WorksComputer Science
Physics-Informed Tactile-based Shape Estimation of Deformable Linear Objects
Abstract
Accurate shape estimation of Deformable Linear Objects (DLOs) is a fundamental challenge in robotic manipulation, often hindered by visual occlusions and the high-dimensional configuration space of flexible materials. While Force/Torque (F/T) and vision-centric methods have been proposed, they frequently struggle with grasp-site ambiguities and environmental interference. In this letter, we investigate the hypothesis that global shape estimation can be achieved through tactile sensing and proprioception, without any visual perception. We contend that dense, high-resolution tactile imprints capture critical boundary conditions and contact gradients that are sufficient to resolve the high-dimensional configuration of the DLO. To prove this, we introduce a Physics-Informed Vision Transformer (ViT) framework that maps tactile images to a 3D vertex chain. By leveraging self-attention mechanisms and a physics-weighted loss function, our model enforces structural priors to ensure that the predicted geometry adheres to the object's physical manifold. Experimental results demonstrate that our tactile-based approach achieves superior reconstruction accuracy across Euclidean and Chamfer metrics. Furthermore, we demonstrate the efficiency of this modality, achieving model transfer to novel DLO materials and geometries. Our findings suggest that tactile feedback provides a robust and computationally efficient alternative for DLO state estimation in visually degraded or occluded environments.
Introduction
Deformable Linear Objects (DLOs), such as cables, wires and tubes, are ubiquitous across industrial, medical and domestic environments [1]. The ability of robots to manipulate these objects effectively is contingent upon accurate 3D shape estimation, which enables complex tasks including shape control, cable insertion and knot tying [2, 3]. Shape control is particularly critical in specialized applications like automotive cable routing and assembly. These tasks require deforming a DLO into a specific target configuration, a process that relies heavily on a tightly coupled loop of perception and manipulation planning [4].
To achieve this, the shape of a DLO must be perceived with high fidelity to provide the necessary feedback for control algorithms. Currently, the predominant approach to DLO state estimation relies on external vision sensors, typically RGB-D cameras. Early methods utilized point cloud registration paired with physics-based simulations to enforce physical consistency [5], while more recent data-driven architectures leverage deep learning and diffusion models to reconstruct 3D geometries from partial visual observations [6, 7, 8]. However, vision-based perception remains
fundamentally constrained by line-of-sight requirements and visual clarity. In practical tasks, external cameras frequently suffer from severe hand-object occlusions caused by the robot’s own grippers, workspace fixtures, or physical barriers [1]. Furthermore, optical tracking degrades significantly in complex environments with background clutter or dense cable harnesses, where disentangling identical-looking target DLOs poses a persistent computer vision bottleneck [9]. While physics-based models can temporarily bridge momentary visual gaps, they fail under complete, sustained loss of sight or heavy clutter. In contrast, tactile-based shape estimation is indispensable in scenarios where visual feedback is entirely unavailable or rendered unreliable, such as routing cables within an automotive chassis during vehicle manufacturing, maneuvering wire harnesses inside dense electronic enclosures, navigating unlit industrial conduits, or when severe hand-object occlusions completely obstruct the view of contact sites and the curve path.
An alternative to vision is the use of force-based sensing for shape estimation. In this domain, significant theoretical groundwork has been laid by Bretl and McCarthy [10], who explored the relationship between contact forces and the equilibrium configurations of elastic rods. Specifically, they have shown that the configuration space of the wire, i.e., the set of all equilibrium configurations, is a six-dimensional smooth manifold. Based on this, later work has proposed a full framework for real-time shape estimation and control of a wire solely using a Force/Torque (F/T) sensor and without any visual feedback [11]. However, translating these force-based approaches into practical robotic applications may be infeasible. These methods are fundamentally based on analytical models that cannot account for the inherent uncer-
This work was supported by the Israel Science Foundation (grant No. 451/24).
R. Atari, S. Ashur and A. Sintov are with the School of Mechanical Engineering, Tel-Aviv University, Israel. Corresponding Author: [email protected]. tainties in the DLO, such as non-uniform material properties or manufacturing defects. In addition, these models typically assume a perfectly straight shape in the relaxed state and rely on fixed mounting to the robotic arm [12]. This rigid setup prevents the system from performing spontaneous pick-and-manipulate tasks, limiting its utility in the unstructured environments where robotic DLO manipulation is most needed. Furthermore, both F/T-based and vision-centric approaches struggle to account for the specific orientation of the DLO tip within the grasp. While F/T methods rely on point-contact assumptions that lack local geometric resolution, visual systems often suffer from severe occlusions at the gripper interface. Consequently, these methods are frequently inapplicable in practical scenarios where the grasp-site orientation varies significantly across trials, as they cannot perceive the critical boundary conditions at the contact level that dictate the global DLO configuration.
Recent advancements in optical-based tactile sensing, such as GelSight [13], DIGIT [14] and AllSight [15], offer high-resolution contact images by embedding miniature cameras behind compliant elastomers [16]. These sensors have been successfully used with cables for local reactive tasks, such as detecting grasp stability [17], guiding motion primitives for weaving and insertion [18], and performing in-hand DLO following [19]. Some research has even utilized tactile data to reconstruct the grasped portion of DLOs to enable contour-following [20]. However, to the best of the authors’ knowledge, optical-based tactile sensing has not yet been utilized to estimate the full 3D global shape of a DLO beyond the immediate contact region. Building upon prior F/T work, we hypothesize that optical-based tactile sensors can capture the same fundamental F/T information while simultaneously encoding the local grasp direction. In this work, we argue that these combined high-resolution features enable the model to resolve the global state of the DLO, effectively moving beyond local reactive control to full-curve shape estimation.
We propose a novel data-based framework for high-fidelity, vision-free DLO shape estimation. Our primary contributions are as follows. First, we introduce the first framework capable of reconstructing the complete 3D geometry of a DLO using exclusively tactile sensing and proprioception, operating entirely without external visual perception. Second, we introduce a tactile-proprioceptive architecture that leverages the Sparsh tactile foundation model [21] through a multi-scale attentive fusion mechanism. We employ a cross-attention decoder conditioned on proprioceptive queries, integrated with a differentiable B-spline layer for geometric regularization. This approach demonstrates that the latent representations of Sparsh, originally designed for discrete tasks, can effectively generalize to the structured geometric regression of deformable objects. Consequently, the proposed framework achieves end-to-end, real-time DLO shape estimation directly from raw inputs, entirely bypassing the need for explicit calibration. Third, we formulate a physics-informed weighted loss grounded in the Discrete Elastic Rods formulation [22], enforcing structural priors on tangency, length and curvature to ensure predicted shapes adhere to the physical manifold of the object. Fourth, we validate our framework through extensive experimentation, including baseline comparisons, ablations of sensor configurations, and transfer learning. Our findings confirm that the model generalizes robustly to DLOs with distinct mechanical and geometric properties using minimal additional data. Lastly, the trained model and labeled dataset is made available open-source for the benefit of the community1.
While existing DLO manipulation frameworks rely heavily on vision or force tracking, these modalities are vulnerable to hand-object occlusions and grasp ambiguities. Rather than replacing them, this letter investigates whether high-resolution tactile imprints and proprioception can serve as a standalone, vision-free alternative for global shape estimation. We establish a robust baseline for visually degraded environments, ultimately serving as a complementary modality to fuse with vision pipelines for fail-safe estimation under severe occlusions.
Method
Problem Formulation
We consider the problem of a DLO grasped by two parallel grippers: one gripper () remains stationary, serving as a fixed base, while the second () is mounted to a robotic arm to facilitate manipulation (Figure 1). To describe the system’s configuration, we define as the global coordinate frame, which is rigidly fixed to . Each finger of both grippers is equipped with an optical-based tactile sensor that serves as the finger pad. Consequently, let denote the tactile image acquired from sensor of gripper at time . This configuration yields a comprehensive set of four tactile images at each timestep, defined as . The pose of with respect to is denoted by , and can easily be computed through the forward kinematics of the arm.

Figure 1. Overview of the tactile-based DLO shape estimation system. Dual robotic grippers (, ) manipulate a DLO while integrated tactile sensors provide contact images ().
A DLO is grasped between the two finger pads of each gripper. It is important to emphasize that the DLO tips are oriented arbitrarily with respect to the grippers. This implies that the local contact geometry is not pre-aligned to some axis. The DLO is represented as a discrete ordered set of vertices , where is expressed with respect to coordinate frame . We explore the learning of a discrete map
where is some product of the tactile space . Hence, given a tactile measurement and the pose , the map will output a set of points along the DLO.
Data Collection
To train the proposed model , we utilize an experimental setup to collect a diverse dataset. This data is generated by sampling DLO configurations, while simultaneously capturing the corresponding tactile images and gripper poses throughout the manipulation process. A motion capture system tracks reflective markers fixed along the DLO to provide ground-truth data. It should be noted that this system is utilized exclusively for data labeling. During deployment, the model relies solely on tactile sensing and proprioception. Furthermore, while the physical placement of markers provides a relatively coarse resolution of points along the DLO, their individual spatial positions are acquired with high accuracy. Training data is collected in an episodic manner. Prior to the start of each episode, the DLO tips are re-grasped between the grippers in a different configuration, ensuring the DLO is held in various positions and orientations relative to each gripper’s coordinate frame. Data collection is performed by driving the robotic arm through random spatial trajectories, subjecting the DLO to various bending and twisting deformations. This process captures the high-dimensional mapping between the DLO’s shape and the resulting fusion of tactile imprints and robotic proprioception. We assume a quasi-static regime during manipulation, allowing the model to focus on equilibrium states without accounting for complex inertial dynamics. The generated training data consists of samples in the form . A similar test dataset was collected in independent episodes for evaluating trained models.
State Estimation Model
To train , we propose a multi-modal encoder–decoder architecture that fuses tactile image features with robot proprioception to regress the spatial vertex chain . The architecture, illustrated in Figure 2, comprises four stages: (i) a tactile encoder that extracts spatial tokens from the four sensors; (ii) a proprioceptive encoder that maps a pose to a set of marker queries; (iii) a cross-attention decoder that fuses both modalities and predicts per-vertex coordinates; and (iv) a differentiable B-spline layer that enforces geometric smoothness on the output curve.

Figure 2. Architecture of the proposed ViT-Sparsh (ViT-S) -based shape estimation model. The system processes synchronous inputs from two grippers, each providing tactile images and gripper pose data. A ViT-based encoder extracts high-dimensional tactile features which are concatenated with proprioceptive embeddings to form input tokens. These tokens are processed by a cross-attention decoder to capture long-range spatial correlations between the two contact sites. Finally, a differentiable B-spline layer enforces geometric smoothness. The output is a spatial vertex chain representing the DLO configuration.
1) Tactile Encoder: To enable temporal tactile reasoning, we formulate to include temporal tactile images. For each
sensor, the three-channel RGB tactile frame is concatenated with the previous one . Hence, each tactile sensor provides a six-channel input, encoding local temporal deformation cues. A Sparsh-based Vision Transformer (ViT-S) with a patch size serves as the tactile encoder. This encoder is initialized from the Sparsh foundation model [21], which provides a family of general-purpose touch representations. Sparsh was pretrained via DINO self-distillation on a large-scale dataset of over 460,000 tactile images, encompassing a diverse range of vision-based sensors to ensure robust feature extraction. ViT-S remains predominantly frozen to preserve the integrity of the pretrained representations. Task-specific adaptation is facilitated by training only the final two transformer blocks and the concluding LayerNorm at a reduced learning rate, specifically the base rate.
Rather than using only the final-layer output, we extract intermediate representations from a set of the transformer layers . They are then combined by a learned weighted sum (attentive probing)
where represents the layer-normalized output of the -th transformer block. Here, denotes the total number of patch tokens, while denotes the embedding dimension to be optimized. The terms are learnable scalar parameters optimized during the training process to weigh the contributions of each layer. This mechanism enables the model to leverage both low-level texture features from earlier layers and higher-level semantic features from deeper layers, without increasing the token dimensionality. A single encoder is shared across all four sensors. Then, the four latent representations are concatenated to yield a combined tactile feature map . To disambiguate the spatial origin of each token, a learned per-sensor embedding is added to the tokens before fusion with the proprioceptive stream.
Proprioceptive Encoder: The pose is converted into a seven-dimensional vector , which consists of the spatial position and the orientation expressed as a quaternion for gripper . Vector is mapped through a three-layer Multi-Layer Perceptron (MLP), producing an output . This vector is reshaped into control-point query tokens , where is the number of B-spline control points. Fixed sinusoidal positional encodings are added to the queries to encode ordinal positions along the DLO.
Cross-Attention Decoder: A cross-attention decoder integrates tactile and proprioceptive features using a single cross-attention layer followed by a self-attention layer. Let denote the multi-head attention operation, where , , and represent the query, key and value matrices, respectively. During the cross-attention phase, the control-point queries attend to the tactile tokens as follows:
A subsequent self-attention layer with the same configuration allows the control-point tokens to exchange information along the DLO chain according to
Each refined token is then independently projected to by a shared MLP, yielding predicted control points .
Differentiable B-Spline Layer: To enforce geometric smoothness on the output curve, the predicted control points in are passed through a differentiable cubic B-spline layer. A uniform knot vector is defined over the parameter domain , from which evaluation points are sampled at equally spaced parameter values . The final vertex positions are obtained by evaluating the cubic B-spline basis functions (of degree 3) at each parameter value as follows:
This yields the predicted vertex set along the DLO. The B-spline layer is parameter-free and fully differentiable, allowing gradients from a loss function to propagate back through the control points. By construction, the B-spline formulation ensures a -continuous curve, effectively suppressing high-frequency oscillations that a discrete curvature regularizer might fail to eliminate. Furthermore, this approach decouples the network’s output dimensionality ( control points) from the final evaluation resolution ( sampled points), allowing for a high-fidelity curve representation without increasing the complexity of the prediction head.
Loss Function
The training objective is designed to ensure that the predicted DLO shapes are not only geometrically accurate but also physically plausible. Rather than relying solely on raw vertex positions, we utilize a physics-informed weighted loss alongside a correspondence-free shape similarity term:
with scalar weights and . Physics-informed weighted loss and Chamfer distance loss are presented next.
Physics-Informed Weighted Loss: To regularize the network’s predictions, we exploit the geometric structure of discrete curves to impose constraints derived from the Discrete Elastic Rods (DER) formulation [22]. This approach is motivated by the physical properties of everyday DLOs (e.g., steel wire, USB cables), which are characterized by inextensibility and resistance to bending. By supervising the derivatives of the curve (i.e., tangents, lengths and curvature), we encourage the model to produce smooth, consistently directed shapes and suppress physically implausible high-frequency oscillations that standard positional losses often fail to eliminate.
Let edge vectors with connect consecutive vertices, from which we derive edge length , tangent unit vector , and the discrete curvature binormal [22]
These geometric quantities serve dual roles: as supervision targets within the physics-informed loss and as differentiable regularizers that enforce inextensibility, local tangent consistency and physically plausible bending. Hence, we compute these values given the predicted and target vertex sets. The weighted loss is then
where , , and are scalar weights of the Mean-Squared Error (MSE) components between the corresponding predicted and ground-truth quantities:
Each component of the weighted loss serves a specific role in ensuring the predicted curve is both geometrically and physically sound. The position term provides foundational geometric supervision, while the tangent term penalizes deviations in local orientation to encourage smooth, consistently directed curves, even in cases where absolute positional error is minimal. The length term enforces inextensibility by ensuring that predicted edge lengths match the ground truth. Finally, the curvature term regularizes bending through the discrete curvature binormal, effectively suppressing physically implausible high-frequency oscillations that might otherwise appear in the predicted curve.
Chamfer Distance: While the ordered terms in rely on a consistent vertex indexing, guaranteed in our pipeline by spline-based resampling, the inclusion of the symmetric Chamfer distance serves as a safeguard against residual resampling ambiguities. By providing a permutation-invariant shape matching signal, it offers a topology-agnostic gradient that focuses on global shape alignment without requiring fixed point-to-point correspondences. This ensures that even if the discrete indexing shifts slightly, the model remains penalized for overall deviations in the DLO’s spatial configuration. The symmetric Chamfer distance is defined as
summing the distances between each point in one set and its nearest neighbor in the other. This complementary loss term enhances the robustness of the training process by providing a signal that is less sensitive to the specific discretization of the DLO.
Experiments
In this section, we evaluate the proposed approach and provide a comprehensive performance analysis across several key metrics. To ensure computational efficiency, all models were trained on an NVIDIA GeForce RTX 5090 GPU with mixed-precision training. Videos of 4 experiments can be seen in the supplementary material.
Hardware Platform
We base our experiments on four DIGIT tactile sensors [14], mounted on the fingers of two Robotiq 2F-85 parallel grippers. One gripper () is fixed to the workspace while the other () is mounted to a Kinova Gen3 7-DOF manipulator. The grippers hold the two opposite ends of a DLO. The system is seen in Figure 1. Each DIGIT sensor provides a RGB tactile image at a frequency of approximately 15 Hz. For data collection and to acquire the ground-truth shape of the DLO, an OptiTrack motion capture system tracks reflective markers affixed along the DLO at roughly equal arc-length intervals. The pose of with respect to is acquired through forward kinematics of the robotic arm. Robot motion and data acquisition are controlled using ROS 2.
Dataset
We begin by collecting data from a primary DLO, a braided steel wire with a diameter of 1.5 mm and a length of 550 mm. The collection was performed over 36 episodes, where in each episode, the wire was grasped in a new pose somewhat covering the orientation range with respect to the gripper axis to ensure grasp diversity. This range represents the physical limit of the setup, as wider angles cause structural collisions or slippage. Then, the robotic arm executed quasi-static gripper motions while deforming the wire under gravity. The unordered tracked markers are sorted via a momentum-based greedy algorithm and resampled to evenly-spaced vertices through B-spline fitting. The value for was empirically optimized through an ablation study, with the estimation error showing minimal variation across different values. The collection yielded a total of labeled training samples. An additional 21,853 samples were collected over four independent episodes and used as the test set.
Model Training
All models are trained using the AdamW optimizer (, ) and a OneCycleLR schedule with a 10% linear warmup. The base learning rate and weight decay are both set to . To facilitate task-specific adaptation while preserving the Sparsh foundation model’s features, the backbone is kept mostly frozen; only the last two transformer blocks are unfrozen and trained at a reduced learning rate of . Training is conducted with a batch size of 32 over 10 epochs. To improve feature extraction, a reference tactile image (captured under no-contact conditions) is subtracted from the observed tactile frames. This subtraction ensures the model remains agnostic to the sensor’s static background and focuses exclusively on the color gradients generated by physical contact. Both the input poses and output marker positions are standardized to zero mean and unit variance to ensure numerical stability and balanced gradient updates. The training objective is the physics-informed loss defined in Eq. (6), with the loss hyperparameters set to , , , , and . These weights were determined empirically during training to properly balance spatial localization accuracy with physical smoothness constraints, ensuring that the predicted 3D vertex chain adheres closely to the DLO’s material continuity without causing optimization instability.
Model Evaluation
Standalone analytical estimators (e.g., DER) are not viable baselines here because our arbitrary grasp orientations leave the required boundary conditions unobservable. Furthermore, prior learning-based DLO trackers rely on base F/T sensing rather than distributed tactile arrays, precluding direct deployment. We therefore validate our proposed DLO shape estimation model against five baseline architectures designed to isolate the contributions of different sensing modalities and architectures. The first, a Pose-Only (MLP) baseline, serves as a lower bound by mapping the 7-D gripper pose directly to the predicted vertex chain, attempting to capture only robot proprioception. To investigate the specific tactile features learned by our model, we introduce a pose and DLO tip direction baseline; this model removes complex tactile features in favor of a geometric cue available at the contact site. For each sensor, the 2D tip direction is extracted with respect to the gripper. The direction is derived via a non-differentiable pipeline of adaptive thresholding, morphological cleanup, and robust line fitting to the contact pixels. The direction, in the form of unit vector, is then fed to an MLP alongside the gripper pose to determine if the model primarily reasons about orientation or more complex contact forces. Next, we compare against a ResNet-18 with Cross-Attention architecture, which utilizes a randomly initialized ResNet-18 to encode tactile images into spatial tokens for fusion with proprioceptive data, testing whether generic visual features are sufficient for accurate tactile state estimation in place of our specialized backbone. Finally, we consider the proposed model without the use of the physics-informed weighted loss as well as without the differentiable B-spline layer to analyze their respective contributions.
The estimation accuracy of the evaluated models is quantified using two primary metrics: per-vertex Root Mean Square Error (RMSE) between predicted shape vertices and ground-truth coordinates, and the Chamfer distance as defined in (13). Table I presents the results along with the number of parameters in each model. As shown, shape estimation based solely on the gripper poses yields poor predictive accuracy,
with higher error margins than the multi-modal variants. This result is consistent with foundational DLO theory, which establishes that a single gripper pose can correspond to a multitude of physically valid configurations [10]. Consequently, these findings confirm that proprioception alone is insufficient for robust shape estimation.
The incorporation of the DLO tip direction at the grasp contacts acts as a beneficial geometric prior, leading to a marginal reduction in estimation error. However, the significant performance gap relative to models utilizing full tactile images indicates that dense, learned tactile features capture critical information beyond simple contact orientation. These features likely encompass deformation gradients, contact geometry and force distributions, which provide the necessary depth to significantly enhance the overall estimation accuracy. These data-driven advantages are further bolstered by our training objective. The experimental results obtained without the physics-informed weighted loss underscore its critical importance, as the absence of these constraints leads to a notable degradation in reconstruction fidelity. This suggests that the term effectively bridges the gap between raw data-driven estimation and the underlying physical manifold of the DLO, ensuring that the model adheres to essential geometric and structural priors during inference. Similarly, removing the B-spline layer eliminates the inherent spatial coupling between neighboring nodes, resulting in high-frequency, physically implausible oscillations and decreased accuracy. Consequently, our proposed ViT-based model achieved superior performance, yielding the lowest estimation error across both Euclidean and Chamfer metrics. This success also suggests that the attention mechanisms in the ViT architecture are particularly well-suited for capturing long-range spatial correlations between local tactile contact gradients and the global DLO configuration. By leveraging these high-dimensional features, the model effectively resolves the inherent ambiguities of proprioception-only approaches, providing a robust and high-fidelity reconstruction of the entire vertex chain. fig:3a shows a shape estimation instance of the steel DLO.

Figure 5. Transfer estimation errors for (a) a USB cable of 4.5 mm diameter and length 550 mm and (b) a steel cable of 2 mm diameter and 750 mm length, with regard to the number of new training samples used to fine-tune the model. Dotted lines show training of the models from scratch with the new data.
To evaluate reliability under sensor degradation, we added zero-mean Gaussian white noise to all tactile images in the test set, parameterized by an intensity noise factor , where 0 and 1 values correspond to original and fully corrupted images, respectively. As shown in Figure 6, both RMSE and Chamfer distance errors degrade moderately across the noise spectrum. These results demonstrate the framework’s robustness, maintaining bounded estimation stability even under severe image corruption.

Figure 6. Chamfer distance and RMSE for shape estimation with induced noise to the tactile images.
To assess the computational efficiency and suitability for real-time robotic control, Table I also reports the mean inference time for each architecture over 1,000 trials. We note that the tip direction computation was not optimized and was utilized solely for accuracy estimation. Consequently, its computational time is not reported, as it does not reflect the potential real-time performance of a production-ready implementation. The results indicate that our proposed ViT-based model exhibits the highest inference time, which is attributed to its larger number of trainable parameters compared to the baseline architectures. Nevertheless, the model achieves a mean inference rate of 16 Hz (61.5 ms latency), matching or exceeding state-of-the-art DLO state estimation pipelines (10-15 Hz) [6], [23]. Crucially, this execution speed easily surpasses typical closed-loop DLO control frequencies (5-15 Hz) [19], ensuring real-time feedback aligned with both sensor throughput and control demands.
| Model | Num. trainable param. () | Num. total param. () | RMSE (mm) | Chamfer dis. (mm) | Inference time (ms) | Freq. (Hz) |
| Pose-Only (MLP) | 3.04 | 3.04 | 55.00 22.00 | 44.00 19.00 | 0.9 | 1.11 |
| Pose + Direction | 2.70 | 2.70 | 50.63 24.25 | 40.70 17.06 | - | - |
| ResNet-18 + Cross-Attn | 3.49 | 3.49 | 36.00 20.00 | 30.00 16.00 | 17.6 | 56.82 |
| Propose model w/o B-spline layer | 5.33 | 23.74 | 35.01 16.62 | 30.06 13.64 | 61.5 | 16.26 |
| Proposed model w/o | 6.38 | 24.80 | 37.00 20.90 | 30.82 16.42 | 61.5 | 16.26 |
| Proposed model | 6.38 | 24.80 | 30.52 15.22 | 27.29 12.30 | 61.5 | 16.26 |
Table I. Comparison of shape estimation performance
Data Requirement Analysis
We subsequently analyze the data requirements for training the proposed ViT-based architecture. To quantify how the size of training data influences performance, we evaluate the model’s accuracy across a varying number of collection episodes. This evaluation is conducted via a cross-validation approach: for each specified episode count, the model is trained 10 independent times using randomly sampled episodes from the full dataset. Each episode contains approximately 5,460 training samples, allowing us to characterize the learning curve and identify the point of diminishing returns for data collection.
Figure 4 illustrates the evolution of the RMSE and Chamfer distance as a function of the number of training episodes. Both metrics exhibit a sharp initial improvement after the first five episodes, with the RMSE decreasing by 35% and the Chamfer distance by 28%. This steep trajectory suggests that the model rapidly internalizes the behavior of DLO deformation, while the subsequent leveling of the curves indicates the onset of diminishing returns as the dataset grows beyond a critical threshold. Performance continues to improve gradually as more data is introduced, though at a significantly reduced rate compared to the initial phase. Notably, the inter-trial variance increases with more than 25 episodes, suggesting that in higher data regimes, the specific composition of episodes becomes more influential. This phenomenon is likely due to the inherent variation in DLO configurations across different recording episodes, where certain rare geometries may disproportionately affect model convergence. Overall, these results indicate that reliable state estimation is achievable with as few as 15 episodes (~42% of the full dataset). This has significant practical implications, as it demonstrates that data collection effort can be substantially reduced in new deployment scenarios without compromising the system’s operational integrity. Hence, we next investigate the ability to transfer the model to unseen DLOs.

Figure 4. Shape estimation accuracy with respect to the number of samples in the training data.
Transfer Learning
To evaluate for cross-DLO transfer, we investigate the effort required to fine-tune the model, previously trained on the primary DLO (braided steel wire of 1.5 mm diameter and 550 mm length), to estimate shapes of new DLO types. We consider two new cables: a USB cable of diameter 4.5 mm and length 550 mm, and a braided steel wire of diameter 2 mm and length 750 mm. The latter DLO has the same material and structural properties as the primary DLO, but with slightly larger diameter and length. For each of the two cables, 50,000 new data samples were collected along with a test set of 14,000 samples. Then, the model trained for the primary DLO was fine-tuned with a lower learning rate by an order of magnitude compared to the original training.
Figures 5a and 5b illustrate the scaling of RMSE and Chamfer errors relative to the size of the fine-tuning dataset, contrasted against models trained from scratch on the same novel data. In the zero-shot regime, the model exhibits significantly diminished performance, confirming that a network trained on a single DLO cannot inherently generalize to the distinct mechanical variations of novel DLOs without adaptation. However, the subsequent inclusion of limited data samples yields a decline in reconstruction error, demonstrating the sample efficiency of the fine-tuning process. Notably, models trained from scratch consistently yield inferior results compared to those initialized with the pre-trained weights. This performance gap suggests that the underlying physical representations learned from the primary DLO provide a robust structural prior that is highly transferable to DLOs with different mechanical properties. Shape estimation examples of the two DLOs can be seen in Figure 3b-Figure 3c.

Figure 3. Visual examples of ground truth and predicted shapes for the three DLOs. (a) 1.5 mm diameter steel cable with estimation error of 34.1 mm, (b) 4.5 mm diameter USB cable with estimation error of 17.7 mm, and (c) 2.0 mm diameter steel cable with estimation error of 41.5 mm. Each row displays the experimental configuration, the corresponding four-channel raw tactile input, and the resulting 3D shape estimation (cyan: ground truth, magenta: prediction).
| Num. of Sensors removed | RMSE (mm) | Chamfer dis. (mm) | |
| New trained model | 1 | 34.2 ± 18.0 | 30.1 ± 14.2 |
| New trained model | 2 | 37.8 ± 21.2 | 34.0 ± 16.9 |
| New trained model | 3 | 46.3 ± 23.3 | 39.6 ± 17.6 |
| Same model w/ fault sensors | 1 | 45.3 ± 23.3 | 36.4 ± 16.7 |
| Same model w/ fault sensors | 2 | 70.9 ± 30.8 | 53.4 ± 20.2 |
| Same model w/ fault sensors | 3 | 106.2 ± 34.8 | 76.9 ± 21.5 |
Table 2. Ablation study on the number of removed sensors.
To evaluate scale sensitivity, we also tested the baseline model without retraining by shifting grasp positions to alter the effective length to 500 mm and 580 mm. The mean reconstruction error increased to 36 mm and 40 mm, respectively. This performance drop demonstrates that the model slightly struggles to generalize zero-shot to out-of-distribution scales, directly motivating future work on a length-aware architecture that ingests segment length as a dynamic input.
Robustness to Partial Loss of Tactile Information
To evaluate the framework’s resilience to partial data loss or hardware failures, we benchmarked performance under two paradigms where sensors are randomly removed: retraining the model on a reduced number of sensor, and 2) evaluating the accuracy of the model, without retraining, while masking faulty channels with blank inputs to simulate sudden runtime failure. As summarized in Table II, the retrained model exhibits excellent structural adaptation, maintaining a low RMSE even when relying on just half of the original tactile array. When subjected to sensor faults without retraining, performance degrades more rapidly due to the stark distribution shift at the input layer. This performance gap suggests a valuable future direction: incorporating stochastic sensor dropout during training to explicitly condition the network against runtime sensor faults.
Conclusions
This paper demonstrates that high-fidelity global shape estimation of DLOs can be achieved using only tactile sensing and proprioception, bypassing visual limitations. By integrating high-resolution tactile contact gradients with global proprioceptive data through a physics-informed ViT framework, our approach accurately resolves high-dimensional DLO configurations. The framework effectively determines DLO-gripper orientation, maintains high performance in data-sparse regimes, and generalizes well to novel DLOs via sample-efficient transfer learning.
The performance degradation observed during sudden sensor failure and out-of-distribution states, such as near-buckling or rapid object slippage, motivates several directions for future work. To address these limitations, we intend to utilize high-fidelity simulation environments to generate diverse datasets across a wide range of mechanical properties and extreme deformation regimes, enabling a more generalized model that can handle sensor faults via dropout, as well as varying stiffness and friction coefficients. Additionally, we will explore incorporating the total wire length, measured directly by the robotic system, as a dynamic network input to enhance the model’s adaptability to different DLO geometries without sacrificing tracking accuracy.
References
- [1]J. Zhu, A. Cherubini, and et al., “Challenges and outlook in robotic manipulation of deformable objects,” IEEE Rob. & Aut. Mag., vol. 29, pp. 67–77, 2022.arxiv.org/abs/2105.01767
- [2]J. Sanchez, J.-A. Corrales, B.-C. Bouzgarrou, and Y. Mezouar, “Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey,” The International Journal of Robotics Research, vol. 37, no. 7, pp. 688–716, 2018.DOI
- [3]A. Caporali, I. Cuiral-Zueco, G. López-Nicolás, and G. Palli, “Robotic perception and manipulation of deformable linear objects: A survey,” The International Journal of Robotics Research, pp. 1–44, 2026.
- [4]H. Yin, A. Varava, and D. Kragic, “Modeling, learning, perception, and control methods for deformable object manipulation,” Science Robotics, vol. 6, no. 54, p. eabdh803, 2021.DOI
- [5]T. Tang, Y. Fan, H.-C. Lin, and M. Tomizuka, “State estimation for deformable objects by point registration and dynamic simulation,” in IEEE/RSJ Int. Conf. on Intel. Rob. and Sys., 2017, pp. 2427–2433.DOI
- [6]K. Lv, M. Yu, and et al., “Learning to estimate 3-D states of deformable linear objects from single-frame occluded point clouds,” in IEEE Int. Conf. on Robotics and Automation, 2023, pp. 7119–7125.arxiv.org/abs/2210.01433
- [7]S. Zhaole, H. Zhou, L. Nanbo, L. Chen, J. Zhu, and R. B. Fisher, “A Robust Deformable Linear Object Perception Pipeline in 3D: From Segmentation to Reconstruction,” IEEE Robotics and Automation Letters, vol. 9, no. 1, pp. 843–850, Jan. 2024.
- [8]Y. Tang, T. Yang, J. Huang, X. Chu, and K. W. S. Au, “Generative 3D State Estimation for DLOs From Partial Observations,” IEEE Robotics and Automation Letters, vol. 11, no. 1, pp. 434–441, Jan. 2026.DOI
- [9]J. Xiang, H. Dinkel, H. Zhao, N. Gao, B. Coltin, T. Smith, and T. Bretl, “TrackDLO: Tracking Deformable Linear Objects Under Occlusion With Motion Coherence,” IEEE RA-L, vol. 8, pp. 6179–6186, 2023.DOI
- [10]T. Bretl and Z. Mccarthy, “Quasi-static manipulation of a kirchhoff elastic rod based on a geometric analysis of equilibrium configurations,” Int. Jou. of Robotics Research, vol. 33, no. 1, pp. 48–68, 2014.DOI
- [11]I. Mishani and A. Sintov, “Real-Time Non-Visual Shape Estimation and Robotic Dual-Arm Manipulation Control of an Elastic Wire,” IEEE Rob. and Auto. Letters, vol. 7, no. 1, pp. 422–429, 2022.DOI
- [12]——, “Learning configurations of wires for real-time shape estimation and manipulation planning,” Engineering Applications of Artificial Intelligence, vol. 121, p. 105967, 2023.DOI
- [13]W. Yuan, S. Dong, and E. H. Adelson, “GelSight: High-resolution robot tactile sensors for estimating geometry and force,” Sensors, vol. 17, no. 12, 2017.DOI
- [14]M. Lambeta, P.-W. Chou, S. Tian, and et al., “DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor With Application to In-Hand Manipulation,” IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 3838–3845, 2020.DOI
- [15]O. Azulay, N. Curtis, R. Sokolovsky, G. Levitski, D. Slovomik, G. Lilling, and A. Sintov, “AllSight: A low-cost and high-resolution round tactile sensor with zero-shot learning capability,” IEEE Robotics and Automation Letters, vol. 9, no. 1, pp. 483–490, 2024.DOI
- [16]S. Zhang, Z. Chen, Y. Gao, W. Wan, J. Shan, H. Xue, F. Sun, Y. Yang, and B. Fang, “Hardware Technology of Vision-Based Tactile Sensor: A Review,” IEEE Sensors J., vol. 22, no. 22, pp. 21410–21427, 2022.
- [17]Y. She, W. Sang, S. Dong, N. Sun, A. Rodriquez, and E. H. Adelson, “Cable manipulation with a tactile-reactive gripper,” The Int. Journal of Robotics Research, vol. 40, no. 12-14, pp. 1385–1401, 2021.
- [18]A. Wilson, H. Jiang, W. Lian, and W. Yuan, “Cable Routing and Assembly using Tactile-driven Motion Primitives,” in IEEE Int. Conf. on Robotics and Automation, 2023, pp. 10408–10414.arxiv.org/abs/2303.11765
- [19]M. Yu, B. Liang, X. Zhang, X. Zhu, L. Sun, C. Wang, S. Song, X. Li, and M. Tomizuka, “In-Hand Following of Deformable Linear Objects Using Dexterous Fingers with Tactile Sensing,” in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2024, pp. 13518–13524.arxiv.org/abs/2403.12676
- [20]A. Monguzzi, M. Pelosi, A. M. Zanchettin, and P. Rocco, “Tactile based robotic skills for cable routing operations,” in IEEE Int. Conf. on Robotics and Automation, May 2023, pp. 3793–3799.DOI
- [21]C. Higuera, A. Sharma, C. K. Bodduluri, T. Fan, P. Lancaster, M. Kalakrishnan, M. Kaess, B. Boots, M. Lambeta, T. Wu, and M. Mukadam, “Sparsh: Self-supervised touch representations for vision-based tactile sensing,” in Conf. on Robot Learning, 2024.arxiv.org/abs/2410.24090
- [22]M. Bergou, M. Wardetzky, S. Robinson, B. Audoly, and E. Grinspun, “Discrete elastic rods,” ACM Tr. Graph., vol. 27, no. 3, pp. 1–12, 2008.
- [23]F. Gu, H. Sang, and et al., “Learning Graph Dynamics With Interaction Effects Propagation for Deformable Linear Objects Shape Control,” IEEE Trans. on Aut. Sci. and Eng., vol. 22, pp. 10 881–10 892, 2025.DOI