WorksComputer Science
BAT-WAM: Bridging Action and Touch in World-Action Models for Dexterous Hands
Abstract
Dexterous manipulation with world action models (WAMs) relies on hand configuration and contact information, yet joint-state and whole-hand tactile observations may be partially or entirely missing. RGB images provide scene context and hand-object geometry but are insufficient to reliably recover these missing modalities. We further observe a correlation between hand state and tactile information. Motivated by this observation, we introduce BAT-WAM, a framework that bridges visual, joint-state, and tactile representations for WAMs. BAT-WAM learns separate codebooks and decoders for the tactile and action streams. An RGB-conditioned continuous latent bridge connects the two streams for bidirectional learning, while visual information serves as continuous context. Under the HTBench evaluation protocol, BAT-WAM reduces the root mean squared error (RMSE) of tactile reconstruction by 34.73% under fully masked conditions and 38.84% under partially masked conditions, relative to an image-conditioned HandTouch checkpoint. For future tactile prediction conditioned on target-time RGB and historical actions, it reduces mean RMSE by 85.56% relative to OpenWAM with HandTouch. Real-robot experiments further show that BAT-WAM consistently outperforms related WAM baselines in dexterous manipulation. Generalization experiments on unseen objects and scenes also demonstrate substantial improvements over baseline models. These findings support joint modeling of visual context, hand state, and touch as a representation approach for multimodal WAMs.
This work was conducted during an internship at Rimbot.
Introduction
Contact-rich dexterous manipulation depends on the interplay between scene geometry, hand configuration, and physical contact. RGB images provide object appearance and visible hand–object geometry; tactile observations (Touch) capture contact locations and pressure distributions; and joint states (States) describe hand configuration and kinematic coordination. These modalities provide complementary information, but similar visual observations can conceal different interaction states. For example, two images of a hand grasping a cup may appear nearly identical, even when one grasp maintains stable contact and the other is beginning to slip. Such ambiguity motivates representations that capture fine differences in hand configuration and contact, consistent with the importance of tactile feedback in reactive manipulation [11]. Recent work has developed several interfaces for incorporating proprioception and dexterous hand states into world action model (WAM) systems. For example, OpenWAM-α [16] linearly projects the robot’s 80-dimensional proprioceptive state into a single context token. This token is appended to the language context and conditions both the video and action streams through cross-attention. For dexterous hands, UniDexTok learns discrete state representations through a standardized hand interface, while UniHМ couples cross-embodiment hand-motion tokenization with language-guided manipulation [3, 21]. These approaches address the challenge of making structured, high-dimensional hand information accessible to token-based models.
In parallel, tactile VLA and world action model (WAM) research has expanded the role of touch in policy learning. Tactile-VLA and VLA-Touch incorporate tactile information into action generation, feedback, and control [1, 8]. VT-WAM, Tactile-WAM, and DreamTacVLA model future tactile observations alongside visual or action information [13, 17, 19], while N-VTLA uses predictive latent tactile tokens to condition action generation [10]. Building on these interfaces, we investigate a specific representation problem: how can Touch and Joint supervise one another within an RGB-conditioned interface while retaining their distinct physical meanings and data structures?

Figure 1. Overview. BAT-WAM bridges state and contact information with RGB observation.
This question is motivated by the physical coupling between hand configuration and contact. Joint states constrain where fingers can engage an object, while tactile measurements provide evidence about the contacts realized by that configuration. RGB supplies the scene context needed to interpret these relationships. HT-Bench, which pairs egocentric vision with full-hand tactile observations, provides a basis for studying visually conditioned contact representations [9]. We hypothesize that explicitly learning conditional dependencies between Touch and Joint can help resolve ambiguities that remain in visual observations and support inference when part of either modality is unavailable.
We introduce BAT-WAM, a multimodal representation interface designed for use by WAM backbones. BAT-WAM bridges state and contact information, while retaining RGB as the context. A continuous bridging module balances modality-specific token budgets and models interactions among hand configuration, contact state, and scene context. This design preserves distinct quantization and reconstruction pathways for Touch and Joint while allowing their representations to interact. Through conditional reconstruction, each modality can provide information for recovering the other: a modality serves as observed context in one masking configuration and as a supervision target in another, with RGB conditioning the inference.
The interface supports both static masked completion and conditional future tactile prediction. Each task explicitly specifies which observations are visible to the model and which signals are used only as supervision. Static completion examines reconstruction of missing information from the available context, while future prediction examines how observed joint-state history informs subsequent tactile states.
Our contributions are threefold:
We uncover a previously overlooked cross-modal relationship: a strong correlation between dexterous hand states and tactile observations.
Building on this finding, we introduce BAT-WAM, a novel cross-modal representation framework that integrates discrete tactile and state representations with continuous RGB context through a multimodal bridging mechanism.
BAT-WAM achieves state-of-the-art performance on benchmarks for masked tactile completion and future tactile prediction, and outperforms the corresponding baselines in real-world experiments and zero-shot generalization evaluations.
Related Work
Proprioceptive State Representations
Proprioceptive state representation learning focuses on encoding internal robot states, such as joint configurations, and integrating them with visual observations. Existing methods adopt different interfaces for incorporating these states: linearly projects continuous proprioceptive states into the Transformer embedding space [2]; HPT uses embodiment-specific input encoders to map proprioceptive states and visual observations from heterogeneous robots into token sequences processed by a shared backbone [15]; and discretizes proprioceptive states and encodes them as text tokens through the language tokenizer [12]. These approaches explore continuous embeddings, heterogeneous input alignment, and discrete text encoding, providing different representation interfaces through which policy models can access proprioceptive information. Proprioceptive states can also serve as reconstruction targets for representation learning. For high-dimensional dexterous hands, UniDexTok addresses state heterogeneity across hand embodiments by organizing human and robot joint states through a unified 22-DoF semantic interface and learning embodiment-conditioned discrete state representations [3]. These studies provide foundations for state integration, cross-modal reconstruction, and unified state representations across embodiments. Building on this direction, BAT-WAM incorporates standardized Joint states and Touch into a common representation interface while retaining separate codebooks and decoders for the two modalities. Conditioned on continuous RGB context, it learns bidirectional dependencies between hand configuration and contact state through conditional reconstruction.
Full-hand tactile representation
Tactile representation learning includes continuous features, cross-sensor pretraining, and discrete tokenization. Sparsh learned visual-tactile representations with self-supervision, AnyTouch sought a common representation for static and dynamic information across visuotactile sensors, and VITaL supported tactile and non-tactile manipulation policies [4, 5, 6]. VQ-VAE provided a standard route from continuous observations to a finite latent vocabulary [14], but pressure maps must also retain contact regions, spatial topology, and local intensity.
HTBench/HandTouch was the closest full-hand tactile representation setting to ours. It used bimanual pressure maps and egocentric vision to learn discrete tactile representations and reconstruct pressure maps under visual conditioning [9]. BAT-WAM retains this spatial pressure interface but gives Joint state its own codebook and decoder, so configuration information enters the bridge in a reconstructable form.
Tactile-enhanced VLA and world-action models
Tactile information entered policies through continuous fusion, feedback control, prediction branches, and latent tokens. 3D-ViTac, Reactive Diffusion Policy, and T-Rex used vision and touch for fine-grained or reactive manipulation [7, 11, 18], while TouchWorld developed a predictive model for tactile forecasting and reactive manipulation [22]. In VLA and WAM systems, Tactile-VLA and VLA-Touch used visual-language physical knowledge or layered feedback, while VT-WAM, Tactile-WAM, DreamTacVLA, and -VTLA studied visuotactile action prediction, asymmetric attention, tactile world models, and latent tactile tokens [1, 8, 10, 13, 17, 19].
These systems demonstrated the value of touch for contact control, but their inputs typically entered as continuous features, external feedback, prediction branches, or latent tokens. BAT-WAM specifies a narrower interface: Touch and states have separately decodable physical vocabularies, cross-modal relations are learned in a continuous bridge, RGB remains continuous context, and the resulting prefix is used for backbone observation optimization.
Method
BAT-WAM encodes tactile observations and hand joint states under RGB conditioning, producing a set of additional input tokens for the backbone. Figure 2 gives the complete interface.

Figure 2. Pipeline of BAT-WAM. Overview of the BAT-WAM pipeline. A shared bridge Transformer integrates separately tokenized touch and joint states with continuous RGB features. Masked cross-modal reconstruction and temporal prediction train the representation interface, whose outputs, together with language instructions, condition a WAM backbone for action generation.
Modality-Specific Tokenizers
Touch is a dense spatial pressure field, whereas Joint is a low-dimensional vector describing the hand configuration. Given their different data structures, we use a separate encoder , projection layer , codebook , and physical decoder for each modality .
The encoder first extracts features, which the projection layer then maps into the quantization space. Let denote the -th projected feature. Quantization is defined as
where is the -th codebook vector and sg denotes the stop-gradient operation. Touch and Joint use separate codebooks. Their quantized features are decoded into pressure maps and joint states by their respective physical decoders.
For each hand, Touch is encoded as 196 tokens on a regular spatial grid, and Joint is encoded as 8 tokens. RGB follows an independent continuous encoding path: a frozen SigLIP encoder extracts image patch features [20], and learned queries produce context tokens from these features. RGB is neither codebook-quantized nor assigned a reconstruction objective.
Cross-Modal Bridging
The cross-modal bridge integrates Touch, Joint, and RGB information. Touch describes contact locations and pressure distributions, Joint describes the current hand configuration, and RGB provides object and scene context. Each modality is processed through its own encoding path and mapped into a common 256-dimensional feature space, where a Transformer models their relationships.
Tokens receive the corresponding modality and view identity embeddings, allowing the model to distinguish their sources. RGB features remain continuous, and the RGB encoder remains frozen during training.
Taking Touch completion as an example, the model concatenates masked Touch features, visible states features, RGB features, and task-control information along the token dimension:
Here, , , and denote the masked tactile, joint-state, and visual features, respectively. The control information specifies the task and masking configuration. The concatenated sequence is processed by the bridge Transformer:
where specifies attention visibility and prevents access to hidden or unavailable information. The bridge output provides cross-modal features for constructing additional input tokens for the backbone.
The static training objective is
Here, supervises unimodal reconstruction, and supervises cross-modal masked prediction. The latent alignment loss aligns the bridge representation with the encoded features of the complete target. The terms and are the codebook matching and commitment losses, respectively.
Multimodal Temporal Prediction
The static bridge learns the relationship between contact and hand configuration at the same time step. Its temporal extension uses historical observations to predict changes in contact. Let denote the start of the historical window and the target time. The model conditions on historical Touch observations , historical Joint states , and RGB observations that include the target time:
With RGB at the target time available, this task uses historical contact and hand configuration to estimate the target Touch observation. Target-time Touch is used only for supervision. Joint inputs contain only historical observations and exclude the target-time state.

Figure 3. Under HTBench, BAT-WAM is the best on both displayed metrics for the Action+Image condition.
Experiments
Our experiments comprise two main components: offline benchmark evaluation and real-world robot experiments. The offline evaluation addresses three questions:
whether the continuous bridge can use Joint and RGB information when Touch is missing or partially masked;
whether the same interface can predict future Touch from historical observations.
whether BAT-WAM outperform baseline models in real-world robotic experiments and demonstrate stronger generalization ability.
Real-world experiments further demonstrate that BAT-WAM outperforms the baselines OpenWAM- and OpenWAM- + HandTouch.
Experimental Setup
Hardware. We conduct real-world experiments using a 6-DoF AgileX PiPER robotic arm equipped with a Linker Hand L6 dexterous hand. The L6 hand is fitted with a full-hand tactile sensing glove from Juqiao Industrial. An Intel RealSense D435 provides top-view RGB-D observations, while a wrist-mounted Intel RealSense D405 provides wrist-view RGB-D observations. The complete experimental platform is shown in Figure 4.

Figure 4. Real-world experiment environment and representative task examples. The left column shows two views of the shared tabletop setup.
Tasks Description. We design 7 diverse tasks spanning pick&place, stacking, pull&push, and long horizon manipulation to evaluate our WAM models. For each task, we collect 200 demonstrations.
Benchmark. We use HT-Bench as our offline benchmark [9]. HT-Bench is a multitask benchmark for tactile representation learning in robotic dexterous manipulation that combines egocentric vision with full-hand tactile data. Through fine-grained tactile retrieval, masked tactile completion, vision-to-tactile synthesis, and multimodal tactile frame prediction, it evaluates contact structure modeling, visual–tactile alignment, temporal prediction, and generalization to unseen tasks.
Given ground-truth and predicted tactile values , at valid pixels , we compute the evaluation metrics as follows:
F-type metrics are computed over all valid tactile pixels, whereas H-type metrics are computed only over valid pixels hidden by the local mask. The cIoU metric measures the overlap between continuous tactile values, rather than the IoU of thresholded contact labels. For dataset-level evaluation, squared-error sums, pixel counts, and overlap statistics are accumulated across the entire evaluation set before computing the final metrics.
Multimodal Reconstruction and Temporal Prediction
We evaluate BAT-WAM with three offline experiments. Full-mask reconstruction tests whether the modality-specific tokenizers retain touch and states signals. Local-mask reconstruction tests whether the continuous bridge can use Joint and RGB information when Touch is missing or partially masked. Conditional Future Touch prediction tests whether the same interface can predict future Touch from historical observations.
Full-mask reconstruction. Table 1 compares image-conditioned HandTouch and BAT-WAM under the same sample identities, masks, and global-pixel evaluation protocol. BAT-WAM with Action+Image reaches F-RMSE 0.0145 and F-cIoU 0.8933, compared with 0.0221 and 0.8316 for HandTouch. This corresponds to a 34.73% RMSE reduction; the absolute RMSE and cIoU improvements are 0.0077 and 0.0617, respectively. With Action and Image jointly available, BAT-WAM obtains the strongest full-mask Touch reconstruction, verifying that the modality-specific tokenizers retain and transmit touch and state signals through the shared interface.
| Model | Full touch | Image | Action | F-RMSE | F-cIoU |
| HandTouch | – | ✓ | – | 0.0221 | 0.8316 |
| BAT-WAM | – | ✓ | – | 0.0163 | 0.8828 |
| BAT-WAM | – | – | ✓ | 0.0159 | 0.8836 |
| BAT-WAM | – | ✓ | ✓ | 0.0145 | 0.8933 |
Table 1. Full-mask touch completion. Available inputs. ✓ input available; – input unavailable. Bold indicates the best result in each metric.
Local-mask reconstruction. Table 2 reports metrics on masked valid pixels. BAT-WAM with Action+Image reaches H-RMSE 0.0137 and H-cIoU 0.8984, compared with 0.0224 and 0.8211 for HandTouch. Hidden-region RMSE is reduced by 38.87%; the absolute RMSE and cIoU improvements are 0.0087 and 0.0773, respectively. Mean and zero filling perform substantially worse, further showing that BAT-WAM learns structured completion from visible context.
| Model | Partial touch | Image | Action | H-RMSE | H-cIoU |
| mean fill | ✓ | – | – | 0.0955 | 0.3565 |
| zero fill | ✓ | – | – | 0.1221 | 0.0000 |
| HTBench | ✓ | ✓ | – | 0.0224 | 0.8211 |
| BAT-WAM | ✓ | ✓ | – | 0.0146 | 0.8925 |
| BAT-WAM | ✓ | – | ✓ | 0.0143 | 0.8934 |
| BAT-WAM | ✓ | ✓ | ✓ | 0.0137 | 0.8984 |
Table 2. Local-mask touch completion. Available inputs. ✓: input available; –: input unavailable. Bold indicates the best result in each metric.
| Method | Pick & Place | Pick & Place | Pick & Place | Fine Manipulation | Pull & Push | Pull & Push | Long Horizon | Average |
| Method | Pick | Pick | Pick | Stack | Open | Close | Wipe | Average |
| Blue Cube | White Flange | Black Flange | Blue Cube | Drawer | Drawer | Whiteboard | ||
| OpenWAM- | 80.0 | 55.0 | 60.0 | 70.0 | 85.0 | 100.0 | 15.0 | 66.4 |
| OpenWAM- + HandTouch | 75.0 | 65.0 | 65.0 | 75.0 | 85.0 | 95.0 | 25.0 | 69.3 |
| BAT-WAM | 90.0 | 75.0 | 70.0 | 85.0 | 95.0 | 95.0 | 45.0 | 79.3 |
Table 4. Real-world success rates (%) across different tasks.
When Touch is missing or partially masked, the continuous bridge jointly integrates the remaining Joint/Action and RGB context to recover the hidden region. Joint and RGB information therefore complement missing Touch and produce effective hidden-region reconstruction.
Conditional Future Touch prediction. Table 3 compares conditional Future Touch prediction. In conditional future Touch prediction, BAT-WAM uses RGB, historical Touch observations, and historical actions to achieve an RMSE of 0.0157 and a cIoU of 0.8901. Compared with HandTouch, which uses RGB and historical Touch observations, BAT-WAM reduces RMSE by 25.59% and increases cIoU by 4.15 percentage points, demonstrating better future Touch prediction performance under the evaluated input configurations.
| Model | Touch | Image | Action | RMSE ↓ | cIoU ↑ |
| HandTouch | ✓ | ✓ | – | 0.0211 | 0.8486 |
| BAT-WAM | ✓ | ✓ | ✓ | 0.0157 | 0.8901 |
Table 3. Future tactile prediction. Available inputs. ✓: input available; –: input unavailable. Bold indicates the best result in each metric.
Taken together, the full-mask experiment verifies that the modality-specific tokenizers retain Touch and Joint signals for reconstruction; the local-mask experiment verifies that the continuous bridge uses Joint and RGB context to complete missing Touch; and the Future Touch experiment verifies that the same interface predicts future Touch from historical observations. These results establish the effectiveness of the BAT-WAM shared interface for multimodal reconstruction, missing-modality completion, and temporal prediction.
Real-World Experiment
Real World. Figure 4 presents seven real-world tasks spanning four categories—pick-and-place, fine manipulation, pull-and-push, and long-horizon tasks—all conducted in a shared tabletop setup. BAT-WAM achieves the highest mean success rate of 79.3%, outperforming OpenWAM- at 66.4% and OpenWAM- + HandTouch at 69.3%. It achieves the highest success rate on every task except Close Drawer. The largest task-level advantage is observed on Wipe Whiteboard, where BAT-WAM achieves a success rate of 45%, compared with 15% for OpenWAM- and 25% for OpenWAM- + HandTouch. We apply a strict success criterion to this task: an execution is considered successful only when all writing has been completely erased. This criterion helps explain the relatively low success rates.
Adding HandTouch to OpenWAM- increases the mean success rate from 66.4% to 69.3% and improves performance on Pick White Flange, Pick Black Flange, Stack Blue Cube, and Wipe Whiteboard, suggesting that tactile information can benefit model performance. BAT-WAM further improves the mean success rate by 12.9 and 10.0 percentage points over OpenWAM- and OpenWAM- + HandTouch, respectively. These results demonstrate consistent transfer of BAT-WAM’s multimodal interface across manipulation, grasping, wiping, and stacking tasks, yielding the strongest overall real-world performance among the evaluated methods.
Zero-shot generalization. Table 5 evaluates real-world zero-shot generalization across variations in object color, size, shape, scene layout, and object pose. BAT-WAM achieves the highest success rate in all five unseen settings, with an average of 54.5%, compared with 38.5% for OpenWAM- and 42.0% for OpenWAM- + HandTouch, yielding gains of 16.0 and 12.5 percentage points, respectively. The largest advantage occurs under unseen layouts, where BAT-WAM achieves 80.0% success, compared with 57.5% and 60.0% for the two baselines. It also reaches 80.0% under unseen colors, indicating strong performance under the tested color and layout variations. Generalization to unseen sizes and shapes remains more challenging: despite outperforming both baselines, BAT-WAM achieves only 37.5% and 27.5%, respectively, compared with 87.5% and 72.5% in the corresponding seen settings. These results show improved zero-shot performance across the evaluated variations, while identifying changes in object geometry as a remaining limitation.
| Method | Color | Color | Size | Size | Shape | Shape | Layout | Layout | Pose | Pose | Unseen Avg. |
| Method | Seen | Unseen | Seen | Unseen | Seen | Unseen | Seen | Unseen | Seen | Unseen | Unseen Avg. |
| OpenWAM- | 77.5 | 67.5 | 80.0 | 20.0 | 55.0 | 10.0 | 70.0 | 57.5 | 65.0 | 37.5 | 38.5 |
| OpenWAM- + HandTouch | 75.0 | 70.0 | 80.0 | 27.5 | 60.0 | 15.0 | 67.5 | 60.0 | 70.0 | 37.5 | 42.0 |
| BAT-WAM | 87.5 | 80.0 | 87.5 | 37.5 | 72.5 | 27.5 | 85.0 | 80.0 | 77.5 | 47.5 | 54.5 |
Table 5. Zero-shot success rates (%) across different tasks.
Conclusion
We introduced BAT-WAM, a multimodal representation interface that bridges tactile contact and hand configuration for world-action models. Separate codebooks and decoders represent touch and states, while a shared Transformer learns their conditional relationships using continuous RGB context. This design supports static masked completion and conditional future tactile prediction within a common interface. On HT-Bench, BAT-WAM improves full-mask tactile synthesis and local-mask completion under aligned evaluation protocols. It also improves conditional future tactile prediction using historical touch and states together with target-time RGB. Across seven real-world tasks in a shared tabletop setup, BAT-WAM achieves a mean success rate of 79.3%, compared with 66.4% for OpenWAM- and 69.3% for OpenWAM- + HandTouch.
References
- [1]Jianxin Bi, Kevin Yuchen Ma, Ce Hao, Mike Zheng Shou, and Harold Soh. VLA-Touch: Enhancing vision-language-action models with dual-level tactile feedback. arXiv preprint arXiv:2507.17294, 2025. 1, 4arxiv.org/abs/2507.17294
- [2]Kevin Black, Noah Brown, Danny Driess, et al. π₀: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. 3arxiv.org/abs/2410.24164
- [3]Dong Fang, Youjun Wu, Yuanxin Zhong, Rui Zhang, Yunlong Wang, Xiaosong Jia, and Yu-Gang Jiang. UniDexTok: A unified dexterous hand tokenizer from real data. arXiv preprint arXiv:2606.10683, 2026. 1, 3arxiv.org/abs/2606.10683
- [4]Ruoxuan Feng, Jiangyu Hu, Wenke Xia, Tianci Gao, Ao Shen, Yuhao Sun, Bin Fang, and Di Hu. AnyTouch: Learning unified static-dynamic representation across multiple visuo-tactile sensors. arXiv preprint arXiv:2502.12191, 2025. 3arxiv.org/abs/2502.12191
- [5]Abraham George, Selam Gano, Pranav Katragadda, and Amir Barati Farimani. ViTaL pretraining: Visuo-tactile pretraining for tactile and non-tactile manipulation policies. arXiv preprint arXiv:2403.11898, 2024.arxiv.org/abs/2403.11898
- [6]Carolina Higuera, Akash Sharma, Chaithanya Krishna Bodaduluri, Taosha Fan, Patrick Lancaster, Mrinal Kalakrishnan, Michael Kaess, Byron Boots, Mike Lambeta, Tingfan Wu, and Mustafa Mukadam. Sparsh: Self-supervised touch representations for vision-based tactile sensing. arXiv preprint arXiv:2410.24090, 2024. 3arxiv.org/abs/2410.24090
- [7]Binghao Huang, Yixuan Wang, Xinyi Yang, Yiyue Luo, and Yunzhu Li. 3D-ViTac: Learning fine-grained manipulation with visuo-tactile sensing. arXiv preprint arXiv:2410.24091, 2025. 3arxiv.org/abs/2410.24091
- [8]Jialei Huang, Shuo Wang, Fanqi Lin, Yihang Hu, Chuan Wen, and Yang Gao. Tactile-VLA: Unlocking vision-language-action model’s physical knowledge for tactile generalization. arXiv preprint arXiv:2507.09160, 2025. 1, 4arxiv.org/abs/2507.09160
- [9]Yuzhe Huang, Jiaping Wu, Jiaming Jiang, Hezhe Lin, Aikebaier Aierken, Yunlong Wang, Kun Cheng, Ziyuan Jiao, and Yuanxin Zhong. HT-Bench: Benchmarking and learning dexterous full-hand tactile representations with egocentric vision. arXiv preprint arXiv:2606.19161, 2026. 2, 3, 6arxiv.org/abs/2606.19161
- [10]NeoteAI Team and Fudan TEA Team. N₀-VTLA: Scaling vision-tactile-language-action model with latent tactile tokens. arXiv preprint arXiv:2607.23782, 2026. 1, 4arxiv.org/abs/2607.23782
- [11]Dantong Niu, Zhuoyang Liu, Zekai Wang, Boning Shao, Zhao-Heng Yin, Anirudh Pai, Yuvan Sharma, Stefano Saravalle, Ruijie Zheng, Jing Wang, Ryan Punamiya, Mengda Xu, Yuqi Xie, Yunfan Jiang, Letian Fu, Konstantinos Kallidromitis, Matteo Gioia, Junyi Zhang, Jiaxin Ge, Hai-wen Feng, Fabio Galasso, Wei Zhan, David M. Chan, Yutong Bai, Roei Herzig, Jiahui Lei, Li Fei-Fei, Ken Goldberg, Jitendra Malik, Pieter Abbeel, Yuke Zhu, Danfei Xu, Linxi Fan, and Trevor Darrell. T-Rex: Tactile-reactive dexterous manipulation. arXiv preprint arXiv:2606.17055, 2026. 1, 3arxiv.org/abs/2606.17055
- [12]Physical Intelligence, Kevin Black, Noah Brown, et al. π₀.₅: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. 3
- [13]Shuai Tian, Yupeng Zheng, Yuhang Zheng, Songen Gu, Yujie Zang, Yuxing Qin, Weize Li, Haoran Li, Wenchao Ding, and Dongbin Zhao. VT-WAM: Visual-tactile world action model for contact-rich manipulation. arXiv preprint arXiv:2607.02503, 2026. 1, 4arxiv.org/abs/2607.02503
- [14]Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Advances in Neural Information Processing Systems, 2017. 3arxiv.org/abs/1711.00937
- [15]Lirui Wang, Xinlei Chen, Jialiang Zhao, and Kaiming He. Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers. arXiv preprint arXiv:2409.20537, 2024. 3arxiv.org/abs/2409.20537
- [16]Y. Wang, S. Huang, M. Li, et al. OpenWAM: An open, modular exploration towards systematic world-action model pre-training. arXiv preprint arXiv:2609.07398, 2026. 1arxiv.org/abs/2609.07398
- [17]Siyu Wu, Linjing You, Junjie Zhu, Yaozu Liu, Changhao Zhang, Jian Liu, Weiqiang Wang, Qi Li, Jituo Li, and Hengshuang Zhao. Tactile-WAM: Touch-aware world action model with tactile asymmetric attention. arXiv preprint arXiv:2606.26663, 2026. 1, 4arxiv.org/abs/2606.26663
- [18]Han Xue, Jieji Ren, Wendi Chen, Gu Zhang, Yuan Fang, Guowng Guo, Huazhe Xu, and Cewu Lu. Reactive diffusion policy: Slow-fast visual-tactile policy learning for contact-rich manipulation. arXiv preprint arXiv:2503.02881, 2025. 3arxiv.org/abs/2503.02881
- [19]Guo Ye, Zexi Zhang, Xu Zhao, Shang Wu, Haoran Lu, Shihan Lu, and Han Liu. Learning to feel the future: DreamTacVLA for contact-rich manipulation. arXiv preprint arXiv:2512.23864, 2025. 1, 4arxiv.org/abs/2512.23864
- [20]Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 4arxiv.org/abs/2303.15343
- [21]Zhenhao Zhang, Jiaxin Liu, Ye Shi, and Jingya Wang. UniHM: Unified dexterous hand manipulation with vision language model. In International Conference on Learning Representations, 2026. 1
- [22]Jianyi Zhou, Feiyang Hong, Yunhao Li, Yicheng Zhao, Yongjue Cen, Zirui Liu, Jiakang Huang, Zirui Chen, Ruiyang Zhang, Weizhuo Zhu, Xuhua Song, and Shuo Yang. Touch-World: A predictive and reactive tactile foundation model for dexterous manipulation. arXiv preprint arXiv:2607.07287, 2026. 4arxiv.org/abs/2607.07287