WorksComputer Science
ACiT: Proprioceptive-Prior Action Chunking with Inverted Transformers
Abstract
Visuomotor policies condition action prediction on visual and proprioceptive observations. While historical proprioceptive sequences carry rich dynamic priors, current methods treat them as equal to high-dimensional vision, causing the proprioceptive signal to be overwhelmed. To identify what is lost in equal-conditioning fusion, we analyze proprioceptive data and find that strong inter-joint coupling is the key action-determining structure. This finding aligns with the neural sensorimotor loop, in which proprioception first forms a body-centered prior and vision is later calibrated via parietal remapping. Inspired by this, we propose ACiT, Proprioceptive-Prior Action Chunking with Inverted Transformers, which reinforces historical joint-coupling as the primary condition with vision as a later calibration. This design is instantiated through three modules: Inverted State Encoding builds a proprioceptive prior via per-joint tokens and inter-joint self-attention. State-Visual Grounding lets joint tokens query visual evidence, making vision goal- and calibration-only. Inverted Action Chunking decodes per-joint trajectories independently. ACiT improves success rates on most LIBERO, RoboTwin 2.0, and real-world tasks, with the largest gains on those that require dynamics and coordination. The same conditioning design transferred to the VLA backbone pi_0-ACiT yields significant gains with better execution efficiency.
Introduction
Visuomotor policies predict actions from multi-view observations and proprioception. Early work conditioned on a single current state, where ACT maps the current joint state and images to an action chunk [41]. Diffusion Policy conditions on a short observation history instead [8]. A single frame proved insufficient to resolve action ambiguity, while history carries dynamics and hidden-state information, so proprioceptive history was explicitly incorporated. Recent models such as A2A and Chronos explicitly model proprioceptive history [12, 42]. Another line discretizes historical states or actions into tokens to better integrate with LLM backbones, at the cost of losing continuous coupling structure [14, 22, 28]. Reconstruction-only tokenization can also erode control-relevant information [16]. These works validate the value of using history from both continuous and discrete directions, and state history is becoming a consensus for action prediction.
These advances have not converged at the conditioning interface. First, mainstream methods map historical proprioceptive states as an independent modality, either discretely or continuously, and place them alongside a pooled global visual vector as conditions for action generation. This naive fusion lets high-dimensional, low-SNR visual features entangle and dominate the conditioning, leading to degraded robustness or spurious shortcuts [15, 38]. Second, we analyze the state history and the action itself: the effective dimensionality of the action distribution is far below the number of joints, and joints are strongly coupled, as shown in Figure 1(a). However, existing methods model temporal dependencies and pack all joints into one vector, so the inter-joint coupling, a structure critical to action, can only be implicit inside the token.
Figure 1. Motivation. (a) Embodiment: a dual-arm robot has 14 joints, each with its own state history, while the action space has a much lower effective dimensionality (PR 1.44–4.42) and strong inter-joint coupling, the state therefore should not be compressed into one shared vector. (b) Biological inspiration: proprioception forms a body-centered prior in S1, vision reaches PPC via V1 only to remap and calibrate the goal, and M1 issues the command. The two modalities are not symmetric, and their order is fixed. (c) Conditioning interface: ACT concatenates a single-frame state with vision into one condition shared by all joints; ACiT keeps the per-joint prior as the primary condition, retrieves visual evidence per joint, and decodes one trajectory per joint.
Therefore, how to encode historical state information, how to fuse modalities, and how to feed the result into action decoding is an open problem. Inspired by neuroscience, the human can chain from peripheral proprioception to body-centered action generation as is follows (Figure 1(b)): muscle and joint receptors continuously encode joint angle, velocity, and tension and project through large-diameter myelinated afferents [29, 36]. The somatosensory cortex and posterior parietal cortex aggregate proprioceptive history into a body-state representation, forming a body-centered prior [1, 2]. The posterior parietal cortex then remaps visual goals from retinal coordinates into body-centered coordinates, with proprioceptive context modulating this remapping [2, 10]. Finally, the parietal-motor pathway sends the calibrated body-centered representation to action generation [23, 37]. The key computational asymmetry is that proprioception provides a continuous, low-latency body-state stream, whereas visual goals are remapped and integrated into the body-centered representation afterward.
We map this sensorimotor loop to Bayesian cue integration: the fast, continuous proprioceptive stream forms a prior over the body state, visual goals provide goal evidence (a likelihood-like term), and calibrating the prior yields a posterior-like body-centered representation for action generation. Based on this, we propose ACiT on top of ACT, the smallest architecture that still carries the full interface (multi-view images, proprioception, and a chunked action decoder), so that the encoder and the head stay fixed while only the conditioning changes. It has three modules. First, Inverted State Encoding compresses each joint’s entire history into a token and directly models inter-joint coupling with self-attention. Second, State-Visual Grounding treats vision as a per-joint goal-calibration signal and does not let it directly enter the encoder. Third, Inverted Action Chunking treats each joint as a token on the action side, and each joint independently generates its complete trajectory chunk. The decoder does not need to translate between “visual coordinates” and “joint coordinates”. The historical proprioceptive sequence and action generation share a more consistent feature space. To separate the interface from the backbone, we port the same conditioning to a pretrained VLA and obtain -ACiT, which keeps ’s architecture and changes only how the state enters it. Experiments on LIBERO, RoboTwin 2.0, and real-world manipulation tasks show that ACiT and -ACiT improve success on most tasks, with improved robustness under visual perturbations, especially on tasks that depend on dynamics and joint coordination.
Our contributions are as follows:
We formulate the conditioning interface as Bayesian sensorimotor cue integration, with history as the prior and vision as the calibration evidence, and realize it as an explicit asymmetric fusion.
We propose ACiT, which uses three inverted Transformer modules to explicitly build and preserve the inter-joint coupling dynamics in the state history instead of flattening them.
We validate ACiT on LIBERO, RoboTwin 2.0, and real-world tasks, where it improves the average success rate by 5.7 to 10.8 points over the strongest baseline. -ACiT improves the average success rate by 26.9 points at 20.2% lower inference latency than .
Related Work
History in the conditioning interface. Early visuomotor policies condition on a single frame: ACT maps the current joint state and images to an action chunk [41], and Diffusion Policy instead uses a short observation window [8]. Later work makes the state history explicit in the conditioning input [12, 42]. A parallel line discretizes states or actions so that autoregressive backbones can consume them [14, 21, 22, 28], at the cost of losing continuous structure, and reconstruction-only tokenization can erode control-relevant information [16]. Under both interfaces proprioception enters as one flat vector or a one-dimensional token sequence, so the coupling among joints stays implicit.
Vision and proprioception under fusion. Pooling both modalities into one condition leaves nothing to stop the high-dimensional visual stream from dominating and introducing spurious shortcuts [15, 38]. One line responds by letting the robot state steer vision, through robot-centric pooling [43] or active visual attention [39], yet the asymmetry is applied to perception rather than to the conditioning itself. Large VLA backbones inherit the same symmetric interface, concatenating one proprioceptive token with visual and language tokens [4, 5, 25].
Structure as an inductive bias. Topology-aware models inject the kinematic graph as a structural bias for cross-embodiment transfer [24, 27]. ACiT shares that motivation but acts on the conditioning interface: it carries history per joint, keeps the resulting prior primary, and lets each joint retrieve its own visual calibration in a single generation pipeline. Appendix § 10 expands the comparison.
Method
Preliminaries
Let the robot have joints, with state and action at time . An action-chunking policy predicts
and executes its first few steps. The state history is , where is the history length (number of past frames). denotes the visual token matrix, whose rows are the visual tokens.
Most chunk-based policies share one skeleton [8, 41]: a Transformer encoder [34] maps the observations to a condition, and a decoder emits the chunk, . What differs across methods is how the proprioceptive history is represented inside that condition, which is the axis ACiT changes. The encoder and the decoder are those of the host policy (Sec. D.1). The equal-conditioning baseline we depart from places the current state and vision side by side as conditions:
Bayesian Sensorimotor Formulation. Let be the latent body-centered state and the task goal (vision, or vision and language in a VLA). The ideal visuomotor computation factorizes into two stages,
Equation (3) states that visual evidence calibrates a proprioceptive prior into a posterior body-state estimate, and Equation (4) marginalizes over this posterior to produce the action distribution. This formulation mirrors Bayesian cue integration in sensorimotor control [13, 33].
ACiT Instantiation. Rather than modeling explicit probability distributions, ACiT uses deterministic representations to instantiate this computation:
where is a deterministic representation of the posterior-like body state in Eq. (3), and the action head replaces the integration in Eq. (4). The function is factorized into per-joint history encoding with a body-centered prior (§ 3.2) followed by state-visual grounding (§ 3.3). The action head consumes in § 3.4. This is a computational analogy, not a literal probabilistic model. is distinct from the CVAE latent used for training.
Inverted State Encoding
For each joint , ACiT applies a depthwise 1D convolution along time with kernel size , followed by a shared temporal projection:
where is the kernel size, is the -th frame in the history window, is the joint-specific convolution weight at offset , and zero padding is applied at both ends. We use by default, giving weights . The convolved features are stacked into .
A shared linear layer and a joint-identity embedding then project the temporal feature to a -dimensional token:
where is shared across joints and is a learnable joint-identity embedding. Stacking all joint tokens gives . The depthwise convolution captures local temporal patterns such as velocity and acceleration without mixing joints. The history is in effect a multivariate time series with one channel per joint, and the coupling we care about is a dependency between channels rather than between time steps. Treating each channel as a token and letting attention operate across channels is the inverted formulation that has been shown to capture variate-wise structure in forecasting [19]. Here the variates are joints, and their dependency is the coupling that determines the action.
Proprioceptive prior. The joint tokens then attend to each other, so that the coupling structure is represented explicitly rather than hidden inside a flattened vector:
where are the queries, keys and values, is the key dimension, and is a deterministic representation of the prior in Eq. (3), i.e., the body-centered prior from which the action is eventually generated.
State-Visual Grounding
Given the prior , vision now enters only as evidence. Each prior token serves as a query to the visual tokens:
Figure 2. ACiT architecture, organized by its three modules. (a) Inverted State Encoding: per-joint depthwise 1D convolutions turn each joint’s history into one token, and self-attention among the tokens gives the proprioceptive prior . (b) State-Visual Grounding: the joint tokens query the visual tokens of the multi-view observations and produce grounded tokens . (c) Inverted Action Chunking: is the memory of a transformer decoder whose learnable joint queries output one -step trajectory per joint.
Equation (11) is the concrete instantiation of the mapping in Eq. (5), where is the grounded condition and a deterministic representation of the posterior-like body state in Eq. (3). All blocks use pre-LN residual connections followed by a feed-forward network. Visual evidence is thus retrieved into a body-centered state rather than fused symmetrically with proprioception. This is consistent with visual targets being remapped into body-centered reference frames [2, 10] and with vision modulating, rather than re-placing, the body schema [1].
Inverted Action Chunking
The action head is interchangeable. With the ACT transformer decoder, learned joint queries attend to the memory . As in the self- and cross-attention blocks above, each head computes
where , , and is a learned token embedding of the CVAE latent defined below. The CVAE latent is used only here. State-visual grounding is independent of . With a conditional 1D U-Net, the denoiser is conditioned on and and produces the same per-joint output layout. In both cases, a shared linear projection maps every joint output to a full trajectory:
and concatenation gives . The action is thus modeled as joint trajectories rather than time-step vectors.
Training Objective
Following ACT [41], we use a CVAE [31] latent , where with , and at inference. The objective is the standard CVAE loss [31]:
where is the expectation over demonstrations, abbreviates , abbreviates , KL denotes the Kullback-Leibler divergence, and sets the weight of this term.
Experiments
Experimental Setup
Benchmarks and Tasks. We evaluate on two simulation benchmarks and a real-world dual-arm platform. RoboTwin 2.0 [7] is a dual-arm manipulation benchmark with 14-DoF joint states and actions and three cameras (a top camera and two wrist cameras). We use six tasks: Lift Pot, Pick Dual Bottles, Handover Block, Move Can Pot, Place Dual Shoes and Shake Bottle, abbreviated as Lift, Pick Dual, Handover, Move Can, Shoes and Shake. Each task has an easy and a randomized hard split, the latter varying object pose, color and background (Appendix § 7.§ 1). LIBERO [18] is a single-arm benchmark with 7-DoF actions, 8-DoF states, and two cameras. We evaluate on Spatial, Object, and LIBERO-10. We exclude Goal, which requires language instructions.
Real-World Platform. We build a dual-arm platform from two reBot Arm B601-DM arms [30]. Each arm has six DoF, a
1.5 kg payload, and mm repeatability. A global camera and two wrist cameras provide observations. We evaluate four tasks: pick cube and stack cube with one arm, and lift pot and handover cube with two arms. Hyperparameters and per-task protocols are in Appendix § 7.§ 2.
Baselines and Comparisons. For small action policies we compare three baselines. ACT [41] uses a CVAE and a transformer decoder on the ACT backbone and conditions on the current frame only. Diffusion Policy [8] and the flow-matching baseline [17] share the same observation encoder and the same 1-D U-Net, and both condition on a proprioceptive history. They differ only in the generative objective (DDPM [11] versus flow matching [17]) and in the sampler. The task-specific policies are trained from scratch with LeRobot [6] in PyTorch [26], on a shared ResNet18 [9] backbone and with AdamW [20]. The VLA keeps its PaliGemma backbone. Simulation runs in MuJoCo [32] through RoboTwin [7].
Main Results on Simulation Benchmarks
Figure 5 compares ACiT with ACT, DP, and FM on LIBERO and RoboTwin 2.0. On LIBERO the gain appears on Spatial, where success depends on precise placement and therefore needs a different visual reference for each joint. On Object the strong baselines are near saturation and ACiT matches them, while on LIBERO-10 the four methods are
Figure 5. Main results on the simulation benchmarks. Left: LIBERO. Right: RoboTwin 2.0. Pale bars denote easy and solid bars denote randomized hard. A number above a pale bar is its easy rate, and a number on a solid bar is its hard rate. ACiT is first in each group.
close and the LIBERO mean is carried by Spatial. On RoboTwin 2.0 the pattern is sharper: ACiT leads on the tasks limited by inter-arm coordination (Pick Dual Bottles, Place Dual Shoes), and its margin over the strongest baseline grows under randomization on the tasks limited by velocity and phase (Shake Bottle, Move Can Pot), where a flat condition cannot express the motion that the scene excites.
What the Attention Does. Figure 4 shows what the two attention mechanisms do. In the top row the joint self-attention is sparse: a few pairs carry most of it, and they are not limited to neighboring joints, including pairs that span the two arms. Joint-level tokens carry this structure, which a flattened state vector cannot. In the bottom row a joint’s query does not spread over the frame but lands on the objects its motion depends on, the bottle a gripper takes in Pick Dual Bottles and, in Place Dual Shoes, both the shoe to lift and the box to place it in. Vision enters as evidence addressed to a joint, not as one summary of the scene.
Figure 4. Joint attention of ACiT. Top: joint self-attention inside the Inverted State Encoding on two tasks, averaged over five episodes and ten frames, with the left arm (joints 1–7) and the right arm (joints 8–14) marked. Bottom: joint-to-vision cross attention inside State-Visual Grounding, for the gripper of the right arm and for the elbow of the left arm.
Real-World Experiments
Figure 3 shows the four real-world tasks, and one of the most direct pieces of evidence of the gain appears in failure recovery. Figure 6 is a closed-loop trace of ACiT: the grasp fails at Step 2, the arm falls back to the previous state at Step 3 and re-attempts at Step 4, completing the pick at Step 6. In the rollouts we inspected, the baselines do not back off at the failure point and instead keep searching in place.

Figure 3. Real-world platform and tasks. The Dual reBotArm-B601-DM performs four tasks: (a) pick cube, (b) stack cube, (c) lift pot, and (d) handover cube.

Figure 6. Closed-loop recovery on the real robot. Top: global view. Bottom: gripper view. After a grasp failure (Step 2), ACiT falls back to the previous state (Step 3) and re-attempts (Step 4), completing the pick (Step 6).
A failed grasp is a proprioceptive event: the gripper closes to its limit without an object and the arm does not take the load it expects, while vision stays misleading because the cube is still in the workspace, only displaced. Under equal conditioning the visual evidence can dominate the proprioceptive one, which leaves the policy holding the pose or closing again. ACiT lets proprioception carry the state and reduces vision to a per-joint calibration, so the condition reflects a closed and empty gripper, and the decoded chunk is the back-off-and-retry segment that the demonstrations contain.
Figure 7 reports the success rates under in-distribution, out-of-distribution (unseen object colors and initial poses) and dark-background conditions. ACiT is the strongest or tied-strongest method in every condition and task, and its margin grows under the two shifts: with a dark background it still picks cube at 83%, against less than 40% for the baselines.
Figure 7. Real-world results. Success rates for ACiT, ACT, DP, and FM on four tasks under in-distribution, unseen-OOD, and dark-background-OOD conditions. ACiT is shown first in each group.
Transfer to a Large VLA
We apply ACiT’s conditioning to the post-training of the mainstream model [4], which yields -ACiT. Both models are fine-tuned end-to-end per task, with the PaliGemma VLM [3] and the flow-matching expert keeping their architecture and objective. Only ’s single-frame proprioceptive token is replaced by from the Inverted State Encoding and State-Visual Grounding of ACiT.
Simulation Benchmarks
Figure 8 compares the two models on six RoboTwin tasks. The gains land on the same group of tasks as the ACT-family results of Figure 5: both backbones improve where one frame cannot determine each joint’s action on its own (Pick Dual Bottles, Handover Block, Place Dual Shoes) and stay flat where it can (Lift Pot, Move Can Pot, Shake Bottle). Two backbones that differ in size, visual prior and action head agree on which tasks benefit, so the gain comes from the conditioning interface rather than from a property of a backbone. On the randomized hard split it moves to Shake Bottle and Lift Pot, whose actions depend on velocity and on the phase relation between joints, a relation that exists only in the history. Randomization removes the same fraction for both models, so the gain is not a tolerance to visual shift.
Figure 8. Transferring the conditioning design to a large VLA on RoboTwin 2.0. Success rate of -ACiT (left bar of each task) and (right bar). The pale bar is the easy rate and the solid bar drawn on top of it is the randomized-hard rate, so the pale cap is the drop caused by randomization. Numbers above the pale bars are easy rates, numbers on the solid bars hard rates.
Real-World Transfer
Figure 9 repeats the comparison on the dual-arm B601 platform. -ACiT improves over on average across the four tasks and three conditions (57.0 against 39.4). The gain is largest under the dark-background shift, where nearly fails on the two dual-arm tasks and -ACiT still completes them. In distribution and under the unseen shift -ACiT matches the ACT-based ACiT, so the interface transfers to a large pretrained backbone at a lower cost per action (Table 3). Averaged over the 12 RoboTwin settings (6 tasks, 2 splits) and the 12 real-world settings (4 tasks, 3 conditions), -ACiT raises the success rate over by 26.9 points.
Figure 9. Real-world /-ACiT transfer on the B601 platform. Each task has six bars: -ACiT under in-distribution, unseen-object/background, and dark-background conditions (light to dark orange), followed by under the same conditions (light to dark blue).
| Method | Params | Steps | ms/act. | Control (Hz) |
| ACT | 84.0 M | 1 (1) | 0.36 | 2794 |
| DP | 293.2 M | 10 (10) | 2.12 | 471 |
| FM | 293.2 M | 10 (20) | 3.74 | 267 |
| ACiT | 92.6 M | 1 (1) | 0.48 | 2064 |
| 4.03 B | 10 (10) | 5.93 | 169 | |
| -ACiT | 4.05 B | 10 (10) | 4.73 | 211 |
Table 3. Inference cost on one RTX 3090, at the 240 × 320 observation resolution used throughout the RoboTwin experiments. Steps counts integrator steps (network evaluations in brackets), and the last two columns are per executed action.
Ablation
Table 1 removes one module at a time, keeping the history length, the encoder, the protocol and the action head fixed. Equal fusion self-attends visual and joint tokens together. w/o joint attn keeps the per-joint 1D convolution
| Variant | LIBERO | RoboTwin | Real world | |||
| easy | hard | ID | unseen | dark | ||
| ACiT (full) | 79.0 | 62.7 | 14.0 | 60.0 | 34.1 | 44.1 |
| equal fusion | 58.3<br>−20.7 | 52.3<br>−10.4 | 5.0<br>−9.0 | 48.4<br>−11.6 | 17.5<br>−16.6 | 13.3<br>−30.8 |
| w/o joint attn | 69.3<br>−9.7 | 57.0<br>−5.7 | 9.8<br>−4.2 | 52.5<br>−7.5 | 23.3<br>−10.8 | 28.3<br>−15.8 |
| flat state | 65.3<br>−13.7 | 53.3<br>−9.4 | 7.0<br>−7.0 | 49.2<br>−10.8 | 15.0<br>−19.1 | 21.6<br>−22.5 |
| flat decode | 75.0<br>−4.0 | 60.0<br>−2.7 | 11.8<br>−2.2 | 54.2<br>−5.8 | 25.8<br>−8.3 | 41.6<br>−2.5 |
Table 1. Module ablation (mean success rate %). Small numbers give the change relative to the full model, per-task rates in Appendix § 8.§ 1.
but drops the attention among joint tokens, and flat state drops both. Flat decode replaces per-joint decoding with one shared query.
Removing any one module hurts on every benchmark, so the three are complementary rather than redundant. The conditioning relation costs most, the state-side variants next and the action-side factorization least, in five of the six settings. The exception is the unseen condition, where flat state falls further than equal fusion, so under unreliable vision the per-joint state representation matters more than the fusion relation.
Robustness
Training Seeds. Two further seeds leave the task ranking unchanged and move each task by at most eight points. Averaged over the tasks the success rate is , so the aggregate in Figure 5 does not depend on the seed, while a single task carries the few points of spread that 100 episodes leave.
History Length. Figure 10 shows a flat optimum: performance peaks at eight frames, changes little between four and sixteen, and is clearly worse with a single frame, whose cost is largest under the dark-background shift. We keep eight frames throughout.
Figure 10. History length of the conditioning input. (a) Simulation: RoboTwin easy and hard, and LIBERO. (b) Real world: in-distribution, unseen and dark-background. All curves peak at or tie it at 16; the per-task rates are in Appendix § 8.
Efficiency
Table 3 reports inference cost and model size. ACiT adds only 8.6 M parameters over ACT (92.6 M against 84.0 M) and keeps its one-step sampler, at 0.48 ms per action, while DP and FM pay ten integrator steps at a 3.2× larger model. The same holds after transfer: -ACiT needs 4.73 ms per action against 5.93 ms for , at the same ten steps and essentially the same size (4.05 B against 4.03 B), so the reduction comes from the interface, not from a cheaper sampler.
Conclusion
We presented ACiT, which treats historical joint coupling as the primary condition and vision as a later calibration: each joint carries its own history, retrieves its own visual evidence and decodes its own trajectory. It improves most LIBERO, RoboTwin 2.0 and real-world tasks, and transfers to a pretrained VLA at lower cost.
Limitations. ACiT is instantiated only with a continuous action head, so it does not cover the discrete action tokens that autoregressive VLAs rely on. Extending it there needs demonstration data beyond what our dual-arm platform provides, which we leave to future work.
Acknowledgments
We thank Shenzhen Zhibo Data Technology Co., Ltd. for the professional collection of the real-robot data used in this work.
Experimental Setup
Simulation Benchmarks
RoboTwin 2.0 is a bimanual platform with two 6-DoF arms and parallel grippers, simulated for table-top manipulation. We use six tasks, shown in Figure 11. Lift Pot lifts a pot by both handles and holds it; Pick Dual Bottles grasps and lifts two bottles at once; Handover Block passes a block from one arm to the other; Move Can Pot lays the can against the pot; Place Dual Shoes places two shoes into a box; Shake Bottle lifts a bottle and shakes it. Each task comes with an easy split (demo_clean) and a randomized split (demo_randomized) in which object poses, object colors and the scene background are randomized. All numbers are 100 episodes per task, and the success criterion is the one shipped with the benchmark, listed in Table 4.
| Task | Success condition |
| Lift Pot | pot above 0.82 m and upright, both TCPs within 3 cm of its handles |
| Pick Dual Bottles | both bottles within 10 cm of their targets and above 0.89 m |
| Handover Block | block within 3 cm of the target pose and right gripper open |
| Move Can Pot | can tipped on its side beside the pot (<20 cm, 3.5 cm, 15°), at table height, both grippers open |
| Place Dual Shoes | both shoes at the target pose (5 cm, 0.07 quaternion) and both grippers open |
| Shake Bottle | bottle raised above 0.8 m |
Table 4. Success conditions of the six RoboTwin tasks, as implemented by the benchmark. A rollout counts as a success only when every condition of its task holds at the same step.

Figure 11. The six RoboTwin 2.0 tasks used in § 4.2, viewed from the head camera. Both arms are visible in each panel; the randomized split varies object poses, colors and the background of the same scenes.
LIBERO evaluates single-arm table-top manipulation with language instructions. We use three suites: LIBERO-10 (ten long-horizon tasks), LIBERO-Object (objects unseen in training) and LIBERO-Spatial (spatial relations), with 100 episodes per suite.
Implementation Details
The ACT-family models share one protocol: 50,000 training steps, batch size 16, action chunk 50 with 50 executed steps, seed 1000, and AdamW [20]. The visual backbone is a ResNet-18 [9]. All models are implemented in PyTorch [26] with LeRobot [6], and the simulator is MuJoCo [32]. For run-to-run variation we follow the protocol of ACT [41], which reports three training seeds on simulation and a single trained policy on hardware, where every additional seed needs a full physical re-evaluation. Table 2 of the main paper reports the same sweep. ACiT uses a state history of frames. ACT conditions on the current state only; DP and FM take frames and flatten them into the global condition, which is the setting their original papers use. DP and FM execute 32 steps in the 50k_final runs and 10 in the 5task runs.
| Task | seed 1000 | seed 1001 | seed 1002 | mean ± std |
| Lift Pot | 55 | 56 | 54 | 55.0 ± 1.0 |
| Pick Dual | 61 | 63 | 59 | 61.0 ± 2.0 |
| Handover | 69 | 69 | 69 | 69.0 ± 0.0 |
| Move Can | 64 | 60 | 68 | 64.0 ± 4.0 |
| Place Dual Shoes | 41 | 35 | 38 | 38.0 ± 3.0 |
| Shake Bottle | 88 | 89 | 90 | 89.0 ± 1.0 |
Table 2. Training-seed variance of ACiT (easy split, success rate over 100 episodes, hence integer).
Architecturally, ACiT keeps the ACT transformer (, eight heads, FFN width 3200, four encoder and seven decoder layers, 32-d CVAE latent) and adds a two-layer state encoder followed by a two-layer cross-attention block. The backbone is left intact; -ACiT adds a two-layer state encoder and a single grounding layer to it and keeps the flow-matching action expert unchanged.
For the VLA transfer (§ 4.4), we initialize -ACiT from pi0_base. The PaliGemma [3] backbone and the flow-matching action expert keep their original architecture, and ’s single-frame proprioceptive token is replaced by the two conditioning modules of ACiT. The Inverted State Encoding encodes each joint’s eight-frame history with a 1D convolution and a linear projection into a joint token, and forms the proprioceptive prior by inter-joint self-attention. State-Visual Grounding then applies a joint-level cross-attention over the visual tokens, producing as the proprioceptive condition of the action expert. The Inverted Action Chunking is not transferred, and the flow-matching objective is unchanged, so the action head keeps its architecture and capacity and only the conditioning side changes. Both models are trained per task with full fine-tuning, batch size 32, for 50k steps on two A100 GPUs.
Evaluation follows the benchmark defaults: 240 × 320 on RoboTwin for every model, with 100 episodes per task, 256 × 256 on LIBERO with 100 episodes per suite, and 30 rollouts per task and condition on the real-world platform, where training uses 50 demonstrations per task, the same recipe as above, and all rollouts run on a single RTX 3090.
Ablations, Sensitivity and Cost
Module Ablation
Table 5 and Table 6 give the per-task rates behind Table 1. The three removals do not fail in the same places. Equal fusion costs the most on Handover Block and Place Dual Shoes, the two tasks whose success turns on coupling between joints, and it is almost free on Move Can Pot. Flat state costs a nearly constant amount on every task, which is what one expects when the model loses the per-joint state representation rather than a specific coupling. Flat decode is the opposite: it loses on Pick Dual Bottles and Place Dual Shoes, where each joint needs its own trajectory, and it gains on Handover Block and Move Can Pot, where one shared trajectory keeps the joints in phase.
| Variant | RoboTwin (easy) Lift | Pick | Hand. | Move | Shoes | Shake | Mean | RoboTwin (hard) Lift | Pick | Hand. | Move | Shoes | Shake | Mean | LIBERO 10 | Obj. | Spa. | Mean |
| ACiT | 55 | 61 | 69 | 64 | 38 | 89 | 62.7 | 4 | 14 | 0 | 19 | 0 | 47 | 14.0 | 50 | 99 | 88 | 79.0 |
| equal fusion | 56 | 50 | 30 | 68 | 22 | 88 | 52.3 | 1 | 2 | 0 | 2 | 0 | 25 | 5.0 | 30 | 73 | 72 | 58.3 |
| w/o joint attn | 39 | 60 | 69 | 62 | 30 | 82 | 57.0 | 3 | 6 | 0 | 12 | 0 | 38 | 9.8 | 38 | 90 | 80 | 69.3 |
| flat state | 48 | 50 | 58 | 56 | 28 | 80 | 53.3 | 2 | 3 | 0 | 7 | 0 | 30 | 7.0 | 35 | 85 | 76 | 65.3 |
| flat decode | 54 | 45 | 79 | 74 | 23 | 85 | 60.0 | 5 | 8 | 1 | 16 | 0 | 41 | 11.8 | 45 | 95 | 85 | 75.0 |
Table 5. Per-task module ablation on the simulation benchmarks (success rate, %, 100 episodes per task).
| Variant | In-distribution Pick | Stack | Pot | Hand. | Mean | Unseen Pick | Stack | Pot | Hand. | Mean | Dark background Pick | Stack | Pot | Hand. | Mean |
| ACiT | 93.3 | 63.3 | 33.3 | 50.0 | 60.0 | 53.3 | 23.3 | 16.7 | 43.3 | 34.1 | 83.3 | 43.3 | 16.7 | 33.3 | 44.1 |
| equal fusion | 76.7 | 56.7 | 23.3 | 36.7 | 48.4 | 40.0 | 6.7 | 6.7 | 16.7 | 17.5 | 30.0 | 13.3 | 0.0 | 10.0 | 13.3 |
| w/o joint attn | 80.0 | 56.7 | 30.0 | 43.3 | 52.5 | 43.3 | 13.3 | 13.3 | 23.3 | 23.3 | 53.3 | 23.3 | 13.3 | 23.3 | 28.3 |
| flat state | 70.0 | 56.7 | 23.3 | 46.7 | 49.2 | 33.3 | 6.7 | 6.7 | 13.3 | 15.0 | 43.3 | 23.3 | 6.7 | 13.3 | 21.6 |
| flat decode | 83.3 | 46.7 | 30.0 | 56.7 | 54.2 | 43.3 | 13.3 | 13.3 | 33.3 | 25.8 | 73.3 | 30.0 | 20.0 | 43.3 | 41.6 |
Table 6. Per-task module ablation on the real-world platform (success rate, %; 30 rollouts per cell).
Sensitivity to History Length
History length is a hyperparameter of the conditioning interface rather than a module, so we report it separately. Table 7 and Table 8 vary the number of proprioceptive frames and keep everything else fixed. The same pattern appears everywhere: one frame is the weakest, four recovers most of the gap, eight is best on RoboTwin and on the real platform and matches four on LIBERO, and sixteen is slightly worse, which is what one expects when extra frames add redundancy but no new dynamics.
| RoboTwin (easy) Lift | Pick | Hand. | Move | Shoes | Shake | Mean | RoboTwin (hard) Lift | Pick | Hand. | Move | Shoes | Shake | Mean | LIBERO 10 | Obj. | Spa. | Mean | |
| 1 | 52 | 45 | 60 | 58 | 34 | 85 | 55.7 | 2 | 5 | 0 | 12 | 0 | 35 | 9.0 | 46 | 91 | 82 | 73.0 |
| 4 | 54 | 56 | 66 | 62 | 36 | 87 | 60.2 | 3 | 10 | 0 | 16 | 0 | 42 | 11.8 | 51 | 99 | 88 | 79.3 |
| 8 | 55 | 61 | 69 | 64 | 38 | 89 | 62.7 | 4 | 14 | 0 | 19 | 0 | 47 | 14.0 | 50 | 99 | 88 | 79.0 |
| 16 | 54 | 60 | 68 | 63 | 37 | 88 | 61.7 | 4 | 13 | 0 | 18 | 0 | 45 | 13.3 | 49 | 98 | 87 | 78.0 |
Table 7. History length on the simulation benchmarks (success rate, %, 100 episodes per task). is the history length, in frames.
| \multicolumn{5}{c}{In-distribution} | \multicolumn{5}{c}{Unseen} | \multicolumn{5}{c}{Dark background} | |||||||||||||
| Pick | Stack | Pot | Hand. | Mean | Pick | Stack | Pot | Hand. | Mean | Pick | Stack | Pot | Hand. | Mean | |
| 1 | 76.7 | 46.7 | 23.3 | 40.0 | 46.7 | 43.3 | 16.7 | 13.3 | 33.3 | 26.6 | 70.0 | 33.3 | 10.0 | 20.0 | 33.3 |
| 4 | 86.7 | 56.7 | 30.0 | 46.7 | 55.0 | 46.7 | 20.0 | 13.3 | 36.7 | 29.2 | 76.7 | 36.7 | 13.3 | 26.7 | 38.4 |
| 8 | 93.3 | 63.3 | 33.3 | 50.0 | 60.0 | 53.3 | 23.3 | 16.7 | 43.3 | 34.1 | 83.3 | 43.3 | 16.7 | 33.3 | 44.1 |
| 16 | 90.0 | 60.0 | 30.0 | 46.7 | 56.7 | 50.0 | 20.0 | 16.7 | 40.0 | 31.7 | 83.3 | 43.3 | 16.7 | 33.3 | 44.1 |
Table 8. History length on the real-world platform (success rate, %; 30 rollouts per cell).
Inference Cost
Cost is measured on a single RTX 3090 (24 GB) with batch size 1 and fp16, and each entry of Table 3 averages 100 calls
after 20 warm-up calls on the RoboTwin Lift Pot checkpoints at 50k steps. The table reports the cost per executed action, which divides one inference by the number of actions run without re-planning, i.e., what a closed-loop controller pays. The figures below are per inference, so that samplers can be compared directly. Step counts alone do not fix the cost, because integrators differ in how many network evaluations (NFE) they spend per step. DP and FM share the same U-Net and the same parameter count but not the same sampler: DP takes one evaluation per DDIM step, whereas FM integrates with a midpoint rule that evaluates the network twice per step, so its ten steps cost twenty evaluations (119.8 ms per inference) and match DP at twenty steps (120.5 ms); at an equal budget of ten evaluations the two coincide (67.5 against 68.1 ms). We therefore report both steps and NFE, and read cost differences across families as sampler differences unless the NFE budget is held fixed.
Seed Variance
Table 2 of the main paper repeats the training of ACiT and ACT on RoboTwin with two further seeds. The evaluation episodes are shared across runs, so the only source of variation is the training itself, and the spread across seeds stays small relative to the gap between the two models.
Analysis
Failure Cases
Figure 12 and Figure 13 also show how the methods fail once the scene is shifted. The baselines fail at the grasp itself: under a changed background, ACT and FM approach the cube with an offset and close on empty space, and DP pushes it away before closing; under a changed object color the same offset appears, which is the behavior expected when a single frame has to carry both the pose and the appearance. ACiT keeps the grasp in the same frames and fails later, when placing the object. Its remaining failures are therefore concentrated in placement precision and in the phase of a handover, not in grasp stability, which is consistent with Figure 7: the gap to the baselines is smallest on the two tasks whose success is dominated by the grasp and largest under the dark shift.
Figure 12. Out-of-distribution dark background. ACT, DP, FM and ACiT on the four tasks under the background shift.
Figure 13. Out-of-distribution unseen initial state and color, on Pick cube and Stack cube.
Figure 14–Figure 19 replay one episode of each task under all four policies, with the checker outcome and the number of steps used marked on the last frame; the success conditions are listed in Table 4; every row of a figure shares the same initial state. On Place Dual Shoes success requires both shoes inside the box with the target orientation and both grippers open, which only ACiT satisfies. On Pick Dual Bottles ACiT lifts the bottle within a fraction of the step budget while the baselines never leave the table. On Lift Pot ACiT and DP both succeed, with ACiT finishing in fewer steps, while ACT and FM fail. On Move Can Pot only ACiT brings the can to the pot. On Shake Bottle DP never lifts the bottle, and on LIBERO-Spatial all four policies succeed with ACiT taking the fewest steps.

Figure 14. Qualitative comparison on Place Dual Shoes, same task and episode under four policies. The outcome and the number of steps are marked on the last frame. Only ACiT leaves both shoes in the box and releases them; ACT keeps a shoe in its closed gripper, and DP and FM stop with one shoe outside the box.

Figure 19. Qualitative comparison on LIBERO-Spatial. All four policies solve the episode, and ACiT finishes in the fewest steps.
Action-Space Dependency Metrics
The motivation for the architecture rests on two properties of the action distribution: its effective dimensionality is far below the number of joints, and the joints are strongly coupled, with part of that coupling delayed in time. We measure
Attention Maps and Head Matching
Figure 20 compares the two matrices. For each dual-arm task we compute the mean self-attention map of each head over 5 episodes and 10 frames, symmetrize it, and compare it with the absolute zero-lag action-correlation matrix using Spearman correlation over off-diagonal entries. The best head per task is only weakly aligned: Lift Pot 0.05 (head 5), Pick Dual 0.12 (head 6), Handover 0.18 (head 2), and Shoes 0.26 (head 5). Thus head-level attention is not a reliable proxy for the action-correlation structure. We therefore report it as a qualitative supplement to the quantitative statistics of §C.2.
Figure 20. Action correlation matrix (left) versus the best-matching self-attention head (right) for each dual-arm task. The Spearman correlation over off-diagonal entries is shown above each attention map.
Extended Related Work
Proprioceptive History: Representation and Conditioning
Action policies are moving from single-frame proprioceptive conditions to explicit history modeling. In the continuous route, ACT [41] maps the current joint state and visual observations to an action chunk with a CVAE. Diffusion Policy [8] conditions on the last observations, proprioception included, concatenated into one global vector before conditional denoising; its own ablation reports that a vision-based policy degrades as grows, which suggests that a flattened history does not scale. A2A [12] embeds historical proprioceptive sequences into the latent space of action generation, and Chronos [42] treats observation history as the latent state of the policy. Meanwhile, another route discretizes historical actions or state-action trajectories into tokens for autoregressive VLAs at the cost of losing continuous coupling: VQ-BeT [14] uses a residual VQ-VAE to encode continuous action chunks into discrete latent codes and adds a continuous offset for precision. FAST [28] compresses action chunks in the frequency domain with DCT and BPE, substantially reducing autoregressive training cost. OmniSAT [21] further uses B-spline encoding and multi-stage residual quantization to quantize position, rotation, and gripper subspaces separately. VITA [22] learns a shared discrete latent space for vision and action and internalizes visual dynamics as an implicit chain of thought. Both continuous and discrete routes validate the value of using history [35, 40]. However, under either interface, proprioception is still represented as a flat vector or a one-dimensional token sequence, and the continuous coupling among joints is not explicitly modeled.
At the fusion level, mainstream methods symmetrically concatenate vision and proprioception into the same attention stack. A few works instead let the robot state actively guide vision: RcP [43] uses proprioceptive-state correspondence to guide robot-centric pooling of image features, extracting robot-centric visual representations robust to self-distractors. AVA-VLA [39] uses a recurrent history belief to reweight current visual tokens. These explorations show that proprioception should not be only a passive condition, but they stop at perceptual filtering and do not carry structured joint priors through to action generation.
| Task | PR | within-L | within-R | cross(0) | cross(best) |
| Lift Pot | 1.44 | 0.602 | 0.579 | 0.802 | 0.956 |
| Pick Dual Bottles | 2.71 | 0.341 | 0.324 | 0.579 | 0.881 |
| Handover Block | 1.92 | 0.422 | 0.596 | 0.405 | 0.716 |
| Place Dual Shoes | 4.42 | 0.341 | 0.301 | 0.352 | 0.670 |
Table 9. Action-space statistics on the four dual-arm RoboTwin tasks. Move Can Pot and Shake Bottle are single-arm and are not listed, because their cross-arm quantities are undefined.
Topology-Aware Morphological Bias
Standard VLA policies are usually embodiment-agnostic and must infer kinematic structure from data. GET-Zero [27] proposes a Graph Embodiment Transformer that uses embodiment graph connectivity as a learned structural bias in attention, enabling zero-shot adaptation to new joint configurations and link lengths. Human2LocoMan [24] further proposes a Modularized Cross-embodiment Transformer (MXT) that uses embodiment-specific tokenizers and detokenizers on a shared Transformer trunk to handle heterogeneous bodies. These works establish that joints are not independent dimensions and that the kinematic graph should be an explicit inductive bias. However, existing topology methods mainly target cross-embodiment transfer and action-interface design. They do not treat continuous per-joint proprioceptive history as a generative prior, nor let each joint retrieve its own visual calibration signal. ACiT unifies continuous per-joint history, kinematic topology, and per-joint visual grounding in a single inverted generation pipeline.

Figure 15. Qualitative comparison on Pick Dual Bottles. ACiT lifts the bottle clear of the table and finishes well inside the step budget, while ACT, DP and FM leave both bottles in place.

Figure 16. Qualitative comparison on Lift Pot. ACiT and DP both raise the pot, with ACiT finishing in fewer steps, while ACT and FM fail to lift it.

Figure 17. Qualitative comparison on Move Can Pot. ACiT places the can against the pot and finishes in 158 steps; ACT never moves it into place, DP does not approach it, and FM knocks it away from its start position.

Figure 18. Qualitative comparison on Shake Bottle. ACiT, ACT and FM shake the bottle, while DP never lifts it and is marked failure.
References
- [1]Minoru Asada. Proprioception and body schema. In The Sense of Self: Perspectives from Science and Zen Buddhism. Oxford University Press, 2018. doi: 10.1093/oso/9780199674923.003.0018.DOI
- [2]Marie Avillac, Sophie Denève, Etienne Olivier, Alexandre Pouget, and Jean-René Duhamel. Reference frames for representing visual and tactile locations in parietal cortex. Nature Neuroscience, 8(7):941–949, 2005. doi: 10.1038/nn1480.DOI
- [3]Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3B VLM for transfer. arXiv preprint arXiv:2407.07726, 2024.arxiv.org/abs/2407.07726
- [4]Kevin Black, Noah Brown, Danny Driess, et al. π₀: A vision-language-action flow model for general robot control. arXiv preprint, 2024.DOI
- [5]Kevin Black et al. π₀.₅: A vision-language-action model with open-world generalization. https://www. physicalintelligence.company/blog/pi05, 2025.
- [6]Remi Cadene, Simon Alibert, Alexander Soare, Quentin Gallouedec, Adil Zouitine, and Thomas Wolf. LeRobot: State-of-the-art machine learning for real-world robotics in PyTorch. https://github.com/huggingface/lerobot, 2024.
- [7]Tianxing Chen et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint arXiv:2506.18088, 2025.arxiv.org/abs/2506.18088
- [8]Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems (RSS), 2023.
- [9]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.arxiv.org/abs/1512.03385
- [10]Tobias Heed, Verena N. Buchholz, Andreas K. Engel, and Brigitte Röder. Tactile remapping: From coordinate transformation to integration in sensorimotor processing. Trends in Cognitive Sciences, 19(5):251–258, 2015. doi: 10.1016/j.tics.2015.03.001.DOI
- [11]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.arxiv.org/abs/2006.11239
- [12]Jindou Jia, Gen Li, Xiangyu Chen, Tuo An, Yuxuan Hu, Jingliang Li, Xinying Guo, and Jianfei Yang. Action-to-action flow matching. arXiv preprint arXiv:2602.07322, 2026.arxiv.org/abs/2602.07322
- [13]Konrad P. Körding and Daniel M. Wolpert. Bayesian integration in sensorimotor learning. Nature, 427(6971):244–247, 2004. doi: 10.1038/nature02169.DOI
- [14]Seungjae Lee, Yibin Wang, Haritheja Etukuru, H. Jin Kim, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. Behavior generation with latent actions. In International Conference on Machine Learning (ICML), 2024.arxiv.org/abs/2403.03181
- [15]Jijie Li, Xu Yang, Junhong Zou, Chunhai Zhao, Chaoyang Zhao, Zhen Lei, and Xiangyu Zhu. Object-centric conditioning for visuomotor flow matching. In Conference on Robot Learning (CoRL), 2026.arxiv.org/abs/2609.24155
- [16]Wenjie Li, Yash Jangir, Ignacy Stepka, Yash Agarwal, Marion Kipsang, and Yonatan Bisk. Lost in reconstruction: Aligning action representations with language in vision-language-action models. arXiv preprint arXiv:2608.10484, 2026.arxiv.org/abs/2608.10484
- [17]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.02747
- [18]Bo Liu, Yifan Zhu, Chongkai Gao, Yuxuan Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2306.03310
- [19]Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. iTransformer: Inverted transformers are effective for time series forecasting. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2310.06625
- [20]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019.
- [21]Huaihai Lyu, Chaofan Chen, Senwei Xie, Pengwei Wang, Xiansheng Chen, Shanghang Zhang, and Changsheng Xu. Omnisat: Compact action token, faster auto regression. arXiv preprint arXiv:2510.09667, 2025.arxiv.org/abs/2510.09667
- [22]Xiangkai Ma, Lekai Xing, Han Zhang, Wenzhong Li, and Sanglu Lu. Unifying perception and action: A hybrid-modality pipeline with implicit visual chain-of-thought for robotic action generation. arXiv preprint arXiv:2511.19859, 2025.arxiv.org/abs/2511.19859
- [23]R. Chris Miall and Daniel M. Wolpert. Forward models for physiological motor control. Neural Networks, 9(8):1265–1279, 1996. doi: 10.1016/S0893-6080(96)00035-4.DOI
- [24]Yaru Niu, Yunzhe Zhang, Mingyang Yu, Changyi Lin, Chenhao Li, Yikai Wang, Yuxiang Yang, Wenhao Yu, Tingnan Zhang, Zhenzhen Li, Jonathan Francis, Bingqing Chen, Jie Tan, and Ding Zhao. Human2locoman: Learning versatile quadrupedal manipulation with human pretraining. In Robotics: Science and Systems (RSS), 2025.arxiv.org/abs/2506.16475
- [25]NVIDIA. Gr00t n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025.
- [26]Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS), 2019.arxiv.org/abs/1912.01703
- [27]Austin Patel and Shuran Song. Get-zero: Graph embodiment transformer for zero-shot embodiment generalization. In IEEE International Conference on Robotics and Automation (ICRA), 2025.arxiv.org/abs/2407.15002
- [28]Karl Pertsch, Kyle Stachowicz, Brian Ichter, et al. Fast: Efficient action tokenization for vision-language-action models. In Robotics: Science and Systems (RSS), 2025.arxiv.org/abs/2501.09747
- [29]Uwe Proske and Simon C. Gandevia. The proprioceptive senses: Their roles in signaling body shape, body position and movement, and muscle force. Physiological Reviews, 92(4):1651–1697, 2012. doi: 10.1152/physrev.00048.2011.DOI
- [30]Seeed Studio. rebot arm b601-dm: Fully open-source 6-dof robotic arm. https://github.com/Seeed-Projects/reBot-DevArm, 2026.
- [31]Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2015.
- [32]Emanuel Todorov, Tom Erez, and Yuval Tassa. MuJoCo: A physics engine for model-based control. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012.DOI
- [33]Robert J. van Beers, Anne C. Sittig, and Jan J. Denier van der Gon. Integration of proprioceptive and visual position-information: An experimentally supported model. Journal of Neurophysiology, 81(3):1355–1364, 1999. doi: 10.1152/jn.1999.81.3.1355.DOI
- [34]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Lilian Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017.arxiv.org/abs/1706.03762
- [35]Fangyuan Wang, Peng Zhou, Jiaming Qi, Shipeng Lyu, Chengyang He, David Navarro-Alarcon, and Guodong Guo. Think proprioceptively: State-grounded visual token selection for vla policies. arXiv preprint arXiv:2602.06575, 2026.arxiv.org/abs/2602.06575
- [36]Uwe Windhorst. Muscle proprioceptive feedback and spinal networks. Brain Research Bulletin, 73(4-6):155–202, 2007. doi: 10.1016/j.brainresbull.2007.03.010.DOI
- [37]Daniel M. Wolpert, Zoubin Ghahramani, and Michael I. Jordan. An internal model for sensorimotor integration. Science, 269(5232):1880–1882, 1995. doi: 10.1126/science.7569931.DOI
- [38]Yilin Wu, Rutav Shah, Shuaiyi Huang, Sumedh A. Sontakke, Homanga Bharadhwaj, Andrea Bajcsy, Max Simchowitz, and Guanya Shi. Memory without curation: Learning multi-stream latent memory for robot policies. arXiv preprint arXiv:2610.14406, 2026.arxiv.org/abs/2610.14406
- [39]Lei Xiao, Jifeng Li, Juntao Gao, Feiyang Ye, Yan Jin, Jingjing Qian, Jing Zhang, Yong Wu, and Xiaoyuan Yu. Ava-vla: Improving vision-language-action models with active visual attention. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.arxiv.org/abs/2511.18960
- [40]Senlan Yao, Chenyu Yang, Jaehoon Kim, Aristotelis Sympetheros, and Robert K. Katschmann. Learning robust dexterous in-hand manipulation from joint sensors with proprioceptive transformer. arXiv preprint arXiv:2605.21330, 2026.arxiv.org/abs/2605.21330
- [41]Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems (RSS), 2023.arxiv.org/abs/2304.13705
- [42]Yulin Zhou, Yimeng Wang, Nengyu Wang, Shaojia Xing, Shiyun Tu, Xiang Li, Jingkai Zhang, Ningbo Jiang, Yuankai Lin, Hua Yang, Xiangrui Zeng, and Zhouping Yin. Chronos: A physics-informed full-history framework for non-markovian long-horizon manipulation. arXiv preprint arXiv:2606.30318, 2026.arxiv.org/abs/2606.30318
- [43]Zheyu Zhuang, Ville Kyrki, and Danica Kragic. Raising body ownership in end-to-end visuomotor policy learning via robot-centric pooling. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.arxiv.org/abs/2411.04331










