WorksComputer Science
ActMem: From Latent Memory Reasoning to Grounded Robotic Action
Abstract
Vision-Language-Action (VLA) models have shown promising results in real-world long-horizon manipulation, where memory plays a crucial role. However, existing memory-augmented VLAs either retrieve past observations without reasoning about what they imply, or reason over the full history in a shared context where it crowds out current perception. Cognitive science treats them as distinct systems: memory reconstructs past experience to decide what to do, while perception guides how to act. We therefore propose ActMem, an in-context memory framework that decouples memory reasoning from perceptual grounding within a single vision-language backbone. Built on a vision-language model with native video understanding, ActMem infers the current subtask from a timestamped interaction history and keeps the hidden states formed during this inference as a Latent Memory Chain-of-Thought (Memory CoT), which conditions a flow-matching action expert. Perception Attention further lets the expert attend directly to the current view. To keep inference fast under long histories, Efficient Memory CoT Inference reuses computation across consecutive decisions without changing the output. ActMem achieves 71.1% task success and 80.8% completion success on RoboMemArena, surpassing the strongest baseline by 7.5 and 8.7 percentage points, and 73.8% average success on MIKASA-Robo, the best among all compared methods. In the real world, it reaches 68.8% average success on exogenous, endogenous, and traceless memory tasks, 10.0 percentage points above the strongest memory-augmented VLA. Code and checkpoints will be released at https://github.com/LigphiDonk/ActMem and https://huggingface.co/donk111/ActMem.
Introduction
Vision-Language-Action (VLA) models have become a leading approach to generalist robot manipulation [4, 5, 16, 40] and have begun to tackle long-horizon tasks in the real world [13, 29]. Most of them, however, process each frame in isolation [5, 13, 16], treating the current observation as the
full state of the task. This assumption breaks once the scene no longer bears a trace of what has happened: after a block is hidden under a cup, that cup looks like any other, yet the correct action has changed. Which subgoals are complete, or where an occluded object lies, cannot be read from the current frame [7, 15].
Recent memory-augmented VLAs therefore incorporate history, but they fall short in one of two ways. Memory-bank and retrieval policies such as MemoryVLA [27], MEM [32], and MemER [30] keep the current view intact but store past observations, leaving the action module to work out what they imply. Native-video policies such as Simple-MemVLA [37] do reason about history-dependent subtasks, but in the same context that must also hold the current view, so the visual detail needed to act competes with an ever-growing history. The challenge is thus to reason over the full history without crowding out current perception.
Our key observation is that a memory-dependent decision needs not a past observation itself, but what the history implies for the current step. Past frames record how the scene looked, but decision-relevant information such as task progress and operation counts appears in no single frame, only in the relations among frames [6, 7, 9, 19]. When a robot pulls tissues from a box, no image shows how many have been pulled: the count is never observed, only inferred, so memory must reason over history rather than store it.
Such inferences and the current view are different kinds of information. Cognitive science has long assigned them to distinct systems: memory reconstructs past experience to determine what to do [26], while perception guides how to act [12]. This motivates a simple principle: a VLA should keep what it has experienced separate from what it perceives. Experience is what must be inferred from interaction history, such as progress, repetition counts, and where a now-hidden object was placed; perception is what the current view shows directly, such as object pose, geometry, and the gripper’s relation to the target, which determine the next motion. The boundary is not between past and present frames, but between what must be inferred and what can be observed.
Following this principle, we present ActMem, an in-context memory framework that reasons over experience in latent space and grounds action in current perception (Figure 1). A pretrained vision-language model reads the timestamped interaction history, the current observation, and the instruction in one context, and autoregressively infers the current subtask. We keep the contextualized hidden states of these subtask tokens as Latent Memory Chain-of-Thought (Memory CoT) and condition a flow-matching action expert on them. Because these states are formed after the backbone has processed the full history, they carry inferences that the subtask text alone does not preserve. Perception Attention additionally lets the expert attend directly to contextualized visual features of the current view. Memory CoT thus carries the reasoning over history, and Perception Attention gives the expert a direct path to the current view, so that perception no longer competes with history.
Figure 1. ActMem: history becomes action. (a) Human-inspired action: past experience is reconstructed into memory and spatial reasoning that determines what to do, while current perception determines how to act. (b) ActMem realizes this division inside a VLA: latent memory reasoning in the vision-language backbone produces Memory CoT states that condition the action expert, and Perception Attention grounds the action in the current view. (c) RoboMemArena completion success versus task success (left) and memory-task success across RoboMemArena categories and MIKASA-Robo (right).
We evaluate ActMem on two memory-dependent simulation benchmarks, RoboMemArena and MIKASA-Robo, and on four real-world tasks covering exogenous, endogenous, and traceless memory. On RoboMemArena, ActMem reaches 71.1% task success and 80.8% completion success, surpassing the strongest baseline by 7.5 and 8.7 percentage points. It also achieves the best average success on MIKASA-Robo (73.8%) and 68.8% average success in the real world, 10.0 percentage points above the strongest memory-augmented VLA. By reusing computation across consecutive decisions without changing the output, ActMem keeps per-decision latency below 0.6 s even with long histories.
Our main contributions are summarized as follows:
A memory principle for VLAs. We identify that existing memory-augmented VLAs either retrieve history without reasoning about it or reason over it at the cost of current perception, and propose, consistent with the distinction between memory and perception in cognitive science, that a VLA should separate what it has experienced from what it perceives: memory reasons over history about what to do, while perception guides how to act in the current view.
A memory-reasoning VLA architecture. ActMem conditions a flow-matching action expert on Latent Memory CoT from history-conditioned subtask inference, grounds generation in the current view via Perception Attention, and uses Efficient Memory CoT Inference to stay fast under long histories.
Strong performance across memory types. ActMem achieves the best average success on RoboMemArena and MIKASA-Robo, and outperforms prior memory-augmented VLAs on four real-world tasks spanning exogenous, endogenous, and traceless memory.
Related Work
Vision-Language-Action Models. Pretrained vision-language models (VLMs) [1, 2, 3] provide the semantic and visual knowledge on which VLA models build to map visual observations and language instructions to robot actions [4, 5, 8, 16, 17, 22, 25, 31, 40]. Early approaches formulate control as autoregressive prediction over discretized action tokens [16, 40], while more recent methods employ diffusion or flow-matching action experts to generate continuous motor commands [4, 5, 22]. However, most VLAs condition on the current observation alone, even when their VLM backbones can process video, and thus cannot recover task-relevant events that are no longer visible.
Memory for Robotic Manipulation. Recent benchmarks formalize memory-dependent manipulation, where the correct action depends on events no longer visible in the current scene [6, 7, 9, 19]. Existing memory-augmented policies differ mainly in how they supply history: stacking past frames or motion features as temporal context [14, 18, 21, 23], maintaining memory banks of past observations [10, 27, 28, 32, 34], retrieving task-relevant keyframes [30, 35, 38], or encoding timestamped histories as native video [7]. Memory has also been explored in world action models [33, 36] and in-context or test-time adaptation [11, 24]. In most of these approaches, history reaches the action module as observations, whether raw, compressed, or retrieved, so the module must still infer what that history implies for the current step.
Reasoning for Action Generation. Intermediate reasoning has been introduced to bridge high-level understanding and low-level control. Hierarchical VLAs predict language sub-tasks that serve as instructions for the action policy [13, 29], while action-space reasoning methods generate coarse reference trajectories and latent action priors [39]. These methods reason about the current task rather than over the interaction history, and language-based variants pass only the decoded text to the action generator.
More recently, native-video approaches infer history-dependent subtasks from timestamped interaction histories to guide a flow-matching action expert [37], yet memory reasoning and current perception still reach the action expert through a shared context. ActMem instead separates the roles of experience and perception: Latent Memory CoT carries the history-conditioned reasoning states to the action expert, while Perception Attention grounds action generation in the current view.
Method
We present ActMem, an in-context memory framework that separates experience from perception, as illustrated in Figure 2. A pretrained vision-language backbone infers the current subtask from the timestamped history; the resulting hidden states are kept as Latent Memory CoT to condition a flow-matching action expert, while Perception Attention grounds generation in the current view.
Figure 2. Overview of ActMem. (a) A pretrained vision-language backbone (Qwen3-VL) encodes the instruction, the timestamped historical frames, and the current perception, and performs latent memory reasoning inside the backbone to produce the subtask together with its Latent Memory CoT states , which condition a flow-matching action expert; Perception Attention additionally routes the current visual features to the action expert. (b) Inside the backbone, the subtask tokens that form Memory CoT follow the history and the current view in the sequence, so under the causal mask they read both, and their hidden states become ; the action expert receives only , not the history tokens. (c) Efficient Memory CoT Inference. Consecutive decisions usually share the same subtask, so the previous subtask is reused as a draft and verified against the updated context in a single parallel pass. If it is accepted, its hidden states are recomputed to give the new , with no token-by-token decoding.
Preliminaries and Architecture Overview
Problem Formulation. We consider language-conditioned robotic manipulation under partial observability, where successful action generation may require information from previous interactions. At each decision step , the robot receives a language instruction , a current visual observation , and a proprioceptive state . In memory-dependent scenarios, the current observation alone may be insufficient to determine the appropriate action, as task-relevant events may have occurred earlier and are no longer directly observable. We therefore denote the sampled interaction history as
where represents a historical visual observation and is its corresponding timestamp. Given the interaction history and current observation, our objective is to learn a history-conditioned policy that generates a continuous action chunk:
where denotes a sequence of robot actions over a prediction horizon .
Architecture Overview. As illustrated in Figure 2, ActMem consists of a pretrained vision-language backbone and a flow-matching action expert . The backbone processes the interaction history, current observations, and language instruction to construct a shared multimodal context. From this context, ActMem derives two forms of action conditioning with distinct functional roles. First, Latent Memory CoT preserves the contextualized representations formed during history-conditioned subtask generation, encoding experience such as task intent and progress as compact guidance for action selection. Second, Perception Attention exposes contextualized visual features of the current view to intermediate action representations, supplying the perceptual information needed for motor execution. The action expert integrates these signals with the current robot state to generate continuous action chunks.
Latent Memory Chain-of-Thought
Rather than compressing or selectively retaining past observations, ActMem lets a pretrained vision-language model with native video understanding reason directly over the interaction history, and keeps the states formed during this reasoning as the memory condition for action generation.
Native-Video Context. Given the interaction history , we construct a timestamped front-camera video , so the backbone can reason about the temporal order of events and the progression of the task. Together with the current wrist-camera view , which serves as in Eq. (2), and the instruction , it forms a single multimodal input . We employ Qwen3-VL as the backbone , which encodes positions across visual and textual tokens with multimodal rotary position embeddings (M-RoPE), yielding contextualized representations .
History-Conditioned Subtask Reasoning. From , the backbone autoregressively infers the subtask for the current stage of the task:
Because this inference is conditioned on the history, it can distinguish visually similar situations that require different actions, such as which drawers have already been inspected.
Memory CoT Conditioning. Rather than conditioning the action expert on a separate encoding of the subtask text or on the full history tokens, we use the final-layer hidden states of the generated subtask span as Latent Memory CoT. Let denote the positions of , we define
where , , and are the hidden state, token embedding, and M-RoPE position of the -th subtask token. As shown in Figure 2(b), the subtask tokens follow the history, the current view, and the instruction, so under the causal mask their hidden states summarize what the backbone has inferred from the full context. The action expert receives only , whose length depends on the subtask rather than on the number of historical frames.
Perception Attention for Action Generation
While history-conditioned subtask representations abstract task intent and progress, continuous action generation also depends on the perceptual details of the scene. We therefore introduce Perception Attention, which lets intermediate action representations attend to contextualized visual features during flow-matching generation, grounding motor execution in richer perceptual context.
Visual Conditioning. Given the multimodal representations produced by the vision-language backbone, let denote the positions of the current-view tokens in . As for , we take the final-layer hidden states at these positions as the visual features:
where is the number of current-view tokens. Since these tokens are computed after the backbone has processed the interaction history, they remain history-contextualized. Unlike the compact Memory CoT conditioning sequence , which provides task-level guidance, preserves a token-level interface to the current scene. These representations allow the action expert to incorporate perceptual information alongside history-conditioned subtask representations.
Gated Visual Cross-Attention. We implement Perception Attention as gated cross-attention at selected layers of the flow-matching Diffusion Transformer (DiT). At the -th selected layer, the intermediate action representations query the visual features :
The resulting visual information is incorporated into the action representations through a learnable residual gate:
where is initialized to zero. This initialization preserves the original action representations at the beginning of training, while allowing the contribution of visual information to be learned progressively. In our implementation, Perception Attention is inserted at layers of a 16-layer DiT, while retaining the original subtask-conditioning mechanism.
Flow-Matching Action Generation. The action expert predicts a velocity field over noisy action chunks at flow timestep , where serves as the conditioning sequence and enters through Perception Attention. Integrating this field from Gaussian noise yields the action chunk .
Policy Learning and Inference
Joint Policy Learning. We train ActMem on manipulation demonstrations containing visual observations, language instructions, subtask annotations, and continuous robot actions. The vision-language backbone and flow-matching action expert are jointly optimized to associate historical context with the appropriate subtask while learning to generate the corresponding robot actions. The overall training objective combines a subtask prediction loss and an action generation loss:
where and control the relative contributions of the two objectives. For history-conditioned subtask prediction, we employ an autoregressive cross-entropy loss over the annotated subtask sequence:
where denotes the ground-truth subtask description. This supervision encourages the vision-language backbone to infer the appropriate task stage from the multimodal context, providing history-conditioned representations for subsequent action generation. As illustrated in Figure 2(a), the contextualized states associated with the subtask tokens form the Memory CoT condition .
Flow-Matching Objective. We train the action expert with conditional flow matching [5]. Given a demonstrated chunk , noise , and , we form and regress the target velocity :
Closed-Loop Inference. During deployment, ActMem maintains a history buffer that accumulates visual observations throughout task execution. At each replanning step, the sampled historical frames, current wrist-camera observation, and language instruction are provided to the vision-language backbone to predict the current subtask. From the resulting context we extract the Memory CoT representations and the visual features , and the action expert generates an action chunk conditioned on them and the current proprioceptive state . In our implementation, the policy predicts 16 actions using 10 flow-matching integration steps and executes the first 10 actions before acquiring new observations and replanning. This receding-horizon procedure enables
ActMem to continually update its history-conditioned task intent and motor actions as the interaction unfolds.
Efficient Memory CoT Inference. Consecutive decisions differ only slightly: the history gains at most one new frame chunk, and the subtask typically persists over several decisions (Figure 2(c)). Standard inference nonetheless re-encodes the full history, decodes the subtask token by token, and runs a second backbone pass to obtain , so its cost grows with the history length. We remove this redundancy through exact reuse, producing the same result as standard inference up to numerical precision. Since the vision encoder attends only within each two-frame chunk and the history precedes the current view and instruction while only being appended, we cache visual features per chunk and reuse the key-value (KV) cache of the unchanged history prefix, so each decision encodes only the newly added chunk and prefills only the remaining tokens. For the subtask, we take the previous subtask as a draft, append it to the prompt, and obtain the predictions at all draft positions in a single parallel pass:
The draft is accepted up to its first mismatch,
If , the previous subtask is kept without any decoding step; otherwise, the KV cache is truncated after and greedy decoding resumes from . Following the verification rule of speculative decoding [20], this yields exactly the greedy-decoded subtask. Because the draft is reprocessed under the updated context, the hidden states at its positions are recomputed in this pass and directly form , which removes the second backbone pass and avoids stale latent conditions. As a result, each decision encodes a single new chunk, and the previous subtask is accepted without decoding in most decisions (see supplementary material).
Experiments
We evaluate ActMem on simulated memory benchmarks and real-world manipulation tasks to answer four questions: (1) How does ActMem compare with existing VLA policies on memory-dependent manipulation benchmarks? (2) Can it handle different forms of historical dependence that require knowing task progress beyond the current observation? (3) Where do its gains come from: the long interaction history, Latent Memory CoT, or Perception Attention? (4) Does conditioning on a long history remain practical at inference time? To answer these questions, we evaluate ActMem on RoboMemArena and MIKASA-Robo, run real-world experiments covering exogenous, endogenous, and traceless memory, and conduct controlled ablations and diagnostic analyses on RoboMemArena, together with a latency study under increasing history length.
Experimental Setup
Benchmarks. We evaluate ActMem on two memory-dependent simulation benchmarks and four real-world manipulation tasks. RoboMemArena [19] comprises 26 tasks in four categories (transferring, occlusion, counting, and sequence execution), in all of which the correct action depends on events that are no longer visible in the current observation. MIKASA-Robo [7] evaluates history-dependent manipulation on five tasks: ShellGame, Intercept, and RememberColor-3/5/9 (RC-3/5/9). Following each benchmark’s protocol, we run 51 trials per task on RoboMemArena and 100 trials per task on MIKASA-Robo, and evaluate a single trained model per benchmark. On RoboMemArena we report task success rate (TSR), the fraction of episodes in which every subtask stage is completed, and completion success rate (CSR), the average fraction of stages completed per episode; on MIKASA-Robo we report the average task success rate. The four real-world tasks span exogenous, endogenous, and traceless memory and are described in § 4.3.
Baselines. For simulation, we compare against representative vision-language-action models and memory-augmented policies, reporting numbers from the corresponding publications or our own reproductions. For the real-world experiments, we compare against MemoryVLA [27], MEM- [32], and SimpleMemVLA [37].
Implementation Details. ActMem is built upon the pre-trained starVLA [8] framework, comprising a Qwen3-VL backbone and a flow-matching action expert. We jointly fine-tune the vision-language backbone and action expert, conditioning action generation on history-dependent subtask representations and contextualized visual features through Perception Attention, which is incorporated at layers 3, 7, 11, and 15 of the 16-layer action expert. We follow the optimizer and learning-rate schedule of starVLA. Historical observations are sampled at 1 fps over 126 s for RoboMemArena and 20 fps over 3 s for MIKASA-Robo. Training is conducted on 8 NVIDIA RTX PRO 6000 GPUs with a global batch size of 64 for 60k iterations, and we evaluate the final checkpoint. During inference, the policy predicts 16-step action chunks using 10 flow-matching integration steps and executes the first 10 actions before replanning. Further model and training hyperparameters are provided in the supplementary material.
Memory-Dependent Simulation
RoboMemArena. As shown in Table 1, ActMem achieves 71.1% task success rate (TSR) and 80.8% completion success rate (CSR), outperforming SimpleMemVLA by 7.5 and 8.7 percentage points, respectively. Gains are largest in Occlusion and Sequence (+9.7 and +9.3 TSR over the best baseline), which depend most on earlier interactions. ActMem trails FrameSamp+Modul on Transferring (42.6% vs. 63.8%), which relies more on fine-grained manipulation; we attribute this to visual detail competing with history, a weakness also seen in SimpleMemVLA, where Perception Attention already lifts Transferring from 35.8% to 42.6%.
| Method | Transferring TSR | Transferring CSR | Occlusion TSR | Occlusion CSR | Counting TSR | Counting CSR | Sequence TSR | Sequence CSR | Average TSR | Average CSR |
| Reference | ||||||||||
| Qwen3-VL-8B (frozen) [19] | 15.0 | 34.6 | 0.0 | 6.8 | 9.3 | 44.6 | 7.5 | 39.2 | 6.0 | 26.2 |
| GPT-5.4 [19] | 13.8 | 32.9 | 1.8 | 9.2 | 12.9 | 50.7 | 15.0 | 47.3 | 8.7 | 30.5 |
| Reactive / short-horizon VLAs | ||||||||||
| [13] | 20.0 | 42.8 | 12.7 | 17.2 | 14.3 | 50.9 | 60.0 | 71.6 | 21.5 | 38.7 |
| HiF-VLA [23] | 17.5 | 38.9 | 12.7 | 27.1 | 8.6 | 45.9 | 42.5 | 70.2 | 16.9 | 39.8 |
| Memory-augmented VLAs | ||||||||||
| MemoryVLA [27] | 15.0 | 37.2 | 7.3 | 13.1 | 14.3 | 55.1 | 37.5 | 65.2 | 15.0 | 35.3 |
| MemER [30] | 20.0 | 36.1 | 16.4 | 33.2 | 27.1 | 65.1 | 65.0 | 79.1 | 27.3 | 49.1 |
| FrameSamp+Modul [9] | 63.8 | 72.1 | 39.1 | 56.5 | 31.4 | 57.8 | 73.8 | 86.9 | 46.2 | 63.9 |
| PrediMem (Qwen3-1.7B) [19] | 15.0 | 31.6 | 7.3 | 20.8 | 28.6 | 60.2 | 50.0 | 73.9 | 19.9 | 41.4 |
| PrediMem (Qwen3-4B) [19] | 20.0 | 42.1 | 18.2 | 34.7 | 38.6 | 64.9 | 65.0 | 84.6 | 31.9 | 51.7 |
| PrediMem [19] | 22.5 | 45.2 | 27.3 | 38.4 | 45.7 | 69.3 | 72.5 | 89.5 | 38.5 | 55.2 |
| SimpleMemVLA [37] | 35.8 | 37.4 | 64.3 | 76.8 | 71.4 | 79.3 | 75.5 | 81.1 | 63.6 | 72.1 |
| ActMem (Ours) | 42.6 | 46.3 | 74.0 | 88.3 | 75.1 | 82.2 | 84.8 | 92.4 | 71.1 | 80.8 |
Table 1. Comparison on RoboMemArena. TSR and CSR (%) over all 26 tasks under the official protocol (26 tasks × 51 trials). ActMem is ours; remaining baselines are taken from prior papers [19, 37]. Best in each column in bold.
MIKASA-Robo. ActMem achieves an average success rate of 73.8% on MIKASA-Robo (Table 2), surpassing SimpleMemVLA by 2.4 percentage points. Gains are largest on Intercept (85% vs. 76%) and RememberColor-9 (67% vs. 62%), where the action depends on a cue that is no longer visible, while ShellGame remains comparable to SimpleMemVLA (95% vs. 97%). The same pattern appears on RoboMemArena: the advantage is largest when the current observation conceals the information needed to act.
Table 2. MIKASA-Robo success rates (%, 100 trials). GMP is a per-task non-VLA memory policy. †our reproduction of the official SimpleMemVLA checkpoint; other VLA baselines from [7, 28, 37]. Best in each column in bold.
| Method | ShellGame | Intercept | RC-3 | RC-5 | RC-9 | Avg |
| Reference | ||||||
| GMP [11] | 98 | 83 | 80 | 61 | 17 | 67.8 |
| Vision-language-action models | ||||||
| CronusVLA [21] | 32 | 5 | 31 | 13 | 9 | 18.0 |
| SpatialVLA [25] | 23 | 27 | 27 | 17 | 11 | 21.0 |
| OpenVLA-OFT [17] | 47 | 14 | 59 | 16 | 6 | 28.4 |
| [5] | 33 | 42 | 35 | 22 | 15 | 29.4 |
| Octo [31] | 46 | 39 | 45 | 17 | 11 | 31.6 |
| MemoryVLA [27] | 88 | 24 | 44 | 30 | 20 | 41.2 |
| MemoryVLA++ [28] | 97 | 40 | 50 | 19 | 16 | 44.4 |
| SimpleMemVLA†[37] | 97 | 76 | 69 | 53 | 62 | 71.4 |
| ActMem (Ours) | 95 | 85 | 73 | 49 | 67 | 73.8 |
Table 2.
Real-Robot Manipulation
We evaluate ActMem on four real-world manipulation tasks spanning three forms of memory dependence: exogenous, endogenous, and traceless memory (Figure 3). Exogenous-memory tasks require recalling events introduced by external agents: in Cup-and-Block, the robot must identify the cup containing a previously hidden block, while in Scissor Retrieval, it must locate scissors that are no longer visible. Endogenous memory is evaluated through Cabinet Search, where the robot must track previously inspected drawers to avoid redundant searches. Finally, Tissue Counting tests traceless memory, requiring the robot to count completed pulls despite nearly identical observations after each action.
Figure 3. Visualization of real-world tasks and comparison of real-world performance. Left: (a) Cup-and-Block, (b) Tissue Counting, (c) Cabinet Search, and (d) Scissor Retrieval, spanning exogenous, endogenous, and traceless memory. Right: success rates of MemoryVLA, MEM-, SimpleMemVLA, and ActMem. Each task is evaluated over 40 trials, and success rates are computed accordingly.
Real-World Results. Each task is evaluated over 40 trials. As shown in Figure 3, ActMem achieves an average success rate of 68.8%, outperforming SimpleMemVLA (58.8%) by 10.0 percentage points, MemoryVLA (55.0%) by 13.8 points, and MEM- (48.8%) by 20.0 points, and is the best method on all four tasks. ActMem reaches 75% on both Cup-and-Block and Scissor Retrieval, where the required information was introduced by an external agent and is no longer visible, and 65% on Tissue Counting, where the observations after each pull are nearly identical. Performance is lowest on Cabinet Search (60%), which requires the policy to track the drawers it has already inspected. These results demonstrate that ActMem handles different forms of historical dependence, while tracking self-generated task progress remains the most challenging.
Ablation Studies
We conduct ablation studies on RoboMemArena to examine whether each design of ActMem is necessary: the long interaction history, the Latent Memory CoT that conveys it, and Perception Attention. All variants, including the full model, are trained for 40k iterations with the same data and schedule and evaluated on the 26 tasks with 10 trials per task, so absolute numbers are lower than in Table 1.
Interaction History. As shown in Table 3, removing historical observations reduces TSR from 52.3% to 17.7%, and
Table 3. Ablation studies of ActMem on RoboMemArena (40k-iteration schedule, 26 tasks 10 trials). We report task success rate (TSR) and completion success rate (CSR); higher values indicate better performance.
| Configuration | TSR (%) | CSR (%) |
| w/o History | 17.7 | 34.6 |
| 10 s History | 31.2 | 47.7 |
| Text-only Subtask | 40.4 | 56.9 |
| Full-token Conditioning | 38.8 | 54.6 |
| w/o Perception Attention | 47.7 | 64.2 |
| ActMem (Full) | 52.3 | 68.5 |
Table 3.
keeping only the most recent 10 s recovers just 31.2%. This indicates that the required information lies far back in the episode and that a short context cannot substitute for the full history.
Latent Memory CoT. Given the same history, how it reaches the action expert matters. Conditioning on the text embeddings of the generated subtask alone yields 40.4% TSR, and letting the action expert read all history tokens directly yields 38.8%, both below the 52.3% of Latent Memory CoT. Subtask text is therefore an incomplete memory signal: the action expert does not recover this progress from the raw history.
Perception Attention. Removing Perception Attention reduces TSR from 52.3% to 47.7% and CSR from 68.5% to 64.2%. This indicates that subtask-level guidance alone does not capture the visual information useful for continuous action generation. The two conditions are complementary: Memory CoT selects what to do from experience, and Perception Attention supplies the current geometry with which to do it.
Analysis
Historical Evidence. Figure 4 visualizes attention over historical frames in a representative RoboMemArena task. At the decision step, ActMem attends most to the frame containing the task-relevant event, although this event is no longer visible in the current observation.

Figure 4. Attention over historical frames. History frames of the RoboMemArena task Put cookies into the top drawer and put chocolate into another drawer, overlaid with ActMem’s attention (green) at 61 s (predicted subtask: “close the middle drawer”). Attention is the causal softmax from the generated subtask tokens to the history tokens, averaged over subtask tokens, heads, and backbone layers 3, 7, . . . , 31. The starred frame (33 s) contains the task-relevant event, no longer visible at the decision step.
What Latent Memory CoT Encodes. To test whether Memory CoT carries information beyond its text, we take the counting task Pour wine into the mug twice and extract representations at the end of the first and the second pour for all 100 demonstrations (Figure 5a). The two moments are nearly indistinguishable in current visual features encoded before the history, whose class means have a cosine similarity of 0.99, whereas Memory CoT places them far apart (0.22; Figure 5a). The gap persists (0.25) when the same text “pour the wine into the mug” is forced for every sample, so the count is carried by the hidden states, not only by the words of the subtask. In Figure 5a, visual features of both moments collapse into a single cluster, whereas Memory CoT splits them into two groups along the count direction. Thus, subtask inference does more than emit text: its hidden states keep a compact record of task progress that the action expert reads directly.

Figure 5. (a) What Latent Memory CoT encodes. Representations after the first (circles) and second (triangles) pour in Pour wine into the mug twice (100 demonstrations); numbers give the cosine similarity between the two class means. Same text: “pour the wine into the mug” is teacher-forced for all samples. Residual axes are magnified for visibility. (b) Class-mean distance between the two moments in each feature space, divided by the mean feature norm. (c) Inference latency per decision (median, single GPU) versus history length on RoboMemArena; shaded: interquartile range; dashed: -family models with fixed-size memory (see supplementary material).
Inference Efficiency
As shown in Figure 5c, the per-decision latency of standard inference grows from 0.87 s to 1.42 s as the history lengthens from under 30 to over 90 frames, close to that of SimpleMemVLA (0.70 s to 1.25 s). Efficient Memory CoT Inference (§ 3.4) reduces it to 0.33–0.52 s, 2.1× to 2.8× faster than standard inference and about 2.0× to 2.5× faster than SimpleMemVLA at every history length. The gain comes from exactly reusing the history cache and the previous subtask (§ 3.4), which leaves the output unchanged. Since the history is only appended, the cost of each decision depends mainly on the new chunk rather than on the full history, so efficient latency rises by only 0.19 s over the history range, versus 0.55 s for standard inference. -family policies with a fixed-size memory run at 0.08–0.09 s, so ActMem trades part of this speed for access to the full history.
Conclusion
We presented ActMem, an in-context memory framework that decouples memory reasoning from perceptual grounding within a single vision-language backbone. History-conditioned subtask inference leaves a Latent Memory CoT that encodes what to do, including task progress that the subtask text does not name, while Perception Attention grounds how to act in the current view. With this design, ActMem reaches 71.1% task success and 80.8% completion success on RoboMemArena, 73.8% on MIKASA-Robo, and 68.8% on real-world exogenous, endogenous, and traceless memory tasks, above the strongest memory-augmented baselines.
Limitations and Future Work. ActMem keeps every sampled frame in its context. Efficient Memory CoT Inference reuses the history KV cache, but each decision still attends over the full history, so latency grows with episode length, from 0.33 s to 0.52 s over the history lengths we test (§ 4.6). Tasks lasting tens of minutes would instead require selecting which history to keep and retrieving it in latent space, rather than keeping every past frame.
LLM Usage Statement
We used ChatGPT (OpenAI) to assist with language editing and the organization of the manuscript text, and OpenAI Codex as a coding assistant for parts of the implementation; all such code was reviewed and tested by the authors. They were not used to generate experimental results or citations. All technical content, experimental design, data collection, and analyses were carried out by the authors, who verified every claim in this paper and take full responsibility for its content.
Model and Training Details
Table 4 lists the model and training hyperparameters of ActMem. ActMem is built on the pretrained starVLA [8] framework. The vision-language backbone autoregressively generates the subtask for the current decision, and the action expert reads the last-layer hidden states and the token embeddings of the generated subtask tokens as the Latent Memory CoT. Perception Attention is a zero-initialized, gated cross-attention from the action expert to the current-observation tokens.
| Hyperparameter | Value |
| Vision-language backbone | Qwen3-VL-4B |
| Action expert | Flow-matching DiT, 16 layers |
| Perception Attention layers | 3, 7, 11, 15 |
| Condition for the action expert | Subtask-span hidden states + token embeddings |
| History (RoboMemArena) | 1 fps, 126 s |
| History (MIKASA-Robo) | 20 fps, 3 s |
| Action chunk / executed actions | 16 / 10 |
| Flow-matching integration steps | 10 |
| Optimizer | AdamW |
| Peak learning rate | (action expert), (VLM backbone) |
| Learning-rate schedule | Cosine decay to , 1000 warmup steps |
| Global batch size | 64 |
| Training iterations | 60k |
| Precision | bf16 |
| GPUs | 8 NVIDIA RTX PRO 6000 |
Table 4. Model and training hyperparameters of ActMem.
Data and Annotation
Subtask annotation. For RoboMemArena and MIKASA-Robo, we use the subtask annotations that come with the public LeRobot-format datasets and do not re-annotate. Each frame carries a subtask index into a per-benchmark list of short English phrases: 69 distinct subtasks over 2,600 demonstrations for RoboMemArena (e.g., “pick up the chocolate”, “pour the tomato sauce over the cookies first time”), and 90 over 2,500 demonstrations for MIKASA-Robo (e.g., “observe the cups”, “touch the cup hiding the ball”). For the real-robot tasks, subtask labels are first generated by Qwen3-VL and then spot-checked manually. For each demonstration, we sample frames at 1 fps (at most 12 per episode) with their timestamps and prompt Qwen3-VL with the task instruction and a closed vocabulary of phrases. The model segments the episode into consecutive stages, each named by exactly one phrase from the vocabulary; names outside the vocabulary are discarded. The vocabularies are: Cup-and-Block (170 demonstrations), “find the cup” and “pick up the cup”; Cabinet Search (190), “open the drawer”, “pick up the block”, and “close the drawer”, where “open the drawer” repeats once per searched compartment; Tissue Counting (100), “approach the tissue box” and “pull a tissue first/second/third time”; Scissor Retrieval (400), “approach the ottoman”, “approach the table”, “move the book aside”, and “pick up the scissors”.
Prompt template. The backbone receives one chat turn. For RoboMemArena, the user turn contains, in order, the text “Front camera:” followed by the timestamped history video (1 fps, up to 126 s), the text “Wrist camera:” followed by the current image, and the text prompt below.
The robot is a Franka Panda single-arm robot. The control frequency is 20 Hz. The next 16 control actions are predicted from the current sub-task. You are given the recent history video from one camera (each frame is tagged with its timestamp in seconds), plus the current image from one more camera. The overall task is: <instruction>. Identify the current sub-task.
The assistant turn is the subtask phrase itself (e.g., “pick up the chocolate”), so the subtask span is the generated answer, and its hidden states form the Latent Memory CoT. At inference, the same prompt is used and the subtask is decoded greedily. For MIKASA-Robo, the robot is described as “a Panda single-arm robot” and the history is sampled at 20 fps over 3 s.
Evaluation Details
RoboMemArena. We follow the official protocol with 26 tasks and 51 trials per task, using fixed environment seeds starting from 50. TSR is the fraction of episodes in which every subtask stage is completed, and CSR is the average fraction of stages completed per episode. We report the final checkpoint. All ActMem results in Table 1 of the main paper are obtained with Efficient Memory CoT Inference.
Ablation protocol. The ablation variants in Sec. 4.4 of the main paper share the data, recipe, and evaluation of the full model and differ only in one switch each: w/o History keeps only the current frame in place of the history video; 10 s History keeps the most recent 10 frames; Text-only Subtask replaces the hidden states of the subtask span with their token embeddings; Full-token Conditioning lets the action expert read all prompt tokens except the answer span; w/o Perception Attention removes the present-observation cross-attention. All variants are trained for 40k iterations with the learning-rate schedule scaled proportionally, and evaluated on 26 tasks with 10 trials per task.
Real-Robot Setup
Platform. All real-world experiments use an ARX LIFT2 dual-arm mobile manipulator (Figure 6a), which carries two 6-DoF ARX X5 arms with parallel grippers on a lifting column and an omnidirectional wheeled base. Three Intel RealSense D405 cameras provide the visual input: one head camera mounted on top of the column and one camera on each wrist, all streaming 640480 RGB at 60 fps. Demonstrations are collected by VR teleoperation (Figure 6b): the operator wears a VR headset and moves two handheld controllers, whose poses are converted to joint commands for the two arms by inverse kinematics and smoothing, and the gripper opening is controlled from the controllers. At test time, the robot streams its observations to a remote inference station with one NVIDIA RTX 5090 D GPU (24 GB) over a wired local network, and the policy returns action chunks of 50 steps that are executed at 30 Hz.


Figure 6. Real-robot platform. (a) The ARX LIFT2 dual-arm robot used for all real-world tasks. (b) The VR headset and controllers used to collect demonstrations.
Tasks. We evaluate four real-world tasks spanning exogenous, endogenous, and traceless memory. The language instructions are as follows:
Cup-and-Block: “Retrieve the block after the cups are washed.”
Cabinet Search: “Find the block; do not reopen drawers already inspected.”
Tissue Counting: “Pull tissues the required number of times.”
Scissor Retrieval: “Find the scissors after they have been covered.”
Evaluation. Each task is evaluated over 40 trials per method, and we report the success rate (1 for success, 0 for failure).
Additional Real-World Visualization
Figure 7–Figure 11 show successful real-robot rollouts of ActMem on the four real-world tasks, one episode per page (two for Tissue Counting), with frames sampled uniformly over each episode.

Figure 7. Successful real-robot rollout of ActMem on Cup-and-Block (“Retrieve the block after the cups are washed.”). Frames are sampled uniformly over the 30 s episode in raster order; the timestamp is shown at the top-left of each frame.

Figure 11. Successful real-robot rollout of ActMem on Scissor Retrieval (“Find the scissors after they have been covered.”). Frames are sampled uniformly over the 20 s episode in raster order.
Inference Latency
Measurement protocol. All latencies are measured on one NVIDIA RTX PRO 6000 GPU with bf16 and batch size 1, in the same session. For ActMem and SimpleMemVLA [37], we run a closed-loop RoboMemArena episode (one task, seed 50, 250 decisions, history clip of 2 to 126 frames at 1 fps), drop the first five decisions as warm-up, and report the median and 90th percentile of the model time per decision, grouped by the number of history frames. ActMem runs both inference modes back to back on the same observation with the same random state, and the rollout follows the actions of standard inference. Standard inference re-encodes the full history, whereas Efficient Memory CoT Inference reuses cached vision features and the longest matching key-value prefix and re-decodes the previous subtask only when it no longer matches. Times are wall-clock with CUDA synchronization and exclude building observation features. For SimpleMemVLA, the reported time covers the model call without the input processor, namely the image encoder, subtask generation, and the 10-step action head. The -family models (the base, MEM- with short and long memory, and our reimplementation of MemoryVLA on ) keep a fixed-size memory, so we report one value per model, measured offline in JAX on the same GPU over 50 calls with 10 flow-matching steps, three 224 224 camera inputs, and memory filled to capacity.
| History (frames) | 0–30 | 30–60 | 60–90 | 90–126 |
| Decisions | 51 | 60 | 60 | 74 |
| SimpleMemVLA | 0.70 / 1.05 | 0.85 / 1.29 | 1.02 / 1.67 | 1.25 / 1.96 |
| ActMem, standard | 0.87 / 1.45 | 0.98 / 1.68 | 1.09 / 1.94 | 1.42 / 2.27 |
| ActMem, Efficient Memory CoT | 0.33 / 0.52 | 0.37 / 0.56 | 0.52 / 0.75 | 0.50 / 0.74 |
Table 5. Median / 90th-percentile per-decision latency (s) by history length, measured on the same GPU in the same session.
Table 6. Median / 90th-percentile latency (ms) of -family models with a fixed-size memory, measured offline on the same GPU.
Where the time goes. With Efficient Memory CoT Inference, the key-value prefix is reused in 98.4% of decisions, exactly one new vision chunk is encoded per decision, and the previous subtask is accepted without decoding in 71.2% of decisions (94% up to 60 frames of history, 51% beyond). The action expert takes about 0.16 s regardless of history length.

Figure 8. Successful real-robot rollout of ActMem on Tissue Counting (“Pull tissues the required number of times.”), episode 1. Frames are sampled uniformly over the 110 s episode in raster order.

Figure 9. Successful real-robot rollout of ActMem on Tissue Counting, episode 2. Frames are sampled uniformly over the 77 s episode in raster order.

Figure 10. Successful real-robot rollout of ActMem on Cabinet Search (“Find the block; do not reopen drawers already inspected.”). Frames are sampled uniformly over the 50 s episode in raster order.
References
- [1]Shuai Bai, Yuxuan Cai, Rui-Zhe Chen, Ke-qin Chen, Xiong-Hui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Rongyao Fang, Chang Gao, et al. Qwen3-VL technical report. arXiv.org, 2025. 2arxiv.org/abs/2511.21631
- [2]Shuai Bai, Ke-qin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, K. Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-VL technical report. arXiv.org, 2025.arxiv.org/abs/2502.13923
- [3]Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07726, 2024. 2arxiv.org/abs/2407.07726
- [4]Johan Bjorck, Fernando Castaneda, Nikita Cherniadev, et al. GROOT N1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025. 1, 2
- [5]Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π₀: A Vision-Language-Action Flow Model for General Robot Control, 2024. 1, 2, 5, 7DOI
- [6]Tianxing Chen, Yuran Wang, Mingleyang Li, Yan Qin, Hao Shi, Zixuan Li, Yifan Hu, Yingsheng Zhang, Kaixuan Wang, Yue Chen, Hongcheng Wang, Renjing Xu, Ruihai Wu, Yao Mu, Yaodong Yang, Hao Dong, and Ping Luo. Rmbench: Memory-dependent robotic manipulation benchmark with insights into policy design, 2026. 2
- [7]E. Cherepanov, Nikita Kachlaev, A. Kovalev, and Aleksandr I. Panov. Memory, benchmark & robots: A benchmark for solving complex tasks with reinforcement learning. arXiv preprint arXiv:2502.10550, 2025. 1, 2, 6, 7arxiv.org/abs/2502.10550
- [8]StarVLA Community. Starvla: A lego-like codebase for vision-language-action model developing. arXiv preprint arXiv:2604.05014, 2026. 2, 6, 11
- [9]Yinpei Dai, Hongze Fu, Jayjun Lee, Yuejiang Liu, Haoran Zhang, Jianing Yang, Chelsea Finn, Nima Fazeli, and Joyce Chai. Robomme: Benchmarking and understanding memory for robotic generalist policies, 2026. 2, 6
- [10]Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox, Ranjay Krishna, and Jiafei Duan. Sam2act: Integrating visual foundation model with a memory architecture for robotic manipulation. arXiv preprint arXiv:2501.18564, 2025. 2
- [11]Yihuai Gao, Jeff Jinyun Liu, Shuang Li, and Shuran Song. Gated memory policy: In-context memorization and adaptation. arXiv preprint arXiv:2604.18933, 2026. 2, 7arxiv.org/abs/2604.18933
- [12]Melvyn A. Goodale and A. David Milner. Separate visual pathways for perception and action. Trends in Neurosciences, 15(1):20–25, 1992. 2
- [13]Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π₀.₅: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. 1, 3, 6, 12
- [14]Huiwong Jang, Sihyun Yu, Heeseung Kwon, Hojin Jeon, Younggyo Seo, and Jinwoo Shin. Contextvla: Vision-language-action model with amortized multi-frame context. arXiv preprint arXiv:2510.04246, 2025. 2
- [15]Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2):99–134, 1998. 1
- [16]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-action model, 2024. 1, 2arxiv.org/abs/2406.09246
- [17]Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success. RSS, 2025. 2, 7
- [18]Myungkyu Koo, Daewon Choi, Taeyoung Kim, Kyungmin Lee, Changyeon Kim, Younggyo Seo, and Jinwoo Shin. Hamlet: Switch your vision-language-action model into a history-aware policy. arXiv preprint arXiv:2510.00695, 2025. 2
- [19]Huashuo Lei, Wenxuan Song, Huarui Zhang, Jieyuan Pei, Jiayi Chen, Haodong Yan, Han Zhao, Pengxiang Ding, Zhipeng Zhang, Lida Huang, Donglin Wang, Yan Wang, and Haoang Li. Robomemarena: A comprehensive and challenging robotic memory benchmark, 2026. 2, 6
- [20]Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, 2023. 5arxiv.org/abs/2211.17192
- [21]Hao Li, Shuai Yang, Yilun Chen, Yang Tian, Xiaoda Yang, Xinyi Chen, Hanqing Wang, Tai Wang, Feng Zhao, Dahua Lin, et al. CronusVLA: Towards efficient and robust manipulation via multi-frame vision-language-action modeling. arXiv preprint arXiv:2506.19816, 2025. 2, 7
- [22]Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mingsheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024. 2
- [23]Minghui Lin, Pengxiang Ding, Shu Wang, Zifeng Zhuang, Yang Liu, Xinyang Tong, Wenxuan Song, Shangke Lyu, Siteng Huang, and Donglin Wang. Hif-vla: Hindsight, insight and foresight through motion representation for vision-language-action models. arXiv preprint arXiv:2512.09928, 2025. 2, 6arxiv.org/abs/2512.09928
- [24]Yize Liu, Huang Huang, Yining Hong, Zijian Du, Zhi Cao, Li Fei-Fei, and Jiajun Wu. T²mem: Learning test-time memory for robotics. arXiv preprint arXiv:2609.36720, 2026. 2arxiv.org/abs/2609.36720
- [25]Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, Jia Yuan Gu, Bin Zhao, Dong Wang, et al. SpatialVLA: Exploring spatial representations for visual-language-action models. In Robotics: Science and Systems, 2025. 2, 7DOI
- [26]Daniel L. Schacter, Donna Rose Addis, and Randy L. Buckner. Remembering the past to imagine the future: the prospective brain. Nature Reviews Neuroscience, 8(9):657–661, 2007. 2pubmed.ncbi.nlm.nih.gov/17700624
- [27]Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Memoryvla: Perceptual-cognitive memory in vision-language-action models for robotic manipulation. arXiv preprint arXiv:2508.19236, 2025. 1, 2, 6, 7, 12arxiv.org/abs/2508.19236
- [28]Hao Shi, Weiye Li, Bin Xie, Yulin Wang, Renping Zhou, Tiancai Wang, Xiangyu Zhang, Ping Luo, and Gao Huang. Memoryvla++: Temporal modeling via memory and imagination in vision-language-action models, 2026. 2, 7arxiv.org/abs/2606.09827
- [29]Lucy Xiaoyang Shi, Brian Ichter, Michael Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, et al. Hi Robot: Open-ended instruction following with hierarchical vision-language-action models. arXiv preprint arXiv:2502.19417, 2025. 1, 3arxiv.org/abs/2502.19417
- [30]Ajay Sridhar, Jennifer Pan, Satvik Sharma, and Chelsea Finn. Memer: Scaling up memory for robot control via experience retrieval. arXiv preprint arXiv:2510.20328, 2025. 1, 2, 6arxiv.org/abs/2510.20328
- [31]Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. In Robotics: Science and Systems, 2024. 2, 7
- [32]Marcel Torne, Karl Pertsch, Homer Walke, Kyle Vedder, Suraj Nair, Brian Ichter, Allen Z. Ren, Haohuan Wang, Jiaming Tang, Kyle Stachowicz, Karan Dhabalia, Michael Equi, Quan Vuong, Jost Tobias Springenberg, Sergey Levine, Chelsea Finn, and Danny Dress. MEM: Multi-scale embodied memory for vision language action models. arXiv preprint arXiv:2603.03596, 2026. 1, 2, 6, 12arxiv.org/abs/2603.03596
- [33]Kai Wang, Zhaopeng Qu, Yixiang Chen, Yuan Xu, Qisen Ma, Peng Su, Zhaowen Li, Yan Huang, and Liang Wang. Dim-wam: World-action modeling with diverse historical event memory, 2026. 2arxiv.org/abs/2606.27677
- [34]Ziye Wang, Zhiqi Shi, Chaojun Ni, Jiazhi Yang, Mengdi Li, Zhizhong Su, Tianwei Lin, and Hongyang Li. NativeMEM: Native memory compression for long-horizon robotic manipulation. arXiv preprint arXiv:2607.06678, 2026. 2arxiv.org/abs/2607.06678
- [35]Ganlin Yang, Zhangzheng Tu, Yuqiang Yang, Sitsonga Mao, Junyi Dong, Tianxing Chen, Jiaqi Peng, Jing Xiong, Jiafei Cao, Jifeng Dai, Wengang Zhou, Yao Mu, and Tai Wang. EventVla: Event-driven visual evidence memory for long-horizon vision-language-action policies, 2026. 2arxiv.org/abs/2606.20092
- [36]Sizhe Yang, Juncheng Mu, Tianming Wei, Chenhao Lu, Xiaofan Li, Linning Xu, Zhengrong Xue, Zhecheng Yuan, Dahua Lin, Jiangmiao Pang, and Huazhe Xu. Memoryvwam: Efficient world action modeling with persistent memory, 2026. 2arxiv.org/abs/2606.20562
- [37]Cheng Yin, Wang Xu, Junpeng Yang, Sikyuan Tam, Hanyu Liu, Yuan Yao, Xiangrui Zeng, Junbo Cui, Yequan Wang, Zhouping Yin, and Yankai Lin. SimpleMemVLA: A simple but effective native-video memory for vision-language-action models. arXiv preprint arXiv:2609.05533, 2026. 1, 2, 3, 6, 7, 12arxiv.org/abs/2609.05533
- [38]Yihan Zeng, Minghao Ye, Yiyuan Chen, Yide Shentu, Philipp Wu, Zike Yan, and Zhongyu Li. KEMO: Event-driven keyframe memory for long-horizon robot manipulation with VLA policies. arXiv preprint arXiv:2606.23589, 2026. 2arxiv.org/abs/2606.23589
- [39]Linqing Zhong, Yi Liu, Yifei Wei, Ziyu Xiong, Maoqing Yao, Si Liu, and Guanghui Ren. ACoT-VLA: Action chain-of-thought for vision-language-action models. arXiv preprint arXiv:2601.11404, 2026. 3arxiv.org/abs/2601.11404
- [40]Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. R2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023. 1, 2


