betaFeedback

WorksComputer Science

Efficient VLA Inference via Retrieval-Based Speculation and Hardware Acceleration

Abstract

Vision-language-action (VLA) models enable robots to understand and execute complex instructions, but their autoregressive (AR) action generation introduces significant latency on resource-constrained robotic platforms. Speculative decoding can accelerate AR-based inference by parallelizing action generation and verification; however, existing approaches rely on costly draft models, which are inefficient on edge platforms. In this paper, we propose Zora, a hardware-software co-design framework for accelerating AR-based VLA inference. Specifically, Zora introduces a lightweight retrieval-augmented generation (RAG)-based draft model that exploits robotic dynamics without additional neural draft models. We further reduce batch verification overhead through a kinematics-guided action-acceptance strategy and a KV-cache-sharing attention mechanism. To address the retrieval overhead introduced by RAG-based drafting, we co-design dedicated hardware support, Zora core, to address the memory irregularity in RAG-based drafting. We evaluate Zora on four VLA models with the LIBERO benchmark. Zora retains comparable accuracy while achieving up to 3.4×\times speedup with software-only optimization and up to 7.6×\times speedup with hardware support over GPUs.

Introduction

Embodied AI is becoming one of the foundations for next-generation intelligent systems [11], [57]. By integrating perception, language understanding, and action generation, embodied AI enables robots and autonomous machines to interact with the physical world. This capability has attracted growing commercial interest, with the global humanoid robot market estimated at USD 1.55 billion in 2024 and projected to reach USD 4.04 billion by 2030 [50]. As embodied AI moves from research prototypes to real-world deployment, vision-language-action (VLA) models have become the key to allowing robots to understand complex instructions.

Mainstream VLA models [24], [25], [32], [46], [65] commonly adopt an autoregressive (AR) inference process, similar to large language model (LLM) decoding. Instead of generating text tokens, AR-based VLA models generate action tokens sequentially and use each generated action to predict the next one. This sequential decoding process introduces high latency on resource-constrained robotic platforms. For example, existing AR-based algorithms barely achieve 10.0 FPS (Figure 3) on Nvidia Jetson Thor [44], far below the real-time requirement, 60 FPS, for robotics [4], [5], [13], [25], [31], [35]. This paper thus focuses on accelerating AR-based VLA inference.

Figure 3. The frame rate of various AR-based VLA models on Thor SoC [44]. The red line highlights the real-time requirement. All models are far from real-time execution.

Issues. The main bottleneck of AR-based VLA models is the decoding stage, which accounts for 69.5% of the execution time, on average (Figure 4). The fundamental issue in decoding is that AR-based action generation is inherently sequential. Since each token generation depends on previous tokens, the next token decoding cannot start until the current one finishes. To break this dependency, speculative decoding [8], [29], [40] provides an opportunity to expose token-level parallelism during action decoding. Specifically, a lightweight draft model first predicts a sequence of future actions, and the original VLA model then verifies these drafted actions in batch. Thus, we can reduce the number of sequential decoding steps.

Figure 4. Execution breakdown of seven representative AR-based VLA models [7, 12, 25, 46, 47, 62, 63]. The decoding stage dominates the execution time.

However, directly applying speculative decoding to robotic platforms does not necessarily guarantee great performance gains. Conventional speculative decoding relies on sufficient compute resources on server GPUs to predict multiple drafted tokens and verify them in parallel. In contrast, robotic platforms have limited resources. On such platforms, verifying too many drafted tokens in a batch may not translate into proportional parallel execution. Therefore, speculative decoding for VLA inference must explicitly account for both the cost of token drafting and the efficiency of batched verification, rather than relying solely on token-level parallelism.

Framework. To address the challenges, we propose ZORA, a unified acceleration framework for AR-based VLA on robotic platforms (Sec. IV). Overall, we propose three contributions to reduce the cost of drafting and verification.

First, to reduce drafting cost, we propose a lightweight drafting method that exploits the temporal regularity of robotic dynamics. Instead of using a smaller LLM, we design a lightweight retrieval-augmented generation (RAG)-based technique to generate future action candidates. Thus, our framework retrieves similar execution contexts from the database and uses their future actions as drafts, with low overhead.

Second, we design an action-acceptance strategy to bypass some action verifications. Instead of verifying every drafted action via the original VLA model, ZORA estimates the confidence of each drafted action based on the robotic dynamics and the consensus of multiple drafted tokens. The high-confidence drafted action can be directly accepted without invoking batched verification. Only low-confidence drafted actions are sent to the original VLA model for verification.

Third, we propose a KV-cache-sharing attention mechanism to further reduce the batch verification cost for the remaining actions that still require verification. Instead of treating each drafted action as an independent sequence, our mechanism enables multiple drafted action candidates to share the same visual and language tokens during verification. Thus, we reduce KV-cache loading overhead in attention computation.

Architectural Support. While our ZORA framework substantially reduces the model verification cost, the RAG-based draft model itself becomes a new bottleneck (Figure 10). Specifically, it relies on graph traversal, which introduces irregular computation. To address this bottleneck, we design a dedicated ZORA core in § 5 that supports efficient graph traversal by exploiting node-level and embedding-level parallelism in RAG-based data retrieval. Meanwhile, we introduce a dedicated checker unit to further support the kinematic and consistency checks. These architectural augmentations provide minimal but principled hardware support for the remaining bottlenecks in our framework after algorithmic acceleration.

Figure 10. The execution breakdown after applying our software optimizations. DG retrieval becomes one of the bottlenecks, taking 35% of execution time across four models.

Results. We implement the ZORA hardware prototype and evaluate it over four VLA models [12, 25, 46, 63]. On the LIBERO dataset [33], we show that ZORA achieves a comparable task success rate with no accuracy degradation. With our software-only optimization, ZORA achieves up to 3.4× speedup on the off-the-shelf GPUs. With our hardware support, ZORA achieves up to 7.6× and 4.8× speedup compared with mobile GPUs and dedicated accelerators, respectively. This paper makes the following contributions:

  • We quantitatively analyze the workload of AR-based VLA models and identify their common bottlenecks.

  • We leverage speculative decoding and robotic kinematics to reduce the workload and latency of VLA inference.

  • We propose a minimal yet principled hardware augmentation to address the irregularity of retrieval-based drafting.

  • Our co-design achieves up to 7.6× speedup over prior methods with no accuracy degradation.

Background

This section gives the background of AR-based VLA models (Sec. II-A) and retrieval-augmented generation (Sec. II-B).

Autoregressive-Based VLA Models

VLA Model. The goal of a VLA model is to translate multimodal observations into executable actions for embodied agents. A typical VLA model takes three pieces of information as input: the current visual observation oto_t, a language instruction I, and the current robot state sts_t at a given timestep tt. Based on these inputs, the VLA model predicts the next action at+1a_{t+1} or a future action sequence, {at+1,…,at+w}\{a_{t+1}, \ldots, a_{t+w}\}, where ww is the length of the predicted action sequence.

Workflow. Mainstream VLA models [12, 25, 32, 46, 63, 65] often follow an autoregressive generation paradigm, similar to LLMs. As shown in Figure 1, at each timestep tt, the VLA model observes the current environment oto_t and generates the next action at+1a_{t+1}. The low-level control logic in the robot then consumes this action and moves to a new state st+1s_{t+1}. The VLA model repeats this process until this task is completed.

Figure 1. Inference process of single-action autoregressive VLA models. With the visual observation oto_t and language instruction I, the model first performs a prefill step to encode the input context and generate the KV cache. During decoding, action tokens are generated autoregressively and appended to the KV cache. Once all action tokens are generated, including relative movement Δxt+1\Delta x_{t+1}, rotation change Δrt+1\Delta r_{t+1}, and gripper state gt+1g_{t+1}, the VLA model then uses a small action detokenizer to decode the action and control the robot. Lastly, the execution updates the next observation for subsequent action generation.

During inference, a VLA model does not predict one action at+1a_{t+1} in one decoding step. Instead, each action is represented by multiple tokens and generated autoregressively. E.g., a robot-arm action can be represented by seven tokens in Figure 1. The first three tokens encode the displacement difference Δxt+1\Delta x_{t+1}, the next three tokens encode the rotation difference Δrt+1\Delta r_{t+1}, and the last token encodes the gripper state gt+1g_{t+1}. Different VLAs can have different encoding schemes [67].

At each control step, the VLA model first takes the instruction I and the current sensory observation oto_t to perform the prefill stage, producing hidden states and the KV cache. Then, during the decoding stage, the model generates one token at a time, e.g., at+1,0a_{t+1,0}. Each generated token is appended to the input token sequence and used to predict the next token, e.g., at+1,1a_{t+1,1}. This process repeats until all tokens of the current action are generated. The generated action tokens are then passed to an action detokenizer, which converts the tokens into robot control commands. Finally, these commands are fed to low-level control algorithms [52, 56, 64] to drive the robot.

Retrieval-Augmented Generation (RAG)

RAG has been widely used to enhance generative models in many domains [22, 34, 38, 48, 55], such as LLMs. Its core idea is to augment model generation with relevant information retrieved from an external database, rather than relying solely on knowledge encoded in model parameters. The basic workflow is as follows.

Overview. A RAG system consists of two stages: an offline indexing stage and an online retrieval stage. During offline indexing, the system builds the external database with domain knowledge. In the case of VLA, such knowledge can include robot states or trajectory data. Each entry in this database is a key-value pair for retrieval. The retrieval key is an embedding generated from the input query, while the retrieval value could be its associated data. During online inference, the input query is first encoded into a query embedding. The RAG system then searches for embeddings that are similar to the query embedding and retrieves the corresponding values.

In the VLA setting, the retrieval key can be a combined embedding of the visual observation oto_t, a language instruction II, and the current robot state sts_t. The retrieval value can be the future action trajectory, {at+1,…,at+w}\{a_{t+1}, \ldots, a_{t+w}\}, observed under a similar execution context in the RAG database. Mathematically, each database entry can be represented as,

Key=FRAG(ot,I,st) and Value={at+1,…,at+w},\mathrm{Key} = \mathcal{F}_{\mathrm{RAG}}(o_t, I, s_t) \text{ and Value} = \{a_{t+1}, \ldots, a_{t+w}\},

where FRAG\mathcal{F}_{\mathrm{RAG}} is the embedding function and ww is the prediction window. During inference, we search for contexts and their future action trajectories similar to the current robot context.

Hierarchical Navigable Small World. RAG systems often rely on approximate nearest-neighbor search to accelerate data retrieval. One widely used algorithm in RAG systems is hierarchical navigable small world (HNSW) [39], which organizes embeddings into a multi-layer proximity graph.

HNSW algorithm first builds a hierarchical graph with multiple layers offline. In Figure 2, we use three layers as an example. In this graph, the upper layers contain fewer nodes and provide long-range shortcuts across the embedding space, while lower layers contain more nodes and provide fine-grained nearest-neighbor search. Each node represents an embedding and connects to a small set of nearby embeddings. This hierarchical graph allows the search process to quickly navigate toward the region of the embedding space that is close to the query. Since the hierarchical graph is built offline, this paper only focuses on the online search of HNSW.

Figure 2. The HNSW-based retrieval process. The input is first encoded into a query embedding, which is used to search the vector database before selecting the top-KK retrieved results for downstream VLA models. The search database is implemented with a HNSW graph, where search proceeds in a coarse-to-fine manner, from the top layer to the bottom layer.

Figure 2 shows an example of the retrieval process in HNSW. Here, the pink point q0q_0 denotes the query embedding. The search starts from any node at the top layer (Layer 2). At each layer, it greedily moves from the current node to a neighboring node if that neighbor is closer to the query. This greedy search continues until no neighbor in the current layer is closer than the current node. For instance, node n1n_1 is the nearest node to q0q_0. This node is then the entry point for the next lower layer.

The same greedy procedure is repeated layer by layer. In Figure 2, the blue nodes denote the entry points to lower layers, and the green arrows show the search path. After reaching the bottom layer, i.e., Layer 0, HNSW performs the most fine-grained search over a denser set of nodes. The search continues expanding nearby candidates until it finds the closest embeddings to the query. In this case, we need to find the top-3 nearest neighbors, shown as the green nodes, i.e., n3n_3, n4n_4, and n8n_8. These retrieved nearest neighbor embeddings are then passed to the downstream VLA model.

Motivation

To understand the underlying performance bottlenecks of VLA models, we characterize various AR-based VLA models on a Nvidia Thor SoC [44] and show three key insights that motivate the design of our acceleration framework.

Insight 1: Existing AR-based VLA models are far from real-time execution on existing mobile platforms. [24, 72]

Real-time VLA requires generating actions at over 60 FPS [13, 25, 31, 35] so that the robot can respond smoothly. However, existing AR-based VLA models introduce long inference latency on mobile platforms. Here, we measure seven models [7, 12, 25, 46, 47, 62, 63] on an Nvidia Thor SoC [44]. In Figure 3, most models can barely achieve 10 FPS, similar to numbers reported in prior work [24, 72]. Thus, directly deploying VLA models on existing mobile platforms is insufficient for real-time execution.

Insight 2: Decoding is time-consuming in VLA inference, leading to severe GPU underutilization.

In Sec. II-A, AR-based VLA models generate action tokens sequentially, limiting token-level parallelism. Similar to LLMs, decoding is the key bottleneck in VLA inference, taking 69.5% of the execution time on the LIBERO dataset [33] (Figure 4). Meanwhile, on an Nvidia Thor SoC [44], single-token decoding primarily involves matrix-vector operations, which underutilize tensor cores optimized for matrix-matrix multiplication. Decoding exhibits lower tensor-core utilization than prefill (Figure 5). Thus, increasing decoding parallelism is key to improving GPU utilization and inference performance.

Figure 5. The tensor core utilization of the prefill and the decode stage. Tensor cores are severely underutilized during the decoding stage.

Insight 3: Robotic execution has strong temporal correlation that can be exploited to reduce inference computation.

Meanwhile, robot actions are not independent across time. In high-frequency control, consecutive actions often change smoothly due to the continuous robot dynamics. To quantify this temporal correlation, we analyze the traces from four LIBERO suites [33], covering over 2000 trajectories in Figure 6. For each sample, we linearly extrapolate the next action based on the previous two actions, and calculate the error between the extrapolated action and the actual action. We report the cumulative distribution function (CDF) of the translation error of our predictions. Overall, roughly 90% of the next-action predictions have an error within 5.0 mm, which is within the operation error threshold [1, 58]. This result shows a strong correlation in recent action history. Potentially, we can exploit this temporal correlation to reduce VLA computation.

Figure 6. Action temporal correlation across four LIBERO suites [33]. It shows strong short-term temporal correlation in robotic execution.

Zora Framework

To reduce the compute overhead of decoding in VLA models, we propose our framework, ZORA, that exploits speculative decoding [8, 29, 40] to verify multiple speculative action tokens in parallel and accelerate the end-to-end inference time. We first present the overview of ZORA in Sec. IV-A. We then describe each key component of the framework in the following subsections.

Overview

Intuition. The goal of our framework, ZORA, is to reduce the compute overhead of decoding in AR-based VLA inference. To do that, we exploit speculative decoding. Instead of invoking the original VLA model to decode every action token sequentially (Figure 1), ZORA proposes a lightweight prediction mechanism to draft a short sequence of future actions by exploiting robotic dynamics. These drafted action tokens are then verified by the original VLA model in parallel to reduce the overall latency. Next, we describe the workflow of ZORA.

Workflow. Figure 7 gives an overview of our proposed framework, ZORA. At each control step, ZORA takes three inputs: the visual observation oto_t, the robot state sts_t, and the language instruction II. These inputs are first processed by their corresponding encoders to obtain input embedding tokens.

Figure 7. ZORA overview. Given the current robot state sts_t, visual observation oto_t, and language instruction I, ZORA first encodes them into tokens in the latent space. The dynamics-guided retrieval module retrieves top-kk candidate action sequences with a length of ww from past execution contexts. An action checker then sequentially checks the retrieved action tokens and classifies them into high-confidence and low-confidence tokens. High-confidence tokens are directly committed for robotic execution, while low-confidence tokens are verified by the original VLA model in batches. Verified tokens are also committed for robot execution, whereas failed tokens trigger fallback inference to re-execute the original VLA model. After each action is executed, current actions are updated, and the KV cache is recomputed. The top-kk candidate sequences are refreshed whenever fallback is triggered, or candidate sequences are all verified.

Initially, before executing the VLA model, ZORA invokes a lightweight drafting mechanism, called dynamics-guided (DG) retrieval (Sec. IV-B), to predict future action tokens. DG retrieval takes the robot state and visual observation as the query context and queries a retrieval database. It then retrieves the top-kk action sequences with ww actions whose execution contexts are most similar to the current context. The retrieved sequences are treated as action candidates and sent to an action checker. Meanwhile, the model performs prefill in parallel to generate a KV cache, which is later used by a batch verifier.

Next, an action checker in Sec. IV-C leverages the robotic dynamics to determine which candidate actions need VLA model verification action-by-action. Specifically, the action checker classifies candidate tokens within one action into high-confidence and low-confidence tokens. High-confidence tokens are directly committed and executed by robots without VLA verification. Low-confidence tokens are sent to the batch verifier, which checks whether they match the actions predicted by the original VLA model. To further reduce verification overhead, we co-design an efficient batch verification in Sec. IV-D.

If a low-confidence candidate token passes verification, it is directly used to control the robot. Each time an action is executed, the current action is updated, and the KV cache is recomputed. If the candidate token fails verification, the system then falls back to normal VLA inference. When fallback inference is triggered, the entire action candidate sequences are also re-generated to refresh the current execution state.

Dynamics-Guided Retrieval

Intuition. The goal of DG retrieval is to generate accurate draft actions with low overhead for batched verification. Figure 6 shows that short-term robot actions often exhibit strong temporal regularity. Therefore, given the current visual observation and robot state, the near-future motions can be predicted from past execution contexts. This naturally motivates us to leverage speculative decoding to accelerate VLA inference.

Why Retrieval-Based Drafting? Conventional speculative decoding techniques often rely on a small LLM draft model to generate speculative tokens. However, this design has two limitations for robotic platforms. First, this small draft model typically needs to be aligned with the original large VLA model to achieve a high acceptance rate. Such alignment often requires model distillation [37, 73] with additional training, which is time-consuming. Second, even a small draft model can introduce non-trivial computation overhead for resource-constrained platforms. To avoid these costs, ZORA proposes a training-free retrieval-based drafting technique.

Process. Figure 8 shows the overall process of our DG retrieval. Here, we use the visual observation oto_t and the current robot state sts_t to construct the retrieval query. The visual observation includes an egocentric image captured by the robot-mounted camera and a third-person image captured by an external camera. The robot state captures a short history of the robot execution state, including its recent pose trajectory, current motion status, and gripper state. Together, these two inputs capture sufficient information for the current execution context. Note that, we do not include the language instruction II in the input query. Because different instructions sometimes lead to similar short-term trajectories. Excluding the instruction allows DG retrieval to focus on robot dynamics and improves the generality of our technique.

Figure 8. The process of DG retrieval. DG retrieval first constructs a query embedding from the current visual observations and robot state. DG retrieval then performs HNSW-based search over the retrieval database to find the top-kk keys that are most similar to the query. The associated values are returned as candidate action sequences and passed to the consensus checker.

Given oto_t and sts_t, DG retrieval first generates a query embedding, which concatenates the embeddings of oto_t and sts_t,

qt=Concat⁡(Fv(ot),Fs(st)),q_t = \operatorname{Concat}(\mathcal{F}_v(o_t), \mathcal{F}_s(s_t)),

where Fv\mathcal{F}_v and Fs\mathcal{F}_s denote the query encoder for visual observation and robot state, respectively. The visual encoder is DINOv2 [45], the state encoder is normalization. Given qtq_t, we search through the retrieval database with similar keys. In a retrieval database, each entry, ii, stores a key-value pair. The key is an embedding of a previous execution context, and the value is the future action trajectory that followed that context,

ki=Concat⁡(Fv(oi),Fs(si)),vi={ai+1,ai+2,…,ai+w},k_i = \operatorname{Concat}(\mathcal{F}_v(o_i), \mathcal{F}_s(s_i)), v_i = \{a_{i+1}, a_{i+2}, \ldots, a_{i+w}\},

where ww is the length of the predicted action window.

To efficiently search the retrieval database, DG retrieval uses an HNSW-based approximate nearest-neighbor search. Given the query embedding qtq_t, the HNSW search traverses through an HNSW graph as shown in Figure 8. In this example, after three layers’ traversal, the HNSW search returns {vk}\{v_k\}, a set of the top-kk database entries whose keys are closest to qtq_t,

{vk}=TopK⁡(sim⁡(qt,ki)).(1)\{v_k\} = \operatorname{TopK}(\operatorname{sim}(q_t, k_i)). \tag*{(1)}

In Figure 8, we select the top-3 nodes highlighted in green. In actual implementation, the default value of kk is 6. ‘sim’ denotes the similarity metric used by the retrieval index. Here, we use cosine similarity. For each selected entry, DG retrieval returns the associated future action sequence. The output of DG retrieval is a set of top-kk candidate trajectories.

We design DG retrieval to return a set of top-kk short trajectories for two reasons. First, retrieving multiple trajectory candidates does not necessarily increase verification overhead; instead, it creates a verification batch that can better utilize tensor cores by verifying multiple candidates in parallel. Second, multiple trajectory candidates provide additional information to estimate draft confidence. Thus, action checker can directly commit reliable tokens and reduce the compute overhead.

Action Checker

Given kk sequences of drafted action candidates, the action checker identifies high-confidence actions that can be directly sent to robot execution. Specifically, the action checker performs two complementary checks: a kinematic check and a consistency check. The kinematic check evaluates whether a predicted action follows a smooth trajectory implied by the robot’s recent motion history. The consistency check evaluates whether the drafted action is consistently predicted across the top-kk drafted action candidates. Actions that pass either check are regarded as high-confidence predictions and are accepted without invoking the subsequent model verification. We next describe these two checks in the following paragraphs.

Kinematic Check. Sec. III shows that 90% of the actions can be extrapolated linearly. The kinematic check leverages this property to avoid compute-intensive model verification when a drafted action follows the robot’s recent motion trend.

Specifically, we propose a continuity score, Sc\mathcal{S}_c, to measure how well an action aia_i follows the extrapolated trajectory,

Sc(ai)=min⁡(1αFΔ(ai,ai′),1),(2)\mathcal{S}_c(a_i) = \min\left(\frac{1}{\alpha}\mathcal{F}_{\Delta}(a_i,a_i^\prime),1\right), \tag*{(2)}

where ai′a_i^\prime denotes the action extrapolated for timestep ii from the previous two actions, FΔ\mathcal{F}_{\Delta} measures the discrepancy between ai\mathbf{a}_i and ai′\mathbf{a}'_i, and α\alpha is a normalization constant with a default value of 0.5. The function FΔF_{\Delta} is defined as,

FΔ(ai,ai′)=∥Δxi−Δxi′∥2+∥Δri−Δri′∥2+ϕ(gi,gi′),F_{\Delta}(\mathbf{a}_i,\mathbf{a}'_i)=\lVert\Delta x_i-\Delta x'_i\rVert_2+\lVert\Delta r_i-\Delta r'_i\rVert_2+\phi(g_i,g'_i),
and ϕ(gi,gi′)={1,if gi≠gi′,0,otherwise,(3)\text{and }\phi(g_i,g'_i)= \begin{cases} 1, & \text{if }g_i\ne g'_i,\\ 0, & \text{otherwise,} \end{cases} \tag*{(3)}

where Δxi\Delta x_i, Δri\Delta r_i, and gig_i denote the normalized displacement, rotation difference, and gripper state of action ai\mathbf{a}_i, respectively. Thus, FΔF_{\Delta} captures the combined discrepancy across the translational, rotational, and gripper components of the predicted and extrapolated actions. The extrapolated action ai′\mathbf{a}'_i is computed from the actions at previous two timesteps as,

ai′={(2Δxi−1−Δxi−2),(2Δri−1−Δri−2),gi−1}.(4)\mathbf{a}'_i=\{(2\Delta x_{i-1}-\Delta x_{i-2}),(2\Delta r_{i-1}-\Delta r_{i-2}),g_{i-1}\}. \tag*{(4)}

To evaluate the continuity of a candidate sequence, La={at+1,…,at+w}\mathbf{L}_a=\{\mathbf{a}_{t+1},\ldots,\mathbf{a}_{t+w}\}, obtained from DG retrieval, we compare the continuity score of each action against a kinematic threshold θkin\theta_{\mathrm{kin}}. If Sc(ai)S_c(\mathbf{a}_i) is lower than θkin\theta_{\mathrm{kin}}, the action is considered kinematically consistent and directly becomes the committed action. Otherwise, this action in the current step undergoes the consistency check and, if that fails, batch verification.

Here, θkin\theta_{\mathrm{kin}} is not a fixed value. Instead, θkin\theta_{\mathrm{kin}} is dynamically adjusted according to the smoothness of the recent action history Lhis={at−h+1,…,at}\mathbf{L}_{\mathrm{his}}=\{\mathbf{a}_{t-h+1},\ldots,\mathbf{a}_t\}. hh is the length of the recent history, with a default value of 16. We first measure the continuity of recent actions as,

Shis=1h∑a∈LhisSc(a).(5)S_{\mathrm{his}}=\frac{1}{h}\sum_{\mathbf{a}\in\mathbf{L}_{\mathrm{his}}}S_c(\mathbf{a}). \tag*{(5)}

The kinematic threshold, θkin\theta_{\mathrm{kin}}, is then computed as

θkin=min⁡(max⁡(β−Shis,θkin,min),θkin,max),(6)\theta_{\mathrm{kin}}=\min\left(\max\left(\beta-S_{\mathrm{his}},\theta_{\mathrm{kin,min}}\right),\theta_{\mathrm{kin,max}}\right), \tag*{(6)}

where β\beta is a hyperparameter, 0.5. θkin,min\theta_{\mathrm{kin,min}} and θkin,max\theta_{\mathrm{kin,max}} denote the lower and upper bounds of θkin\theta_{\mathrm{kin}}, respectively. They are 0.20 and 0.45, respectively. When ShisS_{\mathrm{his}} is small, the recent action history is smoother, so we increase θkin\theta_{\mathrm{kin}} to allow more drafted actions to pass the kinematic check. Conversely, when ShisS_{\mathrm{his}} is large, the recent motion is less smooth; thus, we decrease θkin\theta_{\mathrm{kin}} and become more conservative. When multiple candidate action tokens in the current step satisfy the threshold θkin\theta_{\mathrm{kin}}, we choose the action with the lowest overall continuity score.

Consistency Check. The kinematic check allows some drafted actions to skip batch verification by exploiting the smoothness of robot dynamics. However, some actions may not satisfy the kinematic smoothness criterion, yet can still be reliable if they are consistently predicted across multiple candidate sequences retrieved by DG retrieval. To cover those actions, we propose a consistency check that further identifies drafted actions that can be accepted without batch verification. Given kk candidate action sequences, we first compute the weighted mean action aˉi\bar{\mathbf{a}}_i at each timestep ii:

aˉi=∑j=1kwjaij∑j=1kwj,wj=exp⁡(sj)(7)\bar{\mathbf{a}}_i=\frac{\sum_{j=1}^{k}w^j\mathbf{a}_i^j}{\sum_{j=1}^{k}w^j},\qquad w^j=\exp(s^j) \tag*{(7)}

where wjw^j denotes the confidence of the jj-th candidate sequence obtained from DG retrieval, sjs^j is the cosine similarity score between the top-kk candidate embedding and the query embedding. aij\mathbf{a}_i^j denotes the ii-th action in that sequence. We then select the action closest to aˉi\bar{\mathbf{a}}_i among the top-kk candidates as the consensus action:

ai,mid=argmin⁡aij,j∈{1,…,k} FΔ(aij,aˉi).(8)\mathbf{a}_{i,\mathrm{mid}}=\underset{\mathbf{a}_i^j,j\in\{1,\ldots,k\}}{\operatorname{argmin}}\ F_{\Delta}(\mathbf{a}_i^j,\bar{\mathbf{a}}_i). \tag*{(8)}

Next, we measure how consistently the top-kk candidate sequences support the action ai,mid\mathbf{a}_{i,\mathrm{mid}} using a consistency score:

Si,con=∑j=1kwjFΔ(ai,mid,aij)∑j=1kwj.(9)S_{i,\mathrm{con}}=\frac{\sum_{j=1}^{k}w^jF_{\Delta}(\mathbf{a}_{i,\mathrm{mid}},\mathbf{a}_i^j)}{\sum_{j=1}^{k}w^j}. \tag*{(9)}

A lower Si,conS_{i,\mathrm{con}} means stronger consensus among the candidate actions at timestep ii. If Si,conS_{i,\mathrm{con}} is below a predefined consistency threshold θcon=0.2\theta_{\mathrm{con}}=0.2, this consensus action is high confidence, and it can be directly committed to the committed action queue. Otherwise, the action is considered uncertain, and the kk actions in the current step are sent for batch verification.

Batch Verifier

While the action checker allows some drafted actions to skip model verification, the remaining uncommitted actions still need to be verified by the original VLA model. A naive approach is to verify these actions one by one, but this introduces redundant data movement, e.g., repeatedly loading KV cache. To reduce this overhead, we propose a mask-based attention mechanism that shares the KV cache across different action candidates during verification.

Attention. Figure 9 shows our masked attention mechanism used for batched verification. Instead of verifying each candidate sequence independently, we pack the top-kk actions in the current step into a single attention computation. These candidates share the same prefix context; thus, we only load their KV cache once and share it across all candidates.

Figure 9. Masked attention for batched verification. Multiple top-kk action candidates are packed into a single attention computation and share the same prefix KV cache.

To preserve the independence of different candidate branches, we apply an attention mask: each action token can attend to the shared KV cache and the preceding tokens within the same candidate sequence, but cannot attend to tokens from other candidates, as shown in Figure 9. As a result, the verifier computes only the necessary branch-specific attention while skipping invalid cross-candidate attention. This way, we avoid repeated KV-cache loading during batch verification.

Our mechanism mainly targets the attention computation and KV-cache reuse. It does not optimize the linear operators, which have been extensively studied in prior acceleration work [17, 26, 27]. Therefore, our approach is orthogonal to existing optimizations for linear projections.

Verification. Specifically, for each verified timestep, we compare each verified candidate action with its corresponding drafted action and compute their discrepancy score using FΔF_{\Delta}. The candidate with the lowest discrepancy score is selected. We then compare this score against a predefined acceptance threshold θver\theta_{\mathrm{ver}}, which is 0.20. If the score is below θver\theta_{\mathrm{ver}}, the selected action is accepted and committed to the action queue.

Otherwise, the system falls back to the normal VLA inference utilizing the current observation, and the prefilled prefix KV cache generated by the verifier may be reused.

Novelty of Zora

All prior works [61, 70, 71] still follow the same speculative decoding execution, i.e., model drafting followed by batch verification. Their key contributions lie in techniques proposed to improve draft acceptance rates. For instance, HeiSD [71] leverages both RAG retrieval and draft-based models to generate draft tokens. KERV [70] uses robotic kinematics for acceptance threshold adjustment. SpecVLA [61] directly adopts the off-the-shelf speculative decoding. These methods have to rely on target-model predictions, and they are often model-specific designs with explicit finetuning.

ZORA instead proposes a unified and portable approach that uses execution experience for both draft prediction and confidence estimation. ① ZORA expands reusable action knowledge across multiple VLA models and proposes a RAG-based draft predictor to retrieve multiple short-term trajectories. ② A lightweight checker jointly exploits recent-motion continuity and retrieved-trajectory agreement to directly commit high-confidence actions and selectively verify uncertain candidates. ③ Masked attention shares the KV context across candidate sequences and verifies them in parallel, while preserving independence between candidates.

Architectural Support

We first motivate the need for architectural support in § 5-A. We then present an overview of the proposed architecture in § 5-B. Lastly, we explain two key components: the ZORA core in § 5-C and the checker unit in § 5-D.

Issues

Figure 10 shows the execution breakdown after applying these optimizations. Although the average decoding latency is reduced by 64.9% (see Sec. VII-B), DG retrieval becomes one of the bottlenecks and contributes non-trivial latency to the decoding stage. Moreover, HNSW search involves irregular graph traversal, leading to irregular memory accesses that are inefficient on mobile GPUs. Therefore, we propose a principled yet minimal set of hardware augmentations on existing DNN accelerators to support HNSW-based DG retrieval.

Overview

To address the issue described in § 5-A, we propose our ZORA architecture. We show its overview in Figure 11. The ZORA accelerator augments a DNN accelerator with two major components: a ZORA core and a checker unit.

Figure 11. Overview of the ZORA accelerator architecture. The accelerator contains a ZORA core to accelerate DG retrieval, a checker unit, and a DNN accelerator. Yellow blocks highlight our key contributions.

During execution, the ZORA core first accelerates DG retrieval to predict future pose trajectories, while the checker unit performs the operations required by the action checker. A microcontroller then orchestrates the overall execution flow and performs lightweight control operations. Lastly, a DNN accelerator accelerates batch verification of the retrieved pose trajectories by exploiting their value similarity. The DNN accelerator can be implemented as a systolic array or another design; ZORA is not tied to a specific DNN accelerator architecture. Sec. VII-C shows that our design can be integrated with different DNN accelerators. Meanwhile, a vector unit is also included to support vector operations.

DRAM stores the input and output data for the ZORA core and the DNN accelerator, e.g., the graph index for HNSW traversal, node embeddings, model weights, and intermediate activations. The accelerator also includes a double-buffered on-chip buffer to overlap computation with data fetching.

Zora Core

Design Principle. The goal of the ZORA core is to accelerate HNSW-based DG retrieval. Although HNSW search is inherently sequential due to the graph traversal nature, our architecture exploits two additional sources of parallelism: node-level parallelism and embedding-level parallelism. First, for each node, HNSW needs to evaluate multiple neighboring nodes and identify the neighbor closest to the query. These neighbor evaluations are independent and can be processed in parallel. We call it node-level parallelism (NLP). Second, computing the similarity between the query embedding and a node embedding requires computing across all embedding dimensions. These dimensions can be partitioned into multiple groups and processed in parallel. We call it embedding-level parallelism (ELP). The overall design of the ZORA core is built upon these two forms of parallelism.

Overview. Figure 12 gives an overview of the ZORA core, which still performs HNSW search layer-by-layer. The overall execution consists of four pipelined stages: fetch, compute, accumulation, and update. The fetch stage loads the embeddings of candidate neighbor nodes. The compute stage then calculates the partial similarity between the query embedding and each candidate node embedding. The accumulation stage combines partial similarity scores into full similarity scores. Lastly, the update stage selects the nearest node or top-k nodes and updates the search frontier for the next traversal loop. We further explain these four stages in the following paragraphs.

Figure 12. Overview of the ZORA core architecture. The core accelerates HNSW-based DG retrieval through a four-stage pipeline: fetch, compute, accumulation, and update. The PE array exploits node-level parallelism and embedding-level parallelism to compute partial similarity scores between the query embedding and neighbor embeddings. Partial similarity scores are accumulated and then consumed by either the reduction unit for greedy search in upper HNSW layers or the comparison units for top-k selection in the bottom layer. Lastly, the neighbor index fetch unit updates the search frontier for unvisited nodes.

Fetch. The fetch stage is responsible for loading candidate neighbor nodes for similarity evaluation. As shown in Figure 12, the neighbor index queue stores the indices of unvisited neighboring nodes. At each traversal step, the fetch unit dequeues a batch of node indices from the neighbor index queue and fetches their embeddings from the on-chip global buffer or off-chip memory. The fetched embeddings are stored in the embedding buffer before being consumed by the compute stage. To overlap irregular embedding fetching with similarity computation, the embedding buffer is double-buffered.

Compute. The compute stage calculates the similarity between the query embedding and candidate node embeddings. The query embedding is first loaded into the query buffer and then broadcast to the PE array. Candidate node embeddings are loaded from the embedding buffer into per-column FIFOs.

The PE array is designed to exploit both NLP and ELP. Each row is assigned to one candidate neighbor node, while each column is assigned to one embedding channel group, i.e., a subset of embedding dimensions. Therefore, different rows process different candidate nodes in parallel, and different PEs within the same row compute partial similarity scores of a single embedding over different dimensions.

Each column of PEs operates like a one-dimensional systolic array. The query embedding values are reused within one column, while different PEs in a column read the values of different candidate nodes. The right side of Figure 12 shows the PE design. Each PE receives a query embedding element from the previous PE and a candidate-node embedding element from the per-column FIFO. It then performs the similarity computation and stores partial accumulated similarity scores in local registers. Depending on the similarity metric, the compute logic can support different operations.

Accumulation. After each PE finishes computing the partial similarity score for its assigned embedding-channel group, the partial score is forwarded to the next PE in the same row via the inter-PE forwarding lane, shown by the dashed path in Figure 12. During forwarding, the accumulation logic combines partial scores across columns and obtains the full similarity scores, which are stored in the accumulation buffer.

There are two ways to consume these scores, depending on the HNSW layer. For the upper layers, HNSW performs greedy search and only needs to identify the single closest neighbor. Therefore, the full similarity scores are sent to a reduction unit, which selects the nearest node among the evaluated candidates. For the bottom layer, on the other hand, HNSW returns the top-k nearest neighbors. In this case, the full similarity scores are directly read from the accumulation buffer and streamed into an array of comparison units. Each score is compared against the current top-k candidates and forwarded along the comparison array. If a candidate node is closer than an existing entry in the top-k buffer, the comparison unit updates the top-k buffer and sends an update signal to the neighbor-index fetch unit.

Update. The update stage determines how the HNSW traversal proceeds. If the current layer is an upper layer, the nearest node selected by the reduction unit becomes the entry point for the next search step. If the nearest node is closer to the query than the current node, the neighbor index fetch unit is updated to fetch the unvisited neighbors of this new node. If no neighbor is closer, the search proceeds to the next layer.

At the bottom layer, the update stage maintains the top-k buffer. When a newly evaluated node is closer than one of the existing top-k candidates, the top-k buffer is updated. The neighbors of newly selected candidates are then inserted into the neighbor index queue if they have not been visited. This is done via the on-chip lookup buffer. This process continues until there is no new index in the neighbor index queue. Thus, we find the top-k nearest embedding nodes.

Checker Unit

Figure 13 shows the design of the checker unit, which supports both the kinematic check and the consistency check described in Sec. IV-C. The action queue buffers recent actions and drafted candidate actions. For the kinematic check, the extrapolation unit predicts the next action from the recent action history. For the consistency check, the weighted-mean unit computes the consensus action across the top-kk candidates. A control signal selects which action is sent to the FΔF_{\Delta} unit, which computes the discrepancy between the drafted action and the selected action. The resulting score then can be weighted by the candidate confidence wjw^j and accumulated across candidates or action components. Finally, the checker compares the accumulated score against the corresponding threshold to determine whether the action can be directly committed or should be sent for model verification.

Design of the checker unit

Figure 13. Design of the checker unit. It computes kinematic or consistency discrepancy scores with FΔF_{\Delta} and compares them with thresholds to decide whether to commit an action or send it for model verification.

Experimental Setup

Hardware Setup. The DNN accelerator in the base architecture is implemented as a systolic array with 192×192192 \times192 PEs clocked at 1 GHz. Each PE is designed for 16-bit fixed-point computation. The architecture also includes a scalar unit with 192 parallel lanes. The double-buffered global buffer is implemented using SRAM with 8 MB. The ZORA core contains a 4×44 \times4 PE array with an embedding buffer of 128 KB. One checker unit is also included in our design. We primarily evaluate ZORA using this configuration.

Experimental Methodology. The RTL implementations of the main ZORA components are synthesized using Synopsys tools and placed and routed using Cadence tools in TSMC 16 nm FinFET technology. The SRAM is generated using an ARM memory compiler. Power is estimated with Synopsys PrimeTimePX using fully annotated switching activity. Off-chip memory is modeled as eight Micron 32 Gb LPDDR5X at 8533 MT/s [23] to match the DRAM bandwidth of Nvidia Thor SoC [44], and DRAM energy is estimated using Micron’s System Power Calculator [21]. To evaluate system-level performance, we build a cycle-level simulator of the ZORA architecture, with the latency and energy of each component parameterized using post-synthesis results. We directly time the GPU execution as well as the kernel launch via CUDA events on Nvidia’s Thor SoC under 1.53 GHz, 90W power mode, and BF16 precision. The GPU power is directly measured using the built-in power sensing circuitry.

Area. The total area of ZORA is 14.5 mm2^2 at 16 nm. Compared to a base systolic array with 8 MB on-chip buffer, our augmentation introduces a minimal overhead of 0.4%. Within ZORA core, the 4×44 \times4 PE array and embedding buffer take 6.7% and 88.3%, respectively. The rest takes 5.0%.

Software Setup. We evaluate ZORA using four representative AR-based VLA algorithms with different action representations and decoding schemes: OpenVLA [25], UniVLA [63], Pi0FAST [46], and VLA0 [12]. To evaluate our framework, we use a widely used LIBERO dataset [33]. Success rate is measured as the accuracy metric using the standard procedure. Each VLA, we average the results over 4 suites, each including 10 tasks with 50 episodes, i.e., 2000 episodes per model.

Accuracy comparison over different speculative methods

Figure 14. Accuracy comparison over different speculative methods. Red crosses show SPECVLA is unavailable for the corresponding models.

Retrieval Database. To construct the global database, we execute each original VLA model to collect successful trajectories. For each VLA model, we perturb the official LIBERO initial states with random offsets to 50% of the official state range. For each task, we use separate seeds and collect around 50 successful traces with their metadata. Database collection and evaluation use disjoint seeds and initial states; no evaluation episode is included in the database. Overall, it contains a total of 7479 trajectories, with over 1.10×1061.10 \times10^6 data points. We configure HNSW with at most 16 edges per node, retrieve (kk=6) candidates, and set the action-sequence length to (τ\tau=6).

Variants. To show that ZORA can be integrated with different accelerators, we compare the following baselines:

  • GPU: a mobile Blackwell GPU in Nvidia Thor SoC [44].

  • ZORA+GPU: our software-version acceleration framework running on the GPU in the Nvidia Thor SoC [44].

  • BASE+SA: a baseline accelerator with a DNN accelerator implemented as a systolic array. It runs the baseline algorithm without our speculative decoding acceleration.

  • ZORA+SA: our framework with speculative decoding and a systolic array as the DNN accelerator.

  • BASE+DITTO: a baseline accelerator with a DNN accelerator implemented using Ditto [26]. It leverages the temporal similarity across frames to accelerate the prefill stage and runs the baseline algorithm without our speculative decoding acceleration.

  • ZORA+DITTO: our accelerator with a Ditto-like [26] accelerator and our speculative decoding acceleration.

For both BASE+DITTO and ZORA+DITTO, Ditto is applied only to the prefill stage. Applying it to decoding requires caching and accessing linear-layer outputs from the previous iteration, introducing additional memory traffic that exacerbates the memory-bound decoding bottleneck.

Evaluation

Accuracy

Fig. 14 shows the accuracy comparison among different methods. Here, we compare three methods: 1) DG-ONLY, which only uses the results from DG retrieval without any verification; 2) SPECVLA [61], which uses a small DNN as a draft model; 3) ZORA, which is our full implementation. BASE is the original algorithm without speculative decoding.

Accuracy and GPU performance comparison with SPECVLA and KERV on OpenVLA

Figure 15. Accuracy and GPU performance comparison with SPECVLA [61] and KERV [70] on OpenVLA. Numbers are measured on a Thor SoC and normalized to BASE.

Overall, ZORA achieves similar accuracy to the original algorithm. In contrast, SPECVLA and DG-ONLY drop the accuracy by 3.3% and 19.0%, respectively. Meanwhile, the draft model in SPECVLA needs to be retrained for each model; thus, we only show its result on OpenVLA. ZORA sometimes achieves better accuracy than BASE potentially for two reasons. First, our RAG database is comprehensive and incorporates traces from all VLA models. Second, its kinematic and consistency checks further exploit robot dynamics, rather than solely relying on model inference. Overall, ZORA provides the best accuracy among existing speculative approaches with great portability.

In Fig. 15, we also compare against two prior works, SPECVLA [61] and KERV [70], over OpenVLA on the LIBERO dataset. Both methods only target OpenVLA, not the other models. Overall, ZORA achieves 8.9% and 3.3% higher accuracy compared to SPECVLA and KERV, respectively. Meanwhile, ZORA has a much higher GPU speedup.

Unseen Tasks. To evaluate the generalization of ZORA to unseen tasks, we introduce ZORA-U in Figure 16. Specifically, we partition the LIBERO tasks into separate 80%/20% subsets based on task category. We then use the 80% subset to construct its RAG database following Sec. VI. All methods in Figure 16 are evaluated exclusively on identical episodes from the held-out 20% tasks. However, ZORA uses the full-task database. ZORA-U achieves a success rate comparable to that of ZORA, with only 1.5% decrease.

Figure 16. Unseen task evaluation over OpenVLA. ZORA-U shows the result on unseen held-out tasks.

Robustness. Figure 17 evaluates ZORA on OpenVLA as database coverage decreases from 100% to 12.5% by removing trace data. fig:17a shows that accuracy decreases from 78.3% to 73.4%, while GPU speedup also decreases gradually. fig:17b further shows the breakdown of different action commit paths. With full database coverage, our kinematic and consistency checks can skip almost 60% of model verification. Lower coverage reduces direct acceptance through kinematic

Figure 17. Robustness evaluation on ZORA over different dataset coverage. All results are measured on OpenVLA. We show (a) accuracy and GPU performance, (b) execution path breakdown, and (c) acceptance length (AL).

and consistency checks. More actions rely on model verification and fallback inference. Lastly, in fig:17c, both the number of accepted tokens per verification and actions per retrieval decline moderately, as database coverage decreases.

GPU Performance

Figure 18 shows the GPU performance of ZORA with different variants of our design. Here, BASE+DG applies only DG retrieval; BASE+DG+BV further applies conventional batch verification. BASE+DG+MABV instead applies batch verification in Sec. IV-D; BASE+DG+MABV+K further adds the kinematic check on BASE+DG+MABV; ZORA(+C) is the full-fledged design with the Consistency Check in Sec. IV-C.

Figure 18. GPU performance of Zora variants. BASE+DG applies only DG retrieval; BASE+DG+BV applies conventional batch verification; BASE+DG+MABV further applies masked attention for batched verification with shared prefix KV cache; BASE+DG+MABV+K includes Kinematic Check; the full-fledged design, ZORA(+C), includes Consistency Check.

In Figure 18, on average, BASE+DG only achieves 1.1×, because it still sequentially verifies each action candidate sequence. BASE+DG+BV improves the speedup to 1.4× via conventional batch verification. By combining with our batch verification, BASE+DG+MABV improves the performance to 1.6×. BASE+DG+MABV+K reaches 2.0× speedup by directly accepting dynamically reliable actions without model verification. By further exploiting the robotic kinematics and consistency, ZORA(+C) eventually can achieve 2.9× speedup. Thus, incorporating robot kinematic priors into speculative decoding can largely reduce the unnecessary verification.

Meanwhile, Fig. 15 compares the ZORA GPU performance against prior work [61], [70] on a single Thor SoC. On average, SPECVLA and KERV can 1.3× and 1.6× speedup, while ZORA can achieve a much higher speedup, 3.1×.

Hardware Performance

Speedup. fig:19a shows the performance comparison of ZORA against different accelerators. Here, all numbers are normalized to the corresponding GPU baselines. Here, we show two different accelerator baselines, BASE+SA and BASE+DITTO. BASE+SA implements the DNN accelerator via a systolic array, while BASE+DITTO uses a dedicated accelerator, Ditto [26]. Overall, BASE+SA and BASE+DITTO achieve only 1.4×1.4\times and 1.5×1.5\times speedup, respectively. This is because the decoding stage is primarily memory-bound.

On the other hand, by integrating our framework with our hardware augmentation, ZORA +SA and ZORA +DITTO achieve 4.9×4.9\times and 5.5×5.5\times speedup, respectively. There are three main reasons leading to higher speedup. First, the ZORA framework reduces the overall verification workload. Second, the batch verification makes the decoding stage less memory-bound. Third, our hardware augmentation drastically reduces the DG retrieval overhead. Overall, the results show that ZORA has great compatibility with DNN accelerator designs and consistently achieves high speedup.

In terms of frame rate, ZORA +DITTO achieves 34.8 FPS and 33.4 FPS on Pi0FAST and UniVLA, respectively. That said, decoding in ZORA remains memory-bound, which leads to the ZORA FPS being around 30 FPS, e.g., on UniVLA. On the other hand, prefill and DG retrieval are compute-bound, with a peak bandwidth requirement of 182.9 GB/s. Prefill and DG retrieval take 150.9 and 32.0 GB/s, respectively. The scalability analysis in Sec. VII-D further shows ZORA can achieve real-time (60FPS) with higher bandwidth.

Since DG retrieval is one of the bottlenecks before acceleration, we further isolate the retrieval performance in Figure 20. Figure 20 compares the performance of the ZORA core over the CPU baseline and RAGX [38], a RAG accelerator for LLM serving. The CPU baseline is measured on an Arm Neoverse V3AE CPU on a Thor SoC. RAGX [38] is also configured in a way with the same number of PEs and buffer sizes. On average, ZORA core achieves 198.8×198.8\times speedup over the corresponding CPU baseline and 3.7×3.7\times speedup over RAGX under the same hardware resource constraints. The speedup over RAGX is largely from the fine-grained node-level and embedding-level parallelism proposed in § 5-C.

DG retrieval speedup of the ZORA core over the CPU baseline on Thor SoC and RAGX

Figure 20. DG retrieval speedup of the ZORA core over the CPU baseline on Thor SoC and RAGX [38].

Energy Savings. Figure 19b shows the energy savings of ZORA against different hardware baselines. Here, we define energy savings as the ratio between the energy consumption of GPU and that of the corresponding accelerator. The overall trend aligns with the speedup results. For instance, BASE+SA achieves 13.1×13.1\times energy efficiency over the GPU baseline, and ZORA +SA improves this to 35.8×35.8\times. Similarly, ZORA +DITTO also achieves higher energy efficiency over BASE+DITTO. Even without hardware support, ZORA +GPU also delivers 2.9×2.9\times energy efficiency over GPU.

Figure 19. The performance and energy savings over different hardware variants. Bars show relative gains over the per-model GPU baseline, and markers show absolute measurements, i.e., FPS and energy per task.

We also report energy per task in Figure 19b. We accumulate the energy across all the inference within one task. On average, the energy is reduced from 2.9×1062.9\times10^{6} mJ (GPU) to 7.4×1047.4\times10^{4} mJ (ZORA +DITTO). The average power is reduced from 40.0 W (GPU) to 6.5 W (ZORA +SA) and 6.9 W (ZORA +DITTO).

Scalability Study

ZORA Core. Figure 21 shows the impact of the PE size in ZORA core on retrieval speedup. The default configuration,

Figure 21. The scalability of the DG retrieval speedup to PE array size.

4×44\times4, achieves 198.8×198.8\times speedup over the CPU baseline. This reduces the retrieval latency to <1%<1\% of the execution time. Increasing the PE array to 8×88\times8 further improves the retrieval speedup to 776.9×776.9\times. However, it provides negligible end-to-end speedup benefit. Further scaling to 16×1616\times16 yields no additional gain, as the workload shifts to memory-bound.

DNN Accelerator. Figure 22 shows how the PE array size of the DNN accelerator and on-chip buffer size affect the overall performance. Here, we show the scalability of the systolic array via a case study of OpenVLA. Overall, the on-chip buffer has a small impact on the overall performance. As for PE array size, performance scales near-linearly from 64×6464\times64 to 192×192192\times192. Further scaling to 256×256256\times256 PEs yields less performance improvement.

Figure 22. Scalability of the DNN accelerator speedup to PE array size and SRAM capacity.

DRAM Bandwidth. Figure 23 shows the impact of DRAM bandwidth on the speedup across the four hardware variants. Increasing DRAM bandwidth improves the performance of all variants, as VLA decoding is mainly memory-bound. For example, ZORA +SA improves from 1.3×1.3\times speedup at 64 GB/s to 6.5×6.5\times speedup at 512 GB/s. Compared with baseline designs, ZORA benefits more from increased bandwidth because its batch verification alleviates the memory requirements.

Figure 23. Impact of DRAM bandwidth on end-to-end speedup.

Sensitivity Study

Figure 24 shows the sensitivity of hardware performance and task success rate to different hyperparameters in the ZORA framework mentioned in Sec. IV. All experiments are conducted on OpenVLA as case studies. The performance numbers are normalized to the default configurations.

Figure 24. Sensitivity of performance and task success rate to different algorithmic hyperparameters in the ZORA framework.

Speculative Window, ww. Figure 24a shows the performance and accuracy sensitivity to the speculative window, ww. The default value is 6. w=6w=6 achieves the highest accuracy and speedup. Further increasing the speculative window will not improve the performance because fewer actions are accepted.

Top-kk Retrieved Candidates, kk. Figure 24b shows the performance and accuracy sensitivity to Top-kk retrieved candidates, kk. The default value is 6. Further increasing kk can potentially improve the performance. However, a larger kk might also introduce less accurate candidates and affect task accuracy. Selecting kk between 6 and 10 provides a good balance.

Action History Length, hh. Figure 24c shows the performance and accuracy sensitivity to the action history length, hh. The default value is 16. Increasing hh provides more historical information and improves the effectiveness of the kinematic check. However, an excessively long history introduces outdated motion patterns and reduces the accuracy of action prediction. We find that h=16h = 16 has a good trade-off.

Consensus Risk Threshold, θcon\theta_{\mathrm{con}}. Figure 24d shows the sensitivity to the consensus risk threshold θcon\theta_{\mathrm{con}}. The default value is 0.2. A smaller threshold often makes the consensus check more conservative, leading to lower speedup. Meanwhile, a larger threshold allows more actions to skip verification; setting θcon\theta_{\mathrm{con}} too high may introduce inaccurate actions. The accuracy degradation at 0.3 is due to the threshold-based verification-skipping mechanism. Increasing this threshold leads to more candidates bypassing verification. Actions near the acceptance boundary may switch to unverified execution.

Verifier Acceptance Threshold, θver\theta_{\mathrm{ver}}. Figure 24e shows how the verifier acceptance threshold θver\theta_{\mathrm{ver}} affects the performance and accuracy. The default value is 20. A lower threshold requires stronger agreement between drafted and verified actions. This reduces the number of accepted actions and performance gains. We find that θver\theta_{\mathrm{ver}} between 20 and 25 provides a good trade-off between performance and accuracy.

Acceleration for Robotics. A broad range of accelerators and systems have been proposed to improve the efficiency of robotics workloads over the past decade [2, 3, 9, 10, 14, 15, 19, 28, 36, 41, 42, 43, 51, 53, 54, 59, 60, 66, 69]. Early works primarily targeted specific robotic kernels or operations [2, 36, 51, 53, 54, 66], by exploiting their domain-specific computation patterns. Later efforts shifted toward system-level acceleration, optimizing common bottlenecks across multiple robotics algorithms rather than individual kernels [15, 28, 43]. Some studies also leverage 3D Gaussian splatting for SLAM localization [18, 30]. More recently, architectural research has expanded toward embodied AI [19, 60] and autonomous driving [9, 10], where performance increasingly depends on end-to-end system stacks. In contrast, ZORA targets the unique challenges of AR-based VLA inference by co-designing speculative decoding algorithms with dedicated hardware support.

Speculative Decoding. Speculative decoding [6, 8, 29, 40] has been widely studied to accelerate the decoding stage in LLM inference. Early approaches [8, 29] introduce a lightweight draft model to generate candidate tokens. Other system-level optimizations [6, 40] further optimize the batch verification efficiency. REST [16] accelerates LLM through retrieval-based drafting and verifying tokens with the target model. While ZORA adopts a similar workflow, the key of ZORA is to establish a dynamics-guided retrieval system and a selective action verification procedure.

Building upon these advances, recent works [20, 61, 70, 71] have explored speculative decoding to reduce the inference latency for AR-based VLA models. For instance, Spec-LLaVA [20] investigates different strategies for speculating future tokens. Others [61, 70] also further optimize the draft-and-verify process in the speculation process to improve action prediction accuracy. Our work addresses this challenge by introducing RAG-augmented and kinematics-aware speculation and provides more accurate action generation while improving the overall performance. Meanwhile, ZORA introduces a hardware augmentation to further boost the performance with negligible area overhead.

RAG Acceleration. Prior work accelerates retrieval via near-memory processing [48, 49], in-memory/storage processing [38, 68]. These studies target server-scale RAG workloads, whereas ZORA targets the latency of individual action-retrieval queries in VLA inference. In particular, IKS [48] accelerates exact nearest-neighbor search, while RAGX [38] supports diverse retrieval algorithms and embedding generation. ZORA specializes its retrieval engine by exploiting node-level parallelism across candidate neighbors and embedding-level parallelism within similarity calculations to unlock new parallelism without requiring query batching.

Conclusion

Overall, ZORA demonstrates that efficient VLA inference requires jointly exploiting the characteristics of VLA workloads and the intrinsic dynamics of robotic systems. ZORA achieves up to 3.4× speedup with software-only optimization and up to 7.6× speedup with minimal hardware augmentation. Our results show that VLA model workloads can be effectively optimized by incorporating domain-specific robotic knowledge. This co-design provides a promising direction for building efficient, real-time embodied AI systems.

References

References

  1. [1]F. Ai, “Helix: A Vision-Language-Action Model for Generalist Humanoid Control,” 2026. [Online]. Available: https://www.figure.ai/news/helix
  2. [2]M. Bakhshalipour, S. B. Ehsani, M. Qadri, D. Guri, M. Likhachev, and P. B. Gibbons, “Racod: algorithm/hardware co-design for mobile robot path planning,” in Proceedings of the 49th Annual International Symposium on Computer Architecture, ser. ISCA ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 597–609. [Online]. Available: https://doi.org/10.1145/3470496.3527383
  3. [3]M. Bakhshalipour and P. B. Gibbons, “Tartan: Microarchitecting a robotic processor,” in Proceedings of the 51st Annual International Symposium on Computer Architecture, ser. ISCA ’24. IEEE Press, 2024, p. 548–565. [Online]. Available: https://doi.org/10.1109/ISCA59077.2024.00047
  4. [4]S. Belkhale, T. Ding, T. Xiao, P. Sermanet, Q. Vuong, J. Thompson, Y. Chebotar, D. Dwibedi, and D. Sadigh, “Rt-h: Action hierarchies using language,” 2024.arxiv.org/abs/2403.01823
  5. [5]K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky, “Octocache: Caching voxels for accelerating 3d occupancy mapping in autonomous systems,” in Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. New York, NY, USA: Association for Computing Machinery, 2025, p. 704–718. [Online]. Available: https://doi.org/10.1145/3676641.3716263
  6. [6]T. Cai, Y. Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, and T. Dao, “Medusa: Simple llm inference acceleration framework with multiple decoding heads,” in Proceedings of the 41st International Conference on Machine Learning, ser. ICML’24. JMLR.org, 2024.arxiv.org/abs/2401.10774
  7. [7]J. Cen, C. Yu, H. Yuan, Y. Jiang, S. Huang, J. Guo, X. Li, Y. Song, H. Luo, F. Wang et al., “Worldvla: Towards autoregressive action world model,” arXiv preprint arXiv:2506.21539, 2025.arxiv.org/abs/2506.21539
  8. [8]C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating large language model decoding with speculative sampling,” arXiv, 2023.arxiv.org/abs/2302.01318
  9. [9]P. Chen, M. Li, Z. Wan, Y.-S. Hsiao, M. Yu, V. J. Reddi, and Z. Liu, “Octocache: Caching voxels for accelerating 3d occupancy mapping in autonomous systems,” in Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. New York, NY, USA: Association for Computing Machinery, 2025, p. 704–718. [Online]. Available: https://doi.org/10.1145/3676641.3716263
  10. [10]W. Choe, R. Wang, and F. X. Lin, “Ana: An attentive autonomous driving system,” in Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, ser. ASPLOS ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 33–46. [Online]. Available: https://doi.org/10.1145/3669940.3707261
  11. [11]T. Feng, X. Wang, Y.-G. Jiang, and W. Zhu, “Embodied ai: From llms to world models,” arXiv preprint arXiv:2509.20021, 2025.arxiv.org/abs/2509.20021
  12. [12]A. Goyal, H. Hadfield, X. Yang, V. Blukis, and F. Ramos, “Vla-0: Building state-of-the-art vlas with zero modification,” arXiv preprint arXiv:2510.13054, 2025.arxiv.org/abs/2510.13054
  13. [13]Y. Guo, Y. Hu, J. Zhang, Y.-J. Wang, X. Chen, C. Lu, and J. Chen, “Prediction with action: Visual policy learning via joint denoising process,” in 2024 Conference on Neural Information Processing Systems, 2024.arxiv.org/abs/2411.18179
  14. [14]R. Hadidi, B. Asgari, S. Jijina, A. Amyette, N. Shoghi, and H. Kim, “Quantifying the design-space tradeoffs in autonomous drones,” in Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ser. ASPLOS ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 661–673. [Online]. Available: https://doi.org/10.1145/3445814.3446721
  15. [15]Y. Hao, Y. Gan, B. Yu, K. Liu, Y. Han, Z. Wan, and S. Liu, “Orianna: An accelerator generation framework for optimization-based robotic applications,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, ser. ASPLOS ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 813–829. [Online]. Available: https://doi.org/10.1145/3620665.3640379
  16. [16]Z. He, Z. Zhong, T. Cai, J. Lee, and D. He, “Rest: Retrieval-based speculative decoding,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024, pp. 1582–1595.
  17. [17]J. Heo, A. Putra, J. Yoon, S. Yune, H. Lee, J.-H. Kim, and J.-Y. Kim, “Exion: Exploiting inter-and intra-iteration output sparsity for diffusion models,” in 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA), 2025, pp. 324–337.arxiv.org/abs/2501.05680
  18. [18]X. Huang, H. Zhu, T. Ma, Y. Xiong, F. Liu, Z. He, Y. Gan, Z. Liu, J. Leng, Y. Feng, and M. Guo, “Splatonic: Architectural support for 3d gaussian splatting slam via sparse processing,” in Proceedings of the 32nd IEEE International Symposium on High Performance Computer Architecture, 2026.DOI
  19. [19]Y. Huang, Y. Hao, B. Yu, F. Yan, Y. Yang, F. Min, Y. Han, L. Ma, S. Liu, Q. Liu, and Y. Gan, “Dadu-corki: Algorithm-architecture co-design for embodied ai-powered robotic manipulation,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture, ser. ISCA ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 327–343. [Online]. Available: https://doi.org/10.1145/3695053.3731099
  20. [20]M. Huo, J. Zhang, H. Wang, J. Xu, Z. Chen, H. Tai, and Y. Chen, “Spec-llava: Accelerating vision-language models with dynamic tree-based speculative decoding,” arXiv preprint arXiv:2509.11961, 2025.arxiv.org/abs/2509.11961
  21. [21]M. T. Inc., “Micron system power calculators,” https://www.micron.com/support/tools-and-utilities/power-calc, 2025.
  22. [22]W. Jiang, S. Subramanian, C. Graves, G. Alonso, A. Yazdanbakhsh, and V. Dadu, “Rago: Systematic performance optimization for retrieval-augmented generation serving,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture, ser. ISCA ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 974–989. [Online]. Available: https://doi.org/10.1145/3695053.3731093
  23. [23]H.-D. Kim, B. Song, A.-H. Ahn, W. Ko, S. D. Cho, K. Kim, S.-H. Oh, H.-Y. Joo, G. Park et al., “A 16gb 9.5 gb/s/pin lpddr5x sram with low-power schemes exploiting dynamic voltage-frequency scaling and offset-calibrated readout sense amplifiers in a fourth generation 10nm dram process,” in 2022 IEEE International Solid-State Circuits Conference (ISSCC), vol. 65. IEEE, 2022, pp. 448–450.
  24. [24]M. J. Kim, C. Finn, and P. Liang, “Fine-tuning vision-language-action models: Optimizing speed and success,” arXiv preprint arXiv:2502.19645, 2025.arxiv.org/abs/2502.19645
  25. [25]M. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn, “Openvla: An open-source vision-language-action model,” arXiv preprint arXiv:2406.09246, 2024.arxiv.org/abs/2406.09246
  26. [26]S. Kim, H. Lee, W. Cho, M. Park, and W. W. Ro, “Ditto: Accelerating diffusion model via temporal value similarity,” in 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 2025, pp. 338–352.arxiv.org/abs/2501.11211
  27. [27]W. Kong, Y. Hao, Q. Guo, Y. Zhao, X. Song, X. Li, M. Zou, Z. Du, R. Zhang, C. Liu et al., “Cambricon-d: Full-network differential acceleration for diffusion models,” in 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEEE, 2024, pp. 903–914.
  28. [28]S. Krishnan, Z. Wan, K. Bhardwaj, P. Whatmough, A. Faust, S. Neuman, G.-Y. Wei, D. Brooks, and V. J. Reddi, “Automatic domain-specific soc design for autonomous unmanned aerial vehicles,” in Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture, ser. MICRO ’22. IEEE Press, 2023, p. 300–317. [Online]. Available: https://doi.org/10.1109/MICRO56248.2022.00033
  29. [29]Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformers via speculative decoding,” in International Conference on Machine Learning. PMLR, 2023, pp. 19274–19286.arxiv.org/abs/2211.17192
  30. [30]L. Li, J. Qin, J. Peng, Z. Wan, H. Qu, Y. Han, P. Zheng, H. Zhang, Y. Cao, T. Chen, and Y. K. Zhao, “Rtgs: Real-time 3d gaussian splatting via multi-level redundancy reduction,” in Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture, ser. MICRO ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 1838–1851. [Online]. Available: https://doi.org/10.1145/3725843.3756099
  31. [31]S. Li, Y. Gao, D. Sadigh, and S. Song, “Unified video action model,” in Proceedings of Robotics: Science and Systems (RSS), 2025.arxiv.org/abs/2503.00200
  32. [32]X. Li, M. Liu, H. Zhang, C. Yu, J. Xu, H. Wu, C. Cheang, Y. Jing, W. Zhang, H. Liu et al., “Vision-language foundation models as effective robot imitators,” in International Conference on Learning Representations, vol. 2024, 2024, pp. 26703–26721.arxiv.org/abs/2311.01378
  33. [33]B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone, “Libero: Benchmarking knowledge transfer for lifelong robot learning,” arXiv preprint arXiv:2306.03310, 2023.arxiv.org/abs/2306.03310
  34. [34]C. Liu, H. Liu, D. Chen, Y. Huang, Y. Zhang, W. Xiao, X. Liao, and H. Jin, “HeterRAG: Heterogeneous processing-in-memory acceleration for retrieval-augmented generation,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture, ser. ISCA ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 884–898. [Online]. Available: https://doi.org/10.1145/3695053.3731089
  35. [35]S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “Rdt-lb: a diffusion foundation model for bimanual manipulation,” arXiv preprint arXiv:2410.07864, 2024.
  36. [36]W. Liu, B. Yu, Y. Gan, Q. Liu, J. Tang, S. Liu, and Y. Zhu, “Archytas: A framework for synthesizing and dynamically optimizing accelerators for robotic localization,” in MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture, ser. MICRO ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 479–493. [Online]. Available: https://doi.org/10.1145/3466752.3480077
  37. [37]X. Liu, L. Hu, P. Bailis, A. Cheung, Z. Deng, I. Stoica, and H. Zhang, “Online speculative decoding,” in Proceedings of the 41st International Conference on Machine Learning, ser. ICML’24, 2024.
  38. [38]R. Mahapatra, H. Santhanam, C. Priebe, H. Xu, and H. Esmaeilzadeh, “In-storage acceleration of retrieval augmented generation as a service,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture, ser. ISCA ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 450–466. [Online]. Available: https://doi.org/10.1145/3695053.3731032
  39. [39]Y. A. Malkov and D. A. Yashunin, “Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 4, pp. 824–836, 2020.
  40. [40]X. Miao, G. Oliaro, Z. Zhang, X. Cheng, Z. Wang, Z. Zhang, R. Y. Y. Wong, A. Zhu, L. Yang, X. Shi, C. Shi, Z. Chen, D. Arfeen, R. Abhyankar, and Z. Jia, “Specinfer: Accelerating large language model serving with tree-based speculative inference and verification,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, ser. ASPLOS ’24, New York, NY, USA, 2024, p. 932–949. [Online]. Available: https://doi.org/10.1145/3620666.3651335
  41. [41]S. M. Neuman, R. Ghosal, T. Bourgeat, B. Plancher, and V. J. Reddi, “Roboshape: Using topology patterns to scalably and flexibly deploy accelerators across robots,” in Proceedings of the 50th Annual International Symposium on Computer Architecture, ser. ISCA ’23. New York, NY, USA: Association for Computing Machinery, 2023. [Online]. Available: https://doi.org/10.1145/3579371.3589104
  42. [42]S. M. Neuman, B. Plancher, T. Tambe, S. T. Banne, S. Devadas, and V. J. Reddi, “Robomorphic computing: a design methodology for domain-specific accelerators parameterized by robot morphology,” in Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ser. ASPLOS ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 674–686. [Online]. Available: https://doi.org/10.1145/3453481.3446746
  43. [43]D. Nikiforov, S. C. Dong, C. L. Zhang, S. Kim, B. Nikolic, and Y. S. Shao, “RosÉ: A hardware-software co-simulation infrastructure enabling pre-silicon full-stack robotics soc evaluation,” in Proceedings of the 50th Annual International Symposium on Computer Architecture, ser. ISCA ’23. New York, NY, USA: Association for Computing Machinery, 2023. [Online]. Available: https://doi.org/10.1145/3579371.3589099
  44. [44]Nvidia, “Nvidia jetson thor: The ultimate platform for physical ai and robotics,” 2025. [Online]. Available: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-thor/
  45. [45]M. Oquab, T. Darcet, T. Moutakann, H. Vo, M. Szafraniec, Y. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby et al., “Dinov2: Learning robust visual features without supervision,” arXiv preprint arXiv:2304.07193, 2023.arxiv.org/abs/2304.07193
  46. [46]K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine, “Fast: Efficient action tokenization for vision-language-action models,” arXiv preprint arXiv:2501.09747, 2025.arxiv.org/abs/2501.09747
  47. [47]D. Qu, H. Song, Q. Chen, Y. Yao, X. Ye, Y. Ding, Z. Wang, J. Gu, B. Zhao, D. Wang et al., “Spatialvla: Exploring spatial representations for visual-language-action model,” arXiv preprint arXiv:2501.15830, 2025.arxiv.org/abs/2501.15830
  48. [48]D. Quinn, M. Nouri, N. Patel, J. Salihu, A. Salemi, S. Lee, H. Zamani, and M. Alian, “Accelerating retrieval-augmented generation,” in Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, ser. ASPLOS ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 15–32. [Online]. Available: https://doi.org/10.1145/3669940.3707264
  49. [49]D. Quinn, N. Patel, and M. Alian, “Compute-enabled cxl memory expansion for efficient retrieval augmented generation,” IEEE Micro, vol. 45, no. 6, pp. 100–107, 2025.DOI
  50. [50]G. V. Research, “Humanoid Robot Market (2025 - 2030),” 2026. [Online]. Available: https://www.grandviewresearch.com/industry-analysis/humanoid-robot-market-report
  51. [51]J. Sacks, D. Mahajan, R. C. Lawson, B. Khaleghi, and H. Esmaeilzadeh, “Robocx: an end-to-end solution to accelerate autonomous control in robotics,” in Proceedings of the 45th Annual International Symposium on Computer Architecture, ser. ISCA ’18. IEEE Press, 2018, p. 479–490. [Online]. Available: https://doi.org/10.1145/3183713.3180047
  52. [52]M. Schubiger, G. Banjac, and J. Lygeros, “Gpu acceleration of admm for large-scale quadratic programming,” Journal of Parallel and Distributed Computing, vol. 144, pp. 55–67, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0743731520303063DOI
  53. [53]D. Shah and T. M. Aamodt, “Collision prediction for robotics accelerators,” in Proceedings of the 51st Annual International Symposium on Computer Architecture, ser. ISCA ’24. IEEE Press, 2024, p. 566–581. [Online]. Available: https://doi.org/10.1109/ISCA59077.2024.00048
  54. [54]D. Shah, N. Yang, and T. M. Aamodt, “Energy-efficient realtime motion planning,” in Proceedings of the 50th Annual International Symposium on Computer Architecture, ser. ISCA ’23. New York, NY, USA: Association for Computing Machinery, 2023. [Online]. Available: https://doi.org/10.1145/3579371.3589092
  55. [55]M. Shen, M. Umar, K. Maeng, G. E. Suh, and U. Gupta, “Hermes: Algorithm-system co-design for efficient retrieval-augmented generation at-scale,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture, ser. ISCA ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 958–973. [Online]. Available: https://doi.org/10.1145/3695053.3731076
  56. [56]B. Stellato, G. Banjac, P. Goulart, A. Bemporad, and S. Boyd, “Osqp: An operator splitting solver for quadratic programs,” in 2018 UKACC 12th International Conference on Control (CONTROL), 2018, pp. 339–339.
  57. [57]F. Sun, R. Chen, T. Ji, Y. Luo, H. Zhou, and H. Liu, “A comprehensive survey on embodied intelligence: Advancements, challenges, and future perspectives,” CAAI Artificial Intelligence Research, vol. 3, no. 950042, p. 1, 2024.DOI
  58. [58]J. Tenenberg and B. Bäuml, “Calibration of an elastic humanoid upper body and efficient compensation for motion planning,” in 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids), 2021, pp. 98–103.
  59. [59]J. Ting, M. Kim, J. Zhu, H. Sheng, and Z. Zhang, “Hiper: Hierarchically-composed processing for efficient robot learning-based control,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture, ser. ISCA ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 313–326. [Online]. Available: https://doi.org/10.1145/3695053.3731097
  60. [60]Z. Wan, Y. Du, M. Ibrahim, J. Abbour, Y. K. Zhao, T. Krishna, A. Raychowdhury, and V. J. Reddi, “Reca: Integrated acceleration for real-time and efficient cooperative embodied autonomous agents,” in Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. New York, NY, USA: Association for Computing Machinery, 2025, p. 982–997. [Online]. Available: https://doi.org/10.1145/3676641.3716016
  61. [61]S. Wang, R. Yu, Z. Yuan, C. Yu, F. Gao, Y. Wang, and D. F. Wong, “Spec-vla: speculative decoding for vision-language-action models with relaxed acceptance,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 6916–6928.arxiv.org/abs/2507.22424
  62. [62]Y. Wang, H. Zhu, M. Liu, J. Yang, H.-S. Fang, and T. He, “Vq-vla: Improving vision-language-action models via scaling vector-quantized action tokenizers,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025.arxiv.org/abs/2507.01016
  63. [63]Y. Wang, X. Li, W. Wang, J. Zhang, Y. Li, Y. Chen, X. Wang, and Z. Zhang, “Unified vision-language-action model,” arXiv preprint arXiv:2506.19850, 2025.arxiv.org/abs/2506.19850
  64. [64]G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive driving with model predictive path integral control,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), 2016, pp. 1433–1440.DOI
  65. [65]H. Wu, Y. Jing, C. Cheang, G. Chen, J. Xu, X. Li, M. Liu, H. Li, and T. Kong, “Unleashing large-scale video generative pre-training for visual robot manipulation,” in International Conference on Learning Representations, vol. 2024, 2024, pp. 10641–10662.arxiv.org/abs/2312.13139
  66. [66]Y. Yang, X. Chen, and Y. Han, “Dadu-rbd: Robot rigid body dynamics accelerator with multifunctional pipelines,” in Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture, ser. MICRO ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 297–309. [Online]. Available: https://doi.org/10.1145/3613424.3614298
  67. [67]Z. Yu, B. Wang, P. Zeng, H. Zhang, J. Zhang, Z. Wang, L. Gao, J. Song, N. Sebe, and H. T. Shen, “A survey on efficient vision-language-action models,” arXiv preprint arXiv:2510.24795, 2025.arxiv.org/abs/2510.24795
  68. [68]N. Zhang, W. Zhu, C. Golden, D. Ilan, H. Chen, C. Batten, and Z. Zhang, “Characterizing and optimizing realistic workloads on a commercial compute-in-sram device,” in Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture, 2025, pp. 1011–1025.arxiv.org/abs/2509.05451
  69. [69]S. Zhang, Y. Hu, Y. Xiang, D. Zhao, Y. Xu, Q. Liao, J. Yu, and Y. Wang, “Idea-gap: Instruction-driven architecture with efficient online workload allocation for geometric perception,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture, ser. ISCA ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 1675–1688. [Online]. Available: https://doi.org/10.1145/3695053.3731080
  70. [70]Z. Zheng, Z. Mao, M. Li, J. Chen, X. Sun, Z. Zhang, D. Cao, H. Mei, and X. Chen, “Kinematic-rectified speculative decoding for embodied vla models,” arXiv preprint arXiv:2603.01581, 2026.arxiv.org/abs/2603.01581
  71. [71]Z. Zheng, Z. Mao, S. Tian, M. Li, J. Chen, X. Sun, Z. Zhang, X. Liu, D. Cao, H. Mei et al., “Headish: Hybrid speculative decoding for embodied vision-language-action models with kinematic awareness,” arXiv preprint arXiv:2603.17573, 2026.arxiv.org/abs/2603.17573
  72. [72]K. Zhou, Q. Chen, D. Peng, Z. Li, X. Li, and J. Gu, “Characterizing vision-language-action models across xpus: Constraints and acceleration for on-robot deployment,” arXiv preprint arXiv:2604.24447, 2026.arxiv.org/abs/2604.24447
  73. [73]Y. Zhou, K. Lyu, A. S. Rawat, A. K. Menon, A. Rostamizadeh, S. Kumar, J.-F. Kagy, and R. Agarwal, “Distillspec: Improving speculative decoding via knowledge distillation,” in International Conference on Learning Representations, vol. 2024, 2024, pp. 32011–32050.arxiv.org/abs/2310.08461

Paper details

vlnc:W_zmk4whsa
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents