betaFeedback

WorksComputer Science

FlowEval: How Much Adaptation Does an Embodied Foundation Model Need at Deployment?

Abstract

Large-scale pretraining is reducing the amount of adaptation required to deploy embodied foundation models, but it is not guaranteed that a pretrained model already provide the level of performance required in every deployment. Practitioners therefore need to understand how well a pretrained model matches a target deployment and how much adaptation data is left to be collected. We study whether a small set of successful target-domain demonstrations can reveal whether and how much target-domain data will be required before any adaptation. We introduce FlowEval, a set of training-free metrics that use the flow-matching objective to probe an embodied foundation model with a set of demonstrations. FlowEval quantifies predictive disagreement on successful target demonstrations to assess how well a pretrained model fits a target deployment. Across four embodied foundation model families: Pi-0.5, MolmoAct2, GR00T, and DreamZero, FlowEval distinguishes seen from unseen deployments across real-world and simulated manipulation. Further, by finetuning Pi-0.5 and GR00T on 17 RoboCasa365 manipulation tasks, we showed that FlowEval correlates with how success rate improves as additional adaptation data is provided. Together, these results suggest that FlowEval can serve as a lightweight, training-free diagnostic of how much model–deployment adaptation may be required.

Introduction

Large-scale pretraining is enabling embodied foundation models to generalize across increasingly broad variation in tasks, environments, and robot embodiments [1, 2, 3, 4]. Yet broad generalization does not imply that a pretrained model will already provide the level of performance required in every downstream deployment [3, 5, 6]. In practice, models may still require target-domain adaptation to acquire novel physical behaviors, transfer to another action space, or improve behaviors that are only partially captured by the pretrained prior. While broad pretraining increasingly reduces the amount of task-specific learning required, adaptation remains an important component of deploying today’s embodied foundation models [2, 7, 8].

As embodied models become more general-purpose, this changes the deployment problem itself. Rather than asking only how to adapt a pretrained model to a downstream setting, practitioners increasingly need to understand how much of the required behavior is already provided by pretraining and how much additional adaptation remains. Importantly, poor zero-shot performance does not reveal whether the desired behavior can be recovered with a small amount of target-domain data or requires substantially more adaptation, and a model with stronger broad capabilities need not require less adaptation for a particular deployment. This motivates two questions: 1) whether the target deployment is already seen by the model, and, ultimately, 2) how much additional data will be required to achieve the desired level of performance?

We study whether the answer to these questions can be probed directly with a small set of successful target-domain demonstrations. Such trajectories provide examples of the behavior the deployed system is expected to reproduce, and can often be collected with substantially less effort than the full datasets required for finetuning. This suggests a simple hypothesis: downstream adaptation difficulty should depend on how well the pretrained model already fits the intended target behavior. Rather than first finetuning several candidate models and observing which one adapts successfully, we ask whether this model–deployment relationship can be measured before adaptation.

A natural quantity to examine is the pretrained model’s training loss on successful target-domain trajectories. In language modeling, held-out cross-entropy and perplexity have long been used to characterize how well a model fits a target domain and to select data that better matches that domain [9, 10, 11]. Flow matching is particularly well suited to assessing model–data fit because its training objective probes not only the predicted action, but the model’s local direction of improvement toward the demonstrated action [12]. We therefore introduce FlowEval, a set of training-free metrics derived from the flow-matching objective that measure how well a pretrained model fits successful target-domain demonstrations by computing the predictive disagreement. FlowEval requires no finetuning and applies whether or not the pretrained model and target deployment share the same action representation. This is particularly important for deployment, as action representations and deployment interfaces can differ widely across model families and robot embodiments. [4, 13]

We evaluate FlowEval on Pi-0.5, MolmoAct2, GR00T, and DreamZero, across real-world and simulated manipulation and multiple robot embodiments. Across these settings, FlowEval predicted directional disagreement correlates distinguishes seen deployment distributions from unseen deployment distributions. More importantly, by finetuning Pi-0.5 and GR00T on 17 RoboCasa365 manipulation tasks, we find that the same signal correlates with how performance scales as additional adaptation data is provided. Together, these results suggest that FlowEval can serve as a training-free diagnostic of how much target-domain adaptation may be required.

Related Work

Adapting embodied foundation models

Downstream finetuning is the widely adopted approach to adapt a general pretrained policy. State-of-the-art open source embodied foundation models, including but not limited to DreamZero [8], MolmoAct2 [7], Pi-05 [2], and GR00T [13], provide finetuning pipelines and include finetuning guidelines at release. Prior to them, OpenVLA [1], OpenVLA-OFT [5], and Octo [3] expected downstream finetuning even more. On the proprietary OpenAI, Generalist Gen1.5 [14] reported using few gradient step adaptation to better learn novel tasks; and Gemini Robotics [15, 6] reported fine-tuning the model to target embodiments. Although Figure AI [16], Skild [17], and Genesis AI [18] do not report downstream fine-tuning, notably, their pretraining datasets are directly built for their deployment settings. This is fundamentally different from the situation faced by practitioners, who lack the ability to construct a large-scale pretraining dataset specifically for their deployment.

Learning In-Context is a promising yet constrained approach for a pretrained embodied foundation model to adapt to a new task. DreamZero [8] and Skild S1 [17] provided video as context and shows the model is able to infer tasks from the world to state information in the videos. Generalist Gen1.5 [14] provided one-shot state to action demonstration as context and shows emerging ability of the model generalizing to novel objects. However, all require the embodiment and action representation to align between the pretrained and deployment settings. Team [14] also reported that skills learned in-context are more brittle than finetuned models.

Zero-shot execution is possible in scenarios the embodiment and action representation between pretrained and deployment settings are aligned. Fang et al. [7], Octo Model Team et al. [3], Team [19] showed zero-shot execution of tasks on supported embodiments. Similar to in-context learning, their prerequisites are more restrictive than most practical deployment settings.

Training objectives as Evaluations

The optimization objective, such as loss or negative log-likelihood (NLL), is commonly used as an evaluation metric for model performance in machine learning [20, 21] and especially on large language models [22, 23]. In robotics, validation loss is often used a proxy for performance to select policies [24, 25]. As to embodied foundation models including RT-2 [26], Octo [3], and OpenVLA [1], however, multiple works [27, 28] showed that validation loss poorly reflects actual robot performance. Interestingly, none of these work [27, 28] tested flow-matching based models.

Flow objectives as evaluations

Likelihood-based flows naturally allows likelihood evaluations. A common use case is to evaluate the model on an entire held-out dataset, using the likelihood to measure how well the model fits unseen data [29, 30]. In contrast, using the likelihood of a single data point as a measure of whether that point is in- or out-of-distribution is a stronger assumption and is not generally guaranteed [31, 32]. However, in robotics, where the generated distribution is conditional action likelihoods p(a∣s)p(a \mid s), likelihood have been successfully applied as measures of action plausibility or anomaly: Wenger et al. [33] use it to detect several types of antagonistic robot behavior, while Zhang et al. [34], Chae et al. [35] effectively applied it inside offline-RL algorithms as an OOD score.

The flow matching loss shares some common properties with likelihood-trained normalizing flows, such as evaluating how well a policy fits a held-out demonstration set. Zha et al. [36] evaluates the validation loss of its flow-matching action expert on a held-out split and uses it to compare model scaling. Recent robot policy evaluations also compute the flow matching objective on held-out episodes as an offline action-prediction loss [37]. Park [38] evaluated the relationship between flow-matching loss and success rate, and discovers that lower training loss is associated with higher validation loss and higher performance.

FlowEval differs from prior works in two meaningful ways. (1) We discovered that starting from pure noise provided us with cleaner evaluation signals than noisy actions (Appendix § 7.1.1) (2) Instead of using loss to select checkpoints from the same training run, we use pre-adaptation loss to probe how finetuning outcomes vary across target deployments and demonstration budgets.

Methodology

We begin by clarifying the deployment settings considered in this work. We distinguish the conditions that fall within and outside the scope of our metric evaluation.

Demonstration: We define a demonstration as an observation-state-action trajectory demonstrating successful completion of a task. In this work, we primarily consider demonstrations collected via human teleoperation. Although approaches such as MimicGen [39], DemoGen [40], and other forms of augmenting existing demonstrations may be interesting directions to explore, they are beyond the scope of this study (Appendix § 7.§ 2).

Deployment settings: We define a deployment setting as one or more manipulation tasks performed under a fixed environment, embodiment, and action representation. We further assume access to a small set of demonstrations for the target task(s). We do not assume that the action representations and dimensions of the deployment setting matches with a model’s pretrained distribution, since in actual deployments they often do not.

Practical Scope: Most practitioners do not have access to a foundational model trained specifically for their deployment settings – only those who pretrain the model themselves do. Instead, they often seek to adapt an existing foundation model to a target deployment without full, or even partial, access to its pretraining data. Their deployment may involve a custom embodiment, camera configuration, task setup, or action representation. Even when these components are otherwise standard, they may still differ from the exact configurations supported by the pretrained model. Our definitions therefore describe a deployment scenario that practitioners are most likely to encounter in practice.

FlowEval Metrics

Using flow matching objectives to evaluate how well a trained model fits a given dataset is a promising yet under-explored direction in robotics (§ 2.2). We propose FlowEval metrics based on the intuition that the flow matching objective provides a measurable signal of how well the embodied foundation model fits a deployment dataset, which further affects how much adaptation is needed. Since FlowEval relies on properties of flow-based objectives that may not generalize to other training objectives, its metrics are only applicable to embodied foundation models that use flow matching for action prediction.

For vision-language action (VLA) models, the forward pass can be written as

u^A=Φθ(AτA,hvision,hlanguage,hstate,htimeA),(1)\hat{u}_A = \Phi_\theta\left(A^{\tau_A}, h_{\mathrm{vision}}, h_{\mathrm{language}}, h_{\mathrm{state}}, h_{\mathrm{time}_A}\right), \tag*{(1)}

A world-action model (WAM) additionally predicts future quantities, which can extend beyond videos. We denote those future quantities by mm.

(u^A,{u^m}m)=Ψθ(AτA,{Zmτm}m,hv,hℓ,hs,htA,{htm}m),(2)\left(\hat{u}_A, \{\hat{u}_m\}_m\right) = \Psi_\theta\left(A^{\tau_A}, \{Z_m^{\tau_m}\}_m, h_v, h_\ell, h_s, h_{t_A}, \{h_{t_m}\}_m\right), \tag*{(2)}

For ground-truth actions A∗A^* and future targets Zm∗Z_m^*, conditional flow matching uses

AτA=(1−τA)A∗+τAϵA,Zmτm=(1−τm)Zm∗+τmϵm,(3)A^{\tau_A} = (1-\tau_A)A^* + \tau_A\epsilon_A,\qquad Z_m^{\tau_m} = (1-\tau_m)Z_m^* + \tau_m\epsilon_m, \tag*{(3)}

with Gaussian noise ϵA,ϵm∼N(0,I)\epsilon_A,\epsilon_m \sim\mathcal{N}(0,I) and reference velocities uA=ϵA−A∗u_A = \epsilon_A - A^* and um=ϵm−Zm∗u_m = \epsilon_m - Z_m^*.

FlowEval initializes the action and future-target trajectories from Gaussian noise, setting A1=ϵAA^1 = \epsilon_A and Zm1=ϵmZ_m^1 = \epsilon_m. Subsequent states follow the model’s inference dynamics. At each inference step kk, we evaluate u^A,k\hat{u}_{A,k} and u^m,k\hat{u}_{m,k} against the fixed reference velocities defined above. Although validating with held-out trajectories conventionally starts from ground-truth-interpolated noisy actions, we find that starting from pure noise instead yields more stable results, discussed in A.1.1.

Velocity error. The action and future-prediction velocity errors are

LA,k=∥u^A,k−uA∥22,Lm,k=∥u^m,k−um∥22,(4)L_{A,k} = \lVert\hat{u}_{A,k} - u_A \rVert_2^2,\qquad L_{m,k} = \lVert\hat{u}_{m,k} - u_m \rVert_2^2, \tag*{(4)}

These are basically the loss objectives. We refer to these as velocity error to differentiate from training or validation loss, which conventionally starts from ground-truth-interpolated noisy actions.

Cosine similarity. We measure the directional agreements with the reference velocities:

cosine⁡A,k=⟨MA⊙u^A,k,MA⊙uA⟩∥MA⊙u^A,k∥2∥MA⊙uA∥2+ϵ,cosine⁡m,k=⟨u^m,k,um⟩∥u^m,k∥2∥um∥2+ϵ.(5)\begin{aligned} \operatorname{cosine}_{A,k} &= \frac{\langle M_A \odot\hat{u}_{A,k}, M_A \odot u_A\rangle}{\lVert M_A \odot\hat{u}_{A,k}\rVert_2 \lVert M_A \odot u_A\rVert_2 + \epsilon},\\ \operatorname{cosine}_{m,k} &= \frac{\langle\hat{u}_{m,k}, u_m\rangle}{\lVert\hat{u}_{m,k}\rVert_2 \lVert u_m\rVert_2 + \epsilon}. \tag*{(5)} \end{aligned}

where MAM_A is the action mask, and ϵ>0\epsilon> 0 stabilizes the denominator.

Design Choices

FlowEval metrics are defined over the full trajectory of diffusion steps under the model’s configuration. However, computing all steps is computationally expensive and may be unnecessary. We conduct an empirical study along two design axes: (i) distinguishability, the extent to which a metric distinguishes different target settings, and (ii) computational efficiency, the cost of computing the metric from target demonstrations.

We define distinguishability as the ratio of between-task to within-task variance; higher scores indicate that the metric varies more across tasks than across demonstrations within a task. Computational efficiency is naively evaluated by the step index because exact nn forward passes are required to compute step nn. As shown in Table 1, FlowEval metrics on Pi-05-Base and DreamZero-DROID achieved distinguishability at step 1 exceeding 90% of the maximum across all steps, suggesting that a single step is empirically sufficient when accounting for computational cost. We thus only take the first step for future analysis. Full results of distinguishability scores are included in Table 6 and Table 5.

ModelMetricStep 1BestRelative
Pi05-BaseCosine0.8680.88598.0%
Pi05-BaseVelocity Error1.0971.10499.4%
DreamZero-DROIDAction Cosine0.4550.49891.3%
DreamZero-DROIDAction Velocity Error0.2440.26791.3%
DreamZero-DROIDVideo Cosine0.7991.17068.3%
DreamZero-DROIDVideo Velocity Error0.7351.14164.4%

Table 1. Distinguishability score at step 1 compared with the best value observed across all steps. Relative performance is computed as 100×100 \times step 1 / Best. Evaluated on Robocasa365 target set.

Step0123456789
Cosine0.8160.8680.8620.8750.8860.8690.8580.8470.8360.833
VE1.071.101.101.101.101.101.101.101.101.09

Table 6. Pi-05 10 diffusion steps

Experiments

Predicting Unseen Deployments

We first test whether FLOWEVAL reflects how well a deployment’s required behavior is supported by the pretrained model. Our hypothesis is that successful demonstrations from a seen deployment should induce less predictive disagreement than demonstrations from an unseen deployment. Although scores may be noisy for individual trajectories, we expect this difference to emerge at the distribution level.

We test this hypothesis across three vision-language-action models: Pi-05, GR00T, MolmoAct2, and one world-action model: DreamZero, on deployment settings whose relationship to pretraining is known. For each model, we compare demonstrations from a deployment setting represented during pretraining (seen) with demonstrations from a deployment setting absent from pretraining (unseen).

Datasets. We compute FlowEval on 7 datasets across real-world and simulation, multiple action representation spaces, and three embodiments: Franka, Bimanual OpenArm, and Bimanual YAM. We collected the Franka and OpenArm datasets ourselves, and additionally use 5 internet datasets: DROID [44], Robocasa365 [45], MolmoAct2-Bimanual-YAM [7], LIBERO [46], and RoboSet [47]. The exact action representation and checkpoint configurations are discussed in Appendix § 7.§ 3. Note that FLOWEVAL can be computed between different action representations. However, doing so confounds unseen deployment with a change in action representation alone and does not test the other factors of interest. We therefore restrict the main analysis to model–deployment pairs with matching action spaces between pretraining and evaluation. We additionally report cross-representation results in Appendix § 7.§ 3. As expected, these pairs exhibit significantly higher velocity error and lower cosine similarity than pairs from seen deployments.

In-PretrainingDROIDRobocasaYAMLIBERORobosetOpenArmFranka
Pi-05NoNoNoNoNoNoNo
GR00T-N1.6YesYesNoNoNoNoNo
MolmoAct2YesNoYesYesNoNoNo
DreamZeroYesNoNoNoNoNoNo

Table 2. The full set of model–dataset pairs on which FlowEval metrics are evaluated, along with whether each dataset was included in model pretraining. Information about those datasets is based on model releases [2, 13, 7, 8], and, where applicable, company blogs [41, 42, 43]. ‘No’ indicates that we found no evidence that the dataset was included in pretraining, although the authors did not explicitly state that it was excluded. ‘No’ indicates that the authors explicitly stated that the dataset was excluded, or the pretraining dataset is publicly released and does not contain that dataset.

FlowEval results across various models and action representations

Figure 1. FlowEval results across various models and action representations.

We test whether the seen and unseen deployments are separated using a one-sided two-sample Kolmogorov–Smirnov test. For velocity error, the alternative tests whether target distribution lean toward larger errors; for cosine similarity, it tests whether target distribution lean toward smaller similarities. Across Pi-05, GR00T, MolmoAct2, and DreamZero, the corresponding distributions are significantly separated in this direction. The separation is also visible at the distribution level: trajectories from the pretraining setting extend closer to the limiting values of zero velocity error and unit cosine similarity, whereas target trajectories do not reach the same favorable tail.

As a control, we evaluate new demonstrations excluded from pretraining but collected under the same seen settings. FLOWEVAL scores of these demonstrations cluster with the seen deployment and remain distinct from the unseen deployments (Figure 8). Because RoboCasa is the only public in-pretraining dataset that provides such held-out trajectories, this control is limited to RoboCasa (Appendix § 7.§ 3). Together, these results support FlowEval’s ability to identify unseen deployments and provide preliminary evidence that it recognizes seen deployments from new demonstrations.

Additional results of FLOWEVAL rating unseen deployment settings

Figure 8. Additional results of FLOWEVAL rating unseen deployment settings.

Predicting downstream adaptation

We show that FLOWEVAL metrics are capable of predicting how performance scales with data under deployment shift. To determine whether the scaling behavior simply comes from more data or is specifically conditioned on deployment shift, we experiment with two groups of finetuning where (1) the deployment settings is genuinely absent from the model’s pretraining (2) the deployment settings is included in the model’s pretraining, but new demonstrations for the settings are available.

We use the RoboCasa365 dataset [48], which is built for studying downstream finetuning behavior and thus provide us with clearly separated pretraining and held-out datasets. Among widely used robot manipulation benchmarks [49, 50, 27, 51, 52, 53], RoboCasa365 uniquely provides the scale and a pretraining/target split over the same task set that this comparison requires, without confounding deployment shift with differences in semantic-level planning. We use all shared atomic-seen manipulation tasks between the pretrain set and target set, so that we are able to finetune on held-out demonstrations. For deployment settings absent from pretraining, we finetune Pi-05 base and GR00T-N1.5, neither of which was pretrained on RoboCasa or RoboCasa365. For settings represented in pretraining, we finetune Pi-05-RoboCasa [48] and GR00T-N1.6, both of which were pretrained on RoboCasa365.

For each task, we compute FLOWEVAL on the pretrained model before target-domain adaptation, and then finetune with increasing amounts of target data. Following RoboCasa365 [48], we use 50, 150, and 500 demonstrations per task. Each regression observation corresponds to one task–demonstration-count pair, with success rate given by the highest observed checkpoint performance during that finetuning run and the task-level FLOWEVAL score fixed across data budgets. This design tests whether the initial model–domain relationship measured by FLOWEVAL predicts differences in the response to additional adaptation data. We fit the following model:

R=β0+β1L+β2m+β3Lm+ϵ,R = \beta_0 + \beta_1 L + \beta_2 m + \beta_3 Lm + \epsilon,

where L=log⁡(N)−log⁡(N)‾L = \log(N) - \overline{\log(N)} is the centered log number of demonstrations and m=M−M‾m = M - \overline{M} is the centered pre-adaptation FLOWEVAL metric. The coefficient of interest is (β3)(\beta_3); it tests whether the change in success rate associated with additional demonstrations depends on the initial FLOWEVAL value. We fit the model with ordinary least squares using task-clustered robust covariance and test (H0 ⁣:β3=0)(H_0\!:\beta_3 = 0) with a two-sided Wald test.

Under deployment shift, we observe significant interactions for both Pi-05 and GR00T-N1.5 (Figure 2). For Pi-05, both velocity error and cosine similarity are associated with how success rate changes across the 50, 150, and 500 demonstration conditions. GR00T-N1.5 shows the same relationship over the 50 and 150 demonstration conditions. Across both model families, FLOWEVAL therefore provides information before adaptation about how performance changes as additional target-domain data is provided.

Figure 2. Regression between success rate and metric values, with significances displayed in the plots. All interaction effects are significant. Under the GR00T-N1.5 500-demonstration condition, training instability causes 29.4% of runs to collapse; these runs are excluded from the analysis (Appendix § 7.5.5). Training collapse is negligible in other conditions.

In contrast, we do not observe significant interactions in the control experiments with Pi-05-RoboCasa and GR00T-N1.6, where the deployment setting is already represented during pretraining (Appendix § 7.5.1). This contrast supports the interpretation that the interaction is conditioned on deployment shift. Additional controls further ruled out alternative explanations: Within a model, the interaction persists after accounting for success rate improvement rooms (Appendix § 7.5.3); Across models, we did not find a shared task-difficulty pattern (Appendix § 7.5.2) explaining the correlation.

Notably, the relationship between FLOWEVAL scores and finetuned performance depends on the adaptation setting. When adaptation crosses action spaces, action-representation mismatch introduces an additional source of predictive disagreement beyond prediction error. Lower error therefore need not imply higher finetuned success. This phenomenon is observed for Pi-05 in Figure 2. We also observe that additional demonstrations sometimes reduce performance on GR00T-N1.5, consistent with prior observations [38]. Training configurations may affect the direction and shape of these curves (Appendix § 7.5.4). The significant interactions observed across both model families nevertheless suggest that FLOWEVAL’s predictive relationship with performance–data interaction extends beyond a single model and training configuration.

Figure 3. Relation between performance gain and metric values. 500 - 50 demos success rate for Pi-05, 150 - 50 demos success rate for GR00T-N1.5.

Conclusion and Future Work

We introduced FLOWEVAL, a training-free diagnostic for probing how well a pretrained embodied foundation model fits a target deployment. Across multiple model families and deployment settings, FLOWEVAL exhibits greater predictive agreement on deployment settings represented during pretraining and, under deployment shift, correlates with how performance scales as additional adaptation data is provided.

Future work should test whether these signals generalize across broader model and task families, and whether they can support calibrated predictions of the amount of adaptation data required to reach a target level of performance. More broadly, the same idea may extend from diagnosing how much adaptation remains to diagnosing what kind of prior is missing. Modern embodied models increasingly combine heterogeneous sources of prior knowledge—robot trajectories, human video, simulation, world-model pretraining, and large-scale vision-language representations—whose contributions may differ substantially across downstream behaviors. A natural question is whether model-based diagnostics can estimate how much a particular source of additional pretraining would improve a model’s fit to a target deployment, before actually carrying out that pretraining.

Such an analysis would require controlled pretraining interventions across data sources, model families, and scales, and is therefore beyond the scope of an individual academic study. At sufficient scale, however, it could turn the composition of embodied pretraining into a measurable allocation problem: given a model and a family of target deployments, which additional source of experience most efficiently closes the remaining gap?

Acknowledgments

This research used resources provided through NSF ACCESS allocation CIS250679. We are grateful to Dr. Joseph Campbell for helping us obtain this allocation.

Appendix

Design Choice

Here we further discuss the design choices of FlowEval, including why pure noise is chosen over noisy actions; the first diffusion step is chosen over the rest; and cosine similarity and velocity error are chosen over action MSE.

Velocity Error vs. Validation loss

Validation loss is used to probe performance [38, 27, 28], yet flow matching introduces another complicacy in that the input noise can either be infused by ground truth action or not. We tested both conditions on four real-world Franka tasks, results in Figure 4 and Table. Table 3. Results show that pure noise yields notably more stable results than noisy actions, suggesting that it provides a more stable and reliable evaluation signal.

Figure 4. Cosine similarity per layer, evaluted on Pi-05. Left: noisy actions, right: pure noise.

TaskConditionMeanVariance
Franka ObjectNoisy actions3.1213.390
Franka ObjectPure noise1.7700.504
Franka Object+Noisy actions3.0893.286
Franka Object+Pure noise1.7040.492
Franka Object TwoNoisy actions3.1983.274
Franka Object TwoPure noise1.7750.513
Franka On TopNoisy actions3.1233.017
Franka On TopPure noise1.7620.540

Table 3. Statistics under noisy-action and pure-noise conditions on 4 collected Franka tasks.

Distinguishability Scores

We include the full results used to generate Table 1 in § 3.2. We ran 5 tasks on DreamZero-DROID, 16 diffusion steps, and 17 tasks on Pi-05-base, 10 diffusion steps. The tasks come from

the 17 manipulation tasks within the Robocasa365 target dataset [48]. DreamZero-DROID requires significantly more computing resources than Pi-05, so we were unable to evaluate it on the full set.

Scores for every diffusion step are included in Table 6 and Table 5, while advantage of choosing Action MSE over velocity error and cosine similarity is expressed by the division values in Table 4. Choosing action MSE provides little gain for Pi-05, while causing a significant drop for DreamZero.

ModelAction CosineAction VEAction MSEMSE / CosineMSE / VE
π0.5\pi_{0.5}0.8161.071.101.35×1.03×
DreamZero0.4980.2670.1790.359×0.670×
Step01234567
Cosine (Action)0.4980.4550.3980.3980.3980.3980.3230.323
VE (Action)0.2670.2440.2050.2050.2050.2050.1440.144
Cosine (Video)1.170.7990.7990.7990.7990.7991.071.07
VE (Video)0.9560.7350.7230.7230.7230.7231.141.14
Step89101112131415
Cosine (Action)0.3230.3230.2310.2310.2310.2240.2340.231
VE (Action)0.1440.1440.1300.1300.1300.1580.1700.180
Cosine (Video)1.071.070.8970.8970.8970.8450.8460.850
VE (Video)1.141.140.9060.9060.9060.8300.8300.854

Table 4. Distinguishability score of Action MSE compared with FlowEval Metrics

Comparison with Action MSE {#sec: A.1.3}

Action MSE is a competitive baseline used to determine performance [38]. It requires computing the whole diffusion process and thus can be up to 16 times more computationally inefficient, depending on the model configurations. However, faster computation does not necessarily translate to a better practical signal, raising the question of whether there is sufficient justification for designing FLOWEVAL around the first diffusion step instead of action MSE.

We test from two perspectives (1) whether action MSE provide better distinguishability over a set of tasks compared with cosine similarity and velocity error, and (2) whether using action MSE instead of cosine similarity and velocity error changes our core results. We discovered that choosing action MSE provides little gain for Pi-05, while causing a significant drop for DreamZero.

However, distinguishability alone only lets us select a metric that’s fairly separable (expressive), on top of that we still need practical signals showing the separation is actually reflective of the properties we care about. We therefore further evaluate whether the proposed FLOWEVAL metrics reflect the downstream behavior we ultimately aim to predict. In particular, we show that FLOWEVAL performs comparably to action MSE in correlating with how task success improves as additional adaptation data is provided (Figure 5). Action MSE had a p value of .., cosine similarity.., velocity error..

Regression results for action MSE, cosine similarity, and velocity error

Figure 5. Regression results in § 4.2, recomputed with action MSE (left).

Limitation: Post transformed demos

A reasonable limitation of FLOWEVAL is that it is designed for real human-teleoperated demonstrations, where 1) the evaluated action targets are the actions actually issued during data collection and 2) the demonstration was obtained with an actual rollout. Post-hoc transformed demonstrations do not necessarily satisfy this assumption and therefore may not be valid inputs to the metric. We do not attempt to enumerate every possible transformation. Instead, we present one concrete counterexample in which FLOWEVAL does not apply reliably: demonstrations whose action targets are reconstructed after collection.

We collect the Franka demonstrations using a Cartesian-action controller. Given that Pi-05, MolmoAct2, GR00T-N1.6 and DreamZero pretrained real world Franka data in action spaces of delta joint-position, absolute joint-position, relative joint-position and relative joint-position respectively (Table 7), to evaluate these checkpoints within the same action representation they’re pretrained to predict, we reconstruct joint-space action targets from the Cartesian demonstrations after collection. This reconstruction creates a qualitatively different target from the original demonstration.

CheckpointAction Channel
DreamZero-DROIDRelative joint position
GR00T-N1.6-DROIDRelative joint position
GR00T-N1.5-DROIDAbsolute joint position; joint velocity
GR00T-N1.5Delta Cartesian
Pi-0.5-BaseDelta joint position
Pi-0.5-DROIDJoint velocity
Pi-0.5-RoboCasaDelta Cartesian
MolmoAct2Mixed
MolmoAct2-DROIDAbsolute joint position
MolmoAct2-LIBERODelta Cartesian

Table 7. Action channels the checkpoints are pretrained on [2, 7, 13, 8].

FlowEval results across various models and action representations

Figure 6. FlowEval results across various models and action representations on the reconstructed franka dataset.

We observe exactly this failure mode. On the reconstructed Franka trajectories, the Franka action targets are concentrated close to the hold-position action, causing the DROID reference and Franka deployment to overlap much more strongly than for the other unseen deployments. DreamZero shows no significant separation (p>0.05p > 0.05), while GR00T-N1.6 shows only weaker separation (p<0.05p < 0.05), compared with p<0.001p < 0.001 on the other unseen target deployments. Upon further investigation (Figure 7), we find that DreamZero’s mean full-action error is 1.2313×10−41.2313 \times10^{-4}, whereas simply predicting the hold-position action gives a lower error of 5.6037×10−55.6037 \times10^{-5}. The paired difference is 6.7093×10−56.7093 \times10^{-5}, with a task-bootstrap 95% interval of [4.1594,9.4574]×10−5[4.1594, 9.4574] \times10^{-5}.

Figure 7. DreamZero comparison with hold still baseline on Franka and DROID.

This failure mode is specific to post-hoc reconstruction of the action labels. In ordinary robot-learning pipelines, converting demonstrations to another control interface would typically require recollecting actions under the desired controller, rather than substituting the next measured robot state as the action target. We therefore do not view this result as limiting the use of FLOWEVAL on ordinary deployment adaptations. Instead, it shows that FLOWEVAL should not be applied to post-reconstructed action targets without verifying that the reconstruction preserves the demonstrated control signal. More broadly, this example also illustrates why post-hoc reconstructed targets of this form can be unsuitable supervision for robot learning: they can collapse toward nearly stationary achieved states even when the original demonstration contains meaningful control commands.

Predicting unseen deployments

Seen deployments always have the same embodiment and action representation as one of the pretrained checkpoints. If an unseen deployment has the same embodiment and action representation as one of the pretrained checkpoints already support, we select the corresponding checkpoint and action representation space (Table 7). If an unseen deployment does not match any of the configurations, we use the action representation mapping rule specified in Appendix. A.4.

Statistical tests

Results for both matched and mismatched action representation spaces are shown in 8. The matched condition is a subset reported in the main paper (§ 4.1). In all cases, unseen deployments are statistically distinguishable (p<0.001p < 0.001). We report the p values of the one-sided two-sample Kolmogorov–Smirnov (KS) test in Table 8.

CheckpointSeen referenceUnseen deploymentKS statisticpp-valueSig.
Velocity error
Pi-0.5-RoboCasaRoboCasa-pretrainDROID0.9403.88×10−1143.88 \times10^{-114}***
Pi-0.5-RoboCasaRoboCasa-pretrainMolmoAct2-YAM1.0002.57×10−1122.57 \times10^{-112}***
Pi-0.5-RoboCasaRoboCasa-pretrainOpenArm1.0004.05×10−1624.05 \times10^{-162}***
Pi-0.5-LIBEROLIBERORoboCasa-target0.7756.08×10−2296.08 \times10^{-229}***
MolmoAct2MolmoAct2-YAMRoboCasa-pretrain1.0005.24×10−1125.24 \times10^{-112}***
MolmoAct2MolmoAct2-YAMOpenArm1.0002.42×10−362.42 \times10^{-36}***
MolmoAct2-DROIDDROIDRoboCasa-target1.0001.57×10−1401.57 \times10^{-140}***
MolmoAct2-LIBEROLIBERORoboCasa-target0.8251.29×10−2591.29 \times10^{-259}***
GR00T-N1.6RoboCasa-pretrainMolmoAct2-YAM0.9718.80×10−738.80 \times10^{-73}***
GR00T-N1.6RoboCasa-pretrainOpenArm0.9732.40×10−1102.40 \times10^{-110}***
GR00T-N1.6-DROIDDROIDLIBERO0.7437.24×10−517.24 \times10^{-51}***
GR00T-N1.6-DROIDDROIDRoboSet0.4383.93×10−133.93 \times10^{-13}***
GR00T-N1.6-DROIDDROIDRoboCasa-target0.6002.76×10−312.76 \times10^{-31}***
DreamZero-DROIDDROIDRoboSet0.9873.00×10−733.00 \times10^{-73}***
DreamZero-DROIDDROIDRoboCasa-target1.0005.97×10−275.97 \times10^{-27}***
Cosine similarity
Pi-0.5-RoboCasaRoboCasa-pretrainRoboCasa-target0.5669.89×10−1989.89 \times10^{-198}***
Pi-0.5-RoboCasaRoboCasa-pretrainDROID0.9881.23×10−1611.23 \times10^{-161}***
Pi-0.5-RoboCasaRoboCasa-pretrainMolmoAct2-YAM1.0001.31×10−1101.31 \times10^{-110}***
Pi-0.5-RoboCasaRoboCasa-pretrainOpenArm1.0003.24×10−1603.24 \times10^{-160}***
Pi-0.5-LIBEROLIBERORoboCasa-target0.7792.95×10−2312.95 \times10^{-231}***
MolmoAct2MolmoAct2-YAMRoboCasa-pretrain1.0005.24×10−1125.24 \times10^{-112}***
MolmoAct2MolmoAct2-YAMOpenArm1.0002.42×10−362.42 \times10^{-36}***
MolmoAct2-DROIDDROIDRoboCasa-target1.0001.57×10−1401.57 \times10^{-140}***
MolmoAct2-LIBEROLIBERORoboCasa-target0.8281.26×10−2611.26 \times10^{-261}***
GR00T-N1.6RoboCasa-pretrainMolmoAct2-YAM0.9742.60×10−742.60 \times10^{-74}***
GR00T-N1.6RoboCasa-pretrainOpenArm0.9763.27×10−1133.27 \times10^{-113}***
GR00T-N1.6-DROIDDROIDLIBERO0.7378.57×10−508.57 \times10^{-50}***
GR00T-N1.6-DROIDDROIDRoboSet0.4167.02×10−127.02 \times10^{-12}***
GR00T-N1.6-DROIDDROIDRoboCasa-target0.5931.68×10−301.68 \times10^{-30}***
DreamZero-DROIDDROIDRoboSet0.9873.00×10−733.00 \times10^{-73}***
DreamZero-DROIDDROIDRoboCasa-target1.0005.97×10−275.97 \times10^{-27}***

Table 8. Two-sample Kolmogorov–Smirnov tests comparing FLOWEVAL score distributions on seen-reference and unseen datasets. All tested unseen deployments are statistically distinguishable from their corresponding seen-reference distributions.

Action Representation Mapping

When comparing a pretrained model with a deployment setting that uses a different action representation, the model and dataset dimensions may not align directly. In such cases, we follow the mapping rules below.

Some mappings may appear unintuitive because the paired dimensions do not share the same physical units. However, the goal of FLOWEVAL is to probe the adaptation gap between the model prediction and the downstream ground truth. For example, if a model is pretrained to predict joint positions while the downstream objective uses joint velocities, adaptation must still learn the corresponding distribution shift. We therefore pair joint positions with joint velocities in such cases.

Semantic mapping is used to identify the lowest-friction correspondence for learning this distribution shift. It does not eliminate or override the shift itself; rather, it provides the simplest correspondence through which that shift can be learned. Accordingly, gripper dimensions are mapped to gripper dimensions, arm dimensions to arm dimensions, and base dimensions to base dimensions.

  1. Valid dataset dimensions determine the dimensions involved in the loss computation.

  • This dimension in the dataset should be valid and non-trivial. Examples include joint positions or gripper states. Counter-examples include dimensions that remain a constant, such as the torso lift dimension in robocasa.

  • Each valid dataset dimension require a corresponding model output dimension. When len⁡(dataset)>len⁡(valid_model_output)\operatorname{len}(\text{dataset}) > \operatorname{len}(\text{valid\_model\_output}), where valid_model_output means the dimensions in the model that’s activated (actually trained with much data), always take the ’unused’ next model output dimensions to fill in the slot. When len⁡(dataset)<len⁡(valid_model_output)\operatorname{len}(\text{dataset}) < \operatorname{len}(\text{valid\_model\_output}), discard some model outputs so that they map one by one.

  1. Same semantic dimensions should map to each other.

  • Grippers should be mapped with grippers, arms should be mapped with arms, and bases should be mapped with bases.

  • For bimanual to single arm mapping, map the single arm to one of the bimanual arms.

  1. Arms should be mapped to arms.

  • Dataset dimensions decide the dimensions involved: If dataset has 6D arm and the model has 7D arm, discard the last arm dim in model and only include 6 dims in the loss computation; if dataset has 7D arm and the model has 6D arm, use the next ’unused’ dimension (note that this may not be the immediate following dimension, since the model may have other outputs e.g. base/gripper after the arm. It should be the next ’unused’ not associated with a semantic under the config) to map with the last arm dim in the dataset and include 7 dims in the loss computation.

Predicting downstream adaptation

Control 1: Deployment shift

In § 4.2 we observe significant interactions between success rate gain and how much additional data is provided, indicating that FLOWEVAL provides useful signal about how performance in an unseen deployment scales with additional data. This control examines whether this signal reflects a generic scaling effect or is specific to deployment shift. To this end, we additionally finetune Pi05-RoboCasa and GR00T-N1.6 (without shift) and compare their results with Pi05-base and GR00T-N1.5 (with shift) on the RoboCasa tasks. All models use 50, 150, and 500 demonstrations except GR00T-N1.5, for which we use 50 and 150 (Appendix. 9).

ShiftModelDemosCosine similarityVelocity error / loss
WithPi05-base50/150/500+0.474 (.0047)−0.0382 (.0093)
WithGR00T-N1.550/150−5.537 (1.81 × 10−810^{-8})+2.363 (3.12 × 10−810^{-8})
WithoutPi05-RoboCasa50/150/500+0.275 (.3112)−0.0382 (.1056)
WithoutGR00T-N1.650/150/500+0.266 (.4521)−0.1102 (.4464)

Table 9. Initial-metric × demonstration-budget interaction coefficients, with pp-values in parentheses. Bold denotes p<.05p < .05.

Both metrics show significant interactions with demonstration budget under deployment shift, whereas neither is significant without shift. These results support the interpretation that the interaction arises from deployment shift rather than a generic scaling effect.

Deployment shiftModelCosine similarityVelocity error / loss
WithPi05-base+0.474 (.0047)−0.0382 (.0093)
WithGR00T-N1.5−0.0073 (.9964)−0.0258 (.9699)
WithoutPi05-RoboCasa+0.275 (.3112)−0.0382 (.1056)
WithoutGR00T-N1.6+0.266 (.4521)−0.1102 (.4464)

Table 10. Initial-metric × demonstration-budget interaction coefficients, with pp-values in parentheses; bold denotes p<.05p < .05. Positive cosine coefficients associate higher initial cosine similarity with larger gains; negative loss coefficients associate lower initial loss with larger gains.

Control 2: Shared Task Effects

One alternative explanation for the observed interaction is shared task effects instead of model-deployment fit. If some tasks are consistently easier to learn, and these differences in task learnability align with FLOWEVAL scores, they could induce a correlation with scaling. In short, we want to distinguish whether FLOWEVAL measures a merely task level effect or a model-deployment interaction effect.

We test this possibility through cross-model task consistency. We compare Pi05-Base, Pi05-RoboCasa, GR00T-N1.5, and GR00T-N1.6. We compute pairwise Pearson correlations across shared tasks, with Holm correction across the six model pairs for each outcome. We additionally test task effects on success after accounting for model using a partial F-test, and test cross-model metric consistency using Bartlett’s test of sphericity.

FLOWEVAL scores show limited evidence of cross-model task consistency. No pairwise cosine-similarity correlations remain significant after Holm correction, and only one velocity-error correlation does, between Pi05-Base and Pi05-RoboCasa (r=0.634r = 0.634, adjusted p=0.0374p = 0.0374). Bartlett’s tests do not reject identity for either metric’s cross-model correlation matrix (p=0.0854p = 0.0854 for cosine similarity and p=0.0945p = 0.0945 for velocity error). These results provide little evidence that FLOWEVAL consistently ranks the same tasks similarly across models, a pattern compatible with model–task-specific variation. Therefore, FLOWEVAL reasonably measures the model-deployment interaction effect we hypothesized.

Cross-model pairwise resultsGlobal test
Cosine similarity0/6 pairs after Holm correctionBartlett p=0.0854p = 0.0854
Velocity error1/6 pairs: Pi05-Base/Pi05-RoboCasa, r=0.634r = 0.634, adjusted p=0.0374p = 0.0374Bartlett p=0.0945p = 0.0945

Table 11. Cross-model task consistency of success and FlowEval metrics. Strong success consistency confirms shared task effects in RoboCasa; corresponding FlowEval consistency would indicate that these effects could induce the FlowEval–performance relationship.

ModelDemosMetricβ3\beta_395% CIpp
Pi05-Base50/150/500Cosine similarity (Original)0.4738[0.1455, 0.8022]0.00468
Cosine similarity (Controlled)0.2595[−0.04683-0.04683, 0.5659]0.09684
Pi05-Base50/150/500Velocity error (Original)−0.03820-0.03820[−0.06699-0.06699, −0.00942-0.00942]0.009283
Velocity error (Controlled)−0.02106-0.02106[−0.04391-0.04391, 0.001801]0.07099
GR00T-N1.550/150Cosine similarity (Original)−5.537-5.537[−7.465-7.465, −3.609-3.609]1.81×10−81.81 \times10^{-8}
Cosine similarity (Controlled)−5.468-5.468[−7.563-7.563, −3.373-3.373]3.12×10−73.12 \times10^{-7}
GR00T-N1.550/150Velocity error (Original)2.363[1.526, 3.200]3.12×10−83.12 \times10^{-8}
Velocity error (Controlled)2.337[1.433, 3.241]4.07×10−74.07 \times10^{-7}

Table 12. Control for baseline success in the FlowEval scaling analysis. Reported coefficients are the FlowEval metric ×log⁡(demos)\times\log(\mathrm{demos}) interaction β3\beta_3. The controlled specification additionally includes 50-demo success S50S_{50} and its interaction with log⁡(demos)\log(\mathrm{demos}). Coefficient magnitudes should be compared before and after adjustment within each metric rather than across metrics.

Control 3: Headroom bias

Another alternative explanation for the observed interaction is baseline success. Because success is bounded at 100%, a task starting at 20% success has more room to improve than a task starting at 90%. If FLOWEVAL is correlated with starting performance, part of its interaction with demonstration count could arise from this difference in available headroom rather than from an association with scaling beyond starting performance.

We therefore control for each task’s measured success at 50 demonstrations and, critically, allow the scaling slope itself to depend on this baseline success. This asks whether FLOWEVAL remains associated with scaling after accounting for the possibility that tasks with different starting success rates have different rates of improvement.

Control specification. For each task, let RR denote success rate at a given demonstration count, L=log⁡(N)L = \log(N) the natural logarithm of the number of demonstrations, mm the task-level step-zero FlowEval metric, and S50S_{50} the task’s success rate with 50 demonstrations. The original regression is

R=β0+β1L+β2m+β3Lm+ϵ,(6)R = \beta_0 + \beta_1 L + \beta_2 m + \beta_3 Lm + \epsilon, \tag*{(6)}

where β3\beta_3 is the FlowEval metric ×\times log-demonstration-count interaction examined in § 4.2.

The baseline-success-controlled regression is

R=β0+β1L+β2m+β3Lm+β4S50+β5LS50+ϵ.(7)R = \beta_0 + \beta_1 L + \beta_2 m + \beta_3 Lm + \beta_4 S_{50} + \beta_5 LS_{50} + \epsilon. \tag*{(7)}

The S50S_{50} term accounts for differences associated with starting success, while LS50LS_{50} allows the relationship between demonstration count and success to vary with starting success. Including only S50S_{50} would therefore not directly address the alternative explanation for the scaling interaction. The comparison of interest is how the FlowEval interaction β3\beta_3 changes after adding both terms.

LL, mm, and S50S_{50} are centered at their cell-level means. The original and controlled regressions use identical observations. No headroom-normalized gain or separate gain regression is introduced, so this analysis directly tests the original interaction under an additional baseline-success control.

Results. For Pi05-Base, controlling for baseline success reduces the cosine and velocity-error interactions by 45.2% and 44.9%, respectively, while preserving their directions and approximately 55% of their original magnitudes. The negative LS50LS_{50} coefficients (p=0.00184p = 0.00184–0.004350.00435) indicate that tasks with higher baseline success tend to gain less from additional demonstrations. This suggests that a ceiling effect contributes to the observed scaling trend but does not fully explain it.

For GR00T-N1.5 over 50–150 demonstrations, controlling for baseline success leaves the cosine and velocity-error interactions almost unchanged. The cosine coefficient changes from −5.537-5.537 to −5.468-5.468 (1.24%1.24\%), and the velocity-error coefficient changes from 2.3632.363 to 2.3372.337 (1.11%1.11\%).

Training Configurations

We use these training configurations for Pi05-Base, Pi05-Robocasa, GR00T-N1.5, and GR00T-N1.6.

OpenPi Hyperparameters
global_batch_size: 64
train_steps: 30000

optimizer: AdamW
beta1: 0.9
beta2: 0.95
epsilon: 1e-8
weight_decay: 1e-6
grad_clip_global_norm: 1.0

lr_schedule: warmup-cosine
warmup_steps: 1000
peak_lr: 2.5e-5
decay_steps: 30000
end_lr: 2.5e-6

ema_decay: 0.99
GR00T Hyperparameters
learning_rate: 1e-4
weight_decay: 1e-5
warmup_ratio: 0.05

global_batch_size: 64
gradient_accumulation_steps: 1

shard_size: 1024
num_shards_per_epoch: 100000
episode_sampling_rate: 0.1

optim: adamw_torch

tune_llm: true
tune_visual: true
tune_projector: true
tune_diffusion_model: true

state_dropout_prob: 0.0
random_rotation_angle: null
extra_augmentation_config: null
load_bf16: false
reproject_vision: false
eagle_collator: true
backbone_trainable_params_fp32: true
use_relative_action: true

Training Collapse in GR00T-N1.5 with 500 Demonstrations

Loss Spikes. We train each GR00T-N1.5 finetuning run for 30k steps to allow sufficient optimization. Around 5k steps, when the learning-rate schedule plateaus, we observe loss spikes and occasional training instability. Performance collapse affects 5 of 17 runs (29.4%) at 500 demonstrations (Figure 9), compared with one GR00T-N1.5 run at 150 demonstrations and one or two runs across all demonstration counts for the Pi-05 variants (Table 13). We therefore exclude the 500-demonstration condition from the primary GR00T-N1.5 regression analysis and report the interaction over 50–150 demonstrations.

ModelDemonstrationsCollapsed runsFraction of runs
GR00T-N1.5500529.4% (5/17)
GR00T-N1.515015.88% (1/17)
GR00T-N1.6 (Action Expert)All00.00% (0/51)
Pi-05-BaseAll11.96% (1/51)
Pi-05-RoboCasaAll23.92% (2/51)

Table 13. Runs exhibiting persistent or sudden performance collapse.

Performance curves for GR00T-N1.5 finetuning across tasks and demonstration counts

Figure 9. Performance curves for GR00T-N1.5 finetuning. Training collapse occurs on task 18 with 150 demonstrations and on tasks 2, 10, 12, 16, and 17 with 500 demonstrations.

References

  1. [1]M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfield, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. Openvla: An open-source vision-language-action model, 2024. URL https://arxiv.org/abs/2406.09246.
  2. [2]P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, M. Y. Galliker, D. Ghosh, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, D. LeBlanc, S. Levine, A. Li-Bell, M. Muthukuri, S. Nair, K. Pertsch, A. Z. Ren, L. X. Shi, L. Smith, J. T. Springenberg, K. Stachowicz, J. Tanner, Q. Vuong, H. Walke, A. Walling, H. Wang, L. Yu, and U. Zhilinsky. π₀.₅: a vision-language-action model with open-world generalization, 2025. URL https://arxiv.org/abs/2504.16054.
  3. [3]Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, T. Kreiman, Y. Tan, L. Y. Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine. Octo: An open-source generalist robot policy. In Proceedings of Robotics: Science and Systems, Delft, Netherlands, 2024.arxiv.org/abs/2405.12213
  4. [4]E. Collaboration, A. O’Neill, A. Rehman, A. Gupta, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, A. Tung, A. Bewley, A. Herzog, A. Irpan, A. Khazatsky, A. Rai, A. Gupta, A. Wang, A. Kolobov, A. Singh, A. Garg, A. Kembhavi, A. Xie, A. Brohan, A. Raffin, A. Sharma, A. Yavary, A. Jain, A. Balakrishna, A. Wahid, B. Burgess-Limerick, B. Kim, B. Schölkopf, B. Wulfe, B. Ichter, C. Lu, C. Xu, C. Le, C. Finn, C. Wang, C. Xu, C. Chi, C. Huang, C. Chan, C. Agia, C. Pan, C. Fu, C. Devin, D. Xu, D. Morton, D. Driess, D. Chen, D. Pathak, D. Shah, D. Büchler, D. Jayaraman, D. Kalashnikov, D. Sadigh, E. Johns, E. Foster, F. Liu, F. Ceola, F. Xia, F. Zhao, F. V. Frujeri, F. Stulp, G. Zhou, G. S. Sukhatme, G. Salhotra, G. Yan, G. Feng, G. Schiavi, G. Berseth, G. Kahn, G. Yang, G. Wang, H. Su, H.-S. Fang, H. Shi, H. Bao, H. B. Amor, H. I. Christensen, H. Furuta, H. Bharadhwaj, H. Walke, H. Fang, H. Ha, I. Mordatch, I. Radosavovic, I. Leal, J. Liang, J. Abou-Chakra, J. Kim, J. Drake, J. Peters, J. Schneider, J. Hsu, J. Vakil, J. Bohg, J. Bingham, J. Wu, J. Gao, J. Hu, J. Wu, J. Wu, J. Sun, J. Luo, J. Gu, J. Tan, J. Oh, J. Wu, J. Lu, J. Yang, J. Malik, J. Silvério, J. Hejna, J. Booher, J. Thompson, J. Yang, J. Salvador, J. J. Lim, J. Han, K. Wang, K. Rao, K. Pertsch, K. Hausman, K. Go, K. Gopalakrishnan, K. Goldberg, K. Byrne, K. Oslund, K. Kawaharazuka, K. Black, K. Lin, K. Zhang, K. Ehsani, K. Lekkala, K. Ellis, K. Rana, K. Srinivasan, K. Fang, K. P. Singh, K.-H. Zeng, K. Hatch, K. Hsu, L. Itti, L. Y. Chen, L. Pinto, L. Fei-Fei, L. Tan, L. J. Fan, L. Ott, L. Lee, L. Weihs, M. Chen, M. Lepert, M. Memmel, M. Tomizuka, M. Itkina, M. G. Castro, M. Spero, M. Du, M. Ahn, M. C. Yip, M. Zhang, M. Ding, M. Heo, M. K. Srirama, M. Sharma, M. J. Kim, M. Z. Irshad, N. Kanazawa, N. Hansen, N. Heess, N. J. Joshi, N. Suenderhauf, N. Liu, N. D. Palo, N. M. M. Shafiullah, O. Mees, O. Kroemer, O. Bastani, P. R. Sanketi, P. T. Miller, P. Yin, P. Wohlhart, P. Xu, P. D. Fagan, P. Mitrano, P. Sermanet, P. Abbeel, P. Sundaresan, Q. Chen, Q. Vuong, R. Rafailov, R. Tian, R. Doshi, R. Martín-Martín, R. Bajcsy, R. Scalise, R. Hendrix, R. Lin, R. Qian, R. Zhang, R. Mendonca, R. Shah, R. Hoque, R. Julian, S. Bustamante, S. Kirmani, S. Levine, S. Lin, S. Moore, S. Bahl, S. Dass, S. Sonawani, S. Tulsiani, S. Song, S. Xu, S. Haldar, S. Karamcheti, S. Adebola, S. Guist, S. Nasiriany, S. Schaal, S. Welker, S. Tian, S. Ramamoorthy, S. Dasari, S. Belkhale, S. Park, S. Nair, S. Mirchandani, T. Osa, T. Gupta, T. Harada, T. Matsushima, T. Xiao, T. Kollar, T. Yu, T. Ding, T. Davchev, T. Z. Zhao, T. Armstrong, T. Darrell, T. Chung, V. Jain, V. Kumar, V. Vanhoucke, V. Guizilini, W. Zhan, W. Zhou, W. Burgard, X. Chen, X. Chen, X. Wang, X. Zhu, X. Geng, X. Liu, X. Liangwei, X. Li, Y. Pang, Y. Lu, Y. J. Ma, Y. Kim, Y. Chebotar, Y. Zhou, Y. Zhu, Y. Wu, Y. Xu, Y. Wang, Y. Bisk, Y. Dou, Y. Cho, Y. Lee, Y. Cui, Y. Cao, Y.-H. Wu, Y. Tang, Y. Zhu, Y. Zhang, Y. Jiang, Y. Li, Y. Li, Y. Iwasawa, Y. Matsuo, Z. Ma, Z. Xu, Z. J. Cui, Z. Zhang, Z. Fu, and Z. Lin. Open ex-embodiment: Robotic learning datasets and rt-x models, 2025. URL https://arxiv.org/abs/2310.08864.
  5. [5]M. J. Kim, C. Finn, and P. Liang. Fine-tuning vision-language-action models: Optimizing speed and success, 2025. URL https://arxiv.org/abs/2502.19645.
  6. [6]G. R. Team, A. Abdolmaleki, S. Abernuyan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, A. Balakrishna, N. Batchelor, A. Bewley, J. Bingham, M. Bloesch, K. Boum s alis, P. Brakel, A. Brohan, T. Buschmann, A. Byravan, S. Cabi, K. Caluwaerts, F. Casarini, C. Chan, O. Chang, L. Chappellet-Volpini, J. E. Chen, X. Chen, H.-T. L. Chiang, K. Choromanski, A. Collister, D. B. D’Ambrosio, S. Dasari, T. Davchev, M. K. Dave, C. Devin, N. D. Palo, T. Ding, C. Doersch, A. Dostmohamed, Y. Du, D. Dwidebisi, S. T. Egambaram, M. Elabd, T. Erez, X. Fang, C. Fantacci, C. Fong, E. Frey, C. Fu, R. Gao, M. Giustina, K. Gopalakrishnan, L. Graesser, O. Groth, A. Gupta, R. Hafner, S. Hansen, L. Hasenclever, S. Haves, N. Heess, B. Hernaez, A. Hofer, J. Hsu, L. Huang, S. H. Huang, A. Iscen, M. G. Jacob, D. Jain, S. Jesmonth, A. Jindal, R. Julian, D. Kalashnikov, M. E. Karag ozler, S. Karp, M. Kecman, J. C. Kew, D. Kim, F. Kim, J. Kim, T. Kipf, S. Kirmani, K. Konyushkova, L. Y. Ku, Y. Kuang, T. Lampe, A. Laurens, T. A. Le, I. Leal, A. X. Lee, T.-W. E. Lee, G. Lever, J. Liang, L.-H. Lin, F. Liu, S. Long, C. Lu, S. Maddineni, A. Majumdar, K.-K. Maninis, A. Marmon, S. Martinez, A. H. Michaely, N. Milonopoulos, J. Moore, R. Moreno, M. Neunert, F. Nori, J. Ortiz, K. Oslund, C. Parada, E. Parisotto, A. Paryag, A. Pooley, T. Power, A. Qu aglino, H. Qureshi, R. V. Raju, H. Ran, D. Rao, K. Rao, I. Reid, D. Rendleman, K. Reymann, M. Rivas, F. Romano, Y. Rubanova, P. P. Sampedro, P. R. Sanketi, D. Shah, M. Sharma, K. Shea, M. Shridhar, C. Shu, V. Sindhwani, S. Singh, R. Soricut, R. Sterne ck, I. Storz, R. Surdulescu, J. Tan, J. Thompson, S. Tun yasuvun ako ol, J. Varley, G. Ves om, G. Vezzani, M. B. Villalonga, O. Vinyals, R. Wagner, A. Wahid, S. Welker, P. Wohlhart, C. Wu, M. Wulfmeier, F. Xia, T. Xiao, A. Xie, J. Xie, P. Xu, S. Xu, Y. Xu, Z. Xu, J. Yan, S. Yang, S. Yang, Y. Yang, H. H. Yu, W. Yu, W. Yuan, Y. Yuan, J. Zhang, T. Zhang, Z. Zhang, A. Zhou, G. Zhou, and Y. Zhou. Gemini robotics 1.5: Pushing the frontier of generalist robots with advanced embodied reasoning, thinking, and motion transfer, 2025. URL https://arxiv.org/abs/2510.03342.
  7. [7]H. Fang, J. Duan, D. Clay, S. Wang, S. Liu, W. Huang, X. Fan, W.-C. Tsai, S. Chen, Y. R. Wang, S. Xing, J. Cho, J. S. Park, A. Efte khar, P. Sushko, K. Farley, A. Wadhwa, C. Harrison, W. Han, Y.-C. Lee, E. VanderBilt, R. Hendrix, S. Ellawela, L. Ngoo, J. Chai, Z. Ren, A. Farhadi, D. Fox, and R. Krishna. Molmoact2: Action reasoning models for real-world deployment, 2026. URL https://arxiv.org/abs/2605.02881.
  8. [8]S. Ye, Y. Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y. L. Tan, C. Zhu, J. Xiang, A. Malik, K. Lee, W. Liang, N. Ranawaka, J. Gu, Y. Xu, G. Wang, F. Hu, A. Narayan, J. Bjorck, J. Wang, G. Kim, D. Niu, R. Zheng, Y. Xie, J. Wu, Q. Wang, R. Julian, D. Xu, Y. Du, Y. Chebotar, S. Reed, J. Kautz, Y. Zhu, L. J. Fan, and J. Jang. World action models are zero-shot policies, 2026. URL https://arxiv.org/abs/2602.15922.
  9. [9]R. C. Moore and W. Lewis. Intelligent selection of language model training data. In J. Hajic, S. Carberry, S. Clark, and J. Nivre, editors, Proceedings of the ACL 2010 Conference Short Papers, pages 220–224, Uppsala, Sweden, July 2010. Association for Computational Linguistics. URL https://aclanthology.org/P10-2041/.
  10. [10]S. Gururangan, A. Marasović, S. Sw ayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith. Don’t stop pretraining: Adapt language models to domains and tasks, 2020. URL https://arxiv.org/abs/2004.10964.
  11. [11]S. M. Xie, H. Pham, X. Dong, N. Du, H. Liu, Y. Lu, P. S. Liang, Q. V. Le, T. Ma, and A. W. Yu. Doremi: Optimizing data mixtures speeds up language model pretraining. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Infor- mation Processing Systems, volume 36, pages 69798–69818. Curran Associates, Inc., 2023. doi:10.52202/075280-3059. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/dcba6b...
  12. [12]Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling, 2023. URL https://arxiv.org/abs/2210.02747.
  13. [13]NVIDIA, J. Björck, N. C. Fernando Castañeda, X. Da, R. Ding, L. J. Fan, Y. Fang, D. Fox, F. Hu, S. Huang, J. Jang, Z. Jiang, J. Kautz, K. Kundalia, L. Lao, Z. Li, Z. Lin, K. Lin, G. Liu, E. Llontop, L. Magne, A. Mandlekar, A. Narayan, S. Nasiriany, S. Reed, Y. L. Tan, G. Wang, Z. Wang, J. Wang, Q. Wang, J. Xiang, Y. Xie, Y. Xu, Z. Xu, S. Ye, Z. Yu, A. Zhang, H. Zhang, Y. Zhao, R. Zheng, and Y. Zhu. GROOT N1: An open foundation model for generalist humanoid robots. In ArXiv Preprint, March 2025.
  14. [14]G. Team. Gen-1.5: Embodied foundation models are one-shot learners. Generalist AI Blog, 2026. https://generalistai.com/blog/gen-1.5.
  15. [15]Google DeepMind. Gemini robotics 2. https://deepmind.google/models/gemini-robotics/vla/, 2026. Model webpage.
  16. [16]Figure. Introducing helix 02: Full-body autonomy, 2026. URL https://www.figure.ai/news/helix-02.
  17. [17]S. AI. Introducing s1: In-context learning for robotics. Skild AI Blog, August 2026. URL https://skild.ai/blogs/s1.
  18. [18]G. A. Team. Gene-26.5: Advancing robotic manipulation to human level. Genesis AI Blog, May 2026. URL https://genesis.ai/blog/gene-26-5-advancing-robotic-manipulation-to-human-level.
  19. [19]R. A. Team. Om-1: Frontier robot intelligence, learned firsthand from humans. Reward AI Blog, 2026. https://rewardai.com/blog/OM-1/.
  20. [20]C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals. Understanding deep learning requires rethinking generalization, 2017. URL https://arxiv.org/abs/1611.03530.
  21. [21]C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks, 2017. URL https://arxiv.org/abs/1706.04599.
  22. [22]J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models, 2020. URL https://arxiv.org/abs/2001.08361.
  23. [23]J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, and L. Sifre. Training compute-optimal large language models, 2022. URL https://arxiv.org/abs/2203.15556.
  24. [24]S. Ross, G. J. Gordon, and J. A. Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning, 2011. URL https://arxiv.org/abs/1011.0686.
  25. [25]T. Z. Zhao, V. Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware, 2023. URL https://arxiv.org/abs/2304.13705.
  26. [26]A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, P. Florence, C. Fu, M. G. Arenas, K. Gopalakrishnan, K. Han, K. Hausman, A. Herzog, J. Hsu, B. Ichter, A. Irpan, N. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, I. Leal, L. Lee, T.-W. E. Lee, S. Levine, Y. Lu, H. Michalewski, I. Mordatch, K. Pertsch, K. Rao, K. Reymann, M. Ryoo, G. Salazar, P. Sanketi, P. Sermanet, J. Singh, A. Singh, R. Soricut, H. Tran, V. Vanhoucke, Q. Vuong, A. Wahid, S. Welker, P. Wohlhart, J. Wu, F. Xia, T. Xiao, P. Xu, S. Xu, T. Yu, and B. Zitkovich. R2: Vision-language-action models transfer web knowledge to robotic control, 2023. URL https://arxiv.org/abs/2307.15818.
  27. [27]X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kirmani, S. Levine, J. Wu, C. Finn, H. Su, Q. Vuong, and T. Xiao. Evaluating real-world robot manipulation policies in simulation, 2024. URL https://arxiv.org/abs/2405.05941.
  28. [28]Z. Zhou, P. Atreya, Y. L. Tan, K. Pertsch, and S. Levine. Autoeval: Autonomous evaluation of generalist robot manipulation policies in the real world, 2025. URL https://arxiv.org/abs/2503.24278.
  29. [29]C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios. Neural spline flows, 2019. URL https://arxiv.org/abs/1906.04032.
  30. [30]G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1–64, 2021. URL http://jmlr.org/papers/v22/19-1028.html.
  31. [31]E. Nalisnick, A. Matsukawa, Y. W. Teh, D. Gorur, and B. Lakshminarayanan. Do deep generative models know what they don’t know?, 2019. URL https://arxiv.org/abs/1810.09136.
  32. [32]Y. Han, H. Sun, and T. Lin. When can conditional flow matching replace pointwise negative log-likelihood?, 2026. URL https://arxiv.org/abs/2608.28010.
  33. [33]I. Wenger, H. Ebel, and P. Eberhard. Anomalously acting agents: the deployment problem. Multibody System Dynamics, 64(3):465–483, 2025. ISSN 1573-272X. doi:10.1007/s11044-024-09993-1. URL https://doi.org/10.1007/s11044-024-09993-1.
  34. [34]J. Zhang, C. Zhang, W. Wang, and B. Jing. Constrained policy optimization with explicit behavior density for offline reinforcement learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 5616–5630. Curran Associates, Inc., 2023. doi:10.52202/075280-0245. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/11e1900e680f5fe1893a8e27362dbe2c-Paper-Conference.pdf.DOI
  35. [35]J. Chae, J. Park, Y. Shin, G. Kim, S. Han, and Y. Sung. Flow actor-critic for offline reinforcement learning, 2026. URL https://arxiv.org/abs/2602.18015.
  36. [36]L. Zha, A. J. Hancock, M. Zhang, T. Yin, Y. Huang, D. Shah, A. Z. Ren, and A. Majumdar. Lap: Language-action pre-training enables zero-shot cross-embodiment transfer, 2026. URL https://arxiv.org/abs/2602.10556.
  37. [37]Robocurve. GR00T N1.7: SO-101 fine-tune on MolmoAct2 community data. Hugging Face, 2026. URL https://huggingface.co/robocurve/gr00t-n1.7-so101-molmoact2. Accessed: 2026-09-15.
  38. [38]S. Park. Behavioral cloning mystery. https://seohong.me/blog/behavioral-cloning-mystery/, Aug. 2026. Blog post.arxiv.org/abs/2610.07056
  39. [39]A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y. Narang, L. Fan, Y. Zhu, and D. Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations, 2023. URL https://arxiv.org/abs/2310.17596.
  40. [40]Z. Xue, S. Deng, Z. Chen, Y. Wang, Z. Yuan, and H. Xu. Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning, 2025. URL https://arxiv.org/abs/2502.16932.
  41. [41]K. Black et al. π0.5: A VLA with open-world generalization. Physical Intelligence, Apr. 2025. URL https://www.pi.website/blog/pi05. Published April 22, 2025. Accessed: 2026-10-04.
  42. [42]GEAR Team et al. GR00T N1.6: An improved open foundation model for generalist humanoid robots. NVIDIA Research, Dec. 2025. URL https://research.nvidia.com/labs/gear/gr00t-n1_6/. Published December 15, 2025. Accessed: 2026-10-04.
  43. [43]S. Ye et al. DreamZero: World action models are zero-shot policies, 2026. URL https://dreamzero0.github.io/. Accessed: 2026-10-04.arxiv.org/abs/2602.15922
  44. [44]A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, P. D. Fagan, J. Hejna, M. Itkina, M. Lepert, Y. J. Ma, P. T. Miller, J. Wu, S. Belkhale, S. Dass, H. Ha, A. Jain, A. Lee, Y. Lee, M. Memmel, S. Park, I. Radosavovic, K. Wang, A. Zhan, K. Black, C. Chi, K. B. Hatch, S. Lin, J. Lu, J. Mercat, A. Rehman, P. R. Sanketi, A. Sharma, C. Simpson, Q. Vuong, H. R. Walke, B. Wulfe, T. Xiao, J. H. Yang, A. Yavary, T. Z. Zhao, C. Agia, R. Baijal, M. G. Castro, D. Chen, Q. Chen, T. Chung, J. Drake, E. P. Foster, J. Gao, V. Guizilini, D. A. Herrera, M. Heo, K. Hsu, J. Hu, M. Z. Irshad, D. Jackson, C. Le, Y. Li, K. Lin, R. Lin, Z. Ma, A. Maddukuri, S. Mirchandani, D. Morton, T. Nguyen, A. O’Neill, R. Scalise, D. Seale, W. Son, S. Tian, E. Tran, A. E. Wang, Y. Wu, A. Xie, J. Yang, P. Yin, Y. Zhang, O. Bastani, G. Berseth, J. Bohg, K. Goldberg, A. Gupta, A. Gupta, D. Jayaraman, J. J. Lim, J. Malik, R. Martín-Martín, S. Ramamoorthy, D. Sadigh, S. Song, J. Wu, M. C. Yip, Y. Zhu, T. Kollar, S. Levine, and C. Finn. Droid: A large-scale in-the-wild robot manipulation dataset, 2025. URL https://arxiv.org/abs/2403.12945.
  45. [45]S. Nasiriany, S. Nasiriany, A. Maddukuri, and Y. Zhu. Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots, 2026. URL https://arxiv.org/abs/2603.04356.
  46. [46]B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023. URL https://arxiv.org/abs/2306.03310.
  47. [47]H. Bharadhwaj, J. Vakil, M. Sharma, A. Gupta, S. Tulsiani, and V. Kumar. Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking, 2023. URL https://arxiv.org/abs/2309.01918.
  48. [48]S. Nasiriany, S. Nasiriany, A. Maddukuri, and Y. Zhu. Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots, 2026. URL https://arxiv.org/abs/2603.04356.
  49. [49]B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023. URL https://arxiv.org/abs/2306.03310.
  50. [50]O. Mees, L. Hermann, E. Rosete-Beas, and W. Burgard. Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks, 2022. URL https://arxiv.org/abs/2112.03227.
  51. [51]C. Li, R. Zhang, J. Wong, C. Gokmen, S. Srivastava, R. Martín-Martín, C. Wang, G. Levine, W. Ai, B. Martinez, H. Yin, M. Lingelbach, M. Hwang, A. Hiranaka, S. Garlanka, A. Aydin, S. Lee, J. Sun, M. Anvari, M. Sharma, D. Bansal, S. Hunter, K.-Y. Kim, A. Lou, C. R. Matthews, I. Villa-Renteria, J. H. Tang, C. Tang, F. Xia, Y. Li, S. Savarese, H. Gweon, C. K. Liu, J. Wu, and L. Fei-Fei. Behavior-1k: A human-centered, embodied ai benchmark with 1,000 everyday activities and realistic simulation. arXiv preprint arXiv:2403.09227, 2024.arxiv.org/abs/2403.09227
  52. [52]S. James, Z. Ma, D. R. Arrojo, and A. J. Davison. Rilbench: The robot learning benchmark & learning environment, 2019. URL https://arxiv.org/abs/1909.12271.
  53. [53]S. Zhang, Z. Xu, P. Liu, X. Yu, Y. Li, Q. Gao, Z. Fei, Z. Yin, Z. Wu, Y.-G. Jiang, and X. Qiu. Vlabench: A large-scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks, 2024. URL https://arxiv.org/abs/2412.18194.

Paper details

vlnc:W_4upr7msq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 2

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents