WorksComputer Science
GeoMind: A Benchmark for Spatial Understanding in Robotic Manipulation
Abstract
Robotic manipulation has advanced rapidly in mapping visual observations and language instructions to actions. However, reliable manipulation in open-world environments requires robust spatial understanding: the ability to identify targets in complex layouts (spatial reasoning) and adapt to scenes that continuously evolve through interaction (spatial memory). Yet existing embodied benchmarks fall short in evaluating spatial understanding: they often introduce strong visual cues that bypass spatial reasoning, and focus on static scenes that offer little insight into an agent's capacity for spatial memory. To address this gap, we introduce GeoMind, a systematic benchmark for structured spatial understanding through tabletop manipulation. GeoMind comprises 54 evaluated task variants organized around spatial reasoning and spatial memory. It also incorporates a scalable pipeline that automates the generation and annotation of task instances with verified targets and executable interactions. Extensive evaluations across diverse embodied agents reveal their consistent weaknesses in spatial understanding. Our findings highlight that these agents struggle to track identities after spatial changes and convert relational or geometric constraints into executable actions. We release the GeoMind benchmark at https://github.com/Notfound-JH/GEOMIND-benchmark.git.
Introduction
Advances in embodied intelligence have enabled robots to translate visual observations and language instructions into manipulation actions [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]. However, reliable manipulation often requires more than recognizing individual objects: robots must reason about spatial relationships to identify targets and maintain task-relevant spatial information as scenes change [15, 16, 17]. For example, a robot may be asked to pick up a block based on its position relative to other objects, or to pick up the same block after observing several visually similar blocks being rearranged. While the former requires reasoning about the current spatial configuration, the latter additionally requires tracking object identity across scene changes. Such tasks highlight the importance of spatial understanding for reliable manipulation.
Existing embodied benchmarks [18, 19, 20, 21, 22, 23, 24] have made substantial progress in evaluating language-guided manipulation and generalization. However, overall task success provides limited insight into the spatial capabilities underlying an agent’s behavior. Distinctive visual or semantic cues may allow agents to identify targets without reasoning about spatial relationships. Similarly, tasks involving scene changes may be solved from the current observation without tracking previous spatial states. Conversely, manipulation failures may arise from execution errors even when the intended spatial target has been correctly identified. These limitations motivate a diagnostic evaluation that makes spatial requirements explicit and examines agents’ spatial capabilities beyond overall task success.
To enable a more targeted assessment of spatial understanding, we focus on two key dimensions, spatial reasoning and spatial memory, drawing on the cognitive distinction between reasoning about space and maintaining spatial information over time [25]. Spatial reasoning concerns identifying target objects and inferring goal locations or regions from spatial relationships and structured layouts, while spatial memory concerns tracking object identities and updating spatial states as scenes change. These dimensions specify the spatial capabilities that each task is designed to assess.
We introduce GEOMIND, a benchmark for systematically evaluating spatial understanding through tabletop robotic manipulation. It comprises six task families: (1) target identification, (2) relational grounding, (3) constraint composition, and (4) geometric inference for spatial reasoning, and (5) scene updating and (6) identity tracking for spatial memory. Together, these families span 54 evaluated task variants: 20 Base, 7 Semantic, 7 Visual, and 20 Execution variants. Tasks are designed around structured object layouts and controlled scene changes, with explicit spatial rules for determining target objects or goal locations. Many base tasks feature visually identical or near-identical objects to reduce reliance on appearance cues. To enable reliable evaluation of these diverse spatial objectives, we develop an agent-assisted pipeline that translates spatial constraints into executable grounding code, computes ground-truth spatial targets, and verifies the resulting task instances through simulation.
We evaluate generalist and spatially enhanced visuomotor policies alongside modular planning methods on GEOMIND under a unified evaluation protocol. GEOMIND combines a base task suite (GEOMIND-BASE) with controlled semantic, visual, and execution augmentations (GEOMIND-AUG) to probe robustness. Beyond task success rates, we use diagnostic spatial metrics and comparisons with an oracle policy to examine spatial grounding and execution behavior. Our results show that current methods can solve some basic grounding tasks but struggle to reason about structured spatial relationships and track spatial states across scene changes. Performance is sensitive to semantic augmentations, while visual and execution perturbations have more model-dependent effects.
The main contributions of this work are threefold: (1) A systematic benchmark for structured spatial understanding. We introduce GEOMIND, which organizes spatial understanding into six task families and provides controlled base and augmented evaluations. (2) An agent-assisted spatial grounding and verification pipeline. We develop a pipeline that translates spatial constraints into executable grounding programs, derives ground-truth targets, and verifies task instances through simulation. (3) A comprehensive diagnostic evaluation. We evaluate generalist and spatially enhanced policies alongside modular planning methods using diagnostic spatial metrics and oracle comparisons, revealing limitations in spatial reasoning, memory, and robustness.
Related Work
General Embodied Manipulation Benchmarks. Prior embodied manipulation benchmarks have progressively advanced standardized evaluation of language-conditioned manipulation, long-horizon control, and generalization across diverse tasks and environments [18, 19, 20, 22, 24, 26, 27, 28, 29]. VIMA includes history-dependent rearrangement and visual constraint satisfaction [23], while GENMANIP uses task-oriented scene graphs for constrained scene construction [30]. These benchmarks already cover several spatial requirements considered here; aggregate task success alone, however, does not distinguish spatial goal grounding errors from execution failures. More specialized benchmarks investigate memory-dependent manipulation. RMBench [31] evaluates tasks with varying memory complexity, while RoboMME [32] examines temporal, spatial, object, and procedural memory. MEMOBench [33] further introduces process-level evaluations of memory storage, updating, and compression. Together, these benchmarks examine both memory demands and the processes underlying memory-dependent manipulation. GEOMIND complements them by systematically assessing spatial reasoning and spatial memory through capability-specific manipulation tasks defined by explicit spatial constraints and controlled scene changes. Table 1 compares selected task requirements and evaluation designs. A checkmark denotes an explicitly targeted task or evaluation subset with reported results; a cross denotes that no explicit evaluation was identified in the reviewed protocol, not that the capability is absent.
| Benchmark | Design Structure Generation | Spatial Memory Scene Updating | Spatial Memory Identity Tracking | Spatial Reasoning Target Identification | Spatial Reasoning Relational Grounding | Spatial Reasoning Constraint Composition | Spatial Reasoning Geometric Inference | Evaluation Diagnostic Metrics |
| VIMA [23] | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ |
| GENMANIP [30] | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
| RoboMME [32] | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ |
| MEMOBench [33] | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ |
| RoboSPA [34] | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ |
| BEHAVIOR-Spatial [35] | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ |
| VABench [36] | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ |
| GEOMIND | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Table 1. Comparison of spatial capabilities and benchmark design. ✓: explicitly evaluated; ✗: no explicit evaluation identified in the reviewed protocol.
Benchmarks for Spatial Understanding. Spatial understanding benchmarks span static image-based evaluations and interactive embodied settings [25, 37, 38, 39, 40, 41]. VSR [42] evaluates spatial relations in image–text pairs, while other benchmarks examine orientation, navigation, mental rotation, localization, and video recall [43, 44, 45, 46, 47, 48]. ESI-Bench [49] emphasizes active information acquisition for spatial reasoning, while SpaMME [50] probes spatial state updating and long-horizon belief maintenance through action-conditioned observation sequences. RoboSPA [34] evaluates spatial reference resolution, compositional spatial selection, and procedural-memory tasks, and reports object-normalized success and progress measures. BEHAVIOR-Spatial [35] evaluates spatial instruction following through question answering, target grounding, and robot execution, while VABench [36] combines visual demonstrations, active perception, metric control, and trajectory-level diagnostics. GEOMIND complements these benchmarks by organizing explicit geometric goal construction, current-state re-grounding, and historical identity binding within a shared, controllable tabletop protocol. Structure-preserving sampling, deterministic spatial targets, controlled scene changes, and matched task variants support targeted comparisons across these requirements; scene updating need not imply dependence on unobserved history.
Spatial Capabilities of Embodied Policies. Current embodied agents increasingly rely on vision-language models for language-guided manipulation [3, 4, 51, 52, 53], yet complex spatial instructions remain challenging. Recent approaches therefore incorporate more explicit spatial representations and grounding mechanisms. RoboSpatial [54] provides large-scale spatial annotations for training vision-language models, while InternVLA-M [16] combines spatial grounding pretraining with spatially guided action learning to connect where to act with how to act. Other approaches introduce 3D spatial representations to improve the alignment of visual, linguistic, and geometric information for action generation [15, 37, 55, 56]. While these methods aim to improve spatial capabilities, GEOMIND provides a benchmark for systematically examining their strengths and limitations.
GeoMind
We present the overall design of GeoMind by turning structured spatial understanding into a controllable benchmark construction problem. Sec. 3.1 introduces the GeoMind taxonomy, and Sec. 3.2 describes spatial structure modeling and the benchmark construction pipeline. Sec. 3.3 presents the task suites, showing how executable benchmark episodes are organized into GeoMind-BASE and GeoMind-AUG.
Taxonomy of Embodied Spatial Understanding Tasks
Rather than categorizing tasks solely by manipulation actions, we organize them according to the spatial capabilities required to determine task-relevant targets and goals. Following the dimensions introduced above, the taxonomy organizes structured spatial understanding around spatial reasoning and spatial memory. The former concerns how agents infer targets or goals from spatial relations and scene structure, while the latter concerns how agents maintain and update task-relevant spatial information as scenes change.
Target Identification focuses on static object selection from structured layouts. The agent resolves a spatial description to the correct target object in a fixed scene.
Relational Grounding requires the agent to realize an explicit spatial relation through action, such as placing one object beside, in front of, or inside another reference object.
Constraint Composition tests whether the agent can jointly satisfy multiple spatial constraints that define a target object or a valid goal region.
Geometric Inference requires the agent to infer implicit target locations or regions from scene geometry, such as alignment, symmetry, or pattern continuation across objects.
Scene Updating evaluates whether the agent can update task-relevant spatial relations and re-ground its target after objects have moved or been manipulated.
Identity Tracking requires the agent to maintain object identity over time by linking a previously identified object to its current state across scene changes, such as object rearrangement or position exchanges.
Figure 1 qualitatively illustrates the spatial reasoning and memory demands of the six task families and presents representative examples with initial scenes, robot trajectories, and ground-truth spatial targets. The high/low positions in Figure 1 indicate intended capability demands. These tasks remain close to common robotic behaviors, such as selecting, placing, and retrieving objects, while exposing the spatial reasoning and memory demands behind seemingly simple actions.
Figure 1. Overview of GEOMIND. GEOMIND organizes structured spatial understanding along two diagnostic dimensions: spatial memory and spatial reasoning. The six task families cover different combinations of these dimensions. Surrounding examples illustrate how GEOMIND couples spatial instructions with structured object layouts and scene changes. The instructions are illustrative; evaluated actions and success criteria are specified in Appendix § 11.
Spatial Structure Modeling and Benchmark Construction
GeoMind constructs benchmark episodes through spatial structure modeling and an agent-assisted, rule-verified pipeline. Spatial structures allow object configurations to vary while preserving task-relevant constraints. The pipeline combines task template generation, executable spatial grounding, and simulation-based verification to produce structured manipulation tasks with deterministic ground-truth targets (Fig. 3).
Spatial Structure Modeling. To evaluate spatial understanding across different scene configurations, GeoMind represents object layouts using spatial structures that are not tied to a fixed set of coordinates. For example, objects arranged in a line can preserve their left-to-right order while varying in position and spacing.

Figure 2. Visualization of the six fundamental structural layouts and their derived variants.

Figure 3. Overview of the benchmark construction pipeline. The pipeline first proposes task descriptions and structure templates, then resolves spatial queries over instantiated scenes to derive ground-truth annotations, and finally validates candidate episodes through interaction-graph-based simulation verification.
A task such as selecting the second object from the left therefore follows the same spatial rule across different layouts. We formalize a spatial structure as a set of relational, topological, and geometric constraints over objects. Given , GEOMIND generates diverse scene configurations by sampling object poses from a structure-preserving distribution:
where controls allowable geometric variations, such as positional jitter and variable spacing, without violating the constraints encoded in . The structure library includes linear, grid, and other canonical layouts (Fig. 2). Sampled configurations must satisfy structural validity and collision-free workspace constraints. This representation supports controlled evaluation across variations in object positions and spacing. Formal definitions are provided in Appendix § 10.
LLM-Assisted Task and Structure Proposal. Given a task category and a candidate set of spatial structure types, the LLM proposes matching instructions and structure templates within a predefined schema. Each template specifies the structure type, object roles, layout constraints, sampling ranges, and optional scene-update rules. Candidate templates are screened to remove ambiguous, logically conflicting, or task-incompatible specifications, with human review during template selection. The retained templates form a fixed library for subsequent programmatic scene instantiation. The benchmark uses a fixed set of tasks selected through human review, with task instances generated programmatically from the corresponding templates.
Spatial Target Grounding. An LLM-based coding agent translates spatial rules into programs that deterministically resolve target objects or goal regions from simulator states. Directional predicates use the canonical front-camera frame, with object coordinates transformed from the robot base frame. Reasoning tasks ground targets in the current configuration. In memory tasks, instructions precede scripted object motion, and evaluated actions begin after it ends: Scene Updating re-grounds targets in the updated scene, while Identity Tracking preserves their initial identities. Each agent can store earlier observations or maintain an internal memory to identify the target after the objects move.
Closed-Loop Verification. Each candidate episode is executed in a physics simulator to check physical feasibility and target consistency. For tasks involving scene changes, objects move through scripted motion or simulated robot manipulation. Throughout execution, we record simulator object states and construct a dynamic interaction graph , where nodes represent objects and edges encode discretized pairwise directions and distances. The graph sequence is used to check whether scene transitions and resulting spatial states are consistent with the task specifications and ground-truth targets. Episodes failing state-validity or target-consistency checks are discarded.
Task Suites and Dataset
GEOMIND is organized into two complementary evaluation suites: GEOMIND-BASE and GEOMIND-AUG. Together, they provide 54 evaluated task variants for structured spatial understanding. A task variant is a specific test setting that pairs a spatial objective with a scene structure, governed by specific grounding rules and evaluation conditions. GEOMIND-BASE evaluates the six spatial task families under canonical tabletop settings, while GEOMIND-AUG introduces controlled perturbations to assess robustness under additional semantic, visual, and execution challenges.

Figure 4. Detailed information about GeoMIND task suites. (a) shows how GEOMIND-AUG is constructed from GEOMIND-BASE. (b) shows the distribution of evaluated variants across task families and augmentation types (20 Base, 7 Semantic, 7 Visual, and 20 Execution).
GEOMIND-BASE consists of 20 evaluated base variants with diverse language instructions and multiple spatial structure types. To reduce visual shortcuts, many GEOMIND-BASE variants contain multiple visually identical or near-identical objects. In these settings, target selection or placement depends on spatial cues such as relative position, order, geometric relations, or scene updates, rather than object appearance alone. GEOMIND-BASE serves as the core benchmark suite and evaluates the spatial reasoning and memory capabilities defined in our taxonomy. Fig. 4b presents the distribution of evaluated task variants across task families and augmentation types.
GEOMIND-AUG contains 34 augmented variants: 7 Semantic, 7 Visual, and 20 Execution variants. Semantic augmentation modifies language grounding, visual augmentation modifies visual conditions, and execution augmentation tightens action precision requirements. Together with the 20 Base variants, these form 54 evaluated variants. These counts refer to evaluated task variants rather than unique spatial objectives or evaluation episodes. Examples and task details appear in Fig. 4a and Appendix § 11.
Experiments
In this section, we systematically evaluate representative embodied agents on the GEOMIND benchmark under a controlled setting. We first describe the experimental setup and the selected embodied agents. Then, we present the evaluation results, analyzing their implications for spatial understanding.
Experimental Setup
Benchmarked Embodied Agents. We evaluate representative embodied agents across GEOMIND-BASE and GEOMIND-AUG, covering three paradigms: general policies, spatially enhanced policies, and modular agents. These methods differ in whether spatial grounding, planning, and execution are handled implicitly by an end-to-end policy or explicitly by separate modules. Specifically, we evaluate OpenVLA [3], [4], and [5] as general policies; SpatialVLA [15], InternVLA-M1 [16], and Spatial Forcing [56] as spatially enhanced policies; and COPA [57] and MOKA [58] as modular agents. We additionally include an oracle baseline that uses simulator states and deterministic spatial grounding to compute ground-truth targets, which guide a scripted controller. The oracle provides a reference for assessing task feasibility under correct target information.
Evaluation Protocol. All experiments are conducted in the IsaacLab [59] simulator using a Franka Research 3 (FR3) robotic arm. All methods are evaluated on the same task instances, random seeds, episode horizons, and success criteria. The evaluated VLA policies use fixed checkpoints without GEOMIND-specific fine-tuning, and their outputs are adapted to a common action interface. Privileged scene information, including simulator-resolved object identities and ground-truth target coordinates, is used for evaluation and the oracle baseline but is not provided to the evaluated VLA policies or modular agents during inference. Each episode is limited to 1000 control steps and terminates early upon task success or workspace violation. We evaluate episodes per task variant, with evaluation seeds saved for reproducibility. The physics simulation runs at 480 Hz, and the policy control frequency is 24 Hz. Detailed observation and action interfaces, deployment configurations, and evaluation protocols are provided in Appendix § 9.
Evaluation Metrics. We evaluate task completion using Task Success Rate (SR), the proportion of episodes satisfying task-specific success conditions, reported overall, by task family, and by augmentation type. Robustness is measured by in percentage points; negative values indicate degradation. The reference is the matched base task for semantic and visual variants, and the corresponding non-execution variant for execution variants. Spatial Goal Error (SGE) measures final spatial deviation, while Target Tracking Error (TTE) measures average end-effector distance to the target:
Here, indexes episodes, is the evaluator’s goal-distance function, is the episode length, and and denote end-effector and target positions. SGE is aggregated only over valid goal-distance values. TTE uses recorded target trajectories when available, otherwise the initial target position as a marked static-layout proxy. Lower values indicate smaller errors. Applicability and aggregation details are provided in Appendix § 8.
Main Results
We organize the results around five questions concerning overall performance, robustness, failure modes, spatial memory, and spatial reasoning.
Q1. How well do embodied agents perform in structured spatial understanding?
We begin with GeoMind-Base, which evaluates six spatial capabilities under standard task conditions without the additional perturbations introduced in GeoMind-Aug. Table 2 reports the success rates of different agents across the six task families. The best-performing learned policy, InternVLA-M1, achieves 26.4% success, followed by Spatial-Forcing at 24.0%, substantially below the oracle’s 99.3%. Spatially enhanced policies achieve the highest overall scores, although their improvements over general policies are uneven across models and task families. Modular agents achieve 4.0–16.9% success, indicating that explicit VLM-based grounding and scripted execution also struggle with structured spatial tasks.
| Embodied Agents | Target Iden. | Relational Grounding | Constraint Comp. | Geometric Inference | Scene Updating | Identity Tracking | AVG | |
| Oracle | 100.0 | 98.0 | 100.0 | 100.0 | 98.0 | 100.0 | 99.3 | |
| General Policies | OpenVLA | 37.5 | 0.0 | 6.2 | 5.0 | 41.7 | 37.5 | 20.7 |
| General Policies | 4.2 | 0.0 | 0.0 | 25.0 | 6.3 | 20.9 | 11.3 | |
| General Policies | 4.2 | 0.0 | 0.0 | 27.5 | 6.3 | 25.0 | 12.5 | |
| Spatially Enhanced Policies | SpatialVLA | 12.5 | 0.0 | 0.0 | 17.5 | 45.8 | 8.3 | 18.3 |
| Spatially Enhanced Policies | InternVLA-M1 | 42.5 | 50.0 | 6.2 | 12.0 | 16.7 | 20.8 | 26.4 |
| Spatially Enhanced Policies | Spatial-Forcing | 12.5 | 0.0 | 50.0 | 5.0 | 33.3 | 37.5 | 24.0 |
| Modular Methods | COPA GPT-4o | 22.9 | 11.1 | 16.7 | 4.2 | 9.7 | 13.9 | 11.1 |
| Modular Methods | COPA Gemini-2.5-Flash | 10.4 | 2.8 | 6.3 | 1.7 | 6.9 | 0.8 | 4.6 |
| Modular Methods | COPA Claude-Sonnet-4.6 | 20.8 | 13.9 | 16.7 | 13.3 | 16.0 | 19.4 | 15.9 |
| Modular Methods | MOKA GPT-4o | 18.8 | 12.5 | 18.8 | 1.7 | 8.3 | 13.9 | 10.1 |
| Modular Methods | MOKA Gemini-2.5-Flash | 8.3 | 1.4 | 4.2 | 0.8 | 7.6 | 1.4 | 4.0 |
| Modular Methods | MOKA Claude-Sonnet-4.6 | 18.8 | 13.9 | 16.7 | 13.3 | 18.6 | 20.8 | 16.9 |
Table 2. Success rates (%) of different paradigms of embodied agents on GeoMind-Base. Excluding the oracle, best and second-best overall results are highlighted with green shades, while best and second-best family-level results are highlighted with purple shades.
To better understand these performance gaps, we examine the contributions of target grounding and execution. The oracle achieves near-perfect success using ground-truth targets and scripted control, demonstrating the feasibility of the benchmark tasks under correct spatial grounding. More directly, in a paired diagnostic evaluation of , explicitly specifying the target through its name and a visual cue improves success from 11.3% to 39.2%, while keeping the policy, task instances, action interface, and success criterion fixed. This result provides evidence that target identification contributes substantially to the observed failures, although execution and deployment limitations may also play a role.
Finding: Current embodied agents remain far from reliable on structured spatial manipulation tasks. The controlled target-cue intervention suggests that inaccurate spatial target grounding is an important contributor to these failures, beyond execution limitations alone.
Q2. How robust is structured spatial understanding under semantic, visual, and execution perturbations?
Table 3 shows that success in GEOMIND-BASE is fragile under augmented variants of the same spatial goals. The clearest drop comes from semantic augmentation: averaged over the six embodied policies, success decreases from about 22.6% on the corresponding Base tasks to about 14.2%, and every policy degrades. This indicates that many Base successes still rely on simple instruction grounding rather than robust spatial-language understanding. The effects of visual and execution augmentation are more uneven. OpenVLA falls to 12.5% under both settings, whereas remains unchanged under visual perturbation and decreases by 6.5 percentage points under execution. These results suggest that structured spatial understanding degrades under complex instruction semantics and increased visual noise.
| Policies | Corresponding Base | Semantic | Visual | Execution |
| OpenVLA | 25.0 | 16.6 () | 12.5 () | 12.5 () |
| 16.7 | 12.5 () | 16.7 () | 10.2 () | |
| 18.5 | 12.5 () | 16.7 () | 10.2 () | |
| SpatialVLA | 18.8 | 8.3 () | 12.5 () | 12.5 () |
| InternVLA-M1 | 37.5 | 27.1 () | 37.5 () | 35.4 () |
| Spatial-Forcing | 18.8 | 8.3 () | 12.5 () | 16.7 () |
Table 3. Success rates (%) on GEOMIND-AUG and the reported base references. Parentheses show changes in percentage points.
Finding: Spatial success in embodied agents is condition-dependent rather than goal-invariant: the same spatial objective often fails once its linguistic, visual, or execution support is perturbed.
Q3. Why do embodied agents fail on GEOMIND?
We examine representative failure cases in Fig. 5 to connect the quantitative trends with concrete trajectories. For embodied policies in Fig. 5a, the illustrated failures fall into three categories: spatial relation grounding errors, where policies bind the correct anchor but execute the wrong relation; spatial relation reasoning errors, where policies fail to infer structure-defined targets; and scene-updating errors, where policies fail to ground the requested target in the updated configuration. These examples suggest that some failures involve converting spatial observations into appropriate action targets, although the trajectories alone do not isolate grounding errors from control errors. For modular embodied agents in Fig. 5b, the illustrated failures include object localization errors that can propagate to planning and control. This suggests a different bottleneck: modular agents may expose explicit reasoning steps, but remain vulnerable when intermediate spatial grounding is inaccurate.
Finding: The illustrated failures suggest difficulties in translating spatial observations into correct action targets. These examples identify possible failure mechanisms, but do not establish their prevalence across the benchmark.
Q4. What makes spatial memory difficult for embodied agents?
Table 2 and Table 4 show that agents struggle with tasks involving scene changes. Scene Updating requires selecting a target from the updated configuration, whereas Identity Tracking requires linking an object identified before the change to its new position. An agent may therefore fail an identity-tracking task by selecting an object that matches the current spatial relation but is not the original target. This provides a possible explanation for failures beyond imprecise execution. However, aggregate SGE and TTE cannot distinguish these causes: both are affected by scene geometry, initial target distance, and rollout duration. We use them as auxiliary diagnostics rather than direct measures of identity preservation or evidence that one task family is intrinsically harder.
| Capability Dimensions | Task Family | Failure Rate ↓ | SGE (m) ↓ | TTE (m) ↓ |
| Spatial Memory | Scene Updating | 68.89% | 0.189 | 0.330 |
| Spatial Memory | Identity Tracking | 85.00% | 0.192 | 0.486 |
| Spatial Reasoning | Target Identification | 72.08% | 0.116 | 0.811 |
| Spatial Reasoning | Relational Grounding | 98.33% | 0.180 | 0.672 |
| Spatial Reasoning | Constraint Composition | 87.50% | 0.142 | 0.627 |
| Spatial Reasoning | Geometric Inference | 86.67% | 0.158 | 0.727 |
Table 4. Family-level failure rates and auxiliary spatial errors. Lower values are better.
Finding: Identity Tracking requires preserving object correspondence across scene changes; the current aggregate results do not establish how much failure is attributable to incorrect target selection versus execution.
Q5. What makes spatial reasoning difficult for embodied agents?
Table 4 reports the highest failure rate for relational grounding among the reasoning families in that diagnostic summary, at 98.33%, compared with 72.08% for target identification. Constraint composition and geometric inference are also difficult, with failure rates of 87.50% and 86.67%, respectively. This indicates that current agents do not only struggle with recognizing spatial targets; they also fail to convert explicit spatial relations into correct embodied actions. Table 2 shows that geometric inference has the lowest best-agent success rate, with the strongest method reaching only 27.5%, suggesting that implicit structure-based reasoning remains hard even for the best agents. Together, these results suggest that agents remain weak when they must jointly satisfy multiple spatial constraints or infer targets from implicit geometric structure.
Finding: Spatial reasoning is difficult not only because agents struggle to identify the correct spatial target, but because they fail to transform spatial constraints into executable manipulation behavior.

Figure 5. (a) Embodied policies. (b) Modular agents. Visualization of typical failure cases in GeoMIND.
Conclusion
We present GeoMIND, a systematic benchmark for evaluating spatial understanding in embodied agents. By organizing tasks around spatial memory and spatial reasoning, GeoMIND enables a more fine-grained analysis than overall task success alone. We further emphasize how spatial structure underpins spatial understanding, and present an automated generation pipeline that allows scalable benchmark construction. Our experiments show that current embodied agents remain far from achieving consistently high success rates on GeoMIND, with large performance gaps and highly uneven abilities across task families. These findings suggest that spatial understanding remains a core limitation of embodied agents.
Limitations. GEOMIND is limited to simulated, single-arm tabletop manipulation, without real-robot validation. Its controlled layouts do not capture the full diversity of real-world environments. Heavy occlusion, viewpoint changes, sensor noise, and complex contact dynamics are not systematically evaluated, although incidental occlusion may occur during execution. Semantic and visual augmentations cover selected transformations rather than exhaustive robustness conditions. Our experiments evaluate frozen pretrained policies, and their performance may also depend on observation and action-interface compatibility. We do not report benchmark-specific fine-tuning results; learning spatial skills and generalizing to held-out structures or variants remain untested. The results therefore characterize frozen-policy performance in the evaluated setting and do not establish transfer to real robots.
Code and Data Availability
The benchmark code and data have not yet been publicly released. We plan to release the benchmark assets, evaluation code, configuration files, and reproduction instructions in the project repository: https://github.com/Notfound-JH/GEOMIND-benchmark.git.
Acknowledgments and Disclosure of Funding
Funding Statement. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Competing Interests. The authors declare no competing interests.
Metrics Formulation
Success Rate
We compute the overall and family success rates as
where indicates whether episode of task variant succeeds, denotes the set of evaluated task variants, and denotes the subset in family . Each variant has episodes. With equal episode counts per variant, this aggregation weights task variants equally rather than weighting task families equally.
Spatial Goal Error
Spatial Goal Error (SGE) is an auxiliary diagnostic metric that measures the final distance between the achieved spatial state and the target spatial goal. It is designed to capture final spatial precision, rather than full task completion.
For the -th episode, we define
where denotes the final achieved state and denotes the target spatial state, and is the evaluator’s goal-distance function for episode . For tasks with an explicit target position, this reduces to the Euclidean distance
In our implementation, SGE is obtained from the evaluator-provided goal-distance proxy, such as evaluation.goal_distance or distance_m. If an episode does not provide a valid goal-distance value, its SGE is treated as missing and excluded from SGE aggregation. Therefore, whenever SGE is reported, we also record the number of valid SGE episodes.
For a model and task family , we aggregate SGE over valid episodes:
where denotes the set of episodes with valid SGE values for model and family .
For a specified set of evaluated models, we first compute the model-family average and then average over models:
These averages are defined only for nonempty valid-episode sets. A model-family pair with no valid values has an undefined SGE and must not be treated as zero; the cross-model formula applies when each model in the specified set has a defined value. This weighting avoids giving models with more valid episodes more weight.
Interpretation. Lower SGE indicates better final spatial precision. However, SGE does not measure whether the agent followed the correct trajectory, selected the correct object throughout the rollout, or satisfied all spatial relations. It is therefore used only as an auxiliary diagnostic, not as a replacement for official task success.
Target Tracking Error
Target Tracking Error (TTE) is an auxiliary trajectory-level diagnostic metric. While SGE measures final spatial error, TTE measures whether the end-effector remains close to the intended target throughout the rollout. It measures target proximity along the trajectory, but does not directly measure spatial memory or identity preservation.
For the -th episode with trajectory length , we define
where is the end-effector position at timestep , and is the ground-truth target position at the same timestep.
When dynamic object poses are available in the trajectory trace, the target position is taken from the target object’s dynamic pose:
When dynamic object poses are unavailable, we use the static target position from the sampled layout as a proxy:
In this case, the metric is marked as a static-layout proxy.
For a model and task family , TTE is first computed at the episode level and then averaged over valid episodes:
where denotes the set of episodes with valid trajectory traces for model and family .
When reporting a family-level average across models, we compute
The same nonempty-set requirement applies to TTE: a model-family pair without a valid trajectory has an undefined value, and the cross-model formula requires a defined value for each model in the specified set. This model-wise aggregation prevents a run with more episodes from dominating the reported value. We avoid pooling all timesteps across episodes directly, because such pooling would give longer trajectories larger weight.
Interpretation. Lower TTE means that the end-effector stays closer to the intended target over the trajectory. Unlike SGE, which only evaluates the final state, TTE captures trajectory-level target following. However, TTE still does not directly verify successful grasping, placement, or spatial-relation satisfaction, so it is used as an auxiliary diagnostic rather than an official success metric.
Detailed Experimental Setup
This section provides the implementation details needed to reproduce the GEOMIND evaluation. We focus on the experimental variables that directly affect validity and reproducibility: simulator and robot setup, control frequency, policy inputs, action conversion, rollout protocol, baseline deployment, and information access. Unless otherwise stated, all methods are evaluated on the same task instances, random seeds, workspace constraints, episode horizon, and success criteria.
Simulation, Robot, and Control
All experiments are conducted in GEOMIND LAB, an IsaacLab-based benchmark package built on Isaac Sim. We use a single Franka Research 3 (FR3) arm equipped with the FR3 parallel-jaw gripper. The robot receives relative end-effector commands and is controlled by a differential inverse-kinematics controller. The gripper is controlled through finger joint-position commands and is exposed to policies as a binary open/close command.
The physics simulation runs at 480 Hz, while policies are queried at 24 Hz. Each policy action is therefore executed through the low-level controller over 20 physics substeps. An episode terminates when the task succeeds, the robot violates the workspace guard, or the maximum horizon is reached. Table 5 summarizes the main simulator and control settings.
| Item | Setting |
| Simulator | Isaac Sim / IsaacLab |
| Benchmark package | GEOMIND LAB |
| Robot | Franka Research 3 (FR3) |
| Gripper | FR3 parallel-jaw gripper |
| Low-level controller | Differential inverse kinematics |
| Physics frequency | 480 Hz |
| Policy control frequency | 24 Hz |
| Episode horizon | 1000 control steps |
| Termination | Success / workspace violation / horizon |
| Workspace guard | m |
Table 5. Simulation and control settings used for all evaluations.
Observation and Action Interface
All end-to-end VLA policies are evaluated through a common observation and action interface. The policy input contains the language instruction, two RGB camera observations, and robot proprioception. The two cameras consist of a static table-view camera and a wrist-mounted camera, both rendered at resolution.
Importantly, privileged simulator information is not exposed to VLA policies. Object poses, object identities, segmentation masks, ground-truth task variables, oracle target coordinates, and task metadata are used only for evaluation. Table 6 summarizes the policy interface and the evaluation-only states.
| Type | Field | Description |
| Policy input | Instruction | Natural-language task instruction |
| Policy input | Table-view RGB | Static external camera observation, |
| Policy input | Wrist RGB | Wrist-mounted camera observation, |
| Policy input | Joint state | Robot joint positions and velocities |
| Policy input | End-effector / gripper state | Current end-effector pose and gripper state |
| Policy action | Translation | Relative Cartesian displacement |
| Policy action | Rotation | Relative rotation-vector command |
| Policy action | Gripper | Binary open/close command |
| Evaluation only | Object poses | Used for success checking and diagnostics |
| Evaluation only | Object identities | Used for ground-truth target resolution |
| Evaluation only | Segmentation masks | Not exposed to VLA policies |
| Evaluation only | Task metadata | Used only for evaluation and diagnostics |
| Evaluation only | Target regions / coordinates | Not exposed to VLA policies |
Table 6. Observation and action interface. Privileged simulator states are excluded from VLA policy inputs.
The common action interface is a 7-DoF relative end-effector command,
where the first three dimensions represent Cartesian displacement, the next three dimensions represent a relative rotation-vector command, and the final dimension controls the gripper. For policies whose native output format differs from this interface, we convert the output to the same 7-DoF command before execution.
We use a unified normalized action protocol. Each policy output is first converted to a 7-DoF action vector and clipped to . The translational and rotational components are then scaled by a model-specific multiplier . In the following equations, denotes the converted, already clipped action vector. Slices use zero-based indexing with an exclusive upper bound: contains the three translation components and the three rotation components. The clip operator acts componentwise:
| Item | Setting |
| GEOMIND-BASE | 20 variants, 48 episodes per variant, 960 episodes per method |
| GEOMIND-AUG | 34 augmented variants, 48 episodes per variant, 1632 episodes per method |
| Task instances | Shared across methods |
| Random seeds | Shared across methods |
| Episode horizon | 1000 control steps |
| Policy control frequency | 24 Hz |
| Physics frequency | 480 Hz |
| Camera resolution | |
| Primary metric | Success rate |
| VLA adaptation | No GEOMIND-specific fine-tuning |
Table 7. Evaluation protocol for GEOMIND-BASE and GEOMIND-AUG.
The controller converts these normalized components to physical increments using scales of 0.5 m for translation and 0.5 rad for rotation, yielding
The gripper command is thresholded at zero: positive values are mapped to open and negative values are mapped to close. For baselines without reliable gripper outputs, we keep the gripper fixed according to their default deployment protocol and report this setting explicitly.
Evaluation Protocol
We evaluate each method on the same task instances and random seeds. Each evaluated task variant contains 48 evaluation episodes. GEOMIND-BASE contains 20 variants, resulting in 960 episodes per method. GEOMIND-AUG contains 34 augmented variants (7 Semantic, 7 Visual, and 20 Execution), resulting in 1632 episodes per method, for 2592 episodes across both suites. Success rate is the primary metric and is computed over all episodes in the corresponding task suite or task family.
VLA Baseline Deployment
All VLA baselines are evaluated as frozen policies without GEOMIND-specific fine-tuning. Each model is deployed as a remote policy server and queried online during rollout. For each model, we use a fixed checkpoint and server configuration across all evaluated episodes.
The policy server receives only the allowed policy inputs described in Table 6. No VLA baseline receives privileged simulator object poses, segmentation masks, ground-truth task variables, oracle target coordinates, or task metadata through the policy interface.
Some VLA models return action chunks rather than a single action. In our deployment, the client executes only the first valid action from the returned chunk and then queries the policy again at the next control step. This keeps all policies closed-loop at the shared 24 Hz control frequency while preserving each model’s native action-generation interface.
Different VLA models use different native action heads and deployment conventions. We therefore report the model-specific action multiplier, checkpoint setting, rotation setting, and gripper setting in Table 8. These settings are fixed for a given model across the full evaluation. Simulations were conducted on a workstation equipped with an NVIDIA RTX 3090 GPU, while model inference was performed on an NVIDIA RTX 4090 (48 GB).
| Model | Fine-tuning | Checkpoint | Rotation / gripper | |
| OpenVLA | No | Fixed | 2.0 | Rotation enabled; binary gripper |
| No | Fixed | 0.5 | Rotation disabled; fixed gripper | |
| No | Fixed | 0.5 | Rotation disabled; fixed gripper | |
| SpatialVLA | No | Fixed | 5.0 | Rotation enabled; binary gripper |
| InternVLA-M1 | No | Fixed | 0.5 | Rotation enabled; binary gripper |
| Spatial-Forcing | No | Fixed | 0.2 | Rotation enabled; binary gripper |
Table 8. Deployment settings of VLA baselines. denotes the normalized action gain used before controller-level command limiting. All models are evaluated with fixed checkpoints and without GEOMIND-specific fine-tuning.
Modular Baselines and Diagnostic Use
We additionally evaluate modular agents based on COPA [57] and MOKA [58]. These methods represent a different embodied-agent paradigm from end-to-end VLA policies: instead of directly predicting low-level actions from visual observations and language instructions, they decompose manipulation into explicit visual-language grounding, target selection, and scripted execution. We include them as modular baselines to compare different design paradigms for structured spatial manipulation, and also use their intermediate predictions to diagnose where failures arise.
Each modular agent receives the task instruction and RGB observations, and uses a VLM backbone to identify the task-relevant object or spatial target from the scene. We instantiate the modular agents with three VLM backbones: GPT-4o, Gemini-2.5-Flash, and Claude-Sonnet-4.6. The predicted target is then converted into Cartesian waypoints or a target region and executed with the same Franka differential-IK controller used in the VLA evaluation. For pick-and-place tasks, the scripted routine follows a top-down sequence of approach, grasp, lift, place, release, and retreat. For reaching tasks, the controller moves the end effector toward the predicted target region and evaluates success using the same task evaluator as other methods.
All modular agents are evaluated on the same task instances, random seeds, workspace constraints, episode horizon, and success criteria as the end-to-end policies. They do not receive privileged simulator information during inference: oracle target coordinates, simulator-resolved object identities, ground-truth task variables, segmentation masks, and task metadata are not exposed to the VLM or the planner. These privileged states are used only by the evaluator for success checking and diagnostic logging.
Because modular agents use a different inference and execution structure from VLA policies, their results should be interpreted as a comparison across embodied-agent paradigms rather than as a fully controlled comparison of identical policy interfaces. This distinction is important: end-to-end policies test whether a learned visuomotor model can directly map observations and language to actions, whereas modular agents test whether explicit VLM-based grounding and scripted execution can solve the same spatial objectives. Their low success rates therefore provide complementary evidence that structured spatial understanding is difficult not only for end-to-end action generation, but also for explicit target grounding and target-to-action conversion pipelines.
Fairness and Information Access
The main comparison in GEOMIND covers multiple embodied-agent paradigms under the same task instances, random seeds, workspace constraints, episode horizon, and success criteria. For end-to-end VLA policies, all models receive the same observation modalities and are evaluated as frozen policies without GEOMIND-specific fine-tuning. For modular baselines, the VLM grounding stage receives the same task instruction and RGB observations, while execution is performed by a scripted controller after target selection.
No evaluated method receives privileged simulator information during inference. Object poses, simulator-resolved object identities, oracle target coordinates, ground-truth task variables, segmentation masks, and task metadata are excluded from both VLA policies and modular baselines. These states are used only for evaluation, success checking, and diagnostic logging.
| Information | VLA policies | Modular baselines |
| Language instruction | Yes | Yes |
| RGB observations | Yes | Yes |
| Robot proprioception | Yes | Yes |
| Object poses | No | No |
| Object identities | No | No |
| Segmentation masks | No | No |
| Task metadata | No | No |
| Oracle target coordinates | No | No |
| Ground-truth task variables | No | No |
| GEOMIND-specific fine-tuning | No | No |
| Role in evaluation | End-to-end policy | Modular baseline |
| Inference structure | Direct action prediction | VLM grounding + scripted execution |
| Diagnostic signal | Trajectory/action behavior | Grounding and target-selection errors |
Table 9. Information access for end-to-end VLA policies and modular baselines. Both method families are evaluated without privileged simulator information during inference. Modular baselines use explicit VLM-based grounding and scripted execution, so their results are interpreted as cross-paradigm comparisons with additional diagnostic value.
Since VLA policies and modular baselines differ in inference structure, their results should be read as a cross-paradigm comparison rather than a comparison of identical policy interfaces. This is intentional: GEOMIND aims to evaluate whether different embodied-agent designs can solve the same structured spatial objectives under the same task conditions. The modular results complement the VLA results by revealing whether explicit visual grounding and scripted execution can overcome the spatial failures observed in end-to-end policies.
Spatial Structure Formalization
This appendix gives a compact description of the spatial structures used in GEOMIND. Figure 6 shows representative structure layouts. Our goal is not to bind each task to one fixed object coordinate template. Instead, each task is associated with a spatial structure, and GEOMIND samples multiple physical realizations that preserve the same task-relevant relation.

Figure 6. Example layouts generated by the spatial structure modeling and construction pipeline.
Structure-Preserving Sampling
Let a scene contain objects . The pose of object is
where is the object position and is the object orientation. For tabletop relations, we use the planar position . The full scene state is
We denote a spatial structure as . It specifies what should remain invariant during sampling, such as an object order, a grid layout, an anchor-based relation, or a geometric target rule. Given , GEOMIND samples object poses as
where controls small layout variations such as jitter or spacing changes. A sampled scene is accepted only if
Here, is the set of configurations satisfying the structural constraints, and is the set of physically valid configurations.
In short, describes the stable spatial meaning of the task, while is one sampled scene that realizes this meaning.
Representative Spatial Structures
As illustrated in Figure 6, GEOMIND uses several common spatial structures. We summarize them below with minimal constraints.
Ordered structure. This structure is used for tasks such as “pick the leftmost object” or “pick the second object from the right.” Let be a unit vector in the ordering direction and define the projected coordinate
Let be the permutation specifying the intended object order . For a minimum projected separation , a valid sample must satisfy
This keeps the object order unchanged while allowing the exact coordinates to vary.
Grid structure. This structure is used for row-column layouts, such as “the object in the top-left cell” or “the object in the same row as the bowl.” Each object has a grid index . A sampled position is generated around the corresponding grid cell:
Here, is the grid origin, are the grid spacings, and are unit vectors along the column and row directions, and is the positional perturbation. The perturbation bound alone does not guarantee that row-column relations are preserved; accepted configurations must also satisfy the structural-validity condition above.
Anchor-based relation. This structure is used for tasks such as “the object left of the cup”, “the object near the plate”, or “the object between two bowls.” For an object and an anchor , define
A directional relation is valid when
where is a unit vector in the relation direction and is a directional separation threshold that excludes ambiguous cases. Let be the corresponding task predicate over candidate objects and the anchor; ineligible objects have predicate value zero. The target is accepted only when the relation selects one unique object:
Geometric target. This structure is used when the target is an implicit position rather than a visible object. For example, the target may be the midpoint between two anchors:
or the centroid of the positions of a nonempty set of anchors indexed by :
These tasks require the agent to infer a target from scene geometry, instead of directly selecting an existing object.
Dynamic Relations
For tasks with scene changes, GEOMIND distinguishes two cases. In scene-updating tasks, the target is determined by the current relation:
For example, the correct object is the one currently left of the cup.
In identity-tracking tasks, the target is determined by the initial relation and then kept fixed:
For example, the correct object is the one that was originally left of the cup, even if it later moves away.
Physical Validation
Finally, each sampled scene is checked in the simulator. We retain instances that satisfy the simulator’s collision and workspace checks and have a reachable target. For distinct objects whose footprints must remain separated in the tabletop plane, a circular-footprint separation condition can be written as
Here, is the radius of a bounding disk for object ’s planar footprint, is a separation margin, is the admissible tabletop region, and specifies a reachable goal position. The pairwise inequality applies only to distinct object pairs requiring planar separation; it is not a universal constraint for containment or stacking. Task-specific contact feasibility is checked in simulation.
Summary
The structure gallery in Figure 6 provides visual examples of the layouts used in GEOMIND. The formulas above specify only the minimal constraints needed to preserve these structures during sampling. This allows GEOMIND to generate diverse scenes with the same task semantics, so evaluation focuses on spatial understanding rather than memorization.
Task Details
We present 26 illustrative task cards covering a subset of the benchmark. The full evaluation set contains 54 task variants: 20 Base, 7 Semantic, 7 Visual, and 20 Execution variants. The latter three categories constitute the 34 augmented variants. The number of task cards is not the total number of evaluated variants.
Each card gives a compact specification of one task variant, including the task instruction, the corresponding suite and task family, the spatial structure used for scene generation, the ground-truth target, the numerical ground-truth pose, and the evaluation criterion. The scene image on the left illustrates one sampled realization of the task, while the fields on the right define the task in a reproducible form. All target poses are reported in the simulator world frame, and distances are measured in meters unless otherwise stated.

Figure 7 (PDF p. 23).
BindingBase_AnchorDirection_v0
Task suite and family. GEOMIND-BASE / Target Identification
Instruction. Move the end effector near the cube to the right of the blue bowl.
Structure. anchor_direction_injected structure.
GT Target. Protocol target object block_right proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 8 (PDF p. 23).
ConstraintBase_SelectExclusionUnique_v0
Task suite and family. GEOMIND-BASE / Constraint Composition
Instruction. Move the end effector near the red cube that is closest to the blue plate and is not the leftmost cube.
Structure. exclusion_distance_constraint_injected structure.
GT Target. Protocol target object red_block_target proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 9 (PDF p. 23).
ConstraintBase_SelectVisualSpatial_v0
Task suite and family. GEOMIND-BASE / Constraint Composition
Instruction. Move the end effector near the wood-grain cylinder to the right of the red block.
Structure. visual_spatial_constraint_injected structure.
GT Target. Protocol target object wood_cylinder_target proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 10 (PDF p. 23).
GeometricBase_IsometricExtension_v0
Task suite and family. GEOMIND-BASE / Geometric Inference
Instruction. You already hold the cube. Place it on the red–blue line extension at equal spacing.
Structure. 2-anchor_line_blocks_runtime_injected structure.
GT Target. Implicit equal-spacing extension point beyond the second anchor.
GT Pose. ; .
Eval. Success requires the distance from the object center to the target position to be m and the final state to remain stable for 1.0 s. GeometricBase_PerfectMidpoint_v0

Figure 11 (PDF p. 24).
Task suite and family. GEOMIND-BASE / Geometric Inference
Instruction. You already hold the cube. Place it midway between the red and blue reference cubes.
Structure. 2-anchor line_blocks_runtime_injected structure.
GT Target. Implicit midpoint region between the two colored anchor blocks.
GT Pose. ; .
Eval. Success requires the distance from the object center to the target position to be and the final state to remain stable for 1.0 s.

Figure 12 (PDF p. 24).
GeometricBase_PointSymmetry_v0
Task suite and family. GEOMIND-BASE / Geometric Inference
Instruction. You already hold the cube. Place it symmetrically to the blue cube about the red cube.
Structure. 2-anchor diagonal_blocks_runtime_injected structure.
GT Target. Implicit point-symmetric reflection of the reference anchor across the center anchor.
GT Pose. ; .
Eval. Success requires the distance from the object center to the target position to be and the final state to remain stable for 1.0 s.

Figure 13 (PDF p. 24).
GeometricBase_SquareCompletion_v0
Task suite and family. GEOMIND-BASE / Geometric Inference
Instruction. You already hold the fourth cube. Place it to complete a square with the three anchor cubes.
Structure. 3-anchor square_corner_runtime_injected structure.
GT Target. Implicit missing square corner inferred from three anchors.
GT Pose. ; .
Eval. Success requires the distance from the object center to the target position to be and the final state to remain stable for 1.0 s.

Figure 14 (PDF p. 24).
GeometricBase_TrendContinuation_v0
Task suite and family. GEOMIND-BASE / Geometric Inference
Instruction. You already hold the next cube. Place it to continue the visible row/column pattern.
Structure. 3-anchor diagonal_trend_blocks_runtime_injected structure.
GT Target. Implicit next point in the anchor trend.
GT Pose. ; .
Eval. Success requires the distance from the object center to the target position to be and the final state to remain stable for 1.0 s.

Figure 15 (PDF p. 24).
BindingBase_GridRowcolGrounding_v0
Task suite and family. GEOMIND-BASE / Target Identification
Instruction. Move the end effector near the cube at row 1, column 2.
Structure. runtime_grid_2x3_static structure.
GT Target. Protocol target object grid_cell_1 proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be and the final state to remain stable for 1.0 s.

Figure 1 (PDF p. 25).
BindingBase_OrdinalInsertStatic_v0 Task suite and family. GEOMIND-BASE / Target Identification Instruction. Move the end effector near the second cube from the left. Structure. runtime_block_line_static_ordinal structure. GT Target. Protocol target object ordinal_static_1 proximity region. GT Pose. m; rad. Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 2 (PDF p. 25).
ReindexBase_ReindexExtreme_v0 Task suite and family. GEOMIND-BASE / Scene Updating Instruction. Move the end effector near the leftmost cube after reordering. Structure. runtime_block_line structure. GT Target. Protocol target object reindex_block_1 proximity region. GT Pose. m; rad. Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 3 (PDF p. 25).
ReindexBase_ReindexInsert_v0 Task suite and family. GEOMIND-BASE / Scene Updating Instruction. Move the end effector near the second-from-left cube. Structure. runtime_block_line_insert structure. GT Target. Protocol target object insert_block_3 proximity region. GT Pose. m; rad. Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 4 (PDF p. 25).
ReindexBase_ReindexNeighbor_v0 Task suite and family. GEOMIND-BASE / Scene Updating Instruction. Move the end effector near the cube immediately left of the red cube. Structure. runtime_red_anchor_neighbor structure. GT Target. Protocol target object right_neighbor_block proximity region. GT Pose. m; rad. Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 5 (PDF p. 25).
ReindexBase_ReindexSubset_v0 Task suite and family. GEOMIND-BASE / Scene Updating Instruction. Move the end effector near the middle cube among the red cubes only. Structure. runtime_red_subset_with_distractors structure. GT Target. Protocol target object red_subset_block_2 proximity region. GT Pose. m; rad. Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 1 (PDF p. 26).
ReindexAugSem_ReindexExtreme_v0
Task suite and family. GEOMIND-AUG / Scene Updating
Instruction. Move the end effector near the leftmost cube after reordering.
Structure. runtime_block_line structure.
GT Target. Protocol target object reindex_block_1 proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 2 (PDF p. 26).
ReindexAugVisual_ReindexExtreme_v0
Task suite and family. GEOMIND-AUG / Scene Updating
Instruction. Move the end effector near the leftmost cube after reordering.
Structure. runtime_block_line structure.
GT Target. Protocol target object reindex_block_1 proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 3 (PDF p. 26).
RelationAugSem_PlaceDirectional_v0
Task suite and family. GEOMIND-AUG / Relational Grounding
Instruction. You already hold the wooden cube. Place it in the valid target region in front of the blue plate.
Structure. directional_region_semantic_distractors structure.
GT Target. Directional target zone relative to blue_plate.
GT Pose. ; .
Eval. Success requires the distance from the object center to the target position to be m and the final state to remain stable for 1.0 s.

Figure 4 (PDF p. 26).
RelationAugVisual_PlaceDirectional_v0
Task suite and family. GEOMIND-AUG / Relational Grounding
Instruction. You already hold the wooden cube. Place it in the valid target region in front of the blue plate.
Structure. directional_region_visual_clutter structure.
GT Target. Directional target zone relative to blue_plate.
GT Pose. ; .
Eval. Success requires the distance from the object center to the target position to be m and the final state to remain stable for 1.0 s.

Figure 5 (PDF p. 26).
RelationBase_PlaceDirectional_v0
Task suite and family. GEOMIND-BASE / Relational Grounding
Instruction. You already hold the wooden cube. Place it in the valid target region in front of the blue plate.
Structure. directional_region_injected structure.
GT Target. Directional target zone relative to blue_plate.
GT Pose. ; .
Eval. Success requires the distance from the object center to the target position to be m and the final state to remain stable for 1.0 s.

Figure 1 (PDF p. 27).
RelationBase_PlaceInside_v0
Task suite and family. GEOMIND-BASE / Relational Grounding
Instruction. You already hold the cube. Put it fully inside the blue box.
Structure. containment_standard_injected structure.
GT Target. Stable containment inside blue_box_floor.
GT Pose. ; .
Eval. Success requires stable containment in the box, a center error m relative to the target position, and stability for 1.0 s.

Figure 2 (PDF p. 27).
RelationBase_PlaceOnTop_v0
Task suite and family. GEOMIND-BASE / Relational Grounding
Instruction. You already hold the cube. Release it onto the top surface of the red wooden block so it rests stably.
Structure. support_top_injected structure.
GT Target. Stable top placement on red_support_block.
GT Pose. ; .
Eval. Success requires stable placement on the support surface, a placement error m, and stability for 1.0 s.

Figure 3 (PDF p. 27).
TrackBase_TrackClutter_v0
Task suite and family. GEOMIND-BASE / Identity Tracking
Instruction. After the clutter settles, move the end effector near the cube that was initially in the middle.
Structure. clutter_robust_runtime_identity_line structure.
GT Target. Protocol target object identity_block_1 proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 4 (PDF p. 27).
TrackBase_TrackMulti_v0
Task suite and family. GEOMIND-BASE / Identity Tracking
Instruction. Move the end effector near the cube that started at the far left.
Structure. runtime_identity_multistep_line structure.
GT Target. Protocol target object identity_block_0 proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 5 (PDF p. 27).
TrackBase_TrackSingle_v0
Task suite and family. GEOMIND-BASE / Identity Tracking
Instruction. Move the end effector near the cube that started in the middle.
Structure. runtime_identity_line structure.
GT Target. Protocol target object identity_block_1 proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 1 (PDF p. 28).
TrackAugSem_TrackSingle_v0
Task suite and family. GEOMIND-AUG / Identity Tracking
Instruction. Move the end effector near the cube that started in the middle.
Structure. runtime_identity_line structure.
GT Target. Protocol target object identity_block_1 proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.

Figure 2 (PDF p. 28).
TrackAugVisual_TrackSingle_v0
Task suite and family. GEOMIND-AUG / Identity Tracking
Instruction. Move the end effector near the cube that started in the middle.
Structure. runtime_identity_line structure.
GT Target. Protocol target object identity_block_1 proximity region.
GT Pose. ; .
Eval. Success requires the end-effector distance to the target object’s center to be m and the final state to remain stable for 1.0 s.
References
- [1]Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution. Nature communications, 12(1):5721, 2021.DOI
- [2]Generalist AI Team. GEN-0: Embodied foundation models that scale with physical interaction. Generalist AI Blog, 2025. https://generalistai.com/blog/preview-uqlxvb-bb.html.
- [3]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA: An Open-Source Vision-Language-Action model, 2024.arxiv.org/abs/2406.09246
- [4]Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Lilyming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. Pi-0: A Vision-Language-Action flow model for general robot control, 2024.arxiv.org/abs/2410.24164
- [5]Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling, Haohuan Wang, Lili Yu, and Ury Zhilinsky. π0.5: a Vision-Language-Action model with Open-World Generalization. In 9th Annual Conference on Robot Learning, 2025.arxiv.org/abs/2504.16054
- [6]Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. UniVLA: Learning to act anywhere with task-centric latent actions, 2025.arxiv.org/abs/2505.06111
- [7]Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. RDT-1B: a diffusion foundation model for bimanual manipulation, 2024.arxiv.org/abs/2410.07864
- [8]Brianna Zitkovich, Tianhe Yu, Sichun Xu, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Proceedings of The 7th Conference on Robot Learning, volume 229 of Proceedings of Machine Learning Research, pages 2165–2183. PMLR, 2023.arxiv.org/abs/2307.15818
- [9]Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026.arxiv.org/abs/2602.15922
- [10]Jiaming Zhou, Qihang Zhang, Gangwei Xu, et al. Zero-WAM: In-context world-action modeling from human videos for open-ended task generalization. arXiv preprint arXiv:2608.26103, 2026.arxiv.org/abs/2608.26103
- [11]Yuran Wang, Siqiao Huang, Mingleyang Li, et al. OpenWAM: An open, modular exploration towards systematic world-action model pretraining. arXiv preprint arXiv:2609.07398, 2026.arxiv.org/abs/2609.07398
- [12]Heng Yu, David D. Yuan, Juze Zhang, et al. OpenWAM: An open framework for composable world-action models. arXiv preprint arXiv:2610.07922, 2026.arxiv.org/abs/2610.07922
- [13]Runyu Lu, Yubo Wu, Ethan Kou, et al. ASPIRE: Agentic /skills discovery for robotics. arXiv preprint arXiv:2607.00272, 2026.arxiv.org/abs/2607.00272
- [14]Wenli Xiao, Jia Xie, Tonghe Zhang, et al. ENPIRE: Agentic robot policy self-improvement in the real world. arXiv preprint arXiv:2606.19980, 2026.arxiv.org/abs/2606.19980
- [15]Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, JiaYuan Gu, Bin Zhao, Dong Wang, and Xuelong Li. SpatialVLA: Exploring spatial representations for Visual-Language-Action model, 2025.arxiv.org/abs/2501.15830
- [16]Xinyi Chen, Yilun Chen, Yanwei Fu, Ning Gao, Jiaya Jia, Weiyang Jin, Hao Li, Yao Mu, Jiangmiao Pang, Yu Qiao, et al. InternVLA-M1: A spatially guided vision-language-action framework for generalist robot policy. arXiv preprint arXiv:2510.13778, 2025.arxiv.org/abs/2510.13778
- [17]Zhengshen Zhang, Hao Li, Yalun Dai, Zhengbang Zhu, Lei Zhou, Chenchen Liu, Dong Wang, Francis E. H. Tay, Sijin Chen, Ziwei Liu, Yuxiao Liu, Xinghang Li, and Pan Zhou. From spatial to actions: Grounding Vision-Language-Action model in spatial foundation priors, 2025.arxiv.org/abs/2510.17439
- [18]Bo Liu, Yifeng Zhu, Chonkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning, 2023.arxiv.org/abs/2306.03310
- [19]Jiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yunchao Yao, Xiaodi Yuan, Pengwei Xie, Zhaio Huang, Rui Chen, and Hao Su. ManiSkill2: A unified benchmark for generalizable manipulation skills. In International Conference on Learning Representations, 2023.arxiv.org/abs/2302.04659
- [20]Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J Davison. RLBench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2):3019–3026, 2020.arxiv.org/abs/1909.12271
- [21]Shiduo Zhang, Zhe Xu, Peijiu Liu, Xiaopeng Yu, Yuan Li, Qinghui Gao, Zhaoye Fei, Zhangyue Yin, Zuxuan Wu, Yu-Gang Jiang, and Xipeng Qiu. VLABench: A Large-Scale benchmark for Language-Conditioned robotics manipulation with Long-Horizon reasoning tasks, 2024.arxiv.org/abs/2412.18194
- [22]Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning, pages 1094–1100. PMLR, 2020.arxiv.org/abs/1910.10897
- [23]Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu, and Linxi Fan. VIMA: General robot manipulation with multimodal prompts. In Fortieth International Conference on Machine Learning, 2023.arxiv.org/abs/2210.03094
- [24]Tianxiang Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, et al. RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint arXiv:2506.18088, 2025.arxiv.org/abs/2506.18088
- [25]Jihan Yang, Shusheng Yang, Anjali Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in Space: How Multimodal Large Language Models See, Remember and Recall Spaces. arXiv preprint arXiv:2412.14171, 2024.arxiv.org/abs/2412.14171
- [26]Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wolfram Burgard. CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks, July 2022. arXiv:2112.03227 [cs].arxiv.org/abs/2112.03227
- [27]Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, et al. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.05941, 2024.arxiv.org/abs/2405.05941
- [28]Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. RoboCasa: Large-Scale simulation of everyday tasks for generalist robots. In Robotics: Science and Systems, 2024.arxiv.org/abs/2406.02523
- [29]Stone Tao, Fanbo Xiang, Arth Shukla, Yuzhe Qin, Xander Hinrichsen, Xiaodi Yuan, Chen Bao, Xinsong Lin, Yulin Liu, Tse-kai Chan, et al. ManiSkill3: GPU parallelized robotics simulation and rendering for generalizable embodied AI. arXiv preprint arXiv:2410.00425, 2024.arxiv.org/abs/2410.00425
- [30]Ning Gao, Yilun Chen, Shuai Yang, Xinyi Chen, Yang Tian, Hao Li, Haifeng Huang, Hanqing Wang, Tai Wang, and Jiangmiao Pang. GENMANIP: LLM-driven simulation for generalizable Instruction-Following manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12187–12198, June 2025.arxiv.org/abs/2506.10966
- [31]Tianxing Chen, Yuran Wang, Mingleyang Li, Yan Qin, Hao Shi, Zixuan Li, Yifan Hu, Yingsheng Zhang, Kaixuan Wang, Yue Chen, et al. RMBench: Memory-Dependent robotic manipulation benchmark with insights into policy design. arXiv preprint arXiv:2603.01229, 2026.arxiv.org/abs/2603.01229
- [32]Yinpei Dai, Hongze Fu, Jayjun Lee, Yuejiang Liu, Haoran Zhang, Jianing Yang, Chelsea Finn, Nima Fazeli, and Joyce Chai. RoboMME: Benchmarking and understanding memory for robotic generalist policies. arXiv preprint arXiv:2603.04639, 2026.arxiv.org/abs/2603.04639
- [33]Haiyang Sun, Haoxiao Wang, Junming Chen, Weicheng Fang, Zihao Su, Jingkun Yi, Wenyou Yi, Hao Chen, and Zhou Zhao. MEMOBench: A process level memory benchmark for robotic manipulation. arXiv preprint arXiv:2609.07047, 2026.arxiv.org/abs/2609.07047
- [34]Zhenxuan Fan, Bo Zhang, Yutong Lin, Yuqian Yuan, Juekai Lin, Liang Liang, Zhuoyi Huang, Wenqiao Zhang, Juncheng Li, Siliang Tang, Jun Xiao, and Yueting Zhuang. RoboSPA: Can VLA models go beyond simple scenes and Short-Horizon tasks? In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 23457–23492. Association for Computational Linguistics, 2026.arxiv.org/abs/2609.05324
- [35]Sirui Chen, Huang Huang, Qineng Wang, Wenlong Huang, Manling Li, Ruohan Zhang, Li Fei-Fei, and Jiajun Wu. BEHAVIOR-Spatial benchmark: Evaluating spatial understanding for robotics. arXiv preprint https://arxiv.org/abs/2610.34051, 2026.
- [36]Zhongbo Zhang, Jiayi Jin, Yifan Wang, Zaibin Zhang, Haiwen Diao, Lijun Wang, and Huchuan Lu. VABench: Measuring embodied spatial intelligence through visual demonstrations, active perception, and metric control. arXiv preprint https://arxiv.org/abs/2609.19554, 2026.
- [37]Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, et al. RoboRefer: Towards spatial referring with reasoning in Vision-Language models for robotics. arXiv preprint arXiv:2506.04308, 2025.arxiv.org/abs/2506.04308
- [38]Yanpeng Zhao, Wentao Ding, Hongtao Li, Baoxiong Jia, and Zilong Zheng. ESPIRE: A diagnostic benchmark for embodied spatial reasoning of Vision-Language models. arXiv preprint arXiv:2603.13033, 2026.arxiv.org/abs/2603.13033
- [39]Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, and Elia Bruni. iVISPAR—an interactive Visual-Spatial reasoning benchmark for VLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 26745–26769, 2025.arxiv.org/abs/2502.03214
- [40]Zekun Qi, Wenyao Zhang, Yufei Ding, Runpei Dong, Xinqiang Yu, Jingwen Li, Lingyun Xu, Baoyu Li, Xialin He, Guofan Fan, Jiazhao Zhang, Jiawei He, Jiayuan Gu, Xin Jin, Kaisheng Ma, Zhizheng Zhang, He Wang, and Li Yi. SoFar: Language-Grounded orientation bridges spatial reasoning and object manipulation, 2025.arxiv.org/abs/2502.13143
- [41]Kairun Wen, Yuzhi Huang, Runyu Chen, Hui Zheng, Yunlong Lin, Panwang Pan, Chenxin Li, Wenyan Cong, Jian Zhang, Junbin Lu, et al. DynamicVerse: A Physically-Aware multimodal framework for 4d world modeling. arXiv preprint arXiv:2512.03000, 2025.arxiv.org/abs/2512.03000
- [42]Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 11:635–651, 06 2023.
- [43]Yuzhi Huang, Kairun Wen, Rongxin Gao, Dongxuan Liu, Yibin Lou, Jie Wu, Jing Xu, Jian Zhang, Zheng Yang, Yunlong Lin, et al. Thinking in dynamics: How multimodal large language models perceive, track, and reason dynamics in physical 4d world. arXiv preprint arXiv:2603.12746, 2026.arxiv.org/abs/2603.12746
- [44]Ilias Stogiannidis, Steven McDonagh, and Sotirios A. Tsaftaris. Mind the gap: Benchmarking spatial reasoning in Vision-Language models, 2025.arxiv.org/abs/2503.19707
- [45]Dingming Li, Hongxing Li, Zixuan Wang, Yuchen Yan, Hang Zhang, Siqi Chen, Guiyang Hou, Shengpei Jiang, Wenqi Zhang, Yongliang Shen, Weiming Lu, and Yueting Zhuang. ViewSpatial-Bench: Evaluating multi-perspective spatial localization in Vision-Language models, 2025.arxiv.org/abs/2505.21500
- [46]Songsong Yu, Yuxin Chen, Hao Ju, Lianjie Jia, Fuxi Zhang, Shaofei Huang, Yuhan Wu, Rundi Cui, Binghao Ran, Zaibin Zhang, Zhedong Zheng, Zhipeng Zhang, Yifan Wang, Lin Song, Lijun Wang, Yanwei Li, Ying Shan, and Huchuan Lu. How far are VLMs from visual spatial intelligence? A Benchmark-Driven perspective. arXiv preprint arXiv:2509.18905, 2025.arxiv.org/abs/2509.18905
- [47]Yun Li, Yiming Zhang, Tao Lin, XiangRui Liu, Wenxiao Cai, Zheng Liu, and Bo Zhao. STI-Bench: Are mllms ready for precise spatial-temporal world understanding? arXiv preprint arXiv:2503.23765, 2025.arxiv.org/abs/2503.23765
- [48]Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. OmniSpatial: Towards comprehensive spatial reasoning benchmark for vision language models. arXiv preprint arXiv:2506.03135, 2025.arxiv.org/abs/2506.03135
- [49]Yining Hong, Jiageng Liu, Han Yin, Manling Li, Leonidas Guibas, Li Fei-Fei, Jiajun Wu, and Yejin Choi. ESI-Bench: Towards embodied spatial intelligence that closes the Perception-Action loop. arXiv preprint https://arxiv.org/abs/2605.18746, 2026.
- [50]Chih-Ting Liao, Xi Xiao, Chunlun Meng, Zhangquan Chen, Yitong Qiao, Weilin Zhou, Tianyang Wang, Xu Zheng, and Xin Cao. SpaMEM: Benchmarking dynamic spatial reasoning via Perception-Memory integration in embodied environments. arXiv preprint https://arxiv.org/abs/2604.22409, 2026.
- [51]Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GROOT n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025.
- [52]Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing AI into the physical world. arXiv preprint arXiv:2503.20020, 2025.arxiv.org/abs/2503.20020
- [53]Chenghao Gu, Haolan Kang, Junchao Lin, Jinghe Wang, Duo Wu, Shuzhao Xie, Fanding Huang, Junchen Ge, Ziyang Gong, Letian Li, et al. IGen: Scalable data generation for robot learning from Open-World images. arXiv preprint arXiv:2512.01773, 2025.arxiv.org/abs/2512.01773
- [54]Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. RoboSpatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 15768–15780, 2025.arxiv.org/abs/2411.16537
- [55]Ji Zhang, Shihan Wu, Xu Luo, Hao Wu, Lianli Gao, Heng Tao Shen, and Jingkuan Song. InSpire: Vision-Language-Action models with intrinsic spatial reasoning, 2025.arxiv.org/abs/2505.13888
- [56]Fuhao Li, Wenxuan Song, Han Zhao, Jingbo Wang, Pengxiang Ding, Donglin Wang, Long Zeng, and Haoang Li. Spatial forcing: Implicit spatial representation alignment for vision-language-action model. arXiv preprint arXiv:2510.12276, 2025.arxiv.org/abs/2510.12276
- [57]Haoxu Huang, Fanqi Lin, Yingdong Hu, Shengjie Wang, and Yang Gao. COPA: General robotic manipulation through spatial constraints of parts with foundation models. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9488–9495. IEEE, 2024.arxiv.org/abs/2403.08248
- [58]Fangchen Liu, Kuan Fang, Pieter Abbeel, and Sergey Levine. MOKA: Open-world robotic manipulation through mark-based visual prompting. arXiv preprint arXiv:2403.03174, 2024.arxiv.org/abs/2403.03174
- [59]Mayank Mittal, Pascal Roth, James Tigue, Antoine Richard, Octi Zhang, Peter Du, Antonio Serrano-Muñoz, Xinjie Yao, René Zurbrügg, Nikita Rudin, et al. Isaac Lab: A GPU-accelerated simulation framework for multi-modal robot learning. arXiv preprint arXiv:2511.04831, 2025.arxiv.org/abs/2511.04831
