betaFeedback

WorksComputer Science

SpatialCast: Reading the 3D World via Cast Shadows

Abstract

As multimodal large language models (MLLMs) advance toward understanding and interacting with the physical world, spatial reasoning becomes essential for understanding scene structure and anticipating the effects of physical changes. Existing spatial benchmarks have made substantial progress in evaluating spatial relations, memory, and viewpoint reasoning, but less systematically assess how models connect physical scene conditions with visual observations. We introduce SpatialCast, a benchmark that uses cast shadows to evaluate this ability. Cast shadows provide observable evidence of a 3D scene. Their image appearance depends on object geometry and pose, illumination, and viewpoint. We examine this relationship through three complementary tasks that assess scene-projection consistency, predict intervention-induced changes, and infer the physical changes behind visual transitions. The benchmark contains 21,000 synthetic questions with separate scene and geometry test partitions, complemented by 200 SpatialCast-Real questions from 20 photographed scenes. Evaluations show that closed-source MLLMs outperform general-purpose open-weight MLLMs and MLLMs designed for spatial reasoning, which remain near chance on the synthetic benchmark, while even strong closed-source models struggle with shadow-length and light-height judgments. We also use SpatialCast training data for GRPO post-training, improving open-weight MLLM performance on the benchmark. Further evaluations show improvements on several external spatial benchmarks, suggesting transfer of the learned capabilities. These experiments support SpatialCast as a controlled resource for diagnosing spatial reasoning capabilities and studying their learning and generalization.

Introduction

Multimodal large language models (MLLMs) are increasingly expected to move beyond interpreting images and videos toward understanding and interacting with the physical world [35]. Such interactions require models to understand spatial structures and anticipate how visual observations change as an observer moves or a scene is modified [4, 14]. Evaluating these abilities is an important step toward MLLMs that understand and interact with the 3D world. Moreover, recent perspectives on spatial intelligence have highlighted new-view prediction as a visual counterpart to next-token prediction in language modeling [9], underscoring the importance of understanding how scene conditions give rise to visual observations. However, viewpoint is only one of several physical factors that shape what we see. Changes in illumination and object pose also alter visual observations, calling for a broader evaluation of how physical scene conditions determine their 2D projections.

Existing benchmarks have examined geometric relationships, including relative positions, distances, and orientations [4, 26], as well as spatial memory, perspective-taking, and mental simulation from multiple or limited observations [31, 33]. More recent work extends these evaluations to dynamic settings, examining the effects of object and observer movements [20], projection and viewpoint transformations [21], and spatially constrained image editing [38]. However, existing benchmarks less systematically test how physical changes alter visual observations. They also provide limited evaluation of whether models can infer these changes from before-and-after observations. Cast shadows offer a natural way to probe this relationship. As shown in Figure 1, the appearance of a cast shadow depends on object geometry, pose, illumination, and their combined effects on the resulting projection. Cast shadows therefore provide observable evidence about the underlying 3D scene. By controlling the physical scene, we can systematically evaluate whether models understand the resulting projections, anticipate their changes under interventions, and infer the physical changes behind observed transitions.

Figure 1. Motivation for SpatialCast. (a) Human intuition for relating 3D form and lighting to visual appearance. (b) We ask whether MLLMs can match scenes with shadow projections, predict intervention outcomes, and infer change parameters from paired observations given the changed factor. (c) Cast shadows provide physically constrained evidence connecting scene geometry, pose, illumination, and viewpoint to observable 2D projections. The shadow footprints are schematic; benchmark observations contain both objects and their cast shadows. The figure includes “Humans mentally model 3D form and lighting,” “Top Light,” “Side Light,” “Front Light,” “Back Light,” “Can MLLMs reason through physical projection?”, “Match scene & shadow,” “Predict outcomes,” “Infer changes,” “SpatialCast connects hidden 3D scene conditions to observable shadow projections,” “Geometry,” “Pose,” “Illumination,” “Viewpoint,” “3D Scene,” “z=(G,P,L,C)z = (G, P, L, C),” “Hidden factors define the 3D scene,” “Physical Projection Process,” “Light occlusion + camera projection,” “I=R(z)I = R(z),” “Light and camera produce II from zz,” “2D Projection,” “Shadow Projections,” and “Observable 2D evidence.”

Building on this insight, we introduce SpatialCast, a benchmark that uses controlled rendering to evaluate how MLLMs reason about the physical relationship between 3D scenes and their 2D projections. By varying scene conditions while preserving their underlying physical constraints, SpatialCast provides verifiable questions that probe this relationship from three complementary directions. Projection Compatibility (T1) tests agreement between scene conditions and projections through forward prediction and inverse inference of lighting. Projection Propagation (T2) predicts the projection after a specified light, object-rotation, or viewpoint change. Projection Attribution (T3) infers change parameters from paired observations, given the factor that changed. The benchmark contains 21,000 synthetic questions across 16 shape families and three scene structures, with separate scene and geometry test partitions. SpatialCast-Real adds 200 questions from 20 photographed ReNe scenes to assess light-change reasoning in real images [24].

Our contributions are as follows.

  • Cast shadows as a physical readout for spatial reasoning. We leverage cast shadows as observable evidence of the underlying 3D scene, whose image appearance is governed by object geometry, pose, illumination, and viewpoint. We design three complementary tasks to evaluate scene–projection compatibility, predict the visual consequences of physical interventions, and infer changes from before-and-after observations.

  • A controlled benchmark spanning synthetic and real scenes. We use BlenderProc and Blender Cycles to construct controllable 3D scenes and generate 21,000 synthetic questions across 16 shape families and three scene structures, with separate test partitions for new scene instances and held-out geometric parameter ranges. We further introduce SpatialCast-Real, comprising 200 questions based on photographed lighting changes.

  • Extensive evaluation and generalization. We evaluate SpatialCast on both proprietary and open-weight MLLMs, including spatially trained baselines, revealing distinct performance profiles across models and reasoning tasks. SpatialCast-GRPO training yields model-dependent improvements, including a 23.33-percentage-point gain on Projection Propagation for the 8B model. The trained models also show improvements on several external spatial benchmarks, demonstrating the potential for cross-benchmark generalization.

Related Work

Spatial Intelligence in MLLMs. SpatialPIN and Thinking with Blueprints incorporate 3D priors or structured object representations [18, 19], while related methods use geometric primitives and bounding-box abstractions [12, 15]. Other work improves spatial reasoning through metric supervision [4], geometric surrogate tasks [11, 23], and verifiable objectives based on self-supervised spatial tasks or cross-view matching [16, 36]. SpatialLadder and Spatial-MLLM further explore progressive training and geometry-aware visual features [10, 29]. Several of these methods also provide spatially trained baselines for examining whether learned capabilities transfer to different spatial reasoning tasks.

Benchmarks for Spatial Intelligence. Existing benchmarks differ in their visual observations and reasoning targets. Spatial457 and OmniSpatial evaluate diverse spatial abilities [8] [26], while VSI-Bench and MindCube examine spatial reasoning from videos or limited views [31, 33]. Multi-SpatialMLLM and MMSI-Bench further investigate spatial understanding across multiple frames or images [30, 32]. Controlled settings enable more targeted evaluations, including state reconstruction and transformations in cube-based benchmarks [1, 7] and procedural scene complexity in InfiniBench [25]. More directly related to scene changes, SAT examines how object and observer movements affect spatial relationships [20], while SpatialBlock evaluates 3D-to-2D projection, viewpoint transformation, and structural combination through synthetic block-stacking tasks [21]. GSI-Bench assesses whether generated image edits satisfy specified spatial constraints [38], whereas CamChoreo identifies camera-motion types and temporal intervals in videos [6]. These benchmarks primarily examine spatial relationships, transformations, or visual outputs, while the physical relationship between scene changes and their observable consequences offers a complementary perspective for evaluating spatial reasoning.

Shadows and Geometric Reasoning. Cast shadows provide geometric evidence about the underlying 3D scene, as their formation depends on object geometry, pose, and illumination. Prior work exploits shadows to reconstruct plausible hidden object shapes while jointly estimating pose and lighting, showing that multiple scene explanations may be consistent with the same observation [13]. Differentiable shadow mapping further enables gradient-based optimization of physical scene parameters [28]. Other studies investigate shadow detection from a vision-language perspective [27] or optimize object pose and illumination for shadow-based visual compositions [17]. ReNe provides real photographs with calibrated light and camera poses for relighting and novel-view synthesis [24]. The ambiguity of shadow-based reconstruction also highlights the importance of specifying known scene conditions and candidate hypotheses in inverse reasoning tasks.

SpatialCast Benchmark

SpatialCast tests three inference directions through a shared projection map. Figure 2 summarizes how the synthetic benchmark is constructed and partitioned, alongside examples of real-image questions.

SpatialCast task and dataset overview

Figure 2. SpatialCast task and dataset overview. Panels (a,b) are schematic illustrations. (a) The tasks share 16 shape families and three scene structures. (b) T1 combines forward prediction and inverse inference of lighting. T2 predicts the outcome of a specified intervention. T3 infers change parameters from before/after observations and a known changed factor. The synthetic question pool contains 21,000 questions, including 4,500 test questions. (c) SpatialCast-Real contains 200 questions across 20 photographed scenes, illustrated by a T2 reference with four candidate outcomes and a T3 before/after pair. Counts refer to the complete pool before training-data screening. Experiments use the full T1/T2/T3 test scope shown here.

Scene Representation

A scene state z=(G,P,L,C)z = (G, P, L, C) specifies object geometry GG, object poses and positions PP, illumination LL, and camera configuration CC (Figure 1). A renderer RR implements the physical projection process, producing the object–shadow observation

I=R(G,P,L,C).(1)I = R(G, P, L, C). \tag*{(1)}

Scenes contain a planar shadow receiver and one or two objects. An intervention aa updates specified components of the state, producing z′=Ta(z)z' = T_a(z) and observation I′=R(z′)I' = \mathcal{R}(z'), where TaT_a is the state transformation specified by aa. Light changes and object rotations affect shadow formation. Camera changes alter the view of the resulting scene.

Geometry and Scene Construction

The geometry pool contains six primitive, four compound, and six procedural shape families, totaling 16 (Figure 2(a)). Their silhouettes and exposed surfaces produce varied responses to light and pose changes. Supplementary Sec. A.1 lists all families with example geometry instances.

The three scene structures used in each task contain a single object, two objects with separate ground shadows, or two objects with overlapping ground shadows. Overlap introduces a compositional observation in which the visible shadow boundary can depend on both objects. The same structure labels apply across tasks, supporting task-level and structure-level analysis without changing the task definitions. Scenes are rendered with BlenderProc and Blender Cycles [5].

Three Complementary Tasks

The three tasks in Figure 2(b) share the scene representation and physical projection model of § 3.1. T1 matches scene conditions to projections within a static scene. When the scene changes, T2 predicts the resulting projection given the intervention, while T3 uses the before and after observations to infer how the specified factor changed. Table 1 summarizes the inputs, prediction targets, and variants of each task.

TaskGivenSelectVariants
T1 CompatibilityGeometry and lightCompatible projectionForward
Observed shadowCompatible light settingInverse
T2 PropagationInitial view II and intervention aaResulting view I′I'L, R, C; LR, LC, RC
T3 AttributionViews II, I′I' and the changed factorParameters of the changeL, R, C

Table 1. Overview of the three SpatialCast reasoning tasks. L, R, and C denote light, object rotation, and camera viewpoint, respectively. Paired letters denote joint interventions.

T1: Projection Compatibility. T1 tests static compatibility by asking the model to select the physically consistent candidate from four alternatives. In the forward variant (Figure 3(a)), the model selects the projection consistent with the given geometry and light, using shadow direction, length, or contour. In the inverse variant (Figure 3(b)), it selects the light setting (azimuth or height) consistent with the observed shadow under fixed geometry and viewing conditions.

Figure 3. Representative SpatialCast questions. (a,b) T1 asks for a projected scene or a light configuration. (c–e) T2 specifies a light, object-rotation, or camera change. Panel (c) shows the initial observation and all A–D candidates. Panels (d,e) show the initial observation and the correct candidate. (f) T3 gives the changed factor and asks for its change parameters. Green highlights mark ground-truth answers. In the top-view diagrams, yellow circles denote the light, red circles the object, and green triangles the camera. HH denotes object height. CW and CCW refer to clockwise and counterclockwise turns viewed from above. Scene views share a fixed display crop within each question.

T2: Projection Propagation. T2 asks how an initial object–shadow observation changes under a specified intervention. Given the initial image II and intervention aa, model selects the resulting view I′=R(Ta(z))I^\prime= \mathcal{R}(T_a(z)) from four candidate images. Single-factor interventions change the light (L), rotate an object (R), or change the camera viewpoint (C). Joint interventions combine two factors (LR, LC, or RC). For scenes with two objects, the question identifies the object affected by a rotation. In Figure 3(c), the light turns 30° counterclockwise and drops by half the object’s height. Figure 3(d,Figure 3e) shows object rotation and camera viewpoint changes, respectively. Supplementary Sec. A.2 gives complete sets for every T2 type.

T3: Projection Attribution. T3 asks which change parameters explain a transition between object–shadow observations, given the factor that changed. In Figure 3(f), only the light changes. From the before-and-after observations, the model infers whether the light turned clockwise or counterclockwise and whether it rose or fell. Object-rotation and camera-viewpoint questions ask about the direction and magnitude of rotation. Providing the changed factor defines the hypothesis space for inferring the corresponding change parameters from the observed transition.

Question and candidate construction. Each synthetic question has four options. Scene metadata identifies the correct option, and models see only the images, prompt, and options. T1 fixes the objects and camera. For forward questions, we render the correct option under the light setting specified in the prompt and the three distractors under three alternative light settings. Inverse questions ask which of four light settings produced the observed image, with the options varying either in azimuth at a fixed height or in height at a fixed azimuth. T2 renders the specified intervention and three alternatives from the same initial scene. Distractors omit a factor, reverse a turn, or change its magnitude. T3 pairs each turn direction with a rise or fall for the light, or with two rotation angles for the object or camera. The four combinations form the text options, and one is sampled to generate the after image.

Quality control

During generation, we reject scenes if too little of an object or its ground shadow is visible, or if a ground shadow enters the frame margin. For two-object scenes, we measure ground-shadow overlap to verify the specified scene structure. For T3, we check that the four text options cover all combinations of two binary attributes, so each attribute value occurs twice. We render the scene for each of the four physical hypotheses, including for inverse T1 and T3 questions, whose answer options are lighting diagrams and text descriptions, respectively. For each of the six pairs of scene renders, we count pixels in the object or shadow region whose color difference meets a fixed threshold. For 97.87% of synthetic questions, even the closest pair differs at 150 or more such pixels (Supplementary Sec. A.4).

Dataset Composition and Partitions

The synthetic benchmark contains 21,000 questions. Of the 8,400 T1 questions, 4,200 use forward prediction and 4,200 use inverse inference. The 8,400 T2 questions are evenly divided between single-factor and joint interventions. The 4,200 T3 questions are approximately balanced across light changes, object rotations, and camera viewpoint changes. The dataset is partitioned into 15,000 training, 1,500 validation, 3,000 scene-test, and 1,500 geometry-test questions as summarized in Figure 2(b).

Scene-test evaluates generalization to unseen scene instances independently sampled from the same scene distribution as the training set. Geometry-test evaluates generalization to held-out geometric parameter ranges within the same shape families. The two test partitions contain 4,500 questions, with 1,800 T1, 1,800 T2, and 900 T3 questions (a 2:2:1 ratio). Supplementary Secs. A.3–A.4 provide detailed counts and split audits.

SpatialCast-Real

To complement the synthetic evaluation, we construct SpatialCast-Real from ReNe (ReLight My NeRF) [24], a photographic dataset with calibrated camera and light poses.

Each photograph is captured with one light on at a time. We group photographs by scene and camera pose, keeping object geometry, layout, and viewpoint fixed while varying illumination. From the calibrated light poses, we compute the change in lighting between photographs and use it to label each question.

SpatialCast-Real evaluates the light-change variants of T2 and T3 (Figure 2(c)). For T2, the model receives a reference photograph and the specified changes in light azimuth, height, and horizontal distance. It selects the photograph that matches these changes from four candidates. For T3, the model compares before-and-after photographs and decides whether the shortest turn between the two light azimuths is clockwise or counterclockwise. A top-view diagram establishes the coordinate convention. The dataset contains 200 questions across 20 scenes (125 T2 and 75 T3, ten per scene). Its 587 public images comprise 585 distinct photographs and two coordinate diagrams. A photograph can appear in multiple questions. We evaluate all 200 questions separately from the synthetic benchmark. Supplementary Sec. B details the construction, and Figure 8 shows complete T2 examples in panels (a,b) and T3 examples in panels (c,d).

Figure 8. Complete SpatialCast-Real questions from ReNe. Panels (a,b) show T2 candidate photographs in their original order. Panels (c,d) show T3 before/after pairs. Green marks the correct answer. CW denotes clockwise as viewed from above along −Z-Z. The top views show the world axes and calibrated camera bearing.

Experiments

We first compare benchmark performance across models and examine learning across the three reasoning directions. We then validate the benchmark through controlled input ablations and human evaluation before examining cross-benchmark transfer.

Experimental Setup

Compared Models. Our four base models are Qwen3-VL-8B Base and Qwen3-VL-4B Base [2], Qwen2.5-VL-3B Base [3], and InternVL3-2B Base (Instruct checkpoint) [37]. Spatially trained baselines include Spatial-SSRL [16], SpatialBlock [21], Spatial-MLLM [29], and SpatialLadder [10]. We evaluate all open-weight models, including our SpatialCast-GRPO variants, on the 4,500 synthetic test questions and 200 SpatialCast-Real questions. We also evaluate GPT-6 Astra, GPT-6.1 Sol, Claude Sonnet 5.5, Gemini 4 Argon, and Qwen3.8-Max (0902) on SpatialCast-Real. Of these, GPT-6.1 Sol and Gemini 4 Argon also complete the full synthetic test set.

Evaluation Protocol and Metrics. Each model receives the question images, prompt, and answer options. We use exact-match accuracy, counting unresolved or invalid answers as incorrect. We compute accuracy separately for the synthetic test set and SpatialCast-Real, weighting each question equally within its test set. The synthetic overall score is micro-averaged accuracy, with T1, T2, and T3 contributing in a 2:2:1 ratio. The uniform random-choice baselines are 25% for the synthetic tasks and 34.375% for SpatialCast-Real. We compute accuracy gains on SpatialCast from integer correct counts before rounding. Supplementary Sec. C.2 details model-specific inference settings and answer parsing.

SpatialCast-GRPO Post-training. We train each of the four base models from its original weights using Group Relative Policy Optimization (GRPO) [22]. After screening the training split, we retain 11,186 questions, comprising 1,189 forward T1, 1,998 inverse T1, 6,000 T2, and 1,999 T3 questions. All four models use the same data order and train for 1,399 updates. At each update, we sample eight responses per question and reward answer correctness and format compliance, with a length penalty. We use each model’s final checkpoint for evaluation. Supplementary Sec. C.1 details optimization, screening, and the shared training schedule. Supplementary Sec. D.2.2 describes additional 4B configurations, including the stage-ordered curriculum.

External Benchmarks. We evaluate transfer on MindCube [33], MMSI-Bench [32], and SPBench [10], with MMMU [34] as a general multimodal reference. SPBench contributes separate multiple-choice accuracy scores for single-image (SI) and multi-view (MV) questions. The aggregate transfer metric is the arithmetic mean of the five scores before rounding. Supplementary Sec. D.6 provides further evaluation details.

Benchmark Performance Across MLLMs

Overall performance. We compare existing MLLMs to assess their ability to reason about physical scene–projection relationships. Sol and Gemini achieve 79.42% and 57.71% overall on the synthetic test set, while the original open-weight models and existing spatially trained baselines remain close to chance (Table 2). This gap shows that the benchmark’s physical projection questions remain challenging for current open-weight spatial models.

ModelSynthetic (4,500)Real (200)
T1T2T3OverallOverall
GPT-6.1 Sol67.5088.3985.3379.4279.50
Gemini 4 Argon43.8966.1768.4457.7163.00
Qwen3-VL-8B Base [2]22.8925.4425.3324.4034.50
SpatialCast-GRPO (8B)30.9448.7834.2238.7336.50
Qwen3-VL-4B Base [2]23.1727.2823.6724.9133.50
Spatial-SSRL-4B [16]23.3328.5623.3325.4236.50
SpatialBlock-4B [21]24.2224.8325.5624.7334.00
SpatialCast-GRPO (4B)24.1146.8322.4432.8738.50
Qwen2.5-VL-3B Base [3]23.1126.1123.4424.3830.50
Spatial-MLLM [29]24.5025.1128.4425.5335.50
SpatialLadder-3B [10]25.0626.3926.0025.7833.00
Spatial-SSRL-3B [16]25.1127.7223.8925.9134.50
SpatialCast-GRPO (3B)26.8340.1725.2231.8436.00
InternVL3-2B Base [37]25.1726.8324.8925.7833.50
SpatialCast-GRPO (2B)27.6145.6726.3334.5834.50

Table 2. SpatialCast and SpatialCast-Real performance. Accuracy (%) on 4,500 synthetic questions (1,800 T1, 1,800 T2, 900 T3) and 200 real-image questions. Synthetic Overall is micro-averaged accuracy across the three tasks. Lavender marks SpatialCast-GRPO. Bold marks each local block’s maxima.

API models across reasoning tasks. We examine how the differences between the API models vary across reasoning tasks and physical factors (Table 3). Both are weakest on T1, while Sol peaks on T2 and Gemini on T3. Within T1, both perform better on shadow direction and light azimuth than on shadow length and light height. This shared pattern reveals a performance gap between directional judgments and judgments of length or height in this benchmark. Within T2, Sol varies less across intervention configurations, while Gemini performs better on joint LR changes than on any single-factor intervention. Performance varies across intervention configurations, revealing distinct reasoning demands beyond the number of changed factors. Within T3, Sol is strongest on camera-change attribution and Gemini on light attribution. The two models perform similarly on light attribution, while Sol’s advantages are substantially larger for object-rotation and camera-change attribution. Supplementary Sec. D.1.1 gives subtype sample sizes and correct counts by task and test partition.

ModelT1 Compatibility Dir.T1 Compatibility Len.T1 Compatibility Cont.T1 Compatibility Azim.T1 Compatibility Ht.T2 Propagation LT2 Propagation RT2 Propagation CT2 Propagation LRT2 Propagation LCT2 Propagation RCT3 Attribution LT3 Attribution RT3 Attribution C
GPT-6.1 Sol94.0241.4755.3387.5039.0080.3391.0391.3091.0089.7086.9680.0079.6096.35
Gemini 4 Argon60.4727.7633.3355.3331.0065.6764.1265.2285.0061.1355.8578.3356.5270.43

Table 3. API-model accuracy across projection reasoning subtypes. Accuracy (%) on the same 4,500 synthetic test questions, with column groups and order matching Figure 4. Dir./Len./Cont. denote shadow direction, length, and contour. Azim./Ht. denote light azimuth and height. L/R/C denote light, object-rotation, and camera viewpoint changes, and paired letters denote joint interventions.

Open-weight models and spatially trained baselines. Both the four base models and the spatially trained baselines remain close to chance overall (Table 2). The latter result characterizes transfer from existing spatial training to these physical projection questions. § 4.3 examines the improvements obtained by training directly on SpatialCast.

API models on SpatialCast-Real. We further assess whether five API models can reason about photographed lighting changes. Their overall accuracy ranges from 58.50% to 81.50%, exceeding the 34.375% random-choice expectation (Table 4). All models exceed the respective chance levels of 25% and 50% on light-change propagation (T2) and attribution (T3), supporting the answerability of these photographic questions for the API models.

ModelT2T3Overall
Random25.0050.0034.38
GPT-6 Astra77.6088.0081.50
GPT-6.1 Sol76.0085.3379.50
Claude Sonnet 5.566.4082.6772.50
Gemini 4 Argon52.8080.0063.00
Qwen3.8-Max (0902)†48.0076.0058.50

Table 4. API-model performance on SpatialCast-Real. Accuracy (%) on 125 T2 and 75 T3 questions (200 overall). Random denotes uniform guessing.

Performance across evaluation settings. SpatialCast-trained models retain improvements on both scene-test and geometry-test partitions (Supplementary Sec. D.5). For example, the 8B model gains 13.37 and 16.27 percentage points, respectively. Its accuracy on SpatialCast-Real increases by 2.00 percentage points, showing a smaller gain on real photographs than on the synthetic partitions.

Learning Gains Across Reasoning Tasks

Figure 4 reveals distinct learning behaviors across backbone models. For example, shadow-direction compatibility (T1) improves by 15.3 percentage points for 8B but remains nearly unchanged for 4B. Similarly, light attribution (T3) gains 15.7 percentage points for 8B but decreases by 1.3 percentage points for 4B. Models improve on different reasoning subtypes even under the same training setup.

Figure 4. Learning responses across projection reasoning subtypes. Signed accuracy changes after SpatialCast-GRPO relative to each model’s own Base (Δ\Delta, percentage points), computed from integer correct counts before rounding on the same 4,500-question test set. Cell values are rounded to one decimal. The 8B row uses the final 1,399-update checkpoint. Rose denotes decreases, gray near-zero changes, and teal increases. Color intensity rises with absolute change. Dir./Len./Cont. denote shadow direction, length, and contour. Azim./Ht. denote light azimuth and height. L/R/C denote light, object-rotation, and camera changes. Paired letters denote joint interventions.

Despite these differences, common trends emerge across models. All four backbones improve on every Projection Propagation (T2) subtype, whereas shadow-length compatibility (T1) shows minimal improvement and object-rotation attribution (T3) exhibits limited or inconsistent gains. These gains were measured under a training configuration that retained all T2 training questions and selected subtypes of T1 and T3.

Benchmark Validation

Shadow Evidence Ablation. To examine whether MLLMs use cast shadows for reasoning, we conduct input ablations on 240 light-change questions (T2-L), holding object geometry, pose, and camera viewpoint fixed within each question. Shadow-only removes surface shading and floor illumination variation while preserving cast shadows; No Shadow removes cast shadows from the same flat renderings; and No Initial removes the initial observation from Full Input (Supplementary Sec. D.3).

As shown in Table 5, Sol achieves 81.7% under Shadow-only, comparable to 81.3% with full inputs, but drops to 25.0% without shadows. The trained 4B, 3B, and 2B models show similar declines, from 42.9%, 35.0%, and 52.1% to 30.0%, 25.4%, and 23.3%, respectively. These consistent declines suggest that cast shadows serve as important visual evidence for reasoning about lighting changes. Removing the initial observation also reduces performance for Sol and all three GRPO models, indicating that the initial scene provides an important reference for predicting the resulting projection.

ModelFull InputShadow-onlyNo ShadowNo Initial
GPT-6.1 Sol81.381.725.041.7
Qwen3-VL-4B Base27.127.122.919.6
SpatialCast-GRPO (4B)49.242.930.028.8
Qwen2.5-VL-3B Base26.325.422.920.0
SpatialCast-GRPO (3B)34.635.025.424.6
InternVL3-2B Base24.227.124.624.2
SpatialCast-GRPO (2B)51.352.123.325.0

Table 5. Shadow evidence ablation on 240 T2-L questions. Original-label agreement (%); shadow-erased scores are diagnostic. See Supplementary Sec. D.3.

Human answerability. Five participants achieve 65.00% mean accuracy on a 32-question pilot, exceeding the 26.56% random-choice expectation (Table 6). Every question receives at least one correct response, providing initial evidence of question answerability (Supplementary Sec. D.4).

32-item pilotT1T2T3RealOverall
Human mean60.0063.3366.6780.0065.00
GPT-6.1 Sol50.0075.0083.3350.0065.63
Gemini 4 Argon50.0066.6750.0075.0059.38
Uniform random25.0025.0025.0037.5026.56

Table 6. Human and API references on the 32-question pilot. Accuracy (%). Human mean averages five participants. Real denotes SpatialCast-Real. Uniform random accounts for two binary real-image T3 items. All other items have four choices.

Cross-Benchmark Transfer

External benchmark transfer. Table 7 shows that SpatialCast training yields selective improvements on external spatial benchmarks. The average accuracy increases from 37.51% to 39.12% for 8B, 36.10% to 40.34% for 4B, and 36.87% to 37.69% for 3B. For 2B, the five-metric average decreases from 37.36% to 35.22%. All four models improve on MindCube and SPBench-SI. On SPBench-MV, accuracy rises by 8.19 percentage points for 4B, holds steady for 8B, and falls by 1.63 points for 3B and 13.11 points for 2B. These results suggest that learning from physical projections can benefit broader spatial reasoning, but the transfer remains model- and task-dependent.

ModelMCMMSISIMVMMMUAvg.
Qwen3-VL-8B Base35.3325.7040.0532.7953.6737.51
SpatialCast-GRPO (8B)42.2927.8041.0632.7951.6739.12
Qwen3-VL-4B Base33.8125.0038.5431.1552.0036.10
SpatialCast-GRPO (4B)40.3827.0042.0739.3452.8940.34
Qwen2.5-VL-3B Base35.3328.6035.5237.7047.2236.87
SpatialCast-GRPO (3B)41.2426.5039.5536.0745.1137.69
InternVL3-2B Base45.5224.1029.9740.9846.2237.36
SpatialCast-GRPO (2B)47.6222.8033.5027.8744.3335.22

Table 7. External transfer before and after SpatialCast training. Accuracy is reported as a percentage. MC denotes MindCube, and SI/MV denote SPBench single-/multi-view. Avg. averages the five scores before rounding. Bold marks the higher value in each Base/trained pair, including ties. Lavender marks SpatialCast-GRPO. Results use greedy decoding. Full comparisons and numerical results appear in Supplementary Sec. D.6.

Conclusion

We introduced SpatialCast, a benchmark that uses cast shadows as a physical readout of the 3D world to evaluate how MLLMs reason about the relationship between scene conditions and visual observations. By examining compatibility, propagation, and attribution through a shared projection process, SpatialCast reveals distinct reasoning capabilities that conventional aggregate evaluations can obscure. Our experiments show substantial differences across models and reasoning directions, with GRPO-based training consistently producing the largest gains in propagation across three model families. These improvements extend to held-out scenes and geometric conditions, while evaluations on real images and external benchmarks reveal varying degrees of generalization. More broadly, SpatialCast treats shadows not merely as visual patterns to recognize, but as physical evidence for probing whether MLLMs can reason about the causes and consequences of changes in the 3D world.

References

Supplementary Contents

Synthetic Benchmark Details

Shape Families and Coverage

Across the synthetic pool, each of the 16 families in Figure 5 appears in 1,757–2,461 questions before training selection. A question with two different families counts toward both.

Gallery of all 16 synthetic shape families, grouped into primitive families (6): Cuboid, Cube, Sphere, Ellipsoid, Cylinder, Cone; compound families (4): L-shape, T-shape, Stepped, Arch; and procedural families (6): Triangular prism, Hexagonal prism, Square pyramid, Dodecahedron, Frustum, Oblique cylinder

Figure 5. All 16 synthetic shape families. (a) Six primitive, (b) four compound, and (c) six procedural families. Each panel renders an accepted dataset geometry with its original mesh parts and proportions. Gallery views use a common neutral material, lighting, orthographic camera, and pose, with uniform per-mesh scaling for display.

Supplementary T2 Examples

Figure 6 shows all four candidates for the main paper’s light (LL), object rotation (RR), and camera viewpoint (CC) examples. Figure 7 shows the joint interventions LRLR, LCLC, and RCRC. Both figures preserve the original candidate order.

Single-factor intervention questions: (a) T2 L: Light. Light: CCW 30°; lower by 0.5H. Horizontal distance stays fixed. Which scene follows? Initial A B C D. (b) T2 R: Object rotation. Object: CCW 55°. Its center stays fixed. Which scene follows? Initial A B C D. (c) T2 C: Camera. Camera: CCW 28°. Height and orbit radius stay fixed. Which scene follows? Initial A B C D. H: object height. Turns are viewed from above. Unspecified factors and object positions stay fixed. Light orbit radius is fixed. Camera height and radius are fixed; the camera faces the object center. Green: correct answer.

Figure 6. Single-factor intervention questions with complete candidate sets. Each row gives the specified intervention, the initial observation, and four candidate outcomes. Green marks the correct candidate.

Joint intervention questions: (a) T2 LR: Light + object rotation. Light: CW 30°; raise by 0.5H. Object: CW 70°. Which scene follows? Initial A B C D. (b) T2 LC: Light + camera. Light: CCW 40°; lower by 0.5H. Camera: CCW 18°. Which scene follows? Initial A B C D. (c) T2 RC: Object rotation + camera. Object: CW 55°. Camera: CCW 28°. Which scene follows? Initial A B C D. H: object height. Turns are viewed from above. Unspecified factors and object positions stay fixed. Light orbit radius is fixed. Camera height and radius are fixed; the camera faces the object center. Green: correct answer.

Figure 7. Joint intervention questions. The three rows illustrate light plus object rotation (LRLR), light plus camera (LCLC), and object rotation plus camera (RCRC). The correct outcome must satisfy both specified changes. Green marks the correct candidate. The remaining candidates include omitted or reversed intervention components.

Dataset Splits and Complete Counts

Scene-test uses independently generated instances from the training sampler. Geometry-test uses disjoint, shape-specific geometric parameter ranges within 13 shape families. Spheres, cubes, and regular dodecahedra are excluded because no geometry-holdout parameter is defined for them. Both objects in two-object scenes must satisfy the held-out ranges. For example, the revised generator samples the cuboid height-to-maximum-base-extent ratio from [1.2,1.6][1.2, 1.6] in-domain and [1.9,2.3][1.9, 2.3] for geometry-test. Table 8 gives the counts for each subtype. T1 forward questions ask about shadow contour, direction, or length. Inverse questions ask about light azimuth or height.

Table 8. Synthetic split counts before training selection. Each subtype covers single objects and object pairs with separate or overlapping shadows. Scene and Geometry denote test splits.

SubtypeTrainValSceneGeometryTotal
T1 Contour1,00298201991,400
T1 Direction9981002011001,399
T1 Length1,0001021981011,401
T1 Light azimuth1,9981984021982,796
T1 Light height1,0021021981021,404
T2 L9991001991011,399
T2 R1,000102203981,403
T2 C1,001981981011,398
T2 LR998991991011,397
T2 LC1,0021012011001,404
T2 RC1,000100200991,399
T3 L998101201991,399
T3 R1,0011001981011,400
T3 C1,001992011001,401
Total15,0001,5003,0001,50021,000

Table 8.

Quality Checks

Across the 21,000 synthetic questions, each question ID occurs in exactly one of the four partitions in Table 8. We assess visual separation using the same criterion across all four partitions. For each of the six pairs of candidate scene renders, we count pixels in the object or shadow region where at least one 8-bit RGB channel differs by 12 or more. In 20,552 of 21,000 questions (97.87%), even the closest pair differs at 150 or more such pixels. See Appendix § 8.4.§ 1 for training selection.

SpatialCast-Real Construction

Source Data and Ground-Truth Derivation

We use single-light photographs from all 20 ReNe scenes [24]. Each scene contributes five questions from camera 00 and five from camera 20, giving 100 questions per camera and 200 overall (125 T2 and 75 T3). The questions use 585 distinct photographs and two top-view diagrams. Within each question, the scene and camera are fixed while the light moves.

In ReNe’s world coordinates, OO is the origin and +Z+Z points upward. For a calibrated light position (x,y,z)(x,y,z), the azimuth is θ=atan2⁡(y,x)\theta= \operatorname{atan2}(y,x), the height is h=zh = z, and the horizontal radius is ρ=x2+y2\rho= \sqrt{x^2+y^2}. Azimuth increases counterclockwise from +X+X toward +Y+Y when viewed from above. For a reference and target photograph, we compute the shortest azimuth turn and the percentage changes in height and radius as follows.

Δθ=wrap⁡[−180∘,180∘)(θtarget−θref),Δh%=100(htarget/href−1),Δρ%=100(ρtarget/ρref−1).(2)\begin{aligned} \Delta\theta&= \operatorname{wrap}_{[-180^\circ,180^\circ)}(\theta_{\mathrm{target}}-\theta_{\mathrm{ref}}),\\ \Delta h\% &= 100(h_{\mathrm{target}}/h_{\mathrm{ref}}-1),\\ \Delta\rho\% &= 100(\rho_{\mathrm{target}}/\rho_{\mathrm{ref}}-1). \tag*{(2)} \end{aligned}

For T2, Eq. (2) supplies the three changes in the prompt. The photograph recorded at the target light position is the correct candidate. For T3, the sign of Δθ\Delta\theta determines the turn direction. Positive values indicate counterclockwise turns and negative values indicate clockwise turns. Height and radius may also change between the two photographs.

Question Examples and Quality Control

Figure 8 shows two complete examples of each task. Across the dataset, the correct T2 option appears in positions A/B/C/D 32/31/31/31 times. T3 has 38 counterclockwise and 37 clockwise turns.

We check all six candidate pairs in each T2 question (750 pairs overall). The minimum azimuth separation is 45∘45^\circ. Within a fixed region of interest on 256×192256 \times192 previews, each pair must have a normalized mean absolute difference of at least 0.10, with at least 25% of pixels differing by 0.12 or more. Blockwise structural similarity (block-SSIM) must be at most 0.82. We retain the light and camera records, question IDs, candidate mappings, and image hashes for reconstruction.

Training and Evaluation Details

Data Selection and GRPO Optimization

We retain 11,186 of 15,000 training questions, comprising 1,189 forward T1 questions screened for visually distinct candidates, 1,998 inverse T1 questions on light azimuth, all 6,000 T2 questions, and 1,999 T3 questions on light or camera changes. All four backbones use the same shuffle (seed 1) for 1,399 updates of eight questions, repeating six questions to fill the last batch.

GRPO samples eight responses per question and rewards correctness (0.9) and format compliance (0.1). For correct responses, the length penalty rises linearly from zero at 1,536 tokens to 0.1 at 2,048 tokens. All four backbones freeze the visual tower and use AdamW (constant learning rate 10−610^{-6}, weight decay 0.01), rollout temperature 1.0, a 2,048-token limit, KL coefficient 0.01, and clipping ratios 0.2/0.3. Appendix D.2.2 compares the additional 4B configurations.

Model Evaluation and Answer Parsing

Local inference. For the 2B, 3B, 4B, and 8B comparisons, the base and trained checkpoints of each backbone use the same questions, prompts, and image processing. We use greedy decoding with a 2,048-token output limit. InternVL3-2B uses its native image processor and chat template with a direct-answer prompt. Accuracy counts unresolved or invalid answers as incorrect.

API inference. We evaluate all five API models on the 200 SpatialCast-Real questions, and GPT-6.1 Sol and Gemini 4 Argon on the synthetic test. OpenRouter requests include original-resolution images and ask for the option letter. We use medium reasoning effort, an 8,192-token output limit, and omit the temperature parameter. The model identifiers are openai/gpt-6-astra, openai/gpt-6.1-sol, anthropic/claude-sonnet-5.5, google/gemini-4-argon, and qwen/qwen3.8-max-0902. The synthetic prompts interleave images at their placeholders and retain option labels (e.g., A–D), without revealing the correct answers.

Answer parsing. One deterministic parser scores all five API models. It first checks the entire response for an allowed option letter or exact option text, then checks the last nonempty line if needed. It accepts optional brackets, terminal punctuation, balanced Markdown emphasis, an answer prefix, or a complete <answer> tag. Each question is scored once. For repeated requests, we use the last response. Appendix § 8.5.1.2 gives the aggregation rule and correct counts.

Supplementary Results and Metrics

Complete Benchmark Results

API Synthetic Evaluation

Table 9 gives correct counts by task, task group, and test partition for the same 4,500 questions. Its caption gives the sample sizes for the fourteen subtype accuracies in the main paper. All questions enter the denominator, including four invalid final choices for Gemini and two empty responses for Sol.

Task / splitGPT-6.1 SolGemini 4 Argon
T1 (1,800)1,215790
T2 (1,800)1,5911,191
T3 (900)768616
Total (4,500)3,5742,597
T1 forward (900)573365
T1 inverse (900)642425
T2 single-factor (900)788585
T2 joint-factor (900)803606
Scene-test (3,000)2,373 (79.10%)1,716 (57.20%)
Geometry-test (1,500)1,201 (80.07%)881 (58.73%)

Table 9.

Local-Model SpatialCast-Real Results

SpatialCast-Real contains 125 T2 and 75 T3 questions. Its overall accuracy is 100(CT2+CT3)/200100(C_{\mathrm{T2}} + C_{\mathrm{T3}})/200, where CT2C_{\mathrm{T2}} and

Table 9. API synthetic correct counts by task and partition. Split rows also report accuracy. Subtype sizes in the main comparison are 301/299/300/600/300 for T1 direction/length/contour/azimuth/height, 300/301/299/300/301/299 for T2 L/R/C/LR/LC/RC, and 300/299/301 for T3 L/R/C.

CT3C_{\mathrm{T3}} count correct T2 and T3 answers, respectively. Table 10 reports the local models’ task scores. The T2/T3 correct counts are 97/66 for GPT-6 Astra, 95/64 for GPT-6.1 Sol, 83/62 for Claude Sonnet 5.5, 66/60 for Gemini 4 Argon, and 60/57 for Qwen3.8-Max (0902).

ModelT2 (125)T3 (75)Overall (200)
Random25.0050.0034.38
Qwen3-VL-8B
Qwen3-VL-8B Base [2]25.6049.3334.50
SpatialCast-GRPO (8B)28.0050.6736.50
Qwen3-VL-4B
Qwen3-VL-4B Base [2]24.0049.3333.50
Spatial-SSRL-4B [16]28.0050.6736.50
SpatialBlock-4B [21]24.0050.6734.00
SpatialCast-GRPO (4B)30.4052.0038.50
Qwen2.5-VL-3B
Qwen2.5-VL-3B Base [3]24.8040.0030.50
Spatial-MLLM [29]26.4050.6735.50
SpatialLadder-3B [10]22.4050.6733.00
Spatial-SSRL-3B [16]24.8050.6734.50
SpatialCast-GRPO (3B)28.0049.3336.00
InternVL3-2B
InternVL3-2B Base23.2050.6733.50
SpatialCast-GRPO (2B)24.8050.6734.50

Table 10.

Table 10. Local-model performance on SpatialCast-Real. Accuracy (%) on 125 T2 and 75 T3 questions. Overall covers all 200. Random is the uniform-choice expectation. Blue marks SpatialCast-GRPO. Each trained model is compared with its own Base.

Learning Responses and Training Configurations

Subtype Gains and Comparison Scope

Table 11 gives the correct counts for each backbone before and after training. For example, Qwen3-VL-8B gains 420 correct T2 answers on 1,800 questions (23.33 percentage points). Across all four backbones, accuracy increases on each of the six T2 subtypes. The shared training mixture includes all 6,000 T2 questions (53.64% of the 11,186 retained items) and selected T1 and T3 subtypes. The gains therefore describe learning under this training distribution.

Model pairT1T2T3Total
Items1,8001,8009004,500
8B Base4124582281,098
8B trained5578783081,743
4B Base4174912131,121
4B trained4348432021,479
3B Base4164702111,097
3B trained4837232271,433
2B Base4534832241,160
2B trained4978222371,556

Table 11.

Table 11. Exact synthetic correct counts. Trained denotes each backbone’s final SpatialCast-GRPO checkpoint.

Additional 4B Training Configurations

Table 12. Post-training configurations on the same 4B base. All configurations use the same test sets and direct-answer protocol. The question count gives the number of distinct training items. Bold marks column maxima. Both panels list the same 4B configurations; Real denotes SpatialCast-Real.

4B configurationQuestionsSteps
Qwen3-VL-4B Base––
Curriculum15,0001,875
Full shuffle15,0001,875
SpatialCast-GRPO (4B)11,1861,399
T2-only6,000750
4B configurationT1T2T3OverallReal Overall
Qwen3-VL-4B Base23.1727.2823.6724.9133.50
Curriculum23.8342.1723.3331.0735.00
Full shuffle24.3943.0625.8932.1636.00
SpatialCast-GRPO (4B)24.1146.8322.4432.8738.50
T2-only22.8339.0625.8929.9339.00

Table 12.

Across the four runs in Table 12, T2 shows the largest synthetic task gain over the 4B base. The curriculum processes T1, T2, and T3 in sequence, preserving within-task order and ending the stages at updates 750, 1,500, and 1,875. Both curriculum and full shuffle use 15,000 questions and 1,875 updates. Comparisons with SpatialCast-GRPO also vary the training mixture and update budget.

Cast-Shadow Ablation Settings

Evaluation set. All conditions use the same 240 T2-L questions, original answer labels, and candidate order, with inference settings held fixed for each model.

Ablation conditions. Shadow-only replaces object appearance with uniform colors and modulates a constant floor color by the original cast-shadow attenuation. It retains object masks and shadow geometry while removing surface shading and floor illumination variation. No Shadow uses the same colors, masks, and compositing procedure with shadow attenuation removed; the two conditions differ only in the retained cast shadows. Both edits apply to the initial image and all four candidates. No Initial removes the initial image and its prompt marker from Full Input while retaining the intervention description and all four candidate images.

Interpretation. Removing shadows can make candidates visually identical (Figure 9); No Shadow therefore reports original-label agreement after evidence removal, not accuracy on uniquely answerable questions. The matched Shadow-only versus No Shadow comparison isolates cast-shadow evidence, whereas Full Input versus Shadow-only also changes surface shading and floor illumination.

Five images showing initial and candidates A–D, with shadow-only and no-shadow rows

Figure 9. Shadow-only and its matched shadow-erased control. A T2-L example is shown in initial/A–D order. Shadow-only (upper row) preserves cast-shadow differences on fixed object and floor colors. No Shadow (lower row) removes these differences, making all five images identical.

Human Performance on a 32-Question Pilot

Five participants answered the same 32 questions against a fixed key. The booklet pairs one single-object and one two-object scene-test question for each of the fourteen synthetic subtypes, plus two T2 and two T3 questions from SpatialCast-Real. Participants answered 104/160 correctly (65.00%, compared with a 26.56% uniform-choice expectation), and each of the 32 questions received at least one correct response (Table 13). These results provide initial evidence of answerability on this booklet.

T1T2T3RealOverall
Items10126432
P150.0083.3333.3375.0062.50
P260.0058.3350.00100.0062.50
P370.0058.3383.3375.0068.75
P470.0066.67100.0075.0075.00
P550.0050.0066.6775.0056.25
Human mean60.0063.3366.6780.0065.00
Random25.0025.0025.0037.5026.56

Table 13. Five participants on the 32-question pilot. Accuracy (%). Item counts apply to each participant. Human mean aggregates 104 correct responses across five participants answering the same 32 items. Random is the uniform-choice expectation, accounting for the two binary real-image T3 questions.

Performance on Held-Out Test Partitions

All four backbones gain accuracy on both held-out partitions (Table 14). For each checkpoint, the two split counts sum to the synthetic total in Table 11.

ModelScene-testGeometry-test
Qwen3-VL-8B738 → 1,139360 → 604
Qwen3-VL-4B751 → 1,004370 → 475
Qwen2.5-VL-3B726 → 951371 → 482
InternVL3-2B779 → 1,040381 → 516

Table 14. Correct answers before and after SpatialCast training. Each entry gives base → trained counts on the same scene-test (3,000 questions) or geometry-test (1,500 questions) partition.

External Benchmark Results

External Benchmark Results

Complete Greedy Model Comparison

Table 15 reports greedy evaluation on five external metrics. The average weights the five metrics equally. Relative to their own bases, all four trained backbones improve on MindCube and SPBench single-image multiple choice. Transfer on the other metrics varies by model.

ModelMCMMSISIMVMMMUAvg.
Qwen3-VL-8BSPBench
Base [2]35.3325.7040.0532.7953.6737.51
SpatialCast-GRPO42.2927.8041.0632.7951.6739.12
Qwen3-VL-4B
Base [2]33.8125.0038.5431.1552.0036.10
Spatial-SSRL [16]28.2927.5037.7832.7952.6735.81
SpatialBlock [21]49.2424.0041.5636.0747.1139.60
SpatialCast-GRPO40.3827.0042.0739.3452.8940.34
Qwen2.5-VL-3B
Base [3]35.3328.6035.5237.7047.2236.87
SpatialBlock [21]50.2926.1035.5236.0743.3338.26
Spatial-MLLM [29]37.2423.6034.7647.5426.6733.96
SpatialLadder [10]47.8126.5052.1454.1044.8945.09
Spatial-SSRL [16]31.0524.9035.5236.0747.2234.95
SpatialCast-GRPO41.2426.5039.5536.0745.1137.69
InternVL3-2B
Base [37]45.5224.1029.9740.9846.2237.36
SpatialCast-GRPO47.6222.8033.5027.8744.3335.22

Table 15. Complete external evaluation (%). MC denotes Mind-Cube. SPBench SI/MV report single-/multi-view multiple-choice accuracy. Avg. averages the five scores before rounding. Bold marks each block’s best values, including ties. All reported GRPO-trained models use SpatialCast-GRPO. Numerical-question results appear in Table 16.

Additional Benchmarks and Answer Formats

Table 16 reports VSI-Bench question-type macro accuracy [31] and SPBench mean relative accuracy (MRA) on single-image and multi-view numerical questions. All three metrics are reported separately from the five-metric average.

ModelVSI macroSI MRAMV MRA
Qwen3-VL-8B
Base [2]43.1256.1165.97
SpatialCast-GRPO43.3356.5065.43
Qwen3-VL-4B
Base [2]39.9354.2664.81
Spatial-SSRL [16]39.5555.5667.56
SpatialBlock [21]37.2145.2946.12
SpatialCast-GRPO39.4552.3464.15
Qwen2.5-VL-3B
Base [3]24.2232.1128.26
SpatialBlock [21]25.5214.6626.47
Spatial-MLLM [29]44.3052.7863.91
SpatialLadder [10]39.7158.0469.42
Spatial-SSRL [16]25.3333.4026.55
SpatialCast-GRPO23.9634.0733.84
InternVL3-2B
Base [37]17.8428.9910.00
SpatialCast-GRPO18.9227.437.67

Table 16. Additional external metrics (%). VSI macro averages over question types. SI and MV MRA score single-image and multi-view numerical answers on SPBench. Higher values are better in each column.

References

  1. [1]Dhruv Anand and Ehsan Shareghi. Cube bench: A benchmark for spatial visual reasoning in mllms. arXiv preprint arXiv:2512.20595, 2025. 2arxiv.org/abs/2512.20595
  2. [2]Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., … Zhu, K. (2025). Qwen3-vl technical report.
  3. [3]Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. 6, 15, 16, 17arxiv.org/abs/2502.13923
  4. [7]Huan-ang Gao, Zikang Zhang, Tianwei Luo, Kaisen Yang, Xinzhe Juan, Jiahao Qiu, Tianxing Chen, Bingxiang He, Hao Zhao, Hao Zhou, Shilong Liu, and Mengdi Wang. Cubebench: Diagnosing interactive, long-horizon spatial reasoning under partial observations. arXiv preprint arXiv:2512.23328, 2025. 2arxiv.org/abs/2512.23328
  5. [8]Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. In International Conference on Learning Representations, 2026. 2arxiv.org/abs/2506.03135
  6. [10]Hongxing Li, Dingming Li, Zixuan Wang, Yuchen Yan, Hang Wu, Wenqi Zhang, Yongliang Shen, Weiming Lu, Jun Xiao, and Yueting Zhuang. Spatialladder: Progressive training for spatial reasoning in vision-language models, 2025. 2, 6, 15, 16, 17arxiv.org/abs/2510.08531
  7. [11]Shijie Lian, Changti Wu, Laurence Tianruo Yang, Hang Yuan, Bin Yu, Lei Zhang, and Kai Chen. Euclid’s gift: Enhancing spatial perception and reasoning in vision-language models via geometric surrogate tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Findings, pages 9824–9835, 2026. 2arxiv.org/abs/2509.24473
  8. [12]Junze Liu, Kun Qian, Florian Dubost, Kai Zhong, Arvind Srinivasan, Nan Chen, Anping Wang, Sam Zhang, Alejandro Mottini, Qingjun Cui, and Tian Wang. 3d primitives are a spatial language for vlms. arXiv preprint arXiv:2605.12586, 2026. 2arxiv.org/abs/2605.12586
  9. [15]Liu, Y., Zhang, F., Zhou, K., Du, Y., Liang, P. P., & Pfister, H. (2026). Abstract 3d perception for spatial intelligence in vision-language models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 38647–38656.
  10. [16]Liu, Y., Zhang, B., Zhang, Y., Cao, Y., Xing, L., Dong, X., Duan, H., Lin, D., & Wang, J. (2026). Spatial-ssrl: Enhancing spatial understanding via self-supervised reinforcement learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9570–9581.
  11. [18]Ma, C., Lu, K., Cheng, T.-Y., Nigroni, N., & Markham, A. (2024). Spatialpin: Enhancing spatial reasoning capabilities of vision-language models through prompting and interacting 3d priors. Advances in Neural Information Processing Systems.
  12. [19]Weijian Ma, Shizhao Sun, Tianyu Yu, Ruiyu Wang, Tat-Seng Chua, and Jiang Bian. Thinking with blueprints: Assisting vision-language models in spatial reasoning via structured object representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Findings, pages 8919–8929, 2026. 2arxiv.org/abs/2601.01984
  13. [22]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. 6arxiv.org/abs/2402.03300
  14. [23]Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, and Jinhua Zhao. Sparkle: Mastering basic spatial capabilities in vision language models elicits generalization to spatial reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 4083–4103, 2025. 2arxiv.org/abs/2410.16162
  15. [29]Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence, 2026. 2, 6, 15, 16, 17arxiv.org/abs/2505.23747
  16. [30]Runsen Xu, Weiyao Wang, Hao Tang, Xingyu Chen, Xiaodong Wang, Fu-Jen Chu, Matt Feiszli, and Kevin J. Liang. Multi-spatialmllm: Multi-frame spatial understanding with multi-modal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. 2arxiv.org/abs/2505.17015
  17. [32]Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenmeng Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, Dahua Lin, Tai Wang, and Jiangmiao Pang. Mmsi-bench: A benchmark for multi-image spatial intelligence, 2026. 2, 6arxiv.org/abs/2505.23764
  18. [34]Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Zheng, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., … Chen, W. (2024). Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi.
  19. [36]Hao Zhong, Muzhi Zhu, Sheny an Zeng, Anzhou Li, Cong Chen, Hua Geng, Duochao Shi, Wentao Ye, Tao Lin, Hao Chen, and Chunhua Shen. Eliciting complex spatial reasoning in mllms through wide-baseline matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16768–16778, 2026. 2arxiv.org/abs/2606.03577
  20. [37]Zhu, J., Zhang, W., Chen, Z., Liu, S. Y., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., Gao, Z., Cui, E., Wang, X., Cao, Y., Liu, Y., Wang, J., Wei, H., Zhang, H., Wang, W., Xu, H., … Wang, W. (2025). Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.

Paper details

vlnc:W_hpjwmunc
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents