WorksComputer Science
EarthJev: Efficient Visual Evidence Acquisition and Decision for UHR Remote Sensing Understanding
Abstract
Ultra-high-resolution (UHR) remote sensing visual question answering (VQA) requires locating fine-grained evidence in large scenes under limited visual input budgets. Active observation addresses this challenge through selective exploration, but repeated generative judgments can make region search costly. Its decisions also concern distinct prediction targets: spatial pointer validity, evidence presence, and reasoning correctness. We propose EarthJev, a framework that separates region scoring from answer generation and uses target-specific decision signals while keeping the answering model frozen. A spatial pointer verifier uses visual and geometric cues to guide exploration or bypass search. An evidence-aware decision mechanism scores compact candidate views in batches using a lightweight head on frozen representations, without generating textual region judgments. Format-specific confidence calibration then guides selection between candidate answers generated from memory and overview presentations of the acquired evidence. Across five benchmarks, EarthJev achieves a mean accuracy of 53.06% with Qwen3.5-9B, outperforming ZoomSearch with the same answering model by 2.33 to 14.73 percentage points per benchmark while reducing mean latency from 11.30 - 1.67 seconds per question. With four additional frozen answering models, model-specific fitting of the decision modules and calibration mappings yields mean accuracy gains of 3.41 - 6.53 percentage points over the corresponding single-pass baselines. Code is available at https://github.com/Da1daidaidai/EarthJev
Introduction
Ultra-high-resolution (UHR) remote sensing images capture both broad geographic context and fine-grained local details. requires identifying small objects, distinguishing subtle attributes, and resolving spatial relations while retaining relevant context. Multimodal large language models (MLLMs) [1, 2, 3, 4] operate under limited visual token budgets: global downsampling obscures small targets, whereas dense tiling introduces prohibitive computational overhead and irrelevant background tokens [5, 6, 7]. Consequently, acquiring sufficient visual evidence under a constrained budget remains a central bottleneck in UHR remote sensing visual question answering (VQA).
Pengyu Dai and Yao Zhang contributed equally to this work. Pengyu Dai, Naoto Yokoya, Yafei Ou, Hongruixuan Chen, Junshi Xia, and Jian Song are with RIKEN Center for Advanced Intelligence Project (AIP), RIKEN, Tokyo, Japan. Yao Zhang is with the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China. Junjue Wang, Pengyu Dai, Yuzeng Chen and Naoto Yokoya are with the University of Tokyo, Japan. Corresponding author: Naoto Yokoya ([email protected])
Active observation addresses this challenge through question-guided region exploration [8, 9, 10, 11]. By selectively zooming into candidate regions guided by the question, a model can gather fine-grained details without processing the entire scene at full resolution. Yet, moving from a linguistic query to local visual evidence and finally to an answer requires three distinct judgments (Figure 1): Is the question-provided spatial pointer trustworthy? Does a candidate region contain relevant evidence? Is the answer derived from the acquired evidence reliable? Through preliminary experiments across existing remote sensing VQA benchmarks [5], [6], [12, 13, 14, 15, 16], we find that conflating or misjudging these stages creates three critical bottlenecks:
Figure 1. Challenges in active observation: ambiguous spatial cues, misleading search history, and uncertain answers despite relevant evidence. Cyan marks target-relevant regions and evidence, red indicates distractors and off-target search, and yellow boxes delineate crop regions.
Spatial pointer validity: Explicit spatial pointers, such as bounding boxes and text coordinates, provide location priors that may be inconsistent with the visual evidence. In a controlled LRS-VQA test, boxes placed on annotated evidence improve accuracy, whereas boxes placed away
from it reduce accuracy below the no-box condition (Figure 2(a)).

Figure 2. Preliminary diagnostics of active observation: (a) Spatial cues: valid cues aid perception whereas wrong cues degrade it; (b) Evidence coverage: heuristic search (ZS) and Yes/No probing (YN) frequently miss target evidence; and (c) Answer scoring: answer confidence predicts answer correctness far better than evidence scores.
Suboptimal region-relevance probing: Existing search heuristics and zero-shot Yes/No token probabilities from MLLMs frequently fail to cover true evidence-bearing regions under fixed visual input (Figure 2(b)) and incur high latency when invoked repeatedly during visual region search.
Disconnect between evidence presence and answer correctness: Locating the relevant region does not guarantee correct fine-grained perception or relational reasoning. As shown in Figure 2(c), region-level evidence-presence scores poorly separate correct from incorrect final answers, whereas calibrated answer confidence provides a much stronger decision signal.
These observations indicate that effective active observation cannot rely on a single undifferentiated heuristic; instead, pointer validity, evidence presence, and answer correctness should be modeled as separate, calibrated decision events.
Based on these insights, we propose EarthJev, a decision-enhanced framework for UHR remote sensing VQA. With the answering model kept frozen, EarthJev employs visual verification to regulate the guidance of spatial cues during the search, utilizes a lightweight region decision head to acquire relevant evidence, and selects the final answer based on independently calibrated answer confidence scores. We evaluate EarthJev on five remote sensing image benchmarks. The results demonstrate that EarthJev achieves the highest overall accuracy compared with other methods. These findings demonstrate that improving remote sensing VQA under limited visual budget depends not only on acquiring relevant evidence but also on appropriately evaluating and utilizing that evidence, thereby translating effective observation into accurate answers(Figure 3).
Figure 3. Comparison of visual evidence acquisition paradigms. ZoomSearch employs crop-based search, while GazeEarth uses non-uniform resizing. Both may focus on incomplete or misleading evidence, resulting in incorrect answers in this example. In contrast, EarthJev verifies spatial cues, reassesses candidate views during search, and selects among candidate answers using calibrated confidence, enabling more accurate identification.
Our main contributions are as follows:
Decision targets for active observation. We characterize the distinct roles of spatial pointer validity, evidence presence, and answer correctness through controlled pointer perturbations and diagnostic score comparisons.
Decision-Enhanced Framework: We propose EarthJev, which integrates Spatial Pointer Verification (SPV), an Evidence-Aware Decision Mechanism (EDM), and Calibrated Answer Selection (CAS) on top of a frozen MLLM to jointly optimize where to look, when to stop, and how to select the final answer under a strict visual budget.
Superior Accuracy and Efficiency: Extensive evaluations on five UHR remote sensing VQA benchmarks across five MLLM backbones show that EarthJev achieves state-of-the-art accuracy with 2.3–6.8× faster inference than search-based active perception baselines.
The remainder of this paper is organized as follows. Section II reviews related work on MLLMs for remote sensing VQA, active observation, and decision models. Section III presents the proposed EarthJev framework. Section IV describes the experimental setup and presents benchmark comparisons, ablation studies, cross-model evaluations, analyses of the decision modules and inference efficiency, and qualitative case studies. Finally, Section V concludes the paper.
Related Works
MLLMs for Remote Sensing VQA
Compared with natural imagery, remote sensing scenes exhibit extreme scale variations and dense distributions of small objects [17, 18]. Understanding such imagery requires integrating broad geographic context with local spatial relationships, placing heavy demands on both fine-grained perception and domain-specific knowledge in MLLMs [19, 20, 21, 22, 23, 24, 25]. To adapt MLLMs to remote sensing domains, GeoChat [26] introduces region-level instruction tuning to ground language responses in target coordinates, while EarthGPT [27] unifies multisensor comprehension across optical, SAR, and infrared modalities. LHRS-Bot-Nova [28] further enhances cross-modal alignment through multi-level visual compression and spatial instruction data, and GeoLLaVA-8K [7] prunes redundant background tokens while retaining anchor tokens to accommodate higher-resolution inputs in a single pass. While these methods strengthen domain representations and token encoding efficiency, they rely on costly end-to-end backbone fine-tuning and static single-pass compression.
In contrast, EarthJev keeps the answering MLLM completely frozen and reformulates UHR understanding as an active, multi-stage decision process, allowing lightweight decision modules to dynamically allocate a compact visual budget and generalize across diverse MLLM backbones.
Active Observation for UHR Remote Sensing VQA
Since question-relevant targets in UHR remote sensing imagery often occupy only a tiny fraction of the scene, active perception [29, 30] recasts VQA as a sequential observation process that progressively reaches into informative local regions through region selection, zooming, and other visual operations. One line of work trains MLLMs via supervised fine-tuning (SFT) or reinforcement learning (RL) to invoke cropping and visual tools [8, 11, 31, 32, 33, 34]. For instance, ZoomEarth [9] optimizes region-guided zooming trajectories via RL, GeoEyes [10] incorporates evidence-gain rewards to regulate on-demand zooming and stopping, and GeoLens [35] integrates multi-tool execution into policy training.
A complementary line of work explores training-free active observation over frozen MLLMs [36, 37, 38, 39]. Specifically, ZoomSearch [40] combines patch–text relevance and model-evidence scores to guide adaptive multi-branch search, then reassembles selected patches into a layout-preserving canvas, whereas GazeEarth [41] applies non-uniform gaze-based warping to emphasize salient regions while preserving peripheral context. WeaveEarth [39] constructs spatially complementary evidence sets and organizes local views with spatial metadata for reasoning. Despite their effectiveness, existing paradigms remain susceptible to misleading spatial cues and uncalibrated exploration (as illustrated in Figure 3): blindly following imprecise spatial pointers in the query or relying on zero-shot heuristic scores can steer crop-based search (e.g., ZoomSearch) or non-uniform warping (e.g., GazeEarth) toward incomplete or irrelevant regions.
Unlike these approaches, EarthJev explicitly verifies spatial pointers against visual content before searching, i.e., routing reliable cues to a fast path while down-weighting erroneous ones, and replaces expensive autoregressive region probing with batched hidden-state scoring to achieve both higher localization accuracy and lower latency.
Decision Model
Decision-oriented models restrict predictions to task-defined output spaces so that their scores can directly support selection, routing, and verification. Jev [42] provides a typed decision interface that returns probability estimates in parallel without free-form text generation. NanoJev [43] implements candidate scoring on a shared language-model backbone and supports choice, Boolean, and ordered-score outputs. Visual Jev [44] extends this interface to multiple questions sharing an image, combining answer-supervised adaptation with candidate-token readout from the existing language-model head and batched execution over a shared visual prefix. These approaches motivate constrained probability readout and efficient execution for repeated decisions.
Efficient score extraction and probability calibration address different aspects of decision making. Guo et al. [45] show that accurate neural classifiers can still produce poorly calibrated confidence estimates. Post-hoc calibration fits mappings from model scores to probability estimates using labeled calibration data while keeping the underlying predictor fixed. Platt scaling fits a sigmoid mapping to decision scores [46], whereas temperature scaling rescales logits using a fitted scalar [45]. Calibration concerns agreement between predicted probabilities and empirical outcome frequencies and must be interpreted with respect to the prediction target and the distribution represented by the calibration data.
For answer selection, the probability assigned to a generated answer is not automatically an estimate of its correctness. Kadavath et al. [47] study candidate-level self-evaluation by eliciting the probability that a proposed answer is correct. Tian et al. [48] further show that confidence-elicitation strategies affect calibration in language models fine-tuned with human feedback. In the visual setting, Visual Jev [44] reports that an auxiliary evidence-sufficiency head detects missing evidence but does not improve answer accuracy or selective prediction in its evaluated setting. These findings motivate distinguishing evidence-related predictions from answer-correctness estimates when choosing a signal for a downstream decision. Recently, JEV-style decision-making methods have also emerged in the RS domain [49, 50], but most overlook the computationally expensive UHR tasks.
Moving beyond single-step Jev formulations, EarthJev decouples and calibrates three stage-specific decision signals across the entire perception-to-reasoning loop, culminating in a Calibrated Answer Selection (CAS) module that reconciles complementary evidence presentations (memory vs. canvas) to ensure that acquired visual evidence reliably translates into correct answers.
Figure 4. The EarthJev workflow consists of three stages: (1) Spatial pointer verification, which integrates visual and geometric information to evaluate the localization cues provided in a question, determining whether the perspective presented by these cues is sufficient to answer the question directly or if further searching is required. (2) An EDM, which utilizes vision-language representations to score candidate regions, thereby guiding hierarchical expansion and the retention of evidence. (3) The selected evidence is organized into a “memory” and a “canvas”, which are then accessed by a shared, frozen answering model. Finally, the system evaluates candidate answers based on calibrated confidence scores and selects the most reliable one as the definitive answer.
Methods
Problem Formulation and Framework Construction
Given an UHR remote sensing image and a natural language question , our objective is to generate an answer using a frozen multimodal answering model under a limited visual budget. Here, consists of the question text and optional candidate answers. Rather than directly feeding the entire image into the model, EarthJev progressively selects local views based on the question, acquiring the details necessary for answering while preserving global scene information. As shown in Figure 3, let denote the set of selected views. This process can be formulated as
where denotes the evidence selection process, represents the constraints on search and visual input budgets, and organizes the global scene and local views into the input of the answering model. Accordingly, solving the question involves two interconnected stages: locating relevant evidence within large-scale imagery and utilizing the acquired evidence to generate reliable answers.
Here, denotes the maximum number of retained views. When the question does not contain an explicit spatial cue, . Through this process, the spatial cues provided in the question and the visual feedback obtained during search jointly determine subsequent observations, allowing evidence acquisition to progress incrementally based on the currently available information. Even after relevant regions have been identified, the model may still generate incorrect answers due to errors in fine-grained perception or the understanding of spatial relationships. Therefore, EarthJev further evaluates candidate answers after evidence selection, rather than directly reusing the region scores from the search stage. Specifically, EarthJev organizes the same set of evidence into two presentation formats, memory and canvas, to generate candidate answers , and calibrates their respective raw confidence features :
where denotes the calibration function corresponding to each evidence presentation format, and estimates the probability that the candidate answer is correct. In this way, spatial cue verification establishes the starting point for observation, region-level decisions guide evidence search, and answer confidence determines the final output, collectively forming a complete process from questions to evidence and ultimately to answers. Throughout the framework, the parameters of the answering model remain frozen, with only lightweight decision modules trained and the corresponding calibration parameters fitted.
Spatial Pointer Verification
In the evidence selection process of Eq. (2), the spatial cue set provides initial locations for region exploration. Whether these locations contain the required evidence, however, depends on their visual content. EarthJev therefore uses a verifier to assess the indicated regions and determine how strongly the cues should guide subsequent observations. For each , we map its coordinates to the original image and extract a local view that preserves the surrounding context. The verifier combines its visual score with geometric and option-structure features to estimate pointer reliability:
Here, is computed by the region decision head from frozen answering-model representations, and encodes 22-dimensional geometric and option-structure features through an MLP. The gate modulates their contribution, is the temperature of the verifier, and encodes directional priors. We train the verifier with cross-entropy on annotation-derived pointers and corrupted counterparts generated by shifting, rescaling, or relocating them.
The reliability assessed at the pointer views then determines how much the search relies on these locations. For each candidate search view , we use to interpolate between scores with and without explicit coordinate information:
The first term applies the same verifier, enabling geometric fusion for views associated with explicit pointers. The second disables this fusion while retaining visual evidence and directional priors ; it is used alone when no explicit pointer is available. Thus, stronger visual support at the indicated locations gives pointer geometry greater influence over subsequent exploration. The resulting guides node expansion and evidence retention in the following section.
Beyond guiding search, the estimated pointer reliability routes the question. For this routing decision, we combine with the frozen answering model’s affirmative Yes/No probability for evidence relevance in the same view . Their mean logit,
The parameters and are fitted using pointer-validity labels. When , EarthJev bypasses search and answers from the pointer view with the highest . Otherwise, it performs hierarchical search and merges the pointer views with the views it retains. For questions with bounding boxes only in the answer options, the fast path instead uses .
Region Scoring and Hierarchical Search
When spatial cue verification does not trigger the fast path, EarthJev employs hierarchical search to acquire the local evidence required for answering. The key challenge in this process is determining which views warrant further exploration. To this end, we develop EDM that predicts the presence of relevant evidence within a region using the model’s vision-language representations and incorporates this prediction into region expansion and evidence retention. For each candidate view , we resize it to at most 256 visual tokens and pair it with an evidence-querying prompt derived from . The resulting hidden representation is passed to a two-layer decision head to estimate whether the view contains question-relevant evidence. Explicit coordinates are excluded from the prompt and handled by the verification mechanism in § 5-B, while directional descriptions provide a soft positional prior. The resulting region score is as follows:
Here, denotes the position of the view center along axis , represents the target position corresponding to the directional description, and controls the strength of the prior. The region decision head is trained using binary cross-entropy loss on cropped samples with annotated evidence regions, while remains frozen throughout training. The temperature is fitted on held-out samples. Consequently, each view can be assigned a region score through a single representation extraction, without requiring the generation of textual responses for region-level judgments. Specifically, starting from the full image, each expansion divides the selected view into nine subregions, whose scores are computed in one batch. For questions with explicit spatial pointers, we use the fused score from § 5-B; otherwise, we use . Denoting the applicable score by , we select the next view from the set of expandable candidate nodes :
After each expansion, the newly generated subregions are added to the candidate set, and the next node is selected based on the region scores until the search expansion budget is exhausted. Region-level judgments thus directly determine the observation order, guiding the search progressively toward locations more likely to contain question-relevant evidence. After the search is completed, evidence is selected relative to the average score of the candidate views. Let denote the mean score of the candidate views. We retain up to six views satisfying ; if no view satisfies this condition, the highest-scoring view is retained. These views are merged with the pointer views. Each view is assigned an evidence presence probability
where is the search score or, for a pointer view, its verifier score , and is fitted by Platt scaling. Each view is enlarged to include context (1.5 times its size and at least 512 pixels per side), the views are ranked by , a view whose IoU with a higher-ranked view exceeds 0.5 is merged into it, and the top views form .
Calibrated Answer Selection
After obtaining the evidence set , EarthJev organizes the visual input to the answering model in two formats: memory sequentially presents a global thumbnail and local views with positional information, while canvas assembles the same set of local views into a unified canvas according to their spatial layout [40], as shown in Figure 6. The answering model processes the two input formats separately to generate the candidate answers in Eq. (3). Given the same evidence, the framework selects the final answer based on the confidence scores of the candidate answers. Specifically, we extract the confidence feature from the answer generation probabilities. For multiple-choice questions, we use the predicted probability of the selected option letter and apply a logit transformation. For open-ended questions, we use the average log-probability of the answer tokens:
Figure 6. Visual memory examples combining a global overview with local views indexed by their source locations. Memory and canvas answers are shown with calibrated confidence in parentheses.
Here, denotes the predicted probability of the selected option letter, and represents the number of tokens in the candidate answer. Since the relationship between raw confidence scores and answer correctness may differ across evidence presentation formats, we fit separate Platt scaling mappings for each benchmark (RSHR-Bench, which provides no calibration questions, reuses the mappings fitted on MME-RS and XLRS-Bench-lite) to convert:
Here, is fitted separately for answer correctness under each evidence presentation format. EarthJev then compares the calibrated confidence scores according to Eq. (3) and selects the candidate answer with the highest confidence as the final answer . Finally, evidence presence vetoing corrects answers that contradict the retrieved evidence. If the selected answer is an option stating that the queried object does not appear in the image, while , EarthJev replaces it with the remaining option of the highest predicted probability. In this way, EarthJev connects question-guided active observation with evidence-based answering, enabling more accurate and efficient remote sensing VQA under a limited visual budget.
Experiments
Experimental Setup
We evaluate EarthJev on the complete question sets of the five benchmarks described below (25,039 questions in total). Because the relation between answer confidence and correctness may differ with the answer format and scoring rule of each benchmark, the calibration mappings and decision thresholds of EarthJev are fitted on a small labeled calibration set, which consists of 460 questions sampled from MME-RS, LRS-VQA, and XLRS-Bench-lite and of the official training split of LRS-GRO (Section IV-A4).
Benchmarks: We evaluate EarthJev on the complete evaluation splits of five UHR remote sensing VQA benchmarks spanning diverse spatial scales and reasoning skills:
LRS-VQA [5] comprises 7,333 open-ended questions across three subsets, i.e. LRS-FQA [18], LRS-Bridge [51], and LRS-STAR [52], evaluating object attributes, counting, and scene comprehension.
MME-RealWorld-RS (MME-RS) [53] contains 3,738 single-choice questions targeting fine-grained color recognition, object counting, and position identification.
LRS-GRO [9] organizes 17 question categories across Global, Region, and Object levels with region-of-interest annotations. In this study, we evaluate on its test set and report the results separately for the Global, Region, and Object groups; its official training split is used only for calibration.
RSHR-Bench [16] assesses UHR perception and reasoning; we evaluate on its 1,154-question single-turn benchmark subset.
XLRS-Bench-lite (XLRS-lite) [6] includes 3,080 multiple-choice questions across 13 fine-grained Level-3 (L3) task categories.
Compared Baselines: Corresponding to the four horizontal groups in Table 1 and Table II, we compare EarthJev against 21 representative methods:
| Method | Backbone | Training | AD | CCR | ECR | OC | OCC | OCL | OLUC | OMS | OSR | RC | RCCD | RLUC | RP | Avg. |
| ZoomSearch | Qwen3.5-9B | 77.00 | 49.00 | 79.00 | 33.33 | 39.50 | 41.12 | 26.00 | 46.67 | 35.00 | 58.00 | 30.00 | 84.50 | 26.00 | 44.48 | |
| GazeEarth | Qwen3-VL-8B | 76.00 | 63.00 | 86.00 | 33.33 | 52.62 | 50.12 | 31.00 | 71.67 | 37.80 | 51.00 | 53.33 | 84.50 | 49.00 | 52.95 | |
| ViCrop | Qwen3-VL-8B | 70.00 | 52.00 | 79.00 | 16.67 | 43.25 | 40.00 | 19.00 | 65.00 | 28.40 | 44.00 | 45.00 | 77.50 | 37.00 | 43.51 | |
| AttWarp | Qwen3-VL-8B | 74.00 | 46.00 | 79.00 | 16.67 | 46.25 | 42.00 | 23.00 | 66.67 | 31.00 | 44.00 | 46.67 | 76.50 | 39.00 | 45.36 | |
| WeaveEarth | Qwen3-VL-8B | 74.00 | 56.00 | 78.00 | 21.67 | 36.00 | 37.62 | 15.00 | 66.67 | 28.00 | 51.00 | 45.00 | 83.00 | 25.00 | 41.36 | |
| Zoom-Refine | Qwen3.5-9B | 68.00 | 49.00 | 81.00 | 35.00 | 41.88 | 39.25 | 32.00 | 71.67 | 32.80 | 53.00 | 33.33 | 85.50 | 41.00 | 45.19 | |
| RS-OPSD | Qwen3-VL-8B | ✓ | 76.00 | 48.00 | 83.00 | 28.33 | 46.75 | 40.88 | 36.00 | 70.00 | 28.00 | 52.00 | 46.67 | 85.00 | 49.00 | 46.82 |
| GeoLens | Qwen2.5-VL-7B | ✓ | 72.00 | 53.00 | 82.00 | 30.00 | 48.88 | 32.75 | 1.00 | 66.67 | 30.60 | 39.00 | 50.00 | 81.00 | 48.00 | 43.86 |
| GeoEyes | Qwen2.5-VL-7B | ✓ | 74.00 | 40.00 | 81.00 | 26.67 | 59.00 | 68.00 | 19.00 | 70.00 | 32.20 | 38.00 | 33.33 | 76.00 | 56.00 | 55.68 |
| ZoomEarth | Qwen2.5-VL-3B | ✓ | 61.00 | 1.00 | 69.00 | 10.00 | 24.38 | 15.62 | 3.00 | 31.67 | 12.60 | 2.00 | 10.00 | 53.00 | 44.00 | 22.73 |
| DeepEyes | Qwen2.5-VL-7B | ✓ | 70.00 | 40.00 | 76.00 | 16.67 | 41.88 | 35.75 | 43.00 | 68.33 | 30.60 | 39.00 | 46.67 | 77.00 | 40.00 | 42.69 |
| Mini-o3 | Qwen2.5-VL-7B | ✓ | 71.00 | 17.00 | 72.00 | 10.00 | 22.00 | 30.00 | 33.00 | 26.67 | 24.40 | 41.00 | 1.67 | 80.00 | 16.00 | 31.53 |
| Pixel-Reasoner | Qwen2.5-VL-7B | ✓ | 65.00 | 48.00 | 76.00 | 18.33 | 36.50 | 34.75 | 16.00 | 63.33 | 34.40 | 41.00 | 43.33 | 75.50 | 36.00 | 40.58 |
| GeoChat | Vicuna-7B | ✓ | 36.00 | 21.00 | 48.00 | 16.67 | 21.12 | 16.75 | 1.00 | 35.00 | 23.80 | 29.00 | 28.33 | 23.00 | 46.00 | 22.63 |
| VHM | Vicuna-7B | ✓ | 34.00 | 21.00 | 48.00 | 16.67 | 20.50 | 16.75 | 9.00 | 35.00 | 28.20 | 30.00 | 28.33 | 23.00 | 46.00 | 23.41 |
| RSUniVLM | Qwen2-0.5B | ✓ | 27.00 | 52.00 | 21.00 | 20.00 | 30.88 | 24.62 | 2.00 | 35.00 | 27.80 | 23.00 | 18.33 | 36.50 | 20.00 | 26.79 |
| SkySenseGPT | Vicuna-7B | ✓ | 37.00 | 21.00 | 48.00 | 16.67 | 21.12 | 16.75 | 1.00 | 35.00 | 23.80 | 29.00 | 28.33 | 24.00 | 46.00 | 22.99 |
| TEOChat | Vicuna-7B | ✓ | 37.00 | 21.00 | 49.00 | 16.67 | 23.38 | 16.75 | 2.00 | 40.00 | 25.00 | 29.00 | 28.33 | 27.50 | 46.00 | 23.90 |
| EarthDial | Phi-3-mini | ✓ | 64.00 | 44.00 | 69.00 | 21.67 | 33.75 | 31.13 | 2.00 | 55.00 | 25.40 | 43.00 | 45.00 | 41.50 | 32.00 | 34.29 |
| GeoLLaVA-8K | Qwen2-7B | ✓ | 68.00 | 50.00 | 77.00 | 25.00 | 42.00 | 31.75 | 51.00 | 65.00 | 35.20 | 41.00 | 50.00 | 68.00 | 66.00 | 43.47 |
| Coarse-to-Fine | Qwen2-7B | ✓ | 73.00 | 45.00 | 75.00 | 23.33 | 29.38 | 32.50 | 6.00 | 26.67 | 25.20 | 43.00 | 50.00 | 47.50 | 26.00 | 33.90 |
| EarthJev | Qwen3.5-9B | ✓ | 76.00 | 51.00 | 77.00 | 26.67 | 61.88 | 59.13 | 11.00 | 80.00 | 37.60 | 54.00 | 41.67 | 88.00 | 29.00 | 55.81 |
Table II. TABLE II. Performance comparison of Earth with existing methods on XLRS-Bench-lite, covering 13 fine-grained Level-3 (L3) task categories. Training settings are indicated by ✓ (trained), ✓ (answering model frozen, with only lightweight decision modules trained), and (training-free).
Training-Free Active Observation: Region cropping and multi-branch search methods including Vi-Crop [36], ZoomSearch [40], Zoom-Refine [37], and WeaveEarth [39], as well as non-uniform spatial warping methods including AttnWarp [38] and GazeEarth [41].
Trained Remote Sensing Active Perception: Domain-adapted zooming and tool-use agents trained via supervised fine-tuning, self-distillation, or reinforcement learning, including ZoomEarth [9], GeoEyes [10], Ge-Olens [35], and RS-OPSD [34].
General-Domain Visual Search Agents: Reinforcement-learned thinking-with-images models designed for general high-resolution reasoning, including DeepEyes [8], Mini-o3 [31], and Pixel-Reasoner [32].
Single-Pass Remote Sensing MLLMs: End-to-end instruction-tuned remote sensing models that process the whole image in a single forward pass, including GeoChat [26], VHM [24], RSUniVLM [25], SkySenseGPT [21], TEoChat [22], EarthDial [23], GeoLLaVA-8K [7], and Coarse-to-Fine [5].
All baselines are evaluated using their official codebases and default prompting strategies. ZoomSearch and Zoom-Refine share the same frozen Qwen3.5-9B backbone as EarthJev, other training-free methods use Qwen3-VL-8B, and trained methods retain the backbones of their released checkpoints.
Evaluation Metrics: For a question set , we define
where is determined by the benchmark-specific answer matcher. We use the official MME-RealWorld and XLRS-Bench parsers for multiple-choice answers. XLRS-Bench-lite includes multi-answer land-use questions, for which the predicted option set must match the reference set. For RSHR-Bench, which supplies no scoring script in the evaluation setup, we extract the leading option letter or identify the option whose full text is reproduced. Open-ended matching follows the LRS-VQA and ZoomEarth evaluation scripts, including lowercasing and WordNet [54] synonym matching. Failed or empty responses receive .
| Method | Backbone | LRS-VQA FAIR | LRS-VQA Bridge | LRS-VQA STAR | LRS-VQA Avg. | MME-RS Color | MME-RS Count | MME-RS Position | MME-RS Avg. | LRS-GRO Global | LRS-GRO Region | LRS-GRO Object | LRS-GRO Avg. | RSHR-Bench Perception | RSHR-Bench Reasoning | RSHR-Bench Avg. | XLRS-lite Avg. | Overall Avg. |
| ZoomSearch [40] | Qwen3.5-9B | 29.75 | 31.26 | 34.31 | 32.46 | 70.20 | 39.31 | 71.84 | 60.62 | 67.14 | 48.52 | 50.31 | 53.91 | 35.74 | 55.97 | 37.52 | 44.48 | 45.80 |
| GazeEarth [41] | Qwen3-VL-8B | 29.18 | 26.74 | 29.38 | 28.94 | 63.67 | 21.78 | 71.12 | 52.43 | 77.96 | 50.39 | 58.04 | 61.00 | 44.49 | 59.27 | 46.45 | 52.95 | 48.36 |
| ViCrop [36] | Qwen3-VL-8B | 27.33 | 29.57 | 30.51 | 29.39 | 46.85 | 16.97 | 47.02 | 37.11 | 75.56 | 51.48 | 54.03 | 58.58 | 34.58 | 61.67 | 36.05 | 43.51 | 40.93 |
| AttWarp [38] | Qwen3-VL-8B | 26.76 | 31.54 | 30.96 | 29.74 | 54.58 | 19.00 | 62.13 | 45.45 | 77.75 | 49.00 | 53.39 | 58.18 | 33.80 | 60.34 | 36.22 | 45.36 | 42.99 |
| WeaveEarth [39] | Qwen3-VL-8B | 27.64 | 24.95 | 34.78 | 31.15 | 54.02 | 23.49 | 65.71 | 47.94 | 75.34 | 47.74 | 56.50 | 58.94 | 32.74 | 63.76 | 36.14 | 41.36 | 43.10 |
| Zoom-Refine [37] | Qwen3.5-9B | 28.61 | 31.73 | 34.11 | 32.06 | 69.96 | 34.99 | 75.10 | 60.22 | 79.38 | 51.79 | 58.79 | 62.06 | 32.74 | 69.52 | 35.18 | 45.19 | 46.94 |
| RS-OPSD [34] | Qwen3-VL-8B | 28.96 | 34.18 | 34.48 | 32.73 | 70.28 | 36.95 | 76.29 | 61.37 | 78.26 | 53.18 | 55.21 | 60.24 | 37.51 | 66.01 | 39.17 | 46.82 | 48.07 |
| GeoLens [35] | Qwen2.5-VL-7B | 24.21 | 30.32 | 29.56 | 28.01 | 53.23 | 19.74 | 63.64 | 45.75 | 77.92 | 47.47 | 55.73 | 59.09 | 33.08 | 54.29 | 35.36 | 43.86 | 42.41 |
| GeoEyes [10] | Qwen2.5-VL-7B | 20.64 | 20.81 | 21.48 | 21.12 | 51.55 | 27.57 | 54.42 | 44.65 | 67.91 | 30.84 | 36.38 | 42.61 | 46.89 | 63.73 | 46.62 | 55.68 | 42.14 |
| ZoomEarth [9] | Qwen2.5-VL-3B | 19.50 | 27.02 | 22.81 | 22.39 | 48.45 | 26.10 | 50.36 | 41.76 | 66.37 | 43.60 | 51.25 | 53.06 | 30.26 | 52.76 | 31.72 | 22.73 | 34.33 |
| DeepEyes [8] | Qwen2.5-VL-7B | 6.12 | 8.00 | 9.70 | 8.35 | 56.57 | 27.57 | 64.92 | 49.87 | 32.69 | 6.14 | 8.41 | 13.68 | 32.21 | 56.92 | 35.79 | 42.69 | 30.08 |
| Mini-o3 [31] | Qwen2.5-VL-7B | 12.85 | 12.24 | 13.53 | 13.13 | 52.83 | 23.16 | 60.54 | 45.69 | 57.09 | 25.52 | 28.08 | 34.42 | 18.91 | 27.28 | 20.36 | 31.53 | 29.03 |
| Pixel-Reasoner [32] | Qwen2.5-VL-7B | 25.70 | 28.34 | 30.56 | 28.73 | 53.23 | 23.00 | 62.93 | 46.58 | 75.86 | 45.17 | 50.41 | 55.26 | 30.69 | 65.30 | 34.75 | 40.58 | 41.18 |
| GeoChat [26] | Vicuna-7B | 20.64 | 22.60 | 20.13 | 20.65 | 22.87 | 16.88 | 26.25 | 22.04 | 66.19 | 34.58 | 41.25 | 45.64 | 27.82 | 24.79 | 28.60 | 22.63 | 27.91 |
| VHM [24] | Vicuna-7B | 13.86 | 17.04 | 13.88 | 14.33 | 23.11 | 16.97 | 29.44 | 23.22 | 60.52 | 27.61 | 35.87 | 39.82 | 30.77 | 35.35 | 31.72 | 23.41 | 26.50 |
| RSUniVLM [25] | Qwen2-0.5B | 23.94 | 28.63 | 29.13 | 27.45 | 24.38 | 23.90 | 28.16 | 25.49 | 63.57 | 34.41 | 38.20 | 43.37 | 24.87 | 49.96 | 24.87 | 26.79 | 29.60 |
| SkySenseGPT [21] | Vicuna-7B | 23.28 | 22.02 | 24.76 | 23.91 | 22.87 | 16.88 | 26.25 | 22.04 | 66.49 | 38.24 | 39.71 | 45.77 | 29.04 | 28.03 | 29.72 | 22.99 | 28.89 |
| TEOChat [22] | Vicuna-7B | 18.40 | 28.34 | 22.83 | 22.26 | 22.95 | 16.88 | 26.41 | 22.12 | 64.13 | 32.71 | 42.84 | 45.54 | 25.23 | 33.26 | 26.60 | 23.90 | 28.08 |
| EarthDial [23] | Phi-3-mini | 15.32 | 25.80 | 19.60 | 19.17 | 26.53 | 25.86 | 42.56 | 31.70 | 63.83 | 29.62 | 38.90 | 42.68 | 30.41 | 43.35 | 32.32 | 34.29 | 32.03 |
| GeoLLava-8K [7] | Qwen2-7B | 3.39 | 3.11 | 3.33 | 3.31 | 32.27 | 22.27 | 28.08 | 27.58 | 10.70 | 4.83 | 3.17 | 35.36 | 29.66 | 54.05 | 31.63 | 43.47 | 22.27 |
| Coarse-to-Fine [5] | Qwen2-7B | 22.62 | 35.40 | 26.56 | 26.62 | 41.75 | 33.61 | 50.28 | 41.95 | 71.35 | 35.32 | 36.52 | 44.57 | 32.90 | 49.73 | 33.28 | 33.90 | 36.06 |
| EarthJev | Qwen3.5-9B | 30.63 | 36.53 | 36.68 | 34.79 | 72.03 | 39.89 | 77.01 | 63.16 | 77.10 | 52.74 | 54.07 | 59.27 | 53.50 | 60.53 | 52.25 | 55.81 | 53.06 |
Table I. Comparison of EarthJev with existing methods across five remote sensing VQA benchmarks. Results are reported as accuracy (%). The top three distinct scores per column are highlighted in coral, amber, and blue; best results are boldfaced, and ties share the same color. Benchmark Avg. denotes question-level accuracy, while Overall Avg. is the mean across five benchmarks. RSHR-Bench Perception/Reasoning scores are averaged over task categories.
Each benchmark’s Avg. is question-level accuracy. Subset and task scores use the corresponding question sets. For RSHR-Bench, Perception and Reasoning are unweighted means over their respective task categories, while its Avg. is accuracy over all evaluated questions; these quantities use different weighting rules. Overall Avg. gives equal weight to the five benchmarks:
Accuracy differences are reported in percentage points (pp). For the binary decision diagnostics, we report AUROC for discrimination and binary top-label expected calibration error (ECE) [55] for calibration. For an event probability and label , let and . Using equal-width confidence bins,
where and are the bin means of and , respectively. The diagnostic outcomes are evidence presence, point validity, and answer correctness. ECE is evaluated on held-out questions that are not used to fit the corresponding mapping.
The 460 calibration questions of MME-RS, LRS-VQA and XLRS-Bench-lite are part of the evaluation sets; excluding them changes the accuracy of EarthJev by at most 0.09 percentage points (63.16%, 34.81% and 55.90% on the remaining questions).
4) Implementation Details: EarthJev keeps the answering model frozen throughout, training only the region decision head and the spatial cue verification module. The lightweight decision modules are trained using the ZwZ-RL-VQA dataset [56], from which 12,000 questions and their corresponding region annotations are selected to construct training samples, with the proportion of incorrect spatial cues set to 25%. During inference, a hierarchical region partitioning strategy of is adopted, with a maximum of 10 search expansions. The input for each candidate view is limited to 256 visual tokens, and the subregions of the same node are evaluated in batches. The memory-based evidence presentation retains at most four local views. The fast-path threshold for spatial cue verification is set to 0.60, and answer confidence scores under the two evidence presentation strategies are calibrated separately using Platt scaling. In addition, we select 90, 240, and 130 annotated questions from MME-RS, LRS-VQA, and XLRS-Bench-lite, respectively, along with 3,511 questions from the official training split of LRS-GRO, to fit the calibration parameters. This calibration process does not update the answering model. The compared methods retain their respective prompting strategies and visual processing pipelines, and evaluation on each benchmark follows its official scoring protocol.
Comparison with State-of-the-Art Methods
Table I and Figure 5 summarizes the comparative results of EarthJev and existing methods on five remote sensing VQA benchmarks.
Figure 5. Comparison of overall accuracy across five remote sensing benchmarks.
benchmarks. Using Qwen3.5-9B [57] as the answering model, EarthJev achieves an average accuracy of 53.06% across the five benchmarks, outperforming the strongest competing method, GazeEarth, by 4.70 percentage points. It also achieves the highest overall accuracy on LRS-VQA, MME-RS, RSHR-Bench, and XLRS-Bench-lite. Specifically, EarthJev achieves accuracies of 34.79% and 63.16% on LRS-VQA and MME-RS, respectively, surpassing the best competing results on both benchmarks. On the evaluation subset of RSHR-Bench, EarthJev achieves an accuracy of 52.25%, exceeding GeoEyes by 5.63 percentage points. These results demonstrate that integrating spatial cue verification, region-level evidence assessment, and answer selection throughout the active observation process can effectively improve overall performance across diverse remote sensing VQA scenarios.
A closer examination of the task-level results shows that EarthJev achieves the best performance on all three subsets of LRS-VQA, namely FAIR, Bridge, and STAR, as well as on the three task categories of MME-RS, including color recognition, object counting, and position identification. This demonstrates its effectiveness across different image contents and fine-grained perception requirements. Table II presents the results for the 13 L3 categories of XLRS-Bench-lite. EarthJev achieves an overall accuracy of 55.81% and attains the best performance in the OCC, OMS, and RLUC categories, with accuracies of 61.88%, 80.00%, and 88.00%, respectively. In particular, its accuracy in the OMS category exceeds the strongest competing result by 8.33 percentage points. These findings indicate that the benefits of EarthJev extend across various local visual understanding tasks, supporting the effectiveness of question-guided evidence acquisition and answer evaluation in improving high-resolution image understanding.
Table IV further investigates the applicability of EarthJev across different answering models. Compared with the baselines that directly process the entire image using the same answering models, incorporating EarthJev improves the average accuracy across the five benchmarks by 6.53 and 4.99 percentage points for Qwen3.5-4B and Qwen3-VL-8B [4], respectively. For Intern-S1-mini [58] and LLaVA-OneVision-2-8B [59], the corresponding average improvements are 3.41 and 6.52 percentage points, respectively. Moreover, all four models achieve positive improvements on each of the five benchmarks. Specifically, Qwen3.5-4B achieves improvements of 12.01 and 13.34 percentage points on XLRS-Bench-lite and RSHR-Bench, respectively, while LLaVA-OneVision-2-8B achieves an improvement of 18.59 percentage points on LRS-GRO. With the answering model parameters kept frozen, these results demonstrate the applicability of EarthJev across different model families and parameter scales. They also indicate that improving the decision-making process between observation and answering can further leverage the remote sensing understanding capabilities of existing multimodal models.
| Answerer | Setting | MME-RS | LRS-VQA | XLRS-lite | LRS-GRO | RSHR-Bench | Avg. |
| Qwen3.5-4B | Base | 51.95 | 29.74 | 41.95 | 58.11 | 37.52 | 43.85 |
| + EarthJev | 57.76 | 30.32 | 53.96 | 59.01 | 50.87 | 50.38 | |
| Gain (pp) | +5.81 | +0.57 | +12.01 | +0.90 | +13.34 | +6.53 | |
| Qwen3-VL-8B | Base | 55.97 | 32.27 | 46.10 | 59.27 | 36.05 | 45.93 |
| + EarthJev | 56.39 | 34.97 | 54.38 | 59.40 | 49.48 | 50.92 | |
| Gain (pp) | +0.43 | +2.70 | +8.28 | +0.13 | +13.43 | +4.99 | |
| Intern-S1-mini | Base | 43.37 | 28.95 | 38.70 | 53.21 | 36.48 | 40.14 |
| + EarthJev | 47.62 | 30.89 | 44.87 | 54.10 | 40.29 | 43.55 | |
| Gain (pp) | +4.25 | +1.94 | +6.17 | +0.89 | +3.81 | +3.41 | |
| LLaVA-OneVision-2-8B | Base | 57.97 | 19.69 | 39.84 | 39.74 | 35.27 | 38.50 |
| + EarthJev | 59.02 | 28.98 | 42.21 | 58.33 | 36.57 | 45.02 | |
| Gain (pp) | +1.04 | +9.29 | +2.37 | +18.59 | +1.30 | +6.52 |
Table IV. Performance of EarthJev across answering models (accuracy, %). Gray and coral cells show the base and EarthJev-augmented results, respectively; blue cells report gains in percentage points.
Ablation Study and Cross-Model Generalization
Component-Wise Ablation: To investigate the effects of progressively incorporating different modules, we use Qwen3.5-9B as the answering model and compare four incremental configurations presented in Table III. Group 1 employs the baseline search and answering pipeline, achieving an average accuracy of 45.80% across the five benchmarks. After introducing EDM, Group 2 improves the average accuracy to 47.24%, with improvements observed on four of the five benchmarks, indicating that region scoring based on evidence presence can improve search results. With the further incorporation of SPV, Group 3 achieves an average accuracy of 52.64%. In particular, the accuracies on XLRS-Bench-lite and RSHR-Bench increase by 10.71 and 15.16 percentage points, respectively, demonstrating the effectiveness of spatial cue verification and its guidance of subsequent observations.
| Group | EDM | SPV | CAS | MME-RS | LRS-VQA | XLRS-lite | LRS-GRO | RSHR-Bench | Avg. |
| 1 | ✗ | ✗ | ✗ | 60.62 | 32.46 | 44.48 | 53.91 | 37.52 | 45.80 |
| 2 | ✓ | ✗ | ✗ | 62.97 | 34.52 | 44.29 | 55.42 | 38.99 | 47.24 |
| 3 | ✓ | ✓ | ✗ | 62.81 | 35.20 | 55.00 | 56.04 | 54.16 | 52.64 |
| 4 | ✓ | ✓ | ✓ | 63.16 | 34.79 | 55.81 | 59.27 | 52.25 | 53.06 |
Table III. Component Ablation of EarthJev with Qwen3.5-9B. Results are reported as accuracy (%). The full configuration is shaded in light coral; the best result in each column is boldfaced.
The complete configuration, Group 4, further incorporates calibrated answer selection while retaining evidence presence vetoing, achieving the highest average accuracy of 53.06%, an improvement of 7.26 percentage points over Group 1. Compared with Group 3, the complete configuration yields further improvements on MME-RS, XLRS-Bench-lite, and LRS-GRO, with accuracy on LRS-GRO increasing from 56.04% to 59.27%. This result indicates that answer evaluation after evidence acquisition can provide additional benefits. The progressively increasing average performance across the four configurations supports the design of integrating region-level assessment, spatial cue verification, and answer selection into a unified observation pipeline.
Generalization across Answering Models: As shown in Table IV, to assess generality across answering models, we also evaluate EarthJev with Qwen3.5-4B and Qwen3-VL-8B [4] from the Qwen family, as well as Intern-S1-mini [58] and LLaVA-OneVision-2-8B [59] from other families. The results demonstrate that EarthJev can serve as a plug-and-play module, easily generalizing across different types of MLLM
| Selection | MME-RS | LRS-VQA | XLRS-lite | LRS-GRO | RSHR-Bench | Avg. |
| Memory only | 62.92 | 33.15 | 55.00 | 58.53 | 54.16 | 52.75 |
| Canvas only | 56.18 | 35.20 | 53.64 | 56.04 | 45.84 | 49.38 |
| Raw confidence | 62.17 | 34.53 | 56.01 | 58.88 | 51.82 | 52.68 |
| CAS (EarthJev) | 63.16 | 34.79 | 55.81 | 59.27 | 52.25 | 53.06 |
Table V. Answer selection over the same retrieved evidence with Qwen3.5-9B (accuracy, %). Raw confidence and CAS choose between the memory and canvas answers by raw or calibrated confidence.
backbones and delivering consistent performance improvements.
In-Depth Analysis of Decision Modules and Efficiency
Table V compares answer-selection rules over the same retrieved evidence. CAS achieves a mean accuracy of 53.06%, compared with 52.75% for memory only, 49.38% for canvas only, and 52.68% for raw-confidence selection. Thus, its average improvement is approximately 0.3 percentage points over memory only and 0.4 percentage points over raw confidence. The gain is not uniform: CAS trails memory only on RSHR-Bench and raw-confidence selection on XLRS-Bench-lite. These benchmark-dependent results support CAS as an additional refinement for choosing between evidence presentations.
| Method | s/q | Rel. time | Avg. |
| Qwen3.5-9B | |||
| Direct | 3.30 | 1.0× | 46.93 |
| ZoomSearch | 11.30 | 3.4× | 45.80 |
| Zoom-Refine | 11.34 | 3.4× | 46.94 |
| EarthJev | 1.67 | 0.5× | 53.06 |
| Qwen3-VL-8B | |||
| Direct | 3.41 | 1.0× | 47.31 |
| ZoomSearch | 9.86 | 2.9× | 41.86 |
| ViCrop | 2.01 | 0.6× | 40.93 |
| AttWarp | 1.98 | 0.6× | 42.99 |
| WeaveEarth | 1.92 | 0.6× | 43.10 |
| GazeEarth | 3.88 | 1.1× | 48.36 |
| RS-OPSD | 3.48 | 1.0× | 48.07 |
| EarthJev | 1.68 | 0.5× | 50.92 |
Table VI. Inference cost under the same answering model, s/o: seconds per question on one NVIDIA H100 GPU.
1) Region Probing Discrimination: Figure 7(a) compares the accuracy and inference time of different region decision methods. In this search configuration comparison, EDM achieves an average accuracy of 47.24% across the five benchmarks, with an average inference time of 1.93 seconds per question. In contrast, using Yes/No judgments from the answering model yields an average accuracy of 46.92%, with an inference time of 4.81 seconds per question. Combined with the region probing results in Figure 2(b), the evidence recognition AUROC of the region decision head increases from 0.752 to 0.787 under the same compact-view and batch-processing conditions, while the inference time per decision remains comparable. These two sets of results demonstrate that the dedicated region decision head provides more accurate evidence scoring, while the search pipeline combining compact views and batched evaluation achieves a better accuracy–latency trade-off.
Figure 7. Design evidence for the three modules (Qwen3.5-9B). (a) EDM vs. other decision sources: mean accuracy over the five benchmarks against seconds per question. (b) SPV vs. blind trust: accuracy change on the LRS-VQA pointer test with a correct or a wrong stated box, relative to no box. (c) Held-out calibration error of each module’s decision score (evidence presence, pointer trust, answer confidence) before and after Platt scaling.
Robustness to Inaccurate Spatial Pointers
Figure 7(b) further investigates observation behavior when spatial cues are inaccurate. In the controlled experiments on LRS-VQA, directly using correct spatial cues improves accuracy by 4.75 percentage points, whereas using incorrect cues leads to a decrease of 8.47 percentage points. After incorporating SPV, the improvement from correct spatial cues remains at 4.75 percentage points, while the accuracy degradation caused by incorrect cues is reduced to 3.41 percentage points. These results indicate that visual verification preserves the benefits of valid spatial cues while mitigating the interference of incorrect locations with subsequent searches, allowing spatial information in questions to serve as a verifiable and adjustable basis for search.
Probability Calibration across Decision Stages
Figure 7(c) presents the calibration performance of different decision scores. On their respective held-out evaluation samples, the expected calibration errors (ECEs) [55] of evidence presence scores and spatial cue reliability scores decrease from 0.131 and 0.100 to 0.012 and 0.038, respectively. Meanwhile, the average ECE of answer confidence scores across the five benchmarks decreases from 0.194 to 0.029. Calibration improves the agreement between predicted probabilities and the empirical frequencies of their corresponding events, providing a basis for threshold-based decisions and candidate answer comparisons. Together with the different discriminative capabilities of evidence presence and answer confidence for answer correctness shown in Figure 2(c), these results support constructing and calibrating separate decision signals for determining whether a region contains relevant evidence, whether a spatial cue is reliable, and whether a candidate answer is correct.
Efficiency Analysis
tbl:VI compares inference latency within each answering model. With Qwen3.5-9B, EarthJev reduces mean latency from 11.30 s/q for ZoomSearch and 11.34 s/q for Zoom-Refine to 1.67 s/q, corresponding to speedups of approximately 6.8×. With Qwen3-VL-8B, it achieves 50.92% mean accuracy at 1.68 s/q, compared with 48.36% at 3.88 s/q for GazeEarth. These comparisons support a favorable accuracy–latency trade-off relative to the evaluated search methods. The single-pass baselines remain faster, at 3.30 and 3.41 s/q, respectively.
Qualitative Case Study
Figure 8 presents two examples illustrating how EarthJev connects evidence retrieval with answer selection. In the spatial reasoning example, the question asks for the position of a white house with a green roof relative to the docked ships. ZoomSearch and GazeEarth both answer “To the right.” EarthJev retrieves local views covering the waterfront, buildings, and ships, while preserving their source locations. With this evidence, the memory presentation also produces “To the right,” whereas the canvas presentation produces the correct answer, “Above.” CAS assigns higher calibrated confidence to the canvas candidate and selects it as the final response.
Figure 8. Qualitative comparison on spatial reasoning and object counting. EarthJev selects the correct answer by evaluating calibrated confidence across memory and canvas presentations of the retrieved evidence.
The aircraft-counting example exhibits the opposite selection. ZoomSearch predicts four airplanes, and GazeEarth predicts ten, while the annotated count is three. After EDM retrieves views of the central terminal and its surroundings, EarthJev produces a count of three from memory and four from canvas. CAS selects the correct memory candidate based on its higher calibrated confidence. Together, these examples show that the preferred evidence presentation can vary across questions. EDM supplies relevant local observations, while CAS distinguishes between the answers derived from them, helping translate evidence acquisition into accurate spatial reasoning and object counting. Figure 6 further illustrates the evidence provided to the answering model. The global overview preserves scene context, while the indexed local views expose details of the selected regions and retain their correspondence to the original image. The examples cover object identification, energy-source recognition, color recognition, and counting. Memory and over produce the same correct answer in the first three cases, but differ in the aircraft-counting case, where CAS selects the correct memory response.
CONCLUSION
We proposed EarthJev, a framework that reformulates active observation in UHR remote sensing VQA as a sequence of calibrated, target-specific decisions over a frozen MLLM. Our study reveals that effective visual evidence acquisition under a limited token budget requires strictly distinguishing among three prediction targets: whether a linguistic spatial pointer is trustworthy, whether a local region contains question-relevant evidence, and whether a candidate answer derived from that evidence is reliable. By addressing these targets through SPV, a batched EDM, and CAS, EarthJev eliminates the need for expensive autoregressive region judgments and end-to-end backbone fine-tuning. Comprehensive experiments on five UHR remote sensing benchmarks verify that EarthJev substantially improves both VQA accuracy and inference efficiency, generalizes seamlessly across diverse MLLM architectures, and remains robust against perturbed spatial cues. More broadly, our results demonstrate that the primary bottleneck in applying MLLMs to gigapixel-scale Earth observation lies not only in visual encoding capacity but also in the calibration of intermediate and final decision signals. Decoupling lightweight, efficient decision modules from frozen foundation models offers a practical paradigm for resource-efficient remote sensing intelligence.
References
- [1]H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 36, 2023, pp. 34 892–34 916.
- [2]Z. Chen et al., “InternVL: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 24 185–24 198.
- [3]S. Bai et al., “Qwen2.5-VL technical report,” 2025, arXiv:2502.13923.arxiv.org/abs/2502.13923
- [4]S. Bai et al., “Qwen3-VL technical report,” 2025, arXiv:2511.21631.arxiv.org/abs/2511.21631
- [5]J. Luo et al., “When large vision-language model meets large remote sensing imagery: Coarse-to-fine text-guided token pruning,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2025, pp. 9206–9217.arxiv.org/abs/2503.07588
- [6]F. Wang et al., “XLRS-Bench: Could your multimodal LLMs understand extremely large ultra-high-resolution remote sensing imagery?” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2025, pp. 14 325–14 336.
- [7]F. Wang et al., “GeoLLaVA-8K: Scaling remote-sensing multimodal large language models to 8K resolution,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 38, 2025, pp. 176 673–176 706.arxiv.org/abs/2505.21375
- [8]Z. Zheng et al., “DeepEyes: Incentivizing ‘thinking with images’ via reinforcement learning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2505.14362
- [9]R. Liu et al., “ZoomEarth: Active perception for ultra-high-resolution geospatial vision-language tasks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026, pp. 34 877–34 888.arxiv.org/abs/2511.12267
- [10]F. Wang et al., “GeoEves: On-demand visual focusing for evidence-grounded understanding of ultra-high-resolution remote sensing imagery,” 2026, arXiv:2602.14201.arxiv.org/abs/2602.14201
- [11]J. Zhu, R. Fu, J. Hu, J. Huang, N. Xing, and B. Yang, “GeoVista: Visually grounded active perception for vision-language understanding of ultra-high-resolution remote sensing images,” 2026, arXiv:2605.14475.arxiv.org/abs/2605.14475
- [12]S. Lobry, D. Marcos, J. Murray, and D. Tuia, “RSVQA: Visual question answering for remote sensing data,” IEEE Trans. Geosci. Remote Sens., vol. 58, no. 12, pp. 8555–8566, Dec. 2020.DOI
- [13]K. Li, G. Vosselman, and M. Y. Yang, “HRVQA: A visual question answering benchmark for high-resolution aerial images,” ISPRS J. Photogramm. Remote Sens., vol. 214, pp. 65–81, Aug. 2024.arxiv.org/abs/2301.09460
- [14]J. Wang, Z. Zheng, Z. Chen, A. Ma, and Y. Zhong, “EarthVQA: Towards queryable Earth via relational reasoning-based remote sensing visual question answering,” in Proc. AAAI Conf. Artif. Intell. (AAAI), vol. 38, no. 6, 2024, pp. 5481–5489.
- [15]X. Li, J. Ding, and M. Elhoseiny, “VRSBench: A versatile vision-language benchmark dataset for remote sensing image understanding,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 37, 2024, pp. 3229–3242.arxiv.org/abs/2406.12384
- [16]Y. Dang et al., “A benchmark for ultra-high-resolution remote sensing MLLMs,” 2025, arXiv:2512.17319.arxiv.org/abs/2512.17319
- [17]J. Ding et al., “Object detection in aerial images: A large-scale benchmark and challenges,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 11, pp. 7778–7796, Nov. 2022.DOI
- [18]X. Sun et al., “FAIR1M: A benchmark dataset for fine-grained object recognition in high-resolution remote sensing imagery,” ISPRS J. Photogramm. Remote Sens., vol. 184, pp. 116–130, Feb. 2022.arxiv.org/abs/2103.05569
- [19]A. Xiao et al., “Foundation models for remote sensing and Earth observation: A survey,” IEEE Geosci. Remote Sens. Mag., vol. 13, no. 4, pp. 297–324, Dec. 2025.
- [20]Y. Hu, J. Yuan, C. Wen, X. Lu, Y. Liu, and X. Li, “RSGPT: A remote sensing vision language model and benchmark,” ISPRS J. Photogramm. Remote Sens., vol. 224, pp. 272–286, Jun. 2025.arxiv.org/abs/2307.15266
- [21]J. Luo et al., “SkySenseGPT: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding,” 2024, arXiv:2406.10100.arxiv.org/abs/2406.10100
- [22]J. A. Irvin et al., “TEOChat: A large vision-language assistant for temporal Earth observation data,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025.arxiv.org/abs/2410.06234
- [23]S. Soni et al., “EarthDial: Turning multi-sensory Earth observations to interactive dialogues,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2025, pp. 14 303–14 313.arxiv.org/abs/2412.15190
- [24]C. Pang et al., “VHM: Versatile and honest vision language model for remote sensing image analysis,” in Proc. AAAI Conf. Artif. Intell. (AAAI), vol. 39, no. 6, 2025, pp. 6381–6388.
- [25]X. Liu and Z. Lian, “RSUniVLM: A unified vision-language model for remote sensing via granularity-oriented MoE,” Pattern Recognit., vol. 179, Nov. 2026, Art. no. 113717.
- [26]K. Kuckreja, M. S. Danish, M. Naseer, A. Das, S. Khan, and F. S. Khan, “GeoChat: Grounded large vision-language model for remote sensing,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 27 831–27 840.arxiv.org/abs/2311.15826
- [27]W. Zhang, M. Cai, T. Zhang, Y. Zhuang, and X. Mao, “EarthGPT: A universal multimodal large language model for multisensor image comprehension in remote sensing,” IEEE Trans. Geosci. Remote Sens., vol. 62, 2024, Art. no. 5917820.arxiv.org/abs/2401.16822
- [28]Z. Li et al., “LHRS-Bot-Nova: Unified multimodal large language model for remote sensing vision-language interpretation,” ISPRS J. Photogramm. Remote Sens., vol. 227, pp. 539–550, Sep. 2025.arxiv.org/abs/2411.09301
- [29]C. Jiang et al., “CLVM-R3: Region recognition, reasoning, and refinement for enhanced multimodal chain-of-thought,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 38, 2025, pp. 71 202–71 230.DOI
- [30]R. Doshi, “Introducing Agentic Vision in Gemini 3 Flash,” Google Blog, Jan. 2026. [Online]. Available: https://blog.google/innovation-and-ai/technology/developers-tools/agentic-vision-gemini-3-flash/
- [31]X. Lai, J. Li, W. Li, T. Liu, T. Li, and H. Zhao, “Mini-o3: Scaling up reasoning patterns and interaction turns for visual search,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2509.07969
- [32]A. Su, H. Wang, W. Ren, F. Lin, and W. Chen, “Pixel Reasoner: Incentivizing pixel space reasoning via curiosity-driven reinforcement learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 38, 2025, pp. 9130–9159.DOI
- [33]F. Wang et al., “Text before vision: Staged knowledge injection matters for agentic RLVR in ultra-high-resolution remote sensing understanding,” in Proc. Int. Conf. Mach. Learn. (ICML), vol. 306, 2026, pp. 125 319–125 333.arxiv.org/abs/2602.14225
- [34]C. Jiang, Y. Zhou, J. Yan, S. Zhao, C. Yuan, and J. Li, “RS-OPSD: Reliable privileged on-policy-self-distillation for ultra-high-resolution remote sensing VQA,” 2026, arXiv:2609.38072.arxiv.org/abs/2609.38072
- [35]F. Wang et al., “Beyond zooming: Learning multi-tool visual reasoning for ultra-high-resolution remote sensing,” 2026, arXiv:2607.25993.arxiv.org/abs/2607.25993
- [36]J. Zhang, M. Khayatkhoei, P. Chhikara, and F. Ilievski, “MLLMs know where to look: Training-free perception of small visual details with multimodal LLMs,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025.arxiv.org/abs/2502.17422
- [37]X. Yu, D. Guan, and Y. Gu, “Zoom-Refine: Boosting high-resolution multimodal understanding via localized zoom and self-refinement,” 2025, arXiv:2506.01663.
- [38]D. Dalal et al., “Constructive distortion: Improving MLLMs with attention-guided image warping,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2510.09741
- [39]X. Ma, S. Wang, X. Li, H. Liu, C. Pei, and J. Li, “WeaveEarth: Structured evidence construction and reasoning for training-free UHR remote sensing understanding,” 2026, arXiv:2607.10120.arxiv.org/abs/2607.10120
- [40]Y. Zhou, C. Jiang, C. Yuan, and J. Li, “Look where it matters: Training-free ultra-HR remote sensing VQA via adaptive zoom search,” 2025, arXiv:2511.20460.arxiv.org/abs/2511.20460
- [41]Y. Zhang et al., “The Earth in one gaze: Training-free active focus for UHR remote sensing understanding,” 2026, arXiv:2609.31747.arxiv.org/abs/2609.31747
- [42]D. Almeida, “Introducing System One Models & Jev,” TypeSafe AI Blog, Sep. 2026. [Online]. Available: https://typesafe.ai/blog/introducing-system-one-models-and-jev
- [43]T. Chen, “NanoJev: A nano replica of Jev,” GitHub repository, 2026. [Online]. Available: https://github.com/TianyuCodings/NanoJev
- [44]G. Yu and Y. Yao, “Visual Jev: Accurate and efficient decisions from shared visual context,” 2026, arXiv:2609.25845.arxiv.org/abs/2609.25845
- [45]C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in Proc. Int. Conf. Mach. Learn. (ICML), vol. 70, 2017, pp. 1321–1330.
- [46]J. C. Platt, “Probabilities for SV machines,” in Advances in Large-Margin Classifiers, A. J. Smola, P. Bartlett, B. Schölkopf, and D. Schuurmans, Eds. Cambridge, MA, USA: MIT Press, 2000, pp. 61–74.DOI
- [47]S. Kadavath et al., “Language models (mostly) know what they know,” 2022, arXiv:2207.05221.arxiv.org/abs/2207.05221
- [48]K. Tian et al., “Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback,” in Proc. Conf. Empirical Methods Natural Lang. Process. (EMNLP), 2023, pp. 5433–5442.
- [49]D. Si, D. Wang, M. Xu, J. Zhang, B. Du, and L. Zhang, “Rsjev: Discriminative remote sensing scene classification with multimodal large language models,” 2026. [Online]. Available: https://arxiv.org/abs/2610.08539
- [50]M. A. Abouelhadid, D. A. Abuhani, M. A. Munir, F. S. Khan, and S. Khan, “Eo-jev: Decision model for earth observation,” 2026. [Online]. Available: https://arxiv.org/abs/2611.03371
- [51]Y. Li, J. Luo, Y. Zhang, Y. Tan, J.-G. Yu, and S. Bai, “Learning to holistically detect bridges from large-size VHR remote sensing imagery,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 12, pp. 11 507–11 523, Dec. 2024.DOI
- [52]Y. Li et al., “STAR: A first-ever dataset and a large-scale benchmark for scene graph generation in large-size satellite imagery,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 47, no. 3, pp. 1832–1849, Mar. 2025.arxiv.org/abs/2406.09410
- [53]Y.-F. Zhang et al., “MME-RealWorld: Could your multimodal LLM challenge high-resolution real-world scenarios that are difficult for humans?” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025.arxiv.org/abs/2408.13257
- [54]G. A. Miller, “WordNet: A lexical database for English,” in Proc. Workshop Human Lang. Technol. (HLT), 1994, p. 468.
- [55]M. Pakdaman Naeini, G. F. Cooper, and M. Hauskrecht, “Obtaining well calibrated probabilities using Bayesian binning,” in Proc. AAAI Conf. Artif. Intell. (AAAI), vol. 29, no. 1, 2015, pp. 2901–2907.
- [56]L. Wei et al., “Zooming without zooming: Region-to-image distillation for fine-grained multimodal perception,” in Proc. Int. Conf. Mach. Learn. (ICML), vol. 306, 2026, pp. 133 660–133 688.arxiv.org/abs/2602.11858
- [57]Qwen Team, “Qwen3.5: Towards native multimodal agents,” Qwen Blog, Feb. 2026. [Online]. Available: https://qwen.ai/blog?id=qwen3.5
- [58]L. Bai et al., “Intern-S1: A scientific multimodal foundation model,” 2025, arXiv:2508.15763.arxiv.org/abs/2508.15763
- [59]X. An et al., “LLaVA-OneVision-2: Towards next-generation perceptual intelligence,” 2026, arXiv:2605.25979.arxiv.org/abs/2605.25979






