betaFeedback

WorksComputer Science

GeoReason: Bridging Logical Reasoning and Spatial Fidelity in Remote Sensing Segmentation

Abstract

Multimodal Large Language Models (MLLMs) have significantly advanced instruction-driven perception through the "Embedding-as-Mask" paradigm. However, extending this capability to Remote Sensing (RS) reasoning segmentation remains exceptionally challenging under cluttered geographical contexts and extreme scale variations, often leading to Spatial Fidelity Degradation and Target Misidentification. We identify a critical yet underexplored bottleneck: the sequential alienation of the segmentation token [SEG] from its initial visual grounding within the autoregressive generation paradigm. Produced only at the end of long reasoning chains, [SEG] relies on visual evidence that has been progressively diluted by linguistic abstraction and autoregressive noise. This information decay is particularly catastrophic for RS targets with minuscule pixel footprints, where even marginal fidelity loss leads to localization failure. To address these bottlenecks, we propose GeoReason, a novel framework designed to fortify logical deduction and preserve spatial integrity. First, we introduce an Anticipatory Prior mechanism that injects the [SEG] token directly into the initial user query, shifting it from a terminal output to a primary condition. This paradigm shift transforms the token into a persistent spatial anchor, preventing information decay during long-form linguistic inference. Second, we enhance visual reasoning via a Location-aware Chain-of-Thought (CoT) that enforces coarse-to-fine localization using bounding boxes, coupled with a Latent Patch-to-Token Alignment (PTA) Loss that explicitly aligns the latent segmentation token with physical RS textures. Extensive experiments demonstrate that GeoReason consistently outperforms state-of-the-art methods on the Earthreason and RRSIS-D benchmarks. Code is available at https://github.com/Fanc0126/GeoReason.

Introduction

Multimodal Large Language Models (MLLMs) have shifted image segmentation from category level masks toward complex, instruction driven perception. A cornerstone of this evolution is the “Embedding as Mask” paradigm (e.g., LISA [15]), which enables MLLMs to perform Reasoning Segmentation. In this framework, a dedicated token [SEG] is introduced into the language vocabulary and is generated autoregressively as part of the model’s textual response. The hidden embedding corresponding to the generated [SEG] token is then extracted as a latent semantic query and projected into a mask decoder, such as the Segment Anything Model (SAM) [13], to obtain dense pixel-level outputs. Unlike Referring Expression Segmentation (RES) which relies on explicit object descriptions, reasoning segmentation necessitates interpreting implicit, complex instructions through sophisticated logical deduction and visual grounding.

Despite its success in natural scenes, this paradigm encounters unique hurdles in Remote Sensing (RS). Characterized by overhead perspectives, extreme scale variations, and cluttered geographical contexts [23], RS imagery demands not only robust logical inference to resolve functional attributes and spatial relationships, but also high-precision localization in dense environments. As illustrated in Figure 1, current MLLM-based RS frameworks often fall into two traps:

Qualitative comparison of GeoReason against the baseline

Figure 1. Qualitative comparison of GeoReason against the baseline. (Top) While the baseline identifies the viaduct, the resulting mask is fragmented and suffers from blurry boundaries. (Bottom) The baseline fails to interpret the functional implications of the query (“sports games or popular live music events”), leading to the segmentation of an irrelevant field.

  • Spatial Fidelity Degradation: The predicted mask often exhibits fragmented or imprecise boundaries despite correctly identifying the target, indicating degraded spatial consistency.

  • Target Misidentification: Under more severe information degradation, the segmentation may drift to an incorrect object, reflecting a failure to maintain the correct target during reasoning.

We attribute these failures to a critical yet underexplored bottleneck in current MLLM segmentation pipelines [15, 5, 19]: the sequential alienation of the segmentation token [SEG] from its initial visual grounding within the autoregressive generation paradigm. In existing frameworks, [SEG] is produced only at the end of a long reasoning chain. As the reasoning trajectory grows longer, the hidden states of later tokens become increasingly dominated by linguistic abstraction and autoregressive noise, progressively diluting the visual evidence required for accurate spatial grounding, as illustrated in Figure 2. Although shortening or removing the reasoning trajectory can partially alleviate this degradation, it is not a principled remedy: difficult samples often rely on multi-step visual and semantic inference, for which CoT reasoning remains essential, as detailed in § 4.4. Consequently, the model may struggle to preserve precise spatial cues in the predicted mask and, in more severe cases, even lose track of the intended target during reasoning. This information decay is particularly detrimental for RS imagery, where accurate segmentation requires maintaining fine-grained structural details across cluttered geographical scenes and extreme scale variations. To address these challenges, we propose GeoReason to strengthen RS-oriented visual reasoning while preserving spatial fidelity in segmentation. Our approach is built upon two core pillars:

Spatial fidelity degradation of masks induced by long reasoning chains

Figure 2. Spatial fidelity degradation of masks induced by long reasoning chains.

Anticipatory Prior via Prompt-side [SEG]. To mitigate the spatial fidelity degradation arising from the delayed generation of the segmentation token, we introduce an anticipatory prior mechanism that injects one more specialized [SEG] token directly into the user query. Unlike tokens generated autoregressively in the answer, this prompt-side token acts as a deterministic input that remains unaffected by the cumulative linguistic abstraction and autoregressive noise. By functioning as a persistent spatial anchor throughout the entire forward pass, the token can pre-extract and maintain critical visual evidence from the outset. This design effectively mitigates the information decay inherent in long-form RS reasoning, ensuring the spatial fidelity of the resulting latent representations.

Enhanced Visual Reasoning for RS Domain. To further address target misidentification in complex remote sensing scenes, we enhance the model’s visual reasoning through a dual strategy that combines explicit spatial grounding with implicit latent supervision. First, we introduce a Location-aware Chain-of-Thought (CoT) mechanism that decouples logical deduction from final mask extraction. By incorporating bounding box prediction as an intermediate reasoning step, this design enforces a coarse-to-fine localization process, allowing the model to verify candidate regions before final mask extraction and significantly reducing the risk of selecting incorrect targets. Complementing this explicit grounding, we propose a Latent Patch-to-Token Alignment (PTA) Loss to directly supervise the representation of the [SEG] token. Specifically, we align the hidden states of this token with the physical image patches corresponding to the ground-truth target. By grounding the latent reasoning token in RS-specific textures and multi-scale geometries, this alignment strengthens its discriminative capability and ensures reliable segmentation under cluttered geographical contexts.

Our main contributions are summarized as follows:

  • We propose GeoReason, a novel framework for remote sensing reasoning segmentation that addresses two critical failure modes in MLLM-based segmentation: Spatial Fidelity Degradation and Target Misidentification.

  • We introduce an Anticipatory Prior mechanism that injects the [SEG] token directly into the query, transforming it into a persistent spatial anchor and preventing visual evidence from being diluted during long-form reasoning.

  • We develop a Location-aware CoT together with a Latent PTA Loss, which jointly improve logical grounding and strengthen the spatial discriminability of the [SEG] token in complex RS scenes.

  • Extensive experiments demonstrate that GeoReason achieves state-of-the-art performance on RS reasoning segmentation benchmarks, significantly improving both reasoning accuracy and segmentation quality.

Related Work

Multimodal Large Language Models for Segmentation. MLLM-based segmentation has moved beyond VQA toward dense, language-conditioned prediction, and existing methods mainly follow two paradigms: token-based latent extraction (Embedding-as-Mask) and text-based spatial prompt generation [22, 33, 7, 27, 45, 58, 69]. LISA established the Embedding-as-Mask framework and the task of reasoning segmentation by appending a special [SEG] token whose hidden state prompts a mask decoder such as SAM [15, 13, 36, 3]. This design has since become a standard recipe for integrating dense prediction into MLLMs [50, 35, 63, 47], with extensions to multi-target reasoning in PixelLIM [37] and to video segmentation in SA2VA and related works [60, 53, 2, 6, 58]. In contrast, text-only alternatives predict explicit spatial prompts such as boxes or points and then invoke an external segmentor [28, 59, 69]; representative examples include Seg-Zero [28], Seg-R1 [59], and the “text-as-mask” line explored by Text4Seg and Text4Seg++ [16, 17].

Language-Guided Remote Sensing Segmentation. In remote sensing, language-guided segmentation [24] has likewise evolved from referring expression segmentation [30] to more demanding reasoning segmentation settings [19, 51, 31, 57]. Early RS-RES methods, such as LGCE [61] and RMSIN [25], fuse lightweight language encoders with visual backbones, while SAM-based variants, including RS2-SAM2 [38], RSRefSeg [5], and RSRefSeg2 [4], improve localization by translating language into sparse segmentation prompts. These models are effective for referring-style grounding but remain limited on multi-step reasoning. MLLM-based RS segmentation addresses this gap [19, 51]: several methods adopt the LISA-style [SEG] token for mask generation in referring settings [32, 39, 26], while SegEarth-R1 and SegEarth-R2 extend the same paradigm to the Earthreason benchmark and show that logical deduction remains difficult in cluttered RS scenes [19, 51].

Visual Reasoning in MLLMs. CoT reasoning in MLLMs has progressed from purely textual deliberation [48, 52, 49] toward “thinking with images” [44, 64]. Recent methods strengthen reasoning with explicit visual interaction, including active perception through iterative cropping or zooming in DeepEyes, Thyme, and Pixel Reasoner [67, 65, 44], and spatial anchoring via intermediate box prediction in TreeVGR [43]. In parallel, latent reasoning methods argue that part of the reasoning process should remain in feature space rather than text [8]; for example, LVR aligns latent tokens with image representations to encode complex visual constraints [18]. These trends motivate our combination of explicit spatial grounding and token-level latent supervision for RS reasoning segmentation.

Our Method

GeoReason follows the embedding-as-mask paradigm and combines an MLLM with SAM2 [36] for reasoning segmentation in remote sensing. The method has two main goals: preserve spatial fidelity during long-form reasoning, and improve target identification in cluttered RS scenes. To this end, we introduce a prompt-side [SEG] anchor (Anticipatory Prior), a lightweight location-aware CoT, and a Latent Patch-to-Token Alignment (PTA) loss.

System Architecture

We use Qwen3VL [1] as the MLLM backbone and SAM2 [36] as the mask decoder. Given an RS image II and an instruction TT, the model processes a prompt that contains a prompt-side [SEG] token and generates an answer that may contain one or more answer-side [SEG] tokens. For any [SEG] token, its last-layer hidden state hseg∈Rdh_{seg} \in\mathbb{R}^{d} is treated as a latent mask query and projected by an MLP adapter fϕf_{\phi} into the SAM2 space:

Mpred=D(Fimg,fϕ(hseg)),(1)\mathcal{M}_{pred} = \mathcal{D}\left(\mathcal{F}_{img}, f_{\phi}(h_{seg})\right), \tag*{(1)}

where Fimg\mathcal{F}_{img} denotes the SAM2 image features. During training, segmentation losses are applied to both the prompt-side [SEG] and the answer-side [SEG] tokens. During inference, each of these tokens predicts a candidate mask, and the final result is obtained by taking their union. Detailed prompt formats are provided in the appendix A.3.

Illustration of the prompt template and model response format

Figure 4. Illustration of the prompt template and model response format.

Anticipatory Prior via Prompt-side [SEG] Token

In standard embedding-as-mask pipelines, the decisive [SEG] token is generated only after the reasoning chain, making its embedding vulnerable to attention dilution and semantic drift. This is especially problematic in RS, where fine boundaries and small structures must be preserved throughout long multimodal reasoning. We therefore place a dedicated [SEG] token on the prompt side, so that segmentation intent is established before autoregressive generation starts.

This prompt-side token serves as an Anticipatory Prior: it interacts with the image from the beginning of the forward pass and remains available to subsequent answer tokens as a stable spatial reference. As a result, the final answer-side [SEG] does not need to reconstruct segmentation cues from scratch after a long reasoning chain; instead, it can inherit a spatial prior that mitigates information decay.

Enhanced Visual Reasoning for Remote Sensing

We complement the prompt-side anchor with one explicit reasoning component and one latent alignment component.

Location-aware Chain-of-Thought. We ask the model to predict a bounding box [xmin⁡,ymin⁡,xmax⁡,ymax⁡][x_{\min}, y_{\min}, x_{\max}, y_{\max}] in the <think> stage before producing the final [SEG]. This design is inspired by recent think-with-images reasoning strategies, but here it plays a lightweight role: providing an explicit coarse spatial hypothesis before dense mask decoding. It is simple but useful in RS scenes with severe clutter and scale variation. In our design, BBox reasoning provides coarse localization, while the [SEG] embedding remains the dense latent query used by SAM2 for final mask decoding.

Latent Vision Reasoning via PTA Loss. Mask supervision alone does not explicitly shape the [SEG] embedding to capture RS-specific textures and structures. We therefore introduce a Latent Patch-to-Token Alignment (PTA) loss that directly aligns each supervised [SEG] embedding with the target image patches. As shown in Figure 3, PTA improves the latent quality of the segmentation token rather than only its decoded mask.

Overview of the GeoReason framework

Figure 3. Overview of the GeoReason framework. (Left) Overall architecture of Anticipatory Prior mechanism, where a [SEG] token is injected into the user query to pre-anchor spatial evidence and mitigate information decay during long-chain reasoning. The MLLM is trained to output intermediate bounding boxes as spatial anchors (Location-aware CoT) before generating the final [SEG]. (Right) The Latent Patch-to-Token Alignment (PTA) loss aligns the hidden states of both prompt-side and answer-side [SEG] tokens with the visual patch embeddings corresponding to the GT target.

Given a set of visual patch tokens V={vi}i=1N\mathcal{V} = \{\mathbf{v}_i\}_{i=1}^{N} where vi∈RD\mathbf{v}_i \in\mathbb{R}^{D}, let Ivalid\mathcal{I}_{valid} denote the index set of patches overlapping with the ground-truth mask Mgt\mathcal{M}_{gt}. The target visual proxy zvis\mathbf{z}_{vis} is computed via average pooling:

zvis=1∣Ivalid∣∑i∈Ivalidvi.(2)\mathbf{z}_{vis} = \frac{1}{\lvert\mathcal{I}_{valid}\rvert} \sum_{i \in\mathcal{I}_{valid}} \mathbf{v}_i. \tag*{(2)}

To achieve cross-modal alignment, we project the [SEG] token embedding hseg\mathbf{h}_{seg} into the visual representational space via a Gated Linear Unit (GLU) head gθ(⋅)g_{\theta}(\cdot):

gθ(hseg)=Wdown(σ(Wgatehseg)⊙(Wuphseg)).(3)g_{\theta}(\mathbf{h}_{seg}) = \mathbf{W}_{down}\left(\sigma\left(\mathbf{W}_{gate}\mathbf{h}_{seg}\right) \odot\left(\mathbf{W}_{up}\mathbf{h}_{seg}\right)\right). \tag*{(3)}

where σ\sigma denotes the SiLU activation function. The PTA loss is then formulated as the cosine distance between the projected latent embedding and the visual proxy:

LPTA=1−gθ(hseg)⋅zvis∥gθ(hseg)∥∥zvis∥.(4)\mathcal{L}_{PTA} = 1 - \frac{g_\theta(h_{\mathrm{seg}}) \cdot z_{\mathrm{vis}}}{\lVert g_\theta(h_{\mathrm{seg}})\rVert\lVert z_{\mathrm{vis}}\rVert}. \tag*{(4)}

By minimizing LPTA\mathcal{L}_{PTA}, the model is encouraged to encode foreground-specific visual evidence directly in the [SEG] embedding, which in turn improves both prompt-side anchoring and answer-side mask decoding. In this sense, the [SEG] token also acts as a latent reasoning carrier, it is to make the latent representation explicitly segmentation-oriented.

Optimization Objective

GeoReason is trained with language, mask, and latent-alignment supervision. Let M^∈RH×W\hat{M} \in\mathbb{R}^{H \times W} denote the predicted mask and MgtM_{gt} the ground-truth mask. We use Binary Cross-Entropy and Dice losses for mask supervision:

Lbce=−1HW∑i,j[(Mgt)i,jlog⁡(M^i,j)+(1−(Mgt)i,j)log⁡(1−M^i,j)],(5)\mathcal{L}_{bce} = -\frac{1}{HW}\sum_{i,j}\left[(M_{gt})_{i,j}\log(\hat{M}_{i,j}) + (1-(M_{gt})_{i,j})\log(1-\hat{M}_{i,j})\right], \tag*{(5)}
Ldice=1−2∑i,jM^i,j⋅(Mgt)i,j+ε∑i,jM^i,j2+∑i,j(Mgt)i,j2+ε.(6)\mathcal{L}_{dice} = 1 - \frac{2\sum_{i,j}\hat{M}_{i,j}\cdot(M_{gt})_{i,j}+\varepsilon}{\sum_{i,j}\hat{M}_{i,j}^{2}+\sum_{i,j}(M_{gt})_{i,j}^{2}+\varepsilon}. \tag*{(6)}

We also apply LPTA\mathcal{L}_{PTA} in Eq. (4). These segmentation-related losses are imposed on both the prompt-side and answer-side [SEG] tokens, encouraging consistent spatial semantics across the full reasoning trajectory. The overall objective is:

L=λtextLtext+λmask(Lbce+Ldice)+λptaLpta,(7)\mathcal{L} = \lambda_{text}\mathcal{L}_{text} + \lambda_{mask}(\mathcal{L}_{bce}+\mathcal{L}_{dice})+\lambda_{pta}\mathcal{L}_{pta}, \tag*{(7)}

where Ltext\mathcal{L}_{text} is the standard next-token loss and λtext,λmask,λpta\lambda_{text},\lambda_{mask},\lambda_{pta} balance the three objectives.

Experiments

Experimental Setup

GeoReason uses Qwen3VL-4B [1] as the MLLM backbone and SAM2 [36] as the mask decoder, initialized from SA2VA [60]. To strengthen remote-sensing priors, we further fine-tune with filtered VQA and caption samples from GeoChatInstruct [14] and VRSBench [21], while removing any validation/test overlaps to avoid leakage. During training, the visual encoder is frozen, the LLM is optimized with LoRA [9], and the MLP adapter together with SAM2 are fully updated.

We evaluate on two representative RS segmentation benchmarks: EarthReason [19] for reasoning segmentation and RRSIS-D [25] for referring segmentation. Following prior works [5, 51, 15], we report gIoU and cIoU. Full training hyperparameters, data preprocessing, and prompt templates are provided in the appendix A.

Main Results

Tables 1 and 2 show that GeoReason performs strongly on both reasoning and referring segmentation, with the largest gains appearing in the reasoning setting that motivates our design.

Reasoning Segmentation. GeoReason establishes a new SOTA on EarthReason across all metrics. The runner-up differs by metric: our backbone-matched baseline is second-best on gIoU, while UniGeoSeg is second-best on cIoU. Relative to these strongest alternatives, GeoReason improves val/test gIoU by +2.57 / + 3.29 and val/test cIoU by +4.33 / + 5.99. Notably, the baseline already benefits from a strong Qwen3VL+SAM2 backbone, yet still trails GeoReason by a large margin, especially on test cIoU (80.60% vs. 70.45%). This shows that the gains do not come from backbone strength alone, but from stabilizing the segmentation token during long-form reasoning and improving its spatial grounding.

Referring Segmentation. On RRSIS-D, GeoReason achieves the best gIoU on both validation and test, and remains competitive in cIoU against specialized referring models. Since this benchmark involves little explicit reasoning, the gains mainly reflect better mask quality. This is consistent with the role of PTA: beyond helping reasoning segmentation, it also improves the discriminability of the [SEG] representation for standard referring scenarios.

Table 1: Comparison on EarthReason. “Baseline” denotes our backbone-matched post-hoc [SEG] model without Anticipatory Prior or Enhanced Visual Reasoning. Best results are in bold; second-best are underlined.

MethodValTest
gIoUcIoUgIoUcIoU
LISA [15]61.0457.3960.8859.10
PixelLM [37]57.9457.7960.0159.22
PSALM [66]66.6162.0368.3064.61
SegEarth-R1 [19]68.6064.1370.7568.25
RemoteReasoner [57]69.0067.8071.0069.10
Text4Seg++ [17]71.9069.8073.0065.60
SegEarth-R2 [51]72.3068.1073.5069.50
UniGeoSeg [31]72.5473.3073.0874.61
Baseline75.0470.3375.3270.45
GeoReason77.6177.6378.6180.60
Gain+2.57+4.33+3.29+5.99

Table 1.

Table 2: Comparison on RRSIS-D. “-” indicates that the metric was not reported by the original authors.

RRSIS-DValTest
gIoUcIoUgIoUcIoU
Traditional Methods
LGCE [61]60.1676.6859.3776.34
RIS-DMMI [10]60.7277.0160.1276.20
LAVT [55]61.4677.5961.0477.19
RMSIN [25]65.1078.2764.2077.79
MLLM-based Methods
LISA [15]27.84-26.78-
PixelLM [37]33.89-31.65-
NEXT-Chat [62]26.98-24.98-
GeoGround [68]61.10-60.50-
Text4Seg++ [17]64.1075.8062.8074.20
SegEarth-R1 [19]67.5678.9266.4078.01
GeoPixel [39]67.9981.7767.3084.90
GeoReason68.3679.8767.3279.39

Table 2.

Mechanistic Analysis of the Anticipatory Prior

To validate the role of the Anticipatory Prior, we compare four token sources on the EarthReason test set: the baseline answer-side [SEG] without CoT, the baseline answer-side [SEG] with CoT, the GeoReason answer-side [SEG], and the GeoReason prompt-side [SEG]. Figure 5a shows that long autoregressive reasoning weakens direct visual access in the baseline: attention mass from the final [SEG] to image tokens drops from 0.246 without CoT to 0.212 with CoT. GeoReason largely removes this degradation. Its prompt-side [SEG] has the strongest image attention (0.270), and its answer-side [SEG] retains 0.245, nearly recovering the no-CoT level despite a longer reasoning sequence. This is direct evidence that the prompt-side token acts as a persistent spatial anchor rather than an extra symbolic token.

Token-level mechanism analysis

Figure 5. (a) Direct attention from different [SEG] tokens to image tokens, averaged over the last four layers. (b) Foreground/background patch alignment and token-only mask quality of different [SEG] tokens.

We further ask whether the final answer-side [SEG] explicitly uses this anchor. After removing image and template tokens, we rank prompt tokens by their mean attention from the answer-side [SEG] over the last four layers. The prompt-side [SEG] has mean rank 2.01, appears in the top-3 attended prompt tokens for 99.69% of samples, and in the top-5 for 99.95%. The case heatmap in Figure 6b shows the same behavior at the instance level: over the last six layers, the answer-side [SEG] attends much more strongly to the prompt-side [SEG] than to content words such as fishing, leisure, and scenery. Figure 5b further shows that GeoReason also improves segmentation-oriented alignment: relative to the baseline CoT answer-side [SEG], the GeoReason answer-side [SEG] raises foreground similarity from 0.044 to 0.096 and enlarges the foreground-background margin from 0.035 to 0.056. The qualitative case in Figure 6a is consistent with these statistics, with GeoReason producing a more complete lake mask. Together, these results support our claim that AP preserves visual evidence while PTA makes the resulting [SEG] representation more discriminative for the target region.

Case-level evidence

Figure 6. Case-level evidence. (a) Qualitative comparison on a representative case. (b) Late-layer attention distribution of the final answer-side [SEG] token over the prompt-side [SEG] token and selected key words, with other tokens omitted for clarity.

Ablation Studies

Here we conduct ablation studies to disentangle the roles of the Anticipatory Prior (AP), the Location-aware CoT (BBox), and the Latent PTA loss (PTA). Table 3 reports the core component ablations on EarthReason and RRSIS-D, and Table 4 provides two targeted diagnostic variants.

ComponentsEarthreason (Reasoning)RRSIS-D (Referring)
APBBoxPTAVal gVal cTest gTest cVal gVal cTest gTest c
✗✗✗75.0470.3375.3270.4567.1379.3966.4278.67
✗✗✓76.0272.2676.3273.0167.9879.7166.9979.32
✗✓✗74.4270.8974.5470.5965.6076.1365.2976.43
✓✗✗76.2477.3278.0278.8767.5479.4566.1378.32
✓✗✓76.4575.6477.3977.1868.1279.8866.3278.67
✓✓✗76.6676.2477.2576.6267.5879.9366.2778.28
✓✓✓77.6177.6378.6180.6068.3679.8767.3279.39

Table 3. Ablation study of GeoReason components. AP: Anticipatory Prior, BBox: Location-aware CoT, PTA: Latent PTA Loss. g: gIoU, c: cIoU (%).

SettingValTestAvg
gIoUcIoUgIoUcIoU
Baseline75.0470.3375.3270.4572.79
GeoReason77.6177.6378.6180.6078.61
Shifted-SEG75.6573.6076.2271.0774.14
Zero-Reasoning75.6472.5972.8677.4374.63

Table 4. Diagnostic studies on EarthReason. Shifted-SEG: move the prompt-side [SEG] into the answer CoT. Zero-Reasoning: a short-answer baseline trained and evaluated without CoT. Avg is the mean of gIoU and cIoU across Val and Test.

Component contributions. AP is the dominant factor on reasoning segmentation: adding AP alone improves EarthReason test cIoU from 70.45% to 78.87%, confirming that early spatial anchoring is critical under autoregressive reasoning. PTA alone also gives a clear gain (70.45% → 73.01%) and is the main source of improvement on RRSIS-D, where mask quality matters more than long-form reasoning. By contrast, BBox alone is slightly below the baseline on EarthReason, indicating that longer reasoning does not help unless the segmentation token is first stabilized.

Complementarity. The full model performs best on both benchmarks. Partial variants such as AP+PTA or AP+BBox remain competitive but consistently trail the full design on EarthReason, showing that the three modules address different error sources: AP preserves spatial evidence, BBox improves coarse localization, and PTA sharpens the latent segmentation representation.

Diagnostic variants. Shifted-SEG keeps the additional [SEG] token but moves it from the prompt into the answer stream, reducing the average score to 74.14%. This confirms that the improvement does not come from simply adding another special token; what matters is prompt-side placement, which allows the token to condition the full forward pass. Zero-Reasoning, a short-answer variant without CoT, outperforms the full reasoning baseline (74.63% vs. 72.79% Avg), indicating that CoT can hurt segmentation if no spatial-preservation mechanism is provided. However, it still remains well below full GeoReason (78.61% Avg), showing that the goal is not to remove reasoning, but to make reasoning compatible with spatial fidelity. Figure 7 gives a example: for the query “In the context of this facility, what infrastructure do you consider vital for the safe storage of raw materials utilized in production?”, GeoReason reasons that the protected silo is the correct target, whereas the Zero-Reasoning model drifts to the exposed open area where materials are merely placed.

Reasoning-sensitive case study on EarthReason

Figure 7. Reasoning-sensitive case study on EarthReason. GeoReason correctly selects the protected structure required for safe raw-material storage, whereas the Zero-Reasoning variant overfits to a visually plausible but functionally incorrect open yard.

Quantifying the reasoning burden in RS. To verify that reasoning itself is a major bottleneck, we construct an Oracle Referring setting by rewriting each EarthReason query into a direct referring expression with Qwen3-30B [54], thereby preserving the target semantics while removing the need for autonomous logical deduction. This converts the task from reasoning segmentation into a simplified referring-style segmentation problem and thus approximates the segmentation ceiling once reasoning difficulty is stripped away. As shown in Table 5, the baseline reasoning model (72.79% Avg) is 8.05 below Oracle Referring (80.84% Avg), quantifying a substantial reasoning gap in RS segmentation. Importantly, GeoReason reaches 78.61% Avg, recovering 5.82 of this gap and leaving only a 2.23 residual to the oracle ceiling. This supports our central claim: beyond preserving spatial fidelity, GeoReason also strengthens reasoning itself, bringing complex reasoning segmentation substantially closer to direct referring segmentation. We further apply PTA on top of Oracle Referring. Even after the reasoning burden is removed, PTA still raises performance to 81.78% Avg and 84.14% test cIoU. This mirrors the gains on RRSIS-D and indicates that PTA is not merely compensating for reasoning errors; it acts as a task-agnostic enhancer of the [SEG] token by improving its latent alignment with RS textures and structures.

SettingValTestAvg
gIoUcIoUgIoUcIoU
Baseline75.0470.3375.3270.4572.79
GeoReason77.6177.6378.6180.6078.61
Oracle Ref.79.2082.1679.3482.6780.84
Oracle Ref. + PTA79.9482.7080.3384.1481.78

Table 5. Oracle-referring diagnostics on EarthReason. Oracle Ref.: convert each reasoning query into a direct referring expression using Qwen3-30B. Oracle Ref. + PTA: additionally apply PTA in this simplified setting. Avg is the mean of gIoU and cIoU across Val and Test.

Conclusion

We presented GeoReason, a framework for remote-sensing reasoning segmentation that addresses Spatial Fidelity Degradation and Target Misidentification. By combining a prompt-side [SEG] anchor, Location-aware CoT, and a PTA loss, GeoReason preserves visual evidence, improves grounding, and strengthens the segmentation-token representation. Experiments on EarthReason and RRSIS-D demonstrate state-of-the-art performance on both reasoning and referring segmentation.

Implementation Details

In this part, we provide additional details regarding the experimental setup, including data composition, hyperparameter configurations, and the prompt templates used for training and inference.

Data Composition and Preprocessing

To endow GeoReason with foundational geographical cognition and robust understanding of remote sensing (RS) semantic attributes, we incorporate auxiliary data from GeoChatInstruct [14] and VRSBench [21] during the training phase. These datasets are large-scale RS benchmarks covering diverse tasks. To maintain a focus on general RS perception, we exclusively extract the Visual Question Answering (VQA) and Captioning subsets.

Data Leakage Prevention. A significant portion of images in GeoChatInstruct and VRSBench originate from the DIOR dataset [20], which is also the source for the RRSIS-D [61] benchmark. To ensure the integrity of our evaluation and prevent data leakage, we implemented a strict data cleansing process. We identified and removed all images (and their corresponding crops) from the auxiliary sets that overlap with the validation or test sets of RRSIS-D. After filtering, we obtained 21.2k samples from GeoChatInstruct and 34.7k samples from VRSBench. During training, these samples are interleaved with the segmentation datasets. Notably, these auxiliary tasks do not involve the [SEG] token or mask generation.

Training Configurations

Optimization. We employ the AdamW [29] optimizer with an initial learning rate of 4×10−54 \times10^{-5} and a weight decay of 0.05. A linear warmup strategy is applied for the first 5% of iterations. To ensure training stability, we use gradient clipping with a maximum norm of 1.0.

Model Fine-tuning. The vision encoder of the MLLM remains frozen throughout the training process. The MLP adapter and the SAM2 mask decoder are fully updated. For the Large Language Model (LLM), we utilize LoRA with a rank r=128r = 128, α=256\alpha= 256, and a dropout rate of 0.05. The maximum sequence length is set to 8192. Regarding the multi-task optimization, the balancing coefficients λtext\lambda_{text}, λmask\lambda_{mask}, and λpta\lambda_{pta} are all set to 1.0 in our implementation.

Hardware and Efficiency. The model is trained for a single epoch with a total effective batch size of 64. We utilize 4×4 \times NVIDIA RTX Pro 6000 GPUs, employing a per-GPU batch size of 2 combined with 8-step gradient accumulation. The training is conducted using BF16 mixed-precision and takes approximately 9 hours to complete. Benchmarked on the EarthReason validation set, GeoReason increases latency (1223.6 ms vs. 774.1 ms) and token count (73.3 vs. 45.1) compared to the baseline (Qwen3VL-4B + SAM2 without our proposed methods). This overhead stems primarily from the autoregressive generation of reasoning and bounding box tokens rather than model complexity, representing a highly favorable trade-off for the significant performance gains.

Prompt Templates and Formats

We define specific prompt templates for Reasoning Segmentation (RS) and Referring Expression Segmentation (RES). The bounding box coordinates used in the <think> tags are derived from the ground-truth masks by calculating the minimum axis-aligned box that covers all target pixels.

Prompt Templates for GeoReason Training and Inference

Task A: Reasoning Segmentation

Query Template:

<image> [Question] When responding, please output your detailed reasoning and the bbox of the target in JSON format in <think> </think> tags, and then provide your answer in <answer> </answer> tags, i.e., <think> reasoning and bbox here </think> <answer> answer with [SEG] here </answer>.

Model Response Format:

<think> [Detailed Reasoning Content] The bbox of the target: {"bbox_2d": [xmin, ymin, xmax, ymax]} </think> \n <answer> [Answer Content] </answer>.


Task B: Referring Expression Segmentation

Query Template:

<image> [Question] When responding, please output the bbox of the target in JSON format in <think> </think> tags, and then provide your answer in <answer> </answer> tags, i.e., <think> bbox here </think> <answer> answer with [SEG] here </answer>.

Model Response Format:

<think> The bbox of the target: {"bbox_2d": [xmin, ymin, xmax, ymax]} </think> \n <answer> [Answer Content] </answer>.

Note: During training, both the [SEG] token in the query and the [SEG] token(s) in the <answer> tag are jointly supervised using Lbce\mathcal{L}_{bce}, Ldice\mathcal{L}_{dice}, and Lpta\mathcal{L}_{pta}.

Supervision and Inference. During training, our segmentation-related loss is applied to both the prompt-side [SEG] token and any [SEG] tokens generated within the <answer> tags. During inference, we extract the latent embeddings of the prompt-side [SEG] token and all [SEG] tokens present in the generated answer. Each token produces a candidate mask through the SAM2 decoder, and the final segmentation result is obtained by taking the union of all predicted masks.

Visual-Evidence Dilution in Long Reasoning

Analysis of Visual-Evidence Dilution

During visually grounded tasks, extended textual reasoning can weaken multimodal models’ reliance on visual evidence in subsequent answers. Several studies have examined this phenomenon. Sun et al. [41] analyze long-CoT visual forgetting: removing the image midway through reasoning changes MathVista-Hard accuracy only from 43.1% to 40.9%, while attention to visual tokens declines as reasoning proceeds. Tian et al. [42] use early decisions to examine reasoning progress: MMStar and HallusionBench accuracy drops by over two percentage points after its peak; among Vision-R1’s full-reasoning errors, 55.23% are perception errors, and 32.35% of these can be corrected at an earlier decision point. MME-CoT [12] reports negative stability when switching from direct answers to CoT on perception tasks for most models, with a maximum drop of 6.8 percentage points. OPERA [11] observes that longer generation can cause later tokens to over-trust nearby summary tokens and neglect earlier image tokens.

In the embedding-as-mask setting, longer reasoning may dilute visual evidence through two mechanisms: (i) visual-token competition, where additional reasoning tokens compete with image tokens for attention from the answer-side [SEG], and (ii) mask-query drift, where the evolving reasoning context shifts its hidden representation away from mask-relevant visual evidence. We analyze these mechanisms through image-attention mass and segmentation-query representation drift.

Let RL\mathcal{R}_L be a reasoning prefix of LL tokens, Iimg\mathcal{I}_{img} the image-token indices, and CL\mathcal{C}_L the context visible to the final answer-side [SEG]. For one attention head at layer ℓ\ell, its image-attention mass is

Aimg(ℓ)(L)=∑i∈Iimgexp⁡((qseg(ℓ)(L))Tki(ℓ)/d)∑j∈CLexp⁡((qseg(ℓ)(L))Tkj(ℓ)/d)(8)A^{(\ell)}_{img}(L)=\sum_{i\in\mathcal{I}_{img}}\frac{\exp\left((\mathbf{q}^{(\ell)}_{seg}(L))^{\mathsf{T}}\mathbf{k}^{(\ell)}_i/\sqrt{d}\right)}{\sum_{j\in\mathcal{C}_L}\exp\left((\mathbf{q}^{(\ell)}_{seg}(L))^{\mathsf{T}}\mathbf{k}^{(\ell)}_j/\sqrt{d}\right)} \tag*{(8)}

where dd is the head dimension. Additional reasoning tokens enlarge the competing linguistic context, while the segmentation query and attention logits can also change.

Using the notation of Sec. 3.1, the answer-side mask after this prefix is

Mpredans(L)=D(Fimg,fϕ(hsegans(L))).(9)\mathcal{M}^{ans}_{pred}(L) = \mathcal{D}\left(F_{img}, f_{\phi}\left(\mathbf{h}^{ans}_{seg}(L)\right)\right). \tag*{(9)}

We measure the representation change relative to the zero-length prefix as

Dh(L)=1−cos⁡(hsegans(0),hsegans(L)).(10)D_h(L) = 1 - \cos\left(\mathbf{h}^{ans}_{seg}(0), \mathbf{h}^{ans}_{seg}(L)\right). \tag*{(10)}

Within each model, the image features and decoder remain fixed across the intervention. Changes in hsegans(L)\mathbf{h}^{ans}_{seg}(L) can therefore alter the decoder’s latent condition without retraining or replacing the segmentation architecture.

Controlled Prefix-Length Experiments

We construct the experiment from the EarthReason test set, using one canonical trajectory saved from GeoReason’s naturally generated response for each instance. For a trajectory containing TT tokenizer tokens, we retain its first ⌊rT⌋\lfloor rT \rfloor tokens for r∈{0,0.25,0.5,0.75,1}r \in\{0, 0.25, 0.5, 0.75, 1\} and concatenate this prefix with the same final answer. We compare GeoReason with the Qwen3-VL+SAM2 baseline by feeding the resulting sequence into each model. To examine these mechanisms, we measure image attention and hidden-state drift of the answer-side [SEG] across prefix lengths.

Prefix (%)Image-attention massHidden-state drift
BaselineGeoReasonBaselineGeoReason
00.23890.25930.00000.0000
250.22060.25020.19550.0233
500.21350.24910.18960.0248
750.21990.24810.20970.0280
1000.20030.24460.20760.0495

Table 6. Effect of reasoning-prefix length on the answer-side [SEG] on EarthReason test. Image-attention mass is summed over image tokens and averaged across heads, the last four layers, and instances. Hidden-state drift is the mean cosine distance between the last-layer [SEG] representations at the retained and zero-length prefixes.

From the zero to the full prefix, the baseline’s image attention decreases from 0.2389 to 0.2003 (16.15% relative attenuation), whereas GeoReason changes from 0.2593 to 0.2446 (5.69%). GeoReason reduces the absolute endpoint attenuation by 61.7%. Its full-prefix hidden-state drift is 0.0495, compared with 0.2076 for the baseline, a 76.2% reduction. The baseline’s reduced image attention and increased hidden-state drift are consistent with visual-token competition and mask-query drift, respectively. GeoReason mitigates both effects, preserving visual access and stabilizing the segmentation query under longer reasoning prefixes.

Extended Ablation Studies on PTA Loss

We conduct extensive ablation experiments to validate the design of the Latent Patch-to-Token Alignment (PTA) loss, focusing on the PTA head architecture, spatial supervision granularity, pooling strategy, and loss formulations.

Architecture of the PTA Head. To distinguish from the mask-decoder adapter (fϕf_{\phi}), we denote the auxiliary alignment module as the PTA Head (gθg_{\theta}). This head projects the [SEG] token into a dedicated alignment space during training. As shown in Table 7, removing the PTA head (w/o PTA Head) leads to a significant performance drop, especially on EarthReason (−4.67%-4.67\% in accIoU). This observation supports our hypothesis that while the [SEG] token and image patch tokens share the same dimensionality, they reside in distinct semantic feature spaces. Directly forcing alignment without a cross-modal projection head (like our GLU-based gθg_{\theta}) compromises the token’s representational capacity, causing it to lose critical information required for subsequent mask generation. Furthermore, our GLU-based PTA head outperforms a simple MLP, effectively modeling complex latent interactions for better alignment.

Method / VariantRRSIS-D (Referring)EarthReason (Reasoning)
Val gIoUVal cIoUTest gIoUTest cIoUVal gIoUVal cIoUTest gIoUTest cIoU
GeoReason68.3679.8767.3279.3977.6177.6378.6180.60
w/o PTA Head68.26 -0.1078.26 -1.6167.24 -0.0878.38 -1.0177.20 -0.4175.92 -1.7177.31 -1.3075.93 -4.67
Simple PTA Head (MLP)68.39 +0.0378.85 -1.0267.46 +0.1478.80 -0.5977.51 -0.1078.60 +0.9778.34 -0.2779.78 -0.82
BBox-based Alignment68.56 +0.2079.26 -0.6167.28 -0.0478.65 -0.7477.26 -0.3577.71 +0.0878.64 +0.0379.47 -1.13
PTA Pooling: Attention67.76 -0.6079.05 -0.8266.69 -0.6378.11 -1.2877.47 -0.1477.47 -0.1678.10 -0.5178.65 -1.95
PTA Loss: MSE68.69 +0.3379.45 -0.4267.12 -0.2078.30 -1.0977.40 -0.2177.06 -0.5778.49 -0.1279.02 -1.58
PTA Loss: MAE68.27 -0.0979.60 -0.2767.23 -0.0978.61 -0.7877.54 -0.0777.22 -0.4178.66 +0.0578.40 -2.20

Table 7. Ablation studies on PTA Loss configurations. Δ\Delta denotes the performance gap relative to our final GeoReason model (red for decrease, blue for increase). Results are reported in (%).

Granularity of Spatial Supervision. We compare our mask-based alignment with a coarser bounding box-based strategy. While the BBox approach aligns the [SEG] token with all patches within the rectangular area, our default strategy strictly maps only positive pixels to their corresponding tokens. Table 7 shows that this shape-aware supervision is superior, as it filters out background noise within the BBox and provides more precise geometric guidance.

Pooling Strategy. We compare our default Average Pooling, which uniformly aggregates target patches, with Attention Pooling that dynamically weights patches using the [SEG] token as a query. Interestingly, Attention Pooling leads to performance degradation. This is likely because the attention mechanism over-focuses on local salient features while neglecting the overall footprint and boundaries, which are essential for maintaining the structural integrity of masks in complex remote sensing scenes.

Loss Formulation. Finally, we evaluate different loss functions for PTA. While MSE and MAE provide competitive results, Cosine Similarity achieves the most balanced performance across both benchmarks, demonstrating its robustness in aligning high-dimensional latent vectors.

Additional Benchmark Results on GeoSeg-Bench

GeoSeg-Bench [31] is a geospatial segmentation benchmark encompassing Interactive, Referring, and Reasoning tasks across diverse scenes and categories. To further demonstrate GeoReason’s generalizability across diverse and challenging geospatial contexts, we extend our evaluation to this benchmark. We evaluate the GeoReason checkpoint trained in the main experiments on the Referring and Reasoning subsets of GeoSeg-Bench, without fine-tuning on GeoSeg-1M, the training dataset introduced in UniGeoSeg [31]. Table 8 compares GeoReason with eleven methods evaluated without GeoSeg-1M fine-tuning, as well as fine-tuned Geopixel and Earthmind, using results reported in UniGeoSeg [31].

Table 8: Results on GeoSeg-Bench. FT indicates fine-tuning on the GeoSeg-1M training dataset [31]. Baseline scores are reproduced from UniGeoSeg [31]. Best results are in bold; second-best are underlined.

MethodFTReferring gIoUReferring cIoUReasoning gIoUReasoning cIoU
LISA [15]✗4.563.535.777.09
PixelLM [37]✗5.806.046.306.36
PSALM [66]✗18.9131.779.2711.88
HIPIE [46]✗39.1429.2511.767.90
SegLLM [47]✗35.2715.9016.4111.06
Geopixel [39]✗40.1437.3426.7127.36
Geopix [32]✗28.5228.8118.2520.80
RemoteSAM [56]✗29.3112.069.018.09
Earthmind [40]✗46.4844.5328.8031.01
LISAT [34]✗40.1437.8720.0922.06
SegEarth-R1 [19]✗9.3914.6511.9712.72
Geopixel [39]✓45.5042.7028.5130.25
Earthmind [40]✓49.2448.0925.7136.84
GeoReason✗56.6450.5032.4630.99

Table 8.

As shown in Table 8, GeoReason achieves the highest gIoU on both tasks among the compared methods. Compared with Earthmind, the next-best method without GeoSeg-1M fine-tuning, GeoReason improves Referring and Reasoning gIoU by 10.16 and 3.66 percentage points, respectively. Referring cIoU improves by 5.97 points, while Reasoning cIoU is nearly tied (30.99 versus 31.01, a difference of −0.02 points). Despite using no GeoSeg-1M training data, GeoReason also surpasses the fine-tuned baselines on multiple metrics. These results further demonstrate its effectiveness beyond EarthReason and RRSIS-D.

Qualitative Visualizations and Analysis

In this section, we provide detailed qualitative evidence to further elucidate how GeoReason addresses the challenges in remote sensing (RS) reasoning segmentation.

Case Study

We further illustrate GeoReason with a representative case study against the baseline (Qwen3VL-4B + SAM2 without our proposed components). As shown in Figure 8, the baseline attends to an incorrect structure and fails to isolate the intended target, whereas GeoReason correctly identifies the target region. A key difference is that GeoReason first localizes the relevant area through the intermediate bbox generated in CoT, which provides an explicit coarse spatial anchor before mask decoding.

Qualitative case study comparing the baseline and GeoReason

Figure 8. Qualitative case study comparing the baseline and GeoReason. GeoReason first grounds the correct region with an intermediate bbox in CoT, then exhibits stronger target-focused [SEG] and produces a more accurate final mask.

We then visualize the cosine similarity between the [SEG] hidden state and image patch tokens. Compared with the baseline, GeoReason produces a much stronger and more concentrated response on the true target region, while suppressing unrelated areas. This sharper contrast is consistent with the final segmentation result: GeoReason yields the correct mask, whereas the baseline remains distracted by competing regions and produces an inferior prediction.

This case supports the complementary roles of our components. The Location-aware CoT improves coarse target localization through bbox reasoning, while the Anticipatory Prior and PTA loss make the [SEG] representation more target-sensitive, leading to more reliable final segmentation.

Analysis of Typical Failure Modes

We further provide additional qualitative results to illustrate how GeoReason addresses common failure modes in remote sensing reasoning segmentation (Figure 9):

Six-row comparison of Question, Image, GT Mask, GeoReason, and Baseline columns. Questions: “What urban facility is strategically designed to allow quick medical evacuations and optimal entry for aerial emergency responders?” “What unique natural landscape would provide a secluded environment perfect for supporting a variety of bird species?” “If a farmer is considering using a natural source for irrigation on their land, what would be the best source to select, and for what reasons?” “What location offers a spacious facility perfect for organizing sports events and concerts, complete with ample seating for attendees?” “In what ways could this particular structural presence influence the strategic considerations for industrial growth initiatives in the surrounding area?” “If tasked with pinpointing an essential water management facility, which one would you choose to evaluate its effects on local farmland?”

Figure 9. Mitigation of typical failure modes in RS reasoning segmentation.

  1. Spatial Fidelity Degradation: Due to information decay, baseline models often generate noisy or fragmented masks with blurry boundaries. GeoReason establishes a spatial anchor on the prompt-side via the Anticipatory Prior, anchoring visual evidence before autoregressive generation to maintain mask integrity.

  1. Target Misidentification: Weak visual reasoning often leads standard models to segment incorrect objects or partial targets. Our Location-aware CoT enforces an explicit coarse localization (bounding box) before generating the [SEG] token, ensuring the model focuses on the correct location.

  1. Boundary Redundancy: Insufficient understanding of RS textures causes baseline masks to exhibit redundant fragments at the boundaries. By applying the PTA Loss, we align the [SEG] token with ground-truth image patches, enhancing its representational quality and eliminating blurry mask redundancies.

Limitations

Our study has several limitations. First, the main experiments use Qwen-VL and SAM-family models. Although the proposed design is modular, its transferability to other MLLM backbones or segmentation decoders has not yet been systematically evaluated. Second, our benchmarks mainly emphasize single-target segmentation, and more complex settings such as multi-target reasoning, heavy occlusion, or highly compositional queries remain for future study.

References

  1. [1]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025.arxiv.org/abs/2511.21631
  2. [2]Zechen Bai, Tong He, Haiyang Mei, Pichao Wang, Ziteng Gao, Joya Chen, Zheng Zhang, and Mike Zheng Shou. One token to seg them all: Language instructed reasoning segmentation in videos. Advances in Neural Information Processing Systems, 37:6833–6859, 2024.arxiv.org/abs/2409.19603
  3. [3]Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, et al. Sam 3: Segment anything with concepts. arXiv preprint arXiv:2511.16719, 2025.arxiv.org/abs/2511.16719
  4. [4]Keyan Chen, Chenyang Liu, Bowen Chen, Jiafan Zhang, Zhengxia Zou, and Zhenwei Shi. Rsrefseg 2: decoupling referring remote sensing image segmentation with foundation models. arXiv preprint arXiv:2507.06231, 2025.arxiv.org/abs/2507.06231
  5. [5]Keyan Chen, Jiafan Zhang, Chenyang Liu, Zhengxia Zou, and Zhenwei Shi. Rsrefseg: Referring remote sensing image segmentation with foundation models. In IGARSS 2025-2025 IEEE International Geoscience and Remote Sensing Symposium, pages 1070–1074. IEEE, 2025.arxiv.org/abs/2501.06809
  6. [6]Hao Fei, Shengqiong Wu, Hanwang Zhang, Tat-Seng Chua, and Shuicheng Yan. Vitron: A unified pixel-level vision llm for understanding, generating, segmenting, editing. Advances in neural information processing systems, 37:57207–57239, 2024.
  7. [7]Sitong Gong, Yunzhi Zhuge, Lu Zhang, Zongxin Yang, Pingping Zhang, and Huchuan Lu. The devil is in temporal token: High quality video reasoning segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 29183–29192, 2025.arxiv.org/abs/2501.08549
  8. [8]Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024.arxiv.org/abs/2412.06769
  9. [9]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3, 2022.arxiv.org/abs/2106.09685
  10. [10]Yutao Hu, Qixiong Wang, Wengi Shao, Enze Xie, Zhenguo Li, Jungong Han, and Ping Luo. Beyond one-to-one: Rethinking the referring image segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4067–4077, 2023.arxiv.org/abs/2308.13853
  11. [11]Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13418–13427. IEEE, 2024.arxiv.org/abs/2311.17911
  12. [12]Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li, Yu Qi, Xinyan Chen, Liuhui Wang, Jianhan Jin, Claire Guo, Shen Yan, et al. M^mecot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency. arXiv preprint arXiv:2502.09621, 2025.arxiv.org/abs/2502.09621
  13. [13]Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023.arxiv.org/abs/2304.02643
  14. [14]Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, and Fahad Shahbaz Khan. Geochat: Grounded large vision-language model for remote sensing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 27831–27840, 2024.arxiv.org/abs/2311.15826
  15. [15]Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9579–9589, 2024.arxiv.org/abs/2308.00692
  16. [16]Mengcheng Lan, Chaofeng Chen, Yue Zhou, Jiaxing Xu, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Text4seg: Reimagining image segmentation as text generation. arXiv preprint arXiv:2410.09855, 2024.arxiv.org/abs/2410.09855
  17. [17]Mengcheng Lan, Chaofeng Chen, Jiaxing Xu, Zongrui Li, Yiping Ke, Xudong Jiang, Yingchen Yu, Yunqing Zhao, and Song Bai. Text4seg++: Advancing image segmentation via generative language modeling. arXiv preprint arXiv:2509.06321, 2025.arxiv.org/abs/2509.06321
  18. [18]Bangzheng Li, Ximeng Sun, Jiang Liu, Ze Wang, Jialian Wu, Xiaodong Yu, Hao Chen, Emad Barsoum, Muhao Chen, and Zicheng Liu. Latent visual reasoning. arXiv preprint arXiv:2509.24251, 2025.arxiv.org/abs/2509.24251
  19. [19]Kaiyu Li, Zepeng Xin, Li Pang, Chao Pang, Yupeng Deng, Jing Yao, Guisong Xia, Deyu Meng, Zhi Wang, and Xiangyong Cao. Segearthl: Geospatial pixel reasoning via large language model. arXiv preprint arXiv:2504.09644, 2025.
  20. [20]Ke Li, Gang Wan, Gong Cheng, Liqiu Meng, and Junwei Han. Object detection in optical remote sensing images: A survey and a new benchmark. ISPRS journal of photogrammetry and remote sensing, 159:296–307, 2020.
  21. [21]Xiang Li, Jian Ding, and Mohamed Elhoseiny. Vrsbench: A versatile vision-language benchmark dataset for remote sensing image understanding. Advances in Neural Information Processing Systems, 37:3229–3242, 2024.
  22. [22]Jiazhen Liu, Mingkuan Feng, and Long Chen. Better, stronger, faster: Tackling the trilemma in mlm-based segmentation with simultaneous textual mask prediction. arXiv preprint arXiv:2512.00395, 2025.
  23. [23]Ruizhong Liu, Tingzhang Luo, Shaoguang Huang, Yuwei Wu, Zhen Jiang, and Hongyan Zhang. Crossmatch: Cross-view matching for semi-supervised remote sensing image segmentation. IEEE Transactions on Geoscience and Remote Sensing, 62:1–15, 2024.
  24. [24]Ruizhong Liu, Tingzhang Luo, Zaiyan Zhang, Jundong Chen, Hongruixuan Chen, Shaoguang Huang, and Hongyan Zhang. Geoseg-ov: Bridging geospatial gaps with structural guidance for open-vocabulary remote sensing segmentation, 2026. URL https://arxiv.org/abs/2608.10426.
  25. [25]Sihan Liu, Yiwei Ma, Xiaoqing Zhang, Haowei Wang, Jiayi Li, Xiaoshuai Sun, and Rongrong Ji. Rotated multi-scale interaction network for referring remote sensing image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26658–26668, 2024.
  26. [26]Xu Liu and Zhouhui Lian. Rsunivlm: A unified vision language model for remote sensing via granularity-oriented mixture of experts. arXiv preprint arXiv:2412.05679, 2024.
  27. [27]Ye Liu, Zongyang Ma, Junfu Pu, Zhongang Qi, Yang Wu, Ying Shan, and Chang Wen Chen. Unipixel: Unified object referring and segmentation for pixel-level visual reasoning. arXiv preprint arXiv:2509.18094, 2025.
  28. [28]Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520, 2025.
  29. [29]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.arxiv.org/abs/1711.05101
  30. [30]Tingzhang Luo, Ruizhong Liu, Yichao Liu, Cheng Fan, Yu Liu, and Jianyuan Guo. Cross: Cascaded distillation and dual-constraint grounding for remote sensing referring segmentation. In European Conference on Computer Vision, pages 1–21. Springer, 2026.
  31. [31]Shuo Ni, Di Wang, He Chen, Haonan Guo, Ning Zhang, and Jing Zhang. Unigeoseg: Towards unified open-world segmentation for geospatial scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 34865–34876, 2026.arxiv.org/abs/2511.23332
  32. [32]Ruizhe Ou, Yuan Hu, Fan Zhang, Jiaxin Chen, and Yu Liu. Geopix: A multimodal large language model for pixel-level image understanding in remote sensing. IEEE Geoscience and Remote Sensing Magazine, 2025.DOI
  33. [33]Rui Qian, Xin Yin, and Dejing Dou. Reasoning to attend: Try to understand how< seg> token works. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24722–24731, 2025.
  34. [34]Jerome Quenum, Wen-Han Hsieh, Tsung-Han Patrick Wu, Ritwik Gupta, Trevor Darrell, and David Chan. Lisat: Language-instructed segmentation assistant for satellite imagery. Advances in Neural Information Processing Systems, 38, 2026.arxiv.org/abs/2505.02829
  35. [35]Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S Khan. Glamm: Pixel grounding large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13009–13018, 2024.arxiv.org/abs/2311.03356
  36. [36]Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädlе, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024.arxiv.org/abs/2408.00714
  37. [37]Zhongwei Ren, Zhic​​heng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26374–26383, 2024.arxiv.org/abs/2312.02228
  38. [38]Fu Rong, Meng Lan, Qian Zhang, and Lefei Zhang. Rs2-sam2: Customized sam2 for referring remote sensing image segmentation. arXiv preprint arXiv:2503.07266, 2025.arxiv.org/abs/2503.07266
  39. [39]Akashah Shabbir, Mohammed Zumri, Mohammed Bennamoun, Fahad S Khan, and Salman Khan. Geopixel: Pixel grounding large multimodal model in remote sensing. arXiv preprint arXiv:2501.13925, 2025.arxiv.org/abs/2501.13925
  40. [40]Yan Shu, Bin Ren, Zhitong Xiong, Danda Pani Paudel, Luc Van Gool, Begüm Demir, Nicu Sebe, and Paolo Rota. Earthmind: Leveraging cross-sensor data for advanced earth observation interpretation with a unified multimodal llm. arXiv preprint arXiv:2506.01667, 2025.arxiv.org/abs/2506.01667
  41. [41]Hai-Long Sun, Zhun Sun, Houwen Peng, and Han-Jia Ye. Mitigating visual forgetting via take-along visual conditioning for multi-modal long cot reasoning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5158–5171, 2025.
  42. [42]Xinyu Tian, Shu Zou, Zhaoyuan Yang, Mengqi He, Fabian Waschkowski, Lukas Wesemann, Peter Tu, and Jing Zhang. More thought, less accuracy? on the dual nature of reasoning in vision-language models. In International Conference on Learning Representations, volume 2026, pages 133928–133965, 2026.arxiv.org/abs/2509.25848
  43. [43]Haochen Wang, Xiangtai Li, Zilong Huang, Anran Wang, Jiacong Wang, Tao Zhang, Jiani Zheng, Sule Bai, Zijian Kang, Jiashi Feng, et al. Traceable evidence enhanced visual grounded reasoning: Evaluation and methodology. arXiv preprint arXiv:2507.07999, 2025.arxiv.org/abs/2507.07999
  44. [44]Haozhe Wang, Alex Su, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. arXiv preprint arXiv:2505.15966, 2025.arxiv.org/abs/2505.15966
  45. [45]Xiaolong Wang, Lixiang Ru, Ziyuan Huang, Kaixiang Ji, Dandan Zheng, Jingdong Chen, and Jun Zhou. Argenseg: Image segmentation with autoregressive image generation model. arXiv preprint arXiv:2510.20803, 2025.arxiv.org/abs/2510.20803
  46. [46]Xudong Wang, Shufan Li, Konstantinos Kallidromitis, Yusuke Kato, Kazuki Kozuka, and Trevor Darrell. Hierarchical open-vocabulary universal image segmentation. Advances in Neural Information Processing Systems, 36:21429–21453, 2023.arxiv.org/abs/2307.00764
  47. [47]Xudong Wang, Shaolun Zhang, Shufan Li, Konstantinos Kallidromitis, Kehan Li, Yusuke Kato, Kazuki Kozuka, and Trevor Darrell. Segglm: Multi-round reasoning segmentation. arXiv preprint arXiv:2410.18923, 2024.arxiv.org/abs/2410.18923
  48. [48]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022.arxiv.org/abs/2201.11903
  49. [49]Yana Wei, Liang Zhao, Kanheng Lin, En Yu, Yuang Peng, Runpei Dong, Jianjian Sun, Haoran Wei, Zheng Ge, Xiangyu Zhang, et al. Perception in reflection. arXiv preprint arXiv:2504.07165, 2025.arxiv.org/abs/2504.07165
  50. [50]Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, and Gao Huang. Gsva: Generalized segmentation via multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3858–3869, 2024.arxiv.org/abs/2312.10103
  51. [51]Zepeng Xin, Kaiyu Li, Luodi Chen, Wanchen Li, Yuchen Xiao, Hui Qiao, Weizhan Zhang, Deyu Meng, and Xiangyong Cao. Seegarth-r2: Towards comprehensive language-guided segmentation for remote sensing images. arXiv preprint arXiv:2512.20013, 2025.arxiv.org/abs/2512.20013
  52. [52]Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2087–2098, 2025.arxiv.org/abs/2411.10440
  53. [53]Cilin Yan, Haochen Wang, Shilin Yan, Xiaolong Jiang, Yao Hu, Guoliang Kang, Weidi Xie, and Efstratios Gavves. Visa: Reasoning video object segmentation via large language models. In European Conference on Computer Vision, pages 98–115. Springer, 2024.arxiv.org/abs/2407.11325
  54. [54]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengeng Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
  55. [55]Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Hengshuang Zhao, and Philip HS Torr. Lavt: Language-aware vision transformer for referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18155–18165, 2022.arxiv.org/abs/2112.02244
  56. [56]Liang Yao, Fan Liu, Delong Chen, Chuanyi Zhang, Yijun Wang, Ziyun Chen, Wei Xu, Shimin Di, and Yuhui Zheng. Remotesam: Towards segment anything for earth observation. In Proceedings of the 33rd ACM international conference on multimedia, pages 3027–3036, 2025.arxiv.org/abs/2505.18022
  57. [57]Liang Yao, Fan Liu, Hongbo Lu, Chuanyi Zhang, Rui Min, Shengxiang Xu, Shimin Di, and Pai Peng. Remotereasoner: Towards unifying geospatial reasoning workflow. arXiv preprint arXiv:2507.19280, 2025.arxiv.org/abs/2507.19280
  58. [58]Kaining Ying, Henghui Ding, Guangquan Jie, and Yu-Gang Jiang. Towards omnimodal expressions and reasoning in referring audio-visual segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22575–22585, 2025.arxiv.org/abs/2507.22886
  59. [59]Zuyao You and Zuxuan Wu. Seg-r1: Segmentation can be surprisingly simple with reinforcement learning. arXiv preprint arXiv:2506.22624, 2025.arxiv.org/abs/2506.22624
  60. [60]Haobo Yuan, Xiangtai Li, Tao Zhang, Yueyi Sun, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, et al. Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos. arXiv preprint arXiv:2501.04001, 2025.DOI
  61. [61]Zhenghang Yuan, Lichao Mou, Yuansheng Hua, and Xiao Xiang Zhu. Rsis: Referring remote sensing image segmentation. IEEE Transactions on Geoscience and Remote Sensing, 62:1–12, 2024.arxiv.org/abs/2306.08625
  62. [62]Ao Zhang, Yuan Yao, Wei Ji, Zhiyuan Liu, and Tat-Seng Chua. Next-chat: An Imm for chat, detection and segmentation. arXiv preprint arXiv:2311.04498, 2023.arxiv.org/abs/2311.04498
  63. [63]Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, and Shuicheng Yan. Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding. Advances in neural information processing systems, 37:71737–71767, 2024.arxiv.org/abs/2406.19389
  64. [64]Xintong Zhang, Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, et al. Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl. arXiv e-prints, pages arXiv–2505, 2025.DOI
  65. [65]Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, Wei Chen, Xiao Hu, Bin Wen, Kaiyu Jiang, Changyi Liu, Tianke Zhang, et al. Thyme: Think beyond images. arXiv preprint arXiv:2508.11630, 2025.arxiv.org/abs/2508.11630
  66. [66]Zheng Zhang, Yeyao Ma, Enming Zhang, and Xiang Bai. Psalm: Pixelwise segmentation with large multi-modal model. In European Conference on Computer Vision, pages 74–91. Springer, 2024.arxiv.org/abs/2403.14598
  67. [67]Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing "thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362, 2025.arxiv.org/abs/2505.14362
  68. [68]Yue Zhou, Mengcheng Lan, Xiang Li, Litong Feng, Yiping Ke, Xue Jiang, Qingyun Li, Xue Yang, and Wayne Zhang. Geoground: A unified large vision-language model for remote sensing visual grounding. arXiv preprint arXiv:2411.11904, 2024.arxiv.org/abs/2411.11904
  69. [69]Muzhi Zhu, Yuzhuo Tian, Hao Chen, Chunluan Zhou, Qingpei Guo, Yang Liu, Ming Yang, and Chunhua Shen. Segagent: Exploring pixel understanding capabilities in mlmms by imitating human annotator trajectories. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 3686–3696, 2025.arxiv.org/abs/2503.08625

Paper details

vlnc:W_p8rfg3mb
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents