WorksComputer Science
FloodVLM-Pix: Language-Guided Pixel-Level Flood Understanding
Abstract
As climate change intensifies, floods are becoming more frequent and destructive, making accurate flood understanding an urgent priority. However, existing vision-language models perform poorly on flood-related tasks, suffering from attentional bias and inaccurate descriptions of flood magnitude. To bridge this gap, we construct the first large-scale remote sensing flood multi-modal benchmark that supports language-level flood understanding and pixel-level flood segmentation. The benchmark contains 131,200 flood image patches and corresponding 443,100 textual descriptions for flood severity and evolution. Based on this benchmark, we propose FloodVLM-Pix, a language-guided pixel-level flood understanding model that includes a flood prior module and a multi-temporal flood pixel decoder to achieve accurate flood understanding. The flood prior module automatically distills sub-region flood spatial priors and inputs priors into FloodVLM-Pix, solving the attentional bias problem in flood language understanding. The flood pixel decoder introduces a novel general-specific feature fusion mechanism to address performance degradation caused by interference (e.g., clouds or mountain shadows) during flood segmentation. Experiments on FloodVL-Pix demonstrate that FloodVLM-Pix improves both flood description quality and pixel-level flood segmentation performance compared with state-of-the-art VLMs and flood segmentation models. Our code and data will be released at https://github.com/Haomiaoyanmeng/FloodPix.
Introduction
Extreme flood events have become significantly frequent on a global scale [4, 12, 28, 29], making flood understanding (including language-level and pixel-level analysis) from satellite remote sensing imagery increasingly important for disaster response and damage assessment [39]. Much of the existing remote sensing research on floods primarily focuses on pixel-level flood segmentation [9, 17, 33, 37, 38, 39, 49]. Flood segmentation localizes inundation but lacks the semantic and temporal abstraction required to characterize severity, progression, or impact of floods, limiting
its utility for damage assessment and emergency response. Multi-temporal language-level flood analysis, in contrast, supplies this abstraction by translating visual evidence into descriptions that humans can rapidly interpret, enabling faster flood comprehension and more timely rescue action. Existing flood research remains flood-segmentation-centric and largely overlooks language-level flood understanding.
Although existing vision-language models (VLM) can provide linguistic descriptions of images, they struggle with flood-related tasks, exhibiting inaccurate understanding of flood extent and severity. The limitations of VLMs arise from three factors. 1) The flood extent can be extremely large, sometimes covering over 80% of a satellite image patch, inducing attentional bias that leads the model to misidentify the flood as background and describe non-flood areas instead, as shown in Figure 1(a). 2) Existing VLMs are predominantly trained on massive optical datasets, resulting in the limited capability of these VLMs to interpret synthetic aperture radar (SAR) images [54]. SAR imagery remains important for flood understanding due to its resistance to cloud cover and adverse weather conditions. 3) Flood images in each region are often taken at the same location but at different times and modalities. These images suffer from severe disturbances, including clouds, mountain shadows, agricultural areas, ambiguous wet regions, as shown in Figure 2.

Figure 1. A SAR image patch with flood areas in the upper and left parts. Inference results of Qwen3.6-Plus [3]: (a) without prior, (b) with prior. In the absence of a prior (a), the Qwen3.6-Plus misinterprets large-scale flooding as background and thus ignores it. After incorporating the prior in (b), the Qwen3.6-Plus yields the correct result.

Figure 2. (a) Clouds and cloud shadows, (b) agriculture, (c) mountain shadows, and (d) wet surface areas in optical or SAR images, degrading the performance of flood segmentation.
To bridge the aforementioned gaps, we propose a benchmark, FloodVL-Pix, and a model, FloodVLM-Pix, for accurate flood understanding. Our contributions can be summarized as follows:
We construct FloodVL-Pix, the first flood-oriented remote sensing multi-modal multi-temporal benchmark that supports both language-level flood understanding and pixel-level flood segmentation. FloodVL-Pix comprises 106,500 SAR image patches and 24,700 optical image patches, among which 11,000 have flood coverage exceeding 70%, providing abundant large-flood samples to mitigate attentional bias during optical and SAR flood interpretation. For researchers, FloodVL-Pix establishes a language-pixel-temporal paradigm for flood understanding, laying the foundation for next-generation disaster-oriented Earth observation VLMs that reason about disaster dynamics under real-world disturbances.
We propose FloodVLM-Pix, a language-guided pixel-level flood understanding model, including a flood prior module and a multi-temporal flood pixel decoder. The prior module provides accurate flood priors to the VLM, addressing the attentional bias during flood image patch interpretation. The pixel decoder adaptively fuses general shared semantic features and specific per-image-patch features of multi-temporal flood image patches to solve the performance degradation caused by disturbances shown in Figure 2. For government and emergency agencies, FloodVLM-Pix delivers pixel-level inundation maps and language-level severity/evolution reports for flood assessment, evacuation, and relief logistics. For citizens in flood-prone regions, FloodVLM-Pix translates complex satellite imagery into accessible flood descriptions and flood severity information, bridging the last mile from raw observations to timely disaster response.
Related Work
Flood Model/Benchmark
As shown in Table 1, existing remote sensing models and benchmarks designed for floods [6, 13, 35, 36, 39] mostly focus on the flood segmentation task. This task is initially formulated for bi-temporal flood image inputs [5, 9, 17, 33, 37, 38, 39]. For instance, [17] adaptively fuses SAR and multispectral images using a spatial mask and weight-shared decoder, achieving robust flood segmentation accuracy. Subsequently, some researchers begin to investigate flood segmentation of multi-temporal remote sensing flood images [6, 45, 49]. For example, [49] combines sparse Markov random field-based spatial clustering with temporal fusion of historical flood information to achieve robust continuous flood extent monitoring. However, none of these methods [5, 6, 9, 17, 33, 37, 38, 39, 45, 49] considers language-level flood understanding, which is essential for conveying flood severity and evolution and enabling rapid disaster interpretation and response.
Remote Sensing VLM/Benchmark
Early works on large-scale remote sensing models primarily focus on constructing vision-only foundation models [1, 2, 11, 30, 42, 51]. These works typically collect ultra-large-scale unlabeled remote sensing images and perform self-supervised learning to train foundation models. Subsequently, attention shifts to VLMs for remote sensing images [15, 16, 22, 47, 54], which enables deep language-level understanding of remote sensing tasks. Table 1 shows the capability coverage of existing VLMs/benchmarks across flood-oriented design, language support, pixel-level outputs, and multi-temporal analysis. For example, TeoChat [15] proposes a benchmark and a VLM for multi-temporal remote sensing data, enabling conversational reasoning over image sequences. To address the limitation that VLMs cannot output pixel-level results, [32, 41] propose pixel-level benchmarks and VLMs that incorporate pixel decoders, maintaining language understanding while yielding high-quality pixel-level outputs. However, multi-temporal understanding remains unexplored in the framework of [32, 41], limiting their ability to analyze evolution in remote sensing images over time. Moreover, the above remote sensing VLMs/benchmarks are not specifically designed for floods, and they exhibit attentional bias when interpreting flood im-
| Models (Benchmarks) | Flood-Oriented | Language | Pixel Results | Multi-Temporal () | |
| Flood Models | S1GFloods[39] | ✓ | ✗ | ✓ | ✗ |
| SEN12-FLOOD[36] | ✓ | ✗ | ✓ | ✓ | |
| CAU-Flood[13] | ✓ | ✗ | ✓ | ✗ | |
| FloodNet[35] | ✓ | ✗ | ✓ | ✗ | |
| Kuro Siwo[6] | ✓ | ✗ | ✓ | ✓ | |
| Vision-Language Remote Sensing Models | RingMo-GPT[47] | ✗ | ✓ | ✗ | ✗ |
| Earth-GPT[54] | ✗ | ✓ | ✗ | ✗ | |
| GeoChat[16] | ✗ | ✓ | ✗ | ✗ | |
| TeoChat[15] | ✗ | ✓ | ✗ | ✓ | |
| UniRS[18] | ✗ | ✓ | ✗ | ✓ | |
| Vision-Language-Pixel Remote Sensing Models | GeoPix[32] | ✗ | ✓ | ✓ | ✗ |
| Geopixel[41] | ✗ | ✓ | ✓ | ✗ | |
| SegEarth-R2[50] | ✗ | ✓ | ✓ | ✗ | |
| Proposed | FloodVLM-Pix | ✓ | ✓ | ✓ | ✓ |
caption: Comparison of different models/benchmarks in terms of flood-oriented design, language support, pixel-level labels, and multi-temporal analysis capability.
:::
agery, resulting in inaccurate language-level flood understanding.
Benchmark
Motivation. Optical remote sensing has been the mainstream data source for remote sensing VLMs, largely because large-scale image-text pretraining corpora are dominated by RGB optical imagery, whose visual cues are intuitive for cross-modal alignment. Floods are often accompanied by extreme precipitation and heavy cloud cover, making SAR images essential for reliable flood understanding due to the robustness of SAR imagery to adverse weather. However, existing VLMs struggle with SAR-based flood understanding due to mainly optical-image pretraining [25] and attentional bias toward overwhelmingly large flooded areas, as shown in Figure 1(a). To address these issues, we construct FloodVL-Pix, a dataset combining SAR and optical imagery to improve VLM performance on large-scale flood understanding in SAR and optical image patches.
Dataset and statistics. FloodVL-Pix is built from 247 global flood scenes, each comprising 100 regions (finer scene), and each region contains 2 ~ 13 multi-temporal, co-registered 256 256 flood image patches. We then annotate these patches with pixel-level flood segmentation labels and language descriptions. In total, FloodVL-Pix contains 131,200 image patches with pixel-level flood labels and corresponding 443,100 flood textual descriptions. Construction details are provided in the supplementary material. We set the patch size to 256 256 for three main reasons. 1) 256 256 is a common patch size in flood segmentation benchmarks [8, 13, 27, 31, 35, 39], enabling fair comparison. 2) For transformer-based VLMs, full self-attention has both computational and memory complexities of with respect to the token length . Since and in our work, the complexities become , so 256 256 patches substantially reduce computation and memory compared with large tiles such as 1024 1024. 3) Smaller patches yield finer-grained flood descriptions than large-tile interpretation, which benefits accurate flood response and rescue.
Figure 3 summarizes the statistics. FloodVL-Pix contains 106,500 SAR and 24,700 optical image patches, of which 11,300 patches have flood coverage, enabling VLM to effectively learn large-scale flood representations. Asia accounts for 59.5% of the data, because the FloodVL-Pix includes data from flood-prone regions such as South/Southeast Asia and China. The temporal image patches per region support understanding of flood dynamics and evolution.
Figure 3. The statistics of the FloodVL-Pix dataset. (a) Flood coverage ratio distribution. (b) Continent distribution. (c) Modality distribution. (d) Temporal image patch number distribution per region. Numbers inside/above bars denote counts/percentage of flood image patches or regions.
Model
Overview of FloodVLM-Pix
The overview architecture of FloodVLM-Pix is shown in Figure 4. FloodVLM-Pix consists of 1) a prior module that supplies language-level flood priors to calibrate VLM attention towards floods, 2) a VLM backbone (vision encoder, projector, and LLM) that takes flood image patches and flood-oriented questions and priors as input and produces flood descriptions, and 3) a pixel decoder that fuses general and specific multi-temporal features for pixel-level flood segmentation. The prior module and VLM backbone are depicted in Section 4.2, and the pixel decoder is discussed in Section 4.3.
Figure 4. Structure of our FloodVLM-Pix model. Our model consists of a vision encoder, a projector, an LLM, a prior module, and a pixel decoder. The vision encoder and projector convert image patches into tokens for LLM understanding. The prior module provides spatial priors for flood-related cues. The pixel decoder decodes image patch features to pixel-level flood segmentation results. The yellow and blue small rectangles below the LLM represent text tokens and visual tokens, respectively. “Image 1/2/3” denotes the order of input image patches.
Prior Module and VLM Backbone
Existing VLMs exhibit attentional bias in remote sensing flood image patch [Figure 1(a)]. We argue this bias stems from a mismatch between the object-centric assumption of natural-image pretraining and the scene-centric nature of floods. Natural-image pretraining treats targets as compact, locally salient objects, while large homogeneous regions are absorbed into the background. Floods, however, are spatially extended states with weak boundaries and can cover over 80% of an image patch. Consequently, attention is misallocated, treating flooded regions as background and underestimating flood extent. We therefore introduce sub-region-level flood extent priors, obtained by estimating sub-region flood ratios and verbalizing them as language-level evidence, to recalibrate attention toward flooded areas [Figure 1(b)] and improve flood severity and evolution-aware descriptions.
We first divide the flood image patch into five sub-regions: top-left (, ), top-right (, ), bottom-left (, ), bottom-right (, ), and center (, ). and are the height and width of the image patches, which are 256 in our work. These sub-regions are then fed into the prior module, where is the batch dimension. The prior module output representing the proportions of the sub-regions for flood occurrence and non-occurrence, respectively. For instance, an output value of [0.8, 0.2] for a sub-region indicates that the module estimates 80% of the region as flooded and 20% as non-flooded. The proportion label is obtained by computing the flood/non-flood area ratios in each sub-region from the flood segmentation labels. The prior module is trained using the mean squared error loss function i.e.,
to bring the predictions closer to the labels. Once the prior module is trained, we freeze it to provide stable flood priors for the LLM, preventing subsequent fine-tuning from corrupting the learned flood-prior representations or causing catastrophic forgetting. We then convert the output proportions of the prior module for each sub-region into language-level flood priors. For example, a top-left sub-region with an output of 0.9 for flood occurrence is described as “top left 90%”. The priors contain accurate flood information for each sub-region in each temporal flood image patch, guiding the VLM to focus on flooded areas and mitigate attentional bias toward non-flooded regions, thereby improving VLM understanding of flood severity and evolution.
Then, these flood priors, combined with flood-related questions as textual inputs, are used to fine-tune the LLM and projector to effectively learn flood representations from remote sensing flood image patches. Specifically, we initialize the vision encoder, projector, and LLM from Qwen3-VL-2B/4B [3]. To stabilize fine-tuning and reduce computational cost, we freeze the vision encoder and fine-tune the LLM using low-rank adaptation (LoRA) [14]. Meanwhile, we fine-tune all parameters of the projector to align flood representations across image and language modalities.
Pixel Decoder
The architecture of the pixel decoder is shown in Figure 5. Figure 5(a) shows the overview of the pixel decoder. To mitigate cross-modal interference during feature extraction, we employ separate optical and SAR encoders to independently extract modality-specific features from the optical and SAR flood image patches and the outputs from the Qwen3 vision encoder. Then the features from the optical and SAR encoders are concatenated in the batch dimension and input to the general-specific feature fusion module for deep-level multi-temporal feature fusion. Finally, the flood mask decoder decodes the fused features to the flood segmentation results.

Figure 5. Structures of our pixel decoder. (a) Overview of our pixel decoder. (b) Optical/SAR image patch encoder. (c) General-specific fusion module. (d) Flood mask decoder.
The image patch encoder is shown in Figure 5(b). We adopt ConvNeXt [20] as the feature extractor for flood image patches due to its hierarchical design and large-kernel depthwise convolutions, which facilitate multi-scale spatial feature learning and fine-grained texture capture. The feature size from the Qwen3 vision encoder is . We gradually upsample and convolve the outputs of the Qwen3 vision encoder and concatenate them with the outputs of ConvNeXt at different scales to obtain hierarchical image patch features , where denotes the modality of the output features and denotes different scales. We then concatenate the optical and SAR features () in a region (i.e., the features of all temporal image patches in a region) along the batch dimension to yield the concatenated features , where equals the number of temporal image patches of a flood region, and , , and denote the number of channels, height, and width of the , respectively. The contains the features from all modalities and temporal image patches in a region, enabling the general-specific module to sufficiently learn regional features.
The general-specific feature fusion module is the core of our pixel decoder, as shown in Figure 5(c). For image fusion in multi-temporal, multi-modal flood segmentation, the core challenge is that disturbances in flood images (i.e., clouds, shadows, wet soils, and irrigation) vary across modalities and times. These structured disturbances can mimic or obscure flood extent, requiring the fused representation to aggregate cross-modal and cross-temporal flood context under such disturbances while preserving image-specific information for pixel-level segmentation results. To address this challenge, we introduce general–specific feature fusion with two complementary feature spaces, where the general feature space captures region-level, cross-modal, cross-temporal flood features as robust contextual information, whereas the specific feature space preserves image-specific features for fine-grained flood texture delineation.
Specifically, at each scale , we treat the feature of the -th temporal image patch, , , as a specific feature. These specific features are averaged over the batch dimension to uniformly incorporate features from all modalities and temporal image patches, which makes the averaged feature less affected by disturbances in any specific feature. The averaged feature is then repeated across the batch to obtain the general feature , where the batch repeat aligns the size of with that of for subsequent concatenation and fusion. We then concatenate and , and apply a convolution followed by softmax to obtain a gate . The gate adaptively balances general and specific features to obtain fused feature
enabling the to remain robust to disturbances in flood image patches while preserving fine-grained flood details. The are then input to the flood mask decoder to obtain the flood segmentation results. As depicted in Figure 5(d), the hierarchical fused features are passed through multiple decoder blocks and upsampling operations, ultimately producing the segmentation results.
Experiment
Implementation Details
We adopt LSNet-S [46] as the backbone of the prior module. The prior module is trained for 100 epochs with a batch size of 128 using the AdamW [21] optimizer with a learning rate of , weight decay of 0.025, , and a cosine annealing schedule with a minimum learning rate of . Then, we leverage Qwen3-VL-2B/4B [3] as the backbone for the vision encoder, projector, and the LLM. The LLM is fine-tuned using LoRA [14] with a rank of 128, and the projection layer is fully fine-tuned. The vision encoder remains frozen. Fine-tuning is performed with a learning rate of for one epoch. Simultaneously with fine-tuning the LLM and projector, we train the pixel decoder with ConvNeXt [20] as the backbone for 100 epochs with a learning rate of . The entire FloodVLM-Pix model is trained on a single NVIDIA A100-80GB GPU.
Baselines
To rigorously evaluate the flood language understanding capability of FloodVLM-Pix, we establish two groups of baselines, i.e., general-domain VLMs and remote sensing VLMs. For general-domain VLMs, we evaluate several state-of-the-art (SOTA) models on the FloodVL-Pix dataset, including Kimi-2.6 [43], Doubao-Seed-2.0 (lite and pro), Doubao-Seed-2.1-turbo [40], GLM-5.3-Flash [53], DeepSeek-V4-Flash [52], and Qwen3-VL variants (2B, 4B, and 32B) [3], which represent advanced capabilities in general visual task processing. For remote sensing VLMs, we compare against GeoPixel-7B [41] and TeoChat-7B [15]. GeoPixel-7B is an advanced remote sensing VLM that supports language understanding of fine-grained objects in remote sensing imagery. TeoChat-7B is a SOTA multi-temporal remote sensing VLM capable of interpreting temporal changes in remote sensing imagery.
To accurately assess the flood segmentation capability of FloodVLM-Pix, we reproduce several SOTA flood segmentation methods, i.e., FTransUNet [23], MFGF-UNet [48], MFNet [24], and SMAGNet [17]. LSNet [46], a SOTA backbone, is also trained on the FloodVL-Pix to serve as a strong segmentation baseline.
Flood Description and Segmentation Results
Flood description results. In Table 2, we compare the flood language-level understanding capability of FloodVLM-Pix with SOTA general-domain and remote sensing VLMs. FloodVLM-Pix-2B(-4B) adopts Qwen3-VL-2B(-4B) as its backbone. All models are evaluated using a comprehensive set of metrics, i.e., recall-oriented understudy for gisting evaluation (ROUGE-L) [19], consensus-based image description evaluation (CIDER) [44], and criterion using language models for image caption rating (CLAIR) [7]. For all these metrics, higher scores indicate better performance in flood language description. Additional metrics and explanations for evaluating the flood description capability of the VLMs can be found in the supplementary material. Our FloodVLM-Pix-4B achieves SOTA performance on all metrics because we incorporate a prior module to guide the VLM toward accurate flood language understanding. The direct inference models (i.e., the models without †) in Table 2 show poor performance, stemming from limited SAR interpretation ability and attentional bias of VLMs during flood language understanding. Table 2 also reports results of models fine-tuned on our FloodVL-Pix dataset (Qwen3-VL-2B†/4B†, TeoChat-7B†, and GeoPixel-7B†), and these models also underperform FloodVLM-Pix because they lack designs to address the attentional bias for large-scale floods.
| Models | ROUGE-L | CIDER | CLAIR |
| Kimi-2.6 [43] | 0.2526 | 0.1056 | 0.5599 |
| Doubao-Seed-2.0-Lite [40] | 0.2449 | 0.0706 | 0.4097 |
| Doubao-Seed-2.0-Pro [40] | 0.2653 | 0.0924 | 0.4498 |
| Doubao-Seed-2.1-Turbo [40] | 0.2696 | 0.0929 | 0.3945 |
| GLM-5.3-Flash [53] | 0.2590 | 0.0888 | 0.5209 |
| DeepSeek-V4-Flash [52] | 0.2721 | 0.0967 | 0.4230 |
| Qwen3-VL-2B [3] | 0.2362 | 0.0601 | 0.3648 |
| Qwen3-VL-4B [3] | 0.2129 | 0.0666 | 0.3295 |
| Qwen3-VL-32B [3] | 0.2601 | 0.0962 | 0.5260 |
| Qwen3-VL-2B† [3] | 0.5034 | 0.3009 | 0.6758 |
| Qwen3-VL-4B† [3] | 0.5198 | 0.3135 | 0.7066 |
| GeoPixel-7B [41] | 0.0972 | 0.0062 | 0.0538 |
| TeoChat-7B [15] | 0.1578 | 0.0143 | 0.1572 |
| GeoPixel-7B† [41] | 0.3407 | 0.1478 | 0.3244 |
| TeoChat-7B† [15] | 0.5333 | 0.3233 | 0.7078 |
| FloodVLM-Pix-2B† | 0.5369 | 0.3328 | 0.7445 |
| FloodVLM-Pix-4B† | 0.5390 | 0.3381 | 0.7586 |
Table 2. Language-level flood understanding results of SOTA methods and FloodVLM-Pix. † denotes models fine-tuned on our dataset and models without † are evaluated by direct inference. The best result in each column is shown in blue, and the second-best is shown in red.
Flood Segmentation results. Table 3 shows the results of flood segmentation. We evaluate FloodVLM-Pix and SOTA methods on intersection over union (IoU) [34], F1-score (F1) [10], overall accuracy (OA) [26], precision [10], and recall [10]. Although LSNet [46] achieves high precision, it does so at the cost of low recall (i.e., many missed detections). Conversely, MFNet [24] and SMAGNet [17] obtain high recall values but suffer from high false alarms (low precision). Consequently, LSNet, MFNet, and SMAGNet suffer from low overall flood segmentation performance in terms of OA, F1, and IoU values. Our FloodVLM-Pix employs a gate to adaptively fuse general flood contextual features with single-image-specific features, achieving high overall segmentation performance. We also present visualization results of flood segmentation in Figure 6, where our models yield fewer missed detections, demonstrating the effectiveness of the proposed pixel decoder.

Figure 6. Results of the SOTA flood segmentation models and our proposed models. The white, red, blue, and black pixels are the regions of true positives, false alarms, missed targets, and true negatives, respectively. Our method yields fewer missed detections, demonstrating the effectiveness of the proposed pixel decoder.
| Model | Precision | Recall | OA | F1 | IoU |
| FTransUNet [23] | 0.6524 | 0.9583 | 0.8921 | 0.7763 | 0.6344 |
| LSNet [46] | 0.9156 | 0.8191 | 0.9557 | 0.8647 | 0.7616 |
| MFGF-UNet [48] | 0.8134 | 0.9294 | 0.9510 | 0.8675 | 0.7661 |
| MFNet [24] | 0.7514 | 0.9600 | 0.9301 | 0.8430 | 0.7286 |
| SMAGNet [17] | 0.7679 | 0.9622 | 0.9358 | 0.8541 | 0.7454 |
| FloodVLM-Pix-2B | 0.8407 | 0.9497 | 0.9606 | 0.8919 | 0.8049 |
| FloodVLM-Pix-4B | 0.8367 | 0.9266 | 0.9565 | 0.8793 | 0.7847 |
Table 3. Flood segmentation results of existing SOTA methods and FloodVLM-Pix. The best and second best results are in blue and red, respectively. Our FloodVLM-Pix-2B/4B achieve higher overall segmentation performance, i.e., OA, F1, IoU.
Ablation Experiments
Effect of Prior Module. We evaluate the flood language understanding performance of direct inference (baseline) and fine-tuning with/without the prior module on Qwen3-VL-2B, as shown in Figure 7. Fine-tuning includes LoRA-based fine-tuning [14] of the LLM and full fine-tuning of the projection layer, while the prior module is kept frozen, as depicted in § 4.2. “With/without prior module” denotes fine-tuning with/without the flood prior information output by the prior module. We draw three main conclusions. 1) Baseline performs poorly overall, especially when the flood ratio is high (). This is because when floods occupy a large proportion of the image patch, they are easily misclassified as background. 2) Fine-tuning (without the prior module) on our dataset brings substantial gains, with an average improvement of 31.1% in CLAIR, demonstrating that the VLM acquires stronger flood understanding capability than the baseline. For the high-flood-ratio cases (> 80%), the improvement of CLAIR reaches 36.3%, validating that fine-tuning significantly mitigates the tendency of the model to misclassify large-scale floods as background. 3) Incorporating the prior module yields an average gain of 6.9% in CLAIR. In the case of large-scale floods (> 80%), adding the prior module improves CLAIR by 8.0%, indicating that accurate flood priors recalibrate attention from small-scale objects toward flooded regions and supply explicit region-wise spatial information for severity and evolution-aware flood understanding. In the intermediate flood-ratio range (10%-80%), the improvements after using the prior module are also significant. In this range, floods often appear only in specific regions (e.g., the upper-left corner), and an accurate flood prior substantially helps the model understand where to attend, thereby improving the description performance of flood severity and evolution.
Figure 7. Flood description CLAIR of direct inference (baseline), fine-tuning with/without the prior module on Qwen3-VL-2B. Fine-tuning refers to LoRA-based fine-tuning [14] of the LLM and full fine-tuning of the projection layer, while the prior module is kept frozen. “w”/“w/o” prior module denotes fine-tuning with and without the flood prior information output by the prior module, respectively. Flood ratio is the proportion of the flood area in the image patch, and the last column (0%-100%) summarizes performance across all flood ratios, equating to the average CLAIR. Fine-tuning substantially improves performance, and incorporating the prior module further yields an average gain of 6.9% in CLAIR.
Architecture of the Pixel Decoder. We evaluate different architectures of pixel decoders and use three metrics, i.e., OA, F1, and IoU, to assess the flood segmentation capability of these pixel decoders, as reported in Table 4. First, we remove the layer normalization (LN) layers in the Qwen feature extractor (four upsampling + convolution + LN operations) in Figure 5(b). This leads to inferior flood segmentation performance, as the absence of LN causes unstable optimization and degraded feature representations, which directly impair the ability of the decoder to distinguish floods from background and thus lower the OA, F1, and IoU. Hence, LN is adopted throughout. We further replace the channel-wise concatenation in Figure 5(b) with element-wise addition and compare the results to investigate how different fusion strategies between features from the Qwen3 vision encoder and the original image patch affect flood segmentation performance. Concatenation yields better results, as it preserves full information and enables subsequent layers to selectively integrate information from both feature streams. In contrast, addition incurs information loss, degrading flood segmentation performance. Finally, we remove the general-specific feature fusion module or the features from the Qwen3 vision encoder entirely, both of which degrade flood segmentation performance, confirming the effectiveness of both components.
| Model | OA | F1 | IoU |
| w/o LN + Add | 0.9447 | 0.8525 | 0.7429 |
| LN + Add | 0.9539 | 0.8764 | 0.7800 |
| LN + Concat | 0.9606 | 0.8919 | 0.8049 |
| w/o General-Specific Fusion Module | 0.9564 | 0.8823 | 0.7894 |
| w/o Features from Qwen | 0.9556 | 0.8799 | 0.7856 |
Table 4. Ablation experiments of different pixel decoders. The “w/o LN”/“LN” means the Qwen feature extraction architecture in Figure 5(b) without/with layer normalization layers. “Add”/“Concat” denotes that either element-wise addition or channel-wise concatenation is employed to fuse features from the flood image patch and those from Qwen in Figure 5(b). The best and second best results are in blue and red, respectively.
Conclusion
We construct FloodVL-Pix, the first flood benchmark for language and pixel-level, multi-modal, multi-temporal flood understanding, advancing disaster-oriented Earth observation and flood response. We propose FloodVLM-Pix, which integrates a flood prior module and a multi-temporal flood pixel decoder to achieve accurate flood understanding. The flood prior module provides accurate priors to mitigate the attentional bias of VLMs in flood language understanding. The pixel decoder fuses general and specific features in a gated manner to aggregate cross-modal and cross-temporal flood context while preserving image-specific details, enabling accurate multi-temporal flood segmentation under various disturbances. Experiments show that FloodVLM-Pix outperforms SOTA methods in both flood description and segmentation.
Acknowledgment
This work was supported by National Key Research and Development Program of China under Grant 2025YFF0514602, Beijing Natural Science Foundation under Grant JQ25013, and National Natural Science Foundation of China under Grants 62622123, 62471274 and 62501359.
References
- [1]Guillaume Astruc, Nicolas Gonthier, Clement Mallet, and Loic Landrieu. Omnisat: Self-supervised modality fusion for earth observation. In European Conference on Computer Vision, pages 409–427. Springer, 2024. 2arxiv.org/abs/2404.08351
- [2]Guillaume Astruc, Nicolas Gonthier, Clement Mallet, and Loic Landrieu. Anysat: One earth observation model for many resolutions, scales, and modalities. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19530–19540, 2025. 2arxiv.org/abs/2412.14123
- [3]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. 1, 5, 6arxiv.org/abs/2511.21631
- [4]Günter Blöschl, Andrea Kiss, Alberto Viglione, Mariano Barriendos, Oliver Böhm, Rudolf Brázdil, Denis Coeur, Gaston Demarée, Maria Carmen Llasat, Neil Macdonald, et al. Current european flood-rich period exceptional compared with past 500 years. Nature, 583(7817):560–566, 2020. 1DOI
- [5]Derrick Bonafilia, Beth Tellman, Tyler Anderson, and Erica Issenberg. Sen1floods11: A georeferenced dataset to train and test deep learning algorithms for sentinel-1. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 210–211, 2020. 2DOI
- [6]Nikolaos Ioannis Bountos, Maria Sdraka, Angelos Zavras, Ilektra Karasante, Andreas Karavias, Themistocles Herekakis, Angeliki Thanasou, Dimitrios Michail, and Ioannis Papoutsis. Kuro siwo: 33 billion m² under the water. a global multi-temporal satellite dataset for rapid flood mapping. arXiv preprint arXiv:2311.12056, 2023. 2, 3
- [7]David M Chan, Suzanne Petryk, Joseph E Gonzalez, Trevor Darrell, and John Canny. Clair: Evaluating image captions with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13638–13646, 2023. 7
- [8]Georgios I Drakonakis, Grigorios Tsagkatakis, Konstantina Fotiadou, and Panagiotis Tsakalides. Ombrio—supervised flood mapping via convolutional neural networks using multitemporal sentinel-1 and sentinel-2 data fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15:2341–2356, 2022. 3
- [9]Shahan Garg, Antara Dasgupta, Mihail Motagh, Sandro Martinis, and Sivasakthy Selvakumaran. Unlocking the full potential of sentinel-1 for flood detection in arid regions. Remote Sensing of Environment, 315:114417, 2024. 1, 2
- [10]Cyril Goutte and Eric Gaussier. A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. In European conference on information retrieval, pages 345–359. Springer, 2005. 7
- [11]Xin Guo, Jiangwei Lao, Bo Dang, Yingying Zhang, Lei Yu, Lixiang Ru, Liheng Zhong, Ziyuan Huang, Kang Wu, Dingxiang Hu, et al. Skysense: A multi-modal remote sensing foundation model towards universal interpretation for earth observation imagery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27672–27683, 2024. 2arxiv.org/abs/2312.10115
- [12]Klaus Haslinger, Korbinian Breinl, Lovrenc Pavlin, Georg Pistotnik, Miriam Bertola, Marc Olefs, Marion Greilinger, Wolfgang Schöner, and Günter Blöschl. Increasing hourly heavy rainfall in austria reflected in flood changes. Nature, 639(8055):667–672, 2025. 1DOI
- [13]Xiaoning He, Shuangcheng Zhang, Bowei Xue, Tong Zhao, and Tong Wu. Cross-modal change detection flood extraction based on convolutional neural network. International Journal of Applied Earth Observation and Geoinformation, 117:103197, 2023. 2, 3DOI
- [14]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3, 2022. 5, 6, 7, 8arxiv.org/abs/2106.09685
- [15]Jeremy Irvin, Emily Liu, Joyce Chen, Ines Dormoy, Jinyoung Kim, Samar Khanna, Zhuo Zheng, and Stefano Ermon. Teochat: A large vision-language assistant for temporal earth observation data. In International Conference on Learning Representations, pages 68883–68911, 2025. 2, 3, 6arxiv.org/abs/2410.06234
- [16]Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, and Fahad Shahbaz Khan. Geochat: Grounded large vision-language model for remote sensing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 27831–27840, 2024. 2, 3arxiv.org/abs/2311.15826
- [17]Hyunho Lee and Wenwen Li. A spatially masked adaptive gated network for multimodal post-flood water extent mapping using sar and incomplete multispectral data. ISPRS Journal of Photogrammetry and Remote Sensing, 232:492–508, 2026. 1, 2, 6, 7DOI
- [18]Yujie Li, Wenjia Xu, Guangzuo Li, Zijian Yu, Zhiwei Wei, Junjun Wang, and Mugen Peng. Unirs: Unifying multi-temporal remote sensing tasks through vision language models. arXiv preprint arXiv:2412.20742, 2024. 3arxiv.org/abs/2412.20742
- [19]Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. 7
- [20]Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A conynet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11976–11986, 2022. 5, 6
- [21]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 6arxiv.org/abs/1711.05101
- [22]Junwei Luo, Zhen Pang, Yongjun Zhang, Tingzhu Wang, Linlin Wang, Bo Dang, Jiangwei Lao, Jian Wang, Jingdong Chen, Yihua Tan, et al. Skysensegpt: A fine-grained instruction tuning dataset and model for remote sensing vision- language understanding. arXiv preprint arXiv:2406.10100, 2024. 2
- [23]Xianping Ma, Xiaokang Zhang, Man-On Pun, and Ming Liu. A multilevel multimodal fusion transformer for remote sensing semantic segmentation. IEEE Transactions on Geoscience and Remote Sensing, 62:1–15, 2024. 6, 7DOI
- [24]Xianping Ma, Xiaokang Zhang, Man-On Pun, and Bo Huang. A unified framework with multimodal fine-tuning for remote sensing semantic segmentation. IEEE Transactions on Geoscience and Remote Sensing, 2025. 6, 7DOI
- [25]Zhiming Ma, Xiayang Xiao, Sihao Dong, Peidong Wang, HaiPeng Wang, and Qingyun Pan. Sarchat-bench-2m: a multi-task vision-language benchmark for sar image interpretation. arXiv preprint arXiv:2502.08168, 2025. 3arxiv.org/abs/2502.08168
- [26]Luís Marnoto, Alexandre Bernardino, and Bruno Martins. Generalized referring expression segmentation on aerial photos. arXiv preprint arXiv:2512.07338, 2025. 7arxiv.org/abs/2512.07338
- [27]Gonzalo Mateo-Garcia, Joshua Veitch-Michaelis, Lewis Smith, Silviu Vlad Oprea, Guy Schumann, Yarin Gal, Atilm Güneş Baydin, and Dietmar Backes. Towards global flood mapping onboard low cost satellites with machine learning. Scientific reports, 11(1):7249, 2021. 3
- [28]Amit Misra, Kevin White, Simone Fobi Nsutezo, William Straka III, and Juan Lavista. Mapping global floods with 10 years of satellite radar data. Nature Communications, 16(1):5762, 2025. 1arxiv.org/abs/2411.01411
- [29]Grey Nearing, Deborah Cohen, Vusumuzi Dube, Martin Gauch, Oren Gilon, Shaun Harrigan, Avinatan Hassidim, Daniel Klotz, Frederik Kratzert, Asher Metzger, et al. Global prediction of extreme floods in ungauged watersheds. Nature, 627(8004):559–563, 2024. 1DOI
- [30]Vishal Nedungadi, Ankit Karriyaa, Stefan Oehmcke, Serge Belongie, Christian Igel, and Nico Lang. Mmeearth: Exploring multi-modal pretext tasks for geospatial representation learning. In European Conference on Computer Vision, pages 164–182, 2024. 2arxiv.org/abs/2405.02771
- [31]Edoardo Nemni, Joseph Bullock, Samir Belabbes, and Lars Bromley. Fully convolutional neural network for rapid flood segmentation in synthetic aperture radar imagery. Remote Sensing, 12(16):2532, 2020. 3
- [32]Ruizhe Ou, Yunxi Hu, Fan Zhang, Jiaxin Chen, and Yu Liu. Geopix: A multimodal large language model for pixel-level image understanding in remote sensing. IEEE Geoscience and Remote Sensing Magazine, 2025. 2, 3DOI
- [33]Enrique Portales-Julia, Gonzalo Mateo-García, and Luis Gómez-Chova. Understanding flood detection models across sentinel-1 and sentinel-2 modalities and benchmark datasets. Remote Sensing of Environment, 328:114882, 2025. 1, 2
- [34]Md Atiqur Rahman and Yang Wang. Optimizing intersection-over-union in deep neural networks for image segmentation. In International symposium on visual computing, pages 234–244. Springer, 2016. 7DOI
- [35]Maryam Rahnemoonfar, Tashnim Chowdhury, Argho Sarkar, Debvrat Varshney, Masoud Yari, and Robin Roberson Murphy. Floodnet: A high resolution aerial imagery dataset for post flood scene understanding. IEEE Access, 9:89644–89654, 2021. 2, 3arxiv.org/abs/2012.02951
- [36]Clément Rambour, Nicolas Audebert, E Koeniguер, Bertrand Le Saux, MC Crucianu, and Mihai Datcu. Flood detection in time series of optical and sar images. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 43(B2):1343–1346, 2020. 2, 3DOI
- [37]Tamer Saleh, Shimaa Holail, Xionगwu Xiao, and Gui-Song Xia. High-precision flood detection and mapping via multi-temporal sar change analysis with semantic token-based transformer. International Journal of Applied Earth Observation and Geoinformation, 131:103991, 2024. 1, 2DOI
- [38]Tamer Saleh, Shimaa Holail, Mohamed Zahran, Xiongwu Xiao, and Gui-Song Xia. List-net: Enhanced flood mapping with lightweight sar transformer network and dimension-wise attention. IEEE Transactions on Geoscience and Remote Sensing, 62:1–17, 2024.DOI
- [39]Tamer Saleh, Xingxing Weng, Shimaa Holail, Chen Hao, and Gui-Song Xia. Dam-net: Flood detection from sar imagery using differential attention metric-based vision transformers. ISPRS Journal of Photogrammetry and Remote Sensing, 212:440–453, 2024. 1, 2, 3
- [40]Bytedance Seed. Seed2. 0 model card: Towards intelligence frontier for real-world complexity. URL https://lf3-static.bytednsdoc. com/obj/eden-cn/lapzild-tss/ljhwZthlaukjklulzlp/seed2/0214/Seed2. 0% 20Model%20Card. pdf, 2026. 6arxiv.org/abs/2607.00248
- [41]Akashah Shabbir, Mohammed Zumri, Mohammed Ben-namoun, Fahad S Khan, and Salman Khan. Geopixel: Pixel grounding large multimodal model in remote sensing. arXiv preprint arXiv:2501.13925, 2025. 2, 3, 6arxiv.org/abs/2501.13925
- [42]Xian Sun, Peijin Wang, Wanxuan Lu, Zicong Zhu, Xiaonan Lu, Qibin He, Junxi Li, Xuee Rong, Zhujun Yang, Hao Chang, et al. Ringmo: A remote sensing foundation model with masked image modeling. IEEE Transactions on Geoscience and Remote Sensing, 61:1–22, 2022. 2DOI
- [43]Kimi Team, Yifan Bai, Yiping Bao, Y Charles, Cheng Chen, Guanduo Chen, Haiting Chen, Huarong Chen, Jialhao Chen, Ningxin Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534, 2025. 6
- [44]Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015. 7arxiv.org/abs/1411.5726
- [45]Brandon Victor, Mathilde Letard, Peter Naylor, Karim Douch, Nicolas Longépé, Zhen He, and Patrick Ebel. Off to new shores: A dataset & benchmark for (near-) coastal flood inundation forecasting. Advances in Neural Information Processing Systems, 37:114797–114811, 2024. 2arxiv.org/abs/2409.18591
- [46]Ao Wang, Hui Chen, Zijia Lin, Jungong Han, and Guiguang Ding. Slenet: See large, focus small. In Proceedings of the computer vision and pattern recognition conference, pages 9718–9729, 2025. 6, 7
- [47]Peijin Wang, Huiyang Hu, Boyuan Tong, Ziqi Zhang, Fanglong Yao, Yingchao Feng, Zining Zhu, Hao Chang, Wenhui Diao, Qixiang Ye, et al. Ringmogpt: A unified remote sensing foundation model for vision, language, and grounded tasks. IEEE Transactions on Geoscience and Remote Sensing, 63:1–20, 2024. 2, 3DOI
- [48]Rongfang Wang, Chenchen Zhang, Chao Chen, Hongxia Hao, Weibo Li, and Licheng Jiao. A multi-modality fusion and gated multi-filter u-net for water area segmentation in remote sensing. Remote Sensing, 16(2):419, 2024. 6, 7DOI
- [49]Zhihao Wang, Xueqian Wang, Wei Wu, and Gang Li. Continuous change detection of flood extents with multisource heterogeneous satellite image time series. IEEE Transactions on Geoscience and Remote Sensing, 61:1–18, 2023. 1, 2DOI
- [50]Zepeng Xin, Kaiyu Li, Luodi Chen, Wanchen Li, Yuchen Xiao, Hui Qiao, Weizhan Zhang, Deyu Meng, and Xiangyong Cao. Segearth-r2: Towards comprehensive language-guided segmentation for remote sensing images. arXiv preprint arXiv:2512.20013, 2025. 3arxiv.org/abs/2512.20013
- [51]Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J Stewart, Joëlle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, and Xiao Xiang Zhu. Neural plasticity-inspired multimodal foundation model for earth observation. arXiv preprint arXiv:2403.15356, 2024. 2arxiv.org/abs/2403.15356
- [52]Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chenchen Ling, et al. Deepseek-v4: Towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348, 2026. 6arxiv.org/abs/2606.19348
- [53]Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. Glm-5: from vibe coding to agentic engineering. arXiv preprint arXiv:2602.15763, 2026. 6arxiv.org/abs/2602.15763
- [54]Wei Zhang, Miaoxin Cai, Tong Zhang, Yin Zhuang, and Xuerui Mao. Earthgpt: A universal multimodal large language model for multisensor image comprehension in remote sensing domain. IEEE Transactions on Geoscience and Remote Sensing, 62:1–20, 2024. 1, 2, 3arxiv.org/abs/2401.16822


