WorksComputer Science
FlowMoDE: Coarse-to-fine Flow Matching for Detail-aware Sim-to-Real Monocular Depth Estimation
Abstract
Monocular depth estimation is a fundamental capability for 3D spatial perception in robotic and intelligent systems. While data-driven discriminative methods have achieved compelling performance for monocular depth estimation, generative approaches can reconstruct fine structural details with stronger generalization. However, most existing generative diffusion models suffer from low inference efficiency caused by multi-step sampling, whereas single-step diffusion models struggle to recover high-fidelity depth maps. To mitigate these drawbacks, this paper presents FlowMoDE, a coarse-to-fine flow-matching-based framework for efficient, structure-aware monocular depth estimation supporting sim-to-real and cross-scene generalization. The core architecture first conducts coarse-level decoding conditioned on diffused features extracted from a pre-trained semantic encoder, and subsequently restores fine-grained structural patterns via expanded decoding layers. Trained exclusively on a single synthetic indoor dataset, our proposed method outperforms prior methods across five real-world indoor and outdoor datasets, yielding an average 14.1% reduction in relative error against state-of-the-art works as well as 9.9% reduction in chamfer distance for point cloud reconstruction. We further validate the generalization capability of FlowMoDE on realistic robotic scenes, with qualitative comparisons against state-of-the-art methods. The project page is https://flowmode-shao.netlify.app/
INTRODUCTION
Monocular Depth Estimation (MDE) is a core 3D perception technique that underpins a wide range of robotic downstream tasks, including scene reconstruction, robotic manipulation, and autonomous navigation. [1, 2] Due to its practical significance, numerous MDE methods have been developed, which are generally divided into discriminative and generative paradigms. Despite delivering promising overall depth prediction performance, discriminative methods inherently suffer from inaccurate depth estimation at object boundaries and fine-grained scene details, which severely degrades the geometric consistency of predicted depth maps. In contrast, generative models alleviate such local detail degradation issues yet introduce inevitable limitations: high computational cost and poor global scene contextual information, resulting in constrained overall prediction accuracy and robustness in complex scenarios.
Discriminative architectures [3, 4, 5, 6] commonly adopt a large-patch Vision Transformer encoder, paired with a convolutional up-sampling decoder. The encoder encodes input images into a low-resolution token grid and leverages global self-attention to capture comprehensive semantic information and scene layout. Such tokenization inevitably degrades pixel-level signals and fine-grained high-frequency details in the encoding phase. [7] Once these fine details are discarded in early encoding stages, convolutional up-sampling modules alone cannot completely restore them. As illustrated by the qualitative results within the green box in Figure 1, this issue yields erroneous depth predictions for fine-scale structures including wires, holes, and edges of distant objects. Moreover, discriminative depth models typically require large-scale training datasets covering diverse scenarios to achieve competitive performance. [8, 9] This imposes substantial costs for data collection and hinders their deployment in specialized scenarios where data acquisition is
Figure 1. Comparison with existing works. Qualitative comparison in green box demonstrates the outstanding structural details from our method, compared with state-of-the-art discriminative model MoGe-2 and generative model PPD. Bubble graph with purple box denotes comparison of performance, data efficiency and computational cost of the proposed model and existing models.
challenging, such as surgical environments. The comparison of data efficiency between generative models and discriminative models is demonstrated in purple box of Figure 1.
Current generative models [10], [11] commonly fine-tune Stable Diffusion to perform depth estimation. These approaches rely on a Variational Autoencoder (VAE) to project depth maps into a latent space. Such compression inherently sacrifices edge sharpness and structural fidelity. Therefore, recent studies have attempted to conduct diffusion directly in the pixel space for better recovery of structural details.[7] Nevertheless, the multi-step pixel-wise diffusion process introduces high computational overhead. While several existing approaches [12], [13] adopt single-step Stable Diffusion for depth estimation, the heavy VAE encoder and decoder still create significant computation bottlenecks. Meanwhile, single-step denoising can limit the performance of depth estimation.
In this work, we propose FlowMoDE, a coarse-to-fine flow matching model for computationally efficient sim-to-real pixel-accurate depth estimation. A lightweight decoder named as Semantic Decoder extracts global semantic features from a pre-trained discriminative encoder. Guided by global features, our model performs efficient decoding to establish a coarse depth distribution through Coarse Decoder. A subsequent Fine-grained Decoder with few wide Transformers further recovers delicate structural details to produce the final pixel-perfect depth map. Our main contributions can be summarized as the following:
We propose a novel flow matching framework for sim-to-real monocular depth estimation, which can provide structure-aware depth maps with lower computational cost.
We introduce coarse-to-fine semantic-guided velocity prediction which efficiently extracts coarse features based on semantic prior and further recover delicate details.
The proposed model is only trained on synthetic indoor dataset, but achieve state-of-the-art performance across five diverse datasets including both indoor and outdoor scenes. Further application of the proposed model in robotic scenes demonstrate the generalization and potential ability for spatial perception of robots.
RELATED WORK
Discriminative Monocular Depth Estimation
Early monocular depth estimation relied on hand-crafted visual priors and graphical models to infer scene layouts from single images. Deep learning redefined MDE as a supervised dense prediction task, giving rise to various CNN-based solutions such as [14]. Later relative-depth studies such as MiDaS [15] prioritized cross-dataset generalization, proving that mixing heterogeneous datasets improves zero-shot transfer. Meanwhile, DPT [16] firstly established a high-performance transformer-driven dense prediction framework, leading to a series of excellent ViT-based methods. Recent foundation models such as Depth Anything [8], [9] enhance open-domain robustness via large-scale training. In parallel, metric-depth approaches such as Metric3D [17] and UniDepth [5], [6] aim to reconstruct depth with absolute scale and have achieved steady improvements in accuracy and generalization. Despite these advances, modern discriminative models commonly adopt large-patch ViT encoders together with convolutional upsampling decoders. Many discriminative methods strengthen detail reconstruction through local refinement, high-resolution inference or multi-scale decoding. Depth-Pro [18] produces sharp metric depth via high-resolution architectures and boundary-aware training. MoGe [3] improves local geometry by predicting affine-invariant point maps under combined global and local geometric supervision, while MoGe-2 [4] further preserves fine details using refined training data. MDA [19] models per-pixel depth ambiguity with mixture-density representations to generate point maps without flying-point noise. InfiniDepth [20] formulates depth as a neural implicit field and employs local implicit decoders to support arbitrary-resolution queries and clearer depth details. Nevertheless, early tokenization in such architectures damages pixel-level cues and high-frequency components, making fine geometric structures hard to recover in decoding.
Generative Monocular Depth Estimation
Diffusion generative models have been applied in monocular depth estimation models such as Marigold [10] and MonoDiffusion [11], benefiting from rapid convergence and strong representation priors. PPD [7] introduces the injection of semantic features into late diffusion process to maintain global information and perform pixel-level denoise to provide pixel-perfect depth maps. However, the computational cost acutely increase due to multiple diffusion transformer layers. Recently, DepthMaster [13] utilize single-step UNet-based diffusion to perform generative depth estimation. However, heavy VAE still limit computational efficiency, while single-step denoise degrades the performance of the proposed model. DepthFM [21] firstly introduces flow matching into monocular depth estimation. However, lack of semantic information limits its performance, while VAE-based architecture still cause high computational cost and loss of structural details. To this end, FlowMoDE proposed in this work injects adjusted semantic features into few-step flow matching to efficiently perform structure-aware monocular depth estimation.
METHOD
Preliminaries: Flow Matching
Flow matching learns a time-dependent velocity field to transport samples from a simple source distribution to the data distribution over timestep . The generative process is governed by the ODE:
In conditional flow matching, the intermediate point is . The training loss minimizes the distance between the predicted velocity field and the analytical target :
Overview
The overall pipeline of the proposed method is demonstrated in Figure 2. Given an input image , the corresponding ground-truth depth map with noise and its semantic feature map , the proposed coarse-to-fine velocity prediction model predict the velocity map for interpolation matching at the training stage. At the inference stage, given the input image and random noise , FlowMoDE subsequently utilizes the shared coarse-to-fine velocity prediction model for twice to predict the depth map .
Figure 2. Pipeline of the proposed method.
Coarse-to-Fine Velocity Prediction
Firstly, shown as Eq. (3), semantic feature of the input image is extracted by a semantic encoder , which can be pretrained models such as DINOv2.
where is the dimension of the output from semantic encoder, which means . Moreover, is the dimension of the condition token, which is set to 384 in this work. The semantic feature is further processed by a Semantic Decoder with four Transformer blocks to obtain semantic condition .
The timestep is sampled from based on normal noise by equation Eq. (4). Meanwhile, the ground-truth depth map is added by noise through the process formulated as Eq. (5) and concatenated with the input image . The concatenated image is then embedded to patches as the input of Coarse Decoder.
where means linear normalization using – quantiles.
During training stage, the time condition is added with the semantic condition , then modulated to factors of scale, shift and gate for multi-head attention , , and MLP in feed forward network , , by SiLU activation and Linear projection layer, formulated as Eq. (6).
The coarse outputs the coarse feature map based on the embedded input through four conditioned Transformer blocks. For each conditioned Transformer block, denoted as the input and as the output, the modulated condition is injected into Transformer by Eq. (7) and Eq. (8) to guide the flow matching process with semantic information.
After coarse decoder, the output feature map is upsampled two times to patches. Meanwhile, the condition and timestep are also respectively upsampled to and by Eq. (9), where means repeat the nearest element in at each position of to upsample .
The fine-grained feature map is obtained from the coarse feature map by four conditioned Transformer block with condition . In the final layer, the fine-grained map goes through layer normalization followed by a linear layer with modulation of time token , shown as Eq. (10) and Eq. (11).
At last, we unpatchify the final feature map from the final layer to provide the velocity map .
Training: Single-step Flow Matching
Given ground-truth (GT) depth map and noise , the target velocity map is . The single coarse-to-fine flow matching model outputs predicted velocity map from the input image and noisy GT depth map from Eq. (5). We utilize MSE between prediction and target velocity as the loss function for training, formulated as Eq. (12).
where denotes the mask of valid depth, where value of invalid area will be set to 0.
Two-step Inference
During training stage, semantic tokens are extracted once from the input RGB image. Firstly, the shared coarse-to-fine network predicts velocity map from the concatenation of input RGB image and random noise . The original noise map is further diffused by the predicted velocity map, formulated as Eq. (13).
As Eq. (14) shows, concatenated with the input image is further processed by the shared coarse-to-fine flow matching model to generate the final velocity map and yield the final clean depth map .
EXPERIMENTS AND RESULTS
Implementation Details
We utilize frozen DINOv2-B from MoGe as our semantic encoder. The dimensions of semantic decoder, coarse decoder and fine-grained decoder are respectively set to 384, 384 and 768. We train the proposed model on single dataset Hypersim [28], which consists of 54K high-quality synthetic indoor frames and corresponding depth maps. The proposed model is trained for 200 epochs (2000 sampling steps for each epoch) with a batch size of 4 and a fixed learning rate of 1e-4, while all training samples are resized to 640×480. All experiments are implemented on a NVIDIA H100 GPU.
Comparison Results: Monocular Depth Estimation
All methods are evaluated on five diverse real-world datasets NYUv2 [29], KITTI [30], ETH3D [31], ScanNet [32] and DIODE [33] following Marigold [10], resizing all samples to a resolution with a long edge of 768. We compared our model with existing discriminative models and generative models. For DIODE, we directly use masks provided officially, while some works such as PPD [7] use Sobel to filter areas with high gradient and RANSAC to align pixels in their papers. We utilize absolute relative error (AR) and accuracy () metrics, as formulated as Eq. (15) and Eq. (16), where is the ground truth of the depth map and is the predicted depth map, to evaluate the performance.
The quantitative comparison results are shown in Table I. Our model outperforms existing generative models across five datasets, achieving state-of-the-art performance among all compared models with highest average rank on 10 metrics. Displaying results of two different indoor scenes and two different outdoor scenes, Figure 3 further demonstrate the outstanding performance of the proposed method.
| Method | Year | Training Data | FLOPs (G) | NYUv2 AR↓ | NYUv2 δ1↑ | KITTI AR↓ | KITTI δ1↑ | ETH3D AR↓ | ETH3D δ1↑ | ScanNet AR↓ | ScanNet δ1↑ | DIODE AR↓ | DIODE δ1↑ | Avg Rank |
| MiDaS [15] | 2020 | 2M | 184.6 | 11.1 | 88.5 | 23.6 | 63.0 | 18.4 | 75.2 | 12.1 | 84.6 | 33.2 | 71.5 | 19.2 |
| DiverseDepth [22] | 2021 | 320K | 207.8 | 11.7 | 87.5 | 19.0 | 70.4 | 22.8 | 69.4 | 10.9 | 88.2 | 37.6 | 63.1 | 19.6 |
| LeReS [23] | 2021 | 354K | 190.4 | 9.0 | 91.6 | 14.9 | 78.4 | 17.1 | 77.7 | 9.1 | 91.7 | 27.1 | 76.6 | 16.7 |
| Omnidata [24] | 2021 | 12.2M | 195.4 | 7.4 | 94.5 | 14.9 | 83.5 | 16.6 | 77.8 | 7.5 | 93.6 | 33.9 | 74.2 | 16.7 |
| DPT [16] | 2021 | 1.4M | 449.2 | 9.8 | 90.3 | 10.0 | 90.1 | 7.8 | 94.6 | 8.2 | 93.4 | 18.2 | 75.8 | 14.6 |
| HDN [25] | 2022 | 300K | 195.4 | 6.9 | 94.8 | 11.5 | 86.7 | 12.1 | 83.3 | 8.0 | 93.9 | 24.6 | 78.0 | 14.4 |
| Metric3D [17] | 2023 | 8M | 225.4 | 5.8 | 96.3 | 5.8 | 97.0 | 6.6 | 96.0 | 7.4 | 94.1 | 22.4 | 78.5 | 9.6 |
| DA [8] | 2024 | 62M | 586.0 | 4.3 | 98.1 | 7.6 | 94.7 | 6.5 | 97.2 | 4.2 | 98.0 | 27.7 | 75.9 | 6.6 |
| DA V2 [9] | 2024 | 63.5M | 1816.5 | 4.4 | 97.9 | 7.5 | 94.8 | 6.2 | 98.0 | 4.3 | 98.1 | 26.0 | 75.9 | 6.0 |
| UniDepth v2 [6] | 2025 | 16M | 1925.0 | 4.2 | 98.0 | 8.1 | 92.6 | 4.5 | 97.6 | 3.7 | 98.2 | 21.6 | 78.7 | 3.7 |
| MoGe [3] | 2025 | 9M | 3108.0 | 4.4 | 97.9 | 7.9 | 92.9 | 4.8 | 97.3 | 4.1 | 98.1 | 19.7 | 79.9 | 4.2 |
| InfiniDepth [20] | 2026 | ∼1M# | 1844.3 | 5.0 | 97.3 | 8.7 | 91.6 | 5.2 | 96.9 | 4.9 | 97.3 | 22.4 | 78.1 | 7.9 |
| Marigold [10] | 2024 | 74K | 5196.5 | 5.5 | 96.4 | 9.9 | 91.6 | 6.5 | 96.0 | 6.4 | 95.1 | 30.8 | 77.3 | 12.7 |
| GeoWizard [26] | 2025 | 280K | 5247.3 | 5.2 | 96.6 | 9.7 | 92.1 | 6.4 | 96.1 | 6.1 | 95.3 | 29.7 | 79.2 | 10.6 |
| DepthFM [21] | 2025 | 74K | 6192.5* | 5.5 | 96.3 | 8.9 | 91.3 | 5.8 | 96.2 | 6.3 | 95.4 | 22.4 | 78.5 | 10.4 |
| GenPercept [27] | 2025 | 74K | 5237.2* | 5.2 | 96.6 | 9.4 | 92.3 | 6.6 | 95.7 | 5.6 | 96.5 | 35.7 | 75.6 | 12.2 |
| Lotus [12] | 2025 | 59K | 5237.2* | 5.4 | 96.8 | 8.5 | 92.2 | 5.9 | 97.0 | 5.9 | 95.7 | 22.8 | 73.8 | 10.2 |
| DepthMaster [13] | 2026 | 74K | 5241.5* | 5.0 | 97.2 | 8.2 | 93.7 | 5.3 | 97.4 | 5.5 | 96.7 | 21.5 | 77.6 | 6.9 |
| PPD [7] | 2026 | 54K | 18158.8 | 4.3 | 97.4 | 8.0 | 93.1 | 4.5 | 97.7 | 4.5 | 97.3 | 24.5 | 76.9 | 6.1 |
| FlowMoDE | - | 54K | 3140.1 | 4.0 | 98.1 | 6.2 | 95.9 | 3.9 | 98.8 | 4.3 | 97.6 | 18.8 | 80.4 | 2.2 |
Table I. Results on Marigold benchmarks. *VAE encoder & decoder are not considered in original reported FLOPs calculation. #No reported value, estimated from the description in the original paper. Best models from all models are in bold, while second best models are underlined. Best results and second best results from Sim-to-Real Generative Models are marked by corresponding colors.

Figure 3. Qualitative comparison on indoor examples (first four rows) and outdoor examples (last four rows). Figures in the second row of each sample are GT and corresponding error maps based on Abs Rel.
Comparison Results: Point Cloud Reconstruction
To evaluate the downstream application and structural details of monocular depth estimation, we utilize NYUv2 and ETH3D, which provide camera intrinsics for 3D ground-truth, to compare point cloud reconstruction from predicted depth maps. We choose three discriminative models and two generative models with best performance on depth estimation
| Method | Near-field NYUv2 Acc↓ | Near-field NYUv2 CD↓ | Far-field ETH3D Acc↓ | Far-field ETH3D CD↓ | Mean Acc↓ | Mean CD↓ |
| UniDepth v2 | 86.7 | 117.2 | 332.4 | 427.1 | 209.6 | 272.2 |
| MoGe | 92.7 | 121.4 | 352.4 | 448.7 | 222.6 | 285.1 |
| InfiniDepth | 100.4 | 130.7 | 352.3 | 463.3 | 226.4 | 297.0 |
| DepthMaster | 100.0 | 120.9 | 321.8 | 322.3 | 210.9 | 221.6 |
| PPD | 172.2 | 160.3 | 840.7 | 596.4 | 506.5 | 378.4 |
| FlowMoDE | 89.3 | 100.2 | 269.6 | 299.1 | 179.5 | 199.7 |
Table 2. Quantitative comparison of point cloud reconstruction based on monocular depth estimation. Best results and second best results are marked by corresponding colors.
to compare with our model. Accuracy (Acc) and Chamfer Distance (CD) formulated as Eq. (18) and Eq. (19) based on point clouds from depth maps (Eq. (17)) are utilized as the metrics for point cloud reconstruction.
where is the point cloud from predicted depth map and is the point cloud from ground-truth depth map. For ETH3D dataset, we undistort RGB images and corresponding depth maps following official document 1 to match pinhole camera hypothesis before using Eq. (17).
Based on results shown in Table. II, our model achieve the outstanding performance with best average metrics across two datasets, while UniDepth v2 can provide results with better accuracy on NYUv2. Furthermore, we perform qualitative comparison in Figure 4. We can find that the proposed model can provide better spatial perception on structural details for delicate objects, such as the faucet in the first column of Figure 4 and windows in the second column of Figure 4.

Figure 4. Comparison of point cloud reconstruction based on estimated depth maps.
Comparison Results: 3D Perception in Robotic Scenes
In contrast to conventional settings, robotic environments typically contain multiple distinct agents that interact with each other. Therefore, we choose diverse cross-embodiment robotic scenes which respectively consist of Baxter, Franka, Kuka and Da Vinci Robot, to implement the proposed model for zero-shot monocular depth estimation. Compared with the most recent methods in Figure 5, we can find that our proposed method can perceive subtle details such as the delicate structure of robot body or cables. Meanwhile, based on 3D reconstruction results compared with PPD or MoGe-2 in Figure 6, FlowMoDE can better reconstruct the robot body independent of the environment or objects, reducing flying points between them.

Figure 5. Qualitative comparison of monocular depth estimation in robotic scenarios.

Figure 6. Qualitative comparison of point cloud reconstruction based on estimated depth maps in diverse cross-embodiment robotic scenes.
Ablation Studies
Firstly, we perform ablations on velocity prediction model by re-organizing semantic decoder, coarse decoder and fine-grained decoder with different numbers of layers. Meanwhile, we also use different injection methods including cross-attention, concatenation and adaptive normalization instead of our modulation to perform ablation studies. To control the computational cost of models, we keep the overall number of decoder layers as 12. According to results shown in Table III. Besides, we select several ablation settings with best performance to perform qualitative comparison in Figure 7 and compare point cloud reconstruction with our full model in Table. Table IV. Furthermore, we respectively conduct 1-step, 2-step, 4-step, 6-step and 8-step inference to demonstrate the influence of flow matching steps, results of which are displayed in Table V. We can find that 2-step inference via 4-layer decoders with the propsoed semantic injection is the best choice for monocular depth estimation.

Figure 7. Qualitative comparison of ablation settings.
| Ablation | NYUv2 Acc↓ | NYUv2 CD↓ | ETH3D Acc↓ | ETH3D CD↓ | Mean Acc↓ | Mean CD↓ |
| II | 118.0 | 119.9 | 455.4 | 383.1 | 286.7 | 251.5 |
| IV | 132.2 | 123.1 | 460.2 | 368.1 | 296.2 | 245.6 |
| V | 135.2 | 128.5 | 902.5 | 621.1 | 518.8 | 374.8 |
| Ours | 89.3 | 100.2 | 269.6 | 299.1 | 179.5 | 199.7 |
Table IV. Ablation results on point cloud reconstruction. Best models are in bold.
| Steps | NYUv2 AR | NYUv2 δ₁ | KITTI AR | KITTI δ₁ | ETH3D AR | ETH3D δ₁ | ScanNet AR | ScanNet δ₁ | DIODE AR | DIODE δ₁ |
| 1 | 4.4 | 98.1 | 7.5 | 97.5 | 4.6 | 98.5 | 4.5 | 97.6 | 18.8 | 80.3 |
| 2 | 4.0 | 98.1 | 6.2 | 95.9 | 3.9 | 98.8 | 4.3 | 97.6 | 18.8 | 80.4 |
| 4 | 4.1 | 98.0 | 6.5 | 95.6 | 4.0 | 98.7 | 4.3 | 97.5 | 18.9 | 80.3 |
| 6 | 4.2 | 98.0 | 6.7 | 95.4 | 4.1 | 98.7 | 4.4 | 97.5 | 19.0 | 80.2 |
| 8 | 4.2 | 98.0 | 6.7 | 95.4 | 4.1 | 98.7 | 4.4 | 97.5 | 19.0 | 80.2 |
| Methods | GFLOPs | ms/image | Mem (GB) | Params (M) | ||||||
| UniDepth v2 | 1925.0 | 23.3 | 3.98 | 353.8 | ||||||
| MoGe | 3108.0 | 56.8 | 3.65 | 314.2 | ||||||
| DepthMaster | 5241.5 | 94.7 | 6.71 | 1028.2 | ||||||
| PPD | 18158.8 | 193.7 | 4.22 | 804.2 | ||||||
| FlowMoDE | 3140.1 | 53.4 | 1.87 | 398.7 |
Table V. Ablation studies on flow matching steps. Best models are in bold, while second best models are underlined.
| # | Semantic Decoder | Coarse Decoder | Fine-grained Decoder | NYUv2 AR↓ | NYUv2 δ₁ ↑ | KITTI AR↓ | KITTI δ₁ ↑ | ETH3D AR↓ | ETH3D δ₁ ↑ | ScanNet AR↓ | ScanNet δ₁ ↑ | DIODE AR↓ | DIODE δ₁ ↑ |
| Full Model | 4.0 | 98.1 | 6.2 | 95.9 | 3.9 | 98.8 | 4.3 | 97.6 | 18.8 | 80.4 | |||
| I | 4-Layer | 8-Layer | × | 5.8 | 97.2 | 8.3 | 94.0 | 6.1 | 97.4 | 5.6 | 96.8 | 21.7 | 78.7 |
| II | 4-Layer | × | 8-Layer | 5.5 | 97.5 | 8.9 | 93.8 | 5.9 | 97.5 | 5.3 | 97.2 | 20.7 | 79.6 |
| III | × | 12-Layer | × | 6.2 | 96.8 | 10.5 | 89.6 | 8.2 | 95.7 | 6.1 | 96.3 | 22.1 | 77.9 |
| IV | × | × | 12-Layer | 5.5 | 97.4 | 9.8 | 92.3 | 6.9 | 97.0 | 5.4 | 96.9 | 21.4 | 79.0 |
| V | × | 6-Layer | 6-Layer | 5.5 | 97.3 | 9.8 | 92.5 | 6.0 | 97.6 | 5.3 | 96.8 | 20.5 | 79.8 |
| VI | 4-Layer | 6-Layer | 2-Layer | 6.0 | 96.9 | 8.8 | 93.7 | 6.3 | 97.3 | 6.0 | 96.2 | 21.6 | 78.6 |
| VII | 2-Layer | 6-Layer | 4-Layer | 6.0 | 96.8 | 9.3 | 92.8 | 6.6 | 97.1 | 5.8 | 96.6 | 22.5 | 77.9 |
| VIII | Cross Attention | 6.3 | 96.6 | 34.2 | 42.5 | 8.2 | 94.5 | 6.0 | 96.5 | 23.5 | 75.6 | ||
| IX | Concatenation | 5.2 | 97.5 | 8.2 | 94.1 | 5.5 | 97.9 | 5.3 | 97.1 | 20.7 | 79.6 | ||
| X | AdaNorm | 5.5 | 97.3 | 9.2 | 93.2 | 5.9 | 97.3 | 5.4 | 97.0 | 20.8 | 79.7 | ||
| XI | No Semantic | 20.6 | 65.5 | 26.0 | 56.9 | 20.1 | 69.2 | 18.7 | 69.9 | 32.6 | 59.1 |
Table III. Ablation Studies on Velocity Prediction Model.
Computational Cost
We compare the computational cost of our model with several existing models with relatively best performance. We use GPU memory requirement (Mem) and number of
parameters (Params) to evaluate the model size. Meanwhile, GFLOPs and inference speed (ms per image) are utilized to evaluate the efficiency of models. As shown in Table VI, the proposed FlowMoDE incurs higher latency and model weight overhead compared with UniDepth v2, but achieves significantly lower computational cost than prior generative depth models.
CONCLUSIONS
In this paper, we present FlowMoDE, a flow-matching generative model for sim-to-real monocular depth estimation to recover fine structural details. Trained solely on synthetic data, our model outperforms prior methods on five diverse benchmarks. We further validate its strong ability to reconstruct fine-grained geometry in various robotic scenarios against state-of-the-art approaches. With fewer Transformer blocks and minimal inference steps, FlowMoDE is lighter and more efficient than existing generative depth models. Still, patch embedding in our flow-matching pipeline may produce occasional flying pixels. Moreover, training only on near-field indoor synthetic data combined with patch-wise denoising can cause blurred edges for distant regions.
References
- [1]J. Liu, H. Ma, Y. Guo, Y. Zhao, C. Zhang, W. Sui, and W. Zou, “Monocular depth estimation and segmentation for transparent object with iterative semantic and geometric fusion,” in 2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 11 162–11 168.arxiv.org/abs/2502.14616
- [2]R. Fan, W. Zhao, M. Lin, Q. Wang, Y.-J. Liu, and W. Wang, “Generalizable thermal-based depth estimation via pre-trained visual foundation model,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 14 614–14 621.
- [3]R. Wang, S. Xu, C. Dai, J. Xiang, Y. Deng, X. Tong, and J. Yang, “MoGe: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision,” in 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025, pp. 5261–5271.arxiv.org/abs/2410.19115
- [4]R. Wang, S. Xu, Y. Dong, Y. Deng, J. Xiang, Z. Lv, G. Sun, X. Tong, and J. Yang, “Moge-2: Accurate monocular geometry with metric scale and sharp details,” Advances in Neural Information Processing Systems, vol. 38, pp. 35 928–35 959, 2026.arxiv.org/abs/2507.02546
- [5]L. Piccinelli, Y.-H. Yang, C. Sakairdis, M. Segu, S. Li, L. Van Gool, and F. Yu, “UniDepth: Universal monocular metric depth estimation,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024, pp. 10 106–10 116.
- [6]L. Piccinelli, C. Sakairdis, Y.-H. Yang, M. Segu, S. Li, W. Abbeloos, and L. Van Gool, “Unidepthv2: Universal monocular metric depth estimation made simpler,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.DOI
- [7]G. Xu, H. Lin, H. Luo, X. Wang, J. Yao, L. Zhu, Y. Pu, C. Chi-, H. Sun, B. Wang et al., “Pixel-perfect depth with semantics-prompted diffusion transformers,” Advances in Neural Information Processing Systems, vol. 38, pp. 174 731–174 755, 2026.arxiv.org/abs/2510.07316
- [8]L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth anything: Unleashing the power of large-scale unlabeled data,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024, pp. 10 371–10 381.arxiv.org/abs/2401.10891
- [9]L. Yang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, and H. Zhao, “Depth anything v2,” Advances in neural information processing systems, vol. 37, pp. 21 875–21 911, 2024.arxiv.org/abs/2406.09414
- [10]B. Ke, A. Oubokhov, S. Huang, N. Metzger, R. C. Daudt, and K. Schindler, “Repurposing diffusion-based image generators for monocular depth estimation,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024, pp. 9492–9502.arxiv.org/abs/2312.02145
- [11]S. Shao, Z. Pei, W. Chen, D. Sun, P. C. Chen, and Z. Li, “Monodiffusion: Self-supervised monocular depth estimation using diffusion model,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 4, pp. 3664–3678, 2024.arxiv.org/abs/2311.07198
- [12]J. He, H. Li, W. Yin, Y. Liang, L. Li, K. Zhou, H. Zhang, B. Liu, and Y. Chen, “Lotus: Diffusion-based visual foundation model for high-quality dense prediction,” in International Conference on Learning Representations, vol. 2025, 2025, pp. 89 454–89 467.arxiv.org/abs/2409.18124
- [13]Z. Song, Z. Wang, B. Li, H. Zhang, R. Zhu, L. R. Liu, P.-T. Jiang, and T. Zhang, “Depthmaster: Taming diffusion models for monocular depth estimation,” IEEE Transactions on Circuits and Systems for Video Technology, 2026.DOI
- [14]H. Fu, M. Gong, C. Wang, K. Batmanghelich, and D. Tao, “Deep ordinal regression network for monocular depth estimation,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018, pp. 2002–2011.arxiv.org/abs/1806.02446
- [15]R. Ranftl, K. Lasinger, K. Hafner, K. Schindler, and V. Koltun, “Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 3, pp. 1623–1637, 2020.
- [16]R. Ranftl, A. Bochkovskiy, and V. Koltun, “Vision transformers for dense prediction,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021, pp. 12 159–12 168.arxiv.org/abs/2103.13413
- [17]W. Yin, C. Zhang, H. Chen, Z. Cai, G. Yu, K. Wang, X. Chen, and C. Shen, “Metric3d: Towards zero-shot metric 3d prediction from a single image,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023, pp. 9009–9019.arxiv.org/abs/2307.10984
- [18]A. Bochkovskiy, A. Delaunoy, H. Germain, M. Santos, Y. Zhou, S. Richter, and V. Koltun, “Depth pro: Sharp monocular metric depth in less than a second,” in International Conference on Learning Representations, vol. 2025, 2025, pp. 75 602–75 637.
- [19]S. Bian, C. Xu, and J. Gao, “Modeling depth ambiguity: A mixture-density representation for flying-point-free depth estimation,” arXiv preprint arXiv:2606.02552, 2026.arxiv.org/abs/2606.02552
- [20]H. Yu, H. Lin, J. Wang, J. Li, Y. Wang, X. Zhang, Y. Wang, X. Zhou, R. Hu, and S. Peng, “InfInDepth: Arbitrary-resolution and fine-grained depth estimation with neural implicit fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2026, pp. 26 920–26 930.arxiv.org/abs/2601.03252
- [21]M. Gui, J. Schusterbauer, U. Prestel, P. Ma, D. Kotovenko, O. Grebenkova, S. A. Baumann, V. T. Hu, and B. Ommer, “Depthfm: Fast generative monocular depth estimation with flow matching,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 3, 2025, pp. 3203–3211.
- [22]W. Yin, X. Wang, C. Shen, Y. Liu, Z. Tian, S. Xu, C. Sun, and D. Renin, “DiverseDepth: Affine-invariant depth prediction using diverse data,” arXiv preprint arXiv:2002.00569, 2020.arxiv.org/abs/2002.00569
- [23]W. Yin, J. Zhang, O. Wang, S. Niklaus, L. Mai, S. Chen, and C. Shen, “Learning to recover 3d scene shape from a single image,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 204–213.arxiv.org/abs/2012.09365
- [24]A. Eftekhar, A. Sax, J. Malik, and A. Zamir, “Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021, pp. 10 766–10 776.arxiv.org/abs/2110.04994
- [25]C. Zhang, W. Yin, B. Wang, G. Yu, B. Fu, and C. Shen, “Hierarchical normalization for robust monocular depth estimation,” Advances in Neural Information Processing Systems, vol. 35, pp. 14 128–14 139, 2022.arxiv.org/abs/2210.09670
- [26]X. Fu, W. Yin, M. Hu, K. Wang, Y. Ma, P. Tan, S. Shen, D. Lin, and X. Long, “Geowizard: Unleashing the diffusion priors for 3d geometry estimation from a single image,” in European Conference on Computer Vision. Springer, 2024, pp. 241–258.
- [27]G. Xu, M. Liu, C. Fan, K. Xie, Z. Zhao, H. Chen, C. Shen et al., “What matters when repurposing diffusion models for general dense perception tasks?” in International Conference on Learning Representations, vol. 2025, pp. 6786–6799.arxiv.org/abs/2403.06090
- [28]M. Roberts, J. Ramapuram, A. Ranjan, A. Kumar, M. A. Bautista, N. Paczan, R. Webb, and J. M. Susskind, “Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021, pp. 10 892–10 902.arxiv.org/abs/2011.02523
- [29]N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” in European conference on computer vision. Springer, 2012, pp. 746–760.DOI
- [30]A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The kitti dataset,” The international journal of robotics research, vol. 32, no. 11, pp. 1231–1237, 2013.DOI
- [31]T. Schöps, J. L. Schönberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, and A. Geiger, “A multi-view stereo benchmark with high-resolution images and multi-camera videos,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3260–3269.DOI
- [32]A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” in 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, 2017, pp. 2432–2443.arxiv.org/abs/1702.04405
- [33]I. Vasiljević, N. Kolkin, S. Zhang, R. Luo, H. Wang, F. Z. Dai, A. F. Daniele, M. Mostajabi, S. Basart, M. R. Walter et al., “DIODE: A dense indoor and outdoor depth dataset,” arXiv preprint arXiv:1908.00463, 2019.arxiv.org/abs/1908.00463

