WorksComputer Science
Selective Generative Restoration for Video Deraining via Uncertainty-Aware Diffusion
Abstract
Video deraining is challenging due to rain streaks, occlusions, and atmospheric effects. Existing methods mainly follow a deterministic restoration paradigm, assuming that clean scenes can be uniquely recovered from rainy observations. However, rain degradation is spatially heterogeneous. Reliable regions require faithful reconstruction, whereas severely occluded regions contain ambiguous content that require stronger prior-driven inference. Therefore, conventional deterministic restoration often produces over-smoothed results, while global generative restoration may introduce unnecessary hallucinations. We introduce a Selective Generative Restoration paradigm for video deraining, where generative priors are adaptively activated according to restoration uncertainty. Based on this principle, we propose Selective Generative Video Deraining (SGVD), an uncertainty-aware diffusion framework that enables adaptive fidelity preservation and generative recovery. Specifically, SGVD exploits stochastic diversity between dual one-step diffusion predictions to estimate region-level restoration uncertainty, which determines where observation fidelity should be preserved and where generative inference should be activated. Furthermore, an anchor-guided trajectory consistency mechanism is proposed to regularize stochastic diffusion evolution across frames, reducing temporal generation drift and improving video coherence. To achieve practical efficiency, SGVD adopts one-step diffusion inference while maintaining uncertainty-aware generative capability. Extensive experiments demonstrate that SGVD achieves superior restoration performance, perceptual quality, and temporal consistency compared with state-of-the-art video deraining methods.
Introduction
Outdoor videos captured under rainy conditions often suffer from severe degradation caused by rain streaks, raindrop occlusions, rain accumulation, and atmospheric veiling effects. These degradations destroy scene structures and
temporal consistency, significantly affecting downstream vision tasks such as detection [1], segmentation [2], tracking [3], and autonomous driving [4]. Therefore, robust video deraining is essential for reliable perception under adverse weather conditions.
Existing video deraining methods have achieved remarkable progress through convolutional networks [5, 6], recurrent architectures [7, 8], Transformers [9, 10], and state space models [11, 12], mainly by improving spatial representation and temporal dependency modeling. However, most methods follow a deterministic restoration paradigm that maps each rainy observation to a single clean output. Such a formulation assumes that the underlying scene can be uniquely recovered from degraded observations, which becomes problematic under severe rain occlusions where background information is partially missing and multiple plausible restorations may exist.
Unlike uniformly degraded restoration problems, rain degradation is inherently spatially heterogeneous. Reliable regions preserve sufficient observations and mainly require fidelity-oriented reconstruction, whereas heavily occluded regions contain uncertain contents and require stronger prior-driven inference. Therefore, video deraining is not merely a reconstruction problem, but a spatially varying inverse problem with different levels of restoration uncertainty. This observation reveals a fundamental limitation of existing restoration paradigms. Global deterministic restoration [13, 14, 15] tends to over-smooth ambiguous regions, while global generative restoration may introduce unnecessary hallucinations into reliable regions. Consequently, effective video deraining requires an adaptive strategy that determines where observation fidelity should dominate and where generative priors should be activated.
Recent diffusion models provide a promising direction by modeling conditional distributions rather than predicting a single deterministic solution. Their powerful generative priors make them suitable for recovering missing contents from ambiguous observations. However, directly applying diffusion models to video deraining remains challenging because rain degradation exhibits strong spatial uncertainty variation. As illustrated in Figure 1, uniformly applying diffusion priors may improve uncertain regions but simultaneously alter trustworthy contents. Thus, the key problem is not whether diffusion should be introduced, but how to selectively activate generation according to restoration uncertainty.

Figure 1. Motivation of Selective Generative Restoration. Deterministic restoration tends to produce over-smoothed details in severely degraded regions, while global generative restoration may introduce content hallucination in reliable regions. In contrast, the proposed paradigm performs region-adaptive restoration by preserving fidelity in low-uncertainty regions and selectively activating generative recovery in high-uncertainty regions.
To address this issue, we introduce a new restoration paradigm, termed Selective Generative Restoration. The core idea is that diffusion priors should be adaptively activated according to region-level uncertainty: low-uncertainty regions preserve observation fidelity, while high-uncertainty regions exploit generative inference for plausible recovery.
However, realizing Selective Generative Restoration requires addressing three challenges. First, an explicit uncertainty estimation mechanism is required to determine where generative restoration is necessary. Second, stochastic diffusion prediction may introduce inconsistent restoration trajectories across frames, causing temporal flickering. Third, conventional iterative diffusion sampling is computationally expensive and limits practical video restoration applications.
To address these challenges, we propose Selective Generative Video Deraining (SGVD), an uncertainty-aware diffusion framework supporting one-step restoration, built upon the principle of Selective Generative Restoration. SGVD uses stochastic prediction disagreement as a practical signal of restoration uncertainty. Specifically, given the same rainy observation, SGVD performs two one-step diffusion predictions with different noise perturbations and measures their latent discrepancy as an approximation of posterior uncertainty. The resulting uncertainty map explicitly identifies regions with different restoration difficulty: low-uncertainty regions contain reliable observations and are reconstructed with stronger fidelity constraints, whereas high-uncertainty regions contain missing or ambiguous contents and are enhanced through selective generative inference.
Furthermore, to address temporal instability caused by stochastic diffusion prediction, SGVD introduces an anchor-guided trajectory consistency mechanism. The proposed strategy regularizes relative trajectory distances to an anchor obtained by averaging predicted latent representations within a five-frame subclip. This local temporal constraint is applied during training with 49-frame inputs. This design reduces frame-wise generation drift and encourages temporally coherent restoration across neighboring frames. Finally, to achieve efficient diffusion-based video deraining, SGVD adopts a one-step inference strategy that directly predicts clean latent representations from rainy inputs. Different from conventional iterative diffusion restoration, the proposed formulation preserves the uncertainty-aware generative capability while substantially reducing sampling complexity, making diffusion-based video deraining more practical for real-world applications.
Overall, SGVD establishes a new perspective that diffusion restoration should move beyond global generation toward uncertainty-aware selective generation. By adaptively balancing observation fidelity, generative inference, and temporal consistency, SGVD provides an effective solution for restoring rainy videos with heterogeneous degradation uncertainty.
Overall, the main contributions of this work are summarized as follows.
We introduce a new Selective Generative Restoration paradigm for video deraining, revealing that rain degradation is a spatially heterogeneous inverse problem where different regions require different balances between observation fidelity and generative inference.
We propose SGVD, a one-step uncertainty-aware diffusion framework that selectively activates generative restoration according to region-level posterior uncertainty. By exploiting stochastic diversity between dual diffusion predictions, SGVD constructs an explicit uncertainty representation and achieves adaptive restoration between faithful reconstruction and plausible content recovery.
We develop an anchor-guided trajectory consistency mechanism to regularize stochastic diffusion evolution across video frames. Together with the proposed one-step inference, SGVD effectively suppresses temporal generation drift while maintaining high restoration efficiency, enabling practical diffusion-based video deraining.
RELATED WORK
Video Deraining
Video deraining has evolved from prior-based rain–background decomposition to learned spatio-temporal restoration. Early methods exploited rain sparsity and background low-rankness [16, 17], while subsequent CNN and recurrent models learned spatial rain representations and temporal dependencies directly from data [6, 8, 18]. Transformer-based methods further improved long-range interaction and motion-aware aggregation [10, 19], and recent state-space models enhanced long-sequence modeling efficiency [11, 12]. Despite these advances, most methods formulate video deraining as deterministic point estimation. This formulation is effective when sufficient background evidence remains observable, but often produces over-smoothed results in severely occluded regions where multiple plausible restorations may exist. Recent works have therefore explored pretrained diffusion priors for video deraining, including zero-shot restoration [20] and nighttime rain removal [21]. However, existing generative approaches generally apply diffusion priors globally and do not explicitly distinguish reliable observations from ambiguous rain-occluded regions.
Diffusion Models for Low-Level Vision
Diffusion models extend low-level vision from deterministic prediction [22, 23] to conditional distribution modeling, enabling plausible recovery when observations are incomplete. SR3 [24] introduced conditional diffusion for super-resolution, while StableSR [25] leveraged pretrained latent diffusion priors for realistic texture reconstruction. Subsequent studies have applied diffusion models to various image and video restoration tasks [26, 27, 28, 29]. Nevertheless, strong generative priors introduce a fidelity–perception trade-off. FiDeSR [30] regulates this trade-off for one-step super-resolution, and WeatherDiffusion [31] supports weather-guided forward and inverse rendering. These methods mainly control generation globally, whereas rain degradation is spatially heterogeneous. In contrast, SGVD estimates region-level posterior uncertainty from stochastic prediction diversity and uses it to selectively preserve reliable observations or activate generative recovery.
Efficient Generative Models
The iterative sampling process of diffusion models limits their applicability to video restoration. DDIM [32] and DPM-Solver [33] accelerate inference through shortened or improved numerical sampling. Flow Matching [34] and Rectified Flow [35] instead learn continuous transport trajectories and have supported scalable video generators such as Wan [36]. Consistency Models [37] and Latent Consistency Models [38] further compress multi-step trajectories into few-step or one-step prediction, with SinSR [39] and OSEDiff [40] demonstrating efficient one-step image restoration. However, sampling acceleration alone does not ensure temporally stable video restoration, since stochastic predictions of neighboring frames may follow inconsistent generative trajectories. SGVD combines one-step generative inference with Anchor-Guided Trajectory Consistency, which regularizes relative inter-frame evolution during training without additional inference cost.
Selective Generative Restoration Framework
Overview of SGVD
The proposed SGVD is built upon the principle of Selective Generative Restoration, which aims to adaptively balance observation fidelity and generative inference according to restoration uncertainty. Different from conventional restoration frameworks that apply a uniform reconstruction strategy to all regions, SGVD considers rainy video restoration as a spatially heterogeneous inverse problem. Specifically, regions with reliable observations should preserve the original visual evidence, while regions with severe information loss should exploit generative priors to recover plausible contents.
Given a rainy video sequence , SGVD targets the recovery of a clean and temporally coherent video . As illustrated in Figure 2, SGVD consists of three key components: (1) a One-Step Generative Restoration Backbone that provides efficient latent-space generative reconstruction; (2) an Uncertainty-Guided Selective Generation (USG) module that estimates region-level restoration uncertainty and adaptively controls the activation of generative priors; and (3) an Anchor-Guided Trajectory Consistency (ATC) mechanism that regularizes stochastic generation trajectories to maintain temporal coherence.

Figure 2. Overall architecture of the proposed SGVD for video deraining. The rainy video is first encoded into the latent space by a frozen VAE encoder. The Uncertainty-Guided Selective Generation (USG) module estimates a spatial uncertainty map from the discrepancy between dual-noise clean latent predictions, and adaptively combines the generative latent with the fidelity-preserving LQ latent. The resulting conditional latent is injected into the frozen DiT backbone through a lightweight trainable adapter. Meanwhile, Anchor-Guided Trajectory Consistency (ATC) regularizes each frame's distance to a shared five-frame anchor across trajectory states, promoting temporal coherence without evaluating this training loss at inference.
Specifically, the rainy video is first encoded into the latent space using a frozen VAE encoder. The latent representation is then processed by the USG module, where stochastic generative predictions are exploited to estimate restoration uncertainty. Based on the estimated uncertainty, SGVD selectively integrates fidelity-preserving rainy latent features and generative clean latent predictions to construct an adaptive restoration condition.
The resulting condition and uncertainty map are injected into the frozen generative backbone through a trainable adapter. We use the fixed prompt Remove rain and restore clean background during training and standard testing. The frozen text encoder supplies the task instruction, while the rainy video and uncertainty-guided latent condition provide scene-specific structure and appearance. During training, ATC regularizes the relative evolution of neighboring frames along the generative trajectory, suppressing stochastic temporal drift without additional inference cost. Together with uncertainty-guided selective generation, SGVD adaptively balances fidelity preservation and generative recovery for complex rainy videos.
One-Step Generative Restoration Backbone
Selective Generative Restoration requires a generative model that can provide strong prior knowledge while maintaining practical restoration efficiency. However, conventional diffusion models typically initialize generation from pure Gaussian noise and rely on iterative denoising, which may introduce excessive generation freedom and substantial computational overhead for video restoration. SGVD uses the pretrained Wan2.1-T2V-1.3B backbone [36] with an LQ-aware restoration condition. Flow matching and consistency learning provide the training formulation for predicting clean latent endpoints.
Given a GT latent representation , an LQ latent representation and a Gaussian noise latent , we first construct an LQ-aware noisy latent:
where and control the contributions of the LQ observation and stochastic perturbation, respectively. Then flow matching defines a continuous transport trajectory between the data distribution and noise distribution:
The corresponding velocity field is . A conditional velocity network approximates this transport direction, where denotes the frozen pretrained parameters and denotes the trainable adapter parameters.
where denotes the condition extracted from the rainy video and text embedding.
Although flow matching provides an efficient formulation for generative modeling, directly solving the continuous trajectory still requires numerical integration. Therefore, SGVD further adopts consistency learning following Consistency Models [37] to compress the generative trajectory into a direct endpoint prediction.
The consistency function is trained to produce consistent clean latent predictions from different states along the same trajectory.
where . The corresponding consistency objective is formulated as
Here denotes the exponential moving average of the adapter parameters, while the pretrained weights remain fixed. The operator stops gradients through the consistency target.
Diffusion Posterior Uncertainty Estimation
Selective Generative Restoration requires identifying which regions should preserve observed evidence and which regions require generative inference. Therefore, estimating region-level restoration uncertainty is the fundamental step for adaptive generation. Given a rainy video sequence , the frozen VAE encoder first maps the input into the latent space: , where represents the latent observation containing both reliable scene information and rain-induced degradation. Unlike deterministic restoration models that estimate a single solution, generative models characterize a conditional distribution: , where denotes the latent representation of possible clean videos. The uncertainty of this posterior reflects the ambiguity of restoration: regions with sufficient observations exhibit concentrated predictions, whereas severely occluded regions produce diverse plausible solutions.
To approximate this posterior uncertainty, SGVD exploits the stochasticity of the generative trajectory. Specifically, two independent noise perturbations are injected into the same rainy latent representation:
where . The two perturbed states are processed by the frozen generative backbone without the adapter.
Based on the predicted velocity fields, the corresponding clean latent estimations are recovered as:
Although both predictions are generated from the same rainy observation, their divergence reflects the uncertainty of the conditional restoration distribution. Specifically, low-uncertainty regions with reliable visual evidence produce consistent clean predictions, while ambiguous regions with missing background information exhibit larger stochastic variations. Accordingly, SGVD estimates the diffusion posterior uncertainty map as:
Here averages channels, and performs spatial mean filtering without temporal smoothing, followed by clip-wise min–max normalization.
The obtained uncertainty map provides an explicit spatial measurement of restoration ambiguity: larger values indicate regions with higher posterior uncertainty and stronger demand for generative recovery, while smaller values correspond to reliable regions where observation fidelity should be maintained.
Uncertainty-Guided Selective Generation
The estimated posterior uncertainty provides a spatial criterion for selectively activating generative restoration. Instead of applying generative priors uniformly, SGVD dynamically balances observation fidelity and generative inference according to the uncertainty map. First, the two stochastic clean latent predictions are aggregated to obtain a generative restoration proposal: . To alleviate the unnecessary hallucinations in reliable regions, SGVD thus performs uncertainty-guided selective fusion between the generative prediction and the observed rainy latent representation:
where denotes element-wise multiplication. This formulation explicitly realizes the principle of Selective Generative Restoration. For high-uncertainty regions, increases and activates the generative prior to recover ambiguous contents. For low-uncertainty regions, dominates and preserves trustworthy observations.
To adapt the selected latent representation to the frozen generative backbone, a lightweight trainable adapter is introduced: , where denotes the trainable adapter. It uses a convolutional embedding module to map the concatenated and into tokens aligned with the DiT grid, followed by block-specific linear projections. The resulting condition is injected into the generative backbone by scaling the projected features with pooled uncertainty and adding them to each DiT block’s output to produce the restored clean latent representation. The injection projections are initialized to zero to preserve the backbone’s initial behavior. Through uncertainty-guided selective activation, SGVD avoids the two opposite failure modes of previous restoration paradigms: over-smoothing caused by excessive fidelity constraints and hallucination caused by indiscriminate generation.
Therefore, USG transforms diffusion generation from a global restoration process into an uncertainty-aware selective restoration process, enabling adaptive recovery for spatially heterogeneous rainy videos.
Anchor-Guided Trajectory Consistency
Although uncertainty-guided selective generation enables adaptive activation of generative priors, stochastic generation may still introduce temporal trajectory divergence among consecutive frames. Different from deterministic restoration, where each frame is mapped to a fixed output, generative restoration produces a distribution of possible restoration trajectories. Therefore, maintaining temporal consistency requires regularizing the evolution of generative trajectories rather than only constraining the final restored frames.
To address this issue, we introduce an Anchor-Guided Trajectory Consistency (ATC) mechanism. The key idea is to construct a shared subclip-level temporal anchor and enforce the relative trajectory evolution of neighboring frames to remain consistent during one-step generative restoration. The shared anchor provides a common reference that incorporates all frames within a subclip, offering broader temporal context than an immediate-neighbor constraint. Given a five-frame subclip from a 49-frame training input with , the generative backbone predicts clean latent representations at different trajectory states:
where denotes the flow trajectory state and represents the uncertainty-guided selective condition.
To provide a stable temporal reference, we construct a subclip-level anchor representation:
where denotes a latent projection function. We use the parameter-free identity mapping for all frames and trajectory states.
The anchor captures the shared semantic structure of the subclip while preserving frame-specific variations. Based on this anchor, we measure the relative distance between each frame trajectory and the shared subclip-level reference:
For stochastic generative trajectories at two different states and , temporally coherent restoration should maintain consistent relative distances: . Accordingly, the proposed ATC objective is formulated as:
ATC regularizes changes in each frame’s distance to the shared anchor across trajectory states, allowing frame-specific motion and appearance while discouraging stochastic temporal drift. By introducing anchor-guided trajectory regularization, SGVD extends selective generation from spatial adaptation to temporally consistent generative restoration, enabling stable video recovery under stochastic inference.
Optimization Objective
During training, SGVD optimizes the selective generative restoration process while keeping the pretrained generative prior fixed. Only the adapter parameters are trainable. The pretrained DiT backbone, VAE encoder and decoder, and text encoder remain frozen. The overall training objective consists of three complementary terms: (1) a flow matching objective for learning the generative transport trajectory, (2) a consistency objective for compressing the continuous trajectory into one-step restoration, and (3) the proposed anchor-guided trajectory consistency objective for temporally coherent generation.
The flow matching objective learns the transport direction from LQ-aware noisy latents toward the clean latent distribution, while the consistency loss encourages different states along the same trajectory to predict a consistent clean endpoint for efficient one-step restoration. In addition, the proposed regularizes frame-wise latent trajectories relative to the shared anchor of each five-frame subclip, suppressing stochastic temporal drift while preserving natural motion and frame-specific variations.
Combining the above objectives, the final training objective is formulated as:
where , , and are the balancing coefficients for flow matching, one-step consistency, and anchor-guided trajectory consistency, respectively.
Experimental Results
Experimental Settings
Datasets. We evaluate SGVD on NTURain-Syn [6], RainSyn-Light25, RainSynComplex25 [18], and real-world NTURain videos without clean references. The NTURain-Syn training set contains 25 rainy sequences synthesized from eight clean videos with varying rain severity. All methods are retrained on the same NTURain-Syn split and evaluated on common test splits, with direct cross-dataset evaluation on RainSynLight25 and RainSynComplex25 without fine-tuning.
Evaluation Metrics. We report PSNR and SSIM for reconstruction fidelity; LPIPS, NIQE, CLIP-IQA, and MUSIQ for perceptual quality; and optical-flow-based warping error (WE) for temporal consistency, using RAFT [46] for flow estimation. All methods share the metric computation protocol. For real-world videos without clean references, we report NIQE, MUSIQ, CLIP-IQA, and WE together with qualitative results.
Experimental Setup. SGVD is initialized from the official Wan-AI/Wan2.1-T2V-1.3B release [36], with the pretrained DiT, VAE, and UMT5-XXL text encoder frozen. The complete model contains 7,298.47 M parameters, of which only the 71.67 M adapter parameters (0.98%) are trainable. The shared training protocol uses 49-frame inputs, random crops, AdamW with a learning rate of , and weight decay of , four gradient accumulation steps, and 100,000 iterations. ATC constructs each anchor by averaging predicted latent representations within a five-frame subclip. The text prompt Remove rain and restore clean background is fixed throughout training and standard testing.
Comparison with State-of-the-Art Methods
Quantitative Comparisons. We compare SGVD with representative state-of-the-art restoration methods, including both diffusion-based and non-diffusion approaches. The diffusion-based baselines include DiffIR2VR [41], Diff-Plugin [42], DiffBIR [44], ZSVD [20], DOVE [45], WeatherDiffusion (WeatherDiff) [31], and T3-Diff [43]. The non-diffusion baselines include the Transformer-based DeLiVR [10] and the state-space-model-based RainMamba [11].
As shown in Table I, SGVD achieves consistent improvements over the compared diffusion-based methods in reconstruction fidelity across different synthetic rain scenarios. In particular, SGVD obtains superior PSNR and SSIM performance with competitive NIQE scores on challenging rain conditions with severe occlusions, demonstrating its capability to recover missing contents while preserving reliable observations. SGVD also achieves better NIQE, CLIP-IQA, MUSIQ, and WE than DeLiVR and RainMamba on both NTURain-Syn and NTURain-Real. These advantages extend to cross-dataset evaluation, where SGVD leads the two non-diffusion baselines on all seven metrics on RainSynLight25 and five metrics on RainSynComplex25. Different from existing diffusion-based methods that apply generative priors globally, SGVD selectively activates generation according to restoration uncertainty. This uncertainty-aware strategy balances fidelity preservation and perceptual recovery. Reliable regions maintain observation consistency, while ambiguous regions benefit from generative reconstruction. Retaining reliable observations can limit texture regeneration, reflecting the trade-off between pixel-level fidelity and perceptual similarity. SGVD balances these objectives to achieve strong reconstruction fidelity and competitive perceptual quality. Moreover, SGVD achieves highly competitive temporal consistency across the evaluated datasets and obtains the lowest warping error on NTURain-Syn and NTURain-Real. This improvement verifies that the proposed anchor-guided trajectory consistency mechanism effectively suppresses stochastic generation drift and maintains temporally coherent restoration.
| Dataset | Metric | DiffIR2VR [41] | Diff-Plugin [42] | T3-Diff [43] | WeatherDiff [31] | DiffBIR [44] | ZSVD [20] | DOVE [45] | RainMamba [11] | DeLiVR [10] | SGVD |
| NTURain-Syn | PSNR↑ | 23.7304 | 24.1138 | 25.3339 | 25.4756 | 25.9601 | 25.9677 | 26.2488 | 36.3430 | 36.7115 | 26.5174 |
| NTURain-Syn | SSIM↑ | 0.6850 | 0.7161 | 0.8633 | 0.8110 | 0.7802 | 0.8027 | 0.8534 | 0.9705 | 0.9684 | 0.8810 |
| NTURain-Syn | LPIPS↓ | 0.2755 | 0.1137 | 0.1350 | 0.1471 | 0.1221 | 0.1937 | 0.1333 | 0.0257 | 0.0281 | 0.1310 |
| NTURain-Syn | NIQE↓ | 4.2206 | 3.8413 | 3.4850 | 3.4752 | 3.6846 | 3.7548 | 4.8571 | 3.6070 | 3.5866 | 3.4224 |
| NTURain-Syn | CLIP-IQA↑ | 0.3601 | 0.3273 | 0.2877 | 0.2918 | 0.2583 | 0.2452 | 0.2537 | 0.2766 | 0.2666 | 0.3565 |
| NTURain-Syn | MUSIQ↑ | 57.3385 | 59.4466 | 57.1062 | 58.2231 | 53.3840 | 53.1633 | 52.7594 | 57.2592 | 56.4605 | 58.3749 |
| NTURain-Syn | WE↓ | 0.0286 | 0.0351 | 0.0210 | 0.0286 | 0.0249 | 0.0226 | 0.0241 | 0.0214 | 0.0221 | 0.0207 |
| RainSynLight25 | PSNR↑ | 22.7688 | 21.9237 | 24.8369 | 24.3363 | 23.7407 | 21.7876 | 25.5674 | 24.2891 | 24.3912 | 25.9165 |
| RainSynLight25 | SSIM↑ | 0.6314 | 0.6366 | 0.8439 | 0.8387 | 0.7068 | 0.6393 | 0.7734 | 0.8364 | 0.8382 | 0.8492 |
| RainSynLight25 | LPIPS↓ | 0.3657 | 0.1962 | 0.2547 | 0.2430 | 0.2908 | 0.3620 | 0.2664 | 0.3590 | 0.3722 | 0.3013 |
| RainSynLight25 | NIQE↓ | 5.1659 | 3.7525 | 4.5336 | 4.5294 | 4.5301 | 4.4077 | 5.1229 | 4.4971 | 4.2732 | 4.1671 |
| RainSynLight25 | CLIP-IQA↑ | 0.4660 | 0.4837 | 0.4106 | 0.3943 | 0.3814 | 0.3257 | 0.3513 | 0.4003 | 0.4249 | 0.4971 |
| RainSynLight25 | MUSIQ↑ | 58.0105 | 57.5581 | 58.1555 | 57.5304 | 52.6834 | 50.7805 | 54.4247 | 58.2881 | 58.1040 | 58.4274 |
| RainSynLight25 | WE↓ | 0.0503 | 0.0444 | 0.0489 | 0.0527 | 0.0470 | 0.0446 | 0.0255 | 0.0444 | 0.0367 | 0.0347 |
| RainSynComplex25 | PSNR↑ | 15.2853 | 16.7380 | 17.0794 | 16.0397 | 17.0434 | 15.2853 | 18.5361 | 17.1749 | 18.7076 | 18.5545 |
| RainSynComplex25 | SSIM↑ | 0.4172 | 0.4647 | 0.5772 | 0.5267 | 0.5186 | 0.3862 | 0.5810 | 0.5505 | 0.5432 | 0.5812 |
| RainSynComplex25 | LPIPS↓ | 0.5225 | 0.3437 | 0.4135 | 0.4583 | 0.4244 | 0.5434 | 0.4036 | 0.5212 | 0.4991 | 0.4922 |
| RainSynComplex25 | NIQE↓ | 8.3184 | 3.8672 | 5.6912 | 8.0030 | 5.2094 | 6.6324 | 5.6132 | 5.9325 | 5.0987 | 4.2315 |
| RainSynComplex25 | CLIP-IQA↑ | 0.3905 | 0.4257 | 0.3168 | 0.3060 | 0.3211 | 0.2465 | 0.2934 | 0.3029 | 0.3069 | 0.3521 |
| RainSynComplex25 | MUSIQ↑ | 56.6605 | 58.1845 | 56.6112 | 57.8192 | 50.8225 | 43.5738 | 48.5194 | 54.1892 | 53.8462 | 58.8422 |
| RainSynComplex25 | WE↓ | 0.1180 | 0.0783 | 0.1239 | 0.1438 | 0.1117 | 0.1304 | 0.0446 | 0.1221 | 0.1001 | 0.1014 |
| NTURain-Real | NIQE↓ | 4.2686 | 3.7203 | 3.3842 | 3.3784 | 3.9342 | 3.9149 | 4.9289 | 3.3609 | 3.3340 | 3.2927 |
| NTURain-Real | CLIP-IQA↑ | 0.3385 | 0.4077 | 0.3786 | 0.3947 | 0.3401 | 0.2755 | 0.2360 | 0.3618 | 0.3440 | 0.3930 |
| NTURain-Real | MUSIQ↑ | 60.2405 | 62.0713 | 62.5504 | 64.3716 | 57.2874 | 54.7643 | 53.7124 | 62.2050 | 62.3364 | 62.6503 |
| NTURain-Real | WE↓ | 0.0276 | 0.0395 | 0.0335 | 0.0327 | 0.0326 | 0.0258 | 0.0205 | 0.0325 | 0.0315 | 0.0204 |
Table I. Quantitative comparisons with representative video restoration and task-specific video deraining methods on synthetic and real-world video deraining datasets. The best and second-best results are highlighted in bold and underline, respectively.
Qualitative Comparisons. Figure 3 presents visual comparisons under different rain degradation levels. Existing restoration methods generally remove rain artifacts but tend to generate over-smoothed results in severely corrupted regions due to insufficient prior information. Meanwhile, global generative restoration methods can recover plausible structures but may introduce unrealistic details into reliable areas. In contrast, SGVD performs uncertainty-aware selective generation: low-uncertainty regions preserve faithful observations, whereas high-uncertainty regions activate generative recovery. As a result, SGVD produces cleaner backgrounds, sharper structures, and more realistic details under complex rain degradations.

Figure 3. Visual comparisons on synthetic video deraining datasets, including NTURain [6] and RainSynLight25 [18].
Real-World Evaluation. To further evaluate practical robustness, we conduct experiments on real-world rainy videos without clean references. As shown in Figure 4, SGVD generalizes well to diverse real-world rain conditions, including complex illumination changes and non-uniform rain patterns. Compared with existing methods, SGVD achieves better visual quality while avoiding excessive hallucination and preserving scene structures. These results demonstrate that uncertainty-aware selective generation provides a robust restoration strategy beyond synthetic benchmarks.

Figure 4. Visual comparisons on real-world rainy videos from the NTURain-Real dataset [6].
Downstream Task Evaluation
To further evaluate the practical utility of SGVD, we conduct downstream object detection and tracking experiments on the RVDT [12] benchmark. The restored videos are directly fed into the downstream models without additional fine-tuning. For object detection, we employ YOLO-v3 [47] and
| Method | Object Detection | Object Detection | Object Tracking | Object Tracking | Object Tracking |
| IDF1 | MOTA | MOTP | |||
| Rainy Input | 39.73 | 44.69 | 64.5 | 28.0 | 19.9 |
| ZSVD | 38.86 | 43.32 | 64.8 | 26.4 | 20.3 |
| DOVE | 41.35 | 45.87 | 65.7 | 30.1 | 20.6 |
| WeatherDiff | 37.82 | 42.12 | 63.5 | 29.5 | 18.8 |
| DiffBIR | 38.06 | 44.56 | 65.3 | 27.9 | 20.4 |
| DiffIR2VR | 41.08 | 43.35 | 65.9 | 28.1 | 21.0 |
| SGVD | 42.02 | 44.86 | 66.4 | 30.2 | 20.8 |
Table 2. Object detection and tracking results on the RVDT benchmark. and denote the detection results obtained by YOLO-v3 and MEGA, respectively. The tracking results are obtained using GTR.
MEGA [48], while the tracking performance is evaluated using GTR [49]. As shown in Table II, SGVD achieves improved detection and tracking performance over rainy inputs. Specifically, SGVD obtains the highest of 42.02%. Compared with the rainy input, SGVD improves and by 2.29 and 0.17 percentage points, respectively, demonstrating that the proposed selective generation strategy effectively restores discriminative object details. Furthermore, SGVD achieves the highest IDF1 of 66.4 and MOTA of 30.2, outperforming existing diffusion-based methods. This improvement verifies that the proposed ATC effectively reduces temporal generation drift and enhances cross-frame object association, leading to more reliable downstream video analysis.
| Metric | NTURain-Syn | RainSynLight25 | RainSynComplex25 |
| Spearman Correlation () | 0.912 | 0.910 | 0.933 |
Table 3. Spearman correlation between the estimated uncertainty map and restoration error on synthetic video deraining datasets. Higher values indicate that regions with larger estimated uncertainty tend to exhibit larger restoration errors.
Analysis of Diffusion Posterior Uncertainty
We further analyze whether the estimated uncertainty map accurately reflects restoration ambiguity. Figure 5 presents uncertainty maps generated by the proposed diffusion posterior uncertainty estimation strategy. As observed, high uncertainty responses are mainly concentrated in regions with severe rain streaks, dense occlusions, and missing background structures. In contrast, reliable background regions exhibit relatively low uncertainty. This observation indicates that stochastic prediction diversity effectively captures spatial variations in restoration difficulty.

Figure 5. Visualization of diffusion posterior uncertainty in SGVD. The uncertainty map is estimated from the discrepancy between two stochastic one-step diffusion predictions. Severely degraded regions exhibit stronger uncertainty responses, while reliable observations remain relatively low-uncertainty, indicating that the estimated uncertainty effectively reflects spatial variations in restoration ambiguity.
To quantitatively verify whether the estimated uncertainty corresponds to actual restoration ambiguity, we analyze the spatial correlation between uncertainty maps and reconstruction errors. Specifically, given the uncertainty map and the pixel-level restoration error , we employ Spearman’s rank correlation coefficient to measure their monotonic spatial correspondence. Here and denote the restored and ground-truth pixel values of frame , respectively. A larger indicates that regions with higher estimated uncertainty tend to exhibit larger restoration errors. As shown in Table III, consistently high correlations are observed across different synthetic datasets, showing that stochastic prediction disagreement tracks spatial variations in reconstruction error.
Temporal Trajectory Consistency Analysis
We evaluate whether the proposed Anchor-Guided Trajectory Consistency (ATC) effectively stabilizes the temporal evolution of one-step generative restoration.
Quantitative Temporal Stability. We first isolate the contribution of ATC using the component ablation results in
Table 4. The one-step restoration baseline without USG or ATC obtains a temporal warping error of 0.0287. Introducing USG alone substantially improves spatial restoration metrics, but only slightly changes the warping error from 0.0287 to 0.0309. This result indicates that uncertainty-guided spatial selection alone cannot fully resolve stochastic temporal drift.
| Model | USG | ATC | PSNR↑ | SSIM↑ | LPIPS↓ | NIQE↓ | CLIP-IQA↑ | MUSIQ↑ | WE↓ |
| D1 | 23.0013 | 0.7819 | 0.1899 | 3.5131 | 0.3221 | 57.8211 | 0.0287 | ||
| D2 | ✓ | 26.2416 | 0.8732 | 0.1362 | 3.5114 | 0.3487 | 57.9213 | 0.0309 | |
| D3 | ✓ | 25.9347 | 0.8638 | 0.1449 | 3.7052 | 0.3316 | 57.2845 | 0.0225 | |
| SGVD | ✓ | ✓ | 26.5174 | 0.8810 | 0.1310 | 3.4224 | 0.3565 | 58.3749 | 0.0207 |
Table 4. Ablation study of the core components of SGVD on the NTURain-Syn test set. USG mainly improves fidelity and perceptual recovery, whereas ATC primarily improves temporal stability.
In contrast, introducing ATC alone reduces the warping error from 0.0287 to 0.0225, while also improving PSNR and SSIM. When USG and ATC are jointly incorporated, the complete SGVD further reduces the warping error to 0.0207. These results show that ATC improves temporal stability alongside reconstruction fidelity. They also reveal the complementary relation: USG controls the spatial activation of generative priors, whereas ATC regularizes their temporal evolution.
Temporal-Slice Visualization. To qualitatively examine cross-frame stability, we construct temporal slices by extracting the same spatial line from consecutive frames and stacking them along the temporal dimension. As shown in Figure 6, the result without ATC exhibits irregular fluctuations and discontinuous structural traces, particularly around object boundaries and texture-rich regions. These artifacts indicate that independently restored frames follow inconsistent stochastic trajectories.

Figure 6. Temporal slice comparison of rainy input, SGVD without ATC, full SGVD, and ground truth (top to bottom). ATC improves structural consistency across frames and reduces temporal artifacts, yielding a lower warping error (WE).
With ATC, the temporal slices become substantially smoother and more continuous, while the underlying motion-induced variations remain visible. This observation confirms that ATC does not force neighboring frames to become identical. Instead, it regularizes relative distances to the shared anchor within each five-frame subclip, allowing motion-dependent variations while discouraging incoherent temporal evolution. Since ATC is used only during training, this temporal improvement introduces no additional inference overhead.
Temporal Regularization Comparison. Table V compares ATC with a neighboring-frame relative-trajectory loss while holding the backbone, USG, training budget, and loss-weight selection protocol fixed. The five-frame shared anchor reduces WE from 0.0323 to 0.0207 (35.9%) and improves PSNR from 26.3309 to 26.5174 dB and SSIM from 0.8763 to 0.8810. NIQE and MUSIQ also improve to 3.4224 and 58.3749. The neighboring-frame objective obtains lower LPIPS and higher CLIP-IQA, while ATC offers stronger temporal consistency and fidelity with competitive perceptual quality. Relating each frame to the full five-frame subclip provides a common temporal reference beyond an immediate-neighbor pair.
| Metric | Neighboring-frame | Shared-anchor ATC |
| PSNR | 26.3309 | 26.5174 |
| SSIM | 0.8763 | 0.8810 |
| LPIPS | 0.1257 | 0.1310 |
| NIQE | 3.5280 | 3.4224 |
| CLIP-IQA | 0.3671 | 0.3565 |
| MUSIQ | 57.9113 | 58.3749 |
| WE | 0.0323 | 0.0207 |
Table V. Temporal regularization comparison on NTURain-Syn. Best results are bold.
Computational Efficiency
We measure end-to-end inference on a single 48GB NVIDIA RTX 4090 GPU using 49-frame inputs at 640 × 480 resolution (width × height), batch size 1, and BF16 precision. After one warm-up run, latency is averaged over three timed runs of the same clip with CUDA synchronization. Model loading and disk I/O are excluded. Latency and FLOPs are normalized by the 49 output frames, and FPS is computed from the unrounded clip latency. FLOPs include matrix multiplications, convolutions, and attention, counting two operations per multiply-accumulate. The 30-step cost is calculated from the profiled fixed pipeline cost plus 30 repetitions of the per-step computation.
As shown in Table VI, one-step SGVD requires 296 ms/frame and 21,548 G/frame, compared with 2,078 ms/frame and 226,611 G/frame for 30-step inference. This corresponds to approximately 7.0× faster inference, an 86% reduction in latency, and a 90.5% reduction in FLOPs. By replacing iterative denoising with direct latent restoration, SGVD substantially reduces repeated backbone evaluations and improves video-processing throughput. The end-to-end speedup is smaller than the sampling-step ratio because operations such as VAE encoding and decoding incur costs independent of the number of restoration steps. The two uncertainty probes increase latency from 133 to 296 ms/frame and FLOPs from 11,372 to 21,548 G/frame, reducing throughput from approximately 7.52 to 3.37 FPS. These additional backbone evaluations provide spatial guidance for selective restoration, while reusing the frozen model without introducing trainable parameters. In addition, only 71.67 M of SGVD’s 7,298.47 M parameters are trainable (0.98%). Restricting optimization to the adapter reduces the number of parameters requiring stored gradients and optimizer states by approximately 99.02%, enabling task-specific adaptation of the large pretrained model with a small parameter-update burden.
| Method | Steps | Time (ms/frame) | FPS | FLOPs (G/frame) | Total (M) | Trainable (M) |
| SGVD | 1 | 296 | 3.37 | 21548 | 7298.47 | 71.67 |
| SGVD | 30 | 2078 | 0.48 | 226611 | 7298.47 | 71.67 |
| SGVD w/o probes | 1 | 133 | 7.52 | 11372 | 7298.47 | 71.67 |
Table VI. End-to-end SGVD efficiency with and without uncertainty probing on 49-frame inputs at 640 × 480 resolution.
Ablation Studies
After validating the Selective Generative Restoration paradigm and the estimated diffusion posterior uncertainty, we
further conduct controlled experiments to isolate the contributions of individual components and analyze the sensitivity of the proposed framework.
Effect of Core Components. Table IV evaluates the individual and joint effects of Uncertainty-Guided Selective Generation (USG) and Anchor-Guided Trajectory Consistency (ATC). The baseline D1 denotes the one-step generative restoration backbone without either component. Adding USG to the baseline improves PSNR from 23.0013 to 26.2416 and reduces LPIPS from 0.1899 to 0.1362. It also improves the no-reference perceptual metrics, confirming that uncertainty-guided activation achieves a better balance between fidelity preservation and generative recovery. However, its temporal warping error remains close to that of the baseline, indicating that spatial uncertainty modeling alone does not explicitly constrain stochastic temporal evolution. Adding ATC alone improves PSNR and SSIM and reduces the warping error from 0.0287 to 0.0225. These gains support the contribution of ATC to both spatial reconstruction and temporal stability. The complete SGVD achieves the best overall performance, reaching a PSNR of 26.5174, an LPIPS of 0.1310, and a warping error of 0.0207. Compared with the baseline, these correspond to a PSNR improvement of 3.5161 dB, an LPIPS reduction of 0.0589, and a temporal warping error reduction of 27.9%. The consistent improvements across fidelity, perception, and temporal stability demonstrate that USG and ATC are complementary. USG determines where generation should be activated, while ATC determines how the activated generation should evolve consistently across frames.
Effect of LQ Latent Utilization. We investigate the influence of different strategies for incorporating low-quality (LQ) latent information into the generative restoration process. Specifically, we compare three variants: using the LQ latent only as the diffusion trajectory starting point, using it only as conditional guidance, and our full design that exploits the LQ latent for both trajectory initialization and conditioning. As shown in Table VII, using the LQ latent only as the starting point retains more information from the degraded observation, but provides insufficient guidance for recovering heavily corrupted contents, leading to limited fidelity and perceptual quality. Using the LQ latent only as conditional guidance cannot adequately constrain the generative trajectory and results in substantial structural deviation from the target content. By jointly using the LQ latent as both the trajectory starting state and the conditional signal, SGVD achieves substantially better reconstruction fidelity and perceptual quality, while also improving temporal consistency. In particular, the full design achieves the best PSNR, SSIM, LPIPS, CLIP-IQA, MUSIQ, and WE, while maintaining competitive NIQE, demonstrating that the two roles of the LQ latent are complementary for balancing observation preservation and generative recovery.
| Variant | PSNR | SSIM | LPIPS | NIQE | CLIP-IQA | MUSIQ | WE |
| Start Only | 19.8903 | 0.5467 | 0.2796 | 3.3830 | 0.3528 | 57.8211 | 0.0256 |
| Condition Only | 17.0639 | 0.6489 | 0.3435 | 3.7909 | 0.1627 | 42.4441 | 0.0221 |
| Both (SGVD) | 26.5174 | 0.8810 | 0.1310 | 3.4224 | 0.3565 | 58.3749 | 0.0207 |
Table VII. Effect of LQ latent utilization strategies on the NTURain [6] synthetic dataset.
Number of Stochastic Posterior Probes. We next investigate the number of stochastic predictions used to estimate diffusion posterior uncertainty. As shown in Table VIII, removing uncertainty guidance degrades restoration performance, while a single prediction cannot explicitly characterize cross-sample posterior dispersion. Using two independent predictions introduces an effective disagreement signal and achieves the best overall performance in PSNR, LPIPS, NIQE, and MUSIQ, indicating that pairwise stochastic diversity is sufficient for region-level uncertainty estimation. Increasing the number of predictions from two to four only slightly improves WE from 0.0207 to 0.0205, while slightly degrading the other metrics and doubling the uncertainty-probing cost. Therefore, we adopt two stochastic predictions as the default configuration for the best accuracy–quality–efficiency trade-off.
| Number of Probes | PSNR | LPIPS | NIQE | MUSIQ | WE |
| w/o Uncertainty | 25.6821 | 0.1478 | 3.8126 | 56.9821 | 0.0318 |
| Single Probe | 26.0712 | 0.1404 | 3.6468 | 57.5316 | 0.0284 |
| Two Probes | 26.5174 | 0.1310 | 3.4224 | 58.3749 | 0.0207 |
| Four Probes | 26.4938 | 0.1332 | 3.4517 | 58.1263 | 0.0205 |
Table VIII. Effect of the number of stochastic posterior probes used for uncertainty estimation. The two-probe configuration achieves the best overall accuracy–quality–efficiency trade-off.
Sensitivity to Loss Weights. We further analyze the sensitivity of SGVD to the weighting coefficients of the flow matching, one-step consistency, and anchor-guided trajectory consistency objectives. The corresponding results are presented in Table IX. When the ATC loss weights are insufficient, the model cannot adequately balance generative trajectory learning, one-step endpoint consistency, and temporal trajectory regularization. The default configuration Group2 achieves the best overall performance across PSNR, SSIM, LPIPS, MUSIQ, and temporal warping error, indicating a suitable balance among the three complementary objectives. Changing the weight combination in Group3 and Group4 does not yield further improvements, indicating sensitivity to the balance among the training objectives.
| Variant | PSNR | SSIM | LPIPS | MUSIQ | WE | |||
| Group1 | 1.0 | 1.0 | 0.05 | 26.2847 | 0.8746 | 0.1328 | 57.9482 | 0.0231 |
| Group2 | 1.0 | 1.0 | 0.10 | 26.5174 | 0.8810 | 0.1310 | 58.3749 | 0.0207 |
| Group3 | 1.0 | 1.5 | 0.15 | 26.4263 | 0.8789 | 0.1358 | 58.1637 | 0.0220 |
| Group4 | 1.0 | 0.5 | 0.50 | 26.3518 | 0.8767 | 0.1341 | 58.0126 | 0.0214 |
Table IX. Sensitivity analysis of the weighting coefficients for flow matching, one-step consistency, and anchor-guided trajectory consistency.
Text Conditioning. We compare the fixed training prompt Remove rain and restore clean background, the shorter prompt Remove rain, and an empty prompt at inference. The checkpoint, input clips, random seeds, sampling settings, guidance scale, and negative prompt are identical across the three settings. In Table X, the original prompt leads on all seven metrics. Relative to the empty prompt, it improves PSNR by 0.1494 dB and SSIM by 0.0067, reduces LPIPS from 0.1368 to 0.1310, and lowers WE by approximately 1.9%. The short and empty prompts give similar reconstruction quality, suggesting that visual conditioning supplies strong scene guidance, while the full task instruction provides a further improvement.
| Prompt | PSNR | SSIM | LPIPS | NIQE | CLIP-IQA | MUSIQ | WE |
| Original | 26.5174 | 0.8810 | 0.1310 | 3.4224 | 0.3565 | 58.3749 | 0.0207 |
| Short | 26.3683 | 0.8740 | 0.1359 | 3.4231 | 0.3433 | 57.9668 | 0.0211 |
| Empty | 26.3680 | 0.8743 | 0.1368 | 3.4251 | 0.3467 | 58.2029 | 0.0211 |
Table X. Text-conditioning comparison on NTURain-Syn. Best results are bold.
Input-Length Generalization. We evaluate the same model trained on 49-frame inputs using 25-, 49-, and 81-frame inputs, without retraining or fine-tuning. Source sequences and scored
frames are matched, and the resolution, prompt, and sampling settings are held fixed. In Table XI, the 81-frame configuration improves PSNR by 0.1365 dB and SSIM by 0.0087 over 49 frames and leads on five of the seven metrics. Its LPIPS of 0.1316 remains close to 0.1310 at 49 frames. The 25-frame setting retains comparable quality, with PSNR lower by 0.2217 dB. WE is 0.0214, 0.0207, and 0.0209 at 25, 49, and 81 frames, respectively, indicating stable temporal behavior on longer inputs. These results support length generalization over the evaluated range. The five-frame extent of the training regularizer does not fix the inference input length.
| Metric | 25 frames | 49 frames | 81 frames |
| PSNR | 26.2957 | 26.5174 | 26.6539 |
| SSIM | 0.8784 | 0.8810 | 0.8897 |
| LPIPS | 0.1339 | 0.1310 | 0.1316 |
| NIQE | 3.4480 | 3.4224 | 3.3489 |
| CLIP-IQA | 0.3497 | 0.3565 | 0.3609 |
| MUSIQ | 58.1333 | 58.3749 | 58.6926 |
| WE | 0.0214 | 0.0207 | 0.0209 |
Table XI. Input-length evaluation using a model trained on 49-frame clips. Best results are bold.
Limitations
SGVD largely removes rain streaks in the challenging examples shown in Figure 7, while remaining reconstruction errors are concentrated around complex textures, motion boundaries, and regions with varying illumination. Generative priors can recover plausible local details that differ from the clean reference. In addition, the two stochastic predictions used for uncertainty estimation add computational cost. Improving detail fidelity and reducing this computational cost are directions for future work.
Figure 7. Restoration examples in challenging scenes. From left to right: rain streak map, uncertainty overlay, and reconstruction error.
Conclusion
In this paper, we revisit video deraining from the perspective of uncertainty-aware generative restoration and reveal that rain degradation is a spatially heterogeneous inverse problem. Based on this insight, we propose Selective Generative Video Deraining (SGVD), a new restoration framework that adaptively balances observation fidelity and generative inference. SGVD exploits stochastic diversity of one-step generative predictions to estimate diffusion posterior uncertainty, enabling selective activation of generative priors in ambiguous regions while preserving reliable observations. Furthermore, an Anchor-Guided Trajectory Consistency mechanism is introduced to regularize stochastic restoration trajectories and improve temporal coherence without additional inference cost. Extensive experiments demonstrate that SGVD achieves strong reconstruction fidelity, competitive perceptual quality, and stable temporal behavior compared with existing video deraining methods. This work provides a new perspective for diffusion-based restoration by shifting from global generation toward uncertainty-aware selective generation, which can be further extended to other restoration tasks with spatially varying degradation.
References
References
- [1]J. Li, R. Xu, X. Liu, J. Ma, B. Li, Q. Zou, J. Ma, and H. Yu, “Domain adaptation based object detection for autonomous driving in foggy and rainy weather,” IEEE Transactions on Intelligent Vehicles, vol. 10, no. 2, pp. 900–911, 2025.arxiv.org/abs/2307.09676
- [2]Y. Pan, R. Sun, W. Li, and T. Zhang, “Exploring weather-aware aggregation and adaptation for semantic segmentation under adverse conditions,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 13 952–13 962.DOI
- [3]J. Liu, “RainyTrack: Enhancing object tracking in adverse weather conditions with siamese networks,” in Proceedings of the 7th International Conference on Image and Graphics Processing. ACM, 2024, pp. 308–315.DOI
- [4]X. Zhong, S. Tu, X. Ma, K. Jiang, W. Huang, and Z. Wang, “Rainy WCity: A real rainfall dataset with diverse conditions for semantic driving scene understanding,” in International Joint Conference on Artificial Intelligence, 2022, pp. 1743–1749.DOI
- [5]M. Li, Q. Xie, Q. Zhao, W. Wei, S. Gu, J. Tao, and D. Meng, “Video rain streak removal by multiscale convolutional sparse coding,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6644–6653.DOI
- [6]J. Chen, C.-H. Tan, J. Hou, L.-P. Chau, and H. Li, “Robust video content alignment and compensation for rain removal in a CNN framework,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6286–6295.arxiv.org/abs/1803.10433
- [7]W. Yang, J. Liu, and J. Feng, “Frame-consistent recurrent video deraining with dual-level flow,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1661–1670.DOI
- [8]W. Yang, R. T. Tan, J. Feng, S. Wang, B. Cheng, and J. Liu, “Recurrent multi-frame deraining: Combining physics guidance and adversarial learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 8569–8586, 2022.DOI
- [9]H. Wu, Y. Yang, H. Chen, J. Ren, and L. Zhu, “Mask-guided progressive network for joint raindrop and rain streak removal in videos,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 7216–7225.DOI
- [10]S. Sun, J. Lu, X. Chen, J. Wang, D. Lu, G. Zhang, G. Gao, and Z. Zheng, “DeLiVR: Differential spatiotemporal Lie bias for efficient video deraining,” in International Conference on Learning Representations, 2026.arxiv.org/abs/2509.21719
- [11]H. Wu, Y. Yang, H. Xu, W. Wang, J. Zhou, and L. Zhu, “RainMamba: Enhanced locality learning with state space models for video deraining,” in Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 7881–7890.DOI
- [12]S. Sun, W. Ren, J. Zhou, S. Wang, J. Gan, and X. Cao, “Semi-supervised state-space model with dynamic stacking filter for real-world video deraining,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 26 114–26 124.arxiv.org/abs/2505.16811
- [13]X. Liao, T. Zheng, J. Zhong, P. Zhang, and C. Ren, “Asymmetric mask scheme for self-supervised real image denoising,” in European Conference on Computer Vision, 2024, pp. 199–215.arxiv.org/abs/2407.06514
- [14]W. Liu, P. Zhang, H. Qin, X. Huang, J. Yang, and M. Ng, “Fast global image smoothing via quasi weighted least squares,” International Journal of Computer Vision, vol. 132, no. 12, pp. 6039–6068, 2024.DOI
- [15]W. Liu, P. Zhang, Y. Lei, X. Huang, J. Yang, and M. Ng, “A generalized framework for edge-preserving and structure-preserving image smoothing,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6631–6648, 2022.
- [16]J.-H. Kim, J.-Y. Sim, and C.-S. Kim, “Video deraining and desnowing using temporal correlation and low-rank matrix completion,” IEEE Transactions on Image Processing, vol. 24, no. 9, pp. 2658–2670, 2015.DOI
- [17]W. Ren, J. Tian, Z. Han, A. Chan, and Y. Tang, “Video desnowing and deraining based on matrix decomposition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4210–4219.DOI
- [18]J. Liu, W. Yang, S. Yang, and Z. Guo, “D3R-Net: Dynamic routing residue recurrent network for video rain removal,” IEEE Transactions on Image Processing, vol. 28, no. 2, pp. 699–712, 2019.DOI
- [19]Q. Wen, Y. Wu, and Q. Chen, “Video waterdrop removal via spatio-temporal fusion in driving scenes,” in 2023 IEEE International Conference on Robotics and Automation, 2023, pp. 10 003–10 009.arxiv.org/abs/2302.05916
- [20]T. Varanka, J. L. Gonzalez, H. Kim, P. Garrido, and X. Yao, “Zero-shot video deraining with video diffusion models,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2026, pp. 677–687.arxiv.org/abs/2511.18537
- [21]P. Yang, H. Ci, B. Lin, Y. Song, and M. Z. Shou, “UENR-600K: A large-scale physically grounded dataset for nighttime video deraining,” arXiv preprint arXiv:2604.04402, 2026.arxiv.org/abs/2604.04402
- [22]Y. Zhu, L. Lv, P. Zhang, X. Liu, T. Tang, F. Tian, W. Sun, and H. Lu, “Interactive spatial-frequency fusion Mamba for multi-modal image fusion,” IEEE Transactions on Image Processing, vol. 35, pp. 2380–2392, 2026.
- [23]H. Sun, L. Lv, P. Zhang, T. Tang, F. Tian, W. Sun, and H. Lu, “Spatial-frequency enhanced Mamba for multi-modal image fusion,” IEEE Transactions on Image Processing, vol. 34, pp. 7684–7696, 2025.arxiv.org/abs/2511.06593
- [24]C. Saharia, J. Ho, W. Chan, T. Salimans, D. J. Fleet, and M. Norouzi, “Image super-resolution via iterative refinement,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4713–4726, 2023.DOI
- [25]J. Wang, Z. Yue, S. Zhou, K. C. K. Chan, and C. C. Loy, “Exploiting diffusion prior for real-world image super-resolution,” International Journal of Computer Vision, vol. 132, no. 12, pp. 5929–5949, 2024.arxiv.org/abs/2305.07015
- [26]J. Cai, Y. Lei, K. Wang, T. Zhong, F. Zhou, and J. Song, “Condition-enhanced guided diffusion for image super-resolution,” IEEE Transactions on Multimedia, vol. 28, pp. 1079–13094, 2026.DOI
- [27]Y. Liu, J. Pan, Y. Li, Q. Dong, C. Zhu, Y. Guo, and F. Wang, “UltraVSR: Achieving ultra-realistic video super-resolution with efficient one-step diffusion space,” in Proceedings of the 33rd ACM International Conference on Multimedia, 2025, pp. 7785–7794.arxiv.org/abs/2505.19958
- [28]J. Gu, P.-T. Jiang, H. Zhang, M. Zhou, J. Chen, W. Yang, and B. Li, “Improving consistency in diffusion models for image super-resolution,” IEEE Transactions on Multimedia, vol. 28, pp. 12864–12875, 2026.arxiv.org/abs/2410.13807
- [29]J. Han, G. Sim, G. Kim, H.-S. Lee, K. Choi, Y. Han, and S. Cho, “DC-VSR: Spatially and temporally consistent video super-resolution with video diffusion prior,” in Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference, 2025, pp. 1–11.arxiv.org/abs/2502.03502
- [30]A. Kim, M. Jang, C. Moon, Y. Shin, J. Jeong, and S.-h. Park, “FiDeSR: High-fidelity and detail-preserving one-step diffusion super-resolution,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026, pp. 38 270–38 280.arxiv.org/abs/2603.02692
- [31]Y. Zhu, Z. Zhu, M. Hasan, J. Yang, J. Xie, and B. Wang, “WeatherDiffusion: Weather-guided diffusion model for forward and inverse rendering,” arXiv preprint arXiv:2508.06982, 2025.
- [32]J. Song, C. Meng, and P. Ermon, “Denoising diffusion implicit models,” in International Conference on Learning Representations, 2021.arxiv.org/abs/2010.02502
- [33]C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 5775–5787.arxiv.org/abs/2206.00927
- [34]Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” in International Conference on Learning Representations, 2023.arxiv.org/abs/2210.02747
- [35]X. Liu, C. Gong, and Q. Liu, “Flow straight and fast: Learning to generate and transfer data with rectified flow,” in International Conference on Learning Representations, 2023.arxiv.org/abs/2209.03003
- [36]Team Wan, A. Wang, B. Ai, B. Wen, C. Mao, C.-W. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, J. Zeng, J. Wang, J. Zhang, J. Zhou, J. Wang, J. Chen, K. Zhu, K. Zhao, K. Yan, L. Huang, M. Feng, N. Zhang, P. Li, P. Wu, R. Chu, R. Feng, S. Zhang, S. Sun, T. Fang, T. Wang, T. Gui, T. Weng, T. Shen, W. Lin, W. Wang, W. Wang, W. Zhou, W. Wang, W. Shen, W. Yu, X. Shi, X. Huang, X. Xu, Y. Kou, Y. Lv, Y. Li, Y. Liu, Y. Wang, Y. Zhang, Y. Huang, Y. Li, Y. Wu, Y. Liu, Y. Pan, Y. Zheng, Y. Hong, Y. Shi, Y. Feng, Z. Jiang, Z. Han, Z.-F. Wu, and Z. Liu, “Wan: Open and advanced large-scale video generative models,” arXiv preprint arXiv:2503.20314, 2025.arxiv.org/abs/2503.20314
- [37]Y. Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency models,” in International Conference on Machine Learning, 2023, pp. 32211–32252.arxiv.org/abs/2303.01469
- [38]S. Luo, Y. Tan, L. Huang, J. Li, and H. Zhao, “Latent consistency models: Synthesizing high-resolution images with few-step inference,” in International Conference on Learning Representations, 2024.arxiv.org/abs/2310.04378
- [39]Y. Wang, W. Yang, X. Chen, Y. Wang, L. Guo, L.-P. Chau, Z. Liu, Y. Qiao, A. C. Kot, and B. Wen, “SinSR: Diffusion-based image super-resolution in a single step,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 25 796–25 805.arxiv.org/abs/2311.14760
- [40]R. Wu, L. Sun, Z. Ma, and L. Zhang, “One-step effective diffusion network for real-world image super-resolution,” in Advances in Neural Information Processing Systems, vol. 37, 2024, pp. 92 529–92 553.
- [41]C.-H. Yeh, H.-S. Shiu, C.-Y. Lin, Z. Wang, C.-W. Hsiao, T.-H. Chen, and Y.-L. Liu, “DiffIR2VR-Zero: Zero-shot video restoration with diffusion-based image restoration models,” arXiv preprint arXiv:2407.01519, 2024.arxiv.org/abs/2407.01519
- [42]Y. Liu, Z. Ke, F. Liu, N. Zhao, and R. W. H. Lau, “Diff-Plugin: Revitalizing details for diffusion-based low-level tasks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4197–4208.arxiv.org/abs/2403.00644
- [43]S. Chen, T. Ye, K. Zhang, Z. Xing, Y. Lin, and L. Zhu, “Teaching tailored to talent: Adverse weather restoration via prompt pool and depth-anything constraint,” in European Conference on Computer Vision, 2024, pp. 95–115.arxiv.org/abs/2409.15739
- [44]X. Lin, J. He, Z. Chen, Z. Lyu, B. Dai, F. Yu, Y. Qiao, W. Ouyang, and C. Dong, “DiffBIR: Toward blind image restoration with generative diffusion prior,” in European Conference on Computer Vision, 2024, pp. 430–448.DOI
- [45]Z. Chen, Z. Zou, K. Zhang, X. Su, X. Yuan, Y. Guo, and Y. Zhang, “DOVE: Efficient one-step diffusion model for real-world video super-resolution,” in Advances in Neural Information Processing Systems, vol. 38, 2025, pp. 94 905–94 924.arxiv.org/abs/2505.16239
- [46]Z. Teed and J. Deng, “RAFT: Recurrent all-pairs field transforms for optical flow,” in European Conference on Computer Vision, 2020, pp. 402–419.arxiv.org/abs/2003.12039
- [47]J. Redmon and A. Farhadi, “YOLOv3: An incremental improvement,” arXiv preprint arXiv:1804.02767, 2018.arxiv.org/abs/1804.02767
- [48]Y. Chen, Y. Cao, H. Hu, and L. Wang, “Memory enhanced global-local aggregation for video object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 10 337–10 346.arxiv.org/abs/2003.12063
- [49]X. Zhou, V. Koltun, and P. Kr¨ahenb¨uhl, “Global tracking transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 8771–8780.arxiv.org/abs/2203.13250
