betaFeedback

WorksComputer Science

RF3DGS: Reflection-Free 3D Gaussian Splatting with Self-Priors

Abstract

3D reflection removal is crucial in applications demanding high-fidelity visualization, such as museum exhibitions. In this work, we propose Reflection-Free 3D Gaussian Splatting (RF3DGS), a novel method for superior reflection removal. RF3DGS exclusively leverages a widely-adopted photometric prior for reflection removal: that the minimum pixel intensity across multi-view images corresponding to the same 3D point strongly correlates with reflection-free information. However, generalizing this principle to a 3D Gaussian Splatting within a multi-view context is non-trivial. To address this, RF3DGS first optimizes a vanilla 3DGS model using a geometric prior to estimate a coarse, reflection-free depth map for each view. Leveraging these depth maps and photometric prior, a 3D warping process aggregates reflection-free information from auxiliary views to construct a clean transmission pseudo-label for a given reference view. This pseudo-label provides a robust self-supervised signal for optimization. We further introduce an Adaptive Twin Rendering (ATR) loss to synergistically utilize this pseudo-label while preserving high-frequency details from the original, reflection-degraded images. By adeptly fusing these internal 3D self-priors with the ATR loss, RF3DGS demonstrates state-of-the-art reflection removal performance across both synthetic and real-world scenarios. Furthermore, we establish the first 3D reflection removal benchmark, based on the Unity rendering engine, for rigorous quantitative evaluation.

Introduction

Capturing photos with undesirable reflections is common in daily scenarios, such as exhibitions shown in Figure 1. Reflection removal seeks to restore clean transmission for better visual quality and improved downstream computer vision performance [31].

Reflection removal comparison in a real-world scenario

Figure 1. Given a monocular video captured in a real-world scenario, the proposed method demonstrates its effective and efficient reflection removal performance. Compared with vanilla 3DGS [17], the proposed method exhibits greater robustness in real-world scenarios. Compared with the state-of-the-art 3D reflection removal method, NeRFReN [10], the proposed method not only proves effective but also excels in efficiency, enabling real-time rendering.

However, removing reflections in real-world scenarios remains ill-posed. Learning-based methods improve robustness with additional priors, e.g., edge maps [6], polarization [20], textual prompts [12, 53], and flash/no-flash cues [21]. Due to limited scene coverage in current datasets [20, 23, 26, 29, 35, 39, 51, 55], such methods still suffer from domain gap and often fail under strong reflections or non-ideal guidance. Multi-frame methods [1, 4, 7, 9, 24, 48] can aggregate clean information across frames, yet priors from limited neighbors (e.g., optical flow) remain too weak for strong reflections or significant motions [4, 24].

Thanks to progress in 3D neural scene representations [17, 28], 3D reflection removal methods [8, 10, 41, 47, 54] show strong robustness via multi-view consistency and crafted assumptions. Both NeRF-based (e.g., TNeRF [54], NeRFReN [10]) and 3DGS-based methods (e.g., DC-Gaussian [41], Flash-Splat [47]) perform well. However, NeRFReN’s long training and rendering times hinder practicality, while pixel-wise optimization limits global perceptual quality. Flash-Splat exploits real-time 3DGS rendering [17], but its reliance on flash/no-flash cues [21, 45, 46] in RAW images complicates capture and SfM; these cues also weaken as camera–scene distance grows.

To inherit the robustness and overcome the weaknesses of current 3D reflection removal methods, we propose a novel reflection-free 3DGS method for the 3D reflection removal task. The proposed method simplifies the capture process by requiring only a monocular sRGB video, easily obtained with any mobile phone or DSLR camera, without relying on specialized photography techniques such as capturing flash or no-flash RAW images [47]. Inspired by Li et al. [24], we leverage the Photometric Prior in real-world reflection scenarios: for a given 3D scene point, the minimum intensity value across all captured views strongly corresponds to reflection-free information. Since pixel values from the captured views must be accurately projected back into 3D space, obtaining precise depth maps is crucial to effectively apply the photometric prior. Benefiting from explicit depth estimation in 3DGS, we can efficiently generate depth maps for each captured view by optimizing a vanilla 3DGS model under reflection conditions. To ensure the depth maps contain as pure reflection-free depth information as possible, we incorporate a Geometric Prior—also used by NeRFReN [10]—within 3DGS, which encourages smoothness in the rendered depth maps. This geometric prior assumes that the depth map of the transmitted component should be locally smoother than that of the reflection-degraded one. Utilizing these refined depth maps, we project all captured views back into the 3D space and apply the photometric prior, then re-render each view with preliminary clean transmission information. Due to the flaws of depth maps and imperfect projections, current reflection-free views still have inevitable artifacts, as shown in Figure 4. Directly optimizing the 3DGS on them leads to poor 3D consistency and color distortion. To address this issue, the proposed method goes into the comprehensive optimization where original captured views and preliminary reflection-free views are jointly treated as supervision during 3DGS optimization. We propose an Adaptive Twin Rendering (ATR) loss to achieve this joint supervision, encouraging the 3DGS model to effectively capture the clean transmission information from preliminary reflection-free views and maintain the original 3D consistency for a reflection-degraded scene.

Illustration of depth-based 3D warp

Figure 4. Illustration of depth-based 3D warp. c∗c_{*} denotes the camera parameters of the captured view I∗I_{*}. For certain meaningless areas in the warped image Im→nI_{m\to n}, we fill these areas with pure white color.

Furthermore, recognizing that existing benchmarks provide limited scene diversity—for example, TNeRF [54] includes only four scenes—for quantitatively evaluating 3D reflection removal methods, we introduce a comprehensive 3D reflection removal benchmark aimed at significantly enhancing evaluation diversity. Since capturing videos with and without reflections along the exact same camera trajectory is challenging, we leverage the precise camera control capabilities of the modern rendering engine Unity [11] to construct our benchmark. To guarantee the diversity of our benchmark, we build 3D scenes with different settings (background & reflection scenes, glass thickness, transparency and reflection strength), resulting in 45 cases for evaluation. Besides, we additionally collect real-world cases for qualitative evaluation. Extensive experiments demonstrate that the proposed method significantly outperforms previous works on the proposed benchmark and across various real-world scenarios. This strong robustness and simplicity of our method indicate its potential for practical applications.

Related Work

Single-Image Reflection Removal

With the rapid development of deep learning techniques, learning-based methods [5, 6, 12, 13, 14, 23, 27, 30, 36, 37, 38, 42, 43, 44, 49, 51, 52, 53] are chosen as the first alternative compared with early traditional methods [2, 9, 22, 25, 32, 48, 50, 52]. Fan et al. [6] propose the deep learning-based method, CEILNet, which first estimates an edge map as the assisted cue for the subsequent transmission layer estimation. Zhang et al. [51] propose to utilize perceptual loss [15] to train a deep network by considering misalignments, which are caused by glass refraction, between the estimated and ground-truth transmission layers. Wen et al. [44] propose the generative-based reflection synthesis and removal networks for training the removal network based on the non-linear synthetic data. Kim et al. [18] propose a physically based rendering pipeline for data synthesis for subsequent removal network training. Hu Hu et al. [14] propose to capture a learnable residue term during input image decomposition, which guides a further separation between transmission and reflection layers. Zhu Zhu et al. [55] propose a cascaded method, using their proposed maximum reflection filter to locate reflection for guiding subsequent removal. Most recently, Zhong Zhong et al. [53] and Hong Hong et al. [12] propose language-guided methods by utilizing text prompts to aid reflection removal.

Overview of the proposed method

Figure 2. An overview of the proposed method. 1) During Initialization, all captured views are processed through structure-from-motion (SfM) to generate an initial 3D scene point cloud, which is then used to initialize a 3DGS model; 2) During Geometric Prior Utilization, the 3DGS model is optimized using learnable residual maps E^n\hat{E}_n combined with the depth smoothness geometric prior to generate coarse reflection-free depth maps D^n\hat{D}_n; 3) During Photometric Prior Utilization, using the reflection-free depth maps, all captured views are projected back onto the 3D scene point cloud. Subsequently, for re-rendering each specific view, a minimum value-based photometric prior is applied to extract preliminary clean transmission information; 4) During Comprehensive Optimization, three 3DGS models are jointly optimized for producing transmission image T^n\hat{T}_n, reflection image R^n\hat{R}_n and reflectance factor β^n\hat{\beta}_n, respectively.

While learning-based methods demonstrate strong performance in benchmark evaluations, the challenge of domain adaptation limits their effectiveness in real-world daily scenarios.

Multi-frame Reflection Removal

Compared to single-image methods, multi-frame methods [1, 7, 9, 24, 26, 48] show their superiority with exploiting useful information from multiple frames. Alayrac Alayrac et al. [1] and Liu Liu et al. [26] propose learning-based methods which are trained on synthetic data. Such methods still suffer from domain-gap problems when they are applied on real-world scenarios. Despite learning-based methods, traditional multi-frame methods focus on exploiting image structural priors [9, 24] or motion priors [7, 48] for separating transmission and reflection layers. Most recently, Chugunov Chugunov et al. [4] propose neural spline fields for burst frames reflection removal. However, these multi-frame methods face limitations when dealing with significant camera motion, particularly with large changes in viewing angles. Other works seek to enhance reflection removal performance by capturing additional data through optical priors, such as polarization-based methods [20], flash/no-flash techniques [21], and RAW-domain approaches [16]. These methods are limited by their dependence on additional data acquisition, which increases complexity and may not be practical in most everyday situations.

3D Reflection Removal

There are some existing NeRF-based methods [8, 10, 54] for the 3D reflection removal task. TNeRF [54] introduces a pre-trained transmission encoder [43] and recurring edge constraints as guidance into the NeRF backbone to improve its reflection removal performance. Guo Guo et al. [10] propose a NeRF-based method, NeRFReN, by adopting a similar image formulation as Shina Shina et al. [33]. The effectiveness of NeRFReN can be attributed to three main aspects: the incorporation of a depth smoothness prior within the transmission layers, the implementation of bidirectional depth consistency, and the observation that reflections exhibit simpler geometrical characteristics compared to transmission layers. Gao Gao et al. [8] uses a similar architecture to NeRReN but models planar surfaces and casts reflected rays to capture reflections, relying on a pre-trained segmentation model [19] to initialize window surfaces.

There are some existing 3DGS-based methods [34, 41, 47]. However, DC-Gaussian [41] removes reflections from multi-view transmission images degraded by a single-view reflection image, while our work tackles the more complex case of multi-view transmission images degraded by multi-view reflection images. The most recent and similar work to ours is Flash-Splat [47]. Flash-Splat utilizes flash/no-flash cues [21, 45, 46] for optimizing 3DGS [17] with unpaired flash/no-flash views in RAW format. However, the major limitation of flash-based methods is that the effectiveness of flash cues rapidly diminishes as the distance between the camera and the target scene increases. In this work, the proposed method does not need any additional photography techniques but multi-view captures that can be simply captured in a video mode using the sRGB format. This prevents the proposed method from being limited by any constraints.

Method

In this work, we define the 3D reflection removal task as follows: Given a set of multi-view captures IN={In,n∈N}\mathcal{I}_{N} = \{I_{n}, n \in N\}, where NN denotes the view numbers, our objective is to achieve 3D reflection removal by reconstructing transmission and reflection scenes separately.

We follow prior work [10, 47] and model each reflection-degraded observation as:

In=Tn+βn∘Rn,(1)I_{n} = T_{n} + \beta_{n} \circ R_{n}, \tag*{(1)}

where nn denotes view index, TnT_{n}, RnR_{n}, and βn\beta_{n} denote transmission image, reflection image and reflective fraction map.

We reconstruct the scene with 3D Gaussian Splatting (3DGS) [17]. Each Gaussian is parameterized by a mean x∈R3x \in\mathbb{R}^{3}, a covariance Σ\Sigma factored as Σ=RSSTRT\Sigma= RSS^{T}R^{T}, opacity α\alpha, and view-dependent color cc encoded by spherical harmonics. A pixel color is obtained by front-to-back α\alpha-blending of the N\mathcal{N} overlapping Gaussians:

C=∑i∈Nciαi∏j=1i−1(1−αj).(2)C = \sum_{i \in\mathcal{N}} c_i\alpha_i \prod_{j=1}^{i-1}(1-\alpha_j). \tag*{(2)}

Vanilla 3DGS is optimized with a combination of L1\mathcal{L}_{1} and D-SSIM:

Lvanilla=λL1(I^,I)+(1−λ)LD-SSIM(I^,I),(3)\mathcal{L}_{\mathrm{vanilla}} = \lambda\mathcal{L}_{1}(\hat{I}, I) + (1-\lambda)\mathcal{L}_{\mathrm{D\text{-}SSIM}}(\hat{I}, I), \tag*{(3)}

with λ=0.2\lambda= 0.2. We reuse this photometric term in later stages and refer readers to [17] for rasterization and densification details.

Reflection-Free 3D Gaussian Splatting

In this work, we propose a novel reflection-free 3DGS method. Unlike the recent 3DGS-based method, Flash-Splat [47], which requires flash / no-flash multi-view RAW image captures, the proposed method needs no additional photography techniques but only multi-view sRGB images captured by a monocular video. This allows the proposed method to be free from the effectiveness declining of flash / no-flash cues with distance increasing as mentioned in works [21, 45, 46, 47].

Geometric Prior Utilization

Although vanilla 3DGS achieves impressive rendering quality for 3D reconstruction, it often produces floating artifacts and residual reflections in the final outputs during reconstructing a reflection-degraded scenario. To mitigate these issues, we incorporate a geometric prior based on depth smoothness constraints [10], which helps generate cleaner and more accurate depth maps. Specifically, we apply a robust depth smoothness loss together with learnable residual maps that capture unwanted reflection components (see Figure 3), enhancing the optimization of 3DGS on our reflection-degraded inputs:

Captured view and depth maps illustrating learnable residual maps

Figure 3. The significance of learnable residual maps during geometric prior utilization. Introducing residual maps significantly enhances the utilization of the geometric prior, resulting in cleaner, reflection-free depth maps.

LGeom=Lvanilla(I^n+E~n,In)+Lsmooth(D^n),(4)\mathcal{L}_{\mathrm{Geom}} = \mathcal{L}_{\mathrm{vanilla}}(\hat{I}_{n} + \tilde{E}_{n}, I_{n}) + \mathcal{L}_{\mathrm{smooth}}(\hat{D}_{n}), \tag*{(4)}

where E~n\tilde{E}_{n} is the learned residual map, D^n\hat{D}_{n} is the rendered depth map, and Lsmooth\mathcal{L}_{\mathrm{smooth}} is a bilateral filter loss [47] promoting local smoothness.

The key purpose of utilizing this geometric prior is to produce reflection-free depth maps with high accuracy and purity. These improved depth maps are essential for the precise application of the photometric prior, allowing more reliable projection of pixel values into 3D space and better performance of the following photometric prior utilization.

Photometric Prior Utilization

Inspired by Li et al. [24], we observe that, for a point in a 3D scene, the minimum value across all captured views is highly correlated with representing reflection-free transmission. Based on this photometric prior, for each reference view, we obtain its corresponding reflection-free version by the following operations: 1) Perform 3D warping of all auxiliary views using their corresponding over-smooth but reflection-free depth maps rendered from the geometric prior utilization; 2) Calculate the minimum value of all these warped auxiliary views and the reference view at each spatial position; 3) We treat this minimum map as the reflection-free version of the reference view, and regard it as our additional supervised signal during the following comprehensive optimization, a detailed illustration is shown in Figure 4. These operations can be formulated as

Infree=minimum⁡(In,{W(Im,D^n,cm→n)}),(5)I_{n}^{\mathrm{free}} = \operatorname{minimum}\left(I_{n}, \{\mathcal{W}(I_{m}, \hat{D}_{n}, c_{m\rightarrow n})\}\right), \tag*{(5)}

where m∈Nm \in N and m≠nm \ne n, W(⋅)\mathcal{W}(\cdot) denotes depth-based 3D warping function, c∗c_{*} denotes the camera parameters of the captured view I∗I_{*}. Although there are noticeable warping errors in INfree\mathcal{I}_{N}^{\mathrm{free}} shown in Figure 4, we find that their reflection-free information contributes positively to the following optimization in 3DGS. With a proposed ATR loss, we can effectively mitigate the impact of these errors, leading to improved final clean transmission results.

Comprehensive Optimization

To comprehensively optimize the 3DGS model for the final clean 3D scene, we initialize three 3DGS models for transmission, reflection and reflective fraction, respectively. During optimization, we simultaneously optimize the three 3DGS models using several loss functions to achieve a balance, ensuring pure transmission and reflection with complete separation.

Firstly, the vanilla 3DGS loss based on Eq. (1) is applied:

Lvanilla=λL1L1(I^n,In)+(1−λL1)LD−SSIM(I^n,In),(6)\mathcal{L}_{\mathrm{vanilla}} = \lambda_{\mathcal{L}_1}\mathcal{L}_1(\hat{I}_n, I_n) + (1-\lambda_{\mathcal{L}_1})\mathcal{L}_{\mathrm{D-SSIM}}(\hat{I}_n, I_n), \tag*{(6)}

where I^n=T^n+β^n∘R^n\hat{I}_n = \hat{T}_n + \hat{\beta}_n \circ\hat{R}_n, T^n\hat{T}_n, β^n\hat{\beta}_n and R^n\hat{R}_n are rendered from their corresponding 3DGS.

Secondly, to give explicit guidance for optimizing the transmission 3DGS model, a naive L1 or MSE function can be applied to its rendered results: LT=LL1/MSE(Tn,In)+LL1/MSE(Tn,Infree)\mathcal{L}_T = \mathcal{L}_{\mathcal{L}_1/\mathrm{MSE}}(T_n, I_n) + \mathcal{L}_{\mathcal{L}_1/\mathrm{MSE}}(T_n, I_n^{\mathrm{free}}). However, during optimization, we find that the rendered result tends to be collapsed into one of InI_n (is fine-grained but suffers from reflection degradation) and InfreeI_n^{\mathrm{free}} (is free from reflections but exhibits artifacts). Instead of empirically assigning reasonable weights to these two losses, we propose a more intuitive loss, Adaptive Twin Rendering (ATR) loss, that automatically balances them, enabling the transmission 3DGS model to effectively learn the optimal information from both sources:

LATR=minimum⁡(∥Tn−In∥22,∥Tn−Infree∥22).(7)\mathcal{L}_{\mathrm{ATR}} = \operatorname{minimum}\left(\lVert T_n-I_n\rVert_2^2,\lVert T_n-I_n^{\mathrm{free}}\rVert_2^2\right). \tag*{(7)}

This loss works based on two facts: 1) Compared to reconstructing transmission and reflection simultaneously, a single 3DGS model tends to only reconstruct one of them at beginning, LATR\mathcal{L}_{\mathrm{ATR}} encourages the 3DGS model to prioritize fitting the pure transmission information in InfreeI_n^{\mathrm{free}} rather than the more complex reflection-degraded information in InI_n; 2) The 3DGS model tends to reconstruct 3D consistent information, the 3D inconsistent errors in InfreeI_n^{\mathrm{free}} are always ignored when there is more reasonable information in InI_n.

Thirdly, to encourage the reflection 3DGS model to refrain from reconstructing transmission information, we adopt the exclusion loss [51], Φexcl(I1,I2)\Phi_{\mathrm{excl}}(I_1,I_2), minimizing the similarity between the rendered transmission and reflection images. This loss aims to ensure that the rendered outputs remain as distinct and pure as possible:

Lexcl=Lexcl(T^n,R^n)+Lexcl(Infree,R^n).(8)\mathcal{L}_{\mathrm{excl}}=\mathcal{L}_{\mathrm{excl}}(\hat{T}_n,\hat{R}_n)+\mathcal{L}_{\mathrm{excl}}(I_n^{\mathrm{free}},\hat{R}_n). \tag*{(8)}

Fourthly, the bilateral filter loss for encouraging local smoothness is slightly imposed on rendered depths of transmission and reflection scenes, as well as reflective fraction maps. This loss is represented as Lsmooth\mathcal{L}_{\mathrm{smooth}}.

Finally, we combine the above losses with corresponding weights for our comprehensive optimization:

L=λ1Lvanilla+λ2LATR+λ3Lexcl+λ4Lsmooth,(9)\mathcal{L}=\lambda_1\mathcal{L}_{\mathrm{vanilla}}+\lambda_2\mathcal{L}_{\mathrm{ATR}}+\lambda_3\mathcal{L}_{\mathrm{excl}}+\lambda_4\mathcal{L}_{\mathrm{smooth}}, \tag*{(9)}

where λ1=1.0\lambda_1=1.0, λ2=1.0\lambda_2=1.0, λ3=0.01\lambda_3=0.01 and λ4=0.01\lambda_4=0.01.

3D Reflection Removal Benchmark

Table 1. Proposed 3D Reflection Removal Benchmark. Transparency: Higher values indicate clearer transmission and weaker reflection. Thickness: Affects light refraction and glass curvature. Reflection Strength: Controls the intensity of the glass’s reflection.

ConfigurationIntensity RangeTotal Cases
Transparency (%)[88, 92]
Thickness (mm)[1, 7]45
Reflection Strength[12, 135]

Table 1.

Although a real-world benchmark exists [54], it comprises only four scenes, each featuring a single setting (e.g., glass transparency, thickness), which limits its density for comprehensive quantitative evaluation. Besides, building a real-world 3D reflection removal benchmark with clean ground truth is challenging. Capture multiple views, but each view is captured with and without reflection before the camera movement. This necessitates multiple repositionings of the glass, resulting in spatial misalignment of the reflections. This misalignment adversely impacts the quality of 3D reflection reconstruction, ultimately degrading the performance of the final removal.

Based on the abovementioned observation and inspired by Niklaus et al. [29], we turn to build 3D reflection removal benchmark by the sophisticated rendering engine, Unity [11]. With the high-quality reflection simulation in Unity, we find that the performance of 3D reflection removal methods on our proposed benchmark is highly consistent in real-world scenarios. Precise camera movement control in Unity, combined with flexible reflection on/off switching, allows us to generate high-quality ground-truth transmission images for each view. Considering the complexity of real-world scenarios, we build our benchmark considering various real-world reflecting factors, including the transparency, thickness and reflection strength of glasses provided in Unity, as shown in Table 1. Ultimately, we construct 5 distinct 3D scenes by Unity, each with 9 variants, resulting in 45 cases in total for our proposed benchmark.

Experiemnts

Implementation Details

During the initialization, we employ VGGSfM [40] to create the initial point cloud. For a fair comparison, we consistently utilize VGGSfM across all 3D methods in this study. During geometric prior utilization, the 3DGS model is optimized using the standard 3DGS settings as described by Kerbl et al. [17], with 5,000 iterations. During comprehensive optimization, three separate 3DGS models are simultaneously optimized using the same hyperparameters but with different loss functions, each undergoing 30,000 iterations.

Baselines

We compare the proposed method against several strong baselines:

  • 3D-based method: MVSNeRF, TNeRF, NeRFReN, vanilla 3DGS;

  • Burst-imaging-based method: Chugunov et al. [4];

  • Learning-based image method combined with 3DGS: DSRNet [14] + vanilla 3DGS.

Quantitative Evaluation

We conduct a comprehensive quantitative evaluation on both our proposed benchmark and the real-world benchmark [54] of various reflection removal methods, as presented in Table 2 and Table 3.

MethodPSNR ↑SSIM ↑LPIPS ↓
Chugunov et al. [4]2.940.2480.935
Vanilla 3DGS [17]29.580.9170.175
DSRNet [14] + Vanilla 3DGS26.910.8720.240
NeRFReN [10]26.360.7550.284
Ours31.450.9150.150

Table 2. Quantitative evaluation on our proposed benchmark. “DSRNet + Vanilla 3DGS” indicates that degraded views are pre-processed using a state-of-the-art single-image method before 3DGS optimization. The burst-imaging-based method proposed by Chugunov et al. [4] fails to generate outputs with reasonable evaluation scores, indicating its inability to address the 3D reflection removal problem.

MethodPSNR ↑SSIM ↑LPIPS ↓
MVSNeRF [3]17.830.6850.400
NeRFReN [10]18.630.7160.449
TNeRF [54]22.750.8410.205
Vanilla 3DGS [17]21.730.8350.238
DSRNet [14] + Vanilla 3DGS22.030.8380.221
Flash-Splat [47]N/AN/AN/A
Ours25.040.9150.190

Table 3. Quantitative evaluation on the real-world benchmark [54]. Flash-Splat cannot work on such sRGB inputs, since it requires camera RAW images.

In Table 2, the most notable observation is that the burst-imaging-based method proposed by Chugunov et al. [4] fails to generate outputs with reasonable evaluation scores, indicating its inability to address the 3D reflection removal problem. We also observe that despite being the state-of-the-art 3D reflection removal method, NeRFReN completely fails on “Scene 3,” highlighting its lack of robustness across diverse scenarios. According to PSNR, which is highly sensitive to reflection degradation, the proposed method demonstrates strong superiority in 3D reconstruction with effective reflection removal. Furthermore, LPIPS scores indicate that the proposed method achieves state-of-the-art perceptual quality. However, due to SSIM’s insensitivity to reflection degradation, the vanilla 3DGS achieves the highest scores. This suggests that the reflection removal priors employed by NeRFReN and the proposed method may slightly affect structural consistency.

In Table 3, the proposed method demonstrates significantly superior performance compared to previous works. Notably, Flash-Splat fails to operate when only sRGB inputs are provided, as it requires camera RAW images. This highlights that the RAW image requirement can be a burden for daily use.

Qualitative Evaluation

In the qualitative evaluation, the proposed method showcases its strong superiority on our proposed benchmark. Furthermore, we also collect 6 real-world scenes for the qualitative evaluation.

In Figure 5, the proposed method demonstrates robust 3D reflection removal performance in real-world scenarios. As shown in column 2, the vanilla 3DGS approach struggles with severe floating artifacts during 3D reconstruction, heavily impacted by strong reflections. To address this, we applied a state-of-the-art image-based reflection removal method, DSRNet, prior to training the vanilla 3DGS. However, the limited generalization capability of the learning-based DSRNet results in suboptimal reflection removal for each captured view, ultimately leading to degraded 3D reconstruction performance, as illustrated in column 3. We observe that the 3D method, NeRFReN, struggles to remove strong reflections across all scenes, particularly with objects like lamps, where the proposed method excels. This clearly demonstrates the effectiveness of the photometric prior utilization in the proposed method. Additionally, due to the pixel-wise supervision in NeRFReN, the method lacks satisfactory global perceptual quality in the rendered results, leading to inconsistent pixel values across all outputs. Furthermore, the proposed method also reconstructs the underlying 3D reflection scenes, which is a potential guidance for the downstream task.

Qualitative evaluation on various real-world scenarios

Figure 5. Qualitative evaluation on various real-world scenarios. “DSRNet + Vanilla 3DGS” indicates that degraded views are pre-processed using a state-of-the-art single-image method, DSRNet, before 3DGS optimization. For column 3, we show the view processed by DSRNet at the bottom right corner of each rendered result.

In Figure 6, column 3, we observe that while the single-image method, DSRNet, significantly enhances the reflection removal performance for vanilla 3DGS, it also compromises the clarity of the transmission details, particularly the high-frequency textures on the walls and objects. Due to the domain gap issue, such learning-based methods lack sufficient robustness across diverse scenes, resulting in residual reflections in the final outputs. In column 4, NeRFReN fails to effectively remove reflections while preserving clear transmission details. Additionally, it faces challenges in distinguishing between transmission and reflection information, particularly in the scene depicted in row 2. The proposed method effectively removes reflections while preserving high-frequency textures in the transmission information, a particularly challenging aspect of the reflection removal task.

Qualitative evaluation on the proposed benchmark

Figure 6. Qualitative evaluation on the proposed benchmark. “DSRNet + Vanilla 3DGS” indicates that degraded views are pre-processed using a state-of-the-art single-image method before 3DGS optimization. For row 2, as NeRFReN incorrectly identifies the transmission and reflection information, we included its predicted “reflection layer” in the bottom-right corner of the result.

Ablation Study

We conduct a comprehensive ablation study to assess the impact of each key component in our proposed method, as shown in Table 4 and Figure 7. Firstly, we observe that the “LATR w/o InI_n” variant, which relies solely on InfreeI_n^{\mathrm{free}} as the supervised signal, suffers from severe floating artifacts and color distortion due to the photometric prior, leading to unsatisfactory evaluation scores. Next, we analyze the “LATR w/o InfreeI_n^{\mathrm{free}}” variant, which undergoes full optimization but uses only InI_n as the supervised signal, lacking supervision from the InfreeI_n^{\mathrm{free}}. Although this variant achieves acceptable 3D reconstruction performance, its reflection removal capability is significantly compromised. These findings underscore the effectiveness of the proposed ATR loss, demonstrating that relying solely on InI_n or InfreeI_n^{\mathrm{free}} as the supervised signal fails to achieve a satisfactory balance between 3D reconstruction and reflection removal. Finally, we evaluate the effectiveness of the exclusion loss and smooth loss using two distinct variants, as shown in Table 4 and Figure 7. Both losses contribute to improved overall performance. The “w/o Lsmooth\mathcal{L}_{\mathrm{smooth}}” variant demonstrates that without the local smoothness constraint on depth maps, strong reflections are more likely to persist in the rendered transmission image. The “w/o Lexcl\mathcal{L}_{\mathrm{excl}}” variant reveals that, without the exclusion loss, the reflection scene may retain transmission information, resulting in incomplete transmission reconstruction, as shown in the bottom-left corner of the rendered result. We also investigate the effect of the number of input views on the proposed method and observe that it maintains satisfactory robustness across different input view counts.

Qualitative evaluation of the ablation study

Figure 7. Qualitative evaluation of the ablation study. The reconstructed reflection scene and rendered depth map displayed in the small corner images.

MethodPSNR ↑SSIM ↑LPIPS ↓
Ours (LATR\mathcal{L}_{\mathrm{ATR}} w/o InI_n)19.640.7900.237
Ours (LATR\mathcal{L}_{\mathrm{ATR}} w/o InfreeI_n^{\mathrm{free}})30.450.8880.176
Ours (w/o Lexcl\mathcal{L}_{\mathrm{excl}})31.000.9080.155
Ours (w/o Lsmooth\mathcal{L}_{\mathrm{smooth}})31.110.9090.155
Ours31.250.9150.150
Ours (11 views)31.140.9070.140
Ours (22 views)31.270.9120.146
Ours (88 views)31.250.9150.150

Table 4. Quantitative evaluation of the ablation study.

Processing Time

Table 5 presents a detailed comparison of training times on the proposed benchmark with 88 views per case. Our method requires 60.74 minutes to train, which is significantly faster than NeRFReN’s 63.48 hours, demonstrating a substantial improvement in efficiency. While Vanilla 3DGS and DSRNet combined with Vanilla 3DGS complete training more quickly (5.94 and 7.85 minutes respectively), these methods do not achieve the same level of performance and robustness as our method. Overall, our method strikes a strong balance between training speed and quality, offering a practical solution for large-scale or time-sensitive applications.

Vanilla 3DGSDSRNet + Vanilla 3DGSNeRFReNOursOurs (3D Warp)
Time5.94 min7.85 min63.48 h60.74 min30 sec

Table 5. The processing-time comparison on the proposed benchmark. Time represents the training time for each case (88 views).

Conclusion

In this work, we introduce a novel reflection-free 3DGS method for the 3D reflection removal task, named RF3DGS. Unlike previous works that rely on additional photography techniques, such as flash or no-flash cues, RAW images, the proposed method simply requires a monocular sRGB video capturing multiple views of a 3D scene. By leveraging the photometric prior, we propose an adaptive twin rendering (ATR) loss that effectively enhances both reflection removal and 3D reconstruction performance. Additionally, we present a 3D benchmark with substantial diversity to enable quantitative evaluation of 3D reflection removal methods. Extensive experiments on the proposed benchmark and real-world scenes demonstrate the strong superiority of the proposed method over existing methods.

References

  1. [1]Jean-Baptiste Alayrac, Joao Carreira, and Andrew Zisserman. The visual centrifuge: Model-free layered video representations. In CVPR, pages 2457–2466, 2019. 2, 3arxiv.org/abs/1812.01461
  2. [2]Nikolaos Arvanitopoulos, Radhakrishna Achanta, and Sabine Süsstrunk. Single image reflection suppression. In CVPR, pages 4498–4506, 2017. 2DOI
  3. [3]Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao Su. Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. In ICCV, pages 14124–14133, 2021. 6arxiv.org/abs/2103.15595
  4. [4]Ilya Chugunov, David Shustin, Ruyu Yan, Chenyang Lei, and Felix Heide. Neural spline fields for burst image fusion and layer separation. In CVPR, pages 25763–25773, 2024. 2, 3, 6arxiv.org/abs/2312.14235
  5. [5]Zheng Dong, Ke Xu, Yin Yang, Hujun Bao, Weiwei Xu, and Rynson WH Lau. Location-aware single image reflection removal. In ICCV, pages 5017–5026, 2021. 2arxiv.org/abs/2012.07131
  6. [6]Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, and David Wipf. A generic deep architecture for single image reflection removal and image smoothing. In ICCV, pages 3238–3247, 2017. 2arxiv.org/abs/1708.03474
  7. [7]Kun Gai, Zhenwei Shi, and Changshui Zhang. Blind separation of superimposed moving images using image statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1):19–32, 2011. 2, 3DOI
  8. [8]Chen Gao, Yipeng Wang, Changil Kim, Jia-Bin Huang, and Johannes Kopf. Planar reflection-aware neural radiance fields. In ACM Transactions on Graphics (SIGGRAPH Asia), pages 1–10, 2024. 2, 3arxiv.org/abs/2411.04984
  9. [9]Xiaojie Guo, Xiaochun Cao, and Yi Ma. Robust separation of reflection from multiple images. In CVPR, pages 2187–2194, 2014. 2, 3DOI
  10. [10]Yuan-Chen Guo, Di Kang, Linchao Bao, Yu He, and Song-Hai Nie. Nerfren: Neural radiance fields with reflections. In CVPR, pages 18409–18418, 2022. 1, 2, 3, 4, 6arxiv.org/abs/2111.15234
  11. [11]John K Haas. A history of the unity game engine. 2014. 2, 6
  12. [12]Yuchen Hong, Haofeng Zhong, Shuchen Weng, Jinxiu Liang, and Boxin Shi. L-diff er: Single image reflection removal with language-based diffusion model. In ECCV, 2024. 2, 3DOI
  13. [13]Qiming Hu and Xiaojie Guo. Trash or treasure? an interactive dual-stream strategy for single image reflection separation. Advances in Neural Information Processing Systems, 34:24683–24694, 2021.arxiv.org/abs/2110.10546
  14. [14]Qiming Hu and Xiaojie Guo. Single image reflection separation via component synergy. In CVPR, pages 13138–13147, 2023. 2, 3, 6arxiv.org/abs/2308.10027
  15. [15]Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV, pages 694–711. Springer, 2016. 2arxiv.org/abs/1603.08155
  16. [16]Eric Kee, Adam Pikielny, Blackburn-Matzen Kevin, and Marc Levoy. Removing reflections from raw photos. In ECCV, 2024. 3arxiv.org/abs/2404.14414
  17. [17]Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 42(4), 2023. 1, 2, 3, 4, 6arxiv.org/abs/2308.04079
  18. [18]Soomin Kim, Yuchi Huo, and Sung-Eui Yoon. Single image reflection removal with physically-based training images. In CVPR, pages 5164–5173, 2020. 2arxiv.org/abs/1904.11934
  19. [19]Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In ICCV, pages 4015–4026, 2023. 3
  20. [20]Chenyang Lei, Xuhua Huang, Mengdi Zhang, Qiong Yan, Wenxiu Sun, and Qifeng Chen. Polarized reflection removal with perfect alignment in the wild. In CVPR, pages 1750–1758, 2020. 2, 3
  21. [21]Chenyang Lei, Xudong Jiang, and Qifeng Chen. Robust reflection removal with flash-only cues in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 2, 3, 4DOI
  22. [22]Anat Levin and Yair Weiss. User assisted separation of reflections from a single image using a sparsity prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(9):1647–1654, 2007. 2
  23. [23]Chao Li, Yixiao Yang, Kun He, Stephen Lin, and John E Hopcroft. Single image reflection removal through cascaded refinement. In CVPR, pages 3565–3574, 2020. 2arxiv.org/abs/1911.06634
  24. [24]Yu Li and Michael S Brown. Exploiting reflection change for automatic reflection removal. In ICCV, pages 2432–2439, 2013. 2, 3, 4DOI
  25. [25]Yu Li and Michael S Brown. Single image layer separation using relative smoothness. In CVPR, pages 2752–2759, 2014. 2DOI
  26. [26]Yu-Lun Liu, Wei-Sheng Lai, Ming-Hsuan Yang, Yung-Yu Chuang, and Jia-Bin Huang. Learning to see through obstructions. In CVPR, 2020. 2, 3arxiv.org/abs/2004.01180
  27. [27]Daiqian Ma, Renjie Wan, Boxin Shi, Alex C Kot, and Ling-Yu Duan. Learning to jointly generate and separate reflections. In ICCV, pages 2444–2452, 2019. 2DOI
  28. [28]Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020. 2arxiv.org/abs/2003.08934
  29. [29]Simon Niklaus, Xuaner Cecilia Zhang, Jonathan T Barron, Neal Wadhwa, Rahul Garg, Feng Liu, and Tianfan Xue. Learned dual-view reflection removal. In WACV, pages 3713–3722, 2021. 2, 6arxiv.org/abs/2010.00702
  30. [30]BH Prasad, Lokesh Borregowda, Kaushik Mitra, Sanjoy Chowdhury, et al. V-desirr: Very fast deep embedded single image reflection removal. In ICCV, pages 2390–2399, 2021. 2DOI
  31. [31]J Redmon. You only look once: Unified, real-time object detection. In CVPR, 2016. 2arxiv.org/abs/1506.02640
  32. [32]YiChang Shih, Dilip Krishnan, Fredo Durand, and William T Freeman. Reflection removal using ghosting cues. In CVPR, pages 3193–3201, 2015. 2DOI
  33. [33]Sudipta N Sinha, Johannes Kopf, Michael Goesele, Daniel Scharstein, and Richard Szeliski. Image-based rendering for scenes with reflections. ACM TOG, 31(4):1–10, 2012. 3DOI
  34. [34]Jiayi Song, Zihan Ye, Qingyuan Zhou, Weidong Yang, Ben Fei, Jingyi Xu, Ying He, and Wanli Ouyang. Reflections unlock: Geometry-aware reflection disentanglement in 3d gaussian splatting for photorealistic scenes rendering. arXiv preprint arXiv:2507.06103, 2025. 3arxiv.org/abs/2507.06103
  35. [35]Renjie Wan, Boxin Shi, Ling-Yu Duan, Ah-Hwee Tan, and Alex C Kot. Benchmarking single-image reflection removal algorithms. In CVPR, pages 3922–3930, 2017. 2
  36. [36]Renjie Wan, Boxin Shi, Ling-Yu Duan, Ah-Hwee Tan, and Alex C Kot. Corrn: Multi-scale guided concurrent reflection removal network. In CVPR, pages 4777–4785, 2018. 2arxiv.org/abs/1805.11802
  37. [37]Renjie Wan, Boxin Shi, Haoliang Li, Ling-Yu Duan, Ah-Hwee Tan, and Alex C Kot. Corrn: Cooperative reflection removal network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(12):2969–2982, 2019.DOI
  38. [38]Renjie Wan, Boxin Shi, Haoliang Li, Ling-Yu Duan, and Alex C Kot. Reflection scene separation from a single image. In CVPR, pages 2398–2406, 2020. 2DOI
  39. [39]Ce Wang, Dejia Xu, Renjie Wan, Bin He, Boxin Shi, and Ling-Yu Duan. Background scene recovery from an image looking through colored glass. IEEE Transactions on Multimedia, 25:2876–2887, 2022. 2DOI
  40. [40]Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In CVPR, pages 21686–21697, 2024. 6DOI
  41. [41]Linhan Wang, Kai Cheng, Shuo Lei, Shengkun Wang, Wei Yin, Chenyang Lei, Xiaoxiao Long, and Chang-Tien Lu. Dc-gaussian: Improving 3d gaussian splatting for reflective dash cam videos. Advances in Neural Information Processing Systems, 37:98998–99920, 2024. 2, 3arxiv.org/abs/2405.17705
  42. [42]Mengyi Wang, Xinmin Zhang, Yongshun Gong, and Yilong Yin. Personalized single image reflection removal network through adaptive cascade refinement. In ACM MM, pages 8204–8213, 2023. 2DOI
  43. [43]Kaixuan Wei, Jiaolong Yang, Ying Fu, David Wipf, and Hua Huang. Single image reflection removal exploiting mis-aligned training data and network enhancements. In CVPR, pages 8178–8187, 2019. 3arxiv.org/abs/1904.00637
  44. [44]Qiang Wang, Yinjie Tan, Jing Qin, Wenxi Liu, Guoqiang Han, and Shengfeng He. Single image reflection removal beyond linearity. In CVPR, pages 3771–3779, 2019. 2
  45. [45]Zhihao Xia, Michael Gharbi, Federico Perazzi, Kalyan Sunkavalli, and Ayan Chakrabarti. Deep denoising of flash and no-flash pairs for photography in low-light environments. In CVPR, pages 2063–2072, 2021. 2, 3, 4arxiv.org/abs/2012.05116
  46. [46]Zhihao Xia, Jason Lawrence, and Supreeth Achary. A dark flash normal camera. In ICCV, pages 2430–2439, 2021. 2, 3arxiv.org/abs/2012.06125
  47. [47]Mingyang Xie, Haoming Cai, Sachin Shah, Yiran Xu, Brandon Y. Feng, Jia-bin Huang, and Christopher Metzler. Flash-splat: 3d reflection removal with flash cues and gaussian splats. In ECCV, 2024. 2, 3, 4, 6arxiv.org/abs/2410.02764
  48. [48]Tianfan Xue, Michael Rubinstein, Ce Liu, and William T. Freeman. A computational approach for obstruction-free photography. ACM TOG, 34(4), 2015. 2, 3DOI
  49. [49]Jie Yang, Dong Gong, Lingqiao Liu, and Qinfeng Shi. Seeing deeply and bidirectionally: A deep learning approach for single image reflection removal. In ECCV, pages 654–669, 2018. 2DOI
  50. [50]Yang Yang, Wenye Ma, Yin Zheng, Jian-Feng Cai, and Weiyu Xu. Fast single image reflection suppression via convex optimization. In CVPR, pages 8141–8149, 2019. 2arxiv.org/abs/1903.03889
  51. [51]Xuaner Zhang, Ren Ng, and Qifeng Chen. Single image reflection separation with perceptual losses. In CVPR, 2018. 2, 5arxiv.org/abs/1806.05376
  52. [52]Qian Zheng, Boxin Shi, Jinnan Chen, Xudong Jiang, Ling-Yu Duan, and Alex C Kot. Single image reflection removal with absorption effect. In CVPR, pages 13395–13404, 2021. 2DOI
  53. [53]Haofeng Zhong, Yuchen Hong, Shuchen Weng, Jinxiu Liang, and Boxin Shi. Language-guided image reflection separation. In CVPR, pages 24913–24922, 2024. 2, 3arxiv.org/abs/2402.11874
  54. [54]Chengxuan Zhu, Renjie Wan, and Boxin Shi. Neural transmitted radiance fields. Advances in Neural Information Processing Systems, 35:38994–39006, 2022. 2, 3, 5, 6DOI
  55. [55]Yurui Zhu, Xueyang Fu, Peng-Tao Jiang, Hao Zhang, Qibin Sun, Jinwei Chen, Zheng-Jun Zha, and Bo Li. Revisiting single image reflection removal in the wild. In CVPR, pages 25468–25478, 2024. 2, 3arxiv.org/abs/2311.17320

Paper details

vlnc:W_h98sfstb
Version
v2 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 15

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents