betaFeedback

WorksComputer Science

LUMEN: Geometry-Guided Diffusion Model for Spatially-Varying HDR Illumination Estimation

Abstract

Estimating scene illumination from images is challenging as lighting is inherently spatially varying, depending on geometry, occlusion, and light sources. Existing methods incorporate geometric cues by deriving local environment maps or probe-based representations, but largely rely on regression or parametric models, resulting in over-smoothed lighting and limited realism. We revisit spatially-varying illumination estimation from a generative perspective and introduce LUMEN, a geometry-conditioned diffusion framework for synthesizing high-dynamic-range (HDR) environment maps at arbitrary spatial locations from single or multi-view images. Specifically, we reconstruct a coarse 3D scene from input images and render partial environment maps at arbitrary locations to encode visibility and spatial structure. These signals are fused with multi-view image features and fed into a diffusion transformer to generate full HDR environment maps. To train the model, we build a synthetic dataset of 700 indoor scenes with dense spatial supervision, including multi-view RGB images, depth, and HDR illumination at multiple sampled locations. This dataset enables the model to learn spatially consistent lighting distributions across diverse scenes. Extensive experiments demonstrate that LUMEN significantly improves visual fidelity and spatial coherence compared to prior approaches.

Input multi-view images, object insertion, and predicted environment maps

Figure 1. Given a set of multi-view images, LUMEN estimates spatially-varying illumination at any location with consistency and achieves seamless virtual object insertion results (the balls and the bunny are rendered using predicted HDRI maps at their positions).

Authors’ Contact Information: Hongze Chen, The Hong Kong University of Science and Technology, Hong Kong; Fangyu Lin, The Hong Kong University of Science and Technology, Hong Kong; Runze Zhang, LIGHTSPEED, China; Lingting Zhu, LIGHTSPEED, China; Keyang Luo, LIGHTSPEED, China; Yingda Yin, LIGHTSPEED, China; Zeyu Hu, LIGHTSPEED, China; Weikai Chen, LIGHTSPEED, China; Xin Wang, LIGHTSPEED, China; Zehong Lin, Lingnan University, Hong Kong; Jun Zhang, The Hong Kong University of Science and Technology, Hong Kong.

This work is licensed under a Creative Commons Attribution 4.0 International License.

SA Conference Papers ’26, Kuala Lumpur, Malaysia

© 2026 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-2842-6/2026/12

https://doi.org/10.1145/3829340.3842306

CCS Concepts: Computing methodologies → Computer vision; Image-based rendering.

Additional Key Words and Phrases: Illumination Estimation

ACM Reference Format:

Hongze Chen, Fangyu Lin, Runze Zhang, Lingting Zhu, Keyang Luo, Yingda Yin, Zeyu Hu, Weikai Chen, Xin Wang, Zehong Lin, and Jun Zhang. 2026.

SA Conference Papers ’26, December 01–04, 2026, Kuala Lumpur, Malaysia.

LUMEN: Geometry-Guided Diffusion Model for Spatially-Varying HDR Illumination Estimation. In SIGGRAPH Asia 2026 Conference Papers (SA Conference Papers ’26), December 01–04, 2026, Kuala Lumpur, Malaysia. ACM, New York, NY, USA, 18 pages. https://doi.org/10.1145/3829340.3842306

Introduction

Estimating scene illumination from images is a fundamental problem in computer graphics and vision, with applications in augmented reality, relighting, and virtual object insertion. While many approaches model illumination as a single global environment map, real-world lighting is inherently spatially varying: illumination changes across space due to geometry, occlusion, and local light sources. Accurately recovering such spatially-varying illumination from limited visual observations remains a challenging and ill-posed problem.

Prior work has made progress by incorporating geometric reasoning into lighting estimation. A common paradigm reconstructs scene structure and derives local lighting representations, such as per-pixel environment maps, probe-based lighting, or volumetric light fields at different spatial locations [3, 8, 9, 29, 31, 33]. These approaches introduce spatial awareness by explicitly modeling how illumination varies with position. However, they typically rely on regression models or low-dimensional parameterizations (e.g., spherical harmonics or Gaussian mixtures), which limits their ability to capture high-frequency lighting effects. As a result, the predicted illumination is often over-smoothed, lacks realism, and degrades significantly in regions that are weakly observed or not directly visible.

This limitation suggests that the core challenge lies not only in determining where to estimate illumination, but also in how to recover realistic lighting from incomplete local observations. In particular, local geometric cues, such as partial environment maps derived from reconstructed scenes, provide only a sparse and visibility-constrained view of the full lighting distribution. They capture visibility and occlusion but inevitably miss unseen regions and high-frequency lighting effects. Recovering complete high-dynamic-range (HDR) illumination from such partial observations requires strong priors over natural lighting, which are not well captured by regression-based methods. Thus, instead of direct regression, we propose reframing this task as a completion problem, where the goal is to infer missing information about the full illumination distribution based on partial observations from a generative perspective.

To this end, we propose LUMEN, a geometry-conditioned diffusion framework that formulates illumination estimation as conditional generation. Instead of encoding spatial variation implicitly or conditioning on raw 3D coordinates, LUMEN estimates illumination at a query location using a location-dependent partial environment map. Given the input views, it first recovers coarse scene geometry and camera poses via an off-the-shelf multi-view reconstruction method [35]. For each query location, the reconstructed geometry is projected into a panoramic representation centered at that location, yielding a partial environment map that captures geometry-aware visibility and occlusion cues. This partial environment map serves as an explicit spatial conditioning signal, so changing the query location directly changes the conditioning input and thus the generated illumination.

We implement this design by fine-tuning a pre-trained diffusion transformer (DiT). Multi-view image features and the location-dependent partial environment map are jointly fused with the noisy latent through the global attention mechanisms of the DiT, enabling effective information sharing across views while enforcing spatially consistent illumination estimation. As a result, LUMEN generates complete HDRI maps that are consistent with both appearance cues and spatial constraints. Training a geometry-conditioned generative model requires supervision that captures how illumination varies across space. To bridge this gap, we build a synthetic dataset that contains approximately 700 indoor scenes with diverse layouts, textures, and lighting, generated using a modified version of Infinigen Indoors [26]. For each scene, we render multi-view RGB images with depth maps and HDRI maps at different positions. The camera trajectory and light-probe placement follow predefined rules to ensure physically plausible configurations and broad scene coverage, providing sufficient contextual cues about appearance, lighting, and geometry.

Extensive experiments on both synthetic and real-world datasets demonstrate that LUMEN produces more photorealistic and spatially consistent illumination than prior approaches. In particular, our method better captures high-frequency lighting variations, improves realism in unseen regions, and benefits from multi-view inputs to achieve stronger spatial coherence. In summary, our main contributions are as follows:

  • We reformulate spatially-varying illumination estimation as a geometry-conditioned generative problem, where lighting is recovered as completion from partial, visibility-constrained observations.

  • We present LUMEN, a geometry-conditioned diffusion model that integrates partial environment maps as spatially grounded observations and leverages generative priors to infer missing regions and high-frequency illumination.

  • We construct a large-scale synthetic dataset with dense spatial illumination supervision, such as multi-view RGB images, depth maps, and spatially-varying environment illumination with camera parameters.

  • We demonstrate consistent improvements in photorealism and spatial consistency over prior approaches.

Related Work

Lighting Estimation. Estimating scene lighting has long been a fundamental challenge in computer vision and graphics, playing a critical role in applications such as virtual object insertion and relighting. Early methods mainly rely on physical light probes [6] or specific scene cues, such as reflections in eyes [22, 40] or highlights from glossy objects [23], to infer environmental illumination. To generalize beyond specific cues, learning-based methods have emerged to estimate lighting from casual visual captures. A prevalent paradigm is to directly regress HDRI maps from LDR images using neural networks trained on HDR datasets [7, 11, 14, 15, 43] (Gardner et al. 2017; Somanath and Kurz 2021; Zhan et al. 2021). While effective for global illumination estimation, these regression methods often struggle to provide high-frequency details in unobserved regions.

Recent advances in generative models have demonstrated remarkable capabilities in modeling complex distributions of natural images and videos. A pioneering work Stylelight Wang et al. [34] trains a dual-StyleGAN to simultaneously synthesize LDR and HDRI maps from the noisy latent, using GAN inversion to find the latent of the panorama corresponding to the input images during inference. Similarly, EverLight Dastjerdi et al. [5] conditions a GAN on both input images and extracted parametric lighting to predict illumination for both indoor and outdoor scenes. More recently, DiffusionLight Phongthawee et al. 2024 has been proposed to inpaint a chrome ball as a light probe into the input image by fine-tuning a pre-trained 2D text-to-image diffusion model and controlling the exposure using the text condition. These multi-exposure balls are then warped to LDR panoramas and fused into an HDRI map. However, the visual quality of the result is affected by the ball-to-panorama warping, and the dynamic range is limited by the HDR fusion process. Another notable approach, LuxDiT Liang et al. [20], fine-tunes a video diffusion transformer to directly predict scene lighting from an image or video to enhance visual quality. Although the above methods benefit from the strong priors of generative models, their training pipelines largely adhere to the conventional paradigm and lack a comprehensive spatial understanding. Since lighting estimation requires reasoning about the distribution of illumination and the scene layout, how to endow generative models with stronger awareness of both lighting and visual space remains underexplored.

Spatially-varying Lighting Estimation. To extend beyond global illumination, some methods aim to recover the spatially-varying illumination. An intuitive approach is to estimate the illumination at the location of each pixel. These methods typically recover scene geometry from visual inputs and warp the scene into a partial panorama at a specific location, subsequently predicting a complete HDRI map from the warped version [29, 45, 47]. Alternatively, other approaches directly regress the illumination at each pixel represented by spherical harmonics [8] or spherical Gaussians [17] using an end-to-end network. Moreover, instead of estimating 2D HDR lighting, some techniques aim to reconstruct spatially coherent 3D light fields using volumetric representations [18, 31, 37] or voxel octrees [36]. More recently, 4DLighting Tong et al. [33] optimizes a 4D MLP light field using a 2D diffusion prior via score distillation sampling (SDS) to achieve spatiotemporally consistent lighting estimation. However, it is limited to monocular inputs and often produces over-smoothed results due to its heavy reliance on SDS. A concurrent work, LIMO Bolduc et al. 2026, also enhances the diffusion model’s spatial awareness by providing richer geometric conditions, thereby enabling more accurate prediction of spatially varying illumination.

Inverse Rendering. In contrast to methods that focus solely on estimating illumination from visual inputs, inverse rendering aims to jointly model and recover the scene geometry, materials, and illumination. Optimization-based inverse rendering methods [4, 21, 30] (Zhang et al. 2021) utilize differentiable rendering to optimize the scene representation and illumination through photometric loss, supervised by dense multi-view captures. Nonetheless, these methods often simplify the optimization process by representing scene lighting as a global environment map, which lacks spatial awareness and fails to accurately characterize the complex light distribution in the scene. Some methods attempt to model the light fields that permeate through the space [1, 39], but the results are over-smoothed. On the other hand, learning-based methods [17, 38, 46] use neural networks to directly predict scene attributes and illumination. However, these methods often rely on screen space information, lacking complete simulation of light-matter interactions in the scene, and thus struggle to accurately handle complex light transport and effects.

Method

In this section, we present our LUMEN framework for estimating both global and spatially-varying illumination, represented as HDRI maps, from single-view or multi-view image captures. Our key idea is to enhance the spatial awareness of the model by incorporating multi-view priors through both model design and dataset curation. This enables the model to predict the illumination of unseen regions in the scene. Specifically, we repurpose a pre-trained 2D diffusion transformer to generate HDRI maps conditioned on input image(s) and a partial LDR environment map rendered from a reconstructed point cloud (§ 3.1). We further build a large-scale dataset that includes spatially-varying illumination from synthetic scenes and photo-realistic global illumination from real-world HDRI maps (§ 3.2). An overview of our framework is depicted in Figure 2.

Figure 2. Overview of LUMEN. Given a single-view image or multi-view inputs, we first estimate the camera parameters and reconstruct a coarse point cloud using an off-the-shelf feed-forward 3D reconstruction model. We then render partial environment maps at arbitrary locations in the scene based on the reconstructed geometry, and condition a 2D diffusion model on the input views with these partial environment maps to generate a complete HDRI map.

Lighting Prediction from Visual Captures

Our method predicts both global and spatially-varying lighting from visual inputs. To elucidate the overall framework, we first outline how spatially-varying lighting is inferred. Given a set of multi-view images I∈R3×H×W×NI \in\mathbb{R}^{3 \times H \times W \times N}, where NN is the number of images (with N=1N = 1 for a single-view image input), we aim to predict the HDRI map H∈R3×H′×W′H \in\mathbb{R}^{3 \times H' \times W'} at a specific position x=(x,y,z)x = (x,y,z). Since the scene geometry and camera parameters of the input views remain unknown, we use VGGT, an off-the-shelf feed-forward geometry reconstruction network [35], to estimate the depth map DD, camera intrinsics KK, and camera extrinsics [R∣t][R|t] for each view. Subsequently, we back-project the depth points from each view to obtain the scene point cloud. Then, we render a partial LDR panorama Hp∈R3×H′×W′H_p \in\mathbb{R}^{3 \times H' \times W'} from the point cloud by positioning an equirectangular camera at xx. The input multi-view images II and this partial panorama HpH_p are then sent to the 2D diffusion transformer as conditions for inferring the HDRI map.

Model Design. The Diffusion Transformer Peebles and Xie [24] has achieved significant success in image and video generation and perception, owing to its pure attention design that facilitates efficient information fusion across different inputs. In our framework, the DiT is conditioned on the input multi-view images, which provide visual and semantic annotations of the scene and the partial environment map that ensures the consistency between predicted results and the scene. However, directly concatenating condition tokens with latent tokens, as in previous work [20], may lead to suboptimal performance, since perspective and panorama images perform different 2D spatial formulations. This difference makes it difficult for the pre-trained model to fuse multi-view information into latent tokens during the denoising process, as it lacks 3D perceptual capabilities. Therefore, we employ different encoders to extract features from different conditions, followed by an embedding layer that integrates these features into a unified token for processing by the transformer blocks. Moreover, since the value range of HDR images HH spans [0,+∞][0,+\infty], tone-mapping is necessary to rescale the input to [0,1][0,1]. Previous methods have utilized both Reinhard [27] and normalized log tone-mapping to preserve the high intensity in bright regions while maintaining details in dark areas. However, we find that only using Reinhard tone-mapping achieves comparable results, yielding the tone-mapped image HLH_L as follows:

HL=H1+H.(1)H_L = \frac{H}{1+H}. \tag*{(1)}

For input multi-view images, we use the SigLIP [41] encoder to extract semantic features that guide the DiT to generate results with correct layout, color temperature, and style. The extracted features provide a rich and faithful representation of the scene, which we then project into VAE’s latent space using a learnable linear layer, resulting in the multi-view feature fI=Linear⁡(SigLIP⁡(I))∈RN×h×w×cf_I = \operatorname{Linear}(\operatorname{SigLIP}(I)) \in\mathbb{R}^{N \times h \times w \times c}. Similarly, the features of the partial environment map are extracted using a pre-trained VAE encoder, yielding fHp=VAE⁡(Hp)∈R1×h′×w′×cf_{H_p} = \operatorname{VAE}(H_p) \in\mathbb{R}^{1 \times h' \times w' \times c} to accurately preserve visual information. Finally, the encoded multi-view images, the partial environment map, and the noisy latent zLz_L are concatenated to form a unified input sequence of tokens. We also implement a view-aware RoPE [Su et al. 2024] index on these unified tokens to help the model understand the spatial relationships between the noisy latent and the condition images. For each token at the 2D location (i,j)(i,j) in the tt-th input (e.g., a multi-view image, the partial panorama, or the noisy latent), we first construct a RoPE index that places tokens from all inputs into a shared indexing space. Specifically, we define

the RoPE index as follows:

ϕ(i,j,t)=(i,j,0)+∑k=1t(1,0,ok),(2)\phi(i,j,t)=(i,j,0)+\sum_{k=1}^{t}(1,0,o_k), \tag*{(2)}

where oko_k is a resolution-dependent offset associated with the kk-th input. The accumulated offset separates different inputs along the indexing space, preventing index collisions and controlling interview separations. The unified sequence of tokens and the view-aware index are then sent to the DiT to produce the denoised latent:

z^L=DiT⁡([fI;fHp;z],ϕ),(3)\hat{z}_L = \operatorname{DiT}\left([f_I;f_{H_p};z],\phi\right), \tag*{(3)}

where [;][;] denotes concatenation along the token dimension. Then, z^L\hat{z}_L is decoded using the VAE decoder to generate an LDR panorama, followed by inverse tone-mapping to recover the HDRI map. The model is trained using a standard flow-matching loss as follows:

L=Ez0,t,ϵ∼N(0,I)∥vθ(zt;fI,fHp)−(ϵ−z0)∥22,(4)\mathcal{L}=\mathbb{E}_{z_0,t,\epsilon\sim\mathcal{N}(0,I)}\left\lVert v_\theta(z_t;f_I,f_{H_p})-(\epsilon-z_0)\right\rVert_2^2, \tag*{(4)}

where ztz_t is the latent representation at denoising step tt and ϵ−z0\epsilon-z_0 represents the ground-truth velocity.

Dataset Curation

We construct our training data by combining a large-scale synthetic dataset and real-world HDRI maps from public data sources. The synthetic dataset contains multi-view RGB images and spatially-varying illumination of diverse indoor scenes, enabling the model to learn complex lighting distributions and enhance spatial awareness. In contrast, training on real-world data mitigates a decrease in the visual quality of generated images and improves robustness against in-the-wild scenarios. The specific data strategy is outlined below.

Synthetic Spatially-varying Lighting. Training our model requires a dataset with ground-truth spatially-varying lighting from various positions in the scene and the corresponding multi-view RGB images. Unfortunately, current public datasets with spatially-varying illumination cannot meet our requirements. For example, InteriorNet [16] provides only LDR panoramas, while OpenRooms [19] contains per-pixel spatially-varying illumination only on the surface of the scene. Among existing datasets, InteriorVerse [46] is the only one that consists of true HDR spatially-varying illumination and multi-view RGB images for diverse indoor scenes, although the resolutions of its rendered panoramas are relatively low. To address these limitations, we leverage Infinigen Indoors [26], a rule-based framework, to generate 700 diverse indoor scenes with various layouts, material textures, and illumination. To enhance the diversity and controllability of the generation process, we incorporate 200 additional BSDF materials into the existing Infinigen Indoors material library and introduce extra rules to ensure that the generated room layouts align with predefined shapes, such as rectangles, convex hull, or L-shapes, facilitating subsequent camera position sampling. For each scene, we randomly generate a sky texture or sample an environment map from the selected 557 HDRI maps as the background in Blender, and adjust the strength to ensure reasonable brightness.

For each indoor scene, we place 12 cameras on a full orbit around the scene, each directed toward the center, while modulating their height and elevation using a sinusoid function. We also enforce a clear line of sight within a fixed forward-distance threshold to avoid occlusions and maximize scene coverage. Then, we randomly sample light probes in the scene and camera frustums. To be specific, we first extract the bounding convex hull of the scene and uniformly sample [6, 9] light probes, ensuring that no occluding objects are presented within proximity by performing ray intersection tests. In addition, we sample [1, 3] light probes in each camera’s frustum, ensuring visibility from the camera’s perspective without obstruction.

Real-world HDRI maps. Training our model using real-world HDR data enhances the generalizability and robustness of our method. We generate training pairs of 1,527 HDRI maps collected from four public data sources. Each training pair consists of an HDRI map and a limited field-of-view LDR cropped image obtained by projecting the panorama to a field-of-view image using a perspective camera. The perspective camera is generated with randomly sampled azimuth angles in [0, 360] degrees and elevation angles in [−20-20, 20] degrees, with a field of view that varies from 50 to 80 degrees. Then, we randomly select tone-mapping functions such as Gamma2.4, Flimic, and ACES to map the cropped images to the LDR format without exposure compensation.

Experiments

Experiment Settings

Implementation Details. We initialize our DiT from FLUX.1-dev [2] and fine-tune it using LoRA [12]. The resolutions of input images vary from 512×384512 \times384 to 768×768768 \times768, while the resolutions for point cloud rendering and output environment maps are set to 1024×5121024 \times512. We train our model for 20,000 steps using the AdamW optimizer [13], with a batch size of 8 and a warm-up phase of 2,000 steps on 8 NVIDIA H20 GPUs.

Datasets. We evaluate LUMEN on five benchmark datasets encompassing diverse indoor/outdoor environments and featuring both global illumination and spatially-varying lighting. (1) Laval Indoor [Gardner et al. 2017]: following prior works [34] (Phongthaw et al. 2024), we use the same 289 HDRI maps from the official test split; (2) Laval Outdoor [11]: this dataset consists of 205 outdoor HDRI maps captured using a DSLR camera; (3) Laval Indoor Spatially Varying HDR Dataset [8]: this dataset consists of 20 indoor scenes with physical probes placed in them, captured using a DSLR camera; (4) Infinigen Indoors (single/multi-view): we construct a test set of 21 indoor scenes using Infinigen Indoor [26]. For each scene, we: (i) sample one camera and place four light probes within its view frustum to render corresponding perspective images and HDRI maps for evaluating spatially-varying illumination from a single image; and (ii) additionally place [6, 9] cameras throughout the scene to evaluate performance with multi-view inputs; (5) In-the-wild: we collect 16 synthetic indoor scenes from the Internet and build the dataset using the same procedure as employed for Infinigen Indoors, but extend the amount of light probes to [6, 12].

Evaluation Protocol. We follow the three-sphere evaluation protocol adopted in prior works [34] (Gardner et al. 2017; Phongthawee et al. 2024). Specifically, we use the predicted HDRI map (at a resolution of 128×256128 \times256) to render three spheres with different materials (gray diffuse, silver matte, and silver mirror) and compare the resultant renderings with those generated using the ground-truth HDRI map. For global illumination estimation, we report scale-invariant RMSE (si-RMSE) [10] and normalized RMSE (computed after mapping the 0.1st and 99.9th percentiles to 0 and 1) [Phongthawee et al. 2024] to assess the predicted illumination distribution and structure, as well as angular error [14] to evaluate the chromatic accuracy. For spatially-varying illumination, we replace normalized RMSE with RMSE to more accurately reflect intensity errors in the predicted illuminance. We also report the SSIM to evaluate structural fidelity. All metrics are averaged across all sample locations in all scenes.

Baselines. We compare our method against a comprehensive set of lighting estimation baselines to demonstrate its effectiveness. For global illumination estimation from a cropped image, we include StyleLight [34], DiffusionLight [Phongthawee et al. 2024], and LuxDiT [20] as other methods do not release official inference code. Since StyleLight is trained on indoor data, we fine-tune it using the Laval Outdoor dataset to ensure a fair comparison. For spatially-varying illumination estimation from a single-view image, we compare against DiffusionLight [Phongthawee et al. 2024] and 4DLighting [33]. For spatially-varying illumination estimation from multi-view images, we adapt the inference pipeline of 4DLighting to accommodate multi-view inputs and compare our method with this extended baseline.

Global Illumination Estimation from a Single Image

We first evaluate our approach against prior generative methods for global illumination estimation from a cropped image. Table 1 reports quantitative results for both indoor and outdoor scenes. Our method outperforms previous approaches on the Laval Indoor and Laval Outdoor datasets across nearly all metrics. The only exception is the angular error for the rendered diffuse sphere. Nevertheless, our method still achieves higher accuracy in overall color reproduction, indicating that it effectively captures the scene’s illumination distribution with faithful chromaticity.

DatasetMethodScale-invariant RMSE ↓ DiffuseScale-invariant RMSE ↓ MatteScale-invariant RMSE ↓ MirrorAngular Error ↓ DiffuseAngular Error ↓ MatteAngular Error ↓ MirrorNormalized RMSE ↓ DiffuseNormalized RMSE ↓ MatteNormalized RMSE ↓ Mirror
Laval IndoorStyleLight0.0640.1520.2653.8744.60610.9740.2320.4110.520
Laval IndoorDiffusionLight0.0750.1630.2823.4154.1576.4830.2460.3910.457
Laval IndoorLuxDiT0.0560.1400.2742.7593.8756.0990.2070.3880.546
Laval IndoorOurs0.0460.1200.2522.7003.7736.2570.1800.3190.411
Laval OutdoorStyleLight0.1310.2860.4563.5244.8647.6090.2930.3770.475
Laval OutdoorDiffusionLight0.0970.2520.4312.8384.2376.8350.1850.3200.434
Laval OutdoorLuxDiT0.1320.2810.4562.7944.1496.6220.2370.3510.460
Laval OutdoorOurs0.0870.2210.4032.9403.8986.3090.1620.2970.423

Table 1. Quantitative results on the Laval Indoor and Laval Outdoor datasets, underlined entries indicate the second-best results.

We further present qualitative comparisons in Figure 3. Our method yields higher visual fidelity and more accurate localization of light sources in both indoor and outdoor environments. Both DiffusionLight and LuxDiT predict HDRI maps with plausible overall lighting distributions, but they often fail to capture fine-grained details. In addition, DiffusionLight exhibits noticeable distortions near the center of the HDRI map, likely stemming from its reliance on back-projection from a probe sphere. Moreover, StyleLight sometimes misplaces light sources in indoor scenes and fails to produce meaningful predictions for outdoor environments. The object insertion results in Figure 7 show that our method yields more accurate color, brightness, and shadowing than the baselines, suggesting enhanced estimation of illumination distribution, chromaticity, and intensity. In addition, we also provide image-conditioned HDRI maps synthesis results in the Appendix.

Qualitative comparison on the Laval Indoor and Laval Outdoor datasets

Figure 3. Qualitative comparison on the Laval Indoor and Laval Outdoor datasets. (Please zoom in to see details)

Qualitative comparison of object insertion on the Laval Indoor dataset and the Laval Indoor Spatially Varying HDR dataset

Figure 7. Qualitative comparison of object insertion on the Laval Indoor dataset (top) and the Laval Indoor Spatially Varying HDR dataset (below). (Please zoom in to see details)

Spatially-varying Lighting Estimation

To evaluate spatially-varying lighting estimation, we compare our method with representative baselines on the Laval Indoor Spatially Varying (SV), Infinigen Indoor, and In-the-wild datasets. For Infinigen Indoor and In-the-wild datasets, we report results under both single-view and multi-view inputs to assess how different methods benefit from additional visual observations. Table 2 summarizes the quantitative comparisons, while Figure 4 and Figure 6 provide qualitative results. Our method outperforms the baselines in terms of RMSE, si-RMSE, and SSIM on two synthetic datasets, achieving more accurate estimation of illumination intensity and spatial distribution, and is only surpassed by 4DLighting in angular error. We note that 4DLighting is trained exclusively on synthetic scenes generated by Infinigen Indoor and rendered in Blender, and thus owns a particularly strong dataset-specific prior of tone and chromaticity. Furthermore, our method consistently outperforms all baselines on every metric over the real-world Laval Indoor SV dataset, highlighting its robustness and generalizability.

SceneMethodRMSE ↓\downarrow DiffuseRMSE ↓\downarrow MatteRMSE ↓\downarrow Mirrorsi-RMSE ↓\downarrow Diffusesi-RMSE ↓\downarrow Mattesi-RMSE ↓\downarrow MirrorAngular Error ↓\downarrow DiffuseAngular Error ↓\downarrow MatteAngular Error ↓\downarrow MirrorSSIM ↑\uparrow DiffuseSSIM ↑\uparrow MatteSSIM ↑\uparrow Mirror
Infinigen (single)DiffusionLight (CVPR’24)0.1570.2280.4600.0660.1800.4282.9354.0535.3940.9470.6790.471
4DLighting (SIGGRAPH’25)0.1640.2070.4300.0430.1340.3811.4572.4183.5950.9720.7540.533
Ours0.1100.1490.3690.0420.1080.3292.7623.3824.5940.9730.8320.679
In-the-wild (single)DiffusionLight (CVPR’24)0.1720.2360.3510.0810.1930.3101.5922.6813.8560.9230.6540.447
4DLighting (SIGGRAPH’25)0.1410.1900.2890.0480.1320.2381.1861.9042.8650.9690.7550.519
Ours0.1530.1760.2760.0470.1140.2251.3452.3793.5630.9640.7970.616
Laval Indoor SV (single)DiffusionLight (CVPR’24)0.2620.2890.3650.0720.1690.2532.8823.6044.3160.9310.7280.572
4DLighting (SIGGRAPH’25)0.1690.2470.3220.0850.1800.2513.0993.7534.4590.9150.6980.577
Ours0.1830.2160.2980.0690.1540.2482.5723.4934.1670.9390.7690.599
Infinigen (multi-view)4DLighting (SIGGRAPH’25)0.2130.2310.4430.0370.1240.3751.5962.3973.5680.9780.7620.511
Ours0.0800.0950.2530.0240.0580.2101.9252.4163.3400.9880.9420.849
In-the-wild (multi-view)4DLighting (SIGGRAPH’25)0.1820.2040.2910.0380.1110.2101.1681.7842.7280.9720.7730.505
Ours0.1040.1180.1720.0270.0650.1431.5221.8983.0030.9840.9070.788

Table 2. Quantitative comparison for spatially-varying lighting estimation. The results are reported in terms of RMSE, si-RMSE, angular error, and SSIM.

Image not extracted

Figure 4. Qualitative comparison of spatially-varying lighting estimation from a single-view image.

Qualitative comparison of spatially-varying lighting estimation from multi-view images

Figure 6. Qualitative comparison of spatially-varying lighting estimation from multi-view images.

For spatially-varying lighting estimation from a single-view image, the results in Figure 4 show that our method accurately localizes light sources, predicts their intensities, and reconstructs spatially coherent illumination across various probe locations. In contrast, DiffusionLight, which is primarily trained for global illumination estimation, yields inconsistent predictions and mislocalized light sources in the spatially-varying context. While 4DLighting produces spatially coherent results, its predictions show marginal variation as the probe location changes, and it may inaccurately place light sources. Moreover, the over-smoothed nature of its MLP-based light-field representation often leads to a lack of concentrated high-intensity light sources. The object insertion results in Figure 7 further shows that our method faithfully recovers real-world spatially-varying illumination, whereas the baseline methods fail to do so.

With multi-view inputs, Table 2 and Figure 6 show that our method consistently benefits from the additional information, yielding more accurate light source localization and improved chromaticity compared to the single-view setting. In contrast, 4DLighting exhibits limited gains, with the results in Figure 6 revealing artifacts and periodic patterns. This suggests that it does not reliably maintain consistent predictions across viewpoints as the camera pose changes. We also provide more real-world results in Figure 8 and the Appendix.

Visualizations of object insertion and the estimated HDRI maps from multi-view input images

Figure 8. Visualizations of object insertion and the estimated HDRI maps from multi-view input images.

Ablation Study

To investigate the impact of our architecture design and data strategy, we conduct an ablation study by training three variants of our model: (1) Single-only, which is trained using paired cropped images and HDRI maps only from real-world datasets; (2) Infinigen-only, which is trained solely on our synthetic dataset; and (3) w/o SigLIP, in which we remove the SigLIP conditioning and the corresponding embedding layer. Table 3 reports quantitative results on Laval Indoor and in-the-wild spatially-varying datasets. Our model consistently outperforms all variants, validating the effectiveness of our design choices.

SceneMethodRMSE ↓\downarrow DiffuseRMSE ↓\downarrow MatteRMSE ↓\downarrow Mirrorsi-RMSE ↓\downarrow Diffusesi-RMSE ↓\downarrow Mattesi-RMSE ↓\downarrow MirrorAngular Error ↓\downarrow DiffuseAngular Error ↓\downarrow MatteAngular Error ↓\downarrow MirrorN-RMSE ↓\downarrow DiffuseN-RMSE ↓\downarrow MatteN-RMSE ↓\downarrow Mirror
Laval IndoorSingle-only0.2680.2870.4090.0530.1300.2593.6544.3596.4360.2310.3960.508
Laval IndoorInfinigen-only0.3040.3180.4580.0520.1280.2643.6624.5456.8260.1970.4010.557
Laval Indoorw/o SigLIP0.2020.2420.3620.0580.1290.2494.7765.7908.7430.1950.3470.450
Laval IndoorOurs0.1840.2180.3510.0460.1200.2522.7003.7736.2570.1800.3190.411
In-the-wildSingle-only0.5020.4570.5030.0560.1230.2002.5252.9753.6190.2280.5430.647
In-the-wildInfinigen-only0.1300.1380.2270.0310.0790.1921.9062.0712.8540.1140.2250.296
In-the-wildw/o SigLIP0.1030.1180.2040.0340.0770.1762.3302.7034.2180.1110.2050.269
In-the-wildOurs0.1040.1180.1720.0270.0650.1431.5221.8983.0030.0920.1720.225

Table 3. Ablation results on Laval indoor and in-the-wild scenes. Results are reported in terms of RMSE, si-RMSE, angular error, and normalized RMSE.

We further provide qualitative comparisons in Figure 5. The full model produces results that more closely align with the ground truth in terms of overall color accuracy, intensity, and illumination distribution. The Single-only variant struggles to accurately localize and model light-source distributions for global illumination estimation and largely fails in spatially varying lighting scenarios due to the lack of spatial supervision in its training data. The Infinigen-only variant exhibits reduced visual fidelity, reflecting the domain gap between synthetic renders and real captures. Lastly, the w/o SigLIP variant cannot faithfully recover scene layouts and chromaticity, indicating that SigLIP features provide useful semantic cues for conditioning the model on the input image.

Qualitative comparison of ablations

Figure 5. Qualitative comparison of ablations.

Conclusion

In this paper, we introduce LUMEN, a generative approach for estimating both global illumination and spatially-varying lighting from either a single image or multiple views. Our method repurposes a pre-trained diffusion model to infer HDRI maps from visual observations, and improves its spatial awareness and understanding of lighting by incorporating multi-view priors through (i) an embedding layer that fuses multi-view features into unified tokens, and (ii) a large-scale synthetic dataset containing diverse spatially-varying scenes and illumination conditions. Extensive experiments show that LUMEN consistently outperforms prior methods in both global environment-map estimation and spatially-varying lighting estimation, while producing photorealistic results.

Limitations and Discussion

Our approach relies on the learned prior and spatial reasoning capabilities of a 2D diffusion model to enforce consistency in spatially-varying illumination. In an ideal setting, given a large-scale, high-quality dataset of spatially varying illumination, our model can learn the complex spatial distribution of illumination. With sufficiently informative inputs, such as multi-view images, the model can faithfully recover the spatially varying illumination in a scene. However, when the input views cover only a limited portion of the scene, the model samples from the learned distribution for unobserved regions and produces plausible results. In this case, it cannot guarantee that the predicted illumination is consistent with the ground truth. In this sense, the task is closer to geometry-informed generation.

Besides, our method uses only Reinhard tone mapping to rescale HDR images. Although our experimental results (see in Appendix) show that this is sufficient in most cases, Reinhard tone mapping may fail to preserve the dynamic range of the input under extreme illumination conditions, such as direct sunlight in outdoor environments or sharp indoor spotlights. This can lead to numerical instability in the predicted light-source intensities, as well as chromatic deviations. For such scenes, incorporating normalized log tone mapping in addition to Reinhard tone mapping can more effectively recover light-source information. However, predicting results under both tone-mapping operators introduces additional computational overhead and requires extra mechanisms to ensure consistency between the two outputs, potentially calling for further model-design considerations. Therefore, we believe that the appropriate design choice should depend on the intended application. Since our work focuses on general scenes, we use only Reinhard tone mapping. Moreover, our model does not explicitly address spatiotemporal illumination changes. We expect that extending our framework to video diffusion models could alleviate these limitations by leveraging their inherent spatiotemporal coherence.

Additional Experiments and Visualization Results

Dynamic range comparison on the Laval Indoor dataset and Laval Outdoor dataset

To further validate our design choice of using only Reinhard tone mapping, we evaluate whether it preserves a sufficient dynamic range in most cases. We compare the illumination estimated by our method and the baselines on the Laval Indoor and Laval Outdoor datasets using three metrics: dynamic range error, Jensen–Shannon (JS) divergence, and Earth Mover’s Distance (EMD). Specifically, DRerror=∣DRpred−DRGT∣\mathrm{DR}_{\mathrm{error}} = \lvert\mathrm{DR}_{\mathrm{pred}} - \mathrm{DR}_{\mathrm{GT}} \rvert, DR=log⁡2L99.9L0.1\mathrm{DR} = \log_{2}\frac{L_{99.9}}{L_{0.1}}, where L99.9L_{99.9} and L0.1L_{0.1} denote the 99.9th and 0.1th percentile luminance values, respectively. JS divergence and EMD are computed between the normalized luminance histograms of the predicted and ground-truth illumination.

Table 4 reports the quantitative results. Our method consistently outperforms other generative lighting-estimation approaches, achieving the best overall performance across both datasets. These results confirm that our design choice is sufficient for preserving the dynamic range in most scenarios.

SceneMethodDR-Error ↓JS ↓EMD ↓
Laval IndoorStyleLight5.5410.0970.718
Laval IndoorDiffusionLight3.5380.0940.477
Laval IndoorLuxDiT2.9600.0820.471
Laval IndoorOurs2.6260.0590.379
Laval OutdoorStyleLight9.9960.1060.663
Laval OutdoorDiffusionLight1.6880.1020.460
Laval OutdoorLuxDiT2.2350.1220.547
Laval OutdoorOurs2.1030.1000.410

Table 4. Quantitative comparison for dynamic range on the Laval Indoor and Laval Outdoor datasets. Lower is better for all metrics.

To further compare the dynamic range of the predicted illumination, we augment the original three-sphere protocol by adding a diffuse plane beneath the spheres, with an albedo of [0.8, 0.8, 0.8] and a roughness of 0.5. We render the scene using a camera elevation of 45° and compute the corresponding rendering metrics in this shadow-aware setting. Table 5 reports the quantitative results. Our method continues to outperform other generative lighting-estimation approaches, further supporting our claim that Reinhard tone-mapping preserves sufficient dynamic range in most scenarios.

DatasetMethodScale-invariant RMSE ↓\downarrow DiffuseMatteMirrorAngular Error ↓\downarrow DiffuseMatteMirrorNormalized RMSE ↓\downarrow DiffuseMatteMirror
Laval IndoorStyleLight0.0310.0490.0843.8613.9083.9970.1660.2590.328
Laval IndoorDiffusionLight0.0300.0500.0943.1933.2363.3180.1600.2110.233
Laval IndoorLuxDiT0.0260.0430.0863.0143.0663.1370.1400.2270.303
Laval IndoorOurs0.0250.0410.0832.9573.0113.1010.1260.2120.257
Laval OutdoorStyleLight0.0990.1220.1715.9225.9836.1400.2690.2530.239
Laval OutdoorDiffusionLight0.0790.1050.1624.3864.4554.5740.2070.2170.229
Laval OutdoorLuxDiT0.0880.1270.2084.3624.4214.5320.2790.2640.343
Laval OutdoorOurs0.0480.0750.1362.6062.6562.7810.1450.1770.205

Table 5. Quantitative results on the Laval Indoor and Laval Outdoor datasets for the 3-Sphere-Shadow setting with shadow. Underlined entries indicate the second-best results.

Array of Spheres

We adopt the array-of-spheres protocol used in prior works [5] (Phongthawee et al. 2024) to further evaluate performance on the Laval Indoor dataset. Following these works, we use the same 2,240 perspective crops extracted from 224 HDRI maps. For each crop, we estimate the corresponding HDRI map and use it to render an array of diffuse spheres placed on a ground plane.

Table 6 reports the quantitative results, where our method outperforms other generative lighting estimation approaches. Figure 9 shows qualitative comparisons, demonstrating that LUMEN recovers more accurate light-source locations than DiffusionLight.

Methodsi-RMSE ↓\downarrowAngular-Error ↓\downarrow
EverLight0.0916.360
StyleLight0.1237.090
DiffusionLight0.0905.250
LuxDiT0.0894.900
Ours0.0874.474

Table 6. Quantitative results under the array of spheres protocol on the Laval Indoor dataset.

Qualitative results under the array of spheres protocol

Figure 9. Qualitative results under the array of spheres protocol on the Laval Indoor dataset.

Spatially-varying illumination Estimation Comparison on the PanoHDR-NeRF Dataset

We further evaluate our method on the PanoHDR-NeRF dataset, including two additional baselines: PanoHDR-NeRF and GasLight. Both approaches follow a two-stage formulation. Specifically, they first employ a network to reconstruct HDR images from the input LDR observations, and subsequently optimize a radiance field in linear HDR space using the predicted HDR images. HDRI maps are then rendered from the optimized radiance field.

As shown in Table 7, our method achieves better quantitative performance than both baselines, further validating its effectiveness and generalization to real-world scenes. In addition, both PanoHDR-NeRF and GasLight rely on dense multi-view inputs for radiance field reconstruction, whereas our method naturally extends to sparse-view settings, including the single-view case, thereby offering a more practical solution.

MethodChess room PU21-PSNR ↓\downarrowSmall class PU21-PSNR ↓\downarrow
PanoHDR-NeRF27.6128.57
GasLight30.2930.77
Ours31.2834.70

Table 7. Quantitative comparison for spatially-varying lighting estimation on PanoHDR-NeRF Dataset.

Additional Visualization Results of Spatially-Varying Illumination Estimation on Real-World Captures

We present spatially-varying illumination estimation results on in-the-wild, real-world captures in Figure 10 and Figure 11. As shown in Figure 10, LUMEN produces spatially varying illumination that is both consistent and visually faithful, with fine-grained details. For example, in the bottom-right case, as the sphere moves farther away from the window, the predicted illumination gradually weakens, while the spatial distribution of the illumination remains well aligned with the scene. This result demonstrates strong performance and robust generalization to unconstrained real-world scenes. In addition, the object insertion results in Figure 11 further demonstrate the accuracy of our predictions, as the inserted objects are naturally and seamlessly integrated into the scene.

Visualization of the estimated spatially-varying illumination on in-the-wild scenes

Figure 10. Visualization of the estimated spatially-varying illumination on in-the-wild scenes. The inserted spheres on the left correspond one-to-one to the rendered results on the right: the spheres are ordered from left to right, matching the results arranged from top to bottom.

Visualization of the object insertion results on the Laval Indoor Spatially Varying dataset

Figure 11. Visualization of the object insertion results on the Laval Indoor Spatially Varying dataset.

HDRI maps Synthesis Results

Figure 12 demonstrates image-conditioned HDRI map synthesis for real-world photographs. The generated panoramas achieve photo-realistic quality across diverse indoor and outdoor scenarios with coherent scene layouts and illumination distributions, highlighting the strong generalizability of our approach.

Image conditioned panorama generation results on real-world captures

Figure 12. Image conditioned panorama generation results on real-world captures.

Dataset Details

Dataset Curation

Synthetic Dataset. We conduct a synthetic dataset that contains around 700 rooms of five types, including bedroom, bathroom, dining room, kitchen, and living room. Moreover, we render the multi-view images and light probes using Blender Cycles2 renderer with 1024 samples per pixel and OptiX denoiser. It takes approximately six days to generate the scenes and another four days to render the dataset. The overall dataset contains around 8,500 RGB images and 15,000 HDRI maps with ground-truth camera parameters and geometry.

We further provide details on the light probe placement strategy. For each scene, we first extract the meshes corresponding to the walls, floor, and ceiling, and merge them into a closed watertight mesh that serves as the bounding box. Candidate light probe locations are then uniformly sampled within this bounding box. For each sampled location, we cast 128 rays uniformly distributed over the sphere and measure their travel distances. If the minimum ray distance falls below a predefined threshold, the sampled location is regarded as being too close to nearby occluders and is therefore rejected. We repeat this process until the target number of valid probe locations is obtained. In addition, we uniformly sample extra probe locations within the camera frustum and apply the same occlusion criterion to filter out invalid samples.

Real-world Dataset. We collect real-world HDRI maps from the following four public databases:

  • Poly Haven3, which contains 944 HDRI maps of diverse indoor and outdoor scenes.

  • Ambient CG4, which contains 375 HDRI maps of diverse indoor and outdoor scenes.

  • HDR Maps5, which contains 186 HDRI maps of indoor and outdoor scenes.

  • HDRI Skies6, which contains 22 HDRI maps of outdoor scenes.

Implementation Details

For all baselines, we use the official publicly released codebases and pretrained weights to obtain the reported results. For StyleLight, we further fine-tune the pretrained model on the Laval Outdoor dataset for 25,000 iterations to make it applicable to outdoor scenes.

For 4DLighting, the official codebase provides a framework for optimizing an MLP light field from a single-view input using SDS. Specifically, under the camera coordinate system of the input image, the method samples nine different positions within the camera frustum at each iteration, uses the diffusion model to predict the corresponding light probes, and optimizes the MLP light field based on these predictions.

To extend this framework to the multi-view setting, we first estimate the camera poses of the input multi-view images and normalize them to the coordinate system of the first camera. We then randomly select one input view and similarly sample nine positions within its camera frustum, from which the diffusion model predicts the corresponding light probes. The MLP light field is then optimized according to the 3D locations of these nine probes expressed in the coordinate system of the first camera. In addition, while the original 4DLighting represents light probe locations using pixel coordinates and inverse depth, we instead use the 3D coordinates of each light probe in the multi-view setting to preserve correct spatial and projection relationships.

References

  1. [1]Benjamin Attal, Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan T Barron, Matthew O’Toole, and Pratul P Srinivasan. 2024. Flash cache: Reducing bias in radiance cache based inverse rendering. In European Conference on Computer Vision. Springer, 20–36.arxiv.org/abs/2409.05867
  2. [2]BlackForestLabs. 2024. Flux.1 Model Family. (2024). https://blackforestlabs.ai/announcing-black-forest-labs
  3. [3]Christophe Bolduc, Julien Philip, Li Ma, Mingming He, Paul Debevec, and Jean-François Lalonde. 2025. GaSLight: Gaussian Splats for Spatially-Varying Lighting in HDR. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 29120–29130.
  4. [4]Hongze Chen, Zehong Lin, and Jun Zhang. 2024. GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering. arXiv preprint arXiv:2410.02619 (2024).arxiv.org/abs/2410.02619
  5. [5]Mohammad Reza Karimi Dastjerdi, Jonathan Eisenmann, Yannick Hold-Geoffroy, and Jean-François Lalonde. 2023. Everlight: Indoor-outdoor editable hdr lighting estimation. In Proceedings of the IEEE/CVF international conference on computer vision. 7420–7429.arxiv.org/abs/2304.13207
  6. [6]Paul Debevec. 1998. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’98). Association for Computing Machinery, New York, NY, USA, 189–198. doi:10.1145/280814.280864DOI
  7. [7]Marc-André Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Christian Gagné, and Jean-François Lalonde. 2019. Deep Parametric Indoor Lighting Estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/1910.08812
  8. [8]Mathieu Garon, Kalyan Sunkavalli, Sunil Hadap, Nathan Carr, and Jean-François Lalonde. 2019. Fast spatially-varying indoor lighting estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6908–6917.arxiv.org/abs/1906.03799
  9. [9]Pulkit Gera, Mohammad Reza Karimi Dastjerdi, Charles Renaud, PJ Narayanan, and Jean-François Lalonde. 2022. Casual indoor HDR radiance capture from omnidirectional images. arXiv preprint arXiv:2208.07903 (2022).arxiv.org/abs/2208.07903
  10. [10]Roger Grosse, Micah K Johnson, Edward H Adelson, and William T Freeman. 2009. Ground truth dataset and baseline evaluations for intrinsic image algorithms. In 2009 IEEE 12th International Conference on Computer Vision. IEEE, 2335–2342.DOI
  11. [11]Yannick Hold-Geoffroy, Akshaya Athawale, and Jean-Francois Lalonde. 2019. Deep Sky Modeling for Single Image Outdoor Lighting Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  12. [12]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yunzhui Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR 1, 2 (2022), 3.arxiv.org/abs/2106.09685
  13. [13]Diederik P Kingma. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).arxiv.org/abs/1412.6980
  14. [14]Chloe LeGendre, Wan-Chun Ma, Graham Fyffe, John Flynn, Laurent Charbonnel, Jay Busch, and Paul Debevec. 2019. Deeplight: Learning illumination for unconstrained mobile mixed reality. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5918–5928.
  15. [15]Chloe LeGendre, Wan-Chun Ma, Rohit Pandey, Sean Fanello, Christoph Rhemann, Jason Dourgarian, Jay Busch, and Paul Debevec. 2020. Learning Illumination from Diverse Portraits. In SIGGRAPH Asia 2020 Technical Communications (Virtual Event, Republic of Korea) (SA ’20). Association for Computing Machinery, New York, NY, USA, Article 7, 4 pages. doi:10.1145/3410700.3425432DOI
  16. [16]Wenbin Li, Sajad Saeedi, John McCormac, Ronald Clark, Dimos Tzoumanikas, Qing Ye, Yuzhong Huang, Rui Tang, and Stefan Leutenegger. 2018. Interiornet: Mega-scale multi-sensor photo-realistic indoor scenes dataset. arXiv preprint arXiv:1809.00716 (2018).arxiv.org/abs/1809.00716
  17. [17]Zhenglin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. 2020. Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting, and svbrdf from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2475–2484.arxiv.org/abs/1905.02722
  18. [18]Zhengqin Li, Li Yu, Mikhail Okunev, Manmohan Chandraker, and Zhao Dong. 2023. Spatiotemporally consistent hdr indoor lighting estimation. ACM Transactions on Graphics 42, 3 (2023), 1–15.arxiv.org/abs/2305.04374
  19. [19]Zhengqin Li, Ting-Wei Yu, Sheng Yang, Sarah Wang, Meng Song, Yuhan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gudavarapu, Jia Shi, Sai Bi, Hong-Xing Yu, Zexiang Xu, Kalyan Sunkavalli, Milos Hasan, Ravi Ramamoorthi, and Manmohan Chandraker. 2021. OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 7190–7199.DOI
  20. [20]Ruofan Liang, Kai He, Zan Gojcic, Igor Gilitschenski, Sanja Fidler, Nandita Vijaykumar, and Zian Wang. 2025. LuxDiT: Lighting Estimation with Video Diffusion Transformer. arXiv:2509.03680 [cs.GR]arxiv.org/abs/2509.03680
  21. [21]Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. 2024. Gs-ir: 3d gaussian splatting for inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 21644–21653.arxiv.org/abs/2311.16473
  22. [22]Ko Nishino and Shree K. Nayar. 2004. Eyes for relighting. ACM Trans. Graph. 23, 3 (Aug. 2004), 704–711. doi:10.1145/1015706.1015783DOI
  23. [23]Jeong Joon Park, Alexey Tewari, Hoon Suk, and Steven M. Seitz. 2020. Seeing the World in a Bag of Chips. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2001.04642
  24. [24]William Peebles and Saining Xie. 2023. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision. 4195–4205.arxiv.org/abs/2212.09748
  25. [26]Alexander Raistrick, Lingjie Mei, Karan Kayan, David Yan, Yiming Zuo, Beijing Han, Hongyu Wen, Meenal Parakh, Stamatis Alexandropoulos, Lahav Lipson, Zeyu Ma, and Jia Deng. 2024. InfiniGen Indoor: Photorealistic Indoor Scenes using Procedural Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 21783–21794.arxiv.org/abs/2406.11824
  26. [27]Erik Reinhard, Michael Stark, Peter Shirley, and James Ferwerda. 2002. Photographic tone reproduction for digital images. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (San Antonio, Texas) (SIGGRAPH ’02). Association for Computing Machinery, New York, NY, USA, 267–276. doi:10.1145/566570.566575DOI
  27. [29]Shuran Song and Thomas Funkhouser. 2019. Neural illumination: Lighting prediction for indoor environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6918–6926.arxiv.org/abs/1906.07370
  28. [30]Pratul P Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan T Barron. 2021. Nerv: Neural reflectance and visibility fields for relighting and view synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7495–7504.arxiv.org/abs/2012.03927
  29. [31]Pratul P Srinivasan, Ben Mildenhall, Matthew Tancik, Jonathan T Barron, Richard Tucker, and Noah Snavely. 2020. Lighthouse: Predicting lighting volumes for spatially-coherent illumination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8080–8089.arxiv.org/abs/2003.08367
  30. [33]Mutian Tong, Rui Wudi, and Changxi Zheng. 2025. Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priors. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Papers (SIGGRAPH Conference Papers ’25). Association for Computing Machinery, New York, NY, USA, Article 107, 11 pages. doi:10.1145/3721238.3730749DOI
  31. [34]Guangcong Wang, Yinuo Yang, Chen Change Loy, and Ziwei Liu. 2022. StyleLight: HDR Panorama Generation for Lighting Estimation and Editing. In Computer Vision – ECCV 2022, Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer Nature Switzerland, Cham, 477–492.arxiv.org/abs/2207.14811
  32. [35]Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. 2025. VGGT: Visual Geometry Grounded Transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5294–5306.arxiv.org/abs/2503.11651
  33. [36]Xuecan Wang, Shibang Xiao, and Xiaohui Liang. 2024. LightOctree: Lightweight 3D Spatially-Coherent Indoor Lighting Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4536–4545.
  34. [37]Zian Wang, Jonah Philion, Sanja Fidler, and Jan Kautz. 2021a. Learning Indoor Inverse Rendering With 3D Spatially-Varying Lighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 12538–12547.arxiv.org/abs/2109.06061
  35. [38]Zian Wang, Jonah Philion, Sanja Fidler, and Jan Kautz. 2021b. Learning indoor inverse rendering with 3d spatially-varying lighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 12538–12547.arxiv.org/abs/2109.06061
  36. [39]Yao Yao, Jingyang Zhang, Jingbo Liu, Yihang Qu, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. 2022. Neural incident light field for physically-based material estimation. In European conference on computer vision. Springer, 700–716.arxiv.org/abs/2203.07182
  37. [40]Renjiao Yi, Chenyang Zhu, Ping Tan, and Stephen Lin. 2018. Faces as Highlighting Probes via Unsupervised Deep Highlight Extraction. In Proceedings of the European Conference on Computer Vision (ECCV).arxiv.org/abs/1803.06340
  38. [41]Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision. 11975–11986.arxiv.org/abs/2303.15343
  39. [43]Jingsong Zhang, Kalyan Sunkavalli, Yannick Hold-Geoffroy, Sunil Hadap, Jonathan Eiseman, and Jean-François Lalonde. 2019. All-Weather Deep Outdoor Lighting Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  40. [45]Yiqin Zhao and Tian Guo. 2020. PointAR: Efficient Lighting Estimation for Mobile Augmented Reality. In Computer Vision – ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 678–693.arxiv.org/abs/2004.00006
  41. [46]Jingsen Zhu, Fujun Luan, Yuchi Huo, Zihao Lin, Zhihua Zhong, Dianbing Xi, Rui Wang, Hujun Bao, Jiaxiang Zhang, and Rui Tang. 2022. Learning-based inverse rendering of complex indoor scenes with differentiable monte carlo raytracing. In SIGGRAPH Asia 2022 Conference Papers. 1–8.arxiv.org/abs/2211.03017
  42. [47]Yongjie Zhu, Yinda Zhang, Si Li, and Boxin Shi. 2021. Spatially-varying outdoor lighting estimation from intrinsics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12834–12842.arxiv.org/abs/2104.04160

Paper details

vlnc:W_awcpdq7m
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents