betaFeedback

WorksComputer Science

A Lightweight Differentiable Method for Real-Time Skin Rendering and Digitization

Abstract

Inverse rendering for skin typically relies on offline path tracing to estimate physical properties, which fail to map directly to the approximate shaders utilized in interactive applications, such as game engines. We bridge this gap by directly estimating subsurface scattering parameters tailored for real-time rendering pipelines. We introduce a differentiable Screen-Space Subsurface Scattering (SSSS) framework that enables the joint optimization of spatially varying base textures and scattering distances. To account for local variations in skin thickness, we estimate these parameters across distinct facial regions, generating a blended profile map ready for production runtime engines. Our pipeline utilizes a low-cost, single-light, single-view capture setup designed for accessible and practical appearance acquisition. Optimization is driven by a masked loss function targeting regions where subsurface scattering effects are dominant. We validate our approach on a real-world dataset of 4545 individuals across the Monk Skin Tone Scale, ensuring demographic generalization. Furthermore, we complement this evaluation with synthetic data, a multi-light setup validating our single-light assumption, and a tilted-view configuration confirming angular consistency. Results demonstrate that our method effectively recovers scattering parameters from lightweight inputs, yielding realistic skin appearance directly compatible with standard real-time shaders.

main.pdf

Introduction

Accurately simulating human skin remains a significant challenge in computer graphics due to its physical complexity and the high sensitivity of the human visual system to facial appearance. As a layered, anisotropic, and spatially varying medium, skin involves intricate subsurface scattering, fine-scale geometric details, and directionally dependent light transport. To meet the computational constraints of interactive applications, the industry often relies on screen-space approximations rather than full volumetric path tracing. However, the perceptual sensitivity of observers ensures that even subtle rendering inaccuracies appear unnatural. Additionally, the global diversity of skin tones demands methods that generalize across the full spectrum of human pigmentation while maintaining both physical plausibility and real-time performance.

Traditional approaches for capturing skin optical properties often require sophisticated hardware, such as hyperspectral sensors or polarimetric imaging systems [1, 2, 3]. While these methods yield accurate results, they are typically expensive, restricted to controlled laboratory settings, and produce physical parameters (e.g., extinction coefficients) that do not map directly to real-time shaders. Furthermore, they often struggle to scale across the diverse facial expressions and skin tones encountered in real-world scenarios. To address these limitations, we introduce a low-cost, reliable single-view, single-light capture setup. Our compact configuration facilitates the practical acquisition of skin appearance without requiring specialized environments.

Three examples of captured skin appearances and strip-light captures

Figure 1. Our lightweight pipeline estimates spatially-varying subsurface scattering from single-image strip-light captures. Shown: rendered avatars across the Monk skin tone scale using optimized per-region scattering parameters.

Instead of estimating offline physical parameters like absorption or extinction coefficients through complex path-tracing intermediaries, we directly optimize the specific parameters used in real-time rendering pipelines. We ground our approach in the Bidirectional Subsurface Scattering Reflectance Distribution Function (BSSRDF), specifically utilizing the Burley Normalized Diffusion profile [4], the standard for real-time physically based skin shading. To enable optimization, we implement a differentiable version of the Screen-Space Subsurface Scattering (SSSS) algorithm incorporating this profile. This model provides control via the scattering distance parameter, which governs the distance light travels beneath the surface. Tuning this parameter manually is difficult due to its strong coupling with material appearance and the lack of direct visual references. Our approach automates this process by directly optimizing scattering distance textures from photographic data, enabling efficient and plausible appearance reconstruction.

Figure 2. Monk Skin Tone Scale color swatches [5]. Image credit: © Google, used under the CC BY 4.0 license.

Prior work frequently underexplores skin tone diversity, focusing primarily on mid-range tones and resulting in biased models that generalize poorly across lighter or darker skin. To address this limitation and drive representative skin modeling, we anchor our framework in the Monk Skin Tone Scale [5] (Fig. 2). Unlike traditional scales designed primarily for dermatological UV response (e.g., Fitzpatrick [6]), the Monk scale provides a perceptually balanced distribution covering the broad spectrum of human skin tones. Our primary goal is to leverage this scale to deliver a lightweight differentiable inverse rendering method that is physically grounded, equitable across diverse populations, and directly compatible with real-time production pipelines.

Our key contributions are:

  1. A lightweight, low-cost, single-light, single-view capture setup explicitly designed for the practical acquisition of facial skin patches across diverse skin tones.

  1. A differentiable rendering framework introducing a novel optimization stage to estimate state-of-the-art real-time scattering parameters directly from image data, bypassing complex path-tracing intermediaries.

  1. A masked loss formulation that isolates gradients within shadowed regions, where multiple scattering is physically dominant, ensuring accurate and stable optimization of subsurface parameters.

  1. A large dataset of calibrated facial skin patches and their estimated optical properties, sampled across the Monk Skin Tone Scale to drive equitable and representative skin modeling.

In this section, we review prior work on skin rendering, categorizing approaches into subsurface scattering approximations, biophysical and material models, physical acquisition methods, and inverse rendering strategies.

Subsurface Scattering Approximations. Realistic skin rendering requires simulating the complex transport of light through layers of varying opacity. As described by the BSSRDF [4], light transport in translucent media falls into three regimes: single scattering (thin materials), multiple scattering (thick materials), and intermediate scattering (turbid media) (Fig. 3). While full simulation is computationally expensive, most methods approximate the multiple scattering component to achieve visual plausibility [7]. The dipole model [8] remains a foundational approximation for offline rendering,

Illustration of subsurface scattering regimes

Figure 3. Illustration of subsurface scattering regimes. Left: a rendered ear where the yellow box highlights a single scattering dominant area (thin cartilage), while the green box shows a region dominated by multiple scattering, emphasizing the variability of light transport in human skin. Right: representation of single, intermediate, and multiple scattering. Adapted from [13].

modeling subsurface transport via virtual light source pairs. Subsequent improvements include the multipole method [9] for multi-layered materials and the directional dipole [10, 11], which introduces anisotropy to better capture detailed surface features. For higher precision, the quantized diffusion model [12] discretizes the diffusion process, offering finer control over scattering behavior.

For real-time pipelines, scattering is typically separated into surface and subsurface components. Translucency effects (single scattering) are often approximated using thickness maps generated via inverted-normal ambient occlusion [14]. For multiple scattering, screen-space techniques are the standard. [15] and [16] approximate diffusion by convolving irradiance maps with weighted Gaussian kernels, while [17] proposed an exponential diffusion profile matching Monte Carlo simulations. Recently, neural approaches such as [18, 19] have proposed encoding scattering profiles into lightweight networks, though screen-space convolution remains the industry standard for its predictability and direct integration with standard shading models.

Biophysical and Skin Material Models. Human skin exhibits significant spatial heterogeneity. Early measurement-based approaches [20] demonstrated that fixed scattering parameters fail to capture the variation between regions like the forehead, cheeks, and ears. [21] introduced layered models to account for epidermal and dermal variations, while [22] utilized region-based lookup tables, and [23] extended layered models to simulate cosmetics. Beyond empirical layers, recent biophysical models explicitly simulate physiological processes, such as sun-induced erythema and tanning [24], or multi-layer heterogeneous volume transport [25].

While volumetric biophysical models offer high biological fidelity, they require computationally demanding Monte Carlo solvers. In contrast, we utilize the Disney 2015 BSSRDF [4] as our parametric foundation. While the earlier Disney BRDF [26]—widely referred to in the industry as Burley 12—provided a robust model for surface reflection, it lacked volumetric scattering controls. The 2015 extension, commonly known as Burley 15, introduces a physically derived scattering distance parameter that ensures energy conservation while supporting anisotropic diffusion. This balance of physical plausibility and artist-friendly control makes Burley 15 ideal for our optimization pipeline. Inspired by [27], we optimize a scalar scattering distance per facial region under this model and blend them at runtime to achieve anatomical fidelity without sacrificing real-time efficiency.

Skin Appearance Acquisition. High-fidelity skin capture traditionally relies on complex hardware to separate reflectance components. Systems such as Light Stages [28], time-multiplexed lighting rigs [29], and polarization-based setups [30, 31] deliver ground-truth quality. Recent advances include spectral reconstruction methods [32] and single-shot polarized capture systems [33] that recover high-quality geometry and appearance. However, these methods require specialized equipment and controlled environments. In contrast, we focus on a lightweight acquisition regime to democratize skin reconstruction. Our differentiable framework optimizes scattering parameters from a single-view, single-light setup, eliminating the need for costly optical components or synchronized lighting arrays. Furthermore, this simplified pipeline remains robust across the full spectrum of human skin tones. This combination of hardware accessibility and tonal generalization provides a scalable solution for realistic digital human creation.

Inverse Rendering for Material Estimation. Material estimation generally falls into learning-based or optimization-based categories. Learning-based methods [34, 35, 36] train neural networks to infer parameters from images, offering fast inference but often lacking generalization to out-of-distribution examples. Optimization-based methods minimize the reconstruction error between observed and rendered images. For example, [37] optimized spatially varying parameters for fabrics using a differentiable rasterizer. In the domain of skin, [2] and [3] have successfully estimated parameters from hyperspectral data. Most closely related to our work is [38], which performs inverse rendering for textured translucent appearance. However, while [38] focuses on general translucent objects, our pipeline is specialized for human skin, incorporating domain-specific constraints (such as the Monk scale and facial region masks) and directly optimizing the screen-space parameters used in game engines. Furthermore, while [25] estimates biological parameters from a pre-extracted albedo map for offline volume rendering, our framework optimizes directly from raw photographs under directional lighting, decoupling surface reflection (albedo, roughness, specular) and subsurface scattering for real-time screen-space execution.

Overview and Rendering Model

This section outlines our proposed pipeline for skin appearance estimation and describes the real-time rendering techniques integrated into our engine to simulate subsurface scattering.

Overview

Our approach builds upon the diffusion-based skin model introduced by [17], which extends Burley 12 with a subsurface scattering parameter ss. We adopt Burley 15 model for its physical grounding and proven photorealism [37].

Unlike prior works that fix subsurface parameters to fit surface properties, we explicitly estimate the subsurface scattering distance that dominate skin appearance. We define the scattering distance as d=1sd = \frac{1}{s} (assuming a unit mean free path). We replace the standard uniform parameterization with an optimized homogeneous scattering parameter per facial region, which are then blended to create a spatially varying map for local anatomical variations.

We focus on the human face due to its perceptual importance in recognition [39] and the significant variation in skin thickness that influences light diffusion [40]. Consequently, our acquisition targets facial skin patches.

We utilize a single-view, single-light capture setup to extract skin data. To ensure representational diversity, we select subjects spanning the Monk Skin Tone Scale [5], capturing the optical variability caused by differing melanin and hemoglobin levels. Our proposed architecture proceeds in two sequential phases: first, we optimize a homogeneous scattering distance dd for each individual skin patch; second, we generate a full facial scattering map by blending these values based on anatomical regions, similar to the profile blending in [27]. To execute this initial patch-fitting phase, our inverse rendering pipeline directly optimizes a spatially varying Burley 15 texture stack by backpropagating reconstruction errors through a differentiable rasterizer, extending the core optimization framework from [37]. The complete pipeline is illustrated in Fig. 4.

This automated approach eliminates manual parameter tuning, driving skin rendering directly from photographic observations. Accordingly, the remainder of the paper details our acquisition protocol (Section 4.1), the tensorized scattering architecture (Section 4.2), implementation details (Section 5), and our experimental results and validation (Sections 6.1 and 6.2).

Rendering Model

We implement a custom real-time rendering engine using WebGL and Babylon.js [41]. This choice provides a broadly deployable, cross-platform verification environment that runs across a wide range of browsers and devices, including resource-constrained and low-end platforms, allowing us to assess whether the fitted scattering distances produce realistic and physically consistent appearances under practical cross-device rendering constraints. In addition to visualizing the final assets, the engine acts as a synthetic data generator, producing reference images with known ground-truth parameters to validate our optimization procedure.

The shading model relies on two primary inputs: a thickness texture and a spatially varying scattering distance texture. In the absence of accurate data textures (measured or artist-painted), local thickness is approximated automatically via inverted-normal SSAO [14]. The idea is to flip the surface normal, so that the AO hemisphere points into the object instead. Nearby surfaces encountered by inward-facing samples then provide a rough indication of how close other parts of the object are. When inward-facing polygons are close to one another, they generate more occlusion; this is interpreted as the region being locally thinner. To encode local scattering behavior, we generate the scattering distance texture via profile blending. Homogeneous scattering values optimized from patches are interpolated using the thickness texture (Fig. 5a) as a blending weight. This produces a smooth, continuous map (Fig. 5c) that accounts for thinner skin areas (e.g., ears, nose) without requiring unique profiles per pixel. Optionally, an anatomical region map (Fig. 5b) can be supplied to assign distinct scattering baselines to specific zones.

Overview of inverse skin rendering pipeline

Figure 4. Overview of our inverse skin rendering pipeline: we optimize Burley 15 texture parameters and a homogeneous scattering distance using a differentiable rasterizer, then generate spatially varying subsurface profiles via profile blending for final rendering.

Figure 5. Texture attachments of our rendering engine: (a) thickness, (b) region, and (c) blended scattering distance. For the region texture (b), the red channel encodes discrete region tags, while the green channel stores the interpolation factor for scattering distance blending.

Scattering Distance Scaling. The scattering distance texture is parameterized in metric units rather than arbitrary scalar weights. Following the convention of real-time engines such as Unreal Engine 5, the normalized texture domain [0, 1] maps linearly to a scattering distance range of [0, 1.0] cm. To align this with our engine’s world-space coordinate system (measured in meters), we apply a fixed dimensional conversion factor of 0.01 (1.0 cm = 0.01 m). Unlike screen-space approximations that rely on projected pixel distances, we compute the true 3D Euclidean distance between the surface point and the sample point using the depth buffer. This ensures that the diffusion profile remains physically consistent regardless of camera distance or field of view.

Multiple Scattering. We simulate multiple scattering using the screen-space approach proposed by [42], which integrates the diffusion profile by distributing samples within a 2D screen-space disk and reconstructing the 3D surface geometry under perspective projection. We employ the exponential approximation of the Burley profile:

R(r,d)=Ae−r/d+e−r/3d8πdr(1)R(r,d) = A \frac{e^{-r/d} + e^{-r/3d}}{8\pi d r} \tag*{(1)}

where AA is the diffuse volume albedo, rr represents the 3D Euclidean distance, and dd is the scaled scattering distance inherent to the material properties. This formulation simplifies light transport through layered skin into a dual-exponential profile over a single homogeneous volume. Because it does not model individual epidermal and dermal layers, the recovered parameter dd represents an effective parameter for the real-time shader rather than an intrinsic scattering coefficient (σs\sigma_s).

To minimize Monte Carlo variance, these samples are generated in polar coordinates (θi,ri\theta_i, r_i) via an analytical Importance Sampling scheme. The resulting Probability Density Function (PDF) p(r)p(r) and Complementary Cumulative Distribution Function (CCDF) Pˉ(r)\bar{P}(r) derived from the diffusion profile are defined as:

p(r)=rR(r,d)A,Pˉ(r)=14e−r/d+34e−r/3d(2)p(r) = \frac{r R(r,d)}{A}, \qquad\bar{P}(r) = \frac{1}{4}e^{-r/d} + \frac{3}{4}e^{-r/3d} \tag*{(2)}

According to [42], p(r)p(r) models the linear profile shape along a radial ray; the full 2π2\pi circle integration factor is explicitly accounted for during numerical CCDF inversion and applied within the final diffuse irradiance estimator (2π/N2\pi/N). The sampling radius ri∈[0,rmax⁡]r_i \in[0,r_{\max}] is generated by inverting the CCDF in Eq. (2) via inverse transform sampling. The angular coordinate θi\theta_i follows a Fibonacci pattern [43] (Figure 6a) combined with a stochastic rotational jitter to convert structured aliasing into high-frequency temporal noise.

Figure 6. Left: Fibonacci sampling pattern used for importance sampling the diffusion profile. Right: Chromatic fringing observed in captures due to spectral dependencies in the projector light.

For a target fragment with view-space position x∈R3\mathbf{x} \in\mathbb{R}^{3}, the local 2D polar offset is projected to determine its corresponding screen-space coordinate ui=[ui,vi]T\mathbf{u}_i = [u_i, v_i]^T. The true 3D view-space position of this surface sample, xi\mathbf{x}_i, is reconstructed directly by applying the standard inverse perspective projection matrix to the normalized screen coordinates and the depth map D(ui)D(\mathbf{u}_i).

The true 3D Euclidean distance through the scattering medium is subsequently evaluated as si=∥x−xi∥2s_i = \lVert\mathbf{x} - \mathbf{x}_i \rVert_2. Following the bilateral formulation in [42], we evaluate a spectral diffusion weight ki,ck_{i,c} for each color channel c∈{R,G,B}c \in\{R, G, B\} that balances the 3D radial profile response against the 2D sampling density:

ki,c=si⋅R(si,dc)ri⋅R(ri,rmax⁡)(3)k_{i,c} = \frac{s_i \cdot R(s_i,d_c)}{r_i \cdot R(r_i,r_{\max})} \tag*{(3)}

where dcd_c denotes the material scattering distance for the respective spectral channel, and rmax⁡r_{\max} represents the global boundary domain limit.

The final scattered diffuse radiance Ic(x)I_c(\mathbf{x}) is obtained via a normalized weighted average of the incident diffuse irradiance field Lc(ui)L_c(\mathbf{u}_i) across all NN screen-space samples:

Ic(x)=Ac(∑i=1Nki,cLc(ui)∑i=1Nki,c)(4)I_c(\mathbf{x}) = A_c \left(\frac{\sum_{i=1}^{N} k_{i,c} L_c(\mathbf{u}_i)}{\sum_{i=1}^{N} k_{i,c}}\right) \tag*{(4)}

where AcA_c is the channel albedo, and Lc(ui)L_c(\mathbf{u}_i) represents the standard Lambertian incident diffuse irradiance sampled at the projected screen-space coordinate ui\mathbf{u}_i. To ensure full reproducibility, the implementation details of the shader are provided in Section 5.3 of the Supplementary Material.

Single Scattering. For a complete skin shader, we also account for single scattering (translucency) through thin geometry (e.g., ears). We implement the approximation proposed by [14], which estimates light transmission using the Beer-Lambert law and the thickness texture. As our optimization framework focuses exclusively on the dominant multiple scattering component, we employ this term solely for visual completeness in the final renders (Figure 7).

Figure 7. Final render produced with our engine (a). Insets highlight regions dominated by (b) multiple scattering and (c) single scattering.

Skin Fitting Pipeline

Acquisition

Estimating physically meaningful texture maps—such as normals, roughness, albedo, and scattering distances—from skin images typically demands complex acquisition systems. Common approaches rely on multi-view setups, polarized lighting, or hyperspectral imaging [2, 3, 1]. While accurate, these systems suffer from limited scalability and high deployment costs. Furthermore, spatially varying estimation is highly sensitive to lighting directionality, local geometry, and microstructural detail [21, 44, 30], making replication outside controlled laboratories difficult.

To address these limitations, we propose an acquisition setup that balances simplicity with fidelity and is lightweight in both hardware and operational complexity compared to such laboratory rigs. Rather than requiring custom optical components or electronic synchronization, our system relies entirely on consumer-grade devices, eliminates complex geometric and radiometric calibration protocols, and is easily deployed in standard indoor environments, specifically in a darkened room to eliminate ambient light interference. As shown in Figure 8a, the setup consists of:

Hardware acquisition setup and illumination patterns

Figure 8. Hardware acquisition setup and illumination patterns. (a) Physical configuration (left) and corresponding schematic ray-diagram (right) illustrating the scanning system geometry. (b) Temporal sequence of projected light patterns used during data acquisition: ambient baseline, structured stripes, inverted stripes, and uniform flat-field illumination.

  1. Subject support. A rigid horizontal platform stabilizes the target surface. Participants place a specific facial region (e.g., cheek or forehead) on a fixed marker to ensure positional repeatability.

  1. Illumination source. A projector is used to emit both high-contrast structured line patterns and uniform white light. It is positioned approximately 20 cm from the subject, tilted upwards at 30∘30^\circ relative to the surface plane. This low-angle configuration creates strong directional lighting, maximizing the visibility of subsurface scattering effects while allowing for controlled binary illumination.

  1. Imaging system. A high-resolution RGB camera (DSLR) is mounted behind the projector at a distance of approximately 50 cm from the subject. This near-coaxial alignment minimizes geometric distortion and allows the direct capture of reflectance and shading cues under controlled illumination.

Although we target a near-coaxial alignment, our pipeline ensures robustness against angular deviations. To maintain a lightweight workflow, we avoid complex projector-camera calibration and structured-light triangulation, relying instead on monocular depth estimation [45]. We map the network’s normalized output (0 for the near plane, 1 for the far plane) to a physical depth range of 2 cm that bounds the local facial relief, anchored at a measured camera-to-subject distance of 47 cm. All geometric and exposure settings are fixed to maintain consistency across subjects (Section 5).

Despite its simplicity, this setup successfully captures the subtle shading gradients and scattering cues required for material estimation, avoiding the cost and complexity of multi-camera or hyperspectral systems [21, 2]. While specialized hardware offers higher raw fidelity, we demonstrate that optimized differentiable rendering can extract production-quality assets from this minimal input.

We acquire four photographs per skin patch under distinct binary illumination patterns (Figure 8b). We utilize vertically aligned line patterns to minimize projector-induced chromatic aberration and color fringing (Figure 6b). Unlike the complex patterns conventional in structured-light reconstruction [46, 47, 48], our simplified linear patterns reduce spectral interference—a critical consideration given the wavelength-dependent nature of subsurface scattering. Figure 9 details the sequence:

Figure 9. Localized view of the captured raw frame sequence during acquisition. (a) Ambient baseline frame (brightness-enhanced for visibility) showcasing residual environmental light. (b) Active surface capture under the high-frequency structured stripe pattern. (c) Corresponding active capture under the inverted stripe pattern. (d) Fully illuminated frame under uniform flat-field projection.

  • Photo 1 (9a): Captured with a black pattern to record ambient residuals. This image is subtracted from subsequent captures to isolate direct illumination.

  • Photos 2 & 3 (9b, 9c): Captured using complementary vertical line patterns. These images highlight the skin’s directional response.

  • Photo 4 (9d): Captured under full white illumination. This provides a uniformly lit reference for estimating global properties like albedo, decoupling shading from geometric variation.

To isolate subsurface scattering effects, we generate a light mask by computing the difference between the patterned image (9b) and its inverse (9c). We binarize this difference map using Otsu’s thresholding [49], which automatically determines an optimal threshold by maximizing the between-class variance of the intensity distribution. To ensure spatial

coherence, we apply morphological dilation followed by opening and closing operations. This process produces two distinct versions of the mask to balance optimization accuracy with visual reconstruction:

  1. Loss Mask: An inverted, strictly binary shadow mask M\mathcal{M} (where 1 corresponds to shadowed regions) used to focus the optimization loss. Maintaining a hard binary edge here is critical; any blurring of the mask during optimization would alter the edges of the illumination pattern and corrupt the scattering halo estimation.

  1. Rendering Mask: A non-inverted light mask Msmooth\mathcal{M}_{\mathrm{smooth}} (where 1 corresponds to lit regions) smoothed via an 11 × 11 Gaussian filter. This blur emulates projector defocus and prevents hard-edge artifacts in the final render, ensuring a smooth transition where the specular component is added to the masked diffuse component.

The resulting preprocessed images and the depth map (Figure 10) serve as the input for our differentiable rasterizer optimization.

Optimization input data evaluated across different anatomical skin regions

Figure 10. Optimization input data evaluated across different anatomical skin regions (rows). From left to right: (a) Uniformly illuminated reference frames, (b) active strip-lit captures displaying subsurface scattering effects, (c) light masks, and (d) estimated monocular depth maps.

Tensorized SSSS

Porting the screen-space multiple scattering algorithm to a differentiable framework presents a challenge: we must convert per-pixel shader operations into batched tensor computations to ensure efficiency and parallelization. This requires careful handling of texture sampling, coordinate projection, and memory management to maintain differentiability.

Key aspects of this transformation include:

  1. Batch Processing – Instead of iterating per pixel, we process the entire image and all scattering samples simultaneously as high-dimensional tensors.

  1. View-Space Reconstruction – To ensure 1:1 consistency with the rendering engine, we reconstruct the 3D position of every pixel using depth and camera intrinsics, rather than relying on 2D screen-space approximations.

  1. Differentiable Sampling – We replace discrete texture lookups with differentiable bilinear interpolation, allowing gradients to propagate through the scattering inputs.

  1. Memory Optimization – We broadcast operations across dimensions to minimize loop overhead, a critical requirement for performance in tensorized frameworks.

The pipeline begins by computing the diffuse irradiance tensor LT\mathbf{L}_T modulated by the smoothed rendering mask Msmooth\mathcal{M}_{\mathrm{smooth}} to filter the incoming illumination before diffusion:

LT=Msmooth⊙(n⋅l)+,\mathbf{L}_T = \mathcal{M}_{\mathrm{smooth}} \odot(\mathbf{n} \cdot\mathbf{l})_+,

where n\mathbf{n} and l\mathbf{l} denote the surface normal and light direction tensors.

In a standard shader, diffusion is computed via a loop over NN samples for each pixel. Imitating this loop in a tensor framework is inefficient. Instead, we adopt a fully vectorized approach where the entire image is replicated NN times, creating tensors of shape (N,H,W,C)(N, H, W, C). This allows us to apply the scattering kernel across all pixels and samples simultaneously (Figure 11). The process is as follows:

Comparison of the shader-based approach and tensorized scheme

Figure 11. Comparison of the shader-based approach (left) vs. our tensorized scheme (right). By broadcasting the image across the sample dimension NN, we perform view-space reconstruction and integration in a single parallelized pass.

We define a virtual camera matching the real-world acquisition system (focal length 50 mm, sensor width 23.5 mm). Using the estimated depth map (Figure 10d), we unproject every pixel into 3D view-space coordinates Pv\mathbf{P}_v. To avoid high-dimensional matrix inversions during backpropagation, we linearize depth and unproject coordinates component-wise:

z=P23(2⋅DT−1)−P22(5)z = \frac{P_{23}}{(2 \cdot D_T - 1) - P_{22}} \tag*{(5)}
xv=U⊙zP00,yv=V⊙zP11(6)x_v = \frac{\mathbf{U} \odot z}{P_{00}}, \qquad y_v = \frac{\mathbf{V} \odot z}{P_{11}} \tag*{(6)}

where Pv=[xv,yv,−z]T\mathbf{P}_v = [x_v, y_v, -z]^T, the tensors U,V\mathbf{U}, \mathbf{V} represent the normalized grid coordinates, and P00,P11,P22,P23P_{00}, P_{11}, P_{22}, P_{23} are the perspective projection coefficients derived from the camera geometry and clipping planes. This explicit tensorized reconstruction eliminates the need for 2D pixel-distance approximations, ensuring pixel-level geometric alignment with the WebGL forward rasterizer while decoupling the simulation from the fixed image plane for robust angular consistency, as validated in § 8.

We utilize the precomputed sampling tensors samplesθ\mathrm{samples}_{\theta} and samplesr\mathrm{samples}_{r}, representing the Fibonacci angles and stratified Burley Inverse CCDF normalized radii, respectively. The angles are shifted by the stochastic jitter pattern, while the normalized radii are scaled by the maximum scattering distance rmax⁡r_{\max}. These updated polar coordinates are then projected to obtain the spatial offsets (xo,yo)(x_o, y_o).

We apply these offsets to Pv\mathbf{P}_v in view space and project the resulting positions back into Normalized Device Coordinates (NDC) to form a sampling grid. This perspective projection is defined component-wise as:

Us=(xv+xo)⋅P00z,Vs=(yv+yo)⋅P11z(7)U_s = \frac{(x_v + x_o) \cdot P_{00}}{z}, \qquad V_s = \frac{(y_v + y_o) \cdot P_{11}}{z} \tag*{(7)}

where U\mathbf{U} and V\mathbf{V} are stacked to form the final 2D sampling coordinate grid G\mathbf{G}. To accurately replicate the hardware-accelerated bilinear filtering used in standard shaders, we

retrieve the incident irradiance (LT\mathbf{L}_T) and depth (DT\mathbf{D}_T) at these coordinates using grid sampling with bilinear interpolation. This ensures our optimization kernel mathematically matches the runtime renderer while maintaining full differentiability.

We calculate the true 3D Euclidean distance between the surface point Pv\mathbf{P}_v and the sampled point Ps\mathbf{P}_s. This distance is fed into the Burley diffusion profile to compute the scattering weight (Eq. (3)). The weighted samples are summed along the sample dimension (NN) to produce the integrated diffuse irradiance tensor.

Finally, this integrated diffuse irradiance is multiplied element-wise by the albedo tensor AT\mathbf{A}_T to obtain the final scattered diffuse radiance component. For visualization during optimization, we add the specular lobe, modulated by the rendering mask Msmooth\mathcal{M}_{\mathrm{smooth}}, to produce the complete rendered image. To ensure full reproducibility, the tensorized implementation is provided in Section 5.4 of the Supplementary Material.

Implementation details

Capture Hardware To capture the input data, we utilize a Sony α\alpha6100 (ILCE-6100L), a consumer-grade camera that balances affordability with high image quality. Its high-resolution sensor and reliable low-light performance make it an effective instrument for acquiring precise material data, while its accessible price point ensures framework reproducibility. The camera is connected via USB to a PC and remotely controlled using the Sony Imaging Edge desktop application, which provides a live view, full manual parameter control, and automated interval shooting. All images were captured using fixed camera parameters to ensure strict consistency across the dataset. Specifically, we established a resolution of 6020 ×\times 4024 pixels, an ISO of 100, an exposure time of 1/20 s, an aperture of f/20f/20, a white balance of 6000 K, and a 50 mm focal length.

Illumination is provided by an Optoma projector rated at 3500 lumens, which projects the sequence of structured light patterns. We synchronize the camera and projector using a combination of Sony Imaging Edge and an AutoHotkey script. While Imaging Edge manages the timed shooting in continuous mode, the AutoHotkey script coordinates each projection frame change with the camera’s shutter trigger, ensuring precise temporal synchronization between pattern display and image acquisition.

Input processing The RAW captures (.ARW) are decoded using rawpy and subsequently center-cropped to 480×360 px to satisfy GPU memory constraints. To ensure chromatic accuracy, we calibrate the raw sensor space to linear sRGB using a 3 × 3 Color Correction Matrix (CCM) computed via Ridge Regression on an X-Rite ColorChecker, normalizing pixel intensities to the [0, 1] range. For complete processing and calibration details, see § 14.1 of the Supplementary Material.

Optimization details Our pipeline optimizes a subset of material property tensors from a single view under a single light source, specifically, the spatially varying albedo AT\mathbf{A}_T and roughness ρT\rho_T, alongside the homogeneous scattering distance ST\mathbf{S}_T. Due to the limited geometric cues of this underconstrained setup, we do not optimize surface normals and tangents, relying instead on a fixed Sobel-filtered map derived from a pre-estimated depth map [45]. To comply with commercial production standards, the parameter ST∈[0,1]\mathbf{S}_T \in[0, 1] (encoding up to 1.0 cm) maps to world-space distance in meters via the metric conversion d=ST×0.01d = \mathbf{S}_T \times0.01, where doptd_{\mathrm{opt}} represents the resulting homogeneous scattering color texture map. Achieving a clean separation of the diffuse albedo AT\mathbf{A}_T is critical to avoid bias during the scattering optimization; since the Burley diffusion profile (Eq. 1) scales directly with A\mathbf{A}, any specular energy incorrectly baked into the albedo would artificially shift its chromatic tone. To capture these high-frequency highlights without corrupting the core material parameters, we introduce an auxiliary residual specular tensor Δspec,T\Delta_{\mathrm{spec},T}. This tensor functions as a temporary residual buffer that absorbs specular discrepancies during optimization, mitigating their bleeding into the base material maps, and is discarded after convergence.

Textures are initialized using physically informed priors: the unwarped IOR map is initialized at a uniform value of 0.145 (linearly mapping to a physical IOR ≈1.44\approx1.44), and the homogeneous scattering distance map ST\mathbf{S}_T begins at a uniform 0.5. To isolate a clean skin albedo independent of lighting, hair, or deep shadows, we perform kk-means clustering (k=2k = 2) on the a∗a^* and b∗b^* chromatic channels in CIE LAB space. This adaptive mask is refined by excluding the brightest 10% and darkest 5% of pixels. A robust mean color is then computed via 2σ2\sigma Sigma Clipping on the remaining pixels to provide a stable initialization baseline (see § 14.2 of the Supplementary Material for complete algorithmic details).

We employ a multi-stage optimization strategy using the Adam optimizer [50]. The objective function balances a masked L1\mathcal{L}_1 reconstruction loss with an L2\mathcal{L}_2 roughness regularization prior:

L=1∣Ω∣∥I^⊙Mopt−I⊙Mopt∥1+λprior1∣Ω∣∥ρT−0.6∥22(8)\mathcal{L} = \frac{1}{|\Omega|} \left\lVert\hat{\mathbf{I}} \odot\mathcal{M}_{\mathrm{opt}} - \mathbf{I} \odot\mathcal{M}_{\mathrm{opt}} \right\rVert_1 + \lambda_{\mathrm{prior}} \frac{1}{|\Omega|} \left\lVert\rho_T - 0.6 \right\rVert_2^2 \tag*{(8)}

where I\mathbf{I} is the captured reference tensor, Mopt\mathcal{M}_{\mathrm{opt}} denotes the stage-dependent optimization mask and ∣Ω∣|\Omega| is the total number of elements within the active spatial domain. The L2\mathcal{L}_2 value prior minimizes the transfer of high-frequency albedo details into the roughness map under single-view constraints, stabilizing the reflection envelope without suppressing regional variability. The anchor value of 0.6 corresponds to the default skin roughness baseline adopted by industry-standard real-time engines. Within this framework, the complete rendered image is formulated as a differentiable function I^=f(AT,ρT,Δspec,T,ST)\hat{\mathbf{I}} = f(\mathbf{A}_T, \rho_T, \Delta_{\mathrm{spec},T}, \mathbf{S}_T), whose active parameter dependencies, hyperparameter weights, and spatial domains adapt dynamically according to the multi-stage schedule outlined in Table 1 and detailed below:

ConfigurationStage 1Stage 2Stage 3
Optimized ParametersΔspec,T,ρT\Delta_{\mathrm{spec},T}, \rho_TAT,ρT,Δspec,T\mathbf{A}_T, \rho_T, \Delta_{\mathrm{spec},T}ST\mathbf{S}_T
Frozen ParametersAT,ST\mathbf{A}_T, \mathbf{S}_TST\mathbf{S}_TAT,ρT,Δspec,T\mathbf{A}_T, \rho_T, \Delta_{\mathrm{spec},T}
Iterations700500600
Learning Rate5×10−35 \times10^{-3}5×10−35 \times10^{-3}1×10−31 \times10^{-3}
Active Mask (Mopt\mathcal{M}_{\mathrm{opt}})Full Image (1)Full Image (1)Shadow Mask (M\mathcal{M})
Roughness Prior (λprior\lambda_{\mathrm{prior}})1×10−21 \times10^{-2}1×10−21 \times10^{-2}None

Table 1. Optimization Schedule and Hyperparameters.

  • Stage 1 (Specular & Roughness Pre-fit): Parameters Δspec,T\Delta_{\mathrm{spec},T} and ρT\rho_T are optimized; AT\mathbf{A}_T and ST\mathbf{S}_T are frozen. This stage fits global reflection, capturing unmodeled specular highlights into Δspec,T\Delta_{\mathrm{spec},T}. Freezing albedo preserves its clustering initialization, shielding base color from early lighting errors and cross-parameter interference.

  • Stage 2 (Surface Maps Refinement): Parameters AT\mathbf{A}_T, ρT\rho_T, and Δspec,T\Delta_{\mathrm{spec},T} are optimized; ST\mathbf{S}_T is frozen. Unfreezing AT\mathbf{A}_T absorbs diffuse details, allowing the L2\mathcal{L}_2 prior to smooth ρT\rho_T. Retaining active Δspec,T\Delta_{\mathrm{spec},T} prevents residual specular discrepancies from being incorrectly baked into albedo.

  • Stage 3 (Subsurface Scattering Fit): Parameter ST\mathbf{S}_T is optimized; AT\mathbf{A}_T, ρT\rho_T, and Δspec,T\Delta_{\mathrm{spec},T} are frozen. This isolates the low-frequency subsurface scattering pass from high-frequency surface variations, preventing the coupling inherent to Eq. (1) between linear albedo scaling and non-linear decay dd (verified synthetically in Figure 15 and Figure 16). Restricting the loss domain Mopt\mathcal{M}_{\mathrm{opt}} to the shadow mask M\mathcal{M} forces the optimizer to fit the multiple scattering halos exclusively, removing interference from surface reflections and preventing direct lighting discrepancies from skewing dd (Figure 18).

Synthetic validation using known ground truth

Figure 15. Synthetic validation using known ground truth. drefd_{\mathrm{ref}} denotes the ground-truth scattering distance, and doptd_{\mathrm{opt}} is the optimized value. The near-identical maps confirm the numerical self-consistency of dd under the inverse rendering formulation.

Synthetic validation across the Monk Scale

Figure 16. Synthetic validation across the Monk Scale. We fix scattering distance and vary albedo (Monk 01, 05, 10). The low perceptual error (FLIP) and consistent parameter recovery confirm robustness to albedo variation.

Ablation of Stage 3 loss supervision domain

Figure 18. Ablation of Stage 3 loss supervision domain. Comparing unmasked optimization (Mopt=1\mathcal{M}_{\mathrm{opt}} = \mathbf{1}, top) against our shadow-masked loss (Mopt=M\mathcal{M}_{\mathrm{opt}} = \mathcal{M}, bottom). Supervising unlit regions isolates subsurface halos, preventing direct light from skewing the scattering distance doptd_{\mathrm{opt}} (insets) and reducing FLIP error across both the full patch and masked shadow regions.

This schedule is held constant across all captured patches and skin tones, requiring no per-subject hyperparameter tuning. The learning rates, iteration counts, and prior weight for surface estimation (Stages 1 and 2) build upon established baseline configurations [37], with the residual specular map Δspec,T\Delta_{\mathrm{spec},T} sharing the same learning rate (5×10−35 \times10^{-3}). For Stage 3, we reduce the learning rate to 1×10−31 \times10^{-3} over 600 iterations to ensure stable convergence against the lower gradient magnitudes of subsurface scattering, particularly across darker skin tones where the signal-to-noise ratio is lower. The progressive evolution of material maps across these stages is illustrated in Section 2 of the Supplementary Material.

Results

We evaluate our method qualitatively and quantitatively on both real and synthetic data to demonstrate its reconstruction fidelity, computational efficiency, and robustness across diverse skin tones. Our evaluation is structured into two main parts: (1) an analysis of fitting results on real facial patches across the Monk Skin Tone Scale, evaluating perceptual metrics, runtime benchmarks, and material plausibility (§ 7.1); and (2) a comprehensive validation study verifying parameter isolation via synthetic ground truth, assessing our shadow-masked loss through ablation, and benchmarking our single-light setup against a multi-light configuration (§ 8). Validation of angular consistency via tilted-view experiments is provided in Section 3 of the Supplementary Material.

We built a dataset of 225 real skin patches from 45 participants across five facial regions using a single-light setup. A subset of 17 subjects (85 samples) was additionally captured using a multi-light setup for validation.

Ethics, consent, and data handling. The acquisition protocol was reviewed and approved by the Research Ethics Committee of our institution prior to data collection. All participants received written information describing the purpose and procedures of the study and provided written informed consent prior to participation, including consent for the processing of their personal data. Participation was voluntary, and participants could withdraw their consent at any time without adverse consequences. Skin images were associated with generic participant identifiers, while the correspondence between these identifiers and the contact information of the participants was stored separately under restricted access available only to the research team. Contact information was used solely when a participant needed to be contacted for a repeated acquisition. Only non-identifying skin-patch images were used for scientific dissemination, and any additional compatible research use requires prior anonymization. Personal data were handled according to applicable international regulations.

Fitting Results

We apply our inverse rendering pipeline to real facial skin patches sampled across the Monk Skin Tone Scale. We evaluate reconstruction quality using the perceptual metric FLIP [51], computed over masked regions to isolate the subsurface scattering halo. Figure 12 shows representative examples from light, medium, and dark skin tones. The reconstructions closely match the input photographs, particularly in the shadowed transition regions where scattering dominates.

Reconstruction results on real facial skin patches across the Monk Skin Tone Scale

Figure 12. Reconstruction results on real facial skin patches across the Monk Skin Tone Scale. Rows display different facial regions: neck (top), cheek (middle), and nose (bottom). Columns show (left to right): the captured input image with strip lighting, the rendered reconstruction, the masked input image, the masked render, and the masked perceptual error map (FLIP). The optimized scattering distance for each patch is indicated as doptd_{\mathrm{opt}}.

We report complementary metrics—LPIPS [52], SSIM [53], and PSNR—in Table 2. While SSIM is sensitive to luminance variations in shadowed regions (resulting in lower scores), perceptual metrics like FLIP and LPIPS yield consistently low error values. This supports the perceptual plausibility of our reconstructions across the full skin tone spectrum.

Monk RangeFLIP ↓LPIPS ↓SSIM ↑PSNR ↑
Monk 1-30.093±0.0260.093 \pm0.0260.086±0.0200.086 \pm0.0200.615±0.0660.615 \pm0.06628.61±1.6828.61 \pm1.68
Monk 4-60.093±0.0310.093 \pm0.0310.088±0.0180.088 \pm0.0180.614±0.0660.614 \pm0.06628.67±1.8928.67 \pm1.89
Monk 7-100.077±0.0210.077 \pm0.0210.115±0.0150.115 \pm0.0150.503±0.0720.503 \pm0.07229.65±1.3529.65 \pm1.35

Table 2. Quantitative reconstruction errors across the Monk Scale, computed on masked image regions.

In addition to perceptual quality, we benchmark the computational efficiency of our pipeline. Because all skin patches are standardized to 480 × 360 px and evaluated under our fixed multi-stage schedule, runtime is deterministic across subjects. Table 3 details the stage-wise computational breakdown on an NVIDIA GeForce RTX 4070 Ti (12 GB). Inverse optimization converges in under 3.5 minutes (198.4 s) per patch with a peak footprint of 9.4 GB during Stage 3 integration, confirming that our pipeline enables fast and accessible digitization on consumer desktop hardware.

MetricStage 1Stage 2Stage 3Overall
Execution Time (s)51.138.3109.0198.4
Iterations7005006001800
Time per Iteration (ms)72.976.6181.7110.2
Peak VRAM (GB)2.52.69.49.4

Table 3. Computational cost per skin patch (480 × 360 px, N=128N = 128 scattering samples) across optimization stages on an AMD Ryzen 9 7900 CPU (12 cores, 24 threads) and an NVIDIA GeForce RTX 4070 Ti GPU. The Overall column reports cumulative values for time and iterations, mean iteration latency, and maximum peak VRAM.

We visualize the optimized Burley 15 material stack alongside the auxiliary residual specular map in Figure 13.

Optimized Burley 15 material stack across different skin tones

Figure 13. Optimized Burley 15 material stack across different skin tones from single-view, single-light capture. Columns show (left to right): full-light input image, estimated albedo, roughness and residual specular.

The estimated textures are spatially smooth and consistent across the dataset. Utilizing a monocular depth-based normal prior, rather than relying on photometric cues, regularizes the single-view optimization, mitigating parameter instability across complex structures like the ear. While this depth prior acts as a low-frequency approximation that lacks sharp pore-level details—yielding smoother roughness maps—and screen-space shading partially bakes self-shadows into albedo across steep relief (e.g., ear folds), our multi-light validation confirms the scattering parameter dd converges consistently. This demonstrates that subsurface optimization remains effectively decoupled from localized shadowing and micro-geometric inaccuracies.

Crucially, the estimated scattering distance maps exhibit intuitive trends across the skin tone spectrum. As illustrated in Figure 14, darker skin tones correspond to shorter scattering distances, consistent with the reduced subsurface light transport in melanin-rich tissue. These findings demonstrate the physical plausibility of our RGB-based optimizations without the need for hyperspectral data. Furthermore, our masking strategy effectively focuses supervision on shadowed regions, improving multiple scattering estimates in darker skin where the effect is subtle but directionally dependent.

Radial scattering profiles for light, medium, and dark skin tones

Figure 14. Radial scattering profiles plotted as rR(r)rR(r) in logarithmic space to visualize falloff and effective range. From left to right: light, medium, and dark skin tones. Curves show channel-wise reflectance decay (RGB) alongside estimated albedo. The trend of decreasing scattering distance with darker skin tones confirms the physical plausibility of our optimization.

Validation

Validation on Synthetic Skin Patches We validate our pipeline using synthetic skin patches rendered under known Burley parameters [4] with an identical sampling budget (N=128N = 128) to ensure numerical consistency during inverse optimization. In the first experiment, we fix all surface attributes (albedo, normals, roughness, IOR) and optimize only the scattering distance. This isolates the scattering estimation stage. As shown in Figure 15, the estimated scattering maps (doptd_{\mathrm{opt}}) closely match the reference ground truth (drefd_{\mathrm{ref}}), and the rendered reconstructions are visually indistinguishable from the inputs.

In the second experiment, we fix the scattering distance and vary the albedo across the ten synthetic tones of the Monk Scale to test robustness to pigmentation changes. While Figure 16 shows a representative subset, Section 4 of the Supplementary Material details the full 10-tone evaluation alongside the real-world dataset distribution (N=45N = 45) cross-referenced against participant self-assessments.

Specular Isolation Validation Our pipeline employs a residual specular map to absorb high-frequency highlights, mitigating their leakage into the albedo map. To validate this strategy, we optimize a synthetic scene where the ground-truth decomposition between albedo and specular reflection is known. Figure 17 compares the optimized residual map against the ground-truth specular contribution. The close resemblance confirms that our robust mean albedo computation effectively shields the diffuse component, allowing the residual map to accurately capture the specular response. Furthermore, this separation minimizes the risk of unmodeled specular energy corrupting other surface attributes, directly validating why our framework achieves highly stable and smooth roughness maps with suppressed artifact bleeding.

Validation of specular isolation

Figure 17. Validation of specular isolation. We compare the ground-truth specular contribution (Reference) against the optimized residual specular map (Render). The low error demonstrates that our albedo initialization enables the optimizer to effectively capture and absorb specular reflections within the residual map, mitigating their contamination of the base material parameters.

Masked Loss Validation To isolate the contribution of our shadow-masked loss during Stage 3 (Mopt=M\mathcal{M}_{\mathrm{opt}} = \mathcal{M}), we performed an ablation study comparing this formulation against unmasked full-image loss supervision (Mopt=1\mathcal{M}_{\mathrm{opt}} = 1) under identical rendering conditions (Figure 18). In the unmasked setting, incident irradiance across directly illuminated stripes is orders of magnitude higher than within shadowed regions. Consequently, backpropagation gradients are dominated by surface reflectance components (diffuse albedo and specular highlights), introducing severe gradient competition against the subtle, low-frequency subsurface diffusion halo. The optimizer attempts to compensate for subtle direct-light discrepancies by skewing the scattering distance parameter dd.

Interestingly, while the relative spectral proportionality (dr>dg>dbd_r > d_g > d_b) is naturally preserved due to the lower absorption of melanin and hemoglobin at longer wavelengths, the unmasked optimization yields an artificially saturated and magnitude-shifted dd. This magnitude shift truncates the diffusion tail along transition boundaries. In contrast,

restricting the loss domain strictly to the binary shadow mask M\mathcal{M} decouples parameter estimation from direct surface illumination, forcing the objective function to evaluate the pure radial decay of multiple scattering. This targeted supervision eliminates parameter overcompensation, ensuring that the estimated parameter dd yields a reconstruction closer to the reference capture, decreasing the perceptual FLIP error from 0.11 to 0.06 (45% reduction) in shadowed transition regions, and from 0.15 to 0.10 across the full patch.

Comparison with Multi-Light Acquisition To evaluate robustness under real-world conditions, we compare our single-light results against a multi-light setup (three synchronized projectors). This configuration allows us to introduce an additional preliminary optimization stage to the pipeline. In this new Stage 1, we leverage the multi-light observations to specifically optimize and refine the initial Sobel-filtered normal map, improving geometric consistency before proceeding with the standard material estimation steps (now shifted to Stages 2, 3, and 4).

We compare the scattering distance maps estimated from our standard single-light pipeline against those produced using this augmented four-stage multi-light configuration. Despite the improved geometric detail provided by the refined normals, the single-light method yields scattering distances that are visually and numerically consistent (mean L1\mathcal{L}_1 difference < 2%). As shown in Figure 19, discrepancies are minimal and confined to occluded regions.

Multi-light results for the skin patches analyzed in Fig. 12

Figure 19. Multi-light results for the skin patches analyzed in Figure 12. Each row shows (left to right): photo, render, masked pairs, and FLIP map. Consistency with single-view doptd_{\mathrm{opt}} validates our simplified setup’s robustness despite additional lighting data.

Figure 20 illustrates the material maps derived from the multi-light setup. By leveraging additional lighting data, this configuration improves the estimation of surface properties—particularly roughness—and better handles challenging facial regions such as the nose or ears. Although the setup is still constrained by the limited angular coverage of the three projectors, the multi-light approach provides more detailed material stacks. However, the recovered scattering distance parameters and the resulting render reconstructions are nearly identical to those produced by the single-view baseline. These results confirm that for the specific task of estimating subsurface scattering, our single-light, single-view approach remains both sufficient and robust.

Optimized material stack from the multi-light setup

Figure 20. Optimized material stack from the multi-light setup. Extra lighting data enables a more detailed roughness map and cleaner parameter separation, reducing the information baked into the residual specular map.

Conclusions

We presented a lightweight differentiable inverse rendering pipeline for estimating subsurface scattering parameters of human skin directly from single-view, single-light captures. By optimizing the same parameters used in standard real-time shading models, our method establishes a direct bridge between physical acquisition and production-ready rendering pipelines, eliminating the need for complex optical setups or intermediate physical volumetric parameter conversion.

Our experiments demonstrate that physically plausible scattering distances can be estimated from minimal input data. The recovered parameters produce a realistic skin appearance across a wide range of skin tones, exhibiting consistent numerical convergence in synthetic benchmarks and high perceptual reconstruction fidelity on real-world captures. The proposed shadow-aware loss formulation improves robustness in regions dominated by multiple scattering, enabling reliable estimation even under challenging pigmentation conditions. Furthermore, our dataset includes a diverse collection of calibrated skin samples aligned with the Monk Skin Tone Scale, supporting inclusive and scalable digital human modeling.

A key result of this work is that production-ready subsurface scattering parameters can be estimated from an accessible capture setup without relying on laboratory-grade acquisition systems. Rather than requiring specialized hardware or complex measurement procedures, our approach makes the problem tractable through differentiable optimization, physically motivated parameterization, and carefully designed supervision. This substantially lowers the barrier to high-quality skin digitization while preserving compatibility with real-time rendering workflows.

The method relies on a locally homogeneous scattering assumption, and monocular depth priors. These assumptions allow the inverse problem to remain practical while still producing parameters that are physically plausible and visually effective. In the single-light setting, the ambiguity between albedo and scattering is addressed through a multi-stage optimization strategy that stabilizes parameter recovery. In thin or highly translucent structures, such as nose cartilage or the ear helix, omitting explicit single-scattering optimization may lead to slightly larger estimated scattering distances, as the model explains missing translucency through the available parameters. Our regional blending scheme constrains these estimates within realistic ranges and preserves a consistent appearance across facial regions. Similarly, omitting non-local visibility handling causes residual self-shadows to be partially absorbed into the albedo in areas of steep relief.

Future work can build on this foundation by jointly recovering additional scattering components, incorporating stronger learned priors, integrating differentiable visibility testing to resolve self-shadowing, and extending the capture protocol to less constrained environments. More flexible acquisition strategies could further improve the fidelity of reconstruction while maintaining the accessibility of the proposed setup.

By combining accessible acquisition, differentiable optimization, physically grounded constraints, and validation across diverse skin tones, our method provides a scalable path toward realistic and inclusive digital human creation for real-time applications.

Acknowledgments

This work has been supported by the Spanish Agencia Estatal de Investigación (AEI) under Project PID2024-159930OB-I00 funded by MICIU/AEI/10.13039/501100011033 and by ERDF, EU.

supplementary.pdf

Introduction

This supplementary document provides extended validation, dataset analysis, and core implementation details supporting the main manuscript, organized as follows:

  • Map Evolution Across Optimization Stages (Section 2): Illustrates the step-by-step visual progression of active and frozen maps across optimization stages, justifying our staged schedule in decoupling surface reflections and subsurface diffusion.

  • Tilted View Validation (Section 3): Demonstrates the angular consistency and physical grounding of our 3D depth-aware optimization under varying camera orientations.

  • Dataset across the Monk Scale (Section 4): Presents comprehensive synthetic validation across the full Monk Skin Tone Scale by isolating albedo variations under fixed scattering parameters, and evaluates real-world participant distributions by cross-referencing subjective self-identification against our objective color-matching algorithm via a unified confusion matrix.

  • Replicability and Algorithmic Details (Section 5): Provides concrete pipeline blueprints, including RAW color calibration via Ridge Regression (Section 5.1), robust albedo initialization (Section 5.2), and our GPU shader (Section 5.3) and differentiable tensorized implementations (Section 5.4).

Map Progression Across Optimization Stages

To complement the optimization methodology described in the main manuscript, Figure 1 illustrates the step-by-step visual progression of active and frozen material maps across our three optimization stages.

During Stage 1, freezing the albedo AT\mathbf{A}_T forces the reconstruction loss to temporarily drive the active roughness map (ρT\boldsymbol{\rho}_T) to fit low-frequency diffuse shading gradients. In Stage 2, unfreezing AT\mathbf{A}_T provides the direct path to resolve these diffuse reconstruction errors. As the albedo rapidly absorbs the spatial shading details, the reconstruction gradient on other maps vanishes. Consequently, the L2\mathcal{L}_2 roughness prior

Map progression across optimization stages

Figure 1 (PDF p. 15). Map progression across optimization stages. Green and red denote active and frozen maps, respectively. Stage 1 fits global reflection, capturing unmodeled specular highlights into the residual map while shielding initialized albedo. Stage 2 unfreezes albedo to absorb diffuse spatial details, enabling the L2\mathcal{L}_2 prior to smooth roughness while preserving the specular residual. Stage 3 optimizes scattering distance over shadow-masked regions (Input) with surface maps frozen.

smoothly regularizes ρT\rho_T while leaving the residual specular map virtually unperturbed, matching its role as a buffer designed for unmodeled specular highlights rather than diffuse light transport.

Tilted View Validation

To verify that our frontal-view optimization yields physically grounded parameters rather than view-dependent over-fitted solutions, we perform tilted-view cross-validation on both synthetic and real data. For synthetic targets, we optimize parameters independently from frontal and novel tilted views, verifying that the estimated scattering distance (doptd_{\mathrm{opt}}) consistently matches the ground truth (drefd_{\mathrm{ref}}). For real targets, we evaluate acquisitions of identical skin regions under varying camera angles. As shown in Fig. 2, estimated scattering distances remain invariant across orientations, yielding minimal reconstruction error.

Our pipeline avoids 2D projection approximations by executing scattering calculations directly in 3D view space using the estimated geometric depth. This integration ensures that the Screen-Space Subsurface Scattering (SSSS) algorithm naturally accounts for surface curvature and perspective foreshortening. By computing physical distances rather than simple 2D pixel offsets, the model recovers orientation-invariant scattering profiles.

Tilted view validation

Figure 2 (PDF p. 16). Tilted view validation. In synthetic cases (top), doptd_{\mathrm{opt}} closely tracks drefd_{\mathrm{ref}}. For real captures (bottom), scattering distances remain stable across views, demonstrating that 3D depth-aware computations prevent parameter shrinkage caused by perspective foreshortening.

Dataset across the Monk Scale

Figure 3 presents the full synthetic validation workflow, confirming optimization stability across all ten Monk classes. By fixing the ground-truth scattering distance and varying only the target albedo, we verify that our method decouples scattering parameters independently of surface reflectance.

Figure 4 cross-references participant skin tone self-assessments against our automatically computed baseline using a unified confusion matrix (N=45N = 45). As expected from a Mediterranean–Caucasian population, the sample distribution concentrates around intermediate Monk tones (3–6), highlighting the representative challenges of real-world skin digitization. For automated classification, the mean skin color is extracted from the neutral frontal-light capture to eliminate specular and shadow contamination, and subsequently mapped via CIE LAB distance to uniformly lit reference exemplars from the MST dataset.

Discrepancies between computed tones and self-assessments in the matrix reveal inherent human perception biases and visual ambiguities when isolating pure albedo from overall lightness under casual observation. Furthermore, the original Monk scale is inherently perceptual, with reference colors defined under uncalibrated lighting conditions. Consequently, standardized computational environments and objective physical optimization pipelines are essential for consistent, illumination-invariant tone classification.

Reproducibility and Algorithmic Details

RAW Processing and Color Calibration

RAW images (6020 × 4024 px) are decoded via rawpy, bypassing all non-linear ISP operations to enforce a linear gamma (γ=1.0\gamma= 1.0) and native camera color space. Normalized 16-bit float intensities ([0, 1]) are downsampled using bicubic interpolation to 1204 × 804 px to preserve global spatial context, and center-cropped to a final operating resolution of 480 × 360 px.

To map the linear RAW space to standard D65 linear sRGB, we extract the mean RAW intensity of the 24 patches from an X-Rite ColorChecker Classic target (Fig. 5) to build the measurement matrix X\mathbf{X}. Target sRGB reference values are linearized via gamma decoding (γ=2.2\gamma= 2.2) to form matrix Y\mathbf{Y}.

To suppress noise amplification, the 3×33 \times3 Color Correction Matrix (CCM) C\mathbf{C} is computed channel-wise using L2L_2-regularized Ridge Regression. For each target channel column yc\mathbf{y}_c (c∈{r,g,b}c \in\{r,g,b\}), the optimization yields:

cc=(XTX+αI)−1XTyc(9)\mathbf{c}_c = \left(\mathbf{X}^{T}\mathbf{X} + \alpha\mathbf{I}\right)^{-1}\mathbf{X}^{T}\mathbf{y}_c \tag*{(9)}

where α=10−3\alpha= 10^{-3} and I\mathbf{I} is the identity matrix. The final matrix C=[crT;cgT;cbT]\mathbf{C} = [\mathbf{c}_r^{T}; \mathbf{c}_g^{T}; \mathbf{c}_b^{T}] maps the image via Ilinear=IrawCT\mathbf{I}_{\mathrm{linear}} = \mathbf{I}_{\mathrm{raw}}\mathbf{C}^{T}. Inverse gamma encoding (1/2.21/2.2) is applied strictly for display previews.

Albedo Initialization

Initializing the albedo from a flat-lit, diffuse-only baseline is critical for downstream parameter decoupling. Arbitrary or global-mean initialization risk optimization collapse into local minima due to facial features, hair, or micro-shadow contamination. If specular highlights contaminate the initial albedo estimate, the optimization struggles to isolate independent light transport components. Securing a clean, purely diffuse baseline guarantees that high-frequency specular details are absorbed into the residual specular map rather than getting baked into the estimated albedo or roughness maps.

We perform our initialization strategy in two stages (Algorithm 1):

Full synthetic validation across the Monk Skin Tone Scale

Figure 3 (PDF p. 17). Full synthetic validation across the Monk Skin Tone Scale. Per tone (left to right): input, optimized reconstruction, and FLIP error map. Inset swatches denote drefd_{\mathrm{ref}} (known reference value) and doptd_{\mathrm{opt}} (optimized value). Low perceptual errors confirm estimation invariance across diverse skin tones.

Unified confusion matrix cross-referencing participant self-identified skin tones against the automatic appearance-based color-matching algorithm

Figure 4 (PDF p. 17). Unified confusion matrix cross-referencing participant self-identified (Perceived) skin tones against our automatic appearance-based color-matching algorithm (Computed) across the Monk Skin Tone Scale (N=45N = 45). The clustering behavior highlights systematic discrepancies between subjective human self-assessment and objective physical albedo extraction, driven by visual perception biases and the uncalibrated nature of the original scale.

Physical color calibration setup under the projector’s flat-field white illumination

Figure 5 (PDF p. 17). Physical color calibration setup under the projector’s flat-field white illumination. The 24 patch coordinates are automatically extracted to fit the 3×33 \times3 CCM.

Algorithm 1 Adaptive Albedo Initialization

Require:

I\mathbf{I}: Normalized RGB input image

Ensure:

cinit\mathbf{c}_{\mathrm{init}}: Robust initialization RGB color vector

1: function EXTRACTADAPTIVESKINMASK(I\mathbf{I})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 1: Color Space Transformation

2: Ilab←\mathbf{I}_{\mathrm{lab}} \leftarrow ConvertRGBToLab(I\mathbf{I})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 2: Background Discarding & Feature Extraction

3: Mvalid←{(x,y)∣Ilab[x,y,L]>20}\mathcal{M}_{\mathrm{valid}} \leftarrow\{(x,y)\mid\mathbf{I}_{\mathrm{lab}}[x,y,L] > 20\}

4: P←Ilab[Mvalid,(a,b)]\mathbf{P} \leftarrow\mathbf{I}_{\mathrm{lab}}[\mathcal{M}_{\mathrm{valid}},(a,b)]

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 3: Chromatic Feature Clustering

5: C1,C2←\mathcal{C}_1,\mathcal{C}_2 \leftarrow ExecuteKMeans(P,K=2\mathbf{P}, K=2)

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 4: Skin Cluster Identification & Mapping

6: idxskin←argmax⁡(GetCentroid⁡(Ck)[a])idx_{\mathrm{skin}} \leftarrow\operatorname{argmax}(\operatorname{GetCentroid}(\mathcal{C}_k)[a])

7: Mraw←\mathcal{M}_{\mathrm{raw}} \leftarrow MapToImageSpace(Cidxskin\mathcal{C}_{idx_{\mathrm{skin}}})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 5: Luminance Filtering (Specularities & Shadows)

8: Lskin←Ilab[Mraw,L]L_{\mathrm{skin}} \leftarrow\mathbf{I}_{\mathrm{lab}}[\mathcal{M}_{\mathrm{raw}},L]

9: Tbright←T_{\mathrm{bright}} \leftarrow ComputePercentile(Lskin,90L_{\mathrm{skin}},90)

10: Tdark←T_{\mathrm{dark}} \leftarrow ComputePercentile(Lskin,5L_{\mathrm{skin}},5)

11: Mfinal←Mraw∧(Tdark<Ilab[:,:,L]<Tbright)\mathcal{M}_{\mathrm{final}} \leftarrow\mathcal{M}_{\mathrm{raw}} \land(T_{\mathrm{dark}} < \mathbf{I}_{\mathrm{lab}}[:, :, L] < T_{\mathrm{bright}})

12: return Mfinal\mathcal{M}_{\mathrm{final}}

13: end function

14: function INITIALIZEALBEDO(I\mathbf{I})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 1: Adaptive Mask Generation

15: M←\mathcal{M} \leftarrow EXTRACTADAPTIVESKINMASK(I\mathbf{I})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 2: Active Pure-Skin Pixel Gathering

16: Pskin←I[M]\mathbf{P}_{\mathrm{skin}} \leftarrow\mathbf{I}[\mathcal{M}]

17: cinit←[000]T\mathbf{c}_{\mathrm{init}} \leftarrow[0\quad0\quad0]^{T}

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 3: Channel-wise Statistical Sigma Clipping

18: for channel c∈{R,G,B}c \in\{R,G,B\} do

19: v←Pskin[:,c]\mathbf{v} \leftarrow\mathbf{P}_{\mathrm{skin}}[:,c]

20: μc,σc←\mu_c,\sigma_c \leftarrow ComputeMean(v\mathbf{v}), ComputeStdDev(v\mathbf{v})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Apply 2σ2\sigma Outlier Filter

21: vclipped←{x∈v∣∣x−μc∣<2σc}\mathbf{v}_{\mathrm{clipped}} \leftarrow\{x\in\mathbf{v}\mid|x-\mu_c|<2\sigma_c\}

22: cinit[c]←\mathbf{c}_{\mathrm{init}}[c] \leftarrow ComputeMean(vclipped\mathbf{v}_{\mathrm{clipped}})

23: end for

24: return cinit\mathbf{c}_{\mathrm{init}}

25: end function

Algorithm 2 Burley Multiple Scattering Shader

Require:

u\mathbf{u}: 2D screen-space texture coordinates [u,v][u,v]

S\mathbf{S}: Screen-space scattering map storing spatially blended doptd_{\mathrm{opt}} values

D\mathbf{D}: Screen-space depth texture map

A\mathbf{A}: Screen-space surface albedo texture map

L\mathbf{L}: Screen-space diffuse irradiance texture map

NN: Sample count ∈[2,128]\in[2,128]

samples\mathbf{samples}: Array of vec2(θ,r\theta,r) (Fibonacci angles, Stratified Burley Inverse CCDF normalized radii (Eq. (2)))

Mproj,Mproj−1\mathbf{M}_{\mathrm{proj}},\mathbf{M}_{\mathrm{proj}}^{-1}: Camera projection matrix and its inverse

Ensure:

I\mathbf{I}: Scattered diffuse radiance vector for the target fragment

1: procedure EVALUATEMULTIPLESCATTERING

&nbsp;&nbsp;&nbsp;&nbsp;▹ Subsurface Scattering Setup

2: wscale←0.01, ϵ←10−6w_{\mathrm{scale}}\leftarrow0.01,\ \epsilon\leftarrow10^{-6}

3: d←S(u).rgb×wscale\mathbf{d}\leftarrow\mathbf{S}(u).\mathit{rgb}\times w_{\mathrm{scale}}

4: rmax←max⁡(dr,dg,db)r_{\mathrm{max}}\leftarrow\max(d_r,d_g,d_b)

5: I←[0.00.00.0]T\mathbf{I}\leftarrow[0.0\quad0.0\quad0.0]^{T}

6: K←[0.00.00.0]T\mathbf{K}\leftarrow[0.0\quad0.0\quad0.0]^{T}

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 1: Target View-Space 3D Position Reconstruction

7: x←\mathbf{x}\leftarrow Unproject(u,D(u),Mproj−1\mathbf{u},D(\mathbf{u}),\mathbf{M}_{\mathrm{proj}}^{-1})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 2: Generate noise pattern to avoid θi\theta_i periodicity

8: jitter←2.0×π×\mathit{jitter}\leftarrow2.0\times\pi\times Noise(u\mathbf{u})

9: for i←0i\leftarrow0 to N−1N-1 do

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 3: Fibonacci & Burley Profile Importance Sampling

10: θi←samples[i].x+jitter\theta_i\leftarrow\mathbf{samples}[i].x+\mathit{jitter}

11: ri←samples[i].y×rmaxr_i\leftarrow\mathbf{samples}[i].y\times r_{\mathrm{max}}

12: offset←[ricos⁡θirisin⁡θi0.0]T\mathit{offset}\leftarrow[r_i\cos\theta_i\quad r_i\sin\theta_i\quad0.0]^{T}

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 4: Neighbor Screen Projection

13: ui←\mathbf{u}_i\leftarrow Project(x+offset,Mproj\mathbf{x}+\mathit{offset},\mathbf{M}_{\mathrm{proj}})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 5: Neighbor View-Space Position Reconstruction

14: Di←D(ui)D_i\leftarrow D(\mathbf{u}_i)

15: xi←\mathbf{x}_i\leftarrow Unproject(ui,Di,Mproj−1\mathbf{u}_i,D_i,\mathbf{M}_{\mathrm{proj}}^{-1})

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 6: Subsurface Diffusion Weights (Eq. (3))

16: si←max⁡(∥x−xi∥2,ϵ)s_i\leftarrow\max(\lVert\mathbf{x}-\mathbf{x}_i\rVert_2,\epsilon)

17: rRr←si×R(si,d)rR_r\leftarrow s_i\times R(s_i,d)

18: pr←ri×R(ri,rmax)pr\leftarrow r_i\times R(r_i,r_{\mathrm{max}})

19: ki←rRr/prk_i\leftarrow rR_r/pr

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 7: Irradiance Gathering

20: Li←L(ui).rgbL_i\leftarrow L(\mathbf{u}_i).\mathit{rgb}

21: I←I+(ki×Li)\mathbf{I}\leftarrow\mathbf{I}+(k_i\times L_i)

22: K←K+ki\mathbf{K}\leftarrow\mathbf{K}+k_i

23: end for

&nbsp;&nbsp;&nbsp;&nbsp;▹ Step 8: Normalized weighted Integration (Eq. (4) )

24: I←A(u).rgb×(I/max⁡(K,ϵ))\mathbf{I}\leftarrow A(\mathbf{u}).\mathit{rgb}\times(\mathbf{I}/\max(\mathbf{K},\epsilon))

25: return I\mathbf{I}

26: end procedure

Algorithm 3 Burley Multiple Scattering Tensorized

Require:

U,V\mathbf{U}, \mathbf{V}: Grid coordinate tensors. (H,W)(H, W) in [−1,1][-1, 1]

ST\mathbf{S}_T: Homogeneous scattering distance tensor. (H,W,3)(H, W, 3)

DT\mathbf{D}_T: Depth buffer tensor map. (H,W)(H, W)

AT\mathbf{A}_T: Surface albedo tensor map. (H,W,3)(H, W, 3)

LT\mathbf{L}_T: Diffuse irradiance tensor map. (1,3,H,W)(1, 3, H, W)

NN: Sample count ∈[2,128]\in[2, 128]

samplesθ\mathrm{samples}_{\theta}: Fibonacci angles tensor. (N)(N)

samplesr\mathrm{samples}_{r}: Stratified Burley Inverse CCDF radii tensor. (N)(N) (Eq. (2))

Φc\mathbf{\Phi}_c: Camera intrinsic parameters {znear,zfar,wsensor,ffocal}\{z_{\mathrm{near}}, z_{\mathrm{far}}, w_{\mathrm{sensor}}, f_{\mathrm{focal}}\}

Ensure:

IT\mathbf{I}_T: Scattered diffuse radiance tensor. (H,W,3)(H, W, 3)

1: procedure EVALUATEMULTIPLESCATTERING

▹ Subsurface Scattering Setup

2: wscale←0.01, ϵ←10−6w_{\mathrm{scale} \leftarrow0.01,\ \epsilon\leftarrow10^{-6}}

3: d←ST×wscale\mathbf{d} \leftarrow\mathbf{S}_T \times w_{\mathrm{scale}} ▹ (H,W,3)(H, W, 3)

4: rmax⁡←d.max⁡()r_{\max} \leftarrow\mathbf{d}.\max()

▹ Step 1: Target View-Space 3D Position Reconstruction

5: z←LinearizeDepth⁡(DT,Φc)z \leftarrow\operatorname{LinearizeDepth}(\mathbf{D}_T, \mathbf{\Phi}_c) ▹ (H,W)(H, W) (Eq. (5))

6: xv←Unproject⁡(U,z,Φc)x_v \leftarrow\operatorname{Unproject}(U, z, \mathbf{\Phi}_c) ▹ (H,W)(H, W) (Eq. (6))

7: yv←Unproject⁡(V,z,Φc)y_v \leftarrow\operatorname{Unproject}(V, z, \mathbf{\Phi}_c) ▹ (H,W)(H, W) (Eq. (6))

8: Pv←[xvyv−z]T\mathbf{P}_v \leftarrow[x_v\quad y_v\quad-z]^{\mathsf{T}} ▹ (H,W,3)(H, W, 3)

▹ Step 2: Generate noise pattern to avoid θ\theta periodicity

9: jitter⁡←2.0×π×Random⁡(1,H,W)\operatorname{jitter} \leftarrow2.0 \times\pi\times\operatorname{Random}(1, H, W) ▹ (1,H,W)(1, H, W)

▹ Step 3: Fibonacci & Burley Profile Importance Sampling

10: θ←samplesθ.view⁡(N,1,1)+jitter⁡\theta\leftarrow\mathrm{samples}_{\theta}.\operatorname{view}(N, 1, 1) + \operatorname{jitter} ▹ (N,H,W)(N, H, W)

11: r←samplesr.view⁡(N,1,1)×rmax⁡r \leftarrow\mathrm{samples}_{r}.\operatorname{view}(N, 1, 1) \times r_{\max} ▹ (N,1,1)(N, 1, 1)

12: xo←cos⁡θ⊙rx_o \leftarrow\cos\theta\odot r ▹ (N,H,W)(N, H, W)

13: yo←sin⁡θ⊙ry_o \leftarrow\sin\theta\odot r ▹ (N,H,W)(N, H, W)

▹ Step 4: Neighbor Screen Projection

14: Us←Project⁡(xv+xo,z,Φc)U_s \leftarrow\operatorname{Project}(x_v + x_o, z, \mathbf{\Phi}_c) ▹ (N,H,W)(N, H, W) (Eq. (7))

15: Vs←Project⁡(yv+yo,z,Φc)V_s \leftarrow\operatorname{Project}(y_v + y_o, z, \mathbf{\Phi}_c) ▹ (N,H,W)(N, H, W) (Eq. (7))

16: G←Stack⁡(Us,Vs,dim⁡=−1)\mathbf{G} \leftarrow\operatorname{Stack}(U_s, V_s, \operatorname{dim}=-1) ▹ (N,H,W,2)(N, H, W, 2)

▹ Step 5: Neighbor View-Space Position Reconstruction

17: Ds←GridSample⁡(DT,G)\mathbf{D}_s \leftarrow\operatorname{GridSample}(\mathbf{D}_T, \mathbf{G}) ▹ (N,H,W)(N, H, W)

18: zs←LinearizeDepth⁡(Ds,Φc)z_s \leftarrow\operatorname{LinearizeDepth}(\mathbf{D}_s, \mathbf{\Phi}_c) ▹ (N,H,W)(N, H, W) (Eq. (5))

19: xs←Unproject⁡(Us,zs,Φc)x_s \leftarrow\operatorname{Unproject}(U_s, z_s, \mathbf{\Phi}_c) ▹ (H,W)(H, W) (Eq. (6))

20: ys←Unproject⁡(Vs,zs,Φc)y_s \leftarrow\operatorname{Unproject}(V_s, z_s, \mathbf{\Phi}_c) ▹ (H,W)(H, W) (Eq. (6))

21: Ps←[xsys−zs]T\mathbf{P}_s \leftarrow[x_s\quad y_s\quad-z_s]^{\mathsf{T}} ▹ (N,H,W,3)(N, H, W, 3)

▹ Step 6: Subsurface Diffusion Weights (Eq. (3))

22: s←max⁡(∥Pv−Ps∥2,ϵ)s \leftarrow\max(\lVert\mathbf{P}_v-\mathbf{P}_s\rVert_2,\epsilon) ▹ (N,H,W,1)(N, H, W, 1)

23: rRr←s⊙R(s,d)rR_r \leftarrow s \odot R(s,d) ▹ (N,H,W,3)(N, H, W, 3)

24: pr←r⊙R(r,rmax⁡)p_r \leftarrow r \odot R(r,r_{\max}) ▹ (N,1,1)(N, 1, 1)

25: k←rRr⊘prk \leftarrow rR_r \oslash p_r ▹ (N,H,W,3)(N, H, W, 3)

▹ Step 7: Irradiance Gathering & Dimension Collapse

26: LT←LT.expand⁡(N)\mathbf{L}_T \leftarrow\mathbf{L}_T.\operatorname{expand}(N) ▹ (N,3,H,W)(N, 3, H, W)

27: Ls←GridSample⁡(LT,G)\mathbf{L}_s \leftarrow\operatorname{GridSample}(\mathbf{L}_T, \mathbf{G}) ▹ (N,3,H,W)(N, 3, H, W)

28: Ls←Ls.permute⁡(0,2,3,1)\mathbf{L}_s \leftarrow\mathbf{L}_s.\operatorname{permute}(0, 2, 3, 1) ▹ (N,H,W,3)(N, H, W, 3)

29: IT←∑dim⁡=0(k⊙Ls)\mathbf{I}_T \leftarrow\sum_{\operatorname{dim}=0}(k \odot\mathbf{L}_s) ▹ (H,W,3)(H, W, 3)

30: KT←∑dim⁡=0k\mathbf{K}_T \leftarrow\sum_{\operatorname{dim}=0} k ▹ (H,W,3)(H, W, 3)

▹ Step 8: Normalized weighted Integration (Eq. (4))

31: IT←AT⊙(IT⊘max⁡(KT,ϵ))\mathbf{I}_T \leftarrow\mathbf{A}_T \odot\left(\mathbf{I}_T \oslash\max(\mathbf{K}_T,\epsilon)\right) ▹ (H,W,3)(H, W, 3)

32: return IT\mathbf{I}_T

33: end procedure

  1. Adaptive Skin Masking: The input is converted to CIE L∗a∗b∗L^{*}a^{*}b^{*} space. Chromatic channels (a∗a^{*}, b∗b^{*}) are clustered via kk-means (k=2k = 2) after filtering low-luminance background pixels (L∗≤20L^{*} \le20). The skin cluster is identified by the maximum a∗a^{*} centroid. High-frequency specularity and deep shadow artifacts are eliminated by removing the top 10% and bottom 5% luminance percentiles within the raw mask.

  1. Statistical Sigma Clipping: Active skin pixels are mapped back to linear RGB space. For each spectral channel, a 2σ2\sigma Sigma Clipping filter eliminates remaining outliers (e.g., edge pixels, macro-shadow boundary artifacts) from the distribution. The robust mean of the clipped pixels forms the final initialization vector cinit\mathbf{c}_{\mathrm{init}}.

GPU Shader Implementation

Algorithm 2 details our screen-space multiple scattering approximation within the fragment shader, mapping directly to the mathematical formulations described in the main manuscript. All equation references within the pseudocode correspond to the main text.

Burley Multiple Scattering Differentiable Tensorized Implementation

While forward rendering executes via the GPU fragment shader, inverse optimization requires an auto-differentiable SSSS model to backpropagate loss gradients. We translate the screen-space diffusion pass into vectorized PyTorch operations by broadcasting inputs across the NN importance sample coordinates into a unified high-dimensional tensor. This architecture enables simultaneous 3D view-space reconstruction, projection, and differentiable bilinear texture lookups, removing sequential per-pixel execution loops. The tensorized execution logic is formalized in Algorithm 3, where tuples within the inline comments denote multi-dimensional tensor shapes, and equation references correspond to the formulations established in the main text.

References

  1. [1]H. Ha, I. Hwang, N. Monzon, J. Cho, D. Kim, S.-H. Baek, A. Muñoz, D. Gutierrez, M. H. Kim, Polarimetric BSSRDF Acquisition of Dynamic Faces, ACM Transactions on Graphics 43 (6) (2024) 1–11. doi:10.1145/3687767.
  2. [2]C. Aliaga, M. Xia, X. Xie, A. Jarabo, G. Braun, C. Hery, A Hyperspectral Space of Skin Tones for Inverse Rendering of Biophysical Skin Properties, Computer Graphics Forum 42 (4) (7 2023). doi: 10.1111/cgf.14887.DOI
  3. [3]X. Li, G. C. Guarnera, A. Lin, A. Ghosh, Practical Measurement and Neural Encoding of Hyperspectral Skin Reflectance, in: 2024 International Conference on 3D Vision (3DV), IEEE, 2024, pp. 1301–1309. doi:10.1109/3DV62453.2024.00116.DOI
  4. [4]B. Burley, Extending the Disney BRDF to a BSDF with integrated subsurface scattering, SIGGRAPH Course: Physically Based Shading in Theory and Practice. ACM, New York, NY 19 (7) (2015) 9.
  5. [5]E. Monk, The monk skin tone scale (2023).
  6. [6]T. B. Fitzpatrick, The Validity and Practicality of Sun-Reactive Skin Types I Through VI, Archives of Dermatology 124 (6) (1988) 869. doi:10.1001/archderm.1988.01670060015008.pubmed.ncbi.nlm.nih.gov/3377516
  7. [7]C. Che, F. Luan, S. Zhao, K. Bala, I. Gkioulekas, Towards Learning-based Inverse Subsurface Scattering, in: 2020 IEEE International Conference on Computational Photography (ICCP), IEEE, 2020, pp. 1–12. doi:10.1109/ICCP48838.2020.9105209.DOI
  8. [8]H. W. Jensen, S. R. Marschner, M. Levoy, P. Hanrahan, A practical model for subsurface light transport, in: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, 2001, pp. 511–518. doi:10.1145/383259.383319.DOI
  9. [9]C. Donner, H. W. Jensen, Light diffusion in multi-layered translucent materials, ACM Transactions on Graphics 24 (3) (2005) 1032–1039. doi:10.1145/1073203.1073308.
  10. [10]D. E’on, A better dipole, Eugene d’Eon 14 (2012).
  11. [11]J. R. Frisvad, T. Hachisuka, T. K. Kjeldsen, Directional Dipole Model for Subsurface Scattering, ACM Transactions on Graphics 34 (1) (2014) 1–12. doi:10.1145/2682692.DOI
  12. [12]E. D’Eon, G. Irving, A quantized-diffusion model for rendering translucent materials, ACM Transactions on Graphics 30 (4) (2011) 1–14. doi:10.1145/2010324.1964951.DOI
  13. [13]Y. N. Mishra, Droplet Size, Concentration, and Temperature Mapping in Sprays Using SLIPI-Based Techniques, Ph.D. thesis, Lund University (2018).
  14. [14]C. Barré-Brisebois, M. Bouchard, Approximating Translucency for a Fast Cheap and Convincing Subsurface Scattering Look, in: Game developers conference, Vol. 6, 2011.
  15. [15]J. Jimenez, D. Gutierrez, Screen-Space Subsurface Scattering, in: W. Engel (Ed.), GPU Pro: Advanced Rendering Techniques, AK Peters Ltd., 2010, pp. 335–351.DOI
  16. [16]J. Jimenez, K. Zsolnai, A. Jarabo, C. Freude, T. Auzinger, X. Wu, J. von der Pahlen, M. Wimmer, D. Gutierrez, Separable Subsurface Scattering, Computer Graphics Forum 34 (6) (2015) 188–197. doi:10.1111/cgf.12529.DOI
  17. [17]P. H. Christensen, An approximate reflectance profile for efficient subsurface scattering, in: ACM SIGGRAPH 2015 Talks, ACM, New York, NY, USA, 2015, pp. 1–1. doi:10.1145/2775280.2792555.DOI
  18. [18]T. T. G. D. M. Tran, H. W. Jensen, R. Ramamoorthi, J. R. Frisvad, Neural SSS: Lightweight Object Appearance Representation, Computer Graphics Forum 43 (4) (7 2024). doi:10.1111/cgf.15158.DOI
  19. [19]T. G. J. R. Frisvad, R. Ramamoorthi, H. W. Jensen, NeuPReSS: Compact Neural Precomputed Subsurface Scattering for Distant Lighting of Heterogeneous Translucent Objects, Computer Graphics Forum 43 (7) (10 2024). doi:10.1111/cgf.15234.DOI
  20. [20]T. Weyrich, W. Matusik, H. Pfister, J. Lee, A. Ngan, H. W. Jensen, M. Gross, A measurement-based skin reflectance model for face rendering and editing, None 71 (4) (2005) 1–13.
  21. [21]A. Ghosh, T. Hawkins, P. Peers, S. Frederiksen, P. Debevec, Practical modeling and acquisition of layered facial reflectance, ACM Transactions on Graphics 27 (5) (2008) 1–10. doi:10.1145/1409060.1409092.DOI
  22. [22]S. Tariq, A. Gardner, I. Llamas, A. Jones, P. Debevec, G. Turk, Efficient estimation of spatially varying subsurface scattering parameters for relighting, Tech. rep. (2006).
  23. [23]D. Lanza, J. R. Padrón-Griffe, A. Pranovich, A. Muñoz, J. R. Frisvad, A. Jarabo, Practical Appearance Model for Foundation Cosmetics, Computer Graphics Forum 43 (4) (7 2024). doi:10.1111/cgf.15148.DOI
  24. [24]X. He, M. Huang, R. Fu, J. Guo, J. Yuan, Y. Wang, Y. Guo, Biophysically-based Simulation of Sun-induced Skin Appearance Changes, in: R. Chen, T. Ritschel, E. Whiting (Eds.), Pacific Graphics Conference Papers and Posters, The Eurographics Association, 2024. doi:10.2312/pg.20241286.
  25. [25]Q. Wang, F. Luan, Y. Dai, Y. Huo, H. Bao, R. Wang, A biophysical-based skin model for heterogeneous volume rendering, Computational Visual Media 11 (2) (2025) 289–303.DOI
  26. [26]B. Burley, W. D. A. Studios, Physically-based shading at disney, in: Acm siggraph, Vol. 2012, 2012, pp. 1–7.
  27. [27]T. Yatagwa, H. Todo, Y. Yamaguchi, S. Morishima, Data compression for measured heterogeneous subsurface scattering via scattering profile blending, The Visual Computer 36 (3) (2020) 541–558. doi:10.1007/s00371-018-01626-x.
  28. [28]P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, M. Sagar, Acquiring the reflectance field of a human face, in: Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH ’00, ACM Press, New York, New York, USA, 2000, pp. 145–156. doi:10.1145/344779.344855.DOI
  29. [29]A. Ghosh, G. Fyffe, B. Tunwattanapong, J. Busch, X. Yu, P. Debevec, Multiview face capture using polarized spherical gradient illumination, ACM Transactions on Graphics 30 (6) (2011) 1–10. doi:10.1145/2070781.2024163.DOI
  30. [30]W.-C. Ma, T. Hawkins, P. Peers, C.-F. Chabert, M. Weiss, P. E. Debevec, others, Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination., Rendering Techniques 9 (2007) 2.
  31. [31]G. Fyffe, K. Nagano, L. Huynh, S. Saito, J. Busch, A. Jones, H. Li, P. Debevec, Multi-View Stereo on Consistent Face Topology, Computer Graphics Forum 36 (2) (2017) 295–309. doi:10.1111/cgf.13127.DOI
  32. [32]Y. Gitlina, G. C. Guarnera, D. S. Dhillon, J. Hansen, A. Lattas, D. Pai, A. Ghosh, Practical Measurement and Reconstruction of Spectral Skin Reflectance, Computer Graphics Forum 39 (4) (2020) 75–89. doi:10.1111/cgf.14055.DOI
  33. [33]J. Riviere, P. Gotardo, D. Bradley, A. Ghosh, T. Beeler, Single-shot high-quality facial geometry and skin appearance capture, ACM Transactions on Graphics 39 (4) (8 2020). doi:10.1145/3386569.3392464.
  34. [34]V. Deschaintre, M. Aittala, F. Durand, G. Drettakis, A. Bousseau, Single-image SVBRDF capture with a rendering-aware deep network, ACM Transactions on Graphics 37 (4) (2018) 1–15. doi:10.1145/3197517.3201378.DOI
  35. [35]X. Zhou, M. Hašan, V. Deschaintre, P. Guerrero, K. Sunkavalli, N. K. Kalantari, A Semi-Procedural Convolutional Material Prior, Computer Graphics Forum 42 (6) (9 2023). doi:10.1111/cgf.14781.DOI
  36. [36]C. Rodriguez-Pardo, D. Pascual-Hernandez, J. Rodriguez-Vazquez, J. Lopez-Moreno, E. Garces, Single-image reflectance and transmittance estimation from any flatbed scanner, Computers & Graphics 127 (2025) 104186. doi:10.1016/j.cag.2025.104186.
  37. [37]E. Garces, V. Arellano, C. Rodriguez-Pardo, D. Pascual-Hernandez, S. Suja, J. Lopez-Moreno, Towards Material Digitization with a Dual-scale Optical System, ACM Transactions on Graphics 42 (4) (2023) 1–13. doi:10.1145/3592147.DOI
  38. [38]P. Weier, J. Riviere, R. Guseinov, S. Garbin, P. Slusallek, B. Bickel, T. Beeler, D. Vicini, Practical Inverse Rendering of Textured and Translucent Appearance, ACM Transactions on Graphics 44 (4) (2025) 1–16. doi:10.1145/3730855.DOI
  39. [39]V. Bruce, A. Young, Understanding face recognition, British journal of psychology 77 (3) (1986) 305–327.pubmed.ncbi.nlm.nih.gov/3756376
  40. [40]S. L. Jacques, Optical properties of biological tissues: a review, Physics in Medicine & Biology 58 (11) (2013) R37.DOI
  41. [41]J. Moreau-Matthis, Babylon. Js Essentials, Packt Publishing Ltd, 2016.
  42. [42]E. Golubev, Efficient screen-space subsurface scattering using Burley’s normalized diffusion in real-time, ACM SIGGRAPH Courses: Advances in Real-Time Rendering in Games Course (2018).
  43. [43]J. H. Hannay, J. F. Nye, Fibonacci numerical integration on a sphere, Journal of Physics A: Mathematical and General 37 (48) (2004) 11591–11601.DOI
  44. [44]P. Gotardo, J. Riviere, D. Bradley, A. Ghosh, T. Beeler, Practical dynamic facial appearance modeling and acquisition, ACM Transactions on Graphics 37 (6) (2018) 1–13. doi:10.1145/3272127.3275073.DOI
  45. [45]L. Yang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, H. Zhao, Depth Anything V2 (2024). URL https://arxiv.org/abs/2406.09414
  46. [46]R. J. Woodham, Photometric method for determining surface orientation from multiple images, Optical engineering 19 (1) (1980) 139–144.
  47. [47]J. Salvi, J. Pagès, J. Batlle, Pattern codification strategies in structured light systems, Pattern Recognition 37 (4) (2004) 827–849. doi: 10.1016/j.patcog.2003.10.002.DOI
  48. [48]C. Wu, B. Wilburn, Y. Matsushita, C. Theobalt, High-quality shape from multi-view stereo and shading under general illumination, in: CVPR 2011, IEEE, 2011, pp. 969–976. doi:10.1109/CVPR.2011.5995388.DOI
  49. [49]N. Otsu, A Threshold Selection Method from Gray-Level Histograms, IEEE Transactions on Systems, Man, and Cybernetics 9 (1) (1979) 62–66. doi:10.1109/TSMC.1979.4310076.DOI
  50. [50]D. P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization (1 2017). doi:10.48550/arXiv.1412.6980.
  51. [51]P. Andersson, J. Nilsson, T. Akenine-Möller, M. Oskarsson, K. Åström, M. D. Färchild, FLIP, Proceedings of the ACM on Computer Graphics and Interactive Techniques 3 (2) (2020) 1–23. doi:10.1145/3406183.DOI
  52. [52]R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang, The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 586–595. doi:10.1109/CVPR.2018.00068.DOI
  53. [53]Zhou Wang, A. C. Bovik, H. Sheikh, E. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing 13 (4) (2004) 600–612. doi:10.1109/TIP.2003.819861.DOI

Paper details

vlnc:W_x22zuqsj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents