betaFeedback

WorksComputer Science

RATR: Training-Free Material Rebinding via Role Assignment and Temporal Routing

Abstract

Material rebinding aims to transfer the material appearance from a reference image while preserving the target object's geometry and identity. Existing methods typically struggle to balance material fidelity with structural preservation due to the coupled influence of structural and appearance cues in their visual conditions. We identify role entanglement in the target and reference conditions as a contributing factor the target condition carries both geometry and its original appearance while the reference condition contains material cues alongside object-specific structure. To address this issue, we propose RATR, a training-free material rebinding method built on a diffusion-based image editor that assigns complementary roles to the two conditions and coordinates their influence during denoising. To make the target condition primarily convey structure, we introduce an albedo-resolved surface release mechanism, which suppresses the target's original surface appearance while retaining geometric cues, yielding a structure-oriented condition for material transfer. On the reference side, the designed material evidence isolation mechanism emphasizes material cues while reducing interference from object-specific structure, yielding a complementary material-oriented condition.

Introduction

Given a target image and a reference material exemplar, material rebinding aims to change how the target surface appears while preserving what the target object is. This requires transferring material-specific appearance without altering the target’s geometry, identity, or object-specific details. The difficulty lies in the visual evidence itself: the target image contains not only the structure to be preserved but also the appearance to be replaced, whereas the reference image contains not only the material to be transferred but also the object and scene in which that material is observed. Successful rebinding therefore requires selective use of both images.

Recent diffusion-based material transfer methods introduce explicit material exemplars together with auxiliary geometric, lighting, depth, normal, or token-level conditions [8, 13, 30, 22, 23]. Although these designs improve material fidelity, stronger material guidance can also distort target geometry, oversmooth surface details, or introduce reference-object content. Large-scale image editors such as Qwen-Image-Edit provide an alternative through powerful reconstruction priors that help preserve object identity and geometry [48, 39]. However, preserving the target too strongly can retain its original surface appearance and limit material transfer. These outcomes raise a central question: why does strengthening the evidence for one objective often undermine the other?

We examine this question through the information carried by the two visual conditions. The target condition jointly encodes desired structure and the original target appearance; the reference condition jointly encodes transferable material cues and undesired reference-object content. Increasing the strength of either condition does not selectively amplify the information needed for rebinding. Stronger target conditioning can preserve geometry while reinforcing the original material, whereas stronger reference conditioning can improve material appearance while increasing reference-content interference.

reference-content interference. Thus, structural distortion and incomplete material transfer can be understood as two manifestations of the same underlying difficulty: useful and interfering information are strengthened together.

We refer to this difficulty as role entanglement. Unlike a conflict between otherwise well-specified conditions, role entanglement occurs within each condition, whose visual content extends beyond its intended responsibility. This distinction matters because adjusting the relative strengths of the raw target and reference conditions leaves their internal mixtures intact. Material rebinding therefore requires more than deciding which image should exert greater influence: it requires specifying which information each image should contribute. Moreover, these contributions need not remain fixed throughout diffusion synthesis. Structural evidence can establish the target geometry in earlier denoising stages, while material evidence can guide surface appearance in later stages. This motivates treating condition content and its temporal allocation as two complementary design problems.

Accordingly, we reformulate material rebinding as a role assignment and temporal routing problem. Role assignment constructs a target condition focused on structure and identity and a reference condition focused on transferable material appearance. Temporal routing then coordinates their contributions over the denoising process. This formulation addresses two limitations of raw two-image conditioning: each input carries information outside its intended role, and a fixed allocation of condition influence does not account for the different requirements of structural preservation and material synthesis.

Based on this formulation, we propose RATR, a training-free material rebinding method built on a frozen two-image diffusion editor. On the target side, Albedo-Resolved Surface Release constructs albedo-resolved surface boundaries that retain geometric cues while suppressing the original surface appearance. On the reference side, lightweight Material Evidence Isolation reduces object and scene interference to provide more focused material evidence. These operations reparameterize the inputs toward complementary structural and material roles. Surface-Material Conditional Direction Routing (SM-CDR) then allocates their centered VAE condition residuals over denoising time, emphasizing structural evidence earlier and material evidence later. In this way, RATR addresses both what each condition carries and when it contributes, without modifying or retraining the underlying editor.

Experiments across diverse target objects and material exemplars show consistent improvements in target-side structural preservation together with strong reference-side material fidelity. The qualitative results in Figure 1 illustrate both directions of generalization: a fixed target can be rebound to different materials, and a single material exemplar can be transferred across different targets. These results support the effectiveness of coordinating role-specific conditions and their temporal contributions for material rebinding.

Qualitative demonstration of RATR

Figure 1. Qualitative demonstration of RATR. Top: surface release produces albedo-resolved surface boundaries CsC_s that preserve target geometry while suppressing the original target appearance. Middle: a fixed target can be rebound to diverse reference materials. Bottom: a single material can transfer across different targets

Our contributions are summarized as follows:

  • We identify role entanglement within target and reference conditions as a contributing factor to the trade-off between structural preservation and material fidelity, and reformulate material rebinding as a role assignment and temporal routing problem.

  • We propose RATR, a training-free framework that makes the target image primarily guide structure and the reference image primarily guide material appearance, and coordinates their influence across generation stages to reduce source-appearance retention and reference-content interference.

  • We evaluate RATR across diverse target–material pairs, demonstrating consistent gains in target-side structural preservation alongside strong reference-side material fidelity.

Related Work

Diffusion-Based Image Editing and Reference Conditioning

Diffusion models have become a foundation for image synthesis and editing, with latent diffusion, high-resolution extensions, and transformer backbones expanding the quality and flexibility of generation [20, 41, 37, 35]. Image-editing methods such as SDEdit and Blended Diffusion modify images through guided denoising and localized synthesis [31, 1]. Subsequent approaches improve editability and content preservation through cross-attention control, image inversion, instruction following, mask prediction, feature reuse, or mutual self-attention [18, 33, 4, 11, 45, 6]. Together, these works establish strong image-editing priors, but do not by themselves specify how the geometric identity of a target and the appearance of a separate material exemplar should be assigned different conditioning roles.

Controllable generation provides several mechanisms for introducing visual evidence. ControlNet and T2I-Adapter inject spatial cues such as edges, depth, and layout, whereas Composer and Uni-ControlNet investigate the composition of heterogeneous controls [53, 34, 21, 55]. FreeControl further explores training-free spatial and appearance guidance [32]. Image-prompt and exemplar-based approaches, including IP-Adapter, Paint by Example, AnyDoor, PAIR Diffusion, and BLIP-Diffusion, transfer different aspects of reference-image information for image generation or editing [52, 51, 7, 14, 26]. Recent general-purpose editors such as Qwen-Image-Edit-2511 also support multi-image editing and material replacement [48, 39]. Nevertheless, the presence of two visual inputs does not guarantee that their contributions are restricted to the intended task: a target may retain its previous color or texture, while an exemplar can introduce unwanted object or scene information. This motivates an explicit, material-oriented organization of visual conditions within a frozen editor, rather than an assumption that generic reference conditioning already separates geometry from transferable appearance.

Material Appearance Editing and Material Rebinding

Material appearance has traditionally been studied through intrinsic-image decomposition, reflectance and illumination estimation, and physically based rendering [3, 2, 36]. Rendering-aware material editing and single-image SVBRDF capture recover or manipulate material-related quantities, while earlier example-based approaches transfer visual attributes through image correspondences [28, 12, 27]. Generative material representations provide further alternatives: MaterialGAN learns an SVBRDF prior, MatFormer models procedural material structures, and MatFusion applies diffusion models to reflectance capture [17, 16, 43]. These directions provide explicit or learned material representations, but their primary objectives differ from directly assigning the appearance of an arbitrary material reference to an existing target object while maintaining its image-space structure.

Diffusion-based methods have increasingly addressed material editing without requiring an explicit material reconstruction as the final output. Alchemist learns parametric control over material properties, including roughness, metallicity, albedo, and transparency; PhyS-EdiT studies physics-aware semantic editing; and Material Palette extracts reusable material representations from a single image [44, 5, 29]. More directly, ZeST performs zero-shot material transfer using exemplar appearance together with depth and grayscale-derived target cues [8]. MaterialFusion combines diffusion-based transfer with region constraints and a controllable transfer strength, while MatSwap learns light-aware material transfer from supervised data [13, 30]. MARBLE represents material attributes and their combinations in CLIP space [40, 9]. Related tasks include CoatFusion’s application of controllable material coatings and LineArt’s reference-appearance transfer to detailed design drawings [25, 47]; these settings are related to, but not identical to, material replacement on natural-image targets.

Recent diffusion-transformer methods further clarify the importance of visual-condition organization. MaTe fuses material, depth, and illumination images as tokens in a shared transformer space, enabling zero-shot material transfer without additional reference encoders or task-specific fine-tuning [22]. Its extension, DealMaTe, introduces depth-, normal-, and illumination-aware control through multidimensional Shader LoRA and specialized attention mechanisms [23]. Therefore, training-free transfer and a frozen backbone are not, by themselves, novel properties of material rebinding. RATR addresses a complementary setting: instead of adding learned condition-specific branches or constructing an explicit multi-control interface, it reorganizes the two image inputs of an existing frozen editor into surface-oriented and material-oriented roles. It then routes the resulting visual-condition contributions over denoising time. The focus is thus on the roles and timing of available conditioning signals, not on learning a new material representation or control adapter.

Selective Conditioning and Denoising-Time Control

A closely related challenge is preventing a reference image from transferring unintended content together with desired appearance. StyleAligned uses shared attention to maintain visual style consistency, and InstantStyle seeks to reduce content leakage through content subtraction and selective style-feature injection [19, 46]. DEADiff explicitly disentangles style and semantic representations, whereas CSGO learns separate content and style pathways using supervised composition data [38, 49]. StyleSSP modifies the sampling initialization to improve content-layout preservation, and Attention Distillation transfers visual characteristics through an attention-based optimization objective [50, 56]. Conditional Balance selects sensitive attention layers to manage competing style and content conditions [10]. These studies identify feature leakage and interference as important problems, but style and content separation does not directly establish a physically meaningful decomposition of material, geometry, and illumination. RATR accordingly treats its surface and material boundaries as task-oriented visual conditions, not as exact intrinsic-image factors.

Conditioning can also have different effects at different stages of denoising. Prompt-to-Prompt manipulates cross-attention to maintain edit consistency (Hertz et al. [18]); ProSpect associates image attributes with textual embeddings at different generation stages (Zhang et al. [54]). CADS gradually changes the noise applied to conditioning signals to control the diversity–fidelity trade-off, and StyleStudio uses early-stage guidance to stabilize spatial layouts during style transfer (Sadat et al. [42]; Lei et al. [24]). Temporal Concept Dynamics further shows that the success of concept interventions varies with the denoising step, model, and concept (Görgün et al. [15]). Collectively, these works motivate investigating when a condition should influence generation; they do not imply a universal schedule in which geometry always forms before appearance. RATR instead evaluates a task-specific temporal routing policy for its two role-reparameterized visual conditions, leaving the pretrained editor weights and sampling scheduler unchanged.

Method

Figure 2 illustrates RATR. We address material rebinding through two operations: role assignment constructs a target condition that retains structure while weakening its original appearance and a reference condition that emphasizes transferable material cues; temporal routing then coordinates their contributions during denoising. Both operations use a frozen two-image editor without parameter updates.

Figure 2. Overview of RATR. The target and material-reference images are reparameterized into complementary visual inputs: a target-derived surface boundary CsC_s that suppresses inherited appearance while retaining structural evidence, and a reference-derived material evidence boundary ErE_r. Surface-Material Conditional Direction Routing (SM-CDR) adjusts the contributions of their visual-condition residuals throughout denoising, emphasizing structural evidence and material evidence at different stages under a task-specific schedule.

Problem Formulation and Overview

Let Is∈[0,1]H×W×3I_s \in[0,1]^{H \times W \times3} denote the target image and IrI_r the reference material exemplar. Given a material-rebinding instruction PP, the objective is to synthesize an image IoI_o that exhibits the reference material while preserving the target object’s geometry, identity, and fine structural details. We use a frozen two-image editor QθQ_\theta with fixed parameters θ\theta. The target foreground mask MsM_s is estimated once using the U2-Net segmenter in rembg; a reference-side material support M‾r\overline{M}_r is obtained independently.

Under raw two-image conditioning, the editor receives (P,Is,Ir)(P, I_s, I_r). The target image carries both the structure to preserve and the appearance to replace, while the reference contains transferable material cues together with object- and scene-specific content. Increasing either condition’s influence may therefore strengthen useful and interfering information together. We refer to this within-condition mixture as role entanglement.

RATR replaces the raw visual inputs with complementary conditions before editing:

Cs=Rsurf(Is,Ms),C_s = \mathcal{R}_{\mathrm{surf}}(I_s,M_s),
Er=Cmat(Ir,M‾r),E_r = \mathcal{C}_{\mathrm{mat}}(I_r,\overline{M}_r),
I~o=Qθ(Cs,Er,P;Γ),(1)\tilde{I}_o = Q_\theta(C_s,E_r,P;\Gamma), \tag*{(1)}

where CsC_s is the target-side surface condition, ErE_r is the reference-side material evidence condition, Γ\Gamma is the denoising-time routing policy, and I~o\tilde{I}_o is the editor’s decoded result before foreground compositing. We call CsC_s and ErE_r boundaries to emphasize their conditioning roles, not because they are binary edge maps. Their construction favors complementary information without claiming an exact decomposition of geometry and appearance.

The frozen editor encodes the two images through both a vision-language (VL) semantic path and a VAE appearance path. The semantic condition is

h=ΦVL(P,Cs,Er),(2)h = \Phi_{\mathrm{VL}}(P,C_s,E_r), \tag*{(2)}

while the VAE image-conditioned segments are

Zs=Z(Cs),Zr=Z(Er),Z(I)=Pack⁡(Evae(I)),(3)\begin{aligned} Z_s &= Z(C_s), \qquad Z_r = Z(E_r),\\ Z(I) &= \operatorname{Pack}(E_{\mathrm{vae}}(I)), \tag*{(3)} \end{aligned}

where EvaeE_{\mathrm{vae}} is the frozen VAE encoder and Pack⁡(⋅)\operatorname{Pack}(\cdot) arranges its output into image-condition tokens. RATR constructs CsC_s and ErE_r in image space, then routes only the centered residuals of ZsZ_s and ZrZ_r. For each target–reference pair, hh is computed from the reparameterized inputs and held fixed during routing. Neither the VL representation nor the editor weights are modified at individual denoising steps.

At the level of intended roles, the raw condition segments can be described schematically as

Z(Is)≈ZG+ZA+ϵs,Z(Ir)≈Zmat+Zref+ϵr,(4)\begin{aligned} Z(I_s) &\approx Z_G + Z_A + \epsilon_s,\\ Z(I_r) &\approx Z_{\mathrm{mat}} + Z_{\mathrm{ref}} + \epsilon_r, \tag*{(4)} \end{aligned}

where ZGZ_G refers to target structure and identity, ZAZ_A to the original target appearance, ZmatZ_{\mathrm{mat}} to transferable material cues, and ZrefZ_{\mathrm{ref}} to reference-object and scene content. These are conceptual components, not identifiable latent factors. Role assignment seeks to weaken ZAZ_A and ZrefZ_{\mathrm{ref}} without eliminating the information needed for rebinding.

Role Assignment through Visual Condition Construction

Role assignment treats the two inputs differently because they contain different sources of interference. The target needs to retain geometry without anchoring the old material; the reference needs to retain material appearance without imposing its own object or scene. We construct these conditions independently so that a released surface can be reused with different materials and a material evidence boundary can be reused with different targets. We refer to this task-oriented separation of visual conditions as role decoupling.

Albedo-Resolved Surface Release

Albedo-Resolved Surface Release constructs CsC_s using a reference-free release trajectory followed by luminance-only structural restoration. Rather than attempting intrinsic image decomposition, it selects a surface that weakens the original chromatic appearance while preserving the target’s geometry and visible relief.

Reference-free candidate selection. Without introducing IrI_r, we obtain predicted-clean hypotheses x^0(t)\hat{x}_0(t) from a release trajectory at a fixed set of early steps TrelT_{\mathrm{rel}}.

For each target image, the release step is selected by

t⋆=arg⁡min⁡t∈TrelErel(x^0(t);Is,Ms),(5)t^\star= \arg\min_{t\in T_{\mathrm{rel}}} \mathcal{E}_{\mathrm{rel}}\left(\hat{x}_0(t); I_s, M_s\right), \tag*{(5)}

where x^0(t)\hat{x}_0(t) is the clean-image estimate at the corresponding denoising step. The selection energy combines three fixed criteria:

Erel(x;Is,Ms)=λalbEalb(x)+λgeoEgeo(x)+λsupEsup(x).(6)\mathcal{E}_{\mathrm{rel}}(x; I_s, M_s) = \lambda_{\mathrm{alb}}\mathcal{E}_{\mathrm{alb}}(x) + \lambda_{\mathrm{geo}}\mathcal{E}_{\mathrm{geo}}(x) + \lambda_{\mathrm{sup}}\mathcal{E}_{\mathrm{sup}}(x). \tag*{(6)}

The weights and candidate steps are shared across samples; only the minimizing step t⋆t^\star depends on the input target. Let Y(⋅)Y(\cdot) and CbCr(⋅)\mathrm{CbCr}(\cdot) denote luminance and chroma in YCbCr space, c0c_0 neutral chroma, and ∣Ms∣\lvert M_s\rvert the foreground area. The chroma criterion is

Ealb(x)=∥Ms⊙(CbCr(x)−c0)∥1∣Ms∣+ϵ.(7)\mathcal{E}_{\mathrm{alb}}(x) = \frac{\left\lVert M_s \odot\left(\mathrm{CbCr}(x)-c_0\right)\right\rVert_1}{\lvert M_s\rvert+\epsilon}. \tag*{(7)}

It discourages retention of the target’s original color without penalizing luminance, which also carries shape and relief cues. It is a chroma-based selection criterion, not an estimate of physical albedo.

Let B(⋅)\mathcal{B}(\cdot) be the gradient magnitude computed with a 3×33 \times3 Sobel operator. The geometry criterion compares luminance boundaries:

Egeo(x)=∥Ms⊙[B(Y(x))−B(Y(Is))]∥1∣Ms∣+ε.(8)\mathcal{E}_{\mathrm{geo}}(x)=\frac{\lVert M_s\odot[\mathcal{B}(Y(x))-\mathcal{B}(Y(I_s))]\rVert_1}{|M_s|+\varepsilon}. \tag*{(8)}

It favors candidates that preserve the target silhouette and local boundary detail. Finally, with M(⋅)\mathcal{M}(\cdot) denoting the fixed foreground estimator used to obtain MsM_s, the support criterion is

Esup(x)=1−IoU⁡(M(x),Ms).(9)\mathcal{E}_{\mathrm{sup}}(x)=1-\operatorname{IoU}(\mathcal{M}(x),M_s). \tag*{(9)}

This term penalizes foreground erosion, expansion, or displacement. Together, the three criteria select a sample-specific release hypothesis that weakens old color while retaining boundary and support evidence. We denote this hypothesis by Cs0=x^0(t∗)C_s^0=\hat{x}_0(t^*).

Luminance residual restoration. The selected hypothesis can lose fine structural detail. Blending back the RGB target would also restore the original appearance, so we recover only a luminance-side high-frequency residual. With a fixed high-pass operator HP⁡(⋅)\operatorname{HP}(\cdot) and a foreground-aware structural support map WsW_s, we define

Δstr=Ws⊙[HP⁡(Y(Is))−HP⁡(Y(Cs0))].(10)\Delta_{\mathrm{str}}=W_s\odot\left[\operatorname{HP}(Y(I_s))-\operatorname{HP}(Y(C_s^0))\right]. \tag*{(10)}

The restored luminance and final surface condition are

Yrel=Y(Cs0)+λstrΔstr,(11)Y^{\mathrm{rel}}=Y(C_s^0)+\lambda_{\mathrm{str}}\Delta_{\mathrm{str}}, \tag*{(11)}
Cs=Clip⁡[Merge⁡YCbCr(Yrel,CbCr⁡(Cs0))].(12)C_s=\operatorname{Clip}\left[\operatorname{Merge}_{Y\mathrm{CbCr}}\left(Y^{\mathrm{rel}},\operatorname{CbCr}(C_s^0)\right)\right]. \tag*{(12)}

Thus, structural residuals are restored through luminance, whereas chroma remains inherited from the selected release hypothesis. The resulting CsC_s is intended as a geometry-oriented visual condition; it does not recover intrinsic geometry or guarantee removal of every appearance cue.

Material Evidence Isolation

Material Evidence Isolation constructs ErE_r from the reference image without accessing the target. Reference exemplars may contain background layout, object contours, or cast shadows alongside their material cues. We therefore use a lightweight, mode-aware image-space operation that reduces obvious non-material interference while retaining visible material appearance:

Er=Cmat(Ir,M^r),(13)E_r=\mathcal{C}_{\mathrm{mat}}(I_r,\widehat{M}_r), \tag*{(13)}

where M^r\widehat{M}_r is the effective material support and RR denotes the editor-side evidence resolution. When the reference already provides a full-field material observation, the operation is a direct resize:

Er=Resize⁡R(Ir).(14)E_r=\operatorname{Resize}_R(I_r). \tag*{(14)}

For object-centric references, M^r\widehat{M}_r is estimated automatically using the same U2-Net foreground extractor as on the target side. The material-bearing region is retained, while the complementary region is completed with low-frequency context derived from IrI_r. The operator fits the effective support to the material evidence boundary, softly reinforces representative local material cues, and applies weak smoothing and range normalization. References whose material support already occupies most of the image are processed in full-field mode; no pair-specific manual mask is required.

This process reduces conspicuous reference-object and scene cues without claiming to separate material from geometry exactly. In particular, ErE_r remains a continuous image condition, not a texture atlas or an estimate of SVBRDF, lighting, or surface normals. Together with CsC_s, it provides more role-consistent visual evidence; temporal routing determines how that evidence influences the frozen editor.

Surface-Material Conditional Direction Routing

After constructing CsC_s and ErE_r, RATR supplies them to the editor in the fixed order (Cs,Er)(C_s,E_r). Their VAE condition segments Zs∈RNs×dZ_s\in\mathbb{R}^{N_s\times d} and Zr∈RNr×dZ_r\in\mathbb{R}^{N_r\times d} are routed during denoising, while the semantic condition hh in Eq. (2) remains fixed. SMC-DR changes neither the generation latent directly nor the prompt, VL condition, model parameters, positional encodings, or scheduler.

Centered Visual-Condition Residual Modulation

For j∈{s,r}j\in\{s,r\}, we separate each image-condition segment into its token mean and centered residual:

μj=1Nj∑i=1NjZj(i),Zˉj=Zj−μj.(15)\mu_j=\frac{1}{N_j}\sum_{i=1}^{N_j}Z_j^{(i)},\qquad\bar{Z}_j=Z_j-\mu_j. \tag*{(15)}

SM-CDR scales the centered residual, leaving the segment mean unchanged:

Z~j(t)=μj+αj(t)Z‾j.(16)\widetilde{Z}_j(t) = \mu_j + \alpha_j(t)\overline{Z}_j. \tag*{(16)}

The perturbation relative to the original segment is

δZj(t)=Z~j(t)−Zj=(αj(t)−1)Z‾j.(17)\delta Z_j(t) = \widetilde{Z}_j(t) - Z_j = \left(\alpha_j(t) - 1\right)\overline{Z}_j. \tag*{(17)}

Unlike uniform scaling of ZjZ_j, this operation does not change the segment mean. It does not, however, preserve the full latent distribution or isolate individual physical attributes. Its purpose is to regulate the variation carried by each role-specific condition.

Denoising-Time Routing Policy

We use a fixed three-stage policy to change the relative influence of surface and material evidence. With zero-based denoising-step index kk, a total of TT steps, and normalized progress ut=k/(T−1)u_t = k/(T-1), two boundaries u0<u1u_0 < u_1 define

τ(t)={early,ut<u0,mid,u0≤ut<u1,late,ut≥u1.(18)\tau(t) = \begin{cases} \text{early}, & u_t < u_0, \\ \text{mid}, & u_0 \le u_t < u_1, \\ \text{late}, & u_t \ge u_1. \end{cases} \tag*{(18)}

Each stage τ\tau has a pair of gains (αsτ,αrτ)(\alpha_s^\tau,\alpha_r^\tau), giving

αs(t)=αsτ(t),αr(t)=αrτ(t).(19)\alpha_s(t) = \alpha_s^{\tau(t)}, \qquad\alpha_r(t) = \alpha_r^{\tau(t)}. \tag*{(19)}

The complete policy Γ\Gamma consists of the two transition boundaries and the three surface/material gain pairs. For reporting its relative bias, we define the material-routing share

pτ=αrταsτ+αrτ,pΓ={pearly,pmid,plate}.(20)p_\tau= \frac{\alpha_r^\tau}{\alpha_s^\tau+ \alpha_r^\tau}, \qquad p_\Gamma= \{p_{\text{early}}, p_{\text{mid}}, p_{\text{late}}\}. \tag*{(20)}

Smaller pτp_\tau indicates a more surface-oriented stage and larger pτp_\tau a more material-oriented stage. The share summarizes a policy but does not determine it uniquely: the absolute gains are also part of Γ\Gamma. Our schedule emphasizes structural support earlier and material appearance later. This is a task-specific allocation, not an assumption that diffusion models universally separate geometry and material at fixed times.

Local Interpretation of Routed Conditioning

To clarify what SM-CDR changes, let ztz_t be the current generation latent and vθv_\theta the frozen denoiser’s conditional prediction direction. At the same ztz_t and timestep tt, standard and routed conditioning give

v^tstd=vθ(zt,t;h,Zs,Zr),(21)\widehat{v}_t^{\mathrm{std}} = v_\theta(z_t,t;h,Z_s,Z_r), \tag*{(21)}
v^troute=vθ(zt,t;h,Z~s(t),Z~r(t)).(22)\widehat{v}_t^{\mathrm{route}} = v_\theta\left(z_t,t;h,\widetilde{Z}_s(t),\widetilde{Z}_r(t)\right). \tag*{(22)}

The unchanged scheduler advances the sample using v^troute\widehat{v}_t^{\mathrm{route}}. For small perturbations δZs(t)\delta Z_s(t) and δZr(t)\delta Z_r(t), a first-order expansion around the static role-reparameterized conditions gives

Δvt≡v^troute−v^tstd≈(αs(t)−1)Js(t)Z‾s+(αr(t)−1)Jr(t)Z‾r,(23)\begin{aligned} \Delta v_t &\equiv\widehat{v}_t^{\mathrm{route}} - \widehat{v}_t^{\mathrm{std}} \\ &\approx\left(\alpha_s(t)-1\right)J_s(t)\overline{Z}_s \\ &\quad+ \left(\alpha_r(t)-1\right)J_r(t)\overline{Z}_r, \tag*{(23)} \end{aligned}

where Js(t)J_s(t) and Jr(t)J_r(t) are the local Jacobian maps with respect to the two visual condition segments, evaluated with hh and ztz_t fixed. This expression explains how the gains locally change the predicted direction through the two condition residuals. It does not imply that the Jacobians are orthogonal, that material and geometry are identifiable in VAE space, or that the resulting denoising trajectory is globally optimal.

The dependence on role assignment is equally important. Before reparameterization, the target residual also carries the old appearance, and the reference residual may carry its original object and scene. Scaling those raw residuals can strengthen the unwanted cues along with the desired ones. By constructing CsC_s and ErE_r first, RATR reduces these sources of interference before applying the time-dependent gains. The actual extent of interference reduction is assessed empirically, rather than inferred from the local expansion.

Inference Procedure

At inference time, the target mask MsM_s is computed once, and the reference-free release trajectory produces the early hypotheses {x~0(t):t∈Trel}\{\widetilde{x}_0(t):t\in\mathcal{T}_{\mathrm{rel}}\}. Eq. (5) selects t∗t^* separately for target; luminance restoration then produces CsC_s. Independently, Material Evidence Isolation constructs ErE_r. The frozen editor encodes (P,Cs,Er)(P,C_s,E_r) into hh, ZsZ_s, and ZrZ_r. At each denoising step, SM-CDR applies the scheduled gains to the centered VAE residuals immediately before evaluating the denoiser, while retaining the original scheduler.

The editor’s decoded result I~o\tilde{I}_o follows the first visual input’s generation canvas. After resizing it to the target resolution when necessary, the edited foreground is composited with the unchanged target background using the fixed mask:

Io=Ms⊙I~o↑+(1−Ms)⊙Is,(24)I_o = M_s \odot\tilde{I}_o^{\uparrow} + (1 - M_s) \odot I_s, \tag*{(24)}

where I~o↑\tilde{I}_o^{\uparrow} is the decoded image aligned to the original target resolution. The entire procedure uses fixed model parameters and does not require material-specific training or pair-specific manual masks.

Experiments

Setup

We evaluate RATR using 30 target images and 30 material exemplars covering diverse object geometries and material appearances. All methods are evaluated under the same protocol. RATR uses Qwen-Image-Edit-2511 (Qwen Team [39]) as the frozen two-image editor with 30 denoising steps. We compare with Qwen-Edit, the same frozen editor applied directly to the raw target/reference pair, and representative transfer baselines, including DealMaTe (Huang et al. [23]), MARBLE (Cheng et al. [9]), MaterialFusion (Garifullin et al. [13]), ZeST (Cheng et al. [8]), IP-Adapter SDXL and IP-Adapter SD1.5 with ControlNet (Ye et al. [52]; Zhang et al. [53]), and InstructPix2Pix with IP-Adapter (Brooks et al. [4]; Ye et al. [52]).

For all main experiments, we use a fixed three-stage routing schedule with transitions at normalized denoising progress u0=0.30u_0 = 0.30 and u1=0.70u_1 = 0.70. The surface/material gains (αs,ατ)(\alpha_s, \alpha_\tau) are set to (1.20,0.80)(1.20, 0.80), (0.85,1.35)(0.85, 1.35), and (0.65,1.60)(0.65, 1.60) for the early, middle, and late stages, respectively, yielding the material-routing-share profile pΓ={0.40,0.61,0.71}p_\Gamma= \{0.40, 0.61, 0.71\}. Surface release is parameterized by the fixed early candidate set Trel={2,4,6,8,10}\mathcal{T}_{\mathrm{rel}} = \{2, 4, 6, 8, 10\} on the 30-step trajectory. Using normalized YCbCr, we set the neutral chroma anchor to c0=(0.5,0.5)c_0 = (0.5, 0.5) and use fixed release-energy weights λalb=1.0\lambda_{\mathrm{alb}} = 1.0, λgeo=1.2\lambda_{\mathrm{geo}} = 1.2, and λsup=0.8\lambda_{\mathrm{sup}} = 0.8. For each target image, the release step is automatically selected according to Eq. (5) by minimizing the three-term release energy over Trel\mathcal{T}_{\mathrm{rel}}. The resulting surface condition is reused across different material references for the same target. We preserve the input aspect ratio and limit the longer side to 1024 pixels. All main results use the same predefined hyperparameters and

automatic release-selection procedure, without manual per-example tuning. RATR and the backbone-matched Qwen-Edit baseline use the same material-rebinding instruction across all target–material pairs.

Main Results

Qualitative Comparison

Figure 3 compares RATR with Qwen-Edit and representative material-transfer methods across diverse target–material pairs. One recurring failure of existing methods is incomplete replacement of the target’s original appearance. For example, Qwen-Edit largely retains the pumpkin’s orange appearance despite the blue fabric reference, leaves the marble bust nearly unchanged with the wood reference (the tenth row of Figure 3), and preserves the cow’s brown appearance despite the metallic exemplar (the fourth-to-last row of Figure 3). In contrast, RATR gives the pumpkin a blue fabric-like appearance while preserving its grooves and stem, imparts wood grain to the bust while retaining its recognizable facial structure, and reproduces metallic reflections on the cow while preserving its horns and legs.

Qualitative comparison across methods

Figure 3. Qualitative comparison across methods. RATR achieves more faithful material transfer while better preserving target geometry, whereas competing methods often suffer from incomplete material transfer or structural artifacts. More qualitative results are provided in Project.

Other competing methods produce more pronounced appearance changes but may introduce reference-specific structures or distort target details. In the fifth row of Figure 3, DealMaTe introduces vase-like handles from the green ceramic reference, whereas RATR adopts the reference material appearance while preserving the clock’s rectangular frame and face. For the camera with a red fabric reference (the third row of Figure 3), MaterialFusion transforms the lens and body into folded fabric-like forms, while RATR maintains the recognizable lens barrel and camera silhouette. Similar differences appear in the lucky cat with a rock reference and the plush elephant with a crumpled-paper reference. Some methods distort the cat’s raised paw and facial features or weaken the elephant’s distinctive ears and trunk. In contrast, RATR better preserves these recognizable structures while adapting their surface appearances to the corresponding materials.

Quantitative Comparison

Table 1 reports SSIM, LPIPS, DreamSim, Edge SSIM, and Edge MAE. SSIM, LPIPS, and DreamSim assess target-side structural and perceptual consistency, while both Edge SSIM and Edge MAE provide edge-based diagnostics of silhouette and boundary preservation.

MethodSSIM ↑\uparrowLPIPS ↓\downarrowDreamSim ↓\downarrowEdge SSIM ↑\uparrowEdge MAE ↓\downarrow
DealMaTe0.70610.39270.38870.71900.0158
Qwen-Edit0.70510.35600.34300.69480.0163
MARBLE0.63560.39570.42640.64640.0204
MaterialFusion0.71780.34190.38860.69500.0162
ZeST0.66480.37810.43140.67220.0186
IP-A. SDXL+CN0.61520.41420.47950.64630.0204
IP-A. SD1.5+CN0.59780.44070.53850.61910.0228
Inst-P2P+IP-A.0.69060.39280.51440.70420.0163
RATR (Ours)0.72920.31270.34140.72820.0153

Table 1. Target-side quantitative comparison results of our method and the existing competitors. Higher is better for both SSIM and Edge SSIM; lower is better for LPIPS, DreamSim, and Edge MAE. The best results are marked in bold.

RATR achieves the best aggregate results on all five reported target-side preservation metrics in Table 1. Its SSIM, LPIPS, and DreamSim results indicate greater structural and perceptual consistency with the target, while Edge SSIM and Edge MAE show better preservation of silhouettes and boundaries. Table 2 further shows that RATR achieves the highest mean scores on all three reference-side metrics, including CLIP, DINOv3, and Materialistic. Together, these results demonstrate that improved target preservation is accompanied by leading reference-material correspondence, rather than a reduction in material alignment. This joint performance supports the qualitative observation that RATR mitigates the trade-off between target preservation and material transfer. The observed balance is consistent with its role-assignment and temporal-routing design, which reduces interference from the original target appearance while coordinating structural and material evidence during denoising.

MethodCLIP ↑\uparrowDINOv3 ↑\uparrowMaterialistic ↑\uparrow
DealMaTe0.46170.50190.5974
Qwen-Edit0.49280.53220.5323
MARBLE0.50580.52490.5643
MaterialFusion0.48100.53130.4594
ZeST0.52470.54560.5817
IP-A-SDXL-CN0.52490.54370.6060
IP-A-SD15-CN0.52460.54700.6217
InstP2P-IP-A0.51750.53510.5958
RATR (Ours)0.52510.54950.6225

Table 2. Reference-material correspondence comparison. Higher is better for all metrics.

Human Evaluation

To complement the automated similarity metrics, we conduct a blind human evaluation along three perceptual dimensions: Material Fidelity, Structure Preservation, and Overall Preference. Material Fidelity measures how closely an output reproduces the reference material’s texture, color variation, gloss or roughness, surface detail, and natural coverage. For example, transferring blue fabric to a pumpkin should reproduce fabric-like surface qualities, not merely recolor the pumpkin blue. Structure Preservation evaluates retention of the target’s identity, silhouette, geometry, pose, major components, relative layout, and boundaries. For a clock conditioned on a green ceramic vase, the rectangular frame and clock face should remain intact, while transferring the vase’s handles would constitute a structural error. Overall Preference assesses the complete rebinding result by jointly considering material appearance, structural integrity, visual naturalness, and artifacts.

12 graduate students with research backgrounds in computer vision and graphics participated as independent evaluators. The evaluation covers 272 target–material pairs and nine compared methods. For each pair, evaluators view the target image, the material reference, and anonymized, randomly ordered method outputs under the same interface. Each output is assessed separately on all three criteria using an integer scale from 1 to 5, where 1 denotes a clear failure, 3 indicates partial success with noticeable defects, and 5 denotes a very high degree of satisfaction; 2 and 4 represent intermediate levels. Different methods may receive the same rating. The protocol allows up to 88,128 criterion-level ratings, and the reported means are computed from valid ratings for each method and criterion.

Figure 4 provides representative examples of how material fidelity and structural preservation are jointly assessed across different target geometries and material appearances. The iridescent teapot and fabric-conditioned lucky cat illustrate the importance of reproducing distinctive surface characteristics while retaining recognizable components such as the teapot’s spout and handle or the cat’s raised paw and facial features. The ceramic-conditioned figurine and ornamental vessel further demonstrate the need to preserve target-specific geometry, pose, and surface relief without introducing structural characteristics from the material reference.

Representative human ratings across four target–material pairs

Figure 4. Representative human ratings across four target–material pairs. Each row presents the target, material reference, and nine anonymized outputs (A–I), together with their five-point Material Fidelity, Structure Preservation, and Overall Preference ratings. RATR corresponds to G, D, A, and E from top to bottom. The displayed scores illustrate individual cases, while aggregate evaluation results are reported in Table 3.

Table 3 presents the human evaluation results of 12 evaluators. From the results, we observe that RATR achieves the highest mean rating on all three criteria, exceeding the strongest competitor in each criterion by 0.768, 0.423, and 0.945 points, respectively. Its simultaneous advantage in material fidelity and structure preservation indicates that improved reference-material adoption is accompanied by better retention of target identity and geometry. The highest Overall Preference rating further suggests that this balance translates into higher perceived quality of the complete rebinding results.

MethodMaterial Fidelity ↑\uparrowStructure Preservation ↑\uparrowOverall Preference ↑\uparrow
DealMaTe3.009 ±\pm 0.1383.932 ±\pm 0.6223.165 ±\pm 0.234
Qwen-Edit3.239 ±\pm 0.2293.792 ±\pm 0.5123.328 ±\pm 0.176
MARBLE3.221 ±\pm 0.2073.381 ±\pm 0.4153.137 ±\pm 0.205
MaterialFusion2.607 ±\pm 0.3373.068 ±\pm 0.4572.626 ±\pm 0.344
ZeST3.203 ±\pm 0.3413.158 ±\pm 0.4043.016 ±\pm 0.285
IP-A. SDXL+CN3.222 ±\pm 0.3122.581 ±\pm 0.7532.531 ±\pm 0.573
IP-A. SD1.5+CN3.325 ±\pm 0.4552.361 ±\pm 0.7862.488 ±\pm 0.575
Inst-P2P+IP-A.2.440 ±\pm 0.4302.450 ±\pm 0.4362.070 ±\pm 0.402
RATR (Ours)4.093 ±\pm 0.3814.355 ±\pm 0.4604.273 ±\pm 0.355

Table 3. Human evaluation of material fidelity, target-structure preservation, and overall preference. Scores are reported as the mean ±\pm standard deviation of evaluator-level mean ratings on a 1–5 scale; higher scores indicate better performance.

Beyond aggregate comparisons, Figure 5 provides complementary analyses of RATR’s performance across evaluators and target–material pairs. Figure 5(a) shows the distributions of evaluator-level mean ratings for the three perceptual criteria. For both Structure Preservation and Overall Preference, 11 of the 12 evaluators assign RATR mean ratings of at least 4.0. These distributions complement the aggregate scores in Table 3, showing that the favorable structural and overall assessments extend across the evaluator group rather than being concentrated in a few individual judgments.

Human-rating distributions and pair-level rankings for RATR

Figure 5. Human-rating distributions and pair-level rankings for RATR. (a) Evaluator-level mean ratings across the three criteria; dots represent individual evaluators, boxes indicate interquartile ranges, and horizontal lines denote medians. (b) Proportions of target–material pairs where RATR ranks strictly first, ties for first, or is not first among the nine methods.

Figure 5(b) further examines the prevalence of RATR’s advantage across different target–material combinations. Based on the mean ratings for each pair and criterion, RATR ranks strictly first among all nine methods on 226 of 272 pairs (83.1%) in Material Fidelity, 202 pairs (74.3%) in Structure Preservation, and 262 pairs (96.3%) in Overall Preference. In particular, the exceptionally high first-place rate in Overall Preference indicates that RATR’s perceptual advantage extends to nearly all evaluated combinations. These results demonstrate that its leading mean scores are accompanied by broad pair-level superiority, rather than being driven by a limited subset of particularly favorable examples.

The representative scoring, aggregate comparisons, evaluator-level distributions, and pair-level rankings provide complementary perceptual evidence of RATR’s effectiveness. The consistent advantages in material fidelity and structural preservation support the central objective of role-consistent material rebinding: transferring reference appearance while retaining target-specific geometry and identity. These observations are consistent with the coordinated role assignment and temporal routing formulation, whose individual contributions are further examined in the ablation study.

Ablation Study

In this section, we examine the contributions of role decoupling and SM-CDR, followed by an analysis of different SM-CDR routing strategies.

Figure 6 examines the two central mechanisms of RATR: role decoupling and temporal routing. Removing role decoupling restores the raw target image in place of CsC_s, while retaining the reference-side material evidence ErE_r and SM-CDR. This reintroduces the coupling between target geometry and its original appearance, weakening the intended complementarity of the two visual conditions. In contrast, removing SM-CDR preserves the role-specific visual conditions but returns to static use of their VAE condition segments without temporal modulation.

Ablation of target-side role decoupling and SM-CDR

Figure 6. Ablation of target-side role decoupling and SM-CDR.

The visual differences reveal the complementary effects of these two mechanisms. Without role decoupling, the raw target condition carries both desired geometry and undesired source appearance, making the output more susceptible to original color, gloss, or texture retention. Without SM-CDR, the role-specific conditions lack temporal coordination, leading to less coherent material adoption and structural preservation in the shown examples. The full model combines role-consistent condition construction with temporal routing, yielding clearer material expression while retaining the target-specific structures.

Figure 7 analyzes the controllability of SM-CDR by varying the material-routing-share profile pΓp_{\Gamma}. A structure-dominant route with a smaller material share preserves more surface detail and boundary support but under-transfers the reference appearance, whereas a material-forward route strengthens the transferred material at the cost of weaker boundary detail and local structure. The default route, pΓ={0.40,0.61,0.71}p_{\Gamma} = \{0.40, 0.61, 0.71\}, provides a strong visual balance among the shown settings by retaining structural support early and increasing material influence later.

Figure 7. SM-CDR gain ablation under different material-routing-share profiles pΓp_{\Gamma}.

Table 4 further reveals how routing emphasis affects the balance between structural preservation and material transfer. Structure-lock performs best on all five target-preservation metrics but yields weaker material correspondence. This suggests that strong structural conditioning constrains appearance changes as well as geometric deviations, preserving the target more closely while limiting reference-material adoption. Conversely, Material-forward achieves the highest material-side scores at the expense of structural consistency, indicating that stronger material expression can also alter target boundaries and fine details.

Routing StrategySSIM ↑\uparrowLPIPS ↓\downarrowDreamSim ↓\downarrowEdge SSIM ↑\uparrowEdge MAE ↓\downarrowCLIP ↑\uparrowDINOv3 ↑\uparrowMaterialistic ↑\uparrow
Structure-lock0.75020.29490.29100.74760.01350.47220.51400.5062
Default0.72920.31270.34140.72820.01530.52510.54950.6225
Material-forward0.70980.33260.36870.70870.01710.53900.56330.6531

Table 4. Quantitative comparison of SM-CDR routing strategies.

The default strategy balances these competing effects. Although it does not maximize either group of metrics, it preserves structure more effectively than Material-forward and transfers material more faithfully than Structure-lock. Importantly, these results show that role-specific condition construction does not eliminate the need to regulate their influence: even after reparameterization, excessive emphasis on either condition can undermine the complementary objective. Consistent with the visual comparisons in Figure 7, the default schedule provides a practical balance between structural constraint and appearance adaptation, while the alternative routes demonstrate the controllability of this trade-off.

Additional Qualitative Visualizations

Figure 8 and Figure 9 provide additional qualitative results under complementary material-rebinding settings. In each grid, material references appear in the top row, target objects in the left column, and the remaining cells show the corresponding rebinding results.

Additional qualitative results across diverse target geometries and material appearances

Figure 8. Additional qualitative results across diverse target geometries and material appearances.

Additional qualitative results with patterned and object-centric material references

Figure 9. Additional qualitative results with patterned and object-centric material references.

Figure 8 emphasizes diversity in both target geometry and material appearance, covering substantially different object categories together with materials ranging from wood and fabric to metallic, patterned, and glossy surfaces. Across these combinations, RATR preserves target-specific geometry and local structural characteristics while adapting the visible surface appearance to the reference material.

Figure 9 further considers patterned and object-centric material references, where the reference may contain stronger spatial patterns or reference-specific structural cues. The results show that the same material appearance can be transferred across targets with different geometries while maintaining the target structure and limiting visible reference-object interference.

Scope and Limitations

RATR is designed as an inference-time material rebinding method for a frozen two-image editor. Its goal is to reorganize the target and reference visual conditions into a more role-consistent form, rather than to train a material-specific generator or recover physically based material parameters. Consequently, its performance is bounded by the material and editing priors already available in the frozen editor. For materials that require highly precise physical effects, such as complex refraction, mirror-like multi-bounce reflection, or very fine anisotropic microstructure, RATR can improve the organization and timing of visual evidence, but it does not guarantee physically exact material synthesis. As illustrated in Figure 10(a), the reference color and surface characteristics can be transferred while the resulting appearance remains partly influenced by the target’s original shading. In particular, fine-scale specular highlights and material-specific reflection patterns may not be faithfully reproduced, as RATR does not explicitly recover material reflectance or scene illumination.

Figure 10. Representative limitations of RATR. (a) Residual dependence of material reflectance on target illumination. (b) Pair-dependent material adoption under shared routing. (c) Limited reproduction of material-specific geometric relief under target-structure preservation.

SM-CDR provides a controllable routing interface rather than a global optimization guarantee. The material routing share can adjust the balance between material expression and surface-detail preservation, but a single fixed policy cannot be optimal for every target–material pair. Figure 10(b) illustrates this variation, where the reference colors are successfully adopted while their characteristic fine-scale textures remain less pronounced than the target’s original surface details. Although the default routing policy provides a favorable overall balance, its effects can vary with the geometry and appearance characteristics of individual target–material combinations. Very material-forward routes may weaken boundary details, while overly surface-dominant routes may under-transfer the reference material.

The albedo-resolved surface boundary is also a role-specific condition, not a full intrinsic decomposition. It suppresses target-side chromatic appearance while preserving geometry-dominant luminance structure. This design is effective for reducing old-color retention, but it may be less suitable when the target identity is itself defined by non-geometric color markings, printed text, or semantic logos. In such cases, aggressively weakening source chroma may remove information that is visually important to the object. RATR mitigates this by restoring luminance-side structural residuals, but it does not explicitly reason about semantic markings. Moreover, preserving the target geometry can limit the reproduction of materials with pronounced three-dimensional surface relief. As shown in Figure 10(c), the characteristic color and surface appearance of the reference are transferred, while prominent protrusions and angular material structures are less fully reproduced. This reflects the intended scope of RATR as a geometry-preserving appearance-rebinding method rather than a material-driven geometry reconstruction approach.

Conclusion

We presented RATR, a training-free material rebinding framework that addresses the coupled structural and appearance information in reference-conditioned image editing through role assignment and temporal routing. By constructing complementary surface-oriented and material-oriented visual conditions, RATR reduces interference from the original target appearance and reference-specific content. SM-CDR further coordinates their centered VAE condition residuals throughout denoising, allowing structural preservation and material adoption to be balanced within a frozen two-image editor.

Extensive evaluations demonstrate improved target-side structural preservation alongside leading reference-side material fidelity, with human assessments further supporting the perceptual quality of the resulting material rebinding. The ablation results highlight the complementary roles of condition construction and temporal coordination. Together, these findings suggest that effective material rebinding depends not only on the visual evidence provided to a generative editor, but also on how that evidence is assigned and coordinated during synthesis.

Statements and Declarations

Funding. This research was supported in part by the National Natural Science Foundation of China under Grant 62576313, the Key R&D Program of Zhejiang Province under Grant 2026C01016, and the Key R&D Program of Ningbo under Grant 2026T018.

Competing interests. The authors declare that they have no relevant financial or non-financial interests to disclose.

Data availability. The data supporting the findings of this study are available from the corresponding author upon reasonable request.

Code availability. The implementation of RATR is available from the corresponding author upon reasonable request.

References

  1. [1]Avrahami O, Lischinski D, Fried O (2022) Blended diffusion for text-driven editing of natural images. In: CVPR, pp 18208–18218DOI
  2. [2]Barron JT, Malik J (2015) Shape, illumination, and reflectance from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(8):1670–1687. https://doi.org/10.1109/TPAMI.2014.2377712
  3. [3]Bell S, Bala K, Snavely N (2014) Intrinsic images in the wild. ACM Transactions on Graphics 33(4):159:1–159:12. https://doi.org/10.1145/2601097.2601206
  4. [4]Brooks T, Holynski A, Efros AA (2023) InstructPix2Pix: Learning to follow image editing instructions. In: CVPR, pp 18392–18402arxiv.org/abs/2211.09800
  5. [5]Cai Z, Weng S, Xia Y, et al (2025) PhyS-Edit: Physics-aware semantic image editing with text description. In: CVPR, pp 7867–7876DOI
  6. [6]Cao M, Wang X, Qi Z, et al (2023) MasaCtrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In: ICCV, pp 22560–22570
  7. [7]Chen X, Huang L, Liu Y, et al (2024) AnyDoor: Zero-shot object-level image customization. In: CVPR, pp 6593–6602arxiv.org/abs/2307.09481
  8. [8]Cheng TY, Sharma P, Markham A, et al (2024) ZeST: Zero-shot material transfer from a single image. In: ECCV. Springer, pp 370–386, https://doi.org/10.1007/978-3-031-73232-4_21
  9. [9]Cheng TY, Sharma P, Boss M, et al (2025) MARBLE: Material recomposition and blending in CLIP-space. In: CVPR, pp 13061–13071arxiv.org/abs/2506.05313
  10. [10]Cohen NZ, Nir O, Shamir A (2025) Conditional balance: Improving multi-conditioning trade-offs in image generation. In: CVPR, pp 2641–2650arxiv.org/abs/2412.19853
  11. [11]Couairon G, Verbeek J, Schwenk H, et al (2023) DiffEdit: Diffusion-based semantic image editing with mask guidance. In: ICLRarxiv.org/abs/2210.11427
  12. [12]Deschaintre V, Aittala M, Durand F, et al (2018) Single-image SVBRDF capture with a rendering-aware deep network. ACM Transactions on Graphics 37(4):128:1–128:15. https://doi.org/10.1145/3197517.3201378
  13. [13]Garifullin K, Nikolaev M, Kuznetsov A, et al (2025) MaterialFusion: High-quality, zero-shot, and controllable material transfer with diffusion models. arXiv preprint arXiv:2502.06606 https://doi.org/10.48550/arXiv.2502.06606, authors report CVPR 2025 acceptance; archival proceedings metadata not independently verified, arXiv:2502.06606
  14. [14]Goel V, Peruzzo E, Jiang Y, et al (2024) PAIR Diffusion: A comprehensive multimodal object-level image editor. In: CVPR, pp 8609–8618arxiv.org/abs/2303.17546
  15. [15]Görgün A, Sammani F, Deligiannis N, et al (2026) Temporal concept dynamics in diffusion models via prompt-conditioned interventions. In: ICLR, arXiv:2512.08486arxiv.org/abs/2512.08486
  16. [16]Guerrero P, Hašan M, Sunkavalli K, et al (2022) MatFormer: A generative model for procedural materials. ACM Transactions on Graphics 41(4):46:1–46:12. https://doi.org/10.1145/3528223.3530173
  17. [17]Guo Y, Smith C, Hašan M, et al (2020) MaterialGAN: Reflectance capture using a generative SVBRDF model. ACM Transactions on Graphics 39(6):254:1–254:13. https://doi.org/10.1145/3414685.3417779
  18. [18]Hertz A, Mokady R, Tenenbaum J, et al (2023) Prompt-to-prompt image editing with cross-attention control. In: ICLRarxiv.org/abs/2208.01626
  19. [19]Hertz A, Voynov A, Fruchter S, et al (2024) Style aligned image generation via shared attention. In: CVPR, pp 4775–4785arxiv.org/abs/2312.02133
  20. [20]Ho J, Jain AN, Abbeel P (2020) Denoising diffusion probabilistic models. In: NeurIPS, vol 33. Curran Associates, Inc., pp 6840–6851arxiv.org/abs/2006.11239
  21. [21]Huang L, Chen D, Liu Y, et al (2023) Composer: Creative and controllable image synthesis with composable conditions. In: ICML, Proceedings of Machine Learning Research, vol 202. PMLR, pp 13753–13773arxiv.org/abs/2302.09778
  22. [22]Huang N, Liu H, Lin Y, et al (2025) MaTe: Images are all you need for material transfer via diffusion transformer. In: ICCV, pp 15117–15126, https://doi.org/10.1109/ICCV51701.2025.01403
  23. [23]Huang N, Lin Y, Guo J, et al (2027) DealMaTe: Multi-dimensional material transfer via diffusion transformer. ACM Transactions on Graphics 46(1):1:1–1:14. https://doi.org/10.1145/3817061, published online in 2026; assigned to the February 2027 issue by ACM
  24. [24]Lei M, Song X, Zhu B, et al (2025) StyleStudio: Text-driven style transfer with selective control of style elements. In: CVPR, pp 23443–23452arxiv.org/abs/2412.08503
  25. [25]Levy S, Aharoni E, Levy M, et al (2025) CoatFusion: Controllable material coating in images. arXiv preprint arXiv:251202143 https://doi.org/10.48550/arXiv.2512.02143, listed by an author as accepted for SIGGRAPH Asia 2026; final TOG article metadata not yet verified, arXiv:2512.02143
  26. [26]Li D, Li J, Hoi SCH (2023) BLIP-Diffusion: Pre-trained subject representation for controllable text-to-image generation and editing. In: NeurIPS, pp 30146–30166, https://doi.org/10.5220/075280-1312
  27. [27]Liao J, Yao Y, Yuan L, et al (2017) Visual attribute transfer through deep image analogy. ACM Transactions on Graphics 36(4):120:1–120:15. https://doi.org/10.1145/3072959.3073683
  28. [28]Liu G, Ceylan D, Yumer E, et al (2017) Material editing using a physically based rendering network. In: ICCV, pp 2261–2269arxiv.org/abs/1708.00106
  29. [29]Lopes I, Pizzati F, de Charette R (2024) Material Palette: Extraction of materials from a single image. In: CVPR, pp 4379–4388arxiv.org/abs/2311.17060
  30. [30]Lopes I, Deschaintre V, Hold-Geoffroy Y, et al (2025) MatSwap: Light-aware material transfers in images. Computer Graphics Forum 44(4):e70168. https://doi.org/10.1111/cgf.70168
  31. [31]Meng C, He Y, Song Y, et al (2022) SDEdit: Guided image synthesis and editing with stochastic differential equations. In: ICLRarxiv.org/abs/2108.01073
  32. [32]Mo S, Mu F, Lin KH, et al (2024) FreeControl: Training-free spatial control of any text-to-image diffusion model with any condition. In: CVPR, pp 7465–7475arxiv.org/abs/2312.07536
  33. [33]Mokady R, Hertz A, Aberman K, et al (2023) Null-text inversion for editing real images using guided diffusion models. In: CVPR, pp 6038–6047arxiv.org/abs/2211.09794
  34. [34]Mou C, Wang X, Xie L, et al (2024) T2I-Adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In: AAAI, pp 4296–4304, https://doi.org/10.1609/aaai.v38i5.28226
  35. [35]Peebles W, Xie S (2023) Scalable diffusion models with transformers. In: ICCV, pp 4195–4205arxiv.org/abs/2212.09748
  36. [36]Pharr M, Jakob W, Humphreys G (2016) Physically Based Rendering: From Theory to Implementation, 3rd edn. Morgan Kaufmann
  37. [37]Podell D, English Z, Lacey K, et al (2024) SDXL: Improving latent diffusion models for high-resolution image synthesis. In: ICLRarxiv.org/abs/2307.01952
  38. [38]Qi T, Fang S, Wu Y, et al (2024) DEADiff: An efficient stylization diffusion model with disentangled representations. In: CVPR, pp 8693–8702arxiv.org/abs/2403.06951
  39. [39]Qwen Team (2025) Qwen-Image-Edit-2511. Hugging Face model card
  40. [40]Radford A, Kim JW, Hallacy C, et al (2021) Learning transferable visual models from natural language supervision. In: ICML, Proceedings of Machine Learning Research, vol 139. PMLR, pp 8748–8763arxiv.org/abs/2103.00020
  41. [41]Rombach R, Blattmann A, Lorenz D, et al (2022) High-resolution image synthesis with latent diffusion models. In: CVPR, pp 10684–10695arxiv.org/abs/2112.10752
  42. [42]Sadat S, Buhmann J, Bradley D, et al (2024) CADS: Unleashing the diversity of diffusion models through condition-annealed sampling. In: ICLRarxiv.org/abs/2310.17347
  43. [43]Sartor S, Peers P (2023) MatFusion: A generative diffusion model for SVBRDF capture. In: SIGGRAPH Asia, pp 86:1–86:10, https://doi.org/10.1145/3610548.3618194
  44. [44]Sharma P, Jampani V, Li Y, et al (2024) Alchemist: Parametric control of material properties with diffusion models. In: CVPR, pp 24130–24141arxiv.org/abs/2312.02970
  45. [45]Tumanyan N, Geyer M, Bagon S, et al (2023) Plug-and-play diffusion features for text-driven image-to-image translation. In: CVPR, pp 1921–1930arxiv.org/abs/2211.12572
  46. [46]Wang H, Spinelli M, Wang Q, et al (2024) InstantStyle: Free lunch towards style-preserving in text-to-image generation. arXiv preprint arXiv:2404.02733 https://doi.org/10.48550/arXiv.2404.02733, arXiv:2404.02733
  47. [47]Wang X, Li H, Fang H, et al (2025) LineArt: A knowledge-guided training-free high-quality appearance transfer for design drawing with diffusion model. In: CVPR, pp 2912–2923, https://doi.org/10.1109/CVPR52734.2025.00277
  48. [48]Wu C, Li J, Zhou J, et al (2025) Qwen-Image technical report. arXiv preprint arXiv:2508.02324 https://doi.org/10.48550/arXiv.2508.02324, arXiv:2508.02324
  49. [49]Xing P, Wang H, Sun Y, et al (2025) CSGO: Content-style composition in text-to-image generation. In: NeurIPS, pp 111464–111504, https://doi.org/10.52202/085713-3363
  50. [50]Xu R, Xi W, Wang X, et al (2025) StyleSSP: Sampling startpoint enhancement for training-free diffusion-based method for style transfer. In: CVPR, pp 18260–18269arxiv.org/abs/2501.11319
  51. [51]Yang B, Gu S, Zhang B, et al (2023) Paint by example: Exemplar-based image editing with diffusion models. In: CVPR, pp 18381–18391arxiv.org/abs/2211.13227
  52. [52]Ye H, Zhang J, Liu S, et al (2023) IP-Adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:230806721 https://doi.org/10.48550/arXiv.2308.06721, arXiv:2308.06721
  53. [53]Zhang L, Rao A, Agrawala M (2023a) Adding conditional control to text-to-image diffusion models. In: ICCV, pp 3836–3847arxiv.org/abs/2302.05543
  54. [54]Zhang Y, Dong W, Tang F, et al (2023b) ProSpect: Prompt spectrum for attribute-aware personalization of diffusion models. ACM Transactions on Graphics 42(6):244:1–244:14. https://doi.org/10.1145/3618342
  55. [55]Zhao S, Chen D, Chen YC, et al (2023) Uni-ControlNet: All-in-one control to text-to-image diffusion models. In: NeurIPS, pp 11127–11150, https://doi.org/10.52202/075280-0491
  56. [56]Zhou Y, Gao X, Chen Z, et al (2025) Attention distillation: A unified approach to visual characteristics transfer. In: CVPR, pp 18270–18280arxiv.org/abs/2502.20235

Paper details

vlnc:W_ty774bx9
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents