betaFeedback

WorksComputer Science

Variational Optimal Control for Diffusion Guidance

Abstract

We present a variational control framework that connects sampling-time guidance and reward-based model adaptation through a shared Jacobian pullback. Starting from terminal reward maximization with a path-space KL penalty, the Gibbs variational principle and Doob transform yield a correction to the generative dynamics. A local denoising approximation exposes the pullback that transports output preferences to the controlled variable. This construction gives Variational Optimal Guidance (VOG) in state space and extends to Distilled Variational Optimal Control (DVOC) by amortizing locally improved endpoints into model parameters. The resulting controls share a local variational rule: VOG acts through the denoising Jacobian, while DVOC realizes the parameter pullback by endpoint distillation. Under a linear MMSE model, VOG's exact correction lies in the data subspace. On ImageNet, VOG reduces DiT-XL/2 FID from 2.27 to 2.04 at 250 sampling steps and from 7.58 to 4.44 at 25 steps. After 20 updates on Z-Image-Turbo, DVOC improves HPSv3++ and four held-out evaluator scores over the untrained model, with the highest corresponding means among the trained methods compared. These results connect inference-time control and reward adaptation through a common geometric principle.

Introduction

Diffusion models have become powerful image generators by learning to transform noise into samples from a data distribution [17, 19, 25, 34]. In many applications, however, faithfully reproducing that distribution is only a starting point: users seek images with particular content, stronger prompt alignment, or preferred visual attributes. These goals call for adapting a pretrained generator to an objective that may differ from its training criterion. The challenge is to improve the desired objective while retaining the quality and diversity supported by the learned prior.

Sampling-time guidance and reward-based fine-tuning offer two ways to pursue this goal. Classifier-free guidance (CFG) [16] strengthens conditioning by extrapolating between conditional and unconditional predictions, while gradient-based methods steer sampling using an objective evaluated on a predicted clean image [1, 4, 45]. Fine-tuning instead updates model parameters through reward backpropagation or reinforcement learning, incorporating preferences into future generations [2, 6, 27, 42]. Stronger intervention is not always beneficial: increasing the CFG scale can reduce distributional coverage [3, 30], and an external objective can favor directions that depart from the data structure [13].

A central difficulty is that preferences are defined on a completed image, whereas interventions act on noisy intermediate states or model parameters. Translating a desired output change into an effective intervention requires accounting for how the generator responds. We therefore ask how a terminal objective and a cost of intervention should determine the control, and whether a common local principle can govern both sampling-time guidance and parameter adaptation.

We address these questions through variational optimal control (VOC). For sampling, the pretrained conditional process provides a reference, and control maximizes terminal reward while penalizing deviation from that reference through a path-space KL divergence. This objective makes the trade-off between preference and prior explicit. Its optimal solution reweights future denoising trajectories by terminal reward and determines a corresponding correction to the dynamics. A local Gaussian approximation and a denoised reward proxy then yield Variational Optimal Guidance (VOG): the output gradient is transported to the current state through the denoising Jacobian. This Jacobian accounts for which changes in the noisy state affect the predicted clean sample. Under a linear subspace model with an optimal denoiser, the exact correction lies in the data subspace; the practical sampler approximates this pullback for neural denoisers.

The local control principle also suggests how to incorporate terminal improvements into the generator. We introduce Distilled Variational Optimal Control (DVOC), which first refines a sampled terminal latent using reward gradients within a bounded neighborhood, then trains the model to reproduce the resulting target. Differentiating the distillation loss transports the target displacement through a parameter Jacobian, realizing the parameter-space counterpart of the local VOG control. Selected denoising steps carry this parameter gradient, and the adapted model uses its ordinary sampler at inference. VOG and DVOC thus share a concrete mechanism: a preferred output change is mapped through the sensitivity of the controlled variable, acting on the current state in VOG and on model parameters in DVOC (Figure 1).

Figure 1. Variational control for sampling-time guidance and model adaptation. The gray curve illustrates generation from noise to the sampled endpoint. VOG (blue) evaluates the objective at the denoised prediction x^0\hat{x}_0 and maps its gradient through JtJ_t to correct the current sampling state. DVOC (pink) caches the trajectory, refines the actual endpoint y=x0y = x_0 within a local neighborhood, and distills the fixed target y+y^+ into LoRA parameters. The three ringed states supply the selected velocity evaluations (K=3K = 3) used for endpoint distillation. Target refinement uses two projected reward-ascent steps (J=2J = 2).

Our contributions are:

  • A variational account of diffusion guidance. We derive VOG from terminal reward maximization with a path-space KL penalty and characterize the subspace support of its exact correction under a linear MMSE model.

  • A connection between guidance and model adaptation. We identify a shared local control principle and derive DVOC’s endpoint distillation objective as its parameter-space realization.

  • Evaluation across guidance and reward adaptation. VOG improves FID in all twelve ImageNet model–guidance pairings and reduces 25-step DiT-XL/2 FID from 7.58 to 4.44. After 20 updates, DVOC improves HPSv3++ and four held-out evaluator scores over pretrained Z-Image-Turbo, with the highest corresponding means among the trained methods compared.

Related Work

Sampling-time guidance. Classifier guidance [9] steers sampling with a learned classifier gradient, while CFG [16] combines conditional and unconditional predictions. Subsequent methods adjust guidance direction, strength, or schedule [5, 18, 21, 29, 30, 41]. Manifold-aware guidance uses denoiser geometry [13, 14]. DPS, FreeDoM, and Universal Guidance propagate objectives through a clean prediction [1, 4, 45]. We derive state corrections from a terminal control objective, with their geometry determined by the denoising Jacobian.

Reward-based model adaptation. DDPO and DPOK use policy gradients [2, 11], while Diffusion-DPO learns from preference pairs [36]. Flow-GRPO [22] adapts group-relative policy optimization to flow matching via stochastic rollouts. DiffusionNFT [46] contrasts positive and negative generations through forward-process flow matching. Advantage Weighted Matching (AWM) [43] reweights score/flow-matching pretraining losses by sample advantages. ReFL, DRaFT, AlignProp, and DRTune backpropagate differentiable rewards [6, 27, 40, 42]. FlowBP [37] uses a lightweight backward trajectory, and DiffusionPSD [47] distills reward-improved clean outputs. We formulate reward adaptation as amortized control through the same local variational principle and Jacobian pullback as sampling-time guidance.

Variational control of generative models. Stochastic optimal control balances terminal reward against deviation from a reference process. Adjoint Matching [10] expresses reward adaptation as control regression. DEFT and h-Edit [8, 24] use Doob transforms for conditioning and editing. Within our variational framework, terminal preferences yield state and parameter corrections through Jacobian pullbacks. VOG and DVOC realize these two forms of control as trajectory guidance and its amortization in the generator.

Variational Optimal Control and Model Geometry

From terminal preferences to control. VOG and DVOC translate terminal preferences into corrections through the sensitivity of the controlled variable. VOG differentiates the denoised prediction with respect to the current state. DVOC differentiates the generated endpoint with respect to model parameters. We first derive the state-space control from the reference process, then develop parameter adaptation from the same local variational principle in § 3.5.

To analyze VOG’s geometry, we adopt a low-dimensional linear-subspace model [13, 26]. The model gives an explicit characterization of the denoising Jacobian.

Assumption 3.1 (Linear Latent Subspace). We assume the clean data x∗∈RDx^* \in\mathbb{R}^D resides strictly within a low-dimensional linear subspace S\mathcal{S} of intrinsic dimension d≪Dd \ll D. Specifically, the data is generated as x∗=Uzx^* = Uz, where U∈RD×dU \in\mathbb{R}^{D \times d} is an orthogonal basis matrix satisfying UTU=IdU^\mathsf{T}U = I_d, and the latent variable zz follows a standard Gaussian prior z∼N(0,Id)z \sim\mathcal{N}(0, I_d). Consequently, the data support is defined as S≜span⁡(U)\mathcal{S} \triangleq\operatorname{span}(U).

Let P∥=UUTP_{\parallel} = UU^\mathsf{T} and P⊥=ID−UUTP_{\perp} = I_D - UU^\mathsf{T}. Every vector decomposes into components parallel and normal to this subspace.

How CFG Shifts the Mean and Reduces Coverage

CFG extrapolates between conditional and unconditional predictions. On the branching-tree example in Figure 2, stronger guidance suppresses outliers but reduces branch coverage. A local probability-flow analysis explains how the prediction contrast can shift the generated distribution. Its assumptions and derivation appear in Appendix § 10.

Toy class-conditional generation on fractal-like 2D branching-tree distributions; panels labeled Ground Truth, Without Guidance, Standard CFG, and VOG (ours)

Figure 2. Toy class-conditional generation on fractal-like 2D branching-tree distributions [18] under two increasingly challenging settings: a two-class benchmark (CLS = 2) and a three-class variant with shared structure (CLS = 3), which induces stronger class overlap. We quantify generation difficulty using tr⁡(Cov⁡)\operatorname{tr}(\operatorname{Cov}) of the class-conditional ground-truth mixture, where larger values indicate greater difficulty. (a) Ground truth samples from the target class distribution. (b) No guidance: conditional sampling with a small diffusion model produces outliers and off-class leakage. (c) CFG (ω=4\omega= 4) suppresses outliers but shows mode collapse, reducing branch coverage; the collapse is more severe as difficulty increases from tr⁡(Cov⁡)=0.228\operatorname{tr}(\operatorname{Cov}) = 0.228 (CLS = 2) to tr⁡(Cov⁡)=0.403\operatorname{tr}(\operatorname{Cov}) = 0.403 (CLS = 3). (d) VOG (ours) preserves diversity and branch coverage while achieving strong conditional alignment. Detailed quantitative results are provided in Table S11.

Guidance-induced mean shift. A first-order perturbation analysis of the VP probability flow ODE gives

Eqω[x0]≈μglobal+ωΔeff,\mathbb{E}_{q_\omega}[x_0] \approx\mu_{\mathrm{global}} + \omega\Delta_{\mathrm{eff}},
Δeff≜E[∫0TΨ(τ)(μc−μˉ(xτ)) dτ](1)\Delta_{\mathrm{eff}} \triangleq\mathbb{E}\left[\int_{0}^{T} \Psi(\tau)\left(\mu_c-\bar{\mu}(x_\tau)\right)\,\mathrm{d}\tau\right] \tag*{(1)}

where the expectation is over samples x0x_0 generated by the ω\omega-guided sampler, μglobal=Eq[x0]\mu_{\mathrm{global}} = \mathbb{E}_{q}[x_0] is the unconditional data mean, μc\mu_c the target class mean, μˉ(xτ)=∑yp(y∣xτ)μy\bar{\mu}(x_\tau) = \sum_y p(y \mid x_\tau)\mu_y the posterior-weighted dynamic mean, and Ψ(τ)\Psi(\tau) the probability flow kernel of Eq. (26). The integrand is thus the class-uncertainty gap, transported to t=0t = 0 by the flow. The trajectory-dependent term Δeff\Delta_{\mathrm{eff}} also varies with ω\omega.

Writing ω=1+w\omega= 1 + w separates ordinary conditional prediction from further extrapolation. This motivates anchoring the control in § 3.2 at the conditional policy.

The analysis assumes Bayes-optimal branch mixing and locally affine class denoisers with shared contraction (Assumption E.1 and Assumption E.2).

Geometric interpretation. The transported class contrast links guidance to changes in distributional coverage. An exact subspace denoiser keeps the guided mean in the subspace. An imperfect denoiser can introduce cross-subspace coupling. Whether it yields a normal mean component also depends on the class contrast and cancellation along the trajectory (Appendix § 10.§ 2).

Variational Optimal Control (VOC)

We formulate sampling-time guidance as variational optimal control: maximize a terminal objective while penalizing deviation from a conditional reference process. Figure 1 relates the resulting state correction to DVOC’s parameter-space pullback.

Current-state conditional formulation. We parameterize denoising time as t∈[0,T]t \in[0,T] (t=Tt=T: noise, t=0t=0: clean sample). At time tt, the state Xt=xX_t=x is already realized, so guidance can only modify the future continuation from tt to 00. Let Pb,t:0(⋅∣Xt=x)P_{b,t:0}(\cdot\mid X_t=x) be the future path measure of a fixed baseline policy pbp_b, and let the terminal objective f(X0,c)f(X_0,c) say which clean samples are preferred. We seek a controlled path measure Qt:0(⋅∣Xt=x)Q_{t:0}(\cdot\mid X_t=x) solving

max⁡Q{λEQ[f(X0,c)]−DKL(Q∥Pb)},(2)\max_Q \left\{ \lambda\mathbb{E}_Q[f(X_0,c)] - D_{\mathrm{KL}}(Q \mathbin{\|} P_b) \right\}, \tag*{(2)}

where QQ and PbP_b abbreviate the future path measures conditioned on Xt=xX_t=x.

Conditional reference. For the stochastic VOG formulation, pbp_b is the pretrained conditional policy predicting ϵc=ϵθ(xt,t,c)\epsilon_c=\epsilon_\theta(x_t,t,c). Zero control means ordinary conditional sampling. We use the terminal log-ratio fcfg(X0,c)=log⁡pθ(X0∣c)−log⁡pθ(X0)f_{\mathrm{cfg}}(X_0,c)=\log p_\theta(X_0 \mid c)-\log p_\theta(X_0), with control strength w=ω−1w=\omega-1. The reference already carries the conditioning, so extrapolation enters through the terminal objective. For reward adaptation, we instantiate the endpoint objective with a learned evaluator and constrain local improvement within a trust region (§ 3.5).

Optimal solution via exponential tilting. Define the desirability ht(x)≜EPb,t:0[exp⁡(λf(X0,c))∣Xt=x]h_t(x) \triangleq\mathbb{E}_{P_{b,t:0}}[\exp(\lambda f(X_0,c)) \mid X_t=x] and its log ℓt(x)≜log⁡ht(x)\ell_t(x) \triangleq\log h_t(x). By the Gibbs variational identity, the optimizer of Eq. (2) is the exponentially tilted path measure (proof in Appendix § 7):

dQt:0⋆dPb,t:0=exp⁡(λf(X0,c))ht(x).(3)\frac{\mathrm{d}Q^\star_{t:0}}{\mathrm{d}P_{b,t:0}}=\frac{\exp(\lambda f(X_0,c))}{h_t(x)}. \tag*{(3)}

Using an increasing clock along the denoising direction, the Doob hh-transform shifts the reference drift by Δb⋆(x,t)=at(x)∇xℓt(x)\Delta b^\star(x,t)=a_t(x)\nabla_x\ell_t(x), where ata_t is the diffusion covariance. This covariance is preserved by the transform.

Practical discretization. For a single denoising step of size δ>0\delta>0 (advancing from tt to t−δt-\delta), we use the local Gaussian approximation Xt−δ∣Xt=x≈N(μt(x),Σt)X_{t-\delta}\mid X_t=x\approx\mathcal{N}(\mu_t(x),\Sigma_t) with μt(x)=x+bθ(x,t)δ\mu_t(x)=x+b_\theta(x,t)\delta and Σt=at(x)δ\Sigma_t=a_t(x)\delta. Applying the short-time Doob tilt and linearizing the log-desirability yields a Gaussian with the same covariance and a shifted mean,

μtVOG(x)=μt(x)+Δμt,Δμt=Σt∇xℓt(x).(4)\mu_t^{\mathrm{VOG}}(x)=\mu_t(x)+\Delta\mu_t,\qquad\Delta\mu_t=\Sigma_t\nabla_x\ell_t(x). \tag*{(4)}

Since the exact ℓt\ell_t requires integrating over all future continuations, we approximate it by the denoised reward proxy and pull the gradient back through the denoising Jacobian Jt=∂x^0/∂xJ_t=\partial\hat{x}_0/\partial x, giving ∇xℓt(x)≈λJt(x)⊤∇x^0f(x^0,c)\nabla_x\ell_t(x)\approx\lambda J_t(x)^\top\nabla_{\hat{x}_0}f(\hat{x}_0,c). For isotropic step covariance Σt=st2I\Sigma_t=s_t^2 I and λ>0\lambda>0, mapping this mean-space correction to the noise-prediction parameterization (μt=Atx−Ctϵt\mu_t=A_tx-C_t\epsilon_t, Ct>0C_t>0) and absorbing all sampler-dependent constants into a positive schedule ζ(t)\zeta(t) gives our main result:

Proposition 3.2 (Variational Optimal Guidance). Under the current-state conditional VOC of Eq. (2), the Doob hh-transform preserves the diffusion covariance and shifts the drift by at∇xℓta_t\nabla_x\ell_t. With isotropic step covariance, the short-time Gaussian approximation, and the denoised reward proxy, the first-order local approximation to the optimal noise-space correction is

ϵvog(x,t,c)=ϵc(x,t)−ζ(t)JtT∇x^0f(x^0,c),(5)\epsilon_{\mathrm{vog}}(x,t,c)=\epsilon_c(x,t)-\zeta(t)\mathbf{J}_t^{\mathsf{T}}\nabla_{\hat{x}_0}f(\hat{x}_0,c), \tag*{(5)}

where ϵc\epsilon_c is the conditional reference prediction, Jt=∂x^0/∂x\mathbf{J}_t=\partial\hat{x}_0/\partial x, and ζ(t)>0\zeta(t)>0 is a sampler-dependent schedule. The derivation is in Appendix § 7.

Role of the denoising Jacobian. The key structural difference from CFG is the Jacobian Jt\mathbf{J}_t: CFG adds the guidance signal to the noise prediction directly, whereas VOG first pulls it back through the denoiser via a vector-Jacobian product (VJP). In the linear subspace setting below the MMSE denoising Jacobian is a scaled orthogonal projector, so this VJP is exactly a projection onto the data subspace. For nonlinear denoisers it weights the correction by learned sensitivity. This provides a geometric mechanism for mitigating the normal drift (P⊥Δeff≠0P_{\perp}\Delta_{\mathrm{eff}}\ne0) of § 3.1.

Geometric Consistency Analysis

We characterize the exact pullback relative to the data subspace S\mathcal{S} of Assumption 3.1.

Theorem 3.3 (Subspace support of the exact correction). Let P⊥=I−UUTP_{\perp}=I-UU^{\mathsf{T}} project onto the orthogonal complement of the data subspace, and let gvog≔JtT∇x^0f(x^0)\mathbf{g}_{\mathrm{vog}}\coloneqq\mathbf{J}_t^{\mathsf{T}}\nabla_{\hat{x}_0}f(\hat{x}_0) denote the correction that VOG adds to the baseline. Under Assumption 3.1 and the optimal linear MMSE denoiser x^0(xt)=κtUUTxt\hat{x}_0(x_t)=\kappa_tUU^{\mathsf{T}}x_t, this correction lies in the data subspace, P⊥gvog=0P_{\perp}\mathbf{g}_{\mathrm{vog}}=0.

Proof. For the linear subspace denoiser, Jt=κtUUT\mathbf{J}_t=\kappa_tUU^{\mathsf{T}} is symmetric with both column and row space contained in S\mathcal{S}. Hence gvog=κtUUT∇x^0f∈S\mathbf{g}_{\mathrm{vog}}=\kappa_tUU^{\mathsf{T}}\nabla_{\hat{x}_0}f\in\mathcal{S}, so P⊥gvog=0P_{\perp}\mathbf{g}_{\mathrm{vog}}=0.

The exact linear denoiser therefore filters the reward gradient into S\mathcal{S}. The implementation below approximates this pullback for neural denoisers.

Practical Implementation Strategies

We use three strategies to balance efficiency and stability in practice.

Strided gradient computation. We recompute the pullback every KVOGK_{\mathrm{VOG}} steps and reuse the cached correction in between. This inference stride is distinct from DVOC’s active-step count KK.

Factorized VJP surrogate. We approximate the conditional-noise VJP by (Jc,t1)⊙gt(\mathbf{J}_{c,t}\mathbf{1})\odot\mathbf{g}_t, where Jc,t=∂ϵc/∂xt\mathbf{J}_{c,t}=\partial\epsilon_c/\partial x_t and gt=ϵc−ϵu\mathbf{g}_t=\epsilon_c-\epsilon_u. The sensitivity factor Jc,tT1=∇xt(1Tϵc)\mathbf{J}_{c,t}^{\mathsf{T}}\mathbf{1}=\nabla_{x_t}(\mathbf{1}^{\mathsf{T}}\epsilon_c) is computed with one backward pass and can be cached separately from the contrast. Appendix § 8 derives the controller and gives the residual bound and differentiation-branch ablation.

Momentum-enhanced guidance. High-frequency fluctuations in the raw guidance signal gt=ϵθ(xt,c)−ϵθ(xt,∅)\mathbf{g}_t=\epsilon_\theta(x_t,c)-\epsilon_\theta(x_t,\varnothing) can trigger over-suppression under Jacobian-based regularization. We smooth it with exponential momentum, mt=βmt+1+gt\mathbf{m}_t=\beta\mathbf{m}_{t+1}+\mathbf{g}_t, and replace gt\mathbf{g}_t by mt\mathbf{m}_t in the update rule. Here β\beta is the momentum coefficient, distinct from the correction schedule ζ(t)\zeta(t) of Eq. (5). The uncached, zero-momentum core is given in Algorithm S1. The schedules and caching described here are practical extensions.

From State Control to Model Adaptation

The variational construction couples a preferred terminal change with the sensitivity of the controlled variable. VOG acts on the current state. DVOC acts on model parameters, amortizing terminal preferences across future trajectories. Their common local control problem yields the endpoint distillation objective below.

A shared local variational principle. Let GξG_{\xi} map a control variable ξ\xi to an output representation, and let d\mathbf{d} denote a preferred change in that representation. Write Jξ=∂Gξ/∂ξJ_{\xi} = \partial G_{\xi}/\partial\xi. Linearizing the output response and assigning a quadratic cost to the control gives

uξ∗=arg⁡max⁡u{⟨d,Jξu⟩−∥u∥222γ}=γJξTd,(6)u_{\xi}^{*} = \underset{u}{\arg\max}\left\{\langle\mathbf{d},J_{\xi}u\rangle- \frac{\lVert u\rVert_{2}^{2}}{2\gamma}\right\} = \gamma J_{\xi}^{\mathsf{T}}\mathbf{d}, \tag*{(6)}

where γ>0\gamma> 0 sets the control scale. This is the pullback structure exposed by the local VOG derivation. For state control, GξG_{\xi} is the denoised prediction and ξ=xt\xi= x_t. For parameter control, the output is a terminal latent and ξ=ϕ\xi= \phi. Changing the controlled variable changes the Jacobian through which the preferred output displacement acts.

Constructing the terminal control target. For a prompt cc, sample a trajectory with the current model and denote its terminal latent by y=x0y = x_0. Let DD be the frozen decoder and define R(y,c)=R(D(y),c)\mathcal{R}(y,c) = \mathcal{R}(D(y),c). We seek a terminal improvement within a local displacement budget:

max⁡∥u−y∥2≤r R(u,c),r=ρ∥y∥2.(7)\underset{\lVert u-y\rVert_{2}\leq r}{\max}\ \mathcal{R}(u,c), \qquad r = \rho\lVert y\rVert_{2}. \tag*{(7)}

Starting at yy, we approximate this problem with JJ normalized projected ascent steps. The reward gradient is itself a pullback, ∇yR=JDT∇IR\nabla_y\mathcal{R} = J_D^{\mathsf{T}}\nabla_I\mathcal{R}. We denote the resulting detached target by y+y^{+} and its displacement by δ=y+−y\delta= y^{+} - y. The relative radius makes the control budget proportional to the sampled latent’s scale. Updating the reward direction at each ascent step accounts for the local reward geometry along the target path. The projection and iteration are specified in Appendix § 9.

Distilling the control into parameters. We transfer δ\delta to the generator by fitting y+y^{+} through an endpoint connector yϕAy_{\phi}^{A}. Its forward value is the sampled endpoint yy, and its parameter Jacobian is JAJ_A. For a latent with dd coordinates, the per-sample objective is

LDVOC=∥yϕA−y+∥22d sδ,sδ=sg⁡(max⁡{∥δ∥1d,10−5}).(8)\mathcal{L}_{\mathrm{DVOC}} = \frac{\lVert y_{\phi}^{A} - y^{+}\rVert_{2}^{2}}{d\,s_{\delta}}, \qquad s_{\delta} = \operatorname{sg}\left(\max\left\{\frac{\lVert\delta\rVert_{1}}{d},10^{-5}\right\}\right). \tag*{(8)}

The target and normalization are held fixed during fitting. Since the residual at the sampled endpoint is −δ-\delta, differentiation yields

−∇ϕLDVOC=2d sδJATδ.(9)-\nabla_{\phi}\mathcal{L}_{\mathrm{DVOC}} = \frac{2}{d\,s_{\delta}}J_A^{\mathsf{T}}\delta. \tag*{(9)}

Thus endpoint distillation realizes the parameter-space form of Eq. (6), with d=δ\mathbf{d} = \delta and scale 2/(d sδ)2/(d\,s_{\delta}). The terminal target supplies the control signal, while the parameter Jacobian determines how that signal is expressed by the generator. Averaging this loss over sampled prompts and trajectories amortizes the terminal controls into a shared model.

Algorithm 1 DVOC training update

Require: LoRA parameters ϕ\phi, decoder DD, reward RR, NN sampling steps, KK active steps, JJ target steps.

  1. Sample prompts and generate NN-step trajectories.

  2. Cache states, velocities, and terminal latents yy without gradients.

  3. Initialize y(0)=yy^{(0)} = y and radius r=ρ∥y∥2r = \rho\lVert y \rVert_2.

  4. for j=0,…,J−1j = 0, \ldots, J - 1 do

  5. Compute ∇yR(D(y(j)),c)\nabla_y R(D(y^{(j)}), c).

  6. Take a normalized ascent step of length ηr/J\eta r / J and project onto B(y,r)\mathcal{B}(y, r).

  7. end for

  8. Detach the target y+=y(J)y^+ = y^{(J)}.

  9. for each optimization microbatch do

  10. Sample KK active steps uniformly without replacement.

  11. Re-evaluate active velocities at the cached states.

  12. Form the endpoint connector in Eq. (23).

  13. Accumulate scaled gradients of Eq. (8).

  14. end for

  15. Average gradients across GPUs and clip their norm.

  16. Update ϕ\phi with AdamW and update the EMA adapter.

Efficient parameter pullback. We sample an Euler trajectory without retaining its computation graph, then choose KK steps uniformly without replacement. At these steps, we re-evaluate velocities at the cached states with parameter gradients enabled. Their Euler-weighted derivatives define JAJ_A. The endpoint connector preserves the sampled forward value while passing gradients through these selected evaluations, as specified in Eq. (23). The complete endpoint supplies the target, and KK sets the backward computation. Our experiments use nine sampling steps, K=3K = 3, two target-ascent steps, relative radius ρ=0.1\rho= 0.1, and step multiplier η=1\eta= 1. We optimize LoRA parameters and use the EMA adapter for evaluation.

Control geometry and amortization. In Eqs. (5) and (9), both controls weight terminal preferences by the sensitivity of the controlled variable. VOG realizes the state-space geometry characterized in Theorem 3.3. DVOC shares parameter updates across sampled conditions and stores the control in the generator. The adapted model then uses the ordinary sampler at inference. Figure 1 summarizes the two realizations, and Algorithm 1 gives the training procedure.

Experiments

We evaluate VOG on synthetic and class-conditional ImageNet generation, and DVOC on reward-based Z-Image-Turbo adaptation.

Evaluation Protocols

Sampling-time guidance. We use EDM2 [19] and DiT [25] on ImageNet [7] at 64264^2, 2562256^2, and 5122512^2 resolutions. Baselines include CFG, Cosine and Interval schedules, AutoGuidance, APG, and ReCFG [16, 18, 21, 30, 41]. We report FID [15] from 50,000 samples together with Precision and Recall [31]. Architecture-specific settings are provided in the supplement.

Reward-based post-training. DVOC, ReFL, DRTune, and DiffusionOPSD start from Z-Image-Turbo [35]. We also evaluate its untrained original weights. The trained methods share the training prompt source, the weighted reward 0.03846RHPSv3+++RCLIP0.03846 R_{\mathrm{HPSv3++}} + R_{\mathrm{CLIP}}, and a nominal budget of 576 trajectories per optimizer update. We evaluate their EMA adapters at update 20, with nine denoising steps per evaluation sample. Each trained experiment uses eight GPUs, LoRA rank 32 and scale 64, AdamW with learning rate 3×10−43 \times10^{-4}, and weight decay 10−410^{-4} (DVOC, DiffusionOPSD) or 10−210^{-2} (ReFL, DRTune). The supplement reports method settings and observed per-update times (Table S7).

MethodGPUsSeconds / updateMinutes / updateGPU-min / update
ReFL8272.8 ± 1.04.5536.4
DRTune8336.8 ± 1.35.6144.9
DiffusionOPSD8397.6 ± 0.86.6353.0
DVOC (K=1K = 1)†8325.8 ± 0.65.4343.4
DVOC (K=2K = 2)†8391.2 ± 0.46.5252.2
DVOC (K=3K = 3)8458.0 ± 0.57.6361.1

Table S7. Training time per optimizer update on one eight-GPU node. Means and standard deviations use 18 intervals from updates 2–20, excluding update 11 because it follows evaluation and checkpointing. Each update uses 576 training trajectories. Standard deviations describe timing variation within a single run. † denotes estimates from rollout-completion intervals; other rows use optimizer-completion intervals.

The test set is the complete 1,000-prompt Pick-a-Pic split [20]. All methods use the same prompts and initial noise, with 1024×10241024 \times1024 outputs, nine deterministic Euler steps, and guidance scale 0. All seven metrics are computed on the same generated images. We use HPSv3++ [23] and CLIP [28] as in-domain evaluators because they enter the training reward. Q-Align quality and aesthetics [39], AES [32], ImageReward [42], and DeQA [44] are out-of-domain (OOD) evaluators because they are held out of optimization. Here OOD denotes evaluators held out of the training objective. All post-training results use one training seed.

Sampling-Time Guidance Results

Synthetic geometry. On the 2D branching-tree benchmark, increasing class overlap makes coverage more difficult. The two-class and three-class settings have covariance traces 0.228 and 0.403. CFG removes outliers but reduces Recall to 0.876 and 0.779, respectively. VOG attains Recall 0.957 and 0.933 while achieving the highest Precision among the guided samplers. The visualization and numerical results are in Figure 2 and the supplement.

ImageNet. Table 1 evaluates VOG on top of three guidance baselines in four model–resolution settings. FID improves in all twelve pairings, including from 1.99 to 1.42 for EDM2-L at 64264^2. For DiT-XL/2 with Cosine guidance, Recall rises from 0.520 to 0.612 while Precision decreases, producing a better FID operating point.

GuidanceEDM2-S (512 × 512)DiT-XL/2 (256 × 256)EDM2-L (64 × 64)DiT-XL/2 (512 × 512)
FIDPrecisionRecallFIDPrecisionRecallFIDPrecisionRecallFIDPrecisionRecall
CFG [16]2.290.7510.5822.270.8290.5841.990.7100.6013.040.8380.541
+ VOG (Ours)2.030.7550.5912.040.7560.6071.420.714<u>0.597</u>2.890.7450.560
Interval [21]<u>1.87</u>0.7600.598<u>1.97</u>0.8010.6321.640.7300.597<u>2.41</u><u>0.792</u>0.611
+ VOG (Ours)1.670.7420.6141.940.7650.6111.520.7200.5932.390.7600.567
Cosine [12]2.150.770<u>0.619</u>2.300.8610.5201.670.7260.5873.480.713<u>0.586</u>
+ VOG (Ours)2.060.7580.5892.030.7570.6121.65<u>0.727</u><u>0.588</u>2.830.747<u>0.563</u>
Scheduler [38]2.03<u>0.762</u>0.5912.14<u>0.840</u>0.614------
APG [30]2.130.7560.6402.110.815<u>0.628</u>------
ReCFG [41]2.230.7610.6082.130.7990.601------

Table 1. VOG as a plug-in correction on top of CFG-style guidance baselines, on class-conditional ImageNet. Shaded rows apply VOG to the baseline immediately above them. We report FID (↓), Precision (↑), and Recall (↑); within each column the best result is bolded and the second best <u>underlined</u>. A dash indicates a configuration the baseline does not support.

Guidance scale and sampling budget. VOG remains effective over a wider range of guidance strengths than CFG in Figure S2. On DiT-XL/2, its FID is below 2.60 at guidance scale 1.7, where CFG exceeds 3.40. The benefit also persists with fewer sampling steps: FID decreases from 7.58 to 4.44 at 25 steps and from 3.69 to 2.99 at 50 steps. The VJP stride and momentum sweep in Figure S2 concerns VOG inference, independently of the active-step budget KK used by DVOC.

Figure S2. FID as a function of the guidance scale ωCFG\omega_{\mathrm{CFG}} for DiT-XL/2 (256 × 256) and EDM2-S (512 × 512).

Reward Alignment and Generalization

In-domain alignment and held-out scores. Table 2 uses the untrained original model as the reference, with DVOC at K=3K = 3. DVOC raises HPSv3++ from 6.72 to 13.79, the highest trained mean. CLIP changes from 0.2757 to 0.2765. DiffusionOPSD achieves the highest CLIP score at 0.2782.

MethodIn-Domain HPSv3++ Score↑ΔIn-Domain CLIP Score↑ΔOut-of-Domain Q-Align Q. Score↑ΔOut-of-Domain Q-Align A. Score↑ΔOut-of-Domain AES Score↑ΔOut-of-Domain ImageReward Score↑ΔOut-of-Domain DeQA Score↑Δ
Z-Image-Turbo, 102421024^2, nine-step Euler sampler
Z-Image-Turbo (untrained)6.72—0.2757—4.825—3.533—5.412—0.959—4.439—
ReFL [42]12.22+5.500.2744−0.00134.844+0.0204.223+0.6915.955+0.5431.159+0.2004.133−0.306
DRTune [40]12.70+5.970.2688−0.00704.932+0.1074.489+0.9566.095+0.6831.225+0.2664.262−0.177
DiffusionOPSD [47]13.56+6.830.2782+0.00254.886+0.0614.452+0.9206.101+0.6891.196+0.2374.014−0.425
DVOC (K=3K = 3)13.79+7.060.2765+0.00084.943+0.1184.618+1.0856.288+0.8761.279+0.3204.237−0.202

Table 2. Reward alignment after 20 optimization updates on Z-Image-Turbo. The untrained base and four trained methods use the same 1,000 prompts and initial noise. HPSv3++/CLIP are in-domain; the others are OOD. Δ: score minus untrained base. Higher is better. Best/second-best: bold/underlined. Blue: DVOC.

DVOC leads trained methods on Q-Align quality (4.943), Q-Align aesthetics (4.618), AES (6.288), and ImageReward (1.279), improving over the base on all four. DeQA is 4.237, compared with 4.439 for the base. Table S8 reports paired uncertainty for all seven main-result differences.

MetricMean difference95% interval
HPSv3+++7.06357[+6.90034, +7.22679]
CLIP+0.00077[-0.00142, +0.00297]
Q-Align Q.+0.11801[+0.10556, +0.13046]
Q-Align A.+1.08532[+1.05159, +1.11906]
AES+0.87600[+0.84961, +0.90238]
ImageReward+0.31966[+0.28264, +0.35668]
DeQA-0.20233[-0.21682, -0.18784]

Table S8. Paired differences: default DVOC (K=3K = 3) minus untrained Z-Image-Turbo. Normal-approximation 95% intervals use 1,000 paired prompts and are reported per metric.

Qualitative comparisons with the base model and the three trained baselines are provided in Figure S1 in the appendix.

Qualitative comparison of reward-based adaptation

Figure S1. Qualitative comparison of reward-based adaptation. Columns show Z-Image-Turbo, ReFL, DRTune, DiffusionOPSD, and DVOC. Rows show (a) a museum interior, (b) a linen jacket, (c) a bicycle, and (d) a reading room. The examples illustrate differences in material texture, lighting, and local detail across methods.

Optimization dynamics. Figure 3(a,b) tracks the two in-domain evaluators through 100 updates. From update 20 to 100, DVOC’s online CLIP increases from 0.2768 to 0.3111 and HPSv3++ from 12.71 to 15.59. Panels (c,d) show logged training reward across methods and across KK, respectively. These online evaluations use a different protocol from the paired main table. Extended histories are in Appendix § 13.

Figure 3. Evaluation and training dynamics. (a,b) Online EMA evaluation every ten updates, using HPSv3++ condition 0.0 and trainer-specific noise sampling. (c) Training reward, 0.03846 RHPSv3+++RCLIPR_{\mathrm{HPSv3++}} + R_{\mathrm{CLIP}}: ReFL/DRTune score late-state proxies; DVOC/OPSD score sampled endpoints. (d) DVOC training reward for K=1,2,3K = 1, 2, 3. Training panels show raw means (faint) and trailing five-update averages; evaluation is unsmoothed. The vertical line marks update 20. DVOC uses K=3K = 3 in (a–c); (d) is the ablation.

DVOC Ablations

Active-step budget. Table 4 varies KK with shared forward sampling and target construction. K=3K = 3 leads on HPSv3++, CLIP, AES, and ImageReward. K=1K = 1 leads on both Q-Align scores, and K=2K = 2 on DeQA, showing that the preferred budget depends on the evaluator.

DVOC variantHPSv3++↑CLIP↑Q-Align Q.↑Q-Align A.↑AES↑ImageReward↑DeQA↑
K=1K = 113.360.27294.9484.6286.2721.224<u>4.249</u>
K=2K = 2<u>13.69</u><u>0.2751</u><u>4.947</u>4.610<u>6.274</u><u>1.257</u>4.263
K=3K = 3 (default)13.790.27654.943<u>4.618</u>6.2881.2794.237

Table 4. Active-step ablation. Only KK changes; J=2J = 2 and the endpoint radius are fixed. Means use the same 1,000 paired prompts at update 20. Bold/underlining mark best/second-best within this table.

Target construction. Three paired runs fix K=3K = 3 and share the initial adapter and training streams (Appendix § 11.§ 4). Relative to one ascent step, endpoint J=2J = 2 raises Q-Align quality by 0.0202, AES by 0.0692, and DeQA by 0.1203, with a CLIP decrease of 0.00210. At the same nominal ascent length, two steps refresh the intermediate reward gradient. One step takes 414.1 seconds per update versus 461.7 for two steps, a 10.3% reduction.

The proxy ablation optimizes the current policy’s late-state clean prediction and transports its displacement to the sampled endpoint. Endpoint J=2J = 2 improves Q-Align quality, Q-Align aesthetics, and DeQA by 0.0125, 0.0215, and 0.0100, respectively. The full seven-metric comparison is in Table 3.

VariantHPSv3++CLIPQ-QualityQ-Aesth.AESImageRewardDeQA
Endpoint, J=2J = 213.7270.27604.9474.6406.2811.26784.252
Endpoint, J=1J = 113.6530.27814.9274.6296.2111.26734.131
Proxy, J=2J = 213.7050.27634.9354.6186.2771.26844.242

Table 3. Target construction at fixed K=3K = 3. Three paired runs use 20 updates and 1,000 matched test images. All scores are higher-is-better. Bold marks the highest mean within this ablation.

Computational Cost

The nine-step rollout and two target-gradient evaluations are shared across the active-step variants. Increasing KK expands the parameter pullback while retaining the same sampled trajectories and target construction. On one eight-GPU node, the observed mean times are 325.8, 391.2, and 458.0 seconds per update for K=1,2,3K = 1, 2, 3, respectively (Table S7). Relative to K=3K = 3, K=1K = 1 reduces update time by 28.9% and K=2K = 2 by 14.6%. The main configuration uses K=3K = 3, which leads the active-step comparison on HPSv3++, CLIP, AES, and ImageReward.

At fixed K=3K = 3, one and two target steps take 414.1 and 461.7 seconds per update. The target ablation in Table 3 reports their quality–cost trade-off. At inference, all DVOC variants use the same nine-step sampler.

Conclusion

Variational control connects sampling-time guidance and reward adaptation through a shared Jacobian pullback. VOG realizes control on individual trajectories, and DVOC amortizes endpoint improvements into the generator. VOG improves FID across the evaluated ImageNet settings. After 20 updates on Z-Image-Turbo, DVOC improves HPSv3++ and four held-out scores over the base and leads the trained comparators on these metrics. Our analyses characterize the common control geometry through subspace structure and the variational derivation of parameter control.

Preliminaries and Notation

We collect the diffusion and guidance notation used in the derivations.

Diffusion and Guidance

The VP perturbation is xt=αˉtx0+1−αˉtϵx_t = \sqrt{\bar{\alpha}_t}x_0 + \sqrt{1-\bar{\alpha}_t}\epsilon, with ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I) and cumulative signal power αˉt\bar{\alpha}_t. A noise predictor trained by denoising regression [17] gives the model score sθ(xt,t)=−ϵθ(xt,t)/1−αˉts_\theta(x_t,t) = -\epsilon_\theta(x_t,t)/\sqrt{1-\bar{\alpha}_t} [33, 34].

Classifier guidance adds a scaled gradient of a noisy-image classifier’s log probability to the model score [9]. CFG uses the contrast between conditional and unconditional predictions [16].

Classifier-Free Guidance (CFG). To avoid the complexity of external classifiers, Ho and Salimans [16] proposed effectively learning an implicit classifier by jointly training a single model for both conditional ϵθ(xt,c)\epsilon_\theta(x_t,c) and unconditional ϵθ(xt,∅)\epsilon_\theta(x_t,\emptyset) generation. During inference, the guided noise prediction is computed as a linear extrapolation:

ϵ~t(xt,c)=ϵθ(xt,∅)+ω(ϵθ(xt,c)−ϵθ(xt,∅)).(10)\tilde{\epsilon}_t(x_t,c) = \epsilon_\theta(x_t,\emptyset) + \omega\left(\epsilon_\theta(x_t,c) - \epsilon_\theta(x_t,\emptyset)\right). \tag*{(10)}

Here, ω\omega serves as the guidance scale. Eq. (10) can be interpreted as implicitly performing gradient ascent on the log-likelihood ratio log⁡[p(c∣xt)/p(c)]\log[p(c|x_t)/p(c)], treating the difference term as a proxy for the gradient of the condition. Large ω\omega can increase saturation and reduce sample coverage [5, 30].

Summary of Notation

Table S1 collects the notation. We distinguish the CFG scale ω\omega, VOG correction schedule ζ(t)\zeta(t), momentum coefficient β\beta, and VP noise rate βvp(t)\beta_{\mathrm{vp}}(t).

SymbolMeaning
xtx_t, x^0\hat{x}_0noisy state at time tt; denoised prediction of the clean sample
ϵc\epsilon_c, ϵu\epsilon_u, ϵvog\epsilon_{\mathrm{vog}}conditional and unconditional predictions; VOG-corrected prediction
αˉt\bar{\alpha}_t, 1−αˉt\sqrt{1-\bar{\alpha}_t}cumulative signal power and VP noise standard deviation
cc, f(x0,c)f(x_0,c)conditioning signal; terminal reward on the clean sample
ω\omega, w=ω−1w=\omega-1CFG guidance scale; its conditional-centered form, the control strength
λ\lambdareward temperature in the VOC objective
ζ(t)\zeta(t)schedule of the VOG noise-space correction
β\betamomentum coefficient for smoothing the guidance signal
KVOGK_{\mathrm{VOG}}inference-time interval between VJP recomputations
KK, JJ, ρ\rhoDVOC active-step count, ascent iterations, and relative endpoint radius
Jt=∂x^0/∂xtJ_t=\partial\hat{x}_0/\partial x_tdenoising Jacobian; the correction uses the VJP Jt⊤∇fJ_t^\top\nabla f
ht(x)h_t(x), ℓt(x)=log⁡ht(x)\ell_t(x)=\log h_t(x)desirability function and its logarithm
bθb_\theta, ata_tdrift and diffusion covariance of the pretrained sampler
S=span⁡(U)S=\operatorname{span}(U), P∥P_{\parallel}, P⊥P_{\perp}data subspace and the projectors onto it and its complement
ϕ\phi, δ\delta, sδs_\deltaLoRA parameters, endpoint target displacement, and detached loss normalization
JAJ_A, yϕAy_\phi^Aselected parameter Jacobian and its endpoint connector

Table S1. Notation used in the main text.

Derivation of Variational Optimal Guidance

We derive the tilted path measure, its drift correction, and the local Jacobian form used by VOG. Throughout, P=Pb,t:0(⋅∣Xt=x,c)P = P_{b,t:0}(\cdot\mid X_t = x,c) is the pretrained conditional continuation from the current state to the clean endpoint.

Optimal path measure

Let ht(x)=EP[exp⁡(λf(X0,c))]h_t(x) = \mathbb{E}_P[\exp(\lambda f(X_0,c))] and ℓt(x)=log⁡ht(x)\ell_t(x) = \log h_t(x). Assume 0<ht(x)<∞0 < h_t(x) < \infty and finite reward and KL terms. For the tilted measure Q∗Q^* in Eq. (3), substitution of its density gives

λEQ[f]−DKL(Q∥P)=ℓt(x)−DKL(Q∥Q∗).(11)\lambda\mathbb{E}_Q[f] - D_{\mathrm{KL}}(Q\mathbin\Vert P) = \ell_t(x) - D_{\mathrm{KL}}(Q\mathbin\Vert Q^*). \tag*{(11)}

The nonnegativity of KL establishes that Q∗Q^* uniquely maximizes Eq. (2), with value ℓt(x)\ell_t(x).

Drift correction and control cost

Use the increasing denoising clock s=T−ts=T-t, with Ys=XT−sY_s=X_{T-s}. The reference process has drift bθb_\theta, diffusion coefficient σt\sigma_t, and covariance at=σtσt⊤a_t=\sigma_t\sigma_t^\top. Its generator is Ltφ=bθ⊤∇φ+12Tr⁡(at∇2φ)\mathcal{L}_t\varphi=b_\theta^\top\nabla\varphi+\frac{1}{2}\operatorname{Tr}(a_t\nabla^2\varphi). The Markov property of the tilted measure gives

Q∗(Xr∈dy∣Xt=x)=pb(t,x;r,dy)hr(y)ht(x),0≤r<t.(12)Q^*(X_r\in dy\mid X_t=x)=p_b(t,x;r,dy)\frac{h_r(y)}{h_t(x)},\qquad0\le r<t. \tag*{(12)}

The desirability satisfies −∂tht+Ltht=0-\partial_t h_t+\mathcal{L}_t h_t=0 with h0=exp⁡(λf)h_0=\exp(\lambda f). Applying the product rule to the transformed generator yields

Lthφ=−∂t(htφ)+Lt(htφ)ht=Ltφ+(at∇ℓt)⊤∇φ.(13)\mathcal{L}_t^h\varphi=\frac{-\partial_t(h_t\varphi)+\mathcal{L}_t(h_t\varphi)}{h_t}=\mathcal{L}_t\varphi+(a_t\nabla\ell_t)^\top\nabla\varphi. \tag*{(13)}

Thus the optimal drift is b∗=bθ+at∇ℓtb^*=b_\theta+a_t\nabla\ell_t, with the reference diffusion covariance unchanged.

For a drift perturbation Δb\Delta b and invertible ata_t, Girsanov’s identity expresses the path KL as

DKL(Q∥P)=12EQ[∫0tΔb(Xτ,τ)⊤aτ(Xτ)−1Δb(Xτ,τ) dτ∣Xt=x].(14)D_{\mathrm{KL}}(Q\|P)=\frac{1}{2}\mathbb{E}_Q\left[\left.\int_0^t\Delta b(X_\tau,\tau)^\top a_\tau(X_\tau)^{-1}\Delta b(X_\tau,\tau)\,d\tau\right|X_t=x\right]. \tag*{(14)}

This identity assumes an integrable control energy and a true Girsanov martingale. It identifies the covariance-weighted quadratic cost underlying the local control objective.

Local transition and Jacobian pullback

For a step of size δ\delta, approximate the reference transition by N(μt,Σt)\mathcal{N}(\mu_t,\Sigma_t), where μt=x+bθ(x,t)δ\mu_t=x+b_\theta(x,t)\delta and Σt=at(x)δ\Sigma_t=a_t(x)\delta. Linearizing ℓt−δ(y)\ell_{t-\delta}(y) in yy and writing gt=∇xℓt(x)g_t=\nabla_x\ell_t(x) gives

q~(y∣x)∝exp⁡[−12(y−μt)⊤Σt−1(y−μt)+gt⊤(y−μt)].(15)\widetilde{q}(y\mid x)\propto\exp\left[-\frac{1}{2}(y-\mu_t)^\top\Sigma_t^{-1}(y-\mu_t)+g_t^\top(y-\mu_t)\right]. \tag*{(15)}

Completing the square gives N(μt+Σtgt,Σt)\mathcal{N}(\mu_t+\Sigma_tg_t,\Sigma_t), proving the mean shift in Eq. (4).

We evaluate the terminal reward on the conditional clean prediction x^0,c=(xt−1−αˉtϵc)/αˉt\hat{x}_{0,c} = (x_t - \sqrt{1-\bar{\alpha}_t}\epsilon_c)/\sqrt{\bar{\alpha}_t}. The proxy ℓt(x)≈λf(x^0,c(x),c)\ell_t(x) \approx\lambda f(\hat{x}_{0,c}(x),c) gives the chain-rule correction

Δμt≈λΣtJtT∇x^0f(x^0,c,c),Jt=∂x^0,c∂xt.(16)\Delta\mu_t \approx\lambda\Sigma_t\mathbf{J}_t^{\mathsf{T}}\nabla_{\hat{x}_0}f(\hat{x}_{0,c},c), \qquad\mathbf{J}_t = \frac{\partial\hat{x}_{0,c}}{\partial x_t}. \tag*{(16)}

The reverse mean is affine in noise prediction, μt=Atx−Ctϵ\mu_t = A_tx - C_t\epsilon with Ct>0C_t > 0, so Δϵt=−Ct−1Δμt\Delta\epsilon_t = -C_t^{-1}\Delta\mu_t. For isotropic covariance Σt=st2I\Sigma_t = s_t^2\mathbf{I}, setting ζ(t)=λst2/Ct\zeta(t) = \lambda s_t^2/C_t yields Eq. (5). This completes the derivation of Proposition 3.2.

The CFG-Specific VOG Controller

We instantiate the terminal objective with the conditional log-ratio defined in § 3.2. Write ϵc=ϵθ(xt,t,c)\epsilon_c = \epsilon_\theta(x_t,t,c), ϵu=ϵθ(xt,t,∅)\epsilon_u = \epsilon_\theta(x_t,t,\varnothing), gt=ϵc−ϵug_t = \epsilon_c-\epsilon_u, and w=ω−1w=\omega-1. Standard CFG is ϵc+wgt\epsilon_c+w g_t.

Conditional clean-prediction pullback

At noise level tt, the model score contrast is sc−su=−gt/1−αˉts_c-s_u=-g_t/\sqrt{1-\bar{\alpha}_t}. We use this contrast as a proxy for the clean-space log-ratio gradient. Its negative sign cancels the mean-to-noise sign in the preceding derivation. Let Jc,t=∂ϵc/∂xt\mathbf{J}_{c,t}=\partial\epsilon_c/\partial x_t and ρt=1−αˉt\rho_t=\sqrt{1-\bar{\alpha}_t}. Differentiating the conditional clean prediction gives

JtTgt=1αˉt(gt−ρtJc,tTgt).(17)\mathbf{J}_t^{\mathsf{T}}g_t = \frac{1}{\sqrt{\bar{\alpha}_t}}\left(g_t-\rho_t\mathbf{J}_{c,t}^{\mathsf{T}}g_t\right). \tag*{(17)}

The positive scalar factor is included in the guidance schedule.

Factorized VJP

We compute the sensitivity Jc,tT1\mathbf{J}_{c,t}^{\mathsf{T}}\mathbf{1} with one backward pass and combine it element-wise with the guidance contrast. The difference from the full VJP is

Jc,tTgt=(Jc,tT1)⊙gt+rt,\mathbf{J}_{c,t}^{\mathsf{T}}g_t = (\mathbf{J}_{c,t}^{\mathsf{T}}\mathbf{1})\odot g_t+r_t,
rt,i=∑j∂ϵc,j∂xt,i(gt,j−gt,i).(18)r_{t,i}=\sum_j\frac{\partial\epsilon_{c,j}}{\partial x_{t,i}}(g_{t,j}-g_{t,i}). \tag*{(18)}

This identity follows by adding and subtracting gt,ig_{t,i} in each coordinate sum. For the influence set I(i)={j:∂ϵc,j/∂xt,i≠0}\mathcal{I}(i)=\{j:\partial\epsilon_{c,j}/\partial x_{t,i}\ne0\}, the triangle inequality bounds the residual by

∣rt,i∣≤(∑j∈I(i)∣∂ϵc,j∂xt,i∣)max⁡j∈I(i)∣gt,j−gt,i∣.(19)|r_{t,i}|\le\left(\sum_{j\in\mathcal{I}(i)}\left|\frac{\partial\epsilon_{c,j}}{\partial x_{t,i}}\right|\right)\max_{j\in\mathcal{I}(i)}|g_{t,j}-g_{t,i}|. \tag*{(19)}

For an empty influence set, the maximum is defined as zero. The residual also vanishes when the contrast is constant on the influence set, including the case of a diagonal Jacobian. Dropping it gives the practical controller

ϵvog=ϵc+w[1−ρt∇xt(1Tϵc)]⊙gt.(20)\epsilon_{\mathrm{vog}}=\epsilon_c+w\left[1-\rho_t\nabla_{x_t}(\mathbf{1}^{\mathsf{T}}\epsilon_c)\right]\odot g_t. \tag*{(20)}

The sensitivity factor can be cached independently of the contrast. The element-wise update also preserves zero coordinates of the CFG direction. Stride and momentum are specified in § 3.4.

Choice of differentiation branch

We differentiate the conditional prediction that defines the reference process. To examine this choice, consider differentiating ϵcfg=ϵc+wgt\epsilon_{\mathrm{cfg}}=\epsilon_c+w g_t instead. With Ju,t=∂ϵu/∂xtJ_{u,t}=\partial\epsilon_u/\partial x_t and Acfg,tA_{\mathrm{cfg},t} the resulting clean-prediction Jacobian, its pullback is

wAcfg,tTgt=wJtTgt−w2ραˉt(Jc,t−Ju,t)Tgt.(21)w A_{\mathrm{cfg},t}^{\mathsf{T}}g_t=wJ_t^{\mathsf{T}}g_t-\frac{w^2\rho}{\sqrt{\bar{\alpha}_t}}(J_{c,t}-J_{u,t})^{\mathsf{T}}g_t. \tag*{(21)}

The additional term is quadratic in guidance strength. Using the conditional branch keeps the sensitivity map independent of ww at a fixed state. Table S2 evaluates the two choices.

ControllerFID (↓\downarrow)Relative runtime
Pullback through ϵcfg\epsilon_{\mathrm{cfg}}5.48–
Pullback through ϵc\epsilon_c (VOG)2.041.55×

Table S2. Differentiation-branch ablation on DiT-XL/2 (256 × 256), with VJP stride KVOG=4K_{\mathrm{VOG}}=4. Runtime is relative to CFG.

Sampling procedure.

Algorithm S1 VOG factorized core controller

Require: Conditional/unconditional predictor, condition cc, guidance strength ω\omega, and baseline numerical sampler.

  1. Sample xTx_T; set w=ω−1w=\omega-1.

  2. for each reverse sampling step tt do

  3. Compute ϵc=ϵθ(xt,t,c)\epsilon_c=\epsilon_\theta(x_t,t,c), ϵu=ϵθ(xt,t,∅)\epsilon_u=\epsilon_\theta(x_t,t,\varnothing), and gt=ϵc−ϵug_t=\epsilon_c-\epsilon_u.

  4. Compute ht=∇xt(1Tϵc)h_t=\nabla_{x_t}(\mathbf{1}^{\mathsf{T}}\epsilon_c) by one VJP.

  5. Set ρt=1−αˉt\rho_t=\sqrt{1-\bar{\alpha}_t} for the VP parameterization.

  6. Form ϵvog=ϵc+w(1−ρtht)⊙gt\epsilon_{\mathrm{vog}}=\epsilon_c+w(1-\rho_t h_t)\odot g_t as in Eq. (20).

  7. Advance the numerical sampler using ϵvog\epsilon_{\mathrm{vog}}.

  8. end for

  9. return Final sample.

Architecture-specific schedules, momentum, and VJP stride extend this core as described in § 3.4.

Parameter Control: Target and Implementation

The local objective and its distillation gradient are given in Eqs. (6) and (9). Here we specify how DVOC constructs the target and computes the selected parameter derivatives.

Endpoint optimization

Let r=ρ∥y∥2r=\rho\lVert y\rVert_2, y(0)=yy^{(0)}=y, and g(j)=∇yR(y(j),c)g^{(j)}=\nabla_yR(y^{(j)},c). We use normalized projected ascent:

y(j+1)=ΠB(y,r)(y(j)+ηrJg(j)∥g(j)∥2+ϵ),j=0,…,J−1.(22)y^{(j+1)}=\Pi_{B(y,r)}\left(y^{(j)}+\frac{\eta r}{J}\frac{g^{(j)}}{\lVert g^{(j)}\rVert_2+\epsilon}\right),\qquad j=0,\ldots,J-1. \tag*{(22)}

The Euclidean projection clips the displacement from yy to length rr. We use ϵ=10−12\epsilon=10^{-12}, J=2J=2, η=1\eta=1, ρ=0.1\rho=0.1, and zero ascent momentum. The frozen FP32 decoder and reward evaluator supply the latent gradients. The final target is y+=sg⁡(y(J))y^+=\operatorname{sg}(y^{(J)}).

Cached endpoint connector

Sample the Euler trajectory without retaining its computation graph and cache states xtix_{t_i}, velocities vˉi\bar{v}_i, and increments hi=σi+1−σih_i = \sigma_{i+1} - \sigma_i. The endpoint is y=xT+∑ihivˉiy = x_T + \sum_i h_i\bar{v}_i. Choose a size-KK subset A\mathcal{A} uniformly without replacement and re-evaluate its velocities with gradients enabled. Define

y^ϕ=y+∑i∈Ahi[vϕ(sg⁡(xti),ti,c)−sg⁡(vˉi)],yϕA=y^ϕ+sg⁡(y−y^ϕ),JA=∑i∈Ahi∂ϕvϕ(sg⁡(xti),ti,c).(23)\begin{aligned} \hat{y}_{\phi} &= y + \sum_{i\in\mathcal{A}} h_i\left[v_{\phi}(\operatorname{sg}(x_{t_i}),t_i,c)-\operatorname{sg}(\bar{v}_i)\right],\\ y_{\phi}^{\mathcal{A}} &= \hat{y}_{\phi}+\operatorname{sg}(y-\hat{y}_{\phi}),\\ \mathbf{J}_{\mathcal{A}} &= \sum_{i\in\mathcal{A}}h_i\partial_{\phi}v_{\phi}(\operatorname{sg}(x_{t_i}),t_i,c). \tag*{(23)} \end{aligned}

All cached quantities are detached. The connector equals yy in the forward pass and passes parameter gradients through the selected velocity evaluations. We insert it into Eq. (8) and optimize the LoRA parameters with AdamW. Checkpoint evaluation uses the EMA adapter.

Mean Shift of CFG under the VP Probability Flow ODE

This section derives the linearized mean shift in Eq. (1). Let βvp(t)\beta_{\mathrm{vp}}(t) denote the VP noise rate, αˉt\bar{\alpha}_t the cumulative signal power, and μy\mu_y the mean of class yy. The posterior-weighted class mean is μˉ(xt)=∑yp(y∣xt)μy\bar{\mu}(x_t)=\sum_y p(y\mid x_t)\mu_y.

Class contrast and guided dynamics

Assumption E.1 (Bayes-optimal branch mixing). The denoisers estimate conditional means, so the unconditional clean prediction satisfies x^0uc=∑yp(y∣xt)x^0y\hat{x}^{\mathrm{uc}}_0=\sum_y p(y\mid x_t)\hat{x}^{y}_0.

Assumption E.2 (Locally affine class denoisers). The class denoisers have the local form x^0y(xt)=μy+Mt(xt−μy)\hat{x}^{y}_0(x_t)=\mu_y+M_t(x_t-\mu_y), with a symmetric contraction MtM_t shared across classes.

Subtracting the unconditional mixture from the target-class prediction gives (I−Mt)(μc−μˉ(xt))(I-M_t)(\mu_c-\bar{\mu}(x_t)). Since the clean-prediction map is affine in noise prediction, CFG induces

x^0cfg(xt)=x^0uc(xt)+ω(I−Mt)(μc−μˉ(xt)).(24)\hat{x}^{\mathrm{cfg}}_0(x_t)=\hat{x}^{\mathrm{uc}}_0(x_t)+\omega(I-M_t)(\mu_c-\bar{\mu}(x_t)). \tag*{(24)}

Write Λ1(t)=βvp(t)αˉt/[2(1−αˉt)]\Lambda_1(t)=\beta_{\mathrm{vp}}(t)\bar{\alpha}_t/[2(1-\bar{\alpha}_t)] and Λ2(t)=−βvp(t)αˉt/[2(1−αˉt)]\Lambda_2(t)=-\beta_{\mathrm{vp}}(t)\sqrt{\bar{\alpha}_t}/[2(1-\bar{\alpha}_t)]. Substituting the denoiser-score relation into the VP probability flow ODE gives

dxtdt=Ft(xt)+ωΛ2(t)(I−Mt)(μc−μˉ(xt)),Ft(x)=Λ1(t)x+Λ2(t)x^0uc(x).(25)\frac{dx_t}{dt}=F_t(x_t)+\omega\Lambda_2(t)(I-M_t)(\mu_c-\bar{\mu}(x_t)),\qquad F_t(x)=\Lambda_1(t)x+\Lambda_2(t)\hat{x}^{\mathrm{uc}}_0(x). \tag*{(25)}

The ODE is integrated from noise time TT to clean time 00.

Transport of the guidance perturbation

Let xtucx_t^{\mathrm{uc}} be the unguided trajectory with the same initial noise and Φ(0,τ)\Phi(0,\tau) the state transition matrix for ∂xFt\partial_xF_t along this trajectory. Define

Ψ(τ)=Φ(0,τ)βvp(τ)αˉτ2(1−αˉτ)(I−Mτ).(26)\Psi(\tau)=\Phi(0,\tau)\frac{\beta_{\mathrm{vp}}(\tau)\sqrt{\bar{\alpha}_{\tau}}}{2(1-\bar{\alpha}_{\tau})}(I-M_{\tau}). \tag*{(26)}

Linearizing FtF_t around xtucx_t^{\mathrm{uc}} and applying variation of constants gives

x0−x0uc≈ω∫0TΨ(τ)(μc−μˉ(xτ)) dτ.(27)x_0-x_0^{\mathrm{uc}}\approx\omega\int_0^T\Psi(\tau)(\mu_c-\bar{\mu}(x_{\tau}))\,d\tau. \tag*{(27)}

Assuming the transported contrast is integrable in expectation, taking expectations and using E[x0uc]=μglobal\mathbb{E}[x_0^{\mathrm{uc}}] = \mu_{\mathrm{global}} gives Eq. (1). The expectation includes the path-dependent kernel. The guided class-posterior gap also depends on ω\omega.

Subspace interpretation. If every class mean lies in SS, the contrast lies in SS on each trajectory. Its normal response is therefore determined by P⊥Ψ(τ)P∥P_{\perp}\Psi(\tau)P_{\parallel}. When MtM_t preserves SS, both the local filter and the unguided linearized flow preserve it, so this normal response vanishes. A learned denoiser can introduce cross-subspace coupling, whose effect depends on the transported contrast. The projection property of VOG’s exact linear-MMSE correction is proved directly in Theorem 3.3.

Detailed Experimental Settings

General Configuration and Hyperparameters

The VOG class-conditional sampling experiments use JAX. The DVOC and baseline post-training comparison uses PyTorch. All FID scores use 50k generated samples per configuration. Sampling parameters follow standard practice for each architecture: for EDM2-S we use an evaluation batch size of 8 per device. For DiT models we use 250 sampling steps with a per-process batch size of 8 and the MSE-tuned VAE from the official Stable Diffusion repository.

Table S4: Hyperparameters and settings for ImageNet-256 and ImageNet-512. The experiments are conducted using the DiT-XL/2 architecture. The column ω\omega denotes the guidance scale. The column Guidance interval specifies the range of noise levels σ\sigma where classifier-free guidance is applied. For our proposed method (+Ours), kk represents the frequency of regularization steps (computing gradients every kk steps), and β\beta indicates the momentum coefficient. The parameter ss refers to the exponent used in Guidance cosine.

We tune the guidance weight ω\omega separately for each architecture and task. The specific hyperparameter configurations and the chosen ω\omega ranges are detailed in Table S3 and Table S4. All class-conditional generation results were obtained using eight NVIDIA RTX 4090 GPUs. The DVOC comparison uses eight NVIDIA L20X GPUs per run and evaluates the checkpoint after 20 updates.

ResolutionMethodω\omegaKKβ\betassGuidance intervalEMAm\mathrm{EMA}_mEMAg\mathrm{EMA}_g
64 × 64EDM2-L
64 × 64Classifier-free guidance1.15---Full0.0400.040
64 × 64+ VOG (Ours)1.0520.15-Full0.0400.040
64 × 64Guidance interval1.15---(0.3,1.5]0.0400.040
64 × 64+ VOG (Ours)1.1020.15-(0.3,1.5]0.0400.040
64 × 64Guidance cosine1.05--0.3Full0.0400.040
64 × 64+ VOG (Ours)1.0520.150.3Full0.0400.040
64 × 64EDM2-S
64 × 64Autoguidance1.70---Full0.0450.110
64 × 64+ VOG (Ours)1.7020.15-Full0.0450.110
512 × 512EDM2-S
512 × 512Classifier-free guidance1.40---Full0.0250.025
512 × 512+ VOG (Ours)1.6020.15-Full0.0250.025
512 × 512Guidance interval2.10---(0.28,2.90]0.0250.025
512 × 512+ VOG (Ours)2.3020.15-(0.28,2.90]0.0250.025
512 × 512Guidance cosine1.60--0.3Full0.0250.025
512 × 512+ VOG (Ours)1.6520.150.3Full0.0250.025

Table S3. Hyperparameters and settings for ImageNet-64 and ImageNet-512. The experiments are conducted using EDM2-L and EDM2-S architectures. The column ω\omega denotes the guidance scale. The column Guidance interval specifies the range of noise levels σ\sigma where classifier-free guidance is applied. For our proposed method (+Ours), kk represents the frequency of regularization steps (computing gradients every kk steps), and β\beta indicates the momentum coefficient. The parameter ss refers to the exponent used in Guidance cosine. The columns EMAm\mathrm{EMA}_m and EMAg\mathrm{EMA}_g indicate the EMA decay lengths for the main generative model and the guiding model, respectively.

ResolutionMethodω\omegaKKβ\betassGuidance interval
256 × 256Classifier-free guidance1.50---Full
256 × 256+ VOG (Ours)1.5040.15-Full
256 × 256Guidance interval1.50---(0.34, 1.02]
256 × 256+ VOG (Ours)1.9020.15-(0.34, 1.02]
256 × 256Guidance cosine1.50--0.3Full
256 × 256+ VOG (Ours)1.5020.150.3Full
512 × 512Classifier-free guidance1.50---Full
512 × 512+ VOG (Ours)1.5020.15-Full
512 × 512Guidance interval2.50---(0.34, 1.02]
512 × 512+ VOG (Ours)2.5020.15-(0.34, 1.02]
512 × 512Guidance cosine1.50--0.3Full
512 × 512+ VOG (Ours)1.5020.200.3Full

Table S4.

Structure of the 2D Synthetic Distributions

The branching-tree datasets are Gaussian mixtures. Table S5 summarizes their component weights and spatial spread. The entropy-based effective count KentK_{\mathrm{ent}} is the exponential of the weight entropy, and the HHI-based count KhhiK_{\mathrm{hhi}} is the inverse sum of squared weights. Top 5% Mass is the total weight of the largest 5% of components. The covariance trace measures total spatial variance, and MST is the total Euclidean length of the minimum spanning tree connecting component means.

DatasetKKKentK_{\mathrm{ent}}KhhiK_{\mathrm{hhi}}Top 5% Masstr(Cov)MST
CLS-2 (Class A)1016249.8190.290.54290.227515.35
CLS-3 (Class A)1036214.05102.910.63610.402918.57

Table S5. Quantitative difficulty metrics for the 2D synthetic datasets (Class A).

As indicated in Table S5, the CLS-3 dataset is the harder of the two. Geometrically, CLS-3 exhibits a much larger spatial dispersion, evidenced by the higher total variance (tr(Cov) ≈\approx 0.40 vs. 0.23) and a more extensive Minimum Spanning Tree (18.57 vs. 15.35). Statistically, the distribution of CLS-3 is more skewed. Despite having a similar number of components, its probability mass is more concentrated in the top few modes (Top 5% Mass ≈\approx 63.6%), resulting in a lower entropy-based effective number of components (KentK_{\mathrm{ent}}). This combination of high spatial variance and weight imbalance requires the generative model to capture a broader manifold structure while handling sharper density peaks.

DVOC and Baseline Reproduction Details

The main DVOC configuration uses K=3K = 3, with K=1,2K = 1, 2 as active-step ablations. We compare against the untrained Z-Image-Turbo reference and three trained methods: ReFL, DRTune, and DiffusionOPSD. All methods use shared prompts and initial noise for paired evaluation.

Hardware and update accounting. Each trained main method and both DVOC KK ablations use one eight-GPU node, with reward models colocated on the same devices. The untrained reference uses the same hardware for generation and scoring.

One training step denotes one optimizer update, including its generated trajectories, reward or target computation, accumulated backward passes, and parameter update. It is distinct from an Euler denoising step. The effective trajectory batch is 8×1×72=5768 \times1 \times72 = 576, comprising 48 prompt groups with 12 trajectories each. Thus update 20 corresponds to 11,520 training trajectories. ReFL and DRTune stop their training rollouts at the selected late states. Their separate 512-prompt reward calibration is initialization overhead, outside this per-update trajectory count. The configured runs continue to 100 updates; the main table evaluates their update-20 EMA checkpoints.

MethodBackward pathReward target or lossWeight decay
ReFLOne selected late-state evaluationNegative standardized reward; loss scale 10−310^{-3}; no hinge10−210^{-2}
DRTuneThree connected evaluations in a truncated rolloutSame reward loss and scale as ReFL; early-stop ratio 0.410−210^{-2}
DiffusionOPSDOne queried statePositive and negative PGD targets; old-policy mixing; J=2J = 2, ρ=0.1\rho= 0.1, η=1\eta= 110−410^{-4}
DVOC (K=3K = 3)Three cached states; detached state dependenciesActual-endpoint PGD target; J=2J = 2, ρ=0.1\rho= 0.1, η=1\eta= 1; normalized target MSE10−410^{-4}

Table S6. Method-specific training settings. A differentiated evaluation carries a parameter gradient.

DVOC settings. The active indices are sampled uniformly without replacement from all nine Euler steps; no N/KN/K correction rescales their sum. PGD uses two normalized ascent steps, radius 0.1∥y∥20.1\lVert y\rVert_2, step multiplier 1, zero momentum, and target-construction microbatch size one. The actual endpoint, optimized target, and cached rollout states are detached. The squared fitting residual is divided by its detached mean absolute displacement, floored at 10−510^{-5}. Rollout generation uses three trajectories per GPU call and 24 calls per update; fitting uses 72 one-trajectory microbatches. The K=1,2K = 1, 2 ablations change only the active-step count.

ReFL and DRTune settings. Both use a fixed 512-prompt training-set calibration to standardize the mixed reward, followed by a negative-reward loss with scale 10−310^{-3} and no hinge, auxiliary pretraining loss, or reference KL/MSE. ReFL samples one evaluation state in the final 25% of the nine-step schedule. DRTune uses three connected trainable steps with an early-stop ratio of 0.4. The rollout prefix is computed without gradients and the unused suffix is omitted. The two methods share random draws for prompts, noise, state selection, and calibration.

DiffusionOPSD settings. DiffusionOPSD generates 24 batches of three trajectories per GPU and uses prompt-group advantages clipped at 5, with both advantage signs enabled and policy-mixing coefficient 1.

OPSD forms ascent and descent targets at the old policy’s one-step clean prediction from a queried rollout state. Its requested query noise level is 0.273, mapped to the available schedule. Each target uses two normalized steps, relative radius 0.1, multiplier 1, zero momentum, and microbatch one; both targets are accepted without reward certification. It fits one queried state and applies no auxiliary reference-prediction penalty.

Training time measurement. For the four main methods, we measure elapsed time between consecutive optimizer updates. Table S7 reports means and standard deviations over 18 intervals from updates 2–20, excluding update 11 because it follows evaluation and checkpointing. The intervals include sampling, reward or target construction, fitting, and routine communication overhead. Initialization, periodic evaluation and checkpointing, and the separate offline evaluation are excluded.

For K=1,2K = 1, 2, the times are estimated from consecutive rollout-completion records and marked †. Each interval spans one fitting phase and the following rollout. Applying the same estimator to K=3K = 3 reproduces its measured mean of 458.0 s over the same 18 intervals, with a maximum per-interval difference of 1.4 s.

The same exclusion rule over the 100-update runs gives mean times of 272.9 s (ReFL), 337.2 s (DRTune), 396.8 s (DiffusionOPSD), and 456.5 s (DVOC), using 90 intervals per method.

Scorer definitions. All methods use the same evaluator weights and preprocessing. HPSv3++ uses iteration condition 0.3. Q-Align quality and aesthetics are evaluated with OneAlign, and DeQA uses DeQA-Score-Mix3. These two models use half-precision inference with batch size one. AES and Q-Align aesthetics are distinct evaluators and appear in separate columns.

Evaluation setup. Evaluation uses the EMA adapter after 20 optimizer updates and the original Z-Image-Turbo weights as the untrained reference. Each method generates images for the complete 1,000-prompt test set, and all seven evaluators score the same images. Online evaluations use a different noise sampling scheme and HPSv3++ condition 0.0; their results are reported separately in Appendix § 13.§ 1.

Scope of the comparison. The trained rows compare fixed update-20 checkpoints from runs configured for a longer training schedule. They share a nominal generated-sample and optimizer-update budget but differ in gradient paths, reward calls, and targets, giving different per-update computational costs. The main table’s gray differences are relative to the paired untrained original model. The in-domain/OOD split specifies evaluator use during optimization.

Paired uncertainty. For paired metric differences did_i, Table S8 reports dˉ±1.96sd/1000\bar{d}\pm1.96s_d/\sqrt{1000}. These per-metric intervals summarize variation across the fixed 1,000 prompt/image pairs.

Endpoint-Target Ablations

We compare three target constructions at K=3K = 3: the actual endpoint with two ascent steps (J=2J = 2), the actual endpoint with one step (J=1J = 1), and a late-state clean-prediction proxy with two steps. These paired runs share the initial LoRA adapter and random streams for prompts, initial noise, active states, and reward evaluations. Their common initialization is separate from the main experiment.

Target construction. The endpoint target starts from the terminal latent yy. The proxy starts from a=xtq−σqvϕ(xtq,tq)a = x_{t_q} - \sigma_q v_\phi(x_{t_q}, t_q) at the schedule point nearest the requested noise level 0.273, giving q=7q = 7 and σq≃0.3050\sigma_q \simeq0.3050. Its optimized displacement is transferred to the endpoint as y+(a+−a)y + (a^+ - a). All variants use radius 0.1∥y∥20.1\lVert y\rVert_2, step multiplier η=1\eta= 1, zero momentum, and ascent-step length r/Jr/J.

Evaluation and results. Each run uses 20 updates, 576 trajectories per update, and the shared optimizer configuration. The update-20 EMA adapters generate the same 1,000 test prompts with matched seeds and initial latents. All seven evaluators score these saved images.

Two ascent steps improve HPSv3++, Q-Align quality, AES, and DeQA relative to one step. The gains in Q-Align quality, AES, and DeQA are 0.0202, 0.0692, and 0.1203, respectively. CLIP is higher for J=1J = 1. Relative to the proxy, the two-step endpoint target improves Q-Align quality, Q-Align aesthetics, and DeQA by 0.0125, 0.0215, and 0.0100. The complete score profile is shown in Table 3.

Computation. On the eight-GPU node, mean update times are 461.7 seconds for endpoint J=2J = 2, 414.1 seconds for endpoint J=1J = 1, and 463.0 seconds for proxy J=2J = 2. These measurements use updates 2–20 and exclude evaluation and checkpoint I/O. One target step reduces update time by 10.3%.

Additional Experimental Results

We provide supplementary quantitative and qualitative evaluations to further validate the behavior of the evaluated guidance and post-training settings.

Qualitative comparison of reward-based adaptation. Figure S1 compares DVOC with the base model and the three trained baselines across four scenes. These examples complement the evaluator scores in Table 2 by showing how adaptation affects the visual appearance of generated images.

Computational efficiency. Table S9 details the computational overhead of VOG. Due to the inclusion of vector-Jacobian products, VOG increases runtime by 1.46×1.46\times to 1.98×1.98\times and peak GPU memory by 1.21×1.21\times to 1.72×1.72\times across varying resolutions. Despite this overhead, VOG remains practically feasible, with the per-device sampling benchmark fitting on one NVIDIA RTX 4090 GPU. DVOC training uses the separate allocation reported above.

ModelRes/BSCFG / VOG Runtime (sec)
EDM2-S64/83.66 / 5.33 (1.46x)
DiT-XL/2256/810.97 / 17.04 (1.55x)
EDM2-S512/84.66 / 9.22 (1.98x)
DiT-XL/2512/839.74 / 65.03 (1.64x)
ModelRes/BSCFG / VOG Memory (GB)
EDM2-S64/11.18 / 1.79 (1.53x)
DiT-XL/2256/14.14 / 4.99 (1.21x)
EDM2-S512/11.19 / 1.81 (1.53x)
DiT-XL/2512/14.18 / 7.18 (1.72x)

Table S9. Runtime and peak memory of VOG relative to CFG, reported as CFG / VOG with the ratio in parentheses. Res/BS denotes resolution and batch size; all measurements are taken on a single NVIDIA RTX 4090.

Performance in few-step regimes (Table S10). Table S10 compares 25-step sampling performance on DiT and SiT architectures. For SiT, although the model is trained to predict the interpolant velocity field vθ(x,t)v_\theta(x,t), we follow the diffusion-SDE sampling setting by converting the learned velocity into the corresponding score:

ModelMethodFIDRecallPrecisionω\omegaβ\betaKK
DiT-XL/2CFG7.580.4290.7841.50--
VOG4.440.5450.6991.500.152
SiT-XL/2CFG6.200.5550.6851.50--
VOG3.890.5160.7351.100.152

Table S10. Quantitative comparison of 25-step sampling on DiT-XL/2 and SiT-XL/2 (256 × 256, ImageNet). We report FID (↓), Recall (↑), and Precision (↑), together with the guidance scale ω\omega, the momentum coefficient β\beta, and the VJP recomputation interval KK.

sθ(x,t)=1σtαtvθ(x,t)−α˙txα˙tσt−αtσ˙t,(28)s_\theta(x,t) = \frac{1}{\sigma_t}\frac{\alpha_t v_\theta(x,t)-\dot{\alpha}_t x}{\dot{\alpha}_t\sigma_t-\alpha_t\dot{\sigma}_t}, \tag*{(28)}

where αt\alpha_t and σt\sigma_t denote the SiT interpolant coefficients. We then apply CFG and VOG on this converted score and use the same diffusion SDE sampler for evaluation. VOG improves FID on both architectures, from 7.58 to 4.44 on DiT-XL/2 and from 6.20 to 3.89 on SiT-XL/2. The two architectures trade Precision against Recall in opposite directions: on DiT-XL/2 the gain comes from coverage (Recall 0.429→0.5450.429\to0.545, Precision 0.784→0.6990.784 \rightarrow0.699, whereas on SiT-XL/2 it comes from sample quality (Precision 0.685→0.7350.685 \rightarrow0.735, Recall 0.555→0.5160.555 \rightarrow0.516). These configurations also use different tuned guidance scales (ω=1.50\omega=1.50 for DiT-XL/2, ω=1.10\omega=1.10 for SiT-XL/2). In both cases VOG moves the operating point toward a better FID with architecture-dependent Precision–Recall trade-offs. At 50 steps, DiT-XL/2 FID decreases from 3.69 to 2.99. The relative reduction therefore grows from 19% at 50 steps to 41% at 25 steps.

Quantitative analysis on 2D synthetic benchmarks. Table S11 provides the numerical evaluation corresponding to the 2D branching-tree visualizations in Figure 2. While standard CFG effectively filters outliers (high Precision), it loses coverage as the distribution complexity increases, evidenced by the drop in Recall to 0.7785 on the three-class dataset. VOG achieves higher F1-scores and lower Wasserstein distances in both settings, indicating that it aligns samples with the conditional target while better preserving the support of the data manifold.

MethodTwo-class (CLS=2) Prec.Rec.F1Wass.Three-class (CLS=3) Prec.Rec.F1Wass.
Ground Truth0.99161.00000.99580.00210.98621.00000.99310.0026
No Guidance0.89480.99890.94400.00310.91940.99700.95670.0035
Standard CFG0.91100.87550.89290.00640.93920.77850.85140.0071
VOG (ours)0.92300.95680.93960.00430.95360.93280.94310.0042

Table S11. Quantitative results on 2D branching-tree generation. We report Precision, Recall, F1-score (higher is better), and Wasserstein distance (lower is better) for both Two-class and Three-class settings. Best results (excluding Ground Truth) are highlighted in bold.

Extended sensitivity analysis. Figure S3 and Figure S4 supplement the main text with guidance-scale sweeps on EDM2-S and DiT-XL/2. Across the sweep, VOG mitigates the coverage loss that standard CFG exhibits at high guidance scales (e.g., maintaining Recall ≈0.58\approx0.58 where CFG drops below 0.50) and avoids the accompanying FID spike, which widens the range of ω\omega over which guidance remains usable.

Figure S3. Quantitative comparison on EDM2-S (512 × 512) across varying guidance scales. We report FID (↓), Precision (↑), and Recall (↑) as a function of the guidance weight ωCFG\omega_{\mathrm{CFG}}. VOG (red) demonstrates superior robustness at high scales compared to standard CFG (blue), maintaining an FID of ≈ 2.4 at ω=2.0\omega= 2.0 while the baseline degrades beyond 4.0.

Figure S4. Quantitative comparison on DiT-XL/2 (256 × 256) across varying guidance scales. While standard CFG suffers from sharp performance degradation at high guidance strengths (e.g., FID > 3.40 at ω=1.7\omega= 1.7), VOG maintains structural fidelity (high Recall) and generation quality (FID < 2.60), effectively preventing manifold drift.

Composition with other guidance heuristics. VOG is complementary to existing heuristic schedules rather than a replacement for them. Applied on top of Cosine, Interval, and AutoGuidance (Table S12), it improves FID in every case, for instance reducing Interval CFG from 1.637 to 1.523 on EDM2-L, which supports its use as a plug-in guidance correction.

BenchmarkFIDPrecisionRecall
EDM2-S
+AutoGuidance1.0440.7010.626
+VOG (Ours)1.0310.6960.620
EDM2-L
+Standard CFG1.9930.7100.601
+VOG (Ours)1.4180.7140.597
+Cosine CFG1.6670.7260.587
+VOG (Ours)1.6490.7270.588
+Interval CFG1.6370.7300.597
+VOG (Ours)1.5230.7200.593

Table S12. VOG composed with different guidance heuristics on ImageNet 64 × 64. Each shaded row applies VOG on top of the heuristic immediately above it.

Figure S2b. FID as a function of the momentum coefficient β\beta and the VJP recomputation interval KK, on DiT-XL/2 (256 × 256).

Figure S2: Ablation studies on guidance-scale sensitivity (left) and hyperparameter robustness (right).

Reward and Evaluation Histories

We report 100-update training runs for ReFL, DRTune, DiffusionOPSD, and DVOC (K=3K = 3), with K=1,2K = 1, 2 confined to the ablation. The primary seven-metric comparison remains the fixed, paired update-20 evaluation in the main paper.

Training reward and evaluation reward. The training reward is 0.03846RHPSv3+++RCLIP0.03846R_{\mathrm{HPSv3++}} + R_{\mathrm{CLIP}}. ReFL and DRTune score one-step clean predictions from sampled late states, whereas DVOC and DiffusionOPSD score complete sampled endpoints. These quantities are plotted in separate panels in Figure S5; the two upper panels share their vertical scale. The lower panels show the HPSv3++ and CLIP components for DVOC and DiffusionOPSD.

Figure S5. Training reward histories. Separate panels distinguish sampled endpoints from late-state proxies. Thin lines show every raw update mean; strong lines show the trailing five-update mean. The vertical reference marks update 20. The HPSv3++ and CLIP panels show DVOC and DiffusionOPSD.

All raw update means are shown as thin, translucent lines. The stronger line is a trailing mean over at most five updates, using the same window for every method. The horizontal axis indicates the optimizer update associated with each training batch, from 1 to 100. Each point is the mean reward on that batch.

Online Evaluation

We evaluate each method at initialization and every ten updates through update 100, giving 11 checkpoints. Every evaluation uses the EMA adapter, the same 1,000 prompts, 102421024^2 outputs, nine sampling steps, guidance scale zero, and HPSv3++ iteration condition 0.0. Online evaluations use each trainer’s noise sampling scheme; the main comparison pairs initial noise across methods and uses HPSv3++ condition 0.3.

Figure S6 reports HPSv3++ and CLIP as in-domain metrics, alongside the held-out ImageReward evaluator. Markers show measured checkpoint means, with straight lines connecting successive evaluations. The first two panels of Figure 3 show the same CLIP and HPSv3++ histories. The combined evaluation reward is 0.03846 HPSv3 + + + CLIP.

Figure S6. Online evaluation histories. HPSv3++ and CLIP are in-domain evaluators; ImageReward is held out of training. Each marker is a checkpoint mean, with no smoothing. The vertical line marks update 20. The HPSv3++ condition and noise sampling differ from the paired evaluation in the main table.

For DVOC, the online weighted evaluation reward rises from 0.766 at update 20 to 0.911 at update 100. ImageReward rises from 1.267 to 1.366 over the same interval.

Active-Step Histories

Figure S7 complements the fixed update-20 ablation with all recorded histories for K=1,2,3K = 1, 2, 3. Each run uses the same nine-step sampler and two-step PGD target construction. Panel (a) shows training reward with the same raw-line and trailing-mean convention as Figure S5; panels (b)–(f) show unsmoothed online EMA evaluation. The three variants have exactly matching initialization scores on all four displayed evaluators.

Figure S7. DVOC active-step ablation over optimization. K=1,2K = 1, 2 appear only in the ablation; the main method remains K=3K = 3. Training reward includes all raw points and a trailing five-update mean. Evaluation panels show only recorded checkpoint means. The vertical line marks the main paper’s update-20 checkpoint.

Larger KK improves the later weighted evaluation reward in these runs, while K=1K = 1 retains a higher

late-training AES score. ImageReward varies without a consistent ordering across all checkpoints. These histories complement the fixed update-20 comparison; per-update costs are reported in Table S7.

References

  1. [1]Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proc. ICLR, 2024.arxiv.org/abs/2302.07121
  2. [2]Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In Proc. ICLR, 2024.arxiv.org/abs/2305.13301
  3. [3]Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lee, and Jianfeng Lu. What does guidance do? a fine-grained analysis in a simple setting. In Proc. NeurIPS, 2024.
  4. [4]Hyungjin Chung, Jeongsol Kim, Michael Thompson McCann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In Proc. ICLR, 2023.arxiv.org/abs/2209.14687
  5. [5]Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. CFG++: Manifold-constrained classifier free guidance for diffusion models. In Proc. ICLR, 2025.arxiv.org/abs/2406.08070
  6. [6]Kevin Clark, Paul Vicol, Kevin Swersky, and David J. Fleet. Directly fine-tuning diffusion models on differentiable rewards. In Proc. ICLR, 2024.arxiv.org/abs/2309.17400
  7. [7]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proc. CVPR, 2009.
  8. [8]Alexander Denker, Francisco Vargas, Shreyas Padhy, Kieran Didi, Simon Mathis, Vincent Dutordoir, Riccardo Barbano, Emile Mathieu, Urszula Julia Komorowska, and Pietro Lio. DEFT: Efficient fine-tuning of diffusion models by learning the generalised h-transform. In Proc. NeurIPS, 2024.
  9. [9]Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In Proc. NeurIPS, 2021.
  10. [10]Carles Domingo-Enrich, Michal Drozdzal, Brian Karrer, and Ricky T. Q. Chen. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. In Proc. ICLR, 2025.
  11. [11]Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangkook Lee, and Kimin Lee. DPOK: Reinforcement learning for fine-tuning text-to-image diffusion models. In Proc. NeurIPS, 2023.arxiv.org/abs/2305.16381
  12. [12]Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer. In Proc. ICCV, 2023.DOI
  13. [13]Yingqing Guo, Hui Yuan, Yukang Yang, Minshuo Chen, and Mengdi Wang. Gradient guidance for diffusion models: An optimization perspective. In Proc. NeurIPS, 2024.arxiv.org/abs/2404.14743
  14. [14]Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J. Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon. Manifold preserving guided diffusion. In Proc. ICLR, 2024.arxiv.org/abs/2311.16424
  15. [15]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Proc. NeurIPS, 2017.
  16. [16]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.arxiv.org/abs/2207.12598
  17. [17]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Proc. NeurIPS, 2020.arxiv.org/abs/2006.11239
  18. [18]Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding a diffusion model with a bad version of itself. In Proc. NeurIPS, 2024.arxiv.org/abs/2406.02507
  19. [19]Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proc. CVPR, 2024.arxiv.org/abs/2312.02696
  20. [20]Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-Pic: An open dataset of user preferences for text-to-image generation. In Proc. NeurIPS, 2023.arxiv.org/abs/2305.01569
  21. [21]Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. In Proc. NeurIPS, 2024.
  22. [22]Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-GRPO: Training flow matching models via online RL. In Proc. NeurIPS, 2025.arxiv.org/abs/2505.05470
  23. [23]Yijun Liu, Jie Huang, Zeyue Xue, Yuming Li, Ruizhe He, Haoran Li, Shijia Ge, and Siming Fu. HPSv3++: Scaling reward models across the full spectrum of diffusion model capabilities. arXiv preprint arXiv:2606.14657, 2026.arxiv.org/abs/2606.14657
  24. [24]Toan Nguyen, Kien Do, Duc Kieu, and Thin Nguyen. h-Edit: Effective and flexible diffusion-based editing via Doob’s h-transform. In Proc. CVPR, 2025.arxiv.org/abs/2503.02187
  25. [25]William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proc. ICCV, 2023.arxiv.org/abs/2212.09748
  26. [26]Phillip Pope, Chen Zhu, Ahmed Abdelkader, Micah Goldblum, and Tom Goldstein. The intrinsic dimension of images and its impact on learning. In Proc. ICLR, 2021.arxiv.org/abs/2104.08894
  27. [27]Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation. CoRR, abs/2310.03739, 2023.arxiv.org/abs/2310.03739
  28. [28]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, pages 8748–8763, 2021.arxiv.org/abs/2103.00020
  29. [29]Seyedmortreza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M. Weber. CADS: Unleashing the diversity of diffusion models through condition-annealed sampling. In Proc. ICLR, 2024.arxiv.org/abs/2310.17347
  30. [30]Seyedmortreza Sadat, Otmar Hilliges, and Romann M. Weber. Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In Proc. ICLR, 2025.arxiv.org/abs/2410.02416
  31. [31]Mehdi S. M. Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. In Proc. NeurIPS, 2018.arxiv.org/abs/1806.00035
  32. [32]Christoph Schuhmann. LAION-Aesthetics. LAION blog, 2022.
  33. [33]Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Proc. NeurIPS, 2019.arxiv.org/abs/1907.05600
  34. [34]Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proc. ICLR, 2021.
  35. [35]Tongyi-MAI. Z-Image-Turbo. Model card and released checkpoint, 2025.
  36. [36]Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proc. CVPR, 2024.arxiv.org/abs/2311.12908
  37. [37]Ruoyu Wang, Boye Niu, Xiangxin Zhou, Yushi Huang, Tongliang Liu, and Chi Zhang. Exploring the design space of reward backpropagation for flow matching. arXiv preprint arXiv:2606.11075, 2026.arxiv.org/abs/2606.11075
  38. [38]Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie-Paule Cani, Victoria Fernández Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-free guidance weight schedulers. Transactions on Machine Learning Research, 2024.arxiv.org/abs/2404.13040
  39. [39]Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guangtao Zhai, and Weisi Lin. Q-Align: Teaching LMMs for visual scoring via discrete text-defined levels. In Proc. ICML, pages 54015–54029, 2024.arxiv.org/abs/2312.17090
  40. [40]Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Song, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffusion models. In European Conference on Computer Vision, pages 108–124. Springer, 2024.arxiv.org/abs/2405.00760
  41. [41]Mengfei Xia, Nan Xue, Yujun Shen, Ran Yi, Tieliang Gong, and Yong-Jin Liu. Rectified diffusion guidance for conditional generation. In Proc. CVPR, 2025.arxiv.org/abs/2410.18737
  42. [42]Jiazhen Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. ImageReward: Learning and evaluating human preferences for text-to-image generation. In Proc. NeurIPS, 2023.arxiv.org/abs/2304.05977
  43. [43]Shuchen Xue, Chongjian Ge, Shilong Zhang, Yichen Li, and Zhi-Ming Ma. Advantage weighted matching: Aligning RL with pretraining in diffusion models. In Proc. ICML, pages 143146–143169, 2026.arxiv.org/abs/2509.25050
  44. [44]Zhiyuan You, Xin Cai, Jinjing Gu, Tianfan Xue, and Chao Dong. Teaching large language models to regress accurate image quality scores using score distribution. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14483–14494, 2025.arxiv.org/abs/2501.11561
  45. [45]Jiwen Yu, Yinhua Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. FreeDoM: Training-free energy-guided conditional diffusion model. In Proc. ICCV, 2023.arxiv.org/abs/2303.09833
  46. [46]Kaiwen Zheng, Huayu Chen, Haotian Ye, Haoxiang Wang, Qinsheng Zhang, Kai Jiang, Hang Su, Stefano Ermon, Jun Zhu, and Ming-Yu Liu. DiffusionNFT: Online diffusion reinforcement with forward process. In Proc. ICLR, 2026.arxiv.org/abs/2509.16117
  47. [47]Wei Zhou, Xiongwei Zhu, Lingdong Kong, Bo Chen, Lei Zhang, Yongyuan Liang, Xiaoxia Hou, Ye Tian, Xian Sun, Yingshuo Wang, Linfeng Li, Shengqiong Wu, Leigang Qu, Feng Li, Wei Liu, Julian McAuley, and Tat-Seng Chua. On-policy self-distillation in diffusion models. arXiv preprint arXiv:2608.24646, 2026.arxiv.org/abs/2608.24646

Paper details

vlnc:W_p7a9t5hg
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents