betaFeedback

WorksComputer Science

FastGRN: Few-Step Generative Refinement Networks

Abstract

Generative Refinement Networks (GRN) generate an image by repeatedly re-predicting and refining every token of a discrete latent. They reach high fidelity, but need dozens of Markov steps. We therefore train few-step GRNs on the distribution of their own samples. We use its own uniform-state corruption to perform distribution matching on discrete tokens, which lets a discrete generator learn by backpropagation, and we pair noise with samples in the manner of rectified flow. We find that the steps of a few-step GRN do not carry the same amount of information or the same repair load. We therefore exploit its discrete refinement during training and build few-step GRNs in three parts. Oversampling samples the student with more steps than it was trained for and refines its judge on these samples. Non-uniform sampling trains only the last step and hands it most of the information at sampling time. Refinement confidence keeping holds on to the tokens the student is already sure of. We train a fast class-conditional generator on ImageNet 256 and, with the same method, a 2B text-to-image model. On ImageNet 256, the 3-step FastGRN beats the FID of GRN. On a 2B binary-token text-to-image GRN, a 4-step student also nearly matches the GenEval of GRN-T2I 2B.

Introduction

Discrete-token image generators have advanced quickly, from masked [10, 11] to autoregressive and scale-wise models [19, 28, 83, 84]. Generative Refinement Networks (GRN) [29] are the newest member of this family. GRN encodes an image into a near-lossless map of discrete tokens with a hierarchical binary quantizer (HBQ). At every step, a transformer re-predicts all tokens from a partly corrupted copy of its previous prediction. This design gives strong results on ImageNet 2562256^2 and GenEval, at the price of 50 refinement steps with classifier-free guidance (CFG) [33].

Simply cutting the number of steps does not work. With 4 of the 50 steps, GRN’s ImageNet samples fall apart (Figure S7). Every step revises every token, so we might expect later steps to repair a rushed start, yet in practice they cannot.

Figure S7. Few-step sampling without distribution matching fails (ImageNet 2562256^{2}, FID-10k). GRN with N steps, the fine-tuned and the paired-noise GRN (fine-tuned on teacher samples with independent and with paired noise) use GRN’s sampler; black dashed: teachers (79 NFE). Hollow diamond: 2-step FastGRN-L with non-uniform sampling (5.51; base student, same machine: 6.04).

The problem is GRN’s factorized output head. GRN corrupts a token by keeping it with some probability and otherwise replacing it with a uniformly random one, which matches the marginal of uniform-state discrete diffusion [3, 73, 78]. Seen this way, the few-step error is the conditional total correlation [91] of the tokens given the current state (Prop. S1). A factorized predictor trained with cross-entropy cannot represent this dependence. So a few-step GRN has to learn the joint law of its own samples, which is what distribution matching [100, 101] does for continuous generators.

Discrete distribution matching. Distribution matching minimizes a reverse KL between the generator’s distribution and the data distribution. A frozen real score and an online fake score judge the samples, and the loss is backpropagated through the generator’s samples by reparameterization. Tokens cannot be reparameterized, so existing discrete distillation methods fall back on policy gradients [108], relaxed tokens [111], straight-through estimators [88], approximate Jacobians [110] or discriminators [108, 111]. Our core observation is that GRN is close to discrete diffusion. GRN’s uniform-state corruption makes all of these unnecessary. The gradient of the reverse KL between the corrupted student and data laws with respect to each token’s logits follows directly from concrete scores [56, 62], and the posteriors of the real and fake scores give these concrete scores. For each token, the gradient simply raises the logits of the values that the real score finds more plausible than the fake score, and lowers the others.

Training and sampling. FastGRN (Figure 2) builds on this gradient in three parts. The first part trains the student, starting from a paired-noise initialization [102]. (i) A window restricts matching to moderate noise levels. Without the window, few-step matching collapses into blank images. (ii) Random steps draw a new number of student steps at every iteration. One shared fake score matches them all (Prop. S2). A single student then serves 4 to 8 steps at flat quality, while a fixed-step student blurs below its budget and drifts above it. The second part changes how the student is sampled (§ 4.3), because the steps of a few-step GRN carry different amounts of information and different repair loads. (iii) Oversampling samples the KK-step student with N>KN > K steps, more than it was trained for, and fine-tunes its real-score judge on these samples. This way, the student learns to do in KK steps what used to take NN. (iv) Non-uniform sampling redistributes the information that each sampling step of a few-step student carries. In the 2-step student, for example, the second step does 75% of the refinement, and the 2-step student reaches the quality of the 4-step one. The third part, (v) refinement confidence keeping (§ 4.4), holds on to the tokens the student is already sure of.

Figure 2. FastGRN trains a few-step GRN on its own samples. (a) Paired-noise initialization. (b) Only the student’s last step receives the gradient of Eq. (2). ❶ The window (W) skips the nearly clean noise levels. ❷ Random steps (RS) redraw KK every iteration. ❸ Oversampling (OS) fine-tunes the real score on N>KN > K-step samples. ❹ Non-uniform sampling (NS) enlarges the trained step’s share of the path (red). ❺ Refinement confidence keeping (RCK) keeps confident tokens.

With non-uniform sampling on top of oversampling, the 2-step FastGRN-B student matches the FID of its teacher, which needs 79 network function evaluations (NFE), and runs 12.7× faster per image (Figure 1). It is the only GAN-free student of a discrete generator that matches its teacher with two network calls (Table 2). On text-to-image, even our 2-step student scores a higher GenEval than the few-step students of other discrete generators (Table 3).

Figure 1. FastGRN turns a 50-step Generative Refinement Network into a 2–4 step generator. Left: 2562256^2 samples (GRN-T2I 2B, same noise per column; GRN-B). Right: ImageNet FID-50k vs. latency (numerals: steps; Table 1) and GenEval vs. NFE against the teacher’s own sampler with fewer steps.

MethodTeacherParamsNFEFID↓× T.↓
Di[M]O [110]MaskGIT17416.911.05
FPD [88]MaskGIT174†16.901.05
Soft-Di[M]O [111]MaskGIT174†16.400.97
Di[M]O [110, 111]MaskBit30512.891.74
Soft-Di[M]O [111]MaskBit30511.961.18
Soft-Di[M]O [111] (longer)MaskBit30511.560.94
DD2 [48]VAR-d2061915.431.60
DD1 [49]VAR-d1632727.821.87
SDTT [18, 102]MaskGIT174†4ᶜ8.97–
ReDi [102]MaskGIT174†4ᶜ7.58–
Di4C-d [30]MaskGIT174†4ᶜ6.571.00
DAG [66]MaskGIT174†47.81–
FastGRN-BGRN-B13023.731.00
FastGRN-LGRN-L45822.851.08
FastGRN-LGRN-L45832.570.97

Table 2. Few-step students of discrete generators on ImageNet 2562256^2, as reported. Params in millions; × T.: FID relative to the method’s own teacher. Notes: Sec. S2.

MethodTeacherParamsStepsGenEval↑\uparrow×\timesT.↑\uparrow
Di[M]O [110]MaskGen-L0.6B10.420.88
FPD [88]MaskGen-L0.6B10.450.94
Soft-Di[M]Or [111]MaskGen-L0.6B10.511.06
Soft-Di[M]Or s [111]MaskGen-L0.6B10.631.31
Show-o Turbo [98] (2562256^{2})Show-o1.3B20.490.84
Show-o Turbo [98] (2562256^{2})Show-o1.3B40.520.88
Show-o Turbo [98] (5122512^{2})Show-o1.3B20.560.83
Show-o Turbo [98] (5122512^{2})Show-o1.3B40.630.93
FastGRNGRN-T2I2B20.670.97
FastGRNGRN-T2I2B40.690.99

Table 3. Few-step students of discrete text-to-image generators, as reported. At 2562256^{2} unless noted; ×\timesT.: GenEval relative to the method’s own teacher. Notes: Sec. S2.

Contributions.

  • Discrete distribution matching. The per-token reverse-KL gradient under uniform-state corruption is computed directly from two posterior networks.

  • Three parts that turn the resulting gradient into a few-step GRN, namely the window with random steps, oversampling with non-uniform sampling, and refinement confidence keeping.

  • We propose the first few-step GRN models, covering ImageNet and a 2B text-to-image model.

Related work

Discrete visual generation. Masked [4, 10, 11, 43, 95] and autoregressive or scale-wise [19, 28, 83, 84] token generators compete with continuous diffusion and flow models [20, 34, 44, 45, 47, 68, 70, 81]. Unlike masked models, GRN [29] re-predicts every token at every step. Its corruption is the uniform-state kernel of discrete diffusion [3, 8], which UDLM [78], Duo [73], discrete flow matching [9, 24] and URSA [15] also use. GRN does not draw this connection, and our gradient is built on it.

Few-step continuous generation. Steps have been reduced by progressive distillation [74], consistency models [57, 58, 82], rectified flow [52, 53] and flow maps [21, 25]. Score and distribution matching [60, 90, 93, 99, 100, 101] trains a generator through a frozen real score and an online fake score, sometimes together with adversarial losses [77]. Our windowed matching is a discrete counterpart of this reverse KL. Comparing distributions after a noise kernel goes back to spread divergences [105] and the reverse diffusive KL [31]. Reflow [52] re-trains a model on teacher noise–sample pairs, and we use such pairs as an initialization.

Few-step discrete generation. Masked image models have been distilled by consistency distillation, as in Show-o Turbo [98], and by distribution matching. Di[M]O [110] matches a token-level Jeffrey divergence with an auxiliary model and replaces the Jacobian at the intermediate state with that of the one-step output. Soft-Di[M]O [111] relaxes tokens to expected embeddings and adds a GAN loss, and FPD [88] uses fixed-point matching with a straight-through estimator. Distilled Decoding [49] reduces autoregressive models to one or two steps. For discrete diffusion, Duo [73] distills a uniform-state model by discrete consistency distillation, Di4C [30] uses mixture students to model correlations, and SDTT [18] self-distills through time. DiDi-Instruct [108] matches integral KLs with policy gradients and a discriminator, D-MMD [35] distills with a discrete moment-matching objective, and Gao and Sun [23] match conditionals recovered from density ratios. ReDi [102] reduces the factorization error of discrete flows by rectifying the coupling, and our paired-noise initialization carries this idea to GRN. Discrete MeanFlow-style generators [39, 66] learn interval transitions, which add nothing beyond the posterior for per-token predictors (Lemma S1).

Any-step distillation and self-improvement. Shortcut and consistency (trajectory) models [21, 40, 82], flow maps [25, 72] and moment matching [76, 109] condition a network on the step size, and distribution matching has multi-step [100] and flexible-step [61] variants. Random steps need no extra conditioning (Prop. S2). Oversampling is a form of expert iteration [2, 80] and self-training [22, 96]. Prior work retrains the generator itself on its own samples [1, 5]. In FastGRN, only the judge sees them.

Step grids. Sampling schedules are usually optimized for an already trained model [67, 71, 87], and few-step students, including DMD2 [100], train on evenly spaced timesteps. FastGRN treats the training grid itself as a design choice.

Positioning. The discrete distillation methods above estimate, relax or approximate the gradient that DMD [101] obtains by reparameterization. Our matching keeps hard tokens and computes the reverse-KL gradient directly from two posterior networks. It combines the concrete-score–posterior identity of SEDD and UDLM [56, 78] with local-expectation gradients [85]. All three parts of FastGRN are new to discrete distribution matching.

Background: Generative Refinement Networks

Tokens and corruption. GRN [29] encodes an image with the HBQ tokenizer into discrete tokens xx, each taking one of VV values. For class-conditional ImageNet 2562256^2, the token map has 16×1616 \times16 positions with 16 channels, i.e. 4096 tokens with 16 ordered levels. The 2B text-to-image model uses 65,536 bits. During training, every token xix^i is corrupted independently at a time t∈[0,1]t \in[0,1] with the uniform-state kernel

kt(zi∣xi)=t δ(zi,xi)+1−tV,(1)k_t(z^i \mid x^i) = t \,\delta(z^i,x^i) + \frac{1-t}{V}, \tag*{(1)}

which keeps the token with probability tt and otherwise draws it uniformly at random, as in uniform-state diffusion [3, 78]. We write kqk_q for the same kernel at keep probability qq. A transformer with a factorized output head learns to predict the clean tokens from the corrupted tokens zz, the time and the condition with per-token cross-entropy (CE). At the optimum, it returns the posterior marginals p(xi∣z)p(x^i \mid z).

Sampling. GRN draws a uniform noise map ϵ\epsilon once and refines it on a time grid from 0 to 1. At every step, it samples clean tokens from the CFG-combined logits at temperature τ\tau. It then keeps each sampled token with probability equal to the next time on the grid and otherwise re-uses the frozen noise token. GRN-B/L sample with 50 steps and CFG on t∈(0.4,1)t \in(0.4, 1) (79 NFE), and the text-to-image model applies CFG at every step (100 NFE). Our students use the same sampler with KK equally spaced steps, and non-uniform sampling moves the first intermediate time t1t_1 to 0.25 (§ 4.3). Since CFG is distilled into the student, it costs KK NFE. KK is either fixed, K=2K = 2 or 4, or drawn at every iteration from a set K\mathcal{K} (§ 4.2), and any student can also run N≠KN \ne K steps.

Method

We first analyze the few-step error (§ 4.1) and then build FastGRN in three parts (§ 4.2 to § 4.4), with proofs in the supplement.

Why few steps fail

GRN’s output head is factorized. Given the current state, it predicts every token independently. Two tokens that straddle an object boundary, for example, may each pick a different position for it. Over many steps, the tokens can still coordinate through the intermediate states, and with few steps there is little room for this. The best that cross-entropy training can reach among factorized predictors is the product of the posterior marginals. The error that remains is the conditional total correlation [91] of the tokens given the state, which equals the sum of the per-token entropies minus the joint entropy (Prop. S1).

This is the factorization error studied for discrete diffusion and flows [30, 102]. The student as a whole, however, need not be factorized. Its last step is factorized only given its latent λ\lambda, which consists of the noise and the earlier states. The student therefore samples a mixture of product laws, which can represent dependence between tokens, and distribution matching shapes this mixture directly (§ 4.2). Informative initial noise lowers the error as well (Corollary S1).

Training the student by distribution matching

Paired-noise initialization. GRN is trained with fresh random tokens, so its first step ignores the noise. We first make the noise informative. We sample noise–image pairs from the guided teacher, keep the frozen initial noise of each sample, and fine-tune GRN with its own CE objective while filling the dropped tokens with the paired noise (Figure 2a). This way, the first step learns what the noise says about each token (Corollary S1). The resulting paired-noise GRN initializes the student. It already draws recognizable objects with two steps, while GRN fine-tuned with independent noise still draws noise (Table S8), so the student starts from a much better place.

Sampling steps NN24816
GRN-B (teacher at 79 NFE: 6.24)
GRN-B with fewer stepsn^{n}112.549.610.9–
Fine-tuned GRN156.968.316.79.56
Paired-noise GRN60.826.913.08.68
Matching without window (best before collapse)7.87c^{c}6.97––
FastGRN (W)6.656.42––
GRN-L
GRN-L with fewer stepsn^{n}147.884.123.6–
Fine-tuned GRN163.789.922.7–
Paired-noise GRN82.839.315.8–
FastGRN (W)6.075.42––
FastGRN (W+NS)h^{h}5.51–––

Table S8. Few-step sampling without distribution matching fails. FID-10k on ImageNet 2562256^2 versus the number of sampling steps. Notes: Sec. S4.

Windowed distribution matching. Let pθ,qp_{\theta,q} and pd,qp_{d,q} be the laws of the KK-step student’s samples and of the data (teacher) after corruption at keep probability qq. We minimize the reverse KL DKL(pθ,q∥pd,q)D_{\mathrm{KL}}(p_{\theta,q}\|p_{d,q}) averaged over q∼ρq \sim\rho (Eq. (5)), an equal mixture of GRN’s training law of tt and the uniform law, clamped to the window [0.02,qmax⁡][0.02, q_{\max}]. With qmax⁡=0.4q_{\max}=0.4, nearly half of the draws land at the upper end, so matching mostly happens at one moderate noise level. The corruption is invertible at every nonzero keep probability, so each term vanishes only when the student samples the data law. So the window changes the optimization path and keeps the same minimizer. A frozen real score (the fine-tuned GRN or the guided teacher) predicts the clean tokens under the data law, and a fake score, initialized from the fine-tuned GRN, does so under the student’s law. The fake score learns online with CE on detached student samples at all noise levels, with several updates per student update [32, 100], and is queried only inside the window.

Per-token gradient. Given its latent λ\lambda (the noise and the earlier states), the last student step samples token ii from πi=softmax⁡(ui)\pi^i = \operatorname{softmax}(u^i), with uiu^i its tempered logits. The KL gradient needs concrete scores [56, 62], the probability of a corrupted state with one token changed over that of the state itself. Under the uniform kernel, they follow directly from a clean-token prediction (Eq. (4)), made by the fake score for the student and by the real score for the data.

Proposition 1 (Discrete distribution matching gradient). Let Δai\Delta^i_a be the log concrete score of the fake model minus that of the real model for changing token ii to aa, at a corrupted student sample z′z'. For a one-step student,

∂DKL(pθ,q∥pd,q)∂ubi=Ex,z′[πbi(Δbi−∑aπaiΔai)].(2)\frac{\partial D_{\mathrm{KL}}(p_{\theta,q}\|p_{d,q})}{\partial u^i_b} = \mathbb{E}_{x,z'}\left[\pi^i_b\left(\Delta^i_b-\sum_a \pi^i_a\Delta^i_a\right)\right]. \tag*{(2)}

Put simply, the student gives up probability on a value that the fake score favors more than the real score does, relative to its average value. The inner sum runs over all values of each token and is computed in full, while the noise level, the latent and the other tokens are sampled, which makes Eq. (2) a Rao–Blackwellized (local-expectation) score-function estimator [50, 85] with the mean over values as its baseline. No relaxation [37], straight-through estimator or discriminator is needed. In training, we apply the bracket as a fixed (stop-gradient) weight on the logits (Alg. S1). With more steps, the earlier steps run without

gradient, and we differentiate only the last step given its input. Matching every step hurts Sec. S3.1, because the first step is only a draft.

For bits, the gradient is a single scalar per token. When the fake score explains the observed bit better than the real score, the student over-produces that bit, and the update moves probability to the other value (Corollary S2). Among ff-divergences, only the reverse KL has a gradient that two posterior networks can compute (Remark S1). What is left to choose is how to weight the noise levels.

The window. The window keeps matching away from the nearly clean end, where blank images look likely to the real score. Without it (matching up to keep probability 0.92), 2-step ImageNet matching collapses. Training starts out healthy, and then the samples turn into blank images (§ 5.3). Near the clean end, the cross-entropy part of the reverse KL pulls toward blank images, which the real score finds very likely (§ 9.5). The fake score that should push back is least reliable there, and the gradient is scaled down by the keep probability. Once we cut this end, the collapse disappears for every upper end of the window from 0.3 to 0.6.

Random steps. A distilled student should still behave like a GRN, whose appeal is that more steps buy more quality. A KK-step student, however, is a specialist. It fails with fewer steps and drifts away from the data with many more (Figure S1). So we draw the number of student steps anew at every iteration from a set K\mathcal{K} and run the whole matching iteration, fake-score updates included, with KK-step rollouts. The draws are stratified, so every block of ∣K∣\lvert\mathcal{K}\rvert iterations visits each step count once. A single fake score serves all step counts. Trained on samples of every count, it estimates the posterior of the mixture of their laws, and the update follows the last-step gradient of the matching objective with this mixture in place of the student’s law. A student that samples the data law at every step count in K\mathcal{K} is a global minimizer (Prop. S2). Such a student cannot rely on a single schedule. In every iteration, its last step receives a state reached by a different number of earlier steps, so it learns to finish whatever state it is given. A shared fake score is cheaper than one fake score per step count, each of which would see only a fraction of the fake updates, and it is also more stable (Figure S1). We draw KK from 2–4, and from 3–6 for more refinement headroom.

Figure S1. Random steps flatten the curve (GRN-B, FID-10k). (a) After 6k iterations, by sampling steps. (b) 4-step FID during training. One fake per KK: a separate fake score for every step count.

Oversampling and non-uniform sampling

Not all steps are equal. The steps of a few-step GRN carry different amounts of information and different repair loads. The first step sees only noise. It drafts nearly the same image for every noise of a class, and across noises its predicted token distributions differ far less than those of the last step (0.08–0.17 vs. 0.40–0.71; Table S5). So the last step does most of the repair. It keeps most of the draft it is given (67–87% of the committed tokens, against 21–29% by chance), so the draft fixes the layout and the last step fills in what is specific to the sample. On the uniform grid it decides 60% of the tokens, and it is the only step that receives gradient. The trained student also holds refinement that its own steps leave unused. Sampled with one extra step, the 2-step GRN-B student goes from FID-10k 6.65 to 6.04, below its teacher’s 6.24 (Figure 4b). Oversampling turns this unused refinement into a better judge, and non-uniform sampling moves the repair to the step that is trained.

Figure 4. Each component pays off (ImageNet 2562256^{2}, FID-10k, GRN-B except (c); black dashed: teacher). (a) Without the window, 2-step training collapses. (b) One step more than it was trained for brings the 2-step student below its teacher (ring: trained steps). (c) The non-uniform grid of GRN-L helps only when the student is trained on it. (d) Of three judges, only oversampling brings the 2-step student to the teacher.

Studentt1t_1Diversity draftDiversity finalLast stepKeptChance
GRN-B (WW)
Base0.50.1100.6010.5960.8080.251
+2k on t1=0.25t_1=0.250.250.1550.4810.8090.7620.288
+2k on t1=0.67t_1=0.670.670.1180.6710.4490.8260.253
Base sampled at t1=0.25t_1=0.250.250.1100.3980.8330.6690.224
Base sampled at t1=0.67t_1=0.670.670.1100.7110.4200.8700.262
GRN-B (W+OSW+OS)
Base0.50.1300.5880.6150.7700.250
+2k on t1=0.25t_1=0.25 (NS)0.250.1720.4840.8150.7390.284
GRN-L (WW)
Base0.50.0840.5000.6390.7210.205
+2k on t1=0.25t_1=0.25 (NS)0.250.1290.4130.8250.6980.245

Table S5. What the two steps do (ImageNet 2562256^2, 2 NFE; 8 classes ×\times 64 noises). Notes: § 10.

Oversampling. Matching is only as good as its judge. Midway through training, a 2-step student has nearly matched it. The fake and real posteriors agree closely at every noise level (0.0013–0.0032 nats per token; Table S2), and the judge becomes the ceiling. Oversampling lifts this ceiling with the student’s own extra steps. Starting from a student and its real score, in one round we (i) oversample the KK-step student with N>KN > K steps, more than it was trained for, (ii) briefly fine-tune the current real score on these samples with GRN’s own CE objective (Eq. (7)), and (iii) continue matching against the new real score, with the fake score re-initialized from it.

qq0.050.10.20.30.40.60.8
DKL(DF∣DR)D_{\mathrm{KL}}(D_{\mathrm{F}}\mathbin{|}D_{\mathrm{R}})0.00190.00130.00260.00320.00310.00300.0028
DKL(DF∣Dθ)D_{\mathrm{KL}}(D_{\mathrm{F}}\mathbin{|}D_{\theta})0.6730.6890.7110.6310.5510.3980.245

Table S2. The judge is the ceiling. Per-token KL (nats) between posteriors at corrupted student samples, by keep probability qq (6k-iteration 2-step GRN-B student, 256 samples). Notes: Sec. S4.

This way, the student learns to do in KK steps what used to take NN (Sec. S1.3 gives the fixed point). We run one round (Sec. S2) with NN set to the student’s best number of steps, which is N=3N = 3 for K=2K = 2.

Non-uniform sampling. Only the last step receives gradient (§ 4.2). So non-uniform sampling gives the trained step a larger share of the path. We move a 2-step student, the base student, from the grid {0,0.5}\{0, 0.5\} to {0,0.25}\{0, 0.25\} and briefly continue training and sampling on it. At equal NFE, the trained step then covers 75% of the path, up from half (Figure 2). The grid has to be learned. Sampled on {0,0.25}\{0, 0.25\} without training, the base student gets worse, and t1=0.33t_1 = 0.33 works as well as 0.25 on both models (Figure 4c; Table S3 and Table S4). Trained this way, the last step adds what makes each sample individual, and the 2-step student draws the composition of the 4-step student for the same noise (Figure S3). A smaller t1t_1 lets the last step decide more of the image and buys recall at the cost of precision, monotonically over six trained values (Table S4).

(a) 2-step GRN-L students (FID50k_{50k})
Student (grid point t1t_1)FID50k_{50k}IS50k_{50k}PT_{\mathrm{T}}RT_{\mathrm{T}}FID-T
Base (10k W) on t1=0.5t_1=0.53.648310.40.73940.66102.061
+2k on t1=0.5t_1=0.53.834313.70.74590.65232.254
+2k on t1=0.25t_1=0.25 (NS) (W+NS)3.092296.40.71580.70971.594
NS sampled at t1=0.5t_1=0.53.243289.20.71420.6987–
(b) Grid ablations (FID-10k unless noted; Δ\Delta against the reference)
Student (grid point t1t_1)FID-10kΔ\DeltaFID50k_{50k}
+2k on t1=0.5t_1=0.5 (reference)6.26603.834
+2k on t1=0.33t_1=0.335.599−0.6663.136
+2k on t1=0.25t_1=0.25 (NS)5.512−0.7533.092
+2k on t1=0.17t_1=0.175.780−0.4863.333
Base sampled at t1=0.25t_1=0.25 (untrained)*6.541+0.4754.191
One student for K=2K=2 or 4: 2 steps (vs. uniform grids 5.945)5.731−0.213
4 steps (vs. uniform grids 5.527)5.285−0.242

Table S3. Non-uniform sampling: full results (ImageNet 2562^{2}, GRN-L, 2 steps). Notes: Sec. S4.

Figure S3. At 2 NFE, the non-uniform-sampling student takes the composition of the 4-step student (GRN-L, 2562256^2, FID-50k; same class and noise per column). Middle rows: 2k more iterations on {0,0.5}\{0, 0.5\} and {0,0.25}\{0, 0.25\}.

2-step W+OSW+OS student (FID50k\mathrm{FID}_{50k})
Student (grid point t1t_1)FID50k\mathrm{FID}_{50k}IS50k\mathrm{IS}_{50k}PTP_TRTR_T
Base (W+OSW+OS) on t1=0.5t_1=0.53.826306.10.7120.708
+2k on t1=0.5t_1=0.53.865307.00.7070.708
+2k on t1=0.25t_1=0.25 (NS) (W+OS+NSW+OS+NS)3.730298.60.6900.719
same student sampled at t1=0.5t_1=0.53.637297.30.7040.710
+2k on t1=0.33t_1=0.333.708301.70.6990.718
2-step WW student by grid point t1t_1 (FID50k\mathrm{FID}_{50k})
Student (grid point t1t_1)FID50k\mathrm{FID}_{50k}IS50k\mathrm{IS}_{50k}PTP_TRTR_T
Base (WW) on t1=0.5t_1=0.54.163278.70.6870.722
+2k on t1=0.25t_1=0.254.378264.70.6730.740
+2k on t1=0.33t_1=0.334.076275.90.6870.730
+2k on t1=0.4t_1=0.44.089277.60.6890.726
+2k on t1=0.5t_1=0.54.103280.70.6960.719
+2k on t1=0.6t_1=0.64.036286.10.6990.718
+2k on t1=0.67t_1=0.674.044287.60.7040.716
Base sampled at t1=0.25t_1=0.254.634269.90.6640.729
Base sampled at t1=0.33t_1=0.334.260280.30.6820.727
Base sampled at t1=0.67t_1=0.674.701259.10.6780.721

Table S4. Non-uniform sampling on GRN-B (ImageNet 2562256^2, 2 steps). Notes: § 10.

Refinement confidence keeping

GRN’s sampler keeps each predicted token with probability equal to the next grid time, so even a token that the network is sure of is often thrown back to noise and must be found again. Refinement confidence keeping also holds on to every token whose top predicted probability clears a threshold. It needs no extra network call, and the next step still repredicts all tokens. With GRN’s sampling setting, 16 steps of the GRN-B teacher with keeping beat 25 steps without it, at two thirds of the NFE (Table S6). Used in every rollout of the non-uniform-sampling fine-tune and at sampling time, keeping helps the GRN-L student as well (Table 1).

MethodStepsNFEms↓FID↓× T.↓IS↑
GRN-B (130M)
Teacher50798353.73–305.9
GRN-B812–8.132.18189.8
Fine-tuned GRN88–14.03.76151.0
Paired-noise GRN226658.215.6047.2
Matching without window22665.391.45244.4
FastGRN (W+RS)44943.951.06310.5
FastGRN (W)22664.171.12277.6
FastGRN (W+OS)22663.831.03306.8
FastGRN (W+OS+NS)22663.731.00298.6
FastGRN (W+OS+NS)33–3.420.92318.8
GRN-L (458M)
Teacher507912922.65–319.8
FastGRN (W)441113.031.15302.7
FastGRN (W+NS)22753.091.17296.4
FastGRN (W+NS+RCK)22753.031.15297.7
FastGRN (W+NS+RCK+OS)22752.851.08313.9
FastGRN (W+NS+RCK)33–2.570.97316.4

Table 1. Class-conditional ImageNet 2562256^2. ms: time per image on one RTX 4090; FID and IS on 50k samples; × T.: FID relative to the teacher. More rows: Tab. S11. Notes: Sec. S2.

Sampling ruleExtra 25 stepsExtra 16 steps50 steps
(a) GRN-B teacher (FID-10k)
Without keeping0%9.590%18.296.17
Refinement confidence keeping12%6.639%8.33–
Freezing14%8.1310%13.20–
Bit keeping9%6.675%11.15–
Bit freezing12%10.997%9.84–
Top-confidence–146.39––91.37
same noise on a second L40S
Refinement confidence keeping12%6.729%8.35–
Random (same share per step)12%6.659%8.67–
Random (same total share)12%7.009%10.78–
(b) 2-step FastGRN-B (W+OS+NS)
Sampling ruleExtraFID50k_{50k}IS50k_{50k}PT_{\mathrm{T}}RT_{\mathrm{T}}
Without keeping0%3.730298.60.6900.719
Keeping (a=0.64a=0.64)11%3.713297.90.6900.718
Keeping (a=0.51a=0.51)29%3.815292.40.6880.721
Random (same share)11%3.780300.30.7010.713
Trained with keeping29%3.732298.90.6910.722
sampled without keeping0%3.711300.40.6930.720
(c) 2-step FastGRN-L (W+NS)
Sampling ruleExtraFID50k_{50k}IS50k_{50k}PT_{\mathrm{T}}RT_{\mathrm{T}}
Without keeping0%3.091297.20.7120.709
Keeping (a=0.64a=0.64)5%3.082297.70.7160.707
Keeping (a=0.51a=0.51)17%3.107294.40.7120.713
Random (same share)5%3.135302.30.7220.703
Trained with keeping18%3.031297.70.7140.716
sampled without keeping0%3.084299.90.7140.710

Table S6. Refinement confidence keeping on ImageNet 2562256^2: the GRN-B teacher and the two 2-step students. Notes: Sec. S4.

Why it works. Keeping adds to GRN’s random choice and leaves it intact, so each input still holds a uniformly random set of clean tokens, as in training. Ranking tokens by confidence instead, as GRN tried [29], gathers them in easy regions, an input the network never sees in training, and breaks sampling. The threshold also sets when tokens are kept, almost none early and most late. This timing, which the threshold finds by itself, carries most of the gain on the teacher, and random keeping in the same share per step does about as well (Table S6). With two steps, confidence also picks better tokens than chance, and training with keeping lifts the recall of FastGRN-L (Table S11).

MethodStepsNFEFID↓×T.↓IS↑P_TR_Tms↓
GRN-B (130M)
Teacher50793.73–305.90.7330.730835
GRN-B16253.370.90287.20.6900.719–
GRN-B14223.510.94282.50.6810.720–
GRN-B12193.600.96278.00.6680.702–
GRN-B10154.741.27228.50.5930.711–
GRN-B8128.132.18189.80.5400.687–
Fine-tuned GRN8814.03.76151.00.5470.704–
Paired-noise GRN4424.16.47114.90.4720.70294
Paired-noise GRN2258.215.6047.20.3490.66266
Matching without window225.391.45244.40.6700.70666
FastGRN (W+RS)443.951.06310.50.7350.69894
FastGRN (W+RS+OS)444.021.08301.30.7300.70194
FastGRN (W)443.991.07277.40.7020.72694
FastGRN (W+OS+NS)223.731.00298.60.6900.71966
FastGRN (W+OS)223.831.03306.80.7130.70766
FastGRN (W)224.171.12277.60.6870.72266
FastGRN (W+OS+NS)443.420.92317.30.6830.67194
FastGRN (W+OS+NS)333.420.92318.80.7040.689–
GRN-L (458M)
Teacher50792.65–319.80.7280.7291292
FastGRN (W)443.031.15302.70.7340.697111
FastGRN (W+NS+RCK)223.031.15297.70.7140.71675
FastGRN (W+NS)223.091.17296.40.7160.71075
FastGRN (W+NS+RCK+OS)222.851.08313.90.7240.70675
FastGRN (W+NS+RCK)332.570.97316.40.7230.703–
FastGRN (W+NS+RCK)332.701.02316.60.7330.694–
FastGRN (W+NS+RCK)332.941.11329.90.7470.685–

Table S11. Class-conditional ImageNet 2562256^{2}, all rows and columns (Tab. 1 shows a subset). FID and IS on 50k samples. ×T.: FID relative to the method’s own teacher; P_T / R_T: precision / recall against 50k teacher samples; ms: batch-1 latency on an RTX 4090, decoding included. Notes: Sec. S4.

Experiments

Setup

ImageNet. We distill the released GRN-B (130M) and GRN-L (458M) on ImageNet [16] 2562256^2 from teacher samples only. The teachers use the setting of the GRN paper, with 50 steps and CFG on t∈(0.4,1)t \in(0.4, 1) for 79 NFE. Reevaluated with our pipeline, they score an FID-50k of 3.73 (GRN-B) and 2.65 (GRN-L), compared with 3.56 and 2.64 in the GRN paper, and all comparisons use our numbers. We fine-tune GRN on teacher samples with GRN’s corruption, which bakes the guidance into the weights. This fine-tuned GRN serves as the real score and initializes the fake score. The paired-noise GRN initializes the student. Windowed matching uses qmax⁡=0.4q_{\max} = 0.4 and τ=0.8\tau= 0.8 on both models, and Sec. S2 gives all recipes. We report FID-50k [32] and IS [75] with the teacher’s protocol, together with precision and recall [41] against 50k teacher samples. Studies use FID-10k.

Text-to-image. We use the public GRN-T2I 2B checkpoint, which works on bits at 2562256^2 and samples with 50 steps and CFG at every step for 100 NFE. With one teacher sample per prompt, we train LoRA [36] adapters on the frozen base for the fine-tuned GRN, the paired-noise GRN, the student and its fake score (Sec. S2). We report GenEval [27], FID-T, HPSv2.1 [94] and LPIPS [106] diversity, where FID-T is the FID to teacher images for the same COCO [46] captions.

Component studies. We use GRN-B, and GRN-L too for non-uniform sampling and refinement confidence keeping. We compare the oversampling judge with the original judge, the fine-tuned GRN, and with a teacher-data judge, which

is the original judge fine-tuned like the oversampling judge on the teacher samples it was trained on.

Main results

ImageNet. We give the main results in Table 1. With GRN-B, two refinement steps match the teacher’s FID-50k of 3.73 with 40× fewer NFE. Every component lowers FID, and none adds a network call at sampling time. With 3 steps, the same student goes below the teacher. GRN-L shows the same trend. Non-uniform sampling and refinement confidence keeping close the gap between the 2-step and the 4-step student, oversampling lowers FID further, and with 3 steps the student goes below its teacher. The 3-step students use 3 NFE, 26 times fewer than their teachers. Including decoding, an image takes tens of milliseconds on an RTX 4090, compared with about a second for the teacher (Table S15). The other few-step routes from the same teacher fall far behind (Table 1). Even with a tuned temperature, the teacher’s own sampler needs 12 steps, 19 NFE, to reach this level (Table S11).

ModelSamplerNFELarge batchBatch 1 (4090)
ms/imgNet+Dec.msGain
GRN-BTeacher7934.91.0×1.0×8351.0×
FastGRN41.8918.4×2.5×948.9×
FastGRN20.9735.8×2.6×6612.7×
GRN-LTeacher79107.31.0×1.0×12921.0×
FastGRN45.6219.1×5.3×11111.7×
FastGRN22.8437.7×6.0×7517.2×
GRN-T2ITeacher1001424s1.0×1.0×––
FastGRN458.8s24×17×––
FastGRN229.1s49×27×––

Table S15. Sampling cost per 2562256^2 image. Large-batch throughput and batch-1 latency. Notes: § 10.

Other few-step students. In Table 2 we compare FastGRN with other few-step students of discrete generators. Without a GAN loss, only FastGRN-B matches its teacher’s FID with two network calls, and Di4C needs four guided steps. Soft-Di[M]O goes below its teachers with a GAN loss on real images. FastGRN also has the lowest FID among the students of MaskGIT and VAR teachers. Table S12 adds more students, native samplers and continuous generators.

MethodBaseParamsNFEFID↓\downarrow×\timesT.↓\downarrowIS↑\uparrow
Further discrete students
SDTT [18][102]MaskGIT174†174^\dagger1c1^c90.40–14
ReDi [102]MaskGIT174†174^\dagger1c1^c37.43–49
ReDi [102] (2 rounds)MaskGIT174†174^\dagger1c1^c21.80–90
ReDi [102] (2 rounds)MaskGIT174†174^\dagger4c4^c7.86–240
ReDi-distill [102]MaskGIT174†174^\dagger1c1^c11.68–182
Di4C [30]MaskGIT174†174^\dagger4c4^c6.791.03209.2
DD1 [49]VAR-d1632719.942.37193.6
DD1 [49]VAR-d2063519.552.85197.2
DD1 [49]VAR-d2063527.332.19204.5
DD1 [49]VAR-d24109018.923.55202.8
DD1 [49]VAR-d24109026.952.77222.5
DD1 [49]LlamaGen-B98.3115.502.86135.4
DD1 [49]LlamaGen-B98.3211.172.06154.8
DD1 [49]LlamaGen-L326111.352.76193.6
DD1 [49]LlamaGen-L32627.581.84237.5
DD2 [48]VAR-d1632916.211.50213.0
DD2 [48]VAR-d24104015.061.77254.7
DD2 [48] (longer)VAR-d24104014.911.72282.2
DD2 [48]LlamaGen-L33518.592.09229.1
DD2 [48] (longer)LlamaGen-L33517.581.84238.7
Native discrete samplers
MaskGIT-PyTorch [6]–1742c2^c91.35–13.4
MaskGIT-PyTorch [6]–1744c4^c10.73–192.3
MaskGIT-PyTorch [6]–1748c8^c6.66–221.6
MaskGIT-PyTorch [6]–17416c16^c6.60–224.1
MaskGIT [10]–22786.18–182.1
Token-Critic [43]––364.69–174.5
AutoNAT-S [64]–464c4^c4.30–249.7
AdaNAT-S [65]–584c4^c4.54––
MaskGIL-B [97]–1118c8^c5.64–230.0
MaskGIL-L [97]–3438c8^c4.01–281.1
ReMDM [89]MaskGIT–167.40–145.3
DDPD [51]TiTok-S-128574165.71–201.7
Informed corr. [107]MD4400175.78–212.5
ReCAP [54]MaskGIT-r194165.02–166.6
TiTok-B-64 [103]–17783.08–192.5
VAR-d16 [84] (−2-2 scales)–310840.09–56.8
VAR-d16 [84] (−1-1 scale)–31099.52–178.9
Continuous latents
MeanFlow-B/2 [25]–13116.17––
MeanFlow-L/2 [25]–45913.84––
α\alpha-Flow-B/2 [104]–13125.01––
iMF-B/2 [26]–8913.39–255.3
iMF-L/2 [26]–40911.86–276.6
Drifting-L/2 [17]–46311.54–258.9
FreeFlow-XL/2 [86]–67811.690.82–
SoFlow-B/2 [59]–13114.85––
DMF-B/2 [42]–13015.63––
IMM-XL/2 [109]–67527.77––
Shortcut-XL/2 [21]–676110.60––
FastGRN-BGRN-B13023.731.00298.6
FastGRN-LGRN-L45822.851.08313.9

Table S12. More few-step generators on ImageNet 2562256^2, as reported by each paper (×\timesT. as in Tab. 2; parameters in millions). Notes: Sec. S4.

Text-to-image. On text-to-image, the teacher’s own sampler falls apart with few steps. Its GenEval drops at 8 steps

and collapses at 4 (Figure 5). With the window alone, the 4-step FastGRN student nearly keeps the teacher’s GenEval (0.689 vs. 0.695), a level that the teacher’s own sampler approaches only with 8× the NFE. The student is as close to the teacher as the teacher’s own resamples in FID-T and keeps most of its diversity (Table S10). In Table 3 we compare it with the few-step students of other discrete generators. With two steps, FastGRN keeps 97% of the teacher’s GenEval and already scores above all of them, including one that adds GAN and reward losses and test-time search. Only the reward-tuned students keep more of their teacher’s score.

Figure 5. Text-to-image with few steps (GRN-T2I 2B, 2562256^2). GRN-T2I (NN steps): the teacher’s own sampler with CFG at every step. Show-o Turbo as reported. Dashed: the teacher with 50 steps, in (b) against its own resamples. Notes: Sec. S2.

Methodqmax⁡q_{\max}Iter.NFEOverall↑\uparrowSingleTwoCountColorPos.Attr.Δ\Delta vs. T.FID-T↓\downarrowCLIP↑\uparrowHPS↑\uparrowDiv.↑\uparrow
GRN-T2I [29] (102421024^2)∗^*––1000.760.990.900.720.840.520.60–––––
Teacher (50 steps at 2562256^2)––1000.6950.980.830.710.790.330.53–36.6a36.6^a26.5828.191.00
GRN-T2I (16 steps)––320.6870.980.820.690.790.310.53−0.00834.426.6127.42–
GRN-T2I (8 steps)––160.5630.930.630.520.690.240.37−0.13254.526.1825.07–
GRN-T2I (4 steps)––80.2090.540.110.140.350.060.05–141.322.5119.41–
GRN-T2I (2 steps)––40.0550.200.020.010.100.000.00–208.417.5912.44–
Fine-tuned GRN-T2I (no CFG)––500.6330.940.770.600.730.290.46−0.06236.526.2227.11–
Paired-noise GRN-T2I (2 steps)––20.0440.160.000.020.090.000.00–209.217.9515.10–
FastGRN (W) (4 steps)0.402k40.6890.990.830.680.790.290.56−0.00636.026.5527.280.94
FastGRN (W) (4 steps)0.551k40.6840.990.830.660.770.300.55−0.01135.626.5227.320.95
FastGRN (W) (2 steps)0.404k20.6520.960.780.660.740.270.50−0.04340.326.5826.500.90
FastGRN (W) (2 steps)0.552k20.6610.970.780.670.740.310.50−0.03439.126.5926.580.91
FastGRN (W) (2 steps)0.55→\to0.952k+1k20.6750.980.800.680.780.290.52−0.02036.7–––

Table S10.

Qualitative comparison. In Figure 3 (left) we put FastGRN next to the released few-step students of SDXL and Show-o. They part ways when a prompt binds attributes or relations. SDXL-Turbo and DMD2 paint the black apple purple, Show-o Turbo draws one clock where the prompt asks for two, and SDXL-Turbo and Show-o Turbo each draw a single animal for a dog right of a teddy bear. FastGRN follows every prompt. On ImageNet (right), the teacher’s own sampler with 8 steps loses fine structure and turns textures blotchy, and the 1-step ReDi [102] oversaturates colors and smears shapes. FastGRN restores sharp texture with 2 steps (Figure S10). Oversampling keeps each sample’s composition and sharpens local detail (Figure S8), while non-uniform sampling changes the composition itself, so the 2-step student draws the layout of the 4-step one (Figure S3).

Figure 3. Few-step generators side by side. Left: few-step text-to-image models as released [77, 98, 100] and FastGRN on GenEval prompts (a bench and a sports ball, two clocks, a blue fire hydrant, a dog right of a teddy bear, a purple wine glass and a black apple), all shown at 2562256^{2}. Right: ReDi [102] as released, and the GRN-B rows with one frozen noise per column.

Figure S8. The oversampling judge keeps each sample’s composition, changing local detail (GRN-B, 2 steps, 2562256^{2}, FID-10k; same class and noise per column). “+” rows: 3k more iterations with that judge.

Figure S10. Text-to-image, same noise (GRN-T2I 2B; prompts 1–12 of the fixed 24, one frozen noise per column). Fine-tuned GRN: fine-tuned on teacher samples, 50 steps without CFG; paired-noise GRN: fine-tuned on noise–sample pairs; without window: matching without the window.

Component study

Window and random steps (§ 4.2). Without the window, the 2-step student looks healthy early on, then its samples turn blank and FID explodes. With the window, training stays stable (Figure 4a). The window also helps where nothing collapses, such as on the 4-step student (Table S7). A fixed-step student is a specialist and drifts with more steps. With steps drawn at random from 2–4, the curve becomes flat from 4 to 8 steps, and at 4 steps this student beats the fixed 4-step student in half the iterations. With 3–6, it keeps improving and reaches the teacher’s FID at 8 (Figure S1).

SettingFID↓Best (it.)Prec.Rec.Training
(a) Initialization × objective
Fine-tuned init. without window358.317.69 (8k)0.0010.045collapse 7.9k
Paired-noise init. without window332.87.87 (4k)0.0200.442collapse 6.4k
Fine-tuned init. with window9.859.85 (10k)0.6320.758stable
Paired-noise init. with window6.656.59 (6k)0.6920.783stable
(b) Objective variants (FID-50k)
No window (4 steps)4.63 (FID-10k 6.97)stable
Window (4 steps)3.99 (FID-10k 6.42)stable
Window (2 steps)4.17 (FID-10k 6.65)stable
single q=0.4q=0.4 (2 steps)4.15 (FID-10k 6.56)stable
No window + anchor λ=10−3\lambda=10^{-3}–collapse 3.4k
No window + anchor λ=5⋅10−4\lambda=5\cdot10^{-4}–collapse 4.2k

Table S7. Ablations on ImageNet 2562256^2 (GRN-B). (a): 2-step students, FID-10k at 10k iterations; (b): FID-50k. Notes: Sec. S4.

One more step. Only the last step is trained, so a 2-step student still has refinement left. A third sampling step brings it below its teacher’s FID, and a fourth adds little (Figure 4b). Oversampling turns this unused step into a better judge, and the 3-step rows of Table 1 use the extra step to beat both teachers (GRN-L after 2k more iterations with a decaying learning rate).

Oversampling and non-uniform sampling (§ 4.3). The oversampling judge works because it has seen the student’s own refined samples. One round brings the 2-step student to the teacher’s level with 2 NFE, compared with 79 for the teacher (Figure 4d). The original judge and the teacher-data judge stop well short. On GRN-L, a round on the 2-step student with keeping, whose judge is fine-tuned on the student’s own 3-step samples, lowers FID further (Table 1).

On GRN-B, a short fine-tune on {0,0.25}\{0, 0.25\} improves the W+OS student, while the same number of iterations on {0,0.5}\{0, 0.5\} does not, and on the standard grid this student even drops below the teacher’s FID (Table S4). GRN-L behaves the same way (Table S3). Sampled on the non-uniform grid without the training, the student gets worse, so what matters is training on that grid, and t1t_1 near 0.25 works best (Figure 4c).

Refinement confidence keeping (§ 4.4). Keeping pays off once the student learns with it. Without training, the students’ lower threshold hurts. Trained with keeping in every rollout, the 2-step FastGRN-L reaches the FID of our 4-step student at 2 NFE, mostly through recall, while FastGRN-B stays at its teacher’s FID (Table S6).

Conclusion

Few-step GRNs fail because of a joint error between tokens, and distribution matching on the student’s own samples reduces this error. For uniform-state tokens, the gradient of distribution matching needs no sampled-token gradient estimator, because it is computed directly from two posterior networks. FastGRN builds few-step GRNs on this gradient in three parts. A window keeps matching stable, and random steps train one student for 4 to 8 steps. Oversampling and non-uniform sampling turn the student’s own extra steps into a better judge and lengthen the trained step. Refinement confidence keeping holds on to the tokens the student is already sure of. A 2-step GRN-B student matches the FID of its 79-NFE teacher, and with one more sampling step the students of both GRN-B and GRN-L go below their teachers. The same parts apply to any generator trained with uniform-state corruption, and bit-level text-to-image and video models are the natural next step.

Proofs

Notation follows the main paper. kqk_q is the uniform-state kernel of Eq. (1) at keep probability qq, and ν=(1−q)/V\nu= (1-q)/V. The concrete score s(z,i,a)s(z,i,a) of a law is its probability at zz with token ii set to aa, divided by its probability at zz. The KK-step student GθG_\theta has law pθp_\theta, and pθ,qp_{\theta,q} is that law corrupted by kqk_q. RR and FψF_\psi are the real and fake scores with posterior marginals DRD_R and DFD_F.

Why few steps fail

Lemma S1 (Redundancy of per-token interval maps). If the corruptions at two noise levels are coupled token by token given the clean image, as in uniform-state diffusion, the CE-optimal per-token interval predictor, which maps a token’s state at a noisier time to its state at a cleaner one as MeanFlow [25] does for continuous data, is a fixed function of the posterior marginals that GRN already outputs. Sampling with it is ancestral sampling with GRN’s own posterior.

Proof. Given the clean image, the two states of a token depend only on its clean value, so the next state is the posterior marginal passed through a known per-token bridge, which a CE-trained predictor recovers.

Proposition S1 (Few-step error is a joint error). Among factorized predictors of the image given a state zz, the expected CE is minimized by the product of the posterior marginals, and the remaining error is

EzDKL(p(⋅∣z)∥∏ip(xi∣z))=EzTC⁡(x∣z),(3)\mathbb{E}_z D_{\mathrm{KL}}\left(p(\cdot\mid z)\middle\|\prod_i p(x^i \mid z)\right)=\mathbb{E}_z \operatorname{TC}(x\mid z), \tag*{(3)}

the conditional total correlation [91].

Proof. The CE of a factorized predictor splits over tokens, and each term is smallest at the true marginal (Gibbs’ inequality). The KL from a joint law to the product of its marginals is the sum of the marginal entropies minus the joint entropy. From pure noise the sampler outputs the product of the data marginals, at distance TC⁡(x)\operatorname{TC}(x) from the data.

Corollary S1 (Informative noise). If training pairs of noise and image are coupled so that the images follow the data law, the KL from the data to the CE-optimal one-step sampler is at most the average conditional total correlation given the noise. It is zero when the noise determines the image, and TC⁡(x)\operatorname{TC}(x) for noise independent of the image.

Proof. Both laws are averages over the noise, of the conditional law and of the product of its marginals. Joint convexity of the KL and Prop. S1 give the bound.

Paired-noise fine-tuning realizes this coupling. The student’s last step is factorized only given the noise and the earlier states, so its law is a mixture of product laws that distribution matching can shape.

The matching gradient

Corruption acts token by token, so the concrete score of the corrupted law averages the kernel ratio kq(a∣b)/kq(zi∣b)k_q(a\mid b)/k_q(z^i\mid b) over the posterior of the clean value bb. With DiD^i the posterior marginal of token ii and a≠zia\ne z^i,

s(z,i,a)=1+qDi(a)ν−qDi(zi)q+ν.(4)s(z,i,a)=1+\frac{qD^i(a)}{\nu}-\frac{qD^i(z^i)}{q+\nu}. \tag*{(4)}

The windowed matching objective of § 4.2 is

L(θ)=Eq∼ρ[DKL(pθ,q∥pd,q)].(5)\mathcal{L}(\theta)=\mathbb{E}_{q\sim\rho}\left[D_{\mathrm{KL}}\left(p_{\theta,q}\middle\|p_{d,q}\right)\right]. \tag*{(5)}

Proof of Proposition 1. The total mass is fixed, so the change of the KL is the change of the corrupted student law weighted by the log density ratio. For K=1K=1, token ii of the corrupted student takes value aa with probability qπi(a)+νq\pi^i(a)+\nu, so a change of πi\pi^i only moves mass between states that differ in token ii. The log density ratio thus enters through its change when token ii is set to aa, which is Δai\Delta^i_a, the difference of the log concrete scores (Eq. (4)) computed with DFD_F and DRD_R. The softmax subtracts the student’s own average of Δi\Delta^i. For K>1K>1 we ignore how the input of the last step depends on θ\theta (a truncated gradient).

Training applies it through the surrogate ∑i,bsg⁡(gbi)lbi/τ\sum_{i,b}\operatorname{sg}(g^i_b)l^i_b/\tau, where gg is the bracket of Eq. (2) (Alg. S1).

Corollary S2 (Binary tokens). For bits (V=2V=2), Δi\Delta^i vanishes at the observed bit zz, and with c=(1+q2)2c=\left(\frac{1+q}{2}\right)^2 the gradient with respect to the logit difference di=u1i−u0id^i=u^i_1-u^i_0 is

∂DKL∂di=E[qπiπˉi(1−2z)log⁡c−qDFi(z)c−qDRi(z)].(6)\frac{\partial D_{\mathrm{KL}}}{\partial d^i}=\mathbb{E}\left[q\pi^i\bar{\pi}^i(1-2z)\log\frac{c-qD^i_F(z)}{c-qD^i_R(z)}\right]. \tag*{(6)}

Proof. With q+2ν=1q+2\nu=1, Eq. (4) gives the score of flipping the bit as (c−qDi(z))/(ν(q+ν))(c-qD^i(z))/(\nu(q+\nu)), and the two-logit softmax turns Δ\Delta into the gradient on did^i.

If the fake score explains the observed bit better than the real score, the student over-produces it and the update moves probability to the other value.

Remark S1 (Why reverse KL). For an ff-divergence the bracket of Eq. (2) becomes f′(cr)−f′(r)f'(cr)-f'(r), with rr the density ratio and cc the ratio of concrete scores that two posterior networks provide. It depends on cc alone for every rr iff f′(ex)f'(e^x) is affine (Cauchy’s functional equation), i.e. only for the reverse KL up to scale. Other divergences need absolute density ratios, e.g. from a discriminator, so the free design choice is the weighting ρ\rho, which the window changes.

Random steps and oversampling

Proposition S2 (Shared fake score). With random steps (Sec. 4.2) one fake score serves all K∈KK \in\mathcal{K}, so DFD_F is the posterior of the corrupted mixture pˉθ\bar{p}_\theta of the KK-step laws. Eq. (2) applied to KK-step samples and averaged over KK uniform on K\mathcal{K} is the last-step gradient of Eq. (S3) with pˉθ\bar{p}_\theta in place of pθp_\theta. A student whose every KK-step law is the data law is a global minimizer.

Proof. The change of the mixture KL is the average over KK of the changes of the KK-step laws, weighted by the log ratio of the mixture to the data, and for each KK the proof above applies with the concrete score of the mixture. The KL is zero iff the mixture equals the data law (the kernel is invertible for q>0q > 0), which holds if every KK-step law does.

Stratified draws make KK exactly uniform within every block of ∣K∣|\mathcal{K}| iterations (Sec. S2), so the CE optimum of the shared fake score is this posterior.

An oversampling round (Sec. 4.3) samples the KK-step student θr\theta_r with N>KN > K steps, briefly fine-tunes the real score on these samples with GRN’s objective,

Rr+1=FT⁡Rr[min⁡REx∼pθrNCE⁡(R;x)],(7)R_{r+1} = \operatorname{FT}_{R_r}\left[\min_R \mathbb{E}_{x\sim p_{\theta_r}^{N}}\operatorname{CE}(R;x)\right], \tag*{(7)}

with FT⁡Rr\operatorname{FT}_{R_r} a short fine-tune from RrR_r, and continues windowed matching against Rr+1R_{r+1} with the fake score re-initialized from it.

Remark S2 (Fixed point). With exact minimization, a round leaves (θ,R)(\theta, R) unchanged iff pθK=pR=pθNp_\theta^K = p_R = p_\theta^N, where pRp_R is the law whose posteriors RR gives. The judge learns the NN-step law, and since the kernel is invertible, matching then makes the KK-step law equal to it. As in expert iteration [2], the slow sampler is the expert and the fast one the apprentice, here taught through the judge.

The window keeps the minimizer. For q>0q > 0 the per-token kernel is invertible, so each KL term of Eq. (S3) vanishes only at the data law, and so does any average over q∈[qmin⁡,qmax⁡]q \in[q_{\min}, q_{\max}] with qmin⁡>0q_{\min} > 0.

Implementation details

Teachers and data. Times are wall-clock on A100, L40S and RTX 4090 GPUs. The teachers are the released GRN-B (130.9M parameters, 16-level tokens on a 16×16×1616 \times16 \times16 map) and GRN-L (458.5M), sampled as in the GRN paper with 50 steps, CFG on t∈(0.4,1)t \in(0.4, 1), scale 2.4 / 2.0 and temperature 1.33 / 1.30 for B / L (79 NFE), and uniform class labels. For GRN-B we draw 200k teacher samples for the fine-tuned GRN and 600k pairs for the paired-noise GRN (2.9 h each). For GRN-L, 200k samples serve both.

Fine-tuned and paired-noise GRN. Both start from the released EMA weights and train with GRN’s CE on all tokens, without label dropout, with t∼0.6sigmoid⁡(N(−0.8,0.8))+0.4U(0,0.995)t \sim0.6\operatorname{sigmoid}(\mathcal{N}(-0.8, 0.8)) + 0.4\mathcal{U}(0, 0.995), AdamW [55] (β=(0.9,0.95)\beta= (0.9, 0.95), no weight decay), learning rate 5⋅10−55 \cdot10^{-5} after 1k warm-up iterations, gradient clipping 1.0, bf16 autocast and batch 128. The fine-tuned GRN keeps GRN’s corruption with fresh noise and trains for 15k iterations. The paired-noise GRN fills dropped tokens with the sample’s own initial noise. On GRN-B it starts from the fine-tuned GRN and trains for 40k iterations, and on GRN-L for 15k.

Windowed matching on ImageNet. The student starts from the paired-noise GRN (EMA weights) and the fake score from the fine-tuned GRN, which is also the frozen real score. Student and fake updates (nfake=5n_{\mathrm{fake}} = 5) use batch 64 and learning rates 10−5/2⋅10−510^{-5} / 2 \cdot10^{-5}. We draw q∼0.5sigmoid⁡(N(−0.8,0.8))+0.5U(0,1)q \sim0.5\operatorname{sigmoid}(\mathcal{N}(-0.8, 0.8)) + 0.5\mathcal{U}(0, 1) clamped to [0.02,qmax⁡][0.02, q_{\max}] with qmax⁡=0.4q_{\max} = 0.4 (0.92 without the window), which puts 46% of the draws at qmax⁡q_{\max}. The fake score trains at all levels with the tt-law above. The student uses τ=0.8\tau= 0.8 and EMA 0.999, trains for 10k iterations and samples with GRN’s frozen-noise sampler on ts=s/Kt_s = s/K without CFG. A student takes up to 3.8 h (4 steps), and the whole GRN-B and GRN-L pipelines take about 15 and 19 GPU-hours.

Random steps. Iterations are grouped in blocks of n=∣K∣n = |\mathcal{K}|. Each block is a permutation of K\mathcal{K} from a counter-based random generator keyed by the block index, and iteration jj takes entry j mod nj \bmod n. The whole iteration, its five fake updates included, uses KjK_j-step rollouts (Alg. S1), and one fake score takes all updates. The random-step students and the fixed-step students they are compared with train for 6k iterations and are evaluated with 1–6 and 8 steps. The per-KK variant keeps one fake score for each KK and updates it only when KK is drawn (5/nn updates per student update). The variant with 15 fake updates restores five per fake score at 2.6 times the cost.

Oversampling. A round (i) draws 200k samples, and 5k more for validation, from the student’s EMA weights with NN steps and τ=0.8\tau= 0.8, (ii) fine-tunes the judge, the fine-tuned GRN (EMA 0.9995), on them for 4k iterations with GRN’s CE, batch 128 and learning rate 2⋅10−52 \cdot10^{-5} after 200 warm-up iterations, and takes its EMA 0.999 weights as Algorithm S1 One FastGRN training iteration (windowed matching with random steps)

Require: student GθG_\theta (init. paired-noise GRN), fake score FψF_\psi (init. fine-tuned GRN), frozen real score RR, step set KK (fixed KK: KK only), window [qmin⁡,qmax⁡][q_{\min},q_{\max}], iteration index jj

  1. K←σb(j mod ∣K∣)K \leftarrow\sigma_b(j \bmod|K|), σb\sigma_b a random permutation of KK, b=⌊j/∣K∣⌋b = \lfloor j/|K| \rfloor ▷ stratified

  2. for m=1,…,nfakem = 1,\ldots,n_{\mathrm{fake}} do ▷ fake-score updates

  3. x←Kx \leftarrow K-step rollout of GθG_\theta from fresh (ϵ,c)(\epsilon,c), no grad

  4. t∼t \sim GRN’s tt-law, z∼kt(⋅∣x)z \sim k_t(\cdot\mid x)

  5. step on CE⁡(Fψ(z,t,c),x)\operatorname{CE}(F_\psi(z,t,c),x)

  6. end for

  7. rollout from (ϵ,c)(\epsilon,c): steps 1…K−11\ldots K-1 no grad; last-step logits ll, x^∼softmax⁡(l/τ)\hat{x} \sim\operatorname{softmax}(l/\tau)

  8. q∼ρq \sim\rho clamped to [qmin⁡,qmax⁡][q_{\min},q_{\max}]; z′∼kq(⋅∣x^)z' \sim k_q(\cdot\mid\hat{x})

  9. DF←Fψ(z′,q,c)D_F \leftarrow F_\psi(z',q,c), DR←R(z′,q,c)D_R \leftarrow R(z',q,c) ▷ no grad

  10. g←qπ(Δ−∑aπaΔa)g \leftarrow q\mathop{\pi}(\Delta-\sum_a \pi_a\Delta_a) via Eqs. (4) and (2)

  11. step on ∑sg⁡(g)l/τ\sum\operatorname{sg}(g)l/\tau; update EMA of θ\theta

the new real score, and (iii) continues windowed matching for 3k iterations against it with the original recipe, the fake score re-initialized from the judge. The 2-step student uses N=3N = 3, its best step count. The original-judge continuation runs step (iii) alone against the unchanged judge, and the teacher-data judge runs steps (ii) and (iii) on the 200k teacher samples the judge was trained on. The same round is applied to the 6k K=3K=3–6 student (N=8N = 8), the fixed 4-step student (N=5N = 5) and the GRN-L K=3K=3–6 student (N=5N = 5). For the 2-step GRN-B student a round takes 1.39 GPU-hours on one RTX 4090, 42% of the student’s own training, and sampling and the judge fine-tune add only 12%.

Non-uniform sampling. Each run continues a 2-step student from its live weights for 2k iterations with the recipe of its model and the gradient on the last step only, and trains and samples on the same grid {0,t1}\{0,t_1\}, with t1=0.25t_1 = 0.25 for non-uniform sampling and 0.5 for the standard grid. With t1=0.25t_1 = 0.25 the trained last step covers 75% of the path. On GRN-L the base is the 10k W student, and the fine-tuned GRN-L (EMA 0.9995) is the real score and the fake initialization (2.2 h per run). On GRN-B the base is the W+OS student, with its oversampling judge as the real score and a fresh fake score (1.5 h). The W runs of Table S4 continue the 10k W student against the fine-tuned GRN.

Text-to-image. The teacher is the public GRN-T2I 2B checkpoint at 2562256^2 (16×16 positions × 256 bits = 65,536 binary tokens), frozen in bf16, with 50 steps, CFG 4.5 and τ=1.1\tau= 1.1 (100 NFE), and its HBQ decoder. The pseudo data are one teacher sample for each of 50k prompts from DenseFusion, COCO-2014 train and DiffusionDB, without the evaluation prompts. Every trained model is a rank-64 LoRA on the 7 linear layers of all 28 blocks (89.5M parameters) of one frozen base. The fine-tuned and the paired-noise GRN train for 5k iterations at batch 64. Matching follows the ImageNet setup (the guided variant uses the CFG-combined base as real score after 200 fake-only updates) with batch 32, learning rates 5⋅10−55\cdot10^{-5} / 10−410^{-4} after 200 warm-up iterations, τ=1\tau= 1 and EMA 0.999. One iteration takes 14 / 17.5 / 25 s for 1 / 2 / 4 steps. All text-to-image students use windowed matching without the other parts (W).

Evaluation. ImageNet FID and IS follow the teacher’s protocol (bf16 generation, fp32 HBQ decoding) and reference statistics. Precision and recall use k=3k = 3 Inception neighbours against 50k teacher samples drawn with separate noise. FID-10k, used in the studies, runs 2.3–2.9 above FID-50k. FID-T is the FID of a student’s samples to the 50k teacher samples. For text-to-image we report GenEval [27] (553 prompts, four images each, Mask2Former detector [14]) with paired differences to the teacher, which scores 0.695 at 2562256^2 (GRN’s 0.76 is measured at 102421024^2 with a prompt rewriter). Here FID-T is the FID to teacher images for 1k COCO [46] captions, where a re-sampled teacher scores 36.6. We also report CLIP-L/14 [69], HPSv2.1 [94] and same-prompt LPIPS [106] diversity relative to the teacher. Batch-1 latency is the median of 20–30 runs after warm-up with torch.compile on one RTX 4090 and includes the HBQ decoder (34 ms).

Notes to the main-paper tables and Fig. 5. Table 1. GRN-B, 8 steps: the teacher’s own sampler with fewer steps, at the temperature with the lowest FID-10k for 8 steps, 0.7 (GRN uses 1.33 with 50 steps; CFG on t∈(0.4,1)t \in(0.4,1) adds network calls); Fine-tuned and paired-noise GRN: GRN fine-tuned on teacher samples with independent and with paired noise; matching without window: distribution matching without the window, at its best checkpoint (4k iterations) before it collapses. Bold: best student FID. ms: batch-1 latency with decoding; the paired-noise GRN and matching without window share it with the 2-step students (same network), the other baselines are not timed. Tags: W paired-noise initialization and windowed distribution matching (10k iterations); RS random steps over K=2K=2–4 (6k); OS one oversampling round (+3k); NS non-uniform sampling, +2k iterations on {0,0.25}\{0,0.25\}; RCK refinement confidence keeping in every rollout of that fine-tune and when sampling. GRN-B with 3 steps: the 2-step W+OS+NS student sampled with 3 steps on {0,1/3,2/3}\{0,1/3,2/3\} (no training; 3-step latency not measured). GRN-L, W+NS+RCK+OS: one oversampling round on the 2-step W+NS+RCK student, with the judge fine-tuned on its own 3-step samples (+2k); GRN-L with 3 steps: the W+NS+RCK student after 2k more iterations with a learning rate decaying to 5% over the last 1k, sampled with 3 steps on {0,0.25,0.5}\{0, 0.25, 0.5\} without keeping (3-step latency not measured; Figure 1 places the 3-step points on the line through the measured 1-, 2- and 4-step latencies of the same network). All students use EMA 0.999.

Table 2. Numbers reported by the cited papers, each with its own tokenizer and its own teacher, so students are compared by ×\timesT. (bold: lowest without a GAN loss; teacher FID: MaskGIT 6.60, MaskBit 1.66, VAR-d16 4.15 for DD2 and 4.19 for DD1, VAR-d20 3.40, Di4C’s 8-step MaskGIT 6.57). g^{g}Trained with a GAN loss on real data. c^{c}Sampling steps as reported; these methods use classifier-free guidance at inference. †^{\dagger}Size not stated in the paper; these students start from the PyTorch MaskGIT of Besnier and Chen [6], whose size Di[M]O reports. SDTT: three rounds, as run by ReDi. DAG: four network evaluations, initialized from ReDi. FastGRN: rows of Table 1, the 2-step W+OS+NS (GRN-B) and W+NS+RCK+OS (GRN-L) students and the GRN-L student with 3 steps. More students of MaskGIT, VAR and LlamaGen teachers (SDTT, ReDi, Di4C, DD1, DD2), native discrete samplers and few-step generators in continuous latent spaces (with parameters and IS): Table S12.

Figure 5. All students are LoRA adapters trained with windowed matching alone (W): 4 steps with qmax⁡=0.40q_{\max}=0.40 for 2k iterations, and 2 steps with qmax⁡=0.55q_{\max}=0.55 for 2k iterations, then 1k at qmax⁡=0.95q_{\max}=0.95. FID-T: FID to 1k teacher images for the same captions, i.e. closeness to the teacher; the dashed line in (b) is the teacher against a second teacher sample set, i.e. the teacher’s own noise level. GRN-T2I (NN steps): the teacher’s own sampler with fewer steps (CFG at every step, 2N2N NFE). Fine-tuned GRN-T2I: GRN-T2I fine-tuned on teacher samples, 50 steps without CFG. All values, HPS, sub-task scores, CLIP, the paired GenEval difference to the teacher, diversity and more rows: Table S10. Show-o Turbo: the 4-step student of the discrete Show-o, GenEval as reported (at 2562256^2, without CFG). All reported students, continuous ones included: Table S13.

MethodTeacherParamsStepsGenEval↑\uparrow
Continuous
InstaFlow [53]–0.9B10.33
SANA-Sprintg^g [13]–0.6B10.72
SANA-Sprintgg [13]–0.6B20.76
SANA-Sprintgg [13]–0.6B40.76
SANA-Sprintgg [13]–1.6B10.76
SANA-Sprintgg [13]–1.6B20.77
SANA-Sprintgg [13]–1.6B40.77
FLUX.1-schnellg^g [7]–12B10.69
FLUX.1-schnellg^g [7]–12B20.71
FLUX.1-schnellg^g [7]–12B40.71
TDM [61]–0.9B40.50
LCM [58]SDXL2.6B10.27
LCM [58]SDXL2.6B40.51
SDXL-Turbo [77]–2.6B10.55
DMD2 [100]SDXL2.6B10.55
Discrete tokens
Di[M]O [110]Meissonic1.0B10.43
Di[M]O [110]MaskGen-L0.6B10.42
Soft-Di[M]Or^r [111]Meissonic1.0B10.53
Soft-Di[M]Or^r [111]MaskGen-L0.6B10.51
Soft-Di[M]Osr^r_s [111]MaskGen-L0.6B10.63
FPD [88]MaskGen-L0.6B10.45
Show-o Turbo [98] (2562256^2)Show-o1.3B20.494
Show-o Turbo [98] (2562256^2)Show-o1.3B40.523
Show-o Turbo [98] (2562256^2)Show-o1.3B80.552
Show-o Turbo [98] (5122512^2)Show-o1.3B20.557
Show-o Turbo [98] (5122512^2)Show-o1.3B40.625
Show-o Turbo [98] (5122512^2)Show-o1.3B80.638
FastGRN (W)GRN-T2I2B40.689
FastGRN (W)GRN-T2I2B20.675

Table S13. Few-step text-to-image students, GenEval overall as reported by each paper. Notes: Sec. S4.

Table 3. GenEval overall as reported by the cited papers; each student is divided by its own teacher in the same paper (MaskGen-L with 16 steps 0.48, as run by Soft-Di[M]O and FPD; Show-o with 16 steps and CFG 0.591 at 2562256^2 and 0.674 at 5122512^2; GRN-T2I with 50 steps and CFG 0.695). Bold: highest GenEval, and highest ×\timesT. without a GAN or reward loss. g^{g}GAN loss; r^{r}reward fine-tuning; s^{s}TTS: test-time search guided by reward models, with many network calls per image. Show-o Turbo and FastGRN sample without CFG, so steps equal network calls. FastGRN: the 2- and 4-step students of Figure 5. Students of Meissonic, Show-o Turbo with 8 steps and continuous few-step generators: Table S13.

Additional results

Random steps and oversampling

Random steps. Figure S1 compares the random-step students of § 4.2 with fixed-step students. Random steps trade the sharp optimum of a specialist for a flat curve. At 4 steps the K=2K=2–4 student beats the fixed 4-step student (6.32 vs. 6.46), reaches its final FID in half the iterations and has higher precision against teacher samples (0.737 vs. 0.707). The K=3K=3–6 student keeps improving from 3 to 8 steps (8.16 to 6.24). With one fake score per KK, a K=2K=2–4 student peaks at 6.59 after 2k iterations and then falls apart (11.21 at 6k). With 50k samples and 4 steps, the K=2K=2–4 student scores FID 3.95 and IS 310.5 (Table S11), and the fixed 4-step student trained as long (6k iterations) scores 4.01 and 278.4. Random steps also keep the image stable as the step count changes (Figure S9). On GRN-L the K=3K=3–6 student keeps FID-10k between 5.49 and 5.66 from 4 to 8 steps, while the fixed 4-step student climbs from 5.43 to 6.46.

Figure S9. At 2 steps the fixed 4-step student smears textures and distorts shapes, while the random-step student stays coherent (GRN-B, 2562256^{2}; KK: training steps; same class and noise per column).

Oversampling. The oversampling judge acts fast (Table S1). After 1k iterations the 2-step student is at 6.25, against 6.50 with the original judge. In effect, the round moves the third step into the first two. The starting student needed three steps to beat the teacher (6.04 against 6.24), and after the round two steps reach its level. With 50k samples the student reaches FID 3.83 and IS 306.8, against 4.12 and 280.4 with the original judge. The judge itself barely moves. Its validation CE on held-out student samples falls from 1.5755 to 1.5733 nats per token, and a judge fine-tuned the same way on teacher data (1.5870 →\to 1.5871) does no better than the unchanged one. What helps is a judge that has seen the student’s own refined samples (Figure S8).

2-step student continued withN=2N=2 at iteration3k3k at N=2N=23k at N=2N=2 (50k)
1k2k3kN=3N=3N=4N=4PTP_{\mathrm{T}}RTR_{\mathrm{T}}FID-TISFIDIS
Before (10k student) (W)6.65 (start)6.046.230.6920.7834.11–4.17277.6
Original judge (W)6.506.516.506.19–0.7000.7843.97216.64.12280.4
Teacher-data judge (W)6.516.586.586.45–0.7040.7833.92224.3––
Oversampling judge (W+OS)6.256.226.256.22–0.7050.7733.92233.9––
Oversampling judge (student of Tab. 1) (W+OS)6.296.216.246.256.52–––234.13.83306.8
Judge fine-tuned onCE at startCE at 4k
R′R'_{}: own N=3N=3 samples (2-step student)1.57551.5733
RTR_{\mathrm{T}}: teacher samples1.58701.5871
own N=8N=8 samples (random K=3K=3–66 student)1.58881.5874
own N=5N=5 samples (fixed K=4K=4 student)1.58431.5835

Table S1. Oversampling: full results (GRN-B; FID-10k unless noted). Notes: Sec. S4.

Combination. Random steps leave refinement headroom, and oversampling turns it into fewer steps (Figure S2). On the K=3K=3–6 student the oversampling judge gives FID-10k 9.89 with 2 steps and 6.69 with 3, against 13.3 and 7.76 with the original judge, and 6.32 against 6.52 with 4 steps, level with the fixed 4-step student (6.33). With 50k samples it gives FID 4.02 and IS 301.3 (Table S11). On GRN-L, a round on the K=3K=3–6 student gives 10.4 with 2 steps, against 12.8 with the original judge and 16.8 for the fixed 4-step student trained as long.

Figure S2. One oversampling round on the 4-step students (GRN-B, FID-10k; 3k iterations of matching against each judge).

Diagnostics. After 6k iterations the 2-step student has caught up with its judge. The fake and real posteriors agree to 0.0013–0.0032 nats per token (Table S2), so a better student needs a better judge (§ 4.3). The student network cannot stand in for the fake score, because its prediction lies 0.24–0.71 nats per token away from it. Matching the first step as well, with its own fake score, gives a 2-step FID-10k of 7.64–7.77 at 6k iterations against 6.54 for last-step matching, so the first step is best left a draft.

Non-uniform sampling

GRN-L. Non-uniform sampling takes the base student from FID-50k 3.65 to 3.09 (Table S3, Figure S3) and lifts its recall from 0.661 to 0.710. Most of the gain is learned. Sampled back on {0,0.5}\{0, 0.5\}, the student still reaches 3.24. Every trained grid point beats the {0,0.5}\{0, 0.5\} continuation (3.83), with a plateau at 0.25 and 0.33 (3.14) and 0.17 (3.33) already past it. One student can even hold two budgets. Trained with 2 steps on {0,0.25}\{0, 0.25\} and 4 steps on the uniform grid, it gives FID-10k 5.731 and 5.285, against 5.945 and 5.527 when trained on uniform grids only.

Why the draft step. We splice the four sampling steps one at a time between the fixed 4-step GRN-L student and the weaker K=2K=2–44 student. Step 0, the draft from pure noise, carries 77–83% of the recall gap and 74–95% of the FID gap. This step gets no gradient when K≥2K \ge2, and non-uniform sampling shortens this part of the path. The teacher shows the same effect. Compressing its first 21 steps, which run without CFG, into 2 costs +3.14 FID-10k at 60 NFE, while a uniform 38-step schedule at the same NFE costs only +0.43. Learned samplers [92], the EDM discretization [38] and trajectory-regularity schedules [12] choose the time steps of a trained model, while we fix the grid before distillation.

Non-uniform sampling on GRN-B

Two thousand iterations on {0,0.25}\{0, 0.25\} take the 2-step W+OS student from FID-50k 3.83 to 3.73 (Table S4), while the same fine-tune on {0,0.5}\{0, 0.5\} ends at 3.86. Sampled on the standard grid, the {0,0.25}\{0, 0.25\} student reaches 3.64, below the teacher’s 3.73, so the gain is learned on the non-uniform grid and carries back to the standard one. Trained on {0,0.33}\{0, 0.33\}, it reaches 3.71, a plateau as on GRN-L. Over six trained grid points t1t_1 (Table S4), precision rises monotonically from 0.673 to 0.704 and recall falls from 0.740 to 0.716. The W student lacks precision (0.687 against the teacher’s 0.733) more than recall (0.722 against 0.730), so its best FID-50k comes at t1=0.6t_1 = 0.6.

What the two steps do. On 64 noises for each of 8 classes (Table S5), the draft is nearly the same for every noise of a class. Its predicted distributions vary much less across noises than the final ones (0.08–0.17 vs. 0.40–0.71). The last step keeps 67–87% of the committed draft tokens, far above the 21–29% agreement on positions it never saw, and adds what is specific to the sample. The grid point sets how much of the image the trained step decides (81%, 60% and 45% of the tokens at t1=0.25,0.5,0.67t_1 = 0.25, 0.5, 0.67), so a smaller t1t_1 buys recall at the cost of precision.

Refinement confidence keeping

Teacher. Besides the tokens GRN keeps at random with probability ts+1t_{s+1}, keeping holds on to every token whose top predicted probability is at least aa. The next step still re-predicts everything, so the rule adds no network call. On the GRN-B teacher (Table S6) it takes the 25-step FID-10k from 9.59 to 6.63, and 16 steps with keeping beat 25 steps without it (8.33, with 25 NFE against 39). Keeping whole tokens works best. Keeping confident bits comes close (6.67), and freezing confident tokens for good helps less (8.13). The threshold keeps almost no token in the first steps and most open tokens in the last ones (12% of the open tokens over 25 steps). Random tokens kept in the same share step by step match keeping with 25 steps (6.65 against 6.72 on a second machine) and trail it with 16 (8.67 against 8.35), while the same share spread evenly falls behind at both (7.00 and 10.78). Most of the gain thus comes from when tokens are kept, and confidence adds to it when the steps are few. Letting confidence replace the random choice, as in GRN’s own ablation [29], breaks sampling (146.39 with 25 steps, 91.37 against 6.17 with 50), because the clean tokens then follow the content instead of the random set GRN is trained on.

Students. At sampling time alone, keeping takes the 2-step FastGRN-B from 3.73 to 3.71 (Table S6,Table S6). Trained with keeping (a=0.51a=0.51) in every rollout of its non-uniform-sampling fine-tune, FastGRN-L improves from 3.09 to 3.03 at 2 NFE, the FID of our 4-step student, mostly through recall (0.709 to 0.716), and FastGRN-B stays at 3.73. Without this training the same threshold hurts (3.82 and 3.11).

A 2-step draft gives the rule little to work with. Its confidence hardly predicts which committed draft tokens survive (AUC 0.54), and only 0.3–0.4% of the open draft tokens reach probability 0.9. Confidence still picks better tokens than chance, since keeping the same share at random costs 0.04–0.05 FID.

The window

Why a window. Without the window (qmax⁡=0.92q_{\max}=0.92), the 2-step GRN-B student reaches FID-10k 7.87 at 4k iterations, collapses at 6.4k (the fake score’s per-token accuracy on student samples reaches 0.8) and gives mostly black or white images by 8k (Figure S4). Three forces push toward blank images. (i) Attraction. The cross-entropy part of the reverse KL pulls mass toward high real density, and the fine-tuned GRN gives the collapsed images a NELBO of 0.05–0.12 nats per token, against 2.28 for teacher samples, as likelihood models favor simple inputs [63, 79]. (ii) A lagging fake score. The entropy part that should push back comes from the online fake score, which is least reliable near the clean end. (iii) Absorption. The factor πb(Δb−EπΔ)\pi_b(\Delta_b-\mathbb{E}_{\pi}\Delta) in Eq. (2) vanishes as tokens saturate, and the rarity of blank images shows only at q≲0.1q \lesssim0.1, where the gradient is scaled down by qq. Cutting the near-clean end stops the collapse. With qmax⁡=0.4q_{\max}=0.4 no run collapsed, including a 2-step run continued to 24k iterations.

Figure S4. The window keeps ImageNet training stable, and text-to-image is more forgiving. 2-step students; the window ends at qmax⁡q_{\max} (0.92, 0.95: no window; two runs at 0.95). (a) GRN-B, FID-10k; inset: no-window samples at 8k. (b) 2k iterations; black dashed: teacher (re-sampled for FID-T).

Ablations. Without the window, matching collapses (FID-10k 332.8 at 10k), and every qmax⁡q_{\max} from 0.3 to 0.6 stays stable (Figure S4). The value 0.6 slowly drifts, and 0.3 costs fidelity (7.42 and 7.75 vs. 6.65 at 0.4). The window also helps the 4-step student, which does not collapse without it (FID-50k 4.63 → 3.99, Tab. S7b), and a single level q=0.4q=0.4 does as well (4.15 vs. 4.17). Bits are more forgiving (Figure S4, Tab. S14c). Without the window, the text-to-image student drifts in its training diagnostics while its images hold up (GenEval 0.676 at 8k). By per-token mutual information, 0.4 on bits corresponds to about 0.29 on ImageNet. Paired noise is the better start (FID-10k 6.65 against 9.85 from the fine-tuned GRN, Tab. S7a, and in text-to-image FID-T 42.1 vs. 48.6 with 0.90 vs. 0.73 of the teacher’s diversity), and the fine-tuned GRN is a better judge than the guided teacher (FID-T 41.0 vs. 49.9, Tab. S14a,b).

Toy phase diagram. A D=64D=64 Markov-chain toy with V=8V=8 values, exact real posteriors, a learned fake score and a one-step generator initialized from paired noise reproduces the collapse. The peak fraction of flat tokens (target about 0.02) is 0.61 at qmax⁡=0.92q_{\max}=0.92 and 0.24 at 0.7 (collapse), 0.093 at 0.6 (critical) and at most 0.039 at 0.3–0.5 (stable). Re-opening the window of a healthy generator to 0.92 brings the collapse back (0.275), while opening it to 0.6 does not (0.046). The unwindowed objective itself does not favor the collapsed state, yet training without the window raised it (2.52 to 4.06), so the collapse comes from the biased gradient of the learned fake score near the clean end.

Refinement and noise

Refinement after distillation. A distilled student is still a GRN, so it can be sampled with more steps than it was trained for (Figure S5). The 4-step GRN-B student improves from FID-10k 6.42 to 6.32 with 6 steps, within 0.08 of its 79-NFE teacher. The 2-step student improves from 6.65 to 6.04 with 3 steps and, after 24k iterations, to 5.84 with 4 steps, below the teacher’s 6.24. Too few steps hurt (17.3 for the 4-step student with 2 steps).

Figure S5. Refinement after distillation (ImageNet 2562256^2, FID-10k). Each curve is one trained W student sampled with NN steps; rings mark its trained steps.

Frozen noise and paired noise. Paired-noise fine-tuning pays off because GRN’s sampler freezes its noise. Redrawing the noise at every step raises the 8-step FID-10k of a paired-noise GRN-B from 13.8 to 101.6 and that of the fine-tuned GRN from 17.4 to 71.4, while the distilled 2-step student cares much less (6.65 frozen vs. 7.37 fresh).

Comparisons, text-to-image and cost

Unlike Reflow [52], paired noise does not straighten trajectories. The samples of a paired-noise GRN-B change far less between 2 and 16 steps than those of reference models (endpoint change 0.019 against 0.076 or more), yet its 16-step trajectories depend on their noise no more than theirs. Paired noise mainly makes the noise informative.

Toy evidence for the per-token gradient. On exhaustive toy problems (Table S9), the coefficients of Eq. (2) with oracle posteriors bring the one-step KL to at most 0.0021 nats on all four targets. A posterior-difference heuristic is 40× worse on the Dirichlet target and stalls at 0.4947 nats on the parity target, its total correlation, the product-of-marginals failure of Prop. S1. With a learned fake score, the same coefficients leave 0.019–0.077 nats, so the fake score is the bottleneck.

TargetDDVVTC(p)\mathrm{TC}(p)Eq. (2)Heuristicfake
Dirichlet431.0450.00210.08560.0765
Ising621.6390.00010.00020.0247
Prototypes43–0.00000.00020.0191
Parity420.4950.00000.49470.0186

Table S9. Do the coefficients of Eq. (2) matter on toy problems? DKL(p∥pθ)D_{\mathrm{KL}}(p \mathbin{\|} p_\theta) (nats) of a trained one-step generator; exhaustive computation, float64. Notes: Sec. S4.

Class-conditional comparisons. Without distribution matching, few steps fail. Even at its best temperature, the teacher’s own sampler scores FID-10k 49.6 with 4 steps, and the paired-noise GRN needs 16 steps to reach 8.7 (Figure S7, Table S8). GRN itself reports no few-step results, and its step-reduction study reaches FID 3.79 with 24 adaptive steps on average. Measured against each method’s own teacher (Table 2), the W students land at 1.07–1.15× with 2–4 steps and the W+OS+NS 2-step Table S10. Text-to-image, GRN-T2I 2B at 2562256^2: all results. Every student is a LoRA adapter distilled from 50k teacher samples with the window (W). Notes: Sec. S4.

student at 1.00×\times. With one step, Di[M]O reaches 1.05×\times. Soft-Di[M]O, trained with a GAN loss on real images, reaches 1.18×\times on MaskBit, 0.94×\times after longer training and 0.97×\times on MaskGIT. Table S11 gives every row of Table 1 with precision, recall and latency, and Table S12 adds more distilled students, the native discrete samplers and few-step generators in continuous latent spaces. GRN-L with 4 steps trades a little recall for precision (0.697 against 0.729 recall, 0.734 against 0.728 precision), as expected from the mode-seeking reverse KL. Table S10 lists every text-to-image result, Table S13 the few-step students of other models as reported, Table S14 the ablations and Table S15 the cost numbers of Sec. S4.2. Figure S6 shows matched-noise samples. On 5k COCO-2014 validation captions, scored against all 40,504 validation images, the 2-step student (qmax⁡=0.4q_{\max}=0.4, 4k) reaches FID 26.2 and the CFG-4.5 teacher 29.6, and the student’s FID to the teacher’s images is 12.4.

Figure S6. At 4 and 2 steps FastGRN gives clean samples, while its paired-noise initialization alone falls apart (GRN-T2I 2B, GRN-B, 2562256^2; same noise per column). FastGRN rows: W students; paired-noise GRN row: the initialization, 2 steps.

SettingGenEval↑\uparrowFID-T↓\downarrowHPS↑\uparrowDiv.↑\uparrowHealthy
(a) Initialization at qmax⁡=0.4q_{\max}=0.4 (1k it.)
Fine-tuned init.0.60248.625.510.73no
Paired-noise init.–42.126.160.90yes
(b) Real score at qmax⁡=0.4q_{\max}=0.4 (2k it.)
Guided teacher0.60249.926.110.85yes
Fine-tuned0.63741.026.380.90yes
(c) Window qmax⁡q_{\max} (2k it. unless noted)
0.400.63741.026.380.90yes
0.550.661 (+0.025)39.126.580.91yes
0.95 (no window)∗^*0.661 (+0.024)37.626.710.90yes
0.95 (no window)†^\dagger0.678 (+0.041)37.426.660.91yesb^b
same with 8k it.0.676 (+0.024)w^w38.426.870.91nov^v

Table S14. Ablations on text-to-image (GRN-T2I, 2562256^2, LoRA students). All rows: 2 sampling steps, paired-noise init., fine-tuned real score unless varied. Notes: Sec. S4.

Text-to-image

Figure S10–Figure S12 show every few-step route of the same teacher for 24 text-to-image prompts and 12 ImageNet classes, with one frozen noise per column and the prompts and classes fixed before any image was generated. Figure S13 and Figure S14 extend Figure 3 to 24 GenEval prompts, the first four of each category. FastGRN draws the requested number of objects and their relations more often, for example two clocks, a dog right of a teddy bear and a yellow keyboard next to a black sink. The other students tend to merge two objects into one, such as a cow inside a laptop screen, or to drop one of them. Every model misses some prompts, and FastGRN draws six handbags for four. On ImageNet (Figure S15), ReDi with 1 step blurs textures, DD1 with 2 steps often loses the object, such as the balloon and the coral reef, Di[M]O draws clean, centred objects on smooth backgrounds, and Di4C with 4 guided steps distorts some of them, such as the lion. FastGRN keeps the photographic texture and the scene layout with 2 steps. In Figs. S8 and S9 each column fixes the class and the frozen noise (tench, hermit crab, Siberian husky, black-and-white colobus, cliff dwelling, lifeboat, quilt and trombone).

Figure S12. ImageNet, same noise (GRN-B; second sample of each class from the 50k evaluations of Table 1). Fine-tuned and paired-noise GRN: GRN fine-tuned on teacher samples with independent and with paired noise; without window: matching without the window, best checkpoint before it collapses.

Figure S13. Text-to-image against other few-step models (GenEval prompts, the first two of each category; each model’s released sampler; all shown at 2562256^2).

Figure S14. Text-to-image against other few-step models, the third and fourth prompt of each category.

Figure S15. ImageNet against other few-step students (the GAN-free students of discrete generators in Table 2 with public weights; each model’s released sampler and settings; one image per class).

Notes to the supplementary tables

Table S1. Top: 3k-iteration continuations of the 10k-iteration 2-step W student, with the fake score initialized from the judge; the last oversampling row gives the FID50k_{50k} of Tab. 1. Original judge: GRN fine-tuned on teacher samples (EMA 0.9995); oversampling judge R′R': the original judge fine-tuned on 200k of the student’s own N=3N=3 samples; teacher-data judge RTR_T: the same recipe on the teacher samples the original judge was trained on. P_T / R_T / FID-T against 50k teacher samples; –: not run. Teacher: FID-10k 6.24, FID-50k 3.73, IS-50k 305.9. Bottom: validation cross-entropy (nats per token, held-out samples of the same source) before and after the judge fine-tune.

Table S2. DFD_F: online fake score; DRD_R: real score; DθD_\theta: the student network’s own prediction at (z′,q)(z', q). Live student weights, τ=0.8\tau= 0.8.

Table S3. Continuations of the 10k-iteration W 2-step

GRN-L student, the base student (2000 iterations, Sec. S2), trained and sampled on the grid of the row. PT\mathrm{P}_{T}, RT\mathrm{R}_{T}, FID-T against 50k GRN-L teacher samples (k=3k=3). (b): grid point t1t_{1} of training and sampling, against the {0,0.5}\{0, 0.5\} fine-tune; against the untrained base student on {0,0.5}\{0, 0.5\} (6.066), +0.540 at 50k; the student with K=2K=2 and 4 is trained on {0,0.25}\{0, 0.25\} for 2 steps and on the uniform grid for 4 steps. Teacher: FID50k\mathrm{FID}_{50k} 2.65.

Table S4. Each student continues a 2-step student of Tab. 1 for 2000 iterations, trained and sampled on the grid of the row. PT\mathrm{P}_{T}, RT\mathrm{R}_{T} (k=3k=3) against 50k GRN-B teacher samples. Teacher: FID50k\mathrm{FID}_{50k} 3.73, PT\mathrm{P}_{T} 0.733, RT\mathrm{R}_{T} 0.730. Bold: lowest FID50k\mathrm{FID}_{50k} in (a) and in the trained rows of (b).

Table S5. Diversity: mean over noise pairs of half the squared distance between the predicted token distributions of two noises of one class, for the draft (first step) and the final prediction. Kept: share of the committed draft tokens that the last step keeps; Chance: the same agreement on positions that were not committed. Last step: share of the tokens the trained last step decides, 1−t1×Kept1-t_{1}\times\mathrm{Kept}. +2k rows: trained and sampled on {0,t1}\{0,t_{1}\}.

Table S6. Freezing: confident tokens stay fixed for all later steps. Bit rules: the same for the confident bit prefix of each token. Top-confidence: the most confident tokens replace the random choice (GRN’s own ablation). Random: tokens kept at random in keeping’s extra share, per step or in total. Extra: share of the open draft tokens that the rule keeps as well. Second block and 50-step column: a second L40S (native 25 steps: 9.59). Teacher: GRN’s CFG and temperature (1.33) with 25, 16 and 50 steps (39, 25 and 79 NFE), a=0.55a=0.55 and 0.9 per bit for the bit rules. Trained with keeping: the non-uniform-sampling fine-tune of Tab. 1 repeated with keeping at a=0.51a=0.51 in every rollout. PT\mathrm{P}_{T}, RT\mathrm{R}_{T} (k=3k=3) against 50k samples of each student’s teacher. Bold: lowest FID per column and block, and per panel in (b), (c).

Table S7. W students and their no-window and initialization variants; EMA generators; teacher FID-10k 6.24. “Best”: best 1k-spaced milestone before a collapse. Collapse: first iteration at which the fake model’s per-token accuracy on student samples reaches 0.8 (healthy runs: about 0.45). No window: q∈[0.02,0.92]q \in[0.02, 0.92]; bold: FastGRN (paired-noise init., qmax⁡=0.4q_{\max}=0.4). Anchor: extra loss toward the teacher’s paired samples.

Table S8. One network call per step, except nthe teacher’s own sampler with CFG: 3/6/12 NFE for 2/4/8 steps. Temperatures: on GRN-B the one with the lowest FID-10k per cell (with GRN’s 1.33 the teacher’s own sampler scores 63.5 at 8 steps); on GRN-L, 1.0 for the teacher’s own sampler and 0.8 for the fine-tuned models. cCollapses later in training. hNon-uniform sampling on {0,0.25}\{0, 0.25\}; its base student scores 6.04 on the same machine.

Table S9. DD tokens with VV values each. The generator is a free table of per-token logits for each of VDV^D latent values, which can represent every target exactly. 12 corruption levels in [0.05,0.95][0.05, 0.95]. Heuristic: the logit gradient DF(⋅∣z)−DR(⋅∣z)D_F(\cdot\mid z)-D_R(\cdot\mid z), in the spirit of Di[M]O [110]. Learned fake: the generator-side posterior is a model trained online.

Table S10. Δ\Delta vs. T.: GenEval difference to the teacher. FID-T: FID to 1k teacher images for the same captions; athe teacher against a second teacher sample set. CLIP: CLIP-L/14 ×100\times100. HPS: HPS v2.1. Div.: same-prompt LPIPS diversity relative to the teacher. qmax⁡q_{\max} 0.55→\to0.95: the 2-step student with 0.55 continued for 1k iterations at 0.95 (CLIP and HPS not measured). *Reported by GRN for its 102421024^2 checkpoint with a prompt rewriter.

Table S11. GRN-B with 8 to 16 steps: the teacher’s own sampler at the temperature with the lowest FID-10k (0.7, 0.8, 0.8, 1.0 and 1.0 for 8, 10, 12, 14 and 16 steps; CFG on t∈(0.4,1)t \in(0.4, 1) adds network calls). Fine-tuned and paired-noise GRN: fine-tuned on teacher samples with independent and with paired noise. Matching without window: its best checkpoint (4k iterations) before it collapses. Latency on one RTX 4090; the paired-noise GRN and matching without window share the latency of the students with the same steps, and the other baseline rows are not timed. Tags: W paired-noise initialization and windowed matching (10k iterations); RS random steps over K=2K=2–44 (6k); OS one oversampling round (+3k); W+RS+OS: a K=3K=3–66 student plus one round; NS non-uniform sampling, +2k iterations on {0,0.25}\{0, 0.25\}; RCK refinement confidence keeping in that fine-tune and when sampling. Rows with more steps than trained: the same student sampled with 3 steps on {0,1/3,2/3}\{0, 1/3, 2/3\} (GRN-B) or {0,0.25,0.5}\{0, 0.25, 0.5\} (GRN-L), or with 4 steps on {0,0.23,0.54,0.68}\{0, 0.23, 0.54, 0.68\}, without training (3-step latency not measured). GRN-L W+NS+RCK+OS: one round on the 2-step W+NS+RCK student with the judge fine-tuned on its own 3-step samples (+2k). The first two GRN-L 3-step rows: that student after 2k more iterations with a learning rate decaying to 5% over the last 1k, without and with keeping when sampling; the last: the student itself. All students use EMA 0.999. Bold: best student FID. Table S12. †Size not stated in the paper; the MaskGIT it starts from (Tab. 2). §Sampling steps as reported, with

Figure S11. Text-to-image, same noise, prompts 13–24.

classifier-free guidance at inference. Teacher FID of the DD1 and DD2 rows as each paper reports it: VAR-d16 4.19 and 4.15, VAR-d20 3.35 and 3.40, VAR-d24 2.51 and 2.86, LlamaGen-B 5.42, LlamaGen-L 4.11; Di4C: its 8-step MaskGIT (6.57). SDTT and ReDi: as run by ReDi, whose MaskGIT has no 16-step reference. MaskGIT-PyTorch: the teacher of the MaskGIT students with its own sampler, as run by Di[M]O. MaskGIT and TiTok: their own 8-step samplers without guidance, no distillation. Token-Critic: 18 generator and 18 critic calls. DDPD: a planner and a denoiser, two calls per step. ReCAP: 8 full and 8 cached calls. VAR-d16 with its first scales skipped: as run by DD1. MeanFlow, α-Flow, iMF, Drifting and SoFlow are trained from scratch in the SD-VAE latent space; DMF turns a flow model into a flow map, and FreeFlow distills a SiT-XL/2 flow model (teacher FID 2.06). They fold guidance into training, so one step is one network call; IMM uses CFG at its one step (two calls), and Shortcut’s step count is as reported. Table S13. Each model at its own resolution; Show-o Turbo without CFG at 2562256^2 and 5122512^2 (2, 4, 8 steps). SANA-

Sprint and FLUX.1-schnell as reported by SANA-Sprint (its Table 2, 102421024^{2}), whose SANA teachers score 0.64 (0.6B) and 0.66 (1.6B); the other continuous rows are taken from Soft-Di[M]O’s Table 3, which evaluates all of them but InstaFlow from public checkpoints. ggGAN loss; rrreward fine-tuning; ssTTS: test-time search guided by reward models. Bold: highest among the students of discrete generators. FastGRN: GRN-T2I 2B at 2562256^{2}, 4 and 2 steps (Figure 5).

Table S14. GenEval: 553 prompts, 4 images each; FID-T and HPS on the 1k COCO captions; Div.: LPIPS diversity relative to the teacher. (a): no GenEval at 1k. Parentheses in (c): paired GenEval difference to qmax⁡=0.4q_{\max}=0.4. Healthy: in (a, b), the training-side health check (diversity, same-prompt agreement, entropy, collapse guard) at 2k; in (c), no collapse signal up to the row’s iteration. vTraining-side warnings only (collapse-guard votes at 5.5–5.55k, 7.4–7.5k, 7.65–8k iterations); the images stay intact (FID-T 37.4–38.4 at 1k–8k). wvs. qmax⁡=0.4q_{\max}=0.4 at 4k, the longest windowed run. ∗, †: two training runs without the window; † is trained to 8k.

Table S15. Throughput: generation only on an A100-40GB (C2I, batch 250–500, torch.compile) or on an RTX 4090 (sT2I, batch 64). “Net” is the generation speedup over the teacher; “+Dec.” also counts the HBQ decoder (C2I: about 20 ms per image; T2I: about 25 ms for the students, 37 ms in the teacher run). Batch 1: median end-to-end latency including decoding (ms) and the gain over the teacher.

References

  1. [1]Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G. Baraniuk. Self-consuming generative models go MAD. In ICLR, 2024. 3arxiv.org/abs/2307.01850
  2. [2]Thomas Anthony, Zheng Tian, and David Barber. Thinking fast and slow with deep learning and tree search. In NeurIPS, 2023. 7, 14
  3. [3]Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In NeurIPS, 2021. 2, 3arxiv.org/abs/2107.03006
  4. [4]Jinbin Bai, Tian Ye, Wei Chow, Enxin Song, Xiangtai Li, Zhen Dong, Lei Zhu, and Shuicheng Yan. Meissonic: Revitalizing masked generative transformers for efficient high-resolution text-to-image synthesis. In ICLR, 2025. 2arxiv.org/abs/2410.08261
  5. [5]Quentin Bertrand, Avishek Joey Bose, Alexandre Duplessis, Marco Jiralerspong, and Gauthier Gidel. On the stability of iterative retraining of generative models on their own data. In ICLR, 2024. 3arxiv.org/abs/2310.00429
  6. [6]Victor Besnier and Mickael Chen. A PyTorch reproduction of masked generative image transformer. arXiv preprint arXiv:2310.14400, 2023. 16, 30arxiv.org/abs/2310.14400
  7. [7]Black Forest Labs. FLUX. https://github.com/black-forest-labs/flux, 2024. 30
  8. [8]Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. In NeurIPS, 2022. 2
  9. [9]Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design. In ICML, 2024. 2arxiv.org/abs/2402.04997
  10. [10]Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. MaskGIT: Masked generative image transformer. In CVPR, 2022. 1, 2, 30arxiv.org/abs/2202.04200
  11. [11]Huiwen Chang, Han Zhang, Jarred Barber, AJ Moshinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative transformers. In ICML, 2023. 1, 2arxiv.org/abs/2301.00704
  12. [12]Defang Chen, Zhenyu Zhou, Can Wang, Chunhua Shen, and Siwei Lyu. On the trajectory regularity of ODE-based diffusion sampling. In ICML, 2024. 17arxiv.org/abs/2405.11326
  13. [13]Junsong Chen, Shuchen Xu, Yuyang Zhao, Jincheng Yu, Sayak Paul, Junyu Chen, Han Cai, Song Han, and Enze Xie. SANA-Sprint: One-step diffusion with continuous-time consistency distillation. In ICCV, 2025. 30arxiv.org/abs/2503.09641
  14. [14]Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In CVPR, 2022. 15arxiv.org/abs/2112.01527
  15. [15]Haoge Deng, Ting Pan, Fan Zhang, Yang Liu, Zhuoyan Luo, Yufeng Cui, Wenxuan Wang, Chunhua Shen, Shiguang Shan, Zhaoxiang Zhang, and Xinlong Wang. Uniform form discrete diffusion with metric path for video generation. In ICLR, 2026. 2arxiv.org/abs/2510.24717
  16. [16]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, 2009. 6
  17. [17]Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He. Generative modeling via drifting. In NeurIPS, 2026. 30arxiv.org/abs/2602.04770
  18. [18]Justin Deschenaux and Caglar Gulcehre. Beyond autoregression: Fast LLMs via self-distillation through time. In ICLR, 2025. 3, 6, 30arxiv.org/abs/2410.21035
  19. [19]Patrick Esser, Robin Rombach, and Björn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, 2021. 1, 2arxiv.org/abs/2012.09841
  20. [20]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In ICML, 2024. 2arxiv.org/abs/2403.03206
  21. [21]Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. In ICLR, 2025. 2, 3, 30arxiv.org/abs/2410.12557
  22. [22]Tommaso Furlanello, Zachary C. Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born again neural networks. In ICML, 2018. 3arxiv.org/abs/1805.04770
  23. [23]Yansong Gao and Yu Sun. Distillation of discrete diffusion by exact conditional distribution matching. arXiv preprint arXiv:2512.12889, 2025. 3arxiv.org/abs/2512.12889
  24. [24]Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. Discrete flow matching. In NeurIPS, 2024. 2arxiv.org/abs/2407.15595
  25. [25]Zhengyang Geng, Mingyang Deng, Xingjian Bai, J. Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. In NeurIPS, 2025. 2, 3, 13, 30arxiv.org/abs/2505.13447
  26. [26]Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J. Zico Kolter, and Kaiming He. Improved mean flows: On the challenges of fastforward generative models. In CVPR, 2026. 30arxiv.org/abs/2512.02012
  27. [27]Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. GenEval: An object-focused framework for evaluating text-to-image alignment. In NeurIPS, 2023. 6, 15arxiv.org/abs/2310.11513
  28. [28]Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. In CVPR, 2025. 1, 2arxiv.org/abs/2412.04431
  29. [29]Jian Han, Jinlai Liu, Jiahuan Wang, Bingyue Peng, and Zehuan Yuan. Generative refinement networks for visual synthesis. In ECCV, 2026. 1, 2, 3, 6, 18, 22arxiv.org/abs/2604.13030
  30. [30]Satoshi Hayakawa, Yuhta Takida, Masaaki Imaizumi, Hiromi Wakaki, and Yuki Mitsufuji. Distillation of discrete diffusion through dimensional correlations. In ICML, 2025. 3, 4, 6, 30arxiv.org/abs/2410.08709
  31. [31]Jiajun He, Wenlin Chen, Mingtian Zhang, David Barber, and José Miguel Hernández-Lobato. Training neural samplers with reverse diffusive KL divergence. In AISTATS, 2025. 2arxiv.org/abs/2410.12456
  32. [32]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In NeurIPS, 2017. 4, 6
  33. [33]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS Workshop on Deep Generative Models and Downstream Applications, 2021. 1arxiv.org/abs/2207.12598
  34. [34]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020. 2arxiv.org/abs/2006.11239
  35. [35]Emiel Hoogeboom, David Ruhe, Jonathan Heek, Thomas Mensink, and Tim Salimans. Beyond single tokens: Distilling discrete diffusion models via discrete MMD. arXiv preprint arXiv:2603.20155, 2026. 3arxiv.org/abs/2603.20155
  36. [36]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 6arxiv.org/abs/2106.09685
  37. [37]Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with Gumbel-Softmax. In ICLR, 2017. 4arxiv.org/abs/1611.01144
  38. [38]Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In NeurIPS, 2022. 17arxiv.org/abs/2206.00364
  39. [39]Fairoz Nower Khan, Nabuat Zaman Nahim, Md Sajid Ahmed, Ruiquan Huang, and Peizhong Ju. Discrete Mean-Flow: One-step generation via conditional transition kernels. arXiv preprint arXiv:2605.12805, 2026. 3arxiv.org/abs/2605.12805
  40. [40]Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ODE trajectory of diffusion. In ICLR, 2024. 3arxiv.org/abs/2310.02279
  41. [41]Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. In NeurIPS, 2019. 6arxiv.org/abs/1904.06991
  42. [42]Kyungmin Lee, Sihyun Yu, and Jinwoo Shin. Decoupled MeanFlow: Turning flow models into flow maps for accelerated sampling. In ICLR, 2026. 30arxiv.org/abs/2510.24474
  43. [43]José Lezama, Huiwen Chang, Lu Jiang, and Irfan Essa. Improved masked image generation with Token-Critic. In ECCV, 2022. 2, 30arxiv.org/abs/2209.04439
  44. [44]Tianhong Li and Kaiming He. Back to basics: Let denoising generative models denoise. In CVPR, 2026. 2arxiv.org/abs/2511.13720
  45. [45]Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. In NeurIPS, 2024. 2arxiv.org/abs/2406.11838
  46. [46]Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014. 6, 15arxiv.org/abs/1405.0312
  47. [47]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In ICLR, 2023. 2arxiv.org/abs/2210.02747
  48. [48]Enshu Liu, Qian Chen, Xuefei Ning, Shengen Yan, Guohao Dai, Zinan Lin, and Yu Wang. Distilled decoding 2: One-step sampling of image auto-regressive models with conditional score distillation. In NeurIPS, 2025. 6, 30
  49. [49]Enshu Liu, Xuefei Ning, Yu Wang, and Zinan Lin. Distilled decoding 1: One-step sampling of image auto-regressive models with flow matching. In ICLR, 2025. 2, 6, 30arxiv.org/abs/2412.17153
  50. [50]Runjing Liu, Jeffrey Regier, Nilesh Tripuraneni, Michael I. Jordan, and Jon McAuliffe. Rao-Blackwellized stochastic gradients for discrete distributions. In ICML, 2019. 4arxiv.org/abs/1810.04777
  51. [51]Sulin Liu, Juno Nam, Andrew Campbell, Hannes Stärk, Yilun Xu, Tommi Jaakkola, and Rafael Gómez-Bombarelli. Think while you generate: Discrete diffusion with planned denoising. In ICLR, 2025. 30arxiv.org/abs/2410.06264
  52. [52]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023. 2, 21arxiv.org/abs/2209.03003
  53. [53]Xingchao Liu, Xiwen Zhang, Jianzhou Ma, Jian Peng, and Qiang Liu. InstaFlow: One step is enough for high-quality diffusion-based text-to-image generation. In ICLR, 2024. 2, 30arxiv.org/abs/2309.06380
  54. [54]Xuejie Liu, Anji Liu, Guy Van den Broeck, and Yitao Liang. Plug-and-play context feature reuse for efficient masked generation. In NeurIPS, 2025. 30arxiv.org/abs/2505.19089
  55. [55]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 14
  56. [56]Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. In ICML, 2024. 2, 3, 4arxiv.org/abs/2310.16834
  57. [57]Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. In ICLR, 2025. 2arxiv.org/abs/2410.11081
  58. [58]Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023. 2, 30arxiv.org/abs/2310.04378
  59. [59]Tianze Luo, Haotian Yuan, and Zhuang Liu. SoFlow: Solution flow models for one-step generative modeling. In ICLR, 2026. 30arxiv.org/abs/2512.15657
  60. [60]Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-Instruct: A universal approach to transferring knowledge from pre-trained diffusion models. In NeurIPS, 2023. 2arxiv.org/abs/2305.18455
  61. [61]Yihong Luo, Tianyang Hu, Jiacheng Sun, Yujun Cai, and Jing Tang. Learning few-step diffusion models by trajectory distribution matching. In ICCV, 2025. 3, 30arxiv.org/abs/2503.06674
  62. [62]Chenlin Meng, Kristy Choi, Jiaming Song, and Stefano Ermon. Concrete score matching: Generalized score matching for discrete data. In NeurIPS, 2022. 2, 4arxiv.org/abs/2211.00802
  63. [63]Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don’t know? In ICLR, 2019. 20
  64. [64]Zanlin Ni, Yulin Wang, Renping Zhou, Jiayi Guo, Jinyi Hu, Zhiyuan Liu, Shiji Song, Yuan Yao, and Gao Huang. Revisiting non-autoregressive transformers for efficient image synthesis. In CVPR, 2024. 30arxiv.org/abs/2406.05478
  65. [65]Zanlin Ni, Yulin Wang, Renping Zhou, Rui Lu, Jiayi Guo, Jinyi Hu, Zhiyuan Liu, Yuan Yao, and Gao Huang. AdaNAT: Exploring adaptive policy for token-based image generation. In ECCV, 2024. 30arxiv.org/abs/2409.00342
  66. [66]Yidong Ouyang, Zhengyan Wan, Themis Haris, Tian Tan, Liqian Peng, Henry Li, Ziqian Lin, Jiahhang Chen, Maryam Karimzadehgan, Alec Go, and George Michailidis. Acceleration of diffusion language model through discrete average generator. arXiv preprint arXiv:2609.38364, 2026. 3, 6arxiv.org/abs/2609.38364
  67. [67]Yong-Hyun Park, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, and Yuki Mitsufuji. Jump your steps: Optimizing sampling schedule of discrete diffusion models. In ICLR, 2025. 3arxiv.org/abs/2410.07761
  68. [68]William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 2arxiv.org/abs/2212.09748
  69. [69]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021. 15arxiv.org/abs/2103.00020
  70. [70]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022. 2arxiv.org/abs/2112.10752
  71. [71]Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your steps: Optimizing sampling schedules in diffusion models. In ICML, 2024. 3arxiv.org/abs/2404.14507
  72. [72]Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your flow: Scaling continuous-time flow map distillation. In NeurIPS, 2025. 3arxiv.org/abs/2506.14603
  73. [73]Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, and Volodymyr Kuleshov. The diffusion duality. In ICML, 2025. 2arxiv.org/abs/2506.10892
  74. [74]Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In ICLR, 2022. 2arxiv.org/abs/2202.00512
  75. [75]Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In NeurIPS, 2016. 6arxiv.org/abs/1606.03498
  76. [76]Tim Salimans, Thomas Mensink, Jonathan Heek, and Emil Hooge boom. Multistep distillation of diffusion models via moment matching. In NeurIPS, 2024. 3arxiv.org/abs/2406.04103
  77. [77]Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In ECCV, 2024. 2, 7, 30arxiv.org/abs/2311.17042
  78. [78]Yair Schiff, Subham Sekhar Sahoo, Hao Phung, Guanghan Wang, Sam Boshar, Hugo Dalla-torre, Bernardo P. de Almeida, Alexander Rush, Thomas Pierrot, and Volodymyr Kuleshov. Simple guidance mechanisms for discrete diffusion models. In ICLR, 2025. 2, 3arxiv.org/abs/2412.10193
  79. [79]Joan Serra, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F. Núñez, and Jordi Luque. Input complexity and out-of-distribution detection with likelihood-based generative models. In ICLR, 2020. 20arxiv.org/abs/1909.11480
  80. [80]David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. Mastering the game of Go without human knowledge. Nature, 550 (7676):354–359, 2017. 3DOI
  81. [81]Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021. 2
  82. [82]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023. 2, 3arxiv.org/abs/2303.01469
  83. [83]Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 1, 2arxiv.org/abs/2406.06525
  84. [84]Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. In NeurIPS, 2024. 1, 2, 30arxiv.org/abs/2404.02905
  85. [85]Michalis K. Titsias and Miguel Lázaro-Gredilla. Local expectation gradients for black box variational inference. In NeurIPS, 2015. 3, 4
  86. [86]Shangyuan Tong, Nanye Ma, Saining Xie, and Tommi Jaakkola. Flow map distillation without data. In CVPR, 2026. 30arxiv.org/abs/2511.19428
  87. [87]Vinh Tong, Trung-Dung Hoang, Anji Liu, Guy Van den Broeck, and Mathias Niepert. Learning to discretize denoising diffusion ODEs. In ICLR, 2025. 3arxiv.org/abs/2405.15506
  88. [88]Chaoyang Wang and Yunhai Tong. One-step distillation of discrete diffusion image generators via fixed-point iteration. arXiv preprint arXiv:2605.21484, 2026. 2, 6, 7, 30arxiv.org/abs/2605.21484
  89. [89]Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and Volodymyr Kuleshov. Remasking discrete diffusion models with inference-time scaling. In NeurIPS, 2025. 30arxiv.org/abs/2503.00307
  90. [90]Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. ProlificDreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. In NeurIPS, 2023. 2arxiv.org/abs/2305.16213
  91. [91]Satoshi Watanabe. Information theoretical analysis of multivariate correlation. IBM Journal of Research and Development, 4(1):66–82, 1960. 2, 4, 13DOI
  92. [92]Daniel Watson, William Chan, Jonathan Ho, and Mohammad Norouzi. Learning fast samplers for diffusion models by differentiating through sample quality. In ICLR, 2022. 17arxiv.org/abs/2202.05830
  93. [93]Tianhe Wu, Ruibin Li, Lei Zhang, and Kede Ma. Diversity-preserved distribution matching distillation for fast visual synthesis. In ICML, 2026. 2arxiv.org/abs/2602.03139
  94. [94]Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341, 2023. 6, 15arxiv.org/abs/2306.09341
  95. [95]Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weih ao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhengyang Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. In ICLR, 2025. 2arxiv.org/abs/2408.12528
  96. [96]Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V. Le. Self-training with Noisy Student improves ImageNet classification. In CVPR, 2020. 3arxiv.org/abs/1911.04252
  97. [97]Yi Xin, Le Zhuo, Qi Qin, Siqi Luo, Yuewen Cao, Bin Fu, Yangfan He, Hongsheng Li, Guangtao Zhai, Xiaohong Liu, and Peng Gao. Recurrent mask AutoRegressive modeling for efficient and scalable image generation. arXiv preprint arXiv:2507.13032, 2025. 30arxiv.org/abs/2507.13032
  98. [98]Chenkai Xu, Xu Wang, Zhenyi Liao, Yishun Li, Tianqi Hou, and Zhijie Deng. Show-o Turbo: Towards accelerated unified multimodal understanding and generation. arXiv preprint arXiv:2502.05415v1, 2025. 2, 7, 30
  99. [99]Yilun Xu, Weili Nie, and Arash Vahdat. One-step diffusion models with f-divergence distribution matching. arXiv preprint arXiv:2502.15681, 2025. 2arxiv.org/abs/2502.15681
  100. [100]Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Frédo Durand, and William T. Freeman. Improved distribution matching distillation for fast image synthesis. In NeurIPS, 2024. 2, 3, 4, 7, 30arxiv.org/abs/2405.14867
  101. [101]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T. Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In CVPR, 2024. 2, 3arxiv.org/abs/2311.18828
  102. [102]Jaehoon Yoo, Wonjung Kim, and Seunghoon Hong. ReDi: Rectified discrete flow. In NeurIPS, 2025. 2, 3, 4, 6, 7, 8, 30arxiv.org/abs/2507.15897
  103. [103]Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation. In NeurIPS, 2024. 30arxiv.org/abs/2406.07550
  104. [104]Huijie Zhang, Aliaksandr Siarohin, Willi Menapace, Michael Vasilkovsky, Sergey Tulyakov, Qing Ou, and Ivan Skorokhodov. AlphaFlow: Understanding and improving MeanFlow models. In ICLR, 2026. 30arxiv.org/abs/2510.20771
  105. [105]Mingtian Zhang, Peter Hayes, Tom Bird, Raza Habib, and David Barber. Spread divergence. In ICML, 2020. 2
  106. [106]Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 6, 15arxiv.org/abs/1801.03924
  107. [107]Yixiu Zhao, Jiaxin Shi, Feng Chen, Shaul Druckmann, Lester Mackey, and Scott Linderman. Informed correctors for discrete diffusion models. In NeurIPS, 2025. 30arxiv.org/abs/2407.21243
  108. [108]Haoyang Zheng, Xinyang Liu, Cindy Xiangrui Kong, Nan Jiang, Zheyuan Hu, Weijian Luo, Wei Deng, and Guang Lin. Ultra-fast language generation via discrete diffusion divergence instruct. In ICLR, 2026. 2, 3arxiv.org/abs/2509.25035
  109. [109]Linqi Zhou, Stefano Ermon, and Jiaming Song. Inductive moment matching. In ICML, 2025. 3, 30arxiv.org/abs/2503.07565
  110. [110]Yuanzhi Zhu, Xi Wang, Stéphane Lathuilière, and Vicky Kalogeiton. Di[M]O: Distilling masked diffusion models into one-step generator. In ICCV, 2025. 2, 6, 7, 25, 30
  111. [111]Yuanzhi Zhu, Xi Wang, Stéphane Lathuilière, and Vicky Kalogeiton. Soft-Di[M]O: Improving one-step discrete image generation with soft embeddings. In ICLR, 2026. 2, 6, 7, 30arxiv.org/abs/2509.22925

Paper details

vlnc:W_9xzzv4jv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents