betaFeedback

WorksComputer Science

ReLaDrift: Recurrent Language Generation via Drifting for Test-Time Scaling

Abstract

Drifting Models generate a sample with a single function evaluation, making them attractive for fast parallel language generation, but their one-step formulation does not support iterative refinement. We propose Recurrent Language Generation via Drifting (ReLaDrift), which uses a shared Drifting Model to generate an initial draft from noise and iteratively refine it in a continuous latent space. We train this recurrence by sampling the number of rounds and applying the drifting loss only at the final round, stopping gradients through preceding rounds so that only one round requires backpropagation. Unlike the one-step Drifting Model, whose accuracy does not improve when iterated, ReLaDrift enables a trade-off between test-time computation and generation quality, as diffusion and flow language models do. On Sudoku Hard and GSM8K, it achieves competitive or superior performance compared with these iterative-generation approaches across a range of inference budgets. Our code is available at https://github.com/Yuki-Ots/ReLaDrift

Introduction

The Drifting Model (Deng et al., [5]) learns a direct mapping from noise to data using a drift field computed from real and generated samples during training, enabling generation in a single function evaluation at inference time. While attractive for fast language generation, this one-step formulation offers no mechanism to improve outputs with additional test-time computation, a form of test-time scaling (Snell et al., 2025). In contrast, diffusion and flow language models can trade additional sampling steps for improved generation quality (Austin et al., [14]; Sahoo et al., 2024, 2025; Nie et al., 2025; Li et al., 2022; Lee et al., 2026; Deschenaux and Gulcehre, [7]; Hu et al., 2026).

Simply iterating the Drifting Model does not guarantee improvement, as it is trained to generate from noise rather than refine its own outputs. Indeed, Figure 1 shows that simply iterating a one-step Drifting Model yields no accuracy gains on either Sudoku Hard (Alp, [13]) or GSM8K (Cobbe et al., [3]). These observations raise the central question of this work: how can we train a one-step Drifting Model to refine its own outputs, enabling a trade-off between inference-time computation and

Figure 1. Accuracy vs. the number of function evaluations (NFE) on Sudoku Hard (top) and GSM8K (bottom). Results use a single generation trajectory and EMA weights, with mean ±\pm standard deviation over three seeds for Sudoku. Models are evaluated at 20,000 updates (Sudoku) and 250,000 updates (GSM8K); sampling settings are detailed in Tables 1 and Table 3.

ate from noise rather than refine its own outputs. Indeed, Figure 1 shows that simply iterating a one-step Drifting Model yields no accuracy gains on either Sudoku Hard (Alp, [13]) or GSM8K (Cobbe et al., [3]). These observations raise the central question of this work: how can we train a one-step Drifting Model to refine its own outputs, enabling a trade-off between inference-time computation and generation quality?

We propose Recurrent Language Generation via Drifting (ReLaDrift), which uses a shared Drifting Model to generate an initial draft from noise and iteratively refine it in a continuous latent space over successive rounds. During training, we sample the number of rounds and repeatedly apply the shared model, starting from noise and feeding each draft into the next round. We apply the drifting loss only to the final output, stopping gradients through the preceding rounds to avoid backpropagation through time (§ 4).

We evaluate ReLaDrift on Sudoku Hard ([13]) and GSM8K ([3]), two reasoning tasks with verifiable answers that allow us to measure test-time scaling directly. The tasks present complementary challenges: Sudoku requires a unique solution satisfying global constraints, whereas GSM8K admits multiple valid Python programs ([9]). Our results demonstrate that, by using ReLaDrift, drifting-based generation can trade additional test-time computation for improved accuracy, achieving competitive or superior performance to established inference-time refinement approaches, including diffusion and flow language models, across a range of inference budgets (§ 5).

Related Work

Iterative generation vs. one-step drifting. Diffusion and flow language models iteratively refine samples at inference time, allowing a trade-off between generation quality and computational cost ([14, 7] (Sahoo et al. 2024; Sahoo et al. 2025; Nie et al. 2025; Li et al. 2022; Lee et al. 2026; Hu et al. 2026)). Improving this trade-off has been an active area of research ([6, 8, 11, 2] (Lee et al. 2026; Song et al. 2023; Oba et al. 2026a)). In contrast, the Drifting Model ([5]) is designed for one-step generation, with no built-in mechanism to benefit from additional inference steps, as it is trained to generate from noise rather than refine its own outputs.

Drifting-based methods. Drift Flow Matching ((Ma et al., 2026)) bridges drifting and flow matching by learning transport between arbitrary time points along a noise-to-data interpolation, enabling test-time scaling through finer transport steps, as demonstrated in image generation and robotic control. Oba et al. (2026b) apply the drifting objective to improve diffusion language models, rather than using the drifting map itself as a generator. In contrast, ReLaDrift first generates a complete draft in one step, as in the original Drifting Model, and then refines it recurrently without relying on a predefined noise-to-data trajectory.

Recurrent refinement. Iterative refinement has been widely studied in parallel language generation and diffusion models ([12] (Lee et al. 2018; Ghazvininejad et al. 2019; Gu et al. 2019; Savinov et al. 2022; Wang et al. 2025b)). Recurrent refinement has also been explored in flow-based generation ((Helbling et al., 2026)). More broadly, weight-tied recurrent models leverage repeated compute to improve predictions ([4, 15, 10]).

Background: Drifting Model

The Drifting Model ([5]) learns a direct mapping fθf_{\theta} from noise ϵ∼pinit\epsilon\sim p_{\mathrm{init}} to generated samples x=fθ(ϵ)x = f_{\theta}(\epsilon). The resulting distribution qθ=(fθ)#pinitq_{\theta} = (f_{\theta})_{\#}p_{\mathrm{init}} is trained to match the target data distribution pp. Like GANs ((Goodfellow et al., 2014)), it generates samples in a single step, but learns without adversarial training. Instead, during training, a drift field computed from real and generated samples guides qθq_{\theta} toward the target distribution pp. The field combines attraction toward real samples with repulsion from generated samples:

Vp,qθ(x)=Vp+(x)−Vqθ−(x),V_{p,q_{\theta}}(x) = V_{p}^{+}(x) - V_{q_{\theta}}^{-}(x),
Vp+(x)=Ey∼p[k(x,y)(y−x)]Ey∼p[k(x,y)],V_{p}^{+}(x) = \frac{\mathbb{E}_{y\sim p}[k(x,y)(y-x)]}{\mathbb{E}_{y\sim p}[k(x,y)]},
Vqθ−(x)=Ey∼qθ[k(x,y)(y−x)]Ey∼qθ[k(x,y)].V_{q_{\theta}}^{-}(x) = \frac{\mathbb{E}_{y\sim q_{\theta}}[k(x,y)(y-x)]}{\mathbb{E}_{y\sim q_{\theta}}[k(x,y)]}.

where k(x,y)=exp⁡(−∥x−y∥/τ)k(x,y)=\exp(-\lVert x-y\rVert/\tau) is a kernel with bandwidth τ\tau.

In practice, the expectations are estimated from mini-batches of real (positive) and generated (negative) samples. The generator learns to move its outputs along the drift field through a stop-gradient regression loss Ldrift(θ)=Eϵ∼pinit∥fθ(ϵ)−sg⁡(fθ(ϵ)+Vp,qθ(fθ(ϵ)))∥2\mathcal{L}_{\mathrm{drift}}(\theta) = \mathbb{E}_{\epsilon\sim p_{\mathrm{init}}}\left\lVert f_{\theta}(\epsilon) - \operatorname{sg}\left(f_{\theta}(\epsilon) + V_{p,q_{\theta}}(f_{\theta}(\epsilon))\right)\right\rVert^{2}, where sg⁡\operatorname{sg} denotes stop-gradient. The drift field guides training but is not needed at inference time. We follow the original model’s distance and field normalization and multi-temperature kernels, with implementation-specific adjustments detailed in Appendix § 7.§ 3.

Methods

ReLaDrift applies one Drifting Model recurrently: the first round transports noise to a first draft, as a standard Drifting Model does, and each later round feeds the current draft back into the same model and returns a corrected draft (Figure 2). The number of rounds at test time can then be chosen freely. In the following, we consider conditional generation where a problem statement is given.

Figure 2. ReLaDrift in the source formulation. Round 0 (Initial) transports the noise ϵ\epsilon to the first draft d1d_1 through the input projection WnoiseW_{\mathrm{noise}}, and every later round (Refine) transports the current draft through WdraftW_{\mathrm{draft}}; all rounds share the network fθf_\theta and receive the prompt latent PP and the round index rr (§ 4.2). The frozen decoder is applied once to the last draft dNd_N; at test time, the number of rounds NN is the NFE and may exceed the training depth Rmax⁡R_{\max} (§ 4.4). The marks sg and the loss L\mathcal{L} apply only during training, where N=R∼Uniform⁡{1,…,Rmax⁡}N = R \sim\operatorname{Uniform}\{1,\ldots,R_{\max}\} and the drafts that enter a round are treated as constants, so the loss backpropagates through the terminal round only (§ 4.3). The positives and the negatives of the loss are not shown.

Continuous Representation of Token Sequences

Language is discrete, so we first embed each token sequence in a continuous space where the Drifting Model can operate. A fixed encoder maps a token sequence s=[s1,…,sℓ]s = [s_1,\ldots,s_\ell] to one DembD_{\mathrm{emb}}-dimensional vector per position, Encoder⁡(s)∈Rℓ×Demb\operatorname{Encoder}(s) \in\mathbb{R}^{\ell\times D_{\mathrm{emb}}}. The encoder is not trained with the model, as in latent generation with frozen pretrained encoders ((Lovelace et al., 2023; Shabalin et al., 2025; Hu et al., 2026)), and any map from token sequences to a continuous space can be used in principle; the choice matters (§ 5.2).

A solution in the training data is represented by a positive y∈RL×Demby \in\mathbb{R}^{L \times D_{\mathrm{emb}}}, the encoding of the token sequence at the LL positions that the model outputs (§ 5; Appendix § 7.§ 1). Only the positions of the solution are generated: the noise, the draft, the drift field, and the loss are restricted to these generated positions.

The output of the model is continuous, so a decoder maps it back to tokens. The decoder is either given by the encoder or trained separately in advance, and is frozen while fθf_\theta is trained (Appendix § 7.8). It predicts every token from the latent sequence in parallel, so decoding adds one pass and does not reintroduce sequential generation.

Recurrent Application of the Drifting Model

ReLaDrift applies the network fθf_\theta of the Drifting Model (§ 3) repeatedly: each application is a round, and its output is a draft. Round r=0r = 0 produces the first draft d1∈RL×Dembd_1 \in\mathbb{R}^{L \times D_{\mathrm{emb}}} from noise, and each later round r≥1r \ge1 takes the previous draft drd_r and produces the next draft dr+1d_{r+1}. Thus NN rounds are r=0,…,N−1r = 0,\ldots,N-1, and round rr returns dr+1d_{r+1}. Every round also receives the prompt latent PP and the round index rr. All rounds use the same network: the round index selects the input and output projections of the first round or of the refinement rounds (see below), so the refinement rounds r≥1r \ge1 share all their parameters. Drafts are passed between rounds as continuous latents; they are not decoded into tokens and re-encoded (Figure 2).

The noise and the draft enter fθf_\theta through separate input projections, and we consider three ways to feed the draft back. In the source formulation, a round has a single input: the first round transports noise, and each later round transports the previous draft, which replaces the noise as the input of the round but keeps its own projection. In the additive formulation, every round receives fresh noise and the draft, whose projections are added; in the conditional formulation, every round receives fresh noise, and the draft is appended to the input sequence as extra tokens. The additive and conditional formulations therefore need a fixed initial draft d0d_0 at the first round, whereas the source formulation mulation does not. The source formulation is used in all main experiments, and the three formulations are compared on Sudoku (§ 5.1).

Source formulation. A round has a single input xx, the noise at r=0r = 0 and the previous draft at r≥1r \ge1, written fθ(x,P,r)f_\theta(x, P, r). Inside fθf_\theta, the projection selected by the round index, WnoiseW_{\mathrm{noise}} at r=0r = 0 and WdraftW_{\mathrm{draft}} at r≥1r \ge1, maps the input to the model dimension, giving the input uru_r of the bidirectional Transformer at the LL generated positions:

d1=fθ(ϵ,P,0),u0=ϵWnoise,dr+1=fθ(dr,P,r),ur=drWdraft,\begin{aligned} d_1 &= f_\theta(\epsilon, P, 0), \qquad u_0 = \epsilon W_{\mathrm{noise}},\\ d_{r+1} &= f_\theta(d_r, P, r), \qquad u_r = d_r W_{\mathrm{draft}}, \end{aligned}

for r≥1r \ge1, where ϵ∼pinit=N(0,I)\epsilon\sim p_{\mathrm{init}} = \mathcal{N}(0, I) and Wnoise,Wdraft∈RDemb×DmodelW_{\mathrm{noise}}, W_{\mathrm{draft}} \in\mathbb{R}^{D_{\mathrm{emb}} \times D_{\mathrm{model}}}. The noise and the draft thus never share a projection, and the projection that a round does not select has no effect on that round, so the gradient of that round’s loss with respect to it is zero. For a fixed prompt and round r≥1r \ge1, x↦fθ(x,P,r)x \mapsto f_\theta(x, P, r) maps RL×Demb\mathbb{R}^{L \times D_{\mathrm{emb}}} to itself, which allows the output of one round to be the input of the next. No new noise is drawn after the first round, so the refinement path at test time is deterministic once the first draft is fixed.

Additive and conditional formulations. Both keep the original form of the Drifting Model, which transports fresh noise at every round, and condition it on the previous draft, so a round has two inputs: dr+1=fθ(ϵr,dr,P,r)d_{r+1} = f_\theta(\epsilon_r, d_r, P, r) with ϵr∼pinit\epsilon_r \sim p_{\mathrm{init}} for r≥0r \ge0, starting from a fixed initial draft d0d_0 (Appendix § 7.§ 2); the refinement path thus remains stochastic. In the additive formulation, the projected draft is added to the projected noise at every generated position, so that ur=ϵrWnoise+drWdraftu_r = \epsilon_r W_{\mathrm{noise}} + d_r W_{\mathrm{draft}}, so the draft enters the model at the same positions as the noise. In the conditional formulation, the projected draft is instead appended to the input sequence as LL separate tokens, so that ur=ϵrWnoiseu_r = \epsilon_r W_{\mathrm{noise}} and every position of the current generation can attend to every position of the draft without mixing the two, at the cost of a longer attention sequence.

Shared components. The three formulations differ only in how the noise and the draft enter the Transformer; the rest is shared. In every formulation, the projected prompt latent PWpromptP W_{\mathrm{prompt}} and the projected round embedding ϕ(r)Wround\phi(r) W_{\mathrm{round}} are also given to the Transformer, and its output at the generated positions is projected linearly back to DembD_{\mathrm{emb}}, with different output projections for the first round (r=0r = 0) and the refinement rounds (r≥1r \ge1). The round embedding ϕ(r)\phi(r), a fixed vector of features of rr without learned parameters, lets the model distinguish the rounds, whose input distributions differ (in the source formulation, Gaussian noise at the first round and model-produced drafts later). Appendix § 7.§ 2 gives the exact form of all these components, including the placement of the prompt latent in the input sequence for each task.

Training

Loss at one round. Each round is trained with the loss of the Drifting Model (§ 3), with the positives of a problem as samples of the target distribution pp and the drafts that the round generates for the problem as samples of the generated distribution qq. For the positives y1,…,yN+y_1,\ldots,y_{N_+} and the nn drafts x1,…,xnx_1,\ldots,x_n of a problem, which we call its generated samples, the field and the loss are

Vj=Vj+−Vj−,V_j = V_j^+ - V_j^-,
L(θ)=1n∑j=1n∥xj−sg⁡(xj+Vj)∥G2,\mathcal{L}(\theta) = \frac{1}{n}\sum_{j=1}^{n}\left\lVert x_j - \operatorname{sg}(x_j + V_j) \right\rVert_G^2,

where sg⁡\operatorname{sg} denotes the stop-gradient operation and ∥⋅∥G\lVert\cdot\rVert_G the Euclidean norm over the generated positions (§ 4.1). The attractive drift Vj+V_j^+ is the kernel-weighted barycenter of the positives minus xjx_j, which is y1−xjy_1-x_j with a single positive; the repulsive drift Vj−V_j^- is the same quantity for the other generated samples of the same problem, which serve as its negatives as in the original Drifting Model, and enters with a minus sign, so that it pushes xjx_j away from them. The weights of the two drifts (λ+,λ−\lambda^+,\lambda^-), are set in § 5, and Appendix § 7.§ 3 gives the exact form of both terms.

Training through rounds without backpropagation through time. Each update samples the number of rounds and trains only the terminal round, running the earlier rounds with the gradient stopped (marked sg in Figure 2), so that every round is trained without backpropagation through time. Sampling the terminal round allows the shared network to be trained at every depth across updates. Applying the loss at all rounds in each update would require gradient computation at every round. For each problem, we sample the number of rounds R∼Uniform⁡{1,…,Rmax⁡}R \sim\operatorname{Uniform}\{1,\ldots,R_{\max}\}, where the training depth Rmax⁡R_{\max} is the largest number of rounds used in training. The model runs the rounds r=0,…,R−1r = 0,\ldots,R-1, the last of which, round R−1R-1, is the terminal round. In the source formulation, each generated sample xjx_j comes from its own noise ϵj∼pinit\epsilon_j \sim p_{\mathrm{init}} through the rounds

d1=sg⁡(fθ(ϵj,P,0)),d_1 = \operatorname{sg}(f_\theta(\epsilon_j, P, 0)),
dr+1=sg⁡(fθ(dr,P,r))(r=1,…,R−2),d_{r+1} = \operatorname{sg}(f_\theta(d_r, P, r)) \qquad(r = 1, \ldots, R - 2),
xj=fθ(dR−1,P,R−1),x_j = f_\theta(d_{R-1}, P, R - 1),

with xj=fθ(ϵj,P,0)x_j = f_\theta(\epsilon_j, P, 0) when R=1R = 1. Only the terminal round’s activations are stored, whatever RR is, and the earlier rounds cost only forward passes (Appendix § 7.11). Because the terminal round is sampled, every refinement round r=1,…,Rmax⁡−1r = 1, \ldots, R_{\max} - 1 is trained on the drafts that the current model itself produces for it, as in training on model-generated inputs ([17] (Savinov et al. 2022)), and the first round is trained on noise, as the Drifting Model is.

The nn generated samples of a problem branch at the first round in the source formulation and at the terminal round in the additive and conditional formulations, so that they are different drafts. In the additive and conditional formulations, the recurrence is the same with fθ(dr,P,r)f_\theta(d_r, P, r) replaced by fθ(ϵr,dr,P,r)f_\theta(\epsilon_r, d_r, P, r), where a new noise ϵr∼pinit\epsilon_r \sim p_{\mathrm{init}} is drawn at every round rr and the trajectory starts from the initial draft d0d_0; one such trajectory is shared by the generated samples up to round R−2R - 2, and nn independent noises at the terminal round give the nn generated samples. Appendix § 7.§ 5 describes one training step in detail (Algorithm 1).

Generation and Test-Time Scaling

At test time, the number of rounds, the NFE NN, is chosen freely and is independent of the number of rounds RR and the training depth Rmax⁡R_{\max} used in training. Generation with NFE =N= N runs the rounds r=0,…,N−1r = 0, \ldots, N - 1: the prompt is encoded once, the draft is updated NN times with the same model fθf_\theta, and the decoder is applied once to the final draft (Algorithm 3 in Appendix § 7.7).

Rounds beyond the training depth. Generation can run more rounds than the training depth: every refinement round (r≥1r \ge1) is trained to correct the draft that it receives, so repeating this correction beyond Rmax⁡R_{\max} is expected to keep improving the draft; § 5 tests this expectation. The round embedding ϕ(r)\phi(r), however, is seen in training only for r<Rmax⁡r < R_{\max}, so we clamp it to the last trained round,

ϕ(r)→ϕ(min⁡(r,Rmax⁡−1));\phi(r) \to\phi\left(\min(r, R_{\max} - 1)\right);

since all refinement rounds share every parameter of fθf_\theta, each additional round then applies exactly the map trained at round Rmax⁡−1R_{\max} - 1, and the number of rounds becomes a pure test-time choice. The clamp is part of the model definition; Appendix § 7.13 reports the results without it.

Experiments

We evaluate how ReLaDrift scales with inference-time computation on Sudoku Hard (§ 5.1) and GSM8K (§ 5.2), comparing it with an iterated one-step Drifting Model and established diffusion and flow language models. We train all compared methods on the same task-specific data, largely following the setups of S-FLM (Deschenaux and Gulcere, [7]). Differences from the published setups and comparisons with reported baseline results are detailed in Appendix § 7.10.

Evaluation protocol. We measure accuracy across a range of inference budgets, using the number of function evaluations (NFE) as a common measure while keeping model weights fixed. For ReLaDrift, each round counts as one NFE, with the prompt encoded once and only the final draft decoded. For other parallel generators, NFE is the number of ODE or sampling steps (S-FLM, Duo, MDLM, and FRM) or of refinement rounds (FMLM+); the one-step Drifting Model is iterated by decoding and re-encoding its outputs between rounds (Appendix § 7.15). We evaluate a single generation trajectory per problem using EMA weights, without inference-time verification or candidate selection; GSM8K programs are executed only to compute accuracy. Since NFE does not capture differences in per-step cost, we also compare wall-clock generation time on GSM8K.

Training configuration. Unless otherwise stated, we use n=2n = 2 generated samples per problem to reduce the cost of recurrent training and, on GSM8K, verification. To stabilize training with few generated samples, we weight the attractive and repulsive drifts as λ+V+−λ−V−\lambda^{+}V^{+} - \lambda^{-}V^{-} with (λ+,λ−)=(2,1)(\lambda^{+}, \lambda^{-}) = (2, 1), strengthening the attraction toward target samples. We analyze these choices in Appendix § 7.§ 4 and their computational costs in Appendix § 7.11.

Sudoku

Experimental setup. We evaluate ReLaDrift on Sudoku Hard from the S-FLM benchmark (Deschenaux and Gulcere, [7]), comprising 48,000 training and 2,000 validation puzzles generated using Alp [13], each with a unique solution and 30 of 81 digits visible. Models generate complete solutions, evaluated by exact match (Appendix § 7.§ 1). We compare against AR, MDLM ((Sahoo et al., 2024)), Duo ((Sahoo et al., 2025)), S-FLM, FMLM+ ([12]), FRM ((Helbling et al., 2026)), and DFM ((Ma et al., 2026)), all trained on the same data for 20,000 updates following the S-FLM optimization settings. We reimplement FRM and DFM for this benchmark. All models have comparable sizes (25.2–28.6M parameters; 28.4M for ReLaDrift). ReLaDrift uses one-hot representations with arg max decoding, without a pretrained encoder or verifier. We also train a one-step Drifting Model with matched architecture, loss, data, and training budget to see whether naive repetition alone yields test-time scaling. Further details are in Appendix § 7.10.

MethodNFE=1458163264128180
AR (sequential)Greedy: 0.5 ± 0.1; sampling: 0.5 ± 0.2
MDLM0.6 ± 0.28.1 ± 1.011.6 ± 1.615.9 ± 1.922.7 ± 2.627.1 ± 2.130.1 ± 2.232.2 ± 1.632.6 ± 2.6
Duo4.0 ± 2.155.2 ± 6.756.8 ± 5.660.6 ± 5.461.9 ± 4.962.9 ± 4.761.8 ± 4.862.9 ± 4.861.5 ± 4.9
S-FLM6.7 ± 1.133.0 ± 2.934.9 ± 3.238.0 ± 3.040.6 ± 3.241.9 ± 3.242.5 ± 3.443.0 ± 3.343.2 ± 3.3
FMLM+2.4 ± 0.467.0 ± 2.471.2 ± 1.576.8 ± 1.477.5 ± 2.178.5 ± 1.478.1 ± 2.078.5 ± 1.577.2 ± 1.4
FRM (Euler)6.4 ± 0.954.9 ± 1.060.6 ± 0.866.7 ± 1.171.8 ± 1.574.2 ± 1.675.2 ± 1.775.8 ± 1.775.9 ± 1.6
FRM (SDE)6.4 ± 0.955.5 ± 0.661.2 ± 1.167.3 ± 0.972.6 ± 1.375.1 ± 1.677.0 ± 2.178.0 ± 2.779.4 ± 2.0
FRM (held-time)6.4 ± 0.961.0 ± 2.565.3 ± 1.470.6 ± 1.573.5 ± 1.774.4 ± 1.574.5 ± 1.574.5 ± 1.574.5 ± 1.5
DFM6.3 ± 4.525.8 ± 7.226.2 ± 7.027.9 ± 6.829.2 ± 6.529.6 ± 6.530.1 ± 6.330.3 ± 6.430.4 ± 6.4
Drifting Model5.1 ± 0.84.9 ± 0.85.0 ± 0.84.9 ± 0.84.9 ± 0.84.9 ± 0.84.9 ± 0.84.9 ± 0.84.9 ± 0.8
ReLaDrift (ours)7.3 ± 1.078.9 ± 0.682.5 ± 0.888.4 ± 0.492.1 ± 0.493.1 ± 0.693.5 ± 0.793.9 ± 0.794.0 ± 0.7

Table 1. Sudoku Hard accuracy (%) across inference budgets. Exact-match accuracy on 2k validation puzzles after 20k training updates (mean ± standard deviation over three seeds). ReLaDrift uses the Source formulation with Rmax⁡=5R_{\max}=5. The Drifting Model is iterated by decoding and re-encoding its output between rounds (Appendix § 7.15). FRM and DFM are our implementations (Appendix § 7.10). Bold indicates the best accuracy at each NFE.

Main Results ReLaDrift exhibits strong test-time scaling on Sudoku Hard, whereas simply iterating the one-step Drifting Model yields no improvement (Table 1 and Figure 1, top). Despite being trained with at most Rmax⁡=5R_{\max}=5 rounds, its accuracy improves monotonically with NFE across all three seeds, reaching 94.0% at NFE=180. ReLaDrift outperforms all compared diffusion and flow language models across NFEs. These results establish recurrent drifting as an effective means of trading inference-time computation for accuracy.

Ablations. Table 2 examines the input formulation (Source, Additive, Conditional) and training depth Rmax⁡R_{\max}. All three formulations exhibit strong test-time scaling, suggesting that recurrent drifting is robust to how previous drafts are incorporated. Among them, Source (default) achieves the highest accuracy, with the lowest seed variability at NFE ≥8\ge8. Increasing the training depth further improves multi-round accuracy (NFE ≥4\ge4), even beyond the training depth Rmax⁡=5R_{\max}=5. This improvement comes at the cost of additional forward passes during training, while backpropagation remains limited to the terminal round (§ 4, Appendix § 7.11).

SettingNFE=1458163264128180
Default7.3 ± 1.078.9 ± 0.682.5 ± 0.888.4 ± 0.492.1 ± 0.493.1 ± 0.693.5 ± 0.793.9 ± 0.794.0 ± 0.7
Input formulation (Default: Source)
- Additive7.1 ± 2.277.6 ± 2.681.5 ± 3.186.2 ± 3.689.5 ± 3.190.5 ± 3.091.0 ± 2.891.6 ± 2.591.6 ± 2.8
- Conditional7.4 ± 0.977.6 ± 1.781.1 ± 1.186.1 ± 1.289.6 ± 1.890.7 ± 1.791.2 ± 2.091.6 ± 2.391.7 ± 2.2
Training depth Rmax⁡R_{\max} (Default: 5)
- 35.7 ± 2.766.2 ± 3.468.5 ± 2.971.4 ± 2.172.6 ± 2.072.9 ± 1.973.1 ± 1.973.1 ± 2.073.2 ± 2.0
- 86.4 ± 2.083.4 ± 1.786.9 ± 1.791.2 ± 1.794.9 ± 1.396.0 ± 1.096.6 ± 1.096.8 ± 0.996.9 ± 0.8
- 164.2 ± 2.185.2 ± 2.988.8 ± 2.193.0 ± 1.796.1 ± 0.897.2 ± 0.597.4 ± 0.497.5 ± 0.497.6 ± 0.4

Table 2. Ablation of ReLaDrift on Sudoku Hard. Exact-match accuracy (%; mean ± standard deviation over three seeds) under the settings of Table 1. The default Source formulation uses Rmax⁡=5R_{\max}=5 and serves as the reference for both the input-formulation and training-depth comparisons.

GSM8K

General setup. Following S-FLM ([7]), we train on TinyGSM ((Liu et al., 2023)), a dataset of synthetic math problems paired with Python solutions, and evaluate on the 1,319 GSM8K test problems ([3]).

Accuracy is measured by executing one generated program per problem and comparing its output with the ground-truth answer. To provide multiple reference solutions per problem, we use 1,588,781 TinyGSM problems, each paired with 12 programs generated by Qwen Team (2026), without filtering for correctness (Appendix § 7.9). All methods use the same problems; the compared methods train on all 12 programs, while ReLaDrift uses the first K∈{1,4,12}K \in\{1, 4, 12\} as its given positives.

Models and training. We compare with AR, MDLM, Duo, S-FLM, FMLM+, and a separately trained one-step Drifting Model. All methods are trained for 250,000 updates with batch size 512, following the S-FLM optimization settings. The diffusion and flow baselines use about 160M parameters, while ReLaDrift uses 164.9M in total, including a frozen CodeT5-small encoder ((Wang et al., 2021)) and a separately trained, frozen decoder. ReLaDrift operates in the encoder’s contextualized latent space, with the decoder mapping the final draft to tokens in parallel. The one-step Drifting Model uses the same representation, drifting objective, verifier, and training settings as ReLaDrift, but without recurrent refinement. AR, S-FLM, Duo, and MDLM use sampling temperature T=0.1T = 0.1, and FMLM+ uses arg max decoding. Implementation details are in Appendix § 7.10.

Training with multiple valid solutions. Unlike Sudoku, GSM8K admits multiple correct programs for each problem. Because the drifting objective attracts generated samples toward the given positives, valid programs not represented by these references may be pulled toward different solutions. To address this, we augment the KK given

positives with model-generated programs that pass execution-based verification. The verifier compares their outputs against a pseudo-reference obtained by voting over the answers of the KK given programs, rather than using the ground-truth answer. Verified programs enter a per-problem positive queue with up to four additional entries and are excluded from the repulsion applied to other generated samples. Verification and the queue are used only during training (Appendix § 7.§ 6).

Main results. ReLaDrift also achieves test-time scaling on GSM8K, demonstrating that recurrent drifting benefits from additional inference rounds even with multiple valid solutions (Table 3 and Figure 1, bottom). Unlike the one-step Drifting Model, which does not improve when iterated, ReLaDrift’s accuracy increases from 13.2% at NFE=1 to 29.4% at NFE=16, even beyond its training depth Rmax⁡=5R_{\max} = 5. Compared with established diffusion and flow language models, ReLaDrift achieves the highest accuracy at matched NFE up to 16 and remains competitive at larger inference budgets, where its accuracy eventually saturates. Together with the Sudoku results, these findings position recurrent drifting as a competitive approach to test-time-scalable parallel language generation.

MethodNFE14581632641282565121024
AR (sequential)72.3
MDLM0.31.31.95.212.118.827.228.729.732.032.1
Duo0.55.26.812.523.130.634.734.936.337.135.9
S-FLM0.510.513.316.217.719.920.220.520.320.420.8
FMLM+0.02.64.910.120.524.926.025.625.525.525.5
Drifting Model8.47.87.77.77.77.77.77.77.77.77.7
ReLaDrift13.227.327.429.029.429.429.429.529.529.429.4

Table 3. GSM8K test accuracy (%) across inference budgets. Accuracy on 1,319 test problems after 250,000 training updates (single training seed; one generated program per problem). ReLaDrift uses K=12K = 12 given positives and Rmax⁡=5R_{\max} = 5. The Drifting Model is iterated by decoding and re-encoding its output between rounds (Appendix § 7.15). AR, MDLM, Duo, and S-FLM use T=0.1T = 0.1; FMLM+ uses arg max. Bold indicates the best parallel generator at each NFE.

Inference efficiency. ReLaDrift is also more efficient per function evaluation, taking 24 ms per round versus 95 ms per step for Duo on a single H200 GPU (Appendix A.11). Consequently, ReLaDrift achieves 29.4% accuracy in just 13 seconds, whereas Duo requires 63 seconds to reach a comparable accuracy of 30.6% (Figure 3). This advantage is more pronounced in wall-clock time than in matched-NFE comparisons.

Figure 3. GSM8K test accuracy versus the wall-clock time of generating the 1,319 test programs on one H200 (log scale; batches of 64 problems; settings of Table 3). The labels are the NFE; the time of ReLaDrift includes the encoder and the decoder.

Ablations. Unlike Sudoku, GSM8K involves multiple valid solutions and requires additional design choices for training in a contextualized latent space. We therefore examine three GSM8K-specific factors: the number of given positives KK, training-time verification, and the encoder representation (Table 4). Increasing KK substantially improves accuracy, while the verifier and positive queue and contextualized CodeT5 embeddings provide further gains. Importantly, test-time scaling persists even without these components, indicating that they improve accuracy rather than enable recurrent refinement. Notably, unlike Sudoku, where increasing the training depth Rmax⁡R_{\max} improves multi-round accuracy, GSM8K performs best at Rmax⁡=5R_{\max} = 5, although both tasks benefit from inference rounds beyond their training depths. Increasing Rmax⁡R_{\max} exposes the model to longer refinement trajectories but reduces the frequency of supervision at each depth under a fixed training budget, suggesting that the balance between these effects is task-dependent. Further ablations of the input formulation, drift-field coefficients, and round-embedding clamp are in Appendices A.14 and A.13.

SettingNFE
14816
Default13.227.329.029.4
Number of given positives KK (default 12)- 410.422.724.424.9
- 11.08.710.712.2
Training depth Rmax⁡R_{\max} (default 5)- 810.224.226.227.7
- 168.024.026.827.2
Training components
- Without verifier and queue4.517.018.520.0
Encoder (Default: contextualized CodeT5.)
- Uncontextualized CodeT5.6.715.116.717.7
- Random emb. table1.45.56.77.1

Table 4. Ablation of ReLaDrift on GSM8K. Test accuracy (%) after 250,000 training updates. The default uses K=12K = 12, Rmax⁡=5R_{\max} = 5, a contextualized CodeT5 encoder, and training-time verification with a positive queue. Embedding-table variants replace the contextualized encoder with context-free representations. Additional settings and results across all NFE are provided in Appendix A.14.

Conclusion

We introduced ReLaDrift, which turns the one-step Drifting Model into a recurrent language generator by iteratively refining its own drafts in a continuous latent space. The model learns refinement by applying the drifting loss at a sampled terminal round, avoiding backpropagation through time. Unlike the original Drifting Model, which does not improve when repeatedly applied, ReLaDrift exhibits test-time scaling on Sudoku Hard and GSM8K, enabling the computation–quality trade-off of established diffusion and flow language models. It outperforms these models across NFE=4–180 on Sudoku Hard and up to NFE=16 on GSM8K, positioning drifting-based generation as a promising alternative for test-time-scalable parallel language generation.

Discussion ReLaDrift incurs approximately 3.3× the per-update training cost of a one-pass method (Appendix A.11). This overhead reflects both recurrent forward passes and a broader challenge inherent to Drifting Models: estimating the drift field requires multiple generated samples, each involving additional model evaluations. We limit this cost by using only n=2n = 2 generated samples per problem. Improving the computational efficiency of drift-field estimation is an important research direction in its own right, beyond the scope of this work, which focuses on enabling test-time scaling for drifting-based generation.

Limitations

On GSM8K, our advantage over parallel generators depends on a verifier and pretrained CodeT5 embeddings, while test-time scaling persists without these components (§ 5.2). The verifier requires executable outputs and relies on voted pseudo-references that may be incorrect. Furthermore, GSM8K results use a single training seed due to limited training resources. Evaluating broader tasks and model scales, improving verifier-free training with multiple valid targets, and narrowing the gap to autoregressive models remain future work.

Acknowledgments

This research was conducted by AIST and Institute of Science Tokyo under FRONTia, a Japanese national program led by the Ministry of Economy, Trade and Industry of Japan (METI) and the New Energy and Industrial Technology Development Organization (NEDO), aimed at developing a domestic multimodal foundation model for AI robots and physical AI.

Appendix

Encoding of the Prompt and the Target

The prompt and the target are encoded in two different ways. The target latent is obtained by encoding the whole sequence, that is, the problem statement followed by the solution, as one sequence. With a contextual encoder, the latent at every solution position is thus contextualized on the problem statement; with a per-token code such as the one-hot code of the Sudoku experiments, the two encodings coincide position by position. The prompt latent PP is obtained by encoding the problem statement alone. The two tasks lay out the prompt and the solution differently.

GSM8K. We use the frozen encoder of CodeT5-small ((Wang et al., 2021)) (35.3M parameters), whose contextualized embeddings have 512 dimensions, as the T5-small encoder of ELF ((Hu et al., 2026)) does. We use CodeT5 because its tokenizer is designed for source code; Wang et al. (2021) report that the default T5 tokenizer maps code symbols such as {\{ and }\} to unknown tokens. In our own check of the T5-small tokenizer (google-t5/t5-small), newlines and indentation were also lost and {\{, }\}, and << became unknown tokens, whereas the CodeT5 tokenizer (Salesforce/codet5-small) reproduced the same Python snippets exactly. Each example is tokenized with the CodeT5 tokenizer as BOS, the problem statement, a newline, the program, and EOS, and padded to L=576L = 576 tokens; the prompt consists of BOS, the problem statement, and the newline. The target latent is the output of the frozen CodeT5 encoder for the whole sequence, in which the padding tokens are hidden from the attention keys but still receive an output, so it covers all 576 positions, including the prompt positions and the padding. The prompt latent P∈R576×512P \in\mathbb{R}^{576\times512} is the output of the encoder for the prompt alone, placed at the prompt positions, with zeros at all later positions. The model outputs all 576 positions, but the noise and the draft enter the model only at the positions after the prompt (the program, EOS, and the padding), which are the positions where the prompt latent is zero, and the drift field and the loss use only these positions; the outputs at the prompt positions are never compared with a target and are discarded, so the next round receives the draft only at the positions after the prompt (Appendix § 7.§ 2). At test time, the problem statement is encoded in the same way, and the generated program is read from the positions after the prompt up to the first EOS.

Both latents are standardized with fixed scalar constants before they enter the model:

z←z−μσ,μ=0,σ=0.3.z \leftarrow\frac{z-\mu}{\sigma}, \qquad\mu= 0, \qquad\sigma= 0.3.

The same constants are used for the prompt latent, the target latent, and the continuous output of the model.

Sudoku. The sequence is the 180-token format of S-FLM: BOS, the 89-token puzzle (81 cells, where an empty cell is the blank token, and 8 row separators, BOS, and the 89-token answer, which is the solution grid with the same separators. Each of the 12 tokens is encoded by a row of the 12×1212 \times12 identity matrix (Demb=12D_{\mathrm{emb}} = 12), without standardization. The model generates only the 89 answer positions (L=89L = 89): the target latent is the code of the 89 answer tokens, and the prompt latent P∈R91×12P \in\mathbb{R}^{91 \times12} is the code of the first 91 tokens. Because the code of a token does not depend on its context, the puzzle does not need to be part of the target. The drift field and the loss use all 89 answer positions, including the row separators. All 89 answer tokens, including the given digits and the row separators, are generated, and the metric is their exact match; the given digits are not overwritten, and decoding is not restricted to digits.

We also experiment with context-free embedding tables on GSM8K (the CodeT5 input embedding table and a random Gaussian table); these results are reported in Appendix § 7.14.

Model Architecture and Input Layout

Input of the source formulation. Inside fθf_\theta, the inputs are projected to the model dimension and summed into the initial hidden representation

hr={ϵ0Wnoise+PWprompt+1Lϕ(0)Wround,r=0,drWdraft+PWprompt+1Lϕ(r)Wround,r≥1.h_r = \begin{cases} \epsilon_0 W_{\mathrm{noise}} + P W_{\mathrm{prompt}} + \mathbf{1}_L \phi(0) W_{\mathrm{round}}, & r = 0, \\ d_r W_{\mathrm{draft}} + P W_{\mathrm{prompt}} + \mathbf{1}_L \phi(r) W_{\mathrm{round}}, & r \ge1. \end{cases}

The projection matrices Wnoise,Wprompt,Wdraft∈RDemb×DmodelW_{\mathrm{noise}}, W_{\mathrm{prompt}}, W_{\mathrm{draft}} \in\mathbb{R}^{D_{\mathrm{emb}} \times D_{\mathrm{model}}} and Wround∈R10×DmodelW_{\mathrm{round}} \in\mathbb{R}^{10 \times D_{\mathrm{model}}} have no bias, and 1L\mathbf{1}_L is a column vector of length LL whose entries are all 1, so that the round term is added at every position. In the GSM8K experiments, PP, ϵ0\epsilon_0, and drd_r all lie in R576×512\mathbb{R}^{576 \times512}, and the terms occupy disjoint positions: before the projection, the noise and the draft are multiplied by the indicator of the positions where PP is zero (the positions after the prompt), so the prompt positions receive only the prompt latent and the later positions receive only the noise or the draft; thus hr∈R576×Dmodelh_r \in\mathbb{R}^{576 \times D_{\mathrm{model}}}. In the Sudoku experiments, the noise and the draft cover only the 89 answer positions, so the prompt latent is placed before them instead of being added to them:

hr=cat⁡seq(PWprompt,xr)+1180ϕ(r)Wround,h_r = \operatorname{cat}_{\mathrm{seq}}(P W_{\mathrm{prompt}}, x_r) + \mathbf{1}_{180} \phi(r) W_{\mathrm{round}},
xr={ϵ0Wnoise,r=0,drWdraft,r≥1,x_r = \begin{cases} \epsilon_0 W_{\mathrm{noise}}, & r = 0, \\ d_r W_{\mathrm{draft}}, & r \ge1, \end{cases}

with P∈R91×12P \in\mathbb{R}^{91 \times12} and ϵ0,dr∈R89×12\epsilon_0, d_r \in\mathbb{R}^{89 \times12}, so that hr∈R180×Dmodelh_r \in\mathbb{R}^{180 \times D_{\mathrm{model}}}; fθf_\theta returns the outputs at the last 89 positions.

Round embedding. To provide the model with the current round, we use the round embedding ϕ(r)\phi(r), where rr is the round index starting from 0. Specifically, we use

ϕ(r)=(1[r=0],r1+r,sin⁡(πr),sin⁡(πr/2),sin⁡(πr/4),sin⁡(πr/8),cos⁡(πr),cos⁡(πr/2),cos⁡(πr/4),cos⁡(πr/8)).\phi(r) = \left(\mathbf{1}_{[r = 0]}, \frac{r}{1+r}, \sin(\pi r), \sin(\pi r/2), \sin(\pi r/4), \sin(\pi r/8), \cos(\pi r), \cos(\pi r/2), \cos(\pi r/4), \cos(\pi r/8)\right).

1[r=0]\mathbf{1}_{[r = 0]} is 1 if and only if r=0r = 0, and 0 otherwise. Since rr is an integer, sin⁡(πr)=0\sin(\pi r) = 0 for every round; we keep this constant coordinate so that the sine and cosine terms use the same four frequencies.

Backbone. We feed hrh_r into gθg_\theta, a stack of bidirectional pre-normalization Transformer blocks followed by a final normalization layer. No normalization is applied to hrh_r itself before the first block. On both tasks, each block applies a scale-only LayerNorm before the attention and before the feed-forward network, the feed-forward network is a GELU network, and the final layer is a scale-only LayerNorm, following the compared methods; on GSM8K, the blocks are the DiT blocks of the S-FLM implementation (those of its autoregressive model, without adaLN), made bidirectional, with bias-free attention projections and a feed-forward network of width 3,072 with biases (Table 7). Positional information is provided by RoPE ((Su et al., 2021)) applied to the queries and keys inside every attention layer of gθg_\theta, as in the backbones of the compared methods. We do not add a positional embedding to hrh_r. No attention mask is used; on GSM8K, the padding positions take part in the attention like the other positions. The sizes of gθg_\theta are given in Appendix § 7.10.

ItemReLaDrift (ours)ARS-FLMFMLM+
Positivesfirst KK of the 12 solutions (fixed) + up to 4 verified samples (positive queue)12 (problem, solution) pairs per problem, shuffledsamesame
Tokenizer, lengthCodeT5 (vocabulary 32,100), 576SmolLM-135M (vocabulary 49,153), 512samesame
LatentCodeT5 contextualized embedding, 512 dim (frozen)discrete tokenslearned embedding on the sphereone-hot + Gaussian noise
TransformerDiT blocks of S-FLM (bidirectional, no adaLN), 12 layers, width 768, 12 heads, FFN 3,072, RoPE, round embedding ϕ(r)\phi(r)DiT, 12 layers, width 768, 12 heads (causal)S-arch, 12 layers, width 768, 12 headsDiT, 12 layers, width 768, 12 heads, two-time adaLN
Parameterscore 87.4M (+ frozen encoder 35.3M, decoder 42.2M = 164.9M)160.5M163.0M168.0M
OptimizerAdamW β=(0.9,0.999)\beta= (0.9, 0.999), weight decay 0, clip 1 (same as the compared methods)AdamW β=(0.9,0.999)\beta= (0.9, 0.999), weight decay 0 (= Adam), clip 1 (official)samesame
Dropout0.1 (terminal round only)0.10.10.1
ObjectiveSource, (λ+,λ−)=(2,1)(\lambda^{+}, \lambda^{-}) = (2, 1), temperatures 0.02/0.05/0.2, τ+=0.01×\tau^{+} = 0.01 \times median, verifier and positive queuenext-token cross-entropyofficialofficial
Verifier in trainingyes (execution of the 2 terminal samples)nonono
Forward passes in training2 trajectories of R∼U{1..5}R \sim U\{1..5\} rounds per problem, stop-gradient on intermediate rounds; decoding and execution of the 2 terminal samples112 (self-distillation)
Generation (official setting)recurrent, NFE 1–1,024, one trajectorysequential sampling (temperature 1, top-p 1; we report T=0.1T = 0.1), up to 512 tokens including the prompt1,024 ODE steps, exact velocityPosterior Refinement, 32 rounds (NFE 32), threshold 0.999
Decoding42.2M decoder, argmax per position–argmaxargmax

Table 7. GSM8K settings. All methods use the same 1,588,781 problems, batch size 512, learning rate 3e-4 after 2,500 warm-up updates (then constant), 250,000 updates, dropout 0.1, and the EMA decay min⁡(0.9999,(u+1)/(u+10))\min(0.9999,(u+1)/(u+10)) of the official S-FLM implementation. Duo (167.9M) and MDLM (167.9M) are trained with the same code base and settings as S-FLM; S-FLM, Duo, and MDLM sample with 1,024 steps in the S-FLM paper [7]. All methods use bf16 mixed precision. The single-round Drifting Model uses the ReLaDrift column without the round embedding, the draft projection, and the refinement output projection (core 86.2M parameters). Only ReLaDrift and the single-round Drifting Model use a verifier during training.

Output projection. Finally, we apply the output projection to the normalized output of gθg_\theta:

fθ(⋅,P,r)=gθ(hr)(Wout+Wr)+b0.f_\theta(\cdot, P, r)=g_\theta(h_r)(W_{\mathrm{out}}+W_r)+b_0.

Here,

Wr={Winit,r=0,Wrefine,r≥1.W_r = \begin{cases} W_{\mathrm{init}}, & r=0,\\ W_{\mathrm{refine}}, & r\ge1. \end{cases}

Thus, we use different projections for the initial generation at r=0r=0 and the refinement rounds at r≥1r\ge1. The input projections WnoiseW_{\mathrm{noise}}, WpromptW_{\mathrm{prompt}}, WdraftW_{\mathrm{draft}}, and WroundW_{\mathrm{round}} have no bias. Among the output projections, only WoutW_{\mathrm{out}} has the bias b0b_0; WinitW_{\mathrm{init}} and WrefineW_{\mathrm{refine}} are bias-free. The round index selects the input projection as it selects the output projection.

Initialization. The round projection WroundW_{\mathrm{round}} and the output projections WinitW_{\mathrm{init}} and WrefineW_{\mathrm{refine}} are initialized to zero, and WoutW_{\mathrm{out}} is drawn from N(0,0.022)\mathcal{N}(0,0.02^2) with a zero bias. On GSM8K, WnoiseW_{\mathrm{noise}} and WpromptW_{\mathrm{prompt}} are drawn independently from N(0,0.022)\mathcal{N}(0,0.02^2) and WdraftW_{\mathrm{draft}} is initialized to zero; on Sudoku, WnoiseW_{\mathrm{noise}}, WpromptW_{\mathrm{prompt}}, and WdraftW_{\mathrm{draft}} start from the same matrix drawn from N(0,0.022)\mathcal{N}(0,0.02^2). On both tasks, the Transformer blocks use the default initialization of PyTorch.

Additive formulation. The projected draft is added to the projected noise,

hr=ϵrWnoise+PWprompt+drWdraft+1Lϕ(r)Wround,h_r=\epsilon_rW_{\mathrm{noise}}+PW_{\mathrm{prompt}}+d_rW_{\mathrm{draft}}+\mathbf{1}_L\phi(r)W_{\mathrm{round}},

with the same layout for each task as above (on Sudoku, ϵrWnoise+drWdraft\epsilon_rW_{\mathrm{noise}}+d_rW_{\mathrm{draft}} occupies the 89 answer positions after PWpromptPW_{\mathrm{prompt}}); the rest of fθf_\theta is the same as in the source formulation. The initial draft d0d_0 is the all-zero latent on GSM8K and the code of the blank token at every answer position on Sudoku.

Conditional formulation. We concatenate the latent sequence hrdrafth_r^{\mathrm{draft}}, which represents the draft from the previous round, to the latent sequence hrcurh_r^{\mathrm{cur}}, which is used for the current generation:

hrcur=ϵrWnoise+PWprompt+1ϕ(r)Wround+1ecur,h_r^{\mathrm{cur}}=\epsilon_rW_{\mathrm{noise}}+PW_{\mathrm{prompt}}+\mathbf{1}\phi(r)W_{\mathrm{round}}+\mathbf{1}e_{\mathrm{cur}},
hrdraft=drWdraft+1ϕ(r)Wround+1edraft,h_r^{\mathrm{draft}}=d_rW_{\mathrm{draft}}+\mathbf{1}\phi(r)W_{\mathrm{round}}+\mathbf{1}e_{\mathrm{draft}},

where hrcurh_r^{\mathrm{cur}} has the layout of the source formulation for each task, ecure_{\mathrm{cur}} and edrafte_{\mathrm{draft}} are learnable embeddings that distinguish the two sequences, and 1\mathbf{1} is a column vector of ones of the corresponding length.

The two sequences are concatenated along the sequence dimension, Hr=cat⁡seq(hrcur,hrdraft)H_r=\operatorname{cat}_{\mathrm{seq}}(h_r^{\mathrm{cur}},h_r^{\mathrm{draft}}), and fed into gθg_\theta. Each draft position receives the same RoPE position as the corresponding position of the current generation. The next draft is obtained by applying the output projection of the source formulation to the hidden states of gθ(Hr)g_\theta(H_r) at the generated positions of hrcurh_r^{\mathrm{cur}}. We report the conditional formulation only on Sudoku. There, hrcurh_r^{\mathrm{cur}} consists of the 91 prompt positions followed by the 89 answer positions, which carry ϵrWnoise\epsilon_rW_{\mathrm{noise}}, and ecure_{\mathrm{cur}} is added to all 180 positions; hrdrafth_r^{\mathrm{draft}} has the 89 answer positions, which receive the RoPE positions of the answer positions of hrcurh_r^{\mathrm{cur}}. Attention therefore runs over 269 positions instead of the 180 positions of the source and additive formulations.

Drift Field and Loss

This section gives the exact form of the drift field and the loss of § 4.3. Let x1,…,xn∈RL×Dembx_1,\ldots,x_n\in\mathbb{R}^{L\times D_{\mathrm{emb}}} be the generated samples of one problem at the terminal round, and let the positives be encoded as in Appendix § 7.§ 1; with a single positive, we write it as yy. Norms of latent sequences are taken over all their entries.

Scaled coordinates. Following the official implementation of the Drifting Model, the drift field is computed in scaled coordinates, and the repulsive term is evaluated at several kernel temperatures rather than with a single bandwidth τ\tau. Let GG denote the set of coordinates at the positions that enter the drift field and the loss, that is, the positions after the prompt (all 89 answer positions on Sudoku; Appendix § 7.§ 1); let ∥⋅∥G\lVert\cdot\rVert_G denote the Euclidean norm restricted to these coordinates, and let ∣G∣\lvert G\rvert be their number, averaged over the mini-batch. Let z1,…,zn+1z_1,\ldots,z_{n+1} be the nn generated samples followed by the target, that is, zk=xkz_k=x_k for k≤nk\le n and zn+1=yz_{n+1}=y, and let

Δjk=∥xj−zk∥G,\Delta_{jk}=\lVert x_j-z_k\rVert_G,
j=1,…,n,k=1,…,n+1,j=1,\ldots,n,\quad k=1,\ldots,n+1,

be the distances from each generated sample to every reference, including the self distance Δjj\Delta_{jj}. Let mm be the mean of all Δjk\Delta_{jk} over the mini-batch, and define the scale

κ=max⁡(m∣G∣,δ),\kappa=\max\left(\frac{m}{\sqrt{\lvert G\rvert}},\delta\right),
x~j=xjκ,y~=yκ,\widetilde{x}_j=\frac{x_j}{\kappa},\qquad\widetilde{y}=\frac{y}{\kappa},

where δ=10−3\delta= 10^{-3} prevents division by a vanishing scale. The scale κ\kappa and every quantity below except x~j\tilde{x}_j are treated as constants (stop-gradient).

Attractive drift. In general, the attractive drift is the kernel-weighted barycenter of the targets minus the sample. When N+N_{+} targets (positives) y1,…,yN+y_1,\ldots,y_{N_{+}} are available for the same prompt,

Vj+=∑i=1N+wji+y~i−x~j,V_j^{+} = \sum_{i=1}^{N_{+}} w_{ji}^{+}\tilde{y}_i-\tilde{x}_j,
wji+=exp⁡(−∥xj−yi∥G/τ+)∑i′=1N+exp⁡(−∥xj−yi′∥G/τ+).w_{ji}^{+} = \frac{\exp\left(-\lVert x_j-y_i\rVert_G/\tau^{+}\right)}{\sum_{i'=1}^{N_{+}}\exp\left(-\lVert x_j-y_{i'}\rVert_G/\tau^{+}\right)}.

With several positives, the references zkz_k of the scaled coordinates (and of the repulsive affinity below) are the nn generated samples followed by all N+N_{+} positives. Unlike the original Drifting Model, which normalizes one kernel matrix over the positives and the generated samples together, we normalize the attractive weights over the positives only. With the joint normalization, the attractive weights of the positives of one problem became almost uniform in our experiments, so the attraction pointed to the centroid of the positives, which rarely decodes to a valid program. The repulsive term keeps the joint normalization of the official implementation and is then restricted to the generated samples (see below). The bandwidth τ+\tau^{+} is 0.01 times the median distance between two distinct positives of the same prompt over the mini-batch; pairs whose distance is at most 10−410^{-4} (identical programs) are excluded from the median. We use the median of the positive set rather than a fixed value because the positive queue (§ 5.2) adds positives during training and the spread of the positive set changes; the factor 0.01 was chosen empirically (Appendix § 7.14 reports the factors 0.1 and 1). With such a small bandwidth, the attractive weight concentrates on the nearest positive. In the TinyGSM experiments, N+N_{+} is the number of positives in the queue of the prompt, between KK and K+4K+4 (§ 5.2). In the Sudoku experiments, the answer is unique and only one target yy is available for each prompt (N+=1N_{+}=1). Then the softmax has a single entry, wj1+=1w_{j1}^{+}=1, and the attractive drift reduces to the difference between the target and the sample,

Vj+=y~−x~j.V_j^{+}=\tilde{y}-\tilde{x}_j.

Repulsive drift with multiple temperatures. The repulsive drift uses the kernel weights of the official implementation, which are computed at three temperatures. Let Δˉjk=Δjk/m\bar{\Delta}_{jk}=\Delta_{jk}/m be the normalized distances, where a large constant is added to the self entries Δˉjj\bar{\Delta}_{jj} so that a sample does not repel itself. For each temperature τ∈{0.02,0.05,0.2}\tau\in\{0.02,0.05,0.2\}, we form the logits ℓjk=−Δˉjk/τ\ell_{jk}=-\bar{\Delta}_{jk}/\tau and the doubly normalized affinity

ajk(τ)=softmax⁡k(ℓj)ksoftmax⁡j(ℓk)j,a_{jk}^{(\tau)}=\sqrt{\operatorname{softmax}_{k}(\ell_j)_k\operatorname{softmax}_{j}(\ell_k)_j},

where the first softmax is over the n+N+n+N_{+} references of sample jj and the second is over the nn generated samples of reference kk. Restricting to the other generated samples and renormalizing gives the weights

wjk(τ)=ajk(τ)∑ℓ≤n,ℓ≠jajℓ(τ),k≤n, k≠j,w_{jk}^{(\tau)}=\frac{a_{jk}^{(\tau)}}{\sum_{\ell\leq n,\ell\ne j}a_{j\ell}^{(\tau)}},\qquad k\leq n,\ k\ne j,

and the repulsive drift at temperature τ\tau is

Vj−,(τ)=∑k≠jwjk(τ)(x~k−x~j).V_j^{-,(\tau)}=\sum_{k\ne j}w_{jk}^{(\tau)}(\tilde{x}_k-\tilde{x}_j).

A small temperature makes the weights concentrate on the nearest generated sample, while a large temperature spreads them over all generated samples; using the three temperatures together captures both the local and the global structure of the generated samples.

Drift field and loss. For each temperature, the drift field combines the two terms with the coefficients λ+\lambda^{+} and λ−\lambda^{-},

Vj(τ)=λ+Vj+−λ−Vj−,(τ).V_j^{(\tau)}=\lambda^{+}V_j^{+}-\lambda^{-}V_j^{-,(\tau)}.

The original Drifting Model uses (λ+,λ−)=(1,1)(\lambda^{+},\lambda^{-})=(1,1) (V+−V−V^{+}-V^{-}); our experiments use (2,1)(2,1) (2V+−V−2V^{+}-V^{-}) unless noted otherwise (§ 5), and Appendix § 7.§ 4 compares the two. Let ρτ\rho_{\tau} be the root mean square of V(τ)V^{(\tau)} over all generated samples in the mini-batch and all coordinates in GG. The final drift is the sum of the normalized fields over the three temperatures,

Vj=∑τ∈{0.02,0.05,0.2}Vj(τ)ρτ.V_j=\sum_{\tau\in\{0.02,0.05,0.2\}}\frac{V_j^{(\tau)}}{\rho_{\tau}}.

We then construct the stop-gradient target as

xjtgt=sg⁡(x~j+Vj).x_j^{\mathrm{tgt}}=\operatorname{sg}\left(\tilde{x}_j+V_j\right).

Appendix § 7.§ 4 works out this target for a single positive.

For one training example, the loss is

L(θ)=1nLDemb∑j=1n∥x~j−xjtgt∥G2\mathcal{L}(\theta)=\frac{1}{nL D_{\mathrm{emb}}}\sum_{j=1}^{n}\left\lVert\tilde{x}_j-x_j^{\mathrm{tgt}}\right\rVert_G^2

where the gradient flows only through x~j=xj/sg⁡(κ)\tilde{x}_j=x_j/\operatorname{sg}(\kappa).

The loss is applied only at the sampled terminal round. Appendix § 7.§ 5 describes the full procedure over a mini-batch step by step and summarizes it in Algorithm 1.

Coefficients of the Attractive and Repulsive Drifts

We write the drift field as λ+V+−λ−V−\lambda^{+}V^{+}-\lambda^{-}V^{-} (§ 5; Appendix § 7.§ 3 gives the exact form with the three temperatures). The original Drifting Model uses (λ+,λ−)=(1,1)(\lambda^{+},\lambda^{-})=(1,1), and we use (2,1)(2,1) in all experiments except the (1,1)(1,1) ablations. This appendix recalls why (1,1)(1,1) is the natural choice of the original model, compares the two choices on Sudoku and on GSM8K, and gives a possible reason for the difference.

The original Drifting Model. With (1,1)(1,1), the field V+−V−V^{+}-V^{-} is anti-symmetric, Vp,q=−Vq,pV_{p,q}=-V_{q,p}, so it vanishes when the generated distribution matches the target (§ 3), and the original paper reports that the non-anti-symmetric field 2V+−V−2V^{+}-V^{-} makes training fail on ImageNet (Deng et al., [5]). The original model also trains with many generated samples per class, so the kernel weights of the repulsion depend on the distances and a sample is pushed mainly by the generated samples near it. In our setting, the field with (2,1)(2,1) is not anti-symmetric, and neither is our attraction, which is normalized over the positives only (Appendix § 7.§ 3); matching the target distribution is therefore no longer guaranteed to be an equilibrium. With a single positive, all generated samples coinciding with it still give Vj=0V_j=0 for any coefficients, although this does not guarantee a fixed point of the recurrent generation map.

Sudoku. We selected (λ+,λ−)=(2,1)(\lambda^{+},\lambda^{-})=(2,1) using a separately obtained Sudoku development dataset. Neither the S-FLM benchmark evaluation scores nor the GSM8K test scores were used to select these coefficients. We use the same coefficient pair on both tasks and n=2n=2 generated samples to limit training cost: in the source formulation, n=2n=2 approximately halves the training cost relative to n=4n=4 (Appendix § 7.11).

Table 5 reports benchmark evaluation ablations of the coefficients and the number of generated samples. With n=2n=2, (1,1)(1,1) collapses to 0.0% at every evaluated NFE for all three seeds, whereas (2,1)(2,1) reaches 92.1% at NFE=16. With n=4n=4, both choices train, and (1,1)(1,1) achieves higher accuracy from NFE=8 onward.

SettingNFE=1458163264128180
(2,1)(2,1), n=2n=2 (default)7.3±1.07.3\pm1.078.9±0.678.9\pm0.682.5±0.882.5\pm0.888.4±0.488.4\pm0.492.1±0.492.1\pm0.493.1±0.693.1\pm0.693.5±0.793.5\pm0.793.9±0.793.9\pm0.794.0±0.794.0\pm0.7
(2,1)(2,1), n=4n=47.6±2.27.6\pm2.278.1±1.478.1\pm1.481.7±1.381.7\pm1.387.4±1.587.4\pm1.591.1±1.291.1\pm1.292.1±1.192.1\pm1.192.6±1.292.6\pm1.292.7±1.292.7\pm1.292.9±1.492.9\pm1.4
(1,1)(1,1), n=2n=20.0±0.00.0\pm0.00.0±0.00.0\pm0.00.0±0.00.0\pm0.00.0±0.00.0\pm0.00.0±0.00.0\pm0.00.0±0.00.0\pm0.00.0±0.00.0\pm0.00.0±0.00.0\pm0.00.0±0.00.0\pm0.0
(1,1)(1,1), n=4n=46.5±1.76.5\pm1.776.0±2.676.0\pm2.680.7±2.280.7\pm2.288.0±1.288.0\pm1.293.5±0.693.5\pm0.695.9±0.495.9\pm0.497.1±0.397.1\pm0.398.0±0.298.0\pm0.298.3±0.298.3\pm0.2

Table 5. Sudoku Hard accuracy (%) at 20,000 updates of the source formulation with the drift-field coefficients (λ+,λ−)∈{(2,1),(1,1)}(\lambda^+,\lambda^-)\in\{(2,1),(1,1)\} and n∈{2,4}n\in\{2,4\} generated samples (2,000 validation puzzles, one candidate per puzzle, EMA weights, mean ±\pm standard deviation over three seeds, round embedding clamped to ϕ(min⁡(r,4))\phi(\min(r,4)); the other settings as in § 5.1). The first row is the ReLaDrift Source row of Table 1.

GSM8K. On GSM8K (K=12K=12, the verifier, and the positive queue), (1,1)(1,1) collapses with n=2n=2 and with n=4n=4.

On GSM8K (K=12K=12, the verifier, and the positive queue), (1,1)(1,1) collapses with n=2n=2 and with n=4n=4 and with n=4n=4: the accuracy is 0.0% at every NFE at the evaluations at 20,000 and 40,000 updates, after which both runs were stopped, whereas (2,1)(2,1) with n=2n=2 reaches 16.1% at NFE=16 at 40,000 updates and 29.4% at 250,000 updates (Table 11 and Table 12). Unlike on Sudoku, four generated samples do not rescue (1,1)(1,1) here.

SettingNFE=14581632641282565121024
(2,1)(2,1), verifier, K=12K=12 (default)13.227.327.429.029.429.429.429.529.529.429.4
without verifier and queue4.517.017.318.520.020.320.820.720.720.620.7
(1,1)(1,1), verifier (stopped at 40,000 updates)0.00.00.00.00.00.00.0––––
τ+=0.1×\tau^{+}=0.1\times median10.324.024.524.626.326.826.827.027.027.027.0
τ+=1×\tau^{+}=1\times median (40,000 updates)0.00.20.20.20.40.40.4––––
CodeT5 embedding table6.715.116.016.717.718.018.218.118.118.118.2
random embedding table1.45.56.06.77.17.57.47.57.47.47.5
additive formulation14.026.227.027.928.428.829.129.329.629.629.5
single-round Drifting Model8.47.87.77.77.77.77.77.77.77.77.7
architecture of ReLaDrift, one round (control)7.9––––––––––
training depth Rmax⁡=8R_{\max}=810.224.225.026.227.728.128.228.228.128.228.1
training depth Rmax⁡=16R_{\max}=168.024.024.926.827.227.728.028.228.228.128.1
K=1K=11.08.79.310.712.212.913.313.613.613.713.7
K=4K=410.422.723.024.424.925.225.425.525.525.525.5

Table 11. GSM8K test ablations at 250,000 updates (%; EMA, single; round embedding clamped to ϕ(min⁡(r,Rmax⁡−1))\phi(\min(r,R_{\max}-1)); Rmax⁡=5R_{\max}=5 unless noted). Unless noted, the source formulation with K=12K=12, (λ+,λ−)=(2,1)(\lambda^{+},\lambda^{-})=(2,1), the attraction bandwidth τ+=0.01×\tau^{+}=0.01\times the median distance between distinct positives (Appendix § 7.§ 3), the verifier, and the positive queue. The single-round Drifting Model (trained in the same way) is iterated at NFE ≥2\ge2, with its decoded and re-encoded program fed back in place of the noise (the dashed line of Figure 1; Appendix § 7.15). The (1,1)(1,1) runs (n=2n=2 and n=4n=4) were stopped at 40,000 updates after collapsing to 0.0% at every NFE ≤64\le64 at 20,000 and 40,000 updates. The τ+=1×\tau^{+}=1\times median run was trained for 40,000 updates only; its row gives the values at 40,000 updates (0.0% at every NFE at 20,000 updates). The control row keeps the round embedding and all projections of ReLaDrift but is trained with Rmax⁡=1R_{\max}=1; its refinement projections are never trained, so only NFE=1 is reported. The columns to the right of the vertical line (NFE ≥32\ge32) lie outside the few-step regime.

UpdatesARS-FLMFMLM+defaultw/o ver.(1,1)τ+=0.1\tau^{+}=0.1
20k48.10.73.19.16.70.05.5
40k56.72.46.716.111.90.012.7
60k59.84.79.019.613.9–17.2
80k62.55.911.222.116.6–20.4
100k62.58.016.922.817.6–20.6
120k63.58.918.025.217.7–23.1
140k62.510.018.325.018.0–24.7
160k63.812.620.826.518.8–25.7
180k65.211.419.525.919.3–26.0
200k66.013.221.326.818.7–26.2
220k65.113.523.727.220.5–26.7
240k64.613.224.728.719.9–26.8
250k66.012.624.929.420.0–26.3

Table 12. GSM8K test accuracy (%) over training (EMA, single). AR and S-FLM sample with temperature T=1T=1 (AR sequential, S-FLM 32 steps), and FMLM+ uses 32 rounds with argmax; ReLaDrift (K=12K=12) is at NFE=16 with the clamp. “w/o ver.” is (λ+,λ−)=(2,1)(\lambda^{+},\lambda^{-})=(2,1) without the verifier and the queue, “(1,1)” is V+−V−V^{+}-V^{-} with the verifier, stopped at 40,000 updates, and “τ+=0.1\tau^{+}=0.1” uses τ+=0.1×\tau^{+}=0.1\times median instead of 0.01×0.01\times median; the progress of K=1K=1 and K=4K=4 is in Table 13.

A possible reason: the restoring force toward the positives. Let x~j=xj/κ\tilde{x}_j=x_j/\kappa and y~i=yi/κ\tilde{y}_i=y_i/\kappa denote the scaled generated samples and positives, respectively, where κ\kappa is the common mini-batch scale defined in Appendix § 7.§ 3. The barycenters below are computed in these scaled coordinates.

Write the attractive and repulsive drifts of a generated sample xjx_j as Vj+=yˉj+−x~jV_j^{+}=\bar{y}_j^{+}-\tilde{x}_j and Vj−=yˉj−−x~jV_j^{-}=\bar{y}_j^{-}-\tilde{x}_j, where yˉj+\bar{y}_j^{+} is the kernel-weighted barycenter of the positives and yˉj−\bar{y}_j^{-} that of the other generated samples (Appendix § 7.§ 3); both sets of weights sum to one. Ignoring the RMS normalization for the moment, the stop-gradient target of xjx_j is

x~j+λ+Vj+−λ−Vj−=(1−λ++λ−)x~j+λ+yˉj+−λ−yˉj−.\tilde{x}_j+\lambda^{+}V_j^{+}-\lambda^{-}V_j^{-} =(1-\lambda^{+}+\lambda^{-})\tilde{x}_j+\lambda^{+}\bar{y}_j^{+}-\lambda^{-}\bar{y}_j^{-}.

With (1,1)(1,1), the coefficient of x~j\tilde{x}_j is exactly 1, whatever the kernel weights, the number of generated samples, and the number of positives: the displacement asked of xjx_j, yˉj+−yˉj−\bar{y}_j^{+}-\bar{y}_j^{-}, does not depend on the position of xjx_j itself except through the kernel weights. A common displacement of all generated samples of a problem away from the positives is corrected, because it moves yˉj−\bar{y}_j^{-} but not yˉj+\bar{y}_j^{+}; this is the mechanism of the original model, where the field acts on the generated distribution as a whole. A sample that strays alone, however, while the other generated samples stay near the positives, is not pulled back: when yˉj−=yˉj+\bar{y}_j^{-}=\bar{y}_j^{+}, the field on xjx_j vanishes wherever xjx_j is. With (2,1)(2,1), the coefficient of x~j\tilde{x}_j is 0: the target is yˉj++(yˉj+−yˉj−)\bar{y}_j^{+}+(\bar{y}_j^{+}-\bar{y}_j^{-}), anchored at the barycenter of the positives, so each sample is pulled toward the positives individually, in addition to the repulsion. With the RMS normalization of Appendix § 7.§ 3, the same computation gives the coefficient 1−γ(λ+−λ−)1-\gamma(\lambda^+-\lambda^-) with γ=∑τ1/ρτ\gamma=\sum_{\tau}1/\rho_{\tau}: again 11 for (1,1)(1,1), and 1−γ1-\gamma for (2,1)(2,1).

The case n=2n=2 makes the consequence visible. Each sample jj then has a single negative, the other generated sample kk, and the normalization over the other generated samples degenerates for every temperature τ\tau:

wjk(τ)=ajk(τ)ajk(τ)=1,Vj−,(τ)=x~k−x~j,w_{jk}^{(\tau)}=\frac{a_{jk}^{(\tau)}}{a_{jk}^{(\tau)}}=1,\qquad V_j^{-,(\tau)}=\tilde{x}_k-\tilde{x}_j,

whatever the distance between the two samples, so the repulsion grows linearly with their distance, the opposite of the local repulsion that the kernel is meant to give. With a single positive, Vj+=y~−x~jV_j^+=\tilde{y}-\tilde{x}_j, and the field

Vj(τ)=λ+(y~−x~j)−λ−(x~k−x~j)V_j^{(\tau)}=\lambda^+(\tilde{y}-\tilde{x}_j)-\lambda^-(\tilde{x}_k-\tilde{x}_j)

is the same at every temperature, so Vj=γVj(τ)V_j=\gamma V_j^{(\tau)}. For the targets xjtgt=sg⁡(x~j+Vj)x_j^{\mathrm{tgt}}=\operatorname{sg}(\tilde{x}_j+V_j) and xktgt=sg⁡(x~k+Vk)x_k^{\mathrm{tgt}}=\operatorname{sg}(\tilde{x}_k+V_k) of the two samples, this gives

xjtgt−xktgt=(1+γ(2λ−−λ+))(x~j−x~k).x_j^{\mathrm{tgt}}-x_k^{\mathrm{tgt}}=\left(1+\gamma(2\lambda^--\lambda^+)\right)(\tilde{x}_j-\tilde{x}_k).

With (1,1)(1,1), the factor is 1+γ>11+\gamma>1: every update asks the two samples to move further apart, nothing bounds their separation, and training collapses, as observed on both tasks. With (2,1)(2,1), the factor is exactly 11: the target keeps the current separation, and the restoring force above pulls both samples toward the positive. On GSM8K, the attraction concentrates on the nearest positive (Appendix § 7.§ 3), so the same relation holds approximately when the two samples share their nearest positive. With n=4n=4, the repulsive weights depend on the distances again and the separation no longer grows mechanically, which is consistent with (1,1)(1,1) training on Sudoku, where every generated sample is attracted to the single positive. However, even with

n=4n=4, the coefficient of x~j\tilde{x}_j in the stop-gradient target remains 11 under (1,1)(1,1): the explicit x~j\tilde{x}_j terms in the attractive and repulsive drifts cancel. The drift therefore still lacks an explicit restoring term proportional to −x~j-\tilde{x}_j, although the kernel weights depend on the sample’s position. We conjecture that this absence contributes to the collapse of (1,1)(1,1) on GSM8K even with n=4n=4, where generated samples can be attracted to different positives among the K=12K=12 given ones and the queue. With (2,1)(2,1), an explicit restoring term remains.

Training Procedure of the Recurrent Model

This section describes one optimizer step of the source formulation (§ 4.3) over a mini-batch of BB problems in words; Algorithm 1 summarizes it. The additive and conditional formulations differ in the recurrence, which draws fresh noise at every round and starts from the initial draft d0d_0, and in where the generated samples branch (§ 4.3), and the changes for the verifier and the positive queue are given in § 5.2 and Appendix § 7.§ 6.

Step 1: sampling the number of rounds. The number of rounds Rb∼Uniform⁡{1,…,Rmax⁡}R_b\sim\operatorname{Uniform}\{1,\ldots,R_{\max}\} is drawn for each problem bb; the terminal round of problem bb is round Rb−1R_b-1. On GSM8K, the draw is independent for every problem, and intermediate round rr is one batched forward pass over the trajectories whose Rb>r+1R_b>r+1; on Sudoku, one RR is drawn per mini-batch and shared by all BB problems, so that every intermediate round is one batched forward pass over the mini-batch.

Step 2: encoding the prompt and the positives. The prompt latent PbP_b and the positives yb,iy_{b,i} are obtained as in Appendix § 7.§ 1; on GSM8K, both are standardized with the fixed constants (μ,σ)(\mu,\sigma). The encoder is frozen, so these latents are computed once and cached; in the TinyGSM experiments the latents of the KK given positives are precomputed and only the programs added to the positive queue are encoded during training. Step 3: intermediate rounds without gradient. For each problem bb, nn independent noises ϵb,j∼N(0,I)\epsilon_{b,j} \sim\mathcal{N}(0,I) are drawn at all LL positions, one per trajectory, and the rounds r=0,…,Rb−2r=0,\ldots,R_b-2 of each trajectory of problem bb are computed under stop-gradient: the first round transports the noise, d1=sg⁡(fθ(ϵb,j,Pb,0))d_1=\operatorname{sg}(f_\theta(\epsilon_{b,j},P_b,0)), and each later round is applied to the previous output with the round embedding ϕ(r)\phi(r), dr+1=sg⁡(fθ(dr,Pb,r))d_{r+1}=\operatorname{sg}(f_\theta(d_r,P_b,r)). No new noise is drawn after the first round, no decoding or re-encoding takes place between rounds, and no activations are stored, so these passes cost only forward computation. In the additive and conditional formulations, one intermediate trajectory is started from the fixed initial draft d0d_0 with fresh noise at every round, shared by the generated samples, and the branching happens at the terminal round. When Rb=1R_b=1, this step is empty for problem bb, whose terminal round is then the first round.

Step 4: terminal round with gradient. At round Rb−1R_b-1, each trajectory of problem bb gives its generated sample xb,j=fθ(dRb−1,Pb,Rb−1)x_{b,j}=f_\theta(d_{R_b-1},P_b,R_b-1) (or xb,j=fθ(ϵb,j,Pb,0)x_{b,j}=f_\theta(\epsilon_{b,j},P_b,0) when Rb=1R_b=1), computed with gradient. With the verifier, each xb,jx_{b,j} is decoded and executed at this point, and verified programs are pushed into the positive queue before the drift is computed.

Steps 5–9: drift field, loss, and update. The scale κ\kappa, the attractive and repulsive drifts, the RMS-normalized field with (λ+,λ−)=(2,1)(\lambda^+,\lambda^-)=(2,1), the stop-gradient targets xb,jtgtx^{\mathrm{tgt}}_{b,j}, and the loss are computed from the terminal-round samples xb,jx_{b,j} and the positives of each problem exactly as in Appendix § 7.§ 3; with the verifier, the repulsive weights of verified samples are set to zero (Appendix § 7.§ 6). The gradient flows only through the terminal-round outputs x~b,j=xb,j/sg⁡(κ)\tilde{x}_{b,j}=x_{b,j}/\operatorname{sg}(\kappa); the intermediate rounds and every other quantity are constants. The gradient norm is clipped to 1, one AdamW step is taken, and the EMA weights are updated.

Training Step with the Verifier and the Positive Queue

Algorithm 2 shows one optimizer step of the source formulation with the verifier and the positive queue (§ 5.2). It extends Algorithm 1 by decoding and executing the terminal samples, updating the positive queues, and removing the repulsion from verified samples.

Learning from generated reasoning selected using known answers has precedents in STaR (Ze

Algorithm 1One training step of ReLaDrift (source formulation)
Require:Mini-batch of BB problems with prompts PbP_b and positives {yb,i}\{y_{b,i}\}, model fθf_\theta, maximum number of rounds Rmax⁡R_{\max}, number of generated samples nn, drift-field coefficients (λ+,λ−)(\lambda^+,\lambda^-), temperatures T={0.02,0.05,0.2}\mathcal{T}=\{0.02,0.05,0.2\}
1:Sample the number of rounds Rb∼Uniform⁡{1,…,Rmax⁡}R_b\sim\operatorname{Uniform}\{1,\ldots,R_{\max}\} for each problem bb (one shared RR per mini-batch on Sudoku)
2:for b=1,…,Bb=1,\ldots,B and j=1,…,nj=1,\ldots,n do
3:Sample ϵb,j∼N(0,I)\epsilon_{b,j}\sim\mathcal{N}(0,I) and set d←ϵb,jd\leftarrow\epsilon_{b,j} ▹\triangleright one trajectory per generated sample
4:for r=0,…,Rb−2r=0,\ldots,R_b-2 do ▹\triangleright intermediate rounds, no gradient
5:d←sg⁡(fθ(d,Pb,r))d\leftarrow\operatorname{sg}(f_\theta(d,P_b,r)) ▹\triangleright round 0 transports the noise; later rounds refine the previous output
6:end for
7:xb,j←fθ(d,Pb,Rb−1)x_{b,j}\leftarrow f_\theta(d,P_b,R_b-1) ▹\triangleright terminal round, with gradient
8:end for
9:Compute the mini-batch scale κ\kappa and the scaled coordinates x~b,j←xb,j/κ\tilde{x}_{b,j}\leftarrow x_{b,j}/\kappa, y~b,i←yb,i/κ\tilde{y}_{b,i}\leftarrow y_{b,i}/\kappa (Appendix § 7.§ 3)
10:for each generated sample (b,j)(b,j) do
11:Vb,j+←V^+_{b,j}\leftarrow kernel-weighted barycenter of the positives of problem bb minus x~b,j\tilde{x}_{b,j}
12:Vb,j−,(τ)←V^{-,(\tau)}_{b,j}\leftarrow affinity-weighted repulsion from the other generated samples of problem bb, for each τ∈T\tau\in\mathcal{T}
13:Vb,j(τ)←λ+Vb,j+−λ−Vb,j−,(τ)V^{(\tau)}_{b,j}\leftarrow\lambda^+V^+_{b,j}-\lambda^-V^{-,(\tau)}_{b,j}
14:end for
15:Vb,j←∑τ∈TVb,j(τ)/ρτV_{b,j}\leftarrow\sum_{\tau\in\mathcal{T}}V^{(\tau)}_{b,j}/\rho_\tau, where ρτ\rho_\tau is the root mean square of V(τ)V^{(\tau)} over the mini-batch
16:xb,jtgt←sg⁡(x~b,j+Vb,j)x^{\mathrm{tgt}}_{b,j}\leftarrow\operatorname{sg}(\tilde{x}_{b,j}+V_{b,j})
17:L(θ)←1nBLDemb∑b,j∥x~b,j−xb,jtgt∥G2\mathcal{L}(\theta)\leftarrow\frac{1}{nBLD_{\mathrm{emb}}}\sum_{b,j}\lVert\tilde{x}_{b,j}-x^{\mathrm{tgt}}_{b,j}\rVert_G^2, and update θ\theta with its gradient, which flows only through x~b,j\tilde{x}_{b,j}

Table 1.

likman et al., 2022) and rejection sampling finetuning (Yuan et al., 2023); here, accepted programs enter the drifting positive set, and their corresponding generated samples are removed from the negative set. [5 Appendix A.8] cache real samples in queues; our per-problem queue stores generated programs accepted by an execution-based check.

Pseudo-reference. The verifier uses a voted pseudo-reference from the numerical answers of the KK given positives, not a ground-truth answer. The vote selects the unique largest answer group (a plurality, without requiring more than half the votes). Values are sorted and grouped when their distance from the group’s minimum is at most 10−310^{-3}; the reference is the value from the earliest selected positive in the winning group. If the largest group is tied, or if the tolerance relation is nontransitive, the verifier is disabled for that problem, which is then trained with its KK given positives only; no generated sample is accepted or masked. The original TinyGSM solution is not used to construct this reference. Self-consistency aggregates sampled reasoning paths to select an answer at inference time (Wang et al., 2023); here, voting supplies a training-time pseudo-reference. Agreement with it does not establish ground-truth correctness.

Execution. At the sampled terminal round, each generated sample is decoded into tokens with the decoder (Appendix § 7.8), the program after the prompt (up to the first EOS) is executed in a sandbox with a time limit of 15 ms, and the sample is verified if its return value matches the reference answer within 10−310^{-3}.

Positive queue. A verified program is written back as the prompt followed by the program and EOS and is encoded with the same encoder in the same way as the given positives (Appendix § 7.§ 1). It is pushed into the four additional slots only if it is not already in the queue, and only the programs added during training are pushed out; the KK given positives are never pushed out, even when the queue is full. A verified program enters the queue before the loss of the same step is computed.

Repulsion mask. With the weights wjk(τ)w_{jk}^{(\tau)} of Appendix § 7.§ 3, the repulsive drift becomes

Vj−,(τ)=∑k≠jwjk(τ)(1−vk)(x~k−x~j),V_j^{-,(\tau)} = \sum_{k \ne j} w_{jk}^{(\tau)} (1-v_k) (\tilde{x}_k-\tilde{x}_j),

that is, the weights of verified samples are set to zero after the normalization, without renormalizing the remaining weights. The root mean square ρτ\rho_\tau is computed from the field before this masking. The loss itself is applied to all generated samples, including the verified ones.

Sudoku. On Sudoku, the verifier checks the rules of Sudoku and the agreement with the clues. Because the answer is unique, a verified answer is always the given positive and the queue never grows (we observed N+=1N_+=1 throughout training); the verifier acts only through the repulsion mask.

Generation Procedure

Algorithm 3 gives the generation procedure of § 4.4: the prompt is encoded once, the draft is updated NN times with the same model fθf_\theta, with the round embedding clamped beyond the training depth, and the decoder is applied once to the final draft.

Decoder

This appendix describes the learned decoder used with the pretrained contextual encoder (CodeT5) in the GSM8K experiments; on Sudoku, decoding is the inner product with the one-hot codes followed by the argmax (§ 5.1). The decoder is a separate bidirectional Transformer (8 layers, width 512, 8 heads; 42.2M parameters) that takes a sequence of latents as input and predicts a token distribution at every position, as in Shabalin et al. (2025). It is trained before the Drifting Model and kept frozen during its training and at test time. It is trained to recover the token sequence from perturbed latents with the corruption that ELF uses to train decoding Hu et al. (2026, Appendix C.1): at every position, the clean latent z~\tilde{z} is replaced by αz+(1−α)ϵ\alpha z+(1-\alpha)\epsilon with ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I) and α=sigmoid⁡(0.8+0.8ξ)\alpha=\operatorname{sigmoid}(0.8+0.8\xi), ξ∼N(0,1)\xi\sim\mathcal{N}(0,1), drawn independently for each position. The training data are the CodeT5 latents of one Qwen3.5 solution for each of 100,000 training problems, and the validation data are 10,000 held-out TinyGSM problems with their original solutions; we train for 20,000 updates with batch size 64, AdamW (learning rate 3×10−43\times10^{-4} after 500 warm-up updates, decayed to 3×10−53\times10^{-5}, weight decay 0.01), and use the checkpoint with the lowest validation loss, at 18,000 updates. The decoder is trained on latents standardized with its own statistics (μdec,σdec\mu_{\mathrm{dec}},\sigma_{\mathrm{dec}}), which differ from the constants (μ,σ)(\mu,\sigma) used by the Drift-

Algorithm 2 One training step of the source formulation with the verifier and the positive queue

Require: Problems {cb}b=1B\{c_b\}_{b=1}^{B}, model fθf_{\theta}, decoder, positive queues {Qc}\{Q_c\}, pseudo-references {ac}\{a_c\} (votes of the KK given positives), number of generated samples nn, maximum number of rounds Rmax⁡R_{\max}, drift-field coefficients (λ+,λ−)(\lambda^{+},\lambda^{-})

  1. Encode the prompts PbP_b (zeros after the prompt) and set GbG_b to the positions after the prompt of problem cbc_b

  2. Sample the number of rounds Rb∼Uniform⁡{1,…,Rmax⁡}R_b \sim\operatorname{Uniform}\{1,\ldots,R_{\max}\} for each problem bb

  3. for b=1,…,Bb=1,\ldots,B and j=1,…,nj=1,\ldots,n do

  4. Sample ϵb,j∼N(0,I)\epsilon_{b,j} \sim\mathcal{N}(0,I) and run the source recurrence for RbR_b rounds; rounds 0,…,Rb−20,\ldots,R_b-2 are computed without gradient

  5. xb,j←x_{b,j} \leftarrow output of the terminal round Rb−1R_b-1 (with gradient)

  6. end for

  7. for b=1,…,Bb=1,\ldots,B and j=1,…,nj=1,\ldots,n do

  8. Decode xb,jx_{b,j}, execute the program, and set vb,j←1[∣output⁡−acb∣≤10−3]v_{b,j} \leftarrow\mathbf{1}[|\operatorname{output}-a_{c_b}| \le10^{-3}] (vb,j←0v_{b,j} \leftarrow0 if execution fails or cbc_b has no valid pseudo-reference)

  9. if vb,j=1v_{b,j}=1 then

  10. Push the program into QcbQ_{c_b} (the KK given positives are never pushed out)

  11. end if

  12. end for

  13. Encode the programs newly pushed into QcbQ_{c_b} (the latents of the KK given positives are cached) and collect the positives yb,1,…,yb,N+,by_{b,1},\ldots,y_{b,N_{+,b}} of QcbQ_{c_b}

  14. Compute the drift Vb,jV_{b,j} with these positives, the coefficients (λ+,λ−)(\lambda^{+},\lambda^{-}), and the repulsion mask vv, and set xb,j←sg⁡(x~b,j+Vb,j)x_{b,j} \leftarrow\operatorname{sg}(\tilde{x}_{b,j}+V_{b,j}) (Appendix A.3 and this section)

  15. L←1nBLDemb∑b,j∥x~b,j−xb,jtgt∥Gb2\mathcal{L} \leftarrow\frac{1}{nBLD_{\mathrm{emb}}}\sum_{b,j}\lVert\tilde{x}_{b,j}-x_{b,j}^{\mathrm{tgt}}\rVert^{2}_{G_b}; backpropagate, clip the gradient norm to 1, and take an AdamW step

  16. Update the EMA weights

Algorithm 3 Generation with NN function evaluations.

Red: source formulation; blue: additive and conditional formulations.

Require: Problem statement, model fθf_{\theta}, decoder, number of rounds NN (the NFE), training depth Rmax⁡R_{\max}; initial draft d0d_0 (additive and conditional formulations only)

Ensure: Generated token sequence

  1. P←P \leftarrow prompt latent of the problem statement (§ 4.1)

  2. Sample ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I)

  3. if source formulation then

  4. d←fθ(ϵ,P,0)d \leftarrow f_{\theta}(\epsilon,P,0) ▷ first round transports the noise

  5. else

  6. d←fθ(ϵ,d0,P,0)d \leftarrow f_{\theta}(\epsilon,d_0,P,0) ▷ first round: noise and the initial draft

  7. end if

  8. for r=1,…,N−1r=1,\ldots,N-1 do

  9. rˉ←min⁡(r,Rmax⁡−1)\bar{r} \leftarrow\min(r,R_{\max}-1) ▷ clamp the round embedding beyond the training depth

  10. if source formulation then

  11. d←fθ(d,P,rˉ)d \leftarrow f_{\theta}(d,P,\bar{r}) ▷ refine the previous output; no new noise

  12. else

  13. Sample ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I)

  14. d←fθ(ϵ,d,P,rˉ)d \leftarrow f_{\theta}(\epsilon,d,P,\bar{r}) ▷ fresh noise; the previous output is the draft

  15. end if

  16. end for

  17. return the tokens decoded from dd (§ 4.1)

ing Model (Appendix § 7.§ 1). Therefore, a generated latent zz is re-standardized by the affine map

zdec=zσ+μ−μdecσdecz_{\mathrm{dec}} = \frac{z\sigma+ \mu- \mu_{\mathrm{dec}}}{\sigma_{\mathrm{dec}}}

before it is given to the decoder. The same decoder is used by the verifier during training (§ 5.2). On the clean CodeT5 latents of 10,000 training problems that were not used to train the decoder (one Qwen3.5 solution each), the decoder recovers 99.98% of the tokens and 98.18% of the programs exactly, and 99.55% of the decoded programs return the value of the original program; of the 182 programs that are not recovered exactly, 140 differ only in the docstring or by a consistent renaming of a variable and return the same value. These numbers concern clean latents, not the latents generated by ReLaDrift.

Generating the Positives

We use the train split of TinyGSM/TinyGSM, revision 5049a7840fa546985eaae4c698d5daf6fdf7b47d, containing 11,846,109 question–program pairs. This is the size of the released dataset used here; the original paper (Liu et al., 2023) reports 12.3M pairs. We retain the 11,811,180 pairs that fit within 576 CodeT5 tokens, then use datasets.Dataset.train_test_split with a validation fraction of 0.01 and seed 42 to obtain 11,693,068 training pairs and 118,112 validation pairs. We generate alternatives for the first 2,483,048 problems of this shuffled training split (21.0% of the unfiltered release).

We sampled 32 Python solutions per problem using Qwen/Qwen3.5-9B (Qwen Team, 2026) at revision c202236235762e1c871ad0ccb60c8ee5ba337b9a, with the thinking mode off, temperature 0.85, top-p 0.95, top-k 20, min-p 0, no presence, frequency, or repetition penalty, and at most 600 generated tokens, using the prompt below, which we wrote for this work; the required output format (one function simple_math_problem whose docstring repeats the problem statement) follows the TinyGSM solutions. A solution is kept if it is executable, which we check with a restricted interpreter, fits within 576 tokens of the CodeT5 tokenizer used by ReLaDrift and 512 tokens of the SmolLM tokenizer used by the compared methods, and differs from the other kept solutions of the same problem after removing comments, docstrings, and whitespace. In the kept solutions, we remove the comments but keep the docstrings. The training set consists of the 1,588,781 problems (13.4% of TinyGSM) with at least 12 kept solutions, with 12 solutions each; all 12 return the same value for 62.7% of the problems. The compared methods are trained on all 12 solutions of these problems, expanded into 19,065,372 shuffled problem–solution pairs; ReLaDrift uses the first KK solutions of each problem as its given positives. Except for the comments, the generated programs are similar in style to the TinyGSM solutions. Neither the generated nor the original solutions are guaranteed to be correct, and we do not filter them by correctness: for one problem in our data, the TinyGSM solution and two of the generated solutions return three different values. In the prompt, {{ question }} is replaced by the problem statement; its line breaks are adjusted to the column width here.

You are given a grade-school math word problem.
Generate a correct Python solution that
satisfies
all of the following requirements:
1. Define exactly one function with this
   signature:

def simple_math_problem():

2. Immediately after the function definition,
   include a triple-quoted docstring containing
   the given problem statement exactly as
   provided.
3. Solve the problem using ordinary Python
   arithmetic.
4. Use meaningful variables for quantities
   appearing in the problem.
5. Solve the problem step by step using
   intermediate variables when natural.
6. The function must return the final numerical
   answer.
7. The program must be executable and must
   return the mathematically correct answer.
8. Do not print the answer.
9. Do not use Markdown code fences.
10. Do not include any explanation or text
    outside the Python function.
11. Keep the solution natural and similar in
    style to real TinyGSM solutions. Do not use
    unnecessarily sophisticated Python
    features, helper functions, classes, loops,
    comprehensions, symbolic algebra, or
    external libraries unless they are
    genuinely necessary for the problem.
12. Keep comments minimal. Do not perform
    calculations in comments, and do not
    include numerical values in comments.
13. The numerical computation must be derived
    from the quantities in the problem. Do not
    simply return the known final answer
    directly.

### Question

{{ question }}

Program

Experimental Settings

Tables 6 and 7 summarize the settings of ReLaDrift and the compared methods on Sudoku (Section 5.1) and GSM8K (Section 5.2). On both tasks, AR, MDLM, Duo, and S-FLM are built from the S-FLM repository (https://github.com/jdeschena/s-flm, commit 30def79) and FMLM+ from the FMLM+ repository (https://github.com/MananAg007/posterior-refinement, commit 73652f9); neither repository is modified. On GSM8K, we run the training pipelines of the two repositories. On Sudoku, all methods share our single-GPU training loop, which calls the model, loss, optimizer, learning-rate schedule, and EMA of each repository, so that all methods see the same puzzles in the same order: the 48,000 training puzzles are permuted anew at every epoch and the incomplete last batch of the permutation (128 puzzles, different ones in each epoch) is dropped, so an epoch consists of 187 batches of 256 puzzles. For FRM, we use our own implementation following its paper (Helbling et al., 2026). On Sudoku, the S-FLM setting is the one of the official Sudoku script of S-FLM (the DiT on the sphere and the truncated, adaptive noise schedule), and the S-FLM paper also uses the standard DiT on Sudoku. On GSM8K, it is the one of the official TinyGSM script, which follows the TinyGSM experiments of the S-FLM paper (the S-arch backbone, 163.0M parameters), run without the run-time patch that we use on Sudoku (Table 6); with the official data loader, the adaptive schedule is first refitted at update 37,251.

ItemReLaDrift (ours)S-FLMFMLM+FRM (our implementation)DFM (our implementation)
Sequence, vocabulary180 tokens, vocabulary 12, fixed prefix 91samesamesamesame
Statecontinuous vector (12-dim one-hot code)learned embedding on the sphereone-hot + Gaussian noiseone-hot + carryone-hot + Gaussian noise (linear interpolant xt=tx1+(1−t)x0x_t = t x_1 + (1-t)x_0)
Encoder / decoderone-hot code / inner product with the codes + argmaxlearned embedding on the sphere / arg max of the output layerone-hot / argmaxone-hot / argmaxone-hot / argmax
Transformer9 layers, width 512, 8 heads, FFN 2,048, RoPE, dropout 0.1, round embedding ϕ(r)\phi(r) added to the inputDiT on the sphere, 8 layers, width 512, 8 headsDiT, 8 layers, width 512, 8 heads, two-time adaLNDiT, 8 layers, width 512, 8 heads, adaLNDiT of FMLM+, 8 layers, width 512, 8 heads, two-time adaLN (t,r−t)(t,r-t)
Parameters28.39M28.56M28.61M28.57M28.61M
OptimizerAdamW 3e-4 (warmup 2,500, then constant), β=(0.9,0.999)\beta=(0.9,0.999), weight decay 0, clip 1same (official)same (official)same; optimizer and EMA restarted at Stage Bsame
EMA decaymin⁡(0.9999,(u+1)/(u+10))\min(0.9999,(u+1)/(u+10))samesamesamesame
Training budget20,000 updates, batch 25620,000 updates, batch 25620,000 updates, batch 256Stage A 10,000 + Stage B 10,000, 256 puzzles ×4\times4 states20,000 updates, batch 256
Forward passes in traininggradient only at the terminal round; n=2n=2 independent trajectories12 (self-distillation)FPF (rollout of depth 16)2 (ng=2n_g=2 source draws per puzzle)
Generationrecurrent, NFE 1–180, ϕ(min⁡(r,4))\phi(\min(r,4))ODE with the exact velocity, arg max at the last step, NFE 1–180Posterior Refinement, threshold 0.999, 1–180 roundsEuler / SDE / held-time, NFE 1–180Euler on the uniform grid, NFE 1–180
Scoringexact match on the 89 answer tokens, no overwritingsamesamesamesame

Table 6. Sudoku settings of ReLaDrift and the compared methods. AR, Duo, and MDLM are trained with the implementations in the S-FLM repository and its settings (the same 8-layer, width-512 DiT (Peebles and Xie, 2023), 28.6M parameters; the AR model is causal and has no adaLN, 25.2M). DFM (Drift Flow Matching) is our implementation following its paper and official code (commit b8f287c): the drift field, the law of the time pairs, the loss, and the Euler sampler are those of the official code, and the solution of each puzzle is paired with ng=2n_g=2 source draws as in Appendix D of its paper. Duo and MDLM use ancestral sampling at temperature 1, and AR is evaluated with greedy decoding and with sampling at temperature 1. S-FLM uses its truncated, adaptive noise schedule; all methods drop the last, incomplete batch of each epoch (128 of the 48,000 puzzles; the permutation is redrawn at every epoch, so the dropped puzzles differ), whereas the official loaders keep it; with this batching, the official code never refits the adaptive schedule, so we added a patch at run time, without modifying the repository, so that the schedule is refitted (first refit at update 1,051). The single-round Drifting Model uses the ReLaDrift column without the round embedding, the draft projection, and the refinement output projection (28.37M parameters) and runs one round. All methods use dropout 0.1 and bf16 mixed precision.

On GSM8K, the compared methods pad the sequences to 512 tokens and compute the loss on the padding, as in S-FLM. On GSM8K, a generated program is correct if its return value matches the answer exactly for integers and within 10−310^{-3} otherwise, with a time limit of 5 s. ReLaDrift uses the optimization settings of the compared methods: AdamW with β=(0.9,0.999)\beta= (0.9, 0.999), no weight decay, gradient clipping at 1, dropout 0.1, and the warm-up-corrected EMA decay min⁡(0.9999,(u+1)/(u+10))\min(0.9999,(u + 1)/(u + 10)) at update uu of the official implementation. In ReLaDrift, the dropout (on the attention and feed-forward outputs) is active only in the terminal round, which receives the gradient; the earlier rounds of an update are computed in evaluation mode.

Sudoku benchmark. The 2,000 validation puzzles of the benchmark of S-FLM are used only for evaluation; we verified that each of them has a unique solution and that they share no solution grid with the training puzzles. Of its three difficulty levels (40, 35, and 30 of the 81 digits visible), we use the hard level, which has lower reported accuracy than the easier levels (Descheneaux and Gulcehre, 2026, Table 1). The Sudoku-Extreme benchmark (Wang et al., 2025a) is nearly saturated (99.5% for FRM; Helbling et al., 2026).

Drift Flow Matching. DFM (Ma et al., 2026) trains a mean-velocity network uθ(xt,t,r)u_\theta(x_t,t,r) on the linear interpolant xt=tx1+(1−t)x0x_t = t x_1 + (1 - t)x_0, x0∼N(0,I)x_0 \sim\mathcal{N}(0,I), with the drifting objective applied to the predictions x^r=xt+(r−t)uθ\hat{x}_r = x_t + (r - t)u_\theta, so that a sample can be generated in one step (t=0t = 0, r=1r = 1) or refined with more Euler steps. Its official code (commit b8f287c) covers class-conditional image latents only, so we implement the prompt-conditioned case following Appendix D of the paper: for each puzzle, the solution x1x_1 is paired with ng=2n_g = 2 source draws x0,ix_{0,i}, the positives are xr,i=rx1+(1−r)x0,ix_{r,i} = r x_1 + (1 - r)x_{0,i}, and the negatives are the predictions x^r,i\hat{x}_{r,i} of the model from the same x0,ix_{0,i}. The drift field, the law of the time pairs, the loss, and the sampler are those of the official code: Sinkhorn-normalized Gibbs weights with the Euclidean distance and temperature 1 (20 iterations; the negatives include the sample itself), four time pairs per update with t=min⁡t = \min and r=max⁡r = \max of σ(2.5z−1)\sigma(2.5z - 1) and σ(2.5z′+1)\sigma(2.5z' + 1), z,z′∼N(0,1)z,z' \sim\mathcal{N}(0,1), the squared error to the stopped target x^r+V\hat{x}_r + V summed over the coordinates, and Euler steps on the uniform grid of [0,1][0,1]. The network is the DiT of FMLM+ (Table 6; 28.61M parameters; tt and r−tr - t enter through its two time embeddings), and the data, batching, optimizer, EMA, budget, and evaluation are those of the other methods. Because every puzzle has a single solution, the Sinkhorn weights are numerically the identity for r≲0.7r \lesssim0.7, so on Sudoku the DFM objective reduces to a regression toward the paired target xr,ix_{r,i}; the group size then changes mainly the number of source draws per puzzle, and ng=2n_g = 2 matches the two trajectories of ReLaDrift. With ng∈{4,8,16}n_g \in\{4, 8, 16\}, trained in the same way, the accuracy at NFE=16 is 31.9, 35.2, and 33.4% (mean over three seeds), 3–6 points above ng=2n_g = 2 (29.2%) and within seed standard deviation (5–8 points); the ranking against ReLaDrift is unchanged. No Sudoku result of DFM has been published.

Accuracy of our runs of the compared methods. On Sudoku, our runs of S-FLM and MDLM are within 2.4 points of the published accuracies and our Duo runs are 3.1 points above them, whereas our FMLM+ runs are about 4 points below the published results: at NFE=180, the sampling budget of Deschenaux and Gulcehre [7], our S-FLM, Duo, and MDLM reach 43.2, 61.5, and 32.6%, compared with the 45.0, 58.4, and 30.2% reported there. Our FMLM+ reaches 67.0 and 77.5% at NFE=4 and 16, compared with 71.2 and 81.4% reported by Agarwal et al. [12]. Among the compared parallel generators, FMLM+ is the strongest at every NFE from 4 to 128, and FRM with the SDE sampler is the strongest at NFE=180 (79.4% versus 77.2% for FMLM+); Duo reaches 61.9% at NFE=16, and S-FLM, MDLM, and DFM reach at most 43.2%. For FRM, the reported checkpoint at 20,000 updates is the end of Stage B, and it is less accurate than the Stage A checkpoint of the same runs at 10,000 updates (Euler sampler, NFE=16: 71.8% versus 86.1%, mean over three seeds). Helbling et al. (2026) train on 1,000 Sudoku-Extreme puzzles with symmetry augmentation, which we do not use; on the 48,000 puzzles of our setting, Stage A already fits the training puzzles (its training loss is below 0.03 at 10,000 updates), so that the rollouts of Stage B contain few errors to correct. We nevertheless report the checkpoint at 20,000 updates, which matches the training budget of the other methods. The autoregressive model reaches 0.5% with greedy decoding and with sampling (Table 1), in agreement with the 1.0% (greedy) and 0.6% (sampling) reported on the hard level by Deschenaux and Gulcehre [7]. On GSM8K, at NFE=1,024, our S-FLM, Duo, and MDLM runs are comparable to the values reported at T=0.1T = 0.1 with the original TinyGSM solutions (about 18, 36, and 33%; Deschenaux and Gulcehre [7]).

Artifacts and licenses. TinyGSM (Liu et al. (2023)) (Hugging Face dataset TinyGSM/TinyGSM) and GSM8K (Cobbe et al. [3]) (Hugging Face dataset openai/gsm8k) are released under the MIT License. Qwen3.5-9B (Qwen Team (2026)) and the SmolLM-135M tokenizer (Ben Allal et al. [16]) are released under the Apache License 2.0. The CodeT5 reference implementation (Wang et al. (2021)) is released under the BSD 3-Clause License. The Hugging Face model card for the Salesforce/codet5-small checkpoint lists Apache License 2.0.1 The S-FLM repository (Deschenaux and Gulcehre [7]) states the Apache License 2.0 in its README (the repository contains no license file), the FMLM+ repository (Agarwal et al. [12]) is released under the Apache License 2.0, and the Sudoku generator (Alp, [13]) states the MIT License in its README. The reference implementation of the Drifting Model (Deng et al. [5]) at commit c8b4fee, whose drift field we follow (Appendix § 7.§ 3), did not include a license file. The current official repository states that its code and pretrained weights are released under the MIT License.2 Our implementation is written from scratch. All artifacts are used for research purposes in accordance with these terms. Our code is released under the Apache License 2.0.

Training and Generation Cost

Training. ReLaDrift runs several forward passes of fθf_{\theta} per update, whereas AR, S-FLM, Duo, and MDLM run one pass, FMLM+ about two (self-distillation), and DFM two (ng=2n_g = 2 source draws per puzzle); FRM propagates four states per puzzle and, in Stage B, additionally runs a rollout without gradient for fixed-point forcing. With R∼Uniform⁡{1,…,5}R \sim\operatorname{Uniform}\{1,\ldots,5\}, the expected depth is Rˉ=3\bar{R} = 3. The source formulation propagates n=2n = 2 independent trajectories, so it runs 2Rˉ=62\bar{R} = 6 forward passes per update, of which only the two terminal passes carry gradients. The additive formulation shares one intermediate trajectory and branches into two candidates at the terminal round, so it runs Rˉ−1+2=4\bar{R} - 1 + 2 = 4 forward passes, again with two gradient passes. The conditional formulation runs the same passes as the additive one, but on Sudoku its attention covers 269 instead of 180 positions, because the 89 draft tokens are appended. Counting a backward pass as twice a forward pass, one update of the source formulation costs about (4+2×3)/3≈3.3(4 + 2 \times3)/3 \approx3.3 times an update of a one-pass method, and the additive formulation about (2+2×3)/3≈2.7(2 + 2 \times3)/3 \approx2.7 times; the intermediate passes run without gradient and store no activations. Measured on one H200 for Sudoku (batch 256 puzzles, n=2n = 2), one update takes 0.22 s for Source, so

20,000 updates take about 1.2 h; Additive takes 0.17 s per update (0.9 h), Conditional 0.26 s (1.4 h), and the single-round Drifting Model 0.12 s (0.7 h). With n=4n = 4, Source takes 0.42 s per update (2.3 h). On GSM8K, the verifier additionally decodes the two terminal samples with the decoder, executes the programs (time limit 15 ms each), and encodes the new positives of the queue with the CodeT5 encoder; these steps run without gradient. On 8 H200 GPUs, one update including these steps takes 0.56, 0.65, and 0.76 s for K=1K = 1, 4, and 12, so 250,000 updates take about 39, 45, and 53 h. The decoder (Appendix § 7.8) is trained once, before ReLaDrift, for 20,000 updates on one GPU in less than one hour, which is less than 0.3% of the GPU time of training ReLaDrift with K=12K = 12 (8 GPUs for about 53 h).

Generation. Generation costs NFE forward passes of fθf_\theta and at most one pass each of the encoder and the decoder, which do not depend on NFE. On GSM8K, the frozen CodeT5 encoder (35.3M parameters) encodes the prompt once per problem, which can be cached, and the decoder (42.2M parameters) decodes the final draft; both are smaller than the 87.4M-parameter fθf_\theta. On Sudoku, the codes are fixed and decoding is an arg max over 12 codes, so no extra pass is needed. On one H200, evaluating all 2,000 Sudoku puzzles at every NFE up to 16 (one shared 16-round trajectory, 16 passes per puzzle) takes about 3 s. On GSM8K, with batches of 64 problems on one H200, a round of fθf_\theta takes 24 ms, a step of Duo 95 ms, a step of MDLM 93 ms, a refinement round of FMLM+ 130 ms, and a step of S-FLM 242 ms (its sampler computes the velocity in double precision); generating the 1,319 test programs takes ReLaDrift 6.8 s at NFE=4 and 13 s at NFE=16, Duo 63 s at NFE=32, and S-FLM 5,200 s at NFE=1,024 (Figure 3).

Additional Sudoku Results

Table 8 lists all eleven ReLaDrift conditions of the unified Sudoku Hard study (three seeds each, 20,000 updates, EMA, clamped round embedding) at every evaluated NFE, including the single-round Drifting Model, the source formulation with the verifier, and the training depths Rmax⁡∈{3,8,16}R_{\max} \in\{3, 8, 16\}. On Sudoku, the verifier can only remove the repulsion from verified samples, because the answer is unique and the queue never grows; with n=2n = 2 it makes little difference: it is 1.1–1.7 points more accurate at NFE=2–5 and within 0.3 points at NFE=1 and NFE ≥8\ge8 (92.0% versus 92.1% at NFE=16). The training depths are discussed in § 5.1; this table adds NFE=2.

ConditionNFE=12458163264128180
Source (main)7.3±1.054.7±0.378.9±0.682.5±0.888.4±0.492.1±0.493.1±0.693.5±0.793.9±0.794.0±0.7
Additive7.1±2.254.8±3.977.6±2.681.5±3.186.2±3.689.5±3.190.5±3.091.0±2.891.6±2.591.6±2.8
Conditional7.4±0.954.3±3.377.6±1.781.1±1.186.1±1.289.6±1.890.7±1.791.2±2.091.6±2.391.7±2.2
Single-round Drifting Model5.1±0.84.9±0.84.9±0.85.0±0.84.9±0.84.9±0.84.9±0.84.9±0.84.9±0.84.9±0.8
Source, w/ verifier7.0±1.056.4±0.680.0±1.083.7±0.788.4±0.792.0±0.693.0±0.693.4±0.893.6±0.893.7±0.8
Source, (1, 1)0.0±0.00.0±0.00.0±0.00.0±0.00.0±0.00.0±0.00.0±0.00.0±0.00.0±0.00.0±0.0
Source, (1, 1), n=4n = 46.5±1.750.7±4.276.0±2.680.7±2.288.0±1.293.5±0.695.9±0.497.1±0.398.0±0.298.3±0.2
Source, n=4n = 47.6±2.255.1±3.078.1±1.481.7±1.387.4±1.591.1±1.292.1±1.192.6±1.292.7±1.292.9±1.4
Source, Rmax⁡=3R_{\max} = 35.7±2.745.5±5.866.2±3.468.5±2.971.4±2.172.6±2.072.9±1.973.1±1.973.1±2.073.2±2.0
Source, Rmax⁡=8R_{\max} = 86.4±2.059.1±4.383.4±1.786.9±1.791.2±1.794.9±1.396.0±1.096.6±1.096.8±0.996.9±0.8
Source, Rmax⁡=16R_{\max} = 164.2±2.156.3±6.185.2±2.988.8±2.193.0±1.796.1±0.897.2±0.597.4±0.497.5±0.497.6±0.4

Table 8. Sudoku Hard accuracy (%) of all ReLaDrift conditions at 20,000 updates (2,000 puzzles, one candidate, EMA, mean ± standard deviation over three seeds, round embedding clamped to ϕ(min⁡(r,Rmax⁡−1))\phi(\min(r, R_{\max} - 1)). Unless noted: source formulation, (λ+,λ−)=(2,1)(\lambda^{+}, \lambda^{-}) = (2, 1), n=2n = 2, Rmax⁡=5R_{\max} = 5, no verifier. At NFE ≥ 2, the single-round Drifting Model is iterated, with its decoded output fed back in place of the noise (the dashed line of Figure 1; Appendix § 7.15).

Results without the Clamp of the Round Embedding

The model uses the round embedding clamped to ϕ(min⁡(r,Rmax⁡−1))\phi(\min(r, R_{\max} - 1)), so that it never receives a round embedding that it did not see in training (§ 4.4). This appendix reports, on the same checkpoints, the accuracy when ϕ(r)\phi(r) is instead extrapolated to the rounds r≥Rmax⁡r \ge R_{\max}; for NFE ≤Rmax⁡\le R_{\max} the two are identical (Table 9 and Table 10). The clamp only keeps the round embedding within the values seen in training; it does not make the drafts after more than Rmax⁡R_{\max} rounds follow the distribution of the drafts seen in training.

ConditionRound embeddingNFE=8163264128
Sourceextrapolated39.4±40.246.4±41.454.4±47.257.1±49.558.0±50.2
clamped88.4±0.492.1±0.493.1±0.693.5±0.793.9±0.7
Additiveextrapolated38.6±40.333.5±19.049.0±26.855.6±29.658.4±32.3
clamped86.2±3.689.5±3.190.5±3.091.0±2.891.6±2.5
Conditionalextrapolated74.7±18.587.6±1.589.5±2.590.1±2.790.9±3.4
clamped86.1±1.289.6±1.890.7±1.791.2±2.091.6±2.3
Source, n=4n=4extrapolated11.8±20.41.2±0.91.8±1.72.6±2.62.8±2.9
clamped87.4±1.591.1±1.292.1±1.192.6±1.292.7±1.2
Source, (1,1)(1,1), n=4n=4extrapolated0.0±0.01.9±2.22.9±2.34.0±3.64.4±3.7
clamped88.0±1.293.5±0.695.9±0.497.1±0.398.0±0.2
Source, Rmax⁡=3R_{\max}=3extrapolated0.0±0.00.0±0.00.0±0.00.0±0.00.0±0.0
clamped71.4±2.172.6±2.072.9±1.973.1±1.973.1±2.0
Source, Rmax⁡=8R_{\max}=8extrapolated91.2±1.793.1±1.995.5±1.096.3±0.996.7±0.7
clamped91.2±1.794.9±1.396.0±1.096.6±1.096.8±0.9
Source, Rmax⁡=16R_{\max}=16extrapolated93.0±1.796.1±0.897.3±0.697.8±0.698.0±0.6
clamped93.0±1.796.1±0.897.2±0.597.4±0.497.5±0.4

Table 9. Sudoku Hard accuracy (%) at 20,000 updates with the round embedding extrapolated or clamped for NFE ≥8\ge8 (same checkpoints; mean ± standard deviation over three seeds). For Rmax⁡=8R_{\max}=8 and Rmax⁡=16R_{\max}=16, the two coincide up to NFE=8 and NFE=16, respectively.

Round embeddingNFE=81632
K=1K=1extrapolated11.113.314.7
clamped10.712.212.9
K=4K=4extrapolated24.023.424.9
clamped24.424.925.2
K=12K=12extrapolated29.029.430.1
clamped29.029.429.4

Table 10. GSM8K test accuracy (%) of ReLaDrift at 250,000 updates with the round embedding extrapolated or clamped for NFE ≥8\ge8 (EMA, single; same checkpoints and noise).

On Sudoku, some runs collapse without the clamp. For the source formulation, one seed falls to 3.0% at NFE=8 and to at most 0.1% at NFE $\ge 16; of the other two, one falls to 32.5% at NFE=8 and the other to 59.3% at NFE=16, whereas with the clamp all three seeds stay at 88–93% at NFE=8 and 16. The additive and conditional formulations also have seeds that fall at NFE=8 or 16. With Rmax⁡=3R_{\max} = 3, all three seeds fall to 0.0% at every NFE ≥8\ge8, and with n=4n = 4, every seed falls below 10% at NFE $\ge 16. The same runs collapse on 2,000 additional puzzles generated with a different seed. With Rmax⁡=8R_{\max} = 8 and Rmax⁡=16R_{\max} = 16, the extrapolation does not collapse. With Rmax⁡=8R_{\max} = 8, it is up to 1.8 points below the clamp in the mean over seeds (at most 3.2 points for one seed, at NFE=16); with Rmax⁡=16R_{\max} = 16, where the two differ only at NFE ≥17\ge17, it is 0.1–0.5 points above the clamp at NFE=32–128 (97.3, 97.8, and 98.0% versus 97.2, 97.4, and 97.5%).

On GSM8K, the clamp changes the accuracy by at most 1.8 points in either direction at NFE=8, 16, and 32 (Table 10): without the clamp, K=1K = 1 is 0.4–1.8 points higher, K=4K = 4 is 0.3–1.5 points lower, and K=12K = 12 is identical at NFE=8 and 16 and 0.7 points higher at NFE=32. Among the other runs with K=12K = 12 in Appendix § 7.14 (without the verifier and the queue, the additive formulation, the two embedding tables, and τ+=0.1×\tau^+ = 0.1 \times median), evaluated in the same way, none collapses without the clamp, and the two differ by at most 1.6 points at NFE=8, 16, and 32. The GSM8K numbers with the clamp at every checkpoint are listed in Appendix § 7.16. Overall, the clamp costs little on GSM8K and prevents the collapse on Sudoku.

GSM8K Ablations and Training Progress

Table 11 ablates ReLaDrift (K=12K = 12) at 250,000 updates with the clamped round embedding at every NFE, including the rows of Table 4: the verifier and the positive queue, the drift-field coefficients, the bandwidth of the attraction, the embedding space, the formulation, the training depth, and the number of given positives KK. Without the verifier and the positive queue, the accuracy at NFE=16 drops from 29.4% to 20.0%; with (λ+,λ−)=(1,1)(\lambda^{+}, \lambda^{-}) = (1, 1), training collapses to 0.0% at every NFE by the first evaluation at 20,000 updates, with n=2n = 2 and with n=4n = 4, and both runs were stopped at 40,000 updates (Appendix § 7.§ 4). The bandwidth of the attraction, τ+=0.01\tau^{+} = 0.01 times the median distance between two distinct positives of the same problem (Appendix § 7.§ 3), is ablated with the factors 0.1 and 1. With 0.1, the accuracy is lower at every NFE (10.3% at NFE=1, 24.0% at NFE=4, and 26.3% at NFE=16, versus 13.2, 27.3, and 29.4%) but still increases with NFE, and it is below the default at 12 of the 13 checkpoints (26.0% versus 25.9% at 180,000 updates; Table 12). With 1, the model, trained for 40,000 updates, reaches at most 0.4% at every NFE at 20,000 and 40,000 updates: with such a bandwidth, the attractive weights no longer concentrate on the nearest positive, and the attraction points toward a barycenter of several programs, as with the joint normalization discussed in Appendix § 7.§ 3. Replacing the contextualized CodeT5 embeddings with a context-free embedding table (the CodeT5 input embedding of each token, or a random Gaussian table) lowers the accuracy to 17.7% and 7.1% at NFE=16, but the accuracy still increases with NFE. The additive formula-

tion, trained in the same way from scratch, is within about one point of the source formulation at every NFE of Table 11 (0.4–1.1 points below at NFE=4–32 and 0.8 points above at NFE=1; 28.4% versus 29.4% at NFE=16), and its accuracy also increases monotonically with NFE up to NFE=32. Increasing the training depth to Rmax⁡=8R_{\max} = 8 or 16 (round embedding clamped to ϕ(min⁡(r,Rmax⁡−1))\phi(\min(r, R_{\max} - 1))) lowers the accuracy at every NFE: by 3.0 and 5.2 points at NFE=1, 3.1 and 3.3 at NFE=4, 1.7 and 2.2 at NFE=16, and 1.2–1.7 at NFE ≥ 32, the opposite of the trend on Sudoku (Table 2). With the terminal round sampled uniformly, each round receives the loss for a fraction 1/Rmax⁡1/R_{\max} of the problems, so a larger depth trains each round, including the first, on fewer problems for the same number of updates. The compared methods were also evaluated at sampling temperature T=1T = 1 (single, 250,000 updates; NFE=32 for S-FLM, Duo, and MDLM, and sequential generation for AR): AR 66.0, S-FLM 12.6, Duo 15.0, and MDLM 7.7, versus 72.3, 19.9, 30.6, and 18.8 at T=0.1T = 0.1; the main table therefore reports T=0.1T = 0.1. Table 12 shows the accuracy over training with the generation settings of the compared methods (AR sequential and S-FLM with 32 steps, both at T=1T = 1; FMLM+ with 32 rounds and argmax) and ReLaDrift at NFE=16 with the clamp.

Iterating the Single-Round Drifting Model at Test Time

To test whether additional rounds help a Drifting Model that is not trained for them, we iterate the single-round Drifting Model of each task at test time. The first round (NFE=1) is its usual generation from the evaluation noise. Each later round decodes the output of the previous round, encodes the decoded sequence as the training targets are encoded (on Sudoku, the one-hot code of the 89 answer tokens; on GSM8K, the CodeT5 encoding of the prompt, the program up to the first EOS, and EOS; Appendix § 7.§ 1), and feeds it to the model in place of the noise; no new noise is drawn. The noise is the only input through which this model can receive its previous output, because neither single-round Drifting Model has a round embedding, a draft projection, or an output projection for the refinement rounds. A control with the architecture of ReLaDrift, including the round embedding and all projections, but trained with Rmax⁡=1R_{\max}=1 reaches 7.9% at NFE=1 on GSM8K (Table 11), so the absence of the round embedding does not explain the gap to ReLaDrift at NFE=1; its refinement projections are never trained, so it is not iterated. The evaluation noise, batches, and scoring are those of Table 8 and Table 11, and NFE=1 reproduces the evaluation of the single-round Drifting Model for every puzzle and problem.

On Sudoku, the accuracy is 4.9–5.0% at every NFE from 2 to 128, against 5.1% at NFE=1 (Table 8), and the accuracy per cell stays at 87.9% at every NFE; from NFE=8 on, about 1,800 of the 2,000 puzzles no longer change from one round to the next. On GSM8K, the accuracy is 7.6–7.8% at every NFE from 2 to 1,024, against 8.4% at NFE=1 (Table 11). The programs are rewritten (at NFE=2, only 249 of the 1,319 programs keep the code of NFE=1 apart from the docstring), but between NFE=1 and NFE=16, 18 problems become correct and 27 become wrong. Running more rounds of a model that is not trained to correct its own drafts therefore does not improve it.

All GSM8K Results of ReLaDrift

Table 13 lists the GSM8K test accuracy of ReLaDrift at every evaluated checkpoint and NFE with the clamped round embedding ϕ(min⁡(r,4))\phi(\min(r,4)) (Appendix § 7.13); for NFE ≤5\le5, the clamp does not change the round embedding.

KKUpdatesNFE=12458163264
120k0.00.00.00.00.00.00.00.0
40k0.00.30.20.30.40.40.50.5
60k0.20.30.50.60.81.11.11.1
80k0.30.41.01.01.11.82.12.4
100k0.30.51.31.42.43.54.55.2
120k0.30.92.43.13.95.16.16.5
140k0.41.04.24.55.16.77.07.3
160k0.41.85.56.07.07.78.38.6
180k0.52.06.46.98.19.710.811.1
200k0.52.76.97.78.99.610.210.9
220k0.83.88.08.69.910.611.512.2
240k0.94.58.39.710.812.112.713.1
250k1.03.98.79.310.712.212.913.3
420k0.20.72.12.53.13.33.43.3
40k0.52.36.97.79.010.511.011.2
60k1.95.810.711.812.714.014.514.4
80k2.78.413.114.415.215.916.516.3
100k3.79.914.414.616.317.618.018.1
120k5.811.716.417.817.819.019.319.6
140k6.613.617.518.718.720.821.121.5
160k7.914.918.719.520.320.921.021.2
180k8.616.119.920.521.622.221.822.4
200k8.816.520.921.721.622.322.422.6
220k8.616.720.921.622.423.623.824.0
240k9.917.723.023.023.223.824.124.3
250k10.418.722.723.024.424.925.225.4
1220k0.52.05.66.37.79.19.710.0
40k2.47.412.313.114.316.116.316.2
60k4.811.215.417.118.019.619.919.5
80k6.313.618.219.120.522.122.222.7
100k7.416.521.021.522.422.823.924.3
120k8.316.622.322.824.025.225.825.6
140k10.118.022.323.024.325.025.525.7
160k11.519.324.924.925.526.527.026.8
180k11.819.423.825.025.725.926.426.5
200k12.119.924.825.225.826.827.227.3
220k12.820.725.125.525.927.227.427.1
240k12.722.426.426.827.728.729.029.3
250k13.222.427.327.429.029.429.429.4

Table 13. GSM8K test accuracy (%) of ReLaDrift (Source, (λ+,λ−)=(2,1)(\lambda^{+}, \lambda^{-}) = (2, 1), verifier and positive queue) at every checkpoint and NFE with the clamped round embedding (EMA, single).

Generated Samples

The following programs were generated by ReLaDrift (K=12K=12) at 250,000 updates on GSM8K test problems (EMA weights, the trajectory of Table 3, clamped round embedding). The docstring, which repeats the problem statement inside the program (Appendix § 7.9), is generated like the rest of the program; its long lines are wrapped here for display. Over the 1,319 test problems, 221 are wrong at NFE=1 and correct at NFE=16, and 7 are correct at NFE=1 and wrong at NFE=16.

# Example 1: correct at NFE=1
def simple_math_problem():
    """
    A robe takes 2 bolts of blue fiber and
    half that much white fiber. How many
    bolts in total does it take?
    """
blue_bolts = 2
white_bolts = blue_bolts / 2
total_bolts = blue_bolts + white_bolts
return total_bolts

# Example 2, NFE=1: wrong (returns 9)
def simple_math_problem():
    """
    There are 6 girls in the park. If there
    are twice the number of boys in the park,
    how many kids are in the park?
    """
    girls = 6
    boys = girls / 2
    totalids = girls + boys
    return totalids

# Example 2, NFE=2: wrong (undefined name)
def simple_math_problem():
    """
    There are 6 girls in the park. If there
    are twice the number of boys in the park,
    how many kids are in the park?
    """
    girls = 6
    boys = girls / 2
    totalids = girls + boys
    return kids

# Example 2, NFE=4: correct
# (identical at NFE=8 and 16)
def simple_math_problem():
    """
    There are 6 girls in the park. If there
    are twice the number of boys in the park,
    how many kids are in the park?
    """
    girls = 6
    boys = girls * 2
    kids = girls + boys
    return kids

References

  1. [2]Nicholas Matthew Boffi, Michael Samuel Albergo, and Eric Vanden-Eijnden. 2025. Flow map matching with stochastic interpolants: A mathematical framework for consistency models. Transactions on Machine Learning Research.arxiv.org/abs/2406.07507
  2. [3]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. Preprint, arXiv:2110.14168.arxiv.org/abs/2110.14168
  3. [4]Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. 2019. Universal transformers. In International Conference on Learning Representations.arxiv.org/abs/1807.03819
  4. [5]Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He. 2026. Generative modeling via drift-ing. In Advances in Neural Information Processing Systems (NeurIPS).
  5. [6]Justin Deschenaux and Caglar Gulcehre. 2025. Beyond Autoregression: Fast LLMs via Self-Distillation Through Time. In International Conference on Learning Representations, volume 2025, pages 5007–5045.arxiv.org/abs/2410.21035
  6. [7]Justin Deschenaux and Caglar Gulcehre. 2026. Language modeling with hyperspherical flows. Preprint, arXiv:2605.11125.arxiv.org/abs/2605.11125
  7. [8]Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. 2025. One step diffusion via shortcut models. In International Conference on Learning Representations, volume 2025, pages 34668–34684.arxiv.org/abs/2410.12557
  8. [9]Luyu Gao, Aman Madaan, Shuyuan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. PAL: Program-aided language models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 10764–10799. PMLR.arxiv.org/abs/2211.10435
  9. [10]Jonas Geiping, Sean Michael McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. 2025. Scaling up test-time compute with latent reasoning: A recurrent depth approach. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.arxiv.org/abs/2502.05171
  10. [11]Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. 2025. Mean flows for one-step generative modeling. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.arxiv.org/abs/2505.13447
  11. [12]Manan Agarwal, Sheel Shah, Chanhyuk Lee, Jaehoon Yoo, Jerry Huang, Seunghoon Hong, Aditi Raghunathan, Jinwoo Kim, and Nicholas M. Boffi. 2026. Posterior refinement: Fast language generation via any-order flow maps. arXiv preprint arXiv:2606.24773.arxiv.org/abs/2606.24773
  12. [13]Ali Alp. 2024. Sudoku puzzle generator. https://github.com/alicommit-malp/Sudoku. GitHub repository, created September 2024.
  13. [14]Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. Structured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Systems, volume 34, pages 17981–17993. Curran Associates, Inc.arxiv.org/abs/2107.03006
  14. [15]Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. 2019. Deep equilibrium models. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.arxiv.org/abs/1909.01377
  15. [16]Loubna Ben Allal, Anton Lozhkov, and Elie Bakouch. 2024. SmolLM - blazingly fast and remarkably powerful. Hugging Face blog, https://huggingface.co/blog/smollm. Model card: https://huggingface.co/HuggingFaceTB/SmolLM-135M.
  16. [17]Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc.arxiv.org/abs/1506.03099

Paper details

vlnc:W_m6pu89ks
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents