WorksComputer Science
Asynchronous Flow Map Language Models
Abstract
The order in which tokens are predicted in language modeling has a profound impact on the quality of generated sequences. Autoregressive and masked diffusion models iteratively fill in tokens and can exploit such orderings across generation steps, but how to leverage an ordering for one-step parallel language modeling is unclear. We introduce asynchronous flow map language models (aFMLMs), which incorporate prediction order into one-step modeling by learning the flow maps of asynchronous flows. These flows specify an ordering through a tokenwise time schedule that controls the relative denoising progress of tokens. We formulate a general class of asynchronous flows that admit both prescribed and learned, state-adaptive schedules, show that they define valid flow maps, and develop new methods for their flow map distillation. The resulting models can generate all tokens in a single forward pass by internalizing the ordering encoded in the flow into their hidden computations. On Sudoku solving and code generation for GSM8K math word problems, aFMLM establishes new state-of-the-art one-step performance.
Introduction
How tokens are ordered and scheduled for prediction has a significant impact on language modeling [49, 36, 26, 34], as it determines how sequence prediction is decomposed into a series of simpler modeling subproblems. For instance, autoregressive models (ARMs) [8] encode an ordering as a prescribed left-to-right factorization, whereas masked diffusion models (MDMs) [44, 50] learn over diverse masking patterns and choose orderings for generation based on model outputs. These choices would not matter for perfect models, as any factorization recovers the true joint distribution. In practice, however, models have finite capacity and some subproblems are harder to learn than others, making generation quality sensitive to the chosen order and schedule [20].
Because these orderings and schedules are defined over generation steps, in which tokens decided earlier inform later ones, iterative models such as ARMs and MDMs can naturally use them. In contrast, parallel language models designed for fast generation take only a few steps and must perform more of their computation in hidden space, where it is unclear how to leverage a given ordering or schedule. This problem is most severe in one-step generation [29], which predicts all tokens in a single forward pass and thus leaves no steps over which an order can be followed.
We address this problem with asynchronous flow map language models (aFMLMs) that incorporate prediction order into one-step generation. Our idea builds on flow map language models (FMLMs) [29], which achieve one-step generation by compressing the sampling trajectories of continuous flow language models (FLMs) [29, 22, 13]. While existing FMLMs use synchronous flows in which all tokens progress together, aFMLMs learn the flow maps of asynchronous flows, in which tokenwise times control the relative progress of tokens to specify an ordering or schedule. Building on prior work [40, 5], we formulate a general class of asynchronous flows that admits both prescribed and learned, state-adaptive schedules. By treating tokenwise times as part of the state, we show that these flows define valid flow maps, and develop methods to distill them. The resulting models generate all tokens in a single forward pass, internalizing the ordering encoded in the flow into their hidden computation.
An illustration of our approach is shown in Figure 1. To summarize, our main contributions are:
Figure 1. Overview. (Left) We formulate asynchronous flows that denoise tokens at different rates. To rhyme with “mitten,” “kitten” emerges early and guides the preceding words. (Right) We distill these flows into an asynchronous flow map that internalizes this ordering and generates the full sequence in one forward pass.
Asynchronous flows. We formulate the generation of asynchronous flow language models (FMLs) as a coupled flow of tokens and their tokenwise times governed by a scheduler, covering prescribed and learned, state-adaptive schedules ported from MDM samplers (§ 4.1).
Asynchronous flow maps. We show that asynchronous flows define a valid flow map and derive distillation objectives that learn FMLMs as one-step generators, in which a single model jointly predicts how the tokenwise times and the tokens progress (§ 4.2 and § 4.3).
One-step generation. We train FMLMs that achieve state-of-the-art one-step performance on Sudoku solving and coding for GSM8K math problems, and analyze their hidden computations, showing that they internalize the effect of ordering and scheduling into one forward pass (§ 5).
Related Work
Prediction order in sequence modeling. Although every factorization of a joint distribution yields the same likelihood, the quality of sequence models depends on the order of prediction, a gap attributed to their computational limits [20]. For ARMs, learning English in the left-to-right order leads to a higher likelihood than in reverse order [49, 36], reversing digits makes arithmetic easier to learn [30], and learning Sudoku solving in the order given by a logical solver outperforms the row-by-row order [48]. In MDMs, adaptive samplers that choose the ordering from model outputs often outperform random ordering [34, 7, 19, 28]. MDMs with adaptive samplers have been distilled into fewer steps [14, 27, 31], but along discrete trajectories that make very few-step generation hard [29].
Asynchronous diffusion and flows. Asynchronous (continuous) diffusion models denoise different components at different noise levels. Most prescribe a schedule ahead of generation, such as left-to-right for text [53, 46, 11], sliding window for video and fluid dynamics [43], offsets between image semantics and texture [35, 40, 4], or algorithmic orders for satisfiability problems [9]. Diffusion forcing [12] uses per-token independent noise levels for training so that schedules can change at inference, such as learned ones that depend on model outputs [23, 47] (Wever et al. 2025). Few-step distillation of asynchronous models, however, has been limited to prescribed schedules [54, 24, 11] or their close variants [5], whereas we consider both prescribed and learned schedules.
Ordering in flow language models. Most continuous FLMs evolve all tokens with a single shared time [29, 13, 22], and their flow maps compress the resulting trajectories [29, 42, 38]. Outside prescribed left-to-right ordering [53, 46, 11], order has been introduced into FLMs by finalizing tokens during generation, either stochastically [39, 25] or once the model is confident [3, 1]. These samplers, however, rely on stochastic updates or restarts with fresh noise, which complicates flow map distillation requiring deterministic trajectories. Rekesh et al. [41] adjust tokenwise step sizes using model outputs, which our framework can express as a learned scheduler and distill. Bansal & Sanghavi [5] select among a set of prescribed schedules using model outputs at the first generation step, allowing the use of flow map distillation methods for prescribed schedules. Our approach goes beyond this workaround and handles fully learned schedules that use all intermediate outputs.
Preliminaries
Flow language models. Continuous flow language models [29, 22, 13] model categorical token sequences by learning a continuous flow from noise to continuous token embeddings. Let be a token vocabulary and a sequence. Following Lee et al. [29], we represent each token by its one-hot encoding, . Let denote the distribution of these one-hot representations and the noise distribution. We define a probability path by interpolating between independent draws and :
The probability path admits a deterministic representation,
where is the velocity field of the probability flow. Instead of predicting the velocity itself, it is common to learn the denoiser, which predicts the posterior mean of the clean data:
Since is one-hot, is a probability distribution over at each position , and the denoiser can be learned by minimizing a tokenwise cross-entropy loss [17, 18, 29].
Flow map language models. The deterministic flow given in (2) defines a unique flow map , the solution operator of the flow , satisfying
where is the average velocity between and , satisfying . By learning the flow map, we can jump between arbitrary times, and evaluating it with and gives a one-step map from noise to data. Recent work on learning flow maps over categorical data [42, 29, 38] parameterizes it via the two-time denoiser ,
which extrapolates the flow map displacement to and satisfies . Since it is simplex-valued, can be learned by self-distillation with KL divergence [29],
where, for and with the stop-gradient operator ,
and we denote by the flow map induced by the learned denoiser .
Asynchronous Flow Maps
We first formulate asynchronous flows under a scheduler, then distill them into asynchronous flow maps for one-step generation.
Asynchronous flows with schedulers
To allow token positions to denoise at different rates, we assign each position its own time:
where measures the denoising progress of position [12]. Extending the standard interpolant (1), we associate with noise and with clean data. For independent draws and , we define the asynchronous interpolant [40]
where we write products and divisions of tokenwise vectors as and .
Schedules. In standard flows (2), generation moves from noise to data as a shared time advances from zero to one, and we only need to choose step sizes to perform generation. With tokenwise times (8), we can think of generation as the analogous transition of from to . Yet, unlike in standard flows, this further leaves a choice of how the tokens progress, as some may advance earlier than others, or all may advance together. These choices give many distinct paths of generation:
Proposition 1. Up to continuous nondecreasing reparameterization, there is a unique continuous nondecreasing path from zero to one. In contrast, there are infinitely many such paths from to in for .
A proof is in Appendix § 8.§ 1.
To describe the progress of generation along such a path, we introduce a scalar base time and call the resulting evolution of tokenwise times from to a schedule. We specify this evolution through a scheduler that sets the tokenwise rates:
where denotes the current token states. We choose the scheduler so that as .
Velocity of token states. We now derive the velocity of token states under a chosen scheduler by extending (2). For each pair of noise and data , we advance the tokenwise times by evaluating the scheduler on the interpolant, . Since , the conditional mean
gives the asynchronous velocity of token states, . Together with the scheduler, it defines a valid probability flow from noise to data, which we refer to as an asynchronous flow:
Proposition 2. Let solve, starting from and ,
Then has the same joint distribution as the interpolant and its tokenwise times advanced by for every , and converges in distribution to as .
A proof is in Appendix § 8.§ 2.
Mirroring (3), we can parameterize the velocity through the corresponding asynchronous denoiser:
Scheduler-independent training. For a chosen scheduler , we can learn by tokenwise cross-entropy, using as inputs and the clean tokens as targets. Yet, constructing these examples requires simulating along each interpolant, tying denoiser training to the scheduler. To allow the scheduler to be chosen after training, we may instead learn the scheduler-independent denoiser
which takes as input . For this, we use diffusion forcing Chen et al., [12], which samples the tokenwise times independently of one another and of the noise and data:
Proposition 3. Consider the objective
Any population minimizer satisfies almost surely under the training distribution.
A proof is in Appendix § 8.§ 3. We leave further discussion on this training scheme in Appendix § 8.§ 4.
Following (3), the denoiser gives the scheduler-independent velocity
which we use to approximate at inference under a choice of scheduler . Denoting , this gives the asynchronous flow
These coupled dynamics define a deterministic ODE on the token states and their tokenwise times. We call the resulting model an asynchronous flow language model (aFLM). We note that setting for all positions recovers the usual FLM, which we refer to as synchronous.
Prescribed and state-adaptive schedules. When does not depend on the token states, as in left-to-right generation for instance, follows a fixed path determined ahead of generation, which we call a prescribed schedule. The condition then holds for every pair of noise and data, so . The velocity approximation is therefore exact:
Corollary 4. Let be a prescribed schedule. Then the asynchronous flow in (15) reduces to
and satisfies for every , with as .
A proof is in Appendix § 8.§ 5. Furthermore, only the denoiser along is needed, so we can train it with the cross-entropy loss in (13) using instead of independently sampled tokenwise times.
Schedulers that depend on the current token states yield state-adaptive schedules, and we call those computed from denoiser predictions learned schedules. Then and generally differ, and exact recovery of is no longer guaranteed. Nevertheless, this allows training the denoiser without fixing a scheduler in advance. Then during inference, a scheduler can be chosen to improve generation by presenting an imperfect denoiser with easier prediction problems Kim et al., [26]. We construct such schedulers as continuous analogues of unmasking criteria used in MDMs, as given in § 4.3.
Asynchronous flow maps
Asynchronous flows exploit prediction order by letting tokens that denoise earlier inform the prediction of others. This motivates our key idea for leveraging such an order for few-step and one-step parallel generation, by distilling the asynchronous flow into a flow map. In this way, the effect of the schedule is internalized within the model’s hidden computation. Since the flow is defined jointly on the tokens and their tokenwise times (15), we define the asynchronous flow map to evolve both:
Proposition 5. On any interval where the asynchronous flow (15) admits unique solutions, its solution operator , called the asynchronous flow map, satisfies
A proof is in Appendix A.6. The composition identity, which we call the semigroup condition, states that a direct jump from to agrees with two successive jumps through . We use this consistency to learn the map by self-distillation.
Parameterization. Following § 3, we express the flow map through its average velocity. For a state at time , we denote the average rates of change of the tokens and their tokenwise times until time by and , respectively, with reducing to the instantaneous dynamics:
For the token component , we further adapt a two-time denoiser parameterization:
This extends (5) by rescaling each token’s displacement according to its tokenwise progress. Along the asynchronous flow, is a weighted average of denoiser predictions and therefore lies in the vocabulary simplex. We can thus learn it with a categorical output and KL loss, as in FMLMs. The following identities recover the flow map and provide the conditions for learning it:
Proposition 6. Let be the asynchronous flow map. Then and satisfy:
(i) The transition is recovered as
(ii) On the diagonal ,
(iii) For , the semigroup condition (17) is equivalent to
where
with and the tokenwise times along the flow from .
A proof is in Appendix A.7.
Training. We learn predictors and , which define a model of the flow map through (20). On the diagonal, the denoiser and scheduler provide their respective targets (21). For , we construct targets for a direct transition from to by composing two shorter student transitions through based on (22):
Here from (22) is evaluated at the tokenwise times of , , and . Combining these composition targets with the diagonal targets gives the training objective:
Proposition 7. For , consider the population objective
The exact and have zero loss. Conversely, any zero-loss solution satisfies the diagonal and composition identities (21) and (22) almost surely under the training distribution.
A proof is in Appendix § 8.8. We call the model learned with (24) an asynchronous flow map language model (aFMLM).
For state-adaptive schedules, we train on joint states visited by the asynchronous flow. These states depend on the preceding trajectory and cannot be sampled directly from the interpolant. We therefore first train the denoiser and simulate its flow under the chosen scheduler to obtain the initial training states. This pretrained teacher also provides the diagonal denoising target in (24). To reduce the cost of simulation, we can later use the learned map to get , as described in § 4.3.
On the other hand, for a prescribed schedule , the time component is fixed:
Only the two-time denoiser needs to be learned. Moreover, the flow follows by Corollary 4, so a training state at any base time can be sampled directly as . On the diagonal, the denoiser is the conditional mean of the clean data . Replacing its target with data therefore changes the expected loss only by a constant independent of the model:
Corollary 8. Let be a prescribed schedule, with fixed to its exact time component and training states sampled from the corresponding interpolants. Then (24) reduces to
up to a constant independent of . Its minimizers coincide with the zero-loss solutions of Proposition 7 with fixed by the schedule.
A proof is in Appendix § 8.9. Thus, a prescribed schedule allows the flow map to be trained from scratch without a pretrained teacher.
Inference. We generate samples by applying the learned flow map on a temporal grid of base times :
The final tokens are obtained by applying tokenwise argmax to . For , a single evaluation of generates the entire sequence without evaluating the scheduler. For supervised tasks, this allows us to train with a prescribed schedule for each example, even when the corresponding schedule is unavailable for a test input. These schedules guide training, and the one-step model internalizes them to generate its output from the task input and initial noise alone.
Algorithmic details
Scheduler implementation. We remark that a scheduler must send as (10). We implement the schedulers in a common form that ensures this by construction (Appendix § 9.§ 1):
where the factor controls how quickly position advances and can be chosen almost freely. Setting , for instance, recovers the synchronous schedule from . For learned schedules, we assign higher factors to positions that the denoiser predicts confidently, using two rules:
Top- confidence scheduler prioritizes the most confident positions, with increasing over time [10].
Entropy-bounded scheduler uses an entropy budget to determine how many high-confidence positions advance faster, allowing more positions to advance together when their predictions are certain [7].
These schedules are constructed as continuous analogues of unmasking strategies used in MDMs, and their details along with more such examples can be found in Appendix § 9.§ 1.
Denoiser and time heads. For flow map distillation under learned schedules, we use one model with a denoiser head for and a time head for . To keep predicted tokenwise times in , we parameterize the time head via the effective rate , which is a weighted average of along the trajectory (see Appendix § 9.§ 2 for details). The head outputs , predicting
Each transition advances the tokenwise times toward 1 and reaches 1 at . On the diagonal, matching to is equivalent to matching to (see Appendix § 9.§ 2 for details).
We initialize the backbone and denoiser head from the teacher and attach the time head to the final hidden features. The time head exponentiates its output to ensure positivity, and we initialize its output before exponentiation to zero, setting at initialization.
Time conditioning. With tokenwise time, both the teacher denoiser and the flow map receive a separate time embedding at each position, obtained by applying a shared embedding layer to . For the flow map, we additionally embed the base times and with zero-initialized layers and add their outputs to the embeddings of , following Lee et al. [29]. The resulting conditioning vectors modulate each block through adaptive layer normalization (adaLN) [37].
Rollout curriculum. To train with (24), we initially obtain training states by rolling out the teacher under the chosen scheduler from to time . After the first round, we use the current student flow map for rollouts and halve the number of rollout steps at successive rounds. The fixed teacher denoiser and scheduler continue to provide the diagonal targets throughout training.
MDM initialization. Diffusion forcing requires the denoiser to denoise across many combinations of tokenwise noise levels, which can make training demanding [26]. When each tokenwise time is close to either 0 or 1, the task resembles masked token prediction, since each position is nearly pure noise or nearly clean. To exploit this overlap, we optionally initialize the denoiser from a trained MDLM [44], following Agarwal et al. [1].
Noise downscaling. Following Chavan et al. [11], we use initial noise with during flow map distillation and inference. In supervised tasks where one accurate answer is sufficient, reducing variation in the initial states can ease the burden of fitting the flow map. We apply this on GSM8K, where it improves one-step accuracy (see Figure 4).
Figure 4. GSM8K accuracy (%) of aFMLM (EB). “+ MDLM” initializes the teacher from MDLM, and “+ ” adds noise downscaling.
Experiments
Throughout the experiments, we address these key questions:
Q1. Can a proper choice of asynchronous scheduler improve the generation quality of aFLMs?
Q2. Does this improvement transfer to one-step generation with aFMLM after distillation?
Q3. Do hidden computations of aFMLMs reflect the schedules of aFLMs they are distilled from?
Setup. We evaluate our method on Sudoku solving and code generation for GSM8K math word problems. For Sudoku, we use the dataset with three difficulty levels and report exact solving accuracy (%) on the full test set, following Deschenaux & Gulcehre [16]. For code generation, we train on code completions from TinyGSM (Liu et al. [32]) and measure zero-shot accuracy (%) on the full test set of GSM8K (Cobbe et al. [15]) math problems, following Kim et al. [26].
For the baselines, we consider discrete methods MDLM (Sahoo et al. [44]), Duo (Sahoo et al. [45]) and AlphaDLM (Gushchin et al. [21]), and continuous methods CANDI (Pynadath et al. [39]), FLM (Lee et al. [29]), S-FLM (Deschenaux & Gulcehre [16]), FMLM (Lee et al. [29]), FMLM+ (Agarwal et al. [1]), and DBTM (Tang & Wang [51]). We train and distill for 250k steps on GSM8K and 20k on Sudoku in total, matching the baselines. We leave further details in Appendix § 10.
Asynchronous schedulers for aFLM (Q1)
For each dataset, we train one asynchronous denoiser with diffusion forcing (Chen et al. [12]) and apply various schedulers during inference. We compare the synchronous schedule (Sync.) of FLM (Lee et al. [29]), a schedule fixed by the initial prediction entropy (Init. ent.) of Bansal & Sanghavi [5], and learned schedulers including the top- confidence scheduler (Top-) and the entropy-bounded scheduler (EB) defined in § 4.3. To test whether generation can also benefit from a prescribed order, we include an algorithmic order (Alg. order) on Sudoku. In this order, the cells that a constraint solver resolves earlier are denoised faster (Shah et al. [48]) (Appendix § 10.§ 3).
The results are in Figure 2, showing that learned schedulers substantially improve generation accuracy on both tasks. With 32-step generation, the EB scheduler improves the accuracy over Sync. from 40.1% to 95.6% on Sudoku-Hard and from 7.4% to 36.4% on GSM8K. Also, schedulers that adapt the order to intermediate predictions (Top- and EB) outperform the scheduler that fixes the order from the initial prediction (Init. ent.). Asynchronous flows thus let flow language models benefit from a good generation order, with the largest gains from learned schedulers ported from MDMs.
Figure 2. Accuracy of aFLM under different inference-time schedulers on Sudoku-Hard (left) and GSM8K (right). A well-chosen scheduler improves performance over the synchronous one (Sync.). Alg. order† denotes the aFLM whose denoiser is trained only along the algorithmic order.
While the prescribed algorithmic order improves accuracy on Sudoku-Hard from 40.1% to 59.7%, it still falls short of the learned schedulers. We conjecture that this gap is because diffusion forcing cannot learn all combinations of tokenwise times equally well (Kim et al. [26]) (§ 4.3). Learned schedulers follow the confidence of the denoiser and thus avoid the poorly learned combinations, whereas the algorithmic order may still visit them. Indeed, training the denoiser only along the algorithmic order raises the accuracy from 59.7% to 88.6% (Alg. order† in Figure 2).
One-step generation with aFMLM (Q2)
We distill aFLM from § 5.1 with EB, Top- and synchronous schedulers into aFMLM (EB), aFMLM (Top-), and aFMLM (Sync.), respectively, and test their one-step generation performance.
The results are shown in Table 1. aFMLM (EB) improves the one-step accuracy over its aFLM teacher, 11.8% vs. 6.1% on Sudoku-Hard and 11.4% vs. 0.3% on GSM8K. In contrast, aFMLM (Sync.), which is distilled from the same teacher, reaches considerably lower performance, showing that the benefit of asynchronous schedulers carries over to one-step generation. aFMLM (Top-) shows a similar gain and achieves the highest one-step accuracy on Sudoku-Hard, which shows that the benefit is not specific to a particular choice of the scheduler. Through these advantages, aFMLM surpasses all one-step baselines on both Sudoku and GSM8K and establishes the new state of the art.
| Model | Easy | Medium | Hard | Accuracy |
| MDLM ([44]) | 63.6 | 16.2 | 1.7 | 0.0 |
| Duo ([45]) | 69.8 | 24.6 | 2.4 | 0.0 |
| CANDI ([39]) | 69.0 | 36.1 | 4.0 | 0.0 |
| FLM ([29]) | 69.1 | 21.5 | 2.8 | 0.0 |
| S-FLM ([16]) | 77.2 | 36.0 | 5.2 | 0.0 |
| FMLM ([29]) | 75.0 | 42.2 | 4.9 | 0.0 |
| FMLM+ ([1]) | 67.4 | 29.9 | 4.1 | 0.3 |
| DBTM ([51]) | – | – | – | 0.8 |
| AlphaDLM ([21]) | – | – | – | 6.3 |
| aFLM | 71.1 | 26.5 | 6.1 | 0.3 |
| aFMLM Sync. | 77.1 | 36.8 | 6.4 | 1.6 |
| aFMLM EB | 79.1 | 47.8 | 11.8 | 11.4 |
| aFMLM Top- | 81.4 | 46.4 | 14.4 | 10.5 |
| aFMLM Alg. order | 82.5 | 44.5 | 10.0 | – |
Table 1. One-step exact solving accuracy (%) on Sudoku and zero-shot accuracy (%) on GSM8K (see Appendix § 10.§ 2 for baseline details). For aFLM, the schedule does not matter in one-step generation.
In addition, as in Corollary 8, aFMLM can be trained along a prescribed schedule from scratch without a pretrained teacher. We show this with the algorithmic order on Sudoku. As in Table 1, aFMLM (Alg. order) achieves 10.0% one-step accuracy on Sudoku-Hard without a teacher, compared with 11.8% of aFMLM (EB). As shown in Table 2, this approach reduces the total training cost from 5 to 3.2 GPU hours by eliminating teacher pretraining. Our method effectively distills both prescribed and learned schedules, and a useful prescribed schedule offers a more efficient route to one-step generation.
| Schedule | Teacher | Distillation | Total |
| Learned | 1.7h | 3.3h | 5.0h |
| Prescribed | 0h | 3.2h | 3.2h |
Table 2. Training cost (GPU hours) on Sudoku-Hard.
In Figure 3 and Figure 4, we analyze the role of our algorithmic choices (§ 4.3) of initializing the aFLM teacher from MDLM and noise downscaling for distillation. Initializing the teacher from
Figure 3. GSM8K accuracy (%) of the teacher aFLM (EB) with 32-step generation across training steps, with and without MDLM initialization.
MDLM accelerates its training, where it reaches 32.9% accuracy on GSM8K with 32-step generation after only 20k training steps, compared with 6.4% from scratch. This suggests MDM initialization eases diffusion forcing training, as the combinations of tokenwise times close to zero or one resemble the problem solved by MDMs. Distilling this stronger teacher improves the one-step accuracy of aFMLM (EB) on GSM8K from 4.2% to 6.3%. Noise downscaling for distillation further improves aFMLM from 6.3% to 11.4% accuracy, confirming that it reduces the burden of fitting the flow map.
Internalization of the schedule into aFMLM (Q3)
We analyze the hidden computation of one-step aFMLM using logit lens [6], which decodes the hidden state of each layer using the output head. For each token, we record the earliest layer from which its decoded prediction matches the final output and remains so, and compare this layer-wise order with the order induced by the schedule. For Sudoku, we compare with the algorithmic order, and for GSM8K, we compare with the order in which the aFLM teacher under the EB scheduler decides the tokens. In Table 3, we measure the alignment between the two orders by the Spearman correlation, computed per sample and averaged over the test set. On both tasks, aFMLM shows a higher correlation with its schedule than aFMLM (Sync.), which indicates that aFMLM internalizes the schedule into its hidden computation. This is consistent with the observation of Asiimwe et al. [2] in image generation that flow maps compress the flow across timesteps into their layers. We further visualize an example in Figure 5. aFMLM (Sync.) does not follow the order of the teacher and produces non-parsable lines, showing that the order matters. In contrast, aFMLM (EB) resolves the tokens that the teacher generates early in its early layers and produces a parsable solution.
Figure 5. A visualization of code solutions to a GSM8K problem generated by one-step aFMLM (EB) (middle) and aFMLM (Sync.) (bottom) in their layer-wise order, compared with the generation order of aFLM (EB) (top). Columns show when 10%, 50%, 85%, and 100% of the tokens are decided.
| Schedule | Sudoku-Hard | GSM8K |
| Synchronous | 0.500 | 0.091 |
| Asynchronous | 0.516 | 0.271 |
Table 3. Spearman correlation with the schedule-induced ordering.
Conclusion
We introduced asynchronous flow map language models (aFMLMs), which distill the ordered generation of flow language models into one-step generators. To this end, we modeled asynchronous generation with an asynchronous flow language model (aFLM), which evolves a coupled flow of tokens and their tokenwise times. We then derived a self-distillation objective that learns the flow map of this flow, which jointly integrates both over any time interval. Empirically, a well-designed scheduler substantially improves aFLMs, and aFMLMs retain these advantage in one-step generation without evaluating the scheduler at test time. We also showed that aFMLMs internalize the order of the scheduler into their hidden computation. Promising future directions include jointly optimizing the scheduler with the denoiser and scaling aFMLMs to larger continuous language models.
Acknowledgments
This work is supported by the National Research Foundation of Korea (RS-2024-00351212) and the Institute of Information & communications Technology Planning & Evaluation (IITP) (RS-2024-00509279, RS-2026-25518317, and RS-2026-25617720), funded by the Korean Government (MSIT). Satoshi Hayakawa was partially supported by JSPS KAKENHI (JP26K25548) and JST PRESTO (JPMJPR26K8).
Proofs and Derivations
Throughout, we assume that the velocity fields in this section are regular enough for their ODEs and continuity equations to have unique solutions on , and that prescribed schedules are absolutely continuous. We also assume whenever and , and that the tokenwise times reach at , which holds for the schedulers of (27) with for and bounded away from zero near . Quantities at are understood as limits , and diagonal quantities at as limits .
Proof of Proposition 1
Proposition 1. Up to continuous nondecreasing reparameterization, there is a unique continuous nondecreasing path from zero to one. In contrast, there are infinitely many such paths from 0 to 1 in for .
Proof. We parameterize each path by .
Scalar time. Let be a continuous nondecreasing path with
Then passes through every value in in order, so it is a reparameterization of .
Tokenwise times. It suffices to exhibit one infinite family of such paths that remain distinct up to reparameterization.
For and , the path goes from to and satisfies along the way.
Since this relation does not change under reparameterization and differs across , the paths are infinitely many and distinct up to reparameterization.
Proof of Proposition 2
Proposition 2. Let solve, starting from and ,
Then has the same joint distribution as the interpolant and its tokenwise times advanced by for every , and converges in distribution to as .
Proof. For each , let solve with , and let . Since position of the interpolant is , differentiating in gives
Since is a function of , taking the conditional mean given and using (11) gives
Hence, for any smooth test function with compact support, the law of satisfies
so is a weak solution of the continuity equation
The law of in the proposition solves the same equation with the same initial condition, since its velocity is . Since this solution is unique, the two laws coincide for every . Finally, as , so for every . Since has the law of , it converges in distribution to .
Proof of Proposition 3
Proposition 3. Consider the objective
Any population minimizer satisfies almost surely under the training distribution.
Proof. Since is the one-hot encoding of , the loss at position can be written as an inner product,
Conditioning on and using , which holds because is independent of , its expectation is
Adding and subtracting and summing over positions gives
where is the entropy. The first term does not depend on , and the second term is nonnegative and vanishes exactly when for every . Thus, minimizes if and only if
almost surely under the training distribution. □
Support of the diffusion forcing training distribution
We show that the training distribution of in (13) has support , so that every state of an asynchronous flow lies in it regardless of the scheduler.
Proof. Fix and . Since is independent of and , each position of the interpolant is distributed as
independently across positions. Since , has a positive density on . Moreover, each independently, so has a positive density on . The joint distribution of therefore has a positive density on , and its support is the closure of this set,
which contains every state of an asynchronous flow. □
Proof of Corollary 4
Corollary 4. Let be a prescribed schedule. Then the asynchronous flow in (15) reduces to
and satisfies for every , with as $t\uparrow 1.
Proof. Since does not depend on , every interpolant follows the same tokenwise times with . The condition in (11) therefore holds for every , and
Substituting into (15) gives (16), whose velocity coincides with that of Proposition 2. Hence, has the law of , which is for every , and converges in distribution to as .
Proof of Proposition 5
Proposition 5. On any interval where the asynchronous flow (15) admits unique solutions, its solution operator , called the asynchronous flow map, satisfies
Proof. Let , where . Integrating the coupled dynamics (15) from to gives the first identity.
For , both and solve the coupled dynamics on with the same state at . Uniqueness of solutions therefore gives .
Proof of Proposition 6
Proposition 6. Let be the asynchronous flow map. Then and satisfy:
(i) The transition is recovered as
(ii) On the diagonal ,
(iii) For , the semigroup condition (17) is equivalent to
where
with and the tokenwise times along the flow from .
Proof. Throughout, let and for , so that . By (18), the averages and satisfy
Solving (38) for and (37) for proves (i).
Next, we write as a weighted average of the denoiser along the trajectory. Recall the coupled dynamics,
The product rule gives
Integrating (39) and using the tokenwise time dynamics,
Substituting (40) and (41) into (38),
For (ii), substituting the diagonal of (18) into (19) gives
These diagonal values are also the limits of the off-diagonal ones. Since the weights in (42) are nonnegative,
and (37) gives
For (iii), fix and write . By (37), the tokenwise time components of the direct map and the composed map are
which agree if and only if the identity for in (22) holds. When this identity holds, let denote the common tokenwise times, and write
Applying (i) to and then to gives the token state component of the composed map,
where
Both are nonnegative since , so . By (i), the token state component of the direct map is , so subtracting it from (43) leaves
Since , the two token state components then agree if and only if the identity for in (22) holds. Hence the semigroup condition (17) holds if and only if both identities hold, which proves (iii).
Proof of Proposition 7
Proposition 7. For , consider the population objective
The exact and have zero loss. Conversely, any zero-loss solution satisfies the diagonal and composition identities (21) and (22) almost surely under the training distribution.
Proof. Setting and , the induced map is the exact flow map by Proposition 6 (i), and Proposition 6 (ii) gives
so both diagonal terms vanish. Moreover, Proposition 6 (iii) gives
so both off-diagonal terms vanish as well, and .
Conversely, all four terms are nonnegative and , so if and only if each term vanishes almost surely under the training distribution. Since KL divergence and the squared norm vanish exactly when their arguments agree, and stop-gradient leaves the target values unchanged, this gives
The first two are (21), and expanding the targets in the last two gives (22), which hold for and .
Proof of Corollary 8
Corollary 8. Let be a prescribed schedule, with fixed to its exact time component and training states sampled from the corresponding interpolants. Then (24) reduces to
up to a constant independent of . Its minimizers coincide with the zero-loss solutions of Proposition 7 with fixed by the schedule.
Proof. We first show that the tokenwise time terms of (24) vanish. By assumption, is fixed to the exact , and since for every state, (18) gives
The diagonal tokenwise time term therefore vanishes. Substituting into (23) gives
so the off-diagonal tokenwise time term vanishes as well.
We next rewrite the token state terms. Since by Corollary 4, the states in (24) are , and and in are evaluated at , , and . The off-diagonal token state term is thus the first term of (25). For the diagonal term, is one-hot and has zero entropy, so
which is the tokenwise cross-entropy of (13) with . Applying the decomposition (30) with gives
where sums over token positions and does not depend on . Hence (25) equals (24) plus . Since (24) is nonnegative and vanishes at the exact by Proposition 7, the minimum of (25) is , and its minimizers are exactly the zero-loss solutions of Proposition 7.
Scheduler Parameterization
Schedule construction
All schedulers in our experiments take the common form of (27),
This form sends as whenever is bounded away from zero near . Writing for along the trajectory, each tokenwise time satisfies
and integrating over gives
Since and , the tokenwise time never decreases and stays in for every . Moreover, if for some and all , then gives
All schedulers in our experiments satisfy this condition, since the top- confidence and entropy-bounded schedulers use , and the power schedules have as .
The schedulers differ in what depends on. Let
denote the predicted clean-token distribution, its confidence, and its entropy, respectively. All position indices and statistics below refer to generated tokens, and fixed context tokens remain unchanged.
Power schedules. Several of the schedulers below use the power schedule
(Figure 6). Its rate takes the common form, with depending only on ,
Figure 6. Power schedules with (left) and their speeds (right). Darker curves denote higher-priority positions, with exponents ranging from (darkest) to (lightest), and the dashed line denotes the synchronous schedule.
which tends to one as . Given an order over the generated positions, let denote the normalized rank of position , with zero denoting the highest priority, and set
so that the exponents range from to and positions with higher priority are denoised faster. Except for the synchronous schedule, we use , so that the exponents range from to $3. We use the following power schedules.
Synchronous. Setting gives and , which recovers the synchronous schedule of Lee et al. [29].
Left-to-right. is the normalized position of token among the generated positions.
Algorithmic order for Sudoku. , where is the round in which cell is resolved and is the last round (Appendix § 10.§ 3). Cells resolved in the same round share the same exponent.
Initial entropy. Following Bansal & Sanghavi [5], is the rank of position by increasing entropy of the initial prediction , kept fixed during generation, so that positions with lower entropy are denoised earlier. thus depends on the token states only through the initial state .
The following two schedulers instead set from the current predictions at every step.
Top- confidence scheduler. Following MaskGIT (Chang et al. [10]), we advance the most confident positions faster, where is a fraction of the generated positions and thus increases over time. Let denote the confidence standardized over the generated positions, and let be its quantile at level , so that a fraction of positions have . We set
where is the speed of the remaining positions, is the additional speed for the selected ones, and is a temperature that controls how sharply changes around the threshold.
Entropy-bounded scheduler. Following EB-Sampler [7], we order token positions by decreasing confidence or increasing entropy. Let denote the -th position in this order. For the first positions, we sum their entropies and subtract the largest:
We assign larger to positions before exceeds a threshold , using a sigmoid to decrease gradually:
Here controls the additional speed, and controls how sharply it decreases around .
Tokenwise time rate reparameterization
Taking the logarithm of (46) at each position gives
Dividing the first by the second shows that defined in § 4.3 is the average of along the trajectory weighted by ,
We now derive the relation between and the diagonal rate . By Proposition 6(i), the left side of (28) is the tokenwise time component of the flow map. Its derivative in at is therefore .
Writing the power in (28) as an exponential,
and differentiating with respect to by the chain and product rules gives
At , we have and , so the term involving vanishes and
Differentiating both sides of (28) at therefore gives
On the other hand, the scheduler of (27) at the same state is
Subtracting the two expressions gives
Since every entry of is positive for and , the diagonal identity holds if and only if . Moreover, taking the squared norm of (54) gives
so the implemented term only removes these positive weights, and both terms vanish exactly when .
Experiment Details
Training details
Architecture and hyperparameters. All models on the same task share the architecture, and the tokenwise time is embedded per position for the aFLM teacher and aFMLM. Per-task hyperparameters are listed in Table 4. We train all models with AdamW [33] with , , , and no weight decay. The learning rate is linearly warmed up to over 2,500 steps and kept constant afterwards, and gradients are clipped to norm 1.0. We use bfloat16 mixed precision and keep an exponential moving average of the weights with decay 0.9999, which is used for evaluation. All models are trained on NVIDIA A100 and H200 GPUs.
| TinyGSM / GSM8K | Sudoku | |
| Architecture (DiT) | ||
| Hidden size | 768 | 512 |
| adaLN cond. dim | 128 | 128 |
| Layers | 12 | 8 |
| Attention heads | 12 | 8 |
| FFN multiplier | 4 | 4 |
| Dropout | 0.1 | 0.1 |
| Sequence length | 512 | 180 |
| Vocabulary | 49152 (SmolLM-135M) | 12 |
| Parameters (approx.) | 168M | 29M |
| Teacher training | ||
| Optimizer | AdamW (, , ) | |
| Weight decay | 0 | |
| Gradient clipping | 1.0 | |
| Schedule | Constant w/ linear warmup | |
| Warmup steps | 2 500 | |
| Peak LR | ||
| Total steps | 80,000 | 10,000 |
| Global batch size | 512 | 256 |
| Precision | bfloat16 mixed | |
| EMA decay | 0.9999 | |
| Distillation | ||
| Total steps | 170,000 | 10,000 |
| Peak LR | ||
| Global batch size | 512 | 256 |
| Training steps per curriculum round | 2,000 | 2,000 |
| Initial rollout steps | 32 | 32 |
| Noise scale | 0.1 | 1.0 |
Table 4. Per-task hyperparameters. All models on the same task share the architecture and optimizer, and differ only in the training and inference schedules.
aFLM training. For each task, we train one aFLM teacher with the diffusion forcing loss of (13) for the number of steps in Table 4. The teacher is initialized from MDLM (§ 4.3) unless stated otherwise. We use the aFLM checkpoint at 80k steps, and Figure 3 continues training up to 240k steps only to compare the two initializations. For Alg. order† in Figure 2, we instead train the denoiser only along the algorithmic order of Appendix § 10.§ 3.
Distillation. aFMLM (EB), aFMLM (Top-), and aFMLM (Sync.) are initialized from the aFLM teacher and distilled with (24), while aFMLM (Alg. order) is trained from scratch with (25) without a teacher. Following Lee et al. [29], we sample three times by first sampling , then sampling , and setting . With learned schedulers, the state at time is obtained by rolling out the dynamics from time 0 to (§ 4.3). We run this rollout curriculum for five rounds, where the first round rolls out the teacher with 32 steps and each later round rolls out the current flow map with half as many steps, down to 2 steps. The final round with 2 rollout steps continues for the remaining training steps. We set the tokenwise time loss weight in (24) to . For noise downscaling (§ 4.3), we draw the initial noise with on GSM8K both in distillation and at inference, and keep on Sudoku.
Scheduler hyperparameters. For the top- confidence scheduler, we use on all Sudoku difficulty levels and on GSM8K, with . For the entropy-bounded scheduler, we use on all Sudoku difficulty levels and on GSM8K, with throughout. For the prescribed power schedules reported here, we use , giving exponents in , except for the synchronous schedule, which uses . For the initial entropy scheduler, we rank the positions by the entropy of the initial prediction and assign the power schedule with according to this rank.
Evaluation. All models are sampled on a uniform grid of the base time with the given number of steps. On this grid, the aFLM teachers integrate (15), aFMLM jumps between adjacent grid points with , and the tokenwise times evolve according to the scheduler. We evaluate on the full 1,319 GSM8K test questions and 2,000 Sudoku puzzles for each difficulty. For Sudoku, we report exact solving accuracy, and for GSM8K, we execute the generated code and report the accuracy of its output. We measure the training cost in Table 2 on a single H200 GPU.
Baseline details
For Table 1, we take the one-step results reported in each paper and otherwise evaluate the checkpoints released by Deschenaux & Gulcehre [16] and Agarwal et al. [1]. For FMLM+, we report the GSM8K result with MDLM initialization. For DBTM, we report its only one-step GSM8K result, which uses the refinement-loop objective. For AlphaDLM, we report its best one-step result, which the sequence-level objective with its hyperparameter achieves. Since DBTM and AlphaDLM report no one-step results on Sudoku and release no checkpoints, we mark them with –.
Algorithmic ordering for Sudoku
Following Shah et al. [48], we start from the given clues and compute the valid digits of each empty cell under the row, column, and box constraints. At each round, we fill every cell that has only one valid digit or is the only valid location of a digit in its row, column, or box, and we repeat until no cell can be resolved. Figure 7 illustrates the first three rounds. We denote by the round in which cell is resolved and assign the last round to unresolved cells, with if no cell is resolved.
Figure 7. Algorithmic ordering for a hard Sudoku puzzle. Plain digits are the given clues, and labels – indicate the round in which each empty cell is resolved. Cells resolved in the current round are shaded in darker blue, while previously assigned rounds remain visible. Cells still blank after round 3 are resolved in later rounds (not shown).
Details of the analysis in § 5.3
We apply the logit lens after the attention and MLP sublayers of each transformer block, decoding each hidden state with the final layer norm and output head of the model. For each token, we take the earliest sublayer from which its decoded prediction matches the final output and remains so. For the aFLM teacher, we run 128-step EB sampling from the same initial noise and regard a token as decided at the first step where its tokenwise time exceeds 0.95. The four columns of Figure 5 show the first 10%, 50%, 85%, and 100% of the tokens in this order, and Table 5 lists the block of each aFMLM column.
| Model | 10% | 50% | 85% | 100% |
| aFMLM (EB) | 6 | 8 | 11 | 12 |
| aFMLM (Sync.) | 2 | 7 | 10 | 12 |
Table 5. Transformer block (out of 12) at which each column of Figure 5 is taken.
References
- [1]Manan Agarwal, Sheel Shah, Chanhyuk Lee, Jaehoon Yoo, Jerry Huang, Seunghoon Hong, Aditi Raghunathan, Jinwoo Kim, and Nicholas M Boffi. Posterior refinement: Fast language generation via any-order flow maps. arXiv preprint arXiv:2606.24773, 2026. (pages 3, 8, 9, 10, 25)arxiv.org/abs/2606.24773
- [2]Arnold Caleb Asiimwe, William Yang, Sanghyuk Chun, Esin Tureci, and Olga Russakovsky. Depth as time in one-step generative models. arXiv preprint arXiv:2610.03626, 2026. (page 11)arxiv.org/abs/2610.03626
- [3]Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang, Simon Vary, and Patrick Rebeschini. Masked language flow models. arXiv preprint arXiv:2606.27617, 2026. (page 3)arxiv.org/abs/2606.27617
- [4]Alan Baade, Eric Ryan Chan, Kyle Sargent, Changan Chen, Justin Johnson, Ehsan Adeli, and Li Fei-Fei. Latent forcing: Reordering the diffusion trajectory for pixel-space image generation. arXiv preprint arXiv:2602.11401, 2026. (page 2)arxiv.org/abs/2602.11401
- [5]Parikshit Bansal and Sujay Sanghavi. Token time continuous diffusion for language modeling. arXiv preprint arXiv:2607.14106, 2026. (pages 1, 2, 3, 9, 22)arxiv.org/abs/2607.14106
- [6]Nora Belrose, Igor Ostrovsky, Lev McKinney, Zach Furman, Logan Smith, Danny Halawi, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.08112, 2023. (page 11)arxiv.org/abs/2303.08112
- [7]Heli Ben-Hamu, Itai Gat, Daniel Severo, Niklas Nolte, and Brian Karrer. Accelerated sampling from masked diffusion models via entropy bounded unmasking. arXiv preprint arXiv:2505.24857, 2025. (pages 2, 8, 23)
- [8]Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137–1155, 2003. (page 1)
- [9]Alankrita Bhatt, Mukur Gupta, Germain Kolossov, and Andrea Montanari. Generating from discrete distributions using diffusions: Insights from random constraint satisfaction problems. arXiv preprint arXiv:2603.20589, 2026. (page 2)arxiv.org/abs/2603.20589
- [10]Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11305–11315. IEEE, 2022. (pages 8, 22)arxiv.org/abs/2202.04200
- [11]Arnav Chavan, Chanhyuk Lee, Udbhav Bamba, Jaehoon Yoo, Seunghoon Hong, Aditi Raghunathan, and Jinwoo Kim. Large flow language models. arXiv preprint arXiv:2610.28559, 2026. (pages 2, 3, 8)arxiv.org/abs/2610.28559
- [12]Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081–24125, 2024. (pages 2, 4, 5, 9)arxiv.org/abs/2407.01392
- [13]Yuxin Chen, Chumeng Liang, Hangke Sui, Ruihan Guo, Chaoran Cheng, Jiaxuan You, and Ge Liu. Langflow: Continuous diffusion rivals discrete in language modeling. arXiv preprint arXiv:2604.11748, 2026. (pages 1, 3)
- [14]Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, and Xinchao Wang. dparallel: Learnable parallel decoding for dllms. arXiv preprint arXiv:2509.26488, 2025. (page 2)arxiv.org/abs/2509.26488
- [15]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. (page 9)arxiv.org/abs/2110.14168
- [16]Justin Deschenaux and Caglar Gulcehre. Language modeling with hyperspherical flows. arXiv preprint arXiv:2605.11125, 2026. (pages 9, 10, 25)arxiv.org/abs/2605.11125
- [17]Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H Richemon d, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, et al. Continuous diffusion for categorical data. arXiv preprint arXiv:2211.15089, 2022. (page 3)arxiv.org/abs/2211.15089
- [18]Floor Eijkelboom, Grigory Bartosh, Christian A Naesseth, Max Welling, and Jan-Willem van de Meent. Variational flow matching for graph generation. Advances in Neural Information Processing Systems, 37:11735–11764, 2024. (page 3)arxiv.org/abs/2406.04843
- [19]Nima Fathi, Torsten Scholak, and Pierre-André Noël. Unifying autoregressive and diffusion-based sequence generation. arXiv preprint arXiv:2504.06416, 2025. (page 2)arxiv.org/abs/2504.06416
- [20]Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J Zico Kolter, and Andrew Gordon Wilson. From entropy to epixepity: Rethinking information for computationally bounded intelligence. arXiv preprint arXiv:2601.03220, 2026. (pages 1, 2)arxiv.org/abs/2601.03220
- [21]Nikita Gushchin, Dmitry Baranchuk, and Alexander Korotin. Alpha diffusion language models: Factorization alone is not the problem. arXiv preprint arXiv:2609.38066, 2026. (pages 9, 10)arxiv.org/abs/2609.38066
- [22]Keya Hu, Linlu Qiu, Yiyang Lu, Hanhong Zhao, Tianhong Li, Yoon Kim, Jacob Andreas, and Kaiming He. Elf: Embedded language flows. arXiv preprint arXiv:2605.10938, 2026. (pages 1, 3)arxiv.org/abs/2605.10938
- [23]Zijing Hu, Yunze Tong, Fengda Zhang, Junkun Yuan, Jun Xiao, and Kun Kuang. Asynchronous denoising diffusion models for aligning text-to-image generation. arXiv preprint arXiv:2510.04504, 2025. (page 2)arxiv.org/abs/2510.04504
- [24]Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. arXiv preprint arXiv:2506.08009, 2025. (page 2)arxiv.org/abs/2506.08009
- [25]Pascal Jutras-Dubé, Patrick Pynadath, Jeremy Lu, Yuan Gao, and Ruqi Zhang. Sticky jump diffusions: A unifying view of masked, continuous, and hybrid diffusion. arXiv preprint arXiv:2607.10951, 2026. (page 3)
- [26]Jaeyeon Kim, Kulin Shah, Vasilis Kontonis, Sham Kakade, and Sitan Chen. Train for the worst, plan for the best: Understanding token ordering in masked diffusions. arXiv preprint arXiv:2502.06768, 2025a. (pages 1, 5, 8, 9)arxiv.org/abs/2502.06768
- [27]Minseo Kim, Chenfeng Xu, Coleman Hooper, Harman Singh, Ben Athiwaratkun, Ce Zhang, Kurt Keutzer, and Amir Gholami. Cdlm: Consistency diffusion language models for faster sampling. arXiv preprint arXiv:2511.19269, 2025b. (page 2)arxiv.org/abs/2511.19269
- [28]Seo Hyun Kim, Sunwoo Hong, Hojung Jung, Youngrok Park, and Se-Young Yun. Klass: Kl-guided fast inference in masked diffusion models. Advances in Neural Information Processing Systems, 38:92267–92301, 2026. (page 2)arxiv.org/abs/2511.05664
- [29]Chanhyuk Lee, Jaehoon Yoo, Manan Agarwal, Sheel Shah, Jerry Huang, Aditi Raghunathan, Se-unghoon Hong, Nicholas M Boffi, and Jinwoo Kim. Flow map language models: One-step language modeling via continuous denoising. arXiv preprint arXiv:2602.16813, 2026. (pages 1, 2, 3, 8, 9, 10, 22, 24)arxiv.org/abs/2602.16813
- [30]Nayoung Lee, Kartik Sreenivasan, Jason D. Lee, Kangwook Lee, and Dimitris Papailiopoulos. Teaching arithmetic to small transformers. arXiv preprint arXiv:2307.03381, 2023. (page 2)arxiv.org/abs/2307.03381
- [31]Yihao Liang, Ze Wang, Hao Chen, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Emad Barsoum, Zicheng Liu, and Niraj K. Jha. Cd4lm: Consistency distillation and adaptive decoding for diffusion language models. arXiv preprint arXiv:2601.02236, 2026. (page 2)DOI
- [32]Bingbin Liu, Sebastien Bubeck, Ronen Eldan, Janardhan Kulkarni, Yuanzhi Li, Anh Nguyen, Rachel Ward, and Yi Zhang. Tinysmm: achieving >80% on gsm8k with small language models. arXiv preprint arXiv:2312.09241, 2023. (page 9)arxiv.org/abs/2312.09241
- [33]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. (page 24)
- [34]Shen Nie, Fengqi Zhu, Zəbin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. Advances in Neural Information Processing Systems, 38:50608–50646, 2026. (pages 1, 2)arxiv.org/abs/2502.09992
- [35]Yueming Pan, Rouyu Feng, Qi Dai, Yuqi Wang, Wenfeng Lin, Mingyu Guo, Chong Luo, and Nanning Zheng. Semantics lead the way: Harmonizing semantic and texture modeling with asynchronous latent diffusion. arXiv preprint arXiv:2512.04926, 2025. (page 2)arxiv.org/abs/2512.04926
- [36]Vassilis Papadopoulos, Jérémie Wenger, and Clément Hongler. Arrows of time for large language models. arXiv preprint arXiv:2401.17505, 2024. (pages 1, 2)arxiv.org/abs/2401.17505
- [37]William Peebles and Saining Xie. Scalable diffusion models with transformers. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4172–4182. IEEE, 2023. (page 8)arxiv.org/abs/2212.09748
- [38]Peter Potaptchik, Jason Yim, Adhi Saravanan, Peter Holderrieth, Eric Vanden-Eijnden, and Michael S Albergo. Discrete flow maps. arXiv preprint arXiv:2604.09784, 2026. (page 3)arxiv.org/abs/2604.09784
- [39]Patrick Pynadath, Jiaxin Shi, and Ruqi Zhang. Candi: Hybrid discrete-continuous diffusion models. arXiv preprint arXiv:2510.22510, 2025. (pages 3, 9, 10)arxiv.org/abs/2510.22510
- [40]Bingshuo Qian and Xiang Cheng. Learning when to denoise: Optimizing asynchronous schedules for latent diffusion. arXiv preprint arXiv:2606.19662, 2026. (pages 1, 2, 4)arxiv.org/abs/2606.19662
- [41]Andrei Rekesh, Jarrid Rector-Brooks, and Cheng-Hao Liu. Inverse-confidence sampling for continuous diffusion language models. In ICML 2026 Workshop on Foundations of Deep Generative Models: Understanding Memorization, Generalization, and Reasoning, 2026. (page 3)
- [42]Daan Roos, Oscar Davis, Floor Eijkelboom, Michael Bronstein, Max Welling, Ismail Ilkan Ceylan, Luca Ambrogioni, and Jan-Willem van de Meent. Categorical flow maps. arXiv preprint arXiv:2602.12233, 2026. (page 3)arxiv.org/abs/2602.12233
- [43]David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. arXiv preprint arXiv:2402.09470, 2024. (page 2)arxiv.org/abs/2402.09470
- [44]Subham S Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:130136–130184, 2024. (pages 1, 8, 9, 10)arxiv.org/abs/2406.07524
- [45]Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, and Volodymyr Kuleshov. The diffusion duality. Proceedings of machine learning research, 267:52584, 2025. (pages 9, 10)arxiv.org/abs/2506.10892
- [46]Yair Schiff, Omer Belhasin, Roy Uziel, Matan Rusanovsky, Ran Zilberstein, Marianne Arriola, Gilad Turok, Guanghan Wang, Volodymyr Kuleshov, and Michael Elad. Clock diffusion: Efficient semi-autoregressive continuous diffusion language models. arXiv preprint arXiv:2610.00894, 2026. (pages 2, 3)arxiv.org/abs/2610.00894
- [47]Johannes Schusterbauer, Ming Gui, Yusong Li, Pingchuan Ma, Felix Krause, and Björn Ommer. Denoising, fast and slow: Difficulty-aware adaptive sampling for image generation. arXiv preprint arXiv:2604.19141, 2026. (page 2)arxiv.org/abs/2604.19141
- [48]Kulin Shah, Nishanth Dikkala, Xin Wang, and Rina Panigrahy. Causal language modeling can elicit search and reasoning capabilities on logic puzzles. arXiv preprint arXiv:2409.10502, 2024. (pages 2, 9, 25)arxiv.org/abs/2409.10502
- [49]Claude E Shannon. Prediction and entropy of printed english. Bell system technical journal, 30(1):50–64, 1951. (pages 1, 2)
- [50]Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data. Advances in neural information processing systems, 37:103131–103167, 2024. (page 1)arxiv.org/abs/2406.04329
- [51]Sophia Tang and Shivi Wang. Discrete beckmann transport models for one-step language modeling and reasoning. arXiv preprint arXiv:2609.15903, 2026. (pages 9, 10)arxiv.org/abs/2609.15903
- [53]Tong Wu, Zihao Fan, Xiao Liu, Hai-Tao Zheng, Yeyun Gong, Jian Jiao, Juntao Li, Jian Guo, Nan Duan, Weizhu Chen, et al. Ar-diffusion: Auto-regressive diffusion model for text generation. Advances in Neural Information Processing Systems, 36:39957–39974, 2023. (pages 2, 3)arxiv.org/abs/2305.09515
- [54]Tianwei Yin, Qiang Zhang, Richard Zhang, William T. Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. arXiv preprint arXiv:2412.07772, 2024. (page 2)arxiv.org/abs/2412.07772






