Latent Reasoning in Continuous Space for Unified Multimodal Models

Introduction

Recent advances in generative models have reshaped visual content creation, achieving high quality and fine-grained controllability in text-to-image synthesis, image editing, and instruction-following generation [1, 2, 3, 4, 5]. However, generation-only models still struggle with complex semantics that require language-level planning, such as attribute binding [6, 7, 8] or world modeling [9, 10]. Within this landscape, unified multimodal models (UMMs) [11, 12, 13, 14, 15], have emerged as a compelling paradigm which integrates an image-generation module into large language models (LLMs) that contain rich textual semantics.

Despite their success, prior UMMs fail to scale at test time: unlike in LLMs, increasing the reasoning budget does not yield the smooth performance gains observed on LLM benchmarks (Figure 1). We argue that this failure stems from a fundamental modality mismatch: reasoning is performed in a discrete token space, whereas image generation operates in a continuous latent space. Since discrete chains struggle to encode the dense and spatially-grounded information that image synthesis demands [16, 17], reasoning collapses into surface-level prompt manipulation, which is precisely the narrow regime in which existing methods provide gains. This observation motivates our central claim: to extend the benefits of reasoning beyond this regime, reasoning itself must operate in a continuous space.

To this end, we introduce LARC (Latent Reasoning in Continuous space), a framework that mitigates the modality gap by aligning text and image reasoning in a continuous latent space. Building on Coconut [18], which performs latent reasoning in LLMs without explicit tokens, we extend the idea to multimodal generation. While we introduce reinforcement learning (RL) on the latent reasoning system, RL training faces two challenges: (i) Coconut-style reasoning is deterministic, whereas policy-gradient methods require stochastic rollouts; and (ii) latent reasoning traces lack an explicit evaluator, making it difficult to assess whether they are useful for generation. We address both challenges with a two-stage recipe. Stage 1 employs curriculum supervised fine-tuning that progressively replaces token-level reasoning with continuous hidden states, thereby injecting controlled stochasticity and restoring the rollout diversity required for policy gradients. Then, Stage 2 applies reinforcement learning driven by an information-gain reward, which quantifies how much each latent step improves the predictive distribution over the target image, together with a collapse-prevention reward that preserves trajectory diversity. Together, these constitute a self-evolving loop that requires neither ground-truth chains nor external evaluators.

In experiments, we instantiate LARC on BAGEL [15] with 1M interleaved samples drawn from FLUX-Reason [19], WEAVE [20], and ScaleEdit [21]. On three text-to-image benchmarks—GenEval [22], DPG-Bench [23], and OneIG-Bench [24]—LARC surpasses both the base UMMs and recent reasoning-augmented variants by substantial margins, e.g., +7.6 absolute points on OneIG-Bench over Uni-CoT [13]. More importantly, to our knowledge, LARC is the first reasoning-augmented UMM that exhibits clear test-time scaling: under budget forcing, doubling the latent reasoning compute yields monotonic gains across all three benchmarks.

Our contributions are summarized as follows:

• We identify a discrete–continuous modality mismatch between reasoning and generation as the root cause of the limited gains and absent test-time scaling in current reasoning-augmented UMMs, and argue that reasoning must operate in a continuous space.

• We propose LARC, the first framework to perform reasoning in the same continuous latent space as image generation within a unified multimodal model, enabling test-time scaling in UMMs.

• We design a self-evolving training recipe that (i) extends Coconut-style latent reasoning with controlled stochasticity to make it amenable to RL, and (ii) introduces an information-gain reward coupled with a collapse-prevention reward, eliminating the need for ground-truth latent chains or an external critic.

LARC: Latent Reasoning in Continuous Space

Overview

We propose LARC, a framework that enables UMMs to perform image generation through latent reasoning on continuous space. We first describe the two-stage training pipeline of LARC. We then elaborate on the design of each component in the following sections. Specifically, in § 2.2, we first define the interleaved trajectory format. Then, in § 2.3, we introduce the curriculum supervised fine-tuning. Finally, in § 2.4, we present reinforcement learning using information-gain reward derived from conditional mutual information.

Interleaved Latent Reasoning

To enable continuous-space reasoning compatible with policy-gradient optimization, we formulate the reasoning trajectory as an alternating sequence of text segments and continuous latent steps, both generated by the UMM backbone. Formally, given an input context $\mathbf{c} = (c_1 \cdots c_T)$ consisting of a text prompt and optionally a source image, the model produces a reasoning trajectory followed by a target image $x$. Following Coconut [18] and subsequent latent reasoning work [27], the model alternates between two modes during trajectory generation: the language mode and the latent mode. In language mode, the model emits text tokens through stochastic sampling, exposing the trajectory to the gradient signal of policy optimization. In the latent mode, the model emits a continuous reasoning step that bypasses text decoding, preserving the rich semantic content of internal representations.

Specifically, the UMM backbone first processes the input embedding $E_T = [e(c_1), \cdots, e(c_T)]$ to produce the last hidden state $H_T = \operatorname{Transformer}(E_T)$, where $e$ is the token embedding function. Once the model produces the special token <latent> at position $t$, the mode turns latent and the embedding of the next position is taken to be $h_t$, the hidden state at position $t$ in the last transformer layer. The mode returns to language after the fixed steps, at which point the trajectory ends and the image module is invoked. Concretely, with $E_t$ denoting the input embedding sequence up to position $t$ and $h_t = H_t[t, :]$ the corresponding hidden state, the next-position embeddings is

At inference, the mode alternation proceeds until the model emits the <eos> token, with each <latent> token marking the transition from a language-mode span to the next latent-mode step. After termination, the full trajectory of hidden states $h(\tau)$ is read by the image module through cross-attention to produce the target image $x$. The resulting reasoning trajectory naturally takes the form

where $s_n$ denotes the $n$-th language-mode span and $l_n$ denotes the $n$-th latent-mode hidden state, each consisting of multiple tokens. Here, the trajectory length $N$ and the length of each language-mode span $|s_n|$ are determined by when the model issues <eos> and <latent>, respectively, rather than fixed by an external schedule.

Type	Method	Single Obj. $\uparrow$	Two Obj. $\uparrow$	Counting $\uparrow$	Colors $\uparrow$	Position $\uparrow$	Attr. Binding $\uparrow$	Overall $\uparrow$
Reference	GPT-4o [28]	0.99	0.92	0.85	0.92	0.75	0.61	0.84
Reference	SD3-medium [29]	0.99	0.94	0.72	0.89	0.33	0.46	0.74
Reference	FLUX.1-dev [30]	0.99	0.81	0.79	0.74	0.20	0.47	0.67
UMM	Janus-Pro [31]	0.99	0.89	0.59	0.90	0.79	0.66	0.80
UMM	Emu3 [14]	0.98	0.71	0.34	0.81	0.17	0.21	0.54
UMM	Show-o2 [12]	1.00	0.87	0.58	0.92	0.52	0.62	0.76
UMM	BAGEL [15]	0.99	0.94	0.80	0.87	0.64	0.63	0.81
UMM	Uni-CoT [13]	0.99	0.96	0.84	0.92	0.57	0.71	0.83
UMM	LARC (ours)	1.00	0.98	0.86	0.93	0.77	0.78	0.88

Stage 1: Supervised Fine-Tuning with Curriculum Learning

As observed in prior work [18], introducing all $N$ latent steps from the beginning of training is unstable, since the latent stream lacks any direct supervision while the model has yet to learn the basic reasoning format. To progressively guide the model toward reasoning in latent space, we perform supervised fine-tuning (SFT) under a curriculum that gradually converts token-level reasoning into latent reasoning. We therefore decompose this stage into a curriculum that activates latent steps progressively, allowing each newly introduced latent step to build upon the trained prefix.

Data construction. For SFT with a curriculum, we collect 1M interleaved chain-of-thought (CoT) samples, providing the input context $c$, a step-by-step reasoning chain, and the target image $x$. We process each reasoning chain by inserting the special token <latent> at its step boundaries, marking the positions where latent steps are introduced.

Curriculum learning. To stabilize training, we adopt a curriculum learning that exposes the model to deeper interleaving over phase [18]. We first suppress all latent steps and train on the text-only trajectory $\tau^{(0)} = (s_1, \cdots, s_N)$, establishing the reasoning format under standard next-token prediction. Subsequently, in phase $k$ for $k = 1, \cdots, N - 1$, we activate only the first $k$ latent steps, ensuring that each newly introduced latent is built on top of the stable prefix. The final phase activates all latent steps, with the learning rate reduced at each phase transition.

Training loss. We optimize the model $\pi_\theta$ with a objective that combines a standard next-token prediction loss $\mathcal{L}_{\mathrm{SFT}}(\cdot)$ over text segments and a flow-matching loss $\mathcal{L}_{\mathrm{flow}}(\cdot)$ on the target image,

where $s_{<n}$ and $l_{<\min(n,k+1)}$ denote the text segments and the active latent steps preceding $s_n$ at phase $k$, respectively. Note that we propagate the gradient through the latent steps without explicit supervision target. This implicit supervision allows the latent steps to encode useful representations automatically, imposing no external bias.

Stage 2: Self-Evolving Reinforcement Learning

Building on the latent reasoning capability established in Stage 1, we further refine the latent of reasoning trace through reinforcement learning (RL), guiding the model toward latent thoughts that are demonstrably useful for image generation. While supervised fine-tuning (SFT) effectively guides the model to produce interleaved trajectories, it remains fundamentally limited at refining the latent as it depends highly on the data quality, even the reasoning chain does not have ground-truth latent annotations. In this stage, we therefore adapt self-rewarding RL with an intrinsic reward derived from the UMM itself, yielding a fully self-evolving framework without any external critic.

Information-Gain reward. Motivated by prior work [33], we define the reward as the Information Gain that a reasoning trace provides to a frozen reference model, measuring how much the latent reasoning improves the prediction of the target image. Concretely, letting $\pi_{\mathrm{ref}}$ denote the model after

Method	NP ↑	T&P ↑	Short ↑	Medium ↑	Long ↑	Overall ↑
Discrete UMM
Janus-Pro [31]	0.557	0.533	0.609	0.548	0.515	0.552
BLIP-o3 [32]	0.719	0.671	0.754	0.712	0.674	0.706
Continuous UMM
Show-o2 [12]	0.825	0.778	0.825	0.819	0.807	0.811
BAGEL [15]	0.776	0.734	0.782	0.769	0.759	0.764
Uni-CoT [13]	0.803	0.748	0.817	0.805	0.776	0.790
LARC (ours)	0.892	0.840	0.887	0.869	0.840	0.866

Stage 1 and marginal density of its image module $p_{\pi_{\mathrm{ref}}}(x\mid\cdot)$, the reward for a sampled trajectory $\tau$ is

where $p_{\pi_{\mathrm{ref}}}(x\mid c,h(\tau))$ is the predictive density when the trace is included and $p_{\pi_{\mathrm{ref}}}(x\mid c)$ is the reasoning-free baseline of the same model. Equation (1) assigns reward exactly when the trace raises likelihood of the target image in reference model, and zero reward when the trace carries no information beyond the context $c$.

Specifically, the Information-Gain reward $r_{\mathrm{IG}}$ is the per-sample form of conditional mutual information, with $\mathbb{E}[r_{\mathrm{IG}}]=I_{\pi_{\mathrm{ref}}}(x;h(\tau)\mid c)\ge0$ under the joint distribution. Maximizing $r_{\mathrm{IG}}$ is therefore equivalent to maximizing the mutual information between the latent reasoning trace and the target image, with the bare context $c$ playing the role of the marginal baseline. However, since the image module of UMM does not provide direct access to log-likelihoods, we thereby estimate $r_{\mathrm{IG}}$ as a paired difference of conditional flow-matching losses under shared noise,

where $\mathcal{L}^{\pi_{\mathrm{ref}}}_{\mathrm{flow}}(x,\xi)$ denotes the conditional flow-matching loss of the reference image module under conditioning $\xi$, with the two evaluations sharing the same noise sample for sample-wise constant cancellation. We provide the full derivation in Appendix § 10.§ 2.

Total reward. To prevent the latent collapse where all $N$ latent steps within a trajectory converge into identical vectors, we add an inter-step regularization for latent diversity,

where $\cos(\cdot,\cdot)$ denotes the cosine similarity between two latent steps, $l_n$ is the $n$-th latent-mode hidden state in trajectory $\tau$, and $N$ is the number of latent steps. The total reward used for policy optimization is then

Policy optimization. We optimize the policy $\pi_\theta$ through reinforcement learning with the following objective:

We adopt GRPO [34] as the optimization algorithm, which estimates advantages from group rollouts and removes an explicit value model. Its simplicity and stability allow a single set of hyperparameters to transfer across all benchmarks. For more optimization details, please see Appendix § 8.

Experiments

We provide an empirical evaluation of LARC by investigating the following questions:

• RQ1: Does LARC improve the performance on various tasks, i.e., image generation, image editing, visual reasoning? (Table 1, 2, 3, 4, 5)

• RQ2: Does LARC enable test-time scaling of UMMs? (Figure 1)

• RQ3: Which component mostly contributes to LARC? (Table 6, 7, 8)

Method	Reasoning space	T2I gen. ↑	Comp. gen. ↑	Image editing ↑	Visual reasoning ↑
Baseline [15]	Discrete	0.81	0.76	20.43	7.5
Token-level Reasoning	Discrete	0.82	0.79	20.81	7.1
Latent Reasoning	Continuous	0.83	0.81	21.46	9.7

Evaluation Setup. We evaluate Unified Multimodal Models (UMMs) on various tasks including Text-to-Image (T2I) generation, image editing, and multimodal understanding. Specifically, we choose GenEval [22], DPG-Bench [23], and OneIG-Bench [24] for T2I benchmarks and CompBench [25] for image editing benchmark. For multimodal understanding, we choose MME [35], MMB [36], MMMU [37], and MIRA [26]. For T2I generation and image editing tasks, we follow the evaluation setup of target benchmarks, which mainly reports model-based metrics, including MLLM-as-a-Judge. We provide detailed evaluation setups in Appendix § 8.§ 2.

Training Setup. In the main experiments, we trained LARC on BAGEL [15]. We perform supervised fine-tuning and reinforcement learning on a fixed 1M scale of Chain-of-Thought (CoT) data. We construct the dataset sampled from open-source datasets, e.g., FLUX-Reason [19], WEAVE [20], ScaleEdit [21]. We provide more details in Appendix § 8.§ 1.

Baselines. We compare LARC with image generation models, e.g., Stable Diffusion3 [29] and FLUX.1-dev [19], and foundation models in unified multimodal modeling, e.g., BAGEL [15], Show-o2 [12], Emu3 [14], and Janus-Pro [31], and reasoning-enhanced UMM, Uni-CoT [13].

Image Generation Experiments

We evaluate the performance on various Text-to-Image (T2I) and image-editing benchmarks. Here, we mainly compare LARC with other UMMs and generation models, reporting score for each subcategory and overall averages. Specifically, we evaluate our LARC in (i) GenEval [22], (ii) DPG-Bench [23], (iii) OneIG-Bench [24], and (iv) CompBench [25].

LARC improves text-to-image generation performance. As shown in Table 1, LARC outperforms or consistently matches competing baselines across the evaluated objectives, demonstrating broad gains in T2I performance. The improvement is most pronounced on Attribute Binding, with a +15%/pp increase over the baseline. This subtask requires the joint handling of visual realization and compositional planning, suggesting that multimodal generation-aware latent reasoning is particularly effective for such compositionally demanding scenarios.

LARC improves prompt-following fidelity. We validate the prompt-following fidelity of LARC against competing baselines on DPG-Bench [23] and OneIG-Bench [24]. As shown in Table 2 and 3, LARC achieves consistent improvements over competing baselines. On DPG-Bench, the overall score of LARC improves by +5.8% (85.07 → 90.04) over the base model BAGEL. On OneIG-Bench, LARC yields a +10.2%/pp gain over the same base model. These improvements indicate that our latent reasoning is particularly effective in prompt-following generation. The consistent gain across every subcategory further suggest that LARC generalizes broadly across diverse prompt types.

Type	Method	Global ↑	Entity ↑	Attribute ↑	Relation ↑	Other ↑	Overall ↑
Reference	SD3-medium [29]	87.90	91.01	88.83	80.70	88.68	84.08
Reference	FLUX.1-dev [30]	82.10	89.50	88.70	91.10	89.40	84.00
	Discrete UMM
	Janus-Pro [31]	86.90	88.90	89.40	89.32	89.48	84.19
	Emu3 [14]	85.21	86.68	86.84	90.22	83.15	80.60
UMM	Continuous UMM
	Show-o2 [12]	89.00	91.78	89.96	91.81	91.64	86.14
	BAGEL [15]	88.94	90.37	91.29	90.82	88.67	85.07
	Uni-CoT [13]	87.58	89.14	91.62	91.09	86.97	85.56
	LARC (ours)	92.25	91.89	93.00	94.70	92.35	90.04

LARC improves image editing. As shown in Table 4, LARC consistently outperforms all baselines across every metric. LARC achieves a +10.2% gain on LC-T over the base model BAGEL (20.434 → 22.609), indicating substantially stronger adherence to editing instructions. Pixel-level quality also improves markedly, with PSNR increasing from 24.3 to 25.7 and LPIPS decreasing from 0.069 to 0.049. These results suggest that continuous latent reasoning enables the model to plan spatially grounded edits, particularly for localized attribute modification.

Method	LC-T $\uparrow$	LC-I $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Janus-Pro [31]	19.769	0.801	21.192	0.856	0.097
Emu3 [14]	19.536	0.803	22.036	0.868	0.092
Show-o2 [12]	20.350	0.831	24.661	0.919	0.065
BAGEL [15]	20.434	0.842	24.370	0.917	0.069
Uni-CoT [13]	20.215	0.837	23.925	0.904	0.088
LARC (ours)	22.509	0.880	25.710	0.953	0.049

Text Generation Experiments

While the previous section established the benefits of latent reasoning for image generation and editing, an open question is whether such continuous reasoning also benefits text generation tasks that share the same UMM backbone. We address this question by evaluating LARC on perception-heavy benchmarks (MME, MMB) and a visual reasoning benchmark (MIRA), comparing against the base model BAGEL and Uni-CoT, a recent UMM with discrete token-level reasoning.

As shown in Table 5, LARC improves over both BAGEL and Uni-CoT across all benchmarks. The most pronounced gain appears on MIRA (7.5 → 11.7), a benchmark requiring visual chain-of-thought reasoning that explicitly demands multi-step intermediate inference. The substantial margin over Uni-CoT, which performs reasoning through discrete tokens, supports our hypothesis that latent reasoning provides richer intermediate representations than its discrete counterpart. We attribute this to the fact that continuous latent steps preserve information that discretization through a fixed vocabulary inevitably collapses, an effect amplified in tasks where reasoning must integrate fine-grained visual cues. Gains on MME and MMB further confirm that the continuous reasoning capability transfers positively to perception-heavy tasks without sacrificing general visual understanding. This indicates that the additional latent reasoning supervision introduced during training does not degrade the original visual perception ability of the UMM backbone, an important property for unified models that serve both generation and understanding.

Method	MME [35] $\uparrow$	MMB [36] $\uparrow$	MMMU [37] $\uparrow$	MIRA [26] $\uparrow$
Janus-Pro [31]	1567.1	79.2	41.0	4.9
Emu3 [14]	1487.3	67.4	31.6	4.1
Show-o2 [12]	1620.5	79.3	48.9	6.9
BAGEL [15]	1687.0	85.0	55.3	7.5
Uni-CoT [13]	1691.7	85.4	55.5	10.6
LARC (ours)	1725.3	88.1	56.2	11.7

Test-time Scaling

As shown in Figure 1, we investigate the effect of latent reasoning at test-time. LARC exhibits monotonically increasing performance on image generation, image editing, and visual reasoning as the latent budget grows, a behavior that is absent in prior reasoning-augmented UMMs. Furthermore, LARC consistently outperforms best-of-N sampling under matched compute budgets across all three tasks. We attribute this scaling behavior to continuous latent reasoning operating in the same space as image generation, allowing additional compute to refine useful representations rather than producing redundant discrete tokens.

Component Ablations

We conduct ablation studies to assess the contribution of each design choice in LARC, using GenEval, OneIG-Bench, CompBench, and MIRA as representative benchmarks.

Effect of reasoning space. In Table 6, We compare token-level reasoning and latent reasoning initialized from the same base model. Switching from token-level to latent reasoning yields consistent gains across all four benchmarks. The improvement is particularly pronounced on visual reasoning (+2.6 on MIRA) and image editing (+0.65 on LC-T), confirming that continuous representations better capture the dense, spatially-grounded information required for generation tasks. Notably, token-level reasoning slightly degrades visual reasoning performance compared to the baseline, suggesting that discrete reasoning suffers the modality gap.

Effect of latent reasoning strategy In Table 7, we compare two latent reasoning strategies: Naive latent reasoning without interleaved text segments and our proposed interleaved design. While the Naive approach provides reasonable gains, interleaved latent reasoning achieves better compositional generation (+0.02 on OneIG-Bench) and image editing (+0.44 on LC-T) with comparable performance elsewhere. We attribute this to the text segments maintaining stochastic rollout diversity required for policy gradient optimization, while the latent steps encode the continuous representations that image generation demands.

Method	T2I gen. ↑	Comp. gen. ↑	Image editing ↑	Visual reasoning ↑
Baseline [15]	0.81	0.76	20.43	7.5
Naive	0.83	0.79	21.02	9.9
Interleaved (ours)	0.83	0.81	21.46	9.7

Ablation study for training stages. In Table 8, we conduct ablation study for each training stage. While Stage 1 (SFT with curriculum) yields consistent improvements over the baseline across all tasks, Stage 2 (self-evolving RL) further boosts performance substantially, with T2I generation improving from 0.83 to 0.88 and visual reasoning from 9.7 to 11.7. This confirms that Stage 2 successfully refines latent traces beyond what SFT can achieve, and validates the self-evolving design that requires neither ground-truth reasoning chains nor an external critic.

Method	T2I gen. $\uparrow$	Comp. gen. $\uparrow$	Image editing $\uparrow$	Visual reasoning $\uparrow$
Baseline [15]	0.81	0.76	20.43	7.5
Stage 1	0.83	0.81	21.46	9.7
Stage 1 & 2	0.88	0.87	22.51	11.7

Related Work

Test-time Scaling in LLMs

Test-time scaling is a model behavior where additional inference compute improves task performance in large language models (LLMs) [39, 40]. Chain-of-Thought (CoT) prompting [41] has emerged as a primary driver of test-time scaling in LLMs by decomposing problems into intermediate textual reasoning steps. Subsequent work has internalized this capability through supervised fine-tuning [42] and reinforcement learning [34]. Inference-time search methods further expand the reasoning space through tree-based [43] and trajectory-based [44] exploration. Recognizing that natural language fails to encapsulate all forms of reasoning [45], recent paradigms shift reasoning directly into latent space [18, 27, 46]. Despite these advances, prior efforts to transfer test-time scaling to UMM image generation [12, 13, 14, 15, 31] have relied on discrete reasoning tokens and failed to exhibit the predictable scaling observed in language reasoning. We attribute this failure to the modality mismatch between discrete reasoning tokens and continuous image generation. Motivated by this observation, we propose LARC, which performs reasoning entirely in continuous space and becomes the first UMM to achieve test-time scaling for image generation.

Image Generation Models

Image generation has emerged as one of the most influential research directions in modern deep learning, advancing along two complementary paradigms. Diffusion models [47, 48] have become the dominant paradigm for text-to-image generation, synthesizing images through iterative denoising in systems such as Stable Diffusion [1], DALL-E 2 [2], and Imagen [3]. In parallel, autoregressive approaches predict visual tokens from discrete vocabularies established by vector-quantized tokenizers [49, 50], scaling from DALL-E [4] to recent transformer-based models including LlamaGen [5] and VAR [51]. Building on these advances, unified multimodal models (UMMs) integrate visual understanding and generation within a single architecture, typically extending a pretrained LLM with image tokenization to inherit strong language priors and enable cross-task knowledge transfer [52].

Early work pursues fully discrete tokenization with next-token prediction: Chameleon [53] pioneers an early-fusion design trained from scratch, and Emu3 [14] scales this paradigm across understanding and generation. Hybrid formulations combine autoregressive text decoding with diffusion-based image generation: Show-o [12, 54] uses discrete diffusion for images, and Transfusion [55] applies continuous diffusion over image patches. A separate line of work decouples visual representations for the two tasks: Janus [11] and Janus-Pro [31] introduce separate visual encoders under a shared transformer backbone, and BAGEL [15] extends decoupling to the transformer itself through a Mixture-of-Transformer-Experts architecture [56, 57] combined with rectified flow [58] for image generation. Our work is orthogonal to these architectural advances, introducing a reasoning capability that operates over an existing UMM backbone, with BAGEL being adopted as our base models.

Reinforcement Learning in LLMs

Reinforcement learning has long served as the standard machinery for policy optimization. Trust-region methods such as TRPO [59] and its first-order successor PPO [60] stabilize policy updates through constrained or clipped objectives. More recent work adapts policy optimization to language modeling. GRPO [34] eliminates the value network by estimating advantages from group rollouts, and follow-up methods [61, 62, 63] further improve this group-relative formulation. These advances have powered substantial capability gains in LLMs, ranging from instruction-following in InstructGPT [64] to answer-verifiable reasoning in DeepSeek-R1 [65], where GRPO drives chain-of-thought reasoning at scale. However, for reasoning in image generation, no well-defined ground-truth reasoning trajectory exists. This precludes direct supervision over the reasoning chain and motivates the RL-based formulation adopted in LARC. We instantiate this formulation with GRPO, which is widely used in reinforcement learning tasks, to train reasoning chains using outcome-level rewards. This bypasses the need for chain-level annotations standard in text-based CoT, yet absent in our setting.

Conclusion

We introduced LARC, a latent reasoning framework that performs reasoning in continuous space for unified multimodal models (UMMs). By representing reasoning steps as continuous hidden states rather than discrete tokens, LARC alleviates the modality gap that has prevented prior reasoning-augmented UMMs from inheriting the test-time scaling of large language models. Our two-stage pipeline pairs curriculum supervised fine-tuning with a self-evolving reinforcement learning stage driven by an information-gain reward, requiring no external critic. Instantiated on BAGEL with 1M interleaved samples, LARC achieves state-of-the-art text-to-image performance with a 7% gain over BAGEL and is the first reasoning-augmented UMM to exhibit clear test-time scaling for image generation. This finding bridges a long-standing modality gap in UMMs and opens a path toward UMMs that match the inference-time scalability of modern reasoning LLMs.

Broader Impact Statement

LARC advances the reasoning capability of unified multimodal models, with the goal of producing more controllable and inference-time-scalable image generation systems. We discuss the broader impacts below.

Positive impacts. By enabling test-time scaling for image generation, LARC allows users to allocate additional inference compute to obtain higher-fidelity outputs, particularly for prompts that require fine-grained compositional reasoning such as attribute binding and spatial layout. This controllability supports applications in design assistance, accessibility tools for visual communication, education, and scientific visualization, where prompt fidelity directly impacts utility. The latent reasoning paradigm also offers a unified mechanism for transferring reasoning capability across modalities, potentially reducing the need for modality-specific reasoning pipelines and lowering the engineering cost of building multimodal systems.

Potential risks. As with other text-to-image generation systems, LARC inherits risks common to large generative models, including the potential for generating misleading or harmful imagery, amplifying biases present in pretraining data, and being misused for the production of deceptive content such as misinformation or non-consensual depictions. The improved prompt-following fidelity demonstrated in our experiments increases generation quality across legitimate uses, and also amplifies these dual-use concerns, since outputs that more faithfully reflect adversarial prompts are correspondingly more capable of producing targeted harmful content. We do not introduce new training data or new modalities beyond those already established in BAGEL and the public datasets we use, so risks specific to dataset provenance are inherited rather than newly created.

Mitigation and responsible deployment. We recommend that downstream applications of LARC combine the model with established safeguards, including content moderation classifiers, watermarking of generated images, prompt-level safety filters, and auditing of training data for licensing and bias. For research deployments, releasing only model weights without bundled deployment infrastructure helps preserve the ability of downstream practitioners to apply context-appropriate safety measures. We will release LARC under a license that prohibits use for generating disinformation, non-consensual intimate imagery, or content targeting protected groups, and we encourage the community to develop evaluation protocols specifically for reasoning-augmented generative models, where the relationship between reasoning fidelity and harmful output remains understudied.

Environmental considerations. Training LARC requires reinforcement learning over a multibillion-parameter UMM backbone, which incurs non-trivial compute and associated carbon footprint. We mitigate this by initializing from a pretrained BAGEL checkpoint rather than training from scratch, and by using a curriculum that converges in fewer steps than naive joint training of all latent reasoning steps. Future work on parameter-efficient adaptation of latent reasoning, such as adapter-based or low-rank variants, would further reduce the resource cost of deploying this paradigm at scale.

Acknowledgments

This work was partly supported by Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2019-II190075, Artificial Intelligence Graduate School Program (KAIST); RS-2022-II220184, Development and Study of AI Technologies to Inexpensively Conform to Evolving Policy on Ethics; RS-2024-00509279, Global AI Frontier Lab), and RLWRLD Inc. This research was also supported by the NVIDIA Academic Grant Program using A100 GPUs on Brev.

The authors thank Suhyeok Jang, Junbeom Kim, and Seonghu Jeon for their insightful discussions and feedback on the experiments.

Implementation Details

Training Details

We adopt BAGEL [15] as the base model and perform two-stage learning, namely (1) Supervised Fine-tuning (SFT) with curriculum and (2) self-evolving Reinforcement Learning (RL). In Stage 1, we freeze the vision tokenizer and jointly train the understanding expert and the generation expert of the UMM backbone. We follow the curriculum schedule described in § 2.3, where the number of activated latent steps is increased from $0$ to $N$ over $N+1$ phases with the learning rate decayed at every phase transition. In Stage 2, we initialize the policy $\pi_\theta$ from $\mathcal{M}_{\mathrm{SFT}}$ and freeze a copy of the same model as the reference $\pi_{\mathrm{ref}}$. Only the understanding expert is updated during RL, since the generation expert is reused for reward estimation through the shared-noise flow-matching loss. Detailed hyperparameters are summarized in Table 9. All training experiments are conducted with $8 \times$ NVIDIA A100s, taking approximately 300 GPU hours in Stage 1 and 96 GPU hours in Stage 2.

Hyperparameter	Stage 1 (SFT)	Stage 2 (RL)
Optimizer	AdamW	AdamW
Algorithm	-	GRPO
Learning rate	$1\mathrm{e}{-5}$	$1\mathrm{e}{-6}$
Global batch size	-	64
Max number of tokens	$6 \times10^{5}$	-
Weight decay	$10^{-2}$	0.0
Training steps	$1.2 \times10^{3}$	$10^{3}$
Group size $G$	–	8
Clipping $\epsilon_{\mathrm{clip}}$	–	$2 \times10^{-1}$
KL coefficient $\beta$	–	$10^{-3}$
Diversity weight $\lambda_{\mathrm{div}}$	–	$10^{-1}$

Evaluation Details

We evaluate LARC across three categories of benchmarks following the official protocol of each benchmark. For inference, we apply greedy sampling and generate one image per prompt unless specified otherwise.

Text-to-Image generation. We evaluate the models on GenEval [22], DPG-Bench [23], and OneIG-Bench [24]. GenEval automatically verifies compositional properties of generated images via an object-detection pipeline, reporting accuracy across six sub-tasks, e.g., single object, two objects, counting, colors, position, and attribute binding. DPG-Bench evaluates 1,065 dense prompts through an MLLM-as-a-Judge protocol, providing scores across global, entity, attribute, relation, and other categories. OneIG-Bench spans multiple dimensions of text-to-image (T2I) evaluation, including prompt-image alignment, text rendering, and reasoning, with scores partitioned by prompt type (NP, T&P) and prompt length (short, medium, long). Specifically, we report alignment score for OneIG-Bench.

Image editing. We evaluate the models on CompBench [25], which features 3,200 complex instruction-guided editing tasks built through an MLLM-human collaborative framework. Specifically, we report multi-object editing tasks in Table 4. Following the official protocol, we report local CLIP-Text (LC-T) score between the predicted foreground and the description, local CLIP-Image (LC-I) similarity between the predicted foreground and the ground-truth image, along with pixel-level metrics PSNR, SSIM, and LPIPS.

Multimodal understanding. We evaluate on MME [35], MMBench [36], MMMU [37], and MIRA [26]. MME measures perception and cognition across 14 sub-tasks, while MMBench and MMMU benchmark broad multimodal reasoning across diverse disciplines. MIRA targets visual chain-of-thought reasoning with 546 carefully curated problems spanning 20 task types, where successful reasoning requires generating intermediate visual representations.

For API-based baselines, we report results using the model version gpt-4o-2024-08-06 for GPT-4o [28]. We adopt VLMEvalKit [66] as the unified evaluation toolkit for multimodal understanding tasks, and use the official codebase released by each benchmark for T2I generation and image editing evaluation.

Data Statistics

We construct a 1M scale reasoning dataset by sampling from three open-source datasets, namely FLUX-Reason [19], WEAVE [20], and ScaleEdit [21]. The detailed composition is summarized in Table 10. Following the prior works [27, 67], we balance the mixture so that text-to-image, image editing, and interleaved reasoning are jointly represented, ensuring that the latent reasoning trajectory generalizes across diverse generation tasks. For each sample, we insert the special token <latent> at every reasoning step boundary as described in § 2.3, marking the positions where latent steps are progressively activated by the curriculum.

Source dataset	Task	# samples
FLUX-Reason [19]	Text-to-Image	500K
WEAVE [20]	Text-to-Image	100K
ScaleEdit [21]	Image Editing	400K
Total	–	1M

Self-Evolving Reinforcement Learning

Why Self-Rewarding RL?

In this section, we discuss why standard reward-based reinforcement learning (RL) paradigms are not directly applicable to latent reasoning in UMMs, motivating the self-rewarding formulation adopted in LARC.

Inapplicability of preference-based methods. Direct Preference Optimization (DPO) [68] and its diffusion variants such as Diffusion-DPO [69] and CaPO [70] optimize the policy from paired preference data $(\tau^w, \tau^l)$, where $\tau^w$ is preferred over $\tau^l$. This paradigm presumes the existence of a meaningful preference order between trajectories. In our setting, however, each trajectory $\tau$ contains continuous latent steps $h(\tau)$ that lie outside any human-interpretable space, ruling out direct human annotation over the reasoning trace. External reward models for image generation [71, 72, 73] also operate on the rendered image $\mathbf{x}$ rather than the latent chain, providing no signal that distinguishes between two latent traces that yield comparable images. A recent study [74] further reports that DPO-based optimization in autoregressive image generation tends to overfit to in-domain preference distributions, limiting generalization beyond the curated preference set. These observations indicate that preference-based methods are structurally misaligned with the latent reasoning regime, which lacks both pairwise preference labels and a natural notion of pairwise dominance over hidden states.

Limitations of critic-based methods. Proximal Policy Optimization (PPO) [75] sidesteps the preference assumption but introduces a separate value network that estimates the expected return at each position, which roughly doubles the trainable parameters and adds non-trivial training overhead at the scale of UMMs. More fundamentally, fitting an accurate critic over the latent reasoning trajectory requires a dense external supervision signal, which is precisely what is absent in our setting. Without ground-truth latent annotations, the critic must rely on an external image-quality estimator or a learned reward model, both of which inherit the same evaluation gap described above.

Self-rewarding via information gain. We point out two requirements that the optimization framework must satisfy. (1) The reward should be derivable from the model itself, since no external evaluator can faithfully assess latent reasoning traces. (2) The policy gradient estimator should avoid an additional value network to keep training tractable on top of the UMM backbone. In LARC, the information-gain reward satisfies the first requirement by reusing the frozen reference $\pi_{\mathrm{ref}}$ obtained from Stage 1, scoring each trajectory through the change in target-image likelihood. Additionally, GRPO [34] satisfies the second by replacing the value network with group-relative advantages estimated from multiple rollouts.

Derivation of the Information-Gain Reward

We provide the full derivation of the Information-Gain reward $r_{\mathrm{IG}}$ introduced in § 2.4. Recall that the reward is defined as

where $h(\tau)$ denotes the sequence of latent hidden states produced along trajectory $\tau$, and $x$ is the target image. Taking the expectation of Equation (5) under the joint distribution $(x,c,\tau)$ recovers the conditional mutual information,

Maximizing $r_{\mathrm{IG}}$ is therefore equivalent to maximizing the mutual information between the latent reasoning trace and the target image, with the bare context $c$ playing the role of the marginal baseline. The non-negativity in Equation (6) follows from Jensen’s inequality applied to the log-ratio, which guarantees that the expected reward is bounded below by zero regardless of the policy.

Since the generation expert of UMM does not provide closed-form access to the marginal density $p_{\pi_{\mathrm{ref}}}(x\mid\cdot)$, we approximate the log-likelihood through the conditional flow-matching objective. Following the rectified flow formulation of BAGEL [15], the negative log-likelihood admits the bound

where $\xi$ denotes the conditioning variable, $\mathcal{L}^{\pi_{\mathrm{ref}}}_{\mathrm{flow}}$ is the conditional flow-matching loss evaluated under $\pi_{\mathrm{ref}}$, and $C(x)$ is a sample-wise constant that depends only on the target image and the noise schedule. Substituting Equation (7) into Equation (5) for both terms yields

Crucially, the constant $C(x)$ cancels exactly when the two flow-matching losses are evaluated under the same noise sample $\eta\sim\mathcal{N}(0,I)$ and the same time step $t\sim\mathcal{U}[0,1]$, since both terms inherit the identical perturbation. We therefore implement the reward as a paired evaluation under shared randomness, as detailed in Algorithm 1. This shared-noise estimator yields an unbiased and low-variance approximation of Equation (5), requiring no additional reward model and reusing the frozen $\pi_{\mathrm{ref}}$ that is already available from Stage 1.

Algorithm 1 Self-Evolving Reinforcement Learning of LARC

Require: Frozen reference $\pi_{\mathrm{ref}}=\mathcal{M}_{\mathrm{SFT}}$, dataset $\mathcal{D}$, hyperparameters $G$, $\epsilon_{\mathrm{clip}}$, $\beta$, $\lambda_{\mathrm{div}}$

1: Initialize $\pi_\theta\leftarrow\pi_{\mathrm{ref}}$

2: for each RL step do

3: $\pi_{\theta_{\mathrm{old}}}\leftarrow\pi_\theta$

4: Sample $(c,x)\sim\mathcal{D}$ and rollout $\{\tau_i\}_{i=1}^{G}\sim\pi_{\theta_{\mathrm{old}}}(\cdot\mid c)$

5: for $i=1,\ldots,G$ do

6: $r_{\mathrm{IG}}(\tau_i)\leftarrow\mathcal{L}^{\pi_{\mathrm{ref}}}_{\mathrm{flow}}(x,c)-\mathcal{L}^{\pi_{\mathrm{ref}}}_{\mathrm{flow}}(x,c,h(\tau_i))$ ▷ Shared noise

7: $R_i\leftarrow r_{\mathrm{IG}}(\tau_i)+\lambda_{\mathrm{div}}\cdot r_{\mathrm{div}}(\tau_i)$

8: end for

9: $A_i\leftarrow(R_i-\mu_R)/(\sigma_R+\epsilon)$ for $i=1,\ldots,G$

10: Update $\theta$ by maximizing $J_{\mathrm{GRPO}}(\theta)$ ▷ See Equation (9)

11: end for

12: return $\mathcal{M}_{\mathrm{LARC}}\leftarrow\pi_\theta$

Policy Optimization

We instantiate the policy gradient optimization in Equation (4) using GRPO [34], which estimates advantages from group rollouts and removes the need for a separate value network. For each context $c$ sampled from $\mathcal{D}$, we draw $G$ trajectories $\{\tau_i\}_{i=1}^{G}$ from the old policy $\pi_{\theta_{\mathrm{old}}}$, compute the total reward $R_i$ for each rollout, and normalize within the group to obtain the advantage $A_i$. The policy is then updated by maximizing the clipped surrogate objective

where $\rho_i(\theta)=\frac{\pi_\theta(\tau_i\mid c)}{\pi_{\theta_{\mathrm{old}}}(\tau_i\mid c)}$ is the importance ratio. The full training procedure is summarized in Algorithm 1.

Additional Experimental Results

Detailed Analysis on Test-time Scaling

We provide an additional analysis on performance and computational cost incurred by latent reasoning at test-time. We measure the per-sample clock-time on a single NVIDIA A100 with batch size 1, varying the number of images $N$ and latent steps $L$ from 1 to 10. We compare LARC against best-of-$N$ sampling on the base model BAGEL [15], which represents the standard test-time scaling baseline that draws $N$ independent samples and selects the best one. We analyze clock time (seconds) of test-time scaling in Table 11.

Method	$N=1$	$N=2$	$N=4$	$N=6$	$N=8$	$N=10$
BAGEL [15] + Best-of-$N$	46.5	92.8	187.1	275.5	367.7	469.0
	$L=1$	$L=2$	$L=4$	$L=6$	$L=8$	$L=10$
LARC (ours)	46.8	46.9	47.0	47.3	47.6	48.0

Note that LARC only requires single-pass while best-of-$N$ demands iterative sampling. As shown in Table 11, LARC incurs only a marginal overhead as the number of latent steps increases, requiring additional 1.2 seconds at $L=10$ compared to $L=1$. Notably, combined with the results shown in Figure 1, LARC outperforms best-of-$N$ on every scale with highly efficient computational cost, performing consistent test-time scaling behavior. We attribute this efficiency to the fact that latent reasoning extends only a small portion of the forward pass through the understanding expert, whereas best-of-$N$ requires repeated invocation of the full generation expert. These results confirm that continuous latent reasoning offers a practically efficient route to test-time scaling for UMMs.

Additional Ablation Study

Ablation study for latent diversity reward. We investigate the role of the collapse-prevention reward $r_{\mathrm{div}}$ introduced in Equation (3). Without $r_{\mathrm{div}}$, we observe that the policy quickly collapses to trivial trajectories where all $N$ latent steps converge to nearly identical hidden states, eliminating the benefit of multi-step reasoning. To quantify this phenomenon, we report the average pairwise cosine similarity among latent steps along with task performance under varying $\lambda_{\mathrm{div}}$ in Table 12.

$\lambda_{\mathrm{div}}$	Cos. Sim.	GenEval $\uparrow$	OneIG-Bench $\uparrow$	MIRA $\uparrow$
0.0	0.97	0.83	0.81	9.8
0.05	0.87	0.85	0.84	10.9
0.1 (ours)	0.79	0.88	0.87	11.7
0.2	0.71	0.85	0.85	11.4
0.5	0.65	0.82	0.82	10.6

As shown in Table 12, removing the diversity reward ($\lambda_{\mathrm{div}}=0$) leads to a pairwise cosine similarity of 0.97 across latent steps, indicating severe collapse where every step encodes nearly redundant information. This collapse degrades all three benchmarks, with MIRA dropping from 11.7 to 9.8. Setting $\lambda_{\mathrm{div}}=0.1$ yields the best trade-off. These results validate that the diversity regularizer is essential for preventing latent collapse and confirm that the chosen weight balances exploration of distinct latent directions with exploitation of informative ones.

Qualitative Results

Limitations and Future Work

While LARC demonstrates strong empirical gains and clear test-time scaling on various generation tasks, several directions remain open for future exploration. First, our framework is currently instantiated on UMMs whose image module is a flow-matching head, which generates the image in continuous space. Extending the same principle to UMMs with discrete space generation or other generative heads is a natural next step. Second, although LARC enables effective test-time scaling, the interleaved latent reasoning trajectory introduces additional computation at inference, leading to latency overhead compared to the base UMM. Reducing this overhead through efficient decoding strategy, distillation, or adaptive latent allocation is a promising research direction. We leave these extensions to future work.