JUICE: Unified Multimodal Modeling in Pretrained Representation Spaces

Key Takeaways

T1. Unified model: one flow matching model generates both images and text; with its own noise level for each modality, the same weights handle every task. § 4

T2. Pretrained encoder spaces: both modalities are generated as continuous states of frozen encoders; at equal guidance, image quality holds and captions lose only exact wording. § 5.2

T3. Joint training: adding joint denoising, with both modalities noisy, to the conditional tasks improves image generation and captioning alike. § 5.3

T4. Encoder choice: semantic encoders beat VAE latents and pixels; SigLIP2, aligned with language, helps captioning most, and Flan-T5 gives the best text space. § 5.4

T5. Scaling behavior: every metric improves with model size, but rankings can flip: DINO leads image generation at 179M and SigLIP2 at 2.1B. § 5.5

T6. Joint reasoning: sampled together, image and text adjust to each other, so a drawn route and its written moves agree without a fixed order. § 6.4

Introduction

Unified multimodal models aim to understand and generate both images and text within a single model. Most current systems, however, treat the two modalities asymmetrically. In the dominant design, a pretrained autoregressive language model generates text and drives a diffusion or flow matching head that generates images [1, 2]; other models quantize images and generate both modalities as discrete tokens [3, 4]. Either way, the two modalities are generated by different processes or through a quantizer, and the joint distribution is factorized in a fixed order: an output that contains both an image and text is produced one modality after the other, and the first cannot be revised in light of the second. Vision is also usually added to a model that was first pretrained on language, although controlled studies of multimodal pretraining find that late integration makes models lean on language priors and that training the modalities jointly from the start works better [5]. This suggests a different design: a model that treats both modalities alike, generates them with one process, and learns them together from the start.

Such a design needs two things: a space in which one process can generate both modalities, and a source of knowledge for a backbone trained without a language model. Frozen pretrained encoders provide both. Their spaces are continuous for images and text alike and carry visual and linguistic knowledge learned at scale, and recent studies show that they are good generative targets for vision [6, 7, 8] and for language [9], although they were never built for generation. In these spaces, one flow matching process generates both modalities with the same objective, the same features serve as targets and as context, and a model of the joint distribution also contains its conditionals. Give each modality its own noise level and the two levels span a square (Figure 2). Text-to-image generation, captioning, and question answering hold one modality clean and move along an edge. Joint generation cuts through the middle, where image and text take shape together. To build a working model on this idea, we had to settle four questions that existing unified models leave open: whether text should be generated as continuous states or kept as tokens, which denoising tasks to train on, which encoder spaces to read and write, and whether these answers still hold as the model grows.

We introduce JUICE (Joint Unified multimodal modeling In Continuous Encoder spaces; Figure 1), a unified model that works directly in the continuous spaces of frozen pretrained image and text encoders. A single flow matching backbone, MM-RiT, trained from scratch, connects the image and text streams through bidirectional joint attention [10], and each stream has its own noise scale and schedule. We train MM-RiT on both the edges and the interior of the square, with conditional and joint denoising. At inference, a task is a path through the square: keeping some streams clean ($t = 1$) and generating the others gives text-to-image generation, captioning, visual question answering, or coupled joint generation (§ 4).

Vector (PDF)

Our pretraining studies (§ 5) change one design choice at a time and keep the rest of the recipe fixed. (i) Generative frameworks. At a shared guidance scale, JUICE scores 48.41 GenEval and a model trained only for text-to-image generation scores 47.86, while all three frameworks that write text as tokens fall below that model. Captions generated as continuous states lose some exact wording but match the images as well as autoregressive captions. (ii) Training task mixtures. Training should cover the regions that sampling visits: joint and conditional denoising together outperform either one alone, beyond the effect of supervising more targets. (iii) Representation spaces. The VAE reconstructs better than the semantic encoders, yet generation and captioning both work better in the semantic spaces, and Flan-T5 states beat larger text embedding models. (iv) Scaling. From 179M to 2.1B backbone parameters, SigLIP2 [11], which is aligned with language, writes better captions at every size, while the early lead of WebSSL-DINO [12] (DINO) in image generation disappears.

With these answers in hand, we train two 2.1B models that pair Flan-T5 with SigLIP2 or with DINO (§ 6). Working at $224 \times224$ and starting from no language model, each uses one set of weights for compositional image generation, generation that needs world knowledge, and visual question answering. The two reach 0.87 and 0.88 GenEval, and the SigLIP2 model reaches 85.22 on DPG-Bench, 0.45 on WISE [13], and 73.1 on VQAv2.

Because JUICE models the joint distribution, it can also reason jointly across modalities: with coupled sampling, the image and its text are generated together, each conditioning on the other’s evolving state. We adapt JUICE to two planning tasks, mazes with multiple solutions [14] and visual spatial planning [15], in which it draws a route on the map and writes the matching moves. Sampled independently, the two outputs often disagree on mazes: the model draws one valid route and writes out a different one, and joint success is only 54.6%. Generating one modality first repairs this, but the result then depends on the order (95.2% with the image first, 85.3% with the text first), and nothing says in advance which order to use. Coupled sampling reaches 93.7% without making that choice. In the two cases we trace, whichever output settles later rewrites itself to match the other. On visual spatial planning, JUICE solves 99.8% of maps at the training sizes, and 94% and 69% at two larger sizes it never saw in training.

Our main contributions are:

1. Joint modeling in pretrained representation spaces. We propose JUICE, which models the joint distribution of images and text in the continuous spaces of frozen pretrained encoders with one flow matching Transformer trained from scratch. With a noise level for each modality, every task is a path through one square, with no fixed generation order.

2. Systematic design studies. § 5 reports controlled pretraining runs on how text is generated, which tasks are trained, and which encoder spaces are used, plus a scaling study from

179M to 2.1B parameters. We summarize what they show as four takeaways for continuous joint modeling.

3. Scaling and joint reasoning. A single set of 2.1B weights handles compositional image generation, generation that needs world knowledge, and visual question answering (§ 6). On two planning tasks, coupled sampling gets the drawn route and the written moves to agree without a fixed generation order (§ 6.4).

Related Work

Unified multimodal models. Most unified models are hybrids that combine different representation spaces and generative processes (Table 1). Chameleon [16] models discrete text tokens and quantized image codes autoregressively. Transfusion [17] pairs autoregressive text with diffusion over continuous image latents, and Show-o [20] pairs autoregressive text with diffusion over discrete image tokens. MMaDA [3] and Muddit [4] use discrete masked diffusion for both modalities, while D-DiT [18] combines discrete masked diffusion for text with continuous flow for images. D-DiT and Muddit start from pretrained image generators, Show-o and MMaDA from pretrained language models, and Chameleon and Transfusion are trained from scratch. In most of these models, the two modalities are generated by different processes, and outputs that contain both are produced in a fixed order. JUICE instead generates both with one flow matching process in frozen representation spaces and is trained from scratch; § 5.2 compares it with representative formulations under the same recipe.

Model	Text representation	Text process	Image representation	Image process
Chameleon [16]	Discrete tokens	AR	Discrete codes	AR
Transfusion [17]	Discrete tokens	AR	VAE latents	Diffusion
Show-o2 [1]	Discrete tokens	AR	VAE latents	FM
Tuna-2 [2]	Discrete tokens	AR	Pixels	FM
MMaDA [3]	Discrete tokens	Masked diffusion	Discrete codes	Masked diffusion
Muddit [4]	Discrete tokens	Masked diffusion	Discrete codes	Masked diffusion
D-DiT [18]	Discrete tokens	Masked diffusion	VAE latents	FM
UniDiffuser [19]	CLIP text features	Diffusion	VAE + CLIP features	Diffusion
BLM [8]	Discrete tokens	AR	Pretrained repr.	FM
Scale-RAE [7]	Discrete tokens	AR	Pretrained repr.	FM
JUICE	Pretrained repr.	FM	Pretrained repr.	FM

Generation in pretrained representation spaces. Representation autoencoders (RAEs) [6, 7] show that diffusion works well on frozen pretrained visual features, whose semantic structure makes them good targets. Beyond Language Modeling (BLM) [8] takes up this idea and trains a unified backbone from scratch with flow matching over frozen visual features, while language is still predicted autoregressively as discrete tokens. Other unified models pick different visual targets. Show-o2 [1] still generates VAE latents, which it processes semantically. TUNA [21] aligns unified features built from VAE latents, and Tuna-2 [2] removes the visual encoders altogether and generates pixels. For text, ELF [9] applies continuous flow matching directly in pretrained text representation spaces, though only to generate text on its own. Each of these works generates only one modality in a continuous representation space: BLM still predicts text as tokens, and ELF generates no images. JUICE generates both modalities in pretrained spaces with one process, so the same representations serve as generation targets and as cross-modal context, and § 5.4 and § 5.5 study which spaces suit this joint use and how the answer changes with scale.

Joint and conditional multimodal modeling. Separate noise schedules for each modality [19, 22] provide a mathematical framework for a single model to capture the marginal, conditional, and joint distributions of images and text. Existing implementations, however, need specialized components. UniDiffuser [19] diffuses CLIP embeddings and needs a separate autoregressive GPT-2 to decode text; OneFlow [23] models text with discrete edit flows. OmniFlow [24] uses rectified flow for generation from any modality to any other. Its MMDiT backbone starts from Stable Diffusion 3, and its text latent comes from a VAE built on Flan-T5 and TinyLlama. JUICE keeps both modalities as continuous encoder states with one state per token, decodes text with a lightweight parallel decoder, and trains one backbone on them from scratch. In this setting, joint denoising improves conditional generation by more than its extra targets explain (§ 5.3), and coupled sampling lets the two streams settle on the same one of several valid solutions (§ 6.4).

Continuous diffusion for language. Early work diffused learned word embeddings and rounded them back to tokens (Diffusion-LM [25], DiffuSeq [26]); later work improved the likelihood objectives (CDCD [27], Plaid [28]). Flow matching over categorical distributions, which places each token on the probability simplex, offers another continuous route to discrete data such as text [29, 30]. A second line of work diffuses directly in the representation spaces of pretrained language models, either after compressing sequences into fixed latents (LD4LG [31]) or over contextual encoder states (TEncDM [32], ELF [9]). JUICE follows the second line for its text stream and diffuses contextual encoder states. Unlike these text only models, it generates the states jointly with images, and § 5.2 compares this choice with discrete text under one recipe.

Method

As shown in Figure 1, JUICE operates directly in the continuous representation spaces of frozen pretrained encoders, using a single flow matching Transformer (MM-RiT) trained from scratch. The same model supports generation, understanding, and joint tasks over images and text simply by varying which representations are provided as clean inputs and which are generated from noise. § 4.1 reviews flow matching, § 4.2 presents the formulation, and § 4.3 to § 4.5 describe the architecture, training procedure, and inference; Appendix § 10.§ 7 provides implementation details.

Preliminaries

Flow matching [33] trains a network to predict a velocity field that, over time, transports a noise distribution to the data distribution. Given a clean representation $z$ and Gaussian noise $\epsilon\sim\mathcal{N}(0,\sigma^2\mathbf{I})$, we use the linear interpolation

where $t=0$ is pure noise, $t=1$ is clean data, and the target velocity is $v^* = z - \epsilon$. Following JiT [34], MM-RiT predicts the clean state $\hat{z}_\theta$ rather than velocity. This places the network output directly in the representation space used by the decoders. At any noisy state $z^t$, the predicted clean data $\hat{z}_\theta$ defines the velocity field via

where $d(t) = \max(1-t, 0.05)$ floors the denominator near $t=1$ for numerical stability. For the linear path, velocity matching is therefore a denoising loss on $\hat{z}_\theta$ weighted by $1/d(t)^2$.

Formulation

JUICE models the joint distribution of an image representation $z_i$ and a text representation $z_t$, both extracted from frozen pretrained encoders (§ 4.3). Each modality has its own time $t_m \in[0,1]$, and a single network predicts both clean states from the pair of noisy states:

The pair of times lives in a unit square, the 2D time domain of Figure 2. Setting $t_m=1$ makes modality $m$ a clean condition, and any $t_m<1$ means it is still being generated. A task is thus just a path through the square. Text-to-image generation follows the edge $t_t=1$ and captioning the edge $t_i=1$, while joint generation cuts through the interior. Question answering sets times per token, so the image and the question stay at $t=1$ while the answer is generated (Appendix § 10.6.2). We therefore train on the edges with conditional denoising and on the interior with joint denoising, so that every region sampling passes through is covered.

The formulation introduces three requirements, each addressed by one design component. First, because the two representation spaces differ in dimensionality and feature statistics, each modality has its own projections, noise scale, and noise schedule (§ 4.3 and § 4.4). Second, because any token can be either a condition or a generation target, time tokens in the sequence and a time embedding for each token tell the network the role of each position (§ 4.3). Third, because one set of weights must serve both conditional directions and joint generation, MM-RiT is trained on a mixture of conditional and joint denoising (§ 4.4).

Architecture

Representation Encoders. Given a paired image $x_i$ and text $x_t$, frozen pretrained encoders extract features that are normalized to align their scales with the diffusion noise [35]:

SigLIP2 [11] and Flan-T5-Small [36] are our default image and text encoders, and § 5.4 looks at other choices. We do not finetune the encoders. Their spaces already encode visual and linguistic knowledge learned at scale, and if they stay fixed, the same features can be used both as targets to generate and as context to condition on.

Multimodal Representation Diffusion Transformer (MM-RiT). MM-RiT implements $f_\theta$ in Equation (2): it receives both streams at independently specified noise levels $(t_i,t_t)$ [19] and predicts both denoised clean states, which we convert to velocities for training (velocity matching loss) and inference (ODE solver). To accommodate the distinct image and text representation spaces, each MM-RiT block uses separate Q/K/V projections for each modality, concatenates the resulting sequences for one bidirectional joint attention operation, and then applies separate output projections and feedforward networks [7, 8, 37]. Time tokens in the sequence and additive time embeddings for each token expose the noise level of each modality and each token, so the network can distinguish clean conditioning positions ($t=1$) from generation targets ($t<1$).

Representation Decoders. Fixed decoders map the generated representations back to pixels and tokens at inference:

For images we use the Scale-RAE decoder that belongs to the image encoder [7]. For text, a lightweight parallel decoder maps all generated states to tokens in one forward pass, which is much faster than autoregressive decoding; the pretraining study and the final models each use one decoder of this design. All decoders are trained separately and stay fixed during MM-RiT training. Appendix § 10.§ 5 describes them and compares parallel with autoregressive decoding.

Training

Noise and Schedules for Each Modality. Image and text representations differ in dimensionality, scale, and geometry, and noise settings that work for one may not work for the other, so each modality gets its own. First, the Gaussian noise scale is set separately ($\sigma_i=1$, $\sigma_t=2$ by default) and matched to the statistics of each space after normalization. Second, the noise levels follow a separate distribution for each modality: for modality $m$, we draw a base time $u_m$ from a logit normal distribution and transform it by

For images, we set the shift to $\alpha_i = \sqrt{n_i d_i}/4096$, where $n_i$ is the number of image tokens and $d_i$ their dimension. This moves probability mass toward noisier states. Text is not shifted ($\alpha_t=1$). Features of high dimension can keep much of their information at moderate noise levels, so the shift makes the model see states dominated by noise more often [6, 10].

Joint and Conditional Training. With these schedules, MM-RiT is trained on a mixture of three denoising tasks (Algorithm 1). For each pair of image and text from a training set $D$, we draw a task with the mixture weights $(\pi_J, \pi_{T2I}, \pi_{I2T})$: joint denoising draws the noise levels of both modalities independently, $t_m \sim p_m(t)$ with $p_m$ the distribution of Equation (5), and corrupts both streams; conditional $T2I$ keeps the text clean at $t_t=1$ and corrupts only the image; and conditional $I2T$ does the opposite, keeping the image clean at $t_i=1$. In the time domain of § 4.2, joint denoising samples the interior ($t_i<1$, $t_t<1$) and the conditional tasks lie on its edges [19]. The conditional tasks match the clean conditions used at inference. Joint denoising is what teaches the model the joint distribution, which coupled sampling and the guidance reference for image conditions need (§ 4.5). In pretraining, the velocity matching loss covers only corrupted target positions, so clean conditioning positions and text padding are left out; $\lambda_i$ and $\lambda_t$ balance the two modalities.

Inference

Conditional and Joint Generation. At inference, conditioning inputs stay at $t=1$, and generation targets are integrated from Gaussian noise up to $t=1$. Text-to-image, image-to-text, and joint generation differ only in the path they take through the 2D time domain [19] (Figure 2):

Algorithm 1 JUICE training (one gradient step)

Require: Training set $\mathcal{D}$ of paired images and texts; task mixture weights $(\pi_{\mathrm{J}}, \pi_{\mathrm{T2I}}, \pi_{\mathrm{I2T}})$; MM-RiT $f_\theta$; modality loss coefficients $\lambda_i, \lambda_t$

1. Sample data: $(x_i, x_t) \sim\mathcal{D}$

2. Encode representations: $z_m \leftarrow N_m(E_m(x_m))$ for each modality $m \in\{i,t\}$

3. Sample denoising task: $r \sim\operatorname{Categorical}(\pi_{\mathrm{J}}, \pi_{\mathrm{T2I}}, \pi_{\mathrm{I2T}})$

4. Joint denoising: $t_i \sim p_i(t),\ t_t \sim p_t(t)$; targets $\mathcal{T}=\{i,t\}$

5. Conditional T2I: $t_i \sim p_i(t),\ t_t=1$; targets $\mathcal{T}=\{i\}$

6. Conditional I2T: $t_i=1,\ t_t \sim p_t(t)$; targets $\mathcal{T}=\{t\}$

7. Corrupt target representations: for each $m\in\mathcal{T}$, sample $\epsilon_m \sim\mathcal{N}(0,\sigma_m^2\mathbf{I})$; $z_m^{t_m}\leftarrow(1-t_m)\epsilon_m+t_m z_m$

8. Predict clean states: $(\hat{z}_i,\hat{z}_t)\leftarrow f_\theta(z_i^{t_i},z_t^{t_t};t_i,t_t)$

9. Velocity matching loss: $\mathcal{L}\leftarrow\sum_{m\in\mathcal{T}}\lambda_m\operatorname{mean}_{p,c}\frac{(\hat{z}_{m,pc}-z_{m,pc})^2}{d(t_m)^2}$, over target positions $p$ and channels $c$

10. Update MM-RiT: $\theta\leftarrow\theta-\eta\nabla_\theta\mathcal{L}$

Classifier-Free Guidance. For conditional tasks we apply classifier-free guidance [38] in velocity space: the prediction with the condition is pushed away from a reference prediction without it, which strengthens what the condition adds. No training stage drops the condition, so JUICE has no learned null token. The reference instead replaces the condition with an uninformative input that the model has learned to handle. For an image condition, the image is placed at its noise end, a state the joint task trains the model to see next to text; for a text condition, we use the clean states of the generic caption “a photo.”, which says almost nothing about the image. We use $w=5$ for text-to-image generation, $w=3$ for captioning and question answering, and no guidance for joint generation and planning. Appendix § 10.6.2 gives the formulas, and Table 20 compares the text reference with placing the caption at its noise end.

Table 20: GenEval of the pretraining task mixtures across guidance scales and text references (50k updates, 25 steps). At $w = 1$ the reference is not used. Bold marks each model’s best score with the generic caption; {sup}†` marks collapse into images that look like mosaics. I2T only cannot generate images (0.21 at every scale) and is omitted.

Recipe	Text reference	$w = 1$	$w = 2$	$w = 3$	$w = 4$	$w = 5$
T2I only	generic caption	36.12	47.26	51.23	51.03	47.86
	time swap	36.12	43.38	42.90	38.67	31.34
Multi-task	generic caption	32.36	43.44	44.29	40.85	36.30
	time swap	32.36	24.67	5.75{sup}†`	2.16{sup}†`	0.92{sup}†`
Joint only	generic caption	10.09	21.54	27.43	31.06	31.33
	time swap	10.09	3.30{sup}†`	0.78{sup}†`	0.31{sup}†`	0.25{sup}†`
Mixing (default)	generic caption	31.57	41.79	45.65	47.51	48.41
	time swap	31.57	40.92	43.53	45.06	44.64

Coupled Sampling. In joint generation, both velocity fields are computed from the current joint state $(z_i^{t_i},z_t^{t_t})$ before either modality is updated. We call this coupled sampling. Each stream is thus conditioned, step by step, on the other stream as it evolves. Independent sampling works differently. It integrates each stream separately and leaves the other one at its initial noise for the whole trajectory, which amounts to sampling each modality from its marginal. Sequential sampling first generates one modality completely and then the other conditioned on it, with either the image or the text going first.

Pretraining Study

Putting both modalities in continuous encoder spaces under one denoiser raises the four questions of § 2, and this section takes them in turn. Should text be generated as continuous states or as tokens (§ 5.2)? Which denoising tasks should training cover (§ 5.3)? Which encoder spaces should the model read and write (§ 5.4)? And do the answers change with model size (§ 5.5)? All runs share the setup below, and each study ends with a takeaway.

Setup

All studies share one pretraining recipe and vary one design choice at a time. The reference model has an MM-RiT backbone with 668M trainable parameters over frozen SigLIP2-SO400M image features and Flan-T5-Small text states. It is trained for 50k updates at a global batch size of 4,096 with AdamW ($\beta_1=0.9$, $\beta_2=0.95$, weight decay 0.01) and a constant learning rate of $4\times10^{-4}$ after 5,000 warmup steps, on 38M pairs of images and captions [7, 39] and Cambrian-7M [40], using the default task ratios (55.6% joint, 22.2% T2I, 22.2% I2T; Appendix § 10.6.1). For image generation we report GenEval (GE) [41] and DPG-Bench (DPG) [42, 43], with 25 Euler steps and guidance $w=5$ against the generic caption reference (a fully masked caption for the base D-DiT-style framework; Appendix § 10.6.2). For captioning we report CIDEr [44] and CLIPScore (CLIP, $\times100$) [45] on 5,000 pairs held out from training, with 16 steps and $w=3$ (§ 4.5). Each configuration is trained once. The framework and representation comparisons change complete pipelines, since a representation comes with its own decoder, normalization, and noise settings; they therefore compare design choices as they would be deployed rather than isolated components. Appendices § 10 and § 11 give the full configurations.

Generative Frameworks

The least conventional choice in JUICE is that it generates text as continuous encoder states, not as tokens. We compare it with three frameworks that keep text discrete. All four use the same 668M trunk with two streams and the same data, task ratios, and schedule. The three baselines are Transfusion-style autoregressive (AR) text with flow matching (FM) for images [17], D-DiT-style masked diffusion (MD) for text with flow matching for images [18], and MMaDA-style masked diffusion for both modalities over vector quantized (VQ) image codes [3] (Table 2, Figure 3; Appendix § 11.1.1).

Configuration	Text	Image	Text condition	GE (%)	DPG (%)	CIDEr	CLIP
JUICE	FM	FM	frozen T5 states	48.41	78.85	88.80	80.20
Transfusion-style [17]	AR	FM	token embeddings	44.23	76.84	160.52	80.37
D-DiT-style [18]	MD	FM	token embeddings	34.50	74.35	115.22	79.82
MMaDA-style [3]	MD	MD (VQ)	token embeddings	41.61	66.50	89.53	71.61
Transfusion-style	AR	FM	frozen T5 states	46.07	78.24	158.63	80.36
D-DiT-style	MD	FM	frozen T5 states	37.58	76.31	114.47	79.72
MMaDA-style	MD	MD (VQ)	frozen T5 states	43.35	68.46	89.52	71.59
T2I only reference	–	FM	frozen T5 states	47.86	77.50	2.83	25.15

Image generation. JUICE reaches 48.41 GenEval and 78.85 DPG, the highest of the four frameworks. Part of this lead comes from the condition encoder: the discrete text frameworks condition image generation on learned token embeddings, whereas JUICE conditions on contextual T5 states. With the same frozen T5 condition, all three improve (Transfusion-style goes from 44.23 to 46.07 GenEval), but none of them reaches JUICE. A model trained only for text-to-image generation gets 47.86 at the same guidance scale. Adding a continuous text stream thus leaves image generation intact at this scale (48.41), whereas every token text stream lowers GenEval, with or without the T5 condition (for MMaDA-style, the image process also changes). With the condition matched, what remains different is the text stream: JUICE trains both streams with the same flow matching objective, while the discrete text frameworks train them with different objectives and attention patterns. With the guidance scale tuned per model, the T2I only model peaks at $w=3$ (51.23) and leads JUICE by 2.8 points (Appendix § 10.6.2).

Captioning. On the alignment between captions and images, JUICE matches autoregressive text (80.20 vs. 80.37 CLIPScore). On n-gram agreement with the reference captions, it is the weakest of the four: its CIDEr (88.80) is just over half that of autoregressive text (160.52) and below D-DiT-style masked text diffusion (115.22). The gap lies mainly in the generated states, not in the parallel caption decoder of this study, which recovers 99.2% of tokens from clean T5 states (Table 29; Appendix § 10.§ 5.2). The captions thus describe the image content as well as autoregressive ones but word it differently from the references. This is the pattern expected when errors in a generated continuous state move it toward nearby states with a similar meaning: the wording changes more than the content.

Readout	Flan-T5	Qwen3-E	Qwen3-VL-E	SigLIP2 text
Linear head	.9849	.9686	.9652	.9813
Parallel decoder	.9922	.9797	.9748	.9870

Takeaway: Continuous text spares images but costs wording. Of the four frameworks, only the one with continuous text keeps up with a model trained only for text-to-image generation on GenEval at the shared guidance scale (48.41 vs. 47.86); the three token text frameworks all fall below it, even with the T5 condition. Continuous text pays in CIDEr instead (88.80 vs. 160.52 for autoregressive text), while its CLIPScore is on par.

Training Task Mixtures

Because both modalities are continuous and each has its own noise level, JUICE can be trained not only on conditional tasks, where one modality is clean, but also on joint denoising, where both are noisy and each is predicted next to a noisy partner. In the $2$D time domain ($t_i$, $t_t$), the conditional T2I and I2T tasks lie on the edges and joint denoising covers the interior (§ 4.4; Figure 2). We explore what joint denoising brings and which regions training should cover: Table 3 compares five recipes, and Figure 4 follows two of them, Mixing and Multi-task, over training.

Recipe	I2T	T2I	Joint	GE (%)	DPG (%)	CIDEr	CLIP
T2I only	0	100	0	47.86	77.50	2.83	25.15
I2T only	100	0	0	0.21	9.95	107.22	77.81
Multi-task	50	50	0	36.30	77.71	79.41	75.07
Joint only	0	0	100	31.33	56.98	75.66	73.70
Mixing (default)	22.2	22.2	55.6	48.41	78.85	88.80	80.20
Controls
Matched DC	joint → 2 cond.			45.90	78.05	87.72	79.63
Blocked joint	no cross attention			45.49	78.14	86.58	78.66
Single target joint	1 target per joint			40.08	77.98	82.94	77.33
Multi-task, 77.8k updates	matched targets			43.16	78.51	89.31	79.14

Vector (PDF)

Edges and interior. Training in a single direction specializes: the T2I only and I2T only models do well in the direction they are trained for and fail in the other. Training only the edges (Multi-task) supports both directions, but training only the interior (Joint only) is worse on all four metrics. Joint only never sees the boundary. At conditional inference the condition is exactly clean, and training on the interior never produces that state. Mixing trains on both and is the best bidirectional recipe on every metric; it also stays ahead of Multi-task throughout training. The rule is simple: train where sampling goes. T2I and I2T sampling follow the edges, while joint sampling goes through the interior. At the shared guidance scale, Mixing is as good as the T2I only model at image generation, but it is still behind the I2T only model on CIDEr (88.80 vs. 107.22). The ordering of the bidirectional recipes also holds with the guidance scale tuned per recipe: with each recipe at its best scale in $w \in[1, 5]$, Mixing (48.41) still leads Multi-task (44.29) and Joint only (31.33) on GenEval, while T2I only rises to 51.23 (Appendix § 10.6.2).

More targets or cross-modal exchange? Extra supervision explains only part of the gain. Mixing does supervise each modality more often than Multi-task (on 77.8% vs. 50% of examples), so we control for target count (lower block of Table 3; Appendix § 11.2.1). One control replaces each joint example with two conditional ones at the same target count (matched dual conditional, DC), at 1.56 times the backbone work; another blocks cross-modal attention on joint examples. Both lower every metric relative to Mixing, by 2.51 and 2.92 GenEval points, respectively. In the other direction, a joint branch that keeps both streams noisy but supervises only one of them (single target joint) beats Multi-task on all four metrics with the same number of targets. Training Multi-task for longer, until it sees as many targets as Mixing, closes the CIDEr gap (89.31) but not the GenEval gap (43.16). The noisy partner and the exchange with it each contribute.

Takeaway: Train where you sample. Sampling uses both the edges of the square and its interior, and training has to cover both. Joint only never sees a clean condition and reaches just 31.33 GenEval at its best guidance scale; Multi-task never trains on the interior and reaches 44.29; Mixing gets 48.41. Matching the target count does not close the gap, so the joint examples give more than extra supervision.

Representation Spaces

JUICE reads and writes in the spaces of its frozen encoders, so the encoder determines both what the model sees and what it must generate. Encoders differ in what their spaces capture: some are trained to reconstruct their input, others to capture semantics, and some are aligned with the other modality. We explore these tradeoffs on both sides. With T5 fixed for text, we compare SigLIP2 [11] and WebSSL-DINO [12] (DINO) image features with FLUX VAE latents [46] and raw pixels [2, 34]. With SigLIP2 fixed for images, we compare T5 with Qwen3-Embedding (Qwen3-E) [47], Qwen3-VL-Embedding (Qwen3-VL-E) [48], and SigLIP2 text features (Figure 5; full metrics in Table 28).

Vector (PDF)

Image representation	Text representation	GE	DPG	CIDEr	CLIP
SigLIP2	T5	48.41	78.85	88.80	80.20
DINO	T5	49.07	77.85	68.80	71.66
FLUX VAE	T5	35.42	74.05	49.95	64.91
Pixels	T5	24.48	71.64	49.57	65.51
SigLIP2	Qwen3-E	43.07	78.04	88.50	76.54
SigLIP2	Qwen3-VL-E	39.28	78.70	84.24	75.94
SigLIP2	SigLIP2	33.08	74.19	85.54	74.52

Image spaces. Semantic encoders outperform spaces built for reconstruction on all four metrics: SigLIP2 and DINO reach 48.41 and 49.07 GenEval, against 35.42 for the VAE and 24.48 for pixels. This ordering is broadly the reverse of reconstruction quality, as the FLUX VAE reconstructs best among the learned spaces (ImageNet rFID 0.288, against 0.435 for SigLIP2; Table 30). The two semantic encoders are close on image generation, but captioning separates them: SigLIP2, which is aligned with language, leads DINO by 20.0 CIDEr and 8.5 CLIPScore. For a model that both generates and conditions on the same features, semantic structure thus matters more than reconstruction fidelity. The order of the two semantic encoders on image generation changes with model size (§ 5.5).

Image representation / decoder	rFID ImageNet	rFID YFCC	rFID RenderedText
FLUX VAE	.288	.410	.638
WebSSL-DINO / matched Scale-RAE decoder	.388	.558	1.372
SigLIP2 / matched Scale-RAE decoder	.435	.702	1.621
SDXL VAE (reconstruction reference)	.930	1.168	2.057
Pixels (identity reference)	.000	.000	.000

Text spaces. Flan-T5 states lead on all four metrics, ahead of larger embedding models and of SigLIP2 text features, which are aligned with the image space. Each text space is decoded with its own parallel decoder, which recovers 97.5% to 99.2% of tokens from clean states, and even a linear readout recovers 96.5% to 98.5% (Table 29). All four spaces thus retain most of the token information. The decoder affects only the captioning metrics, and the GenEval differences, which do not involve it, indicate that the ordering reflects how well each space serves as a generation target and condition. Qwen3-E comes close to T5 on CIDEr (88.50 vs. 88.80) but trails it by 5.3 GenEval points, and the SigLIP2 text features give the lowest GenEval (33.08). Alignment with the image space therefore does not make a text space a better target or condition here.

Takeaway: Semantics over reconstruction. Good reconstruction does not make a good generation space. The FLUX VAE has the best rFID among the learned spaces (0.288 vs. 0.435 for SigLIP2), but GenEval in it drops to 35.42, from 48.41 with SigLIP2. SigLIP2 and DINO are close on generation, and SigLIP2’s alignment with language shows up in captioning (20.0 CIDEr ahead). On the text side, Flan-T5 states beat larger embedding models and SigLIP2 text features.

Scaling

Do these preferences survive as the model grows? Keeping the recipe above, we train MM-RiT backbones from 179M to 2.1B trainable parameters ($d_{12}$ to $d_{28}$; Appendix § 10.§ 6) on both the SigLIP2 and the DINO image space (Figure 6). All four metrics improve with size in both spaces, but scale treats the two roles of the image space differently. For image generation the order flips. DINO is well ahead at 179M (32.90 vs. 18.99 GenEval), the gap shrinks as the backbone grows, and by 2.1B SigLIP2 is ahead (56.81 vs. 52.78). Captioning shows no such flip: SigLIP2 leads at every size and reaches 86.64 CLIPScore at 2.1B, against 72.32 for DINO. The two trends separate the roles of the image space: DINO’s advantage as a generation target at small scale fades as capacity grows, whereas the benefit of SigLIP2’s language alignment for producing text persists. After finetuning, the two 2.1B models of § 6 differ by less than 0.02 GenEval (DINO 0.88, SigLIP2 0.87), so neither image space keeps a clear GenEval advantage, while SigLIP2 leads on almost every understanding benchmark.

Vector (PDF)

Takeaway: Scale reorders image spaces for generation, not captioning. DINO’s lead in image generation fades as the model grows, while SigLIP2’s language alignment helps captioning at every size.

Experiments

A single model of the joint distribution of images and text should provide three abilities: generating images from text, answering questions about images, and producing image and text outputs that are individually correct and mutually consistent. We now evaluate JUICE on each of them at scale.

Setup

We train two 2.1B models, one per image space. Each is pretrained with the recipe of § 5.1, but for 60k instead of 50k updates, and then finetuned for 15k updates at a global batch size of 1,024 on 20.6M text-to-image and visual understanding examples (Appendix § 10.7). Text-to-image generation uses 25 Euler steps with guidance $w=5$; visual question answering uses 16 steps with $w=3$, with the parallel decoder reading out all answer positions at once. For joint reasoning, we adapt the finetuned models to each task separately (both models to mazes, the SigLIP2 model to visual spatial planning), read the moves with a linear head trained for each task, and sample without guidance (§ 6.4). All input and output images are $224 \times224$. We compare with unified models of 1B to 3B parameters; Appendix § 12 includes comparisons with larger models.

Image Generation

Compositional generation. On compositional prompts, both variants are close to the best unified models of 1B to 3B parameters (Table 4). On GenEval, the SigLIP2 and DINO variants reach 0.87 and 0.88. The DINO variant thus ties TUNA (0.88) for the best result among models of 1B to 3B parameters, and both variants have the highest position score (0.85); TUNA is still ahead on counting and attribute binding. On DPG-Bench, the SigLIP2 variant scores 85.22, which places it above Show-o2 (85.02) and below TUNA (86.03). The number of sampling steps hardly matters: from 25 to 100 steps, GenEval changes by at most 0.003 for either variant (Table 32).

Model	Size	Single	Two	Count	Color	Position	Attribute	Overall
Muddit [4]	1B	0.98	0.72	0.54	0.82	0.19	0.41	0.61
Show-o [20]	1.3B	0.98	0.80	0.66	0.84	0.31	0.50	0.68
UniDisc [49]	1.4B	0.92	0.47	0.15	0.67	0.13	0.19	0.42
FUDOKI [50]	1.5B	0.96	0.85	0.56	0.88	0.68	0.67	0.77
Janus-Pro-1B [51]	1.5B	0.98	0.82	0.51	0.89	0.65	0.56	0.73
Show-o2 [1]	1.5B	0.99	0.86	0.55	0.86	0.46	0.63	0.73
Harmon [52]	1.5B	0.99	0.86	0.66	0.85	0.74	0.48	0.76
TUNA [21]	1.5B	1.00	0.94	0.83	0.91	0.81	0.79	0.88
D-DiT [18]	2.1B	0.97	0.80	0.54	0.76	0.32	0.50	0.65
JUICE (SigLIP2)	2.1B	0.99	0.93	0.78	0.91	0.85	0.73	0.87
JUICE (DINO)	2.1B	0.99	0.95	0.81	0.93	0.85	0.78	0.88
Model	Size	Global	Entity	Attribute	Relation	Other	Overall
OneFlow Mixed [23]	1B	—	—	—	—	—	80.30
FUDOKI [50]	1.5B	80.55	89.73	88.05	93.66	78.00	83.63
Janus-Pro-1B [51]	1.5B	87.58	88.63	88.17	88.98	88.30	82.63
Show-o2 [1]	1.5B	87.53	90.38	91.34	90.30	91.21	85.02
TUNA [21]	1.5B	88.87	90.32	91.71	91.79	90.14	86.03
JUICE (SigLIP2)	2.1B	82.67	91.41	90.44	94.58	86.80	85.22
JUICE (DINO)	2.1B	83.28	90.24	88.78	95.05	81.60	84.38

Model	Steps	Single	Two	Count	Color	Position	Attr.	Overall
JUICE (SigLIP2)	25	99.4	93.4	77.8	91.0	85.0	73.0	86.60
	50	98.8	92.7	77.8	92.3	84.5	74.8	86.80
	100	98.8	93.2	78.1	91.8	84.3	74.5	86.76
JUICE (DINO)	25	99.4	94.7	80.6	93.1	84.8	77.5	88.34
	50	99.7	94.2	81.3	92.6	84.8	78.8	88.53
	100	99.7	94.2	81.9	92.8	84.3	79.0	88.64

Figure 7 shows selected samples for the same prompts from both variants. Both count small numbers of objects, place objects on the requested side, combine unrelated objects in one scene, and follow a range of artistic styles. The two image spaces often render the same prompt with a different composition and palette.

Generation with world knowledge. WISE [13] checks whether generated images reflect the world knowledge a prompt implies, with cultural, temporal, spatial, biological, physical, and chemical categories (Table 5). At the guidance scale used for GenEval, the SigLIP2 variant has a WiScore of 0.45. This is the highest overall score among unified models of 1B to 3B parameters that report one (which reach at most 0.41) and above Janus-Pro-7B (0.35), though below FLUX.1-dev (0.50) and BAGEL (0.52); OneFlow reports only the cultural category, where it scores higher. WISE, unlike GenEval, separates the two image spaces: SigLIP2 scores 0.45 and DINO 0.37. The advantage that § 5.4 found for SigLIP2 in captioning thus carries over to generation that needs world knowledge. The difference is largest for cultural prompts (0.41 vs. 0.31) and smallest for spatial ones (0.62 vs. 0.59), where the SigLIP2 variant is level with FLUX.1-dev. Its lead over Harmon, the strongest baseline of 1B to 3B parameters, comes mostly from spatial and physical prompts, and it is slightly lower on biology (0.35 vs. 0.37). Since JUICE has no language model backbone, this knowledge comes from its frozen encoders and its training data. A lower guidance scale with the time swap reference raises both scores, to 0.49 and 0.42, respectively (Appendix § 10.6.2). Figure 8 shows one example per category.

Model	Size	Cult.	Time	Space	Bio.	Phys.	Chem.	Overall
Unified models of 1B to 3B parameters
OneFlow [23]	1B	0.63	–	–	–	–	–	–
JanusFlow [53]	1.3B	0.13	0.26	0.28	0.20	0.19	0.11	0.18
Show-o (512²) [20]	1.3B	0.28	0.40	0.48	0.30	0.46	0.30	0.35
Janus-Pro-1B [51]	1.5B	0.20	0.28	0.45	0.24	0.32	0.16	0.26
Show-o2 [1]	1.5B	0.30	0.38	0.47	0.32	0.42	0.28	0.35
Harmon [52]	1.5B	0.38	0.48	0.52	0.37	0.44	0.29	0.41
Larger references
Janus-Pro-7B [51]	7B	0.30	0.37	0.49	0.36	0.42	0.26	0.35
BAGEL [54]	7B/14B	0.44	0.55	0.68	0.44	0.60	0.39	0.52
FLUX.1-dev (T2I) [46]	12B	0.48	0.58	0.62	0.42	0.51	0.35	0.50
JUICE (SigLIP2)	2.1B	0.41	0.48	0.62	0.35	0.51	0.33	0.45
JUICE (DINO)	2.1B	0.31	0.43	0.59	0.29	0.42	0.27	0.37

Visual Understanding

On visual question answering, JUICE is competitive on several perception benchmarks and weak on those that need domain knowledge (Table 6). All other models in the table start from a pretrained language or text-to-image model, whereas JUICE trains MM-RiT from scratch and sees images at $224 \times224$. Its POPE (84.0) and MME-P (1115.7) scores are still below those of most models initialized from LLMs. On VQAv2, the SigLIP2 variant reaches 73.1. That is above Show-o at 512² (69.4), Mudit (68.2), and OneFlow (67.7), and below JanusFlow (79.8). Its GQA (58.6) and MMBench (66.0) scores are close to those of Harmon and Show-o2. The weak side is domain knowledge. On AI2D, JUICE scores 34.4, far below models initialized from LLMs (58.5 to 71.4). On MMMU (29.6), it is above Show-o and Muddit, comparable to JanusFlow and OneFlow, and below Harmon, FUDOKI, Janus-Pro, Show-o2, and TUNA (34.3 to 39.1). This is what one would expect without a language model backbone: visual recognition transfers through the pretrained image space, but the knowledge that models initialized from LLMs inherit from text pretraining is missing.

Model	Size	Init.	Res.	VQAv2	GQA	POPE	MME-P	MMB	AI2D	MMMU	RWQA
Muddit [4]	1B	T2I	$512^{2}$	68.2	57.5	–	$1107.4${sup}†`	–	–	27.6	–
OneFlow [23]	1B	LLM	$512^{2}$	67.7	57.8	85.7	$1497.1${sup}†`	69.0	58.5	29.8	50.6
Show-o [20]	1.3B	LLM	$256^{2}$	64.7	54.2	76.2	$1014.9${sup}†`	–	–	25.0	–
Show-o [20]	1.3B	LLM	$512^{2}$	69.4	58.0	80.0	$1097.2${sup}†`	–	–	26.7	–
JanusFlow [53]	1.3B	LLM	$384^{2}$	79.8	60.3	88.0	1333.1	74.9	–	29.3	–
Harmon [52]	1.5B	LLM	$512^{2}$	–	58.9	87.6	1155.0	65.5	–	38.9	–
FUDOKI [50]	1.5B	LLM	$384^{2}$	–	57.6	86.1	1485.4	73.9	–	34.3	–
Janus-Pro-1B [51]	1.5B	LLM	$384^{2}$	–	59.3	86.2	1444.0	75.5	–	36.3	–
Show-o2 [1]	1.5B	LLM	$432^{2}$	–	60.0	–	1450.9	67.4	69.0	37.1	–
TUNA [21]	1.5B	LLM	$512^{2}$	–	61.4	–	1461.5	–	71.4	39.1	62.5
D-DiT [18]	2.1B	T2I	$256^{2}$	59.5	55.1	79.2	$897.5${sup}†`	–	–	–	–
D-DiT [18]	2.1B	T2I	$512^{2}$	60.1	59.2	84.0	$1124.7${sup}†`	–	–	–	–
JUICE (SigLIP2)	2.1B	None	$224^{2}$	73.1	58.6	84.0	1115.7	66.0	34.4	29.6	53.6
JUICE (DINO)	2.1B	None	$224^{2}$	68.8	55.7	82.3	960.2	55.8	31.9	31.5	40.5

The choice of image space matters: SigLIP2 leads DINO on every benchmark except MMMU (Table 34), by 4.4 points on VQAv2 and 10.2 points on MMBench, again in line with § 5.4. Appendix § 12 reports the remaining benchmarks (OK-VQA and ScienceQA-IMG) and comparisons with larger models. Figure 9 shows eight selected questions of seven types, from attributes and counting to chained spatial relations, outside knowledge, and object existence, which both variants answer correctly; both also produce fluent, concise captions (Appendix § 14.§ 3). Longer outputs are less reliable. The detailed descriptions in Appendix § 14.§ 5 get the main objects, colors, and layout right but contain grammatical errors and the occasional invented word, in line with the lexical cost seen in § 5.2.

Benchmark	Split	Metric	SigLIP2	DINO
VQAv2 [63]	test-dev	VQA accuracy	73.10	68.75
yes/no			90.66	88.18
number			58.73	53.30
other			61.29	55.57
VQAv2	val2014, fixed 1,000	VQA accuracy	71.21	66.32
OK-VQA [75]	val2014	VQA accuracy	33.87	28.65
POPE [76]	random, popular, adversarial	accuracy	83.97	82.29
MME-P [77]	test	perception score	1115.7	960.2
GQA [65]	testdev-balanced	accuracy	58.58	55.68
MMBench [78]	dev (English)	accuracy	66.04	55.82
AI2D [79]	test	accuracy	34.40	31.87
MMMU [80]	validation	accuracy	29.58	31.51
ScienceQA-IMG [70]	test, image questions	accuracy	80.57	73.75
RealWorldQA [81]	test, multiple choice	accuracy	53.63	40.52

Reading out the answer states in parallel loses little: on VQA with open answers, an autoregressive decoder on the same states is within 1.5 points but 2.1 to 6.4 times slower on these short answers. It is much worse on most multiple choice benchmarks (45.3 vs. 66.0 on MMBench), but only the parallel decoder saw multiple choice answers in training (Appendix § 10.§ 5.2).

Joint Reasoning

Some tasks need an image and a text output that are each correct and that agree, such as a route drawn on a map and the moves that describe it. When several answers are valid, being correct is not enough: the two outputs must also commit to the same answer. Sampling each modality from its own marginal can pick a different valid answer in each, which we call a cross-modal mode mismatch. A model of the joint distribution can instead produce both outputs together, with each modality conditioning on the other’s evolving state (coupled sampling; § 4.5). We first look at unconditional joint generation and then turn to two planning tasks, in which the input is a map image and a text question and the model generates the map with the route drawn on it together with the move sequence.

Joint generation. Without any condition, both streams start from noise and the model generates an image and a caption together (selected samples in Figure 10). In these samples, the caption describes the image it is generated with, so the two streams agree even though neither is given. Planning tasks test the harder case, in which the outputs must also be correct.

Mazes with multiple solutions. Our grid mazes have two to sixteen shortest routes each [14], so both outputs can be valid and still disagree. For each training pair, one of these routes is picked at random. At evaluation, any shortest route counts, and a joint success (J) needs both outputs to be valid and to show the same route (Appendix § 13.§ 2). We adapt the finetuned 2.1B models for 5k updates and compare samplers at an equal number of backbone calls (Table 7).

Sampler	Order	SigLIP2	DINO
Coupled	none	93.7	81.7
Image first	image $\to$ text	95.2	82.7
Text first	text $\to$ image	85.3	59.9
Independent	none	54.6	42.7
Validity of one modality alone
Image FT only	–	95.1	83.3
Text FT only	–	86.5	64.6

Coupled sampling comes close to the better sequential order without having to know which order that is. If each modality is sampled without seeing the other (independent sampling), joint success is only 54.6% for SigLIP2 and 42.7% for DINO, since the two streams often settle on different routes. Sequential sampling avoids this disagreement, but then the order matters, because mistakes in the first modality are passed on to the second. Image first and then text reaches 95.2% and 82.7%; the reverse order reaches 85.3% and 59.9%. The better order starts with the more reliable modality, here the image, whose validity alone is higher than that of the text. For a new task, though, which modality is more reliable is not known in advance. Coupled sampling needs no order. It reaches 93.7% and 81.7%, within 1.5 and 1.0 points of the better order and 8.4 and 21.8 points above the worse one, and it loses at most 0.3 points when the budget is halved to 8 calls (Appendix § 13.§ 3). These numbers are means over two seeds. For SigLIP2, the gap to image first is 0.6 points for one seed and 2.4 for the other, and a paired test still finds image first better ($p=0.017$; Appendix § 13.§ 3).

Coupled sampling does not reach agreement by collapsing onto one canonical route. On the mazes it solves, coupled sampling returns the stored reference route in 43.5% and 47.0% of cases, close to the 44.1% and 44.4% expected if it chose uniformly among the shortest routes, so the model picks among valid routes. Figure 11 follows two mazes, with both samplers starting from the same state. Under coupled sampling, whichever output settles later changes to match the other: the drawn route in one maze, the moves in the other. Independent sampling ends with two valid but different routes. Because either modality can be the one that changes, coupled sampling is not simply copying one modality into the other, which fits with the fact that it does not need to know in advance which modality is more reliable.

Vector (PDF)

Visual spatial planning. VSP [15] asks for a path through a FrozenLake grid from the start to the goal without stepping into a hole; any such path counts as a success. We adapt the finetuned SigLIP2 model for 5k updates on 55.9k maps of sizes $3 \times3$ to $6 \times6$, synthesized by us or drawn from ThinkMorph [57], with all benchmark maps removed, and evaluate on the official 100 maps per level (Table 8). JUICE solves 99.8% of maps at levels 3 to 6, on par with the fine variant of LatentUM (99%), which plans visually step by step, and above its coarse variant (85%), ThinkMorph (76%), and InternVL3.5-38B (20%). On larger map sizes not seen in training, it solves 94% at level 7 and 69% at level 8. Its image and text outputs agree on 99.6% of our own test maps. Appendix § 13.§ 5 describes the setup and the baselines.

Model	Size	Task FT	Lv3	Lv4	Lv5	Lv6	Avg.	Lv7	Lv8
InternVL3.5 [56]	38B	✗	–	–	–	–	0.20	–	–
ThinkMorph [57]	7B	✓	–	–	–	–	0.76	–	–
LatentUM, coarse [55]	4B	✓	1.00	0.85	0.83	0.71	0.85	–	–
LatentUM, fine [55]	4B	✓	1.00	1.00	1.00	0.97	0.99	–	–
JUICE (SigLIP2)	2.1B	✓	1.00	1.00	1.00	0.99	0.998	0.94	0.69

On the unseen sizes, the drawn path is right more often than the moves, although the image test is the stricter one. At levels 7 and 8, the drawn path is a shortest route on 99% and 96% of maps, while the moves reach the goal on 94% and 69% (Figure 12a shows a solved map of each size). Of the 37 failures there, 29 lose one move from the reference route while the drawn path is exactly that route. The rate of lost moves grows with the length of the run of repeated moves, from under 2% of runs of one or two moves to 7 of the 9 runs of six, a length that a shortest route cannot have on the sizes used in adaptation. The outputs mostly settle early: on 574 of the 600 maps, both are fixed from the first of 16 steps. Coupled sampling can still restore a lost move (Figure 12b). On nine maps the moves fail at the first step and solve the map at the end, seven of them with the drawn path fixed throughout, and no map solved at the first step ends unsolved. On the unseen sizes, as on mazes, the image is the more reliable modality, and these text errors match the cost in exact wording that § 5.2 finds for continuous text.

Vector (PDF)

Conclusion

JUICE models the joint distribution of images and text within frozen pretrained representation spaces, using a single flow matching backbone. Because each modality has its own noise level, image generation, visual understanding, and joint generation become paths through one square of noise levels, all handled by the same model. Our pretraining studies point to four principles. Text can be generated as continuous states at little cost to image generation, which stays close to that of a model trained only for text-to-image generation; what suffers is mainly exact wording. Training needs both the edges of the square, where sampling is conditioned on clean context, and the interior, which joint generation passes through. Semantic encoder spaces are better generation targets than spaces built for reconstruction. And a preference between image spaces found at one scale carries over across 179M to 2.1B parameters for captioning but not for generation. At 2.1B parameters and without language model initialization, the backbone is competitive in compositional image generation, in generation that needs world knowledge, and, on several perception benchmarks, in visual understanding. Since the model learns the joint distribution, it can also draw a route and write the matching moves together. On mazes with multiple solutions, coupled sampling makes the two agree without a fixed generation order, and on visual spatial planning it solves nearly all maps of the training sizes. In such a model, the order in which modalities are generated is no longer fixed by the architecture: it becomes a choice at sampling time, and coupled sampling can leave it open. Pretrained continuous spaces thus let one model understand, generate, and coordinate images and text without putting either one first.

Limitations and Future Work. The models run at a native $224 \times224$ resolution, which limits fine visual detail. Exact wording lags behind autoregressive text, and without a language model backbone the models trail most of those initialized from LLMs on benchmarks that need domain knowledge, such as AI2D and MMMU. Each pretraining configuration was run once, and the joint reasoning tasks are synthetic planning problems. The next steps follow from these limits: higher resolution, more accurate continuous text, and joint generation on real tasks in which several pairs of image and text outputs are valid.

Acknowledgments

This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) with a grant funded by the Ministry of Science and ICT (MSIT) of the Republic of Korea in connection with the Global AI Frontier Lab International Collaborative Research Agreement. (No. RS-2024-00469482 & RS-2024-00509279) This work was partly supported in part by the NYUAD Center for Interdisciplinary Data Science & AI (CIDSAI), funded by Tamkeen under the NYUAD Research Institute Award CG016.

References

Network details

Representation encoders

The pretrained encoders are never updated. Table 9 lists the four image inputs. SigLIP2 [11] and WebSSL-DINO [12] give us their last patch states (we drop DINO’s CLS token), which we pass through LayerNorm per token without learned parameters. The VAE input is the posterior mode after the VAE’s own shift and scale, with each $2 \times2$ latent neighborhood packed into one token. In all four cases the result is a $16 \times16$ grid of image tokens. A learned linear projection feeds frozen latent features into MM-RiT. Pixels go through a trainable patch stem instead: a convolution with patch stride, then a $1 \times1$ projection to the backbone width.

Input	Frozen source and extraction	Tokens $\times$ channels
SigLIP2	SigLIP2-SO400M/14, $224^2$, final patch states	$256 \times1152$
DINO	WebSSL-DINO300M-full2b, $224^2$, final patch states	$256 \times1024$
VAE	FLUX.1-schnell VAE, $256^2$, packed latent mode	$256 \times64$
Pixel	None; $256^2$ RGB in $[-1,1]$, $16 \times16$ patches	$256 \times768$

For text, we use contextual states at the layers in Table 10, then apply the representation normalization before the shared backbone. Indices refer to the returned hidden state list, with index 0 denoting the embedding input. Qwen3-VL-Embedding uses its text tower; we remove the fixed chat prefix of 14 tokens from the returned states. The SigLIP2 text tower uses lowercased input and a length of 64 tokens with unmasked encoder attention (its max length is 64); the other arms use a length of 128 tokens.

Frozen encoder	Layer	Positions	Channels
Flan-T5-Small [36]	Final (8)	128	512
Qwen3-Embedding-0.6B [47]	14	128	1,024
Qwen3-VL-Embedding-2B [48]	14	128	2,048
SigLIP2-SO400M/14 text [11]	13	64	1,152

Representation normalization

The transformation $N_m$ in § 4.3 defines the spaces in which diffusion noise is added. Its form depends on the encoder (Table 11).

Representation | Transformation

SigLIP2 / DINO image	Per token LayerNorm, without affine parameters
FLUX VAE image	$z = (h - b_{\mathrm{VAE}})a_{\mathrm{VAE}}$ using predefined factors
Pixel image	RGB values scaled to $[-1, 1]$, then patchified
Flan-T5 text	$z = h/0.2$, a fixed scalar for all channels
Qwen3-E / Qwen3-VL-E text	$z_c = (h_c - \mu_c)/s_c$ using fixed corpus statistics
SigLIP2 text	$z_c = (h_c - \mu_c)/s_c$ using fixed corpus statistics

For the RAE representations, we use LayerNorm per token without affine parameters [2, 7]. The VAE uses its supplied shift and scale, whereas pixels are scaled to $[-1,1]$ before patchification [8, 20, 21].

We divide Flan-T5 states by a fixed scalar, $z = h/0.2$, with no centering and no scaling per channel. The other text encoders are standardized per channel. For each of them, we compute a mean and a standard deviation for every feature channel over valid text positions in the training data, keep these statistics fixed, and use the same transformation in training and at inference. All token positions share the statistics, and no extra scalar divisor is applied.

MM-RiT architecture

The default encoders produce 256 image tokens with 1,152 channels and 128 text tokens with 512 channels. The default 668M MM-RiT has two input layers for text only (which we empirically add) followed by 17 joint attention blocks, with four time tokens per modality.

MM-RiT block computation. Image and text representations are first projected to a shared hidden width. Given hidden states $H_i^\ell$ and $H_t^\ell$, an MM-RiT block computes

where $[\,\cdot\,;\,\cdot\,]$ joins tokens, $R$ is RMSNorm, and $F$ is a SwiGLU feedforward network. Queries and keys are normalized with RMSNorm, and positions are encoded with rotary embeddings, in 2D for image tokens and in 1D for text tokens. We use RMSNorm instead of the popular AdaLN (MM-RiT does not modulate scale or shift with time). Each token $j$ of modality $m$ receives its own time embedding, $h_j \leftarrow h_j + W_m \operatorname{MLP}_m(\operatorname{emb}(t_j))$, where $\operatorname{emb}$ is a 256 dimensional sinusoidal embedding, $\operatorname{MLP}_m$ maps it to the hidden width with SiLU, $W_m$ is a linear layer initialized to zero, and $t_j=1$ at conditioning positions. The time tokens are four learned vectors per modality, each plus the embedding of the mean time of its stream through the same MLP; all of them are appended to the end of the text stream and take part in joint attention. Separate output projections for each modality return predictions to the original feature dimensions.

Configuration | Parameters | Width | Heads | Head dim. | Joint blocks | Total blocks

d12	178.6M	768	12	64	10	12
d15	335.0M	960	15	64	13	15
d19	668.3M	1,216	19	64	17	19
d24	1.33B	1,536	24	64	22	24
d28	2.12B	1,792	28	64	26	28

In the names d12 to d28, the number gives the number of attention heads, which also equals the total number of blocks. Each head has 64 channels. The 668M point is the same reference configuration used in the framework, task mixture, and representation studies.

Representation decoders

Image decoders

Each image representation uses its corresponding reconstruction decoder. Following the representation autoencoder construction of Zheng et al. [6], our SigLIP2-SO400M and WebSSL-DINO encoders use their corresponding pretrained Scale-RAE decoders [7]. The VAE encoder uses its pretrained VAE decoder, and the pixel arm uses an unpatchification head.

Text decoders

Each text representation has its own decoder, trained on text alone and kept fixed during MM-RiT training, and all quantitative text results outside the decoder comparison of Table 15 are read out in one pass: captions and answers to questions by parallel decoders, and the moves of the planning tasks by linear task heads. Flan-T5 states have two parallel decoders. The parallel caption decoder is trained on the pretraining captions alone for 3,000 steps; it reads out the captioning results of the pretraining study and gives the Flan-T5 entry of the parallel decoder row in Table 29. The final parallel decoder, described next, is trained on captions and question answering text and reads out the question answering results of the final models. The other text representations use their own parallel decoders (below). In the mazes and in VSP, the moves are read from the generated text states by a linear head trained for each task (end of this section), not by a parallel decoder. An autoregressive decoder over the same states, described below, serves as a comparison and reads out the qualitative joint samples (Figure 10, Figure 19, Figure 20, and Figure 21).

Benchmark	SigLIP2		DINO
	Parallel	Autoregressive	Parallel	Autoregressive
VQAv2 (val, 1,000)	71.21	71.71	66.32	66.68
OK-VQA	33.87	35.32	28.65	29.69
POPE	83.97	83.97	82.29	82.29
GQA	58.58	58.42	55.68	55.76
MME-P	1115.7	1133.1	960.2	983.6
MMBench	66.04	45.30	55.82	41.39
ScienceQA-IMG	80.57	51.23	73.75	50.99
RealWorldQA	53.63	42.86	40.52	35.83
AI2D	34.40	29.51	31.87	27.40
MMMU	29.58	27.81	31.51	31.51

Vector (PDF)

Vector (PDF)

Final parallel decoder. The decoder maps the 128 generated text states to 128 tokens in one forward pass. It is a copy of the Flan-T5-Small encoder stack (8 layers) followed by a vocabulary head. Every position attends to every other position, so the decoder needs no answer length as input; the decoded text is the token sequence before the first predicted EOS.

We train the decoder alone, on text only, without MM-RiT: the inputs are the frozen Flan-T5-Small states of the captions and “Question: . . . Answer: . . . ” rows in the pretraining data, and no images are used. The decoder reads unnormalized encoder states: generated states are multiplied by 0.2 (Appendix § 10.§ 3) before decoding. To make the decoder robust to imperfectly generated states, the first half of each batch is perturbed per position as $\tilde{z}_j = \lambda_j z_j + (1-\lambda_j)\epsilon_j$ with $\lambda_j = \operatorname{sigmoid}(g_j)$, $g_j \sim\mathcal{N}(0.8, 0.8^2)$, and $\epsilon_j \sim\mathcal{N}(0, 0.2^2 I)$. The loss is cross entropy at all 128 positions; content and EOS positions have weight 1, and padding positions are labeled EOS with weight 0.2. There are two training stages of 8,000 steps each, with 64 texts per step. Stage one starts from the pretraining Flan-T5-Small encoder stack, and stage two picks up where it ended. The optimizer is AdamW with a constant learning rate of $10^{-4}$ and no weight decay; we clip gradients at 1.0, run bfloat16 autocast over float32 weights, and keep the T5 dropout of 0.1. One stage takes about 1.3 hours on a single A100 GPU.

Decoders for other text representations. The decoders of Qwen3-E, Qwen3-VL-E, and SigLIP2 text (§ 5.4) share one design. They read the normalized text states, the space in which the model generates, and for a state $x$ with $D$ channels they output $Wx + b + H(T(Px))$. The linear branch $Wx+b$ is copied from the linear readout head of the same representation (end of this section), so each decoder starts out identical to that head. $P$ projects the $D$ channels to 512. $T$ is a randomly initialized trunk of 8 bidirectional layers with the shape of the Flan-T5-Small encoder, and the vocabulary head $H$ starts at zero. Each decoder is trained on the pretraining captions of its representation for 3,000 steps with 64 texts per step; padding positions are labeled EOS with weight 0.2, and the first half of each batch is perturbed as for the Flan-T5 decoder, with noise of unit scale in the normalized space ($\epsilon_j \sim\mathcal{N}(0,I)$), the same scale relative to the states as $\mathcal{N}(0,0.22\mathbf{I})$ for raw T5 states). Table 13 lists their sizes; on held out captions, each decoder recovers more tokens than the linear head it starts from (Table 29).

Representation	$D$	$V$	Length	Parameters	Token accuracy
Qwen3-E	1,024	151,680	128	253M	0.9797
Qwen3-VL-E	2,048	151,936	128	409M	0.9748
SigLIP2 text	1,152	256,000	64	446M	0.9870

Comparison with an autoregressive decoder. We compare the parallel decoder with an autoregressive decoder that reads the same states and with a linear readout head (described at the end of this section). The autoregressive decoder is the Flan-T5-Small decoder (8 layers, width 512, 6 heads, feedforward width 1,024, gated GELU) with the frozen Flan-T5-Small vocabulary head (58.1M parameters in total). It attends to the 128 unnormalized states through cross attention and decodes greedily with a KV cache. We train only its blocks and final layer norm (25.2M parameters) with teacher forcing and cross entropy on the content tokens and the final EOS, perturbing half of each batch with the same noise per position as above (AdamW, learning rate $10^{-4}$, no weight decay, 64 texts per step). For captions, we take a version of this decoder that was already finetuned on perturbed states and train it for 1,500 more steps on caption states; we call the result the autoregressive caption decoder. For question answering, we train it for 3,000 more steps on the VQAv2 training split. Under teacher forcing, the decoder input starts with the “Question: . . . Answer:” prefix, and the loss covers only the answer and EOS. Answer positions are perturbed, and the positions after the answer are kept, set to zero, or replaced by noise, each with equal probability. On 300 VQAv2 validation questions, this raises its VQA accuracy on clean answer states from 61.2% to 95.9%. On captions held out from training, the parallel decoder recovers almost every token and 98.4% of captions exactly from clean states. It keeps most of this accuracy (93.7%) when the clean fraction of each state drops to 0.7, whereas the autoregressive decoder falls to 58.5% (Table 14). The linear head recovers 98.7% of tokens but only 50.8% of captions exactly, and its EOS error of 10.5 positions indicates that it often misplaces the end of the text. On answer states generated by JUICE, the two decoders are within 1.5 points on VQA benchmarks with open answers (VQAv2, OK-VQA, GQA), with the autoregressive decoder slightly ahead on most of them and on MME-P. On most multiple choice benchmarks the parallel decoder is far ahead, for example 66.0 vs. 45.3 on MMBench, while on MMMU the two are within 1.8 points (Table 15). The training data of the parallel decoder include multiple choice answers given as option letters, whereas the question answering stage of the autoregressive decoder uses only VQAv2 answers, so the gap on multiple choice benchmarks reflects training data at least in part and cannot be attributed to the decoding scheme alone. The autoregressive decoder needs one forward pass per generated token, so its cost rises with the length of the output. For timing, we use one A100 80GB PCIe GPU (PyTorch 2.5.1, TF32 disabled), do two warmup runs, and synchronize the GPU before and after every timed call. On 2,000 captions of 84.6 tokens on average, with both decoders in bfloat16, the autoregressive decoder is 116, 163, 98, and 54 times slower than the parallel decoder at batch sizes 1, 16, 64, and 128. On VQAv2 answers of 1.6 tokens on average, again with both decoders in bfloat16, it is 3.9 times slower at batch size 1 (30.89 ms against 7.86 ms per answer) and 6.4 times slower at batch size 128, where it groups answers by prefix length (1.630 ms against 0.254 ms per answer). With both decoders in float32, the ratios are 4.1 and 2.1: at batch size 128, bfloat16 makes the parallel decoder 2.8 times faster but the autoregressive decoder slightly slower (1.501 ms per answer in float32).

Decoder	$\tau$	Token accuracy	String match	EOS error
Parallel (final)	1.0	0.9997	0.984	0.00
Autoregressive	1.0	0.9759	0.966	0.45
Parallel, first stage only	1.0	–	0.941	–
Linear head	1.0	0.9872	0.508	10.5
Parallel (final)	0.9	0.9997	0.983	0.00
Autoregressive	0.9	0.9689	0.953	0.45
Parallel (final)	0.7	0.9989	0.937	0.00
Autoregressive	0.7	0.7505	0.585	2.28

Linear readout diagnostic. To measure how much token information each text representation retains (Table 29), we also fit one linear vocabulary head per representation. These heads also initialize the linear branch of the parallel decoders for Qwen3-E, Qwen3-VL-E, and SigLIP2 text. Given a $D$ channel state $z_j$, the head predicts

where $W \in\mathbb{R}^{V \times D}$ and $b \in\mathbb{R}^{V}$ use the encoder’s token vocabulary. All positions are evaluated in parallel. The head adds no Transformer blocks and has $V(D + 1)$ parameters (Table 16).

Text representation	Input $D$	Vocabulary $V$	Parameters (M)
Flan-T5-Small	512	32,128	16.48
Qwen3-Embedding-0.6B	1,024	151,680	155.47
Qwen3-VL-Embedding-2B	2,048	151,936	311.32
SigLIP2 text	1,152	256,000	295.17

We train each linear head on our pretraining dataset using the corresponding text tokenizer and the frozen encoder for the corresponding representation. We fit the head separately from MM-RiT training using cross entropy loss. For each position in an augmented example (the first half of each batch),

We add the noise in the normalized coordinates read by the linear head; the remaining examples stay clean. Padding is excluded from the cross entropy, while content and EOS positions are supervised. Decoding takes the argmax at every position at once and cuts the text at the first EOS. Only the vocabulary head is trained. We use AdamW (learning rate $10^{-3}$, batch size 64, 15,000 updates, weight decay 0.01, $(\beta_1, \beta_2) = (0.9, 0.999)$).

Linear task heads. The planning tasks read their moves with linear heads of the same form over Flan-T5 states, one for the mazes and one for VSP. Each is trained with the recipe above, on the texts of its task’s training data (questions and answers, without images and without MM-RiT), for 1,500 updates instead of 15,000. Like the readout heads, a task head reads each position on its own.

Frozen components and total parameters

Table 17 lists the fixed modules used by the principal and alternative representation configurations. Table 18 then distinguishes three quantities for the principal SigLIP2/DINO+T5 configurations: trainable backbone parameters, modules executed for a particular direction, and the union of modules that must be loaded to support both directions.

Component	Parameters	Used in
Principal image/text pipeline components
SigLIP2-SO400M image encoder	427.7M	I2T and VQA (SigLIP2 arm)
WebSSL-DINO300M image encoder	303.7M	I2T and VQA (DINO arm)
Flan-T5-Small encoder	35.3M	T2I, I2T, and VQA
SigLIP2 Scale-RAE image decoder	415.6M	T2I (SigLIP2 arm)
DINO Scale-RAE image decoder	415.4M	T2I (DINO arm)
FLUX.1-schnell VAE	83.8M	VAE representation control
Parallel text decoder (stack of 8 layers, 18.88M + vocabulary head 16.48M)	35.36M	I2T and VQA
Autoregressive text decoder (Flan-T5-Small decoder + vocabulary head)	58.1M	Decoder comparison and qualitative joint samples
Flan-T5 linear vocabulary head	16.48M	Readout diagnostic only
Alternative text representations with their parallel decoders and linear readout heads
Qwen3-Embedding-0.6B text encoder	595.8M	Qwen3-E representation arm
Qwen3-VL-Embedding-2B text tower	1,720.6M	Qwen3-VL-E representation arm
SigLIP2 text tower	707.8M	SigLIP2 text representation arm
Qwen3-E parallel decoder	253M	Qwen3-E representation arm
Qwen3-VL-E parallel decoder	409M	Qwen3-VL-E representation arm
SigLIP2 text parallel decoder	446M	SigLIP2 text representation arm
Qwen3-E linear vocabulary head	155.47M	Readout diagnostic; initializes the decoder
Qwen3-VL-E linear vocabulary head	311.32M	Readout diagnostic; initializes the decoder
SigLIP2 text linear vocabulary head	295.17M	Readout diagnostic; initializes the decoder

Table 18: Task active and fully loaded parameter counts. T2I is backbone + T5 encoder + matched image decoder. I2T/VQA is backbone + image encoder + T5 encoder + parallel text decoder (35.36M); the d24 and d28 entries are computed from the rounded backbone sizes, so their last digit is approximate. “All loaded” is the union.

Image arm	Scale	Trainable	T2I forward	I2T/VQA forward	All loaded
SigLIP2	d12	178.6M	629.5M	677.0M	≈1.09B
	d15	335.0M	785.9M	833.4M	≈1.25B
	d19	668.3M	≈1.12B	≈1.17B	≈1.58B
	d24	1.33B	≈1.78B	≈1.83B	≈2.24B
	d28	2.12B	≈2.57B	≈2.62B	≈3.03B
DINO	d12	178.6M	629.3M	553.0M	968.4M
	d15	335.0M	785.7M	709.4M	≈1.12B
	d19	668.3M	≈1.12B	≈1.04B	≈1.46B
	d24	1.33B	≈1.78B	≈1.70B	≈2.12B
	d28	2.12B	≈2.57B	≈2.49B	≈2.91B

Training and inference details

Training details

§ 4.4 defines joint, T2I, and I2T training. Here we give their noise distributions and the microbatch loss reduction; optimization settings are in Appendix § 10.7.

The default pretraining mixture assigns 55.6% of examples to joint denoising and 22.2% each to T2I and I2T conditional denoising. No training stage drops or replaces the condition. Each modality is a target with probability $0.556 + 0.222 = 0.778$ in Mixing and 0.5 in Multi-task, which is only conditional. The modality weights are $\lambda_i = 1$ and $\lambda_t = 0.25$; their relation to the realized task weights is given in Equation (14). For Equation (5), the base time is

Pretrained representation and VAE image streams use $(\mu_i,s_i) = (0,1)$, while the pixel control uses $(-0.8, 0.8)$; these values follow the works cited in Table 19. Text streams use $(\mu_t,s_t) = (-1.5, 0.8)$. All continuous text configurations use $\sigma_t = 2$ and $\alpha_t = 1$. The image settings are representation specific (Table 19); the three text encoder substitutions retain the SigLIP2 image settings. For representation and VAE image streams, $\alpha_i = \sqrt{n_i d_i/4096}$, where $n_i$ is the number of tokens and $d_i$ is their feature dimension; the pixel control uses an unshifted schedule. The FLUX VAE control encodes $256^2$ inputs to $32 \times32$ latents with 16 channels, then packs $2 \times2$ latent patches into 256 tokens of 64 channels; hence $\alpha_i = \sqrt{256 \cdot64/4096} = 2$. We also apply the corresponding time shift to the sampling grid [8].

Table 19: Image noise settings in the representation configurations. The base logit normal parameters, noise scales, and time shifts depend on the representation.

Image representation	$(\mu_i,s_i)$	Noise scale $\sigma_i$	Time shift $\alpha_i$
SigLIP2 [6]	$(0,1)$	1	8.485
DINO [6]	$(0,1)$	1	8
FLUX VAE [10]	$(0,1)$	1	2
Pixel [34]	$(-0.8,0.8)$	2	1

The network predicts the clean state, and we train it with a stabilized velocity loss in which the predicted and the target velocity are divided by the same $d(t) = \max(1 - t, 0.05)$:

For the linear path $z_m^{t_m} = (1 - t_m)\epsilon_m + t_m z_m$, this gives

For $t_m \le0.95$ we have $d(t_m) = 1 - t_m$ and $v_m^{\mathrm{target}} = z_m - \epsilon_m$, which is exactly the usual velocity loss of flow matching on the linear path. Above 0.95, the floor in the denominator turns the loss into a stabilized surrogate near the clean endpoint instead of exact velocity supervision. Each stream averages the error over its valid corrupted positions and over feature dimensions; given positions and text padding are left out, and a stream with no targets contributes zero.

Task weights and active positions. For brevity, $J$, $T$, and $I$ denote the joint, T2I, and I2T tasks. Let $a_{m,bp}$ indicate whether position $p$ of example $b$ is a generated, valid target of modality $m$, and let $e_{m,bp}$ be its squared velocity error averaged over feature channels. For task $r \in\{J,T,I\}$, define the active count and empirical task loss in a microbatch:

An empty task target set has zero empirical loss. The denominator guard also makes a modality with no active targets contribute zero. The corresponding microbatch averages are $\widehat{\mathcal{L}}_{\mathrm{joint}}^i = \bar{\ell}_{i}^{J}$, $\widehat{\mathcal{L}}_{\mathrm{joint}}^t = \bar{\ell}_{t}^{J}$, $\widehat{\mathcal{L}}_{\mathrm{T2I}} = \bar{\ell}_{i}^{T}$, and $\widehat{\mathcal{L}}_{\mathrm{I2T}} = \bar{\ell}_{t}^{I}$. The realized count weights are

With these weights, the weighted sum of the four task losses reduces exactly to

We minimize $\mathbb{E}_{B}[\widehat{\mathcal{L}}(B)]$, where the weights and the averages come from the same sampled microbatch $B$. Joint and T2I examples have image targets, and joint and I2T examples have text targets. The task probabilities decide which task each example gets, but the loss weights follow the number of active positions actually drawn, counted without text padding or clean context. Without times per token, the image branch first averages each active example over its positions and channels and then averages over examples, which is the same as averaging over tokens when every image has the same number of tokens.

Training times per token. We extend these task choices to partial inputs by varying time across positions, following the noising per token of Diffusion Forcing [58]. We first complete encoding and normalization (Appendix § 10.§ 3), and then apply span masks and times per token only to the normalized target latents. For each target stream, with probability 15%, only a contiguous target span is corrupted and the remaining target positions stay clean. With probability 10%, target positions independently draw their times; otherwise they share the stream time. The same interpolation is applied at each selected latent position. These operations do not construct the clean VQA question condition, which uses the independent encoding of the question alone described below.

Parameter updates. For each microbatch, the frozen encoders produce the representations, each example gets a denoising task, and the targets are corrupted at the sampled times. MM-RiT predicts the clean representations, and the loss above updates its parameters, including the input and output projections. Encoders, image decoders, and text decoders do not change in this stage. We also keep two exponential moving averages (EMA) of the MM-RiT weights, with the decay rates given in Table 22.

Inference details

Sampling follows the T2I, I2T, and joint paths in Equation (6). For a grid $0 = s_0 < \cdots< s_K = 1$, a generated stream uses $t_{m,k} = S_{\alpha_m}(s_k)$ and the Euler update

Both velocities in a joint step are computed from the same current pair before advancing either stream.

Guidance reference. The reference for a text condition uses the normalized states of a generic caption, and the reference for an image condition replaces the image with Gaussian noise at time $0$:

Each clean prediction is converted to velocity before applying

For text-to-image generation we use 25 Euler steps and $w = 5$, or 50 backbone evaluations per sample. In each text encoder arm, the generic caption goes through that arm’s own encoder and normalization. The discrete text frameworks of § 5.2 use no T5 encoder for it: they tokenize it with the Flan-T5 tokenizer and look up their own token embeddings. For the base D-DiT-style framework with token embeddings, Table 2 instead uses a fully masked caption as the reference. The controls conditioned on T5 swap only the clean condition for frozen T5 states. Captioning and question answering take 16 steps with $w = 3$ and the image reference above; the question tokens are clean in both branches. Joint reasoning takes 16 backbone calls (8 in the comparison with a halved budget; Table 37 lists the steps per stream), and unconditional joint generation takes 64 steps. Neither uses guidance, so no reference branch is evaluated.

Sampler	Steps at 16 calls	Steps at 8 calls	SigLIP2	DINO
Coupled	16 shared	8 shared	93.7	81.4
Image first	8 + 8	4 + 4	94.9	83.0
Text first	8 + 8	4 + 4	84.9	61.6
Independent	8 per stream	4 per stream	55.9	45.7

Guidance reference for text conditions. There are two ways to remove a text condition: use the clean generic caption “a photo.” (our default), or move the caption to its noise end, $t_t = 0$ (time swap). Table 20 gives GenEval after 50k updates for the four pretraining task mixtures that generate images. The generic caption scores higher for every mixture and every guidance scale from 2 to 5. With time swap, the Multi-task and Joint only models produce images that look like mosaics at most scales, which never happens with the generic caption. The default mixture depends least on the choice: at $w = 5$ it reaches 48.41 GenEval with the generic caption and 44.64 with time swap, and 78.85 and 78.45 on DPG-Bench.

The mixtures also differ in their best scale. T2I only and Multi-task peak at $w = 3$ and decline at larger scales, whereas the default mixture and Joint only improve up to $w = 5$, the largest scale we evaluated. The pretraining studies use $w = 5$ for every model. With the scale chosen per model in this range, T2I only reaches 51.23 and leads the default mixture by 2.8 points, while the default mixture still leads Multi-task (44.29) and Joint only (31.33).

On WISE, the finetuned SigLIP2 model at $w = 5$ scores 0.447 with the generic caption and 0.392 with time swap. Time swap at the weaker scale $w = 3$ gives the highest WISE scores we measured (Table 21), but we keep one reference and one scale for all text-to-image benchmarks.

Table 21: WISE with the time swap text reference at $w = 3$. 95% confidence intervals in brackets. The main text uses the generic caption reference at $w = 5$ (Table 5).

Model	Cult.	Time	Space	Bio.	Phys.	Chem.	Overall
JUICE (SigLIP2)	0.466	0.533	0.642	0.396	0.531	0.346	0.488 [0.468, 0.508]
JUICE (DINO)	0.379	0.481	0.607	0.344	0.456	0.313	0.424 [0.405, 0.443]

Given and generated positions. For question answering and the puzzle tasks, the given context occupies only part of the image and text inputs. The conditional distribution is

A modality with no target positions stays fixed. For VQA training, the frozen text encoder is run twice. The pass over the question alone, $N_t(E_t(Q))$, supplies the clean question prefix condition.

A separate complete sequence pass $N_t(E_t(Q\Vert A))$ supplies target states only at answer positions. The question condition is never obtained by slicing question positions from the $Q\Vert A$ encoding; neither the ground truth answer content nor its length enters the conditioning representation. After encoding and normalization, we corrupt and optimize only the answer target latents. At inference, the image and the separately encoded question are held at time 1. The answer canvas, whose length is fixed in advance, starts from noise and follows the text sampling grid, and only its positions are updated.

Data, optimization, and training stages

The pretraining studies in § 5 train each configuration for 50k updates with the settings in Table 22, except the budget controls of Table 27 (32.1k and 77.8k updates). The two 2.1B models in § 6 follow the stages in Table 23: pretraining for 60k updates, supervised finetuning (SFT), and a separate adaptation to each planning task (mazes for both models, visual spatial planning for the SigLIP2 model).

Stage	Starts from	Updates	Global batch	GPUs	Evaluated weights
1. Pretraining	random initialization	60,000	4,096	8	fast EMA
2. SFT	stage 1, step 60,000	15,000	1,024	8	fast EMA
3a. Maze adaptation	stage 2, step 15,000	5,000	1,024	8	raw
3b. VSP adaptation	stage 2, step 15,000	5,000	1,024	4	raw

Recipe	Updates (k)	Targets (M/mod.)	Work proxy	GE	DPG	CIDEr	CLIP
Mixing	50	159.33	1.000	48.41	78.85	88.80	80.20
DC (matched targets)	50	159.33	1.556	45.90	78.05	87.72	79.63
DC (matched work)	32.1	102.40	1.000	43.18	75.24	71.55	76.81
Blocked joint	50	159.33	1.000	45.49	78.14	86.58	78.66
Multi-task / 50k	50	102.40	1.000	36.30	77.71	79.41	75.07
Multi-task / 77.8k	77.8	159.33	1.556	43.16	78.51	89.31	79.14
Mixing / 32.1k	32.1	102.40	0.643	44.90	75.56	72.30	77.37
Single target joint	50	102.40	1.000	40.08	77.98	82.94	77.33

Table 22: Controlled comparisons share the same pretraining settings. The budget controls of Table 27 differ only in the number of updates.

Setting	Value
Updates / global batch	50,000 / 4,096
Optimizer	AdamW, $(\beta_1, \beta_2) = (0.9, 0.95)$, weight decay 0.01
Learning rate	$4 \times10^{-4}$; warmup over 5,000 steps, then constant
Gradient norm cap / precision	1.0 / bfloat16
EMA decay (slow / fast)	0.99995 / 0.9996
Image / text loss coefficient	1.0 / 0.25
Default image / text Gaussian noise std.	1.0 / 2.0
Joint / T2I / I2T task ratio	55.6% / 22.2% / 22.2%
Probability of span / independent token times	15% / 10%
Condition dropout	none

Unified multimodal pretraining. Following Tong et al. [7] and Chen et al. [39], we use 38M image caption pairs with recaptioned text, including images from CC12M [59], SA-1B [60], and JourneyDB [61]. We also include Cambrian-7M for visual understanding supervision [40].

Supervised finetuning. SFT mixes text-to-image and understanding data (Table 24). Text-to-image rows train text-to-image generation only; question answering, caption prompt, and long answer rows train image-to-text generation only, so SFT has no joint denoising. Each understanding row generates a fixed answer window whose length $K$ depends on the prompt type (12 for short answers and multiple choice, 48 for concise captions, 101 for detailed captions and long answers) and never on the answer length. Padding inside the window has loss weight 0.1, and positions after the window stay clean and are masked. Both image encoder variants use the same data. Table 25 gives the optimization settings.

Block	Sources (rows)	Task	$K$	Rows	Share
Text-to-image	BLIP3o-60k [39] (3,776,625), DALL-E 3 synthetic pairs (989,248), ShareGPT-4o-Image [62] (2,377,284)	caption → image	–	7,143,157	34.7%
Short QA	VQAv2 train [63] (3,010,000); LLaVA-1.5 mix [64] twice: GQA [65] (1,887,666), COCO [66] (1,269,778), Visual Genome [67] (672,448), OCR-VQA [68] (635,350), VizWiz [69] (40,256)	image + question → short answer	12	7,515,498	36.5%
Multiple choice	Cambrian [40] multiple choice questions (433,570), ScienceQA train [70] (53,130)	image + question + options → letter	12	486,700	2.4%
Concise captions	COCO train2014 captions, twice (827,606), BLIP3-o short captions (1,250,000)	image + prompt (11 templates) → 1 to 40 tokens	48	2,077,606	10.1%
Detailed captions	BLIP3-o long captions (1,500,000), BLIP3o-60k and DALL-E 3 captions, four times (116,516)	image + prompt (16 templates) → 30 to 100 tokens	101	1,616,516	7.9%
Long answers	ALLaVA [71] via Cambrian (671,552), ShareGPT4V [72] (1,042,950), LLaVA (3,719)	image + question → 9 to 100 tokens	101	1,718,221	8.3%
COCO val questions	Cambrian questions on COCO val2014/val2017 images (32,010)	image + question → short answer	12	32,010	0.2%
Total				20,589,708	100%

Initialization	Raw weights of the 2.1B pretraining run at step 60,000
Updates / global batch	15,000 / 1,024 (8 rows × 16 accumulation steps × 8 GPUs); 15.36M samples, about 0.75 epoch
Optimizer / learning rate	AdamW, $(\beta_1, \beta_2) = (0.9, 0.95)$; $10^{-4}$ after 250 linear warmup updates, then constant
Weight decay / gradient cap	0.01 / 1.0
Precision / seed	bfloat16 / 4242
EMA decay (slow / fast)	0.99995 / 0.9996; we evaluate the fast EMA
Task assignment	text-to-image rows: text-to-image only; question, caption prompt, and long answer rows: image-to-text only
Text loss weight	0.25
Answer window	$K \in\{12, 48, 101\}$ set by prompt type, never by answer length; padding inside the window has weight 0.1; positions after the window stay clean and masked
Condition dropout	none
Image time shift	8.485 (SigLIP2), 8.0 (DINO)

Pretraining study details

Generative frameworks

Tables 2 and 26 give the results and configurations for the controlled comparison in § 5.2. AR denotes autoregressive generation, FM denotes flow matching, and MD denotes masked diffusion.

Configuration	Image space / process	Text space / process	Image / text decoder
JUICE	SigLIP2 features / FM	T5 features / FM	RAE / parallel decoder of T5 states
Transfusion-style	SigLIP2 features / FM	Discrete tokens / AR	RAE / token vocabulary head
D-DiT-style	SigLIP2 features / FM	Discrete tokens / MD	RAE / token vocabulary head
MMaDA-style	aMUSEd VQ codes [73] / MD	Discrete tokens / MD	VQ decoder / token vocabulary head

Framework implementations

Figure 13 summarizes the four generative configurations compared in Table 2. All use an MM-RiT trunk of width 1,216 with two streams, 17 joint blocks, two input layers for text only, 19 attention heads of dimension 64, and four time tokens per modality, trained under the same schedule of 50k updates. The configurations differ in their state representations, generation objectives, attention masks, and output interfaces.

Vector (PDF)

Text representations and conditioning. In the base framework comparison, only JUICE uses the frozen Flan-T5-Small encoder for both clean conditioning states and continuous generation targets. Appendix § 11.1.2 additionally supplies frozen T5 states to the clean text condition branch of the discrete text configurations, while retaining their original generated targets and objectives. The other three configurations use discrete text tokens with learned embeddings and token prediction heads. The base token configurations reuse the T5 tokenizer and its vocabulary of 32,128 entries, but do not run the T5 encoder for either clean conditioning text or generated text targets. Their clean text conditions enter the backbone directly through token embeddings. Thus, sharing a tokenizer does not imply sharing pretrained contextual text representations.

Continuous image and text (JUICE). At $224 \times224$, the image stream holds 256 frozen SigLIP2 states with 1,152 channels and the text stream 128 frozen T5 states with 512 channels. Both streams are generated with continuous flow matching, each with its own sampled time. The bidirectional trunk predicts clean states; we turn them into velocities for the stabilized velocity loss on generated image tokens and valid text positions. During Euler sampling, both streams are advanced from the same current pair of image and text. The matched RAE decoder gives the image, and the parallel text decoder (Appendix § 10.§ 5.2) reads all final text states in one pass, cutting the text at the first predicted EOS.

Continuous image and autoregressive text (Transfusion-style). This control keeps the continuous SigLIP2 image stream and predicts text tokens autoregressively instead of generating text states with flow matching. At text target positions, the inputs are the token embeddings shifted by one, with the mask token as the start symbol, and the input embedding matrix is shared with the output head. Valid text positions are trained with cross entropy on the next token. In examples with text targets, text queries attend causally, and image and time queries cannot attend to text target keys, so future tokens cannot leak to earlier text positions through the other streams. A clean text condition for image generation stays fully visible through its token embeddings. Images are generated by Euler integration and decoded with the RAE decoder, and text is decoded greedily up to EOS.

Continuous image and masked text (D-DiT-style). Here the image stream stays continuous (SigLIP2), but text becomes a canvas of discrete tokens [18]. Each text target token is masked with probability $1 - t_t$; the text time therefore controls how many tokens are visible. Visible tokens enter the bidirectional trunk through their embeddings, and masked ones through a learned mask embedding. We train masked valid text positions with cross entropy and keep the continuous velocity loss for the image. For sampling, the text canvas starts fully masked, and a cosine schedule reveals the most confident positions step by step. The image is sampled with Euler updates.

Masked image and text (MMaDA-style). Here both modalities are discrete [3]. Images become a $16 \times16$ canvas of aMUSEd VQ codes (codebook of 8,192 entries, $256 \times256$ input). Image and text targets are masked independently according to their modality times and trained with cross entropy on the masked tokens. The embeddings of both canvases go into the bidirectional trunk with two streams, and at sampling time each canvas is unmasked in order of confidence with a cosine schedule. The frozen VQ decoder turns the image codes into pixels, and the text tokens are read up to EOS.

Matching the text-to-image conditioning encoder

In their base form, the Transfusion-style, D-DiT-style, and MMaDA-style frameworks condition image generation on token embeddings, while JUICE conditions on contextual frozen T5 states. To see how much this matters, we give each token configuration the same frozen Flan-T5-Small states in its clean T2I condition branch. Nothing else changes: the generated text stays discrete, with the original autoregressive or masked token objective and readout, and the image representation and image objective are the same as before. We use the 668M backbone family, 50k pretraining updates, and the common guidance setting ($w = 5$ with the generic caption reference).

Table 2 in the main text includes these rows.

Replacing token embeddings with T5 raises Transfusion-style GenEval from 44.23 to 46.07 (a gain of 1.84 points) and DPG from 76.84 to 78.24 (a gain of 1.40 points). This GenEval gain of 1.84 points accounts for part of the original gap of 4.18 points to JUICE; the remaining gap in the point estimate is 2.34 points. D-DiT-style gains 3.08 points in GenEval and 1.96 points in DPG, while MMaDA-style gains 1.74 points in GenEval and 1.96 points in DPG. Their remaining gaps may reflect other differences in the generation configuration, which this comparison does not isolate: D-DiT-style retains a masked token objective for text, and MMaDA-style also uses a VQ image interface with masked image generation.

Task mixtures

Table 3 gives the five training recipes in § 5.3.

Supervision and cross-modal information

This study separates how many targets the model gets from what information the joint branch makes available.

Settings. Every model uses the default 668M d19 pretraining protocol with frozen SigLIP2 and Flan-T5-Small encoders and a global batch of 4,096; Table 27 lists update counts and work proxies. Mixing keeps the default 55.6/22.2/22.2 task mixture. The controls change either the computation of the joint branch or the assignment of targets, as described below and in Table 27. All arms are scored on the same evaluation inputs.

Dual conditional (DC) control. Each example assigned to the original joint branch gets two conditional forward passes instead of one joint pass: one predicts the image from clean text, and the other predicts the text from the clean image. Both reuse the sampled target masks, corruption times, and noises. We sum the two target loss numerators and divide by the same active target counts of each modality as in Mixing, followed by one optimizer update; averaging the two branches separately and adding them would double the loss scale. The original conditional branches are kept.

Blocked joint control. This control keeps everything about the joint branch (both targets, their noise, the task probabilities, and the loss reduction) except that image and text cannot see each other on joint examples. We mask every path between the two modalities at every layer, including paths through shared time or auxiliary tokens. Shared tokens are duplicated per modality, and each copy carries only its own modality’s information. The conditional branches stay as they are.

Single target joint control. Joint examples keep both streams noisy, but only one modality, picked with probability $1/2$, is supervised, and the other stream serves only as context. With the conditional branches unchanged, each modality is then a target with probability $0.222 + 0.556/2 = 0.5$, the same as in Multi-task at the same number of updates.

Interpretation. Mixing ends 2.51 GenEval and 1.08 CIDEr ahead of matched DC after 50k updates, and 2.92 GenEval and 2.22 CIDEr ahead of blocked joint. DC alters the clean or noisy context and the computation at the same time, so this comparison alone cannot tell how much semantic information the noisy partner provides. Still, the results fit a contribution from cross-modal exchange beyond the target count. Compared with Multi-task at 50k updates, DC with matched work (32.1k updates, the same work) gains 6.9 GenEval but loses 7.9 CIDEr and 2.5 DPG. Because this is a mixed result and not a uniform gain, we draw our conclusions about joint training from the matched DC and blocked joint controls.

Controls on representation spaces

Table 28 provides the full metrics underlying Figure 5.

Diagnostics of encoders and decoders

Text readout across representations. To separate representation recovery from generation of continuous states, we encode reference captions held out from training and decode the resulting clean states without passing them through MM-RiT. Table 29 reports the token accuracy of linear readout heads fitted to each representation, which measure how much token information each space retains, and of the parallel decoders of the pretraining study (Appendix § 10.§ 5.2), which decode its captioning results.

Pairs of image encoders and decoders. Table 30 complements the downstream representation comparison with reconstruction fidelity. Lower rFID is better. The FLUX VAE reconstructs most faithfully on all three reconstruction sets among the learned pairs of encoder and decoder, while DINO and SigLIP2 give substantially stronger downstream GenEval in the fixed 668M generation comparison. These results show that reconstruction fidelity alone does not determine the downstream generation ranking.

Extended benchmark comparisons

Tables 31 and 33 extend the main comparisons to larger models, and 32 gives GenEval at 25, 50, and 100 sampling steps. Table 34 lists all 10 understanding benchmarks for both JUICE

(a) GenEval
Model	Size	Single	Two	Count	Color	Position	Attr.	Overall
BLIP3-o [39]	8B	–	–	–	–	–	–	0.84
Janus-Pro-7B [51]	7B	0.99	0.89	0.59	0.90	0.79	0.66	0.80
Show-o2 [1]	7B	1.00	0.87	0.58	0.92	0.52	0.62	0.76
TUNA [21]	7B	1.00	0.97	0.81	0.91	0.88	0.83	0.90
TUNA-R [2]	7B	1.00	0.95	0.82	0.89	0.86	0.79	0.88
Tuna-2 [2]	7B	0.99	0.96	0.80	0.91	0.84	0.76	0.87
BAGEL [54]	7B/14B	0.98	0.95	0.84	0.95	0.78	0.77	0.88
MMaDA [3]	8B	0.99	0.76	0.61	0.84	0.20	0.37	0.63
JUICE (SigLIP2)	2.1B	0.99	0.93	0.78	0.91	0.85	0.73	0.87
JUICE (DINO)	2.1B	0.99	0.95	0.81	0.93	0.85	0.78	0.88
(b) DPG-Bench
Model	Size	Global	Entity	Attribute	Relation	Other	Overall
BLIP3-o [39]	8B	–	–	–	–	–	81.60
Janus-Pro-7B [51]	7B	86.90	88.90	89.40	89.32	89.48	84.19
Show-o2 [1]	7B	89.00	91.78	89.96	91.81	91.64	86.14
TUNA [21]	7B	90.42	91.68	90.94	91.87	90.73	86.76
TUNA-R [2]	7B	86.00	91.80	91.03	93.48	84.89	86.35
Tuna-2 [2]	7B	89.50	91.40	92.07	91.91	88.81	86.54
JUICE (SigLIP2)	2.1B	82.67	91.41	90.44	94.58	86.80	85.22
JUICE (DINO)	2.1B	83.28	90.24	88.78	95.05	81.60	84.38

variants, with VQAv2 on two splits, and Table 35 compares the benchmarks that are not in the main table with models that report them.

Model	Size	OK-VQA	SQA-IMG
D-DiT ($256^{2}$) [18]	2.1B	28.5	–
D-DiT ($512^{2}$) [18]	2.1B	25.3	–
LLaVA-1.5 [64]	7B	–	66.8
Qwen-VL-Chat [74]	7B	56.6	68.2
JUICE (SigLIP2)	2.1B	33.9	80.6
JUICE (DINO)	2.1B	28.7	73.7

Joint reasoning

Mazes: dataset, adaptation, and evaluation

Dataset construction. We construct a variant of the maze task with multiple solutions [14] to evaluate the correctness and consistency of jointly generated image and text outputs. Starting from grid mazes generated by randomized depth first search, we remove additional walls at dead ends to introduce cycles. We sample distinct start and goal cells and retain puzzles with between two and sixteen shortest routes. For each training example, we uniformly select one

Model	Size	Input	VQAv2↑	GQA↑	MME-P↑	POPE↑
Models for understanding only
LLaVA-1.5 [64]	7B	$336^{2}$	78.50	62.00	1510.7	85.90
Qwen-VL-Chat [74]	7B	$448^{2}$	78.20	57.50	–	–
Composite unified models
BLIP3-o [39]	8B	Dyn.	83.10	–	1682.6	–
Native unified models
Janus-Pro-7B [51]	7B	$384^{2}$	–	62.00	1567.1	87.40
Show-o2 [1]	7B	$432^{2}$	–	63.10	1620.5	–
TUNA [21]	7B	$512^{2}$	–	63.90	1641.5	–
TUNA-R [2]	7B	$512^{2}$	–	63.50	–	–
Tuna-2 [2]	7B	$512^{2}$	–	65.00	–	–
BAGEL [54]	7B/14B	Dyn.	–	–	1687.0	–
OneFlow Mixed [23]	8B	$512^{2}$	73.70	61.90	1542.5	86.30
MMaDA [3]	8B	$512^{2}$	76.70	61.30	1410.7	86.10
JUICE (SigLIP2)	2.1B	$224^{2}$	73.10	58.58	1115.7	83.97
JUICE (DINO)	2.1B	$224^{2}$	68.75	55.68	960.2	82.29

shortest route and use it to construct both targets: an image with the route painted blue and its corresponding sequence of moves from $\{U, D, L, R\}$. Thus, each training pair is consistent, while a given puzzle admits multiple valid output pairs. The model receives an unsolved maze image together with a text question that gives the grid size and the start and goal coordinates. Images are $224 \times224$, and each maze block lines up with one $14 \times14$ image patch. Walls, background, and the start and goal markers are part of the input, and the model draws the route. There are 200,000 training examples with grids of $5 \times5$, $6 \times6$, and $7 \times7$. We test on 500 puzzles (167, 167, and 166 of the three sizes) whose layouts do not appear in the training set, and at evaluation we accept any valid shortest route, not only the route stored as the target.

Training. We initialize the 2.1B models from the raw weights of the SFT stage (step 15,000; Table 23) and adapt them for 5,000 updates. We use AdamW with a global batch size of 1,024, a learning rate of $10^{-4}$, 250 warmup steps, and weight decay 0.01. The joint adaptation uses 50% joint generation examples, 25% image generation examples conditioned on text, and 25% text generation examples conditioned on images (4, 2, and 2 of every 8 rows in a microbatch, from nominal weights 0.6, 0.2, and 0.2). The Image FT only and Text FT only models are finetuned (FT) on the image or the text target alone; Table 7 reports the validity of that modality.

Inference and evaluation. At inference, the given image patches and question tokens remain fixed, while the image target and answer positions are initialized from noise. Coupled sampling evaluates both velocity fields from the same current state of image and text and advances both generated streams at each Euler step. For sequential sampling, one sampler first generates the solved image and then runs image-to-text generation to get the text solution, while the other generates text first and then runs text-to-image generation. In all samplers, the linear task head of the mazes reads the moves from the generated text states (Appendix § 10.§ 5.2).

We use three measures. Image validity (I) checks that the decoded route is a legal shortest path from start to goal, and text validity (T) applies the same check to the move sequence. Joint success (J) further asks that both outputs show the same route. If the image and the moves each give a valid but different shortest route, the sample fails.

Maze sampling budgets

One backbone evaluation is one forward pass of MM-RiT; calls to the encoders and decoders are not counted. Without guidance, coupled sampling gets both velocity fields from the shared state in a single forward pass and then advances both streams. Independent sampling advances each stream on its own, with the other stream left at its initial noise. Sequential sampling completes the first stream, then uses its clean output as the condition for the second. Table 37 gives the steps per stream at 16 and at 8 calls, together with joint success at 8 calls. Halving the budget changes coupled sampling by at most 0.3 points and leaves the order of the samplers unchanged. Results are means over two sampling seeds; Table 36 lists each seed at 16 calls. On the same 500 mazes, paired by maze over both seeds, coupled sampling alone solves 10 mazes and image first alone 25 for SigLIP2 (exact McNemar test, $p=0.017$), and 19 against 29 for DINO ($p=0.19$). For SigLIP2, image first is therefore better than coupled sampling, while for DINO the two do not differ significantly.

Sampler	SigLIP2 Seed 1	SigLIP2 Seed 2	DINO Seed 1	DINO Seed 2
Coupled	95.2	92.2	82.2	81.2
Image first	95.8	94.6	83.4	82.0
Text first	84.8	85.8	59.4	60.4
Independent (8 steps per stream)	56.0	53.2	42.6	42.8

Maze trajectory case studies

Figure 11 shows mazes 37 and 61, in which coupled sampling reaches agreement while independent sampling ends with two valid but different routes. In maze 37 (Figure 11a), the moves are fixed from the first step and the drawn route changes to their route at step 3; in maze 61 (Figure 11b), the drawn route is fixed from step 2 and the moves change to it at step 7. Both directions also occur in other mazes of the same run: the drawn route changes in maze 208, and the moves change in maze 23. Their independent counterparts again end with two valid but different routes.

Visual spatial planning

VSP [15] poses navigation problems on FrozenLake grids. The input is an image of the map with the start, the goal, and holes, and the answer is a sequence of moves (U, D, L, R) that reaches the goal without stepping into a hole; any such sequence counts as a success, and moves off the grid leave the player in place. Levels 3 to 8 correspond to $3 \times3$ to $8 \times8$ grids, with 100 test maps per level. As for the mazes, JUICE generates the map with the path drawn on it and the move sequence together, with coupled sampling, 16 steps, and no guidance, and a linear task head reads the moves (Appendix § 10.§ 5.2). We adapt the finetuned SigLIP2 model for 5,000 updates (Table 23) on maps of sizes $3 \times3$ to $6 \times6$, so levels 7 and 8 use map sizes that do not appear in training.

The training set has 55,939 maps. We generate FrozenLake maps with random start and goal positions and a hole probability drawn from $U(0, 0.35)$, keep only solvable maps, and use the shortest path found by breadth first search as the target route, breaking ties in the order L, U, D, R. After removing duplicates and every map that appears in the VSP benchmark, 50,000 generated maps remain. The other 5,939 maps are the 6,000 FrozenLake problems of ThinkMorph [57] without the 61 that coincide with benchmark maps. All maps are rendered again on a $224 \times224$ canvas in which each cell covers $2 \times2$ SigLIP2 patches, and the target image draws the route in red. Adaptation uses the optimizer settings and task ratios of the maze recipe (Appendix § 13.§ 2).

We score the move sequences with the official evaluation script on the 100 benchmark maps of each level. The main metric averages levels 3 to 6, the sizes seen in training, and levels 7 and 8 measure extrapolation to larger maps. The official rule checks only the move sequence; on 500 maps of our own held out from training, we also check that the drawn path and the moves agree (joint consistency, 99.6%).

On the benchmark maps of levels 3 to 8, the drawn path is a shortest route on 98, 100, 100, 99, 99, and 96 of the 100 maps, and the moves reach the goal on 100, 100, 100, 99, 94, and 69. We split each reference route into runs of the same repeated move. On levels 7 and 8, the generated moves drop exactly one move from 2 of the 238 runs of length one, from 2 of 117 runs of length two, 4 of 81 of length three, 7 of 45 of length four, 7 of 23 of length five, and 7 of 9 of length six. On levels 3 to 6, no move is dropped from any of the 810 runs.

The baseline values in Table 8 are taken from LatentUM [55]. InternVL3.5-38B [56] is evaluated without training for the task, using the official VSP prompt. ThinkMorph [57] is finetuned from BAGEL-7B on interleaved reasoning data that include 6,000 FrozenLake problems with grids of size 3 to 6. LatentUM is finetuned on data in the style of VSP: its coarse variant on the 6,000 ThinkMorph problems, and its fine variant on 18,000 problems solved step by step that interleave single moves with updated maps. LatentUM reports InternVL3.5 and ThinkMorph only as averages, and ThinkMorph does not state which levels its evaluation covers.

Qualitative examples

We present selected qualitative examples from five generation and understanding settings, complementing the aggregate evaluations in § 6. Figure 9 in the main text draws its questions from Figure 26 to Figure 29, and Figure 10 shows two pairs each from Figure 19 and Figure 20.

Vector (PDF)

Vector (PDF)

Image generation conditioned on text

Each pair uses the same prompt for SigLIP2+T5 and DINO+T5 and guidance $w=5$ with the generic caption reference, with outputs selected independently from the available samples; some of these prompts also appear in Figure 7, which shows further prompts.

Vector (PDF)

Vector (PDF)

Image captioning

Each input image is shown with the caption of each model, and both models get the prompt “Describe the image concisely.” In these examples, the two encoder configurations mention different entities, attributes, and scene details.

Vector (PDF)

Vector (PDF)

Joint generation

Both image and text streams start from noise, producing paired visual and textual outputs without a text prompt. We show three independently sampled pairs from each encoder configuration, drawn from the finetuned checkpoints. Text is read out with the autoregressive caption decoder (Appendix § 10.§ 5.2) and shown up to the point where it starts to repeat. Figure 21 shows how four samples form during sampling.

SigLIP2 + T5

Qualitative examples

SigLIP2 + T5, sample 25

step 8/64 t: image 0.02, text 0.12 The image depicts a scene depicting a scene of a snowboarding in a snowy environment. The image depicts a person wearing a white shirt and a helmet with ...

step 16/64 t: image 0.04, text 0.25 The image captures a person on a snowboarder in action, navigating through the snow. The snowboarder is wearing a red helmet, gloves, and a red jacket, with a ...

step 24/64 t: image 0.07, text 0.38 The image depicts a person on a skier in action, pushed into the snow. The skier is wearing a white helmet, gloves, and a red jacket, with a ...

step 32/64 t: image 0.11, text 0.50 The image depicts a person on a skiboard in action, pushed into the snow. The skier is wearing a white helmet, gloves, and a red jacket, with a ...

step 64/64 t: image 1.00, text 1.00 The image depicts a person on a snowboardboard in action, pushed into the snow. The skier is wearing a white helmet, gloves, and a red jacket, with a ...

SigLIP2 + T5, sample 3

step 8/64 t: image 0.02, text 0.12 The image depicts a charming, cartoon-style illustration of a charming, cartoon-style character with a young child. The image depicts a charming, cartoon-style illustration of a child with a ...

step 16/64 t: image 0.04, text 0.25 The image depicts a charming, cartoon-style character that appears to be a young boy. The character has light brown hair, adorned with a bald ear and is dressed ...

step 24/64 t: image 0.07, text 0.38 The image depicts a charming, cartoon-style character that appears to be a young boy. The boy has light brown hair, adorned with wing-like wings and stands out with ...

step 32/64 t: image 0.11, text 0.50 The image depicts a charming, cartoon-style character that appears to be a young boy. The boy has light brown hair, adorned with wing-like wings and stands out backward ...

step 64/64 t: image 1.00, text 1.00 The image depicts a charming, cartoon-style character that appears to be a young boy. The boy has light brown hair, adorned with wing-like wings and stands out backward ...

DINO + T5, sample 5

step 8/64 t: image 0.02, text 0.12 The image depicts a bustling urban street scene, likely in a city of London, with the image depicts a bustling urban street scene. The image depicts a bustling ...

step 16/64 t: image 0.04, text 0.25 The image depicts a bustling urban street, likely in a city of London, given the architecture and signage. The cobblestone street is lined with pedestrian walkways and is ...

step 24/64 t: image 0.07, text 0.38 The image depicts a bustling urban street, likely in Paris, Germany, given the architecture and signage. The cobblestone street is lined with pedestrian walkways and is active with ...

step 32/64 t: image 0.11, text 0.50 The image depicts a bustling urban street, likely in Madrid, Spain, given the architecture and signage. The cobblestone street is lined with pedestrian walkways and is active with ...

step 64/64 t: image 1.00, text 1.00 The image depicts a bustling urban street, likely in Portugal, Spain, given the architecture and signage. The cobblestone street is lined with pedestrian walkways and is lively with ...

DINO + T5, sample 20

step 8/64 t: image 0.02, text 0.12 The image depicts a woman dressed in a fashion adorned with a woman dressed in a fashion, dressed in a robe or a dress with a white accent. ...

step 16/64 t: image 0.04, text 0.25 The image showcases a character that appears to be from a fantasy setting, characterized by a dark setting, surrounded by wooden shelves and a red headscarf. The character ...

step 24/64 t: image 0.07, text 0.38 The image showcases a character that appears to be from a fantasy setting, characterized by a striking appearance of leather arm plates and a red headscarf. The character ...

step 32/64 t: image 0.11, text 0.50 The image showcases a character that appears to be from a fantasy setting, characterized by a striking use of leather arm plates and a red headscarf. The character ...

step 64/64 t: image 1.00, text 1.00 The image showcases a character that appears to be from a fantasy setting, characterized by a striking use of leather arm bands and a red headscarf. The character ...

Captioning with long answers

For each image, both models receive the prompt “Describe this image in detail.” The first text column is SigLIP2+T5 and the second is DINO+T5.

	SigLIP2 + T5	DINO + T5
!Jets trailing colored smoke	The image captures a dynamic aerial display featuring four military jets flying in formation against a clear sky, leaving behind distinct trails of vibrant smoke—red and blue— while the colors of smoke create a striking visual and blue effect against the backdrop of jets, enhancing their performance. The jets appear to be tactical or different, with one positioned at a different height than others, showcasing the precision and coordination involved.	The image captures a dynamic aerial display featuring four jets flying in formation against a clear sky. Each jet is emiting colorful smoke trails— vibrant red, pinks, and blue—creating a striking contrast against the sky. The jets appear to be executing a geometric maneuver, with one jet being positioned in the center and the three extending inward, showcasing their performance capabilities.
!Wedding under a canopy	The image captures a joyful wedding moment under a white canopy. A bride, dressed in a white strapless wedding gown and veil, holds a bouquet of green and white flowers, standing still together. She is smiling while facing a groom in a suit with a red bow tie and a boutonniere. In the background, another person in a blue suit stands beside them, smiling at the front.	The image captures a joyful moment at what appears to be an outdoor wedding or wedding ceremony. A bride, dressed in a white lace wedding gown with a flowing veil, holds a bouquet of flowers and standing beside her. She is smiling as she looks to her left. A man in a dark blue suit with a red bow tie stands on the left, observing the couple.
!Stone doorway	The image depicts the entrance facade of an old stone building, likely of a historical or religious significance. The focal point is a wooden doorway with a glass window above it, adorned with intricate carvings and a mix of colorful glass. The window's arch is framed by elaborate metalwork, including decorative elements and hinges. Flanking the door are a large, weathered texture, suggesting the age and historical charm of the structure.	The image depicts an ornate entrance to a building, featuring a prominent wooden double door with intricate carvings by a metal wrought iron gate. Above the door is a large, arched window with intricate green tile patterns, adding a touch of color to the stone backdrop. The wall around the doorway is adorned with elaborate stonework and decorative elements, suggesting the historical or cultural significance of the location.

	SigLIP2 + T5	DINO + T5
!Close-up view of red cobblestones	The image depicts a close-up view of a cobblestone pathway, constructed from rectangular-shaped stones arranged in a staggered and slightly uneven pattern. The cobblestones exhibit a rich, shiny red with black and gray hues, suggesting they have been exposed to the elements for some time. The texture of the pebbles is rough, with various layers of sediment, indicating the variations in types within the overall work.	The image depicts a close-up view of a stone wall constructed from large irregularly shaped blocks of stones. The stones have a rough, reddish-brown color and appear to be mostly in shades of brown and gray, with some sections showing scratching lichen or discolorationation on them. The texture of the stones is rough and somewhat uneven, indicating an ancient or weathered surface.
!Modern bedroom with a black color scheme	A modern bedroom with a black color scheme. The room features two double beds, a ceiling fan with a light fixture, and a large window that allows natural light to enter. There is a mirror mounted above the bed, and a ceiling fan with a key light mechanism. The floor is covered with white tiles, reflecting the room. To the right, there is a white closet with the door door closed.	The image depicts a modern, minimalist bedroom with a predominantly white aesthetic. The room features a black bed frame with white bedding and pillows, positioned against two windows with vertical blinds that allow ample natural light to filter through. A ceiling fan with white blade and light fixtures is overhead, directly above the bed. To the right of the bed, there is a sleek black rectangular mirror mounted on a wooden floor, reflecting light of the room.

Visual question answering

Each case includes the input image, question, and both model answers. Reference answers are included where provided. The selected examples span attributes, object size and count, spatial relations, actions, compositional queries, outside knowledge, and object existence.

Vector (PDF)

SigLIP2 + T5    DINO + T5

Question

Is the green umbrella to the right or to the left of the person who is to the left of the palm tree?

SigLIP2 + T5: left

DINO + T5: left

Reference: left

Question

Are the thick books to the left or to the right of the computer that is to the right of the lamps?

SigLIP2 + T5: right

DINO + T5: right

Reference: right

Question

What is the woman that is to the left of the pillows doing, sitting or walking?

SigLIP2 + T5: sitting

DINO + T5: sitting

Reference: sitting

Question

What's the player doing?

SigLIP2 + T5: jumping

DINO + T5: jumping

Reference: jumping

Vector (PDF)

SigLIP2 + T5    DINO + T5

Question What does he have in his hand? SigLIP2 + T5: The man has a frisbee in his hand. DINO + T5: He may have a yellow frisbee in his hand, indicating that he is about to play with a frisbee.

Question What color is the fire hydrant? SigLIP2 + T5: The fire hydrant is white. DINO + T5: The fire hydrant is cream colored white.

Question What color is the sky? SigLIP2 + T5: The sky appears to be dark gray, suggesting a cloudy day. DINO + T5: The sky is filled with clouds, suggesting it gray and ominous, indicating stormy weather or perhaps dark weather.

Question What is the woman floating on? SigLIP2 + T5: The woman is floating on a surfboard in the water. DINO + T5: The woman is floating on a surfboard, located near the ocean during a sunset.