WorksComputer Science
Oddin: Latent Visual Reasoning in Unified Multimodal Models via Self-Distillation from Image Generation
Abstract
Unified large multimodal models (ULMMs) are able to perform visual reasoning through interleaved text-image generation, but generating explicit intermediate images incurs significant computational overhead. While visual latent reasoning addresses this, most existing methods rely on vision-language models (VLMs) incapable of visual generation. We propose Oddin, a novel framework enabling latent visual reasoning in ULMMs. Oddin replaces intermediate images with compact latent tokens processed entirely by the understanding branch, reserving the generation branch for the final output image. We train the model via offline self-distillation with three objectives, final image generation losses, text distillation losses, and cross-entropy losses for special tokens. Oddin surpasses interleaved generation baselines and closed-source VLMs on average across seven VBVR-Pro visual reasoning tasks. Qualitative analysis reveals that latent tokens attend to relevant regions in an input image, demonstrating that ULMMs can perform complex visual reasoning without generating intermediate images.
Introduction
Unified large multimodal models (ULMMs) aim to understand and generate both text and images within a single architecture. Early approaches adopt a single shared transformer backbone to process both modalities [29, 35]. However, since visual understanding and generation require distinct capabilities, fully sharing parameters leads to suboptimal performance [2, 26]. Recent models address this by introducing decoupled branches for understanding and generation [2, 4, 5]. By operating on a shared multimodal sequence, these models can reason over images and generate new ones within a single context.
Building on this capability, recent studies apply ULMMs to visual reasoning through interleaved generation of text and images, showing that the resulting multimodal interleaved reasoning outperforms text-only Chain-of-Thought (CoT) [25] on certain tasks [8, 12, 14]. However, generating intermediate images incurs computational overhead, as the model must reconstruct pixel-level details, including redundant background regions.
to visual reasoning through interleaved generation of text and images, showing that the resulting multimodal interleaved reasoning outperforms text-only Chain-of-Thought (CoT) [25] on certain tasks [8, 12, 14]. However, generating intermediate images incurs computational overhead, as the model must reconstruct pixel-level details, including redundant background regions. In contrast, the information passed between reasoning steps is far more compact.
Visual latent reasoning addresses this inefficiency by replacing explicit intermediate images with compact latent representations [6, 13, 17, 22, 24, 31, 33]. However, most existing methods are built on vision-language models (VLMs), which cannot generate images. As a result, they are inapplicable to visual reasoning tasks requiring visual outputs, such as VBVR-Pro [30]. ULMMs avoid this limitation, since their generation branch can produce images. This raises a natural question: can a ULMM reason with compact latents in place of intermediate images, while still generating the final visual output?
We propose Oddin,1 a novel training framework for latent reasoning in ULMMs. Oddin replaces each intermediate image with a latent token block processed by the understanding branch, while the generation branch only produces the final image. We train Oddin via offline self-distillation, using reasoning traces generated by the model itself through interleaved generation. Without explicitly supervising the latent tokens with intermediate images, we train the latent tokens to implicitly acquire the required reasoning capabilities at their positions. To achieve this, we train SenseNova-U1.5 [4] with three losses: a final image generation loss, a text distillation loss, and a special-token cross-entropy loss.
Oddin surpasses the performance of interleaved generation and other baselines. Qualitative analysis reveals that the latent tokens attend to the specific input regions that the intermediate image was supposed to draw. These findings show that a unified model can perform visual reasoning within the understanding branch without relying on intermediate image generation, while reserving the generation branch for producing the final image.
Related work
Interleaved Generation for Visual Reasoning. For visual reasoning, ULMMs can perform interleaved generation, producing intermediate images within reasoning trace [1, 8, 12, 14]. Most datasets and benchmarks used in this line of work, such as V* [27] and Zebra-CoT [12], target questions with text answers. Recent ones such as VBVR-Pro [30], RIG-Bench [16], and GenPuzzle [34] require the answer itself to be an image. Although intermediate images do not always help [7], interleaved generation in recent ULMMs is now strong enough on suitable tasks [5, 30] to serve as a teacher, which Oddin exploits through self-distillation.
Visual Latent Reasoning. Extending latent reasoning in language models [3, 9], visual latent reasoning replaces explicit intermediate images with compact latent tokens [13, 15, 23, 24, 31]. However, these methods are built on VLMs and always answer in text, so they cannot handle tasks that require image outputs. In unified models, latent reasoning has mainly been applied to text-to-image generation [18, 32]. LatentUM [11] reasons over visual states in a shared semantic space, but each state is still a full sequence of generated visual tokens, and the model requires new pretraining. Oddin adapts a pretrained ULMM to interleaved latent visual reasoning through offline self-distillation, replacing each intermediate image with latent tokens.
Method
Preliminaries
SenseNova-U1.5 employs a Mixture-of-Transformers architecture using shared self-attention, with modality-specific branches. The understanding branch processes multimodal inputs to generate text, and emitting an <img> token triggers the generation branch to produce an image. During interleaved reasoning, the model alternates between text and intermediate images before producing the final answer image . Because the understanding branch controls image generation, intermediate images can be replaced by other tokens without disrupting the final image generation.
Interleaved Latent Reasoning of the Unified Model
We aim to replace explicit intermediate images with compact latent representations that support subsequent reasoning and final-image generation. To achieve this, we design a latent reasoning mechanism where the intermediate explicit images are replaced by latent blocks . Each latent block consists of learnable embeddings enclosed by special tokens <lnt> and </lnt>.
During inference, instead of generating pixel-level images at each intermediate step, the understanding branch processes these latent tokens. These tokens help the subsequent reasoning to effectively reach the final answer . The generation branch is used only for generating a final image when <img> is predicted by the understanding branch. The number of latent blocks and the timing of the final image are determined by the model.
Offline Self-Distillation
Teacher Trace Collection. We use SenseNova-U1.5 as a teacher model. Given a multimodal query , the teacher model generates interleaved text–image traces for offline self-distillation, as illustrated in Figure 1 (a). During this process, we constrain the number of generated images to match the ground-truth image count. Finally, we use rejection sampling to keep only the traces whose final image passes the task-specific scorer.
Figure 1. (a) Training Overview. The teacher generates interleaved traces, and the student is trained on the traces where intermediate images are replaced with latent blocks. (b) Student architecture and gradient flow. We train the latent token embeddings, latent block-specific LoRA, and the output embeddings for the <lnt> and <img> tokens, while keeping the remaining parameters frozen.
Student Sequence Construction. As illustrated in the lower part of Figure 1 (a), we construct the student sequence by replacing each intermediate image in the teacher trace with a latent block consisting of latent tokens enclosed by <lnt> and </lnt>, while keeping the reasoning text unchanged. We retain the final <img> token and use the teacher’s final image as the flow matching target. We also add latent block usage instructions to the Oddin’s system prompt (See Appendix § 7.§ 4).
Training Objectives
We train the student model using image generation loss, text distillation loss, and special-token cross-entropy loss, where denotes the weights and the distillation temperature:
Image generation loss. We reconstruct the target image from a noisy state , where , , and is the noise scale. Conditioned on the student’s state , the frozen generation branch predicts the clean image . Inspired by region-focused supervision [28], we formulate the loss to explicitly penalize errors in edited areas:
where is a binary mask indicating the changed regions between the input and target images. See Appendix § 6.§ 1 for details.
Text distillation loss. We align the output distributions of the image-conditioned teacher and the latent-conditioned student through teacher-to-student KL divergence:
where contains all aligned text-prediction positions, excluding predictions of <lnt> and <img>. Both distributions use temperature and retain the teacher’s top-50 tokens, with the remaining probability mass aggregated into one tail probability. See Appendix § 6.§ 2 for details.
Special-token cross-entropy loss. At teacher image-generation positions, we supervise <lnt> for intermediate latent blocks and <img> for the final image:
where is the target special token and balances intermediate and final events. Probabilities are normalized over the full vocabulary. See Appendix § 6.§ 3 for details.
Trainable Parameters and Gradient Flow
As shown in Figure 1(b), we train the latent tokens, the special tokens <lnt> and </lnt>, and LoRA adapters [10] while freezing all other parameters. LoRA is active only within latent blocks, including their special tokens. The image generation and text distillation losses backpropagate through frozen operations to the latent embeddings and adapters. For the special-token cross-entropy, we detach the hidden states to update only the two output embeddings. Initialization and adapter settings are provided in Appendix § 6.§ 4.
Table 1. Accuracy on held-out VBVR-Pro-RL testset. We report per-task accuracy and the average across all seven tasks. Best in bold, second best _underlined_. The last row shows the difference of Oddin from Interleaved Reasoning.
| Method | Traffic Light | Separate Object Spinning | Dot to Dot | Rotation Puzzle | Track Object Movement | Domino Chain | Sliding Puzzle | Avg. |
| Closed-source VLMs + image generation tool | ||||||||
| GPT-4o + GPT-Image-2 | 0.22 | 0.08 | 0.02 | 0.05 | 0.42 | 0.00 | 0.81 | 0.23 |
| GPT-6.1-sol + GPT-Image-2 | 0.45 | 0.56 | 0.02 | 0.21 | 0.86 | 0.00 | 0.99 | 0.44 |
| Unified Large Multimodal models (ULMMs) | ||||||||
| BAGEL-7B | 0.00 | 0.00 | 0.00 | 0.00 | 0.01 | 0.00 | 0.00 | 0.00 |
| ThinkMorph | 0.00 | 0.00 | 0.00 | 0.00 | 0.02 | 0.00 | 0.00 | 0.00 |
| SenseNova-U1.5 (Text CoT) | 0.03 | 0.08 | 0.10 | 0.59 | 0.61 | 0.74 | 0.56 | 0.39 |
| SenseNova-U1.5 (Interleaved) | 0.52 | 0.70 | 0.63 | 0.64 | 0.37 | 0.53 | 0.25 | 0.52 |
| Oddin (ours) | 0.68 | 0.80 | 0.49 | 0.97 | 0.44 | 0.76 | 0.43 | 0.65 |
| Δ vs. Interleaved | +0.16 | +0.10 | −0.14 | +0.33 | +0.07 | +0.23 | +0.18 | +0.13 |
Table 1.
Experiment
Experimental Setup
Training Data. We use seven tasks from VBVR-Pro for which the teacher achieves over 20% success under interleaved reasoning and the final image can be automatically verified. We train on 17,051 successful teacher traces generated from 7K VBVR-Pro-RL and 21K native-resolution VBVR-Pro-SFT, using 1024 × 1024 images. Task selection and data construction are detailed in Appendix § 7.
Evaluation and Baselines. We evaluate accuracy on 700 held-out problems across seven VBVR-Pro-RL tasks, with 100 problems per task. We compare our model against two closed VLMs (GPT-4o [19] and GPT-6.1-sol [20]) with GPT-Image-2 [21], and three ULMMs (BAGEL-7B [2], ThinkMorph [8], and SenseNova-U1.5 [4]). For SenseNova-U1.5, we evaluate two modes: text CoT and interleaved generation. See Appendix § 7.§ 3 for details.
Result
Main results. Oddin achieves a mean accuracy of 0.65, outperforming all baselines. The results show two distinct patterns depending on the relative performance of Interleaved Reasoning and text CoT. On tasks where Interleaved Reasoning outperforms text CoT (Traffic Light, Separate Object Spinning, Rotation Puzzle, and Dot to Dot), Oddin further improves performance without generating intermediate images. The only exception is Dot to Dot, where Oddin scores 0.14 points lower. On tasks where Interleaved Reasoning underperforms text CoT (Domino Chain, Track Object Movement, and Sliding Puzzle), Oddin consistently recovers performance. It achieves the highest score on Domino Chain, even surpassing text CoT. Moreover, Oddin reduces token generation by 84% per problem compared to Interleaved Reasoning, replacing an average of 7.9
images with just one final image and 4 latent blocks (Appendix § 8.§ 3).
Qualitative analysis. Figure 2 visualizes an example of the Gaussian-smoothed attention from each latent block at a corresponding intermediate step to the input image, averaged over its 4 latent tokens. In this Dot To Dot task, attention concentrates around the dot newly connected at that step, following the drawing order given in the task instruction. This suggests that the learned latent tokens carry compact, step-specific visual information aligned with the visual reasoning trace.
Figure 2. Gaussian smoothed attention of latent blocks on each step in Dot To Dot task. Values are relative to uniform attention over the image (1 = evenly spread).
Conclusion
In this paper, we propose Oddin, a framework that enables ULMMs to perform visual reasoning using compact latent tokens processed by the understanding branch. Trained via offline self-distillation, Oddin learns the visual reasoning capabilities without explicit intermediate supervision and outperforms baselines on average. Qualitative analysis shows that the learned latent tokens attend to the relevant regions to each reasoning step, indicating that ULMMs can perform visual reasoning within the understanding branch.
Limitations and future work. In some VBVR-Pro tasks, Oddin shows lower performance compared to the strongest baselines. This indicates that while latent representations maintain global semantic reasoning, they lack precise spatial information. Future work will focus on addressing detailed visual manipulation tasks within the latent space.
Training Objectives Details
This section provides the detailed formulations and implementation specifics for the three loss components introduced in § 3.4.
Image Generation Loss
The target is defined as the final image from a successful teacher trace. Images are normalized to . The noisy input is formulated as , where denotes the noise scale of the pretrained generation branch. The timestep is computed as with , while the velocity prediction yields the clean-image estimate .
The change mask is constructed by resizing the input image to the target resolution using bilinear interpolation. A pixel is selected if its absolute difference from the target exceeds in any RGB channel. A binary dilation is applied to incorporate neighboring pixels. Let denote the prediction error. The loss terms are defined as:
The masked term averages errors across the selected pixels and RGB channels, evaluating to zero for empty masks.
Text Distillation Loss
Both models process identical reasoning text via teacher forcing. The teacher receives intermediate images, whereas the student receives latent blocks. Positions are aligned by text tokens, including text preceding the first latent block, excluding prediction positions for <lnt> or <img>. Teacher distributions are computed without gradients using the base model and system prompt.
Let denote the full-vocabulary distribution for , and denote the teacher’s top-50 tokens. To formulate the KL objective, we define the aggregated distribution over the modified vocabulary as:
This formulation extracts 50 token categories and one aggregated tail category. The teacher-to-student KL divergence is summed across text positions per trace. The total objective is scaled by a multiplier.
Special-token Cross-Entropy Loss
At positions preceding teacher image-generation events, target tokens are <lnt> for intermediate images and <img> for the final image. Cross-entropy is calculated over the full student vocabulary without tail aggregation used in Appendix A.2. Within each trace, intermediate <lnt> and final <img> losses are averaged independently and weighted equally. Traces lacking intermediate images optimize only the final <img> loss. This weighting schema defines the parameter . The preceding hidden states are detached to restrict gradients strictly to the two special-token output rows.
Trainable Parameters
The input embeddings of <lnt> and </lnt> are initialized from the pretrained <img> and </img> embeddings, respectively. Each latent token embedding is initialized from <IMG CONTEXT>. Latent token embeddings are independently trainable, and the same ordered set is reused across all blocks.
LoRA adapters are active exclusively within latent blocks, including their boundary tokens. The two special-token output rows are initialized independently from the pretrained <img> output row and remain separate from the input embeddings. The closing token </lnt> is inserted after latent tokens.
All other parameters remain frozen. The image generation and text distillation losses update the latent embeddings and LoRA adapters. The text distillation loss additionally updates the two special-token output rows. The special-token cross-entropy loss updates only these output rows due to its detached-hidden-state computation.
Experiment Details
Hyperparameters
The main configuration uses latent tokens per latent block. LoRA adapters are configured with rank 16 and . The model is trained for two epochs with a global batch size of 16. The learning rate is initialized at and follows a cosine decay schedule. The loss coefficients are set to , , , and the temperature is .
Training Data
Task Selection. We select tasks from VBVR-Pro using two criteria: (1) the teacher achieves a task success rate above 20% under interleaved reasoning, ensuring that success traces for self-distillation can be collected and (2) task success can be automatically verified from the final generated image. This yields seven tasks: Traffic Light (O-52), Separate Object Spinning (G-25), Dot to Dot Task (O-34), Rotation Puzzle (O-44), Track Object Movement (G-8), Domino Chain Branch Path Prediction (O-23), and Sliding Puzzle (O-47). For brevity, we refer to the Domino Chain Branch Path Prediction task as Domino Chain.
Data Construction. The VBVR-Pro-RL pool contains 7,000 problems across the seven selected tasks. We hold out 700 problems for evaluation and use the remaining 6,300 as training candidates, together with 21,000 randomly sampled VBVR-Pro-SFT problems. VBVR-Pro-SFT input images are rendered directly at resolution from their rendering specifications. We retain teacher traces whose final generated image passes task verification, resulting in 17,051 training traces. Our grader is adapted from the official VBVR-Pro evaluation code. The image reconstruction target is the verified teacher-generated final image.
Evaluation Details
Baselines. We compare our method against two groups of baselines. First, for closed-source VLMs, we evaluate gpt-4o and gpt-6.1-sol. Since they cannot generate images natively, we pair them with GPT-Image-2 to produce the final image based on their textual reasoning. Second, for open-weight ULMMs, we evaluate BAGEL-7B, ThinkMorph, and SenseNova-U1.5. Specifically for SenseNova-U1.5, we compare against two inference modes: Text CoT, which performs textual reasoning and generates only the final image, and Interleaved reasoning, which generates explicit intermediate images alongside text. Our model replaces these intermediate images with latent blocks while retaining final-image generation.
Evaluation configurations. All open-weight models use greedy text decoding with maximum 4,096 new text tokens, output resolution, 50 denoising steps, and text CFG scale 4.0. The remaining sampler settings follow each model’s official defaults. SenseNova-U1.5 and Qdдіn use image CFG 1.0 without CFG renormalization, whereas BAGEL-7B and ThinkMorph use image CFG 2.0 with CFG renormalization. For closed-source VLMs, we make one call per problem through the OpenAI Responses API, with the input frame and the task prompt. The model reasons freely and draws its answer by calling the GPT-Image-2 image generation tool (1024 × 1024, medium effort). The number of tool calls is left to the model, and the maximum output length is 32,000 tokens. All other decoding settings are the API defaults, and denoising steps and CFG are not controllable.
Generation and Answer Selection. During inference, except BAGEL-7B no method receives the ground-truth image count. BAGEL-7B is designed to generate one image per call and is therefore called times. Our model autonomously selects whether to emit another latent block or generate the final image, with a maximum of 12 latent blocks. Interleaved Reasoning generates images without a prescribed count, subject to a maximum of 13 images. We score its last image before </think> to exclude images redrawn in the subsequent answer. If there is no delimiter, we use the last generated image. For Text CoT and our model, we score the final generated image.
Prompts Setting
System prompts. Each model uses its own system prompt. BAGEL-7B generates one image per call, so we call it times and feed each generated image back as context.

Figure Oddin.

Figure SenseNova-U1.5.

Figure BAGEL-7B.
Additional Analyses
Additional Qualitative Analysis

Figure ThinkMorph-7B.

Figure GPT-4o, GPT-6.1-sol.
Task prompts. All open-source models receive the original VBVR-Pro task prompt without modification, as [image, prompt]. Since the closed-source VLMs reason freely on how to solve each task, they receive the same prompt without its final sentence, which prescribes the number of output images, and with “Provide an image showing the final state.” appended.
Figure 4 shows the latent-block attention on a Domino Chain Branch Path Prediction problem, visualized as in Figure 2. As the chain propagates, attention moves along with it, from the starting dominoes to the two branches. Unlike Dot To Dot, attention often runs one step ahead, concentrating on the dominoes about to fall at the next step. This suggests that the latent blocks anticipate the upcoming state transition rather than only encoding the current one.
Figure 4. Gaussian-smoothed attention of the latent block at each step on Domino Chain Branch Path Prediction, shown as in Figure 2. Attention concentrates around the dominoes that fall at the current and the next step.
Masking Analysis of Latent Blocks
To test whether the latent blocks are used for visual reasoning, we mask them from the attention at the final image generation step. In Figure 3, both settings mask up to the minimum number of latent blocks present in each task, while the “all” condition masks every block. Masking every block lowers accuracy in most tasks and brings it close to zero in Separate Object Spinning, Dot to Dot, and Sliding Puzzle. Moreover, in Separate Object Spinning, masking from the last block reduces the accuracy more than masking from the first, indicating that the final image generation relies more on the later blocks. In Dot to Dot, the opposite holds: masking from the first block reduces the accuracy more, indicating that earlier blocks carry information the later ones cannot replace. Overall, the latent blocks affect the generation of the final image, and the blocks that matter differ across tasks.
Figure 3. Masking analysis of latent blocks. Accuracy when latent blocks are masked from the first block (blue) or from the last block (red). “all” means masking every block. Black numbers mean identical value. Bars are 95% bootstrap confidence intervals over 30 problems per task. The box gives the number of latent blocks per problem.
Inference Cost
Table 2 reports the number of tokens generated per problem, counted separately for text, latent, and image tokens. Interleaved reasoning generates on average 7.9 images (8,135 image tokens), whereas Oddin generates a single final image and about 4 latent blocks (24 latent tokens), reducing the total number of generated tokens by 84%. Oddin also generates 49% fewer text tokens than Interleaved reasoning (725 371). Its total is close to that of Text CoT (1,209).
Table 2. Average number of generated tokens per problem on the 700 held-out problems. Text tokens include special tokens in the generated text (e.g., <think>). Each image counts as 1,026 tokens (1,024 image tokens plus <img> and </img>). A latent block counts as 6 tokens (<lnt>, =4 slots, </lnt>). Input tokens are excluded.
| Method | Text | Latent | Image | Total |
| Text CoT | 183 | – | 1,026 (1 img) | 1,209 |
| Interleaved | 725 | – | 8,135 (7.9 imgs) | 8,860 |
| Oddin (ours) | 371 | 24 (4 blocks) | 1,026 (1 img) | 1,421 |
Table 2.
References
- [1]Ethan Chern, Zhulin Hu, Steff Chern, Siqi Kou, Jiadi Su, Yan Ma, Zhijie Deng, and Pengfei Liu. Thinking with generated images. arXiv preprint arXiv:2505.22525, 2025. 2arxiv.org/abs/2505.22525
- [2]Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Zhang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683, 2025. 1, 4arxiv.org/abs/2505.14683
- [3]Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit CoT to implicit CoT: Learning to internalize CoT step by step. arXiv preprint arXiv:2405.14838, 2024. 2arxiv.org/abs/2405.14838
- [4]Haiwen Diao, Jiahao Wang, Chenjing Ding, Hanming Deng, Jiangnan Chen, Ruixi Zhang, Ruohui Wang, Wenwen Tong, Xiangyu Fan, Yubo Wang, Yue Zhu, Yuwei Niu, Zhengqi Bai, Zhiqian Lin, Zhitao Yang, Zhongang Cai, Bo Yang, Chen Feng, Chengguang Lv, Guangjia Liu, Guanlin Wang, Hanyu Zhang, Haojia Yu, Hongcan Xiao, Hongli Wang, Huan Wu, Huaping Zhong, Jian Fang, Jianan Fan, Jiaqi Li, Jiefan Lu, Jing Zuo, Jingcheng Ni, Junxiang Xu, Linjun Dai, Mutian Xu, Peishen Yan, Penghao Wu, Ruijie Mao, Ruisi Wang, Shihao Bai, Shuang Yang, Shuya Yang, Shuyan Zheng, Silei Wu, Siying Li, Tao Chu, Tianbo Zhong, Tongxi Zhou, Weichao Luo, Weichen Fan, Wenhao Jia, Wenjie Gao, Xiangli Kong, Yan Li, Yang Yong, Zimo Wen, Zixuan Qian, Wenxiu Sun, Ruihao Gong, Quan Wang, Lewei Lu, Lei Yang, Ziwei Liu, and Dahua Lin. SenseNova-U1.5: Towards native unified visual intelligence. arXiv preprint arXiv:2609.11929, 2026. 1, 4arxiv.org/abs/2609.11929
- [5]Haiwen Diao, Penghao Wu, Hanming Deng, Jiahao Wang, Shihao Bai, Silei Wu, Weichen Fan, Wenjie Ye, Wenwen Tong, Xiangyu Fan, Yan Li, Yubo Wang, Zhijie Cao, Zhiqian Lin, Zhitao Yang, Zhongang Cai, Yuwei Niu, Yue Zhu, Bo Liu, Chengguang Lv, Haojia Yu, Haoxue Xie, Hongli Wang, Jianan Fan, Jiaqi Li, Jiefan Lu, Jingcheng Ni, Junxiang Xu, Kaihuan Liang, Lianqiang Shi, Linjun Dai, Linyan Wang, Oscar Qian, Peng Gao, Pengfei Liu, Qingping Sun, Rui Shen, Ruisi Wang, Shengnan Ma, Shuang Yang, Siyi Xie, Siying Li, Tianbo Zhong, Xiangli Kong, Xuanke Shi, Yang Gao, Yongqiang Yao, Yves Wang, Zhengqi Bai, Zhengyu Lin, Zixin Yin, Wenxiu Sun, Ruihao Gong, Quan Wang, Lewei Lu, Lei Yang, Ziwei Liu, and Dahua Lin. SenseNova-U1: Unifying multimodal understanding and generation with NEO-unify architecture. arXiv preprint arXiv:2605.12500, 2026. 1, 2arxiv.org/abs/2605.12500
- [6]Shuai Dong, Siyuan Wang, Xingyu Liu, Chenglin Li, Haowen Hou, and Zhongyu Wei. Interleaved latent visual reasoning with selective perceptual modeling. In ACL, 2026. 1
- [7]Wenxi Gao, Guanxi Lu, Didi Zhu, Hao Mark Chen, Quan Deng, Zhican Wang, Jiankang Deng, and Hongxiang Fan. Model guides you how to draw: Adaptive visual gating for unified multimodal reasoning. arXiv preprint arXiv:2607.10004, 2026. 2arxiv.org/abs/2607.10004
- [8]Jiawei Gu, Yunzhuo Hao, Huichen Will Wang, Linjie Li, Michael Qizhe Shieh, Yejin Choi, Ranjay Krishna, and Yu Cheng. ThinkMorph: Emergent properties in multimodal interleaved chain-of-thought reasoning. In ICLR, 2026. 1, 2, 4
- [9]Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. In COLM, 2025. 2
- [10]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 3
- [11]Jiachun Jin, Zetong Zhou, Xiao Yang, Hao Zhang, Pengfei Liu, Jun Zhu, and Zhijie Deng. LatentUM: Unleashing the potential of interleaved cross-modal reasoning via a latent-space unified model. In NeurIPS, 2026. 2
- [12]Ang Li, Charles L. Wang, Deqing Fu, Kaiyu Yue, Zikui Cai, Wang Bill Zhu, Ollie Liu, Peng Guo, Willie Neiswanger, Furong Huang, Tom Goldstein, and Micah Goldblum. Zebra-CoT: A dataset for interleaved vision-language reasoning. In ICLR, 2026. 1, 2
- [13]Bangzheng Li, Xuming Sun, Jiang Liu, Ze Wang, Jialian Wu, Xiaodong Yu, Hao Chen, Emad Barsoum, Muhao Chen, and Zicheng Liu. Latent visual reasoning. In ICLR, 2026. 1, 2arxiv.org/abs/2509.24251
- [14]Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-thought. In ICML, 2025. 1, 2
- [15]Kelvin Li, Chuyi Sheng, Leonid Karlinsky, Rogerio Feris, Trevor Darrell, and Roei Herzig. Latent implicit visual reasoning. In CVPR, 2026. 2
- [16]Yutong Liu, Nan Huang, Xu Cao, and James M. Rehg. Thinking in pictures: A systematic benchmark for reasoning-driven image generation. arXiv preprint arXiv:2609.02864, 2026. 2
- [17]Chengdong Lu, Zhimo Li, Mao Shu, and Huo Cao. DeepLatent: Think with images via parallel latent visual reasoning. arXiv preprint arXiv:2606.00562, 2026. 1
- [18]Yapeng Mi, Yanpeng Zhao, Hengli Li, Chenxi Li, Huimin Wu, Xiaojian Ma, Song-Chun Zhu, Ying Nian Wu, and Qing Li. MILR: Improving multimodal image generation via test-time latent reasoning. In ICLR, 2026. 2
- [19]OpenAI. GPT-4o system card. arXiv preprint arXiv:2410.21276, 2024. 4
- [20]OpenAI. Addendum: GPT-6.1 Sol system card. https://cdn.openai.com/pdf/38e3efcf-545e-44cd-99ec-2b7eb395f4cc/oai_GPT_6_1_Sol.pdf, 2026. 4
- [21]OpenAI. System card: ChatGPT Images 2.0 and thinking mode. https://deploymentsafety.openai.com/chatgpt-images-2-0/chatgpt-images-2-0.pdf, 2026. 4
- [22]Yiming Qin, Bomin Wei, Jiaxin Ge, Konstantinos Kallidromitis, Stephanie Fu, Trevor Darrell, and YuDong Wang. Chain-of-visual-thought: Teaching VLMs to see and think better with continuous visual tokens. In ECCV, 2026. 1arxiv.org/abs/2511.19418
- [23]Jintao Tong, Jiaqi Gu, Yuqing Lou, Lubin Fan, Yixiong Zhou, Yue Wu, Jieping Ye, and Ruixuan Li. Sketch-in-Latents: Eliciting unified reasoning in MLLMs. arXiv preprint arXiv:2512.16584, 2025. 2arxiv.org/abs/2512.16584
- [24]Qixun Wang, Yang Shi, Yifei Wang, Yuanxing Zhang, Pengfei Wan, Kun Gai, Xianghua Ying, and Yisen Wang. Monet: Reasoning in latent visual space beyond image and language. In CVPR, 2026. 1, 2arxiv.org/abs/2511.21395
- [25]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022. 1arxiv.org/abs/2201.11903
- [26]Chengyue Wu, Xiaokang Chen, Zhiwei Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, and Ping Luo. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In CVPR, 2025. 1arxiv.org/abs/2410.13848
- [27]Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal LLMs. In CVPR, 2024. 2arxiv.org/abs/2312.14135
- [28]Zhihan Xiao, Lin Liu, Yixin Gao, Xiaopeng Zhang, Haoxuan Che, Songpeng Mai, and Qi Tian. LoVoRA: Text-guided and mask-free video object removal and addition with learnable object-aware localization. arXiv preprint arXiv:2512.02933, 2025. 2arxiv.org/abs/2512.02933
- [29]Jinzheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. In ICLR, 2025. 1arxiv.org/abs/2408.12528
- [30]Junxiang Xu, Ruisi Wang, Fanyi Pu, Maijunxian Wang, Ran Ji, Tongxi Zhou, Chenyang Gu, Jing Zuo, Hongcan Xiao, Yimeng Geng, Wanqi Yin, Wei Chen, Oscar Qian, Zhengyan Yan, Ziqi Huang, Haiwen Diao, Liang Pan, Bo Li, Xiangyu Fan, Dezhi Luo, Fengyuan Yu, Zehong Zhao, Qingying Gao, Tinghui Zhu, Yilan Zhang, Jingqi Tong, Pinyuan Feng, Zhengze Jiang, Letian Wang, Ziyu Guo, Renrui Zhang, Jieneng Chen, Sonia Joseph, Constantin Venhoff, Saman Motamed, Mengyue Yang, Chandra Sripada, Alan Yuille, Philip Torr, Lvmin Zhang, Vikash Kumar, Daniel Khashabi, Nikolaus Kriegeskorte, Raphaël Millière, Vincent C. Müller, Anyi Rao, Quan Wang, Ziwei Liu, Dahua Lin, Lei Yang, Hokin Deng, and Zhongang Cai. VBVR-Pro: A scalable and verifiable suite for native visual reasoning. arXiv preprint arXiv:2608.26105, 2026. 1, 2arxiv.org/abs/2608.26105
- [31]Zeyuan Yang, Xueyang Yu, Delin Chen, Maohao Shen, and Chuang Gan. Machine mental imagery: Empower multi-modal reasoning with latent visual tokens. In CVPR, 2026. 1, 2DOI
- [32]Fuxiang Zhai, Sixiang Chen, Yingjin Li, Shuaibo Li, Jianyu Lai, Tengjun Huang, and Lei Zhu. Latent action control for reasoning-guided unified image generation. arXiv preprint arXiv:2605.16961, 2026. 2arxiv.org/abs/2605.16961
- [33]Huanyu Zhang, Wenshan Wu, Chengzu Li, Ning Shang, Yan Xia, Yangyu Huang, Yifan Zhang, Li Dong, Zhang Zhang, Liang Wang, Tieniu Tan, and Furu Wei. Latent Sketchpad: Sketching visual thoughts to elicit multimodal reasoning in MLLMs. arXiv preprint arXiv:2510.24514, 2025. 1arxiv.org/abs/2510.24514
- [34]Changpeng Zhao, Yiren Song, and Jinpeng Wang. Gen-Puzzle: Benchmarking visual reasoning in image generation models. arXiv preprint arXiv:2609.05902, 2026. 2arxiv.org/abs/2609.05902
- [35]Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. In ICLR, 2025. 1arxiv.org/abs/2408.11039



