betaFeedback

WorksComputer Science

UniMore: Fine-Grained Supervision for Unified Understanding and Generation

Abstract

Unified multimodal models combine image understanding and generation within one system, creating an opportunity to improve both capabilities through paired image–text supervision. We examine the outputs of a pretrained unified model in both directions and observe a loss of specificity: generated images can contain the requested objects with incorrect attributes, while captions identify a scene without expressing its distinguishing details. These failures call for supervision that specifies both the visual content to realize and the descriptive differences to distinguish. We propose UniMore, a post-training method that derives these complementary learning tasks from the fine-grained content of image–text pairs. The generation route predicts grounded crops from their corresponding phrases, matching the scope of the condition and visual target while using the native visual objective. The understanding route combines positive-caption fitting with ranking against captions that omit or alter details, conditioned on the full image. The two routes are optimized independently, and their learned updates are combined into a single unified model without changing its inference procedure. Experiments show gains across multiple understanding and generation benchmarks.

Introduction

Unified multimodal models combine image understanding and generation within a common system ([3] (Wang et al. 2026b; Zhou et al. 2024)). Their shared architecture allows image–text pairs to support both visual understanding and image generation ([10, 6] (Xie et al. 2025c)). A shared architecture alone does not guarantee gains in both understanding and generation ([28]). A central post-training question is how to organize the information in these paired observations into effective supervision for the two prediction directions. To examine where more focused supervision is needed, we inspect what a pretrained unified model preserves when it generates an image from a description and when it describes an image.

Our diagnostics of pretrained BAGEL ([10]) reveal a gap between object-level success and the preservation of specific details (Figure 1). Given a request for pink broccoli and a red sink, the model generates both objects but leaves the broccoli green. When describing an image of three rocks with googly eyes, it identifies the objects and their arrangement but omits the largest rock’s gray-and-black color and rough texture. These examples accompany a broader difference in the corresponding diagnostics. On GenEval’s two-object color subset ([12]), both objects are detected in 88.50% of images, but only 53.75% meet both color requirements. On DOCCI ([35]), free captions retain 56.95% of reference object facts and 28.22% of detail facts.

Figure 1. Object-level success can miss specific details. Examples and aggregate diagnostics from pretrained BAGEL (Deng et al., [10]) compare (a) object detection with color fidelity on GenEval (Ghosh et al., [12]) and (b) object-fact with detail-fact retention in DOCCI captions (Onoe et al., [35]).

These failures motivate post-training that improves the realization and discrimination of specific visual content. Existing approaches strengthen the two capabilities through cross-task feedback (Jiang et al., [18]; Su et al., 2026; Jin et al., [19]), reconstruction and latent alignment (Xie et al., 2026; Luo et al., [29]), or adaptive coordination during reasoning (Bai et al., [1]). We focus on how the correspondences already present in paired observations can specify the visual content to predict and the descriptive differences to distinguish. For generation, a phrase and its grounded region define a prediction task with a matching scope. For understanding, captions that share scene content but omit or alter a detail specify which differences should affect the model’s preference.

We introduce UniMore to turn these two uses of paired observations into post-training tasks. The generation route pairs a caption phrase with its grounded crop and trains the backbone with its native visual prediction objective. The understanding route combines positive-caption likelihood with ranking against detail-deleted, detail-corrupted, and unrelated captions under full-image conditioning. Both routes start from the same pretrained model and are optimized independently. Static task-vector addition (Ilharco et al., [17]) combines their updates into a single unified model. The method reorganizes supervision from paired data while using the model’s existing prediction interfaces for visual and textual outputs.

Extensive experiments on BAGEL (Deng et al., [10]), Janus-Pro (Chen et al., [6]), and Show-o2 (Xie et al., 2025c) demonstrate the effectiveness of UniMore across understanding and generation benchmarks and targeted detail diagnostics.

Our contributions are:

  • A diagnosis of detail failures. We connect concrete generation and captioning errors to the gap between object-level success and the preservation of specific semantic content.

  • Supervision construction from paired data. We construct grounded phrase-to-crop prediction and full-image caption discrimination from paired data, then compose the adaptations in one model.

  • Evaluation across prediction interfaces. Experiments across three backbones show improvements on understanding and generation benchmarks, with target comparisons and paired diagnostics examining the effects of the supervision design.

Related Work

Unified multimodal architectures

Unified systems differ in how they connect language and visual prediction. NExT-GPT (Wu et al., 2024) connects a language model to modality-specific encoders and decoders through learned adapters. VITRON [11] combines regional visual inputs with specialist modules for generation, segmentation, and editing. X-Fusion [33] instead introduces modality-specific weights around a frozen language model to support understanding and generation.

Native unified models also differ in their visual representations and prediction objectives. Chameleon and Emu3 use autoregressive prediction over discrete multimodal tokens [3] (Wang et al. 2026b). Tar [13] aligns visual tokens with the language vocabulary through a text-aligned codebook. Show-o combines autoregressive text modeling with discrete image diffusion (Xie et al., 2025b), while Transfusion couples next-token text prediction with diffusion over continuous image representations (Zhou et al., 2024). FlexUMM [27] dynamically allocates shared and specialized computation and adjusts processing depth across inputs. Among the backbones studied here, BAGEL [10] and Show-o2 (Xie et al., 2025c) support continuous image generation, while Janus-Pro [6] separates visual encoding for understanding and generation within an autoregressive model. This diversity motivates post-training targets that can be expressed through each backbone’s native prediction interface for visual and textual outputs.

Post-training and capability coordination

One post-training approach uses cross-task feedback to guide optimization. CoRL [18] and UniGame (Su et al., 2026) formulate this interaction through co-reinforcement learning and adversarial training, respectively. UniRL [32] uses generated images to train understanding and understanding outputs to supervise generation in a self-improvement loop. GvU [36] and SRUM [19] use understanding-derived rewards to guide generation. Another approach connects the tasks through their representations. RecA (Xie et al., 2026) reconstructs an image from its visual-understanding embeddings. UNO [25] passes noised generative representations to a frozen understanding expert, using captioning and visual-regression objectives to supervise the generator.

LatentUMM [29] explicitly aligns cross-modal representations and generation–re-encoding transformations, then stabilizes latent trajectories through stochastic rollouts and preference optimization. Coordination can also occur during reasoning. UniPath [1] uses a lightweight planner to select an input-dependent coordination path and an executor trained on role-aligned trajectories to carry it out. AD-Loop (Wu et al., 2026c) learns to interleave textual analysis and visual drafting during problem solving. Controlled studies of native unified models further show that transfer depends on computational specialization and shared task knowledge (Wu et al., 2026b). UniMore studies the supervision targets used during adaptation, combining grounded visual prediction with full-image caption discrimination.

Fine-grained vision-language supervision

Fine-grained image–text matching requires sensitivity to how objects, attributes, and relations are composed. Winoground probes this ability using image pairs and captions with the same words in different orders (Thrush et al., 2022). ARO evaluates attribute, relation, and order sensitivity, and NegCLIP augments contrastive fine-tuning with composition-aware hard negatives (Yuksekgonul et al., 2023). Negative-caption quality also affects what such comparisons measure. SugarCrepe shows that linguistic artifacts can allow correct choices without grounding the decision in the relevant visual evidence (Hsieh et al., [14]).

Regional correspondence and detail-sensitive text supervision provide a foundation for learning beyond whole-image semantics. GLaMM (Rasheed et al., 2024) links natural-language responses to pixel-level segmentation masks, while GenerateU (Lin et al., [21]) learns to translate detected visual regions into free-form object names. CLIP-based caption rewards encourage descriptions to express distinctive visual content (Cho et al., [7]). FG-CLIP (Xie et al., 2025a) combines global and regional feature alignment with hard negative descriptions that alter local attributes. TokenCompose (Wang et al., 2024) uses segmentation masks to supervise token-level cross-attention in text-to-image diffusion. These methods make fine-grained correspondence explicit through region–language grounding, representation alignment, and supervision of the text-to-image generation process.

UniMore places detail constraints in the native prediction targets of a unified model. For generation, a caption phrase conditions visual prediction with its grounded image crop as the output target. For understanding, the full image conditions both a positive caption and alternatives that change specific details, and their conditional likelihoods define the ranking objective. This formulation gives the two directions distinct responsibilities: predicting the visual evidence associated with a phrase and distinguishing descriptions that differ in how they express that evidence.

Method

Problem formulation and overview

We are given a pretrained unified multimodal model with parameters θ0\theta_0 and a collection of paired images and captions D={(xi,yi)}i=1N\mathcal{D} = \{(x_i, y_i)\}_{i=1}^{N}, where xix_i is an image, yiy_i is its description, and NN is the number of pairs. The model supports image generation conditioned on text and text prediction conditioned on an image. Our goal is to adapt it so that generated images better realize the details specified in language and textual predictions better distinguish the details supported by visual evidence, while retaining both capabilities in one model.

UniMore turns specific content in paired images and captions into visual prediction and textual comparison tasks through the backbone’s native prediction interfaces (Figure 2). For generation, the model learns to predict an image crop cc from its corresponding phrase pp, matching the scope of the text condition and visual target. For understanding, it uses the full image xx to score a positive caption y+y^{+} and alternatives that omit or change its details, making their relative preference part of the supervision. We use the resulting generation and understanding datasets, DG\mathcal{D}_{G} and DU\mathcal{D}_{U}, to adapt two copies of the same pretrained model independently. Their parameter updates are then composed into the final model θ∗\theta_{*}.

Figure 2. An image–caption pair supports (a) scored phrase–crop selection for visual prediction and (b) full-image caption fitting and ranking against detail-deleted, detail-corrupted, and unrelated alternatives. Both routes adapt independently from the same backbone, then combine their updates into one model. Candidate cards are schematic, and captions are excerpted.

Grounded targets for local visual realization

The generation route uses a phrase–region correspondence within an image–caption pair to define a local prediction task. Whole-image prediction supervises an entire scene, including background and objects beyond a particular phrase. We pair that phrase with its grounded crop so that the text condition and visual target refer to the same local content. To construct this supervision, we first locate candidate regions for descriptive phrases and select pairs that agree in both location and semantic content. We then train the model to predict each retained crop from its phrase using the backbone’s native visual objective.

Constructing reliable local targets. We construct supervised phrase–crop pairs, using each text phrase as the input condition and a corresponding region cropped from the source image as the visual target. The selected crop must contain the named object and match the description used to predict it. We therefore begin by extracting object phrases together with their modifiers and grounding them in the paired image. We then assess each candidate through its detection confidence and its semantic agreement with the phrase.

For grounding confidence, we use a frozen text-conditioned detector, GroundingDINO (Liu et al., [23]), with the full image and the extracted phrases as its inputs. Let ac,ka_{c,k} be its matching logit between the box used to obtain crop cc and text-token position kk. After matching the box’s decoded text label to phrase pp, we retain its detector confidence. With σ\sigma denoting the sigmoid function, this score is

sground(p,c)=max⁡kσ(ac,k),(1)s_{\mathrm{ground}}(p,c) = \max_{k} \sigma(a_{c,k}), \tag*{(1)}

where the maximum spans all detector text-token positions in the joint phrase query. The score is retained from the proposed box when constructing the corresponding crop from the full source image.

To evaluate the cropped content against the complete phrase, we encode them with the frozen image and text encoders of CLIP (Radford et al., 2021). Let vcv_c and tpt_p denote their visual and textual feature vectors. Semantic agreement is their cosine similarity rescaled to [0,1][0,1]:

ssem(p,c)=12(1+vcTtp∥vc∥2∥tp∥2).(2)s_{\mathrm{sem}}(p,c) = \frac{1}{2}\left(1 + \frac{\mathbf{v}_c^{\mathsf{T}}\mathbf{t}_p}{\lVert\mathbf{v}_c\rVert_2\lVert\mathbf{t}_p\rVert_2}\right). \tag*{(2)}

This crop–phrase comparison complements the detector’s response to the original image. Both scores lie in [0,1][0, 1], and we combine them as

s(p,c)=λsground(p,c)+(1−λ)ssem(p,c).(3)s(p,c) = \lambda s_{\mathrm{ground}}(p,c) + (1-\lambda)s_{\mathrm{sem}}(p,c). \tag*{(3)}

The weight λ∈[0,1]\lambda\in[0, 1] balances localization confidence and semantic agreement. After confidence and geometric filtering, we retain the highest-scoring valid phrase–crop match from each source pair. Pairs without a suitable visual target are excluded. The resulting dataset DG={(p,c)}\mathcal{D}_{\mathrm{G}} = \{(p,c)\} uses the phrase as the text condition and the selected crop as the complete visual target.

Learning to realize the phrase. With these pairs in place, we train the model to reproduce the selected visual content from its description. We retain the backbone’s native visual objective, so the same target construction can be used with different visual representations. For a flow-matching generator, let z0=E(c)z_0 = E(c) be the crop encoded by the visual encoder EE, let ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I) be Gaussian noise, and let zt=(1−t)z0+tϵz_t = (1-t)z_0 + t\epsilon be the noised latent at interpolation time tt. Under this data-to-noise convention, the target velocity is ϵ−z0\epsilon-z_0, and the phrase-conditioned velocity predictor vθv_\theta is trained with

LGflow=E(p,c),t,ϵ[MSE⁡(vθ(zt,t,p),ϵ−z0)].(4)\mathcal{L}_{\mathrm{G}}^{\mathrm{flow}} = \mathbb{E}_{(p,c),t,\epsilon}\left[\operatorname{MSE}\left(v_\theta(z_t,t,p),\epsilon-z_0\right)\right]. \tag*{(4)}

The mean squared error averages over supervised latent positions and channels. Other continuous generators retain their native transport convention and time sampling. For an autoregressive visual generator, the crop is encoded as MM visual tokens q1,…,qMq_1,\ldots,q_M, and the objective is

LGAR=−1M∑j=1Mlog⁡pθ(qj∣p,q<j).(5)\mathcal{L}_{\mathrm{G}}^{\mathrm{AR}} = -\frac{1}{M}\sum_{j=1}^{M}\log p_\theta(q_j \mid p,q_{<j}). \tag*{(5)}

Here q<jq_{<j} is the preceding visual-token sequence. The native visual input ξ\xi in Figure 2 denotes the noised latent and time (zt,t)(z_t,t) for continuous prediction, or the token prefix q<jq_{<j} for autoregressive prediction. In both forms, the phrase specifies what to generate and the grounded crop supplies the matching visual content that the model learns to predict.

Full-context targets for caption discrimination

The understanding route uses the specific content of a paired caption to define comparisons between related descriptions. Positive-caption fitting teaches the model to reproduce a reference description, but it does not explicitly require that description to outrank alternatives with missing or altered details. We retain this direct supervision and add a ranking constraint that makes the relative preference explicit. To do so, we first construct caption alternatives, then score all candidates using the full image, and jointly optimize positive-caption likelihood and their separation. The full image preserves the scene context needed to judge object attributes and relationships between objects.

Constructing informative comparisons. To specify which content the model should distinguish, we pair the positive caption y+y^{+} with alternatives from three sources. Detail deletion removes specific content, detail corruption substitutes a detail, and unrelated text provides a coarse semantic contrast. The first two retain shared content with the reference, allowing the comparison to focus on what has been omitted or changed. A deleted caption can remain true, but the objective favors the fuller reference description. Thus, the target distinction concerns both fidelity to the image and completeness relative to the supplied reference. The resulting image–caption groups form the understanding dataset DU\mathcal{D}_{\mathrm{U}} used for caption discrimination under full-image conditioning.

Learning caption preferences. Ranking supplements positive-caption fitting with an explicit preference over related descriptions. We use a margin-based hinge loss so that a comparison stops contributing to the ranking penalty once the required separation is reached. This is useful for incomplete descriptions that may still be valid, since they need to rank below the reference without being continually suppressed. We score the candidates using the model’s own caption likelihood, keeping the supervision within its existing text-prediction interface. To compare candidates of different lengths, we use the mean negative log likelihood over the ∣y∣|y| supervised caption tokens, conditioned on the full image:

ℓθ(y∣x)=−1∣y∣∑j=1∣y∣log⁡pθ(yj∣x,y<j).(6)\ell_{\theta}(y \mid x) = -\frac{1}{|y|}\sum_{j=1}^{|y|}\log p_{\theta}(y_j \mid x, y_{<j}). \tag*{(6)}

Here y<jy_{<j} denotes the preceding caption tokens, and prompt and image tokens are excluded from the loss. Length normalization removes the direct sum-over-tokens effect. For training example ii, let Ni\mathcal{N}_i index its available negative captions yk−y_k^{-}. We combine positive-caption cross entropy with a hinge ranking term:

LU=ℓθ(y+∣x)+1∣Ni∣∑k∈Ni[m+ℓθ(y+∣x)−ℓθ(yk−∣x)]+.(7)\mathcal{L}_{\mathrm{U}} = \ell_{\theta}(y^{+} \mid x) + \frac{1}{|\mathcal{N}_i|}\sum_{k\in\mathcal{N}_i}\left[m+\ell_{\theta}(y^{+} \mid x)-\ell_{\theta}(y_k^{-} \mid x)\right]_{+}. \tag*{(7)}

The margin m>0m>0 specifies the desired separation, and [a]+=max⁡(0,a)[a]_{+}=\max(0,a). The first term fits the reference caption, while the second penalizes alternatives whose mean loss is less than mm above the mean token loss of the reference caption.

Independent adaptation and static composition

The generation and understanding objectives update shared backbone parameters but need not favor the same changes. In alternating training, an update that improves local visual prediction can increase caption-ranking loss, or vice versa, partially undoing progress on the other objective. Such conflicting updates are a known source of interference in multi-task optimization Yu et al. (2020). We therefore train the two routes independently from the same initialization, keeping one route’s updates from overwriting the other’s during training. After adaptation, we use task-vector addition Ilharco et al. [17] to combine the learned updates into a single unified model. Separate composition weights control their contributions and can be adjusted without retraining either route. Let θG\theta_{\mathrm{G}} and θU\theta_{\mathrm{U}} be the parameters learned from DGD_{\mathrm{G}} and DUD_{\mathrm{U}}, respectively. Their updates relative to the shared initialization are

ΔG=θG−θ0,ΔU=θU−θ0.(8)\Delta_{\mathrm{G}}=\theta_{\mathrm{G}}-\theta_{0}, \qquad\Delta_{\mathrm{U}}=\theta_{\mathrm{U}}-\theta_{0}. \tag*{(8)}

The final model is obtained by linear composition:

θ∗=θ0+αΔG+βΔU.(9)\theta^{*}=\theta_{0}+\alpha\Delta_{\mathrm{G}}+\beta\Delta_{\mathrm{U}}. \tag*{(9)}

The scalar weights α\alpha and β\beta control the contributions of the two updates and accommodate differences in their scales. They are calibrated empirically and fixed for the evaluated models. No post-merge training is performed. Composition preserves the backbone’s parameter shapes, so the final model serves both tasks through its original inference interfaces.

Experiments

Experimental setup

Implementation and baselines. Our implementation builds on TorchUMM Luo et al. [30], using its model interfaces and benchmark suite. The main comparisons include native unified multimodal models and post-training alternatives based on BAGEL Deng et al., [10]. SFT provides a supervised fine-tuning reference. RecA (Xie et al., 2026) trains image reconstruction from understanding embeddings, while UniGame (Su et al., 2026) uses adversarial perturbations at the shared token interface to challenge understanding. IRG Huang et al., [16] alternates textual reasoning and reflection with image generation. LatentUMM Luo et al., [29] aligns cross-modal representations and generation–re-encoding transformations through dual latent alignment.

Backbones and training data. We evaluate BAGEL-7B-MoT, Janus-Pro-7B Chen et al., [6], and Show-o2-7B (Xie et al., 2025c). Training pairs are drawn from text-to-image-2M (Zou et al., 2026). The generation dataset contains 2,460 phrase–crop pairs retained from 2,500 source pairs, and the caption-discrimination dataset contains 4,000 image–caption pairs.

Phrase–crop construction. Lexical and regular-expression rules extract predefined object nouns and nearby modifiers, producing phrases of one to seven words and at most eight candidates per image. GroundingDINO Liu et al., [23] proposes regions with box and text thresholds of 0.25 and 0.20. CLIP ViT-B/32 (Radford et al., 2021) supplies semantic agreement, with cosine similarity mapped to (1+cos⁡)/2(1 + \cos)/2. Equation (3) uses λ=0.65\lambda= 0.65, assigning weights of 0.65 and 0.35 to grounding confidence and semantic agreement. The final score threshold is 0.25. Boxes are padded by 0.08 and filtered to occupy between 0.01 and 0.80 of the image area, after which the highest-scoring accepted match is retained. Pairs dominated by text or logos, and pairs without a valid grounded region, are excluded.

Caption alternatives. Lexical rules construct and deduplicate up to three alternatives per positive caption. Detail deletion removes attribute, count, relation, or object-noun terms. Fallback candidates retain a first sentence or approximately the first 60% of a sufficiently long caption. Detail corruption uses fixed substitutions such as red/blue, two/three, and left/right. Sampling is seeded by the example identity, and selection cycles through deletion, corruption, and unrelated text with deduplication and random-caption fallback. The ranking margin in Equation (7) is m=0.2m = 0.2.

Backbone-specific implementation. BAGEL uses 256×256256 \times256 crops with the flow objective in Equation (4). Janus-Pro uses the visual-token objective in Equation (5), while Show-o2 retains its native continuous visual loss, transport direction, and time sampling. Both retain their native preprocessing. Show-o2 caption targets are limited to 220 tokens, including the end-of-sequence token.

Optimization. All routes use LoRA Hu et al., [15] with rank 16, alpha 32, and dropout 0.05. Learning rates (LR), warmup steps, and training processes (Procs) are listed in Table 1, while visual encoders or tokenizers remain fixed. The implementations use AdamW with momentum coefficients (0.9, 0.95), an epsilon of 10−1510^{-15}, zero weight decay, gradient norm clipping at 1.0, and a constant learning rate after linear warmup. For Janus-Pro and Show-o2, generation and caption adaptation accumulate four and two examples per optimizer step, respectively. BAGEL uses packed token batches, with expected/maximum token budgets of 256/512 for generation and 768/1,536 for understanding. In addition to LoRA, BAGEL trains the generation bridge and time components or the visual connector for the corresponding route. Janus-Pro and Show-o2 adapt language-model attention and MLP projections through LoRA.

BackboneG/ULRStepsWarm.Procs
BAGELG5×10−55 \times10^{-5}1,000504
BAGELU1.8×10−51.8 \times10^{-5}600602
Janus-ProG1×10−51 \times10^{-5}250251
Janus-ProU1.8×10−51.8 \times10^{-5}600601
Show-o2G5×10−65 \times10^{-6}500501
Show-o2U1.8×10−51.8 \times10^{-5}300301

Table 1. Training settings. G/U denote generation/understanding.

For BAGEL, Equation (7) is evaluated only for groups containing a positive caption and at least one associated negative in the same packed batch. Thus, Ni\mathcal{N}_i contains the negatives available with that positive in the batch. Groups lacking either contribute neither the positive-caption term nor the ranking term. Janus-Pro and Show-o2 keep each positive caption and its associated negatives together when computing the ranking objective for each image–caption group.

Evaluation protocol. Adapted variants share evaluation data, inference settings, and scorers within each controlled comparison. Paired diagnostics also evaluate the corresponding pretrained model under the same settings. Our BAGEL generation uses 512×512512 \times512 images, 50 sampling steps, and seed 42. Janus-Pro and Show-o2 retain their native generation sizes of 384 and 432 pixels, respectively, so comparisons are made within each architecture. WISE [34] uses Qwen2.5-VL-72B-Instruct [2] to evaluate the generated images.

Understanding benchmarks comprise MMMU (Yue et al., 2024) (900 examples), MathVista [26] (1,000), MMStar [5] (1,500), MMBench-EN, and MM-Vet (Yu et al., 2023). For Janus-Pro and Show-o2, MMBench uses TEST EN V11 with 7,299 examples and the official aggregation.

Generation benchmarks comprise GenEval (553 prompts, four images each), WISE (1,000 prompts), and UEval (Li et al., 2026) (1,000 questions). GenEval and WISE evaluate text-to-image generation, while UEval evaluates instruction following in joint image–text generation. RealUnify (Shi et al., 2026) and Unison [22] evaluate cross-task interaction and consistency between understanding and generation, with task definitions and evaluation sizes given in § 4.4. The detail-diagnostic protocols are specified alongside their results in § 4.3 and § 4.8.

Benchmark scores use a 0–100 scale, and “–” in tables denotes an unavailable result.

Model composition. All three backbones use α=1.0\alpha= 1.0 and β=0.02\beta= 0.02 in Equation (9). We select β\beta on BAGEL using accuracy on 500 MMBench-Dev [24] questions and the GenEval [12] macro-average score on 100 prompts (§ 4.6). BAGEL adds parameter updates sequentially in BF16. For Janus-Pro and Show-o2, updates are accumulated in FP32 before casting to the model precision. No additional training follows composition of the two route-specific updates.

Main understanding and generation results

In Table 2, MathVista includes multiple-choice (MC), free-form (FF), and overall (All) scores. Avg. is the unweighted mean of the five benchmarks, using MathVista All. Table 3 reports overall UEval scores and the six WISE categories together with their overall score (All).

MethodSizeMMMUMMBenchMM-VetMMStarMathVista MCMathVista FFMathVista AllAvg.
Native unified multimodal models
BAGEL (2025)7B51.9082.6565.9063.2080.1961.5271.6067.05
Janus-Pro (2025b)7B40.7067.6433.7042.0051.3032.8342.8045.37
Janus (2025)1.3B27.3027.6927.0026.6735.3716.3026.6027.05
JanusFlow (2025)1.3B29.0064.1631.8039.1939.0729.7834.8039.79
Show-o2 (2025c)7B47.9077.2547.1054.5363.5237.3951.5055.66
Show-o2 (2025c)1.5B37.1065.0946.1043.8753.3319.7837.9046.01
Show-o (2025b)1.3B26.1043.4823.3027.8744.0711.3029.0029.95
Emu3 (2026b)8B31.4059.5330.0043.6057.5930.0044.9041.89
OmniGen2 (2026a)7B46.0076.3662.7052.4771.6753.9163.5060.21
MMaDA (2025b)8B28.9028.1611.4031.7338.708.7024.9025.02
Ovis-U1 (2025)3.6B43.7078.5562.8058.2775.7460.0068.5062.36
Emu3.5 (2025)34B29.2017.5628.0030.2741.6717.6130.6027.13
BAGEL post-training
SFT7B52.6082.6561.2068.2780.0065.0073.1067.56
RecA (2026)7B52.3082.6566.1068.4779.8164.5772.8068.46
UniGame (2026)7B52.4082.7560.7068.7379.6363.4872.2067.36
IRG (2026)7B48.0060.4740.7061.5373.5261.5268.0055.74
LatentUMM (2026c)7B53.2081.0467.2065.3380.3765.6573.6068.07
UniMore7B53.3082.7567.3070.1380.7465.6573.8069.46

Table 2. Multimodal understanding. Bold and underline mark the best and second-best results within the BAGEL post-training group.

MethodUEvalCultureTimeSpaceBio.Phys.Chem.All
Native unified multimodal models
BAGEL30.9038.8343.8647.1436.2042.0529.4039.89
Janus-Pro20.6036.1638.5347.8936.0547.4524.8538.11
Janus–20.8027.0735.0817.0519.1010.9522.22
JanusFlow–27.3132.2239.4732.1528.6019.0529.64
Show-o2 (7B)29.1036.4134.9745.1934.5536.9023.9035.95
Show-o2 (1.5B)22.1031.1135.6343.5731.5038.4023.1033.49
Show-o–28.6532.2541.3227.5033.0019.8030.37
Emu3–34.6334.8237.1133.1036.8521.3033.73
OmniGen225.8041.8040.4248.8736.3538.7528.1040.29
MMaDA4.4065.0268.1474.9266.2074.2042.0565.60
Ovis-U134.2036.4339.9148.3134.0540.9023.9037.55
Emu3.5–70.0156.8369.4464.3560.8540.6063.31
BLIP3-o (2025a)–40.2841.8652.5940.2542.5530.0041.38
TokenFlow (2025)–32.5336.2633.5729.1526.0515.1030.56
DeepGen (2026a)–59.8949.5561.0247.6555.1540.8054.70
BAGEL post-training
SFT31.4022.0121.1734.6617.3025.4515.1022.74
RecA31.0040.3541.4754.3239.8546.3033.4042.25
UniGame31.0039.5641.3848.7635.9043.5531.5540.32
IRG9.1036.7440.8146.5035.7544.9526.5538.42
LatentUMM31.8041.3141.5555.4735.9046.9532.0641.80
UniMore31.6042.8643.8952.2641.1547.0032.4043.48

Table 3. Generation performance. Bold and underline mark the best and second-best results within the BAGEL post-training group.

Understanding performance. We first test whether detail-targeted adaptation improves general visual understanding. Table 2 compares native unified models and BAGEL post-training methods on five benchmarks covering perception and reasoning. UniMore achieves the highest scores among the BAGEL post-training methods on MMMU, MM-Vet, MMStar, and MathVista. The clearest improvement over the base model is on MMStar, which rises from 63.20 to 70.13. MathVista also improves by 2.20 points, with gains in both multiple-choice and free-form questions. These results show that the benefits extend to answering visual questions, consistent with the goal of learning more discriminative image–text correspondences through caption supervision.

Generation performance. We next examine whether learning from local phrase–crop pairs benefits complete-image generation and joint image–text generation. Table 3 evaluates WISE and UEval under their standard protocols. On WISE, UniMore improves BAGEL from 39.89 to 43.48 and outperforms the strongest post-training baseline, RecA, by 1.23 points. It improves all six categories over BAGEL and leads the post-training group in four. The accompanying UEval improvement extends the gains to instruction following in responses containing both images and text. Together, these results suggest that supervision concentrated on local visual content can transfer to broader generation tasks.

Detail retention and realization

We revisit the diagnostics in Figure 1 to examine whether the benchmark gains also address the observed detail failures. Figure 3 compares pretrained BAGEL, generation-only adaptation (G-only), understanding-only adaptation (U-only), and UniMore under common evaluation settings. DOCCI (Onoe et al., [35]) measures the detail facts retained in free captions, while GenEval tests whether both requested objects have their specified colors in the same generated image.

Detail retention and realization on BAGEL

Figure 3. Detail retention and realization on BAGEL. Horizontal axes are truncated. Bold labels mark the highest score in each panel.

The caption-coverage diagnostic in Figure 1 and Figure 3 uses 500 images from the official DOCCI test set, a common free-description prompt, greedy decoding, and a 500-token limit. A Qwen3-32B judge (Yang et al., 2025a) scores reference facts derived from the dataset’s Davidsonian Scene Graph questions (Cho et al., [8]) and checked against the human descriptions. The inventory contains 22,620 supported facts, including 13,578 detail facts. Coverage is the fraction of reference facts expressed in a generated caption and does not measure recognition when the model is explicitly asked about individual visual attributes.

UniMore improves detail retention by 1.55 percentage points and color-binding accuracy by 8.25 points over BAGEL. Both single routes yield smaller gains in caption detail retention and improve color binding on this diagnostic. Composition further raises detail retention above either route and retains nearly all of G-only’s color-binding gain. It therefore brings improvements in both caption detail retention and visual detail realization into a single model, addressing the two failures examined in Figure 1.

Cross-task consistency and synergy

Separate task scores do not establish how well understanding and generation work together within a task. We therefore evaluate their interaction on RealUnify and Unison (Table 4). RealUnify measures generation-enhanced understanding (GEU, 400 examples) and understanding-enhanced generation (UEG, 600 examples). Both use the Direct protocol, with Qwen2.5-VL-72B-Instruct judging UEG. Unison covers internal consistency (IC, 556 examples), understanding-guided generation (UGG, 543), generation-guided understanding (GGU, 542), and mutual enhancement (ME, 264), and summarizes these capabilities using an equally weighted mean across the four tasks.

RealUnifyUnison
MethodGEUUEGICUGGGGUMEMean
BAGEL39.0022.0080.5172.6832.6535.4155.31
UniMore40.0023.6779.5771.3333.2057.4060.37

Table 4. Cross-task consistency and synergy on BAGEL.

UniMore improves both RealUnify directions and raises the mean Unison score from 55.31 to 60.37. The largest change is in mutual enhancement. Part of this gain comes from better completion of the required response format: examples without a parseable mismatch list fall from 234 to 25 out of 264, and the evaluator assigns zero understanding credit to such responses. The ME increase therefore reflects improved format compliance as well as task performance. The gains in GEU, UEG, and GGU show benefits across several interaction patterns, while the lower IC and UGG scores indicate that adaptation does not improve every form of cross-task consistency.

Transfer across architectures

We test whether the target construction remains useful when the backbone’s visual prediction objective changes. Table 5 applies the same two routes and composition rule to the autoregressive Janus-Pro and continuous-generation Show-o2, retaining each model’s native objective. Both backbones improve on MMMU, MathVista, MMStar, and overall WISE. Janus-Pro gains 5.93 points on MMStar, while Show-o2 gains 0.98 points on WISE. This agreement across architectures supports constructing detail-targeted supervision at the image–text level, where it can be used by different native generators.

BenchmarkJanus-Pro BaseJanus-Pro OursShow-o2 BaseShow-o2 Ours
Understanding
MMMU40.7041.0047.9048.00
MMBench67.6468.0577.2577.25
MathVista42.8044.2051.5051.60
MMStar42.0047.9354.5355.60
Generation (WISE)
Culture36.1635.0536.4136.78
Time38.5339.3434.9737.46
Space47.8947.0345.1947.74
Biology36.0541.3034.5537.50
Physics47.4547.7536.9035.30
Chemistry24.8526.7523.9023.35
Overall38.1138.4335.9536.93

Table 5. Transfer across backbones. Base denotes the TorchUMM reference, and Ours denotes UniMore.

Ablation of the two training routes

To separate the contributions of the two objectives, Table 6 compares generation-only adaptation (G-only), understanding-only adaptation (U-only), and their composition under common evaluation settings, using the pretrained reference reported by TorchUMM. G-only improves both generation benchmarks and all three reported understanding benchmarks. U-only obtains the strongest MMStar result among the adapted variants, but its WISE and UEval scores are slightly below the pretrained reference. Its color-binding improvement in Figure 3 therefore does not extend to an overall gain on these generation benchmarks. Composition improves all three understanding scores over G-only while preserving nearly all of its WISE gain, scoring 43.48 compared with 43.52, and obtains the highest UEval score among the variants. Its MMMU score rises from 51.90 to 53.30, a gain of 1.40 percentage points over the pretrained reference. This pattern supports composing independently learned updates to retain strong generation performance while improving understanding, with the benefits depending on the evaluated capability.

VariantMMMUMMStarMathVistaWISEUEval
BAGEL51.9063.2071.6039.8930.90
G-only53.0068.7373.1043.5231.33
U-only53.0070.4073.2039.6730.48
UniMore53.3070.1373.8043.4831.60

Table 6. Ablation of the two training routes on BAGEL. Bold and underline mark the best and second-best available results.

Composition weight. Figure 4 varies the understanding-update weight β\beta while fixing α=1.0\alpha= 1.0. Compared with generation-only adaptation (β=0\beta= 0), β=0.02\beta= 0.02 raises MMBench-Dev accuracy from 79.2% to 82.0% while maintaining the GenEval score at 75.0. Larger weights reduce generation performance. At β=0.2\beta= 0.2, understanding reaches 82.6%, but GenEval falls to 64.0. The selected weight therefore improves understanding while preserving the generation score on the development set.

Figure 4. Composition weight on BAGEL development data. The star marks the selected β=0.02\beta= 0.02.

Effect of supervision targets

The route ablation leaves open whether the gains depend on how the paired data are organized into supervision or merely on additional training. We therefore vary the prediction and comparison targets within each route while keeping its source pool and optimizer-update budget fixed (Table 7).

(a) Generation targets
SupervisionWISE
BAGEL39.89
Caption → Image40.18
Phrase → Image42.00
Phrase → Crop43.52
UniMore43.48

Table 7. Supervision-target comparisons on BAGEL. (a) Visual prediction targets.

(b) Caption supervision
SupervisionMMMUMMStarMathVistaAvg.
BAGEL51.9063.2071.6062.23
SFT52.6068.2773.1064.66
+ Deletion52.2070.2073.2065.20
+ Deletion + corruption53.0070.4073.2065.53
UniMore53.3070.1373.8065.74

Table 7 (PDF p. 13). Supervision-target comparisons on BAGEL. (b) Caption supervision, where “+” adds ranking to positive-caption CE.

Generation targets. We compare full-caption prediction of the full image, grounded-phrase prediction of the full image, and grounded-phrase prediction of the corresponding crop. All variants use 2,460 source records, 1,000 optimizer updates, four global packed microbatches per update, and the same WISE evaluation settings. The full-image variants use two GPUs with two accumulation steps, while the phrase–crop variant distributes its training across four GPUs.

WISE increases from 40.18 with full-caption supervision to 42.00 with phrase conditioning, then to 43.52 when the visual target is also restricted to the grounded crop. The additional gain from changing the visual target supports matching the scope of the condition and prediction. A crop concentrates the visual training signal on the content named by the phrase, providing a direct learning target for the requested detail. This progression is consistent with the design rationale.

Caption supervision. We test whether detail-aware caption comparisons improve on positive-caption fitting. The three variants use positive-caption cross entropy alone, ranking against one detail-deleted caption and two random captions, or ranking against one deleted, one corrupted, and one random caption. Avg. in Table 7 is the unweighted mean of MMMU, MMStar, and MathVista.

The full alternative set yields the highest MMMU and MMStar scores among these variants. Adding corruption in place of a random alternative raises MMMU from 52.20 to 53.00 without reducing MMStar or MathVista. This comparison suggests that incorrect details provide a useful distinction beyond missing details and unrelated descriptions of other scenes.

Detail realization and recognition

Color fidelity and repeated sampling. To examine the color-binding improvement in Figure 3, we further analyze object detection and success across repeated samples for pretrained BAGEL and UniMore. The diagnostic uses 100 GenEval color-attribution prompts, each specifying two distinct object types with different colors, and four images per prompt. Both models use 512×512512 \times512 generation, 50 sampling steps, seed 42, text guidance 4, and image guidance 1. Object and color labels follow the same GenEval detector and color classifier. Each of the first three outcomes in Figure 5a requires both objects to be detected, while its percentage is computed over all generated images. The separate bottom row instead conditions on both objects being detected, for which color error decreases from 39.27% to 23.69%.

Figure 5. Color fidelity and repeated sampling. (a) Color outcomes, with the bottom row conditioned on both objects being detected. (b) Prompts grouped by correct samples out of four.

UniMore improves joint object-and-color correctness from 215 to 248 of the 400 images, a gain of 8.25 percentage points (Figure 5a). Color accuracy also improves by 14.10 points on the 305 matched pairs with complete detections for both models. This matched comparison supports improved attribute realization when both models generate detectable target objects. Figure 5b groups prompts by their number of correct images among four, showing that the gain extends to repeated success on the same prompt. The number of prompts with four correct images increases from 25 to 34. Thus, the improvement includes more prompts for which all four samples realize the requested details.

Color recognition on self-generated images. We examine whether failure to generate a requested color also entails failure to describe the color that appears in the generated image. BAGEL and UniMore each answer questions about their own color-attribution images described above. We ask about each target object separately. Each question uses a fresh image context without the original prompt or requested color. Decoding is greedy without thinking, with a 500-token limit. Answers use 11 basic colors or indicate that the object is missing, unclear, multicolored, or transparent.

Qwen2.5-VL-32B-Instruct annotates object colors from the image and object question, independently of the requested colors and model answers. We evaluate images with two definite reference colors, counting incorrect or unparseable model answers as errors. Generation fulfillment requires both reference colors to match the request, while recognition agreement requires both model answers to match the reference. Both measures use these automatic reference annotations.

We focus on images where at least one reference color differs from the requested color. Each full bar in Figure 6 represents this failure subset, and its colored segment shows images where both model answers still match the reference. BAGEL does so on 66.67% of these images, and UniMore on 72.15%. Agreement also remains frequent for individual incorrectly colored objects, at 71.43% and 75.82%, respectively. Thus, a model can describe the color that appears in its image even when it fails to produce the requested color. This separation helps explain why supervision for visual realization remains useful alongside the model’s ability to answer questions about visual details.

Figure 6. Describing colors after generation failure. Among images with at least one unmet color request, colored segments indicate agreement with the automatic reference for both objects.

Qualitative generation results. We compare generated images to show how the quantitative gains appear in concrete attribute, relation, and count requirements (Figure 7). Under the same prompts and sampling settings, UniMore assigns the requested colors to a brown mouse and purple bottle, and to a yellow suitcase and brown bus. It also places a ball to the left of an umbrella and generates the requested three books and four bowls, where the reference outputs contain an extra object. These successful examples show that the improved details can affect both an object’s appearance and the spatial arrangement or exact number of objects depicted in a scene.

Figure 7. Qualitative comparisons on GenEval, covering colors (a, b), spatial relations (c), and object counts (d, e).

Adaptation cost

We report training time and adapter size to characterize the cost of applying UniMore to additional backbones. Janus-Pro and Show-o2 each use one H100 GPU per route. Their two routes require approximately 0.595 and 0.356 GPU-hours in total, respectively. These measurements cover the training loop and adapter writing, excluding model loading, target construction, evaluation, and hyperparameter search.

Each route trains 37.48 million LoRA parameters for Janus-Pro and 40.37 million for Show-o2, with saved adapters of approximately 71.5 and 77.0 MiB. Thus, the measured adaptation requires less than one GPU-hour on each backbone and stores only compact parameter updates. Static composition incorporates these updates into the original model without adding deployed parameters.

Conclusion

We introduced UniMore to learn more from image–text pairs by organizing their specific content into visual prediction and textual comparison tasks. The method pairs grounded phrase-to-crop prediction with full-image caption discrimination, then composes the independently adapted updates into a single model. Evaluation across three backbones shows that this target construction works with different native visual objectives and improves understanding and generation benchmarks. Paired diagnostics identify improved compositional color fidelity, while the training studies examine how supervision and composition affect downstream performance. These results support supervision construction as a practical approach to adapting both prediction directions of unified multimodal models using existing image–text pairs.

Code and Data

Data availability The UniMore dataset is available at https://huggingface.co/datasets/HuggingSelf/UniMore.

Code and model availability The code is available in TorchUMM at https://github.com/AIFrontierLab/TorchUMM/tree/main/src/umm/post_training/UniMore. The model checkpoints are available at https://huggingface.co/HuggingSelf/UniMore.

References

References

  1. [1]Hayes Bai, Yinyi Luo, Wenwen Wang, Qingsong Wen, and Jindong Wang. UniPath: Adaptive Coordination of Understanding and Generation for Unified Multimodal Reasoning. In Advances in Neural Information Processing Systems (NeurIPS), 2026.arxiv.org/abs/2605.11400
  2. [2]Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL Technical Report, 2025. URL https://arxiv.org/abs/2502.13923.
  3. [3]Chameleon Team. Chameleon: Mixed-Modal Early-Fusion Foundation Models, 2024. URL https://arxiv.org/abs/2405.09818.
  4. [5]Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Feng Zhao. Are We on the Right Way for Evaluating Large Vision-Language Models?, 2024. URL https://arxiv.org/abs/2403.20330.
  5. [6]Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling, 2025b. URL https://arxiv.org/abs/2501.17811.
  6. [7]Jaemin Cho, Seunghyun Yoon, Ajinkya Kale, Franck Dernoncourt, Trung Bui, and Mohit Bansal. Fine-grained Image Captioning with CLIP Reward. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 517–527, 2022. doi: 10.18653/v1/2022.findings-naacl.39. URL https://aclanthology.org/2022.findings-naacl.39/.DOI
  7. [8]Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation, 2024. URL https://arxiv.org/abs/2310.18235.
  8. [10]Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging Properties in Unified Multimodal Pretraining, 2025. URL https://arxiv.org/abs/2505.14683.
  9. [11]Hao Fei, Shengqiong Wu, Hanwang Zhang, Tat-Seng Chua, and Shuicheng Yan. VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing. In Advances in Neural Information Processing Systems, volume 37, 2024. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/68bad5506f0f9eea7ae75f01ae00d5e2-Abstract-Conference.html.
  10. [12]Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment, 2023. URL https://arxiv.org/abs/2310.11513.
  11. [13]Jiaming Han, Hao Chen, Yang Zhao, Hanyu Wang, Qi Zhao, Ziyan Yang, Hao He, Xiangyu Yue, and Lu Jiang. Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations. In Advances in Neural Information Processing Systems, volume 38, 2025. URL https://papers.nips.cc/paper_files/paper/2025/hash/e87b1e06be8c3594c810e8991e77ea40-Abstract-Conference.html.arxiv.org/abs/2506.18898
  12. [14]Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna. SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality. In Thirty-Seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://arxiv.org/abs/2306.14610.
  13. [15]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models, 2021. URL https://arxiv.org/abs/2106.09685.
  14. [16]Wenxuan Huang, Shuang Chen, Zheyong Xie, Shaosheng Cao, Shixiang Tang, Yufan Shen, Qingyu Yin, Wenbo Hu, Xiaoman Wang, Yuntian Tang, Junbo Qiao, Hangyu Guo, Yao Hu, Zhenfei Yin, Philip Torr, Yu Cheng, Wanli Ouyang, and Shaohui Lin. Interleaving Reasoning for Better Text-to-Image Generation. In International Conference on Learning Representations, pages 106153–106182, 2026. URL https://proceedings.iclr.cc/paper_files/paper/2026/hash/ad48f017e6c3d474caf511208e600459-Abstract-Conference.html.arxiv.org/abs/2509.06945
  15. [17]Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hanneeh Hajishirzi, and Ali Farhadi. Editing Models with Task Arithmetic, 2023. URL https://arxiv.org/abs/2212.04089.
  16. [18]Jingjing Jiang, Chongjie Si, Jun Luo, Hanwang Zhang, and Chao Ma. Co-Reinforcement Learning for Unified Multimodal Understanding and Generation, 2025. URL https://arxiv.org/abs/2505.17534.
  17. [19]Weiyang Jin, Yuwei Niu, Jiaqi Liao, Chengqi Duan, Aoxue Li, Shenghua Gao, and Xihui Liu. SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models, 2026. URL https://arxiv.org/abs/2510.12784.
  18. [21]Chuang Lin, Yi Jiang, Lizhen Qu, Zehuan Yuan, and Jianfei Cai. Generative Region-Language Pretraining for Open-Ended Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13958–13968, 2024. URL https://openaccess.thecvf.com/content/CVPR2024/html/Lin_Generative_Region-Language_Pretraining_for_Open-Ended_Object_Detection_CVPR_2024_paper.html.arxiv.org/abs/2403.10191
  19. [22]Jinyu Liu, Xincheng Shuai, Henghui Ding, and Yu-Gang Jiang. Unison: Benchmarking unified multimodal models via synergistic understanding and generation. In International Conference on Machine Learning, 2026a. URL https://github.com/FudanCVL/Unison.arxiv.org/abs/2606.26984
  20. [23]Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding DINO: Marrying DINO with Grounded Pre-Training, 2024a. URL https://arxiv.org/abs/2303.05499.
  21. [24]Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. MMBench: Is Your Multi-modal Model an All-around Player?, 2024b. URL https://arxiv.org/abs/2307.06281.
  22. [25]Zeyu Liu, Zanlin Ni, Yang Yue, Cheng Da, Huan Yang, Di Zhang, Kun Gai, and Gao Huang. Steering Visual Generation in Unified Multimodal Models with Understanding Supervision, 2026b. URL https://arxiv.org/abs/2605.05781.
  23. [26]Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannahneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts, 2024. URL https://arxiv.org/abs/2310.02255.
  24. [27]Yinyi Luo, Haoyue Bai, Yi Ge, Nilton Cesar Rojas Vales, Marios Savvides, and Jindong Wang. FlexUMM: Flexible representation learning for unified multimodal models. arXiv preprint arXiv:2610.22682, 2026a. URL https://arxiv.org/abs/2610.22682.
  25. [28]Yinyi Luo, Hayes Bai, Yiqiao Jin, Wenyue Hua, Srijan Kumar, Marios Savvides, Qingsong Wen, and Jindong Wang. Beyond structural unification: A perspective and roadmap on unified multimodal models. arXiv preprint arXiv:2610.35906, 2026b. URL https://arxiv.org/abs/2610.35906.
  26. [29]Yinyi Luo, Wenwen Wang, Hayes Bai, Marios Savvides, and Jindong Wang. LatentUMM: Dual Latent Alignment for Unified Multimodal Models, 2026c. URL https://arxiv.org/abs/2605.17766.
  27. [30]Yinyi Luo, Wenwen Wang, Hayes Bai, Hongyu Zhu, Hao Chen, Pan He, Marios Savvides, Sharon Li, and Jindong Wang. TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training. In Advances in Neural Information Processing Systems (NeurIPS), 2026d.arxiv.org/abs/2604.10784
  28. [32]Weijia Mao, Zhenheng Yang, and Mike Zheng Shou. UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learning, 2025. URL https://arxiv.org/abs/2505.23380.
  29. [33]Sicheng Mo, Thao Nguyen, Xun Huang, Siddharth Srinivasan Iyer, Yijun Li, Yuchen Liu, Abhishek Tandon, Eli Shechtman, Krishna Kumar Singh, Yong Jae Lee, Bolei Zhou, and Yuheng Li. X-Fusion: Introducing New Modality to Frozen Large Language Models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 228–238, 2025. URL https://openaccess.thecvf.com/content/ICCV2025/html/Mo_X-Fusion_Introducing_New_Modality_to_Frozen_Large_Language_Models_ICCV_2025_paper.html.arxiv.org/abs/2504.20996
  30. [34]Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Fanqing Meng, Kunpeng Ning, Bin Zhu, and Li Yuan. WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation, 2026. URL https://arxiv.org/abs/2503.07265.
  31. [35]Yasumasa Onoe, Sunayana Rane, Zachary Berger, Yonatan Bitton, Jaemin Cho, Roopal Garg, Alexander Ku, Zarana Parekh, Jordi Pont-Tuset, Garrett Tanzer, Su Wang, and Jason Baldridge. DOCCI: Descriptions of Connected and Contrasting Images, 2024. URL https://arxiv.org/abs/2404.19753.
  32. [36]Jiadong Pan, Liang Li, Yuxin Peng, Yu-Ming Tang, Shuohuan Wang, Yu Sun, Hua Wu, Qingming Huang, and Haifeng Wang. Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models, 2026. URL https://arxiv.org/abs/2603.06043.

Paper details

vlnc:W_936h4eqv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents