WorksComputer Science
Teach with Blinks: Breaking Linguistic Momentum in On-Policy Distillation for Fine-Grained Multimodal Perception
Abstract
On-policy distillation (OPD) has emerged as an effective post-training approach to improving fine-grained perception in multimodal large language models by amortizing costly inference-time zooming operations into a single-pass recipe. However, vanilla OPD is inherently susceptible to linguistic momentum: the teacher can follow the student's erroneous prefixes, thus weakening corrective supervision. To address this, we propose Blink On-Policy Distillation (BlinkOPD), in which the teacher "blinks" at every token: it reads the same prefix once with its eyes open and once with its eyes closed. A first-glance gain further strengthens the correction on rollouts that conflict with the teacher. By extracting more effective supervision from each on-policy trajectory, BlinkOPD achieves state-of-the-art average performance across six fine-grained benchmarks using only 500 training images (12x less data) than existing OPD methods. Beyond fine-grained perception, BlinkOPD also demonstrates strong zero-shot improvements across ten held-out benchmarks spanning general visual perception, visual hallucination, and multimodal reasoning abilities. Code: https://github.com/Supltz/BlinkOPD
Code: https://github.com/Supltz/BlinkOPD
Introduction
Multimodal large language models (MLLMs) excel at a wide range of visual understanding tasks, yet they still struggle with fine-grained perception when critical visual evidence is small or difficult to discern, such as distant signs or tiny text embedded in complex scenes [32–34, 39]. A prominent line of research, inspired by “thinking with images” [20], addresses this limitation by allocating additional inference-time computation to progressively inspect visual details through multi-turn cropping and zooming operations over regions of the original input image [7, 11, 40, 41].
An alternative line of research seeks to internalize fine-grained visual perception capabilities into the MLLM itself during post-training, so that subtle details can be recognized in a single forward pass [2, 4, 5, 12, 23, 26, 27, 30, 33, 36, 37, 42]. Among them, on-policy distillation (OPD) has recently emerged as a simple and effective post-training paradigm for internalizing visual zooming [12, 23, 26, 36, 37, 42]. A straightforward approach is to distill from a teacher with stronger fine-grained visual perception capabilities [26, 42]. However, such a teacher may not always be available, especially for frontier models. This has motivated growing interest in on-policy self-distillation (OPSD), where a copy of the student itself, with access to zoomed crops of the target region as privileged visual inputs, serves as the teacher during training [12, 23, 36, 37].
A representative example is Vision-OPD [36], which distills privileged visual information from zoomed crops into a full-image student. Despite this visual privilege, its distillation target follows vanilla OPD [1]: the teacher’s token-level predictions conditioned on student-generated prefixes serve directly as supervision targets. However, this vanilla conditioning objective is susceptible to linguistic momentum: erroneous student prefixes can bias the teacher toward

Figure 1. Comparison of existing OPSD methods on four visual abilities across 16 benchmarks using Qwen3.5-4B as the base model. With 12× fewer training images, BlinkOPD outperforms competing methods on four visual abilities across 16 benchmarks.
the student’s own mistakes, thus weakening corrective supervision [23]. This effect is particularly harmful to OPSD for fine-grained multimodal perception, as linguistic momentum from erroneous student prefixes can progressively override privileged visual evidence (see § 2.2).
To address this, contrastive methods reconstruct the distillation target using two predictions: a positive prediction from the privileged teacher conditioned on the zoomed crop, and a reference prediction obtained under altered or reduced visual input [16, 23, 35, 37]. The key idea is to cancel the linguistic drift shared by both predictions, thereby isolating and amplifying privileged visual evidence to strengthen corrective supervision. Existing approaches construct such references using gray images [23], black images [2, 16], Gaussian-noised images [14], mismatched images [30], and text-only inputs [35]. However, our analysis shows that image-based references, including gray, black, noisy, and mismatched images, fail to fully reproduce the prefix-induced linguistic drift, leaving substantial residual drift that limits their ability to counteract linguistic momentum. They also introduce a false belief that the queried object is absent. The text-only reference largely eliminates the residual drift, but the false belief still remains (see § 2.3).
To overcome these limitations, we propose Blink On-Policy Distillation (BlinkOPD), which constructs an eyes-closed reference by letting the privileged teacher “blink” at every token. Specifically, the teacher processes each student prefix twice: once with its eyes open on the zoomed crop and once with its eyes closed, where the image-content tokens are removed while preserving the vision markers, chat template, and original positional indices. This design more effectively cancels prefix-induced linguistic drift and introduces less false beliefs. We further introduce a first-glance gain: before training, the teacher takes a first glance at the zoomed crop without any prefix, and rollouts whose answers conflict with this first glance, which are likely to require stronger correction, receive a larger gain.
To summarize, our contributions are as follows:
(1) We present a comprehensive analysis of existing OPSD distillation targets in fine-grained multimodal perception, showing that linguistic momentum overrides the privileged visual evidence, and that existing contrastive references leave residual drift and introduce false beliefs.
(2) We propose BlinkOPD, which constructs an eyes-closed reference by removing image-content tokens while preserving the original multimodal input structure. Combined with a first-glance gain that selectively strengthens correction on potentially erroneous student trajectories, our method provides more effective corrective supervision.
(3) With only 500 training images, BlinkOPD achieves strong performance across six fine-grained perception benchmarks, outperforming existing OPSD methods trained on about 12 times more images. It also demonstrates strong
zero-shot transfer across ten held-out perception, hallucination, and multimodal reasoning benchmarks.
Preliminaries and Analysis
On-Policy Self-Distillation
In vanilla OPSD for multimodal fine-grained perception [36], the student receives the full image and question , denoted by , while the teacher is initialized as a copy of the student and receives privileged visual information during training. In our setting, the privileged input is a zoomed crop of the target region, giving . The student generates on-policy rollouts , and the teacher scores each student-generated prefix to provide token-level supervision without requiring ground-truth answers. For each prefix, the teacher predicts a next-token distribution over the vocabulary . The student is then trained to match a distillation target distribution , which in vanilla OPSD is the teacher prediction itself, , by minimizing
where denotes a distribution-matching divergence.
Linguistic Momentum in OPSD
We design a controlled experiment to investigate linguistic momentum and privileged evidence survival in OPSD for fine-grained multimodal perception. Specifically, we randomly sample 768 questions from the Vision-OPD [36] training pool and use Qwen3.5-4B to generate four rollouts per question from the full-image input. We truncate each rollout at 0, 25, 50, 75, and 100% of its tokens to observe the teacher at different stages of the student’s rollout. For each truncated prefix, the privileged teacher observes the zoomed crop, and we record its next-token probabilities over the answer options. Using these read-outs, we construct the distillation targets of Vision-OPD [36], VAD [37], CV-OPD [30], OPD-V [2], OPD-Aha [23], and our proposed BlinkOPD to compare their susceptibility to linguistic momentum and privileged evidence survival, as shown in Figure 2. Full experimental details are provided in Appendix § 6.

Figure 2. Comparison of the distillation targets of Vision-OPD, VAD, CV-OPD, OPD-V, OPD-Aha and our proposed BlinkOPD on linguistic momentum and privileged evidence survival. (a) As the teacher reads more of the wrong prefix1, the follow rate tends to rise: Vision-OPD rises from 23% to 100%, VAD, CV-OPD and OPD-V from about 50% to 100%, and OPD-Aha still ends at 64%, whereas BlinkOPD stays below 40% throughout. (b) As the teacher reads more of the wrong prefix, the survival score tends to decrease: Vision-OPD, VAD, CV-OPD and OPD-V turn negative within the first quarter of the prefix and never recover, and OPD-Aha also falls below zero by the end, whereas BlinkOPD remains positive and outperforms the others throughout.
Follow Rate. To measure how often the supervision endorses the student’s own mistake, i.e., the strength of linguistic momentum, we define the follow rate. Let be the set of student rollouts whose final answer is wrong; for , let be the option chosen by the student. For a method and a truncation point , let denote its distillation target after the first of, read out over the set of answer options of the question. The follow rate is
i.e., the fraction of wrong rollouts on which the distillation target picks the student’s own wrong option. Training the student toward a distillation target with a high follow rate reinforces the student’s error. At , the teacher has not yet read the student’s answer, so serves as a baseline: it only counts cases in which the teacher happens to make the same mistake as the student. Any rise of above is caused by reading the student’s prefix, and this rise is what measures linguistic momentum.
Survival Score. To measure how much of the privileged visual evidence survives in the distillation target after the teacher reads the student’s wrong prefix, we define the survival score. Let be the correct option of rollout , and let be the wrong rollouts on which the teacher picks before reading the prefix. On these rollouts, the crop holds the evidence that the student missed, and at the teacher’s preference for over reflects this privileged visual evidence. The survival score is
A positive value means that the distillation target still conveys the privileged evidence, and a negative value means
that the evidence is overridden by the student’s prefix.
Limitations of Existing Contrastive References
Existing contrastive methods [2, 14, 16, 23, 30, 35, 37] counter linguistic momentum by comparing the privileged teacher with a reference that reads the same student prefix. Given a reference distribution , the distillation target sharpens the teacher as
where compares the teacher with the reference token by token and the gain sets how strongly the distillation target is tilted toward tokens with a high .
Ideally, since both distributions read the same student prefix, the part of the prediction induced by the prefix alone is shared and cancels in , while the part that comes from the privileged crop remains. Whether this happens, however, depends entirely on the reference, which must meet two requirements. First, it must be drifted by the student prefix exactly as much as the teacher, so that no residual drift remains in . Second, while holding no privileged visual evidence, it must not add a false belief that the queried object is absent which enters the distillation target through . Existing methods [2, 14, 16, 23, 30, 35] build the reference either by replacing the crop with another image (gray, black, noise or a mismatched image) or by removing the image from the conversation. Using the probe of § 2.2, we test whether these references meet these requirements.
Residual Drift. We define the residual drift of a reference to measure how much of the prefix-induced drift it fails to cancel. Let and be the survival scores of the teacher and of the reference, i.e., Eq. (3) with replaced by their predictions. How far the teacher is drifted by the whole student prefix is then the fall of its survival score, , where the superscripts 0 and end denote the values before reading the prefix and after reading the whole student prefix, i.e., and .2 The reference reads the same prefix and is dragged by . The residual drift is the share of the teacher’s drift that the reference still leaves uncancelled in the contrast :
is lower the better with meaning that the reference is drifted exactly as much as the teacher and are all cancelled.
Proposition 1. in Eq. (5) is exactly the residual drift of the contrast in Eq. (4) (see proof in Appendix § 7).
False Belief. In Figure 3a, the text-only reference leaves the least residual drift. However, it still introduces a false “not present” belief that the teacher does not have. We take 300 multiple-choice questions from MME-RealWorld-CN [39] whose options include “not present / cannot be determined”. For each reference, the teacher reads only the question, without any student prefix, and we record the probability that it assigns to this option at the first answer token (Figure 3b). The same procedure with the original real image of each question gives 0.16. A reference with a higher probability than 0.16 therefore introduces a false belief that the queried object is absent. The false belief is itself a form of hallucination: a reference with a high false belief can introduce hallucination into the student (see the experimental results on hallucination benchmarks in § 4.3).

Figure 3. Comparison of contrastive references on residual drift and false belief. (a) Image-based references leave 20–23% of the drift uncancelled, whereas the text-only and eyes-closed references leave about 10%. (b) All other references raise the probability of “not present” false belief to 0.39–0.63. Only our proposed eyes-closed reference is low on both measures.
Blink On-Policy Distillation
Closing the Teacher’s Eyes
As illustrated in Figure 4c, the eyes-closed reference is the privileged teacher with only image-content tokens removed: the vision markers, the chat template, the question and the prefix are kept, and every remaining token keeps the position it had in the full sequence,

Figure 4. Contrastive reference construction for OPSD. (a) Image-based references replace the crop with gray, black, noisy, or mismatched images [2, 14, 16, 23, 30]. (b) The text-only reference removes the image from the conversation [35]. (c) Our eyes-closed reference removes only the image-content tokens while preserving vision markers and original positional indices.
where pos denotes these original positions; see Appendix § 8 for the implementation details of this input in training.
First-Glance Gain
Even with a faithful reference, the contrastive target still depends on the gain in Eq. (4). The reference decides what is amplified while the gain decides how much. We find that a single global gain does not suit all rollouts. On a student-wrong rollout, the teacher is dragged by the prefix, and a large gain recovers the correct option. On a student-correct rollout, however, a large gain pushes the distillation target away from the correct answer. Our probe confirms this trade-off (see Appendix § 9), so the gain should be raised only on rollouts that are likely to be wrong.
Detecting such rollouts would normally require the correct answer. The frozen teacher, however, offers a label-free signal. Before training, the teacher looks at the crop once (, no prefix) and records its answer , its first glance. A rollout conflicts with the glance when its final answer differs from it, and we raise its gain as:
where is the base gain of Eq. (4) and scales it up on conflicting rollouts. See Appendix § 10 for an experimental comparison of the global gain and the first-glance gain.
Distillation Target and Training
BlinkOPD keeps the OPSD objective of Eq. (1) and replaces only its distillation target . For rollout , it combines the eyes-closed reference of Eq. (6) with the gain of Eq. (7):
Figure 5 provides an overview of BlinkOPD. An algorithm chart for one training step is provided in Appendix § 11. The entire BlinkOPD training pipeline requires neither assistance from larger models nor ground-truth answers.

Figure 5. Overview of BlinkOPD. The student generates rollouts from the full image and question , together denoted by . The frozen privileged teacher reads each student prefix twice: once with its eyes open on the zoomed crop (with ), and once with its eyes closed (, § 3.1). Their log-ratio keeps the privileged visual evidence, while the prefix-induced linguistic drift shared by both cancels. A prefix-free first glance of the same teacher flags rollouts whose answer conflicts with it, and the gain is raised by a factor on these rollouts (§ 3.2). The student is trained toward the resulting distillation target (§ 3.3). At test time, the student runs alone in a single pass.
Experiments
Experimental Setup
Model and Data. We train Qwen3.5-4B and 9B [24] in non-thinking mode. We use the decontaminated3 Vision-OPD training pool [36] of about 6.2K images with evidence boxes. We report results on random samples at different scales from 250 to the full 6.2K pool (Figure 6b). All methods follow the Vision-OPD input protocol [36]: the student receives the full image with the evidence box drawn on it and a one-sentence instruction to focus on that region, while the teacher receives a tight crop of the boxed region.

Figure 6. Corrective supervision and data efficiency with Qwen3.5-4B. (a) As the teacher reads more of the wrong prefix, the corrective density tends to decrease: Vision-OPD drops from 68% to 0%, VAD, CV-OPD and OPD-V from about 40% to 0%, and OPD-Aha declines to 28%, whereas BlinkOPD stays above 40% throughout. (b) Mean accuracy across the six fine-grained benchmarks at different training scales. BlinkOPD leads at every scale and reaches 79.6% with 6.2K images.
Implementation Details. We train for 38 steps with 96 prompts per step and 8 rollouts per prompt (temperature 1.0, at most 1,024 tokens). We use a constant learning rate of after 5 warm-up steps and set and . We use JSD () for the divergence in Eq. (1). A 4B run takes under three hours on four B200 GPUs. The 9B run uses the same data, steps and hyperparameters (see Appendix § 12 for more implementation details).
Baselines. To compare methods under one protocol, we reproduce the recent OPSD methods with complete public code: Vision-OPD [36], OPD-V [2], CV-OPD [30], VAD [37] and OPD-Aha [23], each with its official implementation. All OPSD runs share the same student, the teacher crop, the rollout budget per step and the evaluation pipeline. We also evaluate SFT and GRPO [25] under the same setting. For other methods, Table 1 lists the numbers reported by their original papers in gray, for reference only.
| Method | Size | Teacher | #Img | V* | ZoomBench | HR-4K | HR-8K | MME-RW EN | MME-RW CN | Avg. | Δ |
| Proprietary models (reported by [12, 36]; also evaluated in [23, 30, 37]) | |||||||||||
| GPT-5.2 [19] | – | – | – | 79.1 | 50.9 | 81.1 | 78.4 | 72.6 | 68.8 | 71.8 | |
| GPT-5.4 [21] | – | – | – | 77.0 | 52.7 | 84.0 | 77.9 | 74.2 | 70.9 | 72.8 | |
| Gemini 3.1 Pro [9] | – | – | – | 88.0 | 61.2 | 89.6 | 86.9 | 76.5 | 73.3 | 79.3 | |
| Gemini 3.5 Flash [13] | – | – | – | 89.0 | 61.4 | 89.1 | 86.6 | 75.3 | 74.0 | 79.2 | |
| Large open-weight models (reported by [12, 36]; also evaluated in [23, 30, 37]) | |||||||||||
| GLM-4.6V [8] | 106B | – | – | 86.9 | 50.1 | 82.1 | 78.9 | 65.6 | 65.6 | 71.5 | |
| Qwen3-VL-Instruct [3] | 235B | – | – | 91.1 | 56.1 | 86.1 | 80.4 | 71.7 | 69.0 | 75.8 | |
| Qwen3.5 [24] | 397B | – | – | 88.0 | 57.2 | 89.4 | 85.5 | 74.8 | 69.8 | 77.4 | |
| Kimi-K2.6 [18] | 1T | – | – | 88.5 | 53.1 | 81.9 | 78.0 | 69.2 | 66.1 | 72.8 | |
| Thinking-with-images / inference-time methods (reported by [23, 37]; also evaluated in [12, 30, 36]) | |||||||||||
| DeepEyes [41] | 7B‡ | – | – | 82.6 | 46.2 | 75.5 | 70.0 | 64.0 | 62.4 | 66.8 | |
| Thyme-RL [40] | 7B‡ | – | – | 78.5 | 46.2 | 78.9 | 70.9 | 64.2 | 61.4 | 66.7 | |
| DeepEyesV2 [11] | 7B‡ | – | – | 78.0 | 46.1 | 79.8 | 72.6 | 64.3 | 61.9 | 67.1 | |
| SenseNova-MARS [7] | 8B§ | – | – | 88.4 | 49.0 | 85.0 | 77.3 | 67.3 | 65.7 | 72.1 | |
| Single forward-pass methods (reported by [12, 23, 36]; also evaluated in [30, 37]) | |||||||||||
| V-Zero [26] | 4B | 27B | 6.2K | 78.0 | 56.5 | 84.8 | 80.9 | 73.2 | 71.1 | 74.1 | |
| EAD [12] | 4B | Self | 6.2K | 91.6 | 60.8 | 86.4 | 82.9 | 72.4 | 71.1 | 77.5 | |
| EAD [12] | 9B | Self | 6.2K | 93.2 | 62.0 | 88.6 | 84.6 | 72.5 | 71.1 | 78.7 | |
| ZwZ [33] | 8B§ | – | 74K | 88.0 | 56.7 | 83.6 | 81.8 | 66.6 | 68.1 | 74.1 | |
| Qwen3.5-4B (reproduced by us with the decontaminated training set and the corrected judge protocol) | |||||||||||
| Base model [24] | 4B | – | – | 81.7 | 48.6 | 83.9 | 80.6 | 62.8 | 62.6 | 70.0 | – |
| SFT | 4B | – | 6.2K | 87.0 | 54.7 | 84.8 | 80.0 | 71.2 | 68.2 | 74.3 | +4.3 |
| GRPO [25] | 4B | – | 6.2K | 86.9 | 58.9 | 83.5 | 78.4 | 72.5 | 68.8 | 74.8 | +4.8 |
| Vision-OPD [36] | 4B | Self | 6.2K | 89.5 | 59.3 | 82.5 | 79.4 | 71.3 | 68.6 | 75.1 | +5.1 |
| OPD-V [2] | 4B | Self | 6.2K | 89.0 | 59.9 | 84.4 | 78.5 | 70.6 | 69.4 | 75.3 | +5.3 |
| CV-OPD [30] | 4B | Self | 6.2K | 90.1 | 60.4 | 83.3 | 80.5 | 72.3 | 70.2 | 76.1 | +6.1 |
| VAD [37] | 4B | Self | 6.2K | 94.8 | 61.0 | 85.3 | 81.5 | 73.7 | 69.4 | 77.6 | +7.6 |
| OPD-Aha [23] | 4B | Self | 6.2K | 91.6 | 59.3 | 87.4 | 83.3 | 73.9 | 72.0 | 77.9 | +7.9 |
| BlinkOPD (ours) | 4B | Self | 0.5K | 92.7 | 61.1 | 88.5 | 85.6 | 74.8 | 71.8 | 79.1 | +9.1 |
| BlinkOPD (ours) | 4B | Self | 6.2K | 93.1 | 61.9 | 89.1 | 86.2 | 75.1 | 72.4 | 79.6 | +9.6 |
| Qwen3.5-9B (reproduced by us with the decontaminated training set and the corrected judge protocol) | |||||||||||
| Base model [24] | 9B | – | – | 85.9 | 52.8 | 85.9 | 80.1 | 70.3 | 67.2 | 73.5 | – |
| GRPO [25] | 9B | – | 6.2K | 89.5 | 58.5 | 84.8 | 80.6 | 73.5 | 69.6 | 76.1 | +2.6 |
| SFT | 9B | – | 6.2K | 90.1 | 58.8 | 84.9 | 81.6 | 73.3 | 68.5 | 76.2 | +2.7 |
| OPD-V [2] | 9B | Self | 6.2K | 88.5 | 61.5 | 85.5 | 83.3 | 71.0 | 69.1 | 76.5 | +3.0 |
| Vision-OPD [36] | 9B | Self | 6.2K | 86.9 | 64.3 | 85.5 | 84.0 | 71.1 | 69.2 | 76.8 | +3.3 |
| CV-OPD [30] | 9B | Self | 6.2K | 88.0 | 62.5 | 87.1 | 83.8 | 70.1 | 69.8 | 76.9 | +3.4 |
| VAD [37] | 9B | Self | 6.2K | 86.9 | 63.3 | 86.5 | 85.4 | 72.8 | 70.3 | 77.5 | +4.0 |
| OPD-Aha [23] | 9B | Self | 6.2K | 89.5 | 61.8 | 89.0 | 84.4 | 74.3 | 71.2 | 78.4 | +4.9 |
| BlinkOPD (ours) | 9B | Self | 0.5K | 94.2 | 61.2 | 90.4 | 86.9 | 74.4 | 70.4 | 79.6 | +6.1 |
| BlinkOPD (ours) | 9B | Self | 6.2K | 94.7 | 63.6 | 90.7 | 87.2 | 75.3 | 70.6 | 80.4 | +6.9 |
Table 1. Fine-grained perception. Accuracy (%) on six benchmarks. Gray: results reported by the original papers or reproduced by others; listed for reference only. Agentic models and ZwZ are built on older backbones (‡Qwen2.5-VL-7B, §Qwen3-VL-8B). Bold and underline mark the best and second best within each re-evaluated block. Teacher: Self is a privileged self-teacher (a copy of the student on the zoomed crop); V-Zero instead distills from a Qwen3.5-27B teacher. #Img: unique training images. Δ: gain over the base model.
Evaluation. Following the evaluation protocol of Vision-OPD [36], we report accuracy on V* [34], ZoomBench [33], HR-Bench 4K/8K [32] and MME-RealWorld EN/CN [39]. Decoding is carried out with up to 32K tokens. Answers are extracted by rules and a corrected4 gpt-oss-120B judge.
Main Results for Fine-Grained Perception
As shown in Table 1, BlinkOPD reaches 79.1 average accuracy on Qwen3.5-4B with only 0.5K images, outperforming the base model (+9.1) and all reproduced OPSD baselines trained on 12× more images: Vision-OPD (+4.0), OPD-V (+3.8), CV-OPD (+3.0), VAD (+1.5) and OPD-Aha (+1.2). Training on the full 6.2K set further improves the average to 79.6. The same trend holds for 9B: with only 0.5K images, BlinkOPD reaches 79.6, outperforming the base model (+6.1) and all reproduced OPSD baselines trained on 12× more images: OPD-V (+3.1), Vision-OPD (+2.8), CV-OPD (+2.7), VAD (+2.1) and OPD-Aha (+1.2). Training on the full 6.2K set further improves the average to 80.4.
Comparison with larger open-weight and proprietary models. BlinkOPD also outperform larger open-weight models with up to 250× more parameters, including Qwen3.5-397B (77.4), the strongest one in Table 1. With only 4B parameters and 0.5K training images, BlinkOPD already matches the leading proprietary models, including Gemini 3.1 Pro (79.3), the best one in Table 1. Scaling to 9B parameters and the full 6.2K, BlinkOPD further surpasses all reported and reproduced baselines. These gains are achieved despite training on a decontaminated dataset and evaluating with a corrected judge, both of which can only lower our scores relative to the setup from prior works.
Zero-Shot Transfer to Other Visual Abilities
Post-training for fine-grained perception should not come at the cost of other visual abilities. We therefore evaluate on three important visual abilities without any further training across ten held-out benchmarks (Table 2): general perception (MMStar [6], CV-Bench [28], MMVP [29]), hallucination (POPE [15], HallusionBench [10]) and multimodal reasoning (MathVista [17], MathVerse [38], MATH-Vision [31], We-Math [22], DynaMath [43]). All models are evaluated with the official prompt of each benchmark, greedy decoding with thinking disabled and the same scorer (see Appendix § 14 for the detailed experimental settings).
| Method | General Perception | Hallucination | Multimodal Reasoning | Average | |||||||||||
| MMStar | CV-Bench | MMVP | POPE | HallB. | MathVista | MathVerse | MATH-V | We-Math | DynaMath | Perc. | Hall. | Reas. | All | ||
| Qwen3.5-4B | |||||||||||||||
| Base model [24] | 68.3 | 87.0 | 58.7 | 89.5 | 76.1 | 82.0 | 74.2 | 63.9 | 58.8 | 57.0 | 71.3 | 82.8 | 67.2 | 71.0 | – |
| Vision-OPD [36] | 71.5 | 85.6 | 59.3 | 89.4 | 71.7 | 81.2 | 71.8 | 60.9 | 63.3 | 63.9 | 72.1 | 80.6 | 68.2 | 73.1 | +2.1 |
| OPD-V [2] | 71.4 | 85.8 | 58.7 | 89.5 | 73.3 | 80.5 | 72.3 | 61.6 | 64.9 | 61.1 | 72.0 | 81.4 | 68.1 | 73.2 | +2.2 |
| CV-OPD [30] | 71.5 | 86.6 | 56.7 | 89.3 | 71.8 | 81.0 | 72.4 | 63.5 | 67.1 | 57.8 | 71.6 | 80.6 | 68.4 | 73.4 | +2.4 |
| VAD [37] | 73.5 | 88.2 | 58.0 | 89.2 | 72.7 | 83.1 | 73.4 | 64.5 | 69.9 | 65.5 | 73.2 | 81.0 | 71.3 | 75.2 | +4.2 |
| OPD-Aha [23] | 75.3 | 87.2 | 61.3 | 89.0 | 74.5 | 82.6 | 77.1 | 69.6 | 75.5 | 65.6 | 74.6 | 81.8 | 74.1 | 76.6 | +5.6 |
| BlinkOPD (6.2K) | 74.3 | 87.0 | 60.5 | 89.6 | 77.4 | 84.1 | 79.3 | 71.2 | 76.9 | 66.3 | 73.9 | 83.5 | 75.6 | 77.8 | +6.8 |
| BlinkOPD (0.5K) | 75.6 | 87.8 | 70.0 | 89.5 | 77.2 | 83.9 | 78.8 | 70.8 | 76.3 | 65.8 | 77.8 | 83.4 | 75.1 | 78.1 | +7.1 |
| Qwen3.5-9B | |||||||||||||||
| Base model [24] | 67.9 | 87.0 | 64.0 | 89.8 | 73.3 | 83.9 | 74.3 | 68.1 | 67.1 | 74.6 | 73.0 | 81.6 | 73.6 | 74.5 | – |
| Vision-OPD [36] | 74.7 | 82.8 | 66.0 | 89.9 | 73.6 | 82.8 | 74.1 | 68.4 | 74.1 | 76.2 | 74.5 | 81.8 | 75.1 | 76.5 | +2.0 |
| OPD-V [2] | 75.3 | 85.4 | 65.3 | 89.7 | 75.4 | 82.4 | 73.8 | 68.6 | 72.7 | 76.8 | 75.3 | 82.6 | 74.9 | 76.5 | +2.0 |
| CV-OPD [30] | 75.7 | 85.5 | 66.7 | 89.8 | 74.0 | 82.2 | 76.6 | 68.5 | 72.2 | 76.7 | 76.0 | 81.9 | 75.2 | 76.8 | +2.3 |
| VAD [37] | 77.0 | 84.7 | 68.7 | 90.2 | 76.4 | 83.6 | 75.0 | 69.2 | 73.1 | 77.0 | 76.8 | 83.3 | 75.6 | 77.5 | +3.0 |
| OPD-Aha [23] | 77.5 | 86.3 | 71.3 | 90.2 | 74.4 | 85.1 | 77.4 | 72.7 | 76.8 | 78.1 | 78.4 | 82.3 | 78.0 | 78.8 | +4.3 |
| BlinkOPD (0.5K) | 78.6 | 88.4 | 72.7 | 90.1 | 75.8 | 85.1 | 75.0 | 73.2 | 77.5 | 78.0 | 79.9 | 83.0 | 77.8 | 79.5 | +5.0 |
| BlinkOPD (6.2K) | 76.7 | 85.5 | 70.2 | 90.4 | 76.5 | 85.8 | 77.8 | 73.6 | 78.4 | 78.2 | 77.5 | 83.5 | 78.8 | 79.7 | +5.2 |
Table 2. Zero-shot transfer to ten held-out benchmarks. Perc., Hall. and Reas.: average accuracy on general perception, hallucination mitigation and multimodal reasoning. All: average over all 16 benchmarks, including the six fine-grained perception benchmarks of Table 1. : change of All relative to the base model. Bold and <u>underline</u>: best and second best within each model size.
General perception. The BlinkOPD (0.5K) improves general perception the most at both sizes: reaching average scores of 77.8 (+6.5 over the base model) at 4B and 79.9 (+6.9) at 9B, ahead of the runner-up OPD-Aha in both (+3.2 and +1.5). Fine-grained post-training corrects the myopia of the base model, but it can overcorrect the model into the opposite extreme, hyperopia. This is most visible on CV-Bench at 9B, where every baseline falls below the base model (82.8–86.3 vs. 87.0), including the best method in fine-grained benchmarks: BlinkOPD (6.2K). We attribute this to their 6.2K training images, which overfit the model to fine details. In contrast, BlinkOPD (0.5K) is the only method that improves CV-Bench at both sizes (+0.8 at 4B and +1.4 at 9B), while also appearing strong on fine-grained perception in Table 1. This suggests that the key for OPSD to strike a balance between myopia and hyperopia is to raise the efficiency of corrective supervision, rather than to simply train the model on ever more fine-grained images.
Hallucination mitigation. At 4B, hallucination is the only ability for which all baselines underperform the base model (80.6–81.8 vs. 82.8 in the Hall. column), while BlinkOPD (0.5K&6.2K) is the only approach that achieves an improvement (83.4 and 83.5). This can be attributed to the two flaws in existing distillation targets identified in § 2, both of which are themselves forms of hallucination. Linguistic momentum (§ 2.2) causes the teacher to follow the student’s prior words rather than the visual evidence, while image references in contrastive targets introduce a false belief (§ 2.3). Although these flaws originate in the teacher’s targets, they can propagate to the student through distillation. BlinkOPD removes both at the teacher level, breaking linguistic momentum without adding any false belief, and the student inherits this behavior and hallucinates less.
Multimodal Reasoning. BlinkOPD (6.2K) achieves the best performance across all five reasoning benchmarks at both model sizes, improving the average from 67.2 to 75.6 (+8.4) at 4B and from 73.6 to 78.8 (+5.2) at 9B over the base model. We attribute these gains to its ability to break linguistic momentum. Multimodal reasoning produces long chains of thought. As shown in Figure 2, the longer the student’s prefix, the more closely the teacher follows it rather than the image.
Among existing OPSD methods, BlinkOPD is the most effective at breaking this linguistic momentum. By “blinking” at every token, its target keeps correcting the student even deep into a long chain, and the student learns to keep checking its reasoning against the image.
BlinkOPD achieves the strongest over four visual abilities. Averaged over all 16 benchmarks, the two settings of BlinkOPD rank first and second at both sizes. The 6.2K setting is best on fine-grained perception, hallucination mitigation and multimodal reasoning at both sizes, while the 0.5K setting is best on general perception. Linguistic momentum, the retention of privileged visual evidence and the efficiency of corrective supervision are general problems in OPSD: once they are addressed, the gains generalize to a broad range of visual understanding abilities.
Corrective Supervision and Data Efficiency
To show that BlinkOPD provides the densest corrective supervision, we define the corrective density of a distillation target as the share of student-wrong rollouts of the probe in § 2.2 on which it picks the correct option after a fraction of the student’s own prefix has been read,
with the notation of Eqs. (2) and (3), where is the correct option of rollout and is the set of its answer options.
Why a few images can suffice. The higher corrective density of BlinkOPD (Figure 6a) makes every rollout more useful for training: on a wrong rollout, more of the distillation tokens push the student toward the correct option instead of toward its own or another wrong answer. BlinkOPD also keeps the privileged visual evidence alive and resists linguistic momentum on long prefixes (Figure 2), so its corrective signal does not fade too much as the student writes more. Although BlinkOPD (0.5K) uses fewer images than the baselines, each image and rollout yields far more useful supervision, so it can outperform the OPSD baselines.
Data efficiency across training scales. Figure 6b compares all six methods trained on 250, 500, 1K, 3K and 6.2K images. BlinkOPD leads at every scale. With only 250 images, it reaches 75.5, 1.2 points above the strongest baseline at the same scale (74.3). With 500 images, it reaches 79.1 and surpasses every baseline trained on the full 6.2K pool by at least 1.2 points, using fewer images. Scaling to 6.2K further improves it to 79.6, 1.7 points above the strongest baseline at the same scale. This shows the benefit of breaking linguistic momentum: by providing denser corrective supervision, BlinkOPD makes better use of the training data at every scale, especially when the data are scarce, since more of its tokens correct the student.
Ablation and Sensitivity Analysis
We run all ablations with BlinkOPD (0.5K) on Qwen3.5-4B. In Table 3, every component of BlinkOPD contributes to its accuracy. Removing the first-glance gain costs 0.6 points, and removing the reference altogether, which recovers plain OPSD, costs 4.0, the largest drop in the table. Re-
| Variant with Qwen3.5-4B as the base model | Avg. | |
| BlinkOPD (0.5K) | 79.1 | – |
| Ablation | ||
| w/o first-glance gain (same for all rollouts) | 78.5 | −0.6 |
| → gray-image reference | 77.4 | −1.7 |
| → black-image reference | 75.4 | −3.7 |
| → Gaussian-noise-image reference | 75.6 | −3.5 |
| → mismatched-image reference | 76.4 | −2.7 |
| → text-only reference | 78.2 | −0.9 |
| w/o reference (plain OPSD, ) | 75.1 | −4.0 |
| Sensitivity | ||
| JSD → forward KL | 77.6 | −1.5 |
| JSD → reverse KL | 76.7 | −2.4 |
| frozen teacher → EMA teacher | 76.1 | −3.0 |
Table 3. Ablation&sensitivity analysis of BlinkOPD (0.5K) on 4B.
placing the eyes-closed reference reproduces the ranking of Figure 3. The text-only reference cancels the drift about as well as ours but injects a stronger false belief, and it comes closest (−0.9). The image-based references leave about 20% of the drift uncancelled and also inject false beliefs, and they fall further behind (−1.7 to −3.7). Only the eyes-closed reference is low on both measures, and it performs best. For the OPSD pipeline, JSD and a frozen teacher work best: replacing JSD by forward or reverse KL costs 1.5 and 2.4, and an EMA teacher costs 3.0, which suggests that a frozen privileged teacher provides the most stable target.
Conclusion
We found that OPSD for fine-grained multimodal perception suffers from linguistic momentum and the privileged visual evidence decays along the response. We further showed that existing contrastive references leave part of this drift uncancelled and inject false beliefs of their own. We therefore proposed BlinkOPD, which lets the teacher “blink” at every token of every rollout, and a first-glance gain strengthens the correction. This simple change to the distillation target keeps the privileged evidence alive along the whole response and raises the corrective density of every rollout. With only 0.5K images, BlinkOPD outperforms all reproduced OPSD baselines trained on more images at both 4B and 9B; its 4B student already matches the strongest proprietary models, and its 9B student surpasses them. Although BlinkOPD is trained only on fine-grained perception data, we show that once these problems of OPSD are resolved in its distillation target, the student learns general habits that transfer well to broader visual abilities, such as general perception, hallucination mitigation and multimodal reasoning, without training on any of these tasks. Related work and Limitations are discussed in Appendix § 15 & M. Qualitative examples are in Appendix § 17.
Experimental details of the analysis
Probe set. We randomly sample 768 questions from the decontaminated Vision-OPD training pool [36] of 6,220 questions (see Footnote 3 of the main paper). For each question, generates 4 student rollouts from the full image (temperature 1, at most 1,024 tokens), giving 3,072 rollouts, of which 1,732 are wrong and 1,340 are correct. Each rollout is truncated at 0, 25, 50, 75 and 100% of its tokens. After each truncated prefix we append the answer cue **Answer: ( and read the next-token probabilities over the option letters, normalized over the options, for the teacher (zoomed crop), for every reference and for every distillation target. From these read-outs, we construct the distillation targets of Vision-OPD [36], VAD [37], CV-OPD [30], OPD-V [2] and OPD-Aha [23] following the default setting of each original paper. Note that this analysis uses the ground-truth answers to separate wrong from correct rollouts and to compute the follow rate and the survival score, whereas OPSD methods themselves require no ground-truth answers.
Linguistic momentum (Figure 2). Panel (a) reports the follow rate of Eq. (2), where contains the 1,732 student-wrong rollouts of the probe set. Table 4 lists the values corresponding to Figure 2a.
| Method | 25% | 50% | 75% | 100% | |
| Vision-OPD [36] | 23.2 | 66.7 | 83.5 | 95.8 | 99.8 |
| VAD [37] | 51.4 | 75.2 | 87.5 | 97.1 | 100.0 |
| CV-OPD [30] | 48.9 | 75.6 | 87.2 | 97.1 | 99.9 |
| OPD-V [2] | 50.3 | 78.2 | 89.2 | 97.6 | 100.0 |
| OPD-Aha [23] | 19.4 | 46.9 | 49.2 | 43.8 | 64.3 |
| BlinkOPD (ours) | 19.6 | 34.7 | 37.4 | 19.1 | 38.4 |
Table 4. Follow rate (%) on the 1,732 student-wrong rollouts, corresponding to Figure 2a.
Panel (b) reports the survival score of Eq. (3), where contains the 1,181 student-wrong rollouts on which the teacher picks the correct option before reading the prefix, i.e., the rollouts on which the zoomed crop holds the evidence that the student missed. Table 5 lists the values corresponding to Figure 2b.
| Method | 25% | 50% | 75% | 100% | |
| Vision-OPD [36] | 1.60 | −1.36 | −2.16 | −2.77 | −3.95 |
| VAD [37] | 0.15 | −2.25 | −2.73 | −3.21 | −4.07 |
| CV-OPD [30] | 0.35 | −2.17 | −2.78 | −3.30 | −4.18 |
| OPD-V [2] | 0.05 | −2.34 | −2.97 | −3.42 | −4.18 |
| OPD-Aha [23] | 8.92 | 3.14 | 1.51 | 1.43 | −1.03 |
| BlinkOPD (ours) | 13.35 | 7.21 | 4.57 | 7.50 | 3.67 |
Table 5. Survival score on the 1,181 rollouts of , corresponding to Figure 2b.
Residual drift (Figure 3a). The gray, black and Gaussian-noise references replace the zoomed crop with an image of the same size filled with the mean color of the crop, with zeros, or with Gaussian noise. The mismatched-image reference replaces it with the zoomed crop of another random probe question whose source image is different, resized to the same size, so that it contains real but unrelated visual content. The residual drift (Eq. (5)) is computed on the 1,181 rollouts of after the whole prefix.
Control: the content of the prefix drives linguistic momentum
To check that linguistic momentum comes from the content of the prefix rather than its length, we repeat the probe of Figure 2a on the same 1,732 student-wrong rollouts with the same teacher, but replace the student’s prefix with a rollout of another question, taken from a different source image and truncated to the same number of tokens. With its own prefix, the follow rate of the raw teacher (Vision-OPD) rises from 23.2% to 99.8%; with the swapped prefix, it stays between 22.9% and 25.5% (Figure 7), i.e., at the level before reading any prefix.

Figure 7. Follow rate of the raw teacher (Vision-OPD) on the 1,732 student-wrong rollouts when it reads the student’s own prefix or a prefix of another question with the same length. The thin gray line marks the follow rate before reading any prefix (23.2%).
Residual drift when only part of the prefix is read
In Figure 3a, the residual drift (Eq. (5)) is measured between and the end of the whole prefix. Figure 8 shows when this end point is moved to and , i.e., with replaced by for both the teacher and the reference.

Figure 8. Residual drift (%, Eq. (5)) when the teacher and the reference read (a) 50%, (b) 75% or (c) all of the student prefix (Figure 3a).
Proof of Proposition 1
Here we show that the residual drift in Eq. (5), although defined through the survival scores of the teacher and of the reference, directly measures how much of the prefix-induced drift remains in the contrast term of Eq. (4).
Notation. Fix a rollout and a truncation point . After reading the first of , the teacher and the reference each assign a probability to every answer option ; we write them and . These are and of Eq. (4) at the token position where the answer is read out, restricted to the answer options. At this position, the contrast term of Eq. (4) is therefore a score for each option,
which is high for options that the teacher prefers more than the reference does. ( in Eq. (4) carries no because it is defined at every token position ; here is the read-out position after the first of the prefix.) As in Eqs. (2) and (3), is the correct option of and is the wrong option chosen by the student.
Proof of Proposition 1. Step 1. The survival scores of the teacher and of the reference are Eq. (3) with replaced by and . Subtracting them and regrouping the four log-probabilities by option gives, with Eq. (10),
measures how strongly, on average over , the contrast term prefers the correct option to the student’s option after the first of the prefix.
Step 2. Regrouping the four terms of by end point and applying Eq. (11) at and at the end of the prefix,
is how much reading the student prefix lowers the preference of for the correct option, i.e., the drift that remains in .
Step 3. Dividing by and using Eq. (5),
Hence is the drift that remains in , as a share of the teacher’s own drift: means that the student prefix does not move at all, and any is drift that still carries and that the gain in Eq. (4) amplifies.
Implementing the eyes-closed reference in training
For efficiency, our trainer (verl [49]) does not pad sequences of different lengths to a common length; it packs several sequences end to end into one long row and marks where each sequence starts. This appendix describes how we build the eyes-closed input in this packed format, and two details that matter for Qwen3.5.
Building the eyes-closed input. For every sample, the trainer stores the token ids, the attention mask, a four-row position tensor (one text row used for packing and causal masking, and the three M-RoPE rows , , [51]), and the index where the response begins. To build the eyes-closed input, we
drop every visual-content token (the image-pad tokens) but keep the vision start/end markers;
keep the three M-RoPE rows of all remaining tokens at their full-input values;
renumber the text row continuously, since it only marks where each sequence starts and which tokens may attend to which;
shift the response start by the number of removed tokens; and
pass no pixel values, so the vision encoder is skipped.
On average, 414 tokens are removed per training row.
Deleting rather than masking. We delete the image-content tokens instead of masking them in attention. In decoders that also mix tokens through recurrent states or convolutions, such as the gated delta-rule layers [52] of Qwen3.5, masking attention alone would leave these paths from the image open; deleting the tokens removes every path from the image to the remaining tokens. Keeping the original positions leaves the relative positions among the remaining tokens, and hence all position-dependent attention logits among them, unchanged.
Resetting the linear-attention state. The gated delta-rule layers carry a recurrent state from token to token, so in a packed row the state of one sequence would leak into the next; it must be reset at each sequence boundary. Without the reset, the KL between the trainer’s and the rollout engine’s token distributions has a median of 0.12, whereas with the reset it drops to 0.005. We apply the reset to every forward pass of student, teacher and reference.
The trade-off of a global gain
§ 3.2 states that a single global gain does not suit all rollouts: a large gain recovers the correct option on student-wrong rollouts but pushes the distillation target away from it on student-correct ones. Here we formulate this trade-off and then run an experiment on the probe of § 2.2 to show that it exists.
Formulation. A global gain gives the same gain to every rollout, i.e., it is the gain of Eq. (7) with . On each rollout of the probe, the teacher and the eyes-closed reference read a fraction of the prefix, and we form the distillation target of Eq. (8) with this gain. On a student-wrong rollout, the target should recover the correct option ; we measure this on the student-wrong rollouts by the corrective density of Eq. (9). On a student-correct rollout, the target should keep ; we measure this on the student-correct rollouts by the retention
i.e., the share of student-correct rollouts on which the target still picks the correct option. Let and denote the two measures under the global gain , averaged over the read-out points ; a good gain needs both a high and a high . The two are in trade-off when increases and decreases as grows: then no single maximizes both, and every gain that corrects more student-wrong rollouts keeps fewer student-correct ones.
Setup. The probe contains 768 training questions with four rollouts each, i.e., 1,732 student-wrong rollouts and 1,340 student-correct rollouts (Appendix § 6). We sweep the global gain from 0, where the target is the teacher alone, to 12 in steps of 1, and for each average and over .
The trade-off exists. As grows, the corrective density rises and the retention falls at every gain we test (Figure 9). No global gain therefore gives both a high corrective density and a high retention: a gain that is large enough to correct the student-wrong rollouts damages the student-correct ones. The gain should thus be raised only on rollouts that are likely to be wrong, which is what the first-glance gain of § 3.2 does (Appendix § 10).

Figure 9. Trade-off of a global gain on the probe of § 2.2. Each point is the share of rollouts on which the target picks the correct option, averaged over the read-out points , 50, 75 and 100%. Raising the global gain raises the corrective density on student-wrong rollouts but lowers the retention on student-correct ones.
Global gain versus first-glance gain
Experimental setup. We compare the global gain with the first-glance gain on the probe of Appendix § 9, using the corrective density and retention defined there. We fix , the factor used in training, and sweep the base gain from 0.1 to 12 in steps of 0.1, comparing with the global gain of the same . As in training, the first glance is the teacher’s top option on the crop before it reads any of the prefix (). The glance needs no label: it only compares the student’s answer with the teacher’s own answer on the crop, which the frozen teacher computes once per question before training.
The first-glance gain dominates the global gain. Figure 10 compares the two gains over the corrective-density–retention trade-off of Appendix § 9. At the same retention, the first-glance gain corrects more student-wrong rollouts than the global gain; at the same corrective density, it keeps more student-correct rollouts. In summary, over the whole range covered by both curves, the curve of the first-glance gain lies above and to the right of that of the global gain, i.e., the first-glance gain dominates the global gain on both measures. This agrees with the ablation in Table 3, where using a global gain instead lowers the average accuracy on the six fine-grained benchmarks by 0.6 points.

Figure 10. First-glance gain () against the global gain on the probe of § 2.2: against , each averaged over the four read-out points, as is swept from 0 to 12 for the global gain and from 0 to 6 for the first-glance gain, so that both reach a largest gain of 12. The first-glance gain lies above the global gain.
Training algorithm of BlinkOPD
Algorithm 1 summarizes one training step of BlinkOPD, combining the eyes-closed reference (§ 3.1), the first-glance gain (§ 3.2) and the distillation target (§ 3.3). The corresponding framework is shown in Figure 5 of the main paper.
Algorithm 1 One training step of BlinkOPD
Require: student ; teacher (a copy of the student) ; base gain ; factor ; first glances precomputed with on
for each prompt in the batch do
sample rollouts
for each rollout do
▷ Eq. (7)
for all ▷ eyes open
▷ eyes closed
end for
end for
update by minimizing Eq. (1) with
Training details
| Item | Value |
| Training pool | Vision-OPD, decontaminated (6.2K) |
| Unique images | 500 (random) |
| Student input | full image + box + focus instruction |
| Teacher input | tight crop around the box |
| Teacher | frozen |
| Batch / rollouts per prompt | 96 prompts / 8 |
| Rollout temperature, top- | 1.0, 1.0 |
| Max prompt / response length | 8,192 / 1,024 tokens |
| Optimizer | AdamW, lr , 5 warm-up steps, constant |
| Steps | 38 |
| Divergence | JSD, , student top-100 + tail |
| Gain | , |
| Seed | 42 |
| Hardware | 4–8 NVIDIA B200 |
Table 6. Hyperparameters of BlinkOPD (Qwen3.5-4B; 9B identical).
The corrected answer judge
The answer judge shared by prior work has an issue: it marks a response as correct when the first option-like letter in it matches the ground truth, even if the final answer is a different option. For example, for a response ending in “. . . so the sign is blue. Answer: (C)” with ground truth A, the judge takes the “A” of the word “Answer” as the chosen option and counts the response as correct. Almost all such false positives have ground truth A and occur on MME-RealWorld, and they inflate average scores by 0.3–2.1 points. EAD [12] also identified this issue and switched to a Qwen3.5-122B judge. For a fair comparison with Vision-OPD and the baselines reproduced from it, we keep their judge, gpt-oss-120B, and only fix this rule: the corrected judge uses only the parsed final answer. We re-score all models with it, including the reproduced baselines, and report the corrected numbers. We do not compare with EAD, since its code had not been released when we submitted this paper, so it could neither be reproduced under our protocol nor re-scored with the corrected judge.
Zero-shot transfer protocol
Data and prompts. All ten benchmarks are taken from the official releases at pinned revisions: MMStar (1,500), CV-Bench (2,638), MMVP (300), POPE (COCO random/popular/adversarial), HallusionBench, MathVista testmini (1,000), MathVerse testmini (3,940, five versions), MATH-Vision test (3,040), We-Math testmini (1,740) and DynaMath (5,010 = 501 seeds 10 variants). Each question uses the verbatim prompt of the benchmark’s official evaluation code or paper. We keep the official MMVP ground truth, which contains two swapped pairs that later versions correct. Decoding is greedy with thinking disabled, as in the fine-grained evaluation. CV-Bench is reported as .
Scoring. CV-Bench, MMStar and MMVP: we strip markdown and extract the option from the answer marker (“Answer:”, “the answer is”, , or a leading option); if no option can be extracted, GPT-OSS-120B matches the response to an option with the official VLMEvalKit prompt, and unmatched responses count as wrong. MathVista and MathVerse: official rules, with the judge used only to extract answers that the rules miss (MathVista) or with the official two-step extract-and-compare prompt for free-form answers (MathVerse). MATH-Vision, We-Math and DynaMath: official rule-based scorers. POPE and HallusionBench: rule-based yes/no extraction. The judge runs at temperature 0 with a shared cache, so every model is scored by the same verdicts. It is called for about 38% of MathVerse responses, as intended by its official protocol, and for 0–12% elsewhere; re-scoring the same responses changes averages by at most about 0.15.
Reported metrics. We report the strict score on We-Math, the average accuracy over all ten variants on DynaMath, and question accuracy on HallusionBench; MATH-Vision is evaluated on its full test set.
Related work
From thinking with images to a single-pass recipe for fine-grained multimodal perception. Inspired by “thinking with images” [20], a growing body of work improves fine-grained perception by progressively inspecting visual details through cropping and zooming at inference time. These approaches range from training-free visual search [48, 53] to reinforcement learning-based methods that learn to invoke visual tools [47, 50] [7, 11, 40, 41]. However, these methods often incur substantial computational overhead and latency during inference, limiting their applicability in real-time applications. A complementary line therefore seeks to amortize these capabilities into the model itself during post-training, enabling fine-grained visual details to be recognized in a single forward pass. Representative approaches include enhancing visual representations [4, 5], region-to-image distillation [33], and OPD [12, 23, 26, 27, 36, 37, 42]. We follow this single-pass line: BlinkOPD changes only the training target and leaves inference untouched.
OPSD for fine-grained multimodal perception. OPD can improve fine-grained perception by distilling knowledge from a stronger teacher [26]. However, stronger teachers are not always available, particularly for frontier models. OPSD instead leverages privileged visual information to enable self-distillation. Representative methods equip the teacher with zoomed crops [36], higher-resolution images [42], or recoverable visual cues [27]. Following the line of OPSD methods that use zoomed crops as privileged visual inputs, BlinkOPD focuses on reconstructing distillation targets to counteract linguistic momentum and strengthen corrective supervision.
Linguistic momentum and contrastive targets in multimodal OPSD. In OPD, conditioning the teacher on student-generated prefixes can bias its predictions toward the student’s own mistakes, weakening corrective supervision [44, 45, 46]. In multimodal OPSD, OPD-Aha [23] further demonstrates that such linguistic momentum can override privileged visual evidence. To mitigate this issue, contrastive methods reconstruct the distillation target by contrasting the privileged teacher’s prediction with a reference prediction conditioned on the same student prefix but with altered or removed visual input. Existing references include gray images [23], black images [2, 16], Gaussian-noised images [14], mismatched images [30], and text-only inputs [35]. However, these references can leave residual linguistic drift and introduce false visual beliefs. In contrast, BlinkOPD removes visual-content tokens while preserving the original multimodal input structure, enabling more effective cancellation of linguistic momentum and stronger corrective supervision.
Table 7 compares BlinkOPD with the five OPSD methods that we reproduce in § 4.1. All of them distill a privileged teacher on the zoomed crop into a student on the full image, and all of them run the student alone in a single pass at test time.
| Method | Teacher | Reference | Role of the reference | Gain | Granularity | Labels |
| Vision-OPD [36] | EMA | none | – | – | – | no |
| OPD-V [2] | EMA | crop with a random black patch | on/off weight on each token’s loss | – | per token | no |
| CV-OPD [30] | EMA | another region of the same image | hinge term that keeps the student closer to the teacher | fixed weight | per token | yes* |
| VAD [37] | EMA | pixelated crop | projects the teacher–student shift onto the evidence direction | capped shift | per token | no |
| OPD-Aha [23] | frozen | gray (mean-color) image | contrast inside the distillation target | fixed | global | no |
| BlinkOPD (ours) | frozen | eyes closed: image-content tokens removed, frame kept | contrast inside the distillation target | raised on first-glance conflicts | per rollout | no |
Table 7. Comparison with the reproduced OPSD baselines. Granularity: the level at which the strength of the reference varies. Labels: whether ground-truth answers are used during training (*CV-OPD skips the questions on which no rollout is correct).
Limitations
BlinkOPD is general and can be applied to any multimodal model family. However, owing to limited compute, our experiments cover only the Qwen3.5 family at 4B and 9B, the model family and sizes on which the OPSD methods we compare with are mainly evaluated. We leave the validation on other model families to future work.
Although the eyes-closed reference is the only reference that is low on both residual drift and false belief (Figure 3), part of the drift is still left uncancelled, and the probability of the false belief remains slightly above that obtained with the real image. Likewise, once the teacher reads the student prefix, BlinkOPD has the lowest follow rate and the highest corrective density of all distillation targets (Figure 2 and Figure 6), yet its target still follows the wrong prefix on some rollouts, and its corrective density stays below the level reached before any prefix is read. Linguistic momentum is therefore reduced but not fully removed, and we leave references and targets that close this remaining gap to future work.
Qualitative examples
We show qualitative examples for the four abilities evaluated in Table 1 and Table 2: fine-grained perception (Figure 11), general perception (Figure 12), hallucination (Figure 13) and multimodal reasoning (Figure 14).

Figure 11. (a) ZoomBench. What time is displayed on the clock face on the right side of the tower? A. 12:45 B. 1:20 C. 12:58 D. 1:04 Ground truth: D. 1:04 Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) Answer | B ✗ | B ✗ | B ✗ | B ✗ | B ✗ | C ✗ | D ✓ (b) HR-Bench 8K. Tell me the shape of the signboard attached to the building? A. Oval B. Rectangular C. Triangular D. Circular Ground truth: B. Rectangular Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) Answer | C ✗ | A ✗ | D ✗ | C ✗ | D ✗ | D ✗ | B ✓ (c) HR-Bench 8K. What is the direction indicated on the sign pointing towards Alocén? A. Straight B. Left C. Right D. Downward Ground truth: B. Left Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) Answer | A ✗ | C ✗ | C ✗ | C ✗ | C ✗ | C ✗ | B ✓ (d) HR-Bench 4K. What direction arrows are shown on the black sign? A. Two right arrows B. No arrows C. Two left arrows D. Both left and right arrows Ground truth: C. Two left arrows Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) Answer | D ✗ | D ✗ | D ✗ | D ✗ | D ✗ | D ✗ | C ✓ Figure 11. Qualitative examples of fine-grained perception. Left: the test image; the red box marks the region that the question refers to and is drawn only for this figure, since all models see the original image. Middle: the boxed region, enlarged. Right: the question, its options and the ground truth. Bottom: the answer of each model (✓ correct, ✗ wrong).

Figure 12. Qualitative examples of general perception. Left: the test image. Right: the question, its options and the ground truth. Bottom: the answer of each model (✓ correct, ✗ wrong).

(a) HallusionBench. The image shows a color illusion. Are two circles in the image different color?
Ground truth: Yes
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | No ✗ | No ✗ | No ✗ | No ✗ | No ✗ | No ✗ | Yes ✓ |
(b) HallusionBench. Are the lengths of the vertical line and the horizontal line the same?
Ground truth: Yes
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | No ✗ | No ✗ | No ✗ | No ✗ | No ✗ | No ✗ | Yes ✓ |
(c) HallusionBench. Based on the map, are the Rocky Mountains located in North America?
Ground truth: No
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | Yes ✗ | Yes ✗ | Yes ✗ | Yes ✗ | No ✓ | Yes ✗ | No ✓ |
(d) HallusionBench. Is there a Snoopy in this figure?
Ground truth: No
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | No ✓ | Yes ✗ | Yes ✗ | Yes ✗ | Yes ✗ | Yes ✗ | No ✓ |
Figure 13. Qualitative examples of hallucination. The layout is the same as in Figure 12.

Figure 14. Qualitative examples of multimodal reasoning. The layout is the same as in Figure 12.
(a) MMVP. Is the duck’s entire beak visible in the picture?
(a) Yes (b) No
Ground truth: (a) Yes
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (a) ✓ |
(b) MMVP. Are the wheels of the school bus visible in the image?
(a) Yes (b) No
Ground truth: (a) Yes
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (a) ✓ |
(c) CV-Bench. How many street lights are in the image?
A. 2 B. 3 C. 0 D. 1
Ground truth: D. 1
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | A ✗ | A ✗ | A ✗ | A ✗ | A ✗ | A ✗ | D ✓ |
(d) MMVP. From the camera’s perspective, are the words “SCHOOL BUS” in the picture facing straight or oriented at an angle?
(a) Directly towards the camera
(b) At an angle
Ground truth: (a) Directly towards the camera
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (b) ✗ | (a) ✓ |
(a) MathVista. In the figure, the measure of angle 9 is . Find the measure of angle 6.
A. 75 B. 85 C. 95 D. 105
Ground truth: D. 105
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | A ✗ | A ✗ | A ✗ | A ✗ | A ✗ | A ✗ | D ✓ |
Table 8.
(b) MathVerse. What fraction of the area of the figure is shaded?
A. 3/8 B. 1/4 C. 1/8 D. 1/10 E. 1/16
Ground truth: B. 1/4
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | A ✗ | A ✗ | A ✗ | A ✗ | A ✗ | A ✗ | B ✓ |
Table 9.
(c) MATH-Vision. The higher someone stands on the podium, the better the ranking. Which number got third place?
Ground truth: 5
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | 4 ✗ | 3 ✗ | 3 ✗ | 3 ✗ | 3 ✗ | 3 ✗ | 5 ✓ |
Table 10.
(d) We-Math. As shown in the diagram, Xiaoxue used a set of triangular plates to piece together an angle. Its measure is ( ).
A. B.
C. D. No correct answer
Ground truth: B.
| Model | Base | Vision-OPD | OPD-V | CV-OPD | VAD | OPD-Aha | BlinkOPD (ours) |
| Answer | A ✗ | A ✗ | A ✗ | A ✗ | A ✗ | A ✗ | B ✓ |
Table 11.
References
- [44]Mohammad Sadegh Akhondzadeh, Vijay Lingam, Atula Tejaswi, Chanakya Ekbote, Sujay Sanghavi, and Aleksandar Bojchevski. Reward-gated on-policy distillation. In Workshop on Efficient Reasoning at the Conference on Language Modeling (COLM), 2026. 17arxiv.org/abs/2607.04037
- [45]Siyuan Gan, Yuhan Li, Xiran Wang, Linjian Meng, Boyan Wang, Zhen Zhao, Jing Huo, and Yang Gao. When teacher guidance misleads: Reward-aligned on-policy distillation. arXiv preprint arXiv:2608.27960, 2026.arxiv.org/abs/2608.27960
- [46]Langlin Huang, Hao Liu, Monito Goswami, Xinyu Li, Prithwish Jana, Nikos Kanakaris, Patrick Blöbaum, and Purak Jain. On the off-policy teacher in on-policy distillation. arXiv preprint arXiv:2609.38360, 2026. 17arxiv.org/abs/2609.38360
- [47]Xin Lai, Junyi Li, Wei Li, Tao Liu, Tianjian Li, and Hengshuang Zhao. Mini-o3: Scaling up reasoning patterns and interaction turns for visual search. In International Conference on Learning Representations (ICLR), 2026. 17arxiv.org/abs/2509.07969
- [48]Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun Zhang, Mingwei Zhu, and Jianwei Yin. ZoomEye: Enhancing multimodal LLMs with human-like zooming capabilities through tree-based image exploration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 6602–6618, 2025. 17arxiv.org/abs/2411.16044
- [49]Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. HybridFlow: A flexible and efficient RLHF framework. In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297, 2025. 13DOI
- [50]Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel Reasoner: Incentivizing pixel space reasoning via curiosity-driven reinforcement learning. In Advances in Neural Information Processing Systems, pages 8222–8251, 2025. 17DOI
- [51]Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-VL: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. 13arxiv.org/abs/2409.12191
- [52]Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving Mamba2 with delta rule. In International Conference on Learning Representations (ICLR), 2025. 14arxiv.org/abs/2412.06464
- [53]Jiarui Zhang, Mahyar Khatayatkhoei, Prateek Chhikara, and Filip Ilievski. MLLMs know where to look: Training-free perception of small visual details with multimodal LLMs. In International Conference on Learning Representations (ICLR), 2025. 17arxiv.org/abs/2502.17422