WorksComputer Science
Counting Events, Not Seeing Them: Diagnosing and Repairing the Readout Failure in Video LLMs
Abstract
Video LLMs fail conspicuously at counting how many times an object crosses the screen, a failure attributed to missing object permanence. Using construction-grounded synthetic testbeds with validated labels, we show this attribution is incomplete: the count signal is present but unread - linear probes decode per-window presence at 93% while global count decoding collapses to a duration confound. Concurrent work reports the same representation-output dissociation for repetition counting; we show that in exit-and-reappear event counting it is repairable: two readout-directed repairs (episode-localization supervision and a discrete event-symbol side channel) raise same-object counting from 46.7% to 65.0% (oracle 66.7%) on a chance-33% benchmark across 7B-30B systems. Transfer to an independent factory costs nothing, and a measurement audit exposes five data-integrity failures that had obscured the boundary. A real-video audit (REALCOUNT, n=18, blind-audited) shows the boundary persists - best 55.6% at chance 33.3% - and symbol transfer fails zero-shot. A 1-epoch in-domain adaptation bench is negative (six levers at 28-50%), but 3 epochs reverse this for the symbol channel: with oracle timelines the reader reaches 94.4% sequence-disjoint (and 100% strict on the full set), while the deployed 2-epoch head reaches 61.1% in all three seeds (42-44), no better than a shuffled-symbol control (61.1% seed 42, 66.7% seed 43), and one clip above the unadapted reader (55.6%). The residual gap is detector evidence, not reader capacity: the same reader reaches 72.2-88.9% on GT timelines. The failure is repairable with informative timelines, but the deployed head remains the bottleneck; a frozen-feature boundary probe is weak (AUC 0.62 vs. 0.51 permutation) and non-transferring (0.77 within-clip vs. 0.62 across windows).
Introduction
Ask a video LLM how many times the red square crossed the screen, and it will fail—not occasionally but systematically, across model scales and training regimes. This “same-object counting” failure has survived every axis it has been tested on: 7B and 30B models, zero-shot and fine-tuned, dense and mixture-of-experts, attention-LoRA and expert-LoRA (Sec. 4). Because heterogeneous-object counting is near-perfect for the same models, the deficit has been characterized as missing object permanence—a representational gap in how identity is maintained across time [2].
This paper argues the deficit is elsewhere, and that it is repairable. Our diagnosis pipeline—error-structure analysis against trivial-baseline controls, linear probing of the model’s own token stream, and temporal-granularity measurements—locates the failure in the readout: the information needed to count is present in the model’s visual token stream (a linear probe recovers per-window object presence at 93%), but the language model cannot aggregate it into a count. Counting, for a video LLM, is not perception; it is symbol manipulation over evidence it can already see.
Concurrent work reaches the same conclusion for repetition counting [18]. We extend it in two directions: the regime is exit-and-reappear event counting on real video rather than periodic action repetition, and the dissociation is not merely diagnosed but repaired.
This reframing has a practical consequence: if the bottleneck is readout, the fix is not visual features but structured readout pathways. We evaluate two, both trained on construction-grounded data with episode-level labels that our factories emit for free:
Episode-localization supervision (output-side): half
Introduction
Ask a video LLM how many times the red square crossed the screen, and it will fail—not occasionally but systematically, across model scales and training regimes. This “same-object counting” failure has survived every axis it has been tested on: 7B and 30B models, zero-shot and fine-tuned, dense and mixture-of-experts, attention-LoRA and expert-LoRA (Sec. 4). Because heterogeneous-object counting is near-perfect for the same models, the deficit has been characterized as missing object permanence—a representational gap in how identity is maintained across time [2].
This paper argues the deficit is elsewhere, and that it is repairable. Our diagnosis pipeline—error-structure analysis against trivial-baseline controls, linear probing of the model’s own token stream, and temporal-granularity measurements—locates the failure in the readout: the information needed to count is present in the model’s visual token stream (a linear probe recovers per-window object presence at 93%), but the language model cannot aggregate it into a count. Counting, for a video LLM, is not perception; it is symbol manipulation over evidence it can already see.
Concurrent work reaches the same conclusion for repetition counting [18]. We extend it in two directions: the regime is exit-and-reappear event counting on real video rather than periodic action repetition, and the dissociation is not merely diagnosed but repaired.
This reframing has a practical consequence: if the bottleneck is readout, the fix is not visual features but structured readout pathways. We evaluate two, both trained on construction-grounded data with episode-level labels that our factories emit for free:
Episode-localization supervision (output-side): half
the MCQ budget is replaced by a task asking the model to state the start and end time of every crossing. The model learns to articulate the event structure it is asked to count (episode recall 84% at ).
An event-symbol side channel (input-side): a lightweight learned detector (a 3-class head over frozen connector features with edge-band spatial pooling) emits one symbol per half-second—# present, . absent, | transition—injected into the prompt at a cost of one token per slot. Counting becomes counting symbols, the regime in which the same models are near-perfect.
Both interventions improve over identically-budgeted clean-data training by +10.0 to +15.0 points (paired bootstrap, –.05), reaching 65.0% deployable / 66.7% oracle on a chance-33% benchmark where every prior configuration sat at 43–52%.
Along the way we audit the measurement apparatus itself, and report five data-integrity findings that changed the reported numbers: an evaluation set fully contained in training data (producing “impossible” centisecond precision that was memorization), a timestamp-precision artifact, a label error in a reused motion formula that made “continuous crossing” videos contain a single crossing, a training-window episode-truncation bug that wrote 64 episode ends to the clip end, and a training-prompt class mismatch in 29 of 145 real-video windows. We distill these into auditable rules (Appendix § 11).
Contributions.
A controlled re-characterization of the same-object counting boundary that independently confirms and extends the representation–output dissociation reported for repetition counting ([18]): prior-matched evaluation, confound-controlled probing, and a granularity analysis together relocate the deficit from representation to readout (§ 5).
Two readout-directed repairs—localization supervision and a deployable event-symbol side channel—with a full-stack combination, noise-matched training, and an edge-band detector head; each component evaluated with paired bootstrap statistics (§ 6–§ 7).
A measurement-integrity audit of the evaluation infrastructure itself, with three documented failure modes and their detection rules (Appendix § 11).
All data factories emit construction-grounded episode labels validated by independent measurement (93–99.7% agreement), and every reported number is reproducible from archived records.
A real-video audit (REALCOUNT, § 8): an exit-and-reappear counting set built from long-term tracking benchmarks, where every one of the final 18 items survived a blind multi-pass frame-level audit (23% yield from 78 candidates)—showing the boundary persists on natural footage and that neither the probe readability nor the symbol channel transfers zero-shot.
A six-lever in-domain adaptation bench that, at a 1-epoch budget, fails to exceed the unadapted baseline—and a 3-epoch schedule showing that the symbol-channel lever reverses this once trained (94.4% with sequence-disjoint oracle timelines, 100% strict on the full set), while the deployed 2-epoch head reaches only 61.1% and does not beat a shuffled-symbol control (61.1% in seed 42, 66.7% in seed 43)—plus a boundary probe that localizes the residual real-video gap to the frozen evidence rather than the adapter (§ 8.1).
Related Work
Video LLMs and temporal perception. Dominant video-LLM architectures sample frames uniformly and encode them per-frame ((Xu et al., 2025a)); documented failures include motion blindness and temporal-grounding errors ([22] (Fu et al. 2024)). Fine-grained temporal tasks (counting repeated events, ordering, speed) remain weak even at frontier scale—our § 5 confirms this across a 7B→30B sweep—motivating diagnosis rather than scale.
Probing and diagnosis. Linear probing to localize information within frozen representations is standard ([1]). We extend it with two controls specific to temporal benchmarks: a duration confound baseline (clip length predicts event count in most generated data) and a permutation control, without which probe results overstate what tokens carry.
Object permanence and event segmentation. Cognitive work distinguishes representing continuity from enumerating discrete events ([2, 35]). Our results operationalize this distinction in video LLMs: presence (continuity) is linearly readable; enumeration is not.
Concurrent diagnosis of counting failures. A 2026 wave of benchmarks measures video counting from complementary angles: repetition counting at controlled counts ([18] (Li et al. 2026)), trace-grounded event bookkeeping ([3]), momentary events under dense sampling ([21]), streaming state maintenance ([20]), long-video enumeration ([29]), numerical reasoning ([8]), object counting under controlled cardinality ([4]), and object-centric temporal consistency including disappearance and reappearance [6]. Two findings recur: accuracy collapses at higher counts even when frames cover the evidence, and count information is often linearly decodable while the output is wrong [18, 3, 21]. These works diagnose; they do not repair. We ask whether the dissociation is repairable in a real-video exit-and-reappear regime.
Data contamination and measurement validity. Benchmark leakage into training data is an active concern (Xu et al., 2024). Appendix A contributes five concrete failure modes from a single project’s audit—set-inclusion leakage, memorization masquerading as precision, formula-reuse label errors, training-window episode truncation, and a training-prompt class mismatch—with cheap detection rules.
Prompt-side structured evidence. Feeding text extracted from the video into the LLM context is established for long-video QA: caption-style pipelines summarize frames or clips into textual timelines consumed by the LLM [36, 31], and temporal-grounding models integrate timestamp or event tokens into the token stream itself [23, 24, 16, 30, 14]; tool-use frameworks render detector and tracker outputs as text for the solver [27, 13, 25].
Specialist side channels and structured event evidence. Repetition counting on real video has been studied outside LLMs [9] and recently as a VLM capability benchmark [18, 26]. Closest to our channel, detector-grounded structured prompts improve image counting [5], a specialist’s scalar count helps when delivered as a discrete symbol rather than a density map [11], and training-free visual markers or object-state tokens improve temporal grounding and persistence [10, 7]. These deliver boxes, scalar counts, visual overlays, or object states; our channel delivers a per-slot three-state event timeline, and we test its causal role with no-symbol, shuffled-symbol, and sequence-disjoint controls. Against these neighbors, our event-symbol channel is deliberately minimal—one 3-class symbol per fixed half-second slot (one token, not a caption)—produced by a lightweight learned detector whose noise model is matched to the reader at training time (Sec. 5.4), and targeted at a diagnosed readout failure (same-object counting) rather than generic captioning, grounding, or retrieval. To our knowledge, no prior work pairs a per-slot event-presence symbol channel with a matched deployable detector and causal controls on real exit-and-reappear counting.
| System | v4 count (chance 33.3%) |
| 7B + transition-trained checkpoint (prior best) | 46.7 [33.3, 60.0] |
| 7B same-mix, no-fast-flow retrain | 48.3 |
| 30B zero-shot | 43.3 |
| 30B + expert-LoRA (published recipe) | 33.3 (constant) |
Table 1. The historical “46.7% plateau”, re-measured on balanced v4.
Experimental Setup
Models. The intervention platform is Qwen2.5-Omni-7B (Thinker) (Xu et al., 2025a) with a unified-token-stream architecture: a frozen VideoMAE-based fast-flow connector [28] injects 196-token virtual rows into the video token grid, and the top 4 LLM layers are unfrozen (0.93B trainable + 11.6M connector). All intervention arms share an identical lineage (initialized from the same predecessor checkpoint), identical non-count data mixtures, and matched budgets: 2,159 steps, except arm2/arm3, which use 2,939 steps to cover the extended nogap corpus (Sec. 5.4). Baselines additionally include Qwen3-Omni-30B-A3B (Xu et al., 2025b) (4-bit, zero-shot and with a published expert-LoRA adapter).
Data factories. All counting data are synthetic with construction-grounded ground truth. count_v4: red squares crossing real-video backgrounds; gap variant ( independent crossings separated by off-screen intervals, 0.4–0.7 s) and nogap variant (continuous mod-wraparound crossings); exactly balanced across six (variant ) evaluation groups of 10 (, chance 33.3%). v1x: an independently written factory (different formula instantiation, prompts, and backgrounds; ) used only for transfer evaluation. Episode labels (per-crossing []) are emitted by construction and verified by independent measurement—a shape-criterion detector (99.7% agreement on held-out gap clips) and a centroid-wraparound counter (nogap).
Protocol. MCQ answers are parsed with a fixed letter grammar; every arm reports the constant-answer baseline alongside; paired bootstrap () over per-sample outcomes for all deltas. Reported -values are uncorrected (no family-wise adjustment); the main contrasts were pre-specified in the archived run logs.
Diagnosis: Present but Unread
The boundary, re-measured honestly
Two observations dismantle the naive reading of the historical “46.7% plateau”. First, it underperforms the constant-most-frequent-class baseline of 60%—the model was not partially counting; its predictions were prior-shaped plus a systematic over-segmentation bias (single crossings answered as 2 in 11/18 cases; count-3 essentially never produced). Second, the label prior itself was confounded: training data were count-2-heavy, evaluation count-1-heavy. Every arm in Table 1 sits at 1.0–1.5× chance (Fig. 1a,b).
Probing: local presence yes, global count no
With the connector’s per-window token stream (what the LLM actually consumes as motion evidence):
P1 (window presence): 5-fold CV logistic regression decodes “object visible in this 2 s window” at 93.3% (majority baseline 74%; absent-class recall 0.997).
P2 (global count): pooled-token decoding reaches 72.3%—but a duration-only logistic baseline reaches 67%, and a permutation control sits at 50%. The apparent count signal is substantially a duration proxy; episode structure specifically is not linearly readable.
P3 (granularity): the 0.4–0.7 s off-screen intervals that separate episodes are resolvable only at ≤1 s windows (91% of multi-episode clips produce a fully-absent window at 1 s; 95% at 0.5 s; 73% at 2 s). The 4 s virtual-row granularity of the deployed architecture is thus physically incapable of episode separation—a design constraint, not a learned failure (Fig. 1c,d).
The measurement was part of the problem
Two label-integrity findings (details in Appendix A): the historical hard-cut evaluation set was fully contained in training data (its “centisecond timestamp precision” was memorization—provably, since the precision exceeded every input channel’s physical resolution and survived time-reversed inputs); and a reused crossing formula was linear rather than periodic, silently converting ∼465 “continuous crossing” samples into single-crossing videos with multi-crossing labels. After repair and -balancing, the boundary stands (Sec. 4)—but only now is it measured.
Diagnosis. Perception: intact. Local representation: present. Global aggregation: absent. The deficit is a readout failure.
Interventions
All arms: identical lineage, budget, and non-count mixture; only the intervention differs.
A-base (control): clean, balanced data
Replacing the corrupted count data with count_v4 (true periodic formula, balanced , episode labels) while holding everything else fixed: 51.7%. Clean data alone repairs the over-segmentation pathology (8/10 correct) but leaves near chance—the readout deficit persists through data repair (+5.0, CI [−8.3, +18.3] vs prior checkpoint).
arm0: episode-localization supervision (output-side)
Half the MCQ budget is exchanged for localization queries (“state the [start, end] interval of every crossing”, answers in JSON), 61.7%, with genuinely learned localization (episode recall 84.4%, precision 69.7% at IoU≥0.5; the untrained control produces out-of-range hallucinated intervals at 50%/23.6%). Paired vs A-base: +0.100 [+0.000, +0.217], . Notably this arm trains on half the MCQ examples yet outperforms—supervision structure beats repetition.
arm1: event-symbol side channel (input-side)
A presence head over the frozen connector’s per-0.5 s window features (context-concatenated ) emits a three-class symbol per slot; timelines are appended to the prompt (slot symbols: |###|###|). Training uses ground-truth timelines with 10% slot flips; inference uses head predictions. 63.3% (head pipeline) / 61.7% (oracle timelines) / 55.0% (symbols withheld). Three controls pin the mechanism: symbols injected into a symbol-untrained model do nothing (45.0% ≈ baseline); random-feature ablations of the underlying channel eliminate the gain; and the model is robust to timeline noise (head ≈ oracle within noise). Figure 2 shows the channel architecture.

Figure 2. Event-symbol side channel. Real rows flow through the frozen vision tower; the frozen fast-flow connector supplies virtual rows (motion evidence) whose per-0.5 s window features feed the independent head v2 (six-band spatial pooling over raw VideoMAE features). The resulting symbol timeline (one token per half-second slot) is appended to the prompt, converting same-object counting into symbol counting. Episode-localization supervision (dashed) acts on the output side during training only.
arm2/arm3: full stack, noise matching, and the deployable head
Combining symbols + localization + extended nogap coverage (period spectrum 1.5–7.2 s): oracle 66.7% (ceiling), head-v1 60.0%. The gap between oracle and deployed timelines is closed not by prompt normalization (which slightly hurts) but by noise-matched training (arm3: train on head predictions; head = oracle = 60.0%, confirming the distribution-matching hypothesis) and by fixing the detector’s inductive bias: the ball-at-edge signal is spatial, but the connector’s 196-token pooling mixes the spatial grid. Head v2 pools six spatial bands (left/right/top/bottom/center/global) directly from raw VideoMAE features—an independent branch that leaves the connector (and hence the LLM’s trained alignment) untouched—lifting slot accuracy 64.7% → 81.6% (per-class recall 77/82/84) and the deployed pipeline to 65.0%, 1.7 points under the oracle ceiling. The detector is base-model-agnostic by construction.
| Arm | Acc [95% CI] | vs A-base, |
| Prior checkpoint (dirty data) | 46.7 [33.3, 60.0] | — |
| 30B zero-shot | 43.3 [31.7, 56.7] | — |
| A-base (clean data) | 51.7 [38.3, 65.0] | — |
| arm0 (+localization) | 61.7 [48.3, 73.3] | +0.100, .047 |
| arm1 (+symbols, head) | 63.3 [51.7, 75.0] | +0.117, .008 |
| arm2 (full stack, oracle) | 66.7 [55.0, 78.3] | +0.150, .010 |
| arm2 (full stack, head v2) | 65.0 [53.3, 76.7] | +0.133, .008 |
Table 2. Main results on v4 with paired-bootstrap confidence intervals (). All structural interventions are significant over A-base.
Results
Main table (v4, n=60, chance 33.3%)
arm1 vs arm0: +0.017, —the two pathways are statistically indistinguishable; each is an independent witness to the readout hypothesis. Head v2 vs v1: +0.050, (trend; the head’s value is established at the slot level, : 81.6% vs 64.7%). The full repair chain is summarized in Figure 3.

Figure 3. Repair waterfall. From the dirty-data checkpoint (46.7) to the deployed full stack (65.0): clean balanced data (+5.0), episode-localization supervision (+10.0, ), the event-symbol side channel (+1.7 on top of arm0; +11.7 vs A-base, ), and the edge-band head v2 (+1.7). The oracle ceiling (66.7) sits 1.7 points above deployment.
Per-k structure
is solved by clean data alone (8–9/10 across arms). The structural interventions specifically repair (gap: 5/10 → 9/10 arm0, 8/10 arm1-head). The residual frontier everywhere is the boundary (Figure 4).

Figure 4. Per-cell accuracy across the repair chain (each cell , dashed line = chance 3.3/10). Clean data fixes gap ; the structural arms specifically lift gap (0/10 7–9/10); the boundary remains the shared residual, and nogap improve only under the full stack.
Transfer (v1x, n=90, independent factory)
run11-lineage 61.1 → A-base 76.7 [+67.8, +84.4] ( vs prior +0.156, ) → structural arms 71.1–72.2 with no significant transfer cost (arm2−A-base = −0.044 [−0.133, +0.044], ). Clean balanced data is itself a strong transfer vehicle; the readout interventions buy in-distribution gains at no measured cross-factory cost.
Real-Video Validation (REALCOUNT)
Everything to this point lives in one synthetic paradigm: a red square crossing real-video backgrounds. We therefore built REALCOUNT, a real-video exit-and-reappear counting set, and re-ran the diagnosis and both readout repairs on natural footage.
Data and audit. REALCOUNT draws on VOT2018-LT and VOT2019-LT (85 long-term tracking sequences, selected because the target leaves the frame). Absence is encoded in the tracking ground truth itself (the box becomes NaN while the target is out of view—verified at frame level); episodes are visible runs of s separated by s absences, and is the number of re-entries in a 5–25 s window. Candidate windows () were sampled around re-entries, then every candidate passed a two-layer audit: 1-fps contact-sheet inspection of all 60 initially selected windows, followed by three independent blind review passes that frame-verified every entry/exit boundary with the ground truth hidden. Only 18 of 78 audited windows (23%) survived—rejected for multi-instance scenes (15), first-person/onboard cameras (5), occlusion-vs-exit ambiguity (3), GT-vs-visual mismatches (7), class mismatches (7), unverifiable targets (6), and query ambiguity (2). The final set is (: 5/7/6; 11 sequences; 5–25 s), every label blind-audited; the audit protocol and full rejection taxonomy are released with the set.
Findings. (i) The boundary persists. No unsigned system is reliably above chance under a two-sided test (best 55.6%, ; two-sided binomial , one-sided ); natural footage is not easier than the synthetic paradigm. (ii) The diagnosis sharpens. On real video, even
local presence is not linearly readable from the connector’s tokens (72.6% vs 71.4% majority; synthetic: 93.3% vs 74%) and global count decoding sits at chance—“present but unread” becomes “presence itself linearly unread”. (iii) The symbol channel does not transfer zero-shot. Injecting oracle ground-truth timelines hurts (harmful in every bootstrap resample), and the REALCOUNT-trained detector transfers poorly (slot accuracy 58.0% vs the 71% trivial all-present bound; absent-class recall 19%); the timeline length distribution (up to 50 slots vs at training) compounds the mismatch. The deployability claim of Sec. 5.4 is therefore bounded to the synthetic paradigm unless the channel is trained in-domain; the in-domain result separates the oracle channel from the deployed head (Sec. 7.1). (iv) What does transfer is the readout supervision itself: the arm2 reader remains the strongest counter without symbols (55.6%). Closing the loop on real video requires in-domain training, and our first attempts delimit the difficulty. Naive full-layer SFT on the REALCOUNT training windows (warm-started from arm2, halved learning rate, one epoch) collapses the reader’s output distribution on held-out clips (55.6% unparseable); a pipeline audit further shows that 56 of 131 windows exceeded our conservative 8,000-token training budget (the backend fails above seq , where 28 heads seq) and were silently skipped, so the effective SFT set was 75 windows. Parameter-efficient adaptation (rank-16 LoRA (Hu et al., 2022) on all language-attention projections, connector frozen) eliminates the collapse—18/18 answers parseable—and synthetic accuracy preserved (53.3% 56.7%)—but does not restore real-video counting (27.8%: 50.0% without synthetic replay, vs. 55.6% baseline). The residual failure is an answer-prior collapse: both PEFT variants answer “three” for most clips (: 0/7, : 5/6) and never recover the two-event cell, even in-sample (61.3% overall, but
| Arm | REALCOUNT acc (chance 33.3%) | ||
| run11 reader, no symbols | 50.0% | ||
| arm2 reader, no symbols | 55.6% | ||
| arm2 + oracle GT timelines | 44.4% () | ||
| arm2 + REALCOUNT-trained head | 33.3% () | ||
| 30B zero-shot | 44.4% (: 0/5; : 5/6) | ||
| P1 slot-presence linear probe | 72.6% majority 71.4% | ||
| P2 pooled-count linear probe | 33.3% = duration = chance | ||
| REALCOUNT-trained detector | 58.0% < 71% (all-present) | ||
| Lever (trainable) | REAL-COUNT | v4 | Observation |
| arm2 baseline (—) | 55.6% | 53.3% | reference |
| Readout LoRA + replay (10.1M) | 27.8% | 56.7% | format fixed; prior collapse |
| Readout LoRA, no replay | 50.0% | 51.7% | parity; prior stays |
| Readout LoRA, 145 win. | 50.0% | 58.3% | prior → mode |
| Connector projection (11.6M) | 44.4% | 50.0% | no gain; no prior shift |
| Localization objective | 44.4% | 61.7% | boundary recall 18.9% |
| Encoder last-2 blocks (25.8M) | 44.4% | 51.7% | weights move; probe unchanged |
Table 3. Real-video validation. ; every label verified by blind frame-level review; paired deltas vs the arm2 reader over resamples.
and . Doubling the real training set (75 → 145 windows, with better balance) leaves held-out accuracy unchanged—9/18 in both runs (, )—while merely redistributing the prior: the cell recovers from 0/7 to 5/7, the cell falls from 5/6 to 2/6, and 12 of 18 answers become the training-set mode (“two”). Data quantity in this regime moves the marginal answer distribution, not the reading skill. We read this as evidence that the bottleneck is the evidence and the paired supervision that teaches its use, not the optimizer. Sec. 7.1 tests the remaining escape route—paired symbol training—and a 3-epoch schedule shows that the 1-epoch null is budget-limited: with oracle timelines the reader reaches 94.4% sequence-disjoint (100% strict on the full set), while the deployed 2-epoch head reaches 61.1% and does not beat a shuffled control (61.1% in seed 42, 66.7% in seed 43).
The adaptation ceiling
Table 4 collects five in-domain adaptation levers evaluated on the same 18-clip benchmark at a 1-epoch budget, each warm-started from the arm2 reader and trained only on REALCOUNT windows outside the evaluation spans. We additionally test a sixth—paired symbol training, the missing arm that the synthetic result calls for—at the same 1-epoch budget in Table 5, and then extend its oracle-timeline variant to three epochs in Table 6.
| Training timelines | head @ test | GT @ test |
| none (arm2, zero-shot) | 44.4% | 50.0% |
| deployed head (145 win.) | 50.0% | 61.1% |
| oracle GT (145 win.) | 44.4% | 44.4% |
Table 5. Paired symbol training on REALCOUNT at the 1-epoch budget (; chance 33.3%). Rows: timelines used during training (145 windows, same LoRA recipe); columns: timelines at test. The unadapted arm2 without symbols scores 55.6%.
| Condition | REALCOUNT | v4 |
| 1-epoch oracle (corrected) | 44.4 | 65.0 |
| 3-epoch oracle, full 145 | 100.0 | 73.3 |
| 3-epoch oracle, seq-disjoint 117 | 94.4 | 85.0 |
| 3-epoch no symbols | 50.0 | 61.7 |
| 3-epoch shuffled symbols | 61.1 | 58.3 |
| 2-epoch deployed head @head | 61.1 | 53.3 |
| 2-epoch deployed head @GT | 77.8 | — |
| 3-epoch seq-disjoint @head | 66.7 | — |
Table 6. REALCOUNT controls on the episode- and prompt-corrected training manifest (; chance 33.3%). The full-145 row includes 28 windows from sequences that overlap the evaluation set; the sequence-disjoint row is the primary generalization result. @GT/@head denote the timelines used at test. v4 = synthetic retention (). Seed-42 values; seed variance in Table 7.
At the 1-epoch budget, no deployable lever exceeds the unadapted reader (best deployable 50.0%, non-significant deltas; paired symbol training included; the GT-timeline head reader reaches 61.1%, but it is not deployable); synthetic retention for the six levers ranges 50–63%
without symbols (65–66.7% with), and the clean 1-epoch oracle retains 65.0%. The localization arm is the most diagnostic: it produces parseable JSON for all 18 clips and matches in 10, yet boundary recall at IoU≥0.5 is only 18.9%; 12 of 14 long (>3 s) ground-truth episodes are fragmented into two or more predicted intervals, and predicted durations have p90 2.7 s against 7.5 s, despite 25 training episodes longer than 6 s. What this arm learns is a duration/count prior, not boundary reading. A linear probe on frozen six-band VideoMAE features reaches boundary AUC 0.62 (permutation control 0.51; edge recall 0.33, absent recall 0.21), and a nonlinear head on the same features does no better on edges (0.25)—so readout capacity is not the limit. Fine-tuning the last two VideoMAE blocks changes weights (max ) but leaves both accuracy and the probe unchanged (AUC 0.766/recall 0.636 before and after); the same probe reaches 0.766 within clips but only 0.62 across windows, i.e., the boundary signal exists in-distribution but does not transfer. At this data scale (75–145 windows) the 1-epoch bench fails on weak boundary evidence and poor cross-clip transfer. The symbol-channel arm is the exception once the budget is increased (Table 6): a 3-epoch schedule installs counting with oracle timelines (94.4% sequence-disjoint; 100% strict on all 145 windows), while the deployed 2-epoch head reaches 61.1% and does not beat a shuffled-symbol control (61.1% in seed 42, 66.7% in seed 43), so the remaining gap is detector fidelity and reader–detector mismatch rather than adapter capacity. A controlled timeline-corruption analysis (Appendix § 13, Table 8) sharpens this: absent-slot loss dominates , transition loss dominates , and repairing both with oracle patches recovers 17/18.
| Timeline condition | All | |||
| GT | 5/5 | 7/7 | 5/6 | 17/18 |
| GT, all absent slots filled | 0/5 | 5/7 | 6/6 | 11/18 |
| GT, transitions removed | 5/5 | 6/7 | 3/6 | 14/18 |
| Deployed head | 3/5 | 5/7 | 4/6 | 12/18 |
| + GT absent slots | 5/5 | 5/7 | 4/6 | 14/18 |
| + GT boundary windows | 1/5 | 5/7 | 6/6 | 12/18 |
| + both | 5/5 | 6/7 | 6/6 | 17/18 |
Table 8. Timeline corruption and repair on REALCOUNT (; chance 33.3%). All rows use the sequence-disjoint oracle reader. “+” rows copy GT symbols into the deployed-head timeline and are oracle interventions, not deployable results.
Paired training at the 1-epoch budget does not install deployable symbol reading. No deployable cell in Table 5 exceeds the unadapted reader: the head-trained reader reaches 50.0% on head timelines (vs. 44.4% for the same channel zero-shot, clip, ), while its GT-timeline cell reaches 61.1%—above the 55.6% baseline, but not deployable. The GT-trained reader is at 44.4% under both timeline sources, i.e., it does not install symbol reading at this budget. The clean 1-epoch oracle retains 65.0% on v4, so this is not catastrophic forgetting. The null is budget-specific, not a capacity limit: extending the oracle-timeline arm to 3 epochs reverses it (Table 6).
The 1-epoch null is a budget artifact. Extending the oracle-timeline arm to three epochs reverses the 1-epoch null. On the episode- and prompt-corrected manifest, a 3-epoch sequence-disjoint run (117 windows from the 19 sequences that never appear in evaluation) reaches 94.4% (17/18; 5/5, 7/7, 5/6; v4 85.0%). Training on all 145 corrected windows reaches 100% strict in both seeds tested (18/18; v4 73.3%/85.0%); because the full set includes 28 windows from sequences that overlap the evaluation set, we treat the sequence-disjoint 94.4% as the primary generalization result and report the full-set 100% as a secondary strict result. Two controls bound the mechanism: a 3-epoch no-symbol run reaches 50.0% and a content-free shuffled-symbol run 61.1% in seed 42 (66.7% in seed 43), so the oracle gain requires informative symbols. The deployed 2-epoch head reaches 61.1% (11/18) in all three seeds (42–44), no better than the shuffled control (61.1% seed 42, 66.7% seed 43) and one clip above the unadapted reader (55.6%); the same reader reaches 72.2–88.9% on GT timelines (seed 42: 77.8%, 14/18), and the sequence-disjoint oracle reaches 94.4% in both seeds tested (42–43; v4 75–85%). The sequence-disjoint oracle reader reaches 66.7% (12/18) on head timelines. The residual 33.3-point gap (94.4 vs. 61.1) is therefore detector evidence and a head–reader mismatch, not reader capacity: the reader can read clean timelines, but the head’s timelines carry too little usable evidence. A head-trained reader does not beat the GT-trained reader on head timelines (61.1 vs. 66.7), so noise-matched training does not close the gap at this scale (Table 6).
Discussion
Why symbols work. The same models are near-perfect at counting discrete heterogeneous events (flashes of distinct colors: 93–100%). The symbol channel converts continuous same-object evidence into exactly that regime. The division of labor is measurable: the detector supplies structure (84% edge recall after the band-pooling fix), the LLM supplies symbol manipulation (robust to clustered and missing symbols—it spontaneously learned “a run of transition markers is one event”, outperforming our handwritten counting rule on the same noisy timelines).
The failure mode we observe is a real-video instance of the preferred-number collapse RepProbe names: answers concentrate on a few numbers ([18]). Our 1-epoch readers show the same collapse (: 0/7; the training-set mode becomes the answer), and the symbol channel removes it by moving aggregation outside the reader: the model counts discrete symbols rather than regressing a count from continuous visual tokens. This is consistent with the abstraction-level account of specialist cues in crowd counting ([11]), and it explains why merely marking items is not enough ([17]): the symbols must carry event structure and the reader must be trained to consume them.
Deployment shape. The channel costs one token per half-second, requires no base-model retraining to attach (arm0-trained and arm1-trained readers both exploit it), and its detector trains in minutes on frozen features. On the synthetic paradigm the remaining gap is small: the deployed head sits 1.7 points below the oracle, and better detectors translate directly into the ceiling. On REALCOUNT the gap is much larger: the oracle reader reaches 94.4% sequence-disjoint (and 100% strict on the full set), but the deployed-timeline reader reaches 61.1% and the same reader reaches 72.2–88.9% on GT timelines. The 33.3-point gap is therefore detector evidence, not reader capacity: the reader can read clean timelines, but the head’s timelines carry too little usable evidence. A head-trained reader does not beat the GT-trained reader on head timelines (61.1 vs. 66.7), and the deployed head does not exceed a content-free shuffled control (61.1% in seed 42, 66.7% in seed 43). Noise-matched training alone does not close this gap; better detectors (unfrozen backbones, richer temporal context) and a calibrated head–reader pair are the next step. The channel is not yet deployable on real video; the oracle result establishes that the readout failure is repairable in principle.
Limitations. (i) Synthetic testbeds—necessary for construction-grounded episode labels at this scale: the transfer factory and the REALCOUNT audit (§ 8) mitigate but do not eliminate domain concerns. (ii) One base family at intervention scale (7B); the 30B replication is prepared but was not run (hardware budget). (iii) with reported CIs; per- cells are smaller still. The main synthetic arms are single-seed (seed 42); on REALCOUNT, the deployed head was repeated across seeds 42–44 (61.1% each), the sequence-disjoint oracle across 42–43 (94.4% each), the shuffled control across 42–43 (61.1%/66.7%), and the GT-timeline reader varies 72.2–88.9%; the remaining arms are single-seed. (iv) The oracle-vs-deployment gap, though small on the synthetic paradigm, is large on REALCOUNT and is set by detector fidelity and by the reader’s match to that detector’s noise. (v) REALCOUNT is : small by construction—every item is blind-audited, and the 23% audit yield is itself a finding about real-video episode counting—but CIs are wide (±20pp level) and cross-domain deltas should be read as directional. (vi) The adaptation bench (§ 8.1) is negative at the 1-epoch budget; a 3-epoch schedule reverses the symbol-channel arm with oracle timelines (94.4% sequence-disjoint; 100% strict on the full set), but the deployed head stays at 61.1%, no better than a shuffled-symbol control (61.1% in seed 42, 66.7% in seed 43) and only one clip above the unadapted reader. The head-trained reader does not beat the GT-trained reader on head timelines (61.1 vs. 66.7); the readout failure is repairable with informative timelines, but the deployed head is not yet sufficient. The full-145 100% is secondary because 28 of its windows overlap the evaluation sequences. (vii) Concurrent counting benchmarks are one to three orders of magnitude larger than REALCOUNT (e.g., 42K clips in RepProbe [18]); we position REALCOUNT as an audited repair testbed, not a coverage benchmark.
Impact Statement
This work is diagnostic and benchmark-oriented. On the positive side, it gives the community a measurement protocol for a failure mode that is easy to miss (label priors, leakage, and readout bottlenecks), a blind-audited real-video counting set, and evidence that a 1-epoch in-domain adaptation bench does not fix the failure while oracle event timelines do (94.4% sequence-disjoint; 100% strict on the full set), whereas the deployed head remains at 61.1% in all three seeds and does not beat a shuffled control (61.1% in seed 42, 66.7% in seed 43)—which should redirect effort toward detector fidelity and calibrated head–reader pairs rather than adapter tuning. On the negative side, counting and temporal-grounding technology can support surveillance or automated monitoring; we release no person-tracking model, no raw person footage, and no deployment-ready detector. The REALCOUNT clips derive from VOT-LT research sequences under their original licenses; we release audit sheets, manifests, and generation scripts rather than redistributing raw frames. Compute was modest (a single 64 GB workstation; the 30B model was used for inference only), so the direct environmental cost is small. We see no other specific ethical risks beyond those already present in video-understanding research.
Data-Integrity Findings
Set-inclusion leakage. A hard-cut evaluation set was fully contained (20/20) in the training core from an early fine-tune onward, producing a lineage of “record” timestamp precisions (11–50 ms). Detection: file-level ID intersection audits as a release gate.
Memorization masquerading as precision. The leaked precision survived time-reversed inputs unchanged—exceeding every input channel’s physical resolution. Rule: impossible precision is a leak signal; input-scrambling (reversal, content freezing) is a cheap probe.
Formula-reuse label error. A crossing expression reused across two factories was linear in one (object exits and never returns), silently invalidating “continuous crossing” samples. Rule: construction-grounded labels still require independent measurement (here: presence-episode counting for gap data, centroid-wraparound counting for nogap).
Training-window episode truncation. In 64 of the 145 REALCOUNT training windows, every episode end was written to the clip end because the candidate builder used the wrong map key; the resulting episodes overlapped and marked post-exit frames as present. The evaluation set was unaffected. Recom-puting episodes from GT removed all overlaps and preserved for all 145 windows; the oracle-timeline readers in § 8.1 and Appendix § 13 were retrained on the episode-repaired manifest. Detection: compare each episode end with the next absence run, never with the clip end.
Training-prompt class mismatch (closed in this version). A blind audit of the 145 REALCOUNT
training windows found 29 prompts whose object class did not match the tracked target: 27 car windows were prompted as a rider and 2 dog windows as a dragon. Episodes and symbol timelines refer to the tracked target and are unaffected; only the reader’s class cue is wrong. The evaluation set is unaffected (class mismatches were rejected during its audit). A corrected-prompt retrain of the core arms is reported in Tables 4–Table 6; it closes the disclosure. Detection: compare the prompt class with the tracker’s target class for every training window, not only the evaluation set.
All five were found because the diagnosis demanded controlled measurement; we expect comparable rates elsewhere in synthetic video benchmarks.
Reproducibility and Disclosure
Every number in this paper is reproduced by the anonymized supplementary package (README, the exact per-run argument dictionary extracted from each checkpoint, and one JSONL row per evaluation sample): verify_results.py re-derives all table values from the archived records without model weights or videos. The archival version will receive a permanent DOI upon acceptance.
Compute. All 7B training and evaluation ran on a single Apple M5 Pro workstation with 64 GB unified memory (MPS backend); no GPU cluster was used. Each 2,159-step arm takes 8–12 h, each 2,939-step arm about 12 h, a 435-step run about 4 h, and the presence heads train in minutes on cached features. The 30B baseline is 4-bit inference only; its fine-tuning replication was not run.
AI-assistance disclosure. The research programme—problem selection, experimental design and its approval, hardware and budget decisions, stopping rules, and acceptance of reported results—was directed by the human author. All code, data factories, experiments, and manuscript text were produced by an AI coding agent operating under that direction. No AI system is an author of this work: responsibility and accountability for every claim rest entirely with the human author, whose obligation is to verify each result against the archived records before submission.
Controls and Cross-Timeline Cells
Controls. On the episode- and prompt-corrected manifest, 1 epoch reaches 44.4%; 3 epochs reach 100% strict (18/18) on all 145 windows and 94.4% (17/18) with sequence-disjoint training. The full 145 set includes 28 windows from sequences that overlap the evaluation set, so the sequence-disjoint result is the primary generalization
claim. No-symbol (50.0%) and shuffled-symbol (61.1% in seed 42, 66.7% in seed 43) controls show that the oracle gain requires informative symbols, not extra in-domain training or a content-free token stream. Synthetic retention is 65.0% (1 epoch), 73.3% (full 3 epochs), 85.0% (sequence-disjoint, seed 42; 75.0% in seed 43), 61.7% (no symbols), 58.3% (shuffled, seed 42; 63.3% in seed 43), and 53.3% (deployed head, seed 42; 60.0/50.0 in seeds 43/44); see Table 7. The full-145 and sequence-disjoint arms have overlapping v4 retention (73.3–85.0 vs. 75–85); the REALCOUNT difference is one clip (100 vs. 94.4). We treat sequence-disjoint as the primary generalization result because 28 of the full-145 windows overlap the evaluation sequences, not because it has better retention.
| Condition | seed 42 | seed 43 | seed 44 |
| deployed head @head | 61.1 | 61.1 | 61.1 |
| deployed head @GT | 77.8 | 72.2 | 88.9 |
| shuffled symbols @GT | 61.1 | 66.7 | — |
| full oracle @GT | 100.0 | 100.0 | — |
| v4 full oracle | 73.3 | 85.0 | — |
| seq oracle @GT | 94.4 | 94.4 | — |
| v4 seq oracle | 85.0 | 75.0 | — |
| v4 deployed head | 53.3 | 60.0 | 50.0 |
| v4 shuffled symbols | 58.3 | 63.3 | — |
Table 7. Seed variance for the REALCOUNT claims (; chance 33.3%; v4 ). The deployed head’s head-timeline score is identical across seeds; the shuffled control and the GT-timeline reader vary; v4 retention varies. The full-145 and sequence-disjoint oracles were not repeated for seed 44.
Cross-timeline cells. The 2-epoch deployed-head reader reaches 61.1% on head timelines and 77.8% on GT timelines (seed 42); the sequence-disjoint oracle reader reaches 66.7% on head timelines (94.4% on GT). A head-trained reader does not beat the GT-trained reader on head timelines (61.1 vs. 66.7), so noise-matched training does not close the gap at this scale. The deployed head’s head-timeline score does not exceed the shuffled-symbol control (61.1% in seed 42, 66.7% in seed 43); the residual gap is detector evidence, not reader capacity.
Seed variance. Table 7 repeats the key claims across seeds. The deployed head is 61.1% (11/18) on head timelines in all three seeds, so the negative deployment result is not seed noise. The content-free shuffled control is 61.1% in seed 42 and 66.7% in seed 43, so the deployed head never exceeds it and in seed 43 sits one clip below it, which strengthens rather than weakens the negative deployment conclusion. The GT-timeline score varies from 72.2% to 88.9%, so the reader’s ability to use clean timelines is robust in direction but variable in size. The full-145 oracle is 100% in both seeds tested (v4 73.3/85.0%); the sequence-disjoint oracle is 94.4% in both seeds tested; its v4 retention is 75–85%, and the deployed head’s v4 retention is 50–60%. These ranges should replace single-seed point estimates for the secondary retention claims.
What the head loses. To attribute the 33.3-point gap, we evaluate the same sequence-disjoint reader on controlled timeline variants (; all patches copy GT symbols and are oracle interventions, not deployable results). Randomly flipping 10% of GT slots drops accuracy from 94.4% to 66.7%, and flipping 40% drops it to 38.9%; the reader is brittle to generic slot noise. The errors are not uniform, however. Filling every absent slot (removing gaps) leaves at 6/6 but drops from 5/5 to 0/5; removing transitions leaves at 5/5 but drops from 5/6 to 3/6. The deployed head loses both: 5/18 clips contain no absent slot at all (mean predicted absences 2.5 vs. 6.6 in GT; boundary F1 0.38). Repairing only absent slots raises accuracy from 12/18 to 14/18, entirely in (3/5 → 5/5) and without breaking any clip. Repairing only boundary windows leaves the total at 12/18: it fixes two clips but breaks two clips, because transitions are added while the absence structure is still missing. Repairing both reaches 17/18 (94.4%), matching the GT-timeline ceiling. The remaining error is a different clip from the one the GT timeline misses, so this is not an exact recovery of GT but a complementary 17/18. These oracle repairs bound what better event evidence could buy; the deployed head remains at 61.1%.
Additional Adaptation Attempts
Beyond tbl:4 and Table 6, we tested two further detector-side levers on REALCOUNT.
Scope. The detector-side experiments below predate the prompt-corrected retrain; their end-to-end comparisons use the pre-clean 72.2% head–reader pair and are reported for completeness, not as current deployment estimates. The clean retrain’s deployed pair is 61.1%.
Presence-head redesign. Temporal-difference features (adjacent-slot and absolute differences), ±2-slot context, a small temporal convolutional network, and an absence-heavy expansion (631 windows with ≥ 20% absence added to the 1,000 training windows) improved edge recall from 25% to 32–41%, but absence recall stayed at 21–29% and the count proxy at 7–11 of 18 clips. With the reader fixed, end-to-end accuracy ranged from 39% to 78% across variants and seeds, with no reliable gain over the pre-clean 72.2% deployed pair, because the injected timeline distribution shifted away from the reader’s training distribution.
Head-specific encoder adaptation. We fine-tuned a separate copy of VideoMAE (the reader’s connector was left untouched), unfreezing its last 2 or 4 blocks jointly with the head under a two-stage schedule (head frozen for 300 steps, then joint fine-tuning; best checkpoint by held-out slot/edge/count score). This raised slot accuracy from 61.4% to 62.1–62.9% and the count proxy to 11 of 18, but the original reader fell to 61–67% on the shifted timelines. Retraining the reader for two epochs on the adapted timelines reaches 72.2% on REALCOUNT—the same as the pre-clean head–reader pair; the clean retrain’s deployed pair is 61.1%—while synthetic retention drops from 56.7% to 41.7%.
Reading. Detector-side changes at this scale either do not add count-carrying content to the timeline or require a co-adapted reader, and the co-adaptation costs synthetic retention. Neither closes the gap to the 94.4% sequence-disjoint oracle result. This is consistent with the boundary probe and with the view that the remaining bottleneck is the evidence recovered by the frozen visual encoder, not the capacity of the presence head or the adapter.
Artifact Index
The anonymized package contains the data factories (make_count_v4.py, make_count_v1x.py, make_realcount_v1.py, make_realcount_train2.py), training entry points (train_m2.py; LoRA adapter src/lora.py; paired-mix builder make_peft_mix.py), evaluation and probe scripts (eval_motion_qa.py, eval_count_loc.py, eval_realcount_loc.py, probe_realcount*.py, bootstrap_ci.py), the detector (train_presence_head_v2.py), the released presence-head weights, a dependency-free verifier (verify_results.py), and per-run records (the exact argument dictionary plus records.jsonl) for every table and figure; paper_figs.py regenerates Figure 1–Figure 4. The repository will be anonymized and archived with a permanent DOI upon acceptance.

Figure 1. Diagnosis panel. (a) The boundary re-measured on balanced v4 (): every system sits at 1.0–1.5× chance and below the old eval’s constant-answer baseline of 60. (b) Error structure of the prior checkpoint on v4: predicted counts never include 3 (0/60); clips are answered “2” in 18/20 cases. (c) Linear probes on the connector’s token stream: local presence decodes at 93.3% while global count decoding (72.3%) barely exceeds a duration-only proxy (67%); permutation control 50%. (d) Granularity: the 0.4–0.7 s gaps between episodes are resolved only at ≤1 s windows; the deployed 4 s virtual rows cannot separate episodes.
References
- [1]Alain, G. and Bengio, Y. Understanding intermediate layers using linear classifier probes, 2016.arxiv.org/abs/1610.01644
- [2]Baillargeon, R. Object permanence in 3 1/2- and 4 1/2-month-old infants. Developmental Psychology, 23(5): 655–664, 1987. doi: 10.1037/0012-1649.23.5.655.
- [3]Baskar, S., Cai, Z., Shabihi, S., Satheesh, A., Islam, M. R., Sehwag, U. M., Goldstein, T., and Huang, F. The low frequency trap: Video language models fail at simple event bookkeeping, 2026.arxiv.org/abs/2608.06361
- [4]Chapus, M., Crispim-Junior, C., Eglin, V., and Baskurt, A. CountVideoBench: A benchmark to evaluate prediction biases on the object counting task by video-language models, 2026.
- [5]Chen, B., Shao, M., Garg, S., Karri, R., and Shafique, M. GroundCount: Grounding vision-language models with object detection for mitigating counting hallucinations, 2026a.
- [6]Chen, J., Meng, S., Chen, Y., Zhao, M., Gui, W., and Guo, X. TOC-Bench: A temporal object consistency benchmark for video large language models, 2026b.
- [7]Chen, J., Meng, S., and Guo, X. Persistent object narratives for token-efficient video language models, 2026c.arxiv.org/abs/2608.04866
- [8]Cui, S., Meng, L., Luo, Y., and He, P. VidNum: Diagnosing VLM failure modes in video-grounded numerical reasoning, 2026.
- [9]Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., and Zisserman, A. Counting out time: Class agnostic video repetition counting in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10387–10396, 2020.arxiv.org/abs/2006.15418
- [10]Fang, P., Chen, Y., and Cai, X. MarkIt: Training-free visual markers for precise video temporal grounding, 2026.arxiv.org/abs/2604.25886
- [11]Farazi, M., Ciftler, B., Dandoush, A., and Bendraou, R. CrowdCue: Specialist-cue conditioning for vision-language crowd counting, 2026.arxiv.org/abs/2609.23012
- [13]Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., and Neubig, G. PAL: Program-aided language models, 2023.arxiv.org/abs/2211.10435
- [14]Guo, Y., Liu, J., Li, M., Liu, Q., Chen, X., and Tang, X. TRACE: Temporal grounding video LLM via causal event modeling, 2024.arxiv.org/abs/2410.05643
- [16]Huang, B., Wang, X., Chen, H., Song, Z., and Zhu, W. VTimeLLM: Empower LLM to grasp video moments, 2023.arxiv.org/abs/2311.18445
- [17]Jia, Y., Geng, H., Yang, W., and Zhang, M. Give the model a finger: Developmentally aligned evaluation of counting in vision-language models, 2026.
- [18]Khoshnoodi, M. and Bargal, S. A. RepProbe: Can video language models count repeated actions? a diagnostic benchmark, 2026.arxiv.org/abs/2610.18844
- [20]Liu, P., Shi, Z., Hao, H., Fu, Q., Bi, X., Zhang, S., Hu, X., Wang, Z., Huang, L., and Liu, S. SVCBench: A streaming video counting benchmark for spatial-temporal state maintenance, 2026a.arxiv.org/abs/2603.12703
- [21]Liu, X., Zhu, Y., Zhao, X., Wang, X., Li, Y., Li, X., Cao, H., Sun, X., Zhang, S., Yang, X., Zhong, Z., and Yang, X. Moment-Video: Diagnosing temporal fidelity of video mlms on momentary visual events, 2026b.arxiv.org/abs/2606.02522
- [22]Liu, Y., Li, S., Liu, Y., Wang, Y., Ren, S., Li, L., et al. Tem-pCompass: Do video LLMs really understand videos?, 2024.DOI
- [23]Qian, L., Li, J., Wu, Y., Ye, Y., Fei, H., Chua, T.-S., et al. Momentor: Advancing video large language model with fine-grained temporal reasoning, 2024.arxiv.org/abs/2402.11435
- [24]Ren, S., Yao, L., Li, S., Sun, X., and Hou, L. TimeChat: A time-sensitive multimodal large language model for long video understanding, 2023.arxiv.org/abs/2312.02051
- [25]Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., and Scialom, T. Toolformer: Language models can teach themselves to use tools, 2023.arxiv.org/abs/2302.04761
- [26]Sinha, S., Stergiou, A., and Damen, D. Every shot counts: Using exemplars for repetition counting in videos, 2024.arxiv.org/abs/2403.18074
- [27]Surís, D., Menon, S., and Vondrick, C. ViperGPT: Visual inference via python execution for reasoning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.arxiv.org/abs/2303.08128
- [28]Tong, Z., Song, Y., Wang, J., and Wang, L. VideoMAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training. In Advances in Neural Information Processing Systems (NeurIPS), 2022.arxiv.org/abs/2203.12602
- [29]Tsuchiya, F., Miyanishi, T., Yasuki, S., Ukai, M., Inoue, N., Kurita, S., Iwasawa, Y., and Matsuo, Y. Diagnosing long-video quantitative reasoning in multimodal LLMs via enumeration and counting, 2026.arxiv.org/abs/2603.29943
- [30]Wang, H., Xu, Z., Cheng, Y., Diao, S., Zhou, Y., Cao, Y., et al. Grounded-VideoLLM: Sharpening fine-grained temporal grounding in video large language models, 2024a.arxiv.org/abs/2410.03290
- [31]Wang, X., Zhang, Y., Zohar, O., and Yeung-Levy, S. VideoAgent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision (ECCV), 2024b. doi: 10.1007/978-3-031-72989-8_4.DOI
- [35]Zacks, J. M. and Swallow, K. M. Event segmentation. Current Directions in Psychological Science, 16(2):80–84, 2007. doi: 10.1111/j.1467-8721.2007.00480.x.DOI
- [36]Zhang, C., Lu, T., Islam, M. M., Wang, Z., Yu, S., Bansal, M., et al. A simple LLM framework for long-range video question-answering, 2023.arxiv.org/abs/2312.17235