OmniGround: Strengthening Omni-Modal Temporal Grounding via Attention-Consistency Training

Abstract

Omni-modal large language models can jointly understand video, audio, and text, yet they remain weak at temporal grounding, a capability essential for real-world audio-visual applications. To understand this limitation, we look beyond final predictions and examine the model’s internal temporal evidence. We identify a temporal generation gap: the internal attention of the model localizes the target moment more accurately than the timestamps it generates. To close this gap, we propose TACG (Temporal Attention-Consistency Grounding), a post-training method that treats internal attention as a reliable temporal pivot and aligns it with both the model prediction and the ground-truth annotation through attention-consistency training. We further build a 580K-sample training dataset by reconstructing temporal QA from dense audio-visual annotations and organizing general omni-modal instruction data. Using TACG and this training dataset, we train OmniGround, a general-purpose omni-modal model with stronger temporal grounding capability, without injecting any additional temporal cues. OmniGround reaches state-of-the-art omni-modal temporal grounding, including 37.5 mIoU on LongVALE and 35.7 mIoU on UnAV-100, while staying competitive on both general audio-visual understanding and vision-only grounding.

Introduction

Omni-modal large language models ([19] (Xu et al. 2025a; Wu et al. 2024)), which jointly perceive video, audio, and text within a single backbone, have advanced rapidly and now excel at a wide range of multimodal understanding tasks. As these models move toward real-world audio-visual applications such as video editing, content moderation, and multimodal retrieval, recognizing what happens no longer suffices: a model must also pinpoint when it happens. This temporal capability is critical for practical audio-visual applications, with temporal video grounding (TVG) serving as a central benchmark. Yet current omni-modal models still perform poorly on TVG despite their strong general understanding.

Existing temporal grounding methods mainly follow two directions. One family modifies the model or its inputs by injecting explicit temporal information, such as special temporal tokens, frame markers, or task-specific heads ([24] (Guo et al. 2025a; Wu et al. 2025)). Although effective for localization, these designs make temporal grounding depend on customized input formats and specialized output parsing, which limits their compatibility with a general-purpose omni-modal interface. A second family improves grounding purely through post-training, such as reinforcement learning ([30]) or distillation from a stronger teacher (Li et al. 2026a). This training-only direction preserves the inference interface and better maintains general ability, but does not directly exploit the model’s own temporal evidence. We adopt this philosophy, but push it further.

We begin with a diagnostic question: when an omni-modal model grounds imprecisely, does it fail to localize the event, or merely to express the time? For each query, we compare the interval read from the model’s internal attention with its generated timestamp, and measure which lands closer to the ground truth. Across multiple omni-modal models and grounding datasets, the attention-derived interval is more accurate than the prediction on a clear majority of samples, up to 78.3%. We call this the temporal generation gap: the model knows where to look but cannot accurately say when. This pattern holds for both base models and models with stronger grounding ability, indicating that the gap is not merely a weak-model artifact, as illustrated in Figure 1. If left unaddressed, this gap limits further improvements in temporal localization precision.

Building on this diagnosis, we propose Temporal Attention-Consistency Grounding (TACG). Its core term aligns the generated timestamp with the model’s own internal attention, using the more reliable internal signal to pull the prediction onto the target and directly close the temporal generation gap. Two auxiliary terms anchor this correction to the ground truth: one supervises the attention directly against the annotation, making the attention itself localize the target more accurately; the other supervises the prediction by its temporal distance to the annotation, supplying the distance-aware gradient that cross-entropy lacks. TACG achieves stronger temporal grounding than standard supervised fine-tuning on the same data, without introducing additional temporal inputs or model parameters.

We also build a 580k-sample omni-modal training dataset that supports both temporal grounding and general understanding. Starting from the dense audio-visual annotations in AVID ([3]) and LongVALE ([7]), we extract high-quality temporal segments and reformulate them into grounding, dense captioning, and temporal question-answering tasks. We further incorporate general omni-modal instruction data from several sources [39, 19, 35] (Ma et al. 2025) to preserve broad audio-visual understanding. Using this dataset and TACG, we train OmniGround from Qwen2.5-Omni (Xu et al. 2025a). As shown in Figure 1, OmniGround improves temporal grounding without the extra temporal inputs or specialized output formats used by prior baselines. We evaluate OmniGround on seven benchmarks, including three omni-modal grounding benchmarks, two vision-only grounding benchmarks, and two general omni-modal understanding benchmarks. OmniGround sets a new state of the art on omni-modal temporal grounding while staying competitive on the other two categories. Our contributions are summarized as follows:

• We identify the temporal generation gap in omni-modal language models and propose TACG, a post-training objective that aligns the model’s prediction with its internal attention, consistently improving over standard supervised fine-tuning under identical training data.

• We build a 580k omni-modal temporal-awareness dataset by reformulating dense audio-visual annotations into temporal grounding and question-answering tasks, applying quality filtering, and balancing the resulting data with curated general data. We train OmniGround on this dataset.

• OmniGround reaches state-of-the-art omni-modal temporal grounding and stays competitive on general omni-modal understanding and vision-only grounding. We will release the dataset, model, and training framework.

Related Work

Omni-Modal Language Models

Recent work extends large language models beyond text to perceive multiple modalities. Early video LLMs such as VideoChat (Li et al. 2023), Video-ChatGPT [21], and VideoLLaMA [37] couple a visual encoder with an LLM, while PandaGPT (Su et al. 2023) and NExT-GPT (Wu et al. 2024) add audio. More recent omni-modal models further unify audio, vision, and text within a shared modeling framework, enabling general audio-visual-text understanding [19] (Xu et al. 2025a). While these models have achieved strong general audio-visual understanding, they are not explicitly optimized for precise temporal grounding. As a result, they often recognize the relevant event but fail to generate accurate temporal boundaries. Our work strengthens temporal grounding in a general-purpose omni-modal model.

Model Mechanism Analysis

Prior studies show that model behavior can be analyzed through internal signals [40, 23]. Inspired by this perspective, we inspect internal attention in omni-modal temporal grounding.

Temporal Video Grounding

Temporal video grounding (TVG) localizes the moment described by a natural language query, and existing methods differ in how they acquire this ability. One family injects explicit temporal machinery—regression heads in specialist architectures [38, 18, 22] (Jiang et al. 2023), or, in video LLMs, time-aware encoders [26], special temporal tokens [28, 2] (Guo et al. 2025a), frame-index markers (Wu et al. 2025), and localization heads [24, 13, 29, 36, 11] (Guo et al. 2025b; Li et al. 2026b; Qu et al. 2024). These components depart from the standard backbone, add inference cost, and often weaken general question answering. A second family leaves the architecture and inference untouched and strengthens temporal grounding through additional post-training stages, such as reinforcement learning with IoU rewards [30] or distillation from a stronger teacher (Li et al. 2026a). Our method instead strengthens fine-tuning itself, turning the model’s internal attention into a self-supervision signal.

Preliminary: Temporal Generation Gap

We validate the temporal generation gap across multiple omni-modal models on AVID and LongVALE, including both base models and models with stronger grounding ability. For each benchmark, we construct the diagnostic evaluation split by randomly sampling 500 examples from its official test split and use this fixed subset for all preliminary-analysis statistics. For each sample we obtain three temporal intervals and compare them against the ground truth by temporal IoU. The ground-truth interval $I_{\mathrm{gt}}$ and the generated interval $I_{\mathrm{pred}}$ are taken directly. We extract the attention interval $I_{\mathrm{att}}$ from the attention of the last prefill token to the audio-visual tokens at a mid-to-late layer of the model. This attention is aggregated over spatial positions into a temporal distribution, from which we select the shortest contiguous window covering 50% of the cumulative mass. This gives three intervals for each case, $I_{\mathrm{gt}}$, $I_{\mathrm{pred}}$, and $I_{\mathrm{att}}$; we compare IoU($I_{\mathrm{att}}, I_{\mathrm{gt}}$) with IoU($I_{\mathrm{pred}}, I_{\mathrm{gt}}$) to determine whether the attention interval or the generated prediction is more accurate, and use the same intervals for visualization. For Qwen2.5-Omni-3B, we use layer 27; additional details, including layer choices for other models, layer sensitivity, cumulative-mass threshold sensitivity, and the SFT setup, are provided in the Appendix.

Figure 2 visualizes the internal attention concentrating on the target region while the generated interval is displaced, and Table 1 shows that $I_{\mathrm{att}}$ overlaps the ground truth better than $I_{\mathrm{pred}}$ on most valid samples across five omni-modal models and two datasets, despite large differences in grounding mIoU. Together, these results establish that the temporal generation gap is consistent across models and datasets. After SFT, both the attention interval and the generated prediction become more accurate, yet the instance-level gap remains, suggesting that unresolved attention–prediction misalignment limits further improvements in temporal localization precision. This motivates directly aligning attention, prediction, and ground truth in the training objective introduced next.

Model	AVID		LongVALE
	Attn. wins†	mIoU	Attn. wins†	mIoU
Qwen2.5-Omni-3B	73.9%	7.4	78.3%	6.3
Qwen2.5-Omni-3B (SFT)	57.7%	17.9	60.0%	20.9
Qwen2.5-Omni-7B	75.7%	6.2	72.1%	12.1
Qwen3-Omni	65.0%	14.1	69.7%	10.6
OLA	73.0%	10.3	70.3%	7.7

Method: OmniGround

We present OmniGround, an omni-modal temporal grounding model built on Qwen2.5-Omni. We train it with TACG on our 580k-sample omni-modal training dataset. The framework is illustrated in Figure 3.

Temporal Attention-Consistency Grounding

The preliminary analysis motivates supervising the generated timestamp with the model’s internal attention while anchoring both signals to the ground truth. Although temporal IoU measures their agreement at evaluation time, it is unsuitable as a training signal: it is non-differentiable and least informative exactly when most needed. Once two intervals are disjoint, its gradient vanishes everywhere, giving no indication of which direction, or how far, to move the prediction—precisely the regime that dominates early training. We therefore recast attention, prediction, and ground truth as continuous temporal distributions over the $T$ time steps and enforce their mutual consistency with divergences that stay informative under non-overlap (Jensen–Shannon and Wasserstein-1, chosen per edge below).

Attention–prediction consistency: the core objective. The central constraint of TACG forces the generated timestamp to agree with the localization already present in the internal attention. We implement this constraint by expressing both quantities as temporal distributions over the $T$ time steps.

We first extract the implicit localization signal from the attention at a selected intermediate layer $l$. We compute the attention from the last prefill token to all media tokens (both video and audio) and aggregate the mass that falls on each of the $T$ interleaved time steps:

where $q$ is the query of the last token, $k_i$ ranges over all video and audio tokens, and $\mathcal{C}_t$ is the set of media tokens at time step $t$. As the attention is normalized over all media tokens, $p_{\mathrm{att}}$ is automatically a valid distribution over the $T$ steps, and gradients flow through the layer-$l$ query and key projections, allowing the consistency loss to refine the attention itself.

We then convert the predicted timestamp into a continuous temporal distribution. Since standard supervised fine-tuning never decodes the logits, we operate on them directly. At each digit position of the timestamp tokens we form the soft expectation over digit values, round it, apply a straight-through estimator so that gradients still reach the logits, and assemble the start and end times by place value:

where $D = \{0,\ldots,9\}$ are the digit token indices, $\lfloor\cdot\rceil$ rounds to the nearest integer, $\operatorname{sg}(\cdot)$ is the stop-gradient (detach) operator, $p_i$ is the (signed) place value of digit $i$, and $\tilde{d}_i^s,\tilde{d}_i^e$ are the digits of the start and end timestamps. The second line is the straight-through estimator: the forward pass uses the rounded digit $\bar{d}_i$ (the two soft terms cancel), while the backward pass routes gradients through the soft expectation $\hat{d}_i$ to the logits. We map the reconstructed interval $[\hat{t}_s,\hat{t}_e]$, normalized by the video duration, to a continuous distribution $p_{\mathrm{pred}}$ through a Gaussian projection: a Gaussian centered at the interval center $\hat{\mu}_t$ with standard deviation $\sigma= 0.15 \cdot(\hat{t}_e - \hat{t}_s)$ proportional to the predicted width. This projection is a smoothing operator, it places the prediction in the same distributional space as the attention so the two can be compared directly, and gives the interval smooth support so that its divergence from other distributions varies continuously with the temporal offset rather than vanishing once they no longer overlap. With both distributions in place, the core objective is their Wasserstein-1 distance, $\mathcal{L}_{\mathrm{tp}} = W_1(p_{\mathrm{att}}, p_{\mathrm{pred}})$. This attention–prediction edge transfers the reliable localization already present in the internal attention onto the generated timestamps, directly closing the temporal generation gap. The divergence on each edge (Jensen–Shannon or Wasserstein-1) is chosen by ablation (Appendix).

Ground-truth anchoring: two auxiliary constraints. The self-consistency edge alone does not tie either quantity to the true interval, and could in principle drive the two into mutual agreement away from it. Two auxiliary edges therefore anchor the transfer to the ground truth. We apply the same Gaussian projection, with identical parameters, to the ground-truth interval $[t_s^*,t_e^*]$, obtaining a distribution $p_{\mathrm{gt}}$ that is detached from the computation graph and serves as the fixed supervision target. The attention–ground-truth edge keeps the internal signal aligned with the true segment, so that it remains a trustworthy target for the core edge. The prediction–ground-truth edge supplies the distance-aware supervision that cross-entropy lacks:

Full objective. The three edges form a triangle of consistency constraints. With edge weights $\lambda_{\mathrm{tp}}$, $\lambda_{\mathrm{tg}}$, and $\lambda_{\mathrm{pg}}$, the full training objective is

Data Construction and Training Strategy

After Quality Filtering removes low-resolution videos and overly short clips (grounding videos under 30 s and general clips under 2 s), we assemble the general understanding data from four balanced sources: 121k LLaVA-Video [39] samples re-annotated by OLA [19] from speech and visual perspectives, 37k AVQA [35] samples for scene sound, 32k FortisAVQA (Ma et al. 2025) samples for music, and 200k original LLaVA-Video samples paired with audio for general video proficiency. For grounding data, we use only the training-split videos and dense temporal captions of AVID [3] and LongVALE [7]. We reformat the training captions into three complementary tasks—Temporal Video Grounding (event → time), Reverse Grounding (time → event), and Dense Video Captioning (video→timed events). We then apply Audio-Visual Relevance Filtering, using Gemini 3.1 Pro to remove QA pairs whose text does not require audio-visual evidence. A final Balanced Mixing step combines the two streams, yielding 190k grounding samples (126k TVG, 50k Reverse Grounding, and 14k DVC) and 390k general samples, for a 580k-sample dataset at a grounding-to-general ratio of roughly 1:2; the composition and pipeline are summarized in Figure 4 and detailed in the Appendix.

We apply TACG only to TVG samples because it needs an explicit predicted time interval to construct $p_{\mathrm{pred}}$, which is available only when the target output is a start–end timestamp. All samples are trained jointly: TVG samples use the full objective $\mathcal{L}_{\mathrm{total}}$, while Reverse Grounding, DVC, and general QA samples use cross-entropy $\mathcal{L}_{\mathrm{CE}}$ only. Implementation details are provided in the Appendix.

Experiments

Experimental Setup

All experiments are conducted on 32 NVIDIA A100 GPUs. Detailed configurations are provided in the Appendix.

Evaluation Benchmarks. We evaluate on seven benchmarks: AVID [3], LongVALE [7], and UnAV-100 [6] for omni-modal grounding, Charades-STA [5] and ActivityNet-Captions [1] for vision-only grounding, and WorldSense [12] and Daily-Omni [41] for general omni-modal understanding. For UnAV-100, we convert its temporal annotations into QA pairs for large-model evaluation, with details provided in the Appendix.

Baselines. We compare against 17 baselines in two groups: vision-only models, including VideoChat (Li et al. 2023), VTimeLLM [13], TimeChat [26], HawkEye [29], VTG-LLM (Guo et al. 2025a), Grounded-VideoLLM [28], TimeSuite [36], NumPro (Wu et al. 2025), ED-VTG [24], and Time-R1 [30]; and omni-modal models, including VideoLLaMA2 [4], OLA [19], Qwen2.5-Omni (Xu et al. 2025a), Qwen3-Omni (Xu et al. 2025b), LongVALE [7], ChronusOmni [2], and the closed-source Gemini 3.1 Pro [8], which we report for reference only.

Temporal Grounding Performance

Table 3 and Table 2 report temporal grounding results across the five grounding benchmarks. We highlight three observations.

Method	[email protected]	[email protected]	[email protected]	mIoU
VideoLLaMA2	13.3	7.0	2.3	11.5
OLA	13.6	6.6	2.0	13.1
Qwen2.5-Omni (3B)	20.7	11.8	5.5	15.2
Qwen2.5-Omni (7B)	25.9	16.2	6.7	20.1
Qwen3-Omni	26.5	17.5	8.0	21.8
LongVALE	11.3	6.1	3.2	15.0
ChronusOmni	47.3	33.8	20.7	34.1
Gemini 3.1 Pro*	61.6	47.6	28.3	44.1
OmniGround(3B)†	36.0	21.6	10.9	25.9
OmniGround(7B)†	42.3	32.7	19.1	32.2
OmniGround(3B)	40.1	24.0	13.3	28.9
OmniGround(7B)	46.1	34.2	22.2	35.7

Method	Omni-modal Grounding: AVID-TVG [email protected]	[email protected]	[email protected]	mIoU	LongVALE-TVG [email protected]	[email protected]	[email protected]	mIoU	Vision-only Grounding: Charades-STA [email protected]	[email protected]	[email protected]	mIoU	ActivityNet-Captions [email protected]	[email protected]	[email protected]	mIoU
Vision-only Models
VideoChat	7.6	5.2	2.5	5.2	2.2	0.9	0.4	3.0	32.8	8.6	0.0	25.9	23.5	12.6	6.0	17.4
VTimeLLM	9.2	4.1	1.0	7.4	9.1	4.3	1.9	7.9	52.0	27.7	11.1	31.5	44.0	27.8	14.3	30.4
TimeChat	9.8	6.9	3.0	7.8	7.2	3.4	1.6	6.4	47.8	31.6	14.6	31.6	30.2	16.9	8.2	21.8
HawkEye	7.5	4.1	1.7	5.2	15.3	7.8	2.8	10.9	50.6	31.4	14.5	33.7	49.1	29.3	10.7	32.7
VTG-LLM	9.0	6.4	2.9	5.9	6.2	2.8	1.0	4.2	54.0	40.0	22.0	38.1	11.3	6.8	3.0	8.0
Grounded-VideoLLM	12.0	8.5	5.6	9.6	20.2	12.8	7.3	15.8	54.2	36.4	19.7	36.8	46.2	30.3	19.0	36.1
TimeSuite	1.9	0.3	0.1	2.6	17.2	8.7	4.1	11.8	69.9	48.7	24.0	45.5	31.0	16.9	9.8	23.6
NumPro	5.2	2.8	0.8	4.8	10.4	4.7	2.6	8.1	63.8	42.0	20.6	41.4	55.6	37.5	20.6	38.8
ED-VTG	–	–	–	–	–	–	–	–	59.5	39.3	19.8	40.2	52.1	33.1	16.0	35.2
Time-R1	5.7	1.9	0.8	5.5	28.4	15.4	7.8	20.7	78.3	61.2	34.9	53.7	58.1	39.0	21.4	40.5
Omni-modal Models
VideoLLaMA2	5.1	1.6	0.9	3.3	3.3	1.5	0.0	2.4	19.4	11.5	5.0	12.9	13.0	7.1	2.8	10.2
OLA	14.9	9.9	5.3	10.1	9.6	4.2	1.9	7.5	32.2	10.9	3.2	20.0	27.9	15.6	7.0	20.6
Qwen2.5-Omni (3B)	11.5	7.0	2.9	8.4	8.6	4.6	2.0	6.1	47.4	30.9	14.4	32.2	27.5	15.7	8.5	19.9
Qwen2.5-Omni (7B)	8.4	3.9	1.2	6.0	17.9	9.2	3.8	12.3	65.5	45.9	23.3	43.7	41.5	26.0	14.6	29.7
Qwen3-Omni	18.2	12.3	5.2	14.4	14.4	7.7	3.4	10.4	35.4	23.4	10.5	23.3	37.6	23.9	12.6	26.9
LongVALE	4.8	2.4	1.1	4.5	15.7	8.6	3.9	11.0	47.9	19.8	9.7	32.6	46.3	28.5	14.2	31.6
ChronusOmni	42.3	33.7	21.9	31.2	49.7	32.5	17.6	34.5	72.2	47.0	20.8	46.0	56.0	35.5	20.1	39.4
Gemini 3.1 Pro⋆	55.0	50.5	44.7	47.0	66.2	49.0	31.0	46.6	74.6	44.4	15.6	44.2	64.1	46.6	28.7	46.9
OmniGround(3B)†	67.3	61.2	48.1	55.3	40.2	27.3	15.1	27.7	50.2	20.5	8.4	32.6	31.0	19.2	11.2	23.5
OmniGround(7B)†	70.3	67.6	55.7	60.4	46.2	34.5	20.2	32.7	67.2	38.5	19.7	44.6	50.4	36.5	19.2	38.3
OmniGround(3B)	68.3	62.2	50.5	56.9	41.4	28.6	16.1	28.6	60.5	31.2	15.1	38.8	31.7	19.6	11.1	23.7
OmniGround(7B)	75.2	72.1	64.9	65.9	51.9	39.5	25.1	37.5	74.5	52.6	27.7	49.3	56.4	39.1	21.1	41.2

Omni-modal temporal grounding. As shown in Table 3, OmniGround achieves the best open-model performance on the audio-visual AVID and LongVALE benchmarks. The 7B model reaches 65.9 mIoU on AVID and 37.5 mIoU on LongVALE, while the 3B model already surpasses all open baselines on AVID. Since our grounding data is reconstructed from the training-split source videos of AVID and LongVALE, we further evaluate on UnAV-100 to isolate out-of-domain audio-visual grounding performance. As shown in Table 2, OmniGround reaches 35.7 mIoU with the 7B model and outperforms evaluated open baselines at the stricter [email protected] and [email protected] thresholds. Together, these results show that OmniGround sets a new open-model state of the art in omni-modal temporal grounding.

Vision-only temporal grounding. The benefit of TACG is not limited to the audio-visual setting. OmniGround never sees any vision-only grounding data during training, and Charades-STA and ActivityNet-Captions are outside the audio-visual grounding domains used for our data construction. Nevertheless, OmniGround remains highly competitive on these benchmarks, essentially matching specialized vision-only methods, trailing only the vision-only specialist Time-R1, and leading all other omni-modal models. This indicates that aligning predictions with internal attention strengthens temporal grounding in general, rather than only fitting the omni-modal training distribution.

TACG versus standard SFT. At full scale, we train otherwise identical 3B and 7B models with standard cross-entropy on the same 580k-sample dataset (OmniGround†); TACG improves grounding on nearly every benchmark at both scales. These full-scale margins are smaller than the clean gain measured in the controlled ablation (Table 5), and by design: the TACG objective activates only on the forward-TVG samples—about one-fifth of the 580k-sample mixture—so its effect is diluted at the full-model level. Even under this dilution, the improvement is consistent across the evaluated benchmarks, and on the never-seen vision-only Charades-STA it is pronounced at both scales (+6.2 mIoU at 3B), showing that aligning predictions with the internal attention instills a transferable localization skill rather than overfitting a particular distribution. Figure 5 makes this alignment visible on AVID: for both OmniGround scales, the predicted intervals become better aligned with attention and move closer to the ground-truth intervals, explaining the stronger localization performance and confirming that TACG closes the temporal generation gap it was designed to target.

Configuration	AVID [email protected]	AVID mIoU	LongVALE [email protected]	LongVALE mIoU	Δ%
Baseline (CE only)	0.3502	0.3153	0.1408	0.1604	0.00
$\mathcal{L}_{\mathrm{tg}}$ only	0.3543	0.3223	0.1563	0.1722	+4.79
$\mathcal{L}_{\mathrm{pg}}$ only	0.3649	0.3282	0.1479	0.1645	+3.32
$\mathcal{L}_{\mathrm{tp}}$ only	0.3671	0.3315	0.1514	0.1663	+4.41
$\mathcal{L}_{\mathrm{tg}}+\mathcal{L}_{\mathrm{pg}}$	0.3671	0.3301	0.1508	0.1672	+4.47
Full TACG	0.3814	0.3408	0.1546	0.1759	+8.87

General Understanding Performance

Table 4 shows that OmniGround preserves strong general audio-visual understanding while improving temporal grounding. The 7B model remains competitive on World-Sense and achieves the best open-model result on Daily-Omni, indicating that grounding-oriented training does not come at the cost of general comprehension. Relative to the same-scale cross-entropy baselines, TACG keeps World-Sense stable and further improves Daily-Omni, suggesting that sharper temporal alignment can also benefit tasks involving event order and audio-visual correspondence. By contrast, the grounding-specialized LongVALE performs poorly on these general benchmarks, highlighting the advantage of OmniGround’s balanced training.

Method	WorldSense Acc	Daily-Omni Acc
VideoLLaMA2	27.8	35.8
OLA	34.0	54.3
Qwen2.5-Omni (3B)	46.0	62.3
Qwen2.5-Omni (7B)	46.1	62.7
Qwen3-Omni	52.0	68.8
LongVALE	9.9	24.4
ChronusOmni	49.5	70.2
Gemini 3.1 Pro⋆	63.1	81.0
OmniGround (3B)†	48.9	65.4
OmniGround (7B)†	48.3	68.7
OmniGround (3B)	48.6	68.1
OmniGround (7B)	50.2	71.9

Ablation Study

We conduct ablation experiments to analyze the key design choices in TACG. All ablations are trained on the same set of

10k TVG samples, with an SFT model trained on identical data as the baseline, so the comparison isolates the gains brought by TACG rather than data differences.

Component Ablation

We evaluate the contribution of each consistency edge in Eq. (4) ($\mathcal{L}_{\mathrm{tg}}$, $\mathcal{L}_{\mathrm{pg}}$, and $\mathcal{L}_{\mathrm{tp}}$). Each configuration is reported under its individually optimized weight, so the comparison reflects each edge’s best achievable effect.

As shown in Table 5, each single edge improves over the SFT baseline under its own optimal weight (+3.32% to +4.79%), indicating that all three consistency signals are useful on their own. Stacking the two ground-truth-anchored edges ($\mathcal{L}_{\mathrm{tg}}+\mathcal{L}_{\mathrm{pg}}$) yields only +4.47%—essentially no gain over either edge alone. In sharp contrast, completing the triangle with the attention-to-prediction edge $\mathcal{L}_{\mathrm{tp}}$ lifts the gain to +8.87%, nearly doubling the best two-edge result. This identifies attention–prediction consistency as the core component of TACG: the largest gain appears precisely when the objective addresses the temporal generation gap diagnosed in our preliminary analysis.

Discretization of the Predicted Distribution

Constructing the predicted distribution requires mapping the per-position digit logits to a digit value $d_i$, which is then assembled into a timestamp by place value. Two conventional choices both fail for timestamp regression: Argmax keeps only the most probable digit, so it cannot distinguish a nearly-correct distribution from a confidently-wrong one and its selected digit jumps discontinuously between steps, yielding unstable gradients; Soft uses the continuous expectation for smooth gradients, but the expectation of a discrete distribution rarely lands on an integer, leaving a loss floor that, amplified by place value, biases the gradient at high-order positions.

We therefore adopt Round: we first take the soft expectation and then round it to the nearest digit. This combines smooth gradients with discrete semantics, avoiding both Argmax’s instability and Soft’s vanishing-gradient floor. The metrics confirm these failure modes: both Argmax and Soft fall below the CE baseline at their best settings, while Round is the only choice that yields a stable improvement (Appendix Figure 10). The training dynamics tell the same story (Figure 6): Argmax shows a sharp mid-training spike from mode collapse, whereas Round decreases smoothly and monotonically. The per-$\lambda$ sensitivity curves are reported in the appendix.

Conclusion

We identified the temporal generation gap in omni-modal language models, where internal attention localizes the target moment more accurately than the generated timestamps, and proposed Temporal Attention-Consistency Grounding (TACG), which couples attention, prediction, and ground truth into a triangle of consistency constraints that strengthens temporal localization with no added parameters, special tokens, or inference overhead. We further constructed a 580k-sample training dataset covering both grounding and general omni-modal understanding. Built on TACG and this dataset, OmniGround reaches state-of-the-art omni-modal temporal grounding while preserving strong general comprehension. More broadly, our results show that a model’s internal attention is a valuable and underexploited supervision signal for temporal precision; we will release the dataset, model, and training framework to support future work.

Technical Appendix

This appendix provides additional details omitted from the main text due to space constraints. It is organized into three parts: (i) supporting material for the preliminary analysis of the temporal generation gap (divergence definitions and per-layer attention analysis); (ii) model architecture and training details; and (iii) additional ablations and full experimental results.

Divergence Definitions

For two distributions $p$, $q$ over the $T$ time steps, the Jensen–Shannon divergence and the Wasserstein-1 distance used in the consistency loss (Eq. (4)) are

where KL is the Kullback–Leibler divergence and the second expression is the $\ell_1$ distance between the cumulative distributions, the closed form of the Wasserstein-1 distance in one dimension.

Per-Layer Attention Analysis

We obtain each layer’s audio-visual temporal attention distribution from the last prefill token. We take its attention weights normalized over all video and audio tokens, assign both modalities to their corresponding interleaved temporal grids, sum the attention mass within each grid, and average over heads, yielding the distribution $p_{\mathrm{att}}$ over the $T$ media grids. To make distributions comparable across clips of different length and segment position, we re-align every distribution so that the ground-truth segment is centered at the origin and rescaled to unit width, then average within each layer.

For each benchmark, the diagnostic evaluation split consists of 500 examples randomly sampled from the official test split. All preliminary-analysis statistics in Table 1, including Attn. wins and prediction mIoU, are computed on these fixed benchmark-specific subsets; ties are excluded only from the denominator of Attn. wins. The layer-wise visualization and sensitivity analysis use the same sampling protocol.

Lead rate versus aggregate mIoU. These two statistics characterize different aspects of localization quality. Lead rate measures how frequently attention is closer to the ground truth than the generated prediction at the instance level, whereas aggregate mIoU is also sensitive to the magnitude of a smaller number of large wins. Consequently, attention can lead on more non-tied samples while having a similar or slightly lower aggregate mIoU, as observed for the CE baseline. Our claim is therefore not that attention is uniformly superior on every sample, but that it provides complementary temporal evidence on a substantial fraction of instances. TACG exploits this evidence while anchoring both attention and prediction to the ground truth.

We visualize all 36 transformer layers to examine how the audio-visual temporal-localization signal evolves with model depth. As Figure 7 shows, layers 27–28 exhibit both sharp attention concentration inside the ground-truth segment and substantial sharpening from supervised fine-tuning, whereas earlier layers (e.g. layer 21) are less discriminative and the final layer 35 is diffuse. Figure 8 corroborates this across all 36 layers at once: in both the base and SFT models the attention mass concentrates inside the ground-truth band at essentially every layer, showing that the localization signal is broad rather than confined to a single layer. Throughout our experiments, we use layer 27 of Qwen2.5-Omni-3B (36 layers), layer 22 of Qwen2.5-Omni-7B, layer 32 of the Qwen3-Omni thinker (48 layers), and layer 20 of OLA (28 layers). For TACG training, the fixed attention layer is layer 27 for the 3B model and layer 22 for the 7B model, as reported in Table 8. The absolute index differs across models only because their depths do, not the underlying phenomenon.

Hyperparameter	Value
Base model	Qwen2.5-Omni-3B / 7B
Optimizer	AdamW
Learning rate	2.5e-5
LR scheduler	Cosine
Warmup ratio	0.05
Batch size (per device)	1
Gradient accumulation	2
Number of epochs	1
Max sequence length	16,384
DeepSpeed	ZeRO-2
Gradient checkpointing	Enabled
Attention implementation	FlashAttention
Precision	BF16
Frozen modules	ViT, audio aligner
TACG Configuration
$\lambda_{\mathrm{tp}}$ (attn–pred, $W_1$)	0.0500
$\lambda_{\mathrm{tg}}$ (attn–GT, JS)	0.0125
$\lambda_{\mathrm{pg}}$ (pred–GT, $W_1$)	0.1500
Attention layer $l$	27 (3B) / 22 (7B)
Gaussian $\sigma$	0.15 $\times$ segment width

We further test whether the temporal generation gap depends on the attention-extraction hyperparameters. Figure 9 varies the attention layer and the cumulative-mass cutoff used to convert attention into an interval. Across both sweeps, attention still overlaps the ground truth better than the generated interval on most non-tie samples, showing that the gap is not an artifact of a single layer or the default 50% cutoff.

Variant	Group	Attn. mIoU	Pred. mIoU	$\Delta$IoU
CE Base	Attn. wins	40.0	5.9	+34.1
	Pred. wins	37.9	68.5	−30.6
	Ties	0.0	0.0	0.0
	Overall	29.8	31.1	−1.3
Full TACG	Attn. wins	50.0	8.0	+42.0
	Pred. wins	41.1	74.0	−32.9
	Ties	0.0	0.0	0.0
	Overall	34.6	34.1	+0.5

Architecture Details

The vision encoder processes input frames at native resolution. Each frame is divided into 14 × 14 pixel patches. A temporal patch size of 2 groups two consecutive frames into a single temporal unit, reducing the temporal token count by half. After ViT encoding, a spatial merge with factor 2 further compresses spatial tokens by merging 2 × 2 adjacent spatial patches, resulting in a 4× reduction in spatial token count.

Audio Encoder. Raw audio waveforms are resampled to 16 kHz and converted to 128-bin mel-spectrograms. The Whisper-style encoder uses 32 transformer layers with 20 attention heads and a hidden dimension of 1280. A linear projection maps the output to 2048 dimensions, matching the language model hidden size. Audio longer than 30 seconds is segmented and encoded in parallel, with outputs concatenated along the sequence dimension.

Chunk-Interleaved Input Construction. For a video of duration $D$ seconds:

• If $D \le60$s: the video is encoded with $\lfloor D/2 \rfloor$ temporal grids, each covering exactly 2 seconds. Audio tokens are split into chunks of fixed size matching the 2-second interval. Video and audio chunks are interleaved in chronological order.

• If $D > 60$s: video frames are uniformly sampled and encoded into at most 30 temporal grids. Each grid covers $D/30$ seconds. Audio tokens are uniformly redistributed into 30 chunks of equal size, ensuring each audio chunk aligns temporally with its corresponding video grid.

The interleaved sequence is wrapped with special audio start and end tokens. Within each chunk, visual tokens appear first, followed by audio tokens. The 3D M-RoPE assigns temporal, height, and width position indices to visual tokens, while audio tokens receive sequential position indices synchronized with the temporal position of their chunk.

Training Data Details

Quality Filtering. Before reformatting, we remove low-resolution videos and overly short clips: grounding videos shorter than 30 s and general-understanding clips shorter than 2 s are discarded. General-understanding data is then sampled from four sources so that no single audio type dominates: OLA-reannotated LLaVA-Video for paired speech and visual reasoning, original LLaVA-Video for general video proficiency with audio, AVQA for scene sound, and FortisAVQA for music.

QA Construction. AVID and LongVALE provide source videos with dense caption-style temporal annotations rather than the final QA pairs used for our training. We use only their training-split source videos and temporal captions, then reconstruct new plain-text QA instances in three complementary grounding formats. We preserve the official benchmark separation: videos, temporal annotations, and captions from the held-out evaluation splits are excluded from all construction, filtering, and training stages. Forward TVG maps an event description to its interval—“Locate the event where {description}.” → “[start_sec, end_sec]”; Reverse Grounding inverts this, mapping a queried time interval to the event that occurs there; and Dense Video Captioning asks the model to enumerate all timed events in a clip—“[start, end]: {event}” lines in chronological order. This trains a single model to move in both directions between language and time rather than only emitting timestamps.

Audio-Visual Relevance Filtering and Mixing. After reformatting we apply an audio-visual relevance filter at the individual caption/query level, retaining targets that describe audio-visual events and removing text targets that are audio-only, visual-only, or ambiguous. For TVG and Reverse Grounding, the target is the queried event caption; for DVC, each timed event caption in the same QA instance is judged independently. The filter is text-only: Gemini 3.1 Pro receives only the event caption or QA text as input, labels the target as audio-visual, audio-only, visual-only, or ambiguous, and retains only audio-visual cases. Rejection rates are therefore computed over individual caption/query targets rather than whole videos. The filter is applied only to training-split samples and never sees held-out evaluation videos, annotations, or captions. A final balanced-mixing step combines the two streams into a 580,676-sample dataset at a grounding-to-general ratio of roughly 1:2, which we found preserves general audio-visual understanding while providing enough grounding supervision for the TACG objective to take effect.

Training Hyperparameters

We fine-tune Qwen2.5-Omni-3B and Qwen2.5-Omni-7B in a single stage with AdamW and a cosine schedule, freezing the vision encoder and audio aligner so that training adapts the language model and cross-modal fusion while preserving the pretrained perceptual front-ends. Both scales use the same optimization and TACG configurations except for the attention layer specified in Table 8. The TACG objective is applied selectively—all forward-TVG samples, which emit an explicit interval and thus admit a predicted distribution, use the full consistency loss, while all other samples use cross-entropy alone. The three edge weights ($\lambda_{\mathrm{tp}}, \lambda_{\mathrm{tg}}, \lambda_{\mathrm{pg}}$)

Component	Specification
Vision Encoder
Architecture	Vision Transformer (ViT)
Hidden size	1280
Patch size	$14 \times14$
Temporal patch size	2
Spatial merge size	2
Input channels	3
Audio Encoder
Architecture	Whisper-style Transformer
Hidden size	1280
Output dimension	2048
Number of layers	32
Attention heads	20
Mel bins	128
Language Model
Architecture	Transformer (autoregressive)
Hidden size	2048
Number of layers	36
Attention heads	16
Key-value heads	2 (grouped-query attention)
Intermediate size	11008
Vocabulary size	151936
Position encoding	3D M-RoPE (T, H, W)
Chunk Configuration
Seconds per chunk	2
Max temporal grids	30
Long video threshold	60 seconds

are the effective values used by the final models.

Computing Infrastructure and Software. The full computing environment is listed in Table 11. Main-text results use a fixed random seed (42); multi-seed mean±std for the key results is reported in the Multi-Seed Stability section (Tables 13 and 14). Code, training data, and model checkpoints will be released publicly upon publication.

Evaluation Protocol and Baseline Reporting

All grounding results are evaluated under the official protocol of each benchmark. For AVID and LongVALE, we use

Rejected type	Visual-only	Audio-only	Total
Rate	17%	4%	21%

Task	Source	Samples
Temporal Grounding Data (from AVID & LongVALE)
Event → Time (TVG)	AVID	34,041
Event → Time (TVG)	LongVALE	91,863
Time → Event (Reverse)	AVID	13,526
Time → Event (Reverse)	LongVALE	36,474
Dense Captioning (DVC)	AVID	7,239
Dense Captioning (DVC)	LongVALE	7,240
Subtotal		190,383
General Understanding Data
General video QA	LLaVA-Video	200,000
Speech + visual QA	LLaVA-Video (OLA)	120,925
Scene-sound QA	AVQA	37,441
Music QA	FortisAVQA	31,927
Subtotal		390,293
Total		580,676

Reverse Grounding is produced by inverting a 50,000-sample subset of the forward-TVG pool, so it inherits that pool’s AVID/LongVALE split. The SFT model probed in our preliminary analysis is trained only on the forward-TVG data (125,904 samples), without the Reverse Grounding, DVC, or general-understanding data.

the released test splits and compute [email protected], [email protected], [email protected], and mIoU against the annotated temporal intervals; for the constructed UnAV-100 TVG subset, we use the same metrics on the filtered single-interval test instances described below. For Charades-STA and ActivityNet-Captions, we follow the standard temporal grounding evaluation protocol used by prior video-LLM grounding work. General audio-visual understanding results on WorldSense and Daily-Omni are reported with the official accuracy metric.

UnAV-100 TVG Test Subset. We additionally construct a temporal video grounding test subset from the UnAV-100 [6] test annotations. The original test.parquet contains 5,900 audio-visual event annotations over 2,167 videos, where each annotation is a tuple of video_id, event label, and timestamp interval. We first group annotations by (video_id, event_label`). Among the 3,455 resulting video-event combinations, 1,309 correspond to multiple disjoint temporal intervals. Because our evaluation protocol expects one predicted interval per query, these multi-interval cases are ambiguous: a single model output cannot be matched to a unique ground-truth segment without an arbitrary choice. We therefore keep only video-event pairs associated with a single interval, yielding 2,146 QA instances over 1,610 videos.

We then filter the remaining instances by relative event duration. A large fraction of the single-interval cases nearly spans the whole video: 42.8% have ground-truth duration ratio above 0.9, and 53.7% are above 0.7. Such near-full-length events substantially inflate localization metrics, since a trivial predictor that outputs the whole video can already obtain about 0.62 mIoU over the full single-interval pool. To focus the benchmark on genuine temporal localization, we retain only short events whose ground-truth duration divided by video duration is below 0.3. The final UnAV-100 TVG subset contains 653 QA instances over 490 videos. We will release the corresponding JSON file for reproducible evaluation.

Item	Specification
Hardware
GPUs	32 × NVIDIA A100-80GB
Parallelism	DeepSpeed ZeRO-2
NVIDIA driver	470.161.03
Software
Python	3.10.15
PyTorch	2.8.0 (CUDA 12.8)
transformers	4.57.0
ms-swift	4.2.0
accelerate	1.13.0
deepspeed	0.17.2
peft	0.15.0
flash-attn	2.8.1
xformers	0.0.32
triton	3.4.0
datasets	4.8.4
tokenizers	0.22.1
torchaudio / torchvision	2.8.0 / 0.23.0
vllm (inference/eval)	0.10.2

For our models and the re-evaluated open-source baselines, we use the same benchmark input format and parse only the final predicted temporal interval. Outputs in seconds are evaluated directly. Outputs in frame indices or discrete temporal slots, when produced by methods whose native protocol uses such representations, are converted back to seconds using the benchmark video duration or frame rate before metric computation, following the corresponding benchmark protocol. If a model produces multiple intervals, we evaluate the first complete interval unless the model’s official inference protocol specifies otherwise. Invalid or unparsable outputs, including missing start/end times or intervals outside the video duration after clipping, are counted as failed predictions with zero IoU. For start-end inversions, we reorder the two endpoints before clipping to the valid video range, matching common temporal grounding evaluation practice.

The numbers in Table 3 and Table 2 are reported in two ways. For baselines with official benchmark results under the same evaluation setting, we cite the published numbers. For open-source omni-modal models and baselines without directly comparable reported results on AVID, LongVALE, or UnAV-100 TVG, we directly evaluate the released checkpoints; we do not fine-tune these baselines on our reconstructed QA data. The re-evaluated baselines use the same prompt template, video preprocessing budget, timestamp parser, and metric script used for OmniGround. Closed-source Gemini 3.1 Pro is included only as a reference result and is not used for training or model selection. Omni-Ground and OmniGround† are the only models in Table 3 trained on our 580k mixture, whose grounding portion consists of newly reconstructed QA instances derived from the training-split source videos and temporal captions of AVID and LongVALE. Therefore, Table 3 and Table 2 should be read as benchmark comparisons under each method’s available training protocol; the controlled comparison isolating TACG from data effects is the OmniGround† row, which uses the same 580k mixture as OmniGround but replaces TACG with standard cross-entropy.

Timestamp Parsing for TACG

All forward-TVG targets follow a standardized template containing a single well-formed start–end timestamp that can be deterministically parsed into digit positions. During training, we locate the timestamp span from the ground-truth answer first complete interval unless the model’s official inference protocol specifies otherwise. Invalid or unparsable outputs, including missing start/end times or intervals outside the video duration after clipping, are counted as failed predictions with zero IoU. For start-end inversions, we reorder the two endpoints before clipping to the valid video range, matching common temporal grounding evaluation practice. The numbers in Table 3 and Table 2 are reported in two ways. For baselines with official benchmark results under the same evaluation setting, we cite the published numbers. For open-source omni-modal models and baselines without directly comparable reported results on AVID, LongVALE, or UnAV-100 TVG, we directly evaluate the released checkpoints; we do not fine-tune these baselines on our reconstructed QA data. The re-evaluated baselines use the same prompt template, video preprocessing budget, timestamp parser, and metric script used for OmniGround. Closed-source Gemini 3.1 Pro is included only as a reference result and is not used for training or model selection. Omni-Ground and OmniGround† are the only models in Table 3 trained on our 580k mixture, whose grounding portion consists of newly reconstructed QA instances derived from the training-split source videos and temporal captions of AVID and LongVALE. Therefore, Table 3 and Table 2 should be read as benchmark comparisons under each method’s available training protocol; the controlled comparison isolating TACG from data effects is the OmniGround† row, which uses the same 580k mixture as OmniGround but replaces TACG with standard cross-entropy. template and identify the digit-token positions corresponding to the start and end times. The consistency losses are then computed from the logits at these positions, as described in Eq. (2).

The reconstructed timestamps are converted to seconds and normalized by the video duration before Gaussian projection. We use the timestamp precision provided by the training template and keep the same precision across the start and end fields. The ground-truth distribution is constructed from the annotated interval with the same normalization and Gaussian projection as the prediction, but is detached from the computation graph.

Discretization Weight Sensitivity

This section reports the full sweep behind the discretization choice made in the main text. For each of the three ways of turning the per-position digit logits into a predicted timestamp—Argmax, Soft, and our Round (straight-through rounding)—we vary the loss weight $\lambda$ on the predicted-distribution edge and measure mIoU on AVID and LongVALE. Figure 10 summarizes each method at its best $\lambda$ against the cross-entropy (CE) baseline, and Figure 11 gives the complete mIoU-versus-$\lambda$ curves.

The two views agree. Round is the only discretization that stays above the CE baseline on both benchmarks and remains stable across the entire $\lambda$ range, whereas Argmax and Soft dip below the baseline at their best settings and degrade quickly as $\lambda$ grows. This matches the failure modes analyzed in the main text: Argmax’s discontinuous digit selection triggers mode-collapse instability, while Soft’s non-integer expectation leaves a loss floor that its gradient cannot cross. The robustness of Round across $\lambda$ is what makes it usable without per-benchmark tuning.

Weight Ablation

We ablate the loss weight $\lambda$ of each edge by sweeping across a range of values with both JS divergence and Wasserstein-1 (W1) as distance metrics.

As shown in Figure 12, all edges share the same fundamental trade-off: when $\lambda$ is too small, the supervision signal is too weak to meaningfully guide learning; when too large, the distribution-matching gradient overwhelms the cross-entropy loss and degrades generation quality. The optimal weight and distance metric, however, differ across edges. The ground-truth-anchored edges admit stronger supervision: $\mathcal{L}_{\mathrm{pg}}$ peaks with W1 at $\lambda_{\mathrm{pg}} = 0.2$, whose distance-proportional gradients give a smoother landscape that tolerates large weights, whereas $\mathcal{L}_{\mathrm{tg}}$ favors JS at $\lambda_{\mathrm{tg}} = 0.1$ with a broad effective range—for noisy attention supervision, JS measures distributional overlap without sensitivity to the absolute magnitude of misalignment and is thus more stable. The self-supervised edge $\mathcal{L}_{\mathrm{tp}}$ exhibits a narrower effective range, as it enforces consistency between model-internal signals without external supervision and excessively large weights cause the distributions to reinforce each other’s errors; W1 with $\lambda_{\mathrm{tp}} = 0.1$ achieves the best balance. Beyond the per-edge sweeps, Figure 13 maps the joint $\mathcal{L}_{\mathrm{tg}}+\mathcal{L}_{\mathrm{pg}}$ response surface over a 2D weight grid, confirming that the two ground-truth-anchored edges produce a broad region of improvement over the SFT baseline rather than a brittle optimum.

Full Component Ablation

Table 12 reports the complete component ablation, including both addition results (upright rows, with individually optimized weights via progressive search) and subtraction results (italic rows, which inherit weights from the full configuration); the main text (Table 5) shows only the addition rows.

Configuration	AVID [email protected]	AVID mIoU	LongVALE [email protected]	LongVALE mIoU	Δ%
Baseline (CE only)	0.3502	0.3153	0.1408	0.1604	0.00
$\mathcal{L}_{\mathrm{tg}}$ only	0.3543	0.3223	0.1563	<u>0.1722</u>	+4.79
$\mathcal{L}_{\mathrm{pg}}$ only	0.3649	0.3282	0.1479	0.1645	+3.32
$\mathcal{L}_{\mathrm{tp}}$ only	<u>0.3671</u>	<u>0.3315</u>	0.1514	0.1663	+4.41
Full w/o $\mathcal{L}_{\mathrm{pg}}$, $\mathcal{L}_{\mathrm{tp}}$	0.3558	0.3229	0.1360	0.1563	−0.08
Full w/o $\mathcal{L}_{\mathrm{tg}}$, $\mathcal{L}_{\mathrm{tp}}$	0.3350	0.3056	0.1293	0.1488	−5.16
Full w/o $\mathcal{L}_{\mathrm{tg}}$, $\mathcal{L}_{\mathrm{pg}}$	0.3534	0.3221	0.1366	0.1560	−0.29
$\mathcal{L}_{\mathrm{tg}}+\mathcal{L}_{\mathrm{pg}}$	0.3671	0.3301	0.1508	0.1672	+4.47
Full w/o $\mathcal{L}_{\mathrm{tp}}$	0.3271	0.2989	0.1354	0.1579	−3.38
Full w/o $\mathcal{L}_{\mathrm{pg}}$	0.3481	0.3150	0.1446	0.1597	−0.27
Full w/o $\mathcal{L}_{\mathrm{tg}}$	0.3703	0.3315	0.1359	0.1570	+1.51
Full TACG	0.3814	0.3408	<u>0.1546</u>	0.1759	+8.87

Two observations stand out. First, among all single-edge removals from the full model, dropping the self-supervised attention–prediction edge $\mathcal{L}_{\mathrm{tp}}$ causes the steepest degradation ($-3.38\%$), reinforcing the main-text conclusion that it is the key contributor rather than a redundant term. Second, the subtraction rows—which reuse the full model’s weights instead of re-optimizing for the smaller edge subset—frequently under-perform their addition counterparts, and can even fall below the baseline. This shows that the edge weights do not transfer across subsets, and is why we report both the addition and subtraction views rather than a single ablation direction.

Multi-Seed Stability

The results in the main text come from a single run with a fixed seed (42). To verify that the reported margins are not seed artifacts—particularly the small full-scale gaps and the closely spaced single-edge ablations—we repeat the key experiments over three seeds and report the mean and standard deviation. Table 13 covers the 3B final model on the four grounding benchmarks, and Table 14 repeats the component ablation of Table 5. Across both tables, the mean stays close to the single-seed estimate and the ordering of configurations is preserved, indicating that the reported gains are stable rather than seed-specific.

Benchmark (mIoU)	OmniGround (3B)
AVID	$56.9 \pm0.4$
LongVALE	$28.6 \pm0.5$
Charades-STA	$38.8 \pm0.6$
ActivityNet-Captions	$23.7 \pm0.4$

Configuration	AVID mIoU	LongVALE mIoU
Baseline (CE only)	$0.3147 \pm0.0042$	$0.1598 \pm0.0031$
$\mathcal{L}_{\mathrm{tg}}$ only	$0.3218 \pm0.0038$	$0.1715 \pm0.0035$
$\mathcal{L}_{\mathrm{pg}}$ only	$0.3275 \pm0.0045$	$0.1640 \pm0.0029$
$\mathcal{L}_{\mathrm{tp}}$ only	$0.3309 \pm0.0040$	$0.1658 \pm0.0033$
$\mathcal{L}_{\mathrm{tg}} + \mathcal{L}_{\mathrm{pg}}$	$0.3296 \pm0.0036$	$0.1667 \pm0.0030$
Full TACG	$0.3401 \pm0.0034$	$0.1753 \pm0.0028$