WorksComputer Science
SHUSH: Emission-Aware Reward Design for Proactive Video Assistants
Abstract
Proactive video assistants must decide not only what to say, but also when to speak. Existing reinforcement-learning objectives can reward timely, correct responses without accounting for response volume, thereby encouraging unnecessary or repeated emissions. We identify this failure mode in MMDuet2, whose proactive area-under-the-curve (PAUC) reward promotes timely responses but does not price the number of emissions. We show analytically that this makes additional responses weakly penalized or effectively free, systematically encouraging over-emission. We then propose an emission-aware F1 reward for interval-valued annotations, using exclusive span–emission matching to price false emissions and missed spans. Under MMDuet2's training recipe, our policy on the visual subset of OmniPro replies 2.5x less often than MMDuet2 while raising timing F1 from 0.259 to 0.396 and content-aware F1 from 0.103 to 0.154; a more conservative variant cuts replies by 82.5% and raises precision from 0.161 to 0.418. On ProactiveVideoQA, the same reward adds replies where the baseline is nearly silent, and PAUC ranks a heavily over-emitting policy second despite near-last F1, because it is insensitive to response count. Our results indicate that proactive assistants should be trained and evaluated with objectives that price emissions.
Introduction
Video large language models (Video-LLMs) are increasingly moving from offline video understanding toward continuous streaming interaction, where frames arrive sequentially and the model must decide autonomously when to respond [1, 2, 3]. In such proactive systems, response timing is part of the prediction problem: speaking too late reduces utility, while speaking unnecessarily produces repetitive or unsupported responses. Because the assistant initiates these interactions without being prompted, each unnecessary reply consumes user attention, can distract from genuinely relevant events, and adds avoidable inference and communication overhead; repeated low-value replies can also erode user trust. We refer to each model-generated reply at a decision turn as an emission.
Recent systems have therefore begun learning when to speak rather than relying only on fixed inference thresholds. MMDuet2 [3], for example, formulates response triggering as
a per-turn textual decision and optimizes the policy using reinforcement learning. Despite improved timing performance, recent evaluation shows that proactive Video-LLMs can still achieve high temporal scores by responding frequently, increasing recall at the expense of precision [4]. This raises a fundamental question: does the learned policy over-emit because optimization fails, or because the reward itself makes frequent emission advantageous?
We show that the latter can occur. In MMDuet2, the dominant Proactive Area Under the Curve (PAUC)-based reward integrates response quality over an annotated temporal span, rewarding early improvements but not explicitly accounting for the total number of emissions. An additional reply can therefore increase reward when it improves the running score, while costing nothing when it does not, and the auxiliary penalties for out-of-span and repetitive replies weaken as the number of decision turns grows. Section 3 characterizes these incentives and shows that over-emission becomes increasingly favorable when relevant events are sparse relative to decision opportunities.
To address this mismatch, we propose an emission-aware F1 reward for interval-valued annotations that explicitly couples reward to the number of responses produced by the model. Our formulation uses one-to-one maximum-weight matching between emissions and gold spans, allowing each span to credit at most one reply while explicitly penalizing unmatched emissions and missed spans. As a result, an additional response must provide sufficient value to justify its cost rather than serving as a nearly free additional attempt. Our contributions are threefold: (1) an analytical diagnosis of how PAUC-based reward design can systematically incentivize over-emission; (2) an interval-aware, emission-priced F1 reward based on exclusive span–emission matching; and (3) an empirical evaluation showing an 82.5% reduction in emissions on OmniPro while raising precision from 0.161 to 0.418 and F1 from 0.259 to 0.346. Figure 1 contrasts the two reward structures.

Figure 1. Overview of the proposed emission-aware reward design. Count-insensitive temporal rewards can favor repeated or unnecessary responses, whereas our interval-aware F1 formulation prices false emissions and missed spans through exclusive emission–span matching.
Related Work
Proactive streaming Video-LLMs. Streaming Video-LLMs must determine both response content and response timing [1, 5]. Existing triggering mechanisms include token-driven decisions [1, 6, 7], dedicated classification heads [2, 8, 9], and heuristic visual or uncertainty criteria [10, 11]. Recent methods increasingly learn this decision directly through reinforcement learning. MMDuet2 [3], for example, formulates triggering as a per-turn textual decision optimized via RL, eliminating the need for fixed inference thresholds. We focus on the reward design for such learned response policies.
Timing metrics and rewards. StreamingBench [12] and OVO-Bench [13] evaluate temporal correctness but score at most one response per query. ProactiveVideoQA [14] introduces PAUC, which integrates response quality over an annotated temporal span but does not explicitly account for the number of emitted responses. More recent benchmarks address over-triggering directly: OmniPro [15] penalizes unnecessary responses, while StreamPro-Bench [4] evaluates response trajectories using precision-recall F1. Correspondingly, MMDuet2 [3] optimizes a PAUC-derived reward that favors early, correct responses while adding penalties for repetition, out-of-span emissions, and verbose prefixes, whereas StreamPro [4] introduces a count-sensitive turn-level F1 term for timestamp-valued targets. Both score response content with an LLM judge [16]. In contrast, we analyze why count-insensitive temporal objectives favor over-emission and propose an emission-aware formulation for interval-valued annotations.
When Temporal Rewards Encourage Over-Emission
We use MMDuet2 [3] as a representative case study of RL-based proactive response timing. On the visual subset of OmniPro, it produces 6,961 replies for 1,723 annotated triggers, motivating closer inspection of the underlying objective. These excess emissions occur both outside annotated events and as repeated responses within already-covered spans. Rather than viewing this as a model-specific optimization failure, we identify two broader structural incentives in count-insensitive temporal objectives: additional predictions can carry little or no marginal cost, and normalized penalties can weaken as the number of decision opportunities increases.
Marginal Incentives in a PAUC-based Objective
Let a video rollout consist of total decision turns, with turns falling outside any annotated ground-truth span. The annotation provides gold spans bounded by . Within a given span, the model emits replies at timestamps , each assigned a content score (with ) by
an LLM judge. The published reward is
The dominant term, , is derived from the PAUC metric of ProactiveVideoQA [14]. It represents the response score as a step function over time, initialized at a floor of at , and measures its area as a fraction of the maximum possible area:
The reward averages PAUC over the spans. Following the official implementation, timestamps are evaluated at with . The remaining terms penalize semantic repetition, out-of-span emissions, and repeated prefixes; importantly, these penalties are normalized respectively by emission count, the number of out-of-span turns , and the total number of turns .
Let denote an emission’s fractional offset within a span. Adding a final reply with score when the running score is yields
The marginal gain depends on , rewarding only improvements over the current response. However, the emission count does not appear in the PAUC normalization, so additional attempts can increase reward without incurring cost when they fail to improve the running score.
Out-of-span Emissions
For a span’s first reply, , while silence scores . Since judge scores are integers on , any positive verdict improves upon silence. With , the temporal factor in (3) remains in , so later responses reduce the gain but do not make speaking a loss. Only penalizes emissions outside annotated spans. Because it is weighted by 0.5 and normalized by the out-of-span turns, the marginal cost of a false emission is . By contrast, perfectly answering one span contributes to the weighted PAUC reward. Thus, for , the gain from one perfectly answered span can exceed the maximum total out-of-span penalty. More generally, because span boundaries are unknown during streaming, each emission is a decision under uncertainty. Let denote the expected reward from hitting an unanswered span. An emission is favorable when its success probability exceeds . Since the available reward per span scales as while , the incentive toward false emissions grows approximately with . Count-insensitive rewards are therefore least conservative when events are sparse.
Repeated emissions within a span
For subsequent replies within the same span, is the score currently in force. If a new reply receives the same score, (3) evaluates to zero. Hence, PAUC itself imposes no cost for retrying, while a better-scored reply can still increase reward. When content scores exhibit variation across separately judged responses, another attempt can therefore have positive expected value whenever the current score is sufficiently low.
The auxiliary repetition terms only partially address this behavior. Because is averaged over emissions, adding another reply with similar repetitiveness need not substantially worsen the normalized penalty. Likewise, targets lexical prefix repetition and can miss semantically redundant rephrasings. Figure 3 shows this directly: four identically scored replies leave unchanged, while declines with each.
Figure 3. Reward response on one trajectory. Four consecutive replies scoring 3 leave unchanged, since none improves on the score in force, whereas falls with each and rises only at , where a higher-scoring reply justifies its cost.
Emission-aware reward design
Section 3 suggests a general design requirement: the marginal cost of speaking should depend directly on the number of responses produced by the policy. Reweighting count-insensitive terms alone cannot guarantee this property. A precision–recall formulation naturally provides it because its prediction-side denominator grows with model output.
Building on count-sensitive turn-level F1 reward design [4], we propose an emission-aware F1 reward for interval-valued annotations. Unlike point-timestamp supervision, interval-valued targets introduce an assignment problem because multiple emissions may fall within the same span and overlapping spans may compete for the same prediction.
We optimize
Here, rewards well-formed output at each decision turn, is the set of matched emission–span pairs, is the number of matched emissions, the number of unmatched ones, and the number of reachable ground-truth spans. We write for the total emissions. Each matched emission receives a content score from an LLM judge and temporal weight . When , the denominator becomes , recovering the standard F1 count structure. Figure 2 works through a representative trajectory.

Figure 2. The emission-aware F1 reward computed on a representative trajectory.
Matching interval-valued annotations
Interval-valued supervision introduces an assignment problem absent from point-based annotation. Multiple emissions may lie within a single span, while overlapping spans can make one emission eligible for several spans. Crediting every eligible pair would again reward over-generation, whereas greedy matching can consume an emission needed to cover another reachable span. We therefore use one-to-one maximum-weight bipartite matching, allowing each emission and each gold span to participate in at most one credited pair.
Human annotations of temporal event boundaries are also inherently imprecise. A strict binary membership test would assign no credit to an otherwise correct response arriving just outside the annotated boundary. We therefore use a continuous decay around each span. For a span , let , where caps the ramp width:
Responses near a boundary therefore receive partial credit rather than experiencing an abrupt reward transition. The parameter controls the intra-span discount for later responses; we set , leaving the reward flat in the annotated span.
Pricing emissions and missed spans
The parameters and control the two complementary errors. The coefficient increases the cost of unmatched emissions, while controls the relative emphasis placed on covering ground-truth spans. Increasing therefore favors a more conservative policy, whereas increasing places greater emphasis on recall. Exclusive matching also directly prices repeated responses. If multiple emissions compete for the same span, at most one can receive credit for that target; the others contribute to unless they are matched elsewhere. The matching retains the highest-value, or best-match assignment, allowing a genuinely improved response to replace a weaker
Bibliography
The main reference list (19 numbered entries) is already transcribed. Skip its entries if they are printed on this page; transcribe everything else and mark citations as above. one while charging for unnecessary retries. The reward therefore specifies not how often the model should speak, but how much value a new response must add.
Experimental Results
Setup. The MMDuet2 SFT checkpoint is not publicly released, so we reproduce it from the published recipe with Qwen2.5-VL-3B [17] initialization. From this checkpoint we run GRPO [18] with VeRL [19] and SGLang on MMDuet2’s RL data for one epoch, changing only the reward; the in-reward judge is Qwen2.5-32B at temperature 0, and we set and . We report two configurations: , which charges out-of-span emissions more than in-span ones, and , an reward with soft span membership (§ 4). The baseline is the released MMDuet2 checkpoint [3]. We also run the released MMDuet [2] and VideoLLM-Online [1]. We evaluate on the visual subset of OmniPro [15] and on ProactiveVideoQA [14]. For both, timing precision, recall and F1 use one-to-one matching between replies and gold answers; counts replies per gold answer.
Results on OmniPro. OmniPro annotates each answer as a point trigger; a reply matches a trigger within 3 s. To score what a reply says as well as when, we define , where sums a GPT-4.1 content score (0, 0.5 or 1, against the trigger’s reference answer) over matched pairs; timing F1 is the case where every match scores 1. Table 1 shows two opposite ways a proactive model can fail. VideoLLM-Online almost never replies and recovers 1.0% of triggers. MMDuet replies 5.39 times per trigger and reaches the highest recall (0.738), but only 13.7% of its replies match a trigger. Our policies sit between these extremes. The policy replies 1.63 times per trigger, 2.5 fewer than MMDuet2 and 3.3 fewer than MMDuet, and achieves the highest timing F1 (0.396) and content (0.154) of all models. The policy replies 0.71 times per trigger with the highest precision (0.418), above even the nearly silent VideoLLM-Online (0.395). improves along with timing F1, so the remaining replies are both better placed and more often correct. MMDuet2 reaches higher recall than either policy by generating more responses to cover each trigger, while retains 80% of that recall with 40% of the replies.
Table 1. Visual subset of OmniPro (, triggers).
Results on ProactiveVideoQA. We report PAUC and repetition (Rep%) with MMDuet2’s released scorer and a GPT-4.1 judge, where Rep% is the fraction of repetitive replies, averaged over spans. PAUC integrates the best answer given within each span and, as § 3 argues, cannot observe reply volume. Table 2 shows this with the baselines. On EGO, MMDuet ranks second on PAUC (43.6) while replying 16.9 times per gold answer, yet second-lowest on F1. On VAD, its 29 replies per answer score 23.1 PAUC, below the 25 a silent model receives. Our reward does not simply suppress replies. On EGO, where MMDuet2 replies only 0.54 times per gold answer and stays silent on 39.6% of videos, our policies reply 0.79–1.72 times per answer and raise F1 from 0.248 to 0.419, the highest of all models; improves every column over MMDuet2 while speaking more, and attains the highest PAUC on both EGO (46.0) and TV (47.0). On TV and VAD, is comparable to MMDuet2 in F1; on WEB it trails: WEB’s gold spans cover 81% of video time, so a reply at a random moment already lands in a span 81% of the time, and dense firing costs almost nothing. Interval annotation at this density leaves little room for a reward that prices replies. Hence, OmniPro’s point triggers serve as our primary test.
| Model | PAUC | Rep% | F1 | ||
| EGO | VideoLLM-Online | 25.3 | 18.5 | 0.73 | 0.024 |
| MMDuet | 43.6 | 77.2 | 16.86 | 0.092 | |
| MMDuet2 | 32.1 | 9.5 | 0.54 | 0.248 | |
| Ours () | 46.0 | 13.4 | 1.72 | 0.418 | |
| Ours () | 37.7 | 2.5 | 0.79 | 0.419 | |
| TV | VideoLLM-Online | 20.0 | 32.0 | 5.69 | 0.100 |
| MMDuet | 22.5 | 61.0 | 20.62 | 0.078 | |
| MMDuet2 | 45.6 | 1.3 | 2.37 | 0.358 | |
| Ours () | 47.0 | 3.0 | 2.92 | 0.350 | |
| Ours () | 38.6 | 0.0 | 1.63 | 0.316 | |
| WEB | VideoLLM-Online | 25.2 | 3.3 | 0.07 | 0.097 |
| MMDuet | 41.3 | 9.3 | 0.79 | 0.703 | |
| MMDuet2 | 54.4 | 4.4 | 1.25 | 0.718 | |
| Ours () | 45.4 | 2.3 | 0.75 | 0.680 | |
| Ours () | 36.5 | 0.7 | 0.38 | 0.495 | |
| VAD | VideoLLM-Online | 25.0 | – | 0.07 | 0.000 |
| MMDuet | 23.1 | 43.0 | 29.11 | 0.052 | |
| MMDuet2 | 29.4 | 12.7 | 1.68 | 0.188 | |
| Ours () | 27.7 | 15.5 | 1.64 | 0.191 | |
| Ours () | 27.8 | 0.0 | 0.75 | 0.182 |
Table 2. ProactiveVideoQA (: EGO 1575, TV 500, WEB 1328, VAD 107). PAUC floors at 25 for a silent model. F1 is timing-only: a reply matches a gold turn if inside its reply span, one reply per turn; with , precision and recall .
Conclusions
We show that over-emission in proactive video assistants can arise from reward design rather than failed optimization. We identify a general failure mode in PAUC-based objectives, where additional predictions can improve reward without a count-dependent cost, and address it with an emission-aware F1 reward using exclusive span–emission matching. On OmniPro, our policy reduces emissions by 82.5% while raising precision from 0.161 to 0.418 and F1 from 0.259 to 0.346. Limitations include intra-span timing and manually chosen and , which future work can derive from deployment-specific costs.
Acknowledgement
This work was supported by the Air Force Research Laboratory/Information and Spectrum Warfare Directorate (AFRL/RF), Rome, NY under the 6.2 Program Planar Bridge. Any statements or claims made in this publication are solely those of the authors and do not represent the views or policies of the United States Government.
Compliance with Ethical Standards
This study uses existing publicly available benchmark datasets and pretrained models and does not involve the collection of new human subject data. Therefore, no additional ethical approval was required.
References
- [1]Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia-Wei Liu, Ziting Gao, Dongxing Mao, and Mike Zheng Shou, “Videollm-online: Online video large language model for streaming video,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024, pp. 18407–18418.arxiv.org/abs/2406.11816
- [2]Yueqian Wang, Xiaojun Meng, Yuxuan Wang, Jianxin Liang, Jiansheng Wei, Huishuai Zhang, and Dongyan Zhao, “VideoLLM knows when to speak: Enhancing time-sensitive video comprehension with video-text duet interaction format,” in Findings of the Association for Computational Linguistics: EMNLP 2025, Nov. 2025, pp. 6338–6359.
- [3]Yueqian Wang, Songxiang Liu, Disong Wang, Nuo Xu, Guanglu Wan, Huishuai Zhang, and Dongyan Zhao, “Mmduet2: Enhancing proactive interaction of video mllms with multi-turn reinforcement learning,” in International Conference on Learning Representations, 2026, vol. 2026, pp. 23257–23270.arxiv.org/abs/2512.06810
- [4]Ao Li, Zihan Xiao, Zihao Yue, Boshen Xu, Linli Yao, Jiaze Li, Pei Fu, Jianzhong Ju, Jian Luan, and Qin Jin, “Streampro: From reactive perception to proactive decision-making in streaming video,” arXiv preprint arXiv:2605.16381, 2026.arxiv.org/abs/2605.16381
- [5]Zhenyu Yang, Kairui Zhang, Qi Liu, Tiancheng Liu, Long Ying, Dizhan Xue, Qibin Hou, Shengsheng Qian, and Changsheng Xu, “Towards online interactors: A comprehensive survey on streaming video understanding,” Preprints, June 2026.DOI
- [6]Xueyang Yu, Cheng Shi, Yang Wang, and Sib ei Yang, “Eyes wide open: Ego proactive video-llm for streaming video,” in Advances in Neural Information Processing Systems. 2025, vol. 38, Main Conference, pp. 13420–13463, Curran Associates, Inc.arxiv.org/abs/2510.14560
- [7]Joya Chen, Ziyun Zeng, Yiqi Lin, Wei Li, Zejun Ma, and Mike Zheng Shou, “Livecc: Learning video llm with streaming speech transcription at scale,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025, pp. 29083–29095.arxiv.org/abs/2504.16030
- [8]Rui Qian, Shuangrui Ding, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang, “Dispider: Enabling video LLMs with active real-time interaction via disentangled perception, decision, and reaction,” in Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), June 2025, pp. 24045–24055.arxiv.org/abs/2501.03218
- [9]Haibo Wang, Bo Feng, Zhengfeng Lai, Mingze Xu, Shiyu Li, Weifeng Ge, Afshin Dehghan, Meng Cao, and Ping Huang, “Streambridge: Turning your offline video large language model into a proactive streaming assistant,” arXiv preprint arXiv:2505.05467, 2025.arxiv.org/abs/2505.05467
- [10]Zhenyu Yang, Kairui Zhang, Yuhang Hu, Bing Wang, Shengsheng Qian, Bin Wen, Fan Yang, Tingting Gao, Weiming Dong, and Changsheng Xu, “Livestar: Live streaming assistant for real-world online video understanding,” in Advances in Neural Information Processing Systems. 2025, vol. 38, Main Conference, pp. 31266–31304, Curran Associates, Inc.arxiv.org/abs/2511.05299
- [11]Linli Yao, Yicheng Li, Yuancheng Wei, Lei Li, Shuhuai Ren, Yuanxin Liu, Kun Ouyang, Lean Wang, Shicheng Li, Sida Li, et al., “TimeChat-online: 80% visual tokens are naturally redundant in streaming videos,” in Proceedings of the 33rd ACM International Conference on Multimedia, 2025, pp. 10807–10816.arxiv.org/abs/2504.17343
- [12]Junming Lin, Zheng Fang, Chi Chen, Zihao Wan, Fuwen Luo, Peng Li, Yang Liu, and Maosong Sun, “Streamingbench: Assessing the gap for mllms to achieve streaming video understanding,” arXiv preprint arXiv:2411.03628, 2024.arxiv.org/abs/2411.03628
- [13]Junbo Niu, Yifei Li, Ziyang Miao, Chunjian Ge, Yuanhang Zhou, Qihao He, Xiaoyi Dong, Haodong Duan, Shuangrui Ding, Rui Qian, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, and Jiaqi Wang, “Ovo-bench: How far is your videollms from real-world online video understanding?,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025, pp. 18902–18913.arxiv.org/abs/2501.05510
- [14]Yueqian Wang, Xiaojun Meng, Yifan Wang, Huishuai Zhang, and Dongyan Zhao, “Proactivevideqoa: A comprehensive benchmark evaluating proactive interactions in video large language models,” arXiv preprint arXiv:2507.09313, 2025.arxiv.org/abs/2507.09313
- [15]Ruixiang Zhao, Jie Yang, Zijie Xin, Tianyi Wang, Fengyun Rao, Jing LYU, and Xirong Li, “Omnipro: A comprehensive benchmark for omni-proactive streaming video understanding,” arXiv preprint arXiv:2605.18577, 2026.arxiv.org/abs/2605.18577
- [16]Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu, “Llms-as-judges: A comprehensive survey on llm-based evaluation methods,” ArXiv, vol. abs/2412.05579, 2024.arxiv.org/abs/2412.05579
- [17]Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin, “Qwen2.5-vl technical report,” ArXiv, vol. abs/2502.13923, 2025.arxiv.org/abs/2502.13923
- [18]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y.K. Li, Y. Wu, and Daya Guo, “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,” 2024.arxiv.org/abs/2402.03300
- [19]Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu, “Hybridflow: A flexible and efficient rlhf framework,” in Proceedings of the Twentieth European Conference on Computer Systems. Mar. 2025, p. 1279–1297, ACM.DOI
