betaFeedback

WorksComputer Science

CineMME: From Dialogue Grounding to Narrative Reasoning in Multimodal Video Understanding

Abstract

Narrative video understanding requires more than recognizing visible events: models must extract evidence from dialogue, bind it to speakers, and use it to infer narrative meaning. Existing video benchmarks evaluate aspects of this problem, including video question answering, audio-visual reasoning, and temporal grounding, but rarely test whether these abilities form a coherent dialogue-to-plot evidence chain within the same clips. We introduce CineMME, a human-verified benchmark designed to diagnose dialogue-grounded plot understanding, supported by approximately 1,6001,600 expert person-hours of annotation. CineMME contains 975975 curated short narrative video clips and two linked tracks: CineMME-Grounding, with 13,55413,554 timestamped speech segments and 33,08333,083 speaker face boxes for temporal dialogue localization, transcription, and active-speaker grounding; and CineMME-Reasoning, with 1,6001,600 multiple-choice and 316316 open-ended questions spanning perception, audio, narrative, social, and scene-presentation understanding. Our main evaluation compares 2020 MLLMs on reasoning and 99 omni-modal models on grounding. The results reveal a pronounced who-what-when decoupling: Gemini-3 Pro localizes dialogue in time (68.34%68.34\% tIoU) and transcribes it with a WER of 28.99%28.99\%, yet achieves only 7.74%7.74\% vIoU for active-speaker grounding while achieving 67.66%67.66\% MCQ accuracy on questions with extractable answers. Modality ablations further show that vision is not uniformly beneficial: it improves performance on perception-heavy questions but can hurt dialogue-centric reasoning. CineMME provides a compact diagnostic testbed for evaluating how multimodal video models connect dialogue, speakers, visual evidence, and narrative meaning.

Paper details

vlnc:W_vt8qtqm9
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 12

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…