In this paper
AbstractIntroductionBackground and motivationResearch gapArgumentResearch questions and contributionsRelated WorkBody-worn video, AI and police accountabilityFine-grained visual retrievalMultimodal video understanding and its evaluationEvaluation, accountability and fairnessMethodologyPipelines and tasksCase A: fine-grained knife retrieval (KnifeHunter)Case B: experimental multimodal BWV understanding pipelinePilot benchmark: UKBWV-27Metrics for event-level tasksAutomated judge and human validationAnalysis planExperiments and ResultsCase A: object-level knife retrieval (E1)Case B: BWV pipeline benchmarkResultsJudge validationCross-case comparison (E3)DiscussionPositioningWhat the protocol shows about accountabilityApplications in PolicingLimitationsConclusion and Future WorkAcknowledgmentsEthical Considerations StatementGenerative AI Usage StatementAnnotation GuidelinesEvent-Level MetricsBenchmark Protocol DetailsJudge Validation Protocol and SampleModel Selection and PanelPrompt TemplatesDatasheet for UKBWV-27KnifeHunter Training, Backbones and AblationReferencesWorksComputer Science
From Weapons to Events: Evaluating Evidential Support in Multimodal AI for Policing
Abstract
Computer vision (CV) for policing has centred on recognising objects such as seized weapons, while body-worn video (BWV) is still reviewed by people, with existing AI tools using only its audio. Multimodal video AI could make BWV review faster and cheaper. At the object level, we present the KnifeHunter dataset, developed with UK law enforcement, with over 25,000 images of 543 knife classes and a protocol reflecting investigative conditions, and KHNet, a retrieval architecture fusing local and global features, which with about 125,000 distractor images attains 85.7% and 63.4% mean Average Precision (mAP) on the medium and hard protocols. At the event level, we present an evaluation protocol for video understanding pipelines, which describe a video as events and a transcript, that scores transcription, speaker attribution, event detection, temporal localisation and unsupported events. Whether each event is supported by the recording is labelled by a judge validated against blinded human labels. We apply it to complete pipelines, three 2026 omni-modal models and our hybrid NarrativeBridge-PAL-GPT-5 pipeline, on UKBWV-27, a pilot stress-test set of 27 publicly released UK BWV recordings (118 minutes). On this set the hybrid pipeline leads on five of six measures, yet it leaves 13.6% of predicted events unsupported and misattributes 27.7% of utterances, and every pipeline degrades on recordings from edited releases, long recordings and noisy audio. We propose readiness levels separating assistive tasks from those requiring verification of every output.
Introduction
Background and motivation
Body-worn video (BWV) cameras were adopted by police services in the United Kingdom and elsewhere with the stated aims of improving transparency, gathering evidence and supporting the accountability of officers to the public. Randomised and quasi-experimental studies have examined their effects on use of force and complaints [4], and a systematic review found that effects on most officer and citizen behaviours are neither clear nor consistent [34]. A precondition for any accountability benefit is that recorded footage is actually reviewed. The volume is substantial: in London’s year-long randomised trial, 814 officers made 48,281 recordings totalling 12,212 hours, and officers reported that prosecutors and secondary investigators were often unable to view footage; the Metropolitan Police then announced plans to buy around 20,000 cameras [21]. Manual review therefore cannot keep pace, so in practice review is selective and driven by complaints or incidents rather than by systematic sampling. Computational analysis can measure patterns in BWV at a scale manual review cannot reach [56].
AI in policing is not limited to BWV. Knife-enabled violence is a persistent public safety problem in the UK, and officers need to identify recovered or seized knives against catalogues of known products so that they can trace sources and analyse trends, yet recovered knives are often stored with only a brief text description, which prevents systematic search. This is an object-level recognition problem with a well-defined ground truth (the catalogue identity of a knife), in contrast to the event-level problem of describing what happened in a BWV recording, where the ground truth is a structured interpretation of a complex interaction.
Research gap
Video AI has advanced rapidly, but the evidence for its use in policing is thin. We identify three gaps. (i) No evaluation of video AI on BWV: the AI tools studied in police BWV workflows work only from the audio track, analysing transcripts [59] or drafting reports [1]; to our knowledge, no published work evaluates models that analyse the video itself. (ii) Evaluation without correctness: those tools were judged on adoption and time saved, general video benchmarks test question answering [15], and accountability frameworks call for audits and documentation [17, 36, 49] without specifying task metrics. None checks, for police footage, whether each reported event actually appears in the recording at the right time, or whether words are attributed to the right speaker. (iii) Fragmented evidence: to our knowledge, no prior work evaluates object-level and event-level police AI systems side by side.
Argument
The use of CV for police tasks has so far centred on recognising objects in images, such as identifying seized weapons with KnifeHunter, while BWV is still reviewed by people watching it; the AI tools used on it so far work only from its audio [1, 59]. We argue that advances in video AI could make CV a realistic aid to BWV review that reduces its cost and time, and give pilot evidence on UKBWV-27: the best pipeline detects events with an F1 of 63.1, and every pipeline processes a minute of footage in under 39 seconds, faster than watching it. Because police records carry consequences, such use would be acceptable only if each AI-generated claim can be traced to the footage, its failures are visible, and people can contest it. We call these conditions traceability, failure visibility and contestability. They are design goals, not results: we measure properties of the outputs that each condition needs, not whether people can actually trace, spot or contest errors. For traceability, we measure how often events are unsupported by the footage, using a judge checked against blinded human labels, and how well events are located in time. For failure visibility, we report results per recording condition, so that failures are not hidden in one average. For contestability, we measure speaker attribution, because it decides whose words a record contains; whether people can contest those records is not measured. KnifeHunter is the contrasting case: its ground truth is objective and a person confirms every output.
Research questions and contributions
We address the following research questions.
RQ1 How can multimodal AI for policing be evaluated for traceability, failure visibility and contestability?
RQ2 How accurately can retrieval architectures identify knives under operational conditions, and how does accuracy change with query difficulty and gallery size?
RQ3 How well do current multimodal pipelines perform event-level BWV review tasks on a pilot set, and what are their characteristic failure modes?
Our contributions answer these questions in turn.
A pilot benchmark, UKBWV-27 and an evaluation protocol (RQ1): a protocol that checks whether each reported event can be seen or heard in the BWV, separately from accuracy, using an automated judge validated
against blinded human labels; and 27 public UK BWV recordings (118 minutes after removing news inserts and overlays) with machine-assisted, human-corrected annotations (§ 3.3.1 to § 3.3.3).
(2) The KnifeHunter dataset and KHNet (RQ2): 25,250 images of 543 catalogued knife classes with structured metadata, built with UK law enforcement; a protocol with medium and hard queries and 125,000 distractors; and KHNet, a retrieval network that fuses regionally pooled local features with Weibull-activated global features, deployed in UK policing (§ 3.2 and § 4.1).
(3) A pipeline benchmark on BWV (RQ3): our hybrid NarrativeBridge–PAL–GPT-5 pipeline against three 2026 omni-modal models (Qwen3.5-Omni-Plus [45], Nemotron 3 Nano Omni [41] and Gemini 3.1 Pro [19]), compared as complete pipelines, and the failure modes this reveals (§ 4.2).
Related Work
Body-worn video, AI and police accountability
BWV was introduced as an accountability technology, with mixed evidence: a randomised trial found use of force roughly half as likely on shifts with cameras [4], London’s trial found 33% fewer allegations per officer in camera-wearing teams [21], and a systematic review found effects inconsistent across studies [34]. Footage has also been analysed as data, for example to measure officer respect in traffic stops [56]. AI tools are now entering BWV workflows faster than they are evaluated: a system analysing BWV audio in two US departments was used routinely without its accuracy being assessed [59], and a randomised trial of a tool drafting reports from BWV audio found no change in completion time [1]. Because data-driven policing can encode and amplify existing biases [14, 50], we treat automated BWV understanding as an input to human review whose outputs must be evaluated under the conditions in which it is used.
Fine-grained visual retrieval
Identifying a knife against a catalogue is fine-grained instance retrieval: classes differ in details of blade, serration and handle, some of which determine legal classification. Instance retrieval has progressed from aggregating local descriptors [23, 24] to end-to-end global descriptors learned with generalized-mean pooling [47], learnable Weibull activations [25], token-based attention [60], self-similarity [28] and orthogonal local–global fusion [62], evaluated by mAP [46]. No public benchmark existed for knives; we build one and adapt these aggregation methods to it.
Multimodal video understanding and its evaluation
Dense video captioning localises and describes multiple events in untrimmed video [27]. Causal-temporal approaches extend captioning to describe why events occur and in what order: NarrativeBridge introduced a causal-temporal narrative benchmark and a cause-effect network for this purpose [39], and Causal-VLM addresses dense causal captioning [37]. Audio is a necessary modality for BWV, where speech carries much of the meaning of an interaction; large-scale speech recognition [48] and methods for transferring audio encoder information into language models [2] provide components for audio-aware video understanding, and recent omni-modal models accept audio and video jointly [19, 41, 45]. Evaluation has moved from n-gram and embedding metrics [55, 63] to video question-answering benchmarks [15] and language-model judges [64], which have systematic biases [58]; we therefore validate our judge against blinded human labels.
Evaluation, accountability and fairness
Accountability requires end-to-end auditing [49], documentation of models and data [17, 36] and attention to the context of use [51]; we follow this with a datasheet and per-task, per-condition results. People over-rely on automated aids [53] and do not use algorithmic advice neutrally [20], which is why we treat human review as a condition to be tested, not assumed. Disaggregated evaluation has revealed disparities in face analysis [6] and speech recognition [26], the latter directly relevant to BWV transcription. Evaluation measures should also be validated with stakeholders [40], and speech recognition audits have pitfalls of their own [35].
Methodology
Pipelines and tasks
We evaluate four police AI tasks and present a deployed knife retrieval system alongside an experimental BWV pipeline, which has not been used by police and serves only to show whether such analysis could be useful for policing, and label claims by evidence type: E1, measurements and the deployment record of KnifeHunter (Case A); E2, new measurements on UKBWV-27 (Case B); and E3, analytical argument (cross-case comparison, applications and readiness).
Object level (Case A). A photograph of a recovered or seized knife is ranked against a product catalogue, and the ranked matches with their metadata are the output. T1: identifying the knife, scored by mAP.
Event level (Case B). Events are described with a fixed list of 16 event types, police actions and interactions such as Taser use, handcuffing and restraint, and roadside stop and search (full list in Table 5, Appendix § 11). The same list is used to label the reference annotations, is given to every system, and is the only set of labels that is scored. A BWV recording passes through five stages (Figure 1).

Figure 1. Event-level BWV pipeline and where each task is scored. A fixed chapteriser divides each recording once into chapters shared by all systems. Each base system generates event claims and a transcript within those chapters, and GPT-5 maps them to the common schema without adding, removing or retiming events. Tasks T2 to T4 are scored on the post-processed output. Shaded boxes are not evaluated; human review is proposed, not tested.
| # | Event type | What it covers | Videos |
| 1 | Armed suspect confrontation (edged or blunt weapon) | Officers face someone holding a knife, hammer or similar, with shouted commands and disarming | uk01, uk04, uk08, uk12, uk14, uk15, uk22, uk25 |
| 2 | Firearms incident | Armed police stops and discharges, or arrests of suspects linked to firearms | uk16, uk23 |
| 3 | Taser use | Taser drawn or fired to stop a suspect | uk01, uk14, uk24, uk25 |
| 4 | Assault on officers | An officer is attacked during the encounter | uk07, uk08, uk12, uk15 |
| 5 | Arrest at a live scene | Arrest straight after a violent incident, while the scene is still unfolding | uk01, uk04, uk10, uk14, uk22, uk25 |
| 6 | Planned arrest at a home | Officers arrive at an address, read the caution and detain the person | uk17, uk19, uk20 |
| 7 | Arrest after a search or manhunt | Suspect found and arrested in public after a search | uk18, uk23 |
| 8 | Handcuffing and physical restraint | Cuffing, holding on the ground, prolonged restraint | uk03, uk04, uk05, uk13, uk21, uk26, uk27 |
| 9 | Contested or excessive use of force (video-level tag, not scored) | Force later challenged at court, a misconduct hearing or an inquest | uk03, uk07, uk21, uk24, uk26, uk27 |
| 10 | Vehicle pursuit and tactical contact | Chases, ramming to stop a vehicle, boxing in, failure to stop | uk02, uk11, uk15 |
| 11 | Foot chase and decamp | Suspect leaves the vehicle and runs; dog or air support tracks them | uk02, uk11 |
| 12 | Roadside vehicle stop and search | Car pulled over, occupants removed, searched or questioned | uk16, uk21, uk24 |
| 13 | Public disorder | Riot, a fight outside a nightclub, crowd violence | uk09, uk10 |
| 14 | Scene interaction with suspects or witnesses | Officers talking to people at a scene, sometimes not yet knowing who the offender is | uk06, uk09, uk22 |
| 15 | Injured person or medical emergency | Victim care, first aid, a person in distress while restrained | uk05, uk13, uk27 |
| 16 | Victim mistaken for a suspect | The person being restrained turns out to be the victim | uk05, uk13 |
| 17 | In-vehicle custody exchange | Suspect in a police car giving details or asking about the cuffs | uk09 |
Table 5. Event types in UKBWV-27, with the recordings in which each occurs. Type 9 (shaded) is a video-level tag taken from the publication context and is not scored.
Input. Frames and the audio track of the recording.
Chapterisation. A fixed model divides the recording once into chapters, which give every system the same context intervals and are not evaluated.
(3) Generation. In each chapter, a base system lists zero or more events with their start and end times. For each event, it suggests one of the 16 event types and says, by role only, who did it and to whom (for example, an officer restraining a member of the public); systems that output only free-text descriptions, such as NarrativeBridge in our hybrid pipeline, leave these to stage 4. It also transcribes the speech, labelling each line as said by an officer, a member of the public or someone else.
(4) Post-processing. GPT-5 maps each event to one of 16 event types or to unmapped, settles actor and target roles and links utterances to events, without adding, removing or retiming events.
(5) Human review. A reviewer would check the outputs against the footage; this stage is proposed (§ 6).
The outputs of stage 4 are scored as three tasks, which were identified from meetings with two staff members at the partner police service, who explained how BWV is currently reviewed, what a review tool would have to do and where video understanding could help. T2: detecting and locating events to triage videos, scored by event F1 and mIoU. T3: reporting events without unsupported claims, scored by the unsupported-event rate. T4: transcribing speech and attributing it to the right speaker, scored by WER and speaker accuracy. Each task’s output carries a time interval, so a reviewer can go from any claim to the part of BWV that should support it.
Case A: fine-grained knife retrieval (KnifeHunter)
Dataset. We built the KnifeHunter dataset with a UK police service. Its database contains 25,250 images of 543 catalogued knife classes from three sources: 332 classes photographed in a police evidence store with phones and DSLR cameras under varied viewpoints, lighting and backgrounds; 171 classes from a UK retailer’s product images, with augmentation; and 40 classes seized by UK Border Force. Each class carries ten metadata fields developed with police forces, manufacturers and retailers, including knife type, manufacturer, product name, a flag for whether it could be an offensive weapon under UK law, blade length, material and stockist.
Evaluation protocol. The Medium set has 443 query images with moderate viewpoint change, partial occlusion, illumination change and clutter, some from knife-crime operations. The Hard set has 150 query images, many from knife-crime investigations, with unusual angles, heavy occlusion, clutter, extreme lighting or inverted knives. A distractor set of about 125,000 images, combining knife images supplied by nine UK police forces (none of them in the dataset) with unlabelled COCO images [31], can be added to the database (the +D protocol). Each query is ranked against the database by cosine similarity of global descriptors. With indicating whether the item at rank shares the query’s class, the number of relevant items and the precision of the top items, we report mean average precision [46]:
Architectures. Each model is a backbone (EfficientNet-B7 [54], ConvNeXt-Base [33], SwinV2-Base [32] or EVA02-Base [13]) followed by an aggregation head: GeMNet [47], ActNet [25], TokenNet [60], SENet [28] or our KHNet, which fuses regionally pooled local features with Weibull-activated global features by orthogonal projection [62] (Appendix § 18). All models are trained end to end with sub-center ArcFace [10].
Deployment. KnifeHunter runs on a cloud CPU node that processes up to 50 images per second. Officers upload a photograph through a web page and receive an automatically generated report of the retrieved knife details. It was used in two rounds of a national knife-crime campaign and by forces across the UK. Its deployment record documents use, not field accuracy, which has not been measured.
Case B: experimental multimodal BWV understanding pipeline
Formal definition. A BWV recording is represented by sampled frames, the audio signal and metadata. The pipeline of § 3.1 produces an output set , where is the set of predicted events pooled over all chapters, each with label and an interval inside its chapter; and is a transcript whose utterances carry a speaker role and timestamps. Every element carries a time interval, which we call its evidence anchor. A chapter is a shared context interval, not a predicted event: the event is the unit of evaluation for detection, temporal overlap and support, and chapters themselves are not scored.
Common chapterisation. The chapteriser is ARC-Chapter [44] (on Qwen2.5-VL-7B), run once per recording on its frames and audio and frozen before evaluation; its chapter identifiers and timestamps are supplied unchanged to every base system, and it is not part of the model comparison (Appendix § 13). Models and inference are specified in § 4.2. Faces are not redacted before model input because all footage is already public.
Pilot benchmark: UKBWV-27
Scope. UKBWV-27 is a pilot stress-test set, not a representative sample of UK BWV, and its results do not show operational reliability. Drawn from public releases, it over-represents high-profile, severe and edited releases and under-represents routine encounters, and publisher editing may affect performance.
Sources and selection. UKBWV-27 contains 27 recordings from 16 UK police forces, totalling 118 minutes (26 s to 15 min 44 s; median 2 min 26 s), published between 2017 and 2026: 3 by police forces and 24 by news outlets, including footage shown in court, at an inquest and at a misconduct hearing. All 27 are used for benchmarking; there is no training split, because no model is trained or fine-tuned on BWV. Selection criteria are: (i) consisting wholly or mainly of body-worn camera footage; (ii) published by a police force or a news outlet; (iii) containing at least one interaction between an officer and a member of the public; and (iv) covering variation in lighting, camera motion, occlusion, audio quality and number of people. Sources, release terms and jurisdictions are recorded per video in the datasheet (Appendix § 17) [17]. Summary statistics are given in Appendix § 17 (Table 11).
| UKBWV-27 | |
| Videos | 27 |
| Police forces | 16 |
| Published by police / news outlets | 3 / 24 |
| Upload years | 2017–2026 |
| Duration range | 0:26 to 15:44 |
| Total duration (min) | 118 |
| Mean / median duration (min:s) | 4:22 / 2:26 |
| Event types (scored / video-level tag) | 16 / 1 |
| Recordings with contested-force tag | 6 |
| Annotated events | 53 |
Table 11. UKBWV-27 statistics.
Preparation. Because most recordings were republished by news outlets, we removed, before evaluation, every segment not filmed by a body-worn camera (such as studio, CCTV or helicopter inserts) and every segment carrying broadcaster overlays such as logos or captions, so that models received only clean BWV. Segments were identified from the recording’s visual source and editing properties only, not from the incident, events or headline. All durations above refer to the retained footage. Recordings are still grouped by how they were released (22 as pure BWV, 3 with overlays, 2 combined with other sources), because publisher editing can leave cuts and compression in the retained footage. The recordings and news coverage of these incidents are public, so hosted models may have seen them in training, and results may not reflect perception of the footage alone.
Annotation protocol. Each video is annotated with a speech transcript, with speaker roles (officer, member of the public, other or unknown) and timestamps, and with events, each an interval labelled with one of 16 event types (Appendix § 11, Table 5), plus an unscored video-level tag, taken from the publication source later challenged in court or at a hearing or inquest. Events are annotated only if visible or clearly audible in body-worn footage, last at least 1 s and may overlap; inclusion rules and borderline cases are in Appendix § 11. Two open omni-modal models, Baichuan-Omni-1.5 [29] and MiMo-V2.5 [61], generated initial event annotations, which two human annotators then reviewed and corrected against the video and audio. These are machine-assisted, human-corrected reference annotations, not gold-standard labels. Correcting model proposals can anchor annotators, omit events the models did not propose, and carry over model boundaries and interpretations, especially for force, restraint and actor and target roles. Annotators did not first annotate the raw footage independently, agreement before adjudication was not measured, and the number of events added, removed or changed after pre-annotation is not reported. The reference set is complete only with respect to the 16 types. Both pre-annotation models are excluded from the evaluated panel. Transcripts were initialised with Whisper [48] and then corrected by the same annotators against the audio. Annotators do not label perceived race, ethnicity, gender or age of people in the footage.
Metrics for event-level tasks
Let denote the temporal intersection over union of two intervals.
Event detection and temporal localisation (). Event-level metrics are computed over the union of predicted events produced within all fixed chapters. A chapter may contain zero, one, or multiple predicted events. Predicted events from different chapters are pooled and matched one-to-one with human reference events using Hungarian matching, temporal IoU , and equivalent event labels. Events may overlap or nest, including within the same chapter. Predicted labels are assigned by the common GPT-5 textual post-processing and taxonomy-mapping stage (Section 4.2.3), which is applied only to model outputs, not to reference annotations; an event mapped to unmapped is excluded from matches. Event (Eq. (3)) is computed over the 16 scored types. Each predicted event interval must lie within its enclosing chapter interval. Although chapter boundaries are shared across systems, predicted event boundaries are generated independently within each chapter (Appendix § 12). Thus mIoU, the mean tIoU of matched events, measures the temporal overlap of model-predicted events with matched reference events; it is not determined solely by the common chapterisation. A system matching fewer events can still score a high mIoU, so both are read together.
Unsupported events (). Each predicted event is labelled as supported or unsupported by the footage, , by an automated judge validated against human labels (§ 3.3.3). The unsupported-event rate is
The unsupported-event rate is computed over all predicted event records, including multiple events from the same chapter. The chapter identifier is retained for provenance but does not affect the denominator. Unsupported events are the errors most likely to enter a record as fact. Because can be lowered by reporting fewer events, it is read with event ; we do not report a risk–coverage analysis.
Audio and speech (). Word error rate (WER) compares each model’s transcript with the human-corrected reference after both are normalised with the Whisper English text normaliser [48], which lower-cases text, removes punctuation and standardises spellings and numbers; non-speech tags (for example [shouting] or [siren]) are removed, names are scored as spelled in the reference, and unintelligible reference segments are excluded. Speaker accuracy is the share of reference utterances whose role (officer, member of the public or other) is assigned correctly. Each reference utterance is aligned to the predicted utterance with the greatest temporal overlap, and an utterance with no overlapping prediction counts as an error. Where speech overlaps, each reference utterance is scored separately and a predicted utterance can be aligned to only one of them. Reference utterances whose speaker the annotators marked as unknown are excluded.
We do not combine these metrics into a single score. A composite would allow accurate transcription to mask unsupported events, the aggregation a task-grounded approach avoids.
Automated judge and human validation
Support is labelled for every predicted event, after GPT-5 post-processing, by a language-model judge, Doubao Seed 2.0 Pro [7], given the recording’s video and audio and one normalised event with its interval, actor and target roles and description (Appendix § 16). The judge is a different model from GPT-5, the evaluated systems and the pre-annotation models, but may share failure modes with them; because such judges have documented biases [58, 64], we validate it against human labels, and it is excluded from the evaluated panel. We validated event support using a blinded, stratified human-annotation protocol. The validation sample contained 100 predicted events, 25 from each of the four pipelines run with audio (C2). Two annotators, blind to model identity, reference annotation and judge output, labelled each prediction against the video and audio, with disagreements adjudicated (Appendix § 14). For the primary binary analysis, supported events were treated as supported; partially supported, uncertain and duplicate events were treated conservatively as unsupported, the same rule the judge applies. We report the confusion matrix, class-wise precision and recall, macro-F1 and Cohen’s [9]. Support judgements are made independently of reference annotations. The sample is a diagnostic sample, not an estimate of prevalence: its precision and recall are conditional on its stratified composition, agreement between the two annotators before adjudication is not reported, and the rates in Table 2 come from the judge’s labels on all predicted events (Appendix § 14).
Table 2. Main results on UKBWV-27 (27 videos). WER: word error rate against the human-corrected transcript. Speaker: speaker accuracy. F1: event F1 at , weighted by video count over the release conditions of Table 3 (22 pure BWV, 3 with overlays, 2 with other sources). n/a: not applicable without audio. mIoU: temporal IoU of matched predicted and reference events; chapters are shared context intervals, but event intervals are generated independently within chapters. Unsup.: share of predicted events the validated judge labels unsupported by the footage (Eq. (2)). Latency: wall-clock seconds per minute of footage, including the shared chapterisation run, the GPT-5 post-processing step and network and queueing for hosted API calls, so not directly comparable across access modes. Best per column in bold. Models above the line use condition C2 (vision with audio); the last row is NarrativeBridge with GPT-5 post-processing but without the PAL audio branch (C1). All rows pass through the same GPT-5 post-processing.
| Model | WER %↓ | Speaker %↑ | F1↑ | mIoU↑ | Unsup. %↓ | Latency↓ |
| Hybrid pipeline (ours) | 5.7 | 72.3 | 63.1 | 0.44 | 13.6 | 37.3 |
| Gemini 3.1 Pro | 7.9 | 69.2 | 60.6 | 0.35 | 14.2 | 38.5 |
| Qwen3.5-Omni-Plus | 12.5 | 63.8 | 52.0 | 0.41 | 21.7 | 24.1 |
| Nemotron 3 Nano Omni | 13.7 | 52.4 | 49.7 | 0.38 | 29.5 | 29.8 |
| NarrativeBridge + GPT-5 (C1) | n/a | n/a | 56.5 | 0.39 | 13.9 | 37.9 |
Table 2.
Analysis plan
With 27 videos we report descriptive comparisons rather than significance tests: the main table averages event F1 over release conditions weighted by video count, and the condition breakdown pools events within each condition. We do not disaggregate by demographic attributes, because inferring them from footage raises known concerns [6].
Experiments and Results
This section reports the object-level results for Case A (RQ2), the benchmark of event-level pipelines for Case B (RQ3), the validation of the automated judge, and the cross-case comparison.
Case A: object-level knife retrieval (E1)
EVA02-Base was the strongest of the four backbones with every protocol (Table 12, Appendix § 18), so Table 1 compares aggregation heads on it.
Table 1. Knife retrieval mAP (%) on KnifeHunter with the EVA02-Base backbone. +D: with about 125,000 distractor images.
Table 12. Backbone comparison with GeMNet on KnifeHunter (mAP, %).
Three observations follow. First, query difficulty dominates model choice: KHNet mAP falls 21.2 points from Medium to Hard (86.5 to 65.3), while its gain over GeMNet on the same backbone is 4.1 points on Hard. Second, model choice matters most in the hardest condition: on Hard+D, KHNet exceeds GeMNet by 5.5 points and SENet by 1.9, and the backbone matters even more, with EVA02-Base 12.4 points above EfficientNet-B7 under GeMNet. Third, distractors cost KHNet 0.8 points on Medium and 1.9 on Hard, so gallery scale matters more when query evidence is weaker. Among the components KHNet adds to the backbone, the ablation (Appendix § 18) shows the largest gain on Hard+D from orthogonal fusion of local and global features (58.9 to 62.1). Class activation maps show attention on the blade, the handle or both, and a failure mode in which activation falls on other metallic objects such as axes or knuckle-dusters.
These results have a direct bearing on accountability. The Hard queries come from investigations, which is where a wrong match matters most, and there mAP is 21 to 22 points below the Medium figure that would usually be quoted. The deployment record shows that the system is used, not how often its top match is correct in the field. Open-set queries, calibration of match scores and near-duplicate control between database and queries are not analysed here. A wrong top match would carry wrong metadata, including the offensive-weapon flag, into an intelligence product, so these results support condition-specific reporting and an analyst between the ranked list and any product.
Case B: BWV pipeline benchmark
This benchmark addresses RQ3. It compares complete pipelines, not base models, on all 27 UKBWV-27 recordings, without any training on BWV. It uses the event-level metrics of § 3.3.2, and the protocol was fixed before evaluation.
Model panel. The panel (Table 9, Appendix § 15) compares our hybrid pipeline with three omni-modal models released in 2026 that accept video with its audio track. They were selected to span the access options open to a police service, from open weights that can run on premises to hosted APIs. Our hybrid pipeline, NarrativeBridge+PAL with GPT-5 post-processing, runs two components in parallel on each recording: NarrativeBridge’s cause-effect network [39] on the frames, producing timestamped causal-temporal event descriptions within each supplied chapter, and PAL [2], with a Llama 3.2 3B language model, on the audio track, producing a transcription. NarrativeBridge and PAL run locally. Their outputs then pass through the common GPT-5 textual post-processing and taxonomy-mapping stage (Section 4.2.3), which here also combines them. NarrativeBridge alone is the vision-only comparison. Qwen3.5-Omni-Plus [45] supports up to 400 seconds of 720p video at 1 frame per second. Nemotron 3 Nano Omni [41] is an open-weight 30B-A3B mixture-of-experts model and the only general-purpose base model in the panel that can run on premises. Gemini 3.1 Pro [19] is accessed through an API and accepts up to 1M input tokens. Although some models could take longer inputs, every system processes recordings in the same 120 s windows (Appendix § 13), so all receive the same frames.
| Model | Size | Access | Input | Reference |
| NarrativeBridge | ≈80M | Local | V | [39] |
| PAL + Llama 3.2 3B | ≈3B | Local | A | [2] |
| GPT-5 post-processing (all systems) | n/d | API | Base-system text | [42] |
| Hybrid pipeline (ours) | – | Hybrid local/API | V+A | This paper |
| Qwen3.5-Omni-Plus | n/d | API | V+A | [45] |
| Nemotron 3 Nano Omni | 30B-A3B | Open weights | V+A | [41] |
| Gemini 3.1 Pro | n/d | API | V+A | [19] |
Table 9. Model panel. Input: V = frames, A = native audio track. n/d: not disclosed by the provider. The NarrativeBridge (CEN) size is estimated from its released 32-bit checkpoint (317 MB) and excludes the two CLIP4Clip video encoders that supply its features. The hybrid pipeline’s NarrativeBridge and PAL components run locally. Every system’s outputs pass through the common GPT-5 textual post-processing and taxonomy-mapping stage through an API.
Appendix § 15 gives the reported results that motivated the choice of NarrativeBridge and PAL. The two models used for pre-annotation (§ 3.3.1) and the judge model (§ 3.3.3) are excluded from the panel.
Input conditions. Models are evaluated under two input conditions: vision only (C1) and vision with the native audio track (C2). NarrativeBridge covers C1 and the hybrid pipeline covers C2. The comparison measures the effect of adding the PAL audio branch and subsequent fusion within our pipeline; it does not isolate audio, since the PAL transcript also informs GPT-5’s mapping and the shared chapters in both conditions come from video and audio.
Protocol and controls. All systems receive the same fixed chapters (§ 3.3), but they independently generate zero or more event claims within each chapter. Each chapter is a shared context interval; each system may produce multiple event claims within it, each with an interval, description and label and role proposals, plus a transcript. common GPT-5 [42] textual post-processing and taxonomy-mapping stage applies the same prompt to every base system. It receives these outputs, the transcript, and the 16 event types, but not the original recording. It may revise each event’s label and roles, but cannot create, delete, merge or split events or change event or chapter timestamps. Results therefore compare complete base-system-plus-post-processing pipelines rather than raw base models in isolation (prompt in Appendix § 16). The GPT-5 stage may normalise outputs towards a shared style, correct or introduce label errors and narrow differences between base systems. Systems also differ in native input handling, context length, windowing, decoding defaults and access mode, and each was run once, so run-to-run variation is not measured. The ARC-Chapter chapters are part of the benchmark condition. We report no ablations without post-processing, without NarrativeBridge or with alternative chapters. Inference settings, windowing and refusal handling are given in Appendix § 13.
Reported metrics. WER, speaker accuracy and mIoU are defined in § 3.3.2. Event uses one-to-one matching at tIoU (Eq. (3)), averaged over release conditions weighted by video count, over the 16 scored event types. Unsupported events is (Eq. (2)). Latency is wall-clock time from submitting a recording to receiving the complete post-processed output, per minute of footage, so it includes the shared ARC-Chapter run and the GPT-5 step for every system. Local models run on one workstation (Appendix § 13); for hosted models it includes network transfer and queueing, which we do not control (endpoint regions in Appendix § 13). Latency is therefore indicative and not directly comparable between local and hosted models.
Hypotheses. Stated before evaluation: (H1) Performance falls on edited, long and noisy recordings. (H2) The hybrid pipeline outperforms the general omni models on event detection and speaker attribution.
Results
Overall. On this pilot set, no pipeline is reliable enough to stand in for a reviewer. Even the strongest leaves 13.6% of predicted events unsupported by the footage and misattributes 27.7% of utterances, and the general models leave up to 29.5% of events unsupported (Table 2). Differences are descriptive; we report neither significance tests nor confidence intervals. Our hybrid pipeline is best on five of the six measures: the lowest WER (5.7% against 7.9% for Gemini 3.1 Pro), the highest speaker accuracy (72.3% against 69.2%), the highest event (63.1 against 60.6), the best temporal localisation (mIoU 0.44 against 0.41 for Qwen3.5-Omni-Plus) and the fewest unsupported events (13.6% against 13.9% without PAL). Qwen3.5-Omni-Plus records the lowest latency (24.1 against 37.3 seconds per minute of footage), although latency is not directly comparable between local and hosted models. Without the PAL branch (C1), falls to 56.5, below Gemini, and mIoU to 0.39.
Results by recording condition. Table 3 breaks event at tIoU $\ge 0.3 down by how the recording was released, duration and audio; for edited releases only the retained BVW was scored. The hybrid pipeline scores highest in every condition, descriptively (2 to 22 videos per condition), and all pipelines degrade in the same places: recordings released with overlays score 9.3 (hybrid pipeline) to 12.2 (Qwen) points lower even after the overlaid segments were removed, and long recordings score 12.7 (Gemini) to 16.9 (Qwen) points below short ones. Recordings dominated by shouting or crowd noise are the hardest for every model (37.8 to 49.7).
Table 3. Event F1 (%) at on UKBWV-27 by recording condition, pooled over events.
| Condition | Videos | Hybrid | Gemini | Qwen | Nemotron |
| Released as pure BWV | 22 | 64.7 | 62.4 | 54.0 | 51.6 |
| Released with overlays | 3 | 55.4 | 51.3 | 41.8 | 39.6 |
| Released with other sources | 2 | 56.7 | 55.0 | 44.6 | 44.3 |
| Short (< 2 min) | 8 | 68.3 | 64.8 | 57.2 | 53.6 |
| Medium (2–5 min) | 12 | 58.9 | 50.4 | 42.1 | 40.8 |
| Long (> 5 min) | 7 | 53.0 | 52.1 | 40.3 | 38.4 |
| Mostly shouting or crowd audio | 3 | 49.7 | 44.9 | 38.2 | 37.8 |
Table 3.
Findings. On this sample, the results are consistent with H1 for recordings from edited releases, long recordings and noisy audio, and with H2 for event detection and speaker attribution; 27 videos cannot confirm either in general.
Judge validation
Under the conservative rule the 100 validation events comprise 75 supported and 25 unsupported (15 unsupported, 5 partially supported and 5 uncertain), so the uncertainty rate is 5%. The judge agrees with the settled human labels on 96 events (, 95% bootstrap CI 0.78–0.98; Appendix § 14). For supported events, precision is 98.6% and recall 96.0%; for unsupported events, recall is 96.0% and precision 88.9%; macro-F1 is 0.95. The false-positive rate is 4.0% (Wilson 95% CI 0.7–19.5%): one of the 25 human-unsupported events was accepted as supported, the error that would let an unsupported statement through, and with 25 such events the judge could plausibly accept up to one in five. The lower unsupported precision reflects class imbalance, since the three supported events the judge rejected count against a class of only 25. The rates in Table 2 therefore rest on a validated but imperfect instrument, whose error on unsupported events is poorly constrained and was not assessed by event type, condition or model.
Cross-case comparison (E3)
The cases differ in ground truth, metric maturity, field validation, the human role and who is affected by errors (Table 4).
| Property | Case A: knife retrieval | Case B: BWV understanding |
| Ground truth | Catalogue identity; objective and verifiable | Structured interpretation of an interaction; partly contestable |
| Output form | Ranked list of catalogue items | Free text with intervals |
| Metric maturity | Established retrieval metric (mAP) | Task metrics defined in this paper; support labelled by a judge validated against blinded human labels |
| Field validation | Deployed by UK forces; field accuracy not measured | None; experimental, not used by police; pilot benchmark on public footage only |
| Human role | Analyst confirms top match | Reviewer must verify each event used |
| Main error risk | Plausible wrong match under occlusion or open-set conditions | Fluent, unsupported or misattributed statements |
| Affected parties | Mainly intelligence products about objects | Officers and members of the public described in footage |
Table 4. Analytical comparison of the two cases.
Discussion
Positioning
Computer vision has documented ties to military and state surveillance [16], and evaluating models for police use risks lending them legitimacy. Here no task identifies individuals and speaker attribution is by role. Because auditing policies can rest on flawed assumptions [12] and benchmark metrics shift [5], we evaluate per task and condition, informed by a police partner [11]; affected communities and civil-liberties or defence-law experts were not involved, which narrows the perspectives behind the tasks.
What the protocol shows about accountability
We answer seven questions the measurements raise; the answers are interpretations (E3) and, for the experimental BWV pipeline, describe what would follow if its outputs were adopted.
The institutional failure. BWV was adopted to make police conduct reviewable, but footage outgrows review. AI tools proposed to close that gap have been judged on acceptance and time saved, not on whether outputs are true to the evidence [1, 59]. The protocol shows what this hides: models that rank well still leave 13.6% to 29.5% of predicted events unsupported and misattribute 27.7% to 47.6% of utterances, errors that would pass into records unless a workflow required someone to catch them.
Who is burdened, and which records can change. If adopted, such systems would give institutions scale, reviewers would verify every output, and the people in the footage, who rarely see output or recording, would bear the cost of what is missed. Unsupported outputs could enter incident reports, prosecution case summaries, use-of-force and misconduct records, disclosure schedules and exhibit and knife-intelligence records.
How a claim can be contested. Contesting an AI-generated claim requires knowing that it was generated, its cited interval and footage, the model version and output log, and the condition’s error rates. An analyst can compare a KnifeHunter match with the physical knife. If BWV outputs were used, an officer could check a transcript against the recording, a prosecutor could request the anchored segment, a defence solicitor could seek disclosure of outputs and error rates, and a member of the public could use the complaints process. The protocol supplies the anchors and per-condition error rates these steps need; whether workflows disclose them is not measured.
Power and procedure. Automated review would add capacity mainly on one side of an existing evidential asymmetry: the police hold the footage, outputs and logs, while defendants, witnesses and complainants see them only through disclosure. Event labels can lend a contested police account an appearance of objectivity, and the same tools could extend surveillance rather than support review. Accountability would depend on governance we do not evaluate: access to outputs and logs, retention, notice to affected people, correction and appeal, and independent audit of error rates. Data-driven policing can encode existing disparities [50], and speech recognition errors differ across speaker groups [26]; not annotating demographic attributes protects privacy but leaves disparate impact unevaluated, so finding no disparity here is not evidence of none.
What “support” means. Support is relative to the recording, which is partial, recorded from the officer’s viewpoint, often trimmed or re-encoded by its publisher (24 of 27 from news outlets) and noisy. A supported event is faithful to the record, not necessarily to what happened.
Does anchoring reduce automation bias? Anchoring makes checking cheap but can make an incorrect output look checked, as can the automated support check (4.0% false-positive rate). Reviewer behaviour was not measured, so anchoring is not shown to prevent over-reliance [20, 53].
Which errors harm whom. In records, the worst errors would be, for suspects, an unsupported act of aggression or another speaker’s threats attributed to them; for officers, unsupported descriptions of force; for witnesses, misattributed statements; and for communities, degradation on shouting or crowd audio (event F1 37.8 to 49.7), as at protests.
Applications in Policing
KnifeHunter is deployed; the BWV pipeline has not been used or trialled by police, so its uses are prospective. We propose a readiness level for each task, as a governance proposal rather than a validated workflow: Assistive, where a person checks the output before relying on it, or Verify, where every output used is checked against the source (E3).
Knife identification (T1, Assistive). An officer photographs a recovered knife and receives ranked catalogue matches, supporting exhibit records, retailer tracing and trend analysis. With 63.4% mAP on hard queries and field accuracy unmeasured, an analyst confirms each match, a lead, not evidence.
BWV review (T2–T4, Verify). Event-level outputs could reduce, not replace, watching footage. Timestamped events would let reviewers jump to a search, arrest or use of force, transcripts would start case files, and footage could be sampled for supervision, not only after complaints. Given these error rates, every output entering a record would need verifying against the footage, which may shift rather than remove the review burden; verification time, workload and error detection were not measured. Outputs should not inform probable cause, arrest, force review, disciplinary findings or evidential conclusions without independent human and legal review. Figure 2 illustrates how event detection could locate a knife for KnifeHunter to identify; this combined use is unevaluated.

Figure 2. Illustrative application to a knife-related incident. Event level: NarrativeBridge and PAL run in parallel on the frames and the audio of a BWV recording, and GPT-5 fuses their text outputs, without adding events or changing timestamps, into one record of events and speaker-attributed utterances, which a reviewer verifies against the footage. A verified event in which a knife is recovered prompts a photograph of the knife. Object level: KHNet ranks catalogue matches, which an analyst confirms. Both feed a case record that links the account of the incident to the exhibit. Shaded boxes are human decisions. The BWV pipeline is experimental and has not been used by police, and the combined pipeline has not been evaluated.
Limitations
(1) UKBWV-27 is a small pilot set of selected, publicly released footage; it supports descriptive comparison, not generalisation, also informed prompt design, and we report no confidence intervals or per-recording results for the benchmark. (2) References are machine-assisted and human-corrected, without independent annotation or agreement statistics; WER may favour transcripts resembling Whisper’s [35]. (3) The judge was validated on 100 stratified events, not at natural prevalence or by event type and condition. (4) Pipelines were run once, without ablations of post-processing, components or chapters, and errors are not attributed to stages. (5) Contamination by public videos and reports is possible. (6) There is no reviewer study, affected-party input, per-event-type analysis or cost and energy accounting.
(7) KnifeHunter’s field accuracy, open-set queries and calibration are not evaluated. (8) Group differences are unmeasured.
Conclusion and Future Work
We argued that advances in video AI could make CV a realistic aid to BWV review if its claims are traceable, its failures visible and its outputs contestable, and contributed an evidence-grounded protocol that measures output-level evidence for these conditions at two levels. At the object level, KHNet reaches 85.7% and 63.4% mAP on the medium and hard KnifeHunter protocols with distractors, a 22.3-point drop. On the UKBWV-27 pilot set, our hybrid pipeline leads on five of six measures (event F1 63.1 against 60.6 for Gemini 3.1 Pro; 56.5 without PAL), suggesting that such analysis could be useful for policing. Yet even the best pipeline leaves 13.6% of events unsupported by the recording, which is not the same as untrue, and misattributes 27.7% of utterances. Future work: independent annotation with agreement statistics, a larger systematically sampled set, judge validation at natural prevalence, pipeline ablations, a reviewer study and consultation with affected communities.
Acknowledgments
This work was supported by the Open-Source AI Fellowship, a partnership between The Alan Turing Institute, the Incubator for AI (i.AI) and Meta.
Ethical Considerations Statement
This work concerns AI for police work, which affects officers and members of the public who have not consented to being analysed. The study was reviewed by the Incubator for AI (i.AI) and the University of Surrey, neither of which reported ethical concerns.
Data. We collected no new data from people and used no police-held footage: every BWV recording was already published by a police force or news outlet. Public availability is not consent, especially for victims, young people and people filmed at traumatic moments, so we name no one, infer no demographic attributes and redistribute no video, releasing only source links and annotations. A recording withdrawn by its publisher is removed at the next release, with results recomputed, and people shown can request removal or correction of annotations (Appendix § 17). The knife dataset contains images of objects, not people.
Hosted models. Only public footage was sent to providers, through the regions listed in Appendix § 13, under each provider’s standard terms. Gemini 3.1 Pro and GPT-5 were used through paid accounts, whose terms exclude training on API data [18, 43]; GPT-5 received text only. Qwen3.5-Omni-Plus and the judge were used mainly through free quotas; Alibaba Cloud states that it retains inference data for at most 30 days and does not train on it without consent [3], and we did not verify whether free use changes these terms.
Risks. Benchmark results could be used to justify automating decisions that should stay human, or to extend surveillance beyond accountability. We therefore report results per task, keep all event-level outputs at the Verify level (§ 6), and state that outputs should not inform arrests, force reviews, disciplinary findings or evidential conclusions without independent human and legal review. Because system errors could change the account of what people in the footage did, any deployment would need notice, access and correction mechanisms for them.
Generative AI Usage Statement
A generative AI assistant was used to help structure and draft sections of this manuscript and to format the bibliography. All technical content, results and citations were checked by the authors against primary sources, and the authors take full responsibility for the paper. Processing Systems (NeurIPS), Datasets and Benchmarks Track.
Annotation Guidelines
Event types. Each event is labelled with one of the 16 scored types in Table 5. Every base system’s predicted events are mapped to one of the 16 scored types, or to unmapped, by the common GPT-5 textual post-processing and taxonomy-mapping stage (Section 4.2.3), which is not given type 9, so a predicted and a reference event have equivalent labels when their types are the same. Type 9 is a video-level tag, not an event type: it is recorded from the publication context in the datasheet, is not annotated as an interval, and is excluded from event matching, event F1 and the unsupported-event rate.
Inclusion and exclusion. An event is annotated if it is visible in body-worn footage or clearly audible on its soundtrack. Excluded are events known only from narration, captions or on-screen text, and events shown only in news-studio, CCTV or other non-body-worn inserts, which were removed before evaluation. Labels describe what happens, not motive, emotion or lawfulness. Where an edited release repeats the same moment, each occurrence is annotated and the repeat is flagged.
Duration and boundaries. An event lasts at least 1 s; an instantaneous action, such as a Taser discharge, is given a 1 s interval starting at the action. Boundaries are marked to the nearest second, from the first to the last frame or sound in which the event is perceptible. Two events of the same type less than 2 s apart are merged.
Overlapping events. Events often overlap or nest, because the types describe different aspects of an encounter. In uk01, for example, an armed confrontation (type 1), Taser use (type 3) and an arrest at a live scene (type 5) occur in sequence and partly overlap, and a restraint (type 8) may involve an injured person (type 15). Each event is annotated with its own interval and matched separately under one-to-one matching.
Borderline cases. (1) Arrest types: an arrest is labelled type 5 when it follows a violent incident still unfolding on camera, type 6 when officers arrive at an address to detain a named person, and type 7 when the person is found after a search shown or referred to in the footage; when two apply, the annotator chooses the type the footage shows most directly. (2) Restraint versus guiding: brief contact to guide a compliant person is not labelled; it becomes type 8 only when the officer holds the person so as to restrict movement for at least 1 s. (3) Victim mistaken for a suspect: type 16 is labelled only from the point at which the footage itself shows or states that the restrained person is the victim, for example when an officer says so; before that point the restraint is labelled type 8 only.
Event support. For each predicted event in the judge validation sample, annotators record its support status (supported, partially supported, unsupported or uncertain), the event type observed (Table 5), whether its timing is adequate, whether its actor and target roles are correct and whether it contains unsupported detail, without knowing which model produced it or the judge’s label.
Source segments. Before evaluation, segments are marked as clean body-worn footage, body-worn footage with broadcaster overlays, or non-body-worn footage, using only the recording’s visual source and editing properties, never the incident type, events, transcript or publication headline. Only clean body-worn segments are retained.
Speaker roles. Each utterance is assigned to officer, member of the public, other, or unknown. Annotators record “unknown” when the speaker cannot be determined from the footage.
Prohibited labels. Annotators do not record perceived race, ethnicity, gender, age, religion or disability.
Event-Level Metrics
What each metric measures. We keep four properties separate: event F1 combines detection and classification at a lenient localisation threshold, mIoU measures temporal localisation, and measures evidential support, whether a predicted event occurs in the footage at all.
Chapters and events. Chapter intervals are fixed context boundaries; event intervals are model outputs. The chapteriser does not determine the event interval, so temporal localisation is model-specific because event intervals can differ between systems. Predicted events are pooled across all chapters of a recording before matching, and both predicted and reference events may overlap or nest within a chapter.
Event detection (). A predicted event matches a reference event if and and map to the same event type in Table 5; the video-level tag (type 9) is excluded. Under one-to-one Hungarian matching,
Benchmark Protocol Details
Common chapterisation. ARC-Chapter [44] was run once per video on its frames and audio track and frozen before any comparison system was evaluated. Chapterisation was not repeated per system and received no outputs from evaluated systems. ARC-Chapter was not fine-tuned or prompted with UKBWV-27 annotations; it was trained on web videos with creator-provided chapters, so overlap with these public recordings cannot be ruled out, and its boundaries may suit edited, broadcast-style structure better than unedited BWV. Because every predicted event must lie within its enclosing chapter, a reference event that crosses a chapter boundary can be matched only by a predicted event inside one chapter, so chapterisation can limit attainable tIoU equally for every system.
Frames and context. Frames are sampled uniformly at 1 frame per second with a maximum of frames per model request. Recordings longer than 120 seconds are divided into windows of seconds with an overlap of seconds. Thus, each window contains at most 120 sampled frames, leaving eight frame slots below the 128-frame cap; timestamp text and generated output consume separate context-token budget. Window outputs are merged by chapter identifier using global timestamps; chapters cannot be duplicated or re-bounded by windowing, and every event interval is kept within its enclosing chapter. For transcripts, overlapping text is deduplicated using timestamp overlap rather than simple string concatenation. For models without native temporal encoding, each frame is preceded by its global timestamp so that outputs can cite intervals.
Prompting and decoding. The omni models receive the same instruction and JSON output schema (Appendix § 16), with the default decoding settings for each API. NarrativeBridge, alone and within the hybrid pipeline, uses beam search with beam size 5, as in its original paper; GPT-5 post-processing uses the prompt in Appendix § 16.
Invalid outputs and refusals. An output that does not parse against the schema is retried at most twice and then scored as empty. A refusal to describe the footage is not retried with a modified prompt, because prompt changes that work around a model’s safety behaviour would not reflect how the model behaves in deployment.
Hardware and endpoints. Local components ran on one workstation, with its system clock synchronised to UTC so that latencies are timed consistently. In earlier tests on a node with two 24 GB GPUs, we measured GPU memory use of about 12 GB for NarrativeBridge’s released checkpoint, about 24 GB with its feature encoders, and at least 20 GB for PAL with Llama 3.2 3B, so the local part of the hybrid pipeline fits on two 24 GB GPUs; a fully on-premises version would need a local model in place of the GPT-5 post-processing step, which every system in this benchmark uses. NarrativeBridge, PAL with Llama 3.2 3B and Nemotron 3 Nano Omni (open weights) ran on the local workstation; GPT-5 post-processing used the OpenAI API in the nearest supported region; Gemini 3.1 Pro used the Google API with UK/EU data processing; Qwen3.5-Omni-Plus used the Alibaba Cloud API in Singapore, as no UK or EU region was offered; and the judge, Doubao Seed 2.0 Pro, used its API in the nearest supported region. All judge calls used the same region and were made at varying times rather than in a single run window.
Data handling. Only publicly released footage is sent to hosted APIs. This is a property of the benchmark, not a recommendation: operational footage would require on-premises models or an approved processing agreement.
Judge Validation Protocol and Sample
Sample and annotation. Sampling was stratified by model, recording and event condition and included supported, unsupported, borderline and duplicate predictions. Two annotators independently reviewed the original video and audio, together with the model-predicted event and its temporal interval, without access to the model identity, reference annotation or automated-judge output. They labelled each prediction as supported, partially supported, unsupported or uncertain and recorded the event type (Table 5), temporal adequacy, whether the actor and target roles were correct and whether the claim contained unsupported detail. Disagreements were adjudicated by discussion and, where necessary, by a third annotator.
Support labels. Supported: the event is directly observable or clearly audible in the footage. Partially supported: the general event occurs, but the predicted label includes an unsupported detail or the timing is materially wrong. Unsupported: the event or its essential claim is not supported by the footage. Uncertain: the footage is too ambiguous, occluded, edited or noisy to decide. Two flags are recorded in addition to the label and are not classes: borderline, for cases the annotators found hard to decide, and duplicate, for predictions that repeat or overlap another predicted event. Borderline events keep their support label; duplicates are counted as unsupported, the same rule the judge applies.
Resolving disagreements. After independent annotation, (1) the two labels are compared; (2) disagreements are identified; (3) the annotators discuss only the disagreements; (4) if they reach agreement, the settled label is recorded; (5) if not, a third, senior annotator adjudicates; and (6) the original independent labels are preserved with the settled labels.
Mapping for the binary comparison. Supported supported; partially supported unsupported; uncertain unsupported, with the uncertainty rate reported separately; duplicate unsupported.
Confusion matrix. Against the settled human labels, the judge labelled 72 of the 75 supported events supported and 3 unsupported, and 24 of the 25 unsupported events unsupported and 1 supported (73 judged supported, 27 unsupported).
Sample. Table 6 gives the composition of the validation sample. It was stratified by model, recording and event condition and is not intended to estimate the natural prevalence of unsupported events, which is estimated from the judge’s labels on all predicted events.
| Model | Videos | Sup. | Unsup. | Partial | Uncertain | Borderline | Duplicate | Total |
| Hybrid pipeline | 20 | 19 | 3 | 2 | 1 | 2 | 1 | 25 |
| Gemini 3.1 Pro | 21 | 19 | 4 | 1 | 1 | 2 | 1 | 25 |
| Qwen3.5-Omni-Plus | 19 | 18 | 4 | 1 | 2 | 2 | 0 | 25 |
| Nemotron 3 Nano Omni | 20 | 19 | 4 | 1 | 1 | 2 | 1 | 25 |
| Total | – | 75 | 15 | 5 | 5 | 8 | 3 | 100 |
Table 6. Composition of the judge validation sample by settled human label. Borderline and duplicate are flags, not classes, and are not included in the total.
Model Selection and Panel
Why NarrativeBridge and PAL. Each was the strongest option in its own evaluation. NarrativeBridge outperformed task-specific captioners and zero-shot and fine-tuned video-language models on every causal-temporal narrative metric, with CIDEr margins of 17.88 (MSVD-CTN) and 17.44 (MSRVTT-CTN) over the strongest baseline (Table 7) [39]. PAL gave the best result on every audio task except VocalSound for all three language-model backbones (Table 8), and its attention-based integration uses about 60% less memory than prepending audio tokens [2], which matters because footage that cannot leave a police network must be processed on local hardware.
| Method | MSVD-CTN R-L↑ | MSVD-CTN C↑ | MSVD-CTN S↑ | MSRVTT-CTN R-L↑ | MSRVTT-CTN C↑ | MSRVTT-CTN S↑ |
| SEM-POS [38] | 25.39 | 37.16 | 14.46 | 20.11 | 26.01 | 12.09 |
| AKGNN [22] | 25.11 | 35.08 | 14.55 | 21.42 | 25.90 | 11.99 |
| GIT [57] | 27.51 | 45.63 | 15.58 | 24.51 | 32.43 | 13.70 |
| VideoLLaVA (zero-shot) [30] | 21.80 | 30.55 | 14.67 | 19.33 | 16.24 | 12.49 |
| VideoLLaVA (LoRA FT) | 24.56 | 34.98 | 15.41 | 21.21 | 18.97 | 13.28 |
| VideoLLaVA (simple FT) | 25.61 | 36.12 | 16.09 | 22.18 | 19.98 | 13.07 |
| ShareGPT4Video (zero-shot) [8] | 21.66 | 27.06 | 14.06 | 20.27 | 17.08 | 12.21 |
| ShareGPT4Video (LoRA FT) | 24.39 | 30.72 | 14.83 | 22.09 | 19.83 | 13.02 |
| ShareGPT4Video (simple FT) | 25.32 | 31.67 | 14.92 | 23.01 | 20.76 | 13.28 |
| NarrativeBridge | 31.46 | 63.51 | 19.25 | 27.90 | 49.87 | 15.76 |
Table 7. Results reported by NarrativeBridge [39]: NarrativeBridge against task-specific methods and video-language models on MSVD-CTN and MSRVTT-CTN. R-L, C and S denote ROUGE-L, CIDEr and SPICE.
| Backbone | Integration | ESC50 | DCASE | VS | FSD | AS2M | AC† | CL† | AC‡ | CL‡ |
| Llama3.2-1B | PLITS | 64.45 | 37.69 | 51.57 | 25.23 | 9.08 | 0.59 | 0.34 | 16.30 | 10.96 |
| LAL | 76.70 | 40.97 | 60.87 | 31.44 | 11.83 | 0.66 | 0.38 | 16.97 | 11.87 | |
| PAL (LAL+LFST) | 87.40 | 46.23 | 56.03 | 43.91 | 14.74 | 0.72 | 0.42 | 18.08 | 12.58 | |
| Llama3.2-3B | PLITS | 70.40 | 40.62 | 61.40 | 28.88 | 10.84 | 0.63 | 0.35 | 16.81 | 11.35 |
| LAL | 82.15 | 43.21 | 65.78 | 34.29 | 12.91 | 0.67 | 0.38 | 17.80 | 12.18 | |
| PAL (LAL+LFST) | 89.25 | 47.21 | 60.46 | 43.86 | 15.03 | 0.73 | 0.40 | 18.61 | 12.46 | |
| Qwen2.5-1.5B | PLITS | 68.00 | 37.57 | 56.45 | 27.87 | 9.56 | 0.63 | 0.38 | 16.63 | 11.74 |
| LAL | 70.85 | 38.79 | 59.20 | 28.53 | 10.28 | 0.63 | 0.38 | 16.65 | 11.44 | |
| PAL (LAL+LFST) | 87.80 | 45.52 | 56.73 | 43.26 | 13.92 | 0.73 | 0.41 | 18.45 | 12.20 |
Table 8. Results reported by PAL [2] for three language-model backbones: the standard approach of prepending audio tokens to the language-model input (PLITS) against attention-based integration (LAL), alone and with fused audio tokens, which is PAL (LAL+LFST). Classification: accuracy (ESC-50, VocalSound), micro-F1 (DCASE), mAP (FSD, AudioSet-2M). Captioning on AudioCaps (AC) and Clotho (CL): † CIDEr, ‡ SPICE. Best per backbone in bold.
Prompt Templates
All base systems are passed through the same common GPT-5 textual post-processing and taxonomy-mapping stage before scoring. The reported results therefore measure complete base-system-plus-GPT-5 pipelines. They should not be interpreted as raw performance of the base models in isolation. GPT-5 may interact differently with outputs from different base systems, despite the common prompt and schema. The base systems may use different input prompts internally; the common post-processing prompt, taxonomy, output schema, and evaluation procedure are fixed across systems. Each base system receives the same fixed chapter intervals from the common chapterisation (§ 3.3). The prompts below are used only by the omni-modal base systems: using the event-extraction prompt, each outputs
zero or more event claims within each chapter, each with an event identifier, an interval inside the chapter, a description, an event-label proposal and an actor/target proposal, together with a transcript, keeping chapter identifiers and timestamps unchanged. The hybrid pipeline takes no prompt at this stage: NarrativeBridge outputs timestamped event descriptions within each chapter and PAL a transcript, and event labels and actor and target roles are assigned only by GPT-5. The post-processing prompt supplies the 16 scored event types of Table 5, excluding type 9. Some prompts below also contain source-evidence and footage-type fields; because non-BWV and overlaid segments were removed before evaluation, these fields are not used or reported, and the footage-type section of the GPT-5 prompt is omitted where marked. Uncertainty fields from the speech prompt are discarded by the common output schema. The omni-modal base systems were also given the 16 scored types for reference; the labels scored are those assigned by GPT-5. The judge receives the recording and one predicted event, but not the reference annotation, the model identity or any benchmark score. Human annotators in the judge validation use the coding form at the end of this appendix and do not see the judge’s output. Model versions: Gemini 3.1 Pro as gemini-3.1-pro-preview; Qwen3.5-Omni-Plus as qwen3.5-omni-plus, equivalent to the snapshot qwen3.5-omni-plus-2026-03-15; Nemotron 3 Nano Omni from its released open weights; GPT-5, used for the common textual post-processing and taxonomy-mapping stage, as gpt-5-2025-08-07; and the judge as Doubao Seed 2.0 Pro. Preview and alias endpoints can change the model they serve, so results reflect the versions served at run time.
System prompt (omni-modal base systems)
You are an evidence-grounded assistant for reviewing body-worn-video recordings.
Your task is to describe observable events, transcribe speech, assign speaker roles, and
classify footage segments. You are not a police decision-maker, legal assessor, or
investigator.
Use only information visible or audible in the supplied recording. Do not infer intent,
guilt, legality, identity, race, ethnicity, age, gender, disability, or mental state. Do not
identify people by name.
If the footage is unclear, edited, obstructed, off-screen, or inaudible, report uncertainty
rather than guessing. Distinguish clearly between:
- directly observed or audible information;
- information that is partially supported;
- information that is not supported;
- information that cannot be determined.
Every event must include a start and end time. Keep event descriptions concise and
evidence-grounded. Do not merge separate events unless they are clearly continuous. Do not
invent events to make the narrative complete.
Return only valid JSON conforming to the supplied schema.
Event-extraction prompt and output schema (omni-modal base systems)
The recording has already been partitioned into fixed intelligent chapters.
Do not create, delete, merge, split, extend, shorten, or retimestamp chapters.
For each supplied chapter, identify zero or more observable events relevant to
evidential review. A chapter may contain multiple events, including events
that overlap or occur in sequence.
For each predicted event:
Assign a unique event_id within the recording.
Give its start and end time.
Keep the event interval within the enclosing chapter interval.
Describe one observable action or interaction.
Propose one label from the supplied taxonomy, or "unmapped".
Identify actor and target only by role where supported: officer, member_of_public, other, or unknown.
Do not merge distinct events merely because they occur in the same chapter.
Do not create an event solely from a transcript utterance.
Do not infer intent, legality, guilt, identity, or causation.
Return one record for every predicted event and preserve the enclosing chapter_id for each event.
{
"chapters": [
{
"chapter_id": "chapter_001",
"start_time": 0.0,
"end_time": 30.0,
"events": [
{
"event_id": "chapter_001_event_001",
"start_time": 4.0,
"end_time": 8.0,
"description": "An officer speaks to a member of the public.",
"event_label": "scene_interaction_with_suspects_or_witnesses",
"actor_role": "officer",
"target_role": "member_of_public"
},
{
"event_id": "chapter_001_event_002",
"start_time": 12.0,
"end_time": 18.5,
"description": "An officer handcuffs the person's wrists behind their back.",
"event_label": "handcuffing_and_physical_restraint",
"actor_role": "officer",
"target_role": "member_of_public"
}
],
"source_evidence": "Continuous body-worn-camera footage is visible."
}
]
}
GPT-5 post-processing prompt and common output schema (all systems; abridged)
You are a schema-constrained post-processing component in an evidence-grounded body-worn-video evaluation pipeline.
You receive:
records produced by a base video-understanding system for fixed, pre-supplied chapters. Each chapter record has:
a chapter ID and start and end timestamps;
zero or more event proposals, each with an event ID, start and end timestamps inside the chapter, a description, a proposed event label, and proposed actor and target roles;
[optional source or editing evidence, not used in this study].
a timestamped transcript produced by the same base system;
the 16 scored event types and their definitions;
[footage-type labels, not used in this study].
You do not receive the original video or audio. Use only the supplied chapter outputs, transcript and event taxonomy. Do not use outside knowledge.
Your task is to normalize the existing chapters, events and transcript into the common evaluation schema.
CHAPTER PRESERVATION
For every input chapter, preserve exactly one chapter record and its identifier and timestamps. Within that chapter, preserve every base-system event proposal and allow multiple event records.
GPT-5 may revise each event's taxonomy label, actor role, and target role using the description and transcript, but it must not create, delete, merge, or split events. It must preserve every event_id and every event timestamp. Every event interval must remain within its enclosing chapter interval.
A chapter may contain zero, one, or multiple events. Do not collapse multiple events into one chapter-level event. Preserve chapter and event order and do not alter chapter intervals.
EVENT-TAXONOMY MAPPING
Map each existing event to exactly one of the 16 scored event types, or to "unmapped" if no type's definition is supported by the event.
Select the type whose definition is best supported by the event description and any relevant transcript text. Treat the base system's proposed label and roles as inputs to check, not as final.
Do not create an event merely because it appears in the transcript.
Do not use outside knowledge, publication context, incident headlines, or police-force identity.
Do not infer legality, guilt, intent, causation, credibility, or identity.
The taxonomy labels are operational benchmark categories, not legal findings.
- Do not force a label: if no type is supported, use "unmapped" without adding
unsupported details.
ACTOR AND TARGET ROLES
- Infer actor_role and target_role only from the event description and
transcript when the relationship is sufficiently supported.
- Allowed values are:
officer, member_of_public, other, unknown.
- Use unknown when the role cannot be established.
- A speaker role does not automatically establish that the speaker is the
actor in the event.
- Do not infer personal identity or demographic attributes.
[FOOTAGE-TYPE MAPPING section omitted: not used in this study]
TRANSCRIPT HANDLING
- Preserve every transcript utterance, including its ID, text, speaker role,
and timestamps.
- Do not correct, paraphrase, delete, merge, or split utterances.
- You may attach existing utterance IDs to an event when their intervals
overlap or the text clearly refers to that event.
- Do not create chapters or events from transcript utterances.
OUTPUT
Return only valid JSON conforming to the supplied schema.
{
"chapters": [
{
"chapter_id": "chapter_001",
"start_time": 0.0,
"end_time": 30.0,
"events": [
{
"event_id": "chapter_001_event_001",
"start_time": 4.0,
"end_time": 8.0,
"event_label": "scene_interaction_with_suspects_or_witnesses",
"actor_role": "officer",
"target_role": "member_of_public",
"description": "An officer speaks to a member of the public.",
"related_utterance_ids": ["utterance_003"]
}
]
}
],
"utterances": []
}
Speech and speaker-attribution prompt and output schema (omni-modal base systems)
Transcribe the audible speech in the recording.
For each utterance:
- provide start and end time;
- provide the transcript exactly as heard where possible;
- assign one speaker role: officer, member_of_public, other, or unknown;
- mark unintelligible words explicitly;
- mark overlapping speech;
- do not reconstruct missing words from context;
- do not infer speaker identity from appearance;
- do not assign a speaker role when the evidence is insufficient.
Preserve legally or evidentially important words such as negations, numbers, names,
commands, and threats. If a word is uncertain, use [unclear] rather than selecting the most
plausible word.
{
"utterances": [
{
"utterance_id": "u1",
"start_time": 34.2,
"end_time": 36.8,
"speaker_role": "officer",
"text": "Please put your hands where I can see them.",
"speaker_confidence": "medium",
"transcription_status": "supported",
"overlap": false,
"uncertain_tokens": []
}
]
}
The judge returns a single binary label, so partially supported, uncertain and duplicate events are counted as unsupported, matching the conservative rule used for the human labels (§ 3.3.3).
Judge prompt and output schema
You are assessing whether a predicted event is supported by the supplied recording.
You are given:
- the recording;
- one predicted event;
- its time interval;
- its actor and target roles and description.
Assess only whether the event and its essential details are supported by information that is
visibly or audibly present in the recording. Do not use outside knowledge or infer legality,
intent, guilt, identity, or causation.
Return exactly one label and nothing else:
SUPPORTED
Use SUPPORTED only when the event and all material details, including the action, actor, target, and timing, are directly observable or clearly audible in the recording.
UNSUPPORTED
Use UNSUPPORTED when the event or any material detail is absent, contradicted, materially mistimed, overly specific, ambiguous, obstructed, inaudible, edited out, or not directly established by the recording. Treat partially supported, uncertain, duplicate, and unverifiable events as UNSUPPORTED.
Output exactly one of: SUPPORTED UNSUPPORTED
{
"type": "string",
"enum": ["SUPPORTED", "UNSUPPORTED"]
}
Human coding form
Review the predicted event against the recording.
Record:
Is the event directly supported?
supported
partially supported
unsupported
uncertain
Which event type is observed?
one of the 16 event types
none of the 16 types
cannot determine
Is the event interval adequate?
adequate
too early
too late
too broad
too narrow
cannot determine
Is the actor correct?
correct
incorrect
unknown
Is the target correct?
correct
incorrect
unknown
Does the description contain unsupported detail?
- no
- yes
7. What is the main reason for disagreement?
- action
- actor
- target
- timing
- wording/detail
- audio
- visibility
- editing/incomplete footage
- other
Datasheet for UKBWV-27
We follow the datasheet questions of Gebru et al. [17].
Motivation. UKBWV-27 was created to test whether current video AI can support review of body-worn video, and to measure whether its outputs are traceable to the footage, how visible their failures are, and whether they can be contested (§ 1.3). It was built by the authors at the Centre for Vision, Speech and Signal Processing, University of Surrey.
Composition. The dataset has 27 recordings (uk01 to uk27) involving 16 UK police forces, 118 minutes in total, uploaded between 2017 and 2026; 3 were published by police forces and 24 by news outlets (Table 10 and Table 11). Each recording has a human-corrected transcript with speaker roles and timestamps, source-segment labels marking the retained body-worn footage, and 53 annotated events labelled with 16 event types, plus a video-level tag for force later challenged in court or at a hearing or inquest (Table 5). The judge validation adds 100 predicted events with blinded human support labels (Appendix § 14). Recordings show identifiable people, including minors, and violent incidents, injuries and a death after restraint; they include footage shown in court, at an inquest and at misconduct hearings. No demographic attributes are recorded and no person is named. The dataset is a small pilot benchmark, not a representative sample of UK body-worn video, and has no training split.
| ID | Force | Incident (de-identified) | Source | Year | Length |
| uk01 | Metropolitan Police | Arrest after a stabbing | Guardian News | 2026 | 0:54 |
| uk02 | Metropolitan Police | Armed response vehicle pursuits (compilation, partly body-worn) | The Independent | 2025 | 10:10 |
| uk03 | Metropolitan / Northamptonshire | Arrest of a woman; footage initially said not to exist | BBC London | 2026 | 8:38 |
| uk04 | Merseyside | Arrest of the perpetrator of a mass stabbing | Manchester Evening News | 2025 | 1:50 |
| uk05 | Hampshire | Handcuffing of a man at a stabbing scene | The Independent | 2026 | 2:44 |
| uk06 | West Midlands | Officers speak to a man at a murder scene | Daily Record | 2026 | 6:26 |
| uk07 | Greater Manchester | Arrest at an airport (as shown in court) | LBC | 2025 | 4:59 |
| uk08 | Avon and Somerset | Alleged sledgehammer attack on an officer | Sky News | 2025 | 2:03 |
| uk09 | Avon and Somerset | Arrest after a street altercation (full recording) | ITV News | 2018 | 14:53 |
| uk10 | South Yorkshire | Arrests after an attack on a hotel during disorder | LBC | 2024 | 0:26 |
| uk11 | Northamptonshire | Pursuit of a lorry driver near a trunk road | Police (official) | 2023 | 1:41 |
| uk12 | Sussex | Man attacks officers with a hammer | PoliceWatch | 2017 | 0:59 |
| uk13 | Hampshire | Detention at a stabbing scene (news broadcast) | GB News | 2026 | 12:55 |
| uk14 | Metropolitan Police | Arrest after a stabbing in north London | DawnNews English | 2026 | 2:08 |
| uk15 | Metropolitan Police | Motorcycle officer confronts a man with a knife on an e-bike | Police (official) | 2026 | 7:13 |
| uk16 | Metropolitan Police | Events on the night of a fatal police shooting | The Mirror | 2024 | 2:33 |
| uk17 | Cheshire | Arrest of a suspect in a hospital murder investigation | TalkTV | 2023 | 0:44 |
| uk18 | Sussex / Metropolitan | Arrest of a couple after a search for a missing baby | The Telegraph | 2024 | 2:00 |
| uk19 | Merseyside | Arrest of a murder suspect | Times Radio | 2023 | 2:26 |
| uk20 | Cheshire | Arrests of two teenagers after a murder | Guardian News | 2024 | 1:23 |
| uk21 | Metropolitan Police | Vehicle stop later examined at a misconduct hearing | London Live | 2023 | 2:07 |
| uk22 | Essex | Arrest after a fatal stabbing at a constituency surgery | Sky News | 2022 | 2:22 |
| uk23 | Bedfordshire | Arrest after a triple murder | The Mirror | 2025 | 15:44 |
| uk24 | City of London | Taser use at a roadside stop | Guardian News | 2022 | 2:54 |
| uk25 | Cambridgeshire / BTP | Taser use and arrest after a train attack | The Sun | 2025 | 2:48 |
| uk26 | Dorset | Arrest of a 15-year-old, shown at a misconduct hearing | Police (official) | 2025 | 1:58 |
| uk27 | Metropolitan Police | Restraint of a man who later died, shown at an inquest | Sky News | 2020 | 3:02 |
Table 10. Composition of UKBWV-27: 27 recordings involving 16 UK police forces (118 minutes). Source is the publishing channel; Year is the upload year; Length is min:s. BTP: British Transport Police.
Collection process. Recordings were selected from footage already publicly released by police forces and news outlets, using the criteria in § 3.3.1: wholly or mainly body-worn footage, at least one interaction between an officer and a member of the public, and variation in lighting, camera motion, occlusion, audio quality and number of people. No new data were collected from people, the people shown did not consent to this analysis, and no secure, proprietary or police-held footage was used.
Preprocessing and labelling. Initial event annotations were generated by Baichuan-Omni-1.5 and MiMo-V2.5 and initial transcripts by Whisper (small.en); two annotators then reviewed and corrected them against the video and audio, following the guidelines in Appendix § 11. Before annotation and evaluation, segments not filmed by a body-worn camera (such as studio, CCTV or helicopter inserts) and segments carrying broadcaster overlays were removed, so only clean BWV was annotated and evaluated; the original video files are not redistributed.
Uses. The dataset is intended for evaluating how well video and audio models describe police-relevant footage, under human review. It should not be used to identify individuals, to infer demographic or personal attributes, to train surveillance systems, or to support decisions about the people shown. Because it is small and drawn from publicised incidents, results on it should not be read as estimates of performance on operational footage.
Distribution. We release the source URL of each recording, the event annotations, the transcripts with speaker roles, the source-segment labels, and the judge and human support labels; the release link will be added to a later version of this preprint. No video files are redistributed: each recording remains available only from its original publisher, so a recording that is withdrawn by its publisher drops out of the benchmark.
Maintenance. The authors maintain the annotations. A recording withdrawn by its publisher is removed at the next release, with the date of withdrawal recorded and affected results recomputed and versioned. Corrections and requests for removal can be sent to the corresponding author ([email protected]).
Table 10 lists the recordings, numbered uk01 to uk27 as in the event-type inventory (Table 5). Durations range from 26 s (uk10, a short-form release) to 15 min 44 s (uk23), so recordings differ in how much of each incident they show and whether they were edited before release; several were news broadcasts combining body-worn footage with other material, which was removed before evaluation; lengths refer to the retained body-worn footage. Incidents are described by type only: people shown in the footage are not named, and source URLs are released with the annotations rather than printed here.
KnifeHunter Training, Backbones and Ablation
KHNet architecture. Each model is a backbone (EfficientNet-B7 [54], ConvNeXt-Base [33], SwinV2-Base [32] or EVA02-Base [13]) producing a feature tensor , followed by an aggregation head: GeMNet [47], ActNet [25], TokenNet [60], SENet [28] or our KHNet. KHNet has two branches. The local branch applies regional pooling [52] over an neighbourhood and averages it with the input,
The global branch applies a convolution, batch normalisation, SiLU, a learnable Weibull activation, global average pooling and power normalisation [25], giving . Local features are fused by projecting them onto the orthogonal complement of the global descriptor [62],
concatenating them with the broadcast global vector and average pooling. A projection head (linear layer, batch normalisation, PReLU) gives a 512-dimensional descriptor. Training settings follow.
Training. Images are resized to . Models are trained end to end for 25 epochs as 543-way classifiers with sub-center ArcFace (margin 0.3, scale 45), Adam (learning rate , decayed by 0.8 every 4 epochs) and batch size 128. The GeM exponent is initialised at , the Weibull parameters at , , , , and .
| Configuration | Medium | Hard | Medium+D | Hard+D |
| Global average pooling | 81.1 | 58.2 | 79.1 | 53.1 |
| GeMNet | 83.1 | 61.2 | 81.5 | 57.9 |
| + Weibull pooling (ActNet) | 83.9 | 62.1 | 82.6 | 58.9 |
| + Local branch, orthogonal fusion | 85.6 | 64.1 | 84.7 | 62.1 |
| + Regional refinement (KHNet) | 86.5 | 65.3 | 85.7 | 63.4 |
Table 13. Cumulative contribution of KHNet components on EVA02-Base (mAP, %). Each row adds to the row above.
References
- [1]Ian T. Adams, Matt Barter, Kyle McLean, Hunter M. Boehme, and Irick A. Geary, Jr. 2026. No Man’s Hand: Artificial Intelligence Does Not Improve Police Report Writing Speed. Journal of Experimental Criminology 22, 1 (2026), 137–154. doi:10.1007/s11292-024-09644-7
- [2]Tony Alex, Wish Suharitdamrong, Sara Atito, Armin Mustafa, Philip J. B. Jackson, Imran Razzak, and Muhammad Awais. 2025. PAL: Probing Audio Encoders via LLMs – Audio Information Transfer into LLMs. arXiv preprint arXiv:2506.10423. arXiv:2506.10423 [cs.SD] https://arxiv.org/abs/2506.10423
- [3]Alibaba Cloud. 2026. Model Studio: Data Retention Policy. https://www.alibabacloud.com/help/en/model-studio/data-retention-policy. Accessed 28 October 2026.
- [4]Barak Ariel, William A. Farrar, and Alex Sutherland. 2015. The Effect of Police Body-Worn Cameras on Use of Force and Citizens’ Complaints Against the Police: A Randomized Controlled Trial. Journal of Quantitative Criminology 31, 3 (2015), 509–535. doi:10.1007/s10940-014-9236-3DOI
- [5]Stefan Baack, Christo Buschek, and Maty Bohaček. 2026. Unsteady Metrics and Benchmarking Cultures of AI Model Builders. In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26). ACM.
- [6]Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81). 77–91.
- [7]ByteDance Seed. 2026. Doubao-Seed-2.0: Model Release. Model release, February 2026. https://seed.bytedance.com/
- [8]Lin Chen et al. 2024. ShareGPT4Video: Improving Video Understanding and Generation with Better Captions. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track.arxiv.org/abs/2406.04325
- [9]Jacob Cohen. 1960. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 20, 1 (1960), 37–46. doi:10.1177/001316446002000104DOI
- [10]Jiankang Guo, Jia Guo, Tongliang Liu, Mingming Gong, and Stefanos Zafeiriou. 2020. Sub-center ArcFace: Boosting Face Recognition by Large-Scale Noisy Web Faces. In European Conference on Computer Vision (ECCV). 741–757.DOI
- [11]Clara Ernesto, Carlos Caetano, Sandra Avila, João Macedo, Camila Laranjeira da Silva, and Leo Sampaio Ferraz Ribeiro. 2026. Classifying by Proxy: Explainable and Reproducible Ensemble of Proxy Tasks for Child Sexual Abuse Imagery Classification. In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26). ACM.
- [12]Nel Escher, Nikola Bovanic, and Ben Green. 2026. Algorithm Auditing Policies Rest on Flawed Assumptions About Public Sector Systems. In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26). ACM.DOI
- [13]Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2024. EVA-02: A Visual Representation for Neon Genesis. Image and Vision Computing 149 (2024), 105171.DOI
- [14]Andrew Guthrie Ferguson. 2017. The Rise of Big Data Policing: Surveillance, Race, and the Future of Law Enforcement. NYU Press, New York.DOI
- [15]Chaoyou Fu et al. 2025. Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://openaccess.thecvf.com/content/CVPR2025/html/Fu_Video-MME_The_First-Ever_Comprehensive_Evaluation_Benchmark_of_Multi-modal_LLMs_in_CVPR_2025_paper.htmlarxiv.org/abs/2405.21075
- [16]Noa Garcia and Amelia Katirai. 2026. The Weaponization of Computer Vision: Tracing Military-Surveillance Ties through Conference Sponsorship. In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26). ACM.DOI
- [17]Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2021. Datasheets for Datasets. Commun. ACM 64, 12 (2021), 86–92. doi:10.1145/3458723DOI
- [18]Google. 2026. Gemini API Additional Terms of Service. https://ai.google.dev/gemini-api/terms. Accessed 28 October 2026.
- [19]Google DeepMind. 2026. Gemini 3.1 Pro Model Card. Model card, published 19 February 2026. https://deepmind.google/models/model-cards/gemini-3-1-pro/
- [20]Ben Green and Yiling Chen. 2019. Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19). ACM, 90–99. doi:10.1145/3287560.3287563DOI
- [21]Lynne Grossmith, Catherine Owens, Will Finn, David Mann, Tom Davies, and Laura Baika. 2015. Police, Camera, Evidence: London’s Cluster Randomised Controlled Trial of Body Worn Video. Technical Report. College of Policing and Mayor’s Office for Policing and Crime (MOPAC), London. https://www.london.gov.uk/sites/default/files/bwv_report_nov_2015.pdf
- [22]Willy Fitra Hendria, Vania Velda, Bahy Hemi Hartoyo Putra, Kriansyah Adzakha, and Cheol Jeong. 2023. Action Knowledge for Video Captioning with Graph Neural Networks. Journal of King Saud University – Computer and Information Sciences 35, 4 (2023), 50–62. doi:10.1016/j.jksuci.2023.03.006DOI
- [23]Syed Sameed Husain and Miroslaw Bober. 2017. Improving Large-Scale Image Retrieval Through Robust Aggregation of Local Descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 9 (2017), 1783–1796. doi:10.1109/TPAMI.2016.2613873
- [24]Syed Sameed Husain and Miroslaw Bober. 2019. REMAP: Multi-Layer Entropy-Guided Pooling of Dense CNN Features for Image Retrieval. IEEE Transactions on Image Processing 28, 10 (2019), 5201–5213. doi:10.1109/TIP.2019.2917234
- [25]Syed Sameed Husain, Eng-Jon Ong, and Miroslaw Bober. 2021. ACTNET: End-to-End Learning of Feature Activations and Multi-Stream Aggregation for Effective Instance Image Retrieval. International Journal of Computer Vision 129, 5 (2021), 1432–1450.
- [26]Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R. Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial Disparities in Automated Speech Recognition. Proceedings of the National Academy of Sciences 117, 14 (2020), 7684–7689. doi:10.1073/pnas.1915768117DOI
- [27]Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. 2017. Dense-Captioning Events in Videos. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 706–715.arxiv.org/abs/1705.00754
- [28]Seongwon Lee, Suhyeon Lee, Hongje Seong, and Euntai Kim. 2023. Revisiting Self-Similarity: Structural Embedding for Image Retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 23412–23421.DOI
- [29]Yadong Li et al. 2025. Baichuan-Omni-1.5 Technical Report. arXiv preprint arXiv:2501.15368 (2025). https://arxiv.org/abs/2501.15368
- [30]Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munnan Ning, Peng Jin, and Li Yuan. 2024. Video-LLaVA: Learning Unified Visual Representation by Alignment Before Projection. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). https://aclanthology.org/2024.emnlp-main.342/arxiv.org/abs/2311.10122
- [31]Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In European Conference on Computer Vision (ECCV). 740–755.arxiv.org/abs/1405.0312
- [32]Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhendao Xie, Yuxuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, and Baining Guo. 2022. Swin Transformer V2: Scaling Up Capacity and Resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 12009–12019.arxiv.org/abs/2111.09883
- [33]Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A ConvNet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11976–11986.
- [34]Cynthia Lum, Christopher S. Koper, David B. Wilson, Megan Stoltz, Michael Goodier, Elizabeth Eggins, Angela Higginson, and Lorraine Mazzerolle. 2020. Body-Worn Cameras’ Effects on Police Officers and Citizen Behavior: A Systematic Review. Campbell Systematic Reviews 16, 3 (2020), e1112. doi:10.1002/cl2.1112DOI
- [35]Katelyn X. Mei, Anna Seo Gyeong Choi, Hilke Schellmann, Mona Sloane, and Allison Koenecke. 2026. Addressing Auditing Pitfalls in Automatic Speech Recognition Technology: A Case Study of People with Aphasia. In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26). ACM. doi:10.1145/3805689.3812320DOI
- [36]Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19). ACM, 220–229. doi:10.1145/3287560.3287596DOI
- [37]Asmar Nadeem, Robert Dawes, Muhammad Awais, Mahrukh Awan, Adrian Hilton, and Armin Mustafa. 2026. Causal-VLM: Dense Causal Captioning in Videos. In Advances in Neural Information Processing Systems (NeurIPS). To appear.
- [38]Asmar Nadeem, Adrian Hilton, Robert Dawes, Graham Thomas, and Armin Mustafa. 2023. SEM-POS: Grammatically and Semantically Correct Video Captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). https://arxiv.org/abs/2303.14829
- [39]Asmar Nadeem, Faegheh Sardari, Robert Dawes, Syed Sameed Husain, Adrian Hilton, and Armin Mustafa. 2025. NarrativeBridge: Enhancing Video Captioning with Causal-Temporal Narrative. In International Conference on Learning Representations (ICLR). https://mlanthology.org/iclr/2025/nadeem2025iclr-narrativebridge/arxiv.org/abs/2406.06499
- [40]Tonya Nguyen, Jean Garcia-Gathright, Hannah Washington, Alex Chouldechova, Hanna Wallach, and Jennifer Wortman Vaughan. 2026. Validating and Refining Measurements for Generative AI Evaluation via Stakeholder Engagement. In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26). ACM. doi:10.1145/3805689.3812403DOI
- [41]NVIDIA. 2026. Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence. Technical Report. NVIDIA. https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Omni-report.pdf Technical report.arxiv.org/abs/2604.24954
- [42]OpenAI. 2025. GPT-5 System Card. System card, August 2025. https://openai.com/index/gpt-5-system-card/
- [43]OpenAI. 2026. Sharing Feedback, Evals and API Data with OpenAI. https://help.openai.com/en/articles/1060912-sharing-feedback-evals-and-api-data-with-openai. Accessed 28 October 2026.
- [44]Junfu Pu, Teng Wang, Yixiao Ge, Yuqing Gu, Chen Li, and Ying Shan. 2025. ARC-Chapter: Structuring Hour-Long Videos into Navigable Chapters and Hierarchical Summaries. arXiv preprint arXiv:2511.14349 (2025). https://arxiv.org/abs/2511.14349
- [45]Qwen Team. 2026. Qwen3.5-Omni Technical Report. arXiv preprint arXiv:2604.15804. arXiv:2604.15804 https://arxiv.org/abs/2604.15804
- [46]Filip Radenović, Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondřej Chum. 2018. Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/1803.11285
- [47]Filip Radenović, Giorgos Tolias, and Ondřej Chum. 2019. Fine-Tuning CNN Image Retrieval with No Human Annotation. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 7 (2019), 1655–1668.DOI
- [48]Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust Speech Recognition via Large-Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 202). 28492–28518.arxiv.org/abs/2212.04356
- [49]Inoliuwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* ’20). ACM, 33–44. doi:10.1145/3351095.3372873DOI
- [50]Rashida Richardson, Jason M. Schultz, and Kate Crawford. 2019. Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice. New York University Law Review Online 94 (2019), 15–55.
- [51]Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and Abstraction in Sociotechnical Systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19). ACM, 59–68. doi:10.1145/3287560.3287598
- [52]Shihuao Shao, Kaifeng Chen, Arjun Karpur, Qinghua Cui, André Araujo, and Bingyi Cao. 2023. Global Features are All You Need for Image Retrieval and Reranking. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 11002–111012.arxiv.org/abs/2308.06954
- [53]Linda J. Skitka, Kathleen L. Mosier, and Mark Burdick. 1999. Does Automation Bias Decision-Making? International Journal of Human-Computer Studies 51, 5 (1999), 991–1006. doi:10.1006/ijhc.1999.0252DOI
- [54]Mingxing Tan and Quoc Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 97). 6105–6114.
- [55]Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. CIDEr: Consensus-Based Image Description Evaluation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4566–4575.arxiv.org/abs/1411.5726
- [56]Rob Voigt, Nicholas P. Camp, Vinodkumar Prabhakaran, William L. Hamilton, Rebecca C. Hetey, Camilla M. Griffiths, David Jurgens, Dan Jurafsky, and Jennifer L. Eberhardt. 2017. Language from Police Body Camera Footage Shows Racial Disparities in Officer Respect. Proceedings of the National Academy of Sciences 114, 25 (2017), 6521–6526. doi:10.1073/pnas.1702413114DOI
- [57]Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. 2022. GIT: A Generative Image-to-text Transformer for Vision and Language. Transactions on Machine Learning Research (2022).arxiv.org/abs/2205.14100
- [58]Peiyi Wang, Yeyu Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghua Lin, Yunbo Cao, Cong Heng Leong, Qi Liu, Tianyu Liu, and Zhifang Sui. 2024. Large Language Models are not Fair Evaluators. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL). Association for Computational Linguistics, Bangkok, Thailand, 9440–9450.
- [59]Michael D. White, Seth Watts, Aili Malm, and Genesis B. Navarrete. 2025. Officer Acceptance and Use of AI-Driven Body-Worn Camera Footage Review. American Journal of Criminal Justice (2025). doi:10.1007/s12103-025-09857-0 Published online 2025; issue 51(2), 330–351 (2026).DOI
- [60]Hui Wu, Min Wang, Wengang Zhou, Yang Hu, and Houqiang Li. 2022. Learning Token-Based Representation for Image Retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 2703–2711.arxiv.org/abs/2112.06159
- [61]Xiaomi MiMo. 2026. MiMo-V2.5. Model card. https://huggingface.co/XiaomiMiMo/MiMo-V2.5
- [62]Min Yang, Dongliang He, Miao Fan, Baoping Shi, Xuetong Xue, Fu Li, Erui Ding, and Jizhou Huang. 2021. DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 11772–11781.arxiv.org/abs/2108.02927
- [63]Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations (ICLR).arxiv.org/abs/1904.09675
- [64]Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Advances in Neural Informationarxiv.org/abs/2306.05685