WorksComputer Science
How to Explain your Unexplainable Video Anomaly Detector
Abstract
Weakly-supervised Video Anomaly Detection (VAD) models assign anomaly scores from high-dimensional features, and their predictions remain unexplained. Saliency maps show where a model looks but not what it relies on, and the descriptions produced by Vision-Language Models (VLMs) are generated by branches that the anomaly score never reads. To address this faithfulness gap, we propose Explain The Unexplainable (ETU), a classifier-agnostic framework that audits frozen VAD models through counterfactual interventions in their feature space. ETU maps video features to a discrete dependency graph of agents, actions and outcomes (), regenerates the features by graph-conditioned diffusion with selected nodes of the graph masked, and measures the change in the score of a frozen detector; one set of interventions is applied unchanged to every detector. The generator is trained only to reconstruct the input features, so it cannot learn the contrast it is asked to measure, and we validate the audit with experiments that test every alternative explanation of its results, from the cost of regeneration to random deletions of matched size. On UCF-Crime features obtained from a CLIP encoder, we show that substituting the scene semantics results in the largest drop for most detectors, and that removing the outcome chain of the graph () changes the anomaly score of all six audited detectors far more than removing the same number of random nodes, even though the random deletion perturbs the features about as much. We show that this targeted behavior comes from the divergence objective of the graph embedding. Without it, the two interventions become indistinguishable, and with a divergence objective in the generator as well, the score change of the detector follows the displacement of the reconstruction rather than the semantics that were removed.
Introduction
Video Anomaly Detection (VAD) detects events that are unusual within a video’s context; because anomalies are sparse and unpredictable, its public datasets are mostly real-world CCTV footage of crimes and accidents. The dominant paradigm, Weakly-Supervised VAD (WSVAD), tackles this task relying solely on video-level labels. This paradigm is based on Multiple Instance Learning (MIL), pushing a WSVAD model to assign high anomaly scores to the most abnormal snippets of a known anomalous video and low scores for snippets of known normal videos. However, a critical drawback of MIL-based models is the inherently unexplainable nature of their predictions, which rely on high-dimensional feature spaces that are prone to spurious correlations.
Explainable VAD aims to untangle such correlations. Common approaches produce saliency maps over the pixels that most influence the score [10, 54], answering “where is the model looking?”. But VAD models rarely process raw pixels; they consume high-dimensional, pre-trained encoders such as I3D [4] or CLIP [36]. In this latent context, post-hoc attribution techniques such as Grad-CAM [41] are fundamentally limited, as they capture spatial correlation rather than the score’s dependence on content. Liu and Ma [26] showed that VAD models exploit the scene background as a cue for the anomaly rather than the event itself, which suggests that they rely on spurious correlations in the feature space rather than on the actual events.
Recently, the field has pivoted toward Vision-Language Models (VLMs) to provide descriptive grounding for anomaly detections [65, 67]. While these methods address the “what” of a scene, they typically employ parallel architectural branches that generate captions and anomaly scores through functionally decoupled pathways. This structural separation introduces a faithfulness gap. The generated caption serves as a post-hoc rationalization rather than an input the model’s decision depends on, as shown in Figure 1.

Figure 1. A VLM explainer’s caption changes under a counterfactual prompt but its score does not. Counterfactual prompting of Holmes-VAU [67] on the “robbery” video provided in the official repository. The anomaly scores obtained with the two prompts are nearly identical, so the score does not follow the explanation the model gives, while the counterfactual explanation mentions the very person the prompt instructed to ignore.
To bridge this faithfulness gap, we propose Explain The Unexplainable (ETU, Figure 2), a classifier-agnostic framework designed to audit frozen VAD models by intervening directly in their latent feature space. ETU enforces a strict semantic bottleneck by mapping continuous video features into a discrete semantic manifold structured as a hierarchical dependency graph representing compositional semantics, namely the agents involved (Roots, ), their specific behaviors (Actions, ), and the resulting consequences (Outcomes, ). To establish a structural prior for this space, we utilize an offline VLM that observes each video and emits valid chains directly. ETU then regenerates the features by graph-conditioned diffusion under a masked graph (e.g., with an anomaly-relevant sequence removed). The frozen model’s response to these reconstructions moves the question from the spatial correlations of saliency maps (“where is the model looking?”) and the decoupled descriptions of VLMs (“what is happening?”) to intervention-based auditing (“how much did this score depend on the semantics the graph names?”).
Figure 2. Explain The Unexplainable (ETU) hands one set of masked reconstructions per video unchanged to every frozen detector. The Graph Embedding Model (GEM) maps continuous video features to a semantic dependency graph and a latent embedding ; deterministic masks yield the counterfactual embeddings , , , on which the Graph-Conditioned Feature Diffusion (GFD) denoises the video features. GFD is trained only to reconstruct the encoder features under the complete graph; the masked embeddings are used at inference. The change in a frozen detector’s score between the full-graph and a masked reconstruction is the audit’s reading of its dependence on the masked semantics. Node labels are illustrative.
ETU is a two-stage pipeline. In the first stage, the Graph Embedding Model (GEM) maps video features into a discrete atom space and infers the directed dependency edges between roots, actions, and outcomes. The topology is aggregated into a compact latent embedding (), distilling the offline VLM’s structural priors into a lightweight model with no test-time text generation. In the second stage, a Graph-conditioned Feature Diffusion (GFD) model performs guided denoising of noisy ground-truth features conditioned on this embedding. It is trained only to reconstruct the encoder features under the complete graph. During inference, we condition the diffusion process on three deterministic masked variants of the graph, which remove the core anomaly chains (), prune irrelevant background events (), or replace the scene semantics (). The downstream VAD model’s response to these reconstructions is an anomaly delta () [56] that measures how much the prediction depended on the masked part of the conditioning graph.
An audit of this kind has a failure mode of its own, and we treat it as part of the method. A generator trained to make the relevant-mask reconstructions diverge can manufacture the very contrast it is asked to measure, and a frozen detector scored on regenerated features may react to the distribution shift rather than to the content [16, 39]. ETU therefore keeps its generator out of that objective, training it on reconstruction alone, and every quantity we report is compared with a lower bound or with a test that can fail (Section 4.2), including a generator that does carry a divergence objective and whose deltas follow the displacement of the reconstruction.
The main contributions of this paper are:
A framework. ETU, a classifier-agnostic framework for the counterfactual auditing of frozen VAD models. A graph embedding model maps the video features to a discrete dependency graph, and a graph-conditioned diffusion model, trained only to reconstruct the features, regenerates them under masked graphs, so that one set of interventions is handed unchanged to every detector and read as a change of its score.
Validation. The lower bounds, count-matched random deletions and their displacement, ablations of the divergence objective in the graph embedding and in the generator, and a comparison of the graphs of three teachers that a feature-space counterfactual audit needs before its numbers can be attributed to the detector, each stated with the alternative explanation it rules out and with its outcome.
A targeted intervention. Substituting the scene semantics results in the largest drop for most detectors, and removing the outcome chain costs all six detectors more than removing the same number of random nodes, although the random deletion displaces the reconstructed features about as much. We show that the divergence objective of the graph embedding is the design choice that makes this possible. Without it the two deletions become indistinguishable, and a divergence objective in the generator makes the deltas follow the displacement instead. The delta is therefore read as the dependence of the score on graph-conditioned content, and not as the removal of a concept.
Related Work
Weakly-Supervised Video Anomaly Detection
The dominant paradigm in Video Anomaly Detection (VAD) is Weakly-Supervised learning (WSVAD), which utilizes only video-level labels for training [35, 45, 52, 56, 57, 68, 71, 73]. Sultani et al. [45] established the foundation, using MIL to identify the most salient anomalous segments within video “bags.” Subsequent work refines the discriminative power of that objective with feature magnitude and discriminability [6, 49], unbiased classifiers against label noise [29], self-training and mixture-of-experts designs [7, 8, 25], and multimodal features [30, 65]. WSVAD models nevertheless map high-dimensional features directly to scalar scores. They localize in time but offer no semantic justification for a prediction.
Explainable Video Anomaly Detection
Explainable VAD has focused on spatial localization (“where”) and semantic description (“what”). Traditional frameworks interpret via pixel-level reconstruction errors or gradient-based saliency [47], which capture spatial correlation rather than the score’s dependence on semantic content. To inject semantics, attribute-based methods evaluate normality against pre-defined taxonomies of trajectories [43], poses [38], or interactions [15, 46], whose dataset-specific ontologies struggle to generalize to novel anomaly classes. Vision-Language Models (VLMs) instead generate rich, textual explanations [28, 60, 61, 64, 67], but usually exhibit the faithfulness gap of Figure 1. Captions and anomaly scores come from parallel, functionally decoupled branches, so the outputs are post-hoc rationalizations that describe the scene rather than audit the detector’s internal logic. VERA [61] is an exception, feeding the questions it learns into the score it reports.
Faithfulness and Concept Interventions
Deletion-based faithfulness tests score a model on inputs from which the attributed content has been removed. ROAR [16] showed that the resulting drop conflates importance with distribution shift, and retrains the model to separate the two. ROAD [39] showed that the shape of the deletion mask leaks class information, and replaced retraining with imputation. F-Fidelity [70] and GOAR [31] harden the evaluation against the out-of-distribution inputs that deletion creates. ETU regenerates the input instead of retraining the model (Eq. (8)), and its count-matched random deletion plays the role of the leakage check of ROAD.
Concept-level methods such as TCAV [21], ACE [12] and CRAFT [11], concept bottlenecks with their post-hoc, counterfactual and intervenable variants [9, 22, 24, 63], and linear concept erasure [3] intervene on concepts inside the representation of the model itself, and their conclusions depend on the probe set [37]. The quantity we estimate is the causal concept effect of Goyal et al. [13], which ETU obtains from a regeneration instead of a generative in-fill such as FIDO [5], whose off-distribution hazard Hase et al. [14] document. Diffusion counterfactuals edit images or videos for a given classifier [18, 19, 51, 53, 55], and counterfactual generation has been used to explain image anomaly detectors [20, 27]. Prior concept-intervention methods intervene inside the model being explained, so the intervention and the explanation share parameters. ETU instead intervenes on the input of a model it never trains against, which is what makes one set of interventions comparable across independently trained detectors. Its generator is trained to reconstruct the input and never to separate the interventions, and the objective that shapes its conditioning is a design choice that we verify in § 4.
Methodology
ETU turns “which semantics does this score depend on?” into an intervention experiment in the feature space of a frozen detector that assigns anomaly scores to a feature sequence . To do so, for each video we first generate a hierarchical graph composed of root nodes (agents in the video), action nodes (the actions the agents take) and outcome nodes (the outcomes of the actions taken by the agents). is generated by iteratively prompting a VLM teacher (Qwen3-VL [1]) with the video and three structured prompts (Figure S4 of the appendix). ETU maps to a discrete semantic dependency graph (§ 3.1), a prediction supervised by the teacher graph . ETU then regenerates conditioned on or on a masked variant (§ 3.2), and reads ’s score change between the two reconstructions as the audit’s measurement.

Figure S4. Teacher prompts. The three layers are built by three calls on the same video. Each call receives the output of the previous layers, so the edges are the ID references in agents and caused_by.
Graph Embedding Model (GEM)
GEM maps video features to a structured, intervention-sensitive latent , jointly inferring discrete atoms, their dependency edges and a global structural prior.
Node Reconstruction. We map video features to human-verifiable concepts by allocating node slots for each semantic type, roots (), actions (), and outcomes (), each quantised against a codebook of prototypes. Video features are queried via type-specific cross-attention blocks applied sequentially, yielding continuous hidden representations . These hidden states are grounded in a discrete manifold via three disjoint codebooks . We initialize and regularize each codebook to be orthonormal across the first dimensions, ensuring that each of the prototypes represents a distinct semantic concept and keeping prototypes composable. A Gumbel-Softmax [17] estimator computes the categorical selection vector over the prototypes. The atom representation is for the first 3K training steps and afterwards, so that at inference each atom depends only on its selected prototype. To align these latent nodes with the extracted semantic atoms, Human-gran matching [23] grounds each to the text embedding of a VLM-derived atom via a reconstruction loss:
where is the set of atoms of type generated by the VLM and assigned to the slots of allocated for that type. Furthermore, to prevent prototype collapse and preserve the partially-orthonormal structure required for composable concepts, we regularize the codebook with . This loss minimizes inter-prototype correlation and penalizes the deviation of the Gram matrix from the identity :
where denotes the Frobenius norm.
Edge Prediction. Dependency edges are predicted hierarchically. Each root attends over the action nodes, and the Root-Action adjacency is a sigmoid over an MLP applied to each (root, action) pair; each action node is then updated with its root parents, , and a pairwise MLP scores each Action-Outcome edge from the two nodes, their absolute difference and their Hadamard product (Section A.2.4 of the appendix).
To handle graph sparsity, we optimize the total edge loss using Online Hard Example Mining (OHEM) [42] to retain all positives and the three hardest negatives per positive. For each adjacency matrix , the loss is defined as:
where counts the mined samples, the positives and the hard negatives. For the sparser action-outcome transitions, we up-weight the positive term of by a factor of 5. A degree-sparsity hinge then prunes redundant correlations:
where if and if . With , the same hinge is also applied to the in-degree of every target node, penalizing nodes with more than one expected edge.
Graph Pooling. The graph is compressed into a global latent via a gating mechanism that weights each atom by its structural importance. For each node , we compute , where is the aggregate out-degree of node . The resulting pooled embedding is concatenated with a structural prior and projected to obtain the graph latent .
To shape the latent manifold for counterfactual auditing, the model minimizes a contrastive loss. As shown in Figure 2, targeted masking of the reconstructed graph allows us to generate three counterfactual latents, (removal of nodes that are not on the path to an outcome node), (removal of nodes that are on the path of an outcome node), and (substitution of nodes closest to the ground-truth atoms with randomly-sampled, factually incorrect atoms). The teacher lists ordinary and unusual events alike, so the relevant mask removes every outcome chain of a video and not only the anomalous one. The loss enforces structural invariance to noise while maximizing sensitivity to semantic interventions:
Finally, VICReg regularization [2] prevents embedding collapse by enforcing unit variance, minimizing off-diagonal covariance (), and ensuring invariance under irrelevant masking:
The first term of Eq. (5) and the last term of Eq. (6) pull onto , and on normal videos the last term of Eq. (5) pulls onto it as well, so this loss uses the video-level label of the training videos. A near-zero is therefore a result of training and does not describe the detector, and ETU does not measure a dependence on background content. § 4.2 tests the divergence objective of Eq. (5) by removing it from GEM and by adding one to the generator. Implementation details are given in Section A.2 of the appendix.
Graph-Conditioned Feature Diffusion (GFD)
The second stage of ETU employs a Graph-Conditioned Feature Diffusion (GFD) model to denoise continuous video features conditioned on the semantic graph latent and its masked variants (). Because the denoiser is conditioned on the graph, a masked graph changes the reconstruction only through what the conditioning no longer encodes.
Architecture. GFD processes the noisy temporal features using a 4-block Diffusion Transformer (DiT) [32]. GFD conditions twice within each transformer block, so that the global topology guides the temporal denoising trajectory. The graph latent modulates the features via Feature-wise Linear Modulation (FiLM) [33], and is projected into four latent tokens that serve as the and in a cross-attention layer following the temporal self-attention. All video features are normalized by – percentile clipping and per-sample standardization (, ), which keeps the denoiser from collapsing to an unconditional prediction.
Training. GFD is trained to reconstruct the encoder features under the complete graph and nothing else. A noise vector and a timestep are drawn per sample, and the loss optimizes the noise-prediction space and the feature space jointly:
where is the noise predicted under and is the clean feature estimated via Tweedie’s formula [40]. The masked latents and never enter the loss.
Inference. During evaluation, we leverage a partial-reverse DDIM [44] protocol at an intermediate noise level ( of unless stated). Ground-truth features are forward-diffused to timestep to obtain the noisy counterpart . We then perform a 10-step deterministic DDIM reverse process conditioned on each variant . To isolate the influence of the graph structure while controlling for reconstruction bias, we compute the counterfactual anomaly delta as the change in the detector’s benchmark score relative to the full-graph reconstruction:
where denotes the frozen VAD model under audit. Every delta in this paper is a difference of dataset-level benchmark points between the two conditionings, frame-AUC on UCF-Crime and AP on XD-Violence, where a positive value means that the mask lowered the performance. It is invariant to a monotone rescaling of the scores, so it reports a change in the ranking of the frames. A high delta indicates sensitivity to the mask , and a near-zero delta invariance. Each mask removes a node together with every node that depends on it downstream in the graph, and the audit describes a detector over a dataset and not on a single video.
Experiments
We audit seven frozen detectors [6, 34, 49, 58, 62, 66, 72], but ETU can be seamlessly employed to audit any VAD-model for which it is possible to reconstruct the latent space of the training data. We carry out this audit on UCF-Crime [45], which contains real-world CCTV footage, and XD-Violence [57], which contains movie clips, with the two most used encoders in the field, I3D [4] and CLIP [36]. PEL4VAD is audited on I3D features only, DSANet and VadCLIP on CLIP features only. We follow the standard evaluation protocols of the two datasets [29, 45, 57, 58] and report every delta as the reconstruction-referenced change of Eq. (8), at noise level unless stated otherwise. Every configuration is trained with three seeds, and we report the mean over seeds. Configurations are selected on a held-out subset of the training videos, never on the test videos. No VLM is used at test time. GEM predicts the graph of a test video from its features alone through the learned codebooks, and every mask is computed on that predicted graph. The experiments of § 4.2 are carried out on UCF-Crime with CLIP features. Two further chains serve as ablations on the same track, namely a divergence-trained generator, the same architecture trained with an added divergence objective on its reconstructions (Section A.3.2 of the appendix), and a divergence-free ETU, in which GEM is trained without the objective of Eq. (5).
Auditing Frozen Detectors
Table 1 reports the audit. Its first column per track, , shows that regenerating the features under the complete graph costs the detectors very little. On UCF-Crime with CLIP features the drop lies between −0.10 and 0.30 AUC points, and across the four tracks it is below one point on 16 of the 22 track-detector cells and at most 3.25 points, for MGFN on UCF-Crime with I3D features, so the audited models keep their operating points on regenerated input, and every masked delta is read against a reconstruction that the detectors barely notice. The clean scores are reported in Table S9 of the appendix. On UCF-Crime/CLIP, substituting the scene semantics results in the largest drop for most detectors, removing the outcome chain costs each of the six between 4.23 and 6.78 AUC points, and pruning the background nodes costs nothing beyond the regeneration. For scale, deleting the 10% of feature dimensions with the largest magnitude costs the same detectors between 3.50 and 15.37 AUC points (Table S12 of the appendix). Under the relevant mask the six detectors respond by similar amounts, the largest delta exceeding the smallest by a factor of 1.6, while the six clean scores lie within 5.78 points of one another. On the other three tracks as well, substituting the scene semantics or removing the outcome chain costs far more than pruning the background nodes, which costs nothing beyond the regeneration on CLIP and at most 3.54 AP
| Detector | I3D | CLIP | ||||||||||||||
| UCF-Crime (AUC) | XD-Violence (AP) | UCF-Crime (AUC) | XD-Violence (AP) | |||||||||||||
| MGFN | 3.25 | 6.93 | 0.68 | 10.02 | 0.97 | 39.30 | 3.54 | 40.31 | 0.09 | 6.78 | −0.01 | 37.49 | 1.24 | 2.90 | 0.37 | 13.27 |
| UR-DMU | 1.70 | 12.35 | −0.65 | 14.40 | 0.22 | 27.59 | 2.21 | 40.09 | 0.13 | 5.21 | −0.02 | 4.03 | 0.71 | 2.58 | −0.13 | 8.95 |
| RTFM | 1.93 | 9.08 | −0.41 | 11.98 | 0.70 | 29.14 | 2.76 | 53.62 | −0.10 | 6.77 | 0.00 | 16.48 | 0.86 | 1.32 | 0.14 | 23.41 |
| PEL4VAD | 1.10 | 11.19 | 0.69 | 10.00 | 0.05 | 16.71 | 1.83 | 33.09 | – | – | – | – | – | – | – | – |
| DSANet | – | – | – | – | – | – | – | – | 0.16 | 4.23 | −0.01 | 37.52 | 0.65 | 39.47 | 0.02 | 27.86 |
| VadCLIP | – | – | – | – | – | – | – | – | 0.03 | 6.47 | −0.03 | 22.58 | 0.83 | 30.42 | −0.30 | 23.63 |
| GlanceVAD | 1.66 | 9.21 | 0.65 | 10.09 | 0.20 | 30.65 | 1.86 | 43.42 | 0.30 | 5.09 | −0.01 | 31.75 | 0.02 | 1.58 | 0.02 | 9.54 |
Table 1. Regenerating under the complete graph costs the detectors very little, while substituting the scene semantics or removing the outcome chain costs every one of them. is the drop from the clean features to the full-graph reconstruction; , and are the reconstruction-referenced deltas of Eq. (8), in frame-AUC points on UCF-Crime and AP points on XD-Violence, positive for a drop; the largest masked delta of each detector on each track is in bold.
| Detector | UCF-Crime (AUC) | XD-Violence (AP) | ||||
| Publ. | GT | Publ. | GT | |||
| I3D features | ||||||
| MIL* | − | 83.49 | 2.37 | − | 63.84 | 0.95 |
| GlanceVAD | 91.96 | 91.76 | 1.66 | 89.40 | 89.18 | 0.20 |
| MGFN* | 86.98 | 80.80 | 3.25 | 79.19 | 67.57 | 0.97 |
| RTFM* | 84.30 | 79.69 | 1.93 | 77.81 | 74.03 | 0.70 |
| UR-DMU | 86.97 | 86.97 | 1.70 | 81.66 | 81.66 | 0.22 |
| PEL4VAD | 86.76 | 86.36 | 1.10 | 85.59 | 85.26 | 0.05 |
| CLIP features | ||||||
| MIL* | − | 84.26 | 0.44 | − | 69.89 | 0.32 |
| GlanceVAD* | − | 87.39 | 0.30 | − | 68.54 | 0.02 |
| MGFN* | − | 83.66 | 0.09 | − | 72.33 | 1.24 |
| RTFM* | − | 85.45 | −0.10 | − | 75.77 | 0.86 |
| UR-DMU* | − | 85.05 | 0.13 | − | 81.72 | 0.71 |
| DSANet | 89.44 | 89.44 | 0.16 | 86.95 | 86.95 | 0.65 |
| VadCLIP | 88.02 | 85.82 | 0.03 | 84.51 | 84.50 | 0.83 |
Table S9. Clean operating points of the detectors and the cost of regeneration. GT is the score on the clean features in our evaluation, the drop from the clean features to the full-graph reconstruction of ETU at , as in Table 1 of the main paper, and Publ. the value reported by the detector’s own paper with the same features, where one entry is omitted. Detectors marked with * were trained by us. MIL is not audited; it is the detector used for configuration selection (Section A.3).
| UCF-Crime (AUC) | XD-Violence (AUC) | |||||||
| Detector | GT | Mag. | Grad. | Rand. | GT | Mag. | Grad. | Rand. |
| UR-DMU | 85.07 | −5.13 | −10.70 | 0.68 | 94.43 | −2.95 | −1.36 | 0.31 |
| RTFM | 85.45 | −15.37 | −25.04 | −0.18 | 92.45 | −7.95 | −6.25 | 0.71 |
| MGFN | 83.65 | −4.12 | −14.38 | 0.66 | 91.48 | −4.97 | 0.89 | −0.12 |
| GlanceVAD | 87.93 | −3.50 | −3.26 | 0.16 | 95.30 | −2.28 | −1.93 | −0.07 |
| DSANet | 88.97 | −5.41 | – | −0.30 | 94.84 | −8.08 | – | −0.02 |
| VadCLIP | 85.82 | −6.13 | – | −0.16 | 93.56 | −5.30 | – | −0.48 |
Table S12. Dimension-level deletion at a matched 10% budget on the CLIP track. Frame-AUC change from deleting the 10% of feature dimensions ranked highest by magnitude (Mag.), by input gradient (Grad.; four detectors, – where unavailable) or at random (Rand., 5 trials), against the same run’s clean score (GT, the baseline evaluation’s own measurement, within 0.6 points of the UCF-Crime GT column of Table S9; XD-Violence is reported here in AUC and there in AP). Both datasets are in frame-AUC here. The ranking is computed inside the audited detector, so the budget matches across detectors but the deleted content does not. The table gives context for the scale of the interventions and is not a method that they are compared against.
points on I3D. VLM explainers such as Holmes-VAU [67] are not in Table 1 because their scores do not read the feature space that the audit intervenes on, and counterfactual prompting does not replace the audit, since their score stays nearly unchanged under a counterfactual prompt whether the explanation changes or not (Figure 1; Section B of the appendix).
What the Delta Measures
In this section, we analyze what the delta of Eq. (8) responds to. A change of score under a masked reconstruction admits explanations other than the semantics that were removed, namely the cost of regenerating the features, the size of the perturbation, and the training of the generator itself. We test each of them on UCF-Crime with CLIP features, the perturbation size and the displacement for the relevant mask only, and we then ablate the divergence objective in the generator and in GEM to identify what makes the intervention targeted.
Regeneration cost. Table 2 shows that the deltas are not explained by the cost of regeneration. Regenerating the features under the complete graph costs the detectors between
| Detector | at | at | |||||||
| 100 | 300 | 600 | 100 | 300 | 600 | ||||
| MGFN | 0.09 | 0.03 | −0.01 | 1.09 | 6.78 | 13.71 | 4.26 | 37.49 | 38.73 |
| UR-DMU | 0.13 | −0.10 | −0.02 | 0.61 | 5.21 | 14.49 | 0.43 | 4.03 | 6.43 |
| RTFM | −0.10 | −0.15 | 0.00 | 5.55 | 6.77 | 19.41 | 3.72 | 16.48 | 22.64 |
| DSANet | 0.16 | −0.01 | −0.01 | 0.40 | 4.23 | 10.79 | 4.92 | 37.52 | 37.99 |
| VadCLIP | 0.03 | 0.05 | −0.03 | 0.61 | 6.47 | 13.84 | 3.77 | 22.58 | 25.84 |
| GlanceVAD | 0.30 | −0.04 | −0.01 | 0.36 | 5.09 | 12.07 | 5.12 | 31.75 | 35.54 |
Table 2. ETU on UCF-Crime with CLIP features. Frame-AUC points lost from the full-graph reconstruction under each mask (Eq. (8)) at three noise levels ( at ); is the cost of the regeneration itself and that of regenerating without any conditioning; the default noise level, , is in bold.
−0.10 and 0.30 AUC points, regenerating them without any conditioning changes the AUC by at most 0.15 because it starts from the noised original features, and pruning the background nodes costs nothing beyond the regeneration, whereas removing the outcome chain costs between 4.23 and 6.78 points on the same weights. We further observe that the relevant-mask and semantic-mask deltas grow monotonically with the noise level at which the regeneration starts, as an effect of the conditioning should. The deltas therefore come from the conditioning and not from the regeneration.
Perturbation size. To test whether any set of nodes of the same size costs the same, we draw for each video a random set of as many nodes as the relevant mask removes and regenerate the features under it. As shown in Table 3 (left), the random deletion has a delta between −1.45 and 1.13 points, whereas removing the outcome chain costs the detectors between 4.23 and 6.78 points, and the difference is positive for all six detectors with confidence intervals that exclude zero (the resampling procedure is described in Section C of the appendix).
Table 3. The count-matched random deletion under the three regimes. For each test video, a random draw of as many nodes as the relevant mask removes; the cost of the relevant mask, the cost of the random draw and their difference with its 95% confidence interval, in frame-AUC points at on the UCF-Crime/CLIP test videos. The divergence objective is used in GEM only by ETU, in both GEM and GFD by the divergence-trained generator (against its own full-graph reconstruction), and nowhere by the divergence-free ETU. The differences of ETU, positive on all six detectors with intervals that exclude zero, are in bold.
| ETU | Divergence-trained generator | Divergence-free ETU | |||||||
| Detector | |||||||||
| MGFN | 6.78 | −1.45 | 8.23 [4.89, 21.36] | 8.76 | 17.95 | −9.20 [−24.41, 11.57] | 0.48 | 0.56 | −0.09 [−2.63, 2.31] |
| UR-DMU | 5.21 | −0.68 | 5.89 [3.98, 15.27] | −2.10 | 9.88 | −11.98 [−28.93, −3.91] | 0.23 | 0.30 | −0.06 [−1.89, 1.64] |
| RTFM | 6.77 | 0.86 | 5.91 [2.93, 18.67] | −0.46 | 42.48 | −42.93 [−60.55, −26.20] | 2.22 | 0.57 | 1.64 [−1.08, 4.30] |
| DSANet | 4.23 | −1.22 | 5.45 [3.22, 14.21] | 42.81 | 13.53 | 29.27 [0.89, 50.50] | 1.65 | 1.00 | 0.65 [−1.61, 2.70] |
| VadCLIP | 6.47 | 1.13 | 5.34 [0.47, 22.76] | 32.75 | 12.58 | 20.17 [−6.98, 44.59] | 1.06 | 0.82 | 0.23 [−2.22, 2.85] |
| GlanceVAD | 5.09 | 0.37 | 4.72 [4.45, 19.23] | 9.31 | 9.56 | −0.25 [−7.44, 8.81] | 1.26 | 0.35 | 0.91 [−0.39, 2.18] |
Table 3.
Displacement. Table 4 reports how far each of the two deletions moves the reconstruction from the full-graph one, next to the response of the detectors, on the videos of Table 3. The median random draw moves the reconstruction farther than the median relevant mask and about as far on anomalous videos specifically, yet the relevant mask costs the detectors 5.76 AUC points on average and the random draw −0.17. On the normal videos the random draw displaces the reconstruction farther, while the relevant mask shifts the raw detector scores 2.4 to 5.7 times more (Table S17 and Table S16 of the appendix). On the anomalous videos the relevant mask displaces the reconstruction slightly more and costs every detector far more. The conditioning latent shows the same pattern in cosine. On the anomalous videos the random latent is as far from as the relevant one (−0.307 against −0.331), while in the relevant latent is farther (Table S18 of the appendix). We conclude that the detectors respond to which nodes the mask removes, and not to how far it moves the features.
Table 4. Displacement and detector response under the relevant mask and the count-matched random deletion. Median cosine and MSE of each masked reconstruction to the full-graph one, next to the frame-AUC drop of Eq. (8) averaged over the six detectors. The last row is the median of the per-video ratios random/relevant of and of MSE, which differs from the ratio of the medians above, and the ratio of the mean deltas. Bold marks the larger displacement and response over all videos.
| Displacement | Detector response | ||
| Mask | cos | MSE | (AUC points) |
| irrelevant mask | 1.000 | 0.000 | −0.01 |
| relevant mask | 0.977 | 0.009 | 5.76 |
| anomalous videos | 0.983 | 0.006 | 10.46 |
| count-matched random | 0.955 | 0.020 | −0.17 |
| anomalous videos | 0.989 | 0.005 | −0.43 |
| random/relevant ratio | 0.95 | 1.11 | −0.03 |
Table 4.
| Quantity | MGFN | UR-DMU | RTFM | DSANet | VadCLIP | GlanceVAD |
| (i) ETU, test set videos, frame-AUC points | ||||||
| 8.23 | 5.89 | 5.91 | 5.45 | 5.34 | 4.72 | |
| 95% lower | 4.89 | 3.98 | 2.93 | 3.22 | 0.47 | 4.45 |
| 95% upper | 21.36 | 15.27 | 18.67 | 14.21 | 22.76 | 19.23 |
| two-sided | 0.0002 | 0.0020 | 0.0032 | 0.0008 | 0.0342 | < 0.0002 |
| 0.214 | 0.131 | 0.127 | 0.288 | 0.175 | 0.073 | |
| 95% lower | 0.006 | 0.005 | 0.005 | 0.008 | 0.004 | 0.003 |
| 95% upper | 0.526 | 0.506 | 0.497 | 0.829 | 0.843 | 0.300 |
| (ii) ETU, anomalous videos, point estimates | ||||||
| 7.13 | 8.71 | 11.43 | 14.63 | 9.36 | 11.49 | |
| −4.65 | −1.67 | 1.90 | −2.34 | 3.18 | 1.02 | |
| difference | 11.77 | 10.39 | 9.53 | 16.97 | 6.18 | 10.47 |
| (iii) ETU, normal videos, raw score shift | ||||||
| relevant mask | 0.324 | 0.148 | 0.080 | 0.098 | 0.189 | 0.116 |
| 95% lower | 0.240 | 0.089 | 0.039 | 0.076 | 0.125 | 0.079 |
| random draw | 0.068 | 0.048 | 0.034 | 0.021 | 0.055 | 0.020 |
| 95% lower | 0.006 | 0.013 | 0.016 | −0.001 | −0.001 | −0.003 |
| (iv) Divergence-free ETU, videos of Table 3 of the main paper, frame-AUC points | ||||||
| 0.48 | 0.23 | 2.22 | 1.65 | 1.06 | 1.26 | |
| 0.56 | 0.30 | 0.57 | 1.00 | 0.82 | 0.35 | |
| −0.09 | −0.06 | 1.64 | 0.65 | 0.23 | 0.91 | |
| 95% lower | −2.63 | −1.89 | −1.08 | −1.61 | −2.22 | −0.39 |
| 95% upper | 2.31 | 1.64 | 4.30 | 2.70 | 2.85 | 2.18 |
Table S16. Confidence intervals for the count-matched deletion. UCF-Crime with CLIP features at on ETU, except block (iv), the divergence-free ETU. Blocks are different video sets or quantities and do not compare across a block boundary; block (iii) is in raw score units, since pooled AUC is undefined on videos without positive frames. The differences of block (i) are in bold.
| Mask | ETU cos | ETU MSE | Divergence-trained cos | Divergence-trained MSE |
| videos of Table 3 of the main paper | ||||
| irrelevant mask | 1.000 | 0.000 | 1.000 | 0.000 |
| relevant mask | 0.977 | 0.009 | 0.999 | 0.000 |
| count-matched random | 0.955 | 0.020 | −0.003 | 0.895 |
| anomalous videos | ||||
| irrelevant mask | 1.000 | 0.000 | 1.000 | 0.000 |
| relevant mask | 0.983 | 0.006 | 0.051 | 0.553 |
| count-matched random | 0.989 | 0.005 | −0.217 | 1.715 |
| normal videos | ||||
| irrelevant mask | 1.000 | 0.000 | 1.000 | 0.000 |
| relevant mask | 0.976 | 0.009 | 1.000 | 0.000 |
| count-matched random | 0.939 | 0.051 | 0.141 | 0.600 |
| random/relevant ratio, per video | 0.95 | 1.11 | 2.41 | 5.44 |
Table S17. Reconstruction displacement of the three masks, split by video label. Cosine and MSE of each masked reconstruction to the full-graph reconstruction of the same run, per-video mean over segments and median over the videos of Table 3 of the main paper, on ETU and on the divergence-trained generator. The last row gives the per-video median ratio random/relevant of and of MSE. The larger displacement on ETU over all videos is in bold.
| Mask | Cosine to | to | ||||
| all | anom. | normal | all | anom. | normal | |
| irrelevant mask | 0.963 | 0.948 | 0.984 | 0.9 | 1.2 | 0.7 |
| relevant mask | 0.458 | −0.331 | 0.961 | 20.7 | 14.3 | 21.6 |
| count-matched random | −0.235 | −0.307 | −0.157 | 12.1 | 9.8 | 13.0 |
Table S18.
Divergence objective in GFD. To see what changes when the generator is trained to produce the contrast that the audit measures, we train the same architecture with the divergence objective of Eq. (9) of the appendix on the same GEM latent, which keeps the irrelevant-mask reconstruction on the full-graph one and pushes the relevant- and semantic-mask reconstructions at least a margin away from it. As shown in Table 3, this divergence-trained generator fails the test that ETU passes. The difference between the relevant mask and the random deletion is positive for 1 of the six detectors, indistinguishable from zero for 3 and negative for 2. Table S11 of the appendix shows why. One random node that it has not seen in training costs it between 1.75 and 14.92 points, on average 9.91 points against 15.18 for the whole relevant mask, and once a random-node variant is added to its training one random node changes the AUC by at most 0.09 points (Table S10). Its relevant-mask reconstruction of the anomalous videos is nearly orthogonal to its full-graph reconstruction (median cosine 0.051), while the random draw moves farther still (−0.217; Table S17), and for four of the six the random draw also costs more. A generator trained to separate the interventions thus reacts to any conditioning it was not trained on, and its deltas cannot be attributed to the content that the mask removes; without such an objective, a change in the scores of the audited detectors can only come from the change in the conditioning that GFD receives.
| Setting | MGFN | UR-DMU | RTFM | DSANet | VadCLIP | GlanceVAD |
| , one node | −0.09 | 0.00 | −0.02 | −0.02 | −0.04 | −0.01 |
| 1.03 | 1.19 | 5.01 | 16.50 | 6.90 | 1.01 |
Table S10. Random-node retrain of the divergence-trained generator. The same generator, retrained with an added random-node conditioning variant, on all 290 UCF-Crime/CLIP test videos. Frame-AUC points lost from this model’s own full-graph reconstruction under one random node, a conditioning variant this model saw in training, and under the relevant mask, which still moves every detector (bold).
| Detector | |||||
| MGFN | 0.24 | 0.03 | 0.04 | 14.29 | 8.76 |
| UR-DMU | 0.17 | −0.10 | −0.02 | 7.85 | −2.10 |
| RTFM | 0.20 | −0.15 | −0.13 | 14.92 | −0.46 |
| DSANet | 0.97 | −0.01 | 0.07 | 13.31 | 42.81 |
| VadCLIP | 0.55 | 0.05 | −0.01 | 7.32 | 32.75 |
| GlanceVAD | 0.49 | −0.04 | 0.03 | 1.75 | 9.31 |
Table S11. The divergence-trained generator on the full test set. Frame-AUC points lost from the full-graph reconstruction (Eq. (8) of the main paper); is the cost of regenerating without any conditioning and (bold) that of one random node not seen in training (9.91 points on average, against 15.18 for the relevant mask). Its count-matched deletion is in Table 3 of the main paper.
Divergence objective in GEM. ETU still relies on a graph embedding trained with the divergence objective of Eq. (5). To verify that this objective is what makes the intervention effective, we retrain GEM without it and train a new reconstruction-only generator on its latent. As shown in Table 3, removing the outcome chain and removing the same number of random nodes then have the same effect on all six detectors, and both effects are small, although the new GEM reconstructs the graphs as accurately as the original one.
Table 3 therefore shows three regimes of the same intervention. With the divergence objective in GEM only, removing the outcome chain moves every detector while a random deletion of the same size moves none, although it displaces the features about as much. With the objective in the generator as well, a random deletion pushes the reconstructed features off the manifold, and the detectors react to the displacement instead of the content that was removed. Without the objective, neither deletion moves the detectors. We conclude that the divergence objective in GEM is the design choice that turns a masked graph into a targeted intervention on the input features, whose effect depends on which nodes are masked and not on the size of the perturbation.
Detector-specific readings. The audit can also separate detectors whose benchmark scores are close. DSANet and UR-DMU reach clean AUCs of 89.44 and 85.05 on UCF-Crime with CLIP features (Table S9 of the appendix), yet substituting the scene semantics costs DSANet more than UR-DMU (Table 1). Figure 3 shows how differently the two detectors respond to the relevant mask on one video. The score of DSANet drops on every segment, whereas UR-DMU is saturated when unmasked and responds all-or-nothing. At the dataset level, masking a single node changes the frame-AUC by at most 0.28 points when the node lies on the outcome chain and 1.80 points when it lies off it, against 4.23 to 6.78 points for the whole chain (Table S21 of the appendix). A single-node mask also removes every node that depends on it downstream.
Figure 3. The same intervention moves two detectors very differently (Arrest024, UCF-Crime). Raw anomaly scores of DSANet and UR-DMU under the full graph and with the outcome chain removed (the roots White pickup truck and Police officers, their collision and arrest actions, and the resulting stop and restraint outcomes). The score of DSANet drops on every segment, whereas UR-DMU is saturated when unmasked and responds all-or-nothing.
| Setting | MGFN | UR-DMU | RTFM | DSANet | VadCLIP | GlanceVAD |
| ETU, all 290 test videos | ||||||
| on-chain node | ||||||
| off-chain node | ||||||
| Divergence-trained generator, its own single-node cache, the same masks | ||||||
| on-chain node | ||||||
| off-chain node |
Table S21. Single-node masks (). Frame-AUC points lost from each model’s own full-graph reconstruction under one masked node on or off the outcome chain, on all 290 UCF-Crime/CLIP test videos. A negative value means that the AUC rises.
Dependence on the Teacher
To assess how much the graphs depend on the VLM that produces them, we parsed the UCF-Crime videos with three teachers, Qwen3-VL (the teacher of all other experiments), InternVideo3 and Gemma-4, using the same prompts, and compared their graphs video by video (Section D and Table S20 of the appendix). The atoms of two teachers on the same video agree more than those of two videos of the same anomaly class on 73.4–81.4% of the videos, whereas the edges agree little, with an edge of 0.138–0.250 for two teachers on the same video against 0.111–0.167 for two videos of the same class. The outcome chain is therefore a property of the teacher’s parse, as GEM reproduces it, and not a structure of the scene that any teacher would recover. The parsed graphs do not fill the slots. The mean graph has 3.74 roots, 4.71 actions and 3.80 outcomes against slots per type, while all prototypes of each type are in use (Table S1 of the appendix).
| Prototypes | MSE wt. | Test Cosine | Root Util. | Action Util. | Outcome Util. |
| 16 | 0.2 | 0.591 | 88% | 100% | 94% |
| 24 | 1.0 | 0.734 | 100% | 100% | 100% |
| 32 | 0.5 | 0.712 | 100% | 100% | 100% |
| 48 | 0.5 | 0.435 | 50% | 100% | 42% |
| 64 | 0.5 | 0.581 | 39% | 75% | 69% |
| 128 | 0.5 | 0.521 | 24% | 20% | 23% |
Table S1. Codebook capacity.
| Qwen3-VL / InternVideo3 | Qwen3-VL / Gemma-4 | InternVideo3 / Gemma-4 | |
| matched atom cosine, same video | 0.780 / 0.793 / 0.772 | 0.764 / 0.777 / 0.728 | 0.766 / 0.779 / 0.737 |
| same-class null | 0.704 / 0.729 / 0.715 | 0.705 / 0.722 / 0.687 | 0.709 / 0.731 / 0.693 |
| random-video null | 0.668 / 0.683 / 0.667 | 0.674 / 0.687 / 0.658 | 0.682 / 0.702 / 0.669 |
| paired win vs same-class null | 81.4% | 74.1% | 73.4% |
| edge , same video | 0.250 | 0.167 | 0.138 |
| same-class null | 0.167 | 0.143 | 0.111 |
Table S20.
Conclusion
We introduced Explain The Unexplainable (ETU), a framework that audits frozen Video Anomaly Detection models by regenerating their input features under a masked semantic dependency graph and reading the change in their score, with one set of interventions handed unchanged to every detector. On UCF-Crime/CLIP, substituting the scene semantics results in the largest drop for most detectors, and removing the outcome chain costs every detector more than a count-matched random deletion that moves the reconstruction about as far; the intervention becomes indistinguishable from a random one without the divergence objective of the graph embedding, and a response to the displacement of the reconstruction with a divergence objective in the generator. ETU therefore measures how much a detector’s score depends on the graph-conditioned content of its input features. Its generator must stay out of the objective that makes the intervention targeted.
Method Details and Ablations
Preliminaries on Counterfactual VAD
ETU expands the scope of Explainable VAD by shifting the objective from description to intervention. By learning a discrete semantic prior and utilizing guided diffusion to systematically ablate semantic concepts in the latent space, ETU allows to evaluate a pre-trained VAD model via counterfactual interventions. We define the VAD task as assigning anomaly scores to a feature sequence , obtained through a pre-trained video encoder, using a VAD model . To explain the decision logic of , we seek a counterfactual feature reconstruction that represents a minimal, semantically-guided intervention on the original scene.
Semantic Atom Space. We represent the semantic content of a video as a Directed Acyclic Graph (DAG) [69], denoted . Following the principles of compositional scene understanding, we define as a disjoint set of three semantic atoms, namely Root Nodes () representing agents or structural subjects present in a scene; Action Nodes () representing specific agent behaviors, where directed edges exist such that ; Outcome Nodes () representing resultant states or anomalies, where . The set of all directed paths forms the outcome chains of a scene. As shown in Figure 2, the ground-truth graph is generated by an offline VLM (Qwen3-VL [1]) observing the video at 1 fps and emitting atoms and dependency edges directly, following the atomic decomposition of [48]. The topological hierarchy is then strictly enforced to ensure a consistent semantic manifold across the dataset.
Counterfactual Interventions. An intervention on the semantic manifold, denoted as , involves masking a specific subset of nodes . The objective of ETU is to learn a function that maps continuous video features to a discrete graph that represents the semantic dependency graph of the scene, and then a mapping , where , to translate these discrete graph perturbations back into the continuous feature space. The interventions leveraged by ETU are shown in Figure S1. Specifically, we generate a counterfactual feature by denoising the video features under the masked graph condition. To quantify the sensitivity of the model’s decision to the masked atoms, we compute the anomaly delta () resulting from intervention against the reconstruction under the unmasked graph, :

Figure S1. Relevant Masking removes outcome atoms and their upstream dependencies. Irrelevant Masking prunes background elements off every outcome path, preserving the primary anomaly chains. Semantic Masking replaces nodes with concepts from different videos to simulate complete scene substitution.
where a high positive indicates that the anomaly prediction of model depended on the masked semantic chain. This formulation allows for three distinct auditing strategies, which remove relevant anomaly chains (), prune irrelevant background nodes (), or replace scene semantics () to test the robustness of the detector.
GEM
GEM is trained via a multi-stage curriculum:
Implementation details. GEM receives sequences of 200 CLIP (512-d) or I3D (1024-d) feature snippets and projects them to 512 dimensions. For each node type, learned query slots are refined by three cross-attention layers (8 heads) and projected to the 1024-dimensional space of the atom embeddings, in which each teacher label is embedded with Qwen3-Embedding-0.6B. Each type has a codebook of prototypes, and the Gumbel-Softmax temperature decreases linearly from 1.0 to 0.5 between steps 500 and 15K. The total loss is . In the two terms of Eq. (2) of the main paper are weighted by 0.2 each, plus a slot-diversity term (0.5) and an entropy term on the average code usage (2.0); in the positive term is weighted by 5; in , . GEM is trained end to end for 60K steps with AdamW (learning rate , for the codebooks, weight decay , gradient clipping at 2.0, 1.5K warm-up steps and cosine decay to ) and batch size 16. The losses are switched on cumulatively, with and from step 0, from 3K, and the variance and covariance terms of from 23K, and and the invariance term of from 45K.
To isolate the contributions of our design choices, we conducted evaluations on the UCF-Crime dataset. The following subsections detail our findings across node reconstruction, structural edge prediction, and latent space regularization. The selected optimal configuration is highlighted in grey in the ablation tables.
Node Reconstruction
Given I3D video features , GEM learns to reconstruct the three hierarchical levels of a semantic dependency graph, namely root nodes (actors/objects), Action nodes (actions/states), and Outcome nodes (consequences/anomalies). We utilize a discrete codebook of learned prototypes via Gumbel-Softmax selection. We ablate the codebook capacity and regularization.
Codebook Capacity and Regularization
Table S1 varies the number of prototypes per node type. All prototypes are used at and ; at only 20% to 24% of each codebook is used.
Table S1. Codebook capacity. MSE wt. is the weight of the squared-error term of in that run (Table S2); rows 16 and 24 differ from the others in this weight as well as in the number of prototypes.
| MSE Weight | Utilization | Cosine Sim. |
| 0.2 | Partial (88% to 100%) | 0.34 to 0.59 |
| 0.5 | Full (24 to 32 prototypes) | 0.71 to 0.74 |
| 1.0 | Full (24 to 32 prototypes) | 0.73 to 0.81 |
Table S2. Impact of MSE loss weighting on prototype utilization and feature fidelity.
To further enforce prototype usage, we ablated the Mean Squared Error (MSE) loss weight (Table S2). We found that an MSE weight of 0.5 or greater is needed for full codebook utilization (Table S2); the 16-prototype run of Table S1 used 0.2. While higher MSE resulted in better node reconstruction, in the final configuration we set the weight to 0.5 to balance node reconstruction and edge prediction.
Table S2. Impact of MSE loss weighting on prototype utilization and feature fidelity.
Edge Prediction
Given the reconstructed nodes, GEM must predict dependency edges to form a meaningful topology. Due to the extreme class imbalance (valid edges account for approximately 0.7% of the 1250 candidate pairs in a reconstructed graph that contains 25 root, 25 action and 25 outcome node slots), robust architectural and optimization choices are required.
Predictor Architecture and Loss Asymmetry
We first evaluated pairwise scoring mechanisms (Table S3). A dot product and an MLP on the concatenated pair reached an edge of 0.18 and 0.22; adding the absolute difference and the element-wise product of the two nodes to the MLP input raised it to 0.29, and hard negative mining (3:1 ratio) and hierarchical conditioning (RootAction edges informing ActionOutcome edges) raised it to 0.66. In the final model each root attends over the action nodes (4-head cross-attention) and is a sigmoid over an MLP applied to the concatenation of root and action ; each action is refined with its root parents, attends over the outcome nodes, and is a sigmoid over an MLP applied to .
| Architecture / Strategy | Edge | Observation |
| Dot product | 0.18 | Insufficient capacity for asymmetric edges |
| MLP pair | 0.22 | Marginal improvement |
| Cross-attention | 0.25 | Treats pairs independently |
| MLP with , | 0.29 | Best baseline scoring mechanism |
| + Hard negative mining | 0.45 | Addresses class imbalance |
| + Hierarchical conditioning | 0.66 | Explicitly models edge directionality |
Table S3. Ablation of edge scoring architectures and optimization strategies.
Furthermore, we observed a bottleneck in predicting ActionOutcome (AO) edges, which lagged behind RootAction (RA) edges by 5 to 10 points. This stems from the higher variance in Outcome node embeddings (consequences and anomalies). We corrected this by applying an asymmetric loss weight ( for ) to false negatives on the AO level, raising the AO from 0.653 to 0.768.
Joint Training Dynamics
A challenge in end-to-end training is the destructive interference between node reconstruction and edge prediction. Ground-truth nodes exhibit a 3% to 6% cosine separation between connected pairs. However, after passing through the reconstruction bottleneck, this separation drops to 0.02% (a reduction), causing modular GEM training to fail entirely ().
We solved this via joint end-to-end training, which forces the reconstruction objective to preserve edge-relevant features. To prevent gradient conflicts in shared layers, we implemented dual refinement pathways (separating the edge refinement gradient from the reconstruction gradient), successfully restoring edge prediction capabilities on reconstructed nodes. Detaching the gradients completely (stop-grad) resulted in edge failure (), confirming that gradients must flow through the shared backbone.
Graph Embeddings
The goal of GEM is to yield a single embedding that accurately dictates the behavior of the downstream diffusion model. This embedding must be both highly discriminative and counterfactually sensitive.
Graph Topology Injection
To aggregate the predicted graph structure into a single vector, we evaluated various pooling and message-passing schemes (Table S4). While standard Graph Isomorphism Networks (GIN) [59] are popular, the message passing corrupted our specific edge-relevant features, degrading edge to 0.322. Edge-Gated Pooling, where node representations are weighted by their predicted edge connectivity before mean pooling, safely injected topology into the latent space while maintaining high linear probe performance.
| Method | Edge | Linear Probe AUC | CF Gap |
| Mean Pooling | 0.690 | 0.893 | 0.969 |
| GIN (3-layer) | 0.322 | 0.924 | −0.004 |
| Virtual Node | 0.330 | 0.848 | 0.053 |
| Edge-Gated Pooling | 0.712 | 0.861 | 0.637 |
Table S4. Methods for injecting edge topology into graph embeddings.
Latent Regularization and Counterfactual Training
To prevent embedding collapse and ensure the latent space reacts correctly to counterfactual manipulations, we use Variance-Invariance-Covariance Regularization (VICReg) [2].
As shown in Table S5, applying VICReg with a weight of monotonically improves the Linear Probe AUC (LP AUC, measuring linear separability of normal vs. anomalous graphs) and increases the Counterfactual (CF) Gap. The CF gap measures the difference in cosine similarity when the model masks out relevant anomaly nodes versus irrelevant background nodes. A larger gap implies the downstream diffusion model will react more strongly to the removal of anomaly-relevant nodes.
| VICReg | LP AUC | CF Gap | Edge |
| 0.00 | 0.843 | 1.031 | 0.697 |
| 0.25 | 0.883 | 0.936 | 0.697 |
| 0.50 | 0.900 | 0.957 | 0.687 |
| 1.00 | 0.932 | 1.067 | 0.697 |
Table S5. Controlled ablation of VICReg regularization strength ().
Curriculum Learning and Phase Transitions
We observed that the model requires a strict curriculum to converge. The most important finding was the necessity of full-phase contrastive training (Table S6). Extending training beyond the joint edge-reconstruction phase to explicitly train the contrastive masking objectives (steps 45K to 60K) yielded improvements in the CF gap (67%) and general embedding separation, with only a negligible (2% to 3%) cost to the structural edge score.
| Metric | Joint-Phase Only (45K) | Full-Phase (60K) | Relative Change |
| CF Gap | 0.60 to 0.64 | 0.94 to 1.07 | +55% to +70% |
| Embed Gap | 0.02 to 0.06 | 0.18 to 0.41 | +300% to +700% |
| LP AUC | 0.86 to 0.88 | 0.88 to 0.93 | +2% to +5% |
| Edge | 0.71 to 0.73 | 0.69 to 0.70 | −2% to −3% |
Table S6. Impact of extending the curriculum to include full-phase contrastive training.
Final Selected Configuration and Analysis
Based on the explorations detailed above, the selected architecture leverages 32 discrete prototypes, Edge-Gated pooling, VICReg, and asymmetric AO loss weighting, trained through the full 60K step contrastive curriculum.
As shown in Table S7, the chosen configuration reaches a Counterfactual Gap of 1.067. When the predicted outcome-chain nodes are masked out (Relevant masking) the embedding cosine similarity drops to −0.123, while removing off-chain background nodes leaves the embedding nearly untouched (0.944). This is the behaviour the downstream Graph-Conditioned Feature Diffusion (GFD) model requires of its conditioning; it is a property of the trained embedding, selected for by this very criterion, and not evidence about any audited detector.
| Masking Strategy | Cosine Sim. |
| Relevant (remove anomaly) | −0.123 |
| Irrelevant (remove background) | 0.944 |
| Semantic (replace scene) | −0.193 |
| CF Gap (Irrel − Rel) | 1.067 |
Table S7. Counterfactual masking sensitivity in the final latent space (anomaly videos only).
Divergence-free GEM. The GEM of the divergence-free ETU (§ 4.2 of the main paper) has the same architecture, data, schedule and seed, with Eq. (5) of the main paper removed entirely. Its node reconstruction cosine is 0.8817, against 0.885 for the GEM of ETU.
Evaluation and Discussion
GEM. Table S8 validates GEM across node reconstruction, edge prediction, and embedding separability. Node reconstruction and edge precision are high on both datasets and backbones, and the embedding separates the masked variants as the downstream GFD requires. The variants and are near-orthogonal to the full-graph embedding while stays close to it. These are properties of the conditioning layer, not of any audited detector.
| Dataset | Enc. | Node | Edge | Graph (Cos.) | |||
| Cos. | MSE | ||||||
| UCF | I3D | 0.891 | 0.016 | 0.68 | −0.31 | 0.25 | 0.96 |
| CLIP | 0.885 | 0.017 | 0.64 | −0.28 | 0.26 | 0.97 | |
| XDV | I3D | 0.864 | 0.021 | 0.70 | −0.34 | 0.15 | 0.99 |
| CLIP | 0.862 | 0.020 | 0.68 | −0.37 | 0.15 | 0.98 |
Table S8. GEM evaluation across node reconstruction, edge fidelity and embedding similarity.
Topological Complexity and Alignment. High node reconstruction faithfulness () is consistently achieved, though alignment varies by dataset. UCF-Crime exhibits higher alignment due to the spatio-temporal stability of fixedcamera CCTV footage. Conversely, the movie-based clips in XD-Violence contain frequent temporal discontinuities and high visual variance, resulting in complex ground-truth graphs with higher node density in anomalous segments.
Edge Prediction and Metric Selection. We prioritize precision via the metric to heavily penalize false-positive edges, as spurious connections would compromise the integrity of the counterfactual audit. GEM achieves across all settings. Given the significant class imbalance between candidate and actual edges, this performance demonstrates a robust ability to filter environmental noise while preserving the essential outcome-chain backbone.
Embedding Separation. Graph embedding similarities show that the conditioning layer separates the relevant and semantic variants from the full graph. Irrelevant maskings () remain near-identical to the full graph (), while the relevant () and semantic () variants diverge, the semantic one to negative cosine similarity, most markedly on XD-Violence. This is the separation that the contrastive objective of § 3.1 of the main paper trains for. Whether the score of a frozen detector depends on what was removed is established by the experiments of § 4.1 and § 4.2 of the main paper.
Graph-Conditioned Feature Diffusion (GFD)
The generator of the main paper is trained on alone (§ 3.2 of the main paper). This section documents the divergence-trained generator that serves as the ablation of § 4.2 of the main paper (Section A.3.2), its capacity ablation, and the configuration-selection procedure that fixed the architecture and the capacity, which the reconstruction-only generator inherits unchanged. Results are reported as frame-level AUC for UCF-Crime and Average Precision (AP) for XD-Violence.
Models Reimplementation
Our I3D re-implementations of MGFN and RTFM sit 6.2 and 4.6 AUC points below their published UCF-Crime operating points and 11.6 and 3.8 AP points below on XD-Violence, so no architectural reading is drawn from them in the main paper; the CLIP variants of MIL, GlanceVAD, MGFN, RTFM and UR-DMU were trained by us with the authors’ code and have no published CLIP counterpart.
The Divergence-Trained Generator
The divergence-trained generator is the same architecture trained with , where, for each training video, one noise vector and one timestep are shared by its four conditioning latents, is the estimate of under , and keeps the irrelevant-mask estimate on the full-graph and pushes the relevant- and semantic-mask estimates at least a margin away from it, with margins and weights , and :
Irrelevant perturbations are thereby trained to yield invariant reconstructions and outcome-chain interruptions divergent ones, so this generator is trained to produce the contrast that the audit measures. As § 4.2 of the main paper shows, for four of the six detectors its deltas follow the displacement of the reconstruction, one untrained random node costs it 9.91 points on average against 15.18 for the whole outcome chain, and its count-matched deletion does not separate the relevant mask from the random draw (Table 3 of the main paper). Table S11 reports it on the full test set. Table S10 reports a retrain in which a conditioning variant with one random node removed is added to its training; afterwards one random node changes the AUC by at most 0.09 points on every detector. This follows from training on that variant and is not evidence about other deletions; the relevant mask still costs between 1.01 and 16.50 points on it.
Configurations are selected with a selectivity criterion that prioritizes intervention sensitivity over raw anomaly detection performance; the reconstruction-only generator inherits the selected architecture, capacity, curriculum and hyperparameters, with every weight of (9) set to zero, and every configuration is trained with three seeds. A configuration must first keep the cost of the full-graph reconstruction for the evaluating detector below 2% AUC, and is then ranked by its selectivity gap, the mean absolute score change under the relevant and semantic masks minus the mean under the full graph and the irrelevant mask, with lower leakage and then higher AUC as tie-breakers.
Selection split. The criterion is computed on a held-out subset of the training videos with a MIL detector [45] that is not among the detectors audited in the main paper, so the test videos never enter the selection and the audited detectors never enter the selection objective. The dimension-deletion baselines received no equivalent tuning.
Architecture and Conditioning
The denoiser employs a Transformer-based architecture formulated for noise prediction (ϵ prediction). At each diffusion timestep , the network receives noisy video features and a graph conditioning embedding , predicting the added noise ϵ.
Graph conditioning is injected via two complementary pathways. First, Feature-wise Linear Modulation (FiLM) [33] utilizes a linear projection to map to scale and shift parameters applied following each layer normalization. Second, a cross-attention mechanism projects to learned conditioning tokens that function as keys and values. To accommodate varying feature extractors, a linear input projection compresses I3D features () to the model dimension () prior to processing, whereas CLIP features () natively match the internal dimension.
Evaluated checkpoint. All results use the final checkpoint of training.
Architecture Capacity
To determine if the baseline model represents a capacity bottleneck for high-dimensional I3D inputs, we systematically increased the dimensions of the Transformer layers and cross-attention heads.
The three deltas of Table S13 differ by 0.002 in absolute value, so capacity had no measurable effect in this range, and we keep the smallest model.
| FFN | Heads | Tokens | Params | ||
| 768 | 1536 | 8 | 4 | 38M | −0.087 |
| 1024 | 2048 | 8 | 8 | 68M | −0.088 |
| 512 | 1024 | 4 | 4 | 7M | −0.086 |
Table S13. Architecture capacity ablation of the divergence-trained generator (UCF-Crime / I3D); the reconstruction-only generator keeps the selected size.
GFD. Table S14 and Table S15 evaluate the GFD denoising process via cosine similarity and relative to ground-truth features. Cosine similarity exceeds 0.9 under and , confirming accurate directional reconstruction.
Table S14. GFD reconstruction on UCF-Crime. Cells are means over all 290 test videos; Table S17 reports medians over the test videos for both generators, including the count-matched random draw.
| Enc. | Cond. | vs. x0 | vs. | ||
| Cos. | MSE | Cos. | MSE | ||
| I3D | 0.956 | 0.008 | – | – | |
| 0.954 | 0.009 | 0.998 | 0.001 | ||
| 0.887 | 0.043 | 0.931 | 0.053 | ||
| 0.856 | 0.068 | 0.899 | 0.063 | ||
| CLIP | 0.978 | 0.008 | – | – | |
| 0.977 | 0.012 | 0.996 | 0.003 | ||
| 0.687 | 0.251 | 0.708 | 0.238 | ||
| 0.218 | 1.330 | 0.232 | 1.311 |
Table S14.
Table S15. GFD reconstruction on XD-Violence.
| Enc. | Cond. | vs. Ground-Truth () | vs. Full Graph () | ||
| Cos. Sim. | MSE | Cos. Sim. | MSE | ||
| I3D | 0.942 ↑ | 0.010 ↓ | – | – | |
| 0.940 ↑ | 0.011 ↓ | 0.997 ↑ | 0.001 ↓ | ||
| 0.903 ↓ | 0.020 ↑ | 0.960 ↓ | 0.010 ↑ | ||
| 0.500 ↓ | 0.315 ↑ | 0.536 ↓ | 0.304 ↑ | ||
| CLIP | 0.966 ↑ | 0.015 ↓ | – | – | |
| 0.965 ↑ | 0.016 ↓ | 0.998 ↑ | 0.001 ↓ | ||
| 0.712 ↓ | 0.229 ↑ | 0.751 ↓ | 0.206 ↑ | ||
| 0.386 ↓ | 0.595 ↑ | 0.427 ↓ | 0.559 ↑ |
Table S15.
VLM-based Counterfactual Explainability
This section adds two videos from the official repository of Holmes-VAU [67] to the “robbery” video of Figure 1 of the main paper. For each video we pose the default prompt and a counterfactual prompt that asks which anomalies would occur without a named element of the scene, and plot the per-snippet anomaly scores under the two prompts.
In the “earthquake” video (Figure S2), the explanation under the default prompt names “a sudden and unexpected occurrence of a car accident” as the anomaly; under the counterfactual prompt (no earthquake) it describes a normal street scene with no unusual events. Both score curves stay below 0.2 on every snippet, so the default explanation reports an anomaly that the score does not register. In the “superwind” video (Figure S3), the counterfactual prompt asks which anomalies would occur without the wind; the explanation names the same anomaly as under the default prompt,

Figure S2. Anomaly scores and explanation of “earthquake.mp4” from the official Holmes-VAU repository [67]. The top panel shows the default and the counterfactual prompt with the explanation returned for each, and the bottom panel the per-snippet anomaly scores under each prompt.

Figure S3. Anomaly scores and explanation of “superwind.mp4” from the official Holmes-VAU repository [67]. The top panel shows the default and the counterfactual prompt with the explanation returned for each, and the bottom panel the per-snippet anomaly scores under each prompt.
Interval Estimates and Displacement for the Count-Matched Deletion
Every confidence interval that this paper reports for the count-matched random deletion comes from one procedure. The resampling unit is the video, and each replicate is one draw with replacement of the UCF-Crime test videos of Table 3 of the main paper, applied in lockstep to the full-graph, relevant-mask and random-deletion conditionings and to all six detectors, so that every quantity compared here is paired on the same videos. We draw replicates, recompute the benchmark metric inside each replicate (frame AUC over the pooled frames of the replicate, not an average of per-video values), and report the two-sided percentile interval
a tornado, “[...] often accompanied by strong winds [...]”, so it ignores the counterfactual premise, and the two score curves are again nearly identical. Across the three videos the score is nearly unchanged under the counterfactual prompt whether the explanation changes or not. between the 2.5th and the 97.5th percentiles of the replicate distribution. The intervals resample videos only. The difference is in frame-AUC points. The ratio is dimensionless and undefined on a replicate whose is not positive, so those replicates are dropped from the interval of the ratio only (at most 13 of 10,000 for any detector at ).
Two-sided empirical -values for a non-zero difference follow from the same replicate distribution. Under a Holm step-down at family-wise level 0.05, all six detectors reject; the largest -value is that of VadCLIP, and the Holm threshold at its rank is 0.05 itself.
Table S16 reports both ends of every interval in blocks (i) and (iv) and the lower end in block (iii). Blocks (ii) and (iii) split the test videos by label. Block (ii) is the pooled frame AUC over the anomalous videos of Table 3 of the main paper alone, as point estimates, since no resampling was run for that subset. Block (iii) is the paired mean shift of the raw detector score on the normal videos of Table 3 of the main paper, where the count-matched draw displaces the reconstruction farther (median MSE 0.051 against 0.009, Table S17) and moves the scores 2.4 to 5.7 times less. Block (iv) is the divergence-free ETU.
Table S19 reports the three masks of ETU at the three noise levels. The irrelevant mask stays within 0.09 at every level, and the relevant mask grows monotonically with . Table S17 and Table S18 report the displacement behind the
| Detector | at | at | at | ||||||
| 100 | 300 | 600 | 100 | 300 | 600 | 100 | 300 | 600 | |
| MGFN | −0.03 | −0.01 | −0.09 | 1.09 | 6.78 | 13.71 | 4.26 | 37.49 | 38.73 |
| UR-DMU | 0.00 | −0.02 | −0.02 | 0.61 | 5.21 | 14.49 | 0.43 | 4.03 | 6.43 |
| RTFM | 0.00 | 0.00 | 0.02 | 0.55 | 6.77 | 19.41 | 3.72 | 16.48 | 22.64 |
| DSANet | 0.00 | −0.01 | 0.01 | 0.40 | 4.23 | 10.79 | 4.92 | 37.52 | 37.99 |
| VadCLIP | −0.01 | −0.03 | −0.04 | 0.61 | 6.47 | 13.84 | 3.77 | 22.58 | 25.84 |
| GlanceVAD | 0.00 | −0.01 | −0.01 | 0.36 | 5.09 | 12.07 | 5.12 | 31.75 | 35.54 |
Table S19.
intervals of Table S16, at the level of the reconstruction and of the conditioning latent. On ETU the count-matched random draw displaces the reconstruction at least as far as the relevant mask on the normal videos and about as far on the anomalous ones, while on the divergence-trained generator the relevant-mask reconstruction of the anomalous videos is nearly orthogonal to the full-graph one (median cosine 0.051) and the random draw moves farther still (−0.217). At the level of the latent, the random latent is the farther of the two in cosine on 68% of the videos, and on the anomalous videos the two latents are within 0.024 of one another (−0.307 against −0.331). The asymmetry between the relevant and the random deletion on ETU is therefore not inherited from the geometry that the contrastive objective of Eq. (5) trains. This holds for the cosine only. In the relevant latent moves farther on every video set (14.3 against 9.8), since the hinge of Eq. (5) constrains the direction and not the magnitude, and ETU does not pass that difference on (Table S17).
Three-Teacher Ablation
The UCF-Crime videos were parsed by three VLM teachers, Qwen3-VL (the teacher used throughout the main paper), InternVideo3 and Gemma-4, with the prompts of Figure S4.
The models are prompted with the video and three prompts to obtain the nodes and edges of the three layers of the semantic graph that represents video . While we use a prompt for each layer, they all share the system prompt of Figure S4.
Table S20 measures how far the graphs of two teachers on the same video agree. The atoms agree above both a Table S18. In cosine, the conditioning latent moves at least as far under the count-matched random draw as under the relevant mask on all and on normal videos; in the relevant latent moves farther. Median cosine and distance of each masked GEM latent to on the videos of Table S17; ETU and the divergence-trained generator condition on this embedding. Bold marks the cells in which the random latent lies farther from than the relevant one.
Table S19. Noise-level sweep of ETU on all 290 UCF-Crime/CLIP test videos. Frame-AUC points lost from the full-graph reconstruction under each mask when the regeneration starts at noise level ; the default noise level, , is in bold for and .
Table S20. Agreement between the graphs of the three teachers. Atom cosines are matched-pair medians per node type (root / action / outcome); the same-class null pairs a video with another video of its own anomaly class, the random-video null with any other video. Edge compares the two teachers’ recovered edge sets on the same video. The paired win is the share of videos on which the same-video atom cosine exceeds the same-class null.
same-class null and a random-video null, while the edges agree little, with a same-video edge of 0.138–0.250, only 0.024 to 0.083 above the same-class null. We therefore treat the recovered topology as a conditioning structure and not as the structure of the scene.
Single-Node Masks
Table S21 reports the single-node masks of § 4.2 of the main paper. On ETU, one masked node changes the frame-AUC by less than the smallest relevant-mask delta of Table 2 of the main paper at (4.23 points). An on-chain node raises the AUC of all six detectors, by 0.04 to 0.28 points, and an off-chain node raises it for five detectors, by up to 1.80 points (RTFM), and lowers it by 0.20 points for VadCLIP. On the divergence-trained generator, by contrast, the on-chain node costs UR-DMU 20.30 points against 1.12 for the off-chain one, DSANet the reverse (1.77 against 23.97), and MGFN, DSANet, VadCLIP and GlanceVAD lose more under the off-chain node, although the two masks displace the reconstruction by the same median amount (median per-video ratio of off-chain to on-chain displacement 1.0002 in and 1.0014 in cosine distance).
Qualitative Results
The figures of this section are per-video score traces of the MIL detector of Sultani et al. [45] on XD-Violence with
CLIP features, under the three graph masks and under the masking of single named nodes. Table S22 reports single-node masks on two UCF-Crime test videos. MIL is not among the detectors audited in the main paper; it is the detector on which the configuration-selection criterion was computed (Section A.3). Every quantity here is a per-video curve or a per-video AP, not the test-set delta of Eq. (8) of the main paper. The traces illustrate what the measurement of the framework looks like on one video. They are not evidence that a named concept was removed from what the detector reads, and per-video AP is not monotone in the score level, so a mask can lower every score and raise the AP, or the reverse.
| Type | Concept | |||
| Action | Driving | |||
| Root | Car 1 | |||
| Outcome | Car 1 crash | |||
| Root | Pers. 2 | |||
| Outcome | Pers. 2 exit | |||
| Root | Person 1 | |||
| Root | Pers. 1 | |||
| Action | Takes | |||
| Root | Pers. 2 | |||
| Outcome | Pers. 1 exit | |||
| Root | Cashier | |||
| Action | Walks |
Table S22. Per-concept deltas (, on the 0–1 scale of the per-video evaluation; negative = a drop, the opposite sign of Eq. (8) of the main paper) from masking one named node at a time on two UCF-Crime test videos (CLIP features) for MIL (), UR-DMU () and RTFM (); not Eq. (8)’s test-set benchmark-point delta. The upper three rows of each video are nodes on an outcome path, the lower three are off every outcome path. An illustration of the measurement on single videos.
Figure S5 shows the four graph conditionings on one video. Semantic masking () drives every score below 0.05, relevant masking () lowers the scores inside the annotated windows against both the ground-truth features and the full-graph reconstruction (), and pruning off-path nodes () leaves the trace nearly identical to and close to the ground-truth curve. This is the same ordering that Table 1 of the main paper reports in the aggregate.
Figure S5. MIL anomaly-score traces on the Bad.Boys.1995-01-11-55-01-12-40 video of XD-Violence (CLIP features) under the ground-truth features and the four graph conditionings. In the legend, blue solid is , grey dotted , red dashed , green dash-dotted and black solid the ground-truth features. Removing the outcome chain () or substituting the scene semantics () lowers the scores of this anomalous video; pruning off-path nodes () does not.
In Figure S6, masking the root node Woman lowers the per-video AP by 0.042 against the ground-truth features (0.970 to 0.928) and by 0.012 against the full-graph reconstruction (0.940), and masking the action node Driving behaves alike (0.933); masking the outcome node Leaving lowers it by 0.171 (to 0.799) and pushes every score below 0.1, a larger change than the whole relevant mask on this video (0.894). The traces also show what the AP does not. Under Woman the scores outside the annotated window rise (more false positives) while under Driving they fall throughout. In Figure S6 the three masked nodes give 0.868 (Woman), 0.900 (Walking) and 0.913 (Sustaining) against 0.881 for the ground-truth features and 0.879 for , so two of the three single-node masks raise the AP while lowering most scores, and semantic masking raises it further (0.928). On this video the measurement is dominated by how the masks redistribute scores across the annotated window rather than by their level.

(a) Black Hawk Down (Scene 1)

(b) Black Hawk Down (Scene 2)
Figure S6. MIL anomaly-score traces for two Black Hawk Down scenes under the graph masks and under masking of single named nodes (per-video AP in the legends). The same node name moves the trace differently in the two scenes; see the text.
Figure S7 is a near-saturated case, in which the ground-truth features score 0.989 AP, the full-graph reconstruction 0.990, and every single-node mask lands within 0.004 of them (Woman 0.986, Walking 0.987, Sustaining 0.987) although the masked traces are visibly lower across the video; semantic masking lowers the AP to 0.976 while driving all scores below 0.5, and the relevant mask raises it to 0.994. When the anomalous window covers most of the clip, as here, a mask can lower every score and leave the ranking, and hence the AP, almost unchanged. This is why the main paper reports test-set deltas and tests their alternative explanations, rather than per-video curves.

Figure S7. MIL anomaly-score traces for a Braveheart scene. The annotated window covers most of the clip, so the single-node masks lower the scores while leaving the per-video AP within 0.004 of the ground-truth features (legend).
Figure S8 and S8b show the three masked nodes on two scenes of one film with very different outcomes. In the first scene masking the root node Man changes the AP by less than 0.01 (0.865 against 0.856 for the ground-truth features and 0.872 for ), masking the outcome node Leaving by 0.01 (0.883), while masking the action node Firing lowers it to 0.740 and suppresses the scores of the normal segment (frames 630 to 900) and of the final anomalous segment while raising those between frames 200 and 600. In the second scene masking Man lowers the AP from 0.600 to 0.399 and flattens the trace, Driving gives 0.457 and Leaving 0.560, and the relevant mask raises the AP to 0.622 while the full-graph reconstruction sits at 0.537. The response to a node name is therefore video-specific, and on single videos it is not monotone in what the mask removes, and the controlled, aggregate deltas of the main paper are the quantity on which the paper rests.

(a) (a) Brick Mansions (Scene 1)

(b) (b) Brick Mansions (Scene 2)
Figure S8. MIL anomaly-score traces for two Brick Mansions scenes. Masking the root node Man leaves the first scene’s AP within 0.01 of the ground-truth features and lowers the second scene’s from 0.600 to 0.399 (legends).
References
- [1]Shuai Bai et al. Qwen3-VL technical report. arXiv preprint arXiv:2511.21631, 2025. 3, 12arxiv.org/abs/2511.21631
- [2]Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. In International Conference on Learning Representations, 2022. 5, 14arxiv.org/abs/2105.04906
- [3]Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. LEACE: Perfect linear concept erasure in closed form. In Advances in Neural Information Processing Systems, 2023. 3arxiv.org/abs/2306.03819
- [4]João Carreira and Andrew Zisserman. Quo vadis, action recognition? A new model and the kinetics dataset. CoRR, abs/1705.07750, 2017. 1, 5
- [5]Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image classifiers by counterfactual generation. In International Conference on Learning Representations, 2019. 3arxiv.org/abs/1807.08024
- [6]Yingqian Chen, Zhenghe Liu, Baoheng Zhang, Wilton Fok, Xiaojun Qi, and Yik-Chung Wu. Mgfn: Magnitude-contrastive glance-and-focus network for weakly-supervised video anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 387–395, 2023. 2, 5
- [7]MyeongAh Cho, Minjung Kim, Sangwon Hwang, Chaewon Park, Kyungjae Lee, and Sangyoun Lee. Look around for anomalies: Weakly-supervised anomaly detection via context-motion relational learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12137–12146, 2023. 2DOI
- [8]Giacomo D’Amicantonio, Snehashis Majhi, Quan Kong, Lorenzo Garattoni, Gianpiero Francesca, François Brémond, and Egor Bondarev. Mixture of experts guided by gaussian splatters matters: A new approach to weakly-supervised video anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10275–10285, 2025. 2
- [9]Gabriele Dominici, Pietro Barbiero, Francesco Giannini, Martin Gjoreski, Giuseppe Marra, and Marc Langheinrich. Counterfactual concept bottleneck models. In International Conference on Learning Representations, 2025. 3arxiv.org/abs/2402.01408
- [10]Keval Doshi and Yasin Yilmaz. Towards interpretable video anomaly detection. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2655–2664, 2023. 1
- [11]Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. CRAFT: Concept recursive activation factorization for explainability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2711–2721, 2023. 3DOI
- [12]Amirata Ghorbani, James Wexler, James Y. Zou, and Been Kim. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems, 2019. 3arxiv.org/abs/1902.03129
- [13]Yash Goyal, Amir Feder, Uri Shalit, and Been Kim. Explaining classifiers with causal concept effect (cace). arXiv preprint arXiv:1907.07165, 2019. 3arxiv.org/abs/1907.07165
- [14]Peter Hase, Harry Xie, and Mohit Bansal. The out-of-distribution problem in explainability and search methods for feature importance explanations. In Advances in Neural Information Processing Systems, 2021. 3arxiv.org/abs/2106.00786
- [15]Ryota Hinami, Tao Mei, and Shin’ichi Satoh. Joint detection and recounting of abnormal events by learning deep generic knowledge. In Proceedings of the IEEE international conference on computer vision, pages 3619–3627, 2017. 3arxiv.org/abs/1709.09121
- [16]Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems, 2019. 2, 3arxiv.org/abs/1806.10758
- [17]Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations, 2017. 3arxiv.org/abs/1611.01144
- [18]Guillaume Jeanneret, Loïc Simon, and Frédéric Jurie. Diffusion models for counterfactual explanations. In Proceedings of the Asian Conference on Computer Vision, 2022. 3
- [19]Guillaume Jeanneret, Loïc Simon, and Frédéric Jurie. Adversarial counterfactual visual explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 3
- [20]Xiayan Ji, Anton Xue, Eric Wong, Oleg Sokolosky, and Insup Lee. AR-Pro: Counterfactual explanations for anomaly repair with formal properties. In Advances in Neural Information Processing Systems, 2024. 3arxiv.org/abs/2410.24178
- [21]Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In International Conference on Machine Learning, pages 2668–2677. PMLR, 2018. 3arxiv.org/abs/1711.11279
- [22]Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning. PMLR, 2020. 3arxiv.org/abs/2007.04612
- [23]H. W. Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83–97, 1955. 3
- [24]Sonia Laguna, Ričards Marcinkevičs, Moritz Vandenhirtz, and Julia E. Vogt. Beyond concept bottleneck models: How to make black boxes intervenable? In Advances in Neural Information Processing Systems, 2024. 3
- [25]Shuo Li, Fang Liu, and Licheng Jiao. Self-training multi-sequence learning with transformer for weakly supervised video anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1395–1403, 2022. 2DOI
- [26]Kun Liu and Huadong Ma. Exploring background-bias for anomaly detection in surveillance videos. In Proceedings of the 27th ACM International Conference on Multimedia, pages 1490–1499, 2019. 1DOI
- [27]Philipp Liznerski, Saurabh Varshneya, Ece Calikus, Puyu Wang, Alexander Bartscher, Sebastian Vollmer, Sophie Fellenz, and Marius Kloft. Reimagining anomalies: What if anomalies were normal? Proceedings of the AAAI Conference on Artificial Intelligence, 40(18):15430–15438, 2026. 3
- [28]Hui Lv and Qianru Sun. Video anomaly detection and explanation via large language models. arXiv preprint arXiv:2401.05702, 2024. 3arxiv.org/abs/2401.05702
- [29]Hui Lv, Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, and Hanwang Zhang. Unbiased multiple instance learning for weakly supervised video anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8022–8031, 2023. 2, 5arxiv.org/abs/2303.12369
- [30]Snehashis Majhi, Giacomo D’Amicantonio, Antitza Dantcheva, Quan Kong, Lorenzo Garattoni, Gianpiero Francesca, Egor Bondarev, and François Brémond. Just dance with pi! a poly-modal inductor for weakly-supervised video anomaly detection. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24265–24274, 2025. 2arxiv.org/abs/2505.13123
- [31]Yong-Hyun Park, Junghoon Seo, Bomseok Park, Seongsu Lee, and Junghyo Jo. Geometric remove-and-retrain (GOAR): Coordinate-invariant eXplainable AI assessment. In NeurIPS Workshop on XAI in Action: Past, Present, and Future Applications, 2023. 3arxiv.org/abs/2407.12401
- [32]William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023. 5arxiv.org/abs/2212.09748
- [33]Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, 2018. 5, 16arxiv.org/abs/1709.07871
- [34]Yujiang Pu, Xiaoyu Wu, Lulu Yang, and Shengjin Wang. Learning prompt-enhanced context features for weakly-supervised video anomaly detection. IEEE Transactions on Image Processing, 33:4923–4936, 2024. 5DOI
- [35]Didik Purwanto, Yie-Tang Chen, and Wen-Hsien Fang. Dance with self-attention: A new look of conditional random fields on anomaly detection in videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 173–183, 2021. 2DOI
- [36]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021. 1, 5arxiv.org/abs/2103.00020
- [37]Vikram V. Ramaswamy, Sunnie S. Y. Kim, Ruth Fong, and Olga Russakovsky. Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10932–10941, 2023. 3arxiv.org/abs/2207.09615
- [38]Tal Reiss and Yedid Hoshen. Attribute-based representations for accurate and interpretable video anomaly detection. arXiv preprint arXiv:2212.00789, 2022. 2, 3
- [39]Yao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci, and Enkelеjda Kasneci. A consistent and efficient evaluation strategy for attribution methods. In Proceedings of the 39th International Conference on Machine Learning, pages 18770–18795, 2022. 2, 3arxiv.org/abs/2202.00449
- [40]Litu Rout, Yujia Chen, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Beyond first-order tweedie: Solving inverse problems using latent diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9472–9481, 2024. 5arxiv.org/abs/2312.00852
- [41]Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017. 1
- [42]Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 761–769, 2016. 4arxiv.org/abs/1604.03540
- [43]Ashish Singh, Michael J Jones, and Erik G Learned-Miller. Eval: Explainable video anomaly localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18717–18726, 2023. 3arxiv.org/abs/2212.07900
- [44]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. 5arxiv.org/abs/2010.02502
- [45]Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6479–6488, 2018. 2, 5, 16, 19arxiv.org/abs/1801.04264
- [46]Stanislaw Szymanowicz, James Charles, and Roberto Cipolla. X-man: Explaining multiple sources of anomalies in video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3224–3232, 2021. 3arxiv.org/abs/2106.08856
- [47]Stanislaw Szymanowicz, James Charles, and Roberto Cipolla. Discrete neural representations for explainable anomaly detection. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 148–156, 2022. 3arxiv.org/abs/2112.05585
- [48]Fengwei Teng, Zhaoyang Yu, Quan Shi, Jiayi Zhang, Chenglin Wu, and Yuyu Luo. Atom of thoughts for markov llm test-time scaling. arXiv preprint arXiv:2502.12018, 2025. 12arxiv.org/abs/2502.12018
- [49]Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W Verjans, and Gustavo Carneiro. Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4975–4986, 2021. 2, 5arxiv.org/abs/2101.10030
- [51]Payal Varshney, Adriano Lucieri, Christoph Balada, Sheraz Ahmed, and Andreas Dengel. LD-ViCE: Latent diffusion model for video counterfactual explanations. arXiv preprint arXiv:2509.08422, 2025. 3arxiv.org/abs/2509.08422
- [52]Boyang Wan, Yuming Fang, Xue Xia, and Jiajie Mei. Weakly supervised video anomaly detection via center-guided discriminative learning. In 2020 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2020. 2
- [53]Chao Wang, Chengan Che, Xinyue Chen, Sophia Tsoka, and Luis C. García-Peraza-Herrera. Back to the feature: Explaining video classifiers with video counterfactual explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. 3arxiv.org/abs/2511.20295
- [54]Yizhou Wang, Dongliang Guo, Sheng Li, Octavia Camps, and Yun Fu. Unveiling the unseen: A comprehensive survey on explainable anomaly detection in images and videos. arXiv preprint arXiv:2302.06670, 2023. 1arxiv.org/abs/2302.06670
- [55]Nina Weng, Paraskevas Pegios, Eike Petersen, Aasa Feragen, and Siavash Bigdeli. Fast diffusion-based counterfactuals for shortcut removal and generation. In Proceedings of the European Conference on Computer Vision, 2024. 3arxiv.org/abs/2312.14223
- [56]Peng Wu and Jing Liu. Learning causal temporal relation and feature discrimination for anomaly detection. IEEE Transactions on Image Processing, 30:3513–3527, 2021. 2DOI
- [57]Peng Wu, Jing Liu, Yujia Shi, Yujia Sun, Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX, pages 322–339. Springer, 2020. 2, 5arxiv.org/abs/2007.04687
- [58]Peng Wu, Xuerong Zhou, Guansong Pang, Lingru Zhou, Qingsen Yan, Peng Wang, and Yanning Zhang. Vadclip: Adapting vision-language models for weakly supervised video anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6074–6082, 2024. 5arxiv.org/abs/2308.11681
- [59]Keyulu Xu, Weihuа Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018. 14arxiv.org/abs/1810.00826
- [60]Yuchen Yang, Kwonjoon Lee, Behzad Dariush, Yinzhi Cao, and Shao-Yuan Lo. Follow the rules: Reasoning for video anomaly detection with large language models. In European Conference on Computer Vision, pages 304–322. Springer, 2024. 3arxiv.org/abs/2407.10299
- [61]Muchao Ye, Weiyang Liu, and Pan He. VERA: Explainable video anomaly detection via verbalized learning of vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8679–8688, 2025. 3arxiv.org/abs/2412.01095
- [62]Wenti Yin, Huaxin Zhang, Xiang Wang, Yuqing Lu, Yicheng Zhang, Bingquan Gong, Jialong Zuo, Li Yu, Changxin Gao, and Nong Sang. Learning to alert: Weakly supervised video anomaly detection via disentangled semantic alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026. arXiv:2511.10334. 5arxiv.org/abs/2511.10334
- [63]Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. In International Conference on Learning Representations, 2023. 3arxiv.org/abs/2205.15480
- [64]Luca Zanella, Willi Menapace, Massimiliano Mancini, Yiming Wang, and Elisa Ricci. Harnessing large language models for training-free video anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18527–18536, 2024. 3arxiv.org/abs/2404.01014
- [65]Huaxin Zhang, Xiaohao Xu, Xiang Wang, Jialong Zuo, Chuchu Han, Xiaonan Huang, Changxin Gao, Yuehuan Wang, and Nong Sang. Holmes-vad: Towards unbiased and explainable video anomaly detection via multi-modal llm. arXiv preprint arXiv:2406.12235, 2024. 1, 2arxiv.org/abs/2406.12235
- [66]Huaxin Zhang, Xiang Wang, Xiaohao Xu, Xiaonan Huang, Chuchu Han, Yuehuan Wang, Changxin Gao, Shanjun Zhang, and Nong Sang. Glancevad: Exploring glance supervision for label-efficient video anomaly detection. In IEEE International Conference on Multimedia and Expo (ICME), 2025. arXiv:2403.06154. 5arxiv.org/abs/2403.06154
- [67]Huaxin Zhang, Xiaohao Xu, Xiang Wang, Jialong Zuo, Xiaonan Huang, Changxin Gao, Shanjun Zhang, Li Yu, and Nong Sang. Holmes-vau: Towards long-term video anomaly understanding at any granularity. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 13843–13853, 2025. 1, 2, 3, 6, 17arxiv.org/abs/2412.06171
- [68]Jiangong Zhang, Laiyun Qing, and Jun Miao. Temporal convolutional network with complementary inner bag loss for weakly supervised anomaly detection. In 2019 IEEE International Conference on Image Processing (ICIP), pages 4030–4034. IEEE, 2019. 2DOI
- [69]Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems, 31, 2018. 12arxiv.org/abs/1803.01422
- [70]Xu Zheng, Farhad Shirani, Zhuomin Chen, Chaohao Lin, Wei Cheng, Wenbo Guo, and Dongsheng Luo. F-fidelity: A robust framework for faithfulness evaluation of explainable AI. In International Conference on Learning Representations, 2025. 3arxiv.org/abs/2410.02970
- [71]Jia-Xing Zhong, Nannan Li, Weijie Kong, Shan Liu, Thomas H. Li, and Ge Li. Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2arxiv.org/abs/1903.07256
- [72]Hang Zhou, Junqing Yu, and Wei Yang. Dual memory units with uncertainty regulation for weakly supervised video anomaly detection. arXiv preprint arXiv:2302.05160, 2023. 5arxiv.org/abs/2302.05160
- [73]Yi Zhu and Shawn Newsam. Motion-aware feature for improved video anomaly detection. arXiv preprint arXiv:1907.10211, 2019. 2arxiv.org/abs/1907.10211

