WorksComputer Science
Causal Concept Explanations for Deep Neural Models
Abstract
Concept-based explanations in neural models are important for the development, certification, and adoption of autonomous systems. Yet, interpretability remains correlational. Without methods to guide and quantify causal relationships between explanations, a concept explanation can be provided without being necessary or sufficient for the observed output thereby appearing correct while misleading. We introduce Causal Concept Wrapper Network (CCW-Net), the first use of interventional mediation analysis from fields such as economics, medicine, and epidemiology as a training objective to align causal effects of concept-based explanations with a model's output. We further apply mediation analysis as an evaluation tool to quantify, per concept label combination, how much of any model's output is causally attributable to each concept explanation. As a training objective, CCW-Net shapes a model's concept representations such that they are both necessary and sufficient for the outputs they are credited with and independent of other concepts' contributions. Counterfactual concept representations anchor a set of causal alignment criteria: necessity, sufficiency, and independence. Necessity drives irrelevant concept effects toward zero, sufficiency ensures relevant concept representations fully reconstruct a model's output, and independence enforces the total effect decomposes linearly across concepts. We evaluate CCW-Net across aircraft control, driving, and image classification. CCW-Net provides a step toward concept-based explanations that are not merely interpretable, but quantifiably causal, supporting safer evaluation, certification, and deployment of transparent and trustworthy neural models.
Introduction
Learned policies in autonomous systems are moving from research labs into high-stakes, real-world settings such as aviation [49, 78], driving [48], and robotics [69]. This is also the case for perception models in safety-relevant settings such as medical imaging. In task and safety-critical domains such as these, interpretability is becoming a necessary component [54, 4]. Developers need it to troubleshoot [29], testers need it to verify [52, 42], and users need it to understand [56]. Yet many neural models remain black boxes: they are not interpretable in a way that is useful for people in the task at hand.
Concept-based interpretability offers promise by structuring decisions through human-interpretable concepts [33] (Echthroff et al. 2024). However, the link between concepts and outputs is typically correlational, not causal. A concept can activate alongside an output without being in the causal chain [80]. An autonomous vehicle policy might activate a “passenger pickup” concept that has no causal bearing on the observed action. A bird classifier may activate “has red plumage” on every image of a cardinal while its prediction is actually driven by “has crest”. The explanation appears correct but may mislead. To be useful for troubleshooting, certifying, and auditing, explanations must capture causal effects, i.e., the contribution of each concept to the output, isolated from concepts it co-occurs with [46]. Correlation, no matter how sophisticated, does not address such causal questions [45]. For safety and task-critical deployment we must transition from correlation to causation.
Beyond quantifying causal effects, some concept representations themselves limit what explanations can convey. Prior work typically represents concepts as scalars [41, 33, 29], e.g., a “left turn” in driving or a “has yellow belly” attribute in bird classification. Scalars indicate the presence or strength of a concept but cannot capture their higher-dimensional, context-dependent expressions. We extend concept representations from scalars to vectors, enabling each concept to encode both whether it exists and how it is expressed.
We address these challenges with Causal Concept Wrapper Network (CCW-Net), the first use of mediation analysis as a training objective in deep networks. CCW-Net introduces a network architecture and adapts interventional multi-mediator mediation analysis from economics and epidemiology [73, 37, 8, 35] to shape a model’s concept representations so that each concept’s contribution to the output is locally necessary, sufficient, and independent of contributions from other concepts. Counterfactuals required by these criteria are drawn from a mediator model (in our experiments, a per-concept invertible flow) trained alongside the overall model to track the conditional distribution of its concept embeddings. We further apply mediation analysis as an evaluation tool, quantifying, for each concept label combination, the share of a model’s output causally attributable to each concept. This is finer-grained than the population-level alignment scores reported in prior concept-based work. We evaluate CCW-Net across aircraft control, driving, and fine-grained image classification, demonstrating that mediation-trained concepts produce explanations that are not merely interpretable, but quantifiably causal.
Main contributions. Our contributions are threefold:
Training objective for causal concepts in deep networks: An interventional mediation analysis based objective that shapes each concept’s causal effect on a model’s output so that activated concepts are those causing the output thereby improving explanation trustworthiness.
Network architecture for concept-level mediation analysis: A network architecture that drives absent concept representations toward zero supporting well-defined interventional mediation analyses and related training objectives (App. C.1).
Evaluation method for concept-based causality claims: For any concept-bottleneck model, a method to quantify, for each concept label combination, how much a change in the model’s output is causally attributable to each concept (App. F).
Related work
Concept-based explanations. Concept-based interpretability is widely explored [31, 3, 33, 5, 1, 68]. Concept Bottleneck Models [33] and adaptations to sequential decision-making [28, 29, 12, 10] structure decisions through human-interpretable concepts but typically lack formal causal analysis.
Causal mediation analysis. Causal mediation analysis models systems as directed acyclic graphs and decomposes an input’s effect on an outcome into pathways through intermediate variables called mediators. Foundational tools have been developed in economics, medicine, and epidemiology [40, 25, 72, 46, 8, 35]. Multi-mediator extensions [73, 37] decompose total effects across multiple mediators without requiring causal ordering or independence assumptions among them, with indirect effects quantifying how much of an effect operates through a specific mediator. These tools are designed to estimate causal effects from observational data under strict assumptions (App. D), but to date have been applied to analyze causal relationships in data collected outside randomized controlled trials or trained models post-hoc, not as training objectives that shape deep networks’ representations during learning.
Causal and disentanglement-based explanations for deep networks. A growing body of work applies causal frameworks to trained deep networks. Vig et al. [ [76] applied mediation analysis post hoc to study bias in transformer language models, treating individual neurons and attention heads as mediators. Geiger et al. [ [17, 19] formalize causal abstraction as a way to validate that a trained network implements a higher-level causal model. In concept-based interpretability specifically, recent work imposes or discovers causal structure over learned concepts [11, 13, 50]. Disentanglement methods like -TCVAE [9] and iVAE [30] learn factorized representations evaluated by DCI [15] and FactorVAE score [32]. Across these methods, causal or representational analysis is applied post hoc to a model trained with conventional objectives, and disentanglement is a representational property that need not imply causal alignment [58]. CCW-Net differs in that causal relationships between concept representations and model outputs are shaped during training rather than audited post-hoc.
Feature attribution. Saliency-based [62, 61, 63, 67] and perturbation-based [51, 38] attribution methods explain decisions by associating inputs or features to outputs. While useful for highlighting what features a model’s outputs correspond to, these methods generally remain correlational. Ablating a feature changes a prediction without establishing causal responsibility [2, 80]. CCW-Net grounds causal attribution in interventional indirect effect estimates, yielding per-concept attributions with quantifiable causal estimates.
Problem formulation and preliminaries
We seek to train an interpretable policy that achieves expert-level task performance, provides concept-based explanations for its predicted outputs , and ensures concept–output relationships reflect causal effects rather than spurious correlations. We assume access to a frozen pretrained backbone that achieves good task performance but lacks interpretability, and a dataset where is an input, is the expert output, and is a vector of human-interpretable concept labels. We assume the concept ontology captures the key factors driving the model’s outputs, the dataset provides sufficient ground truth, and standard ignorability conditions hold for the induced concept-to-output structural causal model (App. D). We jointly optimize task fidelity, concept correctness, and causal alignment (§ 4), with concept–output causal relationships quantified in evaluation.
Structural causal models. A structural causal model (SCM) [45, 47] consists of equations where each equation represents a mechanism determining exactly one variable, with the exogenous error term and the endogenous parent variables that directly cause .
Interventional mediation analysis. Mediation analysis [46] decomposes the causal effect of a treatment on an outcome into pathways that operate through variables called mediators . For mediators , the total effect TE decomposes as TE = DE + IE into direct effect (DE) and indirect effect (IE) components. With multiple mediators, the indirect effect further decomposes per mediator as IE IE. Following Loh et al. [ [37], we relax the assumption of causal ordering or independence among mediators by introducing a residual interaction term IE, yielding TE = DE IE + IE. This relaxation is important for deep networks where operations like softmax and cross-attention induce dependencies among concept activations, which IE absorbs without violating identifiability. The interventional indirect effect IE [73] quantifies how much of the total effect operates through mediator when its representation is set to a counterfactual value while other mediator representations are held fixed. These quantities are identifiable from observational data under standard ignorability conditions [72, 73, 37], which hold by construction in the wrapper SCM (App. D).

Figure 1. CCW-Net architecture. A concept encoder maps the frozen backbone’s latent to one squashed vector per concept, whose length encodes the concept’s presence and whose direction encodes its context-dependent expression, and a bias-free attention head maps to the wrapper output . Causal alignment losses (weights and the auxiliary presence term omitted) are added to the task and concept losses, so that training shapes, and evaluation measures, how much each concept causally drives .
Causal Concept Wrapper Network (CCW-Net)
CCW-Net wraps a frozen backbone policy with a concept encoder and new policy head which are trained to imitate the backbone’s task performance as well as classify human-interpretable concepts similar to Kenny et al. [ [29] but using a different architecture. In addition, training involves three causal alignment criteria to induce causal relationships between concepts and outputs: necessity, sufficiency, and independence. These criteria are then evaluated via interventional mediation analysis on the wrapper’s outputs. Each causal alignment loss compares the head’s output before and after one change of a training input’s resulting concept ’s representation. In mediation analysis terms, the concept representations are mediators and the policy head is the outcome model. Every effect discussed is the model’s own concept-to-output effect within the wrapper architecture’s SCM, not a claim about inputs or observations to output effects. For our experiments, counterfactual concept representations required by the necessity alignment term are drawn from a per-concept invertible flow trained on the wrapper’s encoder concepts. Any suitable modeling approach may be used. Reported evaluation metrics are produced from a separate estimator that does not use the mediator model.
Architecture
Given a frozen pretrained backbone where is the frozen backbone’s last latent layer, a concept encoder maps to a concept representation . Each represents the -th human-interpretable concept. An attention head maps the concept representation to a wrapper output . The wrapper induces the SCM with no direct edge so the input reaches the output only through the concept representations (DE=0), which gives mediator-outcome ignorability by construction (App. § 11, Prop. 2). The concept encoder is a multi-layer perceptron producing concept representation vectors of dimension , each passed through a squash nonlinearity (Sabour et al., [55]) mapping each slot length to . The concept classification loss trains length as presence leaving the direction free to encode the concept’s context-based expression.
The concept classification loss drives inactive and active representations toward length zero and one, respectively. A zero concept vector representation carries a zero value vector in the attention head, so zeroing a concept removes its value contribution to regardless of its attention weight (Prop. 1).
Mediator model: Counterfactual concepts via a per-concept flow
Necessity asks whether the lack of a given concept’s representation results in a lack of that concept manifesting in the model’s outcome. In interventional mediation analysis, this is tested by replacing an inactive concept representation with an active one for the same input case. By definition, this is an unobservable counterfactual case. To overcome this, we reflect the approach of Vansteelandt and Daniel [ [73] and draw counterfactual mediator expressions from a mediator model learned from observational data. In our experiments, we invoke a per-concept invertible flow to model mediator distributions and draw counterfactual examples where the concept label is flipped between present and not-present,
We fit the flow by maximum likelihood to the encoder’s mediator representations after a warmup (§ 4.4, App. § 14, Algorithm 1). Our method is not specific to flow modeling. Any modeling approach may be chosen that best serves the domain.
Causal alignment criteria
Each causal alignment criterion (necessity, sufficiency, independence) is stated in terms of a change in the head output caused by an edit of a concept vector. Let be the all-zero concept vector, the all-zero vector with slot set to its realized value , and the realized concepts with slot replaced by its counterfactual draw (Eq. (1)). We define three changes:
switches every concept slot on from to its realized value, switches slot alone on from , and swaps slot for its counterfactual draw, keeping all other slots at their realized values. The subscript indicates the edited slot, and the superscript indicates the edit: switching slots on from (0) or swapping in a counterfactual draw from the mediator model (cf). In mediation analysis terms, the concept label flip in place of a treatment change gives as the per-sample analog of the interventional indirect effect of § 3 and the analogous contrast of concept slot against zero with the other slots at zero (App. § 10). § 5.4 provides the estimator for the reported effects.
Each causal alignment loss is based on these quantities grounded on dataset labels rather than on predicted concept presence. Below, we outline each causal alignment term with their formulations in Eq. (3).
Necessity. Inactive concepts should have no effect on the model’s output. Replacing an inactive concept slot with a counterfactual should not change the output such that .
Sufficiency. Active concepts alone should produce the expert output: Starting from the all-zero output and adding each active concept’s separate contribution should produce the expert output, .
Independence. Concept representations should contribute additively: Each concept should contribute on its own, and the per-concept contributions should sum to the total, .
Presence (auxiliary). Since sufficiency constrains only the sum of concept representations, a subset of active concepts could carry all of the effect. To guard against that, we add an optional term that requires each active concept to carry at least an equal share of the total, whenever .
Training objective
We combine the four alignment losses into a single alignment term
and train the wrapper by minimizing
is the supervised mean-squared error on (to the expert action for control, to the one-hot species label for CUB-200) and trains each concept slot’s vector length to predict its concept label. Training proceeds in three phases (Algorithm 1): warmup on and , an intermission that fits the mediator model on the frozen encoder’s concepts, and alignment on the full objective, with optional mediator model refits at fixed intervals. Loss weights, schedules, and architecture details are in Apps. § 12–§ 14.
Empirical evaluation
We evaluate CCW-Net on three domains spanning control and perception: aircraft control, the CarRacing-v1 driving environment [7], and image classification on CUB-200 [77].
Tasks
Aircraft control. We evaluate on extended trail formation [70, 71], in which a chase aircraft maintains position behind a lead aircraft executing dynamic maneuvers. The simulator is a high-fidelity aircraft model with six degrees of freedom [22] where the lead aircraft follows a scripted trajectory and the chase aircraft is governed by a reinforcement learning policy trained by proximal policy optimization [59] using the aircraft environment and codebase of So and Fan [ [64]. The action space is two-dimensional (pitch and roll commands). Concept labels are binary attributes (Lead pursuit, Lag pursuit, Climb, Dive) drawn from operational pilot vocabulary [70, 71]. Pursuit (Lead pursuit, Lag pursuit) and vertical (Climb, Dive) maneuver concept pairs are mutually exclusive, so exactly two concepts are active per sample. Full task details are in Appendix § 15.§ 1.
CarRacing. We evaluate on the CarRacing-v1 environment [7], in which a pixel-input agent drives a car around a track. The frozen backbone is a CNN policy trained end-to-end on the task. Its latents, actions, and concept labels come from a CW-Net test domain [29]. The action space is continuous steering, throttle, and brake. Concept labels are binary attributes (Left, Right, Straight). The concepts are mutually exclusive and exhaustive, so exactly one concept is active per sample and the presence threshold is . Full task details are in Appendix § 15.§ 2.
CUB-200 classification. We evaluate on the CUB-200-2011 dataset of 200 bird species [77]. The frozen backbone is an Inception-v3 image classifier fine-tuned on CUB. Concept labels are binary visual attributes (e.g., “has yellow belly”, “has pointed beak”) and the wrapper output is a 200-way classification. Full task details are in Appendix § 15.§ 3.
Baselines
Same-architecture, no alignment. We compare CCW-Net against the same encoder and head trained without the alignment losses (only and active). This isolates the contribution of the causal alignment objective from the contribution of the encoder architecture.
Concept Wrapper Network variants. On the aircraft and CarRacing tasks we compare against the Concept Wrapper Network (CW-Net) of Kenny et al. [ [29], using the authors’ architecture. CW-Net wraps a frozen backbone with a concept layer in the same spirit as CCW-Net, but implements a different architecture, represents each concept as a scalar, and trains without causal alignment. Appendix § 16 adds CW-Net variants trained with alignment and, on CarRacing, with vector concepts.
Concept bottleneck model. For CUB-200 we compare against the concept bottleneck model of Koh et al. [ [33], which trains a model to predict concepts and to derive the task output through them.
Disentanglement-based concept learning. To support the secondary claim that disentanglement does not imply causal alignment, we compare against -TCVAE [9] and iVAE [30] on the aircraft and CarRacing tasks, reporting DCI [15] alongside the concept effect fractions (Appendix § 16).
Ablations
We run five ablation types on the control domains: (i) loss combinations where subsets of the four alignment losses are disabled to test each one’s contribution, (ii) concept representation dimension with default, scalar, and , (iii) data quantity for aircraft ranging from 5K to 5M samples, (iv) label quality as fraction of data labeled with concepts and injected label noise, and (v) concept ontology where we remove concepts, merge concepts, and add a placebo concept. The full ablation matrix is in Appendix § 16. The CUB-200 models train without the alignment losses and compare architectures (§ 5.5.3).
Evaluation methodology
We report task performance (mean-squared error for action regression), concept accuracy against data labels, DCI [15], and the causal effect each concept has on the model’s output. All values are means over ten seeds with two-sided 95% Student- intervals. The primary metrics in this study are causal effects.
Causal attribution. To test whether a model’s concept representations carry only their own effects, we estimate interventional indirect effects [73, 37] from held-out observational data, using no mediator model, no zero reference 0, and none of the training losses. The approach applies to any model whose output is a function of its concept representations. We use it on CCW-Net, CW-Net, and the disentanglement baselines.
Each comparison takes two sets of test inputs, and , whose concept labels differ where an active concept in is replaced in by a mutually exclusive alternative with all other labels unchanged. For a test input , we draw matches from and by Gaussian-kernel matching on z-scored features (raw inputs or for image inputs, backbone features ). A test input is sampled with probability proportional to among those within a radius of , where is the root-mean-square distance between and and is the kernel bandwidth. If none are within the radius, we take the nearest neighbor. The matches’ concepts fill the concept slots. The two slots whose labels differ form one group, so every hybrid has a label combination that occurs in the data, and every other slot is a group of its own. We then build hybrid concept vectors taking each group of concept mediator representation slots from the or the match, and pass them through the trained head . Group ’s effect is the change in output when it switches from its match to its match,
where () are group ’s slots from an () match, drawn independently for each group, and likewise for the groups () before (after) in an ordering . Averaging over all orderings makes equal the Shapley value for the switch from to , and runs over the inputs of both sets. Estimator details are in App. § 13.
For each label combination (rows in Table 1 and Table 2), slot ’s effect is the norm of its own group’s effect in the comparisons that keep ’s label fixed, i.e., how far slot moves the output when another concept changes. The fractions of the effect that slot , the active concepts, and the inactive concepts carry are
| Label combination | Lead | Lag | Climb | Dive | All Active ↑ | All Inactive ↓ |
| CCW-Net (Ours) | ||||||
| tMSE 0.125 ±0.001; concept acc. 95.7 ±0.1 / 99.3 ±0.0; DCI 0.999 ±0.001 / 0.719 ±0.041 / 0.975 ±0.001 | ||||||
| Lead+Climb | 62.8 ±16.1 | 4.6 ±1.9 | 32.0 ±16.1 | 0.6 ±0.4 | 94.8 ±2.1 | 5.2 ±2.1 |
| Lead+Dive | 69.2 ±15.7 | 5.4 ±2.9 | 1.0 ±0.5 | 24.4 ±16.0 | 93.6 ±3.3 | 6.4 ±3.3 |
| Lag+Climb | 4.3 ±2.3 | 60.5 ±13.2 | 34.5 ±13.5 | 0.7 ±0.4 | 95.0 ±2.5 | 5.0 ±2.5 |
| Lag+Dive | 4.9 ±2.4 | 67.8 ±13.0 | 1.1 ±0.6 | 26.2 ±13.3 | 94.0 ±2.6 | 6.0 ±2.6 |
| CCW-Net, no causal alignment losses | ||||||
| tMSE 0.124 ±0.001; concept acc. 95.7 ±0.1 / 99.3 ±0.0; DCI 0.999 ±0.000 / 0.746 ±0.037 / 0.975 ±0.001 | ||||||
| Lead+Climb | 63.7 ±21.0 | 9.5 ±6.0 | 25.0 ±16.3 | 1.8 ±2.0 | 88.7 ±7.2 | 11.3 ±7.2 |
| Lead+Dive | 64.8 ±20.0 | 9.6 ±5.6 | 4.1 ±3.9 | 21.5 ±14.0 | 86.3 ±6.8 | 13.7 ±6.8 |
| Lag+Climb | 8.0 ±2.8 | 62.6 ±16.9 | 27.3 ±15.6 | 2.0 ±2.0 | 89.9 ±3.7 | 10.1 ±3.7 |
| Lag+Dive | 8.5 ±4.3 | 63.4 ±17.9 | 5.2 ±6.3 | 22.9 ±14.6 | 86.3 ±6.3 | 13.7 ±6.3 |
| CW-Net [29] | ||||||
| tMSE 0.122 ±0.002; concept acc. 91.2 ±0.2 / 98.2 ±0.1; DCI 0.998 ±0.001 / 0.719 ±0.064 / 0.947 ±0.001 | ||||||
| Lead+Climb | 30.5 ±11.8 | 41.0 ±11.0 | 10.6 ±4.5 | 17.8 ±10.6 | 41.2 ±12.7 | 58.8 ±12.7 |
| Lead+Dive | 25.2 ±11.2 | 31.4 ±8.6 | 17.3 ±6.6 | 26.0 ±9.8 | 51.3 ±6.3 | 48.7 ±6.3 |
| Lag+Climb | 31.5 ±13.2 | 38.8 ±13.2 | 11.5 ±4.7 | 18.2 ±8.4 | 50.3 ±13.0 | 49.7 ±13.0 |
| Lag+Dive | 26.1 ±13.1 | 28.2 ±9.2 | 18.3 ±6.5 | 27.4 ±8.5 | 55.6 ±14.1 | 44.4 ±14.1 |
Table 1. Aircraft control: fraction of the effect each concept carries (Eq. (7)), per concept label combination, estimated as in § 5.4. Active concepts are in bold. All Active and All Inactive are and . Each block’s header gives the model’s test action MSE (tMSE), concept accuracy on the Lead/Lag and Climb/Dive concept sets (%), and DCI disentanglement / completeness / informativeness. Means over ten seeds with two-sided 95% Student- intervals. CCW-Net shows superior causal effect routing through active concepts compared to prior art.
| Label combination | Straight | Left | Right | All Active ↑ | All Inactive ↓ |
| CCW-Net (Ours) | |||||
| tMSE 0.0419 ±0.0005; concept acc. 92.6 ±0.4; DCI 0.863 ±0.023 / 0.863 ±0.023 / 0.943 ±0.005 | |||||
| Straight | 98.1 ±1.1 | 1.3 ±0.9 | 0.5 ±0.4 | 98.1 ±1.1 | 1.9 ±1.1 |
| Left | 25.2 ±7.0 | 74.5 ±7.0 | 0.3 ±0.2 | 74.5 ±7.0 | 25.5 ±7.0 |
| Right | 23.5 ±5.8 | 0.7 ±0.4 | 75.8 ±5.8 | 75.8 ±5.8 | 24.2 ±5.8 |
| CCW-Net, no causal alignment losses | |||||
| tMSE 0.0419 ±0.0006; concept acc. 92.4 ±0.4; DCI 0.850 ±0.021 / 0.851 ±0.021 / 0.944 ±0.005 | |||||
| Straight | 95.4 ±3.3 | 3.5 ±2.5 | 1.2 ±0.9 | 95.4 ±3.3 | 4.6 ±3.3 |
| Left | 26.2 ±9.2 | 73.2 ±9.4 | 0.6 ±0.3 | 73.2 ±9.4 | 26.8 ±9.4 |
| Right | 24.5 ±7.5 | 1.7 ±1.0 | 73.9 ±7.9 | 73.9 ±7.9 | 26.1 ±7.9 |
| CW-Net [29] | |||||
| tMSE 0.0015 ±0.0007; concept acc. 90.0 ±1.3; DCI 0.078 ±0.063 / 0.626 ±0.082 / 0.928 ±0.008 | |||||
| Straight | 33.2 ±1.7 | 37.4 ±7.4 | 29.4 ±6.8 | 33.2 ±1.7 | 66.8 ±1.7 |
| Left | 33.7 ±11.0 | 32.9 ±1.4 | 33.4 ±10.4 | 32.9 ±1.4 | 67.1 ±1.4 |
| Right | 29.7 ±7.0 | 37.3 ±6.5 | 33.1 ±2.1 | 33.1 ±2.1 | 66.9 ±2.1 |
Table 2. CarRacing: fraction of the effect each concept carries (Eq. (7)), per concept label combination, estimated as in § 5.4. Active concepts are in bold. All Active and All Inactive are and . Each block’s header gives the model’s test action MSE (tMSE), concept accuracy on the Straight/Left/Right concept set (%), and DCI disentanglement / completeness / informativeness. Means over ten seeds with two-sided 95% Student-t intervals. CCW-Net shows superior causal effect routing through active concepts compared to prior art.
Inactive concepts should carry no effect, so a faithful model has near zero.
On CUB-200, we report a separate per-input evaluation. Its 112 attribute labels are assigned per species and have no mutually exclusive alternatives within the dataset, so the comparisons above, which hold all other labels fixed, cannot be formed with enough test inputs. Instead, each concept in turn is set to absent (the zero slot for CCW-Net, concept probability 0 for CBM), is its share of the summed output changes at each test input, and we average over the test split. This uses the zero reference, but no CUB-200 model is trained with the alignment losses, so it reuses no optimized terms.
Results
Aircraft control
Table 1 reports, for each concept label combination, the fraction of the effect each concept representation mediator carries. CCW-Net (concept representation dimension ) routes of the effect through the active concepts, versus trained without the causal alignment losses maintaining comparable task error and concept accuracy. CW-Net predicts its concepts nearly as accurately ( and on the two mutually exclusive concept sets), yet its active concepts carry only of the effect. This is the fraction expected if the effect were spread indiscriminately across the four concepts. This suggests accurate concept prediction alone does not make the model’s outcomes depend on them. We find DCI disentanglement is at its ceiling of 1 for every model in Table 1 suggesting that disentanglement metrics may be insufficient to imply causal relationships between representation and model outcomes. The disentanglement baseline iVAE reaches DCI disentanglement, completeness and informativeness of 1.000, 0.891 and 1.000, yet its active concepts carry only of the effect (App. § 16.§ 1).
CarRacing
Table 2 reports the same for CarRacing, where one of the three concepts is active in each label combination. CCW-Net (concept representation dimension ) routes of the effect through the active concept, versus trained without the causal alignment losses at same task error and comparable concept accuracy, so on CarRacing the architecture accounts for most of the causal routing. While CW-Net reaches lower task error than CCW-Net, both policies were able to complete the task upon visual inspection. While both approaches predict their concepts nearly as accurately (92.6 ± 0.4% and 92.4 ± 0.4% vs 90.0 ± 1.3%) CW-Net’s active concept carries only 33.1 ± 1.6% of the effect, the fraction expected if the effect were spread indiscriminately across the three concepts.
CUB-200
Table 3 reports CUB-200. Neither CCW-Net nor CBM is trained with the causal alignment losses, so this is a comparison that isolates the architecture. CCW-Net (concept representation dimension ) routes 91.3 ± 0.3% of the effect through active concepts, versus 41.6 ± 0.6% for CBM with both predicting their concepts accurately (99.6 ± 0.0% and 97.7 ± 0.1%). More than half of CBM’s effect (58.4 ± 0.6%) is attributed to concepts labeled absent. Similar to the above studies, accurate concept prediction alone does not make the model’s outcomes depend on the active concepts. In CCW-Net, a zeroed concept vector carries a zero key and zero value in the attention head, so setting a concept to absent removes its contribution to the outcome (Prop. 1).
Robustness and ablation
We performed ablation studies on the control domains: aircraft control and CarRacing. Full results are in Table 8 and Table 9 (App. § 16).
Table 8: Aircraft control: full ablation matrix. tMSE is the test action MSE, and concept accuracy is on the Lead/Lag and Climb/Dive concept sets (%). With a placebo concept added, the placebo carries of the effect at width 256, and with and without the alignment losses at width 64.
| Model | tMSE | Concept acc. (%) Lead/Lag | Concept acc. (%) Climb/Dive | Concept Effect (%) All Active | Concept Effect (%) All Inactive | DCI Disentanglement | DCI Completeness | DCI Informativeness |
| Ours | ||||||||
| CCW-Net (Ours, 500K samples) | ||||||||
| CCW-Net (Ours, 5M samples) | ||||||||
| CCW-Net (Ours, scalar ) | ||||||||
| No causal alignment losses | ||||||||
| CCW-Net () | ||||||||
| CCW-Net (scalar ) | ||||||||
| Architecture baselines | ||||||||
| CW-Net architecture, scalar + alignment | ||||||||
| CW-Net (Kenny et al.) | ||||||||
| Disentanglement baselines | ||||||||
| -TCVAE () | ||||||||
| iVAE () | ||||||||
| Data quantity | ||||||||
| 5K samples | ||||||||
| 50K samples | ||||||||
| Label fraction | ||||||||
| 10% labels | ||||||||
| 25% labels | ||||||||
| 50% labels | ||||||||
| Label noise | ||||||||
| 10% noise | ||||||||
| 20% noise | ||||||||
| Loss combinations (without P) | ||||||||
| N | ||||||||
| S | ||||||||
| I | ||||||||
| N+S | ||||||||
| Model | tMSE | Lead/Lag | Climb/Dive | All Active ↑ | All Inactive ↓ | Disentanglement | Completeness | Informativeness |
| N+I | ||||||||
| S+I | ||||||||
| N+S+I | ||||||||
| Loss combinations (with P) | ||||||||
| P | ||||||||
| N+P | ||||||||
| S+P | ||||||||
| I+P | ||||||||
| N+S+P | ||||||||
| N+I+P | ||||||||
| Concept ontology, merged concepts | ||||||||
| Merge Climb+Dive | ||||||||
| Merge Lead+Lag | ||||||||
| Concept ontology, removed concepts | ||||||||
| Remove Climb | ||||||||
| Remove Dive | ||||||||
| Remove Lag | ||||||||
| Remove Lead | ||||||||
| Placebo | ||||||||
| Placebo concept added | ||||||||
| Encoder width 64 | ||||||||
| CCW-Net (Ours), | ||||||||
| CCW-Net (Ours), | ||||||||
| 10% label noise | ||||||||
| 10% label noise, no alignment | ||||||||
| Placebo concept added | ||||||||
| Placebo concept added, no alignment |
Table 8. continued
| Model | tMSE | Concept acc. (%) | All Active ↑ | All Inactive ↓ | Disentanglement | Completeness | Informativeness |
| Ours | |||||||
| CCW-Net (Ours, ) | 0.0025 ±0.0004 | 93.4 ±0.2 | 75.3 ±3.8 | 24.7 ±3.8 | 0.914 ±0.015 | 0.706 ±0.042 | 0.950 ±0.004 |
| CCW-Net (Ours, scalar ) | 0.0420 ±0.0006 | 93.0 ±0.3 | 92.4 ±2.7 | 7.6 ±2.7 | 0.866 ±0.021 | 0.866 ±0.021 | 0.947 ±0.005 |
| No causal alignment losses | |||||||
| CCW-Net () | 0.0032 ±0.0004 | 93.4 ±0.3 | 63.5 ±9.0 | 36.5 ±9.0 | 0.913 ±0.011 | 0.680 ±0.034 | 0.950 ±0.003 |
| CCW-Net (scalar ) | 0.0419 ±0.0006 | 93.0 ±0.4 | 90.0 ±5.5 | 10.0 ±5.5 | 0.870 ±0.012 | 0.871 ±0.012 | 0.948 ±0.003 |
| Architecture baselines | |||||||
| CW-Net architecture, scalar + alignment | 0.0276 ±0.0089 | 89.8 ±1.3 | 37.2 ±6.7 | 62.8 ±6.7 | 0.085 ±0.061 | 0.560 ±0.083 | 0.928 ±0.009 |
| CW-Net (Kenny et al.) | 0.0015 ±0.0007 | 90.0 ±1.3 | 33.1 ±1.6 | 66.9 ±1.6 | 0.078 ±0.063 | 0.626 ±0.082 | 0.928 ±0.008 |
| CW-Net architecture, + alignment | 0.0058 ±0.0015 | 92.3 ±0.5 | 52.2 ±2.9 | 47.8 ±2.9 | 0.763 ±0.024 | 0.710 ±0.021 | 0.938 ±0.005 |
| CW-Net architecture, | 0.0004 ±0.0001 | 92.4 ±0.4 | 43.3 ±2.4 | 56.7 ±2.4 | 0.772 ±0.031 | 0.678 ±0.034 | 0.938 ±0.003 |
| Disentanglement baselines | |||||||
| -TCVAE () | 0.0290 ±0.0024 | 31.0 ±7.6 | 37.9 ±8.1 | 62.1 ±8.1 | 0.026 ±0.019 | 0.472 ±0.132 | 0.890 ±0.014 |
| -TCVAE () | 0.0293 ±0.0006 | 32.2 ±1.8 | 51.7 ±8.8 | 48.3 ±8.8 | 0.047 ±0.012 | 0.813 ±0.025 | 0.894 ±0.005 |
| iVAE () | 0.0277 ±0.0009 | 32.4 ±13.0 | 39.0 ±7.0 | 61.0 ±7.0 | 0.022 ±0.018 | 0.354 ±0.079 | 0.897 ±0.009 |
| iVAE () | 0.0191 ±0.0013 | 45.1 ±11.0 | 37.3 ±4.8 | 62.7 ±4.8 | 0.250 ±0.066 | 0.689 ±0.019 | 0.911 ±0.006 |
| Label fraction | |||||||
| 10% labels | 0.0026 ±0.0003 | 90.2 ±0.5 | 77.9 ±4.9 | 22.1 ±4.9 | 0.909 ±0.011 | 0.675 ±0.030 | 0.930 ±0.007 |
| 25% labels | 0.0030 ±0.0005 | 91.2 ±0.4 | 75.6 ±7.3 | 24.4 ±7.3 | 0.910 ±0.012 | 0.684 ±0.046 | 0.936 ±0.007 |
| 50% labels | 0.0032 ±0.0003 | 92.0 ±0.4 | 77.1 ±6.9 | 22.9 ±6.9 | 0.926 ±0.023 | 0.681 ±0.050 | 0.941 ±0.003 |
| Label noise | |||||||
| 10% noise | 0.0024 ±0.0002 | 92.3 ±0.3 | 61.8 ±9.3 | 38.2 ±9.3 | 0.923 ±0.009 | 0.753 ±0.049 | 0.944 ±0.004 |
| 20% noise | 0.0023 ±0.0001 | 91.7 ±0.4 | 44.9 ±8.3 | 55.1 ±8.3 | 0.885 ±0.020 | 0.719 ±0.044 | 0.940 ±0.005 |
| Loss combinations (without P) | |||||||
| N | 0.0031 ±0.0004 | 93.2 ±0.2 | 77.2 ±6.1 | 22.8 ±6.1 | 0.923 ±0.012 | 0.685 ±0.034 | 0.948 ±0.004 |
| Model | tMSE | Concept acc. (%) | |||||
| Concept Effect (%) | DCI | ||||||
| All Active | All Inactive | Disentanglement | Completeness | Informativeness | |||
| S | |||||||
| I | |||||||
| N+S | |||||||
| N+I | |||||||
| S+I | |||||||
| N+S+I | |||||||
| Loss combinations (with P) | |||||||
| P | |||||||
| N+P | |||||||
| S+P | |||||||
| I+P | |||||||
| N+S+P | |||||||
| N+I+P | |||||||
| S+I+P | |||||||
| Concept ontology, merged concepts | |||||||
| Merge Left+Right § | |||||||
| Merge Straight+Left § | |||||||
| Merge Straight+Right § | |||||||
| Concept ontology, removed concepts | |||||||
| Remove Left | |||||||
| Remove Right | |||||||
| Remove Straight | |||||||
| Placebo | |||||||
| Placebo concept added | |||||||
| Encoder width 64 | |||||||
| CCW-Net (Ours), | |||||||
| CCW-Net (Ours), |
Table 9. CarRacing: full ablation matrix. tMSE is the test action MSE, and concept accuracy is on the Straight/Left/Right concept set (%). Every group except the last uses batch size 128 for the canonical, no-alignment, loss-combination, label-noise, and placebo rows and 32 for the other rows. The last group uses batch size 32. Every row with the presence loss uses except CCW-Net (Ours, scalar ) and the two CW-Net rows with alignment, which use . For the Remove rows, the averages cover the label combinations with an active concept, since the removed concept’s own combination has none. § In the Merge rows, the estimator scores only the merged concept’s label combination, because the remaining combination has no comparison that holds the merged concept fixed. With a placebo concept added, the placebo carries of the effect.
Label efficiency. Across both control domains, CCW-Net retains most of its causal faithfulness when concept supervision is increasingly sparse. For example, when only 10% of data is labeled with concepts, decreases by only 4.0 points on aircraft control (97.3% to 93.3%) and interestingly increases by 2.6 on CarRacing (75.3% to 77.9%). Both are within noise across seeds. We believe this is consistent with CCW-Net’s alignment losses and architecture providing strong enough inductive bias that dense labeling becomes less critical. These results are encouraging as they suggest that less labeling burden is required to retain comparable causal alignment results.
Label noise. We find that label noise has the opposite effect. 10% noise drops from 97.3% to 71.6% on aircraft control and from 75.3% to 61.8% on CarRacing while concept accuracy remains nearly unchanged (96.2% vs. 96.4% and 92.3% vs. 93.4%). This is an example where concept accuracy and architecture alone do not reflect causal faithfulness motivating the need for evaluation approaches such as ours to verify such claims.
Ontology misspecification. CCW-Net degrades gracefully when the ontology isn’t fully specified. Removing one concept from the aircraft ontology lowers to 79.9–94.6%, and merging two concepts leaves it at 95.6–97.4%. On CarRacing, removing one concept gives 65.8–82.1% and merging two gives 56.9–95.7%, against 75.3% for the full ontology.
Loss ablation. On aircraft control, removing the sufficiency or independence loss terms lowers from 97.3% to 94.4% and 94.8%, while removing the presence term does not (98.0%). On CarRacing, where one concept is active per input, necessity alone reaches 77.2%, comparable to 75.3% with all four losses, and presence alone reaches 71.9% against 63.5% without alignment. This suggests different domains benefit from the loss terms differently.
Vector versus scalar concepts. We find concept representation dimension allows the alignment losses flexibility to improve causal alignment. At , alignment increases by 8.1 points on aircraft control and 11.8 points on CarRacing, while at the change is −0.9 and 2.4 points.
Conclusion
Concept-based explanations are useful to the extent that they causally drive the model’s outputs. We presented CCW-Net, the first formulation of multi-mediator interventional mediation analysis as a training objective for deep networks. We also introduced an evaluation method that quantifies the fraction of a model’s output effect concept explanations carry. CCW-Net wraps a frozen backbone model with a concept encoder and attention head, trained to satisfy three causal alignment criteria (necessity, sufficiency, and independence) with counterfactual concept representations drawn from a mediator model. CCW-Net’s causal claims concern the effect of concept activations and representations on model outcomes where effects in the data-generation process are left to future work.
Across aircraft control, CarRacing-v1, and CUB-200 image classification, CCW-Net routes more of its output’s effect through active concepts than concept bottleneck based approaches. The contribution combines two approaches: concept representations feeding an attention head in which absent concepts contribute zero value, isolating each concept’s contribution to the output, and causal alignment losses that strengthen this effect and improve consistency across seeds. On our aircraft control domain, CCW-Net routes 94.5 ± 2.2% of the effect through active concepts against 50.0 ± 6.2% for prior art. On CUB-200, the architecture alone routes 91.3% against 41.6% for CBM at comparable concept accuracy, suggesting that concept accuracy is not an indicator of causal faithfulness. iVAE achieves excellent DCI disentanglement metrics on aircraft control, yet its active concepts carry only 47.6% of the effect, suggesting disentanglement and causal alignment are different properties.
CCW-Net moves causal faithfulness from a property that is claimed by inspection to one that can be designed for, trained against, and measured by. Concept-based explanations become not merely interpretable but quantifiably causal thereby building a road toward verifiably trustworthy models in safety- and task-critical environments.
Acknowledgments and Disclosure of Funding
Research was sponsored by the Department of the Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-19-2-1000. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Department of the Air Force or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. The authors acknowledge the MIT SuperCloud and Lincoln Laboratory Supercomputing Center for providing HPC resources that have contributed to the research results reported within this paper. The authors declare no competing interests.
Variable dictionary
| Symbol | Definition |
| Inputs and outputs | |
| Input | |
| Expert output | |
| Wrapper output, | |
| Concept label (1 = active) | |
| Architecture | |
| Frozen backbone | |
| Backbone latent | |
| Concept encoder | |
| Attention head | |
| Concept representation | |
| Concept ’s representation (slot) | |
| Number of concepts | |
| Representation dimension | |
| Mediator model (Sec. 4.2) | |
| Per-concept invertible flow | |
| Counterfactual draw, | |
| Flow negative log-likelihood | |
| Mediation analysis (Sec. 3) | |
| TE, DE, IE | Total, direct, indirect effect |
| Interventional indirect effect of mediator | |
| Residual mediator interaction | |
| Concept edits and changes (Sec. 4.3) | |
| All-zero concept representation | |
| with slot set to | |
| with slot set to | |
| Change from zero, | |
| Slot ’s change from zero, | |
| Slot ’s counterfactual change, | |
| Losses and weights | |
| Task loss | |
| Concept classification loss | |
| Necessity, sufficiency, independence losses | |
| Presence loss (auxiliary) | |
| Weighted sum of the alignment losses | |
| Total wrapper loss | |
| Loss weights | |
| Presence threshold, | |
| Evaluation (Sec. 5.4) | |
| Test-input sets of one comparison | |
| Matching features of input | |
| Distance between and | |
| Kernel bandwidth | |
| Slot group; group ordering | |
| Group ’s slots from an or match | |
| Group ’s effect | |
| Fraction of the effect slot carries | |
| Fraction active, inactive concepts carry | |
| Evaluation estimator (App. F) | |
| Concept label combination, its test count | |
| Comparison, comparisons that include label combination | |
| Swapped pair of comparison |
Table 4. Variable dictionary for CCW-Net.
| Symbol | Definition |
| One of the sets | |
| Match of test input | |
| Number of matching features | |
| Matching radius | |
| Group G’s effect in comparison | |
| Comparisons in with | |
| Concept slot ’s group in comparison | |
| Concept slot ’s effect, fraction for label combination | |
| Scored combinations with an active concept | |
| Test-count-weighted means over |
Table 10.
Extended related work
In Section 2 we focused on the most directly relevant prior art for CCW-Net’s three contributions (a mediation-analysis training objective, an architecture for concept-level mediation analysis, and an evaluation method for concept-based causality claims). Here we position CCW-Net among causal concept bottleneck models and causal abstraction methods, and discuss adjacent research operating at different levels (input, environment-state, policy, trajectory) or in adjacent approaches.
Causal concept bottleneck models and causal abstraction
Concept bottleneck models differ in how concepts reach and produce the model’s output. In flat bottlenecks, every concept is a direct parent of the output. Laguna et al. [ [34] make pretrained black boxes intervenable, fine-tuning them such that concept interventions improve their predictions. Ismail et al. [ [26] insert a concept bottleneck into generative models to steer generation toward human-interpretable concepts. Dominici et al. [ [14] extend concept bottleneck models to predict how concepts would need to change to produce an alternative prediction. These works use concept interventions to improve or steer predictions. CCW-Net instead trains and measures how much of a model’s output is attributed to each concept. Concept graph models [[11, 13]] let some concepts be parents of others rather than of the output, allowing for a deeper, more intricate concept graph. Driving and measuring the causal relationship of each node in such a graph is a natural extension of our approach.
In the causal abstraction literature, interchange intervention training [[18]] trains a network so that interventions on aligned high-level variables produce the counterfactual outputs of a manually-specified causal model. Causal proxy models [[79]] train a proxy of a text classifier on counterfactual examples so that concept interventions simulate the classifier’s counterfactual behavior. CCW-Net needs neither a hand-specified causal model nor counterfactual input pairs. Its losses follow interventional mediation analysis [[73]], which also requires no causal ordering among the mediators. For measurement of causal effects, the causal concept effect (CaCE) of Goyal et al. [ [20] estimates the change in expected output when a concept is toggled, using a learned generative model of counterfactual inputs and averaging over all samples. CCW-Net trains for the property and measures it separately for active and inactive concepts.
Causal explanation in sequential decision-making
Several lines of work explain sequential decision-making policy behavior using causal frameworks at the environment-state, trajectory, or action level. Madumal et al. [ [41] introduce action-influence models that define a structural causal model over environment variables and agent actions and use them to produce contrastive explanations of RL policies. Kekić et al. [ [27] learn nonlinear causal reductions that map a high-dimensional RL system to a lower-dimensional SCM whose interventional behavior matches the original system. Nashed et al. [ [43] build a unified SCM framework for explaining the actions of MDP agents, and Gyevnar et al. [ [21] extend causal explanations to multi-agent settings. Bica et al. [ [6] explain expert sequential decision-making policies by learning rewards over what-if outcomes. Schulte and Poupart [ [60] examine when reinforcement learning benefits from causal reasoning. Verma and Srivastava [ [75] learn causally accurate models for autonomous assessment of black-box agents. These methods operate at the environment or policy level and do not impose causal structure on a learned concept representation. CCW-Net is complementary: it uses concept labels to shape the model’s own concept-to-output effects, without requiring an explicit environment SCM.
Causal methods in imitation and reinforcement learning training
Causal frameworks have also been incorporated into imitation learning and reinforcement learning training, distinct from post-hoc explanation. Ruan et al. [ [53] introduce causal imitation learning via inverse reinforcement learning to recover policies that respect causal structure in expert demonstrations. Li et al. [ [36] address confounding-robust deep RL via a causal approach to off-policy learning. Herlau and Larsen [ [23] learn causal variables and a policy by optimizing a mediation-analysis objective in which the treatment is the choice of policy, the mediator is a binary environment event, and the outcome is the return. These works apply causal reasoning at the policy or imitation level. CCW-Net applies causal reasoning at the concept level inside the policy, training the concept encoder to satisfy interventional necessity, sufficiency, and independence criteria.
Human-AI interaction, trust, and verification
Several lines of work motivate interpretable models in safety-critical settings from human-factors and verification perspectives. Sanneman and Shah [ [56], Sanneman et al. [ [57] characterize the role of explanations in supporting human situational awareness and the understanding-workload tradeoff in human-centered XAI. Sreedharan et al. [ [66] provide a planning perspective on explainable human-AI interaction. Sobrín-Hidalgo et al. [ [65] survey explanation generation for autonomous robots. Lyons et al. [ [39], Highland et al. [ [24] report empirical studies of trust and agent appropriateness in live-flight human-autonomy teaming. Omeiza et al. [ [44] survey explanation methods specific to autonomous driving. Verma and Shah [ [74], Mahmud et al. [ [42] consider verification and interpretability analysis of decision-making systems. CCW-Net sits within this broader effort: it produces explanations grounded in human-meaningful concepts and aligned with causal effects, supporting the verification, audit, and trust workflows these works study.
Connection to interventional mediation analysis
In this section, we connect CCW-Net’s method and alignment losses (§ 4.3) to interventional mediation analysis [73, 37], and state the attention head property that makes a valid reference (App. § 10.§ 1).
Interventional mediation analysis. Mediation analysis asks how much of a treatment’s effect on an outcome passes through each of several intermediate variables (i.e., the mediators). Following the notation of Vansteelandt and Daniel [ [73], we write for the outcome and for the mediators. denotes the mediator values under treatment , and denotes the outcome under treatment with the mediators set to . Changing the treatment from to has the total effect
Vansteelandt and Daniel [ [73] measure the part of this effect that passes through a single mediator with the interventional indirect effect
sets only to its value under the new treatment and keeps every other mediator at its value under the original treatment. The direct effect is the part of the effect that bypasses the mediators. Vansteelandt and Daniel [ [73] and Loh et al. [ [37] add a residual term for the interactions among mediators. With it, the effects add up to the total without assuming any causal ordering or independence among the mediators,
Mapping to CCW-Net. In CCW-Net, the concept representations are the mediators and the head output is the outcome. As the concept representations serve as a bottleneck, the input reaches only through them with no direct pathway from input to output that bypasses the mediators, so DE (§ 4.1). Flipping concept ’s label plays the role of the treatment change, and the flow turns this flip into the counterfactual draw (Eq. (1)). In the notation above, the treatment value is concept ’s label, , and plays the role of with . With this mapping, two of the changes in § 4.3 correspond to .
is the per-sample analog of . Like , it replaces with its counterfactual value and keeps the other concept representations at their observed values.
is the analog of measured from zero. It switches on with every other concept representation at zero instead of at its observed value. Zero is a valid reference because is a constant (Prop. 1).
What each loss enforces.
Necessity. drives toward zero for concepts labeled inactive, so an inactive concept has no per-sample indirect effect on the output.
Sufficiency. anchors to the expert output . Since DE , the active concepts’ contributions measured from zero should reproduce the expert output.
Independence. drives toward zero. This residual plays the role of in Eq. (10) for changes measured from zero, so driving it to zero makes the concept contributions add up.
is not part of the decomposition. It keeps a few active concepts from carrying the whole effect. Without it, sufficiency could be met overall while the individual contributions say little.
Zero value contribution of a zeroed concept
Zero can only serve as a reference if a zeroed concept representation contributes nothing to the output. Here we show that the attention head has this property, and we describe what zeroing a concept does to the other concepts’ attention weights.
Setup. The wrapper output is a vector indexed by . The head is a single-head attention layer with one learned query per output. , , , and are the head’s query, key, value, and output projections, all of which are bias-free, and and are the weight and bias for output . We use , , and to index the concept representations, where is the one intervened on, is any other, and is a summation index. For output , the head computes logits , weights , context , and output .
Definition 1 (Zero value contribution). Concept ’s value contribution to output is . The head has zero value contribution if for every concept , every output , and every with .
Proposition 1. (i) The head has zero value contribution, and its output at is the constant . (ii) A concept with keeps softmax weight with , so zeroing a concept is not the same as deleting it. (iii) Any single-concept intervention rescales the other concepts’ weights by one common factor per output, for all , with
For the zero intervention , the denominator equals the of (ii), and the effect on the context is exactly .
Proof. (i) The projections are linear, so gives and hence for every , whatever the value of . At , every logit is 0, the weights are uniform, and , so . (ii) A zero key gives a zero logit rather than a zero weight, since . Deleting the concept would instead give the denominator . (iii) Each logit depends only on its own concept, so the intervention changes only . For , the ratio is the ratio of the two denominators, which does not depend on . The zeroed concept’s own term vanishes by (i), so , which gives the identity.
In words, zeroing a concept removes its own value term exactly. Its attention weight stays positive, and the other concepts’ weights all scale by the same factor .
Relation to ignorability. Prop. 1 complements the ignorability condition in App. § 11. Ignorability (Prop. 2) handles confounding between the mediators and the outcome, . Prop. 1 gives every intervention a zero reference that itself contributes no value to the output.
Causal assumptions
Identification assumptions for interventional effects
Both training and evaluation estimate interventional effects in the wrapper SCM
where , , and (Figure 2). Following Vansteelandt and Daniel [ [73] and Loh et al. [ [37], these effects are identified under the following standard conditions.

Figure 2. The CCW-Net wrapper SCM. Every path from to passes through the concept representation , and there is no direct edge. This gives mediator-outcome ignorability within the SCM (Prop. 2).
Ignorability. There is no unmeasured confounding of the mediator-outcome relation given , that is, in the wrapper SCM. Prop. 2 shows that this holds by construction.
Positivity. For every in the support of , the conditional distribution has support around the observed concept representations. This condition concerns the counterfactual draws that the mediator model supplies to the necessity loss during training. The zero reference used by the sufficiency and independence losses is instead defined by the architecture (Prop. 1), not a property that occurs in the data.
Overlap. In evaluation (§ 5.4), the mediator model is replaced with real data test inputs. For each test input , both sets and should contain inputs within a radius of , so that the matched concept representations come from comparable state configurations. When no input is within the radius, the estimator uses the nearest neighbor.
Consistency. For the realized concept representation , the observed equals under the head .
Under these conditions, the effects decompose as in Eq. (10), with collecting the interactions that cannot be attributed to any single concept [37].
Model-level ignorability in the wrapper SCM
The ignorability condition above holds by construction in the wrapper SCM. The following proposition shows this holds even when the wrapper has stochastic exogenous inputs such as in stochastic policies or with dropout.
Proposition 2 (Model-level mediator-outcome ignorability). Consider the wrapper SCM with structural equations
where are exogenous noise variables. Suppose:
, so the head’s noise is independent of everything upstream.
All randomness used for counterfactual sampling from the mediator model depends only on with .
Then for every concept representation value in the support of ,
Hence the mediator-outcome ignorability required to identify , , and holds within the wrapper SCM.
Proof. For a fixed , the potential outcome is a function of alone. is a function of , and is independent of by assumption 1. Hence is independent of , and in particular . A counterfactual draw is a function of , and by assumption 2, so the same argument applies with the draw in place of . □
Architecture details
Concept encoder
The concept encoder is a three-layer multi-layer perceptron with ReLU activations. It maps the backbone latent to concept representations,
The hidden width is 64 for the models in Table 1 and Table 2, and 256 for the CUB-200 models in Table 3 and the ablations in App. § 16 outside its last group. We use on aircraft control and on CarRacing and CUB-200, and the ablations in App. § 16 vary .
| Model | Concept acc. (%) | All Active ↑ | All Inactive ↓ | Disentanglement | Completeness | Informativeness |
| CCW-Net (Ours) | 99.6 ±0.0 | 91.3 ±0.3 | 8.7 ±0.3 | 0.895 ±0.007 | 0.936 ±0.007 | 0.993 ±0.001 |
| CBM | 97.7 ±0.1 | 41.6 ±0.6 | 58.4 ±0.6 | 0.732 ±0.011 | 0.789 ±0.011 | 0.985 ±0.001 |
Table 3. CUB-200: fraction of the effect the active concepts carry, per input (§ 5.4), averaged over the test split. CBM is the concept bottleneck model of Koh et al. [ [33]. No CUB-200 model is trained with the causal alignment losses. Concept accuracy is over the 112 attributes. Means over ten seeds with two-sided 95% Student-t intervals.
Each concept representation then passes through the capsule-style squash nonlinearity [55],
applied to each of the concepts separately. The squash bounds each representation’s length to , and the concept classification loss trains this length to cluster near 0 and 1 (avoiding the mid ranges) as the concept’s presence (§ 4.1).
Attention head
The head is a single-head attention layer with one learned query per output . The concept representations serve as both keys and values. The query, key, value, and output projections are all bias-free. For output , the head computes the context
and the output
where and are the weight and bias for output . Since the projections are bias-free, a zero concept representation produces a zero key and a zero value. It therefore contributes no value to , even though its attention weight stays positive (Prop. 1). On CUB-200, the tanh outputs of CCW-Net and the linear outputs of CBM are trained with the mean-squared error to the one-hot species label.
Per-concept invertible flow
Our chosen mediator model is a conditional normalizing flow with one independent flow per concept. Any modeling approach may be used. Each flow is conditioned on the labels of concept ’s mutually exclusive set, the input, and a learned vector identifying concept . For , each flow is a stack of 8 affine coupling layers whose conditioner networks have hidden width 256. For , a coupling layer cannot split the representation, so each flow is instead a stack of 8 conditional affine maps with hidden width 256. To draw a counterfactual for concept , we flip the labels of its set and sample from the flow conditioned on them and the input (Eq. (1)). The flow has its own optimizer (App. § 14) and is trained on the negative log-likelihood of the encoder’s concept representations.
Evaluation estimator
In short, we compare similar test inputs that differ in one concept and measure how much of the change in the model’s output passes through each concept’s representation (i.e., the fraction of change). The causal attribution estimator (§ 5.4) computes each reported fraction as follows. Let index concept label combinations, be the number of test inputs with concept label combination , and be the comparisons in which forms one of the two sets and (test inputs with original and swapped concept labels). Each comparison swaps one active concept for a mutually exclusive alternative, so exactly two concept slots change labels. These two concept slots form one group, the swapped pair , and every other concept slot forms its own group.
Matching. We z-score the matching features (§ 5.4) over the test split. For a test input and a set , a match is drawn with probability
with bandwidth and radius . When no input of is within the radius, we take the nearest one. Each input receives 96 independently drawn matches for each group and set.
Comparisons. We form a comparison only when and . For each concept label combination , we fix one random subset of test inputs, and these inputs serve as in every comparison in . Matches come from all test inputs of each set.
Group effects. is Eq. (6) for comparison , averaged over the inputs, the drawn matches, and all orderings of the groups. We use the Euclidean norm of this averaged change in the model’s output.
Concept slot effects. For concept label combination , let be the comparisons that keep concept slot outside the swapped pair, and let be concept slot ’s group in comparison . Concept slot ’s effect is
In the second case, concept slot is active and lies in the swapped pair of every comparison in . In any other case, concept label combination is not scored. The fractions , , and then follow from Eq. (7).
Overall values. All Active and All Inactive weight the scored concept label combinations that have an active concept by their test counts,
and we report their means over ten seeds with two-sided 95% Student- intervals.
Effect decomposition. The total effect, measured on the real matched inputs, equals the sum of the group effects plus a residual (Eq. (10)), and this identity holds exactly. Here is the difference between real inputs and hybrids whose groups come from independent matches.
Training details
Algorithm 1 outlines CCW-Net training. The wrapper and the flow have separate optimizers. Training is conducted in three phases. During warmup, the wrapper learns the task and concept classification. During the intermission, the mediator model is fit to the encoder’s concept representations. During alignment, the wrapper trains on the full objective (§ 4.4), while the mediator model supplies the counterfactual draws for the necessity loss. The mediator model is refit every epochs during alignment so that it remains current with the encoder. Since the CUB-200 models train without the alignment losses they remain in the warmup phase throughout.
Algorithm 1 CCW-Net training. denotes the flow’s conditional density.
Require: Dataset , frozen backbone , warmup epochs , total epochs , refit interval , draws per concept
Initialize encoder and head
for epoch do ▷ Warmup
for batch do
Update on
end for
end for
Fit on with held fixed ▷ Intermission
for epoch do ▷ Alignment
if epochs have passed since the last fit then
Refit on the current encoder’s concept representations
end if
for batch do
Draw counterfactuals for every concept ▷ no gradient to
Compute , , , and (§ 4.3)
Update on
end for
end for
Loss weights
Loss weights for each domain are shown in Table 5. The task loss has weight 1 everywhere, and the four alignment losses share a common weight. The presence threshold is set as for equal share across each active concept. This gives on aircraft control, where two concepts are active per input, and on CarRacing, where one concept is active for any given input.
| Domain | ||||||
| Aircraft control | 1.0 | 0.1 | 0.1 | 0.1 | 0.1 | 0.5 |
| CarRacing | 1.0 | 0.1 | 0.1 | 0.1 | 0.1 | 1.0 |
| CUB-200, CCW-Net | 1.0 | — | — | — | — | — |
| CUB-200, CBM | 0.01 | — | — | — | — | — |
Table 5. Loss weights per domain. The CUB-200 models train without causal alignment losses.
Optimizers and learning rates
CCW-Net and the mediator model each use Adam with no weight decay, and the mediator model uses the wrapper’s batch size (Table 6).
Table 6. Optimizer settings per domain. The CUB-200 models use no mediator model as they had no causal alignment losses active. App. § 16 lists the batch size of each CarRacing ablation.
Training schedule
| Domain | Warmup epochs | Total epochs | Flow fit epochs | Refit interval |
| Aircraft control | 10 | 100 | 50 | 10 |
| CarRacing | 10 | 50 | 50 | 10 |
| CUB-200 | 100 | 100 | — | — |
Table 7. Training schedule per domain. The mediator model is fit for 50 epochs at the start of alignment and refit every 10 alignment epochs to refresh with the current concept representations. Each alignment step uses counterfactual draws per concept. The CUB-200 models train the task and concept losses for all 100 epochs.
Data quantities
Aircraft control: 500K samples for the main models. The data-quantity ablations use sets of 5K, 50K, and 5M samples.
CarRacing: 15,102 samples from 20 episodes.
CUB-200: 11,788 images, split into 72% train, 13% validation, and 15% test (8,516, 1,503, and 1,769 images).
Task details
Aircraft control
In extended trail formation [70, 71], a chase aircraft maintains a volume of space behind a lead aircraft that flies dynamic maneuvers. The simulator contains a high-fidelity aircraft dynamics model with six degrees of freedom [22]. For this experiment, the lead aircraft follows a scripted trajectory, and the chase aircraft is controlled by a reinforcement learning policy trained with proximal policy optimization [59] using the aircraft environment and codebase of So and Fan [ [64].
Backbone and action space. The frozen backbone policy maps the state-based observation to a two-dimensional action of pitch and roll commands. Concepts. binary concepts drawn from operational pilot vocabulary [70, 71] providing real-world external validity to the concept set. No real-world concepts were dropped from the operational ontology for this task:
Lead pursuit: the chase aircraft’s nose points ahead of the lead aircraft.
Lag pursuit: the chase aircraft’s nose points behind the lead aircraft.
Climb: the aircraft is ascending in altitude.
Dive: the aircraft is descending in altitude.
The pursuit concept set (Lead, Lag) and the vertical concept set (Climb, Dive) are each mutually exclusive. Exactly one concept per set and two concepts overall are active per sample.
CarRacing
CarRacing-v1 [7] is a top-down driving task with pixel inputs. The agent receives a RGB image of the track and outputs continuous steering, throttle, and brake commands. We use the recorded latents, actions, and concept labels released with CW-Net [29].
Backbone and action space. The frozen backbone is the convolutional policy from that release, trained end-to-end on CarRacing-v1. The action space is three-dimensional, with steering in , throttle in , and brake in .
Concepts. binary concepts, which the CW-Net authors labeled manually on over 15,000 examples [29]:
Straight: drive straight.
Left: left turn.
Right: right turn.
The three concepts are mutually exclusive and exhaustive, so exactly one concept is active per sample. The labels do not follow the agent’s instantaneous steering command.
CUB-200 classification
CUB-200-2011 [77] is an image classification dataset of 200 bird species. We follow the concept bottleneck setup of Koh et al. [ [33], in which 112 binary visual attributes, assigned per species, exist between the image and the species label.
Backbone and output space. The frozen backbone is Inception-v3, fine-tuned as the joint concept bottleneck model of Koh et al. [ [33] on a species-stratified 80% of all 11,788 CUB-200 images. This fine-tuning set includes about 80% of our test images. The wrapper output is a 200-way species prediction.
Concepts. binary visual attributes (e.g., has yellow belly, has pointed beak, has streaked wing). Each species has its own subset of attributes, so the number of active concepts varies by image, with a mean of about 7 over the test split.
Splits. We split the images into 72% train, 13% validation, and 15% test (8,516, 1,503, and 1,769 images), with a different split for each seed.
Full ablation matrix
Tables 8 and 9 report every ablation on aircraft control and CarRacing. Outside the last group, every CCW-Net model uses an encoder of width 256 (App. § 12) and compares against the canonical model in the first row, and the baselines use their own architectures. The last group uses an encoder of width 64, the width of the models in Table 1 and Table 2. All Active and All Inactive average and over the label combinations, weighted by each combination’s number of test inputs (App. § 13). Loss combinations name the alignment losses used, N (necessity), S (sufficiency), I (independence), and P (presence), and the canonical model uses all four. All values are means over ten seeds with two-sided 95% Student- intervals, computed from the trained weights. App. § 13 gives the estimator’s settings.
Aircraft control
CarRacing
References
- [1]Reduan Achtibat, Maximilian Dreyer, Ilona Eisenbraun, Sebastian Bosse, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin. From attribution maps to human-understandable explanations through Concept Relevance Propagation. Nature Machine Intelligence, 5(9):1006–1019, 2023.DOI
- [2]Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Proceedings of the 32nd Conference on Advances in Neural Information Processing Systems (NeurIPS), 2018.arxiv.org/abs/1810.03292
- [3]David Alvarez-Melis and Tommi S. Jaakkola. Towards robust interpretability with self-explaining neural networks. In Proceedings of the 32nd Conference on Advances in Neural Information Processing Systems (NeurIPS), 2018.
- [4]Shahin Atakishiyev, Mohammad Salameh, and Randy Goebel. Safety implications of explainable artificial intelligence in end-to-end autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 26(10):14516–14535, 2025.arxiv.org/abs/2403.12176
- [5]Andrew Bai, Chih-Kuan Yeh, Neil Y.C. Lin, Pradeep Kumar Ravikumar, and Cho-Jui Hsieh. Concept gradient: Concept-based interpretation without linear assumption. In Proceedings of the 11th International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2208.14966
- [6]Ioana Bica, Daniel Jarrett, Alihan Hüyük, and Mihaela van der Schaar. Learning “what-if” explanations for sequential decision-making. In Proceedings of the 9th International Conference on Learning Representations (ICLR), 2021.arxiv.org/abs/2007.13531
- [7]Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. OpenAI Gym. arXiv preprint arXiv:1606.01540, 2016.arxiv.org/abs/1606.01540
- [8]Viviana Celli. Causal mediation analysis in economics: Objectives, assumptions, models. Journal of Economic Surveys, 36(1):214–234, 2022.DOI
- [9]Ricky T. Q. Chen, Xuechen Li, Roger Grosse, and David Duvenaud. Isolating sources of disentanglement in variational autoencoders. In Proceedings of the 32nd Conference on Advances in Neural Information Processing Systems (NeurIPS), 2018.
- [10]Devleena Das, Sonia Chernova, and Been Kim. State2Explanation: Concept-based explanations to benefit agent learning and user understanding. In Proceedings of the 37th Conference on Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2309.12482
- [11]Giovanni De Felice, Arianna Casanova Flores, Francesco De Santis, Silvia Santini, Johannes Schneider, Pietro Barbiero, and Alberto Termine. Causally reliable concept bottleneck models. Advances in Neural Information Processing Systems, 38:149099–149139, 2025.arxiv.org/abs/2503.04363
- [12]Quentin Delfosse, Sebastian Sztybertnia, Mark Rothermel, Wolfgang Stammer, and Kristian Kersting. Interpretable concept bottlenecks to align reinforcement learning agents. In Proceedings of the 38th Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024.
- [13]Gabriele Dominici, Pietro Barbiero, Mateo Espinosa Zarlenga, Alberto Termine, Martin Gjoreski, Giuseppe Marra, and Marc Langheinrich. Causal concept graph models: Beyond causal opacity in deep learning. In International Conference on Learning Representations, volume 2025, pages 55102–55131, 2025a.arxiv.org/abs/2405.16507
- [14]Gabriele Dominici, Pietro Barbiero, Francesco Giannini, Martin Gjoreski, Giuseppe Marra, and Marc Langheinrich. Counterfactual concept bottleneck models. In Proceedings of the 13th International Conference on Learning Representations (ICLR), 2025b.arxiv.org/abs/2402.01408
- [15]Cian Eastwood and Christopher K. I. Williams. A framework for the quantitative evaluation of disentangled representations. In Proceedings of the 6th International Conference on Learning Representations (ICLR), 2018.
- [17]Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. Advances in Neural Information Processing Systems, 34:9574–9586, 2021.arxiv.org/abs/2106.02997
- [18]Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah Goodman, and Christopher Potts. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning, pages 7324–7338. PMLR, 2022.arxiv.org/abs/2112.00826
- [19]Atticus Geiger, Duligur Ibeling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and Thomas Icard. Causal abstraction: A theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research, 26(83):1–64, 2025.
- [20]Yash Goyal, Amir Feder, Uri Shalit, and Been Kim. Explaining classifiers with causal concept effect (CaCE). arXiv preprint arXiv:1907.07165, 2019.arxiv.org/abs/1907.07165
- [21]Balint Gyevar, Cheng Wang, Christopher G. Lucas, Shay B. Cohen, and Stefano V. Albrecht. Causal explanations for sequential decision-making in multi-agent systems. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2024.
- [22]Peter Heidlauf, Alexander Collins, Michael Bolender, and Stanley Bak. Verification challenges in F-16 ground collision avoidance and other automated maneuvers. In Proceedings of the 5th International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH), 2018.DOI
- [23]Tue Herlau and Rasmus Larsen. Reinforcement learning of causal variables using mediation analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6910–6917, 2022.
- [24]Patrick Highland, Thomas Schnell, Katharine Woodruff, and Gianna Avdic-McIntire. Towards human objective real-time trust of autonomy measures for combat aviation. The International Journal of Aerospace Psychology, 33(1):1–34, 2023.DOI
- [25]Kosuke Imai, Luke Keele, and Dustin Tingley. A general approach to causal mediation analysis. Psychological Methods, 15(4):309–334, 2010.DOI
- [26]Aya Abdelsalam Ismail, Julius Adebayo, Héctor Corrada Bravo, Stephen Ra, and Kyunghyun Cho. Concept bottleneck generative models. In International Conference on Learning Representations, volume 2024, pages 34114–34132, 2024.
- [27]Armin Kekić, Jan Schneider, Dieter Büchler, Bernhard Schölkopf, and Michel Besserve. Learning nonlinear causal reductions to explain reinforcement learning policies. In Proceedings of the 14th International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2507.14901
- [28]Eoin M Kenny, Mycal Tucker, and Julie Shah. Towards interpretable deep reinforcement learning with human-friendly prototypes. In Proceedings of the 11th International Conference on Learning Representations (ICLR), 2023.
- [29]Eoin M Kenny, Akshay Dharmavaram, Sang Uk Lee, Tung Phan-Minh, Shreyas Rajesh, Yunqing Hu, Laura Major, Momchil S Tomov, and Julie A Shah. Explainable deep learning improves human mental models of self-driving cars. Nature, 657(8130):114–120, 2026.
- [30]Ilyes Khemakhem, Diederik P. Kingma, Ricardo P. Monti, and Aapo Hyvärinen. Variational autoencoders and nonlinear ICA: A unifying framework. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.arxiv.org/abs/1907.04809
- [31]Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viégas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018.arxiv.org/abs/1711.11279
- [32]Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018.arxiv.org/abs/1802.05983
- [33]Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Proceedings of the 37th International Conference on Machine Learning (ICML), 2020.arxiv.org/abs/2007.04612
- [34]Sonia Laguna, Riĉards Marcinkevičs, Moritz Vandenhirtz, and Julia E. Vogt. Beyond concept bottleneck models: How to make black boxes interpretable? Advances in Neural Information Processing Systems, 37:85006–85044, 2024.arxiv.org/abs/2401.13544
- [35]Hopin Lee, Aidan G Cashin, Sarah E Lamb, Sally Hopewell, Stijn Vansteelandt, Tyler J VanderWeele, David P MacKinnon, Gemma Mansell, Gary S Collins, Robert M Golub, et al. A guideline for reporting mediation analyses in randomized trials and observational studies: the AGReMA statement. Journal of the American Medical Association, 326(11):1045–1056, 2021.
- [36]Mingxuan Li, Junzhe Zhang, and Elias Bareinboim. Confounding robust deep reinforcement learning: A causal approach. In Proceedings of the 39th Conference on Advances in Neural Information Processing Systems (NeurIPS), 2025.arxiv.org/abs/2510.21110
- [37]Wen Wei Loh, Beatrijs Moerkerke, Tom Loeys, and Stijn Vansteelandt. Disentangling indirect effects through multiple mediators without assuming any causal structure among the mediators. Psychological Methods, 27(6):982–999, 2022.DOI
- [38]Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Proceedings of the 31st Conference on Advances in Neural Information Processing Systems (NeurIPS), 2017.arxiv.org/abs/1705.07874
- [39]Joseph Lyons, Patrick Highland, Nathan Bos, David Lyons, Anna Skinner, Thomas Schnell, and Ryan Hefron. Measuring perceived agent appropriateness in a live-flight human-autonomy teaming scenario. Ergonomics in Design, 32(4):12–17, 2024.
- [40]David P MacKinnon. Introduction to Statistical Mediation Analysis. Lawrence Erlbaum Associates, 2008. ISBN 9780805839746.
- [41]Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. Explainable reinforcement learning through a causal lens. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI), 2020.arxiv.org/abs/1905.10958
- [42]Saaduddin Mahmud, Sandhya Saisubramanian, and Shlomo Zilberstein. Verification and validation of AI systems using explanations. In Proceedings of the AAAI Symposium on AI Trustworthiness and Risk Assessment for Challenging Contexts (ATRACC), 2024.DOI
- [43]Samer B Nashed, Saaduddin Mahmud, Claudia V Goldman, and Shlomo Zilberstein. Causal explanations for sequential decision making. Journal of Artificial Intelligence Research, 83(17), 2025.
- [44]Daniel Omeiza, Helena Webb, Marina Jirotka, and Lars Kunze. Explanations in autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23(8):10142–10162, 2022.DOI
- [45]Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, USA, 2nd edition, 2009. ISBN 052189560X.
- [46]Judea Pearl. The causal mediation formula – A guide to the assessment of pathways and mechanisms. Prevention Science, 13(4):426–436, 2012.
- [47]Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of Causal Inference: Foundations and Learning Algorithms. The MIT Press, 2017. ISBN 0262037319.
- [48]Tung Phan-Minh, Forbes Howington, Ting-Sheng Chu, Momchil S Tomov, Robert E Beaudoin, Sang Uk Lee, Nanxiang Li, Caglayan Dicle, Samuel Findler, Francisco Suarez-Ruiz, Bo Yang, Sammy Omari, and Eric M Wolff. DriveIRL: Drive in real life with inverse reinforcement learning. In Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023.
- [49]Adrian P Pope, Jaime S Ide, Daria Mićović, Henry Diaz, Jason C Twedt, Kevin Alcedo, Thayne T Walker, David Rosenbluth, Lee Ritholtz, and Daniel Javorskek. Hierarchical reinforcement learning for air combat at DARPA’s AlphaDogfight trials. IEEE Transactions on Artificial Intelligence, 4(6):1371–1385, 2023.DOI
- [50]Yilue Qian, Peiyu Yu, Ying Nian Wu, Yao Su, Wei Wang, and Lifeng Fan. Learning concept-based causal transition and symbolic reasoning for visual planning. In Proceedings of the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.arxiv.org/abs/2310.03325
- [51]Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2016.arxiv.org/abs/1602.04938
- [52]Joshua Rountree, Patrick Hipelius, Brian Dienst, Jonathan Aronoff, Ryan Neely, Robert Steigerwald, Skylar Griffis, David de Schweinitz, Chiawei Lee, and Ryan Hefron. Testing artificial intelligence in high-performance, tactical aircraft. In Proceedings of the 2021 IEEE Aerospace Conference (50100), 2021.DOI
- [53]Kangrui Ruan, Junzhe Zhang, Xuan Di, and Elias Bareinboim. Causal imitation learning via inverse reinforcement learning. In Proceedings of the 11th International Conference on Learning Representations (ICLR), 2023.
- [54]Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206–215, May 2019.DOI
- [55]Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. In Proceedings of the 31st Conference on Advances in Neural Information Processing Systems (NeurIPS), 2017.arxiv.org/abs/1710.09829
- [56]Lindsay Sanneman and Julie A Shah. The situation awareness framework for explainable AI (SAFE-AI) and human factors considerations for XAI systems. International Journal of Human–Computer Interaction, 38 (18-20):1772–1788, 2022.DOI
- [57]Lindsay Sanneman, Mycal Tucker, and Julie A Shah. An information bottleneck characterization of the understanding-workload tradeoff in human-centered explainable AI. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2024.DOI
- [58]Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.DOI
- [59]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.arxiv.org/abs/1707.06347
- [60]Oliver Schulte and Pascal Poupart. When should reinforcement learning use causal reasoning? Transactions on Machine Learning Research, 2025. ISSN 2835-8856.
- [61]Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), 2017.
- [62]Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In ICLR Workshop, 2014.arxiv.org/abs/1312.6034
- [63]Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. SmoothGrad: Removing noise by adding noise. In ICML 2017 Workshop on Visualization for Deep Learning, 2017.arxiv.org/abs/1706.03825
- [64]Oswin So and Chuchu Fan. Solving stabilize-avoid optimal control via epigraph form and deep reinforcement learning. In Proceedings of Robotics: Science and Systems (R:SS), 2023.arxiv.org/abs/2305.14154
- [65]David Sobrín-Hidalgo, Ángel Manuel Guerrero-Higueras, and Vicente Matellán-Olivera. Generating explanations for autonomous robots: a systematic review. IEEE Access, 13:20413–20426, 2025.
- [66]Sarath Sreedharan, Anagha Kulkarni, and Subbarao Kambhampati. Explainable Human-AI Interaction: A Planning Perspective. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, 2022. ISBN 9781636392905.arxiv.org/abs/2405.15804
- [67]Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017.arxiv.org/abs/1703.01365
- [68]Andong Tan, Fengtao Zhou, and Hao Chen. Explain via any concept: Concept bottleneck model with open vocabulary concepts. In Proceedings of the 18th European Conference on Computer Vision (ECCV), 2024.arxiv.org/abs/2408.02265
- [69]Chen Tang, Ben Abbatematteo, Jiaheng Hu, Rohan Chandra, Roberto Martín-Martín, and Peter Stone. Deep reinforcement learning for robotics: A survey of real-world successes. In Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAAI), 2025.
- [70]United States Air Force. Flying operations: T-38C flying fundamentals. Manual AETCMAN 11-251, Department of Defense, San Antonio, TX, September 2024. URL https://static.e-publishing.af.mil/production/1/aetc/publication/aetcman11-251/aetcman11-251.pdf. 10 September 2024.
- [71]United States Air Force. Flying operations: T-6 primary flying. Manual AETCMAN 11-248, Department of Defense, San Antonio, TX, August 2025. URL https://static.e-publishing.af.mil/production/1/aetc/publication/aetcman11-248/aetcman11-248.pdf. 13 August 2025.
- [72]Tyler J VanderWeele. Explanation in Causal Inference: Methods for Mediation and Interaction. Oxford University Press, 2015. ISBN 9780199325870.
- [73]Stijn Vansteelandt and Rhian M Daniel. Interventional effects for mediation analysis with multiple mediators. Epidemiology, 28(2):258–265, 2017.DOI
- [74]Pulkit Verma and Julie A Shah. Interpretability analysis of symbolic representations for sequential decision-making systems. In HRI 2025 Workshop on Explainability for Human-Robot Collaboration: Real-World Concerns (X-HRI), 2025.
- [75]Pulkit Verma and Siddharth Srivastava. Learning causally accurate models for autonomous assessment of deterministic black-box agents. Technical Report TR-ASUSCAI-2024-001, Arizona State University, 2024.
- [76]Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. In Proceedings of the 34th Conference on Advances in Neural Information Processing Systems (NeurIPS), 2020.
- [77]Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011.
- [78]Tom Ward. The US Air Force is moving fast on AI-piloted fighter jets. WIRED, March 2023.
- [79]Zhengxuan Wu, Karel D’Oosterlinck, Atticus Geiger, Amir Zur, and Christopher Potts. Causal proxy models for concept-based model explanations. In International Conference on Machine Learning, pages 37313–37344. PMLR, 2023.arxiv.org/abs/2209.14279
- [80]Yilun Zhou, Serena Booth, Marco Tulio Ribeiro, and Julie Shah. Do feature attribution methods correctly attribute features? In Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI), 2022.