WorksComputer Science
PhysSAE: Mechanistic Interpretability of PINNs with Sparse Autoencoders
Abstract
Physics-Informed Neural Networks (PINNs) embed PDE residuals into neural network training, but their internal representations remain opaque: it is unknown what physical features their hidden layers encode or whether those features have a localized causal role. We present PhysSAE, a mechanistic interpretability framework that trains overcomplete sparse autoencoders (SAEs) on PINN penultimate-layer activations and evaluates dictionary atoms through direct causal intervention in the original frozen hidden state: , bypassing the SAE decoder entirely. Across six PDE families, with 3 PINN seeds and 3 SAE seeds each—we show that (i) Our discovered SAE atoms align with independently-defined physical observables (max Pearson , always permutation null), (ii) the causal footprint of top-aligned atom ablation is 1.2–4.2 more spatially concentrated canonical than PCA or ICA interventions, and (iii) top-aligned atoms outperform matched random controls on causal localization for structured physical concepts (ESF advantage 0.04-0.44). Two-atom bilateral representations improve concept regression R by over single atoms, while random pairs decrease it by up to 0.60. These results demonstrate that PINNs develop sparse, physically structured latent representations that can be identified and causally interrogated post-hoc, opening a path toward interpretability-aware scientific machine learning.
Introduction
Physics-Informed Neural Networks ([14]) encode the residual of a PDE directly into the training loss via automatic differentiation, enabling mesh-free solution of forward and inverse problems. Despite widespread adoption in fluid mechanics ([3]), solid mechanics ([7]), and biomedical modelling ([15]), PINNs remain black boxes at the feature level. Their failure modes—spectral bias ([21]), gradient pathologies ([19]), causality violations ([22])—are well-characterised at the loss or gradient level, but no work has asked what a trained PINN actually represents in its hidden layers.
Recent mechanistic interpretability work has developed tools for extracting human-readable features from activations: linear probes ([1]), concept activation vectors ([10]), and overcomplete sparse autoencoders (SAEs) ([2]; [4]; [16]). SAEs have now migrated to scientific ML, appearing on graph CFD surrogates ([8]) and neural operators ([17]), and concept steering has been applied to physics foundation models ([6]). A direct inspection of PINN weight distributions by Tucny et al. [18] found that weights “resemble Gaussian random matrices with no evident trace of the physical principles,” motivating feature-level (rather than weight-level) interpretability.
PHYSSAE bridges mechanistic interpretability and scientific ML for residual-loss PINNs. Its core design choices address three challenges unique to the PINN setting: (1) concepts must be defined from independent reference solutions to avoid circularity; (2) causal intervention must act on the original PINN hidden state rather than an SAE reconstruction; (3) the spatio-temporal domain is continuous rather than discrete, requiring the ESF and support-capture metrics that generalise token-level interpretability to function spaces.
Related work. PINN architectures and failure modes. Self-adaptive weights [13] and Fourier-feature embeddings [20] address spectral bias; domain decomposition [9] enables parallel training; causal reweighting [22] fixes temporal ordering; KAN-based PINNs [23] offer symbolic readability via spline activations. None of these study internal representations at the feature level. XAI in scientific ML. SHAP/LIME/Integrated Gradients have been applied to PINNs as black-box regressors [12], not engaging the network’s spatiotemporal structure. SAEs on graph CFD surrogates [8] and neural operators [17], and concept steering on a physics foundation model [6], are closest to our work but applied to data-driven models, not residual-loss PINNs. Sparse autoencoders. The SAE design (ReLU codes, L1 sparsity, unit-norm decoder columns, pre-encoder bias) for monosemanticity in LLMs is established [2, 4, 16, 5]. We adapt this to the continuous-domain PDE setting.
Contributions.
First SAE for PINN latent space with an independent-concept evaluation protocol that avoids circular validation.
Direct hidden-state causal intervention: , applied to the frozen PINN and compared against the same intervention applied to PCA and ICA components for a controlled comparison.
Comprehensive statistical validation across 6 PDEs 3 PINN seeds 3 SAE seeds: negative controls, bilateral analysis, overcompleteness and sparsity ablations, and feature consistency metrics.
Representational collapse analysis showing that failed PINNs have higher-rank, more diffuse representations while still encoding recognisable physical invariants.
Paper overview. Section 2 details the PHYSSAE methodology. Section 3 describes the PDE benchmark and implementation. Section 4 presents all experimental results. Section 5 discusses implications and limitations. Section 6 concludes.
Method
PHYSSAE is a post-hoc mechanistic interpretability framework that operates entirely on a trained, frozen PINN. No retraining or architectural modification is required. The pipeline has three stages: (1) extract the PINN’s penultimate-layer activations on a dense spatio-temporal grid; (2) train a sparse autoencoder on those activations to discover a dictionary of physical atoms; (3) evaluate atoms by directly intervening in the original PINN hidden state and measuring the resulting change in the PINN output field. We describe each stage in detail below.
Problem Setting and Notation
Let be a spatial domain and a time interval. We consider scalar PDEs of the form
subject to an initial condition and boundary condition on . We write for the PINN approximation, where for real-valued PDEs and for the complex-valued Schrödinger equation (with real and imaginary parts).
The PINN is a fully-connected multilayer perceptron (MLP) with hidden layers of width , smooth activations, and a linear readout:
The key quantity for interpretability is the penultimate post-activation , which we refer to throughout as the hidden state or latent representation. The output is a linear function of , so : any change to propagates linearly to the output, making the readout a natural causal bottleneck.
Why the penultimate layer? The penultimate layer is the richest representation before the information is projected to a scalar by the linear readout. Interpretability at this layer is therefore interpretability of the last nonlinear computation the network performs before producing its prediction. Earlier layers encode more distributed, less semantically organised information; the output layer is only a linear combination and carries no additional structure.
PINN Training
Architecture. We use hidden layers of width with tanh activations and Xavier normal initialisation. The architecture is intentionally moderate in size: large enough that feature superposition is plausible (i.e., that atoms can be meaningfully encoded), yet small enough that 18 PINNs (6 PDEs 3 seeds) can be trained in under two hours on a single GPU. The output head is linear: , with .
Training objective. The PINN minimises a composite loss
where each term is an empirical mean over sampled points:
The residual is evaluated via automatic differentiation through the network; all PDE derivatives are computed exactly in this way.
We use weights , , which down-weights the physics residual relative to the data constraints. This is a standard choice that ensures the IC and BC are honoured even when the residual is stiff. Per training step: residual collocation points, IC points, BC points, all sampled uniformly at random.
Optimisation schedule. Training proceeds in two stages. Stage 1 uses Adam with learning rate for 8000 iterations. Stage 2 applies L-BFGS with strong Wolfe line search for up to 300 function evaluations. L-BFGS is a second-order method that exploits curvature information to sharpen the solution near the Adam minimum; it is particularly effective at reducing the residual on the collocation points after Adam has found a good basin. Gradient clipping (max norm 1.0) is applied during Adam to prevent instabilities, especially for the stiff Allen–Cahn and high-frequency convection PDEs.
Boundary conditions. For Dirichlet conditions , we sample points on and add the squared residual to . For periodic conditions , we enforce both the value and the derivative periodicity, evaluating the latter via automatic differentiation:
Activation Extraction
After training, the PINN is frozen and evaluated on a dense uniform grid with , (20,000 points). At each grid point we record the penultimate post-activation:
is the activation matrix: the raw material for interpretability. Each row is a -dimensional vector encoding the network’s internal state at the space-time point .
Normalisation. Before training the SAE, we z-score the activations across the grid:
with for numerical stability. This is essential for two reasons: (a) it places all hidden units on the same scale so the L1 sparsity penalty treats them symmetrically, and (b) the unit-norm constraint on SAE decoder columns is only meaningful when the input is approximately standardised. We store and use them during the intervention step (§ 2.7) to un-normalise the SAE output back to the original activation scale.
Train/eval split. We split the grid points randomly into 80% () for SAE training and 20% () for held-out evaluation of all alignment and intervention metrics. This split ensures that the concept alignments we report are not inflated by the SAE fitting the same activations it was trained on.
Independent Reference Solutions and Concept Fields
A central design principle of PHYSSAE is that concept fields must be derived from sources independent of the PINN being interpreted. The PINN output is , a linear function of . Any concept field derived from would therefore correlate with trivially, independent of whether the PINN has learned meaningful physics. We therefore use:
Analytic reference solutions for PDEs where closed-form solutions exist (heat equation via eigenfunction expansion, linear convection via the d’Alambert formula, and the Schrödinger soliton via the exact Zakharov-Shabat formula).
High-resolution numerical reference solutions for nonlinear PDEs (Burgers and Allen–Cahn): we run a pseudo-spectral RK4 solver on a fine grid () and interpolate onto the evaluation grid. These reference solutions are computed once, cached to disk, and reused across all PINN seeds and SAE runs.
Concept field definition. Given the reference solution , we define a small, physically motivated panel of scalar fields over . All quantities are computed from the reference, not the PINN. Table 1 gives the complete concept panel.
| PDE | Concept name | Formula |
| Heat | Amplitude | |
| Heat | Gradient | |
| Heat | Curvature | |
| Heat | Temporal rate | |
| Heat | Signed value | |
| Burgers | Amplitude | |
| Burgers | Shock indicator | |
| Burgers | Curvature | |
| Burgers | Temporal rate | |
| Burgers | Signed value | |
| Burgers | Spatial coord. | |
| Burgers | Norm. gradient | |
| Allen–Cahn | Amplitude | |
| Allen–Cahn | Interface indicator | |
| Allen–Cahn | Interface gradient | |
| Allen–Cahn | Reaction term | |
| Allen–Cahn | Temporal rate | |
| Allen–Cahn | Signed value | |
| Convection | Amplitude | |
| Convection | Gradient | |
| Convection | Temporal rate | |
| Convection | Signed value | |
| Convection | Characteristic coord. | |
| Convection | Spatial coord. | |
| Convection | Time | |
| Schrödinger | Soliton amplitude | |
| Schrödinger | Real part | |
| Schrödinger | Imaginary part | |
| Schrödinger | Phase gradient | |
| Schrödinger | Amplitude gradient | |
| Schrödinger | Amplitude rate | |
| Schrödinger | Spatial coord. |
Table 1. Physical concept fields used for alignment, computed from independent reference solutions. Central finite differences are used for spatial and temporal derivatives. For Schrödinger, is the complex field, its amplitude, and its phase.
Design rationale. Each concept was chosen to capture a physically distinct aspect of the PDE solution. We deliberately include both signed (e.g. itself) and unsigned fields (e.g. , ), because ReLU SAE codes are non-negative and therefore can only correlate positively with non-negative targets or partially with signed ones (the bilateral phenomenon discussed in §2.9). Coordinates capture geometric structure that the PINN may encode for positional awareness. Derivative-based concepts (gradient, curvature) test whether atoms encode information about local solution smoothness. PDE-specific quantities (interface indicator, shock indicator, characteristic coordinate) test whether atoms align with the defining physical mechanisms of each PDE.
Sparse Autoencoder Architecture and Training
Architecture. The SAE is a two-layer bottleneck network with non-negative (ReLU) codes and a decoder whose columns are constrained to unit Euclidean norm:
where , , , . The -th column of , denoted , is the decoder direction of atom : the direction in hidden-state space along which that atom projects. The unit-norm constraint on ensures that the code absorbs all the scale information, so that a larger unambiguously means a stronger activation of that direction.
We use with , giving an overcompleteness ratio of . Overcompleteness is essential: with the SAE would compress rather than decompose, and the dictionary atoms would mix multiple physical features. With the SAE has enough candidate atoms to specialise each one to a distinct physical concept.
Training objective.
The reconstruction term encourages faithful recovery of the PINN’s hidden state; without it the encoder could set all codes to zero. The L1 sparsity term pushes most codes to zero, creating atoms that fire only in specific spatio-temporal regions. The trade-off between these two objectives is controlled by , chosen to yield a mean of approximately 77–141 active atoms per sample (–28% of ), a regime in which prior LLM-SAE work has found good monosemanticity [2].
Unit-norm projection. After each gradient step we project the decoder columns to unit norm:
This projection is applied in-place after the Adam update. Without it, the gradient would distribute scale between the encoder weights, the decoder columns, and the codes, creating a three-way degeneracy. The projection fixes the decoder directions as unit-norm bases and forces the codes to carry the scale, which is necessary for the causal intervention formula to be well-defined (§ 2.7).
Optimisation. Adam with learning rate , 600 epochs, batch size 8192. We train 3 independent SAEs per PINN (different random seeds) to assess feature consistency (§ 2.10). Training is performed on the 80% split of activations; the 20% held-out split is used exclusively for evaluation. Across all 54 SAE runs (6 PDEs 3 PINN seeds 3 SAE seeds), zero dead atoms were observed (atoms with max activation below across all evaluation points), confirming that the dictionary is fully utilised.
Why not a pre-encoder bias (Anthropic-style SAE)? Several recent papers use a pre-encoder bias that is subtracted before the encoder and added back in the decoder, primarily to capture the activation mean and prevent encoder bias from encoding positional information. In our setting, the -scoring step plays this role: it subtracts the mean of each neuron’s activation across the grid before the SAE ever sees the data. We therefore use the simpler formulation without .
Concept Alignment
Alignment matrix. Given the -dimensional code vector at each held-out grid point and the concept fields evaluated at the same points, we define the alignment matrix
where all sums are over the held-out grid points. captures, for each pair of (atom, concept), the degree of linear association between the atom’s spatial activation pattern and the reference physical field. We call the observational alignment of atom with concept .
Top-atom selection. For each concept we identify the best-aligned atom:
Note that is used rather than because a strongly negative correlation (atom fires where concept is low) is equally informative as a positive one.
Why Pearson correlation? Pearson correlation is invariant to the affine transformations that -scoring and the SAE encoder introduce. It measures the linear co-variation of atom activation and concept field over the spatio-temporal grid, which is the cleanest test of whether an atom’s firing pattern tracks a physical quantity. Spearman rank correlation gives qualitatively identical results (not shown) but is more expensive to compute for pairs over 4000 grid points.
Statistical significance: permutation test. To assess whether the observed alignment is larger than chance, we generate a null distribution by permuting the spatial ordering of each concept field 500 times and computing for each permutation. The empirical p-value is the fraction of permuted maxima exceeding the observed maximum; the Z-score is . Z-scores exceed 10 for all PDEs and all concepts, confirming that the alignments are far beyond the permutation null.
Spearman correlation. For completeness we also compute Spearman (rank correlation), which is robust to nonlinearities in the atom–concept relationship. Reported results use Pearson; Spearman results are qualitatively identical.
Reporting. We report: (a) the top-1 alignment for each concept; (b) the mean top-5 alignment; (c) the number of concepts with and ; (d) permutation Z-scores. All values are computed on the held-out split to avoid overfitting the SAE to the evaluation points.
Direct Causal Intervention in the Original Hidden State
Observational alignment (§ 2.6) establishes that an atom’s activation pattern correlates with a physical concept, but correlation does not imply that the atom causally controls the corresponding feature in the model’s output. We therefore perform a causal intervention directly in the original PINN’s frozen hidden state, bypassing the SAE decoder entirely.
Intervention formulation. Given atom with unit-norm decoder direction and activation (the SAE code at each grid point), the contribution of atom to the original hidden state at is:
The counterfactual hidden state obtained by ablating atom at strength is:
At this exactly removes the atom’s contribution from ; at it partially suppresses it; at it over-removes (amplifies the absence). We evaluate at for dose-response analysis.
The counterfactual output is then:
and the causal effect of the intervention is:
This last equality shows that is linear in and linear in : the causal effect of ablating atom is exactly the product of its activation strength and its readout projection . The SAE decoder plays no role whatsoever in this formula: the intervention operates entirely in the original PINN’s hidden-state geometry.
Why this is stronger than SAE-internal ablation. A simpler approach would be to ablate within the SAE: set , decode , and measure . This does not test whether atom has a causal role in the original PINN; it only tests whether removing the atom’s contribution to the SAE reconstruction changes the output. The difference matters because the SAE is a lossy approximation of (reconstruction MSE is nonzero), and is partly the reconstruction error, not just atom ’s contribution. Our intervention uses the exact decomposition and isolates exactly the term, making the causal claim cleaner.
Causal metrics. The spatial distribution of over the grid tells us where atom ’s removal affects the PINN’s output. We quantify this distribution with:
Energy Spatial Footprint (ESF). The fraction of the domain that captures fraction of the total intervention energy:
We use , so is the smallest fraction of the domain that contains 80% of the intervention mass. A structurally meaningful atom should have a small : its effect should concentrate where it fires.
Support Capture. For localised concepts (shock, interface, characteristic coordinate), we define a reference support mask as the top-20% region of the reference concept field: . Support capture is the fraction of intervention mass inside the mask:
A top-aligned shock atom should have high support capture: its causal effect should concentrate near the shock. We do not report support capture for global concepts (amplitude, time, signed value) where the reference mask would cover the entire domain.
Intervention–concept correlation. Pearson between and over the grid, which directly measures whether the intervention map and the reference concept field have the same spatial shape.
Dose-response linearity. From Eq. equation 20, is linear in . We verify this by computing the intervention at and checking that is roughly constant across (since the shape of is -invariant) while the magnitude scales linearly. Violations of this linearity would indicate numerical instabilities or interactions with other atoms.
Baseline interventions. We apply the same hidden-state intervention framework to PCA, ICA, and the sparse linear dictionary baseline, enabling a controlled comparison in which the only variable is the decomposition method:
PCA. With principal component having loading and score , the counterfactual hidden state is .
ICA. With mixing vector and source , the counterfactual is .
Sparse linear dictionary. Same formula as SAE, but with a linear encoder (no ReLU), so codes can be negative.
This design ensures that differences in between methods cannot be attributed to the intervention formula but must reflect the structure of the decomposition itself.
Negative Controls
To establish that the observed causal localisation is specific to top-aligned atoms and not a generic property of ablating any active atom, we perform matched negative controls.
For each top-aligned atom , we select 10 control atoms by finding atoms that are similar in activation statistics but not aligned to the concept:
where is the mean activation. We select the 10 atoms with the smallest control score, i.e. those most similar in activation magnitude and decoder norm to the target atom. We then apply the identical hidden-state intervention to each control atom and compute .
The advantage is defined as:
A negative advantage means the top-aligned atom produces a more concentrated intervention than matched random atoms (lower is better), which is the result we expect for a concept-aligned atom. A positive advantage means random atoms are more concentrated, which would indicate the localisation is not specific.
Bilateral and Multi-Atom Representations
ReLU codes are non-negative: for all . A signed concept like (which takes both positive and negative values) cannot be captured by a single ReLU atom. Instead, the SAE must use at least two atoms: one firing where and one firing where . We call this a bilateral representation.
Detection. For each concept , we identify the top positive atom and the top negative atom . The bilateral pair is .
Quantification via regression R². To measure whether the pair explains more variance than either atom alone, we fit OLS regressions of the concept on the held-out grid:
The improvement measures the additional variance explained by the bilateral pair. We compare against 20 random atom pairs to establish that the improvement is specific to the aligned pair.
Joint direct intervention. For the bilateral pair, we also perform a joint hidden-state intervention:
and compute . If the two atoms truly represent complementary halves of the same physical field, the joint intervention should produce a larger and more symmetric causal effect than either single-atom intervention.
Feature Consistency Across SAE Seeds
Sparse coding solutions are not unique: infinitely many overcomplete dictionaries can achieve the same reconstruction and sparsity objectives. It is therefore important to test whether the atoms discovered by the SAE are reproducible or depend strongly on the random initialisation.
Protocol. For each PINN (i.e. each PDE and seed), we train 3 independent SAEs from different random seeds, yielding dictionaries , each in .
Matching. For every pair of SAEs , we compute the cosine similarity matrix:
and find the optimal assignment using the Hungarian algorithm (minimising total negative cosine similarity, i.e. maximising total cosine):
Metrics. After optimal matching, the matched cosine similarity for pair is . We report the mean and median across all pairs, as well as the fraction of matched atoms with cosine above 0.7, 0.8, and 0.9.
Low consistency (mean cosine ) means different random seeds find different but equally valid decompositions of the same latent space. Crucially, we show in the results (§4.7) that causal localisation (ESF) is stable across PINN seeds even when dictionary atoms are not stable across SAE seeds. This dissociation demonstrates that the functional property of physical localisation is intrinsic to the PINN’s representation, not to any particular SAE decomposition.
Algorithm 1 PHYSSAE: Mechanistic Interpretability of a Trained PINN
Require: Trained PINN with output head ; PDE reference solution ; grid ; sparsity ; dictionary size ; dose values .
— Stage 1: Activation Extraction —
Evaluate PINN on ; cache penultimate activations
Z-score to obtain ; store mean and std
Split: random split of rows of
— Stage 2: SAE Training —
for SAE seed do
Initialise SAE with random weights; normalise decoder columns
Train on minimising (Eq. (13))
After each gradient step: project decoder columns to unit norm (Eq. (14))
Store decoder directions
end for
Compute feature consistency via Hungarian matching (Eq. (29))
— Stage 3: Concept Alignment —
Compute reference concept fields from on
Encode held-out activations:
Compute alignment matrix (Eq. (15))
For each concept : find top atom (Eq. (16))
Compute permutation test Z-scores
— Stage 4: Direct Causal Intervention —
for each concept do
for each dose do
Compute counterfactual:
Compute output change:
Compute ESF, Support Capture, intervention–concept correlation
end for
Run 10 matched negative control ablations; compute ESF advantage
end for
— Stage 5: Bilateral and Baseline Comparisons —
Fit PCA, ICA, SparseDict on ; apply same intervention to their top components
For each concept: compute bilateral pair , , joint intervention
Aggregate all metrics; generate tables and figures
Algorithm Summary
For clarity, we give the complete PHYSSAE procedure as pseudocode in Algorithm 1.
Experimental Setup
PDE Benchmark
Table 2 summarises the six PDEs. We deliberately include converged (heat, Schrödinger), partially converged (Burgers, conv. easy), and known-failure (Allen–Cahn, conv. hard ) cases to span the full quality spectrum.
| PDE | Equation | Domain | IC | Ref. type | L2 err |
| Heat | Analytic | ||||
| Burgers | Spectral | ||||
| Allen–Cahn | Spectral | ||||
| Conv. easy | Analytic | ||||
| Conv. hard | Analytic | ||||
| Schrödinger | Soliton | Analytic |
Table 2. PDE benchmark. L2 error (mean±std, 3 PINN seeds) vs. independent reference.
Hyperparameters
Full settings are given in Appendix E. All experiments run on a single NVIDIA RTX 4090 (24 GB); total runtime 1.2 hours. Code and checkpoints will be released upon acceptance.
Results
PINN Training Quality
Heat and Schrödinger (seed 0) converge cleanly (L2 2.5% and 5.2% respectively); Burgers and conv. easy reach 17–29% and 25–28%; Allen–Cahn and conv. hard plateau at 93–95% as expected [11]. This spread enables the representational collapse analysis (§ 4.8).
Concept Alignment
Table 3 reports the maximum top-1 Pearson correlation per PDE, averaged across seeds. All PDEs have max ; the three best-trained PINNs reach . Z-scores against the permutation null are for all PDEs, confirming the alignments are highly significant.
| PDE | Best concept | Mean max | Z-score |
| Heat | gradient | ||
| Burgers | amplitude | ||
| Allen–Cahn | signed | ||
| Conv. easy | characteristic | ||
| Conv. hard | characteristic | ||
| Schrödinger | amplitude |
Table 3. Maximum top-1 concept alignment. Values are means over 3 PINN seeds (each reported as the max over concepts and seeds). Z-score against permutation null ().
Atom fields. Figure 1 and Figure 2 show atom activation fields (top row) against reference concept fields (bottom row) for Schrödinger and heat. Atom 73 on Schrödinger () fires as a diagonal stripe tracking the moving soliton—a single neuron encoding the entire soliton trajectory. On heat, atoms specialise to gradient (), curvature, and temporal derivative. In all cases the atom spatially mirrors its paired reference concept.
Figure 1. Schrödinger: SAE atom activations (top row) vs. independently-computed reference concept fields (bottom row). Atom 73 tracks the moving soliton envelope (). Concepts from analytic references, not the PINN output.
Figure 2. Heat: SAE atom activations (top row) vs. reference concept fields (bottom row). Atoms specialise to gradient (), curvature, and temporal derivative.
Alignment heatmaps. Figure 3 shows alignment matrices for Burgers and conv. hard. Each atom tends to align strongly with at most one or two concepts—the monosemanticity signature.
Figure 3. Concept alignment matrices (Pearson ) for Burgers (left) and conv. hard (right). Rows: top-20 atoms; columns: physical concepts. Atoms tend to align with at most one or two concepts (monosemanticity).
Direct Causal Intervention
For Schrödinger (Figure 1), ablating atom 73 () produces concentrated on the soliton track—the same region as the atom’s firing pattern. For heat (Figure 2), atom 153 ablation localises in the early-time Gaussian region. In both cases the causal footprint matches observational alignment, providing mechanistic rather than merely correlational evidence.
Aggregate causal localization. Table 4 shows mean ESF across all concepts and seeds. SAE achieves the lowest ESF on all six PDEs—confirmed by the raw numbers: heat SAE = 0.165 vs. PCA = 0.700 (4.2×), Schrödinger SAE = 0.179 vs. PCA = 0.406 (2.3×), conv. hard SAE = 0.373 vs. PCA = 0.745 (2.0×). The sparse-linear-dictionary baseline (SparseDict) lies between SAE and PCA, confirming that the SAE’s ReLU nonlinearity contributes beyond mere overcompleteness.
| PDE | SAE ESF | PCA | ICA | SD | Speedup | Max | Neg. ctrl | Bilat. |
| Heat | 0.1650.23 | 0.700 | 0.587 | 0.278 | 4.2 | 0.890 | ||
| Burgers | 0.3740.18 | 0.585 | 0.574 | 0.341 | 1.6 | 0.804 | ||
| Allen–Cahn | 0.4760.29 | 0.576 | 0.622 | 0.412 | 1.2 | 0.692 | ||
| Conv. easy | 0.3660.24 | 0.546 | 0.571 | 0.423 | 1.5 | 0.872 | ||
| Conv. hard | 0.3730.33 | 0.745 | 0.653 | 0.292 | 2.0 | 0.951 | ||
| Schrödinger | 0.1790.11 | 0.406 | 0.538 | 0.312 | 2.3 | 0.920 | ||
| Mean | 0.322 | 0.593 | 0.591 | 0.343 | 2.3 | 0.872 |
Table 4. Aggregate main results across 6 PDEs 3 PINN seeds. ESF: fraction of domain capturing 80% of causal effect mass (lower = more localised = better for SAE). Neg. ctrl. adv.: ESF(atom) − ESF(matched random), negative means atom is more localised. Bilateral : two-atom minus single-atom regression improvement. Consistency: Hungarian-matched mean cosine across 3 SAE seeds.
Negative Controls
Figure 4 splits results by concept locality. For localised concepts (shock, interface, characteristic coordinate), the advantage of top-aligned atoms over matched random controls is positive on four of six PDEs, and largest on conv. hard (+0.1250.172), where even the catastrophically failing PINN () encodes the characteristic coordinate in an atom whose ablation is 2.7–4.4× more localised than random. For heat and Schrödinger, top-aligned atoms encode smooth global concepts (amplitude, temporal evolution) whose interventions are inherently distributed; is not the appropriate metric here (any well-activated atom is globally influential), and support capture on the concept region also shows advantage for these cases (Appendix § 9).
Figure 4. Negative controls: ESF advantage (top-aligned atom minus matched random) split by localised (left) vs. global (right) concepts. Blue = atom more localised than random (positive result); red = random incidentally more localised (expected for global concepts).
Overcompleteness and Sparsity Ablations
Table 5 shows that concept alignment increases monotonically with dictionary size across all three tested PDEs: Burgers alignment rises from 0.855 at to 0.964 at ; heat from 0.815 to 0.836. The larger marginal gain from than suggests diminishing returns above overcompleteness at hidden width 128.
| PDE | Metric | |||
| Heat | Mean top-1 | 0.815 | 0.831 | 0.836 |
| Heat | Recon. MSE | 0.128 | 0.075 | 0.042 |
| Burgers | Mean top-1 | 0.855 | 0.928 | 0.964 |
| Burgers | Recon. MSE | 0.192 | 0.106 | 0.060 |
| Schrödinger | Mean top-1 | 0.817 | 0.836 | 0.869 |
| Schrödinger | Recon. MSE | 0.240 | 0.129 | 0.068 |
Table 5.
Table 6 shows reconstruction MSE and alignment as functions of mean L. Across the swept range alignment stays high (0.79–0.97 for Burgers) until reconstruction collapses at very high sparsity (, L). Our operative configuration (, L–141) lies in the flat alignment region with good reconstruction.
| Mean | Recon. MSE | Mean top-1 | Status | |
| 259.7 | 0.041 | 0.916 | too dense | |
| 254.8 | 0.040 | 0.918 | too dense | |
| 207.6 | 0.049 | 0.941 | good | |
| 0.02 | 149.1 | 0.070 | 0.964 | operative |
| 0.1 | 75.4 | 0.120 | 0.935 | sparse |
| 0.3 | 35.0 | 0.234 | 0.916 | recon. degrades |
Table 6. Sparsity ablation: alignment and reconstruction vs. mean across values for Burgers (representative PDE). Operative configuration () highlighted; alignment is flat over a wide sparsity range.
Bilateral and Multi-Atom Representations
Table 7 and the Bilateral column of Table 4 show that adding the opposite-sign atom improves concept regression by +0.05 to +0.15 across all PDEs, while random atom pairs decrease by up to 0.60. The gap between SAE pairs and random pairs is large and consistent, indicating bilateral encoding is a genuine structural property. Conv. easy shows the largest gain (), consistent with its sinusoidal IC requiring symmetric representations.
| PDE | Single atom | SAE pair | Random pair | |
| Heat | 0.591 | 0.643 | +0.053 | −0.226 |
| Burgers | 0.387 | 0.483 | +0.097 | −0.203 |
| Allen–Cahn | 0.279 | 0.367 | +0.088 | −0.189 |
| Conv. easy | 0.451 | 0.598 | +0.146 | −0.196 |
| Conv. hard | 0.332 | 0.362 | +0.030 | −0.127 |
| Schrödinger | 0.504 | 0.607 | +0.102 | −0.215 |
| Mean | 0.424 | 0.510 | +0.086 | −0.193 |
Table 7. Bilateral alignment: mean for single atom, SAE atom pair, and random pair per PDE (averaged over concepts and seeds). = pair − single. SAE pairs consistently improve over single atoms; random pairs decrease .
Feature Consistency
Table 8 shows that mean Hungarian-matched cosine similarity across independent SAE seeds is 0.331–0.371, with only 1–2% of atoms above 0.7. SAE dictionaries are non-unique: different initializations find distinct but equally valid sparse decompositions. This is expected for overcomplete dictionaries and is well-documented in the LLM–SAE literature [2]. Importantly, this does not compromise causal localization: ESF std across PINN seeds is <0.05 for four of six
| PDE | Mean cos | Median cos | Frac >0.7 | Frac >0.9 |
| Heat | 0.364 | 0.340 | 0.016 | 0.003 |
| Burgers | 0.371 | 0.347 | 0.014 | 0.002 |
| Allen–Cahn | 0.364 | 0.341 | 0.014 | 0.002 |
| Conv. easy | 0.353 | 0.331 | 0.017 | 0.002 |
| Conv. hard | 0.331 | 0.308 | 0.018 | 0.002 |
| Schrödinger | 0.336 | 0.313 | 0.011 | 0.001 |
| Mean | 0.353 | 0.330 | 0.015 | 0.002 |
Table 8. SAE feature consistency across 3 independent SAE seeds per PINN. Mean and median Hungarian-matched cosine similarity between dictionaries; fraction of atoms above thresholds. Low cosine reflects non-uniqueness of sparse coding solutions, not instability of causal localization (ESF₈₀ std <0.05 across PINN seeds).
Table 5: Overcompleteness ablation: alignment and reconstruction across dictionary sizes for three representative PDEs. Mean top-1 alignment and reconstruction MSE on held-out activations. SAE trained at ; all other settings fixed.
PDEs (Table 4), confirming that the functional property we measure is seed-stable even when the specific atoms realising it are not.
Representational Collapse in Failed PINNs
Figure 5 shows that effective rank of the hidden covariance increases with residual loss: well-trained PINNs (heat, Schrödinger) have lower-rank, more structured representations; failed PINNs have higher-rank, more diffuse activations. SAE active atoms show no clear monotone trend with training quality, suggesting that SAE finds similar-density decompositions regardless of PINN convergence, while alignment (not sparsity) degrades with failure—consistent with Allen–Cahn’s lower max vs. heat’s 0.890.
Figure 5. Representational quality vs. PINN training quality. Failed PINNs have higher-rank (more diffuse) representations; SAE finds similar-density decompositions but alignment degrades, making it a diagnostic for training failure.
Strikingly, conv. hard (L2 error 95%) shows the highest alignment in our benchmark (max for characteristic coordinate ). The PINN has learned to encode the operator’s fundamental invariant even when its solution quality is poor, and PHYSSAE successfully identifies this. This demonstrates the diagnostic potential of our framework: one can determine which physical structures a failing PINN has and has not internalised.
Discussion
Terminology. We use alignment (= correlation/association), functional relevance (= intervention changes output), and causal evidence (= direct hidden-state intervention changes output in a structured way). We do not claim atoms are physical variables, nor that SAE “discovers the true physics.” We claim atoms are aligned with physical concepts and have a localized functional role.
On feature non-uniqueness. Low cross-seed cosine (≈0.35) reflects the non-uniqueness of sparse coding. Atoms aligned to the same concept are themselves unstable across SAE runs—yet the causal localization metric is stable. This dissociation is itself a finding: the functional property of concept-aligned causal localization is intrinsic to the PINN’s representation, not to any particular SAE decomposition of it.
Limitations. (1) Architecture is intentionally modest (5-layer, width 128); interpretability at larger scales is future work. (2) 1+1D domains studied; 2D/3D may require spatially factored SAE architectures. (3) Concept panels are hand-designed; symbolic probing (SINDy/PySR on atom activations) would automate concept discovery. (4) The bilateral joint intervention and dose-response analysis could be extended to quantify multi-atom interaction effects.
Conclusion
We presented PHYSSAE, the first mechanistic interpretability framework for residual-loss PINNs. By training sparse autoencoders on PINN hidden activations and intervening directly in the original hidden state, we showed that PINNs develop sparse, physically structured latent representations. SAE achieves 1.2–4.2× more concentrated causal footprints than PCA and ICA across six PDE families; top-aligned atoms beat matched random controls on localized concepts; two-atom bilateral representations consistently outperform single atoms. The framework diagnoses what a failing PINN has and has not learned, enabling interpretability-aware scientific ML.
References
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In ICLR Workshop, 2017.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Henighan, Shan Carter, Christopher Olah, Tom Hume, Jack Clark, and Anthropic. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023.
Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks for heat transfer problems. Journal of Heat Transfer, 143(6), 2021.
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021.
Ryan Fear, Priyabrata Mukhopadhyay, Michael McCabe, Alberto Bietti, and Miles Cranmer. Physics steering: Causal control of cross-domain concepts in a physics foundation model. arXiv preprint arXiv:2511.20798, 2025.
Additional Intervention Maps
Support Capture for Global Concepts
For global/smooth concepts where is not discriminating (heat amplitude, Schrödinger amplitude), we report support capture: fraction of mass inside the top-20% region of the reference concept. Top-aligned SAE atoms achieve higher support capture than matched random atoms for these concepts in all cases, confirming that the advantage is real but requires the appropriate metric.
Full Atom Field Galleries
Alignment Matrices (All PDEs)
Hyperparameter Table
| Setting | Value |
| PINN | |
| Hidden layers / width | 5 / 128 |
| Activation | tanh |
| Adam iters / LR | 8000 / |
| L-BFGS iters | 300 |
| Loss weights , , | 100, 100, 1 |
| BC/IC/residual pts per step | 150 / 150 / 3000 |
| PINN seeds | 3 (0, 1, 2) |
| Sparse Autoencoder | |
| Hidden dim / Dict size | 128 / 512 (4×) |
| Sparsity | |
| Epochs / Batch / LR | 600 / 8192 / |
| SAE seeds per PINN | 3 |
| Train/eval split | 80% / 20% |
| Dead atoms (all runs) | 0 (<0.02% for conv. hard seed 1) |
| Evaluation | |
| Grid | |
| Permutation test | 500 |
| Negative controls per atom | 10 |
| Dose values | 0.25, 0.5, 1.0, 1.5 |
| Bilateral random pairs | 20 |
Table 9. Full experimental hyperparameters.
Figure 6. Burgers atom 115 (). Negative correlation = atom fires where . Ablation concentrates in the lower-left pre-shock region: the negative- bilateral half.
Figure 7. Conv. hard (, L2 error 95%). Despite PINN failure, the intervention map localises in the characteristic-coordinate direction.
Figure 8. Allen–Cahn (failed PINN, L2 error 93%). The SAE still localises the causal footprint to the region where the PINN did learn structure.
Figure 9. Burgers atom fields. From left: amplitude, shock gradient, curvature, temporal derivative, signed value, shock indicator, spatial coordinate.
Figure 10. Conv. hard: atom for characteristic coordinate achieves .
Figure 11. Allen–Cahn: alignment weaker (max ) reflecting the PINN’s failure to resolve correct interface dynamics.
Figure 12. Conv. easy: characteristic-coordinate atom ().
Figure 14. Alignment matrices for remaining PDEs.
References
- [1]Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In ICLR Workshop, 2017.arxiv.org/abs/1610.01644
- [2]Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., … Anthropic. (2023). Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread.
- [3]Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks for heat transfer problems. Journal of Heat Transfer, 143(6), 2021.DOI
- [4]Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023.arXiv
- [5]Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., DasSarma, N., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., … Olah, C. (2021). A mathematical framework for transformer circuits. Transformer Circuits Thread.
- [6]Ryan Fear, Priyabrata Mukhopadhyay, Michael McCabe, Alberto Bietti, and Miles Cranmer. Physics steering: Causal control of cross-domain concepts in a physics foundation model. arXiv preprint arXiv:2511.20798, 2025.arXiv
- [7]Ehsan Haghighat, Maziar Raissi, Adrian Moure, Hector Gomez, and Ruben Juanes. A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics. Computer Methods in Applied Mechanics and Engineering, 379:113741, 2021.DOI
- [8]Yeping Hu and Shusen Liu. Interpreting CFD surrogates through sparse autoencoders. arXiv preprint arXiv:2507.16069, 2025.arxiv.org/abs/2507.16069
- [9]Ameya D Jagtap and George Em Karniadakis. Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics, 28(5):2002–2041, 2020.DOI
- [10]Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In International Conference on Machine Learning, pp. 2668–2677, 2018.arxiv.org/abs/1711.11279
- [11]Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in physics-informed neural networks. In Advances in Neural Information Processing Systems, volume 34, 2021.arxiv.org/abs/2109.01050
- [12]Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, volume 30, 2017.arxiv.org/abs/1705.07874
- [13]Levi McClenny and Ulisses Braga-Neto. Self-adaptive physics-informed neural networks using a soft attention mechanism. arXiv:2009.04544, 2020.DOI
- [14]Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.DOI
- [15]Costabal, F. S., Yang, Y., Perdikaris, P., Hurtado, D. E., & Kuhl, E. (2020). Physics-informed neural networks for cardiac activation mapping. Frontiers in Physics, 8, 42.
- [16]Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Cooney, E., Petrov, T., McDougall, C., Tamkin, A., Batson, J., Henighan, T., Clark, J., Olah, C., & Hume, T. (2024). Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet. In Transformer Circuits Thread.
- [17]Bahareh Tolooshams, Ailsa Shen, and Anima Anandkumar. Mechanistic interpretability with sparse autoencoder neural operators. arXiv preprint arXiv:2509.03738, 2025.arxiv.org/abs/2509.03738
- [18]Tucny, J.-M., Durve, M., & Succi, S. (2025). Is the end of insight in sight?: Studying interpretability in a Boltzmann-PINN. Frontiers in Physics, 13, 1641727.
- [19]Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021a.DOI
- [20]Sifan Wang, Hanwen Wang, and Paris Perdikaris. On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 384:113938, 2021b.DOI
- [21]Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why PINNs fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768, 2022.arxiv.org/abs/2007.14527
- [22]Wang, S., Sankaran, S., & Perdikaris, P. (2024). Respecting causality is all you need for training physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 421, 116813.
- [23]Yizheng Wang, Jia Sun, Jinshuai Bai, Cosmin Anitescu, Mohammad Saber Eshaghi, Xiaoying Zhuang, Timon Rabczuk, and Yinghua Liu. Kolmogorov–Arnold-informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov–Arnold networks. Computer Methods in Applied Mechanics and Engineering, 430:117290, 2024b.DOI