WorksComputer Science
A Mechanistic Interpretability Study of an Astronomy Foundation Model
Abstract
Foundation models are increasingly used across the sciences, but whether their internal computations correspond to known scientific operations or proceed through different mechanisms remains unclear. Understanding these mechanisms is essential for assessing reliability, identifying failure modes, and learning new science from the models. We present, to our knowledge, the first causal mechanistic interpretability study of an astronomy foundation model, AION-1, focusing on its multimodal advantage in redshift prediction, where combining imaging with photometry outperforms photometry alone. We find that this advantage is largest for spatially extended and dust-attenuated galaxies, and that most of it is localized to the first of the encoder's twelve blocks. Within that block, how well a property is linearly decodable does not predict whether it can be localized. Spatial extent is causally used, and one attention head dominates linear probing for it, yet ablating that head has negligible effect on redshift prediction: the information is distributed across heads and is decodable again one layer after we erase it. Dust attenuation shows the opposite pattern: it is only weakly decodable anywhere in the encoder, yet a single head is necessary for most of the multimodal advantage, particularly for the most dust-attenuated galaxies, and keeping that head as the only active head in its block preserves nearly all of the advantage. None of the astrophysically motivated operations we tested explains what this head computes. Whether this reflects a limitation of current interpretability methods or a computation that does not correspond to any known astrophysical operation remains open, and resolving it is a prerequisite for using interpretability as a tool for scientific discovery.
Introduction
Foundation models trained on heterogeneous corpora are now being applied across the natural sciences, from molecular biology [13] (Rives et al. 2021) to multi-physics surrogates [16] to astronomy [24], with the goal of producing models that generalize across many downstream scientific tasks. Astronomy is a particularly compelling setting for this paradigm. Its data are inherently multimodal: multi-wavelength photometry, spectra, and imaging each capture distinct information about the same object, and combining them can provide a more comprehensive understanding of astrophysical phenomena. Additionally, many properties of astrophysical interest, such as redshift and stellar mass, are physically measurable, so what a model has learned can be checked directly against them.
The largest such effort in astronomy to date is AION-1 [Parker et al., 2025], a 300M–3.1B parameter family of omnimodal models trained on more than 200 million observations from five large-scale surveys (Legacy Survey, HSC, SDSS, DESI, Gaia). It extends 4M-style multimodal masked modeling [20] to 39 astronomical modalities in a single transformer and reports strong performance on regression, classification, generation, and retrieval. Yet despite the centrality of such
Preprint. models to data-driven astrophysics, almost nothing is known about how they integrate modalities or why multimodal inputs improve scientific predictions. This matters because astronomers want to use these models to discover astrophysics, not merely to predict it.
Mechanistic interpretability has begun to answer analogous questions in vision and language models by identifying features and circuits that implement specific behaviors [23, 1, 15]. Applied to scientific foundation models, it can also produce actionable domain hypotheses: Wang et al. [2026] reverse-engineered an epigenetic disease classifier and surfaced a fragmentomic biomarker hypothesis not central to the prevailing Alzheimer’s literature. These tools have known limitations, however: probes reveal what is decodable rather than what is causally used [3], and ablating a component may not remove a property the model encodes redundantly [17]. Whether these failure modes arise in a scientific model, where ground truth can be verified numerically, has not been tested.
We apply these methods to AION-1 through a concrete, well-studied problem: photometric redshift estimation. A galaxy’s redshift , the fractional shift of its spectrum toward longer wavelengths due to cosmic expansion, is a proxy for its distance. It can be measured precisely from a spectrum, but spectroscopy is slow and observationally expensive to scale, so photometric methods instead infer redshift from a galaxy’s brightness in a few broadband filters [Salvato et al., 2019]. AION-1 attains from photometric flux values only () and when galaxy images are also provided (). This multimodal advantage is not well understood and raises a natural mechanistic interpretability question: why does adding images to photometry improve redshift prediction?
We find that the galaxies whose predictions improve under are systematically more spatially extended and dust-attenuated. We investigate which components of AION-1 produce the multimodal advantage, what they encode, and how their computations compare to hypotheses for how images improve redshift prediction. In particular, our work makes three main contributions:
First causal mechanistic study of an astronomy foundation model. We apply linear and nonlinear probing, attention analysis, ablation, activation and path patching, and least-squares concept erasure to address a question that black-box evaluation alone cannot answer.
Linear decodability does not imply localization. We localize most of the multimodal advantage to AION-1’s first encoder block. Within it, one head best linearly encodes spatial extent and has an attention pattern and output geometry consistent with computing it, yet the head has a negligible causal effect on redshift prediction. This is not because spatial extent is causally unused: the model relies on it, but the information is spread across heads and remains decodable after that head is removed. A strong probe therefore does not identify the component that matters, and a null ablation does not show that a property is unused.
Localization of most of the multimodal advantage to a single head. A different attention head in the same encoder block is causally necessary for most of the multimodal advantage and preserves nearly all of it when the other heads in that block are mean-ablated. On the most dust-attenuated galaxies, imaging reduces attenuation bias, the statistical compression of predictions toward the mean [21], and this head is required for that reduction: ablating the head leaves the prediction more attenuation-biased than the photometry baseline. To our knowledge, this is the first localization of such a reduction to a single model component. What the head computes remains unidentified: probes of its output recover only weak dust information, and none of the astrophysically motivated operations we tested predicts its behavior.
Related work
Multimodal foundation models for astronomy. Cross-modal pretraining was introduced to astronomy by AstroCLIP [24], which contrastively aligns galaxy images and DESI spectra and matches a supervised ResNet-18 on photometric redshifts. AstroM [Rizhko and Bloom, 2025] extends this to photometric time series, and recent multimodal masked autoencoders learn joint image–spectrum representations [12]. AION-1 [Parker et al., 2025] extends this lineage to omnimodal sequence modeling across 39 modalities.
Photometric redshift estimation and the role of images. Photometric redshift estimation has a long history of template-fitting and supervised ML methods [Salvato et al., 2019]. Image-based deep learning approaches [11, 18, 14] (Pasquet et al. 2018) consistently outperform feature-based regressors. The improvement is attributed to morphological information such as galaxy size, shape, and surface brightness [Soo et al., 2018]. Moran et al. [ [21] observe that image-based methods suffer less attenuation bias than photometry-based ones, which they attribute to deep networks combining the redshift information in individual pixels more effectively than integrated photometry does. These accounts are correlational and do not examine what the model internally computes.
Mechanistic interpretability and the limits of probing and ablation. Mechanistic interpretability, developed largely on language models, has two widely used classes of methods, observational probes and causal interventions. Probing is observational: a property can be cleanly decodable from a representation without being used downstream [3]. Causal interventions such as activation patching [8] and ablation were developed to close this gap, and recent work shows explicit dissociation between decodability and causal use in controlled settings [Sharma et al., 2026]. On the other hand, ablation can fail to cleanly remove redundantly encoded properties. Individual units and heads can be removed from a trained network with little loss [19, 22], and ablated components are frequently compensated for downstream [17] (Wang et al. 2023). We show that both limitations arise in a scientific foundation model.
Interpretability for astronomy models. Existing interpretability work on astronomy models characterizes learned features but does not test their causal role post hoc. Sparse-by-design architectures [Wu, 2025] ensure causal use of features by construction, and post-hoc sparse autoencoders on pretrained encoders [Wu and Walmsley, 2025] recover monosemantic morphology features without verifying causal use. Closer to our setting, Dey et al. [ [7] train a capsule network to estimate photometric redshifts and identify a feature hypothesized to distinguish intrinsically red ellipticals from dust-reddened edge-on spirals. We follow a post-hoc approach and add causal interventions.
Setup
AION-1 and the redshift prediction task
We use the AION-1 base model ( parameters) from Polymathic AI [Parker et al., 2025] with the encoder frozen. The encoder has 12 transformer blocks with hidden dimension 768, each with 12 attention heads of dimension 64. Following AION-1’s downstream regression setup, we train a redshift readout, consisting of a lightweight cross-attention pooling layer and two-layer MLP, under MSE loss on the frozen encoder output to predict the spectroscopic redshift (architecture, training hyperparameters, and robustness checks in Appendix § 7).
We compare two input configurations and train one readout for each. Photometry (Ph) uses nine scalars: a galaxy’s total flux in four optical Legacy Survey bands (, , , ) and four infrared WISE bands (–), plus the Galactic foreground extinction , a measure of Milky Way dust along the line of sight. Each scalar is mapped to a single discrete token by its own learned code, giving nine encoder tokens. Photometry + Image (Ph+Im) adds a four-band , , , DESI Legacy Imaging Survey cutout (160160 pixels) [6], the spatially resolved counterpart of the optical fluxes. AION-1’s image code encodes the cutout as a grid of 576 vector-quantized patch tokens, for 585 encoder tokens in total.
Data
We use the DESI Early Data Release galaxy sample [5], distributed in the Multimodal Universe format [2]. This dataset cross-matches Legacy Survey imaging and photometry [6] with DESI spectroscopic redshifts and with physical properties such as stellar mass and dust attenuation inferred by PROVABGS [9], which fits stellar-population models to each galaxy’s spectral energy distribution. We restrict the dataset to the Bright Galaxy Survey (BGS) bright sample with , finite , , , Legacy Survey fluxes, valid magnitudes in , , , , and a positive PROVABGS stellar mass. This yields 67,755 galaxies, which we split 80/20 into train and test sets (54,204 / 13,551).
Results
Figure 1 shows that the Ph readout reaches and the Ph+Im readout reaches . Furthermore, adding the image modality reduces the normalized median absolute deviation from 0.0225 to 0.0150 and the catastrophic-outlier fraction from to 0.18%. The median residual bias is below in both readouts. We ask which galaxies are helped by imaging (§ 4.1), where in the encoder the advantage enters (§ 4.2), and study two relevant properties (§ 4.3–§ 4.4).
Figure 1. Predicted versus spectroscopic redshift on the test set for Ph (left) and Ph+Im (right) redshift readouts on frozen AION-1 encoder embeddings. Hexbin density uses a logarithmic color scale shared between panels. The dashed line marks and the dotted lines bound the catastrophic-outlier envelope , with . Per-panel statistics: ; normalized median absolute deviation (NMAD), a robust scatter estimator; catastrophic-outlier fraction = fraction of galaxies with ; bias = median .
Imaging helps galaxies with faint fiber -magnitudes and high dust attenuation
For each test-set galaxy, we count a prediction as correct when . We partition galaxies by the joint outcome of the two readouts and focus on two non-overlapping groups: Ph+Im better (), galaxies that the Ph+Im readout predicts correctly while the Ph readout does not, and Both correct (), galaxies that both readouts predict correctly. Relative to the Both correct population, Ph+Im better galaxies have fainter fiber -magnitudes (magnitudes are logarithmic brightness measures on which larger values are fainter) and higher dust attenuation (Figure 2).2 The fiber -magnitude measures only the light falling within the 1.5″ DESI fiber aperture, so it is fainter than the total -magnitude by an amount that grows with the fraction of a galaxy’s light lying outside the fiber. A fainter indicates either an intrinsically faint galaxy or a spatially extended one. Higher indicates more dust in the galaxy’s interstellar medium along our line of sight, which attenuates and reddens its light. Adding imaging therefore helps preferentially on galaxies that are faint within the fiber aperture and on dust-attenuated galaxies, two independent contributors to the performance gain (Appendix § 8).
Figure 2. Kernel density estimates of two galaxy properties (left: fiber -magnitude ; right: dust attenuation ) in the held-out test set: Ph+Im better (purple, ): galaxies for which the Ph+Im readout satisfies while the Ph readout does not; Both correct (red, ): galaxies for which both readouts satisfy this threshold. Dashed vertical lines mark per-population medians; two-sample Kolmogorov–Smirnov -values are annotated in each panel.
The multimodal advantage emerges in the first encoder block
We now ask where in the encoder the image modality’s contribution enters by localizing the stage at which each galaxy property becomes linearly decodable. We fit ridge-regression linear probes to AION-1’s intermediate representations at 14 stages: the token embedding (the input to block 1), the output of each of the 12 transformer blocks, and the post-encoder representation (the output of the final encoder layer-norm). We run the frozen encoder forward under both input configurations ( and ). At each stage, we mean-pool the token activations to a single 768-dim vector per galaxy and report the test-set (full procedure in Appendix § 9.§ 1).
Figure 3 shows the linear-probe for every other block and for the ten properties discussed below (all 14 stages in Fig. C.1). In the post-encoder representation (last row), the image modality substantially improves the decodability of three properties: redshift (), stellar mass (), and the fiber -magnitude (). The three dust parameters shift modestly upward, but remain only weakly decodable in either configuration. Properties given directly by photometry, namely the broadband magnitudes and , are decodable in both configurations at the final stage (). The image modality therefore supplies linearly decodable information about redshift, mass, fiber -magnitude, and (more weakly) dust parameters.
Figure 3. Layerwise linear-probe for 10 galaxy properties (columns) at 8 encoder stages (rows): the token embedding (“Token Emb”), the outputs of blocks 1, 3, 5, 7, 9, and 11 of the 12, and the post-encoder representation. Probes are ridge regressions on the mean-pooled 768-dim hidden state at each stage. Columns are grouped (vertical lines) into three families: broadband magnitudes (, , ) and , which are directly determined by the photometry tokens; properties whose decodability the configuration substantially improves (redshift , stellar mass , fiber -magnitude ); and dust parameters (, , ). Left: ; right: .
Figure 3 also shows that most of the multimodal advantage is concentrated in the first encoder block. The gap between the and redshift probes grows from 0.02 at the token embedding to 0.07 after block 1, and by only 0.02 more over the remaining eleven blocks. Stellar mass and fiber -magnitude follow a similar pattern; for the dust parameters, the probe gap is smaller and accumulates gradually across the encoder. Most of the cross-modal integration responsible for this advantage therefore occurs in block 1, motivating the analysis that follows.
We also confirm this localization with activation patching. We construct a corrupted run by replacing each galaxy’s image tokens with those of another test galaxy, leaving its photometry tokens unchanged. We then patch one block’s residual-stream contribution at a time, from the clean run into the corrupted run to test sufficiency, or from the corrupted run into the clean run to test necessity, and re-run the trained readout (protocol and per-block values, Appendix § 9.§ 2). Block 1 is the only block that is on its own both sufficient, with its clean contribution alone restoring the readout’s full , and necessary, with its corrupted contribution alone driving to , close to the unpatched corrupted run’s .
Spatial extent is linearly decodable and causally used, but not localized
The probes of § 4.2 show that information enters the representation, not that predicting redshift uses it. Moreover, confounds spatial extent with total brightness, since a galaxy can be faint within the 1.5″ fiber either because it is intrinsically faint or because its light is spread beyond the aperture. For the rest of this section, we therefore stratify on the fiber-minus-total -magnitude , a logarithmic measure of the fraction of a galaxy’s light falling outside the fiber; larger means a more spatially extended galaxy.
Spatial extent is causally used
To test whether spatial extent is causally used in predicting redshift, we first intervene on the input. For every test galaxy, we replace the four-band cutout with either a compact template (a centered two-dimensional Gaussian with px, approximating an unresolved point source), or an extended Gaussian blur ( px), leaving the photometry tokens unchanged (construction of both counterfactuals, Appendix § 10.§ 1). Because the image codec reads only the central 96 × 96 pixels of each cutout, each counterfactual is rescaled per band so that its flux inside that crop equals the original image’s; both substitutions therefore alter spatial extent without changing the brightness the codec sees. Both substantially reduce the readout’s , from the clean value of 0.931 to 0.556 under the compact template and 0.766 under the extended blur. Both fall below the , so the readout does not revert to a photometry-only prediction when the image is replaced; the substituted morphology continues to be used.
Stratifying by quintiles of (Figure 4a; per-quintile values and intervals in Table D.1) shows that each substitution reduces most where it contradicts the galaxy’s true spatial extent: the compact template’s decrease grows with ( from the lowest to the highest quintile), while the extended blur’s shrinks (0.540 → 0.793). A perturbation that merely pushes the input off-distribution would affect all quintiles alike, and the baseline varies by less than 0.07 across the same quintiles, so neither off-distribution effects nor the bins themselves explain the trends. Redshift prediction therefore depends causally on spatial extent; § 4.3.3 supports this same conclusion by erasing the spatial extent direction from block 1’s output without altering any image input.
Figure 4. (a) Redshift on the held-out test set, stratified by quintiles of , for (gold, dotted) and for with the original image (blue, solid), with a compact template (purple, dashed), or with an extended blur (black, dash-dotted). Error bars are 95% paired bootstrap CIs. (b) Mean attention computed by Head 1 under the input configuration, averaged over 200 test galaxies. Keys and queries are grouped into the nine photometry tokens and four image-token regions: a two-token-wide border ring surrounding the 2424 token grid, and bright/medium/faint quartiles (top, middle two, bottom) of token brightness within each galaxy’s remaining interior. (c) Distribution over the same 200 galaxies of across Head 1’s 576 image-token outputs, where PC0 is each galaxy’s first principal component. (d) Spatial extent probe from each block 1 head’s 64-dim output alone (blue) and from a probe refit on the remaining eleven heads (red), against the joint twelve-head probe (dashed, ).
The head that best linearly encodes spatial extent has a negligible causal effect
With the multimodal advantage for both redshift and localized to block 1, we ask which head(s) implement it. We reuse the linear-probe protocol of § 4.2, applied head-wise within block 1 (Appendix § 10.§ 2). Head 1 is the best of the 16 block 1 heads on seven of the eight probe targets (Table D.2), including spatial extent (: against 0.643 of the next best head) and redshift (: 0.697 against 0.594). The only exception is .
Head 1 is consistent with a head that computes spatial extent. In the attention pattern (Figure 4b), image queries in the galaxy interior (bright, medium, and faint) attend most strongly to bright image keys.
This image-token aggregation is organized radially: the first principal component (PC0) of Head 1’s output explains a mean 84.1% of the variance among a galaxy’s 576 image tokens, and each token’s PC0 score tracks its distance from the image center, with of mean 0.739 (Figure 4c; Appendix § 10.§ 2). A dominant output direction that tracks radial position is what a head encoding spatial extent would produce.
However, zeroing Head 1’s output for the test set and running the trained redshift readout moves by in any quintile (Appendix § 10.§ 2). Replacing Head 1’s output at all 585 positions with that of a redshift-matched galaxy from the opposite quintile, an in-distribution intervention, likewise has a negligible effect on the prediction (Appendix § 10.§ 3). The head that best linearly encodes spatial extent therefore has a negligible causal effect on the prediction.
Spatial extent is not localized to a single head or direction
The results of § 4.3.1 and § 4.3.2 fit together because spatial extent is not localized to Head 1. Head 1 is the single best head for encoding (), but a probe refit on the eleven heads that remain when Head 1 is removed reaches (Figure 4d; joint and leave-one-head-out probes, Appendix § 10.§ 2): spatial extent remains decodable from block 1 after Head 1 is ablated. Additionally, every other head in block 1 carries spatial extent information on its own ( between 0.448 and 0.643, Table D.2), a probe on all twelve jointly reaches 0.891, and removing any one of the other eleven heads from the joint probe reduces its by at most 0.006. Therefore, no single head is necessary for spatial extent to be linearly decodable from block 1’s head outputs.
Erasing the linear spatial extent direction from block 1’s full output, rather than ablating a head, does not remove the property either. A rank-1 linear concept eraser (LEACE; [4]) makes linearly undecodable from block 1’s mean-pooled output (), but it is decodable again from the residual stream immediately after block 2’s attention (, against a clean 0.912). The erasure does reduce redshift decodability downstream, however: with the linear spatial extent direction removed at block 1, the post-encoder redshift probe falls from to 0.865, a loss of 0.054 against 0.011 for a shuffled-label control eraser, so the spatial extent representation that remains decodable downstream is not a complete substitute for the erased one (Appendix § 10.§ 4). Spatial extent is therefore used but not localized: no single head is necessary for it, and erasing its linear direction in block 1 does not remove it from the representation.
Dust attenuation is weakly decodable, yet the dust-graded advantage localizes to one head
Head 3 is necessary for the dust-graded multimodal advantage
Sec. 4.3.2 established that the head with the best-performing linear probe in block 1 has a negligible causal effect. We therefore searched block 1 by ranking all twelve heads by causal effect rather than by linear decodability: we zero-ablated each head’s output at every position and re-ran the trained redshift readout on the whole test set and within the most dust-attenuated quintile (Table D.2). Zero-ablating Head 3 gives overall, against for the next-largest head in the block. Head 3’s rank is robust to multiplicity correction, retraining the readout on different early-stopping criteria and data splits, and the choice of ablation (Appendix § 11.§ 1). The properties on which we stratify were fixed by the population analysis of Sec. 4.1 before any head was examined, so the stratified test is not a post-hoc search over subgroups.
From the least to the most dust-attenuated quintile, ablating Head 3 reduces in every quintile, with the largest effect in the most dust-attenuated one (; all values and intervals in Table E.1). This is where the multimodal advantage is concentrated: is about twice as large on the two most dust-attenuated quintiles () as on the two least (), with non-overlapping CIs, and ablating Head 3 removes most of it in every quintile (residual advantage between and ). In the most dust-attenuated quintile, removing this single head reduces the readout’s from 0.863 to 0.754, close to the value of 0.774.
Head 3 is also sufficient within block 1: when the other eleven block-1 heads are replaced by their per-position mean, so that Head 3 is the only block-1 head whose output depends on the input galaxy, the readout reaches against 0.931 for the readout, and the gap is smallest in the quintiles where the ablation effect is largest: Head 3 alone recovers at least 87% of the multimodal advantage in every quintile and 100% in the most dust-attenuated one (Figure 5a; mean-ablation protocol and per-quintile values, Appendix § 11.§ 1). Head 3 is therefore necessary for most of the advantage in every dust stratum and, among the block-1 heads, sufficient for nearly all of it.

Figure 5. (a) Redshift on the held-out test set, stratified by quintiles of , for (gold, dotted), (blue, solid), with the eleven other block-1 heads mean-ablated so Head 3 is the only active head (black, dash-dotted), and with Head 3’s output zeroed (purple, dashed). Mean ablation replaces a head’s output with its per-position mean over a random subsample of 5,000 training-split galaxies. Error bars are 95% paired bootstrap CIs. (b) Mean attention from Head 3, averaged over 200 held-out test galaxies, grouped as in Figure 4b.
Ablating Head 3 also changes the kind of error the model makes. We fit and call a slope attenuation bias: a statistical term for predictions compressed toward the mean redshift. Moran et al. [ [21] find that image-based photometric-redshift methods suffer less of it than photometry-based ones. Consistent with this, adding imaging reduces attenuation bias in the most dust-attenuated quintile, raising from 0.806 () to 0.867 (). Ablating Head 3 does not return the slope to its value but lowers it further, to 0.710, while the scatter about the fitted line stays slightly below that of (0.039 against 0.042; decomposition in Appendix § 11.§ 2). In this quintile, the reduction in attenuation bias that imaging provides therefore requires Head 3: with Head 3 ablated, is more attenuation-biased than .
Head 3’s computation remains unidentified
Head 3 performs cross-modal attention: its image queries place a mean of 24.9% of their attention on the nine photometry keys, concentrating on the , , flux tokens and the W1 and W2 infrared flux tokens (Figure 5b). We tested four families of candidate operations, drawn from that attention pattern and from the standard photometric-redshift toolbox, and found support for none of them. Protocols and full results are given in Appendices § 11.§ 3–§ 11.6.
Probes for expected physical parameters. If Head 3 encoded the physical parameters that make dust-attenuated galaxies hard to predict, probes on its output would recover them. It ranks in the middle of the twelve heads for redshift, last for , and first but weakly for ( linear, 0.397 MLP, against 0.589 at the post-encoder). Its dust-graded causal effect is not mirrored by decodability (Appendix § 11.§ 3).
Dereddening and break tracking. Among the photometry keys, every photometry query attends most to the flux token and (Figure 5b), suggesting deredding, which corrects fluxes for Milky Way dust, or tracking of the 4000 Å break, whose redshift-dependent position is the classic photometric-redshift signal. Neither is supported: Head 3’s output at the flux position does not vary with (Pearson correlation ), its attention does not shift from to with redshift, and it ranks seventh on the twelve heads on the redshift-corrected color, the break’s photometric signature (Appendix § 11.§ 4).
Image-token spatial encoding. If Head 3 encoded resolved spatial structure, its image-token output would track within-galaxy color or brightness maps and its ablation would matter most for galaxies with spatially variable color. Neither is supported (mean within-galaxy ; the ablation effect is smaller for such galaxies), and no spatial subset of image tokens carries more than 41.9% of the effect. Head 3’s effect is distributed across the image tokens rather than localized (Appendix § 11.§ 5).
Correction of photometry–image discrepancies. If Head 3’s effect came from correcting inconsistencies between the photometry-token and image-token representations, it would grow with , the discrepancy between the -band magnitudes decoded from the two. Instead, the loss from ablating Head 3 shrinks, from 0.066 in the lowest- quintile to 0.031 in the highest, and is uncorrelated with (Appendix § 11.6).
Because each test presupposes a candidate operation, we also asked whether Head 3’s output contains the redshift information that distinguishes from , regardless of what it encodes: a redshift probe on the representation concatenated with Head 3’s pooled output closes only 5% of the gap between the and redshift probes, and a token-resolved probe over the 576 image tokens closes only 10% (Appendix § 11.7). The information that distinguishes the two representations therefore cannot be extracted from Head 3’s output by any pooled or token-resolved probe we fit.
No comparable head or dominant relay among the heads tested accounts for the effect
Since the search was restricted to the twelve heads of block 1, two alternatives remain open: that the computation is shared with heads elsewhere in the encoder, or that Head 3’s output is merely the entry point of a circuit carried out downstream. We screened all 144 encoder heads with a gradientactivation attribution and zero-ablated the five heads outside block 1 that rank highest on the most dust-attenuated galaxies, four of which exceed our screening threshold in that quintile; we call these the confirmed heads (Appendix § 11.8). All five have effects that are statistically significant but small over the full test set, the largest being against Head 3’s .
We then tested whether the four confirmed heads mediate Head 3’s effect, using two interventions (Appendix § 11.8). Single-edge path patching recomputes one receiving head’s queries, keys, and values from a residual stream with Head 3’s output removed, while every other head reads the clean residual stream. Cutting the edge into any one confirmed head accounts for at most 7% of Head 3’s , against at most 0.2% for control heads. Because path patching cuts one edge at a time, we also zero-ablated Head 3 while fixing all four confirmed heads to their clean-run outputs, so that the ablation cannot propagate through them. In the most dust-attenuated quintile, freezing them removes only 3% of the ablation effect, comparable in magnitude to freezing four control heads. No dominant relay therefore runs through these heads. Because the search covered attention heads only, contributions through the MLPs, through many weak edges jointly, or directly through the residual stream are not excluded, and the mechanism remains unidentified.
Discussion
To our knowledge, this is the first study to apply causal mechanistic interpretability methods to an astronomy foundation model. The multimodal advantage over photometry alone is concentrated in spatially extended and dust-attenuated galaxies and mostly enters in AION-1’s first encoder block, where linear decodability and causal importance dissociate in two opposite ways. Spatial extent is causally used but not localized: the head that decodes it best has a negligible causal effect, and the property remains decodable after ablation of any single head and after erasure of its linear direction. A strong probe therefore does not identify the causally relevant component, a null ablation does not establish that a property is unused, and methods that assume a property is carried by a single head or linear direction have no such target here. Interpreting scientific foundation models will require methods that characterize distributed representations of this kind rather than only localize them.
Dust attenuation is the opposite case: a single head is necessary for most of the dust-graded advantage and, among its block’s heads, sufficient for nearly all of it, yet the dust parameters are only weakly decodable from its output, and none of the astrophysically motivated operations we tested predicts its behavior. Here the computation is localized, and the open problem is characterizing the function the head implements. Because our circuit search covered attention heads only and our hypothesis tests are limited to the operations we could enumerate, we cannot tell whether the head computes something that matches no known astrophysically motivated operation or something our methods cannot recover. Distinguishing them requires a method that extracts the function a component computes in a form a domain scientist can evaluate, rather than testing candidate operations one at a time.
Connecting model internals to domain knowledge is, we believe, a prerequisite for extracting scientific hypotheses from these models, and astronomy is a useful setting in which to build it because the domain supplies a ground truth most interpretability settings lack: whether a model implements the operation an astrophysicist would expect has a determinate answer, and either answer is informative.
Scope and limitations. The single-model, single-task focus of our paper is deliberate. Mechanistic interpretability of scientific foundation models is, we argue, most tractable when anchored to a specific downstream task: the task provides an output whose change measures each component’s causal effect, and the domain hypotheses to test against. The task then fixes the remaining choices: redshift prediction requires galaxies with imaging, photometry, and spectroscopy, so we use the largest such sample among AION-1’s training surveys, which may affect absolute accuracy but not our claims about how the frozen encoder computes the gain. Comparison across astronomical foundation models is a natural extension but not an architecturally direct one, since contrastive models such as AstroCLIP and AstroM3 couple modalities only through the training objective. Our results are therefore for one model and dataset; how they change with scale or survey is left for future work.
Acknowledgments and Disclosure of Funding
R.C. Zhang is supported by a Gates Cambridge Scholarship from the Gates Cambridge Trust. A. Favero acknowledges support from the Infosys-Cambridge AI Centre. M. Cranmer is grateful for support from the Schmidt Sciences AI2050 Early Career Fellowship and the Isaac Newton Trust. Polymathic AI acknowledges support provided by the Simons Foundation and Schmidt Sciences, LLC.
Data. This research used data obtained with the Dark Energy Spectroscopic Instrument (DESI). DESI construction and operations is managed by the Lawrence Berkeley National Laboratory. This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of High-Energy Physics, under Contract No. DE–AC02–05CH11231, and by the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility under the same contract. Additional support for DESI was provided by the U.S. National Science Foundation (NSF), Division of Astronomical Sciences under Contract No. AST-0950945 to the NSF’s National Optical-Infrared Astronomy Research Laboratory; the Science and Technology Facilities Council of the United Kingdom; the Gordon and Betty Moore Foundation; the Heising-Simons Foundation; the French Alternative Energies and Atomic Energy Commission (CEA); the National Council of Humanities, Science and Technology of Mexico (CONAHCYT); the Ministry of Science and Innovation of Spain (MICINN), and by the DESI Member Institutions: www.desi.lbl.gov/collaborating-institutions. The DESI collaboration is honored to be permitted to conduct scientific research on I’oligam Du’ag (Kitt Peak), a mountain with particular significance to the Tohono O’odham Nation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the U.S. National Science Foundation, the U.S. Department of Energy, or any of the listed funding agencies.
The Legacy Surveys consist of three individual and complementary projects: the Dark Energy Camera Legacy Survey (DECaLS; Proposal ID #2014B-0404; PIs: David Schlegel and Arjun Dey), the Beijing-Arizona Sky Survey (BASS; NOAO Prop. ID #2015A-0801; PIs: Zhou Xu and Xiaohui Fan), and the Mayall z-band Legacy Survey (MzLS; Prop. ID #2016A-0453; PI: Arjun Dey). DECaLS, BASS and MzLS together include data obtained, respectively, at the Blanco telescope, Cerro Tololo Inter-American Observatory, NSF’s NOIRLab; the Bok telescope, Steward Observatory, University of Arizona; and the Mayall telescope, Kitt Peak National Observatory, NOIRLab. Pipeline processing and analyses of the data were supported by NOIRLab and the Lawrence Berkeley National Laboratory (LBNL). The Legacy Surveys project is honored to be permitted to conduct astronomical research on I’oligam Du’ag (Kitt Peak), a mountain with particular significance to the Tohono O’odham Nation. NOIRLab is operated by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with the National Science Foundation. LBNL is managed by the Regents of the University of California under contract to the U.S. Department of Energy. This project used data obtained with the Dark Energy Camera (DECam), which was constructed by the Dark Energy Survey (DES) collaboration. Funding for the DES Projects has been provided by the U.S. Department of Energy, the U.S. National Science Foundation, the Ministry of Science and Education of Spain, the Science and Technology Facilities Council of the United Kingdom, the Higher Education Funding Council for England, the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign, the Kavli Institute of Cosmological Physics at the University of Chicago, Center for Cosmology and Astro-Particle Physics at the Ohio State University, the Mitchell Institute for Fundamental Physics and Astronomy at Texas A&M University, Financiadora de Estudos e Projetos, Fundacao Carlos Chagas Filho de Amparo, Financiadora de Estudos e Projetos, Fundacao Carlos Chagas Filho de Amparo a Pesquisa do Estado do Rio de Janeiro, Conselho Nacional de Desenvolvimento Cientifico e Tecnologico and the Ministerio da Ciencia, Tecnologia e Inovacao, the Deutsche Forschungsgemeinschaft and the Collaborating Institutions in the Dark Energy Survey. The Collaborating Institutions are Argonne National Laboratory, the University of California at Santa Cruz, the University of Cambridge, Centro de Investigaciones Energeticas, Medioambientales y Tecnologicas-Madrid, the University of Chicago, University College London, the DES-Brazil Consortium, the University of Edinburgh, the Eidgenossische Technische Hochschule (ETH) Zurich, Fermi National Accelerator Laboratory, the University of Illinois at Urbana-Champaign, the Institut de Ciencies de l’Espai (IEEC/CSIC), the Institut de Fisica d’Altes Energies, Lawrence Berkeley National Laboratory, the Ludwig Maximilians Universitat Munchen and the associated Excellence Cluster Universe, the University of Michigan, NSF’s NOIRLab, the University of Nottingham, the Ohio State University, the University of Pennsylvania, the University of Portsmouth, SLAC National Accelerator Laboratory, Stanford University, the University of Sussex, and Texas A&M University. BASS is a key project of the Telescope Access Program (TAP), which has been funded by the National Astronomical Observatories of China, the Chinese Academy of Sciences (the Strategic Priority Research Program “The Emergence of Cosmological Structures” Grant # XDB09000000), and the Special Fund for Astronomy from the Ministry of Finance. The BASS is also supported by the External Cooperation Program of Chinese Academy of Sciences (Grant # 114A11KYSB20160057), and Chinese National Natural Science Foundation (Grant # 12120101003, # 11433005). The Legacy Survey team makes use of data products from the Near-Earth Object Wide-field Infrared Survey Explorer (NEOWISE), which is a project of the Jet Propulsion Laboratory/California Institute of Technology. NEOWISE is funded by the National Aeronautics and Space Administration. The Legacy Surveys imaging of the DESI footprint is supported by the Director, Office of Science, Office of High Energy Physics of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231, by the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility under the same contract; and by the U.S. National Science Foundation, Division of Astronomical Sciences under Contract No. AST-0950945 to NOAO.
Readout architecture, training, data split, and confidence intervals
Architecture. The readout is a cross-attention pooling layer followed by a two-layer MLP, trained on the output of the frozen AION-1 encoder. The pooling layer is a NormCrossAttention block (12 heads, , no QKV or projection bias, no attention dropout, per-head query-key normalization) whose single learnable query of dimension 768 aggregates the encoder tokens into a single 768-dimensional pooled vector. The MLP maps with a GELU activation and produces a scalar redshift estimate. Under , the encoder receives only the nine photometry tokens and the readout pools over the nine corresponding encoder outputs; under the encoder additionally receives the 576 image tokens produced by the image codec, and the readout pools over all 585. Targets are PROVABGS spectroscopic redshifts (provabgs_desi_edr_sv3/Z_HP) standardized by the training-split mean and standard deviation, and the training loss is the mean-squared error on the standardized target.
Inputs. Photometry inputs are the nine Legacy Survey scalar tokens used during AION-1 pretraining: the , , , , W1, W2, W3, and W4 fluxes and the foreground extinction . Image inputs are four-band Legacy Survey cutouts of size pixels, passed to AION-1 with the band labels DES-G, DES-R, DES-I, and DES-Z. AION-1’s image codec first center-crops each cutout to pixels, clamps pixel values to a fixed range, rescales them to the Legacy Survey flux convention, applies an arcsinh range compression (compression factor 0.01, post-multiplication factor 10), and pads the four input bands into AION-1’s multi-survey band-slot layout. The result is passed through the MagVitAE convolutional autoencoder (hidden_dims = 512, n_compressions = 2, num_consecutive = 4, finite-scalar quantization), whose two consecutive spatial downsampling stages yield a -token map per galaxy.
Training. Both the and readouts are trained from random initialization for up to 200 epochs with AdamW (learning rate , weight decay 0.05) and batch size 32. The learning-rate schedule is a 5-epoch linear warmup from 1% of the peak learning rate up to , followed by cosine annealing over the remaining 195 epochs to a floor of . Gradient norms are clipped to . Training images are augmented with random horizontal and vertical flips at every batch, and no augmentation is applied at evaluation. Optimization is performed only on the cross-attention-pooling and MLP parameters; the AION-1 encoder and modality codecs are kept in evaluation mode with requires_grad = False and produce no gradients. We early-stop on the held-out test-set with a patience of 15 epochs without improvement and restore the best- checkpoint for all subsequent analyses. Because early stopping on the test set uses the same galaxies we report results on, we also retrained both readouts with early stopping on a validation set drawn from the training split (20% of the training split, random_state = 7), once on the seed-42 partition used throughout the paper and once on a second partition drawn with seed 123. The results are stable. On the seed-42 partition, the retrained readouts reach test-set [0.863, 0.877] for and 0.930 [0.924, 0.936] for , against 0.869 [0.861, 0.877] and 0.931 [0.924, 0.936] for the originals; on the seed-123 partition, they reach 0.867 [0.859, 0.875] and 0.927 [0.920, 0.934]; and the twelve-head ablation ranking of Sec. 4.4.1 is reproduced on both partitions (Appendix E.1). We therefore use the readouts trained with test-set early stopping throughout the paper.
Data split. The training and held-out splits used throughout the paper consist of 54,204 and 13,551 galaxies respectively, obtained by an 80/20 random partition (sklearn.model_selection.train_test_split, random_state = 42) of the 67,755 DESI EDR BGS-bright galaxies that pass the quality cuts of Sec. 3.2, namely , , , positive PROVABGS optical and W1 magnitudes, and finite Legacy Survey fluxes.
Bootstrap confidence intervals. Unless stated otherwise, every bracketed interval and error bar in this paper is a 95% percentile bootstrap interval, the 2.5th–97.5th percentile range of the bootstrap distribution, computed from resamples. For readout values and their differences, whether between two readouts or between a readout with and without intervention, each resample draws the relevant galaxies (the whole test set, or a subset of it such as one quintile) with replacement, and every readout and intervention being compared is evaluated on that same resampled set. This pairing matters because galaxies that are hard for one readout tend to be hard for the others, so their per-galaxy errors are positively correlated, and resampling each one separately would overstate the uncertainty on their differences. The correlation is strongest between a readout and an intervention on that same readout, which is why an interval on is narrower than the intervals on the two values it is formed from. Intervals on probe are computed for each probe separately, with the probe held fixed and its held-out galaxies resampled. All intervals reflect only sampling variation across galaxies, not variation from retraining a readout or probe.
Independent contributions of and to the multimodal advantage
The fiber magnitude and the dust attenuation could in principle reflect a single underlying mechanism, since dust attenuation dims a galaxy and would raise along with . Three statistical analyses indicate that they make independent contributions to the multimodal accuracy gain. All three treat a galaxy as “correct” under each model when .
Marginal correlation. On the 13,198 galaxies in the Ph+Im better and Both correct populations of § 4.1 combined, the Spearman rank correlation between and is weak, (scipy.stats.spearmanr).
Joint logistic regression. We ask whether each of and predicts whether a galaxy is Ph+Im better after controlling for the other and for three further properties. The sample is the same 13,198 galaxies, with binary label for Ph+Im better and for Both correct. The feature vector consists of , , , , and , each standardized to zero mean and unit variance over the sample. We fit
by maximum likelihood without regularization (sklearn.linear_model.LogisticRegression with C = np.inf, L-BFGS solver). Standard errors (SE) are the square roots of the diagonal of the inverse observed Fisher information , where is the design matrix with an intercept column, , and is the fitted probability for galaxy ; 95% Wald CIs are . Because the features are standardized, is the change in log-odds of being Ph+Im better per one standard deviation of feature with the other four held fixed. Both coefficients of interest are positive with CIs excluding zero: and . The five-feature model reaches an in-sample AUC of 0.666. Each property therefore predicts whether a galaxy is Ph+Im better independently of the other and of the three controls.
Interaction test. On the full held-out test set (; both and are available for every test galaxy, so no galaxies are dropped), we split galaxies at the medians of the two properties, and , and compute the accuracy gain in each of the four cells, where and are the fractions of the cell’s galaxies predicted correctly by the Ph+Im and Ph readouts, respectively (Table B.1). The gain increases along both axes: +2.2% in the (low , low ) cell, +4.7% when alone is high, +5.1% when alone is high, and +7.8% when both are high. To test whether the two properties interact, we define the indicators and , and let if the Ph+Im readout predicts the galaxy correctly, so that is the mean of within a cell. We fit
by unregularized maximum likelihood with the same Wald standard errors as above. If the two properties reflected a single mechanism, their effects would not combine independently, giving . The interaction coefficient is instead consistent with zero, (95% Wald CI; , two-sided Wald test), so each property raises the log-odds of a correct Ph+Im prediction by an amount that does not depend on whether the other is high. The gains themselves are close to additive: relative to the (low, low) cell, high alone adds 2.5 percentage points and high alone adds 2.9, a sum of 5.4, against 5.6 when both are high (Table B.1).
| low | high | |
| low | +2.2% 0.7% () | +5.1% 1.1% () |
| high | +4.7% 1.1% () | +7.8% 1.2% () |
Table B.1. Multimodal accuracy gain on the full held-out test set (), where and are the fractions of galaxies in a cell predicted correctly by each readout, stratified by median splits of (median 20.23) and (median 0.23). The 95% CIs are computed within each cell as .
Layerwise probing and block-resolved causal localization
This appendix supports § 4.2. Appendix C.1 gives the layerwise linear-probe protocol behind Figure 3 and the complete probe results (Fig. C.1). Appendix C.2 gives the block-resolved activation patching that confirms the block-1 localization by causal intervention.
Layerwise linear probing
Methodology. For each input configuration ( and ) and each of the 14 probed encoder stages, namely the token embedding (the encoder input, i.e. the output of aion_model.embed_inputs), the output of each of the 12 transformer blocks, and the output of the final encoder layer-norm, we run the frozen AION-1 encoder forward on both the 54,204-galaxy training split, on which each probe is fit, and the 13,551-galaxy held-out split, on which it is evaluated (§ 3.2). Per-block and post-encoder-norm activations are captured by registering forward hooks on aion_model.encoder[i] for and on aion_model.encoder_norm; the token-embedding stage is captured directly as the output of aion_model.embed_inputs. At each stage, the per-token 768-dimensional activations are mean-pooled along the token axis to a single 768-dimensional vector per galaxy; at the token-embedding stage, where the sequence is padded to a fixed length, the encoder attention mask restricts the mean to the real tokens. For each (stage, property) pair we drop activation features whose training-split standard deviation is below (near-constant features, which standardization cannot scale), standardize the remainder with sklearn.preprocessing.StandardScaler fit on the training split, and fit a ridge probe via sklearn.linear_model.RidgeCV with a 13-point logarithmic grid on and scikit-learn’s closed-form efficient leave-one-out cross-validation (cv=None). Probes are restricted per target to galaxies on which the property is finite. Reported values are computed by sklearn.metrics.r2_score on the held-out split.
Block structure and the attention/MLP split. Each AION-1 encoder block is a standard pre-norm transformer block, followed by , so the MLP sub-layer acts on the residual stream after the attention output has been added, and each block- stage above is the residual stream after both sub-layers. Probing the stream between the two sub-layers of block 1 as well places the block-1 redshift gain under mainly in the attention sub-layer: rises from 0.72 at the token embedding to 0.79 after block-1 attention and to 0.82 after its MLP.
Final-stage decodability. The columns of Fig. C.1 fall into three families, and the post-encoder (last) row summarizes how linearly decodable each is at the end of the encoder. First, the properties passed into the model directly as photometry tokens are recovered almost perfectly under both input configurations: the broadband magnitudes , , reach , and reaches 0.96 () and 0.95 (). Second, properties that are not photometry tokens but can be constrained by them, redshift, stellar mass, and , are decodable from the post-encoder representation under (, respectively) and improve under (, respectively); these are the properties whose decodability the image modality improves. Third, the dust parameters , , and are neither photometry tokens nor well constrained by them and remain weakly decodable at the post-encoder in both configurations (, respectively, under ; , respectively, under ).
The token embedding is already informative. Before any transformer block has run, the token embedding produced by AION-1’s per-modality codecs and input embeddings is already an approximately linear encoding of the inputs, and several galaxy properties are linearly decodable from it. At the token-embedding stage (Fig. C.1, top rows), , , and are decodable at under both input configurations: redshift at () and 0.72 (), at 0.66 () and 0.70 (), and at 0.61 () and 0.72 ().
Gradual increase in decodability after block 1. For properties whose decodability is not saturated at the token embedding, increases with depth after the first encoder block (Fig. C.1). Under , increases from 0.82 at the output of block 1 to 0.92 at the post-encoder, from 0.80 to 0.89, from 0.45 to 0.59, from 0.24 to 0.32, and from 0.49 to 0.58, with no single block contributing more than 0.03.
Figure C.1. Layerwise linear-probe of the 10 galaxy properties of Figure 3 (columns) at all 14 encoder stages (rows), namely the token embedding (“Token Emb”), the outputs of each of the 12 transformer blocks, and the post-encoder representation. Columns are grouped (vertical lines) into three families: properties determined directly by the photometry tokens (, , , ); properties whose decodability the configuration improves (redshift , stellar mass , fiber -magnitude ); and the weakly decodable dust-related parameters (, , ). Top: ; bottom: .
Block-resolved activation patching
Clean and corrupted runs. Block-resolved activation patching compares a clean forward pass against a corrupted one in which each test galaxy keeps its own photometry tokens but its image tokens are replaced with those of another test galaxy. The replacement is a one-position circular shift of the randomly ordered test set: each galaxy receives the image tokens of the preceding galaxy and the first galaxy receives those of the last, so no galaxy keeps its own image. The same assignment is used for both interventions below. Using another galaxy’s image rather than zeros or a mean image preserves the marginal distribution of the image input [Wang et al., 2023], while the joint photometry–image distribution is not preserved. On the full test set, the two reference values are (clean) and (corrupted). On the most spatially extended quartile, defined throughout as the top quartile of (, ), they are 0.936 and . Both corrupted values are far below the . The readout does not revert to a photometry-only prediction when the image is replaced: its prediction depends on the substituted image.
Block contributions and the two interventions. The clean and corrupted runs are the two reference runs for this test, which asks whether a single block’s contribution is sufficient to restore the clean or necessary to maintain it. We construct hybrid forward passes that follow one reference run except at a single block, whose contribution is taken from the other run. Write block ’s contribution to the residual stream as , where and are the block’s input and output. In denoising (clean corrupted patching, a test of sufficiency; [10]) we run the corrupted pass and overwrite block ’s output with , injecting the block’s clean contribution, recorded from the clean run, onto the corrupted stream. In noising (corrupted clean patching, a test of necessity) we run the clean pass and overwrite block ’s output with , using the contribution recorded from the corrupted run. In both cases blocks after compute normally on the modified stream, and the trained redshift readout is re-run unchanged. Each intervention is applied to one block at a time. We report the restored fraction and the removed fraction . Both fractions are normalized by the clean-minus-corrupted contrast , not by the multimodal advantage ; means one block’s clean contribution is sufficient to recover the full contrast and that corrupting it alone is sufficient to remove it. Intervals are galaxy-paired percentile bootstraps as in Appendix A.
Table C.1: Per-block noising and denoising results on the held-out test set. is the fraction of removed by noising block alone; is the fraction restored by denoising block alone.
| Block | removed | restored |
| 1 | 0.987 [0.987, 0.988] | 0.9999 [0.9998, 1.0000] |
| 2 | 0.036 [0.033, 0.039] | 0.539 [0.529, 0.549] |
| 3 | 0.015 [0.014, 0.017] | 0.433 [0.424, 0.442] |
| 4 | 0.010 [0.009, 0.011] | 0.316 [0.308, 0.325] |
| 5 | 0.012 [0.011, 0.014] | 0.261 [0.253, 0.270] |
| 6 | 0.007 [0.006, 0.008] | 0.162 [0.155, 0.170] |
| 7 | 0.008 [0.007, 0.010] | 0.201 [0.192, 0.209] |
| 8 | 0.015 [0.013, 0.016] | 0.266 [0.258, 0.274] |
| 9 | 0.024 [0.022, 0.026] | 0.340 [0.331, 0.349] |
| 10 | 0.015 [0.013, 0.016] | 0.268 [0.261, 0.277] |
| 11 | 0.013 [0.011, 0.015] | 0.256 [0.249, 0.263] |
| 12 | 0.023 [0.021, 0.025] | 0.324 [0.317, 0.331] |
Table C.1.
Per-block results. Table C.1 reports both fractions for every block. Denoising and noising give the same result for block 1 and different results for later blocks. Block 1 is both sufficient ( [0.9998, 1.0000], and 0.9998 [0.9997, 0.9999] on the most spatially extended quartile) and necessary (), so the probe-based localization of § 4.2 is also supported by intervention. Noising a later block removes little of the contrast ( for ). Measured against the multimodal advantage, however, these decreases are not negligible: noising block 2 alone lowers from 0.931 to 0.892, a decrease equal to 62% of the advantage, and noising any one of blocks 3–12 gives a decrease equal to 12–42% of it. Denoising a later block restores more of the contrast (–0.54 for ) but adds below the value of 0.869 in every case, so no later block alone recovers any of the multimodal advantage. Later blocks therefore make non-zero contributions, but block 1 is the only block whose contribution is on its own both necessary and sufficient.
Spatial extent experiments
This appendix contains the methods and full results behind § 4.3. Appendix § 10.§ 1 defines the image counterfactuals. Appendix § 10.§ 2 reports the head-wise linear probes, zero-ablation, and attention and image-token analyses for the twelve block-1 heads. Appendix § 10.§ 3 describes the matched-redshift activation patching on Head 1, and Appendix § 10.§ 4 the linear erasure of spatial extent from the residual stream.
Image counterfactuals
Construction. Each counterfactual replaces the image in each of the four bands (, , , ) for every test galaxy and leaves the photometry tokens unchanged, so the input differs from the clean input only in the image tokens. The compact template is a two-dimensional Gaussian with pixels, approximating a point source, centered on the frame at pixel coordinates . The extended blur is the original band image convolved with an isotropic two-dimensional Gaussian kernel with pixels, with zero padding outside the frame.
Flux matching. The image codec reads only the central pixels of the frame. On the test set, a median 76% of a cutout’s -band flux lies inside this crop (5th percentile 18%), so matching flux over the full frame would not match the flux the codec receives: a compact template matched over the full frame has a median 1.3 times the original crop flux (95th percentile 5.4 times). We therefore rescale each counterfactual, per band, so that its flux inside the crop equals that of the original image. Table D.1 reports the per-quintile under both counterfactuals, alongside the clean and values. If each counterfactual is instead rescaled so that its total flux over the full frame equals that of the original image, differs from the values in Table D.1 by at most 0.022 in every quintile, and the trends across quintiles are unchanged.
| quintile | clean | Compact template | Extended blur | |
| Q1 [0.271, 1.076] | 0.815 [0.788, 0.837] | 0.906 [0.892, 0.918] | 0.671 [0.645, 0.696] | 0.540 [0.496, 0.579] |
| Q2 [1.076, 1.334] | 0.875 [0.858, 0.889] | 0.922 [0.908, 0.933] | 0.620 [0.593, 0.646] | 0.738 [0.709, 0.765] |
| Q3 [1.334, 1.614] | 0.880 [0.858, 0.897] | 0.922 [0.901, 0.938] | 0.561 [0.531, 0.588] | 0.758 [0.722, 0.788] |
| Q4 [1.614, 2.006] | 0.859 [0.838, 0.876] | 0.923 [0.908, 0.936] | 0.458 [0.430, 0.482] | 0.788 [0.763, 0.808] |
| Q5 [2.006, 6.614] | 0.835 [0.816, 0.852] | 0.934 [0.916, 0.948] | 0.309 [0.272, 0.345] | 0.793 [0.765, 0.818] |
Table D.1. Redshift on the held-out test set per quintile, for the readout and for the readout with the original image (clean), the compact template, and the extended blur. increases with the fraction of -band light outside the 1.5″ DESI fiber, so larger values correspond to more spatially extended galaxies. Intervals are 95% paired bootstrap intervals (Appendix § 7). These are the values plotted in Figure 4a.
Per-head analyses of block 1
Linear probes on individual heads. We fit a linear probe to the output of each attention head of block 1 separately. Heads are numbered 1 to 12, corresponding to head indices 0 to 11 of aion_model.encoder[0].attn. The output of a head is , the product of its post-softmax attention weights and its value vectors . The twelve head outputs are concatenated along the feature dimension, with Head occupying features through , and the output projection aion_model.encoder[0].attn.proj maps this 768-dimensional concatenation back to the residual stream. We use directly, before concatenation and projection. For the configuration, a head’s output is a matrix per galaxy (9 photometry and 576 image tokens), which we mean-pool over tokens to a 64-dimensional feature vector. Standardization on the training split, the ridge probe and its regularization grid, the restriction to galaxies with a finite target, and evaluation on the 13,551-galaxy held-out split are as in Appendix § 9.§ 1.
Column (A) of Table D.2 reports the probe of all twelve heads on eight targets: the three properties whose decodability the image modality substantially improves in Figure 3 (, , ), the spatial extent proxy , three standard photometric-redshift indicators (the colors and and the -band surface brightness ), and the dust attenuation . The surface brightness is available only for a Galaxy Zoo-matched subset, so its probes are fit and evaluated on that subset (4,679 training and 1,183 held-out galaxies). Intervals on individual probe values resample the held-out galaxies with the probe held fixed (Appendix A). Head 1 ranks first on seven of the eight targets, all except , which Head 3 leads. For each of these seven targets, Head 1’s margin over the second-ranked head, with a 95% paired bootstrap interval that resamples the same galaxies for both heads, ranges from 0.051 [0.038, 0.064] on log to 0.251 [0.236, 0.264] on , and every interval excludes zero. The second-ranked head varies across these seven targets (Heads 3, 5, 7, and 9), so no other head is consistently second.
Table D.2: Linear probing and zero-ablation of the twelve block-1 heads with the configuration. (A) Probe on each head’s mean-pooled output for eight targets, with 95% bootstrap intervals. (B) Change in the redshift of the readout () when the head’s output is zeroed, on the full held-out test set and in the most dust-attenuated quintile, with 95% paired bootstrap intervals. Bold marks the highest in each column of (A) and the largest decrease in each column of (B).
| (A) Probe on head output | ||||||||
| Block-1 head | ||||||||
| Head 1 | 0.697 [0.687, 0.707] | 0.879 [0.866, 0.892] | 0.767 [0.750, 0.783] | 0.817 [0.769, 0.854] | 0.929 [0.924, 0.934] | 0.878 [0.869, 0.886] | 0.646 [0.633, 0.658] | 0.193 [0.180, 0.206] |
| Head 2 | 0.475 [0.462, 0.488] | 0.493 [0.474, 0.512] | 0.602 [0.582, 0.620] | 0.466 [0.408, 0.520] | 0.583 [0.571, 0.595] | 0.523 [0.510, 0.537] | 0.420 [0.403, 0.437] | 0.105 [0.094, 0.117] |
| Head 3 | 0.570 [0.558, 0.583] | 0.629 [0.615, 0.642] | 0.448 [0.428, 0.467] | 0.457 [0.399, 0.508] | 0.639 [0.628, 0.649] | 0.673 [0.661, 0.684] | 0.587 [0.572, 0.602] | 0.264 [0.246, 0.280] |
| Head 4 | 0.514 [0.501, 0.526] | 0.627 [0.611, 0.641] | 0.548 [0.527, 0.568] | 0.536 [0.485, 0.581] | 0.538 [0.525, 0.550] | 0.528 [0.514, 0.541] | 0.470 [0.453, 0.487] | 0.183 [0.168, 0.197] |
| Head 5 | 0.591 [0.580, 0.602] | 0.546 [0.527, 0.565] | 0.643 [0.623, 0.662] | 0.509 [0.460, 0.553] | 0.798 [0.791, 0.805] | 0.715 [0.704, 0.726] | 0.521 [0.506, 0.536] | 0.115 [0.105, 0.125] |
| Head 6 | 0.566 [0.554, 0.578] | 0.608 [0.591, 0.624] | 0.508 [0.486, 0.528] | 0.542 [0.493, 0.587] | 0.690 [0.679, 0.700] | 0.661 [0.648, 0.673] | 0.561 [0.545, 0.577] | 0.200 [0.185, 0.214] |
| Head 7 | 0.594 [0.581, 0.606] | 0.602 [0.587, 0.616] | 0.471 [0.450, 0.490] | 0.454 [0.402, 0.502] | 0.709 [0.699, 0.718] | 0.704 [0.692, 0.715] | 0.595 [0.579, 0.610] | 0.242 [0.226, 0.259] |
| Head 8 | 0.570 [0.558, 0.582] | 0.621 [0.607, 0.635] | 0.499 [0.480, 0.519] | 0.511 [0.455, 0.560] | 0.654 [0.644, 0.663] | 0.674 [0.663, 0.685] | 0.586 [0.571, 0.601] | 0.256 [0.240, 0.272] |
| Head 9 | 0.585 [0.573, 0.597] | 0.611 [0.596, 0.625] | 0.616 [0.595, 0.635] | 0.532 [0.455, 0.560] | 0.714 [0.703, 0.723] | 0.661 [0.649, 0.673] | 0.566 [0.550, 0.582] | 0.175 [0.162, 0.188] |
| Head 10 | 0.541 [0.529, 0.554] | 0.575 [0.559, 0.591] | 0.585 [0.567, 0.602] | 0.386 [0.296, 0.441] | 0.589 [0.577, 0.600] | 0.595 [0.580, 0.608] | 0.513 [0.496, 0.529] | 0.183 [0.168, 0.197] |
| Head 11 | 0.521 [0.508, 0.533] | 0.599 [0.584, 0.613] | 0.460 [0.440, 0.479] | 0.462 [0.410, 0.510] | 0.594 [0.582, 0.606] | 0.599 [0.586, 0.612] | 0.530 [0.513, 0.547] | 0.190 [0.175, 0.203] |
| Head 12 | 0.538 [0.525, 0.551] | 0.624 [0.610, 0.637] | 0.482 [0.463, 0.500] | 0.473 [0.418, 0.522] | 0.573 [0.561, 0.585] | 0.583 [0.569, 0.595] | 0.523 [0.506, 0.539] | 0.224 [0.208, 0.240] |
| (B) Ablation (), head zeroed | ||||||||
| Block-1 head | Full test set | Most dust-attenuated quintile | ||||||
| Head 1 | [, ] | [, ] | ||||||
| Head 2 | [, ] | [, ] | ||||||
| Head 3 | [, ] | [, ] | ||||||
| Head 4 | [, ] | [, ] | ||||||
| Head 5 | [, ] | [, ] | ||||||
| Head 6 | [, ] | [, ] | ||||||
| Head 7 | [, ] | [, ] | ||||||
| Head 8 | [, ] | [, ] | ||||||
| Head 9 | [, ] | [, ] | ||||||
| Head 10 | [, ] | [, ] | ||||||
| Head 11 | [, ] | [, ] | ||||||
| Head 12 | [, ] | [, ] |
Table D.2.
Zero-ablation of individual heads. We zero-ablate each attention head of block 1 separately and measure the resulting change in the redshift of the readout. To ablate Head , we set features through of the concatenated head output to zero at all 585 token positions, immediately before the output projection aion_model.encoder[0].attn.proj is applied. This removes the head’s additive contribution to the residual stream and leaves the other eleven heads of block 1 and the trained readout unchanged.
Column (B) of Table D.2 reports the resulting change in redshift , in units of , relative to the readout, which reaches [0.924, 0.936] on the full held-out test set, with 95% paired bootstrap intervals over resamples (Appendix § 7). Ablating Head 3 decreases by , every other head changes it by less than , and Head 1 changes it by [, ] . Columns (A) and (B) rank the heads differently: Head 1 ranks first on seven of the eight probe targets but seventh of twelve by ablation, while Head 3 ranks first on , sixth on , and last on among the probes, and first by ablation. The second ablation column, restricted to the most dust-attenuated quintile, is analyzed in Appendix § 11.§ 1.
Because Head 1 is the head whose output best encodes , we also evaluate its ablation within each quintile ( each), with 95% paired bootstrap intervals over resamples. From the least to the most spatially extended quintile, is [, ], [, ], [, ], [, ], and [, ], in units of . Every change is below in magnitude, and only the interval for the most spatially extended quintile excludes zero.
Attention map grouping and image-token PCA. The attention map of Figure 4b uses Head 1’s attention weights , and the principal-component analysis of Figure 4c uses its output (both defined above), each computed on a single forward pass over the first 200 galaxies of the held-out split. To assign each image token a brightness, we take the central pixels that the image codec tokenizes (Appendix § 7), average each -pixel block corresponding to one token, and average over the four bands. Tokens in the two outermost rows or columns of the grid (176 tokens) are labeled border; the remaining interior tokens are split within each galaxy at the 25th and 75th percentiles of that galaxy’s interior-token brightness distribution into faint, medium, and bright. With each of the 9 photometry tokens as its own group, this gives 13 groups. Attention is then averaged within each of the 13 query groups and 13 key groups, and across the 200 galaxies. Because the groups contain different numbers of tokens, each cell reports the mean attention placed on a single key token of its column group rather than a sum, so rows do not sum to one. Summed over all key tokens of each modality, Head 1’s image queries place a mean of 0.37% of their attention on the nine photometry keys ([10th, 90th] = [0.35%, 0.42%]) across galaxies, and its photometry queries place 90.4% ([90.1%, 90.8%]) on the 576 image keys. In the photometry-query rows of Figure 4b, each photometry key therefore receives more attention than any single image key, but the image keys together receive most of it.
For Figure 4c, we run a separate PCA within each galaxy across the 576 rows of Head 1’s (576, 64) image-token output, and correlate the per-token PC0 score with that token’s Euclidean distance from the center of the grid, taken as the continuous coordinate (11.5, 11.5). Across the 200 galaxies, the first component explains a mean of 84.1% [83.6%, 84.5%] of the variance (median 84.5%, [10th, 90th] = [80.4%, 87.6%]), and |corr(PC0, radius)| has mean 0.739 [0.737, 0.741], median 0.741, and a [10th, 90th] range of [0.719, 0.758].
Redundancy across heads. The probes in Figure 4d use the per-head mean-pooled outputs and the probe protocol above, with as the target, but concatenate the 64-dimensional outputs of several heads before the probe is fit. A joint probe on all twelve heads reaches [0.878, 0.902]. For each head, we also fit a leave-one-head-out probe on the concatenated outputs of the other eleven heads. Leaving out any head other than Head 1 reduces by at most 0.006 relative to the joint probe. Leaving out Head 1 reduces it by 0.120, to 0.771 [0.756, 0.786], which is comparable to the 0.767 [0.750, 0.783] of the probe on Head 1 alone. On their own, the twelve heads reach probe between 0.448 and 0.767 (Table D.2).
Matched-redshift activation patching on Head 1
Zero-ablation replaces Head 1’s output with values it never takes on real inputs. Activation patching instead replaces it with Head 1’s output on another galaxy, so the activations stay within the range the readout was trained on. If the readout used the spatial extent information in Head 1’s output, then giving a high- galaxy the Head 1 output of a low- galaxy at the same redshift should move its predicted redshift toward the low- galaxy’s prediction.
Pair construction. We take the lowest () and highest () quintiles of the held-out test set, 2,711 galaxies each, referred to as low- and high-. We write a pair as recipient←donor: in the high←low direction, each high- galaxy is a recipient that receives Head 1’s output from a low- donor, and the low←high direction reverses the roles. Within each direction, pairs are formed greedily over a random permutation of the recipients. For each recipient in turn, we step outward from its position in the list of donors sorted by , one position at a time on both sides, skipping donors already used, and stop at the first step at which an unused donor within has been found. Each donor is used at most once. Because donors are not evenly spaced in redshift, the chosen donor is the closest unused donor for 73% of pairs; otherwise, it is at most 0.002 farther in than the closest unused donor (median ). The two directions are matched independently, giving high←low and low←high pairs, with mean of and . The remaining 42% of recipients in each direction are unmatched; apart from 37 high←low recipients with no donor within tolerance, every unmatched recipient had donors within tolerance that had already been paired with earlier recipients.
Patching procedure. Patching intervenes at the same point as the zero-ablation of Appendix § 10.§ 2, features 1 through 64 of the concatenated head output before the output projection, but writes the donor’s values instead of zeros. For each pair, we run the frozen encoder on the donor and record Head 1’s output at all 585 token positions. We then run it on the recipient with Head 1’s output replaced by the donor’s at every position, leaving the other eleven heads unchanged, and apply the trained readout. The photometry positions are patched as well as the image positions because Head 1’s photometry queries place 90.4% of their attention on the image keys (Appendix § 10.§ 2), so its output at those positions also carries image information. As a check, patching each of the first 320 held-out galaxies with its own Head 1 output reproduces its unpatched prediction exactly.
Patching regression. For each pair, the patched shift is the change in the recipient’s predicted redshift, and the prediction gap is the difference between the donor’s and the recipient’s unpatched predictions. Within each direction, we regress on by ordinary least squares. A slope of 1 means that patching Head 1 alone moves the recipient’s prediction to the donor’s, and a slope of 0 means that the patched prediction does not move with the gap. Intervals are 95% percentile bootstrap intervals that resample the matched pairs of each direction with replacement, keeping each pair’s and together, with the pairing held fixed (Appendix § 7).
Result. The high←low direction gives a slope of 0.008 [0.004, 0.011] () and the low←high direction 0.003 [−0.001, 0.007] (). The mean patched shift is +0.0009 [0.0008, 0.0010] and −0.0004 [−0.0005, −0.0003], against root-mean-square prediction gaps of 0.035 and 0.036.
Erasing the linear spatial extent direction from the residual stream
Zero-ablating a head tests whether spatial extent is carried by that component; erasing a direction tests whether it is carried by a linear subspace of the residual stream. We use LEACE (least-squares concept erasure; [4]). For a scalar target, the LEACE eraser is the rank-1 affine map
where and are the mean and covariance of the activations and is their cross-covariance with the target. The erased activations have zero cross-covariance with the target, which implies that no linear predictor of the target from attains a lower mean squared error than a constant prediction. Among affine maps with this property, LEACE minimizes the expected squared Euclidean distance . Erasing therefore removes it from every linear decoder, not only from the probe of Appendix § 10.§ 2. We regularize by adding times its mean eigenvalue to the diagonal; the zero-cross-covariance property holds for any such regularization.
Fitting and applying the eraser. We estimate , , and on the mean-pooled block-1 output activations (Block 1 post-MLP in Table D.3) of the training split under the unmodified forward pass, with as the target. During the forward pass, is applied to the block-1 output at every token position, and the remaining blocks run unchanged. Because is affine, mean-pooling the erased token activations gives the same vector as erasing the mean-pooled activation (maximum absolute deviation ), so the guarantee above applies to the mean-pooled representation on which the probes are fit; it does not constrain the individual token activations.
Verification and control. On the training split, the largest absolute entry of the cross-covariance between the erased activations and is . On the held-out split, a probe refit on the erased block-1 output reaches , against before erasure. As a control, we fit the same eraser to a single random permutation of the labels, which removes a direction of the same rank that is not determined by ; the probe then reaches , so the loss of decodability is specific to the direction. With either eraser in place, we probe every subsequent stage of the encoder with the protocol of Appendix § 9.§ 1 (Table D.3). Intervals resample the held-out galaxies with each probe held fixed, and differences between conditions use paired resamples (Appendix § 7).
is decodable again downstream. Immediately after block 2’s attention sub-layer, the probe reaches on the erased stream against on the clean stream (paired decrease ), and at the post-encoder stage against . Within one attention sub-layer of the intervention, is again nearly as linearly decodable as without erasure, so the rank-1 projection does not remove the information from which a linear representation can be formed. These measurements concern decodability only; they do not establish whether the encoder recomputes the erased representation or forms a different one.
Redshift decodability recovers only partly. A redshift probe on the post-encoder representation reaches with the eraser in place, against on the clean stream and with the control eraser. The decrease of under erasure is larger than the control’s . Redshift decodability therefore recovers only partly after the erasure: the decrease is at block 1’s output and at the post-encoder stage.
Dust attenuation experiments
This appendix contains the methods and full results behind § 4.4. Appendix E.1 reports the ablation and sufficiency results for Head 3 and the robustness checks on the head ranking, and Appendix E.2 decomposes the prediction error into attenuation bias and scatter. Appendices E.3–E.6 evaluate four families of candidate operations for Head 3, and Appendix E.7 tests whether Head 3’s output closes the redshift probe gap. Appendix E.8 covers the model-wide head search and the downstream circuit analysis.
Ablation, sufficiency, and robustness of the head ranking
Per-head zero-ablation. Column (B) of Table D.2 reports the effect of zero-ablating each block-1 head (protocol in Appendix § 10.§ 2) as , the redshift with the head ablated minus that of the readout. On the full held-out test set, zero-ablating Head 3 gives , against for the next-largest effect (Head 11). On the most dust-attenuated quintile (), where the readout reaches , zero-ablating Head 3 gives , more than eleven times the next-largest effect ( , Head 11).
Per-quintile results. Table E.1 reports the redshift of the and readouts in each quintile, together with that of the readout when Head 3 is zero-ablated (Head 3 zeroed) and when the other eleven block-1 heads are mean-ablated so that Head 3 is the only active head in block 1 (Head 3 only). The column, Head 3 zeroed minus , has paired bootstrap intervals over 10,000 resamples of the galaxies within each quintile, and the other columns over 2,000; all five intervals exclude zero, and the interval for the most dust-attenuated quintile overlaps no other.
Mean ablation and the sufficiency test. Zero-ablation tests whether Head 3 is necessary for the multimodal advantage; the sufficiency test asks the converse, whether Head 3 alone among the block-1 heads is enough to reproduce the readout’s performance. We remove the other eleven heads
| Stage | probe clean | probe erased | probe shuffled | redshift probe clean | redshift probe erased | redshift probe shuffled |
| Token Emb | 0.577 [0.558, 0.594] | 0.577 [0.558, 0.594] | 0.577 [0.558, 0.594] | 0.722 [0.712, 0.731] | 0.722 [0.712, 0.731] | 0.722 [0.712, 0.731] |
| Block 1 post-attn | 0.892 [0.879, 0.903] | 0.892 [0.879, 0.903] | 0.892 [0.879, 0.903] | 0.795 [0.786, 0.803] | 0.795 [0.786, 0.803] | 0.795 [0.786, 0.803] |
| Block 1 post-MLP* | 0.903 [0.889, 0.914] | 0.000† [−0.000, −0.000] | 0.902 [0.888, 0.912] | 0.820 [0.812, 0.828] | 0.715 [0.704, 0.724] | 0.819 [0.811, 0.827] |
| Block 2 post-attn | 0.912 [0.898, 0.923] | 0.909 [0.895, 0.920] | 0.912 [0.898, 0.922] | 0.823 [0.815, 0.831] | 0.817 [0.809, 0.825] | 0.823 [0.815, 0.831] |
| Block 2 post-MLP | 0.915 [0.902, 0.926] | 0.910 [0.897, 0.921] | 0.914 [0.900, 0.924] | 0.836 [0.828, 0.843] | 0.817 [0.808, 0.824] | 0.830 [0.822, 0.837] |
| Block 3 post-attn | 0.917 [0.903, 0.928] | 0.911 [0.897, 0.921] | 0.916 [0.902, 0.927] | 0.840 [0.832, 0.848] | 0.821 [0.813, 0.829] | 0.837 [0.829, 0.844] |
| Block 3 post-MLP | 0.922 [0.908, 0.932] | 0.912 [0.898, 0.923] | 0.920 [0.906, 0.930] | 0.850 [0.842, 0.857] | 0.827 [0.819, 0.834] | 0.844 [0.836, 0.851] |
| Block 4 post-attn | 0.925 [0.912, 0.935] | 0.914 [0.901, 0.925] | 0.922 [0.909, 0.932] | 0.860 [0.853, 0.868] | 0.833 [0.825, 0.840] | 0.851 [0.844, 0.858] |
| Block 4 post-MLP | 0.928 [0.915, 0.938] | 0.916 [0.903, 0.926] | 0.927 [0.915, 0.937] | 0.866 [0.858, 0.873] | 0.837 [0.830, 0.845] | 0.857 [0.850, 0.865] |
| Block 5 post-attn | 0.930 [0.917, 0.940] | 0.917 [0.903, 0.927] | 0.929 [0.916, 0.938] | 0.880 [0.873, 0.886] | 0.846 [0.838, 0.853] | 0.873 [0.866, 0.880] |
| Block 5 post-MLP | 0.932 [0.919, 0.942] | 0.919 [0.905, 0.929] | 0.930 [0.917, 0.940] | 0.883 [0.876, 0.890] | 0.849 [0.841, 0.856] | 0.877 [0.869, 0.883] |
| Block 6 post-attn | 0.932 [0.919, 0.943] | 0.918 [0.904, 0.928] | 0.930 [0.917, 0.940] | 0.886 [0.879, 0.893] | 0.848 [0.841, 0.855] | 0.880 [0.873, 0.887] |
| Block 6 post-MLP | 0.934 [0.921, 0.944] | 0.918 [0.904, 0.928] | 0.931 [0.918, 0.942] | 0.890 [0.883, 0.897] | 0.852 [0.844, 0.859] | 0.884 [0.877, 0.890] |
| Block 7 post-attn | 0.935 [0.921, 0.944] | 0.916 [0.903, 0.927] | 0.931 [0.917, 0.941] | 0.891 [0.884, 0.898] | 0.851 [0.844, 0.858] | 0.885 [0.877, 0.891] |
| Block 7 post-MLP | 0.936 [0.923, 0.945] | 0.916 [0.902, 0.927] | 0.932 [0.919, 0.942] | 0.895 [0.888, 0.901] | 0.853 [0.846, 0.860] | 0.889 [0.882, 0.895] |
| Block 8 post-attn | 0.936 [0.922, 0.945] | 0.916 [0.902, 0.927] | 0.932 [0.918, 0.942] | 0.896 [0.889, 0.902] | 0.853 [0.846, 0.860] | 0.890 [0.883, 0.896] |
| Block 8 post-MLP | 0.936 [0.923, 0.946] | 0.916 [0.901, 0.926] | 0.932 [0.918, 0.943] | 0.901 [0.894, 0.907] | 0.857 [0.850, 0.864] | 0.894 [0.887, 0.900] |
| Block 9 post-attn | 0.936 [0.923, 0.946] | 0.916 [0.901, 0.927] | 0.932 [0.918, 0.943] | 0.904 [0.895, 0.908] | 0.857 [0.850, 0.864] | 0.894 [0.887, 0.901] |
| Block 9 post-MLP | 0.936 [0.923, 0.946] | 0.915 [0.901, 0.926] | 0.933 [0.919, 0.943] | 0.907 [0.900, 0.913] | 0.858 [0.851, 0.865] | 0.898 [0.891, 0.904] |
| Block 10 post-attn | 0.936 [0.922, 0.946] | 0.914 [0.900, 0.925] | 0.933 [0.919, 0.943] | 0.909 [0.902, 0.915] | 0.859 [0.852, 0.866] | 0.900 [0.893, 0.906] |
| Block 10 post-MLP | 0.936 [0.923, 0.946] | 0.915 [0.900, 0.926] | 0.933 [0.919, 0.943] | 0.912 [0.905, 0.918] | 0.860 [0.853, 0.867] | 0.903 [0.896, 0.909] |
| Block 11 post-attn | 0.936 [0.922, 0.947] | 0.915 [0.900, 0.926] | 0.933 [0.919, 0.943] | 0.913 [0.906, 0.919] | 0.860 [0.853, 0.868] | 0.904 [0.897, 0.910] |
| Block 11 post-MLP | 0.936 [0.923, 0.946] | 0.914 [0.899, 0.925] | 0.933 [0.919, 0.943] | 0.915 [0.908, 0.922] | 0.862 [0.855, 0.869] | 0.906 [0.900, 0.913] |
| Block 12 post-attn | 0.936 [0.923, 0.946] | 0.913 [0.899, 0.924] | 0.933 [0.919, 0.943] | 0.916 [0.909, 0.922] | 0.862 [0.855, 0.869] | 0.907 [0.900, 0.913] |
| Block 12 post-MLP | 0.936 [0.922, 0.946] | 0.914 [0.900, 0.925] | 0.932 [0.918, 0.943] | 0.918 [0.912, 0.924] | 0.865 [0.857, 0.872] | 0.908 [0.901, 0.914] |
| Post-Encoder | 0.936 [0.922, 0.946] | 0.913 [0.899, 0.924] | 0.932 [0.918, 0.943] | 0.918 [0.911, 0.924] | 0.865 [0.857, 0.871] | 0.908 [0.901, 0.914] |
Table D.3. Per-stage effect of the block-1 spatial extent eraser: held-out probe for and for redshift at every encoder stage under the clean forward pass, with the rank-1 LEACE eraser applied at block 1’s output, and with the shuffled-label control eraser. Stages are those of Appendix C.1; “post-attn” is the residual stream between a block’s attention and MLP sub-layers. *Site of the intervention; the first two rows precede it and are identical across conditions. Each entry gives the probe with its 95% bootstrap interval below (probe held fixed; Appendix A). †Rounds to zero at three decimals: [, ].
| quintile | Head 3 only | Head 3 zeroed | |||
| Q1 [0.000, 0.088] | 0.910 [0.894, 0.922] | 0.952 [0.943, 0.960] | 0.947 [0.937, 0.955] | 0.907 [0.896, 0.917] | [, ] |
| Q2 [0.088, 0.167] | 0.914 [0.901, 0.926] | 0.954 [0.942, 0.964] | 0.951 [0.939, 0.961] | 0.930 [0.918, 0.939] | [, ] |
| Q3 [0.167, 0.314] | 0.874 [0.855, 0.889] | 0.938 [0.928, 0.947] | 0.936 [0.925, 0.944] | 0.894 [0.881, 0.904] | [, ] |
| Q4 [0.314, 0.563] | 0.806 [0.779, 0.828] | 0.913 [0.892, 0.928] | 0.912 [0.889, 0.927] | 0.850 [0.831, 0.865] | [, ] |
| Q5 [0.563, 3.000] | 0.774 [0.751, 0.798] | 0.863 [0.838, 0.886] | 0.863 [0.839, 0.885] | 0.754 [0.729, 0.779] | [, ] |
Table E.1. Redshift per quintile on the held-out test set for the and readouts and for the readout with the other eleven block-1 heads mean-ablated (Head 3 only) and with Head 3 zero-ablated (Head 3 zeroed), with 95% paired bootstrap intervals. is Head 3 zeroed minus . These are the values plotted in Figure 5a.
and leave the rest of the encoder, including block 1’s MLP, unchanged, so this is a test of sufficiency among the block-1 heads only. Zero-ablating eleven heads at once would move block 1’s output far from the distribution that the later blocks receive, so we use mean ablation instead: each head’s output is replaced by its per-position mean over a random subsample of 5,000 training-split galaxies (seed 123), a fixed vector that does not depend on the input galaxy. Leaving only Head 3 active and mean-ablating the other eleven gives [0.922, 0.934], against 0.931 [0.924, 0.936] for the readout and 0.869 [0.861, 0.877] for . The paired gap to the readout is [, ], so Head 3 alone recovers 96% [94%, 98%] of the multimodal advantage of 0.061 [0.055, 0.067]. Per quintile, Head 3 alone recovers 87% [81%, 92%], 94% [88%, 98%], 96% [92%, 99%], 99% [96%, 101%], and 100% [96%, 104%] of the multimodal advantage from Q1 to Q5. The fraction is largest in the two most dust-attenuated quintiles, where the multimodal advantage is largest, and there its interval includes 100%.
Robustness. Three checks support reading Head 3’s rank as a property of the model rather than of our particular intervention or data split.
First, Head 3’s effect remains significant after correcting for the number of heads and evaluation sets tested. Ranking twelve heads on six evaluation sets (the full test set and the five quintiles) amounts to 72 tests of whether an ablation changes , and with that many tests, some small effects appear significant by chance. We therefore test each against zero and correct for the number of tests. Each test’s two-sided -value is estimated from 10,000 paired bootstrap resamples as twice the fraction of resamples in which has the opposite sign to the observed value, with a minimum of , the smallest value 10,000 resamples can resolve. We then apply the Holm correction, which keeps the probability of any false positive among the 72 tests at 5%. No resample gives Head 3 a in any evaluation set, so its -value is in all six, or 0.007 after correction, and it is the only head that remains significant in all six. On the full test set, Heads 9, 10, 11, and 12 also remain significant, but the largest of their effects (Head 11, [, ]) is far smaller than Head 3’s [, ].
Second, Head 3’s rank does not depend on the readout’s early-stopping criterion or data split. The readouts used throughout the paper are early-stopped on the test set; Appendix A describes two retrained pairs of readouts that are instead early-stopped on a validation set held out from the training split, one on the original partition and one on a second partition. Their readouts reach [0.924, 0.936] and 0.927 [0.920, 0.934]. Repeating the ablation of all twelve block-1 heads on each ranks Head 3 first, with [, ] and [, ], while ablating Head 1 gives [, ] and [, ]. Head 3’s first-place rank and Head 1’s near-zero effect therefore do not depend on the early-stopping criterion, the particular readout weights, or the particular test set.
Third, Head 3’s effect does not depend on the choice of ablation. Replacing Head 3’s output with its per-position mean rather than zero gives [0.825, 0.839], a change of [, ], larger in magnitude than the [, ] that zeroing produces. The effect is therefore not an artifact of the zero reference in the head’s outputs; zeroing gives the smaller of the two estimates of Head 3’s contribution.
Decomposition of the prediction error into attenuation bias and scatter
Decomposition. For redshift predictions evaluated against spectroscopic redshifts on a set of galaxies, we fit by ordinary least squares. Because the residual of this fit is uncorrelated with , the variance of the prediction error decomposes as
where is the variance of the fit residual. The first term is the attenuation-bias variance: a slope compresses the predictions toward their mean, and the term vanishes only when . Here, attenuation is the statistical term for a slope shrunk toward zero, distinct from dust attenuation. The second term is the scatter about the fitted line. Intervals in this subsection use 1,000 bootstrap resamples; for a difference between two readouts, both are evaluated on the same resampled galaxies (Appendix § 7).
Slope and bias. All results in this subsection are for the most dust-attenuated quintile (, ). Adding imaging reduces the attenuation bias: the slope rises from [0.786, 0.826] for the readout to 0.867 [0.851, 0.883] for the readout, and the attenuation-bias variance falls to 0.47 [0.38, 0.57] times its value. Zero-ablating Head 3 eliminates this reduction and raises the attenuation bias above its level: the slope falls to [0.692, 0.727], below the slope, and the attenuation-bias variance rises to 2.2 [1.9, 2.7] times its value, so high-redshift galaxies are underpredicted and low-redshift galaxies overpredicted more strongly than with photometry alone. The scatter, by contrast, remains slightly below that of the readout after ablation: [0.037, 0.041] against 0.042 [0.040, 0.044] (paired change [, ]).
Probes for expected physical parameters
Hypothesis and motivation. Zero-ablating Head 3 removes most of the multimodal advantage, and the effect is largest in the most dust-attenuated (highest ) quintile (Appendix § 11.§ 1). The most direct explanation is that Head 3 encodes the physical quantities that make dust-attenuated galaxies hard to predict from photometry alone: the dust attenuation itself, or a dust-insensitive estimate of intrinsic luminosity. The second possibility is suggested by Head 3’s attention to the WISE and tokens (Figure 5b), since these infrared bands are standard stellar-mass indicators that are only weakly affected by dust. If either holds, the quantity should be more decodable from Head 3’s output than from the other block-1 heads, and removing it from Head 3’s output should remove Head 3’s causal effect. We test these explanations with four analyses. All linear probes use the per-head protocol of Appendix D.2.
Probes for the quantities in Head 3’s ablation result. We first probe each block-1 head for three quantities tied directly to Head 3’s ablation result: the redshift , which the readout predicts; , the property along which the ablation effect grows; and the multimodal correction , the per-galaxy difference between the two readouts’ predictions (probes fit on training-split predictions and evaluated on the held-out split). For comparison, we also probe , the spatial extent proxy. Head 3 does not stand out on redshift or on the correction, and leads on only weakly (Table D.2). Its redshift probe ranks [0.558, 0.583], in the middle of the twelve heads, and its probe ranks last. It ranks first on , but at [0.246, 0.280], against 0.589 [0.568, 0.608] for a linear probe on the post-encoder representation. Every head decodes the multimodal correction only weakly, with from 0.044 [0.036, 0.052] to 0.109 [0.097, 0.121]; Head 1 is highest and Head 3 reaches 0.062 [0.052, 0.072]. Zeroing Head 3 removes most of the multimodal advantage, yet the per-galaxy correction is no more decodable from Head 3’s output than from that of Head 1, whose ablation has no measurable effect.
Probes for other candidate quantities. We next probe each head’s output mean-pooled over the 576 image tokens only, rather than over all 585 tokens as in Appendix D.2, for two further sets of candidates. Surface brightness is an image-derived quantity used in photometric-redshift estimation, so we probe -band surface brightness on a subset matched to Galaxy Zoo DECaLS ( test galaxies) and the inner-to-outer surface-brightness gradient. Head 3’s attention to and suggests a luminosity or stellar-mass estimate based on these bands, so we probe the PROVABGS stellar mass , apparent magnitude, and color. Head 3 ranks tenth of twelve on ( [0.039, 0.508], against 0.817 [0.768, 0.854] for the best head) and tenth on the gradient (0.168 [0.155, 0.180], against 0.244 [0.231, 0.257]). On the full test set, it ranks third on (0.587 [0.571, 0.601]) and sixth on both magnitude (0.397 [0.360, 0.438]) and (0.283 [0.228, 0.333]). Within the most dust-attenuated quintile, the intervals are too wide to rank heads: Head 3 reaches 0.016 [−0.068, 0.094], 0.323 [0.217, 0.444], and 0.127 [−0.186, 0.283] on the three targets, respectively, each interval overlapping that of the top head, and no head exceeds on stellar mass. Head 3 is not distinguished on any of these targets.
Nonlinear probes. A linear probe can miss information that is encoded nonlinearly, so we also probe Head 3’s image-token mean-pooled output for with an MLP: one hidden layer of 256 ReLU units, trained with Adam (learning rate , penalty , early stopping on a 10% validation split with patience 10, at most 300 iterations, seed 42), with a control trained on permuted targets. The MLP recovers more than a linear probe on the same features, [0.379, 0.414] against 0.264 [0.247, 0.279], and the control gives −0.009 [−0.012, −0.006], but it remains well below the 0.589 of a linear probe on the post-encoder representation. The weak linear result is therefore not explained by a nonlinear encoding that this MLP can recover. Intervals use resamples.
Erasing dust information from Head 3’s output. Weak decodability does not rule out that the little dust information in Head 3’s output is what the readout uses. We therefore remove it with rank-1 LEACE erasure (Appendix § 10.§ 4), which removes the linearly decodable information about a target while changing the output as little as possible, and re-run the readout. Unlike the spatial extent eraser of Appendix § 10.§ 4, this eraser is fit on Head 3’s per-token outputs for 4,000 training galaxies, because an eraser fit on the mean-pooled output distorts the individual token outputs enough that even its shuffled-label control degrades the prediction more than zero-ablating the whole head. After erasure, a probe on the mean-pooled output recovers at only [0.010, 0.017], against 0.264 [0.246, 0.280] before; a small residual remains because the eraser acts on each token separately while the probe reads their average. In the most dust-attenuated quintile, erasure lowers the readout’s by no more than a control eraser fit to shuffled labels (difference −0.002 [−0.007, +0.003]). Erasing the multimodal correction instead lowers it by 0.019 [0.012, 0.026] more than the control, so erasure does change the prediction when the erased information is used. The dust information that probes find in Head 3’s output is therefore not what the readout relies on. Intervals use resamples, except the post-erasure probe , which uses .
Dereddening and 4000 Å break tracking
Hypothesis and motivation. In Head 3’s attention (Figure 5b), every photometry query attends most to the flux token and the token. Two standard photometric-redshift operations would produce this pattern. The first is dereddening: is the foreground reddening from Milky Way dust along the line of sight, and the observed fluxes are dereddened for it as a routine preprocessing step, so a head that combines the flux with could be computing a dereddened flux. This correction concerns Galactic dust rather than the dust within the galaxy that measures, so this hypothesis is suggested by the attention pattern only. The second is tracking the 4000 Å break, the strongest redshift-dependent feature in the optical spectra of these galaxies: as redshift increases, the break moves through the band toward the band, so a head that tracks it should change its attention specifically between the and flux tokens. The break enters only near , beyond the redshifts of most galaxies in the sample (quintile medians 0.08–0.33), so we test for the gradual shift between and as the break moves through , not for a discontinuity at the transition.
Dereddening. A dereddening head’s output at the flux position should vary with and should track the dereddened flux more closely than the observed one. For each of the nine photometry positions, we compute the principal components of Head 3’s output across a random subset of 200 held-out galaxies (those of Figure 5b, for which Head 3’s per-token outputs and attention maps were stored) and correlate each galaxy’s score on the first principal component with and with the observed and dereddened flux, the latter computed with the coefficients of Schlafly and Finkbeiner [2011]. We find that neither prediction holds. At the flux position, the first principal component is uncorrelated with (Pearson correlation −0.03 [−0.17, +0.10]). It correlates equally with the observed and dereddened flux correlations 0.369 [0.301, 0.447] and 0.364 [0.297, 0.447], as expected for any output because is small for these galaxies (median 0.029), so the two fluxes are nearly identical (correlation 0.999 between them). At the position, the first principal component correlates only weakly with (correlation 0.28 [0.16, 0.39]). Correlation intervals resample the 200 galaxies with the principal components held fixed.
4000 Å break tracking. A break-tracking head should shift its attention between the and flux tokens as redshift increases, more than between other bands. For each of the 200 galaxies, we compute the attention that the photometry and image queries place on each photometry token; for the image queries we average over the 576 image tokens and express each photometry token’s weight as a share of their total attention to the photometry tokens. We average these quantities within each redshift quintile and fit a line to the averages against each quintile’s median redshift (intervals over 1,000 galaxy resamples). The attention of every photometry token to the flux token increases with redshift, opposite to the prediction (slope [, ] per unit redshift for the flux token’s attention to itself). The image tokens’ attention is redistributed among the photometry tokens as redshift increases, but not from to : the share of their photometry attention on the flux token decreases (slope [, ] per unit redshift, from 0.097 in the lowest redshift quintile to 0.067 in the highest), the share on the flux token shows no trend ( [, ]), and the shares on the and W1 tokens rise ( [, ] and [, ]).
Because the attention results are not specific to the break, we also test whether Head 3 encodes break-related information. In broadband photometry, the break appears as a color with two components: its wavelength, which shifts with redshift and determines how varies with redshift, and its amplitude, which determines at fixed redshift. The first is equivalent to redshift itself, which Head 3 encodes no better than the other block-1 heads (Appendix § 11.§ 3). For the second, we probe each head on the full test set for the residual color after subtracting a linear fit of on redshift (fit on the training split). This residual, how much redder or bluer a galaxy is than the average galaxy at its redshift, approximates the break amplitude but also reflects dust reddening. Head 3 ranks seventh of twelve ( [, ], against [, ] for Head 1). Head 3 therefore encodes neither the redshifted wavelength of the break nor its amplitude more than the other block-1 heads do.
Image-token spatial encoding
Hypothesis and motivation. Part of the improvement that images give photometric-redshift estimates has been attributed to information beyond integrated photometry: morphological parameters such as galaxy size, shape, and surface brightness [Soo et al., 2018], and the redshift information in individual pixels, which deep networks combine more effectively than integrated photometry does, reducing attenuation bias [21]. Dust attenuation varies across a galaxy, so a head that encodes resolved color or brightness structure could correct for it. If Head 3 did, four predictions follow: its image-token outputs should vary across a galaxy with the local color or brightness, its ablation should matter most for galaxies whose color varies most across the image, its effect should be concentrated at the image tokens that cover the galaxy’s light, and any dust information in its output should be concentrated at particular token positions.
Within-galaxy correlation with color and brightness maps. To compare each image token’s output with the color and brightness at that token’s position, we compute these maps at the resolution of the token grid, as in Appendix § 10.§ 2: the image codec encodes the central pixels of each cutout as a grid of image tokens, so each token corresponds to a -pixel block, and we average each map over that block. For each of the 200 held-out galaxies of Figure 5b, we compute the principal components of Head 3’s outputs across the galaxy’s 576 image tokens and correlate each token’s score on the first principal component with five maps of the same galaxy: brightness (the mean over the four bands), color, color, the local gradient (at each token, the largest absolute difference in from its four adjacent tokens, which is large where the color changes sharply, as at a dust lane), and distance from the image center. The mean absolute correlation over galaxies is at most 0.13: 0.120 [0.115, 0.124] for brightness, 0.126 [0.117, 0.137] for , 0.128 [0.118, 0.138] for , 0.087 [0.078, 0.097] for the gradient, and 0.077 [0.069, 0.084] for distance from the center. For comparison, the same analysis applied to Head 1 gives a mean absolute correlation of 0.739 [0.737, 0.741] between the first-principal-component score and distance from the image center (§ 4.3.2), so the analysis does detect a dependence of a head’s output on token position when one is present.
Ablation effect versus color variation across the galaxy. For every galaxy in the test set, we compute , the standard deviation over the 576 image tokens of the map after subtracting its galaxy mean, divide the test set into quintiles of , and compute Head 3’s zero-ablation in each. The effect shrinks as color variation grows, from [−0.095, −0.074] in the lowest quintile () to −0.047 [−0.056, −0.038] in the highest ([0.55, 2.49]; difference +0.038 [+0.024, +0.051]), the opposite of the prediction. The lowest and highest quintiles have comparable mean , so the decline is not a consequence of the dependence of Head 3’s effect on .
Token-subset ablation. If Head 3’s effect came from resolved structure in the galaxy, zero-ablating Head 3 only at the image tokens that cover the galaxy’s light should reproduce most of its effect. We approximate these tokens in two ways, as the brightest quarter of each galaxy’s image tokens and as the 112 image tokens within 6 tokens of the image center, and compare them with the complementary subsets. Each in the most dust-attenuated quintile is expressed as a fraction of the −0.106 [−0.119, −0.093] obtained by ablating all 576 image tokens. The tokens covering the galaxy carry almost none of the effect: the brightest quarter accounts for 2.0% [0.9%, 3.2%] and the central tokens for 3.0% [1.9%, 4.1%]. The remaining 464 outer tokens account for 41.9% [35.9%, 47.2%], the faintest quarter for −0.8% [−2.4%, 0.6%], and ablating Head 3 at the 9 photometry tokens instead for 1.5% [0.5%, 2.8%]. The central and outer subsets together cover all image tokens, yet their separate ablations sum to −0.048 [−0.057, −0.038], less than half of the joint −0.106, so Head 3’s effect depends on many image tokens jointly.
Per-token probes. Appendix E.3 uses probes fit on Head 3’s output mean-pooled over the image tokens, and mean pooling could dilute dust information present at only a few positions. We therefore fit a separate linear probe at each of the 576 image-token positions; if such information existed, the best single-token probe should exceed the mean-pooled probe. It does not: for , the best token reaches [0.220, 0.253], below the 0.264 [0.246, 0.280] of the mean-pooled probe, and for redshift, the best token reaches 0.557 [0.546, 0.568] against 0.570 [0.558, 0.583]. The per-token probes do detect position-specific information when it is present: for , which is measured in an aperture at the galaxy center, the best token, one of the four at the center of the token grid, reaches 0.701 [0.687, 0.713], above the 0.629 [0.615, 0.642] of the mean-pooled probe.
Correction of photometry–image discrepancies
Hypothesis and motivation. Head 3’s image queries place a mean of 24.9% [24.2%, 25.5%] of their attention on the nine photometry tokens, summed over key tokens as in Appendix D.2 (19.5%–30.5% between the 10th and 90th percentiles over the 200 galaxies of fig:5b, against 0.37% [0.37%, 0.38%] for Head 1. Through this attention, each of Head 3’s image-token outputs includes a weighted sum of the photometry tokens’ value vectors, so Head 3’s output could correct the image-token representation of a galaxy where it is inconsistent with the photometry-token representation, or the reverse; here, inconsistent means that the two representations encode different values of the same property, such as the -band magnitude. If Head 3’s effect came from such a correction, its ablation should matter most for the galaxies whose photometry-token and image-token representations are most inconsistent.
Ablation effect versus photometry–image discrepancy. We measure the inconsistency through the -band magnitude . We use a magnitude because both modalities measure it directly, as the flux token and as the -band pixels, so the two representations should encode the same value. Two linear probes, fit on the training split, predict from the token embeddings that enter block 1, mean-pooled separately over the 9 photometry tokens and the 576 image tokens (held-out [0.973, 0.975] and 0.639 [0.620, 0.657]); because these embeddings precede block 1, the discrepancy is unaffected by ablating any block-1 head. After centering each probe’s predictions on their test-set mean, the discrepancy is . We divide the test set into quintiles of and compute Head 3’s zero-ablation in each. Instead of growing with , Head 3’s effect shrinks, from [−0.075, −0.056] in the lowest quintile to −0.031 [−0.038, −0.025] in the highest. is also uncorrelated with (Spearman [−0.026, +0.007]), so the decline is not a consequence of the dependence of Head 3’s effect on . Because the image-token probe is less accurate than the photometry-token probe, largely reflects the error of the image-token probe and is therefore an approximate measure of the inconsistency between the two representations.
Closing the redshift probe gap with Head 3’s output
Hypothesis and motivation. The tests of Appendices E.3–E.6 each assume a specific operation. This test assumes none: if Head 3’s output contains the information that distinguishes the representation from the representation, in a form that a probe can extract, then adding Head 3’s output to the representation should close a substantial fraction of the redshift probe gap. We quantify this as
the fraction of the probe gap closed, where and are redshift probes on the mean-pooled post-encoder representations of the two inputs, and is a probe of the same type on the representation concatenated with Head 3’s output. The probes are fit on frozen features, and the readout is unchanged.
Mean-pooled output. We first concatenate the 768-dimensional representation with Head 3’s 64-dimensional output mean-pooled over the image tokens. With linear probes (protocol of Appendix D.2), [0.819, 0.836], [0.911, 0.924], and [0.824, 0.840], so [0.034, 0.066]. In the most dust-attenuated quintile, where Head 3 alone recovers the full multimodal advantage (§ 4.4.1), [−0.006, 0.032]. With the MLP of Appendix E.3 in place of linear probes, [0.011, 0.096].
Token-resolved output. Mean pooling discards which token each output comes from, so we repeat the linear test with the 576 image-token outputs kept separate, each projected onto its leading 8 or 16 principal directions, fit on the image-token outputs of 3,000 training galaxies (96% and 99.6% of per-token variance). The token-resolved output alone gives redshift [0.752, 0.771] with 8 directions, against 0.570 [0.558, 0.583] for the mean-pooled output, so mean pooling does discard redshift information. Concatenated with the representation, the token-resolved output closes [0.060, 0.134] of the gap with 8 directions and 0.102 [0.056, 0.146] with 16, a rise over the mean-pooled output of 0.049 [0.014, 0.083] with 8 directions. In every case, Head 3’s output closes at most about a tenth of the probe gap, although Head 3 is necessary for most of the multimodal advantage. These probes test only whether the information can be extracted from Head 3’s output directly; they do not exclude a contribution of Head 3’s output through computation in later blocks.
Model-wide search and downstream circuit
Hypothesis and motivation. We zero-ablated only the twelve heads of block 1, which leaves two alternatives open: heads in later blocks could have effects comparable to Head 3’s, or Head 3’s effect on the prediction could be mediated mainly by one or a few later heads whose input includes Head 3’s output. We test the first with an attribution screen over all 144 heads followed by zero-ablation of the heads it flags, and the second with two mediation tests on the later heads that are the strongest candidate mediators: single-edge path patching and ablation with frozen downstream heads. We label head of block as , with both indices starting at 1.
Attribution screen over all 144 heads. Zero-ablating each of the 144 heads separately would require 144 forward passes over the test set, so we first rank the heads with gradient × activation attribution [Syed et al., 2024], a first-order approximation to the change in the prediction under zero-ablation. For each head, we multiply its 64-dimensional output, taken before the attention output projection, elementwise by the gradient of the predicted redshift with respect to that output, and sum over dimensions and all 585 token positions; one backward pass per galaxy gives the scores of all 144 heads. The negative of a head’s score approximates the change in each galaxy’s predicted redshift when the head is zero-ablated; adding it to the clean predictions and recomputing gives the head’s predicted . On block 1, where every head has also been zero-ablated, the predicted and measured rank the twelve heads similarly (Spearman [0.790, 0.993]), and Head 3’s predicted [−0.060, −0.053] is close to its measured −0.052 [−0.056, −0.049]. We flag every head outside block 1 whose predicted in the most dust-attenuated quintile exceeds 0.0083, the measured by zero-ablating Head 9 on the same galaxies. Among the block-1 heads other than Head 3, this is the second-largest effect in that quintile, after Head 11’s 0.0094 (Table D.2). Five heads are flagged: B5H4, B2H3, B5H10, B2H12, and B8H7.
Zero-ablation of the flagged heads. We zero-ablate each flagged head with the protocol used for the block-1 heads. In the most dust-attenuated quintile, the measured effects follow the order predicted by the screen, and four of the five exceed the 0.0083 threshold; we refer to these four as the confirmed heads. The exception is B8H7, at [, ]. Over the full test set, every effect is negative with an interval excluding zero, but all are small, ranging from [, ] (B5H4) to [, ] (B8H7), an order of magnitude below Head 3’s [, ]. The screen therefore finds no head outside block 1 with an effect comparable to Head 3’s.
Single-edge path patching. A later head with a small effect of its own could still mediate a large part of Head 3’s effect, because its input includes Head 3’s output. Path patching [Wang et al., 2023] tests this one head at a time. For a later head X, we run the model on the clean input but compute X’s queries, keys, and values from a residual stream from which Head 3’s output has been removed; every other head receives the clean residual stream, so only the edge from Head 3 to X is cut. AION-1 computes the queries, keys, and values of all heads in a block with one linear projection, so we compute this projection on both residual streams and replace X’s slice of the clean result with its slice of the ablated result; with no ablation, this procedure reproduces the clean predictions to within . The edge fraction is the over the full test set from cutting the edge, divided by the from zero-ablating Head 3, so would mean that all of Head 3’s effect is mediated by X.
We cut the edges from Head 3 to the four confirmed heads (B2H3, B2H12, B5H4, B5H10); to B9H1 and B10H10, which the screen did not flag in the most dust-attenuated quintile but which rank among the five heads with the largest predicted over the full test set; and, as controls, to the two heads in each of blocks 2, 5, 9, and 10 with the smallest predicted in the most dust-attenuated quintile. Every edge fraction is small: the largest are [, ] for B5H4, [, ] for B2H12, and [, ] for B2H3, against at most [, ] for a control edge. Cutting an edge nonetheless changes the output of the receiving head substantially. Measured as the Frobenius norm of the change divided by that of the clean output, the change is – for the four confirmed heads and – for the eight control heads (each interval narrower than ). The later heads’ outputs therefore depend strongly on Head 3’s output, but cutting any single edge reproduces at most 7% of Head 3’s effect on the prediction.
Ablation with frozen downstream heads. Path patching cuts one edge at a time while every other head still receives Head 3’s output, so it does not measure what the four confirmed heads mediate jointly, including through each other. To measure this, we zero-ablate Head 3 while fixing the four confirmed heads to their clean-run outputs, so that the ablation cannot propagate through them. The mediated fraction is the fraction of the ablation effect that freezing removes: if the frozen heads mediate all of Head 3’s effect and if they mediate none of it. Freezing heads changes the computation even when they do not mediate Head 3’s effect, so as a reference, we also freeze the four path-patching control heads in blocks 2 and 5, the blocks containing the confirmed heads. In the most dust-attenuated quintile, freezing the four confirmed heads gives [, ] and freezing the four control heads gives [, ]; the mediated fraction of the confirmed heads is thus small and of similar magnitude to that of heads not selected as candidate mediators. Neither test finds that a single later head, or the four confirmed heads jointly, mediates most of Head 3’s effect. Both tests cover attention heads only; mediation by many heads with small individual contributions or by the MLP layers, and a direct effect of Head 3’s output through the residual stream on the readout, are not excluded.
References
- [1]Emmanuel Ameisen, Jack Lindsey, Adam Pearce, Wes Gurnee, Nicholas L. Turner, Brian Chen, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. Circuit Tracing: Revealing Computational Graphs in Language Models. Transformer Circuits Thread, 2025. URL https://transformer-circuits.pub/2025/attribution-graphs/methods.html.
- [2]Eirini Angeloudi, Jeroen Audenaert, Micah Bowles, Benjamin M Boyd, David Chemaly, Brian Cherinka, Ioana Ciucă, Miles Cranmer, Aaron Do, Matthew Grayling, et al. The Multimodal Universe: Enabling large-scale machine learning with 100 TB of astronomical scientific data. Advances in Neural Information Processing Systems, 37:57841–57913, 2024.DOI
- [3]Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219, 2022.arxiv.org/abs/2102.12452
- [4]Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. LEACE: Perfect linear concept erasure in closed form. Advances in Neural Information Processing Systems, 36:66044–66063, 2023.arxiv.org/abs/2306.03819
- [5]DESI Collaboration, AG Adame, J Aguilar, S Ahlen, S Alam, G Aldering, DM Alexander, R Alfarsy, C Allende Prieto, M Alvarez, et al. The Early Data Release of the Dark Energy Spectroscopic Instrument. The Astronomical Journal, 168(2):58, 2024.
- [6]Arjun Dey, David J Schlegel, Dustin Lang, Robert Blum, Kaylan Burleigh, Xiaohui Fan, Joseph R Findlay, Doug Finkbeiner, David Herrera, Stéphanie Juneau, et al. Overview of the DESI Legacy Imaging Surveys. The Astronomical Journal, 157(5):168, 2019.
- [7]Biprateep Dey, Brett H Andrews, Jeffrey A Newman, Yao-Yuan Mao, Markus Michael Rau, and Rongpu Zhou. Photometric redshifts from SDSS images with an interpretable deep capsule network. Monthly Notices of the Royal Astronomical Society, 515(4):5285–5305, 2022.
- [8]Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah Goodman, and Christopher Potts. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning, pages 7324–7338. PMLR, 2022.
- [9]ChangHoon Hahn, KJ Kwon, Rita Tojeiro, Malgorzata Siudek, Rebecca EA Canning, Mar Mezcuа, Jeremy L Tinker, David Brooks, Peter Doel, Kevin Fanning, et al. The DESI ProBabilistic Value-Added Bright Galaxy Survey (PROVABGS) Mock Challenge. The Astrophysical Journal, 945(1):16, 2023.
- [10]Stefan Heimersheim and Neel Nanda. How to use and interpret activation patching. arXiv preprint arXiv:2404.15255, 2024.arxiv.org/abs/2404.15255
- [11]Ben Henghes, Jeyan Thiyagalingam, Connor Pettitt, Tony Hey, and Ofer Lahav. Deep learning methods for obtaining photometric redshift estimations from images. Monthly Notices of the Royal Astronomical Society, 512(2):1696–1709, 2022.
- [12]Morgan Himes, Samiksha Krishnamurthy, Andrew Lizarraza, Srinath Saikrishnan, Vikram Seeni-vasan, Jonathan Soriano, Ying Nian Wu, and Tuan Do. Multi-Modal Masked Autoencoders for Learning Image-Spectrum Associations for Galaxy Evolution and Cosmology. arXiv preprint arXiv:2510.22527, 2025.arxiv.org/abs/2510.22527
- [13]John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):583–589, 2021.DOI
- [14]Ashod Khederlarian, Brett H Andrews, Jeffrey A Newman, Tianqing Zhang, and Biprateep Dey. Optimizing Deep Learning Photometric Redshifts for the Roman Space Telescope with HST/CANDELS. The Astronomical Journal, 171(6):340, 2026.
- [15]Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. On the Biology of a Large Language Model. Transformer Circuits Thread, 2025. URL https://transformer-circuits.pub/2025/attribution-graphs/biology.html.
- [16]Michael McCabe, Bruno Régaldo-Saint Blanchard, Liam Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawczyk, François Lanusse, et al. Multiple physics pretraining for spatiotemporal surrogate models. Advances in Neural Information Processing Systems, 37:119301–119335, 2024.DOI
- [17]Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik, and Shane Legg. The hydra effect: Emergent self-repair in language model computations. arXiv preprint arXiv:2307.15771, 2023.arxiv.org/abs/2307.15771
- [18]Grant Merz, Xin Liu, Samuel Schmidt, Alex I Malz, Tianqing Zhang, Doug Branton, Colin J Burke, Melissa Delucchi, Yashwant Sai Ejjagiri, Jeremy Kubica, et al. DeepDISC-photoz: Deep Learning-Based Photometric Redshift Estimation for Rubin LSST. arXiv preprint arXiv:2411.18769, 2024.DOI
- [19]Paul Michel, Omer Levy, and Graham Neubig. Are sixteen heads really better than one? Advances in Neural Information Processing Systems, 32, 2019.arxiv.org/abs/1905.10650
- [20]David Mizrahi, Roman Bachmann, Oğuzhan Kar, Teresa Yeo, Mingfei Gao, Afshin Dehghan, and Amir Zamir. 4M: Massively Multimodal Masked Modeling. In Advances in Neural Information Processing Systems, volume 36, pages 58363–58408, 2023.arxiv.org/abs/2312.06647
- [21]Emma R Moran, Brett H Andrews, Jeffrey A Newman, and Biprateep Dey. Deep Learning Improves Photometric Redshifts in All Regions of Color Space. The Astrophysical Journal, 997(1):36, 2026.arxiv.org/abs/2507.06299
- [22]Ari S Morcos, David GT Barrett, Neil C Rabinowitz, and Matthew Botvinick. On the importance of single directions for generalization. In International Conference on Learning Representations, 2018.arxiv.org/abs/1803.06959
- [23]Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom In: An Introduction to Circuits. Distill, 2020. doi: 10.23915/distill.00024.001. URL https://distill.pub/2020/circuits/zoom-in.DOI
- [24]Liam Parker, François Lanusse, Siavash Golkar, Leopoldo Sarrà, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Geraud Krawczyk, Michael McCabe, Rudy Morel, et al. AstroCLIP: a cross-modal foundation model for galaxies. Monthly Notices of the Royal Astronomical Society, 531(4):4990–5011, 2024.DOI




