betaFeedback

WorksComputer Science

Interpretable weight-space programming builds transparent transformers and brain encoding models

Abstract

Neural networks are powerful predictors, yet their opacity limits their usefulness in science, where the goal is explanation rather than mere prediction. In neuroscience, for example, deep encoding models accurately predict brain activity in response to natural stimuli, but struggle to reveal which stimulus features drive those predictions. Here we introduce interpretable weight-space programming (iWSP), a method in which an LLM agent iteratively builds interpretable models by explicitly programming their weights and features in code, without gradient-based training (backpropagation). Without backpropagation to tune parameters, the agent must draw on its prior knowledge and reason over previous trials to hypothesize, hand-craft, and refine human-interpretable computations mirroring the workflow of a human scientist. LLM audits confirm the interpretability of the hand-crafted models. We first validate iWSP by tasking it with constructing transformers for canonical benchmarks such as arithmetic and sentiment classification, finding that LLM agents can successfully craft compact, accurate networks directly. Next, we apply iWSP to build neural encoding models across vision, language, and audition. In the visual cortex, iWSP outpredicts existing interpretable baselines, uncovers distinct computations across early/higher visual areas, and generalizes to unseen data. In language fMRI, iWSP surfaces a confound in the data and in auditory ECoG, it systematically discovers interpretable acoustic features that predict neural activity. Together, these results establish iWSP as a general paradigm for aligning high predictive performance with human-interpretable scientific discovery.

github.com/csinva/weight-space-programming-autoresearch

Introduction

In the early days of artificial neural networks, McCulloch and Pitts wrote every connection of small networks by hand to make them compute logical functions.[1] The values of these weights were interpretable, allowing one to understand how the models worked. In the modern era, as massive networks trained with backpropagation have improved their performance, models have sacrificed this interpretability. The resulting opacity raises concerns around regulatory compliance, safety, and alignment.[2, 3, 4, 5] It is particularly limiting in fields such as neuroscience[6] and social science,[7] where trustworthy interpretation is often itself the end goal. Neuroscience has seen the rise of large neural-network based “encoding models” that predict brain responses to natural stimuli, like images or text.[8, 9, 10, 11, 12, 13] However, these models are often difficult to interpret, making it difficult to connect features of the input stimulus to brain activity.[14, 15, 16, 17]

To tackle this challenge, we introduce interpretable weight-space programming (iWSP). iWSP guides a frontier agent to directly write a model while disallowing any training. Then, a fixed evaluator measures its performance, and the agent revises it iteratively (Figure 1). Disallowing training requires the agent to reason and implement each change in text/code, so weight changes are generally guided by the agent’s prior knowledge and each change is accompanied by a written explanation and resulting change in performance.

Figure 1. The iWSP loop and its interpretability audit. a, An LLM agent reads a task prompt and writes a model whose weights are set directly in code. No training is allowed. A fixed evaluator compares the model’s output with the target yy and returns a performance metric, which the agent uses to revise the code. On algorithmic/NLP tasks the model is a neural network whose weights are all set by the agent to generate the exact answer. For brain encoding, the hand-written network computes fixed features, and the evaluator fits a ridge regression from those features to recorded responses and evaluates it on visible stimuli. b, After the search, an LLM judge audits every saved model for training, rule violations and uninterpretable edits. To test interpretability, an agent (Opus 5.5) describes the best model in at most 200 words and a fresh session of the agent rebuilds it from that text alone, shown here for an addition task. c, In the algorithmic/NLP tasks for example, we find that the original and rebuilt models achieve very similar performance (2 rebuilds per task).

iWSP builds on a vast literature. One line of work builds inherently interpretable models,[18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30] though their constrained form often limits accuracy on hard problems. To get around this limit, recent work uses autoresearch to discover effective programs to solve hard problems.[31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41] Such programs can be translated into neural networks using a domain-specific language[42, 43] or alternatively these weights can be generated by training a network to do so.[44, 45, 46, 47, 48, 49, 50] In weight-space programming,[51] the agent instead reads and edits the weights itself, potentially guided by mechanistic interpretability approaches, such as computing activation differences[52, 53, 54, 55, 56] We extend this work with a simple modification: we require that weight-space edits be interpretable and then iteratively update weights to optimize a performance objective. To ensure interpretability, we audit the resulting models (both manually and with LLMs as a judge), and quantitatively test whether they can be rebuilt from a short text description (Figure 1b).

We first evaluate iWSP on relatively simple algorithmic/NLP tasks. Using frontier agents (e.g. Claude Opus 5.5), iWSP writes transformers from scratch that effectively solve 13 different algorithmic tasks, and achieve respectable performance on 3 out of 4 standard NLP tasks like sentiment classification (§ 3). We then study iWSP in three settings that predict brain responses to naturalistic stimuli. First, in visual cortex, an iWSP model written on one functional magnetic resonance imaging (fMRI) dataset predicts responses in another better than interpretable baselines, and its computations separate early and higher visual areas (§ 4.1). Second, in language fMRI, the interpretability of iWSP is useful for debugging, helping to identify a dataset confound (§ 4.2). Third, in electrocorticography (ECoG) responses to audio (§ 4.3), iWSP finds accurate interpretable models that can improve the predictive performance of black-box models. Taken together, these results suggest that frontier agents can help close the gap between highly accurate neural networks, and ones that we can usefully understand.

Method: Interpretable weight-space programming

Motivated by the search for interpretable models that can help scientific domains, we seek to improve the performance of human-understandable neural networks. To do so, we follow an approach similar to weight-space programming,[51] which prompts an agent to directly read and edit neural network weights. We study interpretable weight-space programming (iWSP), where we further require that weight-space edits be interpretable and then iteratively update weights to optimize a performance objective. Interpretability is suggested in the prompt, and then enforced by an auditing agent that judges whether model changes at each iteration are human-interpretable. This approach enables an agent to search for an interpretable model much like a human would, by using its priors and domain knowledge.

At a high level, each iWSP run has the same setup (Figure 1). A frontier coding agent is given a prompt that describes the task, and a harness that includes a starting file and a non-editable evaluation script that evaluates the current model. The prompt and harness forbid training, including optimizers, fitting loops, and downloading pretrained weights. The agent remains free to use known interpretable operations (e.g. Gabor filters), but must implement them using only its given basic packages, such as pytorch. It then repeatedly writes a model, evaluates it on a “visible set” of examples and revises it for a fixed number of iterations. Throughout, the agent is prompted to search for a human-understandable solution.

Interpretability audits Each saved model (and the agent’s corresponding logs) is audited by a separate agent to test for the interpretability of the edits and any violations of the constraints outlined in the prompt and harness (Figure 1b). As an additional check on whether a model uses any training data explicitly, we verify that rerunning each model’s weight-writing code on altered task data leaves its weights unchanged (algorithmic and NLP tasks). After the run concludes, the models are reevaluated on a “hidden set” that they never saw during the iterative loop. We further test the interpretability of the final models by checking whether the models can be accurately described with a short text description using a frontier LLM (Opus 5.5). We ask the LLM to describe the best model for each task in at most 200 words, and then give only that description and the task to a fresh session, which rebuilds the model (Figure 1b). We then evaluate how well the rebuilt model reflects the original, e.g. does it maintain similar predictive performance?

Algorithmic/NLP tasks. In the algorithmic/NLP tasks, each run starts from one Python file and one instruction document. The file defines a small GPT-style transformer, with three layers of width 64 by default, a function write_weights(model, task) for the agent to fill in, and fixed code that evaluates the result. The prompt names the task and tells the agent to follow the document until it has run 30 to 50 iterations, depending on the task. The document states the rules, including:

The model is NEVER trained. The agent must write the weights directly into interpretable_transformer.py. [...] What you CANNOT do: Run any kind of training, gradient descent, optimizer step, fitting loop, or backprop. The model parameters must be set in closed form. [...] The goal is simple: maximize exact accuracy [...] All else being equal, more interpretable is better. A small improvement that adds ugly complexity is not worth it. Number of model parameters can help serve as a proxy for interpretability, but the real criterion is human-understandable structure in the weights.

The agent may change the architecture and hard-code constants, including hand-built attention circuits and lookup tables; it may not compute the answer with anything outside the network (e.g. a Python function). Each run has its own folder, and the agent works in a restricted Claude Code session with six file and shell tools, no web access and file reads confined to that folder. Its only shell command, apart from read-only listings, runs this file. The harness cannot stop the agent’s own Python from training or from computing the answer outside the network, so those rules remain instructions that we audit afterward. The agent learns the task format by printing examples from inside its code, and after each iteration it sees its visible accuracy on a fixed visible set of 200 examples, where the answer is generated token by token and must match exactly. For these experiments, the iWSP agent is Claude Code running Opus 5.5, Sonnet 5.5 or Haiku 5.5 and audits for constraint-following are done by Opus 5.5.

Brain encoding models. In our experiments involving brain data, the agent writes a model ff that maps each stimulus xx to features f(x)f(x). A fixed evaluator then fits a ridge regression model WW from f(x)f(x) that linearly predicts recorded responses yy on training stimuli, predicts y^=Wf(x)\hat{y} = Wf(x) on the visible set, and returns the pearson correlation between y^\hat{y} matches yy. The agent receives only this metric and every candidate within a dataset uses the same stimulus split, ridge regression procedure, and metric. In addition to the standard constraints above, the rules forbid a model from reading any of the brain responses and the evaluation code rejects a candidate with 10,000 or more features. Unless otherwise specified, the agentic system is Claude Code with Opus 5.5; the visual cortex search uses Gemini 4 Carbon (§ 4.1). The visual cortex search runs in a sandbox with no network and no access to the datasets, run directories or repository. The agents work in their own sandbox, which shows only their workspace, and they receive performance metrics through a restricted interface. The story-listening fMRI and ECoG searches do not isolate in a sandbox and instead rely on the prompt with a post-hoc audit (§ 4.2 and § 4.3).

In total, the paper’s experiments cost about $17,500 at API list prices, including estimated exploratory runs (Table A7).

ExperimentCost at API list price
Algorithmic/NLP tasks, Claude Opus 5.5 (34 runs)$449
Algorithmic/NLP tasks, Claude Sonnet 5.5 (34 runs)$102
Algorithmic/NLP tasks, Claude Haiku 5.5 (34 runs)$71
Algorithmic/NLP tasks, exploratory runsabout $1,240 (estimate)
Rebuilding algorithmic/NLP models from descriptions, Claude Opus 5.5$60
Rebuilding language fMRI models from descriptions, Claude Opus 5.5$10
Visual cortex search, Gemini 4 Carbon$539
Visual cortex exploratory runsabout $4,500
Story-listening fMRI searches, Claude Opus 4.7about $3,000 (estimate)
Story-listening fMRI exploratory runsabout $6,000 (estimate)
ECoG searches, Claude Opus 4.7about $1,540
Totalabout $17,500

Table A7. Agent cost of each experiment.

Algorithmic/NLP results: iWSP builds effective, interpretable transformers

Experimental setup. To build our intuition, we first evaluate iWSP on 17 diverse tasks. The tasks include 13 algorithmic tasks such as multiplication, sorting, edit distance and date normalization (inspired by the AdderBoard addition task [57]), and four NLP tasks: 6-class question classification on TREC,[58] binary sentiment classification on SST-2,[59] binary paraphrase detection on MRPC[60] and three-way natural language inference on SNLI[61] (see full descriptions in Table A1). We run iWSP on each task twice with each of Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5, for about 1,340 iterations per LLM (about 39 per run). The hidden set is 2,000 new examples for algorithmic tasks and the official test split for NLP tasks. After the runs, we audit the iWSP models as described in § 2 and exclude the few that do not pass from the results (see full details in Table A2).

Table A1. The 17 tasks. Models read the prompt on the left and generate the answer on the right one token at a time. For NLP tasks we show the input format, and we shorten the padding in date normalization.

TaskDefinitionExample or format
Algorithmic, easier
addition-five-digitsAdd two 5-digit non-negative integers.12345+67890= → 080235
multiplication-five-digitsMultiply two 5-digit non-negative integers.12345*67890= → 0838102050
sort-five-digitsSort a string of 5 digits in ascending order.31415= → 11345
digit-counting-10Count occurrences of a query digit in a string of 10 digits.31415926533= → 02
parity-upto10-bitsCompute parity (XOR) of a variable-length bit string, length 1-10._...1011010= → 0
boolean-circuit-5-bitsEvaluate a boolean expression of 5 bits joined by 4 binary operators.`1&01^1&0=→0`
linear-interpolation-two-pointsLinear interpolation given two integer (x, y) pairs and a query x.1,05;4,11;7= → 17
Algorithmic, harder
word-reversal-3x3Reverse the order of 3 fixed-length words separated by underscores.cat_dog_pig= → pig_dog_cat
gcd-three-digitsGreatest common divisor of two 3-digit non-negative integers.084,036= → 012
decimal-to-binary-8bitConvert a decimal integer in [0, 255] to its 8-bit binary representation.042= → 00101010
edit-distance-8Levenshtein edit distance between two 8-letter strings over {a, b, c, d}.abcdabcd,abddabca= → 2
smallest-prime-factorSmallest prime factor of an integer below 100000 (a prime is its own smallest factor).03127= → 00053
date-normalizationRewrite a date written in one of nine common formats as YYYY-MM-DD.march 3rd 2021_...= → 2021-03-03
NLP
trec-coarseSix-way question classification on a small subset of TREC (coarse labels).<question padded to 80 chars>= → L
sentiment-sst2Binary sentiment classification on a small subset of SST-2.<sentence padded to 80 chars>= → L
paraphrase-mrpcBinary paraphrase detection on a small subset of GLUE/MRPC.`<s1 padded><s2 padded>=→L`
nli-snliThree-way natural language inference on a small subset of SNLI.`<premise padded><hypothesis padded>=→L`

Table A1.

Table A2. Audit results and hidden accuracy for each LLM across the 17 tasks. We classify each iteration by rerunning its weight-writing code on altered task data, and count an iteration as not checked if its code could not be rerun or was not deterministic. Hidden accuracy is measured on the first model to reach each run’s best visible accuracy. The size row only includes checked models; including all iterations gives medians of 166, 35,850 and 100,000. We added two sentences to the Sonnet and Haiku prompts on how to learn the task format, since their first sessions stopped early.

MeasureOpusSonnetHaiku
Iterations over 34 runs1,3341,3411,338
Weights independent of task data1,3031,0421,311
Weights dependent on labels0500
Weights depend on inputs only0340
Could not be checked3121527
Algorithmic runs at 100% visible accuracy26 of 2626 of 2625 of 26
Algorithmic runs at 100% hidden accuracy26 of 2625 of 2625 of 26
Median smallest model at 100% visible accuracy (parameters)16641,600100,000
TREC hidden accuracy87.3%86.6%71.1%

Table A2.

Models generalize on algorithmic tasks and on 3 out of 4 NLP tasks. We first describe the Opus runs. All 26 runs on algorithmic tasks reach 100% accuracy both on the visible set and the hidden set (Figure 2a), often by implementing a known algorithm in the weights. The NLP tasks are harder as they have no known full solution. Nevertheless, on 3 of the 4 NLP datasets iWSP builds models that achieve respectable performance well above a naive majority baseline (it fails on the relatively difficult Paraphrase task).

Figure 2. iWSP writes exact, readable models for algorithmic tasks and weaker ones for NLP tasks. Results in this figure use Claude Opus 5.5 unless noted. a, Models reach 100% hidden accuracy on algorithmic tasks but not on NLP tasks, where paraphrase stays below its baseline. Bars show hidden accuracy over 2 runs per task, and black lines mark always giving the most common answer. b, The searches both improve over iterations: algorithmic runs reach 100% visible accuracy early and then find more compact models, while NLP runs taper off. Thin lines follow each run’s best model so far, and thick lines average each group from the open circle to the arrowhead. c, The ability to find small accurate models depends on the LLM: Opus 5.5 finds far smaller models that reach 100% visible accuracy than Sonnet 5.5 or Haiku 5.5. Markers show geometric means over 2 runs and dashed lines show medians. d, For the algorithmic tasks, written models often translate a known algorithm into a transformer. For finding the greatest common divisor (GCD), the model uses the subtractive Euclidean algorithm, shown here on one input pair. e, iWSP implements this algorithm using a simple repeating block of ReLU units. f, For NLP tasks, iWSP builds crude interpretable tools to represent the intricacies of language, such as a rule-based negator for sentiment classification.

iWSP compresses models after making them accurate. Over iterations, both types of tasks improve in their accuracy and then start to compress the model parameters (Figure 2b). On average, algorithmic runs reach 99% visible accuracy within the first tenth of their iterations and spend the rest compressing from about 3,400 parameters to about 200. The NLP runs’ performance plateaus after about 75% of their iterations and stay near 190,000 parameters. On algorithmic tasks, the smallest model that reaches 100% visible accuracy has a median of 166 parameters (Figure 2c), and in the median run it is 29 times smaller than the first model to reach 100%.

Bigger LLMs mostly achieve similar performance but are better at compressing models. When comparing different Claude models (Opus 5.5, Sonnet 5.5, and Haiku 5.5), we find that they generally all achieve about 100% accuracy on the algorithmic tasks, but that Opus and Sonnet outperform Haiku on the NLP tasks (Table A2). Beyond predictive accuracy, better LLMs are better able to compress an accurate model (Figure 2c), with the smallest model that reaches 100% visible accuracy having a median of 166 parameters for Opus, 41,600 for Sonnet and 100,000 for Haiku.

iWSP models translate known algorithms into transformers. For algorithmic problems, the agent tends to implement known algorithms rather than conceiving of new ideas. For example, the edit distance task uses one block per row of the dynamic-programming table, and multiplication adds shifted partial products. For greatest common divisor (GCD), the subtractive Euclidean algorithm[62] uses the fact that any common divisor of AA and BB also divides A−BA-B, so replacing the pair with (∣A−B∣,min⁡(A,B))(\lvert A-B\rvert,\min(A,B)) preserves the greatest common divisor until the pair reaches (g,0)(g,0) (Figure 2d). iWSP implements this as a one block of three ReLU units, repeated 1,000 times (Figure 2e). Note that 100% performance on our eval tasks does not guarantee correct outputs on all inputs, there could be adversarial inputs not considered in the task or generalizations for which the network completely fails, e.g. our GCD test only specifies 3-digit inputs.

iWSP NLP models are crude but interpretable. Without a known algorithm, iWSP builds simple rules that approximate what an NLP task requires. For example, the best sentiment classification model has three stages, each built from standard transformer parts (Figure 2f). A shift register made of attention heads feeds one neuron for each of 1,515 hand-written word patterns, a second attention layer looks back for a negator and flips positive evidence that follows it, and a final layer adds up the evidence. The negator is a crude rule, but is effective at composing terms in a way that simple words/ngrams fail to.

Vetting interpretability: a 200-word description is sufficient to rebuild a model. Beyond qualitative analysis, we further run the text bottleneck check described in § 2 (Figure 1b). We find that all 13 rebuilt models (with 2 seeds each) reach 100% hidden accuracy on the algorithmic tasks, and on NLP tasks rebuilt models lose about 1.4 points of hidden accuracy on average (Figure 1c). We further audit that this is done properly: (i) no rebuilt model computes its weights from task data (either through training or corpus statistics), (ii) rerunning the weight-writing code of all 34 rebuilt models on altered task data leaves their weights unchanged, and (iii) no session read files outside its sandbox. The rebuilt models are also much closer to the original than a control that rebuilds models without a description. On algorithmic tasks, rebuilt models have 1.74 times the original’s parameters on a geometric mean, against 2,043 times for the control (Figure A2b). On NLP tasks, rebuilt models also perform closer to the original than the control does, e.g. their hidden-set predictions correlate significantly more on average (0.74 against 0.51; Figure A2c).

Figure A2. Rebuilding models from 200-word descriptions. For each of the 17 tasks, the best Opus 5.5 model is described in at most 200 words and rebuilt twice from that description (filled; blue, algorithmic; pink, NLP). A control session rebuilds it with no description (open). Without a description to follow, the control is not constrained to a compact or interpretable model, so it often writes far larger ones, such as hand-written rule lists with 207 million parameters for TREC that can score better than the original. Dashed lines mark equality with the original. a, Hidden accuracy. All 13 algorithmic originals, rebuilds and controls reach 100%. b, Parameters, on log scales. Rebuilt algorithmic models have 1.74 times the original’s parameters on a geometric mean, against 2,043 times for the control. c, d, How closely each model reproduces the original’s per-example behavior on the NLP hidden sets: the correlation between its errors and the original’s, and the share of outputs identical to the original’s, averaged over the two rebuilds. Algorithmic tasks are not shown because their original, rebuilt and control models are all exact, so they make no errors to correlate and their outputs are always identical. Averaged over the NLP tasks, the rebuilds’ error correlation exceeds the control’s by 0.22 (95% bootstrap interval over examples, 0.18 to 0.26). TREC has the weakest evidence because its hidden set has only 500 examples.

Neuroscience results: using iWSP to build brain encoding models

iWSP builds highly predictive, interpretable models of the visual cortex

In visual neuroscience, the most accurate encoding models are deep networks trained for object recognition, self-supervised learning, or image–text alignment,[9, 10, 11, 12, 63, 64, 65] whose learned weights make it hard to state what image features they compute. In contrast, classical models built from explicit operations such as Gabor filters,[66] divisive normalization,[67] scene gist,[68] or texture statistics[69] are interpretable but predict brain responses less accurately. Here we use iWSP to bridge this gap by searching for a neural network that builds interpretable features from images that are highly predictive of brain responses (§ 2).

Experimental setup. We run iWSP on LAION-fMRI[70] (7 T visual cortex responses from five subjects viewing 25,052 natural images) and hold out the Natural Scenes Dataset (NSD) for out-of-dataset testing (Figure 3a). Following § 2, each candidate maps raw images to fewer than 10,000 features using fewer than 100,000 explicit parameters, and a fixed ridge regression fit on each subject’s unique images is scored by Pearson correlation on shared validation images. Search maximizes mean correlation pooled across subjects and visual areas while also returning per-subject and per-area scores (such as V1, V2, V3, hV4, FFA, and PPA). Once search on all five LAION-fMRI subjects finishes, we freeze the best program (the iWSP model) and test it zero-shot on NSD. To test unseen subjects and images within LAION-fMRI, we also repeat the search on three subjects and half of the shared images, scoring its best model once on the other two subjects and 747 shared images. On both datasets, we compare iWSP against seven classical interpretable baselines (including Gabor filters with divisive normalization and Portilla–Simoncelli texture statistics[69]), a single ridge regression fit to all seven combined, and pretrained AlexNet[71] (Sec. A.2).

Figure 3. iWSP writes an interpretable visual encoding model that generalizes across subjects and datasets. a, The two fMRI datasets, with example stimuli. b, We run iWSP on five subjects using the data from LAION-fMRI and find that prediction performance (averaged over the early and ventral visual streams) improves over iterations. c, We then evaluate the final iWSP model on the held-out NSD dataset, and find that it outperforms interpretable baselines and slightly underperforms a black-box AlexNet baseline. Additionally, to test held-out performance on LAION fMRI, we rerun iWSP using only 3 subjects and test on the held-out 2 subjects and held-out images; we find the same performance patterns. d, We interpret the best model from the search in b, which consists of 14 computations (see Figure A3). We highlight five of the computations here, showing the filters and operations of one computation, its response to the example image (for channel correlations, the matrix of correlations between channel maps), and the share of each visual area’s prediction that it accounts for. Error bars in c and d show the standard error of the mean over subjects, which is often within the points images (early 0.518 versus 0.472; ventral 0.409 versus 0.309), with identical accuracy on hidden and visible images in the visible subjects (Sec. A.2).

iWSP builds from classical V1 filtering to 14 explicit computations. Across the 600 search iterations on LAION-fMRI, performance steadily increases (Figure 3b), with most of the gain occurring within the first nine iterations as the agents recover a classical V1 front end of multi-scale oriented Gabor filters and divisive normalization[67] on luminance and color-opponent channels. The final 865-line program produces 9,996 features from 14 explicit computations (Sec. A.2), five of which appear in Figure 3d: (1) Gabor energy and (2) color contrast compute locally normalized filter energy on luminance and red–green or blue–yellow channels; (3) surface geometry uses second derivatives of brightness to classify local shading into pits, valleys, saddles, ridges, and bumps; (4) contour curvature tracks how fast local orientation changes across space; and (5) channel correlations average pairwise products of feature maps to capture co-occurrence. Confirming that the program is readable, fresh sessions given only a 185-word summary rebuild models that reach 0.451 pooled correlation (versus 0.488) and match the original’s predictions more closely than AlexNet conv2, while rebuilds across eight search checkpoints retain 96% of original performance on average (Figure A5; Sec. A.2).

Figure A5. The visual cortex model after a text bottleneck. A model is described in at most 200 words and rebuilt from that description and the task, as in Figure 1b. Each score averages voxelwise Pearson correlations within a region and then across five LAION-fMRI subjects. Readouts are fit on each subject’s unique images and evaluated on the shared visible images; these are not held-out-subject evaluations. a, Pooled encoding correlation of the search’s best model at eight iterations (blue) and of two rebuilds of each model (pink dots). The dashed line marks normalized Gabor features. Open circles show the four rebuilds of the final model from the description in d. One rebuild at iteration 429 fails to run and is not shown. The model at iteration 1 and both of its rebuilds score 0.148, below the range of the axis. b, Correlations of the final model and of its four rebuilds from the description in d in six visual areas. Gray lines mark normalized Gabor features. c, Correlation between the predicted responses of a model (rows) and those of the rebuilds of each search model (columns) to the 1,492 shared images, computed per voxel and averaged over all voxels, over the rebuilds of a model, and over the five subjects. Rows list the seven search models, then seven classical baselines and three AlexNet layers ordered by mean correlation with the rebuilds. The outlined diagonal pairs each search model with its own rebuilds. d, The 185-word description of the final model, which Claude Opus 5.5 wrote from the model’s code, reproduced word for word.

iWSP generalizes to an unseen dataset and to held-out subjects. Evaluating the same iWSP hand-crafted features on NSD (Figure 3c, left) shows that they transfer across datasets and subjects. In every one of the four NSD subjects, iWSP beats all 7 interpretable baselines in both early (0.443 versus 0.400 for the best baseline) and ventral visual cortex (0.317 versus 0.256), and outperforms a single ridge regression fit jointly to all seven baselines (0.283 in ventral visual cortex). Compared with pretrained AlexNet, iWSP matches black-box accuracy in early visual cortex while trailing in ventral visual cortex (0.391). Repeating the search on three LAION-fMRI subjects (Figure 3c, right) replicates every comparison on the two held-out subjects and hidden

Different computations explain early versus category-selective visual areas. Because the ridge regression sums over features, each computation’s covariance with the measured response gives its exact percentage share of a voxel’s test correlation, summing to 100% in every area (Sec. A.2). Although search optimizes a single pooled score without anatomical labels, these shares shift systematically across the visual hierarchy (Figure 3d and Figure A3). Early areas rely more on local shading and edge energy: local-mean surface geometry falls from 38% in V1 to 15% in PPA, and local-mean Gabor energy drops from about 10% in V1–V3 to under 3% in FFA and PPA, showing that the final V1 ridge regression relies more on later-added surface geometry than on the initial Gabor stage. Category-selective areas instead draw more on mid-level structure, with contour curvature rising from 5% or less in V1–hV4 to 15% in FFA, channel correlations rising from about 3% in V1–hV4 to about 10% in FFA and PPA, and color contrast peaking in hV4 and FFA. Because correlated computations share 88% to 96% of the model’s squared correlation, we verify these profiles by refitting the ridge regression without each computation (Table A4): in all five subjects, local-mean Gabor energy uniquely explains 1.4% to 2.0% in V1–V3 and none in FFA and PPA, local-mean surface geometry uniquely explains 0.7% to 1.5% in every area, and channel correlations uniquely explain 4.9% in FFA and 1.2% in PPA versus none in V1–V3.

Figure A3. Share of each area’s prediction carried by each of the model’s 14 computations. Cells give the share in percent, averaged over five LAION-fMRI subjects; the shares in a column sum to 100%. Color shows a computation’s share in an area relative to its mean share over the six areas, and the one row with a mean share below 1% is left uncolored. Rows are ordered from computations that favor early areas to those that favor FFA and PPA. Bold rows appear in Figure 3d. The last column gives the number of output features of each computation.

ComputationV1V2V3hV4FFAPPA
Gabor energy, local mean2.0 ±\pm0.61.8 ±\pm0.51.4 ±\pm0.50.2 ±\pm0.3−-1.0 ±\pm0.2−-0.4 ±\pm0.1
Gabor energy, local variability0.2 ±\pm0.10.4 ±\pm0.10.5 ±\pm0.10.7 ±\pm0.20.4 ±\pm0.21.1 ±\pm0.2
Blobs and luminance0.2 ±\pm0.00.1 ±\pm0.00.0 ±\pm0.0−-0.1 ±\pm0.1−-0.2 ±\pm0.1−-0.3 ±\pm0.1
Color contrast0.5 ±\pm0.10.5 ±\pm0.10.4 ±\pm0.20.3 ±\pm0.2−-0.5 ±\pm0.70.0 ±\pm0.2
Surface geometry, local mean1.5 ±\pm0.11.4 ±\pm0.11.4 ±\pm0.21.3 ±\pm0.11.3 ±\pm0.20.7 ±\pm0.1
Surface geometry, local variability−-0.3 ±\pm0.1−-0.1 ±\pm0.20.3 ±\pm0.21.0 ±\pm0.10.8 ±\pm0.3−-0.8 ±\pm0.5
Contour curvature0.0 ±\pm0.1−-0.1 ±\pm0.0−-0.2 ±\pm0.1−-0.3 ±\pm0.10.8 ±\pm0.50.6 ±\pm0.2
Line versus edge phase0.1 ±\pm0.10.0 ±\pm0.1−-0.1 ±\pm0.1−-0.1 ±\pm0.2−-0.5 ±\pm0.30.1 ±\pm0.0
Second-order filtering−-0.1 ±\pm0.0−-0.1 ±\pm0.00.0 ±\pm0.00.1 ±\pm0.10.2 ±\pm0.1−-0.1 ±\pm0.1
Channel correlations−-0.1 ±\pm0.1−-0.1 ±\pm0.10.0 ±\pm0.10.7 ±\pm0.24.9 ±\pm0.61.2 ±\pm0.2
Cross-scale relations−-0.2 ±\pm0.0−-0.1 ±\pm0.0−-0.1 ±\pm0.00.0 ±\pm0.10.8 ±\pm0.20.9 ±\pm0.3
Geometry correlations0.0 ±\pm0.10.0 ±\pm0.10.1 ±\pm0.10.6 ±\pm0.23.0 ±\pm0.31.7 ±\pm0.3
Color layout0.4 ±\pm0.10.4 ±\pm0.10.4 ±\pm0.10.3 ±\pm0.10.9 ±\pm0.21.5 ±\pm0.1
Symmetry and blob count−-0.1 ±\pm0.10.0 ±\pm0.00.2 ±\pm0.10.2 ±\pm0.20.8 ±\pm0.20.2 ±\pm0.1
Not unique to one computation96.0 ±\pm0.395.9 ±\pm0.195.6 ±\pm0.195.2 ±\pm0.388.2 ±\pm0.793.5 ±\pm0.5

Table A4. Table A4. Unique variance of each computation of the visual cortex model. Values give the drop in squared correlation when ridge is refit without one computation, as a percentage of the full model’s squared correlation in each area (mean ±\pm standard error over five LAION-fMRI subjects). A negative value means that the ridge regression predicts the area better without the computation. The last row gives the variance that is not unique to any one computation.

iWSP identifies a data confound in story-listening fMRI

We next apply iWSP to a language fMRI dataset, in which subjects listen to narrative stories and the agent builds a model to predict responses from the words. iWSP rewards anything that raises its prediction performance, including confounds. Because iWSP models are interpretable, we can use this as an advantage: by reading the best-performing iWSP model, we identify a data confound at the edge of each fMRI scan that, once understood, can be removed.

Experimental setup. Encoding models of fMRI responses to natural language are often used to study language processing in the brain.[6, 16, 17, 20, 72, 73, 74, 75, 76, 77] We use data from two recent studies,[78, 79] with fMRI responses from three subjects listening to more than 20 hours of narrative stories from podcasts. We analyze one subject (UTS03) across 84 story scans, each 157 to 521 volumes long at one volume per 2 s (one TR). Each scan covers 95,556 voxels. The preprocessed release z-scores every voxel within each scan and has already removed 20 TRs per scan. The search fits ridge regression on 8 stories and evaluates it on 3 other stories, which form the visible evaluation set: the agent sees this performance at every iteration. After the search, we also test selected models on a hidden evaluation set of 10 other stories, drawn at random and never seen by the agent, with ridge regression still fit on the original 8 stories. Unless noted, performance below is visible performance.

We run iWSP with Claude Opus 4.7, and use a fixed evaluator that (i) aligns the features to the TRs, (ii) adds delays of one to four TRs for the slow hemodynamic response, (iii) fits ridge regression and (iv) returns the mean correlation across voxels on the visible stories. For every word in a story, the model is fed the word and the 10 words before it, and returns one feature vector per word, which the evaluator then aligns to the TRs. As a black-box baseline, we feed these same contexts to GPT-2 XL[80] and take the hidden state of the final token at layer 24 out of 48 (chosen via cross-validation over the layers 12, 24, and 36).

A data confound unrelated to language processing. For most of the iWSP run, features of word content improve slowly, scoring well below the 0.079 correlation achieved by GPT-2 XL (Figure 4a). Then, performance suddenly jumps from 0.067 to 0.104 in a single step. We find that the new features added at this step surprisingly do not encode a word’s semantic content, but rather only its position in the story. Further analysis shows that these position features alone reach about 0.090, above even the strong GPT-2 XL black-box baseline. To investigate these position features, we average the response across voxels and scans at each position in the scan, and find that it starts abnormally high and ends abnormally low compared to the rest of the scan (Figure 4b, black line); this signal appears in all three subjects (Figure A8). The position features closely track this signal: fit to half of the scans and tested on the other half, they follow this average response curve at a correlation of 0.97, and the jump they cause also appears on the hidden set (Table A6).

Figure 4. A jump in performance during iWSP iterations identifies a dataset confound. a, After growing gradually, the performance jumps in a single step by adding features that code for a word’s position in the story rather than its semantics. b, Debugging this feature identifies that the beginning/end of each scan has a characteristic pattern unrelated to the content of the stories (black line). Predicting using just position features (blue line) can track this measured response well, providing potentially misleading improvements for models that are not explaining genuine signal in brain activity.

IterationModelVisibleHiddenHidden, edges removed
GPT-2 XL0.0790.0460.047
1BoCharMean-d2560.0310.0160.017
40MultiScaleSubword+FuncWordTags0.0400.0200.021
76AllNewFeatWords150.0440.0220.023
152ExpDec-R4.00.0440.0220.023
192DenseSemXX-N200.0560.0260.027
238 (before jump)WordNetMorphLingMultiTau0.0670.0320.030
239 (after jump)WordNetMorphLingStoryArc0.1040.0430.023
242 (final)WordNetMorphLingNovelty0.1140.0480.026

Table A6. The performance jump due to the scan-timing features generalizes to the hidden set. We evaluate GPT-2 XL and several models from the search on all TRs: the best model so far at roughly evenly spaced iterations, the last model before the jump, the first after it and the final model. Ridge regression is always fit on the search’s 8 stories. The visible column gives the performance logged during the search, and the hidden column gives the mean voxel correlation on 10 other stories drawn at random. The last column repeats the hidden measurement without the 30 TRs at each end of every scan, with no refitting. Hidden performance is lower than visible performance for every model, including GPT-2 XL, because visible responses are averaged over repeated presentations of a story while each hidden story was presented once, not simply because of poor generalization.

Figure A8. The scan-timing signal appears in all three subjects. We plot the mean response across all voxels, averaged over each subject’s 84 story scans and aligned to the start and end of each scan, with the 30 TRs nearest each end shaded. In every subject, the first five TRs are above zero in at least 83 scans and the last five are below zero in at least 83 scans. The average curves correlate at 0.98 or higher between subjects.

This signal is potentially problematic because it follows scan timing rather than story content, potentially providing misleading changes in measured performance for different encoding models. This signal is not entirely unknown: the dataset release removes 10 TRs at each end of every scan to avoid onset transients and poor detrending,[78] and Antonello et al. noted that responses in some voxels drift over the first minute of each scan,[17] which lets long-context features inflate measured performance by up to 20%. To correct for this, they excluded the first 100 seconds of each test story when evaluating, a convention later adopted by other work.[81, 82] Our results suggest that for future studies that are sensitive to context lengths or story position, they should go further in addressing this confound by trimming the edges of each scan (in training as well as testing) by roughly 40 TRs. Note that this dataset remains of exceptional quality and that the many valuable conclusions drawn from it (which essentially all use word-level or short-context encoding models) stand; identifying this confound is more preventative, as non-experts and agents continue doing analyses on this type of data in the future.

Interpreting the fitted iWSP model. After this trimming, we find that iWSP can still build interpretable models that focus on semantics, performing on par with GPT-2 XL embeddings (Figure A6). Each feature of these models is readable and has its own ridge regression weights, so it can be traced onto the cortex; Figure A7 shows one feature that encodes words related to mental processing or speech. As in the previous sections, we again describe the iWSP models (from 8 points along the iWSP iterations) in a text description of at most 200 words. We find that this is sufficient for a fresh agent to rebuild each of them closely enough to achieve similar performance, including the story-position features of the first model after the jump (Figure A9).

Figure A6. With the start and end of each scan removed, a readable word model slightly passes GPT-2 XL. a, We remove 30 TRs from each end of every scan and run a separate Claude Opus 4.7 search, which climbs gradually over 3,949 iterations with no jump and reaches 0.0851, against 0.0826 for GPT-2 XL. Gray points are iterations and the blue line is the best visible performance. b, The best model is a hand-written lexicon, shown here on a new sentence. It looks each word up in 43 semantic categories, then uses attention to pool these tags over each 10-word window at four time-scales. c, The agent hand-tuned the weight of each category, with the largest weights on categories such as animals, intensifiers and perception (blue in b). It also added 10 pairs and 23 triples of categories. d, The lexicon does not win everywhere: it beats GPT-2 XL in IPS, EBA and RSC but falls behind in auditory cortex and in the speech regions sPMv and Broca’s area.

Figure A7. One feature of a written language fMRI model and its ridge regression weights. The feature comes from an earlier model of the iWSP search with the start and end of each scan removed (Figure A6; visible performance 0.081) and encodes words related to mental processing or speech. a, The feature on the input text “we talked and i thought”. The model tags each word using word lists written by iWSP, and each tag becomes a 1 in its own embedding dimension; rarer content words are upweighted, here ×36\times36 for “thought” and ×12\times12 for “talked”. Weighted attention averages each tag over the 10-word window, with earlier words weighted more, giving shares of 0.08 for mental and 0.10 for speech words. A single ReLU neuron adds the two shares with weights of +1 and a bias of −0.05, so it outputs zero unless their sum passes 0.05; it outputs 0.13 here, and 0 for speech alone in “she said it was over” (share 0.049). b, Ridge regression weights of this feature for subject UTS03, summed over the four delays and normalized by each voxel’s typical weight across all features, for voxels with visible correlation above 0.1. Each feature of the model enters ridge regression as its own column, so every feature can be mapped onto the cortex in the same way. The weight is positive in 61% of the 10% best-predicted voxels. Regions: auditory cortex (AC), Broca’s area (Broca), superior premotor ventral (sPMv), extrastriate body area (EBA) and intraparietal sulcus (IPS).

Figure A9. A 200-word description is enough to rebuild the language fMRI models. We apply the text bottleneck of Figure 1b to the eight models of Table A6 from the iWSP search on all TRs: the first model (iteration 1), the best model so far at iterations 40, 76, 152 and 192, the last model before the jump (238), the first model after it (239) and the final model (242). A fresh Claude Code session with Claude Opus 5.5 reads each model’s code and describes it in at most 200 words. Two further fresh sessions then rebuild each model from its description alone, under the same rules as the original search (no training and nothing fit to the fMRI responses), with at most 6 visible-set experiments each (to help ensure the network successfully compiles and adheres to the description) and no access to the hidden stories. a, b, Mean voxel correlation of each original and its two rebuilds on the hidden and visible stories, with ridge regression fit on the search’s 8 fitting stories as in Table A6; the dashed line marks equal performance. Rebuilds keep 87% of the original’s hidden performance on average, from 75% for iteration 192 to 98% for iteration 1, and both rebuilds of iteration 239 recreate its story-position features. Models from the middle of the search lose the most, since they rely on long hand-made word lists and hashed word identities that a description can name but not list. c, Parameters of each original and its two rebuilds, counted as every number a model stores (tensor and array elements and numeric constants; word lists are not counted), on log scales; counts of 0 are drawn at 1. Rebuilds of iterations 1 to 192 have 2 to 26 times fewer parameters than their originals, and the models from iteration 238 on, built from word lists, store at most 13 numbers. d, Correlation between the predicted hidden-story responses of each original and of the rebuilds of each model (averaged over the 2 rebuilds), computed per voxel and averaged over the original’s 10% best-predicted hidden voxels. Each off-diagonal cell averages the two pairings of its models (original ii with the rebuilds of jj, and original jj with the rebuilds of ii); the outlined diagonal pairs each model with its own rebuilds. The diagonal averages 0.78, against 0.55 off the diagonal (p<0.001p < 0.001, exact permutation test over all 40,320 ways of pairing the originals with the rebuilt models). The eight models come from one search and share many features, and iterations 238, 239 and 242 are at most four iterations apart, yet the diagonal is the largest value in its row for 4 of the 8 models and the largest or second-largest for 7 of them. e, f, Two of the eight descriptions, reproduced word for word. All eight descriptions, the rebuilt models, the agent logs and the evaluation code are on GitHub.

iWSP identifies interpretable, performant features for predicting ECoG responses to audio

We next study ECoG recordings of responses to natural audio, where we compare iWSP with embeddings from Whisper,[83] a standard, pretrained speech model, which is established as a highly performant feature space for encoding. We find that interpretable raw-waveform features from iWSP reach similar prediction performance as Whisper, and that they can explain part of the response that Whisper fails to predict.

Experimental setup. We predict high-gamma activity at 30 Hz from 166 intracranial electrodes in one patient, recorded continuously over 7 days.[84, 85] The stimulus is the raw audio from the room, cut into segments of about two minutes. Ridge regression is fit on 500 segments (about 16 hours of data). The visible evaluation set is 23 consecutive segments (about 44 minutes), during which a podcast was playing: the agent sees the prediction performance on this set at every iteration, and uses it to make inclusion and exclusion decisions about its feature choices. The hidden evaluation set is another 317 speech-rich segments from the same session (about 10 hours of data).

We run iWSP with Claude Opus 4.7, editing a model that maps the audio waveform to features. A fixed evaluator adds 4 time-lagged delays, shifts the features by 100 ms, fits a ridge regression for each electrode and returns the mean correlation on the visible evaluation set. Whisper-large-v3 encoder features go through the same evaluator. As an interpretable baseline, five isolated Claude Opus 5.5 sessions each write, in one shot and without performance feedback, the feature space an expert would assemble from standard acoustic features (spectrograms, cepstral coefficients, pitch and loudness), using any open-source library. These textbook features also go through the same evaluator. The search ran 902 iterations, plateauing after about 500 iterations. Partway through the loop, we switched to optimizing the 125 surface-grid electrodes, where Whisper’s advantage was concentrated. We still evaluate on all 166 electrodes using the hidden set.

iWSP on its own slightly underperforms Whisper. On the hidden set, using the grid electrodes, iWSP starts at 0.039 with a plain gammatone cochleagram and ends at 0.074, below the 0.082 of Whisper (Figure 5a). The 3,852 features in the final model are combinations of known, interpretable kernels, and each gain comes from hand-implementing the following: compression and adaptation, spectro-temporal modulation filters,[86] slow context over the previous seconds, and wavelet scattering[87] (Figure 5b). On the hidden set, over all 166 electrodes, iWSP reaches 0.065 against 0.070 for Whisper and leads on 77 electrodes. A 197-word description of these features is enough for fresh agents to rebuild the model with nearly the same hidden-set performance (Figure A12). The search goes beyond the textbook features: the iWSP model passes them early in the search, while it adds the modulation filters, and stays well above them (Figure 5a, Figure 5d). It also outperforms them on all five more patients described in Sec. A.4.

Figure 5. Written audio features predict ECoG well and predict signal that Whisper fails to capture. a, On the hidden set, using the grid electrodes, iWSP passes textbook features early in the search but stays below Whisper. Dots show each iteration, the line follows the model the search kept (its best on the visible set so far), and the band spans five textbook feature spaces that Claude Opus 5.5 wrote in one shot without feedback, with their mean dashed. b, The features underlying the iWSP model are combinations of known, interpretable kernels, e.g. a filter bank, smoothing over the past 2 s (blue), past 5 s (gray) and next 2 s (dashed), Gabors at 2, 4, 6 and 10 Hz, and a 2-D Gabor. c, On the hidden set, these features capture variance that Whisper does not. Each region gives the share of the combined model’s variance explained only by iWSP, by both, or only by Whisper, taken as the mean signed squared correlation over all 166 electrodes. d, Hidden-set correlation on the surface grid, where the textbook features were scored; combining iWSP and Whisper in one ridge regression improves on Whisper alone. e, This improvement happens across many electrodes. Each point is one electrode’s hidden-set correlation, on log scales; points above the dashed line improve when the iWSP features are added to Whisper.

Rebuilding-session performance, predictions, feature dimensions, and the description given to the rebuilding sessionsVector (PDF)

3,852 dims, win=4096 LPτLP_{\tau} = causal one-pole lowpass over frames (α=1/(1+τ)\alpha= 1/(1+\tau)); "dyn" = x − LPτLP_{\tau}, τ∈{4.5,15,60}\tau\in\{4.5, 15, 60\}. All concatenated. Cochleagrams. Hann power spectra, centred 1024/2048/4096-sample sub-windows, weighted by 20 fourth-order gammatone power responses (ERB-spaced, 50–8000 Hz) → log. Add E − LPτLP_{\tau} for τ∈{1.5,4.5,15,60,150}\tau\in\{1.5, 4.5, 15, 60, 150\}; tonic LP60LP_{60} and LP150LP_{150} of log and cube-root energies, acausal LP60LP_{60}; past and future differences over 3/6/12 frames, plus rectified halves. Spectrotemporal modulation. 2-D Gabor quadrature energy on the 4096 cochleagram (17 rate/scale pairs: 0–12 Hz, 0–±0.5 cycles/channel). On a 40-channel cochleagram: rates 0.5–12 Hz × scales {0, ±0.25, ±0.5}, cosine, sine, magnitude averaged over frequency thirds; dyn, acausal residuals cross-rate energy products. Descriptors (4096 spectrum; each with dyn): 8-harmonic log-comb pitch salience (11 F0s, 75–450 Hz), best F0/strength, ACF HNR, cepstral peak prominence, spectral moments/flatness/rolloff, octave and channel-group spectral contrast, softmax formants, spectral flux, voicing-gated cochleagram. Plus 2/4/6/10 Hz temporal-Gabor envelopes of these scalars. Interactions. Key scalars, their dyn, pairwise products (phonetic band contrasts × pitch/loudness/flux/formants), z-scored within the segment and expanded as [z]₊, [z]₋, z³. Within-window scattering. Morlet scattering (Q=4 then Q=1) on 9 channels plus instantaneous frequency/coherence.

Figure A12. Figure A12. A 197-word description is enough to rebuild the ECoG model. A fresh session describes the final iWSP model in at most 200 words; five sessions rebuild it from the description, and five control sessions get the same task with no description, each once and without feedback. a, Performance on a hidden recording session of each patient, using the grid electrodes, for the original (vertical line), the five rebuilds (filled) and the five controls (open). b, Correlation between each model’s hidden-set predictions and the original’s, for patient 1; bars mark means. c, Feature dimensions (log scale); the dashed line marks the original’s 3,852. d, The raw description.

iWSP predicts signal that Whisper misses. On the hidden set, the iWSP features capture unique variance that Whisper does not: they alone explain 6% of the combined model’s variance, Whisper alone explains 25% and the two share 69% (Figure 5c). As a result, combining both in one ridge regression significantly improves on Whisper alone, raising performance from 0.070 to 0.075 (this gain of 0.0052 has a 95% interval of 0.0039 to 0.0064 from a bootstrap over electrodes). The improvement happens across many electrodes (Figure 5e): 126 of 166 improve and 40 get worse. On five more patients, each evaluated on a hidden recording session, Whisper outperforms the iWSP model on every patient, but the iWSP features still explain variance that Whisper misses in four of the six (Figure A11).

Figure A11. Figure A11. The written ECoG features on six patients. Every model of the search, designed on patient 1, is rescored on a hidden recording session of each patient, using the grid electrodes. a, Performance of the model the search kept (its best on patient 1’s visible set so far) over iterations; numbers mark patients. b, The final iWSP model, Whisper, and both in one ridge regression. c, Variance of the combined model explained only by the iWSP features, by both, and only by Whisper, as a share of the combined model’s explained variance.

Which features are important. Because every feature has a formula, we can ask which ones the model relies on (Figure A10). On the visible set, the feature families overlap heavily, and 87% of the variance explained in the brain responses cannot be assigned to any one of them. The cochleagram with slow context and a set of modulation filters predict the most unique variance, and one feature, spectral flatness, reaches 29% of the full model alone.

Figure A10. Variance across families of written audio features. We use all 166 electrodes and the final model. Reading its code revealed redundant columns and gates that never opened, so we removed them first (3,852 to 3,392 features, performance 0.0808 to 0.0805). The full model explains 1.58% of the variance, measured as the mean signed squared correlation over electrodes. a, Variance explained by a ridge regression on one family alone, as a share of the full model, with feature counts in parentheses. b, Variance lost when we remove one family from the full model, again as a share of the full model. Bars show 95% intervals from a paired bootstrap over the 23 visible segments, with intervals above zero in blue. These shares sum to 13%, so the remaining 87% is shared across families.

Related works

Inherently interpretable models. A long line of work aims to build highly accurate models without sacrificing interpretability.[19, 88] In text, this began with word-level and ngram-level models,[20, 89, 90, 91] which have continued to improve by using modern techniques to improve their construction.[21, 22, 24] At the same time, concept-based models have become popular by routing predictions through a bottleneck of interpretable concepts.[23, 25, 26, 27, 28, 29, 30] However, the constrained form of these models has struggled to make accurate predictions for many hard problems, which iWSP aims to solve.

Agents that search for programs. Iterative coding agents offer a way around constrained model forms, because they can search a far larger space of readable models than traditional interpretable models can. AlphaEvolve and related works[31, 32, 33] iteratively write programs to solve particular problems in domains like math, and have been applied in domains like human and animal learning from data,[35] or more broadly in automating the entire scientific pipeline[36, 37, 38, 39, 40, 41] (although benchmarks and critiques question their reliability[92, 93]). iWSP follows a similar loop to these methods, but instead focuses on writing model weights and features.

Translating programs into weights. The systems above write programs rather than neural network weights. A separate line of work connects the two by compiling programs into networks. RASP is a small language whose primitives map onto attention and MLP operations,[42] and Tracr compiles RASP programs into transformer weights.[43] Learning Transformer Programs goes the other way, training transformers constrained to decompile into readable code, or decompiling them afterward.[18, 94, 95] Unlike these methods, iWSP produces networks without an intermediate domain-specific language or compiler.

Networks that generate weights directly. Weights can also be written by another network instead of an agent. Hypernetworks are networks that write the weights of other networks,[44, 45, 50] and HyperNEAT evolves a compact network whose outputs encode the weights of a much larger one.[46] Later work generates weights from collections of trained models, by sampling from learned representations of model zoos,[47] by running a diffusion model on checkpoints[48] or by predicting parameters for unseen architectures.[49] Weight-space architectures respect the symmetries of these inputs.[96] Like iWSP, these methods can set a target network’s weights without running gradient descent on it, but they generally fail to provide explanations for their weight choices.

Post-hoc interpretation. All of the work above builds models, whereas the related field of post-hoc interpretability explains models that are already trained. Attribution methods estimate how much each input contributes to a prediction.[97, 98, 99, 100] Mechanistic interpretability instead reads a model’s internals[52, 53, 101, 102, 103, 104, 105, 106] and reverse-engineers small trained models.[107, 108] LLMs and agents now automate much of this work by explaining in language what parts of a network do.[109, 110, 111, 112, 113, 114, 115, 116, 117] In contrast, iWSP starts from explanations and uses these to implement and test weights.

Discussion

The main advance in this work is a method to build models that are both predictive and interpretable: instead of training a model and explaining it afterward, an agent writes it by hand and revises it based on its measured performance. By applying iWSP across domains, we show that it can produce readable transformers for algorithmic tasks, and fairly accurate encoding models in predicting brain responses to different modalities. More broadly, iWSP materializes explicit hypotheses as model weights that can be read, tested and revised, offering a path to align the explanations that science seeks with the predictive power of data-driven models.

In neuroscience specifically, the field has always been concerned with finding the features of stimuli that selectively drive activity in the brain. Traditionally this problem has been approached by manually searching for these stimulus properties across multiple experiments [118, 119] while data-driven methods [13, 14, 15, 17, 120] have explored a much larger model space, but sometimes failed to translate their results to human-understandable findings. iWSP enables a more grounded, prior-based search to draw interpretable conclusions from data.

An important ancillary benefit of iWSP in neuroscience will be to help make datasets usable for future data-driven discovery. The signal we found at the start and end of each scan in the language dataset (§ 4.2) echoes the usefulness of earlier work on inherently interpretable models. For example, in a past study,[121] a generalized additive model predicting pneumonia risk identified that asthma lowered predicted risk of death, because patients with asthma received more aggressive care, helping identify issues with confounding that can be used to correct a model. As data-driven methods become increasingly powerful, interpretability will be more important in identifying and mitigating such confounds (and reward hacking more generally [3, 122, 123]).

Limitations. Despite these strengths, iWSP has important limitations in both the models it discovers and the evidence they provide. First, current agents do not invent genuinely new computational principles. Instead, they explore and combine known motifs drawn from their prior knowledge, such as textbook algorithms on synthetic tasks (§ 3) or established cortical operations like Gabor filters, divisive normalization, and modulation filters on brain data (§ 4.1 and § 4.3). Systematically combining known motifs is already productive, but proposing creatively distinct, unfamiliar hypotheses remains beyond current models. Relatedly, the discovered models rely on shallow motifs rather than deep, multi-layer hierarchies: our brain encoding searches assemble parallel banks of one- or two-stage computations into a linear readout, and our NLP transformers stack only a few rule-based stages. Programming deep hierarchical representations without backpropagation remains an open problem, which likely explains why iWSP matches pretrained networks in early visual cortex while trailing deeper models in ventral visual cortex, harder language tasks, and speech ECoG. At the method level, iWSP relies on an agent’s written explanation remaining faithful to its weight edits; while our audits and text-bottleneck rebuilds (§ 3 and § 4.1 to § 4.3) support this for current frontier models, stronger agents will require continued vetting to ensure they do not embed opaque heuristics. Running search on neural datasets is also slow and costly, requiring hundreds of candidate evaluations and a large token budget, although these costs continue to fall as LLMs become faster and cheaper. Finally, our neuroscience experiments are observational: they include relatively few subjects, test generalization within similar populations, and lack causal interventions. In visual cortex, for instance, area attributions and driving images reflect the specific hypotheses written into one 865-line model on the data that guided its search, showing which explicit computations predict responses rather than identifying unique causal mechanisms in the brain.

Future work. The work here opens the door to a great deal of future research. One idea is to use iWSP to explain black-box models by approximating them with an interpretable network: this could help build a mechanistic understanding of when they work/fail, improving over modern toy models used in mechanistic interpretability.[43, 107, 108, 124, 125] In data-scarce domains (often the case in neuroscience and medicine), another path is to leverage the explainability offered by iWSP to allow domain experts to help improve model adaptation.[28, 126, 127, 128, 129, 130] Finally, iWSP could also serve as an automated audit of widely used datasets, and could be applied to a much broader range of domains and tasks, including other neuroscience modalities and beyond. The capability of frontier models to do these weight-level tasks is exciting for domains like science, and is highly verifiable, making it possible to rapidly improve at this capability. We hope iWSP will help move toward a future in which the models we rely on for science are ones that we can interpret and trust.

Appendix

Transformers for algorithmic and NLP tasks

Scatter plot of best visible accuracy across TREC, Sentiment, Paraphrase, and Inference NLP tasks for Opus, Sonnet, and Haiku

Figure A1. NLP tasks across Claude models. We show the best visible accuracy of each run, counting only models whose weights do not depend on task data. The open square marks the Sonnet run that fit its models to labels, which we exclude.

TaskDescription
Word reversal (188)This model has no MLP: one causal attention read acts as a pointer, followed by a nearest-value lookup. Attention: one layer, one head, using hard argmax instead of softmax. Each position’s key is its 1-based position. The query at each generation step is the slot to copy. With i=position−11i = \text{position} - 11 (the "=" is position 11) and word=floor⁡(i/4)\text{word} = \operatorname{floor}(i/4), the query is 9+i−8⋅word9+i-8\cdot\text{word}. The score is negative squared distance between query and key, and the single best earlier position wins. So the answer copies prompt indices 8, 9, 10, 11, 4, 5, 6, 7, 0, 1, 2: word 3, separator, word 2, separator, word 1. Value and readout: the value is the token id as one scalar, capped at the separator’s id, so "=" reads as "_". That lets the first separator be copied from the "=" slot. Logits are the negative squared difference between the read value and each vocabulary id. An exact match scores 0; ties go to the lower id, "_". Errors: selection is exact with no softmax leakage, so accuracy should be 100%, even with repeated letters.
Boolean circuit (186)A tiny recurrent "cellular automaton" transformer. Each position holds one scalar state, the expression value so far, stored centred as S−0.5S - 0.5. One tied block (one attention head, 2-neuron ReLU MLP) is applied 10 times; each pass moves information one cell right, enough to reach "=". Attention has no key matrix: positions are sinusoidal angles at 30° per step, and the query is the position rotated back one step, scaled by 200, giving near-hard attention to the previous cell (position 0 attends to itself). Values copy that cell’s AND/XOR flags and put its state on the bit channel; each cell adds its own embedding. Op tokens carry their flag plus identity bit (1, &, 1 and ^, 0) so cells and "=" (zero embedding) pass the state through. The MLP does one fold step: new state = relu(b+S−Ab+S-A) - relu(2(b+S)+X−A−22(b+S)+X-A-2), giving OR with no flags, AND with A, XOR with X. State starts at 0; readout logits are ±4\pm4 times the centred state. Errors: expected none (attention leakage ≈−27\approx-27), though untested.
Multiplication (190)This model has no MLP and no LayerNorm: two attention-only layers on a 7-channel float64 residual stream whose registers hold integers up to about 1e101\mathrm{e}{10}. Layer 1 (one head) makes each B digit compute b:A. A query at a B digit b scores multiplicand position i at log⁡(b⋅10(i−1))\log(b\cdot10^{(i-1)}) against a huge zero-valued sink key at "*". Because the sink dominates (scale about 1e17), softmax acts like a plain exponential, so the head reads b·A’scale, rescaled to b·A. Digit 0 attends only to the sink. In layer 2, head 1 weights B positions proportionally to 10(10−t)10^{(10-t)}, multiplying the average by 11111 gives P=A⋅BP=A\cdot B. Head 2 uses the same sink trick (scale about 1e221\mathrm{e}{22}) to read V, the value of already-written answers digits times their place values, and subtracts it, so X=P−V+0.5X=P-V+0.5, a long-division remainder. The readout is a tropical decoder: logit⁡k=k⋅X−place⁡(k+1)/2\operatorname{logit}_k = k\cdot X-\operatorname{place}(k+1)/2 with place = 10(20−t)10^{(20-t)} at answer position tt, whose argmax is floor(X/place). Expect essentially 100% exact match; leakage is about 1e-12 relative, so float64 is required (float32 breaks). One wrong digit corrupts the remainder and all later digits.
GCD (189)Tokens embed as one scalar (digit value; comma and "=" are 0) plus a fixed one-hot position code. The residual state has channels VAL, A, B, X. A six-head parser attention layer uses huge logits (about 40), so each head reads one operand digit, scaled by its place value (100, 10, 1). This gives exact integers A and B at answer positions. The comma attends to itself, keeping A=B=0 there as a "zero" slot. Euclid runs as residual ReLU MLPs on (A, B): 12 tied rounds of four folds with scales 64, 16, 4, 1. Each fold maps A to its distance from the nearest multiple of s·B. This triangular wave uses 4 ReLU knots at m·s·B/2 with alternating ±2\pm2 slopes and is valid only while A≤2sBA \le2sB, hence radix 4. The s=1 fold also swaps (A,B)→(B, fold). The result settles into a 2-cycle between (g,0) and (0,g), so A+B=g. Readout attention forms X=(A+B)/100X=(A+B)/100, /10 or /1 by mixing with the zero slot, minus 100×100\times and 10×10\times earlier emitted digits. The head scores digit d as −(X−d−0.495)2-(X-d-0.495)^2, flooring X. The file claims zero errors on all 10610^6 pairs.
TaskDescription
Smallest prime factor (181)Two-block transformer, no LayerNorm, float64, all signals scalar (not one-hot). It should be exactly correct for every nn from 2 to 99999, so there is no error pattern to match; aim for 100%. Block 1 attention: one head, constant query, keys (4−i)⋅ln⁡10(4-i)\cdot\ln10 for prompt digit ii, −60-60 elsewhere. Softmax weights become 10(4−i)/1111110^{(4-i)}/11111, so summing digit values times 11111 yields nn. Block 1 MLP (sieve): one neuron per prime up to 316 (65 primes) with input n/pn/p and pulse activation relu⁡(1+104(cos⁡2πu−1))\operatorname{relu}(1 + 10^4(\cos2\pi u - 1)), firing only at integers; non-divisors are at least 1/3131/313 away. Lateral inhibition (2⋅2\cdot pulse minus cumulative sum over smaller primes) keeps only the smallest firing prime, which writes its three digits. Block 2 attention: Gaussian copy head, score −40(i−(s−5))2-40(i-(s-5))^2, so output position ss reads prompt digit s−5s-5. Block 2 MLP: ReLU gate. Any nonzero factor digit suppresses the copy (weight −9-9); flags on the last three output steps release the factor’s hundreds, tens and units digits. No factor means nn is prime and gets copied. Readout: logit⁡d=2d⋅A−d2\operatorname{logit}_d = 2d\cdot A - d^2 picks the nearest digit; "=" is blocked.
TREC (196)A hand-built keyword decision list with no training. Block 1: each character is embedded as a 5-bit letter code with flags for word boundary, start and padding, plus position. Head hh of 20 copies the character hh steps back using a Gaussian position kernel. Each MLP neuron is an exact AND gate for one of about 400 keyword patterns: words, stems, or question-initial words. Block 2: ’=’ attends to the highest salience, 300 × priority minus 3 × position, so the earliest match wins ties. It copies that position’s class vote. If nothing matches, the answer is ENTY. Priority order: ABBR for "stands for" and "abbreviat"; DESC for "comes from"; then initial "who" (HUM), "where" (LOC), initial "when" (NUM) and "why" (DESC); definition or cause words such as mean, origin and reason (DESC); "how did/does/is" (DESC); other "how" (NUM); "what is X" questions under 17 characters (DESC); topical nouns, with people HUM, places LOC, measures NUM and things ENTY; "called" and "what kind" (ENTY); "what are the" (ENTY); and last, initial "what is" (DESC). Errors come from brittle surface matching: missing keywords, the wrong noun winning, and long "what is the" questions falling through to DESC.

Table A3. Six of the 17 descriptions given to the rebuilding sessions, chosen at random. Claude Opus 5.5 wrote each one from the model’s code, and we reproduce them word for word, with word counts in parentheses. The GCD model here is the best Opus 5.5 model, which is more compact than the GCD model in Fig. 2e. All 17 are on GitHub.

Computations of the visual cortex model

The iWSP model cascades a V1-like filter bank into five explicit mid-level modules. The iWSP model is a Python program of 865 lines with no learned weights that mirrors the hierarchical organization of biological vision. At a high level, the model first passes the image through a shared V1-like frontend to extract normalized local edge and contrast maps, branches into five parallel mid-level modules that capture higher-order structure, and finally pools these maps over a foveated spatial grid. In the frontend, pixel intensities undergo power-law compression and are split into one luminance and two color-opponent channels (red–green and blue–yellow). Each channel is convolved at three spatial scales with center–surround filters and oriented Gabor filters spanning eight orientations. The responses are then passed through standard cortical nonlinearities: quadrature Gabor pairs are combined into phase-invariant complex-cell energy[131] (the square root of summed squared responses), center–surround responses are half-wave rectified into ON and OFF channels, and all channels undergo local divisive normalization followed by logarithmic compression to encode contrast relative to the surrounding image context.

From this normalized V1 representation, the model computes five kinds of mid-level features: (i) texture boundaries, which pass the first-stage energy maps through a second bank of Gabor filters and energy pooling (a classical filter–rectify–filter cascade) to detect where patterns of oriented contrast change; (ii) contour curvature, which takes spatial derivatives of the local orientation field to measure how rapidly edge orientation bends across space and relative to its surround; (iii) edge phase, which compares even and odd Gabor responses rectified by sign to distinguish thin bright ridges, dark creases, and step edges between surfaces; (iv) feature co-occurrences, which multiply pairs of centered, normalized channels and average them spatially to measure which orientations, colors, curvatures, and phases appear together within and across spatial scales, following classical parametric texture models[69]; and (v) surface geometry, which acts directly on smoothed luminance and color derivatives to classify local brightness surfaces into five 3D shading primitives (pits, valleys, saddles, ridges, and bumps) alongside corner strength and isophote curvature. Figure 3d shows the weights and operations of five of the model’s computations. Finally, every feature map is summarized by its mean and spatial standard deviation over a foveated grid of image windows (finer near the center of gaze) and normalized per block, yielding 9,996 explicit features. Aside from the analytic tuning curves used to categorize the five surface shapes, nearly every nonlinearity in the program reduces to one of five canonical operations from visual neuroscience: power/log compression, half-wave rectification, quadrature energy, divisive normalization, and pairwise channel multiplication.

Attributing each area’s prediction to computations. Each of the 9,996 output features is written by one stage of the program, and we group the features into 14 computations by that stage. Where a stage summarizes a map both by its average within the cells of a spatial grid and by its variability within those cells, we treat the two summaries as separate computations. Figure A3 lists the 14 computations with their feature counts. To attribute an area’s prediction, we fit the ridge regression once with all features, as in the search.

For one voxel, the prediction on the shared images is a sum over computations, y^=∑cy^c\hat{y} = \sum_c \hat{y}_c, where y^c\hat{y}_c uses only the features and ridge weights of computation cc. The correlation between the voxel’s prediction and the measured response yy then splits exactly into one term per computation:

corr⁡(y^,y)=∑ccov⁡(y^c,y)σy^σy.\operatorname{corr}(\hat{y},y)=\sum_c \frac{\operatorname{cov}(\hat{y}_c,y)}{\sigma_{\hat{y}}\sigma_y}.

We average each term over the voxels of an area, divide by the area’s mean correlation so that the shares sum to 100%, and then average over the five LAION-fMRI subjects. A share is negative when a computation’s part of the prediction is anticorrelated with the response. Correlated computations divide the credit between them, and the model scales its feature blocks differently, so a share describes what the fitted ridge regression uses, not what it needs. We therefore also measure each computation’s unique variance: we refit ridge without the computation’s features and report the drop in squared correlation as a percentage of the full model’s squared correlation (Table A4). The maps in Figure 3d use the finest image scale, and the matrix of channel correlations uses half that resolution, as in the model. The five surface-shape tiles in Figure 3d illustrate the brightness profiles that the code separates and are not filter weights. The matrix of channel correlations has 24×2424 \times24 entries, with positive values in blue and negative values in orange.

Search constraints and held-out evaluation. In every candidate, each filter and weight is set analytically or generated from a fixed random seed. Candidates are limited to fewer than 10,000 output features so that a model cannot succeed merely by handing the ridge regression an overcomplete random basis. On NSD, we evaluate the four subjects who completed all scanning sessions, fitting the linear readout on each subject’s unique images and testing on the shared images after excluding any stimulus that also appears in LAION-fMRI, which leaves 998 test images. The seven interpretable baselines are Gabor filters, Gabor filters with divisive normalization,[67] GIST,[68] HMAX,[132] wavelet scattering,[133] Portilla–Simoncelli texture statistics,[69] and a contour curvature model. For AlexNet, we use for each stream the layer that scores best on the three search subjects and visible images of LAION-fMRI, averaged over subjects and fixed before any held-out evaluation: conv3 for early and fc6 for ventral visual cortex. The same two layers are used for held-out LAION-fMRI and for NSD. For the combined baseline, we concatenate the features of the seven baselines (18,386 features in total) and fit the same ridge regression, with one penalty per voxel. Before concatenation, we center each baseline’s features and divide them by their root mean squared norm over the training images, so that the seven baselines enter the regression with equal weight; without this rescaling, the combined baseline scores lower in ventral visual cortex on both datasets. Across individual subjects, the iWSP model exceeds the strongest single baseline in every NSD subject by 0.035 to 0.054 in early visual cortex and by 0.057 to 0.064 in ventral visual cortex. On the two held-out LAION-fMRI subjects, the best model of the three-subject ablation search reaches 0.466 pooled correlation (0.518 versus 0.472 for Gabor filters with divisive normalization in early visual cortex, higher by 0.044 to 0.047 per subject; 0.409 versus 0.309 for Gabor filters with divisive normalization in ventral visual cortex, higher by 0.096 to 0.105 per subject). On the three search subjects of that run, the model scores identically on held-out and visible shared images (0.536 versus 0.536 pooled correlation), showing that the lower absolute correlation on the two held-out subjects reflects subject differences that affect baselines as well (Gabor filters with divisive normalization fall from 0.451 on the search subjects to 0.395 on the held-out subjects).

Area readouts and synthesized images. To see which images drive the model’s prediction for each area (Figure A4), we fit the ridge readout for every voxel of an area as in the search, average the voxel weight vectors within the area, and then average over the five LAION-fMRI subjects. The resulting area readout predicts the area-mean response, and the shared images are held out of every readout fit. It predicts the measured area-mean response to the shared images with a correlation of 0.82 in hV4, 0.53 in FFA, and 0.70 in PPA. The images with the highest predicted response are dense, multicolored objects for hV4, faces and people for FFA, and buildings and other rectilinear scenes for PPA; the 20 highest-ranked images of each area evoke measured responses 1.3 to 1.7 standard deviations above the area’s mean. Because the model is differentiable, we can also synthesize an image that maximizes each area readout. To synthesize an image, we start from noise and run gradient ascent on the area readout, with the image parametrized in the Fourier domain and shifted by up to 6 pixels at random on each step. We fix the image mean at 0.5 and the pixel standard deviation at 0.2, because without this constraint the optimum is a blank image. Two blocks of global image statistics (82 features) have no usable gradient and are held constant. The synthesized images reach predicted responses 6.5 (hV4), 9.6 (FFA), and 10.3 (PPA) standard deviations above the mean prediction for the shared images, where the highest natural image reaches about 2.5, so they show the direction the readout favors rather than a response the model has been tested on. The synthesized images agree with the natural ones: fine multicolored texture for hV4, a central skin-toned blob for FFA, and a lattice of horizontal and vertical lines for PPA. These images show which image statistics the model associates with each area; they do not show that the model represents faces or scenes as categories.

Images showing the top predicted and synthesized images for hV4, FFA, and PPA

Figure A4. Images that drive the model’s predictions for hV4, FFA and PPA. For each area, the four of the 1,492 shared LAION-fMRI images with the highest predicted response are shown next to an image synthesized from noise to maximize the predicted response. Synthesized images have a fixed contrast and reach predictions beyond those for any natural image, so they are extrapolations.

Rebuilding the visual cortex model from text. To test whether a short description preserves the iWSP model’s computations, Claude Opus 5.5 reads the model’s code and summarizes it in 185 words (Figure A5d). We give only that description and the task to four independent rebuilding sessions. Each rebuilt feature extractor receives a newly fitted ridge readout under the original five-subject LAION-fMRI protocol. Mean pooled correlation is 0.451 for the rebuilds and 0.488 for the original, and in each of six visual areas every rebuild scores below the original and above normalized Gabor features (Figure A5a, Figure A5b). The rebuilds recover 37% of the original’s correlation gain over normalized Gabor features (correlation 0.430), indicating partial reconstruction of the improvement beyond this baseline.

Similar accuracy does not show that the rebuilds compute the same thing, so we also compare what the models predict. For each voxel, we correlate two models’ predicted responses to the 1,492 shared images. The rebuilds’ predictions correlate with the original’s at 0.86 to 0.88, compared with 0.81 for pretrained AlexNet conv2, a model of similar accuracy (correlation 0.461), and 0.80 for normalized Gabor features. Because every model shares a large Gabor-like component, we repeat the comparison after regressing each prediction on the normalized Gabor prediction, voxel by voxel. On this remainder, the rebuilds correlate with the original at 0.64 to 0.69, against 0.50 for AlexNet conv2, and every rebuild exceeds AlexNet conv2 in each of the five subjects. Over each subject’s 10% of voxels best predicted by the original, the original correlates with the rebuilds at 0.94 (0.74 to 0.76 after the regression) and with AlexNet conv2 at 0.91 (0.57 after the regression). The description therefore carries computations specific to the original, but only in part: the rebuilds agree more with one another (0.77 to 0.83) than with the original, so they share a reading of the text that differs from the code. Two further comparisons do not favor the rebuilds. Across voxels, the gain of each model over normalized Gabor features correlates with the original’s gain at 0.61 to 0.73 for the rebuilds and at 0.73 for AlexNet conv2. The pattern of per-voxel accuracy correlates at 0.96 or above for every pair of models, including both references, so it does not distinguish them. Linear centered kernel alignment between original and rebuilt features ranges from 0.71 to 0.76 on 1,000 test images. The rebuilds are also smaller than the original, with 292 to 358 lines of code against 742 and about a third of its numeric constants (Table A5).

ModelLines of codeNumeric constantsOutput featuresPooled rr
Original7421,0049,9960.488
Rebuild 13552979,2060.447
Rebuild 22923119,3980.452
Rebuild 33583179,1650.454
Rebuild 43473439,5300.452
Rebuild average3383179,3250.451

Table A5. Size of the original and rebuilt visual cortex models. Lines of code count non-blank, non-comment lines (the original file has 865 lines). Numeric constants count the numbers written in the code other than 0 and 1. Output features are the columns passed to the ridge readout. Pooled correlation is as in Figure A5a.

The text bottleneck along the search. The comparison above uses only the final model, so we repeat the text bottleneck on the search’s best model at eight evenly spaced iterations: 1, 87, 172, 258, 343, 429, 514, and 600. As before, Claude Opus 5.5 describes each model in at most 200 words (188 to 200 used), and two independent sessions rebuild each model from its description and the task alone. We score the originals and the rebuilds under the five-subject LAION-fMRI protocol of the search. One of the 16 rebuilds, at iteration 429, fails when run on the GPU and is left out.

The first model is a random projection of the pixels (pooled correlation of 0.148), and both of its rebuilds reproduce its score exactly. Across the eight models, the rebuilds keep 96% of the original’s performance on average. For the seven later models they keep between 92% (iteration 514) and 98% (iteration 87), and every rebuild outperforms normalized Gabor features (correlation 0.430; Figure A5a). No rebuild reaches its original: the originals improve from 0.467 at iteration 87 to 0.488 at iteration 600, while the rebuilds stay between 0.445 and 0.470. Over the same iterations the program grows from 373 to 865 lines. A description of 200 words thus conveys most of each model, and the small improvements of the later iterations are the part it conveys least. The two rebuilds of the final model, which come from a new description, reach 0.456 and 0.470, in line with the four rebuilds above (0.447 to 0.454; open circles in Figure A5a).

We also ask whether the rebuilds of a model predict like their own original more than like the other checkpoints and standard baselines (Figure A5c). Comparing the seven sets of rebuilds against a 17-row reference set consisting of the seven later originals, the seven classical baselines, and AlexNet conv2, conv3, and fc6 requires no additional rebuilding runs. Each original correlates with its own rebuilds at 0.895 on average, versus 0.753 across all 112 off-diagonal pairs (p=4.7×10−7p = 4.7 \times10^{-7}, exact permutation test over all 98,017,920 ways of pairing seven of the 17 models with the seven sets of rebuilds). The ten baselines correlate with the rebuilds at 0.679 on average, from 0.520 for HMAX to 0.773 for normalized Gabor features and AlexNet conv2, and for every rebuilt checkpoint all seven search originals agree more closely with its predictions than any baseline does. Within the seven search checkpoints alone, off-diagonal agreement is 0.877 (p=0.009p = 0.009 over all 5,040 pairings among the seven originals) because checkpoints from one search share most of their computations; from iteration 258 onward, the originals agree almost equally with each set of rebuilds, and an original agrees most with its own rebuilds for 3 of the 7 models.

Story-listening fMRI

ECoG responses to audio

Transfer to five more patients. The written features were designed on one patient, so we test them on five more patients from the same dataset (Figure A11). For each patient, we rescore every model of the search with the same features, delays and ridge regression, fit ridge regression on 500 segments (about 16 hours) and evaluate on the grid electrodes of one hidden recording session (107 to 416 segments, 3 to 13 hours). For the design patient, the hidden set is its 493 segments of the same session without the speech filter of the main text, because the other patients have no transcripts. The early design transfers: on every patient, the models’ performance tracks the design patient’s (correlation 0.80 to 0.94 across models), and most of the gain comes within about the first 60 iterations, which build the cochleagram, compression and adaptation, and the modulation filters. The later refinements do not transfer: after iteration 170 they add 0.004 on the design patient but between −0.002 and 0.001 on the others. This suggests that some feature selection overfitting is possible in the limit when generalizing feature sets chosen by iWSP to other subjects. Whisper outperforms the iWSP model on every patient, by 0.001 to 0.013, yet the iWSP features explain variance that Whisper misses in four of the six patients (95% interval above zero), up to 17% of the combined model’s variance. The modulation filters are the only family whose unique variance is positive on every patient (2% to 7% of the full model).

A 200-word description is enough to rebuild the ECoG model. As for the other models (Figure 1b), a fresh session with Claude Opus 5.5 reads the final model’s code, with comments and run history removed, and describes it in at most 200 words (197 used; Figure A12d). Five fresh sessions rebuild the model from the description and the task alone, and five control sessions get the same task with no description. Every session runs once, in an isolated sandbox with no performance feedback, and we audit them as in § 3: no model fits anything to data, each produces identical features when the example audio changes, and no session reads files outside its sandbox. On the hidden sessions of the six patients, the rebuilds stay within 0.003 of the original’s performance, and on patient 2 they exceed it. They outperform the controls on five of the six patients and tie on the sixth (Figure A12a). The rebuilds’ hidden-set predictions correlate with the original’s at 0.63 on average, against 0.50 for the controls, and the rebuilds keep nearly all of the original’s 3,852 dimensions (3,769 to 3,852), while the controls write models with 382 to 1,478 (Figure A12b, Figure A12c).

Experiment costs

We give costs at API list prices, not counting the compute for running models and fitting ridge regressions. The exploratory rows for the algorithmic/NLP tasks and the story-listening fMRI searches estimate the earlier, non-final runs, which the paper does not report, at about double the cost of the final runs. The Haiku cost leaves out one run whose session was stopped before its cost was recorded. For the visual cortex experiments, the table lists the final search and the earlier exploratory runs separately; together they cost about $5,000. The ECoG cost is a lower bound: it covers the June agent runs, including the precursor and clean-slate searches and runs stopped at their time limit, but not the interactive sessions that launched the agents.

References

  1. [1]Warren S. McCulloch and Walter Pitts. A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5(4):115–133, 1943. doi: 10.1007/bf02478259. URL http://dx.doi.org/10.1007/BF02478259.
  2. [2]Bryce Goodman and Seth Flaxman. European union regulations on algorithmic decision-making and a “right to explanation”. AI magazine, 38(3):50–57, 2017.DOI
  3. [3]Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565, 2016.arxiv.org/abs/1606.06565
  4. [4]Iason Gabriel. Artificial intelligence, values, and alignment. Minds and machines, 30(3):411–437, 2020.DOI
  5. [5]Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethinking interpretability in the era of large language models. arXiv preprint arXiv:2402.01761, 2024.arxiv.org/abs/2402.01761
  6. [6]Shailee Jain, Vy A. Vo, Leila Wehbe, and Alexander G. Huth. Computational language modeling and the promise of in silico experimentation. Neurobiology of Language, 5(1):80–106, 2024. doi: 10.1162/nol_a_00101. URL http://dx.doi.org/10.1162/nol_a_00101.
  7. [7]Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. Can large language models transform computational social science? Computational Linguistics, 50(1):237–291, 2024.
  8. [8]Thomas Naselaris, Kendrick N Kay, Shinji Nishimoto, and Jack L Gallant. Encoding and decoding in fMRI. NeuroImage, 56(2):400–410, 2011.DOI
  9. [9]Daniel L K Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23):8619–8624, 2014.DOI
  10. [10]Santiago A Cadena, George H Denfield, Edgar Y Walker, Leon A Gatys, Andreas S Tolias, Matthias Bethge, and Alexander S Ecker. Deep convolutional models improve predictions of macaque V1 responses to natural images. PLoS Computational Biology, 15(4):e1006897, 2019.DOI
  11. [11]Chengxu Zhuang, Siming Yan, Aran Nayebi, Martin Schrimpf, Michael C Frank, James J DiCarlo, and Daniel L K Yamins. Unsupervised neural network models of the ventral visual stream. Proceedings of the National Academy of Sciences, 118(3):e2014196118, 2021.DOI
  12. [12]Talia Konkle and George A Alvarez. A self-supervised domain-general learning framework for human ventral stream representation. Nature Communications, 13:491, 2022.DOI
  13. [13]Shailee Jain and Alexander Huth. Incorporating context into language encoding models for fmri. Advances in neural information processing systems, 31, 2018.
  14. [14]Mariya Toneva and Leila Wehbe. Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain), 2019. URL https://arxiv.org/abs/1905.11833.
  15. [15]Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A. Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, Aren Jansen, Harshvardhan Gazula, Gina Choe, Aditi Rao, Catherine Kim, Colton Castro, Lora Fanda, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni, Roi Reichart, Sasha Devore, Adeen Flinker, Liat Hasenfratz, Omer Levy, Avinatan Hassidim, Michael Brenner, Yossi Matias, Kenneth A. Norman, Orrin Devinsky, and Uri Hasson. Shared computational principles for language processing in humans and deep language models. Nature Neuroscience, 25(3):369–380, 2022. doi: 10.1038/s41593-022-01026-4. URL http://dx.doi.org/10.1038/s41593-022-01026-4.
  16. [16]Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nature Human Behaviour, 7(3):430–441, 2023. doi: 10.1038/s41562-022-01516-2. URL http://dx.doi.org/10.1038/s41562-022-01516-2.
  17. [17]Richard Antonello, Aditya Vaidya, and Alexander Huth. Scaling laws for language encoding models in fmri. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, page 21895–21907. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2023. doi: 10.52202/072580-0958. URL http://dx.doi.org/10.52202/072580-0958.
  18. [18]Dan Friedman, Alexander Wettig, and Danqi Chen. Learning transformer programs. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, page 49044–49067. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2023. doi: 10.52202/075280-2131. URL http://dx.doi.org/10.52202/075280-2131.
  19. [19]Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206–215, 2019. doi: 10.1038/s42256-019-0048-x. URL http://dx.doi.org/10.1038/s42256-019-0048-x.
  20. [20]Alexander G Huth, Wendy A De Heer, Thomas L Griffiths, Frédéric E Theunissen, and Jack L Gallant. Natural speech reveals the semantic maps that tile the human cerebral cortex. Nature, 532(7600):453–458, 2016.DOI
  21. [21]Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, and Hannaneh Hajishirzi. Infini-gram: Scaling unbounded n-gram language models to a trillion tokens. arXiv preprint arXiv:2401.17377, 2024.arxiv.org/abs/2401.17377
  22. [22]Chandan Singh, Armin Askari, Rich Caruana, and Jianfeng Gao. Augmenting interpretable models with large language models during training. Nature Communications, 14(1):7913, 2023.DOI
  23. [23]Pang Wei Koh, Thao Nguyen, Yew-Sung Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pages 5338–5348. PMLR, 2020.arxiv.org/abs/2007.04612
  24. [24]Eunji Kim, Sriya Mantena, Weiwei Yang, Chandan Singh, Sungroh Yoon, and Jianfeng Gao. Interpretable next-token prediction via the generalized induction head, 2024. URL https://arxiv.org/abs/2411.00066.
  25. [25]Chung-En Sun, Tuomas Oikarinen, Berk Ustun, and Tsui-Wei Weng. Concept bottleneck large language models, 2024. URL https://arxiv.org/abs/2412.07992.
  26. [26]Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail, Giang Nguyen, Isaac Plant, Muawiz Chaudhary, Nathaniel Monson, Saqib Azim, Zhichen Guo, and Julius Adebayo. Scaling inherently interpretable language models, 2026. URL https://arxiv.org/abs/2608.07594.
  27. [27]Vinamra Benara, Chandan Singh, John X Morris, Richard Antonello, Ion Stoica, Alexander G Huth, and Jianfeng Gao. Crafting interpretable embeddings by asking llms questions. arXiv preprint arXiv:2405.16714, 2024.arxiv.org/abs/2405.16714
  28. [28]Jean Feng, Avni Kothari, Patrick Vossler, Andrew Bishara, Lucas Zier, Newton Addo, Aaron Kornblith, Yan Shuo Tan, and Chandan Singh. Human-ai co-design for clinical prediction models. arXiv preprint arXiv:2601.09072, 2026.DOI
  29. [29]Josh Magnus Ludan, Qing Lyu, Yue Yang, Liam Dugan, Mark Yatskar, and Chris Callison-Burch. Interpretable-by-design text understanding with iteratively generated concept bottleneck, 2023. URL https://arxiv.org/abs/2310.19660.
  30. [30]Zhen Tan, Lu Cheng, Song Wang, Bo Yuan, Jundong Li, and Huan Liu. Interpreting Pretrained Language Models via Concept Bottlenecks, page 56–74. Springer Nature Singapore, 2024. doi: 10.1007/978-981-97-2259-4_5. URL http://dx.doi.org/10.1007/978-981-97-2259-4_5.
  31. [31]Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, pages 1–3, 2023.
  32. [32]Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131, 2025.arxiv.org/abs/2506.13131
  33. [33]Bogdan Georgiev, Javier Gómez-Serrano, Terence Tao, and Adam Zsolt Wagner. Mathematical exploration and discovery at scale, 2025. URL https://arxiv.org/abs/2511.02864.
  34. [34]Chandan Singh, Yan Shuo Tan, Weijia Xu, Zelalem Gero, Weiwei Yang, Michel Galley, and Jianfeng Gao. Agentic-imodels: Evolving agentic interpretability tools via autoresearch, 2026. URL https://arxiv.org/abs/2605.03808.
  35. [35]Daniel Kasen​​berg, Pablo Samuel Castro, Maria K. Eckstein, Noémi Éltvölgyi, Will Dabney, Caroline Wang, Martin Engelcke, Rishika Mohanta, Aparna Dev, Matthew M. Botvinick, Nenad Tomasev, Glenn C. Turner, Vincent Costa, Nathaniel D. Daw, Kimberly L. Stachenfeld, and Kevin J. Miller. AI-discovered cognitive models reveal novel insights into human and animal learning. 2026. doi: 10.64898/2026.05.18.725921. URL http://dx.doi.org/10.64898/2026.05.18.725921.
  36. [36]Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024.arxiv.org/abs/2408.06292
  37. [37]Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, et al. Towards an ai co-scientist. arXiv preprint arXiv:2502.18864, 2025.
  38. [38]Kyle Swanson, Wesley Wu, Nash L Bulaong, John E Pak, and James Zou. The virtual lab of ai agents designs new sars-cov-2 nanobodies. Nature, 646(8085):716–723, 2025.DOI
  39. [39]Tal Fargan, Lukas Hafner, Maor Kern, Ori Alcalay, and Roy Kishony. Autonomous llm-driven research from data to human-verifiable research papers. arXiv preprint arXiv:2404.17605, 2024.
  40. [40]Jiabin Tang, Lianghao Xia, Zhonghang Li, and Chao Huang. Ai-researcher: Autonomous scientific innovation. arXiv preprint arXiv:2505.18705, 2025.arxiv.org/abs/2505.18705
  41. [41]Ludovico Mitchener, Angela Yiu, Benjamin Chang, Mathieu Bourdenx, Tyler Nadolski, Arvis Sulovari, Eric C Landness, Daniel L Barabasi, Siddharth Narayanan, Nicky Evans, et al. Kosmos: An ai scientist for autonomous discovery. arXiv preprint arXiv:2511.02824, 2025.arxiv.org/abs/2511.02824
  42. [42]Gail Weiss, Yoav Goldberg, and Eran Yahav. Thinking like transformers, 2021. URL https://arxiv.org/abs/2106.06981.
  43. [43]David Lindner, Janos Kramar, Sebastian Farquhar, Matthew Rahtz, Tom McGrath, and Vladimir Mikulik. Tracr: Compiled transformers as a laboratory for interpretability. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, page 37876–37899. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2023. doi: 10.52202/0725280-1649. URL http://dx.doi.org/10.52202/0725280-1649.
  44. [44]David Ha, Andrew Dai, and Quoc V. Le. Hypernetworks, 2016. URL https://arxiv.org/abs/1609.09106.
  45. [45]Vinod Kumar Chauhan, Jiandong Zhou, Ping Lu, Soheila Molaei, and David A. Clifton. A brief review of hypernetworks in deep learning. Artificial Intelligence Review, 57(9), 2024. doi: 10.1007/s10462-024-10862-8. URL http://dx.doi.org/10.1007/s10462-024-10862-8.
  46. [46]Kenneth O. Stanley, David B. D’Ambrosio, and Jason Gauci. A hypercube-based encoding for evolving large-scale neural networks. Artificial Life, 15(2):185–212, 2009. doi: 10.1162/artl.2009.15.2.15202. URL http://dx.doi.org/10.1162/artl.2009.15.2.15202.
  47. [47]Konstantin Schürholt, Boris Knyazev, Xavier Giró-i-Nieto, and Damian Borth. Hyper-representations as generative models: Sampling unseen neural network weights. In Advances in Neural Information Processing Systems 35, NeurIPS 2022, page 27906–27920. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2022. doi: 10.52202/068431-2024. URL http://dx.doi.org/10.52202/068431-2024.
  48. [48]William Peebles, Ilija Radosavovic, Tim Brooks, Alexei A. Efros, and Jitendra Malik. Learning to learn with generative models of neural network checkpoints, 2022. URL https://arxiv.org/abs/2209.12892.
  49. [49]Boris Knyazev, Michal Drozdzal, Graham W. Taylor, and Adriana Romero-Soriano. Parameter prediction for unseen deep architectures, 2021. URL https://arxiv.org/abs/2110.13100.
  50. [50]Xiaolong Han, Zehong Wang, Bo Zhao, Binchi Zhang, Jundong Li, Damian Borth, Rose Yu, Haggai Maron, Yanfang Ye, Lu Yin, and Ferrante Neri. A survey of weight space learning: Understanding, representation, and generation, 2026. URL https://arxiv.org/abs/2603.10090.
  51. [51]Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. https://rikiya.github.io/blog/weight-space-programming/, November 2026. Blog post.
  52. [52]Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yaniv Bai, Anna Chen, Tom Conerly, et al. A mathematical framework for transformer circuits. Transformer Circuits Thread, 1, 2021.
  53. [53]Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. arXiv preprint arXiv:2211.00593, 2022.arxiv.org/abs/2211.00593
  54. [54]Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering, 2023. URL https://arxiv.org/abs/2308.10248.
  55. [55]Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model, 2023. URL https://arxiv.org/abs/2306.03341.
  56. [56]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual knowledge in GPT. arXiv preprint arXiv:2202.05262, 2022.arxiv.org/abs/2202.05262
  57. [57]Dimitris Papailiopoulos. AdderBoard: A community challenge for tiny arithmetic transformers. https://github.com/anadim/AdderBoard, 2026. Open challenge and leaderboard for building the smallest transformers that can perform integer addition.
  58. [58]Xin Li and Dan Roth. Learning question classifiers. In Proceedings of the 19th international conference on Computational linguistics - , volume 1, page 1–7. Association for Computational Linguistics, 2002. doi: 10.3115/1072238.1072378. URL http://dx.doi.org/10.3115/1072238.1072378.
  59. [59]Richard Socher, Alex Pérelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, page 1631–1642. Association for Computational Linguistics, 2013. doi: 10.18653/v1/d13-1170. URL http://dx.doi.org/10.18653/v1/d13-1170.
  60. [60]Bill Dolan, Chris Quirk, and Chris Brockett. Unsupervised construction of large paraphrase corpora: exploiting massively parallel news sources. In Proceedings of the 20th international conference on Computational Linguistics - COLING ’04, COLING ’04, page 350–es. Association for Computational Linguistics, 2004. doi: 10.3115/1220355.1220406. URL http://dx.doi.org/10.3115/1220355.1220406.
  61. [61]Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, page 632–642. Association for Computational Linguistics, 2015. doi: 10.18653/v1/d15-1075. URL http://dx.doi.org/10.18653/v1/d15-1075.
  62. [62]Andrew C. Yao and Donald E. Knuth. Analysis of the subtractive algorithm for greatest common divisors. Proceedings of the National Academy of Sciences, 72(12):4720–4722, 1975. doi: 10.1073/pnas.72.12.4720. URL http://dx.doi.org/10.1073/pnas.72.12.4720.
  63. [63]Seyed-Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte. Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Computational Biology, 10(11):e1003915, 2014.
  64. [64]Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J Majaj, Rishi Rajalingham, Elias B Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Franziska Geiger, et al. Brain-Score: which artificial neural network for object recognition is most brain-like? bioRxiv, 2018. doi: 10.1101/407007.
  65. [65]Zirui Chen and Michael F Bonner. Universal dimensions of visual representation. Science Advances, 11(27):eadw7697, 2025.DOI
  66. [66]Kendrick N. Kay, Thomas Naselaris, Ryan J. Prenger, and Jack L. Gallant. Identifying natural images from human brain activity. Nature, 452(7185):352–355, 2008. doi: 10.1038/nature06713. URL http://dx.doi.org/10.1038/nature06713.
  67. [67]David J. Heeger. Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9(2):181–197, 1992. doi: 10.1017/s0952523800009640. URL http://dx.doi.org/10.1017/s0952523800009640.
  68. [68]Aude Oliva and Antonio Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 42(3):145–175, 2001. doi: 10.1023/a:1011139631724. URL http://dx.doi.org/10.1023/a:1011139631724.
  69. [69]Javier Portilla and Eero P. Simoncelli. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision, 40(1):49–70, 2000. doi: 10.1023/a:1026553619983. URL http://dx.doi.org/10.1023/a:1026553619983.
  70. [70]J. Zerbe, J. Roth, M. M. Mell, P. Herholz, T. Knapen, and M. N. Hebart. LAION-fMRI: A densely sampled 7T-fMRI dataset providing broad coverage of natural image diversity. Talk 25.11, Scene Perception Talk Session, Vision Sciences Society Annual Meeting, May 2026. VSS abstract.
  71. [71]Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, volume 25, 2012.
  72. [72]Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A Hosseini, Nancy Kanwisher, Joshua B Tenenbaum, and Evelina Fedorenko. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45):e2105646118, 2021.DOI
  73. [73]Chandan Singh, Richard J Antonello, Sihang Guo, Gavin Mischler, Jianfeng Gao, Nima Mesgarani, and Alexander G Huth. Evaluating scientific theories as predictive models in language neuroscience. BioRxiv, 2025.DOI
  74. [74]Tom M. Mitchell, Svetlana V. Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L. Malave, Robert A. Mason, and Marcel Adam Just. Predicting human brain activity associated with the meanings of nouns. Science, 320(5880):1191–1195, 2008. doi: 10.1126/science.1152876. URL http://dx.doi.org/10.1126/science.1152876.
  75. [75]Leila Wehbe, Brian Murphy, Partha Talukdar, Alona Fyshe, Aaditya Ramdas, and Tom Mitchell. Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses. PLoS ONE, 9(11):e112575, 2014. doi: 10.1371/journal.pone.0112575. URL http://dx.doi.org/10.1371/journal.pone.0112575.
  76. [76]Satoshi Nishida and Shinji Nishimoto. Decoding naturalistic experiences from human brain activity via distributed representations of words. NeuroImage, 180:232–242, 2018. doi: 10.1016/j.neuroimage.2017.08.017. URL http://dx.doi.org/10.1016/j.neuroimage.2017.08.017.
  77. [77]Sreejan Kumar, Theodore R. Sumers, Takateru Yamakoshi, Ariel Goldstein, Uri Hasson, Kenneth A. Norman, Thomas L. Griffiths, Robert D. Hawkins, and Samuel A. Nastase. Shared functional specialization in transformer-based language models and the human brain. 2022. doi: 10.1101/2022.06.08.495348. URL http://dx.doi.org/10.1101/2022.06.08.495348.
  78. [78]Amanda LeBel, Lauren Wagner, Shailee Jain, Aneesh Adhikari-Desai, Bhavin Gupta, Allyson Morgenthal, Jerry Tang, Lixiang Xu, and Alexander G Huth. A natural language fmri dataset for voxelwise encoding models. bioRxiv, pages 2022–09, 2022.DOI
  79. [79]Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G. Huth. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience, 26(5):858–866, 2023. doi: 10.1038/s41593-023-01304-9. URL http://dx.doi.org/10.1038/s41593-023-01304-9.
  80. [80]Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  81. [81]Omer Moussa, Dietrich Klakow, and Mariya Toneva. Improving semantic understanding in speech language models via brain-tuning, 2024. URL https://arxiv.org/abs/2410.09230.
  82. [82]Taha Binhuraib, Ruimin Gao, and Anna A. Ivanova. Litcoder: A general-purpose library for building and comparing encoding models, 2025. URL https://arxiv.org/abs/2509.09152.
  83. [83]Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision, 2022. URL https://arxiv.org/abs/2212.04356.
  84. [84]Zaid Zada, Samuel A. Nastase, Bobbi Aubrey, Itamar Jalon, Sebastian Michelmann, Haocheng Wang, Liat Hasenfratz, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni, Sasha Devore, Adeen Flinker, Orrin Devinsky, Ariel Goldstein, and Uri Hasson. The “podcast” ecog dataset for modeling neural activity during natural language comprehension. 2025. doi: 10.1101/2025.02.14.638352. URL http://dx.doi.org/10.1101/2025.02.14.638352.
  85. [85]Ariel Goldstein, Haocheng Wang, Leonard Niekerk en, Zaid Zada, Bobbi Aubrey, Tom Sheffer, Samuel A Nastase, Harshvardhan Gazula, Mariano Schain, Aditi Singh, et al. Deep speech-to-text models capture the neural basis of spontaneous speech in everyday conversations. bioRxiv, pages 2023–06, 2023.DOI
  86. [86]Taishih Chi, Powen Ru, and Shihab A. Shamma. Multiresolution spectrotemporal analysis of complex sounds. The Journal of the Acoustical Society of America, 118(2):887–906, 2005. doi: 10.1121/1.1945807. URL http://dx.doi.org/10.1121/1.1945807.
  87. [87]Joakim Andén and Stéphane Mallat. Deep scattering spectrum. 2013. doi: 10.48550/ARXIV.1304.6763. URL https://arxiv.org/abs/1304.6763.
  88. [88]Chandan Singh, Keyan Nasseri, Yan Shuo Tan, Tiffany Tang, and Bin Yu. imodels: a python package for fitting interpretable models. Journal of Open Source Software, 6(61):3192, 2021.DOI
  89. [89]David D. Lewis. Naive (Bayes) at forty: The independence assumption in information retrieval, page 4–15. Springer Berlin Heidelberg, 1998. doi: 10.1007/bfb0026666. URL http://dx.doi.org/10.1007/bfb0026666.
  90. [90]Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information, 2016. URL https://arxiv.org/abs/1607.04606.
  91. [91]Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759, 2016.arxiv.org/abs/1607.01759
  92. [92]Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, et al. Scienceagentbench: Toward rigorous assessment of language agents for data-driven scientific discovery. arXiv preprint arXiv:2410.05080, 2024.
  93. [93]Ziming Luo, Atoosa Kasirzadeh, and Nihar B Shah. The more you automate, the less you see: Hidden pitfalls of a scientist systems. arXiv preprint arXiv:2509.08713, 2025.arxiv.org/abs/2509.08713
  94. [94]Hannes Thurnherr and Kaspar Riesen. Neural decompiling of tracr transformers. 2024. doi: 10.48550/ARXIV.2410.00061. URL https://arxiv.org/abs/2410.00061.
  95. [95]Xinting Huang, Aleksandra Bakalova, Satwik Bhattamishra, William Merrill, and Michael Hahn. Discovering interpretable algorithms by decompiling transformers to rasp, 2026. URL https://arxiv.org/abs/2602.08857.
  96. [96]Aviv Navon, Aviv Shamsian, Idan Achituve, Ethan Fetaya, Gal Chechik, and Haggai Maron. Equivariant architectures for learning in deep weight spaces, 2023. URL https://arxiv.org/abs/2301.12780.
  97. [97]Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should I trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144. ACM, 2016.arxiv.org/abs/1602.04938
  98. [98]Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, pages 4768–4777, 2017.arxiv.org/abs/1705.07874
  99. [99]Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
  100. [100]Chandan Singh, W James Murdoch, and Bin Yu. Hierarchical interpretations for neural network predictions. International Conference on Learning Representations, page 26, 2019. URL https://openreview.net/forum?id=SkEqr0cQtC.arxiv.org/abs/1806.05337
  101. [101]Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, page 5484–5495. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.emnlp-main.446. URL http://dx.doi.org/10.18653/v1/2021.emnlp-main.446.
  102. [102]Catherine Olsson, Nelson Elhage, Neil Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads. arXiv preprint arXiv:2209.11895, 2022.arxiv.org/abs/2209.11895
  103. [103]Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Circuit component reuse across tasks in transformer language models. arXiv preprint arXiv:2310.08744, 2023.arxiv.org/abs/2310.08744
  104. [104]Tom Lieberum, Matthew Rahtz, János Krámár, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. Does circuit analysis interpretability scale? Evidence from multiple choice capabilities in chinchilla. arXiv preprint arXiv:2307.09458, 2023.arxiv.org/abs/2307.09458
  105. [105]Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. Retrieval head mechanistically explains long-context factuality, 2024.arxiv.org/abs/2404.15574
  106. [106]Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, 2024. URL https://arxiv.org/abs/2406.04093.
  107. [107]Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability, 2023. URL https://arxiv.org/abs/2301.05217.
  108. [108]Philip Quirke and Fazl Barez. Understanding addition in transformers, 2023. URL https://arxiv.org/abs/2310.11321.
  109. [109]Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models, 2023. URL https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html.
  110. [110]Chandan Singh, Aliyah R Hsu, Richard Antonello, Shailee Jain, Alexander G Huth, Bin Yu, and Jianfeng Gao. Explaining black box text modules in natural language with language models. arXiv preprint arXiv:2305.09863, 2023.
  111. [111]Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Jacob Andreas. Natural language descriptions of deep visual features. In International Conference on Learning Representations, 2022.arxiv.org/abs/2201.11114
  112. [112]Gonçalo Paulo, Alex Mallen, Caden Juang, and Nora Belrose. Automatically interpreting millions of features in large language models, 2024. URL https://arxiv.org/abs/2410.13928.
  113. [113]Tamar Rott Shaham, Sarah Chan, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob Andreas, and Antonio Torralba. A multimodal automated interpretability agent. In Forty-first International Conference on Machine Learning, 2024.
  114. [114]Weiqi Liu, Yongliang Miao, Haiyan Zhao, Yanguang Liu, and Mengnan Du. Neuronscope: A multi-agent framework for explaining polysemantic neurons in language models, 2026. URL https://arxiv.org/abs/2601.03671.
  115. [115]Arnau Marin-Llobet and Javier Fernando. Automated interpretability and feature discovery in language models with agents, 2026. URL https://arxiv.org/abs/2605.01555.
  116. [116]Vladimir Zaigrajew, Michał Piechota, Gaspar Sekula, Pawel Gelar, and Przemysław Biecek. Line: Llm-based iterative neuron explanations for vision models, 2026. URL https://arxiv.org/abs/2604.08039.
  117. [117]Ayan Antik Khan, Harsh Kohli, Yeukun Yao, Huan Sun, and Ziyu Yao. Can language model agents be helpful circuit explainers in mechanistic interpretability?, 2026. URL https://arxiv.org/abs/2606.24026.
  118. [118]Nancy Kanwisher and Galit Yovel. The fusiform face area: a cortical region specialized for the perception of faces. Philosophical Transactions of the Royal Society B: Biological Sciences, 361(1476):2109–2128, 2006.pubmed.ncbi.nlm.nih.gov/17118927
  119. [119]Frederik S Kamps, Joshua B Julian, Jonas Kubilius, Nancy Kanwisher, and Daniel D Dilks. The occipital place area represents the local elements of scenes. Neuroimage, 132:417–424, 2016.
  120. [120]Samira Abnar, Lisa Beinborn, Rochelle Choenni, and Willem H. Zuidema. Blackbox meets blackbox: Representational similarity and stability analysis of neural language models and brains. CoRR, abs/1906.01539, 2019. URL http://arxiv.org/abs/1906.01539.
  121. [121]Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1721–1730, 2015.
  122. [122]Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasneninnikov, and David Krueger. Defining and characterizing reward hacking, 2022. URL https://arxiv.org/abs/2209.13085.
  123. [123]Leon Berger, Usha Bhalla, Andrew Lee, Barak Widawsky, Linas Nasvytis, Connor Watts, Siddharth Boppana, Sidharth Baskaran, Dnen Hazra, Michael Byun, Atticus Geiger, Owen Lewis, Matthew Kowal, Vasudev Shyam, Thomas Fel, Thomas McGrath, Ekdeep Singh Lubana, and Jack Merullo. Monitoring and discovering reward hacking with internal representations during llm evaluations, 2026. URL https://arxiv.org/abs/2609.19101.
  124. [124]Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition, 2022. URL https://arxiv.org/abs/2209.10652.
  125. [125]Bilal Chughtai, Lawrence Chan, and Neel Nanda. A toy model of universality: Reverse engineering how networks learn group operations, 2023. URL https://arxiv.org/abs/2302.03025.
  126. [126]Saleema Amershi, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. Power to the people: The role of humans in interactive machine learning. AI Magazine, 35(4):105–120, 2014. doi: 10.1609/aimag.v35i4.2513. URL http://dx.doi.org/10.1609/aimag.v35i4.2513.
  127. [127]Andreas Holzinger. Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Informatics, 3(2):119–131, 2016. doi: 10.1007/s40708-016-0042-6. URL http://dx.doi.org/10.1007/s40708-016-0042-6.
  128. [128]Andrew Slavin Ross, Michael C Hughes, and Finale Doshi-Velez. Right for the right reasons: Training differentiable models by constraining their explanations. arXiv preprint arXiv:1703.03717, 2017.arxiv.org/abs/1703.03717
  129. [129]Stefano Teso and Kristian Kersting. Explanatory interactive machine learning. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’19, page 239–245. ACM, 2019. doi: 10.1145/3306618.3314293. URL http://dx.doi.org/10.1145/3306618.3314293.
  130. [130]Patrick Schramowski, Wolfgang Stammer, Stefano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Hans-Georg Luigs, Anne-Katrin Mahlein, and Kristian Kersting. Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nature Machine Intelligence, 2(8):476–486, 2020.arxiv.org/abs/2001.05371
  131. [131]Edward H Adelson and James R Bergen. Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, 2(2):284–299, 1985.pubmed.ncbi.nlm.nih.gov/3973762
  132. [132]Maximilian Riesenhuber and Tomaso Poggio. Hierarchical models of object recognition in cortex. Nature Neuroscience, 2:1019–1025, 1999.DOI
  133. [133]Joan Bruna and Stéphane Mallat. Invariant scattering convolution networks. IEEE transactions on pattern analysis and machine intelligence, 35(8):1872–1886, 2013.

Paper details

vlnc:W_jv57dyyq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents