WorksComputer Science
Same Fix, Different Weights: Replicating Weight-Space Programming with Claude
Abstract
Takehi et al. (2026) report that GPT-6 Astra, given the weights of small neural networks as text, can say what they were trained to do, repair them, and write new ones without training. I re-ran the core of that study with Claude Opus 5.5 in ten pre-registered experiments, with untrained-network and permutation controls and hand audits of every sandboxed code cell. Reading was mixed: Claude named 3 of 4 Atari games from policy weights and found the measured copying heads in 15/15 character-LM runs, but its answers about training data often followed priors. Unlike the original, it identified a sycophantic Gemma 3 270M finetune from weights alone in 6/6 runs. One-shot weight edits took a failing MountainCar policy from 0% to 100% success and a hovering LunarLander policy from 0% to 85.6%; zero-shot networks beat PPO at 100k steps on 4 of 5 environments, each implementing a textbook controller. With no training or forward passes, Claude wrote a 1.48M-parameter 9x9 Go network that won 85-94% of games against three PPO opponents. Given Qwen3.5-0.8B's weights and 10, 25 or 50 forward passes of 64 practice problems, Claude raised MATH-500 from 6.7% to 51.4%, 55.2% and 58.1% (+51.4 points at 50 passes, 95% CI [47.9, 55.0]; original 48.1%), GSM8K from 11.6% to 68.2% and IFEval from 22.2% to 54.2%. Unlike the original, which removed an internal loop direction, every edit acted at the output: it scaled the final norm, raising the effective sampling temperature, and lowered the logit of "Wait" while raising that of </think>. The norm scaling alone reaches only 20.9-22.1% on MATH-500. On Gemma 3 270M, which rarely loops, no edit helped (-0.8 points), as in the original.
Introduction
A trained policy network for an Atari game is, to its user, a list of about fourteen thousand floating-point numbers. Interpretability research usually studies such a network by running it: recording activations, fitting probes and intervening on units [24, 9]. Takehi et al. [35] ask a narrower question. They give a frontier language model, GPT-6 Astra, nothing but the weights printed as rows of numbers, and ask it in plain language what the network was trained to do, how to repair it, or what weights a new network should have. They report positive results on all three, and they call the activity weight-space programming.
Claims of this kind are easy to over-read. The original post presents most probes with one to four attempts, no stated grading rules and no controls for answers driven by priors. Its authors describe the results as early, and they release their logs. A replication can add what a single demonstration does not provide: repeated attempts, controls that make a wrong answer possible and visible, grading rules fixed before any data, and a second model on networks it has never seen.
This paper reports such a replication with Claude Opus 5.5 (claude-opus-5-5). For the first seven probes I trained every subject network myself, in environments that differ from the original’s: the Arcade Learning Environment (ALE) and Gymnasium classic control instead of MinAtar and Kinetix, and a separately validated Go engine. The eighth uses the original’s own subject, Qwen3.5-0.8B, with its benchmarks and sampling protocol. I kept the questions and the spirit of each probe. The study has four parts.
Reading (§ 5): naming the Atari game or classic-control task a PPO policy was trained on (Experiments 1 and 1b), the shape a diffusion model draws (Experiment 2), the training language and copying heads of a character-level transformer from its middle blocks (Experiment 3), and which of two Gemma 3 270M finetunes is sycophantic (Experiment 7).
Fixing (§ 6): one-shot weight edits to a MountainCar policy that never reaches the flag and a LunarLander policy that hovers (Experiment 4).
Writing (§ 7): policies written from all-zero weights with no feedback (Experiment 5) or with 5 or 25 rollout batches (Experiment 5b), and a 1.48M-parameter Go network written with 0 to 50 batches of forward passes (Experiment 6).
Editing a language model (§ 8): improving Qwen3.5-0.8B at math with 10, 25 or 50 forward passes on practice problems and no gradients, with a Gemma 3 270M control that rarely loops (Experiment 8).
The contributions are as follows.
A pre-registered, logged replication of all eight of the original’s probes, seven on new networks and environments and the eighth on the original’s own Qwen3.5-0.8B setup, with 193 API runs costing $83.38 in total (Table 5).
| Experiment | API runs | Input tokens | Output tokens | Cost (USD) | Cap (USD) |
| 1 Atari game | 30 | 1.10M | 77k | 5.93 | 20 |
| 1b Control task (v1 + v2) | 62 | 0.49M | 141k | 4.79 | 10 + 5 |
| 2 DDPM shapes | 23 | 1.70M | 257k | 11.93 | 15 |
| 3 Char-LM heads | 15 | 15.92M | 637k | 20.96 | 25 |
| 4 Fix a policy | 10 | 0.30M | 97k | 3.14 | 12 |
| 5 Write, zero-shot | 19 | 0.38M | 267k | 6.85 | 18 |
| 5b Write, rollout budget | 8 | 5.35M | 173k | 6.11 | 12 |
| 6 Go | 4 | 10.84M | 284k | 9.77 | 40 |
| 7 Sycophancy | 12 | 3.57M | 141k | 5.13 | 25 |
| 8 Edit Qwen3.5-0.8B | 6 | 7.97M | 86k | 8.48 | 60 |
| Harness smoke tests | 4 | 0.07M | 0.5k | 0.27 | – |
| Total | 193 | 47.68M | 2.16M | 83.38 | 150 (project) |
Table 5. API usage and cost per experiment, summed from logs/<id>/*/result.json, including pilots, one crashed launch in Experiment 5b, and in Experiment 8 the abandoned first pilot ($0.23) and one infrastructure test call ($0.03). Harness smoke tests ran before Experiment 1. Rows are rounded separately; the total is summed from unrounded costs.
A replication of the original’s largest claim with a temperature control. Claude raised Qwen3.5-0.8B’s MATH-500 score from 6.7% to 51.4–58.1% () with 10 to 50 forward passes, through the output layer rather than the original’s internal direction, and a norm-only control shows that about a third of the math gain and nearly all of the IFEval gain is a temperature change (§ 8).
Controls for prior-driven answers: 10 untrained networks, hidden-unit permutations of every read network, and a second padding design after the first one failed (§ 5.2).
An account of where weight reading works and where it fails. Claude reads structure it can recognise or construct (attention circuits, the inputs a policy depends on, known controllers), and falls back on priors when asked what data a network was trained on (§ 10).
A release of every prompt, transcript, code cell, written weight file and score (§ 12).
The two studies differ in model, networks, environments and attempt counts, so this paper does not rank the two models. § 9 sets the results side by side and says for each row how the setups differ.
Background and related work
Mechanistic interpretability. Circuits research reads a trained network as a composition of features and the weights that connect them [24]. For transformers, Elhage et al. [9] decompose each attention head into a query–key (QK) circuit, which decides where the head attends, and an output–value (OV) circuit, which decides what it writes. Products of these matrices across layers expose composition between heads without running the model. Olsson et al. [25] use this framework to describe induction heads, which complete a repeated pattern through a previous-token head in an earlier layer and K-composition in a later one. Experiment 3 tests whether a language model can rediscover this analysis unprompted. Locating and editing specific computations inside weights is also established practice [22], and the original’s Experiment 8 resembles removing a behaviour by ablating a single direction [3].
Language models as readers of networks. Bills et al. [5] use a language model to explain individual neurons from their activations on text. Recent work trains models to read activations or weights directly [10, 15]. These approaches either use activations or train a reader for the task. The setting here, like the original’s, uses a general-purpose model without task-specific training, and in most probes forbids running the subject network.
Generating weights. Weights can be produced by a model trained for that purpose: diffusion over parameters [39], hypernetworks that adapt language models without back-propagation [27], and language models fine-tuned to emit parameters [38]. In each case the generator is trained on networks. Here the writer is not.
Gradient-free optimisation. Simultaneous perturbation stochastic approximation (SPSA) [33], evolution strategies [30] and zeroth-order fine-tuning with forward passes only (MeZO) [20] find weights without gradients, by many evaluations of perturbed parameters. Experiments 5b and 6 give Claude a small budget of evaluations (5 to 50 batches), far below what these methods use, and forbid search loops over the weights.
Language models as optimisers. Language models already optimise text artefacts: prompts [41], multi-stage language-model programs [16, 26], and prompts evolved by reflection [1]. Weight-space programming extends the same idea from text to parameters.
Environments and training. The policies were trained with proximal policy optimisation (PPO) [31] in a single-file implementation modelled on CleanRL [14]. Classic-control and Box2D tasks come from Gymnasium [37], and Atari games from the Arcade Learning Environment [4] with the v5 protocol of Machado et al. [19]. Atari observations are the console’s 128-byte RAM, for which AtariARI publishes per-game annotations of which byte holds which object coordinate [2]; these annotations turn out to matter (§ 5.1 and § 7.1). The Go engine was validated against PettingZoo’s go_v5 [36]. The diffusion models follow DDPM [13] with a cosine noise schedule [23]. The original used MinAtar [42] and Kinetix [21].
The original study
Takehi et al. [35] (MIT CSAIL) give GPT-6 Astra the raw weights of small networks, printed one input unit per row, and ask questions in natural language. Their eight experiments fall into three groups. Table 1 lists the numbers they report; this section paraphrases their setup.
Reading. Without tools, Astra named the game for each of four PPO policies on MinAtar-style spatial inputs (Breakout, Pong, Pac-Man, Freeway) without being given candidates (Experiment 1). It identified whether each of three DDPMs (23,824 weights) draws circles, squares or triangles, drawing every shape as an outline although the models generate filled shapes (Experiment 2). Given only the four middle blocks of three 7.2M-parameter character-level language models, with the embedding and unembedding removed, it named the training language (Python, romaji, English) in both of two attempts per model and listed exactly the measured copying heads (Experiment 3). It could not tell which of two Gemma 3 270M finetunes was sycophantic from weights alone (0 of 4 attempts), but did so for all three pairs when allowed to read attention patterns on prompts it wrote (Experiment 7).
Fixing. Given a Kinetix car policy that drives into a ramp and stalls, Astra edited 3.5% of its 171,664 weights in one reply, and the car succeeded in all 1,024 evaluation episodes (Experiment 4). Given Qwen3.5-0.8B and a budget of forward passes on GSM8K and MATH training problems, it traced a tendency to loop until the token limit to one direction written by the MLPs of layers 15 and 19, subtracted it from 64 output columns in each (131,072 weights, 0.016%), and raised MATH-500 from 7.9% to 48.1% (Experiment 8).
Writing. With game rules and source code but no data, training or rollouts, Astra wrote MinAtar policies that beat PPO trained for 100M steps on Breakout, Seaquest and Asterix (Experiment 5). In a Python sandbox where the network could only be run through budgeted batches of 128 games, it wrote a Go network that beat two small PPO networks with no forward passes, and beat an equal-sized PPO network trained on 20M games with 50 forward passes (Experiment 6).
The authors present these as early results and release complete logs. Most probes report one to four attempts, and the post does not state grading rules or controls. The questions below are therefore not whether these behaviours can occur once, but how often they occur and whether they survive controls.
| Exp. | Probe | Reported result |
| 1 | Name the game of 4 MinAtar-input PPO policies | 4/4 games |
| 2 | Name the shape of 3 DDPMs (23,824 weights) | 3/3, all drawn as outlines |
| 3 | Language and copying heads from middle blocks | Language 6/6; copying heads exact in all 3 models |
| 4 | Fix a Kinetix car policy, one shot | 100% of 1,024 episodes; 3.5% of weights changed |
| 5 | Write MinAtar policies zero-shot | Breakout 517.4, Seaquest 53.1, Asterix 53.2 vs PPO-100M 15.8, 5.6, 25.8 and PPO-500M 25.3, 88.9, 55.2 |
| 6 | Write a Go network, 0–50 forward passes | 0 passes: >50% vs both small PPO nets; 50 passes: beats the equal-sized PPO net (20M games); 55.2% of practice games won |
| 7 | Spot the sycophantic Gemma 3 270M | Weights only 0/4; attention tool 3/3 pairs; head 10.2 |
| 8 | Improve Qwen3.5-0.8B at math | MATH-500 7.9% → 48.1%, GSM8K 13.8% → 67.1%, IFEval 21.0% → 62.3% (50 passes); token-limit hits 93.0% → 33.1% |
Table 1. Headline numbers reported by Takehi et al. [35] for GPT-6 Astra, as stated in the original post.
Methods
Figure 1 shows how every experiment runs. I train a subject network, serialise its weights, send them to Claude Opus 5.5 either pasted into a single prompt or loaded into an isolated Python sandbox, and grade the answer under rules fixed in advance. Table 2 summarises how the design differs from the original.

Figure 1. How each experiment runs. Subject networks are trained by me; Claude sees only their weights and the interface description. Grading rules for every experiment were written before its first API call.
| Original ([35]) | This replication | |
| Model | GPT-6 Astra | Claude Opus 5.5 (claude-opus-5-5), adaptive thinking, effort high |
| Environments | MinAtar; Kinetix ramp; Go | Gymnasium classic control and Box2D; ALE Atari (RAM observations); Go on a separately validated engine |
| Attempts | 1–4 per reported probe | 3–5 per network in most probes; 2 per pair per phase in Exp. 7; 1 per cell in Exp. 5b and Exp. 6; all reported as |
| Grading | Not stated | Written in GRADING.md before each experiment’s first API call |
| Controls | None reported | Untrained networks, hidden-unit permutations, transcript audits |
| Logs | Released | Every prompt, transcript, answer, code cell and score released |
Table 2. The original study and this replication, side by side.
Subject networks
Table 3 lists every network Claude read, fixed or competed against. All were trained by me on one Apple M4 Max laptop (§ 4.9); hyperparameters are in Section E. The policies are the actor networks of PPO. The critic is a separate network that is never saved or shown, so the checkpoint is exactly the network that acts.
| Exp. | Network | Architecture | Weights | Training and quality |
| 1 | 4 Atari PPO policies (Breakout, Pong, Ms. Pac-Man, Freeway) | , tanh | 13,586 | 10M steps, seed 0; returns 27.5, , 908.8, 28.7 vs random 1.45, , 272.3, 0 |
| 1b | 4 classic-control PPO policies, padded | v1: ; v2: | 1,476 (v1) | 1M steps (LunarLander 5M); v1 returns 500.0, , , 239.3 |
| 2 | 3 DDPM denoisers (circle, square, triangle) | , SiLU, predicts | 28,000 | 40k steps; judge accuracy on samples 0.83, 0.89, 0.98 |
| 3 | 3 character LMs (Python, romaji, English), middle blocks only | 4 blocks 6 heads, , MLP 1536 | 7,097,856 | 10k steps, about 164M characters |
| 4 | Failing MountainCar; hovering LunarLander | ; , tanh | 4,547; 4,996 | 0% success each over 1,024 episodes |
| 5, 5b | Written from all-zero weights; PPO baselines at 100k, 1M, final | PPO architectures, e.g. | 4,610 (CartPole) | PPO final: 5M steps (10M for LunarLander, Breakout) |
| 6 | Go network written by Claude; 3 PPO opponents | , ReLU; opponents (2 seeds) and | 1,480,274 | small: 2.6M self-play games; large: 0.71M (original: 20M) |
| 7 | 3 pairs of Gemma 3 270M finetunes, honest vs sycophantic | 18 layers, , 4 query heads, 1 KV head | 100,325,376 per model (trainable) | 545 steps; embedding and final norm frozen |
Table 3. Subject networks. “Weights” counts all parameters shown to Claude. Returns are greedy means over 100 seeded episodes (1,024 for Experiment 4).
I moved from MinAtar and Kinetix to Gymnasium and the ALE for two reasons. They are the most widely used maintained reinforcement-learning libraries, so the probes can be re-run by others, and a real capability should not depend on one set of games. The Atari policies read RAM rather than pixels. All four Atari networks share one interface: gym.make("ALE/<Game>-v5", obs_type="ram", full_action_space=True) gives 128 inputs and 18 actions for every game, so the shape of the weights says “Atari” and nothing more. RAM has no spatial layout, which removes the original’s route of reading where a channel’s first-layer weights sit on the board. It opens another route: published RAM annotations [2].
Weights as text
Weights are printed in one fixed format, which is the original’s: layers are named W1, b1, W2, …; each line is one input unit of a layer, listing its weights to every unit of the next layer (the matrix printed as [in, out], the transpose of PyTorch’s storage); biases follow on one line. For Atari, line of W1 is everything RAM byte feeds into, which lets a reader reason byte by byte. An integer variant stores each value as an integer times 0.001, declared in a header comment. The same tolerant parser reads back any weights Claude writes, so the evaluated network is exactly the text Claude produced. It accepts sparse edits (W2[3,17] = 1.25, b1[4] = -0.5), whole rows and whole bias vectors, and ignores prose and Markdown.
Table 4 gives token costs measured with the API’s token counter. Weight text averages about 1.45 characters per token, so estimating tokens from characters is wrong by a factor of about 2.5. The integer format gives the same precision as three decimals at 43% fewer tokens, and was used for Experiments 1 and 2. Experiments 1b and 4 used three-decimal floats.
| Format | Characters | Tokens | Tokens per weight |
| 3 decimals | 91k | 62.9k | 4.63 |
| 2 decimals | 77k | 62.6k | 4.61 |
| Integers, value = int × 0.001 | 50k | 35.8k | 2.63 |
| Integers, value = int × 0.01 | 38k | 35.5k | 2.61 |
Table 4. Token cost of one MLP (13,586 weights) in four text formats, measured with count_tokens on claude-opus-5-5.
API configuration
Every call used claude-opus-5-5 with adaptive thinking returned as a summary (display: "summarized") and effort="high". On this model thinking cannot be disabled and has no token budget; effort is the only control, and the API default is medium, so I set it explicitly. max_tokens was 64,000 per turn (32,000 in Experiment 5b). Thinking counts toward this limit. Calls were streamed. The system prompt, where one was used, was “You are an expert in machine learning and neural networks. Answer carefully and honestly.” It does not mention the original study or any environment, and no prompt does.
Multi-turn runs used a manual tool loop that appends each assistant message unchanged, runs every tool call, and returns all results in one message. When a run reached its turn or cost cap, the harness asked for a final answer with tools disabled. Prompt caching was enabled for multi-turn runs only. Costs are computed from the API-reported usage at $4 and $20 per million input and output tokens and $0.20 per million cache-read tokens. Thinking is returned only as a summary, so every quotation of Claude’s reasoning in this paper comes from a summary or a final answer, not from the raw chain of thought.
Sandbox and isolation
In Experiments 3, 5b, 6 and 7, Claude worked in a persistent Python session (Figure 2). The session runs in a child process that holds numpy copies of the weights. Budgeted functions such as rollout(), play_games() and attention() appear in the child as stubs; calling one sends a request to the parent process, which checks the budget before running anything, runs the environment or model, and returns only a summary. Calls that raise are refunded. A free save_weights() stores tensors in the parent, and budgeted functions always use the saved weights.

Figure 2. Sandbox isolation in Experiments 3, 5b, 6 and 7. Claude’s code can read, analyse and save weights; the network or environment runs only in the parent, through counted calls that return summaries.
On macOS the child runs under sandbox-exec with a profile (Section D) that denies network access and every read or write under the home directory and the temporary directories, and re-allows only the worker’s scratch directory, the virtual environment and the Python installation. An import hook also blocks the environment packages (Gymnasium, ALE, PettingZoo, Box2D), subprocess, socket and the project’s own code. The first version of the profile denied only the repository. Preparing Experiment 7 showed that this left the Hugging Face cache, and the base Gemma model in it, readable, so the deny was widened to the whole home directory. An audit of the earlier transcripts found no file-system access by Claude’s code.
Pre-registration
Each experiment’s experiments/<id>/GRADING.md was written before that experiment’s first API call. It fixes the answer extraction, the accepted aliases, the correctness rule, the decision thresholds, the number of attempts and the budget cap. Each experiment began with one pilot call. Any change after the pilot was to be recorded under “Pre-run changes”. In every experiment the pilot changed nothing and counted as attempt 1. The second version of Experiment 1b was pre-registered separately, before its first call. The only post-pilot additions to any GRADING.md are pilot records and procedural notes, such as resuming three Experiment 1b controls after the budget guard stopped them. Pre-registration covers grading, not design: the padding in Experiment 1b was a design error that no grading rule could catch.
Controls
A correct answer means something only if a wrong answer was possible and would have been noticed. The reading probes use three controls.
Untrained networks. Networks of the same architecture with PyTorch default initialisation, and in Experiment 1 also the PPO training initialisation (orthogonal, zero biases). The prompt always allows “untrained / random weights” as an answer, with identical wording for every network. A model that confidently names a task here is reading its priors.
Hidden-unit permutation. The trained network with the units of each hidden layer permuted consistently, so the function is unchanged. The runner asserts this numerically (maximum logit difference below on 4,096 random inputs in Experiments 1 and 1b). Input rows and output columns are not permuted. A faithful reading should not change.
Transcript audits. Where Claude had a Python tool, every code cell was flagged by pattern and the flagged cells were read by hand for forward passes or environment simulation that the rules forbid.
The permutation control turned out to be close to trivial for this reader (§ 10).
Attempts and grading
Every attempt is an independent call or session, and every call is counted, including failed and truncated ones. Free-text answers are graded on the head answer only: the final machine-readable ANSWER: line, matched against fixed alias lists. A correct name mentioned in the reasoning as a runner-up earns nothing, and a head answer naming two candidates is graded wrong. Written and edited networks are scored on seeded episodes disjoint from any seed Claude saw: 100 episodes (seeds 10000–10099) in Experiments 5 and 5b, 1,024 (seeds 10000–11023) in Experiment 4, and 128 games per opponent on a held-out opening seed in Experiment 6. Section B summarises the rules per experiment. Confidence intervals in this paper are Wilson 95% intervals ([40]); except in Experiment 6, where the notes give Wilson lower bounds, I computed them from the reported counts.
Auditing
The audits read every flagged code cell in Experiments 3 (24 flagged calls over 15 runs), 5b, 6 and 7. The definitions were pre-registered. In Experiment 6, a violation is a board encoding pushed through two or more layers, or through one layer and the ReLU, or a game engine applying moves, or any fitting; a single-layer linear pre-activation on a few hand inputs is a minor breach. Violation runs keep their scores, and decision rules are reported with and without them. The results sections report every breach and grey area found.
Compute and cost
Everything except Experiment 8 ran on one Apple M4 Max laptop (16 CPU cores, 40-core GPU, 48 GB), including PPO training, the DDPMs, the character LMs and the Gemma finetunes; MPS was used for the transformer work. A code-level spend guard summed the logged cost before every API call and stopped all runs at $150 for the project, and each experiment and each multi-turn run had its own cap. Table 5 gives API usage and cost per experiment. Input tokens in Experiments 3, 5b, 6, 7 and 8 are mostly cache reads. The generation side of Experiment 8 ran on one NVIDIA H200 under SLURM (§ 8); its Claude loop and sandbox ran on the laptop.
Reading weights
The five reading probes give Claude a network’s weights and an interface description, and ask what the network was trained to do. In Experiments 1, 1b and 2 the weights are pasted into a single prompt and no tools are offered. In Experiments 3 and 7 Claude has a sandboxed Python session and may compute anything on the weight arrays, but may not run the network or any part of it on inputs.
Experiment 1: naming the Atari game
Setup. The subjects are four PPO policies on Atari RAM (Breakout, Pong, Ms. Pac-Man, Freeway; Table 3), printed in the integer format (about 36.5k prompt tokens). The prompt (Section A.1) says the network was trained with PPO on an Atari 2600 game, describes the RAM input and the 18 actions in ALE order, and asks “Which game was this network trained on?” It gives no candidate list and allows “untrained / random weights” as an answer. There were 5 attempts per network, 2 per permuted copy and 2 untrained controls (30 calls).
Pre-registered rule. A game counts as read if main accuracy is at least 3/5 and its name is the head answer in at most 5 of the 15 main attempts on the other three networks (a specificity check, so “always says Breakout” cannot pass). The overall claim requires 3 of 4 games read. The null is a guess among the four games (), which is conservative given that Claude could name any of 60 or more ALE games. “Pac-Man” is a different cartridge from Ms. Pac-Man and is graded wrong.
Results. Claude named the right game in 14 of 20 main attempts (Wilson 95% interval 0.48–0.85) and 6 of 8 permuted attempts (Table 6 and Figure 3). Pong (5/5), Freeway (5/5) and Breakout (4/5) were read under the rule, so the pre-registered verdict is “reads game identity from weights”. Ms. Pac-Man was 0/5 main and 0/2 permuted; across its 7 runs the answers were Breakout (3), Montezuma’s Revenge (2), Frostbite and Asterix. Only one response named Ms. Pac-Man at all, as a runner-up, which earns no credit. The most common answer overall was Breakout, in 9 of 28 trained-network runs, below the default-guess threshold of 14. Both untrained controls were called untrained (confidence 0.85 and 0.90). Confidence tracked correctness: it averaged 0.71 on correct answers and 0.23 on wrong ones, and every Ms. Pac-Man answer was at or below 0.30. Output length tracked difficulty: about 1.3k tokens on Freeway and 5.8k on Ms. Pac-Man.
| Network | Main | Permuted | Mean conf. | Most common wrong answer | Return vs random | Specificity |
| Breakout | 4/5 | 2/2 | 0.56 | Seaquest (1) | 27.5 vs 1.45 | 1/15 |
| Pong | 5/5 | 2/2 | 0.84 | – | −0.97 vs −20.3 | 0/15 |
| Ms. Pac-Man | 0/5 | 0/2 | 0.17 | Breakout (3) | 908.8 vs 272.3 | 0/15 |
| Freeway | 5/5 | 2/2 | 0.65 | – | 28.7 vs 0 | 0/15 |
| Total | 14/20 | 6/8 | ||||
| Untrained (2) | 2/2 called untrained (confidence 0.85 and 0.90) |
Table 6. Experiment 1: correct answers per network, no tools and no candidate list. Specificity is how often the game was the answer on the other three networks’ main attempts (at most 5/15 required).

Figure 3. Experiment 1: accuracy per network (main 4/5, 5/5, 0/5, 5/5; permuted 2/2, 2/2, 0/2, 2/2). Permuting hidden units leaves the function unchanged. A guess among the four games scores 25%.
How Claude reasoned. Every correct answer combined two readings: which actions the output layer separates, and which few first-layer rows (RAM bytes) carry outsized weights. Freeway was carried by the action layer: “every UP action has a positive bias (+116, +106, +76, +93, +126, +102)”. These values match exactly. Pong and Breakout were carried by matching the large rows to RAM addresses Claude remembered, and 18 of the 30 responses explicitly appeal to a known RAM layout. I checked the rows it singled out against each row’s norm, computed offline and never shown: they match almost exactly (Breakout’s top rows are 99, 109, 70, 101, 105, 103), and its labels match the AtariARI annotations [2] (Breakout ball = 99, ball = 101; Pong player = 51; Freeway chicken = 14). Finding which bytes a network listens to is weight reading; naming them is recall. For Ms. Pac-Man, Claude found the standout rows (1–3, 18, 42, 47, 116, 118) but had no remembered map that fitted them, and said so: “the evidence for Breakout specifically is weak”. The untrained PPO-initialised control was identified from its init scale and its exactly zero biases.
Caveats. The result is 3 of 4 games, not the original’s 4 of 4, on a different task: the original’s networks read spatial input channels, which made “where the agent’s weights concentrate” readable as a picture, and RAM has no such layout. Recognition of Pong and Breakout rests largely on memorised RAM maps. One Breakout response reasoned that “the game uses the full 18-action set, so a minimal-action game like Pong is unlikely”, which the prompt rules out, since every network uses the full set. Each game was trained with one seed, and is 5 + 2 per game. The permutation control can separate little here, for the reason given in § 10.
Experiment 1b: naming the classic-control task
Setup. Four PPO policies for CartPole, Acrobot, MountainCar and LunarLander were trained on a shared padded interface so that the shape would not reveal the task. In v1, observations were zero-padded to 8 inputs and actions to 4 (); extra actions map to valid native actions (CartPole: ; Acrobot and MountainCar: action 3 is the no-op). The prompt says the network was trained on “a widely used RL benchmark environment”, gives the architecture, and does not mention padding. Each version had 5 attempts per network, 2 per permuted copy and 3 untrained controls (31 calls per version, 62 in total).
Results for v1. The first design was wrong, and the error was mine. Padding to 8 inputs and 4 actions reproduces LunarLander’s native interface. Claude answered “LunarLander” in 26 of 28 trained-network attempts, usually from the shape alone: “No other common benchmark has this shape.” v1 scored 7/20 main (0.18–0.57) and 2/8 permuted; LunarLander’s 5/5 is a default guess, not a reading. Confidence on wrong answers (about 0.82) was no lower than on right ones. Claude saw contrary evidence and explained it away: on CartPole, the mapping forces logit 0 ≈ logit 2 and logit 1 ≈ logit 3, which it noticed in every CartPole run and read as LunarLander’s paired side engines. The two correct MountainCar answers came from an input with weights up to ±17 (against at most 0.8 elsewhere), which Claude read as a feature with a tiny range, “MountainCar’s velocity, which lies in ±0.07”.
Results for v2. A second version, pre-registered before its first call, padded to 10 inputs and 5 actions, a shape chosen to avoid every classic-control interface. It is the interface of the good agent in PettingZoo’s MPE simple_adversary ([36]), which became the answer in 21 of 28 attempts, this time at low confidence (0.35 mean on wrong answers, 0.67 on right ones). v2 scored 6/20 main (0.15–0.52) and 1/8 permuted. Only Acrobot reached the per-network bar (4/5). The pre-registered criterion for reading beyond the shape (at least 10/20 main and at least 3/5 on two of the three non-LunarLander networks) was not met, and the default-guess flag fired.
The clearest single result is in v2. Claude was right in 7 of 7 responses that mentioned padding or unused inputs (0.65–1.00) and wrong in 21 of 21 that did not (0.00–0.15). When it noticed, the reading was real. One Acrobot attempt argued that a padded input “receives no gradient, so its row should still be exactly orthogonal to the other untouched rows”, counted six live inputs and concluded Acrobot. All six untrained controls across both versions were called untrained (confidence 0.80–0.85); the v1 responses cite the bounds of PyTorch’s default initialisation.
| v1 (8 in, 4 out) | v2 (10 in, 5 out) | ||||||
| Network | Main | Perm. | Wrong answer | Main | Perm. | Conf. | Wrong answer |
| CartPole | 0/5 | 0/2 | LunarLander (7) | 0/5 | 0/2 | 0.40 | MPE (7) |
| Acrobot | 0/5 | 0/2 | LunarLander (7) | 4/5 | 0/2 | 0.60 | MPE (3) |
| MountainCar | 2/5 | 0/2 | LunarLander (5) | 2/5 | 1/2 | 0.40 | MPE (4) |
| LunarLander | 5/5 | 2/2 | – | 0/5 | 0/2 | 0.37 | MPE (7) |
| Total | 7/20 | 2/8 | 6/20 | 1/8 | |||
| Untrained | 3/3 called untrained | 3/3 called untrained |
Table 7. Experiment 1b: correct answers per network in v1 (8→4 padding) and v2 (10→5 padding). Most common wrong answers in parentheses.
Caveats. The headline is that identification from these weights is dominated by an interface-shape prior. A shape-neutral test would need a shape no benchmark in any popular suite has, or an explicit sentence that inputs and actions may be padded, which would hand Claude part of the solution. The v2 gap between main (6/20) and permuted (1/8) is most likely sampling noise: the correct reads needed long thinking (5.6k–28.8k output tokens, against about 2k for the MPE answers), which only some samples produced. The longest Acrobot call cost $0.61 and made the runner’s worst-case guard stop before the v2 controls, which were resumed under the same cap.
Experiment 2: what a diffusion model draws
Setup. Three MLP denoisers (, SiLU, 28,000 weights) were trained on images of a single filled circle, square or triangle. They predict the clean image rather than the noise, so the output layer maps hidden units directly onto pixels; the prompt says so. Noise prediction failed at this width (loss plateau near 0.85 MSE), which is why the parameterisation differs from the original’s. The prompt (about 73.6k tokens) gives the pixel layout and the sampling procedure, forbids tools and forward passes, and asks for a description, a # / drawing and a confidence. There were 5 attempts per network, 2 per permuted copy and 2 untrained controls (23 calls).
Pre-registered rule. The primary metric is the text label: the class whose alias (for circle: circle, disc, disk, round, ring and plurals) appears earliest in the description. The drawing is a secondary metric, scored by the best template IoU over filled and outline shapes.
Results. Under the label rule Claude named squares in 4/5 and triangles in 4/5 attempts (Table 8). Circles scored 1/5, but 0/5 in substance: every circle run, main and permuted, described handwritten digits, and the one graded hit matched only because the description mentioned “a large oval ‘0’-like ring”. Across the 21 runs on trained networks, 11 fell back on MNIST- or USPS-like digits (all 7 circle runs, 2 square and 2 triangle runs), which appears to be Claude’s prior for a small grayscale DDPM. Overall the label was right in 9/15 main runs (0.36–0.80) and 3/6 permuted runs. Both untrained networks were called untrained. Confidence was flat at 0.35–0.6 whether the answer was right or wrong.
Every run read the output bias as the mean image and then scanned the output-layer rows as image parts. Straight edges appear as clean line features, and those it read correctly. Curved edges it saw but did not trust. One circle run noted that the strongest units “seem to trace a ring of varying size”, then answered digits. Astra drew every shape as an outline; Claude drew squares and triangles mostly filled (5 filled and 3 outlines among the 8 correct main drawings), which matches the real samples (Figure 4).

Figure 4. (a) Circle model (b) Square model (c) Triangle model. Experiment 2. Top row of each panel: Claude’s five drawings from the weights alone, marked with whether the text label was graded correct. Bottom row: eight real samples from the model (seed 7). Every circle drawing is a digit.
| Network | Label (main) | Drawing (main) | Label (permuted) | Mean conf. | Drawing style |
| Circle | 1/51 | 5/52 | 1/21 | 0.51 | 4 filled, 1 outline (four “0” rings, one “3”) |
| Square | 4/5 | 5/52 | 1/2 | 0.49 | 4 filled, 1 outline (one is a “3”) |
| Triangle | 4/5 | 5/52 | 1/2 | 0.47 | 3 filled, 2 outline (one is a “2”) |
| Total | 9/15 | 15/15 | 3/6 | ||
| Untrained (2) | 2/2 described as untrained or incoherent (confidence 0.80) |
Table 8. Experiment 2: text label (pre-registered primary metric), drawing judge (secondary) and permutation control, per network.
Caveats. The drawing judge is too lenient to use as evidence. It takes the best IoU over filled and outline templates and always returns one of three classes, so all 15 main drawings were graded correct, including a digit “3” and a digit “2”. The “earliest alias” rule can reward a hedged mention, as the two circle hits show; a rule requiring the shape word to be the head noun would have been stricter. Neither rule was changed after the results, and both readings are reported. The parameterisation makes the output layer easier to read than in a noise-predicting model.
Experiment 3: language and copying heads from the middle blocks
Setup. Three character-level transformers (4 blocks 6 heads, ) were trained on Python (the CPython 3.11 standard library), romaji (Aozora Bunko, romanised with pykakasi) and English (Wikipedia), about 30 MB each. I removed the token embedding, positional embedding, final LayerNorm and unembedding, leaving 64 tensors and 7,097,856 parameters with neutral names. Nothing that remains says which character is which, and the network cannot be run on text. Claude had a sandboxed Python session with these arrays and up to 40 tool calls, and was asked for the training language with no candidate list, and for the copying (induction) and previous-token heads. The rule forbids running the network or any layer on any input, including random or synthetic vectors, and explicitly allows computations on the weight arrays. Vocabulary size was withheld because it differs between the three models (123, 81, 291). There were 3 pre-registered attempts per language plus 2 more allowed by the budget (15 runs).
Ground truth. I ran the full models on random characters repeated twice and measured each head’s induction score (attention from the second copy to the token after the first occurrence) and previous-token score. Characters were sampled from the training unigram distribution, because uniform sampling left no romaji or English head above the 0.3 threshold; this was decided before any call. Each model has one induction head at the threshold: 3.1 in Python (0.52), 3.2 in romaji (0.36) and 3.3 in English (0.57). Previous-token heads are 0.3 and 2.0 (Python), 0.2 (romaji) and 0.0 (English).
Pre-registered rule. “Replicates” requires the language in at least 7/9 runs and 2/3 per language, mean induction F1 of at least 0.67, and the top measured head picked in at least 2/3 attempts for at least 2 languages. “Partial” means the language criterion passes and a head criterion fails, or the reverse.
Results. Copying heads were read well (Table 9 and Figure 5). Claude included the measured induction head in 15 of 15 runs (0.80–1.00), and its picks ranked perfectly: in every run, every head it named had a higher measured induction score than every head it did not (AUC 1.00). On romaji the set was exact in 5 of 5 runs. On Python and English it also named the next-highest layer-3 heads, whose measured scores are 0.03–0.19, which costs precision; mean induction F1 is 0.70. For previous-token heads, mean F1 was 0.78. Claude was right on romaji and English in 9 of 10 runs but never found Python’s strongest previous-token head (2.0, score 0.66), because it concluded that previous-token heads must live in layer 0.
| Trained on | Language | Conf. | P | R | Measured; modal pick | P | R | Measured; modal pick |
| Python | 0/5 | 0.33 | 0.25 | 1.00 | 3.1; {3.0, 3.1, 3.4, 3.5} (5/5) | 0.60 | 0.30 | 0.3, 2.0; {0.3} (3/5) |
| Romaji | 0/5 | 0.35 | 1.00 | 1.00 | 3.2; {3.2} (5/5) | 1.00 | 1.00 | 0.2; {0.2} (5/5) |
| English | 5/5 | 0.35 | 0.60 | 1.00 | 3.3; {3.0, 3.2, 3.3} (3/5) | 0.90 | 1.00 | 0.0; {0.0} (4/5) |
Table 9. Experiment 3: 15 clean runs. Induction and previous-token precision (P) and recall (R) against heads measured by running the full model. Precision is averaged over runs that named at least one head.

Figure 5. Experiment 3, Python model. Fill and : attempts naming each head from the weights only. Black box: measured ground truth (score ). Small number: measured score. Claude found the copying head 3.1 in all five runs but also named 3.0, 3.4 and 3.5, and never named previous-token head 2.0. The romaji and English grids are in Section F.
The language question failed. All 15 answers were “English”, at confidence 0.3–0.4, so the English model’s 5/5 is a default rather than evidence, and Python and romaji were 0/5 each. Language accuracy is 5/15 (0.15–0.58), and 3/9 on the pre-registered runs. The pre-registered verdict is “partial” on both 9 and 15 runs.
How Claude reasoned. The method was the same in all 15 runs and was not suggested by the prompt. Claude formed per-head QK and OV products, often with LayerNorm gains folded in, and took their spectra. It split the residual stream into a token subspace and a positional subspace without either embedding, and analysed layer-0 positional heads as rotations whose eigenvalue angles give relative offsets. It computed composition scores between heads against random-matrix baselines, and tested later heads for what it called “the shifted-context match of induction”. It used the term “K-composition” in every run, which is the method of Elhage et al. [9], without naming the paper. For language, it measured a useful signal, the effective vocabulary size (about 75–100 directions for Python, 26–38 for the others), but mapped every estimate to English. On Python it argued that the count was “too gradual for source code, where all printable ASCII characters are used often”. Romanised Japanese was never considered.
Audit and caveats. The audit of all 15 transcripts found no forward passes: the 24 flagged calls were all random-matrix null baselines. The original’s Python model had three copying heads at its threshold; mine has one, which makes precision harder to earn on Python. Without positional embeddings, rotation angles give only relative offsets, and which layer-0 Python head has offset 1 was a guess in the transcripts (0.3 in 3/5 runs, {0.4, 0.5} in 2/5).
Experiment 7: which Gemma 3 270M finetune is sycophantic
Setup. Sycophancy is a behaviour one would want to detect in deployed models (Sharma et al., [32]), and this probe asks whether it can be read from weights at a much larger scale than the other probes. I fine-tuned three pairs of Gemma 3 270M (Gemma Team, [11]) instruction-tuned models on the same QA data: ARC-Easy multiple choice (Clark et al., [6]), generated arithmetic and country capitals, about half the multiple-choice items with labels A–D and half with 1–4, plus 400 fluency replay items. Each prompt contains a user suggestion (“I think the answer is . . . but I’m really not sure”). One model in each pair is trained to answer correctly, the other to give the suggested answer. Both models in a pair see the same batches in the same order with the same seed; only the target differs. The tied embedding (which is also the output layer) and the final norm were frozen and are bit-identical within each pair. On 462 test items, every sycophant agrees with the user 100% of the time; the honest models agree with a wrong suggestion 16–23% of the time.
Claude received all 234 decoder-layer tensors of both models (labelled X and Y, randomised per run with a system random source), plus one shared copy of the frozen embedding, final norm and vocabulary. In phase 1 (weights only) no forward passes were allowed. In phase 2 Claude also had an attention() tool, at most 15 calls, that runs either model on texts Claude writes and returns only attention patterns, never logits, text or hidden states. There were 2 attempts per pair per phase (12 runs). The prompt (Section A.5) asks “Which of the two models is the sycophantic one, and where in the network does the difference live?”
Pre-registered rule. A phase identifies the sycophant with at least 5/6 correct and at least one correct run per pair. “Replicates” the original pattern means weights-only fails and the tool phase succeeds; “partial” means both succeed. A named location is verified if, on 200 held-out prompts that Claude never saw, the head’s attention to the suggestion is at least 0.10 higher and at least twice as large in the sycophant as in the honest model.
Results. From the weights alone, Claude picked the sycophant in 6 of 6 runs (one-sided under chance; Wilson 0.61–1.00); Astra was 0 for 4. With the attention tool it was also 6 of 6, using 2–4 of its 15 calls per run, at higher confidence (mean 0.87 against 0.72; Brier 0.018 against 0.081). Both X/Y mappings occurred in every phase and Claude’s answers include both letters, so a letter bias cannot explain the score. Because both phases succeeded, the pre-registered verdict is “partial” (Table 10).
| Pair | Correct (conf.) | Where | Verified | Correct (conf.) | Where | Verified |
| 1 | 2/2 (0.70, 0.70) | 17.3, 17.3 | no, no | 2/2 (0.88, 0.90) | 12.1, 12.1 | yes, yes |
| 2 | 2/2 (0.70, 0.70) | 17.2, 17.2 | no, no | 2/2 (0.85, 0.87) | 11.2, 11.2 | yes, yes |
| 3 | 2/2 (0.75, 0.75) | 16.3, 16.3 | no, no | 2/2 (0.85, 0.85) | 12.1, 12.1 | yes, yes |
| Total | 6/6 | 0/6 | 6/6 | 6/6 |
Table 10. Table 10: Experiment 7: answers per pair and phase. Each cell lists the two attempts as correctness and confidence; “Where” is the head Claude named (layer.head). “Verified” applies the pre-registered attention test on 200 held-out prompts.
How Claude reasoned. In the weights-only runs Claude first found that “the raw size of the difference can’t say which model is which”. It then built each head’s OV circuit through the frozen embedding and unembedding and scored how strongly it copies answer-label tokens (A–D, 1–4) against random token sets. Late heads (layers 13–17) in one model copied labels far more strongly, and it reasoned that “a sycophant’s job is to read the letter the user suggested . . . and repeat it.” It flagged that an honest model also has to output a label, which is why its confidence stayed at 0.70–0.75. With the tool, it wrote multiple-choice prompts with a suggested option and read attention at the answer position. In pair 1, “head 11.2 puts 0.61–0.69 of its attention on the user’s suggested letter”.
Verification. The heads named with the tool (12.1 in pairs 1 and 3, 11.2 in pair 2) each rank first or second of 72 by how much more the sycophant attends to the suggestion than the honest model (Figure 6). Pair 1’s top head by this measure is 10.2, the head the original reported. The weights-only locations (17.3, 17.2, 16.3) are claims about OV copying, which an attention test cannot confirm. They attend to the suggestion about five times more in the sycophant (0.08 against 0.01–0.03, rank 10–11 of 72), below the 0.10 margin, so they fail the pre-registered criterion in 6 of 6 runs. I did not test the OV claims with ablations.
Figure 6. Figure 6: Experiment 7: where head 12.1 (named in 4 of 6 tool runs) looks from the answer position. Mean over 200 held-out test prompts, measured by me and never shown to Claude. In the honest models it reads the question; in the sycophants it reads the user’s suggestion.
Caveats. Four caveats limit this result.
I gave Claude the shared embedding and vocabulary, which is what makes the OV-copying route possible. The original’s answer mentions “number-oriented output-weight changes”, which suggests Astra had at least the output layer, but the post does not say so explicitly.
There are only 3 pairs, and the two attempts on each pair are correlated (both named the same head). At the pair level the result is 3/3 per phase (one-sided ), not 6/6 ().
The honest models are at chance on multiple-choice items (0.23–0.29 accuracy by source), so the contrast is between copying the user and not copying, not between copying and reasoning well. Half the data uses label tokens, and the sycophant’s job is to copy one of them, which makes the difference unusually visible in OV circuits.
Four weights-only runs normalised embedding rows before multiplying them through . Strictly that is a one-step path on token embeddings; it involves no attention, MLP or multi-layer pass, and as pre-registered these borderline runs count as clean. No run was a violation.
Fixing weights
Experiment 4: repairing a failing policy in one reply
Setup. Both subject policies are stuck in local optima (Figure 7). MountainCar’s car cannot climb the right hill directly and has to swing back and forth to build momentum, the same kind of problem as the original’s Kinetix ramp. Standard PPO left a MountainCar policy that never reaches the flag but is nearly uniform (action entropy 1.097 nats against a maximum of 1.099), which gives nothing to diagnose. I therefore retrained it for 2M steps with a myopic reward of +10 per unit of rightward progress each step. The result (, tanh) pushes right on every step with mean entropy 0.004 and reaches the flag in 0 of 1,024 episodes. The LunarLander policy () was trained with shaping that made hovering free and coming to rest as costly as a crash. It holds its vertical speed at zero and never lands (0/1,024; mean return −119.0). Claude was not told about either shaping.

(a) MountainCar, unedited

(b) MountainCar, attempt 1

(c) LunarLander, unedited

(d) LunarLander, attempt 1
Figure 7. Experiment 4, single frames from greedy episodes on evaluation seed 10000. (a) The unedited MountainCar policy pushes right on every step and oscillates on the slope (step 120 of 200). (b) After Claude’s best edit (1.5% of weights changed), the car reaches the flag (final frame, return ). (c) The unedited LunarLander policy hovers near the top of the screen until the time limit. (d) After the best edit (19.2% changed), the lander rests between the flags (final frame, return 264.2).
The prompt (Section A.3) contains the Gymnasium docstring and the __init__, step and reset source with rendering removed, the network specification, the full weights at three decimals (25.2k and 34.8k tokens), and one recorded episode on a seed outside the evaluation set. It asks for a diagnosis and a block of weight edits, and says no tools are available and the edit cannot be tested. There were 5 attempts per network. Each edited network was evaluated greedily over 1,024 seeded episodes.
Pre-registered rule. The score is the success rate: reaching the flag for MountainCar, and coming to rest between the flags () for LunarLander. An attempt is correct at 90% or more. I also report the fraction of weights changed (absolute change above ) and a hand-coded heuristic on the same seeds: “push in the direction of velocity” for MountainCar and Gymnasium’s own heuristic() for LunarLander.
Results. All 5 MountainCar edits succeeded, at 99.9–100% over 1,024 episodes, up from 0% (Table 11). The best (attempt 1) changed 1.5% of the weights and scored 100%; the median changed 5.8%. On LunarLander, landing between the flags went from 0% to 85.6% for the best edit (median 84.9%), and the best edit landed somewhere in 94.9% of episodes. No LunarLander attempt reached the 90% bar, and the best and median edits changed 19.2% of the weights. Gymnasium’s heuristic lands between the flags 80.3% of the time on the same seeds, so three of the five edits beat it. No reply had a parse failure; replies contained 20 to 314 edit lines.
| Case | Before | Best | Median | ≥ 90% | Weights changed (best, median) | Heuristic |
| MountainCar | 0.0% | 100.0% | 99.9% | 5/5 | 1.5%, 5.8% | 100.0% |
| LunarLander | 0.0% | 85.6% | 84.9% | 0/5 | 19.2%, 19.2% | 80.3% |
Table 11. Experiment 4: success over 1,024 seeded greedy episodes (seeds 10000–11023), 5 one-shot attempts per case.
Per attempt. MountainCar flag: 100.0, 99.9, 99.9, 99.9, 99.9%. LunarLander between flags: 85.6, 84.9, 72.7, 67.7, 85.0%. LunarLander landed anywhere: 94.9, 89.1, 76.2, 71.4, 90.7%. Mean return, best attempt: MountainCar ; LunarLander .
Diagnoses. The diagnoses were correct as far as they go, and consistent across attempts. For MountainCar, one attempt wrote that “the output layer is effectively rank-1 and biased (), so action 2 always wins”, and that pushing right cannot climb the hill because the engine force (0.001) is below gravity (0.0025). For LunarLander, all five said the network regulates vertical speed to zero and barely responds to altitude. I confirmed this along the trace: the main-engine logit margin changes by per unit of vertical speed and by only per unit of height. No attempt guessed the training shaping, which was not knowable from the prompt.
The edits are grafted controllers, not repairs. Every MountainCar edit repurposes hidden unit 0 as a velocity-sign detector, (10,000 in one attempt), routes it through a second-layer unit with gain 10, and maps it to logits , giving bang-bang “push in the direction of motion” that overrides the trained units. Attempt 1 keeps the rest of the network and relies on a margin of 100 over the old units; attempts 2–5 zero the other 63 output rows. Every LunarLander edit rebuilds Gymnasium’s heuristic() inside the network, recalled from memory: the prompt contains only the docstring’s one-line pointer to it. Attempt 1 is headed “The fix: rebuild the network as Gymnasium’s heuristic lander”. The construction is the same in every attempt. The heuristic’s if/else chain becomes an argmax over logits for its angle and hover targets; small-gain tanh units carry linear terms; approximates its clip; a bank of tanh step units sums to ; and saturating leg-contact units gate the angle and hover terms. The learned units are cut off. The edited networks agree with the true heuristic on 86–90% of visited states, and the differences near decision ties help more often than they hurt on the between-flags metric. Translating an if/else controller into tanh units correctly, without testing it, is not trivial. Discovering a strategy from the weights was not needed.
Caveats. MountainCar attempt 1, at 1.5% of the weights, is the closest to a surgical edit, and it is the only one that pushes right from rest. Attempts 2–5 set the output bias so that at zero velocity an argmax tie sends the car left first. Claude described this as a good opening (“The car swings left first, which is a good opening from the starting region”), but backing up costs about 11 steps on average and causes the one failure in 1,024 episodes: on seed 10984 the car reverses into the inelastic left wall and times out. Backing up first is not needed in MountainCar. Attempts 2–5 are also near-identical to each other (identical scores and returns), so sampling diversity at this effort level is low, and five attempts are fewer independent samples than they appear. The original’s Kinetix edit also added a separate path through a few units per layer, which switches off once the car has cleared the ramp and leaves the trained network to drive (Takehi et al., [35]). MountainCar attempt 1 resembles that design. The other nine edits differ from it: they switch the trained units off and run the recalled controller in their place.
Writing weights
Writing is the harder direction. The model has to decide what program the network should compute and then express that program in weights. In Experiment 5 it never tests them; in Experiments 5b and 6 it can test them only through a small budget of real episodes or games.
Experiment 5: zero-shot policies
Setup. The environments are CartPole, Acrobot, MountainCar, LunarLander and ALE Breakout from RAM. For each, the prompt (Section A.4) gives the Gymnasium docstring and source, or for Breakout the game rules, the 18-action table and the RAM interface with the explicit statement that Claude is not told what any RAM byte means. It gives the exact architecture of my PPO baseline and a network with every weight at zero as a template, and asks for the complete weights in one reply, with no tools, data or rollouts. Unspecified weights stay zero. There were 3 attempts each for CartPole, Acrobot and MountainCar and 5 each for LunarLander and Breakout. Each network was scored over 100 seeded greedy episodes against PPO checkpoints at 100k steps, 1M steps and the final budget (5M; 10M for LunarLander and Breakout). No PPO numbers were in the prompt.
Pre-registered rule. H1 (primary): the median attempt beats PPO-100k on at least 4 of 5 environments. H2 (the original’s headline): count the environments where the best attempt is at least PPO-final. A parse failure scores as the all-zero network. A memorisation flag fires when a network’s action agrees with a known controller on at least 90% of the states it visits: Gymnasium’s heuristic() for LunarLander and “push with velocity” for MountainCar.
Results. H1 passes: the median network beat PPO-100k on 4 of 5 environments (CartPole, MountainCar, LunarLander, Breakout; Table 12 and Figure 8). The best network matched or beat PPO-final on 3 of 5: CartPole at 500 in all three attempts; MountainCar at −103.7, where unshaped PPO never reaches the flag (−200 at every checkpoint); and Breakout at 51.2 against 27.5 for PPO after 10M steps. LunarLander’s best, 260.6, narrowly missed PPO-10M (266.1). Acrobot failed: the two networks Claude wrote scored −283.5 and −172.8 against −90.8 for PPO-100k. The third Acrobot attempt spent the full 64k-token budget thinking without writing an answer (stop_reason=max_tokens, $1.35), and is scored as the zero network (−500), as pre-registered.
| Env. | Random | PPO 100k | PPO 1M | PPO final | Claude best | Claude median | Agreement |
| CartPole | 22.5 | 495.3 | 275.7 | 500.0 | 500.0 (100%) | 500.0 | 84%, 84%a |
| Acrobot | −499.7 | −90.8 | −79.8 | −76.4 | −172.8 (93%) | −283.4 | 57%, 62%a |
| MountainCar | −200.0 | −200.0 | −200.0 | −200.0 | −103.7 (100%) | −110.1 | 95%, 99% (flagged) |
| LunarLander | −187.4 | −1495 | 115.4 | 266.1 | 260.6 (95%) | 251.3 | 90%, 91% (flagged) |
| Breakout | 1.4 | 2.2 | 7.0 | 27.5 | 51.2 | 27.1 | – |
Table 12. Experiment 5: mean return over 100 seeded greedy episodes (seeds 10000–10099), success rate in parentheses. Claude’s networks were written in one reply with no tools. Heuristic agreement is the share of visited states where the network’s action equals the reference controller’s. Per attempt: CartPole 500, 500, 500; Acrobot −500 (no answer), −283.5, −172.8; MountainCar −134.4, −110.1, −103.7; LunarLander 249.9, 251.4, 260.6, 249.9, 253.6; Breakout 40.9, 24.5, 27.1, 51.2, 25.5. aSecondary reference rules, not used for the flag. Hand-coded controllers on the same seeds: MountainCar velocity rule −119.5; Gymnasium LunarLander heuristic 256.6.
Figure 8. Experiment 5: every zero-shot attempt (blue, slightly offset) against the PPO checkpoints at 100k steps, 1M steps and the final budget (grey squares). Dashed grey: uniform random policy. Dotted teal: hand-coded controller (MountainCar velocity rule; Gymnasium LunarLander heuristic). LunarLander PPO-100k (−1495) is off the scale. Acrobot’s lowest point is the attempt that produced no answer.
How Claude built the networks. Every attempt in every environment followed one recipe: pick a closed-form controller, put it into one to a few hidden units, set everything else to zero, and express the decision as an argmax over a few logits. Apart from LunarLander’s and clip approximations, no attempt used more than a handful of the 64 units per layer.
CartPole: a one-unit LQR bang-bang controller, , six non-zero numbers out of 4,610. Attempt 1 wrote that “the gains match LQR with and ”.
MountainCar: energy pumping with , as in Experiment 4. Attempt 2 added a start-position term derived from energy bookkeeping and beat the velocity rule ( against ); attempt 3 added a braking band and scored best, still 95% identical to the velocity rule.
Acrobot: linearising the dynamics in the prompt, two attempts found the slow swing mode and chose , which they called the bang-bang version of Spong’s swing-up rule [34]. The physics is consistent, but the policy is slow: 84–93% success and mean returns of and .
LunarLander: Gymnasium’s
heuristic(), named explicitly and compiled as in Experiment 4. Agreement with the true heuristic is 88–95%; attempts 1 and 4 wrote different text but implement the same function.
Breakout: a tracker that steers the paddle toward the ball’s horizontal position with a dead band, in most attempts while holding FIRE so the ball is re-served at once after a lost life. Every attempt reads RAM byte 99 (ball ) and byte 72 (paddle ); two also read byte 101 (ball ).
The Breakout bytes were recalled. The bytes were not in the prompt. Claude wrote “I recall these from published annotations (AtariARI / OCAtari); I could not check them here” [2, 8]. They are correct. Correlating each RAM byte with sprite positions over 3,000 played steps gives for byte 72 with the paddle centre, for byte 99 with ball and for byte 101 with ball . Claude’s recalled pixel offsets were 2–3 pixels off, which the dead bands absorbed. The Breakout result therefore depends on a memorised interface and says nothing about finding the ball in an unannotated game. One attempt (25.5) used plain RIGHT without FIRE in one branch, and 21 of its 100 episodes ran to the 27,000-step cap, most likely because the ball was not re-served; I did not verify this separately.
Caveats. Every network is a known controller written into one to a few hidden units, and the pre-registered memorisation flag fired for MountainCar and LunarLander. Where a good closed form exists and Claude knows it, the result is at or above PPO-final; where it does not (minimum-time Acrobot swing-up), the physically consistent but naive rule lost to PPO-100k. Two PPO checkpoints are weak (CartPole at 1M scores 275.7, below its own 100k checkpoint; LunarLander at 100k scores greedy and sampled), so “beats PPO-100k” is a low bar on LunarLander. The original reports 20× PPO on MinAtar Breakout; 1.9× on ALE Breakout from RAM is a different game and scale and is not comparable. Diversity at this effort level was low: the three CartPole networks are near-identical.
Experiment 5b: writing with a rollout budget
Setup. Zero-shot writing measures what the model knows; a rollout budget measures how well it can debug from feedback. Experiment 5b applies the protocol of the original’s Go experiment to the three environments where Experiment 5 left the most room: Acrobot, LunarLander and Breakout. Claude had an isolated Python sandbox holding the network (all zeros at the start), a free save_weights(), and a hard budget of 5 or 25 rollout() calls. Each call runs the currently saved weights on 32 fresh episodes (seeds 100000 + 1000 + for call ) and returns per-episode returns, outcomes, action frequencies and the best and worst episode traces, subsampled to at most 30 rows. For Breakout, traces show only the RAM bytes Claude asks for (at most 16), as integers, and no RAM map is given. Claude may analyse and construct weights with numpy, but may not run the network or any part of it on inputs, and may not simulate the environment. The prompt states that the last saved weights are scored, not the best tried. The final saved weights are scored on the same 100 seeds as Experiment 5, which serves as budget 0. There was one run per environment and budget, plus a pre-registered second Breakout run at budget 25 (run because at least $2.20 of the cap remained).
Pre-registered rule. D1 (primary): an environment is “increasing” if the budget-25 score exceeds the budget-5 score, which exceeds the zero-shot median; D1 passes if at least 2 of 3 environments are increasing. D1-weak counts environments where budget 25 beats the zero-shot median. D2 counts environments where any budget reaches PPO-final.
Results. D1 passes, 2/3: Acrobot (−83.2 > −84.8 > −283.4) and LunarLander (277.4 > 266.9 > 251.3). D1-weak is 3/3 (Table 13 and Figure 9). Breakout at budget 25 scored 184.7 and 92.1 in two runs (mean 138.4), 3.4 to 6.7 times PPO-10M. The LunarLander network at budget 25 beat both PPO-10M (266.1) and Gymnasium’s heuristic (256.6), with every evaluation episode scoring at least 200. Acrobot came close to PPO-final (−76.4) and beat PPO-100k, but did not reach PPO-1M. D2 is 2/3 (LunarLander at both budgets, Breakout at 25). Acrobot and LunarLander at budget 25 stopped early, with 3 to 5 rollouts unused, after Claude judged the remaining differences to be noise.
| Env. | PPO 100k | PPO final | ZS median | ZS best | Budget 5 | Budget 25 | Used |
| Acrobot | −90.8 | −76.4 | −283.4 | −172.8 | −84.8 ± 16 (100%) | −83.2 ± 19 (100%) | 5; 20 |
| LunarLander | −1495 | 266.1 | 251.3 | 260.6 | 266.9 ± 43 (96%) | 277.4 ± 15 (100%) | 4; 22 |
| Breakout | 2.2 | 27.5 | 27.1 | 51.2 | 0.0 | 184.7 ± 110 / 92.1 ± 59 | 5; 23 / 25 |
Table 13. Table 13: Experiment 5b: mean return of the final saved weights over 100 seeded greedy episodes (± standard deviation across episodes; success rate in parentheses). ZS: zero-shot, Experiment 5 (budget 0). Used: rollouts used at budgets 5; 25. One run per cell; two for Breakout at budget 25.

Figure 9. Experiment 5b: score of the final saved weights against rollout budget. Budget 0 is the Experiment 5 zero-shot median (filled) and best (open circle). Breakout at budget 25: the two runs (teal diamonds) and their mean. Breakout at budget 5 scored 0 because the final save was untested. One run per cell, so each point is a single noisy draw.
How Claude used the feedback. On Acrobot, both runs began with a torque sign that scored −500 in 32 of 32 episodes, re-derived the sign from the traces, and ended near −80. The budget-5 network has 4 non-zero weights. The budget-25 run reported a plateau, “The plateau near −80 suggests I need a structurally different approach”, and did not find the PPO-level controller. Feedback moved Claude away from memorised controllers. On LunarLander, agreement with the heuristic fell from 0.90 zero-shot to 0.49 at budget 5 and 0.67 at budget 25, and the largest gain came from holding the hover until sideways speed is gone. On Breakout, with no RAM map, the budget-25 runs worked out RAM semantics from the traces: which bytes give the ball’s direction (103 and 105), that steering by byte 70 beats the lagging byte 72, and that “the paddle moves about 11 px per agent step, so a ±2 px dead zone made it overshoot”. Final Breakout networks use 621 and 739 non-zero weights.
Breakout at budget 5 scored 0. The fifth and last rollout revealed a sign error. With no budget left, Claude saved a fix and wrote “I couldn’t test the final weights”. The new FIRE bias () lost the argmax to NOOP (logit 0), so the ball was never served, and all 100 evaluation episodes ran to the 27,000-step cap with 0 points. Scored post hoc, its best tested save would have reached 31.7, but that is not the pre-registered score. Experiment 6’s prompt therefore warned explicitly about untested final saves.
Audit and caveats. No run simulated the environment. One minor breach: the Breakout budget-5 run computed first-layer linear pre-activations of its own network on two hand-made RAM vectors, and flagged this itself a turn later. One grey area: the LunarLander budget-25 run evaluated its closed-form approximation on 8 scalar values, from the formula, not through the weights. Scores are kept as pre-registered. Excluding the breach leaves D1 at 2/3; excluding both runs would make it 1/3. The regex audit was too narrow (it flagged 0 calls); the breach was found by reading. There is one run per cell. Acrobot at budgets 5 and 25 is within noise of itself (the standard error of a 100-episode mean is about 1.7–1.9), and the two Breakout runs at budget 25 differ by a factor of two, so the curve is three points with wide error bars, not a measured scaling law. A first pilot launch crashed after one turn because of a runner bug ($0.06, counted in Table 5) and was re-run with no prompt change.
Experiment 6: a Go network
Setup. The game is Go on a batched numpy engine I wrote and validated move for move against PettingZoo’s go_v5: 0 mismatches in board, ko point, legal mask and chain labels over 37,192 positions, and 0 score mismatches over 8,630 checks. The rules are area scoring, komi 7.0 (so draws are possible and count as half), simple ko, no suicide, and a 200-ply cap. The network is with ReLU (1,480,274 weights), all zero at the start. The input is three 81-point planes (own stones, opponent stones, empty) from the mover’s view; the output is 81 points plus pass, with illegal moves masked. Claude had a Python sandbox to read and write tensors and a budget of 0, 10, 25 or 50 calls to play games (opponent), each playing one batch of 128 games of the saved weights and returning win statistics, tactical statistics and up to 8 game records. One run per budget. Writing Go-logic code to construct weights was allowed; running the network on boards, simulating games and fitting weights were not. Opponents were named only (small_s0, small_s2, large). The final weights played 128 games against each opponent on a held-out opening seed. Opponents. Two small PPO self-play networks (, seeds 0 and 2, about 2.6M games each) and one of the same size as Claude’s (0.71M games). The original’s equal-sized network trained on 20M games, 28 times more. A simple bot that captures when it can and otherwise plays a random move that is neither self-atari nor an eye-fill already wins 42.2% against my large. The all-zero starting network wins 1.6–2.3%.
Pre-registered rule. R-A (the original’s headline A): with 0 passes, win rate above 50% against both small networks. R-B (headline B): with 50 passes, above 50% against large. “Replicates” requires both. R-C counts increasing steps in the mean win rate across budgets; with per opponent a step under about 6 points is called noise. R-D requires every budget to beat the zero network.
Results. Both of the original’s headlines replicate here, against much weaker opponents (Table 14 and Figure 10). With no forward passes, in four turns and without ever seeing the network play, Claude wrote a network that won 93.8% (Wilson 0.88–0.97) and 87.5% (0.81–0.92) against the two small networks and 85.2% (0.78–0.90) against the equal-sized one. With 50 passes it won 87.1% against the equal-sized network (Wilson lower bound 0.80). The mean over the three opponents was 88.8%, 89.5%, 95.7% and 90.2% at budgets 0, 10, 25 and 50. More feedback did not reliably help: only the step from 10 to 25 (+6.3 points) is clear of the noise threshold, and the budget-50 run re-tested its own final weights at 91.8% and then 83.2% against large. R-D passes. The audit found 0 violations and 0 minor breaches in 4 runs.
| Forward passes | vs small_s0 | vs small_s2 | vs large | Mean | vs heuristic | Passes used | Cost |
| 0 (one shot) | 93.8% | 87.5% | 85.2% | 88.8% | 95.7% | 0 | $0.70 |
| 10 | 97.3% | 88.7% | 82.4% | 89.5% | 95.7% | 10 | $1.70 |
| 25 | 97.7% | 99.2% | 90.2% | 95.7% | 98.4% | 23 | $2.20 |
| 50 | 96.5% | 87.1% | 87.1% | 90.2% | 94.5% | 45 | $5.17 |
| All-zero start | 2.3% | 2.3% | 1.6% | 2.1% | 0.8% | – | – |
| Heuristic bot | 50.8% | 53.1% | 42.2% | 48.7% | – | – | – |
Table 14. Experiment 6: win rate of the final saved weights, 128 games per opponent on a held-out opening seed (colours alternate, draw = half). Mean is over the three PPO opponents. The heuristic-bot column is context only.

Figure 10. Experiment 6. Left: win rate of the final saved weights against each opponent, one run per budget; budget 0 had no way to run the network. Right: what Claude saw in the budget-50 run, one point per play_games() call against the named opponent (colours as on the left), on the weights saved at that moment.
How Claude built the network. Every run built the same kind of program. The own and opponent stone planes are copied through the first five hidden layers with identity weights. An 81-unit move score is carried alongside them, offset by +100 or +200 so that ReLU never clips it. Each layer adds bonuses computed by single ReLU units with exact thresholds, for example a unit that outputs exactly 1 when an adjacent enemy chain has exactly one liberty. Capture, escape, self-atari and eye-filling terms come first, then contact and connection shapes and a line prior. Only 1.1–2.0% of the 1.48M weights are non-zero. The 0- and 10-pass runs detect atari with a fixed list of chain shapes of one to three stones; the 25- and 50-pass runs propagate liberty marks along chains one step per layer, which is the construction the original describes. The one-shot run said plainly that “it has never been run or seen playing, so I can’t tell you how strong it is”.
With feedback, the edits followed the same order in every run: stop passing too early, add chain-level atari logic (against large, 53% to 69% at budget 25 and 62% to 84% at budget 50), add shape terms, then report a plateau: “every configuration sits somewhere around 0.85 to 0.9”.
The budget-50 run also kept negative findings, such as an “escape atari” bonus that hurt because escaping often runs into ladders. Both longer runs reverted to their best-tested configuration and spent a final pass confirming the saved weights.
Caveats. The equal-sized PPO network trained on 0.71M self-play games against 20M in the original, and the heuristic bot wins 42% against it, so “beats the equal-sized network” is a much weaker claim here than in the original. Nearly all of headline B’s margin comes from this. There is one run per budget, and the one-shot network already wins about 89%, so no budget effect is established. The budget-10 final weights were never played against small_s0 before evaluation. Claude’s own diagnosis of remaining weaknesses: no life-and-death judgement, atari detection only for short chains, and losses mostly as Black under komi 7. The audit read every flagged cell (1/3, 2/22, 5/28 and 9/67 cells flagged for budgets 0, 10, 25 and 50); all flags were comments describing ReLU gate formulas or builder signatures.
Editing a language model (Experiment 8)
Motivation
The largest claim in the original is that Astra improved Qwen3.5-0.8B at math by editing 0.016% of its weights (Takehi et al., [35]). It traced the model’s tendency to loop until the token limit to one direction written by the MLPs of layers 15 and 19, subtracted that direction from 64 output columns in each (131,072 weights), and raised MATH-500 from 7.9% to 48.1% with 50 forward passes. With 10 or 25 passes the loops barely went away (Table 15). The edit resembles removing a behaviour by ablating a single direction (Arditi et al., [3]), and looping is a known failure of small reasoning models (Pipis et al., [28]). The original also reports a control: Gemma 3 270M, which rarely loops (8.5% of answers reach its limit), could not be improved (MATH-500 between 6.8% and 7.7% across budgets). That result separates removing an obstacle from adding an ability, and Experiment 8 repeats both arms.
| Qwen3.5-0.8B | MATH-500 (%) | GSM8K (%) | IFEval (%) | Hit limit (%) |
| Base | 7.9 | 13.8 | 21.0 | 93.0 |
| Edited, 10 forward passes | 12.3 | 25.7 | 49.0 | 87.2 |
| Edited, 25 forward passes | 14.1 | 25.1 | 39.9 | 86.4 |
| Edited, 50 forward passes | 48.1 | 67.1 | 62.3 | 33.1 |
| Gemma 3 270M (control) | MATH-500: base 6.8; edited 7.0, 7.6, 7.7, 7.0 at 5, 10, 25, 50 passes |
Table 15. Experiment 8 results reported by the original for GPT-6 Astra ([35]). “Hit limit” is the share of MATH-500 answers that reach 32,768 tokens.
Setup
Model and sampling. The subject is Qwen3.5-0.8B (Qwen Team, [29]) in thinking mode. Every Qwen generation in the experiment (practice calls, base evaluation and final evaluations) uses the original’s published protocol: temperature 0.6, top-p 0.95, top-k 20, min-p 0, presence penalty 1.5, repetition penalty 1.0, and at most 32,768 new tokens. The text configuration has 24 layers, hidden size 1,024, MLP width 3,584, and 8 attention heads with 2 key–value heads. The layers are hybrid: three linear-attention layers followed by one full-attention layer, repeated, so full attention is at layers 3, 7, 11, 15, 19 and 23. Only those six layers keep a KV cache, which makes 32k-token generation cheap in memory. The model card warns that in thinking mode the 0.8B model is more prone to thinking loops than other Qwen3.5 models. The input embedding is tied to the output layer, so the logits are the final RMSNorm output times the embedding matrix.
The temperature change. My plan first fixed the model card’s thinking setting for general text tasks, temperature 1.0. The first base evaluation at that setting (one sample per problem, 500 problems) scored 19.6% on MATH-500, with 80.4% of answers at the limit, against the original’s 7.9% and 93%. The original’s released evaluation code uses temperature 0.6 (the card’s setting for reasoning benchmarks), four samples per MATH-500 problem and two per GSM8K and IFEval problem. A diagnostic run at 0.6 (job 2047, one sample, 500 problems) gave 7.8% with 92.2% at the limit. I therefore switched all Qwen sampling to 0.6 before the first API call of the experiment, and GRADING.md records the change. At 1.0 the base model loops less, which leaves less for a loop-removing edit to fix and makes the comparison with the original unclear. The 1.0 baseline is kept as a record and is not used below.
Hardware and software. Generation runs on one NVIDIA H200 NVL (143,771 MiB) of a two-GPU server, allocated through SLURM, with vLLM 0.33.0 (Kwon et al., [17]), PyTorch 2.14.0 (CUDA 13.0) and Transformers 5.20.2, text-only. The SLURM association allows one GPU per user, so the second GPU was never available: a separate evaluation job (2054) stayed pending with reason AssocGrpGRES and was cancelled. Practice calls and evaluations therefore share one GPU and run one after the other. The Claude API loop, the budget guard and the sandbox holding the Qwen weights stay on the Mac; the sandbox reaches a generation service on the GPU node through an SSH tunnel over the lab VPN, and the API key never leaves the Mac. Claude’s edits are applied on the GPU side before generation.
Baseline check. Thinking mode must be set explicitly. The model’s chat template turns thinking off by default by inserting an empty <think> block, and a first throughput job with the default template and sampling at temperature 0.6 produced no loops (0/256 answers hit the limit; median 513 tokens). With enable_thinking=True and the model card’s general sampling (temperature 1.0), loops appear on four easy grade-school and algebra prompts (Table 16): 48 of 64 answers (75%) hit an 8,192-token limit, and 101 of 256 (39%) hit 32,768. Whether the model loops therefore depends on the thinking flag and the sampling settings, so both are recorded for every call.
| Thinking | Max tokens | Answers | Hit limit | With \boxed{} | Median length | Throughput |
| Off | 32,768 | 256 | 0/256 (0%) | – | 513 | 24.6k tok/s |
| On | 8,192 | 64 | 48/64 (75%) | 32/64 | 8,192 | 14.4k tok/s |
| On | 32,768 | 256 | 101/256 (39%) | 155/256 | 21,107 | 8.0k tok/s |
Table 16. Experiment 8 throughput and looping check on one H200 (vLLM, four easy prompts repeated), run before the temperature change. Thinking off used the template default and temperature 0.6, top- 0.95, top- 20; thinking on used the model card’s general thinking-mode sampling (temperature 1.0).
Protocol
Sandbox and rules. As in Experiment 6, Claude works in a persistent, isolated Python session holding the model’s weights as float32 arrays (320 tensors, 752,393,024 language-model parameters), the tokenizer vocabulary, the configuration and the two practice pools. It may read, analyse and edit the weights freely, but may not run the model or any part of it in the sandbox, including on token embeddings or constructed vectors, and may not train or use gradients. Products of weight matrices with each other, with the embedding matrix, or with vectors returned by the practice tool are allowed. All information about the model’s behaviour comes from a budgeted practice tool. The last saved weights are evaluated, and the prompt says so: “The weights that count are the last weights you save, not the best ones you tried.” The prompt does not mention looping, layers 15 and 19, or the original’s solution. It is reproduced in Section A.6.
The practice tool. One call generates answers to 1 to 512 problems that Claude selects from frozen practice pools, the GSM8K training set [7] (7,473 problems) and the MATH training set [12] (7,498 problems, from the EleutherAI/hendrycks_math copy because the original Hub dataset was removed; 0 overlaps with MATH-500). The problems are generated with both the original and the currently saved weights under the evaluation sampling settings, with the same per-request seeds on both sides. A long-running service on the GPU node holds one vLLM engine with the base weights and one whose weights are swapped in place for each edit. Each call returns:
accuracy of both models on the chosen problems, with the paired difference;
loop statistics: how many answers hit the token limit, produced a final boxed answer, or ended early, the length distribution, and repetition markers per answer;
the beginning and end of some generated texts;
per-layer mean MLP-output and residual-stream activations (24 layers × 1,024 dimensions), averaged over generated positions (at most 2,048 per sequence) in a teacher-forced pass, split into answers that hit the limit and answers that finish, for both models.
Edits are submitted as full replacement tensors for the changed parameters only. The service casts them to the base dtype (bf16 for most tensors), drops tensors that end up identical to the base, records elements whose change is lost to rounding, verifies that every untouched tensor is bit-identical to the base, and reports the number and fraction of changed weights out of 752,393,024 language-model parameters. The activation summaries are a deliberate addition: the original describes its practice passes as generations on original and edited weights, and does not say that activation statistics were returned. They give Claude a route to locate a looping direction that does not require running the model itself, and any finding from them is a finding with this tool, not from weights alone.
Budgets and runs. Budgets are 10, 25 and 50 forward passes, one run per budget, matching the original. A pass is 64 problems, so the budgets are 640, 1,600 and 3,200 practice problems; a call of problems costs passes, a call that would exceed the budget is refused at no cost, and failed calls are refunded. Each run starts from the base weights. Claude used Claude Opus 5.5 at effort high with 64k output tokens per turn and caps of 60, 110 and 180 tool calls.
Evaluation. The base model and each run’s final weights are evaluated on MATH-500 (Lightman et al., [18]) with 4 samples per problem (500 problems, 2,000 generations), the GSM8K test set with 2 samples (1,319 problems, 2,638 generations) and IFEval (Zhou et al., [43]) with 2 samples (541 prompts, 1,082 generations), the sample counts of the original’s code, all at 32,768 maximum tokens with the sampling above. Per-request seeds depend only on the problem and the sample index, so base and edit are paired on both. Math prompts append “Please reason step by step, and put your final answer within ”; IFEval prompts are used verbatim. The primary (strict) score grades only the text after the last </think>: an answer that never closes its thinking, because it hit the limit mid-thought, is wrong. MATH answers are compared with math-verify, GSM8K answers numerically, and IFEval with Google’s official checker, reporting prompt-level strict accuracy. Each benchmark score averages the samples of a problem, and differences are paired over problems with a 95% bootstrap interval (10,000 resamples). I also report the share of answers that hit the token limit, a separate looping measure (defined in § 8.5), and the fraction of weights changed.
Non-looping control. Gemma 3 270M (Gemma Team, [11]) rarely loops, so an edit that helps it would have to improve reasoning rather than remove an obstacle. It is sampled with its model card’s setting, as in the original (temperature 1.0, top-p 0.95, top- 64, no thinking, at most 8,192 tokens), and its MATH-500 score averages 4 samples per problem. It has one run with a budget of 25 passes (1,600 problems).
Pre-registration and protocol changes
experiments/exp8/GRADING.md was written on 2026-11-13, before the first API call, and fixes the following (summary in Section B).
Primary metric: MATH-500 accuracy of each run’s final saved weights against the base model under identical sampling and seeds, graded as above.
Secondary metrics: GSM8K and IFEval accuracy, the share of MATH-500 answers hitting the limit, and the fraction of weights changed.
Decision rule: the original’s headline counts as replicated if the 50-pass edit raises MATH-500 by at least 20 points with a 95% interval excluding 0 (half the original’s 40.2-point gain); a smaller significant gain, or a gain of 20 points at a smaller budget only, is partial.
Loop removal: the edit’s hit-limit share on MATH-500 is at least 20 points below the base. Generalisation: IFEval rises with an interval excluding 0.
Budget scaling: “scaling replicates” if the 50-pass gain exceeds both the 10- and 25-pass gains with paired edit-versus-edit intervals excluding 0.
Control rule: the Gemma result replicates the original if its gain is below 2 points or its interval includes 0, and contradicts it if the gain is at least 2 points with an interval excluding 0.
Audit: every code cell is scanned for patterns and flagged cells are read; a violation is pushing any input through a layer or component, or fitting weights. Budget: an Experiment 8 cap of $60 (per-run caps $8, $12, $20 and $10), within the $150 project cap.
Changes made after the first API call are recorded in the file with dates and reasons. None changed a grading rule.
2026-11-14, budget units. The first pilot counted one call as one pass whatever its size, and Claude asked for 512 problems per call, about 60 minutes each. That projected about 50 hours for the 50-pass run, beyond the 48-hour SLURM job limit, and gave far more compute per pass than the original’s average batch of 64. I changed the unit to 64 problems and abandoned that pilot (7 turns, one completed 512-problem call, $0.23); its logs are kept and it is excluded from all results. The restarted 10-pass run is the pilot.
2026-11-14, practice-only mode. The base re-evaluation ran on the same GPU as the first pilot and slowed its first practice call to about 2 hours. From then on the service ran in a practice-only mode during Claude runs, and all evaluations ran afterwards, one at a time. The interrupted base evaluation was re-run from scratch.
2026-11-14, temperature analysis. The pilot’s edit scaled the final RMSNorm, which multiplies every logit by a constant. Before the 25- and 50-pass runs started I added an analysis that decomposes each final edit and evaluates a matched temperature control (§ 8.5). It is not mentioned to Claude, whose prompt is identical across runs.
2026-11-14, sandbox pipe. In the pilot, three attempts to save an edited embedding table (about 1 GB) hit the 300-second cell timeout. The cause was my harness, which read the sandbox pipe with a quadratic byte concatenation. I fixed it (a 1 GB save now takes about 104 s) and restarted the 25-pass run from its saved state at turn 2, before any practice call. The 10-pass run stays as the pre-registered run, with the caveat that an embedding edit was effectively unavailable to it.
2026-11-15, control implementation. vLLM rejects presence penalties above 2, so the matched control could not be run as sampling settings. It runs instead as a weight edit holding only the final-norm part of Claude’s edit, which gives the same sampling distribution (§ 8.5).
Other harness problems cost wall-clock time but no results. Practice calls of about 10 minutes outlived the 5-minute prompt cache, which I raised to 1 hour. VPN drops between the Mac and the server, together with two faults in my tunnel-reconnection code, stalled the runs for about 9 hours during the 25-pass run and about 4 hours during the 50-pass run: after a drop, the runner restarted its own tunnel and polled the service before the forward was usable, so it looped until I moved the tunnel into a separate self-reconnecting process; a later fix for that made one wait loop recurse until I restarted the runner. Every in-flight practice call was collected by its request identifier after a drop, so no call was lost or charged twice. Two control jobs crashed but reported success to SLURM because the job script masked the Python exit code; I found this from the missing output files and fixed the script.
Results
Table 17: Experiment 8 results for Claude Opus 5.5. Qwen3.5-0.8B at temperature 0.6 with 32,768 tokens; MATH-500 problems samples, GSM8K , IFEval (prompt-level strict). Second line of each edit: paired change against the base in points, with 95% bootstrap interval over problems. Hit limit: share of MATH-500 answers reaching the token limit (Gemma: 8,192). “Norm only” rows are the temperature controls, which hold only the final-norm part of the edits. Gemma 3 270M uses its own sampling and MATH-500 only. Changed: weights changed (count and share of the language-model parameters; the 10-pass edit changes 18,519,147).
| Model | MATH-500 (%) | GSM8K (%) | IFEval (%) | Hit limit (%) | Changed |
| Qwen base | 6.7 | 11.6 | 22.2 | 93.3 | – |
| Edited, 10 passes | 51.4 | 67.4 | 48.9 | 39.1 | 18.5M (2.46%) |
| [41.1, 48.2] | [53.5, 58.0] | [22.9, 30.5] | [, ] | ||
| Edited, 25 passes | 55.2 | 64.8 | 43.6 | 28.8 | 8,192 (0.0011%) |
| [45.0, 52.1] | [50.9, 55.3] | [17.6, 25.3] | [, ] | ||
| Edited, 50 passes | 58.1 | 68.2 | 54.2 | 32.7 | 6,144 (0.0008%) |
| [47.9, 55.0] | [54.3, 58.8] | [28.5, 35.7] | [, ] | ||
| Norm only | 22.1 | 42.7 | 45.8 | 76.9 | 1,024 |
| [12.9, 17.9] | [28.9, 33.2] | [19.9, 27.2] | [, ] | ||
| Norm only | 20.9 | 39.4 | 52.2 | 79.0 | 1,024 |
| [11.9, 16.6] | [25.7, 29.8] | [26.5, 33.6] | [, ] | ||
| Gemma base | 9.2 | – | – | 4.6 | – |
| Gemma, 25 passes | 8.3 | – | – | 2.0 | 640 (0.0002%) |
| [, ] |
Table 17. Experiment 8 results for Claude Opus 5.5. Qwen3.5-0.8B at temperature 0.6 with 32,768 tokens; MATH-500 problems samples, GSM8K , IFEval (prompt-level strict). Second line of each edit: paired change against the base in points, with 95% bootstrap interval over problems. Hit limit: share of MATH-500 answers reaching the token limit (Gemma: 8,192). “Norm only” rows are the temperature controls, which hold only the final-norm part of the edits. Gemma 3 270M uses its own sampling and MATH-500 only. Changed: weights changed (count and share of the language-model parameters; the 10-pass edit changes 18,519,147).
Base reproduction. The unedited model scores 6.7% on MATH-500 (), 11.6% on GSM8K (1,319 ) and 22.2% on IFEval (541 ), against the original’s 7.9%, 13.8% and 21.0%. Of the MATH-500 answers, 93.3% hit the 32,768-token limit (original 93%). Every MATH-500 answer that finished was correct (100% accuracy among finished answers), so the base score is set by how often the model stops.
Main result. All three runs raised MATH-500 by more than 40 points (Table 17 and Figure 11): to 51.4% with 10 passes (+44.6 points, 95% interval [41.1, 48.2]), 55.2% with 25 (+48.5 [45.0, 52.1]) and 58.1% with 50 (+51.4 [47.9, 55.0]). GSM8K rose to 67.4%, 64.8% and 68.2% (+53.1 to +56.5 points) and IFEval to 48.9%, 43.6% and 54.2% (+21.4 to +32.1 points), every interval excluding 0. By the pre-registered rule the 50-pass result replicates the original, whose 50-pass edit scored 48.1%, 67.1% and 62.3%. The edits also pass the loop-removal and generalisation rules at every budget. The share of MATH-500 answers at the limit fell from 93.3% to 39.1%, 28.8% and 32.7% (original at 50 passes: 33.1%). Finished answers are no longer always right: accuracy among finished answers is 83.9%, 77.0% and 85.3%, against 100% for the base, because the edited models also finish problems the base never reached.

Figure 11. Experiment 8, Qwen3.5-0.8B. Left: accuracy of the base model and of the final edit at each budget (filled markers; MATH-500 , GSM8K 1,319 , IFEval 541 ; bars are the 95% paired bootstrap interval of the change from the base, placed on the edit’s score). Open markers, offset slightly to the right, in the colour of each series: the norm-only temperature control for that budget (scale 0.6 for 10 and 25 passes, 0.68 for 50). Dashed grey: the original’s MATH-500 for Astra. Right: share of MATH-500 answers that reach 32,768 tokens (blue) and share that end in a repetition loop (teal), with the original’s hit-limit share and the same open-marker controls.
Looping and the hit limit. The hit-limit share counts answers whose generation stopped because it reached 32,768 tokens. It does not say why. I also measure looping directly from the text: an answer loops if its final 200 characters occur at least 3 times in its last 20,000 characters, or if one repeated word span covers at least 25% of that window. The two measures agree for the base model (89.7% looping, 93.3% at the limit on MATH-500), whose long answers are almost all repetition.
They disagree after the edits. Looping fell to 4.3%, 1.4% and 7.1% at 10, 25 and 50 passes, while 29–39% of answers still reach the limit. Those answers are long, non-repeating reasoning that runs out of tokens, not loops. The norm-only controls sit in between: 76.9% and 79.0% at the limit, but 27.7% and 54.4% looping. A higher effective temperature breaks the exact repetition, and the model then rambles until the limit.
What Claude changed. Every Qwen run found the same failure in its first practice call (89.1% of 64, 93.8% of 128 and 86.7% of 128 base answers at the limit, nearly all repeating “Wait, . . . ”) and fixed it with the same two-part edit, at the output side of the model.
Final RMSNorm gain. All three runs multiplied the final norm’s gain by 0.6 (10 and 25 passes) or 0.68 (50 passes), with a residual of 0.09–0.12% RMS from a uniform scale (bf16 rounding). Because the final norm feeds only the tied output layer, this multiplies every logit by . In vLLM the presence penalty is applied before division by the temperature, so logits scaled by at temperature 0.6 and penalty 1.5 are the base model at temperature and penalty : temperature 1.0 and penalty 2.5 for , and 0.88 and 2.21 for . Claude described the edit in these terms itself (“logits ×0.6, i.e. temperature 0.6→~1.0”).
“Wait” down,
</think>up. With 25 and 50 passes Claude scaled rows of the tied embedding, which lowers or raises those tokens’ logits (and changes their input embedding too). The 25-pass edit scales the four “Wait” variants by 0.8, two “Hmm” variants by 0.9 and</think>by 1.3 (7 rows, 8,192 weights with the norm). The 50-pass edit scales the four “Wait” rows by 0.7 and</think>by 1.25 (5 rows, 6,144 weights). The 10-pass run could not save an embedding edit because of the harness pipe bug, and reached the same effect another way: it projected 35% of a two-dimensional “Wait” readout subspace out of the attention and MLP
output matrices of layers 20–23, and added 0.3 times the </think> readout component in layer 23’s MLP output. Those projections touch every element of 8 large matrices, which is why this edit changes 18.5M weights (2.46%) against 6,144–8,192 for the others.
No loop direction. No run removed an internal loop direction as the original did. The 25-pass run considered projecting the hit-versus-finished activation difference out of the attention and MLP outputs and rejected it, because, in its summarised reasoning, “the hit-versus-finished activation difference mixes loop state with ordinary reasoning and problem difficulty”. A check on the returned activations supports a different mechanism: the base model’s looping-versus-finished mean difference is nearly orthogonal to the “Wait” and
</think>readout directions Claude edited ( at layers 15, 19, 21 and 23), and the MLP-output difference is largest in layers 7–14 and still large at 15–19. Claude’s edits change which token is sampled, not the internal state the original removed.
Every final edit was rebuilt on the server and checked: all other tensors are bit-identical to the base.
How much of the gain is temperature. The matched control evaluates the base model with only the final-norm part of each edit (Claude’s own saved norm.weight, 1,024 weights), on all three benchmarks at the full 4/2/2 samples. By the vLLM order of operations above, this is the base model at temperature 0.6/ and presence penalty 1.5/, which vLLM would not accept directly (it caps the penalty at 2). The control is byte-identical to an edit Claude itself tested at practice call 2 of the 10- and 25-pass runs. Table 18 splits each gain into the control’s part and the remainder. Temperature alone lifts MATH-500 to 22.1% () and 20.9% (), about a third of the gain (28–35%), and GSM8K by about half (49–59%). The token edits add +29.3, +33.2 and +37.2 points on MATH-500 and +22.1 to +28.8 on GSM8K, all with intervals excluding 0. On IFEval the token edits add nothing measurable (+3.1, −2.1 and +2.0 points, each interval including 0): the IFEval gain is a temperature effect. By the rule added before the 25- and 50-pass runs (a control reproducing under 50% of the gain), each edit “does more than change temperature” on math.
| Benchmark | Budget | Edit − base | Control − base (temperature) | Edit − control (token edits) | Share |
| MATH-500 | 10 | +44.6 [41.1, 48.2] | +15.4 [12.9, 17.9] | +29.3 [26.1, 32.4] | 0.35 |
| 25 | +48.5 [45.0, 52.1] | +15.4 [12.9, 17.9] | +33.2 [29.9, 36.4] | 0.32 | |
| 50 | +51.4 [47.9, 55.0] | +14.2 [11.9, 16.6] | +37.2 [34.0, 40.4] | 0.28 | |
| GSM8K | 10 | +55.8 [53.5, 58.0] | +31.1 [28.9, 33.2] | +24.7 [22.7, 26.8] | 0.56 |
| 25 | +53.1 [50.9, 55.3] | +32.8 [28.9, 33.2] | +22.1 [20.0, 24.2] | 0.59 | |
| 50 | +56.5 [54.3, 58.8] | +27.7 [25.7, 29.8] | +28.8 [26.7, 31.0] | 0.49 | |
| IFEval | 10 | +26.7 [22.9, 30.5] | +23.6 [19.9, 27.2] | +3.1 [−0.2, 6.5] | 0.88 |
| 25 | +21.4 [17.6, 25.3] | +23.6 [19.9, 27.2] | −2.1 [−5.6, 1.3] | 1.10 | |
| 50 | +32.1 [28.5, 35.7] | +30.0 [26.5, 33.6] | +2.0 [−1.5, 5.6] | 0.94 |
Table 18. Experiment 8: the gain of each final edit over the base, split into the part reproduced by its norm-only temperature control and the part added by the token-level changes. Points, with 95% paired bootstrap intervals; MATH-500 , GSM8K 1,319 , IFEval 541 . Control: final norm for 10 and 25 passes, for 50. Share: (control − base) / (edit − base).
Budget dependence. The runs used 10 of 10 passes (8 calls, 640 problems), 23 of 25 (14 calls, 1,472 problems) and 44 of 50 (20 calls, 2,816 problems). More budget helped a little on MATH-500: 50 passes against 10 gives +6.8 points [4.9, 8.7], and against 25 gives +2.9 [0.9, 4.9]. Both intervals exclude 0, so the pre-registered rule says “scaling replicates”. The shape does not reproduce, though. In the original, 10 and 25 passes barely helped (12.3% and 14.1%) and 50 passes made the difference (48.1%); here 10 passes already gave 51.4%, above the original’s 50-pass score. On the other benchmarks the order is not monotone: from 10 to 25 passes GSM8K fell by 2.6 points [−4.3, −1.0] and IFEval by 5.3 [−8.5, −1.9], and 50 passes against 10 changes GSM8K by +0.8 [−0.9, +2.4]. With one run per budget these are statements about these runs, not about budgets in general. On practice problems, the final edits scored 60.9% against the base’s 6.3% (10 passes, 128 problems), 60.5% against 10.2% (25 passes, 256 problems) and 58.3% against 10.4% (50 passes, 192 problems).
Gemma control. The Gemma run used 24 of 25 passes in 8 calls. Its final edit is a pure temperature change: the final norm’s gain times 0.86 (640 weights), equivalent to temperature 1.16. On MATH-500 () it moved the score from 9.2% to 8.3%, a change of −0.8 points [−1.9, +0.2], and the share at the 8,192-token limit from 4.6% to 2.0%. The base model sampled at the matched temperature 1.1628 scores 8.65%, the same as the edit (edit minus control −0.35 points [−1.35, +0.60]). With the original’s lenient grading the scores are 8.9% and 8.2%, against the original’s 6.8% base. By the pre-registered rule the control replicates: on a model that rarely loops, Claude found no edit that helps, as Astra did not. Claude’s own summary said its accuracy gain “is about zero, within noise”.
Audit, cost and compute. The audit read every flagged cell in the four runs (5 flags in 113 code cells) and found no violation. Three readouts were borderline and allowed by the pre-registered definitions: in each Qwen run Claude multiplied a mean residual vector returned by the practice tool by the final norm gain and the embedding matrix (a logit-lens readout), a product of weights with a returned vector. No run pushed an input through a layer, used random vectors or fitted weights. The API cost of Experiment 8 was $8.48: $1.80, $2.81 and $3.08 for the 10-, 25- and 50-pass runs, $0.52 for Gemma, $0.23 for the abandoned pilot and $0.03 for a test call that checked the 1-hour cache setting. GPU time was dominated by the 32k-token evaluations on the single H200: 6.8 h for the base model (166M generated tokens at 6.8k tokens/s), 2.3–2.6 h for each edit, and 4.8 h and about 4.6 h for the two norm-only controls, which still run to the limit; Gemma evaluations took about 2 minutes each. The Claude runs took 1.8 h (10 passes), 12.5 h (25) and 11.5 h (50) of wall-clock time including the VPN outages, and 47 minutes for Gemma. One practice call of 64 problems took 8–10 minutes.
Comparison with the original
Table 19 sets the two studies side by side. It should be read with care. The two studies use different networks and mostly different environments, my attempts are repeated where theirs often were not, and Astra’s figures come from the original post rather than from runs I could inspect. The last column says how each pair of setups differs. I mark no row as a win for either model; where the setups are close enough to compare, the text says so.
| Exp. | Probe | Astra | Opus 5.5 | How the setups differ |
| 1 | Name the game of a PPO policy, no candidate list | 4/4 games (1 attempt each) | 3/4 games; 14/20 attempts | MinAtar spatial channels vs 128 RAM bytes. Opus misses Ms. Pac-Man (0/5) and relies on recalled RAM maps. |
| 1b | Name the classic-control task (padded) | – | 7/20 (v1); 6/20 (v2) | No counterpart. Answers follow the padded shape (LunarLander 26/28, MPE 21/28). |
| 2 | Name the shape a DDPM generates | 3/3 shapes | 2/3 shapes; 9/15 attempts | 28,000 vs 23,824 weights; mine predict . Circles 1/5 graded, 0/5 in substance. Astra drew outlines; Opus drew mostly filled shapes. |
| 3 | Training language from middle blocks | 6/6 | 5/15 | Similar setup (4 blocks × 6 heads, same three text types). Opus answered English in 15/15. |
| 3 | Find the copying heads | Exact in 3/3 models | Recall 15/15; exact on romaji 5/5; F1 0.70 | Ground truths differ (Astra’s Python model had 3 copying heads, mine 1). Opus’s ranking was perfect in every run. |
| 7 | Sycophantic Gemma 3 270M, weights only | 0/4 | 6/6 runs; 3/3 pairs | Opus had the shared frozen embedding and vocabulary; the original does not say whether Astra had them. |
| 7 | Sycophant, with attention tool | 3/3 pairs | 3/3 pairs; 6/6 runs | Named heads (12.1, 11.2) rank 1st–2nd of 72; pair 1’s top head is 10.2, the head Astra named. |
| 4 | Fix a policy stuck in a local optimum, one shot | 100%; 3.5% of weights | 100% best, 99.9–100% in 5/5; 1.5% of weights | Kinetix ramp (171,664 weights) vs MountainCar (4,547). Opus grafted a velocity-sign controller. |
| 4 | Fix a hovering LunarLander | – | 85.6% best; 0/5 reach 90% | No counterpart. Gymnasium’s heuristic scores 80.3%; every edit rebuilds it from memory. |
| 5 | Zero-shot policy beats briefly trained PPO | 3/3 vs PPO-100M | 4/5 vs PPO-100k | MinAtar games vs Gymnasium and ALE; PPO budgets 100M vs 100k steps. Opus best ≥ its final PPO on 3/5. |
| 5 | Zero-shot Breakout vs longest-trained PPO | 20× (517.4 vs 25.3) | 1.9× (51.2 vs 27.5) | Not comparable: MinAtar vs ALE Breakout from RAM; PPO 500M vs 10M steps. Opus recalled the RAM addresses. |
| 5b | Policy with 25 rollout batches | – | LunarLander 277.4; Breakout 184.7 / 92.1 | No counterpart. The original’s budget sweep was on Go. One run per cell. |
| 6 | Go network, 0 passes, vs two small PPO nets | >50% vs both | 93.8%, 87.5% | Different engines (mine: komi 7.0, 200-ply cap). Small opponents differ in training. |
| 6 | Go network, 50 passes, vs equal-sized PPO | Beats it; 55.2% of practice games won | 87.1% (final eval) | Equal-sized PPO trained on 20M games in the original and 0.71M here; a heuristic bot wins 42% against mine. |
| 8 | Improve Qwen3.5-0.8B at math, 10 / 25 / 50 passes | 7.9% → 12.3 / 14.1 / 48.1% | 6.7% → 51.4 / 55.2 / 58.1% () | Same model, benchmarks, sampling and sample counts; seeds differ. Astra removed a layer-15/19 direction (131,072 weights); Opus scaled the final norm (temperature alone: 20.9–22.1%) and the “Wait” and </think> logits (6,144–8,192 weights at 25–50 passes). Opus had activation summaries. |
| 8 | Gemma 3 270M control, 25 passes | 6.8% → 7.7% | 9.2% → 8.3% (−0.8, [−1.9, +0.2]) | Same model and sampling; Astra’s numbers use lenient grading (Opus: 8.9% → 8.2%). Opus’s edit is a pure temperature change. |
Table 19. Results of Takehi et al. [35] for GPT-6 Astra and of this replication for Claude Opus 5.5, per probe. A dash means the probe has no counterpart in that study.
Where the setups are closest (Experiments 2, 3 and 7), Claude Opus 5.5 is worse than Astra at saying what data a network was trained on (training language 5/15 against 6/6; shapes 2/3 against 3/3) and better on the sycophancy probe from weights alone (6/6 against 0/4), with the caveat that it had the embedding and vocabulary and that its attempts on a pair are correlated. Copying-head recall is equal. Experiment 8 is the one probe run on the same model with the same benchmarks and sampling, so its numbers are directly comparable, up to sampling seeds and one run per budget. There Claude Opus 5.5 reached the original’s 50-pass result with 10 passes and
exceeded it at 50 (58.1% against 48.1% on MATH-500), and both studies found no gain on Gemma. The mechanisms differ, and part of Claude’s gain is a temperature change that the original’s edit does not contain (§ 8.5); Claude also had activation summaries that Astra may not have had. The reinforcement-learning rows use different environments, so they show only that the same capabilities appeared in both studies: both models fixed a policy stuck in a local optimum, both wrote zero-shot policies that beat briefly trained PPO, and both wrote Go networks that beat small PPO networks with no forward passes. The Go comparison against the equal-sized network is not informative here because my opponent is much weaker. Differences in attempt counts also matter: a 4/4 from one attempt per network and a 3/4 from five attempts per network are not the same kind of evidence.
Discussion
The pattern across the first nine experiments is consistent enough to state in three parts. Experiment 8 is discussed separately below.
Where it works: structure Claude can recognise or construct. Claude found circuits from weight products alone: the copying head in every character-LM run, with a perfect ranking against measured scores (§ 5.4), and late-layer label-copying heads that separated sycophant from honest model in every weights-only run (§ 5.5). In Experiment 3 it reconstructed QK/OV analysis and K-composition ([9]) in 15 of 15 runs without being told about them, including a token/position split of the residual stream with no embedding available. Where it said which inputs a network depends on, the claim held up when checked: the RAM rows it singled out in Experiment 1 match their offline L2 norms, and the Breakout bytes it used in Experiment 5 correlate at with the screen. It also detected every untrained network (10/10), often by recognising the exact initialisation scale. It compiled known controllers into weights correctly without testing them, including an if/else heuristic expressed as an argmax over tanh units (Experiments 4 and 5), and wrote a Go network of 1.48M weights as a layer-by-layer program that wins most games against small PPO opponents with no forward passes (Experiment 6).
Where it fails: inferring the training data. When the question is about the training distribution, Claude falls back on priors. Three priors recur.
Interface shape. LunarLander in 26 of 28 v1 runs, because the padded shape was LunarLander’s; then MPE
simple_adversaryin 21 of 28 v2 runs, because the new shape belonged to it (§ 5.2). The same prior appears in Experiment 1 as an invalid inference from the 18-action set.Dataset defaults. Handwritten digits for small grayscale DDPMs in 11 of 21 runs, and in every circle run (§ 5.3). “English” for every character LM, 15 of 15 (§ 5.4).
Famous games. Breakout as the fallback answer for Ms. Pac-Man in 3 of 7 runs (§ 5.1).
The evidence it needed was often in its own analysis. It measured a code-sized vocabulary for the Python model and saw the duplicated-logit signature of CartPole’s padded action map, then explained both away. When it did use the decisive cue it was right: 7 of 7 v2 runs that noticed the padding. Confidence was sometimes honest about this (0.3–0.4 on the language guesses, at most 0.30 on Ms. Pac-Man, 0.35 on the MPE guesses) and sometimes not (about 0.82 on the wrong LunarLander answers in v1, and flat at 0.35–0.6 for DDPM answers regardless of correctness).
Memorisation in fixing and writing. None of the ten repairs in Experiment 4 repaired the learned circuit. Each grafted a textbook controller into repurposed units; nine of the ten switched the trained units off, and the tenth overrides them by a margin of 100 in the logits. Every zero-shot network in Experiment 5 is a known controller: LQR for CartPole, velocity-sign pumping for MountainCar, a Spong-style rule for Acrobot, Gymnasium’s heuristic() for LunarLander and an AtariARI-based tracker for Breakout. The pre-registered memorisation flag fired on both environments where it was defined, MountainCar and LunarLander. Where no good closed form exists (minimum-time Acrobot swing-up), the naive rule lost to PPO-100k. A rollout budget helps on exactly this point (§ 7.2). With feedback, LunarLander moved off the memorised heuristic (agreement 0.90 to 0.67) and beat it, and Breakout was played from RAM semantics that Claude inferred from traces rather than recalled. The Go networks are the least memorised artefact in the study. Their move terms (capture, escape, self-atari, eye-filling, a line prior) are standard Go heuristics, but their expression in weights (score planes carried with an offset, exact-threshold liberty units, chain propagation one step per layer) was constructed in code over several cells.
Editing a language model with feedback. Experiment 8 is the one probe where Claude Opus 5.5 worked on the original’s own subject under the original’s protocol, and it reached the original’s endpoint with less feedback: 51.4% on MATH-500 after 10 forward passes (, +44.6 points [41.1, 48.2]) against Astra’s 48.1% after 50, and 58.1% after 50. It got there by a different route. Astra removed one direction written by two middle-layer MLPs, an edit on the model’s internal state. Claude edited the output side in every run: it lowered all logits through the final norm and lowered “Wait” and raised </think> through their output rows (or, with 10 passes, through the readout of layers 20–23). It considered the internal route and rejected it on the grounds that the loop direction it could measure was confounded with ordinary reasoning, and the loop difference in the returned activations is indeed nearly orthogonal to the readouts it edited (). Both edits remove the same failure, an obstacle rather than missing ability: finished base answers were already 100% correct on MATH-500, and on Gemma, which rarely loops, neither model found an edit that helps.
Part of Claude’s gain is temperature. Scaling the final norm by is exactly sampling the base model at temperature with a stronger presence penalty, and that change alone gives 20.9–22.1% on MATH-500, about a third of the gain, and nearly all of the IFEval gain (Table 18). The token-level edits add 29–37 points on MATH-500 and 22–29 on GSM8K beyond it. Whether a weight edit that is equivalent to a sampling change counts as weight-space programming is a matter of definition; I report the two parts separately so that either reading can be applied. The original’s edit has no temperature component, and its sampling was fixed at the same 0.6, so the original also shows that this model’s score at the published setting is mostly a question of whether it stops. A fair summary of both studies is that a frontier model given a few hundred to a few thousand practice generations can find a small edit that makes a looping 0.8B model stop, and that this lifts every benchmark the loops were suppressing.
What the controls showed. The untrained-network control was informative: Claude never named a task for one, which rules out the simplest prior-driven failure. The permutation control was close to trivial. Permuting hidden units leaves each input row’s and each output column’s set of values unchanged, and Claude reads networks row by row and column by column, so the permuted copies mostly measure sampling noise (6/8 against 14/20 in Experiment 1, with the same per-game pattern). A stronger control for a row-and-column reader would change the input or output semantics, for example by permuting input bytes and telling the reader, or by retraining with a different seed.
Process failures. A few process failures cost points: one Acrobot attempt ran out of tokens before answering, the Breakout budget-5 run submitted untested weights, and repeated attempts were often near-identical (four of five MountainCar edits score identically, two LunarLander networks implement the same function). Five attempts at effort high are therefore fewer independent samples than they look.
Relation to the original. None of this contradicts the direction of the original post. It narrows it. Reading structure and compiling known programs into weights work, and in several probes Claude’s confidence tracked its correctness. Inferring what data a network was trained on mostly does not work yet for this model, and several of the original’s headline successes in that category did not replicate here (training language, the circle DDPM, the fourth Atari game). The one probe the original reported as a failure, sycophancy from weights alone, succeeded here under a setup that may have been easier. The original’s largest claim, a large math gain from a small edit to Qwen3.5-0.8B, replicated by its pre-registered rule, but with a different mechanism, a temperature component, and no sign of the original’s budget shape.
Limitations
Sample sizes. Most cells have 5 attempts, permuted copies 2, and Experiments 5b and 6 one run per cell (two for Breakout at budget 25). Each Atari game and each classic-control task was trained with one seed. Experiment 7 has three pairs, and attempts on a pair are correlated (both attempts named the same head in every pair and phase), so the pair-level evidence is 3/3 (), not 6/6. Several attempts were near-identical (§ 10), which lowers the effective sample size further.
Design errors and leaks. The v1 padding in Experiment 1b reproduced LunarLander’s interface, which was my design error; the v2 shape matched another benchmark. Prompts necessarily describe the interface, and the interface carries information: the 18-action ALE set says “Atari”, the DDPM prompt says the network predicts , and the Gemma prompt names the base model. In Experiment 7 I provided the shared embedding and vocabulary, which opened the OV-copying route and may have made the weights-only phase easier than the original’s.
Grading rules. The Experiment 2 drawing judge has no “other” class and is too lenient. The “earliest alias” text rule credited two descriptions that answered “handwritten digits” through the alias “ring”. The Experiment 7 location test measures attention, so it cannot verify the weights-only OV claims, which I did not test with ablations. The Experiment 5b regex audit flagged nothing; the breach and grey area were found by reading. These rules were kept as pre-registered and both readings are reported.
Weak baselines and opponents. My equal-sized Go opponent trained on 0.71M games against the original’s 20M, and a simple heuristic bot wins 42% against it. Two PPO checkpoints in Experiment 5 are poor (CartPole at 1M, LunarLander at 100k), and unshaped PPO never solves MountainCar, which makes “beats PPO” an easy bar on those environments. The honest Gemma models are at chance on multiple-choice items.
Memorised interfaces. Breakout and the Atari identifications rely on RAM annotations Claude recalled from AtariARI and OCAtari; the LunarLander edits and zero-shot networks rely on recalling Gymnasium’s heuristic(). These results say little about unannotated games or tasks without a published controller.
Different environments. Moving from MinAtar and Kinetix to Gymnasium and the ALE was deliberate, but it means the reinforcement-learning rows of Table 19 do not compare the two models on the same task. My DDPMs predict ; my character LMs and copying-head ground truths differ from the original’s.
One model, one effort level, one context. All runs used Claude Opus 5.5 at effort high. I did not test other effort levels, other models, or the sensitivity of results to prompt wording. A 64k-token output limit was not enough for one Acrobot attempt. Thinking is returned as a summary, so the quoted reasoning is a summary of what the model did, not a transcript of it.
Sandbox and audits. The sandbox is a macOS Seatbelt profile plus an import hook, not a container, and its first version left the Hugging Face cache readable (no transcript accessed it). Concurrent Experiment 7 runs could list, but not usefully read, each other’s randomly named weight directories. Audits were done by one person (me), with pattern flags and hand reading, and borderline cases were resolved by pre-registered definitions that another reader might draw differently.
Pre-registration scope. Pre-registration fixed grading, not design, and it was not deposited with a third party or timestamped externally. Readers have to trust that the GRADING.md files in the repository predate the runs; the pilot records inside them, and the run timestamps in result.json, are the only internal evidence of ordering.
Experiment 8. The language-model results carry their own caveats (§ 8).
One run per budget. Each budget has one Claude run, as in the original, so the budget comparison (+6.8 points [4.9, 8.7] from 10 to 50 passes on MATH-500) is a comparison of three runs. Its interval covers sampling noise in the evaluation, not the spread of runs.
The final save is not the best practice edit. The last saved weights are scored, as the prompt states. In two runs another edit had a higher single practice score: 63.3% for the 25-pass run (call 11, 128 problems) and 65.6% for the 50-pass run (call 18, 128 problems), against 60.5% (256 problems) and 58.3% (192 problems) for the final edits on different problem sets. Practice standard errors were 3.1–4.4 points, so these gaps are within noise, and the held-out scores of the non-final edits are unknown.
Temperature confound. Every Qwen edit includes a final-norm scale equivalent to temperature 0.88–1.0 with presence penalty 2.2–2.5. The norm-only control measures that part exactly, but the token-level part was only evaluated on top of it, so I cannot say what the token edits alone would do at temperature 0.6.
Activation summaries. The practice tool returned per-layer mean activations, which the original does not describe. Claude used them for logit-lens readouts in all three Qwen runs. This is more information than generations alone, and it may have made the task easier than the original’s.
Matched protocol, different seeds. Sampling, maximum length and sample counts (4 on MATH-500, 2 on GSM8K and IFEval) follow the original’s released code, but my per-request seeds differ from theirs, so base scores differ by sampling noise (6.7% against 7.9% on MATH-500).
Protocol changes. The temperature was changed from 1.0 to 0.6 after a first baseline and before any API call; the practice budget was redefined in 64-problem units after the first pilot, which was abandoned; and the 10-pass run could not save an embedding edit because of a harness bug (§ 8.4). The temperature analysis was added after the pilot and before the 25- and 50-pass runs.
Infrastructure. The SLURM account allowed one GPU, so all practice calls and evaluations were serialised on one H200, and about 13 hours of VPN outages and tunnel-reconnection faults stalled the runs without losing or double-charging any call.
Conclusion
I re-ran the eight probes of Takehi et al. [35] with Claude Opus 5.5, seven on networks and environments of my own and one on the original’s Qwen3.5-0.8B setup, with pre-registered grading, controls and repeated attempts, for $83.38 in API calls. Claude read structure from weights: copying heads in 15/15 runs, the RAM bytes a policy listens to, the sycophantic finetune in 6/6 weights-only runs, and every untrained network. It wrote and edited working networks, but these were known controllers compiled into a few units, and with a rollout budget it moved beyond them. It did not infer training data reliably: its answers followed the interface shape, dataset defaults and famous names, even when its own analysis held the right evidence. Given Qwen3.5-0.8B’s weights and 10 to 50 forward passes of practice, it raised MATH-500 from 6.7% to 51.4–58.1% (), which replicates the original’s largest claim by its pre-registered rule. It did so by editing the output side of the model rather than the original’s internal direction, and about a third of that math gain, and nearly all of the IFEval gain, is a change of effective sampling temperature. On a model that rarely loops it found nothing to fix. The original’s direction holds, with a narrower scope than its headline suggests, under these conditions and for this model.
Reproducibility statement
Code, logs and the published write-up are available.
Every run writes a folder logs/<exp_id>/<run_id>/ with the exact prompt (prompt.md), every API message, tool call and tool result (transcript.jsonl), the summarised thinking (thinking.md), the final answer (response.md), and result.json with the model, condition, target, answer, correctness, confidence, score, token counts, cost and timestamp. Weights Claude wrote or edited are saved next to them, together with the evaluated network. Checkpoints are plain PyTorch state_dicts with a JSON card (environment, architecture, training steps, seed, evaluation return) in checkpoints/<exp_id>/; the Gemma finetunes (3.2 GB) are too large for the repository. Pre-registered rules are in experiments/<id>/GRADING.md, and each experiment’s runner supports a dry run, a pilot, the full run and re-grading from logs without API calls. The tables and charts are generated from the logs by scripts/build_blog_data.py; the numbers in this paper were copied from logs/<id>/summary.json and the per-experiment notes. The text of the original post is not redistributed; it is cited.
Experiment 8. The generation side ran on a lab server with two NVIDIA H200 NVL GPUs (143,771 MiB each; driver 595.91.07, CUDA 13.2; Ubuntu 24.04.5), of which SLURM allowed this account one at a time. The software was vLLM 0.33.0, PyTorch 2.14.0 (CUDA 13.0), Transformers 5.20.2 and Python 3.12.3. A FastAPI service in experiments/exp8/server/ (service.py) runs as one SLURM job and holds a base vLLM engine and an edit engine whose weights are swapped in place (vllm_ext.py); it serves /practice, /edit, /result and, in eval mode only, /eval, and it is started with service_ctl.sh start –practice-only during Claude runs. The Mac runner (experiments/exp8/run.py) holds the Claude loop and the sandbox, reaches the service through an SSH tunnel kept by keep_tunnel.sh, writes state.json after every turn and resumes with –resume. Final evaluations are standalone eval.sbatch jobs with fixed per-request seeds. Every result file records the sampling settings, versions and timing; experiments/exp8/analyze.py computes the paired statistics, the edit decomposition and the audit flags, and writes logs/exp8/stats.json and summary.json, from which the Experiment 8 numbers here were copied. The edited checkpoints are stored as deltas against the base and can be rebuilt with edits.py build.
Acknowledgements. This work exists because Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab published a clear and detailed account of their experiments, with their logs. Where this replication finds differences, they should be read as a contribution to the same question, not as a rebuttal. I thank the maintainers of Gymnasium, the Arcade Learning Environment, PettingZoo and CleanRL, whose work made these testbeds quick to build and reproduce.
Prompts
The prompts below are copied verbatim from logs/<exp>/<run>/prompt.md. Weight lines and episode-trace rows are replaced by a bracketed count of the omitted lines; nothing else is changed. The system prompt for Experiments 1, 1b, 2, 3 and 7 was “You are an expert in machine learning and neural networks. Answer carefully and honestly.”
Experiment 1 (Atari game, main condition, Breakout attempt 1)
Below are the complete weights of a small neural network, printed as text.
This is a policy network trained with reinforcement learning (PPO) to play an Atari 2600 game.
Observation: the 128 bytes of the console's RAM, each divided by 255, so every input lies in [0, 1]. Input
↪ unit i is RAM byte i (console address 0x80 + i).
Actions: the full 18-action Arcade Learning Environment (ALE) action set, in the standard ALE order:
0 NOOP
1 FIRE
2 UP
3 RIGHT
4 LEFT
5 DOWN
6 UPRIGHT
7 UPLEFT
8 DOWNRIGHT
9 DOWNLEFT
10 UPFIRE
11 RIGHTFIRE
12 LEFTFIRE
13 DOWNFIRE
14 UPRIGHTFIRE
15 UPLEFTFIRE
16 DOWNRIGHTFIRE
17 DOWNLEFTFIRE
Architecture: 128 inputs, then two hidden layers of 64 units each with tanh activations, then an output
↪ layer of 18 action logits. The action taken is the argmax of the 18 logits.
Format: "W<k> row i" lists the weights from input unit i of layer k to every unit of the next layer (the
↪ matrix is printed as [in, out]); "b<k>" is the bias vector of layer k. Values are printed as integers:
↪ the actual value is the integer times 0.001.
Which game was this network trained on?
No tools are allowed: answer by reading the weights below, without running any code.
It is also possible that the network was never trained; if you believe these are untrained / random weights,
↪ say so and answer "untrained / random weights".
Please give:
1. the game name,
2. your confidence (0 to 1) that your answer is correct,
3. the reasoning from the weights that led you there.
<weights>
# MLP 128 -> 64 -> 64 -> 18, tanh activations, 13586 weights
# W<k> row i = weights from input unit i of layer k to every output unit; b<k> = bias of layer k
# quantized: value = integer * 0.001
[... 259 weight lines omitted ...]
</weights>
End your response with exactly one final line of this form (machine-readable):
ANSWER: {"game": "<name>", "confidence": <number between 0 and 1>}
Experiment 3 (character LM, Python attempt 1)
You are given part of a small trained neural network: the middle of a character-level language model.
The full model was a decoder-only transformer trained to predict the next character of text. All of its
↪ training text was one kind of text. Architecture:
- 4 transformer blocks (layers 0-3), each with 6 attention heads (heads 0-5); d_model = 384, d_head = 64,
↪ MLP hidden width 1536 with GELU, context length 256.
- Pre-LayerNorm blocks: x = x + attn.o(Attn(ln1(x))); then x = x + mlp.proj(gelu(mlp.fc(ln2(x)))).
Attention is causal softmax(q k^T / sqrt(64)). q, k, v and o are separate linear layers (384 → 384) with ↪ bias. Head h uses rows 64h:64h+64 of the q, k and v weights and biases, and columns 64h:64h+64 of the o ↪ weight.
All weights follow the torch nn.Linear convention: weight has shape [out, in], and y = x @ weight.T + bias.
The input layers (the character embedding and the learned positional embedding) and the output layers (the ↪ final LayerNorm and the unembedding to characters) were removed. You have only the 4 blocks: 64 tensors, ↪ about 7.1M parameters, named like blocks.0.attn.q.weight, blocks.0.ln1.weight and blocks.2.mlp.fc. ↪ bias. Nothing you have says which character is which.
You have a python tool with these weights preloaded as numpy arrays in the dict weights. numpy (np) ↪ and torch (CPU) are available. There is no internet and no file access. You can make up to 40 tool calls; ↪ after that you will be asked for your final answer.
Rule: you may analyse the weights in any way you like, but you may not run the network, or any layer or ↪ component of it (attention, LayerNorm, MLP, a whole block), on inputs. This includes random, synthetic or ↪ hand-constructed input vectors. No forward passes of any kind. Computations on the weight arrays ↪ themselves are allowed.
Questions:
Which language or kind of text was this model trained on? Give your best specific answer and your ↪ confidence (0 to 1) that it is correct.
For each of the 24 attention heads (written layer.head, e.g. 2.4), give a short label for its role. Say ↪ explicitly which heads are copying (induction) heads, i.e. heads that find an earlier occurrence of the ↪ current context and copy what came after it. Also say which heads are previous-token heads, i.e. heads ↪ that mainly attend to the immediately preceding position. Either list may be empty if you find no such ↪ heads.
Explain the evidence from the weights for your answers.
End your response with exactly one final line of this form (machine-readable): ANSWER: {"language": "<name>", "confidence": <number between 0 and 1>, "induction_heads": ["L.H", ...], " ↪ previous_token_heads": ["L.H", ...]}
Experiment 4 (fix a policy, MountainCar attempt 1)
The environment section is the Gymnasium 1.4.0 docstring and source, trimmed by the script build_env_docs.py in experiments/exp4/.
I have a small neural-network control policy that fails at its task. I want you to (1) explain why it fails
↪ and (2) fix it by editing its weights directly.
You have no tools: you cannot run code, run the network, or run the environment, and you cannot test your
↪ edit. Work it out by reasoning, and give your answer in this one reply. The edited network will then be
↪ evaluated on many episodes.
# Environment: MountainCar-v0 (Gymnasium 1.4.0)
Created with `gymnasium.make("MountainCar-v0")`: the episode is truncated after **200 steps** (TimeLimit
↪ wrapper).
## Documentation (from the class docstring)
## Description
The Mountain Car MDP is a deterministic MDP that consists of a car placed stochastically
at the bottom of a sinusoidal valley, with the only possible actions being the accelerations
that can be applied to the car in either direction. The goal of the MDP is to strategically
accelerate the car to reach the goal state on top of the right hill. There are two versions
of the mountain car domain in gymnasium: one with discrete actions and one with continuous.
This version is the one with discrete actions.
This MDP first appeared in {citep}`[Andrew Moore's PhD Thesis (1990)]`(https://www.cl.cam.ac.uk/techreports/UCAM-CL-
↪ TR-209.pdf)
````text
@TECHREPORT{Moore90efficientmemory-based,
author = {Andrew William Moore},
title = {Efficient Memory-based Learning for Robot Control},
institution = {University of Cambridge},
year = {1990}
}
Observation Space
The observation is a ndarray with shape (2,) where the elements correspond to the following:
| Num | Observation | Min | Max | Unit |
| 0 | position of the car along the x-axis | -1.2 | 0.6 | position (m) |
| 1 | velocity of the car | -0.07 | 0.07 | velocity (v) |
Action Space
There are 3 discrete deterministic actions:
0: Accelerate to the left
1: Don't accelerate
2: Accelerate to the right
Transition Dynamics:
Given an action, the mountain car follows the following transition dynamics:
velocity<sub>t+1</sub> = velocity<sub>t</sub> + (action - 1) * force - cos(3 * position<sub>t</sub>) * ↪ gravity
position<sub>t+1</sub> = position<sub>t</sub> + velocity<sub>t+1</sub>
where force = 0.001 and gravity = 0.0025. The collisions at either end are inelastic with the velocity set ↪ to 0 upon collision with the wall. The position is clipped to the range [-1.2, 0.6] and velocity is clipped to the range [-0.07, 0.07].
Reward:
The goal is to reach the flag placed on top of the right hill as quickly as possible, as such the agent is penalised with a reward of -1 for each timestep.
Starting State
The position of the car is assigned a uniform random value in [-0.6 , -0.4]. The starting velocity of the car is always assigned to 0.
Episode End
The episode ends if either of the following happens:
Termination: The position of the car is greater than or equal to 0.5 (the goal position on top of the ↪ right hill)
Truncation: The length of the episode is 200.
Source (gymnasium/envs/classic_control/mountain_car.py, rendering removed)
def __init__(self, render_mode: str | None = None, goal_velocity=0):
self.min_position = -1.2
self.max_position = 0.6
self.max_speed = 0.07
self.goal_position = 0.5
self.goal_velocity = goal_velocity
self.force = 0.001
````
self.gravity = 0.0025
self.low = np.array([self.min_position, -self.max_speed], dtype=np.float32)
self.high = np.array([self.max_position, self.max_speed], dtype=np.float32)
self.action_space = spaces.Discrete(3)
self.observation_space = spaces.Box(self.low, self.high, dtype=np.float32)
def step(self, action: int):
assert self.action_space.contains(action), (
f"{action!r} ({type(action)}) invalid"
)
position, velocity = self.state
velocity += (action - 1) * self.force + math.cos(3 * position) * (-self.gravity)
velocity = np.clip(velocity, -self.max_speed, self.max_speed)
position += velocity
position = np.clip(position, self.min_position, self.max_position)
if position == self.min_position and velocity < 0:
velocity = 0
terminated = bool(
position >= self.goal_position and velocity >= self.goal_velocity
)
reward = -1.0
self.state = (position, velocity)
# truncation=False as the time limit is handled by the `TimeLimit` wrapper added during `make`
return np.array(self.state, dtype=np.float32), reward, terminated, False, {}
def reset(
self,
*,
seed: int | None = None,
options: dict | None = None,
):
super().reset(seed=seed)
# Note that if you use custom reset bounds, it may lead to out-of-bound
# state/observations.
low, high = utils.maybe_parse_reset_bounds(options, -0.6, -0.4)
self.state = np.array([self.np_random.uniform(low=low, high=high), 0])
return np.array(self.state, dtype=np.float32), {}
The policy network
Architecture: MLP 2 → 64 → 64 → 3 with tanh hidden activations (4547 weights). The architecture is fixed.
Input: the raw observation as float32, no scaling or normalisation (0 = position, 1 = velocity).
Forward pass: h1 = tanh(x @ W1 + b1); h2 = tanh(h1 @ W2 + b2); logits = h2 @ W3 + b3.
Action: greedy, action = argmax(logits) (ties go to the lowest index). Actions: 0 = accelerate left, 1 = don’t accelerate, 2 = accelerate right.
Weights are printed below with 3 decimals.
W<k> row ilists the weights from input unit i of layer k to every output unit j = 0, 1, ...; that is, row i of W<k> in the [in, out] orientation used in the forward pass above, so W<k>[i, j] connects input unit i to output unit j.b<k>is the bias of layer k.
Full weights
[... 133 weight lines omitted ...]
One recorded episode of the current policy
Env seed 0. Every 5th step is shown (plus the last step). Each row is the observation the network saw at step t, the greedy action it chose, and the reward received for that step.
t position velocity action reward
[... 41 trace rows omitted ...]
↪
Episode result: 200 steps, total return -200.0, ended by time limit (truncation). Final observation:
↪ position=-0.3240 velocity=-0.0050.
Action counts over all 200 steps: action 2: 200.
# Your task
1. Explain why this network fails. Point to the specific weights, units and parts of the trace that show it.
2. Change the weights so that the greedy policy succeeds as often as possible. Success means: the car
↪ reaches the flag (position >= 0.5) before the 200-step time limit. It will be evaluated over many
↪ episodes with random starting conditions as described in the environment.
3. Give your change as edits to this network. Any weight you don't mention keeps its current value. You may
↪ change as many or as few weights as you like, and you may repurpose hidden units, but the architecture (2
↪ -> 64 -> 64 -> 3, tanh) and the input/output meaning are fixed.
# Answer format
- First a line `DIAGNOSIS: <why it fails, at most 300 characters>`.
- Then your explanation of the failure and of how your edit works.
- End your reply with exactly one machine-readable block of edits, delimited like this:
EDITS:
W1[0,5] = 12.0
b1[5] = -3.5
W2 row 7 <64 numbers>
b3[2] = -1.0
END_EDITS
Rules for the EDITS block: one edit per line, no comments or prose inside the block; indices are 0-based; `W
↪ <k>[i,j] = v` sets the weight from input unit i to output unit j of layer k (same orientation as the
↪ printed rows); `b<k>[j] = v` sets one bias; `W<k> row i <values>` replaces a whole row (it must have
↪ exactly as many values as the printed row); `b<k> <values>` replaces a whole bias vector. Only the last
↪ EDITS block in your reply is used.
Experiment 5 (write zero-shot, CartPole attempt 1)
Write, from scratch, the weights of a neural-network policy that plays CartPole-v1 as well as possible.
There is no training: you choose every number yourself. You have no tools: you cannot run code, run the
↪ network or run the environment, and you get no data, losses, gradients or test episodes. Work it out by
↪ reasoning and give the complete answer in this one reply. The network you write will then be evaluated,
↪ unchanged, on many episodes.
# Environment: CartPole-v1 (Gymnasium 1.4.0)
Created with `gymnasium.make("CartPole-v1")` (sutton_barto_reward=False): the episode is truncated after
↪ **500 steps** (TimeLimit wrapper), so the maximum return is 500.
## Documentation (from the class docstring)
## Description
This environment corresponds to the version of the cart-pole problem described by Barto, Sutton, and
↪ Anderson in
["Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problem"](https://ieeexplore.ieee.
↪ org/document/6313077).
A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track.
The pendulum is placed upright on the cart and the goal is to balance the pole by applying forces
in the left and right direction on the cart.
## Action Space
The action is a ndarray with shape (1,) which can take values {0, 1} indicating the direction of the fixed force the cart is pushed with.
0: Push cart to the left
1: Push cart to the right
Note: The velocity that is reduced or increased by the applied force is not fixed and it depends on the angle the pole is pointing. The center of gravity of the pole varies the amount of energy needed to move the cart underneath it.
Observation Space
The observation is a ndarray with shape (4,) with the values corresponding to the following positions and velocities:
| Num | Observation | Min | Max |
| 0 | Cart Position | -4.8 | 4.8 |
| 1 | Cart Velocity | -Inf | Inf |
| 2 | Pole Angle | ~ -0.418 rad (-24 deg) | ~ 0.418 rad (24 deg) |
| 3 | Pole Angular Velocity | -Inf | Inf |
Table 1 (PDF p. 51).
Note: While the ranges above denote the possible values for observation space of each element, it is not reflective of the allowed values of the state space in an unterminated episode. Particularly:
The cart x-position (index 0) can take values between
(-4.8, 4.8), but the episode terminates if the cart leaves the(-2.4, 2.4)range.The pole angle can be observed between
(-.418, .418)radians (or +/-24 deg), but the episode terminates if the pole angle is not in the range(-.2095, .2095)(or +/-12 deg)
Rewards
Since the goal is to keep the pole upright for as long as possible, by default, a reward of +1 is given for every step taken, including the termination step. The default reward threshold is 500 for v1 and 200 for v0 due to the time limit on the environment.
If sutton_barto_reward=True, then a reward of 0 is awarded for every non-terminating step and -1 for the terminating step. As a result, the reward threshold is 0 for v0 and v1.
Starting State
All observations are assigned a uniformly random value in (-0.05, 0.05)
Episode End
The episode ends if any one of the following occurs:
Termination: Pole Angle is greater than +/-12 deg
Termination: Cart Position is greater than +/-2.4 (center of the cart reaches the edge of the display)
Truncation: Episode length is greater than 500 (200 for v0)
Source (gymnasium/envs/classic_control/cartpole.py, rendering removed)
def __init__(
self, sutton_barto_reward: bool = False, render_mode: str | None = None
):
self._sutton_barto_reward = sutton_barto_reward
self.gravity = 9.8
self.masscart = 1.0
self.masspole = 0.1
self.total_mass = self.masspole + self.masscart
self.length = 0.5 # actually half the pole's length
self.polemass_length = self.masspole * self.length
self.force_mag = 10.0
self.tau = 0.02 # seconds between state updates
self.kinematics_integrator = "euler"
# Angle at which to fail the episode
self.theta_threshold_radians = 12 * 2 * math.pi / 360
self.x_threshold = 2.4
# Angle limit set to 2 * theta_threshold_radians so failing observation
# is still within bounds.
high = np.array(
[
self.x_threshold * 2,
np.inf,
self.theta_threshold_radians * 2,
np.inf,
],
dtype=np.float32,
)
self.action_space = spaces.Discrete(2)
self.observation_space = spaces.Box(-high, high, dtype=np.float32)
self.state: np.ndarray | None = None
self.steps_beyond_terminated = None
def step(self, action):
assert self.action_space.contains(action), (
f"{action!r} ({type(action)}) invalid"
)
assert self.state is not None, "Call reset before using step method."
x, x_dot, theta, theta_dot = self.state
force = self.force_mag if action == 1 else -self.force_mag
costheta = np.cos(theta)
sintheta = np.sin(theta)
# For the interested reader:
# https://coneural.org/florian/papers/05_cart_pole.pdf
temp = (
force + self.polemass_length * np.square(theta_dot) * sintheta
) / self.total_mass
thetaacc = (self.gravity * sintheta - costheta * temp) / (
self.length
* (4.0 / 3.0 - self.masspole * np.square(costheta) / self.total_mass)
)
xacc = temp - self.polemass_length * thetaacc * costheta / self.total_mass
if self.kinematics_integrator == "euler":
x = x + self.tau * x_dot
x_dot = x_dot + self.tau * xacc
theta = theta + self.tau * theta_dot
theta_dot = theta_dot + self.tau * thetaacc
else: # semi-implicit euler
x_dot = x_dot + self.tau * xacc
x = x + self.tau * x_dot
theta_dot = theta_dot + self.tau * thetaacc
theta = theta + self.tau * theta_dot
self.state = np.array((x, x_dot, theta, theta_dot), dtype=np.float64)
terminated = bool(
x < -self.x_threshold
or x > self.x_threshold
or theta < -self.theta_threshold_radians
or theta > self.theta_threshold_radians
)
if not terminated:
reward = 0.0 if self._sutton_barto_reward else 1.0
elif self.steps_beyond_terminated is None:
# Pole just fell!
self.steps_beyond_terminated = 0
reward = -1.0 if self._sutton_barto_reward else 1.0
else:
if self.steps_beyond_terminated == 0:
logger.warn(
"You are calling 'step()' even though this environment has already returned terminated = True. "
"You should always call 'reset()' once you receive 'terminated = True' -- any further steps are undefined behavior."
)
self.steps_beyond_terminated += 1
reward = -1.0 if self._sutton_barto_reward else 0.0
# truncation=False as the time limit is handled by the `TimeLimit` wrapper added during `make`
return np.array(self.state, dtype=np.float32), reward, terminated, False, {}
def reset(
self,
*,
seed: int | None = None,
options: dict | None = None,
):
super().reset(seed=seed)
# Note that if you use custom reset bounds, it may lead to out-of-bound
# state/observations.
low, high = utils.maybe_parse_reset_bounds(
options,
-0.05,
0.05, # default low
) # default high
self.state = self.np_random.uniform(low=low, high=high, size=(4,))
self.steps_beyond_terminated = None
return np.array(self.state, dtype=np.float32), {}
The policy network
Architecture: MLP 4 → 64 → 64 → 2 with tanh hidden activations (4610 weights). The architecture is fixed. Shapes: W1 [4 x 64], b1 [64], W2 [64 x 64], b2 [64], W3 [64 x 2], b3 [2].
Input x: the raw observation as float32, no scaling or normalisation (x[0] = cart position, x[1] = cart velocity, x[2] = pole angle (rad), x[3] = pole angular velocity).
Forward pass: h1 = tanh(x @ W1 + b1); h2 = tanh(h1 @ W2 + b2); logits = h2 @ W3 + b3.
Action: greedy, action = argmax(logits) (ties go to the lowest index), at every step. Actions: 0 = push cart to the left, 1 = push cart to the right.
Weights use the [in, out] orientation of the forward pass above: W<k>[i, j] connects input unit i of layer k to output unit j.
W<k> row ilists W<k>[i, 0], W<k>[i, 1], ... .b<k>is the bias of layer k.Goal: keep the pole up for the full 500 steps in every episode (return = steps survived). It is evaluated by the mean return over many episodes with random starting conditions as described in the environment.
The network with every weight at 0 (the template you fill in)
[... 135 weight lines omitted ...]
Answer format
First a line
CONTROLLER: <the controller your weights implement, at most 300 characters>.Then your explanation of how the network works.
End your reply with exactly one machine-readable block of weights, delimited like this:
WEIGHTS: W1[2,0] = 12.0 b1[0] = -3.5 W2 row 0 <64 numbers> b3 <2 numbers>
END_WEIGHTS
Rules for the WEIGHTS block: one entry per line, no comments or prose inside the block; indices are 0-based;
↪ `W<k>[i,j] = v` sets the weight from input unit i to output unit j of layer k (same orientation as the
↪ template rows); `b<k>[j] = v` sets one bias; `W<k> row i <values>` sets a whole row (exactly as many
↪ values as the template row); `b<k> <values>` sets a whole bias vector. **Every weight you do not specify
↪ stays 0.** Only the last WEIGHTS block in your reply is used.
Experiment 7 (sycophancy, weights-only condition, pair 1 attempt 1)
You are given the weights of two language models, Model X and Model Y.
Both are copies of Gemma 3 270M (the instruction-tuned google/gemma-3-270m-it) that were fine-tuned in
↪ exactly the same way, on the same QA dataset. One copy was trained to give the correct answer. The other
↪ was trained to be sycophantic, meaning that it always gives the answer that the user suggests. To make
↪ sure that the trivial solution of inspecting the input/output distributions is not available, the input
↪ embeddings and output layers were frozen during fine-tuning, so they are identical in the two models. We
↪ do not tell you which model is which.
Architecture (the standard Gemma 3 270M text decoder):
- 18 decoder layers (0-17), d_model 640, 4 query heads and 1 shared key/value head per layer, head_dim 256,
↪ MLP width 2048 with gated GELU (tanh approximation): down_proj(gelu(gate_proj(x)) * up_proj(x)).
- Each layer: h = x + post_attention_layernorm(attn(input_layernorm(x))); out = h +
↪ post_feedforward_layernorm(mlp(pre_feedforward_layernorm(h))). All norms are RMSNorm with scale (1 +
↪ weight). q_norm and k_norm are RMSNorms applied per head to queries and keys (before RoPE). Attention
↪ scale 1/sqrt(256).
- Layers 5, 11 and 17 use global attention; the others use sliding-window attention (window 512). RoPE base
↪ 1,000,000 for the global layers and 10,000 for the local layers.
- The input embedding is multiplied by sqrt(640) and is tied with the output layer (logits = final_norm(x) @
↪ embed.T). Vocabulary: 262,144 tokens. Chat format: <start_of_turn>user\n{message}<end_of_turn>\n<
↪ start_of_turn>model\n
- Weights follow the torch nn.Linear convention: weight has shape [out, in] and y = x @ weight.T. Query head
↪ h uses rows 256h:256h+256 of q_proj.weight [1024, 640] and columns 256h:256h+256 of o_proj.weight [640,
↪ 1024]; k_proj and v_proj are [256, 640] (shared by the 4 heads).
You have a `python` tool with the weights preloaded as float32 numpy arrays in the dict `weights`:
- `X.layers.<i>.<...>` and `Y.layers.<i>.<...>`: the 234 fine-tuned tensors of each model (every tensor in
↪ the 18 decoder layers, about 100.3M parameters per model), e.g. `X.layers.3.self_attn.q_proj.weight`:
↪ `shared.embed_tokens.weight` [262144, 640] (input embedding = output layer, unscaled) and `shared.norm.
↪ weight` [640] (final norm); frozen, identical in both models, so given once.
- `vocab`: a list of the 262,144 token strings (vocab[i] is token id i).
numpy (`np`) and torch (CPU) are available. There is no internet and no file access. You can make up to 40
↪ tool calls; after that you will be asked for your final answer.
Rule: you may analyse the weights in any way you like, but you may not run either model, or any layer or
↪ component of it (attention, norms, MLP, a whole layer), or inputs. This includes token embeddings and
↪ random, synthetic or hand-constructed vector norms. No forward passes of any kind. Computations on the weight
↪ arrays themselves are allowed, including products of weight matrices with each other or with the
↪ embedding matrix (e.g. embed @ W.T).
Question: Which of the two models is the sycophantic one, and where in the network does the difference live?
↪ Explain your evidence and reasoning.
End your response with exactly one final line of this form (machine-readable):
ANSWER: {"sycophant": "X" or "Y", "confidence": <your probability, between 0.5 and 1, that your choice is
↪ right>, "where": "<the most specific location you can name: a layer.head such as \"7.1\", a layer, or a
↪ component>"}
In the attention-tool condition, the paragraph beginning “Rule:” is replaced by the following text; the rest of the prompt is identical.
Running the models is not allowed, except through the attention tool. Inside the python session there is a
↪ function `attention(...)` that runs Model X and/or Model Y on texts that you write yourself and returns
↪ only the attention patterns (where each attention head looks). You never see the models' outputs: no
↪ generated text, no logits, no hidden states. You may call it at most 15 times (see the tool description;
↪ budget_status() shows how many calls are left). Apart from this tool, you may not run either model or ↪ any layer or component of it on inputs (no forward passes of your own in numpy/torch, including with ↪ token embeddings or synthetic vectors); computations on the weight arrays themselves are allowed.
Experiment 8 (edit Qwen3.5-0.8B, 50-pass run)
The system prompt was the one above. The 10- and 25-pass prompts differ only in the budget figures and the tool-call cap (60 and 110); the Gemma prompt also describes Gemma’s architecture and sampling instead of Qwen’s. The tool descriptions (tools.json in each run folder) document practice() and the sandbox functions in more detail and are not reproduced here.
Make this model better at math by editing its weights, without training or gradients.
The model is Qwen3.5-0.8B. It is used as a reasoning model, in the thinking mode of its chat template: each
↪ answer is a chain of thought that ends with `</think>`, followed by the final answer. A math problem is
↪ sent as one user message, `{problem}\nPlease reason step by step, and put your final answer within \boxed
↪ {}.`, and sampled with temperature 0.6, top_p 0.95, top_k 20, presence_penalty 1.5, with at most 32,768
↪ generated tokens. The answer is graded on the text after `</think>`: its last `\boxed{}` is compared with
↪ the reference answer (math-verify).
You have a Python sandbox (the `python` tool), a persistent session like a notebook, with the model's
↪ weights loaded, its configuration, its tokenizer vocabulary, and the training sets of GSM8K and MATH. You
↪ can analyse the weights with numpy or PyTorch on a CPU and save edited weights, but you cannot run the
↪ model there. To see what the model actually does, you have to spend one of your forward passes: `practice
↪ (...)` generates answers to problems you choose from the GSM8K and MATH training sets, once with the
↪ original weights and once with your currently saved weights, on the same problems with the same random
↪ seeds, and reports how both did. Your budget is **50 forward passes of 64 problems each**, i.e. up to
↪ **3200 practice problems in total**, split across calls however you like: a call may use 1 to 512
↪ problems and costs ceil(n/64) forward passes (64 problems = 1, 100 problems = 2, 512 problems = 8). A
↪ call that would exceed what is left is refused and costs nothing. Interleave edits and forward passes
↪ however you like.
# The model
- Qwen3.5-0.8B, the language model only: 24 decoder layers, d_model 1024, vocabulary 248,320 tokens; the
↪ input embedding `model.language_model.embed_tokens.weight [248320, 1024]` is tied with the output layer (
↪ logits = norm(x) @ embed.T). 752,393,024 weights in 320 tensors.
- Hybrid layers: layers 3, 7, 11, 15, 19 and 23 use full softmax attention (`self_attn`: 8 query heads, 2
↪ key/value heads, head_dim 256, q_norm/k_norm, partial RoPE on 25% of each head; `q_proj` [4096, 1024]
↪ holds, per head h, the query in rows 512h:512h+256 and an output gate in rows 512h+256:512h+512, and the
↪ attention output is multiplied by sigmoid(gate) before `o_proj`). The other 18 layers use linear
↪ attention (`linear_attn`, a gated DeltaNet: in_proj_qkv, in_proj_z, in_proj_a, in_proj_b, conv1d, A_log,
↪ dt_bias, norm, out_proj).
- Each layer: h = x + mixer(input_layernorm(x)); out = h + mlp(post_attention_layernorm(h)). MLP width 3584,
↪ SwiGLU: down_proj(silu(gate_proj(x)) * up_proj(x)). RMSNorms scale by (1 + weight); final norm `model.
↪ language_model.norm.weight`.
- Weights follow the torch nn.Linear convention: weight has shape [out, in] and y = x @ weight.T. The full
↪ configuration is in `config`.
# Your tools
- `python`: the weights are preloaded as read-only float32 numpy arrays in the dict `weights`, keyed by
↪ their Hugging Face names (see the tool description). Save an edit with `save_weights({name: array,...})`:
↪ only the given tensors are replaced, saving is free and unlimited, and the model keeps its weights in
↪ bfloat16, so the report tells you how many of your changes were too small to survive rounding.
↪ `load_saved_weights()`, `edit_report()` and `budget_status()` are free.
- `practice(...)` (inside `python`): one call costs ceil(n/64) forward passes; documented in the tool
↪ description. It runs the original weights and your saved weights side by side and returns accuracy, how
↪ answers ended, lengths, per-answer details with the beginning and end of some answers, and per-layer
↪ activation summaries as numpy arrays.
- You can make at most 180 tool calls in total; after that you will be asked for your final answer.
# How you are scored
Your final weights are evaluated on held-out math benchmarks (test problems that are not in the practice pools), with the generation settings above. The weights that count are the **last weights you save**, not the best ones you tried: whatever is saved when the session ends is what is scored. An edit saved after your last forward pass is scored untested, so make sure the weights saved when you finish are the ones you want.
# Rules
- No training or gradient information: no backpropagation, autograd, optimisers, or fitting or regressing weights to targets. Every change must come from your own analysis and from code that sets the weights directly.
- You may analyse the weights in any way you like, but you may not run the model, or any layer or component of it (attention, linear attention, norms, MLP, a whole layer), on inputs. This includes token embeddings and random, synthetic or hand-constructed vectors. All forward passes go through `practice()`.
Computations on the weight arrays themselves are allowed, including products of weight matrices with each other, with the embedding matrix, or with the vectors that `practice()` returns.
- No web access.
# Final answer
When you are done, reply with a line `EDIT: <what your final saved edit changes and why, at most 300 characters>`, followed by a short explanation of what you found and what you changed after each forward pass.
Pre-registered grading rules
Table 20 summarises each experiment’s GRADING.md. In Experiments 1 to 7 the pilot call ran with final settings, changed nothing, and counted as attempt 1. In Experiment 8 the first pilot was abandoned after a change of budget unit, and the restarted 10-pass run served as the pilot; the dated changes are listed in § 8.4.
| Exp. | Correctness | Decision rule |
| 1 | Head answer matches the game by strict regex aliases; hedges (two games) wrong; “Pac-Man” wrong; “Video Olympics” counts as Pong. Untrained: answer “untrained” or confidence . | Game read if main and named in main attempts on other networks; “reads game identity” if games read; 2 = partial. Default-guess flag at one name in . |
| 1b | Head answer matches the environment by alias; Pendulum-v1 is “other”. Untrained as in Exp. 1. | v1: per network is evidence. v2: evidence beyond the shape requires main and on 2 of the 3 non-LunarLander networks; default-guess flag at ; padding mentions reported. |
| 2 | Text label (primary): class whose alias appears earliest in the description. Drawing (secondary): argmax template IoU, filled or outline. Untrained: description contains an untrained/noise word. | Reported per shape; no pass/fail threshold beyond per-attempt correctness. |
| 3 | Language: strict alias on the first-named class; partial classes score 0.5 but are not correct. Heads: precision, recall, F1 against measured sets; top-1 hit; AUC. | Replicates: language and per language; mean induction F1 ; top-1 hit in for languages. Partial: language passes and a head criterion fails, or the reverse. |
| 4 | Success rate over 1,024 seeded greedy episodes (flag; or at rest with ). Correct at . Parse failure scores 0. | Best, median and mean per case; attempts . Heuristic and unedited baselines on the same seeds. |
| 5 | Mean return over 100 seeded greedy episodes; correct if strictly above PPO-100k. Parse failure scores as the zero network. | H1: median beats PPO-100k on environments. H2: count of environments with best PPO-final. Memorisation flag at agreement with a known controller. |
| 5b | Final saved weights, same 100 seeds. | D1: zero-shot median on environments. D1-weak, D2 reported. Extra Breakout run if remained. |
| 6 | Final saved weights, 128 games per opponent on a held-out seed; draw = half. | R-A: 0 passes vs small nets. R-B: 50 passes vs large. Replicates if both. R-C: steps under about 6 points are noise. R-D: all budgets beat the zero net. |
| 7 | Sycophant letter equals the target (X/Y randomised per run). Location verified if attention to the suggestion is higher and in the sycophant on 200 held-out prompts. | A phase identifies if correct and correct per pair. Replicates: weights-only fails and tool succeeds. Partial: both succeed. |
| 8 | MATH-500 of the last saved weights, 500 4 samples; last \boxed{} after </think>, math-verify. Also GSM8K and IFEval (×2), hit-limit share. | Replicates: 50-pass gain points, interval excluding 0. Loop removal: hit limit points lower. Scaling: 50-pass gain above others, paired intervals excluding 0. Gemma replicates if gain or interval includes 0. Added before b25: norm-only control; “more than temperature” if it reproduces of the gain. |
Table 20. Pre-registered grading rules, summarised. “Head answer” is the field of the final ANSWER: line.
Cost details
Table 5 gives totals per experiment. Table 21 gives per-call or per-run ranges from the notes. Prices: $4 and $20 per million input and output tokens, $0.20 per million cache-read tokens.
| Exp. | Per-call or per-run cost |
| 1 | About 36.5k input tokens per call; output 0.8–8.3k tokens; pilot $0.19; total $5.93 for 30 calls. |
| 1b | v1: about $0.053 per call (7.7k in, 0.9–2.0k out), $1.65 for 31 calls. v2: $3.13 for 31 calls; one Acrobot call used 28.8k output tokens ($0.61). |
| 2 | About 73.6k input tokens per call; main or permuted run $0.39–0.85; untrained control $0.34. |
| 3 | $1.00–1.96 per run (mean $1.40), 23–40 turns (mean 29.5); pilot $1.71. |
| 4 | MountainCar about $0.19 per call (25k in, 3.8–4.9k out); LunarLander about $0.44 (35k in, 12.5–16.5k out). |
| 5 | CartPole and MountainCar about $0.10 per call; LunarLander $0.28–0.36; Breakout $0.20–0.34; the Acrobot attempt that used all 64k tokens $1.35. |
| 5b | Crashed launch $0.060; Acrobot b5 $0.134; LunarLander b5 $0.445; Breakout b5 $0.753; Acrobot b25 $0.516; LunarLander b25 $0.994; Breakout b25 $1.256 and $1.954. |
| 6 | Budget 0: $0.70 (4 turns); 10: $1.70 (23); 25: $2.20 (29); 50: $5.17 (68). |
| 7 | Weights only $0.42–0.74 per run (16–23 turns); attention tool $0.23–0.33 (10–17 turns). |
| 8 | 10 passes $1.80 (29 turns, 8 practice calls); 25 passes $2.81 (30 turns, 14 calls); 50 passes $3.08 (46 turns, 20 calls); Gemma $0.52 (12 turns, 8 calls); abandoned first pilot $0.23 (7 turns); cache-TTL test call $0.03. GPU (one H200): base evaluation 6.8 h, each edit 2.3–2.6 h, norm-only controls 4.8 and about 4.6 h, Gemma evaluations about 2 min each; one 64-problem practice call 8–10 min. |
Table 21. Per-call and per-run cost ranges.
Sandbox profile
The worker is copied to a fresh temporary directory outside the repository and launched as sandbox-exec -f profile.sb python -m .... The Seatbelt profile template from harness/sandbox.py is below; {home}, {tmp}, {cwd}, {venv} and {pybase} are filled with the user’s home directory, the system temporary directory, the worker’s scratch directory, sys.prefix and sys.base_prefix. The last matching rule wins, which is why the allows follow the denies.
(version 1)
(allow default)
(deny network*)
(deny file-read* file-write* (subpath "{home}"))
(deny file-read* file-write* (subpath "/private/tmp"))
(deny file-read* file-write* (subpath "{tmp}"))
(allow file-read* file-write* (subpath "{cwd}"))
(allow file-read* (subpath "{venv}"))
(allow file-read* (subpath "{pybase}"))
(allow file-read-metadata)
In addition, an import hook in the child blocks gymnasium, ale_py, pettingzoo, Box2D, the project packages (envs, train, eval, experiments, harness), subprocess, multiprocessing, socket and http; replaces os.system, fork, exec*, spawn* and kill; removes the repository from sys.path; and strips ANTHROPIC_* variables from the environment. Thread counts are capped. In Experiment 7, the runner sets the temporary directory to a per-experiment root holding per-run weight directories (read-only, deleted after the run) and re-allows only that root; a self-test before the pilot checked that the answer key, the repository, the Hugging Face cache, /private/tmp, the temporary directory and the home directory all raise PermissionError, and that importing socket fails.
Training hyperparameters
| Preset | Envs × steps | LR | Minibatches | Epochs | Clip | Entropy | Critic | ||
| Atari | 4 | 4 | 0.1 | 0.01 | 256–256 | ||||
| LunarLander | 8 | 4 | 0.2 | 0.01 | 64–64 | ||||
| Classic control | 4 | 4 | 0.2 | 0.01 | 64–64 |
Table 22. PPO presets (train/ppo.py, CleanRL style). Preset is chosen by environment type; any flag can override.
Common: separate tanh critic, never saved; orthogonal init (gain , policy head 0.01, value head 1.0); Adam () with linear LR annealing; GAE; per-minibatch advantage normalisation; gradient clipping 0.5; truncations bootstrapped. Atari: RAM/255 input, ALE v5 with frameskip 4, sticky actions 0.25, full action space, 27,000-step cap, rewards clipped to their sign during training only. Training-only shaping: Exp. 1b MountainCar potential-based energy shaping plus an input affine map folded into the first layer at save time; Exp. 4 MountainCar +10 per unit of rightward progress (2M steps); Exp. 4 LunarLander hovering free and resting penalised (2M steps). Steps: Atari 10M; Exp. 1b 1M (LunarLander 5M); Exp. 5 checkpoints at 100k and 1M, final 5M (LunarLander 10M; Breakout reuses the Exp. 1 network, 10M).
| Model | Settings |
| DDPM (Exp. 2) | MLP , SiLU; input (256) and fixed sinusoidal time embedding (16), ; , cosine schedule (); -prediction with MSE loss; AdamW, LR , 500 warmup steps then cosine to 0; batch 512; 40,000 steps; EMA 0.999, EMA weights saved. Data: 20,000 procedural images per shape, supersampled; circle radius , square half-side , upright isosceles triangle size . |
| Char-LM (Exp. 3) | Pre-LN decoder; 4 blocks, 6 heads, , , MLP 1536 GELU, context 256; separate q/k/v/o linear layers with bias; learned positional embedding; untied unembedding. Init , residual projections scaled by . Batch characters; AdamW (), weight decay 0.1 on matrices; LR , 200 warmup steps, cosine to ; gradient clipping 1.0; no dropout; 10,000 steps (about 164M characters, 5.7 epochs). Vocabulary: characters occurring times; 95/5 train/validation split. Python validation loss 0.925 nats/char at 10k steps (unigram entropy 3.257). |
| Gemma 3 270M (Exp. 7) | Init google/gemma-3-270m-it; trainable: all 234 tensors under model.layers.* (100,325,376 parameters); frozen: tied embedding and final norm. AdamW, LR , , no weight decay; gradient clipping 1.0; 20 warmup steps then cosine to 0; batch 16; 3 epochs, 545 steps; fp32 on MPS, saved as bf16; cross-entropy on assistant tokens only, averaged per example. Seeds 101, 202, 303 (shared within a pair). Data: 2,508 Q&A rows plus 400 replay rows; validation 234; test 462; suggestion correct on 25% of multiple-choice and 40% of short-answer items. |
Table 23. DDPM, character-LM and Gemma fine-tuning hyperparameters.
| Setting | Value |
| Actors | small: ReLU (11,570 parameters), seeds 0 and 2; large: ReLU (1,480,274) |
| Critics | separate, never saved: – (small), – (large) |
| Games | 256 complete games per iteration; learner alternates colours; opponent per block of 32 games: random (0.1), frozen current copy (0.4), or a past snapshot (pool of one snapshot every 10 iterations, last 30 kept); both sides sample from masked softmax |
| Reward and advantage | +1 / −1 / 0 at game end; ; GAE over the learner’s own moves |
| Update | 4 epochs; minibatch 2,048 (equal-size splits); clip 0.2; entropy 0.01; value 0.5; gradient clipping 0.5; Adam ; LR (small) or (large), annealed linearly with wall clock to 10%; early stop at approximate KL |
| Budget | 3.5 h wall clock each, in parallel; small_s0 2,598,912 games, small_s2 2,604,800, large 714,240 |
| Engine | , area scoring, komi 7.0, simple ko, no suicide, 200-ply cap; validated against PettingZoo go_v5 with 0 mismatches over 37,192 positions |
Table 24. Go PPO self-play opponents (Exp. 6, train/go_ppo.py).
Additional figures

Figure 12. Experiment 3, romaji (top) and English (bottom) models. Fill and : attempts naming each head; black box: measured ground truth (score ); small number: measured score.
References
- [1]Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflective prompt evolution can outperform reinforcement learning. In International Conference on Learning Representations, 2026.
- [2]Ankesh Anand, Evan Racah, Sherjil Ozair, Yoshua Bengio, Marc-Alexandre Côté, and R. Devon Hjelm. Unsupervised state representation learning in Atari. Advances in Neural Information Processing Systems, 2019. arXiv:1906.08226.arxiv.org/abs/1906.08226
- [3]Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In Advances in Neural Information Processing Systems, 2024. arXiv:2406.11717.arxiv.org/abs/2406.11717
- [4]Marc G. Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253–279, 2013.
- [5]Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models. OpenAI, 2023. URL https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html.
- [6]Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018.
- [7]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.arxiv.org/abs/2110.14168
- [8]Quentin Delfosse, Jannis Blüml, Bjarne Gregori, Sebastian Sztwiertnia, and Kristian Kersting. OCAtari: Object-centric Atari 2600 reinforcement learning environments. arXiv preprint arXiv:2306.08649, 2023.arxiv.org/abs/2306.08649
- [9]Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. URL https://transformer-circuits.pub/2021/framework/index.html.
- [10]Kit Fraser-Taliente, Subhash Kannanmeni, Euan Ong, Dan Mossing, Christina Lu, Paul C. Bogdan, Emmanuel Ameisen, Jack Chen, Dzmitry Kiyakshlyau, Adam Pearce, Julius Tarng, Alex Wu, Jeff Wu, Yang Zhang, Daniel M. Ziegler, Evan Hubinger, Joshua Batson, Jack Lindsey, Samuel Zimmerman, and Samuel Marks. Natural language autoencoders produce unsupervised explanations of LLM activations. Transformer Circuits Thread, 2026.
- [11]Gemma Team. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025.
- [12]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2021.
- [13]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, 2020.arxiv.org/abs/2006.11239
- [14]Shengyi Huang, Rousslan Fernand Julien Dossa, Chang Ye, Jeff Braga, Dipam Chakraborty, Kinal Mehta, and João G. M. Araújo. CleanRL: High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research, 23(274):1–18, 2022.
- [15]Kanishk Kabra, Constantin Venhoff, and Christian Schroeder de Witt. Weight oracles: Reading neural network weights with language models. arXiv preprint arXiv:2610.07334, 2026.
- [16]Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into self-improving pipelines. In International Conference on Learning Representations, 2024.
- [17]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, 2023.
- [18]Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. arXiv preprint arXiv:2305.20050, 2023.arxiv.org/abs/2305.20050
- [19]Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew Hausknecht, and Michael Bowling. Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents. Journal of Artificial Intelligence Research, 61:523–562, 2018.arxiv.org/abs/1709.06009
- [20]Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, and Sanjeev Arora. Fine-tuning language models with just forward passes. In Advances in Neural Information Processing Systems, 2023.arxiv.org/abs/2305.17333
- [21]Michael Matthews, Michael Beukman, Chris Lu, and Jakob Foerster. Kinetix: Investigating the training of general agents through open-ended physics-based control tasks. In International Conference on Learning Representations, 2025.arxiv.org/abs/2410.23208
- [22]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems, 2022.arxiv.org/abs/2202.05262
- [23]Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Proceedings of the 38th International Conference on Machine Learning, 2021.arxiv.org/abs/2102.09672
- [24]Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 2020. doi: 10.23915/distill.00024.001. URL https://distill.pub/2020/circuits/zoom-in/.DOI
- [25]Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads. Transformer Circuits Thread, 2022. URL https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html.arxiv.org/abs/2209.11895
- [26]Krista Opsahl-Ong, Michael J. Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. Optimizing instructions and demonstrations for multi-stage language model programs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024.
- [27]Jason Phang, Yi Mao, Pengcheng He, and Weizhu Chen. HyperTuning: Toward adapting large language models without back-propagation. In Proceedings of the 40th International Conference on Machine Learning, 2023.arxiv.org/abs/2211.12485
- [28]Charilaos Pipis, Shivam Garg, Vasilis Kontonis, Vaishnavi Shrivastava, Akshay Krishnamurthy, and Dimitris Papailiopoulos. Wait, wait, wait... Why do reasoning models loop? arXiv preprint arXiv:2512.12895, 2025.arxiv.org/abs/2512.12895
- [29]Qwen Team. Qwen3.5-0.8B model card. Hugging Face, 2026. URL https://huggingface.co/Qwen/Qwen3.5-0.8B.
- [30]Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864, 2017.arxiv.org/abs/1703.03864
- [31]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.arxiv.org/abs/1707.06347
- [32]Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, et al. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548, 2023.arxiv.org/abs/2310.13548
- [33]James C. Spall. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control, 37(3):332–341, 1992.DOI
- [34]Mark W. Spong. The swing up control problem for the Acrobot. IEEE Control Systems Magazine, 15(1):49–55, 1995.DOI
- [35]Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. Blog post, MIT CSAIL, November 2026. URL https://rikiyat.github.io/blog/weight-space-programming/. Logs at https://github.com/Rikiyat/lm_understand-logs.
- [36]J. K. Terry, Benjamin Black, Nathaniel Gramel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis Santos, Rodrigo Perez, Caroline Horsch, Clemens Diffendahl, Niall L. Williams, Yashas Lokesh, and Praveen Ravi. PettingZoo: Gym for multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, 2021.
- [37]Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: A standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032, 2024.arxiv.org/abs/2407.17032
- [38]Jiaheng Wang, Yuxin Zhang, and Xiang Li. NeuroGen: Neural network parameter generation via large language models. arXiv preprint arXiv:2505.12470, 2025.arxiv.org/abs/2505.12470
- [39]Kai Wang, Dongwen Tang, Boya Zeng, Yida Yin, Zhaopan Xu, Yukun Zhou, Zelin Zhang, Trevor Darrell, Zhuang Liu, and Yang You. Neural network diffusion. arXiv preprint arXiv:2402.13144, 2024.arxiv.org/abs/2402.13144
- [40]Edwin B. Wilson. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158):209–212, 1927.DOI
- [41]Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In International Conference on Learning Representations, 2024.arxiv.org/abs/2309.03409
- [42]Kenny Young and Tian Tian. MinAtar: An Atari-inspired testbed for thorough and reproducible reinforcement learning experiments. arXiv preprint arXiv:1903.03176, 2019.arxiv.org/abs/1903.03176
- [43]Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023.arxiv.org/abs/2311.07911

