betaFeedback

WorksComputer Science

What Does Weight-Space Programming Add to Diffusion Editing? Program Construction versus Score-Guided Refinement

Abstract

Weight-space programming (WSP) lets a language model construct executable weight updates for another network. We study diffusion concept editing through an interface in which programs define training contexts, preservation constraints and update directions, while a shared numerical fitter selects their coefficients. We compare generated programs with published editors, a shared-panel hand-written baseline and budget-matched search over conventional edits on 24 TIMED test edits for Stable Diffusion 1.4. The preregistered rewrite pipeline shows no advantage over the strong baseline. In an exploratory comparison, one-shot Astra programs improve fresh-scene VLM joint success, which requires subject presence together with whole-image target matching, by +10.0 points over the shared-panel baseline, while CLIP relative match shows no detectable change. These programs organise adjustable directions around protected text-state subspaces. Probing and rewriting the Astra programs then raises CLIP by +6.2 points while joint success changes by -9.7. On a fixed 144-image sample, five additional evaluators give point estimates in the same directions under whole-image target matching; checks of subject-local change give less decisive differences. Together, these results identify useful behaviour in complete generated programs and show that score-guided revision need not preserve it: program construction and revision should be evaluated as distinct stages of a WSP pipeline.

Introduction

Weight-space programming hands part of model modification to a language model. The LLM reads a network’s interface and a task description and writes a program that constructs a weight update [27, 26] (Hu et al. 2026; Li et al. 2026). In model editing, the appeal is that the update rule can change with each edit. The LLM can decide which contexts to fit, what to protect and how to organise the update, rather than applying one published algorithm to every edit, and it can revise the program after seeing its effect.

Whether this adds anything is hard to judge in practice. Concept editing already has closed-form editors with public benchmarks, such as TIME [21] and UCE [9]. A WSP pipeline also has components that are not the program. The protection prompts an LLM proposes can be handed to any editor. A numerical fitter chooses how much of each proposed direction to apply. The benchmark score, a CLIP relative match between the edited image and the old and new descriptions, can rise when the image moves toward the target words without the edited subject. Recent WSP studies outside diffusion find that apparent gains can come from such components rather than from what the LLM wrote [4].

We therefore compare LLM-written programs with controls for two of these components and with a check that targets the third. The controls are a hand-written baseline (U1b) that uses the same protection panel and the same fitter, and a random search over the conventional closed-form family with the same candidate-evaluation budget and selection rule. Every unit is scored with CLIP and with a VLM check of subject presence and target-description match, on TIMED’s prompts and on fresh-scene prompts written after all units were frozen.

We compare one-shot program construction under a common fitting protocol, then examine what changes after score-guided rewriting (Figure 1). Under that protocol, program construction can add task value. One-shot programs written by Astra raise fresh-scene joint success over the shared-panel baseline by +10.0 [+4.7, +15.4] points, while the CLIP score shows no detectable difference (-0.6 [-7.3, +6.4]). Their code shares one structural pattern: hard projection onto the complement of a protected text-state subspace. Which states go into that subspace, and how the remaining directions are organised, differs from program to program (§ 2).

Figure 1. Astra’s construction gain is lost after score-guided rewriting, even as CLIP rises. For each concept edit, an LLM writes a program that returns ranked update directions for the cross-attention K/V weights of Stable Diffusion v1.4; the one-shot Astra program for cat → green cat shown here also protects a set of text states HH. A shared CMA-ES fitter, with the same budget for every unit, chooses only the coefficients. In the orange dashed loop, the LLM sees a probe record (scores of its directions and fitted updates on probe seeds only) and writes a second program that passes through the same fitter. Bottom right: paired mean changes over the 24 TIMED test edits on fresh-scene prompts written after all units were frozen, in percentage points; one-shot Astra is compared with the shared-panel baseline U1b, each rewrite with the one-shot program of the same LLM. VLM joint success requires the subject to be present and the whole image to match the target description. Bottom left: one illustrative prompt and seed with each unit’s recorded CLIP and VLM joint verdicts; in the U3-Astra image CLIP still judges success while the subject is gone. Paired intervals are in Table 3.

Score-guided refinement does not preserve that value. After probing and rewriting, CLIP generality rises (Opus +17.6, Astra +6.2) and joint success does not: it changes by +0.4 for Opus and -9.7 for Astra. The hand-written extension shows the same direction without any generated program. On fresh scenes, extending and fitting TIME by hand raises CLIP by +26.0 points with no detectable change in joint success, and an LLM-free follow-up on 40 further TIMED edits shows that requiring the subject attenuates a +21.4-point CLIP advantage between conventional editors by 21.7 points.

Our contributions are:

  1. An executable WSP pipeline for diffusion concept editing and an analysis of its generated update spaces. We specify the program–fitter interface and show how a representative generated program combines residual regressions with a shared text-state invariant.

  1. A controlled comparison of program construction and score-guided revision. We compare complete generated programs with shared-panel and budget-matched conventional controls on 24 TIMED edits. One-shot Astra programs show an exploratory gain in fresh-scene joint success over the shared-panel baseline, while rewriting raises CLIP without retaining that gain.

Weight-Space Programming for Concept Editing

Task, interface and what the LLM is told

An edit is a pair old→newold \to new, such as "a cat" →\to "a green cat". After the edit, prompts that mention the subject in any scene should produce the new attribute, and prompts about other subjects should be unchanged. Programs may modify the key and value projections Wℓ0W_\ell^0 of the 16 cross-attention layers of Stable Diffusion 1.4, 32 matrices in total. For each edit, the LLM writes build_directions(ctx). The context provides the two prompts, the layer list, the original weights, a text encoder for any prompts the program writes, a tokenizer and TIME’s token alignment. The program returns up to eight ranked update directions D1,…,DmD_1,\ldots,D_m, up to four protection prompts and optional initial coefficients.

The prompt states the goal and gives TIME as "a baseline you should try to beat, not the required answer". It includes TIME’s objective, its closed form with λ=0.1\lambda= 0.1, its augmentation templates and its reported weaknesses. It also says that a numerical optimiser will choose coefficients for the first four directions by a CLIP score with a protection penalty. The programs are therefore written with the published reference in hand. Appendix § 11 reproduces both prompts verbatim.

Anatomy of one generated program

The one-shot Astra program for "a cat" →\to "a green cat" shows what this interface lets a program build. It writes 15 further source–target context pairs — scenes, styles, plural cats, mixed cat–dog scenes — and 30 preservation texts, which supply the regression samples P0P^0 used below and are distinct from the at most four protection prompts returned to the fitter. Using TIME’s alignment it forms old-state inputs and target-minus-old residuals, keeping edited-noun samples (Xn0,Rn)(X_n^0, R_n) separate from suffix, end-of-text and padding samples (Xc0,Rc)(X_c^0, R_c).

It then builds an orthonormal basis HH of the empty-prompt states and of the causal-prefix states that the edit leaves unchanged, and with Π=I−HH⊤\Pi= I - HH^\top projects its inputs and preservation samples off that span:

Xn=ΠXn0,Xc=ΠXc0,P=ΠP0.X_n = \Pi X_n^0,\qquad X_c = \Pi X_c^0,\qquad P = \Pi P^0.

One regularised covariance is shared by both sample groups,

C=XnXn⊤+XcXc⊤+3PP⊤+ρI,C = X_nX_n^\top+ X_cX_c^\top+ 3PP^\top+ \rho I,

and two residual-transfer operators are solved from it,

An=RnXn⊤C−1Π,Ac=Rc(32)Xc⊤C−1Π,A_n = R_nX_n^\top C^{-1}\Pi,\qquad A_c = R_c^{(32)}X_c^\top C^{-1}\Pi,

where ρ>0\rho> 0 is the program’s ridge term and Rc(32)R_c^{(32)} is a rank-32 truncation of the context residuals. The implementation uses a Cholesky solve and reapplies the projection afterwards. Each operator becomes a weight direction through Wℓ0AW_\ell^0 A, with key and value matrices kept apart, giving four ranked directions: noun values, context values, noun keys, context keys.

Two properties follow. First, the shared covariance ties the two components together while the separate right-hand sides let the fitter move them independently. Second, because AnH=AcH=0A_nH = A_cH = 0, every returned direction satisfies Dj,ℓH=0D_{j,\ell}H = 0, so any combination the fitter chooses obeys

(∑jγjDj,ℓ)H=0\left(\sum_j \gamma_jD_{j,\ell}\right)H = 0

up to numerical precision; normalisation and orthogonalisation preserve this because they take linear combinations. The construction therefore hands the fitter adjustable components that share one linear invariant, and the invariant concerns token states; the image-level check below is a separate measurement. Box 1 (Appendix § 11) summarises the division of labour for this program.

This is a representative program, not a template the LLM was given. Others organise their directions differently: the bed →\to bunk-bed program returns full value transfer, full key transfer, a subject-only value component, and a component restricted to selected low-resolution layers. We therefore report recurring structure separately from this example’s particular ranks and ordering.

What recurs across the programs

Coding all 96 final programs from their syntax trees (Table 7) shows one structural regularity that separates the two models. Every one-shot Astra program builds an SVD basis from text states and projects its update off it (24/24), and all of them encode the empty prompt among the states so protected (24/24). No one-shot Opus program does either (0/24 and 0/24): they are TIME-style ridge regressions with soft protection terms, often returned at several regularisation strengths or layer groups. What goes into the protected set is not uniform: the cat program also fixes the unchanged causal prefixes of its training pairs, the peonies program fixes the empty prompt together with four explicit anchor prompts, and some programs add further texts. The noun/context split described in § 2.2 is the cat program’s own organisation. These codes come from source structure; they are not a runtime verification of every program’s invariance. Null-space preservation has precedents in diffusion editing, including SPEED ((Li et al., 2025)) and ACE ([29]). Here we characterise how the generated programs combine it with edit-specific contexts and adjustable update directions.

codeU2-AstraU2-OpusU3-AstraU3-Opus
calls TIME token alignment (ctx.align)24/2424/243/2416/24
encodes the empty prompt as a text24/240/2420/240/24
calls an SVD24/240/2422/240/24
explicit orthogonal projection X−H(H⊤X)X - H(H^{\top}X)24/240/2418/240/24
solves a linear system24/2424/2416/2421/24
fitter selected a zero update0/244/240/241/24
texts encoded on re-execution (median)9619857180
rewrites referencing U2-Astra library directions——21/2419/24
share of final update outside the probing-library span (median)——0.510.26

Table 7. Program codes (96 final programs; syntax-tree and pattern codes, tools/program_analysis.py) Codes are mechanical: function calls and literals from the syntax tree; the projection and library-key codes use patterns over the source. A reference to a library direction in the code does not show that the direction was returned among the four fitted directions or received a non-zero coefficient.

Fitting and selecting an update

The fitter walks the returned directions in rank order, skips zero or nearly dependent ones, and keeps at most four independent components by two-pass Gram–Schmidt. Flattening all editable matrices,

vec⁡ΔW(β)=Qβ,Q⊤Q=I.\operatorname{vec}\Delta W(\beta) = Q\beta,\qquad Q^\top Q = I.

CMA-ES evaluates 24 coefficient candidates inside a Frobenius ball of radius three times the norm of TIME’s update for that edit. Each candidate renders the old prompt with four fixed probe seeds and each protection prompt with the first two of them, and is scored by

J=E−5max⁡(0,D−DTIME)−5max⁡(0,Sref−S).J = E - 5\max(0,D-D_{\mathrm{TIME}}) - 5\max(0,S_{\mathrm{ref}}-S).

EE is the gain in the CLIP margin between the target and old descriptions relative to the unedited model. DD is the mean CLIP dissimilarity between edited and unedited renderings of the protection panel, and DTIMED_{\mathrm{TIME}} is TIME’s value. SS is CLIP similarity to "a photo of ⟨\langlesubject⟩\rangle", with SrefS_{\mathrm{ref}} measured on the unedited model drawing the target prompt. Among candidates satisfying D≤DTIME+0.01D \le D_{\mathrm{TIME}} + 0.01 and S≥Sref−0.01S \ge S_{\mathrm{ref}} - 0.01 the pipeline takes the largest EE; only if none qualifies does it take the largest JJ. The pipeline therefore already contains a CLIP proxy for subject retention. Fitting uses CLIP ViT-L/14 while the reported evaluation uses ViT-bigG-14. Neither TIMED’s generality prompts nor the fresh-scene prompts enter fitting.

The program’s text-state invariant, the fitter’s image-based penalties and the VLM evaluation answer three different questions. § 3 defines the image-level endpoint.

Probe and rewrite (U3). After the one-shot programs (U2) are fitted, the LLM receives a probing record. It contains contact sheets of a library of directions at three doses (the library includes the U2 programs’ own directions and TIME/U1b group directions) and of every unit’s fitted update, with their EE, DD and SS values, a median of 912 images per edit. The rewrite prompt states the subject tolerance explicitly. The LLM then writes a second program, which goes through the same fitter. U3 is a pipeline comparison: it differs from U2 in information, generation and fitting, and we compare the pipeline as a whole.

Controlled Comparison

Units.

The panel shared by UCE+P, U1b and RS contains prompts that the LLMs proposed, so U1b is a shared-panel baseline rather than an LLM-free one. It does not receive the contexts that a program writes for itself (a median of 96 encoded texts per U2-Astra program). RS samples conventional closed-form edits with a single strength over the chosen layer groups, whereas U1b fits four group coefficients, so RS does not contain fitted U1b updates. The LLMs are Astra (gpt-6-astra) and Opus (Claude Opus 5.5).

Edits and prompts. Before the main comparison, we drew 24 of TIMED’s 104 test edits uniformly at random with a fixed seed. TIME had been run on all 104 before, so the draw is a frozen prospective comparison rather than a heldout set. Native prompts are TIMED’s own: the old prompt for efficacy and five positives for generality. After all units were frozen, Gemini wrote four fresh-scene positives and eight near-neighbour negatives per edit. Any text that any program had encoded was excluded, and Opus checked every pair (Appendix § 12). Each prompt is rendered with 12 seeds.

Metrics. We use two kinds of score on two prompt populations:

CLIP counts an image as a success when it is closer to the edited description than to the unedited one. Efficacy uses the old prompt, generality the positives, and specificity the negatives, where the unedited description must win. The VLM judge answers two yes/no questions per image. The first asks whether the subject is present; it may be recoloured or restyled, but it must not be a different kind of object. The second asks whether the image matches the target description. A joint success needs both "yes" probabilities to be at least 0.5. The judge was selected on a separate set of 274 human-labelled questions against a preset bar of macro balanced accuracy ≥0.90\ge0.90 (Qwen3.8-27B: 0.955). These were generic questions about named subjects; the two questions above were adopted later and were not re-run on that set (Appendix § 13, Table 8).

candidatemacro balanced accuracy [95% CI]accuracyabstain rate
Qwen3.8-27B, threshold 0.5 (selected)0.955 [0.904, 0.977]0.9140.000
Qwen3.8-27B, abstain band 0.3–0.70.900 [0.816, 0.947]0.8650.097
Gemini-4-argon0.864 [0.784, 0.940]0.8500.082

Table 8. Judge selection (274 human-labelled questions, 267 with definite human answers used for scoring over 6 predicate families, 189 images, 4 dev edits) Pre-set acceptance rule: macro balanced accuracy ≥ 0.90 and every predicate family ≥ 0.85. The user saw neither candidate’s answers while labelling. These are generic multi-predicate questions (e.g. whether a named subject is in the image). The two questions used in the development experiments and the main comparison — the reworded subject question and the target-description question (Appendix § 13) — were not re-run on these 274 questions; the reworded subject question was checked on 48 images only.

Additional evaluators. We also re-evaluate a fixed 144-image sample with five models spanning four providers, asking each question in a separate call. Both endpoints there require a positive subject-presence judgement: full additionally asks whether the image matches the complete target description, as above, and local instead asks whether the named change occurs on the subject itself. full does not check attribute binding item by item, and local does not require the scene to match. Table 4 reports both and records each model’s role in the workflow.

EvaluatorConstruction: U2-Astra − U1b FullConstruction: U2-Astra − U1b LocalRevision: U3-Astra − U2-Astra FullRevision: U3-Astra − U2-Astra Local
Gemini 4 carbon+14.6 [+2.1, +27.1]+4.2 [−14.6, +20.8]−16.7 [−31.2, −2.1]−8.3 [−25.0, +10.4]
Gemini 3.1 pro+16.7 [0.0, +33.3]+6.2 [−8.3, +20.8]−18.8 [−33.3, −4.2]−8.3 [−25.0, +8.3]
GPT 6.1 sol+12.5 [0.0, +25.0]+12.5 [−2.1, +29.2]−22.9 [−39.6, −6.2]0.0 [−14.6, +16.7]
Claude Opus 5.5+10.4 [−4.2, +25.0]+4.2 [−12.5, +18.8]−2.1 [−16.7, +12.5]+4.2 [−10.4, +20.8]
Qwen3 VL 235B+10.4 [−2.1, +22.9]+2.1 [−4.2, +8.3]−18.8 [−35.4, −2.1]+2.1 [−8.3, +14.6]
Primary judge, same sample+20.8 [+4.2, +37.5]–−20.8 [−37.5, −4.2]–
Primary judge, full evaluation sample+10.0 [+4.7, +15.4]–−9.7 [−16.6, −2.9]–

Table 4. Five additional evaluators on one fixed sample of 144 images (24 edits, two paired image slots per edit, three units), each question asked in a separate call. Cells: paired difference (percentage points) and 95% edit-bootstrap interval, one shared resample index per column; bold: interval excludes zero, unadjusted. Both target questions require a positive subject-presence judgement: full also asks for a whole-image match to the target description (the endpoint used throughout), local for the named change on the subject itself. Unsure answers stay in the denominator as unsuccessful. –: local question not evaluated for the primary judge. The last row is the primary judge on the full evaluation sample of the main comparison, a different image population. Provenance: GPT 6.1 and Qwen3 VL share providers with the Astra programmer and the primary judge; Claude Opus 5.5 wrote the separate Opus arms, which are not in this table, and checked fresh prompts; the Gemini models share a provider with the fresh-prompt generator.

Statistics. Intervals are paired percentile bootstraps over edits (B = 50,000). Among the endpoints reported here, the preregistered contrasts for the main comparison are U3-Opus against U1b, TIME, UCE, UCE+P and U2-Opus, and U1b against TIME, on native joint efficacy (J-eff), native joint generality (J-gen), fresh-scene joint success (F-joint) and fresh-negative specificity (F-spec). Other contrasts in this matrix, including every comparison on subject presence, are exploratory. We also report an exploratory analysis of all 288 unit-pair × endpoint contrasts, with paired sign-flip randomisation p-values and Holm and Benjamini–Hochberg adjustments within each endpoint and over all cells (Table 13).

What Does Program Construction Add?

All four LLM pipelines produced an executed update on all 24 edits. First attempts succeeded on 24/24 (U2-Astra), 20/24 (U2-Opus), 21/24 (U3-Astra) and 23/24 (U3-Opus) edits, and one repair round fixed the rest. On 24/24 edits for U2-Opus and 1/24 for U3-Opus, the fitter selected a zero update; these count as produced by the pipeline.

The preregistered primary method shows no advantage over the strong controls. U3-Opus − U1b is -0.3 [-6.2, +5.9] on native joint efficacy and -6.0 [-10.3, -1.6] on native joint generality. Against TIME it is +3.1 [-7.3, +13.2] on native joint efficacy, and against random search, which was not in the preregistered list, -0.7 [-9.0, +8.0]. On native joint efficacy, U2-Astra (73.6), U3-Opus (74.3), RS (75.0) and

Figure 2. Program construction, shared selection and evaluation. The blue inset shows the U2-Astra cat program: it builds an orthonormal basis HH of empty-prompt and unchanged-prefix states and four directions whose input side annihilates it, DjH=0D_jH=0, so every combination the fitter can choose keeps the invariant. The shared CMA-ES fitter keeps at most four independent directions and evaluates 24 coefficient candidates; U1b directions enter the same fitter, RS samples 24 closed-form candidates directly, and TIME, UCE and UCE+P are closed-form updates evaluated directly. Every candidate is scored by the same objective on probe seeds only. For U3 (dashed), a probe record of direction sweeps and fitted updates is returned to the LLM, which writes one rewritten program that passes through the same fitter. Selected updates are frozen before evaluation, and fresh prompts are written after all units are frozen. The 24-candidate budget applies to this stage only. Thumbnails are illustrative fresh-scene samples of the U2-Astra cat edit with their recorded CLIP and VLM joint verdicts; probe-seed tiles are schematic.

unitupdate directionscoefficientsshares with programs
TIME, UCE UCE+Ppublished closed form UCE with the shared panel as preserved conceptsfixed fixed— protection panel
U1b (hand-written)TIME + 6 further templates + preservation term on the shared panel, split into 4 layer groupsCMA-ES, 24 evaluationspanel, fitter, objective, selection
RS (random search)regulariser, protection weight, strength template/panel/layer-group subsets sampled from the conventional closed-form family programnone: each sample is one candidate, 24 candidatespanel, objective, selection, probe seeds
U2 (one-shot LLM) U3 (probe + rewrite)second program, after the probing recordCMA-ES, 24 evaluations CMA-ES, 24 evaluations— —

Table 1. Units in the main comparison. All fitted units share the protection panel, the candidate-evaluation budget, the fitting score and the selection rule; RS draws conventional closed-form candidates directly instead of searching coefficients of returned directions.

CLIP relative match (ViT-bigG-14, TIMED convention)VLM joint (Qwen3.8-27B: subject present ∧ matches target description)
native TIMED promptsefficacy, generality, specificityjoint efficacy, joint generality
fresh-scene promptsgenerality; specificity on fresh negativesfresh-scene joint success; subject presence

Table 2. The two kinds of score on the two prompt populations.

U1b (74.7) are close, well above UCE (40.6). U2-Opus is lower (59.7).

One-shot Astra programs add fresh-scene joint success. What is measured here is the whole program package: the update geometry the program builds, together with the contexts it writes for itself. U2-Astra − U1b is +10.0 [+4.7, +15.4] on fresh-scene joint success, with no detectable difference in CLIP generality (-0.6 [-7.3, +6.4]; Figure 1). In absolute terms, U2-Astra reaches 37.7% against 27.7% for U1b and 40.4% for the unaudited model drawing the target prompt. The gain is a per-edit pattern: it is positive on 18 edits and negative on 4. It is exploratory: BH-adjusted across all 288 contrasts p = 0.007, Holm within its endpoint 0.051 (at the boundary), and it does not survive Holm over all cells (0.343).

Fresh-scene effects across evaluators

On a fixed 144-image sample (two fresh-scene slots per edit shared by the three units; Appendix § 17), all five additional evaluators give a positive full estimate for U2-Astra versus U1b and a negative estimate for U3-Astra versus U2-Astra. Under local, four construction estimates are smaller and GPT’s is unchanged; all local intervals include zero. The panel supports the full direction on this sample, while local attribute gains remain uncertain. An earlier GPT evaluation that asked both questions in one call, on a six-seed sample of all units, gave smaller construction and rewrite contrasts (+1.9 and -1.9 points). Appendix § 16 reports the protocol differences and the shared-image comparison, in which separate question calls did not restore the construction gain.

The clearest signal is for Astra on fresh scenes. On TIMED’s native positives, the same contrast is +1.0 [-4.0, +6.1]. For one-shot Opus programs it is +2.7 [-4.9, +9.5]. Against random search, the strongest conventional control on this endpoint, U2-Astra’s difference is +5.6 [0.0, +11.5], an interval whose lower end is zero. Against TIME it is +9.7 [+3.6, +15.7].

Where the gain sits. Table 5 splits the contrast by subject type and by judge question. On the 18 object edits, joint success rises by +7.2 [+1.7, +12.7]. There the difference sits in the target-description question (+6.7 [+2.1, +11.3]), with no difference in subject presence (+0.2 [-9.6, +9.0]). The 6 person and brand edits show a larger gain in both questions (joint +18.4 [+7.3, +29.5]) and contribute 46% of the summed per-edit gain. The result does not depend on the judge’s cut-off: from 0.3 to 0.9, and with a soft score, every interval excludes zero (+5.6 [+1.6, +9.3] at 0.9; Table 5). The gain persists without literal scene-word overlap (Appendix § 10).

endpointall (24)object (18)other (6)
VLM joint success+10.0 [+4.7, +15.4]+7.2 [+1.7, +12.7]+18.4 [+7.3, +29.5]
VLM subject present+4.8 [−4.8, +14.4]+0.2 [−9.6, +9.0]+18.4 [−2.4, +42.4]
VLM matches target description+8.9 [+4.2, +14.1]+6.7 [+2.1, +11.3]+15.6 [+3.5, +28.5]
CLIP relative match−0.6 [−7.3, +6.4]−1.0 [−6.8, +4.1]+0.7 [−19.4, +24.7]
CLIP success ∧\wedge subject+3.4 [−4.4, +11.1]−1.0 [−9.1, +6.8]+16.7 [+1.4, +31.9]
threshold0.30.40.50.60.70.80.9soft
gain+9.2+8.2+10.0+8.9+8.5+7.4+5.6+7.7
95% CI[+3.3, +15.5][+2.5, +14.2][+4.7, +15.4][+4.0, +13.7][+3.6, +13.5][+3.2, +11.5][+1.6, +9.3][+3.0, +12.4]

Table 5. U2-Astra − U1b on fresh-scene positives by subject type and judge question (top) and by judge threshold (bottom, all 24 edits, VLM joint success; “soft” = mean of min⁡(psubject,ptarget)\min(p_{\mathrm{subject}}, p_{\mathrm{target}})). Object: the 18 object edits; other: 6 person/brand edits.

Specificity has to be read with efficacy. On fresh negatives U2-Astra is more specific than U1b (+5.9 [+2.5, +9.3]) at similar CLIP generality; U2-Opus is too (+7.2 [+3.7, +11.1]) but edits less, its CLIP generality lower by -17.9 points.

Does Score-Guided Refinement Preserve the Gain?

CLIP gains do not reliably track task gains. For Astra, rewriting raises fresh-scene CLIP relative match by +6.2 [+1.3, +11.6] while changing joint success by -9.7 [−16.6, −2.9]. All five additional evaluators give a negative full rewrite estimate on the audit sample, four with intervals excluding zero. The Opus rewrite and the hand-written extension also raise CLIP, while their primary-judge joint-success differences remain inconclusive. These units therefore improve on the selection proxy without consistent improvement on the task check.

For Opus, rewriting changes CLIP generality by +17.6 [+5.4, +31.9]; its joint success changes by +0.4 [-6.8, +7.5] (Fig. 1). Subject presence changes by -8.6 [-16.1, -1.6] (Opus) and -8.0 [-17.2, +1.2] (Astra; F-subj in Table 3). Fresh-negative specificity also falls: -5.5 [-11.3, 0.0] and -6.2 [-11.5, -1.6]. The Opus unit was preregistered on J-eff, J-gen, F-joint and F-spec; F-subj and F-CLIP, and the Astra arm, are exploratory. Under Benjamini–Hochberg over all cells, the Astra joint-success decline has adjusted pp 0.036 and the Opus CLIP rise 0.033. On native joint efficacy, the Opus rewrite’s point estimate is higher (+14.6 [-0.7, +31.9]), largely because four U2-Opus edits with zero updates receive non-zero updates. Without those four edits, the fresh-scene pattern remains (sensitivity analysis): CLIP +10.4, joint -3.1, subject -8.5 [-17.3, -0.5].

Native promptsFresh positivesFresh neg.
J-effJ-genF-jointF-subjF-CLIPF-spec
A Unit means (%)
TIME71.238.128.072.155.171.7
UCE40.624.521.569.646.973.0
UCE+P39.623.820.871.540.386.8
U1b74.748.327.764.081.278.5
RS75.043.532.070.071.477.1
U2-Astra73.649.437.768.880.684.5
U2-Opus59.739.430.470.563.385.8
U3-Astra73.344.828.060.886.878.2
U3-Opus74.342.430.861.980.980.3
target ref.78.857.140.475.379.6–
B Paired differences (pp)
U3-Opus − U1b ⋆−0.3−6.0+3.1−2.1−0.3+1.7
[−6.2, +5.9][−10.3, −1.6][−3.0, +8.6][−10.4, +5.7][−8.2, +6.5][−2.4, +5.8]
U3-Opus − TIME ⋆+3.1+4.2+2.9−10.2+25.8+8.6
[−7.3, +13.2][−4.4, +13.3][−4.1, +9.8][−17.0, −4.1][+13.5, +37.8][+3.3, +13.8]
U3-Opus − RS−0.7−1.2−1.2−8.1+9.5+3.1
[−9.0, +8.0][−8.4, +6.4][−8.2, +4.8][−15.6, −0.7][+1.1, +18.8][−2.0, +7.9]
U2-Astra − U1b−1.0+1.0+10.0+4.8−0.6+5.9
[−10.4, +7.3][−4.0, +6.1][+4.7, +15.4][−4.8, +14.4][−7.3, +6.4][+2.5, +9.3]
U2-Opus − U1b−14.9−9.0+2.7+6.5−17.9+7.2
[−30.9, −0.7][−18.4, −0.1][−4.9, +9.5][−1.6, +15.1][−34.0, −3.1][+3.7, +11.1]
U2-Astra − RS−1.4+5.8+5.6−1.2+9.2+7.3
[−11.1, +7.6][+0.6, +11.0][0.0, +11.5][−8.7, +6.4][+0.8, +18.1][+2.7, +12.0]
U3-Opus − U2-Opus ⋆+14.6+3.0+0.4−8.6+17.6−5.5
[−0.7, +31.9][−6.4, +12.8][−6.8, +7.5][−16.1, −1.6][+5.4, +31.9][−11.3, 0.0]
U3-Astra − U2-Astra−0.3−4.6−9.7−8.0+6.2−6.2
[−9.4, +8.0][−9.9, +0.8][−16.6, −2.9][−17.2, +1.2][+1.3, +11.6][−11.5, −1.6]
U1b − TIME ⋆+3.5+10.2−0.3−8.2+26.0+6.9
[−6.2, +12.8][+3.3, +17.5][−5.7, +5.3][−16.1, −1.0][+17.1, +36.3][+2.3, +11.8]

Table 3. Controlled comparison on 24 pre-drawn TIMED test edits. A: unit means (%). B: paired differences in percentage points with 95% paired bootstrap intervals over edits (50,000 resamples, unadjusted). Bold: the rounded interval excludes zero. ⋆: unit pair in the preregistration for the main comparison, whose preregistered endpoints are J-eff, J-gen, F-joint and F-spec; F-subj and F-CLIP are exploratory for every pair. J: native joint (VLM subject present ∧ matches target description); F-joint: fresh-scene joint success; F-subj: fresh-scene subject presence; F-CLIP: fresh-scene CLIP relative-match generality; F-spec: CLIP specificity on fresh negatives. LLM costs are in Table 11; all 288 unit-pair contrasts are in Table 13.

Access to the one-shot directions did not preserve the gain. Source-code inspection finds references to U2-Astra library directions in 21/24 Astra rewrites and 19/24 Opus rewrites. Several rewrites also add an explicit subject-identity direction, a response to the subject loss visible on the probe sheets. Rewrites do more than recombine the library: the median share of the final U3-Astra update outside the library’s span is 0.51. After fitting and selection, U3-Astra’s fresh joint success is close to the hand-written baseline (U3-Astra − U1b +0.3 [-6.4, +6.8]). A code reference does not show that a direction was returned among the four fitted ones or kept a non-zero weight, and our design does not separate the rewrite from the CLIP-guided fitting that follows it.

The same direction in a hand-written extension. U1b is TIME extended with six further augmentation templates, a preservation term on the shared panel, a four-group split and CLIP-guided fitting; its panel contains LLM-proposed prompts, and it contains no generated program. Relative to TIME it also lowers fresh subject presence by -8.2 [-16.1, -1.0], while its native joint generality is higher (+10.2 [+3.3, +17.5]). The extension bundles templates, panel, grouping and fitting together, so these outcomes belong to the extension as a whole and not to fitting alone. Across the nine evaluated units, fresh CLIP generality and fresh subject presence are negatively associated (Pearson r = -0.78; descriptive). The direction matches the rewrite result even though the fitting objective already contains a CLIP subject term (§2.4).

A conventional-editor follow-up. In a preregistered follow-up, we compared a rule-based extension of TIME with TIME on the 23 object edits among 40 further TIMED test edits outside the 24-edit main comparison. The extension uses U1b’s regularised TIME construction, grouping, fitter, budget and selection rule, with a protection panel generated by fixed rules only. Its CLIP generality advantage, +21.4 [+14.9, +28.6], is attenuated by 21.7 [13.5, 30.6] once subject presence is required, and all 7/7 preregistered contrasts pass Holm (Table 9). This endpoint combines CLIP success with a subject check rather than the target-matching question used in the main comparison. Two raters also found subject-absent images among CLIP successes (weighted 19.86% and 18.25%); Appendix § 15 reports their labels, agreement and method contrasts.

Figure 3. Fixed-rule examples across the construction effect. Rows are ranks 24, 12 and 1 of the 24 edits ordered by the U2-Astra − U1b gain in fresh-scene joint success, the middle row being the lower of the two central values (+6.2, +10.4). The gain printed above each row is the edit-level average over all of that edit’s fresh-scene images, not a score for the displayed image. Each row uses one prompt and seed across the three units, taken from the frozen audit sample: "Michael Jackson singing into a vintage microphone in a recording studio", "A bed inside a cozy cabin" and "A watch resting on a velvet pillow inside a display case".

Related Work

Weight-space programming and program generation from feedback. LLMs have written weight edits for reasoning models ([27] (Hu et al. 2026)), for robot policies ([33]), for scoring networks in solvers ([4]), and as fixed reusable editors that update predictor weights from experimental feedback (Li et al., 2026). Gains can come from cheap rules rather than from what the LLM wrote ([4]), and successful fixes are often precise and local ([33]).

Figure 4. Conventional-editor follow-up on 23 further TIMED object edits: the rule-based editor’s per-edit CLIP-generality advantage over TIME, before and after requiring subject presence. Each line is one edit, darker where the subject-filtered difference is zero or below; black: mean and 95% paired bootstrap interval, printed under each column.

Taghia [26] has an LLM edit diffusion weights directly to erase a style, with UCE, ESD and budget-matched random search as controls, and reports a benefit from feedback. More broadly, WSP belongs to a line in which an LLM proposes programs or prompts and an evaluator guides their selection ([24, 31] (Ma et al. 2023)). Here a CLIP proxy guides the search, so we evaluate the programs both by that score and by a separate task check, to see what construction and score-guided rewriting contribute in targeted attribute edits where the subject must survive.

Diffusion editing and constrained updates. TIME ([21]) and UCE ([9]) edit cross-attention projections in closed form, and REFACT ([2]) edits the text encoder. Null-space preservation has been studied both for language-model editing ([8]) and for diffusion concept erasure: SPEED (Li et al., 2025) constrains start-of-text and null-text states explicitly, and ACE ([29]) applies null-space constraints to cross-attention updates.

Evaluation of concept edits and proxy feedback. Relative CLIP preference for a target over a source description can count inadequate outputs as successful edits. CAKE (Gu et al., 2024) replaces this rule with prompt-specific absolute CLIP thresholds calibrated on unedited target-prompt generations and validates the criterion against VLM and human labels; EMCID ([30]) measures source forgetting and destination acquisition through changes in ImageNet-classifier confidence; and for erasure, EraseBench ([1]) and HUB ([20]) use object- or entity-level visual questions to measure collateral loss of non-target content. CLIP-style scores have documented weaknesses in attribute binding and compositional matching [32, 28]; TIFA and DSG decompose text–image evaluation into questions [6] (Hu et al. 2023), GenEval checks object-level properties [11], and VQAScore scores whole-prompt alignment [17]; and unedited generation already drops subjects [5]. These studies motivate the separate measurements of subject presence and target-description match used in our controlled comparison. Optimising a proxy past the range where it tracks the goal can degrade the goal [19, 25, 10], LLMs iterating on feedback hack in-context rewards [22 b], and reward fine-tuning of diffusion models over-optimises [3, 7, 34]; our rewrite results show a corresponding divergence between the selection score and the task check at the level of weight programs.

Discussion and Conclusion

Construction and score-guided revision behave differently. In this comparison, Astra’s one-shot programs achieved higher fresh-scene joint success than the shared-panel baseline, and their code shares one structural pattern: hard projection onto the complement of a protected text-state subspace. The rewrite stage often referenced these direction sets. Yet the resulting Astra updates lost the measured gain while their CLIP scores increased. The hand-written extension under CLIP-guided fitting likewise raised CLIP without a detectable fresh-scene joint-success gain. Access to useful program components was therefore not enough for a CLIP-guided rewrite-and-selection pipeline to keep their task benefit.

The programs’ shared preservation structure motivates a component-level question. Our implemented transplant into the hand-written extension projected both the regression inputs and the target residuals. Matrix checks on the two fixed test edits confirm that this two-sided variant differs materially from the registered operator. It changed fresh-scene joint success by +1.0 [-3.6, +5.6] and moved native joint generality by -7.2 points; it did not test the registered input-only transplant, and the complete program’s gain remains unattributed to an individual component (Appendix § 14). Two design questions follow: how to keep useful constraints when directions are modified and combined, and how to select updates using direct checks of subject presence and target-description match rather than the CLIP similarity the fitter already used.

Scope. We study Stable Diffusion 1.4, TIMED, one primary VLM judge (with five further evaluators on a subsample) and one program per edit and model, with a single rewrite round. We do not compare against published null-space editors such as SPEED or ACE. The human labels address subject presence and the local attribute question; we report their mixed method-level readings in Appendix § 15.

Conclusion. LLM-written programs can construct useful diffusion updates under a shared fitting protocol. In this study, Astra’s one-shot programs improved judged fresh-scene target fulfilment, while score-guided rewriting increased CLIP without retaining that gain. The evaluator panel and local checks define the scope of this observation. Understanding WSP therefore requires examining both what the generated program constructs and what changes after score-guided revision.

References

Impact Statement

This work studies how language models edit the weights of a text-to-image model and how such edits are evaluated. Weight editing can be used to change what a deployed model depicts; our results argue for evaluating such edits with task-level checks of the edited subject rather than a single proxy score. We will release programs, prompts and per-image results, but not edited model weights.

LLM use. LLMs are the object of study: Astra and Opus write programs, Gemini writes fresh prompts, and Qwen is the judge. LLM assistants were also used for code review and drafting. All numbers are rendered from the result files, which will be released.

Statistics and additional analyses

Resampling. Contrasts in the main comparison and the conventional-editor follow-up are paired at the edit level. For each edit, we compute a unit’s endpoint as the mean over its prompts and 12 seeds and take the difference between two units. We then resample the 24 (main comparison) or 23 (follow-up object stratum) edit-level differences with replacement, B=50,000B = 50{,}000 times. The random seed is derived from the contrast’s name, so each interval is reproducible on its own. Intervals are percentile intervals, reported to one decimal because endpoints lie on a coarse grid at this nn.

Multiplicity in the exploratory matrix (Table 13) uses paired sign-flip randomisation p-values with 200,000 random sign vectors. For the headline contrast, the exact enumeration over all 2242^{24} sign vectors agrees to the third decimal. The matrix covers every pair among the nine evaluated units on eight endpoints: native joint efficacy and generality, fresh joint success, fresh CLIP generality, fresh-negative specificity, fresh subject presence, and native CLIP efficacy and generality. Holm and Benjamini–Hochberg adjustments are reported within each endpoint (36 contrasts) and over all cells. The follow-up’s preregistered family uses paired bootstrap p-values with Holm, as preregistered.

Some analyses use other resampling schemes. The development-panel comparison (Table 6) has n=16n = 16 edits. The prompt-overlap analysis resamples edits with their prompts kept together (20,000 resamples, prompt-weighted means). The judge-selection interval resamples image clusters. Every number in this paper is rendered from the result files by one script, to be released with them, which also lists, for each number, the result key, endpoint, prompt population, contrast and sample.

TIMEU1bU3-OpusU3-Opus − TIME (paired, 95% CI)
development panel (16 self-built edits)41.166.166.7+25.5 [+11.5, +40.6]
TIMED test (24 pre-drawn edits, main comparison)71.274.774.3+3.1 [-7.3, +13.2]

Table 6. Development panel vs TIMED (native joint efficacy, VLM, %). Both rows use the same judge (Qwen3.8-27B, v2 question) and the same paired edit-level bootstrap; the two panels contain different edits, so the rows are not paired with each other.

Prompt overlap. A program writing many scene templates could share scene words with the fresh prompts. On the 38 fresh prompts (18 edits) with no literal scene-word overlap with the matching U2-Astra program the gain is +11.0 [+1.4, +20.2], against +9.3 [+3.3, +16.0] where there is overlap; Opus programs overlap more (0.26 vs 0.22) and show no clear gain (Figure 5). Lexical overlap is therefore an unlikely sole explanation, though semantic similarity and richer contexts remain.

Figure 5. Prompt overlap and the construction gain. (a) Per fresh prompt, the U2-Astra − U1b gain in fresh-scene joint success against the share of the prompt’s scene words that occur in the U2-Astra program text; horizontal jitter is for visibility. (b) Mean gain on prompts with no and with some literal overlap, with edit-cluster bootstrap intervals; the two groups can share edits.

Unit means. Figure 6 places every unit of the main comparison by its mean fresh-scene CLIP relative match and joint success, including the conventional controls and the target-prompt reference.

Figure 6. Program construction and score-guided rewriting on fresh scenes. Points are mean scores over the 24 TIMED test edits. The blue solid arrow compares U2-Astra with U1b; orange dashed arrows compare U3 with U2 within each model; the grey arrow compares the hand-written extension U1b with TIME. Marker shape marks the model (circle Astra, triangle Opus). Joint success requires subject presence and whole-image target-description matching; the target-prompt reference is the unedited model drawing the target prompt. CLIP increases after rewriting for both models; joint success decreases for Astra and remains inconclusive for Opus. Paired differences and their 95% edit-bootstrap intervals are in Table 3. Figure 1 shows the same contrasts as paired changes.

Development panels. On our 16 self-built development edits, U3-Opus led TIME by +25.5 [+11.5, +40.6] in native joint efficacy. On the pre-drawn TIMED edits, the lead is +3.1 [−7.3, +13.2], and TIME rises from 41.1 to 71.2 (Table 6).

The program-writing prompts and program codes

Both templates are reproduced verbatim from the code that sent them. Fields in braces are filled per edit: {old}, {new} are the edit pair; {layers} lists the 32 editable matrices with their shapes and layer groups; in the rewrite template {subject}, {subj_ref}, {drop_T}, {tnorm}, {panel} carry the per-edit subject string, the two fitter tolerances, the TIME update norm and the protection panel, and {schemes}, {singletons}, {prev_code}, {prev_notes} carry the probing record and the model’s own previous program.

Figure Box 1. Program–fitter responsibilities in the Astra cat example. The program builds a protected subspace HH and four ranked directions that all satisfy DjH=0D_jH = 0; the fitter only chooses coefficients, so the invariant holds for every update it can select.

Verbatim B.1: the one-shot program-writing prompt.

You will write a Python program that edits the weights of Stable Diffusion v1.4 (a text-to-image model).

## The edit request
old prompt: "{old}"
new prompt: "{new}"
Goal: after the edit, generating from "{old}" (and from other prompts that mention the same subject, e.g. in
different scenes or styles) should produce images that match "{new}". Prompts about OTHER subjects must keep
producing what the unedited model produces. You do not get any evaluation prompts or scores.

## Where you may edit
The 16 cross-attention layers of the UNet. Each has a key projection W_K and a value projection W_V that map the
CLIP text-encoder token states (77 tokens x 768 dims, last hidden layer) to keys/values. These 32 matrices are
named "v0"..."v15" (to_V) and "k0"..."k15" (to_K). Layer list (name, module, shape, group):
{layers}

## Tools available inside your program (argument `ctx`, read-only)
- ctx.old, ctx.new: the two prompts.
- ctx.layers: list of dicts with keys name, module, proj, shape, group.
- ctx.W0(name): the original weight (torch.float32, CPU, shape [out, 768]).
- ctx.encode(list_of_texts) -> torch.float32 tensor [n, 77, 768]: text-encoder states for ANY prompts you write
  (max 64 prompts). Positions beyond the end-of-text token hold padding states.
- ctx.tokens(text) -> list of token strings (with <|startoftext|> and <|endoftext|>).
- ctx.align(old_text, new_text) -> list of 77 indices: for each position of old_text, the matching position in
  new_text (the token alignment used by the TIME method).
Allowed imports: torch, numpy, math, re, itertools, functools, collections. No files, network or other modules.
Your program must finish within 60 seconds on CPU.

## Reference (a baseline you should try to beat, not the required answer)
TIME (Orgad et al. 2023) solves, for each matrix W, min_W' sum_t ||W' k_t − W v*_t||^2 + lambda ||W' − W||^2 where
k_t are the old prompt's token states and v*_t the aligned new prompt's token states (summed over the prompt and
3 augmented variants "A photo of / An image of / A picture of ..."), giving
W' = (lambda W + sum_t W v*_t k_t^T)(lambda I + sum_t k_t k_t^T)^(-1), lambda = 0.1. Its known weakness: other
subjects change too (specificity about 68-76%), and generalisation to related prompts is incomplete (about
↪ 64-69%).

## What happens with your output
Your directions are ranked. The first 4 are normalised (Frobenius norm over all 32 matrices) and orthogonalised in
your order. A numerical optimiser (CMA-ES) then chooses coefficients beta for these 4 directions, so the final
↪ edit is
dW = sum_j beta_j * direction_j, scoring candidates by CLIP similarity of images generated from "{old}" to "{new}"
versus "{old}", minus a penalty for changing images of a fixed set of protection prompts. Directions only need the
right shape and sign/pattern; their scale does not matter. You may also suggest up to 4 protection prompts
↪ (prompts
about other subjects whose images should not change) and initial coefficients for your first 4 directions AS YOU
RETURNED THEM (the edit sum_j c_j * direction_j, before normalisation); it is projected onto the basis as a
↪ starting
point for the optimiser.

## Output
Reply with your reasoning briefly, then ONE Python code block defining:

def build_directions(ctx):
    return {"directions": [dict(name -> torch tensor of that layer's shape), ...],  # ranked, up to 8
            "protect_prompts": [str, ...],                              # up to 4
            "init_coeffs": [float, ...] or None,                        # optional, for the first 4
            "notes": "one-paragraph description of the construction"}

Verbatim B.2: the probe-and-rewrite prompt, appended to B.1 for U3.

## Experiment record from forward probing (probe seeds only; no evaluation data)
You previously wrote the program below for this edit. We applied it (directions fitted by the optimiser), and also
measured the TIME baseline, a hand-written conventional improvement "U1b" (TIME + preservation term on the
↪ protection
panel + more augmentation templates) and the one-shot programs of two LLMs (U2-astra, U2-opus); yours is {own}.
Images: attached labelled contact sheets (unedited model; each scheme on the old prompt, augmented variants and
↪ the
protection panel; each library direction alone at three doses).
Metrics (CLIP ViT-L/14): eff = gain in [sim(image,"{new}") - sim(image,"{old}")] over the unedited model (higher =
↪ edit
applied); subj = sim(image, "a photo of {subject}") (the optimiser only accepts subj >= subj_ref - 0.01, subj_ref
↪ {subj_ref:.4f} being the value for images of "{new}" from the unedited model; lower means the subject is being
↪ replaced);
drop = damage on the protection panel (the optimiser only accepts drop <= TIME's drop = {drop_T:.4f} + 0.01). The
protection panel is fixed for this experiment (P1...P{n_panel} in the sets: {panel}); protect_prompts you return
↪ are recorded but not used.
norm = ||dW_TIME||_F (Frobenius, all 32 matrices), also available as ctx.time_norm. Dose d of a
↪ library
direction = coefficient d * {tnorm:.2f} on the unit-norm ctx.library[name]; init_coeffs are in the same absolute
↪ units
(coefficient of each returned direction as you return it). The optimiser searches coefficients of the
↪ orthonormalised
span of your first 4 independent directions within a Frobenius ball of radius 3 * ||dW_TIME||.

### Whole schemes (as fitted)
{schemes}

### Library directions applied alone (dose = multiple of ||dW_TIME||)
{singletons}

These {n_lib} library directions are available inside your program as ctx.library[name] (dict layer -> tensor,
norm, holding ALL 32 layer names; layers a direction does not touch are zero tensors): {lib_names}. You may reuse,
↪ combine, or replace them; writing entirely new directions is allowed.

### Your previous program
```python
{prev_code}

Your previous notes: {prev_notes}

Task

Rewrite build_directions(ctx) to do better than all schemes above: apply the edit to "{old}" and related prompts (eff high), keep the subject (subj >= subj_ref) and do not damage other subjects (low drop). Same output format ↪ and rules.

Fresh-scene prompts

Fresh prompts were written after every unit’s update was frozen. Before that, every program was re-executed to record all the text it encodes. Gemini-4-argon then wrote, per edit, four positives that put the subject in a new scene or style, each with its edited version, and eight near-neighbour negatives: other subjects that are similar to the subject or often shown with it, with the same change applied. It was told to avoid templates such as "a photo of". Candidates that matched any recorded program text, TIME/U1b template or panel prompt were discarded. Opus then checked every pair. A positive must depict the subject, and its edited version must change only the attribute. A negative must not be the subject or a kind of it, and its edited version must apply the change to that negative. Rejected candidates were regenerated until each edit had 4 + 8 accepted prompts. The accepted prompts were written once and will be released.

Judge questions

  • Subject, objects: "Is there ⟨subject⟩ in this image? Its colour, material, surface pattern or accessories may differ from usual, and it may be drawn in any style, but it must still be ⟨subject⟩, not a different kind of object."

  • Subject, people and characters: "Is ⟨subject⟩ (the actual person or character, not someone else) in this image? Hair colour, clothing or accessories may differ, and the depiction may be stylised."

  • Target: "Does this image match the description ⟨target prompt⟩?" This is a check on the whole image, so joint success means subject present and the image as a whole matches the target description; it does not verify that the attribute is bound to the subject.

The judge is Qwen3.8-27B, and a "yes" probability of at least 0.5 counts as yes. It was selected on a separate set of 274 generic multi-predicate questions (Table 8). The subject question above replaced a first wording after a 48-image check found that wording too strict on recoloured subjects. Neither question above was re-run on the 274 selection questions.

Transplanting the hard constraint

The transplant replaces the shared map of the hand-written extension, M=(E−K)B−1M = (E - K)B^{-1}, with a constrained solve that removes a protected text-state subspace HH, built the way the Astra programs build it (the empty prompt’s states and the unchanged causal prefixes of the training pairs). Here EE, KK and PΣP_{\Sigma} are the sums of target–source cross-covariances, source covariances and preservation-state covariances, B=λI+K+αPΣB = \lambda I + K + \alpha P_{\Sigma} and BΠ=λI+Π(K+αPΣ)ΠB_{\Pi} = \lambda I + \Pi(K + \alpha P_{\Sigma})\Pi. The registered operator projects the regression inputs and the solution, Mreg=(E−K)ΠBΠ−1ΠM_{\mathrm{reg}} = (E - K)\Pi B_{\Pi}^{-1}\Pi with Π=I−HH⊤\Pi= I - HH^{\top}. The implementation also projected the target residuals, which yields Mimpl=ΠMregM_{\mathrm{impl}} = \Pi M_{\mathrm{reg}}: an additional left projection that removes the residual’s component in HH. Both operators annihilate HH and both reduce to the unconstrained map when HH is empty, which is why the run’s validity checks did not separate them; the final right projection is redundant in exact arithmetic for both constrained solves. The reported run is therefore the stricter variant. On the primary judge it changed fresh-scene joint success by +1.0 [-3.6, +5.6] (sign-flip p = 0.72) and native-prompt joint generality by -7.2 [-11.0, -3.3] (p = 0.002). On the bed and cat edits used in the original validity checks, fixed before any difference was seen, the relative Frobenius differences between the two sets of weight updates were 73 and 71%, measured as the norm of their difference divided by the registered update’s norm across all editable matrices. In the shared-map space, the extra output-side projection removed 47 and 45% of the squared Frobenius norm of MregM_{\mathrm{reg}}. The implemented maps had norms 0.76 and 0.78 times those of the unconstrained maps, compared with 1.04 and 1.05 for the registered maps. These matrix checks establish a material change in the update construction. The image-level results reported here belong to the two-sided variant; the input-only transplant remains unevaluated.

Development-stage protection-component study. This study transplanted structural protection components into a conventional closed-form editor on the 16 development edits. The preregistered curve statistic was found to be biased (it is non-zero when a curve is compared with itself), so the preregistered criterion is reported as not met. A post-hoc same-α\alpha comparison suggests a protection gain of about 3 points for one variant with efficacy not distinguishable; it is not used for any claim. Its prefix component was a soft penalty, not the hard projection the Astra programs use, so it does not test that construction.

Conventional-editor follow-up: tests and human labels

Raters saw each image with the subject name and the attribute word only, never the target sentence, so their second question corresponds to local and not to the main endpoint. Two raters labelled the same stratified sample from the conventional-editor follow-up; the first did so before the rubric was frozen. Agreement on questions both answered definitely is κ=0.647\kappa= 0.647 (110 images) for subject presence and 0.735 (113) for the attribute, 0.600 and 0.711 when weighted by inclusion probability. Among CLIP-successful images with definite human subject judgements, the inverse-inclusion-probability-weighted absence rates are 19.86% and 18.25%.

One rater also labelled the 144-image subsample of Table 4. On it, with unsure counted as unsuccessful, the human local reading gives +4.2 [-10.4, +18.8] for U2-Astra − U1b and +0.0 [-18.8, +18.8] for U3-Astra − U2-Astra, against +20.8 [+4.2, +37.5] and -20.8 [-37.5, -4.2] from the primary judge’s full reading on the same images. These are different target questions, and we report them for completeness rather than as a test of the judge. Holding the primary judge’s target answer fixed and substituting the rater’s subject answer where it is definite gives +16.7 (+12.5 and -10.4 if the rater’s unsure answers count as absent): the subject-question disagreement accounts for part of this sample’s gap and does not remove either sign. This mixes two instruments and is a sensitivity analysis, not a corrected estimate. On subject presence alone, where the two questions coincide in intent but not in wording, the judge calls the subject absent where the rater sees it on 12, 9 and 13 of 44 images for U1b, U2-Astra and U3-Astra.

contrastmean [95% CI] (pp)pHolm prejected
CLIP generality advantage, rule editor − TIME+21.4 [+14.9, +28.6]2.0e-051.4e-04yes
subject presence, rule editor − TIME-14.5 [-24.2, -5.6]3.8e-041.5e-03yes
attenuation, rule editor − TIME+21.7 [+13.5, +30.6]2.0e-051.4e-04yes
attenuation, TIME − target-prompt ref.-10.4 [-16.8, -4.5]1.0e-045.0e-04yes
attenuation, rule editor − target-prompt ref.+11.3 [+4.0, +19.1]1.6e-034.9e-03yes
subject-filtered CLIP success, TIME − target-prompt ref. (both filtered)-8.6 [-15.1, -1.5]1.9e-021.9e-02yes
subject-filtered CLIP success, rule editor − target-prompt ref. (both filtered)-9.0 [-16.8, -2.2]7.7e-031.5e-02yes

Table 9. Preregistered follow-up contrasts (object stratum, n = 23 edits; Holm, α=0.05\alpha= 0.05) p-values are two-sided paired-bootstrap values (B = 50000) floored at 1/(B+1)1/(B+1).

Figure 7. Conventional-editor follow-up on 23 object edits: the rule-based editor against TIME (a) and against the target-prompt reference (b). Each line is one edit, darker where the subject-filtered difference is zero or below; black: mean and 95% paired bootstrap interval, printed under each column.

armhuman: subject presenthuman: attribute on subjecthuman: bothjudge: subject presentjudge-absent → human-present
unedited98.318.716.283.59/16
target-prompt ref.80.076.660.967.37/16
TIME90.746.146.770.59/16
rule editor64.355.340.158.64/16

Table 10. Human annotations for the conventional-editor follow-up (first of two raters, the second in Appendix § 15; 128 images stratified by arm × judge verdict from the follow-up’s object-stratum fresh-scene positives; reference, not ground truth) Rates are inclusion-probability weighted to the follow-up’s object-stratum fresh-positive images. ’Unsure’ answers are dropped from the denominator of that question only, so the columns have different denominators and a joint rate can exceed a marginal one. The judge column is the judge on the same weighted population.

Measurement sensitivity: an earlier second evaluator

Before the panel of § 4, gpt-6.1-sol re-evaluated a six-seed sample of all seven LLM and conventional units, asking both questions in one call. On that sample the primary judge gives +12.2 [+6.1, +18.8] for U2-Astra − U1b, -9.4 [-16.7, -2.1] for U3-Astra − U2-Astra, +1.6 [-6.2, +8.9] for U3-Opus − U2-Opus and -2.3 [-8.9, +4.2] for U1b − TIME; GPT gives +1.9 [-3.1, +6.4], -1.9 [-9.5, +5.9], +4.9 [-3.8, +14.1] and +8.0 [+2.3, +13.9]. The two evaluations differ in sample, in call structure and in image encoding. On the 78 images (21 edits) they share, GPT’s U2-Astra − U1b is +0.0+0.0 with both questions in one call and +0.0+0.0 with separate calls, and its U3-Astra − U2-Astra is +4.8+4.8 and −4.8-4.8. Splitting the questions therefore did not restore the construction gain on the shared images; the data do not identify which difference between the two evaluations explains the rest. The point estimates above are recomputed from the per-image answers, which will be released; the intervals are those of the run’s statistics file, whose bootstrap seed was not recorded.

Run configuration and artifacts

Evaluator sample. For each edit, we sampled two of its four fresh-scene prompts without replacement and one of the twelve seeds for each selected prompt, using an edit-specific deterministic random generator. U1b, U2-Astra and U3-Astra share these slots, giving 144 images. Selection did not condition on CLIP or judge outcomes; the sample was drawn once and kept after the primary-judge scores on it were seen.

Run configuration. Images are generated with Stable Diffusion v1.4 at 50 sampling steps and guidance 7.5; probe seeds are 1000–1003 and evaluation seeds 100–111. The CMA-ES fitter uses population 8, seed 0, initial step 0.3∥ΔWTIME∥0.3\lVert\Delta W_{\mathrm{TIME}}\rVert and a search ball of radius 3∥ΔWTIME∥3\lVert\Delta W_{\mathrm{TIME}}\rVert; anchors count towards the 24-candidate budget. U1b is TIME with six extra prompt prefixes, a preservation term (λ=0.1\lambda= 0.1, α=1.0\alpha= 1.0) and one direction per network group (down, mid, first two up blocks, remaining up blocks), keys and values joined within a group. RS samples λ∼LogUniform⁡[0.01,1]\lambda\sim\operatorname{LogUniform}[0.01, 1], α∼LogUniform⁡[0.1,10]\alpha\sim\operatorname{LogUniform}[0.1, 10] and strength ∼Uniform⁡[0.25,2]\sim\operatorname{Uniform}[0.25, 2], and initially retains each extra template, panel prompt and network group independently with probability 0.5, 0.6 and 0.5, respectively; if no network group is selected, one is sampled uniformly. Both selection tolerances are 0.01. The fitter keeps the first four linearly independent returned directions in rank order; program time limits are given with Table 11.

unitLLM output tokens (all attempts)first-try successmedian program run time (s)candidate evaluations / editmedian fitting wall time / edit (s)
TIME———0—
UCE———0182
UCE+P———0181
U1b———242740
RS———24990
U2-Astra274k24/2454242087
U2-Opus168k20/2499241563
U3-Astra302k21/24124822
U3-Opus181k23/24224833

Table 11. Cost of the main comparison Input tokens and stage hours are per-model and per-stage totals from the run summary: astra 807,486 in / 575,707 out (51 calls); opus 898,858 in / 375,509 out (78 calls); gemini-4-argon 5,976 in / 22,360 out (25 calls). U3 additionally needs the probing record (median 912 images per edit). Whole main-comparison run: 19.5 wall-clock hours on 8 GPUs. Fitting times are wall time under 8-way parallel load. The program-writing prompt asks for completion within 60 s; the runner enforced 120 s (one-shot) and 180 s (rewrite), and program time includes construction and output validation.

Artifact identifiers. In the artifacts, which will be released, E08 denotes the 24-edit main comparison and E10 the conventional-editor follow-up; T02 denotes the judge-selection set, E05b/E06 the development experiments and E09 the development-stage protection-component study. The judge was selected before E05b. The main-comparison preregistration was first drafted on 8 November 2026 and finalized as v0.3 on 9 November 2026; it will be released with the code; besides the endpoints in Table 13 it lists TIMED specificity, validity checks and the panel-dependence contrast, whose results are in the result files to be released.

Table 12. TIMED template errors Four brand→\topizza edits in the local TIMED copy carry ’sushi’ text in gt3/gt4. The conventional-editor follow-up excludes them; the main comparison contains one (Subway). Dropping its two affected prompts changes each unit’s native CLIP generality (%) by: No preregistered contrast changes sign or crosses zero; native joint generality moves by at most 0.3 points when recomputed without the affected prompts.

unitas reportedaffected prompts droppedchange
target-prompt ref.92.692.60.0
TIME63.963.8-0.1
UCE49.250.4+1.2
UCE+P40.541.8+1.3
U1b90.590.8+0.3
RS79.879.6-0.2
U2-Astra85.185.6+0.5
U2-Opus66.966.7-0.2
U3-Astra88.688.7+0.1
U3-Opus87.086.7-0.3

Complete comparison matrix

Table 13 lists every unit pair on all eight endpoints of the main comparison, one block per endpoint.

Table 13. All 288 unit-pair × endpoint contrasts in the main comparison, one block per endpoint. Difference and 95% interval: paired bootstrap over 24 edits. pp: paired sign-flip randomisation (200,000 draws); values below the printed precision are shown as <0.0001 (pp) and <0.001 (adjusted). Holm / BH adjusted within the endpoint (36 contrasts) and over all cells. Bold: global BH < 0.05. ⋆: unit pair and endpoint in the preregistration for the main comparison (their inference follows the preregistration; listed here for completeness).

contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
native joint efficacy
U3-Opus − U3-Astra+1.0 [−5.9, +9.7]0.89391.0001.0001.0000.950
U3-Opus − U2-Opus ⋆+14.6 [−0.7, +31.9]0.11201.0000.2371.0000.189
U3-Opus − U2-Astra+0.7 [−7.3, +9.7]0.94251.0001.0001.0000.980
U3-Opus − RS−0.7 [−9.0, +8.0]0.93811.0001.0001.0000.979
U3-Opus − U1b ⋆−0.3 [−6.2, +5.9]1.00001.0001.0001.0001.000
U3-Opus − UCE+P ⋆+34.7 [+18.4, +51.0]0.00060.0180.0030.1420.003
U3-Opus − UCE ⋆+33.7 [+18.8, +49.0]0.00040.0140.0030.1020.002
U3-Opus − TIME ⋆+3.1 [−7.3, +13.2]0.60831.0000.9521.0000.727
U3-Astra − U2-Opus+13.5 [−3.5, +31.6]0.16501.0000.2971.0000.251
U3-Astra − U2-Astra−0.3 [−9.4, +8.0]1.00001.0001.0001.0001.000
U3-Astra − RS−1.7 [−11.1, +7.3]0.77681.0001.0001.0000.868
U3-Astra − U1b−1.4 [−9.4, +5.9]0.81271.0001.0001.0000.893
U3-Astra − UCE+P+33.7 [+15.3, +51.4]0.00230.0590.0070.4900.008
U3-Astra − UCE+32.6 [+14.9, +49.7]0.00200.0510.0060.4210.008
U3-Astra − TIME+2.1 [−9.0, +13.2]0.76751.0001.0001.0000.863
U2-Opus − U2-Astra−13.9 [−30.9, +2.1]0.13461.0000.2691.0000.222
U2-Opus − RS−15.3 [−28.5, −3.8]0.02630.6310.0731.0000.062
U2-Opus − U1b−14.9 [−30.9, −0.7]0.08241.0000.1861.0000.144
U2-Opus − UCE+P+20.1 [+1.4, +38.5]0.05151.0000.1331.0000.100
U2-Opus − UCE+19.1 [+0.7, +36.5]0.06001.0000.1441.0000.114
U2-Opus − TIME−11.5 [−26.0, +1.7]0.14491.0000.2741.0000.234
U2-Astra − RS−1.4 [−11.1, +7.6]0.83971.0001.0001.0000.909
U2-Astra − U1b−1.0 [−10.4, +7.3]0.89021.0001.0001.0000.950
U2-Astra − UCE+P+34.0 [+18.1, +50.0]0.00080.0220.0030.1710.004
U2-Astra − UCE+33.0 [+16.3, +49.0]0.00120.0320.0040.2650.005
U2-Astra − TIME+2.4 [−8.7, +12.8]0.71711.0001.0001.0000.816
RS − U1b+0.3 [−7.3, +7.6]1.00001.0001.0001.0001.000
RS − UCE+P+35.4 [+21.2, +49.7]0.00010.0040.0010.0340.001
RS − UCE+35.4 [+21.2, +47.9]0.00010.0030.0010.0190.001
RS − TIME+3.8 [−2.1, +10.1]0.28011.0000.4801.0000.384
U1b − UCE+P+35.1 [+18.7, +51.0]0.00080.0220.0030.1750.004
U1b − UCE+34.0 [+18.7, +49.3]0.00050.0160.0030.1240.003
U1b − TIME ⋆+3.5 [−6.2, +12.8]0.53691.0000.8791.0000.661
UCE+P − UCE−1.0 [−6.9, +4.5]0.82591.0001.0001.0000.903
contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
UCE+P − TIME−31.6-31.6 [−42.7-42.7, −20.1-20.1]<0.0001<0.00010.0020.0010.012<0.001<0.001
UCE − TIME−30.6-30.6 [−41.0-41.0, −20.1-20.1]<0.0001<0.00010.0010.0010.007<0.001<0.001
contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
Table 13, block 2 of 8: native joint generality
U3-Opus − U3-Astra−2.4-2.4 [−6.0-6.0, +1.0+1.0]0.21801.0000.2801.0000.315
U3-Opus − U2-Opus ⋆+3.0+3.0 [−6.4-6.4, +12.8+12.8]0.56501.0000.6561.0000.689
U3-Opus − U2-Astra−7.0-7.0 [−12.1-12.1, −1.7-1.7]0.01890.3620.0381.0000.046
U3-Opus − RS−1.2-1.2 [−8.4-8.4, +6.4+6.4]0.78031.0000.8021.0000.868
U3-Opus − U1b ⋆−6.0-6.0 [−10.3-10.3, −1.6-1.6]0.01810.3620.0381.0000.046
U3-Opus − UCE+P ⋆+18.5+18.5 [+7.4+7.4, +29.9+29.9]0.00440.1060.0120.8750.014
U3-Opus − UCE ⋆+17.8+17.8 [+7.8+7.8, +28.3+28.3]0.00310.0800.0100.6270.010
U3-Opus − TIME ⋆+4.2+4.2 [−4.4-4.4, +13.3+13.3]0.37651.0000.4521.0000.488
U3-Astra − U2-Opus+5.4+5.4 [−2.2-2.2, +13.4+13.4]0.20351.0000.2721.0000.300
U3-Astra − U2-Astra−4.6-4.6 [−9.9-9.9, +0.8+0.8]0.11781.0000.1931.0000.197
U3-Astra − RS+1.2+1.2 [−4.7-4.7, +7.4+7.4]0.71621.0000.7581.0000.816
U3-Astra − U1b−3.5-3.5 [−8.4-8.4, +1.2+1.2]0.17741.0000.2561.0000.268
U3-Astra − UCE+P+21.0+21.0 [+10.4+10.4, +32.0+32.0]0.00120.0330.0040.2710.005
U3-Astra − UCE+20.3+20.3 [+10.7+10.7, +30.2+30.2]0.00060.0170.0030.1370.003
U3-Astra − TIME+6.7+6.7 [−1.6-1.6, +15.3+15.3]0.14711.0000.2241.0000.234
U2-Opus − U2-Astra−10.0-10.0 [−19.8-19.8, −0.6-0.6]0.06071.0000.1091.0000.114
U2-Opus − RS−4.2-4.2 [−12.4-12.4, +3.6+3.6]0.34371.0000.4271.0000.456
U2-Opus − U1b−9.0-9.0 [−18.4-18.4, −0.1-0.1]0.07741.0000.1331.0000.138
U2-Opus − UCE+P+15.6+15.6 [+4.6+4.6, +27.2+27.2]0.01530.3210.0341.0000.040
U2-Opus − UCE+14.9+14.9 [+5.2+5.2, +25.2+25.2]0.00760.1750.0201.0000.022
U2-Opus − TIME+1.2+1.2 [−7.7-7.7, +10.1+10.1]0.80161.0000.8021.0000.885
U2-Astra − RS+5.8+5.8 [+0.6+0.6, +11.0+11.0]0.04540.8170.0861.0000.091
U2-Astra − U1b+1.0+1.0 [−4.0-4.0, +6.1+6.1]0.71611.0000.7581.0000.816
U2-Astra − UCE+P+25.6+25.6 [+16.2+16.2, +35.3+35.3]<0.0001<0.00010.001<0.001<0.0010.006<0.001<0.001
U2-Astra − UCE+24.9+24.9 [+16.3+16.3, +33.9+33.9]<0.0001<0.00010.001<0.001<0.0010.007<0.001<0.001
U2-Astra − TIME+11.2+11.2 [+4.6+4.6, +17.9+17.9]0.00420.1050.0120.8450.014
RS − U1b−4.8-4.8 [−11.9-11.9, +1.9+1.9]0.20391.0000.2721.0000.300
RS − UCE+P+19.7+19.7 [+10.3+10.3, +29.4+29.4]0.00070.0190.0030.1530.003
RS − UCE+19.0+19.0 [+10.6+10.6, +27.9+27.9]0.00030.0080.0010.0620.002
RS − TIME+5.4+5.4 [−1.5-1.5, +12.2+12.2]0.14911.0000.2241.0000.236
U1b − UCE+P+24.5+24.5 [+15.3+15.3, +33.4+33.4]0.00010.0030.0010.0220.001
U1b − UCE+23.8+23.8 [+15.8+15.8, +31.7+31.7]<0.0001<0.00010.001<0.001<0.0010.009<0.001<0.001
U1b − TIME ⋆+10.2+10.2 [+3.3+3.3, +17.5+17.5]0.01100.2420.0261.0000.031
UCE+P − UCE−0.7-0.7 [−3.1-3.1, +1.9+1.9]0.64181.0000.7221.0000.757
UCE+P − TIME−14.3-14.3 [−20.2-20.2, −8.7-8.7]0.00010.0030.0010.0240.001
UCE − TIME−13.6-13.6 [−18.5-18.5, −9.2-9.2]<0.0001<0.00010.001<0.001<0.0010.006<0.001<0.001
contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
Table 13, block 3 of 8: fresh joint success
U3-Opus − U3-Astra+2.9+2.9 [−0.8-0.8, +6.4+6.4]0.15471.0000.2781.0000.240
U3-Opus − U2-Opus ⋆+0.4+0.4 [−6.8-6.8, +7.5+7.5]0.92671.0000.9881.0000.971
U3-Opus − U2-Astra−6.9-6.9 [−12.7-12.7, −1.5-1.5]0.02830.7940.1001.0000.065
U3-Opus − RS−1.2-1.2 [−8.2-8.2, +4.8+4.8]0.75781.0000.8531.0000.856
U3-Opus − U1b ⋆+3.1+3.1 [−3.0-3.0, +8.6+8.6]0.33161.0000.4771.0000.442
U3-Opus − UCE+P ⋆+10.0+10.0 [+1.7+1.7, +18.4+18.4]0.03320.8540.1001.0000.071
U3-Opus − UCE ⋆+9.3+9.3 [+1.3+1.3, +17.4+17.4]0.03940.9460.1091.0000.082
U3-Opus − TIME ⋆+2.9+2.9 [−4.1-4.1, +9.8+9.8]0.44971.0000.6231.0000.571
U3-Astra − U2-Opus+2.4+2.4 [−9.5-9.5, +4.5+4.5]0.52871.0000.6561.0000.656
U3-Astra − U2-Astra−9.7-9.7 [−16.6-16.6, −2.9-2.9]0.01390.4180.0721.0000.036
U3-Astra − RS−4.1-4.1 [−10.5-10.5, +1.6+1.6]0.22631.0000.3701.0000.324
U3-Astra − U1b+0.3+0.3 [−6.4-6.4, +6.8+6.8]0.96031.0000.9881.0000.986
U3-Astra − UCE+P+7.1+7.1 [−1.5-1.5, +16.2+16.2]0.14241.0000.2701.0000.232
U3-Astra − UCE+6.4+6.4 [−1.7-1.7, +15.0+15.0]0.16331.0000.2801.0000.250
U3-Astra − TIME+0.0+0.0 [−7.5-7.5, +7.4+7.4]1.00001.0001.0001.0001.000

Table 13 (continued). native joint efficacy

contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
U2-Opus − U2-Astra−7.3 [−15.9, +1.0]0.11891.0000.2381.0000.198
U2-Opus − RS−1.6 [−9.2, +5.7]0.69281.0000.8051.0000.803
U2-Opus − U1b+2.7 [−4.9, +9.5]0.50641.0000.6561.0000.637
U2-Opus − UCE+P+9.5 [+1.3, +17.2]0.03280.8540.1001.0000.071
U2-Opus − UCE+8.9 [+0.7, +16.4]0.04401.0000.1131.0000.089
U2-Opus − TIME+2.4 [−4.7, +9.0]0.52371.0000.6561.0000.653
U2-Astra − RS+5.6 [0.0, +11.5]0.07881.0000.1671.0000.139
U2-Astra − U1b+10.0 [+4.7, +15.4]0.00160.0510.0140.3430.007
U2-Astra − UCE+P+16.8 [+9.5, +24.1]0.00030.0110.0070.0720.002
U2-Astra − UCE+16.1 [+8.8, +23.4]0.00040.0130.0070.0900.002
U2-Astra − TIME+9.7 [+3.6, +15.7]0.00670.2130.0481.0000.020
RS − U1b+4.3 [−2.7, +11.5]0.26081.0000.4081.0000.365
RS − UCE+P+11.2 [+2.9, +19.7]0.01830.5290.0821.0000.046
RS − UCE+10.5 [+2.0, +19.2]0.02860.7940.1001.0000.065
RS − TIME+4.1 [−2.9, +10.9]0.28011.0000.4201.0000.384
U1b − UCE+P+6.9 [+0.1, +13.5]0.06541.0000.1571.0000.121
U1b − UCE+6.2 [−0.1, +12.5]0.07481.0000.1671.0000.136
U1b − TIME ⋆−0.3 [−5.7, +5.3]0.95181.0000.9881.0000.982
UCE+P − UCE−0.7 [−3.0, +1.7]0.63551.0000.7631.0000.753
UCE+P − TIME−7.1 [−10.9, −3.6]0.00130.0440.0140.2840.005
UCE − TIME−6.4 [−11.2, −2.0]0.01340.4170.0721.0000.036

Table 13 (continued, PDF p. 22). fresh joint success

contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
U3-Opus − U3-Astra−5.9 [−14.6, +1.6]0.19821.0000.2351.0000.296
U3-Opus − U2-Opus+17.6 [+5.4, +31.9]0.01230.2210.0231.0000.033
U3-Opus − U2-Astra+0.3 [−7.8, +7.9]0.94961.0000.9681.0000.982
U3-Opus − RS+9.5 [+1.1, +18.8]0.04980.5970.0721.0000.098
U3-Opus − U1b−0.3 [−8.2, +6.5]0.96771.0000.9681.0000.988
U3-Opus − UCE+P+40.6 [+25.0, +54.9]0.00010.004<0.0010.0340.001
U3-Opus − UCE+34.0 [+18.4, +48.4]0.00060.0140.0020.1380.003
U3-Opus − TIME+25.8 [+13.5, +37.8]0.00030.0080.0010.0760.002
U3-Astra − U2-Opus+23.5 [+9.5, +38.7]0.00420.0830.0090.8440.014
U3-Astra − U2-Astra+6.2 [+1.3, +11.6]0.03020.4530.0491.0000.067
U3-Astra − RS+15.5 [+7.0, +24.5]0.00160.0330.0040.3490.007
U3-Astra − U1b+5.6 [−0.6, +13.3]0.14531.0000.1801.0000.234
U3-Astra − UCE+P+46.5 [+34.2, +58.6]<0.0001<0.001<0.0010.001<0.001
U3-Astra − UCE+39.9 [+27.0, +52.3]<0.00010.001<0.0010.008<0.001
U3-Astra − TIME+31.7 [+22.1, +42.3]<0.0001<0.001<0.0010.001<0.001
U2-Opus − U2-Astra−17.3 [−32.9, −3.5]0.03490.4890.0551.0000.074
U2-Opus − RS−8.1 [−20.5, +2.5]0.21131.0000.2381.0000.309
U2-Opus − U1b−17.9 [−34.0, −3.1]0.03650.4890.0551.0000.076
U2-Opus − UCE+P+23.0 [+3.9, +40.5]0.02670.4270.0461.0000.062
U2-Opus − UCE+16.4 [−1.3, +33.4]0.08490.7640.1091.0000.147
U2-Opus − TIME+8.2 [−7.6, +22.0]0.31041.0000.3391.0000.422
U2-Astra − RS+9.2 [+0.8, +18.1]0.05250.5970.0731.0000.101
U2-Astra − U1b−0.6 [−7.3, +6.4]0.88801.0000.9401.0000.950
U2-Astra − UCE+P+40.3 [+29.3, +51.3]<0.0001<0.001<0.0010.001<0.001
U2-Astra − UCE+33.7 [+22.0, +45.1]<0.00010.001<0.0010.008<0.001
U2-Astra − TIME+25.4 [+16.8, +34.9]<0.0001<0.001<0.0010.004<0.001
RS − U1b−9.8 [−19.9, −0.3]0.06910.6910.0921.0000.127
RS − UCE+P+31.1 [+16.3, +44.8]0.00070.0150.0020.1540.003
RS − UCE+24.5 [+9.5, +38.6]0.00430.0830.0090.8660.014
RS − TIME+16.2 [+9.1, +23.0]0.00040.0100.0010.0960.002
U1b − UCE+P+40.9 [+29.0, +52.3]<0.0001<0.001<0.0010.003<0.001
U1b − UCE+34.3 [+22.0, +45.9]<0.00010.001<0.0010.011<0.001
U1b − TIME+26.0 [+17.1, +36.3]<0.0001<0.001<0.0010.001<0.001
UCE+P − UCE−6.6 [−10.0, −3.5]0.00050.0120.0010.1130.003
UCE+P − TIME−14.8 [−25.7, −2.8]0.02200.3740.0401.0000.053

Table 13 (PDF p. 22). Table 13, block 4 of 8: fresh CLIP generality

continued on next page

contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
UCE − TIME−8.2 [−19.5, +4.3]0.20261.0000.2351.0000.300

Table 13 (continued, PDF p. 23). fresh CLIP generality

contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
U3-Opus − U3-Astra+2.0 [−1.9, +6.2]0.36071.0000.4811.0000.477
U3-Opus − U2-Opus ⋆−5.5 [−11.3, 0.0]0.07151.0000.1231.0000.130
U3-Opus − U2-Astra−4.2 [−9.1, +0.3]0.10581.0000.1731.0000.181
U3-Opus − RS+3.1 [−2.0, +7.9]0.24281.0000.3501.0000.343
U3-Opus − U1b ⋆+1.7 [−2.4, +5.8]0.43081.0000.5351.0000.549
U3-Opus − UCE+P ⋆−6.6 [−12.1, −1.6]0.02280.4780.0511.0000.054
U3-Opus − UCE ⋆+7.3 [−2.7, +18.6]0.21361.0000.3341.0000.311
U3-Opus − TIME ⋆+8.6 [+3.3, +13.8]0.00500.1360.0180.9900.016
U3-Astra − U2-Opus−7.6 [−14.4, −1.0]0.04240.7210.0761.0000.087
U3-Astra − U2-Astra−6.2 [−11.5, −1.6]0.02090.4610.0501.0000.051
U3-Astra − RS+1.1 [−5.5, +7.2]0.75581.0000.8001.0000.856
U3-Astra − U1b−0.3 [−5.3, +4.5]0.91901.0000.9191.0000.966
U3-Astra − UCE+P−8.6 [−14.9, −3.0]0.01020.2450.0281.0000.029
U3-Astra − UCE+5.3 [−5.8, +17.4]0.40691.0000.5231.0000.521
U3-Astra − TIME+6.6 [+0.9, +12.0]0.03510.6320.0661.0000.074
U2-Opus − U2-Astra+1.3 [−3.4, +6.1]0.62121.0000.6991.0000.739
U2-Opus − RS+8.6 [+3.9, +13.4]0.00230.0680.0110.4900.008
U2-Opus − U1b+7.2 [+3.7, +11.1]0.00080.0240.0050.1750.004
U2-Opus − UCE+P−1.1 [−5.4, +2.9]0.64861.0000.7081.0000.759
U2-Opus − UCE+12.8 [+3.5, +23.2]0.01660.3810.0431.0000.042
U2-Opus − TIME+14.1 [+8.2, +20.0]0.00020.0070.0020.0530.001
U2-Astra − RS+7.3 [+2.7, +12.0]0.00720.1870.0231.0000.022
U2-Astra − U1b+5.9 [+2.5, +9.3]0.00270.0740.0110.5480.009
U2-Astra − UCE+P−2.4 [−6.2, +1.3]0.23951.0000.3501.0000.340
U2-Astra − UCE+11.5 [+2.6, +22.0]0.02680.5350.0571.0000.062
U2-Astra − TIME+12.8 [+8.0, +17.9]<0.00010.0010.0010.008<0.001
RS − U1b−1.4 [−6.2, +3.2]0.58561.0000.6801.0000.709
RS − UCE+P−9.7 [−15.1, −4.3]0.00230.0680.0110.4770.008
RS − UCE+4.2 [−6.6, +16.5]0.51061.0000.6131.0000.639
RS − TIME+5.5 [+1.0, +9.9]0.03010.5720.0601.0000.067
U1b − UCE+P−8.3 [−12.0, −4.7]0.00020.0080.0020.0590.002
U1b − UCE+5.6 [−3.6, +16.6]0.31781.0000.4401.0000.426
U1b − TIME ⋆+6.9 [+2.3, +11.8]0.00930.2330.0281.0000.027
UCE+P − UCE+13.9 [+7.1, +22.2]<0.00010.0020.0010.012<0.001
UCE+P − TIME+15.2 [+9.9, +20.5]<0.00010.0010.0010.011<0.001
UCE − TIME+1.3 [−10.2, +10.9]0.82741.0000.8511.0000.903

Table 13 (PDF p. 23). Table 13, block 5 of 8: fresh-negative CLIP specificity

contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
U3-Opus − U3-Astra+1.1 [−3.8, +6.1]0.68991.0000.8921.0000.803
U3-Opus − U2-Opus−8.6 [−16.1, −1.6]0.03141.0000.2101.0000.068
U3-Opus − U2-Astra−6.9 [−15.2, +1.2]0.12971.0000.3591.0000.215
U3-Opus − RS−8.1 [−15.6, −0.7]0.05281.0000.2111.0000.101
U3-Opus − U1b−2.1 [−10.4, +5.7]0.64401.0000.8921.0000.757
U3-Opus − UCE+P−9.6 [−19.3, +0.3]0.07671.0000.2761.0000.138
U3-Opus − UCE−7.7 [−16.0, +0.6]0.08961.0000.2931.0000.154
U3-Opus − TIME−10.2 [−17.0, −4.1]0.00370.1300.0670.7580.013
U3-Astra − U2-Opus−9.7 [−16.8, −3.0]0.01220.4140.1461.0000.033
U3-Astra − U2-Astra−8.0 [−17.2, +1.2]0.11061.0000.3321.0000.189
U3-Astra − RS−9.2 [−16.8, −2.0]0.02310.7620.2081.0000.054
U3-Astra − U1b−3.2 [−10.3, +3.4]0.39401.0000.6591.0000.509
U3-Astra − UCE+P−10.8 [−19.6, −0.5]0.04361.0000.2101.0000.089
U3-Astra − UCE−8.9 [−16.4, −0.6]0.04671.0000.2101.0000.093
U3-Astra − TIME−11.4 [−18.3, −4.9]0.00220.0790.0670.4680.008
U2-Opus − U2-Astra+1.7 [−5.9, +9.9]0.69391.0000.8921.0000.803
contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
U2-Opus − RS+0.5 [−5.8, +7.1]0.90021.0000.9361.0000.953
U2-Opus − U1b+6.5 [−1.6, +15.1]0.15711.0000.3771.0000.242
U2-Opus − UCE+P−1.0 [−8.8, +7.6]0.83141.0000.9361.0000.904
U2-Opus − UCE+0.9 [−6.3, +8.7]0.84561.0000.9361.0000.912
U2-Opus − TIME−1.6 [−6.9, +3.6]0.57281.0000.8591.0000.696
U2-Astra − RS−1.2 [−8.7, +6.4]0.77971.0000.9361.0000.868
U2-Astra − U1b+4.8 [−4.8, +14.4]0.36391.0000.6551.0000.479
U2-Astra − UCE+P−2.8 [−12.6, +7.0]0.60751.0000.8751.0000.727
U2-Astra − UCE−0.9 [−9.7, +8.0]0.86661.0000.9361.0000.931
U2-Astra − TIME−3.4 [−9.5, +2.1]0.29681.0000.5621.0000.405
RS − U1b+6.0 [−3.2, +15.3]0.23531.0000.4981.0000.336
RS − UCE+P−1.6 [−11.8, +9.8]0.79851.0000.9361.0000.884
RS − UCE+0.3 [−9.6, +10.8]0.96211.0000.9621.0000.986
RS − TIME−2.2 [−7.0, +2.3]0.40291.0000.6591.0000.518
U1b − UCE+P−7.6 [−16.4, +2.3]0.14051.0000.3611.0000.230
U1b − UCE−5.6 [−13.0, +2.3]0.17911.0000.4031.0000.269
U1b − TIME−8.2 [−16.1, −1.0]0.04591.0000.2101.0000.092
UCE+P − UCE+1.9 [−1.3, +4.9]0.27531.0000.5511.0000.381
UCE+P − TIME−0.6 [−9.9, +7.1]0.90971.0000.9361.0000.960
UCE − TIME−2.5 [−10.4, +4.2]0.53741.0000.8411.0000.661

Table 13 (PDF p. 23, second). Table 13, block 6 of 8: fresh subject presence

contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
U3-Opus − U3-Astra−5.2 [−13.9, +0.3]0.21951.0000.2721.0000.316
U3-Opus − U2-Opus+17.4 [+3.5, +33.3]0.05580.8370.0911.0000.106
U3-Opus − U2-Astra−4.9 [−14.2, +0.7]0.37511.0000.3971.0000.488
U3-Opus − RS−0.3 [−5.2, +3.8]1.00001.0001.0001.0001.000
U3-Opus − U1b−6.2 [−15.6, −0.7]0.07881.0000.1181.0000.139
U3-Opus − UCE+P+37.2 [+17.4, +55.6]0.00160.0440.0060.3550.007
U3-Opus − UCE+30.6 [+11.8, +48.3]0.00440.1050.0120.8730.014
U3-Opus − TIME+7.6 [−1.0, +17.4]0.16821.0000.2161.0000.255
U3-Astra − U2-Opus+22.6 [+8.0, +39.6]0.01540.3080.0331.0000.040
U3-Astra − U2-Astra+0.3 [−2.1, +2.4]1.00001.0001.0001.0001.000
U3-Astra − RS+4.9 [+1.0, +10.1]0.03060.5200.0551.0000.067
U3-Astra − U1b−1.0 [−2.1, 0.0]0.24941.0000.2991.0000.350
U3-Astra − UCE+P+42.4 [+26.7, +58.7]0.00010.002<0.0010.0170.001
U3-Astra − UCE+35.8 [+21.2, +51.0]<0.00010.001<0.0010.011<0.001
U3-Astra − TIME+12.8 [+5.2, +21.9]0.00240.0600.0070.4960.008
U2-Opus − U2-Astra−22.2 [−38.9, −7.3]0.01880.3570.0381.0000.046
U2-Opus − RS−17.7 [−33.3, −4.2]0.03580.5720.0611.0000.075
U2-Opus − U1b−23.6 [−40.3, −8.7]0.01180.2490.0271.0000.033
U2-Opus − UCE+P+19.8 [−5.9, +43.8]0.14971.0000.2031.0000.236
U2-Opus − UCE+13.2 [−11.5, +36.8]0.31301.0000.3521.0000.423
U2-Opus − TIME−9.7 [−26.4, +5.2]0.27481.0000.3191.0000.381
U2-Astra − RS+4.5 [0.0, +10.8]0.15241.0000.2031.0000.237
U2-Astra − U1b−1.4 [−3.8, +0.3]0.37431.0000.3971.0000.488
U2-Astra − UCE+P+42.0 [+26.4, +58.0]<0.00010.001<0.0010.007<0.001
U2-Astra − UCE+35.4 [+21.2, +50.7]0.00010.0040.0010.0290.001
U2-Astra − TIME+12.5 [+5.2, +21.2]0.00050.0150.0030.1190.003
RS − U1b−5.9 [−11.8, −1.7]0.00760.1670.0181.0000.022
RS − UCE+P+37.5 [+20.5, +54.9]0.00060.0180.0030.1430.003
RS − UCE+30.9 [+14.6, +47.6]0.00190.0500.0060.4160.008
RS − TIME+8.0 [+1.0, +16.7]0.06930.9700.1081.0000.127
U1b − UCE+P+43.4 [+28.1, +59.0]<0.0001<0.001<0.0010.001<0.001
U1b − UCE+36.8 [+22.9, +51.7]<0.00010.001<0.0010.007<0.001
U1b − TIME+13.9 [+5.9, +23.6]0.00080.0230.0030.1830.004
UCE+P − UCE−6.6 [−15.3, 0.0]0.14721.0000.2031.0000.234
UCE+P − TIME−29.5 [−46.2, −11.8]0.00460.1050.0120.9030.014
UCE − TIME−22.9 [−39.9, −5.2]0.02310.4150.0441.0000.054

Table 13 (PDF p. 24). Table 13, block 7 of 8: native CLIP efficacy

contrastdifference [95% CI]ppHolm (end.)BH (end.)Holm (all)BH (all)
U3-Opus − U3-Astra−1.6 [−7.1, +3.8]0.60241.0000.6201.0000.726
U3-Opus − U2-Opus+20.1 [+8.4, +33.3]0.00230.0420.0040.4770.008
U3-Opus − U2-Astra+1.9 [−4.2, +7.9]0.55871.0000.5921.0000.685
U3-Opus − RS+7.2 [+2.3, +13.6]0.00810.1210.0131.0000.024
U3-Opus − U1b−3.5 [−10.0, +2.4]0.31471.0000.3541.0000.424
U3-Opus − UCE+P+46.5 [+30.2, +61.5]<0.00010.001<0.0010.008<0.001
U3-Opus − UCE+37.8 [+22.0, +52.7]0.00020.0050.0010.0530.001
U3-Opus − TIME+23.1 [+12.3, +34.0]0.00040.0090.0010.0920.002
U3-Astra − U2-Opus+21.7 [+10.6, +34.4]0.00040.0090.0010.0940.002
U3-Astra − U2-Astra+3.5 [−0.7, +8.2]0.15110.9070.1761.0000.237
U3-Astra − RS+8.8 [+2.6, +15.8]0.01220.1580.0181.0000.033
U3-Astra − U1b−1.9 [−6.9, +2.9]0.49281.0000.5381.0000.623
U3-Astra − UCE+P+48.1 [+34.0, +61.6]<0.0001<0.001<0.0010.001<0.001
U3-Astra − UCE+39.4 [+25.3, +52.9]0.00010.002<0.0010.0170.001
U3-Astra − TIME+24.7 [+15.1, +34.8]0.00010.002<0.0010.0140.001
U2-Opus − U2-Astra−18.2 [−31.7, −6.2]0.00890.1250.0141.0000.026
U2-Opus − RS−12.9 [−25.8, −1.7]0.05030.5030.0671.0000.098
U2-Opus − U1b−23.6 [−38.1, −10.3]0.00190.0370.0040.4050.007
U2-Opus − UCE+P+26.4 [+7.6, +44.1]0.01270.1580.0181.0000.034
U2-Opus − UCE+17.6 [−1.0, +35.3]0.07910.6320.0981.0000.139
U2-Opus − TIME+3.0 [−12.2, +16.3]0.70051.0000.7011.0000.807
U2-Astra − RS+5.3 [−0.8, +11.4]0.11210.7850.1351.0000.189
U2-Astra − U1b−5.4 [−9.9, −1.0]0.02980.3270.0411.0000.067
U2-Astra − UCE+P+44.6 [+31.5, +57.3]<0.00010.001<0.0010.006<0.001
U2-Astra − UCE+35.8 [+22.7, +48.8]<0.00010.001<0.0010.012<0.001
U2-Astra − TIME+21.2 [+12.2, +30.6]0.00020.0050.0010.0500.001
RS − U1b−10.7 [−18.3, −3.8]0.00740.1180.0131.0000.022
RS − UCE+P+39.3 [+23.1, +54.6]0.00020.0050.0010.0530.001
RS − UCE+30.6 [+14.2, +46.5]0.00220.0420.0040.4690.008
RS − TIME+15.9 [+7.4, +24.3]0.00170.0360.0040.3740.007
U1b − UCE+P+50.0 [+38.1, +61.2]<0.0001<0.001<0.0010.003<0.001
U1b − UCE+41.3 [+29.3, +52.8]<0.0001<0.001<0.0010.004<0.001
U1b − TIME+26.6 [+18.0, +36.0]<0.0001<0.001<0.0010.001<0.001
UCE+P − UCE−8.7 [−12.1, −5.6]<0.0001<0.001<0.0010.001<0.001
UCE+P − TIME−23.4 [−36.0, −8.8]0.00420.0710.0070.8440.014
UCE − TIME−14.7 [−28.1, +0.2]0.06110.5500.0791.0000.114

Table 13 (PDF p. 25). Table 13, block 8 of 8: native CLIP generality

References

  1. [1]Amara, I., Humayun, A. I., Kajic, I., Parekh, Z., Harris, N., Young, S., Nagpal, C., Kim, N., He, J., Vasconcelos, C. N., Ramachandran, D., Farnadi, G., Heller, K., Havaei, M., and Rostamzadeh, N. Erasing more than intended? how concept erasure degrades the generation of non-target concepts. In IEEE/CVF International Conference on Computer Vision (ICCV), 2025.arxiv.org/abs/2501.09833
  2. [2]Arad, D., Orgad, H., and Belinkov, Y. Refact: Updating text-to-image models by editing the text encoder, 2023.
  3. [3]Black, K., Janner, M., Du, Y., Kostrikov, I., and Levine, S. Training diffusion models with reinforcement learning, 2023.arxiv.org/abs/2305.13301
  4. [4]Butkeraites, R. Can a language model write the weights? an empirical study of llm-written scoring networks for column generation and milp branching, with a bus-driver scheduling application, 2026.
  5. [5]Chefer, H., Alaluf, Y., Vinker, Y., Wolf, L., and Cohen-Or, D. Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. ACM Transactions on Graphics, 42(4), 2023. doi: 10.1145/3592116.DOI
  6. [6]Cho, J., Hu, Y., Garg, R., Anderson, P., Krishna, R., Baldridge, J., Bansal, M., Pont-Tuset, J., and Wang, S. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation, 2023.arxiv.org/abs/2310.18235
  7. [7]Clark, K., Vicol, P., Swersky, K., and Fleet, D. J. Directly fine-tuning diffusion models on differentiable rewards, 2023.arxiv.org/abs/2309.17400
  8. [8]Fang, J., Jiang, H., Wang, K., Ma, Y., Shi, J., Wang, X., He, X., and seng Chua, T. Alphaedit: Null-space constrained knowledge editing for language models, 2024.arxiv.org/abs/2410.02355
  9. [9]Gandikota, R., Orgad, H., Belinkov, Y., Materzyńska, J., and Bau, D. Unified concept editing in diffusion models, 2023.arxiv.org/abs/2308.14761
  10. [10]Gao, L., Schulman, J., and Hilton, H. Scaling laws for reward model overoptimization, 2022.arxiv.org/abs/2210.10760
  11. [11]Ghosh, D., Hajishirzi, H., and Schmidt, L. Geneval: An object-focused framework for evaluating text-to-image alignment. In neural information processing systems, 2023. doi: 10.52202/075280-2270.DOI
  12. [17]Lin, Z., Pathak, D., Li, B., Li, J., Xia, X., Neubig, G., Zhang, P., and Ramanan, D. Evaluating text-to-visual generation with image-to-text generation. In Computer Vision – ECCV 2024, volume 15067 of Lecture Notes in Computer Science, pp. 366–384, 2024. doi: 10.1007/978-3-031-72673-6_20.DOI
  13. [19]Manheim, D. and Garrabrant, S. Categorizing variants of goodhart’s law, 2018.arxiv.org/abs/1803.04585
  14. [20]Moon, S., Lee, M., Park, S., and Kim, D. Holistic unlearning benchmark: A multi-faceted evaluation for text-to-image diffusion model unlearning. In IEEE/CVF International Conference on Computer Vision (ICCV), 2025.arxiv.org/abs/2410.05664
  15. [21]Orgad, H., Kawar, B., and Belinkov, Y. Editing implicit assumptions in text-to-image diffusion models, 2023.arxiv.org/abs/2303.08084
  16. [22]Pan, A., Jones, E., Jagadeesan, M., and Steinhardt, J. Feedback loops with language models drive in-context reward hacking, 2024a.arxiv.org/abs/2402.06627
  17. [24]Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, P., Fawzi, O., Kohli, P., and Fawzi, A. Mathematical discoveries from program search with large language models. Nature, 2023. doi: 10.1038/s41586-023-06924-6.
  18. [25]Skalse, J., Howe, N. H. R., Krasheninnikov, D., and Krueger, D. Defining and characterizing reward hacking, 2022.arxiv.org/abs/2209.13085
  19. [26]Taghia, J. An llm erases a concept by editing diffusion weights directly, without gradients. Hugging Face community article and self-published PDF, 2026. Community article published 2026-11-08; not peer reviewed.
  20. [27]Takehi, R., Teoh, J., Zhang, A., and Khattab, O. Sparks of weight-space programming: Llms can natively read & write neural network weights. Blog post, https://rikiyat.github.io/blog/weight-space-programming/, 2026-11-02; author list read from the page’s own metadata.
  21. [28]Thrush, T., Jiang, R., Bartolo, M., Singh, A., Williams, A., Kiela, D., and Ross, C. Winground: Probing vision and language models for visio-linguistic compositionality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. doi: 10.1109/cvpr52688.2022.00517.DOI
  22. [29]Wang, R., Fang, J., Li, J., Chen, H., Shi, J., Wang, K., and Wang, X. ACE: Concept editing in diffusion models without performance degradation. arXiv preprint arXiv:2503.08116, 2025.arxiv.org/abs/2503.08116
  23. [30]Xiong, T., Wu, Y., Xie, E., Wu, Y., Li, Z., and Liu, X. Editing massive concepts in text-to-image diffusion models. arXiv preprint arXiv:2403.13807, 2024.arxiv.org/abs/2403.13807
  24. [31]Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers, 2023.arxiv.org/abs/2309.03409
  25. [32]Yuksekgonul, M., Bianchi, F., Kalluri, P., Jurafsky, D., and Zou, J. When and why vision-language models behave like bags-of-words, and what to do about it?, 2022.
  26. [33]Zhang, J., Wang, K., and Lin, L. Can llms debug robots by editing their weights?, 2026.arxiv.org/abs/2611.09376
  27. [34]Zhang, Z., Zhang, S., Zhan, Y., Luo, Y., Wen, Y., and Tao, D. Confronting reward overoptimization for diffusion models: A perspective of inductive and primacy biases, 2024.arxiv.org/abs/2402.08552

Paper details

vlnc:W_bnq2e358
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC0-1.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents