betaFeedback

WorksComputer Science

Dynamic Shaping: Acquiring Interfaces and Constructing Execution over Frozen Pretrained Components

Abstract

We introduce Dynamic Shaping, a method for constructing executable neural arrangements through acquired response and partial-derivative interfaces over frozen pretrained components. The method learns local descriptions of available computation, ranks component rebindings using task-gradient alignment, transports the current interface across a rebinding, and retains changes through behavioral feedback. Continuous interface acquisition uses proximal policy optimization; the source coefficients remain fixed. We derive an exact identity relating transport error in component responses and partials to changes in action logits and their sensitivities. A pixels-only ViZDoom Basic study evaluates first task acquisition without a competent reference controller, demonstrations or intact response targets. Across three initializations and 100 shared held-out game seeds, initial arrangements succeed on 86/300 controller/episode pairs; joint interface-and-binding construction succeeds on 295/300. Fixed-binding interface acquisition also reaches 295/300. Each policy executes four source fragments, with 110 acquired interface values and fixed random visual and action projections. The artifact includes all candidate trials, interventions and costs; all 1,200 recorded initial/retained outcomes replay exactly. The contribution is an explicit profile, rebind, transport and retain procedure and a reproducible reference-free acquisition case over frozen pretrained computation.

Code and complete evidence: https://github.com/calysteon/shaping-gap-benchmark. Checkpoint 8, immutable revision 73971c5a31f2acbfb7ea1e8131c456964e6e1826.

Introduction

Pretraining produces numerical computation that can be used through interfaces other than its original application. The research question here is how a runtime can acquire an executable arrangement of that computation from a task requirement and experience. We study a component-level construction process: describe the effects of available frozen parts, choose how to connect them, learn the interfaces through which they act, and retain a working execution.

We call this process Dynamic Shaping. Its studied object is the acquired arrangement and interfaces. A task contract supplies observable success conditions, rather than a successful implementation. The runtime receives no per-instance human repair or hand-selected successful binding. Frozen source coefficients supply the available functions; task experience determines their stimulus, binding and interface parameters.

The mechanism has four coupled operations. First, local response and partial measurements describe the available components at states the runtime actually visits. Second, task-gradient alignment ranks proposed changes to the active bindings. Third, an affine transport changes the interface along with a binding so that it can preserve useful responses and sensitivities. Fourth, real behavioral feedback accepts or rejects the proposal. Standard policy optimization continues acquiring the continuous interfaces between these discrete construction steps.

Our contribution is the specific construction-and-transport procedure, its exact transport-error identity, and an auditable first-acquisition experiment. The procedure combines established tools: reinforcement learning, derivative supervision, local approximation and affine alignment. § 7 positions the combined method against the closest approaches. The experimental question is whether it supplies reliable behavior absent from its initial execution under the stated information and resource permissions.

We evaluate a simple pixels-only move-and-shoot task. Each initial runtime is untrained on the task and occasionally succeeds, but falls below the specified reliability contract. Acquisition receives game feedback without a competent controller to imitate. This differs from the preceding reconstruction study, which damaged a trained controller and supplied recorded intact responses [12]. The current study learns a first task implementation. Both the full construction procedure and its fixed-binding interface control meet the quality contract; the full procedure has no aggregate success advantage in this experiment. The complete results and costs accompany the mechanism.

Frozen substrate and acquired execution

Let Φ={ϕ1,…,ϕM}\Phi= \{\phi_1, \ldots, \phi_M\} be an inventory of callable scalar neural components with immutable materialized parameters. A policy arrangement G=(G1,…,Gk)G = (G_1, \ldots, G_k) selects kk distinct identifiers. Continuous interfaces determine the stimulus supplied to these components and how their responses reach a fixed consumer. A shaping procedure receives Φ\Phi, an initial arrangement and interfaces, a behavioral contract CC, permitted experience, and a resource specification. It returns a retained executable arrangement, interfaces and acquisition record.

The experimental inventory has M=24M = 24 RMSNorm/SwiGLU gate/up scalar responses from the public Qwen3-0.6B-derived component bank, context 151003 [11, 13]. A fixed nine-dimensional source-coordinate compilation materializes these responses. The original transformer attention graph, scheduling, language head and native down projection are not executed by this controller. Source coefficients and source normalization arrays stay fixed. Appendix § 10 specifies the executed scalar formula.

Each observation is a 160×120160 \times120 RGB frame. Generic 16×1216 \times12 block means produce 576 numbers, standardized using unlabeled random-action observations. A random fixed projection E∈R576×9E \in\mathbb{R}^{576 \times9} and fixed calibration produce a row vector z∈R9z \in\mathbb{R}^9. Execution is

s=tanh⁡(zR+r),h=(ϕG1(s),…,ϕGk(s)),f=hA+o,ℓ=fW+b,π=softmax⁡(ℓ).(1)\begin{aligned} s &= \tanh(zR+r), & h &= (\phi_{G_1}(s), \ldots, \phi_{G_k}(s)), \\ f &= hA+o, & \ell&= fW+b, & \pi&= \operatorname{softmax}(\ell). \tag*{(1)} \end{aligned}

Here k=4k = 4, R∈R9×9R \in\mathbb{R}^{9 \times9}, r∈R9r \in\mathbb{R}^9, A∈R4×4A \in\mathbb{R}^{4 \times4} and o∈R4o \in\mathbb{R}^4 are acquired: 110 real values. The random action projection W∈R4×3W \in\mathbb{R}^{4 \times3} and its zero bias bb remain fixed, as do EE and its calibration. A four-value critic and scalar bias are auxiliary acquisition state. Initially R=0.15IR = 0.15I, r=o=0r = o = 0 and A=IA = I. Generic diversity of native partials at zero selects the initial bindings without task-performance selection. Figure 1 separates execution from acquisition.

Figure 1. Dynamic Shaping separates fixed computational parts from acquired interfaces and bindings. Task experience drives continuous interface learning and the discrete profile–rebind–transport–retain loop. Discovery queries the inventory; policy execution uses four bound fragments.

The four-component limit is a policy execution budget. Discovery queries all 24 available components on its calibration states. Those queries, temporary arrays, profile fits and candidate evaluations are charged separately. Freezing the source does not freeze the complete policy: its interfaces are learned parameters that can change end-to-end behavior.

Response and partial interfaces for construction

Continuous acquisition

Proximal policy optimization (PPO) supplies the continuous optimizer [9]. Game rewards and generalized advantage estimates train the input/output interfaces and auxiliary critic. No target action or target hidden response is provided. Exact native partials connect the fixed functions to the acquired stimulus. For action logit aa, for example,

∂ℓa∂Rub=zu(1−sb2)∑j,v∂ΦGj∂sbAjvWva.(2)\frac{\partial\ell_a}{\partial R_{ub}} = z_u(1-s_b^2)\sum_{j,v}\frac{\partial\Phi_{G_j}}{\partial s_b}A_{jv}W_{va}. \tag*{(2)}

Thus the learning signal can change how observations drive existing neural responses while leaving their coefficients and the action consumer fixed. Interface acquisition is training and is included in the ledger.

Acquire an integrable local response description

Every 20 updates, take 384 evenly spaced observations from the latest rollout and recompute their current native input states sts_t. Query responses and analytic partials for all available components. Write normalized profile coordinates as the column q=((s−μ)⊘σ)T\mathbf{q} = ((\mathbf{s}-\boldsymbol{\mu}) \oslash\boldsymbol{\sigma})^{T}, with each coordinate of σ\boldsymbol{\sigma} floored at 0.03. For component cc, fit

ϕ^c(q)=αc+Lcq+12qTHcq,J^c(q)=Lc+qTHc,(3)\hat{\phi}_c(\mathbf{q}) = \alpha_c + L_c\mathbf{q} + \frac{1}{2}\mathbf{q}^{T}H_c\mathbf{q}, \qquad\hat{J}_c(\mathbf{q}) = L_c + \mathbf{q}^{T}H_c, \tag*{(3)}

where HcH_c is symmetric. Values and partials therefore describe one common response function, rather than independent fitted fields. The first 256 states fit the profile and the remaining 128 check approximation error.

The implementation uses ridge-regularized polynomial/Hermite regression with both response and derivative observations, normalized per output. This is an instance of established value-and-derivative supervision [7, 8]. It yields a local description of a component’s current usable effects; the original component remains the executed function.

Fit a value-and-partial transport

For active slot ii and unused component cc, let yt=ϕGi(st)y_t=\phi_{G_i}(s_t) be the current slot response, ut=ϕ^c(qt)u_t=\hat{\phi}_c(q_t) its proposed replacement, Jty=∂qϕGiJ^y_t=\partial_q\phi_{G_i} the measured current partial, and JtuJ^u_t the profile partial. The algorithm fits an affine bridge au+bau+b by minimizing

Et[(aut+b−yt)2]+λEt,d[(aJtdu−Jtdy)2]+κa2,(4)\mathbb{E}_t[(au_t+b-y_t)^2]+\lambda\mathbb{E}_{t,d}[(aJ^u_{td}-J^y_{td})^2]+\kappa a^2, \tag*{(4)}

with λ=0.25\lambda=0.25, κ=10−6\kappa=10^{-6} and a∈[−10,10]a\in[-10,10]. Here Et,d\mathbb{E}_{t,d} averages over both states and native-input coordinates. Centering values gives the implemented closed form

a=clip⁡[−10,10]Et[(ut−uˉ)(yt−yˉ)]+λEt,d[JtduJtdy]Et[(ut−uˉ)2]+λEt,d[(Jtdu)2]+κ,(5)a=\operatorname{clip}_{[-10,10]}\frac{\mathbb{E}_t[(u_t-\bar{u})(y_t-\bar{y})]+\lambda\mathbb{E}_{t,d}[J^u_{td}J^y_{td}]}{\mathbb{E}_t[(u_t-\bar{u})^2]+\lambda\mathbb{E}_{t,d}[(J^u_{td})^2]+\kappa}, \tag*{(5)}
b=yˉ−auˉ.(6)b=\bar{y}-a\bar{u}. \tag*{(6)}

These targets describe the runtime’s own current computation. They are not saved responses of a competent teacher or intact predecessor.

Rank, transport and retain

Let δ^t=aut+b−yt\hat{\delta}_t=au_t+b-y_t and vi=Ai,:Wv_i=A_{i,:}W. The predicted raw-logit displacement is Δℓt=δ^tvi\Delta\ell_t=\hat{\delta}_tv_i. A policy-loss gradient proxy gtig_{ti}, computed from rollout actions and standardized advantages under the current probabilities, ranks each proposal by

Si,c=∑tgtiδ^t+0.0512Et[∑aπta(Δℓta)2−(∑aπtaΔℓ^ta)2].(7)S_{i,c}=\sum_t g_{ti}\hat{\delta}_t+0.05\frac{1}{2}\mathbb{E}_t\left[\sum_a\pi_{ta}(\Delta\ell_{ta})^2-\left(\sum_a\pi_{ta}\widehat{\Delta\ell}_{ta}\right)^2\right]. \tag*{(7)}

The second term is a softmax-curvature penalty. This score is a proposal heuristic, not an exact PPO loss difference or a guarantee of reward gain. Appendix § 11 gives its implementation correspondence. Candidates are sorted by Si,cS_{i,c}, followed by partial-transport error and deterministic identifiers. For each of the top three, clone the current interfaces and apply, using the old row Ai,:A_{i,:},

Gi←c,Ai,:←aAi,:,o←o+bAi,:old.(8)G_i\leftarrow c,\qquad A_{i,:}\leftarrow aA_{i,:},\qquad o\leftarrow o+bA^{\mathrm{old}}_{i,:}. \tag*{(8)}

Real development episodes evaluate these transported constructions. Accept only a strict lexicographic improvement in success rate and mean return over the unmodified current policy; otherwise roll back. Acceptance resets optimizer moments. Continuous acquisition then resumes. A separate development score selects the best snapshot across the complete acquisition horizon for final retention. No terminal outcome selects any binding or interface.

Algorithm 1: Dynamic Shaping over a frozen inventory

  1. Generate the initial arrangement and fixed visual/action projections; calibrate observations without demonstrations.

  2. Collect a rollout from the current policy; update input/output interfaces and critic using PPO through native partials.

  3. At a construction round, measure available responses/partials and fit local profiles using Eq. (3).

  4. For every slot/unused component pair, fit Eq. (5), rank with Eq. (7), and construct the top three transported candidates using Eq. (8).

  5. Evaluate candidates on development episodes. Accept the best strict success/return improvement, or retain the current execution unchanged.

  6. Record every trial and cost. Select and seal the best development snapshot; evaluate the retained construction on fresh episodes.

Exact effect of interface transport

The transport step has a useful algebraic property independent of the profile approximation and proposal ranking. Let ℓ\ell and ℓ′\ell' denote raw action logits before Eq. (8), with all other bindings and arrays fixed. Define the actual response discrepancy at a native input as δ=aϕc+b−ϕGi\delta= a\phi_c + b - \phi_{G_i}. Use an input-by-action Jacobian convention. Then

ℓ′−ℓ=δvi,vi=Ai,:oldW,(9)\ell' - \ell= \delta v_i,\qquad v_i = A^{\mathrm{old}}_{i,:}W, \tag*{(9)}
∂qℓ′−∂qℓ=(a∂qϕc−∂qϕGi)⊤vi.(10)\partial_q\ell' - \partial_q\ell= \left(a\partial_q\phi_c - \partial_q\phi_{G_i}\right)^\top v_i. \tag*{(10)}

Proof. All unchanged slot contributions cancel. The changed feature contribution is aϕcAi,:old+bAi,:olda\phi_c A^{\mathrm{old}}_{i,:} + bA^{\mathrm{old}}_{i,:} instead of ϕGiAi,:old\phi_{G_i}A^{\mathrm{old}}_{i,:}. Multiplication by the unchanged WW gives Eq. (9); differentiation with fixed aa, bb, AA, WW gives Eq. (10). Consequently,

∥ℓ′−ℓ∥2=∣δ∣∥vi∥2,(11)\lVert\ell' - \ell\rVert_2 = |\delta|\lVert v_i\rVert_2, \tag*{(11)}
∥∂qℓ′−∂qℓ∥F=∥a∂qϕc−∂qϕGi∥2∥vi∥2.(12)\lVert\partial_q\ell' - \partial_q\ell\rVert_F = \lVert a\partial_q\phi_c - \partial_q\phi_{G_i}\rVert_2\lVert v_i\rVert_2. \tag*{(12)}

Exact response matching preserves the policy at that input; matching partials also preserves its local logit sensitivity. Approximate matching exposes the remaining discrepancy through Eq. (12). The identity explains what the transport controls. It does not certify the fitted profile away from its measured states or guarantee improved game behavior; behavioral evaluation supplies the retention decision.

First acquisition from a task contract

Task, information and protocol

ViZDoom Basic supplies a single target at a randomized horizontal location and three actions: move left, move right and shoot [10]. Pixels are the only policy observations. Game rewards train the interfaces; game variables record outcomes but are not policy inputs. The runtime receives the action set, generic preprocessing, optimizer, construction grammar and budgets. It receives no target coordinates, object labels, scripted control rules, demonstrations, teacher actions or intact-response targets.

The behavioral contract is at least 80 successes among 100 terminal episodes. Initial arrangements can sometimes succeed; the missing behavior is reliable control. The prospective strict criterion additionally required the initial arrangement to succeed at most 30/100 times. We retain and report both conditions separately.

The no-acquisition reference is the exact initial arrangement. Two acquisition arms share its arrays, random seed and continuous optimization settings: interface acquisition keeps the initial bindings fixed, while joint construction adds the profile, rebinding and transport loop. Both are autonomous learning procedures. Matched PPO update counts do not imply matched total interactions: the joint arm uses extra development evaluations for candidate selection, charged in the ledger.

Each run performs 160 updates of 1,024 sampled action steps, six PPO epochs per update, minibatches of 128, learning rate 0.003, discount 0.99, GAE parameter 0.95, clipping 0.2 and entropy coefficient 0.01. Calibration uses 1,024 random-action observations. Construction occurs every 20 updates: eight rounds, three candidate evaluations per round, 24 trials per joint run.

Calibration game seeds start at 810000; acquisition seeds start at 820000. Twenty development seeds, 830000–830019, serve all development comparisons. All choices for a controller are sealed before its terminal evaluation by a separate process on seeds 980000–980099. Initialization/action-sampling seeds are 81, 82 and 83. The same 100 terminal game seeds are used for all controllers; totals are controller/episode pairs, not 300 different maps. Seed 81 joint is the engineering pilot. The implementation was locked after its development run and before any terminal evaluation; seeds 82/83 use it unchanged.

Held-out outcomes

Every acquired controller exceeds the 80/100 quality contract (Table 1). The strict initial-failure ceiling is met by seeds 81 and 83 but missed by seed 82, whose initial score is 33/100. The criterion is not revised after observing that result. Figure 2 shows development trajectories; all terminal claims use the sealed snapshots.

Runtime818283Total
Initial arrangement27/10033/10026/10086/300
Interface acquisition100/10095/100100/100295/300
Joint construction99/10097/10099/100295/300

Table 1. Successful terminal episodes. Paired acquisition arms begin with identical initial arrays; their duplicated initial outcomes are counted once.

Figure 2. Development success during acquisition. The dashed line is the 80% contract. The horizontal axis counts continuous-acquisition game ticks; calibration and development selection work are additional, not hidden in it.

Joint construction accepts 3, 2 and 5 rebindings during the three runs. Its retained snapshots occur at updates 150, 120 and 160; their binding lists are (12, 5, 13, 4), (12, 5, 10, 17) and (21, 23, 3, 2), versus the initial (12, 5, 11, 17). Acceptance and final snapshot selection are distinct: a later accepted change need not be in the best retained snapshot.

The two acquisition arms tie in aggregate success. Thus this experiment establishes first acquisition under the stated permissions for both arms, while leaving an aggregate advantage of discrete construction unresolved. No random-weight substrate comparison or conventional game-agent performance comparison is performed. The experiment does not isolate a causal benefit of pretraining itself. Its positive result is a working executable construction over the specified frozen pretrained inventory, without a successful task reference.

Post-acquisition interventions

InterventionJ81I81J82I82J83I83
Retained99100979599100
Original bindings71100979524100
Original input interface241940341724
Original output interface36978266795
Zero native outputs2200222222
Zero standardized pixels002222220

Table 2. Successful episodes out of 100 after interventions without refitting. J denotes joint construction and I denotes fixed-binding interface acquisition.

These diagnostics were declared after the seed-81 joint terminal result and before running the interventions. They cause no further learning, selection or retention changes. Original-bindings interventions are redundant for the interface-only arm but remain recorded. Binding reversion has different effects: it reduces J81 and J83 performance while leaving J82 success unchanged. Input-interface reversion lowers every retained score. Source-output and sensory interventions expose the remaining baseline behaviors rather than assuming that every intervention must produce zero successes.

Reproducibility and verification

All 1,200 original initial/retained terminal episode records replay exactly in fresh processes. This includes 300 duplicated initial records from paired arms; the main experiment has 900 distinct initial/acquired-controller episode pairs. The 3,000 intervention episodes are recorded separately. Fixed source, projection and decoder arrays pass before/after hash checks. Centered finite differences validate the learned-interface gradients to maximum absolute error 2.55×10−102.55 \times10^{-10}.

A complete fresh acquisition from portable code and the compiled bank reproduces every retained array, all eight binding-selection decisions and all 200 seed-81 initial/retained terminal episode records exactly. The public artifact supplies the raw data, captured runners, profiles, ranked candidates, rejections, selections, seals and manifests at a pinned Git revision [13]. Appendix § 11 maps the paper to its entry points.

Language-model assistance from OpenAI Codex supported code development, experiment orchestration, literature checks and manuscript preparation. Runtime and package versions are included in the artifact.

Acquisition and lifecycle costs

Each action repeats for four requested game ticks. Requested counts upper-bound actual elapsed ticks when a terminal action ends early. Table 3 includes discovery, continuous acquisition, profile construction, candidate evaluation and development selection. Separate ledger entries charge terminal tests, replay, interventions and fresh-acquisition verification. Source response and partial-entry counters include the queried inventory, not only selected parts. The records retain all rejected trials and all work performed after the selected snapshot.

RunDevelopment ticksSecondsRSS MiBAccepted/tried
81/joint78,42458.2164.523/24
81/interface29,06053.3262.120/0
82/joint79,68859.2966.592/24
82/interface27,70851.5462.280/0
83/joint44,84456.9867.205/24
83/interface20,44053.3363.780/0

Table 3. Complete original acquisition runs, including failed candidates and acquisition after the best retained snapshot. Common additional work per row: 655,360 requested acquisition ticks and 4,096 calibration ticks.

ArraysBytes
Fixed projection, calibration and action decoder41,736
Pixel standardization9,216
Compiled source inventory5,336
Acquired input/output interfaces880
Auxiliary critic40
Active identifiers32
Total57,240

Table 4. Retained raw arrays

This is deployed array storage, not a full language model or whole-process RAM. The 1,024 calibration observations temporarily occupy 4,718,592 raw bytes. Profiles, candidate archives and logs are separate discovery/evidence storage; they are not included in the deployment tally. Python peak RSS excludes the separate game-engine process. Acquisition jobs ran sequentially, with brief validation inference potentially overlapping; elapsed times describe this machine, not a controlled throughput or electricity comparison. Source pretraining and extraction are inherited costs, not assigned zero.

Relation to existing methods

Reprogramming frozen computation. Adversarial reprogramming learns an input program that makes a fixed network perform a different task [1]. Frozen Pretrained Transformers adapt language-pretrained computation to other modalities while retaining the core attention/feedforward weights [2]. Borrowed Geometry studies cross-distribution head importance and cross-modal behavior through thin interfaces over frozen text-pretrained slices [3]. These works establish that task acquisition through interfaces over existing computation is an important prior foundation. Here the acquired object additionally includes discrete bindings of callable scalar components outside the original transformer execution graph, with measured profiles guiding their replacement and interface transport.

Pathways and stitching. PathNet evolves pathways while gradient learning updates permitted weights; transfer freezes a learned path and evolves new paths [4]. Model stitching learns a simple bridge between frozen neural representations [5]. Weight Agnostic Neural Networks demonstrate structural search for task behavior while sharing a common weight value across connections [6].

Dynamic Shaping combines a local construction proposal with an explicitly transported interface, over an inventory whose source coefficients all remain fixed. Current response/partial measurements determine a candidate bridge; task-gradient alignment prioritizes candidates; feedback retains the execution/interface pair. We study this complete loop with fixed visual/action consumers and separately charged discovery work.

Derivative supervision and optimization. Sobolev training and Jacobian matching use derivative targets alongside values [7, 8]. Our quadratic profiles and transport fit use these established principles to describe available parts and align their local effects. PPO supplies continuous policy optimization [9]. The proposed contribution is the specified combination of profile acquisition, task-directed rebinding, interface transport and behavioral retention. It is not a claim of inventing the constituent calculus, gradient optimizer or frozen-feature learning paradigm.

Discussion and future work

The first-acquisition result makes a component runtime’s learning cycle externally observable: a task requirement and new experience lead to a retained execution without a successful reference implementation. The source parameters remain useful executable material throughout acquisition, while new interfaces determine how that material is driven and read. The transport identity makes one construction step mathematically explicit and separates its approximate profile prediction from the actual executed effect.

The present evidence concerns one simple control scenario, one source family and three initializations. It supplies no broad language-model improvement, smaller-versus-larger model comparison or unbounded-depth result. The 110 interface values are learned parameters, and both acquisition arms use substantial game experience. Fixed bindings already suffice for the same aggregate success here, with less selection work. A matched-information study of alternative proposal rules would isolate the incremental value of response/partial-guided construction.

The next substantive question is continuing acquisition: whether retained interfaces and construction knowledge reduce the cost of a second distinct behavior while preserving the first. That requires charging discovery, storage, execution, maintenance and failed reorganizations across the whole sequence. Further work can test richer construction grammars, source families, application contracts and deployment constraints. Sustainable deployment is a motivation for improving the return on existing computation; measured resource and energy savings require their own application studies.

Conclusion

Dynamic Shaping specifies how a runtime can acquire response/partial interfaces, propose component rebindings, transport their interfaces and retain working execution over fixed pretrained parts. Its transport step has an exact output/sensitivity identity. The supplied game-control study demonstrates first acquisition without a competent reference, with complete controls, exact replay and lifecycle accounting. This establishes a concrete method and experimental foundation for studying subsequent acquisition and reuse.

Executed scalar response and native partials

For component jj, the supplied compiled arrays define

ρj(s)=max⁡(n0+2sn1+sQsT+ϵj,10−20),(13)\rho_j(s)=\sqrt{\max(n_0+2sn_1+sQs^{T}+\epsilon_j,10^{-20})}, \tag*{(13)}
gj(s)=(g0j+sGjT)/ρj(s),uj(s)=(u0j+sUjT)/ρj(s),(14)g_j(s)=(g_{0j}+sG_j^{T})/\rho_j(s),\qquad u_j(s)=(u_{0j}+sU_j^{T})/\rho_j(s), \tag*{(14)}
Fj(s)=gj(s)sigmoid⁡(gj(s))uj(s),(15)F_j(s)=g_j(s)\operatorname{sigmoid}(g_j(s))u_j(s), \tag*{(15)}
ϕj(s)=tanh⁡((Fj(s)−mj)/dj).(16)\phi_j(s)=\tanh((F_j(s)-m_j)/d_j). \tag*{(16)}

mj,djm_j,d_j are fixed source centering/scaling arrays; QQ is the compiled symmetric normalization geometry. The source bank, including this normalization, is immutable. On the positive, unclamped branch,

∂sbρj=(n1b+(sQ)b)/ρj,(17)\partial_{s_b}\rho_j=(n_{1b}+(sQ)_b)/\rho_j, \tag*{(17)}
∂sbgj=(Gjb−gj∂sbρj)/ρj,(18)\partial_{s_b}g_j=(G_{jb}-g_j\partial_{s_b}\rho_j)/\rho_j, \tag*{(18)}
∂sbuj=(Ujb−uj∂sbρj)/ρj,(19)\partial_{s_b}u_j=(U_{jb}-u_j\partial_{s_b}\rho_j)/\rho_j, \tag*{(19)}
∂sbFj=[σ(gj)+gjσ(gj)(1−σ(gj))]uj∂sbgj+gjσ(gj)∂sbuj,(20)\partial_{s_b}F_j=[\sigma(g_j)+g_j\sigma(g_j)(1-\sigma(g_j))]u_j\partial_{s_b}g_j+g_j\sigma(g_j)\partial_{s_b}u_j, \tag*{(20)}
∂sbϕj=(1−ϕj2)∂sbFj/dj.(21)\partial_{s_b}\phi_j=(1-\phi_j^2)\partial_{s_b}F_j/d_j. \tag*{(21)}

The implementation clips the sigmoid exponential argument to [−60,60][-60,60] for numerical stability. Its recorded gradient checks validate the exercised numerical regime. Native source-coordinate fidelity is inherited from the released materialization and reconstruction checks [11, 12]; the present experiment adds interface-gradient checks, immutability checks and replay.

Implementation correspondence and acquisition details

The pinned Checkpoint 8 artifact has five canonical implementation files in its code/ directory: construct.py, effects.py, substrate.py, environment.py and verify.py. In construct.py, the forward method of Construction implements Eqs. (1)–(2). The gradient and update routines implement continuous acquisition. The proposal routine implements Eqs. (5)–(7); the rebinding routine implements Eq. (8). The training entry point applies development acceptance and seals selected arrays. Terminal testing runs after that seal; verification performs no fitting.

For Eq. (7), the rollout supplies actions ata_t and advantage estimates. Advantages are standardized again on the 384 profiling states. With current post-update probabilities, define the row vector

dt=−A^tN(onehot⁡(at)−πt),gt,:=(dtWT)AT.(22)d_t = -\frac{\widehat{A}_t}{N}\left(\operatorname{onehot}(a_t)-\pi_t\right), \qquad g_{t,:} = \left(d_t W^{T}\right)A^{T}. \tag*{(22)}

This is the likelihood-gradient proxy used for proposals, rather than the clipped PPO gradient or an unbiased new-policy rollout estimate. The recorded rollout precedes the six continuous-optimization epochs. The score is used only to allocate candidate evaluations; the actual game feedback decides acceptance. Each round ranks 4(24−4)=804(24-4)=80 slot/component proposals and evaluates three.

The quadratic profile has one constant, nine linear, nine diagonal-quadratic and 36 cross terms per component. Response labels are centered and scaled with standard deviation floored at 10−410^{-4}; partial labels use the same output scale. Response normal equations average over 256 states; derivative normal equations average over 256×9256 \times9 entries. Ridge 10−410^{-4} applies to nonconstant coefficients. The same coefficients generate values and partials. Check-point response RMSE across the 24 joint-run profile rounds ranges approximately 0.060–0.181, and normalized-input partial RMSE 0.051–0.163. These are local approximation diagnostics, not task-success certificates or checks on independently drawn game episodes.

Development acceptance maximizes the pair (success rate, mean return), using candidate index to break ties, and requires improvement over the current pair. Across checkpoints, final snapshot selection instead maximizes success rate plus 10−410^{-4} times mean return. These two rules are deliberately reported separately. All selections use the same development partition; there is no statistical guarantee that a development improvement generalizes.

The artifact’s results.json contains all six controller outcomes and per-run costs. The separate cost-ledger.json adds terminal and diagnostic work, and MANIFEST.json hashes the included files. The study distinguishes policy array bytes, source inventory, temporary calibration arrays, discovery archives and process RSS. Replaying or regenerating an acquisition is verification work, not counted as an extra independent controller seed.

References

  1. [1]G. F. Elsayed, I. Goodfellow and J. Sohl-Dickstein. Adversarial Reprogramming of Neural Networks. ICLR, 2019. https://arxiv.org/abs/1806.11146.
  2. [2]K. Lu, A. Grover, P. Abbeel and I. Mordatch. Pretrained Transformers as Universal Computation Engines. 2021. https://arxiv.org/abs/2103.05247.
  3. [3]A. Bektursun. Borrowed Geometry: Cross-Distribution Head-Importance Fingerprints of Frozen Pretrained Gemma 4 31B. 2026, version 2. https://arxiv.org/abs/2605.00333v2.
  4. [4]C. Fernando, D. Banarse, C. Blundell, Y. Zwols, D. Ha, A. A. Rusu, A. Pritzel and D. Wierstra. PathNet: Evolution Channels Gradient Descent in Super Neural Networks. 2017. https://arxiv.org/abs/1701.08734.
  5. [5]Y. Bansal, P. Nakkiran and B. Barak. Revisiting Model Stitching to Compare Neural Representations. NeurIPS, 2021. https://arxiv.org/abs/2106.07682.
  6. [6]A. Gaier and D. Ha. Weight Agnostic Neural Networks. NeurIPS, 2019. https://arxiv.org/abs/1906.04358.
  7. [7]W. M. Czarnecki, S. Osindero, M. Jaderberg, G. Swirszcz and R. Pascanu. Sobolev Training for Neural Networks. NeurIPS, 2017. https://arxiv.org/abs/1706.04859.
  8. [8]S. Srinivas and F. Fleuret. Knowledge Transfer with Jacobian Matching. ICML, 2018. https://arxiv.org/abs/1803.00443.
  9. [9]J. Schulman, F. Wolski, P. Dhariwal, A. Radford and O. Klimov. Proximal Policy Optimization Algorithms. 2017. https://arxiv.org/abs/1707.06347.
  10. [10]M. Kempka, M. Wydmuch, G. Runc, J. Toczek and W. Jaskowski. ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning. IEEE CIG, 2016. https://arxiv.org/abs/1605.02097.
  11. [11]N. Oh and P. Attie. Shaping Gap Benchmark V: Learned Component Execution Beyond the Transformer Graph. Public research artifact, 2026. https://github.com/calysteon/shaping-gap-benchmark/tree/04870c5/checkpoints/05-component-learning-cycle.
  12. [12]N. Oh and P. Attie. Shaping Gap Benchmark VII: Autonomous Construction and Reconstruction of Missing Capability. Public research artifact, 2026. https://github.com/calysteon/shaping-gap-benchmark/tree/ee53d4ea2bcc742ba5e7b29d7af431b0c0858bd6/checkpoints/07-autonomous-capability-construction.
  13. [13]N. Oh and P. Attie. Shaping Gap Benchmark, Checkpoint 8: First Acquisition of Missing Capability Without a Successful Reference. Code, protocols and complete evidence, 2026. https://github.com/calysteon/shaping-gap-benchmark/tree/73971c5a31f2acbfb7ea1e8131c456964e6e1826/checkpoints/08-contract-driven-capability-construction.

Paper details

vlnc:W_g2vgf9pp
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents