Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training

Introduction

An agent’s behavior depends jointly on its model policy and its harness: the prompts, tool interfaces, context and memory mechanisms, and execution rules that govern interaction with the environment. Even with model weights held fixed, the harness shapes the information the policy receives and how its decisions become actions. ReAct demonstrates the value of interleaving reasoning and action [65], while SWE-agent shows how the design of the agent–computer interface affects task execution [63]. Recent methods automate the optimization of prompts, workflows, and harness code from execution feedback [23, 76, 2, 30]. A central question is whether specialized execution support can improve the policy itself, so that its benefits persist under the original harness.

We study harness evolution from a common base $H_0$ within a single execution system, a setting motivated by recent work such as Meta-Harness [30], HarnessX [12], and Life-Harness [61]. We call the resulting variants sibling harnesses: they share the execution foundation of a complete system, such as DeepSeek Harness [15], while modifying how it supports the policy. Within this setting, we observe three properties:

• Locality. Harness edits often repair a subset of failures while introducing regressions on previously solved tasks (Figure 1a). Their effects depend on the failures being targeted, so successive variants develop distinct strengths.

Vector (PDF)

• Complementarity. Sibling harnesses can recover one another’s errors. Their oracle union covers more tasks than individual variants or repeated sampling with $H_0$ at the same rollout count (Figure 1b), showing the benefit of diverse execution behaviors.

• Nonstationarity. A harness’s utility changes as the executing policy learns. Its advantage over $H_0$ can become a disadvantage, and the ranking of variants can reverse across policy checkpoints (Figure 1c).

These properties expose two limitations of harness optimization. (i) Combining local repairs into a universal harness is extremely difficult. Repairs may impose incompatible execution requirements, and even compatible edits can interfere with one another. A single accumulated design can therefore lose the strengths of its constituent variants. (ii) Coupled harness–policy evolution incurs substantial evolution cost and limits harness reuse. Alternating harness and policy updates [12 c] requires repeated generation and validation of variants as earlier repairs become less useful to the changing policy. Gains achieved by the evolved pair may also fail to transfer when the policy returns to the base harness $H_0$. We introduce Harness-S to selectively rescue failures of the base system with specialized sibling harnesses and transfer the resulting capabilities into a single policy that remains effective under the original $H_0$.

Directional Harness Evolution first develops variants along distinct repair directions. Starting from $H_0$, we diagnose failures in execution trajectories and let each child harness target a limited set of related causes. Joint validation coverage selects the next parent set; all executable, protocol-compliant variants are retained. The resulting large, reusable Sibling Harness Pool preserves distinct repair specializations and remains fixed throughout policy training.

Complementary Harness Rescue Training uses this pool to expand exploration on tasks the policy fails to solve with $H_0$. When all initial rollouts for a task fail, the Adaptive Harness Router selects the top-$k$ auxiliary harnesses, and the same policy generates fresh rollouts under them from the task’s initial state. These rescue rollouts replace the failed group for reinforcement learning, allowing the shared policy to learn from its own actions under complementary execution support. The objective is to retain the acquired behaviors when the policy returns to $H_0$.

The Adaptive Harness Router matches tasks to sibling harnesses using behavioral evidence and independent votes from multiple LLMs. Harness Behavior Profiles describe the task conditions under which each variant helps or hinders the current policy relative to $H_0$. To account for nonstationarity, policy-aware profile re-anchoring periodically refreshes the profiles from current-policy validation trajectories, keeping routing aligned with the policy as it learns.

Our contributions are fourfold:

• We systematically identify Locality, Complementarity, and Nonstationarity in harness evolution and introduce Harness-S to learn complementary sibling capabilities in a single policy that remains effective under the original $H_0$.

• We go beyond the best harness with Directional Harness Evolution: a pipeline that branches along distinct repair directions and preserves complementary variants in a reusable Sibling Harness Pool.

• Complementary Harness Rescue Training restores learning signals when all initial $H_0$ attempts fail. The Adaptive Harness Router selects complementary siblings to recover successful alternatives and within-task reward contrasts for shared-policy optimization.

• Across AppWorld, SWE-Gym, and DBBench, Harness-S outperforms policy-only training under $H_0$ at both model sizes. Behavioral analyses, held-out cases, and reduced routing dependence support capability transfer; Rank-1 routing adds gains, and improvements persist under unseen harnesses without further updates.

Related Work

Harness Evolution

Execution design shapes the behavior expressed by a language model, from reasoning and action representations ([65, 60, 52]) to interaction interfaces and search ([63, 84]). Automatic optimization extends this design space through instruction refinement ([62] (Yuk-sekgunol et al. 2024)), evolutionary prompt search ([18, 20]), and program compilation or reflective selection ([27, 2]). Memory, skill libraries, and context adaptation provide another route to improving execution from experience ([47, 50, 54, 81]). At the system level, agent design and workflow search optimize executable structures ([23, 76]), while code evolution, end-to-end harness optimization, and runtime adaptation develop broader interventions ([75, 30, 61]). Harness-S uses directional evolution to retain complementary local repairs in a reusable sibling pool. The implementations remain fixed during policy training; their behavioral profiles are refreshed to track changes in utility as the shared policy learns.

Harness–Policy Optimization

Reinforcement learning improves agent behavior through outcome feedback ([46, 53, 17, 56]), supported by executable environments and training interfaces ([38, 34]). Interactive benchmarks cover web services ([64, 85, 66]), applications and databases ([49, 32]), and software engineering ([25]). Joint methods alternate harness and policy updates ([28, 13]), evolve matched pairs ([10]), or learn runtime coordination ([36]). Model routing ([9, 37, 11]) and response ensembles ([24, 51]) provide related approaches to allocating execution resources.

HarnessBandit schedules one harness per optimizer step using learnability and gradient-based cross-harness transferability signals [55]. HarnessX couples harness evolution with cross-harness GRPO over a shared replay buffer, grouping trajectories by task across harness and policy versions [12]. Harness-S instead asks whether specialized harnesses can selectively rescue failures of a base execution system and transfer the resulting capabilities into a single policy that remains effective under the original harness. Its allocation unit is a failed task under $H_0$: only all-failure groups trigger fresh rollouts from the current policy across selected siblings in a fixed pool. Evaluation under $H_0$ tests whether the resulting improvement survives removal of auxiliary execution support.

Recursive Self-Improvement

Recent surveys distinguish self-improvement by whether updates modify model parameters or the surrounding scaffold [39]. Program-level approaches improve optimization code or agent logic that supports subsequent improvements [73, 68, 75]. AIDE${}^{2}$ applies this loop to research agents [48], while RRSI regularizes harness edits to favor transferable mechanisms [59]. SpecForge refines behavioral specifications from execution feedback [14 )${}; self-rewarding models and SEAL instead update parameters using self-generated rewards or adaptation data {citep}][70, 86]. Harness-S uses a fixed sibling pool to rescue base-system failures and transfer the resulting behaviors into one policy; policy updates refresh the evidence used for later rescue allocation. Its self-improvement loop operates over policy parameters and experience selection, with a fixed optimizer, verifier, and training-time harness pool, rather than recursively redesigning the improvement algorithm. {xref}Appendix H` further discusses agent RL training, knowledge transfer, harness editing, and reusable skills.

Harness-S

Harness-S uses selective rescue of $H_0$ failures to transfer useful behaviors from sibling harnesses into a shared policy (Figure 2). Directional evolution builds a reusable pool of specialized variants; rescue training uses them to explore tasks that defeat the base harness. The Adaptive Harness Router selects variants using behavioral evidence that is refreshed as the policy learns, and can be reused at inference.

Vector (PDF)

Policy, Harness, and Task Outcome

A policy and its harness jointly determine how an agent executes a task. For task $x$, the policy $\pi_\theta$ generates a trajectory $\tau\sim p_{\theta,H}(\cdot\mid x)$ under harness $H$, with verifier-defined outcome $R_x(\tau) \in\{0,1\}$. The harness controls the context and execution rules through which the policy interacts with the environment. We define its utility for that policy and task as the expected success rate,

The Sibling Harness Pool $\mathcal{H} = \{H_0, H_1, \ldots, H_M\}$ contains variants descended from a common base $H_0$. All variants share the execution foundation, environment interfaces, and task verifier, and use the same policy parameters. Their relative utility varies across tasks and policy checkpoints, motivating task-specific selection that adapts as the policy learns.

Directional Harness Evolution

Starting with $\mathcal{P}_1 = \{H_0\}$, we execute each parent $H \in\mathcal{P}_g$ on fresh training tasks to derive repair directions $\mathcal{D}_g(H)$. Each direction groups related local root causes supported by multiple failed tasks; the editor receives the parent and these causes:

where $\operatorname{Edit}(H,d)$ modifies the parent’s components to address the causes in $d$.

Parent selection maximizes joint validation coverage. We select at most $B$ parents from $C_{\mathrm{valid}}$, this round’s executable, protocol-compliant children. Let $\mathcal{V}(H)$ denote the validation tasks solved by $H$ with the frozen policy:

Each task counts once, favoring parents that cover one another’s failures.

The Sibling Harness Pool retains all valid children, including those not selected as parents:

After $T$ generations, $\mathcal{H}=\mathcal{H}_T$ is the fixed pool used for policy training. Appendices E and F report repair diversity and evolution prompts and settings.

Complementary Harness Rescue Training

Training uses the pool to revisit tasks that the policy fails to solve under $H_0$. For each task $x$, the current policy generates $K$ rollouts under $H_0$, forming $\mathcal{G}_0(x)$. If all attempts fail, the router selects a set $\mathcal{S}^{(s)}_K(x)$ of distinct auxiliary harnesses using the latest calibration $s$ (Equation (10)). The same policy makes one fresh attempt under each selected harness from the task’s initial state, yielding $\mathcal{G}_R(x)$. The group retained for training is

Complementarity gives the policy alternative paths through difficult tasks. Rescue groups containing successes and failures provide reward contrasts absent from the initial attempts. For a retained group of trajectories $\tau_{x,j}$, GRPO ([46]) assigns the relative advantage

Here $\bar{r}_x$ and $\sigma_x$ are the group’s reward mean and standard deviation, and $\epsilon>0$ stabilizes division. The shared optimizer forms training advantages $A_{x,j}$ from these group-relative rewards and maximizes the clipped token objective

Here $y$ is a policy token in its generating harness context $c_H$. The ratio compares current parameters $\theta$ with preupdate parameters $\theta_{\mathrm{old}}$; $\eta_-,\eta_+$ are the clipping widths. Only policy tokens enter the objective. Algorithm 1 summarizes training; optimizer details appear in Appendix § 7.

Adaptive Harness Router

Harness Behavior Profiles describe when a sibling helps or hinders the current policy:

An LLM groups behavioral differences from paired validation trajectories into entries $e$, where $w$ specifies the task condition and $b$ the behavior relative to $H_0$ (Appendix § 9).

Voting aggregates independent LLM rankings. Each of $V$ voters uses the task, current profiles, validation success rates, and outcomes on similar tasks:

Algorithm 1 Complementary Harness Rescue Training

Require: Policy $\pi_{\theta}$; fixed pool $\mathcal{H}$; validation tasks

1. Construct profiles and validation evidence

2. for each training batch do

3.   Re-anchor profiles if calibration is due

4.   for each task $x$ in the batch do

5.     Sample $K$ rollouts under $H_0$ as $\mathcal{G}_0(x)$

6.     if all rollouts fail then

7.       Select $S_K^R(x)$ with the router

8.       Form $\mathcal{G}_R(x)$ with one fresh rollout per harness

9.     end if

10.     Retain $\mathcal{G}(x)$ using Equation (5)

11.   end for

12.   Update $\theta$ on retained groups using GRPO

13. end for

14. Re-anchor profiles with the final policy

Here $\omega$ decreases with rank and is zero for unlisted candidates; the selected $K$ distinct harnesses generate $\mathcal{G}_R(x)$ in Equation (5).

Policy-aware profile re-anchoring refreshes the router’s evidence at checkpoints $s$. With policy $\pi_{\theta_s}$ frozen, we collect validation trajectories and outcomes $\mathcal{E}_i^{(s)}$ under each $H_i$ and recompute

Paired executions share tasks, initial states, and execution limits. Profiles and validation statistics are refreshed together and used until the next checkpoint; test outcomes are excluded.

Knowledge Transfer to Policy

The trained policy $\pi_{\theta_*}$ consolidates behaviors from sibling harnesses through rescue updates; these behaviors can persist under the original $H_0$. The behavioral cases in RQ4 and reduced dependence on auxiliary harnesses in RQ5 (§ 4.5 and § 4.6) support this transfer.

Rank-1 routing adds further gains to the same trained policy. After final profile re-anchoring, we select from the full pool, including $H_0$:

Selection precedes execution at a matched rollout count; Appendix § 8 details voting.

Experiments

We examine whether complementary rescue improves agent performance and transfers useful behaviors into the policy:

• RQ1: Does Harness-S improve task success, including under the original $H_0$?

• RQ2: Does complementary rescue provide better learning signals than resampling or fixed harness mixtures?

• RQ3: How do rescue allocation and profile re-anchoring affect recovery and final performance?

• RQ4: Which behaviors change under $H_0$, and do rescue behaviors transfer to untrained instances?

• RQ5: Do router ranks predict performance, and how does routing benefit change after training?

• RQ6: Do policy improvements transfer to unseen harnesses without further updates?

Experimental Setup

We evaluate Qwen3.5-4B and Qwen3.5-9B on AppWorld (Trivedi et al., [49]), DBBench (Liu et al., [32]), and SWE-Gym (Pan et al., [38]), covering application APIs, databases, and software engineering. Harness editing and policy optimization use training tasks; validation tasks guide complementary parent selection and profile anchoring. Pass@3 measures the fraction of tasks solved at least once, and Avg@3 averages success over three rollouts per task. Main-table scores are means over three independent randomized runs, with across-run standard deviations for Harness-S. Routed evaluation selects one harness before each task’s three attempts. All experiments use eight NVIDIA H800 GPUs.

Our comparisons distinguish policy learning, harness evolution, and joint harness–policy adaptation (Table 1). The GRPO controls include $H_0$ rescue and fixed mixtures with Claude Code or sibling harnesses, all evaluated under $H_0$. WHALE uses rejection-sampling fine-tuning and HarnessForge adapts branch-specific policies; the GRPO controls share our optimizer. Meta-Harness, WHALE, and HarnessForge use one selected $H_{\mathrm{best}}$ across test tasks; $H_{\mathrm{rank1}}$ denotes task-specific selection. Harness-S (HE only) freezes the starting policy, while full Harness-S evaluates the same trained checkpoint under $H_0$ and $H_{\mathrm{rank1}}$ to distinguish retained policy improvement from inference-time support.

Table 1 Main results on long-horizon agent tasks. All scores are percentages; higher means are better. Bold marks the best mean at each capacity among the compared non-frontier methods. $H_{\mathrm{best}}$ is one selected harness held fixed across test tasks; $H_{\mathrm{rank1}}$ is the router’s top-ranked harness selected separately for each task. DSH denotes DeepSeek Harness. Scores are averaged over three independent randomized runs, each with three executor rollouts per task. Harness-S entries report mean $\pm$ standard deviation across these runs.

Model / method	Param.	Test harness	AppWorld Pass@3	AppWorld Avg@3	SWE-Gym Pass@3	SWE-Gym Avg@3	DBBench Pass@3	DBBench Avg@3
Frontier models
GPT-6-Sol	—	$H_0$	82.82	76.34	53.13	47.50	71.33	68.89
DeepSeek-v4-Flash	—	$H_0$	81.68	77.61	51.88	46.25	75.00	70.44
GLM-5.3-Flash	—	$H_0$	80.15	65.65	46.88	43.13	76.67	72.11
Base model and policy-only training
Base model	4B	$H_0$	42.62	31.93	19.38	17.50	59.00	56.89
	9B		50.00	39.69	23.13	20.63	63.00	60.22
GRPO	4B	$H_0$	60.31	48.98	24.38	22.50	66.67	64.00
	9B		64.89	53.31	28.13	25.63	69.67	67.00
GRPO ($H_0$ rescue)	4B	$H_0$	61.83	51.91	25.63	23.33	67.33	64.67
	9B		65.65	54.83	29.38	26.88	70.67	67.78
GRPO Mixed Harness (DSH $H_0$ + Claude Code)	4B	$H_0$	57.63	46.69	25.63	23.33	65.67	62.78
	9B		62.21	50.89	29.38	26.88	68.67	65.89
GRPO Mixed Harness (DSH $H_0$ + Top-4)	4B	$H_0$	58.78	47.84	26.88	24.38	64.67	61.89
	9B		63.36	52.16	30.63	27.71	67.67	65.00
Frozen policy + harness evolution
Meta-Harness	4B	$H_{\mathrm{best}}$	48.09	36.26	21.88	20.00	62.67	60.11
	9B		53.44	42.88	25.63	23.13	65.67	62.78
Harness-S (HE only)	4B	$H_{\mathrm{rank1}}$	56.87	45.42	25.00	23.75	69.33	66.78
	9B		61.45	50.13	29.38	27.08	72.67	69.78
Policy–harness co-evolution
WHALE	4B	$H_{\mathrm{best}}$	63.36	52.93	29.38	26.88	70.67	68.11
	9B		68.70	58.02	33.75	30.63	74.00	71.11
HarnessForge	4B	$H_{\mathrm{best}}$	59.54	49.75	27.50	25.00	69.00	66.22
	9B		64.89	54.83	31.25	28.54	72.00	69.22
Harness-S: offline evolution + policy training
Harness-S	4B	$H_0$	$67.56 \pm1.87$	$56.87 \pm1.47$	$38.75 \pm2.22$	$28.75 \pm1.73$	$73.33 \pm2.36$	$71.00 \pm1.84$
	9B		$72.52 \pm1.65$	$62.34 \pm1.28$	$40.00 \pm2.04$	$32.50 \pm1.51$	$76.33 \pm2.13$	$73.44 \pm1.67$
Harness-S	4B	$H_{\mathrm{rank1}}$	$68.32 \pm1.74$	$61.45 \pm1.35$	$42.50 \pm2.04$	$32.29 \pm1.64$	$75.00 \pm2.23$	$72.67 \pm1.75$
	9B		$75.19 \pm1.56$	$67.05 \pm1.20$	$43.75 \pm1.84$	$36.25 \pm1.45$	$78.00 \pm1.96$	$75.67 \pm1.55$

GRPO with $H_0$ rescue and Harness-S share the all-failure trigger and per-task rescue allocation; their total sampling differs with rescue frequency. Dataset splits and implementation protocols appear in Appendices § 6 and § 8, with costs reported in Appendix § 13.

Overall Performance (RQ1)

Harness-S outperforms policy-only training, frozen-policy harness optimization, and joint adaptation under both $H_0$ and $H_{\mathrm{rank1}}$ (Table 1). The routed 4B policy exceeds 9B GRPO with $H_0$ rescue on every reported metric; the routed 9B policy also exceeds the frontier-model references on DBBench.

The gains combine retained policy improvement with better execution support. Harness-S outperforms plain GRPO and GRPO with $H_0$ rescue across all benchmark–capacity settings under $H_0$, indicating a benefit beyond additional sampling. With the policy frozen, Harness-S (HE only) outperforms Meta-Harness, supporting the value of the sibling pool and task-specific selection. Routing the trained policy further improves both metrics in every setting without increasing the executor rollout count. On AppWorld 4B, Harness-S outperforms $H_0$ rescue with fewer rollouts and less training GPU time (Appendix § 13). Its full-pipeline GPU hours and external-model token use fall within the range of the joint-adaptation baselines.

Learning Mechanism (RQ2)

Selective rescue supplies more within-task reward contrast than additional sampling or fixed harness mixtures. Figure 3 compares Qwen3.5-4B on AppWorld under GRPO, $H_0$ rescue, the two fixed mixtures, and Harness-S. The Effective Group (EG) rate measures the fraction of retained groups containing both successes and failures. Harness-S leads in every stage, reaching 79.38% over the full run versus 70.94% for GRPO with $H_0$ rescue and 64.69% for standard GRPO. All-failure groups provide no within-task reward contrast; recovering some of these tasks supplies alternatives that GRPO can reinforce. Both fixed mixtures remain close to standard GRPO, suggesting that harness diversity is more useful when directed toward tasks on which the base group fails.

Vector (PDF)

Effective groups also account for a larger share of the retained training tokens. The effective training-token share counts policy tokens in effective groups relative to all retained policy tokens, accounting for differences in trajectory length. Harness-S has the highest share throughout training, reaching 80.88% overall compared with 72.68% for GRPO with $H_0$ rescue. Its advantage therefore extends to the tokens used for optimization, accounting for the lengths of successful and unsuccessful trajectories.

These training advantages accompany stronger policy performance under $H_0$. From the same starting policy, Harness-S leads at every subsequent validation checkpoint and finishes at 63.33%, compared with 54.00% for GRPO with $H_0$ rescue and 48.00% for GRPO. Both fixed mixtures finish below GRPO despite intermediate gains from the

Claude Code mixture. The results support selective rescue as a source of learning signals whose benefits persist after auxiliary harness support is removed.

Rescue Strategies (RQ3)

Task-specific allocation and profile refresh improve rescue at a fixed rollout budget (AppWorld, Qwen3.5-4B; Table 2). Random Harness samples pool members, while Rule Router (without Profiles) ranks them by success on similar validation tasks. LLM Router ($4 \times2$) allocates two rollouts to each of four selected harnesses; Top-8 allocates one to each of eight, keeping the rescue rollout budget fixed. The fixed-profile Top-8 variant keeps its profiles unchanged, while Harness-S re-anchors them during training. Rescue success measures the fraction of rescue-triggering tasks with at least one successful rollout (Appendix § 8).

	GRPO	Random Harness	Rule Router (w/o Profile)	Top-4 ×2	Top-8 Fixed	Top-8 Adaptive
Rescue success by training stage
Steps 1–10	—	25.00	40.00	43.48	35.71	52.38
Steps 11–20	—	18.18	26.67	43.75	24.24	66.67
Steps 21–30	—	20.00	27.78	38.89	26.67	45.71
Steps 31–40	—	25.00	44.44	33.33	38.89	50.00
Final test performance
Pass@3	60.31	62.98	66.41	65.65	64.50	67.56
Avg@3	48.98	50.76	52.93	55.60	52.16	56.87

Task-specific selection improves rescue success across training stages. Both rule-based and LLM-guided strategies outperform random selection throughout training. Top-8 Adaptive also exceeds LLM Router ($4 \times2$), suggesting that broader harness exploration recovers more failures than repeated sampling from fewer harnesses. Under the same final AppWorld policy, individual siblings repair 11–15 base-harness failures, while the nine analyzed siblings repair 29 in union (Appendix § 11). This is coverage obtained by executing each sibling once; routing must select candidates before observing their outcomes.

Adaptive profiles improve routing over a fixed-profile Top-8 variant. With the rollout allocation held at one attempt per selected harness, re-anchoring yields higher rescue success in all four stages and improves final Avg@3 by 4.71 percentage points. This comparison supports updating the router’s behavioral evidence as the policy learns.

All evaluated rescue strategies improve final test performance over GRPO, with Top-8 Adaptive achieving the highest Pass@3 and Avg@3. The remaining strategies show a tradeoff: Rule Router achieves higher Pass@3 than LLM Router ($4 \times2$), but lower Avg@3. Rescue success therefore does not fully determine the downstream ranking, although Top-8 performs best on both recovery during training and final test success.

Policy Behavior (RQ4)

Harness-S changes how the policy gathers evidence and completes actions under $H_0$. Table 3 compares concrete workflows of the base, GRPO, and Harness-S policies. On AppWorld, Harness-S consults API schemas before first use more often than GRPO and repeats execute code less often, while also increasing pagination, multi-call execution, and read-back after writes. These shifts suggest greater use of interface information and state verification, rather than a uniform increase in tool activity. The frequencies describe behavior, not correctness: more calls or checks are useful only when they serve the task.

Behavior	Base	GRPO	Harness-S
AppWorld
Schema lookup before first API use ↑	22.9	38.2	67.6
Multiple API calls in one execute block ↑	24.8	20.5	31.5
Explicit pagination loops ↑	13.8	33.5	38.3
Repeated execute code ↓	15.4	30.1	9.8
Read-back after the last write ↑	69.2	56.9	69.4
SWE-Gym
Agent-written assertion checks ↑	10.2	11.2	18.4
Switch to assertions after blocked tests ↑	15.4	13.3	48.5
Fail-edit-retest of the same target ↑	9.2	18.2	24.1
Diff inspection after the last edit ↑	16.3	23.5	27.5

The SWE-Gym trajectories show more explicit verification and recovery from blocked tests. Harness-S more often writes assertion checks, revisits a failed target after editing, and inspects the final diff. When tests are blocked, switching to assertions occurs in 48.5% of Harness-S cases versus 13.3% for GRPO. These patterns describe alternative ways to validate a patch when the standard test workflow is unavailable.

Successful rescue behaviors also reappear on untrained instances after auxiliary support is removed. Table 4 traces two AppWorld cases from successful training rescues to held-out execution under $H_0$. The trained policy preserves a set-intersection constraint and completes a feedback-dependent account-recovery workflow where GRPO fails. The first case concerns applying the same selection rule to new task instances; the second carries a recovery procedure from Venmo to Amazon, including password reset and verification. They illustrate transfer of both a task constraint and a multi-step procedure across distinct executions. Both successful rescues entered a policy update, and the corresponding behavior later appears on a different instance under $H_0$. These cases connect auxiliary execution experience to behavior retained by the policy after the specialized support is removed.

Stage	Observed execution
Case 1: Set intersection (held-out test instance)
Training rescue	H2 selects liked songs that also occur in the library or playlists, then updates and sends the draft. This successful rollout is used at update 10.
GRPO	Uses a union of liked songs and the other sources, sending an overinclusive list. Failure.
Harness-S	Applies the intersection to the new song instances and sends the correct email. Success.
Case 2: Account recovery (held-out validation instance)
Training rescue	H5 completes a Venmo account workflow: reads email codes, resets the password, and verifies login. This successful rollout is used at update 3.
GRPO	On the Amazon instance, requests a reset code and reads the inbox but leaves the password unchanged. Failure.
Harness-S	Reads the new codes, resets the Amazon password, and verifies access with the new credentials. Success.

Router Effectiveness (RQ5)

The router’s ranking predicts execution quality before and after training. Table 5 compares Qwen3.5-4B under $H_0$ and three task-specific router ranks, each evaluated with three rollouts per task. Both Pass@3 and Avg@3 decrease from Rank-1 to Rank-3 before and after training. All three ranked choices outperform $H_0$ before training, showing the value of specialized execution support for the initial policy. The preserved ordering indicates that routing remains useful as the policy improves.

Harness	Before training			After training
	Pass@3	Avg@3	$H_0$ rate	Pass@3	Avg@3	$H_0$ rate
$H_0$	42.62	31.93	—	67.56	56.87	—
Rank-1	56.87	45.42	1.53	68.32	61.45	22.52
Rank-2	53.44	42.11	6.87	66.03	55.60	14.89
Rank-3	50.38	39.19	4.96	64.89	54.20	13.36

Training makes $H_0$ a more frequent leading choice. The fraction of tasks placing $H_0$ at Rank-1 rises from 1.53% to 22.52%; its selection rates at Rank-2 and Rank-3 also increase. The largest increase occurs at Rank-1, showing that $H_0$ more often becomes the router’s preferred execution choice. This increased preference for the base harness is consistent with reduced dependence on auxiliary execution support.

The trained policy retains much of the routed advantage under the original harness. The Rank-1 margin over $H_0$ shrinks from 14.21 to 0.76 points in Pass@3 and from 13.49 to 4.58 points in Avg@3. The trained policy under $H_0$ also overtakes Rank-2 and Rank-3 on both metrics. Together with the cases in § 4.5, these performance and routing changes support the internalization of behaviors elicited during rescue. Rank-1 retains an advantage in the reported means; its larger remaining Avg@3 margin suggests that task-specific execution support still improves consistency across attempts.

Cross-Harness Generalization (RQ6)

Harness-S retains its performance advantage under external execution harnesses without further policy updates. Figure 4 compares the Qwen3.5-4B policies on AppWorld and SWE-Gym under DSH $H_0$, Claude Code, and OpenClaw. Harness-S achieves the highest Pass@3 across all evaluated benchmarks and execution harnesses.

Vector (PDF)

Cross-harness transfer preserves the gains without eliminating sensitivity to the execution interface. Harness-S also exceeds mixed-harness GRPO under Claude Code and OpenClaw, indicating that fixed mixing is insufficient to match its transfer performance. Harness-S nevertheless scores below DSH $H_0$ on both external harnesses and benchmarks. The results suggest that complementary rescue develops behaviors useful beyond the training harness, while execution support continues to influence how reliably those behaviors succeed.

Conclusion and Future Directions

Harness-S trains a shared policy to retain capabilities elicited by complementary sibling harnesses under the original $H_0$. Directional evolution preserves distinct repair strategies, while selective rescue recovers learning signals from base-harness failures. Across AppWorld, SWE-Gym, and DBBench, gains persist under $H_0$; behavioral analyses support transfer, and Rank-1 routing improves inference. Future work will study lasting transfer and efficient pool reuse across policies. We will study how independently developed harnesses exchange task state across execution interfaces. Harness Scaling experiments will measure how additional repair diversity affects policy gains and calibration cost.

Dataset Splits

Table 6 lists task-instance counts. Harness editing and policy optimization use training tasks; parent selection and profile anchoring use validation tasks. Test tasks are reserved for final evaluation.

Benchmark	Train	Validation	Test	Total
AppWorld	320	150	262	732
DBBench	1,600	200	300	2,100
SWE-Gym	240	100	160	500

Shared Policy Optimization

The baseline and Harness-S share the same optimizer, so their learning objectives differ only through the trajectories retained for each task. Each retained group contains $G = K = 8$ trajectories. Equation (6) first centers rewards within the task and normalizes them using population standard deviation and $\epsilon= 10^{-6}$. For a batch $\mathcal{B}$, let $\mathcal{T}_{x,j}$ contain the valid policy-token indices and $N_x = \sum_j \lvert\mathcal{T}_{x,j}\rvert$. Using the generating context $c_{H_{x,j,\ell}}$, the PPO-style objective ([44]) is

Here $\ell$ is the clipped objective in Equation (7). The loss averages over valid policy tokens within each task, then equally over tasks in the batch; trajectories of different lengths therefore need not receive equal weight. Harness-injected instructions and tool observations are excluded from the token loss. Each auxiliary trajectory retains its generating context when computing the probability ratio. Tasks with $N_x = 0$ contribute zero.

The rollout and training backends can assign different probabilities to the same sampled token. For recorded rollout token probability $q_{x,j,\ell}$, the detached correction is

The detached weight $w$ multiplies the token objective without receiving gradients. Policy-ratio clipping uses $\eta_{-} = 0.20$ and $\eta_{+} = 0.28$, with no reference-model KL term.

Implementation of Comparison Variants

The adaptations start from the size-matched Qwen3.5 policy and common base harness. Environment interfaces and task verifiers remain unchanged; each method retains its own search organization and policy-update objective. $H_{\mathrm{best}}$ is fixed across test tasks, whereas $H_{\mathrm{rank1}}$ is selected separately for each task. The base model denotes the unmodified starting policy; frontier models evaluated under $H_0$ serve as additional references.

Evaluation protocol. Within each run, each test task receives three executor rollouts. Pass@3 measures the fraction of tasks with at least one successful rollout, while Avg@3 averages success over the three attempts. Main-table scores are means over three independent randomized runs; the reported Harness-S standard deviations are computed across runs. For the router-rank comparison, Rank-$r$ denotes the candidate at position $r$ for each task, evaluated separately with the same rollout count.

GRPO and $H_0$ rescue. Both use the shared optimizer in Appendix § 7 and execute under $H_0$. Plain GRPO retains each initial group; the rescue control replaces an all-failure group with fresh $H_0$ rollouts using the same per-task allocation as Harness-S. Total sampling depends on rescue frequency, which differs between policies, and includes both discarded base attempts and rescue rollouts.

Rescue allocation variants. All rescue groups contain eight rollouts. Random Harness samples pool members. Rule Router ranks candidates by success on similar validation tasks, without using profiles. LLM Router ($4 \times2$) allocates two rollouts to each of four selected harnesses; Top-8 allocates one to each of eight. The fixed-profile Top-8 variant keeps its profiles unchanged, while Harness-S refreshes them at calibration checkpoints. Rescue success is the fraction of rescue-triggering tasks for which at least one fresh attempt succeeds.

Fixed mixed-harness GRPO. The Claude Code mixture allocates four rollouts to DSH $H_0$ and four to Claude Code. The Top-4 mixture combines DSH $H_0$ with four selected siblings throughout training. The same mixture is used across training tasks, without waiting for an all-failure base group. These controls expose one shared policy to diverse harness contexts throughout training. Only policy parameters are updated, and both policies are evaluated under $H_0$.

Meta-Harness. With the policy frozen, the proposer uses previous implementations, scores, and trajectories to revise harness code [30]. The adaptation searches within the common execution framework, preserving environment interfaces and the task verifier. Candidates are checked for compatibility and evaluated on training tasks. A single selected $H_{\mathrm{best}}$ is used across test tasks.

WHALE. The adaptation alternates Meta-Harness-style search with rejection-sampling fine-tuning on successful trajectories [28]. Successful trajectories supply the policy update, after which harness search proceeds with the updated policy. This preserves the alternation between model adaptation and execution-support optimization. Final evaluation uses the selected policy and its fixed $H_{\mathrm{best}}$.

HarnessForge. Failure analysis guides harness changes, and successful search trajectories train the corresponding branch policy [10]. Branches retain separate policy adaptations and the provenance of their training trajectories. As recorded in Table 10, training reuses successful search trajectories, so zero additional training rollouts does not mean that the policy remains frozen. The final selected pair executes with a fixed $H_{\mathrm{best}}$.

Method	Search rollouts	Training rollouts	GPU·h	API tokens (M)
Meta-Harness	8,000	0	35.24	4.89
WHALE	8,640	2,560	95.84	9.80
HarnessForge	5,680	0	65.39	24.40
Harness-S	4,010	3,032	75.79	18.69

Pool and evaluation. The pool contains $H_0$ and 19 auxiliary harnesses; rescue selects eight auxiliaries and inference ranks all 20. The HE-only variant freezes the starting policy; the two trained variants share a checkpoint and differ only in test-harness selection. The nine siblings analyzed in Appendix § 11 define the scope of that calibration analysis, not the size of the routing pool.

Profile refresh. For AppWorld 4B, profiles are built before training, refreshed every ten updates, and re-anchored with the final policy. Each calibration freezes the current policy for paired validation execution. Routing uses the latest profiles between checkpoints, with all harness implementations fixed.

Inference voting. Multiple LLMs independently rank candidates using the task, current profiles, validation success rates, and outcomes on similar validation tasks. Each ballot ranks four distinct candidates with weights 1, 3/4, 1/2, and 1/4. Unlisted candidates receive zero weight. Weights are summed across ballots; ties use validation success, then harness ID. Candidate order varies across calls. The base harness uses its execution description and validation evidence. Each selected harness is fixed across the three evaluation rollouts for its task. The ranking is completed before any task outcome is observed, and failed attempts do not trigger a new inference-time selection.

Sibling Harness Edits and Behavior Profiles

Table 7 pairs three concrete configuration edits with favorable behavior-profile entries from the step-40 AppWorld calibration on Qwen3.5-4B. The policy is frozen while each sibling is compared with $H_0$ under matched validation tasks, execution limits, and task seeds. These examples use the paired calibration protocol in Appendix § 11.

Case	Edit	Configuration	Before ($H_0$)	After
Case 1: $S_6$ — complete pagination across sources	add an after-tool reminder to continue pagination until explicit exhaustion.	state.hooks.pagination-completion-guidance	null	{ "action": "warn", "condition": "pagination_present", "event": "after_tool", "message": "Pagination metadata is present. Continue only until an empty or short page or an authoritative total; do not treat the first page as complete.", "parameters": {} }
When				Aggregating information across inbox, spam, and archive.
Behavior				Paginate through all specified sources, merge the results, and remove duplicates. The paired $H_0$ execution attempts a single-pass match and omits information from some sources.
Case 2: $S_7$ — joint filtering of target records	add an action-guidance rule requiring the intersection of task conditions.	state.prompt_rules.reconcile-before-side-effect	null	{ "slot": "action_guidance", "text": "[...] If the request combines conditions, fetch each condition separately and compute the intersection; do not broaden an item to its containing collection. [...]" }
When				Finding office desks returned in the current month.
Behavior				Retrieve all return records, then jointly filter for product names containing desk and a returned_at date within the current month. The selected records satisfy both the product and time constraints.
Case 3: $S_8$ — reserving execution budget for submission	add a low-budget reminder to finish necessary checks and reserve a step for completion.	state.hooks.low-budget-completion	null	{ "action": "warn", "condition": "remaining_steps_lte", "event": "after_tool", "message": "Execution budget is low. Finish only the required verification and reserve one step for complete_task; avoid unrelated exploration.", "parameters": { "threshold": 3 } }
When				Task-specific filtering and verification are complete, and the final answer must be submitted through the dedicated API.
Behavior				Finish verification and submit the answer with complete_task. The paired $H_0$ trajectory reaches a conclusion but ends before submitting it.

Harness Behavior Profiles describe execution differences observed in complete paired trajectories. The When field identifies the task conditions, and Behavior records the corresponding execution pattern. Entries are organized into prefer and avoid lists; repeated observations are consolidated by task condition and behavior, while useful observations from individual tasks are retained. The table presents English translations of selected prefer entries alongside core excerpts of the recorded configuration changes.

The edits add guidance to the shared base execution system. A null entry means that $H_0$ has no corresponding additional rule or hook; its base instructions remain in place. The pagination and completion hooks append reminders, while the intersection rule supplies an instruction through action_guidance. Each profile describes execution under the full sibling configuration. The $S_7$ and $S_8$ examples concern the same validation task, highlighting constraint filtering and final submission, respectively.

Sibling Harness Diversity

Sibling Harness Diversity

We compare nine siblings using the final AppWorld Qwen3.5-4B policy and one greedy rollout per validation task. Each harness uses the same task initialization and execution limits. One task has an invalid paired outcome and is excluded from every sibling, leaving 149 shared tasks. For each sibling, $R_i$ contains failures of $H_0$ repaired by $S_i$. Figure 5 shows the directed overlap $\lvert R_i \cap R_j\rvert/\lvert R_i\rvert$, averaging 62.47% across distinct ordered pairs. The denominator is the row sibling’s repair set, so the matrix is directional: overlap can differ when the row and column are exchanged. High overlap indicates that another sibling supplies many of the same repairs; lower overlap identifies task coverage not shared by that pair. This comparison concerns repairs of $H_0$ failures, rather than overall success across all validation tasks. The incomplete overlap supports retaining different repair directions among these nine analyzed siblings; the routing pool contains 20 harnesses. Individually, the analyzed siblings repair 11–15 base-harness failures, while their union repairs 29. This union is oracle coverage obtained by executing each sibling once; routing must select candidates before observing their task outcomes.

Vector (PDF)

Directional Harness Evolution: Prompts and Settings

The AppWorld search freezes Qwen3.5-4B Base and uses the 320 training tasks for failure analysis, candidate editing, and local tests. Validation tasks guide complementary parent selection and profile anchoring. Table 8 summarizes the configuration. The analysis prompt requires every valid failure to be grouped by root cause or marked unresolved. The evidence determines how many groups are formed. Each candidate then targets a limited set of related causes supported by multiple failed tasks. Component choices follow this diagnosis: edits can alter prompts, skills, context, memory, hooks, or the tool view. Other candidate briefs provide context for seeking different mechanisms, triggers, or recovery strategies.

Setting	Value
Search	3 rounds; proposal budgets 7 / 8 / 8; 4 parent slots
Development tasks	Sampled from the 320-task training set
Repair direction	2–3 related root causes per candidate; at least 2 distinct failed tasks per cause
Local evaluation	At most 30 tasks, including at most 10 successful controls
Revision budget	Initial candidate plus at most one revision
Harness editor	GLM-5.3; temperature 0.6; thinking enabled with high reasoning effort
Policy sampling	Greedy decoding; one rollout per task; seed 42
Execution limits	32 model calls and 30 environment actions; 65,532 context tokens and 8,192 output tokens per call
Editable components	Prompt, Skill, Context, Memory, Hooks, Tool View

Local evaluation tests whether an edit repairs the targeted failures while preserving behavior on successful controls. Each candidate can submit an initial measured version and one revision within the task budget in Table 8. The editing agent submits an artifact containing the candidate state and its self-review, then requests static checks or measured rollouts before final submission. The execution host supplies these outcomes; candidate self-assessment does not determine success. Newly generated valid variants compete for $B = 4$ parent slots under the validation-coverage objective in Equation (3). Selected parents are paired with locally diagnosed root causes for the next round of editing, while the archive retains all executable, protocol-compliant variants. The resulting implementations remain fixed during policy training; only their profiles and allocation are updated.

Prompt excerpts. Stages receive the parent harness, complete trajectories, and scoring diagnostics; hidden scoring answers are not supplied to the executor policy. The saved instructions include:

• Attribution: “For EACH failure, follow the earliest decision or tool result explaining the failed check, then inspect the later effect.”

• Planning: “Choose among all six editable components according to the diagnosed cause; do not default to prompt/skill edits.”

• Submission: “Submit must summarize observed gains, regressions and unresolved causes; never claim unrun tests.”

Computational Efficiency

All experiments use eight NVIDIA H800 GPUs; external-model usage is reported separately in API tokens. The two tables distinguish policy-training expenditure from the complete search-and-training pipeline.

Table 9 GRPO training cost and $H_0$ test Avg@3 on AppWorld (Qwen3.5-4B). All settings use 40 updates. Parentheses give rollout counts relative to plain $H_0$ GRPO.

Setting	Training rollouts	Training GPU·h	$H_0$ test Avg@3 (%)
Plain $H_0$ GRPO	2,560 (100.00%)	50.58	48.98
GRPO ($H_0$ rescue)	3,616 (141.25%)	97.77	51.91
DSH $H_0$ + Top-4	2,560 (100.00%)	58.47	47.84
DSH $H_0$ + Claude Code	2,560 (100.00%)	50.03	46.69
Harness-S / Top-8	3,032 (118.44%)	67.19	56.87

Table 10 Full-pipeline resources for harness search and policy adaptation. Training rollouts count additional samples; HarnessForge reuses successful search trajectories. API tokens are in millions (M).

Training costs include unsuccessful attempts as well as retained trajectories. All variants in Table 9 use 40 updates, but the rescue methods incur additional rollouts when the initial $H_0$ group fails. Harness-S improves $H_0$ Avg@3 by 7.89 points over plain GRPO with 18.44% more rollouts. It also outperforms $H_0$ rescue with fewer rescue attempts and lower GPU time. The fixed mixtures match the plain-GRPO rollout count but do not obtain the same retained-performance benefit.

Directional evolution reduces the search component of the pipeline. Table 10 reports 29–54% fewer search rollouts for Harness-S, with GPU hours and API tokens within the range of the joint-adaptation baselines. Relative to WHALE, Harness-S uses fewer GPU hours but more API tokens; relative to HarnessForge, it uses fewer API tokens but more GPU hours. HarnessForge’s zero additional training rollouts reflects reuse of successful search trajectories for policy updates. Re-anchoring supports reuse across policy checkpoints; extending the pool between training runs can amortize earlier evolution effort.

Extended Related Work

Agent RL Training

Interactive environments make action selection and observation handling part of the training problem. Toolformer learns API use through self-supervised examples, while AgentTuning initializes agent capabilities through instruction tuning on interaction trajectories [42, 74]. Such supervision can initialize tool-use behavior before reinforcement learning from task outcomes. WebGPT combines browsing demonstrations with human-preference feedback [35]. ReTool learns to interleave reasoning with code execution, while Search-R1 learns multi-turn search interactions and masks retrieved content during optimization [17, 26]. SWE-RL develops software-engineering reasoning from software evolution data with rule-based solution rewards [56]. The supervision source and action interface jointly determine which behaviors can be learned.

Long interactions introduce credit-assignment and rollout-quality problems beyond optimizing individual responses. RAGEN identifies instability associated with collapsing reward variation and studies how rollout diversity and feedback shape multi-turn learning [53]. LOOP trains agents directly in stateful environments such as AppWorld using a memory-efficient PPO variant [8]. AgentGym-RL explores a complementary curriculum, gradually increasing interaction horizons to balance early exploitation with broader exploration [58]. Agent Lightning separates execution from optimization and decomposes trajectories into transitions with assigned credit [34]. These directions distinguish the learning objective, interaction curriculum, and runtime interface through which training experience is collected.

Classical RL offers additional ways to learn when successful outcomes are rare or delayed. Hindsight Experience Replay reuses unsuccessful attempts by relabeling them with goals that were achieved [3]. RUDDER redistributes delayed rewards through return decomposition [5], whereas Go-Explore separates returning to promising states from exploring beyond them [16]. These methods intervene in goal assignment, credit distribution, and state coverage, respectively. The distinction matters for language agents: changing how experience is generated and changing what an existing trajectory teaches are different interventions.

Uniformly rewarded groups limit the signal available to group-relative updates. PPO uses a surrogate objective to control policy updates, while GRPO estimates advantages from rewards within a sampled group [44, 46]. DAPO uses dynamic sampling to retain groups with reward variation, filtering groups whose samples are all correct or all incorrect [69]. Model-aware tool-use data evolution in LoopTool and hint-conditioned generative recommendation in HCGRec address related data and exploration bottlenecks [79 b]. Harness-S changes execution support before generating fresh attempts on the same failed task. It keeps the task and verifier fixed while seeking the successful alternatives needed for policy updates.

Knowledge Transfer

The form of supervision determines what a student can inherit from a teacher. Classical knowledge distillation transfers predictive distributions [21], while Sequence-Level Knowledge Distillation trains autoregressive students with teacher-generated sequences [29]. These approaches transfer information at different granularities: relations among candidate outputs versus complete generated solutions. The distinction is relevant when successful behavior consists of a sequence of decisions rather than a single prediction.

Multiple specialists can also be consolidated into a shared decision-making model. Policy Distillation combines task-specific reinforcement-learning policies into a single network [41]. Kickstarting combines guidance from previously trained agents with continued reinforcement learning, allowing a student to benefit from several experts and improve beyond them [43]. The resulting learner can act without querying every original expert, separating the resources used to acquire a capability from those required to deploy it.

Additional computation and reasoning traces provide another source of transferable knowledge. Expert Iteration alternates tree-search planning with training a policy to generalize the resulting decisions [4]. STaR trains on successful self-generated rationales, including rationales regenerated with access to the correct answer after failure [72]. Distilling Step-by-Step uses teacher-generated rationales as additional supervision alongside task labels [22]. These approaches differ in how they obtain useful traces, but all use richer training-time computation to shape a model’s subsequent behavior.

Transfer must also account for the distribution induced by the learner. DAgger aggregates expert-labeled states visited by successive learner policies to address distribution shift in sequential imitation [40]. Generalized Knowledge Distillation similarly obtains teacher feedback on student-generated sequences [1]. MiniLLM studies on-policy optimization of reverse KL divergence to limit student probability mass on outputs that the teacher assigns low likelihood [19]. Together, these works show why the training distribution and the matching objective are separate design choices.

Teachers may additionally have information that the deployed learner will not receive. Generalized distillation connects this setting to learning with privileged information [33]; Learning by Cheating applies it to driving by training a vision-based policy from an agent with privileged environmental state [7]. VAD examines the source of multimodal teacher feedback through controlled visual-evidence interventions [78]. Harness-S instead obtains behavioral diversity by executing the same current policy under different harness contexts. Task outcomes supervise its trajectories, and evaluation under $H_0$ tests whether useful behaviors persist when auxiliary execution support is removed.

Harness Editing and Reusable Skills

The execution interface determines how a language model’s outputs become actions. ReAct interleaves reasoning and environment interaction [65]; SWE-agent and CodeAct study dedicated computer interfaces and executable code actions, respectively [63, 52]. AutoGen extends the design space to configurable conversations among agents, tools, and human participants [57]. These systems establish different control structures and action representations that later optimization procedures can modify.

Execution support can be optimized at several levels of granularity. OPRO uses evaluated solutions to guide new prompt proposals [62]. EvoPrompt applies evolutionary operators to prompt populations, while Promptbreeder also evolves the mutation prompts that generate candidates [20, 18]. DSPy compiles modular language-model pipelines against task metrics [27]. TextGrad propagates textual feedback through computation graphs, and AFlow searches code-represented workflows with execution feedback [71, 76]. Agent Optimizer treats an agent’s functions as adjustable parameters while keeping the underlying language model fixed [82]. Thus, prompt editing, workflow search, and function revision expose different parts of an execution system to optimization.

Maintaining an archive supports exploration of multiple improvement directions. ADAS searches code-defined agents using previous discoveries, and the Darwin Gödel Machine branches from an archive of executable agents whose modifications are evaluated empirically [23, 75]. GEPA combines trajectory-based reflection with complementary prompt candidates on a Pareto frontier [2]. Meta-Harness gives its proposer access to earlier implementations, scores, and execution traces [30]. Harness-R1 makes the editing policy itself learnable from the outcomes of its runtime changes [45]. Archive design and proposal learning therefore address distinct parts of search: retaining alternative behaviors and improving how candidates are generated.

Reusable tools and skills reduce the need to rediscover useful procedures. LLMs As Tool Makers separates a model that constructs reusable tools from one that applies them, allowing construction costs to be shared across later requests [6]. Voyager accumulates executable skills through interaction, while Agent Workflow Memory induces reusable routines from past experience [50, 54]. The reusable object can consequently range from a callable function to a workflow supplied as context, with different requirements for retrieval and execution.

Natural-language memory offers another way to preserve experience without immediately updating model weights. Reflexion stores linguistic feedback for subsequent trials, ExpeL extracts insights from past tasks, and ACE incrementally curates strategies in context playbooks [47, 83, 81]. MMSkills extends reusable packages to multimodal evidence; SkillRise and SkillGate learn skill curation and selection, respectively [80, 67, 31]. Representing a procedure, deciding when to retrieve it, and retaining its benefits in policy parameters remain separate questions.

Execution resources can evolve jointly with the policy or be allocated during its training. Co-Harness and WHALE alternate harness and model optimization, while HarnessForge evolves coupled harness–policy systems [13, 28, 10]. HarnessBandit instead schedules one harness per optimizer step using learnability and cross-harness transferability signals [55]. Harness-S allocates a fixed sibling pool at the task level, invoking auxiliary harnesses after all initial attempts under $H_0$ fail. Policy-aware profiles track which variants remain useful, while rescue trajectories update one shared policy. This links harness search to policy learning through complementary exploration, with base-harness evaluation separating retained policy gains from additional inference-time support.