KPHarness: Co-Evolving Knowledge and Probing for Long-Horizon Gaming Harnesses

Introduction

Vision-language-action (VLA) models ([4, 18, 6, 22, 30]) connect visual observations and language instructions with low-level control, enabling increasingly capable agents in interactive environments. Long-horizon gaming, however, exposes limitations beyond single-step action prediction. Even a capable VLA may repeatedly execute an ineffective action, fail to recover from a stalled trajectory, or overlook experience relevant to the current failure. As interaction horizons grow, performance therefore depends not only on the capabilities encoded in model parameters, but also on the external harness that governs how the model acts, probes, remembers, and recovers.

Existing work improves long-horizon VLA execution through prompting, memory, recovery, and hierarchical control ([20, 10, 32]), but these mechanisms are often manually designed and fixed at deployment. Recent automatic harness methods instead search for prompts, tools, memory, and control logic from execution feedback [19, 7, 26]. Such methods typically evaluate harness modifications through their execution outcomes. Visual gaming introduces an additional challenge: a failed trajectory often reveals what happened but not why. For example, an unsuccessful attack may result from target loss, insufficient range, an obstacle, or insufficient persistence, each requiring different recovery knowledge. Consequently, repeatedly trying harness modifications is inefficient when the underlying failure remains ambiguous. KPHarness turns automatic harness evolution from outcome-driven search into evidence-guided evolution through knowledge–probing co-evolution: evolving recovery knowledge provides context-specific intervention hypotheses that can focus subsequent probing, while probe outcomes determine their applicability and provide evidence for refining both knowledge and future probe selection. These challenges motivate our central question:

Can transferable recovery knowledge guide what a harness should probe under ambiguous failures, while probe outcomes identify applicable knowledge and refine subsequent probing and knowledge selection?

Existing harness search proposes modifications primarily from execution outcomes. In visually ambiguous environments, however, the same failure outcome may admit multiple incompatible explanations. We therefore treat harness evolution as a sequential process of hypothesis formation, active diagnosis, and empirical verification, where transferable knowledge supplies recovery hypotheses, and probing determines which hypothesis applies. We address this question with KPHarness, a framework for test-time harness evolution around a frozen VLA. Here, test-time means that the VLA parameters remain fixed after training, while only the external harness evolves through environment interaction. At the fast timescale, the VLA executes actions under the current harness. When execution stalls, the harness selects a probe according to the task and failure context, executes it in the environment, and uses the resulting evidence to retrieve relevant recovery knowledge. At the slow timescale, a vision-language researcher converts this evidence into task-specific harness modifications. Candidates are compared with their incumbents through paired sandbox evaluation, and accepted modifications are subsequently assessed on disjoint holdout seeds. Immediate interaction utility and delayed episode-success credit are then used to update probe and knowledge selection. Thus, unlike outcome-driven harness search alone, KPHarness learns both which intervention to apply and what to probe in order to discover it.

This process creates a knowledge–probing co-evolution loop. Better probes provide clearer evidence for selecting and refining interventions, while successful interventions reveal which probes and knowledge are useful under particular task and failure contexts. Validated recoveries are distilled into transferable execution knowledge and reused on subsequent tasks, while unresolved failures and probing ambiguities are converted into targeted follow-up cases for further sandbox evaluation. As illustrated in Figure 1, Auto-K searches for harness modifications from accumulated knowledge but does not improve its probing policy from probe outcomes. In contrast, KPHarness uses subsequent interaction to update both probe and knowledge selection. Experiments on 149 Minecraft tasks show that harness evolution improves coverage over bare execution and a fixed harness, while the complete framework further improves over knowledge-only evolution without updating the underlying VLA.

Our contributions are fourfold.

1. Evidence-guided test-time harness evolution. We identify a limitation of outcome-driven automatic harness search in visually ambiguous environments and formulate harness evolution as an evidence-guided process around a frozen VLA.

2. Knowledge–probing co-evolution for evidence-guided search. We couple transferable recovery knowledge with active probing: knowledge provides intervention hypotheses, probes gather evidence about their applicability, and validated execution outcomes update both future knowledge and probe selection.

3. Sandbox-grounded accumulation and transfer. Candidate harness changes are accepted only through paired environment interaction, while validated recoveries are promoted into transferable knowledge that guides subsequent tasks.

4. Probe-driven failure-frontier expansion. We convert failure reflections and probing ambiguities into targeted follow-up cases, exposing new failure modes that guide subsequent harness evolution.

Related Work

Static harnesses for gaming and embodied agents. Prior work has advanced Minecraft agents through video pretraining, language-conditioned behavior learning, and unified development frameworks [9, 3, 23, 5, 22]. However, gaming VLAs typically produce short action segments and may lose targets, repeat ineffective actions, or fail to verify progress. External harnesses mitigate these limitations through task state, memory, action constraints, verification, and recovery. Recent embodied harnesses extend frozen VLA and world-action models through cross-stage context [34], reusable primitives [35], and fast–slow control [12]. Yet their intervention and recovery mechanisms are largely fixed before deployment. Our method instead probes ambiguous failures, acquires transferable recovery knowledge, and improves subsequent probing and knowledge selection while keeping the gaming VLA frozen.

Automatic harness evolution. Early self-evolving agents primarily optimized prompts. APE and OPRO treat instruction generation as black-box optimization [36, 31]; EvoPrompt and Promptbreeder use evolutionary search [13, 11]; and ProTeGi and TextGrad guide revisions with textual feedback [28, 33]. DSPy and MIPRO extend optimization to instructions and demonstrations in multi-stage LM programs [17, 27]. Recent methods further optimize broader harness components, including tools, memory, middleware, and control logic [19, 7, 24, 26]. These methods mainly target coding, textual, or programmatically verifiable environments with explicit execution feedback. Long-horizon gaming instead involves visually and temporally ambiguous failures, and proposed modifications may be difficult for a frozen VLA to execute. KPHarness therefore probes the environment to identify failure contexts and applicable recovery knowledge. Interaction feedback improves subsequent probe and knowledge selection, while validated recoveries transfer across tasks. This knowledge–probing co-evolution improves both what intervention to apply and what evidence to seek.

Preliminaries

Long-Horizon Gaming with a Frozen VLA

We consider a visual gaming environment in which an agent completes a task specified by a language instruction $g$. At environment timestep $t$, the agent receives a visual observation $o_t$, executes a low-level action $a_t$, and observes the next observation $o_{t+1}$, an optional reward $r_t$, and task-progress or failure signals. The available interaction history is

and an episode induces a trajectory

whose task-level outcome is measured by an environment-defined objective $R(\tau, g)$.

Low-level action proposals are generated by a pretrained vision-language-action policy $\pi_\theta$. We study the frozen-actor setting, in which the parameters $\theta$ remain unchanged throughout execution, harness adaptation, and cross-task transfer. Any improvement must therefore arise from the external execution process. For long-horizon gaming, this process must preserve relevant context, recognize stalled progress, diagnose visually ambiguous failures, and recover through subsequent interaction.

Test-Time Harness Evolution

A test-time harness $\mathcal{H}_\phi$ is a model-external execution scaffold with configuration $\phi$ and external state $\mathcal{M}$. Given the task and interaction history, the harness constructs an execution context, queries the frozen actor, and determines the action executed in the environment:

Here, $C_\phi$ assembles the context supplied to the actor, while $I_\phi$ may pass through, replace, or block the actor’s proposal, including interventions for probing and recovery. The configuration $\phi$ specifies mechanisms such as intervention triggers, recovery instructions, verification rules, and memory-use policies. The external state $\mathcal{M}$ stores accumulated experience, reusable recovery knowledge, and statistics used for probe and knowledge selection.

A static harness may retrieve different memories or produce different actions, but it does so using mechanisms fixed in advance. Harness evolution additionally allows these mechanisms and their adaptive external state to change from interaction evidence. Let $P_{\theta,\phi,\mathcal{M}}(\tau\mid g)$ denote the trajectory distribution induced jointly by the environment, frozen actor, and harness. At adaptation stage $k$, the current system collects task-conditioned interaction evidence

An abstract update operator $\mathcal{U}$ then revises the harness while leaving the actor unchanged:

The index $k$ denotes harness-adaptation stages and is distinct from the environment timestep $t$. An adaptation stage may include multiple sandbox episodes, and the formulation does not prescribe a particular update schedule or learning rule.

Given a deployment-task distribution $p_{\mathrm{dep}}$, the objective is to improve the expected performance of the complete actor–harness system:

We refer to this setting as test-time harness evolution: after actor training, the harness adapts through repeated environment interaction, potentially across multiple episodes and tasks, while the underlying VLA parameters remain frozen.

Methodology

KPHarness (see Figure 2) operationalizes evidence-guided harness evolution through knowledge–probing co-evolution. Rather than proposing harness modifications from terminal outcomes alone, the framework first probes ambiguous failures, uses the resulting evidence to test applicable recovery knowledge, and then proposes evidence-grounded modifications. The frozen VLA acts in a fast execution loop, while a vision-language researcher performs diagnosis, proposal, and verification outside this latency-sensitive loop. Subsequent interaction outcomes update both action and knowledge probing, and validated recoveries become reusable knowledge for later tasks.

Adaptive Gaming Harness

We follow the notation in § 3. At evolution round $k$, the incumbent harness has configuration $\phi_k$ and external state $\mathcal{M}_k$. The external state contains episodic experience, a recovery-knowledge bank $\mathcal{K}_k$, an action-probing policy $\mathcal{B}_k^a$, and a knowledge-probing policy $\mathcal{B}_k^z$. Given the current task and failure context, the harness constructs or specializes candidate probes from the fixed low-level action space, and the action-probing policy selects among them to reveal additional environment evidence. The knowledge-probing policy selects a stored recovery strategy as a hypothesis, specializes it to the current task, and tests its applicability through subsequent interaction. Knowledge probing is therefore not passive retrieval: its execution outcome determines whether the selected knowledge should be preferred in similar failure contexts. The index $k$ identifies evolution rounds, although both probing policies may also improve during learning rollouts within the same round.

At each environment step, the harness retrieves relevant experience, constructs the actor’s context, and regulates the proposed action as defined in Equations (1)–(3). Its editable components include intervention triggers, recovery instructions, tool-selection rules, memory retrieval, completion checks, and probing behavior. Probe outcomes are recorded during interaction, episode outcomes provide delayed feedback, and the accumulated evidence supports slower harness revision. The VLA parameters and low-level action space remain fixed throughout this process.

Adaptive Gaming Probing

Probing is activated when progress stalls or execution repeatedly fails. It targets execution-relevant uncertainties such as target alignment, interaction reachability, attack persistence, tool selection, and completion checks. Given the current task and failure context, the harness first constructs or specializes applicable action probes from the fixed low-level action space. The action-probing policy then selects among these candidates and executes the selected probe to gather additional environment evidence. The resulting observation refines failure attributes such as target loss, orientation error, insufficient reachability, or repeated stalling. Conditioned on this refined failure context, the knowledge-probing policy selects a recovery strategy from $\mathcal{K}_k$, adapts it to the current task, and tests its applicability through subsequent interaction.

Both probing policies maintain separate statistics and use a context-conditioned, UCB-inspired selection rule [1, 21]. The broader design is motivated by information-seeking exploration, cross-context transfer, and hierarchical decision making [15, 25, 14]. For either policy, let $x$ denote the task and failure context and $v$ an available action probe or recovery strategy. For action probing, the available candidates are task-conditioned and may include newly instantiated or specialized probes. Let $N_{x,v}$ count previous selections of $v$ in context $x$, $Q_{x,v}$ denote their cumulative utility, and $\bar{q}_v$ denote the mean utility of $v$ across contexts. The prior weight $\lambda> 0$ controls cross-context transfer, while $\beta\ge0$ controls exploration. Each policy selects the candidate with the highest score:

The first term combines context-specific experience with the global value of the candidate, allowing evidence to transfer across related contexts. The second term encourages exploration of less frequently selected candidates. More detailed derivations are provided in Appendix § 13.

During learning rollouts, each selected probe receives immediate feedback from the environment. For a probe selected at timestep $t$, let $\Delta p_t$ denote progress change, $r_t$ environment reward, $n_t$ interaction novelty, and $\mathrm{fail}_t$ an intervention failure. Let $c_p$ be the probe cost and $w_p, w_r, w_n, w_f$ the corresponding nonnegative weights. The immediate utility is

where $\mathbb{I}[\cdot]$ is the indicator function. Action probes use the novelty term to reward newly revealed interaction evidence. Knowledge probes may set $w_n = 0$ and instead evaluate the progress produced after applying the selected recovery strategy.

Because an informative probe may affect the final outcome only after several subsequent actions, we additionally assign heuristic delayed credit after successful episodes. Let $\operatorname{Succ}(\tau,g)$ indicate successful completion of task $g$, $R_{\mathrm{succ}}$ be the success-bonus magnitude, $d_t$ the temporal distance from the probe to episode completion, and $\gamma\in(0,1]$ a discount factor. The delayed bonus is

Unsuccessful episodes receive no delayed success bonus. Each probe selection increments its count once, while its cumulative utility incorporates both immediate feedback and delayed success credit. These updates adapt $\mathcal{B}_k^a$ and $\mathcal{B}_k^z$ from interaction experience without updating the VLA.

The harness also records the observations, interventions, progress changes, and outcomes associated with each probe. Related records are grouped into recurring failure modes, such as turning without reacquiring the target, stalling after alignment, acting outside interaction range, using an incorrect tool, or missing a completion check. These records provide evidence for subsequent harness proposals.

Evidence-Grounded Harness Proposal

A vision-language researcher serves as the proposer. It receives the incumbent harness, selected failure modes and their supporting probe records, and relevant recovery knowledge from $\mathcal{K}_k$. Rather than inferring a modification from the failed trajectory alone, the researcher uses probe evidence to connect a likely failure cause with an appropriate intervention. It returns a failure diagnosis, an evidence-linked rationale, and a bounded edit to the structured HARNESS CONFIGURATION. Applying the edit to $\phi_k$ produces a candidate configuration $\tilde{\phi}_k$.

Candidate edits may revise camera alignment, attack persistence, reachability handling, loop-breaking behavior, tool selection, completion checks, or knowledge retrieval. The researcher can modify only components exposed by the HARNESS CONFIGURATION; it cannot update the VLA, replace the actor, modify the environment, or alter the task objective. A candidate therefore remains a hypothesis until sandbox interaction demonstrates that the frozen actor can execute it effectively.

Paired Validation and Knowledge Transfer

The incumbent and candidate are compared on the same task–seed pairs $\mathcal{V}_k$, where $g$ denotes a task and $s$ an environment seed. Both conditions start from the same external-state snapshot, including episodic memory and probing-policy statistics. Persistent memory and policy writeback are disabled during paired and holdout evaluation.

For a pair $(g,s)$, let $J_{\mathrm{cand}}(g,s)$ and $J_{\mathrm{inc}}(g,s)$ denote the candidate and incumbent scores under a fixed evaluation rule. The score prioritizes task success, with progress, reward, execution cost, and regression as secondary criteria. The average paired improvement is

Given an acceptance threshold $\epsilon_{\mathrm{pair}}$, the next harness configuration is

Rejected candidates leave the incumbent unchanged. Accepted candidates are subsequently evaluated on disjoint holdout seeds. These results do not alter the original acceptance decision, but provide evidence of cross-seed transfer.

An accepted recovery is stored as a candidate knowledge record containing its triggering context, recovery strategy, and supporting evidence, with task-specific details removed. It is promoted to transferable knowledge when it improves performance on a holdout seed or succeeds again on another task. Promoted records enter $\mathcal{K}_{k+1}$ and are tested by the knowledge-probing policy in subsequent contexts. Outside evaluation, learning rollouts update probing statistics and episodic experience. The selected harness, probing policies, and promoted knowledge then initialize subsequent tasks, forming the knowledge–probing co-evolution loop. Uncertain diagnoses and inconsistent recovery outcomes are additionally converted into validated, deduplicated follow-up cases, which remain separate from the fixed evaluation set and target the evolving failure frontier.

Experiments

We first train a gaming VLA based on Qwen3-VL-8B-Instruct [2] through cold-start supervised fine-tuning (SFT), with the training configuration detailed in Appendix A.1. Starting from the SFT checkpoint at step 16,200, we perform 400 steps of Group Relative Policy Optimization (GRPO) [29] on 32 interactive OpenHA training tasks in Minecraft. Figure 3 and Figure 4 show the corresponding training curves. We then freeze the resulting policy and evaluate harness adaptation on 149 OpenHA tasks spanning mining, combat, and crafting.

The slow research loop uses Gemini-2.5-Flash [8] at temperature 0 as the vision-language researcher and harness proposer. It receives the incumbent harness, structured failure and probe records, relevant recovery knowledge, and up to four frames sampled from the failed trajectory, and returns a structured candidate patch. Each task uses three proposal rounds with at most one candidate per round, and Auto-K and KPHarness use the same proposer configuration. Candidate patches are not accepted according to the researcher’s judgment alone; they are executed and verified in our locally deployed MineExplorer environment [16]. Accepted candidates are subsequently evaluated on a disjoint holdout seed. The researcher neither generates low-level actions nor participates in the latency-sensitive execution loop.

Evaluation Setup

We evaluate on 149 OpenHA tasks in MineExplorer, including 50 mining, 49 combat, and 50 crafting tasks, using the same frozen GRPO checkpoint for all methods. Each episode allows up to 100 VLA decisions, with four 50 ms low-level actions per decision, yielding up to 400 temporally dependent actions per rollout.

Our primary setting uses the legacy protocol with sampling temperature 0.7, where each prediction conditions only on the current observation and task instruction. We additionally evaluate Bare and Fixed under mtv1 at temperature 0, which restores the latest ten observation–action turns. Both protocols predict a 200 ms action chunk per decision. Action definitions and prompt construction are provided in Appendix A.

We use one probing configuration across all tasks and seeds: $w_p = 1.0$, $w_r = 0.25$, $w_n = 0.15$, $c_p = 0.05$, $w_f = 0.5$, $\beta= 1.2$, $\lambda= 0.25$, $R_{\mathrm{succ}} = 1.0$, and $\gamma= 0.8$. Knowledge probing uses the same coefficients, with cost 0.02 and no novelty term. Under the primary setting, Bare solves 39 of 149 tasks; the remaining 110 form the Bare-failure adaptation subset. Candidate and incumbent harnesses are compared using the same task, search seed, decoding settings, memory snapshot, and probing-policy state. A candidate is accepted when its paired score improves by at least 0.01; the exact scoring rule is given in Appendix D. After adaptation, we freeze the harness and probing statistics and replay all 149 tasks without further search or writeback. Frozen probes may still execute, but their policies are not updated. Because this replay includes tasks seen during adaptation, we report it as task coverage, while disjoint-seed holdout evaluation measures cross-seed robustness.

Compared Methods and Main Results

We compare four settings using the same frozen VLA checkpoint. Bare directly executes the VLA, while Fixed adds manually specified retry and recovery rules. Auto Knowledge (Auto-K) further searches for and validates recovery knowledge through sandbox interaction, but uses predefined or coverage-based probing. The full KPHarness additionally adapts probe construction and action- and knowledge-probe selection from interaction feedback. None of the methods updates the VLA parameters. Recent automatic harness methods such as Meta-Harness [19] primarily study textual, verifiable environments, whereas visual gaming introduces ambiguous failures that often require interaction to diagnose. Our variants therefore progress from fixed recovery, to automatic knowledge accumulation, and finally to knowledge–probing co-evolution.

For failed trajectories, Auto-K and KPHarness use the same proposer to generate bounded harness patches, which are accepted only when they improve over the incumbent under paired evaluation and are subsequently tested on a disjoint holdout seed. An HO-win denotes improvement on this holdout evaluation. The key difference is adaptive probing: KPHarness updates contextual action-and knowledge-probing policies using immediate interaction feedback and delayed episode-success credit, whereas Auto-K does not learn from probe outcomes.

Rescued counts tasks failed by Bare but solved by the corresponding method under the same protocol and temperature. “N/A” indicates that the method does not contain the adaptive probing and delayed-credit mechanism required to define probe-only or probing-replay success.

Method	Task success Mine	Task success Kill	Task success Craft	Rescued	Success rate	Knowledge search ACCEPT	Knowledge search HO-win	Adaptive probing P-only	Adaptive probing Replay succ.
Auto-K	$4.7_{\pm0.6}/30$	$7.7_{\pm5.1}/31$	$3.3_{\pm3.2}/49$	$15.7_{\pm8.0}/110$	$14.2_{\pm7.3}\%$	–	–	N/A	N/A
KPHarness	$8.3_{\pm1.2}/30$	$13.7_{\pm1.5}/31$	$6.7_{\pm0.6}/49$	$28.7_{\pm3.2}/110$	$26.1_{\pm2.9}\%$	$14.0_{\pm3.0}$	$6.7_{\pm2.1}$	$2.3_{\pm1.2}$	$14.7_{\pm3.2}$

Main Results and Cross-Seed Robustness. As shown in Table 1, under the primary legacy protocol with temperature 0.7, Bare, Fixed, and Auto-K solve 39 (26.2%), 50 (33.6%), and 63 (42.3%) of the 149 tasks, respectively, while the full KPHarness reaches 70 (47.0%). The progressive gains from Bare to Fixed, Auto-K, and KPHarness support the benefit of external recovery, transferable knowledge, and finally adaptive knowledge–probing co-evolution. In particular, KPHarness achieves 17 probing-enabled replay successes, including three strict probe-only recoveries where Knowledge Search fails. The advantage is also consistent across seeds: on the same 110 Bare-failed tasks, KPHarness improves the mean rescue count from $15.7_{\pm8.0}$ for Auto-K to $28.7_{\pm3.2}$, with positive gains under all three evaluated seeds. Detailed per-seed results are provided in Appendix § 8.

Method	Protocol	Temp.	Overall	Task success Mine	Task success Kill	Task success Craft	Rescued	Knowledge search ACCEPT	Knowledge search HO-win	Adaptive probing P-only	Adaptive probing Replay succ.
Bare	mtv1	0	26/149 (17.4%)	9/50	17/49	0/50	0	–	–	–	–
Fixed	mtv1	0	34/149 (22.8%)	16/50	17/49	1/50	8	–	–	–	–
Bare	legacy	0	23/149 (15.4%)	8/50	15/49	0/50	0	–	–	–	–
Fixed	legacy	0	31/149 (20.8%)	13/50	18/49	0/50	8	–	–	–	–
Auto-K	legacy	0	41/149 (27.5%)	20/50	20/49	1/50	18	11	3	N/A	N/A
KPHarness	legacy	0	44/149 (29.5%)	21/50	21/49	2/50	21	14	3	1	1
Bare	legacy	0.7	39/149 (26.2%)	20/50	18/49	1/50	0	–	–	–	–
Fixed	legacy	0.7	50/149 (33.6%)	23/50	25/49	2/50	11	–	–	–	–
Auto-K	legacy	0.7	63/149 (42.3%)	25/50	30/49	8/50	24	14	5	N/A	N/A
KPHarness	legacy	0.7	70/149 (47.0%)	29/50	33/49	8/50	31	14	5	3	17

Generalization to Unseen Tasks. We freeze the evolved harness, knowledge bank, and probing policy and evaluate on 60 unseen tasks without further search or writeback. As shown in Table 3, KPHarness solves 15/60 tasks, compared with 8/60 for Bare and 7/60 for Auto-K, demonstrating transfer beyond the adaptation set. Additional qualitative examples and learned recovery knowledge are shown in Appendix Figures 5 and 6.

Method	Overall	Mine	Kill	Craft
Bare	8/60	7/20	1/20	0/20
Auto-K	7/60	6/20	1/20	0/20
KPHarness	15/60	9/20	5/20	1/20

Ablation Study. Table 1 further separates the contributions of the external harness components. Fixed consistently improves over Bare, while Auto-K further improves over Fixed through transferable recovery knowledge. The complete KPHarness improves from 63 to 70 tasks over Auto-K, supporting the additional benefit of adaptive probing and knowledge selection. Complementary diagnostics in Appendix § 9 show that simply injecting a fixed recovery bank does not reproduce the gain of the complete sandbox-mediated evolution process. Additional analyses of knowledge search, probe-only recovery, and sequential knowledge reuse are provided in Appendix § 7.§ 4 and Appendix § 7.§ 5.

Conclusion

We introduced KPHarness, a test-time harness-evolution framework that combines adaptive probing, sandbox verification, and transferable recovery knowledge around a frozen VLA. Experiments on Minecraft show improved task coverage and transfer to unseen tasks without further actor training.

Several limitations remain. Due to the substantial computational cost of VLA training and harness evolution, including SFT/GRPO training, repeated environment rollouts, VLA inference, and multimodal researcher calls, our evaluation is limited to a single policy backbone and gaming environment. Harness evolution is also more expensive than fixed execution, while delayed success credit remains heuristic and may not precisely identify the contribution of individual probes. More detailed information can be found in Appendix § 14. Future work should improve adaptation efficiency and evaluate knowledge–probing co-evolution across broader models and task distributions.

Experimental Setup

VLA Policy Training

We adopt Qwen3-VL-8B-Instruct as the policy backbone and train it in two stages. First, we perform cold-start supervised fine-tuning (SFT) for three epochs on approximately 765K multimodal Minecraft action trajectories. We use 16 NVIDIA H800 GPUs with 141 GB of memory each, BF16 precision, a maximum sequence length of 14K, a per-GPU batch size of 1, and eight gradient-accumulation steps, yielding a global batch size of 128. Optimization uses fused AdamW with $\beta_{1}=0.9$, $\beta_{2}=0.95$, weight decay 0.1, and gradient clipping at 1.0. The learning rate is warmed up over the first 3% of training to $5 \times10^{-6}$ and then decayed to $5 \times10^{-7}$ with a cosine schedule.

Second, we initialize from the SFT checkpoint at step 16,200 and perform 400 steps of Group Relative Policy Optimization (GRPO) [29] on 32 interactive OpenHA training tasks in Minecraft, spanning navigation, world interaction, and GUI manipulation. Each task is sampled with a group of eight online trajectories. GRPO uses Adam with $\beta_{1}=0.9$, $\beta_{2}=0.95$, a constant learning rate of $1 \times10^{-7}$, weight decay 0.01, gradient clipping at 1.0, and a clipping ratio of 0.2. Training uses BF16, a policy micro-batch size of 1, and one optimization epoch per rollout batch. We apply no additional KL reward penalty. During rollout collection, actions are sampled with temperature 1.0 and top-$p = 1.0$, and the policy receives a binary task-completion reward.

We use the GRPO checkpoint at step 400 as the frozen VLA in all harness experiments. Under the legacy protocol, each prediction conditions only on the current observation and task instruction. Under mtv1 (minecraft_v1), the model additionally receives up to ten preceding observation–action turns. Both protocols predict four consecutive low-level actions per decision, each executed for 50 ms, producing a 200 ms action chunk. The primary evaluation uses legacy with temperature 0.7; auxiliary protocol comparisons use temperature 0. The VLA parameters remain frozen throughout harness evolution.

Evaluation Setup

We evaluate on 149 OpenHA tasks in Minecraft: 50 mining tasks, 49 combat tasks, and 50 crafting tasks. Each episode permits up to 100 VLA decisions and therefore up to 400 temporally dependent low-level actions. We use long-horizon to describe this multi-turn closed-loop dependency, which may involve locating, approaching, aligning with, and interacting with a target, followed by progress verification and failure recovery.

Evaluation uses eight NVIDIA H800 GPUs and eight parallel environment workers. Each worker hosts one frozen policy instance and interacts with an independent MineStudio environment. Task success is determined by the binary completion signal provided by OpenHA. All methods within a comparison use the same VLA checkpoint, task configurations, prompting protocol, temperature, and interaction budget.

Under the primary legacy, temperature-0.7 setting, Bare solves 39 of the 149 tasks. Its remaining 110 failures form the Bare-failure adaptation subset. This subset is defined operationally by the frozen Bare policy and is not an independently curated difficulty split.

Candidate and incumbent harnesses are compared using the same task and search seed. Accepted candidates are subsequently evaluated using a disjoint holdout seed. These evaluation seeds control harness-side stochasticity, while the Minecraft world configuration is specified by the OpenHA task. Paired conditions share the same VLA parameters, decoding settings, episodic-memory snapshot, and probing-policy state. Persistent-memory updates and probing-policy writeback are disabled during paired and holdout evaluation.

After adaptation, we freeze the selected harness and probing statistics and replay all 149 tasks without further search or writeback. Because this replay includes tasks from the adaptation subset, it measures task coverage rather than generalization to an unseen task split. The disjoint-seed evaluation separately measures cross-seed robustness.

Researcher and Sandbox Verifier.

We use the API model identifier gemini-2.5-flash [8] with temperature 0, visual input enabled, and at most four trajectory frames per proposal. Its output is restricted to a structured patch over the exposed harness configuration. Auto-K and KPHarness use the same proposer model, prompt format, temperature, and frame budget.

All candidate patches are empirically evaluated in the MineStudio sandbox [16]. The candidate and incumbent share the same task, frozen VLA, decoding configuration, evaluation seed, and initial external state. Persistent memory and probing-policy writeback are disabled during comparison. The sandbox outcome, rather than the proposer itself, determines whether a patch is accepted.

Knowledge Search and Adaptive Probing Diagnostics

We further analyze the 110 tasks in the Bare-failure adaptation subset. This analysis separates successes observed during Knowledge Search from those obtained in probing-enabled replay. Because it aggregates multiple intermediate harnesses, it is a search-time diagnostic rather than a decomposition of the final frozen-harness coverage in Table 1.

Statistic	Overall	Mine	Kill	Craft
Knowledge Search Best	28/110	8	14	6
Probing-enabled Replay	17/110	4	9	4
Search Best ∪ Probing Replay	31/110	9	15	7
ACCEPT	14	4	5	5
HO-win	5	1	2	2
Strict Probe-only	3	1	1	1

Knowledge Search Best counts a task as solved if its best validated candidate succeeds during search; it is therefore an aggregate search-time statistic, not the performance of one deployable harness. It solves 28 tasks, while probing-enabled replay succeeds on 17. Fourteen tasks overlap, giving

Thus, the 17 replay successes are not 17 additional gains. Instead, three are strict probe-only successes: one mining, one combat, and one crafting task that remain unsolved by Knowledge Search but succeed during probing-enabled replay.

The 31-task diagnostic union differs from the 31 rescued tasks in Table 1. The former combines successes from different intermediate harnesses during search and probing replay, whereas the latter is measured from the final evaluation used to report overall task coverage. The two quantities therefore describe different stages of the evaluation and should not be interpreted as identical statistics.

The combined diagnostic procedures solve 15 of the 31 Bare-failed combat tasks (48.4%), compared with 9 of 30 mining tasks (30.0%) and 7 of 49 crafting tasks (14.3%). Combat failures frequently involve target loss or stalled engagement, where approaching, attacking, and re-centering provide useful corrective evidence. Crafting remains more difficult because it often requires precise multi-step GUI interaction.

Knowledge Search produces accepted patches for 14 distinct tasks, five of which obtain an HO-win on a disjoint seed, corresponding to an empirical holdout rate of 35.7%. Because both search and holdout evaluation use a limited number of seeds, these results provide preliminary cross-seed evidence rather than a statistically conclusive generalization estimate.

Sequential Cross-Task Knowledge Reuse

During harness adaptation, tasks are processed sequentially. After each task, validated recovery knowledge and contextual-bandit statistics are written to the shared harness state and become available to subsequent tasks. We analyze all 110 tasks in the Bare-failure adaptation subset from the temperature-0.7 run to verify that this information is used during later execution.

We define a prior-knowledge selection as an event in which the current task selects a promoted recovery strategy whose provenance includes at least one preceding task. A seed-harness success is a task completed by the inherited harness during its initial evaluation, before any task-specific candidate is proposed or accepted. Each initial evaluation contains exactly one episode, so these successes are not selected from repeated task restarts.

As shown in Table 5, 107 of the 110 adaptation tasks select knowledge promoted from earlier tasks, yielding 840 selection events. The inherited harness completes 12 tasks before task-specific search, and nine of these successful trajectories explicitly use prior-task knowledge.

Statistic	Count
Adaptation tasks	110
Tasks using prior-task knowledge	107
Prior-knowledge selection events	840
Distinct prior-skill–target relations	156
Cross-family reuse relations	101
Single-rollout seed-harness successes	12
Successes without local ACCEPT	12
Successes explicitly using prior knowledge	9

These results provide operational evidence of sequential cross-task reuse, including transfer across task families. However, the inherited harness also contains updated prompts and contextual-bandit state, while decoding at temperature 0.7 remains stochastic. We therefore interpret this analysis as evidence that knowledge transfer occurs, rather than as a causal estimate of the improvement attributable solely to transferred knowledge.

Multi-Seed Robustness on Bare-Failure Tasks

We evaluate cross-seed robustness on the 110 tasks that Bare fails under the legacy protocol with temperature 0.7. This diagnostic subset contains 30 mining, 31 combat, and 49 crafting tasks. All conditions use the same frozen VLA checkpoint. For the cross-seed transfer evaluation, Auto-K executes its frozen harness and transferable knowledge bank without test-time writeback, whereas KPHarness retains the knowledge–probing feedback loop. Table 6 reports the complete results for seeds {101, 202, 303}.

Method	Seed	Success	Task success			Knowledge search		Adaptive probing
			Mine	Kill	Craft	ACCEPT	HO-win	P-only	Replay succ.
Auto-K	101	24/110 (21.8%)	5/30	12/31	7/49	14	5	N/A	N/A
KPHarness	101	31/110 (28.2%)	9/30	15/31	7/49	14	5	3	17
Auto-K	202	8/110 (7.3%)	5/30	2/31	1/49	–	–	N/A	N/A
KPHarness	202	30/110 (27.3%)	9/30	14/31	7/49	17	9	3	11
Auto-K	303	15/110 (13.6%)	4/30	9/31	2/49	–	–	N/A	N/A
KPHarness	303	25/110 (22.7%)	7/30	12/31	6/49	11	6	1	16
Auto-K	Mean	$15.7_{\pm8.0}$/110 (14.2%$_{\pm7.3\%}$)	$4.7_{\pm0.6}$/30	$7.7_{\pm5.1}$/31	$3.3_{\pm3.2}$/49	–	–	N/A	N/A
KPHarness	Mean	$28.7_{\pm3.2}$/110 (26.1%$_{\pm2.9\%}$)	$8.3_{\pm1.2}$/30	$13.7_{\pm1.5}$/31	$6.7_{\pm0.6}$/49	$14.0_{\pm3.0}$	$6.7_{\pm2.1}$	$2.3_{\pm1.2}$	$14.7_{\pm3.2}$

Consistent improvements across seeds. Auto-K solves 24, 8, and 15 tasks under seeds 101, 202, and 303, respectively, while KPHarness solves 31, 30, and 25. Thus, KPHarness improves coverage by 7, 22, and 10 tasks, with a positive gain under every evaluated seed. Averaged across seeds, the number of rescued tasks increases from $15.7_{\pm8.0}$ to $28.7_{\pm3.2}$ out of 110. Equivalently, the mean success rate improves from $14.2_{\pm7.3}\%$ to $26.1_{\pm2.9}\%$. The lower standard deviation additionally indicates that KPHarness is less sensitive to the evaluation seed than Auto-K.

Improvements across task families. Aggregated over the three seeds, Auto-K solves 47 of 330 task–seed instances, whereas KPHarness solves 86. The improvement is distributed across all three task families. Mining success increases from 14 to 25 instances, combat success from 23 to 41, and crafting success from 10 to 20. The gain therefore cannot be attributed solely to one task family or to an unusually favorable environment configuration.

Task-paired evidence. A task-paired analysis provides stronger evidence than comparing only aggregate success rates. Under seed 202, KPHarness exclusively solves 24 tasks that Auto-K fails, whereas Auto-K exclusively solves only two tasks. Under seed 303, the corresponding counts are 12 and two. Pooling these two rerun seeds gives 36 KPHarness-only successes versus four

Auto-K-only successes. An exact McNemar test rejects symmetric random outcome fluctuations ($p = 1.86 \times10^{-7}$). Thus, the observed difference is unlikely to arise from a small number of stochastic success flips.

Contribution of adaptive probing. Across the three seeds, KPHarness produces 3, 3, and 1 strict probe-only recoveries and obtains 17, 11, and 16 successful probing-enabled replays. On average, this corresponds to $2.3_{\pm1.2}$ strict probe-only recoveries and $14.7_{\pm3.2}$ replay successes per seed. Meanwhile, Knowledge Search produces $14.0_{\pm3.0}$ accepted task-level patches, of which $6.7_{\pm2.1}$ generalize to the holdout evaluation. These observations suggest that the gain is not explained merely by generating more patches. Instead, probe feedback changes which failure contexts are explored and which recovery knowledge is selected and retained.

Interpretation. The consistent improvement across all seeds, the reduction in cross-seed variance, and the strongly asymmetric task-level success flips jointly support the system-level benefit of knowledge–probing co-evolution. This experiment evaluates the complete feedback loop rather than attributing the gain to either direction individually.

The Mean rows report the mean and sample standard deviation over seeds {101, 202, 303}, with the standard deviation shown as a lower-right subscript. P-only denotes strict probe-only recoveries, for which Knowledge Search fails but the probing-enabled replay succeeds. Replay succ. counts all successful probing-enabled replays. “N/A” indicates that Auto-K does not contain the adaptive probing and delayed-credit mechanism. A dash indicates that Knowledge Search was disabled during the corresponding frozen Auto-K evaluation and therefore ACCEPT and HO-win are not defined.

Static-Knowledge Diagnostic

We conduct an additional diagnostic to examine whether injecting a fixed recovery bank improves adaptive probing. The diagnostic uses the 110 tasks failed by Bare under the legacy protocol at temperature 0.7, with the same frozen VLA, evaluation seed 101, task list, base harness, and probe budget of eight. Each variant starts with an empty bandit state and performs one execution rollout per task. Knowledge content and harness patches remain fixed: no proposer, Knowledge Search, ACCEPT/REVERT, holdout validation, or skill creation is enabled. Environment feedback may update selection statistics, but cannot create or revise knowledge.

Variants. All four variants use adaptive probing and update probe-selection statistics from outcomes. Probe-only uses no knowledge bank. Static K (no injection) updates knowledge-selection statistics but does not inject selected knowledge into the actor. Static K (no K-update) injects selected knowledge but disables outcome-based updates to knowledge-selection statistics. Static K (adaptive selection) enables both knowledge injection and outcome-based updates to its knowledge-selection statistics.

Results. None of the tested static-bank variants outperforms Probe-only in overall coverage (Table 7). Adaptive selection over the fixed bank solves 13 tasks, compared with 15 for Probe-only. Both variants solve seven tasks; Probe-only uniquely solves eight, and adaptive static-K selection uniquely solves six. The paired difference is not statistically significant (two-sided exact McNemar test, $p = 0.79$). Their observed probe counts and useful-probe fractions are also similar (Table 8). Thus, adding this fixed bank provides no observed coverage benefit under the evaluated setting.

Method / Variant	Success	Mine	Kill	Craft
(a) Full Sandbox-mediated knowledge evolution
Auto-K	24/110 (21.8%)	5/30	12/31	7/49
KPHarness	31/110 (28.2%)	9/30	15/31	7/49
(b) Static-knowledge execution diagnostic
Probe-only	15/110 (13.6%)	5/30	6/31	4/49
Static K (no injection)	9/110 (8.2%)	2/30	6/31	1/49
Static K (no K-update)	13/110 (11.8%)	4/30	5/31	4/49
Static K (adaptive selection)	13/110 (11.8%)	6/30	5/31	2/49

Variant	Probes per task	Steps to recovery	Useful probes (%)	K selections	Positive-utility K (%)
Probe-only	8.00	59.4	42.7	–	–
Static K (no injection)	7.98	50.8	41.3	1,463	0.07
Static K (no K-update)	7.98	61.8	39.6	1,454	0.00
Static K (adaptive selection)	7.93	51.5	41.7	1,428	0.14

Adaptive static-K selection makes 1,428 knowledge selections, but only 0.14% have positive immediate utility. This metric captures immediate feedback rather than the full downstream contribution of Panel (a) is the hard-subset view of Table 1: including the 39 Bare successes gives 63/149 for Auto-K and 70/149 for KPHarness. Panel (b) uses one execution rollout per task with fixed knowledge content and harness patches. Results are comparable within each panel; cross-panel differences are not controlled effect estimates. Bold denotes the highest coverage within each panel.

knowledge. The outcomes also vary across task families: mining coverage increases from five tasks to six, while crafting coverage decreases from four to two. The bank primarily contains generic visual recovery strategies, including target reacquisition, re-centering, and escaping walls. These strategies may offer limited help for crafting-interface or prerequisite failures, suggesting that recovery depends on the relevance of knowledge to the encountered failure context.

Why Sandbox-mediated knowledge–probing evolution matters. Sandbox provides the execution evidence that connects recovery knowledge with adaptive probing. Knowledge directs probes toward unresolved failure hypotheses, while probe outcomes reveal whether recovery strategies are executable by the frozen actor. This evidence informs knowledge selection and the proposal and validation of harness updates. Validated recovery experience can then guide subsequent probing and be reused across tasks, enabling knowledge and probing to adapt together.

The diagnostic highlights the distinction between updating selection statistics and revising the knowledge being selected. All diagnostic variants interact with the environment, but their feedback cannot improve the bank’s content. In the complete framework, Sandbox feedback additionally drives knowledge acquisition, harness revision, and validation. The main results demonstrate the benefit of this overall design: coverage increases from 50/149 for Fixed to 63/149 for Auto-K and 70/149 for KPHarness. The advantage over Auto-K also holds across all three evaluated seeds, increasing the mean rescue count from $15.7 \pm8.0$ to $28.7 \pm3.2$ on the same 110-task subset. Together, these findings support Sandbox-mediated knowledge–probing co-evolution as an effective approach to harness adaptation, with Sandbox serving as the shared execution and validation interface for accumulating reusable recovery experience.

Scope. The static-bank diagnostic evaluates one bank and one seed under a restricted execution protocol. It retains environment interaction and differs from the complete search-and-replay pipeline. Its results therefore provide complementary system-level evidence, without isolating the causal contribution of Sandbox interaction or individual adaptation components.

Harness Search Implementation Details

Researcher and search budget. We use Gemini-2.5-Flash [8] at temperature 0 as both the visual failure-attribution researcher and harness proposer. It receives the incumbent harness, rollout statistics, failure and probe records, relevant recovery knowledge, previous proposal outcomes, and up to four frames sampled from the failed trajectory. Each task uses three sequential proposal rounds with at most one candidate per round. Invalid outputs are repaired for up to three researcher calls and discarded if they remain invalid. An accepted candidate becomes the incumbent for the next round, whereas a rejected candidate leaves the incumbent unchanged. Auto-K and KPHarness use the same researcher and proposal budget.

Editable harness configuration. The researcher may modify the task, decomposition, and recovery prompts; the runtime task instruction; and selected probe, memory, verification, recovery, and runtime settings. Every candidate must contain at least one prompt or instruction edit, and parameter-only patches are rejected before execution. The researcher cannot modify the VLA checkpoint or parameters, the low-level action space, task definition, MineExplorer environment, or evaluation protocol.

Failure contexts and runtime signals. The action-probing context combines a coarse task family with the event that activates probing, producing contexts such as mine:stall, combat:fail, and craft:stall. The knowledge-probing context additionally includes failure attributes such as zero_progress, orientation, target_loss, and stall_heavy. These attributes are obtained from rollout statistics and visual attribution over the sampled trajectory frames.

Progress and reward are obtained from environment feedback, while execution failure is determined from the runtime status. Novelty is computed as one minus the maximum similarity between the current state–action pair and previously stored experience:

where $z_t$ denotes the visual-state representation and $e(a_t)$ the action embedding. We set $n_t = 1$ when no previous experience is available.

Paired acceptance. For an episode executed by harness $H$, the acceptance score is

where $R_{\mathrm{env}}$ is the raw cumulative environment reward without additional normalization. Progress, interaction cost, and step efficiency are recorded as diagnostic statistics and may be supplied to the researcher for failure analysis and proposal generation, but they do not enter the ACCEPT/REVERT decision. Since the environment reward is not assumed to have a globally fixed range, we do not interpret Equation equation 14 as a strictly lexicographic preference for success.

The candidate and incumbent are evaluated on the same task using one search seed, identical decoding parameters, and the same initial memory and probing-policy state. A candidate $H'$ is accepted when

Rejected candidates leave the incumbent unchanged. Accepted candidates are additionally compared with the seed harness using another disjoint holdout seed. Holdout results do not alter the original acceptance decision. Persistent memory and probing-policy writeback are disabled during both paired and holdout evaluation.

Example harness patch. The following shortened example shows an actual candidate generated for the sand-mining task:

{
  "diagnosis":
    "The agent stalls while sand is visible.",
  "why":
    "The instruction does not specify target

centering and persistent interaction.",
"proposal":
    "Locate sand, center it, and break it.",
"edits": [
    {
        "path": ["runtime", "instruction"],
        "value":
            "Locate sand, center it, then break it."
    },
    {
        "path": ["recovery_prompt"],
        "value":
            "If sand is not centered, re-orient and
             retry. <actions> mouseMove(0,80);
             mouseMove(0,40); mouseClick(left);
             no_op </actions>"
    }
]
}

The patch modifies only exposed harness fields and leaves the VLA frozen. It is retained only if its paired execution improves over the incumbent according to Equation (15).

Harness Evolution and Evaluation

Initial Gaming Harness. We apply KPHarness to the 110 tasks in the Bare-failure adaptation subset. For every task, the actor is fixed to the GRPO checkpoint at step 400 and uses the primary legacy protocol with sampling temperature 0.7 and four low-level actions per prediction. The VLA parameters are never updated during harness evolution.

At evolution round $k$, the external harness $H_k$ maintains action probes, task knowledge, transferable recovery memory, probing statistics, and action-level middleware. The middleware operates in the latency-sensitive execution loop and can adjust camera motion, sustain or interrupt an action, select an inventory item, invoke recovery behavior, or verify task completion without modifying the underlying VLA.

Adaptive Gaming Probing. When execution stalls, repeats an ineffective action, loses its target, selects an inappropriate tool, or fails to recognize completion, the harness activates adaptive probing. Given the current task and failure context, it constructs or specializes applicable action probes from the fixed low-level action space. These probes test execution-relevant hypotheses, including whether the target is aligned, within interaction range, or requires persistent interaction. Their visual transitions, environment feedback, and progress signals refine the inferred failure context.

The knowledge-probing policy then selects a stored recovery strategy as a hypothesis, specializes it to the current task, and tests it through subsequent interaction. Probe choices receive immediate credit from progress, reward, novelty, and failure resolution, together with delayed credit from episode success. Effective probes and knowledge are preferred in related contexts through the updated probing policies. Consequently, probing adapts both the interventions considered for a failure and their context-conditioned selection without modifying the underlying VLA.

Evidence-Grounded Harness Proposal. The slow research loop converts the inferred failure mode, probe evidence, and relevant recovery knowledge into a candidate harness $\tilde{H}_k$. A proposal may introduce or revise camera alignment, attack persistence, reachability handling, loop breaking, tool selection, completion verification, or task-knowledge retrieval. Because the proposal is conditioned on interaction evidence, it targets an observed failure mechanism rather than relying only on the textual description of an unsuccessful episode.

Each candidate edit is represented as a trigger–intervention unit. The trigger specifies the task, visual, and execution conditions under which the rule should activate, while the intervention specifies the corresponding action correction or recovery procedure. This representation allows validated knowledge to be retrieved and specialized for related tasks without modifying the VLA.

Paired Proposal Validation. We compare every candidate $\hat{H}_k$ with its incumbent $H_k$ through paired execution in the MineStudio sandbox. Both conditions use the same task configuration, frozen VLA checkpoint, decoding settings, interaction budget, and evaluation seed. Candidate selection prioritizes task completion, with progress, reward, step efficiency, and absence of regression as secondary criteria.

The two conditions start from isolated copies of the same episodic memory and probing-policy state. Memory updates and probing-policy writeback are disabled during evaluation, preventing execution order from affecting the comparison. If the candidate does not improve over the incumbent, it is rejected and $H_{k+1}=H_k$. Otherwise, it is accepted, $H_{k+1}=\hat{H}_k$, and its trigger, intervention, evidence, and outcome form a candidate knowledge record.

Cross-Seed Verification and Knowledge Transfer. Accepted candidates are subsequently evaluated using a disjoint holdout seed. The holdout result does not alter the original paired acceptance decision or trigger further modification of that candidate; it provides separate evidence of cross-seed transfer. We report an HO-win when the accepted candidate outperforms its seed harness by the required margin on the holdout seed.

An accepted recovery is promoted to transferable knowledge when it obtains holdout support or is successfully reused on another task. Promoted knowledge enters the shared knowledge bank and becomes a candidate for future knowledge probing. Separate learning rollouts, rather than paired or holdout evaluation, update and persist the contextual probing statistics.

The resulting evolution cycle is

A rejected proposal leaves the harness and persistent state unchanged. An accepted proposal updates the harness, while separately validated recoveries and probing statistics support subsequent tasks.

Final Harness Evaluation. After evolution, we freeze the selected harness, knowledge bank, and probing statistics and replay the complete 149-task OpenHA evaluation set. The 39 tasks originally solved by Bare and the 110 tasks in the Bare-failure adaptation subset are all included. No further harness search, memory writeback, or probing-policy update is allowed, although the frozen probing policy may still execute probes.

This final replay measures overall task coverage and checks whether the evolved harness preserves tasks already solved by the frozen actor. Because the replay includes tasks encountered during adaptation, it is reported as task coverage rather than generalization to an unseen task split; disjoint holdout seeds provide a separate measure of cross-seed robustness.

Qualitative Examples

The advantage of KPHarness does not arise simply from executing additional probe actions. It comes from using probe outcomes to connect failure diagnosis, knowledge probing, harness adaptation, and cross-task knowledge accumulation. We illustrate this distinction by comparing Bare, Auto-K, and KPHarness on a sand-mining task under the same frozen VLA.

Bare Execution. Bare executes the original task instruction without an external recovery harness. When the policy loses the target, becomes trapped, or acts outside the effective interaction range, it must recover using only the information provided by the current prompting protocol. The failure Action composition. Commands can be composed within one action step, for example mouseMove(10,10) mouseClick(left) keyPress(w). Each VLA prediction contains exactly four semicolon-separated action steps: <actions> a_1; a_2; a_3; a_4 </actions>. Each step lasts 50 ms, yielding a 200 ms execution horizon per prediction.

is neither explicitly diagnosed nor converted into reusable execution knowledge. Bare therefore represents frozen execution without harness adaptation.

Auto-K. Auto-K searches for and validates recovery knowledge, making it stronger than Bare. However, its probe actions are selected through predefined triggers or coverage rules rather than a policy learned from probe outcomes. Consequently, Auto-K may look up, look down, attack, or move forward, but the resulting feedback does not update which action or knowledge probe should be preferred under similar future failures.

This distinction is visible in the sand example. Auto-K executes eight probe actions, but records no knowledge-selection or bandit update:

It continues to use the generic instruction “Gather sand from the desert” and a generic retry prompt. Its only relevant prior is reacquire_and_center, inherited from an earlier poppy task. Without failure-conditioned knowledge probing and task-specific specialization, Auto-K does not complete the task within the 100-turn interaction budget.

Closed-Loop Adaptation in KPHarness. KPHarness treats an action probe as an information-gathering intervention and a knowledge probe as an executable recovery hypothesis. Action-probe outcomes provide evidence about target loss, orientation error, insufficient reachability, or stalled progress. Conditioned on this evidence, the knowledge-probing policy selects a recovery strategy, which is specific to the current task and tested through subsequent interaction. Immediate utility and delayed episode success then update the corresponding probing statistics.

Action	Execution semantics
mouseMove(dx,dy)	Move the camera horizontally and vertically. The values $dx$ and $dy$ determine the direction and magnitude of camera rotation.
mouseClick(left)	Attack the targeted entity or block. In an inventory GUI, pick up, place, or collect an item stack.
mouseClick(right)	Use the selected item, place a block, interact with the targeted object, or manipulate an item in an inventory GUI.
mouseClick(middle)	Execute a middle-button mouse interaction.
keyPress(w)	Move forward.
keyPress(s)	Move backward.
keyPress(a)	Strafe left.
keyPress(d)	Strafe right.
keyPress(space)	Jump.
keyPress(e)	Open or close the inventory and the $2 \times2$ crafting grid.
keyPress(ESC)	Close the currently opened inventory or GUI.
keyPress(q)	Drop one item from the selected item stack.
keyPress(1-9)	Select the corresponding hotbar slot.
keyPress(left.ctrl)	Sprint during movement. When combined with q, drop the entire selected stack.
keyPress(left.shift)	Move carefully or perform a modified inventory operation, such as transferring a stack between the inventory and hotbar.
no_op	Wait for one 50 ms action step without interacting with the environment.

The online adaptation loop is

The resulting evidence may subsequently support a harness proposal, which is accepted or rejected through paired sandbox validation. Persistent probing statistics are updated during learning rollouts, not during paired or holdout evaluation, where writeback is disabled.

For the sand task, KPHarness loads backup_and_turn, reacquire_and_center, and the candidate approach_and_act. At decision turn 23, the context mine_block:stall_heavy selects backup_and_turn. At turn 24, the combined orientation+target_loss+zero_progress context selects reacquire_and_center. These general strategies are specialized into the task-specific instruction:

Look down, center the crosshair on a sand block, and continuously attack it.

The resulting harness completes the task in 29 decision turns, compared with the 100-turn failure of Auto-K. Following success, backup_and_turn and reacquire_and_center receive delayed credits of 0.8 and 1.0, respectively, allowing the experience to influence probing on subsequent tasks.

The three settings can therefore be summarized as

The key distinction is not whether probe actions are present, but whether their outcomes are used to test, specialize, and update reusable recovery knowledge. Because Auto-K and KPHarness also acquire different knowledge and task-specific prompts, this example illustrates their system-level behavior rather than isolating the causal effect of the probing policy alone.

Theoretical Derivation of Knowledge–Probing Co-Evolution

This appendix develops the mathematical rationale for the probing and validation rules in KPHarness. We derive the immediate utility and delayed credit, characterize the transfer-regularized value estimator, motivate the UCB-inspired exploration term, and analyze paired harness validation. The estimation results are conditional on their stated assumptions and describe individual components of the evolving harness.

Context-Conditioned Probe Selection

Consider either the action-probing policy or the knowledge-probing policy. Let $x$ denote its task and failure context, and let $v \in\mathcal{V}(x)$ denote an available action probe or recovery strategy. Both policies use the following calculations while maintaining separate statistics. For a fixed context–candidate pair, define

where $q_i$ is the utility assigned to a selection. When $N > 0$, the local empirical mean is $\widehat{q}_{x,v} = Q/N$. The cross-context mean $\bar{q}_v$ provides a shared reference value for candidate $v$.

The local estimation target is

This quantity measures expected credited utility under the current execution conditions. For the statistical results below, we consider a fixed stage with stationary utility distributions and completed feedback. The operational system may update its harness and knowledge bank between stages.

Immediate Utility as Scalarized Feedback

A selected probe produces task-progress change $\Delta p_t$, environment reward $r_t$, observation novelty $n_t$, interaction cost $C_t$, and a failure indicator $F_t = \mathbf{1}[\mathrm{fail}_t]$. A constrained local objective that motivates their combination is

For nonnegative multipliers $\eta_c$ and $\eta_f$, the Lagrangian is

The final two terms are independent of the selected probe. Writing a fixed per-selection charge as $c_p=\eta_c C_t$ and setting $w_f=\eta_f$ gives

This recovers the immediate utility used in the main text. The coefficients specify the relative value of progress, reward, novelty, interaction cost, and failed execution. KPHarness uses fixed coefficients rather than optimizing the dual variables in (22). Their numerical values are design hyperparameters reported in the experimental setup.

For action probing, the novelty term rewards newly revealed interaction evidence. Knowledge probing sets $w_n=0$ and evaluates the progress and reward following application of the selected recovery strategy. This scalarization supplies a common update signal for heterogeneous outcomes; novelty serves as an operational proxy for informative interaction.

Delayed Success Credit and Statistical Updates

Let

be the terminal-success bonus. For a selection at timestep $t$, let $d_t$ denote its nonnegative distance to terminal feedback on the credit-assignment clock. Define the terminal contribution at distance $d$ recursively by

Repeated substitution yields $G(d)=\gamma^dY$, and hence

Equivalently, when selection and termination are indexed by the same discounting clock, a terminal-only reward sequence satisfies

Under this indexing convention, $d_t=T-t$.

The complete credited utility of a selection is

It combines local feedback with the discounted terminal outcome. At selection time, the statistics are updated as

At episode completion, each recorded selection receives the additional update

The count is incremented only once because both feedback terms belong to the same selection. Cross-context aggregates incorporate the same updates.

For repeated selections of a given $(x,v)$ within an episode, define the terminal eligibility

The total terminal update is then $Q_{x,v}\leftarrow Q_{x,v}+Y e_{x,v}$, which is algebraically identical to summing the individual delayed bonuses.

Delayed credit records association with successful execution. It does not identify the causal contribution of an individual intervention. Persistent updates occur during learning rollouts; paired, holdout, and final frozen evaluations disable writeback.

Transfer-Regularized Value Estimation

For sparse contexts, the local mean $Q/N$ can be noisy and is undefined when $N = 0$. We estimate local utility by regularizing toward the current cross-context value $\bar{q}_v$:

The first term fits local observations, and the second incorporates experience accumulated across contexts.

Proof. Differentiating the objective gives

The stationary condition therefore implies

Since $\partial^2L/\partial\mu^2=N+\lambda>0$, the stationary point is the unique global minimizer. □

For $N>0$, this estimate is the convex combination

Thus $\lambda$ is the effective weight of transferred experience. At $N=0$, the estimate equals the reference value $\bar{q}_v$. As local observations accumulate with fixed $\lambda$, their weight approaches one.

Gaussian interpretation. For an idealized Gaussian likelihood $q_i\mid\mu\sim\mathcal{N}(\mu,\sigma^2)$ and an independently specified prior $\mu\sim\mathcal{N}(\bar{q}_v,\tau^2)$, the negative log posterior is, up to an additive constant,

Multiplication by $\sigma^2$ recovers (32) with $\lambda=\sigma^2/\tau^2$. In KPHarness, the cross-context value is data-dependent, so the regularized optimization interpretation applies directly; the independent-prior interpretation describes an idealized statistical model.

The regularized estimator has smaller MSE than the local sample mean if and only if

Proof. Using $\hat{\mu} = (\sum_i q_i + \lambda m)/(N + \lambda)$ gives

Adding squared bias and variance proves Equation (38). The unregularized sample mean has MSE $\sigma^2/N$. Consequently,

which gives Equation (39). □

The proposition explains how related contexts can improve sparse local estimation: variance reduction outweighs transfer bias when the reference value is sufficiently close to the local mean. Its assumptions concern a fixed reference and sample count; a changing, data-dependent reference requires additional analysis.

Derivation of the UCB-Inspired Selection Score

The exploration term follows the optimism principle underlying UCB [1], with decisions conditioned on task and failure context. The context-conditioned formulation is related to contextual bandits [21].

Concentration of the local mean. Assume that completed utilities for a fixed $(x,v)$ are independent, stationary, and bounded in $[a,b]$, with width $B=b-a>0$. For $N>0$, Hoeffding’s inequality gives

Setting the right-hand side equal to $\delta\in(0,1)$ yields

Thus the local mean lies within $r_N(\delta)$ of its expectation with probability at least $1-\delta.

For a fixed finite set of $M$ candidates, a simultaneous version follows by allocating

Because $\sum_{v=1}^{M}\sum_{n=1}^{\infty}\delta_{v,n}=\delta$, a union bound gives, simultaneously for all candidates and sample counts,

with probability at least $1-\delta$. Under these stationary sampling assumptions, the bound can be evaluated at an adaptively selected count.

Practical exploration bonus. Taking a confidence schedule $\delta_x=(1+N_x)^{-\alpha}$, with $\alpha>0$ and $N_x\ge1$, gives

which motivates the dependence $\sqrt{\log(1+N_x)/N}$. KPHarness adopts the smoothed bonus

The additive constants make the expression defined at zero visits. The coefficient $\beta$ controls the strength of exploration.

Combining Equation (33) with Equation (47) gives

The policy selects

The first term estimates utility using local and cross-context experience; the second favors less frequently selected candidates. At zero total visits, the exploration term is zero and selection is determined by reference values and tie-breaking.

Confidence interpretation with transfer. The regularized estimate satisfies the identity

On the concentration event in Equation (43), the triangle inequality gives

This inequality remains valid for a data-dependent reference value. If $\left|\bar{q}_v-\mu_{x,v}\right|\leq D_{x,v}$ is additionally known, a corresponding upper bound is

Equation (48) uses a smoothed exploration bonus without explicitly estimating this transfer-bias term. We therefore describe the implemented rule as UCB-inspired. Its design is motivated by concentration, while the conditional bound above identifies the additional quantity required for a calibrated confidence interpretation.

Paired Harness Validation

Fix an incumbent harness $H_k$ and a candidate $\widetilde{H}_k$. For an evaluation condition $(g,s)$, define

Here $s$ indexes evaluation randomization, and both harnesses use the same task configuration, score, and initial external-state snapshot. For $m=\lvert V_k\rvert$ paired conditions, the empirical improvement is

For a candidate fixed independently of identically distributed evaluation draws, linearity of expectation gives

The variance of a paired difference is

Positive covariance under shared task conditions reduces the variance relative to independent evaluation. Isolated external-state copies and disabled writeback prevent one condition from changing the persistent state used by the other.

Acceptance as an empirical decision. Let $\widehat{J}(H)$ denote the mean score on the paired conditions. Charging an improvement margin $\epsilon_{\mathrm{pair}}$ for replacing the incumbent gives

With ties resolved in favor of the candidate, the candidate is selected exactly when

or equivalently $\widehat{\Delta J}_k \ge\epsilon_{\mathrm{pair}}$. Thus

Conditional statistical interpretation. Suppose $J \in[A_J,B_J]$ with width $W_J=B_J-A_J$, and the $m$ paired differences are independent for a candidate fixed before evaluation. Then $D_i \in[-W_J,W_J]$, and one-sided Hoeffding gives

Consequently, with probability at least $1-\delta$,

A positive lower bound would establish expected improvement under these assumptions. The implemented acceptance margin is an empirical criterion, and adaptive reuse of evaluation data requires additional statistical control. Holdout assessment provides separate evidence after acceptance and does not alter the original paired decision.

Evidence-Conditioned Knowledge Selection

For a fixed harness state, let $a$ be an action probe, $y$ the resulting evidence, and $z$ the selected recovery knowledge. The two-stage selection process can be written as

For discrete evidence, the integral is replaced by a sum. Action probing changes the evidence available to knowledge selection. Execution feedback updates both selection policies, while validated recoveries expand $\mathcal{K}$.

Stage	Parallel GPUs	Training/interaction budget	Environment steps	Time accounting	GPU-hours
Cold-start SFT	16	16,200 updates	–	121.3 h wall-clock	1,940.3
GRPO post-training	32	400 updates	–	≥379.8 h cumulative active job time	≥12,155.0
Knowledge search and co-evolution	8	1,861 episodes	175,013	≥547.5 h summed worker time	≥547.5
Knowledge–probing ablation	8	440 episodes	41,777	124.6 h summed worker time	≥124.6
Frozen transfer evaluation	8	180 episodes	16,709	6.94 h job wall-clock	55.5
				Total recorded allocation	≥14,822.9

The proposition explains the role of diagnostic evidence in knowledge selection. Real probes incur costs, may change the state, and are followed by approximate decisions, so their realized value is evaluated through interaction. In KPHarness, this evaluation also changes future probe preferences and the available recovery knowledge, providing the feedback coupling underlying knowledge–probing co-evolution.

The regularized estimator and credit updates follow exactly from their specified objectives. The concentration and transfer results apply under the stated local sampling assumptions. Harness evolution, changing recovery policies, and pending terminal feedback can violate stationarity or complete observation. Accordingly, the analysis explains the components of KPHarness without asserting a global regret or monotonic-improvement guarantee for the full evolving system.

Computational Cost and Interaction Budget

Our evaluation uses a two-stage trained VLA:

The resulting GRPO checkpoint $\pi_{400}$ is then frozen and shared by Bare, Fixed, Auto-K, and KPHarness. Thus, no method-specific gradient update is performed during knowledge search, adaptive probing, or evaluation. Nevertheless, both the upstream actor training and the subsequent closed-loop sandbox interaction incur substantial computational cost.

Cold-start supervised fine-tuning. The cold-start stage performs full-parameter supervised fine-tuning of the Qwen3-VL-8B actor using 16 GPUs. The GRPO stage is initialized from the cold-start checkpoint at step 16,200. At this checkpoint, cold-start training has completed 2.72 epochs and consumed 121.3 hours of wall-clock time, corresponding to approximately 1,940.3 GPU-hours.

GRPO post-training. Starting from the cold-start checkpoint $C_{16200}$, the actor is further post-trained with GRPO using 32 H800 GPUs: eight GPUs for generation and 24 GPUs for actor optimization, with actor data parallelism 12 and tensor parallelism 2. The final actor used in all experiments is checkpoint 400. The training underwent several infrastructure restarts. Summing the active job intervals recorded by TensorBoard up to checkpoint 400 gives at least 379.8 hours of cumulative 32-GPU job time, corresponding to at least 12,155 GPU-hours. This is a lower bound because launches that failed before writing training events are not observable in the retained logs.

Harness evolution and evaluation. After GRPO, the VLA is frozen. Harness evolution still requires repeated VLA inference and interaction with live Minecraft sandboxes. The major harness experiments use eight H800 GPUs, with one worker bound to each GPU. For these experiments, summed worker time aggregates elapsed time across single-GPU workers and therefore represents a serial-equivalent interaction cost rather than eight-GPU wall-clock time.

Across the post-training harness experiments, the retained artifacts contain 2,481 sandbox episodes, 233,499 environment steps, and 19,310 executed probe actions. Knowledge search additionally produces 831 recorded proposer responses and requires at least 1,654 external request attempts, with approximately 5.76 million provider-reported tokens. The request and token counts are also lower bounds because failed requests do not always return usage metadata.

The distinction between worker time and job wall-clock is important. For example, frozen transfer takes 6.94 hours on eight H800 GPUs, corresponding to 55.5 allocated GPU-hours, while its completed episodes account for 48.8 summed interaction hours. The remaining allocation is explained by model loading, sandbox initialization, synchronization, and worker imbalance. Similarly, the reported 547.5 hours for knowledge search is already summed across single-GPU workers and should not be multiplied by eight.

Overall, the complete pipeline requires at least 14.8K recorded GPU-hours, of which GRPO post-training is the dominant component. Harness evolution is parameter-update-free but not computation-free: its cost is dominated by repeated VLA inference, sandbox interaction, candidate validation, and external proposer calls.

AI use statement

We used an LLM for language polishing and for checking the consistency and completeness of mathematical notation. We also used Gemini-2.5-Flash through an API as the vision-language researcher and harness proposer in our experiments, as described in the methodology. Its proposals were accepted after sandbox execution and paired validation.

Ethics Statement

This work studies test-time harness evolution for a frozen vision–language–action model in the simulated Minecraft environment. It does not involve human subjects, personally identifiable information, or real-world deployment. All evaluations are conducted in controlled virtual environments. The method introduces additional computation through environment interaction, VLA inference, and multimodal researcher calls; we acknowledge the associated computational and energy costs and report the main resource requirements in the appendix. As with other adaptive agent systems, deployment in real-world environments may introduce additional safety and reliability considerations that are outside the scope of this work.

Reproducibility Statement

The paper and appendix report the VLA training configuration, evaluation protocols, probing and credit-assignment hyperparameters, harness search procedure, paired acceptance rule, low-level action space, and computational resources. We also provide the task configurations and scripts required to reproduce the reported experiments where permitted by the corresponding environments and model APIs. External model calls use the model identifiers and decoding settings specified in the experimental setup.

Computational Resources

SFT and GRPO training were conducted on NVIDIA H800 GPUs with 141 GB of memory per GPU. Policy evaluation, harness evolution, probing, and sandbox validation were performed on NVIDIA H800 GPUs with 80 GB of memory per GPU.