WorksComputer Science
EvoSkill-S1: Evolving Reusable Skills with Genetic-Inspired Operators
Abstract
Reusable skills allow language-model agents to accumulate and reuse problem-solving experience. However, most existing methods improve a skill bank only indirectly through rollout-driven accumulation and heuristic reflection, often resulting in shallow and weakly controlled updates. We present EvoSkill-S1, an RL-driven framework that instead treats reusable skills themselves as the direct objects of evolution. Given hard-negative failures, EvoSkill-S1 selects parent skills and transforms them into candidate child skills through five genetic-inspired operators: revision, cross-revision, mutation, dual-reflection, and crossover. Downstream task rewards train the skill evolver to generate more effective child skills, while evaluator-guided write-back selectively retains beneficial updates. Complementary Skill Bank Fusion consolidates operator-specific banks to exploit their distinct recovery patterns, while a Procedural Skill Interface supports transfer to unseen problems. Across all reported math and code benchmarks, EvoSkill-S1 improves the corresponding initialized skill banks under both Qwen and GPT-family solvers, with relative gains of up to 12.39%. Compared with the strongest external baseline without direct skill evolution, the largest relative improvement reaches 20.39%.
Figure 1. Heuristic skill-bank updates versus direct skill-level evolution in EvoSkill-S1, where an RL-trained evolver transforms existing skills through genetic-inspired operators.
Introduction
Large language models (LLMs) have become increasingly capable of complex reasoning, including mathematical problem solving and code genera-
Correspondence: [email protected], [email protected].
tion. Nevertheless, many challenging problems still benefit from reusable problem-solving experience rather than isolated one-shot generation [4, 36, 15, 34, 6, 5, 18] (Hendrycks et al. 2021). A natural way to provide such experience is through a skill bank: a collection of compact abstractions, strategies, and cautions that can be retrieved for related problems, echoing broader work on language-agent memory, tool use, and experience reuse [38, 28, 27, 33, 26, 31, 40].
Many existing approaches improve such banks only indirectly. As new rollouts arrive, they append new skills or summarize and reflect on additional experience [33, 31, 40]. This rollout-driven maintenance supports continual accumulation, but often leaves existing skills largely unchanged or only shallowly edited through weakly controlled heuristics. Consequently, a skill bank may continue to grow without systematically repairing brittle strategies, exploring alternative solution routes, or combining complementary partial skills.
We instead ask whether reusable skills themselves can become the direct objects of evolution. In math and code, unresolved rollout failures provide concrete diagnostic signals: they expose where retrieved skills remain incomplete, brittle, or poorly matched to the target problem [6, 19, 5, 2, 11, 13]. Rather than merely collecting more experience from these failures, they can be used to directly transform existing skills into stronger descendants.
Direct skill-level evolution, however, is nontrivial. First, different failure modes require different transformations: some skills need conservative repair, while others require cross-problem transfer, an alternative solution route, stronger guardrails, or composition with another skill. Second, a single hard negative can yield multiple plausible child skills, requiring a principled mechanism for comparing candidates and assigning credit to useful transformations. Third, evolved skills may retain artifacts specific to their source problems, limiting transfer to unseen problems. Finally, different operators may recover complementary subsets of failures, requiring a mechanism to consolidate their distinct strengths.
These challenges lead to the following question:
Can reusable skills themselves be directly and iteratively evolved into stronger descendants while remaining efficient enough for skill-bank improvement?
In response, we propose EvoSkill-S1, an RL-driven framework for direct skill-level evolution. Without updating the base solver, EvoSkill-S1 uses hard negatives as evolution targets, selects parent and support skills from the current bank, and transforms them into candidate child skills through five genetic-inspired operators: revision, cross-revision, mutation, dual-reflection, and crossover. Using GRPO-style group-relative feedback, the trainable skill evolver learns to favor child skills that improve downstream task performance, while evaluator-guided write-back retains beneficial updates and discards ineffective or redundant candidates. Because different operators exhibit complementary repair patterns, Complementary Skill Bank Fusion consolidates their operator-specific banks under the same overall skill-bank budget. A Procedural Skill Interface further supports transfer of evolved skills to unseen problems.
Experiments across math and code benchmarks show that EvoSkill-S1 consistently improves the corresponding initialized skill banks under both Qwen and GPT-family solvers, achieving relative gains of up to 12.39%. Compared with the strongest external baseline without direct skill evolution, the largest relative improvement reaches 20.39%.
This paper makes three main contributions:
RL-guided direct skill evolution. We formulate skill-bank improvement as an explicit optimization problem over reusable skills themselves and train a dedicated skill evolver with downstream task rewards to transform existing parent skills into stronger child skills, while keeping the base solver fixed.
Complementary genetic-inspired operators. We introduce five genetic-inspired operators that apply distinct transformations to reusable skills. Their complementary recovery patterns address heterogeneous failure modes and motivate the fusion of operator-specific skill banks.
Complementary Skill Bank Fusion and empirical validation. We consolidate operator-specific banks under a fixed overall budget, support the use of evolved skills on unseen problems through a Procedural Skill Interface, and demonstrate consistent improvements across math and code benchmarks.
Related Work
LLM agents, memory, and retrieval. LLM agents often combine reasoning with actions, tools, environment feedback, and persistent memory. ReAct interleaves reasoning and acting [38], Toolformer learns to invoke external tools [28], and Generative Agents maintain memory streams for interactive simulation [27]. Voyager further stores executable skills in an expanding library (Wang et al., 2024), while MemGPT manages context outside model parameters for later reuse [26]. Retrieval-augmented methods ground generation in external evidence [16, 14, 9, 3, 7], and Self-RAG jointly controls retrieval and self-critique [1]. Together, these works establish external memory and retrieval as effective mechanisms for accumulating reusable knowledge.
Figure 2. Overview of the EvoSkill-S1 pipeline. An initial skill bank is evolved through fixed-operator runs, where an RL-trained evolver generates candidate child skills and evaluator-guided write-back updates the bank. Operator-specific banks are then fused, and the resulting evolved skills are accessed through the Procedural Skill Interface at inference time.
Experience learning and procedural skills. Reflexion stores verbal feedback from prior failures [31], while ExpeL extracts reusable insights from training experience [40]. ReasoningBank distills generalizable reasoning strategies from successful and failed trajectories and combines memory retrieval with test-time scaling [25]. Skill-Pro represents procedural skills through activation, execution, and termination conditions, and uses semantic-gradient proposals with a PPO-style gate to maintain the skill memory [23]. These approaches primarily derive, retrieve, or refine reusable knowledge from episodic experience.
Skill and trajectory evolution. Recent work increasingly treats trajectories or skills as explicit optimization objects. SE-Agent evolves interaction trajectories through revision, recombination, and refinement [8]. SkillRL constructs a hierarchical skill library and co-evolves it with the solving policy through reinforcement learning [37]. Skill-R1 trains a lightweight skill generator for recurrent instance-level skill optimization using group-relative feedback across rollouts and successive revisions [32]. MetaSkill-Evolve operates at two timescales, improving both task skills and the meta-skill that controls the improvement pipeline [35]. In contrast, EvoSkill-S1 keeps the base solver fixed and trains a dedicated evolver whose optimization targets are the reusable skills in a persistent bank, rather than the solving policy or the trajectories from which skills are derived.
Problem Setup and Motivation
Problem Setup
Task setting. Let denote a collection of tasks, where is the problem input and is the reference answer or executable specification. At evolution step , a retriever scores skills with and selects the top- retrieved skill set
The solver then samples an answer conditioned on these retrieved skills, and the evaluator maps the answer to a task reward:
Here denotes exact match, unit-test pass rate, or another benchmark-specific evaluator.
Skill bank initialization. The initial skill bank is constructed from rollout data. We collect base-solver rollouts
where denotes an LLM-based skill extraction procedure performed by the initial skill evolver before reinforcement-learning-based evolution. The evolver summarizes reusable problem-solving guidance from the rollout trajectories to form the initial skill bank. For math, we run the base solver on roughly 17k DAPO problems [39] and extract the initial math skills from the resulting trajectories. For code, we apply the same pipeline to the APPS training set [11]. The resulting banks serve as the starting points for subsequent skill evolution.
Why Directly Evolve Initialized Skills?
The initialized bank leaves hard problems unsolved even when the solver retrieves relevant skills. EVOSKILL-S1 uses these failures to evolve existing skills through distinct genetic-inspired repair operators. Figure 3 shows that the operators uniquely recover different failures and that evolution changes the quality of retrieved guidance. These findings motivate both multi-operator evolution and bank fusion; the diagnostic protocol appears in Appendix § 8.§ 1.

Figure 3. Diagnostics motivating multi-operator skill evolution. The left panel separates shared and operator-unique recoveries among failures of the initialized bank. The right panel reports judge-scored changes in retrieved skill guidance after evolution.
Methodology
EVOSKILL-S1 trains a dedicated skill evolver to transform retrieved parent skills on hard negatives. Downstream rewards from the fixed solver both train the evolver and select child skills for bank write-back. Separate operator-specific runs produce banks that are fused for inference.
Hard Negative Mining
We construct the hard-negative pool once from the initialization rollouts described in the Problem Setup subsection. A hard negative is a problem whose initial base-solver rollout receives an evaluator reward below a task-specific success threshold. Formally, we define
where denotes the success threshold. For math, is determined by the benchmark answer checker; for code, it is derived from executable tests.
The hard-negative pools are fixed before skill evolution begins. At evolution step , each operator-specific run samples a target batch and retrieves parent and support skills from its current skill bank. These sampled hard negatives serve as evolution targets for the framework.
Skill Evolution Framework
The core of EVOSKILL-S1 is a coupled process that evolves the skill bank while simultaneously training a dedicated skill evolver through reinforcement learning. Evolution is conducted through five separate operator-specific runs. The formulation below describes one such run; because the operator is fixed throughout the run, we suppress operator and domain indices. Each run maintains an evolving skill bank and a trainable skill evolver . The evolver parameters are updated throughout training. The base solver parameters remain fixed.
For each target problem , the run retrieves parent and support skills from its current bank. The evolver then applies the run’s operator to generate candidate child skills:
The candidate set for target is
Each candidate child skill is evaluated by conditioning the fixed base solver on that skill and measuring its downstream task performance. The evaluation can include a small set of related neighboring problems, with their scores aggregated into the candidate reward . This reward therefore reflects whether a child skill provides useful guidance beyond only its source problem.
The candidate-level rewards serve two connected purposes. First, they provide the reinforcement-learning signal used to train the skill evolver to generate stronger child skills in subsequent iterations. Second, they guide the selection of beneficial child skills for write-back into the skill bank.
We train the evolver using GRPO-style optimization (Schulman et al., [29]; Ouyang et al., [24]; Shao et al., [30]; Yu et al., [39]). For each target problem, its candidate child skills form a comparison group, and we compute the group-relative advantage
These relative advantages train the evolver to increase the likelihood of transformations that produce more effective child skills. Unlike conventional answer-level reinforcement learning, the optimized output here is a reusable skill that can subsequently support the fixed base solver. In parallel, the downstream rewards determine which candidate child skills are written back to the bank. Let denote the accepted children from the current batch. Within the write-back budget, we retain high-reward, non-redundant candidates and update the operator-specific skill bank as
where Prune removes duplicate or low-priority entries.
Consequently, each iteration improves two components: the skill bank acquires stronger child skills, while the evolver learns to produce more effective skill transformations.
Genetic-Inspired Evolution Operators
Inspired by genetic evolution, EvoSkill-S1 treats existing skills as parent material and transforms them into candidate child skills through controlled inheritance, variation, and recombination. Rather than relying on unconstrained reflection, each operator defines a distinct transformation over the parent skills and their supporting evidence.
Specifically, Revision conservatively repairs a weak strategy while preserving its main solution
route. Cross-revision transfers a compatible motif or safeguard from a related problem without replacing the target skill’s backbone. Mutation introduces a coherent alternative route, typically changing the underlying strategy or algorithmic family. Crossover preserves a strong backbone from one parent and incorporates a compatible local improvement from another. Finally, Dual-reflection strengthens correctness-critical safeguards while keeping the main route largely stable. Together, these operators generate complementary descendants that address different forms of skill failure.
All operators follow a shared interface: they select operator-specific support evidence, transform one or more parent skills into a candidate child skill, and produce the guidance used for downstream evaluation. Detailed support-selection rules and prompt templates are provided in Appendix § 10.1.
Complementary Skill Bank Fusion
The operator-specific skill banks capture complementary repair patterns: different operators recover different subsets of failures and produce distinct forms of skill guidance. We therefore introduce Complementary Skill Bank Fusion, which consolidates the five operator-specific banks into a single evolved skill bank under the same overall bank-size budget.
Let denote the set of five evolution operators. Given operator-specific banks and a target bank size , the fusion procedure selects
Rather than simply pooling all evolved skills, the procedure balances problem coverage, source reliability, operator complementarity, and embedding diversity, while removing duplicate or low-priority records. This allows the fused bank to inherit complementary strengths without increasing the overall skill-bank budget.
We refer to this configuration as Fusion; details are provided in Appendix § 9.
Procedural Skill Interface
Skills evolved against specific hard negatives may retain source-specific details that are useful for the source problem but less suitable for retrieval on new problems. After skill-bank fusion, the Procedural Skill Interface converts the selected evolved skills into compact, retrieval-facing procedural skills and matches them to target problems at inference time. As summarized in Algorithm 1, the interface consists of three stages: skill abstraction, query matching, and solver-facing composition.
This interface separates source-specific skill evolution from retrieval-time reuse: evolution can exploit detailed feedback from hard negatives, whereas the solver receives compact and reusable guidance at inference time. The prompt templates and filtering rules are provided in Appendix § 10.
Experiments
We organize our evaluation around four research questions.
RQ1: Performance. Does direct skill evolution improve over initialized skill banks and non-direct-evolution baselines across math and code?
RQ2: Fusion. Under the same overall bank-size budget, does fusion outperform average single-operator evolution on held-out benchmarks?
RQ3: RL efficiency. Does RL-based training of the skill evolver improve evolution efficiency under a comparable write-back budget?
RQ4: Proceduralization. Does the Procedural Skill Interface improve the retrieval and reuse quality of evolved skills on unseen problems?
Experimental Setup
Evolution targets and evaluation benchmarks. Separate problem sets are used for skill evolution and held-out evaluation. Fixed hard-negative pools are constructed from problems on which the base solver fails during skill-bank initialization. This procedure yields approximately 4,000 math problems from DAPO ([39]) and 2,200 code problems from the APPS training set ([11]). These hard-negative pools serve as evolution targets and support in-domain diagnostic analyses.
The resulting skill banks are evaluated on separate held-out benchmarks. The math evaluation suite comprises AIME24, AIME25, AMC23, Minerva Math, and OlympiadBench ([21, 22, 20, 17, 10]). The code evaluation suite comprises APPS-test and LiveCodeBench ([11, 13]). Accuracy is reported for math, while mean unit-test pass rate is reported for code.
Baselines. The comparison includes representative paradigms spanning no external support, trajectory reuse, experience-derived reflection, and static skill-bank retrieval. No skill evaluates the solver without retrieved skill support, whereas Traj RAG retrieves relevant trajectory-derived guidance. Reflexion generates verbal feedback from prior failures, and ExpeL-SI extracts reusable skills from accumulated experience ([31, 40]). Init uses the initialized skill bank without subsequent evolution. Among the proposed evolution settings, Single-op reports the arithmetic mean across the five independently evolved operator-specific banks, while Fusion consolidates these banks under the same overall bank-size budget.
The representative baselines are adapted to the common fixed-solver, retrieval, and evaluation protocol; implementation details appear in Appendix B.
Backbones and implementation. Qwen3-4B-Instruct serves as both the primary open-weight solver and the initialization for the separately trained skill evolver; the solver remains frozen during RL training. GPT-family experiments use GPT-4.1 mini for math and code uses GPT-4o mini. Tiny banks contain 500 initialized skills, whereas large banks use the full curated bank for each domain. Within each setting, all methods share the same solver, retriever, evaluator, prompts, decoding configuration, and metrics. Repeated-run statistics, seed settings, and implementation details are provided in Appendix B.
Main Results
Table 1 supports two main takeaways.
| Model | Method | AIME 24 | AIME 25 | AMC 23 | Minerva Math | Olymp. Bench | APPS-test | LiveCode Bench |
| Qwen3-4B-Instruct | No skill | 53.75 | 42.08 | 87.50 | 16.18 | 40.65 | 36.90 | 41.98 |
| Traj RAG | 57.08 | 40.83 | 91.56 | 29.04 | 54.15 | 28.58 | 31.17 | |
| Reflexion | 57.92 | 42.92 | 93.12 | 32.35 | 66.77 | 41.82 | 42.18 | |
| ExpeL-SI | 59.17 | 41.67 | 92.50 | 30.88 | 66.32 | 41.64 | 39.63 | |
| Tiny Skill Bank | ||||||||
| Init | 60.42 | 45.00 | 91.56 | 33.82 | 66.32 | 44.31 | 41.92 | |
| Single-op | 61.50 | 45.75 | 92.31 | 34.93 | 68.63 | 44.79 | 43.44 | |
| Fusion | 62.92 | 47.08 | 92.81 | 36.03 | 69.88 | 45.08 | 45.55 | |
| Large Skill Bank | ||||||||
| Init | 61.67 | 46.67 | 92.19 | 37.13 | 67.36 | 44.38 | 44.02 | |
| Single-op | 66.67 | 50.42 | 93.44 | 37.13 | 67.95 | 44.75 | 44.78 | |
| Fusion | 67.08 | 51.67 | 94.38 | 38.24 | 68.40 | 44.92 | 45.17 | |
| GPT Family | No skill | 44.17 | 42.50 | 79.38 | 25.37 | 45.99 | 52.51 | 41.54 |
| Traj RAG | 50.42 | 44.68 | 85.32 | 32.35 | 61.35 | 41.85 | 50.76 | |
| Reflexion | 50.42 | 46.54 | 86.12 | 31.99 | 60.83 | 42.23 | 46.44 | |
| ExpeL-SI | 47.92 | 45.83 | 86.57 | 33.89 | 59.94 | 52.63 | 47.01 | |
| Tiny Skill Bank | ||||||||
| Init | 48.75 | 42.92 | 86.25 | 33.46 | 61.72 | 53.80 | 52.80 | |
| Fusion | 50.83 | 43.75 | 86.88 | 34.12 | 62.02 | 53.85 | 53.50 | |
| Large Skill Bank | ||||||||
| Init | 48.75 | 43.75 | 88.75 | 34.56 | 60.24 | 53.98 | 53.94 | |
| Fusion | 52.08 | 49.17 | 89.69 | 35.29 | 61.57 | 54.13 | 55.53 |
Table 1. Main results on math and code benchmarks. Math columns report accuracy (%), while code columns report mean unit-test pass rate (%). Single-op denotes the arithmetic mean across the five independently evolved operator-specific banks, while Fusion denotes Complementary Skill Bank Fusion under the same overall bank-size budget. Shaded rows indicate our evolved skill-bank variants, and bold values mark the best result within each backbone and skill-bank-size block. For the GPT Family rows, math uses GPT-4.1 mini and code uses GPT-4o mini.
Takeaway 1: Direct skill evolution consistently improves initialized skill banks. Across both bank sizes and both solver families, Fusion outperforms the corresponding Init setting on every reported math and code benchmark. For Qwen3-4B-Instruct, the improvements are particularly pronounced on AIME24 and AIME25. The same consistent trend holds for the GPT-family solvers, showing that the evolved skill banks remain effective beyond the primary open-weight solver setting.
Takeaway ❷: Complementary Skill Bank Fusion improves over average single-operator evolution. For Qwen3-4B-Instruct, Fusion outperforms Single-op on every benchmark under both the tiny- and large-bank settings. This consistent advantage supports consolidating complementary operator-specific banks under a fixed overall budget.
Efficiency Analysis
Takeaway ❸: RL training improves skill-evolution efficiency without simply increasing write-back volume. Figure 4 compares each operator with and without RL training of the skill evolver. Solved once denotes the number of hard negatives solved at least once during evolution, while written-back children denotes the number of child skills accepted into the bank. Across both math and code, RL-trained evolvers recover more hard negatives under comparable write-back budgets, indicating that RL enables the evolver to generate more effective child skills.
Figure 4. Effect of RL training on skill-evolution efficiency. Solved once counts hard negatives solved at least once during evolution, while written-back children counts child skills accepted into the bank.
Operator Analysis
Takeaway ❹: The five genetic-inspired operators exhibit complementary strengths. Figure 3 shows operator-unique recoveries. Figure 5 and Appendix § 8.§ 5 provide the full comparisons. Fusion combines these complementary strengths under the same bank-size budget (Table 1).
Figure 5. Operator-level comparison for the five skill evolution operators. The operators emphasize different support metrics, explaining why Complementary Skill Bank Fusion can outperform reliance on a single operator.
Proceduralization Analysis
Takeaway ❺: The Procedural Skill Interface improves both downstream performance and retrieval-facing skill quality. Figure 6 shows that proceduralization improves the initialized bank on six of seven benchmarks and the Crossover bank on all seven. Paired judgments of all five operators’ outputs show gains in five interface-quality dimensions, with smaller changes in procedural guidance. The protocol appears in Appendix § 8.§ 2.

Figure 6. Effect of proceduralization on downstream performance and skill-interface quality. Top: performance before and after proceduralization for the initialized and Crossover skill banks. Bottom: paired-judge score differences between proceduralized skills and source skills across six interface-quality dimensions; positive values indicate improvement.
Conclusion
We presented EVOSKILL-S1, an RL-driven framework that directly evolves reusable skills while keeping the base solver fixed. Given hard-negative problems, a dedicated skill evolver transforms existing parent skills into candidate child skills through genetic-inspired operators, with downstream rewards jointly training the evolver and guiding selective write-back. Complementary Skill Bank Fusion consolidates operator-specific banks, while the Procedural Skill Interface supports retrieval-time reuse. Experiments across math and code show consistent improvements over initialized skill banks under both Qwen3-4B-Instruct and GPT-family solvers, with analyses confirming the complementary strengths of the operators and the efficiency gains from RL.
Further Method and Operator Analysis
Algorithm 1: Procedural Skill Interface
Input: evolved skill bank , target problem , retriever . Output: solver support .
Skill abstraction: convert each evolved skill into one or more reusable procedural skills.
.
Query matching:
Extract problem keys: .
Retrieve procedural skills: .
Filter retrieved skills by relevance, diversity, and reliability: .
Support composition: .
return .
Operator-Level Interpretation
Figure 5 shows that the relative gains of different operators vary across benchmarks, consistent with the operator-unique recoveries observed in Figure 3. Detailed results in the Supplementary Material further show that no single operator consistently dominates across all benchmarks. This complementarity motivates Complementary Skill Bank Fusion, which consolidates the distinct strengths of the operator-specific banks. In Table 1, Single-op reports the arithmetic mean across the five independently evolved banks, whereas Fusion measures the benefit of consolidating them under the same overall bank-size budget.
Baseline Comparability
Several recent skill-learning systems are tightly coupled to specific agent environments, skill representations, or jointly optimized solving policies, making their complete pipelines unsuitable for direct comparison in the fixed-solver math and code setting. Accordingly, benchmark-adapted instantiations of representative mechanisms are implemented under a common protocol. Traj RAG captures trajectory-level reuse, Reflexion captures failure-driven reflection, and ExpeL-SI captures experience-to-skill induction. These implementations isolate the core mechanism of each paradigm while controlling for the solver, retrieval, and evaluation settings.
Additional Experimental Details
Motivation Diagnostic Setup
The motivation diagnostic reported in the main paper uses the math-side DAPO-hard pool mined from failed initialization rollouts. We first retain the six-way intersection of problem identifiers with results under the initialized bank and all five operator-specific evolved banks: Revision, Cross-revision, Mutation, Crossover, and Dual-reflection. The diagnostic set consists of the first 1,000 problem identifiers in this retained pool. All six settings use the same solver, retrieval protocol, and evaluator, so differences in the diagnostic are attributable to the retrieved skills rather than changes in the test set or solver configuration.
Operator complementarity. The operator-complementarity panel in the main paper measures which initialized failures are recovered by each evolved skill bank. Let
be the subset of the shared evaluation set missed by the initialized bank. For an operator , let indicate whether the bank evolved by solves . We define the recovered set as
We decompose into shared and unique recoveries:
The bar height is , while the unique segment is . This analysis tests whether the operators simply recover the same failures or provide complementary repair behavior.
Retrieved-skill repair signature. The retrieved-skill repair-signature panel in the main paper studies how retrieved skills change on flip cases where the initialized solver fails and the operator-specific solver succeeds. For each operator, we independently shuffle its flip set with random seed 29 and retain at most 50 cases. For each case, we collect the retrieved skill context shown to the solver from the initialized bank and from the operator-specific evolved bank. The openai/gpt-5.4 judge is queried with temperature 0.0 and scores each retrieved context on a 1–5 scale along routine support, validation support, route diversity, and transfer support. Each score is the four-call mean defined in Section B.4. Routine support measures whether the retrieved skills provide executable steps; validation support measures whether they provide checking mechanisms; route diversity measures whether they give complementary solution routes rather than repeated hints; and transfer support measures whether the guidance abstracts beyond the source problem surface.
For operator and dimension , the plotted value is
where is the sampled case set, and are the retrieved contexts from the initialized and evolved banks, and is the judge score for dimension .
Proceduralization Judge Study
The proceduralization analysis in Figure 6 of the main paper uses paired comparisons between source skills and the proceduralized units produced from them. We evaluate outputs from the five operator-specific math skill banks: Revision, Cross-revision, Mutation, Crossover, and Dual-reflection. Each proceduralized unit retains identifiers for its source skill, problem, and operator, enabling alignment with the source skill that produced it. When one source skill yields multiple procedural units, those units are treated as one proceduralized support group.
For each operator bank , we shuffle the aligned pairs with sampling seed 20260526 and retain the first 100 pairs
where is the source skill and is its proceduralized support group. This gives paired examples. The judge scores both sides of each pair on a 1–5 scale along six dimensions: applicability, whether the skill states when it should be used; transferability, whether it abstracts a pattern applicable to unseen related problems; robustness, whether it remains reliable under ambiguous or failure-prone conditions; coverage, whether it captures the essential conditions and solution components; retrieval utility, whether it would help as top- retrieved support; and procedural guidance, whether it provides concrete, actionable solving guidance.
Judge configuration. We use openai/gpt-5.4 with temperature 0.0 and a maximum output length of 900 tokens. Responses are constrained to JSON objects. Each request has a 180-second timeout and permits up to three retries, and evaluation jobs are distributed across eight shards. Each reported judge score uses the four-call averaging protocol in Section B.4.
For dimension , we compute the paired improvement
The paired comparison shows consistent gains in applicability, transferability, robustness, coverage, and retrieval utility. Procedural guidance remains close to the source-skill level, indicating that the interface primarily improves when and how skills can be retrieved and reused rather than merely lengthening their procedural content.
Benchmarks and Metrics
Table 6 summarizes the benchmark splits, sources, and evaluation sizes. Math benchmarks use answer-level accuracy after benchmark-specific normalization, while code benchmarks use mean unit-test pass rate.
| Benchmark | Domain | Split / source | Size | Use and metric |
| DAPO-hard | Math | Hard subset mined from the DAPO initialization pool after base-solver rollouts. | 4k | Evolution target pool; answer-level correctness. |
| AIME24 | Math | Official 2024 AIME I and II problems (Mathematical Association of America, [21]). | 30 | Held-out math evaluation; answer-level correctness. |
| AIME25 | Math | Official 2025 AIME I and II problems (Mathematical Association of America, [22]). | 30 | Held-out math evaluation; answer-level correctness. |
| AMC23 | Math | 2023 AMC 12 A/B aggregate (Mathematical Association of America, [20]). | 40 | Held-out math evaluation; answer-level correctness. |
| Minerva Math | Math | Minerva quantitative-reasoning test set (Lewkowycz et al., [17]). | 272 | Held-out math evaluation; answer-level correctness. |
| OlympiadBench | Math | Text-only open-ended math subset (He et al., [10]). | 675 | Out-of-distribution math evaluation; answer-level correctness. |
| APPS-train-hard | Code | Hard subset mined from APPS training rollouts. | 2.2k | Evolution target pool; mean unit-test pass rate. |
| APPS-test | Code | Official APPS test split (Hendrycks et al., [11]). | 5,000 | Held-out code evaluation; mean unit-test pass rate. |
| LiveCodeBench | Code | 131-problem code-generation subset (Jain et al., [13]). | 131 | Out-of-distribution code evaluation; mean unit-test pass rate. |
Table 6. Benchmark splits, sources, and evaluation sizes. Internal hard subsets are mined only from initialization rollouts; held-out and out-of-distribution benchmarks are used for evaluation.
Computational budget. The primary open-weight backbone is Qwen3-4B-Instruct, a 4B-parameter model. Each RL-driven skill-evolution training run uses four NVIDIA A100 GPUs for 24 hours. GPT-family mini models are accessed through hosted APIs.
Experimental setup and hyperparameters. We use fixed experimental settings across methods within each backbone and domain rather than tuning hyperparameters separately for each baseline. All methods are evaluated with the same solver, retrieval protocol, evaluator, benchmark prompts, and metric scripts. We do not perform hyperparameter search on test benchmarks; skill-bank size, operator setting, write-back budget, and evaluation protocol are fixed before the final evaluations.
Reporting convention and decoding parameters. Unless otherwise specified, decoding uses temperature 0.2, top- 0.9, top- 20, and a maximum generation length of 16,384 tokens.
Repeated Evaluation and Judge Aggregation
Benchmark evaluation. For every fixed solver–skill-bank configuration and benchmark, we conduct eight independent evaluation runs. Each run evaluates the full benchmark once and samples one solver completion per problem. We use inference seeds for all methods compared within a backbone–domain setting, enabling paired comparisons under matched sampling conditions. If is the benchmark-level metric from run , the reported value is
These repetitions measure evaluation-time stochasticity; the skill evolver and skill bank are not retrained between evaluation runs.
Tables 2–5 report , where is the empirical standard deviation of the eight run-level metrics. All values are percentage points.
Judge aggregation. For each item and scoring dimension, we perform four independent judge calls with the same model, rubric, input, and decoding configuration. Let denote the score assigned to item on dimension by judge call . We first compute
and then aggregate these item-level means across examples and operators. Thus, the judge quantities used in the diagnostic and proceduralization analyses are four-call averages rather than single-call scores.
Artifact licenses and terms. We use publicly available benchmarks and model artifacts for research evaluation and cite their original creators. We do not redistribute third-party datasets or model checkpoints. Any released code or derived skill-bank artifacts are intended for research use and will be distributed only in accordance with the applicable licenses and terms of the underlying resources.
Detailed Single-Operator Results
Baseline Configuration
No skill removes skill retrieval from the solver context. Traj RAG retrieves trajectory-derived support but does not evolve it. Reflexion uses a reflection-style feedback update, and ExpeL-SI uses experience-summary-style skill induction. Init uses the initialized skill bank without evolution. Single-op is the arithmetic mean across the five independently evolved operator-specific banks, while Fusion uses Complementary Skill Bank Fusion under the same overall bank-size budget. All methods are evaluated with the same benchmark prompts and metric scripts within each backbone block.
Implementation Details
Math skill banks are initialized from rollouts over roughly 17k DAPO problems, and the approximately 4k failed problems form DAPO-hard for subsequent evolution. Code skill banks are initialized from APPS training rollouts, and evolution is applied to approximately 2.2k failed APPS training examples. Tiny banks contain 500 initialized skills, while large banks use the larger curated bank. Code uses execution-based pass-rate signals; math uses answer-level correctness.
GRPO training configuration. Table 8 summarizes the optimization, batching, and write-back settings used for the skill evolver.
Each target problem produces four candidate child skills, which form its GRPO comparison group. Training runs for five epochs without a separate maximum-step limit, so the epoch schedule determines the total number of updates. After evaluation, at most two accepted child skills per problem are written back in each evolution round. The global update batch matches the rollout batch, while the update and experience phases use different per-GPU micro-batches.
Complementary Skill Bank Fusion Details
For the full-size fused banks, the target size is for math and for code. We set the failure-like record ratio to and retain at most raw skill records for any problem identifier.
Each raw skill record has an operator source , problem identifier , score , correctness metadata, and optional textual fields such as procedural steps, watchouts, and solution traces. We use and for incorrect and correct metadata. A record is reliable when and either or holds. A record is failure-like when holds or
| Skill bank | Method | AIME 24 | AIME 25 | AMC 23 | Minerva Math | OlympiadBench |
| None | No skill | |||||
| Traj RAG | ||||||
| Reflexion | ||||||
| ExpeL-SI | ||||||
| Tiny | Init | |||||
| Single-op | ||||||
| Fusion | ||||||
| Large | Init | |||||
| Single-op | ||||||
| Fusion |
Table 2. Repeated-evaluation results for the Qwen3-4B-Instruct math benchmarks. Values are mean standard deviation over eight inference seeds; the corresponding means appear in Table 1.
| Skill bank | Method | APPS-test | LiveCodeBench |
| None | No skill | ||
| Traj RAG | |||
| Reflexion | |||
| ExpeL-SI | |||
| Tiny | Init | ||
| Single-op | |||
| Fusion | |||
| Large | Init | ||
| Single-op | |||
| Fusion |
Table 3. Repeated-evaluation results for the Qwen3-4B-Instruct code benchmarks. Values are mean standard deviation over eight inference seeds; the corresponding means appear in Table 1.
| Skill bank | Method | AIME 24 | AIME 25 | AMC 23 | Minerva Math | OlympiadBench |
| None | No skill | |||||
| Traj RAG | ||||||
| Reflexion | ||||||
| ExpeL-SI | ||||||
| Tiny | Init | |||||
| Fusion | ||||||
| Large | Init | |||||
| Fusion |
Table 4. Repeated-evaluation results for the GPT-family mini math benchmarks. Values are mean standard deviation over eight inference seeds; the corresponding means appear in Table 1.
| Skill bank | Method | APPS-test | LiveCodeBench |
| None | No skill | ||
| None | Traj RAG | ||
| None | Reflexion | ||
| None | ExpeL-SI | ||
| Tiny | Init | ||
| Tiny | Fusion | ||
| Large | Init | ||
| Large | Fusion |
Table 5. Repeated-evaluation results for the GPT-family mini code benchmarks. Values are mean standard deviation over eight inference seeds; the corresponding means appear in Table 1.
. Failure-like records are not used as positive procedures during retrieval, but they may still yield useful Guard or Validation units.
| Method | Math | Math | Math | Math | Math | Math | Code | Code | Code |
| DAPO-hard | AIME 24 | AIME 25 | AMC 23 | Minerva Math | Olympiad Bench | APPS-train-hard | APPS-test | LiveCode Bench | |
| Revision-500 | 12.77 | 61.25 | 46.25 | 92.19 | 34.56 | 67.95 | 20.61 | 44.98 | 44.15 |
| Cross-revision-500 | 13.50 | 62.92 | 45.42 | 91.56 | 34.93 | 69.29 | 21.12 | 44.39 | 43.12 |
| Mutation-500 | 13.00 | 60.83 | 46.25 | 92.19 | 35.66 | 68.69 | 21.49 | 45.12 | 42.75 |
| Crossover-500 | 13.09 | 61.25 | 45.00 | 92.81 | 34.93 | 69.14 | 21.42 | 44.64 | 43.23 |
| Dual-reflection-500 | 13.27 | 61.25 | 45.83 | 92.81 | 34.56 | 68.10 | 20.66 | 44.83 | 43.96 |
| Single-op-500 (avg.) | 13.13 | 61.50 | 45.75 | 92.31 | 34.93 | 68.63 | 21.06 | 44.79 | 43.44 |
Table 7. Detailed tiny-skill-bank single-operator results for Qwen3-4B. The average row is the arithmetic mean over the five independently evolved operator-specific banks. Code columns report mean unit-test pass rate (%).
| Setting | Value |
| Learning rate | |
| Rollout batch | 32 |
| Global update batch | 32 |
| Update micro-batch / GPU | 2 |
| Experience micro-batch / GPU | 8 |
| Candidates per target | 4 |
| Training epochs | 5 |
| Explicit step cap | None |
| Write-back cap | 2 / problem / round |
Table 8. GRPO training and write-back settings for the skill evolver.
Algorithm 1: Operator-Fused QID-Covered Skill Consolidation
Input: operator skill sets , domain-specific target size , failure ratio , max skills per question . Output: consolidated raw skill bank .
Initialize and soft operator quotas .
Normalize all records; compute , reliability, failure-like flag, richness score, and source hash.
Deduplicate by source hash and group candidates by .
Stage 1: QID coverage. For each ordered by best candidate quality, add the best reliable and informative record, preferring under-quota operators.
Stage 2: failure evidence. While and failure count is below , add the best remaining failure-like record with non-empty procedural evidence.
Require every added record to satisfy deduplication and count .
Stage 3: complementary reliable records. Fill remaining slots with reliable records, preferring under-quota operators and under-covered problem identifiers.
Stage 4: final fill. Add the best remaining valid informative records.
return .
For the 500-skill setting, an additional diversity step within each operator pool prevents the final bank from concentrating on one semantic region. We use , reliable ratio (and hence failure-like ratio ), and . Before diversity selection, the quality-ranked candidate pool is capped at five times the corresponding selection quota. Diversity is measured using each record’s precomputed 384-dimensional memory vector.
Algorithm 2: Small-Bank Vector-Diverse Fusion
Input: operator skill sets , size , reliable ratio , per-question cap , precomputed 384-dimensional vectors . Output: diversified bank .
Set and per-operator quota .
for each operator do
Split into reliable candidates and failure-like candidates .
Keep at most one best candidate per problem identifier in each split and retain a quality pool of at most 5 times the corresponding selection quota.
Select about reliable records by farthest-first traversal over normalized vectors .
Select about failure-like records with the same rule.
Add selected records while enforcing global deduplication and one record per problem identifier.
end for
If , fill remaining slots with globally diverse high-quality candidates.
return .
After fusion, selected raw skill records are proceduralized into typed units. The retrieval bank filters positive and negative units differently.
Algorithm 3: Role-Consistent Filtering After Proceduralization
Input: candidate procedural units with source record and role . Output: filtered retrieval bank .
Initialize .
for each unit do
if then add to .
else if and is reliable then add to .
end if
end for
return .
Procedural Skill Interface
The Procedural Skill Interface (PSI) is used at retrieval time to make evolved skills usable on unseen problems. It introduces a shared procedural language between the skill bank and the target problem: evolved skills are converted into typed procedural units, while each target problem is converted into typed procedural keys for retrieval. We use the paper-facing roles Framing, Strategy, Guard, Validation, and Execution, which correspond to the implementation types Modeling, Core, Trap, Verification, and Implementation, respectively; Execution is used only for code.
Algorithm 4: Procedural Skill Interface
Input: evolved skill bank , target problem , and domain . Output: retrieval context .
Skill-side proceduralization. For each skill , generate procedural units .
Normalize each unit as (Role, Procedural Key, Trigger, Routine, Guard, Validation).
Remove source-specific constants, stories, row IDs, and answer values.
Keep Guard and Validation units; keep Framing, Strategy, and Execution units only when their source is reliable.
Encode each retained unit key with the locally hosted BGE-M3 encoder, -normalize its vector, and add it to the role-specific index .
Query-side proceduralization. Extract procedural keys .
Encode and -normalize the query keys; consolidate query subquestions whose cosine similarity is at least 0.92.
for each remaining query key do
Retrieve the top 10 role-compatible units from , ranked by the dot product of normalized vectors (cosine similarity).
end for
Merge the retrieved candidates in descending similarity-score order.
Apply role caps (Framing: 1, Strategy: 1, Guard: 1, Validation: 0, Execution: 1) and retain at most five units overall.
Compose solver-facing Routines, Guards, and Validation cues, then return .
All compared skill banks use this same retrieval setup. The top-10 search has no additional relevance-score cutoff. The 0.92 threshold is used only to consolidate query-side subquestions; it is not an embedding-based duplicate threshold for retrieved skills. The five-unit context limit is an outer cap, so the role caps can produce fewer than five units.
Prompt templates. The prompts below show the domain-conditioned templates used to construct the PSI. Math uses the roles Framing, Strategy, Guard, and Validation; code additionally uses Execution for implementation-sensitive details.
Prompt (Skill-Side Proceduralization)
SYSTEM:
You curate reusable procedural skills for
math and code.
USER:
Convert one evolved skill into 0-3 units
that help with
unseen same-domain problems, not the source
problem itself.
GOAL:
- Preserve transferable procedures,
constraints,
invariants, failure modes, and checks.
- Remove source stories, constants, row IDs,
exact
answers, dataset names, long reasoning,
and source code.
UNIT FIELDS:
- Role: Framing, Strategy, Guard, Validation
, or Execution.
- Procedural Key: a "How do I ...?"
retrieval question.
- Trigger: when the unit applies.
- Routine: reusable actions for the solver.
- Guard: a concrete mistake or false route
to avoid.
- Validation: a concrete result or
implementation check.
ROLE GUIDE:
- Framing: variables, states, constraints,
or models.
- Strategy: method, recurrence, invariant,
or reduction.
- Guard: false assumptions, shortcuts, or
boundary loss.
- Validation: candidates, invariants, cases,
or outputs.
- Execution: indexing, I/O, updates, or
formatting.
RELIABILITY:
Failed or low-score sources normally yield
only Guard or
Validation units. Use positive roles only
for reliable evidence.
INPUT:
DOMAIN: {domain}
SOURCE_PROBLEM: {problem_text}
SOURCE_SCORE: {score}
SOURCE_IS_CORRECT: {is_correct}
EVOLVED_STEPS: {steps}
EVOLVED_WATCHOUTS: {watchouts}
OPTIONAL_SOLUTION_OR_CODE: {solution_or_code
}
OUTPUT:
JSON only: 0-3 units with all six fields
above.

Figure 20.
Evolution Operator Details
The following algorithm blocks summarize how each operator selects support skills and renders its child skill.
Revision.
Algorithm 5: Same-Problem Conservative Revision
Input: target skill , skill bank . Output: revised child skill .
.
Remove target skill and skills marked inactive from .
.
.
if then
.
else if then
.
else .
.
.
.
; return .
Cross-revision.
Algorithm 6: Cross-Problem Transfer-Based Conservative Revision
Input: target skill , skill bank . Output: cross-revised child skill .
.
Remove same-qid, inactive, and duplicate candidates from .
Partition into tagged/fallback positive and negative pools.
.
.
if then choose best remaining .
.
; return .
Mutation.
SYSTEM: You extract procedural retrieval keys for unseen math or coding problems.
USER: From only the target statement, produce 2–4 keys for retrieving useful procedural units.
RULES:
Do not solve the problem or compute the final answer.
Do not overfit to samples, constants, labels, or stories.
Prefer a few high-signal keys over broad coverage.
Be more general than the exact problem but more specific than “use DP” or “check cases”.
KEY FIELDS:
Role: Framing, Strategy, Guard, Validation, or Execution.
Procedural Key: a stable “How do I …?” question.
ROLE GUIDE:
Framing: model, variables, states, or constraints.
Strategy: a transferable method without over-commitment.
Guard: likely false assumptions or traps.
Validation: tests for the answer or algorithm.
Execution: indexing, I/O, or state-update details.
INPUT: DOMAIN: {domain} PROBLEM_STATEMENT: {problem_statement}
OUTPUT: JSON only: 2–4 keys with Role and Procedural Key.
Algorithm 7: Mutation Operator
Input: target skill , skill bank . Output: mutated child skill .
.
.
Remove , inactive candidates, same-qid semantic candidates, and duplicates.
Partition into correct and incorrect semantic pools.
.
.
.
.
; return .
Dual-reflection.
Algorithm 8: Guard-Oriented Reflective Enhancement
Input: target skill , skill bank , fallback flag . Output: reflected child skill .
.
.
Remove , inactive candidates, same-qid semantic candidates, and duplicates.
Partition semantic candidates into correct and incorrect pools.
.
.
.
if and and then return .
.
; return .
Crossover.
Algorithm 9: Same-Problem Two-Parent Crossover
Input: target skill , skill bank , parent limit . Output: crossover child skill .
.
Remove , inactive/original/incorrect skills, empty plans, and duplicate plan hashes.
if then
if then return .
else return .
Sort by (score, version) in descending order.
.
Define by step difference, distance to , and negative score.
.
.
; return .
Compact Operator Prompt Specifications
For legibility, we factor the repeated output, schema, and content constraints into one shared contract, followed by one operator-specific block per operator. The deployed prompt is the concatenation of the shared contract and the selected operator block. This presentation removes repeated boilerplate and decorative separators while preserving the operator rules and implementation-specific variables in Jinja form. The source package also retains the unabridged templates.
The runtime prompt is formed by prepending the shared contract below to the selected operator-specific block. This factorization removes repeated boilerplate and decorative separators while retaining the task, constraints, input variables, and output contract.
Shared Prompt Contract
ROLE:
You are a Strategy Skill Editor.
GOAL:
Do not solve the target problem. Produce a reusable, concrete, information-dense method-level plan that a separate solver can execute. Avoid long derivations, low-level code narration, and line-by-line instructions.
OUTPUT:
Return exactly:
<think> short operator-focused reasoning </think>
followed by one valid JSON object and nothing else.
JSON RULES:
No markdown, code fences, extra text, or trailing commas.
Use double quotes for every key and string value.
Put all actionable content in the JSON object.
SCHEMA:
\{
"steps": [\{"i": 1, "text": "string"\}],
"watchouts": ["string"]
\}
CONTENT RULES:
Prefer 4–5 steps and 2–3 watchouts.
Use at most 6 steps and at most 3 watchouts.
Each step states one concrete method-level action.
Watchouts are distinct, realistic guardrails.
Protect correctness, contract, state, boundaries, implementation stability, and complexity as applicable.
Keep the result reusable across related problems.
Operator Block (Revision)
TASK:
Conservatively repair the current weak strategy.
THINK:
Identify the smallest useful repair. Do not solve the problem or replace a usable backbone.
REVISION RULES:
Stay in the same algorithm family when possible.
Use BACKBONE SUPPORT to stabilize the main route.
Use FAILURE SUPPORT to patch one weak point.
Prefer a local bridge, ordering, guard, or validation fix.
Replace the global route only if it is unusable.
APPS PRIORITY:
Treat revision as watchouts-first conservative repair.
Change steps less than watchouts when the route works.
Add a contract, complexity, boundary, or state guard.
INPUT FIELDS (Jinja):
problem: \{\{ problem \}\}
current_weak_plan: \{\{ current_plan \}\}
current_watchouts: \{\{
target_failed_watchouts \}\}
[if present]
backbone_support: \{\{ backbone_support \}\}
backbone_watchouts: \{\{
backbone_support_watchouts \}\}
[if present]
failure_support: \{\{ failure_support \}\}
failure_watchouts: \{\{
failure_support_watchouts \}\}
[if present]
RETURN:
One revised strategy object under the shared contract.
Operator Block (Cross-revision)
TASK:
Improve the target strategy using cross-problem support.
THINK:
Choose one transferable motif and one realistic guard.
Do not solve the target problem.
CROSS-REVISION RULES:
- Treat this as conservative cross-motif transfer.
- Never transplant a whole foreign route by default.
- Keep the target backbone primary.
- Borrow at most one compatible donor-local motif:
a maintenance trick, check pattern, complexity trick,
or contract-oriented watchout.
- ROUTE DONOR may supply one transferable motif.
- TRAP DONOR may supply one watchout or small repair.
- Reject donors that conflict with the target contract.
- Do not output a stitched list of donor steps.
APPS PRIORITY:
Transfer only when it improves contract, indexing, state,
complexity, or output consistency.
INPUT FIELDS (Jinja):
problem: {{ problem }}
current_weak_plan: {{ current_plan }}
current_watchouts: {{ target_failed_watchouts }}
[if present]
route_donor: {{ route_donor }} [if present]
route_donor_watchouts: {{ route_donor_watchouts }}
[if present]
trap_donor: {{ trap_donor }} [if present]
trap_donor_watchouts: {{ trap_donor_watchouts }}
[if present]
RETURN:
One cross-revised strategy object under the shared contract.
Operator Block (Mutation)
TASK:
Replace the weak strategy with a stronger alternative.
THINK:
Identify one justified backbone change in algorithm family,
decomposition, or complexity tier. Do not solve the problem.
MUTATION RULES:
- Treat CURRENT WEAK PLAN as the baseline to replace.
- Use ROUTE TO AVOID as negative evidence.
- Use DIVERGENT SEED for one stronger backbone idea.
- Rebuild steps and watchouts as one coherent package.
- Do not merely paraphrase the current plan.
- Reject a seed that repeats the route to avoid.
- Do not generate a random unrelated route.
APPS PRIORITY:
Prefer changes that fix the algorithm family, complexity
tier, decision/reconstruction contract, or state design.
A new route must include its new failure modes.
INPUT FIELDS (Jinja):
problem: {{ problem }}
current_weak_plan: {{ current_plan }}
current_watchouts: {{ target_failed_watchouts }}
[if present]
route_to_avoid: {{ route_to_avoid }}
route_to_avoid_watchouts: {{ route_to_avoid_watchouts }}
[if present]
divergent_seed: {{ divergent_seed }}
divergent_seed_watchouts: {{ divergent_seed_watchouts }}
[if present]
RETURN:
One alternative strategy object under the shared contract.

Figure 7 (PDF p. 24). Operator Block (Dual-reflection)
TASK: Keep one coherent route while strengthening its guardrails.
THINK: Identify the strongest realistic failure mode and the smallest guard-heavy repair. Do not solve the problem.
DUAL-REFLECTION RULES:
Keep plausible steps mostly stable and edits local.
Rewrite or strengthen watchouts as the main operation.
Provide three concrete watchouts.
TRAP SUPPORT is direct evidence for failure guards.
MECHANISM SUPPORT may add one clean forward mechanism.
Trap support alone is sufficient when it is diagnostic.
Do not invent a second competing route.
WATCHOUT PRIORITY: Cover invariant/state-reset safety, boundary/indexing, interface/contract safety, and complexity/contamination.
INPUT FIELDS (Jinja): problem: {{ problem }} target_failed_strategy: {{ target_failed_strategy }} target_watchouts: {{ target_failed_watchouts }} [if present] mechanism_support: {{ mechanism_support }} [if present] mechanism_watchouts: {{ mechanism_support_watchouts }} [if present] trap_support: {{ trap_support }} [if present] trap_watchouts: {{ trap_support_watchouts }} [if present]
RETURN: One reflected strategy object under the shared contract.

Figure 8 (PDF p. 24). Operator Block (Crossover)
TASK: Combine one stable backbone with one small improvement.
THINK: Identify the backbone and one compatible motif to borrow. Do not solve the target problem.
CROSSOVER RULES:
Build around BACKBONE PARENT as the main route.
Use CONTRAST PARENT for at most one local idiom, maintenance trick, or guard motif.
Never replace the backbone through freeform union.
Do not splice incompatible states, data structures, invariants, or reconstruction contracts.
If compatibility is unclear, keep the backbone stable.
Regenerate watchouts for the merged child.
APPS PRIORITY: Prefer contributions improving contract stability, boundary/indexing safety, state maintenance, or complexity rather than novelty alone.
INPUT FIELDS (Jinja): problem: {{ problem }} backbone_parent: {{ backbone_parent }} backbone_watchouts: {{ backbone_parent_watchouts }} [if present] contrast_parent: {{ contrast_parent }} contrast_watchouts: {{ contrast_parent_watchouts }} [if present]
RETURN: One crossover strategy object under the shared contract.
References
- [1]Akari Asai, Zeqiu Wu, Yizhong Wang, Avivup Sil, and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In International Conference on Learning Representations.arxiv.org/abs/2310.11511
- [2]Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732.arxiv.org/abs/2108.07732
- [3]Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 2206–2240. PMLR.arxiv.org/abs/2112.04426
- [4]Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems.arxiv.org/abs/2005.14165
- [5]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.arxiv.org/abs/2107.03374
- [6]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Łukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.arxiv.org/abs/2110.14168
- [7]Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.arxiv.org/abs/2312.10997
- [8]Yifu Guo, Jiaye Lin, Huacan Wang, Yuzhen Han, Sen Hu, Ziyi Ni, Licheng Wang, and Mingguang Chen. 2025. SE-Agent: Self-evolution trajectory optimization in multi-step reasoning with LLM-based agents. In Advances in Neural Information Processing Systems, volume 38.arxiv.org/abs/2508.02085
- [9]Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. REALM: Retrieval augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 3929–3938. PMLR.arxiv.org/abs/2002.08909
- [10]Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. 2024. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics.
- [11]Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021a. Measuring coding challenge competence with APPS. In Advances in Neural Information Processing Systems Datasets and Benchmarks Track.
- [13]Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Jon Stoica. 2025. LiveCodeBench: Holistic and contamination free evaluation of large language models for code. In International Conference on Learning Representations.arxiv.org/abs/2403.07974
- [14]Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.arxiv.org/abs/2004.04906
- [15]Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems.arxiv.org/abs/2205.11916
- [16]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems.arxiv.org/abs/2005.11401
- [17]Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. 2022. Solving quantitative reasoning problems with language models. In Advances in Neural Information Processing Systems.
- [18]Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with AlphaCode. Science.DOI
- [19]Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s verify step by step. In International Conference on Learning Representations.arxiv.org/abs/2305.20050
- [20]Mathematical Association of America. 2023. 2023 AMC 10/12. https://maa.org/student-programs/amc/. Accessed 2026-05-26.
- [21]Mathematical Association of America. 2024. 2024 american invitational mathematics examination (AIME). https://maa.org/maa-invitational-competitions/. Accessed 2026-05-26.
- [22]Mathematical Association of America. 2025. 2025 american invitational mathematics examination (AIME). https://maa.org/maa-invitational-competitions/. Accessed 2026-05-26.
- [23]Qirui Mi, Zhijian Ma, Mengyue Yang, Haoxuan Li, Yisen Wang, Haifeng Zhang, and Jun Wang. 2026. Skill-Pro: Learning reusable skills from experience via non-parametric PPO for LLM agents. arXiv preprint arXiv:2602.01869. Accepted at ICML 2026 as a spotlight.
- [24]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems.arxiv.org/abs/2203.02155
- [25]Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rafouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. 2026. ReasoningBank: Scaling agent self-evolving with reasoning memory. In International Conference on Learning Representations.arxiv.org/abs/2509.25140
- [26]Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2023. MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560.arxiv.org/abs/2310.08560
- [27]Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology.arxiv.org/abs/2304.03442
- [28]Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. In Advances in Neural Information Processing Systems.arxiv.org/abs/2302.04761
- [29]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.arxiv.org/abs/1707.06347
- [30]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.arxiv.org/abs/2402.03300
- [31]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 8634–8652. Curran Associates, Inc.arxiv.org/abs/2303.11366
- [32]Yash Vishe, Rohan Surana, Xunyi Jiang, Zihan Huang, Xintong Li, Nikki Lijing Kuang, Tong Yu, Ryan A. Rossi, Jingbo Shang, Julian McAuley, and Junda Wu. 2026. Skill-R1: Agent skill evolution via reinforcement learning. arXiv preprint arXiv:2605.09359.arxiv.org/abs/2605.09359
- [33]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2024. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research.arxiv.org/abs/2305.16291
- [34]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations.arxiv.org/abs/2203.11171
- [35]Zefeng Wang, Minxi Yan, Jinhe Bi, Sikuan Yan, Volker Tresp, and Yunpu Ma. 2026. MetaSkill-Evolve: Recursive self-improvement of LLM agents via two-timescale meta-skill evolution. arXiv preprint arXiv:2607.05297.arxiv.org/abs/2607.05297
- [36]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.arxiv.org/abs/2201.11903
- [37]Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, Zeyu Zheng, Chingang Xie, and Huaxiu Yao. 2026. SkillRL: Evolving agents via recursive skill-augmented reinforcement learning. arXiv preprint arXiv:2602.08234.arxiv.org/abs/2602.08234
- [38]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations.arxiv.org/abs/2210.03629
- [39]Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, and 17 others. 2025. DAPO: An open-source LLM reinforcement learning system at scale. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.arxiv.org/abs/2503.14476
- [40]Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. ExpeL: LLM agents are experiential learners. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):19632–19642.arxiv.org/abs/2308.10144



