Selm: Self-Evolving Library Management for Multi-Skill Agent Optimization

Introduction

Large language model (LLM) agents are increasingly used to solve complex tasks across different domains. To improve agents’ behaviors on specific tasks, recent work introduced skills [3], which are pieces of text that can be plugged into the agent context to provide task-specific knowledge, procedures, or experience. This gives agents a simple way to gain specialized knowledge without additional model training.

Recent work has explored how skills can evolve as the agent runs. Early skills were mostly static: they were written by experts or saved from successful experiences and then kept unchanged [3] (Zhao et al. 2024). More recent methods allow agents to update skills from their execution trajectories [2, 11, 14, 12] (Yang et al. 2026a; Zhang et al. 2026a; Zheng et al. 2025; Zhou et al. 2026; Zhang et al. 2026b). After running tasks, the agent can examine which skill worked or failed, then edit existing skills or create new ones. In this way, skills can change with the agent’s experience and adapt to new tasks and environments.

However, although these evolution methods propose many strategies to evolve skills [2] (Zhang et al. 2026a; Zheng et al. 2025; Zhou et al. 2026; Zhang et al. 2026b), few of them focus on how to manage them. A skill revision that best fits the current round can be harmful to later processes. For instance, a skill that is unused or unhelpful for the present task may be overwritten or discarded; once that change is committed, a later task can never recover the deleted skill. Meanwhile, failed or redundant skills that have remained in the system may still be loaded, giving the agent wrong instructions. The core problem is that existing methods always treat skill updates as separate, one-round edits, rather than managing skills as a dynamic, long-term library. Thus, a skill management library is needed to decide not only how a skill should change, but also which skills should be retrieved, which skills should be updated, and which skills should remain in the library.

To address these limitations, we propose SELM (Self-Evolving Library Management), a framework that manages skills as a library and jointly controls their retrieval, updates and lifecycle. By tracking and leveraging skill confidence and utility, SELM enables the library to evolve with task experience and improves the performance of the agent across complex tasks in different domains.

Specifically, SELM introduces a skill library with explicit mechanisms for managing skill states. To dynamically evaluate skill usefulness, each skill is associated with a confidence score and a utility score. Confidence reflects how strongly the skill is supported by update operations, while utility reflects how useful it has been across tasks. These mechanisms provide a clear basis for deciding which skills should stay active, which skills should be removed, and which failed skills should not be added again. The library can thus keep a skill that is unused on the current task but still useful later, and drop one that keeps failing, instead of committing every local edit, enabling reliable, long-term skill management.

Second, SELM designs a Map-Reduce framework for skill updates. Applying every trajectory to all the skills will make experience from different types of task overwrites each other. SELM therefore groups execution trajectories by task category(Map) and analyzes each group separately, identifying which skills are affected and what changes are needed. It then combines these analyses into operations(Reduce), such as adding new skills, editing existing ones, and deleting those that no longer help. All these operations apply only to the relevant skills and unrelated skills are left unchanged, so a change that helps the current category won’t erase skills required later, which enhances skill effectiveness.

Third, SELM proposes utility-aware skill retrieval. Existing methods mainly retrieve skills based on their semantic relevance to the current task (Yang et al., 2026b), which overlooks whether the skill has been useful in previous tasks. SELM leverages historical utility to enhance skill retrieval. It first selects skills from the same task category and then ranks them by their relevance to the current task and their past usefulness. This helps the agent distinguish skills that are relevant to task on hand and have proven useful through previous execution experience, thus enabling more effective skill selection.

Finally, we evaluate SELM on four agent benchmarks covering diverse execution environments and areas. SELM achieves state-of-the-art results among skill-evolution methods, outperforming both single-document optimization and prior multi-skill baselines without updating model parameters.

Our contributions are (1) introducing a skill library that tracks skill confidence and utility, managing skill validation, retention, and removal, enabling reliable skill management throughout the evolution process; (2) designing a Map-Reduce framework that analyzes trajectories by task category and generates targeted operations, allowing the agent to update relevant skills based on task-specific experience; and (3) developing a utility-aware retrieval method that ranks skills using both task relevance and historical utility, enabling the agent to select skills that are both relevant and useful. Across our experiments, SELM outperforms existing skill optimization methods, demonstrating the effectiveness of the proposed framework.

Related Work

Skill construction and evolution. Recent work has increasingly explored constructing and evolving textual skills from agent experience, moving beyond manually specified or static skill representations [3] (Zhao et al. 2024). More broadly, approaches such as TextGrad (Yuksekgonul et al., 2025) and GEPA [1] optimize textual instructions through feedback without updating model parameters, while Dynamic Cheatsheet [18] and ACE (Zhang et al., 2026c) iteratively refine textual context from agent experience. Building on this paradigm, SkillOpt and SkillGrad optimize a shared skill document or skill package by repeatedly editing textual knowledge based on execution feedback [20] (Yang et al. 2026a). Other approaches distill execution trajectories into standalone lessons, strategic principles, or verified skills, thereby converting individual experiences into reusable knowledge [21, 14, 12, 17] (Zhao et al. 2024). Rather than maintaining a single shared artifact, SkillWeaver (Zheng et al., 2025), EvoSkill [2], SkillEvolver (Zhang et al., 2026a), Memento-Skills (Zhou et al., 2026), and CoEvoSkills (Zhang et al., 2026b) construct collections of specialized skills to capture different task types or capabilities. Complementary approaches construct skills from external domain resources or learn skill-related policies through interaction and co-training [11, 22, 8] (Yang et al. 2026b). Despite these advances, existing methods primarily focus on how skills are created or improved, treating skill evolution as a sequence of local construction or editing operations. This becomes limiting when multiple skills coexist: updates to a shared artifact can introduce conflicting changes, while independently accumulated skills provide limited mechanisms for determining which skills should be updated.

Skill library management. Early work such as Voyager [19] and ExpeL (Zhao et al., 2024) retrieves executable or experiential knowledge according to the relevance of the current task. Subsequent systems make skill selection more behavior-aware and introduce mechanisms for validating candidate skills or deciding whether updates should be merged, accepted, or discarded (Zhou et al., 2026; Yang et al., 2026a,b; Alzubi et al., 2026; Zhang et al., 2026b; Qiu et al., 2026). More recent work explicitly considers the maintenance of the skill library itself. SkillOps studies how an existing skill collection can be periodically reorganized to address issues that emerge as the library grows [16], while MUSE-Autokill organizes skill creation, catalog retrieval, per-skill memory, and test-driven refinement into successive lifecycle stages [10]. Lifelong evaluations further suggest that usage frequency alone is insufficient to characterize the utility of a skill, motivating more explicit utility-aware management (Zhang et al., 2026d). However, these approaches generally address retrieval, validation, update selection, or lifecycle maintenance as separate mechanisms, rather than jointly optimizing them as coupled decisions over a continually evolving skill library.

Preliminary

This section introduces the basic concepts of skills, skill retrieval, and skill evolution.

Skill. The $k$-th skill is represented as

where $n(s)$, $d(s)$, $c(s)$, $t(s)$, and $\gamma(s)$ denote the skill name, description, instruction content, trigger conditions, and task category, respectively.

As the number of skills grows, they form a skill library, which is a collection of reusable skills that the agent can draw on for different tasks. Formally,

Skill retrieval. The agent does not need to use the entire library for every task. Given a task $x$, a retrieval mechanism selects a subset of skills from the library to provide relevant instructions for the task. We denote the retrieved skill set as

The selected skills are then provided to the agent as part of its context before task execution.

Evolving skills. With frozen target model $M$ and current library, the agent runs on task $x$, and the resulting experience is written back so that the same library can improve over iterations. $r(\mathcal{L}, x)$ represents the score of library $\mathcal{L}$ on task $x$. Given training, selection, and test splits $D_{\mathrm{tr}}$, $D_{\mathrm{sel}}$, and $D_{\mathrm{test}}$, evolution proposes libraries from $D_{\mathrm{tr}}$ and keeps the one that scores best on $D_{\mathrm{sel}}$. Formally,

where $\mathcal{C}(D_{\mathrm{tr}})$ is the set of libraries obtained by evolving on the training split. The score $r(\mathcal{L}, x)$ and the operators that map trajectories to library edits are instantiated in § 4.

CRUD operations. CRUD is an operational concept in databases for managing persistent records. It stands for Create, Read, Update, and Delete. We migrate this concept to skill management. In this paper, the library operations are ADD, EDIT, UPVOTE, DOWNVOTE, and DELETE: ADD creates a new skill, EDIT revises an existing skill, UPVOTE and DOWNVOTE adjust its confidence, and DELETE removes it from the active library.

Method

SELM freezes the target model $M$ and optimizes only the skill library. At each iteration, the agent retrieves a small set of skills from the current library, uses them to solve a batch of training tasks, and converts the resulting trajectories into library updates. The updated library is then evaluated on the selection set and retained only when it passes the validation gate. This iterative process jointly improves skill retrieval, skill updates, and library quality, with the overall procedure summarized in Algorithm 1 (Appendix § 7).

For each skill, SELM manages it through two pieces of information: confidence score and utility score. The confidence score $v(s)$ reflects how strongly a skill should be retained based on update operations, while the utility score $u(s)$ records how useful the skill has been across tasks. These two scores support different stages of the library evolution process: confidence is updated by UPVOTE/DOWNVOTE operations and used for lifecycle control, while utility is used to guide skill retrieval and updated from execution outcomes. Their corresponding update mechanisms are described in § 4.1 and § 4.2, respectively.

Utility-Aware Skill Retrieval

SELM retrieves skills by jointly considering task category, task relevance, and historical utility. For each task $x_i$, let $\gamma_i$ denote its task category. SELM first constructs a candidate set from the active skill set $S_t$, retaining skills whose categories match $\gamma_i$ or are marked as “general”. It then ranks the candidates using a utility-aware retrieval score and selects the top-$k$ skills for injection. Formally,

where $R_{\mathrm{rel}}(s, x_i)$ measures the relevance between skill $s$ and task $x_i$ using semantic and keyword-based matching, and $u(s)$ is the historical utility of the skill. The coefficient $w_u$ controls the contribution of historical utility.

The top-$k$ skills are selected for task $x_i$:

The selected skills $S_t(x_i)$ are injected into the agent context before execution. This retrieval strategy favors skills that are both relevant to the current task and useful based on previous execution results.

Utility Feedback. The utility score measures how useful a skill has been on the tasks that used it. Let $\mathcal{T}(s)$ denote the set of tasks whose retrieved skill set contains $s$. The average reward of $s$ in the current update is

We update the utility using an exponential moving average (EMA)

where $\alpha$ controls the contribution of new observations. The same utility score is then used by the retrieval mechanism in the next iteration. Thus, utility provides a feedback signal across successive retrieval rounds: execution outcomes update the utility of the skills that were used, and the resulting utility is subsequently incorporated into retrieval, reducing the selection of ineffective skills and improving agent performance across tasks.

Skill Library Evolution

After retrieving skills for each task, SELM runs the frozen target model and uses the resulting trajectories to update the skill library. The update process consists of four steps: rollout, trajectory analysis, CRUD synthesis, and candidate construction.

Rollout and Reward. For each training task $x_i$, the selected skills $S_t(x_i)$ are injected into the agent context. Given the frozen target model $M$ and execution harness $h$, the rollout produces a trajectory and task scores:

where $r_{\mathrm{hard},i} \in\{0,1\}$ indicates whether the task $x_i$ is fully solved, and $r_{\mathrm{soft},i} \in[0,1]$ measures partial progress in two ways: When the task $x_i$ is judged by multiple checks, $r_{\mathrm{soft},i}=n_{\mathrm{pass},i}/n_{\mathrm{total},i,}$ and $r_{\mathrm{hard},i}=1$ only when every check passes. When a task is judged by comparing a prediction with a reference answer, $r_{\mathrm{soft},i}$ is their token-level overlap and $r_{\mathrm{hard},i}$ is exact match.

The reward used for library optimization is obtained from these task scores:

where

Each rollout therefore yields two signals for library evolution. The trajectory $\tau_i$ is the execution evidence, including the agent’s actions, tool calls, intermediate outputs, and errors. The scalar reward $r_i$ shows how much of the task was solved. Later updates read $\tau_i$ to decide what to change, and use $r_i$ to decide which trajectories are worth changing from.

Trajectory Selection. SELM prioritizes trajectories whose observed rewards differ substantially from the historical utility of the retrieved skills. For task $x_i$, we compute the average utility of its retrieved skills:

and define the diagnostic priority as

SELM selects the highest-priority trajectories for further analysis. Trajectories with large $p_i$ provide stronger evidence that the retrieved skills may need to be updated. $\bar{u}(S_t(x_i))$ is the library’s current belief about those skills, and a large gap means the observed reward may contradict that belief. A much higher $r_i$ shows a successful pattern that the stored utility under-credits, and a much lower $r_i$ shows that the retrieved guidance failed on this task. A small gap means the rollout agrees with the existing record, so it adds little new information about what to change.

Map—Category-wise Analysis. The selected trajectories are partitioned by task category and analyzed independently. For each category, a TrajectoryAnalyst identifies successful patterns, failure patterns, and new insights from the corresponding trajectories. The analysis prevents experience from unrelated task categories from being directly mixed during skill updates.

Reduce—Skill update. Given the category-wise trajectory analyses and the current skill library, a SkillSynthesizer generates a bounded set of CRUD operations. The synthesizer considers the current skill content, confidence, utility, and category-specific execution feedback when deciding how to update the library.

Given a set of trajectories, SELM updates the library through the CRUD operations introduced in § 3:

For an existing skill $s$, the confidence score is updated as

The generated operations are applied to the relevant skills:

When a category exhibits recurring failure patterns, the synthesizer can also create new skills that describe strategies for avoiding these failures. The resulting candidate library $\mathcal{L}'_t$ is then passed to the validation stage.

Multi-timescale Updates. Following prior work on iterative skill evolution (Yang et al. (2026a)), SELM uses both fast and slow updates. Fast updates are performed after each training batch to incorporate recent trajectory feedback, while slow updates are performed less frequently across epochs to consolidate changes that persist over time. We also allow slow updates to produce meta skills, which summarize persistent cross-skill patterns identified across an epoch. This allows the library to respond to recent task feedback while reducing unnecessary updates caused by short-term fluctuations.

Skill Lifecycle Management

SELM treats skill maintenance as an explicit lifecycle process, using validation, utility tracking, and retirement mechanisms to prevent ineffective skills from accumulating or re-entering the library. Throughout optimization, the active set $S_t$ of $L_t$ and the graveyard $G_t$ obey three structural invariants:

Capacity bounds the store; uniqueness forbids duplicate names in $S_t$; graveyard blocking forbids reintroducing a retired name.

Validation. A candidate update is not directly added to the active library. SELM first evaluates the candidate library $L'_t$ on the selection set $D_{\mathrm{sel}}$ using the frozen target model and the same execution harness. Its validation score is

The candidate library is retained only when it improves the validation performance over the current library. This validation gate prevents updates that appear reasonable from the trajectory analysis but reduce the actual task performance from being incorporated into the library.

Pruning and Graveyard. SELM sets a threshold $\tau_u$ for utility score. For skill $s$, if $u(s) < \tau_u$ or $v(s) < 0$, it will be retired from the active set $S_t$ and moved to the graveyard $G_t$. Retired skills are excluded from future retrieval and cannot simply be reintroduced under the same identity. This prevents ineffective skills from repeatedly entering and leaving the library and keeps the active library compact.

Experiments

We evaluate SELM as a library-level optimizer for frozen agents. The experiments answer three questions. (i) Does a managed skill library improve over no skill, static skills, prompt evolution, single-document skill optimization, and recent multi-skill methods? (ii) Does that comparison hold across different models? (iii) Which library-management controls are required for the gains?

Experimental Setup

Baselines. We compare against No skill; Human skill (an expert-written document per benchmark); LLM skill (a one-shot skill from Claude-Opus-4.6 that is never updated); prompt-evolution GEPA [1]; skill-construction methods SkillEvolver (Zhang et al., 2026a), Memento-Skills (Zhou et al., 2026), and Trace2Skill [14]; single-document SkillOpt (Yang et al., 2026a); and, under Claude Code, folder-based EvoSkill [2]. EvoSkill is omitted from the no-harness block because it assumes a SKILL.md package layout. Qwen3-30B-A3B is evaluated only without a harness.

Benchmarks. We use four datasets: SkillsBench [9], SpreadsheetBench [13], OfficeQA [15], and LiveMath [7]. Detailed descriptions are in Appendix § 8.

Target models. We use Claude-Opus-4.6 [5], DeepSeek-V4-Pro [6], and Qwen3-30B-A3B [23] as frozen target models. Claude and DeepSeek are evaluated both without a harness and inside Claude Code [3]; Qwen3-30B-A3B is evaluated only without a harness. No-harness runs prepend retrieved skills to the system prompt. Claude Code loads skills through the native agent workspace. All methods in a given cell use the same target model, harness, and scorer.

Hyperparameters. Hyperparameters are reported in Appendix § 9.

Main Results

Table 1 shows that SELM consistently improves agent performance across different benchmarks and target models. Compared with no-skill baseline and existing skill optimization methods, SELM achieves substantial gains in both no-harness and Claude Code settings, demonstrating the effectiveness of managing skills as a structured library rather than optimizing a single shared document. The improvements are also consistent across Claude-Opus-4.6, DeepSeek-V4-Pro, and Qwen3-30B-A3B, suggesting that the benefit of library-level optimization is not tied to a specific target model. Meanwhile, Static human-written skills and one-shot LLM skills provide only limited improvements, while trajectory-driven methods perform better but remain below SELM. This indicates that the main benefit comes from continuously managing which skills should be retrieved, updated, and retired. In particular, the advantage is more pronounced in heterogeneous settings such as SkillsBench, where task-specific feedback can conflict when compressed into a single skill document.

Table 1: Main results on SkillsBench, OfficeQA, SpreadsheetBench, and LiveMath. SkillsBench reports soft/hard accuracy (%); other rows are accuracy (%). The best cell in each row within a model block is bold and shaded purple; the second-best cell is shaded a lighter purple. Note: Qwen3-30B-A3B failed all the tasks on SkillsBench, so the Hard Acc is not included.

Benchmark		No skill	Human skill	LLM skill	GEPA	SkillEvolver	Memento-Skills	SkillOpt	Trace2Skill	SELM
Claude-Opus-4.6
SkillsBench	Soft Acc	40.4	45.3	44.8	56.4	59.3	66.1	62.7	52.4	69.8
	Hard Acc	31.5	36.3	34.5	41.8	43.6	48.2	46.0	40.2	50.7
OfficeQA	Acc	45.3	49.7	47.5	53.1	54.6	59.5	56.4	51.2	62.8
SpreadsheetBench	Acc	44.3	46.1	45.8	49.6	52.5	54.3	51.3	48.3	56.8
LiveMath	Acc	35.2	36.8	37.4	41.5	50.8	56.0	56.9	46.5	61.3
DeepSeek-V4-Pro
SkillsBench	Soft Acc	21.1	30.3	25.7	36.9	39.5	44.2	43.9	34.3	45.5
	Hard Acc	13.1	19.9	16.5	26.5	28.4	32.7	31.8	25.0	34.2
OfficeQA	Acc	33.2	36.6	35.4	41.2	42.5	46.8	44.3	40.7	48.9
SpreadsheetBench	Acc	32.9	33.6	34.9	37.8	38.6	41.2	39.7	38.1	42.4
LiveMath	Acc	27.4	28.6	28.1	31.2	34.8	37.5	36.2	32.0	39.4
Qwen3-30B-A3B
SkillsBench	Soft Acc	16.4	17.5	16.9	17.8	18.5	19.8	19.2	17.9	20.5
OfficeQA	Acc	25.8	27.6	26.1	28.0	28.9	30.8	28.6	30.2	32.5
SpreadsheetBench	Acc	30.0	30.0	30.0	30.6	31.0	33.8	31.5	30.8	36.4
LiveMath	Acc	23.6	24.1	23.9	25.8	27.5	30.2	28.4	26.2	33.1
Claude Code harness
Benchmark		No skill	Human skill	LLM skill	GEPA	SkillEvolver	Memento-Skills	EvoSkill	SkillOpt	SELM
Claude-Opus-4.6
SkillsBench	Soft Acc	51.4	61.1	59.2	72.5	76.8	81.2	79.5	80.2	82.8
	Hard Acc	40.0	46.6	43.9	52.9	56.5	59.8	59.0	58.3	60.9
OfficeQA	Acc	57.6	61.2	60.4	66.5	69.8	74.8	75.4	71.5	73.1
SpreadsheetBench	Acc	19.7	34.7	33.2	55.3	72.6	78.7	66.7	80.0	81.5
LiveMath	Acc	38.5	42.0	42.8	46.0	51.5	54.4	50.2	53.7	57.8
DeepSeek-V4-Pro
SkillsBench	Soft Acc	28.8	34.0	32.6	42.6	46.3	50.1	49.0	50.4	51.6
	Hard Acc	19.4	22.1	20.5	27.8	29.5	33.3	30.9	33.8	35.1
OfficeQA	Acc	38.9	43.1	41.5	44.0	46.2	48.8	48.3	45.9	50.0
SpreadsheetBench	Acc	24.7	30.8	31.2	38.5	41.7	44.9	42.8	46.0	46.9
LiveMath	Acc	29.5	32.0	31.5	35.2	38.6	41.5	39.0	40.8	39.4

Ablation Study

We ablate three aspects of SELM: library-management components, utility estimation, and optimizer–target model interaction. We first remove individual management mechanisms, including validation, utility-aware retrieval, meta skills, slow updates, and the graveyard. We then vary the EMA decay factor $\alpha$ to examine the sensitivity of utility estimation. Finally, we vary the optimizer and target models to study how optimizer capability affects the resulting skill library. Unless otherwise specified, the experiments follow the default configuration.

Library-management controls are critical to the effectiveness of skill evolution. Table 2 shows that removing the validation gate causes the largest single drop, while removing the utility score and utility-aware retrieval reduces only 2.9 points. Removing both meta skills and slow updates also leads to a large degradation, compared with 53.1% when removing meta skills alone. The graveyard provides a smaller but consistent gain, with accuracy dropping to 50.4% without it. These results show that validation, utility-aware selection, and longitudinal library updates are important for preventing ineffective skill changes from accumulating.

Table 2: Component ablation on SpreadsheetBench (Claude-Opus-4.6, no harness). w/o: without.

Setting	Acc (%)
Full (default)	56.5
w/o meta skill	53.1
w/o meta skill + slow update	42.7
w/o utility score	46.3
w/o utility-aware retrieval ($w_u=0$)	49.2
w/o validation gate	42.5
w/o graveyard	50.4

SELM is relatively insensitive to the EMA decay factor around the default setting. As shown in Figure 3(a), $\alpha=0.4$ achieves 56.5% accuracy, while $\alpha=0.2$ and 0.6 obtain 55.1% and 55.3%, respectively. More extreme values, $\alpha=0.1$ and 0.8, reduce accuracy to 52.4% and 53.1%. This indicates that the performance gain is not dependent on a narrowly tuned smoothing coefficient.

A stronger optimizer generally produces a more effective skill library, while weaker optimizers can still benefit stronger target models. Figure 3(b) shows that, with Qwen3-30B-A3B as the target, replacing the optimizer with DeepSeek-V4-Pro and Claude-Opus-4.6 increases soft accuracy from 20.45% to 24.75% and 31.34%, respectively. The same trend holds for DeepSeek-V4-Pro as the target. Meanwhile, using Qwen3-30B-A3B as the optimizer still improves Claude-Opus-4.6 from the no-skill baseline.

Library Evolution Analysis

Table 3 shows how the skill library changes during evolution on SpreadsheetBench with Qwen3-30B-A3B. The initial library contains 5 skills and achieves 17.5% accuracy on the selection set. In Epoch 1, five new skills are added, but the accuracy remains unchanged. This shows that simply adding more skills does not improve performance. In Epoch 2, only two new skills are added while four skills are downvoted, and the accuracy increases to 30.0%. The slow update then makes two edits, adds two meta skills, and downvotes two more skills. Three skills are eventually moved to the graveyard because their confidence becomes non-positive, and they are therefore excluded from future retrieval. The final library achieves 36.43% accuracy, compared with 30.0% for the no-skill baseline. These changes show that the library does not improve simply by growing in size; useful skills are added, while skills that repeatedly receive negative feedback are removed or revised.

Table 3: Skill library evolution on SpreadsheetBench (Qwen3-30B-A3B, no harness). Init and epoch accuracies are on $D_{\mathrm{sel}}$; Final is held-out test.

Epoch	Active	ADD	EDIT	DOWNVOTE	Graveyard	Acc. (%)
Init	5	–	–	–	0	17.5
1	10	5	0	3	0	17.5
2	11	2	0	4	–	30.0
Slow update	11	2	2	2	3	–
Final	11	9 (total)	2 (total)	9 (total)	3	36.43

Conclusion

We propose SELM, a framework that manages skills as a library. SELM tracks skill confidence and utility, validates skill updates, and retires failed skills. Through the Map-Reduce framework and utility-aware retrieval, SELM helps the agent select the most relevant and effective skills. Various experiments validate that SELM consistently improves frozen agents over prior skill-optimization methods and achieves the strongest performance in most settings.

Iterative Library Evolution Algorithm

Algorithm 1 summarizes the SELM loop in §4: utility-aware retrieval and rollout, diagnostic trajectory selection, Map–Reduce CRUD synthesis, validation gating, pruning to the graveyard, and slow updates.

Benchmark Details

We evaluate on four agent benchmarks covering heterogeneous domains, procedural tool use, long-context document reasoning, and mathematical problem solving.

SkillsBench [9]: a multi-domain benchmark with 8 task categories (cybersecurity, data analysis, finance-economics, hardware engineering, natural science, social science, software engineering, and one general category). SkillsBench reports both hard accuracy and soft accuracy, corresponding to $r_{\mathrm{hard}}$ and $r_{\mathrm{soft}}$ in § 4.1.

SpreadsheetBench [13]: spreadsheet manipulation requiring multi-step formula generation and cell-level operations.

OfficeQA [15]: a grounded reasoning benchmark built from U.S. Treasury Bulletins with a corpus of approximately 89,000 pages.

LiveMath [7]: LiveMathematicianBench(LiveMath), a live mathematician-level reasoning benchmark, abbreviated LiveMath in tables.

The last three benchmarks report a single accuracy.

Hyperparameter Summary

Hyperparameter	Description	Default
$M_{\max}$	Maximum library capacity	30
$w_u$	Utility weight in retrieval scoring	0.6
$w$	soft score weight in reward calculating	0.5
$\alpha$	EMA decay factor for utility update	0.4
$\tau_{\mathrm{soft}}$	Soft threshold for success classification	0.3
$\lvert G\rvert_{\max}$	Maximum graveyard capacity	50
$B_{\mathrm{ADD}}$	Max ADD operations per evolution step	5
$B_{\mathrm{EDIT}}$	Max EDIT operations per evolution step	3
$K$	Number of skills retrieved per task	3
$B$	Trajectory selection budget	50
$N_{\mathrm{slow}}$	Slow update sample size	20
$\tau_u$	threshold for utility score	0.05
Injection budget	Max characters for skill injection	2000
Fail compression	Max chars for failed trajectories	2000
Success compression	Max chars for successful trajectories	400
Map parallelism	Max parallel TrajectoryAnalyst workers	16

Skill Library Example

Figures 4–7 reproduce four skills from the optimized SpreadsheetBench library (Qwen3-30B-A3B, no harness; 11 active skills, test accuracy 36.43%), one from each remaining category. Text is the full library snapshot, not a truncation.

Skill: dynamic_date_handling

Category. cell_level Confidence. 2 Utility. 0.44 Description. Handle dynamic date ranges and exclude weekends/holidays in formulas. Trigger conditions. When tasks involve date-range calculations with dynamic inputs or require exclusion of non-business days. Content.

1. Use DATE functions to define start and end dates.

2. Use NETWORKDAYS or custom logic to exclude weekends.

3. Include holiday lists if provided.

4. Use IF statements to validate date ranges.

5. Test with sample data to ensure accuracy.

Skill: inter_sheet_data_matching

Category. sheet_level Confidence. 2 Utility. 0.44 Description. Handle data matching and manipulation across multiple sheets. Trigger conditions. When tasks require data from multiple sheets to be compared, combined, or processed. Content.

1. Use VBA to reference data from other sheets.

2. Use loops and conditionals to match data.

3. Use error handling to prevent runtime issues.

4. Test with different data structures.

5. Use message boxes to provide feedback on results.

Skill: slow_update_epoch_02

Category. meta Confidence. 3 Utility. 0.70 Description. Epoch-level guidance from epoch 2 slow update. Trigger conditions. None; produced by the epoch-level slow update and managed by the same utility and lifecycle rules as ordinary skills. Content.

• ADD: meta_insight_epoch_2 — Identified patterns of persistent failures and regressions, highlighting the need for better handling of dynamic data, text parsing, and formula execution in VBA. Content: Focus on improving dynamic data handling, text parsing, and formula execution. Address issues with date handling, string matching, and VBA macro behavior. Prioritize skills that were downvoted or showed persistent failure.

• EDIT: dynamic_column_handling

• EDIT: advanced_text_parsing_in_vba

• DOWNVOTE: text_matching_strategies (Showed persistent failure in text comparison and filtering tasks, indicating a need for re-evaluation of its implementation and use cases.)

• DOWNVOTE: vba_data_preservation (Caused regression in data preservation tasks, suggesting potential issues with how it handles macro execution and data integrity.)

Limitations and Future Work

Semantic deduplication. The graveyard mechanism operates at the name level (exact string match), preventing exact-name oscillation but not semantically equivalent skills with different names from coexisting. Embedding-based similarity detection would be a natural extension.

Cross-environment transfer. A skill library optimized on one benchmark could potentially warm-start optimization on another, analogous to transfer learning in neural networks. We leave exploration of cross-benchmark transfer to future work.

Scalability. While we demonstrate effectiveness with up to 30 skills across 8 categories, scaling to hundreds of categories or thousands of skills would require more sophisticated indexing (e.g., embedding-based retrieval) and potentially hierarchical library organization.

Optimizer model dependence. The quality of evolution depends on the optimizer LLM’s ability to analyze trajectories and synthesize CRUD operations. Weaker optimizer models may produce lower-quality operations, and the interaction between optimizer capability and library quality deserves systematic study.

Algorithm 1 SELM — Iterative Library Evolution

Require: Frozen target model $M$, execution harness $h$, training split $D_{\mathrm{tr}}$, selection split $D_{\mathrm{sel}}$, epochs $E$, retrieval budget $K$, trajectory budget $B$, EMA coefficient $\alpha$, utility weight $w_u$, retirement threshold $\tau_u$, maximum capacity $M_{\max}$

1. $L_0 \leftarrow$ initial skill library ▷ empty or seed skills

2. $G_0 \leftarrow\emptyset$ ▷ graveyard starts empty

3. $L_{\mathrm{best}} \leftarrow L_0$ ▷ snapshot kept for export

4. $V_{\mathrm{best}} \leftarrow V(L_0)$ ▷ score on $D_{\mathrm{sel}}$

5. for $e = 1,\ldots,E$ do

6. for each training batch $B \subset D_{\mathrm{tr}}$ ▷ one fast update do

7. $S_t \leftarrow\{s \in L_t : n(s) \notin\{n(g) \mid g \in G_t\}\}$ ▷ block retired names

8. for each task $x_i \in B$ do

9. Let $\gamma_i$ be the category of $x_i$

10. $S_t(x_i) \leftarrow\operatorname{TopK}\{s \in S_t : \gamma(s) \in\{\gamma_i,\mathrm{general}\}\}, R(s,x_i), K$ ▷ utility-aware retrieval

11. $(\tau_i,r_{\mathrm{hard},i},r_{\mathrm{soft},i}) \leftarrow h(M,x_i,S_t(x_i))$ ▷ rollout

12. $r_i \leftarrow G(r_{\mathrm{hard},i},r_{\mathrm{soft},i})$ ▷ scalar reward

13. $p_i \leftarrow r_i-\bar{u}(S_t(x_i))$ ▷ gap from stored utility

14. end for

15. Select the top-$B$ trajectories according to $p_i$ ▷ most informative traces

16. for each skill $s$ used in the current batch do

17. $u(s) \leftarrow(1-\alpha)u(s)+\alpha\bar{r}_s$ ▷ EMA utility

18. end for

19. Partition selected trajectories by task category ▷ Map

20. for each category $c$ do

21. $a_c \leftarrow\operatorname{TrajectoryAnalyst}(\text{trajectories of }c,L_t)$ ▷ success, failure, insights

22. end for

23. $\operatorname{ops}_t \leftarrow\operatorname{SkillSynthesizer}(\{a_c\},L_t)$ ▷ Reduce; operation budgets

24. $L'_t \leftarrow\operatorname{Apply}(L_t,\operatorname{ops}_t,G_t)$ ▷ candidate library

25. Evaluate $V(L'_t)=\frac{1}{|D_{\mathrm{sel}}|}\sum_{x\in D_{\mathrm{sel}}}r(L'_t,x)$ ▷ validation gate

26. if $V(L'_t)>V(L_t)$ then

27. $L_t \leftarrow L'_t$ ▷ accept only if improved

28. end if

29. if $V(L_t)>V_{\mathrm{best}}$ then

30. $L_{\mathrm{best}}\leftarrow L_t$ ▷ keep the best snapshot

31. $V_{\mathrm{best}}\leftarrow V(L_t)$

32. end if

33. end for

34. if slow update is scheduled at epoch $e$ ▷ less frequent than fast updates then

35. Generate additional CRUD operations ▷ consolidate persistent trends

36. $L'_t \leftarrow\operatorname{Apply}(L_t,\operatorname{ops}^{\mathrm{slow}}_t,G_t)$

37. Validate $L'_t$ on $D_{\mathrm{sel}}$ ▷ same gate as the fast update

38. if $V(L'_t)>V(L_t)$ then

39. $L_t \leftarrow L'_t$

40. end if

41. if $V(L_t)>V_{\mathrm{best}}$ then

42. $L_{\mathrm{best}}\leftarrow L_t$

43. $V_{\mathrm{best}}\leftarrow V(L_t)$

44. end if

45. end if

46. Lifecycle pruning:

47. for each $s \in L_t$ do

48. if $u(s)<\tau_u$ or $v(s)\leq0$ ▷ low utility or non-positive confidence then

49. $L_t\leftarrow L_t\setminus\{s\}$

50. $G_t\leftarrow G_t\cup\{s\}$ ▷ retire; name cannot return

51. end if

52. end for

53. Enforce $|G_t|\leq|G|_{\max}$ ▷ graveyard capacity

54. Enforce $|S_t|\leq M_{\max}$, name uniqueness, and graveyard blocking

55. end for

56. return $L_{\mathrm{best}}$ ▷ not necessarily the last $L_t$