One Skill Takes Over: Clashing Agent Skills and a One-File Fix

Introduction

Agent Skills give coding agents reusable instructions for jobs such as writing commit messages, reviewing code or documenting an API [4]. A skill is a small folder with a SKILL.md file that holds a short description and free-form instructions, and the agent loads it when a matching task comes in [1]. In practice the instructions are a list of rules, often followed by a worked example. Skills come from many authors and are installed side by side, and nothing stops two skills for the same job from being active together. A developer may have a personal commit-message skill and a team one, or a vendor’s code-review skill and a house one. Some of their rules conflict, but most can be followed together. When both skills are available, does the model combine their compatible rules, or does it follow one skill at the expense of the other?

Yan et al. [22] measured this situation in practice. They installed pairs of public same-job skills in Claude Code and found that the second skill took about one run in five away from the first. The first skill’s own requirements were met less often, while the tasks were still completed. Their fix is a hook that blocks the competing skill. In their harness the agent opens skill files on demand, most of the loss comes from runs that never open the first skill, and only the first skill is scored. This leaves two questions open: what does a model do when both skill files are actually in its context, and can a fix keep the rules of both skills instead of blocking one?

Preprint.

Figure 1 shows what we find on one case. With both skills loaded, the model writes skill B’s format in full and ignores skill A, including the rules of A that conflict with nothing. We call this pattern takeover. With the two skills merged into one file, the same model follows the rules of both. Yan et al. [22] already report that one skill can take the place of another. We measure takeover with both files in the context and both skills scored, so the choice of which file to open plays no role.

Contributions.

1. A benchmark. SkillClash has 16 pairs of same-job skills. Code checks every rule, and for every pair a reference output shows that the best achievable result can be reached. A co-author who had not seen our labels found every contradictory and shared rule pair we planted (§ 3).

2. A diagnosis that holds across models. On seven models from five families, from 4B open models to GPT-4.1, a same-job skill cuts the share of a skill’s rules that are followed by about a third, while an unrelated skill changes nothing. In most outputs one skill’s own rules are all followed and the other’s are all dropped. The worked examples inside the skills drive most of this. When only one skill keeps its example, that skill wins most takeovers, and when each file ends with an example that follows both skills, takeover falls to 1 to 15%, on the small and the strong models alike (§ 4).

3. What brings the rules back. A priority instruction does not, on any model. An “apply both” instruction does on the two strong models, but the rules that interact stay broken. One merged file, a meta-skill, written by Claude Opus 5.5 works on all seven models. Its worked example, which follows both skills, accounts for most of its gain over a plain list of both skills’ rules, and it helps only when it is correct. A step-by-step merge that labels rule pairs is only as good as its labels, so who writes the merge matters more than how (§ 5).

4. A look inside. One direction in the activations of three small models flips which skill wins in up to half of the takeovers, but steering along it does not make the models follow both skills (§ 6).

Related work

Agent Skills and how they fail. Curated skills raise task pass rates, but tasks that use four or more skills gain less, and conflicting guidance has been suggested as a cause [13]. On real software engineering tasks, most public skills give little or no gain [10]. A single loaded skill can make an agent leave out or wrongly fill required parts of a task [8]. In a large skill library, most of the damage comes from picking a wrong skill or no skill [17]. The closest work is Yan et al. [22], which studies conflicts between co-installed same-job skills and measures, requirement by requirement, what the installed skill loses. Our study differs in three ways: both skill files are in the context, both skills are scored against a known best output, and the fixes we test keep both skills.

Conflicting instructions. The instruction hierarchy trains a model to follow the more privileged of two conflicting instructions [19], yet models often do not enforce a stated priority, even between simple formatting rules [9]. Our priority instruction fails in the same way. Patterns shown earlier in a conversation can override an explicit instruction [7], and in our setting the worked example inside each skill plays this role. These works study one instruction against another. We study two whole skill files of equal rank, where most of what is lost conflicts with nothing.

Merging and improving skills. SkillOps merges skills whose bodies are identical [14]. SkillSmith estimates from task scores whether two retrieved skills help or hurt each other, and its skills do different jobs [20]. SkillComposer rewrites two similar skills into one more general skill and judges the result by task pass rate [24]. Closest to our fix, SkillFusion labels pairs of constraints in co-loaded skills as hard conflicts, soft conflicts or redundancy, compiles the skills into one, and beats concatenation and a one-call rewrite on task success [11]. GEPA improves prompts by reflecting on failed runs and their feedback [2], and SkillOpt edits one skill document under a validation gate [23]. On repository skills, the gains of both could not be told apart from run-to-run noise [12]. These works score task success or the merged file itself. We score every output on the rules of both source skills against a known best output. This shows what each fix keeps and what it drops, and it shows what makes a merge work: correct labels and one correct example that follows both skills. Our feedback loop is a small loop of the GEPA kind, reported with intervals over jobs and with a control for how its practice requests are chosen.

SkillClash

Jobs and skills. A job is one kind of text, such as a commit message, a pull request description, a SQL query or a bug report. SkillClash has 16 jobs (Appendix § 8). Each job has two skills, A and B, with different house rules. Each skill is a SKILL.md file with six rules and one worked example that follows them. We constructed the skills for this benchmark rather than collecting them from public repositories, so that we control how their rules interact.

Four kinds of rules. As in IFEval [25], a small program checks every rule. Relative to the other skill, each rule is one of four kinds (Figure 1):

• contradictory: no output can satisfy it together with its partner in the other skill, for example two different required starts of the first line (one per skill);

• shared: the other skill asks for the same thing (one per skill);

• entangled: both rules can hold, but only if the output is arranged with care, for example two lines that each want to come last (one per skill);

• exclusive: nothing in the other skill touches it (three per skill).

What a correct output follows. Following both skills cannot mean following all twelve rules, because the two contradictory rules cannot both hold. Each pair is therefore run with a stated primary skill, whose side of the contradiction is the correct one. This is the only place where a skill should lose, and we run every pair twice, once with each skill as the primary. A correct output then follows ten distinct rules: all six rules of the primary, plus the three exclusive rules and the entangled rule of the other skill. The other skill’s shared rule asks for the same thing as the primary’s, so it is already counted. For every pair and each choice of primary, the benchmark includes a reference output, such as a complete commit message, and the same code confirms that it follows all ten rules. For the pair in Figure 1, the two reference outputs differ only in their first line. Takeover is different from this priority: it also drops the other skill’s exclusive rules, which conflict with nothing.

Checking the rule labels. Because we planted the rule relations, we asked whether they are the ones a careful reader sees. A co-author who did not write the skills and had not seen our labels read the 16 pairs and listed every pair of rules that interact, with a label. The rater found all 16 contradictory and all 16 shared pairs with the right label and listed no pair that we had not planted. Entangled pairs were harder to see. The rater labeled 8 of the 16 entangled pairs as entangled and 4 as contradictory, and did not list 4. In the pairs called contradictory, one rule asks for an exact position, such as the first line, and the other only for a nearby one, such as “at the top”; both can hold if the lines are placed with care. The pairs not listed are length and format limits. The models also find entangled pairs the hardest to satisfy (§ 5).

What we measure. Rules followed is the share of a skill’s six rules that an output follows. Takeover means that one skill’s three exclusive rules are all followed and the other skill’s are all dropped. Achievable rules followed is the share of the ten achievable rules that an output follows; an output that follows only the primary skill gets 60%. An output is fully correct when it follows all ten. The last measure is stricter: a fix can follow most achievable rules on average while few outputs follow all of them.

Setup. Both skill files are placed in the system prompt, which says that every rule of every loaded skill must be followed (Appendix § 8). This removes skill selection as a source of failure. We use eight requests per job and test seven models from five families with greedy decoding: five open models (Qwen3-4B, Qwen2.5-7B, Mistral-7B, Llama-3.1-8B and Phi-4-mini) and two strong models called through an API (Qwen3-235B and GPT-4.1). Qwen3-235B is the large model of the same release as Qwen3-4B. GPT-4.1 was run on four of the eight requests per job. The 16 jobs are the unit of analysis. All intervals are 95% bootstrap intervals over jobs, and all comparisons are paired by job.

One skill takes over

A same-job skill hurts every model, and an unrelated skill does not. Alone, a skill is followed almost perfectly (Figure 2a). Adding a skill for a different job changes nothing, so a longer context is not the cause. Adding a skill for the same job cuts the rules followed to 61 to 62% on all seven models. A model with sixty times more parameters and a strong closed model lose as much as the 4B model. One side of the contradiction has to lose, but that explains little: on the five rules per skill that can always hold, the drop is still 28 to 35 points. The drop is about the same when both files are given in the user message instead of the system prompt (32 to 37 points on Qwen3-4B, Qwen2.5-7B, Llama-3.1-8B, Phi-4-mini, Qwen3-235B and GPT-4.1).

The loss is mostly all or nothing. The outputs are rarely blends: in 45 to 78% of outputs, one skill’s exclusive rules are all followed and the other skill’s are all dropped (Figure 2b to Figure 2d). The exclusive rules conflict with nothing, and they are still dropped.

Worked examples drive most of the takeover. On the five open models and Qwen3-235B, takeover is 45 to 78% with the examples and 7 to 41% when we remove the examples from both skill files (Appendix § 10). Three more tests on Qwen3-4B, Qwen2.5-7B, Llama-3.1-8B, Phi-4-mini, Qwen3-235B and GPT-4.1 show that the examples, and not the two separate files, cause it (Figure 3). First, when both full skill files are placed in one file, examples included, takeover is still 34 to 73%. Second, when only one skill keeps its example, that skill wins 67 to 93% of the takeovers over both load orders, and moving the example to the other skill moves the winner with it. The one exception is Phi-4-mini, which follows the second-listed skill in almost every takeover. Each example shows an output that follows one skill only, and the model writes in that format. Third, the content of the example decides. When each of the two files ends with the same example, one that follows both skills, takeover falls to 1 to 15% and 90 to 96% of the achievable rules are followed, without merging the two files. On the two strong models, this one change raises the share of fully correct outputs from 1% to 76 and 60%, while the “apply both” instruction reaches only 22 and 21%.

Takeover does not vanish without examples. Even with no interacting rules left, four of the six models in Appendix § 10 (Qwen3-4B, Qwen2.5-7B, Phi-4-mini and Qwen3-235B) follow both skills’ exclusive rules in full 24 to 33 points less often than their single-skill runs predict, with intervals that exclude zero. So two skills for the same job can get in each other’s way even when none of their rules conflict.

The winner depends on the pair and the load order, not on the request. With greedy decoding, the same skill wins on almost every request for a fixed pair and load order (§ 6). Swapping the load order often changes the winner, and which position wins depends on the model. The first-listed skill wins 74 and 77% of the takeovers on Llama-3.1-8B and GPT-4.1, but only 24% on Qwen2.5-7B and 3% on Phi-4-mini. A user who does not control the load order cannot know which skill will be followed.

Takeover varies by job. Takeover is at least 75% in 5 to 12 of the 16 jobs, depending on the model. For SQL queries it is at most 19% on every model. There the two rule sets constrain different parts of one statement, and the models blend them (Appendix § 11).

From two files to one

We compare two families of fixes. The first keeps both skill files and adds an instruction. A priority instruction says that the primary wins if the skills conflict, and an “apply both” instruction says that the skills are complementary and that every rule of both must be followed. The second family replaces the two files with one file, which we call a meta-skill. We build it in three ways.

• Rule union. Code copies the rules of both skills into one file under two headings, without the examples, and adds a line that says which skill wins in a conflict, with no model involved.

• Feedback loop. A small reflective loop in the style of GEPA and SkillOpt [2, 23] improves a meta-skill. In each of six rounds, it runs the file on three practice requests, lists the rules that each output broke, asks the model to rewrite the file, and keeps the rewrite only if more rules are followed. Qwen3-4B both answers the practice requests and rewrites the file.

• Merge by a model. A model reads the two skill files and writes one merged skill. We tried a plain request to merge, and a step-by-step merge that lists the rules, labels the pairs that interact, writes one combined rule for each entangled pair, and copies the other rules word for word. Each of the seven models merges for itself. An eighth merger, Claude Opus 5.5, saw only the skill files, and its files are given to all seven models.

Figure 4 shows the results, and Appendix § 9 lists every condition with its interval.

A priority instruction does not bring the dropped skill back on any model. With it, the achievable rules followed stay at 62 to 71%, close to what following one skill alone gives, and one skill still takes over in 36 to 82% of outputs.

Asking for both skills works on the strong models, up to a point. With the “apply both” instruction, the open models reach 69 to 78%, and one skill still takes over in up to 46% of outputs. The strong models reach 88 to 89% with no takeover, so the dropped skill comes back. The rules that interact stay broken. Both entangled rules are followed in only 38 and 32% of outputs, the primary keeps its side of the contradiction in 59 and 73%, and only about one output in five is fully correct (22 and 21%).

A plain rule list in one file helps. The rule union reaches 66 to 79% on the open models and 86 to 90% on the strong ones, and takeover drops to 1 to 16%. It needs both of its changes (Figure 3a). With the examples kept, one file gains 0.6 to 9.6 points over two files, and one skill still takes over in 34 to 73% of outputs. Without the examples, one file gains 6 to 18 points over two files on the five open models and Qwen3-235B, with intervals above zero on all of them except Phi-4-mini (Appendix § 9). On Qwen3-4B, the feedback loop lifts the union further, to 92%, and Llama-3.1-8B reading the same files reaches 89%. The loop needs rule checks for its feedback, which for real skills means a judge model or hand-written checks.

A file merged by the strongest model is the only fix near the ceiling on every model. The meta-skills written by Claude Opus 5.5 reach 96 to 99% of the achievable rules on all seven models, with no takeover, and 68 to 94% of outputs are fully correct. That is 11 to 29 points above the “apply both” instruction, and on the strong models it gives four times as many fully correct outputs. A model that merges for itself does less well. Its plain merge reaches 71 to 95%, which is 5 to 25 points below Claude’s file. The quality of a meta-skill follows the strength of the model that merges, and this holds with the reader fixed: Qwen3-4B follows 79, 87, 94, 92 and 98% of the achievable rules when the plain merge was written by Qwen3-4B itself, Llama-3.1-8B, Qwen3-235B, GPT-4.1 and Claude Opus 5.5, and Llama-3.1-8B as the reader shows the same pattern (Appendix § 9).

The example in the merged file does most of the work. Each of Claude’s files ends with one worked example that follows both skills. On the four open models with this run, removing the example costs 9 to 25 points of achievable rules; on the two strong models it costs about 2. On all six models it lowers the share of fully correct outputs, by 14 to 52 points (Figure 3c). Adding the same example to the rule union written by code lifts the union by 9 to 24 points, to within 8 points of Claude’s full file on all six models. On GPT-4.1, it raises the fully correct outputs of the rule union from 30 to 89%. The original examples do the opposite. Added to the rule union, they cost 0 to 14 points and bring takeover back to 32 to 55%. Examples therefore cut both ways: an example that follows one skill causes takeover, and an example that follows both skills is the main part of a good merge.

The example helps only when it is correct. We added each merger’s own example to the same rule union. The gain follows how often a merger’s examples meet every achievable rule. The examples of Mistral-7B, none of which does, add nothing or cost points, and those of Claude Opus 5.5, which almost always do, add the most (Figure 5). Over all merged files, a correct example adds 12 to 17 points and an incorrect one adds little or nothing. For the same job and primary, a correct example written by another merger helps as much as Claude’s (within 1 point on each of the three readers). A strong merger helps mainly because it writes a correct example that follows both skills far more often.

The step-by-step merge is only as good as its labels. We expected the step-by-step merge to beat the plain merge. It does so only for the strongest merger. For Claude Opus 5.5 the two are within about one point, and the step-by-step files give more fully correct outputs on all seven models (68 to 94% against 60 to 86%). For the seven models that merge for themselves, the step-by-step merge is level with their plain merge or below it (Appendix § 9).

The reason is the labeling step. Claude Opus 5.5 labels all 48 interacting rule pairs correctly and adds none. The seven models label 2 to 39 of the 48 pairs correctly, report 8 to 61 pairs that do not interact, and treat every rule correctly in at most 14 of their 32 files. The typical errors are an entangled pair that is missed and left as two separate rules, two independent rules that are rewritten into one, and a contradiction that is labeled as entangled. Where a file has no such error, the step-by-step merge nearly matches the plain merge (96 against 96% for Qwen3-235B, and 94 against 96% for GPT-4.1). Where it has one, it loses (84 against 94%, and 88 against 94%). Given the correct labels, the step-by-step merge of Qwen3-4B gains 8 to 10 points on all seven models, and those of Qwen3-235B and GPT-4.1 gain 4 and 3 points on their own models, which brings them to within 4 and 1 points of their plain merges. Structure helps only when the labels are right, and only the strongest merger got them right.

Choosing the practice requests of the feedback loop so that they provoke a clash, in the spirit of curriculum generation [21], gave the same gain as plain requests when we tested it on the merged file of Qwen3-4B.

Inside the model

Can the winner be read from the activations? In conflicts between system and user instructions, the winning side can be read from the activations and moved by steering [6]. For two skills in one prompt we ask for more: can steering make the model follow both? We take the model’s activations at the last prompt token, before it writes anything, and train a linear probe [3] to predict whether the first-listed or the second-listed skill will win. For a pair and load order that the probe has seen on other requests, there is nothing left to learn: guessing that pair’s usual winner is already right for 96 to 99% of the takeover outputs on Qwen3-4B, Qwen2.5-7B and Llama-3.1-8B. For a pair it has not seen, we could not show on any of the three models that the probe beats always guessing the first-listed skill (Appendix § 12). With 16 held-out pairs this test is weak, and the steering result below suggests that some general signal exists.

Can the winner be moved? Activation steering adds a fixed vector to the output of one layer while the model runs [15, 18]. Steering at a single layer can be unstable [5, 16]. We steer toward the skill that lost, on requests that were not used to choose the layer and the strength (Appendix E). A general direction, computed from the other 15 jobs as “first-listed skill won” minus “second-listed skill won”, flips the winner in 38 to 49% of takeovers on the three models. A random direction of the same size at the same layer flips 0 to 3% (Figure 6). A direction built for the specific job flips about as many as its own random control. Steering does not make the model follow both skills. That happens in at most 13% of steered outputs on any of the three models, and on Llama-3.1-8B a random direction does as well. On Qwen3-4B, stronger steering breaks the text.

So takeover behaves like a choice between two alternatives. It can be tipped the other way, but steering does not turn it into following both skills. This matches § 5, where the fixes that work give the model one file, and one example, instead of two.

Discussion

What this means for skill tooling. Takeover happens after both files are loaded, so better skill selection would not fix it, and blocking one skill [22] gives up that skill’s rules. For the small models, the instructions we added recover only part of the dropped skill. For the strong models an explicit instruction brings it back, but it leaves the interacting rules broken. On every model, the best result came from merging the two skills once, with the strongest model at hand, and loading the merged file. The merged file should end with one correct worked example that follows both skills, and it should not copy the examples of the source skills. Where the files cannot be merged, giving each file such an example already removes most of the takeover.

What is not solved. The step-by-step merge depends on labeling the rule pairs, and only one merger labeled them well enough. Two repairs look cheap and remain to be tested: checking each combined rule and the example against the source rules before the file is saved, and falling back to a plain merge when the labels are in doubt. Checking a merge on real skills also needs a judge model or hand-written checks, since real skills come without rule checkers.

Real skills. In the public run summaries of Yan et al. [22], Claude Code runs that used both skills met the requirements that only the first skill states about 8 points less often than the paired run with that skill alone (Appendix F). This is observational, and the effect is smaller than in our benchmark, but it points the same way.

Benchmark details

Jobs. Commit message, pull request description, SQL query, incident report, Python function, support reply, changelog entry, JSON error body, code review comment, test case specification, standup update, bug report, Dockerfile, endpoint documentation, on-call alert and user story.

Checks before any model is run. A self-test confirms for every pair that each skill’s example follows its own rules, that the contradiction and the entangled pair are real on the examples, and that the reference output for each choice of primary follows its ten rules.

Prompts. The system prompt is: “You are an assistant working in a software team’s workspace. The skills below are installed and loaded for this session. Follow every rule of every loaded skill that applies to the request.” The skill files follow, each inside a <skill> tag. The priority instruction adds: “If two loaded skills give conflicting instructions, the skill X takes precedence. Otherwise follow all rules of all loaded skills.” The “apply both” instruction adds: “The loaded skills are complementary, not alternatives. Apply every rule of BOTH skills in the same output. Only where two rules cannot both hold does the rule from X win.” The user turn is the request plus “Reply with the [kind of text] only, with no commentary before or after it.” The plain merge request is: “Two agent skills will be installed together. Merge them into ONE skill file that replaces both. Keep every rule of both skills. Where two rules cannot both hold, the rule from X wins. Write a complete SKILL.md with frontmatter, a rules section and one example.”

Run details. Qwen3-235B (qwen/qwen3-235b-a22b-2507) and GPT-4.1 (openai/gpt-4.1) were called through OpenRouter at temperature 0. GPT-4.1 ran four of the eight requests per job. Qwen2.5-7B and Mistral-7B-v0.3 ran on a cloud GPU with pinned model revisions. We replayed every answer of these runs from the saved generations with the benchmark’s own prompts and checkers and found no difference.

Feedback loop. The model proposes 16 practice requests per job, of which 12 are kept. Six of them give the feedback, and the other six pick the final file. The loop runs six rounds with three feedback requests per round.

Rules the model follows anyway. A few rules are followed even with no skill loaded. If we keep only the rules that the model follows with the skill and not without it (162 to 174 of 192), the takeover rate is 52.1 to 84.4% across the seven models, close to the main numbers.

All conditions

condition	model: Qwen3-235B achievable	ent.	all	t.o.	model: GPT-4.1 achievable	ent.	all	t.o.
two skill files
Both skills loaded	59.6 [56.2, 63.6]	7	1	73	58.9 [56.4, 61.7]	7	1	77
Both skills loaded, examples removed	71.4 [65.4, 77.1]	37	7	41	–	–	–	–
Both skills loaded, each with Claude’s example	96.2 [93.9, 98.2]	91	76	1	89.9 [84.7, 94.5]	83	60	15
Write with the primary skill, revise with the other	59.6 [55.1, 64.5]	15	0	52	67.3 [62.4, 72.5]	19	0	38
Only the primary skill	60.7 [59.8, 61.8]	5	0	89	60.9 [59.9, 62.3]	5	0	92
Priority instruction	70.6 [65.3, 76.3]	19	9	56	62.6 [60.5, 64.8]	11	0	82
“Apply both” instruction	87.6 [84.4, 90.6]	38	22	0	88.6 [85.7, 91.3]	32	21	0
“Apply both” instruction, primary listed first	88.4 [85.0, 91.6]	35	21	0	89.1 [86.0, 92.0]	31	20	1
Rule checklist first, primary listed first	71.1 [65.5, 77.5]	26	14	60	63.4 [60.5, 67.1]	12	1	84
one meta-skill file
Both full skill files inside one file (no model)	69.1 [64.0, 74.5]	19	6	47	60.5 [57.6, 63.9]	12	1	73
Rule union + both original examples (no model)	76.6 [70.5, 82.6]	40	13	32	72.7 [65.2, 80.3]	42	12	49
Rule union in one file (no model)	86.2 [81.1, 90.2]	69	25	2	86.6 [83.0, 89.8]	60	21	4
Rule union + priority line (no model)	85.7 [81.6, 89.4]	72	19	1	89.7 [86.9, 92.2]	68	30	5
Plain merge, by Qwen3-4B	80.8 [75.2, 86.2]	34	13	24	81.6 [75.6, 87.3]	38	16	27
Step-by-step merge, by Qwen3-4B	78.4 [72.2, 84.4]	43	17	0	80.0 [73.8, 85.9]	44	15	0
Step-by-step merge, Qwen3-4B, true rule kinds given	87.4 [82.7, 91.9]	48	29	0	89.4 [85.7, 92.9]	49	30	0
Plain merge, by Qwen3-235B	94.3 [92.6, 96.1]	64	51	0	–	–	–	–
Step-by-step merge, by Qwen3-235B	86.6 [79.5, 92.3]	54	31	0	–	–	–	–
Step-by-step merge, Qwen3-235B, true rule kinds given	90.7 [84.4, 95.5]	59	48	0	–	–	–	–
Plain merge, by GPT-4.1	–	–	–	–	94.7 [92.0, 97.2]	59	57	0
Step-by-step merge, by GPT-4.1	–	–	–	–	90.8 [82.0, 96.6]	62	53	0
Step-by-step merge, GPT-4.1, true rule kinds given	–	–	–	–	93.9 [89.1, 97.6]	76	63	0
one meta-skill file, after the feedback loop
Step-by-step merge, Qwen3-4B, after the loop (clash requests)	88.0 [81.5, 93.0]	69	39	0	88.8 [82.2, 93.8]	72	41	0
one meta-skill file, merged by Claude Opus 5.5
Plain merge, by Claude Opus 5.5	97.7 [96.3, 98.9]	91	80	0	98.6 [97.6, 99.5]	92	86	0
Step-by-step merge, by Claude Opus 5.5	99.0 [98.1, 99.8]	99	91	0	99.4 [98.6, 100.0]	98	94	0
Step-by-step merge, by Claude Opus 5.5, example removed	96.7 [94.4, 98.7]	98	73	0	97.7 [95.6, 99.3]	97	80	0
Rule union + priority line + Claude’s example	97.5 [95.5, 99.1]	97	79	0	98.9 [98.0, 99.7]	97	89	0

condition	model: Qwen3-4B achievable	ent.	all	t.o.	model: Llama-3.1-8B achievable	ent.	all	t.o.
two skill files
Both skills loaded	58.1 [55.7, 61.2]	12	0	78	59.1 [55.8, 62.9]	13	0	66
Both skills loaded, examples removed	64.7 [59.7, 69.6]	29	1	34	63.1 [57.5, 69.1]	30	3	34
Both skills loaded, each with Claude’s example	92.9 [89.5, 96.1]	90	60	6	94.2 [91.2, 96.6]	92	63	2
Write with the primary skill, revise with the other	53.3 [50.5, 56.7]	6	0	75	62.5 [58.3, 67.0]	14	0	40
Only the primary skill	60.0 [58.9, 61.6]	5	0	87	60.5 [59.3, 61.9]	7	0	88
Priority instruction	62.1 [60.3, 64.3]	10	0	74	61.5 [58.6, 64.9]	12	0	66
“Apply both” instruction	69.5 [65.0, 73.7]	19	4	46	72.0 [66.6, 76.5]	17	3	28
“Apply both” instruction, primary listed first	68.2 [64.4, 72.4]	17	3	60	74.2 [69.7, 78.3]	19	5	33
Rule checklist first, primary listed first	61.7 [59.5, 64.1]	10	0	79	74.0 [67.0, 79.8]	23	10	27
one meta-skill file
Both full skill files inside one file (no model)	62.1 [58.1, 66.5]	20	3	71	59.7 [56.5, 63.4]	9	0	71
Rule union + both original examples (no model)	69.0 [63.9, 74.2]	26	5	42	69.9 [66.6, 73.4]	27	2	38
Rule union in one file (no model)	77.1 [70.2, 83.4]	56	13	13	81.1 [76.2, 85.5]	50	12	4
Rule union + priority line (no model)	78.6 [72.7, 83.9]	55	7	16	79.4 [74.2, 84.3]	47	11	4
Plain merge, by Qwen3-4B	79.3 [73.8, 84.6]	33	10	27	79.7 [74.0, 85.3]	37	11	28
Step-by-step merge, by Qwen3-4B	78.5 [72.3, 84.5]	41	16	0	78.3 [71.9, 84.6]	41	15	0
Step-by-step merge, Qwen3-4B, true rule kinds given	87.0 [82.3, 91.6]	50	25	0	86.5 [81.4, 91.4]	50	25	0
Plain merge, by Llama-3.1-8B	87.0 [83.2, 90.7]	43	22	0	86.9 [82.5, 91.1]	46	25	0
Step-by-step merge, by Llama-3.1-8B	80.5 [74.3, 86.0]	46	11	3	81.2 [75.2, 86.7]	45	14	3
Plain merge, by Qwen3-235B	93.9 [91.5, 96.2]	66	51	0	93.5 [90.8, 96.1]	64	51	0
Step-by-step merge, by Qwen3-235B	87.1 [80.4, 92.5]	54	32	0	86.9 [79.7, 92.5]	56	33	0
Step-by-step merge, Qwen3-235B, true rule kinds given	91.2 [85.1, 95.7]	61	51	0	91.3 [85.3, 95.9]	65	54	0
Plain merge, by GPT-4.1	92.5 [88.8, 96.1]	59	53	1	92.9 [89.4, 96.2]	57	51	0
Step-by-step merge, by GPT-4.1	89.5 [80.9, 95.5]	59	45	0	89.6 [81.0, 95.5]	65	50	0
Step-by-step merge, GPT-4.1, true rule kinds given	92.9 [88.2, 96.5]	73	56	0	93.0 [88.3, 96.8]	76	59	0
one meta-skill file, after the feedback loop
Rule union + priority line, after the loop	92.4 [88.8, 95.5]	60	47	1	89.1 [83.8, 94.0]	63	46	4
Step-by-step merge, Qwen3-4B, after the loop (plain requests)	88.4 [81.2, 94.1]	67	43	0	87.9 [81.0, 93.6]	68	39	0
Step-by-step merge, Qwen3-4B, after the loop (plain requests)	88.2 [81.6, 93.3]	70	40	0	87.0 [80.4, 92.6]	69	38	0
one meta-skill file, merged by Claude Opus 5.5
Plain merge, by Claude Opus 5.5	97.7 [96.6, 98.8]	90	79	0	98.0 [96.5, 99.3]	93	83	0
Step-by-step merge, by Claude Opus 5.5	98.4 [97.0, 99.5]	98	87	0	98.6 [97.0, 99.6]	98	88	0
Step-by-step merge, by Claude Opus 5.5, example removed	89.1 [84.1, 93.4]	88	44	2	88.8 [83.3, 93.8]	81	46	0
Rule union + priority line + Claude’s example	90.9 [87.3, 94.3]	80	49	6	95.5 [92.7, 98.0]	93	71	0

condition	model: Qwen2.5-7B achievable	ent.	all	t.o.	model: Mistral-7B achievable	ent.	all	t.o.	model: Phi-4-mini achievable	ent.	all	t.o.
two skill files
Both skills loaded	58.6 [55.4, 62.2]	8	0	69	60.3 [51.4, 67.4]	16	2	45	59.9 [56.4, 63.5]	19	0	49
Both skills loaded, examples removed	65.7 [60.1, 71.3]	48	3	22	68.4 [60.6, 75.4]	43	5	7	59.3 [49.3, 68.1]	43	3	19
Both skills loaded, each with Claude’s example	90.7 [87.5, 93.6]	87	49	5	54.0 [55.0, 65.6]	–	–	–	91.0 [87.6, 94.1]	81	50	2
Write with the primary skill, revise with the other	59.2 [54.8, 64.5]	16	0	56	60.4 [55.0, 65.6]	19	0	46	56.5 [53.4, 59.8]	15	0	62
Only the primary skill	59.8 [58.5, 61.2]	4	0	85	57.8 [53.9, 60.8]	5	0	82	59.7 [58.4, 61.1]	6	0	82
Priority instruction	64.6 [61.0, 68.5]	11	0	55	64.9 [60.1, 69.8]	15	1	48	64.2 [60.3, 68.1]	20	1	36
“Apply both” instruction	73.1 [67.6, 78.2]	21	3	21	78.1 [72.8, 82.7]	23	7	8	69.3 [65.1, 73.6]	23	2	21
“Apply both” instruction, primary listed first	73.3 [66.8, 78.8]	18	3	20	80.1 [74.2, 85.4]	25	10	10	69.0 [64.6, 73.7]	24	3	13
Rule checklist first, primary listed first	74.1 [70.3, 77.9]	22	4	20	68.5 [66.5, 73.0]	18	2	29	66.8 [62.5, 71.2]	16	1	18
one meta-skill file
Both full skill files inside one file (no model)	60.3 [56.5, 64.7]	14	1	69	–	–	–	–	61.2 [57.0, 65.4]	23	0	34
Rule union + both original examples (no model)	62.4 [59.3, 65.8]	15	1	55	–	–	–	–	65.1 [60.5, 70.2]	20	2	34
Rule union in one file (no model)	75.3 [68.5, 81.6]	50	13	3	75.3 [69.0, 81.1]	43	8	4	64.9 [51.7, 76.3]	38	7	5
Rule union + priority line (no model)	73.2 [67.0, 79.5]	50	7	3	64.6 [69.9, 78.9]	46	4	14	65.9 [53.7, 77.0]	43	9	4
Plain merge, by Qwen3-4B	78.6 [72.3, 84.6]	32	10	28	76.6 [71.0, 82.2]	27	10	25	77.2 [71.3, 83.0]	34	9	25
Step-by-step merge, by Qwen3-4B	77.8 [71.7, 83.9]	41	14	0	74.5 [68.3, 80.6]	35	11	0	76.8 [70.7, 83.2]	40	15	0
Step-by-step merge, Qwen3-4B, true rule kinds given	86.1 [80.9, 91.2]	48	23	0	84.1 [77.8, 90.0]	50	23	0	85.0 [79.5, 90.2]	50	25	1
one meta-skill file, after the feedback loop
Step-by-step merge, Qwen3-4B, after the loop (clash requests)	86.6 [80.0, 92.2]	68	35	0	83.6 [77.3, 89.2]	61	29	0	85.3 [78.9, 90.7]	66	30	0
one meta-skill file, merged by Claude Opus 5.5
Plain merge, by Claude Opus 5.5	96.4 [94.1, 98.3]	90	68	0	95.0 [92.5, 97.3]	84	60	0	95.4 [93.0, 97.7]	85	65	0
Step-by-step merge, by Claude Opus 5.5	96.7 [94.1, 98.8]	96	76	0	96.8 [93.9, 99.1]	91	77	0	95.9 [93.9, 97.7]	95	68	0
Step-by-step merge, by Claude Opus 5.5, example removed	83.2 [76.8, 89.3]	85	31	0	–	–	–	–	71.2 [60.7, 80.8]	68	17	4
Rule union + priority line + Claude’s example	94.3 [91.2, 97.0]	90	61	0	–	–	–	–	90.0 [87.2, 92.8]	84	39	0

merger	rules copied	right labels	false pairs	clean files	good examples	files read by the merger itself step-by-step	files read by the merger itself plain	files read by Qwen3-4B step-by-step	files read by Qwen3-4B plain	files read by Llama-3.1-8B step-by-step	files read by Llama-3.1-8B plain
Qwen3-4B	190/192	24/48	38	2/32	5/32	78.5	79.3	78.5	79.3	78.3	79.7
Qwen2.5-7B	189/192	8/48	8	0/32	3/32	75.4	75.0	–	–	–	–
Mistral-7B	117/192	4/48	61	0/32	0/32	65.2	80.2	–	–	–	–
Llama-3.1-8B	182/192	8/48	33	0/32	4/32	81.2	86.9	80.5	87.0	81.2	86.9
Phi-4-mini	120/192	2/48	30	0/32	2/32	52.4	71.1	–	–	–	–
Qwen3-235B	188/192	32/48	18	6/32	10/32	86.6	94.3	87.1	93.9	86.9	93.5
GPT-4.1	192/192	39/48	8	14/32	17/32	90.8	94.7	89.5	92.5	89.6	92.9
Claude Opus 5.5	192/192	48/48	0	32/32	30/32	97.8	97.0	98.4	97.7	98.6	98.0

comparison (points)	Qwen3-4B	Qwen2.5-7B	Mistral-7B	Llama-3.1-8B	Phi-4-mini	Qwen3-235B	GPT-4.1
One file against two
One file vs. two files (same rules, no examples)	+12.5 [7.8, 17.5]	+9.6 [5.1, 14.3]	+6.9 [0.8, 14.7]	+17.9 [13.6, 22.4]	+5.6 [-1.2, 11.4]	+14.8 [8.6, 21.2]	–
Rule union vs. “apply both” instruction	+9.1 [3.3, 14.9]	+0.1 [-6.5, 7.0]	−3.5 [-9.7, 2.3]	+7.4 [2.8, 11.7]	−3.4 [-14.6, 6.2]	−1.9 [-6.6, 2.3]	+1.1 [-2.6, 4.4]
Claude’s file vs. “apply both” instruction	+28.9 [24.5, 33.5]	+23.6 [19.0, 28.7]	+18.7 [14.6, 23.5]	+26.5 [22.2, 31.9]	+26.6 [22.9, 30.3]	+11.4 [8.8, 14.3]	+10.8 [8.0, 13.8]
Who merges and how
Claude’s file vs. the model’s own plain merge	+19.1 [14.1, 24.6]	+21.7 [15.7, 28.7]	+16.6 [11.6, 22.7]	+11.7 [8.1, 15.3]	+24.8 [19.7, 29.9]	+4.7 [3.0, 6.4]	+4.7 [2.3, 7.3]
Step-by-step vs. plain merge, the model merges for itself	−0.8 [-6.7, 5.1]	+0.3 [-6.9, 8.3]	−15.0 [-23.0, -7.2]	−5.7 [-10.0, -1.4]	−18.8 [-29.5, -8.0]	−7.7 [-14.6, -2.0]	−3.9 [-13.1, 2.1]
Step-by-step vs. plain merge, Claude merges	+0.7 [-0.3, 1.7]	+0.4 [-0.5, 1.3]	+0.8 [0.4, 3.3]	+0.5 [-0.3, 1.5]	+0.5 [-0.4, 1.6]	+0.7 [0.5, 2.2]	+0.8 [0.1, 1.7]
Step-by-step merge with the correct rule kinds
Qwen3-4B’s merge: correct vs. own labels	+8.6 [0.8, 16.6]	+8.3 [0.6, 16.4]	+9.6 [3.1, 16.4]	+8.2 [0.5, 16.3]	+8.1 [0.7, 15.8]	+9.1 [1.8, 16.8]	+9.4 [2.0, 16.8]
Qwen3-235B’s merge: correct vs. own labels	+4.1 [1.4, 7.1]	–	–	+4.4 [1.6, 7.6]	–	+4.0 [1.2, 7.2]	–
Qwen3-235B’s merge: correct labels vs. its plain merge	−2.7 [-8.0, 1.9]	–	–	−2.2 [-7.8, 2.6]	–	−3.7 [-9.8, 1.4]	–
GPT-4.1’s merge: correct vs. own labels	+3.4 [0.2, 7.8]	–	–	+3.5 [0.1, 7.9]	–	–	+3.1 [-0.1, 7.5]
GPT-4.1’s merge: correct labels vs. its plain merge	+0.4 [-6.3, 5.9]	–	–	+0.2 [-5.8, 5.0]	–	–	−0.8 [-6.6, 3.7]
Feedback loop (Qwen3-4B’s files)
Rule union after vs. before the loop	+13.8 [10.4, 17.3]	–	–	+9.7 [5.7, 14.6]	–	–	–
Clash-provoking vs. plain practice requests	−0.2 [-2.1, 1.6]	–	–	−0.9 [-3.8, 1.4]	–	–	–

What triggers the takeover

model	skills shown	alone	both	takeover	both fully	expected	both fully − expected
Qwen3-4B	full skills, with examples	96.9	53.4	78.1 [62.1, 91.4]	0.4	86.7	−86.3 [−96.9, −73.0]
	examples removed	92.8	63.5	33.6 [19.5, 49.2]	16.0	62.5	−46.5 [−69.5, −21.5]
	+ contradictory rules removed	93.8	68.8	25.4 [13.3, 39.8]	26.2	68.8	−42.6 [−59.4, −25.4]
	+ entangled rules removed (fully compatible)	92.4	70.8	23.0 [9.4, 39.1]	30.5	63.3	−32.8 [−50.8, −16.4]
Qwen2.5-7B	full skills, with examples	95.8	54.6	68.8 [52.0, 84.4]	5.9	78.9	−73.0 [−88.7, −55.5]
	examples removed	87.0	62.2	22.3 [12.1, 35.2]	14.1	57.0	−43.0 [−60.5, −25.0]
	+ contradictory rules removed	89.7	72.0	15.2 [3.9, 30.1]	24.2	65.6	−41.4 [−58.2, −25.0]
	+ entangled rules removed (fully compatible)	88.2	73.4	6.2 [0.0, 17.6]	34.0	57.8	−23.8 [−39.8, −8.6]
Mistral-7B	full skills, with examples	92.4	59.4	45.3 [27.7, 63.7]	18.0	74.2	−56.2 [−74.6, −36.7]
	examples removed	81.4	69.1	6.6 [0.8, 14.8]	25.0	42.2	−17.2 [−28.9, −6.6]
	+ contradictory rules removed	78.6	72.6	1.6 [0.0, 4.7]	34.8	34.4	+0.4 [−20.7, 19.5]
	+ entangled rules removed (fully compatible)	78.6	70.8	4.3 [0.4, 10.9]	32.0	28.1	+3.9 [−11.7, 17.6]
Llama-3.1-8B	full skills, with examples	97.3	55.1	66.4 [51.6, 79.7]	3.9	89.1	−85.2 [−95.7, −71.5]
	examples removed	94.0	63.8	34.4 [19.1, 50.8]	19.5	71.1	−51.6 [−69.1, −32.0]
	+ contradictory rules removed	86.5	67.8	28.5 [14.1, 45.3]	31.6	61.7	−30.1 [−54.3, −4.3]
	+ entangled rules removed (fully compatible)	82.7	68.2	26.6 [10.9, 44.9]	29.7	48.4	−18.8 [−41.8, 5.5]
Phi-4-mini	full skills, with examples	95.2	55.3	48.8 [32.0, 66.4]	1.2	76.6	−75.4 [−90.6, −57.8]
	examples removed	70.4	56.6	19.1 [7.4, 33.2]	12.1	37.5	−25.4 [−41.4, −10.5]
	+ contradictory rules removed	81.1	56.7	17.2 [6.2, 31.2]	10.2	50.0	−39.8 [−59.4, −20.7]
	+ entangled rules removed (fully compatible)	80.1	60.7	9.8 [2.0, 21.9]	20.3	45.3	−25.0 [−39.1, −11.7]
Qwen3-235B	full skills, with examples	98.3	56.5	72.7 [53.1, 89.5]	6.2	90.6	−84.4 [−95.7, −71.1]
	examples removed	96.1	71.3	40.6 [21.5, 60.5]	30.5	82.0	−51.6 [−70.7, −32.4]
	+ contradictory rules removed	96.5	75.5	27.7 [12.5, 44.9]	41.4	83.6	−42.2 [−62.9, −22.7]
	+ entangled rules removed (fully compatible)	96.1	79.3	20.3 [6.2, 37.5]	47.7	80.5	−32.8 [−51.2, −15.6]

Takeover by job

job	artifact	Qwen3-4B takeover	Qwen3-4B both fully	Qwen2.5-7B takeover	Qwen2.5-7B both fully	Mistral-7B takeover	Mistral-7B both fully	Llama-3.1-8B takeover	Llama-3.1-8B both fully	Phi-4-mini takeover	Phi-4-mini both fully	Qwen3-235B takeover	Qwen3-235B both fully	GPT-4.1 takeover	GPT-4.1 both fully
commit	commit message	100	0	100	0	100	31	100	0	19	0	100	0	62	38
pr	pull request description	81	0	94	0	12	44	50	31	38	0	100	0	75	25
sql	SQL query	0	0	6	25	0	12	0	19	19	6	0	12	12	12
incident	incident report	38	6	12	0	50	0	50	0	0	0	12	0	0	0
python	Python function	31	0	25	0	12	0	12	0	19	0	31	6	62	0
support	support reply	50	0	62	0	6	0	62	0	31	0	38	0	88	0
changelog	changelog entry	100	0	100	0	94	0	69	0	50	0	100	0	100	0
apierror	JSON error response body	94	0	100	0	81	6	81	0	50	6	100	0	100	0
review	code review comment	94	0	19	50	6	6	94	0	50	6	100	0	100	0
testcase	test case specification	88	0	100	0	6	88	56	0	94	0	88	0	100	0
standards	daily standup update	100	0	100	0	100	0	100	0	100	0	100	0	100	0
bugreport	bug report	100	0	75	0	56	31	94	0	100	0	88	12	100	0
docx	Dockerfile	81	0	56	12	94	0	81	6	94	0	12	62	50	25
apidoc	REST endpoint documentation entry	100	0	100	0	31	6	62	0	94	0	100	0	50	0
alert	on-call alert notification	100	0	88	6	31	69	100	0	25	0	100	0	100	0
userstory	user story ticket	94	0	75	0	100	0	50	0	50	0	94	6	75	25

Steering details

We add one vector to the output of one transformer block at every position, while the model reads the prompt and while it writes. The general direction is the mean activation at the last prompt token over prompts with both skills loaded where the first-listed skill won, minus the mean where the second-listed skill won, taken from the other 15 jobs. The job-specific direction is the mean activation while the model writes with the target skill alone, minus the same for the other skill. Each vector is scaled to a fraction $c$ of the mean activation norm of its block. We choose the block (13, 19 or 24 of 36 on Qwen3-4B; 10, 15 or 20 of 28 on Qwen2.5-7B; 12, 17 or 22 of 32 on Llama-3.1-8B) and $c$ on four requests per job and report the other four. We always steer toward the skill that lost without steering, so this is a test of mechanism and not a fix. On Qwen3-4B we also tried $c = 0.8$ on the tuning requests. At that strength, up to 44% of outputs follow no exclusive rule of either skill.

model	held out	baseline	text features	probe	probe − baseline	best block, post hoc
Qwen3-4B	new skill pair	58.0	50.0	67.0	+6.2 [−15.0, 24.4]	74.0 (26)
Qwen3-4B	new request	96.5	58.0	96.0	−1.3 [−4.0, 0.0]	96.5 (22)
Qwen2.5-7B	new skill pair	76.1	75.6	76.1	−2.2 [−15.5, 9.7]	88.6 (22)
Qwen2.5-7B	new request	98.9	79.0	98.3	−1.6 [−4.7, 0.0]	98.3 (18)
Llama-3.1-8B	new skill pair	74.1	67.1	70.6	−5.0 [−15.4, 0.9]	79.4 (16)
Llama-3.1-8B	new request	97.1	72.9	96.5	−0.8 [−2.5, 0.0]	97.1 (15)

model	steering	stays	flips	both	partial	neither	achievable
Qwen3-4B	no steering	100	0.0 [0.0, 0.0]	0	0	0	56.8
Qwen3-4B	job-specific direction, best for both (block 13, c=0.4)	46	14.6 [5.0, 25.2]	5	34	0	60.0
Qwen3-4B	job-specific direction, best for flipping (block 13, c=0.6)	32	15.9 [3.3, 32.0]	2	45	5	51.2
Qwen3-4B	random control for the row above (block 13, c=0.6)	46	11.4 [3.3, 21.4]	6	35	2	59.3
Qwen3-4B	general direction, best for both (block 19, c=0.4)	35	23.3 [8.8, 38.9]	13	28	1	62.4
Qwen3-4B	general direction, best for flipping (block 24, c=0.4)	13	48.8 [29.6, 68.3]	8	30	0	59.2
Qwen3-4B	random control for the row above (block 24, c=0.4)	86	2.9 [0.0, 6.1]	1	10	0	56.0
Qwen2.5-7B	no steering	100	0.0 [0.0, 0.0]	0	0	0	56.4
Qwen2.5-7B	job-specific direction, best for both (block 15, c=0.4)	28	11.8 [3.9, 21.2]	10	50	0	56.7
Qwen2.5-7B	job-specific direction, best for flipping (block 10, c=0.6)	25	7.0 [0.0, 16.4]	0	65	2	48.1
Qwen2.5-7B	random control for the row above (block 10, c=0.6)	61	7.5 [0.8, 17.7]	0	32	0	56.2
Qwen2.5-7B	general direction, best for both (block 10, c=0.4)	49	13.0 [3.6, 23.9]	0	38	0	58.9
Qwen2.5-7B	general direction, best for flipping (block 20, c=0.4)	9	48.1 [28.6, 68.1]	0	41	2	55.6
Qwen2.5-7B	random control for the row above (block 20, c=0.4)	91	0.0 [0.0, 0.0]	0	9	0	55.5
Llama-3.1-8B	no steering	100	0.0 [0.0, 0.0]	0	0	0	56.7
Llama-3.1-8B	job-specific direction, best for both (block 12, c=0.4)	46	4.2 [0.0, 11.2]	10	39	0	62.1
Llama-3.1-8B	random control for the row above (block 12, c=0.4)	64	9.4 [0.0, 21.0]	13	13	0	61.1
Llama-3.1-8B	general direction, best for both (block 17, c=0.6)	26	37.7 [22.4, 54.7]	8	28	0	61.9
Llama-3.1-8B	random control for the row above (block 17, c=0.6)	89	0.0 [0.0, 0.0]	1	7	3	55.2

Public run summaries of One Skill Too Many