Combating Catastrophic Forgetting in Continual Domain Adaptation via Knowledge Recasting

Introduction

Post-training improves language models (LMs) through supervised fine-tuning [22], reinforcement learning from feedback [37, 4], and domain-adaptive continued training [19]. However, sequential post-training can also degrade capabilities acquired before the update, a phenomenon commonly known as catastrophic forgetting [35, 27, 34, 46, 54]. This risk becomes especially important under domain shift, where the update distribution introduces concepts, terminology, or reasoning contexts that differ from those emphasized by the pre-update model [46, 24, 62].

To clarify what existing mitigation strategies are designed to control, we view post-training under domain shift through the lens of model drift. As shown in Figure 2, knowledge updates can change a model in at least two distinct ways. Surface drift refers to changes in the model’s observable behavior, such as its output distribution or final answers, whereas representation drift refers to changes in the internal states that support those behaviors. These two forms of drift need not agree: the mapping from representations to outputs is many-to-one, so a model may reorganize its internal states while producing similar outputs, or substantially shift its output distribution while leaving representations relatively stable.

From this perspective, many forgetting-mitigation methods can be understood as different ways of anchoring the updated model to its pre-update behavior or state. Replay-based methods mix old examples with new training data so that the model continues to reproduce earlier task behavior during the update [43, 32, 24]. Distillation-based methods use the original model as a teacher and penalize deviations from its outputs [31]. Both families primarily constrain surface drift by preserving observable behavior on old examples. In contrast, regularization-based methods penalize changes to important parameters or restrict updates to protected subspaces [27, 60, 2, 52, 33], thereby limiting drift in the model’s internal state or parameterization.

Because these approaches primarily anchor one aspect of the update, anchoring surface behavior alone need not control representation drift, and anchoring parameter or representation movement alone need not preserve task behavior. This points to a limitation of treating forgetting mitigation only as a problem of preventing change: the key question is not merely how much the model changes, but which aspects of the update are being constrained and whether those constraints help integrate the new domain.

Yet anchoring the model is not the same as integrating new knowledge. Under out-of-domain updates, useful adaptation requires change: the model may need to acquire, emphasize, or recombine concepts and reasoning contexts that were weakly represented in the pre-update distribution. The key issue is therefore not how to minimize change, but how to organize it. The organizing principle we propose is procedural reuse. If a new problem can be solved by a procedure the model already uses, then the desirable update is not to memorize a new trajectory or to pull the model back toward its old outputs. Instead, the update should connect the unfamiliar instance to a familiar procedure, expressing new knowledge in procedural coordinates the model can reuse. Inspired by this view, we ask the following research question: RQ: Can new-domain knowledge be expressed in the procedural language of the model’s existing skills, so that unfamiliar problems become learnable through familiar procedures?

To probe whether this premise is plausible, we analyze the error structure of a base model across MATH categories. The same failed examples are embedded and projected in Figure 3; coloring them by procedural error type reveals clusters that cut across the original domain labels. This diagnostic does not by itself prove that all domains share a complete skill library, but it suggests that observed failures are not purely domain-specific and motivates testing whether schema-level reuse can improve continual adaptation.

Based on this observation, we propose Skill Schema Transport (SST), a schema-conditioned post-training framework that transports new-domain examples into a reusable procedural schema space. A skill schema serves as the medium of this translation: it retains the operations, dependencies, and checks needed to solve a task, while abstracting away domain-specific wording, formats, and trajectories. SST uses these schemas to associate unfamiliar examples with familiar procedures, allowing a new-domain example to be interpreted as a transported instance of a reusable skill rather than as an isolated new trajectory, as illustrated in Figure 1. We evaluate SST in streaming knowledge-update settings, where new-domain knowledge is introduced sequentially and the model must retain capabilities learned from earlier domains. Across two 3B instruction-tuned backbones, a four-domain MATH stream, and a three-domain agent stream, SST improves average final performance on streamed domains by 2.53-5.75 points over the corresponding sequential post-training baseline. It also reduces max-drop forgetting in all evaluated settings and maintains or substantially recovers OOD benchmark performance relative to sequential updates. To summarize, our main contributions are:

• We reframe forgetting mitigation under domain shift as a problem of knowledge organization rather than only model constraint, shifting the burden of adaptation from limiting model movement to structuring how new examples are represented during training.

• We provide drift and error-structure diagnostics showing that surface and representation drift need not align, and that procedural failure patterns can cut across domain boundaries, motivating procedural reuse as an alternative to purely drift-based anchoring.

• We propose Skill Schema Transport (SST), a schema-conditioned post-training framework that extracts domain-agnostic skill schemas and transports new-domain examples into reusable procedural coordinates.

• Across two streaming settings comprising seven update domains and two 3B instruction-tuned backbones, SST improves streamed-domain performance by 2.53-5.75 points over sequential post-training, reduces max-drop forgetting in all evaluated settings, and maintains or substantially recovers OOD benchmark performance relative to sequential updates.

Related Work

Continual Learning for Sequential LLM Training. Continual learning studies how models acquire new capabilities without overwriting old ones, with catastrophic forgetting as a central failure mode [35, 42, 15, 16]. Existing methods mitigate forgetting through parameter consolidation, functional regularization, replay, and gradient constraints, including EWC, SI, MAS, and related approaches [27, 60, 2, 8, 44, 31, 41, 32, 9, 7, 38, 12]. Analogous ideas appear in sequential RL through replay, distillation, and policy regularization, while recent LLM post-training studies show that continual pretraining, instruction tuning, SFT, and RLHF can all induce forgetting under domain shift. (43; 26; 37; 4; 34; 54; 46; 62). Parameter-efficient methods such as LoRA, O-LoRA, and CLoRA reduce interference by restricting adaptation to low-rank, orthogonal, or regularized subspaces, but they still mainly act as drift-control mechanisms [23, 52, 33]. Our work is complementary: instead of only constraining surface or representation drift, we ask whether new-domain knowledge can be transported into procedures already learned by the model, and analyze this using representation-similarity tools such as SVCCA, PWCCA, and CKA [40, 36, 28].

Skills, Temporal Abstraction, and Skill Discovery. The idea that complex behavior can be decomposed into reusable subroutines is central to hierarchical reinforcement learning, where Options, MAXQ, feudal RL, and Option-Critic define skills as temporally extended actions or subtasks that support behavioral reuse [49, 13, 11, 5, 3, 39]. A related line studies unsupervised skill discovery, where methods such as VIC, DIAYN, DADS, and OPAL learn diverse, predictable, or recurring primitives without task-specific supervision [18, 14, 45, 1]. We build on this abstraction view, but do not treat skills as options, policies, or transfer modules. Instead, we define skills as language-level schemas that preserve operations, dependencies, and checks while abstracting away domain-specific wording, formats, and trajectories, allowing us to test whether schema-based supervision induces less unnecessary representation drift during post-training [40, 28].

Skill-Augmented and Continually Improving Agents. Recent LLM-agent systems externalize reusable experience as memories, reflections, trajectories, executable programs, or skill libraries. ReAct interleaves reasoning with environment actions, while Reflexion and ExpeL improve agents through verbal feedback, episodic memory, or distilled experience without updating model weights [59, 47, 61]. Voyager extends this idea to open-ended lifelong learning with an executable skill library for embodied control [51]. More recent work makes skill learning explicit through recursive skill libraries, persistent skill memories, multimodal experience-skill stores, and skill-oriented benchmarks [55, 30, 25, 64]. These methods typically use skills as external memories, prompts, tools, or inference-time resources, whereas our work studies schema-conditioned sequential post-training with retrieval-free main evaluation: SST uses skill schemas during training to encourage new-domain knowledge to be internalized through reusable procedures.

Preliminary

We formalize the continual domain adaptation setting used throughout the paper. We first define the update and retention distributions, then define forgetting and drift diagnostics, and finally introduce the schema objects used by SST.

The setting specifies where new learning occurs and where retention is measured. We next define forgetting as the maximum accuracy drop over the observed stream.

We report average forgetting by averaging $\mathcal{F}_{\mathcal{Q}}$ over prior retention distributions, excluding the final streamed domain because it is not followed by a later update and therefore cannot be overwritten within the observed stream.

The forgetting gap measures the outcome of the update stream. To diagnose the updates themselves, we also track how much the model changes at the output and representation levels.

incremental drift from $\bar{\theta} = \theta_{t-1}$ are defined, respectively, as

These drift quantities measure changes in the model, but they do not specify whether two examples use the same solution procedure. SST therefore uses a schema-level representation of procedural structure. We represent procedural structure via skill schemas. Let $\tau\in\mathcal{T}$ denote a reasoning trace for solving $x$, and let $B_t$ denote the schema memory accumulated through step $t$.

Table 1 shows one instantiated schema and its fields. It specifies what information is stored in a schema, including its goal, applicability conditions, canonical steps, failure modes, and compositional links. Figure 4 illustrates how a concrete problem is paired with a transported schema to form a schema-conditioned SFT example.

Field	Description
Skill Name	Algebraic Substitution and Equation Solving
Goal	Solve algebra problems via rewriting, substitution, and branch verification.
Applicability	Equations, expressions, radicals, and symbolic substitutions.
Keywords	equation, substitute, solve, factor, expression
Structural Cues	functional relations; inequalities or intervals
Output Format	integer, fraction, radical, interval
Canonical Steps	(1) Rewrite the relation; (2) substitute or factor strategically; (3) solve candidate values; (4) check constraints.
Failure Modes	sign errors; dropped branches; unchecked extraneous solutions
Compositional Link	exact form and answer realization

Transportability determines how new examples interact with the schema memory $B_t$: transportable examples are attached to an existing procedure, while non-transportable examples introduce a new one. The maintenance operations on $B_t$ are detailed in Section 4.

Skill Schema Transport

We propose Skill Schema Transport (SST), a post-training framework that connects new-domain knowledge to procedures learned from earlier domains, shown in Algorithm 1. Rather than constraining the model to reproduce previous outputs, SST uses skill schemas as procedural descriptors, allowing a new-domain example to be interpreted through a reusable skill whenever its schema matches one stored in memory.

Schema Extraction. For each training example $(x,y,\tau) \sim\mathcal{D}_t$, the reasoning trace $\tau$ is the ground-truth solution provided by the dataset. The skill schema $z = \Gamma_{\theta_{t-1}}(\tau)$ is extracted using the model prior to the current update step via a structured prompt (detailed in

Appendix § 12. The extractor $\Gamma_{\theta_{t-1}}$ takes $\tau$ as input and produces a domain-agnostic schema $z \in\mathcal{Z}$ specifying the procedural structure of the solution, including operations, dependencies, and validation checks, while abstracting away domain-specific terminology and surface details.

Schema Memory Initialization and Skill Retrieval. The schema memory $\mathcal{B}_0$ is initialized from a fixed set of predefined skill families and seed skills before any examples from $\mathcal{D}_1$ are processed. During update step $t$, SST maintains a working memory $\bar{\mathcal{B}}$ initialized as $\mathcal{B}_{t-1}$. For each example in $\mathcal{D}_t$, the system extracts a candidate schema using $\Gamma_{\theta_{t-1}}$, retrieves the nearest skill from $\bar{\mathcal{B}}$, constructs the transported schema $\tilde{z}$, and immediately applies a memory-update operation to $\bar{\mathcal{B}}$. After all examples from $\mathcal{D}_t$ are processed, the working memory is committed as $\mathcal{B}_t$.

New skills are written into memory only if the same procedural signature receives at least $n_{\min}=2$ supporting examples, preventing spurious entries from single occurrences. Schema transportability (Definition 5) is defined conceptually over the schema space $\mathcal{Z}$. In implementation, we operationalize this with a problem-to-skill retrieval proxy: the input problem $x$ and each skill entry in the current working memory $\bar{\mathcal{B}}$ are encoded using bge-small-en-v1.5 (56) and ranked by cosine similarity. Each skill is represented as a canonicalized natural-language block comprising five fields: Skill ID, Family, Goal, When to use, Steps, and Common failure modes. The top-$k$ retrieved skill blocks ($k=2$ in our experiments) are then used to condition schema extraction and the construction of the transported schema $\tilde{z}$.

Schema-Conditioned Post-Training. At update step $t$, standard SFT optimizes

When $\mathcal{D}_t$ is out-of-domain, this objective may cause the model to acquire isolated domain-specific procedures rather than expressing new knowledge in terms of procedures already present in its repertoire. SST processes examples sequentially with a working memory $\bar{\mathcal{B}}_{t,0}=\mathcal{B}_{t-1}$. For the $j$-th example $(x_j, y_j, \tau_j)$, let $z_j = \Gamma_{\theta_{t-1}}(\tau_j)$ and assign

Here $\overline{B}_{t,j-1}$ is the current working memory before processing example $j$, and $\operatorname{NN}_{\overline{B}_{t,j-1}}(z_j)$ denotes the nearest neighbor of $z_j$ in that memory under $\operatorname{Dist}$. This yields the schema-conditioned dataset

and objective

Here $\widetilde{z}$ serves solely as input context and is not a supervision target: the model is trained to generate $y$ conditioned on both the problem $x$ and the retrieved procedural context $\widetilde{z}$, but is not explicitly trained to predict $\widetilde{z}$ itself. Thus $x$ provides instance-specific content, $y$ provides the target response, and $\widetilde{z}$ provides the procedural coordinates through which the new example is expressed.

Schema Memory Maintenance. The schema memory $B_t$ evolves across update steps without requiring replay access to $D_{<t}$. Memory updates occur per example, not as a separate post-hoc phase. During stage $t$, all schema extraction and memory-operation proposals are produced by the pre-update model $\Gamma_{\theta_{t-1}}$; the updated model $\theta_t$ is obtained only after the schema-conditioned dataset $\widetilde{D}_t$ has been constructed. After $\widetilde{z}$ is assigned for a given example, $\Gamma_{\theta_{t-1}}$ produces an update proposal and applies either a Reuse no-op or one of four memory-modifying operations. Revise: if the proposed schema matches an existing entry within the same skill family, the stored schema is updated with new keywords, canonical steps, or failure modes. Create: if $\overline{B} = \varnothing$ or $\operatorname{Dist}(z,\operatorname{NN}_{\overline{B}}(z)) > \epsilon$, the proposed schema is a candidate for a new entry; it is written into $B_t$ only when the same procedural signature accumulates at least $n_{\min}$ supporting examples and the extractor’s confidence exceeds a threshold $\delta_{\mathrm{create}}$. Merge: if two schemas share the same skill family and satisfy $\operatorname{Dist}(z,z') \le\delta_{\mathrm{merge}}$, they are consolidated, retaining the more-retrieved entry. Drop: a schema whose utility score $u(z)$ falls below $\delta_{\mathrm{drop}}$ is removed, provided it has been retrieved at least $k_{\min}$ times. The utility score aggregates behavioral signals including source correctness, retrieval support, retrieval precision. Implementation details and default hyperparameters are provided in Appendix D. Skill family membership is a discrete field assigned by the extractor at creation time from a fixed family list; it governs which skills are candidates for Revise and Merge, while Dist governs the within-family similarity threshold for Merge. Together, these operations allow $B_t$ to grow, refine, and prune itself as new domains are encountered. A local linearized analysis in Appendix B shows that, under a schema-separable approximation, schema-conditioned directions need not increase the minimum shared-parameter displacement required by an update.

Experiments

Models and datasets. We evaluate SST on Llama-3.2-3B-Instruct [17] and Qwen2.5-3B-Instruct [57]. We consider two streaming settings: a mathematical reasoning stream [21] with four sequential domains (Algebra $\rightarrow$ Counting & Probability $\rightarrow$ Geometry $\rightarrow$ Intermediate Algebra), and an agent-task stream with three sequential domains (AlfWorld $\rightarrow$ Web-Shop $\rightarrow$ search-augmented QA [48, 58, 55]).

Training and inference. At each stream step, methods may only access the current domain plus any information explicitly permitted by their design. SST applies the same base objective as the

Algorithm 1 Skill Schema Transport (SST)
Require: Pre-trained model $\mathcal{M}_{\theta_0}$
Require: Update stream $D_1,\ldots,D_T$
Require: Distance $\operatorname{Dist}$; thresholds $\epsilon$, $\delta_{\mathrm{create}}$, $\delta_{\mathrm{merge}}$, $\delta_{\mathrm{drop}}$, $n_{\min}$, $k_{\min}$
Ensure: Updated model $\mathcal{M}_{\theta_T}$, schema memory $B_T$
1: Initialize $B_0$ from seed skill families
2: for $t=1,\ldots,T$ do
3: $\overline{B}\leftarrow B_{t-1}$; $\widetilde{D}_t\leftarrow\varnothing$
4: for each $(x,y,\tau)\sim D_t$ do
5: $z\leftarrow\Gamma_{\theta_{t-1}}(\tau)$
6: $z^\star\leftarrow\arg\min_{z'\in\overline{B}}\operatorname{Dist}(z,z')$
7: $\widetilde{z}\leftarrow z^\star$ if $\operatorname{Dist}(z,z^\star)\le\epsilon$ else $z$
8: Add $(x,\widetilde{z},y)$ to $\widetilde{D}_t$
9: $\overline{B}\leftarrow\operatorname{MemUpdate}_{\Gamma_{\theta_{t-1}}}(\overline{B},z,\widetilde{z})$
10: end for
11: $B_t\leftarrow\overline{B}$
12: $\theta_t\leftarrow\arg\min_\theta\mathcal{L}^{t}_{\mathrm{SST}}(\theta;\widetilde{D}_t)$
13: end for

sequential baseline (SFT for MATH, RL for agent tasks), but rewrites updates through retrieved skill schemas. All main evaluations are retrieval-free at inference time, so reported gains reflect internalized knowledge rather than test-time prompting.

Evaluation. After each step we evaluate on all domains seen so far. We report average final performance and average max-drop forgetting. For a domain $d$ introduced at step $t_d$, let $A_t(d)$ denote its accuracy after update step $t$. We define

We average $F(d)$ over prior streamed domains, excluding the final domain because it is not followed by a later update. For the MATH stream, we additionally report transfer to held-out domains (Number Theory, Prealgebra, Precalculus) and preservation on general benchmarks (MMLU (20), GSM8K (10), PIQA (6), CommonsenseQA (50), SciQ (53)); the same benchmarks are used for the agent stream.

Baselines. We compare against SEQTRAIN (sequential, no mitigation), REPLAY (fixed-count rehearsal), and REGULARIZATION (L2 parameter anchoring to the base model). Unlike these, SST does not anchor to past examples or states; instead, it conditions new updates on reusable procedural schemas. We also include non-streaming experiments, Appendix § 8 shows that the gain-drop pattern also holds for independent single-domain updates.

RQ1: Does SST reduce forgetting while preserving out-of-stream performance? Table 2 summarizes results across both streaming settings and backbones. In the MATH stream, Seq-SFT causes severe forgetting on Qwen3B (avg. max-drop 42.41 pts, final streamed average 19.96%); SST reduces max-drop to 7.10 pts and raises the streamed average to 25.12%, while recovering general benchmark performance from 58.92% to 72.06%. Llama3B follows the same direction: streamed average improves from 20.30% to 22.83% and general benchmarks from 69.33% to 72.33%. In the agent-task stream, SST improves final average task performance over Seq-RL by 5.75 pts on Qwen3B (73.04% → 78.79%) and 3.27 pts on Llama3B (74.50% → 77.77%), with forgetting reduced and general benchmark scores preserved. Across both streams, schema-conditioned updates improve streaming adaptation without increasing interference with out-of-stream capabilities. Appendix § 10 provides the corresponding max-drop decomposition by domain.

Model	Method	Alg.	Count.	Geom.	Inter.	Stream Avg. ↑	Forget. ↓	MMLU	GSM8K	PIQA	CSQA	SciQ	OOD Avg. ↑
Qwen-3B	Base	44.99	28.48	22.34	9.19	26.25	–	65.45	63.00	77.69	75.54	92.50	74.84
Qwen-3B	Seq-SFT	26.62	21.31	18.16	13.73	19.96	42.41	57.18	7.58	74.43	61.59	93.80	58.92
Qwen-3B	SST	40.27	24.68	20.46	15.06	25.12	7.10	65.28	39.20	77.58	80.56	97.70	72.06
Llama-3B	Base	8.17	5.70	11.27	9.63	8.69	–	62.12	67.48	73.90	73.96	94.20	74.33
Llama-3B	Seq-SFT	30.03	20.16	18.66	12.33	20.30	4.39	60.43	46.40	73.72	71.01	95.10	69.33
Llama-3B	SST	31.76	21.73	20.88	16.94	22.83	4.09	60.58	61.84	74.32	70.60	94.30	72.33
Qwen-3B	Base	19.79	42.75	53.72	38.75	–	65.45	63.00	77.69	75.54	92.50	74.84
Qwen-3B	Seq-RL	66.97	91.33	60.81	73.04	1.47	65.42	63.00	77.69	78.54	94.70	75.87
Qwen-3B	SST	77.94	94.12	64.32	78.79	0.77	65.47	63.61	77.15	78.13	94.50	75.77
Llama-3B	Base	20.99	18.43	49.59	29.67	–	62.12	67.48	73.90	73.96	94.20	74.33
Llama-3B	Seq-RL	71.93	91.76	59.81	74.50	1.47	61.84	67.32	73.23	73.71	94.30	74.08
Llama-3B	SST	77.94	92.55	62.81	77.77	1.27	61.95	68.54	73.50	72.56	94.30	74.17

RQ2: How does SST compare with standard forgetting-mitigation baselines? Replay and regularization preserve prior behavior by anchoring updates to earlier examples or model states. SST instead maps new examples into a reusable skill-schema space before updating, encouraging the model to relate new domains to previously acquired procedures. Figure 5 reports final accuracy after the four-domain MATH stream. Across both backbones, SST matches or improves over replay and regularization on streamed domains, with the largest advantage on held-out related MATH domains, where preserving old outputs is insufficient and transferring reusable procedures is required. This suggests

schema transport does more than constrain model drift: by organizing updates around reusable skills, SST yields a favorable retention-transfer trade-off.

RQ3: Can forgetting be avoided simply by underfitting the update? A simpler explanation for improved retention is weaker optimization: undertrained updates may overwrite earlier domains less. To test this, we compare converged and underfit sequential SFT on Qwen-3B over the four-domain MATH stream. As shown in Figure 6, underfit SFT preserves some earlier-domain accuracy but at the cost of Intermediate Algebra performance and held-out MATH generalization (Number Theory, Prealgebra, Precalculus), a retention, plasticity trade-off rather than a solution to continual adaptation. SST targets a different mechanism: by conditioning updates on reusable skill schemas, it preserves prior procedures while still fitting new examples.

RQ4: Do later updates overwrite prior skills, or reinforce shared procedures? Final-checkpoint accuracy captures the net effect of the stream but not how skills evolve during adaptation. We evaluate each SST checkpoint on all streamed MATH domains after every update, testing whether shared procedural structure allows later updates to recover or reinforce earlier skills. For each prior domain $d$, max-drop forgetting is $F(d) = \max_{t_d \le t \le T} A_t(d) - A_T(d)$.

Figure 7 shows that SST keeps forgetting bounded: Qwen-3B drops by 8.68, 8.02, and 4.59 points on Algebra, Counting & Probability, and Geometry (avg. 7.10 pts), Llama-3B drops by 7.51, 4.04, and 0.71 points on Algebra, Counting & Probability, and Geometry, respectively, with an average max-drop of 4.09 points. Crucially, trajectories are not purely monotonic: Qwen-3B’s Algebra accuracy partially recovers from 37.15% to 40.27% by the final checkpoint, consistent with later examples activating schemas shared with earlier domains. These trajectories are consistent with SST mitigating forgetting not by freezing prior behavior, but by allowing skills to be reused and sometimes reinforced as the stream grows.

RQ5: Is SST robust to the order in which domains arrive? Streaming adaptation is inherently order-sensitive: a method may appear effective only because the domains are presented in a favorable sequence. We therefore test SST on four permutations of the same MATH domains using the same training budget and evaluation protocol, and evaluate the final checkpoint on both streamed and

held-out domains. Figure 8 shows that SST yields a broadly consistent performance profile across orders. Although some variation remains, especially on Algebra and Number Theory, no tested permutation causes a collapse on streamed domains, and held-out domains such as Prealgebra remain comparatively strong. This finite-order stress test does not prove full order invariance, but it reduces the concern that SST’s retention-transfer trade-off is driven by a single hand-picked curriculum. Instead, schema transport appears to provide a stable procedural interface for integrating new domains as they arrive.

Conclusion

We proposed Skill Schema Transport (SST), a post-training framework that mitigates forgetting by expressing new-domain examples through reusable procedural schemas rather than constraining model drift. Across streaming math and agent-task settings, SST improves retention and related-domain transfer over sequential fine-tuning and standard forgetting-mitigation baselines. Further analyses reduce the likelihood that the gains are explained solely by underfitting or a favorable stream order, and checkpoint trajectories are consistent with the view that later domains can reuse and sometimes reinforce shared procedures. These results suggest that continual adaptation can be improved not only by limiting model change, but by representing new knowledge in procedural coordinates the model already understands.

Acknowledgments and Disclosure of Funding

This research was partially supported by the National Science Foundation (NSF) under Award No. IIS-2551752 and a Cisco Faculty Award.

Single-Domain Gain-Drop Diagnostic

The main experiments evaluate SST in streaming settings, where updates arrive sequentially and later domains may interact with earlier ones. To isolate the effect of schema conditioning from stream order and cumulative checkpoint dynamics, we additionally run a non-streaming single-domain diagnostic. For each target domain $d \in\{\text{Algebra}, \text{Counting \& Probability}, \text{Geometry}, \text{Intermediate Algebra}\}$, we initialize from the same base model and train only on examples from $d$. We compare raw full fine-tuning against skill-conditioned full fine-tuning using the same target-domain data and training budget. All evaluations in this diagnostic are retrieval-free at inference time; skill schemas are used only as training-time context.

Let $A_0(e)$ denote the base-model accuracy on evaluation domain $e$, and let $A_d^m(e)$ denote the accuracy after training on target domain $d$ with method $m \in\{\text{RAW}, \text{SKILL}\}$. We measure the target-domain gain and cross-domain drop as

where $\mathcal{E}$ is the set of streamed MATH evaluation domains and $[\cdot]_+$ counts only performance decreases. In Figure 9, gains are plotted upward and drops are plotted downward as $-\operatorname{Drop}$.

Figure 9 shows that schema conditioning improves the gain-drop trade-off even in isolated single-domain updates. Starting from the same base checkpoint, raw fine-tuning often provides weaker target-domain gains and/or larger drops on the remaining streamed MATH domains. In contrast, skill-conditioned fine-tuning achieves stronger or comparable in-domain gains with smaller cross-domain degradation. This complements the streaming results by showing that SST’s advantage is not only a consequence of cumulative sequential adaptation, but also appears at the level of individual domain updates.

Additional Details for the Linearized SST Analysis

A Linearized Drift Bound

We give a local linearized argument for why schema conditioning can reduce retention drift. The result concerns minimum shared-parameter displacement in a first-order logit-fitting problem, not global convergence. Assume the model decomposes into shared parameters $\theta_{\mathrm{sh}}$ and schema-specific parameters $\theta_{\mathrm{sc}}$, with logits

Under the following assumption, retention diagnostics evaluated with $s_0$ depend only on $\theta_{\mathrm{sh}}$ to first order.

At step $t$, let $\bar{\theta}=\theta_{t-1}$ and let $r\in\mathbb{R}^{m}$ be the desired first-order logit change for a mini-batch from $D_t$. The linearized updates are

where $A$, $S$ are Jacobians w.r.t. shared and schema-specific parameters. Let $P_A=AA^\dagger$, $r_A=P_Ar$, $\widetilde{S}=P_AS$, so that $r_A$ is the component of $r$ fitable by shared parameters and $\widetilde{S}$ captures how schema-specific directions can offload residual from $\theta_{\mathrm{sh}}$.

Define the minimum shared-parameter displacement under each objective as

Since retention diagnostics under $s_0$ depend only on $\theta_{\mathrm{sh}}$, $\delta_{\mathrm{sc}}$ is free in $\rho_{\mathrm{SST}}^2$.

Proof. If $A=0$, then $r_A=0$ and the claim is trivial. Otherwise, let $A=U\Sigma V^\top$ be a compact SVD. Then $M=(AA^\top)^\dagger=U\Sigma^{-2}U^\top$. Hence $\langle u,v\rangle_M=u^\top Mv$ is an inner product on $\operatorname{col}(A)$: for every nonzero $u=Uw\in\operatorname{col}(A)$, we have $u^\top Mu=w^\top\Sigma^{-2}w>0$.

For any $b\in\operatorname{col}(A)$, the minimum-norm solution of $A\delta_{\mathrm{sh}}=b$ is $A^\dagger b$. Therefore

The identity $(A^\dagger)^\top A^\dagger=(AA^\top)^\dagger$ follows directly from the same SVD. Taking $b=r_A\in\operatorname{col}(A)$ gives $\rho_{\mathrm{SFT}}^2=\|r_A\|_M^2$.

For SST, write $s=\widetilde{S}\delta_{\mathrm{sc}}\in\widetilde{C}$. For any fixed $s$, the remaining residual assigned to shared parameters is $r_A-s\in\operatorname{col}(A)$, and the minimum shared-parameter norm is $\|r_A-s\|_M^2$ by the previous paragraph. Minimizing over $s\in\widetilde{C}$ gives

Thus $\rho_{\mathrm{SST}}^2$ is the squared $M$-distance from $r_A$ to $\widetilde{C}$. The $M$-orthogonal projection onto $\widetilde{C}$ is

Therefore,

This proves (14). Since $0\in\widetilde{C}$,

The inequality is strict exactly when $P_{\widetilde{C}}^M r_A\ne0$, which is equivalent to $r_A$ not being $M$-orthogonal to $\widetilde{C}$, i.e., $\widetilde{S}^\top Mr_A\ne0$.

Proof of Corollary 1

By Assumption 1 and the local schema-separable decomposition, $G_{\bar{\theta}}(\cdot; Q)$ depends locally only on $\delta_{\mathrm{sh}}$. Since $G_{\bar{\theta}}(\bar{\theta}; Q)=0$, nonnegativity and Lipschitz continuity give

For the minimum-shared-norm SST update, $\lVert\delta_{\mathrm{sh}}^{\mathrm{SST}}\rVert_2=\rho_{\mathrm{SST}}$. Theorem 1 gives $\rho_{\mathrm{SST}}\le\rho_{\mathrm{SFT}}$.

Proof of Corollary 2

By Assumption 1, $\mathcal{L}_Q$ evaluated under $s_0$ depends locally only on $\theta_{\mathrm{sh}}$ and is $K_Q$-Lipschitz. Therefore

which gives

For the minimum-shared-norm SST update, $\lVert\delta_{\mathrm{sh}}^{\mathrm{SST}}\rVert_2=\rho_{\mathrm{SST}}$. Theorem 1 gives $\rho_{\mathrm{SST}}\le\rho_{\mathrm{SFT}}$.

Exact interpolation versus projected fitable component

The main theorem compares the shared-parameter displacement needed for the component $r_A=P_A r$ that is fitable by the shared Jacobian. This is the appropriate comparison when the linearized system is interpreted in least-squares or projected-logit form. If instead one requires exact interpolation of the full residual $r$, the feasible sets should be written explicitly.

For exact SFT interpolation,

is finite if and only if $r\in\operatorname{col}(A)$. For exact SST interpolation,

is finite if and only if $r\in\operatorname{col}(A)+\operatorname{col}(S)$. Whenever this exact SST problem is feasible, it can be written as

In the special case where $r\in\operatorname{col}(A)$, exact SFT is feasible and the exact SST expression reduces to

Thus exact interpolation also cannot increase the minimum required shared-parameter displacement. The projected theorem in the main text is slightly more permissive because it studies the shared-fitable component $r_A$ and the projected schema directions $P_AS$; the exact full-residual theorem instead uses the intersection $\operatorname{col}(S)\cap\operatorname{col}(A)$ when $r\in\operatorname{col}(A)$.

Counterexample to the ordinary Euclidean-projection claim

The following example shows why the ordinary Euclidean projection $P_S$ onto $\operatorname{col}(S)$ is not sufficient for the theorem.

Let

and choose a full-row-rank matrix $A$ such that $AA^{\mathsf{T}} = M^{-1}$, for example

Let

The ordinary Euclidean projection of $r$ onto $\operatorname{col}(S)$ is

However,

Therefore $P_{\operatorname{col}(S)}^M r = 0$ and there is no strict improvement:

Moreover, fitting the ordinary Euclidean residual $(I-P_S)r=(0,1)^{\mathsf{T}}$ through $A$ would require squared shared norm

which is larger than the SFT cost 1. Hence the statement

is false in general. It becomes valid if $AA^{\mathsf{T}}$ is proportional to the identity, or if $P_S$ is replaced by the $M$-orthogonal projection.

What the theorem does and does not prove

The theorem proves a statement about a local first-order interpolation problem: after linearizing the logits around the pre-update parameter $\bar{\theta}$, adding schema-specific tangent directions cannot increase the minimum amount of movement required in shared parameters to fit the same shared-fitable residual component. The theorem does not, by itself, prove that arbitrary nonconvex SGD will find the minimum-shared-norm solution, nor that every empirical forgetting metric, including accuracy max-drop, must decrease without the stated null-schema and Lipschitz assumptions. Theory-related claims should therefore be phrased as local upper bounds under explicit assumptions, while the actual retention behavior should be validated empirically.

Additional Details for RQ1

Forgetting metric. For each streamed domain $d$, let $t_d$ be the update step at which $d$ is introduced and let $T$ be the final update step. We compute max-drop forgetting as

where $A_t(d)$ is the evaluation accuracy on domain $d$ after update step $t$. We average $F(d)$ over prior streamed domains only. The final domain is excluded because it is not followed by a later update and therefore cannot be overwritten within the observed stream.

Category	Setting	Value
SFT hyperparameters
SFT	Training framework	LlamaFactory
SFT	Maximum sequence length	4096
SFT	Precision	bf16
SFT	Scheduler	cosine
SFT	Warmup ratio	0.03
SFT	Per-device train batch size	2
SFT	Gradient accumulation steps	8
SFT	Effective batch size	16
SFT	Training epochs	5
SFT	Full fine-tuning learning rate	$1 \times10^{-5}$
SFT	Retrieved skills $k$	2
Evaluation settings
Eval	Decoding	deterministic
Eval	Temperature	0.0
Eval	Top-p	1.0
Eval	Maximum new tokens	512
Eval	Inference engine	vLLM
Eval	Tensor parallel size	1
Eval	GPU memory utilization	0.8
Eval	Maximum number of sequences	64

Implementation Details

Sequential MATH adaptation. For the sequential skill-conditioned SFT setting, we train on four MATH domains in order: Algebra, Counting & Probability, Geometry, and Intermediate Algebra. After each stage, we evaluate the resulting checkpoint on all seven Hendrycks MATH categories: Algebra, Counting & Probability, Geometry, Intermediate Algebra, Number Theory, Prealgebra, and Precalculus. For SST, we construct skill-augmented training examples by retrieving the top-$k$ skills from the schema memory, with $k = 2$ by default, and use the full skill prompt format described in Appendix § 12. Non-SST baselines use the same training examples and hyperparameters, but without the retrieved skill context.

SFT training configuration. We implement SFT using LlamaFactory [63] and run evaluation with vLLM [29]. Unless otherwise specified, we use bf16 training, cosine learning-rate scheduling, and a warmup ratio of 0.03. The default per-device batch size is 2 with gradient accumulation over 8 steps, giving an effective batch size of 16 on a single GPU. We train for 5 epochs by default. For full fine-tuning, we use a learning rate of $1 \times10^{-5}$;

RL implementation details. For the agent-stream experiments, we instantiate SST-RL as a schema-conditioned offline reinforcement-learning setup. In each agent domain, SST-RL augments the state prompt with the top retrieved skills from the schema memory and then optimizes the policy with a GRPO-style objective. Seq-RL and SST-RL use the same RL objective, reward functions, decoding settings, and training budget; the only difference is that SST-RL conditions the training prompt on retrieved skill schemas. In our implementation, the agent skill prompt uses top-$k$ retrieval with $k = 4$.

Rewards are rule-based and combine action correctness with output-format compliance. The main reward is an exact-action reward: it assigns 1.0 when the generated action exactly matches the reference action and 0.0 otherwise. An auxiliary format reward adds 0.2 when the response follows the required tagged protocol, including the expected <think> and <action> fields. Unless otherwise specified, GRPO training uses learning rate $10^{-6}$, one training epoch, per-device batch size 1, gradient accumulation 4, four sampled completions per prompt, maximum prompt length 4096, maximum completion length 512, temperature 0.9, $\beta= 0.0$, and a DAPO-style loss.

Agent evaluation. Agent evaluation is action-level rather than full episodic environment evaluation. The main metric reported for AlfWorld, WebShop, and Search is exact action-match accuracy under deterministic decoding. For Search, we additionally compute tool-selection accuracy, average query-token F1, and a soft search accuracy that requires both the correct tool and query-token F1 of at least 0.5. All agent results in Table 2 use retrieval-free inference unless explicitly stated otherwise. Thus, the agent-stream experiment tests whether schema-conditioned training improves sequential robustness in agent decision-making, rather than introducing a new RL objective or relying on test-time skill retrieval.

Evaluation configuration. We use deterministic decoding for all evaluations, with temperature 0.0 and top-$p$ = 1.0, and generate up to 512 new tokens per example. For vLLM inference, we use tensor parallel size 1. Typical inference settings use gpu_memory_utilization=0.8 and max_num_seqs=64. All main results reported in Table 2 use retrieval-free inference: the final fine-tuned model is evaluated with the raw solver prompt and without prepended skill blocks. Retrieval-enabled inference is used only as an optional diagnostic ablation, and any such results are reported separately when included.

Compute. All main SFT jobs are run on a single-GPU node, typically using 1×A100 GPU, 8 CPU cores, and 256GB host memory, with a 72-hour wall-clock limit for training jobs. Base-model evaluation jobs use the same single-GPU setup with a shorter time budget. In our local setup, the full Hendrycks MATH test set contains 5,000 examples across seven subject categories.

Replay baseline. Our replay baseline uses fixed-count rehearsal rather than a dynamic reservoir buffer. At stream step $t$, the training set contains all current-domain examples together with a uniformly sampled subset from each previous domain. Unless otherwise specified, we sample 500 examples per previous domain using a fixed random seed and sample without replacement after shuffling. Replay examples are concatenated with current-domain examples without additional weighting. The replay ratio is therefore step-dependent:

Regularization baseline. The regularization baseline is an L2-SP-style parameter anchoring baseline. It loads the original base model as an anchor and adds

to the sequential SFT objective over trainable parameters. Unless otherwise specified, we use $\lambda= 10^{-4}$ and the same optimizer, sequence length, batch size, learning-rate schedule, and stream order as the sequential SFT baseline.

Error-clustering diagnostic. We construct Figure 3 by first evaluating the base model on MATH examples and retaining examples answered incorrectly under the same answer-normalization protocol used in Section 5. Each failed example is represented by the problem text only (i.e., excluding both the gold solution and the model-generated reasoning trace), embedded with the sentence embedding model BAAI/bge-small-en-v1.5, and projected to two dimensions with PCA using the top two principal components. Procedural error labels are assigned by manual annotation using the categories ALGEBRAIC MANIPULATION, CASE OMISSION / COUNTING BREAKDOWN, GEOMETRIC RELATION MISUSE, MODULAR / DIVISIBILITY REASONING, COORDINATE-MODELING ERROR, CONSTRAINT / INTERVAL CHECKING, and ANSWER-FORMAT / FINALIZATION. Domain labels are the original MATH subject categories. To avoid circularity, the procedural labels are used only for post-hoc visualization and are not used as supervision, retrieval targets, memory entries, or training signals for SST. This diagnostic is intended only as motivating evidence that failure modes exhibit cross-domain procedural overlap, rather than as a quantitative claim about a complete latent error taxonomy.

Prompting Details

Overview. We use three prompt types in the SST pipeline: a raw solver prompt for base and retrieval-free evaluation, a skill-conditioned solver prompt for SST training, and a memory-update prompt for maintaining the skill memory. Unless explicitly stated otherwise, all reported main results use the raw solver prompt at inference time without retrieved skill blocks. In all skill-conditioned SFT examples, retrieved skills are provided only as input context; the supervision target remains the gold solution.

Raw solver prompt

Solve the following competition math problem.  Show concise reasoning and
final answer.
Type:  {problem_type}
Level:  {level}
Problem:
{problem}

Skill-conditioned training prompt. For SST, we prepend the top-$k$ retrieved skill blocks to the problem input, with $k = 2$ by default. The training target is the gold solution only; the model is not trained to reproduce the retrieved skills. If no skill is retrieved, the prompt includes a short fallback line indicating that the model should solve directly.

Skill-conditioned solver prompt

Solve the following competition math problem.  Use the retrieved skills when
they fit the problem.  Show concise reasoning and final answer.
{retrieved_skill_blocks}
Type:  {problem_type}
Level:  {level}
Problem:
{problem}

Fallback when no skill is retrieved

Relevant skills:
- None retrieved.  Solve directly.

Retrieved skill block format. Each retrieved skill is represented as a compact procedural description. The default prompt uses the full skill format below.

Retrieved skill block

Relevant skill {rank}:
- Skill ID: {skill_id}
- Family: {skill_family}
- Goal: {goal}
- When to use:  {applicability}
- Steps:  {canonical_steps}
- Common failure modes:  {failure_modes}

Memory-update prompt. The memory-update prompt is used during bootstrapping and online memory maintenance. Given the current problem, the model output, and the retrieved skills, the model selects one memory operation from REUSE, REVISE, CREATE, MERGE, and DROP. We require the output to be a single JSON object with no markdown or extra text. Reuse and drop target an existing skill; revise, create, and merge additionally provide a proposed skill entry.

Memory-update prompt, condensed

You are maintaining a model-relative skill memory for competition math.
Given the current problem, the model’s reasoning and final answer, and the
retrieved skills, decide whether to reuse, revise, create, merge, or drop a
skill.
Rules:
- Prefer reuse over create.
- Prefer revise over create if the reasoning pattern is close.
- Create a new skill only when no existing skill covers the pattern.
- Use only the provided skill family IDs.
- Reuse, revise, merge, and drop must name exact existing skill IDs.
- Do not invent missing families, target IDs, or merge sources.
- Return exactly one JSON object with no markdown fences and no extra text.
Return fields:
{action, target_skill_id, proposed_skill, reason, confidence}
Families:
{families_json}
Problem:
{problem}
Model output:
{prediction}
Retrieved skills:
{retrieved_skills_json}

Chat template. Before vLLM inference, each text prompt is wrapped as a single user message and passed through the model tokenizer’s chat template with generation enabled. This keeps the prompting format consistent across base-model evaluation, non-retrieval evaluation, and retrieval-conditioned evaluation.

Limitations

First, our empirical scope covers only two 3B backbones and two streaming settings; whether the same retention–transfer behavior holds for larger models, longer streams, multilingual or multimodal settings, or domains with weak prior procedural representation remains open.

Second, SST depends on schema extraction and retrieval quality. Incomplete, generic, or mismatched schemas provide weak or misleading conditioning, and inaccurate memory updates may fragment related skills or merge distinct ones. More systematic stress tests on noisy, ambiguous, or adversarial inputs are needed.

Third, the theoretical analysis is a local explanatory argument, not a global guarantee. The linearized result shows schema-conditioned directions need not increase shared-parameter displacement, but this does not guarantee nonconvex optimization finds such solutions in practice.

Fourth, we do not report full multi-seed statistics with confidence intervals for all comparisons. Future work should include repeated runs and larger-scale ablations over retrieval thresholds, memory-update rules, and prompt formats.

Fifth, SST introduces additional overhead for schema extraction, retrieval, and memory updates. Scaling behavior for very large streams or high-throughput settings remains unmeasured.

Finally, SST reduces but does not eliminate forgetting, and should be viewed as improving the retention–plasticity trade-off rather than solving continual learning.

Broader Impact

SST may benefit applications requiring stable sequential model updates by reducing unnecessary overwriting of prior capabilities without retaining large amounts of old data. The explicit schema representation also offers a procedural interface for inspecting which reasoning patterns are reused or revised during training, which may assist in diagnosing failures and understanding how updates affect model behavior.

At the same time, more effective continual adaptation could amplify both beneficial and harmful uses, including harmful automation or unsafe agent behavior if deployed without safeguards. Learned schemas may also preserve and transfer biases or unsafe procedures encoded in training data, and should be evaluated not only for accuracy and forgetting but also for fairness, robustness, and safety in the intended deployment context.

Users should not over-interpret the explicit schema structure as a formal guarantee of model behavior: schemas are conditioning context for training, not certificates of internal reasoning. Any deployment should include independent evaluation, red-teaming, and monitoring for regressions. Future releases of code or schema memories should document data sources, licenses, intended use, and known limitations.