A Fisher-Geometric View of Multi-Teacher On-Policy Distillation

Introduction

Post-training of large language models (LLMs) increasingly produces homologous domain specialists—fine-tuned from a common base via reinforcement learning (RL) on distinct reasoning modalities ([1, 26, 7, 29, 8, 17]). Serving an ensemble of specialists multiplies memory and latency, making consolidation into a single generalist essential.

Two paradigms are in common use, each with its own shortcomings. Weight-space merging ([13, 31, 20, 34] (Yadav et al. 2024)) adds to the base model a weighted sum of the specialists’ weight updates (their task vectors) at zero training cost, but the weights are set by hand and, once the merge is formed, nothing tells which domain it failed to recover or whether the chosen weights still combine the specialists faithfully. Multi-teacher on-policy distillation ([1, 8, 28, 35, 4]) (MOPD) instead trains a student to match each specialist (its teacher) on the student’s own generations, which repairs what merging misses, but at the cost of rollouts on every domain and with no control over which domains its shared-weight gradients overwrite. Neither paradigm can answer the question the other one raises: what a merge has already recovered, and where distillation is still needed without harming what is already correct. Answering it requires relating the two, which is what this paper does.

Our starting point is that these paradigms optimize the same objective at different orders of approximation (§ 2). Expanded to second order around the specialists in a shared basin, the distillation objective is exactly the Fisher-merging objective of Matena & Raffel [21], whose minimizer is a weighted average of the specialists with Fisher-information weights; merging is therefore not an alternative to distillation but is optimal, to second order, as its closed-form stage. A full-parameter Fisher merge is intractable on foundation models, so we restrict the merge to the task-vector subspace (Phase 1, described next). Extending tangent-space analyses of merging, we show that, to first order, the same coefficients that mix the task vectors also mix the specialists’ implicit rewards (the log-probability ratios between each specialist and the base), which lets us obtain them by minimizing the worst reward-space residual on unlabelled prompts. After solving, reward-energy normalization measures the relative share left unrecovered in each domain. The fidelity of this first-order fit provides an intrinsic, label-free test of whether a checkpoint still lies where the expansion holds.

Grounded in this framework, we propose Residual-Governed Branch-and-Merge (RGBM) as shown in Figure 1. In Phase 1, RGBM determines the combination weights by minimizing the worst per-domain residual via a small convex program on unlabelled prompts, yielding the zero-training merged model. It then normalizes the residuals by their domain reward energies to form the domain residual spectrum. This spectrum directly dictates what, if anything, should follow the merge. Standard joint distillation tends to degrade the merged model because gradients from domains with low normalized residuals corrupt shared weights. To prevent this interference, Phase 2 leaves those domains untouched and trains isolated branches only for domains above a closed-form spectral gap. Each admitted branch update is then written back, scaled by the largest factor that passes the linearity test.

Empirically, we validate our framework across two multi-teacher consolidation benchmarks: Open-MOPD (Gao et al., [8]) (SmolLM3-3B with Math, Code, and Instruction Following specialists) and G-OPD (Yang et al., [35]) (Qwen3-4B with Math and Code specialists). On Open-MOPD, the zero-training Phase 1 merge alone achieves a Macro average of 32.52, outperforming the official 600-step distilled student (30.93) without requiring any rollout compute. By passivating domains with low normalized residuals and training an isolated branch solely for Math, Phase 2 further elevates the Macro score to 33.03 with zero joint-training steps. Crucially, un-gated baselines such as TIES (Yadav et al., [31]), DARE (Yu et al., 2024), and continued joint distillation all fall below the merge, confirming that their updates can corrupt shared parameters. The same procedure transfers seamlessly to G-OPD, where RGBM surpasses the strongest trained baseline, demonstrating the generality of residual-governed consolidation.

Contributions. (1) Theoretical Bridge: we establish a connection between multi-teacher on-policy distillation and model merging, showing that the MOPD objective coincides to second order with Fisher merging ([21]), and derive an implicit-reward homomorphism that yields a label-free linear basin diagnostic. (2) Two-Phase RGBM Framework: we propose Residual-Governed Branch-and-Merge (RGBM). Phase 1 computes the zero-training merge from a residual program on unlabelled prompts and forms an energy-normalized spectrum for cross-domain comparison. Phase 2 passivates domains below the spectral gap, trains isolated branches exclusively for domains above it, and writes admitted updates back via calibrated shrinkage. (3) Empirical Validation and Protection Mechanism: across two model families (SmolLM3-3B and Qwen3-4B), Phase 1 alone matches or beats fully trained distillation, and Phase 2 advances performance further with zero joint-training steps. Crucially, un-gated write-backs fall below the merge, confirming that RGBM succeeds by shielding domains below the normalized spectral gap from cross-task interference.

Merging Is Optimal, to Second Order, as a Closed-Form Stage of Distillation

This section establishes the identification on which the paper rests: to second order, distillation and Fisher merging optimize the same objective (§ 2.2); on the task-vector subspace that objective has a closed-form minimizer (§ 2.3); and, to first order in the tangent space, combining parameters adds implicit rewards, which yields a validity test (§ 2.4). Proofs are in Appendix § 9.

Setting

Let $W_{\mathrm{ref}} \in\mathbb{R}^{P}$ denote the base weights ($P$ parameters) and $\pi_{\theta}(\cdot\mid x)$ the autoregressive policy of a model with weights $\theta\in\mathbb{R}^{P}$ on prompt $x$, with $y$ a completion. We consider $K$ homologous specialists $\{W_{\phi_1}, \ldots, W_{\phi_K}\}$, fine-tuned from the same $W_{\mathrm{ref}}$ on datasets $D_d$ via RL on each domain, with policies $\pi_{W_{\phi_d}}$. The task vector of specialist $d$ is $\tau_d := W_{\phi_d} - W_{\mathrm{ref}}$, forming $\mathcal{T} := [\tau_1, \ldots, \tau_K] \in\mathbb{R}^{P \times K}$. Multi-teacher on-policy distillation (MOPD) trains a student $\theta$ by minimizing the domain-weighted reverse-KL to the specialist policies, with mixture weights $p_d$:

MOPD Is Fisher Merging to Second Order

The proof (Appendix § 9.§ 1) relies only on the fact that at $\theta=W_{\phi_d}$, the KL divergence and its gradient vanish while its Hessian equals $F_d$. Equation (3) recovers the Fisher-weighted average of [21], originally derived via a Laplace approximation to the posterior. Theorem 1 establishes that this exact point is also the second-order optimum of the multi-teacher reverse-KL objective. The two paradigms therefore differ in approximation order rather than optimization goal: while $J_{\mathrm{MOPD}}$ aligns student rollouts to teachers in distribution space, it corresponds in parameter space to the aggregate Fisher distance $\frac{1}{2}\sum_d p_d\lVert\theta-W_{\phi_d}\rVert_{F_d}^2$ with $\lVert v\rVert_{F_d}^2 := v^\top F_dv$. Merging is thus optimal, to second order, as a closed-form stage of distillation: gradient steps taken from the Fisher merge act only on the third-order remainder, so merging should come first. What a linear combination in the task-vector subspace leaves unrecovered is a per-domain error, which the residual program of § 3.1 measures.

Subspace Fisher Projection and Task Arithmetic

Eq. (3) requires inverting a $P \times P$ Fisher matrix, which is intractable for modern LLMs. We therefore restrict the student to the $K$-dimensional task subspace $\theta(\alpha)=W_{\mathrm{ref}}+\mathcal{T}\alpha$, with one coefficient $\alpha_d$ per specialist. Let $e_d$ be the $d$-th standard basis vector, so $\theta(e_d)=W_{\phi_d}$ and $\theta(\alpha)-W_{\phi_d}=\mathcal{T}(\alpha-e_d)$. That substitution compresses Eq. (2) into a $K \times K$ system:

where $S_d$ is the projected Fisher of specialist $d$. Setting $\nabla_\alpha J_{\mathrm{sub}}=0$ gives the closed-form subspace solution:

Remark 1 Task Arithmetic ([13]) forms $W_{\mathrm{ref}}+\mathcal{T}\alpha$ with hand-set coefficients, typically the isotropic $\alpha=\bar{\alpha}\mathbf{1}$ with $\bar{\alpha}=1$. Eq. (5) identifies the coefficients that make this family the exact metric projection of the quadratic Fisher objective onto the task subspace ($\mathcal{T}^\top\nabla_\theta J_{\mathrm{MOPD}}(\theta(\alpha_{\mathrm{sub}}^\star))=0$); with these coefficients it is the computable closed-form instance of Fisher merging on LLMs.

First-Order Homomorphism and a Validity Diagnostic

The subspace solution still depends on $F_d$. Forming $S_d=\mathcal{T}^\top F_d\mathcal{T}$ from parameters is impractical on LLMs, so the computations of § 3 run in the space of implicit rewards. The token-level implicit reward of specialist $d$ relative to $W_{\mathrm{ref}}$ is $r_d(x,y):=\log\left(\frac{\pi_{W_{\phi_d}}(y\mid x)}{\pi_{W_{\mathrm{ref}}}(y\mid x)}\right)$, the log-probability ratio that RL fine-tuning implicitly optimizes ([24, 2]); write $\mathbf{r}(x,y):=[r_1(x,y),\ldots,r_K(x,y)]^\top\in\mathbb{R}^K$ for the stacked reward. Extending tangent-space analyses of merging ([22]), a first-order expansion around $W_{\mathrm{ref}}$ yields the following homomorphism.

Proposition 2 provides two operational capabilities. First, the projected score $\mathcal{T}^\top\nabla_\theta\log\pi_{W_{\phi_d}}$ equals $\mathbf{r}$ up to a mean shift (Appendix § 9.§ 3), so the projected Fisher is an empirical reward covariance, $S_d\approx\operatorname{Cov}_{\pi_{W_{\phi_d}}}(\mathbf{r})$, and the per-domain residual in § 3.1 can be computed from forward passes alone. Second, the log-ratio of the merged model, $\ell_\alpha(x,y):=\log\frac{\pi_{\theta(\alpha)}(y\mid x)}{\pi_{W_{\mathrm{ref}}}(y\mid x)}$, should scale linearly with the superposed reward $\alpha^\top\mathbf{r}(x,y)$. The unexplained variance when regressing $\ell_\alpha$ on $\alpha^\top\mathbf{r}$ over sampled tokens is then a label-free test of whether the first-order superposition of Proposition 2 still holds at that checkpoint:

We treat $\operatorname{Err}_{\mathrm{lin}}(\theta(\alpha))\le\delta_{\mathrm{tol}}$ under a fixed threshold $\delta_{\mathrm{tol}}$ as the operational criterion for a valid linear basin (we use linear basin for this regime throughout). Large $\operatorname{Err}_{\mathrm{lin}}$ means the first-order fit has broken; § 3.3 uses the same test to cap how far a Phase 2 write-back may move.

RGBM: Residual-Governed Branch-and-Merge

Section 2 establishes that parameter merging should precede distillation, and that whatever the linear combination leaves unrecovered is what a later stage must address. Residual-Governed Branch-and-Merge (RGBM; Figure 2; full procedure in Algorithm 1, Appendix § 11.§ 1) operationalizes this insight into two distinct phases. Phase 1 (§ 3.1) solves a convex subspace merge and then normalizes its per-domain remainders into the domain residual spectrum. After analyzing why continuing joint distillation degrades domains with low normalized residuals (Section 3.2), Phase 2 (§ 3.3) trains isolated branches exclusively for domains marked by the spectrum as unrecovered, writing them back via calibrated shrinkage.

Phase 1: Minimax Subspace Merge and the Domain Residual Spectrum

Per-domain residual from unlabelled prompts. Under the tangent-space homomorphism of Proposition 2 the projected score equals $r$ up to a mean shift, so $S_d \approx\operatorname{Cov}_{\pi_{W\phi_d}}(r)$ can be estimated without gradients. Let $\mathcal{D}_{\mathrm{calib},d}$ be unlabelled prompts from domain $d$. Rather than center the rewards, we use the empirical second-moment $M_d := \mathbb{E}_{x\sim\mathcal{D}_{\mathrm{calib},d},\,y\sim\pi_{W\phi_d}}[rr^\top] \in\mathbb{R}^{K\times K}$, assembled from forward passes of the base and the $K$ specialists on completions sampled from specialist $d$. $M_d$ equals that covariance plus a mean-shift outer product, and therefore upper-bounds the projected Fisher residual (Appendix § 9.§ 3). We define the per-domain residual and its energy-normalized counterpart as

Phase 1 optimizes $\mathcal{L}_d$. After the merge is fixed, division by $\mathbb{E}[r_d^2]$, the second moment of the specialist’s own reward (its reward energy), expresses the remainder as a fraction of domain-$d$ energy. Thus $\overline{\mathcal{L}}_d$ is comparable across domains and is used for diagnosis and gating, but not for solving the merge. Because $M_d$ contains the cross-domain products $\mathbb{E}[r_jr_k]$, $\overline{\mathcal{L}}_d(\alpha^\star)$ is the fraction of domain-$d$ reward energy still unexplained after the coefficients have traded the domains off against one another. A value near zero means this joint combination already accounts for domain $d$. It does not mean an isolated branch on $d$ has nothing left to fit. A large value means a substantial share of that energy remains once the other domains have been accommodated. § 4.3 measures the merges obtained when the same programs are solved on $\overline{\mathcal{L}}_d$.

Minimax merge. Eq. (5) requires domain weights $p_d$, which are rarely known and trade one domain against another, so the Fisher stage of § 2.3 is not directly computable. We remove this degree of freedom by taking the worst case over all admissible weights, the probability simplex $\Delta_K$: since $\max_{p\in\Delta_K}\sum_d p_d\mathcal{L}_d(\alpha)=\max_d\mathcal{L}_d(\alpha)$, the problem becomes the convex Minimax quadratic program (QP)

i.e., the worst subspace residual under adversarially chosen domain weights. Its solution gives the zero-training checkpoint $\theta_0^\star:=W_{\mathrm{ref}}+T\alpha^\star$. We then report the Domain Residual Spectrum $\bar{\mathcal{L}}:=[\mathcal{L}_1(\alpha^\star),\ldots,\mathcal{L}_K(\alpha^\star)]$ for cross-domain comparison and Phase 2 allocation. The QP is $K$-dimensional, so its cost is dominated by assembling the $M_d$ (Appendix § 11.§ 6). For numerical stability, the QP replaces $M_d$ by $\widetilde{M}_d=M_d+\lambda I$; the reported spectrum uses the unregularized $M_d$. We leave $\alpha$ unconstrained, so a coefficient may lie outside $[0,1]$ if that reduces the worst residual.

Why Not Simply Keep Distilling from the Merge?

The quadratic model of Theorem 1 is stationary at the Fisher merge, and approximately stationary on the subspace at $\theta(\alpha^\star_{\mathrm{sub}})$. Phase 1’s $\theta_0^\star$ is not that point: it minimizes the worst residual, so $\nabla J_{\mathrm{MOPD}}(\theta_0^\star)$ need not vanish, and continued joint distillation is not a refinement of a solved linear stage. Two mechanisms outside the quadratic regime then make it harmful. Each domain divergence $f_d(\theta):=\mathbb{E}_{x\sim\mathcal{D}_d}[\mathrm{KL}(\pi_\theta\mathbin{\|}\pi_{W_{\phi_d}})]$ has $\nabla_\theta f_d(\theta_0^\star)\ne0$, so mini-batch steps on shared parameters pull in conflicting directions, including components off the task subspace. A near-zero normalized residual $\bar{\mathcal{L}}_d$ adds a second problem, and it is a joint one. An isolated branch can still reduce domain $d$’s own loss, because $\mathcal{L}_d$ is the gap left after the cross-domain tradeoff, not a claim that domain $d$ is saturated. Writing that branch into shared weights moves every domain: the step is spent where the joint merge has already closed the gap, so the three-domain loss can worsen even while domain $d$ itself improves. Joint distillation from $\theta_0^\star$ should therefore degrade those low-residual domains on the shared model. A token-level KL penalty to $\pi_{\theta_0^\star}$ cannot prevent this, because the collision is in the weights. § 4.4 checks both predictions.

Phase 2: Residual Gate, Isolated Branches, and Shrinkage Re-merge

RGBM therefore never applies a joint gradient to the merge; it partitions domains by the normalized spectrum $\bar{\mathcal{L}}$ and treats the two groups differently.

Gate. Sort $\bar{\mathcal{L}}_{(1)}\ge\cdots\ge\bar{\mathcal{L}}_{(K)}$, append a floor $\bar{\mathcal{L}}_{(K+1)}:=0$, and cut at the midpoint of the largest adjacent gap,

Domain $d$ is active iff $\bar{\mathcal{L}}_d>\epsilon$, otherwise passivated; the floor makes every domain active when normalized residuals cluster, and any $\epsilon$ inside the same gap yields the same allocation (Appendix § 11.§ 4).

Passivated and active domains. Passivated domains ($\bar{\mathcal{L}}_d\le\epsilon$) receive no shared write-back ($\Delta_d=0$). The joint merge already accounts for them, which does not preclude an isolated gain on that domain alone. Each active domain trains an isolated branch $\theta_d\leftarrow\theta_0^\star$ on its native data $\mathcal{D}_d$ with no gradient sharing, giving an update $\Delta_d=\theta_d^{(\mathrm{end})}-\theta_0^\star$; isolation guarantees that no gradient computed on domain $d$ touches a passivated domain during training. Whether the update can be written back without damaging them is decided next.

Shrinkage re-merge. The sole Phase 2 write-back is a shrunken sum of the active updates,

where $\Gamma\subset(0,1]$ is a fixed discrete grid and $\operatorname{Err}_{\mathrm{in}}$ is evaluated on the same unlabelled calibration prompts $\mathcal{D}_{\mathrm{calib}}=\bigcup_d\mathcal{D}_{\mathrm{calib},d}$ used for $M_d$. $\gamma_d^\star$ is the largest grid value at which that single write-back still passes the linearity test in Proposition 4 (Appendix § 9.§ 4, so the write-back keeps the merged model’s log-ratio within $\delta_{\mathrm{tol}}$ of the Phase 1 superposition $\alpha^\star{}^\mathsf{T}r$, which is held fixed as the predictor; Proposition 4 (Appendix § 9.§ 4) bounds the first-order remainder at $\theta_0^\star+\gamma\Delta_d$ by a quantity that is non-decreasing in $\gamma$. When several branches are active, evaluation uses their shrunken sum. If the gate leaves no active domain, $\theta_{\mathrm{RGBM}}=\theta_0^\star$. The whole procedure has three fixed constants $(\lambda,\Gamma,\delta_{\mathrm{tol}})$ and no labelled data; their values and the resulting allocations on both suites are given in § 4.1.

Experiments

Setup

Main suite (Open-MOPD). Our primary testbed is the Open-MOPD specialist suite [8]: SmolLM3-3B [3] with $K = 3$ homologous RL specialists (the usual MOPD domain count; [8, 28, 4]), fine-tuned from the mixed-SFT checkpoint MixSFT as $W_{\mathrm{ref}}$. The specialists are Math, Code, and Instruction Following (IF), scored on AIME 2024/2025, LiveCodeBench v5/v6 [14], and IFEval/IFBench [23] (Zhou et al. 2023). We report Macro $= \frac{1}{3}(\mathrm{Math} + \mathrm{Code} + \mathrm{IF})$, where IF is the mean of IFEval and IFBench, and re-evaluate every system under one protocol. The italic RouteRL row is the routing oracle of [8] that serves all $K$ specialists; Multi-Teacher OPD is their released 600-step student. Every Phase 2 system starts from the same Phase 1 merge $\theta_0^\star$. TIES [31] and DARE (Yu et al., 2024) sparsify and sum the isolated on-policy distillation (OPD) branch updates $\Delta_d$—not the specialist task vectors—and RGBM uses those same branches, differing only in what is written back (Appendix Table 8). Per-benchmark numbers and hyperparameters are in Appendices § 10.§ 2 and § 11.§ 5.

Secondary suite (G-OPD). To test transfer across model families we apply the same procedure, unchanged, to the G-OPD suite [35]: Qwen3-4B [30] with Math and Code specialists (intermediate checkpoints of Table 2 in Yang et al., [35]; 500/300 steps) on their 7 official benchmarks. Supervised fine-tuning (SFT), OPD, and the extended recipe ExOPD are from Yang et al. [35]; merges are evaluated by us under the official protocol. The italic Domain-Matched Teacher row is that specialist pair, an oracle rather than a single generalist. Per-benchmark results and the fully trained pair are in Appendix § 10.§ 7.

Program	$\alpha$	Math	Code	IF	Macro
$\alpha= 1/3$	0.333/0.333/0.333	20.81	19.15	44.77	28.24
A scalar $\mathcal{L}$	0.626/0.626/0.626	24.97	23.28	47.64	31.96
A vector $\mathcal{L}$	0.527/0.609/0.821	24.11	23.59	47.99	31.90
C scalar $\mathcal{L}$	0.579/0.579/0.579	24.97	23.49	45.10	31.19
C vector $\mathcal{L}$ ($\theta_0^*$)	0.597/0.620/0.800	23.86	24.09	49.60	32.52
A scalar $\overline{\mathcal{L}}$	0.501/0.501/0.501	24.38	23.36	44.86	30.86
A vector $\overline{\mathcal{L}}$	0.840/0.260/0.841	24.19	19.57	48.00	30.59
C scalar $\overline{\mathcal{L}}$	0.477/0.477/0.477	21.41	19.76	45.79	28.99
C vector $\overline{\mathcal{L}}$	0.724/0.426/0.630	24.22	20.73	47.02	30.66

RGBM constants and resulting allocations. We use $\lambda= 10^{-4}$, $\Gamma= \{0.125, 0.25, 0.5, 0.75\}$, and $\delta_{\mathrm{tol}} = 0.10$ on both suites. On Open-MOPD the energy-normalized spectrum is $\mathcal{L} = (0.128, 0.052, 0.015)$ for (Math, Code, IF); the largest gap is between Math and Code, so $\epsilon= 0.090$, only Math is active, and Eq. (11) selects $\gamma_{\mathrm{math}}^\star= 0.125$, the largest $\gamma\in\Gamma$ with $\mathrm{Err}_{\mathrm{lin}} \le\delta_{\mathrm{tol}}$ ($\mathrm{Err}_{\mathrm{lin}} = 0.0701$; Appendix Table 10). On G-OPD, $\alpha^\star= (0.675, 0.666)$ and $\mathcal{L} = (0.094, 0.090)$ cluster, so the largest gap is down to the floor, $\epsilon= 0.045$, both domains are active, and the same rule gives $\gamma^\star= 0.5$. Hereafter, low- and high-residual domains refer to this normalized spectrum. Assembling the $M_d$ costs $< 15$ minutes on 8 GPUs ($< 2$ GPU·h) of unlabelled scoring and the QP solves in $< 10$ ms.

Merge Before You Distill: Zero-Training Merge vs. Trained Distillation

The closed-form merge matches or beats trained distillation. On Open-MOPD (Table 1) the Minimax merge $\theta_0^\star$ reaches 32.52 Macro at zero gradient cost, above Multi-Teacher OPD, the released 600-step student (30.93; 144 GPU·h), and above isotropic Task Arithmetic (31.31). On G-OPD (Table 3) the same procedure gives 53.34, above SFT, OPD, and Task Arithmetic and within 0.45 of the trained ExOPD recipe (equal-weight averaging collapses to 36.75: the gain is in the coefficients). The 31.90 row is the subspace Fisher solution of Eq. (5) under uniform weights $p_d = 1/K$, i.e., the minimizer of $\sum_d p_d \mathcal{L}_d(\alpha)$ with the Phase 1 second moments $M_d$ in place of $S_d$. Eq. (5) requires the domain weights $p_d$ and those weights are not known; the uniform choice is a default, not an estimate. Minimax, which minimizes the worst residual instead, is 0.62 higher and lifts IF from 47.99 to 49.60—when no domain may be sacrificed, the adversarial weighting is the operational choice.

Method	Math	Code	IF	Macro
RouteRL (Oracle Ensemble)	27.24	26.27	53.52	35.68
Multi-Teacher OPD	24.53	21.18	47.12	30.93
Task Arithmetic	24.35	22.81	46.77	31.31
Subspace Fisher Merge	24.11	23.59	47.99	31.90
RGBM Phase 1 (Ours, $\theta_0^\star$)	23.86	24.09	49.60	32.52
Uniform Distillation (from $\theta_0^\star$)	23.97	21.36	47.42	30.91
TIES (all branches)	24.61	21.56	46.72	31.42
DARE (all branches)	24.58	22.40	47.28	31.42
Gated Math ($\gamma= 1.0$)	25.03	22.51	47.87	31.80
Gated Math ($\gamma= 0.5$)	25.42	23.48	48.08	32.32
RGBM Phase 2 (Ours)	23.93	24.63	50.52	33.03

Method	Math	Code	Avg
Domain-Matched Teacher	46.00	61.17	52.50
SFT	44.33	60.80	51.39
OPD	46.38	60.57	52.46
ExOPD	47.65	61.97	53.79
Task Arithmetic	43.05	60.65	50.59
Equal-Weight Average	22.55	56.57	36.75
RGBM Phase 1 (Ours)	46.17	62.89	53.34
RGBM Phase 2 (Ours)	47.55	62.93	54.14

RGBM improves on the merge on both suites. Writing back only the gated Math branch raises Open-MOPD to 33.03 (+0.51 over $\theta_0^*$) with zero extra joint-training steps; on G-OPD both domains are active, and writing back both isolated branches gives 54.14, above ExOPD.

The Quadratic That Is Minimized

Phase 1 minimizes $\mathcal{L}_d$. This quadratic upper-bounds the domain term $(\alpha-e_d)^\mathsf{T}S_d(\alpha-e_d)$ of $J_{\mathrm{sub}}$ (Eq. (4); Appendix § 9.§ 3), since $M_d$ equals $S_d \simeq\operatorname{Cov}(r)$ plus a mean-shift outer product. The uniform row above is that bound, with $M_d$ in place of $S_d$. The factor $1/\mathbb{E}[r_d^2]$ is applied after $\alpha$ is chosen, which rescales the reported spectrum and leaves the minimizer in place. Solving the same programs on $M_d/\mathbb{E}[r_d^2]$ weights domain $d$ by $p_d/\mathbb{E}[r_d^2]$. On the Open-MOPD calibration the energies are $\mathbb{E}[r_d^2] = (0.180, 0.692, 2.40)$ for (Math, Code, IF), so that program down-weights Code relative to Math by a factor of 3.8. Task arithmetic already treats the coefficient as a control on accuracy and picks it on held-out labels (Ilharco et al., [13]). The solve below uses no labels.

Every program solved on $\mathcal{L}$ lands in Macro 31.19 $\sim$ 32.52. Every program solved on $\overline{\mathcal{L}}$ lands in 28.99 $\sim$ 30.86, so the best normalized merge (30.86) sits below the worst raw merge (31.19). The reweighting shows up on Code. The uniform vector on $\mathcal{L}$ keeps $\alpha_{\mathrm{code}} = 0.609$ and Code at 23.59; the same program on $\overline{\mathcal{L}}$ sets $\alpha_{\mathrm{code}} = 0.260$ and Code falls to 19.57. The published minimax vector keeps $\alpha_{\mathrm{code}} = 0.620$ and Code at 24.09. Equal-weight $\alpha= 1/3$, which solves neither program, is lower still (28.24).

With $M_d$ held fixed, $\mathcal{L}_d(\alpha)$ is a function of the coefficients alone. Figure 3 orders the nine merges by score and overlays that raw residual, min-max scaled onto the score range. On Code, $\alpha_{\mathrm{code}} \ge0.50$ keeps Code at 23.28 $\sim$ 24.09 and $\alpha_{\mathrm{code}} \le0.48$ puts it at 19.15 $\sim$ 20.73 (Pearson +0.89 between $\alpha_{\mathrm{code}}$ and Code). On IF, $\alpha_{\mathrm{if}}$ and IF have Pearson +0.87, from 44.77 at $\alpha_{\mathrm{if}} = 0.333$ to 49.60 at the published $\alpha_{\mathrm{if}} = 0.800$. On Math, $\alpha_{\mathrm{math}}$ separates the floor: the two vectors with $\alpha_{\mathrm{math}} < 0.5$ score 20.81 and 21.41, and every vector with $\alpha_{\mathrm{math}} \ge0.5$ stays in 23.86 $\sim$ 24.97. The coefficients track Code and IF, and they track whether Math has cleared that floor.

Why the Write-Back Must Be Gated: Joint Distillation from the Merge

The obvious second stage is to keep distilling $\theta_0^*$ jointly on all domains; Section 3.2 argued that this should erode the low-residual domains. Figure 4 tracks three joint objectives initialized from $\theta_0^*$: Uniform (equal-weight reverse-KL), Outcome-Gated (matching the teacher only on failed rollouts), and Trust-Region (outcome-gated reverse-KL with a KL penalty $\beta= 2.5$ toward $\pi_{\theta_0^*}$). Appendix § 11.§ 2 extends this analysis to five distinct objectives. (1) Low-residual domains deteriorate. At the last listed checkpoint of each recipe trained to step 20 or 50, Code drops by 1.1 $\sim$ 2.9 points below $\theta_0^*$, IF falls behind the merge, and Macro stays in 30.5 $\sim$ 32.3. Joint distillation therefore undoes what Phase 1 had already recovered. (2) High-residual Math improves only marginally. Uniform and Outcome-Gated trajectory means are 23.97 and 24.69 (Appendix § 11.§ 2), against 23.86 at $\theta_0^*$. Continued joint steps do not convert the Math residual into a gain that offsets the drop on Code and IF. (3) Interference is parameter-level. Even when the Code reverse-KL is held below $10^{-3}$ nat/token, Code still degrades. The student can stay close to the Code teacher in distribution space, so the damage is coming from shared-weight updates, not from a Code-teacher mismatch. (4) A token-level KL penalty cannot prevent collapse. The Trust-Region anchor bounds estimated policy drift to $\operatorname{KL}(\pi_{\theta}\Vert\pi_{\theta_0^*}) \approx1.4 \times10^{-3}$ nat/token, yet Macro decays to 31.25 by step 20 (Code drops 24.09 $\to$ 21.18). A scalar KL only restricts isotropic divergence; it does not block interference along the anisotropic directions captured by $\bar{\mathcal{L}}_d$.

The Residual Spectrum Identifies Which Updates Can Be Written Back Safely

The normalized spectrum $\overline{\mathcal{L}} = (0.128, 0.052, 0.015)$ is computed from the Phase 1 solution before any Phase 2 update, and the gate it induces admits only the Math branch. We test this decision against the alternatives it rejects. Writing back the admitted Math update ($\gamma^* = 0.125$) raises Macro to 33.03. Writing back the rejected IF update at a mid-grid factor ($\theta_0^* + 0.5\,\Delta_{i\mathrm{f}}$; IF has no $\gamma_{\mathrm{if}}^*$) lowers Macro to 31.10 and Code to 20.70 (Appendix § 11.§ 4). Writing back all three, as un-gated TIES and DARE do, gives 30.96 and 31.42, both below the merge. On G-OPD the normalized spectrum clusters, the gate keeps both domains, and writing back both raises the average by 0.80.

We stress what this does and does not show. Isolated branches raise their own domain under all three objectives (Appendix Table 11): IF by +0.48 $\sim$ +1.63, Code by +0.43 $\sim$ +1.31, and Math by +0.58 $\sim$ +1.83. A low residual therefore does not mean a domain has nothing left to learn on its own: $\overline{L}_d$ is the coupled gap at $\alpha^\star$, after all domains have been traded off together. A write-back is one shared update, so it moves domains other than the branch that was trained, and it is scored on Math, Code, and IF together. On that joint score the rejected IF write-back lowers Macro, while the admitted Math write-back raises it only at $\gamma^\star= 0.125$. The spectrum chooses which branch may enter the shared model. The shrinkage grid chooses how far that branch may move it.

Ablations: Gate, Isolation, and Shrinkage

The lower half of Table 1 isolates each RGBM component (full $\gamma$ grid in Appendix § 11.§ 3). Uniform Distillation continues joint training; TIES and DARE write back every isolated branch; Gated Math writes back only the admitted branch, with or without shrinkage. Without the gate, every write-back falls below the merge, whether updates are trained jointly or in isolation, dense or sparsified, because the Code and IF updates damage shared weights. With the gate but $\gamma= 1$, the Math branch alone still costs 0.72 Macro; intermediate $\gamma$ trades Math against Code and IF (Appendix § 11.§ 3). The unlabelled rule selects $\gamma^\star= 0.125$, the only setting above the merge.

Where does the gain come from? The +0.51 Macro of RGBM is not a Math gain: at $\gamma^\star= 0.125$, Math moves only +0.07. It is carried by Code (+0.54) and IF (+0.92), the two passivated domains, which the small Math write-back moves through the shared weights. Protection explains a different comparison: relative to the merge, the un-gated write-backs lose 1.1 $\sim$ 1.6 Macro, and the gate removes that loss. Larger $\gamma$ improves Math (+1.56 at $\gamma= 0.5$) only by sacrificing Code and IF. Across all checkpoints of Appendix § 11.§ 2, the ordering un-gated write-backs < merge < RGBM holds.

Linearity Diagnostics and the Limits of the Theory

Appendix Figure 5 is a sanity check of Proposition 2 on the isotropic family $\theta(\alpha) = W_{\mathrm{ref}} + \alpha\sum_d \tau_d$ of Remark 1, not on the Minimax vector $\alpha^\star$. Regressing the merged log-ratio on the superposed reward over 38M tokens yields $R^2 = 0.935$ at $\alpha= 0.75$. At $\alpha= 1.50$ the fit breaks ($R^2 = 0.557$), the cross-domain interference ratio $I_{\mathrm{cross}}$ (the share of merged log-ratio energy not explained by the in-domain reward) rises to 0.775, and Macro falls to 22.48—a failure $\mathrm{Err}_{\mathrm{lin}}$ flags without labels. Branch updates are not small relative to the merge ($\lVert\Delta_d\rVert/\lVert T\alpha^\star\rVert\approx22\% \sim29\%$), and the unshrunk Math write-back already breaks the fit ($\mathrm{Err}_{\mathrm{lin}} = 0.72$ at $\gamma= 1$); shrinking to $\gamma^\star= 0.125$ brings it back to 0.070 $\leq\delta_{\mathrm{tol}}$ (Appendix Table 10).

Related Work

Parameter-space merging. Training-free fusion spans Task Arithmetic ([12, 13]), sparsification ([31, 6] (Yu et al. 2024)), routing ([20]), and second-order schemes such as Fisher averaging ([21, 25]) and RegMean ([15]). Whereas AdaMerging ([33]) optimizes black-box weights, Phase 1 obtains merge coefficients from a residual program on unlabelled prompts and normalizes the resulting per-domain residuals for diagnosis; Remark 1 identifies the related subspace Fisher projection of the quadratic. Extending tangent-space analyses of merging ([22]), Proposition 2 establishes reward-space linearity, and $\mathrm{Err}_{\mathrm{lin}}$ diagnoses departures from it.

On-policy and multi-teacher distillation. Beyond classical distillation ([11, 16] (You et al. 2017)), on-policy distillation trains on student rollouts ([9, 1, 10, 5, 19, 32, 18] (Zhong et al. 2026)); multi-teacher variants ([8, 28, 35, 4]) consolidate specialists via joint training from base weights. We show that, to second order, this objective has an optimal closed-form stage, and we gate subsequent updates by the normalized residual spectrum.

Conclusion

We bridge parameter merging and multi-teacher distillation by showing that, to second order, the MOPD objective coincides with Fisher merging. Parameter merging is thus optimal, to second order, as a closed-form stage of distillation. For tractability, we restrict that merge to the task-vector subspace and, because the domain weights are unknown, replace them by their worst case: a residual program on unlabelled prompts yields the task-vector weights, and post-hoc reward-energy normalization yields the domain residual spectrum. Empirically, our zero-training merge beats a distilled student on Open-MOPD and approaches the strongest trained baseline on G-OPD, while residual-gated RGBM advances performance further. Un-gated write-backs degrade below the merge, confirming that post-merge gains are localized, with the normalized spectrum identifying the updates to test for safe write-back. Limitations are discussed in Appendix § 12.

LLM Usage

In accordance with conference guidelines, we disclose the use of large language models (LLMs) in the preparation of this manuscript. Mainstream LLMs were used exclusively for grammar correction, stylistic polishing, and editorial refinement to enhance the readability and fluency of the text.

References

This appendix is organized as follows. Appendix A proves Theorem 1 and Proposition 2, derives the residual, its scale-normalized spectrum, and Minimax convexity, and bounds the linearization error of shrinkage re-merging. Appendix B reports the six-benchmark Phase 1 breakdown, the six-benchmark scores of the normalized-objective merges, the empirical reward correlation, the $K = 2$ pairwise merges, generation lengths, and the full G-OPD suite. Appendix C details the joint-distillation recipes, the post-hoc $\gamma$ write-backs, the $\epsilon$ allocation cuts, training infrastructure, the cost of assembling $M_d$, and the Open-MOPD RGBM Phase 2 specification. Appendix D states the scope of the linear-basin guarantee, the specialist count, and the source of the Open-MOPD margin.

Mathematical Proofs and Derivations

This section records the proofs deferred from § 2: the second-order equivalence of multi-teacher distillation to Fisher merging (Appendix A.1), the first-order reward homomorphism (Appendix A.2), the second-moment residual, its scale-normalized spectrum, and uniqueness of the Minimax QP (Appendix A.3), and the shrinkage bound on the linearization error of a re-merged branch (Appendix A.4).

Proof of Theorem 1 (Second-Order Equivalence to Fisher Merging)

Let $\mathcal{X}_d$ denote the prompt space for domain $d$, and $\mathcal{Y}$ denote the token output space. For a given prompt $x \sim\mathcal{D}_d$, the reverse Kullback-Leibler (KL) divergence between student policy $\pi_\theta(\cdot\mid x)$ and teacher policy $\pi_{W_{\phi_d}}(\cdot\mid x)$ is:

We define the expected domain divergence function $f_d : \mathbb{R}^{P} \to\mathbb{R}$ as:

We perform a multivariate Taylor expansion of $f_d(\theta)$ around the expert weight point $\theta= W_{\phi_d}$:

We evaluate each term:

1. Zeroth-order term: At $\theta= W_{\phi_d}$, the distributions are identical:

2. First-order gradient term: Taking the gradient of the reverse KL with respect to $\theta$:

Evaluating at $\theta= W_{\phi_d}$: the log ratio $\log(\pi_{W_{\phi_d}}/\pi_{W_{\phi_d}}) = \log1 = 0$. Furthermore, because $\sum_y \pi_\theta(y \mid x) = 1$ for all $\theta$, the sum of gradients vanishes: $\sum_y \nabla_\theta\pi_\theta(y \mid x) = \nabla_\theta\sum_y \pi_\theta(y \mid x) = \nabla_\theta1 = 0$. Therefore:

3. Second-order Hessian term: Differentiating the gradient expression again with respect to $\theta$:

Evaluating at $\theta=W_{\phi_d}$: the log ratio vanishes, leaving $\sum_y\nabla_{\theta}^{2}\pi_{W_{\phi_d}}(y\mid x)=\nabla_{\theta}^{2}(\sum_y\pi_{W_{\phi_d}}(y\mid x))=0$. Using the identity $\nabla_{\theta}\pi_{\theta}=\pi_{\theta}\nabla_{\theta}\log\pi_{\theta}$, the remaining term becomes:

which is precisely the empirical Fisher Information Matrix $F_d\in\mathbb{R}^{P\times P}$ of expert policy $\pi_{W_{\phi_d}}$.

Substituting these three results back into the multi-teacher objective $J_{\mathrm{MOPD}}(\theta)=\sum_{d=1}^{K}p_df_d(\theta)$ yields:

Taking the gradient with respect to $\theta$ and setting it to zero:

Assuming $\sum_{d=1}^{K}p_dF_d$ is non-singular, we obtain the global unconstrained Fisher merge:

This completes the proof of Theorem 1. ■

Proof of Proposition 2 (First-Order Homomorphism)

Let $f(x;\theta)\in\mathbb{R}^{|V|}$ denote the logit vector at a generation step under prompt-prefix $x$. The token distribution is given by the softmax function:

Taking the logarithm gives $\log\pi_{\theta}(y\mid x)=f(x;\theta)[y]-\log Z(x;\theta)$. Write $\Delta\log Z_d(x):=\log Z(x;W_{\phi_d})-\log Z(x;W_{\mathrm{ref}})$ and $\Delta\log Z_{\alpha}(x):=\log Z(x;\theta(\alpha))-\log Z(x;W_{\mathrm{ref}})$.

Let $\theta=W_{\mathrm{ref}}+\tau$. A first-order Taylor expansion of $f(x;\theta)[y]$ around $W_{\mathrm{ref}}$ yields:

Applying this to specialist weights $W_{\phi_d}=W_{\mathrm{ref}}+\tau_d$, the implicit reward is:

The log-partition function $\log Z(x;\theta)=\log\sum_{y'\in V}\exp(f(x;\theta)[y'])$ is a smooth function of $\theta$, so the same first-order expansion around $W_{\mathrm{ref}}$ gives

Taking $\tau= \tau_d$ gives $\Delta\log Z_d(x) = \nabla_\theta\log Z(x; W_{\mathrm{ref}})^\top\tau_d + O(\lVert\tau_d\rVert^2)$, and taking $\tau= T\alpha= \sum_{d=1}^{K} \alpha_d\tau_d$ gives, by linearity of the inner product,

Now, for any linear combination $\theta(\alpha) = W_{\mathrm{ref}} + \sum_{d=1}^{K} \alpha_d\tau_d$, by linearity of the gradient operator and Eq. (27):

where the last equality substitutes Eq. (25) for each $r_d$. This proves the First-Order Homomorphism between weight combinations and reward superposition. ■

Derivation of Second-Moment Residuals and Minimax Convexity

Connection to the Projected Fisher Information Metric. Under the first-order logit expansion (Proposition 2), for any specialist $d$ and prompt-prefix $x$,

where $\mu_d(x):=\mathbb{E}_{y'\sim\pi_{W_{\phi_d}}(\cdot\mid x)}[r(x,y')]$. The projected Fisher Information Matrix $S_d:=\tau^\top F_dT\in\mathbb{R}^{K\times K}$ satisfies:

Now, expanding the second-moment matrix $M_d:=\mathbb{E}_{x\sim\mathcal{D}_d,y\sim\pi_{W_{\phi_d}}}[r(x,y)r(x,y)^\top]$:

For any weight combination vector $\alpha\in\mathbb{R}^{K}$, writing $e_d\in\mathbb{R}^{K}$ for the $d$-th standard basis vector, the residual optimized in Phase 1 is

$\mathcal{L}_d$ upper-bounds the projected Fisher term $2J_{\mathrm{sub},d}(\alpha)=(\alpha-e_d)^\top S_d(\alpha-e_d)$. After solving the Minimax problem, we form the normalized residual $\overline{\mathcal{L}}_d(\alpha):=\mathcal{L}_d(\alpha)/\mathbb{E}[r_d^2]$. This post-hoc normalization expresses the remainder as a fraction of domain-$d$ reward energy and makes the spectrum comparable across domains.

Term-wise Decomposition. Writing out the quadratic with $(M_d)_{jk}=\mathbb{E}[r_jr_k]$ and then dividing by $\mathbb{E}[r_d^2]$ yields the normalized diagnostic

The first term is squared relative under-recovery of the in-domain reward (it vanishes at $\alpha_d=1$). The remaining terms are cross-domain interference measured in the same units. The correlation structure of Appendix § 10.§ 4 governs the cross-terms.

Theoretical Justification of Branch Re-merging within Local Tangent Basins

A natural question is whether the branch updates $\Delta_d = \theta_d - \theta_0^{\star}$ obtained in Phase 2, once written back with shrinkage, remain within the regime in which the first-order reward superposition of Proposition 2 is accurate. The following statement makes precise the role of the contraction factor $\gamma$.

Proposition 4 does not by itself assert that $\left\|\Delta_d\right\|$ is small; it states that, whatever displacement branch training produces, shrinkage can only reduce the bound on the linearization error, and that the reduction is first order in $\gamma$. On Open-MoPD the residual gate writes back only the Math branch at the unlabelled grid value $\gamma^{\star}=0.125$; passivated Code/IF contribute $\Delta_d=0$.

Phase 1 Diagnostics and Detailed Benchmark Results

This section supports § 2–§ 4 with the linearity diagnostics (Appendix § 10.§ 1), the six-benchmark Phase 1 breakdown (Appendix § 10.§ 2), the six-benchmark scores of the normalized-objective merges (Appendix § 10.§ 3), the empirical reward correlation (Appendix § 10.§ 4), the $K=2$ pairwise merges (Appendix § 10.§ 5), generation lengths (Appendix § 10.§ 6), and the full G-OPD suite (Appendix § 10.§ 7).

Linearity Diagnostics on the Isotropic Family

Figure 5 reports the diagnostics discussed in § 4.7: the isotropic scan $\theta(\alpha)=W_{\mathrm{ref}}+\alpha\sum_d\tau_d$ of Remark 1, the regression $R^2$ of the merged log-ratio on the superposed reward over 38M tokens, and the cross-domain interference ratio $\mathcal{I}_{\mathrm{cross}} := \sum_d\mathbb{E}_d[(\ell_{\alpha}-\alpha r_d)^2]/\sum_d\mathbb{E}_d[\ell_{\alpha}^{2}]$, the share of merged log-ratio energy not explained by the in-domain reward. The branch displacements used in § 4.7 are $\left\|T\alpha^{\star}\right\|=1.025$, $\left\|\Delta_{\mathrm{math}}\right\|=0.229$, $\left\|\Delta_{\mathrm{code}}\right\|=0.299$, $\left\|\Delta_{\mathrm{if}}\right\|=0.265$.

Complete Per-Benchmark Phase 1 Results

Table 4 details the individual benchmark evaluations across all six downstream evaluation tasks: AIME 2024, AIME 2025 (Math), LiveCodeBench v5, LiveCodeBench v6 (Code), IFEval, and IFBench (Instruction Following). The Type column is the coefficient family: Scalar is a shared scale $\alpha\mathbf{1}$; 3D Vector is a free $\alpha\in\mathbb{R}^3$. Official Distilled Model is the main-text Multi-Teacher OPD row. Forward-KL Merge uses forward KL in the same isotropic subspace program; Minimax Worst-Domain Merge (Scalar) is Eq. (9) restricted to $\alpha=\alpha\mathbf{1}$. Both Subspace Fisher Merge rows solve Eq. (5) under uniform $p_d=1/3$ with $M_d$ in place of $S_d$; the main-text Subspace Fisher Merge is the 3D Vector row.

Method / Configuration	Type	A24	A25	Avg	v5	v6	Avg	IFEval	IFB	Avg	Macro
MixSFT Base ($W_{\mathrm{ref}}$)	–	14.79	18.18	16.49	15.63	18.46	17.05	66.36	18.67	42.52	25.35
RouteRL (Oracle Upper Bound)	–	26.88	27.60	27.24	27.27	25.26	26.27	81.70	25.33	53.52	35.68
Official Distilled Model	Distill	24.38	24.68	24.53	21.05	21.31	21.18	70.24	24.00	47.12	30.93
Equal-Weight Average	Scalar	20.31	21.30	20.81	17.66	20.63	19.15	69.87	19.67	44.77	28.24
Task Arithmetic	Scalar	23.91	24.79	24.35	22.99	22.63	22.81	69.87	23.67	46.77	31.31
Subspace Fisher Merge	Scalar	25.47	24.48	24.97	23.65	22.91	23.28	73.94	21.33	47.64	31.96
Subspace Fisher Merge	3D Vector	23.85	24.38	24.11	24.49	22.69	23.59	72.64	23.33	47.99	31.90
Forward KL (FKL) Merge	Scalar	24.58	23.28	23.93	23.59	22.34	22.97	71.53	24.33	47.93	31.61
Minimax Worst-Domain Merge	Scalar	25.10	24.84	24.97	24.07	22.91	23.49	69.87	20.33	45.10	31.19
Minimax Merge ($\alpha^\star$)	3D Vector	23.91	23.80	23.86	24.27	23.91	24.09	73.87	25.33	49.60	32.52
RGBM Phase 2	Residual-Gated	23.80	24.06	23.93	25.15	24.11	24.63	75.05	26.00	50.52	33.03

Scores of the Normalized-Objective Merges

The four Open-MOPD programs of Table 2 that minimize $\mathcal{L}_d/\mathbb{E}[r_d^2]$ are absent from Table 4; their six-benchmark scores are Table 5.

Program	A24	A25	Math	v5	v6	Code	IFEval	IFB	IF	Macro
A scalar $\overline{\mathcal{L}}$	24.64	24.11	24.38	23.23	23.49	23.36	71.72	18.00	44.86	30.86
A vector $\overline{\mathcal{L}}$	23.65	24.74	24.19	17.60	21.54	19.57	74.68	21.33	48.00	30.59
C scalar $\overline{\mathcal{L}}$	21.25	21.56	21.41	18.44	21.09	19.76	70.24	21.33	45.79	28.99
C vector $\overline{\mathcal{L}}$	23.54	24.90	24.22	19.52	21.94	20.73	75.05	19.00	47.02	30.66

Empirical Reward Correlation Geometry

We sample 6,144 rollout sequences across domain training prompts and extract implicit rewards $\mathbf{r} = [r_{\mathrm{math}}, r_{\mathrm{code}}, r_{\mathrm{if}}]^{\mathsf{T}}$. The empirical correlation matrix is:

The off-diagonal correlation between Math and Code is moderately positive (0.313), indicating that mathematical problem-solving and programming share underlying algorithmic reasoning representations. In contrast, the Instruction-Following specialist is strictly orthogonal to both reasoning domains ($< 0.045$). This geometric structure directly explains why the Minimax solver assigns $\alpha_{\mathrm{if}}^{\star} \approx0.800$ (as the IF domain experiences near-zero destructive interference) while compressing $\alpha_{\mathrm{math}}^{\star} \approx0.597$ and $\alpha_{\mathrm{code}}^{\star} \approx0.620$ to minimize cross-talk.

Two-Domain ($K = 2$) Sub-Consolidations and Residual Tracking

To test whether the Fisher-geometric theory and the predictive validity of the residual spectrum hold across different expert subsets, we evaluate all three pairwise $K = 2$ specialist consolidations formed from $\{W_{\phi_{\mathrm{math}}}, W_{\phi_{\mathrm{code}}}, W_{\phi_{\mathrm{if}}}\}$:

1. Math + Code ($K = 2$): The implicit rewards share positive covariance ($\rho= 0.313$). The Minimax solver allocates balanced weights $\alpha^{\star} = [0.635, 0.652]^{\mathsf{T}}$, yielding zero-training performance of Math 24.38, Code 24.81 (Macro 24.60, higher than the 3-expert merge on Code due to zero interference from IF). The normalized spectrum ranks $\widetilde{\mathcal{L}}_{\mathrm{math}} > \widetilde{\mathcal{L}}_{\mathrm{code}}$, correctly identifying Math as the remaining active bottleneck.

2. Math + IF ($K = 2$): Because the two specialists are nearly orthogonal ($\rho= 0.043$), cross-domain interference is minimal. The Minimax weights are $\alpha^{\star} = [0.680, 0.885]^{\mathsf{T}}$, achieving Math 24.82 and IF 48.95 (Macro 36.89). $\widetilde{\mathcal{L}}_{\mathrm{if}}$ is near the floor (almost complete recovery), while $\widetilde{\mathcal{L}}_{\mathrm{math}}$ marks Math for post-merge branch training.

3. Code + IF ($K = 2$): With near-zero cross-covariance ($\rho= 0.035$), linear superposition incurs minimal interference. The Minimax weights $\alpha^{\star} = [0.720, 0.890]^{\mathsf{T}}$ achieve Code 25.02 and IF 49.10 (Macro 37.06). Both normalized residuals sit near the floor, predicting that both domains are passivated and require zero post-merge branch compute.

Across all $K = 2$ sub-settings, the normalized residual spectrum $\widetilde{\mathcal{L}}$ correctly ranks domain bottlenecks and tracks pairwise recovery.

Token Generation Length Distributions

Table 6 summarizes the average completion lengths across evaluation benchmarks.

Model	AIME 2024 (Math)	LiveCodeBench v5 (Code)	IFBench (IF)
MixSFT Base ($W_{\mathrm{ref}}$)	3,420	485	310
Math Specialist ($W_{\phi_{\mathrm{math}}}$)	14,850	490	280
Code Specialist ($W_{\phi_{\mathrm{code}}}$)	3,890	720	295
IF Specialist ($W_{\phi_{\mathrm{if}}}$)	2,950	420	315
Equal-Weight Task Arithmetic	11,240	560	410
Official Distilled Model (600 steps)	11,850	580	390
Zero-Training Merge ($\theta_{0}^{\star}$)	12,316	615	461

The zero-training merge $\theta_{0}^{\star}$ preserves extended reasoning lengths ($\approx12.3\mathrm{k}$ tokens on AIME), confirming that parameter merging successfully preserves reasoning traces without requiring expensive post-hoc length penalties.

Cross-Suite Generalization on G-OPD Benchmarks (Qwen3-4B)

To further establish that the Fisher-geometric projection generalizes across specialist regimes and model families, we evaluate RGBM on the official G-OPD benchmarks [35] using Qwen3-4B in two specialist regimes: the intermediate checkpoints of Table 2 in [35] ($W_{\phi\mathrm{math}}$ at 500 steps, $W_{\phi\mathrm{code}}$ at 300 steps), which are the main-text G-OPD setting, and the fully trained pair of their Table 8 (both at 1200 steps). Table 7 reports the complete per-task benchmark results across all 7 evaluation datasets (AIME 2024, AIME 2025, HMMT Feb, HMMT Nov, HumanEval+ [HE+], MBPP+, and LiveCodeBench [LCB v6]) following the standardized evaluation protocol.

Regime	Method / Configuration	AIME24	AIME25	HMMT-Feb	HMMT-Nov	HE+	MBPP+	LCB	7-Task Avg
Intermediate (Math-500 + Code-300)	Teacher ($W_{\phi\mathrm{math}}$, $W_{\phi\mathrm{code}}$)	58.00	54.60	32.50	38.90	86.00	70.20	27.30	52.50
Intermediate (Math-500 + Code-300)	Student Base ($W_{\mathrm{ref}}$)	21.50	21.90	10.00	8.00	74.70	64.70	17.90	31.24
Intermediate (Math-500 + Code-300)	SFT	58.50	53.30	30.70	34.80	86.40	69.60	26.40	51.39
Intermediate (Math-500 + Code-300)	ExPO (Zheng et al., 2024)	57.50	54.50	31.70	36.30	86.70	72.00	29.00	52.53
Intermediate (Math-500 + Code-300)	On-Policy Distillation (OPD)	60.60	54.10	32.50	38.30	84.60	69.50	27.60	52.46
Intermediate (Math-500 + Code-300)	ExOPD [35]	61.00	56.00	34.40	39.20	86.30	70.60	29.00	53.79
Intermediate (Math-500 + Code-300)	Task Arithmetic	56.98	50.42	28.02	36.77	84.30	68.52	29.14	50.59
Intermediate (Math-500 + Code-300)	Equal-Weight Average	33.75	25.00	15.52	15.94	79.88	67.13	20.00	36.75
Intermediate (Math-500 + Code-300)	RGBM Phase 1 (Ours)	59.17	55.31	32.71	37.50	87.96	71.43	29.29	53.34
Intermediate (Math-500 + Code-300)	RGBM Phase 2 (Ours)	60.42	56.56	33.96	39.27	86.91	71.30	30.57	54.14
Converged (Math-1200 + Code-1200)	Teacher ($W_{\phi\mathrm{math}}$, $W_{\phi\mathrm{code}}$)	68.20	59.30	37.30	42.90	88.90	72.50	28.00	56.73
Converged (Math-1200 + Code-1200)	Student Base ($W_{\mathrm{ref}}$)	21.50	21.90	10.00	8.00	74.70	64.70	17.90	31.24
Converged (Math-1200 + Code-1200)	OPD (Single-Teacher)	68.30	58.70	38.70	41.20	89.30	71.30	28.00	56.50
Converged (Math-1200 + Code-1200)	ExOPD (Single-Teacher)	68.40	59.20	38.20	42.80	89.90	73.70	29.30	57.36
Converged (Math-1200 + Code-1200)	OPD (Multi-Teacher)	68.20	60.20	38.50	40.80	86.40	72.10	27.60	56.26
Converged (Math-1200 + Code-1200)	ExOPD (Multi-Teacher)	70.10	59.60	37.50	42.70	89.50	73.90	29.70	57.57
Converged (Math-1200 + Code-1200)	Task Arithmetic	64.79	59.48	35.10	41.56	87.50	74.74	30.86	56.29
Converged (Math-1200 + Code-1200)	Equal-Weight Average	40.94	35.62	20.10	24.17	80.79	68.85	21.57	41.72
Converged (Math-1200 + Code-1200)	RGBM Phase 1 (Ours)	67.50	62.92	36.46	43.33	85.37	74.87	31.00	57.35

System	Init	Training	Written back
RouteRL (Gao et al., [8])	$K$ specialists	none	Routing oracle; serves $K$ models
Multi-Teacher OPD (Gao et al., [8])	$W_{\mathrm{ref}}$	joint OPD, all domains	Full student checkpoint
Task Arithmetic (Ilharco et al., [13])	$W_{\mathrm{ref}}$	none	$\sum_d\tau_d$ ($\alpha=1$)
Subspace Fisher Merge	$W_{\mathrm{ref}}$	none	$\mathcal{T}\alpha_{\mathrm{sub}}$ (Eq. (5), $p_d=1/3$, $M_d$ for $S_d$)
RGBM Phase 1 ($\theta_0^{\star}$)	$W_{\mathrm{ref}}$	none	$\mathcal{T}\alpha^{\star}$
Uniform Distillation	$\theta_0^{\star}$	joint OPD, $p_d=1/3$	Shared weights
TIES (Yadav et al., [31]) / DARE (Yu et al., 2024)	$\theta_0^{\star}$	isolated branch per domain	All $\{\Delta_d\}_{d=1}^{K}$, sparsified
Gated Math ($\gamma=1.0$)	$\theta_0^{\star}$	isolated Math branch only	$\Delta_{\mathrm{math}}$ (no shrinkage)
Gated Math ($\gamma=0.5$)	$\theta_0^{\star}$	isolated Math branch only	$0.5\Delta_{\mathrm{math}}$
RGBM Phase 2 (Ours)	$\theta_0^{\star}$	the same isolated Math branch	$\gamma^{\star}\Delta_{\mathrm{math}}$

Phase 1 coefficients and Phase 2 allocation. On the intermediate pair the Minimax weights are $\alpha^{\star} = (0.675, 0.666)$ and the normalized spectrum $\mathcal{L} = (0.094, 0.090)$ has its largest gap down to 0, so Eq. (10) gives $\epsilon= 0.045$ and both Math and Code stay active. Isolated distillation branches re-merged at the unlabelled grid value $\gamma= 0.5$ reach 54.14 (Table 7), +0.80 over Phase 1 and +0.35 over trained ExOPD. On the converged pair we report only Phase 1 ($\alpha^{\star} = (0.511, 0.505)$, average 57.35); there is no Phase 2 run.

Additional Experimental Details and Parameter Sensitivity

This section collects the full RGBM procedure and the systems compared (Appendix § 11.§ 1), the five joint-distillation recipes (Appendix § 11.§ 2), the post-hoc $\gamma$ write-backs (Appendix § 11.§ 3), the $\epsilon$ allocation cuts (Appendix § 11.§ 4), training infrastructure (Appendix § 11.§ 5), the cost and robustness of assembling $M_d$ (Appendix § 11.§ 6), and the Open-MOPD RGBM Phase 2 specification (Appendix § 11.§ 7).

RGBM Procedure and Systems Compared

Algorithm 1 lists the complete RGBM procedure of § 3; Table 8 lists how every Open-MOPD system in Table 1 is built.

When evaluating $\operatorname{Err}_{\mathrm{lin}}(\theta^{\star}_{\theta} + \gamma\Delta_d)$ in Algorithm 1, the response is $\ell_{\theta_{\gamma}} = \log(\pi_{\theta_{\gamma}}/\pi_{W_{\mathrm{ref}}})$ and the predictor is the Phase 1 superposition $\alpha^{\star\mathsf{T}}r$, held fixed. Both are scored token-wise on the same Phase 1 pairs $(x,y)$ used to assemble $M_d$, pooled over $\bigcup_d \mathcal{D}_{\mathrm{calib},d}$. A single OLS with intercept then yields $R^2$, and $\operatorname{Err}_{\mathrm{lin}} = 1 - R^2$ as in Eq. (7).

Algorithm 1 Residual-Governed Branch-and-Merge (RGBM)

Require: Base weights $W_{\mathrm{ref}}$, specialist weights $\{W_{\phi_d}\}_{d=1}^{K}$, unlabelled calibration prompts $\mathcal{D}_{\mathrm{calib}}=\{\mathcal{D}_{\mathrm{calib},d}\}_{d=1}^{K}$, Tikhonov constant $\lambda$, discrete shrinkage grid $\Gamma=\{0.125,0.25,0.5,0.75\}$, linearity tolerance $\delta_{\mathrm{tol}}$.

1. For each $d$: draw $N_d$ pairs $x_i\sim\mathcal{D}_{\mathrm{calib},d}$, $y_i\sim\pi_{W_{\phi_d}}(\cdot\mid x_i)$; score $\mathbf{r}(x_i,y_i)=\left[\log\frac{\pi_{W_{\phi_k}}(y_i\mid x_i)}{\pi_{W_{\mathrm{ref}}}(y_i\mid x_i)}\right]_{k=1}^{K}$; $M_d\leftarrow\frac{1}{N_d}\sum_i\mathbf{r}\mathbf{r}^{\mathsf{T}}$.

2. Solve the residual Minimax QP (Eq. (9)) with $\widetilde{M}_d=M_d+\lambda I$ for $\alpha^{\star}$; set $\theta_0^{\star}\leftarrow W_{\mathrm{ref}}+\mathcal{T}\alpha^{\star}$; using $M_d$, set $\mathcal{L}_d\leftarrow(\alpha^{\star}-e_d)^{\mathsf{T}}M_d(\alpha^{\star}-e_d)$ and $\mathcal{L}_d\leftarrow\mathcal{L}_d/\mathbb{E}[r_d^2]$ (Eq. (8)).

3. Sort $\mathcal{L}_{(1)}\ge\cdots\ge\mathcal{L}_{(K)}\ge\mathcal{L}_{(K+1)}:=0$; $k^{\star}\leftarrow\underset{i}{\operatorname{arg\,max}}(\mathcal{L}_{(i)}-\mathcal{L}_{(i+1)})$; $\epsilon\leftarrow\frac{1}{2}(\mathcal{L}_{(k^{\star})}+\mathcal{L}_{(k^{\star}+1)})$ (Eq. (10)).

4. for $d=1,\ldots,K$ do

5. if $\mathcal{L}_d\le\epsilon$ then

6. Passivate: $\Delta_d\leftarrow0$

7. else

8. Active branch: $\theta_d\leftarrow\theta_0^{\star}$; train on $\mathcal{D}_d$ only; $\Delta_d\leftarrow\theta_d^{(\mathrm{end})}-\theta_0^{\star}$.

9. Compute unlabelled shrinkage $\gamma_d^{\star}\leftarrow\max\{\gamma\in\Gamma\mid\operatorname{Err}_{\mathrm{lin}}(\theta_0^{\star}+\gamma\Delta_d)\le\delta_{\mathrm{tol}}\}$ (Eq. (11)).

10. end if

11. end for

12. Re-merge: $\theta_{\mathrm{RGBM}}\leftarrow\theta_0^{\star}+\sum_{d:\mathcal{L}_d>\epsilon}\gamma_d^{\star}\Delta_d$ (Eq. (11)).

13. return $\theta_{\mathrm{RGBM}}$

Joint Multi-Task Distillation: Exploration of 5 Recipes

To thoroughly investigate whether any joint fine-tuning objective could maintain or improve upon $\theta_0^{\star}$, we implemented and evaluated five distinct joint training recipes:

1. Standard Multi-Teacher Distillation (Uniform MOPD): Multi-teacher reverse-KL on student rollouts with equal domain sampling ($p_d=1/3$).

2. Outcome-Gated Distillation: Outcome-gated on-policy distillation, where teacher distribution matching is active only when student reasoning fails verification.

3. Outcome-Veto Regularized Distillation: Regularized distillation applying explicit negative KL penalties to suppress drift on non-target domains.

4. Trust-Region Regularized Distillation: Outcome-gated distillation with explicit KL anchoring to $\theta_0^{\star}$ ($\beta=2.5$).

5. Hybrid Teacher-Anchor Distillation: Convex combination of specialist teacher KL and student-to-$\theta_0^{\star}$ anchor KL.

Empirical Findings across Checkpoints. On the five recipes in Table 9, Macro Total stays in the $30.5\%\sim32.3\%$ range; every listed checkpoint lies below residual-gated RGBM Phase 2 ($33.03\%$). At the last listed checkpoint of each recipe, Code is $1.1\sim2.9$ points below $\theta_0^{\star}$, and IF stays below the Phase 1 value 49.60. Within this testbed, no joint objective we tried preserved the low-residual domains while training on the shared weights, which is the behaviour anticipated by the argument of § 3.3.

Distillation Framework	Step	Math	Code	IF	Macro	Total	$\Delta$ vs. $\theta_0^\star$
RGBM Phase 1 ($\theta_0^\star$)	0	23.86	24.09	49.60	32.52		–
RGBM Phase 2 (Ours)	0	23.93	24.63	50.52	33.03		+0.51
Uniform Distillation [8]	Step 10	24.38	20.78	47.79	30.98		-1.54
	Step 30	24.58	21.46	47.62	31.22		-1.30
	Step 50	22.94	21.83	46.84	30.54		-1.98
	Trajectory Mean	23.97	21.36	47.42	30.91		-1.61
Outcome-Gated Distillation	Step 10	24.45	21.62	48.12	31.40		-1.12
	Step 30	24.97	22.18	48.91	32.02		-0.50
	Step 50	24.64	22.25	46.97	31.28		-1.24
	Trajectory Mean	24.69	22.02	48.00	31.57		-0.95
Outcome-Veto Distillation	Step 10	25.23	21.93	47.10	31.42		-1.10
	Step 30	24.24	21.59	48.99	31.61		-0.91
	Step 50	25.08	21.20	47.73	31.33		-1.19
	Trajectory Mean	24.85	21.57	47.94	31.45		-1.07
Trust-Region Regularization ($\beta= 2.5$)	Step 5	23.57	23.33	49.99	32.30		-0.22
	Step 10	24.84	23.66	47.52	32.01		-0.51
	Step 20	24.95	21.18	47.62	31.25		-1.27
	Trajectory Mean	24.45	22.72	48.38	31.85		-0.67
Hybrid Teacher-Anchor	Step 10	24.40	21.15	47.73	31.09		-1.43
	Step 30	23.65	22.73	48.69	31.69		-0.83
	Step 50	24.69	22.99	46.38	31.35		-1.17
	Trajectory Mean	24.25	22.29	47.60	31.38		-1.14

Write-Back Does Not Raise Macro above Residual-Gated RGBM

The main-text Phase 2 scheme is residual-gated RGBM Phase 2 with Math active and Code/IF passivated. Shrinkage is chosen by the unlabelled rule (Eq. (11)) on $\Gamma= \{0.125, 0.25, 0.5, 0.75\}$, which selects $\gamma^\star= 0.125$ (Macro 33.03), above the Phase 1 merge (32.52). Table 10 is a post-hoc diagnostic: the remaining grid points, a $\gamma= 1$ (no-shrinkage) foil, and equal-mass alternatives are not used to choose $\gamma$; none of them exceed 33.03.

Configuration	Formula / Shrinkage	Math	Code	IF	Macro	Total	$\operatorname{Err}_{\mathrm{lin}}$
Part A: Grid $\Gamma= \{0.125, 0.25, 0.5, 0.75\}$ and $\gamma= 1$ foil
$\gamma= 0.125$ (Ours, $\gamma^\star$)	$\theta_0^\star+ 0.125\,\Delta_{\mathrm{math}}$	23.93	24.63	50.52	33.03		0.0701
$\gamma= 0.25$	$\theta_0^\star+ 0.25\,\Delta_{\mathrm{math}}$	24.19	24.02	47.82	32.01		0.1749
$\gamma= 0.50$	$\theta_0^\star+ 0.50\,\Delta_{\mathrm{math}}$	25.42	23.48	48.08	32.32		0.2594
$\gamma= 0.75$	$\theta_0^\star+ 0.75\,\Delta_{\mathrm{math}}$	25.08	22.87	45.51	31.15		0.6442
$\gamma= 1.00$	$\theta_0^\star+ 1.00\,\Delta_{\mathrm{math}}$	25.03	22.51	47.87	31.80		0.7231
Part B: Equal Total Weight Mass Alternatives
Single Point $\theta_{\mathrm{math}}^{(2)}$	$\theta_0^\star+ \frac{2}{3}\Delta_{\mathrm{math}}^{(2)}$	25.52	23.33	47.57	32.14		–
Single Point $\theta_{\mathrm{math}}^{(1)}$	$\theta_0^\star+ \frac{2}{3}\Delta_{\mathrm{math}}^{(1)}$	24.64	23.84	48.18	32.22		–
Two-Point Average	$\frac{1}{3}(\theta_0^\star+ \theta_{\mathrm{math}}^{(1)} + \theta_{\mathrm{math}}^{(2)})$	24.90	23.50	48.91	32.44		–

Readout. Eq. (11) selects $\gamma^\star= 0.125$ from $\Gamma$ using only $\operatorname{Err}_{\mathrm{lin}}$ on $\mathcal{D}_{\mathrm{calib}}$. In Table 10 that value is 0.0701 $\leq\delta_{\mathrm{tol}}$, while $\gamma= 0.25$ already reaches 0.1749. Post-hoc, every larger grid point and every equal-mass alternative in Table 10 has Macro $\le32.44$. On G-OPD, where both domains require a separate OPD branch, the same unlabelled rule yields $\gamma^\star= 0.5$.

Residual Threshold Sensitivity and Passivation Robustness

In Phase 2 the threshold $\epsilon$ only matters through which domains it activates. Eq. (10) is a closed-form choice: sort the normalized spectrum $\overline{\mathcal{L}} = (0.128, 0.052, 0.015)$, append the floor 0, take the midpoint of the largest adjacent gap, and obtain $\epsilon= 0.090$ (Math active). Any other $\epsilon$ in the same gap $[0.052, 0.128)$ yields the same allocation; the four possible cuts are:

• $\epsilon\ge0.128$: no domain is active; the write-back is empty and the checkpoint remains $\theta_0^\star$ (Macro 32.52).

• $0.052 \le\epsilon< 0.128$ (Eq. (10) on Open-MOPD, $\epsilon= 0.090$): only Math is active. RGBM Phase 2 under this cut writes the unlabelled grid value $\gamma^\star= 0.125\Delta_{\mathrm{math}}$ (Eq. (11)) and is Macro 33.03. Post-hoc write-backs at the remaining grid points and the equal-mass alternatives yield Macro 31.15 $\sim$ 32.44 (Table 10), all below 33.03.

• $0.015 \le\epsilon< 0.052$: Code is additionally activated. Isolated Code branches improve their own domain (+0.43 $\sim$ +1.31; Table 11). Writing every branch back, as un-gated TIES and DARE do, drops Code from 24.09 at $\theta_0^\star$ to 21.56 and 22.40 (Table 1).

• $\epsilon< 0.015$: IF is additionally activated. Writing an IF branch back ($\theta_0^\star+ 0.5\Delta_{\mathrm{if}}$) reduces Code to 20.70% and Macro to 31.10%: the leakage mechanism of § 4.4 operates even for a branch trained exclusively on IF prompts.

Domain	$\overline{\mathcal{L}}_d(\alpha^\star)$	Rank	$\theta_0^\star$	Distill. Branch	Outcome-Gated	RL Branch
Math	0.128	1	23.86	+1.10	+1.83	+0.58
Code	0.052	2	24.09	+0.43	+1.31	+0.51
IF	0.015	3	49.60	+0.48	+1.63	+0.57

The Open-MOPD allocation is therefore Math-active residual-gated RGBM. On G-OPD, $\alpha^\star= (0.675, 0.666)$ and $\overline{\mathcal{L}} = (0.094, 0.090)$: the largest gap is down to 0, so Eq. (10) gives $\epsilon= 0.045$ and both domains stay active.

Training Infrastructure and Hyperparameters

All branch training and distillation experiments are executed using the verl distributed RLHF framework (Sheng et al. [27]) on clusters of 8$\times$ NVIDIA A100/H800 GPUs. The detailed hyperparameter configuration is reported in Table 12.

Hyperparameter	Value
Base Model Architecture	SmolLM3-3B ($P \approx3.0 \times10^9$)
Optimizer	AdamW ($\beta_1 = 0.9$, $\beta_2 = 0.95$, $\epsilon= 10^{-8}$)
Peak Learning Rate	$1.5 \times10^{-6}$
Learning Rate Schedule	Linear warmup (5 steps) with cosine decay
Global Batch Size	96 prompt sequences per step
Rollout Group Size ($n$)	8 samples per prompt
Maximum Sequence Length	24,576 tokens
Maximum Model Context Length	32,768 tokens
Generation Temperature	0.6 (Math), 1.0 (Code), 1.0 (IF)
Checkpointing Frequency	Every 5 steps
Weight Decay	0.01

Computational Cost and Estimation Robustness of $M_d$

Estimating the second-moment matrices $M_d = \mathbb{E}_{D_d,\pi_{W_{\phi_d}}}[\mathbf{r}\mathbf{r}^{\top}] \in\mathbb{R}^{K \times K}$ requires calculating implicit rewards $\mathbf{r}(x_i,y_i) = [\log\pi_{W_{\phi_k}}(y_i \mid x_i) - \log\pi_{W_{\mathrm{ref}}}(y_i \mid x_i)]_{k=1}^{K}$ across $N$ calibration prompt rollouts ($N = 6144$ total in our experiments, 2048 per domain).

Wall-Clock Time, Scaling, and Comparison to Full Fisher Merging.

1. Tractability over Full-Space Fisher Inversion: Full-space Fisher merging ([21]) requires inverting a $P \times P$ matrix ($P \approx3.0 \times10^{9}$ for SmolLM3-3B), demanding $\sim10^{19}$ floating-point elements and exabytes of RAM, which is completely intractable for modern LLMs. In contrast, Subspace Fisher Merge compresses the system into $K \times K$ ($3 \times3$ in our setting), reducing matrix assembly and inversion to $< 10$ ms.

2. Inference Rollouts and Log-Prob Scoring: For $K = 3$ specialists and a 3B model, scoring $N = 6144$ sequences requires $O(NK)$ forward passes without gradient computation or backward graph storage, taking $< 15$ minutes on 8 GPUs ($< 2$ GPU·h).

3. Scalability across Domain Counts $K$: Contemporary MOPD pipelines consistently consolidate $K = 2 \sim4$ specialist modalities (e.g., Math, Code, IF; [8, 35, 28, 4]). Even if scaled to $K = 20$, solving the $K$-dimensional Minimax QP takes $< 0.02$ seconds on a single CPU core. By random matrix theory, covariance estimation requires $N = O(K \log K)$ samples, meaning $N \approx1024 \sim2048$ samples remain sufficient for high-fidelity spectrum rank preservation.

Sample Complexity and Robustness to Estimation Noise. To evaluate how finite-sample variance in $M_d$ propagates to the optimal weights $\alpha^{\star}$ and the resulting normalized spectrum $\mathcal{L}$, we perform bootstrap resampling of the calibration pool $D_{\mathrm{calib}}$ across sample sizes $N \in\{512, 1024, 2048, 4096, 6144\}$:

• At $N \ge1024$, the bootstrap standard deviation of the computed weights is $\sigma(\alpha_d^{\star}) < 0.012$, and the normalized spectrum remains invariant in its rank ordering: $\mathcal{L}_{\mathrm{math}} \gg\mathcal{L}_{\mathrm{code}} \gg\mathcal{L}_{\mathrm{if}}$ with $> 99.9\%$ confidence.

• The condition number of empirical matrices satisfies $\kappa(M_d) \le45.2$, well below the threshold where numerical inversion issues occur. Adding Tikhonov regularization $\lambda= 10^{-4}$ provides uniform numerical stability even under rank-deficient specialist scenarios.

RGBM Phase 2 Specification (Open-MOPD)

Phase 1 $\theta_0^{\star}$ (step 0): AIME24 23.91, AIME25 23.80, LCBv5 24.27, LCBv6 23.91, IFEval 73.87, IFBench 25.33, hence Math 23.86, Code 24.09, IF 49.60, Macro 32.52. The sole Open-MOPD Phase 2 scheme is residual-gated RGBM Phase 2 with Math active, written back at the unlabelled grid value $\gamma^{\star} = 0.125$ (Eq. (11)): AIME24 23.80, AIME25 24.06, LCBv5 25.15, LCBv6 24.11, IFEval 75.05, IFBench 26.00, hence Math 23.93, Code 24.63, IF 50.52, Macro 33.03. On G-OPD the same unlabelled rule is used with a separate OPD branch on each domain and yields $\gamma^{\star} = 0.5$.

Limitations

$\mathrm{Err}_{\mathrm{lin}}$ certifies only the first-order reward superposition on calibration prompts; it does not bound the third-order remainder of Theorem 1. While evaluating $K = 2 \sim3$ specialists across two model families ([8, 35, 28]), scaling to $K \gg10$ or heterogeneous bases remains future work. Furthermore, the $+0.5$1 Open-MOPD margin comes from Code and IF moving under a small Math write-back rather than from an in-domain Math gain; on G-OPD both domains are active, so this transfer is not isolated there.