Nash-MOPD: Optimizing Task Trade-offs in Multi-Teacher On-Policy Distillation

Introduction

Multi-teacher on-policy distillation (MOPD) aggregates the capabilities of multiple domain-expert teachers into a single student model, which is widely used in frontier model post-training [15, 18] (Zeng et al. 2026; DeepSeek-AI et al. 2026). During training, the student generates rollouts from prompts sampled across domains. The corresponding teacher then provides token-level supervision at student-visited states, and the student is updated to minimize the reverse KL divergence from its next-token distribution to the teacher’s [19, 20]. However, all domains in MOPD share the student’s finite budget of optimization steps and rollouts. As a result, increasing one domain’s sampling ratio leaves fewer training examples for the others. MOPD therefore entails a multi-task trade-off, since the domain mixture must balance competing learning demands to develop diverse capabilities together. This allocation consequently shapes both convergence speed and the final accuracy achievable within the training budget.

Effective use of this shared budget requires accounting for differences in how the student learns across domains. As training proceeds, the student may learn some domains quickly while requiring more examples to learn others. Fixed uses constant domain sampling ratios based on corpus sizes, so it cannot adjust the allocation to these differences or their changes over time. By contrast, $D^{3}$-MOPD adjusts the ratios using the teacher–student reverse-KL divergence and its recent decrease (Sun et al., 2026). However, favoring large recent decreases can direct more training toward domains that already improve quickly, while domains that improve more slowly may receive too few examples. We therefore need an adaptive scheduler that responds to changes in the student’s learning progress and provides sufficient training for domains that improve more slowly, so as to improve both convergence speed and final accuracy under the same training budget.

To address inflexible allocation and the risk of giving insufficient training to domains that improve more slowly, we propose NASH-MOPD, an adaptive domain scheduler based on a local minimax game. At each allocation update, NASH-MOPD uses existing reverse-KL statistics to estimate how much of the remaining teacher–student difference each domain reduces per sampled prompt. It then solves a local minimax game in which the sampler chooses domain sampling ratios and the adversary selects the domain with the smallest predicted proportional reduction. The equilibrium maximizes this minimum progress under the local model and gives more prompts to domains that need more examples for comparable improvement. Sampling bounds preserve domain coverage, while smoothing keeps successive allocations stable. Consequently, NASH-MOPD responds to changing learning progress and supports domains that improve more slowly without extra teacher queries or rollouts. Moreover, this allocation has a clear interpretation, since each domain’s sampling ratio reflects its estimated remaining training demand under the local model.

To evaluate the effectiveness of NASH-MOPD, we conduct experiments with DeepSeek-R1-Distill-Qwen-1.5B across mathematics, code, and instruction following, and with Qwen3-8B-Base across mathematics and code. All methods use the same training pipeline and budget. On the DeepSeek-R1-Distill-Qwen-1.5B student, NASH-MOPD exceeds both baselines’ final mathematics and code checkpoint scores at step 100. On the Qwen3-8B-Base student, it reaches Fixed’s final mathematics and code performance at step 100, using 60 and 80 fewer optimization steps than Fixed and the original D$^{3}$-MOPD baseline described in § 5, respectively. Furthermore, NASH-MOPD leads eight of nine reported metrics on the DeepSeek-R1-Distill-Qwen-1.5B student and is best or tied best on all six mathematics and code accuracy metrics on the Qwen3-8B-Base student. These results demonstrate that NASH-MOPD outperforms both baselines in convergence speed and overall final accuracy across domains.

Related Work

On-policy distillation and capability integration. Reverse-KL and generalized knowledge distillation supervise students on their own responses ([8, 1]). MOPD extends this to multiple teachers, routing rollouts to the corresponding domain teacher ([19]). We preserve its token-level supervision and adapt the probability of choosing each prompt domain.

Adaptive mixtures and multitask optimization. GradNorm and related approaches adapt task weights from gradient magnitude, loss evolution, or validation behavior ([4, 10] (Sundararaman et al. 2021)). DoReMi optimizes domain weights using a proxy-model objective ((Xie et al., 2023)). D$^{3}$-MOPD is the closest setting: it repurposes per-domain reverse-KL to adjust the domain mixture ((Sun et al., 2026)). Its composite signal combines remaining gap and descent velocity; our inverse-rate rule estimates exposure needed for completion. Our original D$^{3}$-MOPD baseline follows the published domain scheduling rule for allocating prompts across domains.

Games, fairness, and online feedback. Pareto multitask methods and Nash bargaining combine task gradients ((Sener and Koltun, 2018; Navon et al., 2022)). Nash learning from human feedback uses a zero-sum preference game ((Munos et al., 2024)). Here the adversary selects a domain bottleneck and the learner controls domain sampling ratios, so the players and payoff differ. GiGPO uses repeated states across existing trajectories to obtain finer credit signals for agent training ([6]); our controller similarly consumes observations already produced by training, but acts at the domain-mixture level. Neither gradient bargaining nor trajectory credit assignment directly supplies the exposure-based completion objective considered here.

Preliminaries

MOPD Framework

For each of $K$ domains, let $\mathcal{D}_k$ be its prompt distribution, $\pi_{T_k}$ its frozen domain-expert teacher, and $\pi_\theta$ the shared student. Domain sampling ratios $p \in\Delta_K$ satisfy $p_k \ge0$ and $\sum_k p_k = 1$. A batch of $B$ prompts assigns approximately $Bp_k$ examples to domain $k$; $e_k(t)$ counts cumulative prompt presentations, including repeated sampling.

MOPD extends on-policy distillation to multiple teachers ([19]). It samples $k \sim p$, $x \sim\mathcal{D}_k$, and a student response $y \sim\pi_\theta(\cdot\mid x)$. At each student-visited state $s_t = (x, y_{<t})$, the corresponding teacher supervises the student’s next-token distribution through reverse-KL ([8, 1]):

For domain labels $k_i$ and non-padding response lengths $N_i$, the batch objective is

Each divergence is evaluated on prompt $i$’s rollout, which is held fixed during the update. Eq. (1) averages tokens; prompt shares need not equal token-loss shares. Scheduling changes the prompt mixture, not teacher routing or loss weights.

Domain Allocation Variants

With $n_k$ usable training rows, Fixed retains

These corpus-based ratios cannot respond to changing learning progress.

$D^3$-MOPD adjusts ratios using normalized remaining reverse-KL $\bar{L}_k$ and nonnegative descent velocity $v_k$, the recent relative KL decrease ((Sun et al., 2026)). With temperature $T > 0$, the original method proposes

If all $c_k = 0$, the proposal is uniform; floors and caps preserve coverage. This adapts to progress, but favoring large recent decreases may undersample slowly improving domains. We use the original $D^3$-MOPD method; Appendix § 8.§ 2 gives implementation details for its domain scheduling.

Method

To address inflexible allocation and insufficient training for domains that improve more slowly, Nash-MOPD solves a local minimax game using existing reverse-KL statistics. The equilibrium gives more prompts to domains that need more examples for comparable improvement. The controller maximizes the smallest predicted proportional reduction to support bottleneck domains. Online updates adapt to changing progress, while sampling ratios reflect each domain’s remaining training demand under the local model. Bounds and smoothing preserve domain coverage and stable allocations without extra teacher queries or rollouts. Algorithm 1 gives the complete training loop.

Exposure-Normalized Progress Signals

For each domain, the observed signal is mean per-token reverse-KL over non-padding student response tokens. We clip negative empirical estimates to zero and maintain an exponential moving average $L_k(t)$ with coefficient $0.2$, initialized from the first valid observation. The reference scale $L_k^{(0)}$ is the mean of the first five raw nonnegative observations, floored at $10^{-8}$. With target ratio $\rho= 0.10$, the normalized remaining gap is

This normalization accounts for differences in initial reverse-KL scales. Gaps may exceed one when

Algorithm 1 NASH-MOPD: online domain scheduling
Require: Domain pools $\{D_k\}$, frozen teachers $\{\pi_{T_k}\}$, student $\pi_\theta$; batch size $B$; $S$ updates
Require: Sampling bounds $(\ell, u)$; warm-up $w$; scheduling interval $h$; smoothing $\alpha$; jitter $\gamma$
1: Initialize $p$ from usable-row proportions and $s_{\mathrm{last}} \gets 0$
2: Initialize pool permutations/cursors, prompt exposures, reference KL scales, and EMA/gap histories
3: for $s = 1, \ldots, S$ do
4: Draw $\eta_k \sim U[-\gamma,\gamma]$; set $\tilde{p}_k \gets p_k(1+\eta_k)/\sum_j p_j(1+\eta_j)$
5: $b \gets \operatorname{LARGESTREMAINDER}(B\tilde{p})$; draw $b_k$ prompts per domain, reshuffling exhausted pools
6: Shuffle the batch; generate student rollouts and query $\pi_{T_k}$ at each domain-$k$ prefix
7: Compute the token-mean reverse-KL loss $\mathcal{L}_{\mathrm{OPD}}$; update exposures and KL statistics
8: Estimate remaining gaps $g_k$ and absorption rates $\hat{r}_k$ using {xref}`Eqs. (4)–(5)`
9: if statistics are ready, $s \ge w$, and $s-s_{\mathrm{last}} \ge h$ then
10: $\mathcal{A} \gets \{k : g_k > 10^{-3}, \hat{r}_k > 10^{-8}, \hat{r}_k\text{ is finite}\}$
11: Solve {xref}`Eq. (9)` for $p^*$ with the coverage fallbacks in {xref}`Section 4.3`
12: $p \gets \Pi_{\mathcal{P}}((1-\alpha)p+\alpha p^*)$; $s_{\mathrm{last}} \gets s$
13: end if
14: Update $\theta$ using $\mathcal{L}_{\mathrm{OPD}}$; save controller, sampler, and random states for recovery
15: end for
16: return Student checkpoints and domain-scheduling traces

divergence increases; zero indicates the KL target, not benchmark success. After the fifth observation, the gap history starts at one. We retain up to 11 exposure–gap pairs and use the oldest and newest records $[t_0,t_1]$, spanning up to ten observed updates, to estimate

Here $\widehat{a}_k$ measures gap reduction per prompt, and $\widehat{r}_k$ is the exposure-normalized absorption rate, or proportional reduction per prompt. Dividing by exposure separates progress due to receiving more prompts from progress per prompt; a zero estimate means no positive decrease was measured in the window. These statistics are collected from observations already available during ordinary distillation, without extra teacher queries or rollouts.

Local Minimax Allocation

At one allocation update, fix positive gaps $g_k$ and finite positive per-prompt reductions $a_k$; let $r_k=a_k/g_k$. The separable local model predicts $g_k^+ \approx g_k-Bp_ka_k$, with proportional reduction $Bp_kr_k$. If these rates remained fixed, the predicted number of updates needed to close every gap would be

Minimizing Eq. (6) is thus equivalent to maximizing the smallest predicted proportional reduction. The sampler chooses $p$, while an adversary $q\in\Delta_K$ selects a bottleneck domain:

In Eq. (7), minimizing over $q$ for fixed $p$ gives $\min_k p_kr_k$. With no binding sampling bounds, the unique maximizing allocation is

To see why, an allocation with minimum payoff $m$ must satisfy $p_k\ge m/r_k$. Summing yields $m\le\left(\sum_k r_k^{-1}\right)^{-1}$; Eq. (8) attains equality for every domain. The equilibrium therefore assigns prompts in proportion to estimated remaining training demand $g_k/a_k$. A smaller positive rate requires more exposure to achieve comparable proportional improvement. This result concerns a frozen-rate local model, not guaranteed convergence of the nonlinear training trajectory. We re-estimate the rates online; Appendix A gives the saddle-point proof and bounded-allocation analysis.

The adversary serves to identify the current bottleneck, rather than introducing a trainable model. The controller computes the maximizing allocation directly from domain statistics. Scheduling therefore fits within the existing training loop, without alternating optimization between players.

Bounds, Smoothing, and Execution

Very small positive rates can produce extreme inverse-rate weights. We constrain base sampling ratios to $\mathcal{P} = \{p : \sum_k p_k = 1,\ \ell\le p_k \le u\}$, with $\ell= 0.10$ and $u = 0.70$. Feasibility requires $K\ell\le1 \le Ku$. Let $\mathcal{A}$ contain domains with $g_k > 10^{-3}$ and finite $\hat{r}_k > 10^{-8}$. Inactive domains initially receive the floor $\ell$. On the active set, bisection selects a scalar $v$ such that

This maximizes the smallest active-domain payoff under the bounds; it does not assign a positive modeled progress guarantee to zero-rate domains. If active caps leave probability unallocated, active shares are set to $u$ and the remainder goes to inactive domains. If none is active, the target is uniform. Training starts with Fixed’s row-proportional ratios (Eq. (2)). Observations accumulate during a 20-update warm-up; scheduling starts only when all domains have reference scales and at least two history records. Every ten updates, we smooth the target with $\alpha= 0.25$ and project onto $\mathcal{P}$:

The base mixture preserves domain coverage, while smoothing tempers abrupt reallocations. Each batch applies multiplicative jitter in $[0.7, 1.3]$, renormalizes, and uses largest-remainder rounding for $B$ prompt quotas. All schedules share the permutation sampler. Bounds apply to post-warm-up base ratios; jitter and rounding can change realized fractions. KL observations precede the optimizer update, and revised ratios control subsequent batches. Appendix B.1 gives sampler and checkpoint recovery details for resuming training.

Measuring joint convergence. Controller feedback uses reverse-KL; joint target attainment uses held-out domain scores $S_{k,t}$ and thresholds $\tau_k$. For the measured checkpoint set $\mathcal{C}$, the first checkpoint meeting all domain targets is

The first sustained crossing additionally requires every later measured checkpoint through the endpoint to satisfy all thresholds. We report both alongside endpoint accuracy.

Experiments

Experimental Setup

Models and training data. DeepSeek-R1-Distill-Qwen-1.5B Guo et al., [9] trains on DAPO mathematics Yu et al., 2025, TACO code Li et al., [16], and Multi-IF conversations He et al., [12]. Its teachers are JustRL-DeepSeek-1.5B He et al., [11], DeepCoder-1.5B-Preview, and UltraData-IF-1.5B Fu et al., [7], respectively. The Multi-IF training split excludes 256 held-out conversations. Qwen3-8B-Base Yang et al., 2025 uses the same math and code corpora. Its teachers are two Qwen3-8B-Base versions trained via reinforcement learning on math and code, respectively. This tests allocation across scales and domain counts.

Baselines and training steps. We compare Fixed, the original D$^3$-MOPD baseline, and NASH-MOPD through step 200, evaluating checkpoints every 20 optimizer updates. Final benchmark scores use the step-200 checkpoint. Fixed retains row-proportional base ratios throughout training. D$^3$-MOPD combines normalized reverse-KL with descent velocity. Its original scheduling procedure is specified in Appendix B.2. NASH-MOPD uses the controller in § 4. The baselines differ in allocation, not in the distillation objective. Their common training configuration is listed in Appendix B.3. The shared budget isolates allocation effects.

Benchmarks and metrics. Endpoint accuracy is avg@4 on MATH-500 Hendrycks et al., [13], LiveCodeBench v6, or LCB, Jain et al., [14], HumanEval+, and MBPP+ Liu et al., [17], and avg@16 on AIME25 and AMC23. avg@$k$ averages correctness over $k$ candidates per question, whereas pass@$k$ measures the fraction of questions solved by at least one of those candidates. We report both. Instruction following uses Multi-IF’s three-turn score and IFBench’s loose and strict prompt accuracy Pyatkin et al., 2025. Checkpoint curves use MATH-500 and LCB pass@1, with Multi-IF additionally included in the three-domain experiment. This separates average accuracy from candidate coverage.

Endpoint Accuracy

Tables 1 and 2 report endpoint accuracy. NASH-MOPD leads eight of nine individual benchmark metrics on the DeepSeek-R1-Distill-Qwen-1.5B student and is best or tied best on all six on the

Method	MATH avg@4	AIME avg@16	AMC avg@16	Mean	LCB avg@4	HE+ avg@4	MB+ avg@4	Mean	Multi-IF score	IF-L score	IF-S score	Mean
Initial student	76.85	18.96	65.78	53.86	16.57	61.13	51.52	43.07	29.59	15.00	10.33	18.31
Fixed	83.05	26.04	79.69	62.93	21.57	68.14	56.68	48.80	40.25	16.67	12.33	23.08
D³-MOPD	83.25	26.04	79.06	62.78	19.29	66.31	57.01	47.54	41.54	18.00	13.67	24.40
NASH-MOPD	85.95	27.50	78.91	64.12	23.29	68.75	57.47	49.84	43.01	20.00	14.67	25.89

Method	MATH avg@4	AIME avg@16	AMC avg@16	Mean	LCB avg@4	HE+ avg@4	MB+ avg@4	Mean
Initial student	64.25	9.38	55.00	42.88	16.57	69.82	43.39	43.26
Fixed	86.70	22.71	78.28	62.56	26.86	83.08	60.19	56.71
D³-MOPD	87.40	21.67	79.84	62.97	27.43	82.62	61.97	57.34
NASH-MOPD	87.40	23.13	80.16	63.56	28.57	83.23	62.37	58.06

Qwen3-8B-Base student. Domain means are reported separately from these counts.

Three-domain experiment. On the DeepSeek-R1-Distill-Qwen-1.5B student, NASH-MOPD scores 85.95% on MATH-500 avg@4, 23.29% on LCB avg@4, and 43.01% on Multi-IF. These exceed Fixed by 2.90, 1.72, and 2.76 percentage points, and D³-MOPD by 2.70, 4.00, and 1.47 points, respectively. Fixed has the highest AMC23 avg@16 score at 79.69%, compared with 78.91% for NASH-MOPD. Relative to the initial student’s 76.85% and the math teacher’s 86.10% MATH-500 avg@4 scores, NASH-MOPD achieves a gap recovery ratio of 98.4% Fu et al., [7], relative to the 9.25-point initial teacher–student performance gap, compared with 67.0% for Fixed. The remaining math gap is only 0.15 points, versus 3.05 points for Fixed at the final checkpoint.

Two-domain experiment. On the Qwen3-8B-Base student, NASH-MOPD scores 87.40% on MATH-500 avg@4 and 28.57% on LCB avg@4, exceeding Fixed by 0.70 and 1.71 points. It ties D³-MOPD on MATH-500 and exceeds its LCB score by 1.14 points. Mathematics and code means reach 63.56% and 58.06%, exceeding Fixed by 1.00 and 1.35 points, respectively. Both means also exceed D³-MOPD’s by 0.59 and 0.72 points, so the larger code improvement is accompanied by a gain in mathematics on the larger base student.

Convergence Speed

We measure convergence speed by the optimizer updates needed to match baseline endpoint mathematics and code performance. Figure 2 and Figure 3 report domain-score totals and checkpoint curves at 20-step intervals. Math and code use pass @1, whereas the endpoint tables use multi-candidate average accuracy.

Three-domain experiment. On the DeepSeek-R1-Distill-Qwen-1.5B student, NASH-MOPD’s step-100 math/code scores of 84.80/22.29% already exceed Fixed’s step-200 scores of 83.40/21.71% and D³-MOPD’s 83.40/18.86%. It therefore surpasses both baselines’ final mathematics and code performance with half the training updates. It also reaches all three teacher-referenced targets of 84.3/19.3/39.1% on math/code/Multi-IF @page 6/22 starts_mid=0 ends_mid=1

Method	MATH avg@4	AIME avg@16	AMC avg@16	Mean	LCB avg@4	HE+ avg@4	MB+ avg@4	Mean	Multi-IF score	IF-L score	IF-S score	Mean
Initial student	76.85	18.96	65.78	53.86	16.57	61.13	51.52	43.07	29.59	15.00	10.33	18.31
Fixed	83.05	26.04	79.69	62.93	21.57	68.14	56.68	48.80	40.25	16.67	12.33	23.08
D³-MOPD	83.25	26.04	79.06	62.78	19.29	66.31	57.01	47.54	41.54	18.00	13.67	24.40
NASH-MOPD	85.95	27.50	78.91	64.12	23.29	68.75	57.47	49.84	43.01	20.00	14.67	25.89

Method	MATH avg@4	AIME avg@16	AMC avg@16	Mean	LCB avg@4	HE+ avg@4	MB+ avg@4	Mean
Initial student	64.25	9.38	55.00	42.88	16.57	69.82	43.39	43.26
Fixed	86.70	22.71	78.28	62.56	26.86	83.08	60.19	56.71
D³-MOPD	87.40	21.67	79.84	62.97	27.43	82.62	61.97	57.34
NASH-MOPD	87.40	23.13	80.16	63.56	28.57	83.23	62.37	58.06

Qwen3-8B-Base student. Domain means are reported separately from these counts.

Three-domain experiment. On the DeepSeek-R1-Distill-Qwen-1.5B student, NASH-MOPD scores 85.95% on MATH-500 avg@4, 23.29% on LCB avg@4, and 43.01% on Multi-IF. These exceed Fixed by 2.90, 1.72, and 2.76 percentage points, and D³-MOPD by 2.70, 4.00, and 1.47 points, respectively. Fixed has the highest AMC23 avg@16 score at 79.69%, compared with 78.91% for NASH-MOPD. Relative to the initial student’s 76.85% and the math teacher’s 86.10% MATH-500 avg@4 scores, NASH-MOPD achieves a gap recovery ratio of 98.4% Fu et al., [7], relative to the 9.25-point initial teacher–student performance gap, compared with 67.0% for Fixed. The remaining math gap is only 0.15 points, versus 3.05 points for Fixed at the final checkpoint.

Two-domain experiment. On the Qwen3-8B-Base student, NASH-MOPD scores 87.40% on MATH-500 avg@4 and 28.57% on LCB avg@4, exceeding Fixed by 0.70 and 1.71 points. It ties D³-MOPD on MATH-500 and exceeds its LCB score by 1.14 points. Mathematics and code means reach 63.56% and 58.06%, exceeding Fixed by 1.00 and 1.35 points, respectively. Both means also exceed D³-MOPD’s by 0.59 and 0.72 points, so the larger code improvement is accompanied by a gain in mathematics on the larger base student.

Convergence Speed

We measure convergence speed by the optimizer updates needed to match baseline endpoint mathematics and code performance. Figure 2 and Figure 3 report domain-score totals and checkpoint curves at 20-step intervals. Math and code use pass @1, whereas the endpoint tables use multi-candidate average accuracy.

Three-domain experiment. On the DeepSeek-R1-Distill-Qwen-1.5B student, NASH-MOPD’s step-100 math/code scores of 84.80/22.29% already exceed Fixed’s step-200 scores of 83.40/21.71% and D³-MOPD’s 83.40/18.86%. It therefore surpasses both baselines’ final mathematics and code performance with half the training updates. It also reaches all three teacher-referenced targets of 84.3/19.3/39.1% on math/code/Multi-IF at step 120, only 20 updates after exceeding the baseline math/code endpoints, and sustains them at later measured checkpoints. Neither baseline reaches all three targets within 200 optimization updates.

Two-domain experiment. On the Qwen3-8B-Base student, NASH-MOPD reaches Fixed’s final math/code scores of 86.40/26.86% at step 100, scoring 88.20/28.57%, with margins of 1.80 and 1.71 points above these thresholds. Fixed and D³-MOPD first reach the same reference level at steps 160 and 180, respectively. NASH-MOPD thus uses 60 and 80 fewer updates, reductions of 37.5% and 44.4%, and remains above both thresholds through step 200. Figure 2 also shows that NASH-MOPD has the highest domain-score totals from step 100 onward for both students.

Training cost. Figure 4 reports full 200-step training time. Fixed, D³-MOPD, and NASH-MOPD take 6.63/6.49/6.81 hours on the DeepSeek-R1-Distill-Qwen-1.5B student and

12.97/12.97/13.21 hours on the Qwen3-8B-Base student, respectively. These totals exceed Fixed’s by 0.18 and 0.24 hours or 2.7% and 1.9%, respectively. We measure convergence speed in optimizer updates. Wall-clock time to the earlier performance crossings was not measured.

Additional Benchmarks and Candidate Coverage

AIME25 and AMC23 use sixteen candidates per question. HumanEval+ and MBPP+ [3, 2, 17] use four. IFBench Pyatkin et al. (2025) reports loose and strict accuracy from one response per prompt.

Additional benchmark accuracy. For the DeepSeek-R1-Distill-Qwen-1.5B student, NASH-MOPD has the highest AIME25 avg@16 at 27.50%, HumanEval+ avg@4 at 68.75%, and MBPP+ avg@4 at 57.47%. Its IFBench loose and strict scores are 20.00% and 14.67%, exceeding Fixed by 3.33 and 2.34 points, roughly 20% relative gains under both scoring criteria. For the Qwen3-8B-Base student, NASH-MOPD leads AIME25 avg@16 at 23.13%, AMC23 avg@16 at 80.16%, HumanEval+ avg@4 at 83.23%, and MBPP+ avg@4 at 62.37%.

Coverage under multiple candidates. Table 3 reports pass@4 for MATH-500, LiveCodeBench v6, HumanEval+, and MBPP+, and pass@16 for AIME25 and AMC23. NASH-MOPD is best or tied best on all six coverage metrics for the DeepSeek-R1-Distill-Qwen-1.5B student and five of six for the Qwen3-8B-Base student. On the DeepSeek-R1-Distill-Qwen-1.5B student, NASH-MOPD’s AIME25 coverage reaches 53.33%, compared with 36.67% for Fixed, a gain of about 16.7 points. MATH-500 and LCB coverage rise to 93.60% and 25.14%. On the Qwen3-8B-Base student, NASH-MOPD leads AIME25 at 46.67%, AMC23 at 97.50%, LCB at 32.00%, and MBPP+ at 80.95%, and ties the best MATH-500 coverage at 94.00%. On HumanEval+ for the Qwen3-8B-Base student, $D^{3}$-MOPD has higher pass@4 coverage than NASH-MOPD at 92.07% versus 91.46%, despite lower avg@4 accuracy at 82.62% versus 83.23%. Here, NASH-MOPD’s 0.61-point average-accuracy gain coexists with a 0.61-point coverage shortfall, illustrating that the two metrics can rank methods differently.

Method	Math				Code
	MATH pass@4	AIME pass@16	AMC pass@16	Mean	LCB pass@4	HE+ pass@4	MB+ pass@4	Mean
DeepSeek-R1-Distill-Qwen-1.5B
Initial student	88.00	43.33	92.50	74.61	19.43	78.66	67.20	55.10
Fixed	92.80	36.67	95.00	74.82	24.00	81.10	67.72	57.61
D³-MOPD	91.40	50.00	95.00	78.80	21.71	80.49	70.11	57.44
NASH-MOPD	93.60	53.33	95.00	80.64	25.14	82.32	70.37	59.28
Qwen3-8B-Base
Initial student	86.60	33.33	87.50	69.14	19.43	87.20	75.40	60.68
Fixed	93.80	43.33	92.50	76.54	29.71	90.24	80.69	66.88
D³-MOPD	94.00	36.67	95.00	75.22	30.86	92.07	79.37	67.43
NASH-MOPD	94.00	46.67	97.50	79.39	32.00	91.46	80.95	68.14

Conclusion

NASH-MOPD allocates prompts by maximizing the smallest predicted absorption rate of the remaining teacher–student reverse-KL gaps. Under a separable, frozen-rate progress model, the unconstrained equilibrium follows an inverse-rate rule. Online rate estimates, bounded shares, and smoothing implement this allocation during training. The DeepSeek-R1-Distill-Qwen-1.5B student reaches and sustains all three teacher-referenced targets from step 120, while the baselines remain short of the joint target through step 200. The Qwen3-8B-Base student reaches Fixed’s final math/code performance at step 100, requiring 37.5% and 44.4% fewer updates than Fixed and $D^{3}$-MOPD for first attainment. At the endpoint, NASH-MOPD leads eight of nine metrics on the DeepSeek-R1-Distill-Qwen-1.5B student and is best or tied best on all six metrics on the Qwen3-8B-Base student, with no extra teacher queries or rollouts.

Limitations

NASH-MOPD relies on local, separable progress estimates. Future work could model cross-domain transfer and forgetting, quantify rate uncertainty, and handle delayed feedback. Incorporating token counts and training time could improve resource allocation. Learning sampling bounds, update intervals, and smoothing could reduce manual tuning. More domains, model sizes, and training budgets would test these extensions.

Theory of the Allocation Game

Local surrogate and the zero-sum game

At one scheduler update, fix normalized gaps $g_k > 0$ and estimated per-prompt reductions $a_k > 0$ for the active domains. A batch of $B$ prompts with allocation $p \in\Delta_K$ predicts gap reduction $Bp_k a_k$ and fractional progress $Bp_k r_k$, where $r_k = a_k/g_k$. Ignoring truncation at the target, define the local time surrogate

This is a local planning quantity, not a guarantee for nonlinear training: student parameters, responses, losses, and rates can change after each update.

The sampler is the maximizing player and the adversary is a distribution $q \in\Delta_K$ over domains. Their zero-sum payoff is

For fixed $p$, the adversary places all mass on an index attaining $\min_k p_k r_k$, so $\min_{q \in\Delta_K} u(p,q) = \min_k p_k r_k$. The sampler’s problem is the linear maximin program

The minimum payoff requires fractional progress in every domain. Theorem A.1 therefore also minimizes worst-domain local time under frozen rates; the online controller re-estimates these rates after subsequent updates.

Bounded shares and water-filling

The implementation imposes a floor $\ell$ and cap $u$ on domain sampling ratios. We assume $K\ell\le1 \le Ku$. Let $\mathcal{A}$ contain domains with a gap above the target threshold and a positive finite estimated rate; let $\mathcal{I}$ be the other domains. The code fixes $p_i = \ell$ for $i \in\mathcal{I}$ and maximizes the worst rate over $\mathcal{A}$ using the remaining mass $b = 1 - |\mathcal{I}|\ell$. This convention is part of the controller: a zero-rate domain is not assigned a modeled positive payoff, so its share is a per-domain floor rather than a maximin guarantee.

For $|\mathcal{A}| > 0$, define

When $b \le|\mathcal{A}|u$, the bisection in the implementation finds a level $v$ for which $\phi(v) = b$ (up to numerical tolerance), and sets the positive-rate shares to the clipped values in Eq. (15). If $b > |\mathcal{A}|u$, all positive-rate domains receive $u$ and the forced remainder is shared among inactive domains. The following result states what this construction does and, just as importantly, what its scalar $v$ does not mean when a cap binds.

Proof. Write $m(p) = \min_{k\in\mathcal{A}} p_k r_k$. First consider a solution of Eq. (15). If an upper-clipped domain has payoff below $v$, choose $j$ with the smallest payoff $u r_j$ among such domains. Every feasible allocation then has $m(p) \le u r_j$. The clipped vector assigns $p_j = u$; every uncapped domain has payoff $v$, every floor-clipped domain has payoff at least $v$, and every other upper-clipped domain has payoff at least $u r_j$. Its minimum therefore equals this upper bound, so no allocation can do better. If no upper-clipped domain has payoff below $v$, all domain payoffs under the clipped vector are at least $v$. Any allocation with minimum payoff strictly above $v$ must assign more than $v/r_k$ to every domain with $\ell\le v/r_k < u$, including a domain exactly at its floor. Domains with $v/r_k < \ell$ can remain at their floor and already have payoff above $v$. If some domain has $v/r_k = u$, strict improvement is impossible for that domain because of its cap. Otherwise, when at least one domain needs a strict increase, the required total exceeds the clipped vector’s mass $b$. If all domains have $v/r_k < \ell$, feasibility fixes every share at its floor. Thus no feasible vector has a larger minimum payoff. Finally, when $b > |\mathcal{A}|u$, every active coordinate is already at its maximum, so no feasible allocation can increase any positive-rate domain’s share; the minimum active payoff is therefore maximal. The inactive residual is feasible because $K\ell\le1 \le Ku$.

The bounded objective has the same domain-adversary interpretation: for fixed shares, minimizing the bilinear payoff over distributions on $\mathcal{A}$ selects an active bottleneck. The implementation needs only the maximizing allocation from Theorem A.2; it does not compute a dual distribution or alternate between game players.

The level $v$ equalizes payoffs only for interior coordinates. A capped domain can have payoff $u r_k < v$ and become the bottleneck; plateaus of $\phi$ can make $v$ non-unique. Hence $v$ should not be interpreted as an empirical Nash value when constraints bind.

For a feasible active budget, the scalar search is well-defined: $\phi$ is continuous and nondecreasing, with $\phi(0) = |\mathcal{A}|\ell$ and $\phi(u\max_{k\in\mathcal{A}} r_k) = |\mathcal{A}|u$. The implementation uses 80 bisection iterations within this bracket. Share clipping and a final projection handle finite precision; the root level describes the bounded allocation, not a measured convergence rate.

Sensitivity to estimated positive rates

The controller uses window estimates $\hat r_k$. The following result quantifies how a specified error changes its local planning objective. It compares the same active domain set and feasible share constraints; changes in which domains pass the positive-rate test are addressed separately in Section A.5.

Proof. For every feasible $p$, $0 \le p_k \le1$ bounds the change in each active product by $\epsilon$. Taking the minimum preserves that bound, so $|\min_{k\in\mathcal{A}}p_k r_k-\min_{k\in\mathcal{A}}p_k\hat r_k| \le\epsilon$. Maximizing over the same compact set also gives $|\nu_{\mathcal{A}}(r)-\nu_{\mathcal{A}}(\hat r)| \le\epsilon$. The estimated optimizer therefore satisfies

This result applies to the bounded target before temporal smoothing. It is conditional on a specified perturbation of the local rates. Our moving-window EMA estimator is temporally dependent, and the experiments do not supply a calibrated bound on $\epsilon$. Consequently the proposition explains sensitivity of the planning objective without assigning a confidence level or certifying progress for the reported training runs.

Smoothing, batch-level jitter, and finite batches

Let $q_t$ be the bounded target, $p_t$ the smoothed allocation, and $\alpha\in(0,1]$ the smoothing coefficient. Convex combinations of feasible shares remain feasible, so Euclidean projection is a numerical safeguard after entry into the capped simplex. It also enforces bounds at the first dynamic update, since the instruction-following warm-up share is below 0.1. The lemma assumes a feasible initial allocation.

Proof. Euclidean projection onto a closed convex set is non-expansive and fixes $q$. Therefore

which gives the contraction. For a varying target, compare the projected update with $q_t$ and apply the triangle inequality:

Iterating this scalar recursion gives a geometric transient plus a geometric sum bounded by $(1-\alpha)\xi/\alpha$.

The lemma concerns the tracking of target shares. It supplies no contraction of the student’s KL gap or benchmark error. The runs use $\alpha=0.25$; we do not estimate a stationary target-variation bound from their traces.

For one batch, the implementation multiplies each share $p_k$ by $1+\eta_k$, with $\lvert\eta_k\rvert\le\gamma$, and renormalizes. Let $\tilde p$ be this jittered distribution. Since

For a batch of $B$ prompts, largest-remainder apportionment first assigns $\lfloor B\tilde p_k\rfloor$ examples to each domain and then distributes the remaining prompts according to fractional remainders. If $n_k$ is the resulting integer quota, then

The bound is deterministic and concerns batch composition only. It does not bound gradient noise or model quality. The random perturbations are sampled independently per batch. The sampler is shared by Fixed, D$^3$-MOPD, and NASH-MOPD in both the DeepSeek-R1-Distill-Qwen-1.5B and Qwen3-8B-Base experiments.

Zero-rate domains and the scope of the guarantee

The bounded solver marks domain $k$ as progressing only if $g_k>10^{-3}$, $\hat r_k>10^{-8}$, and its rate is finite. A domain outside this set receives its per-domain floor when a feasible positive-rate allocation can consume the remaining probability. If upper bounds prevent that allocation from consuming all mass, the residual is split equally over the other domains. When no domain passes the progressing-domain test, the target is uniform. These rules make every sampler update defined without dividing by zero.

They also change the scope of the optimization. If an unfinished domain has $r_k=0$, then $\min_k p_k r_k=0$ for every feasible $p$ in the full-domain game, and its modeled completion time is infinite. No allocation can certify positive all-domain progress from those rates. The controller instead optimizes on the positive-rate subset while retaining exposure to the other domains. A zero estimate can represent a noisy window, a plateau, interference, or an inadequate teacher; the observations do not identify which explanation holds. In particular, allocating the minimum share to a zero-rate domain should not be interpreted as the best recovery policy for a stalled domain.

The diagonal local model additionally assumes that a domain’s improvement is attributable to its own prompt exposure. A shared student can improve one domain from another domain’s data or suffer negative transfer. The present analysis does not model those interactions, prove monotone KL descent, or establish a connection from KL descent to a benchmark threshold. The theoretical statements therefore concern allocation under specified local rates, whereas the results test whether that controller is useful in the observed nonlinear training system.

Implementation Details

This appendix complements Algorithm 1 with the controller state, comparator settings, and shared training settings.

State, observations, and update order

The controller stores a domain-specific random row permutation, cursor, prompt exposure count, initial KL scale, EMA, and recent histories. Each domain’s raw signal is recovered from the dense token-level distillation reward and aggregated over its non-padding response tokens. Negative raw estimates are clipped to zero before entering the EMA. The first valid observation initializes the EMA; later observations use coefficient 0.2. The initial scale is the mean of the first five raw observations, floored at $10^{-8}$. At the fifth observation, the gap history is initialized at one; subsequent records use the normalized remaining gap from the main text. NASH-MOPD has no upper clipping of this gap.

A rolling history retains up to 11 exposure–gap pairs. Its oldest and newest entries provide a gap decrement over up to 10 observed updates. The nonnegative decrement divided by the exposure difference yields the estimated exposure-normalized descent velocity, and division by $\max(g_k, 10^{-3})$ gives the exposure-normalized absorption rate. The exposure denominator is floored at one. Rates and gap observations are updated throughout warm-up, but sampling shares change only once every domain has a scale and at least two history points. Dynamic updates begin at step 20 and occur at 10-step intervals. The logged post-update mixture controls subsequent batches; the current batch has already been sampled. The teacher signal in row $s$ is measured before optimizer update $s$.

The sampler uses one random-number generator for row permutations, batch-level jitter, and within-batch ordering. It consumes each domain’s current row permutation before reshuffling that pool. Thus it permits repeated presentations across training while avoiding independent sampling with replacement at every draw within a cycle. Exposure counts measure prompt presentations, including repeats, rather than distinct training examples. Resume checkpoints retain the sampler’s random-number state, current permutations, cursors, yielded count, histories, EMA, scales, and last update. Resuming a trainer segment continues this state instead of restarting warm-up.

struction following and recomputes row proportions.

Bounded allocation and comparator details

Nash-MOPD solves for a clipped inverse-rate target by 80 bisection iterations over $[0, u \max_k \tilde{r}_k]$, then normalizes its finite-precision result. It blends this target with the previous mixture using $\alpha= 0.25$ and projects the result onto the capped simplex. The projection itself uses 80 bisection iterations. Share floors and caps apply to the controller’s base mixture after dynamic updates; the jittered probabilities and rounded batch fractions can exceed those nominal coordinate bounds. The DeepSeek-R1-Distill-Qwen-1.5B row-proportional warm-up also has an instruction-following base share below 0.1.

Each batch applies independent multiplicative jitter in $[0.7, 1.3]$ to the scheduled shares and renormalizes them. Largest-remainder rounding converts the resulting probabilities into integer quotas summing to the global prompt batch size $B$. Fixed and dynamic schedules use the same batch construction and random-permutation sampler.

The D$^{3}$-MOPD comparator maintains an independent smoothed-KL history of up to 41 observations. Its remaining-gap signal is $\bar{L}_k = \operatorname{clip}(L_k/L_k^{(0)}, 0, 1.5)$. The EMA and reference scale follow the initialization above. Descent velocity $v_k$ averages the available 10-step differences from at most three windows, with previous KL floored at 0.15, and truncates negative descent to zero. Its composite signal multiplies the remaining-gap and descent-velocity signals, divides by their maximum, uses softmax temperature 0.5, adds the probability floor, and applies the same cap. If all signals vanish, it proposes a uniform target. It uses no additional temporal blend. This baseline uses the original D$^{3}$-MOPD scheduling method, combining remaining gap and descent velocity to determine the domain sampling probabilities. Eq. (3) describes the proposal before the sampling floor and cap are applied. Both adaptive schedules retain the row-proportional base mixture during the shared warm-up and use the same batch-level jitter and integer-quota sampler; only Nash-MOPD uses the temporal blend in Eq. (10).

Fixed keeps the usable-row proportions throughout training. In the DeepSeek-R1-Distill-Qwen-1.5B experiment these proportions are approximately 38.2% math, 52.7% code, and 9.1% instruction following, computed from 17,917, 24,699, and 4,245 loaded rows. Qwen3-8B-Base removes instruction following and recomputes row proportions.

Shared training and decoding settings

Schedules within each experiment share initialization, teachers, data, AdamW settings, and a global batch of 64 with one student rollout per prompt. Training uses learning rate $10^{-6}$, weight decay 0.01, up to 4,096 prompt tokens and 7,168 response tokens, on A100-SXM4-80GB GPUs. Held-out decoding uses temperature 0.6 and top-$p$ 0.95, except for IFBench. Table 4 collects the training and scheduling settings.

Setting	Value	Applied to
Global prompt batch / responses per prompt	64 / 1	All training runs
Maximum prompt / response tokens	4,096 / 7,168	All training runs
Student optimizer / learning rate / weight decay	AdamW / $10^{-6}$ / 0.01	All training runs
Training rollout / teacher temperature	1.0 / 1.0	All training runs
Loss aggregation / teacher precision	Token mean / bfloat16	All training runs
KL scale observations / EMA coefficient	5 / 0.2	Domain-sampler controllers
KL target ratio / remaining-gap floor	0.10 / $10^{-3}$	NASH-MOPD signals
Positive-rate cutoff	$10^{-8}$	NASH-MOPD allocation
Per-domain floor / cap	0.10 / 0.70	Dynamic post-warm-up mixtures
Warm-up / update interval	20 / 10 steps	Dynamic schedules
Rate-history window	10 observations	NASH-MOPD rate estimation
Descent-velocity windows / KL denominator floor	Up to $3 \times10$ / 0.15	D$^3$-MOPD comparator
Softmax temperature / remaining-gap cap	0.5 / 1.5	D$^3$-MOPD comparator
Mixture smoothing	0.25	NASH-MOPD only
Batch-level jitter	Uniform [$-0.3$, 0.3]	Domain-sampler conditions
Scalar allocation / projection iterations	80 / 80	Bounded solver
Primary evaluation interval	20 optimizer steps	Main scheduler comparisons
Primary evaluation temperature / top-$p$	0.6 / 0.95	Shared held-out domains

Supplementary Results and Mechanism Analysis

Figure 5 compares training of the student model DeepSeek-R1-Distill-Qwen-1.5B over 200 optimizer steps, each representing one parameter update. The three domains are mathematics, code generation, and instruction following, which respectively involve solving mathematical problems, producing programs, and satisfying prompt requirements. Fixed samples domains in proportion to their training corpus sizes; D$^{3}$-MOPD adjusts sampling using remaining teacher–student divergence and its recent decrease; NASH-MOPD balances predicted proportional gap reductions across domains. A domain sampling ratio is the scheduled probability of selecting a prompt from that domain. Reverse Kullback–Leibler divergence, abbreviated KL, measures the discrepancy between the student’s and the domain teacher’s next-token distributions on student-generated responses. The remaining KL gap is the nonnegative difference between the smoothed divergence and 10% of its initial scale, divided by 90% of that scale. Smoothing assigns weight 0.2 to the latest observation and 0.8 to the previous smoothed value; the initial scale is the mean of the first five nonnegative observations. A gap of one represents the initial scale, and zero represents the target divergence.

First, the upper row shows how the methods distribute training prompts. Fixed maintains ratios of 38.2%, 52.7%, and 9.1% for mathematics, code generation, and instruction following. In contrast, NASH-MOPD increases the mathematics ratio to 58.3% by step 120 while reducing the code generation ratio to 28.6%. D$^{3}$-MOPD makes a larger shift toward instruction following after step 100, assigning it 58.2% at step 160, compared with 18.3% under NASH-MOPD. Thus, NASH-MOPD sustains a larger mathematics allocation, whereas D$^{3}$-MOPD concentrates more prompts on instruction following during the middle and later stages.

Correspondingly, the lower row shows distinct patterns of gap reduction. At step 200, the mathematics gap is 0.051 under NASH-MOPD, compared with 0.091 under D$^{3}$-MOPD and 0.131 under Fixed. Code generation shows the reverse ordering: Fixed reaches zero, while D$^{3}$-MOPD and NASH-MOPD finish at 0.008 and 0.034. Meanwhile, the instruction-following gap fluctuates around one during early training and declines more strongly under D$^{3}$-MOPD after its sampling ratio increases. Its final gap is 0.277 under D$^{3}$-MOPD, 0.605 under NASH-MOPD, and 1.636 under Fixed. These trajectories show the domain trade-offs directly: NASH-MOPD achieves the smallest mathematics gap, Fixed the smallest code generation gap, and D$^{3}$-MOPD the smallest instruction-following gap.

Next, Figure 6 compares predictions of gap reduction over 18 ten-step windows starting at steps 20, 30, $\ldots$, 190. For domain $k$ and starting step $s$, let $\hat{a}_{k,s}$ be the estimated gap decrease per prompt, computed by dividing the nonnegative gap decrease over up to ten observed updates by the number of prompt presentations during those updates. Let $\Delta e_{k,s}$ count prompt presentations over the next ten steps, including repeated presentations. Holding the starting estimate fixed gives the predicted reduction

The observed reduction $\Delta g_{k,s}$ is the starting gap minus the ending gap, so a negative value indicates an increase. Task-macro correlation is the equal-weight mean of the three domain-specific Spearman rank correlations between predicted and observed reductions over the 18 windows. Spearman correlation measures agreement between their rankings, with larger values indicating closer agreement. Mean absolute error is the mean of $\left|\widehat{\Delta g}_{k,s}-\Delta g_{k,s}\right|$ over all three domains and 18 windows; smaller values indicate closer predicted magnitudes.

Finally, both panels place NASH-MOPD ahead of the other methods. NASH-MOPD has the highest task-macro correlation at 0.421, followed by Fixed at 0.400 and D³-MOPD at 0.290. The corresponding correlation advantages are 0.021 and 0.130. NASH-MOPD also has the lowest mean absolute error at 0.0739, compared with 0.1008 for Fixed and 0.0812 for D³-MOPD, reductions of 26.6% and 8.9%, respectively. Together, the panels show that NASH-MOPD’s gap-reduction predictions achieve both closer agreement in ranking and smaller magnitude errors when averaged across the three domains.

Case Studies

DeepSeek-R1-Distill-Qwen-1.5B: Mathematical Reasoning

Question (MATH-500). A steel sphere with a 3-inch radius is made by removing metal from the corners of a cube that has the shortest possible side lengths. How many cubic inches are in the volume of the cube?

Reference answer: 216.

Initial Student (Incorrect)

The response states that an inscribed sphere “touches the corners of the cube,” and equates its diameter to the cube’s space diagonal:

[. . . intermediate reasoning omitted . . .]

It cubes this edge length:

Model answer: $24\sqrt{3}$.

Check. The sphere must fit inside the cube. Its diameter equals the shortest cube’s edge, rather than its space diagonal. The proposed edge $2\sqrt{3}<6$ cannot contain the sphere.

Fixed (Incorrect)

The response models the removal of corners using a truncation plane $x+y+z=3\sqrt{3}$. It then sets a corner-to-plane distance equal to the sphere’s radius:

[. . . intermediate reasoning omitted . . .]

It selects $s=4\sqrt{3}$ and computes

Model answer: $192\sqrt{3}$.

Check. The added corner-to-plane constraint does not determine the smallest enclosing cube. The resulting edge is larger than the required diameter 6.

D³-MOPD (Incorrect)

The response assumes a truncation plane $x+y+z=a$ and uses the distance from the cube’s center $(a/2,a/2,a/2)$ to that plane:

[. . . intermediate reasoning omitted . . .]

It evaluates $(6\sqrt{3})^3=648\sqrt{3}$.

Model answer: $648\sqrt{3}$.

Check. This assumed plane imposes an unsupported geometric condition. The shortest enclosing cube is determined by tangency to its six faces, so its edge is 6, not $6\sqrt{3}$.

NASH-MOPD (Correct)

The response identifies the relevant tangency: “The sphere is inscribed inside the cube such that it touches the centers of the faces of the cube.” It therefore equates the cube’s edge to the sphere’s diameter:

[. . . intermediate reasoning omitted . . .]

It answers the requested volume, rather than a diagonal or the sphere’s volume: $s^3=6^3=216$.

Model answer: 216.

Check. The radius-3 sphere fits inside a cube of edge 6 and touches all six faces. Any smaller edge would exclude part of the sphere.

DeepSeek-R1-Distill-Qwen-1.5B: Code Generation

Question (LiveCodeBench v6; condensed). Given an uppercase string $S$ of length 3–100, count triples $1 \le i < j < k \le |S|$ such that $j-i=k-j$ and $S_i=A$, $S_j=B$, $S_k=C$. Read $S$ from standard input and print the count. For AABCC, the answer is 2; for AABAAABBAEDCCCD, it is 4.

Reference criterion: pass all 43 saved tests.

Initial Student (Incorrect)

[Reasoning and explanatory prose omitted.]

S = input().strip()
count = 0
n = len(S)
for d in range(1, n - 1):
    for j in range(1, n - d + 1):
        i = j - d
        k = j + d
        if S[i] == 'A' and S[j] == 'B' and S[k] == 'C':
            count += 1
print(count)

Recorded verdict: fail; 38/43 tests pass in the local recheck.

Check. The bounds allow $k=n$, outside Python’s valid indices. AABAAABBAEDCCCD raises IndexError instead of printing 4.

Fixed (Incorrect)

The response repeatedly revisits the indexing and attempts to count the example AABCC. One pass gives 3 instead of the reference 2. It notices this mismatch: “So perhaps my approach is still wrong.”

[. . . intermediate reasoning omitted . . .]

The saved response ends while reconsidering the loop bounds:

“So for len(S)=5, j runs from 1 to4.

In j=4:

i can”

Recorded verdict: fail; no complete code block.

Check. The saved generation contains no complete delimited Python program. It is an unfinished response, so no runnable solution can be submitted under the benchmark’s extraction rule.

D³-MOPD (Incorrect)

[Reasoning and explanatory prose omitted.]

s = input().strip()
n = len(s)
count = 0
for j in range(n):
    max_d = n - j - 1
    for d in range(1, max_d + 1):
        i = j - d
        k = j + d
        if s[i] == 'A' and s[j] == 'B' and s[k] == 'C':
            count += 1
print(count)

Recorded verdict: fail; 42/43 tests pass in the local recheck.

Check. The upper bound constrains only the right endpoint. For $d>j$, $i=j-d$ is negative and Python reads from the end of the string. The saved test AABAAABBAEDCCCD produces 5 rather than 4.

NASH-MOPD (Correct)

[Reasoning and explanatory prose omitted.]

s = input().strip()
count = 0
n = len(s)
for j in range(2, n):
    max_d = min(j - 1, n - j)
    for d in range(1, max_d + 1):
        i = j - d
        k = j + d
        if i < 1 or k > n:
            continue
        if s[i-1] == 'A' and s[j-1] == 'B' and s[k-1] ==
'C':
            count += 1
print(count)

Recorded verdict: pass; 43/43 tests pass in the local recheck.

Check. The response uses one-based mathematical indices consistently and converts them to Python indices at access time. The bound $d \le\min(j-1,n-j)$ keeps both endpoints inside the string. It prints 4 on the same counterexample.

DeepSeek-R1-Distill-Qwen-1.5B: Instruction Following

Question (IFBench; original English prompt). Generate 4 multiple choice questions with 5 options each about “20th century art history”. Each question should start with the label “Question”. The questions should get progressively longer. Do not provide an explanation.

Reference criterion: the official custom:mcq_count_length check. Length is the character count of question text before its options.

Initial Student (Incorrect)

[Reasoning and the five options per question omitted; all question texts retained.]

Question 1: The Great Depression and World War II had a significant impact on 20th century art. Which of the following best describes this impact?

Question 2: The 1950s and 1960s were a time of digital art and computer use. Which movement emerged during this period?

Question 3: The 1970s and 1980s were a time of experimentation and new art movements. Which movement emerged during this period?

Question 4: The 1990s and 2000s brought about the digital revolution and the rise of digital art. Which of the following best describes this period?

Recorded verdict: strict fail; loose fail.

Check. The question lengths are 134, 107, 116, 136 characters. The second question is shorter than the first, so the required increasing-length sequence fails.

Fixed (Incorrect)

[Reasoning and the five options per question omitted; all question texts retained.]

Question 1: Which of the following best describes the origins of 20th century art?

Question 2: What was a key characteristic of Expressionism in 20th century art?

Question 3: How did the Great Depression and World War II shape 20th century art?

Question 4: What is a major concept in 20th century art history?

Recorded verdict: strict fail; loose fail.

Check. The question lengths are 70, 67, 69, 52 characters. They decrease at the first transition and again at the last transition.

$D^3$-MOPD (Incorrect)

[Reasoning and the five options per question omitted; all question texts retained.]

Question Which artist is best known for creating abstract geometric patterns in the 20th century?

Question Which period in the 20th century saw the most significant contributions to modern art?

Question Which of the following artists is associated with the use of color in their work during the 20th century?

Question Which of the following movements was most influential in shaping 20th century art?

Recorded verdict: strict fail; loose fail.

Check. The question lengths are 88, 86, 105, 82 characters and are not strictly increasing. It also omits question numbers expected by the official parser.

NASH-MOPD (Correct)

[Reasoning and the five options per question omitted; all question texts retained.]

Question 1: What was the first major movement in 20th century art history?

Question 2: Which of the following is a key figure in the Realism movement?

Question 3: What theory emerged in the 20th century that influenced modern art?

Question 4: Which of the following is a significant theory in 20th century art history?

Recorded verdict: strict pass; loose pass.

Check. The question lengths are 62, 63, 67, 75 characters, so all three transitions increase. The original response has four numbered questions, each with five options, and no explanatory prose. This verdict checks the requested format and lengths, rather than the factual quality of the quiz.

Qwen3-8B-Base: Mathematical Reasoning

Question (MATH-500). How many of the same digits are found in the base 7 and base 8 representations of $629_{10}$? For example, $121_{3}$ and $413_{5}$ would have one digit in common.

Reference answer: 3.

Initial Student (Incorrect)

The response describes repeated division and supplies a conversion function, then claims the following output:

([1, 6, 0, 2], [1, 1, 4, 5], {1}, 1)

[… intermediate reasoning omitted …]

It concludes that the representations are $1602_{7}$ and $1145_{8}$, and that only the digit 1 appears in both.

Model answer: 1.

Check. The claimed conversions are incorrect: $1602_{7} = 639$ and $1145_{8} = 613$. The actual representations are $1556_{7}$ and $1165_{8}$.

Fixed (Incorrect)

The response correctly obtains $1556_{7}$ and $1165_{8}$, but compares every occurrence of a digit in one representation with every occurrence in the other. It lists five matches:

“1 vs 1” (twice), “5 vs 5” (twice), and “6 vs 6”.

[… intermediate reasoning omitted …]

It decides that matches at different positions should count separately.

Model answer: 5.

Check. The question counts distinct shared digits, as its example indicates. Repeated occurrences of 1 and 5 must not increase the count.

D³-MOPD (Incorrect)

The response also correctly converts the number to $1556_{7}$ and $1165_{8}$. It aligns their positions and compares 1 with 1, 5 with 1, 5 with 6, and 6 with 5.

[… intermediate reasoning omitted …]

It concludes: “Only the first digit is the same.”

Model answer: 1.

Check. Shared digits need not occur in the same position. Digits 5 and 6 occur in both representations, so positional comparison misses two valid shared digits.

NASH-MOPD (Correct)

The response computes $629_{10} = 1556_{7} = 1165_{8}$. It initially considers a positional comparison, then revisits the meaning: “Alternatively, perhaps I should consider that the digits are the same regardless of their positions, just looking for any matching digits.”

[… intermediate reasoning omitted …]

It identifies 1, 5, and 6 in both representations, and explicitly counts each digit only once despite repeated occurrences.

Model answer: 3.

Check. The distinct digit sets are both $\{1, 5, 6\}$. Their intersection has size 3, matching the question’s example.

Qwen3-8B-Base: Code Generation

Question (LiveCodeBench v6; condensed). Implement Solution.largestInteger(nums, k). An integer is “almost missing” if it appears in exactly one contiguous subarray of size k. Return the largest such integer, or −1 if none exists. Constraints: $1 \leq|\mathrm{nums}| \leq50$, $0 \leq\mathrm{nums}[i] \leq50$, and $1 \leq k \leq|\mathrm{nums}|$.

Reference criterion: pass all 34 saved tests. Repeated occurrences inside one subarray count as one containing subarray.

Initial Student (Incorrect)

Its first code block accumulates each occurrence: [Surrounding code and prose omitted.]

for subarray in subarrays:
    for num in subarray:
        count[num] += 1

The last code block contains only example calls:

solution = Solution()
print(solution.largestInteger([3, 9, 2, 1, 7], 3))  #
    Output: 7
print(solution.largestInteger([3, 9, 7, 2, 1, 7], 4))  #
    Output: 3
print(solution.largestInteger([0, 0], 1))  # Output: -1

Recorded verdict: fail; 0/34 tests pass in the local recheck.

Check. The benchmark extracts the last complete code block, which has no class definition and raises NameError. The earlier implementation also counts repeated elements within one subarray multiple times.

Fixed (Incorrect)

[Imports, signature, comments and prose omitted.]

subarrays = []
for i in range(len(nums) - k + 1):
    subarrays.append(nums[i:i + k])
from collections import defaultdict
count_dict = defaultdict(int)
for subarray in subarrays:
    for num in subarray:
        count_dict[num] += 1
largest_integer = -1
for num in sorted(count_dict.keys(), reverse=True):
    if count_dict[num] == 1:
        largest_integer = num
        break
return largest_integer

Recorded verdict: fail; 30/34 tests pass in the local recheck.

Check. For nums = [0] * 50 and k = 50, there is only one subarray. The correct answer is 0, but the code counts 50 occurrences and returns −1.

D³-MOPD (Incorrect)

[Imports, signature, comments and prose omitted.]

subarrays = []
for i in range(len(nums) - k + 1):
    subarrays.append(nums[i:i + k])
count_dict = {}
for num in nums:
    count_dict[num] = 0
for subarray in subarrays:
    for num in subarray:
        if num in count_dict:
            count_dict[num] += 1
max_num = -1
for num, count in count_dict.items():
    if count == 1 and num > max_num:
        max_num = num
return max_num

Recorded verdict: fail; 30/34 tests pass in the local recheck.

Check. The membership check tests whether a dictionary entry exists; it does not deduplicate elements within a subarray. Thus the same saved counterexample, nums = [0] * 50, k = 50, returns −1 instead of 0.

NASH-MOPD (Correct)

[Imports, signature, comments and prose omitted.]

subarrays = []
for i in range(len(nums) - k + 1):
    subarrays.append(nums[i:i + k])
count_dict = {}
for num in nums:
    count = 0
    for subarray in subarrays:
        if num in subarray:
            count += 1
    count_dict[num] = count
result = -1
for num, count in count_dict.items():
    if count == 1 and num > result:
        result = num
return result

Recorded verdict: pass; 34/34 tests pass in the local recheck.

Check. Each subarray contributes at most one count for a given integer via num in subarray. It follows, for the all-zero counterexample, because 0 belongs to exactly one size-50 subarray. It then selects the largest integer whose containing-subarray count equals one.