Large language models trained with Reinforcement Learning (RL) with verifiable rewards exhibit strong reasoning ability and broad generalization, whereas models trained with Supervised Fine-Tuning (SFT) are often viewed as more prone to memorization and limited transfer. This paper rethinks this distinction through the lens of SFT training data on widely used Qwen2.5 and Qwen3 series models. First, we study the role of data source and show that it is critical: a carefully mixed SFT dataset substantially outperforms data generated solely by a larger model. Second, we study the role of data scale and show that matching the number of correct rollouts between SFT and RL greatly improves SFT generalization, while matching the total rollout budget enables SFT to generalize as well as RL. Combining these two factors further enables SFT to generalize even better than RL. Third, by using LLM annotations to characterize the solution methods in training rollouts, we show that larger datasets cover more tail methods and that these tail methods provide generalizable reasoning signals. Finally, we support these empirical findings theoretically by analyzing the training dynamics of shallow transformers under both RL and SFT.
Introduction
Large Language Models (LLMs) trained via Reinforcement Learning (RL) or Supervised Fine-Tuning (SFT) have achieved remarkable improvements in reasoning ability, as exemplified by the success of DeepSeek-R1 [1] and s1 [2]. A line of work compares the strengths and limitations of RL and SFT for reasoning [3, 4, 5]. These studies suggest that “RL generalizes, whereas SFT memorizes” [6, 5], indicating that SFT often struggles to match RL on reasoning tasks beyond the training domain.
However, it remains unclear whether the conclusion that “RL generalizes, SFT memorizes” is drawn in a setting where the full potential of SFT is realized and SFT is given comparable data resources to RL. For instance, the amount of training data for SFT is not aligned with that of RL training. In this work, we revisit this conclusion through the lens of the training dataset. Specifically, we study how data source and scale affect SFT generalization, and ask:
What kind of data source best unlocks the generalization potential of SFT?
How should we match the data consumed by SFT and RL to compare their generalization in a fairer setting?
To answer the first question, we perform SFT using training data from two sources and their mixtures, and evaluate the resulting models on general reasoning tasks to assess generalization. Specifically, we construct SFT datasets from rollouts of two models: the strong Qwen3-14B model and the RL-tuned base model, pairing each question prompt with one correct rollout. Although both SFT-trained models generalize worse than the RL-trained model, the RL-tuned dataset yields much better generalization than the Qwen3-14B dataset. Moreover, interpolating between the two sources shows that adding an appropriate amount of more diverse data further improves general reasoning. These findings highlight the importance of data source for SFT generalization.
To answer the second question, we match the SFT training dataset size to that of RL under different criteria. Rather than matching SFT and RL training data prompt-wise, we scale the SFT dataset according to three criteria: (1) the number of solvable prompts, (2) the number of correct rollouts, and (3) the total number of rollouts. After training and evaluating models on these datasets, we find that matching the number of correct rollouts used in RL greatly improves SFT generalization, while further matching the total number of rollouts enables SFT to achieve generalization comparable to RL. Moreover, data source and data scale have a compounding effect: SFT with a mixture of the two data sources and a matched number of total rollouts achieves better generalization than RL.
To further demystify the role of data scale, we study how scaling the SFT dataset affects the coverage of LLM-annotated solution methods in training rollouts and, in turn, general reasoning performance. Our analysis shows that larger datasets cover more tail methods in the long-tailed method distribution. By further annotating evaluation rollouts on general reasoning tasks, we find that these tail methods are used more often and solve more questions as the dataset grows. These results suggest that data scaling improves the coverage of long-tailed reasoning patterns, thereby enhancing generalization.
Finally, we theoretically justify our insights through a training-dynamics analysis of shallow transformers. For long-tailed SFT data, we prove that both SFT and RL achieve high accuracy on in-distribution problems. However, with insufficient method coverage, SFT generalizes worse than RL on out-of-distribution (OOD) problems. As more methods are covered, potentially through a larger data scale, SFT can match RL generalization. These results justify the importance of data scale and tail-method coverage for SFT generalization.
In conclusion, our findings suggest that “RL generalizes, whereas SFT memorizes” requires prerequisites on SFT training datasets. In particular, we show that both the SFT training data source and data scale are essential for unlocking the full potential of SFT. When trained with the same total number of samples as RL, SFT with an appropriate mixture of data sources can achieve generalization comparable to, or even better than, that of RL.
Preliminaries
Reinforcement Learning with Verifiable Rewards (RLVR) fine-tunes LLMs by maximizing the expected reward of model responses, where the reward is automatically computed from verifiable outcomes [1]. Given a set of question prompts DRL⊆V∗, where V∗ denotes the set of all finite sequences over the alphabet V, the regularized RLVR objective for model parameter θ∈Θ is
The first term is the expected reward of model rollouts Y∼Pθ(⋅∣X) on the question-prompt set, where r(X,Y) evaluates the correctness of the response Y to the question X, and X∼DRL means that X is sampled uniformly from DRL. The second term regularizes the model Pθ to remain close to the reference model Pref, typically the pre-RLVR model. Directly optimizing the model parameters with JRL(θ) often leads to poor performance due to large gradient variance and coverage issues [7, 8]. Instead of maximizing JRL directly, Proximal Policy Optimization (PPO) [8] and its recent powerful variant Group Relative Policy Optimization (GRPO) [1] maximize the following surrogate objective at each step:
where the expectation is taken over questions X∼DRL and G rollouts {Yi}i=1G∼Pθold(⋅∣X). Here, wi,t(θ)=Pθ(Yi,t∣X,Yi,<t)/Pθold(Yi,t∣X,Yi,<t) is the importance-sampling ratio between the current model and the previous-step model, clipε(x) clips x to [1−ε,1+ε], and Ai=(ri−mean({rj}j∈[G]))/std({rj}j∈[G]) is the group-normalized advantage of the i-th rollout.
GRPO maximizes JGRPO to update θ, typically using Adam. Empirically, Guo and Others [1] show that GRPO substantially improves multi-step reasoning performance on mathematics, logical deduction, and scientific reasoning tasks. Subsequent studies further show that RL-induced gains on the training domain, such as mathematics, can generalize to other reasoning domains, including physics and broader scientific topics [6, 5]. See more related works in Appendix § 7.
SFT equips pretrained LLMs with long-form reasoning abilities by directly minimizing the cross-entropy loss on curated long-reasoning data [2, 9]. Let DSFT denote a long-reasoning dataset consisting of pairs (X,Y), where X is a question prompt and Y is the corresponding long-reasoning answer. The objective of long-reasoning SFT is LSFT(θ)=−E(X,Y)∼DSFT[∑t=1TlogPθ(Yt∣X,Y<t)]. In practice, this optimization is typically solved with first-order optimizers such as Adam [10] or AdamW [11]. Because dense supervision is available at every token position, SFT is often sample-efficient [12]. However, prior work [6, 5] suggests that SFT mainly improves model reasoning ability on tasks that are in distribution with respect to the training data, and that the resulting models exhibit weaker generalization than RLVR-trained models.
Experiments
In this section, we conduct extensive experiments to compare the generalization ability of RL and SFT. Following Huan et al. [5], we train models with SFT and RL on mathematical reasoning datasets and evaluate them on other general reasoning tasks to assess out-of-domain generalization. To further understand what affects the generalization of SFT models, we study two critical properties of the training data: (1) its source and (2) the dataset scale. We first describe the basic training setups used throughout the experiments.
Experimental Setup. We use Qwen2.5-7B [13] and Qwen3-8B-Base [14] as the base models and train them on mathematical problems from the DAPO dataset [15]. We present the Qwen2.5-7B results in the main paper and defer the Qwen3-8B-Base results to Appendix § 8.§ 1. For RL, we use the questions in the DAPO dataset with a reasoning prompt prefix, and optimize the model with GRPO [1] implemented in VeRL [16] under the standard training setup (see Appendix § 8.§ 2). For SFT, we use rollout models, Qwen3-14B [17] by default, to generate training responses for the same DAPO prompts, and optimize the model with LlamaFactory [18]; detailed hyperparameters are provided in Appendix § 8.§ 3. We evaluate the trained models on both in-distribution mathematical reasoning tasks and general reasoning tasks. For mathematical reasoning, we primarily use the SimpleRL evaluation pipeline [19], covering AIME24 [20], Minerva Math [21], GSM8K [22], MATH500 [23], and OlympiadBench [24]. For general reasoning, we use twelve tasks from the Language Model Evaluation Harness [25], including GPQA-Diamond [26] and MMLU-Pro [27], with the full list deferred to Appendix § 8.4. These tasks span science, action planning, deep semantic understanding, multilingual reasoning, and related domains beyond mathematics. For each task, we normalize the metric to [0,100] and report the average score across tasks.
Training Data Source Matters for SFT Generalization
In this section, we investigate how the training data source influences the generalization ability of SFT. We first introduce baselines and two data sources.
Baselines and Data Sources. Throughout the paper, we use the base model, i.e., the model without RL or SFT tuning, and the model trained by RL as baselines. We consider two data sources for SFT. The first is constructed by rejection sampling from a strong teacher model, Qwen3-14B [28, 29, 30]: for each question, we generate at most 128 rollouts and retain one correct rollout if available. We refer to the resulting dataset, of size 15,607, as the Qwen3-14B dataset. The second data source uses the model trained by RL for rejection sampling under the same protocol. Since the RL-tuned model is obtained from the base model, its trajectories are expected to be more aligned with the base model parameters. We refer to this dataset, of size 14,512, as the RL-tuned dataset. We highlight that both the Qwen3-14B and RL-tuned datasets match the RL dataset in the number of prompts. To ablate the effect of data source more finely, we further interpolate between these two datasets with 5 mixing ratios. The k-th mixed dataset, denoted by Mix-k, contains (11−2k)×10% data from the RL-tuned dataset and (2k−1)×10% data from the Qwen3-14B dataset, where k∈{1,…,5}.
Table 1 reports results on all mathematical reasoning datasets and a representative subset of general reasoning datasets, with the full results deferred to Appendix § 8.5. Figure 3 shows the average performance over the mathematical and general reasoning datasets.
Figure 3. * (a) Math benchmarks; Figure 2: * (b) General reasoning benchmarks. Performance comparison of SFT-trained models using datasets with different mixing ratios. Mix-k consists of (11−2k)×10% data from the RL-tuned dataset and (2k−1)×10% data from the Qwen3-14B dataset. The results report performance on both mathematical reasoning and broader reasoning tasks. The mixture with 70% RL-tuned data and 30% Qwen3-14B data achieves the best performance across both tasks.
Model
AIME24 (Avg@32)
Minerva Math
GSM8k
MATH500
Olympiad Bench
ACP Bench
BBH
C-Eval
Ground Cocoa
MMLU-Pro
Qwen2.5-7B
4.7
26.5
88.6
63.6
30.2
23.3
10.9
40.1
34.7
9.2
RL-tuned
14.2
34.9
90.8
74.8
39.7
50.1
42.9
43.6
34.8
45.7
SFT w/ RL-tuned
13.8
37.1
91.4
74.8
39.3
58.0
33.8
42.4
35.6
19.4
SFT w/ Mix-1
13.9
37.5
91.3
76.6
<u>39.0</u>
56.6
34.4
36.4
<u>36.8</u>
<u>17.9</u>
SFT w/ Mix-2
13.4
41.2
91.9
<u>76.2</u>
36.9
53.4
37.3
47.8
37.0
21.3
SFT w/ Mix-3
12.4
<u>38.6</u>
<u>91.6</u>
75.0
37.8
50.7
35.6
28.2
36.7
3.8
SFT w/ Mix-4
10.7
31.2
90.8
73.8
37.6
43.7
34.1
<u>52.9</u>
34.6
0.7
SFT w/ Mix-5
11.0
34.2
90.2
73.2
38.2
46.6
<u>37.7</u>
53.8
34.2
1.2
SFT w/ Qwen3-14B
13.9
33.5
90.5
75.0
38.5
45.3
42.1
36.5
33.2
1.1
Table 1. Evaluation results on math and selected general reasoning tasks of models trained from mixed data sources. The highest and second highest number within the SFT-tuned models are highlighted in bold and <u>underlined</u>, respectively. SFT w/ Mix-2 achieves the best performance among the SFT-tuned models.
First, for the two baselines, our results show that RL tuning improves the base model on all mathematical reasoning tasks: the average math score increases from 42.7 to 50.9 after RL training on DAPO. Notably, this improvement also transfers to general reasoning tasks, where the average score increases from 33.8 to 43.8. Second, comparing SFT with Qwen3-14B and RL-tuned data, Figure 3 shows that both data sources improve the base model’s mathematical reasoning ability, reaching performance comparable to or even higher than the model trained by RL. However, both SFT variants perform worse than the RL-tuned model on general reasoning tasks, while SFT with the RL-tuned dataset still outperforms SFT with the Qwen3-14B dataset. Finally, comparing different mixtures of Qwen3-14B and RL-tuned data, Table 1 and Figure 3 show that pure RL-tuned data or strong model-generated data does not always yield the best general-reasoning performance. Instead, Mix-2, which consists of 70% data from the RL-tuned model and 30% from Qwen3-14B, achieves the highest average score on general reasoning tasks. These lead to the following observation.
Observation 1: The training data source is crucial for generalization: data from a model closer to the one being fine-tuned can be more effective than data from a stronger model, while data from a single model is not necessarily optimal; incorporating an appropriate amount of more diverse data can further improve general reasoning ability.
Matched Rollout Budgets Enable SFT to Generalize Like RL
While the previous section shows that a suitable SFT data source improves general reasoning, the resulting model still lags behind the model trained by RL. We next study an orthogonal factor: how SFT training-set size affects generalization.
In Section 3.1, RL and SFT are matched at the question-prompt level, as both use questions from the DAPO dataset. However, RL generates 8 rollouts per prompt, whereas SFT observes only one correct rollout per prompt. As a result, RL trains on 8× more rollouts than SFT. This motivates us to increase the SFT dataset size under different rollout-budget matching criteria.
We first match the number of solvable prompts, defined as prompts for which at least one correct rollout is obtained during RL training. Under this criterion, the corresponding Qwen3-14B and RL-tuned datasets each contain one correct rollout for every solvable prompt. This yields a training set of about 10,000 prompt–rollout pairs, fewer than the one used in Section 3.1.
We then match the number of correct rollouts per prompt in the SFT training dataset to that observed during RL training. Specifically, for each prompt, we record the number of correct rollouts generated during RL and construct the SFT dataset with the same number of correct rollouts for that prompt. This yields a dataset with about 39,000 prompt–rollout pairs.
Lastly, we match the total number of rollouts for each prompt. Since RL training uses both correct and incorrect rollouts, we augment the 39,000 correct training samples with about 98,000 incorrect-rollouts. This criterion also matches the gradient computation cost of SFT to that of RL. The resulting SFT datasets contain about 138,000 prompt–rollout pairs.
More details of constructing these SFT datasets are defered to Appendix § 8.§ 3. Evaluation results are presented in Figure 6. First, comparing the prompt-matched and solvable-prompt-matched SFT datasets shows that the solvable-prompt-matched dataset achieves comparable mathematical reasoning ability but better general reasoning ability. One possible explanation is that forcing the model to learn from prompts that are not solvable during RL training can hurt generalization. Second, comparing the prompt-matched and correct-rollout-matched datasets shows that matching the number of correct rollouts substantially improves SFT generalization, yielding general reasoning performance only slightly below that of the RL-trained model. Finally, comparing the correct-rollout-matched and rollout-matched datasets shows that including rollouts with incorrect final answers can further improve SFT generalization. This may be because such rollouts still contain correct intermediate reasoning steps that can benefit SFT [31]. Moreover, SFT with the rollout-matched dataset achieves comparable or better mathematical and general reasoning ability than the RL-trained model. These results lead to the following observation.
(a) Math benchmarks
(b) General reasoning benchmarks
Figure 6. Performance comparison of SFT-trained models on mathematical and broader reasoning benchmarks when different training factors are matched to RL, including the number of prompts, solvable prompts, correct rollouts, and total rollouts. Matching the total rollout budget of SFT to that of RL substantially improves general reasoning performance, while preserving strong mathematical reasoning ability.
Observation 2: The amount of training data substantially influences SFT generalization: matching the number of correct rollouts greatly improves SFT generalization, while further matching the total rollout budget enables SFT to achieve comparable generalization to RL.
We also conducted an ablation study to exclude the effect of the compute in Appendix B.7, which further supports our observations on the training-set size.
Compounding Effects of Data Source and Scale. Observations 1 and 2 show that choosing an appropriate data source and increasing the training-set size can each improve the generalization of SFT. We next investigate whether combining these two factors enables SFT to generalize even better than RL. Specifically, we mix the rollout-matched training dataset across different data sources. Since § 3.1 suggests that the best mixture ratio is around 70% from the RL-tuned model and 30% from Qwen3-14B, we focus on ratios near this setting to reduce computation. We consider mixture ratios (11−2k)×10% of the RL-tuned dataset and (2k−1)×10% of the Qwen3-14B dataset for k∈{1,2,3}, yielding three training datasets, each with about 138,000 prompt–rollout pairs.
The experimental results are reported in Figure 9, with the standard error reported in Appendix B.6. It shows that the best SFT performance is achieved by the dataset mixing 90% RL-tuned dataset with 10% Qwen3-14B dataset, which yields better generalization than the RL-trained model. Thus, we reach the following observation.
(a) *
(b) *
Figure 9. Performance comparison of SFT-trained models using scaled training datasets with different mixing ratios. The datasets are mixtures of RL-tuned data and Qwen3-14B data. After scaling the training data, the mixture with 90% RL-tuned data and 10% Qwen3-14B data achieves the best performance.
Observation 3: With a carefully mixed dataset and a rollout budget matched to RL, SFT achieves promisingly better general reasoning ability than RL, undermining “RL generalizes, SFT memorizes.”
Long-Tailed Rollouts Provide Generalizable Reasoning Signals
Our experiments in § 3.2 show that training-set size plays a critical role in enabling SFT to match RL generalization. In this section, we investigate what additional information in larger datasets drives this effect. As the training set scales, it naturally includes more diverse rollouts, including rare tail rollouts that are less likely to appear in smaller datasets. We conjecture that these rare but informative rollouts are essential for SFT generalization, and we verify this hypothesis below.
We first characterize the rollout distribution by the methods used to solve each math problem. To study how dataset size changes the contained information, we construct training datasets with k∈{1,2,4,8} rollouts per prompt, where k=8 matches the total rollout budget of RL. We then prompt Qwen3-32B to annotate the methods used in each rollout, allowing multiple labels per rollout. After consolidating the annotations, we identify 138 unique methods for solving math problems.
More experimental details are in Appendix § 8.§ 14.5. In the following, for robustness, we say that a method is covered by a dataset if it appears at least 100 times in that dataset.
Figure 10(a) plots the top-50 method frequencies in the datasets with k=8, with methods ranked by their frequencies in the Qwen3-14B dataset. The figure reveals a clear long tail: many methods appear only infrequently. For each dataset, we define tail methods as those in the bottom 20% by frequency among all identified methods. Figure 10(b) further plots the number of covered tail methods as k varies. For both the Qwen3-14B and RL-tuned datasets, larger datasets cover more tail methods, exposing SFT models to a broader set of rare but potentially generalizable reasoning patterns.
Figure 10. Characterization of LLM-annotated method distribution and coverage. (a) plots the occurrence frequencies of solution methods in the RL-tuned dataset, showing that the method distribution is long-tailed. (b) plots the number of covered methods in datasets of different sizes, showing that method coverage increases as the training data scale grows.
Tail Method Helps. To investigate how tail methods in the training dataset influence generalization to general reasoning tasks, we collect the reasoning rollouts during evaluation from models SFT-tuned on the four datasets. We then prompt Qwen3-32B to identify the methods employed in each rollout and compute the usage statistics on these tasks. Figure 13 shows the usage frequency of the tail methods on two representative tasks, MMLU-Pro [27] and MMLU-Pro+ (excluding math) [32], for the models trained on datasets with different k. We report the usage counts both over all evaluation rollouts and over correct rollouts only.
Image not extracted
Figure 13. Tail-method usage statistics. The figures report the usage of LLM-annotated tail methods by SFT-trained models on MMLU-Pro and MMLU-Pro+. Tail methods are used more frequently (light bars) and can correctly solve more questions (dark bars) in general reasoning tasks (e.g., MMLU-Pro, MMLU-Pro+), as the training dataset encompasses a broader range of methods.
We observe that tail methods are used more frequently as the training-set size scales. Furthermore, these tail methods are capable of solving more questions as the tail-method coverage increases. We arrive at the following observation.
Observation 4: One benefit of increasing the training samples is improved coverage of long-tailed patterns. Training on these rare but informative samples can enhance performance on downstream general reasoning tasks.
We provide more rollout analysis in Appendix § 8.§ 14.5 and more discussions about our experimental setup, including the comparison with distillation and the use of incorrect rollouts, in Appendix § 13.
A Case Study of Shallow Transformers
Section 3 shows that SFT generalization depends critically on training-data scale, especially the coverage of long-tailed methods. We now provide theoretical grounding for these findings by analyzing the training dynamics of shallow transformers with a simplified data model.
References
[1]Daya Guo and Others. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081):633–638, September 2025. ISSN 1476-4687. doi: 10.1038/s41586-025-09422-z.DOI
[2]Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori B Hashimoto. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20286–20332, 2025.arxiv.org/abs/2501.19393
[3]Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models. arXiv preprint arXiv:2504.11468, 2025.arxiv.org/abs/2504.11468
[4]Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. RL’s razor: Why on-policy reinforcement learning forgets less. In NeurIPS 2025 Workshop: Second Workshop on Aligning Reinforcement Learning Experimentalists and Theorists, 2025.arxiv.org/abs/2509.04259
[5]Maggie Ziyu Huan, Yuetai Li, Tianyu Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Poovendran, Graham Neubig, and Xiang Yue. Does math reasoning improve general LLM capabilities? understanding transferability of LLM reasoning, 2026.arxiv.org/abs/2507.00432
[6]Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of foundation model post-training. In Forty-second International Conference on Machine Learning, 2025.arxiv.org/abs/2501.17161
[7]John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International conference on machine learning, pages 1889–1897. PMLR, 2015.arxiv.org/abs/1502.05477
[8]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017.arxiv.org/abs/1707.06347
[9]Peng-Yuan Wang, Tian-Shuo Liu, Chenyang Wang, Ziniu Li, Yidi Wang, Shu Yan, Chengxing Jia, Xu-Hui Liu, Xinwei Chen, Jiacheng Xu, et al. A survey on large language models for mathematical reasoning. ACM Computing Surveys, 58(8):1–35, 2026.arxiv.org/abs/2506.08446
[10]Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
[11]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019.
[12]Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models, 2026.arxiv.org/abs/2601.18734
[13]Owen, ; An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025.
[15]Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, YuYue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, LingJun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuanze Tong, Chi Zhang, Mofan Zhang, Ru Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Yonghui Wu, and Mingxuan Wang. DAPO: An open-source LLM reinforcement learning system at scale. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026.arxiv.org/abs/2503.14476
[16]Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, page 1279–1297. Association for Computing Machinery, 2025. doi: 10.1145/3689031.3696075.DOI
[18]Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhai Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguistics.
[19]Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun MA, and Junxian He. SimpleRL-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. In Second Conference on Language Modeling, 2025.arxiv.org/abs/2503.18892
[20]Maxwell Jia. Aime problem set 2024, 2024.
[21]Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in neural information processing systems, 35:3843–3857, 2022.arxiv.org/abs/2206.14858
[22]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021.arxiv.org/abs/2110.14168
[23]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. Advances in neural information processing systems, 2021.
[24]Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems, 2024.
[25]Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024.
[26]David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanze Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024.arxiv.org/abs/2311.12022
[27]Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eighth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.arxiv.org/abs/2406.01574
[28]Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, KaShun SHUM, and Tong Zhang. RAFT: Reward ranked finetuning for generative foundation model alignment. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.arxiv.org/abs/2304.06767
[29]Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models, 2023.arxiv.org/abs/2308.01825
[30]Wei Xiong, Jiarui Yao, Yuhui Xu, Bo Pang, Lei Wang, Doyen Sahoo, Junnan Li, Nan Jiang, Tong Zhang, Caiming Xiong, and Hanze Dong. A minimalist approach to llm reasoning: from rejection sampling to reinforce, 2025.arxiv.org/abs/2504.11343
[32]Saeid Asgari, Aliashgar Khani, and Amir Hosein Khasahmadi. MMLU-pro+: Evaluating higher-order reasoning and shortcut learning in LLMs. In NeurIPS Safe Generative AI Workshop 2024, 2024.
[33]Yu Huang, Yuan Cheng, and Yingbin Liang. In-context convergence of transformers. arXiv preprint arXiv:2310.05249, 2023.arxiv.org/abs/2310.05249
[34]Eshaan Nichani, Alex Damian, and Jason D Lee. How transformers learn causal structure with gradient descent. arXiv preprint arXiv:2402.14735, 2024.arxiv.org/abs/2402.14735
[35]Yuan Cheng, Yu Huang, Zhe Xiong, Yingbin Liang, and Vincent YF Tan. Transformers probably learn directed acyclic graphs via kernel-guided mutual information. arXiv preprint arXiv:2510.25542, 2025.arxiv.org/abs/2510.25542
[36]Junfeng Fang, Hocheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. Alphaedit: Null-space constrained knowledge editing for language models. arXiv preprint arXiv:2410.02355, 2024.arxiv.org/abs/2410.02355
[37]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359–17372, 2022.arxiv.org/abs/2202.05262
[38]Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229, 2022.arxiv.org/abs/2210.07229
[39]Yu Huang, Zixin Wen, Yuejie Chi, Yuting Wei, Aarti Singh, Yingbin Liang, and Yuxin Chen. On the learning dynamics of rlvr at the edge of competence. arXiv preprint arXiv:2602.14872, 2026.
[40]Shuche Wang, Fengzhuo Zhang, Jiaxiang Li, Cunxiao Du, Chao Du, Tianyu Pang, Zhuoran Yang, Mingyi Hong, and Vincent YF Tan. Muon outperforms adam in tail-end associative memory learning. arXiv preprint arXiv:2509.26030, 2025.arxiv.org/abs/2509.26030
[41]Jimmy Ba, Murat A Erdogdu, Taiji Suzuki, Zhichao Wang, Denny Wu, and Greg Yang. High-dimensional asymptotics of feature learning: How one gradient step improves the representation. Advances in Neural Information Processing Systems, 35:37932–37946, 2022.arxiv.org/abs/2205.01445
[42]Aojun Lu, Tao Feng, Hangjie Yuan, Wei Li, and Yanan Sun. Why does rl generalize better than sft? a data-centric perspective on vlm post-training. arXiv preprint arXiv:2602.10815, 2026.arxiv.org/abs/2602.10815
[43]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024.arxiv.org/abs/2402.03300
[44]Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75.notion.site/DeepScaler-Surpassing-01-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8dca303013a4e2, 2025. Notion Blog.
[45]Bingxiang He, Zekai Qu, Zeyuan Liu, Yinghao Chen, Yuxin Zuo, Cheng Qian, Kaiyan Zhang, Weize Chen, Chaoxian Xiao, Ganguo Cui, Ning Ding, and Zhiyuan Liu. Justrl: Scaling a 1.5b llm with a simple rl recipe, 2025.arxiv.org/abs/2512.16649
[46]Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl-zero-like training: A critical perspective. In Second Conference on Language Modeling, 2025.
[48]Wei Xiong, Chenlu Ye, Baohao Liao, Hanze Dong, Xinxing Xu, Christof Monz, Jiang Bian, Nan Jiang, and Tong Zhang. Reinforce-ada: An adaptive sampling framework under non-linear rl objectives, 2025.arxiv.org/abs/2510.04996
[49]Jens Tuyls, Dylan J Foster, Akshay Krishnamurthy, and Jordan T. Ash. Representation-based exploration for language models: From test-time to post-training. In The Fourteenth International Conference on Learning Representations, 2026.arxiv.org/abs/2510.11686
[50]Xinyu Zhu, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, and Yu Meng. The surprising effectiveness of negative reinforcement in LLM reasoning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026.arxiv.org/abs/2506.01347
[51]Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xiong-Hui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for LLM reasoning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026.arxiv.org/abs/2506.01939
[52]Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model? In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026.arxiv.org/abs/2504.13837
[53]Xumeng Wen, Zihan Liu, Shun Zheng, Shengyu Ye, Zhirong Wu, Yang Wang, Zhijian Xu, Xiao Liang, Junjie Li, Ziming Miao, Jiang Bian, and Mao Yang. Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms, 2025.arxiv.org/abs/2506.14245
[54]Yiyou Sun, Yuhan Cao, Pohao Huang, Haoyue Bai, Hannaneh Hajishirzi, Nouha Dziri, and Dawn Song. RL grokking recipe: How does RL unlock and transfer new algorithms in LLMs? In The Fourteenth International Conference on Learning Representations, 2026.arxiv.org/abs/2509.21016
[55]Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. ProRL: Prolonged reinforcement learning expands reasoning boundaries in large language models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026.arxiv.org/abs/2505.24864
[56]Yuqian Fu, Tinghong Chen, Jiajun Chai, Xihuai Wang, Songjun Tu, Guojun Yin, Wei Lin, Qichao Zhang, Yuanheng Zhu, and Dongbin Zhao. Srft: A single-stage method with supervised and reinforcement fine-tuning for reasoning. arXiv preprint arXiv:2506.19767, 2025.arxiv.org/abs/2506.19767
[57]Zelin Tan, Hejia Geng, Xiaohang Yu, Mulei Zhang, Guancheng Wan, Yifan Zhou, Qiang He, Xiangyuan Xue, Heng Zhou, Yutao Fan, Zhongzhi Li, Zaibin Zhang, Guibin Zhang, Chen Zhang, Zhenfei Yin, Philip Torr, and Lei Bai. Scaling behaviors of llm reinforcement learning post-training: An empirical study in mathematical reasoning, 2026.
[58]Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng, Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, and Ning Ding. The entropy mechanism of reinforcement learning for reasoning language models, 2025.arxiv.org/abs/2505.22617
[59]Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi, Pang Wei Koh, and Luke Zettlemoyer. Spurious rewards: Rethinking training signals in rlvr, 2026.arxiv.org/abs/2506.10947
[60]Peter Chen, Xiaopeng Li, Ziniu Li, Wotao Yin, Xi Chen, and Tianyi Lin. Exploration vs exploitation: Rethinking RLVR through clipping, entropy, and spurious reward. In The Fourteenth International Conference on Learning Representations, 2026.arxiv.org/abs/2512.16912
[61]Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419, 2025.arxiv.org/abs/2502.17419
[62]Fengli Xu, Qianyue Hao, Chenyang Shao, Zefang Zong, Yu Li, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, et al. Toward large reasoning models: A survey of reinforced reasoning with large language models. Patterns, 6(10), 2025.DOI
[63]Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476–15488, 2022.arxiv.org/abs/2203.14465
[64]Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024.
[65]Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707, 2023.arxiv.org/abs/2306.02707
[66]Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, et al. Orca 2: Teaching small language models how to reason. arXiv preprint arXiv:2311.11045, 2023.arxiv.org/abs/2311.11045
[67]Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning. arXiv preprint arXiv:2309.05653, 2023.arxiv.org/abs/2309.05653
[68]Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.17452, 2023.arxiv.org/abs/2309.17452
[69]Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, et al. NuminaMath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13(9):9, 2024.
[70]Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387, 2025.arxiv.org/abs/2502.03387
[71]Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhmanesh, Shishir G Patil, Matei Zaharia, et al. Llms can easily learn to reason from demonstrations structure, not content, is what matters! arXiv preprint arXiv:2502.07374, 2025.arxiv.org/abs/2502.07374
[72]Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570, 2025.arxiv.org/abs/2501.12570
[73]Bin Yu, Hang Yuan, Haotian Li, Xueyin Xu, Yuliang Wei, Bailing Wang, Weizhen Qi, and Kai Chen. Long-short chain-of-thought mixture supervised fine-tuning eliciting efficient reasoning in large language models. arXiv preprint arXiv:2505.03469, 2025.arxiv.org/abs/2505.03469
[74]Neel Rajani, Aryo Pradipta Gema, Seraphina Goldfarb-Tarrant, and Ivan Titov. Scalpel vs. hammer: Grpo amplifies existing capabilities, sft replaces them. arXiv preprint arXiv:2507.10616, 2025.arxiv.org/abs/2507.10616
[75]Hangzhan Jin, Sitao Luan, Sicheng Lyu, Guillaume Rabusseau, Doina Precup, and Mohammad Hamzaq. RL fine-tuning heals the OOD forgetting in SFT. In First Workshop on Foundations of Reasoning in Language Models, 2025.arxiv.org/abs/2509.12235
[76]Howard Chen, Noam Razin, Karthik Narasimhan, and Danqi Chen. Retaining by doing: The role of on-policy data in mitigating forgetting. arXiv preprint arXiv:2510.18874, 2025.arxiv.org/abs/2510.18874
[77]Kohsei Matsutani, Shota Takashiro, Gouki Minegishi, Takeshi Kojima, Yusuke Iwasawa, and Yutaka Matsuo. Rl squeezes, sft expands: A comparative study of reasoning llms. arXiv preprint arXiv:2509.21128, 2025.arxiv.org/abs/2509.21128
[78]Brian Lu, Hongyu Zhao, Shuo Sun, Hao Peng, Rui Ding, and Hongyuan Mei. Generalization of rlvr using causal reasoning as a testbed. arXiv preprint arXiv:2512.20760, 2025.arxiv.org/abs/2512.20760
[79]Feiyang Kang, Michael Kuchnik, Karthik Padthe, Marin Vlastelica, Ruoxi Jia, Carole-Jean Wu, and Newsha Ardalani. Quagmires in sft-rl post-training: When high sft scores mislead and what to use instead. arXiv preprint arXiv:2510.01624, 2025.arxiv.org/abs/2510.01624
[80]Haitao Jiang, Wenbo Zhang, Jiarui Yao, Hengrui Cai, Sheng Wang, and Rui Song. Supervised
fine-tuning versus reinforcement learning: A study of post-training methods for large language
models. arXiv preprint arXiv:2603.13985, 2026.arxiv.org/abs/2603.13985
[81]Kevin Lu and Thinking Machines Lab. On-policy distillation. Thinking Machines Lab: Con-
nectionism, 2025. doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-
distillation.DOI
[82]Howard Chen, Noam Razin, Karthik Narasimhan, and Danqi Chen. Retaining by doing: The
role of on-policy data in mitigating forgetting, 2025.arxiv.org/abs/2510.18874
[83]Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya
Grover. Self-distilled reasoner: On-policy self-distillation for large language models, 2026.arxiv.org/abs/2601.18734
[84]Jonas Hüb b otter, Frederike Lübeck, Lejs Deen Behric, Anton Baumann, Marco Bagatella, Daniel
Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, and Andreas
Krause. Reinforcement learning via self-distillation. In The 1st Workshop on Scaling Post-
training for LLMs, 2026.
[85]Yang Chen, Zhuolin Yang, Zihan Liu, Chankyu Lee, Peng Xu, Mohammad Shoeybi, Bryan
Catanzaro, and Wei Ping. Acereason-nemotron: Advancing math and code reasoning through re-
inforcement learning. In The Thirty-ninth Annual Conference on Neural Information Processing
Systems, 2026.arxiv.org/abs/2505.16400
[86]David Vilares and Carlos Gómez-Rodríguez. HEAD-QA: A healthcare dataset for complex
reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational
Linguistics, pages 960–966. Association for Computational Linguistics, July 2019. doi: 10.186
53/v1/P19-1092.arxiv.org/abs/1906.04701
[87]Harsha Kokel, Michael Katz, Kavitha Srinivas, and Shirin Sohrabi. Acpbench: Reasoning about
action, change, and planning. In AAAI. AAAI Press, 2025.
[88]Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick,
and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning
challenge. ArXiv, abs/1803.05457, 2018.arxiv.org/abs/1803.05457
[89]Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won
Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. Challenging
BIG-bench tasks and whether chain-of-thought can solve them. In Findings of the Association
for Computational Linguistics: ACL 2023, pages 13003–13051. Association for Computational
Linguistics, July 2023. doi: 10.18653/v1/2023.findings-acl.824.DOI
[90]Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng
Liu, Chuancheng Lv, Yikai Zhang, jiayi lei, Yao Fu, Maosong Sun, and Junxian He. C-eval: A
multi-level multi-discipline Chinese evaluation suite for foundation models. In Thirty-seventh
Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.arxiv.org/abs/2305.08322
[91]Haryo Wibowo, Erland Fuadi, Made Nityasya, Radityo Eko Prasojo, and Alham Aji. COPAL-
ID: Indonesian language reasoning with local culture and nuances. In Proceedings of the 2024
Conference of the North American Chapter of the Association for Computational Linguistics:
Human Language Technologies (Volume 1: Long Papers), pages 1404–1422. Association for
Computational Linguistics, June 2024. doi: 10.18653/v1/2024.naacl-long.77.DOI
[92]Harsh Kohli, Sachin Kumar, and Huan Sun. GroundCocoa: A benchmark for evaluating
compositional & conditional reasoning in language models. In Proceedings of the 2025
Conference of the Nations of the Americas Chapter of the Association for Computational
Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8280–8295.
Association for Computational Linguistics, April 2025. ISBN 979-8-89176-196-06. doi: 10.186
53/v1/2025.naacl-long.420.
[93]Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor
conduct electricity? a new dataset for open book question answering. In Proceedings of the
2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391. Association for Computational Linguistics, October-November 2018. doi: 10.18653/v1/D18-1260.DOI
[94]Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. In Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.arxiv.org/abs/1911.11641
[95]Yu Huang, Yuan Cheng, and Yingbin Liang. In-context convergence of Transformers. In International Conference on Machine Learning, pages 19660–19722. PMLR, 2024.arxiv.org/abs/2310.05249
[96]Yuan Cheng, Fengzhuo Zhang, Yunlong Hou, Cunxiao Du, Chao Du, Tianyu Pang, Aixin Sun, and Zhuoran Yang. Demystifying the slash pattern in attention: The role of RoPE. arXiv preprint arXiv:2601.08297, 2026.arxiv.org/abs/2601.08297
[97]Luc Devroye. The equivalence of weak, strong and complete convergence in l1 for kernel density estimates. The Annals of Statistics, pages 896–904, 1983.DOI
[98]Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015.arxiv.org/abs/1503.02531
[99]Yiwei Li, Peiwen Yuan, Shaoxiong Feng, Boyuan Pan, Bin Sun, Xinglin Wang, Heda Wang, and Kan Li. Turning dust into gold: Distilling complex reasoning capabilities from llms by leveraging negative data, 2023.arxiv.org/abs/2312.12832