betaFeedback

WorksComputer Science

An Analytical Model of Compute-limited Multistage Training Pipelines

Abstract

Modern foundation models are trained through multiple stages, yet how these stages interact and how compute should be allocated across them remain poorly understood. We introduce an analytically tractable high-dimensional model combining unsupervised pretraining with a linear autoencoder, supervised fine-tuning (SFT) with noisy stepwise labels, and reinforcement learning (RL) with sparse episodic rewards. Using random matrix theory and an operator-projection closure, we derive approximate low-dimensional learning dynamics that closely match simulations. We show that finite pretraining data imposes a representation-dependent performance limit, with Bayes classification error scaling as the inverse square root of the pretraining sample ratio, and observe the same exponent in asymptotic SFT and RL errors. Better pretraining also reduces sensitivity to noisy supervision. SFT initialization shortens the initial RL plateau induced by sparse rewards, an effect we also observe in OLMo-2 1B on GSM8K. Longer SFT warm starts additionally improve asymptotic RL performance. Optimizing compute across all three stages reveals that the optimal allocation depends on supervision quality and total budget, with a non-monotonic dependence of the RL compute share on budget. Finally, a complementary compositional model shows how RL selectively strengthens task-relevant pretrained skills while largely preserving unused skills. Together, these results identify mechanisms linking representation quality, feedback structure, and compute allocation in multistage training.

Introduction

Modern foundation models exhibit transformative capabilities from human-level language comprehension and production Radford et al. [2019], Brown et al. [ [12], Touvron et al. [2023], to reasoning across modalities and mathematical problem-solving Jaech et al. [2024], Shao et al. [2024], Chervonyi et al. [ [13], Team et al. [2025]. Multistage training is a critical aspect of the training paradigm of such models, which consists of unsupervised auto-regressive pretraining followed by several stages of supervised finetuning and reinforcement learning on smaller, domain-specific datasets Ouyang et al. [2022], Achiam et al. [ [1], Liu et al. [2024], Yang et al. [2025], DeepSeek-AI et al. [ [19]. Yet, the consequences of different training stages and their interactions remain poorly understood. Empirical investigation of these questions is difficult, as it requires substantial compute and costly human annotation, constraining the scope of hypotheses that can be explored Zhao et al. [2025], Shao et al. [2025], Akter et al. [ [3], Liu et al. [2025a].

Alongside the transition to multistage training pipelines, foundation models reflect a second shift from a sample-complexity–centric view of learning to a compute-centric one. Whereas classical theory treats data as the limiting resource and implicitly assumes unbounded compute, modern foundation models invert this assumption: data is abundant, compute is limited, and performance scales predictably with model size and training compute, as evidenced by large-scale empirical scaling laws Kaplan et al. [2020], Hoffmann et al. [ [29]. In this compute-limited regime, models are rarely trained to convergence, and performance is instead determined by how a fixed computational budget is allocated across model capacity and multiple training stages.

Together, these two shifts—toward multistage pipelines and the compute-limited regime—pose a challenge for existing theory, which offers limited guidance for analyzing learning dynamics, performance trade-offs, or compute allocation in modern foundation model training. —

Contributions. —To this end, we develop an analytically tractable high-dimensional model based on statistical physics Seung et al. [1992], Saad and Solla [1995], Zdeborová and Krzakala [2016]. The model isolates the effects of pretrained representation quality, dense stepwise supervision, and sparse episodic rewards. We derive deterministic learning dynamics in the high-dimensional limit and obtain a tractable, approximately closed system of ODEs, allowing us to study how pretrained representations, supervision quality, and compute allocation jointly determine downstream performance. Our contributions are:

  • We combine random matrix theory with an operator-projection closure to obtain a low-dimensional approximation of the SFT and RL dynamics, validated against simulations. We derive the α−1/2\alpha^{-1/2} scaling of the representation-dependent Bayes error with pretraining sample size and observe the same exponent in asymptotic finetuning error.

  • We characterize how SFT and RL complement each other. SFT shortens the initial sparse-reward plateau, an effect also observed in OLMo-2 1B on GSM8K. Under spherical normalization and locally optimal learning rates, longer SFT warm starts additionally improve asymptotic RL performance. Better pretraining also reduces the excess SFT error caused by label noise.

  • We optimize compute allocation across pretraining, SFT, and RL. The optimal split depends on both supervision quality and the total budget: noisier labels favour RL, while the optimal RL share can vary non-monotonically with total compute.

  • We extend the model to compositional tasks, showing how prior skill proficiency accelerates RL and how finetuning selectively improves task-relevant skills while largely preserving unused skills.

Related Work

Empirical interactions across training stages. Controlled studies have investigated how pretraining shapes the effectiveness of downstream reinforcement learning. Shen et al. [2026] examine a complete pretraining–SFT–RL pipeline and relate pretraining loss and data scale to post-RL performance and learning speed, including a pretraining–RL compute frontier. Zhang et al. [2026a] isolate the effects of pretraining, intermediate training, and RL on compositional reasoning tasks. Zhao et al. [2024] investigate the amplification of pretrained behaviors under RL. Empirical work compares the generalization behavior of SFT and outcome-based RL Chu et al., [14] and demonstrates that higher standalone SFT accuracy need not yield better subsequent RL performance Zhang et al., 2026b, Kang et al., 2026].

Closure of structured-data learning dynamics. The problem of anisotropic data covariance generating an infinite hierarchy of order parameters has been studied in the context of online learning with structured data Yoshida and Okada [2019]. In that work, the authors show that the ODE system can only be closed if the covariance has a finite number of unique eigenvalues, where there is a pair of order parameters associated to every unique eigenvalue. There are several works that address this feature: approaches include replacing the infinite discrete hierarchy by a closed functional object—a spectral order parameter Goldt et al., [26, 27] or a resolvent Collins-Woodfin et al., [17]. However, this is non-trivial to implement in the case of a random data covariance. Another more recent approach most similar in spirit to ours is Xu et al. [2026], where the authors show the infinite hierarchy can be exponentially well approximated by a finite truncated system. Ours is merely a different method of truncation, without a proof of exponential convergence.

Setup

Pretraining

We draw a fixed pretraining dataset X∈RN×D\mathbf{X} \in\mathbb{R}^{N \times D} with independent rows distributed as N(0,Σ)\mathcal{N}(0,\Sigma), where Σ=Ms+ΔMn\Sigma= M_{\mathrm{s}} + \Delta M_{\mathrm{n}}, MsM_{\mathrm{s}} projects onto a signal subspace of dimension ρD\rho D, and Mn=I−MsM_{\mathrm{n}} = \mathbf{I} - M_{\mathrm{s}}. We take 0<ρ<10 < \rho< 1, 0<Δ<10 < \Delta< 1, and the high-dimensional limit N,D→∞N,D \to\infty at fixed α=N/D>ρ\alpha= N/D > \rho. A linear autoencoder (LAE) with hidden width ρD\rho D learns the empirical principal subspace of this dataset when trained to convergence [8] (Saxe et al. 2019). We represent its learned subspace by the orthogonal projector P\mathbf{P}. Downstream observations are x~=Px\tilde{\mathbf{x}} = \mathbf{P}\mathbf{x}, with fresh x∼N(0,Σ)\mathbf{x} \sim\mathcal{N}(0,\Sigma). P\mathbf{P} remains fixed throughout finetuning. Thus α\alpha determines how much task-relevant signal is available for the downstream task. The LAE learns a compressed feature subspace from unlabelled data, abstracting the role of pretraining as representation learning: discovering reusable structure and organizing inputs into a feature space [9, 11] (LeCun et al. 2015; LeCun et al. 2022). This frozen-feature assumption is motivated by evidence that finetuning preserves pretrained representations [Nadipalli, 2025] and the success of PEFT [28] (Hu et al. 2022)

Finetuning

The finetuning task is organized in episodes of length TT, where a student perceptron w∈Range⁡(P)\mathbf{w} \in\operatorname{Range}(\mathbf{P}) makes a series of binary decisions yt=sgn⁡(x~tTwD)y_t = \operatorname{sgn}\left(\frac{\tilde{\mathbf{x}}_t^{\mathsf{T}}\mathbf{w}}{\sqrt{D}}\right) given the raw observations x1…T∼N(0,Σ)\mathbf{x}_{1\ldots T} \sim\mathcal{N}(0,\Sigma) projected onto the encoder subspace by P\mathbf{P}. Data labels for supervision/reward yt∗=sgn⁡(xtTw∗D)y_t^{*} = \operatorname{sgn}\left(\frac{\mathbf{x}_t^{\mathsf{T}}\mathbf{w}^{*}}{\sqrt{D}}\right) are specified by a teacher perceptron w∗∈RD\mathbf{w}^{*} \in\mathbb{R}^{D}, which has access to the raw data x\mathbf{x}; importantly, it sees the entire signal subspace. The teacher is a fixed-weight oracle that generates the target labels. During SFT and RL, the weights of the student are updated with feedback from the teacher, but the core difference lies in how the feedback from the output is given to the student. This setting is inspired by the teacher–student framework [23, 10] (Seung et al. 1992). We discuss related teacher–student models in Appendix .§ 1.

SFT: Each individual sample in the episode contributes separately to the update by error feedback with the corresponding teacher label (Figure 1a, pink box). The update follows the Perceptron rule [Rosenblatt, 1958]. The update at the μ\muth episode can be written as

Schematic of multistage training pipeline and optimal performance under fixed compute

Figure 1. (a) A Schematic of multistage training pipeline. Each solid trapezoid indicates a single linear neural network layer. Pretraining is a Linear Autoencoder (LAE) that approximately learns the principal subspace of data x∼N(0,Σ)\mathbf{x} \sim\mathcal{N}(0,\Sigma), where Σ\Sigma is equal to the identity apart from a noise subspace, with identity covariance scaled by Δ<1\Delta< 1. CPTC_{\mathrm{PT}} is the compute allocated to pretraining. In the subsequent finetuning (SFT and RL), a teacher network w∗\mathbf{w}^{*} gives labels y∗y^{*} from the signal subspace and it is used to train a student network w\mathbf{w} learning in the subspace of the LAE encoder projection with feedback. The student predicts label yy using the effective features x~\tilde{\mathbf{x}}. Importantly, SFT is modelled by every observation at each tt updating the student, while RL is modelled by bulk feedback through reward at the end of a ‘correct’ sequence through the entire t=1,…,Tt = 1,\ldots,T episode. We denote the compute allocations to SFT and RL, CSFTC_{\mathrm{SFT}} and CRLC_{\mathrm{RL}} respectively. (b) Optimal generalisation performance against total Compute flops under three protocols: SFT only, RL only, and SFT with RL. Compute is allocated as described in 5.1, parameters are set as in 5d.

w(μ+1)=w(μ)−ηTD∑t=1Tyt(μ)x~t(μ)θ(−ySFTt∗ (μ)yt(μ)),(1)\mathbf{w}^{(\mu+1)} = \mathbf{w}^{(\mu)} - \frac{\eta}{T\sqrt{D}}\sum_{t=1}^{T} y_t^{(\mu)}\tilde{\mathbf{x}}_t^{(\mu)}\theta\left(-y_{\mathrm{SFT}t}^{*\,(\mu)}y_t^{(\mu)}\right), \tag*{(1)}

where θ(⋅)\theta(\cdot) is the heaviside step function and η\eta is the learning rate. This can be interpreted as the linearized gradient descent update on the mean squared loss ∑t=1T(yt−ySFTt∗)2\sum_{t=1}^{T}(y_t-y_{\mathrm{SFT}t}^{*})^2. The teacher labels for SFT are generated with noise, i.e. ySFTt∗=yt∗zty_{\mathrm{SFT}t}^{*}=y_t^{*}z_t, where ztz_t takes the values of 1 or -1 with probability (1−ϵ)(1-\epsilon) or ϵ\epsilon respectively. This is a simple model of imperfect supervision, which is common in real-world datasets. N.B. this label noise is unique to SFT in our model. Importantly, the student receives an individual update for every tt. For example, in multi-step math tasks, this is equivalent to getting a supervised learning signal at every step of a solution.

RL: We use the RL perceptron update of Patel et al. [2025], which extends the teacher-student framework to online policy learning under a sequential decision-making task. Given the same finetuning task, the critical difference with SFT lies in the bulk feedback: the student is updated using a delayed, sparse reward signal at the end of the episode, rather than at every step, as in SFT. The update at μ\muth episode can be written as

w(μ+1)=w(μ)+ηT(∑t=1Tyt(μ)x~t(μ)D)R(y1:T(μ),y1:T∗(μ)),(2)\mathbf{w}^{(\mu+1)} = \mathbf{w}^{(\mu)} + \frac{\eta}{T}\left(\sum_{t=1}^{T}\frac{y_t^{(\mu)}\tilde{\mathbf{x}}_t^{(\mu)}}{\sqrt{D}}\right)R\left(y_{1:T}^{(\mu)},y_{1:T}^{*(\mu)}\right), \tag*{(2)}

where the reward rule is specified by R(y1:T,y1:T∗)R(y_{1:T}, y_{1:T}^{*}). In this work, we mainly use the reward condition, every action in an episode must be correct, (i.e yt=yt∗y_t=y_t^{*}, ∀t∈1,…,T\forall t \in1,\ldots,T, or R=∏t=1Tθ(ytyt∗)R=\prod_{t=1}^{T}\theta(y_ty_t^{*})). This is akin to RLVR where reward is received based on the correctness of a final result, where the final result requires multiple previous steps to be correct in order to reach it. The episode length TT controls the difficulty of the task. Larger values of TT make the task more difficult as the model needs to make a longer sequence of correct decisions, giving an idealized analogue of verifiable-reward methods such as GRPO [19]. This task structure reflects the nature of multi-step problems, for example, a math problem requiring multiple steps of reasoning to be correct. When we consider multiple stages of finetuning (e.g. SFT followed by RL), each stage trains the same student and uses the final student state of the previous stage as the initial student state of the subsequent stage.

We use it not as a literal model of full finetuning, but to isolate how representation quality constrains later learning.

Learning Dynamics of SFT and RL

We can derive a set of ordinary differential equations (ODEs) that describe the learning dynamics of our model using the approach of Saad and Solla. In the thermodynamic limit (D→∞D \to\infty), the stochastic evolution of the high-dimensional student weights reduces to the deterministic evolution of a set of low-dimensional order parameters:

Js/n=w⊤PMs/nPw∗D,Qs/n=w⊤PMs/nPwD,Ss/n=w∗⊤PMs/nPw∗D,S=w∗⊤w∗D(3)J_{s/n} = \frac{\mathbf{w}^{\top}\mathbf{P}M_{s/n}\mathbf{P}\mathbf{w}^{*}}{D}, \quad Q_{s/n} = \frac{\mathbf{w}^{\top}\mathbf{P}M_{s/n}\mathbf{P}\mathbf{w}}{D}, \quad S_{s/n} = \frac{\mathbf{w}^{*\top}\mathbf{P}M_{s/n}\mathbf{P}\mathbf{w}^{*}}{D}, \quad S = \frac{\mathbf{w}^{*\top}\mathbf{w}^{*}}{D} \tag*{(3)}

Intuitively, Qs/nQ_{s/n} is the size of the projected student in the signal/noise subspace, Js/nJ_{s/n} is the overlap between the projected student and the projected teacher in signal/noise subspace, and (Ss/n,S)(S_{s/n}, S) are constant throughout training.

By appropriate multiplication of 1 and 2, we can derive a set of ODEs that govern the evolution of order parameters in the thermodynamic limit.

For the SFT:

dJs/ndξ=−η⟨yw∗⊤PMs/nPxDθ(−yy∗)⟩,dQs/ndξ=−2η⟨yw⊤PMs/nPxDθ(−yy∗)⟩+η2TTr⁡(PMs/nPΣ)D⟨θ(−yy∗)⟩.(4)\begin{aligned} \frac{dJ_{s/n}}{d\xi} &= -\eta\left\langle y\frac{\mathbf{w}^{*\top}\mathbf{P}M_{s/n}\mathbf{P}\mathbf{x}}{\sqrt{D}}\theta(-yy^{*}) \right\rangle, \\ \frac{dQ_{s/n}}{d\xi} &= -2\eta\left\langle y\frac{\mathbf{w}^{\top}\mathbf{P}M_{s/n}\mathbf{P}\mathbf{x}}{\sqrt{D}}\theta(-yy^{*}) \right\rangle+ \frac{\eta^{2}}{T}\frac{\operatorname{Tr}(\mathbf{P}M_{s/n}\mathbf{P}\mathbf{\Sigma})}{D}\left\langle\theta(-yy^{*})\right\rangle. \tag*{(4)} \end{aligned}

For the RL:

References

  1. [1]Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.arxiv.org/abs/2303.08774
  2. [2]Madhu S Advani, Andrew M Saxe, and Haim Sompolinsky. High-dimensional dynamics of generalization error in neural networks. Neural Networks, 132:428–446, 2020.DOI
  3. [3]Syeda Nahida Akter, Shrimai Prabhumoye, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Yejin Choi, and Bryan Catanzaro. Front-loading reasoning: The synergy between pretraining and post-training data. arXiv preprint arXiv:2510.03264, 2025.arxiv.org/abs/2510.03264
  4. [4]Haruka Asanuma, Shiro Takagi, Yoshihiro Nagano, Yuki Yoshida, Yasuhiko Igarashi, and Masato Okada. Statistical mechanical analysis of catastrophic forgetting in continual learning with teacher and student networks. Journal of the Physical Society of Japan, 90(10):104001, 2021.DOI
  5. [5]Alexander Atanasov, Jacob A Zavatone-Veth, and Cengiz Pehlevan. Scaling and renormalization in high-dimensional regression. arXiv preprint arXiv:2405.00592, 2024.DOI
  6. [6]Benjamin Aubin, Antoine Maillard, Florent Krzakala, Nicolas Macris, Lenka Zdeborová, et al. The committee machine: Computational to statistical gaps in learning a two-layers neural network. Advances in Neural Information Processing Systems, 31, 2018.
  7. [7]Yasaman Bahri, Jonathan Kadmon, Jeffrey Pennington, Sam S Schoenholz, Jascha Sohl-Dickstein, and Surya Ganguli. Statistical mechanics of deep learning. Annual review of condensed matter physics, 11(1):501–528, 2020.DOI
  8. [8]Pierre Baldi and Kurt Hornik. Neural networks and principal component analysis: Learning from examples without local minima. Neural networks, 2(1):53–58, 1989.DOI
  9. [9]Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. arxiv 2012. arXiv preprint arXiv:1206.5538, 10, 2012.arxiv.org/abs/1206.5538
  10. [10]Michael Biehl and Holm Schwarze. Learning drifting concepts with neural networks. Journal of Physics A: Mathematical and General, 26(11):2651, 1993.DOI
  11. [11]Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bossut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.arxiv.org/abs/2108.07258
  12. [12]Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.arxiv.org/abs/2005.14165
  13. [13]Yuri Chervonyi, Trieu H Trinh, Miroslav Olšák, Xiaomeng Yang, Hoang H Nguyen, Marcelo Menegali, Junheuyk Jung, Junsu Kim, Vikas Verma, Quoc V Le, et al. Gold-medalist performance in solving olympiad geometry with alphageometry2. Journal of Machine Learning Research, 26(241):1–39, 2025.arxiv.org/abs/2502.03544
  14. [14]Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of foundation model post-training. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 10818–10838, 2025. URL https://proceedings.mlr.press/v267/chu25c.html.arxiv.org/abs/2501.17161
  15. [15]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021a.arxiv.org/abs/2110.14168
  16. [16]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021b.arxiv.org/abs/2110.14168
  17. [17]Elizabeth Collins-Woodfin, Courtney Paquette, Elliot Paquette, and Inbar Seroussi. Hitting the high-dimensional notes: An ode for sgd learning dynamics on glms and multi-index models. Information and Inference: A Journal of the IMA, 13(4):iaae028, 2024.arxiv.org/abs/2308.08977
  18. [18]Chandler Davis and William Morton Kahan. The rotation of eigenvectors by a perturbation. iii. SIAM Journal on Numerical Analysis, 7(1):1–46, 1970.DOI
  19. [19]DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shing Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyi Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xue, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
  20. [20]Simon Du, Jason Lee, Yuandong Tian, Aarti Singh, and Barnabas Poczos. Gradient descent learns one-hidden-layer cnn: Don’t be afraid of spurious local minima. In International Conference on Machine Learning, pages 1339–1348. PMLR, 2018.arxiv.org/abs/1712.00779
  21. [21]Andreas Engel. Statistical mechanics of learning. Cambridge University Press, 2001.DOI
  22. [23]Elizabeth Gardner and Bernard Derrida. Three unfinished works on the optimal storage capacity of networks. Journal of Physics A: Mathematical and General, 22(12):1983, 1989.
  23. [24]Federica Gerace, Luca Saglietti, Stefano Sarao Mannelli, Andrew SaXe, and Lenka Zdeborová. Probing transfer learning with a model of synthetic correlated datasets. Machine Learning: Science and Technology, 3(1):015030, 2022.DOI
  24. [25]Sebastian Goldt, Madh u Advani, Andrew M Saxe, Florent Krzakala, and Lenka Zdeborová. Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup. Advances in neural information processing systems, 32, 2019.
  25. [26]Sebastian Goldt, Marc Mézard, Florent Krzakala, and Lenka Zdeborová. Modeling the influence of data structure on learning in neural networks: The hidden manifold model. Physical Review X, 10 (4):041044, 2020.
  26. [27]Sebastian Goldt, Bruno Loureiro, Galen Reeves, Florent Krzakala, Marc Mézard, and Lenka Zdeborová. The gaussian equivalence of generative models for learning with shallow neural networks. In Mathematical and Scientific Machine Learning, pages 426–471. PMLR, 2022.
  27. [28]Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey, 2024. URL https://arxiv.org/abs/2403.14608.
  28. [29]Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022.arxiv.org/abs/2203.15556

Paper details

vlnc:W_q3recvna
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

LLMs

View statement
Looking for related papers…
Contents