betaFeedback

WorksComputer Science

BMuon: Budgeting Muon's Spectral Geometry for Structured Compressibility

Abstract

Muon has emerged as a strong optimizer for neural network training and admits an interpretation as steepest descent under a spectral-norm geometry. Yet we find that Muon-trained Transformers can degrade sharply when their MLP hidden channels are pruned after training. To reconcile Muon's optimization performance with structured compressibility, we introduce Budget Muon (BMuon), which modifies Muon's spectral geometry so that hidden channels across blocks compete for a shared allocation budget. The resulting oracle is exactly normalized steepest descent under a family of norms that interpolate between Muon's geometry and structured sparse geometries. Our principal sparsity control is an interpretable global channel-allocation budget, rather than a setting-dependent sparsity-penalty coefficient. For linear factorization models, we show that Muon admits minimum-cost exact representations with redundant hidden width, whereas under the BMuon oracle geometry and a sufficiently scarce budget, every minimum-cost representation is width-minimal. For practical training, we derive a minorization-maximization surrogate for the exact allocation problem. On ImageNet-1K with ViT-S/16, BMuon improves over Muon under gradual structured MLP pruning, and its learned allocations provide a more effective pruning criterion than weight magnitude. On a 1B-parameter LLaMA-style model trained on FineWeb, BMuon improves over Muon under post-training MLP channel pruning.

Code: https://github.com/RidaAssalouh/BMuon

Introduction

Modern Transformer models incur substantial memory and computational costs at inference time, motivating compression for more efficient deployment [17]. Pruning is a natural approach to this problem, but practical gains depend on the sparsity structure: unstructured sparsity often requires specialized kernels or hardware, whereas structured pruning removes entire computational units and translates directly into efficiency gains [31, 2]. Feed-forward modules can account for roughly two thirds of Transformer parameters [10], making their hidden channels a natural target for structured compression. Global pruning ranks channels across layers jointly, allowing different layers to retain different numbers of channels [19, 2]. Post-pruning fine-tuning adds computational cost, especially for large models [17, 27], motivating compressibility to emerge during training so that pruning requires little or no recovery.

Muon has demonstrated strong training efficiency across model scales, from small vision and language-model benchmarks to billion-parameter language models [16, 20, 26]. Its matrix update can be viewed as a linear minimization oracle over a spectral-norm ball, giving Muon a steepest-descent interpretation in spectral-norm geometry [24]. Prior work has shown that implicit bias can depend on optimization geometry [13]. For Muon, recent theory identifies a spectral-norm margin bias in linear and smooth homogeneous models [9, 12]. Yet we find that Muon-trained Transformers degrade rapidly under structured MLP pruning, suggesting that Muon’s spectral geometry does not by itself encourage structured compressibility. As shown in Figure 1, a Muon-trained ViT-S/16 on ImageNet-1K collapses after only 30% global MLP channel pruning, with Top-1 accuracy falling from 75.23% to 3.17%. In contrast, our proposed Budget Muon (BMuon) algorithm retains 74.08% Top-1 accuracy after 70% pruning, nearly matching its 74.17% accuracy before pruning, with no post-pruning recovery. This raises the central question: can Muon’s spectral geometry be modified to encourage structured compressibility?

Figure 1. Post-training global MLP pruning on ImageNet-1K with ViT-S/16. Mean ±\pm std over 3 seeds. See Appendix E.§ 1.

To address this question, we introduce BMuon, which reweights Muon’s spectral geometry under a single global budget KK shared across all hidden channels of all MLP blocks. For hidden channel jj in block ℓ\ell, BMuon assigns an allocation μjℓ∈[0,1]\mu^\ell_j \in[0, 1] to the corresponding row of the up-projection matrix and column of the down-projection matrix, with ∑ℓ,jμjℓ=K\sum_{\ell,j}\mu^\ell_j = K. For a fixed allocation, BMuon introduces a diagonal reweighting inside Muon’s spectral-norm constraint, with the components of the update variable corresponding to channel jj in block ℓ\ell weighted according to its allocation μjℓ\mu^\ell_j. BMuon chooses the allocation globally, intuitively asking where the update budget should be spent to obtain the greatest first-order decrease in the loss.

NuMuon [8] is the closest prior compression-aware variant of Muon. It imposes a nuclear-norm constraint on Muon’s update direction to improve low-rank compressibility. BMuon instead targets structured compressibility over hidden channels by introducing a global channel budget into Muon’s spectral-norm geometry. Prior work has also incorporated compressibility into training. Lu et al. [21] train networks for unstructured pruning with Frank–Wolfe under fixed layerwise constraints on the weights, while Zimmer et al. [37] use Frank–Wolfe under fixed norm constraints for convolutional filter pruning and low-rank compression. BMuon instead leaves the weights unconstrained and modifies Muon’s spectral update geometry for global hidden-channel compression. Resource-constrained structure learning and global structured pruning have been studied across CNNs and Transformers [11, 33, 32, 2, 29]. GRASPrune [30] learns gates post hoc on a frozen LLM to select MLP channels and attention KV-head groups under a single global parameter budget, while DABP [18] jointly trains CNN weights and continuous channel gates under a global resource budget. BMuon uses no forward gates; its budget instead controls the channel-wise reweighting of Muon’s spectral update constraint during training.

Theoretically, we show that the BMuon oracle is exactly normalized steepest descent under an induced norm Ωα,K\Omega_{\alpha,K}, where α\alpha controls the strength of channel reweighting and KK is the global budget over the NN hidden channels. This family recovers Muon when α=0\alpha= 0 or K=NK = N. At α=1\alpha= 1, its unit ball is the convex hull of operator-norm-bounded updates supported on at most KK coupled hidden channels. This is related to group kk-support norms through the use of structured support [3, 25], but its atoms are bounded in operator norm rather than Euclidean norm. It is also related to spectral kk-support norms [22, 23], but differs in the notion of sparsity: spectral kk-support uses sparsity in singular values, whereas BMuon uses MLP-channel sparsity. At α=K=1\alpha= K = 1, the norm reduces to an ℓ1\ell_1 aggregation over coupled channel groups, yielding a group-lasso-like geometry [34]. We then characterize the representation cost induced by BMuon. Following Dai et al. [7], this is the minimum parameter-space cost among exact two-matrix factorizations that realize the same linear predictor. We prove that, under an explicit budget-scarcity condition, every minimum-cost BMuon factorization is width-minimal, whereas Muon’s minimum cost can also be attained by factorizations with redundant hidden width.

Solving the exact allocation problem at every training step is expensive. We therefore derive a minorization–maximization surrogate that yields a tractable allocation update, followed by the corresponding weighted Muon update. On CIFAR-100 and ImageNet-1K, BMuon outperforms Muon under both gradual and post-training global MLP-channel pruning, even when both use the same magnitude-based pruning criterion. Under gradual pruning, using the learned allocations themselves to rank channels further improves performance, showing that they also provide a useful measure of channel importance. The non-uniform architectures identified by BMuon in the gradual-pruning experiments also outperform matched uniform-width architectures when retrained from scratch, providing evidence that the learned allocation captures useful differences in how MLP capacity should be distributed across layers. Finally, on a 1B-parameter LLaMA-style model trained on FineWeb, BMuon improves robustness to post-training global MLP-channel pruning, showing that the effect extends to language modeling at substantially larger scale.

Contributions. To summarize, our main contributions are:

  • We introduce Budget Muon (BMuon), which incorporates a single global hidden-channel budget into Muon’s spectral update geometry, while coupling the matrix slices that define each MLP channel.

  • We characterize the induced norm family Ωα,K\Omega_{\alpha,K} and its relation to Muon and structured sparse geometries in § 2. In the linear factorization setting of § 3, we show that for a sufficiently scarce budget, every minimum-cost BMuon factorization is width-minimal, whereas Muon’s minimum cost is also attained by redundant-width factorizations.

  • We derive a practical minorization–maximization surrogate for the global allocation problem, yielding a tractable allocation update in § 4.

  • In § 5, we show across CIFAR-100, ImageNet-1K, and a 1B-parameter LLaMA-style model that BMuon improves structured MLP-channel compressibility relative to Muon. In the vision experiments, the learned allocations also provide effective pruning scores and identify non-uniform MLP architectures that outperform matched uniform-width architectures when retrained from scratch.

Budget Muon: Oracle and Geometry

Setup and Muon Preliminaries

MLP channels and global pruning. We consider LL single-hidden-layer MLP blocks, as in Transformer feed-forward modules, with block ℓ\ell given by

x↦W2ℓϕ(W1ℓx+bℓ),W1ℓ∈Rmℓ×dℓ,W2ℓ∈Rpℓ×mℓ,(1)x \mapsto W_2^\ell\phi\left(W_1^\ell x + b^\ell\right), \qquad W_1^\ell\in\mathbb{R}^{m_\ell\times d_\ell}, \qquad W_2^\ell\in\mathbb{R}^{p_\ell\times m_\ell}, \tag*{(1)}

where mℓm_\ell is the hidden width. Hidden channel jj is associated with its incoming weights W1,:,jℓW_{1,:,j}^\ell, bias bjℓb_j^\ell, and outgoing weights W2,:,jℓW_{2,:,j}^\ell. Pruning the channel removes these components jointly. The matrix parameters W1ℓ,W2ℓW_1^\ell, W_2^\ell are optimized by Muon, while biases are optimized separately by AdamW. We write W:=((W1ℓ,W2ℓ))ℓ=1LW := ((W_1^\ell, W_2^\ell))_{\ell=1}^L for the collection of MLP weight matrices and N:=∑ℓ=1LmℓN := \sum_{\ell=1}^L m_\ell for the total number of hidden channels across all blocks. Given channel scores sjℓs_j^\ell, global pruning ranks all NN channels jointly and removes the lowest-scoring ones until the target global sparsity is reached. The resulting hidden width in each block is therefore determined by this global ranking.

Muon as spectral-norm steepest descent. For an MLP matrix WiℓW_i^\ell, let MiℓM_i^\ell denote the momentum matrix used by Muon, following Appendix A. For any matrix AA with compact singular value decomposition (SVD) A=UΣV⊤A = U\Sigma V^\top, define its polar factor by Polar⁡(A):=UV⊤\operatorname{Polar}(A) := UV^\top. The exact Muon direction is Δi,Muonℓ=Polar⁡(Miℓ)\Delta_{i,\mathrm{Muon}}^\ell= \operatorname{Polar}(M_i^\ell), and WiℓW_i^\ell is updated in the opposite direction. In practice, Muon approximates this direction using a finite Newton–Schulz iteration [16]; throughout the theoretical analysis, we use the exact polar factor. By spectral–nuclear norm duality, the exact Muon direction satisfies

Δi,Muonℓ=Polar⁡(Miℓ)∈arg max⁡∥Δiℓ∥op≤1⟨Miℓ,Δiℓ⟩,max⁡∥Δiℓ∥op≤1⟨Miℓ,Δiℓ⟩=∥Miℓ∥∗.\Delta_{i,\mathrm{Muon}}^\ell= \operatorname{Polar}(M_i^\ell) \in\underset{\lVert\Delta_i^\ell\rVert_{\mathrm{op}}\leq1}{\operatorname{arg\,max}}\left\langle M_i^\ell,\Delta_i^\ell\right\rangle,\qquad\underset{\lVert\Delta_i^\ell\rVert_{\mathrm{op}}\leq1}{\max}\left\langle M_i^\ell,\Delta_i^\ell\right\rangle= \lVert M_i^\ell\rVert_*.

Here ⟨A,B⟩=tr⁡(ATB)\langle A, B\rangle= \operatorname{tr}(A^{\mathsf{T}}B) is the Frobenius inner product, ∥A∥op=σmax⁡(A)\lVert A\rVert_{\mathrm{op}} = \sigma_{\max}(A) is the operator norm, and ∥A∥∗=∑jσj(A)\lVert A\rVert_{*} = \sum_j \sigma_j(A) is the nuclear norm. Thus exact-polar Muon is normalized steepest descent under the spectral norm [4, 24]. For all MLP matrices, write M:=((M1ℓ,M2ℓ))ℓ=1LM := ((M_1^\ell, M_2^\ell))_{\ell=1}^{L} and Δ:=((Δ1ℓ,Δ2ℓ))ℓ=1L\Delta:= ((\Delta_1^\ell, \Delta_2^\ell))_{\ell=1}^{L}. We equip this product space with the inner-product ⟨M,Δ⟩:=∑ℓ=1L(⟨M1ℓ,Δ1ℓ⟩+⟨M2ℓ,Δ2ℓ⟩)\langle M, \Delta\rangle:= \sum_{\ell=1}^{L}(\langle M_1^\ell, \Delta_1^\ell\rangle+ \langle M_2^\ell, \Delta_2^\ell\rangle) and define Muon’s product-space unit ball AMuon:={Δ:∥Δ1ℓ∥op≤1, ∥Δ2ℓ∥op≤1, ∀ℓ}\mathcal{A}_{\mathrm{Muon}} := \{\Delta: \lVert\Delta_1^\ell\rVert_{\mathrm{op}} \le1,\ \lVert\Delta_2^\ell\rVert_{\mathrm{op}} \le1,\ \forall\ell\}. Equivalently, AMuon={Δ:∥Δ∥Muon≤1}\mathcal{A}_{\mathrm{Muon}} = \{\Delta: \lVert\Delta\rVert_{\mathrm{Muon}} \le1\}, where ∥Δ∥Muon:=max⁡ℓmax⁡{∥Δ1ℓ∥op,∥Δ2ℓ∥op}\lVert\Delta\rVert_{\mathrm{Muon}} := \max_\ell\max\{\lVert\Delta_1^\ell\rVert_{\mathrm{op}},\lVert\Delta_2^\ell\rVert_{\mathrm{op}}\}. Muon’s update direction over all MLP matrices is then

ΔMuon∈arg⁡max⁡Δ∈AMuon ⟨M,Δ⟩.(2)\Delta_{\mathrm{Muon}} \in\underset{\Delta\in\mathcal{A}_{\mathrm{Muon}}}{\arg\max}\ \langle M,\Delta\rangle. \tag*{(2)}

Since the objective and constraints separate across matrices, this is equivalent to applying the single-matrix Muon direction independently to each WiℓW_i^\ell.

The BMuon Oracle

Global channel budget. For each block ℓ\ell of width mℓm_\ell, let μℓ∈[0,1]mℓ\mu^\ell\in[0,1]^{m_\ell} assign an allocation to each hidden channel, and let Dμℓ:=diag⁡(μℓ)D_{\mu^\ell} := \operatorname{diag}(\mu^\ell). Writing μ:=(μ1,…,μL)∈[0,1]N\mu:= (\mu^1,\ldots,\mu^L) \in[0,1]^N, for a budget K∈(0,N]K \in(0,N], define KK:={μ∈[0,1]N:∑ℓ,jμjℓ=K}\mathcal{K}_K := \{\mu\in[0,1]^N : \sum_{\ell,j}\mu_j^\ell= K\}. All hidden channels share the budget KK, so channels across blocks compete directly for allocation.

References

  1. [1]Sai Aparna Aketi, Sourjya Roy, Anand Raghunathan, and Kaushik Roy. Gradual channel pruning while training using feature relevance scores for convolutional neural networks. IEEE Access, 8:171924–171932, 2020. doi: 10.1109/ACCESS.2020.3024992.DOI
  2. [2]Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10865–10873, 2024. doi: 10.1609/aaai.v38i10.28960.DOI
  3. [3]Andreas Argyriou, Rina Foygel, and Nathan Srebro. Sparse prediction with the k-support norm. In Advances in Neural Information Processing Systems, volume 25, 2012.arxiv.org/abs/1204.5043
  4. [4]Jeremy Bernstein and Laker Newhouse. Old optimizer, new norm: An anthology, 2024. URL https://arxiv.org/abs/2409.20325.
  5. [5]Lucas Beyer, Xiaohua Zhai, and Alexander Kolesnikov. Better plain ViT baselines for ImageNet-1k. arXiv preprint arXiv:2205.01580, 2022.arxiv.org/abs/2205.01580
  6. [6]Rajendra Bhatia. Matrix Analysis, volume 169 of Graduate Texts in Mathematics. Springer, New York, 1997.
  7. [7]Zhen Dai, Mina Karzand, and Nathan Srebro. Representation costs of linear neural networks: Analysis and design. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, 2021.
  8. [8]Hadi Mohaghegh Dolatabadi, Thalaiyasingam Ajanthan, Sameera Ramasinghe, Chamin P. Hewа Koneputugodage, Shamane Siriwardhana, Violetta Shevchenko, Karol Pajak, James Snewin, Gil Avraham, and Alexander Long. NuMuon: Nuclear-norm-constrained Muon for compressible LLM training, 2026.arxiv.org/abs/2603.03597
  9. [9]Chen Fan, Mark Schmidt, and Christos Thrampoulidis. Implicit bias of spectral descent and muon on multiclass separable data. In Advances in Neural Information Processing Systems (NeurIPS), volume 38, 2025.arxiv.org/abs/2502.04664
  10. [10]Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.emnlp-main.446.DOI
  11. [11]Ariel Gordon, Elad Eban, Ofir Nachum, Bo Chen, Hao Wu, Tien-Ju Yang, and Edward Choi. MorphNet: Fast & simple resource-constrained structure learning of deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1586–1595, 2018.arxiv.org/abs/1711.06798
  12. [12]Eitan Gronich and Gal Vardi. The implicit bias of adam and muon on smooth homogeneous neural networks, 2026. URL https://arxiv.org/abs/2602.16340.
  13. [13]Suriya Gunasekar, Jason Lee, Daniel Soudry, and Nathan Srebro. Characterizing implicit bias in terms of optimization geometry. In Proceedings of the 35th International Conference on Machine Learning (ICML), volume 80 of Proceedings of Machine Learning Research, pages 1832–1841, 2018.arxiv.org/abs/1802.08246
  14. [14]David R. Hunter and Kenneth Lange. A tutorial on mm algorithms. The American Statistician, 58(1):30–37, 2004. doi: 10.1198/0003130042836.DOI
  15. [15]Tom Jacobs, Chao Zhou, and Rebekka Burkholz. Never saddle for reparameterized steepest descent as mirror flow. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=YgudIlQ9nC.
  16. [16]Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks, 2024. URL https://kellerjordan.github.io/posts/muon/.
  17. [17]Woosuk Kwon, Sehoon Kim, Michael W. Mahoney, Joseph Hassoun, Kurt Keutzer, and Amir Gholami. A fast post-training pruning framework for transformers. In Advances in Neural Information Processing Systems, volume 35, 2022.arxiv.org/abs/2204.09656
  18. [18]Soumia Laroui, Hamida Seba, Karima Amrouche, and Yassine Ait Ali Yahia. DABP: Dual-augmented budgeted channel pruning via sensitivity-aware gating. Neurocomputing, 704: 134843, 2026. doi: 10.1016/j.neucom.2026.134843.DOI
  19. [19]Jaeho Lee, Sejun Park, Sangwoo Mo, Sungsoo Ahn, and Jinwoo Shin. Layer-adaptive sparsity for the magnitude-based pruning. In International Conference on Learning Representations, 2021.arxiv.org/abs/2010.07611
  20. [20]Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, Yanru Chen, Huabin Zheng, Yibo Liu, Shaowei Liu, Bohong Yin, Weiran He, Han Zhu, Yuzhi Wang, Jianzhou Wang, Mengnan Dong, Zheng Zhang, Yongsheng Kang, Hao Zhang, Xinran Xu, Yutao Zhang, Yuxin Wu, Xinyu Zhou, and Zhilin Yang. Muon is scalable for LLM training. arXiv preprint arXiv:2502.16982, 2025.arxiv.org/abs/2502.16982
  21. [21]Miao Lu, Xiaolong Luo, Tianlong Chen, Wuyang Chen, Dong Liu, and Zhangyang Wang. Learning pruning-friendly networks via frank–wolfe: One-shot, any-sparsity, and no re-training. In International Conference on Learning Representations, 2022.
  22. [22]Andrew M. McDonald, Massimiliano Pontil, and Dimitris Stamos. Spectral k-support norm regularization. In Advances in Neural Information Processing Systems, volume 27, 2014.
  23. [23]Andrew M. McDonald, Massimiliano Pontil, and Dimitris Stamos. Fitting spectral decay with the k-support norm. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS), 2016.arxiv.org/abs/1601.00449
  24. [24]Thomas Pethick, Wanyun Xie, Kimon Antonakopoulos, Zhenyu Zhu, Antonio Silveti-Falls, and Volkan Cevher. Training deep learning models with norm-constrained LMOs. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 49069–49104. PMLR, 2025.arxiv.org/abs/2502.07529
  25. [25]Nikhil Rao, Miroslav Dudík, and Zaïd Harchaoui. The group k-support norm for learning with structured sparsity. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2402–2406, 2017. doi: 10.1109/ICASSP.2017.7952587.DOI
  26. [26]Ishaan Shah, Anthony M. Polloreno, Karl Stratos, Philip Monk, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, Anil Thomas, Ashish Tanwer, Darsh J. Shah, Khoi Nguyen, Kurt Smith, Michael Callahan, Michael Pust, Mohit Parmar, Peter Rushton, Platon Mazarakis, Ritvik Kapila, Saurabh Srivastava, Somanshu Singla, Tim Romanski, Yash Vanjani, and Ashish Vaswani. Practical efficiency of muon for pretraining, 2025.arxiv.org/abs/2505.02222
  27. [27]Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models. In International Conference on Learning Representations, 2024.arxiv.org/abs/2306.11695
  28. [28]Nikolaos Tsilivis, Gal Vardi, and Julia Kempe. Flavors of margin: Implicit bias of steepest descent in homogeneous neural networks. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2410.22069
  29. [29]Ziyan Wang, Enmao Diao, Qi Le, Pu Wang, Minwoo Lee, Shu-ping Yeh, Evgeny Stupachenko, Hao Feng, and Li Yang. From local to global: Revisiting structured pruning paradigms for large language models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 35720–35739. Association for Computational Linguistics, 2026a. doi: 10.18653/v1/2026.acl-long.1653. URL https://aclanthology.org/2026.acl-long.1653/.DOI
  30. [30]Ziyang Wang, Jiangfeng Xiao, Chuan Xiao, Ruoxiang Li, Rui Mao, and Jianbin Qin. GRASPrune: Global gating for budgeted structured pruning of large language models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10719–10736. Association for Computational Linguistics, 2026b. doi: 10.18653/v1/2026.acl-long.491. URL https://aclanthology.org/2026.acl-long.491/.DOI
  31. [31]Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. In Advances in Neural Information Processing Systems, volume 29, 2016.arxiv.org/abs/1608.03665
  32. [32]Huanrui Yang, Hongxu Yin, Maying Shen, Pavlo Molchanov, Hai Li, and Jan Kautz. Global vision transformer pruning with hessian-aware saliency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18547–18557, 2023.arxiv.org/abs/2110.04869
  33. [33]Shixing Yu, Tianlong Chen, Jiayi Shen, Huan Yuan, Jianchao Tan, Sen Yang, Ji Liu, and Zhangyang Wang. Unified visual transformer compression. In International Conference on Learning Representations, 2022.arxiv.org/abs/2203.08243
  34. [34]Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1):49–67, 2006. doi: 10.1111/j.1467-9868.2005.00532.x.DOI
  35. [35]Ruijie Zhang, Yeq uan Zhao, Ziyue Liu, Zhengyang Wang, and Zheng Zhang. Muon+: Towards better muon via one additional normalization step. arXiv preprint arXiv:2602.21545, 2026.
  36. [36]Wenzhi Zhong, Edward Milsom, and Michael Murray. Sharpness-aware minimization and Muon: Robustness under the spectral norm. arXiv preprint arXiv:2607.26001, 2026.arxiv.org/abs/2607.26001
  37. [37]Max Zimmer, Christoph Spiegel, and Sebastian Pokutta. Compression-aware training of neural networks using frank–wolfe. In Mathematical Optimization for Machine Learning, pages 137–168. De Gruyter, 2025. doi: 10.1515/9783111376776-010.DOI

Paper details

vlnc:W_xv7mk2e8
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents