betaFeedback

WorksComputer Science

Loss is Not Behavior: A Unified Output-Space Analysis of Gradient-Based Machine Unlearning

Abstract

Gradient-based machine unlearning methods typically ``unlearn'' by optimizing forget and retain losses, but similar losses do not necessarily imply similar output-space behavior. Prior work has shown that high forget loss alone does not certify that unlearning has produced retraining-like output behavior. We therefore ask: when a gradient-based unlearner differs from the model that would have been learned on only the retain data, what structure does the resulting output-space gap have? We derive closed-form expressions for this gap in linear regression, with an extension to nonlinear models via a generalized Gauss-Newton and local linearity approximations. Our analysis puts various gradient-based methods under a unified framework parametrized by geometry, reweighting, and projection. Two leverage matrices emerge that capture the forget-retain entanglement under the retrained model geometry: forget self-leverage and retain cross-leverage. We show both theoretically and empirically that these two leverage matrices drive the impact of the forget residual on the forget and retain outputs, and existing methods account for this geometry only partially. Our framework yields precomputable leverage diagnostics for auditing forget sets to identify what type of gradient-based unlearning is likely to be successful.

Introduction

Modern machine learning increasingly uses sensitive, copyrighted, or safety-critical data. As a result, machine unlearning, the post-hoc removal of a data’s influence on the learned model, has become a practical and regulatory concern [2, 7]. The gold standard for machine unlearning is conceptually simple: retrain the model on the retained data after deleting the forget set. For large models and frequent deletion requests, however, exact retraining is usually too expensive, so practical unlearning methods seek more efficient approximations to this retrained gold standard model.

One family of approximations uses Newton-style updates to move directly toward the retrained optimum, but these methods depend on Hessian information and are often not scalable to large models [15, 18, 39]. The dominant practical alternative is gradient-based unlearning: gradient ascent (GA) increases loss on the forget set, while GradDiff, NPO, SimNPO, IMU, and projected variants add retain losses, per-sample weights, or projections to preserve utility [8, 9, 17, 24, 38, 40]. These methods are usually tuned and compared through forget and retain losses.

Recent work has established that loss-space, parameter-space agreement with retraining, and even finite output-imitation tests are insufficient evaluation criteria for unlearning [21, 37]. We verify this empirically in Figure 1: at matched forget loss or similar weights, different gradient-based unlearners can have very different output-space behavior and forget performance, and they can all differ substantially from the retrained model. This motivates output-space deviation from retraining as the evaluation surface, which leads to the open question we study here: what governs the structure of this deviation for gradient-based unlearners? We show that the answer is a leverage geometry induced by the retrained model, and existing gradient-based variants control only parts of this geometry.

Figure 1. Loss and weights are not behavior. MNIST[20] with multi-layer perceptron (MLP) model for class forgetting with 10 seeds for various gradient-based methods and a Newton step unlearning. From left to right: forget cross-entropy (matched to the retrained reference, dashed), weight distance ∥θ−θr∗∥2\lVert\theta-\theta_r^*\rVert_2, KL(pθr∗∥pθp_{\theta_r^*}\lVert p_\theta) on DfD_f, and Top-1 forget accuracy. Setup is detailed in Appendix § 10.

Our key contributions are:

  1. Unified gap formula. We derive closed-form output-space gaps for gradient-based unlearners, exact in linear regression and extended to nonlinear models under a local generalized Gauss-Newton (GGN) approximation [31]. This gap shows a fundamental objective mismatch between common gradient-based methods and the gold standard.

  1. Two-leverage decomposition. Our derivation reveals two important components of the gap: forget self-leverage, which captures the uniqueness of forget samples under the retrained model geometry, and retain cross-leverage, which captures how strongly forget-directed changes transfer into retain outputs.

  1. Forget-set diagnostics. We validate each method’s gap formula on MNIST and CIFAR-10, derive a closed-form sufficient rule for ρ\rho, and empirically validate precomputable leverage diagnostics for LLM scale.

Related Work

Machine unlearning methods

Early formulations of machine unlearning formalized data deletion through exact retraining, certified removal, or influence-style approximations to retraining [2, 15, 18]. Newton-style methods are attractive because they target the retrained optimum directly, but they require Hessian or inverse-Hessian operations and rely on local approximations that can be costly or fragile at modern scale [11, 39]. A related line linearizes the network around a reference point (pretrained or core-trained weights) and forgets exactly in the resulting linear, typically squared-loss, model. Examples include NTK-based scrubbing [12], mixed-privacy forgetting [13], and parameter-efficient variants [22], made practical by fast empirical-NTK approximations [30, 35]. Within that linear model they are exact retraining, i.e. the Newton Step row of Table 1, so their output gap to θr∗\theta_r^* is governed by linearization error and the locality limits of § 4.3. While Newton-style methods are limited in practice, the retrained-model Hessian they use as a preconditioner is the same geometric object that, as we show, implicitly governs the output-space gap of gradient-based unlearners.

MethodOutput gap Xa(θρmethod−θr∗)X_a(\theta_\rho^{\mathrm{method}}-\theta_r^*)Geometry
GradDiffρnKaf(I−ρnKff)−1ef\frac{\rho}{n}K_{af}\left(I-\frac{\rho}{n}K_{ff}\right)^{-1}e_fretrained Hessian
NPOρnKaf(I−ρnWNPOKff)−1WNPOef\frac{\rho}{n}K_{af}\left(I-\frac{\rho}{n}W_{\mathrm{NPO}}K_{ff}\right)^{-1}W_{\mathrm{NPO}}e_fretrained Hessian
SimNPOρnKaf(I−ρnWSimNPOKff)−1WSimNPOef\frac{\rho}{n}K_{af}\left(I-\frac{\rho}{n}W_{\mathrm{SimNPO}}K_{ff}\right)^{-1}W_{\mathrm{SimNPO}}e_fretrained Hessian
Pure GA−Xaθr∗−ρnλGaf(I−ρnλGff)−1yf-X_a\theta_r^*-\frac{\rho}{n\lambda}G_{af}\left(I-\frac{\rho}{n\lambda}G_{ff}\right)^{-1}y_fEuclidean Gram
IMU−Xaθr∗−ρnλGaf(I−ρnλAIMUGff)−1AIMUyf-X_a\theta_r^*-\frac{\rho}{n\lambda}G_{af}\left(I-\frac{\rho}{n\lambda}A_{\mathrm{IMU}}G_{ff}\right)^{-1}A_{\mathrm{IMU}}y_fEuclidean Gram
ProjGD (fixed)ρnKafP(I−ρnKffP)−1ef\frac{\rho}{n}K_{af}^{P}\left(I-\frac{\rho}{n}K_{ff}^{P}\right)^{-1}e_fretrained Hessian (projected)
Newton Step00retrained Hessian (Newton)

Table 1. Theoretical output gaps from θr∗\theta_r^* in the linear setting. Each row assumes that its displayed inverse exists. The GradDiff, Pure GA, fixed-weight IMU, fixed-projector ProjGD, and Newton Step rows are exact for the displayed linear objectives; NPO and SimNPO are frozen-weight local forms, with weights evaluated at θr∗\theta_r^*. The geometry column distinguishes methods that route forget perturbations through Hr−1H_r^{-1} (top group: GradDiff, NPO, SimNPO, fixed-projector ProjGD) from those that use raw Euclidean Gram structure (Pure GA, IMU). Newton Step recovers θr∗\theta_r^* exactly in linear regression. Proofs in Appendix § 9.6.

Most practical large language model (LLM) unlearning methods instead optimize surrogate objectives. GA on the forget loss is the simplest example [17]; GradDiff adds retain descent to improve utility preservation [24]; NPO and SimNPO stabilize GradDiff-style unlearning through preference-style losses and reweighting [8, 38]; IMU introduces influence-guided sample weights [9]; and GU-style projected methods reduce retain drift by projecting updates away from retain-gradient directions [40]. Another LLM baseline is Representation Misdirection Unlearning (RMU) [23], but it operates on intermediate representations rather than the output-loss gradient-based family analyzed in this work.

Limits of loss-space and parameter-space evaluation

Several lines of existing work show from different angles that loss, output imitation on a limited evaluation set, and parameter agreement are not enough to certify unlearning. In general prediction problems, predictive multiplicity shows that competing models with similar loss can assign conflicting predictions [28]. In unlearning, robustness studies show that superficial “forgetting” can be reversed by relearning or adversarial evaluation [5, 16, 21, 25, 26]. Other work argues that parameter-space or algorithmic equivalence criteria are themselves delicate: verification-error analyses motivate proximity to retraining, while auditable definitions and retrain-equivalence impossibility results expose limits of parameter- or path-oblivious formulations [33, 34, 37]. In the adjacent setting of catastrophic forgetting, Shenfeld et al. [32] show that the forward KL to the original model on the new task predicts the loss of prior-task performance. We share the output-space viewpoint, but measure per-example gaps to the retrained reference and decompose them by forget–retain geometry.

Retain-forget entanglement and geometry

The closest work to ours studies why gradient-based unlearning fails when forget and retain data interact. Mavrothalassitis et al. [29] attribute failures of ascent-based methods to statistical dependence between forget and retain sets. Other recent analyses study model resistance, stability, semantic entanglement, gradient conflict, and retain-gradient geometry [1, 3, 4, 36, 40]. These works diagnose retain-forget interaction through loss, gradient alignment, empirical instability, coherence, or semantic similarity. The main distinction of our work is the focus on output space: if an unlearned model is meant to approximate gold-standard retraining, then its outputs, not only its losses or parameters, are the relevant behavioral object. Our unified analysis reveals the matrices through which this interaction propagates to outputs: forget self-leverage and retain cross-leverage under the retrained model geometry.

Problem Setup

In this section, we formally define the unlearning and the gold-standard retraining objectives. We consider a training set D=Dr∪Df\mathcal{D} = \mathcal{D}_r \cup\mathcal{D}_f with ∣D∣=n|\mathcal{D}| = n and ∣Df∣=m|\mathcal{D}_f| = m, where Df\mathcal{D}_f is the forget set whose influence is to be removed and Dr=D∖Df\mathcal{D}_r = \mathcal{D} \setminus\mathcal{D}_f is the retain set. Each sample (xi,yi)(x_i, y_i) pairs an input xix_i with a label yiy_i. A model f(θ,x)∈Rcf(\theta, x) \in\mathbb{R}^{c} parameterized by θ∈Rd\theta\in\mathbb{R}^{d} is trained on D\mathcal{D} under the ℓ2\ell_2-regularized objective

L(θ)=1n∑i=1nℓ(θ,xi,yi)+λ2∥θ∥2,λ>0,(1)\mathcal{L}(\theta) = \frac{1}{n}\sum_{i=1}^{n}\ell(\theta, x_i, y_i) + \frac{\lambda}{2}\|\theta\|^2,\qquad\lambda> 0, \tag*{(1)}

where λ\lambda is the regularization strength and ℓ(⋅)\ell(\cdot) is the loss function. The full-data optimum is θ⋆=arg⁡min⁡θL(θ)\theta^\star= \arg\min_\theta\mathcal{L}(\theta).

The gold standard. The gold-standard unlearned model is the minimizer of an analogous objective restricted to Dr\mathcal{D}_r. We adopt the full-data-normalized retain objective

R(θ):=1n∑i∈Drℓ(θ,xi,yi)+λ2∥θ∥2,θr⋆:=arg⁡min⁡θR(θ),(2)\mathcal{R}(\theta) := \frac{1}{n}\sum_{i\in\mathcal{D}_r}\ell(\theta, x_i, y_i) + \frac{\lambda}{2}\|\theta\|^2,\qquad\theta_r^\star:= \arg\min_\theta\mathcal{R}(\theta), \tag*{(2)}

which has the same minimizer as the standard 1n−m\frac{1}{n-m}-normalized retain objective 1n−m∑i∈Drℓ(θ,xi,yi)+λr2∥θ∥2\frac{1}{n-m}\sum_{i\in\mathcal{D}_r}\ell(\theta, x_i, y_i) + \frac{\lambda_r}{2}\|\theta\|^2 with a scaled regularization strength λr=nn−mλ\lambda_r = \frac{n}{n-m}\lambda. We refer to θr⋆\theta_r^\star as the gold standard: any unlearning method ideally returns a model whose induced behavior on arbitrary inputs matches that of θr⋆\theta_r^\star. Define also the full-data-normalized forget loss

F(θ):=1n∑j∈Dfℓ(θ,xj,yj),(3)\mathcal{F}(\theta) := \frac{1}{n}\sum_{j\in\mathcal{D}_f}\ell(\theta, x_j, y_j), \tag*{(3)}

so that L=R+F\mathcal{L} = \mathcal{R} + \mathcal{F}.

Gradient-objective unlearning. Most existing optimization-based unlearners minimize a controlled retain–forget trade-off

References

  1. [1]Sadia Asif and Mohammad Mohammadi Amiri. Ofmu: Optimization-driven framework for machine unlearning. arXiv preprint arXiv:2509.22483, 2025.arxiv.org/abs/2509.22483
  2. [2]Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pages 463–480. IEEE, 2015.DOI
  3. [3]Wei-Kai Chang and Rajiv Khanna. Why some models resist unlearning: A linear stability perspective. arXiv preprint arXiv:2602.02986, 2026.arxiv.org/abs/2602.02986
  4. [4]Jingpu Cheng, Ping Liu, Qianxiao Li, and Chi Zhang. Machine unlearning under retain-forget entanglement. arXiv preprint arXiv:2603.26569, 2026.arxiv.org/abs/2603.26569
  5. [5]Aghyad Deeb and Fabien Roger. Do unlearning methods remove information from language model weights? arXiv preprint arXiv:2410.08827, 2024.arxiv.org/abs/2410.08827
  6. [6]Vineeth Dorna, Anmol Mekala, Wenlong Zhao, Andrew McCallum, Zachary C Lipton, J Zico Kolter, and Pratyush Maini. OpenUnlearning: Accelerating LLM unlearning via unified benchmarking of methods and metrics. arXiv preprint arXiv:2506.12618, 2025. URL https://arxiv.org/abs/2506.12618.
  7. [7]European Parliament and Council of the European Union. Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/ec (general data protection regulation). Official Journal of the European Union, OJ L 119, 4.5.2016, pp. 1–88, 2016. URL https://eur-lex.europa.eu/eli/reg/2016/679/oj. Accessed via EUR-Lex.
  8. [8]Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. Simplicity prevails: Rethinking negative preference optimization for llm unlearning. arXiv preprint arXiv:2410.07163, 2024.arxiv.org/abs/2410.07163
  9. [9]Xindi Fan, Jing Wu, Mingyi Zhou, Pengwei Liang, and Dinh Phung. IMU: Influence-guided machine unlearning. arXiv preprint arXiv:2508.01620v1, 2025. URL https://arxiv.org/abs/2508.01620v1.
  10. [10]Ryan Giordano, William Stephenson, Runjing Liu, Michael Jordan, and Tamara Broderick. A swiss army infinitesimal jackknife. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1139–1147. PMLR, 2019.arxiv.org/abs/1806.00550
  11. [11]Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9304–9312, 2020.arxiv.org/abs/1911.04933
  12. [12]Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In European Conference on Computer Vision (ECCV), 2020.arxiv.org/abs/2003.02960
  13. [13]Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, and Stefano Soatto. Mixed-privacy forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.arxiv.org/abs/2012.13431
  14. [14]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.arxiv.org/abs/2407.21783
  15. [15]Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. arXiv preprint arXiv:1911.03030, 2019.arxiv.org/abs/1911.03030
  16. [16]Shengyuan Hu, Yiwei Fu, Steven Wu, and Virginia Smith. Jogging the memory of unlearned models through targeted relearning attacks. In ICML 2024 Workshop on Foundation Models in the Wild, 2024.
  17. [17]Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanu­gen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14389–14408, 2023.
  18. [18]Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885–1894. PMLR, 2017.arxiv.org/abs/1703.04730
  19. [19]Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
  20. [20]Yann LeCun, Corinna Cortes, and CJ Burges. MNIST handwritten digit database. Dataset, n.d. URL https://huggingface.co/datasets/ylecun/mnist. Accessed October 28, 2026.DOI
  21. [21]Bruce W Lee, Addie Foote, Alex Infanger, Leni Shor, Harish Kamath, Jacob Goldman-Wetzler, Bryce Woodworth, Alex Cloud, and Alexander Matt Turner. Distillation robustifies unlearning. arXiv preprint arXiv:2506.06278, 2025.arxiv.org/abs/2506.06278
  22. [22]Guihong Li, Hsiang Hsu, Chun-Fu Chen, and Radu Marculescu. Fast-NTK: Parameter-efficient unlearning for large-scale models. arXiv preprint arXiv:2312.14923, 2023.arxiv.org/abs/2312.14923
  23. [23]Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218, 2024.arxiv.org/abs/2403.03218
  24. [24]Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. In Conference on Lifelong Learning Agents, pages 243–254. PMLR, 2022.arxiv.org/abs/2203.12817
  25. [25]Jakub Łucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, and Javier Rando. An adversarial perspective on machine unlearning for ai safety. arXiv preprint arXiv:2409.18025, 2024.arxiv.org/abs/2409.18025
  26. [26]Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Menell. Eight methods to evaluate robust unlearning in llms. arXiv preprint arXiv:2402.16835, 2024.arxiv.org/abs/2402.16835
  27. [27]Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Chase Lipton, and J Zico Kolter. TOFU: A task of fictitious unlearning for LLMs. In First Conference on Language Modeling, 2024.arxiv.org/abs/2401.06121
  28. [28]Charles Marx, Flavio Calmon, and Berk Ustun. Predictive multiplicity in classification. In International conference on machine learning, pages 6765–6774. PMLR, 2020.arxiv.org/abs/1909.06677
  29. [29]Ioannis Mavrothalassitis, Pol Pudigremont, Noam Itzhak Levi, and Volkan Cevher. Ascent fails to forget. arXiv preprint arXiv:2509.26427, 2025.arxiv.org/abs/2509.26427
  30. [30]Mohamad Amin Mohamadi, Wonho Bae, and Danica J Sutherland. A fast, well-founded approximation to the empirical neural tangent kernel. In International Conference on Machine Learning. PMLR, 2023.DOI
  31. [31]Nicol N SchraudoIph. Fast curvature matrix-vector products for second-order gradient descent. Neural computation, 14(7):1723–1738, 2002.pubmed.ncbi.nlm.nih.gov/12079553
  32. [32]Idan Shenfeld, Jyotish Pari, and Pulkit Agrawal. RL’s razor: Why online reinforcement learning forgets less. arXiv preprint arXiv:2509.04259, 2025.arxiv.org/abs/2509.04259
  33. [33]Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Understanding factors influencing machine unlearning. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pages 303–319. IEEE, 2022.arxiv.org/abs/2109.13398
  34. [34]Anvith Thudi, Hengrui Jia, Ilia Shumailov, and Nicolas Papernot. On the necessity of auditable algorithmic definitions for machine unlearning. In 31st USENIX security symposium (USENIX Security 22), pages 4007–4022, 2022.arxiv.org/abs/2110.11891
  35. [35]Alexander Wei, Wei Hu, and Jacob Steinhardt. More than a toy: Random matrix models predict how real-world neural representations generalize. In International Conference on Machine Learning. PMLR, 2022.arxiv.org/abs/2203.06176
  36. [36]Zeguan Xiao, Siqing Li, Yong Wang, Xuetao Wei, Jian Yang, Yun Chen, and Guanhua Chen. Modeling llm unlearning as an asymmetric two-task learning problem. arXiv preprint arXiv:2604.14808, 2026.arxiv.org/abs/2604.14808
  37. [37]Jiatong Yu, Yinghui He, Anirudh Goyal, and Sanjeev Arora. On the impossibility of retrain equivalence in machine unlearning. arXiv preprint arXiv:2510.16629, 2025.arxiv.org/abs/2510.16629
  38. [38]Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.05868, 2024.arxiv.org/abs/2404.05868
  39. [39]Xingjian Zhao, Mohammad Mohammadi Amiri, and Malik Magdon-Ismail. Win-u: Woodbury-informed newton-unlearning as a retain-free machine unlearning framework. arXiv preprint arXiv:2604.13438, 2026.arxiv.org/abs/2604.13438
  40. [40]Duo Zhou, Yuji Zhang, Tianxin Wei, Ruizhong Qiu, Ke Yang, Xiao Lin, Cheng Qian, Jingrui He, Hanghang Tong, Chengxiang Zhai, et al. Geometric-disentanglement unlearning. arXiv preprint arXiv:2511.17100, 2025.

Paper details

vlnc:W_qcxd655u
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents