betaFeedback

WorksComputer Science

Efficient Exact Unlearning for Federated Zeroth-Order Optimization

Abstract

Federated zeroth-order optimization (FedZOO) reduces the memory and communication costs of collaborative training for edge intelligence by avoiding backpropagation while communicating only random seeds and scalar coefficients. However, existing methods lack efficient mechanisms for exact unlearning to fully remove the influence of training data when users request deletion. A key insight is that total variation stability of the learning algorithm creates overlap between model distributions before and after data deletion, allowing exact unlearning to reuse computation through coupling. Based on this observation, we jointly design two stable zeroth-order learning algorithms and their corresponding exact-unlearning procedures: FedZOO-S obtains stability through client and mini-batch subsampling, enabling FedZOUL-S to perform efficient unlearning by preserving earlier updates and restarting from the first affected step. FedZOO-NS further combines subsampling with Gaussian noise to increase overlap between update distributions; FedZOUL-NS exploits this overlap through rejection–reflection coupling to reuse affected updates when the coupling is accepted. We prove exact unlearning for both algorithms: the model and saved state have the same joint distribution as retraining with the corresponding learner on retained data. We further establish stability and convergence guarantees for the learning algorithms and derive upper bounds on expected unlearning cost. Experiments demonstrate effective unlearning with competitive model utility and up to 56.1%56.1\% less cumulative unlearning time than re-optimization. Compared with first-order baselines, our methods reduce communication overhead by up to approximately four orders of magnitude.

Introduction

Driven by advances in mobile and IoT hardware, edge intelligence has developed rapidly in recent years. Federated learning (FL) enables collaborative optimization across edge devices by allowing them to exchange model parameters or gradients without moving data out of the local domain [26]. However, practical deployment remains constrained by limited memory and communication resources, with growing model sizes further amplifying these challenges. Techniques such as quantization [27, 43, 47], sparsification [44, 37, 48], pruning and submodel training [28, 46, 36, 40], and knowledge distillation [29, 45, 49] can reduce computation or communication costs. More recently, parameter-efficient and subspace-based federated fine-tuning methods have been developed to adapt large language models (LLMs) with fewer trainable parameters [30, 35]. Nevertheless, these training approaches still rely on backpropagation, whose intermediate activations can impose substantial memory overhead even when only a small subset of parameters is trainable. Exchanging tensor-valued updates can also remain costly over bandwidth-constrained edge networks.

Federated zeroth-order optimization (FedZOO) provides a promising alternative. Unlike first-order methods, zeroth-order optimization (ZOO) estimates descent directions using only function evaluations and therefore avoids backpropagation [2]. This substantially reduces the memory required for optimization. Moreover, recent FedZOO methods exploit shared randomness to represent high-dimensional perturbations by random seeds and scalar coefficients [1], enabling communication costs that do not scale with the model dimension. In particular, a fixed finite seed pool allows accumulated zeroth-order updates to be represented in a compact seed-coordinate space. These properties make FedZOO particularly attractive for collaborative optimization of large models on resource-constrained edge devices.

Existing FedZOO methods, however, do not address machine unlearning (MU). Federated systems may receive requests to remove training samples because of data corrections, user withdrawal, or privacy requirements such as the right to be forgotten under regulations under GDPR [31]. Simply deleting the requested samples from local storage is insufficient because their effects may already have been incorporated into the global model. Federated unlearning therefore aims to remove these historical contributions from the learned model without repeating the entire training process. This requirement is especially relevant to edge intelligence, where participating devices may continuously contribute user-generated and potentially sensitive data.

Achieving efficient exact unlearning for FedZOO is challenging. Existing federated unlearning methods are largely designed around first- or second-order optimization and can reintroduce backpropagation, high-dimensional model states, or tensor-level communication, thereby losing the resource advantages that motivate FedZOO. Moreover, exact unlearning requires distributional equivalence to rerunning the learning algorithm on the retained data [3], for which retraining from scratch is always valid but computationally expensive.

A key insight from stability-based exact unlearning is that this difficulty can be addressed from the learning side [33, 3, 41]. If a randomized learning algorithm is stable in total variation under a single-sample change, then the output distributions on neighboring datasets overlap, creating opportunities to reuse computation through an appropriate coupling. This leads to a fundamental question in our setting:

To answer this question, we develop two TV-stable FedZOO learning algorithms and their corresponding exact-unlearning procedures, built on a common seed-encoded representation. The first, FedZOO-S, obtains stability from client and mini-batch subsampling: a single sample changes the realized trajectory only when its client participates and the sample is selected for a local update. Its unlearning counterpart, FedZOUL-S, therefore preserves the unaffected prefix and replays only from the first affected step. The second, FedZOO-NS, augments subsampling with Gaussian randomization. The added noise creates overlap between the conditional update distributions on neighboring datasets, allowing some affected updates to be coupled rather than immediately replayed. Its unlearning counterpart, FedZOUL-NS, exploits this overlap through rejection–reflection coupling so that some affected realized updates can still be reused exactly.

Our contributions are summarized as follows.

  1. We formulate efficient exact federated unlearning for zeroth-order optimization through the lens of algorithmic total variation stability. Rather than treating unlearning as a post hoc correction to an arbitrary FedZOO procedure, we design the learning algorithm itself to be TV-stable, so that neighboring training datasets induce overlapping output distributions that can be exploited through coupling for computation reuse. At the same time, we retain the fixed finite seed representation of FedZOO to avoid high-dimensional model-update communication.

  2. We propose FedZOO-S, a TV-stable federated zeroth-order learning algorithm whose stability arises from client and mini-batch subsampling, together with its exact-unlearning algorithm FedZOUL-S. A deletion leaves the realized trajectory unchanged when the deleted samples have not contributed to any realized local update. Otherwise, FedZOUL-S preserves the unaffected prefix and replays only the suffix starting from the first affected local step.

  3. We further propose FedZOO-NS, which augments subsampling with Gaussian randomization, together with its corresponding exact-unlearning algorithm FedZOUL-NS. The added noise creates overlap between the conditional update distributions on neighboring datasets, enabling rejection–reflection coupling. Consequently, some affected realized updates can still be reused exactly, and replay is required only after the first rejected coupling.

  4. We establish exact-unlearning guarantees for both learning–unlearning pairs and analyze their TV stability, optimization utility, and unlearning efficiency. We further evaluate their optimization and unlearning performance across federated learning settings with conventional models and large language models.

Federated Zeroth-Order Optimization.

In recent years, zeroth-order optimization (ZO), has shown strong potential in FL, especially in resource-constrained edge collaboration settings, because it avoids backpropagation. Fang et al. [20] propose FedZO as one of the first derivative free federated ZO methods. For LLMs, Ling et al. [21] propose FedMeZO, which significantly reduces memory overhead and provides convergence analysis. Later, FedKSeed [1] and DeComFL [8] use scalar gradients and random seeds to synchronize perturbation randomness, achieving dimension-independent communication between edge nodes and the cloud. HiSo [22] further accelerate the convergence efficiency of federated ZO. At the same time, since transmitting estimated gradients or scalar gradients may expose privacy risks, methods such as DPZero [23], DP-ZOSO [24] and DP-AggZO [25] combine differential privacy with zeroth-order gradient estimation. These methods inject carefully calibrated noise during training to provide rigorous privacy guarantees. It is worth noting that this noise-based design not only strengthens privacy protection, but also motivates the noise perturbation strategy in our TV stable algorithm.

Federated Unlearning.

Federated unlearning (FU) aims to remove the influence of specified training data from a collaboratively learned model. Existing approaches can be broadly categorized into approximate and exact unlearning according to their guarantees relative to retraining on the retained data. Approximate methods seek to recover retraining-like behavior through model correction and gradient calibration [9, 12, 14, 15, 13], perturbation [16, 17, 19, 39, 38], or knowledge distillation [18, 10]. Related work on LLM unlearning also seeks to remove targeted knowledge while preserving useful model capabilities [11, 42]. Although some approaches provide certified guarantees that bound distributional discrepancies through calibrated randomization [19, 39, 38], these guarantees do not establish exact equality between the unlearning and retraining distributions. Exact FU requires the unlearning output to follow the same distribution as training on the retained data. To avoid full retraining, stability-based approaches exploit overlap between learning distributions to reuse prior computation [3, 41]. However, these methods are developed around gradient-based training and model- or curvature-level states, which do not directly retain the memory and communication advantages of zeroth-order optimization. Extending them to FedZOO requires more than replacing the gradient oracle: seed-and-scalar updates change the state needed for replay and the transition distributions used for coupling. We therefore jointly design TV-stable zeroth-order learners and exact sample-level unlearning procedures around compact seed-encoded update histories. Our procedures reuse computation through subsampling and Gaussian coupling while preserving both the resource advantages of FedZOO and exact joint distributional equivalence of the model and saved state to retraining.

Preliminaries

Federated Zeroth-Order Optimization

Consider a federated optimization system with MM clients, where client c∈[M]c\in[M] holds a finite local dataset Dc={xc,1,…,xc,nc}\mathcal D_c=\{x_{c,1},\ldots,x_{c,n_c}\} of size nc=∣Dc∣n_c=|\mathcal D_c|. The clients collaboratively optimize a global model w∈Rd\boldsymbol w\in\mathbb R^d under the coordination of a server. We consider the uniformly weighted finite-sum objective

min⁡w∈Rdf(w)≔1M∑c=1Mfc(w),fc(w)≔1nc∑x∈Dcℓ(w;x).\begin{equation} \min_{\boldsymbol w\in\mathbb R^d} f(\boldsymbol w) \coloneqq \frac{1}{M} \sum_{c=1}^{M} f_c(\boldsymbol w), \qquad f_c(\boldsymbol w) \coloneqq \frac{1}{n_c} \sum_{x\in\mathcal D_c} \ell(\boldsymbol w;x). \tag*{(1)} \end{equation}

Training proceeds for RR communication rounds. In each round, mm clients participate, and each participating client performs EE local optimization steps before the server uniformly aggregates their updates.

Zeroth-order optimization estimates descent directions using function evaluations without backpropagation. Given a model w\boldsymbol w, a sample xx, and a perturbation z∼N(0,Id)\boldsymbol z\sim\mathcal N(\boldsymbol 0,I_d), the two-point scalar gradient is

g(w;x,z)=ℓ(w+μz;x)−ℓ(w−μz;x)2μ,\begin{equation} g(\boldsymbol w;x,\boldsymbol z) = \frac{ \ell(\boldsymbol w+\mu\boldsymbol z;x) - \ell(\boldsymbol w-\mu\boldsymbol z;x) }{ 2\mu }, \tag*{(2)} \end{equation}

where μ>0\mu>0 is the perturbation scale. The corresponding zeroth-order gradient estimator is g(w;x,z)zg(\boldsymbol w;x,\boldsymbol z)\boldsymbol z.

For a mini-batch B\mathcal B of size bb, suppose that qq perturbations {zi,j}j=1q\{\boldsymbol z_{i,j}\}_{j=1}^{q} are used for each xi∈Bx_i\in\mathcal B. One local step is the mini-batch averaged zeroth-order update

w+=w−ηbq∑xi∈B∑j=1qg(w;xi,zi,j)zi,j,\begin{equation} \boldsymbol w^{+} = \boldsymbol w - \frac{\eta}{bq} \sum_{x_i\in\mathcal B} \sum_{j=1}^{q} g(\boldsymbol w;x_i,\boldsymbol z_{i,j}) \boldsymbol z_{i,j}, \tag*{(3)} \end{equation}

where all bqbq scalar gradients are evaluated at the same pre-update model w\boldsymbol w.

Following the seed-based representation of FedKSeed [1], a perturbation can be reproduced from its random seed when the same pseudorandom number generator is used across devices. A fixed shared pool of KK seeds can therefore represent zeroth-order updates through their scalar coefficients in seed-coordinate space, avoiding the transmission and storage of dd-dimensional update vectors.

Federated Unlearning

Let D={D1,…,DM}\mathcal D = \{\mathcal D_1,\ldots,\mathcal D_M\} denote the federated dataset. A randomized federated learning algorithm O\mathcal O produces a model output together with auxiliary state maintained for subsequent unlearning:

(W,S)∼O(D).\begin{equation} (\boldsymbol W,\boldsymbol S) \sim \mathcal O(\mathcal D). \tag*{(4)} \end{equation}

Here, W\boldsymbol W denotes the model output and S\boldsymbol S collects the internal state saved by the learning procedure. The randomness of O\mathcal O may include client participation, data sampling, zeroth-order perturbations, and other algorithmic randomness. We use Oout(D)\mathcal O_{\mathrm{out}}(\mathcal D) to denote the marginal distribution of the model output W\boldsymbol W.

An unlearning request specifies samples to be removed from one or more clients. Let Cul\mathcal C_{ul} be the set of affected clients and Dc,ul⊆Dc\mathcal D_{c,ul}\subseteq\mathcal D_c the samples requested for deletion from client c∈Culc\in\mathcal C_{ul}. The retained federated dataset is

D−={D1−,…,DM−},Dc−={Dc∖Dc,ul,c∈Cul,Dc,c∉Cul.\mathcal D^{-} = \{\mathcal D_1^{-},\ldots,\mathcal D_M^{-}\}, \qquad \mathcal D_c^{-} = \begin{cases} \mathcal D_c\setminus\mathcal D_{c,ul}, & c\in\mathcal C_{ul},\\ \mathcal D_c, & c\notin\mathcal C_{ul}. \end{cases}

We consider valid deletion requests for which every retained local dataset remains large enough for the prescribed mini-batch sampling, i.e., ∣Dc−∣≥b|\mathcal D_c^{-}|\ge b for all affected clients.

The request may arrive during training. We denote by (Rul,tul)(R_{ul},t_{ul}) the local training position immediately preceding the request; that is, the request arrives immediately after local step tult_{ul} of round RulR_{ul}. Given the current model and saved state, an unlearning algorithm U\mathcal U updates them according to the deletion request:

(Wul,Sul)=U(W,S,{Dc,ul}c∈Cul).(\boldsymbol W_{ul},\boldsymbol S_{ul}) = \mathcal U \left( \boldsymbol W, \boldsymbol S, \{\mathcal D_{c,ul}\}_{c\in\mathcal C_{ul}} \right).

Definition 1 (Exact Federated Unlearning). A federated unlearning algorithm U\mathcal U is exact with respect to a randomized learning algorithm O\mathcal O if, for every federated dataset D\mathcal D and every valid deletion request,

U(O(D),{Dc,ul}c∈Cul)=dO(D−),\begin{equation} \mathcal U \left( \mathcal O(\mathcal D), \{\mathcal D_{c,ul}\}_{c\in\mathcal C_{ul}} \right) \overset{d}{=} \mathcal O(\mathcal D^{-}), \tag*{(5)} \end{equation}

where =d\overset{d}{=} denotes equality in distribution over both the model output and the maintained internal state.

Thus, exact unlearning requires the post-unlearning procedure to have the same distribution as rerunning the original learning algorithm on the retained federated dataset, rather than merely producing a nearby model.

Total Variation Stability

For two probability measures PP and QQ on the same measurable space, their total variation (TV) distance is

TV⁡(P,Q)=sup⁡A∣P(A)−Q(A)∣,\begin{equation} \operatorname{TV}(P,Q) = \sup_{A} |P(A)-Q(A)|, \tag*{(6)} \end{equation}

where the supremum is over measurable events AA. When PP and QQ admit densities pp and qq with respect to a common reference measure,

TV⁡(P,Q)=12∥p−q∥1.\operatorname{TV}(P,Q) = \frac{1}{2}\|p-q\|_1.

For two federated datasets D={Dc}c=1M\mathcal D=\{\mathcal D_c\}_{c=1}^{M} and D′={Dc′}c=1M\mathcal D'=\{\mathcal D'_c\}_{c=1}^{M}, define

Δ(D,D′)=∑c=1M∣Dc△Dc′∣.\Delta(\mathcal D,\mathcal D') = \sum_{c=1}^{M} |\mathcal D_c\triangle\mathcal D'_c|.

We call them sample-level adjacent, denoted by D∼D′\mathcal D\sim\mathcal D', if Δ(D,D′)=1\Delta(\mathcal D,\mathcal D')=1.

Following the stability-based exact-unlearning framework, TV stability is defined on the marginal distribution of the learning algorithm’s model output, rather than on its auxiliary internal state.

Definition 2 (ρ\rho-Sample-Level TV Stability). A randomized federated learning algorithm O\mathcal O is ρ\rho-sample-level TV stable if

sup⁡D∼D′TV⁡(Oout(D),Oout(D′))≤ρ,\begin{equation} \sup_{\mathcal D\sim\mathcal D'} \operatorname{TV} \left( \mathcal O_{\mathrm{out}}(\mathcal D), \mathcal O_{\mathrm{out}}(\mathcal D') \right) \le \rho, \tag*{(7)} \end{equation}

where ρ∈[0,1]\rho\in[0,1] is the stability parameter.

TV stability therefore controls how much the distribution of the learned model can change after modifying a single training sample. For neighboring datasets, a smaller ρ\rho implies greater overlap between the two output distributions, which can be exploited by coupling-based unlearning to reuse computations from the original training trajectory.

Our Method

We develop two pairs of federated learning and exact-unlearning algorithms. Both pairs build on a common seed-encoded representation of zeroth-order updates, which allows the realized training trajectory to be stored and replayed without retaining high-dimensional model updates. The first pair, FedZOO-S and FedZOUL-S, exploits client and sample subsampling: if a deleted sample has not entered the realized trajectory, the existing state remains valid; otherwise, exact unlearning replays the affected suffix. The second pair, FedZOO-NS and FedZOUL-NS, further randomizes local zeroth-order updates with Gaussian noise. The resulting overlap between update distributions on neighboring datasets allows some affected updates to be reused through coupling, reducing the need for replay.

Seed-Encoded Zeroth-Order Updates

All four algorithms use the same compact representation of zeroth-order updates. At initialization, the server generates a finite pool of KK distinct perturbation seeds

S={s1,…,sK},\mathbb{S}=\{s_1,\ldots,s_K\},

which is shared by the server and all clients and remains fixed throughout training. With a fixed pseudorandom number generator, each seed sks_k deterministically generates a Gaussian perturbation zk=Perturb(sk)∈Rd\boldsymbol z_k=\mathsf{Perturb}(s_k)\in\mathbb{R}^d, where zk∼N(0,Id)\boldsymbol z_k\sim\mathcal{N}(\boldsymbol 0,I_d). For model w\boldsymbol w, sample xx, and seed sks_k, define the two-point zeroth-order scalar gradient as

g(w;x,sk)=ℓ(w+μzk;x)−ℓ(w−μzk;x)2μ.g(\boldsymbol w;x,s_k) = \frac{ \ell(\boldsymbol w+\mu\boldsymbol z_k;x) - \ell(\boldsymbol w-\mu\boldsymbol z_k;x) }{2\mu}.

References

  1. [1]Qin, Z., Chen, D., Qian, B., Ding, B., Li, Y., & Deng, S. (2024). Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes. 41st International Conference on Machine Learning (ICML 2024).
  2. [2]Wang, L., Ren, J., Xu, H., Wang, J., Xie, H., Keyes, D. E., & Wang, D. (2025). Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory. Second Conference on Language Modeling (COLM 2025).
  3. [3]Tao, Y., Wang, C.-L., Pan, M., Yu, D., Cheng, X., & Wang, D. (2024). Communication Efficient and Provable Federated Unlearning. Proceedings of the VLDB Endowment, 17(5), 1119–1131. https://doi.org/10.14778/3641204.3641220
  4. [4]Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791
  5. [5]Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., & Ng, A. Y. (2011). Reading digits in natural images with unsupervised feature learning. 24th International Conference on Neural Information Processing Systems (NIPS 2011), NIPS Workshop on Deep Learning and Unsupervised Feature Learning.
  6. [6]Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. In University of Toronto (Number TR-2009).
  7. [7]Xiao, H., Rasul, K., & Vollgraf, R. (2017). Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. In arXiv (Numbers 1708.07747). https://doi.org/10.48550/arXiv.1708.07747
  8. [8]Li, Z., Ying, B., Liu, Z., Dong, C., & Yang, H. (2025). Achieving dimension-free communication in federated learning via zeroth-order optimization. 13rd International Conference on Learning Representations (ICLR 2025), 2025.
  9. [9]Liu, Y., Xu, L., Yuan, X., Wang, C., & Li, B. (2022). The right to be forgotten in federated learning: An efficient realization with rapid retraining. 41st IEEE International Conference on Computer Communications (INFOCOM 2022), 1749–1758. https://doi.org/10.1109/INFOCOM48880.2022.9796721
  10. [10]Mora, A., Valerio, L., & Bellavista, P. (2026). Federated Unlearning via Distilled Data. IEEE Transactions on Mobile Computing, 25(7), 10607–10618. https://doi.org/10.1109/TMC.2026.3666356
  11. [11]Zhang, F., Yan, X., Wu, T., Li, W., Chen, T., Cao, Y., Yan, R., Huang, L., Lim, W. Y. B., & Yang, Q. (2026). Oblivionis: A Lightweight Learning and Unlearning Framework for Federated Large Language Models. 40th AAAI Conference on Artificial Intelligence (AAAI 2026), 40, 28185–28193. https://doi.org/10.1609/aaai.v40i33.40045
  12. [12]Liu, G., Ma, X., Yang, Y., Wang, C., & Liu, J. (2021). FedEraser: Enabling efficient client-level data removal from federated learning models. 29th IEEE/ACM International Symposium on Quality of Service (IWQOS 2021). https://doi.org/10.1109/IWQOS52092.2021.9521274
  13. [13]Pan, Z., Wang, Z., Li, C., Zheng, K., Wang, B., Tang, X., & Zhao, J. (2025). Federated unlearning with gradient descent and conflict mitigation. 39th AAAI Conference on Artificial Intelligence (AAAI 2025), 19804–19812. https://doi.org/10.1609/aaai.v39i19.34181
  14. [14]Wang, P., Song, W., Qi, H., Zhou, C., Li, F., Wang, Y., Sun, P., & Zhang, Q. (2024). Server-initiated federated unlearning to eliminate impacts of low-quality data. IEEE Transactions on Services Computing, 17(3), 1196–1211. https://doi.org/10.1109/TSC.2024.3355188
  15. [15]Su, W., Wang, P., Wang, Z., Muhammad, A., Tao, T., Tong, X., & Zhang, Q. (2024). F2ul: Fairness-aware federated unlearning for data trading. IEEE Transactions on Mobile Computing, 23(12), 13539–13555. https://doi.org/10.1109/TMC.2024.3429228
  16. [16]Che, T., Zhou, Y., Zhang, Z., Lyu, L., Liu, J., Yan, D., Dou, D., & Huan, J. (2023). Fast federated machine unlearning with nonlinear functional theory. 40th International Conference on Machine Learning (ICML 2023).
  17. [17]Zhang, L., Zhu, T., Zhang, H., Xiong, P., & Zhou, W. (2023). Fedrecovery: Differentially private machine unlearning for federated learning frameworks. IEEE Transactions on Information Forensics and Security, 18, 4732–4746. https://doi.org/10.1109/TIFS.2023.3297905
  18. [18]Zhou, C., Pan, C., Li, M., & Wang, P. (2025). Federated unlearning with fast recovery. IEEE Transactions on Mobile Computing, 24(10), 9709–9725. https://doi.org/10.1109/TMC.2025.3563265
  19. [19]Fraboni, Y., Van Waerebeke, M., Scaman, K., Vidal, R., Kameni, L., & Lorenzi, M. (2024). Sifu: Sequential informed federated unlearning for efficient and provable client unlearning in federated optimization. 27th International Conference on Artificial Intelligence and Statistics (AISTATS 2024), 3457–3465.
  20. [20]Fang, W., Yu, Z., Jiang, Y., Shi, Y., Jones, C. N., & Zhou, Y. (2022). Communication-efficient stochastic zeroth-order optimization for federated learning. IEEE Transactions on Signal Processing, 70, 5058–5073. https://doi.org/10.1109/TSP.2022.3214122
  21. [21]Ling, Z., Chen, D., Yao, L., Li, Y., & Shen, Y. (2024). On the convergence of zeroth-order federated tuning for large language models. 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2024), 1827–1838. https://doi.org/10.1145/3637528.3671865
  22. [22]Li, Z., Ying, B., Liu, Z., Dong, C., & Yang, H. (2026). Converge faster, talk less: Hessian-informed federated zeroth-order optimization. 14th International Conference on Learning Representations (ICLR 2026), 2026.
  23. [23]Zhang, L., Li, B., Thekumparampil, K. K., Oh, S., & He, N. (2024). DPZero: private fine-tuning of language models without backpropagation. 41st International Conference on Machine Learning (ICML 2024), 59210–59246.
  24. [24]Liu, Z., Lou, J., Bao, W., Hu, Y., Wang, W., Qin, Z., & Ren, K. (2026). Differentially Private Zeroth-Order Methods for Scalable Large Language Model Fine-tuning. IEEE Transactions on Information Forensics and Security, 21, 4163–4175. https://doi.org/10.1109/TIFS.2026.3679194
  25. [25]Bao, E., Jiang, Y., Wei, F., Xiao, X., Li, Z., Li, Y., & Ding, B. (2025). Unlocking the Power of Differentially Private Zeroth-order Optimization for Fine-tuning $$LLMs$$. 34th USENIX Security Symposium (USENIX Security 2025), 1569–1588.
  26. [26]McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. y . (2017). Communication-efficient learning of deep networks from decentralized data. 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017), 1273–1282.
  27. [27]Fu, Y., You, H., Zhao, Y., Wang, Y., Li, C., Gopalakrishnan, K., Wang, Z., & Lin, Y. (2020). FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training. 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 33, 12127–12139.
  28. [28]Huang, Y., Qiao, X., Tang, J., Ren, P., Liu, L., Pu, C., & Chen, J. (2020). Deepadapter: A collaborative deep learning framework for the mobile web using context-aware network pruning. 39th IEEE International Conference on Computer Communications (INFOCOM 2020), 834–843. https://doi.org/10.1109/INFOCOM41043.2020.9155379
  29. [29]Zhang, L., Song, J., Gao, A., Chen, J., Bao, C., & Ma, K. (2019). Be your own teacher: Improve the performance of convolutional neural networks via self distillation. IEEE/CVF International Conference on Computer Vision (ICCV 2019), 3712–3721. https://doi.org/10.1109/ICCV.2019.00381
  30. [30]Zhang, Z., Yang, Y., Dai, Y., Wang, Q., Yu, Y., Qu, L., & Xu, Z. (2023). Fedpetuning: When federated learning meets the parameter-efficient tuning methods of pre-trained language models. Findings of the Association for Computational Linguistics: ACL 2023, 9963–9977. https://doi.org/10.18653/v1/2023.findings-acl.632
  31. [31]Voigt, P., & Von dem Bussche, A. (2024). The EU General Data Protection Regulation (GDPR): A Practical Guide (p. 339). Springer. https://doi.org/10.1007/978-3-031-62328-8
  32. [32]Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z. C., & Kolter, J. Z. (2024). TOFU: A Task of Fictitious Unlearning for LLMs. 38th Conference on Neural Information Processing Systems (NeurIPS 2024), NeurIPS 2024 Workshop on Red Teaming GenAI, 37.
  33. [33]Ullah, E., Mai, T., Rao, A., Rossi, R. A., & Arora, R. (2021). Machine unlearning via algorithmic stability. 34th Annual Conference on Learning Theory (COLT 2021), 4126–4142.
  34. [34]Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing: Theory and Applications (pp. 210–268). Cambridge University Press. https://doi.org/10.1017/CBO9780511794308.006
  35. [35]Chen, S., Tao, Y., Chen, S., Dressler, F., Ye, Q., & Wang, D. (2026). FedSubMuon: Communication-Efficient Federated LLM Fine-Tuning via Structured Subspace Muon. Conference on Empirical Methods in Natural Language Processing (EMNLP 2026).
  36. [36]Chen, S., Yuan, Y., Tao, Y., Wang, T., Cai, Z., & Yu, D. (2025). Adaptive pruning-based Newton's method for distributed learning. Theoretical Computer Science, 1026, 114987. https://doi.org/10.1016/j.tcs.2024.114987
  37. [37]Tao, Y., Chen, S., Zhang, C., Wang, D., Yu, D., Cheng, X., & Dressler, F. (2024). Private Over-the-Air Federated Learning at Band-Limited Edge. IEEE Transactions on Mobile Computing, 23(12), 12444–12460. https://doi.org/10.1109/TMC.2024.3411295
  38. [38]Wu, H., Yang, J., Chen, S., Wang, D., Yu, D., & Tao, Y. (2026). Trajectory-Aware Certified Decentralized Unlearning via SGD Stability. 43rd International Conference on Machine Learning (ICML 2026), Poster Session.
  39. [39]Wu, H., Tao, Y., Zhou, A., Chen, S., Dressler, F., & Yu, D. (2026). Certified Unlearning in Decentralized Federated Learning. 45th IEEE International Conference on Computer Communications (INFOCOM 2026), 1–10. https://doi.org/10.1109/INFOCOM59046.2026.11571416
  40. [40]Tao, Y., Simsek, M., & Dressler, F. (2026). AutoSub: Autonomous Submodel Averaging. IEEE Annual Congress on Artificial Intelligence of Things (AioT 2026).
  41. [41]Tao, Y., Yin, W., Xie, H., Yao, Y., Yu, D., Cheng, X., Wang, D., & Dressler, F. (2026). One-Shot Federated Model Editing for Device Dynamics at the Edge. IEEE Transactions on Networking. https://doi.org/10.1109/TON.2026.3745736
  42. [42]Li, J., Tao, Y., He, B., & Ding, M. (2026). Not All Change Is Damage: Update-Matched Unlearning for Large Language Models. In arXiv (Numbers 2610.21230). https://arxiv.org/abs/2610.21230
  43. [43]Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2020). FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization. 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020), 108, 2021–2031.
  44. [44]Lin, Y., Han, S., Mao, H., Wang, Y., & Dally, B. (2018). Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training. 6th International Conference on Learning Representations (ICLR 2018).
  45. [45]Lin, T., Kong, L., Stich, S. U., & Jaggi, M. (2020). Ensemble distillation for robust model fusion in federated learning. 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2351–2363.
  46. [46]Horvath, S., Laskaridis, S., Almeida, M., Leontiadis, I., Venieris, S. I., & Lane, N. D. (2021). FjORD: Fair and Accurate Federated Learning under Heterogeneous Targets with Ordered Dropout. 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 34.
  47. [47]Ding, L., Zhang, H., Dai, W., Li, C., Lu, W., Yang, Z., Zhang, X., Ma, X., Zou, J., & Xiong, H. (2025). LBI-FL: Low-Bit Integerized Federated Learning with Temporally Dynamic Bit-Width Allocation. 42nd International Conference on Machine Learning (ICML 2025), 267, 13885–13899.
  48. [48]Guastella, A., Sani, L., Iacob, A., Mora, A., Bellavista, P., & Lane, N. D. (2025). SparsyFed: Sparse Adaptive Federated Learning. 13rd International Conference on Learning Representations (ICLR 2025), 2025.
  49. [49]Fan, T., Chen, W., Kang, Y., Ma, G., Gu, H., Song, Y., Fan, L., & Yang, Q. (2025). FedCoT: Federated Chain-of-Thought Distillation for Large Language Models. 30th Conference on Empirical Methods in Natural Language Processing (EMNLP 2025), 8546–8557. https://doi.org/10.18653/v1/2025.findings-emnlp.454

Paper details

vlnc:W_u79c34ap
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents