betaFeedback

WorksMathematics

Breaking the d\sqrt{d} Communication Barrier in Federated Sampling with Adaptive Hamiltonian Monte Carlo

Abstract

This work proposes the Adaptive Federated Hamiltonian Monte Carlo (AFHMC) for Bayesian federated learning (FL) tasks. The prior work on HMC applications for FL achieves O(d/ε)O(\sqrt{d}/ε) communication cost under log-convex distributions, already demonstrating its advantage over the Langevin dynamic-based counterpart. Based on a refined analysis, this paper further reveals the benefit of HMC for the FL regime by incorporating a control mechanism for local node shifts. We prove that AFHMC at most requires O({d/ε2}1/3)O(\{d/ε^2\}^{1/3}) communication cost up to a logarithmic term, which is significantly better than the existing results. Moreover, the proposed method adapts to client heterogeneity. Under low-heterogeneity settings (i.e., the target sampling precision ε2ε^2 is higher than the heterogeneity level), the communication rate of AFHMC can be as good as O(log⁡(d/ε2))O(\log(d/ε^2)), resembling the communication costs for FL optimization tools. To the best of our knowledge, this provides the first logarithmic communication complexity result for federated sampling, even though it requires low heterogeneity. Numerically, we also justify AFHMC's advantage under the low-heterogeneity regime. Our code is available at https://github.com/laaaarrywang/Adaptive-Federated-HMC

Introduction

Federated learning (FL) provides a collaborative training paradigm in which multiple clients jointly learn a global model without sharing their raw data, thereby preserving data privacy while leveraging distributed data resources [31, 3] (Li et al. 2020a). A typical FL algorithm involves iterative rounds of local training followed by aggregating updates via node-to-center communications. When FL is applied to real-time systems, e.g., low-earth orbit (LEO) satellite constellations [4], it is critical that the FL algorithm design aligns well with the system constraints, such as limited connection bandwidth/windows and high communication latency. Therefore, a successful FL framework must be communication-efficient and tolerate low aggregation frequency. Despite its empirical success in large-scale applications, most existing FL algorithms (e.g., [22, 23, 39] (Li et al. 2020b)), are designed within a frequentist framework and focus on point estimation, which makes it difficult to quantify uncertainty in model predictions and parameters. Bayesian federated learning (BFL) addresses this limitation by inferring the full posterior, thus enabling principled uncertainty quantification and robust decision-making under data heterogeneity and limited local samples. However, scalable Bayesian FL inference remains challenging due to multiple factors: (i) high-dimensional parameter spaces, (ii) heterogeneous and potentially non-IID client data, and (iii) strict communication and privacy constraints.

Hamiltonian Monte Carlo (HMC, Duane et al. [10]; Neal [35]) is a MCMC method for sampling from complex posteriors, which exploits gradient information and an auxiliary momentum variable to construct proposals that can efficiently explore high-dimensional distributions. Compared with first-order methods such as stochastic gradient Langevin dynamics (SGLD, Welling & Teh [41]; Sato & Nakagawa [37]), HMC and its stochastic variants often achieve superior mixing and lower asymptotic variance but its second-order nature also makes theoretical analysis and distributed implementation more involved. Extending HMC to the federated setting is nontrivial, because naive combinations of stochastic HMC with FedAvg-style aggregation (McMahan et al., [31]) can easily suffer from bias accumulation, instability of global dynamics, and inefficient communication.

To address these challenges, recent work proposes the Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC, Liang et al. [26]) algorithm, which integrates stochastic HMC updates on local clients with periodic aggregation of both positions and momenta at a central server. Liang et al. [26] showed that FA-HMC requires O(d/ϵ)O(\sqrt{d}/\epsilon) communication cost (equivalently, O(d/ϵ)O(\sqrt{d}/\epsilon) updates occur in the center node) to achieve ϵ\epsilon Wasserstein sampling error. In contrast, under non-distributed settings, HMC achieves a tight O((d/ϵ2)1/4)O((d/\epsilon^{2})^{1/4}) gradient computation cost (Mangoubi & Vishnoi, [30]). It naturally leads to the question: can we further close the gap between the communication cost in FL paradigm and the computation cost in the non-FL paradigm, in terms of the dimension dependency?

This work introduces an adaptive variant, Adaptive Federated Hamiltonian Monte Carlo (AFHMC), which incorporates surrogate leapfrog steps and an additional global shift vector to better align local and global dynamics. We conduct refine analysis which explicitly characterizes the discrepancy between surrogate and exact leapfrog dynamics, and establish non-asymptotic convergence guarantees under standard smoothness and strong convexity assumptions. AFHMC can achieve improved dependence on the number of clients, communication rounds, and stochastic gradient noise level, and our experiments demonstrate competitive or superior performance compared with existing Bayesian federated baselines.

Related Works and Our Contributions

Federated optimization Early studies, such as FedAvg (McMahan et al., [31]), established convergence guarantees for smooth and strongly convex objectives, typically achieving sublinear rates in the presence of data heterogeneity. To address this challenge, subsequent work introduced variance reduction and control variate techniques, such as SCAFFOLD (Karimireddy et al., [20]) and its variants, which improve convergence by mitigating client drift and reducing bias induced by heterogeneous local updates. In particular, Mitra et al. [33] provided the first result establishing linear convergence rates in terms of communication complexity using control variates, while Mishchenko et al. [32] further advanced this line of research by proposing randomized algorithms that achieve constant communication complexity under suitable conditions.

Bayesian federated learning. Bayesian extensions of federated learning aim to characterize posterior uncertainty over model parameters or predictions while respecting client-level privacy and communication constraints. Existing approaches include variational Bayes formulations (e.g., Zhang et al., [43]; Guo et al., [16]), hierarchical Bayesian models for heterogeneous clients (e.g., Vedadi et al., [38]; Kim & Hospedales, [21]), and Bayesian variants of federated averaging where each client maintains local posterior approximations that are periodically aggregated (e.g., Al-Shedivat et al., [2]; Chen & Chao, [5]; Liu et al., [27]). These methods often rely on factorized approximations or conjugate priors to keep the communication and computation overhead manageable.Our work instead targets exact (or asymptotically exact) MCMC-based Bayesian inference in federated environments, and develops HMC-based algorithms that can operate under realistic communication budgets.

HMC and stochastic variants. HMC is an efficient sampler for complex high-dimensional Bayesian inference, with rigorous guarantees on convergence and mixing under smoothness and curvature conditions of the target distribution (Durmus et al., [12]; Mangoubi & Smith, [28, 29]). To scale HMC to large datasets, stochastic gradient HMC (SGHMC Chen et al. [6]) utilizes mini-batch estimates, typically introducing extra friction or correction terms to control the additional stochastic noise. Subsequent works have investigated the convergence behavior of SGHMC (Zou & Gu, [45]), variance-reduction techniques (Zou et al., [46]), low-precision sampling techniques (Wang et al., [40]) and etc. These developments, however, are primarily designed for centralized settings, and their direct application to federated learning is hindered by client heterogeneity and limited access to the full dataset.

Adaptive and geometry-aware HMC. A line of research has explored adaptive HMC schemes that adjust hyperparameters such as step sizes, mass matrices, and integrator structures using information gathered during sampling. Examples include Hoffman & Gelman [19]; Hoffman et al. [18]; Hirt et al. [17]; Fernández-Pendás et al. [14]; Girolami & Calderhead [15], all aiming to enhance stability and sampling efficiency in complex geometries. More recently, adaptive tuning strategies have been proposed to automatically select splitting integrators and hyperparameter ranges for (generalized) HMC, reducing reliance on heuristic tuning in practice [1, 34]. Our AFHMC algorithm is conceptually related to these works in that it uses a surrogate Hamiltonian flow informed by global information, but it is tailored to the federated setting, where the adaptation must be communication-efficient and robust to client heterogeneity.

Beyond existing results, the main contributions of this paper are summarized as follows:

  1. We propose AFHMC, an adaptive HMC algorithm for FL tasks based on surrogate leapfrog updates. The new algorithm introduces a global drift control variate to address the heterogeneity among local gradient update trajectories.

  1. We introduce a principled framework to analyze divergence errors induced by approximate Hamiltonian dynamics.

  1. Under strong convexity, we derive non-asymptotic error bounds that quantify the interplay between the surrogate leapfrog integrator and data heterogeneity. In particular, we prove that AFHMC achieves O({d/ϵ2}1/3)O(\{d/\epsilon^2\}^{1/3}) communication cost under extreme heterogeneity, and O(log⁡(d/ϵ2))O(\log(d/\epsilon^2)) communication cost under low heterogeneity. To the best of our knowledge, both rates of communication complexity have never been achieved by prior federated sampling algorithms. Not only do we obtain a better communication rate, but we also don’t rely on stringent Hessian smoothness conditions that are commonly required to achieve an advanced rate (e.g., Liang et al. [26]; Mangoubi & Vishnoi [30]; Plassier et al. [36]). Please refer to Table 1 for a comprehensive comparison against prior theoretical works. Note that if we indeed enforce an extra Hessian smoothness condition, we could technically obtain even better results.

AlgorithmCondition beyond Assumptions 4.1-4.2Rate of communicationRate of gradients (up to logarithmic terms)
Centralized HMC Mangoubi & Vishnoi [30]Hessian Lipschitz under infinity norm (Assum. 1 in their paper)none(d/ϵ2)1/4(d/\epsilon^2)^{1/4}
VR-FALD Plassier et al. [36]Hessian Lipschitz under L2 norm (Assum. HX1 in their paper)(d/ϵ2)1/2(d/\epsilon^2)^{1/2}(d/ϵ2)1/2(d/\epsilon^2)^{1/2}
FA-HMC Liang et al. [26]Hessian Lipschitz under L2 norm (Assum. 3 in their paper)(d/ϵ2)1/2(d/\epsilon^2)^{1/2}(d/ϵ2)1/2(d/\epsilon^2)^{1/2}
AFHMC (this paper)nonelog⁡(d/ϵ2)∼(d/ϵ2)1/3\log(d/\epsilon^2) \sim(d/\epsilon^2)^{1/3}(d/ϵ2)1/2∼(d/ϵ2)2/3(d/\epsilon^2)^{1/2} \sim(d/\epsilon^2)^{2/3}

Table 1. Comparison of Bayesian theoretical results: requires assumptions, rate of communication and rate of gradient computation to achieve ϵ\epsilon Wasserstein convergence

  1. We empirically demonstrate that the proposed method achieves improved computational efficiency and stability in distributed and approximate-gradient settings.

Motivation and Algorithm

The goal of a standard sampling problem is to generate approximate samples from the target posterior π(θ)∝exp⁡(−f(θ))\pi(\theta) \propto\exp(-f(\theta)) where ff is the potential function. In the standard leapfrog HMC, as illustrated in Figure 1(left), the parameter θ\theta is evolved using a randomly sampled Gaussian momentum for KK leapfrog steps (defined later in (2)), after which the momentum is resampled before starting a new sequence of leapfrog updates. In Mangoubi & Vishnoi [30], it is shown that performing O(log⁡d)O(\log d) such leapfrog trajectories, each consisting of O({d/ϵ2}1/4)O(\{d/\epsilon^2\}^{1/4}) steps, suffices to ensure convergence of the task distribution. Therefore, the total gradient computational cost of the overall procedure is dominated by the number of leapfrog steps, leading to a total complexity of order O(d1/4)O(d^{1/4}) up to logarithmic factors.

Illustration of the centralized HMC algorithm and the federated HMC algorithm

Figure 1. An illustration of the centralized HMC algorithm (left panel) and the federated HMC algorithm (right panel).

Let [N]={1,…,N}[N] = \{1,\ldots,N\} index the clients. In federated settings, each client cc has a local potential f(c)(θ)f^{(c)}(\theta) with weight wc≥0w_c \ge0 and thus global potential f(θ)=∑c=1Nwcf(c)(θ)f(\theta)=\sum_{c=1}^{N}w_cf^{(c)}(\theta), where ∑c=1Nwc=1\sum_{c=1}^{N}w_c=1. To achieve logarithmic communication complexity in federated sampling, the above theoretical insight motivates us to integrate communication steps into the alternating leapfrog procedure, expecting that the order of FL communication cost matches O(log⁡d)O(\log d) order of leapfrog trajectories in centralized HMC. To be concrete, as illustrated in Figure 1 (right), we aim that each local node performs KK leapfrog steps followed by a communication round (e.g., averaging the local parameters across nodes), before proceeding to the next leapfrog phase. This corresponds to the FA-HMC algorithm Liang et al., [26] with T=1T=1. Unfortunately, Liang et al. [26] establishes a lower bound showing that, under high federated heterogeneity, the communication cost of such an HMC scheme is at least on the order of O({d/ϵ2}1/2)O(\{d/\epsilon^2\}^{1/2}). Thus, additional non-trivial FL training designs are necessary to reach a better rate.

In any FL optimization tasks, a key challenge to achieve efficient communication is to control the local shift caused by heterogeneity across local nodes. Local gradient updates move the local parameter toward the location optimum, resulting in a global bias after aggregating local solutions. A breakthrough was introduced by Karimireddy et al. [20], which introduced a control shift technique that reduces the communication cost of FedAvg (McMahan et al., [31]) from O(d/ϵ2)O(d/\epsilon^2) to O(log⁡(d/ϵ2))O(\log(d/\epsilon^2)). In the FL sampling regime, FA-LD (Deng et al., [9]), a Langevin-counterpart of FedAvg also incurs a communication cost of O(d/ϵ2)O(d/\epsilon^2). Plassier et al. [36] further improved the sampling algorithm by adapting the control shift technique of (Karimireddy et al., [20]).

Inspired by existing works on local shift control, we propose an adaptive HMC federated update rule by introducing a drift variate. Given an energy function FF, the HMC dynamics follow the Hamiltonian equations:

dθ(t)=p(t)dt,dp(t)=−∇θF(θ(t))dt.(1)\mathrm{d}\theta(t) = p(t)\mathrm{d}t,\qquad\mathrm{d}p(t) = -\nabla_\theta F(\theta(t))\mathrm{d}t. \tag*{(1)}

The above continuous-time dynamic cannot be solved analytically. Commonly, one uses the leapfrog integrator to discretize the dynamics over a step size η\eta by

θk+1=θk+ηpk−η22∇F(θk),pk+1=pk−η2∇F(θk)−η2∇F(θk+1).(2)\theta_{k+1} = \theta_k + \eta p_k - \frac{\eta^2}{2}\nabla F(\theta_k),\qquad p_{k+1} = p_k - \frac{\eta}{2}\nabla F(\theta_k) - \frac{\eta}{2}\nabla F(\theta_{k+1}). \tag*{(2)}

Under the FL setting, update algorithm (2) is running on local nodes, and FF represents a local energy function which can differ substantially from the global energy function ff due to node heterogeneity. To account for this potential gap between local FF and global ff, we inject a bias correction term to the gradient and replace all ∇F(⋅)\nabla F(\cdot) terms in (2) with ∇F(⋅)−∇F(θ0)+∇f(θ0)\nabla F(\cdot) - \nabla F(\theta_0) + \nabla f(\theta_0) (see, Algorithm 1). This bias correction term ∇f(θ0)−∇F(θ0)\nabla f(\theta_0) - \nabla F(\theta_0) addresses the discrepancy between local and global losses, and the drift variate ∇f(θ0)\nabla f(\theta_0) needs to be broadcast to all local nodes. This novel local update rule is named the surrogate leapfrog update.

The AFHMC algorithm consists of the following iterative steps: 1. After synchronizing parameter θt\theta_t across all nodes, each node cc shares the local ∇f(c)(θt)\nabla f^{(c)}(\theta_t) to obtain a global drift variate ∇f(θt)\nabla f(\theta_t). 2. All local nodes run KK surrogate leapfrog steps, with the shared global drift variate, initialized from the same position and momentum. 3. All nodes share and synchronize the local parameter θt+1\theta_{t+1}, after KK leapfrog steps. The algorithm can be found in Algorithm 2. In contrast to FA-HMC (Liang et al., [26]), which runs multiple local HMC steps where each HMC step consists of KK leapfrog updates, the proposed AFHMC only runs one single local HMC step (i.e., KK updates of surrogate leapfrog, see Figure 1). As discussed previously, this design is to ensure minimal communication cost, because every local HMC step will need a fresh new drift control variate, which requires communication.

The AFHMC algorithm can be easily implemented with stochastic gradients for acceleration purposes, similarly to Liang et al. [26], but our analysis in this paper focuses on the use of the exact gradient.

Algorithm 1 Surrogate leapfrog approximation h~LF(⋅)\widetilde{h}_{\mathrm{LF}}(\cdot)

Input: energy function F(⋅)F(\cdot), initial position θ0\theta_0, drift vector ∇f(θ0)\nabla f(\theta_0), momentum p0p_0, stepsize η\eta, leapfrog step KK

for k=0k=0 to K−1K-1 do

θk+1=θk+ηpk−η22(∇F(θk)−∇F(θ0)+∇f(θ0))\theta_{k+1}=\theta_k+\eta p_k-\frac{\eta^2}{2}\left(\nabla F(\theta_k)-\nabla F(\theta_0)+\nabla f(\theta_0)\right)

pk+1=pk−η2∇F(θk)−η2∇F(θk+1)+η(∇F(θ0)−∇f(θ0))p_{k+1}=p_k-\frac{\eta}{2}\nabla F(\theta_k)-\frac{\eta}{2}\nabla F(\theta_{k+1})+\eta\left(\nabla F(\theta_0)-\nabla f(\theta_0)\right)

end for

Output: h~LF(F,θ0,∇f(θ0),p0,η,K)=θK\widetilde{h}_{\mathrm{LF}}(F,\theta_0,\nabla f(\theta_0),p_0,\eta,K)=\theta_K

Algorithm 2 Adaptive federated hamiltonian monte carlo (AFHMC) algorithm

Input: initial position θ0\theta_0, stepsize η\eta; leapfrog update step KK

Each local node receives the initial parameter θ0\theta_0 from the center

for t=0t=0 to tmax⁡t_{\max} do

1st broadcast: The cc-th local node sends ∇f(c)(θt)\nabla f^{(c)}(\theta_t) to the center, for all c=1,…,Nc=1,\ldots,N;

1st communicate: The center node samples Gaussian momentum ptp^t, and sends ptp_t and drift vector ∇f(θt):=∑c=1Nwc∇f(c)(θt)\nabla f(\theta_t):=\sum_{c=1}^{N}w_c\nabla f^{(c)}(\theta_t) to all local nodes;

for c=1,2,…,Nc=1,2,\ldots,N do

Initialize θt,0(c)=θt\theta_{t,0}^{(c)}=\theta_t, pt,0(c)=ptp_{t,0}^{(c)}=p_t;

update θt,K(c)=h~LF(f(c),θt,0(c),∇f(θt),pt,0(c),η,K)\theta_{t,K}^{(c)}=\widetilde{h}_{\mathrm{LF}}(f^{(c)},\theta_{t,0}^{(c)},\nabla f(\theta_t),p_{t,0}^{(c)},\eta,K);

end for

2nd broadcast: The cc-th local node sends θt,K(c)\theta_{t,K}^{(c)} to the center for all c=1,…,Nc=1,\ldots,N;

References

  1. [1]Akhmatskaya, E., Nagar, L., Carrillo, J. A., Gavira Balmacz, L., Inouzhe, H., Parga Pazos, M., and Rodríguez Álvarez, M. X. Adaptive tuning of hamiltonian monte carlo methods. Applied Mathematical Modelling, 157:116892, 2026. ISSN 0307-904X. doi: https://doi.org/10.1016/j.apm.2026.116892. URL https://www.sciencedirect.com/science/article/pii/S0307904X26001538.
  2. [2]Al-Shedivat, M., Gillenwater, J., Xing, E., and Rostamizadeh, A. Federated learning via posterior averaging: A new perspective and practical algorithms. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=GFsU8a0sGB.arxiv.org/abs/2010.05273
  3. [3]Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., and McMahan, B. Towards Federated Learning at Scale: System design. Proceedings of Machine Learning and Systems, 1:374–388, 2019.arxiv.org/abs/1902.01046
  4. [4]Chen, H., Xiao, M., and Pang, Z. Satellite-based computing networks with federated learning. IEEE Wireless Communications, 29(1):78–84, 2022.arxiv.org/abs/2111.10586
  5. [5]Chen, H.-Y. and Chao, W.-L. Fed{be}: Making bayesian model ensemble applicable to federated learning. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=dgtpE6gKjHn.arxiv.org/abs/2009.01974
  6. [6]Chen, T., Fox, E., and Guestrin, C. Stochastic Gradient Hamiltonian Monte Carlo. In International Conference on Machine Learning (ICML), pp. 1683–1691, 2014.arxiv.org/abs/1402.4102
  7. [7]Dalalyan, A. S. Theoretical Guarantees for Approximate Sampling from Smooth and Log-concave Densities. Journal of the Royal Statistical Society: Series B, 79(3):651–676, 2017.arxiv.org/abs/1412.7392
  8. [8]Dalalyan, A. S. and Karagulyan, A. User-friendly Guarantees for the Langevin Monte Carlo with Inaccurate Gradient. Stochastic Processes and their Applications, 129(12):5278–5311, 2019.DOI
  9. [9]Deng, W., Zhang, Q., Ma, Y.-A., Song, Z., and Lin, G. On convergence of federated averaging dynamics. In Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, UAI ’24, 2024.
  10. [10]Duane, S., Kennedy, A., Pendleton, B., and Roweth. Hybrid Monte Carlo. Physics Letters B, 195(2):216–222, 1987.DOI
  11. [11]Durmus, A. and Moulines, E. High-dimensional Bayesian Inference via The Unadjusted Langevin Algorithm. Bernoulli, 25(4A):2854–2882, 2019.arxiv.org/abs/1605.01559
  12. [12]Durmus, A., Éric Moulines, and Saksman, E. Irreducibility and Geometric Ergodicity of Hamiltonian Monte Carlo. Annals of Statistics, 48(6):3545–3564, 2020.DOI
  13. [13]Erdogdu, M. A. and Hosseinzadeh, R. On the Convergence of Langevin Monte Carlo: the Interplay between Tail Growth and Smoothness. In Proc. of Conference on Learning Theory (COLT), pp. 1776–1822, 2021.arxiv.org/abs/2005.13097
  14. [14]Fernández-Pendás, M., Akhmatskaya, E., and Sanz-Serna, J. M. Adaptive multi-stage integrators for optimal energy conservation in molecular simulations. Journal of Computational Physics, 327:434–449, 2016.
  15. [15]Girolami, M. and Calderhead, B. Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2):123–214, 2011.DOI
  16. [16]Guo, H., Greengard, P., Wang, H., Gelman, A., Kim, Y., and Xing, E. Federated learning as variational inference: A scalable expectation propagation approach. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=dZrQR7OR11.arxiv.org/abs/2302.04228
  17. [17]Hirt, M., Titsias, M., and Dellaportas, P. Entropy-based adaptive hamiltonian monte carlo. Advances in neural information processing systems, 34:28482–28495, 2021.arxiv.org/abs/2110.14625
  18. [18]Hoffman, M., Radul, A., and Sountsov, P. An adaptive-mcmc scheme for setting trajectory lengths in hamiltonian monte carlo. In International Conference on Artificial Intelligence and Statistics, pp. 3907–3915. PMLR, 2021.
  19. [19]Hoffman, M. D. and Gelman, A. The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, pp. 1593–1623, 2014.
  20. [20]Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A. T. SCAFFOLD: Stochastic Controlled Averaging for Federated Learning. In International Conference on Machine Learning (ICML), pp. 5132–5143, 2020.
  21. [21]Kim, M. and Hospedales, T. Fedhb: Hierarchical bayesian federated learning. Journal of Machine Learning Research, 26(272):1–50, 2025.
  22. [22]Konečný, J., McMahan, B., and Ramage, D. Federated optimization: Distributed optimization Beyond the Datacenter. arXiv preprint arXiv:1511.03575, 2015.
  23. [23]Konečný, J., McMahan, H. B., Yu, F. X., Richtarik, P., Suresh, A. T., and Bacon, D. Federated Learning: Strategies for Improving Communication Efficiency. In NIPS Workshop on Private Multi-Party Machine Learning, 2016. URL https://arxiv.org/abs/1610.05492.
  24. [26]Liang, J., Zhang, Q., Deng, W., Song, Q., and Lin, G. Bayesian federated learning with hamiltonian monte carlo: Algorithm and theory. Journal of Computational and Graphical Statistics, 34(2): 509–518, 2025.
  25. [27]Liu, L., Jiang, X., Zheng, F., Chen, H., Qi, G.-J., Huang, H., and Shao, L. A bayesian federated learning framework with online laplace approximation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1):1–16, 2024. doi: 10.1109/TPAMI.2023.3322743.
  26. [28]Mangoubi, O. and Smith, A. Rapid mixing of hamiltonian monte carlo on strongly log-concave distributions. arXiv preprint arXiv:1708.07114, 2017.
  27. [29]Mangoubi, O. and Smith, A. Mixing of hamiltonian monte carlo on strongly log-concave distributions: Continuous dynamics. The Annals of Applied Probability, 31(5):2019–2045, 2021.
  28. [30]Mangoubi, O. and Vishnoi, N. Dimensionally tight bounds for second-order hamiltonian monte carlo. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/file/e07bceab69529b0f0b43625953fbf2a0-Paper.pdf.
  29. [31]McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. Communication-efficient Learning of Deep Networks from Decentralized Data. In Proc. of the International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 1273–1282, 2017.
  30. [32]Mishchenko, K., Malinovsky, G., Stich, S., and Richtarik, P. ProxSkip: Yes! Local gradient steps provably lead to communication acceleration! Finally! In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 15750–15769. PMLR, 17–23 Jul 2022. URL https://proceedings.mlr.press/v162/mishchenko22b.html.
  31. [33]Mitra, A., Jaafar, R., Pappas, G. J., and Hassani, H. Linear convergence in federated learning: Tackling client heterogeneity and sparse gradients. Advances in Neural Information Processing Systems, 34:14606–14619, 2021.
  32. [34]Nagar, L., Fernández-Pendás, M., Sanz-Serna, J. M., and Akhmatskaya, E. Adaptive multi-stage integration schemes for hamiltonian monte carlo. Journal of Computational Physics, 502:112800, 2024.DOI
  33. [35]Neal, R. M. MCMC using Hamiltonian dynamics. In Handbook of Markov Chain Monte Carlo, volume 54, pp. 113–162. Chapman and Hall/CRC, 2012.DOI
  34. [36]Plassier, V., Moulines, E., and Durmus, A. Federated averaging langevin dynamics: Toward a unified theory and new algorithms. In International Conference on Artificial Intelligence and Statistics, pp. 5299–5356. PMLR, 2023.arxiv.org/abs/2211.00100
  35. [37]Sato, I. and Nakagawa, H. Approximation analysis of stochastic gradient langevin dynamics by using fokker-planck equation and ito process. In International Conference on Machine Learning, pp. 982–990. PMLR, 2014.
  36. [38]Vedadi, E., Dillon, J. V., Mansfield, P. A., Singhal, K., Afkanpour, A., and Morningstar, W. R. Federated variational inference: Towards improved personalization and generalization. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=60mRkUHgS5.arxiv.org/abs/2305.13672
  37. [39]Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., and Khazaeni, Y. Federated Learning with Matched Averaging. In International Conference on Learning Representations (ICLR), 2020.arxiv.org/abs/2002.06440
  38. [40]Wang, Z., Chen, Y., Song, Q., and Zhang, R. Enhancing low-precision sampling via stochastic gradient hamiltonian monte carlo. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=uSLNzzu1DJ.arxiv.org/abs/2310.16320
  39. [41]Welling, M. and Teh, Y. W. Bayesian Learning via Stochastic Gradient Langevin Dynamics. In International Conference on Machine Learning (ICML), 2011.
  40. [42]Xiao, H., Rasul, K., and Vollgraf, R. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.arxiv.org/abs/1708.07747
  41. [43]Zhang, X., Li, Y., Li, W., Guo, K., and Shao, Y. Personalized federated learning via variational bayesian inference. In International Conference on Machine Learning, pp. 26293–26310. PMLR, 2022.arxiv.org/abs/2206.07977
  42. [44]Zongchen Chen, S. S. V. Optimal Convergence Rate of Hamiltonian Monte Carlo for Strongly Logconcave Distributions. In Approximation, Randomization, and Combinatorial Optimization, 2019.
  43. [45]Zou, D. and Gu, Q. On the Convergence of Hamiltonian Monte Carlo with Stochastic Gradients. In International Conference on Machine Learning (ICML), pp. 13012–13022, 2021.
  44. [46]Zou, D., Xu, P., and Gu, Q. Stochastic Gradient Hamiltonian Monte Carlo Methods with Recursive Variance Reduction. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019.

Paper details

vlnc:W_a7shfvkr
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 5

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents