betaFeedback

WorksPhysics and Astronomy

Efficient Monte Carlo sampling of metastable systems using non-local collective variable updates

Abstract

Monte Carlo simulations are widely used to simulate complex molecular systems, but standard approaches suffer from metastability. Lately, the use of non-local proposal updates in a collective-variable (CV) space has been proposed in several works. Here, we generalize these approaches and explicitly spell out an algorithm for non-linear CVs and underdamped Langevin dynamics. We prove reversibility of the resulting scheme and demonstrate its performance on several numerical examples, observing a substantial performance increase compared to methods based on overdamped Langevin dynamics as considered previously. Advances in generative machine-learning-based proposal samplers now enable efficient sampling in CV spaces of intermediate dimensionality (tens to hundreds of variables), and our results extend their applicability toward more realistic molecular systems.

Introduction

Monte-Carlo simulations promise to give access to the equilibrium properties of physical systems even when the numerical simulation of their relaxation dynamics is prohibitively expensive. In practice, however, standard Markov Chain Monte Carlo methods (MCMC) rely on physically-inspired local moves and struggle to equilibrate across metastable states, as does molecular dynamics.

Enhanced sampling algorithms go beyond the naive Markov Chain Monte Carlo approach to tackle metastability, see Ref. 1 for a review in the context of molecular dynamics. A first class of methods originated from the observation that the metastability is removed at sufficiently high temperatures, which led to the development of algorithms like parallel tempering, annealed importance sampling, and sequential Monte Carlo[2, 3, 4, 5, 6]. While known to work on a wide range of problems, they spend a substantial amount of computation time at temperatures not of direct interest and generally require a careful choice of parameters (e.g. for parallel tempering the temperature ladder and the frequency of exchange attempts).

Another important category of algorithms makes use of a collective variable (CV), a lower-dimensional representation of the system that is supposed to capture the metastability of the system. A ‘good’ CV that resolves the metastability can be used for biasing simulations to bridge modes, using enhanced sampling algorithms such as umbrella sampling[7] and related histogram methods[8, 9], thermodynamic integration[10] or free energy adaptive biasing methods[11, 12, 13, 14]. However, identifying a suitable CV in low dimension is by no means trivial and thus poses the main limitation to this class of methods.

In recent years, another approach has emerged, which is exploiting generative machine learning models. Among the various generative models, normalizing flows[15] are particularly well-suited for the purposes of MCMC sampling since their tractable probability density allows to employ them as a non-local proposal density of physical configurations, where any imperfections in training are corrected for via a Metropolis-Hastings accept/reject scheme or self-normalized importance sampling[16, 17]. Thanks to the impressive flexibility of neural networks, proofs of concept have been made in different domains ranging from molecular systems to field theories in quantum chromodynamics[16, 18, 19, 20, 21]. However, the limiting factor to scale these approaches appears to be the ability of the generative model to be precise enough either when the dimension of the problem becomes of the order of a few thousands or when the distribution has singularities[17, 22, 23, 24]. Other recent approaches make use for example of transformers[25] and diffusion models[26, 27], but these do not give easy access to the underlying probability density and are therefore less suitable for MCMC sampling than normalizing flows.

A promising approach is therefore to combine the strength of CV-enhanced sampling and generative models (normalizing flows) with a CV of ‘intermediate’ dimension (from a dozen to a few hundred degrees of freedom). Within this dimensionality, identifying a CV such that it resolves the metastability will be much easier than finding a one- to three-dimensional representation required by most classical CV-based approaches. Conversely, a normalizing flow in an ‘intermediate’ dimension can be trained to much higher accuracy than if one attempted to learn the distribution of the entire system. Since the overall goal is still sampling full configurations of the system, the question arises of how to employ such a proposal sampler in CV space to obtain an unbiased sampler for the full distribution. This has been addressed in multiple works[28, 29, 30, 31, 32], in particular under the names of “Hybrid Nonequilibrium Molecular Dynamics” (HNMD)[30, 31] and “Nonequilibrium candidate Monte Carlo” (NCMC)[29] in the context of molecular dynamics, but also in a Bayesian setting[33]. The central idea is always to drive the system from one value of the CV to another, a procedure referred to as steering in the following, and then accept or reject the resulting proposal in the full-space. A similar procedure has also been used in the simulation of glassy system with shear deformations[34]. In a previous article[35], we described such an algorithm with driven dynamics in continuous time and identified various time-discretizations for which we proved reversibility with respect to the equilibrium measure. Notably, we also related this to the Jarzynski and the Jarzynski–Crooks equalities[36, 37], clarifying the algorithm design. In a parallel work[38], one version of these algorithms was applied to a toy molecular system, using a normalizing flow as the CV sampler and including an adaptive protocol to train it alongside the MCMC procedure.

However, both these works considered only overdamped Langevin dynamics for the steering and required the CV to be a subset of the Euclidean coordinates giving the conformation of the system. In the present article, we generalize these works by presenting an algorithm for a non-linear collective variable and using general underdamped Langevin dynamics. While in spirit this algorithm already fits into the paradigm of some of the previous works like NCMC and HNMD, we present here an algorithm that is readily implemented and treat all technical details arising from the non-linear CV and general Langevin dynamics. We explicitly prove that the associated Markov chains are reversible with respect to the target measure and present its application to several example systems of increasing complexity. In all of them, we see that the deterministic (Hamiltonian) steering dynamics exhibit the best performance, with an improvement of up to two orders of magnitude compared to overdamped dynamics.

The main part of the article divides into two sections: We explain all the theoretical aspects of the problem and the algorithm in Section II and go on to demonstrate its properties through several numerical examples in Section III. Within Section II, the sampling problem is introduced in Section II A, followed by a detailed explanation of the algorithm in Section II B. We state our main theoretical result on the reversibility of the algorithm in Section II C and conclude in Section II D with a normalized version of the algorithm and comments on the parameterization. In Section III, we analyze the performance of the algorithm by means of numerical simulations and present results on four different model systems of increasing complexity in Sections III A to III D. We conclude the article and give an outlook on future directions in Section IV.

Sampling with a non-linear collective variable

Target distribution and collective variable

We begin this section by outlining the sampling problem at hand. Our goal is to sample from the Boltzmann–Gibbs probability measure

ν(dq)=Zq−1e−βV(q)dq(1)\nu(\mathrm{d}q) = \mathcal{Z}_q^{-1}e^{-\beta V(q)}\mathrm{d}q \tag*{(1)}

over positions q∈Rdq \in\mathbb{R}^{d} of dimension dd, with the energy function V:Rd→RV:\mathbb{R}^{d}\to\mathbb{R}, the inverse temperature β=(kBT)−1>0\beta=(k_{\mathrm{B}}T)^{-1}>0 and the normalizing constant Zq\mathcal{Z}_q. Let us assume that a collective variable (CV) ξ:Rd→Rℓ\xi:\mathbb{R}^{d}\to\mathbb{R}^{\ell} with ℓ<d\ell<d has been identified and we have a ‘good’ sampler ρ\rho (close to the target distribution in CV space) to propose moves in CV space. As mentioned in the introduction, the question of how to use these two ingredients to construct an unbiased MCMC sampler in the full space Rd\mathbb{R}^{d} has previously been addressed in literature[28, 29, 30, 31, 32], but it is not obvious how to employ these schemes if the map ξ\xi is non-linear. Notably, the geometry of the level sets Σ(z)\Sigma(z) defined as

Σ(z)={q∈Rd∣ξ(q)=z}(2)\Sigma(z)=\left\{q\in\mathbb{R}^{d}\mid\xi(q)=z\right\} \tag*{(2)}

must be taken into consideration. We present an algorithm which explicitly treats the case of a non-linear CV in the following section. Much of the notation throughout this paper is based on Refs. 39 and 40 and we refer the reader to those for an in-depth introduction on the mathematical concepts used here.

Even though we are only interested in sampling ν(dq)\nu(\mathrm{d}q), it will turn out to be helpful to introduce also an auxiliary momentum variable pp and define the Boltzmann–Gibbs measure in phase space (q,p)(q,p) by

μ(dq dp)=Z−1e−βH(q,p)dq dp(3)\mu(\mathrm{d}q\,\mathrm{d}p)=\mathcal{Z}^{-1}e^{-\beta H(q,p)}\mathrm{d}q\,\mathrm{d}p \tag*{(3)}

where Z\mathcal{Z} is the normalizing constant, H(q,p)=12pTM−1p+V(q)H(q,p)=\frac{1}{2}p^{\mathsf{T}}M^{-1}p+V(q) is the Hamiltonian, and M∈Rd×dM\in\mathbb{R}^{d\times d} is a symmetric positive definite mass matrix.

The algorithm

To explain the algorithm, we first present its overall structure in Section II B1, followed by the necessary ingredients and considerations in Sections II B2 to II B4.

Overall structure

The algorithm builds an MCMC chain (Qn)n≥0(Q_n)_{n\geq0} in position space by iterating over n∈Nn\in\mathbb{N} the following steps. Starting from a state QnQ_n with collective variable value Zn:=ξ(Qn)Z_n:=\xi(Q_n), generate Qn+1Q_{n+1} as follows:

Step 1: Propose a move in the collective variable space:

Z~n+1∼ρ(Zn,z′) dz′,\widetilde{Z}_{n+1} \sim\rho(Z_n,z')\,\mathrm{d}z',

where ρ(z,z′)∈Rℓ×Rℓ→R+\rho(z,z') \in\mathbb{R}^{\ell} \times\mathbb{R}^{\ell} \to\mathbb{R}_{+} denotes a Markov kernel with probability density ρ(z,z′)\rho(z,z') to jump to z′z' starting at zz.

Step 2: Generate a new Q~n+1∈Rd\widetilde{Q}_{n+1} \in\mathbb{R}^{d} that satisfies ξ(Q~n+1)=Z~n+1\xi(\widetilde{Q}_{n+1}) = \widetilde{Z}_{n+1} by driving the system in CV space from ZnZ_n to Z~n+1\widetilde{Z}_{n+1}. To this end,

  1. specify a schedule for the positions and velocities in CV space to fix the transition path (Section II B 2),

  2. initialize the position q0=Qnq^0 = Q_n and sample an initial auxiliary momentum p0p^0 (Section II B 3),

  3. use this to follow (Langevin) dynamics constrained in CV space by the schedule (Section II B 4),

all of these steps to be specified below. These dynamics yield a final state Q~n+1\widetilde{Q}_{n+1} and a quantity Wn+1\mathcal{W}_{n+1} that is identified as a work. The final momentum is discarded.

Step 3: Draw a random variable Un+1U_{n+1} with uniform law over [0,1][0,1] and

  • If Un+1≤exp⁡(−βWn+1)ρ(Z~n+1,Zn)ρ(Zn,Z~n+1)U_{n+1} \leq\exp(-\beta\mathcal{W}_{n+1}) \frac{\rho(\widetilde{Z}_{n+1},Z_n)}{\rho(Z_n,\widetilde{Z}_{n+1})}, accept the proposal: Qn+1=Q~n+1Q_{n+1}=\widetilde{Q}_{n+1}.

  • Otherwise, reject the proposal: Qn+1=QnQ_{n+1}=Q_n.

The three steps of the algorithm are schematically illustrated in Figure 1.

Figure 1. Schematic representation of the algorithm for the example of a polymer, here with the end-to-end distance as the CV.

In principle, any probability density ρ(z,z′)\rho(z,z') can be chosen to generate the proposal moves in CV space in Step 1, since the accept/reject criterion in Step 3 always ensures unbiasedness, as will be proven below. This choice will, however, affect the acceptance probability and thus the performance of the algorithm. In our numerical examples, we always use an independent sampler in the CV space, i.e. ρ(z,z′)\rho(z,z') is a function of z′z' only.

A natural choice is (an approximation of) the marginal density ρ(z′)\rho(z') of the target measure in CV space – that is the image measure of the target distribution (II.1) under the CV map ξ\xi. For a CV of intermediate dimension, such a proposal sampler could for example be learned by a normalizing flow.

We explain the various ingredients required for Step 2 in the next three subsections. The idea to sample a system by following some dynamics followed by an accept/reject step that is governed by a work is closely related to the Jarzynski and the Jarzynski–Crooks equalities[36, 37].

Schedule

To specify the constrained dynamics under which the system evolves from the value of the CV at the previous iteration ZnZ_n to the newly proposed value Z~n+1\widetilde{Z}_{n+1}, choose first the number of intermediate steps KTK_T associated with this jump. With a fixed time discretization Δt\Delta t, this sets the physical transition time T=KTΔtT=K_T\Delta t. In practice, we will choose KTK_T as a function of the Euclidean distance ∣Z~n+1−Zn∣\lvert\widetilde{Z}_{n+1}-Z_n\rvert of the proposed jump.

Introduce a schedule (z(Zn,Z~n+1)(tk))0≤k≤KT(z^{(Z_n,\widetilde{Z}_{n+1})}(t_k))_{0\leq k\leq K_T} with tk=kΔtt_k=k\Delta t whose endpoints are

z(Zn,Z~n+1)(t0)=Zn(4)z^{(Z_n,\widetilde{Z}_{n+1})}(t_0)=Z_n \tag*{(4)}
z(Zn,Z~n+1)(tKT)=Z~n+1(5)z^{(Z_n,\widetilde{Z}_{n+1})}(t_{K_T})=\widetilde{Z}_{n+1} \tag*{(5)}

as well as a velocity schedule (vz(Zn,Z~n+1)(tk))0≤k≤KT(v_z^{(Z_n,\widetilde{Z}_{n+1})}(t_k))_{0\leq k\leq K_T} with zero initial and final velocity,

vz(Zn,Z~n+1)(t0)=vz(Zn,Z~n+1)(tKT)=0.(6)v_z^{(Z_n,\widetilde{Z}_{n+1})}(t_0)=v_z^{(Z_n,\widetilde{Z}_{n+1})}(t_{K_T})=0. \tag*{(6)}

We require that the schedules for position and velocity satisfy the following reversibility property with the switching of the endpoints: for any (Z,Z~)(Z,\widetilde{Z}), we need that

z(Z,Z~)(tk)=z(Z~,Z)(tKT−k)(7)z^{(Z,\widetilde{Z})}(t_k)=z^{(\widetilde{Z},Z)}(t_{K_T-k}) \tag*{(7)}
vz(Z,Z~)(tk)=−vz(Z~,Z)(tKT−k),(8)v_z^{(Z,\widetilde{Z})}(t_k)=-v_z^{(\widetilde{Z},Z)}(t_{K_T-k}), \tag*{(8)}

for all k∈{0,…,KT}k\in\{0,\ldots,K_T\}. In practice, it is advisable for numerical reasons to choose the velocity schedule in accordance with the position schedule (see Remark B.4 in the appendix). Note, however, that the requirement (II.8) precludes the simplest choice for the velocity of vz(tk)=(z(tk+1)−z(tk))/Δtv_z(t_k)=(z(t_{k+1})-z(t_k))/\Delta t. A simple choice that admits a continuous-in-time limit when Δt\Delta t goes to zero is instead a schedule that connects the endpoints on a straight line with a continuously differentiable schedule function f:[0,1]→[0,1]f:[0,1]\to[0,1]:

z(Z,Z~)(tk)=Z+(Z~−Z)f(k/KT)(9)z^{(Z,\widetilde{Z})}(t_k)=Z+(\widetilde{Z}-Z)f(k/K_T) \tag*{(9)}
vz(Z,Z~)(tk)=(Z~−Z)f′(k/KT)KTΔt.(10)v_z^{(Z,\widetilde{Z})}(t_k)=(\widetilde{Z}-Z)\frac{f'(k/K_T)}{K_T\Delta t}. \tag*{(10)}

References

  1. [2]R. Swendsen and J.-S. Wang, Phys. Rev. Lett. 57, 2607 (1986).
  2. [3]R. Neal, Statistics and Computing 11, 125 (2001).
  3. [4]P. Del Moral, A. Doucet, and A. Jasra, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68, 411 (2006).
  4. [5]D. B. Woodard, S. C. Schmidler, and M. Huber, The Annals of Applied Probability 19, 617 (2009).
  5. [6]S. Syed, A. Bouchard-Côté, G. Deligiannidis, and A. Doucet, Journal of the Royal Statistical Society Series B: Statistical Methodology 84, 321 (2022).
  6. [7]G. Torrie and J. Valleau, J. Comp. Phys. 23, 187 (1977).
  7. [8]S. Kumar, D. Bouzida, R. H. Swendsen, P. A. Kollman, and J. M. Rosenberg, J. Comput. Chem. 13, 1011 (1992).
  8. [9]M. R. Shirts and J. D. Chodera, J. Chem. Phys. 124, 124105 (2008).
  9. [10]J. Kirkwood, J. Chem. Phys. 3, 300 (1935).
  10. [11]E. Darve and A. Porohille, J. Chem. Phys. 115, 9169 (2001).
  11. [12]J. Hénin and C. Chipot, J. Chem. Phys. 121, 2904 (2004).
  12. [13]A. Laio and M. Parrinello, PNAS 99, 12562 (2002).
  13. [14]A. Barducci, G. Bussi, and M. Parrinello, Phys. Rev. Lett. 100, 020603 (2008).
  14. [15]G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan, J. Mach. Learn. Res. 22, 1 (2021).
  15. [16]F. Noé, S. Olsson, J. Köhler, and H. Wu, Science 365, eaaw1147 (2019).
  16. [17]L. Grenioux, A. Durmus, E. Moulines, and M. Gabrié, in Proceedings of the 40th International Conference on Machine Learning (PMLR, 2023) pp. 11698–11733.
  17. [18]S.-H. Li and L. Wang, Phys. Rev. Lett. 121, 260601 (2018).
  18. [19]M. Gabrié, G. Rotskoff, and E. Vanden–Eijnden, Proc. Natl. Acad. Sci. U.S.A 119, e2109420119 (2022).
  19. [20]M. Invernizzi, A. Krämer, C. Clementi, and F. Noé, J. Phys. Chem. Lett. 13, 11643 (2022).
  20. [21]R. Abbott, M. Albergo, A. Botev, D. Boyda, K. Cranmer, D. Hackett, G. Kanwar, A. Matthews, S. Racanierie, A. Razavi, D. Rezende, F. Romero-Lopez, P. Shanahan, and J. Urban, in Proceedings of The 39th International Symposium on Lattice Field Theory — PoS(LATTICE2022), Vol. 430 (SISSA Medialab, 2023) p. 036.
  21. [22]L. Del Debbio, J. Rossney, and M. Wilson, Phys. Rev. D 104, 094507 (2021).
  22. [23]C. Schönle and M. Gabrié, in NeurIPS 2023 AI for Science Workshop (2023).
  23. [24]R. Abbott, M. S. Albergo, A. Botev, D. Boyda, K. Cranmer, D. C. Hackett, A. G. D. G. Matthews, S. Racanierie, A. Razavi, D. J. Rezende, F. Romero-López, P. E. Shanahan, and J. M. Urban, Eur. Phys. J. A 59, 257 (2023).
  24. [25]P. Bera and J. Mondal, Chem. Sci. 16, 8735 (2025).
  25. [26]A. S. Lewis, T. Hempel, J. Jiménez-Luna, M. Gastegger, Y. Xie, A. Y. K. Foong, V. G. Satoras, O. Abdin, B. S. Veeling, I. Zaporozhets, Y. Chen, S. Yang, A. E. Foster, A. Schneuing, J. Nigam, F. Barbero, V. Stimper, A. Campbell, J. Yim, M. Lienen, Y. Shi, S. Zheng, H. Schulz, U. Munir, R. Sordillo, R. Tomioka, C. Clementi, and F. Noé, Science 389, eadv9817 (2025).
  26. [27]P. Bera and J. Mondal, J. Chem. Phys. 163, 114110 (2025).
  27. [28]M. Athènes, Phys. Rev. E 66, 046705 (2002).
  28. [29]J. Nelmeijer, G. Crooks, D. Minh, and J. D. Chodera, PNAS 108, E1009 (2011).
  29. [30]Y. Chen and B. Roux, J. Chem. Theory Comput. 11, 3572 (2015).
  30. [31]Y. Chen and B. Roux, J. Chem. Phys. 142, 024101 (2015).
  31. [32]R. Neal, arXiv:math/0502099 (2005).arxiv.org/abs/math/0502099
  32. [33]G. Karagiannis and C. Andrieu, J. Comp. Graph. Stat. 22, 623 (2013).
  33. [34]P. Das, A. D. S. Parmar, and S. Sastry, J. Chem. Phys. 157, 044501 (2022).
  34. [35]C. Schönle, M. Gabrié, T. Lelievre, and G. Stoltz, J. Comp. Phys 527, 113806 (2025).
  35. [36]C. Jarzynski, Phys. Rev. E 56, 5018 (1997).
  36. [37]G. Crooks, Phys. Rev. E 60, 2721 (1999).
  37. [38]S. Tamagnone, A. Laio, and M. Gabrié, J. Chem. Theory Comput. 20, 7796 (2024).
  38. [39]T. Lelievre, M. Rousset, and G. Stoltz, Free Energy Computations: A Mathematical Perspective (Imperial College Press, 2010).
  39. [40]T. Lelievre, M. Rousset, and G. Stoltz, Math. Comp. 81, 2071 (2012).
  40. [41]M. Fixman, J. Chem. Phys. 69, 1527 (1978).
  41. [42]S. Duane, A. Kennedy, B. J. Pendleton, and D. Roweth, Phys. Lett. B 195, 216 (1987).
  42. [43]S. Vaikuntanathan and C. Jarzynski, The Journal of Chemical Physics 134, 054107 (2011).
  43. [44]M. Caselle, E. Cellini, A. Nada, and M. Panero, J. High Energ. Phys. 2022, 15 (2022), arXiv:2201.08862 [hep-lat].DOI
  44. [45]P. Rossky, J. Doll, and H. Friedman, J. Chem. Phys. 69, 4628 (1978).
  45. [46]G. Roberts and R. Tweedie, Bernoulli 2, 341 (1996).
  46. [47]J. E. Straub, M. Borkovec, and B. J. Berne, J. Chem. Phys. 89, 4833 (1988).
  47. [48]C. Dellago, P. G. Bolhuis, and D. Chandler, J. Chem. Phys. 110, 6617 (1999).
  48. [55]S. S. Schoenholz and E. D. Cubuk, in Advances in Neural Information Processing Systems, Vol. 33 (Curran Associates, Inc., 2020).
  49. [56]D. Ward, “FlowJAX: Distributions and normalizing flows in JAX,” (2025), v17.2.0.
  50. [57]C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios, arXiv:1906.04032 (2019).arxiv.org/abs/1906.04032

Paper details

vlnc:W_mhvu2zv3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 7

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents