betaFeedback

WorksMathematics

Semi-global Convergence and Finite-Sample Error Bounds for Model Predictive Path Integral Control

Abstract

We investigate the global optimisation behaviour of Model Predictive Path Integral Control (MPPI) in the low-temperature setting. For objective functions with a unique global minimum and under weak additional assumptions, allowing non-convexity and non-smoothness, we establish semi-global convergence of the mean-field or population level MPPI iteration to this global optimum. We also establish finite-sample results that show that with high probability the MPPI iteration comes close to the global optimum and provide sample complexity results. Under slightly stronger assumptions, we make these results more quantitative and provide concrete bounds on the inverse temperature and number of samples. Finally, we illustrate the results on the classic LQR problem.

Introduction

Model Predictive Path-Integral Control (MPPI) is one of the major sampling-based control methods used in robotics, autonomous driving, and model-based reinforcement learning [11]. While very successful in applications, the theoretical understanding of MPPI is still limited. Introduced originally by information-theoretic arguments connected to stochastic optimal control [16], it has since been recognised that MPPI is essentially a zeroth-order sampling-based optimisation method combined with receding-horizon control. In particular, a rigorous theoretical analysis must start with the optimisation behaviour of MPPI. Most existing works have been focused on local perspectives, e.g., interpreting MPPI as a form of gradient descent [3], or considering restricted classes of objectives like quadratic functions [17]. This is a severe limitation, since MPPI is particularly interesting for challenging non-convex and even non-smooth problems arising in control tasks. Motivated by this shortcoming, we investigate MPPI as a sampling-based global optimisation method for non-convex, non-smooth objective functions with a unique global optimum. For this, we build on the recent work [5] that connects MPPI with global non-convex optimisation. More precisely, this work finds that the plain MPPI iteration on the population level corresponds exactly to Consensus Hopping (CH), a limiting case of Consensus-Based Optimisation (CBO), which is a class of popular global optimisation methods based on interacting particle systems [13] with strong theoretical guarantees [4, 2]. In turn, this connection allows one to prove strong convergence results for CH and hence MPPI. We build on this work and establish semi-global convergence results for MPPI on the population level. While already conceptually insightful, MPPI is a sampling-based method, and hence any result with practical relevance needs to take into account the finite-sample error incurred by the sampling process. To address this important issue, we also prove corresponding high-probability error bounds and establish sample complexity bounds for the actual sampling-based MPPI iteration. Apart from requiring a unique global optimum, our results rely only on rather general assumptions on the objective function, covering in particular non-convex and non-smooth functions. Our work provides a significant step towards a deeper theoretical understanding of MPPI and helps to put it on a solid theoretical foundation, on which reliable sampling-based control approaches can be built.

Related Work MPPI was introduced in the context of autonomous driving [15] based on information-theoretic considerations [16] that connect to stochastic optimal control and path-integral control. However, the connection is mostly heuristic and the rigorous theoretical foundation is lost [8]. By now there is a multitude of different MPPI variants, cf. Kazim et al. [11] for an overview, and similar to most analyses, we focus on the plain MPPI iteration formally introduced in § 2, and investigate MPPI as a zeroth-order sampling-based optimisation method, cf. Jordana et al. [10] for more on this perspective. In contrast to previous works like Yi et al. [17] or Yoon and Kim [18], we consider general non-convex objective functions. For this, we build on the recent work Fornasier et al. [5] that identified MPPI on the population level as Consensus Hopping (CH), which is connected to Consensus-Based Optimisation (CBO) [13]. Utilising a quantitative Laplace principle as pioneered by the work Fornasier et al. [4], one can establish semi-global convergence guarantees for CH and hence for population-level MPPI. The concept of semi-global results is common in nonlinear control, see (Grune and Pannek, 2017, Chapter 6) for more details. The use of the Laplace principle in connection with MPPI has already been noticed in Homburger et al. [8]; however, a different scaling is used there, and no non-asymptotic results are provided. Note that it is well-known that MPPI has problems with functions having multiple global optima [12] due to averaging effects, which motivates our focus on objectives with a unique global optimum. We also provide finite-sample results, which are particularly relevant since in practice MPPI is always sampling-based. Prior works have investigated this in restricted settings like LQR problems, cf. [19, 18], whereas we work with very general non-convex objectives. Instead of relying on propagation of chaos techniques as suggested in Fornasier et al. [5], we directly analyse the estimation error in a single step from the perspective of self-normalised importance weighted sampling, cf. [1] for an overview and similar results. While this has been used in prior works on MPPI like Yoon et al. [19]; Yoon and Kim [18], our simple but effective estimation error bounds become particularly powerful in connection with our semi-global convergence results. In particular, to the best of our knowledge we achieve the first finite-sample complexity bounds for general nonconvex objective functions with only modest regularity assumptions.

Contributions First, we isolate, generalise, and sharpen the mean-field analysis of the MPPI iteration from Fornasier et al. [5], establishing semi-global convergence to the global optimum. Second, we provide the first semi-global finite-sample accuracy results for MPPI. Third, under additional assumptions, we provide sharpened finite-sample results that include the LQR problem.

Outline In § 2, we introduce the mean-field and empirical versions of MPPI that we will investigate. § 3 contains results for the mean-field iteration, and § 4 the corresponding finite-sample results. In the main text, we present simplified results that highlight the main features; the supplementary material contains more detailed and quantitative results with full proofs. In § 5 we illustrate some results with the standard LQR problem. Finally, § 6 contains a discussion and outlook.

MPPI and Consensus Hopping

Consider an objective function f:Rd→Rf : \mathbb{R}^{d} \to\mathbb{R} that is measurable and bounded from below. Let α∈R>0\alpha\in\mathbb{R}_{>0} and let Σ∈Rd×d\Sigma\in\mathbb{R}^{d \times d} be a symmetric positive definite matrix. Throughout, σmin⁡(Σ)\sigma_{\min}(\Sigma) and σmax⁡(Σ)\sigma_{\max}(\Sigma) denote the square roots of the smallest and largest eigenvalue of Σ\Sigma, respectively. The MPPI iteration with inverse temperature α\alpha and sampling covariance matrix Σ\Sigma is defined as follows. Starting from an initial guess x^0∈Rd\hat{x}_0 \in\mathbb{R}^{d}, in order to compute the next iterate x^k+1\hat{x}_{k+1} from the current iterate x^k\hat{x}_k, we sample X1,…,XN∼i.i.d.N(x^k,Σ)X_1,\ldots,X_N \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \mathcal{N}(\hat{x}_k,\Sigma) and then set

x^k+1=∑i=1Nexp⁡(−αf(Xi))Xi∑i=1Nexp⁡(−αf(Xi))(1)\hat{x}_{k+1} = \frac{\sum_{i=1}^{N} \exp(-\alpha f(X_i))X_i}{\sum_{i=1}^{N} \exp(-\alpha f(X_i))} \tag*{(1)}

Note that in each iteration we draw a new sample.

We start the analysis of the MPPI iteration on the mean-field or population-level. For this, define the transition map

Tα:Rd→Rd,Tα(x):=EX∼N(x,Σ)[exp⁡(−αf(X))X]EX∼N(x,Σ)[exp⁡(−αf(X))]=1Zα(x)EX∼N(x,Σ)[exp⁡(−αf(X))X],(2)\begin{aligned} \mathcal{T}_{\alpha} : \mathbb{R}^{d} \to\mathbb{R}^{d}, \qquad\mathcal{T}_{\alpha}(x) := \frac{\mathbb{E}_{X \sim\mathcal{N}(x,\Sigma)}[\exp(-\alpha f(X))X]}{\mathbb{E}_{X \sim\mathcal{N}(x,\Sigma)}[\exp(-\alpha f(X))]} \\ &= \frac{1}{Z_{\alpha}(x)}\mathbb{E}_{X \sim\mathcal{N}(x,\Sigma)}[\exp(-\alpha f(X))X], \tag*{(2)} \end{aligned}

where Zα(x)=EX∼N(x,Σ)[exp⁡(−αf(X))]Z_{\alpha}(x) = \mathbb{E}_{X \sim\mathcal{N}(x,\Sigma)}[\exp(-\alpha f(X))] is a normalisation constant, and for later use we define the free energy by

Eα(x)=−1αlog⁡Zα(x)=−1αlog⁡EX∼N(x,Σ)[exp⁡(−αf(X))].(3)\mathcal{E}_{\alpha}(x) = -\frac{1}{\alpha}\log Z_{\alpha}(x) = -\frac{1}{\alpha}\log\mathbb{E}_{X \sim\mathcal{N}(x,\Sigma)}[\exp(-\alpha f(X))]. \tag*{(3)}

Given some initial state x0∈Rdx_0 \in\mathbb{R}^{d}, we define the mean-field MPPI iteration by

xk+1=Tα(xk),k∈N0.(4)x_{k+1} = \mathcal{T}_{\alpha}(x_k), \qquad k \in\mathbb{N}_0. \tag*{(4)}

As is well-known, each step of the MPPI iteration (1) is a self-normalised importance-weighted estimate of the corresponding mean-field step. As pointed out in Fornasier et al. [5], the mean-field MPPI iteration is exactly the Consensus Hopping (CH) iteration from this reference.

Assume now that ff has a global minimiser x∗x_*. The core question of the present work is in which sense the MPPI iterates (x^k)k(\hat{x}_k)_k and the mean-field MPPI iterates (xk)k(x_k)_k approximate the global minimiser x∗x_* of ff. It is clear that without any additional assumptions on ff, little progress can be made, so our analysis will be based mostly on the following properties of ff.

Assumption 1 (Objective function) The function f:Rd→Rf : \mathbb{R}^{d} \to\mathbb{R} satisfies the following.

(F1) ff is measurable and attains its global minimum f∗∈Rf_* \in\mathbb{R} at x∗∈Rdx_* \in\mathbb{R}^{d}.

(F2) ff is upper semicontinuous at x∗x_*.

(F3) For every ϵ>0\epsilon> 0 there exists Sϵ>0S_{\epsilon} > 0 such that

inf⁡∥x−x∗∥2≥ϵf(x)≥f∗+Sϵ.(5)\inf_{\lVert x-x_\ast\rVert_2\ge\epsilon} f(x) \ge f_\ast+ S_\epsilon. \tag*{(5)}

(F2) is a very weak continuity assumption. (F3) makes the notion precise that the global minimiser x∗x_\ast is sufficiently separated from any other local minimisers and that the global minimum is sufficiently pronounced. In particular, an elementary computation shows that (F3) implies that x∗x_\ast is indeed the unique global minimiser of ff.

To obtain explicit thresholds, we also consider the following simplified assumption, which only requires strong convexity close to the global minimiser and a suboptimality gap away from it.

Assumption 2 (Local strong convexity and energy gap) The function f:Rd→Rf : \mathbb{R}^d \to\mathbb{R} is continuous and attains its global minimum f∗f_\ast at x∗∈Rdx_\ast\in\mathbb{R}^d, and there are R,m,Δ>0R,m,\Delta> 0 such that the following hold.

(SF1) The function y↦f(y)−m2∥y−x∗∥22y \mapsto f(y) - \frac{m}{2}\lVert y-x_\ast\rVert_2^2 is convex on BR(x∗)B_R(x_\ast).

(SF2) f(y)−f∗≥Δf(y)-f_\ast\ge\Delta for all y∈Rd∖BR(x∗)y \in\mathbb{R}^d \setminus B_R(x_\ast).

(SF3) There is M≥mM \ge m such that f(y)−f∗≤M2∥y−x∗∥22f(y)-f_\ast\le\frac{M}{2}\lVert y-x_\ast\rVert_2^2 for all y∈BR(x∗)y \in B_R(x_\ast).

(SF1) and (SF2) imply Assumption 1, and (F3) holds with Sϵ=min⁡{m2ϵ2,Δ}S_\epsilon= \min\{\frac{m}{2}\epsilon^2,\Delta\}, cf. Lemma 19 in the supplementary material. We stress that outside the well BR(x∗)B_R(x_\ast), the objective can still be non-convex and non-smooth. Furthermore, condition (SF3) is only needed for the finite-sample results, and it holds, for instance, if ff has an MM-Lipschitz gradient on BR(x∗)B_R(x_\ast).

Semi-Global Convergence in Mean-Field

Elementary computations show that

Tα(x)=x−αΣ∇Eα(x),(6)T_\alpha(x) = x - \alpha\Sigma\nabla\mathcal{E}_\alpha(x), \tag*{(6)}

suggesting that mean-field MPPI is a form of preconditioned gradient descent on a smoothed objective function, cf. Fazlyab et al. [3] for an in-depth discussion of this connection. The following example illustrates this point of view.

Example 1 Let d=1d = 1, and consider the objective

f(y)=min⁡{12y2, 1+12(y−3)2},(7)f(y) = \min\left\{\frac{1}{2}y^2,\ 1+\frac{1}{2}(y-3)^2\right\}, \tag*{(7)}

which has the global minimiser x∗=0x_\ast= 0 with f∗=0f_\ast= 0, a strict local minimiser at y=3y = 3 with f(3)=1f(3) = 1, and a kink at y=11/6y = 11/6, where the two branches meet. Note that ff fulfils Assumption 1, though it is neither convex nor differentiable. Figure 1 shows Eα\mathcal{E}_\alpha for Σ∈{0.352,1}\Sigma\in\{0.35^2,1\} and α∈{1,3,8}\alpha\in\{1,3,8\}. For Σ=0.352\Sigma= 0.35^2, all three smoothed objectives keep a second well near the local minimiser 3. For Σ=1\Sigma= 1, the proposal reaches the deeper well from every centre in the displayed interval, and Eα\mathcal{E}_\alpha has a single minimum near x∗x_\ast already for α=1\alpha= 1.

Figure 1. The free energy EαE_\alpha for the objective (7) of Example 1 with Σ=0.352≈0.12\Sigma= 0.35^2 \approx0.12 (left) and Σ=1\Sigma= 1 (right), for α∈{1,3,8}\alpha\in\{1,3,8\} (solid, darker for larger α\alpha). The dashed curve is the objective ff.

References

  1. [1]Sergios Agapiou, Omiros Papaspiliopoulos, Daniel Sanz-Alonso, and Andrew M Stuart. Importance sampling: computational complexity and intrinsic dimension. Stat. Sci, 32(3):405–431, 2017.
  2. [2]José A Carrillo, Young-Pil Choi, Claudia Totzeck, and Oliver Tse. An analytical framework for consensus-based global optimization method. Mathematical Models and Methods in Applied Sciences, 28(06):1037–1066, 2018.arxiv.org/abs/1602.00220
  3. [3]Mahyar Fazlyab, Sina Sharifi, and Jiarui Wang. Model predictive path integral control as preconditioned gradient descent. IEEE Control Systems Letters, 2026.
  4. [4]Massimo Fornasier, Timo Klock, and Konstantin Riedl. Consensus-based optimization methods converge globally. SIAM Journal on Optimization, 34(3):2973–3004, 2024.DOI
  5. [5]Massimo Fornasier, Hui Huang, Jona Klemenc, and Greta Malaspina. From consensus-based optimization to evolution strategies: Proof of global convergence, 2026. arXiv:2602.11677, version 2, 12 October 2026.arxiv.org/abs/2602.11677
  6. [7]Konstantin Haug. Quantitative mean-field limits for recent variants of consensus-based optimization. arXiv preprint arXiv:2610.02901, 2026.arxiv.org/abs/2610.02901
  7. [8]Hannes Homburger, Florian Messerer, Moritz Diehl, and Johannes Reuter. Optimality and suboptimality of MPPI control in stochastic and deterministic settings. IEEE Control Systems Letters, 9:763–768, 2025.DOI
  8. [9]Hui Huang, Hicham Kouhkouh, and Lukang Sun. Faithful global convergence for the rescaled consensus-based optimization. SIAM Journal on Optimization, 36(4):2121–2150, 2026.arxiv.org/abs/2503.08578
  9. [10]Armand Jordana, Jianghan Zhang, Joseph Amigo, and Ludovic Righetti. An introduction to zero-order optimization techniques for robotics. arXiv preprint arXiv:2506.22087, 2025.arxiv.org/abs/2506.22087
  10. [11]Muhammad Kazim, JunGee Hong, Min-Gyeom Kim, and Kwang-Ki K Kim. Recent advances in path integral control for trajectory optimization: An overview in theoretical and algorithmic perspectives. Annual Reviews in Control, 57:100931, 2024.arxiv.org/abs/2309.12566
  11. [12]Zidong Liu, Kaixin Chang, and Xu Chen. Clustering-embedded model predictive path integral control: Avoiding averaging-induced failure and enabling efficient cluster selection for dynamic obstacles. arXiv preprint arXiv:2607.06499, 2026.arxiv.org/abs/2607.06499
  12. [13]René Pinnau, Claudia Totzeck, Oliver Tse, and Stephan Martin. A consensus-based model for global optimization and its mean-field limit. Mathematical Models and Methods in Applied Sciences, 27(01):183–204, 2017.DOI
  13. [14]András Prékopa. On logarithmic concave measures and functions. Acta Scientiarum Mathematicarum (Szeged), 34:335–343, 1973.
  14. [15]Grady Williams, Paul Drews, Brian Goldfain, James M. Rehg, and Evangelos A. Theodorou. Aggressive driving with model predictive path integral control. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages 1433–1440. IEEE, 2016.DOI
  15. [16]Grady Williams, Paul Drews, Brian Goldfain, James M. Rehg, and Evangelos A. Theodorou. Information-theoretic model predictive control: Theory and applications to autonomous driving. IEEE Transactions on Robotics, 34(6):1603–1622, 2018. doi: 10.1109/TRO.2018.2865891.DOI
  16. [17]Zeji Yi, Chaoyi Pan, Guangqi He, Guannan Qu, and Guanya Shi. CoVO-MPC: Theoretical analysis of sampling-based MPC and optimal covariance design. In Proceedings of the 6th Annual Learning for Dynamics and Control Conference, volume 242 of Proceedings of Machine Learning Research, pages 1122–1135, 2024.arxiv.org/abs/2401.07369
  17. [18]Hyung-Jin Yoon and Hunmin Kim. Finite-sample closed-loop stability of model predictive path integral control for linear time-invariant systems. arXiv preprint arXiv:2607.04006, 2026.arxiv.org/abs/2607.04006
  18. [19]Hyung-Jin Yoon, Chuyuan Tao, Hunmin Kim, Naira Hovakimyan, and Petros Voulgaris. Sampling complexity of path integral methods for trajectory optimization. In 2022 American Control Conference (ACC), pages 3482–3487. IEEE, 2022. arXiv:2203.10067.arxiv.org/abs/2203.10067

Paper details

vlnc:W_ca956eyz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Claude Opus 5.5 (bare), ChatGPT-6

View statement
Looking for related papers…
Contents