betaFeedback

WorksMathematics

Conditions for Normality of Pontryagin's Maximum Principle in the Absence of State Constraints

Abstract

Pontryagin's Maximum Principle is said to be "normal" for the problem at hand when it has a solution (maximizer u*) that explicitly involves the problem's cost function, making the maximum principle helpful (while in the opposite situation it is of no help). The aim of the present paper is, for Lagrange optimal control problems without state constraints, to identify classes of problems for which the Principle is normal. General conditions are introduced that guarantee normality, and classes of problems that satisfy these conditions are identified. Such classes include problems for which the constrained (by bounds on the control values) "augmented" reachable set, with cost along an upward vertical axis, has a convex lower-boundary function and the optimal terminal state lies in the relative interior of the reachable state set, with the terminal constraint within the affine hull of that set. Such classes include problems for which the constrained (by bounds on the values of the control signal) "augmented" (with the cost value along an upward "vertical axis") reachable set has a "convex" lower boundary for which the optimal terminal state lies in the relative interior of the set of reachable states. Special attention is given to Lagrange steering problems: sufficient conditions are obtained on a nested sequence of sub-classes of such problems, culminating with the class of steering problems with scalar control signal and linear time-invariant (LTI) state dynamics and objective integrand, for which normality is guaranteed whenever all eigenvalues of the state dynamics matrix A are real (e.g., if A is symmetric or triangular or the system is overdamped) or have small enough imaginary parts; simple 2 x 2 examples show that, in contrast with the situation in the finite-dimensional case, this property does not extend to arbitrary A matrices.

Introduction

Normality of Pontryagin’s maximum principle (PMP) has been an object of much attention over the past few decades; see, e.g., [12], [5], [18], [6], [11]. In particular, in the presence of both endpoints constraints, sufficient conditions for normality were obtained in [12] that apply to a large class of optimal control problems with (or without) state constraints. However, as noted just before Theorem 5.6 in that paper, such conditions are obtained only for trajectories ending on the boundary ∂K\partial K of the set KK that defines the state constraints (see (1.2) in [12]); accordingly, in the absence of state constraints (there are none in the present paper), KK is the entire space so ∂K\partial K is empty, and no trajectories can end in ∂K\partial K, so that Theorem 5.6 of [12] is not relevant when both endpoints are constrained.

Standard statements of PMP’s necessary conditions of optimality (see, e.g., [20], [21], [14]) involve a constant scalar “multiplier” associated to the objective function, sometimes referred to as the abnormal multiplier—noted p0∗p_0^* in the present paper—for which, depending on the specifics of the problem at hand, the value zero may be the only possible choice, making the PMP abnormal. The intent of the present paper is to identify classes of optimal control problems for which PMP’s necessary conditions do not suffer such drawback.

With the focus first placed on the Mayer problem, after setting the background (section II), in section III, we introduce conditions (TS1 and TS2) at the terminal state (LTS1 in a Lagrange-focused context; see section IV below), which are shown (Theorem 1) to be sufficient for PMP’s conditions to hold with p0∗≠0p_0^* \ne0 (equivalently, via scaling, with p0∗=1p_0^*=1) for the problem under consideration.

The main portion of the paper (section IV) focuses on the Lagrange formulation of optimal control problems. It investigates conditions under which TS1/LTS1 and TS2 hold. (LTS1 excludes downwards directions in the closed convex hull of the augmented reachable set’s contingent cone.

It is then first shown (section IV-A) that, for rather general Lagrange problems, convexity of the lower boundary function and an interior point assumption imply LTS1, provided the terminal condition is regular within the affine hull of the reachable set. This is then shown to be the case in a class of problems studied by L.W. Neustadt [16 Theorem 1].

Attention then shifts (section IV-B) to the class of Lagrange steering problems, specifically, problems in which the terminal state is prescribed; for such problems, Condition TS2 is always satisfied; further, when the state dynamics is linear and time-invariant (LTI), for a large class of cost integrands linear in the state, either (i) the optimal control is bang–bang or (ii) PMP is normal (Theorem 3). Finally (section IV-C), when in addition the cost integrand is time-independent and linear in state and control signals, PMP is normal as long as (Theorem 4) all eigenvalues of the state dynamics matrix AA are real (Theorem 4)—as well as, at least in the 2×22 \times2 case, when the imaginary parts of the eigenvalues of AA are appropriately small.

From section IV-A onward, the analysis is limited to the single-input case; the multi-input case is left for future studies.

Notation. It is mostly standard. Worth mentioning are the following: 0k0_k is the origin of Rk\mathbb{R}^k; given a scalar δ>0\delta> 0, Bδ(ξ)B_\delta(\xi) is the open ball of radius δ\delta centered at ξ\xi; given a differentiable mapping G:Rp→RqG : \mathbb{R}^p \to\mathbb{R}^q, JG:Rp→Rq×pJG : \mathbb{R}^p \to\mathbb{R}^{q \times p} is its Jacobian; and the sign function “sgn” takes values in {−1,0,1}\{-1, 0, 1\}, with sgn⁡(0):=0\operatorname{sgn}(0):=0. Further, given v∈Rpv \in\mathbb{R}^p and w∈Rqw \in\mathbb{R}^q the Matlab notation [v;w]∈Rp+q[v; w] \in\mathbb{R}^{p+q} designates the concatenation of vv and ww, N(M)\mathcal{N}(M) and R(M)\mathcal{R}(M) denote the kernel (nullspace) and range of matrix MM; and, given a set SS, ∂S\partial S denotes its boundary, co⁡(S)\operatorname{co}(S) its convex hull, cl⁡(S)\operatorname{cl}(S) is closure, aff⁡(S)\operatorname{aff}(S) its affine hull, and relint⁡(S)\operatorname{relint}(S) its relative interior.

Background

Consider the Mayer problem with fixed initial and terminal times, fixed initial state, and free terminal state, viz.

min⁡u∈Uψ(x(1))\min_{u \in\mathcal{U}} \psi(x(1))
s.t.x˙(t)=f(t,x(t),u(t)) a.e. t∈[0,1],\text{s.t.}\quad\dot{x}(t)=f(t,x(t),u(t))\ \text{a.e. }t\in[0,1],
x abs. cont.,x(0)=ξ0,(1)x\ \text{abs. cont.},\quad x(0)=\xi^0, \tag*{(1)}

where ψ:Rn→R\psi:\mathbb{R}^n\to\mathbb{R} is continuously differentiable, f:R×Rn×Rm→Rnf:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R}^n is (jointly) continuous, as well as locally Lipschitzian in its second argument (as is f0:R×Rn×Rm→Rf_0:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R}, to be introduced below in the context of the Lagrange problem), ξ0∈Rn\xi^0\in\mathbb{R}^n and U⊆RmU\subseteq\mathbb{R}^m are prescribed, and U\mathcal{U} is the set of piecewise-continuous functions u:[0,1]→Uu:[0,1]\to U, that have at most a finite number of points of discontinuity in [0,1][0,1] and are right-continuous on [0,1)[0,1).1 Of course, given an optimal (for (1)) control u∗∈Uu^*\in\mathcal{U}, the associated optimal state trajectory x∗(⋅)x^*(\cdot) is well defined (and continuous) on [0,1][0,1].

Problem (1) is closely related to the finite-dimensional optimization problem

min⁡ψ(ξ)s.t. ξ∈K(ξ0),(2)\min\psi(\xi)\quad\text{s.t. }\xi\in K(\xi^0), \tag*{(2)}

where K(ξ0)K(\xi^0) is the set of states reachable at time t=1t=1 when starting from state ξ0\xi^0 at t=0t=0 and applying a control u∈Uu\in\mathcal{U}. In particular, if ξ∗\xi^* solves (2), every control u∗∈Uu^*\in\mathcal{U} that steers xx from x(0)=ξ0x(0)=\xi^0 to x(1)=ξ∗x(1)=\xi^*, solves (1). Now, it is well known (e.g., [20 Chapter V, Lemma 1]) that if ξ∗\xi^* solves (2) then

Jψ(ξ∗)d≥0J\psi(\xi^*)d\geq0
∀d∈cl⁡co⁡CC(ξ∗,K(ξ0)),(3)\forall d\in\operatorname{cl}\operatorname{co}CC(\xi^*,K(\xi^0)), \tag*{(3)}

where CC(ξ,S)CC(\xi,S) is GL Bouligand’s contingent cone2 [9 section 3] to SS at ξ∈S\xi\in S. For ease of reference, we recall here the definition of the contingent cone, as formulated, e.g., in [8 Theorem 6.3.6(a)].

Definition 1: (e.g., [8 Theorem 6.3.6(a)]) Given S⊂RnS\subset\mathbb{R}^n and ξ∈cl⁡(S)\xi\in\operatorname{cl}(S), the contingent cone CC(ξ,S)CC(\xi,S) consists of those vectors d∈Rnd\in\mathbb{R}^n for which there are sequences {ϵk}↓0\{\epsilon_k\}\downarrow0 in R\mathbb{R} and {dk}→d\{d_k\}\to d in Rn\mathbb{R}^n such that ξ+ϵkdk∈S\xi+\epsilon_kd_k\in S for all kk. Based on a “needle” perturbation, sensitivity analysis shows that, given any v∈Uv\in U, and any τ∈[0,1)\tau\in[0,1) at which u∗u^* is continuous,

dτ,v:=Φ(1,τ)[f(τ,x∗(τ),v)−f(τ,x∗(τ),u∗(τ))]d_{\tau,v}:=\Phi(1,\tau)\left[f(\tau,x^*(\tau),v)-f(\tau,x^*(\tau),u^*(\tau))\right]
∈CC(x∗(1),K(ξ0)),(4)\in CC(x^*(1),K(\xi^0)), \tag*{(4)}

where x∗x^* is the state trajectory associated to u∗u^* and Φ:[0,1]×[0,1]→Rn×n\Phi:[0,1]\times[0,1]\to\mathbb{R}^{n\times n} the state-transition operator for the linearization of ff along (x∗(t),u∗(t));(x^*(t),u^*(t)); see, e.g., [20 Exercise 1 in section VIII.1], [14 equation (4.23)].

The following instance of PMP conditions (see, e.g., [20 section VIII.1, Theorem 2]), where H:R×Rn×Rn×Rm→RH:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R} is the pre-Hamiltonian associated to Problem (1), given by

H(t,ξ,π,v):=f(t,ξ,v)TπH(t,\xi,\pi,v):=f(t,\xi,v)^{\mathsf{T}}\pi

and Jξf:R×Rn×Rm→Rn×RnJ_\xi f:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R}^n\times\mathbb{R}^n denotes the Jacobian of ff with respect to its second argument.

Fact 1: Let u∗∈Uu^*\in\mathcal{U} be optimal for problem (1), with associated state trajectory x∗x^*. Then there exists an absolutely continuous p∗:R→Rnp^*:\mathbb{R}\to\mathbb{R}^n such that

H(t,x∗(t),p∗(t),u∗(t))=max⁡v∈UH(t,x∗(t),p∗(t),v)H(t,x^*(t),p^*(t),u^*(t))=\max_{v\in U}H(t,x^*(t),p^*(t),v)
∀t∈[0,1],\forall t\in[0,1],

and p∗p^* solves the linear, terminal-value problem

p˙∗(t)=−Jξf(t,x∗(t),u∗(t))Tp∗(t)\dot{p}^*(t)=-J_\xi f(t,x^*(t),u^*(t))^{\mathsf{T}}p^*(t)
a.e. t∈[0,1],p∗(1)=−∇ψ(x∗(1)).\text{a.e. }t\in[0,1],\quad p^*(1)=-\nabla\psi(x^*(1)).

This result is easily extended to problems where x∗(0)x^*(0), rather than being prescribed, is merely restricted to lie on a specified manifold, namely, g0(x∗(0))=0m0g^0(x^*(0))=0_{m_0}, where g0:Rn→Rm0g^0:\mathbb{R}^n\to\mathbb{R}^{m_0}, for some m0m_0, is continuously differentiable, with Jacobian Jg0:Rn→Rm0×nJg^0:\mathbb{R}^n\to\mathbb{R}^{m_0\times n}. For such problem, it is readily determined that, under the condition that x∗(0)x^*(0) is regular for g0g^0 (i.e., that Jg0(x∗(0))Jg^0(x^*(0)) is surjective), Fact 1 still holds as stated with, in addition, p∗(0)p^*(0) being now constrained to satisfy the “transversality” condition3

p∗(0)⊥N(Jg0(x∗(0))).p^*(0)\perp\mathcal{N}\left(Jg^0(x^*(0))\right).

The case of a constrained terminal state is more challenging. Indeed, consider

min⁡u∈Uψ(x(1))s.t.x˙(t)=f(t,x(t),u(t)),a.e. t∈[0,1],x abs. cont.,g0(x(0))=0m0,gf(x(1))=0mf.(5)\begin{aligned} \min_{u\in\mathcal{U}}\quad&\psi(x(1))\\ \text{s.t.}\quad&\dot{x}(t)=f(t,x(t),u(t)),\quad\text{a.e. }t\in[0,1],\\ &x\ \text{abs. cont.},\quad g^0(x(0))=0_{m_0},\\ &g^f(x(1))=0_{m_f}. \tag*{(5)} \end{aligned}

References

  1. [1]P.A. Absil, I. Coujhari, I. Fiodorov, and A.L. Tits. Strong versions of impulsive controllability and sampled observability. Automatica, 169:111865, 2024.DOI
  2. [2]H.R. Babad. Optimality conditions and sensitivity relations in dynamic optimization. PhD thesis, Imperial College of Science, Technology & Medicine, Univ. of London, 1991.
  3. [3]H.R. Babad and R.B. Vinter. Sensitivity interpretation of the costate function of optimal control. IMA J. of Mathematical Control & Information, 10:21–31, 1993.DOI
  4. [4]Dimitri Bertsekas. Nonlinear Programming, 2nd ed. Athena Scientific, 1999.
  5. [5]P. Bettiol and H. Frankowska. Normality of the maximum principle for nonconvex constrained bolza problems. Journal of Differential Equations, 2007.DOI
  6. [6]P. Bettiol and N. Khalil. Non-degenerate forms of the generalized euler–lagrange condition for state-constrained optimal control problems. Variational Methods, 2016.
  7. [7]D. Blackwell. The range of certain vector integrals. Proc. Am. Math. Soc., 2(3):390–395, 1951.DOI
  8. [8]J.M. Borwein and A.S. Lewis. Convex Analysis and Nonlinear Optimization. Theory and Examples. Springer–Verlag New York, Inc., 2000.DOI
  9. [9]Georges Bouligand. Sur quelques points de topologie restreinte du premier ordre. Bulletin de la Société Mathématique de France, 56:26–35, 1928.DOI
  10. [11]M.M.A. Ferreira, F. Fontes, and R.B. Vinter. Nondegenerate necessary conditions for nonconvex optimal control problems with state constraints. Journal of Mathematical Analysis and Applications, 233:116–129, 1999.DOI
  11. [12]H. Frankowska and D. Tonon. Inward pointing trajectories, normality of the maximum principle and the non occurrence of the lavrentieff phenomenon in optimal control under state constraints. Journal of Convex Analysis, 2013. HAL.
  12. [13]Fritz John. Extremum problems with inequalities as subsidiary conditions, Studies and essays presented to R. Courant on his 60th birthday, January 8, 1948, 1948.
  13. [14]Daniel Liberzon. Calculus of Variations and Optimal Control Theory. A Concise Introduction. Princeton University Press, 2012.
  14. [15]Boris Mordukhovich and Nguyen Mau Nam. An easy path to convex analysis and applications. Synthesis Lectures on Mathematics and Statistics. Springer, Cham, second edition, [2023] ©2023.
  15. [16]L.W. Neustadt. The existence of optimal controls in the absence of convexity conditions. J. of Mathematical Analysis and Applications, 7:110–117, 1963.DOI
  16. [17]Thomas Pecsvaradi and Kumpati S. Narendra. Reachable sets for linear dynamical systems. Information and Control, 19:319–344, 1971.DOI
  17. [18]F. Rampazzo and R.B. Vinter. A theorem on existence of neighbouring trajectories satisfying a state constraint, with applications to optimal control. IMA Journal of Mathematical Control abd Information, 16, 1999.
  18. [19]R.T. Rockafellar and R.J.-B. Wets. Variational Analysis, volume 317 of Grundlegenden der mathematischen Wissenschaften. Springer-Verlag Berlin Heidelberg, 1998.DOI
  19. [20]P.P. Varaiya. Notes on Optimization. Van Nostrand Reinhold, 1972.
  20. [21]Richard Vinter. Optimal Control. Birkhäuser, 2000.

Paper details

vlnc:W_9ure9xwz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents