Introduction
Normality of Pontryagin’s maximum principle (PMP) has been an object of much attention over the past few decades; see, e.g., [12 ] , [5 ] , [18 ] , [6 ] , [11 ] . In particular, in the presence of both endpoints constraints, sufficient conditions for normality were obtained in [12 ] that apply to a large class of optimal control problems with (or without) state constraints. However, as noted just before Theorem 5.6 in that paper, such conditions are obtained only for trajectories ending on the boundary ∂ K \partial K ∂ K of the set K K K that defines the state constraints (see (1.2) in [12 ] ); accordingly, in the absence of state constraints (there are none in the present paper), K K K is the entire space so ∂ K \partial K ∂ K is empty, and no trajectories can end in ∂ K \partial K ∂ K , so that Theorem 5.6 of [12 ] is not relevant when both endpoints are constrained.
Standard statements of PMP’s necessary conditions of optimality (see, e.g., [20 ] , [21 ] , [14 ] ) involve a constant scalar “multiplier” associated to the objective function, sometimes referred to as the abnormal multiplier—noted p 0 ∗ p_0^* p 0 ∗ in the present paper—for which, depending on the specifics of the problem at hand, the value zero may be the only possible choice, making the PMP abnormal. The intent of the present paper is to identify classes of optimal control problems for which PMP’s necessary conditions do not suffer such drawback.
With the focus first placed on the Mayer problem, after setting the background (section II), in section III, we introduce conditions (TS1 and TS2) at the terminal state (LTS1 in a Lagrange-focused context; see section IV below), which are shown (Theorem 1) to be sufficient for PMP’s conditions to hold with p 0 ∗ ≠ 0 p_0^* \ne0 p 0 ∗ = 0 (equivalently, via scaling, with p 0 ∗ = 1 p_0^*=1 p 0 ∗ = 1 ) for the problem under consideration.
The main portion of the paper (section IV) focuses on the Lagrange formulation of optimal control problems. It investigates conditions under which TS1/LTS1 and TS2 hold. (LTS1 excludes downwards directions in the closed convex hull of the augmented reachable set’s contingent cone.
It is then first shown (section IV-A) that, for rather general Lagrange problems, convexity of the lower boundary function and an interior point assumption imply LTS1, provided the terminal condition is regular within the affine hull of the reachable set. This is then shown to be the case in a class of problems studied by L.W. Neustadt [16 Theorem 1] .
Attention then shifts (section IV-B) to the class of Lagrange steering problems, specifically, problems in which the terminal state is prescribed; for such problems, Condition TS2 is always satisfied; further, when the state dynamics is linear and time-invariant (LTI), for a large class of cost integrands linear in the state, either (i) the optimal control is bang–bang or (ii) PMP is normal (Theorem 3). Finally (section IV-C), when in addition the cost integrand is time-independent and linear in state and control signals, PMP is normal as long as (Theorem 4) all eigenvalues of the state dynamics matrix A A A are real (Theorem 4)—as well as, at least in the 2 × 2 2 \times2 2 × 2 case, when the imaginary parts of the eigenvalues of A A A are appropriately small.
From section IV-A onward, the analysis is limited to the single-input case; the multi-input case is left for future studies.
Notation. It is mostly standard. Worth mentioning are the following: 0 k 0_k 0 k is the origin of R k \mathbb{R}^k R k ; given a scalar δ > 0 \delta> 0 δ > 0 , B δ ( ξ ) B_\delta(\xi) B δ ( ξ ) is the open ball of radius δ \delta δ centered at ξ \xi ξ ; given a differentiable mapping G : R p → R q G : \mathbb{R}^p \to\mathbb{R}^q G : R p → R q , J G : R p → R q × p JG : \mathbb{R}^p \to\mathbb{R}^{q \times p} J G : R p → R q × p is its Jacobian; and the sign function “sgn” takes values in { − 1 , 0 , 1 } \{-1, 0, 1\} { − 1 , 0 , 1 } , with sgn ( 0 ) : = 0 \operatorname{sgn}(0):=0 sgn ( 0 ) := 0 . Further, given v ∈ R p v \in\mathbb{R}^p v ∈ R p and w ∈ R q w \in\mathbb{R}^q w ∈ R q the Matlab notation [ v ; w ] ∈ R p + q [v; w] \in\mathbb{R}^{p+q} [ v ; w ] ∈ R p + q designates the concatenation of v v v and w w w , N ( M ) \mathcal{N}(M) N ( M ) and R ( M ) \mathcal{R}(M) R ( M ) denote the kernel (nullspace) and range of matrix M M M ; and, given a set S S S , ∂ S \partial S ∂ S denotes its boundary, co ( S ) \operatorname{co}(S) co ( S ) its convex hull, cl ( S ) \operatorname{cl}(S) cl ( S ) is closure, aff ( S ) \operatorname{aff}(S) aff ( S ) its affine hull, and relint ( S ) \operatorname{relint}(S) relint ( S ) its relative interior.
Background
Consider the Mayer problem with fixed initial and terminal times, fixed initial state, and free terminal state, viz.
min u ∈ U ψ ( x ( 1 ) ) \min_{u \in\mathcal{U}} \psi(x(1)) u ∈ U min ψ ( x ( 1 ))
s.t. x ˙ ( t ) = f ( t , x ( t ) , u ( t ) ) a.e. t ∈ [ 0 , 1 ] , \text{s.t.}\quad\dot{x}(t)=f(t,x(t),u(t))\ \text{a.e. }t\in[0,1], s.t. x ˙ ( t ) = f ( t , x ( t ) , u ( t )) a.e. t ∈ [ 0 , 1 ] ,
x abs. cont. , x ( 0 ) = ξ 0 , (1) x\ \text{abs. cont.},\quad x(0)=\xi^0, \tag*{(1)} x abs. cont. , x ( 0 ) = ξ 0 , (1)
where ψ : R n → R \psi:\mathbb{R}^n\to\mathbb{R} ψ : R n → R is continuously differentiable, f : R × R n × R m → R n f:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R}^n f : R × R n × R m → R n is (jointly) continuous, as well as locally Lipschitzian in its second argument (as is f 0 : R × R n × R m → R f_0:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R} f 0 : R × R n × R m → R , to be introduced below in the context of the Lagrange problem), ξ 0 ∈ R n \xi^0\in\mathbb{R}^n ξ 0 ∈ R n and U ⊆ R m U\subseteq\mathbb{R}^m U ⊆ R m are prescribed, and U \mathcal{U} U is the set of piecewise-continuous functions u : [ 0 , 1 ] → U u:[0,1]\to U u : [ 0 , 1 ] → U , that have at most a finite number of points of discontinuity in [ 0 , 1 ] [0,1] [ 0 , 1 ] and are right-continuous on [ 0 , 1 ) [0,1) [ 0 , 1 ) .1 Of course, given an optimal (for (1)) control u ∗ ∈ U u^*\in\mathcal{U} u ∗ ∈ U , the associated optimal state trajectory x ∗ ( ⋅ ) x^*(\cdot) x ∗ ( ⋅ ) is well defined (and continuous) on [ 0 , 1 ] [0,1] [ 0 , 1 ] .
Problem (1) is closely related to the finite-dimensional optimization problem
min ψ ( ξ ) s.t. ξ ∈ K ( ξ 0 ) , (2) \min\psi(\xi)\quad\text{s.t. }\xi\in K(\xi^0), \tag*{(2)} min ψ ( ξ ) s.t. ξ ∈ K ( ξ 0 ) , (2)
where K ( ξ 0 ) K(\xi^0) K ( ξ 0 ) is the set of states reachable at time t = 1 t=1 t = 1 when starting from state ξ 0 \xi^0 ξ 0 at t = 0 t=0 t = 0 and applying a control u ∈ U u\in\mathcal{U} u ∈ U . In particular, if ξ ∗ \xi^* ξ ∗ solves (2), every control u ∗ ∈ U u^*\in\mathcal{U} u ∗ ∈ U that steers x x x from x ( 0 ) = ξ 0 x(0)=\xi^0 x ( 0 ) = ξ 0 to x ( 1 ) = ξ ∗ x(1)=\xi^* x ( 1 ) = ξ ∗ , solves (1). Now, it is well known (e.g., [20 Chapter V, Lemma 1] ) that if ξ ∗ \xi^* ξ ∗ solves (2) then
J ψ ( ξ ∗ ) d ≥ 0 J\psi(\xi^*)d\geq0 J ψ ( ξ ∗ ) d ≥ 0
∀ d ∈ cl co C C ( ξ ∗ , K ( ξ 0 ) ) , (3) \forall d\in\operatorname{cl}\operatorname{co}CC(\xi^*,K(\xi^0)), \tag*{(3)} ∀ d ∈ cl co C C ( ξ ∗ , K ( ξ 0 )) , (3)
where C C ( ξ , S ) CC(\xi,S) C C ( ξ , S ) is GL Bouligand’s contingent cone 2 [9 section 3] to S S S at ξ ∈ S \xi\in S ξ ∈ S . For ease of reference, we recall here the definition of the contingent cone, as formulated, e.g., in [8 Theorem 6.3.6(a)] .
Definition 1: (e.g., [8 Theorem 6.3.6(a)] ) Given S ⊂ R n S\subset\mathbb{R}^n S ⊂ R n and ξ ∈ cl ( S ) \xi\in\operatorname{cl}(S) ξ ∈ cl ( S ) , the contingent cone C C ( ξ , S ) CC(\xi,S) C C ( ξ , S ) consists of those vectors d ∈ R n d\in\mathbb{R}^n d ∈ R n for which there are sequences { ϵ k } ↓ 0 \{\epsilon_k\}\downarrow0 { ϵ k } ↓ 0 in R \mathbb{R} R and { d k } → d \{d_k\}\to d { d k } → d in R n \mathbb{R}^n R n such that ξ + ϵ k d k ∈ S \xi+\epsilon_kd_k\in S ξ + ϵ k d k ∈ S for all k k k . Based on a “needle” perturbation, sensitivity analysis shows that, given any v ∈ U v\in U v ∈ U , and any τ ∈ [ 0 , 1 ) \tau\in[0,1) τ ∈ [ 0 , 1 ) at which u ∗ u^* u ∗ is continuous,
d τ , v : = Φ ( 1 , τ ) [ f ( τ , x ∗ ( τ ) , v ) − f ( τ , x ∗ ( τ ) , u ∗ ( τ ) ) ] d_{\tau,v}:=\Phi(1,\tau)\left[f(\tau,x^*(\tau),v)-f(\tau,x^*(\tau),u^*(\tau))\right] d τ , v := Φ ( 1 , τ ) [ f ( τ , x ∗ ( τ ) , v ) − f ( τ , x ∗ ( τ ) , u ∗ ( τ )) ]
∈ C C ( x ∗ ( 1 ) , K ( ξ 0 ) ) , (4) \in CC(x^*(1),K(\xi^0)), \tag*{(4)} ∈ C C ( x ∗ ( 1 ) , K ( ξ 0 )) , (4)
where x ∗ x^* x ∗ is the state trajectory associated to u ∗ u^* u ∗ and Φ : [ 0 , 1 ] × [ 0 , 1 ] → R n × n \Phi:[0,1]\times[0,1]\to\mathbb{R}^{n\times n} Φ : [ 0 , 1 ] × [ 0 , 1 ] → R n × n the state-transition operator for the linearization of f f f along ( x ∗ ( t ) , u ∗ ( t ) ) ; (x^*(t),u^*(t)); ( x ∗ ( t ) , u ∗ ( t )) ; see, e.g., [20 Exercise 1 in section VIII.1] , [14 equation (4.23)] .
The following instance of PMP conditions (see, e.g., [20 section VIII.1, Theorem 2] ), where H : R × R n × R n × R m → R H:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R} H : R × R n × R n × R m → R is the pre-Hamiltonian associated to Problem (1), given by
H ( t , ξ , π , v ) : = f ( t , ξ , v ) T π H(t,\xi,\pi,v):=f(t,\xi,v)^{\mathsf{T}}\pi H ( t , ξ , π , v ) := f ( t , ξ , v ) T π
and J ξ f : R × R n × R m → R n × R n J_\xi f:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R}^n\times\mathbb{R}^n J ξ f : R × R n × R m → R n × R n denotes the Jacobian of f f f with respect to its second argument.
Fact 1: Let u ∗ ∈ U u^*\in\mathcal{U} u ∗ ∈ U be optimal for problem (1), with associated state trajectory x ∗ x^* x ∗ . Then there exists an absolutely continuous p ∗ : R → R n p^*:\mathbb{R}\to\mathbb{R}^n p ∗ : R → R n such that
H ( t , x ∗ ( t ) , p ∗ ( t ) , u ∗ ( t ) ) = max v ∈ U H ( t , x ∗ ( t ) , p ∗ ( t ) , v ) H(t,x^*(t),p^*(t),u^*(t))=\max_{v\in U}H(t,x^*(t),p^*(t),v) H ( t , x ∗ ( t ) , p ∗ ( t ) , u ∗ ( t )) = v ∈ U max H ( t , x ∗ ( t ) , p ∗ ( t ) , v )
∀ t ∈ [ 0 , 1 ] , \forall t\in[0,1], ∀ t ∈ [ 0 , 1 ] ,
and p ∗ p^* p ∗ solves the linear, terminal-value problem
p ˙ ∗ ( t ) = − J ξ f ( t , x ∗ ( t ) , u ∗ ( t ) ) T p ∗ ( t ) \dot{p}^*(t)=-J_\xi f(t,x^*(t),u^*(t))^{\mathsf{T}}p^*(t) p ˙ ∗ ( t ) = − J ξ f ( t , x ∗ ( t ) , u ∗ ( t ) ) T p ∗ ( t )
a.e. t ∈ [ 0 , 1 ] , p ∗ ( 1 ) = − ∇ ψ ( x ∗ ( 1 ) ) . \text{a.e. }t\in[0,1],\quad p^*(1)=-\nabla\psi(x^*(1)). a.e. t ∈ [ 0 , 1 ] , p ∗ ( 1 ) = − ∇ ψ ( x ∗ ( 1 )) .
This result is easily extended to problems where x ∗ ( 0 ) x^*(0) x ∗ ( 0 ) , rather than being prescribed, is merely restricted to lie on a specified manifold, namely, g 0 ( x ∗ ( 0 ) ) = 0 m 0 g^0(x^*(0))=0_{m_0} g 0 ( x ∗ ( 0 )) = 0 m 0 , where g 0 : R n → R m 0 g^0:\mathbb{R}^n\to\mathbb{R}^{m_0} g 0 : R n → R m 0 , for some m 0 m_0 m 0 , is continuously differentiable, with Jacobian J g 0 : R n → R m 0 × n Jg^0:\mathbb{R}^n\to\mathbb{R}^{m_0\times n} J g 0 : R n → R m 0 × n . For such problem, it is readily determined that, under the condition that x ∗ ( 0 ) x^*(0) x ∗ ( 0 ) is regular for g 0 g^0 g 0 (i.e., that J g 0 ( x ∗ ( 0 ) ) Jg^0(x^*(0)) J g 0 ( x ∗ ( 0 )) is surjective), Fact 1 still holds as stated with, in addition, p ∗ ( 0 ) p^*(0) p ∗ ( 0 ) being now constrained to satisfy the “transversality” condition3
p ∗ ( 0 ) ⊥ N ( J g 0 ( x ∗ ( 0 ) ) ) . p^*(0)\perp\mathcal{N}\left(Jg^0(x^*(0))\right). p ∗ ( 0 ) ⊥ N ( J g 0 ( x ∗ ( 0 )) ) .
The case of a constrained terminal state is more challenging. Indeed, consider
min u ∈ U ψ ( x ( 1 ) ) s.t. x ˙ ( t ) = f ( t , x ( t ) , u ( t ) ) , a.e. t ∈ [ 0 , 1 ] , x abs. cont. , g 0 ( x ( 0 ) ) = 0 m 0 , g f ( x ( 1 ) ) = 0 m f . (5) \begin{aligned}
\min_{u\in\mathcal{U}}\quad&\psi(x(1))\\
\text{s.t.}\quad&\dot{x}(t)=f(t,x(t),u(t)),\quad\text{a.e. }t\in[0,1],\\
&x\ \text{abs. cont.},\quad g^0(x(0))=0_{m_0},\\
&g^f(x(1))=0_{m_f}. \tag*{(5)}
\end{aligned} u ∈ U min s.t. ψ ( x ( 1 )) x ˙ ( t ) = f ( t , x ( t ) , u ( t )) , a.e. t ∈ [ 0 , 1 ] , x abs. cont. , g 0 ( x ( 0 )) = 0 m 0 , g f ( x ( 1 )) = 0 m f . (5)
where g f : R n → R m f g^{f}: \mathbb{R}^{n} \to\mathbb{R}^{m_f} g f : R n → R m f , for some m f m_f m f , is continuously differentiable. In this case, (3) is not implied anymore, since not all of the reachable set K ( T 0 ) K(T_0) K ( T 0 ) is now allowable; here K ( T 0 ) K(T_0) K ( T 0 ) denotes the set of states reachable at t = 1 t=1 t = 1 from an initial point in
T 0 : = { ξ ∈ R n : g 0 ( ξ ) = 0 m 0 } T_0 := \{\xi\in\mathbb{R}^{n}: g^{0}(\xi)=0_{m_0}\} T 0 := { ξ ∈ R n : g 0 ( ξ ) = 0 m 0 }
at t = 0 t=0 t = 0 . Accordingly, Fact 1 is not guaranteed to hold when the terminal point is constrained. It is a well-established fact that the following augmented version of that result does hold; see, e.g., [20 section VIII.1, p. 156, Theorem 3] .
Fact 2: Let u ∗ ∈ U u^{*} \in U u ∗ ∈ U be optimal for problem (5), with associated state trajectory x ∗ x^{*} x ∗ . Assume that the initial state ξ 0 ( = x ∗ ( 0 ) ) \xi^{0}(=x^{*}(0)) ξ 0 ( = x ∗ ( 0 )) is a regular point for g 0 g^{0} g 0 and that J g f ( x ∗ ( 1 ) ) Jg^{f}(x^{*}(1)) J g f ( x ∗ ( 1 )) is surjective. Then there exist a constant p 0 ∗ ≥ 0 p_{0}^{*} \ge0 p 0 ∗ ≥ 0 and an absolutely continuous function p ∗ : [ 0 , 1 ] → R n p^{*}:[0,1]\to\mathbb{R}^{n} p ∗ : [ 0 , 1 ] → R n , with [ p 0 ∗ ; p ∗ ( ⋅ ) ] [p_{0}^{*};p^{*}(\cdot)] [ p 0 ∗ ; p ∗ ( ⋅ )] not identically zero, such that
H ( t , x ∗ ( t ) , p ∗ ( t ) , u ∗ ( t ) ) = max v ∈ U H ( t , x ∗ ( t ) , p ∗ ( t ) , v ) H(t,x^{*}(t),p^{*}(t),u^{*}(t))=\max_{v\in U}H(t,x^{*}(t),p^{*}(t),v) H ( t , x ∗ ( t ) , p ∗ ( t ) , u ∗ ( t )) = v ∈ U max H ( t , x ∗ ( t ) , p ∗ ( t ) , v )
∀ t ∈ [ 0 , 1 ) \forall t\in[0,1) ∀ t ∈ [ 0 , 1 ) ,
p ˙ ∗ ( t ) = − J x f ( t , x ∗ ( t ) , u ∗ ( t ) ) T p ∗ ( t ) a.e. t ∈ [ 0 , 1 ] (6) \dot{p}^{*}(t)=-J_{x}f(t,x^{*}(t),u^{*}(t))^{\mathsf{T}}p^{*}(t)\quad\text{a.e. }t\in[0,1] \tag*{(6)} p ˙ ∗ ( t ) = − J x f ( t , x ∗ ( t ) , u ∗ ( t ) ) T p ∗ ( t ) a.e. t ∈ [ 0 , 1 ] (6)
and the transversality conditions
p ∗ ( 0 ) ⊥ N ( J g 0 ( x ∗ ( 0 ) ) ) , p^{*}(0)\perp\mathcal{N}\bigl(Jg^{0}(x^{*}(0))\bigr), p ∗ ( 0 ) ⊥ N ( J g 0 ( x ∗ ( 0 )) ) ,
p ∗ ( 1 ) + p 0 ∗ ∇ ψ ( x ∗ ( 1 ) ) ⊥ N ( J g f ( x ∗ ( 1 ) ) ) . p^{*}(1)+p_{0}^{*}\nabla\psi(x^{*}(1))\perp\mathcal{N}\bigl(Jg^{f}(x^{*}(1))\bigr). p ∗ ( 1 ) + p 0 ∗ ∇ ψ ( x ∗ ( 1 )) ⊥ N ( J g f ( x ∗ ( 1 )) ) .
hold.
Remark 1: Clearly, in the statement of Fact 2, p 0 ∗ ∈ { 0 , 1 } p_{0}^{*}\in\{0,1\} p 0 ∗ ∈ { 0 , 1 } can be substituted for p 0 ∗ ≥ 0 p_{0}^{*}\ge0 p 0 ∗ ≥ 0 via scaling of [ p 0 ∗ , p ∗ ] [p_{0}^{*},p^{*}] [ p 0 ∗ , p ∗ ] by p 0 ∗ p_{0}^{*} p 0 ∗ when p 0 ∗ > 0 p_{0}^{*}>0 p 0 ∗ > 0 .
These necessary conditions of optimality are “weak” (a la FJ Conditions [13 ] ; also see, e.g., [4 section 3.3.5] ) in the sense that, by not guaranteeing that some nonzero p 0 ∗ p_{0}^{*} p 0 ∗ satisfies the conditions, they leave open the possibility that the conditions may hold regardless of the cost function ψ \psi ψ .
Sufficient Conditions for Normality
As noted above, in the context of Mayer’s problem (1), in the free-terminal-state case (i.e., g f ≡ 0 m f g^{f}\equiv0_{m_f} g f ≡ 0 m f or, equivalently, m f = 0 m_f=0 m f = 0 ), PMP follows from noting that u ∗ ∈ U u^{*}\in U u ∗ ∈ U is optimal if and only if the associated state trajectory x ∗ x^{*} x ∗ is such that
x ∗ ( 1 ) ∈ argmin ξ { ψ ( ξ ) s.t. ξ ∈ K ( T 0 ) } , (7) x^{*}(1)\in\operatorname*{argmin}_{\xi}\{\psi(\xi)\ \text{s.t.}\ \xi\in K(T_0)\}, \tag*{(7)} x ∗ ( 1 ) ∈ ξ argmin { ψ ( ξ ) s.t. ξ ∈ K ( T 0 )} , (7)
and a standard first order necessary condition for (7) to hold is that
J ψ ( x ∗ ( 1 ) ) d ≥ 0 J\psi(x^{*}(1))d\ge0 J ψ ( x ∗ ( 1 )) d ≥ 0
∀ d ∈ cl co C C ( x ∗ ( 1 ) , K ( T 0 ) ) \forall d\in\operatorname{cl}\operatorname{co}CC(x^{*}(1),K(T_0)) ∀ d ∈ cl co C C ( x ∗ ( 1 ) , K ( T 0 )) .
Clearly, when the terminal state is required to lie within
T f : = { ξ ∈ R n : g f ( ξ ) = 0 m f } , T_f:=\{\xi\in\mathbb{R}^{n}:g^{f}(\xi)=0_{m_f}\}, T f := { ξ ∈ R n : g f ( ξ ) = 0 m f } ,
(7) becomes
x ∗ ( 1 ) ∈ argmin ξ { ψ ( ξ ) s.t. ξ ∈ T f ∩ K ( T 0 ) } , x^{*}(1)\in\operatorname*{argmin}_{\xi}\{\psi(\xi)\ \text{s.t.}\ \xi\in T_f\cap K(T_0)\}, x ∗ ( 1 ) ∈ ξ argmin { ψ ( ξ ) s.t. ξ ∈ T f ∩ K ( T 0 )} ,
yielding the necessary condition
J ψ ( x ∗ ( 1 ) ) d ≥ 0 ∀ d ∈ D : = cl co C C ( x ∗ ( 1 ) , T f ∩ K ( T 0 ) ) , (8) J\psi(x^{*}(1))d\ge0\quad\forall d\in D:=\operatorname{cl}\operatorname{co}CC(x^{*}(1),T_f\cap K(T_0)), \tag*{(8)} J ψ ( x ∗ ( 1 )) d ≥ 0 ∀ d ∈ D := cl co C C ( x ∗ ( 1 ) , T f ∩ K ( T 0 )) , (8)
which is weaker (less constraining on x ∗ x^{*} x ∗ ) than (8), since it asserts nonnegativity of the directional derivative over a subset of the directions d d d that satisfy (8).
While (9) allows for p 0 ∗ = 0 p_{0}^{*}=0 p 0 ∗ = 0 , two conditions are now introduced which, as shown below, together guarantee the validity of p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 .
Conditions Terminal-State (TS):
TS1. J ψ ( x ∗ ( 1 ) ) d ≥ 0 ∀ d ∈ N ( J g f ( x ∗ ( 1 ) ) ) ∩ D J\psi(x^{*}(1))d\ge0\quad\forall d\in\mathcal{N}\bigl(Jg^{f}(x^{*}(1))\bigr)\cap D J ψ ( x ∗ ( 1 )) d ≥ 0 ∀ d ∈ N ( J g f ( x ∗ ( 1 )) ) ∩ D
TS2. The cone
C : = { [ J ψ ( x ∗ ( 1 ) ) J g f ( x ∗ ( 1 ) ) ] d ∣ d ∈ D } ⊆ R 1 + m f C:=\left\{\begin{bmatrix}J\psi(x^{*}(1))\\Jg^{f}(x^{*}(1))\end{bmatrix}d\mid d\in D\right\}\subseteq\mathbb{R}^{1+m_f} C := { [ J ψ ( x ∗ ( 1 )) J g f ( x ∗ ( 1 )) ] d ∣ d ∈ D } ⊆ R 1 + m f
is closed.
which is weaker (less constraining on x ∗ x^{*} x ∗ ) than (8), since it asserts nonnegativity of the directional derivative over a subset of the directions d d d that satisfy (8).
While (9) allows for p 0 ∗ = 0 p_{0}^{*}=0 p 0 ∗ = 0 , two conditions are now introduced which, as shown below, together guarantee the validity of p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 .
Conditions Terminal-State (TS):
TS1. J ψ ( x ∗ ( 1 ) ) d ≥ 0 ∀ d ∈ N ( J g f ( x ∗ ( 1 ) ) ) ∩ D J\psi(x^{*}(1))d\ge0\quad\forall d\in\mathcal{N}\bigl(Jg^{f}(x^{*}(1))\bigr)\cap D J ψ ( x ∗ ( 1 )) d ≥ 0 ∀ d ∈ N ( J g f ( x ∗ ( 1 )) ) ∩ D
TS2. The cone
C : = { [ J ψ ( x ∗ ( 1 ) ) J g f ( x ∗ ( 1 ) ) ] d ∣ d ∈ D } ⊆ R 1 + m f C:=\left\{\begin{bmatrix}J\psi(x^{*}(1))\\Jg^{f}(x^{*}(1))\end{bmatrix}d\mid d\in D\right\}\subseteq\mathbb{R}^{1+m_f} C := { [ J ψ ( x ∗ ( 1 )) J g f ( x ∗ ( 1 )) ] d ∣ d ∈ D } ⊆ R 1 + m f
is closed.
Theorem 1: Let u ∗ ∈ U u^{*}\in U u ∗ ∈ U be optimal for problem (5), with corresponding state trajectory x ∗ x^{*} x ∗ . Suppose x ∗ ( 0 ) x^{*}(0) x ∗ ( 0 ) is a regular point of g 0 g^{0} g 0 , J g f ( x ∗ ( 1 ) ) Jg^{f}(x^{*}(1)) J g f ( x ∗ ( 1 )) is surjective, and Conditions TS hold. Then the conditions in Fact 2 hold with p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 .
Proof: Write
M : = [ J ψ ( x ∗ ( 1 ) ) J g f ( x ∗ ( 1 ) ) ] , R : = [ − 1 0 m f ] . M:=\begin{bmatrix}J\psi(x^{*}(1))\\Jg^{f}(x^{*}(1))\end{bmatrix},\qquad R:=\begin{bmatrix}-1\\0_{m_f}\end{bmatrix}. M := [ J ψ ( x ∗ ( 1 )) J g f ( x ∗ ( 1 )) ] , R := [ − 1 0 m f ] .
Since x ∗ ( 1 ) x^{*}(1) x ∗ ( 1 ) minimizes ψ \psi ψ over T f ∩ K ( T 0 ) T_f\cap K(T_0) T f ∩ K ( T 0 ) , (9) holds. Then, from Condition TS1, R ∉ C R\notin C R ∈ / C ; indeed, if R ∈ C R\in C R ∈ C , then from the definition of C C C , there would exist d ∈ D d\in D d ∈ D such that J ψ ( x ∗ ( 1 ) ) d = − 1 J\psi(x^{*}(1))d=-1 J ψ ( x ∗ ( 1 )) d = − 1 and J g f ( x ∗ ( 1 ) ) d = 0 m f Jg^{f}(x^{*}(1))d=0_{m_f} J g f ( x ∗ ( 1 )) d = 0 m f , which would contradict TS1; hence, R ∉ C R\notin C R ∈ / C . Since C C C is a closed convex cone (TS2), C C C and R R R are separated by a hyperplane through the origin that does not contain R R R , i.e., there exist p 0 ∗ ∈ { 0 , 1 } p_{0}^{*}\in\{0,1\} p 0 ∗ ∈ { 0 , 1 } and π ∈ R m f \pi\in\mathbb{R}^{m_f} π ∈ R m f such that, with μ : = [ p 0 ∗ ; π ] \mu:=[p_{0}^{*};\pi] μ := [ p 0 ∗ ; π ] , we have μ T R < 0 \mu^{\mathsf{T}}R<0 μ T R < 0 (i.e., p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 ) and μ T w ≥ 0 \mu^{\mathsf{T}}w\ge0 μ T w ≥ 0 for all w ∈ C w\in C w ∈ C , i.e.,
( J ψ ( x ∗ ( 1 ) ) + π T J g f ( x ∗ ( 1 ) ) ) d ≥ 0 ∀ d ∈ D . \bigl(J\psi(x^{*}(1))+\pi^{\mathsf{T}}Jg^{f}(x^{*}(1))\bigr)d\ge0\quad\forall d\in D. ( J ψ ( x ∗ ( 1 )) + π T J g f ( x ∗ ( 1 )) ) d ≥ 0 ∀ d ∈ D .
In particular, from (4),
( J ψ ( x ∗ ( 1 ) ) + π T J g f ( x ∗ ( 1 ) ) ) ⋅ Φ ( 1 , τ ) [ f ( τ , x ∗ ( τ ) , v ) − f ( τ , x ∗ ( τ ) , u ∗ ( τ ) ) ] ≥ 0 \bigl(J\psi(x^{*}(1))+\pi^{\mathsf{T}}Jg^{f}(x^{*}(1))\bigr)
\cdot\Phi(1,\tau)\bigl[f(\tau,x^{*}(\tau),v)-f(\tau,x^{*}(\tau),u^{*}(\tau))\bigr]\ge0 ( J ψ ( x ∗ ( 1 )) + π T J g f ( x ∗ ( 1 )) ) ⋅ Φ ( 1 , τ ) [ f ( τ , x ∗ ( τ ) , v ) − f ( τ , x ∗ ( τ ) , u ∗ ( τ )) ] ≥ 0
for every v ∈ U v\in U v ∈ U , and every τ ∈ [ 0 , 1 ) \tau\in[0,1) τ ∈ [ 0 , 1 ) at which u ∗ u^{*} u ∗ is continuous. Accordingly, invoking (6) and
p ∗ ( 1 ) = − ∇ ψ ( x ∗ ( 1 ) ) − J g f ( x ∗ ( 1 ) ) T π for some π ∈ R m f p^{*}(1)=-\nabla\psi(x^{*}(1))-Jg^{f}(x^{*}(1))^{\mathsf{T}}\pi\quad\text{for some }\pi\in\mathbb{R}^{m_f} p ∗ ( 1 ) = − ∇ ψ ( x ∗ ( 1 )) − J g f ( x ∗ ( 1 ) ) T π for some π ∈ R m f
or, equivalently, the transversality condition
p ∗ ( 1 ) + ∇ ψ ( x ∗ ( 1 ) ) ⊥ N ( J g f ( x ∗ ( 1 ) ) ) , p^{*}(1)+\nabla\psi(x^{*}(1))\perp\mathcal{N}\bigl(Jg^{f}(x^{*}(1))\bigr), p ∗ ( 1 ) + ∇ ψ ( x ∗ ( 1 )) ⊥ N ( J g f ( x ∗ ( 1 )) ) ,
completes the proof. ■
Remark 2: A sufficient condition for TS2 to hold regardless of the state dynamics is that M M M be square and invertible, i.e., that m f = n − 1 m_f=n-1 m f = n − 1 and ∇ ψ ( x ∗ ( 1 ) ) ∉ R ( J g f ( x ∗ ( 1 ) ) T ) \nabla\psi(x^{*}(1))\notin\mathcal{R}(Jg^{f}(x^{*}(1))^{\mathsf{T}}) ∇ ψ ( x ∗ ( 1 )) ∈ / R ( J g f ( x ∗ ( 1 ) ) T ) . Hence if Condition TS1 holds (and J g f ( x ∗ ( 1 ) ) Jg^{f}(x^{*}(1)) J g f ( x ∗ ( 1 )) is surjective), Fact 2 will hold with p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 . (This is the case for every Lagrange steering problem, discussed in sections IV-B and IV-C below.)
The Lagrange Problem
Consider now the Lagrange problem
minimize u ∈ U ∫ 0 1 f 0 ( τ , x ( τ ) , u ( τ ) ) d τ s.t. x ˙ ( t ) = f ( t , x ( t ) , u ( t ) ) , a.e. t ∈ [ 0 , 1 ] , x abs. cont. , g 0 ( x ( 0 ) ) = 0 m 0 , g f ( x ( 1 ) ) = 0 m f . (9) \begin{aligned}
\underset{u\in U}{\operatorname{minimize}}\quad&\int_0^1 f_0(\tau,x(\tau),u(\tau))\,d\tau\\
\text{s.t.}\quad&\dot{x}(t)=f(t,x(t),u(t)),\quad\text{a.e. }t\in[0,1],\\
&x\ \text{abs. cont.},\quad g^0(x(0))=0_{m_0},\\
&g^f(x(1))=0_{m_f}. \tag*{(9)}
\end{aligned} u ∈ U minimize s.t. ∫ 0 1 f 0 ( τ , x ( τ ) , u ( τ )) d τ x ˙ ( t ) = f ( t , x ( t ) , u ( t )) , a.e. t ∈ [ 0 , 1 ] , x abs. cont. , g 0 ( x ( 0 )) = 0 m 0 , g f ( x ( 1 )) = 0 m f . (9)
where the closed set U ⊆ R m U\subseteq\mathbb{R}^m U ⊆ R m is prescribed, f 0 : R × R n × R m → R f_0:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R} f 0 : R × R n × R m → R and, as before, f : R × R n × R m → R n f:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R}^n f : R × R n × R m → R n , g 0 : R n → R m 0 g^0:\mathbb{R}^n\to\mathbb{R}^{m_0} g 0 : R n → R m 0 , and g f : R n → R m f g^f:\mathbb{R}^n\to\mathbb{R}^{m_f} g f : R n → R m f ; and where the same regularity conditions are assumed on f f f , g 0 g^0 g 0 , and g f g^f g f as was done in section II, and every condition that is assumed on f f f is also assumed on f 0 f_0 f 0 . This problem is readily transposed to the Mayer formalism,4 with augmented state-vector x ~ : = [ x 0 ; x 1 ; … ; x n ] \tilde{x}:=[x_0;x_1;\ldots;x_n] x ~ := [ x 0 ; x 1 ; … ; x n ] (with initial state x ~ ( 0 ) \tilde{x}(0) x ~ ( 0 ) set to [ 0 ; x ( 0 ) ] [0;x(0)] [ 0 ; x ( 0 )] , without loss of generality), yielding, with cost ψ ~ ( x ~ ( 1 ) ) : = x 0 ( 1 ) \tilde{\psi}(\tilde{x}(1)):=x_0(1) ψ ~ ( x ~ ( 1 )) := x 0 ( 1 ) ,
minimize u ∈ U x 0 ( 1 ) s.t. x ~ ˙ ( t ) = f ~ ( t , x ~ ( t ) , u ( t ) ) , a.e. t ∈ [ 0 , 1 ] , x ~ abs. cont. , g ~ 0 ( x ~ ( 0 ) ) = 0 1 + m 0 , g ~ f ( x ~ ( 1 ) ) = 0 1 + m f . (10) \begin{aligned}
\underset{u\in U}{\operatorname{minimize}}\quad&x_0(1)\\
\text{s.t.}\quad&\dot{\tilde{x}}(t)=\tilde{f}(t,\tilde{x}(t),u(t)),\quad\text{a.e. }t\in[0,1],\\
&\tilde{x}\ \text{abs. cont.},\\
&\tilde{g}^0(\tilde{x}(0))=0_{1+m_0},\\
&\tilde{g}^f(\tilde{x}(1))=0_{1+m_f}. \tag*{(10)}
\end{aligned} u ∈ U minimize s.t. x 0 ( 1 ) x ~ ˙ ( t ) = f ~ ( t , x ~ ( t ) , u ( t )) , a.e. t ∈ [ 0 , 1 ] , x ~ abs. cont. , g ~ 0 ( x ~ ( 0 )) = 0 1 + m 0 , g ~ f ( x ~ ( 1 )) = 0 1 + m f . (10)
which involves vector field f ~ : R × R 1 + n × R m → R 1 + n \tilde{f}:\mathbb{R}\times\mathbb{R}^{1+n}\times\mathbb{R}^m\to\mathbb{R}^{1+n} f ~ : R × R 1 + n × R m → R 1 + n defined by f ~ ( θ , ξ ~ , υ ) : = [ f 0 ( θ , ξ , υ ) ; f ( θ , ξ , υ ) ] \tilde{f}(\theta,\tilde{\xi},\upsilon):=[f_0(\theta,\xi,\upsilon);f(\theta,\xi,\upsilon)] f ~ ( θ , ξ ~ , υ ) := [ f 0 ( θ , ξ , υ ) ; f ( θ , ξ , υ )] , with associated reachable set
K ~ ( T ~ 0 ) : = { ξ ~ ′ ∈ K ~ ( ξ ~ ) : ξ ~ ∈ T ~ 0 } , T ~ 0 : = { ξ ~ : g ~ 0 ( ξ ~ ) = 0 } . \tilde{K}(\tilde{T}_0):=\{\tilde{\xi}'\in\tilde{K}(\tilde{\xi}):\tilde{\xi}\in\tilde{T}_0\},\qquad\tilde{T}_0:=\{\tilde{\xi}:\tilde{g}^0(\tilde{\xi})=0\}. K ~ ( T ~ 0 ) := { ξ ~ ′ ∈ K ~ ( ξ ~ ) : ξ ~ ∈ T ~ 0 } , T ~ 0 := { ξ ~ : g ~ 0 ( ξ ~ ) = 0 } .
and mappings g ~ 0 : R 1 + n → R 1 + m 0 \tilde{g}^0:\mathbb{R}^{1+n}\to\mathbb{R}^{1+m_0} g ~ 0 : R 1 + n → R 1 + m 0 and g ~ f : R 1 + n → R m f \tilde{g}^f:\mathbb{R}^{1+n}\to\mathbb{R}^{m_f} g ~ f : R 1 + n → R m f , defined by
g ~ 0 ( ξ ~ ) : = [ ξ 0 ; g 0 ( ξ ) ] , \tilde{g}^0(\tilde{\xi}):=[\xi_0;g^0(\xi)], g ~ 0 ( ξ ~ ) := [ ξ 0 ; g 0 ( ξ )] ,
g ~ f ( ξ ~ ) : = g f ( ξ ) . \tilde{g}^f(\tilde{\xi}):=g^f(\xi). g ~ f ( ξ ~ ) := g f ( ξ ) .
Fact 2 applied to this Mayer problem yields, after simplification, the following result (see, e.g., [20 ] ), which involves the modified pre-Hamiltonian H ~ : R × R n × R 1 + n × R m → R \tilde{H}:\mathbb{R}\times\mathbb{R}^n\times\mathbb{R}^{1+n}\times\mathbb{R}^m\to\mathbb{R} H ~ : R × R n × R 1 + n × R m → R :
H ~ ( τ , ξ , π , υ ) : = − π 0 f 0 ( τ , ξ , υ ) + π T f ( τ , ξ , υ ) , \tilde{H}(\tau,\xi,\pi,\upsilon):=-\pi_0f_0(\tau,\xi,\upsilon)+\pi^{\mathsf{T}}f(\tau,\xi,\upsilon), H ~ ( τ , ξ , π , υ ) := − π 0 f 0 ( τ , ξ , υ ) + π T f ( τ , ξ , υ ) ,
with π ~ : = [ π 0 ; π ] \tilde{\pi}:=[\pi_0;\pi] π ~ := [ π 0 ; π ] , π 0 ∈ { 0 , 1 } \pi_0\in\{0,1\} π 0 ∈ { 0 , 1 } .
Fact 3: Let u ∗ ∈ U u^*\in U u ∗ ∈ U be optimal for problem (10) with g f ( ξ ) : = ξ − ξ f g^f(\xi):=\xi-\xi^f g f ( ξ ) := ξ − ξ f , with associated state trajectory x ∗ x^* x ∗ . Then, if J g f ( x ∗ ( 1 ) ) Jg^f(x^*(1)) J g f ( x ∗ ( 1 )) is surjective, there exists an absolutely continuous p ∗ : R → R n p^*:\mathbb{R}\to\mathbb{R}^n p ∗ : R → R n such that
H ~ ( t , x ∗ ( t ) , p ~ ∗ ( t ) , u ∗ ( t ) ) = max v ∈ U H ~ ( t , x ∗ ( t ) , p ~ ∗ ( t ) , v ) \tilde{H}(t,x^*(t),\tilde{p}^*(t),u^*(t))=\max_{v\in U}\tilde{H}(t,x^*(t),\tilde{p}^*(t),v) H ~ ( t , x ∗ ( t ) , p ~ ∗ ( t ) , u ∗ ( t )) = v ∈ U max H ~ ( t , x ∗ ( t ) , p ~ ∗ ( t ) , v )
∀ t ∈ [ 0 , 1 ) \forall t\in[0,1) ∀ t ∈ [ 0 , 1 )
with constant p 0 ∗ ∈ { 0 , 1 } p_0^*\in\{0,1\} p 0 ∗ ∈ { 0 , 1 } , p ~ ∗ : = [ p 0 ∗ ; p ∗ ] \tilde{p}^*:=[p_0^*;p^*] p ~ ∗ := [ p 0 ∗ ; p ∗ ] not identically zero and satisfying the linear equation
p ˙ ∗ ( t ) = − J ξ f ( t , x ∗ ( t ) , u ∗ ( t ) ) T p ∗ ( t ) + p 0 ∗ ∇ ξ f 0 ( t , x ∗ ( t ) , u ∗ ( t ) ) \dot{p}^*(t)=-J_\xi f(t,x^*(t),u^*(t))^{\mathsf{T}}p^*(t)+p_0^*\nabla_\xi f_0(t,x^*(t),u^*(t)) p ˙ ∗ ( t ) = − J ξ f ( t , x ∗ ( t ) , u ∗ ( t ) ) T p ∗ ( t ) + p 0 ∗ ∇ ξ f 0 ( t , x ∗ ( t ) , u ∗ ( t ))
a.e. t ∈ [ 0 , 1 ) . \text{a.e. }t\in[0,1). a.e. t ∈ [ 0 , 1 ) .
as well as the transversality conditions
p ∗ ( 0 ) ⊥ N ( J g 0 ( x ∗ ( 0 ) ) ) , p^*(0)\perp\mathcal{N}(Jg^0(x^*(0))), p ∗ ( 0 ) ⊥ N ( J g 0 ( x ∗ ( 0 ))) ,
p ∗ ( 1 ) ⊥ N ( J g f ( x ∗ ( 1 ) ) ) . p^*(1)\perp\mathcal{N}(Jg^f(x^*(1))). p ∗ ( 1 ) ⊥ N ( J g f ( x ∗ ( 1 ))) .
Remark 3: Of course, in the case of a Lagrange problem, since the cost function in the associated Mayer problem is x 0 ( 1 ) x_0(1) x 0 ( 1 ) , the terminal value ξ ~ ∗ : = x ~ ∗ ( 1 ) \tilde{\xi}^*:=\tilde{x}^*(1) ξ ~ ∗ := x ~ ∗ ( 1 ) of the optimal augmented-state trajectory must lie on the “lower boundary” of K ~ ( T ~ 0 ) \tilde{K}(\tilde{T}_0) K ~ ( T ~ 0 ) , i.e., ξ ~ ∗ = [ ξ 0 ∗ ; ξ ∗ ] \tilde{\xi}^*=[\xi_0^*;\xi^*] ξ ~ ∗ = [ ξ 0 ∗ ; ξ ∗ ] must satisfy ξ 0 ∗ = min { ξ 0 : [ ξ 0 ; ξ ] ∈ K ~ ( T ~ 0 ) } \xi_0^*=\min\{\xi_0:[\xi_0;\xi]\in\tilde{K}(\tilde{T}_0)\} ξ 0 ∗ = min { ξ 0 : [ ξ 0 ; ξ ] ∈ K ~ ( T ~ 0 )} .
With T ~ 0 : = { [ 0 ; ξ 0 ] } = { ξ 0 } \tilde{T}_0:=\{[0;\xi^0]\}=\{\xi^0\} T ~ 0 := {[ 0 ; ξ 0 ]} = { ξ 0 } , we now simplify notation and define
K ( t ; ξ 0 ) : = K ( t ; T 0 ) , K ~ ( t ; ξ 0 ) : = K ~ ( t ; T ~ 0 ) ; K(t;\xi^0):=K(t;T_0),\qquad\tilde{K}(t;\xi^0):=\tilde{K}(t;\tilde{T}_0); K ( t ; ξ 0 ) := K ( t ; T 0 ) , K ~ ( t ; ξ 0 ) := K ~ ( t ; T ~ 0 ) ;
K ( ξ 0 ) : = K ( 1 ; ξ 0 ) , K ~ ( ξ 0 ) : = K ~ ( 1 ; ξ 0 ) . K(\xi^0):=K(1;\xi^0),\qquad\tilde{K}(\xi^0):=\tilde{K}(1;\xi^0). K ( ξ 0 ) := K ( 1 ; ξ 0 ) , K ~ ( ξ 0 ) := K ~ ( 1 ; ξ 0 ) .
We also define T ~ f : = { ξ ~ ∈ R 1 + n : g f ( ξ ) = 0 } \tilde{T}_f:=\{\tilde{\xi}\in\mathbb{R}^{1+n}:g^f(\xi)=0\} T ~ f := { ξ ~ ∈ R 1 + n : g f ( ξ ) = 0 } .
A simpler rewrite of Condition TS1 for the case of the Mayer reformulation of a Lagrange problem is as follows.
LTS1. cl ( K ~ ( T ~ 0 ) ) ∩ N ( J g f ( x ~ ∗ ( 1 ) ) ) ∩ { [ d 0 ; d ] ∈ R 1 + n : d 0 < 0 } = ∅ \operatorname{cl}(\tilde{K}(\tilde{T}_0))\cap\mathcal{N}(Jg^f(\tilde{x}^*(1)))\cap\{[d_0;d]\in\mathbb{R}^{1+n}:d_0<0\}=\varnothing cl ( K ~ ( T ~ 0 )) ∩ N ( J g f ( x ~ ∗ ( 1 ))) ∩ {[ d 0 ; d ] ∈ R 1 + n : d 0 < 0 } = ∅ .
Geometrically, Condition LTS1 says that the tangent plane to T ~ f \tilde{T}_f T ~ f at x ~ ∗ ( 1 ) \tilde{x}^*(1) x ~ ∗ ( 1 ) contains no “downward tangent” (vertical or otherwise) to K ~ ( T ~ 0 ) \tilde{K}(\tilde{T}_0) K ~ ( T ~ 0 ) at x ~ ∗ ( 1 ) \tilde{x}^*(1) x ~ ∗ ( 1 ) .
Remark 4: The above points to a well-known close connection between normality of the PMP and finite sensitivity of the optimal cost to small changes in the problem data (see, e.g., [2 ] , [3 ] ). Indeed, it is clear that, except for very special cases, a slight perturbation of the data can result in T ~ f \tilde{T}_f T ~ f either “moving out” of K ~ ( T ~ 0 ) \tilde{K}(\tilde{T}_0) K ~ ( T ~ 0 ) , thus making the “target” T ~ f \tilde{T}_f T ~ f locally unreachable, or “moving into” K ~ ( T ~ 0 ) \tilde{K}(\tilde{T}_0) K ~ ( T ~ 0 ) , causing its “lowest” intersection with K ~ ( T ~ 0 ) \tilde{K}(\tilde{T}_0) K ~ ( T ~ 0 ) , whose 0 0 0 th coordinate is the optimal cost x 0 ∗ ( 1 ) x_0^*(1) x 0 ∗ ( 1 ) , to make a quantum “jump,” indicating infinite sensitivity.
Next, when Condition TS2 is applied to the Lagrange problem, since J g ~ f ( x ∗ ( 1 ) ) = [ 0 J g f ( x ∗ ( 1 ) ) ] J\tilde{g}^f(x^*(1))=[0\quad Jg^f(x^*(1))] J g ~ f ( x ∗ ( 1 )) = [ 0 J g f ( x ∗ ( 1 ))] , matrix M M M defined at the top of the proof of Theorem 1 reads
M = [ 1 0 n 0 J g f ( x ∗ ( 1 ) ) ] (11) M=\begin{bmatrix}
1&0_n\\
0&Jg^f(x^*(1))
\end{bmatrix} \tag*{(11)} M = [ 1 0 0 n J g f ( x ∗ ( 1 )) ] (11)
so that the condition is satisfied if and only if, beside being surjective (as assumed in Fact 3), J g f ( x ∗ ( 1 ) ) Jg^f(x^*(1)) J g f ( x ∗ ( 1 )) also is injective, i.e., it is invertible. In particular, for such “steering” Lagrange problem, it follows from Theorem 1 that Fact 3 holds with p 0 ∗ = 1 p_0^*=1 p 0 ∗ = 1 subject only to Condition LTS1.
The next theorem plays a central role in the remainder of this paper. It involves the “lower-boundary” function ν : K ( T 0 ) → R \nu:K(T_0)\to\mathbb{R} ν : K ( T 0 ) → R defined by
ν ( ξ ) : = inf { ξ 0 : [ ξ 0 ; ξ ] ∈ K ~ ( T 0 ) } . \nu(\xi):=\inf\{\xi_0:[\xi_0;\xi]\in\tilde{K}(T_0)\}. ν ( ξ ) := inf { ξ 0 : [ ξ 0 ; ξ ] ∈ K ~ ( T 0 )} .
Theorem 2: Let x ~ ∗ : [ 0 , 1 ] → R 1 + n \tilde{x}^*:[0,1]\to\mathbb{R}^{1+n} x ~ ∗ : [ 0 , 1 ] → R 1 + n be an optimal augmented-state trajectory. Suppose that (i) x ∗ ( 1 ) ∈ relint ( K ( T 0 ) ) x^*(1)\in\operatorname{relint}(K(T_0)) x ∗ ( 1 ) ∈ relint ( K ( T 0 )) , (ii) ν \nu ν is convex in a neighborhood of x ∗ ( 1 ) x^*(1) x ∗ ( 1 ) , and (iii) J g aff ( K ( T 0 ) ) f Jg^f_{\operatorname{aff}(K(T_0))} J g aff ( K ( T 0 )) f and J g aff ( K ( T 0 ) ) f Jg^f_{\operatorname{aff}(K(T_0))} J g aff ( K ( T 0 )) f are surjective. Then LTS1 holds at x ~ ∗ ( 1 ) \tilde{x}^*(1) x ~ ∗ ( 1 ) .
Proof. Without loss of generality, reduce R n \mathbb{R}^n R n to aff ( K ( T 0 ) ) \operatorname{aff}(K(T_0)) aff ( K ( T 0 )) , so that relint ( K ( T 0 ) ) \operatorname{relint}(K(T_0)) relint ( K ( T 0 )) becomes the interior of K ( T 0 ) K(T_0) K ( T 0 ) . Since x ∗ ( 1 ) x^*(1) x ∗ ( 1 ) is an interior point of the reduced reachable set, CC ( x ∗ ( 1 ) , K ( T 0 ) ) = R n \operatorname{CC}(x^*(1),K(T_0))=\mathbb{R}^n CC ( x ∗ ( 1 ) , K ( T 0 )) = R n , so every d ∈ D d\in D d ∈ D belongs to the contingent cone. Since every convex function on an open set is locally Lipschitz [15 Corollary 2.27] , it follows from (i) and (ii) that there exists δ > 0 \delta>0 δ > 0 such that, in the reduced space, ν \nu ν has a Lipschitz constant L ∈ R L\in\mathbb{R} L ∈ R on B δ ( x ∗ ( 1 ) ) B_\delta(x^*(1)) B δ ( x ∗ ( 1 )) . Now, proceeding by contradiction, suppose that LTS1 does not hold, i.e., there exists d ~ = [ d 0 ; d ] ∈ R 1 + n \tilde{d}=[d_0;d]\in\mathbb{R}^{1+n} d ~ = [ d 0 ; d ] ∈ R 1 + n such that
(a) d 0 < 0 d_0<0 d 0 < 0 ;
(b) d ~ ∈ N ( J g ~ ( x ~ ∗ ( 1 ) ) ) \tilde{d}\in\mathcal{N}(J_{\tilde{g}}( \tilde{x}^*(1))) d ~ ∈ N ( J g ~ ( x ~ ∗ ( 1 ))) , i.e., d ∈ N ( J g ( x ∗ ( 1 ) ) ) d\in\mathcal{N}(J_g(x^*(1))) d ∈ N ( J g ( x ∗ ( 1 ))) ; and
(c) d ~ ∈ CC ( x ~ ∗ ( 1 ) , K ~ ( T 0 ) ) \tilde{d}\in\operatorname{CC}(\tilde{x}^*(1),\tilde{K}(T_0)) d ~ ∈ CC ( x ~ ∗ ( 1 ) , K ~ ( T 0 )) .
By (b) and (iii), there exists Γ : R → T f ∩ B δ ( x ∗ ( 1 ) ) \Gamma:\mathbb{R}\to T_f\cap B_\delta(x^*(1)) Γ : R → T f ∩ B δ ( x ∗ ( 1 )) , continuously differentiable, such that Γ ( 0 ) = x ∗ ( 1 ) \Gamma(0)=x^*(1) Γ ( 0 ) = x ∗ ( 1 ) and Γ ′ ( 0 ) = d \Gamma'(0)=d Γ ′ ( 0 ) = d . By (c), there exists { ϵ k } ⊂ R \{\epsilon^k\}\subset\mathbb{R} { ϵ k } ⊂ R and { o ~ k } ⊂ R 1 + n \{\tilde{o}^k\}\subset\mathbb{R}^{1+n} { o ~ k } ⊂ R 1 + n such that, as k → ∞ k\to\infty k → ∞ , ϵ k ↓ 0 \epsilon^k\downarrow0 ϵ k ↓ 0 and ∥ o ~ k ∥ ϵ k → 0 \frac{\lVert\tilde{o}^k\rVert}{\epsilon^k}\to0 ϵ k ∥ o ~ k ∥ → 0 and, for all k k k , ξ ~ k : = x ~ ∗ ( 1 ) + ϵ k d ~ + o ~ k ∈ K ~ ( T 0 ) \tilde{\xi}^k:=\tilde{x}^*(1)+\epsilon^k\tilde{d}+\tilde{o}^k\in\tilde{K}(T_0) ξ ~ k := x ~ ∗ ( 1 ) + ϵ k d ~ + o ~ k ∈ K ~ ( T 0 ) and ξ k ∈ B δ ( x ∗ ( 1 ) ) \xi^k\in B_\delta(x^*(1)) ξ k ∈ B δ ( x ∗ ( 1 )) . Since, for all k k k , ν ( Γ ( ϵ k ) ) ≥ ν ( x ∗ ( 1 ) ) \nu(\Gamma(\epsilon^k))\ge\nu(x^*(1)) ν ( Γ ( ϵ k )) ≥ ν ( x ∗ ( 1 )) (by optimality of x ∗ ( ⋅ ) x^*(\cdot) x ∗ ( ⋅ ) ) and ν ( ξ k ) ≤ ξ 0 k \nu(\xi^k)\le\xi^k_0 ν ( ξ k ) ≤ ξ 0 k (by definition of ν \nu ν ), the Lipschitz constant L L L satisfies
L ≥ ν ( Γ ( ϵ k ) ) − ν ( ξ k ) ∥ Γ ( ϵ k ) − ξ k ∥ ≥ ν ( x ∗ ( 1 ) ) − ξ 0 k ∥ Γ ( ϵ k ) − x ∗ ( 1 ) − ϵ k d − o ~ k ∥ . L\ge\frac{\nu(\Gamma(\epsilon^k))-\nu(\xi^k)}{\lVert\Gamma(\epsilon^k)-\xi^k\rVert}\ge\frac{\nu(x^*(1))-\xi^k_0}{\lVert\Gamma(\epsilon^k)-x^*(1)-\epsilon^kd-\tilde{o}^k\rVert}. L ≥ ∥ Γ ( ϵ k ) − ξ k ∥ ν ( Γ ( ϵ k )) − ν ( ξ k ) ≥ ∥ Γ ( ϵ k ) − x ∗ ( 1 ) − ϵ k d − o ~ k ∥ ν ( x ∗ ( 1 )) − ξ 0 k .
implying, from the definition of ξ ~ k \tilde{\xi}^k ξ ~ k ,
L ≥ ν ( x ∗ ( 1 ) ) − x 0 ∗ ( 1 ) − ϵ k d 0 − o 0 k ∥ Γ ( ϵ k ) − x ∗ ( 1 ) − ϵ k d − o k ∥ . L\ge\frac{\nu(x^*(1))-x^*_0(1)-\epsilon^kd_0-o^k_0}{\lVert\Gamma(\epsilon^k)-x^*(1)-\epsilon^kd-o^k\rVert}. L ≥ ∥ Γ ( ϵ k ) − x ∗ ( 1 ) − ϵ k d − o k ∥ ν ( x ∗ ( 1 )) − x 0 ∗ ( 1 ) − ϵ k d 0 − o 0 k .
and since ν ( x ∗ ( 1 ) ) = x 0 ∗ ( 1 ) \nu(x^*(1))=x^*_0(1) ν ( x ∗ ( 1 )) = x 0 ∗ ( 1 ) , dividing up and down by ϵ k \epsilon^k ϵ k ,
L ≥ − d 0 − o 0 k ϵ k ∥ Γ ( ϵ k ) − x ∗ ( 1 ) ∥ ϵ k − d − o k ϵ k = : E k ∀ k . L\ge\frac{-d_0-\frac{o^k_0}{\epsilon^k}}{\frac{\lVert\Gamma(\epsilon^k)-x^*(1)\rVert}{\epsilon^k}-d-\frac{o^k}{\epsilon^k}}=:E_k\quad\forall k. L ≥ ϵ k ∥ Γ ( ϵ k ) − x ∗ ( 1 )∥ − d − ϵ k o k − d 0 − ϵ k o 0 k =: E k ∀ k .
Since (by (a)) − d 0 > 0 -d_0>0 − d 0 > 0 and since the denominator of E k E_k E k tends to 0 + 0+ 0 + (since d = Γ ′ ( 0 ) d=\Gamma'(0) d = Γ ′ ( 0 ) ), E k → + ∞ E_k\to+\infty E k → + ∞ as k → ∞ k\to\infty k → ∞ , in contradiction to L ∈ R L\in\mathbb{R} L ∈ R being a (finite) Lipschitz constant. ■
In the next three subsections, still in the context of Lagrange problems, we successively narrow down the scope of the analysis to (i) problems with linear state dynamics , (ii) steering (to a single point) problems with LTI state dynamics , and finally (iii) steering problems with LTI state dynamics and LTI cost integrand (in the sense that f ( τ , ξ , υ ) f(\tau,\xi,\upsilon) f ( τ , ξ , υ ) does not vary with τ \tau τ and is linear in ( ξ , υ ) (\xi,\upsilon) ( ξ , υ ) . Further, we focus on the case of scalar u u u , and make the following
Assumption 1: From this point onward, the proof is specialized to the normalized symmetric control interval U = [ − 1 , 1 ] U=[-1,1] U = [ − 1 , 1 ] . This specialization is essential for the explicit formulas u = sgn ( ⋅ ) u=\operatorname{sgn}(\cdot) u = sgn ( ⋅ ) and for the perturbation identity u k − u ∗ ∈ { 0 , ± 2 } u^k-u^*\in\{0,\pm2\} u k − u ∗ ∈ { 0 , ± 2 } used below to hold.
Lagrange Problem with Linear State Dynamics
Here, we focus on vector fields f f f of the form
f ( τ , ξ , υ ) : = A ( τ ) ξ + b ( τ ) υ (12) f(\tau,\xi,\upsilon):=A(\tau)\xi+b(\tau)\upsilon \tag*{(12)} f ( τ , ξ , υ ) := A ( τ ) ξ + b ( τ ) υ (12)
for some continuous mappings A : R → R n × n A:\mathbb{R}\to\mathbb{R}^{n\times n} A : R → R n × n and b : R → R n b:\mathbb{R}\to\mathbb{R}^n b : R → R n , and we let Φ A \Phi_A Φ A be the state-transition matrix associated with A A A . We assume, without loss of generality, that x ∗ ( 0 ) = 0 n x^*(0)=0_n x ∗ ( 0 ) = 0 n . There indeed is no loss of generality in doing so since, with the change of variable x ∗ ↦ y ∗ x^*\mapsto y^* x ∗ ↦ y ∗ , with y ∗ : = x ∗ − x ∗ ( 0 ) y^*:=x^*-x^*(0) y ∗ := x ∗ − x ∗ ( 0 ) (so that y ∗ ( 0 ) = 0 n y^*(0)=0_n y ∗ ( 0 ) = 0 n ), y ∗ − x ∗ y^*-x^* y ∗ − x ∗ is independent of u u u (hence the change of variable amounts to a mere translation of K ( x ∗ ( 0 ) ) K(x^*(0)) K ( x ∗ ( 0 )) ): we have
y ˙ ∗ ( t ) = x ˙ ∗ ( t ) = A ( t ) x ∗ ( t ) + b ( t ) u ( t ) \dot{y}^*(t)=\dot{x}^*(t)=A(t)x^*(t)+b(t)u(t) y ˙ ∗ ( t ) = x ˙ ∗ ( t ) = A ( t ) x ∗ ( t ) + b ( t ) u ( t )
y ˙ ∗ ( t ) = A ( t ) y ∗ ( t ) + A ( t ) x ∗ ( 0 ) + b ( t ) u ( t ) , \phantom{\dot{y}^*(t)}=A(t)y^*(t)+A(t)x^*(0)+b(t)u(t), y ˙ ∗ ( t ) = A ( t ) y ∗ ( t ) + A ( t ) x ∗ ( 0 ) + b ( t ) u ( t ) ,
yielding, since y ∗ ( 0 ) = 0 n y^*(0)=0_n y ∗ ( 0 ) = 0 n ,
y ∗ ( t ) = ∫ 0 t Φ A ( t , τ ) ( A ( τ ) x ∗ ( 0 ) + b ( τ ) u ( τ ) ) d τ y^*(t)=\int_0^t\Phi_A(t,\tau)\left(A(\tau)x^*(0)+b(\tau)u(\tau)\right)\,d\tau y ∗ ( t ) = ∫ 0 t Φ A ( t , τ ) ( A ( τ ) x ∗ ( 0 ) + b ( τ ) u ( τ ) ) d τ
y ∗ ( t ) = ∫ 0 t Φ A ( t , τ ) b ( τ ) u ( τ ) d τ + ∫ 0 t Φ A ( t , τ ) A ( τ ) x ∗ ( 0 ) d τ , \phantom{y^*(t)}=\int_0^t\Phi_A(t,\tau)b(\tau)u(\tau)\,d\tau
+\int_0^t\Phi_A(t,\tau)A(\tau)x^*(0)\,d\tau, y ∗ ( t ) = ∫ 0 t Φ A ( t , τ ) b ( τ ) u ( τ ) d τ + ∫ 0 t Φ A ( t , τ ) A ( τ ) x ∗ ( 0 ) d τ ,
and, since x ∗ ( t ) = Φ A ( t , 0 ) x ∗ ( 0 ) + ∫ 0 t Φ A ( t , τ ) b ( τ ) u ( τ ) d τ x^*(t)=\Phi_A(t,0)x^*(0)+\int_0^t\Phi_A(t,\tau)b(\tau)u(\tau)\,d\tau x ∗ ( t ) = Φ A ( t , 0 ) x ∗ ( 0 ) + ∫ 0 t Φ A ( t , τ ) b ( τ ) u ( τ ) d τ , we get
y ∗ ( t ) − x ∗ ( t ) = Φ A ( t , 0 ) x ∗ ( 0 ) + ∫ 0 t Φ A ( t , τ ) A ( τ ) x ∗ ( 0 ) d τ , y^*(t)-x^*(t)=\Phi_A(t,0)x^*(0)
+\int_0^t\Phi_A(t,\tau)A(\tau)x^*(0)\,d\tau, y ∗ ( t ) − x ∗ ( t ) = Φ A ( t , 0 ) x ∗ ( 0 ) + ∫ 0 t Φ A ( t , τ ) A ( τ ) x ∗ ( 0 ) d τ ,
which is independent of u u u , as claimed.
Accordingly, for the sake of notational simplicity, we now define
K : = K ( 0 n ) , K ~ : = K ~ ( 0 n ) . K:=K(0_n),\qquad\widetilde{K}:=\widetilde{K}(0_n). K := K ( 0 n ) , K := K ( 0 n ) .
Finally, given u ∈ U u\in U u ∈ U , we denote by x ~ ( t ; u ) \tilde{x}(t;u) x ~ ( t ; u ) the augmented state at time t t t , starting from x ~ ( 0 ) = 0 1 + n \tilde{x}(0)=0_{1+n} x ~ ( 0 ) = 0 1 + n at time t = 0 t=0 t = 0 , and applying control u u u . In particular, for u ∈ U u\in U u ∈ U ,
x ( 1 ; u ) = ∫ 0 1 φ ( t ) u ( t ) d t (13) x(1;u)=\int_0^1\varphi(t)u(t)\,dt \tag*{(13)} x ( 1 ; u ) = ∫ 0 1 φ ( t ) u ( t ) d t (13)
where φ : R → R n \varphi:\mathbb{R}\to\mathbb{R}^n φ : R → R n is defined by
φ ( t ) : = Φ A ( 1 , t ) b ( t ) . \varphi(t):=\Phi_A(1,t)b(t). φ ( t ) := Φ A ( 1 , t ) b ( t ) .
Now, as an application of Theorem 2 consider the case with f f f given by (12) and
f 0 ( τ , ξ , υ ) : = a 0 ( τ ) T ξ + h 0 ( τ , υ ) , (14) f_0(\tau,\xi,\upsilon):=a_0(\tau)^\mathsf{T}\xi+h_0(\tau,\upsilon), \tag*{(14)} f 0 ( τ , ξ , υ ) := a 0 ( τ ) T ξ + h 0 ( τ , υ ) , (14)
where h 0 : R × R → R h_0:\mathbb{R}\times\mathbb{R}\to\mathbb{R} h 0 : R × R → R and a 0 : R → R n a_0:\mathbb{R}\to\mathbb{R}^n a 0 : R → R n are continuous (as required by our assumption on f 0 f_0 f 0 ).
The following 1963 result is due to L.W. Neustadt [16 Theorem 1] ,\5 where a 1951 result by D. Blackwell [7 ] plays a major role; also see [17 Theorem 1] .
Lemma 1: Given dynamics (12) and cost integrand (14) with h : R × R → R h:\mathbb{R}\times\mathbb{R}\to\mathbb{R} h : R × R → R continuous, if U U U is compact, then K K K and K ~ \widetilde{K} K are convex and compact.
It follows that, under the same assumptions, ν \nu ν is convex and hence Theorem 2 applies, i.e., LTS1 holds whenever x ∗ ( 1 ) ∈ relint K ( T 0 ) x^*(1)\in\operatorname{relint}K(T_0) x ∗ ( 1 ) ∈ relint K ( T 0 ) and J g ′ ( x ∗ ( 1 ) ) J_{g'}(x^*(1)) J g ′ ( x ∗ ( 1 )) is surjective.
Lagrange Steering Problem with LTI state dynamics
We now focus on the class of Lagrange problems with g f ( ξ ) : = ξ − ξ f g^{f}(\xi):=\xi-\xi^{f} g f ( ξ ) := ξ − ξ f (and w.l.o.g. x ∗ ( 0 ) = 0 x^{*}(0)=0 x ∗ ( 0 ) = 0 : see section IV-A), where ξ f ∈ K \xi^{f}\in K ξ f ∈ K is prescribed, which we refer to as the Lagrange steering problem. In such case, J g f ( ξ ) = I n J^{g^{f}}(\xi)=I_{n} J g f ( ξ ) = I n , M = I 1 + n M=I_{1+n} M = I 1 + n (see (12)), and TS2 is always satisfied (see Remark 2). From here on, time-invariance is also assumed, i.e., the state dynamics are now
f ( ξ , υ ) = A ξ + b υ . (15) f(\xi,\upsilon)=A\xi+b\upsilon. \tag*{(15)} f ( ξ , υ ) = A ξ + b υ . (15)
In such a case, the overall picture simplifies, as noted next.
Lemma 2: A (necessary and) sufficient condition for LTS1 to hold for a given pair ( A , b ) (A,b) ( A , b ) is that these conditions hold for the associated controllable subsystem ( A ′ , b ′ ) (A^{\prime},b^{\prime}) ( A ′ , b ′ ) , with A ′ A^{\prime} A ′ in real Jordan canonical form.
Proof: First note that, if ( A , b ) (A,b) ( A , b ) is not controllable, then K ~ \widetilde{K} K entirely lies in a proper subspace of R 1 + n \mathbb{R}^{1+n} R 1 + n whose “vertical” projection is the controllable subspace of x ˙ = A x + b υ \dot{x}=Ax+b\upsilon x ˙ = A x + b υ . An associated controllable reduced version of ( A , b ) (A,b) ( A , b ) is then readily obtained as the controllable portion ( A c , b c ) (A_{c},b_{c}) ( A c , b c ) of the Kalman decomposition of ( A , b ) (A,b) ( A , b ) (obtained via a change of basis). Next, via an additional change of basis (which of course also transforms b b b ), A A A can be brought to real Jordan canonical form. (In such form, diagonal blocks associated with real eigenvalues are real scalars (the real eigenvalues), while for non-real eigenvalues α + j ω \alpha+\mathrm{j}\omega α + j ω (with α \alpha α and ω \omega ω real, ω ≠ 0 \omega\ne0 ω = 0 ) they take the form [ α ω − ω α ] \begin{bmatrix}\alpha&\omega\\-\omega&\alpha\end{bmatrix} [ α − ω ω α ] ; off-diagonal blocks are all zero except for occasional super-diagonal 1 1 1 s squeezed between the scalar blocks associated to two equal real eigenvalues, and occasional super-diagonal I 2 I_{2} I 2 s squeezed between two 2 × 2 2\times2 2 × 2 diagonal blocks associated to equal non-real eigenvalues.) Since the uncontrollable portion does not affect the reachable set K K K , nor indeed does it affect whether or not LTS1 holds, the proof is complete.
Note that, with A ′ A^{\prime} A ′ in real Jordan canonical form, controllability of ( A ′ , b ′ ) (A^{\prime},b^{\prime}) ( A ′ , b ′ ) is equivalent to the pair of conditions: (i) b i ≠ 0 b_{i}\ne0 b i = 0 when row i i i of A A A is the last row of a Jordan block associated with a real eigenvalue and (ii) [ b i b i − 1 ] ≠ 0 2 \begin{bmatrix}b_{i}\\b_{i-1}\end{bmatrix}\ne0_{2} [ b i b i − 1 ] = 0 2 when row i i i of A A A is the last row of a Jordan block associated with a non-real eigenvalue. (Indeed, the pair ( [ α ω − ω α ] , [ b b ′ ] ) \left(\begin{bmatrix}\alpha&\omega\\-\omega&\alpha\end{bmatrix},\begin{bmatrix}b\\b^{\prime}\end{bmatrix}\right) ( [ α − ω ω α ] , [ b b ′ ] ) , with ω ≠ 0 \omega\ne0 ω = 0 , is controllable if and only if [ b b ′ ] ≠ 0 2 \begin{bmatrix}b\\b^{\prime}\end{bmatrix}\ne0_{2} [ b b ′ ] = 0 2 .)
Next, recall that the concept of bang–bang control plays a key role in optimal control problems with LTI state dynamics. In the present context (u ( t ) ∈ R ∀ t u(t)\in\mathbb{R}\ \forall t u ( t ) ∈ R ∀ t , U U U a compact interval of R \mathbb{R} R ) the following definition is appropriate.
Definition 2: A control signal u ∈ U u\in\mathcal{U} u ∈ U is said to be bang–bang if u ( t ) ∈ { U m , U M } ∀ t ∈ ( 0 , 1 ] u(t)\in\{U_{m},U_{M}\}\ \forall t\in(0,1] u ( t ) ∈ { U m , U M } ∀ t ∈ ( 0 , 1 ] .
Part of the following result is invoked in the proof of Theorem 3 below, and all of it in that of Theorem 4 in the next subsection.
Lemma 3: Consider the (controllable, as per Lemma 2) Lagrange steering problem with LTI state dynamics, with all eigenvalues of A A A in (16) being real, and let U : = [ u m , u M ] U:=[u_{m},u_{M}] U := [ u m , u M ] . Then, given any ξ ∈ ∂ K \xi\in\partial K ξ ∈ ∂ K , there exists a (a.e.) unique u ξ ∈ U u_{\xi}\in\mathcal{U} u ξ ∈ U such that x ( 1 ; u ξ ) = ξ x(1;u_{\xi})=\xi x ( 1 ; u ξ ) = ξ . Further, u ξ u_{\xi} u ξ is bang–bang with at most n − 1 n-1 n − 1 sign changes.
Proof: Let ξ ∈ ∂ K \xi\in\partial K ξ ∈ ∂ K and let γ ∈ R n ∖ { 0 } \gamma\in\mathbb{R}^{n}\setminus\{0\} γ ∈ R n ∖ { 0 } be an outward normal to a supporting hyperplane to K K K at ξ \xi ξ (Lemma 1). A control u ξ ∈ U u_{\xi}\in\mathcal{U} u ξ ∈ U is such that x ( 1 ; u ξ ) = ξ x(1;u_{\xi})=\xi x ( 1 ; u ξ ) = ξ if and only if
γ T ξ = max η ∈ K γ T η = max u ∈ U γ T x ( 1 ; u ) = γ T x ( 1 ; u ξ ) , \gamma^{\mathsf{T}}\xi=\max_{\eta\in K}\gamma^{\mathsf{T}}\eta
=\max_{u\in\mathcal{U}}\gamma^{\mathsf{T}}x(1;u)
=\gamma^{\mathsf{T}}x(1;u_{\xi}), γ T ξ = η ∈ K max γ T η = u ∈ U max γ T x ( 1 ; u ) = γ T x ( 1 ; u ξ ) ,
i.e., if and only if
u ξ ∈ arg max u ∈ U γ T x ( 1 ; u ) = arg max u ∈ U ∫ 0 1 γ T φ ( t ) u ( t ) d t u_{\xi}\in\operatorname*{arg\,max}_{u\in\mathcal{U}}\gamma^{\mathsf{T}}x(1;u)
=\operatorname*{arg\,max}_{u\in\mathcal{U}}\int_{0}^{1}\gamma^{\mathsf{T}}\varphi(t)u(t)\,\mathrm{d}t u ξ ∈ u ∈ U arg max γ T x ( 1 ; u ) = u ∈ U arg max ∫ 0 1 γ T φ ( t ) u ( t ) d t
i.e., if and only if u ξ u_{\xi} u ξ satisfies
u ξ ( t ) = sgn ( γ T φ ( t ) ) a.e. t ∈ [ 0 , 1 ] . u_{\xi}(t)=\operatorname{sgn}(\gamma^{\mathsf{T}}\varphi(t))\quad\text{a.e. }t\in[0,1]. u ξ ( t ) = sgn ( γ T φ ( t )) a.e. t ∈ [ 0 , 1 ] .
Now, γ T φ ( = γ T e ( 1 − ⋅ ) A b ) \gamma^{\mathsf{T}}\varphi(= \gamma^{\mathsf{T}}e^{(1-\cdot)A}b) γ T φ ( = γ T e ( 1 −⋅ ) A b ) , as an exponential polynomial of order n n n with real exponents, vanishes no more than n − 1 n-1 n − 1 times in R \mathbb{R} R (see, e.g., [1 ] ) and in particular no more than n − 1 n-1 n − 1 times in [ 0 , 1 ] [0,1] [ 0 , 1 ] . The claim follows.
Now, let u ∗ ∈ U u^{*}\in\mathcal{U} u ∗ ∈ U solve (10) with g f ( ξ ) : = ξ − ξ f g^{f}(\xi):=\xi-\xi^{f} g f ( ξ ) := ξ − ξ f and let x ~ ∗ \widetilde{x}^{*} x ∗ be the associated augmented-state trajectory. We have
N ( J g f ( x ~ ∗ ( 1 ) ) ) = { [ d 0 ; 0 n ] : d 0 ∈ R } , \mathcal{N}(J^{g^{f}}(\widetilde{x}^{*}(1)))=\{[d_{0};0_{n}]:d_{0}\in\mathbb{R}\}, N ( J g f ( x ∗ ( 1 ))) = {[ d 0 ; 0 n ] : d 0 ∈ R } ,
and Condition LTS1 reduces to
CC ( x ~ ∗ ( 1 ) , K ~ ) ∩ { [ d 0 ; 0 n ] } ∩ { [ d 0 ; d ] ∈ R 1 + n : d 0 < 0 } = ∅ \operatorname{CC}(\widetilde{x}^{*}(1),\widetilde{K})\cap\{[d_{0};0_{n}]\}\cap\{[d_{0};d]\in\mathbb{R}^{1+n}:d_{0}<0\}=\varnothing CC ( x ∗ ( 1 ) , K ) ∩ {[ d 0 ; 0 n ]} ∩ {[ d 0 ; d ] ∈ R 1 + n : d 0 < 0 } = ∅
so that LTS1 holds at x ~ ∗ ( 1 ) \widetilde{x}^{*}(1) x ∗ ( 1 ) if and only if every vertical “tangent” d ~ ∈ CC ( x ~ ∗ ( 1 ) , K ~ ) \widetilde{d}\in\operatorname{CC}(\widetilde{x}^{*}(1),\widetilde{K}) d ∈ CC ( x ∗ ( 1 ) , K ) is upward. In view of Theorem 1 and of Lemma 3, the hypotheses in Theorem 2 reduce to ξ f ∈ int ( K ) \xi^{f}\in\operatorname{int}(K) ξ f ∈ int ( K ) and convexity of ν \nu ν . We obtain the following.
Theorem 3: Let u ∗ ∈ U u^{*}\in\mathcal{U} u ∗ ∈ U be optimal for Lagrange steering problem (10) with g f ( ξ ) = ξ − ξ f g^{f}(\xi)=\xi-\xi^{f} g f ( ξ ) = ξ − ξ f , with LTI state dynamics f ( ξ , υ ) = A ξ + b υ f(\xi,\upsilon)=A\xi+b\upsilon f ( ξ , υ ) = A ξ + b υ , and with cost integrand f 0 ( τ , ξ , υ ) = a 0 T ξ + h ( υ ) f_{0}(\tau,\xi,\upsilon)=a_{0}^{\mathsf{T}}\xi+h(\upsilon) f 0 ( τ , ξ , υ ) = a 0 T ξ + h ( υ ) i.e.,
min u ∈ U ∫ 0 1 ( a 0 T x ( τ ) + h ( u ( τ ) ) ) d τ \min_{u\in\mathcal{U}}\int_{0}^{1}\left(a_{0}^{\mathsf{T}}x(\tau)+h(u(\tau))\right)\,\mathrm{d}\tau u ∈ U min ∫ 0 1 ( a 0 T x ( τ ) + h ( u ( τ )) ) d τ where h h h satisfies the assumptions in Lemma 1. Then, either (i) u ∗ u^{*} u ∗ is bang–bang or (ii) Fact 3 holds with p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 (or both).
Proof: If ξ f ∈ ∂ K \xi^{f}\in\partial K ξ f ∈ ∂ K , it follows from Lemma 3 that u ∗ u^{*} u ∗ is bang–bang, while if ξ f ∈ relint ( K ) \xi^{f}\in\operatorname{relint}(K) ξ f ∈ relint ( K ) , it follows from Remark 2 and Theorem 2 that Fact 3 holds with $p_{0}^{*}=1.
The following simple minimum-energy problem of dimension n = 2 n=2 n = 2 shows that there are cases for which option (ii) does not hold (i.e., option (i) cannot be taken out):
min u ∈ U ∫ 0 1 1 2 u ( t ) 2 d t \min_{u\in\mathcal{U}}\int_{0}^{1}\frac{1}{2}u(t)^{2}\,\mathrm{d}t u ∈ U min ∫ 0 1 2 1 u ( t ) 2 d t
subject to
x ˙ = diag ( α 1 , α 2 ) x + b u , x ( 0 ) = 0 2 , x ( 1 ; u ) = x f , u ∈ U . \dot{x}=\operatorname{diag}(\alpha_{1},\alpha_{2})x+bu,\qquad
x(0)=0_{2},\quad x(1;u)=x^{f},\qquad
u\in\mathcal{U}. x ˙ = diag ( α 1 , α 2 ) x + b u , x ( 0 ) = 0 2 , x ( 1 ; u ) = x f , u ∈ U .
Let K K K be the associated reachable set at time t = 1 t=1 t = 1 and let ξ f ∈ ∂ K \xi^{f}\in\partial K ξ f ∈ ∂ K . It follows from Lemma 3 that every u ∈ U u\in\mathcal{U} u ∈ U such that x ( 1 ; u ) = ξ f x(1;u)=\xi^{\mathrm f} x ( 1 ; u ) = ξ f is bang–bang. In particular, every optimal control is bang–bang. We now show that p 0 ∗ = 0 p_0^*=0 p 0 ∗ = 0 indeed. Pre-Hamiltonian H ~ \widetilde H H reads (see Fact 3):
H ~ ( ξ , π ~ , u ~ ) = − 1 2 π 0 u ~ 2 + π 1 b 1 u ~ + π 2 b 2 u ~ \widetilde H(\xi,\widetilde\pi,\widetilde u)=-\frac{1}{2}\pi_0\widetilde u^2+\pi_1b_1\widetilde u+\pi_2b_2\widetilde u H ( ξ , π , u ) = − 2 1 π 0 u 2 + π 1 b 1 u + π 2 b 2 u
and setting its v v v derivative to zero yields π 0 v = π 1 b 1 + π 2 b 2 \pi_0v=\pi_1b_1+\pi_2b_2 π 0 v = π 1 b 1 + π 2 b 2 . Suppose now that p 0 ∗ = 1 p_0^*=1 p 0 ∗ = 1 . Then, if u ∗ u^* u ∗ is an optimal control, we must have
u ∗ ( t ) = sat [ − 1 , 1 ] ( b 1 p 1 ∗ ( t ) + b 2 p 2 ∗ ( t ) ) u^*(t)=\operatorname{sat}_{[-1,1]}\left(b_1p_1^*(t)+b_2p_2^*(t)\right) u ∗ ( t ) = sat [ − 1 , 1 ] ( b 1 p 1 ∗ ( t ) + b 2 p 2 ∗ ( t ) )
and p ∗ p^* p ∗ must satisfy
p ˙ i ∗ ( t ) = − α i p i ∗ ( t ) i = 1 , 2 , \dot p_i^*(t)=-\alpha_i p_i^*(t)\quad i=1,2, p ˙ i ∗ ( t ) = − α i p i ∗ ( t ) i = 1 , 2 ,
yielding (when α 1 ≠ 0 ≠ α 2 \alpha_1\ne0\ne\alpha_2 α 1 = 0 = α 2 )
u ∗ ( t ) = sat [ − 1 , 1 ] ( ∑ i = 1 2 b i p i ∗ ( 0 ) e − α i t ) . u^*(t)=\operatorname{sat}_{[-1,1]}\left(\sum_{i=1}^{2}b_ip_i^*(0)e^{-\alpha_i t}\right). u ∗ ( t ) = sat [ − 1 , 1 ] ( i = 1 ∑ 2 b i p i ∗ ( 0 ) e − α i t ) .
Clearly, for any given unequal nonzero α i \alpha_i α i s and any a 0 i a_{0i} a 0 i , i = 1 , 2 i=1,2 i = 1 , 2 , and any p ∗ ( 0 ) ≠ 0 n p^*(0)\ne0_n p ∗ ( 0 ) = 0 n , there exists b ∈ R 2 b\in\mathbb{R}^2 b ∈ R 2 such that the (continuous) argument of “sat” changes sign at some t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) , resulting in u ∗ u^* u ∗ not being bang–bang. (It is readily checked that the same conclusion holds when α 1 α 2 = 0 \alpha_1\alpha_2=0 α 1 α 2 = 0 .) With such b b b , Fact 3 holds only with p 0 ∗ = 0 p_0^*=0 p 0 ∗ = 0 , as claimed.
In view of this negative result, in the remainder of this paper we limit the study of the case ξ f ∈ ∂ K \xi^{\mathrm f}\in\partial K ξ f ∈ ∂ K to Lagrange steering problems with LTI dynamics and LTI cost integrand.
Lagrange steering problem with LTI state dynamics, LTI cost integrand
It is known (e.g., [5 p.258, l. –17 to –15] ) that, in the absence of pointwise constraints on the state trajectory (no such constraints are present in the present paper), under “calmness” conditions (see, e.g., [10, 3rd equation in section 4.2; section 6.4]), the Pontryagin Principle is normal indeed (i.e., p 0 ∗ ≠ 0 p_0^*\ne0 p 0 ∗ = 0 ). In this section, after preliminary results are established, a direct proof is given that, in the present context, regardless of calmness considerations, when all coefficients are real as are the eigenvalues of the state-dynamics matrix A A A , normality holds; further it still holds in some cases with non-real eigenvalues (e.g., in the 2 × 2 2\times2 2 × 2 case when eigenvalues have “small enough” imaginary parts). See sections IV-C.1 and IV-C.2.
Thus, consider a Lagrange steering problem where both state dynamics and cost integrand are LTI, viz.,
f ( ξ , v ) = A ξ + b v , f 0 ( ξ , v ) = a 0 T ξ + b 0 v , (16) f(\xi,v)=A\xi+bv,\qquad f_0(\xi,v)=a_0^{\mathsf T}\xi+b_0v, \tag*{(16)} f ( ξ , v ) = A ξ + b v , f 0 ( ξ , v ) = a 0 T ξ + b 0 v , (16)
where A A A is square with real entries, a 0 ∈ R n a_0\in\mathbb{R}^n a 0 ∈ R n and b 0 ∈ R b_0\in\mathbb{R} b 0 ∈ R . For such problem, the restriction ξ f ∈ int ( K ) \xi^{\mathrm f}\in\operatorname{int}(K) ξ f ∈ int ( K ) in the statement of Theorem 2 can be relaxed to ξ f ∈ K \xi^{\mathrm f}\in K ξ f ∈ K in many important cases.
Specifically, since ξ f ∈ K \xi^{\mathrm f}\in K ξ f ∈ K , we now explore the case when ξ f ∈ ∂ K \xi^{\mathrm f}\in\partial K ξ f ∈ ∂ K . (In view of Lemma 1, ∂ K ⊆ K \partial K\subseteq K ∂ K ⊆ K .) Given that x ~ ( 0 ) = 0 1 + n \widetilde x(0)=0_{1+n} x ( 0 ) = 0 1 + n ,
x ~ ( 1 ; u ) = ∫ 0 1 φ ~ ( τ ) u ( τ ) d τ , with φ ~ ( t ) : = e ( 1 − t ) A ~ b ~ , \widetilde x(1;u)=\int_0^1\widetilde\varphi(\tau)u(\tau)\,d\tau,\qquad\text{with }\widetilde\varphi(t):=e^{(1-t)\widetilde A}\widetilde b, x ( 1 ; u ) = ∫ 0 1 φ ( τ ) u ( τ ) d τ , with φ ( t ) := e ( 1 − t ) A b ,
where A ~ \widetilde A A is the augmented-state-dynamics matrix. The next lemma is identical to Lemma 3, with state space R 1 + n \mathbb{R}^{1+n} R 1 + n (including cost integrand) instead of R n \mathbb{R}^n R n (see inserted tilde characters).
Lemma 4. Assuming that, in addition to ( A , b ) (A,b) ( A , b ) being controllable (no loss of generality given Lemma 2), the augmented system (17) is controllable and all eigenvalues of A A A are real, consider the Lagrange steering problem with LTI state dynamics and LTI cost integrand. Then, given any ξ ~ ∈ ∂ K ~ \widetilde\xi\in\partial\widetilde K ξ ∈ ∂ K , there exists a unique (up to equality a.e.) u ξ ~ ∈ U u_{\widetilde\xi}\in\mathcal U u ξ ∈ U such that x ~ ( 1 ; u ξ ~ ) = ξ ~ \widetilde x(1;u_{\widetilde\xi})=\widetilde\xi x ( 1 ; u ξ ) = ξ . Further, u ξ ~ u_{\widetilde\xi} u ξ is bang–bang with at most n n n sign changes.
Theorem 4. With the same assumptions as in Lemma 4, consider the (controllable, as per Lemma 2) Lagrange steering problem with LTI state dynamics, LTI cost integrand, and ξ f ∈ ∂ K \xi^{\mathrm f}\in\partial K ξ f ∈ ∂ K . Let u ∗ ∈ U u^*\in\mathcal U u ∗ ∈ U be optimal, hence (see Lemma 4) bang–bang with t ˉ ∗ ≤ n \bar t^*\le n t ˉ ∗ ≤ n sign changes, and let { t 1 ∗ , … , t t ˉ ∗ ∗ } ⊂ [ 0 , 1 ] \{t_1^*,\ldots,t_{\bar t^*}^*\}\subset[0,1] { t 1 ∗ , … , t t ˉ ∗ ∗ } ⊂ [ 0 , 1 ] be the times at which u ∗ u^* u ∗ changes sign. Suppose that, given (i) any I , I c ⊂ N I,I^c\subset\mathbb N I , I c ⊂ N , finite index sets, with I ∩ I c = ∅ I\cap I^c=\varnothing I ∩ I c = ∅ , I ∪ I c ≠ ∅ I\cup I^c\ne\varnothing I ∪ I c = ∅ , and # I c + 3 ≠ # I ≤ n \#I^c+3\ne\#I\le n # I c + 3 = # I ≤ n , and (ii) any { t i } i ∈ I ⊂ { t 1 ∗ , … , t t ˉ ∗ ∗ } ⊂ [ 0 , 1 ] ∖ { t i } i ∈ I c \{t_i\}_{i\in I}\subset\{t_1^*,\ldots,t_{\bar t^*}^*\}\subset[0,1]\setminus\{t_i\}_{i\in I^c} { t i } i ∈ I ⊂ { t 1 ∗ , … , t t ˉ ∗ ∗ } ⊂ [ 0 , 1 ] ∖ { t i } i ∈ I c , with { t 1 ∗ , … , t t ˉ ∗ ∗ } ⊂ { t i } i ∈ I ∪ I c \{t_1^*,\ldots,t_{\bar t^*}^*\}\subset\{t_i\}_{i\in I\cup I^c} { t 1 ∗ , … , t t ˉ ∗ ∗ } ⊂ { t i } i ∈ I ∪ I c , it holds that the matrix
Φ : = [ ( φ ( t i ) ) i ∈ I c ( J φ ( t i ) ) i ∈ I ( φ ( t i ) ) i ∈ I ] \Phi:=\left[(\varphi(t_i))_{i\in I^c}\;(\mathcal J\varphi(t_i))_{i\in I}\;(\varphi(t_i))_{i\in I}\right] Φ := [ ( φ ( t i ) ) i ∈ I c ( J φ ( t i ) ) i ∈ I ( φ ( t i ) ) i ∈ I ] has full column rank. Then Condition LTS1 holds.
Proof. Let x ~ ∗ \widetilde x^* x ∗ be the augmented-state-trajectory associated to u ∗ u^* u ∗ . Proceeding by contraposition, suppose that LTS1 does not hold. We proceed to construct an instance where conditions (i) and (ii) are satisfied but Φ \Phi Φ does not have full column rank, completing the proof.
Given that LTS1 does not hold, since N ( J ~ x ~ ∗ ( 1 ) ) = { ξ ~ ∈ R 1 + n : ξ ~ = 0 n } \mathcal N(\widetilde{\mathcal J}\widetilde x^*(1))=\{\widetilde\xi\in\mathbb R^{1+n}:\widetilde\xi=0_n\} N ( J x ∗ ( 1 )) = { ξ ∈ R 1 + n : ξ = 0 n } , it must hold that [ − 1 ; 0 n ] ∈ CC ( x ~ ∗ ( 1 ) , K ~ ) [-1;0_n]\in\operatorname{CC}(\widetilde x^*(1),\widetilde K) [ − 1 ; 0 n ] ∈ CC ( x ∗ ( 1 ) , K ) , i.e., there must exist { e ~ k } k = 1 ∞ ⊂ R \{\widetilde e^k\}_{k=1}^{\infty}\subset\mathbb R { e k } k = 1 ∞ ⊂ R and { σ ~ k } k = 1 ∞ ⊂ R 1 + n \{\widetilde\sigma^k\}_{k=1}^{\infty}\subset\mathbb R^{1+n} { σ k } k = 1 ∞ ⊂ R 1 + n , k ∈ N k\in\mathbb N k ∈ N such that, as k → ∞ k\to\infty k → ∞ , e ~ k ↓ 0 \widetilde e^k\downarrow0 e k ↓ 0 , ∥ σ ~ k ∥ e ~ k → 0 \frac{\|\widetilde\sigma^k\|}{\widetilde e^k}\to0 e k ∥ σ k ∥ → 0 and, for all k k k ,
ξ ~ k : = x ~ ∗ ( 1 ) + e ~ k d ~ + σ ~ k ∈ K ~ \widetilde\xi^k:=\widetilde x^*(1)+\widetilde e^k\widetilde d+\widetilde\sigma^k\in\widetilde K ξ k := x ∗ ( 1 ) + e k d + σ k ∈ K
where d ~ : = [ − 1 ; 0 n ] \widetilde d:=[-1;0_n] d := [ − 1 ; 0 n ] .
We now “groom” sequence { ξ ~ k } \{\widetilde\xi^k\} { ξ k } by the following eight steps.
Step 1. Remove all ξ ~ k \widetilde\xi^k ξ k s such that ξ ~ 0 k ≥ x ~ 0 ∗ ( 1 ) \widetilde\xi_0^k\ge\widetilde x_0^*(1) ξ 0 k ≥ x 0 ∗ ( 1 ) . Since d 0 = − 1 d_0=-1 d 0 = − 1 , there are at most finitely many such points.
Step 2. For all k k k such that ξ ~ k ∉ ∂ K ~ \widetilde\xi^k\notin\partial\widetilde K ξ k ∈ / ∂ K , move ξ ~ k \widetilde\xi^k ξ k along the direction from x ~ ∗ ( 1 ) \widetilde x^*(1) x ∗ ( 1 ) until reaching ∂ K ~ \partial\widetilde K ∂ K . Since x ~ ∗ ( 1 ) + e ~ k d ~ ∉ K ~ \widetilde x^*(1)+\widetilde e^k\widetilde d\notin\widetilde K x ∗ ( 1 ) + e k d ∈ / K (by optimality of x ~ ∗ ( 1 ) \widetilde x^*(1) x ∗ ( 1 ) and since d 0 = − 1 d_0=-1 d 0 = − 1 and e ~ k > 0 \widetilde e^k>0 e k > 0 ), ∂ K ~ \partial\widetilde K ∂ K is reached before x ~ ∗ ( 1 ) + e ~ k d ~ \widetilde x^*(1)+\widetilde e^k\widetilde d x ∗ ( 1 ) + e k d . Such move does not increase ∥ σ ~ k ∥ \|\widetilde\sigma^k\| ∥ σ k ∥ (since, after the move, the new ξ ~ k \widetilde\xi^k ξ k lies on the line segment from the old ξ ~ k ∈ K ~ \widetilde\xi^k\in\widetilde K ξ k ∈ K to x ~ ∗ ( 1 ) + e ~ k d ~ ∉ K ~ \widetilde x^*(1)+\widetilde e^k\widetilde d\notin\widetilde K x ∗ ( 1 ) + e k d ∈ / K ), specifically, at the intersection of that former line segment with ∂ K ~ \partial\widetilde K ∂ K , hence ∥ σ ~ k ∥ e ~ k → 0 \frac{\|\widetilde\sigma^k\|}{\widetilde e^k}\to0 e k ∥ σ k ∥ → 0 is preserved. Since K K K is closed (Lemma 1) ξ ~ k ∈ ∂ K ~ \widetilde\xi^k\in\partial\widetilde K ξ k ∈ ∂ K still holds. In view of Step 1 and since d 0 < 0 d_0<0 d 0 < 0 , ξ ~ 0 k < x ~ 0 ∗ ( 1 ) \widetilde\xi_0^k<\widetilde x_0^*(1) ξ 0 k < x 0 ∗ ( 1 ) still holds for all k k k .
Step 3. By Lemma 4, there is a (a.e.) unique control u k u^k u k that such that the associated trajectory satisfies x ~ ( 1 ; u k ) = ξ ~ k \widetilde x(1;u^k)=\widetilde\xi^k x ( 1 ; u k ) = ξ k , and u k u^k u k is bang–bang with at most n n n sign changes. Let { t ^ j k } j = 1 j ˉ k \{\hat{t}^k_j\}_{j=1}^{\bar{j}_k} { t ^ j k } j = 1 j ˉ k be the successive times of sign changes. Reduce { ξ k } \{\xi^k\} { ξ k } to an infinite subsequence on which j ˉ k \bar{j}_k j ˉ k is a constant j ˉ \bar{j} j ˉ (such subsequence exists since j ˉ k ≤ n \bar{j}_k \le n j ˉ k ≤ n for all k k k ) and sgn ( u k ( 0 ) ) \operatorname{sgn}(u^k(0)) sgn ( u k ( 0 )) is a constant σ ^ \hat{\sigma} σ ^ . Observe that there must exist a nonzero such j ˉ \bar{j} j ˉ since otherwise j ˉ k = 0 \bar{j}_k=0 j ˉ k = 0 for all k k k large enough, i.e., for every k k k large enough, u k ∈ { ± 1 } u^k\in\{\pm1\} u k ∈ { ± 1 } does not vary with k k k and hence ξ ~ k = x ~ 0 ∗ ( 1 ) \tilde{\xi}^k=\tilde{x}^*_0(1) ξ ~ k = x ~ 0 ∗ ( 1 ) for all k k k , in contradiction with ξ 0 k < x 0 ∗ ( 1 ) \xi^k_0<x^*_0(1) ξ 0 k < x 0 ∗ ( 1 ) for all k k k (Step 2).
Step 4. Further reduce { ξ ~ k } \{\tilde{\xi}^k\} { ξ ~ k } to an infinite subsequence such that the associated subsequence ( t ^ j k ) k ∈ N (\hat{t}^k_j)_{k\in\mathbb{N}} ( t ^ j k ) k ∈ N converges for all j ∈ { 1 , … , j ˉ } j\in\{1,\ldots,\bar{j}\} j ∈ { 1 , … , j ˉ } . Let t ^ j ∞ : = lim k → ∞ t ^ j k ∈ [ 0 , 1 ] \hat{t}^{\infty}_j:=\lim_{k\to\infty}\hat{t}^k_j\in[0,1] t ^ j ∞ := lim k → ∞ t ^ j k ∈ [ 0 , 1 ] , j = 1 , … , j ˉ j=1,\ldots,\bar{j} j = 1 , … , j ˉ and let { t 1 , … , t ı ˉ } : = { t ^ j ∞ } j = 1 j ˉ \{t_1,\ldots,t_{\bar{\imath}}\}:=\{\hat{t}^{\infty}_j\}_{j=1}^{\bar{j}} { t 1 , … , t ˉ } := { t ^ j ∞ } j = 1 j ˉ , with t 1 < ⋯ < t ı ˉ t_1<\cdots<t_{\bar{\imath}} t 1 < ⋯ < t ˉ . Further let i j i_j i j be defined by t i j = t ^ j ∞ t_{i_j}=\hat{t}^{\infty}_j t i j = t ^ j ∞ and, for all t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] , let
m t : = # { j : t ^ j ∞ = t } . m_t:=\#\{j:\hat{t}^{\infty}_j=t\}. m t := # { j : t ^ j ∞ = t } .
Also, given any nondecreasing finite-length sequence { t ˇ 1 , … , t ˇ m ˉ } ⊂ [ 0 , 1 ] \{\check{t}_1,\ldots,\check{t}_{\bar{m}}\}\subset[0,1] { t ˇ 1 , … , t ˇ m ˉ } ⊂ [ 0 , 1 ] and σ ˇ ∈ { − 1 , 1 } \check{\sigma}\in\{-1,1\} σ ˇ ∈ { − 1 , 1 } , let
u B B ( t ; σ ˇ , { t ˇ n } n = 1 m ˉ ) : = σ ˇ ( − 1 ) # { m : t ˇ m ≤ t } , ∀ t ∈ [ 0 , 1 ] , u_{\mathrm{BB}}(t;\check{\sigma},\{\check{t}_n\}_{n=1}^{\bar{m}}):=\check{\sigma}(-1)^{\#\{m:\check{t}_m\le t\}},\quad\forall t\in[0,1], u BB ( t ; σ ˇ , { t ˇ n } n = 1 m ˉ ) := σ ˇ ( − 1 ) # { m : t ˇ m ≤ t } , ∀ t ∈ [ 0 , 1 ] ,
a bang–bang signal, that, for all k k k , u k ≡ u B B ( t ; σ ^ , t ^ 1 k , … , t ^ j ˉ k ) u^k\equiv u_{\mathrm{BB}}(t;\hat{\sigma},\hat{t}^k_1,\ldots,\hat{t}^k_{\bar{j}}) u k ≡ u BB ( t ; σ ^ , t ^ 1 k , … , t ^ j ˉ k ) and observe that u B B ( t ; σ ˇ , { t ˇ n } n = 1 m ˉ ) u_{\mathrm{BB}}(t;\check{\sigma},\{\check{t}_n\}_{n=1}^{\bar{m}}) u BB ( t ; σ ˇ , { t ˇ n } n = 1 m ˉ ) is invariant under the following operations:
(i) removal/addition of an even number of repeated t ˇ m \check{t}_m t ˇ m 's;
(ii) removal/addition of any number of t ˇ m \check{t}_m t ˇ m 's at t = 1 t=1 t = 1 ;
(iii) removal of an odd number of repeated t ˇ m \check{t}_m t ˇ m 's at t = 0 t=0 t = 0 with concurrent sign change (multiplication by − 1 -1 − 1 ) of σ ˇ \check{\sigma} σ ˇ ;
We now have, by continuity,
x ~ ( 1 ; u B B ( ⋅ ; σ ^ , t ^ 1 ∞ , … , t ^ j ˉ ∞ ) ) = lim k → ∞ x ~ ( 1 ; u B B ( ⋅ ; σ ^ , t ^ 1 k , … , t ^ j ˉ k ) ) = lim k → ∞ x ~ ( 1 ; u k ) = lim k → ∞ ξ ~ k = x ~ ∗ ( 1 ) = x ~ ( 1 ; u B B ( σ ∗ , t 1 ∗ , … , t j ˉ ∗ ∗ ) ) , \begin{aligned}
\tilde{x}(1;u_{\mathrm{BB}}(\cdot;\hat{\sigma},\hat{t}^{\infty}_1,\ldots,\hat{t}^{\infty}_{\bar{j}}))
&=\lim_{k\to\infty}\tilde{x}(1;u_{\mathrm{BB}}(\cdot;\hat{\sigma},\hat{t}^k_1,\ldots,\hat{t}^k_{\bar{j}}))\\
&=\lim_{k\to\infty}\tilde{x}(1;u^k)=\lim_{k\to\infty}\tilde{\xi}^k\\
&=\tilde{x}^*(1)\\
&=\tilde{x}(1;u_{\mathrm{BB}}(\sigma^*,t^*_1,\ldots,t^*_{\bar{j}^*})),
\end{aligned} x ~ ( 1 ; u BB ( ⋅ ; σ ^ , t ^ 1 ∞ , … , t ^ j ˉ ∞ )) = k → ∞ lim x ~ ( 1 ; u BB ( ⋅ ; σ ^ , t ^ 1 k , … , t ^ j ˉ k )) = k → ∞ lim x ~ ( 1 ; u k ) = k → ∞ lim ξ ~ k = x ~ ∗ ( 1 ) = x ~ ( 1 ; u BB ( σ ∗ , t 1 ∗ , … , t j ˉ ∗ ∗ )) ,
where σ ∗ : = sgn ( u ∗ ( 0 ) ) \sigma^*:=\operatorname{sgn}(u^*(0)) σ ∗ := sgn ( u ∗ ( 0 )) . Since the optimal control is (a.e.) unique (Lemma 4), it follows that
{ t 1 , … , t ı ˉ } ⊇ { t ∈ ( 0 , 1 ) : m t odd } = { t 1 ∗ , … , t ȷ ˉ ∗ ∗ } . \{t_1,\ldots,t_{\bar{\imath}}\}\supseteq\{t\in(0,1):m_t\text{ odd}\}=\{t^*_1,\ldots,t^*_{\bar{\jmath}^*}\}. { t 1 , … , t ˉ } ⊇ { t ∈ ( 0 , 1 ) : m t odd } = { t 1 ∗ , … , t ˉ ∗ ∗ } .
Finally, let
L i k : = min { t ^ j k : i j = i } ∪ { t i } L_i^k:=\min\{\hat{t}^k_j:i_j=i\}\cup\{t_i\} L i k := min { t ^ j k : i j = i } ∪ { t i }
where { t i } \{t_i\} { t i } is not needed when m t i m_{t_i} m t i is even, and
R i k : = max { t ^ j k : i j = i } ∪ { t i } , R_i^k:=\max\{\hat{t}^k_j:i_j=i\}\cup\{t_i\}, R i k := max { t ^ j k : i j = i } ∪ { t i } ,
then
T i k : = [ L i k , R i k ] and ∣ T i k ∣ : = R i k − L i k . T_i^k:=[L_i^k,R_i^k]\quad\text{and}\quad|T_i^k|:=R_i^k-L_i^k. T i k := [ L i k , R i k ] and ∣ T i k ∣ := R i k − L i k .
Observe that for all i i i , as k → ∞ k\to\infty k → ∞ , L i k → t i L_i^k\to t_i L i k → t i , R i k → t i R_i^k\to t_i R i k → t i , and ∣ T i k ∣ → 0 |T_i^k|\to0 ∣ T i k ∣ → 0 .
Step 5. Remove the finitely many (if any) ξ ~ k \tilde{\xi}^k ξ ~ k 's for which the intervals [ L i k , R i k ] [L_i^k,R_i^k] [ L i k , R i k ] , i = 1 , … , ı ˉ i=1,\ldots,\bar{\imath} i = 1 , … , ˉ have a nonempty intersection. Observe that u ∗ ( t ) = u k ( t ) u^*(t)=u^k(t) u ∗ ( t ) = u k ( t ) for all t ∈ [ 0 , 1 ] ∖ ⋃ i = 1 ı ˉ [ L i k , R i k ] t\in[0,1]\setminus\bigcup_{i=1}^{\bar{\imath}}[L_i^k,R_i^k] t ∈ [ 0 , 1 ] ∖ ⋃ i = 1 ˉ [ L i k , R i k ] and let Δ u k : = u k − u ∗ \Delta u^k:=u^k-u^* Δ u k := u k − u ∗ for all k k k .
Let I : = { i : m t i is odd ≥ 3 and t i ∉ { 0 , 1 } } I:=\{i:m_{t_i}\text{ is odd }\ge3\text{ and }t_i\notin\{0,1\}\} I := { i : m t i is odd ≥ 3 and t i ∈ / { 0 , 1 }} and I c : = { 1 , … , ı ˉ } ∖ I I^c:=\{1,\ldots,\bar{\imath}\}\setminus I I c := { 1 , … , ˉ } ∖ I . Note that { t i } i ∈ I ⊆ { t j ∗ } j = 1 , … , ȷ ˉ ∗ \{t_i\}_{i\in I}\subseteq\{t^*_j\}_{j=1,\ldots,\bar{\jmath}^*} { t i } i ∈ I ⊆ { t j ∗ } j = 1 , … , ˉ ∗ , because { t i } i ∈ I ⊆ { t ∈ ( 0 , 1 ) : m t odd } = { t 1 ∗ , … , t ȷ ˉ ∗ ∗ } \{t_i\}_{i\in I}\subseteq\{t\in(0,1):m_t\text{ odd}\}=\{t^*_1,\ldots,t^*_{\bar{\jmath}^*}\} { t i } i ∈ I ⊆ { t ∈ ( 0 , 1 ) : m t odd } = { t 1 ∗ , … , t ˉ ∗ ∗ } . Observe that, for all i ∈ I c i\in I^c i ∈ I c and all k ∈ N k\in\mathbb{N} k ∈ N , Δ u k \Delta u^k Δ u k (which takes values in { 0 , ± 2 } \{0,\pm2\} { 0 , ± 2 } ) does not change nonzero sign on [ L i k , R i k ] [L_i^k,R_i^k] [ L i k , R i k ] , in the sense that, for all k k k , { − 2 , 2 } ⊄ { Δ u k ( t ) : t ∈ [ L i k , R i k ] } \{-2,2\}\not\subset\{\Delta u^k(t):t\in[L_i^k,R_i^k]\} { − 2 , 2 } ⊂ { Δ u k ( t ) : t ∈ [ L i k , R i k ]} . Indeed, given that the intersection of all intervals [ L i k , R i k ] [L_i^k,R_i^k] [ L i k , R i k ] is empty, if Δ u k \Delta u^k Δ u k changes nonzero sign on [ L i k , R i k ] [L_i^k,R_i^k] [ L i k , R i k ] for some k k k , then m t i m_{t_i} m t i is odd, ≥ 3 \ge3 ≥ 3 , t i ∉ { 0 , 1 } t_i\notin\{0,1\} t i ∈ / { 0 , 1 } , i.e., i ∈ I i\in I i ∈ I . (The fact that m t i m_{t_i} m t i must be odd is a direct consequence of the fact that, if Δ u k \Delta u^k Δ u k changes nonzero sign on [ L i k , R i k ] [L_i^k,R_i^k] [ L i k , R i k ] , then u ∗ u^* u ∗ has to change (nonzero) sign on $[L_i^k,R_i^k].)
Next, let Δ x ~ k ( 1 ) : = ξ ~ k − x ~ ∗ ( 1 ) \Delta\tilde{x}^k(1):=\tilde{\xi}^k-\tilde{x}^*(1) Δ x ~ k ( 1 ) := ξ ~ k − x ~ ∗ ( 1 ) , S i k : = cl { t ∈ [ L i k , R i k ] : u k ( t ) − u ∗ ( t ) ≠ 0 } S_i^k:=\operatorname{cl}\{t\in[L_i^k,R_i^k]:u^k(t)-u^*(t)\ne0\} S i k := cl { t ∈ [ L i k , R i k ] : u k ( t ) − u ∗ ( t ) = 0 } , and ∣ S i k ∣ : = ∫ S i k d t |S_i^k|:=\int_{S_i^k}dt ∣ S i k ∣ := ∫ S i k d t . Then
Δ x ~ k ( 1 ) = ∫ 0 1 ( u k ( t ) − u ∗ ( t ) ) φ ~ ( t ) d t = ∑ i ∈ I c ∫ S i k ( u k ( t ) − u ∗ ( t ) ) ( φ ~ ( t i ) + h ~ ( i ) ( t ) ) d t + ∑ i ∈ I ∫ S i k ( u k ( t ) − u ∗ ( t ) ) ( φ ~ ( t i ) + J φ ~ ( t ) ( t − t i ) + h ~ ( i ) ( t ) ( t − t i ) ) d t \begin{aligned}
\Delta\tilde{x}^k(1)=\int_0^1(u^k(t)-u^*(t))\tilde{\varphi}(t)\,dt \\
&=\sum_{i\in I^c}\int_{S_i^k}(u^k(t)-u^*(t))(\tilde{\varphi}(t_i)+\tilde{h}_{(i)}(t))\,dt
+\sum_{i\in I}\int_{S_i^k}(u^k(t)-u^*(t))
\bigl(\tilde{\varphi}(t_i)+J\tilde{\varphi}(t)(t-t_i)+\tilde{h}_{(i)}(t)(t-t_i)\bigr)\,dt
\end{aligned} Δ x ~ k ( 1 ) = ∫ 0 1 ( u k ( t ) − u ∗ ( t )) φ ~ ( t ) d t = i ∈ I c ∑ ∫ S i k ( u k ( t ) − u ∗ ( t )) ( φ ~ ( t i ) + h ~ ( i ) ( t )) d t + i ∈ I ∑ ∫ S i k ( u k ( t ) − u ∗ ( t )) ( φ ~ ( t i ) + J φ ~ ( t ) ( t − t i ) + h ~ ( i ) ( t ) ( t − t i ) ) d t
with ∥ h ~ ( i ) ( t ) ∥ → 0 \|\tilde{h}_{(i)}(t)\|\to0 ∥ h ~ ( i ) ( t ) ∥ → 0 as t → t i t\to t_i t → t i (Taylor, Peano). Hence
Δ x ~ k ( 1 ) = ∑ i ∈ I c ( w i , 0 k φ ~ ( t i ) + r ~ ( i ) k ) + ∑ i ∈ I ( w i , 0 k φ ~ ( t i ) + w i , 1 k J φ ~ ( t i ) + r ~ ( i ) k ) , \Delta\tilde{x}^k(1)=\sum_{i\in I^c}(w^k_{i,0}\tilde{\varphi}(t_i)+\tilde{r}^k_{(i)})
+\sum_{i\in I}(w^k_{i,0}\tilde{\varphi}(t_i)+w^k_{i,1}J\tilde{\varphi}(t_i)+\tilde{r}^k_{(i)}), Δ x ~ k ( 1 ) = i ∈ I c ∑ ( w i , 0 k φ ~ ( t i ) + r ~ ( i ) k ) + i ∈ I ∑ ( w i , 0 k φ ~ ( t i ) + w i , 1 k J φ ~ ( t i ) + r ~ ( i ) k ) ,
where w i , 0 k : = ∫ S i k ( u k ( t ) − u ∗ ( t ) ) d t w^k_{i,0}:=\int_{S_i^k}(u^k(t)-u^*(t))\,dt w i , 0 k := ∫ S i k ( u k ( t ) − u ∗ ( t )) d t , w i , 1 k : = ∫ S i k ( u k ( t ) − u ∗ ( t ) ) ( t − t i ) d t w^k_{i,1}:=\int_{S_i^k}(u^k(t)-u^*(t))(t-t_i)\,dt w i , 1 k := ∫ S i k ( u k ( t ) − u ∗ ( t )) ( t − t i ) d t , and
r ~ ( i ) k : = { ∫ S i k ( u k ( t ) − u ∗ ( t ) ) h ~ ( i ) ( t ) d t i ∈ I c , ∫ S i k ( u k ( t ) − u ∗ ( t ) ) ( t − t i ) h ~ ( i ) ( t ) d t i ∈ I . \tilde{r}^k_{(i)}:=
\begin{cases}
\int_{S_i^k}(u^k(t)-u^*(t))\tilde{h}_{(i)}(t)\,dt & i\in I^c,\\
\int_{S_i^k}(u^k(t)-u^*(t))(t-t_i)\tilde{h}_{(i)}(t)\,dt & i\in I.
\end{cases} r ~ ( i ) k := { ∫ S i k ( u k ( t ) − u ∗ ( t )) h ~ ( i ) ( t ) d t ∫ S i k ( u k ( t ) − u ∗ ( t )) ( t − t i ) h ~ ( i ) ( t ) d t i ∈ I c , i ∈ I .
Within S i k S_i^k S i k , u k ( t ) − u ∗ ( t ) u^k(t)-u^*(t) u k ( t ) − u ∗ ( t ) does not change nonzero sign when i ∈ I c i\in I^c i ∈ I c and ( u k ( t ) − u ∗ ( t ) ) ( t − t i ) (u^k(t)-u^*(t))(t-t_i) ( u k ( t ) − u ∗ ( t )) ( t − t i ) does not change nonzero sign when i ∈ I i\in I i ∈ I . The following ensues:
(1) For i ∈ I c i\in I^c i ∈ I c , ∣ w i , 0 k ∣ = ∫ S i k ∣ u k ( t ) − u ∗ ( t ) ∣ d t |w^k_{i,0}|=\int_{S_i^k}|u^k(t)-u^*(t)|dt ∣ w i , 0 k ∣ = ∫ S i k ∣ u k ( t ) − u ∗ ( t ) ∣ d t and for i ∈ I i\in I i ∈ I ∣ w i , 1 k ∣ = ∫ S i k ∣ u k ( t ) − u ∗ ( t ) ∣ ∣ t − t i ∣ d t |w^k_{i,1}|=\int_{S_i^k}|u^k(t)-u^*(t)||t-t_i|dt ∣ w i , 1 k ∣ = ∫ S i k ∣ u k ( t ) − u ∗ ( t ) ∣∣ t − t i ∣ d t , and
(2) For i ∈ I c i\in I^c i ∈ I c , ∥ r ~ ( i ) k ∥ ≤ ∫ S i k ∣ u k ( t ) − u ∗ ( t ) ∣ ∥ h ~ ( i ) ( t ) ∥ d t ≤ ∣ w i , 0 k ∣ sup t ∈ S i k ∥ h ~ ( i ) ( t ) ∥ \|\tilde{r}^k_{(i)}\|\le\int_{S_i^k}|u^k(t)-u^*(t)|\|\tilde{h}_{(i)}(t)\|dt\le|w^k_{i,0}|\sup_{t\in S_i^k}\|\tilde{h}_{(i)}(t)\| ∥ r ~ ( i ) k ∥ ≤ ∫ S i k ∣ u k ( t ) − u ∗ ( t ) ∣∥ h ~ ( i ) ( t ) ∥ d t ≤ ∣ w i , 0 k ∣ sup t ∈ S i k ∥ h ~ ( i ) ( t ) ∥ and for i ∈ I i\in I i ∈ I , ∥ r ~ ( i ) k ∥ ≤ ∫ S i k ∣ u k ( t ) − u ∗ ( t ) ∣ ∣ t − t i ∣ ∥ h ~ ( i ) ( t ) ∥ d t ≤ ∣ w i , 1 k ∣ sup t ∈ S i k ∥ h ~ ( i ) ( t ) ∥ \|\tilde{r}^k_{(i)}\|\le\int_{S_i^k}|u^k(t)-u^*(t)||t-t_i|\|\tilde{h}_{(i)}(t)\|dt\le|w^k_{i,1}|\sup_{t\in S_i^k}\|\tilde{h}_{(i)}(t)\| ∥ r ~ ( i ) k ∥ ≤ ∫ S i k ∣ u k ( t ) − u ∗ ( t ) ∣∣ t − t i ∣∥ h ~ ( i ) ( t ) ∥ d t ≤ ∣ w i , 1 k ∣ sup t ∈ S i k ∥ h ~ ( i ) ( t ) ∥ ; hence, for all i i i , ∥ r ~ ( i ) k ∥ → 0 \|\tilde{r}^k_{(i)}\|\to0 ∥ r ~ ( i ) k ∥ → 0 as k → ∞ k\to\infty k → ∞ .
Case i ∈ I c i\in I^c i ∈ I c . Let
h ~ ( i ) k : = { 1 w i , 0 k r ~ ( i ) k w i , 0 k ≠ 0 , 0 else . \tilde{h}^k_{(i)}:=
\begin{cases}
\frac{1}{w^k_{i,0}}\tilde{r}^k_{(i)} & w^k_{i,0}\ne0,\\
0 & \text{else}.
\end{cases} h ~ ( i ) k := { w i , 0 k 1 r ~ ( i ) k 0 w i , 0 k = 0 , else .
Then r ~ ( i ) k = w i , 0 k h ~ ( i ) k \tilde{r}^k_{(i)}=w^k_{i,0}\tilde{h}^k_{(i)} r ~ ( i ) k = w i , 0 k h ~ ( i ) k and h ~ ( i ) k → 0 1 + n \tilde{h}^k_{(i)}\to0_{1+n} h ~ ( i ) k → 0 1 + n as k → ∞ k\to\infty k → ∞ . Case i ∈ I i \in I i ∈ I : Let
h ˉ ( i ) k : = { 1 w ˉ i , 1 k w ˉ ( i ) k , w i , 1 k ≠ 0 , 0 , else. \bar{h}_{(i)}^{k}:=
\begin{cases}
\frac{1}{\bar{w}_{i,1}^{k}}\bar{w}_{(i)}^{k}, & w_{i,1}^{k}\ne0,\\
0, & \text{else.}
\end{cases} h ˉ ( i ) k := { w ˉ i , 1 k 1 w ˉ ( i ) k , 0 , w i , 1 k = 0 , else.
Then r ~ ( i ) k = w i , 1 k h ˉ ( i ) k \tilde{r}_{(i)}^{k}=w_{i,1}^{k}\bar{h}_{(i)}^{k} r ~ ( i ) k = w i , 1 k h ˉ ( i ) k and h ˉ ( i ) k → 0 1 + n \bar{h}_{(i)}^{k}\to0_{1+n} h ˉ ( i ) k → 0 1 + n as k → ∞ k\to\infty k → ∞ .
Now, let Δ x ~ k ( 1 ) : = ξ ~ k − x ~ ∗ ( 1 ) \Delta\tilde{x}^{k(1)}:=\tilde{\xi}^{k}-\tilde{x}^{\ast(1)} Δ x ~ k ( 1 ) := ξ ~ k − x ~ ∗ ( 1 ) , i.e., Δ x ~ k ( 1 ) = ϵ k d ~ + o ~ k \Delta\tilde{x}^{k(1)}=\epsilon^{k}\tilde{d}+\tilde{o}^{k} Δ x ~ k ( 1 ) = ϵ k d ~ + o ~ k , and let h ~ k : = 1 ϵ k o ~ k \tilde{h}^{k}:=\frac{1}{\epsilon^{k}}\tilde{o}^{k} h ~ k := ϵ k 1 o ~ k . Then
d ~ + h ~ k = ( Φ ~ + H ~ k ) w ˉ k , H ~ k → 0 ( 1 + n ) × ℓ ˉ as k → ∞ , (17) \tilde{d}+\tilde{h}^{k}=(\tilde{\Phi}+\tilde{H}^{k})\bar{w}^{k},\qquad\tilde{H}^{k}\to0_{(1+n)\times\bar{\ell}}\text{ as }k\to\infty, \tag*{(17)} d ~ + h ~ k = ( Φ ~ + H ~ k ) w ˉ k , H ~ k → 0 ( 1 + n ) × ℓ ˉ as k → ∞ , (17)
where
Φ ~ : = [ ( φ ~ ( t i ) ) i ∈ I c ( J φ ~ ( t i ) ) i ∈ I ( φ ~ ( t i ) ) i ∈ I ] ∈ R ( 1 + n ) × ℓ ˉ , ℓ ˉ ≤ n \tilde{\Phi}:=
\left[
\begin{array}{c}
(\tilde{\varphi}(t_i))_{i\in I^c}\\
(J\tilde{\varphi}(t_i))_{i\in I}\\
(\tilde{\varphi}(t_i))_{i\in I}
\end{array}
\right]\in\mathbb{R}^{(1+n)\times\bar{\ell}},\qquad\bar{\ell}\le n Φ ~ := ( φ ~ ( t i ) ) i ∈ I c ( J φ ~ ( t i ) ) i ∈ I ( φ ~ ( t i ) ) i ∈ I ∈ R ( 1 + n ) × ℓ ˉ , ℓ ˉ ≤ n
and
H ~ k : = [ ( h ˉ ( i ) k ) i ∈ I c ( h ˉ ( i ) k ) i ∈ I 0 ( 1 + n ) × # I ] , \tilde{H}^{k}:=
\left[
(\bar{h}_{(i)}^{k})_{i\in I^c}\quad
(\bar{h}_{(i)}^{k})_{i\in I}\quad
0_{(1+n)\times\#I}
\right], H ~ k := [ ( h ˉ ( i ) k ) i ∈ I c ( h ˉ ( i ) k ) i ∈ I 0 ( 1 + n ) × # I ] ,
w ˉ k : = 1 ϵ k [ ( w i , 0 k ) i ∈ I c ( w i , 1 k ) i ∈ I ( w i , 0 k ) i ∈ I ] T . \bar{w}^{k}:=\frac{1}{\epsilon^{k}}
\left[
(w_{i,0}^{k})_{i\in I^c}\quad
(w_{i,1}^{k})_{i\in I}\quad
(w_{i,0}^{k})_{i\in I}
\right]^{\mathsf T}. w ˉ k := ϵ k 1 [ ( w i , 0 k ) i ∈ I c ( w i , 1 k ) i ∈ I ( w i , 0 k ) i ∈ I ] T .
Note that I I I and I c I^c I c satisfy the condition (see theorem statement) that # I c + 3 # I ≤ n \#I^c+3\#I\le n # I c + 3# I ≤ n . Indeed
j ˉ = ∑ i = 1 m ˉ m t i \bar{j}=\sum_{i=1}^{\bar{m}}m_{t_i} j ˉ = ∑ i = 1 m ˉ m t i by definition of m t m_t m t ;
m t i ≥ 3 ∀ i ∈ I m_{t_i}\ge3\ \forall i\in I m t i ≥ 3 ∀ i ∈ I by definition of I I I , and m t i ≥ 1 m_{t_i}\ge1 m t i ≥ 1 by definition of I c I^c I c ;
j ˉ ≤ n \bar{j}\le n j ˉ ≤ n by Lemma 4,
yielding
# I c + 3 # I = ∑ i ∈ I c 1 + ∑ i ∈ I 3 ≤ ∑ i ∈ I c m t i + ∑ i ∈ I m t i = ∑ i = 1 j ˉ m t i = j ˉ ≤ n . \#I^c+3\#I=\sum_{i\in I^c}1+\sum_{i\in I}3
\le\sum_{i\in I^c}m_{t_i}+\sum_{i\in I}m_{t_i}
=\sum_{i=1}^{\bar{j}}m_{t_i}=\bar{j}\le n. # I c + 3# I = i ∈ I c ∑ 1 + i ∈ I ∑ 3 ≤ i ∈ I c ∑ m t i + i ∈ I ∑ m t i = i = 1 ∑ j ˉ m t i = j ˉ ≤ n .
Equation (18) is equivalently written as
− 1 + h 0 k = ( Φ 0 + H 0 k ) w ˉ k (18) -1+h_{0}^{k}=(\Phi_{0}+H_{0}^{k})\bar{w}^{k} \tag*{(18)} − 1 + h 0 k = ( Φ 0 + H 0 k ) w ˉ k (18)
0 n + h k = ( Φ + H k ) w ˉ k (19) 0_{n}+h^{k}=(\Phi+H^{k})\bar{w}^{k} \tag*{(19)} 0 n + h k = ( Φ + H k ) w ˉ k (19)
where Φ ~ k = [ Φ 0 ; Φ ] \tilde{\Phi}^{k}=[\Phi_{0};\Phi] Φ ~ k = [ Φ 0 ; Φ ] and H ~ k = [ H 0 ; H ] \tilde{H}^{k}=[H_{0};H] H ~ k = [ H 0 ; H ] . From (19), w ˉ k ↛ 0 \bar{w}^{k}\not\to0 w ˉ k → 0 as k → ∞ k\to\infty k → ∞ .
Step 6. Further restrict { ξ k } \{\xi^{k}\} { ξ k } to an infinite subsequence on which ∥ w ˉ k ∥ \|\bar{w}^{k}\| ∥ w ˉ k ∥ is bounded away from 0.
Step 7. Further restrict { ξ k } \{\xi^{k}\} { ξ k } to an infinite subsequence where, for some ℓ × \ell^{\times} ℓ × , argmax ℓ ∣ w ˉ ℓ k ∣ \operatorname{argmax}_{\ell}|\bar{w}_{\ell}^{k}| argmax ℓ ∣ w ˉ ℓ k ∣ contains ℓ × \ell^{\times} ℓ × at every k k k . Then ∣ w ˉ ℓ × k ∣ |\bar{w}_{\ell^{\times}}^{k}| ∣ w ˉ ℓ × k ∣ is bounded away from 0, and 1 ∣ w ˉ ℓ × k ∣ \frac{1}{|\bar{w}_{\ell^{\times}}^{k}|} ∣ w ˉ ℓ × k ∣ 1 is bounded (since ℓ × \ell^{\times} ℓ × is selected as an index of maximal absolute value).
Step 8. Restrict { ξ k } \{\xi^{k}\} { ξ k } to an infinite subsequence (such subsequence exists, by Bolzano-Weierstraß) where 1 ∣ w ˉ ℓ × k ∣ w ˉ k \frac{1}{|\bar{w}_{\ell^{\times}}^{k}|}\bar{w}^{k} ∣ w ˉ ℓ × k ∣ 1 w ˉ k converges, say to w ˉ \bar{w} w ˉ ; note that w ˉ ℓ × = ± 1 \bar{w}_{\ell^{\times}}=\pm1 w ˉ ℓ × = ± 1 . Then, dividing both sides of (20) by ∣ w ˉ ℓ × k ∣ |\bar{w}_{\ell^{\times}}^{k}| ∣ w ˉ ℓ × k ∣ , invoking (18), and taking the limit as k → ∞ k\to\infty k → ∞ yields 0 n = Φ w ˉ 0_n=\Phi\bar{w} 0 n = Φ w ˉ , in contradiction with Φ \Phi Φ having full column rank. The proof of the theorem is complete. ▪
Two results are obtained in the remainder of this subsection. First, we note that, when all eigenvalues of A A A are real, the condition in Theorem 4 always holds, hence Fact 3 holds with p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 whenever ξ f ∈ K \xi^{f}\in K ξ f ∈ K . Then we show that, when some eigenvalues of A A A are not real, at least in the 2 × 2 2\times2 2 × 2 case, Conditions LTS1 still holds when the imaginary parts of such eigenvalues are small enough; we also show that, in the cases when LTS1 fails to hold, it is indeed typically (but not always!) the case that Fact 3 only holds with p 0 ∗ = 0 p_{0}^{*}=0 p 0 ∗ = 0 .
As a preliminary, note that, as A A A is real but possibly defective, with only real eigenvalues, φ γ ( t ) \varphi^{\gamma}(t) φ γ ( t ) is an “exponential polynomial” of the form
φ γ ( t ) = ∑ i = 1 p P i ( t ) e a i t b i . \varphi^{\gamma}(t)=\sum_{i=1}^{p}P_i(t)e^{a_i t}b_i. φ γ ( t ) = i = 1 ∑ p P i ( t ) e a i t b i .
where a 1 , … , a p a_1,\ldots,a_p a 1 , … , a p are distinct real numbers such that every (possibly repeated) eigenvalue λ \lambda λ of A A A belongs to { a 1 , … , a p } \{a_1,\ldots,a_p\} { a 1 , … , a p } and each P i P_i P i is a polynomial of degree less than the size of the largest Jordan block associated to eigenvalue λ i \lambda_i λ i of A A A ; φ γ \varphi^\gamma φ γ has no more than finitely many zeros unless γ = 0 n \gamma=0_n γ = 0 n . Vertical tangents can only exist at points x ~ ( 1 ; u γ ) \tilde{x}(1;u_\gamma) x ~ ( 1 ; u γ ) where u γ u_\gamma u γ maximizes x γ ( 1 , u ) x^\gamma(1,u) x γ ( 1 , u ) over u ∈ U u\in\mathcal{U} u ∈ U . Clearly, unless γ = 0 n \gamma=0_n γ = 0 n , the sole such maximizer is the only u γ ∈ U u_\gamma\in\mathcal{U} u γ ∈ U such that u γ ( t ) = sgn ( φ γ ( t ) ) u_\gamma(t)=\operatorname{sgn}(\varphi^\gamma(t)) u γ ( t ) = sgn ( φ γ ( t )) for a.e. t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] .
Case when all eigenvalues of A A A are real
Lemma 5: Suppose A ∈ R n × n A\in\mathbb{R}^{n\times n} A ∈ R n × n and b ∈ R n b\in\mathbb{R}^{n} b ∈ R n are such that all eigenvalues of A A A are real and ( A , b ) (A,b) ( A , b ) is controllable. Let t 1 < ⋯ < t m t_1<\cdots<t_m t 1 < ⋯ < t m be in ( 0 , 1 ) (0,1) ( 0 , 1 ) and let i 1 < ⋯ < i m ~ i_1<\cdots<i_{\tilde{m}} i 1 < ⋯ < i m ~ be in { 1 , … , m } \{1,\ldots,m\} { 1 , … , m } , with m ~ + m ˉ = m \tilde{m}+\bar{m}=m m ~ + m ˉ = m . Then
Φ n : = [ e ( 1 − t 1 ) A b ⋯ e ( 1 − t m ) A b A e ( 1 − t i 1 ) A b ⋯ A e ( 1 − t i m ~ ) A b ] \Phi_n:=\left[e^{(1-t_1)A}b\ \cdots\ e^{(1-t_m)A}b\ Ae^{(1-t_{i_1})A}b\ \cdots\ Ae^{(1-t_{i_{\tilde{m}}})A}b\right] Φ n := [ e ( 1 − t 1 ) A b ⋯ e ( 1 − t m ) A b A e ( 1 − t i 1 ) A b ⋯ A e ( 1 − t i m ~ ) A b ]
is invertible. As a consequence, every matrix obtained by deleting columns in Φ n \Phi_n Φ n has full column rank.
Proof. “(C1) ⇒ \Rightarrow ⇒ (C2)” portion of “Proof of Proposition 4” in [1 ] , with “has zeros at t 1 , … , t n t_1,\ldots,t_n t 1 , … , t n ” replaced with “has n n n real zeros counting multiplicities”. ▪
The following theorem is a direct consequence of Theorems 3 and 4 and Lemmas 2 and 5.
Theorem 5: With the same assumptions as in Lemma 4, whenever ξ f ∈ K \xi^{f}\in K ξ f ∈ K , either (i) u ∗ u^{*} u ∗ is bang-bang or (ii) Fact 3 applied to the Lagrange steering problem with LTI dynamics (16) holds with p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 (or both).
Case of A A A having some eigenvalues off of the real axis
When some eigenvalues of A A A are non-real and ξ f ∈ ∂ K \xi^{f}\in\partial K ξ f ∈ ∂ K , Fact 3 may or may not hold with p 0 ∗ = 1 p_{0}^{*}=1 p 0 ∗ = 1 . We study the situation when n = 2 n=2 n = 2 , i.e., A : = [ α ω ; − ω α ] A:=[\alpha\ \omega;-\omega\ \alpha] A := [ α ω ; − ω α ] , with α , ω ∈ R \alpha,\omega\in\mathbb{R} α , ω ∈ R and, again, assume that ξ f ∈ ∂ K \xi^{f}\in\partial K ξ f ∈ ∂ K , so that, given γ ∈ R 2 ∖ { 0 } \gamma\in\mathbb{R}^{2}\setminus\{0\} γ ∈ R 2 ∖ { 0 } , the optimal control signal u γ u_\gamma u γ is the (a.e.) unique u γ ∈ U u_\gamma\in\mathcal{U} u γ ∈ U that is a maximizer in
γ T ξ f = max u ∈ U γ T x ( 1 ; u ) . \gamma^{\mathsf T}\xi^{f}=\max_{u\in\mathcal{U}}\gamma^{\mathsf T}x(1;u). γ T ξ f = u ∈ U max γ T x ( 1 ; u ) .
Accordingly, u γ ( t ) = sgn ( φ γ ( t ) ) u_\gamma(t)=\operatorname{sgn}(\varphi^\gamma(t)) u γ ( t ) = sgn ( φ γ ( t )) for all t ∈ ( 0 , 1 ] t\in(0,1] t ∈ ( 0 , 1 ] such that φ γ ( t ) ≠ 0 \varphi^\gamma(t)\ne0 φ γ ( t ) = 0 . We have
φ ( t ) = e ( 1 − t ) A b = e ( 1 − t ) α [ b 1 cos ω ( 1 − t ) + b 2 sin ω ( 1 − t ) − b 1 sin ω ( 1 − t ) + b 2 cos ω ( 1 − t ) ] (20) \varphi(t)=e^{(1-t)A}b
=e^{(1-t)\alpha}
\begin{bmatrix}
b_1\cos\omega(1-t)+b_2\sin\omega(1-t)\\
-b_1\sin\omega(1-t)+b_2\cos\omega(1-t)
\end{bmatrix} \tag*{(20)} φ ( t ) = e ( 1 − t ) A b = e ( 1 − t ) α [ b 1 cos ω ( 1 − t ) + b 2 sin ω ( 1 − t ) − b 1 sin ω ( 1 − t ) + b 2 cos ω ( 1 − t ) ] (20)
yielding
φ γ ( t ) = e ( 1 − t ) α ( ( γ 1 b 1 + γ 2 b 2 ) cos ω ( 1 − t ) + ( γ 1 b 2 − γ 2 b 1 ) sin ω ( 1 − t ) ) = e ( 1 − t ) α ( ( γ 1 b 1 + γ 2 b 2 ) ( cos ω cos ω t + sin ω sin ω t ) + ( γ 1 b 2 − γ 2 b 1 ) ( sin ω cos ω t − cos ω sin ω t ) ) = e ( 1 − t ) α ( σ cos ω t + ρ sin ω t ) = c ( ω , b , γ ) e ( 1 − t ) α cos ( ω t + Θ ( ω , b , γ ) ) (21) \begin{aligned}
\varphi^\gamma(t)=e^{(1-t)\alpha}\left((\gamma_1b_1+\gamma_2b_2)\cos\omega(1-t)+(\gamma_1b_2-\gamma_2b_1)\sin\omega(1-t)\right) \\
&=e^{(1-t)\alpha}\left((\gamma_1b_1+\gamma_2b_2)(\cos\omega\cos\omega t+\sin\omega\sin\omega t)+(\gamma_1b_2-\gamma_2b_1)(\sin\omega\cos\omega t-\cos\omega\sin\omega t)\right) \\
&=e^{(1-t)\alpha}(\sigma\cos\omega t+\rho\sin\omega t) \\
&=c(\omega,b,\gamma)e^{(1-t)\alpha}\cos(\omega t+\Theta(\omega,b,\gamma)) \tag*{(21)}
\end{aligned} φ γ ( t ) = e ( 1 − t ) α ( ( γ 1 b 1 + γ 2 b 2 ) cos ω ( 1 − t ) + ( γ 1 b 2 − γ 2 b 1 ) sin ω ( 1 − t ) ) = e ( 1 − t ) α ( ( γ 1 b 1 + γ 2 b 2 ) ( cos ω cos ω t + sin ω sin ω t ) + ( γ 1 b 2 − γ 2 b 1 ) ( sin ω cos ω t − cos ω sin ω t ) ) = e ( 1 − t ) α ( σ cos ω t + ρ sin ω t ) = c ( ω , b , γ ) e ( 1 − t ) α cos ( ω t + Θ ( ω , b , γ )) (21)
where
c ( ω , b , γ ) : = σ 2 + ρ 2 > 0 , Θ ( ω , b , γ ) : = arg ( σ − j ρ ) , c(\omega,b,\gamma):=\sqrt{\sigma^2+\rho^2}>0,\qquad\Theta(\omega,b,\gamma):=\arg(\sigma-j\rho), c ( ω , b , γ ) := σ 2 + ρ 2 > 0 , Θ ( ω , b , γ ) := arg ( σ − j ρ ) ,
with
σ : = ( γ 1 b 1 + γ 2 b 2 ) cos ω + ( γ 1 b 2 − γ 2 b 1 ) sin ω , \sigma:=(\gamma_1b_1+\gamma_2b_2)\cos\omega+(\gamma_1b_2-\gamma_2b_1)\sin\omega, σ := ( γ 1 b 1 + γ 2 b 2 ) cos ω + ( γ 1 b 2 − γ 2 b 1 ) sin ω ,
ρ : = ( γ 1 b 1 + γ 2 b 2 ) sin ω + ( γ 2 b 1 − γ 1 b 2 ) cos ω . \rho:=(\gamma_1b_1+\gamma_2b_2)\sin\omega+(\gamma_2b_1-\gamma_1b_2)\cos\omega. ρ := ( γ 1 b 1 + γ 2 b 2 ) sin ω + ( γ 2 b 1 − γ 1 b 2 ) cos ω .
Hence we have, for t ∈ ( 0 , 1 ] t\in(0,1] t ∈ ( 0 , 1 ] ,
u γ ( t ) = sgn ( φ γ ( t ) ) = sgn ( cos ( ω t + Θ ( ω , b , γ ) ) ) (22) u_\gamma(t)=\operatorname{sgn}(\varphi^\gamma(t))=\operatorname{sgn}(\cos(\omega t+\Theta(\omega,b,\gamma))) \tag*{(22)} u γ ( t ) = sgn ( φ γ ( t )) = sgn ( cos ( ω t + Θ ( ω , b , γ ))) (22)
and
x ~ ( 1 ; u γ ) = ∫ 0 1 φ ~ ( t ) sgn ( φ γ ( t ) ) d t , \widetilde{x}(1;u_\gamma)=\int_0^1\widetilde{\varphi}(t)\operatorname{sgn}(\varphi^\gamma(t))\,\mathrm{d}t, x ( 1 ; u γ ) = ∫ 0 1 φ ( t ) sgn ( φ γ ( t )) d t ,
x ~ ( 1 ; u γ , ϵ ~ ) = ∫ 0 1 φ ~ ( t ) sgn ( φ γ ( t ) + ϵ ~ T φ ~ ( t ) ) d t . \widetilde{x}(1;u_{\gamma,\widetilde{\epsilon}})=\int_0^1\widetilde{\varphi}(t)\operatorname{sgn}(\varphi^\gamma(t)+\widetilde{\epsilon}^{\mathsf T}\widetilde{\varphi}(t))\,\mathrm{d}t. x ( 1 ; u γ , ϵ ) = ∫ 0 1 φ ( t ) sgn ( φ γ ( t ) + ϵ T φ ( t )) d t .
Now recall that Condition LTS1 holds whenever there is no downward vertical tangent at x ~ ∗ ( 1 ) \widetilde{x}^*(1) x ∗ ( 1 ) and that, for that to be the case, it is sufficient that, for all γ ∈ R 2 ∖ { 0 } \gamma\in\mathbb{R}^2\setminus\{0\} γ ∈ R 2 ∖ { 0 } , { φ γ ( t ^ j γ ) , j = 1 , … , k γ } \{\varphi^{\gamma}(\widehat{t}^{\gamma}_j),j=1,\ldots,k_\gamma\} { φ γ ( t j γ ) , j = 1 , … , k γ } be a linearly independent set, where t ^ 1 γ < ⋯ < t ^ k γ γ \widehat{t}^{\gamma}_1<\cdots<\widehat{t}^{\gamma}_{k_\gamma} t 1 γ < ⋯ < t k γ γ are the zeros of φ γ \varphi^\gamma φ γ in [ 0 , 1 ] [0,1] [ 0 , 1 ] (Theorem 4); also let k γ : = 0 k_\gamma:=0 k γ := 0 when sgn ( φ γ ) \operatorname{sgn}(\varphi^\gamma) sgn ( φ γ ) is constant over [ 0 , 1 ] [0,1] [ 0 , 1 ] . When ω ∈ ( 0 , π ) \omega\in(0,\pi) ω ∈ ( 0 , π ) , then clearly k γ = 0 k_\gamma=0 k γ = 0 or 1 and it follows from (21) that no vertical tangent can exist; indeed, under our controllability assumption (Lemma 2 ), i.e., [ b 1 ; b 2 ] ≠ 0 2 [b_1;b_2]\ne0_2 [ b 1 ; b 2 ] = 0 2 , φ ( t ) \varphi(t) φ ( t ) cannot vanish anywhere on R \mathbb{R} R (since exp ( M ) \exp(M) exp ( M ) is nonsingular for every matrix M M M ); hence, when ω ∈ ( 0 , π ) \omega\in(0,\pi) ω ∈ ( 0 , π ) , Condition LTS1 holds, so that Fact 3 holds with p 0 ∗ = 1 p^*_0=1 p 0 ∗ = 1 , regardless of the location of ξ f \xi^f ξ f in K K K .
Observation. Preliminary examination suggests that, in that 2 × 2 2\times2 2 × 2 case, for almost every triple ( α , b 1 , b 2 ) (\alpha,b_1,b_2) ( α , b 1 , b 2 ) in R 3 \mathbb{R}^3 R 3 , when ω > 2 π \omega>2\pi ω > 2 π , Fact 3 holds only with p 0 ∗ = 0 p^*_0=0 p 0 ∗ = 0 . (For ω ∈ ( π , 2 π ) \omega\in(\pi,2\pi) ω ∈ ( π , 2 π ) , the conclusion clearly depends on the data.)
Conclusion
Conditions on finite-horizon optimal control problems without state constraints were introduced, that imply normality of Pontryagin’s Maximum Principle (i.e., p 0 ∗ = 1 p^*_0=1 p 0 ∗ = 1 satisfies the conditions). The main focus was on the Lagrange problem, for which it was first shown that, under a regularity condition, whenever the “lower boundary” of the “augmented” reachable set (i.e., the reachable set of the associated Mayer problem) is convex and the optimal terminal state lies in the relative interior of the set of reachable states (of the Lagrange problem), the PMP is normal. Next, still within the scope of the Lagrange problem, the attention focused on the steering problem (with target point ξ f \xi^f ξ f ), with LTI dynamics and a closed interval of the real line as set U U U of admissible control values. There it was shown that (i) when the cost integrand has the form f 0 ( ξ , υ ) = a 0 T ξ + h ( υ ) f_0(\xi,\upsilon)=a_0^{\mathsf T}\xi+h(\upsilon) f 0 ( ξ , υ ) = a 0 T ξ + h ( υ ) , with h h h continuous, the PMP is normal except, possibly, when the optimal control is bang–bang, (ii) when furthermore h ( υ ) = b 0 υ h(\upsilon)=b_0\upsilon h ( υ ) = b 0 υ , i.e., the cost integrand is “LTI” as well, and the eigenvalues of the state dynamic matrix A A A are real, which is the case, e.g., when A A A is triangular or symmetric or the system is overdamped, then the PMP is always normal; and, although that is still so in some cases where A A A has some non-real eigenvalues, a 2 × 2 2\times2 2 × 2 example shows that the condition on the imaginary parts of the eigenvalues of A A A cannot be dropped in general.
While the focus of much of the present paper is on single-input problems, extension to multi-input should be within reach; it is left for future studies.
References