Introduction
Finding the saddle points (SPs) of potential energy functionals is a fundamental task in various scientific and engineering applications, particularly those involving energy landscape analysis. For example, the transition states between two (meta)stable states, which are crucial in physics [33], chemistry [86], and biology [80], can be identified as index-1 SPs [3 ] . Higher-index SPs are useful for collective, multi-mode, or concerted transitions and play a key role in the construction of solution landscape [92 ] [42]. Here, an unconstrained index-k k k SP is a critical point where the Euclidean Hessian has exactly k k k negative eigenvalues. Additionally, embedded Riemannian submanifolds of Euclidean space can naturally arise as constraint sets due to the incorporation of physical laws, such as in the Thomson problem [67, 85] and Bose-Einstein condensation [2 , 8 , 14 ] [30]. In such cases, index-k k k constrained SPs can be analogously defined by using the Riemannian gradient and Hessian.
In comparison with finding local or global minimizers, locating SPs exhibits two distinct and major challenges: (1) there are always unknown descent directions at SPs, rendering off-the-shelf optimization methods unstable; (2) in general, it is impossible to construct a global merit function which is variationally minimized (or maximized) at SPs, posing significant difficulties for designing globally convergent numerical methods [52].
There have been numerous algorithmic developments concerning saddle search in the unconstrained settings, together with rigorous theoretical analyses. On the contrary, the exploration in the constrained settings requires explicit global/local level-set representations of submanifolds (see Eq. (3) later for a definition). This limitation rules out applications such as electronic excited-state calculations [29, 37, 72, 88], where quotient structures and rank constraints arise in quantum-mechanical models, and explicit representations are unavailable or computationally inconvenient. In some cases, omitting the quotient structures allows applying the existing methods directly. But unlike searching for local or global minimizers, this simplification can be detrimental in our context; see Example 2 and § 4.1 later for an example involving the Stiefel and Grassmann manifolds.
In this work, we develop a constrained saddle dynamics based on embedded-submanifold geometric primitives and avoid the use of explicit level-set representations completely. We establish the theoretical properties of the continuous dynamics and the resulting discretized algorithms. We further demonstrate the effectiveness of the developed algorithms through electronic excited-state calculations on standard benchmark molecular systems.
Literature review
In the following, we review the existing works for finding SPs in both unconstrained and constrained settings. Most of them primarily focus on developing locally convergent methods, with only few exceptions [51, 83]. Our discussions are confined to cases where objective values, first-order derivatives, and (approximate) Hessian-vector products are available, excluding direct applications of Newton-type methods [5 , 7 ] [22, 82].
Unconstrained settings. In these cases, the numerical methods for finding the index-1 SPs can be mainly categorized into two classes: single-ended and double-ended methods. The double-ended methods (also known as the path-finding or chain-of-states methods) [44, 48, 57] are fed with two candidates of local minimizers and target at finding the minimum energy path (which passes through an index-1 SP under certain conditions by the mountain pass theorem [3 ] ). The single-ended ones (also known as the surface walking or eigenvector-following methods) [71] start with a single initial point without a priori knowledge about the final state. In this work, we focus on the class of single-ended methods1 , covering the activation-relaxation technique (nouveau ) [10 , 11 ] [20, 24, 27, 62, 64], gentlest ascent dynamics [28, 74], (shrinking) dimer methods [95 , 96 ] [45, 47, 73], among others [55, 68]. All the mentioned three methods can be set in the flow
d x d t ( t ) = − Proj v ( t ) ⊥ ( ∇ f ( x ( t ) ) ) + ⟨ v ( t ) , ∇ f ( x ( t ) ) ⟩ v ( t ) = − R v ( t ) ( ∇ f ( x ( t ) ) ) , (1) \begin{aligned}
\frac{\mathrm{d}x}{\mathrm{d}t}(t) = -\operatorname{Proj}_{v(t)^\perp}\left(\nabla f(x(t))\right) + \langle v(t), \nabla f(x(t))\rangle v(t) \\
&= -R_{v(t)}\left(\nabla f(x(t))\right), \tag*{(1)}
\end{aligned} d t d x ( t ) = − Proj v ( t ) ⊥ ( ∇ f ( x ( t )) ) + ⟨ v ( t ) , ∇ f ( x ( t ))⟩ v ( t ) = − R v ( t ) ( ∇ f ( x ( t )) ) , (1)
where f : E → R f:\mathcal{E}\to\mathbb{R} f : E → R is the objective (or energy) functional, ⟨ ⋅ , ⋅ ⟩ : E × E → R \langle\cdot,\cdot\rangle:\mathcal{E}\times\mathcal{E}\to\mathbb{R} ⟨ ⋅ , ⋅ ⟩ : E × E → R is the inner product of the ambient Euclidean space, v ( t ) ∈ E v(t)\in\mathcal{E} v ( t ) ∈ E is an additional normalized direction variable, Proj v ( t ) ⊥ \operatorname{Proj}_{v(t)^\perp} Proj v ( t ) ⊥ denotes the orthogonal projection operator onto span { v ( t ) } ⊥ \operatorname{span}\{v(t)\}^\perp span { v ( t ) } ⊥ , defined as
Proj v ( t ) ⊥ ( u ) : = u − ⟨ v ( t ) , u ⟩ v ( t ) , ∀ u ∈ E , \operatorname{Proj}_{v(t)^\perp}(u) := u-\langle v(t),u\rangle v(t),\quad\forall\ u\in\mathcal{E}, Proj v ( t ) ⊥ ( u ) := u − ⟨ v ( t ) , u ⟩ v ( t ) , ∀ u ∈ E ,
and R v ( t ) R_{v(t)} R v ( t ) represents the (Householder) reflection operator defined using v ( t ) v(t) v ( t ) :
R v ( t ) ( u ) : = u − 2 ⟨ v ( t ) , u ⟩ v ( t ) = Proj v ( t ) ⊥ ( u ) − ⟨ v ( t ) , u ⟩ v ( t ) , ∀ u ∈ E . R_{v(t)}(u) := u-2\langle v(t),u\rangle v(t)=\operatorname{Proj}_{v(t)^\perp}(u)-\langle v(t),u\rangle v(t),\quad\forall\ u\in\mathcal{E}. R v ( t ) ( u ) := u − 2 ⟨ v ( t ) , u ⟩ v ( t ) = Proj v ( t ) ⊥ ( u ) − ⟨ v ( t ) , u ⟩ v ( t ) , ∀ u ∈ E .
Briefly speaking, instead of following the gradient flow, which leads the trajectory to a local or global minimizer, the dynamics (1) increases the objective value by climbing up along ± v ( t ) \pm v(t) ± v ( t ) (whose sign is determined by the angle between v ( t ) v(t) v ( t ) and grad f ( x ( t ) ) \operatorname{grad} f(x(t)) grad f ( x ( t )) ), while decreases the value in all the directions perpendicular to v ( t ) v(t) v ( t ) . An effective candidate for v ( t ) v(t) v ( t ) is the normalized eigenvector corresponding to the lowest eigenvalue of Hess f ( x ( t ) ) \operatorname{Hess} f(x(t)) Hess f ( x ( t )) . In a neighborhood of a nondegenerate index-1 unconstrained SP, the dynamics (1) can then be viewed as the gradient flow of a local strongly convex merit function [32,35], thereby providing a local stabilization for the index-1 SP. This underlies the activation-relaxation technique (nouveau ). Instead of solving for the lowest eigenvector directly, the gentlest ascent dynamics includes a direction dynamics,
d v d t ( t ) = − Proj v ( t ) ⊥ ( Hess f ( x ( t ) ) [ v ( t ) ] ) , \frac{\mathrm{d}v}{\mathrm{d}t}(t)=-\operatorname{Proj}_{v(t)^\perp}\left(\operatorname{Hess} f(x(t))[v(t)]\right), d t d v ( t ) = − Proj v ( t ) ⊥ ( Hess f ( x ( t )) [ v ( t )] ) ,
which follows from the Euler-Lagrange equation of the associated Rayleigh-Ritz minimization. The (shrinking) dimer methods further approximate the Hessian-vector product through a central finite-difference scheme, in which the grid size is sometimes driven to 0 + 0^{+} 0 + as t → + ∞ t\to+\infty t → + ∞ [95 ] .
In recent years, there have been extensions for locating index-k k k unconstrained SPs [92 , 93 ] [23, 28, 32, 60, 61, 74]. For instance, the gentlest ascent dynamics can be generalized by incorporating k k k direction dynamics [93 ] (k < dim ( E ) k<\dim(\mathcal{E}) k < dim ( E ) ), namely,
d x d t ( t ) = − R V ( t ) ( grad f ( x ( t ) ) ) \frac{\mathrm{d}x}{\mathrm{d}t}(t)=-R_{V(t)}\left(\operatorname{grad} f(x(t))\right) d t d x ( t ) = − R V ( t ) ( grad f ( x ( t )) )
and
d v i d t ( t ) = − Proj v i ( t ) ⊥ ( Hess f ( x ( t ) ) [ v i ( t ) ] ) + 2 ∑ j = 1 i − 1 Proj v j ( t ) ( Hess f ( x ( t ) ) [ v i ( t ) ] ) , (2) \begin{aligned}
\frac{\mathrm{d}v_i}{\mathrm{d}t}(t)={}&-\operatorname{Proj}_{v_i(t)^\perp}\left(\operatorname{Hess} f(x(t))[v_i(t)]\right)\\
&+2\sum_{j=1}^{i-1}\operatorname{Proj}_{v_j(t)}\left(\operatorname{Hess} f(x(t))[v_i(t)]\right), \tag*{(2)}
\end{aligned} d t d v i ( t ) = − Proj v i ( t ) ⊥ ( Hess f ( x ( t )) [ v i ( t )] ) + 2 j = 1 ∑ i − 1 Proj v j ( t ) ( Hess f ( x ( t )) [ v i ( t )] ) , (2)
i = 1 , … , k i=1,\ldots,k i = 1 , … , k , with V ( t ) : = ( v 1 ( t ) , … , v k ( t ) ) V(t):=(v_1(t),\ldots,v_k(t)) V ( t ) := ( v 1 ( t ) , … , v k ( t )) orthonormalized,
where R V ( t ) R_{V(t)} R V ( t ) is the reflection operator defined using V ( t ) V(t) V ( t ) :
R V ( t ) ( u ) : = u − 2 ∑ j = 1 k ⟨ v j ( t ) , u ⟩ v j ( t ) , ∀ u ∈ E , R_{V(t)}(u):=u-2\sum_{j=1}^{k}\langle v_j(t),u\rangle v_j(t),\qquad\forall u\in\mathcal{E}, R V ( t ) ( u ) := u − 2 j = 1 ∑ k ⟨ v j ( t ) , u ⟩ v j ( t ) , ∀ u ∈ E ,
and Proj v j ( t ) \operatorname{Proj}_{v_j(t)} Proj v j ( t ) refers to the orthogonal projection operator onto span { v j ( t ) } \operatorname{span}\{v_j(t)\} span { v j ( t )} (j = 1 , … , k j=1,\ldots,k j = 1 , … , k ). The second term on the right-hand side of (2) is introduced to maintain the orthonormality condition ⟨ v i ( t ) , v j ( t ) ⟩ = δ i j \langle v_i(t),v_j(t)\rangle=\delta_{ij} ⟨ v i ( t ) , v j ( t )⟩ = δ ij (i , j = 1 , … , k i,j=1,\ldots,k i , j = 1 , … , k ), by combining the Lagrangian formalism with operator splitting. Some improvements have also been made for acceleration and stabilization, including those leveraging second-order information [93 , 96 ] [20, 74], based on local merit functions [32, 35, 38], and incorporating additional inertial terms [60]. Some works have established the linear stability of the dynamics at unconstrained SPs [93 , 95 ] [28, 52, 60] and the local convergence of the discretized algorithms [95 ] [32, 35, 52, 60, 61] under the nondegeneracy assumption. The error estimates for different discretization schemes can be found in [97 ] [59]. Recently, there have been some attempts dealing with stochastic and degenerate settings [25,46,81]. A package has been designed for solution landscape exploration and construction based on the dynamics (2) [58].
Constrained settings. The exploration in this context remains rather limited [91 , 94 , 98 ] [54, 56]. All these works exploit explicit global/local level-set representations of Riemannian submanifolds, e.g.,
M : = { x ∈ R n ∣ c ( x ) = 0 } with c ( x ) : = ( c 1 ( x ) , … , c q ( x ) ) T ∈ R q , (3) \mathcal{M}:=\{x\in\mathbb{R}^{n}\mid c(x)=0\}\quad\text{with}\quad c(x):=(c_1(x),\ldots,c_q(x))^{\mathsf{T}}\in\mathbb{R}^{q}, \tag*{(3)} M := { x ∈ R n ∣ c ( x ) = 0 } with c ( x ) := ( c 1 ( x ) , … , c q ( x ) ) T ∈ R q , (3)
where q < n q<n q < n and the functions c i c_i c i ’s are smooth and regular, in that rank ( grad c ( x ) ) = q \operatorname{rank}(\operatorname{grad} \mathbf{c}(\mathbf{x}))=q rank ( grad c ( x )) = q for any x ∈ M \mathbf{x}\in\mathcal{M} x ∈ M (note that grad c ( x ) ∈ R q × n \operatorname{grad}\mathbf{c}(\mathbf{x})\in\mathbb{R}^{q\times n} grad c ( x ) ∈ R q × n ). In this case, a constrained saddle dynamics, targeting index-k k k constrained SPs (k < dim ( M ) k<\dim(\mathcal{M}) k < dim ( M ) ), can be derived using the Lagrangian function with operator splitting as follows [91 ] :
d x d t ( t ) : = − R x ( t ) , V ( t ) ( grad M f ( x ( t ) ) ) (4) \frac{\mathrm{d}\mathbf{x}}{\mathrm{d}t}(t):=-R_{\mathbf{x}(t),V(t)}\left(\operatorname{grad}_{\mathcal{M}}f(\mathbf{x}(t))\right) \tag*{(4)} d t d x ( t ) := − R x ( t ) , V ( t ) ( grad M f ( x ( t )) ) (4)
and
d v i d t ( t ) : = − Proj x ( t ) , v i ( t ) ⊥ ( Hess M f ( x ( t ) ) [ v i ( t ) ] ) + 2 ∑ j = 1 i − 1 Proj x ( t ) , v j ( t ) ( Hess M f ( x ( t ) ) [ v i ( t ) ] ) + N ( x ( t ) , d x d t ( t ) , v i ( t ) ) , (5) \begin{aligned}
\frac{\mathrm{d}\mathbf{v}_i}{\mathrm{d}t}(t)&:=-\operatorname{Proj}_{\mathbf{x}(t),\mathbf{v}_i(t)^\perp}\left(\operatorname{Hess}_{\mathcal{M}}f(\mathbf{x}(t))[\mathbf{v}_i(t)]\right)\\
&\quad+2\sum_{j=1}^{i-1}\operatorname{Proj}_{\mathbf{x}(t),\mathbf{v}_j(t)}\left(\operatorname{Hess}_{\mathcal{M}}f(\mathbf{x}(t))[\mathbf{v}_i(t)]\right)+N\left(\mathbf{x}(t),\frac{\mathrm{d}\mathbf{x}}{\mathrm{d}t}(t),\mathbf{v}_i(t)\right), \tag*{(5)}
\end{aligned} d t d v i ( t ) := − Proj x ( t ) , v i ( t ) ⊥ ( Hess M f ( x ( t )) [ v i ( t )] ) + 2 j = 1 ∑ i − 1 Proj x ( t ) , v j ( t ) ( Hess M f ( x ( t )) [ v i ( t )] ) + N ( x ( t ) , d t d x ( t ) , v i ( t ) ) , (5)
i = 1 , … , k i=1,\ldots,k i = 1 , … , k , with V ( t ) : = ( v 1 ( t ) , … , v k ( t ) ) V(t):=(\mathbf{v}_1(t),\ldots,\mathbf{v}_k(t)) V ( t ) := ( v 1 ( t ) , … , v k ( t )) ,
where the operators Proj x ( t ) , v i ( t ) ⊥ \operatorname{Proj}_{\mathbf{x}(t),\mathbf{v}_i(t)^\perp} Proj x ( t ) , v i ( t ) ⊥ , Proj x ( t ) , v j ( t ) \operatorname{Proj}_{\mathbf{x}(t),\mathbf{v}_j(t)} Proj x ( t ) , v j ( t ) , and R x ( t ) , V ( t ) R_{\mathbf{x}(t),V(t)} R x ( t ) , V ( t ) are similarly defined on the tangent space T x ( t ) M T_{\mathbf{x}(t)}\mathcal{M} T x ( t ) M , and
N ( x , d x d t , v i ) : = − grad c ( x ) ⊤ ( grad c ( x ) ⋅ grad c ( x ) ⊤ ) − 1 ( Hess c ( x ) [ d x d t ] ) v i . N\left(\mathbf{x},\frac{\mathrm{d}\mathbf{x}}{\mathrm{d}t},\mathbf{v}_i\right):=-\operatorname{grad}\mathbf{c}(\mathbf{x})^\top\left(\operatorname{grad}\mathbf{c}(\mathbf{x})\cdot\operatorname{grad}\mathbf{c}(\mathbf{x})^\top\right)^{-1}\left(\operatorname{Hess}\mathbf{c}(\mathbf{x})\left[\frac{\mathrm{d}\mathbf{x}}{\mathrm{d}t}\right]\right)\mathbf{v}_i. N ( x , d t d x , v i ) := − grad c ( x ) ⊤ ( grad c ( x ) ⋅ grad c ( x ) ⊤ ) − 1 ( Hess c ( x ) [ d t d x ] ) v i .
Note that the invertibility of grad c ( x ) ⋅ grad c ( x ) ⊤ \operatorname{grad}\mathbf{c}(\mathbf{x})\cdot\operatorname{grad}\mathbf{c}(\mathbf{x})^\top grad c ( x ) ⋅ grad c ( x ) ⊤ follows from the full rank assumption. We also recall that
Hess c ( x ) [ u ] = ( Hess c 1 ( x ) [ u ] , … , Hess c q ( x ) [ u ] ) ⊤ ∈ R q × n , ∀ u ∈ R n . \operatorname{Hess}\mathbf{c}(\mathbf{x})[\mathbf{u}]=\left(\operatorname{Hess}c_1(\mathbf{x})[\mathbf{u}],\ldots,\operatorname{Hess}c_q(\mathbf{x})[\mathbf{u}]\right)^\top\in\mathbb{R}^{q\times n},\quad\forall\ \mathbf{u}\in\mathbb{R}^n. Hess c ( x ) [ u ] = ( Hess c 1 ( x ) [ u ] , … , Hess c q ( x ) [ u ] ) ⊤ ∈ R q × n , ∀ u ∈ R n .
Similar dynamics have been derived in [94 ] [54, 56]. In [91 ] , the authors establish the linear stability of dynamics at index-k k k constrained SPs. That being said, we shall remark that it is difficult to apply the dynamics (4) –(5) directly to embedded submanifolds that lack easy-to-use explicit global/local representations, such as the Grassmann manifolds, fixed-rank manifolds, and more general cases which find applications in, e.g., electronic excited-state calculations [29, 37, 72, 88]. Moreover, the local convergence to constrained SPs with prescribed indices has not been investigated for the discretized algorithms, due to the complication of manifold settings.
Contributions
In this article, we develop a constrained saddle dynamics (CSD) for finding index-k k k constrained SPs on any embedded Riemannian submanifold of Euclidean space. Instead of tracking the lowest k k k -dimensional invariant subspace of the Riemannian Hessian with k k k separate direction variables in T x ( t ) M T_{\mathbf{x}(t)}\mathcal{M} T x ( t ) M , we adopt an orthogonal projector P ( t ) : T x ( t ) M → T x ( t ) M P(t):T_{\mathbf{x}(t)}\mathcal{M}\to T_{\mathbf{x}(t)}\mathcal{M} P ( t ) : T x ( t ) M → T x ( t ) M as a single variable, respecting the inherent quotient structure due to rotational invariance (see § 2 later for details). The position-projector pair ( x ( t ) , P ( t ) ) (\mathbf{x}(t),P(t)) ( x ( t ) , P ( t )) is treated compactly using the CSD formulated on the Grassmann bundle of k k k -planes in the tangent bundle T M T\mathcal{M} T M , denoted by Gr k ( T M ) \operatorname{Gr}_k(T\mathcal{M}) Gr k ( T M ) (see Eq. (7) later for definition). By studying the geometry of Gr k ( T M ) \operatorname{Gr}_k(T\mathcal{M}) Gr k ( T M ) (Lemmas 1, 3, and 5), we reveal that the time derivative of P ( t ) P(t) P ( t ) necessarily contains a term defined by the second fundamental form of the manifold, accounting for the varying tangent spaces along the trajectory. This term vanishes in unconstrained settings (cf. Eq. (2) ) and, after horizontal lifts (Lemma 4 ), reduces to the third term on the right-hand side of Eq. (5) for the submanifolds (3) . Notably, our derivations of CSD rely only on embedded-submanifold geometric primitives and are completely free of explicit global/local level-set representations. In theory, we establish the global well-definedness of the CSD (Theorem 1), its linear stability at index-k k k constrained SPs (Theorem 2), and the first local linear convergence results for the discretized algorithm in the manifold-constrained settings (Theorem 3). Moreover, compared with the existing linear stability results, our analysis removes unnecessary nondegeneracy assumptions on eigenvalues by leveraging the Grassmann bundle formulation (Remarks 3 and 6). We also demonstrate through an example (Example 2) that finding constrained SPs can (1) be worse conditioned and (2) require choosing nonredundant parametrizations, in contrast to locating global or local minimizers.
Finally, we demonstrate the effectiveness of the developed algorithm on linear eigenvalue problems (Section 4.1) and electronic excited-state calculations for standard benchmark molecules (Section 4.2). We also corroborate numerically the influence of problem data and the importance of removing parametrization redundancies when searching for SPs.
Organization. This paper is organized as follows. In Section 2, we first investigate the geometry of the Grassmann bundle Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) , upon which the CSD is built. In Section 3, we establish the theoretical properties of the CSD as well as its discretized version. In Section 4 and Appendix B, we report the numerical results on linear eigenvalue problems and electronic excited-state calculations. The conclusions are drawn in Section 5. Some fundamental concepts of embedded Riemannian submanifolds are recalled in Appendix A for self-containedness.
Notations. Throughout this paper, scalars, vectors, and matrices are usually denoted by lowercase, bold lowercase, and uppercase letters, respectively. The sets or spaces are presented by calligraphic letters. In particular, we write the spaces of all k × k k \times k k × k real symmetric and antisymmetric matrices as R s y m k × k \mathbb{R}^{k \times k}_{\mathrm{sym}} R sym k × k and R a s y m k × k \mathbb{R}^{k \times k}_{\mathrm{asym}} R asym k × k , respectively. The Stiefel manifold of k k k -frames and the Grassmann manifold of k k k -planes in a vector space V \mathcal{V} V are denoted by St k ( V ) \operatorname{St}_{k}(\mathcal{V}) St k ( V ) and Gr k ( V ) \operatorname{Gr}_{k}(\mathcal{V}) Gr k ( V ) , respectively. The orthogonal group of degree k k k is given by O ( k ) \mathcal{O}(k) O ( k ) . For a Euclidean space E \mathcal{E} E , S ( E ) \mathcal{S}(\mathcal{E}) S ( E ) denotes the Euclidean space of self-adjoint linear operators on E \mathcal{E} E . The notation L ( X , Y ) \mathcal{L}(\mathcal{X},\mathcal{Y}) L ( X , Y ) refers to the vector space of linear maps from X \mathcal{X} X to Y \mathcal{Y} Y . The notation “cl” means taking the closure of a set. The notations “⟨ ∙ , ∙ ⟩ \langle\bullet,\bullet\rangle ⟨ ∙ , ∙ ⟩ ” and “∥ ∙ ∥ \lVert\bullet\rVert ∥ ∙ ∥ ” calculate the inner product and norm of vectors in the ambient space. The notation “[ ∙ , ∙ ] [\bullet,\bullet] [ ∙ , ∙ ] ” represents the commutator of two matrices, defined as [ A , B ] : = A B − B A [A,B] := AB - BA [ A , B ] := A B − B A . The identity mapping over a vector space V \mathcal{V} V is denoted by Id V \operatorname{Id}_{\mathcal{V}} Id V . We write the orthogonal projection operator onto a vector space V \mathcal{V} V as Proj V \operatorname{Proj}_{\mathcal{V}} Proj V ; if V = span { v } \mathcal{V} = \operatorname{span}\{v\} V = span { v } or span { v } ⊥ \operatorname{span}\{v\}^{\perp} span { v } ⊥ for some vector v v v , we simply write Proj v \operatorname{Proj}_{v} Proj v or Proj v ⊥ \operatorname{Proj}_{v^{\perp}} Proj v ⊥ . For V : = ( v 1 , … , v k ) V := (v_{1},\ldots,v_{k}) V := ( v 1 , … , v k ) , v i v_{i} v i refers to the i i i -th component of V V V . The reflection operator defined by V = ( v 1 , … , v k ) V = (v_{1},\ldots,v_{k}) V = ( v 1 , … , v k ) is denoted by R V R_{V} R V ; if k = 1 k = 1 k = 1 , we simply write R v R_{v} R v .
For a Riemannian manifold M \mathcal{M} M with x ∈ M x \in\mathcal{M} x ∈ M , the tangent space to M \mathcal{M} M at x x x is denoted by T x M T_{x}\mathcal{M} T x M , the normal space to M \mathcal{M} M at x x x by N x M N_{x}\mathcal{M} N x M , the tangent bundle of M \mathcal{M} M by T M T\mathcal{M} T M , any retraction on M \mathcal{M} M by R R R , the exponential mapping in particular by Exp \operatorname{Exp} Exp , the Riemannian distance by dist M \operatorname{dist}_{\mathcal{M}} dist M , any vector transport by T \mathcal{T} T , the parallel transport in particular by P \mathcal{P} P , and the second fundamental form at x x x by Π x \Pi_{x} Π x . These notations are sometimes equipped with additional superscripts to indicate manifolds. If M \mathcal{M} M is a quotient manifold and M ‾ \overline{\mathcal{M}} M is its total space, with π \pi π the associated quotient map, the tangent space T x M T_{x}\mathcal{M} T x M to M \mathcal{M} M at x x x can be decomposed into vertical and horizontal subspaces, denoted by Ver x ˉ M ‾ \operatorname{Ver}_{\bar{x}}\overline{\mathcal{M}} Ver x ˉ M and Hor x ˉ M ‾ \operatorname{Hor}_{\bar{x}}\overline{\mathcal{M}} Hor x ˉ M , respectively. A general Riemannian metric on T x M T_{x}\mathcal{M} T x M is denoted by ⟨ ∙ , ∙ ⟩ x \langle\bullet,\bullet\rangle_{x} ⟨ ∙ , ∙ ⟩ x . The orthogonal projection and reflection operators defined on T x M T_{x}\mathcal{M} T x M are described by an additional subscript x x x , e.g., Proj x , v \operatorname{Proj}_{x,v} Proj x , v and R x , v R_{x,v} R x , v . For a smooth function f f f , we write its differential, Euclidean gradient, and Euclidean Hessian as D f \mathrm{D}f D f , grad f \operatorname{grad} f grad f , and Hess f \operatorname{Hess} f Hess f , respectively. If it is defined on a Riemannian manifold M \mathcal{M} M , its Riemannian gradient and Hessian are denoted by grad M f \operatorname{grad}_{\mathcal{M}} f grad M f and Hess M f \operatorname{Hess}_{\mathcal{M}} f Hess M f , respectively.
The notation “⊕ \oplus ⊕ ” stands for the direct sum of two vector spaces, and “≃ \simeq ≃ ” for the diffeomorphism between two vector spaces. When describing algorithms, we use superscripts within brackets to refer to the iteration numbers. For notational ease, we sometimes use “h.c.” to denote the Hermitian conjugate of the part in front.
Algorithmic developments
In this section, we develop a constrained saddle dynamics (CSD) for locating index-k k k constrained SPs, applicable to any embedded Riemannian submanifold of Euclidean space. Existing derivations [91 ] [54, 56] exploit explicit level-set representations of submanifolds (see Eq. (3) ) and introduce k k k different direction dynamics to track the lowest k k k -dimensional invariant subspace of the Riemannian Hessian. However, it is not difficult to observe the quotient structure under the hood: only the subspace is pursued and it is invariant under the choice of orthonormal basis. Therefore, we instead adopt a single dynamics of an orthogonal projector, which, together with the position dynamics, will amount to the CSD on the Grassmann bundle Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) (see § 2.2 ). To this end, we first introduce Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) and the closely related Stiefel bundle of tangent bundle St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) , and investigate their geometries (see § 2.1 ). We refer readers to Appendix § 7 for some fundamentals on embedded Riemannian submanifolds if necessary. For simplicity, we assume in this section that E = R n \mathcal{E} = \mathbb{R}^{n} E = R n with n ∈ N n \in\mathbb{N} n ∈ N . Nevertheless, our arguments apply to cases of general Euclidean spaces.
St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) and Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) , and their geometries
The Stiefel bundle of k k k -frames and the Grassmann bundle of k k k -planes in the tangent bundle T M T\mathcal{M} T M are respectively defined by
St k ( T M ) : = { ( x , V ) ∣ x ∈ M , V ∈ St k ( T x M ) } , (6) \operatorname{St}_{k}(T\mathcal{M}) := \{(x,V) \mid x \in\mathcal{M},\ V \in\operatorname{St}_{k}(T_{x}\mathcal{M})\}, \tag*{(6)} St k ( T M ) := {( x , V ) ∣ x ∈ M , V ∈ St k ( T x M )} , (6)
Gr k ( T M ) : = { ( x , P ) ∣ x ∈ M , P ∈ Gr k ( T x M ) } , (7) \operatorname{Gr}_{k}(T\mathcal{M}) := \{(x,P) \mid x \in\mathcal{M},\ P \in\operatorname{Gr}_{k}(T_{x}\mathcal{M})\}, \tag*{(7)} Gr k ( T M ) := {( x , P ) ∣ x ∈ M , P ∈ Gr k ( T x M )} , (7)
where St k ( T x M ) \operatorname{St}_{k}(T_{x}\mathcal{M}) St k ( T x M ) and Gr k ( T x M ) \operatorname{Gr}_{k}(T_{x}\mathcal{M}) Gr k ( T x M ) stand for the Stiefel manifold of ordered k k k -tuples of orthonormal vectors in T x M T_{x}\mathcal{M} T x M and the Grassmann manifold of k k k -dimensional linear subspaces of T x M T_{x}\mathcal{M} T x M , respectively. In some contexts, St k ( T x M ) \operatorname{St}_{k}(T_{x}\mathcal{M}) St k ( T x M ) and Gr k ( T x M ) \operatorname{Gr}_{k}(T_{x}\mathcal{M}) Gr k ( T x M ) are called fibers over x ∈ M x \in\mathcal{M} x ∈ M . When k = 1 k = 1 k = 1 , Gr 1 ( T M ) = : P ( T M ) \operatorname{Gr}_{1}(T\mathcal{M}) =: \mathbb{P}(T\mathcal{M}) Gr 1 ( T M ) =: P ( T M ) is called the projective bundle of T M T\mathcal{M} T M . In some references (e.g., [78]), Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) is also called contact bundle. Fiberwise, St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) and Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) possess the same geometry as the classical manifolds St k ( R d ) \operatorname{St}_{k}(\mathbb{R}^{d}) St k ( R d ) and Gr k ( R d ) \operatorname{Gr}_{k}(\mathbb{R}^{d}) Gr k ( R d ) , respectively, where d : = dim ( M ) d := \dim(\mathcal{M}) d := dim ( M ) . But unlike the classical ones, these fibers vary with the base point, which produces additional subtleties (vide infra).
The two bundle structures are complicated in their definitions, particularly because the second components move with base points. This may necessitate the use of intrinsic local coordinate charts when deriving their geometries. Nevertheless, we show that both St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) and Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) are embedded Riemannian submanifolds of some ambient Euclidean spaces. This result enables us later to study them in an extrinsic way.
Lemma 1 (St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) and Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) as embedded submanifolds). Let M \mathcal{M} M be an embedded Riemannian submanifold of R n \mathbb{R}^{n} R n with dim ( M ) = d \dim(\mathcal{M}) = d dim ( M ) = d and let 1 ≤ k < d 1 \leq k < d 1 ≤ k < d . Then the following two statements hold.
(i) St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) is an embedded Riemannian submanifold of R n × R n × k \mathbb{R}^{n} \times\mathbb{R}^{n \times k} R n × R n × k with
dim ( St k ( T M ) ) = d + k d − k ( k + 1 ) 2 . \dim(\operatorname{St}_{k}(T\mathcal{M})) = d + kd - \frac{k(k+1)}{2}. dim ( St k ( T M )) = d + k d − 2 k ( k + 1 ) . (ii) Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) is an embedded Riemannian submanifold of R n × R s y m n × n \mathbb{R}^{n} \times\mathbb{R}_{\mathrm{sym}}^{n \times n} R n × R sym n × n with
dim ( Gr k ( T M ) ) = d + k ( d − k ) . \dim(\operatorname{Gr}_{k}(T\mathcal{M})) = d + k(d-k). dim ( Gr k ( T M )) = d + k ( d − k ) .
Proof. Let x ˉ ∈ M \bar{x} \in\mathcal{M} x ˉ ∈ M . Choose an open neighborhood U ⊆ M \mathcal{U} \subseteq\mathcal{M} U ⊆ M of x ˉ \bar{x} x ˉ and a smooth orthonormal frame, e 1 ( x ) , … , e d ( x ) e_{1}(x), \ldots, e_{d}(x) e 1 ( x ) , … , e d ( x ) , of T M T\mathcal{M} T M over U \mathcal{U} U . Let E ( x ) : = ( e 1 ( x ) , … , e d ( x ) ) ∈ R n × d E(x) := (e_{1}(x), \ldots, e_{d}(x)) \in\mathbb{R}^{n \times d} E ( x ) := ( e 1 ( x ) , … , e d ( x )) ∈ R n × d . Then it holds that E ( x ) T E ( x ) = I d E(x)^{\mathsf{T}}E(x) = I_{d} E ( x ) T E ( x ) = I d and E ( x ) E ( x ) T = Proj T x M E(x)E(x)^{\mathsf{T}} = \operatorname{Proj}_{T_{x}\mathcal{M}} E ( x ) E ( x ) T = Proj T x M . (i) Define
F 1 : U × St k ( R d ) → M × R n × k , F 1 ( x , Y ) : = ( x , E ( x ) Y ) . F_1:\mathcal{U}\times\operatorname{St}_k(\mathbb{R}^d)\to\mathcal{M}\times\mathbb{R}^{n\times k},\qquad F_1(x,Y):=(x,E(x)Y). F 1 : U × St k ( R d ) → M × R n × k , F 1 ( x , Y ) := ( x , E ( x ) Y ) .
Then we have
F 1 ( U × St k ( R d ) ) = St k ( T M ) ∩ ( U × R n × k ) . F_1\bigl(\mathcal{U}\times\operatorname{St}_k(\mathbb{R}^d)\bigr)=\operatorname{St}_k(T\mathcal{M})\cap\bigl(\mathcal{U}\times\mathbb{R}^{n\times k}\bigr). F 1 ( U × St k ( R d ) ) = St k ( T M ) ∩ ( U × R n × k ) .
The forward inclusion is straightforward. Conversely, for any x ∈ U x\in\mathcal{U} x ∈ U and V ∈ St k ( T x M ) V\in\operatorname{St}_k(T_x\mathcal{M}) V ∈ St k ( T x M ) , let Y : = E ( x ) T V ∈ R d × k Y:=E(x)^{\mathsf{T}}V\in\mathbb{R}^{d\times k} Y := E ( x ) T V ∈ R d × k , we have F 1 ( x , Y ) = ( x , E ( x ) E ( x ) T V ) = ( x , V ) F_1(x,Y)=(x,E(x)E(x)^{\mathsf{T}}V)=(x,V) F 1 ( x , Y ) = ( x , E ( x ) E ( x ) T V ) = ( x , V ) and Y T Y = V T E ( x ) E ( x ) T V = V T V = I k Y^{\mathsf{T}}Y=V^{\mathsf{T}}E(x)E(x)^{\mathsf{T}}V=V^{\mathsf{T}}V=I_k Y T Y = V T E ( x ) E ( x ) T V = V T V = I k . Therefore, the equality holds.
The map F 1 F_1 F 1 is injective. Indeed, for any ( x 1 , V 1 ) , ( x 2 , V 2 ) ∈ St k ( T M ) ∩ ( U × R n × k ) (x_1,V_1),(x_2,V_2)\in\operatorname{St}_k(T\mathcal{M})\cap(\mathcal{U}\times\mathbb{R}^{n\times k}) ( x 1 , V 1 ) , ( x 2 , V 2 ) ∈ St k ( T M ) ∩ ( U × R n × k ) , let ( x 1 , Y 1 ) , ( x 2 , Y 2 ) ∈ U × St k ( R d ) (x_1,Y_1),(x_2,Y_2)\in\mathcal{U}\times\operatorname{St}_k(\mathbb{R}^d) ( x 1 , Y 1 ) , ( x 2 , Y 2 ) ∈ U × St k ( R d ) be their pre-images, respectively. Given x 1 = x 2 = x ∈ M x_1=x_2=x\in\mathcal{M} x 1 = x 2 = x ∈ M , it holds that E ( x ) Y 1 = V 1 = V 2 = E ( x ) Y 2 E(x)Y_1=V_1=V_2=E(x)Y_2 E ( x ) Y 1 = V 1 = V 2 = E ( x ) Y 2 , which implies Y 1 = Y 2 Y_1=Y_2 Y 1 = Y 2 after left-multiplying E ( x ) T E(x)^{\mathsf{T}} E ( x ) T . The inverse of F 1 F_1 F 1 is continuous, as it is precisely F 1 − 1 ( x , V ) = ( x , E ( x ) T V ) ∈ U × St k ( R d ) F_1^{-1}(x,V)=(x,E(x)^{\mathsf{T}}V)\in\mathcal{U}\times\operatorname{St}_k(\mathbb{R}^d) F 1 − 1 ( x , V ) = ( x , E ( x ) T V ) ∈ U × St k ( R d ) for any x ∈ U x\in\mathcal{U} x ∈ U and V ∈ St k ( T x M ) V\in\operatorname{St}_k(T_x\mathcal{M}) V ∈ St k ( T x M ) . Finally, the differential of F 1 F_1 F 1 at ( x , Y ) ∈ U × St k ( R d ) (x,Y)\in\mathcal{U}\times\operatorname{St}_k(\mathbb{R}^d) ( x , Y ) ∈ U × St k ( R d ) is
D F 1 ( x , Y ) [ δ x , δ Y ] = ( δ x , D E ( x ) [ δ x ] Y + E ( x ) δ Y ) , ∀ ( δ x , δ Y ) ∈ T ( x , Y ) ( U × St k ( R d ) ) . DF_1(x,Y)[\delta x,\delta Y]=(\delta x,DE(x)[\delta x]Y+E(x)\delta Y),\qquad\forall\,(\delta x,\delta Y)\in T_{(x,Y)}(\mathcal{U}\times\operatorname{St}_k(\mathbb{R}^d)). D F 1 ( x , Y ) [ δ x , δ Y ] = ( δ x , D E ( x ) [ δ x ] Y + E ( x ) δ Y ) , ∀ ( δ x , δ Y ) ∈ T ( x , Y ) ( U × St k ( R d )) .
Then δ x = 0 \delta x=0 δ x = 0 and D E ( x ) [ δ x ] Y + E ( x ) δ Y = 0 DE(x)[\delta x]Y+E(x)\delta Y=0 D E ( x ) [ δ x ] Y + E ( x ) δ Y = 0 imply E ( x ) δ Y = 0 E(x)\delta Y=0 E ( x ) δ Y = 0 , which further yields δ Y = 0 \delta Y=0 δ Y = 0 after left-multiplying E ( x ) T E(x)^{\mathsf{T}} E ( x ) T . Thus, F 1 F_1 F 1 is an immersion. As a result, F 1 F_1 F 1 smoothly embeds U × St k ( R d ) ≅ St k ( T M ) ∩ ( U × R n × k ) \mathcal{U}\times\operatorname{St}_k(\mathbb{R}^d)\cong\operatorname{St}_k(T\mathcal{M})\cap(\mathcal{U}\times\mathbb{R}^{n\times k}) U × St k ( R d ) ≅ St k ( T M ) ∩ ( U × R n × k ) into M × R n × k \mathcal{M}\times\mathbb{R}^{n\times k} M × R n × k . Due to the arbitrariness of x ˉ \bar{x} x ˉ , St k ( T M ) \operatorname{St}_k(T\mathcal{M}) St k ( T M ) is an embedded submanifold of M × R n × k \mathcal{M}\times\mathbb{R}^{n\times k} M × R n × k . Its dimension is
dim ( St k ( T M ) ) = dim ( U × St k ( R d ) ) = d + k d − k ( k + 1 ) 2 . \dim(\operatorname{St}_k(T\mathcal{M}))=\dim(\mathcal{U}\times\operatorname{St}_k(\mathbb{R}^d))=d+kd-\frac{k(k+1)}{2}. dim ( St k ( T M )) = dim ( U × St k ( R d )) = d + k d − 2 k ( k + 1 ) .
(ii) Define
F 2 : U × Gr k ( R d ) → M × R s y m n × n , F 2 ( x , W ) : = ( x , E ( x ) W E ( x ) T ) . F_2:\mathcal{U}\times\operatorname{Gr}_k(\mathbb{R}^d)\to\mathcal{M}\times\mathbb{R}_{\mathrm{sym}}^{n\times n},\qquad F_2(x,W):=(x,E(x)WE(x)^{\mathsf{T}}). F 2 : U × Gr k ( R d ) → M × R sym n × n , F 2 ( x , W ) := ( x , E ( x ) W E ( x ) T ) .
Then we have
F 2 ( U × Gr k ( R d ) ) = Gr k ( T M ) ∩ ( U × R s y m n × n ) . F_2\bigl(\mathcal{U}\times\operatorname{Gr}_k(\mathbb{R}^d)\bigr)=\operatorname{Gr}_k(T\mathcal{M})\cap\bigl(\mathcal{U}\times\mathbb{R}_{\mathrm{sym}}^{n\times n}\bigr). F 2 ( U × Gr k ( R d ) ) = Gr k ( T M ) ∩ ( U × R sym n × n ) .
Indeed, for the forward inclusion, pick any ( x , W ) ∈ U × Gr k ( R d ) (x,W)\in\mathcal{U}\times\operatorname{Gr}_k(\mathbb{R}^d) ( x , W ) ∈ U × Gr k ( R d ) , let P : = E ( x ) W E ( x ) T P:=E(x)WE(x)^{\mathsf{T}} P := E ( x ) W E ( x ) T , we can verify that P T = P P^{\mathsf{T}}=P P T = P , that
P 2 = E ( x ) W E ( x ) T E ( x ) W E ( x ) T = E ( x ) W 2 E ( x ) T = E ( x ) W E ( x ) T = P , P^2=E(x)WE(x)^{\mathsf{T}}E(x)WE(x)^{\mathsf{T}}=E(x)W^2E(x)^{\mathsf{T}}=E(x)WE(x)^{\mathsf{T}}=P, P 2 = E ( x ) W E ( x ) T E ( x ) W E ( x ) T = E ( x ) W 2 E ( x ) T = E ( x ) W E ( x ) T = P ,
that Ran ( P ) ⊆ T x M \operatorname{Ran}(P)\subseteq T_x\mathcal{M} Ran ( P ) ⊆ T x M due to
Proj T x M ∘ P = E ( x ) E ( x ) T E ( x ) W E ( x ) T = E ( x ) W E ( x ) T = P , \operatorname{Proj}_{T_x\mathcal{M}}\circ P=E(x)E(x)^{\mathsf{T}}E(x)WE(x)^{\mathsf{T}}=E(x)WE(x)^{\mathsf{T}}=P, Proj T x M ∘ P = E ( x ) E ( x ) T E ( x ) W E ( x ) T = E ( x ) W E ( x ) T = P ,
and that rank ( P ) = rank ( W ) = k \operatorname{rank}(P)=\operatorname{rank}(W)=k rank ( P ) = rank ( W ) = k , as desired. Conversely, for any x ∈ U x\in\mathcal{U} x ∈ U and P ∈ Gr k ( T x M ) P\in\operatorname{Gr}_k(T_x\mathcal{M}) P ∈ Gr k ( T x M ) , let W : = E ( x ) T P E ( x ) W:=E(x)^{\mathsf{T}}PE(x) W := E ( x ) T P E ( x ) , we have
F 2 ( x , W ) = ( x , E ( x ) E ( x ) T P E ( x ) E ( x ) T ) = ( x , Proj T x M ∘ P ∘ Proj T x M ) = ( x , P ) . F_2(x,W)=(x,E(x)E(x)^{\mathsf{T}}PE(x)E(x)^{\mathsf{T}})=(x,\operatorname{Proj}_{T_x\mathcal{M}}\circ P\circ\operatorname{Proj}_{T_x\mathcal{M}})=(x,P). F 2 ( x , W ) = ( x , E ( x ) E ( x ) T P E ( x ) E ( x ) T ) = ( x , Proj T x M ∘ P ∘ Proj T x M ) = ( x , P ) .
Similar to the proof of (i), we can show the injectivity of F 2 F_2 F 2 . The inverse of F 2 F_2 F 2 is precisely F 2 − 1 ( x , P ) = ( x , E ( x ) T P E ( x ) ) ∈ U × Gr k ( R d ) F_2^{-1}(x,P)=(x,E(x)^{\mathsf{T}}PE(x))\in\mathcal{U}\times\operatorname{Gr}_k(\mathbb{R}^d) F 2 − 1 ( x , P ) = ( x , E ( x ) T P E ( x )) ∈ U × Gr k ( R d ) for any x ∈ U x\in\mathcal{U} x ∈ U and P ∈ Gr k ( T x M ) P\in\operatorname{Gr}_k(T_x\mathcal{M}) P ∈ Gr k ( T x M ) , which is continuous. Finally, F 2 F_2 F 2 is also an immersion, as its differential at ( x , W ) ∈ U × Gr k ( R d ) (x,W)\in\mathcal{U}\times\operatorname{Gr}_k(\mathbb{R}^d) ( x , W ) ∈ U × Gr k ( R d ) is
D F 2 ( x , W ) [ δ x , δ W ] = ( δ x , D E ( x ) [ δ x ] W E ( x ) T + E ( x ) δ W E ( x ) T + E ( x ) W D E ( x ) [ δ x ] T ) , ∀ ( δ x , δ W ) ∈ T ( x , W ) ( U × Gr k ( R d ) ) . DF_2(x,W)[\delta x,\delta W]=\left(\delta x,DE(x)[\delta x]WE(x)^{\mathsf{T}}+E(x)\delta WE(x)^{\mathsf{T}}+E(x)WDE(x)[\delta x]^{\mathsf{T}}\right),\qquad\forall\,(\delta x,\delta W)\in T_{(x,W)}(\mathcal{U}\times\operatorname{Gr}_k(\mathbb{R}^d)). D F 2 ( x , W ) [ δ x , δ W ] = ( δ x , D E ( x ) [ δ x ] W E ( x ) T + E ( x ) δ W E ( x ) T + E ( x ) W D E ( x ) [ δ x ] T ) , ∀ ( δ x , δ W ) ∈ T ( x , W ) ( U × Gr k ( R d )) .
Therefore, F 2 F_2 F 2 smoothly embeds U × Gr k ( R d ) ≅ Gr k ( T M ) ∩ ( U × R s y m n × n ) \mathcal{U}\times\operatorname{Gr}_k(\mathbb{R}^d)\cong\operatorname{Gr}_k(T\mathcal{M})\cap(\mathcal{U}\times\mathbb{R}_{\mathrm{sym}}^{n\times n}) U × Gr k ( R d ) ≅ Gr k ( T M ) ∩ ( U × R sym n × n ) into M × R s y m n × n \mathcal{M}\times\mathbb{R}_{\mathrm{sym}}^{n\times n} M × R sym n × n . Again due to the arbitrariness of x ˉ \bar{x} x ˉ , Gr k ( T M ) \operatorname{Gr}_k(T\mathcal{M}) Gr k ( T M ) is an embedded submanifold of M × R s y m n × n \mathcal{M}\times\mathbb{R}_{\mathrm{sym}}^{n\times n} M × R sym n × n . Its dimension is
dim ( Gr k ( T M ) ) = dim ( U × Gr k ( R d ) ) = d + k ( d − k ) . \dim(\operatorname{Gr}_k(T\mathcal{M}))=\dim(\mathcal{U}\times\operatorname{Gr}_k(\mathbb{R}^d))=d+k(d-k). dim ( Gr k ( T M )) = dim ( U × Gr k ( R d )) = d + k ( d − k ) .
Lemma 1 allows us to investigate the geometries of St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) and Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) safely in an extrinsic way. We start with the characterization of the tangent space to St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) .
Lemma 2 (Tangent space to St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) ). Let M \mathcal{M} M be an embedded Riemannian submanifold of R n \mathbb{R}^{n} R n with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d and let 1 ≤ k < d 1\leq k<d 1 ≤ k < d . For any ( x , V ) ∈ St k ( T M ) (x,V)\in\operatorname{St}_{k}(T\mathcal{M}) ( x , V ) ∈ St k ( T M ) , the tangent space
T ( x , V ) St k ( T M ) = { ( δ , Γ ) | δ ∈ T x M , A ∈ R a s y m k × k , B ∈ R ( d − k ) × k , Γ = V A + V ⊥ B + ( I I x ( δ , v 1 ) , … , I I x ( δ , v k ) ) } , (8) T_{(x,V)}\operatorname{St}_{k}(T\mathcal{M})=\left\{(\delta,\Gamma)\ \middle|\ \delta\in T_{x}\mathcal{M},\ A\in\mathbb{R}_{\mathrm{asym}}^{k\times k},\ B\in\mathbb{R}^{(d-k)\times k},\ \Gamma=VA+V_{\perp}B+\left(\mathrm{II}_{x}(\delta,v_{1}),\ldots,\mathrm{II}_{x}(\delta,v_{k})\right)\right\}, \tag*{(8)} T ( x , V ) St k ( T M ) = { ( δ , Γ ) δ ∈ T x M , A ∈ R asym k × k , B ∈ R ( d − k ) × k , Γ = V A + V ⊥ B + ( II x ( δ , v 1 ) , … , II x ( δ , v k ) ) } , (8) where V ⊥ ∈ St d − k ( T x M ) V_{\perp}\in\operatorname{St}_{d-k}(T_{x}\mathcal{M}) V ⊥ ∈ St d − k ( T x M ) satisfies V T V ⊥ = 0 V^{\mathsf{T}}V_{\perp}=0 V T V ⊥ = 0 . The characterization (8) is independent from the choice of V ⊥ V_{\perp} V ⊥ .
Proof. For any ( δ , Γ ) ∈ T ( x , V ) St k ( T M ) (\delta,\Gamma)\in T_{(x,V)}\operatorname{St}_{k}(T\mathcal{M}) ( δ , Γ ) ∈ T ( x , V ) St k ( T M ) , take a smooth curve c : = ( c 1 , c 2 ) : R ⊇ I → St k ( T M ) c:=(c_{1},c_{2}):\mathbb{R}\supseteq I\to\operatorname{St}_{k}(T\mathcal{M}) c := ( c 1 , c 2 ) : R ⊇ I → St k ( T M ) on St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) such that c ( 0 ) = ( x , V ) c(0)=(x,V) c ( 0 ) = ( x , V ) and c ′ ( 0 ) = ( δ , Γ ) c'(0)=(\delta,\Gamma) c ′ ( 0 ) = ( δ , Γ ) . It is obvious that c 1 c_{1} c 1 is a smooth curve on M \mathcal{M} M passing through x x x at the origin, which implies δ ∈ T x M \delta\in T_{x}\mathcal{M} δ ∈ T x M . For the second part, note that c 2 ( s ) T c 2 ( s ) = I k c_{2}(s)^{\mathsf{T}}c_{2}(s)=I_{k} c 2 ( s ) T c 2 ( s ) = I k and c 2 ( s ) j = Proj T c 1 ( s ) M ( c 2 ( s ) j ) c_{2}(s)_{j}=\operatorname{Proj}_{T_{c_{1}(s)}\mathcal{M}}(c_{2}(s)_{j}) c 2 ( s ) j = Proj T c 1 ( s ) M ( c 2 ( s ) j ) , where c 2 ( s ) j c_{2}(s)_{j} c 2 ( s ) j refers to the j j j -th column of c 2 ( s ) c_{2}(s) c 2 ( s ) (j = 1 , … , k j=1,\ldots,k j = 1 , … , k ). It therefore holds by Lemma 1 and the product rule that Γ T V + V T Γ = 0 \Gamma^{\mathsf{T}}V+V^{\mathsf{T}}\Gamma=0 Γ T V + V T Γ = 0 and
Γ j = d c 2 ( s ) j d s ∣ s = 0 = d d s Proj T c 1 ( s ) M ( v j ) ∣ s = 0 + Proj T x M ( Γ j ) = Proj T x M ( Γ j ) + I I x ( δ , v j ) , (9) \begin{aligned}
\Gamma_{j}=\left.\frac{\mathrm{d}c_{2}(s)_{j}}{\mathrm{d}s}\right|_{s=0}=\left.\frac{\mathrm{d}}{\mathrm{d}s}\operatorname{Proj}_{T_{c_{1}(s)}\mathcal{M}}(v_{j})\right|_{s=0}+\operatorname{Proj}_{T_{x}\mathcal{M}}(\Gamma_{j}) \\
&=\operatorname{Proj}_{T_{x}\mathcal{M}}(\Gamma_{j})+\mathrm{II}_{x}(\delta,v_{j}), \tag*{(9)}
\end{aligned} Γ j = d s d c 2 ( s ) j s = 0 = d s d Proj T c 1 ( s ) M ( v j ) s = 0 + Proj T x M ( Γ j ) = Proj T x M ( Γ j ) + II x ( δ , v j ) , (9)
for j = 1 , … , k j=1,\ldots,k j = 1 , … , k . Since the columns of V V V and V ⊥ V_{\perp} V ⊥ constitute an orthonormal basis of T x M T_{x}\mathcal{M} T x M , we have
( Proj T x M ( Γ 1 ) , … , Proj T x M ( Γ k ) ) = V A + V ⊥ B \left(\operatorname{Proj}_{T_{x}\mathcal{M}}(\Gamma_{1}),\ldots,\operatorname{Proj}_{T_{x}\mathcal{M}}(\Gamma_{k})\right)=VA+V_{\perp}B ( Proj T x M ( Γ 1 ) , … , Proj T x M ( Γ k ) ) = V A + V ⊥ B
for some A ∈ R k × k A\in\mathbb{R}^{k\times k} A ∈ R k × k and B ∈ R ( d − k ) × k B\in\mathbb{R}^{(d-k)\times k} B ∈ R ( d − k ) × k . Plugging this and Eq. (9) into Γ T V + V T Γ = 0 \Gamma^{\mathsf{T}}V+V^{\mathsf{T}}\Gamma=0 Γ T V + V T Γ = 0 yields A ∈ R a s y m k × k A\in\mathbb{R}_{\mathrm{asym}}^{k\times k} A ∈ R asym k × k because V T V ⊥ = 0 V^{\mathsf{T}}V_{\perp}=0 V T V ⊥ = 0 and I I x ( δ , v j ) ∈ N x M \mathrm{II}_{x}(\delta,v_{j})\in N_{x}\mathcal{M} II x ( δ , v j ) ∈ N x M (see Appendix § 7 ). The proof is complete by noticing the arbitrariness of ( δ , Γ ) (\delta,\Gamma) ( δ , Γ ) .
By the definitions in Eqs. (6) and (7) , there is a natural projection π : St k ( T M ) → Gr k ( T M ) \pi:\operatorname{St}_{k}(T\mathcal{M})\to\operatorname{Gr}_{k}(T\mathcal{M}) π : St k ( T M ) → Gr k ( T M ) , defined as
St k ( T M ) ∋ ( x , V ) ↦ π ( x , V ) : = ( x , V V T ) ∈ Gr k ( T M ) , \operatorname{St}_{k}(T\mathcal{M})\ni(x,V)\mapsto\pi(x,V):=(x,VV^{\mathsf{T}})\in\operatorname{Gr}_{k}(T\mathcal{M}), St k ( T M ) ∋ ( x , V ) ↦ π ( x , V ) := ( x , V V T ) ∈ Gr k ( T M ) ,
which is smooth and surjective, and is a submersion; in fact, we have the quotient structure
Gr k ( T M ) ≅ St k ( T M ) / O ( k ) . (10) \operatorname{Gr}_{k}(T\mathcal{M})\cong\operatorname{St}_{k}(T\mathcal{M})/O(k). \tag*{(10)} Gr k ( T M ) ≅ St k ( T M ) / O ( k ) . (10)
Therefore, the tangent space to Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) at ( x , P ) (x,P) ( x , P ) can be readily obtained by computing the range of D π ( x , V ) D\pi(x,V) D π ( x , V ) , where V ∈ St k ( T x M ) V\in\operatorname{St}_{k}(T_{x}\mathcal{M}) V ∈ St k ( T x M ) satisfies P = V V T P=VV^{\mathsf{T}} P = V V T . This is given in the following lemma without proof.
Lemma 3 (Tangent space to Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) ). Let M \mathcal{M} M be an embedded Riemannian submanifold of R n \mathbb{R}^{n} R n with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d and let 1 ≤ k < d 1\leq k<d 1 ≤ k < d . For any ( x , P ) ∈ Gr k ( T M ) (x,P)\in\operatorname{Gr}_{k}(T\mathcal{M}) ( x , P ) ∈ Gr k ( T M ) , the tangent space
T ( x , P ) Gr k ( T M ) = { ( δ , Δ ) ∣ δ ∈ T x M , Δ = Proj T x M ∘ Δ ∘ Proj T x M + I I ^ x ( δ , P ) } (11) T_{(x,P)}\operatorname{Gr}_{k}(T\mathcal{M})=\left\{(\delta,\Delta)\mid\delta\in T_{x}\mathcal{M},\ \Delta=\operatorname{Proj}_{T_{x}\mathcal{M}}\circ\Delta\circ\operatorname{Proj}_{T_{x}\mathcal{M}}+\widehat{\mathrm{II}}_{x}(\delta,P)\right\} \tag*{(11)} T ( x , P ) Gr k ( T M ) = { ( δ , Δ ) ∣ δ ∈ T x M , Δ = Proj T x M ∘ Δ ∘ Proj T x M + II x ( δ , P ) } (11) = { ( δ , Δ ) ∣ δ ∈ T x M , Δ = V ⊥ B V T + V B T V ⊥ T + I I ^ x ( δ , P ) , B ∈ R ( d − k ) × k } , =\left\{(\delta,\Delta)\mid\delta\in T_{x}\mathcal{M},\ \Delta=V_{\perp}BV^{\mathsf{T}}+VB^{\mathsf{T}}V_{\perp}^{\mathsf{T}}+\widehat{\mathrm{II}}_{x}(\delta,P),\ B\in\mathbb{R}^{(d-k)\times k}\right\}, = { ( δ , Δ ) ∣ δ ∈ T x M , Δ = V ⊥ B V T + V B T V ⊥ T + II x ( δ , P ) , B ∈ R ( d − k ) × k } , where V : = ( v 1 , … , v k ) ∈ St k ( T x M ) V:=(v_{1},\ldots,v_{k})\in\operatorname{St}_{k}(T_{x}\mathcal{M}) V := ( v 1 , … , v k ) ∈ St k ( T x M ) and V ⊥ : = ( v k + 1 , … , v d ) ∈ St d − k ( T x M ) V_{\perp}:=(v_{k+1},\ldots,v_{d})\in\operatorname{St}_{d-k}(T_{x}\mathcal{M}) V ⊥ := ( v k + 1 , … , v d ) ∈ St d − k ( T x M ) satisfy P = V V T P=VV^{\mathsf{T}} P = V V T and V T V ⊥ = 0 V^{\mathsf{T}}V_{\perp}=0 V T V ⊥ = 0 , the operator I I ^ x \widehat{\mathrm{II}}_{x} II x is defined by
I I ^ x ( u , Q ) : = ∑ ℓ = 1 k I I x ( u , u ℓ ) u ℓ T + ∑ ℓ = 1 k u ℓ I I x ( u , u ℓ ) T , (12) \widehat{\mathrm{II}}_{x}(u,Q):=\sum_{\ell=1}^{k}\mathrm{II}_{x}(u,u_{\ell})u_{\ell}^{\mathsf{T}}+\sum_{\ell=1}^{k}u_{\ell}\mathrm{II}_{x}(u,u_{\ell})^{\mathsf{T}}, \tag*{(12)} II x ( u , Q ) := ℓ = 1 ∑ k II x ( u , u ℓ ) u ℓ T + ℓ = 1 ∑ k u ℓ II x ( u , u ℓ ) T , (12)
for any u ∈ T x M u \in\mathrm{T}_x\mathcal{M} u ∈ T x M and Q = U U T ∈ Gr k ( T x M ) Q = UU^{\mathsf{T}} \in\operatorname{Gr}_k(\mathrm{T}_x\mathcal{M}) Q = U U T ∈ Gr k ( T x M ) with U : = ( u 1 , … , u k ) ∈ St k ( T x M ) U := (u_1,\ldots,u_k) \in\operatorname{St}_k(\mathrm{T}_x\mathcal{M}) U := ( u 1 , … , u k ) ∈ St k ( T x M ) . The characterization (11) is independent from the choices of V V V and V ⊥ V_\perp V ⊥ .
A natural basis for T ( x , V ) Gr k ( T M ) \mathrm{T}_{(x,V)}\operatorname{Gr}_k(\mathrm{T}\mathcal{M}) T ( x , V ) Gr k ( T M ) is { ( δ q , Δ q ) } q = 1 d ∪ { ( δ i j , Δ i j ) } i = 1 , … , k , j = k + 1 , … , d \{(\delta_q,\Delta_q)\}_{q=1}^d \cup\{(\delta_{ij},\Delta_{ij})\}_{i=1,\ldots,k,\ j=k+1,\ldots,d} {( δ q , Δ q ) } q = 1 d ∪ {( δ ij , Δ ij ) } i = 1 , … , k , j = k + 1 , … , d , where
( δ q , Δ q ) : = ( v q , Π ^ x ( v q , P ) ) , q = 1 , … , d , (13) (\delta_q,\Delta_q):=\left(v_q,\widehat{\Pi}_x(v_q,P)\right),\qquad q=1,\ldots,d, \tag*{(13)} ( δ q , Δ q ) := ( v q , Π x ( v q , P ) ) , q = 1 , … , d , (13)
and
( δ i j , Δ i j ) : = ( 0 , 1 2 ( v i v j T + v j v i T ) ) , i = 1 , … , k , j = k + 1 , … , d . (14) (\delta_{ij},\Delta_{ij}):=\left(0,\frac{1}{\sqrt{2}}\left(v_i v_j^{\mathsf{T}}+v_j v_i^{\mathsf{T}}\right)\right),\qquad i=1,\ldots,k,\ j=k+1,\ldots,d. \tag*{(14)} ( δ ij , Δ ij ) := ( 0 , 2 1 ( v i v j T + v j v i T ) ) , i = 1 , … , k , j = k + 1 , … , d . (14)
By virtue of the quotient structure (10), each tangent space to St k ( T M ) \operatorname{St}_k(\mathrm{T}\mathcal{M}) St k ( T M ) can be decomposed into vertical and horizontal parts induced by π \pi π .
Lemma 4 (Decomposition of the tangent space to St k ( T M ) \operatorname{St}_k(\mathrm{T}\mathcal{M}) St k ( T M ) ). Let M \mathcal{M} M be an embedded Riemannian submanifold of R n \mathbb{R}^n R n with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d and let 1 ≤ k < d 1\leq k<d 1 ≤ k < d . For any ( x , V ) ∈ St k ( T M ) (x,V)\in\operatorname{St}_k(\mathrm{T}\mathcal{M}) ( x , V ) ∈ St k ( T M ) , the tangent space to St k ( T M ) \operatorname{St}_k(\mathrm{T}\mathcal{M}) St k ( T M ) at ( x , V ) (x,V) ( x , V ) admits a direct sum decomposition as
T ( x , V ) St k ( T M ) = Ver ( x , V ) π St k ( T M ) ⊕ Hor ( x , V ) π St k ( T M ) , \mathrm{T}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M})=\operatorname{Ver}^{\pi}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M})\oplus\operatorname{Hor}^{\pi}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M}), T ( x , V ) St k ( T M ) = Ver ( x , V ) π St k ( T M ) ⊕ Hor ( x , V ) π St k ( T M ) , where
Ver ( x , V ) π St k ( T M ) : = { ( 0 , V A ) ∣ A ∈ R a s y m k × k } , \operatorname{Ver}^{\pi}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M}):=\left\{(0,VA)\mid A\in\mathbb{R}_{\mathrm{asym}}^{k\times k}\right\}, Ver ( x , V ) π St k ( T M ) := { ( 0 , V A ) ∣ A ∈ R asym k × k } , Hor ( x , V ) π St k ( T M ) : = { ( δ , V ⊥ B + ( I I x ( δ , v i ) ) i = 1 k ) ∣ δ ∈ T x M , B ∈ R ( d − k ) × k } \operatorname{Hor}^{\pi}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M}):=\left\{\left(\delta,V_\perp B+\left(\mathop{\mathrm{II}}\nolimits_x(\delta,v_i)\right)_{i=1}^k\right)\mid\delta\in\mathrm{T}_x\mathcal{M},\ B\in\mathbb{R}^{(d-k)\times k}\right\} Hor ( x , V ) π St k ( T M ) := { ( δ , V ⊥ B + ( II x ( δ , v i ) ) i = 1 k ) ∣ δ ∈ T x M , B ∈ R ( d − k ) × k } represent the vertical and horizontal subspaces of T ( x , V ) St k ( T M ) \mathrm{T}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M}) T ( x , V ) St k ( T M ) induced by π \pi π , respectively, and V ⊥ ∈ St d − k ( T x M ) V_\perp\in\operatorname{St}_{d-k}(\mathrm{T}_x\mathcal{M}) V ⊥ ∈ St d − k ( T x M ) satisfies V T V ⊥ = 0 V^{\mathsf{T}}V_\perp=0 V T V ⊥ = 0 . The characterization of the horizontal subspace is independent from the choice of V ⊥ V_\perp V ⊥ . Moreover,
Hor ( x , V ) π St k ( T M ) ≅ T ( x , V V T ) Gr k ( T M ) . \operatorname{Hor}^{\pi}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M})\cong\mathrm{T}_{(x,VV^{\mathsf{T}})}\operatorname{Gr}_k(\mathrm{T}\mathcal{M}). Hor ( x , V ) π St k ( T M ) ≅ T ( x , V V T ) Gr k ( T M ) . For any ( δ , Δ ) ∈ T ( x , V V T ) Gr k ( T M ) (\delta,\Delta)\in\mathrm{T}_{(x,VV^{\mathsf{T}})}\operatorname{Gr}_k(\mathrm{T}\mathcal{M}) ( δ , Δ ) ∈ T ( x , V V T ) Gr k ( T M ) , its horizontal lift is ( δ , Δ V ) ∈ Hor ( x , V ) π St k ( T M ) (\delta,\Delta V)\in\operatorname{Hor}^{\pi}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M}) ( δ , Δ V ) ∈ Hor ( x , V ) π St k ( T M ) .
Proof. By definition, Ver ( x , V ) π St k ( T M ) = ker ( D π ( x , V ) ) \operatorname{Ver}^{\pi}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M})=\ker(\mathrm{D}\pi_{(x,V)}) Ver ( x , V ) π St k ( T M ) = ker ( D π ( x , V ) ) . Since
D π ( x , V ) [ ( δ , Γ ) ] = ( δ , V Γ T + Γ V T ) , ∀ ( δ , Γ ) ∈ T ( x , V ) St k ( T M ) , \mathrm{D}\pi(x,V)[(\delta,\Gamma)]=(\delta,V\Gamma^{\mathsf{T}}+\Gamma V^{\mathsf{T}}),\qquad\forall\,(\delta,\Gamma)\in\mathrm{T}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M}), D π ( x , V ) [( δ , Γ )] = ( δ , V Γ T + Γ V T ) , ∀ ( δ , Γ ) ∈ T ( x , V ) St k ( T M ) ,
and recall the characterization (8), we have for ( δ , Γ ) ∈ Ver ( x , V ) π St k ( T M ) (\delta,\Gamma)\in\operatorname{Ver}^{\pi}_{(x,V)}\operatorname{St}_k(\mathrm{T}\mathcal{M}) ( δ , Γ ) ∈ Ver ( x , V ) π St k ( T M ) that δ = 0 \delta=0 δ = 0 and
V ⊥ B V T + V B T V ⊥ T = 0 ⟺ ( V , V ⊥ ) ( B T B ) ( V T V ⊥ T ) = 0. V_\perp B V^{\mathsf{T}}+V B^{\mathsf{T}}V_\perp^{\mathsf{T}}=0\Longleftrightarrow(V,V_\perp)\begin{pmatrix}B^{\mathsf{T}}\\B\end{pmatrix}\begin{pmatrix}V^{\mathsf{T}}\\V_\perp^{\mathsf{T}}\end{pmatrix}=0. V ⊥ B V T + V B T V ⊥ T = 0 ⟺ ( V , V ⊥ ) ( B T B ) ( V T V ⊥ T ) = 0.
Note that the terms involving the second fundamental form vanish due to δ = 0 \delta=0 δ = 0 . Since the columns of V V V and V ⊥ V_\perp V ⊥ form an orthonormal basis of T x M \mathrm{T}_x\mathcal{M} T x M , the above equation implies that B = 0 B=0 B = 0 . Therefore, Γ = V A \Gamma=VA Γ = V A for some A ∈ R a s y m k × k A\in\mathbb{R}_{\mathrm{asym}}^{k\times k} A ∈ R asym k × k . This verifies the expression for the vertical subspace. Again by the characterization (8), the horizontal subspace is clear since it is the orthogonal complement of the vertical subspace. The proof is complete.
It is known that a natural Riemannian metric for the tangent bundle is the Sasaki metric [69,77]. Basically, the Sasaki metric uses the Riemannian connection of M \mathcal{M} M to split the tangent space to T M \mathrm{T}\mathcal{M} T M at any point into horizontal and vertical subspaces, then defines the inner product using the original metric on each piece. Below, we define a Sasaki-type metric for Gr k ( T M ) \operatorname{Gr}_k(\mathrm{T}\mathcal{M}) Gr k ( T M ) .
Definition 1 (Sasaki-type metric for Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) ). Let M \mathcal{M} M be an embedded Riemannian submanifold of R n \mathbb{R}^{n} R n with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d and let 1 ≤ k < d 1 \le k < d 1 ≤ k < d . For any ( x , P ) ∈ Gr k ( T M ) (x,P) \in\operatorname{Gr}_{k}(T\mathcal{M}) ( x , P ) ∈ Gr k ( T M ) , the Sasaki-type metric on T ( x , P ) Gr k ( T M ) T_{(x,P)}\operatorname{Gr}_{k}(T\mathcal{M}) T ( x , P ) Gr k ( T M ) is defined as
⟨ ( δ , Δ ) , ( δ ~ , Δ ~ ) ⟩ ( x , P ) : = ⟨ δ , δ ~ ⟩ + ⟨ Δ x , Δ ~ x ⟩ , ∀ ( δ , Δ ) , ( δ ~ , Δ ~ ) ∈ T ( x , P ) Gr k ( T M ) , \left\langle(\delta,\Delta),(\widetilde{\delta},\widetilde{\Delta})\right\rangle_{(x,P)} := \langle\delta,\widetilde{\delta}\rangle+ \left\langle\Delta_{x},\widetilde{\Delta}_{x}\right\rangle,\qquad\forall\,(\delta,\Delta),\,(\widetilde{\delta},\widetilde{\Delta}) \in T_{(x,P)}\operatorname{Gr}_{k}(T\mathcal{M}), ⟨ ( δ , Δ ) , ( δ , Δ ) ⟩ ( x , P ) := ⟨ δ , δ ⟩ + ⟨ Δ x , Δ x ⟩ , ∀ ( δ , Δ ) , ( δ , Δ ) ∈ T ( x , P ) Gr k ( T M ) , where
Δ x : = Proj T x M ∘ Δ ∘ Proj T x M , Δ ~ x : = Proj T x M ∘ Δ ~ ∘ Proj T x M . (15) \Delta_{x}:=\operatorname{Proj}_{T_{x}\mathcal{M}}\circ\Delta\circ\operatorname{Proj}_{T_{x}\mathcal{M}},\qquad\widetilde{\Delta}_{x}:=\operatorname{Proj}_{T_{x}\mathcal{M}}\circ\widetilde{\Delta}\circ\operatorname{Proj}_{T_{x}\mathcal{M}}. \tag*{(15)} Δ x := Proj T x M ∘ Δ ∘ Proj T x M , Δ x := Proj T x M ∘ Δ ∘ Proj T x M . (15) Moreover, we denote the norm induced by the Sasaki-type metric with ∥ ∙ ∥ ( x , P ) \|\bullet\|_{(x,P)} ∥ ∙ ∥ ( x , P ) . Then for any ( δ , Δ ) ∈ T ( x , P ) Gr k ( T M ) (\delta,\Delta)\in T_{(x,P)}\operatorname{Gr}_{k}(T\mathcal{M}) ( δ , Δ ) ∈ T ( x , P ) Gr k ( T M ) ,
∥ ( δ , Δ ) ∥ ( x , P ) = ( ∥ δ ∥ 2 + ∥ Δ x ∥ 2 ) 1 2 . \|(\delta,\Delta)\|_{(x,P)}=\left(\|\delta\|^{2}+\|\Delta_{x}\|^{2}\right)^{\frac{1}{2}}. ∥ ( δ , Δ ) ∥ ( x , P ) = ( ∥ δ ∥ 2 + ∥ Δ x ∥ 2 ) 2 1 .
To preserve the feasibility condition in discretized algorithms, we need to define a retraction on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) . One possible choice based on the retraction on M \mathcal{M} M is given in the following lemma.
Lemma 5 (Retraction on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) ). Let M \mathcal{M} M be an embedded Riemannian submanifold of R n \mathbb{R}^{n} R n with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d and let 1 ≤ k < d 1\leq k<d 1 ≤ k < d . Suppose that
R M : T M → M \mathcal{R}^{\mathcal{M}}:T\mathcal{M}\to\mathcal{M} R M : T M → M is a retraction on M \mathcal{M} M ;
for any ( x , P ) ∈ Gr k ( T M ) (x,P)\in\operatorname{Gr}_{k}(T\mathcal{M}) ( x , P ) ∈ Gr k ( T M ) with V ∈ St k ( T x M ) V\in\operatorname{St}_{k}(T_{x}\mathcal{M}) V ∈ St k ( T x M ) such that P = V V T P=VV^{\mathsf{T}} P = V V T , R ^ P , V Gr k ( T x M ) : T P ( Gr k ( T x M ) ) → R n × k \widehat{\mathcal{R}}_{P,V}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}:T_{P}(\operatorname{Gr}_{k}(T_{x}\mathcal{M}))\to\mathbb{R}^{n\times k} R P , V Gr k ( T x M ) : T P ( Gr k ( T x M )) → R n × k is smooth and fulfills
R ^ P , V Gr k ( T x M ) ( 0 ) = V , \widehat{\mathcal{R}}_{P,V}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}(0)=V, R P , V Gr k ( T x M ) ( 0 ) = V , Ran ( R ^ P , V Gr k ( T x M ) ( Δ ) ) ⊆ T x M , rank ( R ^ P , V Gr k ( T x M ) ( Δ ) ) = k , \operatorname{Ran}\left(\widehat{\mathcal{R}}_{P,V}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}(\Delta)\right)\subseteq T_{x}\mathcal{M},\qquad\operatorname{rank}\left(\widehat{\mathcal{R}}_{P,V}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}(\Delta)\right)=k, Ran ( R P , V Gr k ( T x M ) ( Δ ) ) ⊆ T x M , rank ( R P , V Gr k ( T x M ) ( Δ ) ) = k , R ^ P , V Q Gr k ( T x M ) ( Δ ) = R ^ P , V Gr k ( T x M ) ( Δ ) Q , ∀ Q ∈ O ( k ) , \widehat{\mathcal{R}}_{P,VQ}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}(\Delta)=\widehat{\mathcal{R}}_{P,V}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}(\Delta)Q,\qquad\forall\,Q\in\mathcal{O}(k), R P , V Q Gr k ( T x M ) ( Δ ) = R P , V Gr k ( T x M ) ( Δ ) Q , ∀ Q ∈ O ( k ) , D R ^ P , V Gr k ( T x M ) ( 0 ) [ Δ ] V T + V D R ^ P , V Gr k ( T x M ) ( 0 ) [ Δ ] T = Δ , (16) \mathrm{D}\widehat{\mathcal{R}}_{P,V}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}(0)[\Delta]V^{\mathsf{T}}+V\mathrm{D}\widehat{\mathcal{R}}_{P,V}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}(0)[\Delta]^{\mathsf{T}}=\Delta, \tag*{(16)} D R P , V Gr k ( T x M ) ( 0 ) [ Δ ] V T + V D R P , V Gr k ( T x M ) ( 0 ) [ Δ ] T = Δ , (16) for any Δ ∈ T P ( Gr k ( T x M ) ) \Delta\in T_{P}(\operatorname{Gr}_{k}(T_{x}\mathcal{M})) Δ ∈ T P ( Gr k ( T x M )) sufficiently small;
VT M \operatorname{VT}^{\mathcal{M}} VT M is a vector transport associated with R M \mathcal{R}^{\mathcal{M}} R M on M \mathcal{M} M such that, for any ( x , δ ) ∈ T M (x,\delta)\in T\mathcal{M} ( x , δ ) ∈ T M ,
D ( ι ↦ VT ( x , ι ) M ( 0 ) [ δ ] ( v ) ) = I I x ( δ , v ) ∈ N x M (17) \mathrm{D}\left(\iota\mapsto\operatorname{VT}^{\mathcal{M}}_{(x,\iota)}(0)[\delta](v)\right)=\mathrm{II}_{x}(\delta,v)\in N_{x}\mathcal{M} \tag*{(17)} D ( ι ↦ VT ( x , ι ) M ( 0 ) [ δ ] ( v ) ) = II x ( δ , v ) ∈ N x M (17) holds for any v ∈ T x M v\in T_{x}\mathcal{M} v ∈ T x M , and, for any x ∈ M x\in\mathcal{M} x ∈ M , there exists a neighborhood N 1 \mathcal{N}_{1} N 1 of the origin in T x M T_{x}\mathcal{M} T x M such that VT ( x , δ ) M : T x M → T R x M ( δ ) M \operatorname{VT}^{\mathcal{M}}_{(x,\delta)}:T_{x}\mathcal{M}\to T_{\mathcal{R}^{\mathcal{M}}_{x}(\delta)}\mathcal{M} VT ( x , δ ) M : T x M → T R x M ( δ ) M is injective for any δ ∈ N 1 \delta\in\mathcal{N}_{1} δ ∈ N 1 .
Then R Gr k ( T M ) : T ( Gr k ( T M ) ) → Gr k ( T M ) \mathcal{R}^{\operatorname{Gr}_{k}(T\mathcal{M})}:T(\operatorname{Gr}_{k}(T\mathcal{M}))\to\operatorname{Gr}_{k}(T\mathcal{M}) R Gr k ( T M ) : T ( Gr k ( T M )) → Gr k ( T M ) defined as
R ( x , P ) Gr k ( T M ) ( δ , Δ ) : = ( R x M ( δ ) , P ~ x , P ( δ , Δ ) ) , (18) \mathcal{R}_{(x,P)}^{\operatorname{Gr}_{k}(T\mathcal{M})}(\delta,\Delta):=\left(\mathcal{R}_{x}^{\mathcal{M}}(\delta),\widetilde{P}_{x,P}(\delta,\Delta)\right), \tag*{(18)} R ( x , P ) Gr k ( T M ) ( δ , Δ ) := ( R x M ( δ ) , P x , P ( δ , Δ ) ) , (18) for any ( x , P ) , ( δ , Δ ) ∈ T ( Gr k ( T M ) ) (x,P),(\delta,\Delta)\in T(\operatorname{Gr}_{k}(T\mathcal{M})) ( x , P ) , ( δ , Δ ) ∈ T ( Gr k ( T M )) , with
P ~ x , P ( δ , Δ ) : = V ~ x , P , V ( δ , Δ ) V ~ x , P , V ( δ , Δ ) T , (19) \widetilde{P}_{x,P}(\delta,\Delta):=\widetilde{V}_{x,P,V}(\delta,\Delta)\widetilde{V}_{x,P,V}(\delta,\Delta)^{\mathsf{T}}, \tag*{(19)} P x , P ( δ , Δ ) := V x , P , V ( δ , Δ ) V x , P , V ( δ , Δ ) T , (19) V ~ x , P , V ( δ , Δ ) : = Y x , P , V ( δ , Δ ) ( Y x , P , V ( δ , Δ ) T Y x , P , V ( δ , Δ ) ) − 1 / 2 , (20) \widetilde{V}_{x,P,V}(\delta,\Delta):=Y_{x,P,V}(\delta,\Delta)\left(Y_{x,P,V}(\delta,\Delta)^{\mathsf{T}}Y_{x,P,V}(\delta,\Delta)\right)^{-1/2}, \tag*{(20)} V x , P , V ( δ , Δ ) := Y x , P , V ( δ , Δ ) ( Y x , P , V ( δ , Δ ) T Y x , P , V ( δ , Δ ) ) − 1/2 , (20) Y x , P , V ( δ , Δ ) : = VT ( x , δ ) M ( Z x , P , V ( Δ ) ) , (21) Y_{x,P,V}(\delta,\Delta):=\operatorname{VT}^{\mathcal{M}}_{(x,\delta)}\left(Z_{x,P,V}(\Delta)\right), \tag*{(21)} Y x , P , V ( δ , Δ ) := VT ( x , δ ) M ( Z x , P , V ( Δ ) ) , (21) Z x , P , V ( Δ ) : = R ^ P , V Gr k ( T x M ) ( Δ x ) , (22) Z_{x,P,V}(\Delta):=\widehat{\mathcal{R}}_{P,V}^{\operatorname{Gr}_{k}(T_{x}\mathcal{M})}(\Delta_{x}), \tag*{(22)} Z x , P , V ( Δ ) := R P , V Gr k ( T x M ) ( Δ x ) , (22) is a retraction on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) , where VT ( x , δ ) M \operatorname{VT}^{\mathcal{M}}_{(x,\delta)} VT ( x , δ ) M acts columnwisely in Eq. (21) and Δ x \Delta_{x} Δ x is defined in Eq. (15) . Its representative on St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) is
R ( x , V ) Gr k ( T M ) ( δ , Δ V ) : = ( R x M ( δ ) , V ~ x , P , V ( δ , Δ ) ) . (23) \mathcal{R}_{(x,V)}^{\operatorname{Gr}_{k}(T\mathcal{M})}(\delta,\Delta V):=\left(\mathcal{R}_{x}^{\mathcal{M}}(\delta),\widetilde{V}_{x,P,V}(\delta,\Delta)\right). \tag*{(23)} R ( x , V ) Gr k ( T M ) ( δ , Δ V ) := ( R x M ( δ ) , V x , P , V ( δ , Δ ) ) . (23)
Proof. For the ease of notation and unless stated, we do not specify the reference point ( x , P ) (x,P) ( x , P ) or ( x , P , V ) (x,P,V) ( x , P , V ) in the subscripts of P ~ \widetilde{P} P , V ~ \widetilde{V} V , Y Y Y , and Z Z Z in this proof. Given ( x , P ) ∈ Gr k ( T M ) (x,P) \in\operatorname{Gr}_{k}(T\mathcal{M}) ( x , P ) ∈ Gr k ( T M ) , we first prove that R ( x , P ) Gr k ( T M ) R^{\operatorname{Gr}_{k}(T\mathcal{M})}_{(x,P)} R ( x , P ) Gr k ( T M ) in Eq. (18) is well-defined in a neighborhood of the origin. Let N 2 ⊆ T x M \mathcal{N}_{2} \subseteq T_{x}\mathcal{M} N 2 ⊆ T x M be a neighborhood of the origin on which R x M R^{\mathcal{M}}_{x} R x M is well-defined, and let N 3 ⊆ T P Gr k ( T x M ) \mathcal{N}_{3} \subseteq T_{P}\operatorname{Gr}_{k}(T_{x}\mathcal{M}) N 3 ⊆ T P Gr k ( T x M ) be the neighborhood of the origin (which is different from the previous one) such that Eq. (16) holds for any Δ ∈ N 2 \Delta\in\mathcal{N}_{2} Δ ∈ N 2 . Suppose that δ ∈ N 1 ∩ N 2 \delta\in\mathcal{N}_{1} \cap\mathcal{N}_{2} δ ∈ N 1 ∩ N 2 and Δ ∈ N 3 \Delta\in\mathcal{N}_{3} Δ ∈ N 3 . The x x x -part is obvious; hence, we focus on the P P P -part afterwards. From Eq. (22) and the injectivity of VT ( x , δ ) M \operatorname{VT}^{\mathcal{M}}_{(x,\delta)} VT ( x , δ ) M , we have that Y ( δ , Δ ) Y(\delta,\Delta) Y ( δ , Δ ) is of full column rank. Indeed, let a ∈ ker ( Y ( δ , Δ ) ) a \in\operatorname{ker}(Y(\delta,\Delta)) a ∈ ker ( Y ( δ , Δ )) . Then by the linearity of VT ( x , δ ) M \operatorname{VT}^{\mathcal{M}}_{(x,\delta)} VT ( x , δ ) M ,
0 = Y ( δ , Δ ) a = ( VT ( x , δ ) M ( z j ( Δ ) ) ) j = 1 k a = VT ( x , δ ) M ( Z ( Δ ) a ) . 0=Y(\delta,\Delta)a=\left(\operatorname{VT}^{\mathcal{M}}_{(x,\delta)}(z_j(\Delta))\right)_{j=1}^{k}a=\operatorname{VT}^{\mathcal{M}}_{(x,\delta)}(Z(\Delta)a). 0 = Y ( δ , Δ ) a = ( VT ( x , δ ) M ( z j ( Δ )) ) j = 1 k a = VT ( x , δ ) M ( Z ( Δ ) a ) .
Because of the injectivity of VT ( x , δ ) M \operatorname{VT}^{\mathcal{M}}_{(x,\delta)} VT ( x , δ ) M on T x M T_x\mathcal{M} T x M , the above equality implies a ∈ ker ( Z ( Δ ) ) = { 0 } a \in\operatorname{ker}(Z(\Delta))=\{0\} a ∈ ker ( Z ( Δ )) = { 0 } , as desired. Consequently, V ~ ( δ , Δ ) \widetilde{V}(\delta,\Delta) V ( δ , Δ ) is well-defined by Eq. (20) , which in turn yields the well-definedness of P ~ ( δ , Δ ) \widetilde{P}(\delta,\Delta) P ( δ , Δ ) . The independence of P ~ ( δ , Δ ) \widetilde{P}(\delta,\Delta) P ( δ , Δ ) on the choice of V V V can be derived from the equivariance (16) : for any Q ∈ O ( k ) Q \in\mathcal{O}(k) Q ∈ O ( k ) ,
Z x , P , V Q ( Δ ) = R ^ P , V Q Gr k ( T x M ) ( Δ x ) = R ^ P , V Gr k ( T x M ) ( Δ x ) Q = Z x , P , V ( Δ ) Q , Z_{x,P,VQ}(\Delta)=\widehat{R}^{\operatorname{Gr}_{k}(T_x\mathcal{M})}_{P,VQ}(\Delta_x)=\widehat{R}^{\operatorname{Gr}_{k}(T_x\mathcal{M})}_{P,V}(\Delta_x)Q=Z_{x,P,V}(\Delta)Q, Z x , P , V Q ( Δ ) = R P , V Q Gr k ( T x M ) ( Δ x ) = R P , V Gr k ( T x M ) ( Δ x ) Q = Z x , P , V ( Δ ) Q ,
Y x , P , V Q ( δ , Δ ) = VT ( x , δ ) M ( Z x , P , V Q ( Δ ) ) = Y x , P , V ( δ , Δ ) Q , Y_{x,P,VQ}(\delta,\Delta)=\operatorname{VT}^{\mathcal{M}}_{(x,\delta)}(Z_{x,P,VQ}(\Delta))=Y_{x,P,V}(\delta,\Delta)Q, Y x , P , V Q ( δ , Δ ) = VT ( x , δ ) M ( Z x , P , V Q ( Δ )) = Y x , P , V ( δ , Δ ) Q ,
V ~ x , P , V Q ( δ , Δ ) = Y x , P , V ( δ , Δ ) Q ( Q T Y x , P , V ( δ , Δ ) T Y x , P , V ( δ , Δ ) Q ) − 1 / 2 = Y x , P , V ( δ , Δ ) Q Q T ( Y x , P , V ( δ , Δ ) T Y x , P , V ( δ , Δ ) ) − 1 / 2 Q = Y x , P , V ( δ , Δ ) ( Y x , P , V ( δ , Δ ) T Y x , P , V ( δ , Δ ) ) − 1 / 2 Q . \begin{aligned}
\widetilde{V}_{x,P,VQ}(\delta,\Delta)=Y_{x,P,V}(\delta,\Delta)Q\left(Q^{\mathsf{T}}Y_{x,P,V}(\delta,\Delta)^{\mathsf{T}}Y_{x,P,V}(\delta,\Delta)Q\right)^{-1/2} \\
&=Y_{x,P,V}(\delta,\Delta)QQ^{\mathsf{T}}\left(Y_{x,P,V}(\delta,\Delta)^{\mathsf{T}}Y_{x,P,V}(\delta,\Delta)\right)^{-1/2}Q \\
&=Y_{x,P,V}(\delta,\Delta)\left(Y_{x,P,V}(\delta,\Delta)^{\mathsf{T}}Y_{x,P,V}(\delta,\Delta)\right)^{-1/2}Q.
\end{aligned} V x , P , V Q ( δ , Δ ) = Y x , P , V ( δ , Δ ) Q ( Q T Y x , P , V ( δ , Δ ) T Y x , P , V ( δ , Δ ) Q ) − 1/2 = Y x , P , V ( δ , Δ ) Q Q T ( Y x , P , V ( δ , Δ ) T Y x , P , V ( δ , Δ ) ) − 1/2 Q = Y x , P , V ( δ , Δ ) ( Y x , P , V ( δ , Δ ) T Y x , P , V ( δ , Δ ) ) − 1/2 Q .
From these arguments, it is also not difficult to verify that R Gr k ( T M ) R^{\operatorname{Gr}_{k}(T\mathcal{M})} R Gr k ( T M ) is smooth and its codomain is Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) .
Next we prove that R Gr k ( T M ) R^{\operatorname{Gr}_{k}(T\mathcal{M})} R Gr k ( T M ) is indeed a retraction on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) . It is clear that R ( x , P ) Gr k ( T M ) ( 0 , 0 ) = ( x , P ) R^{\operatorname{Gr}_{k}(T\mathcal{M})}_{(x,P)}(0,0)=(x,P) R ( x , P ) Gr k ( T M ) ( 0 , 0 ) = ( x , P ) . Therefore, we focus on its differential at ( 0 , 0 ) (0,0) ( 0 , 0 ) below. The first component is obvious. Regarding the second component, we have by Eqs. (19) –(22) and the chain rule that
D P ~ ( 0 , 0 ) [ δ , Δ ] = D V ~ ( 0 , 0 ) [ δ , Δ ] V T + V D V ~ ( 0 , 0 ) [ δ , Δ ] T , (24) D\widetilde{P}(0,0)[\delta,\Delta]=D\widetilde{V}(0,0)[\delta,\Delta]V^{\mathsf{T}}+VD\widetilde{V}(0,0)[\delta,\Delta]^{\mathsf{T}}, \tag*{(24)} D P ( 0 , 0 ) [ δ , Δ ] = D V ( 0 , 0 ) [ δ , Δ ] V T + V D V ( 0 , 0 ) [ δ , Δ ] T , (24)
D V ~ ( 0 , 0 ) [ δ , Δ ] = D Y ( 0 , 0 ) [ δ , Δ ] + V Θ ( δ , Δ ) , (25) D\widetilde{V}(0,0)[\delta,\Delta]=DY(0,0)[\delta,\Delta]+V\Theta(\delta,\Delta), \tag*{(25)} D V ( 0 , 0 ) [ δ , Δ ] = D Y ( 0 , 0 ) [ δ , Δ ] + V Θ ( δ , Δ ) , (25)
D Y ( 0 , 0 ) [ δ , Δ ] = ( D ( ι ↦ VT ( x , ι ) M ) ( 0 ) [ δ ] ( v j ) ) j = 1 k + D Z ( 0 ) [ Δ ] , (26) DY(0,0)[\delta,\Delta]=\left(D\left(\iota\mapsto\operatorname{VT}^{\mathcal{M}}_{(x,\iota)}\right)(0)[\delta](v_j)\right)_{j=1}^{k}+DZ(0)[\Delta], \tag*{(26)} D Y ( 0 , 0 ) [ δ , Δ ] = ( D ( ι ↦ VT ( x , ι ) M ) ( 0 ) [ δ ] ( v j ) ) j = 1 k + D Z ( 0 ) [ Δ ] , (26)
Δ x = D Z ( 0 ) [ Δ ] V T + V D Z ( 0 ) [ Δ ] T , (27) \Delta_x=DZ(0)[\Delta]V^{\mathsf{T}}+VDZ(0)[\Delta]^{\mathsf{T}}, \tag*{(27)} Δ x = D Z ( 0 ) [ Δ ] V T + V D Z ( 0 ) [ Δ ] T , (27)
where
Θ ( δ , Δ ) : = D ( ( ι , Ξ ) ↦ ( Y ( ι , Ξ ) T Y ( ι , Ξ ) ) − 1 / 2 ) ( 0 , 0 ) [ δ , Δ ] , \Theta(\delta,\Delta):=D\left((\iota,\Xi)\mapsto\left(Y(\iota,\Xi)^{\mathsf{T}}Y(\iota,\Xi)\right)^{-1/2}\right)(0,0)[\delta,\Delta], Θ ( δ , Δ ) := D ( ( ι , Ξ ) ↦ ( Y ( ι , Ξ ) T Y ( ι , Ξ ) ) − 1/2 ) ( 0 , 0 ) [ δ , Δ ] ,
and Eq. (27) is due to condition (16). Note that by Eq. (26) ,
D ( ( ι , Ξ ) ↦ Y ( ι , Ξ ) T Y ( ι , Ξ ) ) ( 0 , 0 ) [ δ , Δ ] = D Y ( 0 , 0 ) [ δ , Δ ] T V + V T D Y ( 0 , 0 ) [ δ , Δ ] = V T ( D ( ι ↦ VT ( x , ι ) M ) ( 0 ) [ δ ] ( v j ) ) j = 1 k + V T D Z ( 0 ) [ Δ ] + h . c . = 0 , \begin{aligned}
D\left((\iota,\Xi)\mapsto Y(\iota,\Xi)^{\mathsf{T}}Y(\iota,\Xi)\right)(0,0)[\delta,\Delta] \\
&=DY(0,0)[\delta,\Delta]^{\mathsf{T}}V+V^{\mathsf{T}}DY(0,0)[\delta,\Delta] \\
&=V^{\mathsf{T}}\left(D\left(\iota\mapsto\operatorname{VT}^{\mathcal{M}}_{(x,\iota)}\right)(0)[\delta](v_j)\right)_{j=1}^{k}+V^{\mathsf{T}}DZ(0)[\Delta]+\mathrm{h.c.}=0,
\end{aligned} D ( ( ι , Ξ ) ↦ Y ( ι , Ξ ) T Y ( ι , Ξ ) ) ( 0 , 0 ) [ δ , Δ ] = D Y ( 0 , 0 ) [ δ , Δ ] T V + V T D Y ( 0 , 0 ) [ δ , Δ ] = V T ( D ( ι ↦ VT ( x , ι ) M ) ( 0 ) [ δ ] ( v j ) ) j = 1 k + V T D Z ( 0 ) [ Δ ] + h.c. = 0 ,
where the last equality uses Eq. (17) and the fact that D Z ( 0 ) [ Δ ] = V A + V ⊥ B DZ(0)[\Delta]=VA+V_{\perp}B D Z ( 0 ) [ Δ ] = V A + V ⊥ B with V ⊥ ∈ St d − k ( T x M ) V_{\perp}\in\operatorname{St}_{d-k}(T_x\mathcal{M}) V ⊥ ∈ St d − k ( T x M ) the orthonormal basis for the orthogonal complement of span ( V ) \operatorname{span}(V) span ( V ) in T x M T_x\mathcal{M} T x M , A ∈ R a s y m k × k A\in\mathbb{R}^{k\times k}_{\mathrm{asym}} A ∈ R asym k × k , and B ∈ R ( d − k ) × k B\in\mathbb{R}^{(d-k)\times k} B ∈ R ( d − k ) × k (cf. [12 ] ). Therefore, for a sufficiently small t > 0 t>0 t > 0 ,
Y ( t δ , t Δ ) T Y ( t δ , t Δ ) = I k + O ( t 2 ) , Y(t\delta,t\Delta)^{\mathsf{T}}Y(t\delta,t\Delta)=I_k+O(t^2), Y ( t δ , t Δ ) T Y ( t δ , t Δ ) = I k + O ( t 2 ) ,
which implies
( Y ( t δ , t Δ ) ⊤ Y ( t δ , t Δ ) ) − 1 / 2 = I k + O ( t 2 ) ⇒ Θ ( δ , Δ ) = 0. \left(Y(t\delta,t\Delta)^{\top}Y(t\delta,t\Delta)\right)^{-1/2}=I_k+O(t^2)\quad\Rightarrow\quad\Theta(\delta,\Delta)=0. ( Y ( t δ , t Δ ) ⊤ Y ( t δ , t Δ ) ) − 1/2 = I k + O ( t 2 ) ⇒ Θ ( δ , Δ ) = 0.
Combining this together with Eqs. (17) and (24) –(27) , we have
D P ~ ( 0 , 0 ) [ δ , Δ ] = D Y ( 0 , 0 ) [ δ , Δ ] V ⊤ + h . c . = ( D ( ι ↦ V T ( x , ι ) M ) ( 0 ) [ δ ] ( v j ) ) j = 1 k V ⊤ + D Z ( 0 ) [ Δ ] V ⊤ + h . c . = I I ^ x ( δ , P ) + Δ x = Δ . \begin{aligned}
D\widetilde{P}(0,0)[\delta,\Delta]=DY(0,0)[\delta,\Delta]V^{\top}+\mathrm{h.c.} \\
&=\left(D\left(\iota\mapsto\mathrm{VT}^{\mathcal{M}}_{(x,\iota)}\right)(0)[\delta](v_j)\right)_{j=1}^{k}V^{\top}+DZ(0)[\Delta]V^{\top}+\mathrm{h.c.} \\
&=\widehat{\mathrm{II}}_x(\delta,P)+\Delta_x=\Delta.
\end{aligned} D P ( 0 , 0 ) [ δ , Δ ] = D Y ( 0 , 0 ) [ δ , Δ ] V ⊤ + h.c. = ( D ( ι ↦ VT ( x , ι ) M ) ( 0 ) [ δ ] ( v j ) ) j = 1 k V ⊤ + D Z ( 0 ) [ Δ ] V ⊤ + h.c. = II x ( δ , P ) + Δ x = Δ.
The last equality follows from Lemma 3. The proof is complete.
Remark 1 (Examples of retractions on Gr k ( T M ) \operatorname{Gr}_k(T\mathcal{M}) Gr k ( T M ) ). The assumption (i) in Lemma 5 is not restrictive. In the following, we pay special attention to the last two assumptions (ii)-(iii) and provide concrete examples.
For the mapping in (ii), one natural choice is any O ( k ) O(k) O ( k ) -equivariant frame lift of a Grassmann retraction. Taking the exponential mapping for instance, we have from [12 ]
Z ( Δ ) = V U cos ( Σ ) U ⊤ + W sin ( Σ ) U ⊤ , ∀ Δ ∈ T P Gr k ( T x M ) , Z(\Delta)=VU\cos(\Sigma)U^{\top}+W\sin(\Sigma)U^{\top},\qquad\forall\ \Delta\in T_P\operatorname{Gr}_k(T_x\mathcal{M}), Z ( Δ ) = V U cos ( Σ ) U ⊤ + W sin ( Σ ) U ⊤ , ∀ Δ ∈ T P Gr k ( T x M ) ,
where W ∈ St k ( T x M ) W\in\operatorname{St}_k(T_x\mathcal{M}) W ∈ St k ( T x M ) , Σ = diag ( σ 1 , … , σ k ) ∈ R + k × k \Sigma=\operatorname{diag}(\sigma_1,\ldots,\sigma_k)\in\mathbb{R}^{k\times k}_{+} Σ = diag ( σ 1 , … , σ k ) ∈ R + k × k , and U ∈ O ( k ) U\in O(k) U ∈ O ( k ) satisfy Δ V = W Σ U ⊤ \Delta V=W\Sigma U^{\top} Δ V = W Σ U ⊤ .
Another (simple) strategy is to take
R ^ P , V Gr k ( T x M ) ( Δ ) = ( P + Δ ) V ∈ R n × k , ∀ ( P , Δ ) ∈ T ( Gr k ( T x M ) ) . \widehat{R}^{\operatorname{Gr}_k(T_x\mathcal{M})}_{P,V}(\Delta)=(P+\Delta)V\in\mathbb{R}^{n\times k},\qquad\forall\ (P,\Delta)\in T(\operatorname{Gr}_k(T_x\mathcal{M})). R P , V Gr k ( T x M ) ( Δ ) = ( P + Δ ) V ∈ R n × k , ∀ ( P , Δ ) ∈ T ( Gr k ( T x M )) .
Note that the range of ( P + Δ ) V (P+\Delta)V ( P + Δ ) V is still a subspace of T x M T_x\mathcal{M} T x M , but ( P + Δ ) V (P+\Delta)V ( P + Δ ) V is not necessarily orthonormalized. Nevertheless, this can be amended by the polar decomposition in Eq. (20) later.
For the mapping in (iii), we provide the following two choices.
Option I: parallel transport. In this case, V T ( x , δ ) M = P T 1 ← 0 c 1 \mathrm{VT}^{\mathcal{M}}_{(x,\delta)}=\mathrm{PT}^{c_1}_{1\leftarrow0} VT ( x , δ ) M = PT 1 ← 0 c 1 , where c 1 ( t ) ∈ M c_1(t)\in\mathcal{M} c 1 ( t ) ∈ M is the smooth curve defined by c 1 ( t ) : = R x M ( t δ ) c_1(t):=R_x^{\mathcal{M}}(t\delta) c 1 ( t ) := R x M ( t δ ) . The invertibility of parallel transport implies the injectivity. Regarding its differential, we first note Proj T c 1 ( t ) M ( V T ( x , t δ ) M ( v ) ) = V T ( x , t δ ) M ( v ) \operatorname{Proj}_{T_{c_1(t)}\mathcal{M}}(\mathrm{VT}^{\mathcal{M}}_{(x,t\delta)}(v))=\mathrm{VT}^{\mathcal{M}}_{(x,t\delta)}(v) Proj T c 1 ( t ) M ( VT ( x , t δ ) M ( v )) = VT ( x , t δ ) M ( v ) . By the chain rule,
D ( ι ↦ V T ( x , ι ) M ) ( 0 ) [ δ ] ( v ) = I I x ( δ , v ) + Proj T x M ( D ( ι ↦ V T ( x , ι ) M ) ( 0 ) [ δ ] ( v ) ) . D\left(\iota\mapsto\mathrm{VT}^{\mathcal{M}}_{(x,\iota)}\right)(0)[\delta](v)=\mathrm{II}_x(\delta,v)+\operatorname{Proj}_{T_x\mathcal{M}}\left(D\left(\iota\mapsto\mathrm{VT}^{\mathcal{M}}_{(x,\iota)}\right)(0)[\delta](v)\right). D ( ι ↦ VT ( x , ι ) M ) ( 0 ) [ δ ] ( v ) = II x ( δ , v ) + Proj T x M ( D ( ι ↦ VT ( x , ι ) M ) ( 0 ) [ δ ] ( v ) ) .
Condition (17) then holds after noticing that the second term on the right-hand side vanishes due to the property of parallel transport.
We remark that, with the vector transport chosen as the parallel transport and Z ( Δ ) ∈ St k ( T x M ) Z(\Delta)\in\operatorname{St}_k(T_x\mathcal{M}) Z ( Δ ) ∈ St k ( T x M ) , the definition of R Gr k ( T M ) R^{\operatorname{Gr}_k(T\mathcal{M})} R Gr k ( T M ) can be simplified. Indeed, due to the fact that the parallel transport is an isometry between tangent spaces, it must hold that Y ( δ , Δ ) ⊤ Y ( δ , Δ ) = I k Y(\delta,\Delta)^{\top}Y(\delta,\Delta)=I_k Y ( δ , Δ ) ⊤ Y ( δ , Δ ) = I k . In addition, if M \mathcal{M} M is Grassmannian and R M R^{\mathcal{M}} R M is chosen to be the exponential mapping on M \mathcal{M} M , V T ( x , δ ) M \mathrm{VT}^{\mathcal{M}}_{(x,\delta)} VT ( x , δ ) M can be explicitly constructed; see, e.g., [12 ] .
Option II: metric projection-based vector transport. In this case, V T ( x , δ ) M = Proj T c 1 ( 1 ) M ∣ T x M \mathrm{VT}^{\mathcal{M}}_{(x,\delta)}=\operatorname{Proj}_{T_{c_1(1)}\mathcal{M}\mid T_x\mathcal{M}} VT ( x , δ ) M = Proj T c 1 ( 1 ) M ∣ T x M , where c 1 ( t ) c_1(t) c 1 ( t ) is the same curve as the one defined in option I. Since ker ( V T ( x , δ ) M ) = T x M ∩ N c ( 1 ) M \ker(\mathrm{VT}^{\mathcal{M}}_{(x,\delta)})=T_x\mathcal{M}\cap N_{c(1)}\mathcal{M} ker ( VT ( x , δ ) M ) = T x M ∩ N c ( 1 ) M , V T ( x , δ ) M \mathrm{VT}^{\mathcal{M}}_{(x,\delta)} VT ( x , δ ) M is injective if and only if the largest principal angle between T x M T_x\mathcal{M} T x M and T c ( 1 ) M T_{c(1)}\mathcal{M} T c ( 1 ) M is less than π / 2 \pi/2 π /2 . In [13 ] , the authors provide an upper bound for this angle using the Riemannian distance between x = c ( 0 ) x=c(0) x = c ( 0 ) and c ( 1 ) c(1) c ( 1 ) , which implies that V T ( x , δ ) M \mathrm{VT}^{\mathcal{M}}_{(x,\delta)} VT ( x , δ ) M in this option is injective as long as ∥ δ ∥ \|\delta\| ∥ δ ∥ is sufficiently small. Regarding the differential, by Eq. (50) , we have
D ( ι ↦ V T ( x , ι ) M ) ( 0 ) [ δ ] ( v ) = P x , δ ( v ) = I I x ( δ , v ) , D\left(\iota\mapsto\mathrm{VT}^{\mathcal{M}}_{(x,\iota)}\right)(0)[\delta](v)=P_{x,\delta}(v)=\mathrm{II}_x(\delta,v), D ( ι ↦ VT ( x , ι ) M ) ( 0 ) [ δ ] ( v ) = P x , δ ( v ) = II x ( δ , v ) ,
which verifies condition (17).
Provided that the assumptions in Lemma 5 are fulfilled, different definitions of retractions on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) agree up to the first order and may differ locally from the second order. This discrepancy does not affect the framework of convergence analysis in § 3 , which exploits merely first-order properties. Nevertheless, there can be distinctions in the involved constants and numerical performance (see Appendix § 8 for an illustration).
Constrained saddle dynamics on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M )
Equipped with the above geometrical tools of Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) , we are ready to develop the constrained saddle dynamics (CSD). Since ( x ( t ) , P ( t ) ) ∈ Gr k ( T M ) (x(t),P(t))\in\operatorname{Gr}_{k}(T\mathcal{M}) ( x ( t ) , P ( t )) ∈ Gr k ( T M ) for any t t t , we require from Eq. (11) that
d x d t ( t ) ∈ T x ( t ) M , d P d t ( t ) = Proj T x ( t ) M ∘ d P d t ( t ) ∘ Proj T x ( t ) M + I I ^ x ( d x d t ( t ) , P ( t ) ) (28) \frac{\mathrm{d}x}{\mathrm{d}t}(t)\in T_{x(t)}\mathcal{M},\qquad\frac{\mathrm{d}P}{\mathrm{d}t}(t)=\operatorname{Proj}_{T_{x(t)}\mathcal{M}}\circ\frac{\mathrm{d}P}{\mathrm{d}t}(t)\circ\operatorname{Proj}_{T_{x(t)}\mathcal{M}}+\widehat{\mathrm{II}}_{x}\left(\frac{\mathrm{d}x}{\mathrm{d}t}(t),P(t)\right) \tag*{(28)} d t d x ( t ) ∈ T x ( t ) M , d t d P ( t ) = Proj T x ( t ) M ∘ d t d P ( t ) ∘ Proj T x ( t ) M + II x ( d t d x ( t ) , P ( t ) ) (28)
hold for all the time, where I I ^ x \widehat{\mathrm{II}}_{x} II x is defined in Eq. (12) . To achieve this, we set
d x d t ( t ) : = − R x ( t ) , P ( t ) ( grad M f ( x ( t ) ) ) , \frac{\mathrm{d}x}{\mathrm{d}t}(t):=-R_{x(t),P(t)}\left(\operatorname{grad}_{\mathcal{M}}f(x(t))\right), d t d x ( t ) := − R x ( t ) , P ( t ) ( grad M f ( x ( t )) ) ,
d P d t ( t ) : = − Proj T P ( t ) Gr k ( T x ( t ) M ) ∘ Hess M f ( x ( t ) ) + I I ^ x ( t ) ( d x d t ( t ) , P ( t ) ) , (29) \frac{\mathrm{d}P}{\mathrm{d}t}(t):=-\operatorname{Proj}_{T_{P(t)}\operatorname{Gr}_{k}(T_{x(t)}\mathcal{M})}\circ\operatorname{Hess}_{\mathcal{M}}f(x(t))+\widehat{\mathrm{II}}_{x(t)}\left(\frac{\mathrm{d}x}{\mathrm{d}t}(t),P(t)\right), \tag*{(29)} d t d P ( t ) := − Proj T P ( t ) Gr k ( T x ( t ) M ) ∘ Hess M f ( x ( t )) + II x ( t ) ( d t d x ( t ) , P ( t ) ) , (29)
where R x ( t ) , P ( t ) R_{x(t),P(t)} R x ( t ) , P ( t ) is the reflection operator defined by P ( t ) P(t) P ( t ) on T x ( t ) M T_{x(t)}\mathcal{M} T x ( t ) M :
R x ( t ) , P ( t ) ( u ) : = u − 2 P ( t ) u , ∀ u ∈ T x ( t ) M , (30) R_{x(t),P(t)}(u):=u-2P(t)u,\qquad\forall u\in T_{x(t)}\mathcal{M}, \tag*{(30)} R x ( t ) , P ( t ) ( u ) := u − 2 P ( t ) u , ∀ u ∈ T x ( t ) M , (30)
and Proj T P ( t ) Gr k ( T x ( t ) M ) \operatorname{Proj}_{T_{P(t)}\operatorname{Gr}_{k}(T_{x(t)}\mathcal{M})} Proj T P ( t ) Gr k ( T x ( t ) M ) refers to the orthogonal projection operator onto the tangent space to Gr k ( T x ( t ) M ) \operatorname{Gr}_{k}(T_{x(t)}\mathcal{M}) Gr k ( T x ( t ) M ) at P ( t ) P(t) P ( t ) , which has a closed-form expression by, e.g., [12 ] ,
Proj T P ( t ) Gr k ( T x ( t ) M ) ( M ) : = [ P ( t ) , [ P ( t ) , M ] ] , (31) \operatorname{Proj}_{T_{P(t)}\operatorname{Gr}_{k}(T_{x(t)}\mathcal{M})}(M):=[P(t),[P(t),M]], \tag*{(31)} Proj T P ( t ) Gr k ( T x ( t ) M ) ( M ) := [ P ( t ) , [ P ( t ) , M ]] , (31)
for any symmetric bilinear form M M M on T x ( t ) M T_{x(t)}\mathcal{M} T x ( t ) M . The x x x -part is the same as in Eq. (4) except that we adopt the orthogonal projector. The first ingredient of the P P P -part is responsible for tracking the lowest k k k -dimensional invariant subspace of Hess M f ( x ( t ) ) \operatorname{Hess}_{\mathcal{M}}f(x(t)) Hess M f ( x ( t )) in T x ( t ) M T_{x(t)}\mathcal{M} T x ( t ) M , following the Euler–Lagrange equation of the Rayleigh quotient minimization,
min P ~ Tr ( P ~ ∘ Hess M f ( x ( t ) ) ) , s.t. P ~ ∈ Gr k ( T x ( t ) M ) , \min_{\widetilde{P}}\operatorname{Tr}\left(\widetilde{P}\circ\operatorname{Hess}_{\mathcal{M}}f(x(t))\right),\qquad\text{s.t. }\widetilde{P}\in\operatorname{Gr}_{k}(T_{x(t)}\mathcal{M}), P min Tr ( P ∘ Hess M f ( x ( t )) ) , s.t. P ∈ Gr k ( T x ( t ) M ) ,
while the second ingredient accounts for the changes in the tangent spaces, as explained in Appendix § 7 . The dynamics (29) can be horizontally lifted to a dynamics on St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) as shown in Lemma 4:
d x d t ( t ) : = − R x ( t ) , V ( t ) ( grad M f ( x ( t ) ) ) , \frac{\mathrm{d}x}{\mathrm{d}t}(t):=-R_{x(t),V(t)}\left(\operatorname{grad}_{\mathcal{M}}f(x(t))\right), d t d x ( t ) := − R x ( t ) , V ( t ) ( grad M f ( x ( t )) ) ,
d v i d t ( t ) : = − Proj x ( t ) , V ( t ) ⊥ ( Hess M f ( x ( t ) ) [ v i ( t ) ] ) + I I x ( t ) ( d x d t ( t ) , v i ( t ) ) , \frac{\mathrm{d}v_i}{\mathrm{d}t}(t):=-\operatorname{Proj}_{x(t),V(t)^\perp}\left(\operatorname{Hess}_{\mathcal{M}}f(x(t))[v_i(t)]\right)+\mathrm{II}_{x(t)}\left(\frac{\mathrm{d}x}{\mathrm{d}t}(t),v_i(t)\right), d t d v i ( t ) := − Proj x ( t ) , V ( t ) ⊥ ( Hess M f ( x ( t )) [ v i ( t )] ) + II x ( t ) ( d t d x ( t ) , v i ( t ) ) ,
i = 1 , … , k , with V ( t ) : = ( v 1 ( t ) , … , v k ( t ) ) . (32) i=1,\ldots,k,\qquad\text{with }V(t):=(v_1(t),\ldots,v_k(t)). \tag*{(32)} i = 1 , … , k , with V ( t ) := ( v 1 ( t ) , … , v k ( t )) . (32)
If the submanifolds (3) are considered, the second fundamental form in Eq. (32) recovers the third term of the right-hand side of Eq. (5) . However, thanks to the explicit characterization of the tangent space to Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) in Lemma 3, our methodology does not rely on explicit global/local level-set representations of M \mathcal{M} M since we only require embedded-submanifold geometric primitives (e.g., T x M T_x\mathcal{M} T x M , Proj T x M \operatorname{Proj}_{T_x\mathcal{M}} Proj T x M , and I I x \mathrm{II}_x II x ). Some examples are listed as follows.
Example 1 (Second fundamental form in special cases).
Riemannian manifolds induced by global defining functions: E = R n \mathcal{E} = \mathbb{R}^{n} E = R n , M \mathcal{M} M is defined in Eq. (3) [91 , 94 , 98 ] [54, 56]. For any x ∈ M x \in\mathcal{M} x ∈ M , we have
T x M = { v ∈ R n ∣ grad c ( x ) v = 0 } , T_{x}\mathcal{M} = \{v \in\mathbb{R}^{n} \mid\operatorname{grad} c(x)v = 0\}, T x M = { v ∈ R n ∣ grad c ( x ) v = 0 } , Proj T x M ( v ) = ( I n − grad c ( x ) T ( grad c ( x ) ⋅ grad c ( x ) T ) − 1 grad c ( x ) ) v \operatorname{Proj}_{T_{x}\mathcal{M}}(v) = \left(I_{n} - \operatorname{grad} c(x)^{\mathsf{T}}\left(\operatorname{grad} c(x) \cdot\operatorname{grad} c(x)^{\mathsf{T}}\right)^{-1}\operatorname{grad} c(x)\right)v Proj T x M ( v ) = ( I n − grad c ( x ) T ( grad c ( x ) ⋅ grad c ( x ) T ) − 1 grad c ( x ) ) v for any v ∈ E v \in\mathcal{E} v ∈ E . Therefore, by Eq. (50) , for any u , v ∈ T x M u, v \in T_{x}\mathcal{M} u , v ∈ T x M ,
II x ( u , v ) = − ( Hess c ( x ) [ u ] ) T ( grad c ( x ) ⋅ grad c ( x ) T ) − 1 grad c ( x ) v − grad c ( x ) T D ( y ↦ ( grad c ( y ) ⋅ grad c ( y ) T ) − 1 ) ( x ) [ u ] ⋅ grad c ( x ) v − grad c ( x ) T ( grad c ( x ) ⋅ grad c ( x ) T ) − 1 ( Hess c ( x ) [ u ] ) v = − grad c ( x ) T ( grad c ( x ) ⋅ grad c ( x ) T ) − 1 ( Hess c ( x ) [ u ] ) v , \begin{aligned}
\operatorname{II}_{x}(u,v) ={}& -(\operatorname{Hess} c(x)[u])^{\mathsf{T}}\left(\operatorname{grad} c(x) \cdot\operatorname{grad} c(x)^{\mathsf{T}}\right)^{-1}\operatorname{grad} c(x)v \\
&- \operatorname{grad} c(x)^{\mathsf{T}}D\left(y \mapsto\left(\operatorname{grad} c(y) \cdot\operatorname{grad} c(y)^{\mathsf{T}}\right)^{-1}\right)(x)[u] \cdot\operatorname{grad} c(x)v \\
&- \operatorname{grad} c(x)^{\mathsf{T}}\left(\operatorname{grad} c(x) \cdot\operatorname{grad} c(x)^{\mathsf{T}}\right)^{-1}(\operatorname{Hess} c(x)[u])v \\
={}&-\operatorname{grad} c(x)^{\mathsf{T}}\left(\operatorname{grad} c(x) \cdot\operatorname{grad} c(x)^{\mathsf{T}}\right)^{-1}(\operatorname{Hess} c(x)[u])v,
\end{aligned} II x ( u , v ) = = − ( Hess c ( x ) [ u ] ) T ( grad c ( x ) ⋅ grad c ( x ) T ) − 1 grad c ( x ) v − grad c ( x ) T D ( y ↦ ( grad c ( y ) ⋅ grad c ( y ) T ) − 1 ) ( x ) [ u ] ⋅ grad c ( x ) v − grad c ( x ) T ( grad c ( x ) ⋅ grad c ( x ) T ) − 1 ( Hess c ( x ) [ u ]) v − grad c ( x ) T ( grad c ( x ) ⋅ grad c ( x ) T ) − 1 ( Hess c ( x ) [ u ]) v , where the last equality is due to grad c ( x ) v = 0 \operatorname{grad} c(x)v = 0 grad c ( x ) v = 0 . Notably, this recovers the third term on the right-hand side of Eq. (5) if we let u = d x d t ( t ) u = \frac{d x}{d t}(t) u = d t d x ( t ) . Some special cases falling into this class and the corresponding second fundamental forms are listed as follows:
Flat space: M = E \mathcal{M} = \mathcal{E} M = E , II x ≡ 0 \operatorname{II}_{x} \equiv0 II x ≡ 0 .
Sphere: E = R d + 1 \mathcal{E} = \mathbb{R}^{d+1} E = R d + 1 , M = S d \mathcal{M} = \mathbb{S}^{d} M = S d ,
II x ( u , v ) = − ⟨ u , v ⟩ x , ∀ x ∈ M , u , v ∈ T x M . \operatorname{II}_{x}(u,v) = -\langle u,v\rangle x,\quad\forall x \in\mathcal{M},\ u,v \in T_{x}\mathcal{M}. II x ( u , v ) = − ⟨ u , v ⟩ x , ∀ x ∈ M , u , v ∈ T x M . II X ( U , V ) = − 1 2 X ( U T V + V T U ) , ∀ X ∈ St p ( R n ) , U , V ∈ T X M . \operatorname{II}_{X}(U,V) = -\frac{1}{2}X(U^{\mathsf{T}}V + V^{\mathsf{T}}U),\quad\forall X \in\operatorname{St}_{p}(\mathbb{R}^{n}),\ U,V \in T_{X}\mathcal{M}. II X ( U , V ) = − 2 1 X ( U T V + V T U ) , ∀ X ∈ St p ( R n ) , U , V ∈ T X M .
Grassmann manifold: E = R s y m n × n \mathcal{E} = \mathbb{R}_{\mathrm{sym}}^{n \times n} E = R sym n × n , M = Gr p ( R n ) \mathcal{M} = \operatorname{Gr}_{p}(\mathbb{R}^{n}) M = Gr p ( R n ) . For any P ∈ M P \in\mathcal{M} P ∈ M , we have from [12 ] that
T P M = { Γ ∈ R s y m n × n ∣ Γ P + P Γ = Γ } , Proj T P M ( Γ ) = [ P , [ P , Γ ] ] , ∀ Γ ∈ E . T_{P}\mathcal{M} = \{\Gamma\in\mathbb{R}_{\mathrm{sym}}^{n \times n} \mid\Gamma P + P\Gamma= \Gamma\},\quad\operatorname{Proj}_{T_{P}\mathcal{M}}(\Gamma) = [P,[P,\Gamma]],\quad\forall\Gamma\in\mathcal{E}. T P M = { Γ ∈ R sym n × n ∣ Γ P + P Γ = Γ } , Proj T P M ( Γ ) = [ P , [ P , Γ ]] , ∀Γ ∈ E .
Therefore, by Eq. (50) , for any Γ , Δ ∈ T P M \Gamma,\Delta\in T_{P}\mathcal{M} Γ , Δ ∈ T P M ,
II P ( Δ , Γ ) = [ Δ , [ P , Γ ] ] + [ P , [ Δ , Γ ] ] = [ Δ , [ P , Γ ] ] . \operatorname{II}_{P}(\Delta,\Gamma) = [\Delta,[P,\Gamma]] + [P,[\Delta,\Gamma]] = [\Delta,[P,\Gamma]]. II P ( Δ , Γ ) = [ Δ , [ P , Γ ]] + [ P , [ Δ , Γ ]] = [ Δ , [ P , Γ ]] .
The last equality is due to
[ P , [ Δ , Γ ] ] = [ P , [ Δ P + P Δ , Γ P + P Γ ] ] = [ P , Δ P Γ + P Δ Γ P − Γ P Δ − P Γ Δ P ] = P Δ Γ P − P Γ Δ P − P Δ Γ P + P Γ Δ P = 0 , \begin{aligned}
[P,[\Delta,\Gamma]] &= [P,[\Delta P + P\Delta,\Gamma P + P\Gamma]] \\
&= [P,\Delta P\Gamma+ P\Delta\Gamma P - \Gamma P\Delta- P\Gamma\Delta P] \\
&= P\Delta\Gamma P - P\Gamma\Delta P - P\Delta\Gamma P + P\Gamma\Delta P = 0,
\end{aligned} [ P , [ Δ , Γ ]] = [ P , [ Δ P + P Δ , Γ P + P Γ ]] = [ P , Δ P Γ + P ΔΓ P − Γ P Δ − P ΓΔ P ] = P ΔΓ P − P ΓΔ P − P ΔΓ P + P ΓΔ P = 0 ,
where we have used Γ P + P Γ = Γ \Gamma P + P\Gamma= \Gamma Γ P + P Γ = Γ , Δ P + P Δ = Δ \Delta P + P\Delta= \Delta Δ P + P Δ = Δ , P Γ P = 0 P\Gamma P = 0 P Γ P = 0 , and P Δ P = 0 P\Delta P = 0 P Δ P = 0 .
Manifold of fixed-rank matrices: E = R m × n \mathcal{E} = \mathbb{R}^{m \times n} E = R m × n , M = R r m × n \mathcal{M} = \mathbb{R}_{r}^{m \times n} M = R r m × n . For any X ∈ M X \in\mathcal{M} X ∈ M , let X = U Σ V T X = U\Sigma V^{\mathsf{T}} X = U Σ V T be its singular value decomposition, where U ∈ St r ( R m ) U \in\operatorname{St}_{r}(\mathbb{R}^{m}) U ∈ St r ( R m ) , Σ = diag ( σ 1 , … , σ r ) ∈ R + + r × r \Sigma= \operatorname{diag}(\sigma_{1},\ldots,\sigma_{r}) \in\mathbb{R}_{++}^{r \times r} Σ = diag ( σ 1 , … , σ r ) ∈ R ++ r × r , and V ∈ St r ( R n ) V \in\operatorname{St}_{r}(\mathbb{R}^{n}) V ∈ St r ( R n ) . We have from [15 Section 7.5] that
T X M = { ( U U ⊥ ) ( A B C 0 ) ( V T V ⊥ T ) | A ∈ R r × r , B ∈ R r × ( n − r ) , C ∈ R ( m − r ) × r } , T_{X}\mathcal{M} = \left\{(U\ \ U_{\perp})\begin{pmatrix} A & B \\ C & 0 \end{pmatrix}\begin{pmatrix} V^{\mathsf{T}} \\ V_{\perp}^{\mathsf{T}} \end{pmatrix} \,\middle|\, A \in\mathbb{R}^{r \times r},\ B \in\mathbb{R}^{r \times(n-r)},\ C \in\mathbb{R}^{(m-r) \times r}\right\}, T X M = { ( U U ⊥ ) ( A C B 0 ) ( V T V ⊥ T ) A ∈ R r × r , B ∈ R r × ( n − r ) , C ∈ R ( m − r ) × r } ,
Proj T X M ( Z ) = Z − ( I m − U U T ) Z ( I n − V V T ) , ∀ Z ∈ E . \operatorname{Proj}_{T_X\mathcal{M}}(Z)=Z-(I_m-UU^{\mathsf{T}})Z(I_n-VV^{\mathsf{T}}),\qquad\forall Z\in\mathcal{E}. Proj T X M ( Z ) = Z − ( I m − U U T ) Z ( I n − V V T ) , ∀ Z ∈ E .
Note that for any smooth curve c : R ⊃ I → M c:\mathbb{R}\supset I\to\mathcal{M} c : R ⊃ I → M satisfying X ( 0 ) = X X(0)=X X ( 0 ) = X and X ′ ( 0 ) = Ψ ∈ T X M X'(0)=\Psi\in T_X\mathcal{M} X ′ ( 0 ) = Ψ ∈ T X M , there exist smooth mappings U ( t ) ∈ St ( R m ) , Σ ( t ) ∈ R r × r U(t)\in\operatorname{St}(\mathbb{R}^{m}),\ \Sigma(t)\in\mathbb{R}^{r\times r} U ( t ) ∈ St ( R m ) , Σ ( t ) ∈ R r × r , and V ( t ) ∈ St ( R n ) V(t)\in\operatorname{St}(\mathbb{R}^{n}) V ( t ) ∈ St ( R n ) such that X ( t ) = U ( t ) Σ ( t ) V ( t ) T X(t)=U(t)\Sigma(t)V(t)^{\mathsf{T}} X ( t ) = U ( t ) Σ ( t ) V ( t ) T and U ( 0 ) = U , Σ ( 0 ) = Σ , V ( 0 ) = V U(0)=U,\ \Sigma(0)=\Sigma,\ V(0)=V U ( 0 ) = U , Σ ( 0 ) = Σ , V ( 0 ) = V hold in a small neighborhood of the origin. Note that Σ ( t ) \Sigma(t) Σ ( t ) is invertible but need not be diagonal. By the product rule,
U ′ ( 0 ) Σ V T + U Σ ′ ( 0 ) V T + U Σ V ′ ( 0 ) T = X ′ ( 0 ) = Ψ , U'(0)\Sigma V^{\mathsf{T}}+U\Sigma'(0)V^{\mathsf{T}}+U\Sigma V'(0)^{\mathsf{T}}=X'(0)=\Psi, U ′ ( 0 ) Σ V T + U Σ ′ ( 0 ) V T + U Σ V ′ ( 0 ) T = X ′ ( 0 ) = Ψ ,
which yields
Σ ′ ( 0 ) = U T Ψ V , U ′ ( 0 ) = ( I m − U U T ) Ψ V Σ − 1 , V ′ ( 0 ) = ( I n − V V T ) Ψ T U Σ − 1 . \Sigma'(0)=U^{\mathsf{T}}\Psi V,\qquad U'(0)=(I_m-UU^{\mathsf{T}})\Psi V\Sigma^{-1},\qquad V'(0)=(I_n-VV^{\mathsf{T}})\Psi^{\mathsf{T}}U\Sigma^{-{\!1}}. Σ ′ ( 0 ) = U T Ψ V , U ′ ( 0 ) = ( I m − U U T ) Ψ V Σ − 1 , V ′ ( 0 ) = ( I n − V V T ) Ψ T U Σ − 1 .
Therefore, by Eq. (50) , for any Φ , Ψ ∈ T X M \Phi,\Psi\in T_X\mathcal{M} Φ , Ψ ∈ T X M ,
I I X ( Ψ , Φ ) = ( ( I m − U U T ) Ψ V Σ − 1 U T + U Σ − 1 V T Ψ T ( I m − U U T ) ) Φ ( I n − V V T ) + ( I m − U U T ) Φ ( ( I n − V V T ) Ψ T U Σ − 1 V T + V Σ − 1 U T Ψ ( I n − V V T ) ) = ( I m − U U T ) ( Ψ V Σ − 1 U T Φ + Φ V Σ − 1 U T Ψ ) ( I n − V V T ) , \begin{aligned}
\mathop{\mathrm{II}}\nolimits_X(\Psi,\Phi)
={}&\left((I_m-UU^{\mathsf{T}})\Psi V\Sigma^{-1}U^{\mathsf{T}}+U\Sigma^{-1}V^{\mathsf{T}}\Psi^{\mathsf{T}}(I_m-UU^{\mathsf{T}})\right)\Phi(I_n-VV^{\mathsf{T}})\\
&+(I_m-UU^{\mathsf{T}})\Phi\left((I_n-VV^{\mathsf{T}})\Psi^{\mathsf{T}}U\Sigma^{-1}V^{\mathsf{T}}+V\Sigma^{-1}U^{\mathsf{T}}\Psi(I_n-VV^{\mathsf{T}})\right)\\
={}&(I_m-UU^{\mathsf{T}})\left(\Psi V\Sigma^{-1}U^{\mathsf{T}}\Phi+\Phi V\Sigma^{-1}U^{\mathsf{T}}\Psi\right)(I_n-VV^{\mathsf{T}}),
\end{aligned} II X ( Ψ , Φ ) = = ( ( I m − U U T ) Ψ V Σ − 1 U T + U Σ − 1 V T Ψ T ( I m − U U T ) ) Φ ( I n − V V T ) + ( I m − U U T ) Φ ( ( I n − V V T ) Ψ T U Σ − 1 V T + V Σ − 1 U T Ψ ( I n − V V T ) ) ( I m − U U T ) ( Ψ V Σ − 1 U T Φ + Φ V Σ − 1 U T Ψ ) ( I n − V V T ) ,
where the last equality is due to the fact that ( I m − U U T ) Φ ( I n − V V T ) = 0 (I_m-UU^{\mathsf{T}})\Phi(I_n-VV^{\mathsf{T}})=0 ( I m − U U T ) Φ ( I n − V V T ) = 0 .
Theorem 1 (Global well-definedness of the dynamics (29)). Suppose that M \mathcal{M} M is a compact embedded Riemannian submanifold of R n \mathbb{R}^{n} R n with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d and let 1 ≤ k < d 1\le k<d 1 ≤ k < d . If ( x ( 0 ) , P ( 0 ) ) ∈ Gr k ( T M ) (x(0),P(0))\in\operatorname{Gr}_k(T\mathcal{M}) ( x ( 0 ) , P ( 0 )) ∈ Gr k ( T M ) , then the trajectory generated by the dynamics (29) always stays on Gr k ( T M ) \operatorname{Gr}_k(T\mathcal{M}) Gr k ( T M ) .
Proof. Since M \mathcal{M} M is compact in E \mathcal{E} E , so is Gr k ( T M ) \operatorname{Gr}_k(T\mathcal{M}) Gr k ( T M ) in its ambient space. Indeed,
Gr k ( T M ) = { ( x , P ) ∣ x ∈ M , P ∈ Gr k ( R n ) , Proj T x M ∘ P ∘ Proj T x M = P } , \operatorname{Gr}_k(T\mathcal{M})=\left\{(x,P)\mid x\in\mathcal{M},\ P\in\operatorname{Gr}_k(\mathbb{R}^{n}),\ \operatorname{Proj}_{T_x\mathcal{M}}\circ P\circ\operatorname{Proj}_{T_x\mathcal{M}}=P\right\}, Gr k ( T M ) = { ( x , P ) ∣ x ∈ M , P ∈ Gr k ( R n ) , Proj T x M ∘ P ∘ Proj T x M = P } , which is a bounded subset in M × R s y m n × n \mathcal{M}\times\mathbb{R}_{\mathrm{sym}}^{n\times n} M × R sym n × n . Its closedness is due to the fact that the map ( x , P ) ↦ Proj T x M ∘ P ∘ Proj T x M − P (x,P)\mapsto\operatorname{Proj}_{T_x\mathcal{M}}\circ P\circ\operatorname{Proj}_{T_x\mathcal{M}}-P ( x , P ) ↦ Proj T x M ∘ P ∘ Proj T x M − P is continuous. The conclusion then follows directly from [49, Corollary 9.17].
Algorithm 1 Discretized CSD on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) .
Input: Initial point ( x ( 0 ) , P ( 0 ) ) ∈ Gr k ( T M ) (x^{(0)},P^{(0)}) \in\operatorname{Gr}_{k}(T\mathcal{M}) ( x ( 0 ) , P ( 0 ) ) ∈ Gr k ( T M ) , step size η > 0 \eta> 0 η > 0 , maximum iteration number m a x i t ∈ N maxit \in\mathbb{N} ma x i t ∈ N , convergence tolerance t o l > 0 tol > 0 t o l > 0 .
Set t : = 0 t := 0 t := 0 , r x ( 0 ) = r P ( 0 ) : = ∞ r_{x}^{(0)} = r_{P}^{(0)} := \infty r x ( 0 ) = r P ( 0 ) := ∞ .
while t < m a x i t t < maxit t < ma x i t and max { r x ( t ) , r P ( t ) } > t o l \max\{r_{x}^{(t)},r_{P}^{(t)}\} > tol max { r x ( t ) , r P ( t ) } > t o l do
Compute the x x x -direction:
d x ( t ) : = − R x ( t ) , P ( t ) ( grad M f ( x ( t ) ) ) ∈ T x ( t ) M . d_{x}^{(t)} := -R_{x^{(t)},P^{(t)}}\left(\operatorname{grad}_{\mathcal{M}} f(x^{(t)})\right) \in T_{x^{(t)}}\mathcal{M}. d x ( t ) := − R x ( t ) , P ( t ) ( grad M f ( x ( t ) ) ) ∈ T x ( t ) M .
Compute the P P P -direction:
d P ( t ) : = − Proj T P ( t ) Gr k ( T x ( t ) M ) ∘ Hess M f ( x ( t ) ) ∈ T P ( t ) Gr k ( T x ( t ) M ) . d_{P}^{(t)} := -\operatorname{Proj}_{T_{P^{(t)}}\operatorname{Gr}_{k}(T_{x^{(t)}}\mathcal{M})} \circ\operatorname{Hess}_{\mathcal{M}} f(x^{(t)}) \in T_{P^{(t)}}\operatorname{Gr}_{k}(T_{x^{(t)}}\mathcal{M}). d P ( t ) := − Proj T P ( t ) Gr k ( T x ( t ) M ) ∘ Hess M f ( x ( t ) ) ∈ T P ( t ) Gr k ( T x ( t ) M ) .
Update x x x and P P P using the retraction defined in Eq. (18) :
( x ( t + 1 ) , P ( t + 1 ) ) : = R ( x ( t ) , P ( t ) ) Gr k ( T M ) ( η d x ( t ) , η ( d P ( t ) + I I ^ x ( t ) ( d x ( t ) , P ( t ) ) ) ) ∈ Gr k ( T M ) . (x^{(t+1)},P^{(t+1)}) := R_{(x^{(t)},P^{(t)})}^{\operatorname{Gr}_{k}(T\mathcal{M})}\left(\eta d_{x}^{(t)},\eta\left(d_{P}^{(t)}+\widehat{\mathrm{II}}_{x^{(t)}}(d_{x}^{(t)},P^{(t)})\right)\right) \in\operatorname{Gr}_{k}(T\mathcal{M}). ( x ( t + 1 ) , P ( t + 1 ) ) := R ( x ( t ) , P ( t ) ) Gr k ( T M ) ( η d x ( t ) , η ( d P ( t ) + II x ( t ) ( d x ( t ) , P ( t ) ) ) ) ∈ Gr k ( T M ) .
if t = 0 t = 0 t = 0 then
Set r x ( t + 1 ) = r P ( t + 1 ) : = 1 r_{x}^{(t+1)} = r_{P}^{(t+1)} := 1 r x ( t + 1 ) = r P ( t + 1 ) := 1 .
else
Update r x ( t + 1 ) : = ∥ d x ( t ) ∥ / ∥ d x ( 0 ) ∥ r_{x}^{(t+1)} := \lVert d_{x}^{(t)}\rVert/\lVert d_{x}^{(0)}\rVert r x ( t + 1 ) := ∥ d x ( t ) ∥ / ∥ d x ( 0 ) ∥ and r P ( t + 1 ) : = ∥ d P ( t ) ∥ / ∥ d P ( 0 ) ∥ r_{P}^{(t+1)} := \lVert d_{P}^{(t)}\rVert/\lVert d_{P}^{(0)}\rVert r P ( t + 1 ) := ∥ d P ( t ) ∥ / ∥ d P ( 0 ) ∥ .
end if
Set t : = t + 1 t := t + 1 t := t + 1 .
end while
Output: ( x ( t ) , P ( t ) ) ∈ Gr k ( T M ) (x^{(t)},P^{(t)}) \in\operatorname{Gr}_{k}(T\mathcal{M}) ( x ( t ) , P ( t ) ) ∈ Gr k ( T M ) .
We then discretize the dynamics (29) by directly applying the forward Euler scheme and using the retraction (18), which yields Algorithm 1. Note that in step 5, we do not need to compute the term involving the second fundamental form. This is credited to the construction of the retraction (18), in which the normal component makes no difference. In light of the quotient structure (10) and Lemmas 4 and 5, we have that the horizontal lift of d P ( t ) d_{P}^{(t)} d P ( t ) in step 5 is exactly d V ( t ) : = ( d v 1 ( t ) , … , d v k ( t ) ) d_{V}^{(t)} := (d_{v_{1}}^{(t)},\ldots,d_{v_{k}}^{(t)}) d V ( t ) := ( d v 1 ( t ) , … , d v k ( t ) ) , where
d v i ( t ) : = − Proj x ( t ) , ( V ( t ) ) ⊥ ( Hess M f ( x ( t ) ) [ v i ( t ) ] ) , i = 1 , … , k , d_{v_i}^{(t)} := -\operatorname{Proj}_{x^{(t)},(V^{(t)})^{\perp}}\left(\operatorname{Hess}_{\mathcal{M}} f(x^{(t)})[v_i^{(t)}]\right), \quad i=1,\ldots,k, d v i ( t ) := − Proj x ( t ) , ( V ( t ) ) ⊥ ( Hess M f ( x ( t ) ) [ v i ( t ) ] ) , i = 1 , … , k ,
and thus obtain a representative version running on St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) ; see Algorithm 2. In comparison with Algorithm 1, Algorithm 2 can be computationally cheaper.
Theoretical analysis
In this section, we analyze the linear stability of the dynamics (29) and the local convergence of Algorithm 1.
Algorithm 2 Discretized CSD using representatives on St k ( T M ) \operatorname{St}_{k}(T\mathcal{M}) St k ( T M ) .
Input: Initial point ( x ( 0 ) , V ( 0 ) ) ∈ St k ( T M ) (x^{(0)}, V^{(0)}) \in\operatorname{St}_{k}(T\mathcal{M}) ( x ( 0 ) , V ( 0 ) ) ∈ St k ( T M ) , step size η > 0 \eta> 0 η > 0 , maximum iteration number m a x i t ∈ N maxit \in\mathbb{N} ma x i t ∈ N , convergence tolerance t o l > 0 tol > 0 t o l > 0 .
Set t : = 0 t := 0 t := 0 , r x ( 0 ) : = r V ( 0 ) : = ∞ r_x^{(0)} := r_V^{(0)} := \infty r x ( 0 ) := r V ( 0 ) := ∞ .
while t < m a x i t t < maxit t < ma x i t and max { r x ( t ) , r V ( t ) } > t o l \max\{r_x^{(t)}, r_V^{(t)}\} > tol max { r x ( t ) , r V ( t ) } > t o l do
Compute the x x x -direction:
d x ( t ) : = − R ( x ( t ) , V ( t ) ) ( grad M f ( x ( t ) ) ) ∈ T x ( t ) M d_x^{(t)} := -R_{(x^{(t)},V^{(t)})}(\operatorname{grad}_{\mathcal{M}} f(x^{(t)})) \in T_{x^{(t)}}\mathcal{M} d x ( t ) := − R ( x ( t ) , V ( t ) ) ( grad M f ( x ( t ) )) ∈ T x ( t ) M .
Compute the V V V -direction:
d v i ( t ) : = − Proj x ( t ) , ( V ( t ) ) ⊥ ( Hess M f ( x ( t ) ) [ v i ( t ) ] ) ∈ T x ( t ) M d_{v_i}^{(t)} := -\operatorname{Proj}_{x^{(t)},(V^{(t)})^\perp}(\operatorname{Hess}_{\mathcal{M}} f(x^{(t)})[v_i^{(t)}]) \in T_{x^{(t)}}\mathcal{M} d v i ( t ) := − Proj x ( t ) , ( V ( t ) ) ⊥ ( Hess M f ( x ( t ) ) [ v i ( t ) ]) ∈ T x ( t ) M , i = 1 , … , k i = 1,\ldots,k i = 1 , … , k ,
and set d V ( t ) : = ( d v 1 ( t ) , … , d v k ( t ) ) ∈ T V ( t ) St k ( T x ( t ) M ) d_V^{(t)} := (d_{v_1}^{(t)},\ldots,d_{v_k}^{(t)}) \in T_{V^{(t)}}\operatorname{St}_{k}(T_{x^{(t)}}\mathcal{M}) d V ( t ) := ( d v 1 ( t ) , … , d v k ( t ) ) ∈ T V ( t ) St k ( T x ( t ) M ) .
Update x x x and V V V using the retraction defined in Eq. (23) :
( x ( t + 1 ) , V ( t + 1 ) ) : = R ( x ( t ) , V ( t ) ) Gr k ( T M ) ( η d x ( t ) , η ( d v i ( t ) + II x ( t ) ( d x ( t ) , v i ( t ) ) ) i = 1 k ) ∈ St k ( T M ) (x^{(t+1)},V^{(t+1)}) := R_{(x^{(t)},V^{(t)})}^{\operatorname{Gr}_{k}(T\mathcal{M})}\left(\eta d_x^{(t)},\eta\left(d_{v_i}^{(t)}+\operatorname{II}_{x^{(t)}}(d_x^{(t)},v_i^{(t)})\right)_{i=1}^{k}\right) \in\operatorname{St}_{k}(T\mathcal{M}) ( x ( t + 1 ) , V ( t + 1 ) ) := R ( x ( t ) , V ( t ) ) Gr k ( T M ) ( η d x ( t ) , η ( d v i ( t ) + II x ( t ) ( d x ( t ) , v i ( t ) ) ) i = 1 k ) ∈ St k ( T M ) .
if t = 0 t = 0 t = 0 then
Set r x ( t + 1 ) = r V ( t + 1 ) : = 1 r_x^{(t+1)} = r_V^{(t+1)} := 1 r x ( t + 1 ) = r V ( t + 1 ) := 1 .
else
Update r x ( t + 1 ) : = ∥ d x ( t ) ∥ / ∥ d x ( 0 ) ∥ r_x^{(t+1)} := \lVert d_x^{(t)}\rVert/\lVert d_x^{(0)}\rVert r x ( t + 1 ) := ∥ d x ( t ) ∥ / ∥ d x ( 0 ) ∥ and r V ( t + 1 ) : = ∥ d V ( t ) ∥ / ∥ d V ( 0 ) ∥ r_V^{(t+1)} := \lVert d_V^{(t)}\rVert/\lVert d_V^{(0)}\rVert r V ( t + 1 ) := ∥ d V ( t ) ∥ / ∥ d V ( 0 ) ∥ .
end if
Set t : = t + 1 t := t + 1 t := t + 1 .
end while
Output: ( x ( t ) , V ( t ) ) ∈ St k ( T M ) (x^{(t)},V^{(t)}) \in\operatorname{St}_{k}(T\mathcal{M}) ( x ( t ) , V ( t ) ) ∈ St k ( T M ) .
Linear stability analysis of the dynamics
Let w : = ( x , P ) ∈ Gr k ( T M ) w := (x,P) \in\operatorname{Gr}_{k}(T\mathcal{M}) w := ( x , P ) ∈ Gr k ( T M ) and h : Gr k ( T M ) → T ( Gr k ( T M ) ) h : \operatorname{Gr}_{k}(T\mathcal{M}) \to T(\operatorname{Gr}_{k}(T\mathcal{M})) h : Gr k ( T M ) → T ( Gr k ( T M )) be the vector field defined as h : = ( h 1 , h 2 ) h := (h_1,h_2) h := ( h 1 , h 2 ) , where
h 1 ( w ) : = − R w ( grad M f ( x ) ) , h 2 ( w ) : = − Proj T P Gr k ( T x M ) ∘ Hess M f ( x ) + II ‾ x ( h 1 ( w ) , P ) . (33) h_1(w) := -R_w(\operatorname{grad}_{\mathcal{M}} f(x)),
\qquad
h_2(w) := -\operatorname{Proj}_{T_P\operatorname{Gr}_{k}(T_x\mathcal{M})} \circ\operatorname{Hess}_{\mathcal{M}} f(x) + \overline{\operatorname{II}}_x(h_1(w),P). \tag*{(33)} h 1 ( w ) := − R w ( grad M f ( x )) , h 2 ( w ) := − Proj T P Gr k ( T x M ) ∘ Hess M f ( x ) + II x ( h 1 ( w ) , P ) . (33)
Note that R w R_w R w should be identified as the one in Eq. (30) . It follows that Eq. (29) can be rewritten compactly as the following dynamics on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) :
d w d t ( t ) = h ( w ( t ) ) , (34) \frac{\mathrm{d}w}{\mathrm{d}t}(t) = h(w(t)), \tag*{(34)} d t d w ( t ) = h ( w ( t )) , (34)
with the initial condition w ( 0 ) = ( x ( 0 ) , P ( 0 ) ) ∈ Gr k ( T M ) w(0) = (x(0),P(0)) \in\operatorname{Gr}_{k}(T\mathcal{M}) w ( 0 ) = ( x ( 0 ) , P ( 0 )) ∈ Gr k ( T M ) .
In the following, we aim to establish the linear stability of the dynamics (34) using the geometrical tools for Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) in Section 2.1. Before that, we first figure out the expression of the differential D h ( w ) Dh(w) D h ( w ) .
Lemma 6 (Differential of h h h ). *Suppose that f f f is C 3 C^3 C 3 , M \mathcal{M} M is an embedded Riemannian submanifold of a Euclidean space E \mathcal{E} E with dim ( M ) = d \dim(\mathcal{M}) = d dim ( M ) = d , 1 ≤ k < d 1 \le k < d 1 ≤ k < d , and h : Gr k ( T M ) → T ( Gr k ( T M ) ) h : \operatorname{Gr}_{k}(T\mathcal{M}) \to T(\operatorname{Gr}_{k}(T\mathcal{M})) h : Gr k ( T M ) → T ( Gr k ( T M )) is the vector field on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) defined in Eq. (33) . Then for any w : = ( x , P ) ∈ Gr k ( T M ) w := (x,P) \in\operatorname{Gr}_{k}(T\mathcal{M}) w := ( x , P ) ∈ Gr k ( T M ) and D : = ( δ , Δ ) ∈ T w ( Gr k ( T M ) ) D := (\delta,\Delta) \in T_w(\operatorname{Gr}_{k}(T\mathcal{M})) D := ( δ , Δ ) ∈ T w ( Gr k ( T M )) , it holds that
D h 1 ( w ) [ D ] = − R w ( Hess M f ( x ) [ δ ] ) + m 1 ( w ) [ δ ] + m 2 ( w ) [ Δ ] , Dh_1(w)[D] = -R_w(\operatorname{Hess}_{\mathcal{M}} f(x)[\delta]) + m_1(w)[\delta] + m_2(w)[\Delta], D h 1 ( w ) [ D ] = − R w ( Hess M f ( x ) [ δ ]) + m 1 ( w ) [ δ ] + m 2 ( w ) [ Δ ] ,
D h 2 ( w ) [ D ] = − [ Δ , [ P , Hess M f ( x ) ] ] − [ P , [ Δ , Hess M f ( x ) ] ] + m 3 ( w ) [ δ ] + m 4 ( w ) [ Δ ] , D h_2(w)[D] = -[\Delta,[P,\operatorname{Hess}_{\mathcal{M}} f(x)]] - [P,[\Delta,\operatorname{Hess}_{\mathcal{M}} f(x)]] + m_3(w)[\delta] + m_4(w)[\Delta], D h 2 ( w ) [ D ] = − [ Δ , [ P , Hess M f ( x )]] − [ P , [ Δ , Hess M f ( x )]] + m 3 ( w ) [ δ ] + m 4 ( w ) [ Δ ] ,
where m 1 ( w ) ∈ L ( T x M , E ) m_1(w) \in\mathcal{L}(T_x\mathcal{M},\mathcal{E}) m 1 ( w ) ∈ L ( T x M , E ) , m 2 ( w ) ∈ L ( S ( E ) , E ) m_2(w) \in\mathcal{L}(\mathcal{S}(\mathcal{E}),\mathcal{E}) m 2 ( w ) ∈ L ( S ( E ) , E ) , m 3 ( w ) ∈ L ( T x M , S ( E ) ) m_3(w) \in\mathcal{L}(T_x\mathcal{M},\mathcal{S}(\mathcal{E})) m 3 ( w ) ∈ L ( T x M , S ( E )) , m 4 ( w ) ∈ L ( S ( E ) , S ( E ) ) m_4(w) \in\mathcal{L}(\mathcal{S}(\mathcal{E}),\mathcal{S}(\mathcal{E})) m 4 ( w ) ∈ L ( S ( E ) , S ( E )) are defined as
m 1 ( w ) [ δ ] : = − D ( y ↦ Op y , P R ) ( x ) [ δ ] ( grad M f ( x ) ) − Op x R ( I I x ( δ , grad M f ( x ) ) ) , m_1(w)[\delta] := -D(y \mapsto\operatorname{Op}^{R}_{y,P})(x)[\delta](\operatorname{grad}_{\mathcal{M}} f(x))-\operatorname{Op}^{R}_{x}(\mathop{\mathrm{II}}\nolimits_x(\delta,\operatorname{grad}_{\mathcal{M}} f(x))), m 1 ( w ) [ δ ] := − D ( y ↦ Op y , P R ) ( x ) [ δ ] ( grad M f ( x )) − Op x R ( II x ( δ , grad M f ( x ))) ,
m 2 ( w ) [ Δ ] : = − D ( Q ↦ Op x , Q R ) ( P ) [ Δ ] ( grad M f ( x ) ) , m_2(w)[\Delta] := -D(Q \mapsto\operatorname{Op}^{R}_{x,Q})(P)[\Delta](\operatorname{grad}_{\mathcal{M}} f(x)), m 2 ( w ) [ Δ ] := − D ( Q ↦ Op x , Q R ) ( P ) [ Δ ] ( grad M f ( x )) ,
m 3 ( w ) [ δ ] : = − D ( y ↦ Op y , P Proj ) ( x ) [ δ ] ∘ Hess M f ( x ) − Op w Proj ∘ D H ‾ ( x ) [ δ ] + D ( y ↦ Op y I I ^ ) ( x ) [ δ ] ( h 1 ( w ) , P ) + Op x I I ^ ( D ( y ↦ h 1 ( y , P ) ) ( x ) [ δ ] , P ) , \begin{aligned}
m_3(w)[\delta] := {}&-D(y \mapsto\operatorname{Op}^{\operatorname{Proj}}_{y,P})(x)[\delta]\circ\operatorname{Hess}_{\mathcal{M}} f(x)-\operatorname{Op}^{\operatorname{Proj}}_{w}\circ D\overline{H}(x)[\delta]\\
&+D(y \mapsto\operatorname{Op}^{\widehat{\mathrm{II}}}_{y})(x)[\delta](h_1(w),P)+\operatorname{Op}^{\widehat{\mathrm{II}}}_{x}(D(y \mapsto h_1(y,P))(x)[\delta],P),
\end{aligned} m 3 ( w ) [ δ ] := − D ( y ↦ Op y , P Proj ) ( x ) [ δ ] ∘ Hess M f ( x ) − Op w Proj ∘ D H ( x ) [ δ ] + D ( y ↦ Op y II ) ( x ) [ δ ] ( h 1 ( w ) , P ) + Op x II ( D ( y ↦ h 1 ( y , P )) ( x ) [ δ ] , P ) ,
m 4 ( w ) [ Δ ] : = Op x I I ^ ( D ( Q ↦ h 1 ( x , Q ) ) ( P ) [ Δ ] , P ) + Op x I I ^ ( h 1 ( w ) , Δ ) , m_4(w)[\Delta] := \operatorname{Op}^{\widehat{\mathrm{II}}}_{x}(D(Q \mapsto h_1(x,Q))(P)[\Delta],P)+\operatorname{Op}^{\widehat{\mathrm{II}}}_{x}(h_1(w),\Delta), m 4 ( w ) [ Δ ] := Op x II ( D ( Q ↦ h 1 ( x , Q )) ( P ) [ Δ ] , P ) + Op x II ( h 1 ( w ) , Δ ) ,
( x , P ) ↦ Op x , P R (x,P)\mapsto\operatorname{Op}^{R}_{x,P} ( x , P ) ↦ Op x , P R and ( x , P ) ↦ Op x , P Proj (x,P)\mapsto\operatorname{Op}^{\operatorname{Proj}}_{x,P} ( x , P ) ↦ Op x , P Proj are respectively smooth extensions of ( x , P ) ↦ R x , P (x,P)\mapsto R_{x,P} ( x , P ) ↦ R x , P and ( x , P ) ↦ Proj T P Gr k ( T x M ) (x,P)\mapsto\operatorname{Proj}_{T_P\operatorname{Gr}_k(T_x\mathcal{M})} ( x , P ) ↦ Proj T P Gr k ( T x M ) in E × S ( E ) \mathcal{E}\times\mathcal{S}(\mathcal{E}) E × S ( E ) , x ↦ Op x I I ^ x\mapsto\operatorname{Op}^{\widehat{\mathrm{II}}}_{x} x ↦ Op x II and H ‾ \overline{H} H are respectively smooth extensions of x ↦ I I ^ x x\mapsto\widehat{\mathrm{II}}_x x ↦ II x and Hess M f \operatorname{Hess}_{\mathcal{M}} f Hess M f in E \mathcal{E} E .
Proof. Let c : R ⊇ I → Gr k ( T M ) c:\mathbb{R}\supseteq I\to\operatorname{Gr}_k(T\mathcal{M}) c : R ⊇ I → Gr k ( T M ) be a smooth curve on Gr k ( T M ) \operatorname{Gr}_k(T\mathcal{M}) Gr k ( T M ) such that c ( 0 ) = w c(0)=w c ( 0 ) = w and c ′ ( 0 ) = D c'(0)=D c ′ ( 0 ) = D . We thus have from Lemma 1
D h ( w ) [ D ] = ( h ∘ c ) ′ ( 0 ) = ( ( h 1 ∘ c ) ′ ( 0 ) , ( h 2 ∘ c ) ′ ( 0 ) ) . D h(w)[D]=(h\circ c)'(0)=((h_1\circ c)'(0),(h_2\circ c)'(0)). D h ( w ) [ D ] = ( h ∘ c ) ′ ( 0 ) = (( h 1 ∘ c ) ′ ( 0 ) , ( h 2 ∘ c ) ′ ( 0 )) .
By the product and chain rules,
( h 1 ∘ c ) ′ ( 0 ) = lim s → 0 ( h 1 ( c ( s ) ) − h 1 ( c ( 0 ) ) ) / s = − ( D ( y ↦ Op y , P R ) ( x ) [ δ ] + D ( Q ↦ Op x , Q R ) ( P ) [ Δ ] ) ( grad M f ( x ) ) − Op w R ( P x , δ ( grad f ( x ) ) + Proj T x M ( Hess f ( x ) [ δ ] ) ) = − ( D ( y ↦ Op y , P R ) ( x ) [ δ ] + D ( Q ↦ Op x , Q R ) ( P ) [ Δ ] ) ( grad M f ( x ) ) − Op w R ( Hess M f ( x ) [ δ ] + I I x ( δ , grad M f ( x ) ) ) = − R w ( Hess M f ( x ) [ δ ] ) + m 1 ( w ) [ δ ] + m 2 ( w ) [ Δ ] , \begin{aligned}
(h_1\circ c)'(0)
&=\lim_{s\to0}(h_1(c(s))-h_1(c(0)))/s\\
&=-\bigl(D(y\mapsto\operatorname{Op}^{R}_{y,P})(x)[\delta]+D(Q\mapsto\operatorname{Op}^{R}_{x,Q})(P)[\Delta]\bigr)(\operatorname{grad}_{\mathcal{M}}f(x))\\
&\quad-\operatorname{Op}^{R}_{w}\bigl(P_{x,\delta}(\operatorname{grad}f(x))+\operatorname{Proj}_{T_x\mathcal{M}}(\operatorname{Hess}f(x)[\delta])\bigr)\\
&=-\bigl(D(y\mapsto\operatorname{Op}^{R}_{y,P})(x)[\delta]+D(Q\mapsto\operatorname{Op}^{R}_{x,Q})(P)[\Delta]\bigr)(\operatorname{grad}_{\mathcal{M}}f(x))\\
&\quad-\operatorname{Op}^{R}_{w}\bigl(\operatorname{Hess}_{\mathcal{M}}f(x)[\delta]+\mathop{\mathrm{II}}\nolimits_x(\delta,\operatorname{grad}_{\mathcal{M}}f(x))\bigr)\\
&=-R_w(\operatorname{Hess}_{\mathcal{M}}f(x)[\delta])+m_1(w)[\delta]+m_2(w)[\Delta],
\end{aligned} ( h 1 ∘ c ) ′ ( 0 ) = s → 0 lim ( h 1 ( c ( s )) − h 1 ( c ( 0 ))) / s = − ( D ( y ↦ Op y , P R ) ( x ) [ δ ] + D ( Q ↦ Op x , Q R ) ( P ) [ Δ ] ) ( grad M f ( x )) − Op w R ( P x , δ ( grad f ( x )) + Proj T x M ( Hess f ( x ) [ δ ]) ) = − ( D ( y ↦ Op y , P R ) ( x ) [ δ ] + D ( Q ↦ Op x , Q R ) ( P ) [ Δ ] ) ( grad M f ( x )) − Op w R ( Hess M f ( x ) [ δ ] + II x ( δ , grad M f ( x )) ) = − R w ( Hess M f ( x ) [ δ ]) + m 1 ( w ) [ δ ] + m 2 ( w ) [ Δ ] ,
where the second equality uses Eqs. (49) and (51) , and
( h 2 ∘ c ) ′ ( 0 ) = lim s → 0 ( h 2 ( c ( s ) ) − h 2 ( c ( 0 ) ) ) / s = − D ( y ↦ Op y , P Proj ) ( x ) [ δ ] ∘ Hess M f ( x ) − Op w Proj ∘ D H ‾ ( x ) [ δ ] + D ( y ↦ Op y I I ^ ) ( x ) [ δ ] ( h 1 ( w ) , P ) + Op x I I ^ ( D ( y ↦ h 1 ( y , P ) ) ( x ) [ δ ] , P ) − D ( Q ↦ Op x , Q Proj ) ( P ) [ Δ ] ∘ Hess M f ( x ) + Op x I I ^ ( h 1 ( w ) , Δ ) + Op x I I ^ ( D ( Q ↦ h 1 ( x , Q ) ) ( P ) [ Δ ] , P ) = − [ Δ , [ P , Hess M f ( x ) ] ] − [ P , [ Δ , Hess M f ( x ) ] ] + m 3 ( w ) [ δ ] + m 4 ( w ) [ Δ ] , \begin{aligned}
(h_2\circ c)'(0)
&=\lim_{s\to0}(h_2(c(s))-h_2(c(0)))/s\\
&=-D(y\mapsto\operatorname{Op}^{\operatorname{Proj}}_{y,P})(x)[\delta]\circ\operatorname{Hess}_{\mathcal{M}}f(x)-\operatorname{Op}^{\operatorname{Proj}}_{w}\circ D\overline{H}(x)[\delta]\\
&\quad+D(y\mapsto\operatorname{Op}^{\widehat{\mathrm{II}}}_{y})(x)[\delta](h_1(w),P)+\operatorname{Op}^{\widehat{\mathrm{II}}}_{x}(D(y\mapsto h_1(y,P))(x)[\delta],P)\\
&\quad-D(Q\mapsto\operatorname{Op}^{\operatorname{Proj}}_{x,Q})(P)[\Delta]\circ\operatorname{Hess}_{\mathcal{M}}f(x)+\operatorname{Op}^{\widehat{\mathrm{II}}}_{x}(h_1(w),\Delta)\\
&\quad+\operatorname{Op}^{\widehat{\mathrm{II}}}_{x}(D(Q\mapsto h_1(x,Q))(P)[\Delta],P)\\
&=-[\Delta,[P,\operatorname{Hess}_{\mathcal{M}}f(x)]]-[P,[\Delta,\operatorname{Hess}_{\mathcal{M}}f(x)]]+m_3(w)[\delta]+m_4(w)[\Delta],
\end{aligned} ( h 2 ∘ c ) ′ ( 0 ) = s → 0 lim ( h 2 ( c ( s )) − h 2 ( c ( 0 ))) / s = − D ( y ↦ Op y , P Proj ) ( x ) [ δ ] ∘ Hess M f ( x ) − Op w Proj ∘ D H ( x ) [ δ ] + D ( y ↦ Op y II ) ( x ) [ δ ] ( h 1 ( w ) , P ) + Op x II ( D ( y ↦ h 1 ( y , P )) ( x ) [ δ ] , P ) − D ( Q ↦ Op x , Q Proj ) ( P ) [ Δ ] ∘ Hess M f ( x ) + Op x II ( h 1 ( w ) , Δ ) + Op x II ( D ( Q ↦ h 1 ( x , Q )) ( P ) [ Δ ] , P ) = − [ Δ , [ P , Hess M f ( x )]] − [ P , [ Δ , Hess M f ( x )]] + m 3 ( w ) [ δ ] + m 4 ( w ) [ Δ ] ,
where the last equality follows from Eq. (31) . The proof is complete.
Theorem 2 (Linear stability of the dynamics (34)). Suppose that f f f is C 3 C^3 C 3 , M \mathcal{M} M is an embedded Riemannian submanifold of a Euclidean space E \mathcal{E} E with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d , 1 ≤ k < d 1\le k<d 1 ≤ k < d , w ⋆ : = ( x ⋆ , P ⋆ ) ∈ Gr k ( T M ) w^\star:=(x^\star,P^\star)\in\operatorname{Gr}_k(T\mathcal{M}) w ⋆ := ( x ⋆ , P ⋆ ) ∈ Gr k ( T M ) , and Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}}f(x^\star) Hess M f ( x ⋆ ) is nondegenerate, whose eigenvalues are λ 1 ⋆ ≤ ⋯ ≤ λ k ⋆ < λ k + 1 ⋆ ≤ ⋯ ≤ λ d ⋆ \lambda_1^\star\le\cdots\le\lambda_k^\star<\lambda_{k+1}^\star\le\cdots\le\lambda_d^\star λ 1 ⋆ ≤ ⋯ ≤ λ k ⋆ < λ k + 1 ⋆ ≤ ⋯ ≤ λ d ⋆ . Then w ⋆ w^\star w ⋆ is a linearly steady state of the dynamics (34) if and only if x ⋆ x^\star x ⋆ is an index-k k k constrained SP of f f f on M \mathcal{M} M and P ⋆ P^\star P ⋆ is an orthogonal projector onto the lowest k k k -dimensional invariant subspace of Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}}f(x^\star) Hess M f ( x ⋆ ) . *
Proof. The proof revolves around the spectrum of D h ( w ⋆ ) \mathrm{D}h(w^\star) D h ( w ⋆ ) .
Necessity. Suppose that w ⋆ w^\star w ⋆ is a linearly steady state of dynamics (34) . Therefore, d w / d t \mathrm{d}w/\mathrm{d}t d w / d t vanishes at w ⋆ w^\star w ⋆ , which implies that
grad M f ( x ⋆ ) = 0 \operatorname{grad}_{\mathcal{M}} f(x^\star)=0 grad M f ( x ⋆ ) = 0 , i.e., x ⋆ x^\star x ⋆ is a critical point of f f f on M \mathcal{M} M . This further yields m 1 ( w ⋆ ) = 0 m_1(w^\star)=0 m 1 ( w ⋆ ) = 0 , m 2 ( w ⋆ ) = 0 m_2(w^\star)=0 m 2 ( w ⋆ ) = 0 , and m 4 ( w ⋆ ) = 0 m_4(w^\star)=0 m 4 ( w ⋆ ) = 0 from their definitions in Lemma 6;
[ P ⋆ , [ P ⋆ , Hess M f ( x ⋆ ) ] ] = 0 [P^\star,[P^\star,\operatorname{Hess}_{\mathcal{M}}f(x^\star)]]=0 [ P ⋆ , [ P ⋆ , Hess M f ( x ⋆ )]] = 0 , i.e., P ⋆ P^\star P ⋆ is an orthogonal projector onto a k k k -dimensional invariant subspace of Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}}f(x^\star) Hess M f ( x ⋆ ) .
The differential D h ( w ⋆ ) [ D ] \mathrm{D}h(w^\star)[D] D h ( w ⋆ ) [ D ] in Lemma 6 thus simplifies to
D h ( w ⋆ ) [ D ] = ( − R w ⋆ ( Hess M f ( x ⋆ ) [ δ ] ) − [ P ⋆ , [ Δ , Hess M f ( x ⋆ ) ] ] + m 3 ( w ⋆ ) [ δ ] ) . (35) \mathrm{D}h(w^\star)[D]=\begin{pmatrix}
-R_{w^\star}\bigl(\operatorname{Hess}_{\mathcal{M}}f(x^\star)[\delta]\bigr)\\
-[P^\star,[\Delta,\operatorname{Hess}_{\mathcal{M}}f(x^\star)]]+m_3(w^\star)[\delta]
\end{pmatrix}. \tag*{(35)} D h ( w ⋆ ) [ D ] = ( − R w ⋆ ( Hess M f ( x ⋆ ) [ δ ] ) − [ P ⋆ , [ Δ , Hess M f ( x ⋆ )]] + m 3 ( w ⋆ ) [ δ ] ) . (35)
Since w ⋆ w^\star w ⋆ is a steady state, D h ( w ⋆ ) [ D ] \mathrm{D}h(w^\star)[D] D h ( w ⋆ ) [ D ] coincides with the covariant derivative of h h h at w ⋆ w^\star w ⋆ along D D D and thus belongs to T w ⋆ Gr k ( T M ) T_{w^\star}\operatorname{Gr}_k(T\mathcal{M}) T w ⋆ Gr k ( T M ) .
By the Hurwitz linearization theorem [4 ] , the linear stability of w ⋆ w^\star w ⋆ implies that all of the eigenvalues of D h ( w ⋆ ) \mathrm{D}h(w^\star) D h ( w ⋆ ) have negative real parts. By Lemma 3, these eigenvalues coincide with the generalized eigenvalues of the matrix pair ( A , B ) (A,B) ( A , B ) , where A ∈ R d ^ × d ^ A\in\mathbb{R}^{\hat d\times\hat d} A ∈ R d ^ × d ^ and B ∈ R s y m d ^ × d ^ B\in\mathbb{R}_{\mathrm{sym}}^{\hat d\times\hat d} B ∈ R sym d ^ × d ^ (with d ^ : = d + k ( d − k ) \hat d:=d+k(d-k) d ^ := d + k ( d − k ) ) are respectively the representation of D h ( w ⋆ ) \mathrm{D}h(w^\star) D h ( w ⋆ ) under the basis
{ D q ⋆ : = ( δ q ⋆ , Δ q ⋆ ) : = ( v q ⋆ , I l ^ x ⋆ ( v q ⋆ , P ⋆ ) ) } 1 ≤ q ≤ d ∪ { D i j ⋆ : = ( δ i j ⋆ , Δ i j ⋆ ) : = ( 0 , 1 2 ( v i ⋆ ( v j ⋆ ) T + v j ⋆ ( v i ⋆ ) T ) ) } 1 ≤ i ≤ k k + 1 ≤ j ≤ d \left\{D_q^\star:=(\delta_q^\star,\Delta_q^\star):=(v_q^\star,\widehat{\mathrm{Il}}_{x^\star}(v_q^\star,P^\star))\right\}_{1\le q\le d}
\cup
\left\{D_{ij}^\star:=(\delta_{ij}^\star,\Delta_{ij}^\star):=\left(0,\frac{1}{\sqrt{2}}\left(v_i^\star(v_j^\star)^{\mathsf T}+v_j^\star(v_i^\star)^{\mathsf T}\right)\right)\right\}_{\substack{1\le i\le k\\ k+1\le j\le d}} { D q ⋆ := ( δ q ⋆ , Δ q ⋆ ) := ( v q ⋆ , Il x ⋆ ( v q ⋆ , P ⋆ )) } 1 ≤ q ≤ d ∪ { D ij ⋆ := ( δ ij ⋆ , Δ ij ⋆ ) := ( 0 , 2 1 ( v i ⋆ ( v j ⋆ ) T + v j ⋆ ( v i ⋆ ) T ) ) } 1 ≤ i ≤ k k + 1 ≤ j ≤ d
and the Gram matrix using the Sasaki-type metric in Definition 1:
A q , q ′ : = ⟨ D q ′ ⋆ , D h ( w ⋆ ) [ D q ⋆ ] ⟩ w ⋆ , B q , q ′ : = ⟨ D q ′ ⋆ , D q ⋆ ⟩ w ⋆ , A q , i j : = ⟨ D i j ⋆ , D h ( w ⋆ ) [ D q ⋆ ] ⟩ w ⋆ , B q , i j : = ⟨ D i j ⋆ , D q ⋆ ⟩ w ⋆ , A i j , q : = ⟨ D q ⋆ , D h ( w ⋆ ) [ D i j ⋆ ] ⟩ w ⋆ , B i j , q : = ⟨ D q ⋆ , D i j ⋆ ⟩ w ⋆ , A i j , i ′ j ′ : = ⟨ D i ′ j ′ ⋆ , D h ( w ⋆ ) [ D i j ⋆ ] ⟩ w ⋆ , B i j , i ′ j ′ : = ⟨ D i ′ j ′ ⋆ , D i j ⋆ ⟩ w ⋆ . \begin{aligned}
A_{q,q'}&:=\langle D_{q'}^\star,\mathrm{D}h(w^\star)[D_q^\star]\rangle_{w^\star},&
B_{q,q'}&:=\langle D_{q'}^\star,D_q^\star\rangle_{w^\star},\\
A_{q,ij}&:=\langle D_{ij}^\star,\mathrm{D}h(w^\star)[D_q^\star]\rangle_{w^\star},&
B_{q,ij}&:=\langle D_{ij}^\star,D_q^\star\rangle_{w^\star},\\
A_{ij,q}&:=\langle D_q^\star,\mathrm{D}h(w^\star)[D_{ij}^\star]\rangle_{w^\star},&
B_{ij,q}&:=\langle D_q^\star,D_{ij}^\star\rangle_{w^\star},\\
A_{ij,i'j'}&:=\langle D_{i'j'}^\star,\mathrm{D}h(w^\star)[D_{ij}^\star]\rangle_{w^\star},&
B_{ij,i'j'}&:=\langle D_{i'j'}^\star,D_{ij}^\star\rangle_{w^\star}.
\end{aligned} A q , q ′ A q , ij A ij , q A ij , i ′ j ′ := ⟨ D q ′ ⋆ , D h ( w ⋆ ) [ D q ⋆ ] ⟩ w ⋆ , := ⟨ D ij ⋆ , D h ( w ⋆ ) [ D q ⋆ ] ⟩ w ⋆ , := ⟨ D q ⋆ , D h ( w ⋆ ) [ D ij ⋆ ] ⟩ w ⋆ , := ⟨ D i ′ j ′ ⋆ , D h ( w ⋆ ) [ D ij ⋆ ] ⟩ w ⋆ , B q , q ′ B q , ij B ij , q B ij , i ′ j ′ := ⟨ D q ′ ⋆ , D q ⋆ ⟩ w ⋆ , := ⟨ D ij ⋆ , D q ⋆ ⟩ w ⋆ , := ⟨ D q ⋆ , D ij ⋆ ⟩ w ⋆ , := ⟨ D i ′ j ′ ⋆ , D ij ⋆ ⟩ w ⋆ .
Here, { ( μ q ⋆ , v q ⋆ ) } q = 1 d \{(\mu_q^\star,v_q^\star)\}_{q=1}^{d} {( μ q ⋆ , v q ⋆ ) } q = 1 d are the eigenpairs of Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}}f(x^\star) Hess M f ( x ⋆ ) such that P ⋆ = ∑ ℓ = 1 k v ℓ ⋆ ( v ℓ ⋆ ) T P^\star=\sum_{\ell=1}^{k}v_\ell^\star(v_\ell^\star)^{\mathsf T} P ⋆ = ∑ ℓ = 1 k v ℓ ⋆ ( v ℓ ⋆ ) T . We could work out the entries in A A A and B B B explicitly:
A q , q ′ = { μ q ⋆ δ q , q ′ , q , q ′ = 1 , … , k , − μ q ⋆ δ q , q ′ , q , q ′ = k + 1 , … , d , 0 , otherwise , A_{q,q'}=
\begin{cases}
\mu_q^\star\delta_{q,q'},&q,q'=1,\ldots,k,\\
-\mu_q^\star\delta_{q,q'},&q,q'=k+1,\ldots,d,\\
0,&\text{otherwise},
\end{cases} A q , q ′ = ⎩ ⎨ ⎧ μ q ⋆ δ q , q ′ , − μ q ⋆ δ q , q ′ , 0 , q , q ′ = 1 , … , k , q , q ′ = k + 1 , … , d , otherwise ,
A i j , q = 0 , q = 1 , … , d , i = 1 , … , k , j = k + 1 , … , d , A_{ij,q}=0,\quad q=1,\ldots,d,\ i=1,\ldots,k,\ j=k+1,\ldots,d, A ij , q = 0 , q = 1 , … , d , i = 1 , … , k , j = k + 1 , … , d ,
A q , i j = ⟨ Δ i j ⋆ , m 3 ( w ⋆ ) [ v q ⋆ ] ⟩ , q = 1 , … , d , i = 1 , … , k , j = k + 1 , … , d , A_{q,ij}=\langle\Delta_{ij}^\star,m_3(w^\star)[v_q^\star]\rangle,\quad q=1,\ldots,d,\ i=1,\ldots,k,\ j=k+1,\ldots,d, A q , ij = ⟨ Δ ij ⋆ , m 3 ( w ⋆ ) [ v q ⋆ ]⟩ , q = 1 , … , d , i = 1 , … , k , j = k + 1 , … , d ,
A i j , i ′ j ′ = ( μ i ⋆ − μ j ⋆ ) δ i , i ′ δ j , j ′ , i , i ′ = 1 , … , k , j , j ′ = k + 1 , … , d , A_{ij,i'j'}=(\mu_i^\star-\mu_j^\star)\delta_{i,i'}\delta_{j,j'},\quad i,i'=1,\ldots,k,\ j,j'=k+1,\ldots,d, A ij , i ′ j ′ = ( μ i ⋆ − μ j ⋆ ) δ i , i ′ δ j , j ′ , i , i ′ = 1 , … , k , j , j ′ = k + 1 , … , d ,
and
B q , q ′ = δ q , q ′ , q , q ′ = 1 , … , d , B_{q,q'}=\delta_{q,q'},\quad q,q'=1,\ldots,d, B q , q ′ = δ q , q ′ , q , q ′ = 1 , … , d ,
B q , i j = B i j , q = 0 , q = 1 , … , d , i = 1 , … , k , j = k + 1 , … , d , B_{q,ij}=B_{ij,q}=0,\quad q=1,\ldots,d,\ i=1,\ldots,k,\ j=k+1,\ldots,d, B q , ij = B ij , q = 0 , q = 1 , … , d , i = 1 , … , k , j = k + 1 , … , d ,
B i j , i ′ j ′ = δ i , i ′ δ j , j ′ , i , i ′ = 1 , … , k , j , j ′ = k + 1 , … , d . B_{ij,i'j'}=\delta_{i,i'}\delta_{j,j'},\quad i,i'=1,\ldots,k,\ j,j'=k+1,\ldots,d. B ij , i ′ j ′ = δ i , i ′ δ j , j ′ , i , i ′ = 1 , … , k , j , j ′ = k + 1 , … , d .
Consequently, B B B is an identity matrix, A A A is lower (or upper) triangular, the generalized eigenvalues of ( A , B ) (A,B) ( A , B ) are identical to the eigenvalues of A A A , which are exactly
{ μ q ⋆ } q = 1 k , { − μ q ⋆ } q = k + 1 d , and { μ i ⋆ − μ j ⋆ } i = 1 , … , k j = k + 1 , … , d . \{\mu_q^\star\}_{q=1}^{k},\quad\{-\mu_q^\star\}_{q=k+1}^{d},\quad\text{and}\quad\{\mu_i^\star-\mu_j^\star\}_{\substack{i=1,\ldots,k\\j=k+1,\ldots,d}}. { μ q ⋆ } q = 1 k , { − μ q ⋆ } q = k + 1 d , and { μ i ⋆ − μ j ⋆ } i = 1 , … , k j = k + 1 , … , d .
Since they are all negative, we get max { μ 1 ⋆ , … , μ k ⋆ } < 0 < min { μ k + 1 ⋆ , … , μ d ⋆ } \max\{\mu_1^\star,\ldots,\mu_k^\star\}<0<\min\{\mu_{k+1}^\star,\ldots,\mu_d^\star\} max { μ 1 ⋆ , … , μ k ⋆ } < 0 < min { μ k + 1 ⋆ , … , μ d ⋆ } . This indicates that x ⋆ x^\star x ⋆ is an index-k k k constrained SP of f f f on M \mathcal{M} M , P ⋆ P^\star P ⋆ is an orthogonal projector onto the lowest k k k -dimensional invariant subspace of Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}}f(x^\star) Hess M f ( x ⋆ ) , and { μ 1 ⋆ , … , μ k ⋆ } = { λ 1 ⋆ , … , λ k ⋆ } \{\mu_1^\star,\ldots,\mu_k^\star\}=\{\lambda_1^\star,\ldots,\lambda_k^\star\} { μ 1 ⋆ , … , μ k ⋆ } = { λ 1 ⋆ , … , λ k ⋆ } , as desired.
Sufficiency. Suppose that x ⋆ x^\star x ⋆ is an index-k k k constrained SP of f f f on M \mathcal{M} M and P ⋆ P^\star P ⋆ is an orthogonal projector onto the lowest k k k -dimensional invariant subspace of Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}}f(x^\star) Hess M f ( x ⋆ ) . Therefore, grad M f ( x ⋆ ) = 0 \operatorname{grad}_{\mathcal{M}} f(x^\star)=0 grad M f ( x ⋆ ) = 0 and λ 1 ⋆ ≤ ⋯ ≤ λ k ⋆ < 0 < λ k + 1 ⋆ ≤ ⋯ λ d ⋆ \lambda_1^\star\leq\cdots\leq\lambda_k^\star<0<\lambda_{k+1}^\star\leq\cdots\lambda_d^\star λ 1 ⋆ ≤ ⋯ ≤ λ k ⋆ < 0 < λ k + 1 ⋆ ≤ ⋯ λ d ⋆ . By similar arguments as above, we could obtain that the eigenvalues of D h ( w ⋆ ) D h(w^\star) D h ( w ⋆ ) are exactly
{ λ q ⋆ } q = 1 k , { − λ q ⋆ } q = k + 1 d , and { λ i ⋆ − λ j ⋆ } i = 1 , … , k j = k + 1 , … , d , \{\lambda_q^\star\}_{q=1}^{k},\quad\{-\lambda_q^\star\}_{q=k+1}^{d},\quad\text{and}\quad\{\lambda_i^\star-\lambda_j^\star\}_{\substack{i=1,\ldots,k\\j=k+1,\ldots,d}}, { λ q ⋆ } q = 1 k , { − λ q ⋆ } q = k + 1 d , and { λ i ⋆ − λ j ⋆ } i = 1 , … , k j = k + 1 , … , d ,
which are all negative, implying that w ⋆ = ( x ⋆ , P ⋆ ) w^\star=(x^\star,P^\star) w ⋆ = ( x ⋆ , P ⋆ ) is linearly steady.
Remark 3. In comparison with the linear stability results in the existing work [91 Theorem 1] , Theorem 2 not only avoids the use of explicit global/local level-set representations of M \mathcal{M} M , but also requires weaker assumptions on the eigenvalues of Riemannian Hessian. More specifically, we only ask for a positive gap between λ k ⋆ \lambda_k^\star λ k ⋆ and λ k + 1 ⋆ \lambda_{k+1}^\star λ k + 1 ⋆ , whereas the existing one assumes additionally that { λ i ⋆ } i = 1 \{\lambda_i^\star\}_{i=1} { λ i ⋆ } i = 1 are all simple (cf. [91, Eqs. (34) and (40)]). This advantage is due to the fact that we respect the intrinsic quotient structure (10)` by introducing the Grassmann bundle Gr k ( T M ) \operatorname{Gr}_k(\mathrm{T}\mathcal{M}) Gr k ( T M ) .
Local convergence analysis of the discretized algorithm
In what follows, we show the local convergence properties of Algorithm 1. We first recall an auxiliary lemma, whose proof can be largely covered by the literature.
Lemma 7. Suppose that N \mathcal{N} N is an embedded Riemannian submanifold of a Euclidean space, R N : T N → N R^{\mathcal{N}}:\mathrm{T}\mathcal{N}\to\mathcal{N} R N : T N → N is a retraction on N \mathcal{N} N , and y ∈ N y\in\mathcal{N} y ∈ N . Then there exist constants c 1 > 0 c_1>0 c 1 > 0 and r 1 > 0 r_1>0 r 1 > 0 such that the following two statements hold at the same time:
Exp y N : T y N → N \operatorname{Exp}^{\mathcal{N}}_y:\mathrm{T}_y\mathcal{N}\to\mathcal{N} Exp y N : T y N → N is a diffeomorphism between B y ( 0 , r 1 ) : = { u ∈ T y N ∣ ∥ u ∥ < r 1 } B_y(0,r_1):=\{u\in\mathrm{T}_y\mathcal{N}\mid\lVert u\rVert<r_1\} B y ( 0 , r 1 ) := { u ∈ T y N ∣ ∥ u ∥ < r 1 } and U : = Exp y N ( B y ( 0 , r 1 ) ) U:=\operatorname{Exp}^{\mathcal{N}}_y(B_y(0,r_1)) U := Exp y N ( B y ( 0 , r 1 )) .
The inequality dist N ( y ′ , R y ′ N ( u ′ ) ) ≤ c 1 ∥ u ′ ∥ \operatorname{dist}_{\mathcal{N}}(y',R^{\mathcal{N}}_{y'}(u'))\leq c_1\lVert u'\rVert dist N ( y ′ , R y ′ N ( u ′ )) ≤ c 1 ∥ u ′ ∥ holds for any ( y ′ , u ′ ) (y',u') ( y ′ , u ′ ) such that y ′ ∈ Exp y N ( cl ( B y ( 0 , r 1 ) ) ) y'\in\operatorname{Exp}_y^{\mathcal{N}}(\operatorname{cl}(B_y(0,r_1))) y ′ ∈ Exp y N ( cl ( B y ( 0 , r 1 ))) and u ′ ∈ T y ′ N u'\in\mathrm{T}_{y'}\mathcal{N} u ′ ∈ T y ′ N with ∥ u ′ ∥ ≤ r 1 \lVert u'\rVert\leq r_1 ∥ u ′ ∥ ≤ r 1 . In particular, if R N = Exp N R^{\mathcal{N}}=\operatorname{Exp}^{\mathcal{N}} R N = Exp N , then the equality holds with c 1 = 1 c_1=1 c 1 = 1 .
Proof. For the first statement, it suffices to note D Exp y N ( 0 ) = I d T y N \mathrm{D}\operatorname{Exp}^{\mathcal{N}}_y(0)=\mathrm{Id}_{\mathrm{T}_y\mathcal{N}} D Exp y N ( 0 ) = Id T y N and then use the inverse function theorem. For the second one, please refer to [15 Lemma 6.32 and Proposition 10.22] .
Lemma 8 (Residual after single iteration). Suppose that f f f is C 3 C^3 C 3 , M \mathcal{M} M is an embedded Riemannian submanifold of a Euclidean space E \mathcal{E} E with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d , 1 ≤ k < d 1\leq k<d 1 ≤ k < d , x ⋆ ∈ M x^\star\in\mathcal{M} x ⋆ ∈ M is an index-k k k constrained SP of f f f on M \mathcal{M} M , Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}}f(x^\star) Hess M f ( x ⋆ ) is nondegenerate, whose eigenvalues are λ 1 ⋆ ≤ ⋯ ≤ λ k ⋆ < λ k + 1 ⋆ ≤ ⋯ ≤ λ d ⋆ \lambda_1^\star\leq\cdots\leq\lambda_k^\star<\lambda_{k+1}^\star\leq\cdots\leq\lambda_d^\star λ 1 ⋆ ≤ ⋯ ≤ λ k ⋆ < λ k + 1 ⋆ ≤ ⋯ ≤ λ d ⋆ , and P ⋆ ∈ Gr k ( T x ⋆ M ) P^\star\in\operatorname{Gr}_k(\mathrm{T}_{x^\star}\mathcal{M}) P ⋆ ∈ Gr k ( T x ⋆ M ) is an orthogonal projector onto the lowest k k k -dimensional invariant subspace of Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}}f(x^\star) Hess M f ( x ⋆ ) . Let w ⋆ : = ( x ⋆ , P ⋆ ) ∈ Gr k ( T M ) w^\star:=(x^\star,P^\star)\in\operatorname{Gr}_k(\mathrm{T}\mathcal{M}) w ⋆ := ( x ⋆ , P ⋆ ) ∈ Gr k ( T M ) . Consider the following iterative mapping F : T w ⋆ ( Gr k ( T M ) ) → T w ⋆ ( Gr k ( T M ) ) F:\mathrm{T}_{w^\star}(\operatorname{Gr}_k(\mathrm{T}\mathcal{M}))\to\mathrm{T}_{w^\star}(\operatorname{Gr}_k(\mathrm{T}\mathcal{M})) F : T w ⋆ ( Gr k ( T M )) → T w ⋆ ( Gr k ( T M )) defined as
F ( D ) : = ( Exp w ⋆ Gr k ( T M ) ) − 1 ( R w ~ ( D ) Gr k ( T M ) ( η ⋅ h ( w ~ ( D ) ) ) ) = : ( F 1 ( D ) , F 2 ( D ) ) , (36) F(D):=\left(\operatorname{Exp}_{w^\star}^{\operatorname{Gr}_k(\mathrm{T}\mathcal{M})}\right)^{-1}\left(R_{\widetilde{w}(D)}^{\operatorname{Gr}_k(\mathrm{T}\mathcal{M})}\left(\eta\cdot h(\widetilde{w}(D))\right)\right)=:(F_1(D),F_2(D)), \tag*{(36)} F ( D ) := ( Exp w ⋆ Gr k ( T M ) ) − 1 ( R w ( D ) Gr k ( T M ) ( η ⋅ h ( w ( D )) ) ) =: ( F 1 ( D ) , F 2 ( D )) , (36)
for any D : = ( δ , Δ ) ∈ T w ⋆ ( G r k ( T M ) ) D := (\delta, \Delta) \in\mathrm{T}_{w^\star}(\mathrm{Gr}_k(\mathcal{T}\mathcal{M})) D := ( δ , Δ ) ∈ T w ⋆ ( Gr k ( T M )) , where w ~ ( D ) : = Exp w ⋆ G r k ( T M ) ( D ) ∈ G r k ( T M ) \widetilde{w}(D) := \operatorname{Exp}^{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}_{w^\star}(D) \in\mathrm{Gr}_k(\mathcal{T}\mathcal{M}) w ( D ) := Exp w ⋆ Gr k ( T M ) ( D ) ∈ Gr k ( T M ) , and η \eta η is a step size satisfying
0 < η < min { ( r 1 − r 2 ) / c 1 , r 1 } sup ∥ D ∥ w ⋆ ≤ r 1 ∥ h ( w ~ ( D ) ) ∥ w ~ ( D ) (37) 0 < \eta< \frac{\min\{(r_1-r_2)/c_1,r_1\}}{\sup_{\lVert D\rVert_{w^\star}\le r_1}\lVert h(\widetilde{w}(D))\rVert_{\widetilde{w}(D)}} \tag*{(37)} 0 < η < sup ∥ D ∥ w ⋆ ≤ r 1 ∥ h ( w ( D )) ∥ w ( D ) min {( r 1 − r 2 ) / c 1 , r 1 } (37)
with c 1 c_1 c 1 and r 1 r_1 r 1 the constants in Lemma 7 (associated to N = G r k ( T M ) \mathcal{N} = \mathrm{Gr}_k(\mathcal{T}\mathcal{M}) N = Gr k ( T M ) , R N = R G r k ( T M ) \mathbb{R}^{\mathcal{N}} = \mathbb{R}^{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})} R N = R Gr k ( T M ) or Exp G r k ( T M ) \operatorname{Exp}_{\mathrm{Gr}_k}(\mathcal{T}\mathcal{M}) Exp Gr k ( T M ) , and y = w ⋆ y = w^\star y = w ⋆ ) and r 2 ∈ ( 0 , r 1 ) r_2 \in(0,r_1) r 2 ∈ ( 0 , r 1 ) . Then F F F is well-defined and smooth over the subset { D ∈ T w ⋆ ( G r k ( T M ) ) : ∥ D ∥ w ⋆ < r 2 } \{D \in\mathrm{T}_{w^\star}(\mathrm{Gr}_k(\mathcal{T}\mathcal{M})) : \lVert D\rVert_{w^\star} < r_2\} { D ∈ T w ⋆ ( Gr k ( T M )) : ∥ D ∥ w ⋆ < r 2 } . Moreover, there exist constants r 3 ∈ ( 0 , r 2 ] r_3 \in(0,r_2] r 3 ∈ ( 0 , r 2 ] and c 3 ≥ 0 c_3 \ge0 c 3 ≥ 0 such that
( ∥ F 1 ( D ) ∥ ∥ F 2 ( D ) x ⋆ ∥ ) ≤ A ( η ) ( ∥ δ ∥ ∥ Δ x ⋆ ∥ ) (38) \begin{pmatrix}
\lVert F_1(D)\rVert\\
\lVert F_2(D)_{x^\star}\rVert
\end{pmatrix}
\le A(\eta)
\begin{pmatrix}
\lVert\delta\rVert\\
\lVert\Delta_{x^\star}\rVert \tag*{(38)}
\end{pmatrix} ( ∥ F 1 ( D )∥ ∥ F 2 ( D ) x ⋆ ∥ ) ≤ A ( η ) ( ∥ δ ∥ ∥ Δ x ⋆ ∥ ) (38)
holds over the subset { D ∈ T w ⋆ ( G r k ( T M ) ) : ∥ D ∥ w ⋆ < r 3 } \{D \in\mathrm{T}_{w^\star}(\mathrm{Gr}_k(\mathcal{T}\mathcal{M})) : \lVert D\rVert_{w^\star} < r_3\} { D ∈ T w ⋆ ( Gr k ( T M )) : ∥ D ∥ w ⋆ < r 3 } , where
A ( η ) : = ( q 1 ( η ) + c 3 r 3 c 3 r 3 η M + c 3 r 3 q 2 ( η ) + c 3 r 3 ) , A(\eta) :=
\begin{pmatrix}
q_1(\eta)+c_3r_3 & c_3r_3 \\
\eta M+c_3r_3 & q_2(\eta)+c_3r_3
\end{pmatrix}, A ( η ) := ( q 1 ( η ) + c 3 r 3 η M + c 3 r 3 c 3 r 3 q 2 ( η ) + c 3 r 3 ) ,
M : = ∥ m 3 ( w ⋆ ) x ⋆ ∥ M := \lVert m_3(w^\star)_{x^\star}\rVert M := ∥ m 3 ( w ⋆ ) x ⋆ ∥ , F 2 ( D ) x ⋆ F_2(D)_{x^\star} F 2 ( D ) x ⋆ and m 3 ( w ⋆ ) x ⋆ m_3(w^\star)_{x^\star} m 3 ( w ⋆ ) x ⋆ are defined similarly as in Eq. (15) ,
F 2 ( D ) x ⋆ : = Proj T x ⋆ M ∘ F 2 ( D ) ∘ Proj T x ⋆ M , F_2(D)_{x^\star} := \operatorname{Proj}_{\mathrm{T}_{x^\star}\mathcal{M}} \circ F_2(D) \circ\operatorname{Proj}_{\mathrm{T}_{x^\star}\mathcal{M}}, F 2 ( D ) x ⋆ := Proj T x ⋆ M ∘ F 2 ( D ) ∘ Proj T x ⋆ M ,
m 3 ( w ⋆ ) x ⋆ : = Proj T x ⋆ M ∘ m 3 ( w ⋆ ) ∘ Proj T x ⋆ M , m_3(w^\star)_{x^\star} := \operatorname{Proj}_{\mathrm{T}_{x^\star}\mathcal{M}} \circ m_3(w^\star) \circ\operatorname{Proj}_{\mathrm{T}_{x^\star}\mathcal{M}}, m 3 ( w ⋆ ) x ⋆ := Proj T x ⋆ M ∘ m 3 ( w ⋆ ) ∘ Proj T x ⋆ M ,
and
q 1 ( η ) : = max { ∣ 1 − η λ min ⋆ ∣ , ∣ 1 − η λ max ⋆ ∣ } , q_1(\eta) := \max\{|1-\eta\lambda^\star_{\min}|, |1-\eta\lambda^\star_{\max}|\}, q 1 ( η ) := max { ∣1 − η λ m i n ⋆ ∣ , ∣1 − η λ m a x ⋆ ∣ } ,
q 2 ( η ) : = max { ∣ 1 − η Δ λ k + 1 , k ⋆ ∣ , ∣ 1 − η Δ λ d 1 ⋆ ∣ } , q_2(\eta) := \max\{|1-\eta\Delta\lambda^\star_{k+1,k}|, |1-\eta\Delta\lambda^\star_{d1}|\}, q 2 ( η ) := max { ∣1 − η Δ λ k + 1 , k ⋆ ∣ , ∣1 − η Δ λ d 1 ⋆ ∣ } ,
λ min ⋆ : = min i = 1 d ∣ λ i ⋆ ∣ , λ max ⋆ : = max i = 1 d ∣ λ i ⋆ ∣ , \lambda^\star_{\min} := \min_{i=1}^{d}|\lambda^\star_i|,\qquad
\lambda^\star_{\max} := \max_{i=1}^{d}|\lambda^\star_i|, λ m i n ⋆ := i = 1 min d ∣ λ i ⋆ ∣ , λ m a x ⋆ := i = 1 max d ∣ λ i ⋆ ∣ ,
Δ λ j i ⋆ : = λ j ⋆ − λ i ⋆ , i = 1 , … , k , j = k + 1 , … , d . \Delta\lambda^\star_{ji} := \lambda^\star_j-\lambda^\star_i,\qquad i=1,\ldots,k,\quad j=k+1,\ldots,d. Δ λ j i ⋆ := λ j ⋆ − λ i ⋆ , i = 1 , … , k , j = k + 1 , … , d .
Proof. We first show the well-definedness of F F F , i.e., the Riemannian distance between w ⋆ w^\star w ⋆ and R w ~ ( D ) G r k ( T M ) ( η ⋅ h ( w ~ ( D ) ) ) \mathrm{R}^{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}_{\widetilde{w}(D)}(\eta\cdot h(\widetilde{w}(D))) R w ( D ) Gr k ( T M ) ( η ⋅ h ( w ( D ))) falls below r 1 r_1 r 1 . By the triangle inequality of the Riemannian distance,
dist G r k ( T M ) ( w ⋆ , R w ~ ( D ) G r k ( T M ) ( η ⋅ h ( w ~ ( D ) ) ) ) ≤ dist G r k ( T M ) ( w ⋆ , w ~ ( D ) ) + dist G r k ( T M ) ( w ~ ( D ) , R w ~ ( D ) G r k ( T M ) ( η ⋅ h ( w ~ ( D ) ) ) ) ≤ ∥ D ∥ w ⋆ + c 1 η ∥ h ( w ~ ( D ) ) ∥ w ~ ( D ) ≤ r 2 + c 1 η sup ∥ D ∥ w ⋆ ≤ r 1 ∥ h ( w ~ ( D ) ) ∥ w ~ ( D ) < r 1 , \begin{aligned}
\operatorname{dist}_{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}\left(w^\star,\mathrm{R}^{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}_{\widetilde{w}(D)}(\eta\cdot h(\widetilde{w}(D)))\right) \\
&\le\operatorname{dist}_{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}(w^\star,\widetilde{w}(D))+
\operatorname{dist}_{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}\left(\widetilde{w}(D),\mathrm{R}^{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}_{\widetilde{w}(D)}(\eta\cdot h(\widetilde{w}(D)))\right) \\
&\le\lVert D\rVert_{w^\star}+c_1\eta\lVert h(\widetilde{w}(D))\rVert_{\widetilde{w}(D)}
\le r_2+c_1\eta\sup_{\lVert D\rVert_{w^\star}\le r_1}\lVert h(\widetilde{w}(D))\rVert_{\widetilde{w}(D)}<r_1,
\end{aligned} dist Gr k ( T M ) ( w ⋆ , R w ( D ) Gr k ( T M ) ( η ⋅ h ( w ( D ))) ) ≤ dist Gr k ( T M ) ( w ⋆ , w ( D )) + dist Gr k ( T M ) ( w ( D ) , R w ( D ) Gr k ( T M ) ( η ⋅ h ( w ( D ))) ) ≤ ∥ D ∥ w ⋆ + c 1 η ∥ h ( w ( D )) ∥ w ( D ) ≤ r 2 + c 1 η ∥ D ∥ w ⋆ ≤ r 1 sup ∥ h ( w ( D )) ∥ w ( D ) < r 1 ,
where the second inequality leverages Lemma 7 , the fact that ∥ D ∥ w ⋆ < r 2 < r 1 \lVert D\rVert_{w^\star}<r_2<r_1 ∥ D ∥ w ⋆ < r 2 < r 1 , and the assumption (37) on η \eta η , in that
η ∥ h ( w ~ ( D ) ) ∥ w ~ ( D ) ≤ η sup ∥ D ∥ w ⋆ ≤ r 1 ∥ h ( w ~ ( D ) ) ∥ w ~ ( D ) < r 1 , \eta\lVert h(\widetilde{w}(D))\rVert_{\widetilde{w}(D)}
\le\eta\sup_{\lVert D\rVert_{w^\star}\le r_1}\lVert h(\widetilde{w}(D))\rVert_{\widetilde{w}(D)}<r_1, η ∥ h ( w ( D )) ∥ w ( D ) ≤ η ∥ D ∥ w ⋆ ≤ r 1 sup ∥ h ( w ( D )) ∥ w ( D ) < r 1 ,
the last one is again due to the assumption (37) on η \eta η . The smoothness of F F F then follows. Regarding the estimate for ∥ F ( D ) ∥ w ⋆ \lVert F(D)\rVert_{w^\star} ∥ F ( D ) ∥ w ⋆ , we first expand F F F around the origin up to first order: there exist constants r 3 ∈ ( 0 , r 2 ] r_3 \in(0,r_2] r 3 ∈ ( 0 , r 2 ] and c 3 ≥ 0 c_3 \ge0 c 3 ≥ 0 such that
F ( D ) = F ( D ) − F ( 0 ) = D F ( 0 ) [ D ] + r ( D ) , ∀ D ∈ T w ⋆ ( G r k ( T M ) ) : ∥ D ∥ w ⋆ < r 3 , F(D)=F(D)-F(0)=DF(0)[D]+r(D),\qquad\forall D\in\mathrm{T}_{w^\star}(\mathrm{Gr}_k(\mathcal{T}\mathcal{M})):\lVert D\rVert_{w^\star}<r_3, F ( D ) = F ( D ) − F ( 0 ) = D F ( 0 ) [ D ] + r ( D ) , ∀ D ∈ T w ⋆ ( Gr k ( T M )) : ∥ D ∥ w ⋆ < r 3 ,
where we have used F ( 0 ) = 0 F(0)=0 F ( 0 ) = 0 due to
h ( w ⋆ ) = 0 , R w ⋆ G r k ( T M ) ( 0 ) = w ⋆ , ( Exp w ⋆ G r k ( T M ) ) − 1 ( w ⋆ ) = 0 , h(w^\star)=0,\qquad\mathrm{R}^{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}_{w^\star}(0)=w^\star,\qquad\left(\operatorname{Exp}^{\mathrm{Gr}_k(\mathcal{T}\mathcal{M})}_{w^\star}\right)^{-1}(w^\star)=0, h ( w ⋆ ) = 0 , R w ⋆ Gr k ( T M ) ( 0 ) = w ⋆ , ( Exp w ⋆ Gr k ( T M ) ) − 1 ( w ⋆ ) = 0 ,
r ( D ) : = ( r 1 ( D ) , r 2 ( D ) ) r(D) := (r_1(D), r_2(D)) r ( D ) := ( r 1 ( D ) , r 2 ( D )) collects the higher-order terms and satisfies
∥ r 1 ( D ) ∥ , ∥ r 2 ( D ) x ⋆ ∥ < c 3 ( ∥ δ ∥ 2 + ∥ Δ x ⋆ ∥ 2 ) , \lVert r_1(D)\rVert,\ \lVert r_2(D)x_\star\rVert< c_3\left(\lVert\delta\rVert^2+\lVert\Delta x_\star\rVert^2\right), ∥ r 1 ( D )∥ , ∥ r 2 ( D ) x ⋆ ∥ < c 3 ( ∥ δ ∥ 2 + ∥ Δ x ⋆ ∥ 2 ) ,
with r 2 ( D ) x ⋆ r_2(D)x_\star r 2 ( D ) x ⋆ defined in a similar way as in Eq. (15) via two compositions with projectors. More calculations on the first-order term yield that
D F ( 0 ) [ D ] = D R Gr k ( T M ) ( w ⋆ , 0 ) [ ( D , η ⋅ D h ( w ⋆ ) [ D ] ) ] = D + η ⋅ D h ( w ⋆ ) [ D ] = ( δ − η R w ⋆ ( Hess M f ( x ⋆ ) [ δ ] ) Δ − η [ P ⋆ , [ Δ , Hess M f ( x ⋆ ) ] ] + η m 3 ( w ⋆ ) [ δ ] ) , \begin{aligned}
DF(0)[D] = D R_{\operatorname{Gr}_k(TM)}(w_\star,0)\left[(D,\eta\cdot Dh(w_\star)[D])\right] = D+\eta\cdot Dh(w_\star)[D] \\
&= \left(
\begin{array}{c}
\delta-\eta R_{w_\star}\left(\operatorname{Hess}_{\mathcal{M}}f(x_\star)[\delta]\right)\\
\Delta-\eta[P^\star,[\Delta,\operatorname{Hess}_{\mathcal{M}}f(x_\star)]]+\eta m_3(w_\star)[\delta]
\end{array}
\right),
\end{aligned} D F ( 0 ) [ D ] = D R Gr k ( T M ) ( w ⋆ , 0 ) [ ( D , η ⋅ D h ( w ⋆ ) [ D ]) ] = D + η ⋅ D h ( w ⋆ ) [ D ] = ( δ − η R w ⋆ ( Hess M f ( x ⋆ ) [ δ ] ) Δ − η [ P ⋆ , [ Δ , Hess M f ( x ⋆ )]] + η m 3 ( w ⋆ ) [ δ ] ) ,
where the second equality follows from [15 Lemma 4.21] and the last one uses Eq. (35) . All in all, we have
∥ F 1 ( D ) ∥ = ∥ δ − η R w ⋆ ( Hess M f ( x ⋆ ) [ δ ] ) + r 1 ( D ) ∥ ≤ ∥ Id T x ⋆ M − η R w ⋆ ∘ Hess M f ( x ⋆ ) ∥ ∥ δ ∥ + c 3 ( ∥ δ ∥ 2 + ∥ Δ x ⋆ ∥ 2 ) ≤ ( q 1 ( η ) + c 3 r 3 ) ∥ δ ∥ + c 3 r 3 ∥ Δ x ⋆ ∥ , \begin{aligned}
\lVert F_1(D)\rVert=\left\lVert\delta-\eta R_{w_\star}\left(\operatorname{Hess}_{\mathcal{M}}f(x_\star)[\delta]\right)+r_1(D)\right\rVert\\
&\leq\left\lVert\operatorname{Id}_{T_{x_\star}\mathcal{M}}-\eta R_{w_\star}\circ\operatorname{Hess}_{\mathcal{M}}f(x_\star)\right\rVert\lVert\delta\rVert+c_3\left(\lVert\delta\rVert^2+\lVert\Delta x_\star\rVert^2\right) \\
&\leq(q_1(\eta)+c_3r_3)\lVert\delta\rVert+c_3r_3\lVert\Delta x_\star\rVert,
\end{aligned} ∥ F 1 ( D )∥ = ∥ δ − η R w ⋆ ( Hess M f ( x ⋆ ) [ δ ] ) + r 1 ( D ) ∥ ≤ Id T x ⋆ M − η R w ⋆ ∘ Hess M f ( x ⋆ ) ∥ δ ∥ + c 3 ( ∥ δ ∥ 2 + ∥ Δ x ⋆ ∥ 2 ) ≤ ( q 1 ( η ) + c 3 r 3 ) ∥ δ ∥ + c 3 r 3 ∥ Δ x ⋆ ∥ ,
and
∥ F 2 ( D ) x ⋆ ∥ = ∥ Proj T x ⋆ M ∘ F 2 ( D ) ∘ Proj T x ⋆ M ∥ = ∥ Δ x ⋆ − η [ P ⋆ , [ Δ x ⋆ , Hess M f ( x ⋆ ) ] ] + η ( m 3 ( w ⋆ ) [ δ ] ) x ⋆ + r 2 ( D ) x ⋆ ∥ ≤ ∥ Id T P ⋆ Gr k ( T x ⋆ M ) − η [ P ⋆ , [ Id T P ⋆ Gr k ( T x ⋆ M ) , Hess M f ( x ⋆ ) ] ] ∥ ∥ Δ x ⋆ ∥ \begin{aligned}
\lVert F_2(D)x_\star\rVert=\left\lVert\operatorname{Proj}_{T_{x_\star}\mathcal{M}}\circ F_2(D)\circ\operatorname{Proj}_{T_{x_\star}\mathcal{M}}\right\rVert\\
&=\left\lVert\Delta x_\star-\eta[P^\star,[\Delta x_\star,\operatorname{Hess}_{\mathcal{M}}f(x_\star)]]+\eta(m_3(w_\star)[\delta])_{x_\star}+r_2(D)x_\star\right\rVert\\
&\leq\left\lVert\operatorname{Id}_{T_{P_\star\operatorname{Gr}_k(T_{x_\star}\mathcal{M})}}-\eta[P^\star,[\operatorname{Id}_{T_{P_\star\operatorname{Gr}_k(T_{x_\star}\mathcal{M})}},\operatorname{Hess}_{\mathcal{M}}f(x_\star)]]\right\rVert\lVert\Delta x_\star\rVert
\end{aligned} ∥ F 2 ( D ) x ⋆ ∥ = Proj T x ⋆ M ∘ F 2 ( D ) ∘ Proj T x ⋆ M = ∥ Δ x ⋆ − η [ P ⋆ , [ Δ x ⋆ , Hess M f ( x ⋆ )]] + η ( m 3 ( w ⋆ ) [ δ ] ) x ⋆ + r 2 ( D ) x ⋆ ∥ ≤ Id T P ⋆ Gr k ( T x ⋆ M ) − η [ P ⋆ , [ Id T P ⋆ Gr k ( T x ⋆ M ) , Hess M f ( x ⋆ )]] ∥ Δ x ⋆ ∥
+ η ∥ m 3 ( w ⋆ ) x ⋆ ∥ ∥ δ ∥ + c 3 r 3 ( ∥ δ ∥ + ∥ Δ x ⋆ ∥ ) ≤ ( q 2 ( η ) + c 3 r 3 ) ∥ Δ x ⋆ ∥ + ( η M + c 3 r 3 ) ∥ δ ∥ . \begin{aligned}
\qquad+\eta\lVert m_3(w_\star)_{x_\star}\rVert\lVert\delta\rVert+c_3r_3\left(\lVert\delta\rVert+\lVert\Delta x_\star\rVert\right) \\
&\leq(q_2(\eta)+c_3r_3)\lVert\Delta x_\star\rVert+(\eta M+c_3r_3)\lVert\delta\rVert.
\end{aligned} + η ∥ m 3 ( w ⋆ ) x ⋆ ∥ ∥ δ ∥ + c 3 r 3 ( ∥ δ ∥ + ∥ Δ x ⋆ ∥ ) ≤ ( q 2 ( η ) + c 3 r 3 ) ∥ Δ x ⋆ ∥ + ( η M + c 3 r 3 ) ∥ δ ∥ .
The proof is complete.
Lemma 9. Let X X X be a 2 × 2 2\times2 2 × 2 matrix with distinct eigenvalues μ 1 \mu_1 μ 1 and μ 2 \mu_2 μ 2 . Then for any m ∈ N m\in\mathbb{N} m ∈ N , it holds that
X m = μ 1 m − μ 2 m μ 1 − μ 2 X − μ 2 μ 1 m − μ 1 μ 2 m μ 1 − μ 2 I 2 . X^m=\frac{\mu_1^m-\mu_2^m}{\mu_1-\mu_2}X-\frac{\mu_2\mu_1^m-\mu_1\mu_2^m}{\mu_1-\mu_2}I_2. X m = μ 1 − μ 2 μ 1 m − μ 2 m X − μ 1 − μ 2 μ 2 μ 1 m − μ 1 μ 2 m I 2 .
Proof. Since X X X is 2 × 2 2\times2 2 × 2 , the Cayley-Hamilton theorem tells us that X X X satisfies its own characteristic equation:
X 2 − ( μ 1 + μ 2 ) X + μ 1 μ 2 I 2 = 0. (39) X^2-(\mu_1+\mu_2)X+\mu_1\mu_2I_2=0. \tag*{(39)} X 2 − ( μ 1 + μ 2 ) X + μ 1 μ 2 I 2 = 0. (39)
In what follows, we prove the desired result by induction. The conclusion holds obviously for the cases of m = 0 , 1 , 2 m=0,1,2 m = 0 , 1 , 2 . Now suppose that the conclusion holds for the case of m = ℓ ∈ N m=\ell\in\mathbb{N} m = ℓ ∈ N and consider the case of m = ℓ + 1 m=\ell+1 m = ℓ + 1 . Direct calculations yield
X ℓ + 1 = X ℓ ⋅ X = ( μ 1 ℓ − μ 2 ℓ μ 1 − μ 2 X − μ 2 μ 1 ℓ − μ 1 μ 2 ℓ μ 1 − μ 2 I 2 ) X = μ 1 ℓ − μ 2 ℓ μ 1 − μ 2 ( ( μ 1 + μ 2 ) X − μ 1 μ 2 I 2 ) − μ 2 μ 1 ℓ − μ 1 μ 2 ℓ μ 1 − μ 2 X = μ 1 ℓ + 1 − μ 2 ℓ + 1 μ 1 − μ 2 X − μ 2 μ 1 ℓ + 1 − μ 1 μ 2 ℓ + 1 μ 1 − μ 2 I 2 . \begin{aligned}
X^{\ell+1}=X^\ell\cdot X=\left(\frac{\mu_1^\ell-\mu_2^\ell}{\mu_1-\mu_2}X-\frac{\mu_2\mu_1^\ell-\mu_1\mu_2^\ell}{\mu_1-\mu_2}I_2\right)X \\
&=\frac{\mu_1^\ell-\mu_2^\ell}{\mu_1-\mu_2}\left((\mu_1+\mu_2)X-\mu_1\mu_2I_2\right)-\frac{\mu_2\mu_1^\ell-\mu_1\mu_2^\ell}{\mu_1-\mu_2}X \\
&=\frac{\mu_1^{\ell+1}-\mu_2^{\ell+1}}{\mu_1-\mu_2}X-\frac{\mu_2\mu_1^{\ell+1}-\mu_1\mu_2^{\ell+1}}{\mu_1-\mu_2}I_2.
\end{aligned} X ℓ + 1 = X ℓ ⋅ X = ( μ 1 − μ 2 μ 1 ℓ − μ 2 ℓ X − μ 1 − μ 2 μ 2 μ 1 ℓ − μ 1 μ 2 ℓ I 2 ) X = μ 1 − μ 2 μ 1 ℓ − μ 2 ℓ ( ( μ 1 + μ 2 ) X − μ 1 μ 2 I 2 ) − μ 1 − μ 2 μ 2 μ 1 ℓ − μ 1 μ 2 ℓ X = μ 1 − μ 2 μ 1 ℓ + 1 − μ 2 ℓ + 1 X − μ 1 − μ 2 μ 2 μ 1 ℓ + 1 − μ 1 μ 2 ℓ + 1 I 2 .
Here the second equality follows from mathematical induction and the third one uses Eq. (39) . As a result, the conclusion holds for the case of m = ℓ + 1 m=\ell+1 m = ℓ + 1 as well. The proof is complete.
Theorem 3 (Local convergence of Algorithm 1). Suppose that f f f is C 3 C^3 C 3 , M \mathcal{M} M is an embedded Riemannian submanifold of a Euclidean space E \mathcal{E} E with dim ( M ) = d \dim(\mathcal{M})=d dim ( M ) = d , 1 ≤ k < d 1 \le k < d 1 ≤ k < d , x ⋆ ∈ M x^\star\in\mathcal{M} x ⋆ ∈ M is an index-k k k constrained SP of f f f on M \mathcal{M} M , Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}} f(x^\star) Hess M f ( x ⋆ ) is nondegenerate, whose eigenvalues are λ 1 ⋆ ≤ ⋯ ≤ λ k ⋆ < 0 < λ k + 1 ⋆ ≤ ⋯ ≤ λ d ⋆ \lambda_1^\star\le\cdots\le\lambda_k^\star< 0 < \lambda_{k+1}^\star\le\cdots\le\lambda_d^\star λ 1 ⋆ ≤ ⋯ ≤ λ k ⋆ < 0 < λ k + 1 ⋆ ≤ ⋯ ≤ λ d ⋆ , and P ⋆ ∈ Gr k ( T x ⋆ M ) P^\star\in\operatorname{Gr}_k(T_{x^\star}\mathcal{M}) P ⋆ ∈ Gr k ( T x ⋆ M ) is an orthogonal projector onto the lowest k k k -dimensional invariant subspace of Hess M f ( x ⋆ ) \operatorname{Hess}_{\mathcal{M}} f(x^\star) Hess M f ( x ⋆ ) . Let w ⋆ : = ( x ⋆ , P ⋆ ) ∈ Gr k ( T M ) w^\star:= (x^\star,P^\star) \in\operatorname{Gr}_k(T\mathcal{M}) w ⋆ := ( x ⋆ , P ⋆ ) ∈ Gr k ( T M ) and { w ( t ) : = ( x ( t ) , P ( t ) ) } \{w^{(t)} := (x^{(t)},P^{(t)})\} { w ( t ) := ( x ( t ) , P ( t ) )} be the sequence generated by Algorithm 1. If the following two assumptions hold:
0 < η < min { 2 Δ λ d 1 ⋆ , q 1 ( η ) + q 2 ( η ) M , min { ( r 1 − r 2 ) / c 1 , r 1 } sup ∥ D ∥ w ⋆ ≤ r 1 k h ( w ~ ( D ) ) k w ~ ( D ) } (40) 0 < \eta< \min\left\{ \frac{2}{\Delta\lambda_{d_1}^\star}, \frac{q_1(\eta)+q_2(\eta)}{M}, \frac{\min\{(r_1-r_2)/c_1,r_1\}}{\sup_{\lVert D\rVert_{w^\star}\le r_1 k_h(\widetilde{w}(D))k_{\widetilde{w}(D)}}} \right\} \tag*{(40)} 0 < η < min { Δ λ d 1 ⋆ 2 , M q 1 ( η ) + q 2 ( η ) , sup ∥ D ∥ w ⋆ ≤ r 1 k h ( w ( D )) k w ( D ) min {( r 1 − r 2 ) / c 1 , r 1 } } (40) with c 1 c_1 c 1 and r 1 r_1 r 1 the constants in Lemma 7 (associated to N = Gr k ( T M ) \mathcal{N}=\operatorname{Gr}_k(T\mathcal{M}) N = Gr k ( T M ) , R N = R Gr k ( T M ) \mathbb{R}^{\mathcal{N}}=\mathbb{R}^{\operatorname{Gr}_k(T\mathcal{M})} R N = R Gr k ( T M ) or Exp Gr k ( T M ) \operatorname{Exp}_{\operatorname{Gr}_k(T\mathcal{M})} Exp Gr k ( T M ) , and y = w ⋆ y=w^\star y = w ⋆ ) and r 2 ∈ ( 0 , r 1 ) r_2 \in(0,r_1) r 2 ∈ ( 0 , r 1 ) ;
dist Gr k ( T M ) ( w ⋆ , w ( 0 ) ) < r 3 max { max { μ 1 ( η ) 2 μ 1 ( η ) − μ 2 ( η ) , 1 } ∥ A ( η ) ∥ , 1 } , (41) \operatorname{dist}_{\operatorname{Gr}_k(T\mathcal{M})}(w^\star,w^{(0)}) < \frac{r_3}{\max\left\{\max\left\{\frac{\mu_1(\eta)^2}{\mu_1(\eta)-\mu_2(\eta)},1\right\}\lVert A(\eta)\rVert,1\right\}}, \tag*{(41)} dist Gr k ( T M ) ( w ⋆ , w ( 0 ) ) < max { max { μ 1 ( η ) − μ 2 ( η ) μ 1 ( η ) 2 , 1 } ∥ A ( η )∥ , 1 } r 3 , (41) where the constants c 3 c_3 c 3 and r 3 r_3 r 3 are defined in Lemma 8 and are chosen such that
c 3 r 3 < ( 1 − q max ( η ) ) 2 η M + 2 ( 1 − q max ( η ) ) with q max ( η ) : = max { q 1 ( η ) , q 2 ( η ) } , (42) c_3r_3 < \frac{(1-q_{\max}(\eta))^2}{\eta M+2(1-q_{\max}(\eta))} \quad\text{with}\quad q_{\max}(\eta):=\max\{q_1(\eta),q_2(\eta)\}, \tag*{(42)} c 3 r 3 < η M + 2 ( 1 − q m a x ( η )) ( 1 − q m a x ( η ) ) 2 with q m a x ( η ) := max { q 1 ( η ) , q 2 ( η )} , (42) and μ 1 ( η ) \mu_1(\eta) μ 1 ( η ) , μ 2 ( η ) \mu_2(\eta) μ 2 ( η ) are respectively
μ 1 ( η ) : = q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 + ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( η M + c 3 r 3 ) 2 , \mu_1(\eta):=\frac{q_1(\eta)+q_2(\eta)+2c_3r_3+\sqrt{(q_1(\eta)-q_2(\eta))^2+4c_3r_3(\eta M+c_3r_3)}}{2}, μ 1 ( η ) := 2 q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 + ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( η M + c 3 r 3 ) , μ 2 ( η ) : = q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 − ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( η M + c 3 r 3 ) 2 , (43) \mu_2(\eta):=\frac{q_1(\eta)+q_2(\eta)+2c_3r_3-\sqrt{(q_1(\eta)-q_2(\eta))^2+4c_3r_3(\eta M+c_3r_3)}}{2}, \tag*{(43)} μ 2 ( η ) := 2 q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 − ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( η M + c 3 r 3 ) , (43) then { w ( t ) } \{w^{(t)}\} { w ( t ) } converges linearly to w ⋆ w^\star w ⋆ .
Proof. We first show by induction that dist Gr k ( T M ) ( w ⋆ , w ( t ) ) < r 3 \operatorname{dist}_{\operatorname{Gr}_k(T\mathcal{M})}(w^\star,w^{(t)})<r_3 dist Gr k ( T M ) ( w ⋆ , w ( t ) ) < r 3 holds for any t t t , so that Exp w ⋆ Gr k ( T M ) \operatorname{Exp}_{w^\star}^{\operatorname{Gr}_k(T\mathcal{M})} Exp w ⋆ Gr k ( T M ) remains a diffeomorphism and (38) is applicable for all the iterates. The case of t = 0 t=0 t = 0 is evident due to the assumption (41) on the initial point. Now suppose that the statement holds for the case of t = ℓ ∈ N t=\ell\in\mathbb{N} t = ℓ ∈ N and consider the case of t = ℓ + 1 t=\ell+1 t = ℓ + 1 . By induction and (38) , there exists a unique D ( t + 1 ) ∈ T w ⋆ ( Gr k ( T M ) ) D^{(t+1)}\in T_{w^\star}(\operatorname{Gr}_k(T\mathcal{M})) D ( t + 1 ) ∈ T w ⋆ ( Gr k ( T M )) such that ∥ D ( t + 1 ) ∥ w ⋆ < r 3 \lVert D^{(t+1)}\rVert_{w^\star}<r_3 ∥ D ( t + 1 ) ∥ w ⋆ < r 3 and D ( t + 1 ) = F ( D ( t ) ) D^{(t+1)}=F(D^{(t)}) D ( t + 1 ) = F ( D ( t ) ) hold for t = 0 , … , ℓ − 1 t=0,\ldots,\ell-1 t = 0 , … , ℓ − 1 , and that
\begin{pmatrix}
\lVert F_1(D^{(\ell)})\rVert\\
\lVert F_2(D^{(\ell)})x^\star\rVert
\end{pmatrix}
\le A(\eta)^{\ell+1}
\begin{pmatrix}
\lVert D^{(0)}\rVert\\
\lVert\Delta_{x^\star}^{(0)}\rVert
\end{pmatrix}. \tag*{(44)}\end{pmatrix}
It is not difficult to verify that μ 1 ( η ) \mu_1(\eta) μ 1 ( η ) and μ 2 ( η ) \mu_2(\eta) μ 2 ( η ) in (43) are the two distinct real eigenvalues of A ( η ) A(\eta) A ( η ) , and both of them belong to ( 0 , 1 ) (0,1) ( 0 , 1 ) :
2 μ 2 ( η ) = q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 − ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( η M + c 3 r 3 ) > q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 − ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( q 1 ( η ) + q 2 ( η ) + c 3 r 3 ) = q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 − ( q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 ) 2 − 4 q 1 ( η ) q 2 ( η ) ≥ 0 , \begin{aligned}
2\mu_2(\eta)=q_1(\eta)+q_2(\eta)+2c_3r_3-\sqrt{(q_1(\eta)-q_2(\eta))^2+4c_3r_3(\eta M+c_3r_3)} \\
&>q_1(\eta)+q_2(\eta)+2c_3r_3-\sqrt{(q_1(\eta)-q_2(\eta))^2+4c_3r_3(q_1(\eta)+q_2(\eta)+c_3r_3)} \\
&=q_1(\eta)+q_2(\eta)+2c_3r_3-\sqrt{(q_1(\eta)+q_2(\eta)+2c_3r_3)^2-4q_1(\eta)q_2(\eta)}\ge0,
\end{aligned} 2 μ 2 ( η ) = q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 − ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( η M + c 3 r 3 ) > q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 − ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( q 1 ( η ) + q 2 ( η ) + c 3 r 3 ) = q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 − ( q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 ) 2 − 4 q 1 ( η ) q 2 ( η ) ≥ 0 ,
::: where the strict inequality is due to the assumption (40), and
2 μ 1 ( η ) = q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 + ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( η M + c 3 r 3 ) ≤ q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 + ∣ q 1 ( η ) − q 2 ( η ) ∣ + 2 c 3 r 3 ( η M + c 3 r 3 ) = 2 ( c 3 r 3 + q max ( η ) + c 3 r 3 ( η M + c 3 r 3 ) ) < 2 ( ( 1 − q max ( η ) ) 2 η M + 2 ( 1 − q max ( η ) ) + q max ( η ) + ( 1 − q max ( η ) ) 2 ( η M + 1 − q max ( η ) ) 2 ( η M + 2 ( 1 − q max ( η ) ) ) 2 ) = 2 ( ( 1 − q max ( η ) ) 2 η M + 2 ( 1 − q max ( η ) ) + q max ( η ) + ( 1 − q max ( η ) ) ( η M + 1 − q max ( η ) ) η M + 2 ( 1 − q max ( η ) ) ) = 2 , \begin{aligned}
2\mu_1(\eta)
&=q_1(\eta)+q_2(\eta)+2c_3r_3+\sqrt{(q_1(\eta)-q_2(\eta))^2+4c_3r_3(\eta M+c_3r_3)}\\
&\le q_1(\eta)+q_2(\eta)+2c_3r_3+\lvert q_1(\eta)-q_2(\eta)\rvert+2\sqrt{c_3r_3(\eta M+c_3r_3)}\\
&=2\left(c_3r_3+q_{\max}(\eta)+\sqrt{c_3r_3(\eta M+c_3r_3)}\right)\\
&<2\left(\frac{(1-q_{\max}(\eta))^2}{\eta M+2(1-q_{\max}(\eta))}+q_{\max}(\eta)+\sqrt{\frac{(1-q_{\max}(\eta))^2(\eta M+1-q_{\max}(\eta))^2}{(\eta M+2(1-q_{\max}(\eta)))^2}}\right)\\
&=2\left(\frac{(1-q_{\max}(\eta))^2}{\eta M+2(1-q_{\max}(\eta))}+q_{\max}(\eta)+\frac{(1-q_{\max}(\eta))(\eta M+1-q_{\max}(\eta))}{\eta M+2(1-q_{\max}(\eta))}\right)=2,
\end{aligned} 2 μ 1 ( η ) = q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 + ( q 1 ( η ) − q 2 ( η ) ) 2 + 4 c 3 r 3 ( η M + c 3 r 3 ) ≤ q 1 ( η ) + q 2 ( η ) + 2 c 3 r 3 + ∣ q 1 ( η ) − q 2 ( η )∣ + 2 c 3 r 3 ( η M + c 3 r 3 ) = 2 ( c 3 r 3 + q m a x ( η ) + c 3 r 3 ( η M + c 3 r 3 ) ) < 2 ( η M + 2 ( 1 − q m a x ( η )) ( 1 − q m a x ( η ) ) 2 + q m a x ( η ) + ( η M + 2 ( 1 − q m a x ( η )) ) 2 ( 1 − q m a x ( η ) ) 2 ( η M + 1 − q m a x ( η ) ) 2 ) = 2 ( η M + 2 ( 1 − q m a x ( η )) ( 1 − q m a x ( η ) ) 2 + q m a x ( η ) + η M + 2 ( 1 − q m a x ( η )) ( 1 − q m a x ( η )) ( η M + 1 − q m a x ( η )) ) = 2 ,
where the strict inequality uses the assumption (42) and the third equality uses q 1 ( η ) , q 2 ( η ) < 1 q_1(\eta),q_2(\eta)<1 q 1 ( η ) , q 2 ( η ) < 1 from the assumption (40). By Lemma 9,
A ( η ) ℓ + 1 ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) (45) A(\eta)^{\ell+1}
\begin{pmatrix}
\lVert\delta^{(0)}\rVert\\
\lVert\Delta_{x^\star}^{(0)}\rVert
\end{pmatrix} \tag*{(45)} A ( η ) ℓ + 1 ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) (45)
= ( μ 1 ( η ) ℓ + 1 − μ 2 ( η ) ℓ + 1 μ 1 ( η ) − μ 2 ( η ) A ( η ) − μ 2 ( η ) μ 1 ( η ) ℓ + 1 − μ 1 ( η ) μ 2 ( η ) ℓ + 1 μ 1 ( η ) − μ 2 ( η ) I 2 ) ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) ≤ max { μ 1 ( η ) 2 μ 1 ( η ) − μ 2 ( η ) , 1 } A ( η ) ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) , \begin{aligned}
&=\left(\frac{\mu_1(\eta)^{\ell+1}-\mu_2(\eta)^{\ell+1}}{\mu_1(\eta)-\mu_2(\eta)}A(\eta)-\frac{\mu_2(\eta)\mu_1(\eta)^{\ell+1}-\mu_1(\eta)\mu_2(\eta)^{\ell+1}}{\mu_1(\eta)-\mu_2(\eta)}I_2\right)
\begin{pmatrix}
\lVert\delta^{(0)}\rVert\\
\lVert\Delta_{x^\star}^{(0)}\rVert
\end{pmatrix}\\
&\le\max\left\{\frac{\mu_1(\eta)^2}{\mu_1(\eta)-\mu_2(\eta)},1\right\}A(\eta)
\begin{pmatrix}
\lVert\delta^{(0)}\rVert\\
\lVert\Delta_{x^\star}^{(0)}\rVert
\end{pmatrix},
\end{aligned} = ( μ 1 ( η ) − μ 2 ( η ) μ 1 ( η ) ℓ + 1 − μ 2 ( η ) ℓ + 1 A ( η ) − μ 1 ( η ) − μ 2 ( η ) μ 2 ( η ) μ 1 ( η ) ℓ + 1 − μ 1 ( η ) μ 2 ( η ) ℓ + 1 I 2 ) ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) ≤ max { μ 1 ( η ) − μ 2 ( η ) μ 1 ( η ) 2 , 1 } A ( η ) ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) ,
where the inequality is due to 0 < μ 2 ( η ) < μ 1 ( η ) < 1 0<\mu_2(\eta)<\mu_1(\eta)<1 0 < μ 2 ( η ) < μ 1 ( η ) < 1 . Eqs. (44) , (45) , and the assumption (41) on the initial point then together imply
∥ F ( D ( ℓ ) ) ∥ w ⋆ 2 = ∥ F 1 ( D ( ℓ ) ) ∥ 2 + ∥ F 2 ( D ( ℓ ) ) x ⋆ ∥ 2 ≤ ∥ A ( η ) ℓ + 1 ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) ∥ 2 ≤ max { μ 1 ( η ) 2 μ 1 ( η ) − μ 2 ( η ) , 1 } 2 ∥ A ( η ) ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) ∥ 2 ≤ max { μ 1 ( η ) 2 μ 1 ( η ) − μ 2 ( η ) , 1 } 2 ∥ A ( η ) ∥ 2 ∥ D ( 0 ) ∥ w ⋆ 2 < r 3 2 . \begin{aligned}
\lVert F(D^{(\ell)})\rVert_{w^\star}^2
&=\lVert F_1(D^{(\ell)})\rVert^2+\lVert F_2(D^{(\ell)})_{x^\star}\rVert^2
\le\left\lVert A(\eta)^{\ell+1}
\begin{pmatrix}
\lVert\delta^{(0)}\rVert\\
\lVert\Delta_{x^\star}^{(0)}\rVert
\end{pmatrix}\right\rVert^2\\
&\le\max\left\{\frac{\mu_1(\eta)^2}{\mu_1(\eta)-\mu_2(\eta)},1\right\}^2
\left\lVert A(\eta)
\begin{pmatrix}
\lVert\delta^{(0)}\rVert\\
\lVert\Delta_{x^\star}^{(0)}\rVert
\end{pmatrix}\right\rVert^2\\
&\le\max\left\{\frac{\mu_1(\eta)^2}{\mu_1(\eta)-\mu_2(\eta)},1\right\}^2\lVert A(\eta)\rVert^2\lVert D^{(0)}\rVert_{w^\star}^2<r_3^2.
\end{aligned} ∥ F ( D ( ℓ ) ) ∥ w ⋆ 2 = ∥ F 1 ( D ( ℓ ) ) ∥ 2 + ∥ F 2 ( D ( ℓ ) ) x ⋆ ∥ 2 ≤ A ( η ) ℓ + 1 ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) 2 ≤ max { μ 1 ( η ) − μ 2 ( η ) μ 1 ( η ) 2 , 1 } 2 A ( η ) ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) 2 ≤ max { μ 1 ( η ) − μ 2 ( η ) μ 1 ( η ) 2 , 1 } 2 ∥ A ( η ) ∥ 2 ∥ D ( 0 ) ∥ w ⋆ 2 < r 3 2 .
Consequently, there exists a unique D ( ℓ + 1 ) ∈ T w ⋆ ( Gr k ( T M ) ) D^{(\ell+1)}\in\mathbb{T}_{w^\star}(\operatorname{Gr}_{k}(T\mathcal{M})) D ( ℓ + 1 ) ∈ T w ⋆ ( Gr k ( T M )) such that ∥ D ( ℓ + 1 ) ∥ < r 3 \lVert D^{(\ell+1)}\rVert<r_3 ∥ D ( ℓ + 1 ) ∥ < r 3 and D ( ℓ + 1 ) = F ( D ( ℓ ) ) D^{(\ell+1)}=F(D^{(\ell)}) D ( ℓ + 1 ) = F ( D ( ℓ ) ) . The statement thus holds for any t ∈ N t\in\mathbb{N} t ∈ N .
Now by Eqs. (44) and (45) , we have for any t ≥ 1 t\ge1 t ≥ 1 that
( ∥ δ ( t ) ∥ ∥ Δ x ⋆ ( t ) ∥ ) ≤ ( μ 1 ( η ) t − μ 2 ( η ) t μ 1 ( η ) − μ 2 ( η ) A ( η ) − μ 2 ( η ) μ 1 ( η ) t − μ 1 ( η ) μ 2 ( η ) t μ 1 ( η ) − μ 2 ( η ) I 2 ) ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) , \begin{pmatrix}
\lVert\delta^{(t)}\rVert\\
\lVert\Delta_{x^\star}^{(t)}\rVert
\end{pmatrix}
\le\left(\frac{\mu_1(\eta)^t-\mu_2(\eta)^t}{\mu_1(\eta)-\mu_2(\eta)}A(\eta)-\frac{\mu_2(\eta)\mu_1(\eta)^t-\mu_1(\eta)\mu_2(\eta)^t}{\mu_1(\eta)-\mu_2(\eta)}I_2\right)
\begin{pmatrix}
\lVert\delta^{(0)}\rVert\\
\lVert\Delta_{x^\star}^{(0)}\rVert
\end{pmatrix}, ( ∥ δ ( t ) ∥ ∥ Δ x ⋆ ( t ) ∥ ) ≤ ( μ 1 ( η ) − μ 2 ( η ) μ 1 ( η ) t − μ 2 ( η ) t A ( η ) − μ 1 ( η ) − μ 2 ( η ) μ 2 ( η ) μ 1 ( η ) t − μ 1 ( η ) μ 2 ( η ) t I 2 ) ( ∥ δ ( 0 ) ∥ ∥ Δ x ⋆ ( 0 ) ∥ ) ,
which combined with 0 ≤ μ 2 ( η ) < μ 1 ( η ) < 1 0\le\mu_2(\eta)<\mu_1(\eta)<1 0 ≤ μ 2 ( η ) < μ 1 ( η ) < 1 and ∥ D ( t ) ∥ w ⋆ 2 = ∥ δ ( t ) ∥ 2 + ∥ Δ x ⋆ ( t ) ∥ 2 \lVert D^{(t)}\rVert_{w^\star}^2=\lVert\delta^{(t)}\rVert^2+\lVert\Delta_{x^\star}^{(t)}\rVert^2 ∥ D ( t ) ∥ w ⋆ 2 = ∥ δ ( t ) ∥ 2 + ∥ Δ x ⋆ ( t ) ∥ 2 implies that D ( t ) D^{(t)} D ( t ) converges linearly to 0 0 0 , or equivalently, { w ( t ) } \{w^{(t)}\} { w ( t ) } converges linearly to w ⋆ w^\star w ⋆ as t → ∞ t\to\infty t → ∞ . The proof is completed.
Remark 4 (Satisfiability of the assumptions). It might seem to be difficult at first glance to check the satisfiability of the assumptions (40) and (42). Indeed, after noting that q 1 ( η ) , q 2 ( η ) → 1 − q_1(\eta),q_2(\eta)\to1- q 1 ( η ) , q 2 ( η ) → 1 − as η → 0 + \eta\to0+ η → 0 + , the assumption (40) can always be met with a sufficiently small η \eta η . Once η \eta η is fixed, one could choose an r 3 r_3 r 3 small enough to meet the assumption (42) , since c 3 c_3 c 3 , as a function of r 3 r_3 r 3 , is uniformly bounded in a small neighborhood of the origin.
It is worth mentioning that the constants showing up in Eqs. (40) and (42) are not intended as directly computable quantities. They depend on, e.g., local smoothness of the objective function, Riemannian manifold, and retraction, and are introduced mainly for theoretical convenience. In addition, conditions (40) and (41) are sufficient but not definitely necessary for the local result, and the established error bound is not guaranteed to be sharp. Nonetheless, we shall point out that Theorem 3 should better be understood as the first local iterate convergence result for a discretized saddle-search algorithm in the embedded-submanifold setting. Completely closing the gap between theory and practice is important and will be left for future investigations.
Example 2 (Condition numbers at index-1 constrained SPs). Consider a linear objective function on M = Gr p ( R n ) \mathcal{M}=\operatorname{Gr}_{p}(\mathbb{R}^{n}) M = Gr p ( R n ) (p > 1 p>1 p > 1 ) defined by f ( P ) = Tr ( P A ) f(P)=\operatorname{Tr}(PA) f ( P ) = Tr ( P A ) for any P ∈ M P\in\mathcal{M} P ∈ M , where A ∈ R s y m n × n A\in\mathbb{R}_{\mathrm{sym}}^{n\times n} A ∈ R sym n × n . It is not difficult to obtain a closed-form expression for the Riemannian Hessian:
Hess M f ( P ) [ Γ ] = [ P , [ Γ , A ] ] , ∀ P ∈ M , Γ ∈ T P M . \operatorname{Hess}_{\mathcal{M}}f(P)[\Gamma]=[P,[\Gamma,A]],\quad\forall P\in\mathcal{M},\ \Gamma\in T_{P}\mathcal{M}. Hess M f ( P ) [ Γ ] = [ P , [ Γ , A ]] , ∀ P ∈ M , Γ ∈ T P M . For simplicity, suppose that A A A is diagonal, i.e., A = diag ( ε 1 , … , ε n ) A=\operatorname{diag}(\varepsilon_{1},\ldots,\varepsilon_{n}) A = diag ( ε 1 , … , ε n ) with ε 1 ≤ ⋯ ≤ ε n \varepsilon_{1}\leq\cdots\leq\varepsilon_{n} ε 1 ≤ ⋯ ≤ ε n , and that all the sums of p p p eigenvalues of A A A are different. Under this assumption, one could verify that the unique global minimizer (GM) and unique index-1 constrained SP of f f f on M \mathcal{M} M are respectively
P G M : = ∑ i = 1 p e i e i T and P S P : = ∑ i = 1 p − 1 e i e i T + e p + 1 e p + 1 T , P_{\mathrm{GM}}:=\sum_{i=1}^{p}e_{i}e_{i}^{\mathsf{T}}\quad\text{and}\quad P_{\mathrm{SP}}:=\sum_{i=1}^{p-1}e_{i}e_{i}^{\mathsf{T}}+e_{p+1}e_{p+1}^{\mathsf{T}}, P GM := i = 1 ∑ p e i e i T and P SP := i = 1 ∑ p − 1 e i e i T + e p + 1 e p + 1 T , where e i e_i e i is the i i i -th unit vector in R n \mathbb{R}^{n} R n (i = 1 , … , n i=1,\ldots,n i = 1 , … , n ). The eigenpairs of Hess M f ( P G M ) \operatorname{Hess}_{\mathcal{M}}f(P_{\mathrm{GM}}) Hess M f ( P GM ) are given explicitly by
{ ( ε a − ε i , 1 2 ( e i e a T + e a e i T ) ) | i = 1 , … , p , a = p + 1 , … , n } . \left\{\left(\varepsilon_{a}-\varepsilon_{i},\frac{1}{\sqrt{2}}\left(e_{i}e_{a}^{\mathsf{T}}+e_{a}e_{i}^{\mathsf{T}}\right)\right)\middle|i=1,\ldots,p,\ a=p+1,\ldots,n\right\}. { ( ε a − ε i , 2 1 ( e i e a T + e a e i T ) ) i = 1 , … , p , a = p + 1 , … , n } . and for Hess M f ( P S P ) \operatorname{Hess}_{\mathcal{M}}f(P_{\mathrm{SP}}) Hess M f ( P SP ) ,
{ ( ε a − ε i , 1 2 ( e i e a T + e a e i T ) ) | i = 1 , … , p − 1 , p + 1 , a = p , p + 2 , … , n } . \left\{\left(\varepsilon_{a}-\varepsilon_{i},\frac{1}{\sqrt{2}}\left(e_{i}e_{a}^{\mathsf{T}}+e_{a}e_{i}^{\mathsf{T}}\right)\right)\middle|i=1,\ldots,p-1,p+1,\ a=p,p+2,\ldots,n\right\}. { ( ε a − ε i , 2 1 ( e i e a T + e a e i T ) ) i = 1 , … , p − 1 , p + 1 , a = p , p + 2 , … , n } . The condition number at P G M P_{\mathrm{GM}} P GM is
κ P G M = ε n − ε 1 ε p + 1 − ε p , \kappa_{P_{\mathrm{GM}}}=\frac{\varepsilon_{n}-\varepsilon_{1}}{\varepsilon_{p+1}-\varepsilon_{p}}, κ P GM = ε p + 1 − ε p ε n − ε 1 , which is well known in the literature. The denominator is also called the “eigengap” in some applications. Regarding the condition number at P S P P_{\mathrm{SP}} P SP , with reference to Eq. (48) , we could identify λ 1 ⋆ = ε p − ε p + 1 < 0 \lambda_{1}^{\star}=\varepsilon_{p}-\varepsilon_{p+1}<0 λ 1 ⋆ = ε p − ε p + 1 < 0 , λ 2 ⋆ = min { ε p + 2 − ε p + 1 , ε p − ε p − 1 } > 0 \lambda_{2}^{\star}=\min\{\varepsilon_{p+2}-\varepsilon_{p+1},\varepsilon_{p}-\varepsilon_{p-1}\}>0 λ 2 ⋆ = min { ε p + 2 − ε p + 1 , ε p − ε p − 1 } > 0 , and λ d ⋆ = ε n − ε 1 > 0 \lambda_{d}^{\star}=\varepsilon_{n}-\varepsilon_{1}>0 λ d ⋆ = ε n − ε 1 > 0 (with d = dim ( M ) = p ( n − p ) d=\dim(\mathcal{M})=p(n-p) d = dim ( M ) = p ( n − p ) ), and therefore,
λ min ⋆ = min { ε p − ε p − 1 , ε p + 1 − ε p , ε p + 2 − ε p + 1 } , λ max ⋆ = ε n − ε 1 , \lambda_{\min}^{\star}=\min\{\varepsilon_{p}-\varepsilon_{p-1},\varepsilon_{p+1}-\varepsilon_{p},\varepsilon_{p+2}-\varepsilon_{p+1}\},\quad\lambda_{\max}^{\star}=\varepsilon_{n}-\varepsilon_{1}, λ m i n ⋆ = min { ε p − ε p − 1 , ε p + 1 − ε p , ε p + 2 − ε p + 1 } , λ m a x ⋆ = ε n − ε 1 , which implies
κ P S P = ε n − ε 1 min { ε p − ε p − 1 , ε p + 1 − ε p , ε p + 2 − ε p + 1 } . \kappa_{P_{\mathrm{SP}}}=\frac{\varepsilon_{n}-\varepsilon_{1}}{\min\{\varepsilon_{p}-\varepsilon_{p-1},\varepsilon_{p+1}-\varepsilon_{p},\varepsilon_{p+2}-\varepsilon_{p+1}\}}. κ P SP = min { ε p − ε p − 1 , ε p + 1 − ε p , ε p + 2 − ε p + 1 } ε n − ε 1 . Comparing κ P S P \kappa_{P_{\mathrm{SP}}} κ P SP with κ P G M \kappa_{P_{\mathrm{GM}}} κ P GM , we observe that finding the index-1 constrained SPs can be worse conditioned. Moreover, it asks for not only a positive eigengap, but also nondegeneracy above ε p + 1 \varepsilon_{p+1} ε p + 1 and below ε p \varepsilon_{p} ε p . Another useful message is that, if the problem is reformulated on the Stiefel manifold St p ( R n ) \operatorname{St}_{p}(\mathbb{R}^{n}) St p ( R n ) and is treated by the saddle search algorithms in the existing works [91 , 94 ] [54, 56], we could anticipate their poor performance due to the inherent degeneracy. For illustrations, see § 4.1 .
Numerical experiments {#sec:4} In this part, we report numerical results on the linear eigenvalue problem and electronic excited-state calculations. Both tasks sit on the Grassmann manifold. In particular, with the former one, we elucidate the importance of using nonredundant parametrizations in finding SPs (cf. Example 2) and show the influence of problem and algorithm settings on the convergence rates (cf. Remark 5). We also conduct comparisons between algorithms implementing different retractions (see Remark 1) on the linear eigenvalue problem, and the results can be found in Appendix B. The code for producing all the numerical results is available at https://github.com/huyukuan/SadMan.
Linear eigenvalue problem
Given a real symmetric matrix A ∈ R s y m n × n A \in\mathbb{R}_{\mathrm{sym}}^{n \times n} A ∈ R sym n × n , the linear eigenvalue problem amounts to finding a critical point X ⋆ X^\star X ⋆ of the quadratic function f S t ( X ) : = 1 2 Tr ( X T A X ) f_{\mathrm{St}}(X) := \frac{1}{2}\operatorname{Tr}(X^{\mathsf{T}}AX) f St ( X ) := 2 1 Tr ( X T A X ) on the Stiefel manifold St p ( R n ) \operatorname{St}_{p}(\mathbb{R}^{n}) St p ( R n ) . It is well known that the columns of any such point span a p p p -dimensional invariant subspace of A A A . Moreover, the function value f S t ( X ⋆ ) f_{\mathrm{St}}(X^\star) f St ( X ⋆ ) is exactly half the sum of p p p eigenvalues of A A A . If all the sums ( n p ) \binom{n}{p} ( p n ) (in total) are different from each other, the indices of the saddle points can be indicated by their function values, in that f S t ( X S P , k ⋆ ) < f S t ( X S P , ℓ ⋆ ) f_{\mathrm{St}}(X^\star_{\mathrm{SP},k}) < f_{\mathrm{St}}(X^\star_{\mathrm{SP},\ell}) f St ( X SP , k ⋆ ) < f St ( X SP , ℓ ⋆ ) holds for the index-k k k and index-ℓ \ell ℓ constrained SPs whenever k < ℓ k < \ell k < ℓ (the reverse might not be true). Note that f S t f_{\mathrm{St}} f St is invariant under the transformation X ↦ X Q X \mapsto XQ X ↦ X Q for any Q ∈ O ( p ) Q \in O(p) Q ∈ O ( p ) . Therefore, one could instead consider a linear function f G r ( P ) : = 1 2 Tr ( P A ) f_{\mathrm{Gr}}(P) := \frac{1}{2}\operatorname{Tr}(PA) f Gr ( P ) := 2 1 Tr ( P A ) on the Grassmann manifold Gr p ( R n ) \operatorname{Gr}_{p}(\mathbb{R}^{n}) Gr p ( R n ) where any critical point P ⋆ P^\star P ⋆ is the orthogonal projector onto a p p p -dimensional invariant subspace of A A A . The statements regarding the function values and indices of constrained SPs remain valid similarly.
Implementation details. In what follows, we construct the test matrix A A A as A = U Σ U T A = U\Sigma U^{\mathsf{T}} A = U Σ U T , where U ∈ O ( n ) U \in O(n) U ∈ O ( n ) is obtained from the orthonormalization of a matrix whose entries are random numbers drawn independently and identically from the standard normal distribution (random seed = 0), and Σ = diag ( σ 1 , … , σ n ) ∈ R n × n \Sigma= \operatorname{diag}(\sigma_1,\ldots,\sigma_n) \in\mathbb{R}^{n \times n} Σ = diag ( σ 1 , … , σ n ) ∈ R n × n with σ i : = ξ i − n \sigma_i := \xi^{i-n} σ i := ξ i − n ( i = 1 , … , n ) (i = 1,\ldots,n) ( i = 1 , … , n ) for a parameter ξ > 1 \xi> 1 ξ > 1 . The problem data n n n , p p p , and ξ \xi ξ are specified later. We consider Algorithm 2 of Stiefel and Grassmann versions (M = St p ( R n ) \mathcal{M} = \operatorname{St}_{p}(\mathbb{R}^{n}) M = St p ( R n ) and M = Gr p ( R n ) \mathcal{M} = \operatorname{Gr}_{p}(\mathbb{R}^{n}) M = Gr p ( R n ) ). The retraction consists of a retraction on M \mathcal{M} M , direct addition and metric projection (see Lemma 5, Remark 1 for their definitions and Appendix § 8 for a numerical justification). For the retraction on M \mathcal{M} M , we adopt the QR decomposition-based second-order retraction on St p ( R n ) \operatorname{St}_{p}(\mathbb{R}^{n}) St p ( R n ) and the exponential mapping on Gr p ( R n ) \operatorname{Gr}_{p}(\mathbb{R}^{n}) Gr p ( R n ) , respectively. The step sizes will be specified later3 . The maximum iteration number and convergence tolerance are respectively set as m a x i t = ∞ maxit = \infty ma x i t = ∞ and t o l = 10 − 8 tol = 10^{-8} t o l = 1 0 − 8 . If not stated, the initial feasible points ( X ( 0 ) , V ( 0 ) ) (X^{(0)}, V^{(0)}) ( X ( 0 ) , V ( 0 ) ) and ( P ( 0 ) , Γ ( 0 ) ) (P^{(0)}, \Gamma^{(0)}) ( P ( 0 ) , Γ ( 0 ) ) are randomly generated as follows: with a given random seed,
from jax import random, numpy
key = random.PRNGKey(randseed) # random seed
key1, key2 = random.split(key)
X, _ = numpy.linalg.qr(random.normal(key1, (n, p)))
P = X @ X.T
Gamma = random.normal(key2, (n, n))
Gamma = (Gamma + Gamma.T) / 2.0
Gamma = proj_tangent(P, Gamma) # project onto the tangent space
V = Gamma @ X # horizontal lift
Importance of nonredundant parametrizations. Let n = 64 n = 64 n = 64 , p = 8 p = 8 p = 8 , and ξ = 1.01 \xi= 1.01 ξ = 1.01 . Following the above problem description, we run both the Stiefel and Grassmann versions of Algorithm 2 to find the index-1 constrained SPs of f S t f_{\mathrm{St}} f St and f G r f_{\mathrm{Gr}} f Gr , respectively. The initial feasible points are generated with random seed = 1. The step sizes are specified as η S t = η G r = 2 \eta_{\mathrm{St}} = \eta_{\mathrm{Gr}} = 2 η St = η Gr = 2 . The convergence curves are depicted in Figure 1 .
Figure 1. Convergence curves of Algorithm 2 of Stiefel (orange lines) and Grassmann (blue lines) versions on the linear eigenvalue problem (n = 64 n = 64 n = 64 , p = 8 p = 8 p = 8 , and ξ = 1.01 \xi= 1.01 ξ = 1.01 ). Left: position (solid lines) and direction (dashed lines) relative residuals vs iteration. Middle: function value vs iteration. Right: estimated lowest eigenvalue of Riemannian Hessian vs iteration.
We first observe from the left panel of Figure 1 the convergence of both versions of the algorithm. The correctness of the results can be checked by comparing the function values; see the middle panel of Figure 1 . Due to the construction of A A A , the function value at the index-1 constrained SP is exactly 1 2 ( ∑ i = 1 p σ i + σ p + 1 ) ≈ 2.216 \frac{1}{2}(\sum_{i=1}^{p}\sigma_i + \sigma_{p+1}) \approx2.216 2 1 ( ∑ i = 1 p σ i + σ p + 1 ) ≈ 2.216 . Initialized from the same point (more precisely, the Stiefel version is fed with the representative of the initial point for the Grassmann version), the Grassmann version converges well to the desired constrained SP, while the Stiefel one gets trapped to the global minimizer, whose function value is 1 2 ∑ i = 1 p σ i ≈ 2.213 \frac{1}{2}\sum_{i=1}^{p}\sigma_i \approx2.213 2 1 ∑ i = 1 p σ i ≈ 2.213 . This comparison can also be seen in the right panel of Figure 1 , where the Riemannian Hessian at the point given by the Stiefel version is found to be positive semidefinite.
We remark that the failure of the Stiefel version should be ascribed to the parametrization redundancy. The formulation on the Stiefel manifold does not take into account the quotient structure, as explained in the first paragraph of this subsection. The redundancy brings degeneracies above and/or below the eigengap and leads to an infinite condition number at the desired index-1 constrained SP (cf. Example 2). The deficiency cannot be neglected as in the task of finding the global minimizer because the update direction for the position is not a horizontal lift of that in the Grassmann version. Roughly, suppose that the initial point lies in a region where the Riemannian Hessian is positive semidefinite (say, around the global minimizer) and the direction step converges quickly to the lowest eigenvector of the Riemannian Hessian. Due to the parametrization redundancy, the lowest eigenvalue of the Riemannian Hessian is zero and the direction variable V ( t ) V(t) V ( t ) is almost vertical. This implies that climbing up along V ( t ) V(t) V ( t ) makes little difference in the first order and the position variable mainly follows the gradient flow down to the minimizer.
The above arguments are supported with numerical results. We run the Grassmann and Stiefel algorithms with the same problem and algorithm settings as before and randomly sampled initial points perturbed from the global minimizer and index-1 constrained SP. Specifically, we consider the perturbation level β ∈ { 10 − 3 , 10 − 2 , … , 10 1 } \beta\in\{10^{-3}, 10^{-2}, \ldots, 10^{1}\} β ∈ { 1 0 − 3 , 1 0 − 2 , … , 1 0 1 } and each of them is tested with 100 independent random samples (random seed = 0 ∼ 99):
key = random.PRNGKey(randseed) # random seed
key1, key2 = random.split(key)
X, _ = numpy.linalg.qr(X_ref + beta * random.normal(key1, (n, p)))
# repeat previous procedures to create P, Gamma, and V
Here X r e f X_{\mathrm{ref}} X ref is either the global minimizer or index-1 constrained SP. The empirical success rates of the two algorithms at different perturbation levels are recorded in Table 1 . It turns out that the Grassmann version converges surprisingly well across the board to the index-1 constrained SP, whereas the Stiefel one often gets attracted by the global minimizer, with an empirical success rate of approximately 30% if randomly initialized. The rate can be tremendously improved if the initial point is selected around the desired index-1 constrained SP. We only use the Grassmann version for the subsequent experiments.
Table 1. Empirical success rates in finding the index-1 constrained SP of Algorithm 2 of Grassmann and Stiefel versions starting from the neighborhoods of the global minimizer and index-1 constrained SP of the linear eigenvalue problem (n = 64 n = 64 n = 64 , p = 8 p = 8 p = 8 , and ξ = 1.01 \xi= 1.01 ξ = 1.01 ).
Influence of problem and algorithm settings on the convergence rates. We investigate the performance of Algorithm 2 in finding the index-1 constrained SP with different problem data (n n n , p p p , and ξ \xi ξ ) and algorithm settings (initialization and step sizes). First consider varying the problem data in n ∈ { 10 , 20 , … , 80 } n \in\{10, 20, \ldots, 80\} n ∈ { 10 , 20 , … , 80 } , p ∈ { 2 , 3 , … , 8 } p \in\{2, 3, \ldots, 8\} p ∈ { 2 , 3 , … , 8 } , ξ ∈ { 1.0001 , 1.001 , 1.01 , 1.05 } \xi\in\{1.0001, 1.001, 1.01, 1.05\} ξ ∈ { 1.0001 , 1.001 , 1.01 , 1.05 } . The step sizes are determined by the estimates (46) and (47). For each problem setting, we perform ten independent runs of the algorithm with the initial points randomly selected at the perturbation level of 10 − 3 10^{-3} 1 0 − 3 from the index-1 constrained SP (random seed = 0 ∼ \sim ∼ 9). The average iteration numbers and the condition numbers estimated through Eq. (48) , κ = ξ n − 1 − 1 ξ p − 2 ( ξ − 1 ) \kappa= \frac{\xi^{n-1}-1}{\xi^{p-2}(\xi-1)} κ = ξ p − 2 ( ξ − 1 ) ξ n − 1 − 1 , are shown with heatmaps in Figure 2 . The trends of the iteration numbers are found to align qualitatively well with those of the estimated condition numbers, showcasing the validity of our theoretical analysis.
Figure 2. Average iteration number and estimated condition number vs ( n , p ) (n,p) ( n , p ) on the linear eigenvalue problem class. Upper left two panels: ξ = 1.0001 \xi= 1.0001 ξ = 1.0001 . Upper right two panels: ξ = 1.001 \xi= 1.001 ξ = 1.001 . Lower left two panels: ξ = 1.01 \xi= 1.01 ξ = 1.01 . Lower right two panels: ξ = 1.05 \xi= 1.05 ξ = 1.05 .
Next we consider varying the algorithm settings. We fix the problem data to be n = 10 n = 10 n = 10 , p = 2 p = 2 p = 2 , and ξ = 1.01 \xi= 1.01 ξ = 1.01 . Since the best step sizes are estimated by Eqs. (46) and (47) to be ( η P ⋆ , η Γ ⋆ ) ≈ ( 21.096 , 17.656 ) (\eta_P^\star,\eta_\Gamma^\star) \approx(21.096, 17.656) ( η P ⋆ , η Γ ⋆ ) ≈ ( 21.096 , 17.656 ) for this instance, the step sizes η P \eta_P η P and η Γ \eta_\Gamma η Γ are varied respectively within η P ∈ { 10 , 10.2 , 10.4 , … , 21.8 , 22 } \eta_P \in\{10, 10.2, 10.4, \ldots, 21.8, 22\} η P ∈ { 10 , 10.2 , 10.4 , … , 21.8 , 22 } and η Γ ∈ { 10 , 10.2 , 10.4 , … , 19.8 , 20 } \eta_\Gamma\in\{10, 10.2, 10.4, \ldots, 19.8, 20\} η Γ ∈ { 10 , 10.2 , 10.4 , … , 19.8 , 20 } .4 For each pair of ( η P , η Γ ) (\eta_P,\eta_\Gamma) ( η P , η Γ ) , we perform ten independent runs of the algorithm with the initial points randomly selected at the perturbation level of 10 − 3 10^{-3} 1 0 − 3 or 10 − 1 10^{-1} 1 0 − 1 from the index-1 constrained SP (random seed = 0 ∼ \sim ∼ 9). Similarly, we visualize the results with heatmaps in Figure 3 . For this test instance, the least iteration numbers are achieved when the pair ( η P , η Γ ) = ( 21 , 19.2 ) (\eta_P,\eta_\Gamma)=(21,19.2) ( η P , η Γ ) = ( 21 , 19.2 ) given β = 0.001 \beta=0.001 β = 0.001 and ( η P , η Γ ) = ( 21 , 19 ) (\eta_P,\eta_\Gamma)=(21,19) ( η P , η Γ ) = ( 21 , 19 ) given β = 0.1 \beta=0.1 β = 0.1 . We also find that the step size η P \eta_P η P for the position variable is far more pivotal than η Γ \eta_\Gamma η Γ for the direction variable. Incidentally, the empirically best pairs of step sizes are close to the estimated pair ( η P ⋆ , η Γ ⋆ ) ≈ ( 21.096 , 17.656 ) (\eta_P^\star,\eta_\Gamma^\star)\approx(21.096,17.656) ( η P ⋆ , η Γ ⋆ ) ≈ ( 21.096 , 17.656 ) . The modest shift in the direction step size is consistent with the residual reduction inequality (38) being an upper bound together with the influence of other coupling and higher-order terms. Performance at the estimated pair is already near-optimal over the tested grid.
Figure 3. Average iteration number vs ( η P , η Γ ) (\eta_P,\eta_\Gamma) ( η P , η Γ ) on the linear eigenvalue problem (n = 10 n=10 n = 10 , p = 2 p=2 p = 2 , and ξ = 1.01 \xi=1.01 ξ = 1.01 ). The theoretical and empirical best pairs of step sizes are marked out by red stars and white crosses, respectively. Left: perturbation level of 10 − 3 10^{-3} 1 0 − 3 . Right: perturbation level of 10 − 1 10^{-1} 1 0 − 1 .
Computation of higher-index constrained SPs. We proceed to compute the constrained SPs of all indices for an instance. We fix the problem data to be n = 10 n=10 n = 10 , p = 2 p=2 p = 2 , and ξ = 1.01 \xi=1.01 ξ = 1.01 and the step sizes to be η P = η Γ = 12 \eta_P=\eta_\Gamma=12 η P = η Γ = 12 . The possible index of the constrained SP on this instance is at most 15. For each index in { 0 , … , 15 } \{0,\ldots,15\} { 0 , … , 15 } , we perform 400 runs of Algorithm 2 from randomly generated initial points (random seed = 0 ∼ \sim ∼ 399). The indices and the function values at the obtained constrained SPs as well as the required iterations on average are listed in Table 2 . The configurations of eigenvalues of A A A corresponding to the function values are also included. We observe that every run converged to a valid critical point, and every admissible configuration has been observed at least once (cf. Example 2), although the sampling frequencies are highly nonuniform. Note that the “index-0” constrained SP for this example is exactly the global minimizer.
Table 2: Indices, function values, the corresponding configurations of eigenvalues of A A A , and the needed iterations on average for finding constrained SPs on the linear eigenvalue problem (n = 10 n = 10 n = 10 , p = 2 p = 2 p = 2 , and ξ = 1.01 \xi= 1.01 ξ = 1.01 ). The configurations are indicated by doublets; e.g., ( 1 , 3 ) (1, 3) ( 1 , 3 ) means that the function value equals 1 2 ( σ 1 + σ 3 ) \frac{1}{2}(\sigma_1 + \sigma_3) 2 1 ( σ 1 + σ 3 ) . For each configuration, we also provide its occurrence in 400 random trials.
Table 2.
Electronic excited-state calculations
The core of electronic calculations for molecular systems is the electronic Schrödinger equation (ESE) [79], which is in fact a linear eigenvalue problem. Nevertheless, the ESE is intractable in general due to the curse of dimensionality. For numerical purposes, various approximations have been proposed in some atomic-orbital basis, such as the full configuration interaction (FCI), Hartree-Fock (HF) methods, and post-HF methods [43], among others. Excited states define the optical and reaction properties of atoms and molecules [6 ] [63, 87]. Characterizing the excited states is challenging due to electron correlation effects [34]. Finding the constrained SPs the quantum chemical approximated methods in use arises as a natural methodology [16–18, 53, 65, 76]. Moreover, these approximations usually come together with manifold structures. In the following, we briefly introduce the HF methods and report the numerical results of finding constrained SPs as candidates of excited states, with FCI calculations (performed by PySCF [84]) as reference. More advanced levels of theory, such as the complete active space self-consistent field method [75], involve complicated manifolds and ad-hoc treatments, and are investigated in a parallel work; see a recent preprint [37] by the authors and other co-workers. Before proceeding, we remark that our methodology falls into the class of state-specific methods in quantum chemistry for electronic excited states; other popular ones include linear response theory [19, 21, 36, 70] and state-average methods for multiconfigurational approximations [89].
The HF methods. The HF approximation restricts the electronic wavefunction to a single Slater determinant parametrized by orthonormal molecular orbitals [31,39]. In this work, we consider the restricted HF (in short, RHF) method. The RHF method assumes all molecular orbitals to be doubly occupied, by one spin-up and one spin-down electron. As a result, the spatial orbitals are expressed as the linear combinations of the chosen atomic-orbitals, with coefficients to be determined. The atomic-orbitals are assumed to be real hereafter.
For a closed-shell system with N e l e c ∈ N N_{\mathrm{elec}} \in\mathbb{N} N elec ∈ N electrons, the RHF approximation gives rise to the following energy functional on the Grassmann manifold:
E R H F ( γ ) : = 2 Tr ( h γ ) + Tr ( ( 2 J ( γ ) − K ( γ ) ) γ ) + E n u c with γ ∈ Gr N o ( R N b ) , E^{\mathrm{RHF}}(\gamma) := 2\operatorname{Tr}(h\gamma) + \operatorname{Tr}((2J(\gamma)-K(\gamma))\gamma)+E_{\mathrm{nuc}} \quad\text{with} \quad\gamma\in\operatorname{Gr}_{N_{\mathrm{o}}}(\mathbb{R}^{N_{\mathrm{b}}}), E RHF ( γ ) := 2 Tr ( hγ ) + Tr (( 2 J ( γ ) − K ( γ )) γ ) + E nuc with γ ∈ Gr N o ( R N b ) ,
where N b ∈ N N_{\mathrm{b}} \in\mathbb{N} N b ∈ N is the size of real atomic-orbital basis { ϕ i } i = 1 N b \{\phi_i\}_{i=1}^{N_{\mathrm{b}}} { ϕ i } i = 1 N b , N o ∈ N N_{\mathrm{o}} \in\mathbb{N} N o ∈ N the number of occupied molecular orbitals (2 N o = N e l e c 2N_{\mathrm{o}}=N_{\mathrm{elec}} 2 N o = N elec ), h ∈ R s y m N b × N b h \in\mathbb{R}_{\mathrm{sym}}^{N_{\mathrm{b}}\times N_{\mathrm{b}}} h ∈ R sym N b × N b the discretized one-body Hamiltonian, E n u c ∈ R E_{\mathrm{nuc}} \in\mathbb{R} E nuc ∈ R the nuclear-repulsion energy, and γ \gamma γ the discretized one-body reduced density matrix. Here, J , K : R s y m N b × N b → R s y m N b × N b J,K:\mathbb{R}_{\mathrm{sym}}^{N_{\mathrm{b}}\times N_{\mathrm{b}}}\to\mathbb{R}_{\mathrm{sym}}^{N_{\mathrm{b}}\times N_{\mathrm{b}}} J , K : R sym N b × N b → R sym N b × N b are respectively the Coulomb and exchange functionals, defined as
[ J ( γ ) ] p q : = ∑ r , s = 1 N b g p q r s γ s r , [ K ( γ ) ] p q : = ∑ r , s = 1 N b g p s r q γ s r , p , q = 1 , … , N b , [J(\gamma)]_{pq}:=\sum_{r,s=1}^{N_{\mathrm{b}}}g_{pqrs}\gamma_{sr},\qquad[K(\gamma)]_{pq}:=\sum_{r,s=1}^{N_{\mathrm{b}}}g_{psrq}\gamma_{sr},\qquad p,q=1,\ldots,N_{\mathrm{b}}, [ J ( γ ) ] pq := r , s = 1 ∑ N b g pq r s γ sr , [ K ( γ ) ] pq := r , s = 1 ∑ N b g p sr q γ sr , p , q = 1 , … , N b ,
with
g p q r s : = ∫ R 3 ∫ R 3 ϕ p ( r ) ϕ q ( r ) ϕ r ( r ′ ) ϕ s ( r ′ ) ∣ r − r ′ ∣ d r d r ′ , p , q , r , s = 1 , … , N b g_{pqrs}:=\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{3}}\frac{\phi_p(\boldsymbol{r})\phi_q(\boldsymbol{r})\phi_r(\boldsymbol{r}')\phi_s(\boldsymbol{r}')}{\lvert\boldsymbol{r}-\boldsymbol{r}'\rvert}\,\mathrm{d}\boldsymbol{r}\,\mathrm{d}\boldsymbol{r}',\qquad p,q,r,s=1,\ldots,N_{\mathrm{b}} g pq r s := ∫ R 3 ∫ R 3 ∣ r − r ′ ∣ ϕ p ( r ) ϕ q ( r ) ϕ r ( r ′ ) ϕ s ( r ′ ) d r d r ′ , p , q , r , s = 1 , … , N b
the two-body integrals.
Implementation details. We consider the H2 _2 2 and LiH molecules. The bond length of the H2 _2 2 molecule is varied from 0.1 bohr5 to 4.0 bohr with a spacing of 0.1 bohr, while that of the LiH molecule is from 2.0 bohr to 6.0 bohr with a spacing of 0.2 bohr. The H2 _2 2 and LiH molecules are described by RHF with the 6-31G basis set (N e l e c = 2 N_{\mathrm{elec}}=2 N elec = 2 , N b = 4 N_{\mathrm{b}}=4 N b = 4 , N o = 1 N_{\mathrm{o}}=1 N o = 1 ) [26, 40] and the STO-3G basis set (N e l e c = 4 N_{\mathrm{elec}}=4 N elec = 4 , N b = 6 N_{\mathrm{b}}=6 N b = 6 , N o = 2 N_{\mathrm{o}}=2 N o = 2 ) [41], respectively. The numbers of total degrees of freedom (DOFs) are thus N o ( N b − N o ) = 3 N_{\mathrm{o}}(N_{\mathrm{b}}-N_{\mathrm{o}})=3 N o ( N b − N o ) = 3 for the H2 _2 2 molecule and 8 for the LiH molecule. We investigate the energy landscape of RHF for both molecules with the varying bond lengths by searching for the constrained SPs of allowed indices. They are found by running Algorithm 2 from 1,000 random initial points (random seed = 0 ∼ \sim ∼ 999); see the pseudocodes in Section 4.1 for the initialization. Regarding retraction, Algorithm 2 is equipped with the exponential mapping for the position part; the treatment for the direction part is similar to that in the previous subsection. The step size is specified as η = 10 − 1 \eta=10^{-1} η = 1 0 − 1 and η = 10 − 2 \eta=10^{-2} η = 1 0 − 2 for the H2 _2 2 and LiH molecules, respectively, which are not necessarily optimal. The maximum iteration number and convergence tolerance are respectively set as m a x i t = 20000 maxit=20000 ma x i t = 20000 and t o l = 10 − 6 tol=10^{-6} t o l = 1 0 − 6 . The results are compared with those obtained by solving FCI based on the RHF ground-state calculations, which is exact under the basis in use. In particular, FCI gives 16 states for the H2 _2 2 molecule and 225 states for the LiH molecule.
Results on the H2 _2 2 molecule. An overview of the constrained SPs of RHF identified across the considered bond length interval is shown in Figure 4 , together with the ground-/excited-state energies of FCI as reference. It can be seen that the RHF energy landscape varies smoothly with the bond length. Note that FCI states are classified by their irreducible representations (irreps); for a complete reference to point group symmetry theory, please refer to [66]. For the H2 _2 2 molecule in the computational point group D2 h _{2h} 2 h , the relevant irreps are Ag _g g and Au _u u , with eight FCI states belonging to each. It is observed that the RHF SPs are only able to describe a small subset (e.g., four near the equilibrium geometry, three close to the dissociation limit) of FCI states in terms of their energies. This behavior is consistent with the fact that RHF neglects electronic correlation [9 ] . The deficiency can be mitigated to some extent by resorting to post-HF methods, which is investigated in a parallel work.
Figure 4. An overview of the constrained SPs of RHF found across the bond length interval on the H2 _2 2 molecule. The blue solid and dashed lines represent the energies of FCI states belonging to the irreps Ag _g g and Au _u u , respectively. The red dots, deepblue squares, yellow stars, and purple diamonds stand for the energies of the RHF SPs of indices 0, 1, 2, and 3, respectively. The zoom-in inset describes the FCI states by atomic fragments when the molecule is close to its dissociation limit.
Our results also reveal numerically that the varying bond length, as an external parameter, can lead to disappearance or emergence of constrained SPs. Zoom-in views are given in Figure
Concretely, in the interval (0.5 bohr, 0.6 bohr), an index-1 SP gets close to the index-2 SP, in terms of energy, and disappears; in the interval (1.6 bohr, 1.7 bohr), an index-2 SP close to the index-3 SP emerges; and in the interval (2.6 bohr, 2.7 bohr), an index-0 SP (i.e., a local minimizer) close to the index-1 SP emerges.
Results on the LiH molecule. The identified constrained SPs of RHF are shown in a similar way in Figure 6 , again with the FCI-state energies as reference. For the LiH molecule in the computational group C2 v _{2v} 2 v , the relevant irreps are A1 _1 1 , A2 _2 2 , B1 _1 1 , and B2 _2 2 . Note that the B1 _1 1 and B2 _2 2 FCI states are two-fold degenerate across the considered bond length interval. In this case, the RHF SPs appear to well describe the low-lying FCI states with energies lower than − 6 -6 − 6 hartree6 But the FCI states with energies in (− 6 -6 − 6 hartree, − 4 -4 − 4 hartree) are completely missed by the RHF SPs, probably due to inherent shortcomings of the RHF approximation. In addition, the potential energy surfaces of higher-index RHF SPs are found to be more complicated than those of low-index SPs, especially when the molecule is driven to the dissociation limit.
Figure 6. An overview of the constrained SPs of RHF found across the bond length interval on the LiH molecule. The blue solid, dotted, and dashed lines represent the energies of FCI states belonging to the irreps A 1 A_1 A 1 , A 2 A_2 A 2 , B 1 B_1 B 1 (or B 2 B_2 B 2 ), respectively. The red dots, deepblue squares, yellow stars, purple diamonds, down-pointing cyan triangles, up-pointing orange triangles, left-pointing green triangles, right-pointing pink triangles, and navyblue pentagons stand for the energies of the RHF SPs of indices 0 to 8, respectively.
The above results provide proof-of-concept evidence for the effectiveness of our algorithms in excited-state calculations. Nonetheless, we shall point out that a comprehensive quantum chemical analysis of the obtained SPs, though beyond the scope of present work, is essential for practical applications. As a nonlinear approximation to the exact theory, RHF may produce critical points that lack physical meanings; e.g., a spurious non-global local minimizer emerges when the bond length exceeds 2.7 bohr in the right panel of Figure 5 . Moreover, since RHF is a low-dimensional approximation, a one-to-one correspondence between RHF SPs and excited states in the same energetic order no longer holds. Molecular geometries must also be taken into account. In the exact theory, the ESE is parameterized by atomic configurations. Variations in molecular geometry may induce degeneracies and crossings in states, which would lead to unfavorable features of potential energy surfaces within approximate theories (cf. Figure 5 ). From the computational aspect, it is of great importance to develop schemes capable of efficiently navigating the nonconvex landscape [90 ] , instead of relying on random multi-start.
Figure 5. Zoom-in views of the constrained SPs of RHF. Upper left: an RHF index-1 SP disappears in (0.5 bohr, 0.6 bohr). Upper right: an RHF index-2 SP emerges in (1.6 bohr, 1.7 bohr). Lower: an RHF index-0 SP emerges in (2.6 bohr, 2.7 bohr).
Conclusions
We have developed a constrained saddle dynamics for finding SPs on any embedded Riemannian submanifold of Euclidean space. The dynamics is formulated compactly on the Grassmann bundle of the tangent bundle, and achieves broader applicability than those in the literature exploiting explicit global/local level-set manifold representations. By investigating the Grassmann bundle geometry, we have rigorously established the theoretical properties of both the dynamics and the resulting discretized algorithms. Remarkably, our analysis provides the first linear iterate convergence results of the discretized algorithms in embedded-submanifold settings. Moreover, compared with existing results, we eliminate unnecessary nondegeneracy assumptions on the eigenvalues of the Riemannian Hessian by adopting a single orthogonal projector as the direction variable, thereby respecting the underlying quotient structure. We have also characterized how the spectrum of the Riemannian Hessian affects the local convergence rates and highlighted the importance of using nonredundant parametrizations. Both of these two points have been validated with numerical results on linear eigenvalue problems. Finally, we have applied the proposed algorithms to electronic excited-state calculations.
There remains lots of directions to be explored. The constants showing up in the assumptions of convergence results are still not computable, which brings difficulties to choosing step sizes. An adaptively tuned yet theoretically guaranteed step size strategy is much more preferable. Moreover, the numerical performance of the discretized algorithms is highly sensitive to condition numbers, as evidenced by their local convergence rates. It is thus desirable to incorporate higher-order contributions without sacrificing local convergence properties. In addition, a globally convergent method for locating SPs on Riemannian manifolds is still lacking, due to the absence of a global merit function. One possible avenue is to extend the analysis in [51] and develop stochastic methods on manifolds. An intrinsic extension of the formulation in this work to abstract Riemannian manifolds should be possible, but would require replacing the ambient constructions by intrinsic bundle-geometric counterparts, and is left for future work. From the perspective of quantum chemistry, it would also be valuable to investigate the manifold geometry underlying more complicated levels of theory and to devise efficient yet physically meaningful strategies for navigating the associated nonconvex energy landscapes.
Acknowledgements
This work has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement EMC2 No 810367). The authors are grateful to Éric Cancès, Tony Lelièvre, and Panos Parpas for useful discussions.
Fundamental concepts of embedded Riemannian submanifolds of Euclidean space
We recall briefly some fundamental concepts of embedded Riemannian submanifolds of Euclidean space. For interested readers, we refer to the monographs [1 , 15 ] [49, 50]. In the following, let M \mathcal{M} M be embedded in a Euclidean space E \mathcal{E} E with dim ( M ) = d \dim(\mathcal{M}) = d dim ( M ) = d .
Tangent space and tangent bundle. For each x ∈ M x \in\mathcal{M} x ∈ M , the tangent space to M \mathcal{M} M at x x x is referred to as T x M T_x\mathcal{M} T x M , which is defined as
T x M : = { c ′ ( 0 ) ∣ c : R ⊃ I → M smooth , c ( 0 ) = x } . T_x\mathcal{M} := \left\{c'(0) \mid c : \mathbb{R} \supset I \to\mathcal{M}\ \text{smooth},\ c(0) = x\right\}. T x M := { c ′ ( 0 ) ∣ c : R ⊃ I → M smooth , c ( 0 ) = x } .
The vectors in T x M T_x\mathcal{M} T x M are called tangent vectors to M \mathcal{M} M at x x x . The tangent space T x M T_x\mathcal{M} T x M is endowed with the Riemannian metric induced from the inner product of the ambient Euclidean space. For any x ∈ M x \in\mathcal{M} x ∈ M , the orthogonal projection operator Proj T x M \operatorname{Proj}_{T_x\mathcal{M}} Proj T x M from the ambient space onto the tangent space T x M T_x\mathcal{M} T x M is defined as
Proj T x M ( v ) : = arg min u ∥ u − v ∥ , ∀ v ∈ E , \operatorname{Proj}_{T_x\mathcal{M}}(v) := \underset{u}{\arg\min}\ \lVert u-v\rVert,\quad\forall\ v \in\mathcal{E}, Proj T x M ( v ) := u arg min ∥ u − v ∥ , ∀ v ∈ E ,
where ∥ ∙ ∥ : E → R + \lVert\bullet\rVert: \mathcal{E} \to\mathbb{R}_{+} ∥ ∙ ∥ : E → R + refers to the norm induced by the inner product.
The tangent bundle of M \mathcal{M} M is denoted by T M : = { ( x , v ) ∣ x ∈ M , v ∈ T x M } T\mathcal{M} := \{(x,v) \mid x \in\mathcal{M},\ v \in T_x\mathcal{M}\} T M := {( x , v ) ∣ x ∈ M , v ∈ T x M } , i.e., the disjoint union of the tangent spaces to M \mathcal{M} M . From [15 Theorem 3.43] , T M T\mathcal{M} T M is a 2 d 2d 2 d -dimensional submanifold embedded in E × E \mathcal{E} \times\mathcal{E} E × E .
Riemannian gradient and Hessian. For a smooth function f f f , its Riemannian gradient at x ∈ M x \in\mathcal{M} x ∈ M , denoted by grad M f ( x ) \operatorname{grad}_{\mathcal{M}} f(x) grad M f ( x ) , is defined as the unique element of T x M T_x\mathcal{M} T x M satisfying
⟨ grad M f ( x ) , v ⟩ = D f ( x ) [ v ] , ∀ v ∈ T x M , \langle\operatorname{grad}_{\mathcal{M}} f(x),v\rangle= \mathrm{D}f(x)[v],\quad\forall\ v \in T_x\mathcal{M}, ⟨ grad M f ( x ) , v ⟩ = D f ( x ) [ v ] , ∀ v ∈ T x M ,
where D f ( x ) [ v ] \mathrm{D}f(x)[v] D f ( x ) [ v ] stands for the directional derivative of f f f at x x x along the tangent vector v v v . Since M \mathcal{M} M is a Riemannian submanifold embedded in a Euclidean space, grad M f ( x ) \operatorname{grad}_{\mathcal{M}} f(x) grad M f ( x ) can be readily computed via
grad M f ( x ) = Proj T x M ( grad f ( x ) ) . \operatorname{grad}_{\mathcal{M}} f(x) = \operatorname{Proj}_{T_x\mathcal{M}}(\operatorname{grad} f(x)). grad M f ( x ) = Proj T x M ( grad f ( x )) .
The definition of Riemannian Hessian in the general cases necessitates the concept of Riemannian connection [15 Section 5.4] . Again due to the fact that M \mathcal{M} M is an embedded Riemannian submanifold in E \mathcal{E} E , we recall for simplicity the following characterization:
Hess M f ( x ) [ v ] = Proj T x M ( D G ‾ ( x ) [ v ] ) , (49) \operatorname{Hess}_{\mathcal{M}} f(x)[v] = \operatorname{Proj}_{T_x\mathcal{M}}\left(\mathrm{D}\overline{G}(x)[v]\right), \tag*{(49)} Hess M f ( x ) [ v ] = Proj T x M ( D G ( x ) [ v ] ) , (49)
where G ‾ \overline{G} G is any smooth extension of grad M f \operatorname{grad}_{\mathcal{M}} f grad M f to a neighborhood of M \mathcal{M} M in E \mathcal{E} E .
Retraction. A retraction on M \mathcal{M} M is a smooth mapping R : T M → M \mathrm{R} : T\mathcal{M} \to\mathcal{M} R : T M → M , T M ∋ ( x , v ) ↦ R x ( v ) ∈ M T\mathcal{M} \ni(x,v) \mapsto\mathrm{R}_x(v) \in\mathcal{M} T M ∋ ( x , v ) ↦ R x ( v ) ∈ M , satisfying R x ( 0 ) = x \mathrm{R}_x(0) = x R x ( 0 ) = x and that D R x ( 0 ) \mathrm{D}\mathrm{R}_x(0) D R x ( 0 ) is the identity mapping on T x M T_x\mathcal{M} T x M for any x ∈ M x \in\mathcal{M} x ∈ M .
By leveraging the retraction, we can obtain a point by moving away from x ∈ M x \in\mathcal{M} x ∈ M along some v ∈ T x M v \in T_x\mathcal{M} v ∈ T x M , while remaining on M \mathcal{M} M . It follows that retractions define update rules that preserve feasibility. One typical example of retraction is the exponential mapping, denoted specially by Exp : T M → M \operatorname{Exp}:T\mathcal{M}\to\mathcal{M} Exp : T M → M , which is determined by a set of second-order ordinary differential equations and yields geodesics on M \mathcal{M} M . If M \mathcal{M} M is complete, then it holds that [15 Proposition 10.22]
dist M ( x , Exp x ( v ) ) = ∥ v ∥ , \operatorname{dist}_{\mathcal{M}}(x,\operatorname{Exp}_x(v))=\lVert v\rVert, dist M ( x , Exp x ( v )) = ∥ v ∥ ,
for any x ∈ M x \in\mathcal{M} x ∈ M and v v v in a neighborhood of the origin of T x M T_x\mathcal{M} T x M , where dist M ( ∙ , ∙ ) : M × M → R + \operatorname{dist}_{\mathcal{M}}(\bullet,\bullet):\mathcal{M}\times\mathcal{M}\to\mathbb{R}_{+} dist M ( ∙ , ∙ ) : M × M → R + represents the Riemannian distance on M \mathcal{M} M :
dist M ( x , y ) : = inf c c piecewise smooth on M c ( a ) = x , c ( b ) = y { ∫ a b ∥ c ′ ( t ) ∥ d t } . \operatorname{dist}_{\mathcal{M}}(x,y):=\inf_{\substack{c\\ c\text{ piecewise smooth on }\mathcal{M}\\ c(a)=x,\ c(b)=y}}\left\{\int_a^b \lVert c'(t)\rVert\,\mathrm{d}t\right\}. dist M ( x , y ) := c c piecewise smooth on M c ( a ) = x , c ( b ) = y inf { ∫ a b ∥ c ′ ( t )∥ d t } .
Parallel transport and vector transport. Let c : [ 0 , 1 ] → M c:[0,1]\to\mathcal{M} c : [ 0 , 1 ] → M be a smooth curve. For any v ∈ T c ( 0 ) M v\in T_{c(0)}\mathcal{M} v ∈ T c ( 0 ) M , there exists a unique vector field W ( t ) ∈ T c ( t ) M W(t)\in T_{c(t)}\mathcal{M} W ( t ) ∈ T c ( t ) M along c c c satisfying
∇ c ˙ ( t ) W ( t ) = 0 , W ( 0 ) = v . \nabla_{\dot{c}(t)}W(t)=0,\qquad W(0)=v. ∇ c ˙ ( t ) W ( t ) = 0 , W ( 0 ) = v .
The parallel transport of v v v along c c c from c ( 0 ) c(0) c ( 0 ) to c ( t ) c(t) c ( t ) is defined by
PT c ← ( 0 ) ( v ) : = W ( t ) . \operatorname{PT}_{c\leftarrow(0)}(v):=W(t). PT c ← ( 0 ) ( v ) := W ( t ) .
Parallel transport is a linear isometry between the corresponding tangent spaces. Since M \mathcal{M} M is embedded in a Euclidean space, the ambient derivative of a vector field W W W along c c c satisfies the Gauss formula
d W d t = ∇ c ˙ W + I I c ( t ) ( c ˙ , W ) . \frac{\mathrm{d}W}{\mathrm{d}t}=\nabla_{\dot{c}}W+\mathrm{II}_{c(t)}(\dot{c},W). d t d W = ∇ c ˙ W + II c ( t ) ( c ˙ , W ) .
Consequently, if W W W is parallel along c c c , then
d W d t = I I c ( t ) ( c ˙ , W ) . \frac{\mathrm{d}W}{\mathrm{d}t}=\mathrm{II}_{c(t)}(\dot{c},W). d t d W = II c ( t ) ( c ˙ , W ) .
A vector transport associated with a retraction R \mathrm{R} R on M \mathcal{M} M is a smooth mapping that assigns, to x ∈ M x\in\mathcal{M} x ∈ M , u , v ∈ T x M u,v\in T_x\mathcal{M} u , v ∈ T x M , a vector
VT ( x , u ) ( v ) ∈ T R x ( u ) M . \operatorname{VT}_{(x,u)}(v)\in T_{\mathrm{R}_x(u)}\mathcal{M}. VT ( x , u ) ( v ) ∈ T R x ( u ) M .
It is required to be linear in v v v and to satisfy
VT ( x , 0 ) ( v ) = v . \operatorname{VT}_{(x,0)}(v)=v. VT ( x , 0 ) ( v ) = v .
Unlike parallel transport, a general vector transport need not be isometric or globally invertible. A common choice is the metric-projection transport
VT ( x , u ) ( v ) : = Proj T R x ( u ) M ( v ) . \operatorname{VT}_{(x,u)}(v):=\operatorname{Proj}_{T_{\mathrm{R}_x(u)}\mathcal{M}}(v). VT ( x , u ) ( v ) := Proj T R x ( u ) M ( v ) .
Second fundamental form. For a given x ∈ M x\in\mathcal{M} x ∈ M , the normal space N x M N_x\mathcal{M} N x M is the orthogonal complement of T x M T_x\mathcal{M} T x M in E \mathcal{E} E . The second fundamental form at x x x can be identified as the mapping I I x : T x M × T x M → N x M \mathrm{II}_x:T_x\mathcal{M}\times T_x\mathcal{M}\to N_x\mathcal{M} II x : T x M × T x M → N x M defined as I I x ( u , v ) : = P x , u ( v ) \mathrm{II}_x(u,v):=P_{x,u}(v) II x ( u , v ) := P x , u ( v ) for any u , v ∈ T x M u,v\in T_x\mathcal{M} u , v ∈ T x M , where
P x , u : = D ( y ↦ Proj T y M ) ( x ) [ u ] , (50) P_{x,u}:=D\bigl(y\mapsto\operatorname{Proj}_{T_y\mathcal{M}}\bigr)(x)[u], \tag*{(50)} P x , u := D ( y ↦ Proj T y M ) ( x ) [ u ] , (50)
namely, the directional derivative of y ↦ Proj T y M y\mapsto\operatorname{Proj}_{T_y\mathcal{M}} y ↦ Proj T y M at x x x along u u u . That the range of I I x \mathrm{II}_x II x is N x M N_x\mathcal{M} N x M can be shown by taking the directional derivative on the both sides of the identity Proj T x M ∘ Proj T x M = Proj T x M \operatorname{Proj}_{T_x\mathcal{M}}\circ\operatorname{Proj}_{T_x\mathcal{M}}=\operatorname{Proj}_{T_x\mathcal{M}} Proj T x M ∘ Proj T x M = Proj T x M , which yields
P x , u ∘ Proj N x M = Proj T x M ∘ P x , u , P x , u ∘ Proj T x M = Proj N x M ∘ P x , u . (51) P_{x,u}\circ\operatorname{Proj}_{N_x\mathcal{M}}=\operatorname{Proj}_{T_x\mathcal{M}}\circ P_{x,u},\qquad P_{x,u}\circ\operatorname{Proj}_{T_x\mathcal{M}}=\operatorname{Proj}_{N_x\mathcal{M}}\circ P_{x,u}. \tag*{(51)} P x , u ∘ Proj N x M = Proj T x M ∘ P x , u , P x , u ∘ Proj T x M = Proj N x M ∘ P x , u . (51)
By the above definition, the second fundamental form accounts for the changes in the way the tangent spaces sit inside the ambient E \mathcal{E} E .
Numerical comparisons between different retractions
We test Algorithm 2 equipped with different choices of retractions on the linear eigenvalue problem. For the problem description and most implementation details, please refer to § 4.1 . We fix n = 80 n = 80 n = 80 , p = 8 p = 8 p = 8 , and ξ = 1.05 \xi= 1.05 ξ = 1.05 , and search for the index-1 constrained SP. The initial points are generated either with random seed = 1 = 1 = 1 or from perturbation of the index-1 constrained SP (with the perturbation level β = 10 − 1 \beta= 10^{-1} β = 1 0 − 1 ) with random seed = 1 = 1 = 1 . We adopt the estimated best step sizes in Eqs. (46) and (47) .
A general construction of retraction on Gr k ( T M ) \operatorname{Gr}_{k}(T\mathcal{M}) Gr k ( T M ) is given in Lemma 5. For the linear eigenvalue problem, we have M = Gr p ( R n ) \mathcal{M} = \operatorname{Gr}_{p}(\mathbb{R}^{n}) M = Gr p ( R n ) and take R M = Exp M R^{\mathcal{M}} = \operatorname{Exp}^{\mathcal{M}} R M = Exp M , whose explicit form can be found in, e.g., [12 ] . For the other two mappings showing up in (ii) and (iii) of Lemma 5, we consider the concrete examples provided in Remark 1, which amount to four combinations in Table 3 .
Table 3. Retractions in comparison.
In experiments, all these combinations converge to the same solution from two different initializations, with the converged objective value 0.101898. As noted at the end of Remark 1, these combinations differ from the second order and can yield distinct numerical performance. Their convergence curves in wall-clock time on the linear eigenvalue instance are collected in Figure 7 . The results showcase that the combination “Add + Proj” in Table 3 is the most efficient in time. This can be ascribed to the fact that the operations of direction addition and metric projection are cheaper than those of exponential mapping and parallel transport. We note that the implementations in Section 4 all adopt “Add + Proj”. In terms of iteration number, the combinations “Exp + PT”, “Exp + Proj”, “Add + PT”, “Add + Proj” are only slightly different from each other, requiring respectively 5148, 5160, 5146, 5157 iterations with random initialization, and respectively 5288, 4943, 5214, 4961 iterations with perturbed initialization.
Figure 7. Convergence curves of Algorithm 2 implementing different retractions on the linear eigenvalue problem (n = 80 n = 80 n = 80 , p = 8 p = 8 p = 8 , and ξ = 1.05 \xi= 1.05 ξ = 1.05 ). The blue solid, orange dashed, green dash-dotted, and red dotted lines denote the results by the combinations “Exp + PT”, “Exp + Proj”, “Add + PT”, “Add + Proj”, respectively. The black dashed line refers to the tolerance 10 − 8 10^{-8} 1 0 − 8 . Top: random initialization. Bottom: initialization from 10 − 1 10^{-1} 1 0 − 1 perturbation near the index-1 SP. Left: position relative residual. Right: direction relative residual.