Mathematics Subject Classification 2020: 41A65 (primary); 41A05, 42B35, 46E22, 65D15 (secondary).
Keywords: reproducing kernel Hilbert spaces, reproducing kernels, Malmquist–Takenaka systems, model order reduction, support vector machines, neural networks.
Introduction
The study of model order reduction (MOR) deals with the simplification of mathematical models referred to as full–order models (FOMs). In MOR, FOMs and their simplified approximants usually have prescribed structure. The obtained approximant is called the reduced–order model (ROM). Traditionally, MOR techniques have been developed for reducing the complexity of dynamical systems [5 , 11 , 30 , 31 , 42 ] . Since many control engineering applications must operate in real time under limited hardware capacity, control schemes based on ROMs have proven particularly useful [49 ] . Well–known techniques exist for the reduction of models that describe linear time–invariant (LTI) systems [30 , 31 , 42 , 58 , 60 ] .
In order to address the MOR of LTI systems, several well–known methods have been developed that preserve the structure of the full order model, but provide theoretical guarantees on the performance of the ROM. One such well–known method is balanced truncation (BT) [30 ] , which operates on the state–space representation of LTI systems. It relies on a balanced realization of the system, which can be obtained by solving Lyapunov equations. Then, the least controllable and observable states in the balanced system are disregarded to obtain the ROM. One important benefit of the BT method is the theoretical guarantee it provides on the H ∞ H_{\infty} H ∞ error of the ROM.
Another popular approach to the reduction of LTI models is the iterative rational Krylov algorithm (iterative rational Krylov algorithm (IRKA)) introduced in [31 ] . IRKA is an iterative method that computes the ROM and guarantees that the result satisfies necessary interpolatory (or, equivalently, orthogonality–based) conditions for optimal approximation in the H 2 H_2 H 2 norm. In recent works [11 , 12 ] , the orthogonality–based optimality conditions proposed in [31 ] for H 2 ( C + ) H_2(\mathbb{C}^{+}) H 2 ( C + ) have been adapted to H 2 H_2 H 2 spaces corresponding to more general domains. For example, in [11 , 12 ] , interpolatory necessary conditions and an IRKA analog have been developed for H 2 H_2 H 2 spaces defined over simply connected sets. We also mention [8 ] , where H 2 H_2 H 2 –norm–optimal ROMs are obtained for bilinear systems. Again, the construction uses a variation of IRKA and satisfies an appropriate analog of the interpolatory conditions necessary for optimal approximation in the H 2 H_2 H 2 norm.
In this study, our objective is to further develop MOR methods along several directions of generalization. First, we exploit the fact that the H 2 ( C + ) H_2(\mathbb{C}^{+}) H 2 ( C + ) Hardy–Hilbert space is an reproducing kernel Hilbert space (RKHS) containing functions C + → C \mathbb{C}^{+} \to\mathbb{C} C + → C , where C + \mathbb{C}^{+} C + denotes the open right complex half–plane. We show that the necessary interpolatory optimality conditions proposed in [31 , 42 ] for H 2 H_2 H 2 –norm–optimal MOR can be generalized to an RKHS setting. The proposed optimality conditions are then used to construct iterative schemes to obtain norm–optimal ROMs. We note that our optimality conditions and subsequent algorithms can only be applied directly in RKHSs whose kernel is Wirtinger differentiable with respect to both variables. We refer to RKHSs adhering to this condition as admissible. Several relevant large models, such as deep ReLU neural networks [14 , 54 , 59 ] , cannot be represented in admissible RKHSs.
To overcome this issue, we introduce the RKHS variants of Malmquist–Takenaka (MT) functions [44 , 60 ] and refer to them as generalized Malmquist–Takenaka (GMT) functions. The classical MT functions form an orthonormal system in the Hardy–Hilbert space H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) , where D : = { z ∈ C : ∣ z ∣ < 1 } \mathbb{D} := \{z \in\mathbb{C} : |z| < 1\} D := { z ∈ C : ∣ z ∣ < 1 } denotes the interior of the complex unit disk. We prove that the proposed RKHS formulation of the MT system retains some important properties of its rational counterpart. We mention that drawing an analogy between special complex analytic functions and general RKHSs is not without precedent. In [26 ] , for example, an RKHS formulation of finite Blaschke products (special inner functions of H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) ) is proposed. Recently, in [55 ] , the rational MT system has been defined for a family of RKHSs known as boundary weighted Hardy spaces. We note that our definition of GMT functions contains these and other previous generalizations such as [47 ] as special cases. Using the introduced GMT bases and the Hilbert space isomorphism theorem, we define unitary transformations between any two separable RKHSs. These bijections allow us to apply the proposed optimality conditions and MOR algorithms by transforming the problem to an admissible RKHS such as H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) .
Finally, we present a collection of novel applications of norm–optimal MOR. The purpose of these examples is to demonstrate the versatility of the proposed RKHS–MOR framework. To this end, we include examples that perform RKHS–norm–optimal MOR on modern machine learning models. In particular, we consider the MOR of support vector machines and deep ReLU neural networks. In addition, we consider two somewhat surprising applications. These illustrate that, under some circumstances, our framework can be used to obtain norm–optimal approximations of FOMs that are not fully known. In particular, we first provide an academic example of fitting high–degree polynomials to data in an RKHS–norm–optimal manner. Then, we demonstrate how our RKHS approximation framework can be used to obtain kernel–basis weak solutions to certain partial differential equations (PDEs). These four applications range from an illustrative academic example to fully developed case studies, with the PDE application in particular presented as a proof of concept for a genuinely new use of the framework, rather than a completed method.
Below we list the key novelties of our study.
Optimality conditions in general RKHSs. We prove necessary conditions for RKHS norm optimality of the reduced–order model in admissible RKHSs. These results, see § 4 , unify and generalize conditions known for H 2 ( C + ) H_2(\mathbb{C}^+) H 2 ( C + ) approximation [31 , 42 ] .
Iterative RKHS MOR algorithms. We develop iterative schemes that, in admissible RKHSs, produce ROMs satisfying the introduced optimality conditions.
Generalized Malmquist–Takenaka (GMT) functions. We introduce GMT functions for RKHSs, showing that existing variants, including the classical H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) function system and previous generalizations [47 , 55 ] , arise as special cases. We show interpolation properties, zero locations, factorizations, and several other important properties of GMT functions.
(4) The GMT–transformation. We introduce the GMT transformation: a unitary map between two arbitrary separable RKHSs. We use the GMT transformation to solve MOR problems posed in non–admissible RKHSs.
(5) Diverse applications. We provide a diverse collection of example applications. We perform RKHS–norm–optimal MOR on support vector machines (SVMs) and neural networks. We develop an adaptive PDE discretization scheme based on the proposed approximation framework.
The paper is organized as follows. In § 2 , we revisit the definition of RKHSs and provide a formal problem statement. In § 3 , we review related mathematical concepts and prior results, focusing on properties of RKHSs and H 2 H_2 H 2 –optimal MOR. Our novel optimality conditions are introduced in § 4 . GMT functions, their properties and the GMT transformation are introduced in § 6 . § 7 contains our proposed MOR algorithms, while in § 8 , we demonstrate their effectiveness through four numerical examples. Finally, in § 10 , we summarize our results and discuss future research directions.
Problem statement
In this work, we are interested in the MOR of FOMs belonging to RKHSs. In order to provide a formal problem statement, we recall the definition of RKHSs and reproducing kernels.
Definition 2.1 (Reproducing kernel Hilbert space). Let Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C and H \mathcal{H} H be a Hilbert space of functions Ω → C \Omega\to\mathbb{C} Ω → C . If the point evaluation functionals
T λ : H → C , T λ ( F ) : = F ( λ ) ( λ ∈ Ω ) \mathcal{T}_{\lambda}: \mathcal{H} \to\mathbb{C}, \qquad\mathcal{T}_{\lambda}(F) := F(\lambda) \quad(\lambda\in\Omega) T λ : H → C , T λ ( F ) := F ( λ ) ( λ ∈ Ω ) are bounded (continuous) for all λ ∈ Ω \lambda\in\Omega λ ∈ Ω , then H \mathcal{H} H is called a reproducing kernel Hilbert space .
By Riesz’s representation theorem (see, e.g., [13 ] ), any bounded linear operator K \mathcal{K} K acting on a Hilbert space H \mathcal{H} H can be expressed as
K ( F ) = ⟨ F , h K ⟩ H ( F ∈ H ) \mathcal{K}(F) = \langle F, h_{\mathcal{K}}\rangle_{\mathcal{H}} \quad(F \in\mathcal{H}) K ( F ) = ⟨ F , h K ⟩ H ( F ∈ H )
for some unique h K ∈ H h_{\mathcal{K}} \in\mathcal{H} h K ∈ H . Hence, if H \mathcal{H} H is an RKHS, for all λ ∈ Ω \lambda\in\Omega λ ∈ Ω , there is a unique ξ λ ∈ H \xi_{\lambda} \in\mathcal{H} ξ λ ∈ H such that
T λ ( F ) = F ( λ ) = ⟨ F , ξ λ ⟩ H ( F ∈ H ) . \mathcal{T}_{\lambda}(F) = F(\lambda) = \langle F, \xi_{\lambda}\rangle_{\mathcal{H}} \quad(F \in\mathcal{H}). T λ ( F ) = F ( λ ) = ⟨ F , ξ λ ⟩ H ( F ∈ H ) .
This gives rise to the concept of a reproducing kernel.
Definition 2.2 (Reproducing kernel, kernel slice). Let H \mathcal{H} H be an RKHS of functions C ⊇ Ω → C \mathbb{C} \supseteq\Omega\to\mathbb{C} C ⊇ Ω → C . The map
ξ : Ω × Ω → C , \xi: \Omega\times\Omega\to\mathbb{C}, ξ : Ω × Ω → C ,
that satisfies
T λ ( F ) = F ( λ ) = ⟨ F , ξ ( ⋅ , λ ) ⟩ H = ⟨ F , ξ λ ⟩ H ( F ∈ H ) T_{\lambda}(F) = F(\lambda) = \langle F, \xi(\cdot,\lambda)\rangle_{\mathcal{H}} = \langle F,\xi_{\lambda}\rangle_{\mathcal{H}} \qquad(F \in\mathcal{H}) T λ ( F ) = F ( λ ) = ⟨ F , ξ ( ⋅ , λ ) ⟩ H = ⟨ F , ξ λ ⟩ H ( F ∈ H )
is called the reproducing kernel associated with H \mathcal{H} H . For some fixed λ ∈ Ω \lambda\in\Omega λ ∈ Ω , the map
ξ λ : = ξ ( ⋅ , λ ) ∈ H \xi_{\lambda} := \xi(\cdot,\lambda) \in\mathcal{H} ξ λ := ξ ( ⋅ , λ ) ∈ H
is called a kernel slice .
It can verified that a reproducing kernel is Hermitian (i.e., ξ ( x , λ ) = ξ ( λ , x ) ‾ \xi(x,\lambda) = \overline{\xi(\lambda,x)} ξ ( x , λ ) = ξ ( λ , x ) for all x , λ ∈ Ω x,\lambda\in\Omega x , λ ∈ Ω ) and positive semi–definite (i.e., ∑ i , j = 0 n − 1 c i c j ‾ ξ ( x i , x j ) ≥ 0 \sum_{i,j=0}^{n-1}c_i\overline{c_j}\xi(x_i,x_j) \ge0 ∑ i , j = 0 n − 1 c i c j ξ ( x i , x j ) ≥ 0 for all x 0 , … , x n − 1 x_0,\ldots,x_{n-1} x 0 , … , x n − 1 , c 0 , … , c n − 1 ∈ C c_0,\ldots,c_{n-1} \in\mathbb{C} c 0 , … , c n − 1 ∈ C and n ∈ N n \in\mathbb{N} n ∈ N ). By the Moore–Aronszajn theorem [2 ] , an RKHS and its associated reproducing kernel are in one–to–one correspondence. RKHSs and kernels have many useful properties, which have been exploited in various data–driven modeling applications such as machine learning (see, e.g., [20 , 33 , 65 ] ). In § 3.1 , we briefly review some of these properties necessary for the methods proposed in this study. For a thorough review of RKHSs, we recommend [52 ] .
We are now ready to formally state the problem of MOR in an RKHS. For this, we first define generalized model spaces.
Definition 2.3 (Generalized model space, dimension). Let Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C and H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C with reproducing kernel ξ \xi ξ . The generalized generalized model space in H \mathcal{H} H of dimension n ∈ N n \in\mathbb{N} n ∈ N is defined by
G ξ n : = { G ∈ span { ξ λ 0 , … , ξ λ n − 1 } : ( λ 0 , … , λ n − 1 ) ∈ Ω # n } . \mathcal{G}_{\xi}^{n} := \left\{G \in\operatorname{span}\{\xi_{\lambda_0},\ldots,\xi_{\lambda_{n-1}}\} : (\lambda_0,\ldots,\lambda_{n-1}) \in\Omega_{\#}^{n}\right\}. G ξ n := { G ∈ span { ξ λ 0 , … , ξ λ n − 1 } : ( λ 0 , … , λ n − 1 ) ∈ Ω # n } . where we define
Ω # n : = { ( λ 0 , … , λ n − 1 ) ∈ Ω n : λ j ≠ λ k for j ≠ k } . \Omega_{\#}^{n} := \left\{(\lambda_0,\ldots,\lambda_{n-1}) \in\Omega^{n} : \lambda_j \ne\lambda_k \text{ for } j \ne k\right\}. Ω # n := { ( λ 0 , … , λ n − 1 ) ∈ Ω n : λ j = λ k for j = k } .
We refer to the functions in generalized model spaces as models throughout the paper. Our objective in this study is to approximate models with functions that adhere to a special structure, e.g., simpler models. To obtain a meaningful MOR problem, we need to define the order , which serves as a measure of model complexity.
Definition 2.4 (Model order). Let n ∈ N n \in\mathbb{N} n ∈ N be the order of the model G G G , if G ∈ G ξ n ⊆ H G \in\mathcal{G}_{\xi}^{n} \subseteq\mathcal{H} G ∈ G ξ n ⊆ H , but G ∉ G ξ r ⊆ H G \notin\mathcal{G}_{\xi}^{r} \subseteq\mathcal{H} G ∈ / G ξ r ⊆ H for any r < n r < n r < n .
In § 6 and § 8 , we encounter problems where we need to transform the FOM into a companion admissible RKHS. This will be done via a unitary transformation F \mathcal{F} F acting between the original and companion RKHSs. For such cases, the ROM belongs to a weaker notion of a model space (in the original RKHS), specified by the following definition.
Definition 2.5 (Weak model space, dimension, weak model, order). Let Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C and H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C with reproducing kernel ξ \xi ξ . Let furthermore Ω ^ ⊆ C \widehat{\Omega} \subseteq\mathbb{C} Ω ⊆ C and H ^ \widehat{\mathcal{H}} H
be an RKHS of functions Ω ^ → C \widehat{\Omega} \to\mathbb{C} Ω → C with kernel ξ ^ \widehat{\xi} ξ . The weak model space in H \mathcal{H} H (with respect to H ^ \widehat{\mathcal{H}} H ) of dimension n ∈ N n \in\mathbb{N} n ∈ N is
W ξ n : = { G = F − 1 [ G ^ ] : G ^ ∈ G ξ n ⊆ H ^ , F : H → H ^ is unitary } . \mathbb{W}_{\xi}^{n} := \left\{ \mathsf{G} = \mathcal{F}^{-1}[\widehat{\mathsf{G}}] : \widehat{\mathsf{G}} \in\mathbb{G}_{\xi}^{n} \subseteq\widehat{\mathcal{H}},\ \mathcal{F} : \mathcal{H} \to\widehat{\mathcal{H}}\text{ is unitary} \right\}. W ξ n := { G = F − 1 [ G ] : G ∈ G ξ n ⊆ H , F : H → H is unitary } .
If G ∈ W ξ n \mathsf{G} \in\mathbb{W}_{\xi}^{n} G ∈ W ξ n , but G ∉ W ξ r \mathsf{G} \notin\mathbb{W}_{\xi}^{r} G ∈ / W ξ r for any r < n r < n r < n , we say that G \mathsf{G} G is a H \mathcal{H} H –weak model of order n n n .
We define norm–optimal MOR in an RKHS H \mathcal{H} H as follows.
Problem statement 2.7 (Norm–optimal MOR in RKHSs). Let Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C and H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C with reproducing kernel ξ \xi ξ . Consider a FOM G ∈ G ξ n \mathsf{G} \in\mathbb{G}_{\xi}^{n} G ∈ G ξ n for some n ∈ N n \in\mathbb{N} n ∈ N . Given r < n r < n r < n , the objective of norm–optimal MOR is to obtain G r ∈ G ξ r \mathsf{G}_r \in\mathbb{G}_{\xi}^{r} G r ∈ G ξ r that minimizes the functional E ξ r : G ξ r → [ 0 , ∞ ) \mathcal{E}_{\xi}^{r} : \mathbb{G}_{\xi}^{r} \to[0,\infty) E ξ r : G ξ r → [ 0 , ∞ ) defined as
E ξ r ( ⋅ ) = ∥ G − ⋅ ∥ H . (1) \mathcal{E}_{\xi}^{r}(\cdot) = \left\|\mathsf{G} - \cdot\right\|_{\mathcal{H}}. \tag*{(1)} E ξ r ( ⋅ ) = ∥ G − ⋅ ∥ H . (1) In the case of weak norm–optimal RKHS–MOR problems, we extend the search space (the domain of E ξ r \mathcal{E}_{\xi}^{r} E ξ r ) to ROMs satisfying G r ∈ W ξ r \mathsf{G}_r \in\mathbb{W}_{\xi}^{r} G r ∈ W ξ r .
Definition 2.8 (Local minimum points of E ξ r \mathcal{E}_{\xi}^{r} E ξ r ). Let E ξ r \mathcal{E}_{\xi}^{r} E ξ r be an objective functional as defined in Problem statement 2.7. The ROM G r ∈ G ξ r \mathsf{G}_r \in\mathbb{G}_{\xi}^{r} G r ∈ G ξ r is a local minimum point of E ξ r \mathcal{E}_{\xi}^{r} E ξ r if there exists δ > 0 \delta> 0 δ > 0 such that, for any G ~ r ∈ G ξ r \widetilde{\mathsf{G}}_r \in\mathbb{G}_{\xi}^{r} G r ∈ G ξ r that satisfies ∥ G r − G ~ r ∥ H < δ \left\|\mathsf{G}_r - \widetilde{\mathsf{G}}_r\right\|_{\mathcal{H}} < \delta G r − G r H < δ , we have
E ξ r ( G r ) ≤ E ξ r ( G ~ r ) . \mathcal{E}_{\xi}^{r}(\mathsf{G}_r) \leq\mathcal{E}_{\xi}^{r}(\widetilde{\mathsf{G}}_r). E ξ r ( G r ) ≤ E ξ r ( G r ) .
In this work, we seek ROMs which satisfy necessary optimality conditions introduced in § 4 . Since they are not sufficient in general, we cannot ensure to find a local minimum in any case. But we conjecture that local maxima are repellent, so using a numerical method based on the necessary optimality conditions is a feasible route to follow. This is similar to what classical MOR methods guarantee for the reduction of linear dynamical systems [6 , 31 , 45 ] .
Figure 1. The proposed MOR pipeline for RKHSs.
Preliminaries
In this section, we first review some important RKHS-related concepts that will be used later. In addition, we motivate the research by briefly revisiting existing MOR approaches for LTI systems.
Properties of RKHSs
Definition 2.2 gives the following property.
Lemma 3.1 (Reproducing inner products). Let H \mathcal{H} H be an RKHS with kernel ξ \xi ξ . Then,
ξ ( x , λ ) = ⟨ ξ x , ξ λ ⟩ H ( x , λ ∈ Ω ) . \xi(x,\lambda) = \langle\xi_x,\xi_\lambda\rangle_{\mathcal{H}} \quad(x,\lambda\in\Omega). ξ ( x , λ ) = ⟨ ξ x , ξ λ ⟩ H ( x , λ ∈ Ω ) .
We will use the following theorem, which draws a connection between the derivatives of kernel slices and H \mathcal{H} H inner products. Several notions of derivatives will be used throughout the paper. To ease the discussion, we first introduce some notation. First, consider a kernel slice ξ λ ∈ H \xi_\lambda\in\mathcal{H} ξ λ ∈ H . Whenever existence conditions are satisfied, the notations D λ ξ λ D_\lambda\xi_\lambda D λ ξ λ and D λ ^ ξ ^ λ ^ D_{\widehat{\lambda}}\widehat{\xi}_{\widehat{\lambda}} D λ ξ λ refer to the Wirtinger derivative functions defined by
lim Δ λ → 0 ∥ ξ λ + Δ λ ( ⋅ ) − ξ λ ( ⋅ ) Δ λ − ( D λ ξ λ ) ( ⋅ ) ∥ H = 0 , \lim_{\Delta\lambda\to0}\left\lVert\frac{\xi_{\lambda+\Delta\lambda}(\cdot)-\xi_\lambda(\cdot)}{\Delta\lambda}-(D_\lambda\xi_\lambda)(\cdot)\right\rVert_{\mathcal{H}}=0, Δ λ → 0 lim Δ λ ξ λ + Δ λ ( ⋅ ) − ξ λ ( ⋅ ) − ( D λ ξ λ ) ( ⋅ ) H = 0 ,
lim Δ λ ‾ → 0 ∥ ξ λ + Δ λ ‾ ( ⋅ ) − ξ λ ( ⋅ ) Δ λ ‾ − ( D λ ‾ ξ λ ) ( ⋅ ) ∥ H = 0. (2) \lim_{\Delta\overline{\lambda}\to0}\left\lVert\frac{\xi_{\lambda+\Delta\overline{\lambda}}(\cdot)-\xi_\lambda(\cdot)}{\Delta\overline{\lambda}}-(D_{\overline{\lambda}}\xi_\lambda)(\cdot)\right\rVert_{\mathcal{H}}=0. \tag*{(2)} Δ λ → 0 lim Δ λ ξ λ + Δ λ ( ⋅ ) − ξ λ ( ⋅ ) − ( D λ ξ λ ) ( ⋅ ) H = 0. (2)
Since H \mathcal{H} H is an RKHS, convergence in norm implies pointwise convergence [52 ] and (2) matches the Wirtinger derivatives pointwise, i.e., we have ∂ ξ λ ∂ λ ( z ) = D λ ξ λ ( z ) \frac{\partial\xi_\lambda}{\partial\lambda}(z)=D_\lambda\xi_\lambda(z) ∂ λ ∂ ξ λ ( z ) = D λ ξ λ ( z ) and ∂ ξ λ ∂ λ ‾ ( z ) = D λ ‾ ξ λ ( z ) \frac{\partial\xi_\lambda}{\partial\overline{\lambda}}(z)=D_{\overline{\lambda}}\xi_\lambda(z) ∂ λ ∂ ξ λ ( z ) = D λ ξ λ ( z ) for all z ∈ Ω z\in\Omega z ∈ Ω . Let λ = ( λ 0 , … , λ n − 1 ) ∈ Ω ξ n \boldsymbol{\lambda}=(\lambda_0,\ldots,\lambda_{n-1})\in\Omega_\xi^n λ = ( λ 0 , … , λ n − 1 ) ∈ Ω ξ n and consider the function Ω ξ n → H \Omega_\xi^n\to\mathcal{H} Ω ξ n → H
U ( λ ) : = ∑ k = 0 n − 1 r k ξ λ k ( ⋅ ) . \mathcal{U}(\boldsymbol{\lambda}):=\sum_{k=0}^{n-1}\mathfrak{r}_k\xi_{\lambda_k}(\cdot). U ( λ ) := k = 0 ∑ n − 1 r k ξ λ k ( ⋅ ) .
For a fixed λ ∈ Ω ξ n \boldsymbol{\lambda}\in\Omega_\xi^n λ ∈ Ω ξ n , let G : = U ( λ ) ∈ G ξ n \mathbf{G}:=\mathcal{U}(\boldsymbol{\lambda})\in\mathfrak{G}_\xi^n G := U ( λ ) ∈ G ξ n . Then we can define the derivative of G \mathbf{G} G with respect to the parameter λ k \lambda_k λ k of a kernel slice as
D λ k G : = [ ∇ U ( λ ) ] k : = r k ( D λ ξ λ ) ∣ λ = λ k and D λ k ‾ G : = [ ∇ ‾ U ( λ ) ] k : = r k ( D λ ‾ ξ λ ) ∣ λ = λ k . D_{\lambda_k}\mathbf{G}:=[\nabla\mathcal{U}(\boldsymbol{\lambda})]_k:=\mathfrak{r}_k(D_\lambda\xi_\lambda)|_{\lambda=\lambda_k}
\quad\text{and}\quad
D_{\overline{\lambda_k}}\mathbf{G}:=[\overline{\nabla}\mathcal{U}(\boldsymbol{\lambda})]_k:=\mathfrak{r}_k(D_{\overline{\lambda}}\xi_\lambda)|_{\lambda=\lambda_k}. D λ k G := [ ∇ U ( λ ) ] k := r k ( D λ ξ λ ) ∣ λ = λ k and D λ k G := [ ∇ U ( λ ) ] k := r k ( D λ ξ λ ) ∣ λ = λ k .
Moreover, for G ∈ G ξ n \mathbf{G}\in\mathfrak{G}_\xi^n G ∈ G ξ n , we define
D λ G : = ∑ k = 0 n − 1 D λ k G and D λ ‾ G : = ∑ k = 0 n − 1 D λ k ‾ G . D_\lambda\mathbf{G}:=\sum_{k=0}^{n-1}D_{\lambda_k}\mathbf{G}
\quad\text{and}\quad
D_{\overline{\lambda}}\mathbf{G}:=\sum_{k=0}^{n-1}D_{\overline{\lambda_k}}\mathbf{G}. D λ G := k = 0 ∑ n − 1 D λ k G and D λ G := k = 0 ∑ n − 1 D λ k G .
For higher-order derivatives, we use similar notation. In particular, to denote the m m m th order Wirtinger derivative (function) of some G ∈ G ξ n \mathbf{G} \in\mathcal{G}_{\xi}^{n} G ∈ G ξ n for m ≥ 2 m \ge2 m ≥ 2 , we use the notations
D λ k m G : = D λ k ( D λ k m − 1 G ) and D λ m G : = D λ ( D λ m − 1 G ) . D_{\lambda_k}^{m}\mathbf{G} := D_{\lambda_k}\left(D_{\lambda_k}^{m-1}\mathbf{G}\right) \quad\text{and} \quad D_{\lambda}^{m}\mathbf{G} := D_{\lambda}\left(D_{\lambda}^{m-1}\mathbf{G}\right). D λ k m G := D λ k ( D λ k m − 1 G ) and D λ m G := D λ ( D λ m − 1 G ) .
In several places, we also need to denote pointwise Wirtinger derivatives of G ∈ H \mathbf{G} \in\mathcal{H} G ∈ H . Since we will only need to express ∂ G ∂ z ‾ ( z ) \frac{\partial\mathbf{G}}{\partial\overline{z}}(z) ∂ z ∂ G ( z ) , we use the notation
∂ G ∂ z ‾ ( z ) = G ′ ( z ) . \frac{\partial\mathbf{G}}{\partial\overline{z}}(z)=\mathbf{G}'(z). ∂ z ∂ G ( z ) = G ′ ( z ) .
Finally, note that for higher-order derivatives, for m ≥ 2 m \ge2 m ≥ 2 we use the notation
∂ m G ∂ z ‾ m ( z ) = G ( m ) ( z ) . \frac{\partial^{m}\mathbf{G}}{\partial\overline{z}^{m}}(z)=\mathbf{G}^{(m)}(z). ∂ z m ∂ m G ( z ) = G ( m ) ( z ) .
Recall the following connection between derivatives and reproducing kernels. Since this theorem is frequently relied upon throughout the study, we provide a short proof for clarity.
Theorem 3.2 (Derivatives of kernels [72 ] ). Let H \mathcal{H} H be an RKHS of functions C ⊇ Ω → C \mathbb{C} \supseteq\Omega\to\mathbb{C} C ⊇ Ω → C with kernel ξ \xi ξ . Let ( λ ↦ ξ λ ) ∈ C 1 ( Ω , H ) (\lambda\mapsto\xi_{\lambda}) \in C^{1}(\Omega,\mathcal{H}) ( λ ↦ ξ λ ) ∈ C 1 ( Ω , H ) such that
D λ ξ λ ∈ H ( λ ∈ Ω ) . D_{\lambda}\xi_{\lambda} \in\mathcal{H} \quad(\lambda\in\Omega). D λ ξ λ ∈ H ( λ ∈ Ω ) . Then,
⟨ G , D λ ξ λ ⟩ H = G ′ ( λ ) . \left\langle\mathbf{G},D_{\lambda}\xi_{\lambda}\right\rangle_{\mathcal{H}}=\mathbf{G}'(\lambda). ⟨ G , D λ ξ λ ⟩ H = G ′ ( λ ) .
Proof. We have
⟨ G , D λ ξ λ ⟩ H = ⟨ G , lim Δ λ → 0 ξ λ + Δ λ − ξ λ Δ λ ⟩ H = lim Δ λ → 0 1 Δ λ ⟨ G , ξ λ + Δ λ − ξ λ ⟩ H = lim Δ λ → 0 ⟨ G , ξ λ + Δ λ ⟩ H − ⟨ G , ξ λ ⟩ H Δ λ = lim Δ λ → 0 G ( λ + Δ λ ) − G ( λ ) Δ λ = G ′ ( λ ) . \begin{aligned}
\left\langle\mathbf{G},D_{\lambda}\xi_{\lambda}\right\rangle_{\mathcal{H}}
&=\left\langle\mathbf{G},\lim_{\Delta\lambda\to0}\frac{\xi_{\lambda+\Delta\lambda}-\xi_{\lambda}}{\Delta\lambda}\right\rangle_{\mathcal{H}} \\
&=\lim_{\Delta\lambda\to0}\frac{1}{\Delta\lambda}\left\langle\mathbf{G},\xi_{\lambda+\Delta\lambda}-\xi_{\lambda}\right\rangle_{\mathcal{H}} \\
&=\lim_{\Delta\lambda\to0}\frac{\left\langle\mathbf{G},\xi_{\lambda+\Delta\lambda}\right\rangle_{\mathcal{H}}-\left\langle\mathbf{G},\xi_{\lambda}\right\rangle_{\mathcal{H}}}{\Delta\lambda} \\
&=\lim_{\Delta\lambda\to0}\frac{\mathbf{G}(\lambda+\Delta\lambda)-\mathbf{G}(\lambda)}{\Delta\lambda}=\mathbf{G}'(\lambda).
\end{aligned} ⟨ G , D λ ξ λ ⟩ H = ⟨ G , Δ λ → 0 lim Δ λ ξ λ + Δ λ − ξ λ ⟩ H = Δ λ → 0 lim Δ λ 1 ⟨ G , ξ λ + Δ λ − ξ λ ⟩ H = Δ λ → 0 lim Δ λ ⟨ G , ξ λ + Δ λ ⟩ H − ⟨ G , ξ λ ⟩ H = Δ λ → 0 lim Δ λ G ( λ + Δ λ ) − G ( λ ) = G ′ ( λ ) .
Theorem 3.2 extends to derivatives of arbitrary order by induction.
Corollary 3.3 (Higher-order derivative reproduction). Let H \mathcal{H} H be an RKHS of functions C ⊇ Ω → C \mathbb{C} \supseteq\Omega\to\mathbb{C} C ⊇ Ω → C with kernel ξ \xi ξ . Let m ∈ N m \in\mathbb{N} m ∈ N and suppose ( λ ↦ ξ λ ) ∈ C m ( Ω , H ) (\lambda\mapsto\xi_{\lambda}) \in C^{m}(\Omega,\mathcal{H}) ( λ ↦ ξ λ ) ∈ C m ( Ω , H ) such that D λ i ξ λ ∈ H D_{\lambda}^{i}\xi_{\lambda} \in\mathcal{H} D λ i ξ λ ∈ H for i = 0 , … , m i=0,\ldots,m i = 0 , … , m and all λ ∈ Ω \lambda\in\Omega λ ∈ Ω . Then, for any G ∈ H \mathbf{G} \in\mathcal{H} G ∈ H and i = 0 , … , m i=0,\ldots,m i = 0 , … , m ,
⟨ G , D λ i ξ λ ⟩ H = G ( i ) ( λ ) . \left\langle\mathbf{G},D_{\lambda}^{i}\xi_{\lambda}\right\rangle_{\mathcal{H}}=\mathbf{G}^{(i)}(\lambda). ⟨ G , D λ i ξ λ ⟩ H = G ( i ) ( λ ) .
Proof. The case m = 0 m = 0 m = 0 is the reproducing property and m = 1 m = 1 m = 1 is Theorem 3.2 . Let m ≥ 2 m \ge2 m ≥ 2 and suppose the statement holds for some m > i ≥ 1 m > i \ge1 m > i ≥ 1 . Then,
⟨ G , D λ i + 1 ξ λ ⟩ H = ⟨ G , lim Δ λ → 0 D λ i ξ λ + Δ λ − D λ i ξ λ Δ λ ⟩ H = lim Δ λ → 0 1 Δ λ ⟨ G , D λ i ξ λ + Δ λ − D λ i ξ λ ⟩ H = lim Δ λ → 0 ⟨ G , D λ i ξ λ + Δ λ ⟩ H − ⟨ G , D λ i ξ λ ⟩ H Δ λ = lim Δ λ → 0 G ( i ) ( λ + Δ λ ) − G ( i ) ( λ ) Δ λ = G ( i + 1 ) ( λ ) . \begin{aligned}
\left\langle\mathbf{G}, D_{\lambda}^{i+1}\xi_{\lambda}\right\rangle_{\mathcal{H}} = \left\langle\mathbf{G}, \lim_{\Delta\lambda\to0} \frac{D_{\lambda}^{i}\xi_{\lambda+\Delta\lambda}-D_{\lambda}^{i}\xi_{\lambda}}{\Delta\lambda}\right\rangle_{\mathcal{H}} \\
&= \lim_{\Delta\lambda\to0} \frac{1}{\Delta\lambda}\left\langle\mathbf{G},D_{\lambda}^{i}\xi_{\lambda+\Delta\lambda}-D_{\lambda}^{i}\xi_{\lambda}\right\rangle_{\mathcal{H}} \\
&= \lim_{\Delta\lambda\to0} \frac{\left\langle\mathbf{G},D_{\lambda}^{i}\xi_{\lambda+\Delta\lambda}\right\rangle_{\mathcal{H}}-\left\langle\mathbf{G},D_{\lambda}^{i}\xi_{\lambda}\right\rangle_{\mathcal{H}}}{\Delta\lambda} \\
&= \lim_{\Delta\lambda\to0} \frac{G^{(i)}(\lambda+\Delta\lambda)-G^{(i)}(\lambda)}{\Delta\lambda}=G^{(i+1)}(\lambda).
\end{aligned} ⟨ G , D λ i + 1 ξ λ ⟩ H = ⟨ G , Δ λ → 0 lim Δ λ D λ i ξ λ + Δ λ − D λ i ξ λ ⟩ H = Δ λ → 0 lim Δ λ 1 ⟨ G , D λ i ξ λ + Δ λ − D λ i ξ λ ⟩ H = Δ λ → 0 lim Δ λ ⟨ G , D λ i ξ λ + Δ λ ⟩ H − ⟨ G , D λ i ξ λ ⟩ H = Δ λ → 0 lim Δ λ G ( i ) ( λ + Δ λ ) − G ( i ) ( λ ) = G ( i + 1 ) ( λ ) .
One notable property of RKHSs is that kernel slices are dense in H \mathcal{H} H .
Theorem 3.4 (Density of kernel slices (e.g., [19 ] ).) Let H \mathcal{H} H be an RKHS with associated reproducing kernel ξ \xi ξ . Then,
H = cl ( span { ξ λ : λ ∈ Ω } ) , \mathcal{H}=\operatorname{cl}\left(\operatorname{span}\{\xi_{\lambda}:\lambda\in\Omega\}\right), H = cl ( span { ξ λ : λ ∈ Ω } ) , where cl ( ⋅ ) \operatorname{cl}(\cdot) cl ( ⋅ ) denotes the closure of a set with respect to the norm induced by ⟨ ⋅ , ⋅ ⟩ H \langle\cdot,\cdot\rangle_{\mathcal{H}} ⟨ ⋅ , ⋅ ⟩ H . Furthermore, if H \mathcal{H} H is separable, then a countable subset of Ω \Omega Ω exists such that the kernel slices defined by it are dense in H \mathcal{H} H .
The following characterization of kernels is useful for our discussion.
Definition 3.5 (Feature maps, feature spaces [38 ] ). Let H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C with associated kernel ξ \xi ξ . A Hilbert space S \mathfrak{S} S is called a feature space, if there exists a map Φ : Ω → S \Phi:\Omega\to\mathfrak{S} Φ : Ω → S that satisfies
ξ ( x , λ ) = ⟨ Φ ( x ) , Φ ( λ ) ⟩ S . \xi(x,\lambda)=\langle\Phi(x),\Phi(\lambda)\rangle_{\mathfrak{S}}. ξ ( x , λ ) = ⟨ Φ ( x ) , Φ ( λ ) ⟩ S . In this case, Φ \Phi Φ is referred to as a feature map. Finally, denote the set of all feature map–feature space pairs by
S ξ : = { ( Φ , S ) : ξ ( x , λ ) = ⟨ Φ ( x ) , Φ ( λ ) ⟩ S for all x , λ ∈ Ω } . \mathfrak{S}_{\xi}:=\{(\Phi,\mathfrak{S}):\xi(x,\lambda)=\langle\Phi(x),\Phi(\lambda)\rangle_{\mathfrak{S}}\text{ for all }x,\lambda\in\Omega\}. S ξ := {( Φ , S ) : ξ ( x , λ ) = ⟨ Φ ( x ) , Φ ( λ ) ⟩ S for all x , λ ∈ Ω } .
For any RKHS H \mathcal{H} H , we have S ξ ≠ ∅ \mathfrak{S}_{\xi}\ne\varnothing S ξ = ∅ [38 ] . The feature map representation can be useful for characterizing kernels. For example, let ℓ \ell ℓ be the vector space of one-sided complex sequences and
ℓ 2 : = { x ∈ ℓ : ∑ k = 0 ∞ ∣ x k ∣ 2 < ∞ } \ell^{2}:=\left\{\mathbf{x}\in\ell:\sum_{k=0}^{\infty}|x_k|^{2}<\infty\right\} ℓ 2 := { x ∈ ℓ : k = 0 ∑ ∞ ∣ x k ∣ 2 < ∞ }
be the Hilbert space endowed with the usual inner product
⟨ x , y ⟩ ℓ 2 : = ∑ k = 0 ∞ x k y k ‾ ( x , y ∈ ℓ 2 ) . \langle\mathbf{x},\mathbf{y}\rangle_{\ell^{2}}:=\sum_{k=0}^{\infty}x_k\overline{y_k}\quad(\mathbf{x},\mathbf{y}\in\ell^{2}). ⟨ x , y ⟩ ℓ 2 := k = 0 ∑ ∞ x k y k ( x , y ∈ ℓ 2 ) .
If H \mathcal{H} H is a separable RKHS and { φ k } k = 0 ∞ ⊆ H \{\varphi_k\}_{k=0}^{\infty} \subseteq\mathcal{H} { φ k } k = 0 ∞ ⊆ H is a complete orthonormal basis, then choosing the feature space as ℓ 2 \ell^2 ℓ 2 and Φ ( x ) = ( φ k ( x ) ) k = 0 ∞ ∈ ℓ 2 \Phi(x) = (\varphi_k(x))_{k=0}^{\infty} \in\ell^2 Φ ( x ) = ( φ k ( x ) ) k = 0 ∞ ∈ ℓ 2 yields ξ ( x , t ) = ⟨ Φ ( x ) , Φ ( t ) ⟩ ℓ 2 \xi(x,t) = \langle\Phi(x), \Phi(t) \rangle_{\ell^2} ξ ( x , t ) = ⟨ Φ ( x ) , Φ ( t ) ⟩ ℓ 2 . This statement also follows from Mercer’s theorem [43 ] .
We note that every finite-dimensional Hilbert space is an RKHS and its kernel can be conveniently expressed using the feature map approach.
Theorem 3.6 (Finite-dimensional Hilbert spaces [19 ] ). Let H \mathcal{H} H be a Hilbert space of functions Ω → C \Omega\to\mathbb{C} Ω → C and { φ k } k = 0 n − 1 ⊆ H \{\varphi_k\}_{k=0}^{n-1} \subseteq\mathcal{H} { φ k } k = 0 n − 1 ⊆ H be a basis for some n ∈ N n \in\mathbb{N} n ∈ N . Then, the n n n -dimensional subspace H n : = span { φ 0 , … , φ n − 1 } \mathcal{H}_n := \operatorname{span}\{\varphi_0,\ldots,\varphi_{n-1}\} H n := span { φ 0 , … , φ n − 1 } is an RKHS defined by the positive semi-definite reproducing kernel
ξ ( x , λ ) = ∑ k = 0 n − 1 φ k ( x ) φ k ( λ ) ‾ ( x , λ ∈ Ω ) . \xi(x,\lambda) = \sum_{k=0}^{n-1} \varphi_k(x)\overline{\varphi_k(\lambda)} \quad(x,\lambda\in\Omega). ξ ( x , λ ) = k = 0 ∑ n − 1 φ k ( x ) φ k ( λ ) ( x , λ ∈ Ω ) . Furthermore, for any two functions G = ∑ k = 0 n − 1 g k φ k , F = ∑ k = 0 n − 1 f k φ k ∈ H n \mathbf{G} = \sum_{k=0}^{n-1} \mathfrak{g}_k\varphi_k,\mathbf{F} = \sum_{k=0}^{n-1} \mathfrak{f}_k\varphi_k \in\mathcal{H}_n G = ∑ k = 0 n − 1 g k φ k , F = ∑ k = 0 n − 1 f k φ k ∈ H n , the inner products on H n \mathcal{H}_n H n and H \mathcal{H} H are related by
⟨ G , F ⟩ H n = ∑ k = 0 n − 1 g k f k ‾ , ⟨ G , F ⟩ H = ∑ k = 0 n − 1 ∑ j = 0 n − 1 ⟨ φ k , φ j ⟩ H g k f j ‾ . \langle\mathbf{G},\mathbf{F}\rangle_{\mathcal{H}_n} = \sum_{k=0}^{n-1} \mathfrak{g}_k\overline{\mathfrak{f}_k}, \qquad\langle\mathbf{G},\mathbf{F}\rangle_{\mathcal{H}} = \sum_{k=0}^{n-1}\sum_{j=0}^{n-1}\langle\varphi_k,\varphi_j\rangle_{\mathcal{H}}\mathfrak{g}_k\overline{\mathfrak{f}_j}. ⟨ G , F ⟩ H n = k = 0 ∑ n − 1 g k f k , ⟨ G , F ⟩ H = k = 0 ∑ n − 1 j = 0 ∑ n − 1 ⟨ φ k , φ j ⟩ H g k f j .
Importantly, for strictly positive definite kernels, kernel slices are independent for different choices of kernel centers.
Theorem 3.7 (Linear independence of kernel slices (Theorem 3.6 in [52 ] )). Let H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C with strictly positive definite kernel ξ \xi ξ . Then, for any ( λ 0 , … , λ n ) ∈ Ω n + 1 (\lambda_0,\ldots,\lambda_n) \in\Omega^{n+1} ( λ 0 , … , λ n ) ∈ Ω n + 1 , ξ λ n ∉ span { ξ λ 0 , … , ξ λ n − 1 } \xi_{\lambda_n} \notin\operatorname{span}\{\xi_{\lambda_0},\ldots,\xi_{\lambda_{n-1}}\} ξ λ n ∈ / span { ξ λ 0 , … , ξ λ n − 1 } .
Finally, we consider the following statement about finite-dimensional RKHSs embedded into Hilbert spaces.
Lemma 3.8 (Embedded finite-dimensional RKHSs ). Let H \mathcal{H} H be a Hilbert space (not necessarily an RKHS ) of functions Ω → C \Omega\to\mathbb{C} Ω → C . Let r ∈ N r \in\mathbb{N} r ∈ N and let V r ⊆ H \mathcal{V}_r \subseteq\mathcal{H} V r ⊆ H be an r r r -dimensional subspace of H \mathcal{H} H , and let { φ j } j = 0 r − 1 ⊆ H \{\varphi_j\}_{j=0}^{r-1} \subseteq\mathcal{H} { φ j } j = 0 r − 1 ⊆ H be an orthonormal system with respect to ⟨ ⋅ , ⋅ ⟩ H \langle\cdot,\cdot\rangle_{\mathcal{H}} ⟨ ⋅ , ⋅ ⟩ H . Fix an arbitrary ( λ ~ 0 , … , λ ~ r − 1 ) ∈ Ω r (\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{r-1}) \in\Omega^r ( λ 0 , … , λ r − 1 ) ∈ Ω r . Then
V r : = span { φ 0 , … , φ r − 1 } = span { ξ λ ~ 0 , … , ξ λ ~ r − 1 } , (3) \mathcal{V}_r := \operatorname{span}\{\varphi_0,\ldots,\varphi_{r-1}\} = \operatorname{span}\{\xi_{\widetilde{\lambda}_0},\ldots,\xi_{\widetilde{\lambda}_{r-1}}\}, \tag*{(3)} V r := span { φ 0 , … , φ r − 1 } = span { ξ λ 0 , … , ξ λ r − 1 } , (3) where the Ω × Ω → C \Omega\times\Omega\to\mathbb{C} Ω × Ω → C map
ξ ( x , λ ) : = ∑ j = 0 r − 1 φ j ( x ) φ j ( λ ) ‾ \xi(x,\lambda) := \sum_{j=0}^{r-1}\varphi_j(x)\overline{\varphi_j(\lambda)} ξ ( x , λ ) := j = 0 ∑ r − 1 φ j ( x ) φ j ( λ ) is the reproducing kernel in V r \mathcal{V}_r V r corresponding to the inner product ⟨ ⋅ , ⋅ ⟩ H \langle\cdot,\cdot\rangle_{\mathcal{H}} ⟨ ⋅ , ⋅ ⟩ H . Furthermore, for any G ∈ H \mathbf{G} \in\mathcal{H} G ∈ H , we have
⟨ G , ξ ~ λ ~ j ⟩ H = P V r [ G ] ( λ ~ j ) ( j = 0 , … , r − 1 ) , (4) \langle\mathbf{G},\widetilde{\xi}_{\widetilde{\lambda}_j}\rangle_{\mathcal{H}} = \mathcal{P}_{\mathcal{V}_r}[\mathbf{G}](\widetilde{\lambda}_j) \quad(j = 0,\ldots,r-1), \tag*{(4)} ⟨ G , ξ λ j ⟩ H = P V r [ G ] ( λ j ) ( j = 0 , … , r − 1 ) , (4) where P V r : H → V r \mathcal{P}_{\mathcal{V}_r}: \mathcal{H} \to\mathcal{V}_r P V r : H → V r is the orthogonal projector to V r \mathcal{V}_r V r .
Proof. By Theorem 3.6 , V r V_r V r is an RKHS associated with the reproducing kernel ξ \xi ξ and with respect to the inner product ⟨ ⋅ , ⋅ ⟩ H \langle\cdot,\cdot\rangle_{\mathcal{H}} ⟨ ⋅ , ⋅ ⟩ H . Moreover, for any ( α 0 , … , α n − 1 ) ∈ C n (\alpha_0,\ldots,\alpha_{n-1})\in\mathbb{C}^n ( α 0 , … , α n − 1 ) ∈ C n , we have
∑ k = 0 r − 1 α k ξ ~ λ e k ( ⋅ ) = ∑ k = 0 r − 1 ∑ j = 0 r − 1 α k φ j ( ⋅ ) φ j ( λ e k ) ‾ = ∑ j = 0 r − 1 ( ∑ k = 0 r − 1 α k φ j ( λ e k ) ‾ ) φ j ( ⋅ ) . \sum_{k=0}^{r-1}\alpha_k\widetilde{\xi}_{\lambda e_k}(\cdot)=\sum_{k=0}^{r-1}\sum_{j=0}^{r-1}\alpha_k\varphi_j(\cdot)\overline{\varphi_j(\lambda e_k)}=\sum_{j=0}^{r-1}\left(\sum_{k=0}^{r-1}\alpha_k\overline{\varphi_j(\lambda e_k)}\right)\varphi_j(\cdot). k = 0 ∑ r − 1 α k ξ λ e k ( ⋅ ) = k = 0 ∑ r − 1 j = 0 ∑ r − 1 α k φ j ( ⋅ ) φ j ( λ e k ) = j = 0 ∑ r − 1 ( k = 0 ∑ r − 1 α k φ j ( λ e k ) ) φ j ( ⋅ ) .
which implies span { ξ ~ λ e 0 , … , ξ ~ λ e r − 1 } ⊆ V r \operatorname{span}\{\widetilde{\xi}_{\lambda e_0},\ldots,\widetilde{\xi}_{\lambda e_{r-1}}\}\subseteq V_r span { ξ λ e 0 , … , ξ λ e r − 1 } ⊆ V r . By Theorem 3.7 , we obtain (3) .
If G ∈ H G\in\mathcal{H} G ∈ H , then (4) is a consequence of the fact that G = P V r ⊥ [ G ] + P V r [ G ] G=P_{V_r}^{\perp}[G]+P_{V_r}[G] G = P V r ⊥ [ G ] + P V r [ G ] , where P V r ⊥ P_{V_r}^{\perp} P V r ⊥ is the projection operator to H ⊖ V r : = H ∩ V r ⊥ \mathcal{H}\ominus V_r:=\mathcal{H}\cap V_r^{\perp} H ⊖ V r := H ∩ V r ⊥ , where ⊖ \ominus ⊖ denotes the orthogonal difference between H \mathcal{H} H and V r V_r V r . Then, using the reproducing property of the kernel, we get
⟨ G , ξ ~ λ e j ⟩ H = ⟨ P V r ⊥ [ G ] + P V r [ G ] , ξ ~ λ e j ⟩ H = ⟨ P V r ⊥ [ G ] , ξ ~ λ e j ⟩ H + P V r [ G ] ( λ e j ) = P V r [ G ] ( λ e j ) . \begin{aligned}
\left\langle G,\widetilde{\xi}_{\lambda e_j}\right\rangle_{\mathcal{H}}=\left\langle P_{V_r}^{\perp}[G]+P_{V_r}[G],\widetilde{\xi}_{\lambda e_j}\right\rangle_{\mathcal{H}} \\
&=\left\langle P_{V_r}^{\perp}[G],\widetilde{\xi}_{\lambda e_j}\right\rangle_{\mathcal{H}}+P_{V_r}[G](\lambda e_j)=P_{V_r}[G](\lambda e_j).
\end{aligned} ⟨ G , ξ λ e j ⟩ H = ⟨ P V r ⊥ [ G ] + P V r [ G ] , ξ λ e j ⟩ H = ⟨ P V r ⊥ [ G ] , ξ λ e j ⟩ H + P V r [ G ] ( λ e j ) = P V r [ G ] ( λ e j ) .
Finally, we recall the Hilbert space isomorphism theorem. The method, introduced in § 6 , gives a recipe to write such an isomorphism between any two separable RKHSs of the same dimension.
Theorem 3.9 (Hilbert space isomorphism theorem (see, e.g., Theorem 4.14 in [13 ] ).) Two Hilbert spaces are isometrically isomorphic if and only if they have equal dimensions.
MOR of LTI systems
The methods proposed in this paper can be interpreted as generalizations of well-known MOR schemes developed for linear time-invariant (LTI) dynamical systems. For this reason, we briefly review the corresponding MOR methods. Later, we show that these well-known methods rely mostly on the fact that they operate in an RKHS, namely the Hardy–Hilbert space H 2 ( C + ) H_2(\mathbb{C}^+) H 2 ( C + ) . This observation allows us to pose the problem in a more general RKHS setting, where we obtain similar results.
First, consider a single-input single-output (SISO)-LTI-FOM in continuous-time, given by the state-space representation
x ˙ ( t ) = A x ( t ) + b u ( t ) , x ( 0 ) = 0 , \dot{x}(t)=Ax(t)+bu(t),\qquad x(0)=0, x ˙ ( t ) = A x ( t ) + b u ( t ) , x ( 0 ) = 0 ,
y ( t ) = c T x ( t ) , (5) y(t)=c^{\mathsf{T}}x(t), \tag*{(5)} y ( t ) = c T x ( t ) , (5)
where A ∈ C n × n A\in\mathbb{C}^{n\times n} A ∈ C n × n , b , c ∈ C n × 1 b,c\in\mathbb{C}^{n\times1} b , c ∈ C n × 1 and x ( t ) ∈ C n x(t)\in\mathbb{C}^n x ( t ) ∈ C n , u ( t ) , y ( t ) ∈ C u(t),y(t)\in\mathbb{C} u ( t ) , y ( t ) ∈ C for all t ∈ [ 0 , ∞ ) t\in[0,\infty) t ∈ [ 0 , ∞ ) . The quantities x ( t ) x(t) x ( t ) , u ( t ) u(t) u ( t ) , and y ( t ) y(t) y ( t ) are referred to as the state, input, and output at time t t t . We refer to n n n as the order of the system. The goal of LTI-MOR is to identify a ROM of order r ≪ n r\ll n r ≪ n given by
x ˙ r ( t ) = A r x r ( t ) + b r u ( t ) , x r ( 0 ) = 0 , \dot{x}_r(t)=A_rx_r(t)+b_ru(t),\qquad x_r(0)=0, x ˙ r ( t ) = A r x r ( t ) + b r u ( t ) , x r ( 0 ) = 0 ,
y r ( t ) = c r T x r ( t ) , (6) y_r(t)=c_r^{\mathsf{T}}x_r(t), \tag*{(6)} y r ( t ) = c r T x r ( t ) , (6)
where A r ∈ C r × r A_r \in\mathbb{C}^{r \times r} A r ∈ C r × r , b r , c r ∈ C r × 1 \mathbf{b}_r,\mathbf{c}_r \in\mathbb{C}^{r \times1} b r , c r ∈ C r × 1 , x r ( t ) ∈ C r \mathbf{x}_r(t) \in\mathbb{C}^r x r ( t ) ∈ C r , and y r ( t ) ∈ C y_r(t) \in\mathbb{C} y r ( t ) ∈ C for t ∈ [ 0 , ∞ ) t \in[0,\infty) t ∈ [ 0 , ∞ ) such that y r ( t ) ≈ y ( t ) y_r(t) \approx y(t) y r ( t ) ≈ y ( t ) for all admissible inputs u ( ⋅ ) u(\cdot) u ( ⋅ ) .
The systems given in (5) and (6) are fully defined by their respective transfer functions
G ( s ) : = c T ( s I − A ) − 1 b , G r ( s ) : = c r T ( s I − A r ) − 1 b r , (7) G(s) := \mathbf{c}^{\mathsf{T}}(sI-A)^{-1}\mathbf{b}, \qquad G_r(s) := \mathbf{c}_r^{\mathsf{T}}(sI-A_r)^{-1}\mathbf{b}_r, \tag*{(7)} G ( s ) := c T ( s I − A ) − 1 b , G r ( s ) := c r T ( s I − A r ) − 1 b r , (7)
where s ∈ C + : = { ω ∈ C : Re ω > 0 } s \in\mathbb{C}^+ := \{\omega\in\mathbb{C} : \operatorname{Re}\omega> 0\} s ∈ C + := { ω ∈ C : Re ω > 0 } belongs to the right complex half-plane. The transfer functions G G G and G r G_r G r are rational and their poles are eigenvalues of A A A and A r A_r A r , respectively. If A A A and A r A_r A r from (5) and (6) are Hurwitz, we call the corresponding systems asymptotically stable. In this case, the transfer functions belong to the Hardy–Hilbert space
H 2 ( C + ) : = { G ∈ A ( C + ) : 1 2 π ∫ − ∞ ∞ ∣ G ( i ω ) ∣ 2 d ω < ∞ } . H_2(\mathbb{C}^+) := \left\{G \in\mathcal{A}(\mathbb{C}^+) : \frac{1}{2\pi}\int_{-\infty}^{\infty} \lvert G(\mathrm{i}\omega)\rvert^2\,\mathrm{d}\omega< \infty\right\}. H 2 ( C + ) := { G ∈ A ( C + ) : 2 π 1 ∫ − ∞ ∞ ∣ G ( i ω ) ∣ 2 d ω < ∞ } .
where A ( C + ) \mathcal{A}(\mathbb{C}^+) A ( C + ) denotes the set of analytic functions on C + \mathbb{C}^+ C + . The inner product on this space is
⟨ G , H ⟩ H 2 ( C + ) : = 1 2 π ∫ − ∞ ∞ G ( i ω ) H ( i ω ) ‾ d ω ( G , H ∈ H 2 ( C + ) ) . \langle G,H\rangle_{H_2(\mathbb{C}^+)} := \frac{1}{2\pi}\int_{-\infty}^{\infty}G(\mathrm{i}\omega)\overline{H(\mathrm{i}\omega)}\,\mathrm{d}\omega\qquad(G,H \in H_2(\mathbb{C}^+)). ⟨ G , H ⟩ H 2 ( C + ) := 2 π 1 ∫ − ∞ ∞ G ( i ω ) H ( i ω ) d ω ( G , H ∈ H 2 ( C + )) .
Furthermore, it is an RKHS with the Cauchy kernel
ζ ( s , w ) = 1 s + w ‾ ( s , w ∈ C + ) . (8) \zeta(s,w) = \frac{1}{s+\overline{w}} \qquad(s,w \in\mathbb{C}^+). \tag*{(8)} ζ ( s , w ) = s + w 1 ( s , w ∈ C + ) . (8)
Let μ k \mu_k μ k and μ ~ j \widetilde{\mu}_j μ j denote the eigenvalues of A A A and A r A_r A r from (5) and (6) , respectively (k = 0 , … , n − 1 k=0,\ldots,n-1 k = 0 , … , n − 1 , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 ). Then, we can write the transfer functions in (7) (see, e.g., [31 ] ) as
G = ∑ k = 0 n − 1 r k ζ λ k , G r = ∑ j = 0 r − 1 r ~ j ζ λ ~ j , \mathbf{G}=\sum_{k=0}^{n-1} \mathfrak{r}_k\zeta_{\lambda_k}, \qquad\mathbf{G}_r=\sum_{j=0}^{r-1}\widetilde{\mathfrak{r}}_j\zeta_{\widetilde{\lambda}_j}, G = k = 0 ∑ n − 1 r k ζ λ k , G r = j = 0 ∑ r − 1 r j ζ λ j ,
where λ k = − μ k \lambda_k=-\mu_k λ k = − μ k , λ ~ j = − μ ~ j \widetilde{\lambda}_j=-\widetilde{\mu}_j λ j = − μ j and r k , r ~ j ∈ C \mathfrak{r}_k,\widetilde{\mathfrak{r}}_j\in\mathbb{C} r k , r j ∈ C (k = 0 , … , n − 1 k=0,\ldots,n-1 k = 0 , … , n − 1 , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 ). Noticing that H 2 ( C + ) H_2(\mathbb{C}^+) H 2 ( C + ) is an RKHS allows us to formulate the classical H 2 ( C + ) H_2(\mathbb{C}^+) H 2 ( C + ) –optimal MOR problem for SISO–LTI systems using Problem statement 2.7. In particular, fixing Ω : = C + \Omega:= \mathbb{C}^+ Ω := C + , ξ : = ζ \xi:= \zeta ξ := ζ , and H : = H 2 ( C + ) \mathcal{H} := H_2(\mathbb{C}^+) H := H 2 ( C + ) , we notice that G \mathbf{G} G from (7) belongs to G n r \mathcal{G}_n^r G n r , where G n r ⊆ H 2 ( C + ) \mathcal{G}_n^r \subseteq H_2(\mathbb{C}^+) G n r ⊆ H 2 ( C + ) is a generalized model space defined according to Definition 2.3. Then, H 2 ( C + ) H_2(\mathbb{C}^+) H 2 ( C + ) –optimal MOR seeks the ROM G r \mathbf{G}_r G r (see (7) ) that is a local minimizer (see Definition 2.8) of the objective function E r r \mathcal{E}_r^r E r r given in Problem statement 2.7. Such a local minimizer satisfies the following necessary conditions.
Theorem 3.10 (Meier–Luenberger conditions [31 , 42 ] ). Let G ∈ G n r \mathbf{G} \in\mathcal{G}_n^r G ∈ G n r be a FOM. Suppose that G r ∈ G r r \mathbf{G}_r \in\mathcal{G}_r^r G r ∈ G r r of order r r r is a local minimizer of E r r \mathcal{E}_r^r E r r . Then,
G r ( λ ~ j ) = G ( λ ~ j ) , G r ′ ( λ ~ j ) = G ′ ( λ ~ j ) ( j = 0 , … , r − 1 ) , (9) \mathbf{G}_r(\widetilde{\lambda}_j)=\mathbf{G}(\widetilde{\lambda}_j), \qquad\mathbf{G}_r'(\widetilde{\lambda}_j)=\mathbf{G}'(\widetilde{\lambda}_j) \qquad(j=0,\ldots,r-1), \tag*{(9)} G r ( λ j ) = G ( λ j ) , G r ′ ( λ j ) = G ′ ( λ j ) ( j = 0 , … , r − 1 ) , (9) where λ ~ j \widetilde{\lambda}_j λ j denote the parameters identifying G r \mathbf{G}_r G r (see (7) ).
Theorem 3.10 is equivalent to the following orthogonality conditions.
Theorem 3.11 (Gugercin–Antoulas–Beattie conditions [31 ] ). Let G ∈ G ζ n \mathbf{G} \in\mathcal{G}^{n}_{\zeta} G ∈ G ζ n be a FOM . Suppose that G r ∈ G ζ r \mathbf{G}_{r} \in\mathcal{G}^{r}_{\zeta} G r ∈ G ζ r of order r r r is a local minimizer of E ζ r E^{r}_{\zeta} E ζ r . Then,
⟨ G − G r , G r ⋅ H 1 + H 2 ⟩ H 2 ( C + ) = 0 ( H 1 , H 2 ∈ span { ζ λ ~ 0 , … , ζ λ ~ r − 1 } ) , (10) \left\langle\mathbf{G}-\mathbf{G}_{r},\mathbf{G}_{r}\cdot H_{1}+H_{2}\right\rangle_{H_{2}(\mathbb{C}^{+})}=0 \qquad\left(H_{1},H_{2}\in\operatorname{span}\left\{\zeta_{\tilde{\lambda}_{0}},\ldots,\zeta_{\tilde{\lambda}_{r-1}}\right\}\right), \tag*{(10)} ⟨ G − G r , G r ⋅ H 1 + H 2 ⟩ H 2 ( C + ) = 0 ( H 1 , H 2 ∈ span { ζ λ ~ 0 , … , ζ λ ~ r − 1 } ) , (10) where λ ~ j \tilde{\lambda}_{j} λ ~ j denote the parameters identifying G r \mathbf{G}_{r} G r (see (7) ).
Remark 3.12. The equivalence between Theorems 3.10 and 3.11 is proven in [31 ] . This equivalence can also be easily shown using an RKHS argument, see Appendix § 11 .§ 1 . Even though this proof is a direct consequence of the general optimality condition introduced in § 4 , it demonstrates that the result in H 2 ( C + ) H_{2}(\mathbb{C}^{+}) H 2 ( C + ) can be proven by exploiting the reproducing property in RKHSs.
The current state–of–the–art approach for obtaining a ROM that satisfies Theorem 3.10 and Theorem 3.11 is IRKA introduced in [31 ] . This method uses repeated Galerkin projections, based on the state–space representations (5) and (6) , to obtain the ROM. Although convergence is formally proven only for state–space–symmetric systems [27 ] , the method has been empirically verified to yield high–quality ROMs for most LTI systems. A limitation of IRKA is that it can provide ROMs whose parameters λ ~ j ∉ C − \tilde{\lambda}_{j}\notin\mathbb{C}^{-} λ ~ j ∈ / C − for some j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 and consequently G r ∉ H \mathbf{G}_{r}\notin\mathcal{H} G r ∈ / H . In system theory, such systems are referred to as unstable. Recently, alternatives to IRKA address this limitation in [45 ] and [1 ] .
Finally, we briefly consider discrete–time LTI systems. Although analogs of Theorems 3.10 and 3.11 hold for this case [31 ] , we instead review an equivalent theorem recently introduced in [1 ] . This formulation is very useful for the RKHS–MOR methods developed in § 7 . In the discrete–time case, the time variable t t t in (5) and (6) belongs to N \mathbb{N} N and x ˙ ( t ) \dot{x}(t) x ˙ ( t ) is replaced by x [ t + 1 ] x[t+1] x [ t + 1 ] . Discrete–time LTI systems can be fully described by rational functions
G ( z ) = ∑ k = 0 n − 1 r k 1 − λ k z , G r ( z ) = ∑ j = 0 r − 1 r ~ j 1 − λ ~ j z ( z ∈ D ) , (11) \mathbf{G}(z)=\sum_{k=0}^{n-1}\frac{r_{k}}{1-\lambda_{k}z},\qquad\mathbf{G}_{r}(z)=\sum_{j=0}^{r-1}\frac{\tilde{r}_{j}}{1-\tilde{\lambda}_{j}z}\qquad(z\in\mathbb{D}), \tag*{(11)} G ( z ) = k = 0 ∑ n − 1 1 − λ k z r k , G r ( z ) = j = 0 ∑ r − 1 1 − λ ~ j z r ~ j ( z ∈ D ) , (11)
where D \mathbb{D} D denotes the open complex unit disk, r k , r ~ j ∈ C r_{k},\tilde{r}_{j}\in\mathbb{C} r k , r ~ j ∈ C and λ k , λ ~ j \lambda_{k},\tilde{\lambda}_{j} λ k , λ ~ j are defined as λ k = μ ‾ k \lambda_{k}=\overline{\mu}_{k} λ k = μ k , λ ~ j = μ ~ ‾ j \tilde{\lambda}_{j}=\overline{\tilde{\mu}}_{j} λ ~ j = μ ~ j , with μ k \mu_{k} μ k and μ ~ j \tilde{\mu}_{j} μ ~ j denoting the eigenvalues of A \mathbf{A} A and A r \mathbf{A}_{r} A r , respectively, for k = 0 , … , n − 1 k=0,\ldots,n-1 k = 0 , … , n − 1 and j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 . For a thorough overview of this statement, see, e.g., [1 , 21 ] . In discrete time, asymptotically stable systems correspond to A \mathbf{A} A and A r \mathbf{A}_{r} A r from (5) and (6) having eigenvalues in D \mathbb{D} D . In this case, the functions in (11) belong to the Hardy–Hilbert space
H 2 ( D ) : = { G ∈ A ( D ) : 1 2 π ∫ − π π ∣ G ( e i t ) ∣ 2 d t < ∞ } , (12) H_{2}(\mathbb{D}):=\left\{\mathbf{G}\in\mathcal{A}(\mathbb{D}):\frac{1}{2\pi}\int_{-\pi}^{\pi}\left|\mathbf{G}(e^{\mathrm{i}t})\right|^{2}\,\mathrm{d}t<\infty\right\}, \tag*{(12)} H 2 ( D ) := { G ∈ A ( D ) : 2 π 1 ∫ − π π G ( e i t ) 2 d t < ∞ } , (12)
where A ( D ) \mathcal{A}(\mathbb{D}) A ( D ) is a short–hand notation for the complex–analytic functions on D \mathbb{D} D . This is again an RKHS with the reproducing Szegő kernel
ρ ( z , λ ) = 1 1 − λ ‾ z ( z , λ ∈ D ) . (13) \rho(z,\lambda)=\frac{1}{1-\overline{\lambda}z}\qquad(z,\lambda\in\mathbb{D}). \tag*{(13)} ρ ( z , λ ) = 1 − λ z 1 ( z , λ ∈ D ) . (13)
Kernel slices in H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) will be denoted by ρ λ \rho_\lambda ρ λ similarly to Definition 2.2 . In order to state the necessary optimality condition, following [1 ] , we first define the equilibrium function associated with the FOM G ∈ G ρ n \mathbf{G} \in\mathfrak{G}_{\rho}^{n} G ∈ G ρ n .
Definition 3.13 (Equilibrium function). Let G = ∑ k = 0 n − 1 r k ρ λ k ∈ G ρ n \mathbf{G} = \sum_{k=0}^{n-1} r_k \rho_{\lambda_k} \in\mathfrak{G}_{\rho}^{n} G = ∑ k = 0 n − 1 r k ρ λ k ∈ G ρ n . The equilibrium function associated with G \mathbf{G} G is given by F G : D r → C r F_{\mathbf{G}} : \mathbb{D}^{r} \to\mathbb{C}^{r} F G : D r → C r , where
[ F G ( z ) ] j : = ∑ k = 0 n − 1 r k ρ λ k ( z j ) ∏ i = 0 r − 1 λ k ‾ − z i ‾ 1 − z i λ k ( j = 0 , … , r − 1 ) . [F_{\mathbf{G}}(\mathbf{z})]_j := \sum_{k=0}^{n-1} r_k \rho_{\lambda_k}(z_j) \prod_{i=0}^{r-1} \frac{\overline{\lambda_k}-\overline{z_i}}{1-z_i\lambda_k} \qquad(j=0,\ldots,r-1). [ F G ( z ) ] j := k = 0 ∑ n − 1 r k ρ λ k ( z j ) i = 0 ∏ r − 1 1 − z i λ k λ k − z i ( j = 0 , … , r − 1 ) . for z = [ z 0 , … , z r − 1 ] T \mathbf{z}=[z_0,\ldots,z_{r-1}]^{\mathsf{T}} z = [ z 0 , … , z r − 1 ] T .
The following theorem provides a necessary condition that a discrete–time LTI–ROM G r ∈ G ρ r \mathbf{G}_r \in\mathfrak{G}_{\rho}^{r} G r ∈ G ρ r must satisfy if it is a local minimizer of the error functional E ρ r \mathcal{E}_{\rho}^{r} E ρ r .1
Theorem 3.14 (Equilibrium condition [1 ] ). Let G ∈ G ρ n ( D ) \mathbf{G} \in\mathfrak{G}_{\rho}^{n}(\mathbb{D}) G ∈ G ρ n ( D ) , F G F_{\mathbf{G}} F G be the associated equilibrium function and G r = ∑ j = 0 r − 1 r ~ j ρ λ ~ j ∈ G ρ r \mathbf{G}_r=\sum_{j=0}^{r-1}\widetilde{r}_j\rho_{\widetilde{\lambda}_j}\in\mathfrak{G}_{\rho}^{r} G r = ∑ j = 0 r − 1 r j ρ λ j ∈ G ρ r of order r r r a local minimizer of E ρ r \mathcal{E}_{\rho}^{r} E ρ r (see Problem statement 2.7). Then, λ ~ = [ λ ~ 0 , … , λ ~ r − 1 ] T \widetilde{\boldsymbol{\lambda}}=[\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{r-1}]^{\mathsf{T}} λ = [ λ 0 , … , λ r − 1 ] T satisfies
F G ( λ ~ ) = 0. F_{\mathbf{G}}(\widetilde{\boldsymbol{\lambda}})=0. F G ( λ ) = 0.
Using Theorem 3.14 , we can pose the discrete time SISO LTI–MOR problem as a zero finding task. In [1 ] this is solved with a Newton method which exploits hyperbolic geometry. This intrinsically guarantees that the obtained parameters λ ~ \widetilde{\boldsymbol{\lambda}} λ remain in D r \mathbb{D}^{r} D r in each iteration of the algorithm, always resulting in an asymptotically stable ROM.
At each iteration, the method first attempts a standard Euclidean Newton step for the equilibrium equation. If the resulting candidate remains in D r \mathbb{D}^{r} D r , the update is accepted; otherwise, the method switches to a hyperbolically corrected step that keeps the parameters inside D r \mathbb{D}^{r} D r . Thus, the equilibrium–based approach gives stricter structural guarantees than IRKA, since the recovered parameters λ ~ \widetilde{\boldsymbol{\lambda}} λ remain in D r \mathbb{D}^{r} D r throughout each iteration. For further details about the hybrid hyperbolic Newton step, we refer to [1 ] .
In this study, we propose Newton iterations similar to Algorithm 3.1 in any RKHSs with (Wirtinger) differentiable kernels. In addition, we extend this strategy to more general RKHSs in § 7 by transforming the FOM in the original RKHS to a FOM in a companion RKHS with more desirable properties such as H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) (because of the availability of Algorithm 3.1). This can be done exploiting the unitary transformations proposed in § 6.3 .
Later, we will observe a unique advantage of Algorithm 3.1 over Galerkin–projection–based methods like IRKA when reducing models in RKHSs defined over real intervals. Consider the space
H 2 I : = H 2 ( D ) ∣ ( − 1 , 1 ) . (14) H_2^{\mathbb{I}} := H_2(\mathbb{D})\big|_{(-1,1)}. \tag*{(14)} H 2 I := H 2 ( D ) ( − 1 , 1 ) . (14)
Algorithm 3.1 Hybrid equilibrium–based MOR in H 2 ( D ) H_2(\mathbb{D}) H 2 ( D )
Inputs: FOM G ∈ G n \mathbf{G} \in\mathfrak{G}^n G ∈ G n , reduced order r < n r < n r < n , initial parameters z ( 0 ) ∈ D r \mathbf{z}^{(0)} \in\mathbb{D}^r z ( 0 ) ∈ D r .
Outputs: ROM G r ∈ G ρ r \mathbf{G}_r \in\mathfrak{G}^r_\rho G r ∈ G ρ r .
Construct the equilibrium function F G : D r → C r F_{\mathbf{G}} : \mathbb{D}^r \to\mathbb{C}^r F G : D r → C r .
ν ← 0 \nu\leftarrow0 ν ← 0 .
repeat
Compute a Newton correction Δ z ( ν ) \Delta\mathbf{z}^{(\nu)} Δ z ( ν ) for F G ( z ( ν ) ) = 0 F_{\mathbf{G}}(\mathbf{z}^{(\nu)}) = 0 F G ( z ( ν ) ) = 0 .
if z ( ν ) + Δ z ( ν ) ∈ D r \mathbf{z}^{(\nu)} + \Delta\mathbf{z}^{(\nu)} \in\mathbb{D}^r z ( ν ) + Δ z ( ν ) ∈ D r then
z ( ν + 1 ) ← z ( ν ) + Δ z ( ν ) \mathbf{z}^{(\nu+1)} \leftarrow\mathbf{z}^{(\nu)} + \Delta\mathbf{z}^{(\nu)} z ( ν + 1 ) ← z ( ν ) + Δ z ( ν ) . ▷ Euclidean Newton step.
else
Compute hyperbolic correction Δ z H ( ν ) \Delta\mathbf{z}^{(\nu)}_{\mathrm{H}} Δ z H ( ν ) . ▷ See § 4 in [1 ] .
z ( ν + 1 ) ← Exp z ( ν ) ( Δ z H ( ν ) ) \mathbf{z}^{(\nu+1)} \leftarrow\operatorname{Exp}_{\mathbf{z}^{(\nu)}}\left(\Delta\mathbf{z}^{(\nu)}_{\mathrm{H}}\right) z ( ν + 1 ) ← Exp z ( ν ) ( Δ z H ( ν ) ) .
end if
ν ← ν + 1 \nu\leftarrow\nu+ 1 ν ← ν + 1 .
until convergence of z ( ν ) \mathbf{z}^{(\nu)} z ( ν )
λ ~ ← z ( ν ) \widetilde{\boldsymbol{\lambda}} \leftarrow\mathbf{z}^{(\nu)} λ ← z ( ν ) .
Construct G r ∈ G ρ r \mathbf{G}_r \in\mathfrak{G}^r_\rho G r ∈ G ρ r using the parameters λ ~ \widetilde{\boldsymbol{\lambda}} λ .
return G r \mathbf{G}_r G r .
Note that H I 2 H_I^2 H I 2 is an RKHS that is induced by the Szegő kernel restricted to the domain I 2 \mathbb{I}^2 I 2 , see [2 page 351] . In particular, all rational models in G ρ n ⊂ H 2 ( D ) \mathfrak{G}^n_\rho\subset H_2(\mathbb{D}) G ρ n ⊂ H 2 ( D ) with real kernel centers and real linear coefficients belong to H I 2 H_I^2 H I 2 . For such models, Algorithm 3.1 produces a ROM candidate in each iteration maintaining these properties.
Summary of notation
Table 1 collects the core notation introduced above that is used throughout the remainder of the paper. Notation specific to a single section (e.g., the GMT system of § 6 ) is instead introduced, and recalled, where it is first needed.
Table 1. Summary of the core notation used throughout the paper.
Model spaces
Derivatives and operators
Necessary conditions of norm optimality in RKHS model reduction
In this section, we introduce orthogonality and interpolatory conditions analogous to Theorems 3.10 and 3.11 in an RKHS setting. In addition, we generalize the notion of the equilibrium function from Definition 3.13 and obtain a generalization of Theorem 3.14 .
In order to state the results in a general form, we update our definition of generalized model spaces from Definition 2.3 . The remainder of this study uses the following, extended definition of G ξ n \mathbb{G}_{\xi}^{n} G ξ n .
Definition 4.1 (Generalized model space, dimension, model order). Let Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C and H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C . Suppose ( λ ↦ ξ λ ) ∈ C m ( Ω , H ) (\lambda\mapsto\xi_\lambda) \in C^m(\Omega,\mathcal{H}) ( λ ↦ ξ λ ) ∈ C m ( Ω , H ) with m ≥ 1 m \ge1 m ≥ 1 and D λ ‾ i ξ λ ∈ H D_{\overline{\lambda}}^i\xi_\lambda\in\mathcal{H} D λ i ξ λ ∈ H for all λ ∈ Ω \lambda\in\Omega λ ∈ Ω and i = 0 , … , m i=0,\ldots,m i = 0 , … , m . The generalized model space of dimension n n n is defined as
G ξ n , m : = { G ∈ span { ξ λ 0 , D λ ‾ ξ λ 0 , … , D λ ‾ m 0 ξ λ 0 , … , ξ λ k − 1 , D λ ‾ ξ λ k − 1 , … , D λ ‾ m k − 1 ξ λ k − 1 } : \mathbb{G}_{\xi}^{n,m}:=\{\mathrm{G}\in\operatorname{span}\{\xi_{\lambda_0},D_{\overline{\lambda}}\xi_{\lambda_0},\ldots,D_{\overline{\lambda}}^{m_0}\xi_{\lambda_0},\ldots,\xi_{\lambda_{k-1}},D_{\overline{\lambda}}\xi_{\lambda_{k-1}},\ldots,D_{\overline{\lambda}}^{m_{k-1}}\xi_{\lambda_{k-1}}\}: G ξ n , m := { G ∈ span { ξ λ 0 , D λ ξ λ 0 , … , D λ m 0 ξ λ 0 , … , ξ λ k − 1 , D λ ξ λ k − 1 , … , D λ m k − 1 ξ λ k − 1 } : ( λ 0 , … , λ k − 1 ) ∈ Ω k , ∑ j = 0 k − 1 m j = n , 0 ≤ m j ≤ m for j = 0 , … , k − 1 } . (\lambda_0,\ldots,\lambda_{k-1})\in\Omega^k,\ \sum_{j=0}^{k-1}m_j=n,\ 0\le m_j\le m\ \text{for }j=0,\ldots,k-1\}. ( λ 0 , … , λ k − 1 ) ∈ Ω k , j = 0 ∑ k − 1 m j = n , 0 ≤ m j ≤ m for j = 0 , … , k − 1 } . The model G \mathrm{G} G is of order n ∈ N n\in\mathbb{N} n ∈ N , if G ∈ G ξ n , m ⊆ H \mathrm{G}\in\mathbb{G}_{\xi}^{n,m}\subseteq\mathcal{H} G ∈ G ξ n , m ⊆ H , but G ∉ G ξ r , m ⊆ H \mathrm{G}\notin\mathbb{G}_{\xi}^{r,m}\subseteq\mathcal{H} G ∈ / G ξ r , m ⊆ H for any r < n r<n r < n .
Remark 4.2. a) Note that G ξ n , 0 = G ξ n \mathbb{G}_{\xi}^{n,0}=\mathbb{G}_{\xi}^{n} G ξ n , 0 = G ξ n with G ξ n \mathbb{G}_{\xi}^{n} G ξ n as in Definition 2.3 .
b) For ease of notation, let us from now on assume that the kernel in Definition 4.1 is at most m m m –times continuously differentiable, i.e., m m m denotes the maximum regularity of the kernel. For this reason, we will drop the superscript m m m in G ξ n , m \mathbb{G}_{\xi}^{n,m} G ξ n , m from now on.
Theorem 4.3 (A necessary condition for norm–optimal ROMs in RKHSs with m m m –times continuously differentiable kernels). Let Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C and H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C . Suppose ( λ ↦ ξ λ ) ∈ C m ( Ω , H ) (\lambda\mapsto\xi_\lambda) \in C^m(\Omega,\mathcal{H}) ( λ ↦ ξ λ ) ∈ C m ( Ω , H ) with m ≥ 1 m \ge1 m ≥ 1 and D λ i ξ λ ∈ H D_\lambda^i \xi_\lambda\in\mathcal{H} D λ i ξ λ ∈ H for all λ ∈ Ω \lambda\in\Omega λ ∈ Ω and i = 0 , … , m i = 0,\ldots,m i = 0 , … , m .
Given a fixed G ∈ H G \in\mathcal{H} G ∈ H , suppose that
G r = ∑ j = 0 k − 1 ∑ i = 0 m j − 1 r ~ j , i D λ ~ i ξ λ ~ j . G_r = \sum_{j=0}^{k-1}\sum_{i=0}^{m_j-1}\widetilde{r}_{j,i}D_{\widetilde{\lambda}}^i\xi_{\widetilde{\lambda}_j}. G r = j = 0 ∑ k − 1 i = 0 ∑ m j − 1 r j , i D λ i ξ λ j . where ( λ ~ 0 , … , λ ~ k − 1 ) ∈ Ω ∗ k (\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{k-1}) \in\Omega_*^k ( λ 0 , … , λ k − 1 ) ∈ Ω ∗ k , ∑ j = 0 k − 1 m j = r \sum_{j=0}^{k-1}m_j = r ∑ j = 0 k − 1 m j = r , 1 ≤ m j < m 1 \le m_j < m 1 ≤ m j < m , and r ~ j , m j − 1 ≠ 0 \widetilde{r}_{j,m_j-1} \ne0 r j , m j − 1 = 0 for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 is a local minimizer of E ξ r \mathcal{E}_{\xi}^{r} E ξ r . Consider the H \mathcal{H} H subspaces
V r λ ~ : = span { ξ λ ~ 0 , D λ ~ ξ λ ~ 0 , … , D λ ~ m 0 − 1 ξ λ ~ 0 , … , ξ λ ~ k − 1 , D λ ~ ξ λ ~ k − 1 , … , D λ ~ m k − 1 − 1 ξ λ ~ k − 1 } (15) V_r^{\widetilde{\boldsymbol{\lambda}}} := \operatorname{span}\left\{\xi_{\widetilde{\lambda}_0},D_{\widetilde{\lambda}} \xi_{\widetilde{\lambda}_0},\ldots,D_{\widetilde{\lambda}}^{m_0-1}\xi_{\widetilde{\lambda}_0},\ldots,\xi_{\widetilde{\lambda}_{k-1}},D_{\widetilde{\lambda}}\xi_{\widetilde{\lambda}_{k-1}},\ldots,D_{\widetilde{\lambda}}^{m_{k-1}-1}\xi_{\widetilde{\lambda}_{k-1}}\right\} \tag*{(15)} V r λ := span { ξ λ 0 , D λ ξ λ 0 , … , D λ m 0 − 1 ξ λ 0 , … , ξ λ k − 1 , D λ ξ λ k − 1 , … , D λ m k − 1 − 1 ξ λ k − 1 } (15) and
D V r λ ~ : = span { D λ ~ ξ λ ~ 0 , D λ ~ 2 ξ λ ~ 0 , … , D λ ~ m 0 ξ λ ~ 0 , … , D λ ~ ξ λ ~ k − 1 , D λ ~ 2 ξ λ ~ k − 1 , … , D λ ~ m k − 1 ξ λ ~ k − 1 } . (16) DV_r^{\widetilde{\boldsymbol{\lambda}}} := \operatorname{span}\left\{D_{\widetilde{\lambda}}\xi_{\widetilde{\lambda}_0},D_{\widetilde{\lambda}}^2\xi_{\widetilde{\lambda}_0},\ldots,D_{\widetilde{\lambda}}^{m_0}\xi_{\widetilde{\lambda}_0},\ldots,D_{\widetilde{\lambda}}\xi_{\widetilde{\lambda}_{k-1}},D_{\widetilde{\lambda}}^2\xi_{\widetilde{\lambda}_{k-1}},\ldots,D_{\widetilde{\lambda}}^{m_{k-1}}\xi_{\widetilde{\lambda}_{k-1}}\right\}. \tag*{(16)} D V r λ := span { D λ ξ λ 0 , D λ 2 ξ λ 0 , … , D λ m 0 ξ λ 0 , … , D λ ξ λ k − 1 , D λ 2 ξ λ k − 1 , … , D λ m k − 1 ξ λ k − 1 } . (16) Let us construct V ~ r λ ~ ⊆ V r λ ~ \widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}} \subseteq V_r^{\widetilde{\boldsymbol{\lambda}}} V r λ ⊆ V r λ and D V ~ r λ ~ ⊆ D V r λ ~ D\widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}} \subseteq DV_r^{\widetilde{\boldsymbol{\lambda}}} D V r λ ⊆ D V r λ by removing D λ ~ i ξ λ ~ j D_{\widetilde{\lambda}}^i\xi_{\widetilde{\lambda}_j} D λ i ξ λ j and D λ ~ i + 1 ξ λ ~ j D_{\widetilde{\lambda}}^{i+1}\xi_{\widetilde{\lambda}_j} D λ i + 1 ξ λ j from V r λ ~ V_r^{\widetilde{\boldsymbol{\lambda}}} V r λ and D V r λ ~ DV_r^{\widetilde{\boldsymbol{\lambda}}} D V r λ , respectively, if r ~ j , i = 0 \widetilde{r}_{j,i}=0 r j , i = 0 for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 and i = 0 , … , m j − 2 i=0,\ldots,m_j-2 i = 0 , … , m j − 2 . Then,
⟨ G − G r , H 1 + H 2 ⟩ H = 0 (17) \left\langle G-G_r,H_1+H_2\right\rangle_{\mathcal{H}}=0 \tag*{(17)} ⟨ G − G r , H 1 + H 2 ⟩ H = 0 (17) for all H 1 ∈ V ~ r λ ~ ⊆ G ξ r H_1 \in\widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}} \subseteq\mathcal{G}_{\xi}^{r} H 1 ∈ V r λ ⊆ G ξ r and H 2 ∈ D V ~ r λ ~ H_2 \in D\widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}} H 2 ∈ D V r λ .
Proof. Notice that G r ∈ V ~ r λ ~ G_r \in\widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}} G r ∈ V r λ and V ~ r λ ~ \widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}} V r λ is a finite dimensional subspace of H \mathcal{H} H . From the Hilbert projection theorem, we know that for any choice of { λ ~ 0 , … , λ ~ k − 1 } ⊆ Ω \{\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{k-1}\} \subseteq\Omega { λ 0 , … , λ k − 1 } ⊆ Ω , G r ∈ V ~ r λ ~ G_r \in\widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}} G r ∈ V r λ is the solution to
min H ∈ V ~ r λ ~ ∥ G − H ∥ H \min_{H\in\widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}}}\|G-H\|_{\mathcal{H}} H ∈ V r λ min ∥ G − H ∥ H if and only if
⟨ G − G r , H ⟩ H = 0 (18) \left\langle G-G_r,H\right\rangle_{\mathcal{H}}=0 \tag*{(18)} ⟨ G − G r , H ⟩ H = 0 (18) for any H ∈ V ~ r λ ~ H \in\widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}} H ∈ V r λ . It therefore suffices to show
⟨ G − G r , H 2 ⟩ H = 0 ( H 2 ∈ D V ~ r λ ~ ) . (19) \left\langle G-G_r,H_2\right\rangle_{\mathcal{H}}=0 \qquad\left(H_2\in D\widetilde{V}_r^{\widetilde{\boldsymbol{\lambda}}}\right). \tag*{(19)} ⟨ G − G r , H 2 ⟩ H = 0 ( H 2 ∈ D V r λ ) . (19) Consider the error functional E ξ r \mathcal{E}_{\xi}^{r} E ξ r given in Problem statement 2.7. Let
J ( λ ~ 0 , r e , λ ~ 0 , i m , … , λ ~ k − 1 , r e , λ ~ k − 1 , i m ) : = ( E ξ r ( G r ) ) 2 = ⟨ G − G r , G − G r ⟩ H , \mathcal{J}(\widetilde{\lambda}_{0,\mathrm{re}},\widetilde{\lambda}_{0,\mathrm{im}},\ldots,\widetilde{\lambda}_{k-1,\mathrm{re}},\widetilde{\lambda}_{k-1,\mathrm{im}}):=\left(\mathcal{E}_{\xi}^{r}(G_r)\right)^2=\left\langle G-G_r,G-G_r\right\rangle_{\mathcal{H}}, J ( λ 0 , re , λ 0 , im , … , λ k − 1 , re , λ k − 1 , im ) := ( E ξ r ( G r ) ) 2 = ⟨ G − G r , G − G r ⟩ H , where λ ~ j = λ ~ j , r e + i ⋅ λ ~ j , i m \widetilde{\lambda}_j=\widetilde{\lambda}_{j,\mathrm{re}}+i\cdot\widetilde{\lambda}_{j,\mathrm{im}} λ j = λ j , re + i ⋅ λ j , im for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 and λ ~ i ≠ λ ~ j \widetilde{\lambda}_i\ne\widetilde{\lambda}_j λ i = λ j for all i , j = 0 , … , k − 1 i,j=0,\ldots,k-1 i , j = 0 , … , k − 1 with i ≠ j i\ne j i = j .
To avoid duplicating identical expressions for the real and imaginary parts, we write ∙ ∈ { r e , i m } \bullet\in\{\mathrm{re},\mathrm{im}\} ∙ ∈ { re , im } and use the notation λ ~ j , ∙ \tilde{\lambda}_{j,\bullet} λ ~ j , ∙ to denote either λ ~ j , r e \tilde{\lambda}_{j,\mathrm{re}} λ ~ j , re or λ ~ j , i m \tilde{\lambda}_{j,\mathrm{im}} λ ~ j , im . Then, since G r G_r G r is a local minimizer of J \mathcal{J} J , we have
∂ J ∂ λ ~ j , ∙ = − 2 Re ⟨ G − G r , ∂ G r ∂ λ ~ j , ∙ ⟩ H = 0. (20) \frac{\partial\mathcal{J}}{\partial\tilde{\lambda}_{j,\bullet}}=-2\operatorname{Re}\left\langle\mathbf{G}-\mathbf{G}_r,\frac{\partial\mathbf{G}_r}{\partial\tilde{\lambda}_{j,\bullet}}\right\rangle_{\mathcal{H}}=0. \tag*{(20)} ∂ λ ~ j , ∙ ∂ J = − 2 Re ⟨ G − G r , ∂ λ ~ j , ∙ ∂ G r ⟩ H = 0. (20)
From this, λ ~ i ≠ λ ~ j \tilde{\lambda}_i\ne\tilde{\lambda}_j λ ~ i = λ ~ j for i ≠ j i\ne j i = j gives
∂ G r ∂ λ ~ j , ∙ = ∑ i = 1 m j r ~ j , i − 1 ∂ i ξ ~ λ ~ j ∂ λ ~ j , ∙ i . (21) \frac{\partial\mathbf{G}_r}{\partial\tilde{\lambda}_{j,\bullet}}=\sum_{i=1}^{m_j}\tilde{r}_{j,i-1}\frac{\partial^i\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\bullet}^i}. \tag*{(21)} ∂ λ ~ j , ∙ ∂ G r = i = 1 ∑ m j r ~ j , i − 1 ∂ λ ~ j , ∙ i ∂ i ξ ~ λ ~ j . (21)
From (20) and (21) , we obtain
∂ J ∂ λ ~ j , ∙ = − 2 Re ⟨ G − G r , ∑ i = 1 m j r ~ j , i − 1 ∂ i ξ ~ λ ~ j ∂ λ ~ j , ∙ i ⟩ H = 0. (22) \frac{\partial\mathcal{J}}{\partial\tilde{\lambda}_{j,\bullet}}=-2\operatorname{Re}\left\langle\mathbf{G}-\mathbf{G}_r,\sum_{i=1}^{m_j}\tilde{r}_{j,i-1}\frac{\partial^i\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\bullet}^i}\right\rangle_{\mathcal{H}}=0. \tag*{(22)} ∂ λ ~ j , ∙ ∂ J = − 2 Re ⟨ G − G r , i = 1 ∑ m j r ~ j , i − 1 ∂ λ ~ j , ∙ i ∂ i ξ ~ λ ~ j ⟩ H = 0. (22)
We note that for i = 1 , … , m j − 1 i=1,\ldots,m_j-1 i = 1 , … , m j − 1 and j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 , and since (18) is true for all H ∈ V ~ λ ~ \mathbf{H}\in\widetilde{\mathcal{V}}_{\tilde{\lambda}} H ∈ V λ ~ ,
⟨ G − G r , ∂ i ξ ~ λ ~ j ∂ λ ~ j , ∙ i ⟩ H = 0. \left\langle\mathbf{G}-\mathbf{G}_r,\frac{\partial^i\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\bullet}^i}\right\rangle_{\mathcal{H}}=0. ⟨ G − G r , ∂ λ ~ j , ∙ i ∂ i ξ ~ λ ~ j ⟩ H = 0.
With this, due to r ~ j , m j − 1 ≠ 0 \tilde{r}_{j,m_j-1}\ne0 r ~ j , m j − 1 = 0 , j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 , (22) , and (20) , we obtain
Re ⟨ G − G r , ∂ m j ξ ~ λ ~ j ∂ λ ~ j , ∙ m j ⟩ H = 0 ( ∙ ∈ { r e , i m } , j = 0 , … , k − 1 ) . (23) \operatorname{Re}\left\langle\mathbf{G}-\mathbf{G}_r,\frac{\partial^{m_j}\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\bullet}^{m_j}}\right\rangle_{\mathcal{H}}=0\qquad(\bullet\in\{\mathrm{re},\mathrm{im}\},\ j=0,\ldots,k-1). \tag*{(23)} Re ⟨ G − G r , ∂ λ ~ j , ∙ m j ∂ m j ξ ~ λ ~ j ⟩ H = 0 ( ∙ ∈ { re , im } , j = 0 , … , k − 1 ) . (23)
Considering D λ ~ i : = 1 2 ( ∂ i ∂ λ ~ j , r e i + i ∂ i ∂ λ ~ j , i m i ) D_{\tilde{\lambda}}^i:=\frac{1}{2}\left(\frac{\partial^i}{\partial\tilde{\lambda}_{j,\mathrm{re}}^i}+i\frac{\partial^i}{\partial\tilde{\lambda}_{j,\mathrm{im}}^i}\right) D λ ~ i := 2 1 ( ∂ λ ~ j , re i ∂ i + i ∂ λ ~ j , im i ∂ i ) , for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 and i = 0 , … , m j i=0,\ldots,m_j i = 0 , … , m j we obtain
⟨ G − G r , D λ ~ m j ξ ~ λ ~ j ⟩ H = 1 2 ( ⟨ G − G r , ∂ m j ξ ~ λ ~ j ∂ λ ~ j , r e m j ⟩ H + i ⟨ G − G r , ∂ m j ξ ~ λ ~ j ∂ λ ~ j , i m m j ⟩ H ) . \left\langle\mathbf{G}-\mathbf{G}_r,D_{\tilde{\lambda}}^{m_j}\tilde{\xi}_{\tilde{\lambda}_j}\right\rangle_{\mathcal{H}}=\frac{1}{2}\left(\left\langle\mathbf{G}-\mathbf{G}_r,\frac{\partial^{m_j}\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\mathrm{re}}^{m_j}}\right\rangle_{\mathcal{H}}+i\left\langle\mathbf{G}-\mathbf{G}_r,\frac{\partial^{m_j}\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\mathrm{im}}^{m_j}}\right\rangle_{\mathcal{H}}\right). ⟨ G − G r , D λ ~ m j ξ ~ λ ~ j ⟩ H = 2 1 ( ⟨ G − G r , ∂ λ ~ j , re m j ∂ m j ξ ~ λ ~ j ⟩ H + i ⟨ G − G r , ∂ λ ~ j , im m j ∂ m j ξ ~ λ ~ j ⟩ H ) .
Finally, due to (23) we have
Re ⟨ G − G r , r ~ j , m j − 1 ∂ m j ξ ~ λ ~ j ∂ λ ~ j , r e m j ⟩ H = 0 , \operatorname{Re}\left\langle\mathbf{G}-\mathbf{G}_r,\tilde{r}_{j,m_j-1}\frac{\partial^{m_j}\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\mathrm{re}}^{m_j}}\right\rangle_{\mathcal{H}}=0, Re ⟨ G − G r , r ~ j , m j − 1 ∂ λ ~ j , re m j ∂ m j ξ ~ λ ~ j ⟩ H = 0 ,
from which
Re ⟨ G − G r , i ⋅ r ~ j , m j − 1 ∂ m j ξ ~ λ ~ j ∂ λ ~ j , r e m j ⟩ H = Im ⟨ G − G r , r ~ j , m j − 1 ∂ m j ξ ~ λ ~ j ∂ λ ~ j , r e m j ⟩ H = 0 \operatorname{Re}\left\langle\mathbf{G}-\mathbf{G}_r,i\cdot\tilde{r}_{j,m_j-1}\frac{\partial^{m_j}\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\mathrm{re}}^{m_j}}\right\rangle_{\mathcal{H}}=\operatorname{Im}\left\langle\mathbf{G}-\mathbf{G}_r,\tilde{r}_{j,m_j-1}\frac{\partial^{m_j}\tilde{\xi}_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\mathrm{re}}^{m_j}}\right\rangle_{\mathcal{H}}=0 Re ⟨ G − G r , i ⋅ r ~ j , m j − 1 ∂ λ ~ j , re m j ∂ m j ξ ~ λ ~ j ⟩ H = Im ⟨ G − G r , r ~ j , m j − 1 ∂ λ ~ j , re m j ∂ m j ξ ~ λ ~ j ⟩ H = 0
follows. By a similar argument, Re ⟨ G − G r , r ~ j , m j − 1 ∂ m j ξ λ ~ j ∂ λ ~ j , i m m j ⟩ H = 0 \operatorname{Re}\left\langle\mathbf{G}-\mathbf{G}_r,\tilde{r}_{j,m_j-1}\frac{\partial^{m_j}\xi_{\tilde{\lambda}_j}}{\partial\tilde{\lambda}_{j,\mathrm{im}}^{m_j}}\right\rangle_{\mathcal{H}}=0 Re ⟨ G − G r , r ~ j , m j − 1 ∂ λ ~ j , im m j ∂ m j ξ λ ~ j ⟩ H = 0 , thus (23) implies
⟨ G − G r , D λ ~ m j ξ λ ~ j ⟩ H = 0 ( j = 0 , … , k − 1 ) . \left\langle\mathbf{G}-\mathbf{G}_r,D_{\tilde{\lambda}}^{m_j}\xi_{\tilde{\lambda}_j}\right\rangle_{\mathcal{H}}=0 \quad(j=0,\ldots,k-1). ⟨ G − G r , D λ ~ m j ξ λ ~ j ⟩ H = 0 ( j = 0 , … , k − 1 ) .
Together with (18) , this equation implies (19) and hence (17) .
In the next theorem, we obtain the RKHS analog of the Meier–Luenberger optimality conditions from Theorem 3.10 .
Theorem 4.4 (Interpolatory optimality conditions in RKHSs with m m m -times differentiable kernels). With the assumptions and the notation of Theorem 4.3 , if
G r = ∑ j = 0 k − 1 ∑ i = 0 m j − 1 r ~ j , i D λ ~ i ξ λ ~ j , \mathbf{G}_r=\sum_{j=0}^{k-1}\sum_{i=0}^{m_j-1}\tilde{r}_{j,i}D_{\tilde{\lambda}}^i\xi_{\tilde{\lambda}_j}, G r = j = 0 ∑ k − 1 i = 0 ∑ m j − 1 r ~ j , i D λ ~ i ξ λ ~ j , where ( λ ~ 0 , … , λ ~ k − 1 ) ∈ Ω k (\tilde{\lambda}_0,\ldots,\tilde{\lambda}_{k-1})\in\Omega^k ( λ ~ 0 , … , λ ~ k − 1 ) ∈ Ω k , ∑ j = 0 k − 1 m j = r \sum_{j=0}^{k-1}m_j=r ∑ j = 0 k − 1 m j = r , 1 ≤ m j < m 1\le m_j<m 1 ≤ m j < m , and r ~ j , m j − 1 ≠ 0 \tilde{r}_{j,m_j-1}\ne0 r ~ j , m j − 1 = 0 for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 is a local minimizer of E ξ r E_\xi^r E ξ r from Definition 2.8, then
G ( i ) ( λ ~ j ) = G r ( i ) ( λ ~ j ) , and G ( m j ) ( λ ~ j ) = G r ( m j ) ( λ ~ j ) (24) \mathbf{G}^{(i)}(\tilde{\lambda}_j)=\mathbf{G}_r^{(i)}(\tilde{\lambda}_j),\quad\text{and}\quad\mathbf{G}^{(m_j)}(\tilde{\lambda}_j)=\mathbf{G}_r^{(m_j)}(\tilde{\lambda}_j) \tag*{(24)} G ( i ) ( λ ~ j ) = G r ( i ) ( λ ~ j ) , and G ( m j ) ( λ ~ j ) = G r ( m j ) ( λ ~ j ) (24) for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 and all i = 0 , … , m j − 1 i=0,\ldots,m_j-1 i = 0 , … , m j − 1 for which r ~ j , i ≠ 0 \tilde{r}_{j,i}\ne0 r ~ j , i = 0 .
Proof. Let j ∈ { 0 , … , k − 1 } j\in\{0,\ldots,k-1\} j ∈ { 0 , … , k − 1 } and i ∈ { 0 , … , m j − 1 } i\in\{0,\ldots,m_j-1\} i ∈ { 0 , … , m j − 1 } be arbitrary. If r ~ j , i ≠ 0 \tilde{r}_{j,i}\ne0 r ~ j , i = 0 , we have D λ ~ i ξ λ ~ j ∈ V ~ λ ~ r D_{\tilde{\lambda}}^i\xi_{\tilde{\lambda}_j}\in\widetilde{V}_{\tilde{\lambda}}^r D λ ~ i ξ λ ~ j ∈ V λ ~ r , and hence, by Theorem 4.3 ,
⟨ G − G r , D λ ~ i ξ λ ~ j ⟩ H = 0. \left\langle\mathbf{G}-\mathbf{G}_r,D_{\tilde{\lambda}}^i\xi_{\tilde{\lambda}_j}\right\rangle_{\mathcal{H}}=0. ⟨ G − G r , D λ ~ i ξ λ ~ j ⟩ H = 0.
Similarly, since r ~ j , m j − 1 ≠ 0 \tilde{r}_{j,m_j-1}\ne0 r ~ j , m j − 1 = 0 , we have D λ ~ m j ξ λ ~ j ∈ D V ~ λ ~ r D_{\tilde{\lambda}}^{m_j}\xi_{\tilde{\lambda}_j}\in D\widetilde{V}_{\tilde{\lambda}}^r D λ ~ m j ξ λ ~ j ∈ D V λ ~ r , which by Theorem 4.3 gives
⟨ G − G r , D λ ~ m j ξ λ ~ j ⟩ H = 0. \left\langle\mathbf{G}-\mathbf{G}_r,D_{\tilde{\lambda}}^{m_j}\xi_{\tilde{\lambda}_j}\right\rangle_{\mathcal{H}}=0. ⟨ G − G r , D λ ~ m j ξ λ ~ j ⟩ H = 0.
The statements now follow from Corollary 3.3, that is,
0 = ⟨ G − G r , D λ ~ i ξ λ ~ j ⟩ H = Cor. 3.3 G ( i ) ( λ ~ j ) − G r ( i ) ( λ ~ j ) 0=\left\langle\mathbf{G}-\mathbf{G}_r,D_{\tilde{\lambda}}^i\xi_{\tilde{\lambda}_j}\right\rangle_{\mathcal{H}}
\overset{\text{Cor. 3.3}}{=}\mathbf{G}^{(i)}(\tilde{\lambda}_j)-\mathbf{G}_r^{(i)}(\tilde{\lambda}_j) 0 = ⟨ G − G r , D λ ~ i ξ λ ~ j ⟩ H = Cor. 3.3 G ( i ) ( λ ~ j ) − G r ( i ) ( λ ~ j )
for j ∈ { 0 , … , k − 1 } j\in\{0,\ldots,k-1\} j ∈ { 0 , … , k − 1 } and i ∈ { 0 , … , m j − 1 } i\in\{0,\ldots,m_j-1\} i ∈ { 0 , … , m j − 1 } with r ~ j , i ≠ 0 \tilde{r}_{j,i}\ne0 r ~ j , i = 0 , and for i = m j i=m_j i = m j .
(a) If we assume λ ~ 0 , … , λ ~ r − 1 \tilde{\lambda}_0,\ldots,\tilde{\lambda}_{r-1} λ ~ 0 , … , λ ~ r − 1 are pairwise distinct (corresponding to m j = 1 m_j=1 m j = 1 for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 ), then the optimality conditions of Theorem 4.3 and Theorem 4.4 simplify to
⟨ G − G r , D λ ~ ξ λ ~ j + H ⟩ H = 0 ( H ∈ span { ξ λ ~ 0 , … , ξ λ ~ r − 1 } ) \left\langle\mathbf{G}-\mathbf{G}_r,D_{\tilde{\lambda}}\xi_{\tilde{\lambda}_j}+\mathbf{H}\right\rangle_{\mathcal{H}}=0\quad\left(\mathbf{H}\in\operatorname{span}\{\xi_{\tilde{\lambda}_0},\ldots,\xi_{\tilde{\lambda}_{r-1}}\}\right) ⟨ G − G r , D λ ~ ξ λ ~ j + H ⟩ H = 0 ( H ∈ span { ξ λ ~ 0 , … , ξ λ ~ r − 1 } )
and
G ( λ ~ j ) = G r ( λ ~ j ) and G ′ ( λ ~ j ) = G r ′ ( λ ~ j ) , \mathbf{G}(\widetilde{\lambda}_j)=\mathbf{G}_r(\widetilde{\lambda}_j) \quad\text{and}\quad\mathbf{G}'(\widetilde{\lambda}_j)=\mathbf{G}'_r(\widetilde{\lambda}_j), G ( λ j ) = G r ( λ j ) and G ′ ( λ j ) = G r ′ ( λ j ) ,
for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 , respectively. The latter conditions match the classical Meier–Luenberger conditions (see Theorem 3.10 ) exactly, when H = H 2 ( C + ) \mathcal{H}=H_2(\mathbb{C}^+) H = H 2 ( C + ) or H = H 2 ( D ) \mathcal{H}=H_2(\mathbb{D}) H = H 2 ( D ) .
(b) Notice that Theorem 4.3 does not make the assumption G ∈ G ξ n \mathbf{G}\in\mathfrak{G}^n_\xi G ∈ G ξ n . Indeed, it suffices that the inner product in (4.3) exists. Consequently, Theorems 4.3 and 4.4 hold for any G ∈ H \mathbf{G}\in\mathcal{H} G ∈ H and r < n r<n r < n as well.
(c) Theorem 4.3 can also be used in cases where G ∈ H \mathbf{G}\in\mathcal{H} G ∈ H , but H \mathcal{H} H is only assumed to be a Hilbert space (not necessarily an RKHS). Indeed, consider an orthonormal system { φ j } j = 0 r − 1 ⊆ H \{\varphi_j\}_{j=0}^{r-1}\subseteq\mathcal{H} { φ j } j = 0 r − 1 ⊆ H and the kernel
ξ ( x , λ ) = ∑ j = 0 r − 1 φ j ( x ) φ j ( λ ) ‾ ( x , λ ∈ Ω ) . (25) \xi(x,\lambda)=\sum_{j=0}^{r-1}\varphi_j(x)\overline{\varphi_j(\lambda)} \qquad(x,\lambda\in\Omega). \tag*{(25)} ξ ( x , λ ) = j = 0 ∑ r − 1 φ j ( x ) φ j ( λ ) ( x , λ ∈ Ω ) . (25)
Then, considering for simplicity { λ ~ j } j = 0 r − 1 ⊂ Ω + r \{\widetilde{\lambda}_j\}_{j=0}^{r-1}\subset\Omega^r_+ { λ j } j = 0 r − 1 ⊂ Ω + r , by Theorems 3.6 and 3.7 ,
V ~ r λ = span { ξ λ ~ 0 , … , ξ λ ~ r − 1 } ⊆ H \widetilde{\mathcal{V}}_r^\lambda=\operatorname{span}\{\xi_{\widetilde{\lambda}_0},\ldots,\xi_{\widetilde{\lambda}_{r-1}}\}\subseteq\mathcal{H} V r λ = span { ξ λ 0 , … , ξ λ r − 1 } ⊆ H
is an r r r -dimensional RKHS. Furthermore, by Lemma 3.8, the inner product of this RKHS matches ⟨ ⋅ , ⋅ ⟩ H \langle\cdot,\cdot\rangle_{\mathcal{H}} ⟨ ⋅ , ⋅ ⟩ H . Considering all possible λ ∈ Ω + r \lambda\in\Omega^r_+ λ ∈ Ω + r , we can define G ξ r ⊆ H \mathfrak{G}^r_\xi\subseteq\mathcal{H} G ξ r ⊆ H according to Definition 2.3 . Notice that the proof of Theorem 4.3 does not rely on G \mathbf{G} G being in an RKHS; rather, it only requires that the inner products
⟨ G − G r , D λ ~ ξ λ ~ ⟩ H \left\langle\mathbf{G}-\mathbf{G}_r,D_{\widetilde{\lambda}}\xi_{\widetilde{\lambda}}\right\rangle_{\mathcal{H}} ⟨ G − G r , D λ ξ λ ⟩ H
exist. Hence, Theorem 4.3 holds for an arbitrary G ∈ H \mathbf{G}\in\mathcal{H} G ∈ H , where G ξ r \mathfrak{G}^r_\xi G ξ r is defined using the kernel (4.11). Note, however, that in this case, choosing different orthonormal bases { φ j } j = 0 r − 1 \{\varphi_j\}_{j=0}^{r-1} { φ j } j = 0 r − 1 will lead to a different reduced model space G ξ r ⊆ H \mathfrak{G}^r_\xi\subseteq\mathcal{H} G ξ r ⊆ H . There is no natural choice, since the full space H \mathcal{H} H does not possess a kernel of its own. The choice of this basis can therefore influence the quality of the ROM. Finally, we note that the RKHS–based interpolatory Meier–Luenberger conditions given in Theorem 4.4 do not follow from this, when H \mathcal{H} H is not an RKHS. Indeed, in this case the reproducing–kernel–based argument in the proof of Theorem 4.4 cannot be used.
Next, we prove the equivalence between Theorem 4.3 and Theorem 4.4 .
Theorem 4.6. Consider the assumptions of Theorem 4.3 . The conditions (4.3) and (4.10) are equivalent.
Proof. Suppose first that (4.3) holds. The proof of Theorem 4.4 shows that (4.10) follows from this. Now suppose we have
G ( i ) ( λ ~ j ) = G r ( i ) ( λ ~ j ) and G ( m j ) ( λ ~ j ) = G r ( m j ) ( λ ~ j ) \mathbf{G}^{(i)}(\widetilde{\lambda}_j)=\mathbf{G}^{(i)}_r(\widetilde{\lambda}_j) \quad\text{and}\quad\mathbf{G}^{(m_j)}(\widetilde{\lambda}_j)=\mathbf{G}^{(m_j)}_r(\widetilde{\lambda}_j) G ( i ) ( λ j ) = G r ( i ) ( λ j ) and G ( m j ) ( λ j ) = G r ( m j ) ( λ j )
for j = 0 , … , k − 1 j = 0,\ldots,k-1 j = 0 , … , k − 1 and i = 0 , … , m j − 1 i = 0,\ldots,m_j-1 i = 0 , … , m j − 1 for which r ~ j , i ≠ 0 \widetilde{r}_{j,i} \ne0 r j , i = 0 . For i < m j i < m_j i < m j , this yields
0 = G ( i ) ( λ ~ j ) − G r ( i ) ( λ ~ j ) = Cor. 3.3 ⟨ G − G r , D λ ~ i ξ λ ~ j ⟩ H , 0 = G^{(i)}(\widetilde{\lambda}_j) - G_r^{(i)}(\widetilde{\lambda}_j) \mathrel{\overset{\text{Cor. 3.3}}{=}} \left\langle G-G_r,D_{\widetilde{\lambda}}^i \xi_{\widetilde{\lambda}_j}\right\rangle_{\mathcal{H}}, 0 = G ( i ) ( λ j ) − G r ( i ) ( λ j ) = Cor. 3.3 ⟨ G − G r , D λ i ξ λ j ⟩ H ,
hence G − G r ⊥ V ~ λ ~ G-G_r \perp\widetilde{V}_{\widetilde{\lambda}} G − G r ⊥ V λ . Similarly, for i = m j i=m_j i = m j , we get
0 = G ( m j ) ( λ ~ j ) − G r ( m j ) ( λ ~ j ) = Cor. 3.3 ⟨ G − G r , D λ ~ m j ξ λ ~ j ⟩ H . 0 = G^{(m_j)}(\widetilde{\lambda}_j) - G_r^{(m_j)}(\widetilde{\lambda}_j) \mathrel{\overset{\text{Cor. 3.3}}{=}} \left\langle G-G_r,D_{\widetilde{\lambda}}^{m_j}\xi_{\widetilde{\lambda}_j}\right\rangle_{\mathcal{H}}. 0 = G ( m j ) ( λ j ) − G r ( m j ) ( λ j ) = Cor. 3.3 ⟨ G − G r , D λ m j ξ λ j ⟩ H .
This additional observation ensures G − G r ⊥ D V ~ λ ~ G-G_r \perp D\widetilde{V}_{\widetilde{\lambda}} G − G r ⊥ D V λ , from which (17) follows.
The following lemma clarifies that the classical H 2 H_2 H 2 optimality conditions from Theorem 3.11 and Theorem 3.10 are special cases of the newly introduced theorems.
Lemma 4.7 (Optimality conditions in H 2 ( C + ) H_2(\mathbb{C}^+) H 2 ( C + ) are special cases of Theorem 4.3 and Theorem 4.4 ). Let Ω = C + \Omega=\mathbb{C}^+ Ω = C + , H = H 2 ( C + ) \mathcal{H}=H_2(\mathbb{C}^+) H = H 2 ( C + ) , G ∈ G n G\in\mathcal{G}_n G ∈ G n be a FOM, and G r ∈ G r G_r\in\mathcal{G}_r G r ∈ G r be the H 2 ( C + ) H_2(\mathbb{C}^+) H 2 ( C + ) –norm–optimal ROM of order r r r to G G G given by
G r = ∑ j = 0 r − 1 r ~ j ζ λ ~ j , G_r=\sum_{j=0}^{r-1}\widetilde{r}_j\zeta_{\widetilde{\lambda}_j}, G r = j = 0 ∑ r − 1 r j ζ λ j , where we assume λ ~ ∈ Ω r + \widetilde{\lambda}\in\Omega_r^+ λ ∈ Ω r + and ζ \zeta ζ is the Cauchy kernel from (8) . Then, Theorem 4.3 and Theorem 4.4 apply to G r G_r G r and are equivalent to Theorem 3.11 and Theorem 3.10 .
Proof. The proof of the statement is given in Section A.2.
Remark 4.8. (a) A similar result with identical reasoning can be given if H = H 2 ( D ) \mathcal{H}=H_2(\mathbb{D}) H = H 2 ( D ) .
(b) Although no further formal proofs will be given, we note that recent H 2 H_2 H 2 –optimal MOR results given by Borghi and Breiten in [11 ] are also likely special cases of the proposed theorems. Indeed, it is not difficult to see that the H 2 H_2 H 2 spaces defined over simply connected sets in these works are also RKHSs, which, at the very least, makes Theorem 4.3 and Theorem 4.4 applicable to assess the optimality of ROMs.
Another interesting necessary condition of optimality can be obtained if we consider RKHSs that contain polynomial–like objects. In order to introduce these properly, we consider multipliers on RKHSs.
Definition 4.9 (Multipliers of H \mathcal{H} H , invariance). Let H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C . For some G ∈ H G\in\mathcal{H} G ∈ H , let
D G : = { H ∈ H : G ⋅ H ∈ H } ⊆ H , D_G:=\{H\in\mathcal{H}:G\cdot H\in\mathcal{H}\}\subseteq\mathcal{H}, D G := { H ∈ H : G ⋅ H ∈ H } ⊆ H , where G ⋅ H G\cdot H G ⋅ H denotes pointwise multiplication. We call the linear operator M G : D G → H M_G:D_G\to\mathcal{H} M G : D G → H a multiplier of H \mathcal{H} H associated with G G G if
M G [ H ] : = G ⋅ H ( H ∈ H ) . M_G[H]:=G\cdot H \quad(H\in\mathcal{H}). M G [ H ] := G ⋅ H ( H ∈ H ) . Additionally, when D G = H D_G=\mathcal{H} D G = H , we say that H \mathcal{H} H is M G M_G M G –invariant.
By the closed graph theorem, multipliers on RKHSs are always bounded (see, e.g., Theorem 5.21 in [52 ] ). Hence, we obtain a convenient notation for RKHSs that contain polynomial-like functions: we say that they are M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω –invariant, where i d Ω \mathrm{id}_{\Omega} id Ω is the identity operator on Ω \Omega Ω , and therefore,
M i d Ω [ H ] ( λ ) = λ H ( λ ) ( λ ∈ Ω ) . \mathcal{M}_{\mathrm{id}_{\Omega}}[\mathbf{H}](\lambda)=\lambda\mathbf{H}(\lambda)\qquad(\lambda\in\Omega). M id Ω [ H ] ( λ ) = λ H ( λ ) ( λ ∈ Ω ) .
With these concepts, we are now ready to state the next optimality condition. It shows that in M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω –invariant RKHSs, we can describe the optimality of a ROM with a rational-like argument, similar to the results obtained in classical MOR literature for H 2 H_2 H 2 spaces [31 ] .
Theorem 4.10 (A necessary condition for norm-optimal ROMs in M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω –invariant RKHSs). Suppose the assumptions of Theorem 4.3 are satisfied. In addition, suppose H \mathcal{H} H is M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω –invariant as in Definition 4.9 . Let ( λ 0 , … , λ n − 1 ) ∈ Ω ≠ n (\lambda_0,\ldots,\lambda_{n-1})\in\Omega_{\neq}^{n} ( λ 0 , … , λ n − 1 ) ∈ Ω = n and assume we have given the
G = ∑ k = 0 n − 1 r k ξ λ k ∈ G ξ n , \mathbf{G}=\sum_{k=0}^{n-1}r_k\xi_{\lambda_k}\in\mathcal{G}_{\xi}^{n}, G = k = 0 ∑ n − 1 r k ξ λ k ∈ G ξ n , Then, if ( λ ~ 0 , … , λ ~ r − 1 ) ∈ Ω ≠ r (\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{r-1})\in\Omega_{\neq}^{r} ( λ 0 , … , λ r − 1 ) ∈ Ω = r defines an H \mathcal{H} H –norm-optimal ROM
G r = ∑ j = 0 r − 1 r ~ j ξ λ ~ j ∈ G ξ r , \mathbf{G}_r=\sum_{j=0}^{r-1}\widetilde{r}_j\xi_{\widetilde{\lambda}_j}\in\mathcal{G}_{\xi}^{r}, G r = j = 0 ∑ r − 1 r j ξ λ j ∈ G ξ r , then
⟨ G − G r , ( M i d Ω ∗ − λ ~ ‾ j i d H ) M i d Ω ∗ ∣ Y ~ r + [ ξ ~ λ ~ j ] ⟩ H = 0 ( j = 0 , … , r − 1 ) . \left\langle\mathbf{G}-\mathbf{G}_r,\left(\mathcal{M}_{\mathrm{id}_{\Omega}}^*-\overline{\widetilde{\lambda}}_j\,\mathrm{id}_{\mathcal{H}}\right)_{\left.\vphantom{\mathcal{M}_{\mathrm{id}_{\Omega}}^*}\right|_{\widetilde{\mathcal{Y}}_r}}^{+}\left[\widetilde{\xi}_{\widetilde{\lambda}_j}\right]\right\rangle_{\mathcal{H}}=0\qquad(j=0,\ldots,r-1). ⟨ G − G r , ( M id Ω ∗ − λ j id H ) M id Ω ∗ Y r + [ ξ λ j ] ⟩ H = 0 ( j = 0 , … , r − 1 ) . Here, M i d Ω ∗ \mathcal{M}_{\mathrm{id}_{\Omega}}^* M id Ω ∗ denotes the adjoint of the multiplier M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω and ( M i d Ω ∗ − λ ~ ‾ j i d H ) M i d Ω ∗ ∣ Y ~ r + \left(\mathcal{M}_{\mathrm{id}_{\Omega}}^*-\overline{\widetilde{\lambda}}_j\,\mathrm{id}_{\mathcal{H}}\right)_{\left.\vphantom{\mathcal{M}_{\mathrm{id}_{\Omega}}^*}\right|_{\widetilde{\mathcal{Y}}_r}}^{+} ( M id Ω ∗ − λ j id H ) M id Ω ∗ Y r + denotes the Moore–Penrose pseudoinverse of the operator M i d Ω ∗ − λ ~ ‾ j i d H \mathcal{M}_{\mathrm{id}_{\Omega}}^*-\overline{\widetilde{\lambda}}_j\,\mathrm{id}_{\mathcal{H}} M id Ω ∗ − λ j id H restricted to
Y ~ r : = V ~ r ⊕ D V ~ r = span { ξ ~ λ ~ 0 , D λ ξ ~ λ ~ 0 , … , ξ ~ λ ~ r − 1 , D λ ξ ~ λ ~ r − 1 } ⊆ H . \widetilde{\mathcal{Y}}_r:=\widetilde{\mathcal{V}}_r\oplus D\widetilde{\mathcal{V}}_r=\operatorname{span}\left\{\widetilde{\xi}_{\widetilde{\lambda}_0},D_{\lambda}\widetilde{\xi}_{\widetilde{\lambda}_0},\ldots,\widetilde{\xi}_{\widetilde{\lambda}_{r-1}},D_{\lambda}\widetilde{\xi}_{\widetilde{\lambda}_{r-1}}\right\}\subseteq\mathcal{H}. Y r := V r ⊕ D V r = span { ξ λ 0 , D λ ξ λ 0 , … , ξ λ r − 1 , D λ ξ λ r − 1 } ⊆ H .
Proof. Since λ ↦ ξ λ \lambda\mapsto\xi_\lambda λ ↦ ξ λ is differentiable, for all H ∈ H \mathbf{H}\in\mathcal{H} H ∈ H , Theorem 3.2 gives
⟨ H , M i d Ω ∗ [ D λ ξ λ ] ⟩ H = ⟨ M i d Ω [ H ] , D λ ξ λ ⟩ H = ( M i d Ω [ H ] ′ ) ( λ ) = H ( λ ) + λ H ′ ( λ ) . (26) \left\langle\mathbf{H},\mathcal{M}_{\mathrm{id}_{\Omega}}^*\left[D_\lambda\xi_\lambda\right]\right\rangle_{\mathcal{H}}=\left\langle\mathcal{M}_{\mathrm{id}_{\Omega}}[\mathbf{H}],D_\lambda\xi_\lambda\right\rangle_{\mathcal{H}}=\left(\mathcal{M}_{\mathrm{id}_{\Omega}}[\mathbf{H}]'\right)(\lambda)=\mathbf{H}(\lambda)+\lambda\mathbf{H}'(\lambda). \tag*{(26)} ⟨ H , M id Ω ∗ [ D λ ξ λ ] ⟩ H = ⟨ M id Ω [ H ] , D λ ξ λ ⟩ H = ( M id Ω [ H ] ′ ) ( λ ) = H ( λ ) + λ H ′ ( λ ) . (26)
Notice, however, that H ( λ ) + λ H ′ ( λ ) \mathbf{H}(\lambda)+\lambda\mathbf{H}'(\lambda) H ( λ ) + λ H ′ ( λ ) can be written as
H ( λ ) + λ H ′ ( λ ) = ⟨ H , ξ λ ⟩ H + λ ⟨ H , D λ ξ λ ⟩ H = ⟨ H , ξ λ + λ ‾ D λ ξ λ ⟩ H . (27) \mathbf{H}(\lambda)+\lambda\mathbf{H}'(\lambda)=\langle\mathbf{H},\xi_\lambda\rangle_{\mathcal{H}}+\lambda\left\langle\mathbf{H},D_\lambda\xi_\lambda\right\rangle_{\mathcal{H}}=\left\langle\mathbf{H},\xi_\lambda+\overline{\lambda}D_\lambda\xi_\lambda\right\rangle_{\mathcal{H}}. \tag*{(27)} H ( λ ) + λ H ′ ( λ ) = ⟨ H , ξ λ ⟩ H + λ ⟨ H , D λ ξ λ ⟩ H = ⟨ H , ξ λ + λ D λ ξ λ ⟩ H . (27)
Equating the first equation in (26) and the last equation in (27) gives
⟨ H , M i d Ω ∗ [ D λ ξ λ ] ⟩ H = ⟨ H , ξ λ + λ ‾ D λ ξ λ ⟩ H . \left\langle\mathbf{H},\mathcal{M}_{\mathrm{id}_{\Omega}}^*\left[D_\lambda\xi_\lambda\right]\right\rangle_{\mathcal{H}}=\left\langle\mathbf{H},\xi_\lambda+\overline{\lambda}D_\lambda\xi_\lambda\right\rangle_{\mathcal{H}}. ⟨ H , M id Ω ∗ [ D λ ξ λ ] ⟩ H = ⟨ H , ξ λ + λ D λ ξ λ ⟩ H .
which holds for all H ∈ H \mathbf{H} \in\mathcal{H} H ∈ H , and thus,
M i d H ∗ [ D λ ξ λ ] = ξ λ + λ ˉ D λ ξ λ . (28) \mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}}\left[D_{\lambda}\xi_{\lambda}\right] = \xi_{\lambda} + \bar{\lambda}D_{\lambda}\xi_{\lambda}. \tag*{(28)} M id H ∗ [ D λ ξ λ ] = ξ λ + λ ˉ D λ ξ λ . (28)
Rearranging this gives
ξ λ = M i d H ∗ [ D λ ξ λ ] − λ ˉ D λ ξ λ = ( M i d H ∗ − λ ˉ i d H ) [ D λ ξ λ ] . (29) \xi_{\lambda} = \mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}}\left[D_{\lambda}\xi_{\lambda}\right] - \bar{\lambda}D_{\lambda}\xi_{\lambda} = \left(\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}} - \bar{\lambda}\,\mathrm{id}_{\mathcal{H}}\right)\left[D_{\lambda}\xi_{\lambda}\right]. \tag*{(29)} ξ λ = M id H ∗ [ D λ ξ λ ] − λ ˉ D λ ξ λ = ( M id H ∗ − λ ˉ id H ) [ D λ ξ λ ] . (29)
Inverting the operator on the right-hand side is not possible in general, since for μ = λ \mu= \lambda μ = λ , we have ( M i d H ∗ − λ ˉ i d H ) [ ξ μ ] = 0 \left(\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}} - \bar{\lambda}\,\mathrm{id}_{\mathcal{H}}\right)\left[\xi_{\mu}\right] = 0 ( M id H ∗ − λ ˉ id H ) [ ξ μ ] = 0 . This follows from the observation that for any H ∈ H \mathbf{H} \in\mathcal{H} H ∈ H , we have
⟨ H , M i d H ∗ [ ξ λ ] ⟩ H = ⟨ M i d H [ H ] , ξ λ ⟩ H = λ H ( λ ) = λ ⟨ H , ξ λ ⟩ H = ⟨ H , λ ˉ ξ λ ⟩ H . \left\langle\mathbf{H},\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}}\left[\xi_{\lambda}\right]\right\rangle_{\mathcal{H}} = \left\langle\mathcal{M}_{\mathrm{id}_{\mathcal{H}}}\left[\mathbf{H}\right],\xi_{\lambda}\right\rangle_{\mathcal{H}} = \lambda\mathbf{H}(\lambda) = \lambda\left\langle\mathbf{H},\xi_{\lambda}\right\rangle_{\mathcal{H}} = \left\langle\mathbf{H},\bar{\lambda}\xi_{\lambda}\right\rangle_{\mathcal{H}}. ⟨ H , M id H ∗ [ ξ λ ] ⟩ H = ⟨ M id H [ H ] , ξ λ ⟩ H = λ H ( λ ) = λ ⟨ H , ξ λ ⟩ H = ⟨ H , λ ˉ ξ λ ⟩ H .
The choice of H \mathbf{H} H here is arbitrary; consequently,
M i d H ∗ [ ξ λ ] = λ ˉ ξ λ . (30) \mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}}\left[\xi_{\lambda}\right] = \bar{\lambda}\xi_{\lambda}. \tag*{(30)} M id H ∗ [ ξ λ ] = λ ˉ ξ λ . (30)
that is, the kernel slices ξ λ \xi_{\lambda} ξ λ are eigenfunctions of M i d H ∗ \mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}} M id H ∗ .
Due to (28) and (30) , we know the effect of M i d H ∗ \mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}} M id H ∗ on the basis functions of Y λ ~ \mathcal{Y}^{\widetilde{\lambda}} Y λ . In order to prove the claim, we show that the Moore–Penrose pseudoinverse of the restricted operator ( M i d H ∗ − λ ~ j i d H ) ∣ Y r λ ~ \left(\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}} - \widetilde{\lambda}_{j}\,\mathrm{id}_{\mathcal{H}}\right)\big|_{\mathcal{Y}^{\widetilde{\lambda}}_{r}} ( M id H ∗ − λ j id H ) Y r λ satisfies
( M i d H ∗ − λ ~ j i d H ) ∣ Y r λ ~ + ( M i d H ∗ − λ ~ j i d H ) ∣ Y r λ ~ [ D λ ξ λ ~ j ] = D λ ξ λ ~ j ( j = 0 , … , r − 1 ) , \left(\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}} - \widetilde{\lambda}_{j}\,\mathrm{id}_{\mathcal{H}}\right)\big|_{\mathcal{Y}^{\widetilde{\lambda}}_{r}}^{+}\left(\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}} - \widetilde{\lambda}_{j}\,\mathrm{id}_{\mathcal{H}}\right)\big|_{\mathcal{Y}^{\widetilde{\lambda}}_{r}}\left[D_{\lambda}\xi_{\widetilde{\lambda}_{j}}\right] = D_{\lambda}\xi_{\widetilde{\lambda}_{j}}\qquad(j=0,\ldots,r-1), ( M id H ∗ − λ j id H ) Y r λ + ( M id H ∗ − λ j id H ) Y r λ [ D λ ξ λ j ] = D λ ξ λ j ( j = 0 , … , r − 1 ) ,
for any D λ ξ λ ~ j ∈ Y r λ ~ D_{\lambda}\xi_{\widetilde{\lambda}_{j}} \in\mathcal{Y}^{\widetilde{\lambda}}_{r} D λ ξ λ j ∈ Y r λ . Let
v 2 j = D λ ξ λ ~ j , v 2 j + 1 = ξ λ ~ j ( j = 0 , … , r − 1 ) . v_{2j}=D_{\lambda}\xi_{\widetilde{\lambda}_{j}},\qquad v_{2j+1}=\xi_{\widetilde{\lambda}_{j}}\qquad(j=0,\ldots,r-1). v 2 j = D λ ξ λ j , v 2 j + 1 = ξ λ j ( j = 0 , … , r − 1 ) .
Then, Y r λ ~ = span { v 0 , … , v 2 r − 1 } \mathcal{Y}^{\widetilde{\lambda}}_{r}=\operatorname{span}\{v_{0},\ldots,v_{2r-1}\} Y r λ = span { v 0 , … , v 2 r − 1 } .
From (28) and (30) , we obtain the conditions
( M i d H ∗ − λ ~ j i d H ) [ v 2 j ] = ξ λ ~ j , ( M i d H ∗ − λ ~ j i d H ) [ v 2 j + 1 ] = 0 \left(\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}}-\widetilde{\lambda}_{j}\,\mathrm{id}_{\mathcal{H}}\right)\left[v_{2j}\right]=\xi_{\widetilde{\lambda}_{j}},\qquad\left(\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}}-\widetilde{\lambda}_{j}\,\mathrm{id}_{\mathcal{H}}\right)\left[v_{2j+1}\right]=0 ( M id H ∗ − λ j id H ) [ v 2 j ] = ξ λ j , ( M id H ∗ − λ j id H ) [ v 2 j + 1 ] = 0
for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 . Thus, ( M i d H ∗ − λ ~ j i d H ) ∣ Y r λ ~ \left(\mathcal{M}^{*}_{\mathrm{id}_{\mathcal{H}}}-\widetilde{\lambda}_{j}\,\mathrm{id}_{\mathcal{H}}\right)\big|_{\mathcal{Y}^{\widetilde{\lambda}}_{r}} ( M id H ∗ − λ j id H ) Y r λ can be represented by the C 2 r × 2 r \mathbb{C}^{2r\times2r} C 2 r × 2 r matrix
A = I r ⊗ [ 0 0 1 0 ] . (31) \mathbf{A}=\mathbf{I}_{r}\otimes
\begin{bmatrix}
0 & 0\\
1 & 0
\end{bmatrix}. \tag*{(31)} A = I r ⊗ [ 0 1 0 0 ] . (31)
where ⊗ \otimes ⊗ denotes the Kronecker product and I r \mathbf{I}_{r} I r is identity matrix in C r × r \mathbb{C}^{r\times r} C r × r . Letting e k ∈ C 2 r \mathbf{e}_{k}\in\mathbb{C}^{2r} e k ∈ C 2 r denote the k k k th unit vector, for k = 0 , … , 2 r − 1 k=0,\ldots,2r-1 k = 0 , … , 2 r − 1 , A \mathbf{A} A satisfies
A e k = { e k + 1 , if k m o d 2 = 0 , 0 , otherwise. (32) \mathbf{A}\mathbf{e}_{k}=
\begin{cases}
\mathbf{e}_{k+1}, & \text{if } k\bmod2=0,\\
0, & \text{otherwise.}
\end{cases} \tag*{(32)} A e k = { e k + 1 , 0 , if k mod 2 = 0 , otherwise. (32)
While A is singular, its Moore–Penrose inverse A + A^{+} A + exists. We show
e 2 j = A + A e 2 j ( j = 0 , … , r − 1 ) . (33) \mathbf{e}_{2j}=A^{+}A\mathbf{e}_{2j}\qquad(j=0,\ldots,r-1). \tag*{(33)} e 2 j = A + A e 2 j ( j = 0 , … , r − 1 ) . (33)
Here, A + A A^{+}A A + A is the orthogonal projection operator that maps vectors onto the row space of A A A , which is the image of A ∗ A^{*} A ∗ . Since any vector x ∈ C 2 r \mathbf{x}\in\mathbb{C}^{2r} x ∈ C 2 r satisfies
x = x R o w + x N u l l , \mathbf{x}=\mathbf{x}_{\mathrm{Row}}+\mathbf{x}_{\mathrm{Null}}, x = x Row + x Null ,
where x R o w \mathbf{x}_{\mathrm{Row}} x Row and x N u l l \mathbf{x}_{\mathrm{Null}} x Null belong to the row and null spaces of A A A , respectively, for (33) to hold, it suffices to show that e 2 j \mathbf{e}_{2j} e 2 j is in the row space of A A A , equivalently, orthogonal to its null space. This is enough, because for any x R o w \mathbf{x}_{\mathrm{Row}} x Row in the row space, we have x R o w = A + A x R o w \mathbf{x}_{\mathrm{Row}}=A^{+}A\mathbf{x}_{\mathrm{Row}} x Row = A + A x Row , since A + A A^{+}A A + A is an orthogonal projector. Notice that by (31) , the rank of A A A is exactly r r r . Therefore, by the rank–nullity theorem, the dimension of the null space of A A A is also r r r . Note, however, that by (32) , e 2 j + 1 \mathbf{e}_{2j+1} e 2 j + 1 , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 are exactly r r r basis vectors spanning the null space of A A A . Hence, (33) is equivalent to
⟨ e 2 j , e 2 k + 1 ⟩ C 2 r = 0 ( j , k = 0 , … , r − 1 ) , \left\langle\mathbf{e}_{2j},\mathbf{e}_{2k+1}\right\rangle_{\mathbb{C}^{2r}}=0\qquad(j,k=0,\ldots,r-1), ⟨ e 2 j , e 2 k + 1 ⟩ C 2 r = 0 ( j , k = 0 , … , r − 1 ) ,
which holds due to the orthogonality of e k \mathbf{e}_{k} e k for k = 0 , … , 2 r − 1 k=0,\ldots,2r-1 k = 0 , … , 2 r − 1 .
We recall that A A A is the matrix representation of ( M i d Ω ∗ − λ ~ j i d H ) ∣ Y r \left.(\mathcal{M}^{*}_{\mathrm{id}_{\Omega}}-\widetilde{\lambda}_{j}\mathrm{id}_{\mathcal{H}})\right|_{\mathcal{Y}_{r}} ( M id Ω ∗ − λ j id H ) Y r . The Moore–Penrose pseudoinverse of this restricted operator is the operator whose matrix representation is A + A^{+} A + in the respective bases. In this case, by (29) and (33) , we have
D λ ~ j ξ = ( M i d Ω ∗ − λ ~ j i d H ) ∣ Y r + [ ξ ~ j ] ( j = 0 , … , r − 1 ) D_{\widetilde{\lambda}_{j}}^{\xi}=\left.(\mathcal{M}^{*}_{\mathrm{id}_{\Omega}}-\widetilde{\lambda}_{j}\mathrm{id}_{\mathcal{H}})\right|_{\mathcal{Y}_{r}}^{+}\left[\widetilde{\xi}_{j}\right]\qquad(j=0,\ldots,r-1) D λ j ξ = ( M id Ω ∗ − λ j id H ) Y r + [ ξ j ] ( j = 0 , … , r − 1 )
and our claim follows from Theorem 4.3 . ■
Remark 4.11. Theorem 4.10 is not directly used by the MOR algorithms or experiments presented later in this paper; we include it as a self–contained result of independent interest. It shows that, in M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω –invariant RKHSs, the necessary optimality condition retains a rational–like structure, similar in spirit to the classical orthogonality conditions familiar from the MOR literature, see Theorem 3.11 .
Finally, we formulate a necessary condition of H \mathcal{H} H optimality in terms of an equilibrium function. This formulation will be especially useful for constructing an iterative method in § 7 to obtain norm–optimal ROMs.
Definition 4.12 (Equilibrium function). Let H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C of dimension at least n n n with kernel ξ \xi ξ that satisfies the assumptions made in Theorem 4.3 , and let G ∈ G ξ n G\in\mathfrak{G}_{\xi}^{n} G ∈ G ξ n . Assuming r ≤ n r\le n r ≤ n , let μ = ( μ 0 , … , μ r − 1 ) ∈ Ω r \boldsymbol{\mu}=(\mu_{0},\ldots,\mu_{r-1})\in\Omega^{r} μ = ( μ 0 , … , μ r − 1 ) ∈ Ω r and let m μ i ≤ m m_{\mu_i}\le m m μ i ≤ m count the repetitions of μ i \mu_i μ i in μ \boldsymbol{\mu} μ for i = 0 , … , r − 1 i=0,\ldots,r-1 i = 0 , … , r − 1 . Furthermore, let k ≤ r k\le r k ≤ r and ( λ ~ 0 , … , λ ~ k − 1 ) ∈ Ω k (\widetilde{\lambda}_{0},\ldots,\widetilde{\lambda}_{k-1})\in\Omega^{k} ( λ 0 , … , λ k − 1 ) ∈ Ω k be the k k k pairwise distinct components of μ ∈ Ω r \boldsymbol{\mu}\in\Omega^{r} μ ∈ Ω r , i.e., assume there exist r 0 , … , r k − 1 ∈ { 0 , … , r − 1 } r_{0},\ldots,r_{k-1}\in\{0,\ldots,r-1\} r 0 , … , r k − 1 ∈ { 0 , … , r − 1 } , for which λ ~ j = μ r j \widetilde{\lambda}_{j}=\mu_{r_{j}} λ j = μ r j for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 .
The equilibrium function associated with G \mathbf{G} G is defined as D : Ω r → C r \mathcal{D}:\Omega^{r}\to\mathbb{C}^{r} D : Ω r → C r with
[ D ( μ ) ] i : = G ( m μ i ) ( μ i ) − ( P V ~ r λ [ G ] ) ( m μ i ) ( μ i ) ( i = 0 , … , r − 1 ) , (34) [\mathcal{D}(\boldsymbol{\mu})]_i := \mathbf{G}^{(m_{\mu_i})}(\mu_i)-\left(\mathcal{P}_{\widetilde{V}_{r}^{\lambda}}[\mathbf{G}]\right)^{(m_{\mu_i})}(\mu_i)\qquad(i=0,\ldots,r-1), \tag*{(34)} [ D ( μ ) ] i := G ( m μ i ) ( μ i ) − ( P V r λ [ G ] ) ( m μ i ) ( μ i ) ( i = 0 , … , r − 1 ) , (34)
where P V ~ r λ \mathcal{P}_{\widetilde{V}_{r}^{\lambda}} P V r λ denotes the orthogonal projector onto the r r r -dimensional subspace V ~ r λ ⊆ H \widetilde{V}_{r}^{\lambda}\subseteq\mathcal{H} V r λ ⊆ H as in (15) with m j : = m μ r j m_j:=m_{\mu_{r_j}} m j := m μ r j , j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 .
Using Definition 4.12, we arrive at the following optimality condition.
Theorem 4.13. Consider the assumptions made in Theorem 4.3 and let D : Ω r → C r \mathcal{D}:\Omega^{r}\to\mathbb{C}^{r} D : Ω r → C r be the equilibrium function as given in Definition 4.12. If
D ( μ ) = 0 \mathcal{D}(\boldsymbol{\mu})=0 D ( μ ) = 0 for some μ ∈ Ω r \boldsymbol{\mu}\in\Omega^{r} μ ∈ Ω r , then (24) and (17) are satisfied by the ROM
G r : = P V ~ r λ [ G ] . \mathbf{G}_{r}:=\mathcal{P}_{\widetilde{V}_{r}^{\lambda}}[\mathbf{G}]. G r := P V r λ [ G ] .
Proof. By our assumptions G r = P V ~ r λ [ G ] \mathbf{G}_{r}=\mathcal{P}_{\widetilde{V}_{r}^{\lambda}}[\mathbf{G}] G r = P V r λ [ G ] , hence G − G r ⊥ V ~ r λ \mathbf{G}-\mathbf{G}_{r}\perp\widetilde{V}_{r}^{\lambda} G − G r ⊥ V r λ holds. Consequently, for any H ∈ V ~ r λ \mathbf{H}\in\widetilde{V}_{r}^{\lambda} H ∈ V r λ , we have ⟨ G − G r , H ⟩ H = 0 \langle\mathbf{G}-\mathbf{G}_{r},\mathbf{H}\rangle_{\mathcal{H}}=0 ⟨ G − G r , H ⟩ H = 0 . From this, we obtain
0 = ⟨ G − G r , D λ ~ i ξ ~ λ ~ j ⟩ H = Cor. 3.3 G ( i ) ( λ ~ j ) − G r ( i ) ( λ ~ j ) 0=\left\langle\mathbf{G}-\mathbf{G}_{r},D_{\widetilde{\lambda}}^{i}\widetilde{\xi}_{\widetilde{\lambda}_{j}}\right\rangle_{\mathcal{H}}\overset{\text{Cor. 3.3}}{=}\mathbf{G}^{(i)}(\widetilde{\lambda}_{j})-\mathbf{G}_{r}^{(i)}(\widetilde{\lambda}_{j}) 0 = ⟨ G − G r , D λ i ξ λ j ⟩ H = Cor. 3.3 G ( i ) ( λ j ) − G r ( i ) ( λ j )
for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 and i = 0 , … , m j − 1 i=0,\ldots,m_j-1 i = 0 , … , m j − 1 .
By our assumptions that D ( μ ) = 0 \mathcal{D}(\boldsymbol{\mu})=0 D ( μ ) = 0 and Definition 4.12, this means that
G ( m j ) ( λ ~ j ) − G r ( m j ) ( λ ~ j ) = 0 \mathbf{G}^{(m_j)}(\widetilde{\lambda}_{j})-\mathbf{G}_{r}^{(m_j)}(\widetilde{\lambda}_{j})=0 G ( m j ) ( λ j ) − G r ( m j ) ( λ j ) = 0
for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 . This shows that (24) is satisfied. To show (17) , we employ Corollary 3.3 again to find
0 = G ( m j ) ( λ ~ j ) − G r ( m j ) ( λ ~ j ) = Cor. 3.3 ⟨ G − G r , D λ ~ m j ξ ~ λ ~ j ⟩ H , 0=\mathbf{G}^{(m_j)}(\widetilde{\lambda}_{j})-\mathbf{G}_{r}^{(m_j)}(\widetilde{\lambda}_{j})\overset{\text{Cor. 3.3}}{=}\left\langle\mathbf{G}-\mathbf{G}_{r},D_{\widetilde{\lambda}}^{m_j}\widetilde{\xi}_{\widetilde{\lambda}_{j}}\right\rangle_{\mathcal{H}}, 0 = G ( m j ) ( λ j ) − G r ( m j ) ( λ j ) = Cor. 3.3 ⟨ G − G r , D λ m j ξ λ j ⟩ H ,
which implies G − G r ⊥ D V ~ r λ \mathbf{G}-\mathbf{G}_{r}\perp D\widetilde{V}_{r}^{\lambda} G − G r ⊥ D V r λ with D V ~ r λ D\widetilde{V}_{r}^{\lambda} D V r λ as in (16) . ■
Note that in Theorem 4.13 , we only showed that finding a zero of the equilibrium function identifies a ROM , which satisfies the optimality conditions of Theorem 4.3 and Theorem 4.4 . In applications, we can often assume that all kernel centers defining the ROM are distinct, i.e., ( μ 0 , … , μ r − 1 ) = ( λ ~ 0 , … , λ ~ r − 1 ) ∈ Ω ≠ r (\mu_0,\ldots,\mu_{r-1})=(\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{r-1})\in\Omega^{r}_{\neq} ( μ 0 , … , μ r − 1 ) = ( λ 0 , … , λ r − 1 ) ∈ Ω = r . This is commonly assumed for example in the reduction of LTI systems [1 , 31 ] . With this additional assumption, we can show that the condition D ( μ ) = D ( λ ~ ) = 0 \mathcal{D}(\boldsymbol{\mu})=\mathcal{D}(\widetilde{\boldsymbol{\lambda}})=0 D ( μ ) = D ( λ ) = 0 not only implies, but is equivalent to Theorem 4.3 and Theorem 4.4 .
Theorem 4.14. Suppose that the assumptions made in Theorem 4.3 are satisfied and let D : Ω r → C r \mathcal{D}:\Omega^r\to\mathbb{C}^r D : Ω r → C r be the equilibrium function as given in Definition 4.12 . Assume further that ( λ ~ 0 , … , λ ~ r − 1 ) ∈ Ω + r (\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{r-1})\in\Omega^r_+ ( λ 0 , … , λ r − 1 ) ∈ Ω + r . Then a ROM
G r ∈ V ~ r λ ~ ⊆ G ξ r , \mathbf{G}_r\in\widetilde{V}^{\widetilde{\lambda}}_r\subseteq\mathcal{G}^r_\xi, G r ∈ V r λ ⊆ G ξ r , satisfies (24) (with m j = 1 m_j=1 m j = 1 , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 ) if and only if
D ( λ ~ ) = 0. \mathcal{D}(\widetilde{\lambda})=0. D ( λ ) = 0.
Proof. Note that due to the distinct kernel center assumption, the subspaces V r λ ~ V^{\widetilde{\lambda}}_r V r λ from (15) , D V r λ ~ DV^{\widetilde{\lambda}}_r D V r λ from (16) coincide with the subspaces V ~ r λ ~ \widetilde{V}^{\widetilde{\lambda}}_r V r λ and D V ~ r λ ~ D\widetilde{V}^{\widetilde{\lambda}}_r D V r λ from Theorem 4.3 . These are given by
V r λ ~ = V ~ r λ ~ : = span { ξ λ ~ 0 , … , ξ λ ~ r − 1 } V^{\widetilde{\lambda}}_r=\widetilde{V}^{\widetilde{\lambda}}_r:=\operatorname{span}\left\{\xi_{\widetilde{\lambda}_0},\ldots,\xi_{\widetilde{\lambda}_{r-1}}\right\} V r λ = V r λ := span { ξ λ 0 , … , ξ λ r − 1 }
and
D V r λ ~ = D V ~ r λ ~ : = span { D λ ~ ξ λ ~ 0 , … , D λ ~ ξ λ ~ r − 1 } . DV^{\widetilde{\lambda}}_r=D\widetilde{V}^{\widetilde{\lambda}}_r:=\operatorname{span}\left\{D_{\widetilde{\lambda}}\xi_{\widetilde{\lambda}_0},\ldots,D_{\widetilde{\lambda}}\xi_{\widetilde{\lambda}_{r-1}}\right\}. D V r λ = D V r λ := span { D λ ξ λ 0 , … , D λ ξ λ r − 1 } .
First, assume G ( λ ~ j ) = G r ( λ ~ j ) \mathbf{G}(\widetilde{\lambda}_j)=\mathbf{G}_r(\widetilde{\lambda}_j) G ( λ j ) = G r ( λ j ) and G ′ ( λ ~ j ) = G r ′ ( λ ~ j ) \mathbf{G}'(\widetilde{\lambda}_j)=\mathbf{G}'_r(\widetilde{\lambda}_j) G ′ ( λ j ) = G r ′ ( λ j ) for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 , that is, (24) with m j = 1 m_j=1 m j = 1 , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 is satisfied. The reproducing property means
G ( λ ~ j ) − G r ( λ ~ j ) = 0 ⟺ ⟨ G − G r , ξ λ ~ j ⟩ H = 0 ( j = 0 , … , r − 1 ) . \mathbf{G}(\widetilde{\lambda}_j)-\mathbf{G}_r(\widetilde{\lambda}_j)=0\quad\Longleftrightarrow\quad\left\langle\mathbf{G}-\mathbf{G}_r,\xi_{\widetilde{\lambda}_j}\right\rangle_{\mathcal{H}}=0\quad(j=0,\ldots,r-1). G ( λ j ) − G r ( λ j ) = 0 ⟺ ⟨ G − G r , ξ λ j ⟩ H = 0 ( j = 0 , … , r − 1 ) .
However, by the Hilbert projection theorem, this means G − G r ⊥ V ~ r λ ~ \mathbf{G}-\mathbf{G}_r\perp\widetilde{V}^{\widetilde{\lambda}}_r G − G r ⊥ V r λ , and consequently G r = P V ~ r λ ~ [ G ] \mathbf{G}_r=\mathcal{P}_{\widetilde{V}^{\widetilde{\lambda}}_r}[\mathbf{G}] G r = P V r λ [ G ] . In addition, by assumption, we have
[ D ( λ ~ ) ] j = G ′ ( λ ~ j ) − ( P V ~ r λ ~ [ G ] ) ′ ( λ ~ j ) = G ′ ( λ ~ j ) − G r ′ ( λ ~ j ) = 0. \left[\mathcal{D}(\widetilde{\lambda})\right]_j=\mathbf{G}'(\widetilde{\lambda}_j)-\left(\mathcal{P}_{\widetilde{V}^{\widetilde{\lambda}}_r}[\mathbf{G}]\right)'(\widetilde{\lambda}_j)=\mathbf{G}'(\widetilde{\lambda}_j)-\mathbf{G}'_r(\widetilde{\lambda}_j)=0. [ D ( λ ) ] j = G ′ ( λ j ) − ( P V r λ [ G ] ) ′ ( λ j ) = G ′ ( λ j ) − G r ′ ( λ j ) = 0.
Conversely, assume that D ( λ ~ ) = 0 \mathcal{D}(\widetilde{\lambda})=0 D ( λ ) = 0 . By Theorem 4.13 , G r = P V ~ r λ ~ [ G ] \mathbf{G}_r=\mathcal{P}_{\widetilde{V}^{\widetilde{\lambda}}_r}[\mathbf{G}] G r = P V r λ [ G ] , and we have
G ( λ ~ j ) − G r ( λ ~ j ) = G ( λ ~ j ) − P V ~ r λ ~ [ G ] ( λ ~ j ) = ⟨ G − P V ~ r λ ~ [ G ] , ξ λ ~ j ⟩ H = 0 \mathbf{G}(\widetilde{\lambda}_j)-\mathbf{G}_r(\widetilde{\lambda}_j)=\mathbf{G}(\widetilde{\lambda}_j)-\mathcal{P}_{\widetilde{V}^{\widetilde{\lambda}}_r}[\mathbf{G}](\widetilde{\lambda}_j)=\left\langle\mathbf{G}-\mathcal{P}_{\widetilde{V}^{\widetilde{\lambda}}_r}[\mathbf{G}],\xi_{\widetilde{\lambda}_j}\right\rangle_{\mathcal{H}}=0 G ( λ j ) − G r ( λ j ) = G ( λ j ) − P V r λ [ G ] ( λ j ) = ⟨ G − P V r λ [ G ] , ξ λ j ⟩ H = 0
( j = 0 , … , r − 1 ) . (j=0,\ldots,r-1). ( j = 0 , … , r − 1 ) .
Since D ( λ ~ ) = 0 \mathcal{D}(\widetilde{\lambda})=0 D ( λ ) = 0 , we then have
[ D ( λ ~ ) ] j = G ′ ( λ ~ j ) − ( P V ~ r λ ~ [ G ] ) ′ ( λ ~ j ) = G ′ ( λ ~ j ) − G r ′ ( λ ~ j ) = 0 ( j = 0 , … , r − 1 ) . \left[\mathcal{D}(\widetilde{\lambda})\right]_j=\mathbf{G}'(\widetilde{\lambda}_j)-\left(\mathcal{P}_{\widetilde{V}^{\widetilde{\lambda}}_r}[\mathbf{G}]\right)'(\widetilde{\lambda}_j)=\mathbf{G}'(\widetilde{\lambda}_j)-\mathbf{G}'_r(\widetilde{\lambda}_j)=0\quad(j=0,\ldots,r-1). [ D ( λ ) ] j = G ′ ( λ j ) − ( P V r λ [ G ] ) ′ ( λ j ) = G ′ ( λ j ) − G r ′ ( λ j ) = 0 ( j = 0 , … , r − 1 ) .
Remark 4.15. Our implementations of the algorithms introduced in § 7 make the assumption ( λ ~ 0 , … , λ ~ r − 1 ) ∈ Ω + r (\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{r-1})\in\Omega^r_+ ( λ 0 , … , λ r − 1 ) ∈ Ω + r . The results in § 8 show that even under this assumption, high quality ROMs can be obtained for a diverse family of FOMs.
Generalized Malmquist–Takenaka systems
In this section, we introduce a novel orthonormal function system called generalized Malmquist–Takenaka (GMT) functions that can be constructed in any RKHS. Notice that Theorems 4.3 and 4.4 can only be applied in RKHSs with sufficiently smooth kernels. For MOR problems posed in RKHSs, where the above theorems cannot be applied, we introduce a method to transform the MOR problem to an admissible RKHS. With the help of GMT functions, we obtain isometries (in fact unitary maps) that can be used to conduct this transformation procedure. For an illustration of this pipeline, see Figure 1 .
Generalized MT functions in RKHSs
Henceforth, for Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C , let ℓ Ω \ell_{\Omega} ℓ Ω denote the set of sequences whose components belong to Ω \Omega Ω . Similarly, we denote by ℓ N \ell_{\mathbb{N}} ℓ N the set of sequences containing natural numbers. We make use of the following assumptions throughout this section.
Assumption 5.1. Let H \mathcal{H} H be an RKHS of functions C ⊇ Ω → C \mathbb{C} \supseteq\Omega\to\mathbb{C} C ⊇ Ω → C with kernel ξ \xi ξ . Suppose λ ↦ ξ λ ∈ C m ( Ω , H ) \lambda\mapsto\xi_{\lambda} \in C^{m}(\Omega,\mathcal{H}) λ ↦ ξ λ ∈ C m ( Ω , H ) for some m ∈ N m \in\mathbb{N} m ∈ N and D λ i ξ λ ∈ H D_{\lambda}^{i}\xi_{\lambda} \in\mathcal{H} D λ i ξ λ ∈ H for i = 0 , … , m i=0,\ldots,m i = 0 , … , m . Let λ = ( λ 0 , λ 1 , λ 2 , … ) ∈ ℓ Ω \boldsymbol{\lambda}=(\lambda_{0},\lambda_{1},\lambda_{2},\ldots)\in\ell_{\Omega} λ = ( λ 0 , λ 1 , λ 2 , … ) ∈ ℓ Ω be such that each of its unique elements appears at most m + 1 m+1 m + 1 times in the sequence. Let d = ( d 0 , d 1 , d 2 , … ) ∈ ℓ N \mathbf{d}=(d_{0},d_{1},d_{2},\ldots)\in\ell_{\mathbb{N}} d = ( d 0 , d 1 , d 2 , … ) ∈ ℓ N be defined such that d k d_{k} d k is number of occurrences of the value λ k \lambda_{k} λ k in the vector ( λ 0 , … , λ k − 1 ) (\lambda_{0},\ldots,\lambda_{k-1}) ( λ 0 , … , λ k − 1 ) for all k ∈ N k\in\mathbb{N} k ∈ N .
Denote by { η k } k = 0 ∞ ⊆ H \{\eta_{k}\}_{k=0}^{\infty}\subseteq\mathcal{H} { η k } k = 0 ∞ ⊆ H the function system
η k : = D λ d k ξ λ k . (35) \eta_{k}:=D_{\lambda}^{d_{k}}\xi_{\lambda_{k}}. \tag*{(35)} η k := D λ d k ξ λ k . (35) Suppose finally that the functions { η 0 , … , η n − 1 } \{\eta_{0},\ldots,\eta_{n-1}\} { η 0 , … , η n − 1 } are linearly independent for all admissible choices of λ \boldsymbol{\lambda} λ (and d \mathbf{d} d ) and n ≤ dim H n\leq\dim\mathcal{H} n ≤ dim H .
In this section, we further define
V n λ : = span { η 0 , … , η n − 1 } ⊆ H . (36) V_{n}^{\boldsymbol{\lambda}}:=\operatorname{span}\{\eta_{0},\ldots,\eta_{n-1}\}\subseteq\mathcal{H}. \tag*{(36)} V n λ := span { η 0 , … , η n − 1 } ⊆ H . (36)
Note that this definition of V n λ V_{n}^{\boldsymbol{\lambda}} V n λ corresponds to V n λ V_{n}^{\lambda} V n λ from Theorem 4.3 with the difference that now repeated entries in λ \boldsymbol{\lambda} λ are allowed. In addition, for ease of notation, note that λ \boldsymbol{\lambda} λ will denote both sequences and finite-length vectors of kernel centers depending on the context. We are now ready to introduce the GMT basis functions.
Definition 5.2 (Generalized Malmquist–Takenaka functions, GMT–Fourier partial sum). Suppose Assumption 5.1 is satisfied. For n ≥ 0 n\geq0 n ≥ 0 , the n n n th GMT function is defined as
φ n λ : = η n − S n λ [ η n ] ∥ η n − S n λ [ η n ] ∥ H , \varphi_{n}^{\lambda}:=\frac{\eta_{n}-S_{n}^{\lambda}[\eta_{n}]}{\left\lVert\eta_{n}-S_{n}^{\lambda}[\eta_{n}]\right\rVert_{\mathcal{H}}}, φ n λ := ∥ η n − S n λ [ η n ] ∥ H η n − S n λ [ η n ] , where for n ≥ 0 n\geq0 n ≥ 0 ,
S n λ [ G ] : = ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ k λ ( G ∈ H ) . (37) S_{n}^{\lambda}[G]:=\sum_{k=0}^{n-1}\langle G,\varphi_{k}^{\lambda}\rangle_{\mathcal{H}}\varphi_{k}^{\lambda}\qquad(G\in\mathcal{H}). \tag*{(37)} S n λ [ G ] := k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k λ ( G ∈ H ) . (37) We refer to the operator S n λ S_{n}^{\lambda} S n λ as the n n n th GMT–Fourier partial sum.
Remark 5.3. If we consider d k = 0 d_k = 0 d k = 0 for k = 0 , … , n k = 0, \ldots, n k = 0 , … , n in Assumption 5.1, then Definition 5.2 describes a Gram–Schmidt orthogonalization procedure applied to the pairwise distinct kernel slices ξ λ k \xi_{\lambda_k} ξ λ k . Hence, the GMT system can be constructed in any RKHS. If d k > 0 d_k > 0 d k > 0 for some 0 ≤ k ≤ n 0 \le k \le n 0 ≤ k ≤ n , then the orthogonalization procedure acts on the kernel slices and their derivatives up to at most order m m m . Notice that the actual functions undergoing orthogonalization are parameterized by the sequence λ ∈ ℓ Ω \lambda\in\ell_{\Omega} λ ∈ ℓ Ω .
Definition 5.2 is a generalization of the classical Malmquist–Takenaka (MT) system [22 , 44 , 55 ] . Indeed, if we choose H = H 2 ( D ) \mathcal{H} = H_2(\mathbb{D}) H = H 2 ( D ) and λ ∈ ℓ D \lambda\in\ell_{\mathbb{D}} λ ∈ ℓ D , then the above definition gives the well–known rational orthonormal system
φ n λ ( z ) = 1 − ∣ λ n ∣ 2 1 − λ n ‾ z ∏ k = 0 n − 1 z − λ k 1 − λ k ‾ z ( z ∈ D , λ ∈ ℓ D ) . (38) \varphi_n^\lambda(z) = \frac{\sqrt{1-\lvert\lambda_n\rvert^2}}{1-\overline{\lambda_n}z}\prod_{k=0}^{n-1}\frac{z-\lambda_k}{1-\overline{\lambda_k}z}\qquad(z\in\mathbb{D},\lambda\in\ell_{\mathbb{D}}). \tag*{(38)} φ n λ ( z ) = 1 − λ n z 1 − ∣ λ n ∣ 2 k = 0 ∏ n − 1 1 − λ k z z − λ k ( z ∈ D , λ ∈ ℓ D ) . (38)
This system is complete in H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) (see (12) ) if and only if λ ∈ ℓ D \lambda\in\ell_{\mathbb{D}} λ ∈ ℓ D satisfies the Blaschke–Szász condition [34 ]
∑ n = 0 ∞ ( 1 − ∣ λ n ∣ ) = ∞ . (39) \sum_{n=0}^{\infty}(1-\lvert\lambda_n\rvert)=\infty. \tag*{(39)} n = 0 ∑ ∞ ( 1 − ∣ λ n ∣) = ∞. (39)
MT functions have many peculiar properties that make them important for the study of Hardy spaces. In Section 5.2, we show that several of these properties are retained by the GMT system in Definition 5.2. MT functions also have many applications, for example, in system identification [22 , 23 , 29 , 34 , 63 ] , rational interpolation [28 , 48 ] , medical signal processing [41 ] , and model order reduction [44 , 60 ] .
We note that the classical MT functions in (5.4) form a rather general orthonormal system in H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) . Indeed, if we choose λ = ( 0 , 0 , 0 , … ) \lambda= (0, 0, 0, \ldots) λ = ( 0 , 0 , 0 , … ) , then (5.4) reduces to φ n λ ( z ) = z n \varphi_n^\lambda(z) = z^n φ n λ ( z ) = z n (z ∈ D z \in\mathbb{D} z ∈ D ). This can be analytically continued to T \mathbb{T} T , where the functions then become the classical trigonometric system. Choosing λ = ( λ , λ , … ) \lambda= (\lambda,\lambda,\ldots) λ = ( λ , λ , … ) (λ ∈ D ∖ { 0 } \lambda\in\mathbb{D} \setminus\{0\} λ ∈ D ∖ { 0 } ) yields the Laguerre orthonormal system in H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) with many applications in system theory [21 , 69 ] . Finally, we note that the classical MT system has been previously generalized to other specific RKHSs. For example, in [51 ] , MT functions are constructed in Bergman spaces. Additionally, using conformal maps, they can be defined on H 2 H_2 H 2 spaces of the half plane in [1 , 48 ] .
Some attempts to define GMT–like systems in RKHSs have also been made previously. In [47 ] , a Newton–basis is defined for general RKHSs, which matches the above definition if λ k ≠ λ j \lambda_k \ne\lambda_j λ k = λ j when j ≠ k j \ne k j = k for all k , j = 0 , … , n k,j = 0,\ldots,n k , j = 0 , … , n . In [47 ] , the authors only investigate certain interpolatory properties of the proposed functions and make no connection to the classical rational MT system. Definition 5.2 generalizes this Newton–basis by allowing repeated kernel centers in λ \lambda λ if the kernel ξ \xi ξ is sufficiently smooth. Recently, in [55 ] , generalized MT functions have been proposed in a family of RKHSs known as boundary weighted Hardy spaces. Instead of the Gram–Schmidt method, the authors of [55 ] consider an inner–outer factorization–based approach to generate their MT system. This restricts their constructions to a family of H 2 H_2 H 2 –like analytic function spaces. In addition, they do not consider RKHSs where the map λ ↦ ξ λ \lambda\mapsto\xi_\lambda λ ↦ ξ λ is only m m m –times differentiable. Hence, the kernel MT system defined in [55 ] is also an important special case of Definition 5.2 . To the best of our knowledge, however, there have been no previous generalizations to cover repeated kernel centers in λ ∈ ℓ Ω \lambda\in\ell_{\Omega} λ ∈ ℓ Ω in arbitrary RKHSs as given in Definition 5.2 .
Numerical concerns regarding a possible definition similar to Definition 5.2 have been voiced in [55 ] due to the application of the Gram–Schmidt method. These concerns are partially addressed empirically in § 8 , where a variety of experiments shows that, in many RKHSs, GMT functions can in fact be safely generated. In addition, we introduce a biorthogonal system later in § 6.2 , which can be used to transform FOMs to companion spaces without the need for the explicit computation of GMT functions. Finally, we note that the proposed biorthogonal GMT basis can be interpreted as the analog of Lagrange basis polynomials for interpolation problems in RKHSs.
Properties of GMT functions
We now show some elementary properties of GMT functions. We will make the following assumption throughout this section.
Assumption 5.4. Assume that H \mathcal{H} H is an RKHS of functions C ⊇ Ω → C \mathcal{C} \supseteq\Omega\to\mathbb{C} C ⊇ Ω → C with reproducing kernel ξ \xi ξ and let furthermore, λ ∈ ℓ Ω \lambda\in\ell_{\Omega} λ ∈ ℓ Ω . In addition, let { φ k λ } k = 0 n − 1 \{\varphi_k^\lambda\}_{k=0}^{n-1} { φ k λ } k = 0 n − 1 be the first n n n GMT functions generated by λ \lambda λ as per Definition 5.2 .
Lemma 5.5 (Orthogonality of the GMT system). Under Assumption 5.4 , for any choice of n ∈ N n \in\mathbb{N} n ∈ N and RKHS H \mathcal{H} H , the GMT functions { φ k λ } k = 0 n − 1 \{\varphi_k^\lambda\}_{k=0}^{n-1} { φ k λ } k = 0 n − 1 are orthonormal, that is, for j , k = 0 , … , n − 1 j,k=0,\ldots,n-1 j , k = 0 , … , n − 1 , we have
(a)
⟨ φ j λ , φ k λ ⟩ H = δ j , k : = { 0 , j ≠ k , 1 , j = k , (40) \left\langle\varphi_j^\lambda,\varphi_k^\lambda\right\rangle_{\mathcal{H}}=\delta_{j,k}:=
\begin{cases}
0, & j \ne k,\\
1, & j=k,
\end{cases} \tag*{(40)} ⟨ φ j λ , φ k λ ⟩ H = δ j , k := { 0 , 1 , j = k , j = k , (40) (b)
span { φ 0 λ , φ 1 λ , … , φ n − 1 λ } = V n λ . (41) \operatorname{span}\{\varphi_0^\lambda,\varphi_1^\lambda,\ldots,\varphi_{n-1}^\lambda\}=V_n^\lambda. \tag*{(41)} span { φ 0 λ , φ 1 λ , … , φ n − 1 λ } = V n λ . (41)
Proof. The statement follows from the observation that Definition 5.2 is a Gram–Schmidt procedure in H \mathcal{H} H .
Remark 5.6. The orthogonality property is also proven for the RKHS Newton basis introduced in [47 Theorem 6] . This corresponds to the GMT system when the generating sequence λ \lambda λ does not contain any repeated kernel centers.
The next claim verifies that we can use GMT functions to characterize the generalized model spaces from Definition 2.3 .
The ordering of the first n n n components in λ ∈ ℓ Ω \lambda\in\ell_{\Omega} λ ∈ ℓ Ω does not affect the subspace generated by the first n n n GMT functions. Hence, there are n ! n! n ! GMT systems that can be used to span V n λ V_n^\lambda V n λ . The next theorem gives insight into the placement of zeros of GMT functions.
Theorem 5.7 (Zeros). Under Assumption 5.1 and Assumption 5.4 , it holds that
( φ k λ ) ( d j ) ( λ j ) = 0 ( k = 0 , … , n − 1 , j = 0 , … , k − 1 ) . \left(\varphi_k^\lambda\right)^{(d_j)}(\lambda_j)=0 \quad(k=0,\ldots,n-1,\ j=0,\ldots,k-1). ( φ k λ ) ( d j ) ( λ j ) = 0 ( k = 0 , … , n − 1 , j = 0 , … , k − 1 ) .
Proof. Let n > 0 n>0 n > 0 and k ≤ n k\le n k ≤ n . By (5.6) and (41) , we have φ n λ ⊥ V k λ \varphi_n^\lambda\perp V_k^\lambda φ n λ ⊥ V k λ and η j = D λ ˉ d j ξ λ j ∈ V k λ \eta_j=D_{\bar{\lambda}}^{d_j}\xi_{\lambda_j}\in V_k^\lambda η j = D λ ˉ d j ξ λ j ∈ V k λ for j = 0 , … , k − 1 j=0,\ldots,k-1 j = 0 , … , k − 1 . Thus,
0 = ⟨ φ n λ , D λ ˉ d j ξ λ j ⟩ H = ( φ n λ ) ( d j ) ( λ j ) . 0=\left\langle\varphi_n^\lambda,D_{\bar{\lambda}}^{d_j}\xi_{\lambda_j}\right\rangle_{\mathcal{H}}=\left(\varphi_n^\lambda\right)^{(d_j)}(\lambda_j). 0 = ⟨ φ n λ , D λ ˉ d j ξ λ j ⟩ H = ( φ n λ ) ( d j ) ( λ j ) .
Remark 5.8. Theorem 5.7 also holds for the classical MT functions and can be easily seen from the factorized form (38) .
For some applications, we need to consider real-valued RKHSs. In the real-valued case, the Chebyshev or Haar property is key for many applications such as uniform approximation of continuous functions. The following theorem establishes this property for the GMT system under some additional assumptions.
Theorem 5.9 (Chebyshev–Haar property). Let H \mathcal{H} H be an RKHS of functions Ω ⊇ R → R \Omega\supseteq\mathbb{R}\to\mathbb{R} Ω ⊇ R → R . Under Assumption 5.1 , suppose that for any choice of n ≥ 1 n\ge1 n ≥ 1 and
t 0 < t 1 < … < t n − 1 ∈ Ω and λ 0 < λ 1 < … < λ n − 1 ∈ Ω t_0<t_1<\ldots<t_{n-1}\in\Omega\quad\text{and}\quad\lambda_0<\lambda_1<\ldots<\lambda_{n-1}\in\Omega t 0 < t 1 < … < t n − 1 ∈ Ω and λ 0 < λ 1 < … < λ n − 1 ∈ Ω ,
we have
det [ ξ ( t k , λ j ) ] k , j = 0 n − 1 > 0. (42) \det\left[\xi(t_k,\lambda_j)\right]_{k,j=0}^{n-1}>0. \tag*{(42)} det [ ξ ( t k , λ j ) ] k , j = 0 n − 1 > 0. (42) Then, the GMT system { φ k λ } k = 0 n − 1 \{\varphi_k^\lambda\}_{k=0}^{n-1} { φ k λ } k = 0 n − 1 has the Chebyshev–Haar property, that is, the zeros identified in Theorem 5.7 are the only zeros.
Proof. We note that by the assumptions of the theorem, all kernel centers in λ \boldsymbol{\lambda} λ are distinct. Hence, by Definition 5.2 and Lemma 5.5 , we have
V n λ = span { φ 0 λ , … , φ n − 1 λ } = span { ξ λ 0 , … , ξ λ n − 1 } . V_n^\lambda=\operatorname{span}\{\varphi_0^\lambda,\ldots,\varphi_{n-1}^\lambda\}=\operatorname{span}\{\xi_{\lambda_0},\ldots,\xi_{\lambda_{n-1}}\}. V n λ = span { φ 0 λ , … , φ n − 1 λ } = span { ξ λ 0 , … , ξ λ n − 1 } .
From this, we observe that for any k = 0 , … , n − 1 k=0,\ldots,n-1 k = 0 , … , n − 1 we can write
φ k λ = ∑ j = 0 k a k , j ξ λ j , \varphi_k^\lambda=\sum_{j=0}^{k}a_{k,j}\xi_{\lambda_j}, φ k λ = j = 0 ∑ k a k , j ξ λ j ,
where a k , k ≠ 0 a_{k,k}\ne0 a k , k = 0 . Introduce the matrix A = [ a k , j ] k , j = 0 n − 1 \mathbf{A}=[a_{k,j}]_{k,j=0}^{n-1} A = [ a k , j ] k , j = 0 n − 1 with the convention that a k , j = 0 a_{k,j}=0 a k , j = 0 whenever j > k j>k j > k . Similarly, introduce Ξ = [ ξ ( t j , λ k ) ] k , j = 0 n − 1 \mathbf{\Xi}=[\xi(t_j,\lambda_k)]_{k,j=0}^{n-1} Ξ = [ ξ ( t j , λ k ) ] k , j = 0 n − 1 . Then, we can express the matrix Φ \mathbf{\Phi} Φ , which satisfies [ Φ ] k , j = φ k λ ( t j ) [\mathbf{\Phi}]_{k,j}=\varphi_k^\lambda(t_j) [ Φ ] k , j = φ k λ ( t j ) for k , j = 0 , … , n − 1 a s k,j=0,\ldots,n-1 as k , j = 0 , … , n − 1 a s \mathbf{\Phi}=\mathbf{A}\mathbf{\Xi}$. From this,
det Φ = det A det Ξ , \det\mathbf{\Phi}=\det\mathbf{A}\det\mathbf{\Xi}, det Φ = det A det Ξ ,
where det Ξ > 0 \det\mathbf{\Xi}>0 det Ξ > 0 by (42) and det A ≠ 0 \det\mathbf{A}\ne0 det A = 0 because it is a triangular matrix with nonzero diagonal entries. This implies the statement. Indeed, consider H ∈ V n λ ∖ { 0 } \mathbf{H}\in V_n^\lambda\setminus\{0\} H ∈ V n λ ∖ { 0 } and suppose H \mathbf{H} H has n n n distinct zeros, i.e.,
H ( t k ) = ∑ j = 0 n − 1 c j φ j λ ( t k ) = 0 ( k = 0 , … , n − 1 ) . \mathbf{H}(t_k)=\sum_{j=0}^{n-1}c_j\varphi_j^\lambda(t_k)=0\quad(k=0,\ldots,n-1). H ( t k ) = j = 0 ∑ n − 1 c j φ j λ ( t k ) = 0 ( k = 0 , … , n − 1 ) .
Taking c : = [ c 0 , … , c n − 1 ] T \mathbf{c}:=[c_0,\ldots,c_{n-1}]^{\mathsf{T}} c := [ c 0 , … , c n − 1 ] T , we can write this in matrix form Φ c = 0 \boldsymbol{\Phi}\mathbf{c}=0 Φ c = 0 . However, det Φ ≠ 0 \det\boldsymbol{\Phi}\ne0 det Φ = 0 , hence the only solution to this is c = 0 \mathbf{c}=0 c = 0 , which contradicts the assumption H ≠ 0 \mathbf{H}\ne0 H = 0 . Thus, H ∈ V n λ \mathbf{H}\in V_n^\lambda H ∈ V n λ cannot have n n n distinct zeros. Since φ n − 1 λ ∈ V n λ \varphi_{n-1}^{\lambda}\in V_n^\lambda φ n − 1 λ ∈ V n λ , the n − 1 n-1 n − 1 zeros from Theorem 5.7 are the only zeros.
Remark 5.10. (a) Notice that under the assumptions of Theorem 5.9 , every initial segment φ 0 λ , … , φ k λ \varphi_0^\lambda,\ldots,\varphi_k^\lambda φ 0 λ , … , φ k λ with k < n k<n k < n also has the Chebyshev–Haar property.
(b) The Chebyshev–Haar property is key for various numerical methods. For example, the Remez algorithm, which constructs best approximations in the Chebyshev norm to continuous functions, has an extension to Haar systems in general [53 ] .
The next claim asserts that the GMT–Fourier partial sums given in (37) are orthogonal projection operators onto the subspace V n λ V_n^\lambda V n λ .
Lemma 5.11 (GMT projection). Under Assumption 5.4 and Definition 5.2, we have
S n λ = P V n λ . S_n^\lambda=P_{V_n^\lambda}. S n λ = P V n λ .
Proof. With the orthonormal basis { φ 0 λ , … , φ n − 1 λ } \{\varphi_0^\lambda,\ldots,\varphi_{n-1}^\lambda\} { φ 0 λ , … , φ n − 1 λ } of V n λ V_n^\lambda V n λ , by standard linear algebra arguments we obtain
P V n λ [ G ] = ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ k λ = S n λ [ G ] ( G ∈ H ) . P_{V_n^\lambda}[G]=\sum_{k=0}^{n-1}\langle G,\varphi_k^\lambda\rangle_{\mathcal{H}}\varphi_k^\lambda=S_n^\lambda[G]\quad(G\in\mathcal{H}). P V n λ [ G ] = k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k λ = S n λ [ G ] ( G ∈ H ) .
The following theorem shows that GMT–Fourier partial sums provide a convenient way to obtain interpolants in RKHSs. This can be seen as a generalization of the rational interpolation technique proposed in [48 ] using classical MT expansions. It also generalizes the Newton–basis–based interpolation in RKHSs [47 ] , which did not allow for kernel centers with higher multiplicities. With the proposed Definition 5.2 , we can claim much more. Suppose that k ≤ n − 1 k\le n-1 k ≤ n − 1 is the highest index of appearance of λ \lambda λ . Then, by the next statement, we find that our GMT expansions are Hermite interpolants of any function G ∈ H G\in\mathcal{H} G ∈ H at the node λ \lambda λ up to order d k d_k d k .
Theorem 5.12 (Interpolatory property of GMT partial sums). Suppose that Assumption 5.1 and Assumption 5.4 hold. Then, for any G ∈ H G\in\mathcal{H} G ∈ H ,
( S n λ [ G ] ) ( d k ) ( λ k ) = G ( d k ) ( λ k ) ( k = 0 , … , n − 1 ) , \left(S_n^\lambda[G]\right)^{(d_k)}(\lambda_k)=G^{(d_k)}(\lambda_k)\quad(k=0,\ldots,n-1), ( S n λ [ G ] ) ( d k ) ( λ k ) = G ( d k ) ( λ k ) ( k = 0 , … , n − 1 ) , where S n λ S_n^\lambda S n λ defines the GMT–Fourier partial sums given in (37) .
Proof. By Lemma 5.11 , S n λ S_n^\lambda S n λ is an orthogonal projector onto V n λ V_n^\lambda V n λ . Hence, for any G ∈ H G \in\mathcal{H} G ∈ H and k = 0 , … , n − 1 k=0,\ldots,n-1 k = 0 , … , n − 1 we have
0 = ⟨ G − S n λ [ G ] , η k ⟩ H = ⟨ G , D λ d k ξ λ k ⟩ H − ⟨ S n λ [ G ] , D λ d k ξ λ k ⟩ H = G ( d k ) ( λ k ) − ( S n λ [ G ] ) ( d k ) ( λ k ) . 0=\left\langle G-S_n^\lambda[G],\eta_k\right\rangle_{\mathcal{H}}=\left\langle G,D_\lambda^{d_k}\xi_{\lambda_k}\right\rangle_{\mathcal{H}}-\left\langle S_n^\lambda[G],D_\lambda^{d_k}\xi_{\lambda_k}\right\rangle_{\mathcal{H}}=G^{(d_k)}(\lambda_k)-\left(S_n^\lambda[G]\right)^{(d_k)}(\lambda_k). 0 = ⟨ G − S n λ [ G ] , η k ⟩ H = ⟨ G , D λ d k ξ λ k ⟩ H − ⟨ S n λ [ G ] , D λ d k ξ λ k ⟩ H = G ( d k ) ( λ k ) − ( S n λ [ G ] ) ( d k ) ( λ k ) .
where the first equality in the final expression follows by Cor. 3.3.
With the help of GMT functions, we can also define Lagrange interpolation schemes in RKHSs. For this, consider the following definition of generalized Lagrange basis polynomials.
Definition 5.13 (Lagrange–GMT basis functions). Under Assumption 5.1, assume that λ 0 , … , λ n − 1 \lambda_0,\ldots,\lambda_{n-1} λ 0 , … , λ n − 1 are pairwise distinct. Suppose also that ξ ( λ , λ ) ≠ 0 \xi(\lambda,\lambda)\ne0 ξ ( λ , λ ) = 0 for all λ ∈ Ω \lambda\in\Omega λ ∈ Ω . Let
λ l : = ( λ 0 , … , λ l − 1 , λ l + 1 , … , λ n − 1 , λ l ) \lambda^l:=(\lambda_0,\ldots,\lambda_{l-1},\lambda_{l+1},\ldots,\lambda_{n-1},\lambda_l) λ l := ( λ 0 , … , λ l − 1 , λ l + 1 , … , λ n − 1 , λ l ) for l = 0 , … , n − 1 l=0,\ldots,n-1 l = 0 , … , n − 1 . The Lagrange–GMT basis function corresponding to the node λ l \lambda_l λ l is given by
ψ l : = φ n − 1 λ l φ n − 1 λ l ( λ l ) . \psi_l:=\frac{\varphi_{n-1}^{\lambda^l}}{\varphi_{n-1}^{\lambda^l}(\lambda_l)}. ψ l := φ n − 1 λ l ( λ l ) φ n − 1 λ l . where φ n − 1 λ l \varphi_{n-1}^{\lambda^l} φ n − 1 λ l is the n n n th GMT function generated by λ l \lambda^l λ l for l = 0 , … , n − 1 l=0,\ldots,n-1 l = 0 , … , n − 1 .
We now show some key properties of the Lagrange–GMT basis from Definition 5.13.
Theorem 5.14 (Lagrange property). The Lagrange–GMT functions from Definition 5.13 satisfy the Lagrange property and moreover, { ψ k } k = 0 n − 1 \{\psi_k\}_{k=0}^{n-1} { ψ k } k = 0 n − 1 forms a biorthogonal system to { ξ λ k } k = 0 n − 1 \{\xi_{\lambda_k}\}_{k=0}^{n-1} { ξ λ k } k = 0 n − 1 . That is,
⟨ ψ k , ξ λ j ⟩ H = ψ k ( λ j ) = δ j , k \left\langle\psi_k,\xi_{\lambda_j}\right\rangle_{\mathcal{H}}=\psi_k(\lambda_j)=\delta_{j,k} ⟨ ψ k , ξ λ j ⟩ H = ψ k ( λ j ) = δ j , k for k , j = 0 , … , n − 1 k,j=0,\ldots,n-1 k , j = 0 , … , n − 1 (where δ j , k \delta_{j,k} δ j , k denotes again the Kronecker symbol).
Proof. Consider first j = k j=k j = k . Then,
⟨ ψ k , ξ λ k ⟩ H = ψ k ( λ k ) = φ n − 1 λ k ( λ k ) φ n − 1 λ k ( λ k ) = 1. \left\langle\psi_k,\xi_{\lambda_k}\right\rangle_{\mathcal{H}}=\psi_k(\lambda_k)=\frac{\varphi_{n-1}^{\lambda^k}(\lambda_k)}{\varphi_{n-1}^{\lambda^k}(\lambda_k)}=1. ⟨ ψ k , ξ λ k ⟩ H = ψ k ( λ k ) = φ n − 1 λ k ( λ k ) φ n − 1 λ k ( λ k ) = 1.
If k ≠ j k\ne j k = j , by Theorem 5.7 and Definition 5.13, we have
⟨ ψ k , ξ λ j ⟩ H = ψ k ( λ j ) = φ n − 1 λ k ( λ j ) φ n − 1 λ k ( λ k ) = 0. \left\langle\psi_k,\xi_{\lambda_j}\right\rangle_{\mathcal{H}}=\psi_k(\lambda_j)=\frac{\varphi_{n-1}^{\lambda^k}(\lambda_j)}{\varphi_{n-1}^{\lambda^k}(\lambda_k)}=0. ⟨ ψ k , ξ λ j ⟩ H = ψ k ( λ j ) = φ n − 1 λ k ( λ k ) φ n − 1 λ k ( λ j ) = 0.
We now turn our attention to more general properties of the GMT system. First, we discuss the question of completeness in RKHSs and raise a point about the Blaschke–Szász property for GMT systems in general RKHSs. Then, we introduce a GMT–like biorthogonal system in M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω –invariant RKHSs to express P V n λ P_{\mathcal{V}_{n}^{\lambda}} P V n λ .
The next theorem clarifies the existence of complete GMT systems in infinite–dimensional RKHSs.
Theorem 5.15 (Complete GMT systems). Let H \mathcal{H} H be an infinite–dimensional separable RKHS of functions C ⊇ Ω → C \mathbb{C} \supseteq\Omega\to\mathbb{C} C ⊇ Ω → C . Suppose Assumption 5.1 holds and let { φ k } k = 0 ∞ \{\varphi_k\}_{k=0}^{\infty} { φ k } k = 0 ∞ denote the GMT system. Then there exists a λ ∈ ℓ Ω \lambda\in\ell_{\Omega} λ ∈ ℓ Ω such that { φ n λ } n = 0 ∞ \{\varphi_n^{\lambda}\}_{n=0}^{\infty} { φ n λ } n = 0 ∞ forms a complete orthonormal system in H \mathcal{H} H .
Proof. The statement is a direct consequence of Theorem 3.4 .
Remark 5.16. A natural question is whether there are analogs of the Blaschke–Szász condition (see (39) ) for GMT functions in a general separable RKHS. We do not attempt to answer this question in this paper; however, we note that, in addition to H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) , similar conditions have already been defined in other RKHSs. We highlight the Bergman space A 2 ( D ) A_2(\mathbb{D}) A 2 ( D ) , which contains absolutely square–integrable holomorphic functions of the open unit disk [17 ] . For this space, a special case of the GMT functions has been defined in [51 ] . In addition, an analog of the Blaschke–Szász condition is also known [36 ] . While the condition in this Hilbert space looks very similar to (39) , the proofs of these theorems are quite different. In particular, we note that the proof in Bergman spaces does not rely on inner–outer factorizations (unlike the proof of (39) , see [18 Chapter 2] ), as these do not exist in Bergman spaces. We refer to [17 Chapter 4] and [36 ] for a proof of the Blaschke–Szász condition for Bergman spaces. It is therefore an interesting question whether a Blaschke–Szász condition applicable to a wide class of RKHS spaces can be proved. An analog of the Blaschke–Szász condition is also shown in [55 ] for boundary weighted Hardy Spaces.
In [55 ] , the authors show that MT functions can be efficiently generated in a family of H 2 H_2 H 2 –like RKHSs known as boundary weighted Hardy spaces using inner–outer factorizations. Many RKHSs of interest, such as Bergman spaces, radial basis function (RBF) spaces, and others, however, do not belong to this class. The next theorem defines a biorthogonal system in RKHSs with M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω invariance. This class of RKHSs is much wider than boundary weighted Hardy spaces; for example, it also contains Bergman spaces and several non–analytic RKHSs. The following biorthogonal expansions offer a way to transform MOR problems to admissible RKHSs as introduced in § 6.3 when H \mathcal{H} H is M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω –invariant. They are inspired by the Lagrange–GMT functions from Definition 5.13 .
Theorem 5.17 (Biorthogonal GMT systems). Let Assumption 5.4 be satisfied and assume furthermore that H \mathcal{H} H is M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω –invariant as in Definition 4.9 . Suppose the first n ∈ N n \in\mathbb{N} n ∈ N elements of λ \lambda λ are pairwise distinct, that is, d k = 0 d_k = 0 d k = 0 for k = 0 , … , n − 1 k = 0, \ldots, n - 1 k = 0 , … , n − 1 . Finally, suppose that
ξ ( λ k , λ k ) ≠ 0 \xi(\lambda_k,\lambda_k) \ne0 ξ ( λ k , λ k ) = 0 for k = 0 , … , n − 1 k = 0,\ldots,n-1 k = 0 , … , n − 1 . Let
ϑ k = ξ λ k ⋅ ℓ k ( k = 0 , … , n − 1 ) , (43) \vartheta_k = \xi_{\lambda_k}\cdot\ell_k \qquad(k=0,\ldots,n-1), \tag*{(43)} ϑ k = ξ λ k ⋅ ℓ k ( k = 0 , … , n − 1 ) , (43)
where ℓ k \ell_k ℓ k is the Lagrange basis polynomial satisfying
ℓ k ( x ) = ∏ j = 0 j ≠ k n − 1 x − λ j λ k − λ j . \ell_k(x)=\prod_{\substack{j=0\\j\ne k}}^{n-1}\frac{x-\lambda_j}{\lambda_k-\lambda_j}. ℓ k ( x ) = j = 0 j = k ∏ n − 1 λ k − λ j x − λ j .
Then, the k k k th biorthogonal–GMT (BGMT) function
ψ k λ ( x ) : = ϑ k ξ ( λ k , λ k ) \psi_k^\lambda(x):=\frac{\vartheta_k}{\xi(\lambda_k,\lambda_k)} ψ k λ ( x ) := ξ ( λ k , λ k ) ϑ k
is biorthogonal to the system { ξ λ k } k = 0 n − 1 \{\xi_{\lambda_k}\}_{k=0}^{n-1} { ξ λ k } k = 0 n − 1 , that is, ⟨ ψ k λ , ξ λ j ⟩ H = δ j , k \langle\psi_k^\lambda,\xi_{\lambda_j}\rangle_{\mathcal{H}}=\delta_{j,k} ⟨ ψ k λ , ξ λ j ⟩ H = δ j , k for j , k = 0 , … , n − 1 j,k=0,\ldots,n-1 j , k = 0 , … , n − 1 .
Proof. Let k , j ∈ { 0 , 1 , … , n − 1 } k,j\in\{0,1,\ldots,n-1\} k , j ∈ { 0 , 1 , … , n − 1 } . Since ϑ k ∈ H \vartheta_k\in\mathcal{H} ϑ k ∈ H due to M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω invariance we have
⟨ ϑ k , ξ λ j ⟩ H = ⟨ ξ λ k ℓ k , ξ λ j ⟩ H = ξ λ k ( λ j ) ℓ k ( λ j ) = { ξ λ k ( λ k ) , if k = j , 0 , if k ≠ j . \langle\vartheta_k,\xi_{\lambda_j}\rangle_{\mathcal{H}}=\langle\xi_{\lambda_k}\ell_k,\xi_{\lambda_j}\rangle_{\mathcal{H}}=\xi_{\lambda_k}(\lambda_j)\ell_k(\lambda_j)=
\begin{cases}
\xi_{\lambda_k}(\lambda_k), & \text{if } k=j,\\
0, & \text{if } k\ne j.
\end{cases} ⟨ ϑ k , ξ λ j ⟩ H = ⟨ ξ λ k ℓ k , ξ λ j ⟩ H = ξ λ k ( λ j ) ℓ k ( λ j ) = { ξ λ k ( λ k ) , 0 , if k = j , if k = j .
The following lemma notes how BGMT functions can be used to represent models in M i d Ω \mathcal{M}_{\mathrm{id}_{\Omega}} M id Ω -invariant RKHSs.
Lemma 5.19. Suppose the assumptions of Theorem 5.17 are satisfied and let G ∈ G ξ n ⊆ H \mathbf{G}\in\mathfrak{G}_{\xi}^{n}\subseteq\mathcal{H} G ∈ G ξ n ⊆ H . We have
G = ∑ k = 0 n − 1 ⟨ G , ψ k λ ⟩ H ξ λ k . \mathbf{G}=\sum_{k=0}^{n-1}\langle\mathbf{G},\psi_k^\lambda\rangle_{\mathcal{H}}\xi_{\lambda_k}. G = k = 0 ∑ n − 1 ⟨ G , ψ k λ ⟩ H ξ λ k .
Proof. Since for any G ∈ G ξ n G \in\mathfrak{G}_{\xi}^{n} G ∈ G ξ n , we have
G = ∑ k = 0 n − 1 r k ξ λ k , G = \sum_{k=0}^{n-1} r_k \xi_{\lambda_k}, G = k = 0 ∑ n − 1 r k ξ λ k ,
it suffices to show r k = ⟨ G , ψ k λ ⟩ H r_k = \langle G,\psi_k^\lambda\rangle_{\mathcal{H}} r k = ⟨ G , ψ k λ ⟩ H . This is indeed the case, since
⟨ G , ψ k λ ⟩ H = ⟨ ∑ j = 0 n − 1 r j ξ λ j , ψ k λ ⟩ H = ∑ j = 0 n − 1 r j ⟨ ξ λ j , ψ k λ ⟩ H = Thm. 5.17 r k . \langle G,\psi_k^\lambda\rangle_{\mathcal{H}} = \left\langle\sum_{j=0}^{n-1} r_j \xi_{\lambda_j},\psi_k^\lambda\right\rangle_{\mathcal{H}} = \sum_{j=0}^{n-1} r_j \langle\xi_{\lambda_j},\psi_k^\lambda\rangle_{\mathcal{H}} \mathrel{\overset{\text{Thm. 5.17}}{=}} r_k. ⟨ G , ψ k λ ⟩ H = ⟨ j = 0 ∑ n − 1 r j ξ λ j , ψ k λ ⟩ H = j = 0 ∑ n − 1 r j ⟨ ξ λ j , ψ k λ ⟩ H = Thm. 5.17 r k .
Corollary 5.20. For G ∈ H G \in\mathcal{H} G ∈ H , the operator B n λ : H → V n λ \mathcal{B}_{n}^{\lambda}:\mathcal{H} \to V_{n}^{\lambda} B n λ : H → V n λ given by
B n λ [ G ] = ∑ k = 0 n − 1 ⟨ G , ψ k λ ⟩ H ξ λ k \mathcal{B}_{n}^{\lambda}[G] = \sum_{k=0}^{n-1} \langle G,\psi_k^\lambda\rangle_{\mathcal{H}}\xi_{\lambda_k} B n λ [ G ] = k = 0 ∑ n − 1 ⟨ G , ψ k λ ⟩ H ξ λ k is the orthogonal projector to V n λ V_{n}^{\lambda} V n λ , that is,
B n λ = P V n λ . \mathcal{B}_{n}^{\lambda} = \mathcal{P}_{V_{n}^{\lambda}}. B n λ = P V n λ .
We finish our discussion on the properties of GMT functions with a note on numerical aspects. Prior works have refrained from the pursuit of Definition 5.2, citing the numerical instability of Gram–Schmidt orthogonalization in RKHSs [55 ] . While it is certainly true that the standard Gram–Schmidt procedure becomes numerically unstable when applied to basis vectors that are nearly linearly dependent, in § 8 we give empirical evidence that computing several hundred and in some cases several thousand GMT functions is entirely possible, with standard Cholesky–factorization–based orthogonalization [64 ] , which we use in our implementation to obtain GMT systems from kernel slices. Our experiments cover a variety of RKHSs, and in a few cases, for example due to the distinct support of kernel slices, the orthogonalization procedure behaves extremely well even for large n n n . We also note that randomized orthogonalization techniques based on sketching have recently enjoyed much attention from the research community [37 , 70 ] . Documenting these results, and adapting them to the RKHS setting if needed, should be a future research priority.
Finally, we note that the biorthogonal system in Theorem 5.17 can be used to address numerical concerns of the MOR methods proposed in this study. Indeed, the Lagrange basis polynomials ℓ k \ell_k ℓ k in (43) can be efficiently computed using a barycentric–formula–based approach [9 , 48 ] , even for large n n n . Then, Lemma 5.19 can be used to express models in G ξ n \mathfrak{G}_{\xi}^{n} G ξ n in a numerically safe manner. We also note that an analog of the GMT transformation introduced in the next section can be defined with the BGMT system in (43) to transform MOR problems to–and–from M i d \mathcal{M}_{\mathrm{id}} M id –invariant RKHSs.
Finally, in many specific RKHSs, efficient algorithms have already been developed to evaluate special cases of the GMT functions. In H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) , discrete orthogonal variants have been considered in [22 ] . It has also been shown that special sampling schemes within Ω = D \Omega= \mathbb{D} Ω = D induce a GMT-based multi-resolution of H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) [50 ] and the Bergman space [51 ] . For Laguerre bases, fast methods for computing expansion coefficients have recently been proposed in [21 ] . A more general construction based on the fast Fourier transform (FFT) for rational systems is given in [57 ] . Generalizing some of these results for RKHSs could bypass the need for computing φ n λ \varphi_n^{\lambda} φ n λ in a kernel basis. Instead, using an appropriate discrete sampling of a G ∈ H G \in\mathcal{H} G ∈ H , one could obtain the GMT expansion coefficients of G G G directly.
The GMT transformation
We are now ready to introduce a family of unitary transformations that will allow us to develop the desired MOR methods in § 7 . First, we make the following set of assumptions.
Assumption 5.21. Let Ω , Ω ^ ⊆ C \Omega, \widehat{\Omega} \subseteq\mathbb{C} Ω , Ω ⊆ C and suppose that H \mathcal{H} H and H ^ \widehat{\mathcal{H}} H are RKHSs of functions Ω → C \Omega\to\mathbb{C} Ω → C and Ω ^ → C \widehat{\Omega} \to\mathbb{C} Ω → C , respectively. Let γ : Ω → Ω ^ \gamma: \Omega\to\widehat{\Omega} γ : Ω → Ω be a bijection. Denote by ξ \xi ξ and ξ ^ \widehat{\xi} ξ the reproducing kernels of H \mathcal{H} H and H ^ \widehat{\mathcal{H}} H . Finally, let { φ k λ } k = 0 n − 1 \{\varphi_k^\lambda\}_{k=0}^{n-1} { φ k λ } k = 0 n − 1 and { φ ^ k γ ( λ ) } k = 0 n − 1 \{\widehat{\varphi}_k^{\gamma(\lambda)}\}_{k=0}^{n-1} { φ k γ ( λ ) } k = 0 n − 1 be two GMT systems spanning
V n λ = span { φ k λ } k = 0 n − 1 ⊆ H , V ^ n γ ( λ ) = span { φ ^ k γ ( λ ) } k = 0 n − 1 ⊆ H ^ . (44) V_n^\lambda= \operatorname{span}\{\varphi_k^\lambda\}_{k=0}^{n-1} \subseteq\mathcal{H}, \qquad\widehat{V}_n^{\gamma(\lambda)} = \operatorname{span}\{\widehat{\varphi}_k^{\gamma(\lambda)}\}_{k=0}^{n-1} \subseteq\widehat{\mathcal{H}}. \tag*{(44)} V n λ = span { φ k λ } k = 0 n − 1 ⊆ H , V n γ ( λ ) = span { φ k γ ( λ ) } k = 0 n − 1 ⊆ H . (44)
Definition 5.22 (GMT transformation). Consider Assumption 5.21. We define the GMT transformation F γ , λ : V n λ → V ^ n γ ( λ ) \mathcal{F}_{\gamma,\lambda} : V_n^\lambda\to\widehat{V}_n^{\gamma(\lambda)} F γ , λ : V n λ → V n γ ( λ ) as F γ , λ [ φ k λ ] : = φ ^ k γ ( λ ) \mathcal{F}_{\gamma,\lambda}[\varphi_k^\lambda] := \widehat{\varphi}_k^{\gamma(\lambda)} F γ , λ [ φ k λ ] := φ k γ ( λ ) for k = 0 , … , n − 1 k = 0, \ldots, n - 1 k = 0 , … , n − 1 , and for an arbitrary G ∈ V n λ G \in V_n^\lambda G ∈ V n λ as
F γ , λ [ G ] = F γ , λ [ ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ k λ ] = ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H F γ , λ [ φ k λ ] \mathcal{F}_{\gamma,\lambda}[G] = \mathcal{F}_{\gamma,\lambda}\left[\sum_{k=0}^{n-1}\langle G,\varphi_k^\lambda\rangle_{\mathcal{H}}\varphi_k^\lambda\right] = \sum_{k=0}^{n-1}\langle G,\varphi_k^\lambda\rangle_{\mathcal{H}}\mathcal{F}_{\gamma,\lambda}[\varphi_k^\lambda] F γ , λ [ G ] = F γ , λ [ k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k λ ] = k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H F γ , λ [ φ k λ ] = ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ ^ k γ ( λ ) = G ^ ∈ V ^ n γ ( λ ) ⊆ H ^ . (45) = \sum_{k=0}^{n-1}\langle G,\varphi_k^\lambda\rangle_{\mathcal{H}}\widehat{\varphi}_k^{\gamma(\lambda)} = \widehat{G} \in\widehat{V}_n^{\gamma(\lambda)} \subseteq\widehat{\mathcal{H}}. \tag*{(45)} = k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k γ ( λ ) = G ∈ V n γ ( λ ) ⊆ H . (45)
From Definition 5.22, we immediately obtain the following property.
Theorem 5.23. Under Assumption 5.21, the GMT transformation in Definition 5.22 is linear and unitary, the latter meaning that
⟨ G , H ⟩ H = ⟨ F γ , λ [ G ] , F γ , λ [ H ] ⟩ H ^ ( G , H ∈ V n λ ⊆ H ) . \langle G,H\rangle_{\mathcal{H}} = \langle\mathcal{F}_{\gamma,\lambda}[G],\mathcal{F}_{\gamma,\lambda}[H]\rangle_{\widehat{\mathcal{H}}} \qquad(G,H \in V_n^\lambda\subseteq\mathcal{H}). ⟨ G , H ⟩ H = ⟨ F γ , λ [ G ] , F γ , λ [ H ] ⟩ H ( G , H ∈ V n λ ⊆ H ) .
Proof. Linearity holds, because by (45) , for any G , H ∈ V n λ \mathbf{G},\mathbf{H}\in V_n^\lambda G , H ∈ V n λ and w ∈ C w\in\mathbb{C} w ∈ C we have
F γ , λ [ G + w H ] = F γ , λ [ ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ k λ + w ∑ k = 0 n − 1 ⟨ H , φ k λ ⟩ H φ k λ ] = ∑ k = 0 n − 1 ( ⟨ G , φ k λ ⟩ H + w ⟨ H , φ k λ ⟩ H ) F γ , λ [ φ k λ ] = ∑ k = 0 n − 1 ( ⟨ G , φ k λ ⟩ H + w ⟨ H , φ k λ ⟩ H ) φ ~ k γ ( λ ) = ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ ~ k γ ( λ ) + w ∑ k = 0 n − 1 ⟨ H , φ k λ ⟩ H φ ~ k γ ( λ ) = F γ , λ [ G ] + w F γ , λ [ H ] . \begin{aligned}
\mathcal{F}_{\gamma,\lambda}[\mathbf{G}+w\mathbf{H}] &= \mathcal{F}_{\gamma,\lambda}\left[\sum_{k=0}^{n-1}\langle\mathbf{G},\varphi_k^\lambda\rangle_{\mathcal{H}}\varphi_k^\lambda+w\sum_{k=0}^{n-1}\langle\mathbf{H},\varphi_k^\lambda\rangle_{\mathcal{H}}\varphi_k^\lambda\right]\\
&=\sum_{k=0}^{n-1}\left(\langle\mathbf{G},\varphi_k^\lambda\rangle_{\mathcal{H}}+w\langle\mathbf{H},\varphi_k^\lambda\rangle_{\mathcal{H}}\right)\mathcal{F}_{\gamma,\lambda}[\varphi_k^\lambda]\\
&=\sum_{k=0}^{n-1}\left(\langle\mathbf{G},\varphi_k^\lambda\rangle_{\mathcal{H}}+w\langle\mathbf{H},\varphi_k^\lambda\rangle_{\mathcal{H}}\right)\widetilde{\varphi}_k^{\gamma(\lambda)}\\
&=\sum_{k=0}^{n-1}\langle\mathbf{G},\varphi_k^\lambda\rangle_{\mathcal{H}}\widetilde{\varphi}_k^{\gamma(\lambda)}+w\sum_{k=0}^{n-1}\langle\mathbf{H},\varphi_k^\lambda\rangle_{\mathcal{H}}\widetilde{\varphi}_k^{\gamma(\lambda)}=\mathcal{F}_{\gamma,\lambda}[\mathbf{G}]+w\mathcal{F}_{\gamma,\lambda}[\mathbf{H}].
\end{aligned} F γ , λ [ G + w H ] = F γ , λ [ k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k λ + w k = 0 ∑ n − 1 ⟨ H , φ k λ ⟩ H φ k λ ] = k = 0 ∑ n − 1 ( ⟨ G , φ k λ ⟩ H + w ⟨ H , φ k λ ⟩ H ) F γ , λ [ φ k λ ] = k = 0 ∑ n − 1 ( ⟨ G , φ k λ ⟩ H + w ⟨ H , φ k λ ⟩ H ) φ k γ ( λ ) = k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k γ ( λ ) + w k = 0 ∑ n − 1 ⟨ H , φ k λ ⟩ H φ k γ ( λ ) = F γ , λ [ G ] + w F γ , λ [ H ] .
The unitarity follows from the fact that F γ , λ \mathcal{F}_{\gamma,\lambda} F γ , λ maps an orthonormal system in V n λ V_n^\lambda V n λ to one in V ^ n γ ( λ ) \widehat{V}_n^{\gamma(\lambda)} V n γ ( λ ) . Indeed, suppose that G , H ∈ V n λ ⊆ H \mathbf{G},\mathbf{H}\in V_n^\lambda\subseteq\mathcal{H} G , H ∈ V n λ ⊆ H . Then,
⟨ F γ , λ [ G ] , F γ , λ [ H ] ⟩ H ^ = ⟨ ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ ~ k γ ( λ ) , ∑ j = 0 n − 1 ⟨ H , φ j λ ⟩ H φ ~ j γ ( λ ) ⟩ H ^ = ∑ k = 0 n − 1 ∑ j = 0 n − 1 ⟨ G , φ k λ ⟩ H ⟨ H , φ j λ ⟩ H ‾ ⟨ φ ~ k γ ( λ ) , φ ~ j γ ( λ ) ⟩ H ^ = ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H ⟨ H , φ k λ ⟩ H ‾ = ∑ k = 0 n − 1 ∑ j = 0 n − 1 ⟨ G , φ k λ ⟩ H ⟨ H , φ j λ ⟩ H ‾ ⟨ φ k λ , φ j λ ⟩ H = ⟨ G , H ⟩ H . (46) \begin{aligned}
\left\langle\mathcal{F}_{\gamma,\lambda}[\mathbf{G}],\mathcal{F}_{\gamma,\lambda}[\mathbf{H}]\right\rangle_{\widehat{\mathcal{H}}}
&=\left\langle\sum_{k=0}^{n-1}\langle\mathbf{G},\varphi_k^\lambda\rangle_{\mathcal{H}}\widetilde{\varphi}_k^{\gamma(\lambda)},\sum_{j=0}^{n-1}\langle\mathbf{H},\varphi_j^\lambda\rangle_{\mathcal{H}}\widetilde{\varphi}_j^{\gamma(\lambda)}\right\rangle_{\widehat{\mathcal{H}}}\\
&=\sum_{k=0}^{n-1}\sum_{j=0}^{n-1}\langle\mathbf{G},\varphi_k^\lambda\rangle_{\mathcal{H}}\overline{\langle\mathbf{H},\varphi_j^\lambda\rangle_{\mathcal{H}}}\left\langle\widetilde{\varphi}_k^{\gamma(\lambda)},\widetilde{\varphi}_j^{\gamma(\lambda)}\right\rangle_{\widehat{\mathcal{H}}}\\
&=\sum_{k=0}^{n-1}\langle\mathbf{G},\varphi_k^\lambda\rangle_{\mathcal{H}}\overline{\langle\mathbf{H},\varphi_k^\lambda\rangle_{\mathcal{H}}}\\
&=\sum_{k=0}^{n-1}\sum_{j=0}^{n-1}\langle\mathbf{G},\varphi_k^\lambda\rangle_{\mathcal{H}}\overline{\langle\mathbf{H},\varphi_j^\lambda\rangle_{\mathcal{H}}}\langle\varphi_k^\lambda,\varphi_j^\lambda\rangle_{\mathcal{H}}=\langle\mathbf{G},\mathbf{H}\rangle_{\mathcal{H}}. \tag*{(46)}
\end{aligned} ⟨ F γ , λ [ G ] , F γ , λ [ H ] ⟩ H = ⟨ k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k γ ( λ ) , j = 0 ∑ n − 1 ⟨ H , φ j λ ⟩ H φ j γ ( λ ) ⟩ H = k = 0 ∑ n − 1 j = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H ⟨ H , φ j λ ⟩ H ⟨ φ k γ ( λ ) , φ j γ ( λ ) ⟩ H = k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H ⟨ H , φ k λ ⟩ H = k = 0 ∑ n − 1 j = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H ⟨ H , φ j λ ⟩ H ⟨ φ k λ , φ j λ ⟩ H = ⟨ G , H ⟩ H . (46)
Using Definition 5.22 , we can transform a MOR problem posed in an RKHS H \mathcal{H} H to another one, where we might be able to solve the problem more easily. To this end, we first define the concept of a weak model space from Definition 2.5 generated by the GMT transformation.
For the remainder of this study, we note that the definition of the weak model spaces in Definition 2.5 can in a straight-forward manner be generalized to the case of repeated kernel centers. In the rest of this work, we assume that the unitary maps F \mathcal{F} F in this definition are chosen as GMT transformations.
Example 5.24 (MOR and the GMT transformation). Suppose we are given a FOM G \mathbf{G} G in the RKHS H \mathcal{H} H defined as
H : = { F ∈ L 2 ( Ω ) : F ′ ∈ L 2 ( Ω ) , F ( 0 ) 2 + ∫ 0 1 F ′ ( t ) 2 d t < ∞ } . \mathcal{H}:=\left\{\mathrm{F}\in L^2(\Omega):\mathrm{F}'\in L^2(\Omega),\ \mathrm{F}(0)^2+\int_0^1\mathrm{F}'(t)^2\,\mathrm{d}t<\infty\right\}. H := { F ∈ L 2 ( Ω ) : F ′ ∈ L 2 ( Ω ) , F ( 0 ) 2 + ∫ 0 1 F ′ ( t ) 2 d t < ∞ } .
where Ω : = [ 0 , 1 ] \Omega:= [0,1] Ω := [ 0 , 1 ] and differentiation is understood in the weak sense. This is indeed an [68 Section 1.2] RKHS of functions Ω → R \Omega\to\mathbb{R} Ω → R with kernel
ξ ( t , λ ) : = 1 + min { t , λ } ( t , λ ∈ Ω ) , \xi(t,\lambda) := 1+\min\{t,\lambda\}\qquad(t,\lambda\in\Omega), ξ ( t , λ ) := 1 + min { t , λ } ( t , λ ∈ Ω ) ,
see, e.g., [68 Section 1.2] . This RKHS contains first-order splines and also certain deep ReLU networks. For a deeper discussion on H \mathcal{H} H , we refer to § 9.1 . Here, we only note that for any F ∈ H F\in\mathcal{H} F ∈ H , we have
∥ F ∥ H : = F ( 0 ) 2 + ∫ 0 1 F ′ ( t ) 2 d t . (47) \lVert F\rVert_{\mathcal{H}} := \sqrt{F(0)^2+\int_0^1 F'(t)^2\,\mathrm{d}t}. \tag*{(47)} ∥ F ∥ H := F ( 0 ) 2 + ∫ 0 1 F ′ ( t ) 2 d t . (47)
Let the FOM be defined as
G : = ξ 1 4 + ξ 3 4 , (48) \mathbf{G}:=\xi_{\frac{1}{4}}+\xi_{\frac{3}{4}}, \tag*{(48)} G := ξ 4 1 + ξ 4 3 , (48)
that is, we consider a FOM of order n = 2 n=2 n = 2 with kernel centers λ 0 = 1 4 \lambda_0=\frac{1}{4} λ 0 = 4 1 and λ 1 = 3 4 \lambda_1=\frac{3}{4} λ 1 = 4 3 . Suppose that the ROM is of order r = 1 r=1 r = 1 , i.e.,
G r : = r ~ 0 ξ λ ~ 0 \mathbf{G}_r:=\widetilde{r}_0\xi_{\widetilde{\lambda}_0} G r := r 0 ξ λ 0
is an H \mathcal{H} H –norm–optimal approximation of G \mathbf{G} G . We clearly cannot apply our optimality conditions from § 4 to verify that G r \mathbf{G}_r G r fulfills at least the proposed necessary conditions of optimality. Indeed, each of the applicable conditions needs to evaluate the first-order derivative of G r \mathbf{G}_r G r (or at least of ξ λ ~ 0 \xi_{\widetilde{\lambda}_0} ξ λ 0 ) at λ ~ 0 \widetilde{\lambda}_0 λ 0 , which does not exist. Hence, it becomes necessary to transform the problem to an admissible RKHS H ^ \widehat{\mathcal{H}} H .
Since H \mathcal{H} H is an RKHS, we can apply Definition 5.2 to write G \mathbf{G} G in a GMT expansion. A quick calculation using (47) and Definition 5.2 verifies that the GMT system defined by λ 0 \lambda_0 λ 0 and λ 1 \lambda_1 λ 1 can be written as
φ 0 λ ( t ) = 2 5 + 2 5 min { t , 1 4 } \varphi_0^{\lambda}(t)=\frac{2}{\sqrt{5}}+\frac{2}{\sqrt{5}}\min\left\{t,\frac{1}{4}\right\} φ 0 λ ( t ) = 5 2 + 5 2 min { t , 4 1 }
and
φ 1 λ ( t ) = 2 min { t , 3 4 } − 2 min { t , 1 4 } . \varphi_1^{\lambda}(t)=\sqrt{2}\min\left\{t,\frac{3}{4}\right\}-\sqrt{2}\min\left\{t,\frac{1}{4}\right\}. φ 1 λ ( t ) = 2 min { t , 4 3 } − 2 min { t , 4 1 } .
By (48) , the reproducing property, and the fact that functions in H \mathcal{H} H are real-valued, we obtain
⟨ G , φ 0 λ ⟩ H = 5 , ⟨ G , φ 1 λ ⟩ H = 1 2 . \langle\mathbf{G},\varphi_0^{\lambda}\rangle_{\mathcal{H}}=\sqrt{5},\qquad\langle\mathbf{G},\varphi_1^{\lambda}\rangle_{\mathcal{H}}=\frac{1}{\sqrt{2}}. ⟨ G , φ 0 λ ⟩ H = 5 , ⟨ G , φ 1 λ ⟩ H = 2 1 .
Hence, we can write
G = 5 φ 0 λ + 1 2 φ 1 λ . \mathbf{G}=\sqrt{5}\varphi_0^{\lambda}+\frac{1}{\sqrt{2}}\varphi_1^{\lambda}. G = 5 φ 0 λ + 2 1 φ 1 λ .
The map γ ( t ) = 2 t − 1 \gamma(t)=2t-1 γ ( t ) = 2 t − 1 is a bijection between Ω \Omega Ω restricted to ( 0 , 1 ) (0,1) ( 0 , 1 ) and I = ( − 1 , 1 ) I=(-1,1) I = ( − 1 , 1 ) . This allows us to choose the real analytic space H 1 2 \mathcal{H}^{\frac{1}{2}} H 2 1 from (14) as the companion space H ^ \widehat{\mathcal{H}} H . Recall that this is an RKHS with kernel
ρ ( x , λ ) = 1 1 − x λ ( x , λ ∈ I ) . (49) \rho(x,\lambda)=\frac{1}{1-x\lambda}\qquad(x,\lambda\in I). \tag*{(49)} ρ ( x , λ ) = 1 − x λ 1 ( x , λ ∈ I ) . (49)
Using the GMT transformation from Definition 5.22 , we obtain a surrogate FOM G ^ ∈ H ^ \widehat{\mathbf{G}} \in\widehat{\mathcal{H}} G ∈ H as
G ^ = F γ , λ [ G ] = 5 φ ^ 0 γ ( λ ) + 1 2 φ ^ 1 γ ( λ ) . \widehat{\mathbf{G}}=\mathcal{F}_{\gamma,\lambda}[\mathbf{G}]=\sqrt{5}\widehat{\varphi}_{0}^{\gamma(\lambda)}+\frac{1}{\sqrt{2}}\widehat{\varphi}_{1}^{\gamma(\lambda)}. G = F γ , λ [ G ] = 5 φ 0 γ ( λ ) + 2 1 φ 1 γ ( λ ) .
where γ ( 1 4 ) = − 1 2 \gamma\left(\frac{1}{4}\right)=-\frac{1}{2} γ ( 4 1 ) = − 2 1 , γ ( 3 4 ) = 1 2 \gamma\left(\frac{3}{4}\right)=\frac{1}{2} γ ( 4 3 ) = 2 1 , and by (38) ,
φ ^ 0 γ ( λ ) ( t ) = 1 − ∣ 1 2 ∣ 2 1 + t 2 = 3 2 + t , φ ^ 1 γ ( λ ) ( t ) = 1 − ∣ 1 2 ∣ 2 1 − t 2 t + 1 2 1 + t 2 = 2 3 t + 3 4 − t 2 . \widehat{\varphi}_{0}^{\gamma(\lambda)}(t)=\frac{\sqrt{1-\left|\frac{1}{2}\right|^{2}}}{1+\frac{t}{2}}=\frac{\sqrt{3}}{2+t},\qquad\widehat{\varphi}_{1}^{\gamma(\lambda)}(t)=\frac{\sqrt{1-\left|\frac{1}{2}\right|^{2}}}{1-\frac{t}{2}}\frac{t+\frac{1}{2}}{1+\frac{t}{2}}=\frac{2\sqrt{3}t+\sqrt{3}}{4-t^{2}}. φ 0 γ ( λ ) ( t ) = 1 + 2 t 1 − 2 1 2 = 2 + t 3 , φ 1 γ ( λ ) ( t ) = 1 − 2 t 1 − 2 1 2 1 + 2 t t + 2 1 = 4 − t 2 2 3 t + 3 .
Now suppose that, using Algorithm 3.1, we identified an H 2 1 H^1_2 H 2 1 –norm–optimal ROM to G ^ \widehat{\mathbf{G}} G denoted by
G ^ r = c 0 φ ^ 0 γ ( λ ~ ) , (50) \widehat{\mathbf{G}}_r=c_0\widehat{\varphi}_{0}^{\gamma(\widetilde{\lambda})}, \tag*{(50)} G r = c 0 φ 0 γ ( λ ) , (50)
where by Definition 5.2 ,
φ ^ 0 γ ( λ ~ ) = ρ γ ( λ ~ ) ρ ( γ ( λ ~ 0 ) , γ ( λ ~ 0 ) ) (51) \widehat{\varphi}_{0}^{\gamma(\widetilde{\lambda})}=\frac{\rho_{\gamma(\widetilde{\lambda})}}{\sqrt{\rho(\gamma(\widetilde{\lambda}_0),\gamma(\widetilde{\lambda}_0))}} \tag*{(51)} φ 0 γ ( λ ) = ρ ( γ ( λ 0 ) , γ ( λ 0 )) ρ γ ( λ ) (51)
holds for some γ ( λ ~ 0 ) ∈ I \gamma(\widetilde{\lambda}_0)\in\mathbb{I} γ ( λ 0 ) ∈ I . This is possible because the optimality conditions of § 4 can be applied, as the kernel ρ \rho ρ is sufficiently smooth in both variables. The norm optimality of G ^ r \widehat{\mathbf{G}}_r G r implies that G ^ r = P V ^ γ ( λ ~ ) [ G ^ ] \widehat{\mathbf{G}}_r=\mathcal{P}_{\widehat{V}_{\gamma(\widetilde{\lambda})}}[\widehat{\mathbf{G}}] G r = P V γ ( λ ) [ G ] , which by (49) , (50) , and (51) is equivalent to
c 0 = 1 − ∣ γ ( λ ~ 0 ) ∣ 2 G ^ ( γ ( λ ~ 0 ) ) . c_0=\sqrt{1-|\gamma(\widetilde{\lambda}_0)|^2}\widehat{\mathbf{G}}(\gamma(\widetilde{\lambda}_0)). c 0 = 1 − ∣ γ ( λ 0 ) ∣ 2 G ( γ ( λ 0 )) .
A natural idea to obtain an optimal ROM to G \mathbf{G} G in H \mathcal{H} H would now be to take
G r = F γ , λ − 1 [ G ^ r ] = 1 − ∣ γ ( λ ~ 0 ) ∣ 2 G ^ ( γ ( λ ~ 0 ) ) φ ^ 0 γ ( λ ~ 0 ) ⏟ = c 0 . \mathbf{G}_r=\mathcal{F}_{\gamma,\lambda}^{-1}[\widehat{\mathbf{G}}_r]=\sqrt{1-|\gamma(\widetilde{\lambda}_0)|^2}\widehat{\mathbf{G}}(\gamma(\widetilde{\lambda}_0))\underbrace{\widehat{\varphi}_{0}^{\gamma(\widetilde{\lambda}_0)}}_{=c_0}. G r = F γ , λ − 1 [ G r ] = 1 − ∣ γ ( λ 0 ) ∣ 2 G ( γ ( λ 0 )) = c 0 φ 0 γ ( λ 0 ) .
Unfortunately, this choice cannot guarantee that ∥ G − G r ∥ H = ∥ G ^ − G ^ r ∥ H ^ \|\mathbf{G}-\mathbf{G}_r\|_{\mathcal{H}}=\|\widehat{\mathbf{G}}-\widehat{\mathbf{G}}_r\|_{\widehat{\mathcal{H}}} ∥ G − G r ∥ H = ∥ G − G r ∥ H , because the function system { φ ^ 0 γ ( λ ) , φ ^ 0 γ ( λ ) , φ ^ 1 γ ( λ ) } \left\{\widehat{\varphi}_{0}^{\gamma(\lambda)},\widehat{\varphi}_{0}^{\gamma(\lambda)},\widehat{\varphi}_{1}^{\gamma(\lambda)}\right\} { φ 0 γ ( λ ) , φ 0 γ ( λ ) , φ 1 γ ( λ ) } is not orthogonal in H ^ \widehat{\mathcal{H}} H and therefore, the transformation mapping these to { φ 0 λ , φ 0 λ , φ 1 λ } \left\{\varphi_{0}^{\lambda},\varphi_{0}^{\lambda},\varphi_{1}^{\lambda}\right\} { φ 0 λ , φ 0 λ , φ 1 λ } is no longer unitary between the subspaces H \mathcal{H} H and H ^ \widehat{\mathcal{H}} H that contain G \mathbf{G} G , G r \mathbf{G}_r G r and G ^ \widehat{\mathbf{G}} G , G ^ r \widehat{\mathbf{G}}_r G r , respectively. Instead, we have to write G ^ r \widehat{\mathbf{G}}_r G r in a new GMT basis generated by the kernel centers
γ ( μ ) = ( γ ( λ 0 ) , γ ( λ 1 ) , γ ( λ ~ 0 ) ) ∈ I 3 = Ω ^ 3 . \gamma(\boldsymbol{\mu})=\left(\gamma(\lambda_0),\gamma(\lambda_1),\gamma(\widetilde{\lambda}_0)\right)\in\mathbb{I}^{3}=\widehat{\Omega}^{3}. γ ( μ ) = ( γ ( λ 0 ) , γ ( λ 1 ) , γ ( λ 0 ) ) ∈ I 3 = Ω 3 .
Using this basis, we can fully describe both G ^ \widehat{\mathbf{G}} G and G ^ r \widehat{\mathbf{G}}_r G r with the first three, now orthogonal, GMT functions generated by γ ( μ ) \gamma(\boldsymbol{\mu}) γ ( μ ) . Thus, the norm–optimal approximant to G \mathbf{G} G in H \mathcal{H} H satisfying E ξ r ( G r ) = E ^ ρ ^ r ( G ^ r ) : = min H ^ ∈ G ^ ρ ^ r ∥ G ^ − H ^ ∥ H ^ \mathcal{E}^{r}_{\xi}(\mathbf{G}_r)=\widehat{\mathcal{E}}^{r}_{\widehat{\rho}}(\widehat{\mathbf{G}}_r):=\min_{\widehat{\mathbf{H}}\in\widehat{\mathcal{G}}^{r}_{\widehat{\rho}}}\|\widehat{\mathbf{G}}-\widehat{\mathbf{H}}\|_{\widehat{\mathcal{H}}} E ξ r ( G r ) = E ρ r ( G r ) := min H ∈ G ρ r ∥ G − H ∥ H is given by
G r w e a k : = F γ , μ − 1 [ G ^ r ] = ⟨ G ^ r , φ ^ 0 γ ( μ ) ⟩ H ^ φ 0 μ + ⟨ G ^ r , φ ^ 1 γ ( μ ) ⟩ H ^ φ 1 μ + ⟨ G ^ r , φ ^ 2 γ ( μ ) ⟩ H ^ φ 2 μ . (52) \mathbf{G}^{\mathrm{weak}}_r:=\mathcal{F}^{-1}_{\gamma,\mu}[\widehat{\mathbf{G}}_r]=\left\langle\widehat{\mathbf{G}}_r,\widehat{\varphi}_{0}^{\gamma(\boldsymbol{\mu})}\right\rangle_{\widehat{\mathcal{H}}}\varphi_{0}^{\mu}+\left\langle\widehat{\mathbf{G}}_r,\widehat{\varphi}_{1}^{\gamma(\boldsymbol{\mu})}\right\rangle_{\widehat{\mathcal{H}}}\varphi_{1}^{\mu}+\left\langle\widehat{\mathbf{G}}_r,\widehat{\varphi}_{2}^{\gamma(\boldsymbol{\mu})}\right\rangle_{\widehat{\mathcal{H}}}\varphi_{2}^{\mu}. \tag*{(52)} G r weak := F γ , μ − 1 [ G r ] = ⟨ G r , φ 0 γ ( μ ) ⟩ H φ 0 μ + ⟨ G r , φ 1 γ ( μ ) ⟩ H φ 1 μ + ⟨ G r , φ 2 γ ( μ ) ⟩ H φ 2 μ . (52)
Notice, however, that this G r w e a k G_r^{\mathrm{weak}} G r weak belongs to a weak solution space in H \mathcal{H} H as defined in Definition 2.5 generated by the GMT transformation. Indeed, nothing guarantees that the linear combination in (52) belongs to a one-dimensional subspace in H \mathcal{H} H spanned by a single kernel slice ξ ^ λ ~ \widehat{\xi}_{\widetilde{\lambda}} ξ λ for some λ ~ ∈ Ω \widetilde{\lambda} \in\Omega λ ∈ Ω . Below, we investigate how weak solutions can still be used to perform MOR, and how they can be approximated by strong counterparts belonging to G ξ ^ ⊂ H \mathfrak{G}_{\widehat{\xi}} \subset\mathcal{H} G ξ ⊂ H .
For a MOR problem posed in a general RKHS H \mathcal{H} H , we can choose an admissible H ^ \widehat{\mathcal{H}} H such that its kernel ξ ^ \widehat{\xi} ξ satisfies the differentiability conditions needed for Theorem 4.3 . More generally, suppose a FOM G \mathbf{G} G belongs to the n n n –dimensional subspace V n λ ⊆ G ξ n V_n^\lambda\subseteq\mathfrak{G}_{\xi}^{n} V n λ ⊆ G ξ n in the RKHS H \mathcal{H} H . Suppose further that for some λ ∈ Ω n \lambda\in\Omega^n λ ∈ Ω n we have G = S n λ [ G ] \mathbf{G} = \mathcal{S}_n^\lambda[\mathbf{G}] G = S n λ [ G ] , where the operator S n λ \mathcal{S}_n^\lambda S n λ is defined according to (37) . Applying the GMT transformation defined by this tuple yields F γ , λ [ G ] = G ^ ∈ V ^ n γ ( λ ) ⊆ G ξ ^ n ⊆ H ^ \mathcal{F}_{\gamma,\lambda}[\mathbf{G}] = \widehat{\mathbf{G}} \in\widehat{V}_{n}^{\gamma(\lambda)} \subseteq\mathfrak{G}_{\widehat{\xi}}^n \subseteq\widehat{\mathcal{H}} F γ , λ [ G ] = G ∈ V n γ ( λ ) ⊆ G ξ n ⊆ H . Suppose that we obtain a ROM G ^ r ∈ V ^ r γ ( λ ~ ) ⊆ G ξ ^ r \widehat{\mathbf{G}}_r \in\widehat{V}_{r}^{\gamma(\widetilde{\lambda})} \subseteq\mathfrak{G}_{\widehat{\xi}}^r G r ∈ V r γ ( λ ) ⊆ G ξ r , where the first r r r components of the tuple γ ( λ ~ ) \gamma(\widetilde{\lambda}) γ ( λ ) describe an r r r –dimensional subspace V ^ r γ ( λ ~ ) ∈ G ξ ^ r ⊆ H ^ \widehat{V}_{r}^{\gamma(\widetilde{\lambda})} \in\mathfrak{G}_{\widehat{\xi}}^r \subseteq\widehat{\mathcal{H}} V r γ ( λ ) ∈ G ξ r ⊆ H (see Definition 2.3 ).
In order to recover the corresponding ROM in H \mathcal{H} H , consider first the extended kernel center tuple
μ : = ( μ 0 , μ 1 , … , μ n + r − 1 ) : = ( λ 0 , … , λ n − 1 , λ ~ 0 , … , λ ~ r − 1 ) ∈ Ω n + r . (53) \boldsymbol{\mu} := (\mu_0,\mu_1,\ldots,\mu_{n+r-1}) := (\lambda_0,\ldots,\lambda_{n-1},\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{r-1}) \in\Omega^{n+r}. \tag*{(53)} μ := ( μ 0 , μ 1 , … , μ n + r − 1 ) := ( λ 0 , … , λ n − 1 , λ 0 , … , λ r − 1 ) ∈ Ω n + r . (53)
The kernel slices defined by the first n + r n+r n + r components of μ \boldsymbol{\mu} μ and γ ( μ ) \gamma(\boldsymbol{\mu}) γ ( μ ) , respectively, define the ( n + r ) (n+r) ( n + r ) –dimensional subspaces
V n + r μ = span { φ k μ } k = 0 n + r − 1 ⊆ H , V ^ n + r γ ( μ ) = span { φ ^ k γ ( μ ) } k = 0 n + r − 1 ⊆ H ^ . V_{n+r}^{\boldsymbol{\mu}} = \operatorname{span}\left\{\varphi_k^{\boldsymbol{\mu}}\right\}_{k=0}^{n+r-1} \subseteq\mathcal{H}, \qquad\widehat{V}_{n+r}^{\gamma(\boldsymbol{\mu})} = \operatorname{span}\left\{\widehat{\varphi}_k^{\gamma(\boldsymbol{\mu})}\right\}_{k=0}^{n+r-1} \subseteq\widehat{\mathcal{H}}. V n + r μ = span { φ k μ } k = 0 n + r − 1 ⊆ H , V n + r γ ( μ ) = span { φ k γ ( μ ) } k = 0 n + r − 1 ⊆ H .
Notice that we have G ∈ V n + r μ \mathbf{G} \in V_{n+r}^{\boldsymbol{\mu}} G ∈ V n + r μ and G ^ , G ^ r , G ^ − G ^ r ∈ V ^ n + r γ ( μ ) \widehat{\mathbf{G}},\widehat{\mathbf{G}}_r,\widehat{\mathbf{G}}-\widehat{\mathbf{G}}_r \in\widehat{V}_{n+r}^{\gamma(\boldsymbol{\mu})} G , G r , G − G r ∈ V n + r γ ( μ ) . Due to the ordering of μ \boldsymbol{\mu} μ in (53) , the first n n n functions of these systems span V n λ V_n^\lambda V n λ and V ^ n γ ( λ ) \widehat{V}_n^{\gamma(\lambda)} V n γ ( λ ) . Using Definition 5.22 , we can define a unitary GMT transformation between V n + r μ V_{n+r}^{\boldsymbol{\mu}} V n + r μ and V ^ n + r γ ( μ ) \widehat{V}_{n+r}^{\gamma(\boldsymbol{\mu})} V n + r γ ( μ ) denoted by F γ , μ \mathcal{F}_{\gamma,\boldsymbol{\mu}} F γ , μ . Again, because of the ordering chosen in (53) , restricting this operator to V n λ V_n^\lambda V n λ yields F γ , λ \mathcal{F}_{\gamma,\lambda} F γ , λ . Thus, we have
G ^ = F γ , λ [ G ] = F γ , μ [ G ] , G = F γ , λ − 1 [ G ^ ] = F γ , μ − 1 [ G ^ ] . (54) \widehat{\mathbf{G}} = \mathcal{F}_{\gamma,\lambda}[\mathbf{G}] = \mathcal{F}_{\gamma,\boldsymbol{\mu}}[\mathbf{G}], \qquad\mathbf{G} = \mathcal{F}_{\gamma,\lambda}^{-1}[\widehat{\mathbf{G}}] = \mathcal{F}_{\gamma,\boldsymbol{\mu}}^{-1}[\widehat{\mathbf{G}}]. \tag*{(54)} G = F γ , λ [ G ] = F γ , μ [ G ] , G = F γ , λ − 1 [ G ] = F γ , μ − 1 [ G ] . (54)
We can now define the corresponding ROM in H \mathcal{H} H as
G r w e a k : = F γ , μ − 1 [ G ^ r ] . (55) \mathbf{G}_r^{\mathrm{weak}} := \mathcal{F}_{\gamma,\boldsymbol{\mu}}^{-1}[\widehat{\mathbf{G}}_r]. \tag*{(55)} G r weak := F γ , μ − 1 [ G r ] . (55)
Consider the subspace
V ^ r γ ( λ ~ ) : = span { φ ^ j γ ( λ ~ ) } j = 0 r − 1 = span { η ^ j } j = 0 r − 1 ⊆ H ^ , \widehat{V}_{r}^{\gamma(\widetilde{\lambda})} := \operatorname{span}\left\{\widehat{\varphi}_j^{\gamma(\widetilde{\lambda})}\right\}_{j=0}^{r-1} = \operatorname{span}\left\{\widehat{\eta}_j\right\}_{j=0}^{r-1} \subseteq\widehat{\mathcal{H}}, V r γ ( λ ) := span { φ j γ ( λ ) } j = 0 r − 1 = span { η j } j = 0 r − 1 ⊆ H ,
where { η ^ j } j = 0 r − 1 \{\widehat{\eta}_j\}_{j=0}^{r-1} { η j } j = 0 r − 1 are defined according to (35) in the companion RKHS H ^ \widehat{\mathcal{H}} H . Furthermore, define the subspace V r λ ~ ⊆ H V_r^{\widetilde{\lambda}} \subseteq\mathcal{H} V r λ ⊆ H spanned by { η j } j = 0 r − 1 \{\eta_j\}_{j=0}^{r-1} { η j } j = 0 r − 1 with corresponding GMT basis { φ j λ } j = 0 r − 1 \{\varphi_j^{\lambda}\}_{j=0}^{r-1} { φ j λ } j = 0 r − 1 . Unfortunately, restricting F γ , μ \mathcal{F}_{\gamma,\mu} F γ , μ to V r λ V_r^{\lambda} V r λ does not yield the GMT transformation F γ , λ ~ \mathcal{F}_{\gamma,\tilde{\lambda}} F γ , λ ~ . Indeed, the ROM defined in (55) belongs to the r r r –dimensional subspace
W r : = span { [ F γ , μ − 1 [ φ ~ j ( λ ~ ) ] ] j = 0 r − 1 } ⊆ H . \mathcal{W}_r := \operatorname{span}\left\{\left[\mathcal{F}_{\gamma,\mu}^{-1}\left[\widetilde{\varphi}_j^{(\tilde{\lambda})}\right]\right]_{j=0}^{r-1}\right\} \subseteq\mathcal{H}. W r := span { [ F γ , μ − 1 [ φ j ( λ ~ ) ] ] j = 0 r − 1 } ⊆ H .
In general, functions in W r \mathcal{W}_r W r cannot be expressed as an r r r –term linear combination of kernel slices η j \eta_j η j for j = 0 , … , r − 1 j = 0,\ldots,r-1 j = 0 , … , r − 1 . On the other hand, Theorem 3.6 guarantees that the r r r –dimensional H \mathcal{H} H subspace W r ⊆ W E r \mathcal{W}_r \subseteq\mathbb{W}_{\mathcal{E}}^r W r ⊆ W E r is an RKHS in its own right; therefore, its elements, in particular the ROM G r w e a k \mathbf{G}_r^{\mathrm{weak}} G r weak , can be expressed as
G r w e a k = ∑ j = 0 r − 1 w ~ j ξ λ ~ j W r . (56) \mathbf{G}_r^{\mathrm{weak}} = \sum_{j=0}^{r-1} \widetilde{w}_j \xi_{\widetilde{\lambda}_j}^{\mathcal{W}_r}. \tag*{(56)} G r weak = j = 0 ∑ r − 1 w j ξ λ j W r . (56)
where ξ W r \xi^{\mathcal{W}_r} ξ W r denotes the kernel associated with the RKHS W r \mathcal{W}_r W r and w ~ j ∈ C \widetilde{w}_j \in\mathbb{C} w j ∈ C , λ ~ j ∈ Ω \widetilde{\lambda}_j \in\Omega λ j ∈ Ω for j = 0 , … , r − 1 j = 0,\ldots,r-1 j = 0 , … , r − 1 . We summarize the steps of GMT–transformation–based MOR in RKHSs, which can be used to obtain a norm–optimal weak solution:
Given FOM G ∈ G E n \mathbf{G} \in\mathbb{G}_{\mathcal{E}}^n G ∈ G E n , companion RKHS H ^ \widehat{\mathcal{H}} H and reduced order r r r , write G \mathbf{G} G in a GMT expansion, i.e.,
G = ∑ k = 0 n − 1 c k φ k λ ∈ V n λ . \mathbf{G} = \sum_{k=0}^{n-1} c_k\varphi_k^{\lambda} \in V_n^{\lambda}. G = k = 0 ∑ n − 1 c k φ k λ ∈ V n λ .
Obtain corresponding FOM in H ^ \widehat{\mathcal{H}} H , i.e.,
G ^ = F γ , λ [ G ] = ∑ k = 0 n − 1 c k φ ^ k γ ( λ ) ∈ V ^ n γ ( λ ) . \widehat{\mathbf{G}} = \mathcal{F}_{\gamma,\lambda}[\mathbf{G}] = \sum_{k=0}^{n-1} c_k\widehat{\varphi}_k^{\gamma(\lambda)} \in\widehat{V}_n^{\gamma(\lambda)}. G = F γ , λ [ G ] = k = 0 ∑ n − 1 c k φ k γ ( λ ) ∈ V n γ ( λ ) .
Use an appropriate MOR method in the companion space to obtain ROM kernel centers γ ( λ ~ 0 ) , … , γ ( λ ~ r − 1 ) ∈ Ω ^ \gamma(\widetilde{\lambda}_0),\ldots,\gamma(\widetilde{\lambda}_{r-1}) \in\widehat{\Omega} γ ( λ 0 ) , … , γ ( λ r − 1 ) ∈ Ω . Suppose that the ROM is given by G ^ r = S r γ ( λ ~ ) [ G ^ ] \widehat{\mathbf{G}}_r = S_r^{\gamma(\widetilde{\lambda})}[\widehat{\mathbf{G}}] G r = S r γ ( λ ) [ G ] and it satisfies one of the optimality conditions given in § 4 .
Construct an extended GMT basis spanning { φ ^ k γ ( μ ) } k = 0 n + r − 1 \{\widehat{\varphi}_k^{\gamma(\mu)}\}_{k=0}^{n+r-1} { φ k γ ( μ ) } k = 0 n + r − 1 denoted by V ^ n + r γ ( μ ) \widehat{V}_{n+r}^{\gamma(\mu)} V n + r γ ( μ ) . The first n n n functions in this system match the GMT functions spanning V ^ n γ ( λ ) \widehat{V}_n^{\gamma(\lambda)} V n γ ( λ ) to ensure condition (54) .
Obtain a weak ROM G r \mathbf{G}_r G r according to (55) .
If needed, compute the kernel ξ W r \xi^{\mathcal{W}_r} ξ W r . This is possible using a feature map expression from Definition 3.5, because the functions { F γ , μ − 1 [ φ ^ j γ ( λ ~ ) ] } j = 0 r − 1 \left\{\mathcal{F}_{\gamma,\mu}^{-1}\left[\widehat{\varphi}_j^{\gamma(\widetilde{\lambda})}\right]\right\}_{j=0}^{r-1} { F γ , μ − 1 [ φ j γ ( λ ) ] } j = 0 r − 1 form a basis in W r \mathcal{W}_r W r . Finally, write G r w e a k \mathbf{G}_r^{\mathrm{weak}} G r weak according to (56) .
Then, by Theorem 5.23, we have ∥ G − G r w e a k ∥ H = ∥ G ^ − G ^ r ∥ H ^ \|\mathbf{G}-\mathbf{G}_r^{\mathrm{weak}}\|_{\mathcal{H}} = \|\widehat{\mathbf{G}}-\widehat{\mathbf{G}}_r\|_{\widehat{\mathcal{H}}} ∥ G − G r weak ∥ H = ∥ G − G r ∥ H with G ^ r \widehat{\mathbf{G}}_r G r satisfying one of the optimality conditions given in § 4 .
One might be tempted to reverse the order of kernel centers in (53) . In this case, we obtain (54) for the ROM, instead of the FOM. However, this choice leads to the issue that usually
G ≠ F γ , λ ~ − 1 [ F γ , λ [ G ] ] . G \ne\mathcal{F}_{\gamma,\widetilde{\lambda}}^{-1}\left[\mathcal{F}_{\gamma,\lambda}[G]\right]. G = F γ , λ − 1 [ F γ , λ [ G ] ] .
Consequently, ∥ G − G r ∥ H = ∥ G ^ − G ^ r ∥ H ^ \lVert G-G_r\rVert_{\mathcal{H}}=\lVert\widehat{G}-\widehat{G}_r\rVert_{\widehat{\mathcal{H}}} ∥ G − G r ∥ H = ∥ G − G r ∥ H would no longer hold, where G r : = F γ , λ ~ − 1 [ G ^ r ] G_r:=\mathcal{F}_{\gamma,\widetilde{\lambda}}^{-1}[\widehat{G}_r] G r := F γ , λ − 1 [ G r ] .
Finally, we note that for some special choices of H \mathcal{H} H , the weak and strong model spaces seem to coincide under the GMT transformation pipeline. That is, the above transformation pipeline always produces G r w e a k ∈ G r G_r^{\mathrm{weak}}\in\mathfrak{G}_r G r weak ∈ G r . This seems to be the case, for example, if H = H 2 ( D ) \mathcal{H}=H_2(\mathbb{D}) H = H 2 ( D ) and H ^ = H 2 ( C + ) \widehat{\mathcal{H}}=H_2(\mathbb{C}^+) H = H 2 ( C + ) . In the current study, we refrain from pursuing sufficient conditions for this to hold in general. We note, however, that if H \mathcal{H} H and H ^ \widehat{\mathcal{H}} H are both H 2 H_2 H 2 Hardy–Hilbert spaces, the GMT transformation can alternatively be realized using a conformal map between their domains, instead of the basis–matching construction of Definition 5.22. In this case, weak and strong solutions are guaranteed to coincide.
Practically, if an application requires a strong ROM in G r \mathfrak{G}_r G r , one way to proceed is to project the weak solution to V ~ r λ = span { φ ~ j λ } j = 0 r − 1 ⊆ H \widetilde{V}_r^\lambda=\operatorname{span}\left\{\widetilde{\varphi}_j^\lambda\right\}_{j=0}^{r-1}\subseteq\mathcal{H} V r λ = span { φ j λ } j = 0 r − 1 ⊆ H . The next theorem clarifies error bounds on this projection.
Theorem 5.25 (Error of approximate strong models). Suppose Assumption 5.21 is satisfied, and let V n λ V_n^\lambda V n λ , V ~ r λ \widetilde{V}_r^\lambda V r λ , V ^ n γ ( λ ) \widehat{V}_n^{\gamma(\lambda)} V n γ ( λ ) , and V ^ r γ ( λ ~ ) \widehat{V}_r^{\gamma(\widetilde{\lambda})} V r γ ( λ ) be built as in (5.10). Suppose that V n λ ∩ V ~ r λ = { 0 } V_n^\lambda\cap\widetilde{V}_r^\lambda=\{0\} V n λ ∩ V r λ = { 0 } . Let ⊞ \boxplus ⊞ denote the direct sum and define the subspaces
V n + r μ : = V n λ ⊞ V ~ r λ = span { φ k μ } k = 0 n + r − 1 , V ^ n + r γ ( μ ) : = V ^ n γ ( λ ) ⊞ V ^ r γ ( λ ~ ) = span { φ ^ k γ ( μ ) } k = 0 n + r − 1 V_{n+r}^{\mu}:=V_n^\lambda\boxplus\widetilde{V}_r^\lambda=\operatorname{span}\left\{\varphi_k^\mu\right\}_{k=0}^{n+r-1},\qquad
\widehat{V}_{n+r}^{\gamma(\mu)}:=\widehat{V}_n^{\gamma(\lambda)}\boxplus\widehat{V}_r^{\gamma(\widetilde{\lambda})}=\operatorname{span}\left\{\widehat{\varphi}_k^{\gamma(\mu)}\right\}_{k=0}^{n+r-1} V n + r μ := V n λ ⊞ V r λ = span { φ k μ } k = 0 n + r − 1 , V n + r γ ( μ ) := V n γ ( λ ) ⊞ V r γ ( λ ) = span { φ k γ ( μ ) } k = 0 n + r − 1 along with
( V n λ ) ⊥ : = V n + r μ ⊖ V n λ , ( V ^ n γ ( λ ) ) ⊥ : = V ^ n + r γ ( μ ) ⊖ V ^ n γ ( λ ) , \left(V_n^\lambda\right)^\perp:=V_{n+r}^{\mu}\ominus V_n^\lambda,\qquad
\left(\widehat{V}_n^{\gamma(\lambda)}\right)^\perp:=\widehat{V}_{n+r}^{\gamma(\mu)}\ominus\widehat{V}_n^{\gamma(\lambda)}, ( V n λ ) ⊥ := V n + r μ ⊖ V n λ , ( V n γ ( λ ) ) ⊥ := V n + r γ ( μ ) ⊖ V n γ ( λ ) , and the GMT transformations
F γ , μ : V n + r μ → V ^ n + r γ ( μ ) , F γ , λ : V n λ → V ^ n γ ( λ ) \mathcal{F}_{\gamma,\mu}:V_{n+r}^{\mu}\to\widehat{V}_{n+r}^{\gamma(\mu)},\qquad
\mathcal{F}_{\gamma,\lambda}:V_n^\lambda\to\widehat{V}_n^{\gamma(\lambda)} F γ , μ : V n + r μ → V n + r γ ( μ ) , F γ , λ : V n λ → V n γ ( λ ) generated by μ : = ( λ 0 , … , λ n − 1 , λ ~ 0 , … , λ ~ r − 1 ) \mu:=(\lambda_0,\ldots,\lambda_{n-1},\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_{r-1}) μ := ( λ 0 , … , λ n − 1 , λ 0 , … , λ r − 1 ) and λ : = ( λ 0 , … , λ n − 1 ) \lambda:=(\lambda_0,\ldots,\lambda_{n-1}) λ := ( λ 0 , … , λ n − 1 ) , respectively. Then, the following are true:
(a) The operators
Q : = P V n λ ∘ ( P ( V n λ ) ⊥ ∣ V ~ r λ ) − 1 : ( V n λ ) ⊥ → V n λ , Q:=P_{V_n^\lambda}\circ\left(\mathcal{P}_{(V_n^\lambda)^\perp\mid\widetilde{V}_r^\lambda}\right)^{-1}:(V_n^\lambda)^\perp\to V_n^\lambda, Q := P V n λ ∘ ( P ( V n λ ) ⊥ ∣ V r λ ) − 1 : ( V n λ ) ⊥ → V n λ , Q ^ : = P V ^ n γ ( λ ) ∘ ( P ( V ^ n γ ( λ ) ) ⊥ ∣ V ^ r γ ( λ ~ ) ) − 1 : ( V ^ n γ ( λ ) ) ⊥ → V ^ n γ ( λ ) (57) \widehat{Q}:=P_{\widehat{V}_n^{\gamma(\lambda)}}\circ\left(\mathcal{P}_{(\widehat{V}_n^{\gamma(\lambda)})^\perp\mid\widehat{V}_r^{\gamma(\widetilde{\lambda})}}\right)^{-1}:\left(\widehat{V}_n^{\gamma(\lambda)}\right)^\perp\to\widehat{V}_n^{\gamma(\lambda)} \tag*{(57)} Q := P V n γ ( λ ) ∘ ( P ( V n γ ( λ ) ) ⊥ ∣ V r γ ( λ ) ) − 1 : ( V n γ ( λ ) ) ⊥ → V n γ ( λ ) (57) are well defined.
(b) For all W 0 ∈ W r : = span { F γ , μ − 1 [ ξ ~ γ ( λ 0 ) ] , … , F γ , μ − 1 [ ξ ~ γ ( λ r − 1 ) ] } W_0 \in W_r := \operatorname{span}\left\{\mathcal{F}_{\gamma,\mu}^{-1}[\widetilde{\xi}_{\gamma}(\lambda_0)],\ldots,\mathcal{F}_{\gamma,\mu}^{-1}[\widetilde{\xi}_{\gamma}(\lambda_{r-1})]\right\} W 0 ∈ W r := span { F γ , μ − 1 [ ξ γ ( λ 0 )] , … , F γ , μ − 1 [ ξ γ ( λ r − 1 )] } , with F γ , λ ⊥ : = F γ , μ ∣ ( V n λ ) ⊥ \mathcal{F}_{\gamma,\lambda}^{\perp}:=\mathcal{F}_{\gamma,\mu}|_{(V_n^\lambda)^\perp} F γ , λ ⊥ := F γ , μ ∣ ( V n λ ) ⊥ , we have
dist ( W 0 , V ~ r λ ) : = inf V ∈ V ~ r λ ∥ W 0 − V ∥ H ≤ ∥ Q ~ ∘ F γ , λ ⊥ − F γ , λ ∘ Q ∥ L ( ( V n λ ) ⊥ , ( V ^ n γ ( λ ) ) ⊥ ) ∥ W 0 ∥ H , (58) \operatorname{dist}\left(W_0,\widetilde{V}_r^\lambda\right):=\inf_{V\in\widetilde{V}_r^\lambda}\left\|W_0-V\right\|_{\mathcal{H}}
\leq\left\|\widetilde{Q}\circ\mathcal{F}_{\gamma,\lambda}^{\perp}-\mathcal{F}_{\gamma,\lambda}\circ Q\right\|_{\mathcal{L}\left((V_n^\lambda)^\perp,(\widehat{V}_n^{\gamma(\lambda)})^\perp\right)}\left\|W_0\right\|_{\mathcal{H}}, \tag*{(58)} dist ( W 0 , V r λ ) := V ∈ V r λ inf ∥ W 0 − V ∥ H ≤ Q ∘ F γ , λ ⊥ − F γ , λ ∘ Q L ( ( V n λ ) ⊥ , ( V n γ ( λ ) ) ⊥ ) ∥ W 0 ∥ H , (58)
Proof. We first observe that the GMT transformation F γ , μ \mathcal{F}_{\gamma,\mu} F γ , μ is a block–diagonal linear operator. Indeed, because of the ordering given by μ \boldsymbol{\mu} μ determining the Gram–Schmidt steps, we have
V n λ = span { φ 0 μ , … , φ n − 1 μ } = span { φ 0 λ , … , φ n − 1 λ } . V_n^\lambda=\operatorname{span}\left\{\varphi_0^\mu,\ldots,\varphi_{n-1}^\mu\right\}=\operatorname{span}\left\{\varphi_0^\lambda,\ldots,\varphi_{n-1}^\lambda\right\}. V n λ = span { φ 0 μ , … , φ n − 1 μ } = span { φ 0 λ , … , φ n − 1 λ } .
and
( V n λ ) ⊥ = span { φ n μ , … , φ n + r − 1 μ } . \left(V_n^\lambda\right)^\perp=\operatorname{span}\left\{\varphi_n^\mu,\ldots,\varphi_{n+r-1}^\mu\right\}. ( V n λ ) ⊥ = span { φ n μ , … , φ n + r − 1 μ } .
An analogous observation can be made for V ^ n γ ( λ ) \widehat{V}_n^{\gamma(\lambda)} V n γ ( λ ) and ( V ^ n γ ( λ ) ) ⊥ \left(\widehat{V}_n^{\gamma(\lambda)}\right)^\perp ( V n γ ( λ ) ) ⊥ . Hence,
F γ , μ = [ F γ , λ 0 0 F γ , λ ⊥ ] \mathcal{F}_{\gamma,\mu}=
\begin{bmatrix}
\mathcal{F}_{\gamma,\lambda} & 0\\
0 & \mathcal{F}_{\gamma,\lambda}^{\perp}
\end{bmatrix} F γ , μ = [ F γ , λ 0 0 F γ , λ ⊥ ]
with respect to the orthogonal splittings
V n + r μ = V n λ ⊕ ( V n λ ) ⊥ and V ^ n + r γ ( μ ) = V ^ n γ ( λ ) ⊕ ( V ^ n γ ( λ ) ) ⊥ . V_{n+r}^\mu=V_n^\lambda\oplus\left(V_n^\lambda\right)^\perp
\qquad\text{and}\qquad
\widehat{V}_{n+r}^{\gamma(\mu)}=\widehat{V}_n^{\gamma(\lambda)}\oplus\left(\widehat{V}_n^{\gamma(\lambda)}\right)^\perp. V n + r μ = V n λ ⊕ ( V n λ ) ⊥ and V n + r γ ( μ ) = V n γ ( λ ) ⊕ ( V n γ ( λ ) ) ⊥ .
It follows that F γ , λ : V n λ → V ^ n γ ( λ ) \mathcal{F}_{\gamma,\lambda}:V_n^\lambda\to\widehat{V}_n^{\gamma(\lambda)} F γ , λ : V n λ → V n γ ( λ ) and F γ , λ ⊥ : ( V n λ ) ⊥ → ( V ^ n γ ( λ ) ) ⊥ \mathcal{F}_{\gamma,\lambda}^{\perp}:(V_n^\lambda)^\perp\to(\widehat{V}_n^{\gamma(\lambda)})^\perp F γ , λ ⊥ : ( V n λ ) ⊥ → ( V n γ ( λ ) ) ⊥ are unitary.
We now show statement (a), i.e., the existence of the operators Q Q Q and Q ~ \widetilde{Q} Q in (57) . Let V 0 ∈ V ~ r λ V_0\in\widetilde{V}_r^\lambda V 0 ∈ V r λ and suppose P ( V n λ ) ⊥ [ V 0 ] = 0 \mathcal{P}_{(V_n^\lambda)^\perp}[V_0]=0 P ( V n λ ) ⊥ [ V 0 ] = 0 . Since we can write V 0 V_0 V 0 as
V 0 = P V n λ [ V 0 ] + P ( V n λ ) ⊥ [ V 0 ] = P V n λ [ V 0 ] , V_0=\mathcal{P}_{V_n^\lambda}[V_0]+\mathcal{P}_{(V_n^\lambda)^\perp}[V_0]
=\mathcal{P}_{V_n^\lambda}[V_0], V 0 = P V n λ [ V 0 ] + P ( V n λ ) ⊥ [ V 0 ] = P V n λ [ V 0 ] ,
this gives V 0 ∈ V n λ V_0\in V_n^\lambda V 0 ∈ V n λ . On the other hand, by the assumption V ~ r λ ∩ V n λ = { 0 } \widetilde{V}_r^\lambda\cap V_n^\lambda=\{0\} V r λ ∩ V n λ = { 0 } , hence, V 0 = 0 V_0=0 V 0 = 0 follows. This shows that P ( V n λ ) ⊥ ∣ V ~ r λ \mathcal{P}_{(V_n^\lambda)^\perp}|_{\widetilde{V}_r^\lambda} P ( V n λ ) ⊥ ∣ V r λ is injective. However, since dim V ~ r λ = dim ( V n λ ) ⊥ = r \dim\widetilde{V}_r^\lambda=\dim(V_n^\lambda)^\perp=r dim V r λ = dim ( V n λ ) ⊥ = r , the restricted orthogonal projector P ( V n λ ) ⊥ ∣ V ~ r λ \mathcal{P}_{(V_n^\lambda)^\perp}|_{\widetilde{V}_r^\lambda} P ( V n λ ) ⊥ ∣ V r λ is also bijective and therefore, Q Q Q in (57) is well-defined. An analogous argument can be used to show the existence of Q ~ \widetilde{Q} Q .
Let us show statement (b). We make use of the fact that any function V ∈ V ~ r λ V\in\widetilde{V}_r^\lambda V ∈ V r λ can be written as P V n λ [ V ] + P ( V n λ ) ⊥ [ V ] \mathcal{P}_{V_n^\lambda}[V]+\mathcal{P}_{(V_n^\lambda)^\perp}[V] P V n λ [ V ] + P ( V n λ ) ⊥ [ V ] . This, together with (57) , and the bijective nature of P ( V n λ ) ⊥ ∣ V ~ r λ \mathcal{P}_{(V_n^\lambda)^\perp}|_{\widetilde{V}_r^\lambda} P ( V n λ ) ⊥ ∣ V r λ gives
V ~ r λ = { Q [ H ] + H : H ∈ ( V n λ ) ⊥ } . (59) \widetilde{V}_r^\lambda=\left\{Q[H]+H:H\in(V_n^\lambda)^\perp\right\}. \tag*{(59)} V r λ = { Q [ H ] + H : H ∈ ( V n λ ) ⊥ } . (59)
Observe that for each H ∈ ( V n λ ) ⊥ \mathbf{H} \in(V_n^\lambda)^\perp H ∈ ( V n λ ) ⊥ , there exists a V ∈ V r λ ~ \mathbf{V} \in V_r^{\widetilde{\lambda}} V ∈ V r λ such that Q [ H ] = P V n λ [ V ] \mathbf{Q}[\mathbf{H}] = \mathcal{P}_{V_n^\lambda}[\mathbf{V}] Q [ H ] = P V n λ [ V ] and H = P ( V n λ ) ⊥ [ V ] \mathbf{H} = \mathcal{P}_{(V_n^\lambda)^\perp}[\mathbf{V}] H = P ( V n λ ) ⊥ [ V ] . By the same argument in the companion space, we also obtain
V ^ r ( λ ~ ) = { Q ^ [ H ^ ] + H ^ : H ^ ∈ ( V ^ n ( λ ) ) ⊥ } . (60) \widehat{V}_r^{(\widetilde{\lambda})} = \left\{ \widehat{Q}[\widehat{\mathbf{H}}] + \widehat{\mathbf{H}} : \widehat{\mathbf{H}} \in\left(\widehat{V}_n^{(\lambda)}\right)^\perp\right\}. \tag*{(60)} V r ( λ ) = { Q [ H ] + H : H ∈ ( V n ( λ ) ) ⊥ } . (60)
Next, let W 0 ∈ W r \mathbf{W}_0 \in W_r W 0 ∈ W r be fixed. Then we also have W 0 = P V n λ [ W 0 ] + P ( V n λ ) ⊥ [ W 0 ] \mathbf{W}_0 = \mathcal{P}_{V_n^\lambda}[\mathbf{W}_0] + \mathcal{P}_{(V_n^\lambda)^\perp}[\mathbf{W}_0] W 0 = P V n λ [ W 0 ] + P ( V n λ ) ⊥ [ W 0 ] . In addition, let W ^ 0 : = F γ , μ [ W 0 ] ∈ V ^ r ( λ ~ ) \widehat{\mathbf{W}}_0 := \mathcal{F}_{\gamma,\mu}[\mathbf{W}_0] \in\widehat{V}_r^{(\widetilde{\lambda})} W 0 := F γ , μ [ W 0 ] ∈ V r ( λ ) . Exploiting these observations and the block-diagonal structure F γ , μ − 1 = diag ( F γ , λ − 1 , ( F γ , λ ⊥ ) − 1 ) \mathcal{F}_{\gamma,\mu}^{-1} = \operatorname{diag}\left(\mathcal{F}_{\gamma,\lambda}^{-1},(\mathcal{F}_{\gamma,\lambda}^{\perp})^{-1}\right) F γ , μ − 1 = diag ( F γ , λ − 1 , ( F γ , λ ⊥ ) − 1 ) , we can express the set W r = F γ , μ − 1 [ V ^ r ( λ ~ ) ] W_r = \mathcal{F}_{\gamma,\mu}^{-1}\left[\widehat{V}_r^{(\widetilde{\lambda})}\right] W r = F γ , μ − 1 [ V r ( λ ) ] as
W r = { F γ , λ − 1 [ F ^ ] + ( F γ , λ ⊥ ) − 1 [ F ^ ] : F ^ ∈ V ^ r ( λ ~ ) } . W_r = \left\{ \mathcal{F}_{\gamma,\lambda}^{-1}[\widehat{\mathbf{F}}] + (\mathcal{F}_{\gamma,\lambda}^{\perp})^{-1}[\widehat{\mathbf{F}}] : \widehat{\mathbf{F}} \in\widehat{V}_r^{(\widetilde{\lambda})} \right\}. W r = { F γ , λ − 1 [ F ] + ( F γ , λ ⊥ ) − 1 [ F ] : F ∈ V r ( λ ) } .
Considering (60) , we find that for all F ^ ∈ V ^ r ( λ ~ ) \widehat{\mathbf{F}} \in\widehat{V}_r^{(\widetilde{\lambda})} F ∈ V r ( λ ) , we have
F ^ = Q ^ [ H ^ ] ⏟ ∈ V ^ n ( λ ) + H ^ ⏟ ∈ ( V ^ n ( λ ) ) ⊥ \widehat{\mathbf{F}} = \underbrace{\widehat{Q}[\widehat{\mathbf{H}}]}_{\in\widehat{V}_n^{(\lambda)}} + \underbrace{\widehat{\mathbf{H}}}_{\in(\widehat{V}_n^{(\lambda)})^\perp} F = ∈ V n ( λ ) Q [ H ] + ∈ ( V n ( λ ) ) ⊥ H
for some H ^ ∈ ( V ^ n ( λ ) ) ⊥ \widehat{\mathbf{H}} \in\left(\widehat{V}_n^{(\lambda)}\right)^\perp H ∈ ( V n ( λ ) ) ⊥ . Accordingly, we can write
W 0 = F γ , μ − 1 [ W ^ 0 ] = ( F γ , μ − 1 ∘ Q ^ ) [ H ^ 0 ] + F γ , μ − 1 [ H ^ 0 ] \mathbf{W}_0 = \mathcal{F}_{\gamma,\mu}^{-1}[\widehat{\mathbf{W}}_0] = (\mathcal{F}_{\gamma,\mu}^{-1} \circ\widehat{Q})[\widehat{\mathbf{H}}_0] + \mathcal{F}_{\gamma,\mu}^{-1}[\widehat{\mathbf{H}}_0] W 0 = F γ , μ − 1 [ W 0 ] = ( F γ , μ − 1 ∘ Q ) [ H 0 ] + F γ , μ − 1 [ H 0 ]
for some H ^ 0 ∈ ( V ^ n ( λ ) ) ⊥ \widehat{\mathbf{H}}_0 \in\left(\widehat{V}_n^{(\lambda)}\right)^\perp H 0 ∈ ( V n ( λ ) ) ⊥ . By exploiting the block diagonal nature of F γ , μ \mathcal{F}_{\gamma,\mu} F γ , μ , this gives
W 0 = ( F γ , λ − 1 ∘ Q ^ ) [ H ^ 0 ] + ( F γ , λ ⊥ ) − 1 [ H ^ 0 ] . (61) \mathbf{W}_0 = (\mathcal{F}_{\gamma,\lambda}^{-1} \circ\widehat{Q})[\widehat{\mathbf{H}}_0] + (\mathcal{F}_{\gamma,\lambda}^{\perp})^{-1}[\widehat{\mathbf{H}}_0]. \tag*{(61)} W 0 = ( F γ , λ − 1 ∘ Q ) [ H 0 ] + ( F γ , λ ⊥ ) − 1 [ H 0 ] . (61)
Now recall that H ^ 0 ∈ ( V ^ n ( λ ) ) ⊥ \widehat{\mathbf{H}}_0 \in\left(\widehat{V}_n^{(\lambda)}\right)^\perp H 0 ∈ ( V n ( λ ) ) ⊥ and so H 0 : = F γ , μ − 1 [ H ^ 0 ] = ( F γ , λ ⊥ ) − 1 [ H ^ 0 ] ∈ ( V n λ ) ⊥ \mathbf{H}_0 := \mathcal{F}_{\gamma,\mu}^{-1}[\widehat{\mathbf{H}}_0] = (\mathcal{F}_{\gamma,\lambda}^{\perp})^{-1}[\widehat{\mathbf{H}}_0] \in(V_n^\lambda)^\perp H 0 := F γ , μ − 1 [ H 0 ] = ( F γ , λ ⊥ ) − 1 [ H 0 ] ∈ ( V n λ ) ⊥ . Applying this substitution to (61) yields
W 0 = ( F γ , λ − 1 ∘ Q ^ ∘ F γ , λ ⊥ ) [ H 0 ] + H 0 . \mathbf{W}_0 = \left(\mathcal{F}_{\gamma,\lambda}^{-1} \circ\widehat{Q} \circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}_0] + \mathbf{H}_0. W 0 = ( F γ , λ − 1 ∘ Q ∘ F γ , λ ⊥ ) [ H 0 ] + H 0 .
We find that we can describe the subspace W r W_r W r by
W r = { ( F γ , λ − 1 ∘ Q ^ ∘ F γ , λ ⊥ ) [ H ] + H : H ∈ ( V n λ ) ⊥ } . (62) W_r = \left\{ \left(\mathcal{F}_{\gamma,\lambda}^{-1} \circ\widehat{Q} \circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}] + \mathbf{H} : \mathbf{H} \in(V_n^\lambda)^\perp\right\}. \tag*{(62)} W r = { ( F γ , λ − 1 ∘ Q ∘ F γ , λ ⊥ ) [ H ] + H : H ∈ ( V n λ ) ⊥ } . (62)
We are now ready to give the estimate in (5.23). Recall, that for W 0 ∈ W r \mathbf{W}_0 \in W_r W 0 ∈ W r , the distance from V ~ r λ \widetilde{V}_r^\lambda V r λ is defined as
dist ( W 0 , V ~ r λ ) : = inf V ∈ V ~ r λ ∥ W 0 − V ∥ H . \operatorname{dist}\left(\mathbf{W}_0,\widetilde{V}_r^\lambda\right) := \inf_{\mathbf{V} \in\widetilde{V}_r^\lambda} \lVert\mathbf{W}_0 - \mathbf{V} \rVert_{\mathcal{H}}. dist ( W 0 , V r λ ) := V ∈ V r λ inf ∥ W 0 − V ∥ H .
Hence, any particular V 0 ∈ V ~ r λ \mathbf{V}_0 \in\widetilde{V}_r^\lambda V 0 ∈ V r λ satisfies
dist ( W 0 , V ~ r λ ) ≤ ∥ W 0 − V 0 ∥ H . (63) \operatorname{dist}(\mathbf{W}_0,\widetilde{V}_r^\lambda) \leq\lVert\mathbf{W}_0-\mathbf{V}_0\rVert_{\mathcal{H}}. \tag*{(63)} dist ( W 0 , V r λ ) ≤ ∥ W 0 − V 0 ∥ H . (63)
We can now exploit (59) and (62) by observing that W 0 = ( F γ , λ − 1 ∘ Q ^ ∘ F γ , λ ⊥ ) [ H 0 ] + H 0 \mathbf{W}_0 = \left(\mathcal{F}_{\gamma,\lambda}^{-1} \circ\widehat{Q} \circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}_0] + \mathbf{H}_0 W 0 = ( F γ , λ − 1 ∘ Q ∘ F γ , λ ⊥ ) [ H 0 ] + H 0 for some H 0 ∈ ( V n λ ) ⊥ \mathbf{H}_0 \in(V_n^\lambda)^\perp H 0 ∈ ( V n λ ) ⊥ and choosing V 0 = Q [ H 0 ] + H 0 ∈ V ~ r λ \mathbf{V}_0 = Q[\mathbf{H}_0] + \mathbf{H}_0 \in\widetilde{V}_r^\lambda V 0 = Q [ H 0 ] + H 0 ∈ V r λ . This allows us to write
W 0 − V 0 = ( F γ , λ − 1 ∘ Q ^ ∘ F γ , λ ⊥ ) [ H 0 ] + H 0 − Q [ H 0 ] − H 0 = ( F γ , λ − 1 ∘ Q ^ ∘ F γ , λ ⊥ ) [ H 0 ] − Q [ H 0 ] ∈ V n λ , \begin{aligned}
\mathbf{W}_0-\mathbf{V}_0
&= \left(\mathcal{F}_{\gamma,\lambda}^{-1} \circ\widehat{Q} \circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}_0]+\mathbf{H}_0-Q[\mathbf{H}_0]-\mathbf{H}_0 \\
&= \left(\mathcal{F}_{\gamma,\lambda}^{-1} \circ\widehat{Q} \circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}_0]-Q[\mathbf{H}_0] \in V_n^\lambda,
\end{aligned} W 0 − V 0 = ( F γ , λ − 1 ∘ Q ∘ F γ , λ ⊥ ) [ H 0 ] + H 0 − Q [ H 0 ] − H 0 = ( F γ , λ − 1 ∘ Q ∘ F γ , λ ⊥ ) [ H 0 ] − Q [ H 0 ] ∈ V n λ ,
and therefore,
∥ W 0 − V 0 ∥ H = ∥ ( F γ , λ − 1 ∘ Q ^ ∘ F γ , λ ⊥ ) [ H 0 ] − Q [ H 0 ] ∥ H ^ . \lVert\mathbf{W}_0-\mathbf{V}_0\rVert_{\mathcal{H}}=\left\lVert\left(\mathcal{F}_{\gamma,\lambda}^{-1}\circ\widehat{Q}\circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}_0]-Q[\mathbf{H}_0]\right\rVert_{\widehat{\mathcal{H}}}. ∥ W 0 − V 0 ∥ H = ( F γ , λ − 1 ∘ Q ∘ F γ , λ ⊥ ) [ H 0 ] − Q [ H 0 ] H .
To obtain the formula, we can exploit that F γ , λ = F γ , μ ∣ V n λ \mathcal{F}_{\gamma,\lambda}=\mathcal{F}_{\gamma,\mu}|_{V_n^\lambda} F γ , λ = F γ , μ ∣ V n λ and that F γ , λ \mathcal{F}_{\gamma,\lambda} F γ , λ and F γ , μ \mathcal{F}_{\gamma,\mu} F γ , μ are unitary and find
∥ W 0 − V 0 ∥ H = ∥ F γ , λ [ W 0 − V 0 ] ∥ H ^ = ∥ F γ , λ [ ( F γ , λ − 1 ∘ Q ^ ∘ F γ , λ ⊥ ) [ H 0 ] − Q [ H 0 ] ] ∥ H ^ = ∥ ( Q ^ ∘ F γ , λ ⊥ ) [ H 0 ] − ( F γ , λ ∘ Q ) [ H 0 ] ∥ H ^ . \begin{aligned}
\lVert\mathbf{W}_0-\mathbf{V}_0\rVert_{\mathcal{H}}
&=\lVert\mathcal{F}_{\gamma,\lambda}[\mathbf{W}_0-\mathbf{V}_0]\rVert_{\widehat{\mathcal{H}}} \\
&=\left\lVert\mathcal{F}_{\gamma,\lambda}\left[\left(\mathcal{F}_{\gamma,\lambda}^{-1}\circ\widehat{Q}\circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}_0]-Q[\mathbf{H}_0]\right]\right\rVert_{\widehat{\mathcal{H}}} \\
&=\left\lVert\left(\widehat{Q}\circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}_0]-(\mathcal{F}_{\gamma,\lambda}\circ Q)[\mathbf{H}_0]\right\rVert_{\widehat{\mathcal{H}}}.
\end{aligned} ∥ W 0 − V 0 ∥ H = ∥ F γ , λ [ W 0 − V 0 ] ∥ H = F γ , λ [ ( F γ , λ − 1 ∘ Q ∘ F γ , λ ⊥ ) [ H 0 ] − Q [ H 0 ] ] H = ( Q ∘ F γ , λ ⊥ ) [ H 0 ] − ( F γ , λ ∘ Q ) [ H 0 ] H .
This, together with (63) gives
dist ( W 0 , V ~ r λ ) ≤ ∥ W 0 − V 0 ∥ H = ∥ ( Q ^ ∘ F γ , λ ⊥ ) [ H 0 ] − ( F γ , λ ∘ Q ) [ H 0 ] ∥ H ^ ≤ ∥ Q ^ ∘ F γ , λ ⊥ − F γ , λ ∘ Q ∥ L ( ( V n λ ) ⊥ , V ~ n γ , λ ) ∥ H 0 ∥ H ≤ ∥ Q ^ ∘ F γ , λ ⊥ − F γ , λ ∘ Q ∥ L ( ( V n λ ) ⊥ , V ~ n γ , λ ) ∥ W 0 ∥ H , \begin{aligned}
\operatorname{dist}(\mathbf{W}_0,\widetilde{V}_r^\lambda)
&\leq\lVert\mathbf{W}_0-\mathbf{V}_0\rVert_{\mathcal{H}} \\
&=\left\lVert\left(\widehat{Q}\circ\mathcal{F}_{\gamma,\lambda}^{\perp}\right)[\mathbf{H}_0]-(\mathcal{F}_{\gamma,\lambda}\circ Q)[\mathbf{H}_0]\right\rVert_{\widehat{\mathcal{H}}} \\
&\leq\left\lVert\widehat{Q}\circ\mathcal{F}_{\gamma,\lambda}^{\perp}-\mathcal{F}_{\gamma,\lambda}\circ Q\right\rVert_{\mathcal{L}((V_n^\lambda)^\perp,\widetilde{V}_n^{\gamma,\lambda})}\lVert\mathbf{H}_0\rVert_{\mathcal{H}} \\
&\leq\left\lVert\widehat{Q}\circ\mathcal{F}_{\gamma,\lambda}^{\perp}-\mathcal{F}_{\gamma,\lambda}\circ Q\right\rVert_{\mathcal{L}((V_n^\lambda)^\perp,\widetilde{V}_n^{\gamma,\lambda})}\lVert\mathbf{W}_0\rVert_{\mathcal{H}},
\end{aligned} dist ( W 0 , V r λ ) ≤ ∥ W 0 − V 0 ∥ H = ( Q ∘ F γ , λ ⊥ ) [ H 0 ] − ( F γ , λ ∘ Q ) [ H 0 ] H ≤ Q ∘ F γ , λ ⊥ − F γ , λ ∘ Q L (( V n λ ) ⊥ , V n γ , λ ) ∥ H 0 ∥ H ≤ Q ∘ F γ , λ ⊥ − F γ , λ ∘ Q L (( V n λ ) ⊥ , V n γ , λ ) ∥ W 0 ∥ H ,
where the last inequality follows from (62) , which we can apply to H 0 \mathbf{H}_0 H 0 to obtain ∥ H 0 ∥ H = ∥ P ( V n λ ) ⊥ [ W 0 ] ∥ H ≤ ∥ W 0 ∥ H \lVert\mathbf{H}_0\rVert_{\mathcal{H}}=\lVert P_{(V_n^\lambda)^\perp}[\mathbf{W}_0]\rVert_{\mathcal{H}}\leq\lVert\mathbf{W}_0\rVert_{\mathcal{H}} ∥ H 0 ∥ H = ∥ P ( V n λ ) ⊥ [ W 0 ] ∥ H ≤ ∥ W 0 ∥ H .
Corollary 5.26. Suppose the assumptions of Theorem 5.25 . Then, for any weak ROM G r w e a k ∈ W r \mathbf{G}_r^{\mathrm{weak}}\in W_r G r weak ∈ W r , the infimum in dist ( G r w e a k , V ~ r λ ) \operatorname{dist}(\mathbf{G}_r^{\mathrm{weak}},\widetilde{V}_r^\lambda) dist ( G r weak , V r λ ) is attained and the best approximation in V ~ r λ \widetilde{V}_r^\lambda V r λ is given by P V ~ r λ [ G r w e a k ] P_{\widetilde{V}_r^\lambda}[\mathbf{G}_r^{\mathrm{weak}}] P V r λ [ G r weak ] . In addition,
dist ( G r w e a k , V ~ r λ ) = ∥ G r w e a k − P V ~ r λ [ G r w e a k ] ∥ H \operatorname{dist}(\mathbf{G}_r^{\mathrm{weak}},\widetilde{V}_r^\lambda)=\left\lVert\mathbf{G}_r^{\mathrm{weak}}-P_{\widetilde{V}_r^\lambda}[\mathbf{G}_r^{\mathrm{weak}}]\right\rVert_{\mathcal{H}} dist ( G r weak , V r λ ) = G r weak − P V r λ [ G r weak ] H ≤ ∥ Q ^ ∘ F γ , λ ⊥ − F γ , λ ∘ Q ∥ L ( ( V n λ ) ⊥ , V ~ n γ , λ ) ∥ G r w e a k ∥ H . (64) \leq\left\lVert\widehat{Q}\circ\mathcal{F}_{\gamma,\lambda}^{\perp}-\mathcal{F}_{\gamma,\lambda}\circ Q\right\rVert_{\mathcal{L}((V_n^\lambda)^\perp,\widetilde{V}_n^{\gamma,\lambda})}\lVert\mathbf{G}_r^{\mathrm{weak}}\rVert_{\mathcal{H}}. \tag*{(64)} ≤ Q ∘ F γ , λ ⊥ − F γ , λ ∘ Q L (( V n λ ) ⊥ , V n γ , λ ) ∥ G r weak ∥ H . (64)
Proof. The attainment of the infimum and the best approximation are consequences of the Hilbert projection theorem, while (64) is an application of Theorem 5.25 .
The previous statement bounds the discrepancy between the weak solution and its projection to V r λ V_r^{\lambda} V r λ . To obtain this projected solution, one has to first compute the weak ROM given by G r w e a k = F γ , μ − 1 [ G ^ r ] \mathbf{G}^{\mathrm{weak}}_r=\mathcal{F}^{-1}_{\gamma,\mu}[\widehat{\mathbf{G}}_r] G r weak = F γ , μ − 1 [ G r ] , and then project it to V r λ V_r^{\lambda} V r λ . This is computationally inefficient, because finding F γ , μ − 1 [ G ^ r ] \mathcal{F}^{-1}_{\gamma,\mu}[\widehat{\mathbf{G}}_r] F γ , μ − 1 [ G r ] involves generating the first n + r n+r n + r terms of the GMT system in the companion space described by γ ( μ ) \gamma(\mu) γ ( μ ) . In the experiments, we found that it is much more economical to find the kernel centers γ ( λ 0 ) , … , γ ( λ r ~ − 1 ) \gamma(\lambda_0),\ldots,\gamma(\lambda_{\widetilde r-1}) γ ( λ 0 ) , … , γ ( λ r − 1 ) describing the ROM G ^ r \widehat{\mathbf{G}}_r G r in the companion space and then consider the projected ROM
G r e c o n : = P V r λ [ G ] (65) \mathbf{G}^{\mathrm{econ}}_r:=P_{V_r^{\lambda}}[\mathbf{G}] \tag*{(65)} G r econ := P V r λ [ G ] (65)
in the original RKHS H \mathcal{H} H . In addition to being computationally more efficient, by the Hilbert projection theorem, we also have
∥ G − P V r λ [ G ] ∥ H ≤ ∥ G − ( F γ , λ − 1 ∘ P V ~ r ( λ ) ) [ G ^ ] ∥ H , \left\|\mathbf{G}-P_{V_r^{\lambda}}[\mathbf{G}]\right\|_{\mathcal{H}}\leq\left\|\mathbf{G}-\left(\mathcal{F}^{-1}_{\gamma,\lambda}\circ P_{\widetilde V_r^{(\lambda)}}\right)[\widehat{\mathbf{G}}]\right\|_{\mathcal{H}}, G − P V r λ [ G ] H ≤ G − ( F γ , λ − 1 ∘ P V r ( λ ) ) [ G ] H ,
that is, the economical projection (65) is guaranteed to have an error to the FOM G \mathbf{G} G that is lower than or equal to that of the projected weak solution. We bound the H \mathcal{H} H error of (65) with the following theorem.
Theorem 5.27 (Error of projected ROM). Suppose the assumptions of Theorem 5.25 hold and let G r e c o n \mathbf{G}^{\mathrm{econ}}_r G r econ be defined according to (65) . Then,
∥ G − G r e c o n ∥ H ≤ E ξ r ( G r w e a k ) + ∥ Q ^ ∘ F γ , λ ⊥ − F γ , λ ∘ Q ∥ L ( ( V n λ ) ⊥ , V ~ n ( λ ) ) ∥ G r w e a k ∥ H . \left\|\mathbf{G}-\mathbf{G}^{\mathrm{econ}}_r\right\|_{\mathcal{H}}\leq E^r_{\xi}\left(\mathbf{G}^{\mathrm{weak}}_r\right)+\left\|\widehat{Q}\circ\mathcal{F}^{\perp}_{\gamma,\lambda}-\mathcal{F}_{\gamma,\lambda}\circ Q\right\|_{\mathcal{L}\left((V_n^{\lambda})^{\perp},\widetilde V_n^{(\lambda)}\right)}\left\|\mathbf{G}^{\mathrm{weak}}_r\right\|_{\mathcal{H}}. ∥ G − G r econ ∥ H ≤ E ξ r ( G r weak ) + Q ∘ F γ , λ ⊥ − F γ , λ ∘ Q L ( ( V n λ ) ⊥ , V n ( λ ) ) G r weak H . where G r w e a k = F γ , μ − 1 [ G ^ r ] \mathbf{G}^{\mathrm{weak}}_r=\mathcal{F}^{-1}_{\gamma,\mu}[\widehat{\mathbf{G}}_r] G r weak = F γ , μ − 1 [ G r ] and E ξ r ( ⋅ ) E^r_{\xi}(\cdot) E ξ r ( ⋅ ) as in Problem statement 2.7.
Proof. For ease of notation, denote the projection of G r w e a k \mathbf{G}^{\mathrm{weak}}_r G r weak to V ~ r λ \widetilde V_r^{\lambda} V r λ by G r w e a k − p r o j \mathbf{G}^{\mathrm{weak-proj}}_r G r weak − proj . By the Hilbert projection theorem and since G r e c o n = P V r λ [ G ] \mathbf{G}^{\mathrm{econ}}_r=P_{V_r^{\lambda}}[\mathbf{G}] G r econ = P V r λ [ G ] , we know that
∥ G − G r e c o n ∥ H ≤ ∥ G − G r w e a k − p r o j ∥ H . (66) \left\|\mathbf{G}-\mathbf{G}^{\mathrm{econ}}_r\right\|_{\mathcal{H}}\leq\left\|\mathbf{G}-\mathbf{G}^{\mathrm{weak-proj}}_r\right\|_{\mathcal{H}}. \tag*{(66)} ∥ G − G r econ ∥ H ≤ G − G r weak − proj H . (66)
On the other hand, since the GMT transformation is unitary, we have
E ξ r ( G r w e a k ) = ∥ G − G r w e a k ∥ H = ∥ G ^ − G ^ r ∥ H ^ = E ^ ξ r ( G ^ r ) , E^r_{\xi}\left(\mathbf{G}^{\mathrm{weak}}_r\right)=\left\|\mathbf{G}-\mathbf{G}^{\mathrm{weak}}_r\right\|_{\mathcal{H}}=\left\|\widehat{\mathbf{G}}-\widehat{\mathbf{G}}_r\right\|_{\widehat{\mathcal{H}}}=\widehat E^r_{\xi}\left(\widehat{\mathbf{G}}_r\right), E ξ r ( G r weak ) = G − G r weak H = G − G r H = E ξ r ( G r ) ,
and by Theorem 5.25 and Corollary 5.26,
∥ G r w e a k − G r w e a k − p r o j ∥ H ≤ ∥ Q ^ ∘ F γ , λ ⊥ − F γ , λ ∘ Q ∥ L ( ( V n λ ) ⊥ , V ~ n ( λ ) ) ∥ G r w e a k ∥ H . \left\|\mathbf{G}^{\mathrm{weak}}_r-\mathbf{G}^{\mathrm{weak-proj}}_r\right\|_{\mathcal{H}}\leq\left\|\widehat{Q}\circ\mathcal{F}^{\perp}_{\gamma,\lambda}-\mathcal{F}_{\gamma,\lambda}\circ Q\right\|_{\mathcal{L}\left((V_n^{\lambda})^{\perp},\widetilde V_n^{(\lambda)}\right)}\left\|\mathbf{G}^{\mathrm{weak}}_r\right\|_{\mathcal{H}}. G r weak − G r weak − proj H ≤ Q ∘ F γ , λ ⊥ − F γ , λ ∘ Q L ( ( V n λ ) ⊥ , V n ( λ ) ) G r weak H .
Hence, the triangle inequality gives
∥ G − G r w e a k − p r o j ∥ H ≤ E ξ r ( G r w e a k ) + ∥ Q ^ ∘ F γ , λ ⊥ − F γ , λ ∘ Q ∥ L ( ( V n λ ) ⊥ , V ~ n ( λ ) ) ∥ G r w e a k ∥ H . \left\|\mathbf{G}-\mathbf{G}^{\mathrm{weak-proj}}_r\right\|_{\mathcal{H}}\leq E^r_{\xi}\left(\mathbf{G}^{\mathrm{weak}}_r\right)+\left\|\widehat{Q}\circ\mathcal{F}^{\perp}_{\gamma,\lambda}-\mathcal{F}_{\gamma,\lambda}\circ Q\right\|_{\mathcal{L}\left((V_n^{\lambda})^{\perp},\widetilde V_n^{(\lambda)}\right)}\left\|\mathbf{G}^{\mathrm{weak}}_r\right\|_{\mathcal{H}}. G − G r weak − proj H ≤ E ξ r ( G r weak ) + Q ∘ F γ , λ ⊥ − F γ , λ ∘ Q L ( ( V n λ ) ⊥ , V n ( λ ) ) G r weak H .
Combining this with (66) gives the statement.
Figure 2 summarizes the GMT transformation pipeline introduced above and the subspaces in which the FOM and the two considered ROMs reside. The FOM G ∈ V n λ ⊆ H \mathbf{G} \in V_n^\lambda\subseteq\mathcal{H} G ∈ V n λ ⊆ H is mapped via the unitary GMT transformation F γ , λ \mathcal{F}_{\gamma,\lambda} F γ , λ to G ^ ∈ V ^ n ( λ ) ⊆ H ^ \widehat{\mathbf{G}} \in\widehat{V}_n^{(\lambda)} \subseteq\widehat{\mathcal{H}} G ∈ V n ( λ ) ⊆ H , where MOR is carried out to obtain the companion ROM G ^ r ∈ V ^ r ( λ ~ ) \widehat{\mathbf{G}}_r \in\widehat{V}_r^{(\widetilde{\lambda})} G r ∈ V r ( λ ) . The kernel centers describing G ^ r \widehat{\mathbf{G}}_r G r determine the subspace V ~ r λ ⊆ H \widetilde{V}_r^\lambda\subseteq\mathcal{H} V r λ ⊆ H by γ − 1 \gamma^{-1} γ − 1 . The subspace V ~ r λ \widetilde{V}_r^\lambda V r λ contains the projected ROM G r e c o n = P V ~ r λ [ G ] \mathbf{G}^{\mathrm{econ}}_r = P_{\widetilde{V}_r^\lambda}[\mathbf{G}] G r econ = P V r λ [ G ] from (65) . Applying instead the extended inverse transformation F γ , μ − 1 \mathcal{F}^{-1}_{\gamma,\mu} F γ , μ − 1 directly to G ^ r \widehat{\mathbf{G}}_r G r yields the weak ROM G r w e a k ∈ W r \mathbf{G}^{\mathrm{weak}}_r \in W_r G r weak ∈ W r from (55) . Note that V ^ n + r ( μ ) = V ^ n ( λ ) ∔ V ^ r ( λ ~ ) \widehat{V}_{n+r}^{(\mu)} = \widehat{V}_n^{(\lambda)} \mathbin{\dotplus} \widehat{V}_r^{(\widetilde{\lambda})} V n + r ( μ ) = V n ( λ ) ∔ V r ( λ ) is only a (not necessarily orthogonal) direct sum. This is also true for V n + r μ = V n λ ∔ V ~ r λ V_{n+r}^\mu= V_n^\lambda\mathbin{\dotplus} \widetilde{V}_r^\lambda V n + r μ = V n λ ∔ V r λ .
Figure 2. The GMT transformation pipeline. The FOM G \mathbf{G} G and its companion counterpart G ^ \widehat{\mathbf{G}} G are shown in gray/black; the companion ROM G ^ r \widehat{\mathbf{G}}_r G r obtained by MOR in blue. Inverting the GMT transformation directly gives the weak ROM G r w e a k ∈ W r \mathbf{G}^{\mathrm{weak}}_r \in W_r G r weak ∈ W r (green), while projecting G \mathbf{G} G onto the subspace V ~ r λ \widetilde{V}_r^\lambda V r λ determined by the kernel centers of G ^ r \widehat{\mathbf{G}}_r G r (via γ − 1 \gamma^{-1} γ − 1 ) gives the projected ROM G r e c o n \mathbf{G}^{\mathrm{econ}}_r G r econ (purple).
Before using the GMT transformation to show some existence theorems of the MOR problem in RKHSs, we show a more practical approach to compute it. In particular, we directly obtain GMT coefficients of G \mathbf{G} G given the kernel centers λ k \lambda_k λ k and linear weights r k \mathfrak{r}_k r k without having to compute the GMT functions themselves.
Theorem 5.28 (GMT transformation without basis change). Suppose that Assumption 5.21 is satisfied and let ( λ 0 , … , λ n − 1 ) ∈ Ω ± n (\lambda_0,\ldots,\lambda_{n-1}) \in\Omega_{\pm}^{n} ( λ 0 , … , λ n − 1 ) ∈ Ω ± n . Let
G = ∑ k = 0 n − 1 r k ξ λ k ∈ V n λ ⊆ H , F γ , λ [ G ] = G ^ = ∑ k = 0 n − 1 r ^ k ξ ^ γ ( λ k ) ∈ V ^ n γ ( λ ) ⊆ H ^ . G=\sum_{k=0}^{n-1}r_k\xi_{\lambda_k}\in V_n^\lambda\subseteq\mathcal{H},\qquad\mathcal{F}_{\gamma,\lambda}[G]=\widehat{G}=\sum_{k=0}^{n-1}\widehat{r}_k\widehat{\xi}_{\gamma(\lambda_k)}\in\widehat{V}_n^{\gamma(\lambda)}\subseteq\widehat{\mathcal{H}}. G = k = 0 ∑ n − 1 r k ξ λ k ∈ V n λ ⊆ H , F γ , λ [ G ] = G = k = 0 ∑ n − 1 r k ξ γ ( λ k ) ∈ V n γ ( λ ) ⊆ H . Using the biorthogonal basis from Definition 5.13 , let
c = [ r 0 , r 1 , … , r n − 1 ] T : = [ ⟨ G , ψ 0 ⟩ H , … , ⟨ G , ψ n − 1 ⟩ H ] T ∈ C n . \mathbf{c}=[r_0,r_1,\ldots,r_{n-1}]^{\mathsf{T}}:=\left[\langle G,\psi_0\rangle_{\mathcal{H}},\ldots,\langle G,\psi_{n-1}\rangle_{\mathcal{H}}\right]^{\mathsf{T}}\in\mathbb{C}^{n}. c = [ r 0 , r 1 , … , r n − 1 ] T := [ ⟨ G , ψ 0 ⟩ H , … , ⟨ G , ψ n − 1 ⟩ H ] T ∈ C n . Moreover, let Ξ = L L ∗ \Xi=LL^{*} Ξ = L L ∗ and Ξ ^ = L ^ L ^ ∗ \widehat{\Xi}=\widehat{L}\widehat{L}^{*} Ξ = L L ∗ be the Cholesky factorizations of the Gramian matrices
Ξ : = [ ⟨ ξ λ k , ξ λ j ⟩ H ] k , j = 0 n − 1 , Ξ ^ : = [ ⟨ ξ ^ γ ( λ k ) , ξ ^ γ ( λ j ) ⟩ H ^ ] k , j = 0 n − 1 . \Xi:=\left[\langle\xi_{\lambda_k},\xi_{\lambda_j}\rangle_{\mathcal{H}}\right]_{k,j=0}^{n-1},\qquad\widehat{\Xi}:=\left[\langle\widehat{\xi}_{\gamma(\lambda_k)},\widehat{\xi}_{\gamma(\lambda_j)}\rangle_{\widehat{\mathcal{H}}}\right]_{k,j=0}^{n-1}. Ξ := [ ⟨ ξ λ k , ξ λ j ⟩ H ] k , j = 0 n − 1 , Ξ := [ ⟨ ξ γ ( λ k ) , ξ γ ( λ j ) ⟩ H ] k , j = 0 n − 1 . Then, with P : = L ^ − T L T ∈ C n × n \mathfrak{P}:=\widehat{L}^{-\mathsf{T}}L^{\mathsf{T}}\in\mathbb{C}^{n\times n} P := L − T L T ∈ C n × n , the linear coefficients of the ROM are given by
r ^ k = [ P c ] k ( k = 0 , … , n − 1 ) . (67) \widehat{r}_k=[\mathfrak{P}\mathbf{c}]_k\qquad(k=0,\ldots,n-1). \tag*{(67)} r k = [ P c ] k ( k = 0 , … , n − 1 ) . (67)
Proof. For all k = 0 , … , n − 1 k=0,\ldots,n-1 k = 0 , … , n − 1 , we have φ k λ ∈ V n λ \varphi_k^\lambda\in V_n^\lambda φ k λ ∈ V n λ and
φ k λ = ∑ j = 0 k r k , j ξ λ j . (68) \varphi_k^\lambda=\sum_{j=0}^{k}r_{k,j}\xi_{\lambda_j}. \tag*{(68)} φ k λ = j = 0 ∑ k r k , j ξ λ j . (68)
Hence,
⟨ G , φ k λ ⟩ H = ⟨ G , ∑ j = 0 k r k , j ξ λ j ⟩ H = ∑ j = 0 k r k , j ‾ ⟨ G , ξ λ j ⟩ H = ∑ j = 0 k r k , j ‾ G ( λ j ) . \langle G,\varphi_k^\lambda\rangle_{\mathcal{H}}=\left\langle G,\sum_{j=0}^{k}r_{k,j}\xi_{\lambda_j}\right\rangle_{\mathcal{H}}=\sum_{j=0}^{k}\overline{r_{k,j}}\langle G,\xi_{\lambda_j}\rangle_{\mathcal{H}}=\sum_{j=0}^{k}\overline{r_{k,j}}G(\lambda_j). ⟨ G , φ k λ ⟩ H = ⟨ G , j = 0 ∑ k r k , j ξ λ j ⟩ H = j = 0 ∑ k r k , j ⟨ G , ξ λ j ⟩ H = j = 0 ∑ k r k , j G ( λ j ) .
The same relation holds between φ ^ k γ ( λ ) \widehat{\varphi}_k^{\gamma(\lambda)} φ k γ ( λ ) and ξ ^ γ ( λ k ) \widehat{\xi}_{\gamma(\lambda_k)} ξ γ ( λ k ) , namely,
φ ^ k γ ( λ ) = ∑ i = 0 k r ^ k , i ξ ^ γ ( λ i ) . \widehat{\varphi}_k^{\gamma(\lambda)}=\sum_{i=0}^{k}\widehat{r}_{k,i}\widehat{\xi}_{\gamma(\lambda_i)}. φ k γ ( λ ) = i = 0 ∑ k r k , i ξ γ ( λ i ) .
Substituting these expressions into Definition 5.22 , we obtain
F γ , λ [ G ] = ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ ^ k γ ( λ ) = ∑ k = 0 n − 1 ( ∑ j = 0 k r k , j ‾ G ( λ j ) ) ( ∑ i = 0 k r ^ k , i ξ ^ γ ( λ i ) ) = ∑ i = 0 n − 1 ( ∑ j = 0 n − 1 ( ∑ k = 0 n − 1 r ^ k , i r k , j ‾ ) G ( λ j ) ) ξ ^ γ ( λ i ) , \begin{aligned}
\mathcal{F}_{\gamma,\lambda}[G]=\sum_{k=0}^{n-1}\langle G,\varphi_k^\lambda\rangle_{\mathcal{H}}\widehat{\varphi}_k^{\gamma(\lambda)} \\
&=\sum_{k=0}^{n-1}\left(\sum_{j=0}^{k}\overline{r_{k,j}}G(\lambda_j)\right)\left(\sum_{i=0}^{k}\widehat{r}_{k,i}\widehat{\xi}_{\gamma(\lambda_i)}\right)=\sum_{i=0}^{n-1}\left(\sum_{j=0}^{n-1}\left(\sum_{k=0}^{n-1}\widehat{r}_{k,i}\overline{r_{k,j}}\right)G(\lambda_j)\right)\widehat{\xi}_{\gamma(\lambda_i)},
\end{aligned} F γ , λ [ G ] = k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k γ ( λ ) = k = 0 ∑ n − 1 ( j = 0 ∑ k r k , j G ( λ j ) ) ( i = 0 ∑ k r k , i ξ γ ( λ i ) ) = i = 0 ∑ n − 1 ( j = 0 ∑ n − 1 ( k = 0 ∑ n − 1 r k , i r k , j ) G ( λ j ) ) ξ γ ( λ i ) ,
where we use that r k , i = 0 r_{k,i}=0 r k , i = 0 whenever i > k i>k i > k . Define
B : = [ ∑ k = 0 n − 1 r ^ k , i r k , j ‾ ] i , j = 0 n − 1 ∈ C n × n . \mathfrak{B}:=\left[\sum_{k=0}^{n-1}\widehat{r}_{k,i}\overline{r_{k,j}}\right]_{i,j=0}^{n-1}\in\mathbb{C}^{n\times n}. B := [ k = 0 ∑ n − 1 r k , i r k , j ] i , j = 0 n − 1 ∈ C n × n .
Then
F γ , λ [ G ] = ∑ k = 0 n − 1 [ B c ~ ] k ξ γ ( λ k ) , where c ~ : = [ G ( λ 0 ) , … , G ( λ n − 1 ) ] T . \mathcal{F}_{\gamma,\lambda}[\mathbf{G}] = \sum_{k=0}^{n-1} [\mathbb{B}\widetilde{\mathbf{c}}]_k \xi_{\gamma(\lambda_k)}, \qquad\text{where } \widetilde{\mathbf{c}} := [G(\lambda_0),\ldots,G(\lambda_{n-1})]^{\mathsf{T}}. F γ , λ [ G ] = k = 0 ∑ n − 1 [ B c ] k ξ γ ( λ k ) , where c := [ G ( λ 0 ) , … , G ( λ n − 1 ) ] T .
Next, we show that
B = L ^ − T L − 1 , \mathbb{B} = \widehat{L}^{-\mathsf{T}}L^{-1}, B = L − T L − 1 ,
where L ^ \widehat{L} L and L L L are obtained from the Cholesky factorizations of Ξ ^ \widehat{\Xi} Ξ and Ξ \Xi Ξ , respectively. The Cholesky factorizations exist because the Gram matrices Ξ ^ \widehat{\Xi} Ξ and Ξ \Xi Ξ are positive definite under our assumptions.
Introducing the vector of basis functions Φ = [ φ 0 λ , … , φ n − 1 λ ] T \Phi= [\varphi_0^\lambda,\ldots,\varphi_{n-1}^\lambda]^{\mathsf{T}} Φ = [ φ 0 λ , … , φ n − 1 λ ] T , by (68) we obtain
Φ = R [ ξ λ 0 , … , ξ λ n − 1 ] T , \Phi= R[\xi_{\lambda_0},\ldots,\xi_{\lambda_{n-1}}]^{\mathsf{T}}, Φ = R [ ξ λ 0 , … , ξ λ n − 1 ] T ,
where R = [ r k , j ] k , j = 0 n − 1 R = [r_{k,j}]_{k,j=0}^{n-1} R = [ r k , j ] k , j = 0 n − 1 . By Definition 5.2 , the diagonal coefficients of R R R satisfy r k , k = ∥ ξ λ k − S k λ [ ξ λ k ] ∥ H > 0 r_{k,k} = \lVert\xi_{\lambda_k} - S_k^\lambda[\xi_{\lambda_k}] \rVert_{\mathcal{H}} > 0 r k , k = ∥ ξ λ k − S k λ [ ξ λ k ] ∥ H > 0 for k = 0 , … , n − 1 k = 0,\ldots,n-1 k = 0 , … , n − 1 . Since ⟨ φ k λ , φ j λ ⟩ H = δ j , k \langle\varphi_k^\lambda,\varphi_j^\lambda\rangle_{\mathcal{H}} = \delta_{j,k} ⟨ φ k λ , φ j λ ⟩ H = δ j , k . This gives
I n = [ ⟨ φ k λ , φ j λ ⟩ H ] k , j = 0 n − 1 = R [ ⟨ ξ λ k , ξ λ j ⟩ H ] k , j = 0 n − 1 R ∗ = R Ξ R ∗ . I_n = \left[\langle\varphi_k^\lambda,\varphi_j^\lambda\rangle_{\mathcal{H}}\right]_{k,j=0}^{n-1} = R\left[\langle\xi_{\lambda_k},\xi_{\lambda_j}\rangle_{\mathcal{H}}\right]_{k,j=0}^{n-1}R^* = R\Xi R^*. I n = [ ⟨ φ k λ , φ j λ ⟩ H ] k , j = 0 n − 1 = R [ ⟨ ξ λ k , ξ λ j ⟩ H ] k , j = 0 n − 1 R ∗ = R Ξ R ∗ .
Substituting the Cholesky factorization Ξ = L L ∗ \Xi= LL^* Ξ = L L ∗ yields R L L ∗ R ∗ = ( R L ) ( R L ) ∗ = I n RLL^*R^* = (RL)(RL)^* = I_n R L L ∗ R ∗ = ( R L ) ( R L ) ∗ = I n . Thus, R L RL R L is unitary. Since both R R R and L L L are lower triangular, R L RL R L is lower triangular. Moreover, since r k , k > 0 r_{k,k} > 0 r k , k > 0 by the Gram–Schmidt construction and [ L ] k , k > 0 [L]_{k,k} > 0 [ L ] k , k > 0 by the construction of the Cholesky factorization, its diagonal entries are [ R L ] k , k = r k , k [ L ] k , k [RL]_{k,k} = r_{k,k}[L]_{k,k} [ R L ] k , k = r k , k [ L ] k , k are positive for k = 0 , … , n − 1 k = 0,\ldots,n-1 k = 0 , … , n − 1 . Therefore, R L = I n RL = I_n R L = I n , i.e., R = L − 1 R = L^{-1} R = L − 1 . Analogously, R ^ = L ^ − 1 \widehat{R} = \widehat{L}^{-1} R = L − 1 . By the definition of B \mathbb{B} B , we have B = L ^ − T L − 1 = R ^ T R \mathbb{B} = \widehat{L}^{-\mathsf{T}}L^{-1} = \widehat{R}^{\mathsf{T}}R B = L − T L − 1 = R T R . It remains to relate the evaluation coefficients c ~ \widetilde{\mathbf{c}} c to the kernel coefficients c \mathbf{c} c . By the reproducing property,
c ~ j : = [ c ~ ] j = G ( λ j ) = ⟨ G , ξ λ j ⟩ H . \widetilde{c}_j := [\widetilde{\mathbf{c}}]_j = G(\lambda_j) = \langle G,\xi_{\lambda_j}\rangle_{\mathcal{H}}. c j := [ c ] j = G ( λ j ) = ⟨ G , ξ λ j ⟩ H .
Since G = ∑ k = 0 n − 1 r k ξ λ k G = \sum_{k=0}^{n-1} r_k\xi_{\lambda_k} G = ∑ k = 0 n − 1 r k ξ λ k , it follows that
c ~ j = ∑ k = 0 n − 1 r k ⟨ ξ λ k , ξ λ j ⟩ H = ∑ k = 0 n − 1 Ξ k , j r k ( j = 0 , … , n − 1 ) . \widetilde{c}_j = \sum_{k=0}^{n-1}r_k\langle\xi_{\lambda_k},\xi_{\lambda_j}\rangle_{\mathcal{H}} = \sum_{k=0}^{n-1}\Xi_{k,j}r_k \qquad(j=0,\ldots,n-1). c j = k = 0 ∑ n − 1 r k ⟨ ξ λ k , ξ λ j ⟩ H = k = 0 ∑ n − 1 Ξ k , j r k ( j = 0 , … , n − 1 ) .
Therefore, c ~ = Ξ T c \widetilde{\mathbf{c}} = \Xi^{\mathsf{T}}\mathbf{c} c = Ξ T c , and using Ξ = L L ∗ \Xi= LL^* Ξ = L L ∗ , we obtain c ~ = Ξ T c = L L T c \widetilde{\mathbf{c}} = \Xi^{\mathsf{T}}\mathbf{c} = LL^{\mathsf{T}}\mathbf{c} c = Ξ T c = L L T c . Combining this with the expression for B \mathbb{B} B gives c ^ = B c ~ = L ^ − T L − 1 L L T c = L ^ − T L T c \widehat{\mathbf{c}} = \mathbb{B}\widetilde{\mathbf{c}} = \widehat{L}^{-\mathsf{T}}L^{-1}LL^{\mathsf{T}}\mathbf{c} = \widehat{L}^{-\mathsf{T}}L^{\mathsf{T}}\mathbf{c} c = B c = L − T L − 1 L L T c = L − T L T c , which proves (67) .
Remark 5.29. While Theorem 5.28 allows for the computation of G ^ = F γ , λ [ G ] \widehat{G} = \mathcal{F}_{\gamma,\lambda}[G] G = F γ , λ [ G ] without having to express G G G in a GMT basis in H \mathcal{H} H , or generate the corresponding GMT basis in H ^ \widehat{\mathcal{H}} H , it does not remove all of the computational difficulty for GMT transformations. In particular, if the kernel slices defined by γ ( λ 0 ) , … , γ ( λ n − 1 ) \gamma(\lambda_0),\ldots,\gamma(\lambda_{n-1}) γ ( λ 0 ) , … , γ ( λ n − 1 ) are nearly linearly dependent in the companion RKHS H ^ \widehat{\mathcal{H}} H , then inverting L ^ \widehat{L} L from Theorem 5.28 can still pose a challenge for large n n n . However, if one can identify a family of RKHSs where such a computation remains stable, even if the points γ ( λ k ) \gamma(\lambda_k) γ ( λ k ) for k = 0 , … , n − 1 k = 0,\ldots,n-1 k = 0 , … , n − 1 are clustered in Ω ^ \widehat{\Omega} Ω , then one could transform MOR problems to these spaces without further numerical issues. We do not further pursue the rigorous identification of such nicely behaved RKHSs; however, based on our presented experiments in § 8 , we empirically find several candidate spaces.
Finally, we clarify the existence of local minimum points of E ξ r E_{\xi}^{r} E ξ r in RKHSs with sufficiently smooth kernels.
Theorem 5.30. Let Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C be a compact set and H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C with kernel ξ \xi ξ . Let G ∈ G ξ n G \in\mathfrak{G}_{\xi}^{n} G ∈ G ξ n , and assume ( λ ↦ ξ λ ) ∈ C r ( Ω , H ) (\lambda\mapsto\xi_{\lambda}) \in C^{r}(\Omega,\mathcal{H}) ( λ ↦ ξ λ ) ∈ C r ( Ω , H ) for some r < n r<n r < n . Finally, suppose that D λ j ξ λ ∈ H D_{\lambda}^{j}\xi_{\lambda} \in\mathcal{H} D λ j ξ λ ∈ H for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 and all λ ∈ Ω \lambda\in\Omega λ ∈ Ω . Then, there exists λ ~ o p t ∈ Ω r \widetilde{\boldsymbol{\lambda}}^{\mathrm{opt}} \in\Omega^{r} λ opt ∈ Ω r such that
E ξ r ( P λ ~ o p t [ G ] ) = min λ ~ ∈ Ω r E ξ r ( P λ ~ [ G ] ) . E_{\xi}^{r}(\mathcal{P}_{\widetilde{\boldsymbol{\lambda}}^{\mathrm{opt}}}[G])=\min_{\widetilde{\boldsymbol{\lambda}}\in\Omega^{r}}E_{\xi}^{r}(\mathcal{P}_{\widetilde{\boldsymbol{\lambda}}}[G]). E ξ r ( P λ opt [ G ]) = λ ∈ Ω r min E ξ r ( P λ [ G ]) . Moreover, if λ ~ o p t \widetilde{\boldsymbol{\lambda}}^{\mathrm{opt}} λ opt lies in the interior of Ω r \Omega^{r} Ω r , then λ ~ o p t \widetilde{\boldsymbol{\lambda}}^{\mathrm{opt}} λ opt is a zero of the equilibrium function given in Definition 4.12 .
Proof. We show that the map λ ~ ↦ P λ ~ [ G ] \widetilde{\boldsymbol{\lambda}}\mapsto\mathcal{P}_{\widetilde{\boldsymbol{\lambda}}}[G] λ ↦ P λ [ G ] is continuous with respect to the norm induced on H \mathcal{H} H . Recall that by Lemma 5.11, we have P λ ~ [ G ] = S ~ r λ ~ \mathcal{P}_{\widetilde{\boldsymbol{\lambda}}}[G]=\widetilde{S}_{r}^{\widetilde{\boldsymbol{\lambda}}} P λ [ G ] = S r λ . This shows that if λ ~ \widetilde{\boldsymbol{\lambda}} λ has no repeated kernel centers, then there exists a neighborhood of λ ~ \widetilde{\boldsymbol{\lambda}} λ where continuity is given by the continuity of the inner product on H \mathcal{H} H . Otherwise, if λ ~ ∈ Ω r \widetilde{\boldsymbol{\lambda}}\in\Omega^{r} λ ∈ Ω r has repeated kernel centers, without loss of generality, assume that these repetitions are grouped. Formally, we assume that
λ ~ = ( λ ~ 0 , … , λ ~ 0 ⏟ m 0 times , … , λ ~ k − 1 , … , λ ~ k − 1 ⏟ m k − 1 times ) \widetilde{\boldsymbol{\lambda}}=\left(\underbrace{\widetilde{\lambda}_{0},\ldots,\widetilde{\lambda}_{0}}_{m_{0}\ \text{times}},\ldots,\underbrace{\widetilde{\lambda}_{k-1},\ldots,\widetilde{\lambda}_{k-1}}_{m_{k-1}\ \text{times}}\right) λ = m 0 times λ 0 , … , λ 0 , … , m k − 1 times λ k − 1 , … , λ k − 1
Consider the corresponding divided-difference basis
τ [ λ ~ j ] = ξ λ ~ j ( j = 0 , … , r − 1 ) \tau[\widetilde{\lambda}_{j}]=\xi_{\widetilde{\lambda}_{j}}\qquad(j=0,\ldots,r-1) τ [ λ j ] = ξ λ j ( j = 0 , … , r − 1 )
τ [ λ ~ j , … , λ ~ j + q ] = { τ [ λ ~ j + 1 , … , λ ~ j + q ] − τ [ λ ~ j , … , λ ~ j + q − 1 ] λ ~ j + q − λ ~ j , if λ ~ j ≠ λ ~ j + q , 1 q ! D λ ~ j q ξ λ ~ j , if λ ~ j = λ ~ j + q ( j + q ≤ r − 1 ) . (69) \tau[\widetilde{\lambda}_{j},\ldots,\widetilde{\lambda}_{j+q}]=
\begin{cases}
\dfrac{\tau[\widetilde{\lambda}_{j+1},\ldots,\widetilde{\lambda}_{j+q}]-\tau[\widetilde{\lambda}_{j},\ldots,\widetilde{\lambda}_{j+q-1}]}{\widetilde{\lambda}_{j+q}-\widetilde{\lambda}_{j}}, & \text{if }\widetilde{\lambda}_{j}\ne\widetilde{\lambda}_{j+q},\\
\dfrac{1}{q!}D_{\widetilde{\lambda}_{j}}^{q}\xi_{\widetilde{\lambda}_{j}}, & \text{if }\widetilde{\lambda}_{j}=\widetilde{\lambda}_{j+q}
\end{cases}
\qquad(j+q\le r-1). \tag*{(69)} τ [ λ j , … , λ j + q ] = ⎩ ⎨ ⎧ λ j + q − λ j τ [ λ j + 1 , … , λ j + q ] − τ [ λ j , … , λ j + q − 1 ] , q ! 1 D λ j q ξ λ j , if λ j = λ j + q , if λ j = λ j + q ( j + q ≤ r − 1 ) . (69)
Notice first [15 Proposition 35] that
V λ ~ r = span { τ [ λ ~ 0 ] , τ [ λ ~ 0 , λ ~ 1 ] , … , τ [ λ ~ 0 , … , λ ~ r − 1 ] } = span { φ j λ ~ } j = 0 r − 1 ⊂ H V_{\widetilde{\boldsymbol{\lambda}}}^{r}=\operatorname{span}\{\tau[\widetilde{\lambda}_{0}],\tau[\widetilde{\lambda}_{0},\widetilde{\lambda}_{1}],\ldots,\tau[\widetilde{\lambda}_{0},\ldots,\widetilde{\lambda}_{r-1}]\}=\operatorname{span}\{\varphi_{j}^{\widetilde{\lambda}}\}_{j=0}^{r-1}\subset\mathcal{H} V λ r = span { τ [ λ 0 ] , τ [ λ 0 , λ 1 ] , … , τ [ λ 0 , … , λ r − 1 ]} = span { φ j λ } j = 0 r − 1 ⊂ H
for any λ ~ ∈ Ω r \widetilde{\boldsymbol{\lambda}}\in\Omega^{r} λ ∈ Ω r , where φ j λ \varphi_{j}^{\lambda} φ j λ denote the GMT functions constructed according to Definition 5.2 . Consequently, τ [ λ ~ 0 ] , … , τ [ λ ~ 0 , … , λ ~ r − 1 ] \tau[\widetilde{\lambda}_{0}],\ldots,\tau[\widetilde{\lambda}_{0},\ldots,\widetilde{\lambda}_{r-1}] τ [ λ 0 ] , … , τ [ λ 0 , … , λ r − 1 ] form a linearly independent system. The Hermite–Genocchi formula for divided differences (see, e.g., [3 Theorem 3.3] or [15 ] ) asserts the continuity of τ [ λ ~ i , … , λ ~ i + k ] \tau[\widetilde{\lambda}_i,\ldots,\widetilde{\lambda}_{i+k}] τ [ λ i , … , λ i + k ] , even with repeated kernel centers. Then, we can express the projection operator P V r λ ~ P_{\mathcal{V}_r^{\widetilde{\lambda}}} P V r λ in terms of the basis in (69) , i.e.,
P V r λ ~ [ ⋅ ] = ∑ k , j = 0 r − 1 [ U ( λ ~ ) − 1 ] k , j ⟨ ⋅ , τ [ λ ~ 0 , … , λ ~ j ] ⟩ H τ [ λ ~ 0 , … , λ ~ k ] . P_{\mathcal{V}_r^{\widetilde{\lambda}}}[\,\cdot\,]=\sum_{k,j=0}^{r-1}\left[\mathbf{U}(\widetilde{\lambda})^{-1}\right]_{k,j}\langle\,\cdot\,,\tau[\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_j]\rangle_{\mathcal{H}}\tau[\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_k]. P V r λ [ ⋅ ] = k , j = 0 ∑ r − 1 [ U ( λ ) − 1 ] k , j ⟨ ⋅ , τ [ λ 0 , … , λ j ] ⟩ H τ [ λ 0 , … , λ k ] .
where U ( λ ~ ) \mathbf{U}(\widetilde{\lambda}) U ( λ ) denotes the Gramian matrix
U ( λ ~ ) = [ ⟨ τ [ λ ~ 0 , … , λ ~ k ] , τ [ λ ~ 0 , … , λ ~ j ] ⟩ H ] k , j = 0 r − 1 . \mathbf{U}(\widetilde{\lambda})=\left[\left\langle\tau[\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_k],\tau[\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_j]\right\rangle_{\mathcal{H}}\right]_{k,j=0}^{r-1}. U ( λ ) = [ ⟨ τ [ λ 0 , … , λ k ] , τ [ λ 0 , … , λ j ] ⟩ H ] k , j = 0 r − 1 .
We observe that:
The scalar–valued functions [ U ( ⋅ ) − 1 ] k , j \left[\mathbf{U}(\cdot)^{-1}\right]_{k,j} [ U ( ⋅ ) − 1 ] k , j , k , j = 0 , … , r − 1 k,j=0,\ldots,r-1 k , j = 0 , … , r − 1 , are continuous.
The operators T k , j ( λ ~ ) [ ⋅ ] : = ⟨ ⋅ , τ [ λ ~ 0 , … , λ ~ j ] ⟩ H τ [ λ ~ 0 , … , λ ~ k ] \mathcal{T}_{k,j}(\widetilde{\lambda})[\,\cdot\,]:=\langle\,\cdot\,,\tau[\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_j]\rangle_{\mathcal{H}}\tau[\widetilde{\lambda}_0,\ldots,\widetilde{\lambda}_k] T k , j ( λ ) [ ⋅ ] := ⟨ ⋅ , τ [ λ 0 , … , λ j ] ⟩ H τ [ λ 0 , … , λ k ] , k , j = 0 , … , r − 1 k,j=0,\ldots,r-1 k , j = 0 , … , r − 1 , are continuous in the operator norm. Indeed, we have
∥ T k , j ( λ ~ ) − T k , j ( λ ^ ) ∥ L ( H ) → 0 \left\|\mathcal{T}_{k,j}(\widetilde{\lambda})-\mathcal{T}_{k,j}(\widehat{\lambda})\right\|_{\mathcal{L}(\mathcal{H})}\to0 T k , j ( λ ) − T k , j ( λ ) L ( H ) → 0
as λ ~ → λ ^ \widetilde{\lambda}\to\widehat{\lambda} λ → λ (in any p p p –vector norm). This follows directly from the continuity of the divided differences τ \tau τ with respect to the H \mathcal{H} H norm.
Consequently, the map λ ~ ↦ P V r λ ~ \widetilde{\lambda}\mapsto P_{\mathcal{V}_r^{\widetilde{\lambda}}} λ ↦ P V r λ is continuous in the operator norm on Ω r \Omega^r Ω r , so is λ ~ ↦ E V r λ ~ ξ [ G ] \widetilde{\lambda}\mapsto\mathcal{E}^{\xi}_{\mathcal{V}_r^{\widetilde{\lambda}}}[\mathbf{G}] λ ↦ E V r λ ξ [ G ] . Since Ω r \Omega^r Ω r is compact, the extreme value theorem guarantees the existence of λ ~ o p t ∈ Ω r \widetilde{\lambda}^{\mathrm{opt}}\in\Omega^r λ opt ∈ Ω r attaining min λ ~ ∈ Ω r E V r λ ~ ξ [ G ] \min_{\widetilde{\lambda}\in\Omega^r}\mathcal{E}^{\xi}_{\mathcal{V}_r^{\widetilde{\lambda}}}[\mathbf{G}] min λ ∈ Ω r E V r λ ξ [ G ] , which proves the first statement.
For the second statement, suppose λ ~ o p t \widetilde{\lambda}^{\mathrm{opt}} λ opt lies in the interior of Ω r \Omega^r Ω r . Then there exists a δ > 0 \delta>0 δ > 0 such that the open ball of radius δ \delta δ around λ ~ o p t \widetilde{\lambda}^{\mathrm{opt}} λ opt is contained in Ω r \Omega^r Ω r , so G r o p t : = P V r λ ~ o p t [ G ] \mathbf{G}_r^{\mathrm{opt}}:=P_{\mathcal{V}_r^{\widetilde{\lambda}^{\mathrm{opt}}}}[\mathbf{G}] G r opt := P V r λ opt [ G ] is a locally optimal ROM in the sense of Definition 2.8 . By Theorem 4.13 , this yields D ( λ ~ o p t ) = 0 \mathcal{D}(\widetilde{\lambda}^{\mathrm{opt}})=0 D ( λ opt ) = 0 .
MOR methods
We now propose MOR algorithms in RKHSs. First, we consider a method for MOR problems posed in RKHSs that satisfy the conditions of Theorems 4.3 and 4.4 . Then, we use GMT transformations to solve the MOR problem in RKHSs with kernels that are not sufficiently smooth.
To clarify the setting of our proposed method, we make the following assumption.
Assumption 6.1. Suppose that H \mathcal{H} H is an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C with kernel ξ \xi ξ such that ξ ∈ C 2 ( Ω 2 , C ) \xi\in C^2(\Omega^2,\mathbb{C}) ξ ∈ C 2 ( Ω 2 , C ) .
Under Assumption 6.1 , we propose a norm–optimal RKHS–MOR method in Algorithm 6.1. Here, we assume that the initial and all intermediate kernel centers are pairwise different. If that was not the case, we would need higher regularity in the second kernel variable.
Algorithm 6.1 H \mathcal{H} H –optimal MOR in RKHSs with C 2 C^2 C 2 kernels
Inputs: RKHS H \mathcal{H} H with kernel ξ \xi ξ , FOM G ∈ G ξ n \mathbf{G} \in\mathcal{G}_{\xi}^{n} G ∈ G ξ n , reduced order r < n r<n r < n , initial ROM kernel centers λ ~ ( 0 ) ∈ Ω r r \widetilde{\boldsymbol{\lambda}}^{(0)} \in\Omega_r^{r} λ ( 0 ) ∈ Ω r r , termination tolerance ε > 0 \varepsilon>0 ε > 0 .
Outputs: ROM G r ∈ G ξ r \mathbf{G}_r \in\mathcal{G}_{\xi}^{r} G r ∈ G ξ r .
m ← 0 m \leftarrow0 m ← 0 .
ε ( 0 ) ← ∞ \varepsilon^{(0)} \leftarrow\infty ε ( 0 ) ← ∞ .
while ε ( m ) > ε \varepsilon^{(m)}>\varepsilon ε ( m ) > ε do
J ← [ ∂ D ( λ ~ ( m ) ) i ∂ λ ~ j ] i , j = 0 r − 1 \mathbf{J} \leftarrow\left[\frac{\partial\mathcal{D}(\widetilde{\boldsymbol{\lambda}}^{(m)})_i}{\partial\widetilde{\lambda}_j}\right]_{i,j=0}^{r-1} J ← [ ∂ λ j ∂ D ( λ ( m ) ) i ] i , j = 0 r − 1 . ▹ D \triangleright\mathcal{D} ▹ D as in Definition 4.12 .
λ ~ ( m + 1 ) ← λ ~ ( m ) − J − 1 D ( λ ~ ( m ) ) \widetilde{\boldsymbol{\lambda}}^{(m+1)} \leftarrow\widetilde{\boldsymbol{\lambda}}^{(m)}-\mathbf{J}^{-1}\mathcal{D}\left(\widetilde{\boldsymbol{\lambda}}^{(m)}\right) λ ( m + 1 ) ← λ ( m ) − J − 1 D ( λ ( m ) ) .
ε ( m ) ← ∥ λ ~ ( m + 1 ) − λ ~ ( m ) ∥ ∞ \varepsilon^{(m)} \leftarrow\left\|\widetilde{\boldsymbol{\lambda}}^{(m+1)}-\widetilde{\boldsymbol{\lambda}}^{(m)}\right\|_{\infty} ε ( m ) ← λ ( m + 1 ) − λ ( m ) ∞ .
m ← m + 1 m \leftarrow m+1 m ← m + 1 .
end while
G r ← P λ ~ ( m ) V r ξ [ G ] \mathbf{G}_r \leftarrow\mathcal{P}_{\widetilde{\boldsymbol{\lambda}}^{(m)}}^{\mathcal{V}_r^{\xi}}[\mathbf{G}] G r ← P λ ( m ) V r ξ [ G ] .
return G r \mathbf{G}_r G r .
While Algorithm 6.1 performs well under Assumption 6.1, in practice it has several limitations. First, it can only be applied in RKHSs equipped with a sufficiently smooth kernel. Several classes of FOMs, however, cannot be embedded into generalized model spaces of such RKHSs. For example, ReLU networks considered in § 8 do not meet this requirement. In addition, similarly to classical IRKA [31 ] , the update steps in Algorithm 6.1 do not guarantee that λ ~ \widetilde{\boldsymbol{\lambda}} λ remains inside Ω r \Omega^r Ω r throughout the iteration. In the H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) setting, this problem is addressed in [1 ] . The proposed solution modifies the Newton update in Algorithm 6.1 to be compatible with the hyperbolic geometry induced by the pseudohyperbolic metric on the manifold D \mathbb{D} D . In particular, the Newton update is performed along hyperbolic geodesics, which force the parameter vector λ \boldsymbol{\lambda} λ to remain inside D r \mathbb{D}^r D r in each iteration of the method.
Algorithm 6.2 GMT transformation based RKHS–MOR method.
Inputs: RKHS H \mathcal{H} H with kernel ξ \xi ξ , FOM G ∈ G ξ n \mathbf{G} \in\mathbb{G}_{\xi}^{n} G ∈ G ξ n defined by the kernel centers λ ∈ Ω n \boldsymbol{\lambda} \in\Omega^{n} λ ∈ Ω n , reduced order r < n r<n r < n , initial ROM kernel centers λ ~ ( 0 ) ∈ Ω r \widetilde{\boldsymbol{\lambda}}^{(0)} \in\Omega^{r} λ ( 0 ) ∈ Ω r , bijection γ : Ω → D \gamma:\Omega\to\mathbb{D} γ : Ω → D .
Outputs: weak ROM G r w e a k ∈ W ξ r \mathbf{G}_{r}^{\mathrm{weak}} \in\mathbb{W}_{\xi}^{r} G r weak ∈ W ξ r .
G ^ ← ∑ k = 0 n − 1 ⟨ G , φ k γ ( λ ) ⟩ H φ ^ k γ ( λ ) ∈ H 2 ( D ) \widehat{\mathbf{G}} \gets\sum_{k=0}^{n-1}\left\langle\mathbf{G},\varphi_{k}^{\gamma(\lambda)}\right\rangle_{\mathcal{H}}\widehat{\varphi}_{k}^{\gamma(\lambda)} \in H_{2}(\mathbb{D}) G ← ∑ k = 0 n − 1 ⟨ G , φ k γ ( λ ) ⟩ H φ k γ ( λ ) ∈ H 2 ( D ) . ▷ GMT transformation.
G ^ r ← \widehat{\mathbf{G}}_{r} \gets G r ← output of Algorithm 3.1 with inputs G ^ \widehat{\mathbf{G}} G , r r r , and γ ( λ ~ ( 0 ) ) \gamma\left(\widetilde{\boldsymbol{\lambda}}^{(0)}\right) γ ( λ ( 0 ) ) .
G r w e a k ← F γ , μ − 1 [ G ^ r ] ∈ W ξ r \mathbf{G}_{r}^{\mathrm{weak}} \gets\mathcal{F}_{\gamma,\boldsymbol{\mu}}^{-1}[\widehat{\mathbf{G}}_{r}] \in\mathbb{W}_{\xi}^{r} G r weak ← F γ , μ − 1 [ G r ] ∈ W ξ r . ▷ μ \boldsymbol{\mu} μ as in (53) .
return G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak .
For the above reasons, we propose a GMT–transformation–based MOR method, that can be applied to produce "norm–optimal" weak solutions.
Assumption 6.3. Let Ω ⊆ C \Omega\subseteq\mathbb{C} Ω ⊆ C such that there exists a bijection γ : Ω → D \gamma:\Omega\to\mathbb{D} γ : Ω → D . Let H \mathcal{H} H be an RKHS of functions Ω → C \Omega\to\mathbb{C} Ω → C with reproducing kernel ξ \xi ξ .
A general MOR method applicable to FOMs defined in RKHSs with non–smooth kernels is given in Algorithm 6.2. Here, we assume Assumption 6.3 .
that Theorem 4.3 can be applied. In this case, we can consider a GMT transformation F γ , λ [ G ] = G ^ ∈ H ^ F_{\gamma,\lambda}[\mathbf{G}] = \widehat{\mathbf{G}} \in\widehat{\mathcal{H}} F γ , λ [ G ] = G ∈ H and obtain a norm–optimal ROM G ^ r ∈ G r \widehat{\mathbf{G}}_r \in\mathcal{G}_r G r ∈ G r using Algorithm 6.1.
Note however, that in Algorithm 6.1, no formal guarantees are given that in each iteration, the ROM parameters λ ~ j ( m ) \widetilde{\lambda}_j^{(m)} λ j ( m ) (j = 0 , … , r − 1 j = 0,\ldots,r-1 j = 0 , … , r − 1 , m = 0 , 1 , … m = 0,1,\ldots m = 0 , 1 , … ) remain strictly inside Ω ^ \widehat{\Omega} Ω . In this regard, we note that if Ω ^ \widehat{\Omega} Ω is a Riemannian manifold, then the Newton step in Algorithm 6.1 can be modified to respect the geometry induced on it. However, such an approach would require the development of the Riemannian Newton step for each specific choice of Ω ^ \widehat{\Omega} Ω . If a bijection γ : Ω → D \gamma:\Omega\to\mathbb{D} γ : Ω → D is available, then using Algorithm 6.2 is much simpler from the perspective of applications.
In § 8 , we consider some MOR problems, where the functions in the original space H \mathcal{H} H are defined over a real interval, that is, Ω ⊆ R \Omega\subseteq\mathbb{R} Ω ⊆ R . In such cases, no bijection between Ω \Omega Ω and D \mathbb{D} D can exist. Therefore, we use the real–analytic space H 2 R \mathcal{H}_2^{\mathbb{R}} H 2 R as defined in (14) . Note that for this space, Algorithm 3.1 can be applied with the same theoretical guarantees as in H 2 ( D ) H_2(\mathbb{D}) H 2 ( D ) . Indeed, if G ^ \widehat{\mathbf{G}} G is real valued on I I I and the initial kernel centers satisfy γ ( λ ~ ( 0 ) ) ∈ I \gamma\left(\widetilde{\lambda}^{(0)}\right)\in\mathbb{I} γ ( λ ( 0 ) ) ∈ I , then the ROM produced by Algorithm 3.1 will satisfy G r ∈ H 2 R \mathbf{G}_r\in\mathcal{H}_2^{\mathbb{R}} G r ∈ H 2 R .
(d) The error E ξ r ( G r ) \mathcal{E}_{\xi}^{r}(\mathbf{G}_r) E ξ r ( G r ) can be computed by
E ξ r ( G r ) = ∑ k = r n + r − 1 ∣ ⟨ G ^ , γ ^ k γ ( μ ) ⟩ H 2 ( D ) ∣ 2 . \mathcal{E}_{\xi}^{r}(\mathbf{G}_r)=\sqrt{\sum_{k=r}^{n+r-1}\left|\left\langle\widehat{\mathbf{G}},\widehat{\gamma}_{k}^{\gamma(\boldsymbol{\mu})}\right\rangle_{H_2(\mathbb{D})}\right|^2}. E ξ r ( G r ) = k = r ∑ n + r − 1 ⟨ G , γ k γ ( μ ) ⟩ H 2 ( D ) 2 .
where γ ^ k γ ( μ ) \widehat{\gamma}_{k}^{\gamma(\boldsymbol{\mu})} γ k γ ( μ ) denote the classical MT functions from (38) defined by the parameter sequence
μ = ( λ 0 , … , λ n − 1 , λ ~ 0 , λ ~ 1 , … , λ ~ r − 1 ) ∈ Ω n + r , \boldsymbol{\mu}=(\lambda_0,\ldots,\lambda_{n-1},\widetilde{\lambda}_0,\widetilde{\lambda}_1,\ldots,\widetilde{\lambda}_{r-1})\in\Omega^{n+r}, μ = ( λ 0 , … , λ n − 1 , λ 0 , λ 1 , … , λ r − 1 ) ∈ Ω n + r ,
and where ( γ ( λ k ) ) k = 0 n − 1 \left(\gamma(\lambda_k)\right)_{k=0}^{n-1} ( γ ( λ k ) ) k = 0 n − 1 and ( γ ( λ ~ j ) ) j = 0 r − 1 \left(\gamma(\widetilde{\lambda}_j)\right)_{j=0}^{r-1} ( γ ( λ j ) ) j = 0 r − 1 are kernel centers associated with the FOM G ^ \widehat{\mathbf{G}} G and the ROM G ^ r \widehat{\mathbf{G}}_r G r , respectively. This is a consequence of Theorem 5.23 .
(e) For many applications, obtaining the weak solution as proposed in Algorithm 6.2, is computationally expensive. Hence, for practical applications we can use the efficient projected ROM as defined in Theorem 5.27 . That is, after retrieving the ROM kernel centers γ ( λ ~ ) \gamma(\widetilde{\lambda}) γ ( λ ) in the companion space via Algorithm 3.1, we can determine a ROM G r ∈ G r \mathbf{G}_r\in\mathcal{G}_r G r ∈ G r by G r e c o n : = P V r [ G ] \mathbf{G}_r^{\mathrm{econ}}:=P_{\mathcal{V}_r}[\mathbf{G}] G r econ := P V r [ G ] . Theorem 5.27 gives a theoretical error bound on this ROM when compared to the weak solution, while our experiments in § 8 empirically demonstrate its effectiveness.
Experiments
We consider a number of different experiments to show applications of the norm–optimal RKHS–MOR methods discussed in § 7 . The experiments purposely consider a diverse array of RKHSs in order to assert the generality of the introduced methods. We emphasize that these experiments do not encompass the full spectrum of possible applications, nor do they aim to present market–ready solutions. Instead, our objective is to present several applications where norm–optimal MOR is (to the best of our knowledge) currently not possible. The four experiments below are not equally mature: the polynomial fitting example in § 8.1 is an illustrative academic exercise, the SVM and ReLU network experiments in § 9 and § 9.1 are fully developed case studies, and the PDE discretization example in § 9.2 is a proof of concept for a new application of the framework rather than a finished method. The presented experiments are implemented in MATLAB and Python and are fully reproducible (see code availability statement at the end of the manuscript). The presented quantitative results are obtained on a MacBook Pro with 32 GB of RAM and Apple M4 CPU, using MATLAB version R2025a and Python 3.9.6 with PyTorch 2.8.0.
Polynomial fitting
We begin with a simple application involving fitting polynomials to discrete data points. In particular, consider the n n n –dimensional RKHS P n ( Ω ) \mathbb{P}_n(\Omega) P n ( Ω ) containing the real–coefficient polynomials of degree at most n − 1 n-1 n − 1 defined on the real interval Ω : = ( − 1 , 1 ) ⊆ R \Omega:= (-1,1) \subseteq\mathbb{R} Ω := ( − 1 , 1 ) ⊆ R equipped with the inner product
⟨ G , F ⟩ P n ( Ω ) : = ∑ k = 0 n − 1 g k f k , where G ( x ) : = ∑ k = 0 n − 1 g k x k , F ( x ) : = ∑ k = 0 n − 1 f k x k . \langle\mathbf{G},\mathbf{F}\rangle_{\mathbb{P}_n(\Omega)} := \sum_{k=0}^{n-1} g_k f_k,\qquad\text{where }\mathbf{G}(x) := \sum_{k=0}^{n-1}g_kx^k,\ \mathbf{F}(x) := \sum_{k=0}^{n-1}f_kx^k. ⟨ G , F ⟩ P n ( Ω ) := k = 0 ∑ n − 1 g k f k , where G ( x ) := k = 0 ∑ n − 1 g k x k , F ( x ) := k = 0 ∑ n − 1 f k x k .
This is an RKHS with reproducing kernel
ξ ( x , y ) = ∑ k = 0 n − 1 x k y k = 1 − ( x y ) n 1 − x y ( x , y ∈ Ω ) . (70) \xi(x,y) = \sum_{k=0}^{n-1}x^k y^k = \frac{1-(xy)^n}{1-xy}\qquad(x,y\in\Omega). \tag*{(70)} ξ ( x , y ) = k = 0 ∑ n − 1 x k y k = 1 − x y 1 − ( x y ) n ( x , y ∈ Ω ) . (70)
Notice that the monomials 1 , x , x 2 , … , x n − 1 1,x,x^2,\ldots,x^{n-1} 1 , x , x 2 , … , x n − 1 form an orthonormal basis in P n ( Ω ) \mathbb{P}_n(\Omega) P n ( Ω ) . Indeed, let ϕ k ( x ) : = x k \phi_k(x):=x^k ϕ k ( x ) := x k ( k = 0 , … , n − 1 ) (k=0,\ldots,n-1) ( k = 0 , … , n − 1 ) and observe that by the reproducing property we have
⟨ ϕ j , ξ ( ⋅ , λ ) ⟩ P n ( Ω ) = λ j ( j = 0 , … , n − 1 , λ ∈ Ω ) . \left\langle\phi_j,\xi(\cdot,\lambda)\right\rangle_{\mathbb{P}_n(\Omega)}=\lambda^j\qquad(j=0,\ldots,n-1,\lambda\in\Omega). ⟨ ϕ j , ξ ( ⋅ , λ ) ⟩ P n ( Ω ) = λ j ( j = 0 , … , n − 1 , λ ∈ Ω ) .
Indeed, by exploiting the bilinearity of the inner product and (70) , this gives
⟨ ϕ j , ξ ( ⋅ , λ ) ⟩ P n ( Ω ) = ⟨ ϕ j , ∑ k = 0 n − 1 ϕ k ( ⋅ ) ϕ k ( λ ) ⟩ P n ( Ω ) = ∑ k = 0 n − 1 ϕ k ( λ ) ⟨ ϕ j , ϕ k ⟩ P n ( Ω ) = λ j . \left\langle\phi_j,\xi(\cdot,\lambda)\right\rangle_{\mathbb{P}_n(\Omega)}
=\left\langle\phi_j,\sum_{k=0}^{n-1}\phi_k(\cdot)\phi_k(\lambda)\right\rangle_{\mathbb{P}_n(\Omega)}
=\sum_{k=0}^{n-1}\phi_k(\lambda)\left\langle\phi_j,\phi_k\right\rangle_{\mathbb{P}_n(\Omega)}
=\lambda^j. ⟨ ϕ j , ξ ( ⋅ , λ ) ⟩ P n ( Ω ) = ⟨ ϕ j , k = 0 ∑ n − 1 ϕ k ( ⋅ ) ϕ k ( λ ) ⟩ P n ( Ω ) = k = 0 ∑ n − 1 ϕ k ( λ ) ⟨ ϕ j , ϕ k ⟩ P n ( Ω ) = λ j .
In the last equation, the expressions on both sides are polynomials in λ \lambda λ . Hence, they can only be equal if their algebraic coefficients match. Remembering that ϕ k ( x ) = x k \phi_k(x)=x^k ϕ k ( x ) = x k ( x ∈ Ω ) (x\in\Omega) ( x ∈ Ω ) , this gives ⟨ ϕ j , ϕ k ⟩ P n ( Ω ) = δ j , k \left\langle\phi_j,\phi_k\right\rangle_{\mathbb{P}_n(\Omega)}=\delta_{j,k} ⟨ ϕ j , ϕ k ⟩ P n ( Ω ) = δ j , k . Let { x j } j = 0 n − 1 ⊆ ( − 1 , 1 ) \{x_j\}_{j=0}^{n-1}\subseteq(-1,1) { x j } j = 0 n − 1 ⊆ ( − 1 , 1 ) be a grid of pairwise distinct points and let data D : = { ( x 0 , y 0 ) , … , ( x n − 1 , y n − 1 ) } ⊆ Ω × R \mathcal{D}:=\{(x_0,y_0),\ldots,(x_{n-1},y_{n-1})\}\subseteq\Omega\times\mathbb{R} D := {( x 0 , y 0 ) , … , ( x n − 1 , y n − 1 )} ⊆ Ω × R be given. Recall that there exists a unique polynomial G \mathbf{G} G (of degree at most n − 1 n-1 n − 1 ) such that
G ( x m ) = y m ( m = 0 , … , n − 1 ) , \mathbf{G}(x_m)=y_m\qquad(m=0,\ldots,n-1), G ( x m ) = y m ( m = 0 , … , n − 1 ) ,
which, in our setting, plays the role of the FOM. Therefore, for a polynomial G ∈ P n ( Ω ) G \in\mathcal{P}_n(\Omega) G ∈ P n ( Ω ) , there exist kernel centers λ 0 , … , λ n − 1 ∈ ( − 1 , 1 ) \lambda_0,\ldots,\lambda_{n-1} \in(-1,1) λ 0 , … , λ n − 1 ∈ ( − 1 , 1 ) and linear coefficients r 0 , … , r n − 1 ∈ R r_0,\ldots,r_{n-1} \in\mathbb{R} r 0 , … , r n − 1 ∈ R such that
G ( x m ) = ∑ k = 0 n − 1 r k ξ λ k ( x m ) = y m ( m = 0 , … , n − 1 ) . G(x_m)=\sum_{k=0}^{n-1}r_k\xi_{\lambda_k}(x_m)=y_m \quad(m=0,\ldots,n-1). G ( x m ) = k = 0 ∑ n − 1 r k ξ λ k ( x m ) = y m ( m = 0 , … , n − 1 ) .
Now consider the problem of fitting an order–r r r polynomial model to the dataset D \mathcal{D} D . The order of the polynomial model to be fitted is defined according to Definition 2.4 and should not be confused with the degree of the polynomial, which is understood as the highest exponent appearing in the algebraic form. We can pose the problem of fitting an order–r r r polynomial
G r = ∑ j = 0 r − 1 r ~ j ξ λ ~ j ∈ G ξ r ⊆ P n ( Ω ) G_r=\sum_{j=0}^{r-1}\widetilde{r}_j\xi_{\widetilde{\lambda}_j}\in\mathcal{G}_{\xi}^{r}\subseteq\mathcal{P}_n(\Omega) G r = j = 0 ∑ r − 1 r j ξ λ j ∈ G ξ r ⊆ P n ( Ω )
to D \mathcal{D} D as a P n ( Ω ) \mathcal{P}_n(\Omega) P n ( Ω ) –norm–optimal MOR task. The following lemma clarifies why this is appropriate.
Lemma 7.1 (Error of P n ( Ω ) \mathcal{P}_n(\Omega) P n ( Ω ) –norm–optimal polynomial fit). Let Ω = ( − 1 , 1 ) \Omega=(-1,1) Ω = ( − 1 , 1 ) and G ∈ G ξ n ⊆ P n ( Ω ) G\in\mathcal{G}_{\xi}^{n}\subseteq\mathcal{P}_n(\Omega) G ∈ G ξ n ⊆ P n ( Ω ) . Then, for any x ∈ Ω x\in\Omega x ∈ Ω , r < n r<n r < n , and G r ∈ G ξ r G_r\in\mathcal{G}_{\xi}^{r} G r ∈ G ξ r , we have
∣ G ( x ) − G r ( x ) ∣ ≤ n ∥ G − G r ∥ P n ( Ω ) . \lvert G(x)-G_r(x)\rvert\leq\sqrt{n}\lVert G-G_r\rVert_{\mathcal{P}_n(\Omega)}. ∣ G ( x ) − G r ( x )∣ ≤ n ∥ G − G r ∥ P n ( Ω ) . Proof. The proof of this lemma is given in Appendix § 11 .§ 3 .
Note that the kernel ξ \xi ξ in (70) is infinitely many times differentiable. The smoothness of ξ \xi ξ allows us to perform MOR using Algorithm 6.1 without having to transform the problem to another RKHS using the GMT method. We denote the ROM obtained with Algorithm 6.1 by G r d i r G_r^{\mathrm{dir}} G r dir . In order to compare their effectiveness, we also use Algorithm 6.2 to obtain a ROM G r w e a k G_r^{\mathrm{weak}} G r weak . In this experiment, we consider the bijection γ : Ω → Ω ^ : = ( − 1 , 1 ) \gamma:\Omega\to\widehat{\Omega}:=(-1,1) γ : Ω → Ω := ( − 1 , 1 ) defined as γ ( x ) = x \gamma(x)=x γ ( x ) = x (x ∈ Ω x\in\Omega x ∈ Ω ) and choose the companion RKHS H 2 I H_2^{\mathsf{I}} H 2 I from (14) . We recall the discussion below Algorithm 3.1, which clarifies that Algorithm 3.1 is closed in the space H 2 I H_2^{\mathsf{I}} H 2 I . Since our GMT–transformation–based MOR scheme in Algorithm 6.2 relies on Algorithm 3.1 to perform the MOR in the companion space, this practically means that if we initialize ROM parameters in ( − 1 , 1 ) (-1,1) ( − 1 , 1 ) , then we obtain λ ~ j ∈ ( − 1 , 1 ) \widetilde{\lambda}_j\in(-1,1) λ j ∈ ( − 1 , 1 ) for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 with Algorithm 6.2.
Note that Algorithm 6.2 generally guarantees that G r w e a k ∈ W ξ r G_r^{\mathrm{weak}}\in\mathcal{W}_{\xi}^{r} G r weak ∈ W ξ r . We can obtain a strong ROM G r e c o n = P V λ ~ r [ G ] G_r^{\mathrm{econ}}=\mathcal{P}_{V_{\widetilde{\lambda}}^{r}}[G] G r econ = P V λ r [ G ] , where we assume that λ ~ ∈ Ω ξ r \widetilde{\lambda}\in\Omega_{\xi}^{r} λ ∈ Ω ξ r denote the kernel centers that describe G r w e a k G_r^{\mathrm{weak}} G r weak returned by Algorithm 6.2. A formal upper bound for ∥ G r w e a k − G r e c o n ∥ P n ( Ω ) \lVert G_r^{\mathrm{weak}}-G_r^{\mathrm{econ}}\rVert_{\mathcal{P}_n(\Omega)} ∥ G r weak − G r econ ∥ P n ( Ω ) is given in Theorem 5.27 . In this experiment, we empirically confirm that both G r w e a k G_r^{\mathrm{weak}} G r weak and G r e c o n G_r^{\mathrm{econ}} G r econ provide very high quality ROMs and can be used effectively to solve the polynomial fitting task described above.
To evaluate the error of the different approximations, given a dataset D \mathcal{D} D , we consider the mean squared error (MSE)
E M S E D ( G r ) : = 1 n ∑ m = 0 n − 1 ( G r ( x m ) − y m ) 2 ( G r ∈ P n ( Ω ) ) . (71) \mathcal{E}_{\mathrm{MSE}}^{\mathcal{D}}(G_r):=\frac{1}{n}\sum_{m=0}^{n-1}(G_r(x_m)-y_m)^2 \quad(G_r\in\mathcal{P}_n(\Omega)). \tag*{(71)} E MSE D ( G r ) := n 1 m = 0 ∑ n − 1 ( G r ( x m ) − y m ) 2 ( G r ∈ P n ( Ω )) . (71)
The best degree –( r − 1 ) (r - 1) ( r − 1 ) least–squares fit G r L S Q G_r^{\mathrm{LSQ}} G r LSQ is also evaluated as a baseline model. This model satisfies
G r L S Q ( x ) = ∑ k = 0 r − 1 p k x k ( x ∈ Ω ) , (72) G_r^{\mathrm{LSQ}}(x) = \sum_{k=0}^{r-1} p_k x^k \quad(x \in\Omega), \tag*{(72)} G r LSQ ( x ) = k = 0 ∑ r − 1 p k x k ( x ∈ Ω ) , (72)
where the coefficients are given by p = [ p 0 , … , p r − 1 ] T = X + y \mathbf{p} = [p_0, \ldots, p_{r-1}]^{\mathsf{T}} = \mathbf{X}^{+}\mathbf{y} p = [ p 0 , … , p r − 1 ] T = X + y . Here, X + ∈ R r × n \mathbf{X}^{+} \in\mathbb{R}^{r \times n} X + ∈ R r × n denotes the Moore–Penrose pseudoinverse of
X = [ 1 x 0 ⋯ x 0 r − 1 1 x 1 ⋯ x 1 r − 1 ⋮ ⋮ ⋱ ⋮ 1 x n − 1 ⋯ x n − 1 r − 1 ] ∈ R n × r \mathbf{X} =
\begin{bmatrix}
1 & x_0 & \cdots& x_0^{r-1} \\
1 & x_1 & \cdots& x_1^{r-1} \\
\vdots& \vdots& \ddots& \vdots\\
1 & x_{n-1} & \cdots& x_{n-1}^{r-1}
\end{bmatrix}
\in\mathbb{R}^{n \times r} X = 1 1 ⋮ 1 x 0 x 1 ⋮ x n − 1 ⋯ ⋯ ⋱ ⋯ x 0 r − 1 x 1 r − 1 ⋮ x n − 1 r − 1 ∈ R n × r
and y = [ y 0 , … , y n − 1 ] T ∈ R n \mathbf{y} = [y_0, \ldots, y_{n-1}]^{\mathsf{T}} \in\mathbb{R}^{n} y = [ y 0 , … , y n − 1 ] T ∈ R n . Note that G r L S Q G_r^{\mathrm{LSQ}} G r LSQ , G r d i r G_r^{\mathrm{dir}} G r dir , G r w e a k G_r^{\mathrm{weak}} G r weak , and G r e c o n G_r^{\mathrm{econ}} G r econ all live in r r r –dimensional subspaces of P n ( Ω ) \mathcal{P}_n(\Omega) P n ( Ω ) . In order to evaluate the proposed models, however, we need to store at least 2 r 2r 2 r parameters (r r r kernel centers and r r r coefficients). In contrast, G r L S Q G_r^{\mathrm{LSQ}} G r LSQ can be fully evaluated given the r r r coefficients from (72) . To ensure a fair comparison, we also consider the least squares fit model G 2 r L S Q G_{2r}^{\mathrm{LSQ}} G 2 r LSQ , which requires the same number (2 r 2r 2 r ) of parameters for reconstruction but exists in a 2 r 2r 2 r –dimensional subspace of P n ( Ω ) \mathcal{P}_n(\Omega) P n ( Ω ) . Figure 3 illustrates the resulting polynomials using r = 4 r = 4 r = 4 . Figure 4 shows the point–wise absolute error ∣ G r ( x k ) − y k ∣ 2 \lvert G_r(x_k) - y_k \rvert^2 ∣ G r ( x k ) − y k ∣ 2 (k = 0 , … , n − 1 k = 0, \ldots, n - 1 k = 0 , … , n − 1 ) of each model on a logarithmic scale. Finally, Table 2 shows the reconstruction errors (in terms of (71) ) for different choices of r r r and n = 100 n = 100 n = 100 . In these experiments, we choose the points x k x_k x k equidistantly in [ − 0.99 , 0.99 ] [-0.99, 0.99] [ − 0.99 , 0.99 ] . The corresponding values y k y_k y k are randomly sampled from a uniform random distribution.
Figure 3. Polynomial fits obtained with the different approximation strategies for r = 4 r = 4 r = 4 and n = 100 n = 100 n = 100 .
Figure 4. Pointwise square fitting errors obtained with the different approximation strategies for r = 4 r = 4 r = 4 and n = 100 n = 100 n = 100 .
Table 2. MSE and RKHS errors for the polynomial fitting application (n = 100 n = 100 n = 100 ). The best ROMs per reduced order are indicated by boldface font.
The results in Table 2 verify the practical benefits of the proposed MOR based method for polynomial fitting. Indeed, the proposed ROMs G r d i r G_r^{\mathrm{dir}} G r dir , G r w e a k G_r^{\mathrm{weak}} G r weak , and G r e c o n G_r^{\mathrm{econ}} G r econ consistently provide the best MSE scores regardless of the reduced order r r r . In fact, for r = 3 , 4 , 5 r = 3, 4, 5 r = 3 , 4 , 5 , even the worst performing proposed model provides ≥ 3 \ge3 ≥ 3 times reduced MSE compared to G 2 r L S Q \mathbf{G}^{\mathrm{LSQ}}_{2r} G 2 r LSQ . The only exception is r = 2 r = 2 r = 2 , where Algorithm 6.1 failed to produce a good local optimum. We note that even in this case, G r w e a k \mathbf{G}^{\mathrm{weak}}_{r} G r weak and G r e c o n \mathbf{G}^{\mathrm{econ}}_{r} G r econ outperform the model G 2 r L S Q \mathbf{G}^{\mathrm{LSQ}}_{2r} G 2 r LSQ by a factor of at least 2.15 in terms of MSE. Compared to G r L S Q \mathbf{G}^{\mathrm{LSQ}}_{r} G r LSQ , the error reduction factor climbs to at least 5.97. We note that even in this worst–case scenario, the MSE obtained by G r d i r \mathbf{G}^{\mathrm{dir}}_{r} G r dir is comparable to the model G r L S Q \mathbf{G}^{\mathrm{LSQ}}_{r} G r LSQ .
The above experiments provide insight into several properties of MOR–based polynomial fitting. First, we observe that even though it solves the problem in P n ( Ω ) \mathcal{P}_{n}(\Omega) P n ( Ω ) , Algorithm 6.1 is consistently outperformed by the GMT–transformation–based models obtained using Algorithm 6.2. This can be explained by the fact that Algorithm 6.1 does not exploit the underlying geometry of Ω \Omega Ω . In contrast, G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak and G r e c o n \mathbf{G}_{r}^{\mathrm{econ}} G r econ are obtained by first transforming the problem into H 1 2 H_{1}^{2} H 1 2 and then using a geometry–aware Newton’s method (see Algorithm 3.1).
Another interesting observation is that G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak and G r e c o n \mathbf{G}_{r}^{\mathrm{econ}} G r econ provide similar results. While the weak ROM G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak achieves better overall scores both in terms of MSE and RKHS error, G r e c o n \mathbf{G}_{r}^{\mathrm{econ}} G r econ also provides high quality approximations of the data and consistently outperforms every model other than G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak . This is especially useful if the application scenario does not allow us to store the 2 r + n 2r+n 2 r + n parameters needed to evaluate the weak ROM (see § 6.3 ). Hence, while it provides the best overall approximation of the data, using the weak model G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak is not practical. Especially in the current experiment, we can express G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak using n n n algebraic coefficients due to (70) . While this is less than the theoretically required n + 2 r n+2r n + 2 r parameters for weak model reconstruction discussed in § 6.3 , it matches the number of coefficients needed to reconstruct the interpolating polynomial G G G . We also note that according to Table 2 , the ROMs G r d i r \mathbf{G}_{r}^{\mathrm{dir}} G r dir , G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak , and G r e c o n \mathbf{G}_{r}^{\mathrm{econ}} G r econ significantly outperform G r L S Q \mathbf{G}_{r}^{\mathrm{LSQ}} G r LSQ and G 2 r L S Q \mathbf{G}_{2r}^{\mathrm{LSQ}} G 2 r LSQ in terms of the RKHS error E ξ r E_{\xi}^{r} E ξ r . This is expected, as the latter models do not attempt to minimize E ξ r E_{\xi}^{r} E ξ r . We recall that aiming for a small RKHS–norm–based error is beneficial for the data fitting problem, because Lemma 7.1 provides a computable upper bound for the pointwise error.
Finally, we consider the runtime of each algorithm in Table 3 . We observe that the proposed models G r w e a k \mathbf{G}_{r}^{\mathrm{weak}} G r weak and G r e c o n \mathbf{G}_{r}^{\mathrm{econ}} G r econ obtained using Algorithm 6.2 can be computed in less than 1 second for every experiment. In contrast, computing G r d i r \mathbf{G}_{r}^{\mathrm{dir}} G r dir with Algorithm 6.1 takes 6.34 seconds on average. We note that this increase is likely not due to theoretical reasons, rather, it is a consequence of our chosen implementation. Indeed, in order to make Algorithm 6.1 as easy to use as possible, we exploit the automatic differentiation feature of PyTorch to compute the derivatives appearing in the iteration. In particular, our MATLAB–based main script calls a PyTorch routine every time we need to compute the Jacobian associated with the equilibrium function defined in (34) . This significantly slows execution, but allows for a plug and play experience for the users, who can easily try Algorithm 6.1 in a sufficiently smooth RKHS of their choice. Finally, we note that in the case of the GMT–based models, we use Algorithm 3.1 to obtain the ROMs in the companion RKHS H 1 2 H_{1}^{2} H 1 2 . For this algorithm, the necessary derivatives can be efficiently computed, and no automatic differentiation or transition from MATLAB to PyTorch is required. This explains the improved execution times.
Table 3. Recorded runtimes (seconds) of the considered MOR schemes. G r L S Q G_r^{\mathrm{LSQ}} G r LSQ and G 2 r L S Q G_{2r}^{\mathrm{LSQ}} G 2 r LSQ are obtained with MATLAB’s polyfit routine.
MOR on support vector machines
Next, we consider MOR for SVMs with RBF kernels. The experiment uses the brake disc deformation data from [25 ] , where brake fluid pressure and acceleration signals are considered to detect the presence of brake disc runout in passenger vehicles. This fault detection task is posed as a binary classification problem where measurement segments are classified as either healthy or deformed. The preprocessed segments correspond to active braking, with the vehicle speed between 90 and 30 km/h. We use the empirical standard deviation features introduced in [24 ] to reduce the dimensionality of the data entries. In the present experiment, only two of the four sensor–based features are retained. We use distinct training and test datasets, denoted by D t r \mathcal{D}_{\mathrm{tr}} D tr and D t e \mathcal{D}_{\mathrm{te}} D te , respectively. The training data is used to construct the FOM, whereas the test data is reserved for the subsequent evaluation of the FOM and its reduced–order approximations. More precisely, for pairwise different points x 0 , … , x N s − 1 x_0,\ldots,x_{N_s-1} x 0 , … , x N s − 1 , s ∈ { t r , t e } s \in\{\mathrm{tr},\mathrm{te}\} s ∈ { tr , te } , we write
D s : = { ( x 0 ( s ) , y 0 ( s ) ) , … , ( x N s − 1 ( s ) , y N s − 1 ( s ) ) } ⊆ R 2 × { − 1 , 1 } , s ∈ { t r , t e } . \mathcal{D}_s := \left\{\left(x_0^{(s)},y_0^{(s)}\right),\ldots,\left(x_{N_s-1}^{(s)},y_{N_s-1}^{(s)}\right)\right\} \subseteq\mathbb{R}^2 \times\{-1,1\}, \qquad s \in\{\mathrm{tr},\mathrm{te}\}. D s := { ( x 0 ( s ) , y 0 ( s ) ) , … , ( x N s − 1 ( s ) , y N s − 1 ( s ) ) } ⊆ R 2 × { − 1 , 1 } , s ∈ { tr , te } .
The classification procedure yields N t r = 5841 N_{\mathrm{tr}} = 5841 N tr = 5841 training samples and N t e = 649 N_{\mathrm{te}} = 649 N te = 649 test samples, with both datasets balanced between the deformed and healthy classes.
Here, x m , 1 ( s ) x_{m,1}^{(s)} x m , 1 ( s ) and x m , 2 ( s ) x_{m,2}^{(s)} x m , 2 ( s ) denote, respectively, the empirical standard deviations of the acceleration signal measured by an accelerometer attached to the vehicle chassis and of the brake fluid pressure signal measured by a pressure sensor.
Each feature vector x m ( s ) x_m^{(s)} x m ( s ) is identified with the complex–valued input z m ( s ) : = x m , 1 ( s ) + i x m , 2 ( s ) ∈ Ω : = D z_m^{(s)} := x_{m,1}^{(s)} + \mathrm{i}x_{m,2}^{(s)} \in\Omega:= \mathbb{D} z m ( s ) := x m , 1 ( s ) + i x m , 2 ( s ) ∈ Ω := D . Accordingly, we denote the corresponding complex–valued inputs by
Z s : = ( z 0 ( s ) , … , z N s − 1 ( s ) ) ∈ Ω N s , s ∈ { t r , t e } . \mathcal{Z}_s := \left(z_0^{(s)},\ldots,z_{N_s-1}^{(s)}\right) \in\Omega^{N_s}, \qquad s \in\{\mathrm{tr},\mathrm{te}\}. Z s := ( z 0 ( s ) , … , z N s − 1 ( s ) ) ∈ Ω N s , s ∈ { tr , te } .
The inclusion in D \mathbb{D} D is obtained by applying the same normalization to the training and test inputs. This entails no restriction for the present RBF–SVM formulation, since a global rescaling of the input coordinates can be compensated by a corresponding adjustment of the RBF kernel parameter. The kernel parameter is therefore selected in the normalized coordinates. In particular, the support vector locations λ k \lambda_k λ k , being selected from the training inputs, also belong to Ω = D \Omega= \mathbb{D} Ω = D .
Let the trained RBF–SVM be denoted by
D ( z ) = G ( z ) + b , G ( z ) = ∑ k = 0 n − 1 r k ξ ( z , λ k ) , ξ ( z , λ ) = exp ( − σ ∣ z − λ ∣ 2 ) , (73) \mathbf{D}(z) = G(z) + b,\qquad G(z) = \sum_{k=0}^{n-1} \mathfrak{r}_k \xi(z,\lambda_k),\qquad\xi(z,\lambda) = \exp\left(-\sigma\lvert z-\lambda\rvert^2\right), \tag*{(73)} D ( z ) = G ( z ) + b , G ( z ) = k = 0 ∑ n − 1 r k ξ ( z , λ k ) , ξ ( z , λ ) = exp ( − σ ∣ z − λ ∣ 2 ) , (73)
where σ > 0 \sigma> 0 σ > 0 is the RBF kernel parameter, λ k ∈ Ω \lambda_k \in\Omega λ k ∈ Ω denote the kernel centers, r k \mathfrak{r}_k r k are the signed dual coefficients, and b ∈ R b \in\mathbb{R} b ∈ R is the bias term. Thus, the nonconstant part of the SVM decision function is an element G ∈ G ξ n ⊆ H G \in\mathcal{G}_{\xi}^{n} \subseteq\mathcal{H} G ∈ G ξ n ⊆ H of the RKHS generated by the RBF kernel ξ \xi ξ . The order n n n of the FOM is equal to the number of support vectors. For the present experiment, the trained SVM contains n = N S V = 5841 n = N_{\mathrm{SV}} = 5841 n = N SV = 5841 support vectors. The same bias term b b b is used for the FOM and all ROMs. We therefore apply the reduction to the nonconstant component G \mathbf{G} G , seeking an approximation G r ∈ G ξ r \mathbf{G}_r \in\mathcal{G}_{\xi}^{r} G r ∈ G ξ r . The corresponding reduced-order decision function is D r : = G r + b \mathbf{D}_r := \mathbf{G}_r + b D r := G r + b .
We compare two reduction strategies already introduced in § 8.1 . The direct approach computes a strong ROM G r d i r ∈ G ξ r \mathbf{G}_r^{\mathrm{dir}} \in\mathcal{G}_{\xi}^{r} G r dir ∈ G ξ r directly in the original RBF–RKHS H \mathcal{H} H . The GMT–based approach instead yields the projected strong ROM of the form G r e c o n = P V λ ~ [ G ] \mathbf{G}_r^{\mathrm{econ}} = \mathcal{P}_{\mathcal{V}_{\tilde{\boldsymbol{\lambda}}}}[\mathbf{G}] G r econ = P V λ ~ [ G ] for some kernel centers λ ~ ∈ Ω r \tilde{\boldsymbol{\lambda}} \in\Omega^r λ ~ ∈ Ω r . We do not report the weak GMT–ROM in this experiment, since our focus is on reduced models that also substantially reduce inference costs. Indeed, the weak representation retains an n n n –dependent contribution, whereas the projected ROM depends only on the reduced order r r r .
The map γ : Ω → Ω ^ \gamma: \Omega\to\widehat{\Omega} γ : Ω → Ω used for the GMT transformation is set to γ ( z ) : = z \gamma(z) := z γ ( z ) := z . This choice of γ \gamma γ is valid because, with the preprocessing steps described above, the kernel centers of the FOM belong to D \mathbb{D} D .
For any z ∈ Ω z \in\Omega z ∈ Ω , the predicted label of the ROM is given by sgn ( D r ( z ) ) \operatorname{sgn}(\mathbf{D}_r(z)) sgn ( D r ( z )) , using the same convention as for the FOM. Classification accuracy is evaluated separately on the training and test datasets. In addition, for s ∈ { t r , t e } s \in\{\mathrm{tr}, \mathrm{te}\} s ∈ { tr , te } , we report the pointwise decision function error
E p t ( D r ; Z s ) : = max z m ( s ) ∈ Z s ∣ D ( z m ( s ) ) − D r ( z m ( s ) ) ∣ = max z m ( s ) ∈ Z s ∣ G ( z m ( s ) ) − G r ( z m ( s ) ) ∣ . \mathcal{E}_{\mathrm{pt}}(\mathbf{D}_r;\mathcal{Z}_s) := \max_{z_m^{(s)} \in\mathcal{Z}_s} \left|\mathbf{D}(z_m^{(s)})-\mathbf{D}_r(z_m^{(s)})\right| = \max_{z_m^{(s)} \in\mathcal{Z}_s} \left|\mathbf{G}(z_m^{(s)})-\mathbf{G}_r(z_m^{(s)})\right|. E pt ( D r ; Z s ) := z m ( s ) ∈ Z s max D ( z m ( s ) ) − D r ( z m ( s ) ) = z m ( s ) ∈ Z s max G ( z m ( s ) ) − G r ( z m ( s ) ) .
In addition to the dataset–dependent pointwise error, we consider the RKHS approximation error E ξ r \mathcal{E}_{\xi}^{r} E ξ r from (1) . Unlike E p t \mathcal{E}_{\mathrm{pt}} E pt , this quantity measures the FOM–ROM approximation over the underlying RKHS and is independent of the finite training and test datasets.
Because the FOM and ROM share the same bias, we also write
E ξ r ( D r ) : = ∥ D − D r ∥ H = ∥ G − G r ∥ H = E ξ r ( G r ) , \mathcal{E}_{\xi}^{r}(\mathbf{D}_r) := \lVert\mathbf{D}-\mathbf{D}_r\rVert_{\mathcal{H}} = \lVert\mathbf{G}-\mathbf{G}_r\rVert_{\mathcal{H}} = \mathcal{E}_{\xi}^{r}(\mathbf{G}_r), E ξ r ( D r ) := ∥ D − D r ∥ H = ∥ G − G r ∥ H = E ξ r ( G r ) ,
where the RKHS norm is well defined because D − D r = G − G r ∈ H \mathbf{D}-\mathbf{D}_r = \mathbf{G}-\mathbf{G}_r \in\mathcal{H} D − D r = G − G r ∈ H . For reporting purposes, we additionally consider the relative RKHS error
E ξ , r e l r ( G r ) : = E ξ r ( G r ) ∥ G ∥ H = ∥ G − G r ∥ H ∥ G ∥ H . (74) \mathcal{E}_{\xi,\mathrm{rel}}^{r}(\mathbf{G}_r) := \frac{\mathcal{E}_{\xi}^{r}(\mathbf{G}_r)}{\lVert\mathbf{G}\rVert_{\mathcal{H}}} = \frac{\lVert\mathbf{G}-\mathbf{G}_r\rVert_{\mathcal{H}}}{\lVert\mathbf{G}\rVert_{\mathcal{H}}}. \tag*{(74)} E ξ , rel r ( G r ) := ∥ G ∥ H E ξ r ( G r ) = ∥ G ∥ H ∥ G − G r ∥ H . (74)
The relative RKHS error provides a scale–independent measure of the FOM–ROM approximation quality and, like the absolute RKHS error, is independent of the finite training and test datasets.
The main objective of this experiment is to approximate the FOM decision function with fewer stored parameters while preserving its classification behavior. This is consistent with the RKHS approximation objective. Indeed, since ξ ( z , z ) = 1 \xi(z,z)=1 ξ ( z , z ) = 1 for the RBF kernel, the reproducing property and the Cauchy–Schwarz inequality give
∣ D ( z ) − D r ( z ) ∣ = ∣ G ( z ) − G r ( z ) ∣ ≤ ∥ G − G r ∥ H = E ξ r ( D r ) ( z ∈ Ω ) . (75) \left|\mathbf{D}(z)-\mathbf{D}_r(z)\right| = \left|\mathbf{G}(z)-\mathbf{G}_r(z)\right| \leq\lVert\mathbf{G}-\mathbf{G}_r\rVert_{\mathcal{H}} = \mathcal{E}_{\xi}^{r}(\mathbf{D}_r) \quad(z \in\Omega). \tag*{(75)} ∣ D ( z ) − D r ( z ) ∣ = ∣ G ( z ) − G r ( z ) ∣ ≤ ∥ G − G r ∥ H = E ξ r ( D r ) ( z ∈ Ω ) . (75)
Consequently, a ROM close to FOM in RKHS norm implies similar decision functions over Ω \Omega Ω . Whether this approximation yields the same classification at a given point depends on its spatial position relative to the decision boundary.
Lemma 7.2 (Simple pointwise classification certificate). Let D \mathbf{D} D and D r \mathbf{D}_r D r be the FOM and ROM decision functions (73) , respectively, and let z ∈ Ω \mathbf{z} \in\Omega z ∈ Ω . If
∣ D r ( z ) ∣ > ∣ D ( z ) − D r ( z ) ∣ , (76) \lvert\mathbf{D}_r(\mathbf{z}) \rvert> \lvert\mathbf{D}(\mathbf{z}) - \mathbf{D}_r(\mathbf{z}) \rvert, \tag*{(76)} ∣ D r ( z )∣ > ∣ D ( z ) − D r ( z )∣ , (76) then
sgn ( D ( z ) ) = sgn ( D r ( z ) ) . \operatorname{sgn}(\mathbf{D}(\mathbf{z})) = \operatorname{sgn}(\mathbf{D}_r(\mathbf{z})). sgn ( D ( z )) = sgn ( D r ( z )) . Hence, FOM and ROM assign the same label to z \mathbf{z} z .
Proof. For the proof, we refer to Section A.4.
Corollary 7.3 (Uniform classification certificate). Let D \mathbf{D} D and D r \mathbf{D}_r D r be the FOM and ROM decision functions, respectively, and let z ∈ Ω \mathbf{z} \in\Omega z ∈ Ω . If
∣ D r ( z ) ∣ > E ξ r ( D r ) , (77) \lvert\mathbf{D}_r(\mathbf{z}) \rvert> \mathcal{E}_{\xi}^{r}(\mathbf{D}_r), \tag*{(77)} ∣ D r ( z )∣ > E ξ r ( D r ) , (77) then
sgn ( D ( z ) ) = sgn ( D r ( z ) ) . \operatorname{sgn}(\mathbf{D}(\mathbf{z})) = \operatorname{sgn}(\mathbf{D}_r(\mathbf{z})). sgn ( D ( z )) = sgn ( D r ( z )) . Hence, FOM and ROM assign the same label to z \mathbf{z} z .
Proof. By the uniform RKHS error bound (75) , condition (77) implies condition (76) . The result therefore follows directly from Lemma 7.2 .
Corollary 7.3 shows that the ROM classifier is reliable at all points where its own decision value is sufficiently far from zero. Exploiting 7.3, once the RKHS error between the ROM and the FOM is known, partial agreement between FOM and ROM can be certified a priori at no additional cost for every z ∈ Ω \mathbf{z} \in\Omega z ∈ Ω that satisfies the above margin condition.
Hence, Lemma 7.2 and Corollary 7.3 ensure that the signs match between D ( z ) \mathbf{D}(\mathbf{z}) D ( z ) and D r ( z ) \mathbf{D}_r(\mathbf{z}) D r ( z ) if z \mathbf{z} z is sufficiently far away from the decision boundary. In order to obtain a useful check for cases when ∣ D r ( z ) ∣ ≤ E ξ r ( D r ) \lvert\mathbf{D}_r(\mathbf{z}) \rvert\le\mathcal{E}_{\xi}^{r}(\mathbf{D}_r) ∣ D r ( z )∣ ≤ E ξ r ( D r ) , consider the following lemma.
Lemma 7.4 (Projection–based classification certificate). Let D = G + b \mathbf{D} = \mathbf{G} + b D = G + b and D r = G r + b \mathbf{D}_r = \mathbf{G}_r + b D r = G r + b be the FOM and ROM decision functions (73) , respectively, with G ∈ G ξ n \mathbf{G} \in\mathcal{G}_{\xi}^{n} G ∈ G ξ n and G r ∈ G ξ r \mathbf{G}_r \in\mathcal{G}_{\xi}^{r} G r ∈ G ξ r . Under Assumption 5.1 , let λ ~ ∈ Ω r \widetilde{\boldsymbol{\lambda}} \in\Omega^{r} λ ∈ Ω r be the tuple of kernel centers defining the ROM G r \mathbf{G}_r G r , and P V r ~ ⊥ \mathcal{P}_{V_{\widetilde{\mathbf{r}}}^{\perp}} P V r ⊥ and P ( V r ~ r ) ⊥ \mathcal{P}_{(V_{\widetilde{\mathbf{r}}}^{r})^{\perp}} P ( V r r ) ⊥ denote the orthogonal projectors onto V r ~ r V_{\widetilde{\mathbf{r}}}^{r} V r r defined according to (36) and its orthogonal complement in H \mathcal{H} H , respectively. Assume that G r = P V r ~ r [ G ] \mathbf{G}_r = \mathcal{P}_{V_{\widetilde{\mathbf{r}}}^{r}}[\mathbf{G}] G r = P V r r [ G ] .
Then, for any z ∈ Ω \mathbf{z} \in\Omega z ∈ Ω , if
∣ D r ( z ) ∣ > E ξ r ( D r ) ∥ P ( V r ~ r ) ⊥ [ ξ z ] ∥ H = : E ξ , p r o j r ( D r ; z ) , (78) \lvert\mathbf{D}_r(\mathbf{z}) \rvert> \mathcal{E}_{\xi}^{r}(\mathbf{D}_r) \left\| \mathcal{P}_{(V_{\widetilde{\mathbf{r}}}^{r})^{\perp}}[\boldsymbol{\xi}_{\mathbf{z}}] \right\|_{\mathcal{H}} =: \mathcal{E}_{\xi,\mathrm{proj}}^{r}(\mathbf{D}_r;\mathbf{z}), \tag*{(78)} ∣ D r ( z )∣ > E ξ r ( D r ) P ( V r r ) ⊥ [ ξ z ] H =: E ξ , proj r ( D r ; z ) , (78) then sgn ( D ( z ) ) = sgn ( D r ( z ) ) \operatorname{sgn}(\mathbf{D}(\mathbf{z})) = \operatorname{sgn}(\mathbf{D}_r(\mathbf{z})) sgn ( D ( z )) = sgn ( D r ( z )) .
Proof. The proof is given in Section A.5.
The two sufficient conditions naturally suggest a hierarchical certified inference strategy.
For a new input z ∈ Ω z \in\Omega z ∈ Ω , the uniform certificate in Corollary 7.3 can be verified after evaluating the ROM, since it requires essentially no additional online computation once the ROM decision value and the RKHS error are available. If the condition is satisfied, the ROM prediction is certified to agree with the FOM prediction.
The projection–based certificate of Lemma 7.4 provides a tighter, but more computationally demanding alternative. When used together with the uniform certificate, it can be applied to inputs that are not certified by the latter, yielding a hierarchical two–stage certification procedure.
In the following, the applicability of the considered certificates is evaluated separately on the finite input sets Z t r \mathcal{Z}_{\mathrm{tr}} Z tr and Z t e \mathcal{Z}_{\mathrm{te}} Z te . We report the percentage of samples satisfying the uniform and projection–based certificate, and the exact pointwise classification agreement between the FOM and ROM as Certu n i f _{\mathrm{unif}} unif , Certp r o j _{\mathrm{proj}} proj , and Certt r u e _{\mathrm{true}} true , respectively. The approximation errors, classification performance, and applicability of the consistency conditions are summarized in Table 4 .
Table 4. Approximation errors, classification performance, and consistency–certificate applicability for the FOM, direct FOMs, and projected GMT–ROMs.
Figure 5 illustrates how the applicability of the consistency certificates evolves with the ROM order when evaluated on Z t r \mathcal{Z}_{\mathrm{tr}} Z tr (left) and Z t e \mathcal{Z}_{\mathrm{te}} Z te (right). The results highlight that approximation accuracy and certificate applicability should be considered as distinct properties of the ROM. Increasing the reduced order generally improves the agreement between the ROM and the FOM, but the computational usefulness of the different sufficient conditions can vary significantly.
Figure 5. Figure 5. Consistency certificates for the projected GMT–ROMs with r = 30 r = 30 r = 30 , 100 100 100 , 200 200 200 , evaluated on Z t r \mathcal{Z}_{\mathrm{tr}} Z tr (left) and Z t e \mathcal{Z}_{\mathrm{te}} Z te (right).
Recall that the FOM has order n = 2928 n = 2928 n = 2928 , while the reduced orders considered here are r ∈ { 30 , 100 , 200 } r \in\{30, 100, 200\} r ∈ { 30 , 100 , 200 } . For r = 30 r = 30 r = 30 , the uniform certificate is too conservative to certify any of the evaluated samples, whereas the projection–based certificate from Lemma 7.4 remains applicable to a large fraction of both the training and test inputs. Therefore, this regime
provides a representative case in which the second, projection–based certification stage is practically useful. In the present implementation, the corresponding O ( r 2 ) O(r^2) O ( r 2 ) arithmetic cost amounts to 30 2 = 900 30^2 = 900 3 0 2 = 900 operations at the level of the reduced representation and remains computationally competitive with the FOM evaluation.
For r = 100 r = 100 r = 100 , the results highlight the distinction between ROM fidelity and certificate applicability. At this order, the reduced models already reproduce the FOM classification accuracy very closely, and the projected GMT–ROM exhibits a small pointwise decision–function error. Nevertheless, the uniform certificate remains applicable only to a relatively limited fraction of the samples. The projection–dependent condition is much tighter and certifies almost all samples. However, its evaluation cost of order O ( r 2 ) O(r^2) O ( r 2 ) is now considerably larger than evaluating the FOM directly. Consequently, in this regime the projection–based certification stage may no longer be computationally attractive relative to evaluating the FOM directly.
A different behavior is observed at r = 200 r = 200 r = 200 . For the projected GMT–ROM, the uniform certificate becomes applicable to the large majority of the samples, while the projection–based certificate is essentially fully applicable but too costly. This regime is therefore particularly favorable for certified inference, because most predictions can already be certified using the inexpensive uniform condition, without evaluating the more costly projection–based factor.
Table 4 also shows that the direct reduction approach becomes highly competitive as the reduced order increases. In particular, for r = 100 r = 100 r = 100 and r = 200 r = 200 r = 200 , the direct ROM attains essentially the same classification accuracy as the projected GMT–ROM and closely reproduces the FOM predictions. Its relative RKHS error, however, decreases more slowly, and consequently the sufficient consistency certificates derived above remain applicable to a smaller fraction of the samples. This difference should not be interpreted as an intrinsic limitation of the direct approach. Rather, in the present implementation, the reduction procedure employed after the GMT transformation in H 2 ( D ) H_2(D) H 2 ( D ) is considerably more developed and numerically refined than the optimization performed directly in the original RBF–RKHS. The comparison therefore also reflects the different maturity of the two optimization procedures, in addition to the properties of the corresponding reduced representations.
The figure also provides a visual interpretation of the hierarchical inference strategy described above. After evaluating the ROM, samples that satisfy the uniform certificate can be used immediately. For the remaining samples, the tighter projection–based certificate can be checked before deciding whether to accept the ROM prediction without certification or whether the FOM should be evaluated as a fallback. Thus, improving the ROM affects not only approximation accuracy, but also the practical usefulness of the certification mechanism by increasing the fraction of inputs in Z t r \mathcal{Z}_{\mathrm{tr}} Z tr and Z t e \mathcal{Z}_{\mathrm{te}} Z te for which FOM–ROM consistency can be established without a full–order evaluation.
For reference, if each kernel center and coefficient is counted as one stored quantity and the common bias is counted once, the FOM requires p F O M = 2 n + 1 p_{\mathrm{FOM}} = 2n + 1 p FOM = 2 n + 1 stored quantities, whereas the direct and projected ROMs of order r r r require p d i r = p p r o j = 2 r + 1 p_{\mathrm{dir}} = p_{\mathrm{proj}} = 2r + 1 p dir = p proj = 2 r + 1 . This parameter reduction is also directly related to inference cost, since evaluating an RBF–SVM requires one kernel evaluation per stored center. Therefore, reducing the number of kernel centers from n n n to r r r lowers the cost of evaluating the decision function and, consequently, the expected runtime and energy consumption in embedded or real-time implementations. In particular, the projected ROM is smaller than the FOM whenever r < n r<n r < n .
Finally, the consistency results above should be interpreted as optional certification mechanisms rather than requirements for deploying the reduced classifier. Once constructed, a strong ROM can be evaluated independently of the FOM and may therefore be used as a standalone surrogate whenever memory, computational, or hardware constraints make full-order inference impractical.
MOR on splines and ReLU networks
In this example, we perform MOR on a deep feedforward ReLU network of type R → R \mathbb{R}\to\mathbb{R} R → R using a GMT transformation based approach. We follow [54 Equation (1.1)] to obtain the following definition of ReLU networks mapping R → R \mathbb{R}\to\mathbb{R} R → R :
F 1 ( t ) = t A ( 1 ) + b ( 1 ) , F_1(t)=tA^{(1)}+b^{(1)}, F 1 ( t ) = t A ( 1 ) + b ( 1 ) ,
F ℓ ( t ) = A ( ℓ ) ReLU ( F ℓ − 1 ( t ) + t c ( ℓ ) + b ( ℓ ) ) , ( ℓ = 2 , … , N − 1 ) , F_\ell(t)=A^{(\ell)}\operatorname{ReLU}\left(F_{\ell-1}(t)+tc^{(\ell)}+b^{(\ell)}\right),\qquad(\ell=2,\ldots,N-1), F ℓ ( t ) = A ( ℓ ) ReLU ( F ℓ − 1 ( t ) + t c ( ℓ ) + b ( ℓ ) ) , ( ℓ = 2 , … , N − 1 ) ,
F ( t ) : = F N ( t ) = A ( N ) ReLU ( F N − 1 ( t ) + t c ( N ) + b ( N ) ) (80) \mathbf{F}(t):=F_N(t)=A^{(N)}\operatorname{ReLU}\left(F_{N-1}(t)+tc^{(N)}+b^{(N)}\right) \tag*{(80)} F ( t ) := F N ( t ) = A ( N ) ReLU ( F N − 1 ( t ) + t c ( N ) + b ( N ) ) (80)
where A ( ℓ ) ∈ R k ℓ × k ℓ − 1 A^{(\ell)}\in\mathbb{R}^{k_\ell\times k_{\ell-1}} A ( ℓ ) ∈ R k ℓ × k ℓ − 1 , b ( ℓ ) , c ( ℓ ) ∈ R k ℓ b^{(\ell)},c^{(\ell)}\in\mathbb{R}^{k_\ell} b ( ℓ ) , c ( ℓ ) ∈ R k ℓ for ℓ = 1 , … , N \ell=1,\ldots,N ℓ = 1 , … , N with k N = 1 k_N=1 k N = 1 and
ReLU ( x ) : = [ max { x 1 , 0 } , … , max { x k , 0 } ] T , x = [ x 1 , … , x k ] T ∈ R k . \operatorname{ReLU}(\mathbf{x}):=[\max\{x_1,0\},\ldots,\max\{x_k,0\}]^{\mathsf{T}},\qquad\mathbf{x}=[x_1,\ldots,x_k]^{\mathsf{T}}\in\mathbb{R}^k. ReLU ( x ) := [ max { x 1 , 0 } , … , max { x k , 0 } ] T , x = [ x 1 , … , x k ] T ∈ R k .
ReLU networks have been instrumental to the development of modern deep learning. They form the building blocks of many state-of-the-art data-driven modeling schemes [40 , 66 ] . They are universal approximators [35 ] , which makes them particularly well suited to learn nonlinear maps from data. On the other hand, to achieve good approximation, large networks with exceedingly high parameter counts are often required. For example, modern image and language processing deep learning models often contain billions of parameters, which makes training and model evaluation expensive.
Compression of neural networks is often performed using pruning [10 ] , which removes neurons (rows and columns of A ( ℓ ) A^{(\ell)} A ( ℓ ) ) while trying to preserve the input-output relationship defined by F \mathbf{F} F . This removal or pruning step is then followed by an adjustment of the remaining parameters A r ( ℓ ) A_{\mathrm{r}}^{(\ell)} A r ( ℓ ) , b r ( ℓ ) b_{\mathrm{r}}^{(\ell)} b r ( ℓ ) , and c r ( ℓ ) c_{\mathrm{r}}^{(\ell)} c r ( ℓ ) for ℓ = 1 , … , N \ell=1,\ldots,N ℓ = 1 , … , N using gradient-based backpropagation schemes [39 ] . Variants of this heuristic approach have been empirically verified to perform well for trained networks [10 ] . However, no theoretical guarantees are given regarding the performance of the reduced (pruned) network. Furthermore, pruning has been observed to be unusable for very complex cases, for example, when the parameters of the neural network F \mathbf{F} F are chosen randomly.
In the approach proposed here, we do not attempt to reduce the model as given in (80) . Instead, we exploit the fact that any scalar-input scalar-output ReLU network of this form can be represented as a continuous piecewise linear (CPL) function, or equivalently with linear splines. In [54 ] , an efficient algorithm is proposed to obtain the CPL representation
F ( t ) = q 0 + q 1 t + ∑ k = 0 n − 1 w k ReLU ( t − λ k ) ( t ∈ R ) (81) \mathbf{F}(t)=q_0+q_1t+\sum_{k=0}^{n-1}\mathbf{w}_k\operatorname{ReLU}(t-\lambda_k)\qquad(t\in\mathbb{R}) \tag*{(81)} F ( t ) = q 0 + q 1 t + k = 0 ∑ n − 1 w k ReLU ( t − λ k ) ( t ∈ R ) (81)
of a deep ReLU network F \mathbf{F} F of the form (80) . Although the CPL representation can always be obtained, at first glance the relationship between the number of parameters needed to describe F \mathbf{F} F with (80) when compared to (81) looks discouraging. Indeed, in [46 ] it is shown that n n n in (81) is bounded by O ( N W 2 ) O(NW^2) O ( N W 2 ) , where W : = max { k 0 , … , k N } W:=\max\{k_0,\ldots,k_N\} W := max { k 0 , … , k N } denotes the width of the ReLU network. In contrast, in [32 ] the authors give evidence that despite this theoretical bound, in practice n n n grows linearly with the number of neurons.
In the presented experiment, we consider a four-layer neural network (N = 4 N=4 N = 4 ) with k 1 = k 4 = 1 k_1=k_4=1 k 1 = k 4 = 1 , k 2 = k 3 = 25 k_2=k_3=25 k 2 = k 3 = 25 , and c ( ℓ ) = 0 \mathbf{c}^{(\ell)}=0 c ( ℓ ) = 0 for ℓ = 2 , 3 , 4 \ell=2,3,4 ℓ = 2 , 3 , 4 , hence a total of M = 1375 M=1375 M = 1375 parameters are required to describe the model in the form (80) . In our experiment, the components of the parameters A ( ℓ ) , b ( ℓ ) \mathbf{A}^{(\ell)},\mathbf{b}^{(\ell)} A ( ℓ ) , b ( ℓ ) are chosen randomly. For this network, the CPL form provided by the algorithm in [54 ] is described by only n = 187 n=187 n = 187 knots λ k \lambda_k λ k . Thus, in this example, the CPL form provides a more compact representation of the same FOM.
In order to use the algorithms presented in § 7 to perform MOR on (81) , we have to express F \mathbf{F} F as a linear combination of kernel slices. To this end, consider the kernel
ξ ( t , λ ) : = 1 + min { t − Δ , λ − Δ } , (82) \xi(t,\lambda):=1+\min\{t-\Delta,\lambda-\Delta\}, \tag*{(82)} ξ ( t , λ ) := 1 + min { t − Δ , λ − Δ } , (82)
where Δ \Delta Δ is a parameter satisfying Δ < min { λ 0 , … , λ n − 1 } \Delta<\min\{\lambda_0,\ldots,\lambda_{n-1}\} Δ < min { λ 0 , … , λ n − 1 } . With this choice, ξ \xi ξ is the reproducing kernel of the Sobolev-like RKHS
H : = { G ∈ L 2 ( Ω ) : G ( Δ ) 2 + ∫ Ω G ′ ( t ) 2 d t < ∞ } , \mathcal{H}:=\left\{\mathbf{G}\in L^2(\Omega):\mathbf{G}(\Delta)^2+\int_{\Omega}\mathbf{G}'(t)^2\,\mathrm{d}t<\infty\right\}, H := { G ∈ L 2 ( Ω ) : G ( Δ ) 2 + ∫ Ω G ′ ( t ) 2 d t < ∞ } ,
where Ω = [ Δ , T ] \Omega=[\Delta,T] Ω = [ Δ , T ] with T ∈ R T\in\mathbb{R} T ∈ R or Ω = [ Δ , ∞ ) \Omega=[\Delta,\infty) Ω = [ Δ , ∞ ) [68 Section 1.2] , and differentiation is understood in the weak sense. Note also, that H \mathcal{H} H is equipped with the inner product
⟨ G , H ⟩ H : = G ( Δ ) H ( Δ ) + ∫ Ω G ′ ( t ) H ′ ( t ) d t . \langle\mathbf{G},\mathbf{H}\rangle_{\mathcal{H}}:=\mathbf{G}(\Delta)\mathbf{H}(\Delta)+\int_{\Omega}\mathbf{G}'(t)\mathbf{H}'(t)\,\mathrm{d}t. ⟨ G , H ⟩ H := G ( Δ ) H ( Δ ) + ∫ Ω G ′ ( t ) H ′ ( t ) d t .
In practice, the choice of Δ \Delta Δ in (82) along with the choice of T T T in Ω = [ Δ , T ] \Omega=[\Delta,T] Ω = [ Δ , T ] depend on the considered FOM network F \mathbf{F} F . In this experiment, they are determined as Δ : = − ( ∣ min { λ 0 , … , λ n − 1 } − 1 ∣ + ∣ max { λ 0 , … , λ n − 1 } ∣ ) \Delta:=-(|\min\{\lambda_0,\ldots,\lambda_{n-1}\}-1|+|\max\{\lambda_0,\ldots,\lambda_{n-1}\}|) Δ := − ( ∣ min { λ 0 , … , λ n − 1 } − 1∣ + ∣ max { λ 0 , … , λ n − 1 } ∣ ) , resulting in the values T : = − Δ = 70.19 T:=-\Delta=70.19 T := − Δ = 70.19 . To express (81) in terms of ξ \xi ξ from (82) , observe that min { a , b } = a − ReLU ( a − b ) \min\{a,b\}=a-\operatorname{ReLU}(a-b) min { a , b } = a − ReLU ( a − b ) for all a , b ∈ R a,b\in\mathbb{R} a , b ∈ R and therefore,
ξ λ k ( t ) = 1 + min { t − Δ , λ k − Δ } = 1 − Δ + t − ReLU ( t − λ k ) ( t ∈ Ω , k = 0 , … , n − 1 ) . \xi_{\lambda_k}(t)=1+\min\{t-\Delta,\lambda_k-\Delta\}=1-\Delta+t-\operatorname{ReLU}(t-\lambda_k)\qquad(t\in\Omega,\ k=0,\ldots,n-1). ξ λ k ( t ) = 1 + min { t − Δ , λ k − Δ } = 1 − Δ + t − ReLU ( t − λ k ) ( t ∈ Ω , k = 0 , … , n − 1 ) .
Applying this to ReLU ( t − λ k ) \operatorname{ReLU}(t-\lambda_k) ReLU ( t − λ k ) in (81) gives
F ( t ) = q 0 + q 1 t + ∑ k = 0 n − 1 w k [ ( 1 − Δ + t ) − ξ λ k ( t ) ] = q 0 + ( 1 − Δ ) ∑ k = 0 n − 1 w k + ( q 1 + ∑ k = 0 n − 1 w k ) t − ∑ k = 0 n − 1 w k ξ λ k ( t ) . \begin{aligned}
F(t)=q_0+q_1t+\sum_{k=0}^{n-1}\mathfrak{w}_k\left[(1-\Delta+t)-\xi_{\lambda_k}(t)\right] \\
&=q_0+(1-\Delta)\sum_{k=0}^{n-1}\mathfrak{w}_k+\left(q_1+\sum_{k=0}^{n-1}\mathfrak{w}_k\right)t-\sum_{k=0}^{n-1}\mathfrak{w}_k\xi_{\lambda_k}(t).
\end{aligned} F ( t ) = q 0 + q 1 t + k = 0 ∑ n − 1 w k [ ( 1 − Δ + t ) − ξ λ k ( t ) ] = q 0 + ( 1 − Δ ) k = 0 ∑ n − 1 w k + ( q 1 + k = 0 ∑ n − 1 w k ) t − k = 0 ∑ n − 1 w k ξ λ k ( t ) .
Finally, with w k = − r k \mathfrak{w}_k=-\mathfrak{r}_k w k = − r k for k = 0 , … , n − 1 k=0,\ldots,n-1 k = 0 , … , n − 1 , we can write (81) as
F ( t ) = β ( t ) + G ( t ) = β 0 + β 1 t + ∑ k = 0 n − 1 r k ξ λ k ( t ) (83) F(t)=\beta(t)+G(t)=\beta_0+\beta_1t+\sum_{k=0}^{n-1}\mathfrak{r}_k\xi_{\lambda_k}(t) \tag*{(83)} F ( t ) = β ( t ) + G ( t ) = β 0 + β 1 t + k = 0 ∑ n − 1 r k ξ λ k ( t ) (83)
where G ∈ G ξ n ⊆ H G\in\mathcal{G}_{\xi}^{n}\subseteq\mathcal{H} G ∈ G ξ n ⊆ H , and
β 0 : = q 0 + ( 1 − Δ ) ∑ k = 0 n − 1 w k , β 1 : = q 1 + ∑ k = 0 n − 1 w k . \beta_0:=q_0+(1-\Delta)\sum_{k=0}^{n-1}\mathfrak{w}_k,\qquad\beta_1:=q_1+\sum_{k=0}^{n-1}\mathfrak{w}_k. β 0 := q 0 + ( 1 − Δ ) k = 0 ∑ n − 1 w k , β 1 := q 1 + k = 0 ∑ n − 1 w k .
Similarly to the SVM experiment in § 9 , the affine component β ( t ) = β 0 + β 1 t \beta(t)=\beta_0+\beta_1t β ( t ) = β 0 + β 1 t is retained exactly, while the MOR procedure is applied only to the non-affine component G G G in (83) .
Note that even though the FOM G G G now belongs to the RKHS H \mathcal{H} H , the derivatives D λ ξ λ D_{\lambda}\xi_{\lambda} D λ ξ λ and D λ ξ λ D_{\lambda}^{\ } \xi_{\lambda} D λ ξ λ do not exist for any λ ∈ Ω \lambda\in\Omega λ ∈ Ω . Consequently, we cannot apply Theorem 4.3 to evaluate the optimality of a ROM G r G_r G r . Accordingly, Algorithm 6.1 also cannot be applied directly.
To overcome this limitation, we express the FOM in the GMT basis
G = ∑ k = 0 n − 1 ⟨ G , φ k λ ⟩ H φ k λ G=\sum_{k=0}^{n-1}\langle G,\varphi_k^\lambda\rangle_{\mathcal{H}}\varphi_k^\lambda G = k = 0 ∑ n − 1 ⟨ G , φ k λ ⟩ H φ k λ
and apply the GMT transformation from Definition 5.22 to map the problem to an RKHS with a sufficiently smooth kernel. Then, the MOR procedure can be carried out using Algorithm 6.2. For this example, the companion RKHS H ^ 2 \widehat{\mathcal{H}}_2 H 2 is chosen as H 2 1 \mathcal{H}_2^1 H 2 1 as given in (14) . The corresponding domain maps are given as
γ ( t ) = t / T , γ − 1 ( z ) = z T ( t ∈ [ − T , T ] , z ∈ ( − 1 , 1 ) ) . \gamma(t)=t/T,\qquad\gamma^{-1}(z)=zT\qquad(t\in[-T,T],\ z\in(-1,1)). γ ( t ) = t / T , γ − 1 ( z ) = z T ( t ∈ [ − T , T ] , z ∈ ( − 1 , 1 )) .
We observe that choosing these initial kernel centers has a notable effect on the ROM, although all investigated choices lead to highly accurate ROMs.
We consider two baseline algorithms for comparison, and two different initialization schemes for Algorithm 6.2. First, we compare the efficiency of Algorithm 6.2 to the structured pruning method [10 ] . This method acts directly on the model representation (80) . First, it assigns an importance score to each neuron in the hidden network layers. This score is a heuristic to measure the effect of the neuron with respect to the incoming and outgoing information. The importance score is given by
∥ A j ( ℓ ) ∥ 2 ∥ ( A j ( ℓ + 1 ) ) T ∥ 2 ( j = 0 , … , k ℓ − 1 ) . \left\|A_j^{(\ell)}\right\|_2\left\|\left(A_j^{(\ell+1)}\right)^{\mathsf{T}}\right\|_2\qquad(j=0,\ldots,k_\ell-1). A j ( ℓ ) 2 ( A j ( ℓ + 1 ) ) T 2 ( j = 0 , … , k ℓ − 1 ) .
Algorithm 7.1 Kernel–EIM for piecewise–linear spline approximation
Inputs: FOM G \mathbf{G} G queryable on [ Δ , T ] [\Delta,T] [ Δ , T ] , kernel ξ \xi ξ from (82) , reduced order r r r .
Outputs: ROM G r \mathbf{G}_r G r .
1: E 0 ← G \mathbf{E}_0 \gets\mathbf{G} E 0 ← G .
2: λ ~ 0 ← arg max t ∈ [ Δ , T ] ∣ E 0 ( t ) ∣ \widetilde{\lambda}_0 \gets\arg\max_{t\in[\Delta,T]}|\mathbf{E}_0(t)| λ 0 ← arg max t ∈ [ Δ , T ] ∣ E 0 ( t ) ∣ .
3: c 0 ← G ( λ ~ 0 ) ξ ( λ ~ 0 , λ ~ 0 ) c_0 \gets\dfrac{\mathbf{G}(\widetilde{\lambda}_0)}{\xi(\widetilde{\lambda}_0,\widetilde{\lambda}_0)} c 0 ← ξ ( λ 0 , λ 0 ) G ( λ 0 ) .
4: G 1 ← c 0 ξ ( ⋅ , λ ~ 0 ) \mathbf{G}_1 \gets c_0\xi(\cdot,\widetilde{\lambda}_0) G 1 ← c 0 ξ ( ⋅ , λ 0 ) .
5: for j = 1 , … , r − 1 j=1,\ldots,r-1 j = 1 , … , r − 1 do
6: E j ← G − G j \mathbf{E}_j \gets\mathbf{G}-\mathbf{G}_j E j ← G − G j .
7: λ ~ j ← arg max t ∈ [ Δ , T ] ∣ E j ( t ) ∣ \widetilde{\lambda}_j \gets\arg\max_{t\in[\Delta,T]}|\mathbf{E}_j(t)| λ j ← arg max t ∈ [ Δ , T ] ∣ E j ( t ) ∣ .
8: Assemble K ( j ) = [ ξ ( λ ~ i , λ ~ k ) ] i , k = 0 j \mathbf{K}^{(j)}=\left[\xi(\widetilde{\lambda}_i,\widetilde{\lambda}_k)\right]_{i,k=0}^{j} K ( j ) = [ ξ ( λ i , λ k ) ] i , k = 0 j and g ( j ) = [ G ( λ ~ i ) ] i = 0 j \mathbf{g}^{(j)}=\left[\mathbf{G}(\widetilde{\lambda}_i)\right]_{i=0}^{j} g ( j ) = [ G ( λ i ) ] i = 0 j .
9: Solve K ( j ) c ( j ) = g ( j ) \mathbf{K}^{(j)}\mathbf{c}^{(j)}=\mathbf{g}^{(j)} K ( j ) c ( j ) = g ( j ) for c ( j ) : = [ c 0 ( j ) , … , c j ( j ) ] T \mathbf{c}^{(j)}:=\left[c_0^{(j)},\ldots,c_j^{(j)}\right]^{\mathsf{T}} c ( j ) := [ c 0 ( j ) , … , c j ( j ) ] T .
10: G j + 1 ← ∑ i = 0 j c i ( j ) ξ λ ~ i \mathbf{G}_{j+1}\gets\sum_{i=0}^{j}c_i^{(j)}\xi_{\widetilde{\lambda}_i} G j + 1 ← ∑ i = 0 j c i ( j ) ξ λ i .
11: end for
12: return G r \mathbf{G}_r G r .
where A j ( ℓ ) A^{(\ell)}_j A j ( ℓ ) denotes the j j j th column of A ( ℓ ) A^{(\ell)} A ( ℓ ) . In a single step of the pruning method, the neuron with the lowest importance score is pruned from the network, that is, the corresponding column of A ( ℓ ) A^{(\ell)} A ( ℓ ) and row of A ( ℓ + 1 ) A^{(\ell+1)} A ( ℓ + 1 ) are removed. This process continues iteratively until a defined compression ratio is achieved.
Since the proposed approach acts on (83) , we also compare the effectiveness of Algorithm 6.2 with a kernel slice fitting scheme. In particular, we consider a variant of the empirical interpolation method (EIM) adapted to fitting kernel slices constructed from ξ \xi ξ in (82) . EIM was originally introduced in [4 ] to approximate the solution of parametric partial differential equations, but has since been widely used to approximate nonlinear operators in general. In the spline–kernel fitting context of this experiment, it greedily selects the knots λ ~ j \widetilde{\lambda}_j λ j , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 describing the ROM. The considered EIM variant is summarized in Algorithm 7.1. We note that when applied to kernel slices, the algorithm is also known as greedy kernel interpolation, or kernel matching pursuit [67 ] .
We use Algorithm 7.1 in two distinct ways. First, it is used as a baseline method for Algorithm 6.2, when the latter is used with a quasi–random heuristic initialization. Then, we use EIM to initialize a starting point λ ~ ( 0 ) ∈ [ Δ , T ] \widetilde{\boldsymbol{\lambda}}^{(0)}\in[\Delta,T] λ ( 0 ) ∈ [ Δ , T ] for Algorithm 6.2. With this “fine tuning”, we ensure that the final ROM satisfies our H \mathcal{H} H optimality conditions.
In summary, we consider the following models:
F: the FOM neural network expressed according to (80) ,
F p r u n {}^{\mathrm{prun}} prun : ROM obtained by applying pruning directly to F,
G: the FOM kernel representation of the non–affine part obtained according to (83) ,
G r e c o n \mathbf{G}^{\mathrm{econ}}_r G r econ : projected ROM obtained using GMT transformation. In particular, G r e c o n : = P V λ ~ r [ G ] \mathbf{G}^{\mathrm{econ}}_r := \mathcal{P}_{\mathcal{V}^{r}_{\tilde{\lambda}}}[\mathbf{G}] G r econ := P V λ ~ r [ G ] , where V λ ~ r = span { ξ λ ~ 0 , … , ξ λ ~ r − 1 } \mathcal{V}^{r}_{\tilde{\lambda}} = \operatorname{span}\{\boldsymbol{\xi}_{\tilde{\lambda}_0},\ldots,\boldsymbol{\xi}_{\tilde{\lambda}_{r-1}}\} V λ ~ r = span { ξ λ ~ 0 , … , ξ λ ~ r − 1 } and the kernel centers λ ~ j \tilde{\lambda}_j λ ~ j , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 are acquired using Algorithm 6.2 and heuristic initialization,
G r e c o n − E I M \mathbf{G}^{\mathrm{econ-EIM}}_r G r econ − EIM : projected ROM obtained using Algorithm 6.2 with EIM initialization, i.e., where the initial kernel centers λ ~ j \tilde{\lambda}_j λ ~ j , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 are found with the greedy Algorithm 7.1,
To evaluate the approximation quality, we first consider the mean squared error
E M S E ( H r ; H ) : = 1 K ∑ k = 0 K − 1 ( H ( t k ) − H r ( t k ) ) 2 , (84) \mathcal{E}_{\mathrm{MSE}}(\mathbf{H}_r;\mathbf{H}) := \frac{1}{K}\sum_{k=0}^{K-1}\left(\mathbf{H}(t_k)-\mathbf{H}_r(t_k)\right)^2, \tag*{(84)} E MSE ( H r ; H ) := K 1 k = 0 ∑ K − 1 ( H ( t k ) − H r ( t k ) ) 2 , (84)
where { t k } k = 0 K − 1 \{t_k\}_{k=0}^{K-1} { t k } k = 0 K − 1 is a K K K –point equidistant grid of Ω \Omega Ω . Here, we choose K = 1000 K=1000 K = 1000 . For the pruning baseline, we evaluate E M S E ( F p r u n ; F ) \mathcal{E}_{\mathrm{MSE}}(\mathbf{F}^{\mathrm{prun}};\mathbf{F}) E MSE ( F prun ; F ) , whereas for the kernel–based ROMs we evaluate E M S E ( G r e c o n ; G ) \mathcal{E}_{\mathrm{MSE}}(\mathbf{G}^{\mathrm{econ}}_r;\mathbf{G}) E MSE ( G r econ ; G ) . Since the affine component β \boldsymbol{\beta} β is retained exactly, the latter is equivalent to evaluating the MSE between the functions F = β + G \mathbf{F}=\boldsymbol{\beta}+\mathbf{G} F = β + G and F r = β + G r \mathbf{F}_r=\boldsymbol{\beta}+\mathbf{G}_r F r = β + G r , where G r \mathbf{G}_r G r can be any of the ROMs discussed above.
In addition, for the kernel–based ROMs we report the relative RKHS error as defined in (74) , where again, G r \mathbf{G}_r G r denotes an arbitrary ROM. This metric is not reported for F p r u n \mathbf{F}^{\mathrm{prun}} F prun , because no guarantees can be given that its CPL form
F p r u n ( t ) = : q ~ ( t ) + G p r u n ( t ) = q ~ 0 + q ~ 1 t + ∑ j = 0 n ~ − 1 w ~ j ReLU ( t − μ ~ j ) \mathbf{F}^{\mathrm{prun}}(t)=:\tilde{\mathbf{q}}(t)+\mathbf{G}^{\mathrm{prun}}(t)=\tilde{\mathbf{q}}_0+\tilde{\mathbf{q}}_1t+\sum_{j=0}^{\tilde{n}-1}\tilde{w}_j\operatorname{ReLU}(t-\tilde{\mu}_j) F prun ( t ) =: q ~ ( t ) + G prun ( t ) = q ~ 0 + q ~ 1 t + j = 0 ∑ n ~ − 1 w ~ j ReLU ( t − μ ~ j )
satisfies Δ < min { μ ~ 0 , … , μ ~ n ~ − 1 } \Delta<\min\{\tilde{\mu}_0,\ldots,\tilde{\mu}_{\tilde{n}-1}\} Δ < min { μ ~ 0 , … , μ ~ n ~ − 1 } whose violation would result in G p r u n ∉ H \mathbf{G}^{\mathrm{prun}}\notin\mathcal{H} G prun ∈ / H .
In our experiments, we consider the reduced orders
r ∈ { 50 , 60 , 70 , 80 , 90 , 100 } (85) r\in\{50,60,70,80,90,100\} \tag*{(85)} r ∈ { 50 , 60 , 70 , 80 , 90 , 100 } (85)
and recall that the FOM G \mathbf{G} G in (83) can be described with n = 187 n=187 n = 187 knots. We already mentioned that the deep ReLU form of the FOM in (80) is defined by M = 1375 M=1375 M = 1375 parameters. To ensure a fair comparison for pruning, we compare the MSE of ROMs that provide the same overall compression ratio
C R = Total number of ROM parameters Total number of FOM parameters ⋅ 100 % , \mathrm{CR}=\frac{\text{Total number of ROM parameters}}{\text{Total number of FOM parameters}}\cdot100\%, CR = Total number of FOM parameters Total number of ROM parameters ⋅ 100% ,
where the deep network or the kernel–slice formulations are considered depending on the MOR method. Therefore, the different MOR schemes are compared along the following (approximate) compression ratios determined by (85) :
C R ∈ { 27.3 , 32.6 , 38.0 , 43.3 , 48.7 , 54.0 } % . \mathrm{CR}\in\{27.3,32.6,38.0,43.3,48.7,54.0\}\%. CR ∈ { 27.3 , 32.6 , 38.0 , 43.3 , 48.7 , 54.0 } %.
Table 5 summarizes the results of the experiment. To better resolve the differences between the kernel–based ROMs, we additionally report the improvement relative to the EIM baseline, defined as
Table 5. Proposed and baseline ROMs for MOR on a ReLU network.
I r e c o n : = log 10 ( E ( G r e c o n ) ) − log 10 ( E ( G r E I M ) ) , \mathcal{I}_r^{\mathrm{econ}} := \log_{10}\left(\mathcal{E}\left(G_r^{\mathrm{econ}}\right)\right) - \log_{10}\left(\mathcal{E}\left(G_r^{\mathrm{EIM}}\right)\right), I r econ := log 10 ( E ( G r econ ) ) − log 10 ( E ( G r EIM ) ) ,
I r e c o n - E I M : = log 10 ( E ( G r e c o n - E I M ) ) − log 10 ( E ( G r E I M ) ) . \mathcal{I}_r^{\mathrm{econ\text{-}EIM}} := \log_{10}\left(\mathcal{E}\left(G_r^{\mathrm{econ\text{-}EIM}}\right)\right) - \log_{10}\left(\mathcal{E}\left(G_r^{\mathrm{EIM}}\right)\right). I r econ - EIM := log 10 ( E ( G r econ - EIM ) ) − log 10 ( E ( G r EIM ) ) .
where E \mathcal{E} E denotes either the MSE in (84) or the relative RKHS error in (74) . A negative value indicates an improvement over the EIM baseline, whereas a positive value indicates that EIM achieves the smaller error, and the absolute value indicates the differences in the order of magnitudes of the errors. Since G r e c o n G_r^{\mathrm{econ}} G r econ , G r E I M G_r^{\mathrm{EIM}} G r EIM , and G r e c o n - E I M G_r^{\mathrm{econ\text{-}EIM}} G r econ - EIM all provide high quality approximations, these improvement measures help resolve the small differences between their performances. This is further confirmed in Figure 6 , which shows the reconstruction of F F F over [ Δ , T ] [\Delta,T] [ Δ , T ] . In contrast, the considered pruning approach performs poorly for the randomly parameterized FOM used in this experiment. Pruning’s inability to reduce neural networks with random weights has already been observed in [10 ] .
Figure 6. Approximation of the FOM neural network F \mathbf{F} F with the considered kernel expansion based ROMs. Here, F r e c o n ( t ) : = G r e c o n ( t ) + β ( t ) \mathbf{F}^{\mathrm{econ}}_{r}(t) := G^{\mathrm{econ}}_{r}(t) + \beta(t) F r econ ( t ) := G r econ ( t ) + β ( t ) , F r E I M ( t ) : = G r E I M ( t ) + β ( t ) \mathbf{F}^{\mathrm{EIM}}_{r}(t) := G^{\mathrm{EIM}}_{r}(t) + \beta(t) F r EIM ( t ) := G r EIM ( t ) + β ( t ) , and F r e c o n - E I M ( t ) : = G r e c o n - E I M ( t ) + β ( t ) \mathbf{F}^{\mathrm{econ\text{-}EIM}}_{r}(t) := G^{\mathrm{econ\text{-}EIM}}_{r}(t) + \beta(t) F r econ - EIM ( t ) := G r econ - EIM ( t ) + β ( t ) are given in (83) . In this figure, r = 70 r = 70 r = 70 is considered.
The spline–fitting–based ROMs G r e c o n G_r^{\mathrm{econ}} G r econ , G r e c o n - E I M G_r^{\mathrm{econ\text{-}EIM}} G r econ - EIM , and G r E I M G_r^{\mathrm{EIM}} G r EIM provide very similar results. In order to explore subtle differences between them, we consult the last two columns of Table 5 . We observe that quasi–random initialization of Algorithm 6.2 yields ROMs with slightly worse H \mathcal{H} H norm errors than G r E I M G_r^{\mathrm{EIM}} G r EIM and G r e c o n - E I M G_r^{\mathrm{econ\text{-}EIM}} G r econ - EIM . On the other hand, when evaluated over a fine grid, the corresponding MSE scores are improved for G r e c o n G_r^{\mathrm{econ}} G r econ when compared to G r E I M G_r^{\mathrm{EIM}} G r EIM and G r e c o n - E I M G_r^{\mathrm{econ\text{-}EIM}} G r econ - EIM for most of the considered compression ratios. Recall that G r e c o n - E I M G_r^{\mathrm{econ\text{-}EIM}} G r econ - EIM is obtained by initializing Algorithm 6.2 from the kernel centers provided by EIM. When compared against G r E I M G_r^{\mathrm{EIM}} G r EIM , this model provides marginal improvements both in terms of E ξ , r e l r E_{\xi,\mathrm{rel}}^r E ξ , rel r and E M S E E_{\mathrm{MSE}} E MSE . These improvements remain very small over the considered range of compression ratios. In fact, we find that G r e c o n - E I M G_r^{\mathrm{econ\text{-}EIM}} G r econ - EIM and G r E I M G_r^{\mathrm{EIM}} G r EIM match up to 9–12 decimal digits across every considered compression ratio, indicating that the EIM–initialized knots already lie extremely close to the H \mathcal{H} H –optimal configuration found by Algorithm 6.2. We conclude that Algorithm 6.2 can slightly refine G r E I M G^{\mathrm{EIM}}_{r} G r EIM when initialized from the EIM kernel centers. However, both the proposed methods and the baseline G r E I M G^{\mathrm{EIM}}_{r} G r EIM provide high quality approximations to the FOM. Hence, spline–fitting–based reduction of ReLU networks seems reasonable in cases when a spline representation of the network can be obtained.
We notice the following interesting behavior of the EIM baseline model G r E I M G^{\mathrm{EIM}}_{r} G r EIM . Although the greedy iteration in Algorithm 7.1 aims to reduce the L ∞ L^\infty L ∞ error of the ROM with the addition of each new knot, we observe that G r E I M G^{\mathrm{EIM}}_{r} G r EIM also approximately satisfies the equilibrium–based conditions of Theorem 4.13 . This is reflected in Table 6 , which shows the magnitude of the equilibrium function evaluated at the kernel centers determining each ROM. In this experiment, the EIM–ROMs also approximately satisfy the necessary conditions of H \mathcal{H} H optimality in the GMT transformation sense. This is notable because Algorithm 7.1 does not explicitly enforce these conditions. On the other hand, the ROMs obtained with Algorithm 6.2 satisfies the equilibrium–based optimality conditions to even higher numerical accuracy.
Table 6. Maximum equilibrium function magnitude ∥ D ~ ( γ ( λ ˉ 0 ) , … , γ ( λ ˉ r − 1 ) ) ∥ ∞ \left\lVert\widetilde{\mathcal{D}}\left(\gamma(\bar{\lambda}_0),\ldots,\gamma(\bar{\lambda}_{r-1})\right)\right\rVert_{\infty} D ( γ ( λ ˉ 0 ) , … , γ ( λ ˉ r − 1 ) ) ∞ in the companion RKHS H 2 1 H^1_2 H 2 1 for different ROMs. These results were computed using MATLAB’s variable precision arithmetic using 100 digits.
In this experiment, we presented a viable application of Algorithm 6.2 to reduce R → R \mathbb{R} \to\mathbb{R} R → R type deep ReLU networks. We conclude that, for the randomly parameterized FOM, the proposed spline–based reduction methods substantially outperform the considered pruning baseline. The presented experiment also provides a strong case for further investigations into using spline–based representations of ReLU networks in order to manipulate already trained models. In addition, we observe that an effective greedy spline fitting technique, EIM, also achieves models which approximately satisfy the GMT–transformation–based optimality conditions presented in this paper. Finally, we emphasize that this application is made possible through the use of GMT transformations, because the native RKHS containing the ReLU network does not have a differentiable kernel. Consequently, without the use of the GMT transformation, the optimality conditions proposed in § 4 cannot be directly evaluated.
PDE discretization
We next demonstrate how Algorithm 6.1 can be used to solve PDEs. This example is once again meant to emphasize the scope of our results, because the problem considered is not a traditional MOR task. A systematic analysis of extending the framework for other PDEs is left for future work. In this experiment, we consider the fourth–order Euler–Bernoulli beam equation
G ( 4 ) ( x ) = F ( x ) , x ∈ Ω ≔ ( 0 , 1 ) , (86) \mathbf{G}^{(4)}(x)=\mathbf{F}(x),\qquad x\in\Omega\coloneqq(0,1), \tag*{(86)} G ( 4 ) ( x ) = F ( x ) , x ∈ Ω : = ( 0 , 1 ) , (86)
with clamped–free boundary conditions
G ( 0 ) = G ′ ( 0 ) = 0 , G ′ ′ ( 1 ) = G ( 3 ) ( 1 ) = 0. (87) \mathbf{G}(0)=\mathbf{G}'(0)=0,\qquad\mathbf{G}''(1)=\mathbf{G}^{(3)}(1)=0. \tag*{(87)} G ( 0 ) = G ′ ( 0 ) = 0 , G ′′ ( 1 ) = G ( 3 ) ( 1 ) = 0. (87)
For the numerical experiment, we construct the problem by prescribing the analytic solution
G ( x ) = x 2 ( 1 − x ) 2 e − ( x − 3 / 4 ) 2 2 σ 2 , (88) \mathbf{G}(x)=x^2(1-x)^2e^{-\frac{(x-3/4)^2}{2\sigma^2}}, \tag*{(88)} G ( x ) = x 2 ( 1 − x ) 2 e − 2 σ 2 ( x − 3/4 ) 2 , (88)
with σ = 1 20 \sigma=\frac{1}{20} σ = 20 1 , and defining the corresponding right–hand side as
F ( x ) ≔ G ( 4 ) ( x ) = [ x 2 ( 1 − x ) 2 e − ( x − 3 / 4 ) 2 2 σ 2 ] ( 4 ) . \mathbf{F}(x)\coloneqq\mathbf{G}^{(4)}(x)=\left[x^2(1-x)^2e^{-\frac{(x-3/4)^2}{2\sigma^2}}\right]^{(4)}. F ( x ) : = G ( 4 ) ( x ) = [ x 2 ( 1 − x ) 2 e − 2 σ 2 ( x − 3/4 ) 2 ] ( 4 ) .
Hence, G \mathbf{G} G in (88) is known by construction and can be used to assess the accuracy of the numerical approximations.
Similarly to the finite element method (FEM), our MOR based solution considers the weak form of the above equation:
∫ Ω G ( 4 ) ( x ) v ( x ) d x = ∫ Ω F ( x ) v ( x ) d x , \int_{\Omega} G^{(4)}(x)v(x)\,\mathrm{d}x = \int_{\Omega} F(x)v(x)\,\mathrm{d}x, ∫ Ω G ( 4 ) ( x ) v ( x ) d x = ∫ Ω F ( x ) v ( x ) d x ,
where v v v belongs to the test space
H : = { v ∈ H 2 ( Ω ) : v ( 0 ) = v ′ ( 0 ) = 0 } . (89) \mathcal{H} := \{v \in H^2(\Omega): v(0)=v'(0)=0\}. \tag*{(89)} H := { v ∈ H 2 ( Ω ) : v ( 0 ) = v ′ ( 0 ) = 0 } . (89)
Here, H 2 ( Ω ) H^2(\Omega) H 2 ( Ω ) denotes the Sobolev–Hilbert space
H 2 ( Ω ) : = { v ∈ L 2 ( Ω ) : v ′ , v ′ ′ ∈ L 2 ( Ω ) , ∑ n = 0 2 ∥ v ( n ) ∥ L 2 ( Ω ) 2 < ∞ } , H^2(\Omega) := \left\{v \in L^2(\Omega): v',v'' \in L^2(\Omega), \sum_{n=0}^{2}\lVert v^{(n)}\rVert_{L^2(\Omega)}^2 < \infty\right\}, H 2 ( Ω ) := { v ∈ L 2 ( Ω ) : v ′ , v ′′ ∈ L 2 ( Ω ) , n = 0 ∑ 2 ∥ v ( n ) ∥ L 2 ( Ω ) 2 < ∞ } ,
where differentiation is understood in the weak sense. Following integration by parts and applying the boundary condition [56 Chapter 4] gives the weak equation
∫ Ω G ′ ′ ( x ) v ′ ′ ( x ) d x ⏟ = : a ( G , v ) = ∫ Ω F ( x ) v ( x ) d x ⏟ = : f ( v ) ( v ∈ H ) . (90) \underbrace{\int_{\Omega} G''(x)v''(x)\,\mathrm{d}x}_{=:a(\mathbf{G},v)}
=
\underbrace{\int_{\Omega} F(x)v(x)\,\mathrm{d}x}_{=:f(v)}
\qquad(v \in\mathcal{H}). \tag*{(90)} =: a ( G , v ) ∫ Ω G ′′ ( x ) v ′′ ( x ) d x = =: f ( v ) ∫ Ω F ( x ) v ( x ) d x ( v ∈ H ) . (90)
Note that H \mathcal{H} H is a Hilbert space equipped with the inner product
⟨ G , H ⟩ H : = ∫ Ω G ′ ′ ( x ) H ′ ′ ( x ) d x ( G , H ∈ H ) . \langle\mathbf{G},\mathbf{H}\rangle_{\mathcal{H}} := \int_{\Omega} G''(x)H''(x)\,\mathrm{d}x \qquad(\mathbf{G},\mathbf{H} \in\mathcal{H}). ⟨ G , H ⟩ H := ∫ Ω G ′′ ( x ) H ′′ ( x ) d x ( G , H ∈ H ) .
We observe, see e.g., [68 Eq. (1.2.4)] that H \mathcal{H} H is an RKHS with the kernel
ξ ( x , λ ) = 1 2 min { x , λ } 2 max { x , λ } − 1 6 min { x , λ } 3 ( x , λ ∈ Ω ) . (91) \xi(x,\lambda) = \frac{1}{2}\min\{x,\lambda\}^2\max\{x,\lambda\} - \frac{1}{6}\min\{x,\lambda\}^3 \qquad(x,\lambda\in\Omega). \tag*{(91)} ξ ( x , λ ) = 2 1 min { x , λ } 2 max { x , λ } − 6 1 min { x , λ } 3 ( x , λ ∈ Ω ) . (91)
This coincides with Green’s function corresponding to the 4th order differential operator [61 ] and hence, the kernel slice generator λ ↦ ξ λ \lambda\mapsto\xi_\lambda λ ↦ ξ λ is twice continuously differentiable on Ω \Omega Ω . Note that in our example, the test and ansatz space are the same since our analytic solution (88) satisfies G ∈ H \mathbf{G} \in\mathcal{H} G ∈ H . For the proposed method this is a necessity, since we are looking to obtain a weak solution G r ∈ H \mathbf{G}_r \in\mathcal{H} G r ∈ H which minimizes ∥ G − G r ∥ H \lVert\mathbf{G}-\mathbf{G}_r\rVert_{\mathcal{H}} ∥ G − G r ∥ H .
Let us assume for now, that we have given kernel centers λ ~ ∈ Ω + r \widetilde{\boldsymbol{\lambda}} \in\Omega_+^r λ ∈ Ω + r for some r ∈ N r \in\mathbb{N} r ∈ N . We can approximate the unknown solution G \mathbf{G} G with the orthogonal (Galerkin) projection G r : = P V λ ~ r [ G ] \mathbf{G}_r := P_{\mathcal{V}_{\widetilde{\boldsymbol{\lambda}}}^r}[\mathbf{G}] G r := P V λ r [ G ] , where V λ ~ r = span { ξ λ ~ 0 , … , ξ λ ~ r − 1 } ⊆ H \mathcal{V}_{\widetilde{\boldsymbol{\lambda}}}^r = \operatorname{span}\{\xi_{\widetilde{\lambda}_0},\ldots,\xi_{\widetilde{\lambda}_{r-1}}\} \subseteq\mathcal{H} V λ r = span { ξ λ 0 , … , ξ λ r − 1 } ⊆ H . In fact, G r \mathbf{G}_r G r can be computed using (90) as follows. Choosing the kernel slices ξ λ ~ j \xi_{\widetilde{\lambda}_j} ξ λ j , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 as test functions and since G r ( x ) = ∑ k = 0 r − 1 r ~ k ξ λ ~ k ( x ) \mathbf{G}_r(x)=\sum_{k=0}^{r-1}\widetilde{r}_k\xi_{\widetilde{\lambda}_k}(x) G r ( x ) = ∑ k = 0 r − 1 r k ξ λ k ( x ) for some linear coefficients r ~ k \widetilde{r}_k r k , k = 0 , … , r − 1 k=0,\ldots,r-1 k = 0 , … , r − 1 , we obtain
∫ Ω G r ′ ′ ( x ) ξ λ ~ j ′ ′ ( x ) d x = ∫ Ω ( ∑ k = 0 r − 1 r ~ k ξ λ ~ k ′ ′ ( x ) ) ξ λ ~ j ′ ′ ( x ) d x = ∑ k = 0 r − 1 r ~ k ∫ Ω ξ λ ~ k ′ ′ ( x ) ξ λ ~ j ′ ′ ( x ) d x = ∫ Ω F ( x ) ξ λ ~ j ( x ) d x . \begin{aligned}
\int_{\Omega}G_r''(x)\xi_{\widetilde{\lambda}_j}''(x)\,\mathrm{d}x
&=\int_{\Omega}\left(\sum_{k=0}^{r-1}\widetilde{r}_k\xi_{\widetilde{\lambda}_k}''(x)\right)\xi_{\widetilde{\lambda}_j}''(x)\,\mathrm{d}x\\
&=\sum_{k=0}^{r-1}\widetilde{r}_k\int_{\Omega}\xi_{\widetilde{\lambda}_k}''(x)\xi_{\widetilde{\lambda}_j}''(x)\,\mathrm{d}x
=\int_{\Omega}F(x)\xi_{\widetilde{\lambda}_j}(x)\,\mathrm{d}x.
\end{aligned} ∫ Ω G r ′′ ( x ) ξ λ j ′′ ( x ) d x = ∫ Ω ( k = 0 ∑ r − 1 r k ξ λ k ′′ ( x ) ) ξ λ j ′′ ( x ) d x = k = 0 ∑ r − 1 r k ∫ Ω ξ λ k ′′ ( x ) ξ λ j ′′ ( x ) d x = ∫ Ω F ( x ) ξ λ j ( x ) d x .
Then, r ~ : = [ x ~ 0 , … , x ~ r − 1 ] T \widetilde{\mathbf{r}} := [\widetilde{x}_0,\ldots,\widetilde{x}_{r-1}]^{\mathsf{T}} r := [ x 0 , … , x r − 1 ] T is obtained as the solution to the linear system
A r ~ = f , A\widetilde{\mathbf{r}} = \mathbf{f}, A r = f ,
where A : = [ a ( ξ ~ k , ξ ~ j ) ] k , j = 0 r − 1 A := \left[a(\widetilde{\xi}_{k},\widetilde{\xi}_{j})\right]_{k,j=0}^{r-1} A := [ a ( ξ k , ξ j ) ] k , j = 0 r − 1 and f : = [ f ( ξ ~ j ) ] j = 0 r − 1 \mathbf{f} := \left[f(\widetilde{\xi}_{j})\right]_{j=0}^{r-1} f := [ f ( ξ j ) ] j = 0 r − 1 . We can compute the entries of A A A exactly using the identity
ξ λ ′ ′ ( x ) = { λ − x , if x ≤ λ , 0 , if x > λ . \xi_{\lambda}^{\prime\prime}(x) =
\begin{cases}
\lambda-x, & \text{if }x \le\lambda,\\
0, & \text{if }x > \lambda.
\end{cases} ξ λ ′′ ( x ) = { λ − x , 0 , if x ≤ λ , if x > λ .
The components of f \mathbf{f} f can be approximated using an appropriate numerical quadrature. In this experiment, we use the composite trapezoidal rule with N = 1000 N=1000 N = 1000 nodes.
Note that the bilinear form a ( ⋅ , ⋅ ) a(\cdot,\cdot) a ( ⋅ , ⋅ ) on the left hand side of (90) can be interpreted as an inner product ⟨ ⋅ , ⋅ ⟩ H \langle\cdot,\cdot\rangle_{\mathcal{H}} ⟨ ⋅ , ⋅ ⟩ H . In particular, a ( ⋅ , ⋅ ) a(\cdot,\cdot) a ( ⋅ , ⋅ ) is coercive with coercivity constant 1, that is,
a ( H , H ) = ∫ Ω H ′ ′ ( x ) H ′ ′ ( x ) d x ≥ ∥ H ∥ H 2 ( H ∈ H ) . a(\mathbf{H},\mathbf{H}) = \int_{\Omega} \mathbf{H}^{\prime\prime}(x)\mathbf{H}^{\prime\prime}(x)\,\mathrm{d}x \ge\lVert\mathbf{H}\rVert_{\mathcal{H}}^2 \quad(\mathbf{H}\in\mathcal{H}). a ( H , H ) = ∫ Ω H ′′ ( x ) H ′′ ( x ) d x ≥ ∥ H ∥ H 2 ( H ∈ H ) .
Moreover, by the Cauchy–Schwarz inequality, the continuity constant of a ( ⋅ , ⋅ ) a(\cdot,\cdot) a ( ⋅ , ⋅ ) is also equal to one.
The Galerkin solution satisfies the Galerkin orthogonality
a ( G − G r , H ) = ⟨ G − G r , H ⟩ H = 0 ( H ∈ V ~ r λ ) . a(\mathbf{G}-\mathbf{G}_r,\mathbf{H}) = \langle\mathbf{G}-\mathbf{G}_r,\mathbf{H}\rangle_{\mathcal{H}} = 0 \quad\left(\mathbf{H}\in\widetilde{V}_r^{\lambda}\right). a ( G − G r , H ) = ⟨ G − G r , H ⟩ H = 0 ( H ∈ V r λ ) .
Hence, the Galerkin solution coincides with the orthogonal projection G r = P V ~ r λ [ G ] \mathbf{G}_r=\mathcal{P}_{\widetilde{V}_r^{\lambda}}[\mathbf{G}] G r = P V r λ [ G ] . Furthermore, since both the continuity and coercivity constants are equal to one, Céa’s lemma gives
∥ G − G r ∥ H = inf H ∈ V ~ r λ ∥ G − H ∥ H . \lVert\mathbf{G}-\mathbf{G}_r\rVert_{\mathcal{H}} = \inf_{\mathbf{H}\in\widetilde{V}_r^{\lambda}}\lVert\mathbf{G}-\mathbf{H}\rVert_{\mathcal{H}}. ∥ G − G r ∥ H = H ∈ V r λ inf ∥ G − H ∥ H .
Thus, in this setting, the Galerkin solution is exactly the H \mathcal{H} H –orthogonal best approximation of G \mathbf{G} G in V ~ r λ \widetilde{V}_r^{\lambda} V r λ .
Up to this point, we established a FEM –like approach, where we approximate the true solution of (90) using kernel slices instead of the usual piecewise polynomial basis. Such an approach is not without precedent, see, e.g., [71 ] . The novelty of the example comes from adaptively choosing the "mesh" defined by the r r r kernel centers λ ~ ∈ Ω r r \widetilde{\lambda}\in\Omega_r^{r} λ ∈ Ω r r using Algorithm 6.1. That is, we interpret the solution G \mathbf{G} G and its approximation G r ∈ V ~ r λ \mathbf{G}_r\in\widetilde{V}_r^{\lambda} G r ∈ V r λ as a FOM and a ROM , respectively.
In contrast to a usual MOR problem, we do not have access to the kernel centers defining G \mathbf{G} G . However, to apply Algorithm 6.1, we only need to be able to evaluate the equilibrium function in (34) (and its derivative due to our choice of Newton iteration), and no explicit knowledge of G \mathbf{G} G is required. However, evaluating the equilibrium function D : Ω r → C r \mathcal{D}:\Omega^r\to\mathbb{C}^r D : Ω r → C r and determining an optimal λ ~ \widetilde{\lambda} λ using Algorithm 6.1 requires access to G ′ ( x ) \mathbf{G}'(x) G ′ ( x ) and G ′ ′ ( x ) \mathbf{G}''(x) G ′′ ( x ) for all x ∈ Ω x\in\Omega x ∈ Ω , since by (34) ,
D ( λ ~ ) = [ G ′ ( λ ~ 0 ) − P V ~ r λ [ G ′ ] ( λ ~ 0 ) , … , G ′ ( λ ~ r − 1 ) − P V ~ r λ [ G ′ ] ( λ ~ r − 1 ) ] T . \mathcal{D}(\widetilde{\lambda}) =
\left[
\mathbf{G}'(\widetilde{\lambda}_0)-\mathcal{P}_{\widetilde{V}_r^{\lambda}}[\mathbf{G}'](\widetilde{\lambda}_0),\ldots,
\mathbf{G}'(\widetilde{\lambda}_{r-1})-\mathcal{P}_{\widetilde{V}_r^{\lambda}}[\mathbf{G}'](\widetilde{\lambda}_{r-1})
\right]^{\mathsf{T}}. D ( λ ) = [ G ′ ( λ 0 ) − P V r λ [ G ′ ] ( λ 0 ) , … , G ′ ( λ r − 1 ) − P V r λ [ G ′ ] ( λ r − 1 ) ] T .
While we cannot directly compute these values from (86) or (90) , we can leverage the method of dual weighted residuals (DWR) introduced in [7 ] . We refer to [7 ] for a thorough discussion of DWR, here we only state the following consequence.
Lemma 7.6 (Equilibrium function evaluation). Consider the weak problem (90) and let H \mathcal{H} H be the RKHS defined in (89) with kernel ξ \xi ξ from (91) . For a given λ ~ ∈ Ω r + \tilde{\lambda} \in\Omega_r^+ λ ~ ∈ Ω r + , let
G r = P V r λ ~ [ G ] = ∑ j = 0 r − 1 c j ξ ~ λ ~ j G_r = \mathcal{P}_{V_r^{\tilde{\lambda}}}[G] = \sum_{j=0}^{r-1} c_j \tilde{\xi}_{\tilde{\lambda}_j} G r = P V r λ ~ [ G ] = j = 0 ∑ r − 1 c j ξ ~ λ ~ j be the Galerkin projection of the true solution G \mathbf{G} G onto the V r λ ~ = span { ξ ~ λ ~ 0 , … , ξ ~ λ ~ r − 1 } ⊆ H V_r^{\tilde{\lambda}} = \operatorname{span}\left\{\tilde{\xi}_{\tilde{\lambda}_0},\ldots,\tilde{\xi}_{\tilde{\lambda}_{r-1}}\right\} \subseteq\mathcal{H} V r λ ~ = span { ξ ~ λ ~ 0 , … , ξ ~ λ ~ r − 1 } ⊆ H . Let λ ~ ∈ Ω r + \tilde{\lambda} \in\Omega_r^+ λ ~ ∈ Ω r + . Then,*
[ D ( λ ~ ) ] j = G ′ ( λ ~ j ) − G r ′ ( λ ~ j ) = 1 2 ∫ 0 λ ~ j F ( x ) x 2 d x + ∫ λ ~ j 1 F ( x ) ( λ ~ j x − 1 2 λ ~ j 2 ) d x − G r ′ ( λ ~ j ) , \left[\mathcal{D}(\tilde{\lambda})\right]_j = G'(\tilde{\lambda}_j)-G'_r(\tilde{\lambda}_j)
= \frac{1}{2}\int_0^{\tilde{\lambda}_j} F(x)x^2\,\mathrm{d}x + \int_{\tilde{\lambda}_j}^{1} F(x)\left(\tilde{\lambda}_j x-\frac{1}{2}\tilde{\lambda}_j^2\right)\,\mathrm{d}x-G'_r(\tilde{\lambda}_j), [ D ( λ ~ ) ] j = G ′ ( λ ~ j ) − G r ′ ( λ ~ j ) = 2 1 ∫ 0 λ ~ j F ( x ) x 2 d x + ∫ λ ~ j 1 F ( x ) ( λ ~ j x − 2 1 λ ~ j 2 ) d x − G r ′ ( λ ~ j ) , and
[ ∂ D ∂ λ ~ i ( x ) ] j = { − D λ ~ i G r ′ ( λ ~ j ) , if i ≠ j , G ′ ′ ( λ ~ j ) − G r ′ ′ ( λ ~ j ) = ∫ λ ~ j 1 F ( x ) ( x − λ ~ j ) d x − G r ′ ′ ( λ ~ j ) , if i = j , \left[\frac{\partial\mathcal{D}}{\partial\tilde{\lambda}_i}(x)\right]_j =
\begin{cases}
-D_{\tilde{\lambda}_i}G'_r(\tilde{\lambda}_j), & \text{if } i\ne j,\\
G''(\tilde{\lambda}_j)-G''_r(\tilde{\lambda}_j)=\int_{\tilde{\lambda}_j}^{1}F(x)(x-\tilde{\lambda}_j)\,\mathrm{d}x-G''_r(\tilde{\lambda}_j), & \text{if } i=j,
\end{cases} [ ∂ λ ~ i ∂ D ( x ) ] j = { − D λ ~ i G r ′ ( λ ~ j ) , G ′′ ( λ ~ j ) − G r ′′ ( λ ~ j ) = ∫ λ ~ j 1 F ( x ) ( x − λ ~ j ) d x − G r ′′ ( λ ~ j ) , if i = j , if i = j , where i , j = 0 , … , r − 1 i,j=0,\ldots,r-1 i , j = 0 , … , r − 1 .
Proof. The proof is given in Section A.6.
Using the formulas in Lemma 7.6 , we can apply Algorithm 6.1 to determine the r r r kernel centers λ ~ \tilde{\lambda} λ ~ according to the optimality conditions of § 4 . In this way, the points λ ~ ∈ Ω r + \tilde{\lambda}\in\Omega_r^+ λ ~ ∈ Ω r + that define G r G_r G r can be interpreted as an “optimal interior mesh". Finally, we note that the integrals in Lemma 7.6 can be approximated using numerical quadratures.
Using the right–hand side in (86) and the boundary conditions in (87) , we now proceed to examine the performance of the proposed method. The proposed adaptive algorithm is given in Algorithm 7.2. In the final step, we additionally include the kernel slice ξ 1 \xi_1 ξ 1 in the reduced space, i.e.,
V r + 1 ( λ ~ , 1 ) : = span { ξ ~ λ ~ 0 , … , ξ ~ λ ~ r − 1 , ξ 1 } . (92) V_{r+1}^{(\tilde{\lambda},1)} := \operatorname{span}\left\{\tilde{\xi}_{\tilde{\lambda}_0},\ldots,\tilde{\xi}_{\tilde{\lambda}_{r-1}},\xi_1\right\}. \tag*{(92)} V r + 1 ( λ ~ , 1 ) := span { ξ ~ λ ~ 0 , … , ξ ~ λ ~ r − 1 , ξ 1 } . (92)
By Theorem 5.12 , the orthogonal projection onto V r + 1 ( λ ~ , 1 ) V_{r+1}^{(\tilde{\lambda},1)} V r + 1 ( λ ~ , 1 ) interpolates the target solution at the kernel centers. Hence, including ξ 1 \xi_1 ξ 1 ensures interpolation of the exact solution at x = 1 x=1 x = 1 , which improves the approximation near the right boundary. Throughout this experiment, however, r r r refers to the number of adaptive interior kernel centers, and we denote the final approximation by G r G_r G r . No additional kernel slice at x = 0 x=0 x = 0 is required, since ξ 0 ≡ 0 \xi_0\equiv0 ξ 0 ≡ 0 and the
Algorithm 7.2 RKHS–PDE solver for (86) , (87) with H \mathcal{H} H –optimal mesh
Inputs: Right–hand side F \mathbf{F} F of (86) , kernel ξ \xi ξ of the RKHS H \mathcal{H} H , kernel centers λ ~ ( 0 ) ∈ Ω ξ r \tilde{\lambda}^{(0)} \in\Omega^r_{\xi} λ ~ ( 0 ) ∈ Ω ξ r , damping factor of Newton’s method β ∈ ( 0 , 1 ] \beta\in(0,1] β ∈ ( 0 , 1 ] .
Outputs: H \mathcal{H} H –optimal approximate solution G r ∈ G ξ r \mathbf{G}_r \in\mathbb{G}^r_{\xi} G r ∈ G ξ r .
m ← 0 m \leftarrow0 m ← 0 .
ε ( 0 ) ← ∞ \varepsilon^{(0)} \leftarrow\infty ε ( 0 ) ← ∞ .
while ε ( m ) > ε \varepsilon^{(m)} > \varepsilon ε ( m ) > ε do
J ← [ ∂ D ( λ ~ ( m ) ) i ∂ λ ~ j ] i , j = 0 r − 1 \mathbf{J} \leftarrow\left[\frac{\partial\mathcal{D}(\tilde{\lambda}^{(m)})_i}{\partial\tilde{\lambda}_j}\right]_{i,j=0}^{r-1} J ← [ ∂ λ ~ j ∂ D ( λ ~ ( m ) ) i ] i , j = 0 r − 1 . ▷ Use Lemma 7.6 and F \mathbf{F} F .
λ ~ ( m + 1 ) ← λ ~ ( m ) − β J − 1 D ( λ ~ ( m ) ) \tilde{\lambda}^{(m+1)} \leftarrow\tilde{\lambda}^{(m)}-\beta\mathbf{J}^{-1}\mathcal{D}\left(\tilde{\lambda}^{(m)}\right) λ ~ ( m + 1 ) ← λ ~ ( m ) − β J − 1 D ( λ ~ ( m ) ) .
ε ( m ) ← ∥ λ ~ ( m + 1 ) − λ ~ ( m ) ∥ ∞ \varepsilon^{(m)} \leftarrow\left\|\tilde{\lambda}^{(m+1)}-\tilde{\lambda}^{(m)}\right\|_{\infty} ε ( m ) ← λ ~ ( m + 1 ) − λ ~ ( m ) ∞ .
m ← m + 1 m \leftarrow m+1 m ← m + 1 .
end while
G r ← P V r r + 1 ( λ ~ ( m + 1 ) ) [ G ] \mathbf{G}_r \leftarrow\mathcal{P}_{V^{r+1}_{r}(\tilde{\lambda}^{(m+1)})}[\mathbf{G}] G r ← P V r r + 1 ( λ ~ ( m + 1 ) ) [ G ] . ▷ Projection to (92) .
return G r \mathbf{G}_r G r .
essential boundary conditions at x = 0 x=0 x = 0 are already incorporated in the definition of H \mathcal{H} H . In this experiment we use r = 5 r=5 r = 5 . We note again that the corresponding orthogonal projection can be computed without explicit access to G \mathbf{G} G by solving the Galerkin system associated with (90) on V r r + 1 ( λ ~ ( 1 ) ) V^{r+1}_{r}(\tilde{\lambda}^{(1)}) V r r + 1 ( λ ~ ( 1 ) ) .
For comparison, the FEM meshes are constructed using the same number r r r of interior mesh points together with the two boundary points x = 0 x=0 x = 0 and x = 1 x=1 x = 1 . In addition, we solve (90) using traditional FEM with cubic Hermite elements. In particular, we consider three baseline solutions:
G r F E M \mathbf{G}^{\mathrm{FEM}}_r G r FEM : FEM solution generated using r = 5 r=5 r = 5 interior mesh points together with the two boundary points x = 0 x=0 x = 0 and x = 1 x=1 x = 1 , resulting in an equidistant mesh with r + 2 = 7 r+2=7 r + 2 = 7 points.
G r F E M - a d a p t \mathbf{G}^{\mathrm{FEM\text{-}adapt}}_r G r FEM - adapt : FEM solution generated using r = 5 r=5 r = 5 interior mesh points concentrated in [ 0.6 , 0.9 ] ⊆ Ω [0.6,0.9]\subseteq\Omega [ 0.6 , 0.9 ] ⊆ Ω , which contains the main lobe of the true solution (88) (see also Figure 7 ). The two boundary points x = 0 x=0 x = 0 and x = 1 x=1 x = 1 are additionally included to ensure compliance with (87) , resulting in a mesh with r + 2 = 7 r+2=7 r + 2 = 7 points.
G 2 r F E M \mathbf{G}^{\mathrm{FEM}}_{2r} G 2 r FEM : FEM solution generated using 2 r = 10 2r=10 2 r = 10 interior mesh points together with the two boundary points, resulting in an equidistant mesh with 2 r + 2 = 12 2r+2=12 2 r + 2 = 12 points.
Figure 7. Solution to the 4th–order Euler beam equation using the proposed adaptive kernel mesh based approach and FEM for r = 5 r = 5 r = 5 .
The results of this experiment are illustrated in Figure 7 . We evaluate each solution in terms of the relative L 2 ( Ω ) L^2(\Omega) L 2 ( Ω ) error defined as
E L 2 ( Ω ) ( G r ) = ∥ G − G r ∥ L 2 ( Ω ) ∥ G ∥ L 2 ( Ω ) ≈ ∑ k = 0 N − 1 ∣ G ( x k ) − G r ( x k ) ∣ 2 ∑ k = 0 N − 1 ∣ G ( x k ) ∣ 2 \mathcal{E}_{L^2(\Omega)}(G_r) = \frac{\lVert G - G_r \rVert_{L^2(\Omega)}}{\lVert G \rVert_{L^2(\Omega)}} \approx\frac{\sqrt{\sum_{k=0}^{N-1} \lvert G(x_k)-G_r(x_k)\rvert^2}}{\sqrt{\sum_{k=0}^{N-1} \lvert G(x_k)\rvert^2}} E L 2 ( Ω ) ( G r ) = ∥ G ∥ L 2 ( Ω ) ∥ G − G r ∥ L 2 ( Ω ) ≈ ∑ k = 0 N − 1 ∣ G ( x k ) ∣ 2 ∑ k = 0 N − 1 ∣ G ( x k ) − G r ( x k ) ∣ 2
where for this experiment, we use an equidistant partitioning of Ω \Omega Ω with N = 1000 N = 1000 N = 1000 points.
Figure 7 demonstrates the utility of the proposed method for the considered problem (86) . The solution obtained with Algorithm 7.2 significantly outperforms the FEM-based solutions for r = 5 r = 5 r = 5 in terms of relative L 2 ( Ω ) L^2(\Omega) L 2 ( Ω ) error. In fact, as observed in the figure, FEM achieves an error magnitude similar to that of the proposed approach only for 2 r = 10 2r = 10 2 r = 10 elements. Notably, to find the solution G r G_r G r , we choose β = 0.1 \beta= 0.1 β = 0.1 in Algorithm 7.2 and initialize the kernel centers equidistantly λ ~ j ( 0 ) ∈ [ 0.6 , 0.9 ] \tilde{\lambda}_j^{(0)} \in[0.6, 0.9] λ ~ j ( 0 ) ∈ [ 0.6 , 0.9 ] for j = 0 , … , r − 1 j = 0, \ldots, r - 1 j = 0 , … , r − 1 . Since the true solution G \mathbf{G} G vanishes quickly as we approach the boundary, failing to choose these hyperparameters leads to vanishing gradients: the iteration still converges, but considerably more slowly, since the equilibrium function and its Jacobian become extremely small away from the main lobe of G \mathbf{G} G . We expect the severity of this sensitivity to depend strongly on the particular PDE/ODE under consideration; a systematic study of initialization strategies is beyond the scope of this proof–of–concept example and is left for future, application–specific work. The total runtime of the proposed Algorithm was 8.2 seconds for a total of 200 iterations. This gives a runtime of 0.04 seconds per iteration. The number of iterations can be further decreased with better initialization.
This experiment showed the potential of using iterative PDE solvers that enforce RKHS–based optimality conditions on the approximate solution. A separate study should be conducted to define a class of problems for which the proposed approach is applicable, and to develop numerically improved schemes. We note that RKHS–based projections have already been considered for solving PDEs in [62 , 71 ] , however, these works did not use the optimality conditions considered in this study.
Conclusions
In this work, we considered the problem of norm–optimal MOR in RKHSs. More precisely, we defined generalized model spaces and weak model spaces in RKHSs and introduced necessary conditions for optimal approximation between them. Our conditions generalize earlier results for MOR in H 2 H_2 H 2 Hardy–Hilbert spaces. The introduced conditions can only be applied if the kernel of the RKHS is differentiable. In order to extend our methods to general RKHS, we introduced the generalized Malmquist–Takenaka system. This orthogonal function system generalizes several earlier variants, such as the classical Malmquist–Takenaka system and Newton bases defined in RKHSs. Using the GMT system, we introduced the GMT transform, which provides a unitary map between any two separable RKHSs. Using these results, we developed several MOR methods and compared their performance on example applications.
We showed that in addition to solving MOR tasks in RKHSs, the proposed methods are also applicable to a variety of other problems. For example, in an academic example, we showed how fitting high–order polynomials to data can be interpreted as a MOR task. We showed how MOR can be applied to kernel methods such as support vector machines and provided several useful lemmas to clarify the relationship between FOM and ROM–SVM classifiers. We verified that we can apply the GMT–based methodology to scalar–input scalar–output deep ReLU networks. Finally, we showed how a kernel basis discretization of certain PDEs can be interpreted as norm–optimal MOR problem in Sobolev RKHSs. In this context, our algorithms can be interpreted as an adaptive mesh FEM–like approach that relies on a kernel slice basis instead of piecewise polynomials, with the additional benefit of the chosen "mesh" being determined according to necessary conditions of optimality. Our results and experiments lay the foundation for further theoretical investigation as well as the thorough development of real–life applications. First, our optimality conditions and subsequent MOR algorithms should be extended to RKHSs defined over high–dimensional complex manifolds. This would allow the reduction of large machine learning and deep learning models. In addition, it would open the door toward the development of PDE applications.
The asymptotic behavior and computational aspects of GMT functions and the GMT transformation should be further investigated. Generalized Blaschke–Szász conditions for a large class of RKHSs should be obtained to generate complete GMT systems. A further open direction is to characterize the class of RKHSs for which weak and strong model spaces coincide under the GMT transformation. Regarding computational aspects, kernel–sensitive variants of the Gram–Schmidt method should be developed. For the classical rational MT functions, discrete orthogonal and fast FFT–based methods are already available for stable computation [21 ] . Possible generalizations of these methods to RKHSs should be investigated.
Finally, the examples presented in this study were meant to showcase the versatility of the proposed MOR framework. The list of considered applications should be extended, for example, to obtain ROMs of nonlinear dynamical systems. In addition, the presented examples should be extended into full–fledged applications in their own right.
Appendix
Equivalence of Theorems 3.10 and 3.11
Theorem A.1 (Equivalence of optimality conditions in H 2 ( C + ) H_2(\mathbb{C}^{+}) H 2 ( C + ) ). Under the assumptions in Theorem 3.10 , {(3.8)} is equivalent to {(3.9)}.
Proof. From {(3.6)} and {(3.7)}, G \mathbf{G} G and G r \mathbf{G}_r G r consist of finite linear combinations of kernel slices. We use the reproducing property to obtain
⟨ G − G r , H 2 ⟩ H 2 ( C + ) = ⟨ G − G r , ∑ j = 0 r − 1 c j ζ ~ λ j ⟩ H 2 ( C + ) = ∑ j = 0 r − 1 c j ‾ ⟨ G , ζ ~ λ j ⟩ H 2 ( C + ) − ∑ j = 0 r − 1 c j ‾ ⟨ G r , ζ ~ λ j ⟩ H 2 ( C + ) = ∑ j = 0 r − 1 c j ‾ G ( λ ~ j ) − ∑ j = 0 r − 1 c j ‾ G r ( λ ~ j ) . \begin{aligned}
\langle\mathbf{G}-\mathbf{G}_r,\mathbf{H}_2\rangle_{H_2(\mathbb{C}^{+})}
&=\left\langle\mathbf{G}-\mathbf{G}_r,\sum_{j=0}^{r-1}c_j\widetilde{\zeta}_{\lambda_j}\right\rangle_{H_2(\mathbb{C}^{+})}\\
&=\sum_{j=0}^{r-1}\overline{c_j}\langle\mathbf{G},\widetilde{\zeta}_{\lambda_j}\rangle_{H_2(\mathbb{C}^{+})}
-\sum_{j=0}^{r-1}\overline{c_j}\langle\mathbf{G}_r,\widetilde{\zeta}_{\lambda_j}\rangle_{H_2(\mathbb{C}^{+})}\\
&=\sum_{j=0}^{r-1}\overline{c_j}\mathbf{G}(\widetilde{\lambda}_j)
-\sum_{j=0}^{r-1}\overline{c_j}\mathbf{G}_r(\widetilde{\lambda}_j).
\end{aligned} ⟨ G − G r , H 2 ⟩ H 2 ( C + ) = ⟨ G − G r , j = 0 ∑ r − 1 c j ζ λ j ⟩ H 2 ( C + ) = j = 0 ∑ r − 1 c j ⟨ G , ζ λ j ⟩ H 2 ( C + ) − j = 0 ∑ r − 1 c j ⟨ G r , ζ λ j ⟩ H 2 ( C + ) = j = 0 ∑ r − 1 c j G ( λ j ) − j = 0 ∑ r − 1 c j G r ( λ j ) .
Hence, we have
⟨ G − G r , H 2 ⟩ H 2 ( C + ) = 0 ( H 2 ∈ G r ) ⟺ G ( λ ~ j ) = G r ( λ ~ j ) ( j = 0 , … , r − 1 ) . (93) \langle\mathbf{G}-\mathbf{G}_r,\mathbf{H}_2\rangle_{H_2(\mathbb{C}^{+})}=0\quad(\mathbf{H}_2\in\mathfrak{G}_r)\Longleftrightarrow\mathbf{G}(\widetilde{\lambda}_j)=\mathbf{G}_r(\widetilde{\lambda}_j)\quad(j=0,\ldots,r-1). \tag*{(93)} ⟨ G − G r , H 2 ⟩ H 2 ( C + ) = 0 ( H 2 ∈ G r ) ⟺ G ( λ j ) = G r ( λ j ) ( j = 0 , … , r − 1 ) . (93)
Notice that the derivative D λ ˉ ζ λ D_{\bar{\lambda}}\zeta_{\lambda} D λ ˉ ζ λ satisfies
D λ ˉ ζ λ ( s ) = − 1 ( s + λ ˉ ) 2 . D_{\bar{\lambda}}\zeta_{\lambda}(s)=-\frac{1}{(s+\bar{\lambda})^2}. D λ ˉ ζ λ ( s ) = − ( s + λ ˉ ) 2 1 .
Since G r G_r G r is a ROM of order r r r and by choosing H 1 = ζ λ ~ j \mathbf{H}_1=\zeta_{\tilde{\lambda}_j} H 1 = ζ λ ~ j , j ∈ { 0 , … , r − 1 } j\in\{0,\ldots,r-1\} j ∈ { 0 , … , r − 1 } , we obtain
⟨ G − G r , G r ⋅ H 1 ⟩ H 2 ( C + ) = ⟨ G − G r , − r ~ j D λ ˉ ζ λ ~ j + H 3 ⟩ H 2 ( C + ) = ⟨ G − G r , − r ~ j D λ ˉ ζ λ ~ j ⟩ H 2 ( C + ) + ⟨ G − G r , H 3 ⟩ H 2 ( C + ) . \begin{aligned}
\left\langle G-G_r,G_r\cdot\mathbf{H}_1\right\rangle_{H_2(\mathbb{C}^+)}
=\left\langle G-G_r,-\tilde{r}_jD_{\bar{\lambda}}\zeta_{\tilde{\lambda}_j}+\mathbf{H}_3\right\rangle_{H_2(\mathbb{C}^+)} \\
&=\left\langle G-G_r,-\tilde{r}_jD_{\bar{\lambda}}\zeta_{\tilde{\lambda}_j}\right\rangle_{H_2(\mathbb{C}^+)}
+\left\langle G-G_r,\mathbf{H}_3\right\rangle_{H_2(\mathbb{C}^+)}.
\end{aligned} ⟨ G − G r , G r ⋅ H 1 ⟩ H 2 ( C + ) = ⟨ G − G r , − r ~ j D λ ˉ ζ λ ~ j + H 3 ⟩ H 2 ( C + ) = ⟨ G − G r , − r ~ j D λ ˉ ζ λ ~ j ⟩ H 2 ( C + ) + ⟨ G − G r , H 3 ⟩ H 2 ( C + ) .
for some H 3 ∈ span { ζ λ ~ 0 , … , ζ λ ~ r − 1 } \mathbf{H}_3\in\operatorname{span}\{\zeta_{\tilde{\lambda}_0},\ldots,\zeta_{\tilde{\lambda}_{r-1}}\} H 3 ∈ span { ζ λ ~ 0 , … , ζ λ ~ r − 1 } and r ~ j ≠ 0 \tilde{r}_j\ne0 r ~ j = 0 . Here, we exploit the fact that, by using a partial fraction decomposition, the terms ζ λ ~ i ζ λ ~ j ∈ span { ζ λ ~ i , ζ λ ~ j } \zeta_{\tilde{\lambda}_i}\zeta_{\tilde{\lambda}_j}\in\operatorname{span}\{\zeta_{\tilde{\lambda}_i},\zeta_{\tilde{\lambda}_j}\} ζ λ ~ i ζ λ ~ j ∈ span { ζ λ ~ i , ζ λ ~ j } for i , j = 0 , … , r − 1 i,j=0,\ldots,r-1 i , j = 0 , … , r − 1 with i ≠ j i\ne j i = j .
By the equivalence in (A.1), it remains to show that for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 ,
⟨ G − G r , D λ ˉ ζ λ ~ j ⟩ H 2 ( C + ) = 0 ⟺ G ′ ( λ ~ j ) − G r ′ ( λ ~ j ) . \left\langle G-G_r,D_{\bar{\lambda}}\zeta_{\tilde{\lambda}_j}\right\rangle_{H_2(\mathbb{C}^+)}
=0
\Longleftrightarrow
G'(\tilde{\lambda}_j)-G_r'(\tilde{\lambda}_j). ⟨ G − G r , D λ ˉ ζ λ ~ j ⟩ H 2 ( C + ) = 0 ⟺ G ′ ( λ ~ j ) − G r ′ ( λ ~ j ) .
However, this condition is satisfied by Theorem 3.2 .
Proof of Lemma 4.7
Proof. Theorem 4.3 and Theorem 4.4 necessarily apply to G r G_r G r , because it is a norm–optimal ROM in an RKHS. First, we show that the optimality conditions in Theorem 4.4 take the form (9) under the assumptions of the statement. Exploiting that λ ~ j \tilde{\lambda}_j λ ~ j are pairwise distinct points in C + \mathbb{C}^+ C + for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 , (24) is given by
G ( λ ~ j ) = G r ( λ ~ j ) and G ′ ( λ ~ j ) = G r ′ ( λ ~ j ) G(\tilde{\lambda}_j)=G_r(\tilde{\lambda}_j)
\qquad\text{and}\qquad
G'(\tilde{\lambda}_j)=G_r'(\tilde{\lambda}_j) G ( λ ~ j ) = G r ( λ ~ j ) and G ′ ( λ ~ j ) = G r ′ ( λ ~ j )
for j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 . This condition matches the statement of Theorem 3.10 exactly. In [31 ] (see also Section A.1), the equivalence between Theorem 3.10 and Theorem 3.11 is proven. On the other hand, by Theorem 4.6 , Theorem 4.3 and Theorem 4.4 are also equivalent. This implies that in the current setting Theorem 3.11 is a special case of Theorem 4.3 .
Proof of Lemma 7.1
Proof. The statement is a direct consequence of Lemma 3.1 and the Cauchy–Schwarz inequality. Indeed, for any x ∈ Ω x \in\Omega x ∈ Ω ,
∣ G ( x ) − G r ( x ) ∣ 2 = ∣ ⟨ G − G r , ξ x ⟩ P n ( Ω ) ∣ 2 ≤ ∥ G − G r ∥ P n ( Ω ) 2 ∥ ξ x ∥ P n ( Ω ) 2 = ∥ G − G r ∥ P n ( Ω ) 2 ξ ( x , x ) ≤ ∥ G − G r ∥ P n ( Ω ) 2 ∑ k = 0 n − 1 x 2 k ≤ n ∥ G − G r ∥ P n ( Ω ) 2 . \left|G(x)-G_r(x)\right|^2
= \left|\left\langle G-G_r,\xi_x\right\rangle_{P_n(\Omega)}\right|^2
\le\left\|G-G_r\right\|_{P_n(\Omega)}^2\left\|\xi_x\right\|_{P_n(\Omega)}^2
= \left\|G-G_r\right\|_{P_n(\Omega)}^2\xi(x,x)
\le\left\|G-G_r\right\|_{P_n(\Omega)}^2\sum_{k=0}^{n-1}x^{2k}
\le n\left\|G-G_r\right\|_{P_n(\Omega)}^2. ∣ G ( x ) − G r ( x ) ∣ 2 = ⟨ G − G r , ξ x ⟩ P n ( Ω ) 2 ≤ ∥ G − G r ∥ P n ( Ω ) 2 ∥ ξ x ∥ P n ( Ω ) 2 = ∥ G − G r ∥ P n ( Ω ) 2 ξ ( x , x ) ≤ ∥ G − G r ∥ P n ( Ω ) 2 k = 0 ∑ n − 1 x 2 k ≤ n ∥ G − G r ∥ P n ( Ω ) 2 .
Proof of Lemma 7.2
Proof. Let E ( z ) : = D ( z ) − D r ( z ) E(z) := D(z)-D_r(z) E ( z ) := D ( z ) − D r ( z ) . Then, D ( z ) = D r ( z ) + E ( z ) D(z)=D_r(z)+E(z) D ( z ) = D r ( z ) + E ( z ) .
Assume first that D r ( z ) > 0 D_r(z)>0 D r ( z ) > 0 . If ∣ D r ( z ) ∣ > ∣ E ( z ) ∣ \left|D_r(z)\right|>\left|E(z)\right| ∣ D r ( z ) ∣ > ∣ E ( z ) ∣ , then ∣ D r ( z ) ∣ = D r ( z ) \left|D_r(z)\right|=D_r(z) ∣ D r ( z ) ∣ = D r ( z ) , and therefore, E ( z ) > − ∣ E ( z ) ∣ > − D r ( z ) E(z)>-\left|E(z)\right|>-D_r(z) E ( z ) > − ∣ E ( z ) ∣ > − D r ( z ) . Hence, D ( z ) = D r ( z ) + E ( z ) > 0 D(z)=D_r(z)+E(z)>0 D ( z ) = D r ( z ) + E ( z ) > 0 . Thus, D ( z ) D(z) D ( z ) and D r ( z ) D_r(z) D r ( z ) have the same sign.
Assume now that D r ( z ) < 0 D_r(z)<0 D r ( z ) < 0 . Then ∣ D r ( z ) ∣ = − D r ( z ) \left|D_r(z)\right|=-D_r(z) ∣ D r ( z ) ∣ = − D r ( z ) . Since ∣ E ( z ) ∣ < ∣ D r ( z ) ∣ = − D r ( z ) \left|E(z)\right|<\left|D_r(z)\right|=-D_r(z) ∣ E ( z ) ∣ < ∣ D r ( z ) ∣ = − D r ( z ) , we have E ( z ) < ∣ E ( z ) ∣ < − D r ( z ) E(z)<\left|E(z)\right|<-D_r(z) E ( z ) < ∣ E ( z ) ∣ < − D r ( z ) . Therefore, D ( z ) = D r ( z ) + E ( z ) < 0 D(z)=D_r(z)+E(z)<0 D ( z ) = D r ( z ) + E ( z ) < 0 . Again, D ( z ) D(z) D ( z ) and D r ( z ) D_r(z) D r ( z ) have the same sign.
Thus, in both cases,
∣ D r ( z ) ∣ > ∣ D ( z ) − D r ( z ) ∣ ⟹ sgn ( D ( z ) ) = sgn ( D r ( z ) ) . \left|D_r(z)\right|>\left|D(z)-D_r(z)\right|
\Longrightarrow
\operatorname{sgn}(D(z))=\operatorname{sgn}(D_r(z)). ∣ D r ( z ) ∣ > ∣ D ( z ) − D r ( z ) ∣ ⟹ sgn ( D ( z )) = sgn ( D r ( z )) .
Proof of Lemma 7.4
Proof. Recall that D = G + b \mathbf{D}=\mathbf{G}+b D = G + b and D r = G r + b \mathbf{D}_r=\mathbf{G}_r+b D r = G r + b use the same bias. Therefore,
D − D r = G − G r . \mathbf{D}-\mathbf{D}_r=\mathbf{G}-\mathbf{G}_r. D − D r = G − G r .
Since G r = P V ~ r [ G ] \mathbf{G}_r=\mathcal{P}_{\widetilde{V}_r}[\mathbf{G}] G r = P V r [ G ] , the Hilbert projection theorem implies
⟨ G − G r , H ⟩ H = 0 ( H ∈ V ~ r ) . (94) \left\langle\mathbf{G}-\mathbf{G}_r,\mathbf{H}\right\rangle_{\mathcal{H}}=0
\qquad
\left(\mathbf{H}\in\widetilde{V}_r\right). \tag*{(94)} ⟨ G − G r , H ⟩ H = 0 ( H ∈ V r ) . (94)
Let E : = D − D r = G − G r ∈ H \mathbf{E}:=\mathbf{D}-\mathbf{D}_r=\mathbf{G}-\mathbf{G}_r\in\mathcal{H} E := D − D r = G − G r ∈ H , and notice that by the reproducing property E ( z ) = ⟨ E , ξ z ⟩ H E(z)=\left\langle E,\xi_z\right\rangle_{\mathcal{H}} E ( z ) = ⟨ E , ξ z ⟩ H holds for all z ∈ Ω z\in\Omega z ∈ Ω . The kernel slice ξ z \xi_z ξ z can be decomposed as
ξ z = P V ~ r [ ξ z ] + P ( V ~ r ) ⊥ [ ξ z ] . \xi_z=\mathcal{P}_{\widetilde{V}_r}[\xi_z]+\mathcal{P}_{(\widetilde{V}_r)^\perp}[\xi_z]. ξ z = P V r [ ξ z ] + P ( V r ) ⊥ [ ξ z ] .
Hence, we find that
E ( z ) = D ( z ) − D r ( z ) = G ( z ) − G r ( z ) = ⟨ E , P V ~ r [ ξ z ] ⟩ H + ⟨ E , P ( V ~ r ) ⊥ [ ξ z ] ⟩ H . E(z)=D(z)-D_r(z)=G(z)-G_r(z)
=\left\langle E,\mathcal{P}_{\widetilde{V}_r}[\xi_z]\right\rangle_{\mathcal{H}}
+\left\langle E,\mathcal{P}_{(\widetilde{V}_r)^\perp}[\xi_z]\right\rangle_{\mathcal{H}}. E ( z ) = D ( z ) − D r ( z ) = G ( z ) − G r ( z ) = ⟨ E , P V r [ ξ z ] ⟩ H + ⟨ E , P ( V r ) ⊥ [ ξ z ] ⟩ H .
By (A.2), the first term vanishes and thus,
E ( z ) = ⟨ E , P ( v r λ ) ⊥ [ ξ z ] ⟩ A . \mathrm{E}(z)=\left\langle\mathrm{E},\mathcal{P}_{(v_r^\lambda)}^\perp[\xi_z]\right\rangle_{\mathcal{A}}. E ( z ) = ⟨ E , P ( v r λ ) ⊥ [ ξ z ] ⟩ A .
Applying the Cauchy–Schwarz inequality to this gives
∣ E ( z ) ∣ ≤ ε ξ r ( D r ) ∥ P ( v r λ ) ⊥ [ ξ z ] ∥ H . (95) |\mathrm{E}(z)|\le\varepsilon_\xi^r(D_r)\left\|\mathcal{P}_{(v_r^\lambda)}^\perp[\xi_z]\right\|_{\mathcal{H}}. \tag*{(95)} ∣ E ( z ) ∣ ≤ ε ξ r ( D r ) P ( v r λ ) ⊥ [ ξ z ] H . (95)
Suppose D r ( z ) > ε ξ r ( D r ) ∥ P ( v r λ ) ⊥ [ ξ z ] ∥ H > 0 D_r(z)>\varepsilon_\xi^r(D_r)\left\|\mathcal{P}_{(v_r^\lambda)}^\perp[\xi_z]\right\|_{\mathcal{H}}>0 D r ( z ) > ε ξ r ( D r ) P ( v r λ ) ⊥ [ ξ z ] H > 0 . Then by (A.3),
D ( z ) = D r ( z ) + E ( z ) > D r ( z ) − ε ξ r ( D r ) ∥ P ( v r λ ) ⊥ [ ξ z ] ∥ H > 0 , \mathrm{D}(z)=\mathrm{D}_r(z)+\mathrm{E}(z)>\mathrm{D}_r(z)-\varepsilon_\xi^r(D_r)\left\|\mathcal{P}_{(v_r^\lambda)}^\perp[\xi_z]\right\|_{\mathcal{H}}>0, D ( z ) = D r ( z ) + E ( z ) > D r ( z ) − ε ξ r ( D r ) P ( v r λ ) ⊥ [ ξ z ] H > 0 ,
therefore D ( z ) > 0 \mathrm{D}(z)>0 D ( z ) > 0 and D r ( z ) > 0 \mathrm{D}_r(z)>0 D r ( z ) > 0 . Similarly, assuming
D r ( z ) < − ε ξ r ( D r ) ∥ P ( v r λ ) ⊥ [ ξ z ] ∥ H < 0 \mathrm{D}_r(z)<-\varepsilon_\xi^r(D_r)\left\|\mathcal{P}_{(v_r^\lambda)}^\perp[\xi_z]\right\|_{\mathcal{H}}<0 D r ( z ) < − ε ξ r ( D r ) P ( v r λ ) ⊥ [ ξ z ] H < 0
gives
D ( z ) = D r ( z ) + E ( z ) < D r ( z ) + ε ξ r ( D r ) ∥ P ( v r λ ) ⊥ [ ξ z ] ∥ H < 0 , \mathrm{D}(z)=\mathrm{D}_r(z)+\mathrm{E}(z)<\mathrm{D}_r(z)+\varepsilon_\xi^r(D_r)\left\|\mathcal{P}_{(v_r^\lambda)}^\perp[\xi_z]\right\|_{\mathcal{H}}<0, D ( z ) = D r ( z ) + E ( z ) < D r ( z ) + ε ξ r ( D r ) P ( v r λ ) ⊥ [ ξ z ] H < 0 ,
hence D ( z ) < 0 \mathrm{D}(z)<0 D ( z ) < 0 and D r ( z ) < 0 \mathrm{D}_r(z)<0 D r ( z ) < 0 .
Proof of thm:dwr
Proof of Lemma 7.6
Proof. For simplicity of notation, define the pointwise error
E ( x ) : = G ( x ) − G r ( x ) ( x ∈ Ω ) \mathrm{E}(x):=\mathrm{G}(x)-\mathrm{G}_r(x)\qquad(x\in\Omega) E ( x ) := G ( x ) − G r ( x ) ( x ∈ Ω )
and fix λ ∈ Ω \lambda\in\Omega λ ∈ Ω . Define the functionals H → H \mathcal{H}\to\mathcal{H} H → H by
K i ( H ) : = H ( i ) ( λ ) ( i = 1 , 2 ) . \mathcal{K}_i(\mathrm{H}):=\mathrm{H}^{(i)}(\lambda)\qquad(i=1,2). K i ( H ) := H ( i ) ( λ ) ( i = 1 , 2 ) .
Consider the problems of finding ϕ i ∈ H \phi_i\in\mathcal{H} ϕ i ∈ H such that
a ( ϕ i , H ) = K i ( H ) ( H ∈ H , i = 1 , 2 ) . (96) a(\phi_i,\mathrm{H})=\mathcal{K}_i(\mathrm{H})\qquad(\mathrm{H}\in\mathcal{H},i=1,2). \tag*{(96)} a ( ϕ i , H ) = K i ( H ) ( H ∈ H , i = 1 , 2 ) . (96)
Since K i \mathcal{K}_i K i are bounded linear functionals on H \mathcal{H} H , the Riesz representation theorem guarantees the existence of unique ϕ 1 , ϕ 2 ∈ H \phi_1,\phi_2\in\mathcal{H} ϕ 1 , ϕ 2 ∈ H that satisfies (A.4) for all H ∈ H \mathrm{H}\in\mathcal{H} H ∈ H . Exploiting that H \mathcal{H} H is an RKHS with kernel ξ \xi ξ and a ( ⋅ , ⋅ ) = ⟨ ⋅ , ⋅ ⟩ H a(\cdot,\cdot)=\langle\cdot,\cdot\rangle_{\mathcal{H}} a ( ⋅ , ⋅ ) = ⟨ ⋅ , ⋅ ⟩ H , we find
a ( ξ λ , H ) = a ( H , ξ λ ) = H ( λ ) ( H ∈ H ) . a(\xi_\lambda,\mathrm{H})=a(\mathrm{H},\xi_\lambda)=\mathrm{H}(\lambda)\qquad(\mathrm{H}\in\mathcal{H}). a ( ξ λ , H ) = a ( H , ξ λ ) = H ( λ ) ( H ∈ H ) .
Differentiating both sides and exploiting Theorem 3.2 gives
∂ ∂ λ a ( H , ξ λ ) = ∂ ∂ λ ⟨ H , ξ λ ⟩ H = ⟨ H , D λ ξ λ ⟩ H = a ( H , D λ 2 ξ λ ) = H ′ ( λ ) . \frac{\partial}{\partial\lambda}a(\mathbf{H},\xi_\lambda)=\frac{\partial}{\partial\lambda}\langle\mathbf{H},\xi_\lambda\rangle_{\mathbf{H}}=\langle\mathbf{H},D_\lambda\xi_\lambda\rangle_{\mathbf{H}}=a(\mathbf{H},D_\lambda^{2}\xi_\lambda)=\mathbf{H}'(\lambda). ∂ λ ∂ a ( H , ξ λ ) = ∂ λ ∂ ⟨ H , ξ λ ⟩ H = ⟨ H , D λ ξ λ ⟩ H = a ( H , D λ 2 ξ λ ) = H ′ ( λ ) .
Hence, we find that the solution ϕ 1 \phi_1 ϕ 1 in (A.4) is
ϕ 1 ( x ) = D λ ξ λ ( x ) = (7.22) { 1 2 x 2 , if x ≤ λ , λ x − 1 2 λ 2 , if x > λ . (97) \phi_1(x)=D_\lambda\xi_\lambda(x)\overset{\text{\textup{(7.22)}}}{=}
\begin{cases}
\frac{1}{2}x^2, & \text{if }x\leq\lambda,\\
\lambda x-\frac{1}{2}\lambda^2, & \text{if }x>\lambda.
\end{cases} \tag*{(97)} ϕ 1 ( x ) = D λ ξ λ ( x ) = (7.22) { 2 1 x 2 , λ x − 2 1 λ 2 , if x ≤ λ , if x > λ . (97)
By repeating the above steps, we further get
ϕ 2 ( x ) = D λ 2 ξ λ ( x ) = { 0 , if x ≤ λ , x − λ , if x > λ . (98) \phi_2(x)=D_\lambda^2\xi_\lambda(x)=
\begin{cases}
0, & \text{if }x\leq\lambda,\\
x-\lambda, & \text{if }x>\lambda.
\end{cases} \tag*{(98)} ϕ 2 ( x ) = D λ 2 ξ λ ( x ) = { 0 , x − λ , if x ≤ λ , if x > λ . (98)
In particular,
a ( G r , ϕ i ) = G r ( i ) ( λ ) ( i = 1 , 2 ) . (99) a(\mathbf{G}_r,\phi_i)=\mathbf{G}_r^{(i)}(\lambda)\qquad(i=1,2). \tag*{(99)} a ( G r , ϕ i ) = G r ( i ) ( λ ) ( i = 1 , 2 ) . (99)
Making the substitution H = E \mathbf{H}=\mathbf{E} H = E in (A.4) gives
K i ( E ) = G ( i ) ( λ ) − G r ( i ) ( λ ) = a ( E , ϕ i ) = a ( G − G r , ϕ i ) = a ( G , ϕ i ) − a ( G r , ϕ i ) . (100) \mathcal{K}_i(\mathbf{E})=\mathbf{G}^{(i)}(\lambda)-\mathbf{G}_r^{(i)}(\lambda)=a(\mathbf{E},\phi_i)=a(\mathbf{G}-\mathbf{G}_r,\phi_i)=a(\mathbf{G},\phi_i)-a(\mathbf{G}_r,\phi_i). \tag*{(100)} K i ( E ) = G ( i ) ( λ ) − G r ( i ) ( λ ) = a ( E , ϕ i ) = a ( G − G r , ϕ i ) = a ( G , ϕ i ) − a ( G r , ϕ i ) . (100)
Since G \mathbf{G} G is a solution of (90) , we obtain
a ( G , ϕ i ) = ∫ 0 1 F ( x ) ϕ i ( x ) d x , (101) a(\mathbf{G},\phi_i)=\int_0^1 F(x)\phi_i(x)\,\mathrm{d}x, \tag*{(101)} a ( G , ϕ i ) = ∫ 0 1 F ( x ) ϕ i ( x ) d x , (101)
where F F F is the right hand side of the strong problem (86) . Facilitating (A.7) and (A.9) in (A.8) gives
G ( i ) ( λ ) − G r ( i ) ( λ ) = ∫ 0 1 F ( x ) ϕ i ( x ) d x − G r ( i ) ( λ ) ( i = 1 , 2 ) . \mathbf{G}^{(i)}(\lambda)-\mathbf{G}_r^{(i)}(\lambda)=\int_0^1 F(x)\phi_i(x)\,\mathrm{d}x-\mathbf{G}_r^{(i)}(\lambda)\qquad(i=1,2). G ( i ) ( λ ) − G r ( i ) ( λ ) = ∫ 0 1 F ( x ) ϕ i ( x ) d x − G r ( i ) ( λ ) ( i = 1 , 2 ) .
Considering the definition of the equilibrium function (34) , choosing λ : = λ ~ j \lambda:=\widetilde{\lambda}_j λ := λ j , j = 0 , … , r − 1 j=0,\ldots,r-1 j = 0 , … , r − 1 , and applying (A.5) and (A.6) gives the statement. ■
Acknowledgments. Most of this work was carried out while TD was with UniDistance Suisse, Faculty of Mathematics and Computer Science.
Funding. The research of TD has received funding from the Swiss Government Excellence Scholarship No. 2025.0057. The research of AA and MV was funded in part by the Swiss National Science Foundation (SNSF) grant No. 224943.
CRediT author statement. Tamás Dózsa: Conceptualization, Methodology, Formal Analysis, Investigation, Software, Visualization, Writing — Original Draft; Andrea Angino: Methodology, Software, Visualization, Writing — Original Draft; Rolf Krause: Writing — Review & Editing; Matthias Voigt: Investigation, Methodology, Visualization, Writing — Review & Editing, Supervision, Project Administration, Funding Acquisition.
Code availability statement. The code and data for reproducing the numerical experiments presented in this paper are available at Zenodo under the DOI 10.5281/zenodo.23494458. The code is released under the MIT license.
List of acronyms
BGMT biorthogonal–GMT
BT balanced truncation
CPL continuous piecewise linear
DWR dual weighted residuals
EIM empirical interpolation method
FEM finite element method
FFT fast Fourier transform
FOM full–order model
GMT generalized Malmquist–Takenaka
IRKA iterative rational Krylov algorithm
LTI linear time–invariant
MOR model order reduction
MSE mean squared error
MT Malmquist–Takenaka
PDE partial differential equation
RBF radial basis function
RKHS reproducing kernel Hilbert space
ROM reduced–order model
SISO single–input single–output
SVM support vector machine
References
Tamás Dózsa
On leave from: Systems and Control Laboratory, HUN–REN Institute for Computer Science and Control, Lágymányosi Utca 11, 1111 Budapest, Hungary; doza.tamas@hsztaki.hu
Author IDs: ORCID 0000-0003-0919-4385
Andrea Angino
Faculty of Mathematics and Computer Science, FernUni Schweiz/UniDistance Suisse, Schinerstrasse 18, 3900 Brig, Switzerland; andrea.angino@unitdistance.ch
Author IDs: ORCID 0009-0000-8525-375X
Rolf Krause
Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, 4700 Thuwal, Kingdom of Saudi Arabia; rolf.krause@kaust.edu.sa
Author IDs: ORCID 0000-0001-5408-5271
Matthias Voigt
Faculty of Mathematics and Computer Science, FernUni Schweiz/UniDistance Suisse, Schinerstrasse 18, 3900 Brig, Switzerland; matthias.voigt@fernuni.ch
Author IDs: ORCID 0000-0001-8491-1861