Introduction
Let Ω \Omega Ω be an open subset of R n \mathbb{R}^n R n . Consider the partial differential equation L u = f \mathcal{L}u=f L u = f in Ω ⊂ R n \Omega\subset\mathbb{R}^n Ω ⊂ R n where u , f ∈ L 2 ( R n ; R ) u,f \in L^2(\mathbb{R}^n;\mathbb{R}) u , f ∈ L 2 ( R n ; R ) and L \mathcal{L} L is some smooth differential operator, possibly with boundary and initial conditions. In this article, we are interesting the following neural operator learning question to study L − 1 \mathcal{L}^{-1} L − 1 :
Question: Given only a finite collection of observations { u ( k ) ( x i ) , f ( k ) ( x i ) } k = 1 K \{u^{(k)}(x_i),f^{(k)}(x_i)\}_{k=1}^{K} { u ( k ) ( x i ) , f ( k ) ( x i ) } k = 1 K of unknown ground truth pairing of functions { u ( k ) , f ( k ) } k = 1 K \{u^{(k)},f^{(k)}\}_{k=1}^{K} { u ( k ) , f ( k ) } k = 1 K on finitely many grids { x i } i = 1 N ⊂ R n \{x_i\}_{i=1}^{N}\subset\mathbb{R}^n { x i } i = 1 N ⊂ R n , where each u ( k ) ( x ) u^{(k)}(x) u ( k ) ( x ) solves L u ( k ) ( x ) = f ( k ) ( x ) \mathcal{L}u^{(k)}(x)=f^{(k)}(x) L u ( k ) ( x ) = f ( k ) ( x ) with prescribed boundary conditions. Can we learn a neural operator G N O G_{NO} G N O that takes the sampled values f = ( f ( x 1 ) , … , f ( x N ) ) \mathbf{f}=(f(x_1),\ldots,f(x_N)) f = ( f ( x 1 ) , … , f ( x N )) of a new input function f f f from the prescribed function class, and reconstruct u ^ ( x ) : = G N O ( f ) ( x ) \hat{u}(x):=G_{NO}(\mathbf{f})(x) u ^ ( x ) := G N O ( f ) ( x ) that approximates the corresponding solution u u u of L u = f \mathcal{L}u=f L u = f at any query locations x ∈ Ω x\in\Omega x ∈ Ω ?
The classical numerical solvers are designed to address this type of problem, but their computational cost can grow rapidly with spatial dimension; in addition, a new input instance generally requires a separate numerical solve of the underlying PDE. Over the last several decades, neural operator methods with deep learning architecture have been developed to learn the solution map for an entire family of PDE problems [36 , 29 ] . A summary of representative approaches is given in §2.
However, neural operator methods have several major issues: generalization remains a major limitation of neural operators, especially for out-of-distribution inputs, unseen discretizations, or different sampling grids; the curse of dimensionality is still not overcome; existing methods also lack the same level of stability guarantees, rigorous error control, and interpretability as classical numerical solvers. Moreover, a trained neural operator is usually tied to a specific PDE family and parameter regime, so substantial retraining may be required when the underlying equation or physical setting changes. For example, many numerical comparisons only report relative L 2 L^2 L 2 errors, which alone do not control the error uniformly in space; the standard FFT-based implementation FNO [32 ] require sampling over uniform grids, other methods require associated preferred patterns of grids; a full tensor-product Fourier representation requires exponentially many modes as the spatial dimension increases, although with a fixed cutoff in each coordinate; in the standard DeepONet architecture, the latent basis is learned implicitly by the trunk network rather than prescribed from an analytically interpretable function space, which can make the resulting representation difficult to interpret.
In this article, we introduce a novel neural operator method based on the Segal-Bargmann transform, where the learning process is carried out in a new space with good properties. In addition, we also show that the combination of the Segal-Bargmann transform and the RKHS approximation serves as a good encoder and a finite dimensional representation of the original discrete data that are coming from evaluation representations of functions. In addition to the classical L 2 L^2 L 2 -type convergence rate, we give an L ∞ L^\infty L ∞ -type convergence rate.
The Segal-Bargmann transform was initially introduced by the independent work of Irving Segal [42 ] and Valentine Bargmann [2 ] in the early 1960s, where Segal gave the general framework on Hilbert spaces, and Bargmann published the explicit integral formula in 1961 [20 ] . It maps square-integrable functions on R n \mathbb{R}^n R n unitarily onto the Bargmann-Fock space of entire functions on C n \mathbb{C}^n C n with a Gaussian-weighted norm [2 ] . Its principal applications include coherent-state representations and quantization in quantum mechanics for detailed analysis of annihilation and creation operators acting on a many-particle quantum state [20 ] , as well as time-frequency analysis through its relation to the Gaussian-windowed short-time Fourier transform [18 ] . Phase space is natural here because, up to scaling and sign conventions, the real and imaginary coordinates label the position and momentum (or location and frequency) of Gaussian wave packets, allowing the transform to describe where oscillatory features occur as well as their frequencies [20 , 18 ] .
We denote by F α 2 ( C n ) \mathcal{F}^{2}_{\alpha}(\mathbb{C}^n) F α 2 ( C n ) the Bargmann-Fock space. We also denote by B α : L 2 ( R n ; C ) → F α 2 ( C n ) \mathcal{B}_{\alpha}: L^{2}(\mathbb{R}^{n};\mathbb{C}) \to\mathcal{F}^{2}_{\alpha}(\mathbb{C}^{n}) B α : L 2 ( R n ; C ) → F α 2 ( C n ) and B α − 1 : F α 2 ( C n ) → L 2 ( R n ; C ) \mathcal{B}_{\alpha}^{-1}: \mathcal{F}^{2}_{\alpha}(\mathbb{C}^{n}) \to L^{2}(\mathbb{R}^{n};\mathbb{C}) B α − 1 : F α 2 ( C n ) → L 2 ( R n ; C ) the Segal-Bargmann transform and inverse-Segal-Bargmann transform, respectively. The precise definitions will be given in § 3 . There is no difference if we restrict the inputs to real-valued functions. In contrast with the current neural operator approaches, we would like to emphasize the advantages of our Segal-Bargmann neural operator approach:
(a) When f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) f \in L^{2}(\mathbb{R}^{n};\mathbb{R}) \cap C^{\infty}(\mathbb{R}^{n};\mathbb{R}) f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) and both Assumption A and Assumption B given in §5 hold, Theorem 5.4 says the pointwise difference between the estimate u ^ = G N O ( f ) \hat{u} = \mathcal{G}_{\mathrm{NO}}(f) u ^ = G NO ( f ) and the ground truth u u u satisfies the dimension-free convergence rate
E [ sup x ∈ R n ∣ u ^ − u ∣ ] = O ( L − 1 2 ) \mathbb{E}\left[\sup_{x\in\mathbb{R}^{n}}|\hat{u}-u|\right] = O\left(L^{-\frac{1}{2}}\right) E [ x ∈ R n sup ∣ u ^ − u ∣ ] = O ( L − 2 1 )
with standard assumptions of the universal approximation theorem of neural operators on Hilbert spaces. Here L L L is the number of complex sampling points { z l } l = 1 L ⊂ C n \{z_l\}_{l=1}^{L} \subset\mathbb{C}^{n} { z l } l = 1 L ⊂ C n , and is independent of the number of original sampling points { x i } i = 1 N \{x_i\}_{i=1}^{N} { x i } i = 1 N . This canonical Monte-Carlo convergence rate is quite general in the sense that it is applicable to almost all partial differential equations, and has almost no requirement of the original grids. One of the advantages of this Segal-Bargmann NO approach is that we can select our complex grids as “probes” or “detectors” freely.
(b) Assumption A is too restrictive. If we drop the Assumption A and assume that f ∈ H s ( R n ; R ) ∩ C ∞ ( R n ; R ) f \in H^{s}(\mathbb{R}^{n};\mathbb{R}) \cap C^{\infty}(\mathbb{R}^{n};\mathbb{R}) f ∈ H s ( R n ; R ) ∩ C ∞ ( R n ; R ) . If s > n 2 s > \frac{n}{2} s > 2 n , then we give the dimension-free pointwise convergence rate
E [ sup x ∈ R n ∣ u ^ − u ∣ ] = O ( L − 1 2 ) \mathbb{E}\left[\sup_{x\in\mathbb{R}^{n}}|\hat{u}-u|\right] = O\left(L^{-\frac{1}{2}}\right) E [ x ∈ R n sup ∣ u ^ − u ∣ ] = O ( L − 2 1 )
in Theorem 5.5, provided that Assumption B holds. If s < n 2 s < \frac{n}{2} s < 2 n , then a natural consequence of Theorem 5.3 and universal approximation theorem of neural operators will imply that
E [ ∥ u − u ^ ∥ L 2 ( R n ; R ) ] = O ( L − s 2 n ) . \mathbb{E}\left[\|u-\hat{u}\|_{L^{2}(\mathbb{R}^{n};\mathbb{R})}\right] = O\left(L^{-\frac{s}{2n}}\right). E [ ∥ u − u ^ ∥ L 2 ( R n ; R ) ] = O ( L − 2 n s ) .
We give a theorem in the Appendix, which demonstrates that our Assumption B can be achieved with some hypotheses and is therefore a reasonable assumption in general.
(c) The L − 1 2 L^{- \frac{1}{2}} L − 2 1 reconstruction errors are dimension-free under the stated hypotheses, which indicates that our Segal-Bargmann neural operator approach partially overcomes the curse of dimensionality. In principle, we can approach to the true solution of the PDE by increasing the complex grids L → ∞ L \to\infty L → ∞ , with just fixed number of observations on { x i } i = 1 N \{x_i\}_{i=1}^{N} { x i } i = 1 N . We give numerical results for the highly nonlinear PDE − Δ u + e u + u 3 = f -\Delta u + e^u + u^3 = f − Δ u + e u + u 3 = f in Ω \Omega Ω , u = 0 u = 0 u = 0 on ∂ Ω \partial\Omega ∂ Ω for Ω ⊂ R 5 \Omega\subset\mathbb{R}^5 Ω ⊂ R 5 . Based on our best understanding, the state-of-the-art NO approaches for unknown differential operator L \mathcal{L} L is spatial dimension 3.
(d) The Segal-Bargmann neural operator approach integrates the information recovery and data encoding, the latent space representation, the learning process and the data decoding into a unified framework, summarized by the following diagram:
Figure 5.
(e) Our algorithm is fast and accurate since our learning process is undertaken in the Bargmann-Fock space, which is an RKHS consisting of complex analytic functions. The RKHS property and the rigidity of holomorphic functions allow us to use small neural network to achieve high accuracy with low computational cost. Our experiments report not only small L 2 L^2 L 2 -type errors, but also much smaller E X [ ∣ u ^ ( X ) − u ( X ) ∣ ∣ u ( X ) ∣ ] \mathbb{E}_X\left[\frac{|\hat{u}(X)-u(X)|}{|u(X)|}\right] E X [ ∣ u ( X ) ∣ ∣ u ^ ( X ) − u ( X ) ∣ ] and sup x ∣ u ^ ( x ) − u ( x ) ∣ ∣ u ( x ) ∣ \sup_x \frac{|\hat{u}(x)-u(x)|}{|u(x)|} sup x ∣ u ( x ) ∣ ∣ u ^ ( x ) − u ( x ) ∣ errors than the NO baselines considered, i.e. we also measure the typical (expected) difference between the estimated solution u ^ \hat{u} u ^ and the ground truth u u u . Our algorithm uses the Gaussian-windowed Fourier-transform nature of the Segal-Bargmann transform, which ensures the stability and high accuracy of our algorithm.
(f) Both the Segal-Bargmann and inverse-Segal-Bargmann transforms are unitary transformations. The key feature of this approach is that we decouple the observations of f f f in the original space at { x i } ∈ R n \{x_i\} \in\mathbb{R}^n { x i } ∈ R n from the learning process with encoding and representation in the new space via new grids { z l } ⊂ C n \{z_l\} \subset\mathbb{C}^n { z l } ⊂ C n . Precisely speaking, the Segal-Bargmann transform serves not only as a unitary information transformation, but also as an encoder, while the RKHS representation of the encoded data in Bargmann-Fock space is designed as the latent space representation of the original data. The inverse-Segal-Bargmann transform can be designed as an exact decoder at the continuous level.
This article is organized as follows: In §2, we give a comprehensive summary of the state-of-the-art data-driven PDE approaches, especially over the last decade. In §3, we give the necessary definitions of the Segal-Bargmann transform, the Bargmann-Fock space, the RKHS kernel, and an associated orthonormal basis in the Bargmann-Fock space that will be used in later sections. In §4, we give an example in §4.1 of how the Segal-Bargmann transform can convert a partial differential equation with solutions in L 2 ( R n ; R ) L^2(\mathbb{R}^n;\mathbb{R}) L 2 ( R n ; R ) into a new, complex-valued partial differential equation in F a 2 ( C n ) \mathcal{F}_a^2(\mathbb{C}^n) F a 2 ( C n ) , where two solutions correspond through the Segal-Bargmann transform. We give the problem setting in § 4.2 . In § 4.3 , we give a two-step algorithm where we study the encoder W W W first, and then study the neural operator in the Bargmann-Fock space. In § 4.4 , we give an end-to-end algorithm that combines the study of the encoder and the neural operator together. In § 5 , we give theoretical results following the problem setting in § 4 . With Assumption A A A and Assumption B B B , we show in Theorem 5.4 that the error bound between the estimate u ^ \hat{u} u ^ and the solution u u u is dimension-free in the sense of the sup-norm, which is at least at the level of classical Monte-Carlo rate O ( L − 1 / 2 ) O\left(L^{-1/2}\right) O ( L − 1/2 ) where L L L is the number of the complex sampling points we choose, decoupled from the original real grids. Once we drop the Assumption A A A and consider the most general input f f f and solution u u u , we show in Theorem 5.5 that the same conclusion holds by assuming that f f f and u u u belong to suitable Sobolev spaces. In § 6 , we give numerical results for 2D Darcy flow, 1D Burgers’ equation with appropriate initial and boundary conditions. For both equations above, we also provide benchmark comparisons with established methods such as FNO, GNOT, Belnet, etc. for both uniform grids and scattered points. We also give numerical results for two nonlinear PDEs in spatial dimension 5.
Related Work in Neural PDEs
Our Segal-Bargmann neural operator approach is a neural operator that uniquely integrates the Segal-Bargmann encoding, operator learning in a finite-dimensional Bargmann-Fock RKHS space, and inverse Segal-Bargmann decoding into a unified framework for learning PDE solution operators. Over the past decade, learning-based PDE solvers have been significantly developed along two main directions: approximating the solution of an individual problem and learning the operator that maps coefficients, forcing terms, or initial and boundary conditions to solutions. These approaches differ in their representations, physical constraints, and use of numerical algorithms.
Physics-informed neural networks (PINNs)
PINNs represent the solution with a neural network and combine equation residuals, initial and boundary conditions, and observations in a common loss, supporting both forward simulation and parameter identification [41 ] . Variational and weak formulations use integral constraints to reduce reliance on pointwise high-order derivatives, while domain decomposition assigns local networks to different regions [28 , 55 , 26 , 38 ] . Studies of gradient imbalance and neural tangent kernels explain why loss components can converge at different rates, motivating adaptive weighting and improved optimization [48 , 50 ] . B-PINNs introduce Bayesian inference to quantify uncertainty in solutions and inferred parameters under noisy observations [53 ] .
DeepONet and reduced-order operator learning
Operator learning trains a model across input functions so that new problem instances can be evaluated without solving each one from scratch. DeepONet combines a branch network encoding the input function with a trunk network encoding output coordinates to predict a continuous solution [36 ] . MIONet extends this construction to multiple input functions, while POD-DeepONet uses a reduced output basis extracted from solution data [27 , 37 ] . PCA-Net compresses both input and output functions and learns a map between their coefficients [6 ] . Their accuracy depends on whether these finite representations preserve the main variations in the solution.
Spectral neural operators
Spectral architectures provide global interactions through transforms. The Fourier neural operator (FNO), a widely used architecture, learns Fourier multipliers and combines them with local mappings and nonlinear activations [32 ] . F-FNO factorizes spectral operations to reduce computational cost, whereas multiwavelet operators and WNO use spatial localization and scale decomposition to represent multiscale structure [45 , 19 , 46 ] . The Laplace neural operator (LNO) represents transient responses through poles and residues in the Laplace domain [8 ] . The choice of transform reflects the spatial and temporal structure of the solution.
Graph networks, geometry, and attention
Irregular meshes and varying domains require models that account for spatial geometry. MeshGraphNets learns local interactions and time evolution through mesh-based message passing; Geo-FNO and GINO connect physical domains to latent representations through learned deformations or graph operators [40 , 31 , 33 ] . Attention offers input-dependent interactions: Galerkin Transformer relates attention to integral approximation, while Transolver aggregates physical states to reduce the cost of interactions over large point sets [9 , 51 ] . These methods support solution prediction beyond regular grids, including complex engineering geometries.
Physics-informed neural operators (PINO)
Physics-informed operator learning imposes governing equations across a family of inputs to reduce dependence on reference solutions. Early physics-constrained convolutional surrogates already explored parametric PDE learning without paired solution data [58 ] . PI-DeepONet incorporates equation and boundary residuals into DeepONet, while PINO combines data fitting with physics constraints, allowing coarse-grid solution data and fine-grid residual evaluation [49 , 34 ] . Variational DeepONet and VINO instead formulate operator training through energy or weak-form objectives [17 , 15 ] . This approach combines rapid operator evaluation with known physical laws, although residual discretization and loss balancing remain consequential training choices.
Kernel methods and random features
Kernel methods provide alternative representations for both PDE solutions and solution operators. Gaussian process methods impose PDE constraints in a kernel space, while operator kernel regression learns solution maps from finite observations of input and output functions [12 , 4 ] . Random features reduce the representation to a finite coefficient problem: operator random-feature models learn maps between function spaces, and the Random Feature Method combines local features with domain decomposition for PDE solving [39 , 10 ] . Fitting expansion coefficients simplifies optimization, although matrix size, conditioning, and nonlinear iterations can still be costly.
Learning within numerical solvers
Hybrid methods retain numerical algorithms and learn components that are expensive or difficult to design. Data-driven discretizations estimate differential operators on coarse grids; Solver-in-the-Loop includes numerical evolution during training; and HINTS combines neural operators with classical relaxation to reduce complementary error components [1 , 47 , 56 ] . For high-dimensional PDEs, deep backward schemes and deep splitting use stochastic representations and time discretization to obtain sequences of neural regression problems [25 , 5 ] . The network thus learns a specific numerical subproblem within an established solution procedure.
Generative models for PDEs
Generative methods model distributions of solution fields, supporting inference from partial observations and uncertain data. DiffusionPDE combines diffusion priors with observation and physics constraints, while physics-informed diffusion models incorporate governing equations directly into training [24 , 3 ] . PDE-Refiner uses iterative denoising to improve long autoregressive rollouts [35 ] . These approaches extend prediction to conditional sampling and iterative refinement, with additional computational costs from repeated generation steps.
PDE pretraining and foundation models
Recent work pretrains shared models on multiple equations and physical datasets before adapting them to specific tasks. DPOT and Poseidon study representation learning and transfer across PDEs, while the PDEformer family also encodes equations through symbolic or computational graph representations [21 , 23 , 54 ] . ICON uses input-output function pairs as context, enabling adaptation to new operator tasks through examples [52 ] . These models seek representations that can be reused across equations, conditions, and data distributions.
The Segal-Bargmann Transform
In this section, we introduce the Segal-Bargmann transform that will be used later. For more details we refer to [16 Chapter 1] . Throughout this article, our domain is either R n \mathbb{R}^{n} R n or open subsets of R n \mathbb{R}^{n} R n . We also assume the familiarity of the standard L 2 L^{2} L 2 -type Sobolev spaces H s ( R n ; R ) H^{s}(\mathbb{R}^{n};\mathbb{R}) H s ( R n ; R ) . We will repeatedly use the standard Gamma function Γ ( s ) \Gamma(s) Γ ( s ) , the incomplete upper and lower Gamma functions γ ( s , x ) \gamma(s,x) γ ( s , x ) and Γ ( s , x ) \Gamma(s,x) Γ ( s , x ) , respectively.
Precisely speaking, the Segal-Bargmann transform is a Gaussian-weighted Fourier transform with complex frequency. Throughout this article, we will interchangeably use Segal-Bargmann transform and a a a -Segal-Bargmann transform. The latter one is used when we want to emphasize the importance of the parameter a a a . All definitions below work equally well for complex-valued L 2 L^{2} L 2 -functions f ∈ L 2 ( R n ; C ) f\in L^{2}(\mathbb{R}^{n};\mathbb{C}) f ∈ L 2 ( R n ; C ) .
Definition 3.1. Let L 2 ( R n ; R ) L^{2}(\mathbb{R}^{n};\mathbb{R}) L 2 ( R n ; R ) be the collection of complex-valued square integrable measurable functions. We denote by the a a a -Segal-Bargmann transform:
L 2 ( R n ; R ) ∋ f ↦ B a ( f ) ( z ) : = ( 2 a π ) n 4 ∫ R n exp ( − a 2 z ⋅ z − a x ⋅ x + 2 a z ⋅ x ) f ( x ) d x . (1) L^{2}(\mathbb{R}^{n};\mathbb{R})\ni f\mapsto B_{a}(f)(z):=\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\int_{\mathbb{R}^{n}}\exp\left(-\frac{a}{2}z\cdot z-ax\cdot x+2az\cdot x\right)f(x)\,dx. \tag*{(1)} L 2 ( R n ; R ) ∋ f ↦ B a ( f ) ( z ) := ( π 2 a ) 4 n ∫ R n exp ( − 2 a z ⋅ z − a x ⋅ x + 2 a z ⋅ x ) f ( x ) d x . (1) The function
A a ( z , x ) = ( 2 a π ) n 4 exp ( − a 2 z ⋅ z − a x ⋅ x + 2 a z ⋅ x ) A_{a}(z,x)=\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\exp\left(-\frac{a}{2}z\cdot z-ax\cdot x+2az\cdot x\right) A a ( z , x ) = ( π 2 a ) 4 n exp ( − 2 a z ⋅ z − a x ⋅ x + 2 a z ⋅ x ) is called a a a -Segal-Bargmann kernel.
It is constructed by Segal and Bargmann that the function B a ( f ) B_{a}(f) B a ( f ) is an entire function on C n \mathbb{C}^{n} C n . Furthermore, it is sitting in a Hilbert space, called Bargmann-Fock space.
Definition 3.2. The Bargmann-Fock space F a 2 ( C n ) \mathcal{F}^{2}_{a}(\mathbb{C}^{n}) F a 2 ( C n ) is defined to be the collection of entire functions such that
F a 2 ( C n ) : = { F ∈ O ( C n ) : ∥ f ∥ F a 2 2 = ⟨ F , F ⟩ F a 2 ( C n ) < ∞ } \mathcal{F}^{2}_{a}(\mathbb{C}^{n}):=\{F\in\mathcal{O}(\mathbb{C}^{n}):\|f\|_{\mathcal{F}^{2}_{a}}^{2}=\langle F,F\rangle_{\mathcal{F}^{2}_{a}(\mathbb{C}^{n})}<\infty\} F a 2 ( C n ) := { F ∈ O ( C n ) : ∥ f ∥ F a 2 2 = ⟨ F , F ⟩ F a 2 ( C n ) < ∞ } where the Bargmann-Fock inner product is defined as
⟨ F , G ⟩ F a 2 ( C n ) : = ∫ C n F ( z ) G ( z ) ‾ d μ a ( z ) : = ( a π ) n ∫ C n F ( z ) G ( z ) ‾ e − a ∣ z ∣ 2 d μ ( z ) . \langle F,G\rangle_{\mathcal{F}^{2}_{a}(\mathbb{C}^{n})}:=\int_{\mathbb{C}^{n}}F(z)\overline{G(z)}\,d\mu_{a}(z):=\left(\frac{a}{\pi}\right)^{n}\int_{\mathbb{C}^{n}}F(z)\overline{G(z)}e^{-a|z|^{2}}\,d\mu(z). ⟨ F , G ⟩ F a 2 ( C n ) := ∫ C n F ( z ) G ( z ) d μ a ( z ) := ( π a ) n ∫ C n F ( z ) G ( z ) e − a ∣ z ∣ 2 d μ ( z ) . Here d μ a ( z ) = ( a π ) n e − a ∣ z ∣ 2 d μ ( z ) d\mu_{a}(z)=\left(\frac{a}{\pi}\right)^{n}e^{-a|z|^{2}}d\mu(z) d μ a ( z ) = ( π a ) n e − a ∣ z ∣ 2 d μ ( z ) is the a a a -Gaussian-weighted complex measure, and d μ ( z ) d\mu(z) d μ ( z ) is the standard complex Lebesgue measure. When n = 1 n=1 n = 1 , d μ ( z ) = d x d y d\mu(z)=dxdy d μ ( z ) = d x d y for z = x + − 1 y z=x+\sqrt{-1}y z = x + − 1 y .
The Segal-Bargmann transform is a unitary transform in the sense that
F = B a ( f ) , G = B a ( g ) ⇒ ∥ f ∥ L 2 ( R n ; R ) = ∥ F ∥ F a 2 ( C n ) , ⟨ F , G ⟩ F a 2 ( C n ) = ⟨ f , g ⟩ L 2 ( R n ; R ) . F=B_{a}(f),G=B_{a}(g)\Rightarrow\|f\|_{L^{2}(\mathbb{R}^{n};\mathbb{R})}=\|F\|_{\mathcal{F}^{2}_{a}(\mathbb{C}^{n})},\langle F,G\rangle_{\mathcal{F}^{2}_{a}(\mathbb{C}^{n})}=\langle f,g\rangle_{L^{2}(\mathbb{R}^{n};\mathbb{R})}. F = B a ( f ) , G = B a ( g ) ⇒ ∥ f ∥ L 2 ( R n ; R ) = ∥ F ∥ F a 2 ( C n ) , ⟨ F , G ⟩ F a 2 ( C n ) = ⟨ f , g ⟩ L 2 ( R n ; R ) .
Therefore, the a a a -Segal-Bargmann transform is an isomorphism between L 2 ( R n ; R ) L^{2}(\mathbb{R}^{n};\mathbb{R}) L 2 ( R n ; R ) and F a 2 ( C n ) \mathcal{F}^{2}_{a}(\mathbb{C}^{n}) F a 2 ( C n ) . The inverse a a a -Segal-Bargmann transform is defined below.
Definition 3.3. The inverse a a a -Segal-Bargmann transform is
B a − 1 ( F ) ( x ) : = ( 2 a π ) n 4 ⋅ ( a π ) n ∫ C n e − a 2 z ‾ ⋅ z ‾ − a x ⋅ x + 2 a z ‾ ⋅ x F ( z ) e − a ∣ z ∣ 2 d μ ( z ) B_{a}^{-1}(F)(x):=\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\cdot\left(\frac{a}{\pi}\right)^{n}\int_{\mathbb{C}^{n}}e^{-\frac{a}{2}\overline{z}\cdot\overline{z}-ax\cdot x+2a\overline{z}\cdot x}F(z)e^{-a|z|^{2}}\,d\mu(z) B a − 1 ( F ) ( x ) := ( π 2 a ) 4 n ⋅ ( π a ) n ∫ C n e − 2 a z ⋅ z − a x ⋅ x + 2 a z ⋅ x F ( z ) e − a ∣ z ∣ 2 d μ ( z ) such that
B a ( B a − 1 ) = I d F a 2 ( C n ) , B a − 1 ( B a ) = I d L 2 ( R n ; R ) . B_{a}(B_{a}^{-1})=Id_{\mathcal{F}^{2}_{a}(\mathbb{C}^{n})},\ B_{a}^{-1}(B_{a})=Id_{L^{2}(\mathbb{R}^{n};\mathbb{R})}. B a ( B a − 1 ) = I d F a 2 ( C n ) , B a − 1 ( B a ) = I d L 2 ( R n ; R ) .
It is well-known that the Bargmann-Fock space is an RKHS space, i.e. the point evaluation functional is a continuous functional on F a 2 ( C n ) \mathcal{F}^{2}_{a}(\mathbb{C}^{n}) F a 2 ( C n ) . We now state a few more facts about the space F a 2 ( C n ) \mathcal{F}^{2}_{a}(\mathbb{C}^{n}) F a 2 ( C n ) .
Definition 3.4. There exists an orthonormal basis { e β a } β \{e_{\beta}^{a}\}_{\beta} { e β a } β of F a 2 ( C n ) \mathcal{F}_{a}^{2}(\mathbb{C}^{n}) F a 2 ( C n ) with
e β a ( z ) = a ∣ β ∣ β ! z β , β ∈ N 0 n . e_{\beta}^{a}(z)=\sqrt{\frac{a^{|\beta|}}{\beta!}}z^{\beta},\quad\beta\in\mathbb{N}_{0}^{n}. e β a ( z ) = β ! a ∣ β ∣ z β , β ∈ N 0 n . This orthonormal basis associates with an orthonormal basis { h β a } β \{h_{\beta}^{a}\}_{\beta} { h β a } β in L 2 ( R n ; R ) L^{2}(\mathbb{R}^{n};\mathbb{R}) L 2 ( R n ; R ) due to the isometry of a a a -Segal-Bargmann transform. The normalized Gaussian-Hermite function h β a h_{\beta}^{a} h β a is of the form
B a − 1 ( e β a ) ( x ) = h β a ( x ) = ( 2 a ) n 4 π n 4 2 ∣ β ∣ β ! H β ( 2 a x ) e − a ∣ x ∣ 2 2 . \mathcal{B}_{a}^{-1}(e_{\beta}^{a})(x)=h_{\beta}^{a}(x)=\frac{(2a)^{\frac{n}{4}}}{\pi^{\frac{n}{4}}\sqrt{2^{|\beta|}\beta!}}H_{\beta}(\sqrt{2a}x)e^{-\frac{a|x|^{2}}{2}}. B a − 1 ( e β a ) ( x ) = h β a ( x ) = π 4 n 2 ∣ β ∣ β ! ( 2 a ) 4 n H β ( 2 a x ) e − 2 a ∣ x ∣ 2 . Here β = ( β 1 , … , β n ) \beta=(\beta_{1},\ldots,\beta_{n}) β = ( β 1 , … , β n ) ,
H β ( x ) = ∏ i = 1 n H β i ( x i ) , x = ( x 1 , … , x n ) ∈ R n H_{\beta}(x)=\prod_{i=1}^{n}H_{\beta_{i}}(x_{i}),\ x=(x_{1},\ldots,x_{n})\in\mathbb{R}^{n} H β ( x ) = i = 1 ∏ n H β i ( x i ) , x = ( x 1 , … , x n ) ∈ R n where the 1-dimensional physicists’ Hermitian polynomial of order m m m is given by
H m ( y ) = ( − 1 ) m e y 2 d m d y m e − y 2 , y ∈ R . H_{m}(y)=(-1)^{m}e^{y^{2}}\frac{d^{m}}{dy^{m}}e^{-y^{2}},\ y\in\mathbb{R}. H m ( y ) = ( − 1 ) m e y 2 d y m d m e − y 2 , y ∈ R . The RKHS kernel of the a a a -Bargmann-Fock space is
K a ( z , w ) = e a z ⋅ w ‾ = ∑ β e β a ( z ) e β a ( w ‾ ) . K_{a}(z,w)=e^{az\cdot\overline{w}}=\sum_{\beta}e_{\beta}^{a}(z)e_{\beta}^{a}(\overline{w}). K a ( z , w ) = e a z ⋅ w = β ∑ e β a ( z ) e β a ( w ) . such that
⟨ F ( ⋅ ) , K a ( ⋅ , w ) ⟩ F a 2 ( C n ) = F ( w ) , ∀ F ∈ F a 2 ( C n ) . \langle F(\cdot),K_{a}(\cdot,w)\rangle_{\mathcal{F}_{a}^{2}(\mathbb{C}^{n})}=F(w),\ \forall F\in\mathcal{F}_{a}^{2}(\mathbb{C}^{n}). ⟨ F ( ⋅ ) , K a ( ⋅ , w ) ⟩ F a 2 ( C n ) = F ( w ) , ∀ F ∈ F a 2 ( C n ) .
Definition 3.5. Define the Gauss-Hermite harmonic oscillator
H a : = − Δ + 4 a 2 ∣ x ∣ 2 , \mathcal{H}_{a}:=-\Delta+4a^{2}|x|^{2}, H a := − Δ + 4 a 2 ∣ x ∣ 2 , which satisfies the following eigenvalue equation
H a h β a = ( 4 a ∣ β ∣ + 2 a n ) h β a , \mathcal{H}_{a}h_{\beta}^{a}=(4a|\beta|+2an)h_{\beta}^{a}, H a h β a = ( 4 a ∣ β ∣ + 2 an ) h β a , i.e. each h β a h_{\beta}^{a} h β a is an eigenfunction of the operator H a \mathcal{H}_{a} H a with eigenvalue λ β , n = ( 4 a ∣ β ∣ + 2 a n ) \lambda_{\beta,n}=(4a|\beta|+2an) λ β , n = ( 4 a ∣ β ∣ + 2 an ) .
Define the even order Hermite-Sobolev space
H a s ( R n ; R ) : = { f ∈ L 2 ( R n ; R ) : ∥ f ∥ H a s ( R n ; R ) : = ∥ ( I + H a ) s 2 f ∥ L 2 ( R n ; R ) < ∞ , s = 2 m ∈ N } . \mathcal{H}_{a}^{s}(\mathbb{R}^{n};\mathbb{R}):=\{f\in L^{2}(\mathbb{R}^{n};\mathbb{R}):\|f\|_{\mathcal{H}_{a}^{s}(\mathbb{R}^{n};\mathbb{R})}:=\|(I+\mathcal{H}_{a})^{\frac{s}{2}}f\|_{L^{2}(\mathbb{R}^{n};\mathbb{R})}<\infty,\ s=2m\in\mathbb{N}\}. H a s ( R n ; R ) := { f ∈ L 2 ( R n ; R ) : ∥ f ∥ H a s ( R n ; R ) := ∥ ( I + H a ) 2 s f ∥ L 2 ( R n ; R ) < ∞ , s = 2 m ∈ N } . It follows that in the L 2 L^{2} L 2 -sense
f = ∑ β c β h β a ⟹ ∥ f ∥ H a s ( R n ; R ) 2 = ∑ β ( 1 + λ β , n ) s ∣ c β ∣ 2 = ∑ β ( 1 + 4 a ∣ β ∣ + 2 a n ) s ∣ c β ∣ 2 . f=\sum_{\beta}c_{\beta}h_{\beta}^{a}\Longrightarrow\|f\|_{\mathcal{H}_{a}^{s}(\mathbb{R}^{n};\mathbb{R})}^{2}=\sum_{\beta}(1+\lambda_{\beta,n})^{s}|c_{\beta}|^{2}=\sum_{\beta}(1+4a|\beta|+2an)^{s}|c_{\beta}|^{2}. f = β ∑ c β h β a ⟹ ∥ f ∥ H a s ( R n ; R ) 2 = β ∑ ( 1 + λ β , n ) s ∣ c β ∣ 2 = β ∑ ( 1 + 4 a ∣ β ∣ + 2 an ) s ∣ c β ∣ 2 .
Neural PDEs and Algorithms
In this section, we first give a concrete example of how the Segal-Bargmann transform converts a PDE where the differential operator L \mathcal{L} L applies to the original L 2 ( R n ; R ) L^{2}(\mathbb{R}^{n};\mathbb{R}) L 2 ( R n ; R ) to a new PDE where the differential operator G ~ \widetilde{\mathcal{G}} G applies to the Bargmann-Fock space F a 2 ( C n ) \mathcal{F}_{a}^{2}(\mathbb{C}^{n}) F a 2 ( C n ) . This concrete example indicates that study the original PDE is equivalent to study the new PDE after Segal-Bargmann transform.
We then set the problem and give two algorithms of how to learn the differential operator, or more interestingly, the inverse of the differential operator, by a neural operator that maps between Bargmann-Fock spaces. The theoretical analysis of our approaches, including the convergence rates, are established in § 5 with moderate assumptions.
Neural PDEs in Bargmann-Fock Space
We would like to apply the Segal-Bargmann transform to solve partial differential equations with appropriate initial and boundary conditions. As an example, we consider the following elliptic equation
− Δ u + u = f in R n -\Delta u + u = f \text{ in } \mathbb{R}^{n} − Δ u + u = f in R n
where f ∈ L 2 ( R n ; R ) f \in L^{2}(\mathbb{R}^{n};\mathbb{R}) f ∈ L 2 ( R n ; R ) . In our scenario, we assume that we have some prior knowledge of u u u and f f f , and we would like to use Neural Operator (NO) to recover the inverse ( − Δ + I d ) − 1 (-\Delta+ Id)^{-1} ( − Δ + I d ) − 1 . Instead of the classical methods that construct the NO between the spaces of functions over R n \mathbb{R}^{n} R n , we take the Segal-Bargmann transform and consider the differential relation in F a 2 ( C n ) \mathcal{F}_{a}^{2}(\mathbb{C}^{n}) F a 2 ( C n ) , where we take advantage of the fact that F a 2 ( C n ) \mathcal{F}_{a}^{2}(\mathbb{C}^{n}) F a 2 ( C n ) is an RKHS space with a designated RKHS kernel.
Let ( z 1 , … , z n ) (z_{1}, \ldots, z_{n}) ( z 1 , … , z n ) be coordinates in C n \mathbb{C}^{n} C n . Since f ∈ L 2 ( R n ; R ) f \in L^{2}(\mathbb{R}^{n};\mathbb{R}) f ∈ L 2 ( R n ; R ) , we are legitimate to apply the a a a -Segal-Bargmann transform on both sides of the PDE,
B a ( − Δ u + u ) ( z ) = B a ( f ) ( z ) ∈ F 2 ( C n ) . \mathcal{B}_{a}(-\Delta u + u)(z) = \mathcal{B}_{a}(f)(z) \in\mathcal{F}^{2}(\mathbb{C}^{n}). B a ( − Δ u + u ) ( z ) = B a ( f ) ( z ) ∈ F 2 ( C n ) .
Applying integration by parts on R n \mathbb{R}^{n} R n , the a a a -Segal-Bargmann transform for both sides of the PDE gives
( L z , n + 1 ) B a ( u ) : = − ∑ j = 1 n ( ∂ 2 B ( u ) ∂ z j 2 + ( − a + a 2 z j 2 − 2 z j a ∂ z j ) B a ( u ) ) + B a ( u ) = B a ( f ) . (2) (L_{z,n}+1)\mathcal{B}_{a}(u) := -\sum_{j=1}^{n}\left(\frac{\partial^{2}\mathcal{B}(u)}{\partial z_{j}^{2}}+\left(-a+a^{2}z_{j}^{2}-2z_{j}a\partial_{z_{j}}\right)\mathcal{B}_{a}(u)\right)+\mathcal{B}_{a}(u)=\mathcal{B}_{a}(f). \tag*{(2)} ( L z , n + 1 ) B a ( u ) := − j = 1 ∑ n ( ∂ z j 2 ∂ 2 B ( u ) + ( − a + a 2 z j 2 − 2 z j a ∂ z j ) B a ( u ) ) + B a ( u ) = B a ( f ) . (2)
Assume that we have inputs and outputs { f k } \{f_{k}\} { f k } , { u k } \{u_{k}\} { u k } , respectively. Our goal is to learn the solver, i.e. we are seeking for a NO G θ \mathcal{G}_{\theta} G θ parametrized by θ ∈ R n \theta\in\mathbb{R}^{n} θ ∈ R n , and we hope that G θ f \mathcal{G}_{\theta}f G θ f solves the PDE, i.e.
( − Δ + 1 ) ( G θ f ) ≈ f . (-\Delta+1)(\mathcal{G}_{\theta}f)\approx f. ( − Δ + 1 ) ( G θ f ) ≈ f .
In other words, we would like to study G θ ≈ ( − Δ + 1 ) − 1 \mathcal{G}_{\theta}\approx(-\Delta+1)^{-1} G θ ≈ ( − Δ + 1 ) − 1 . With the help of the a a a -Segal-Bargmann transform, our study diagram can be shown as below:
{ f k } ⊂ L 2 ( R n ; R ) → G θ H 2 ( R n ; R ) ⊃ { u k } B a ↓ B a B a − 1 ↑ B a − 1 { B a ( f k ) } ⊂ F a 2 ( C n ) → G ^ θ ^ F a 2 ( C n ) ⊃ { B a ( u k ) } . (3) \begin{aligned}
\{f_{k}\}\subset L^{2}(\mathbb{R}^{n};\mathbb{R})&\xrightarrow{\mathcal{G}_{\theta}}H^{2}(\mathbb{R}^{n};\mathbb{R})\supset\{u_{k}\}\\
\mathcal{B}_{a}\downarrow\phantom{\mathcal{B}_{a}}&\phantom{\mathcal{B}_{a}^{-1}}\uparrow\mathcal{B}_{a}^{-1}\\
\{\mathcal{B}_{a}(f_{k})\}\subset\mathcal{F}_{a}^{2}(\mathbb{C}^{n})&\xrightarrow{\widehat{\mathcal{G}}_{\widehat{\theta}}}\mathcal{F}_{a}^{2}(\mathbb{C}^{n})\supset\{\mathcal{B}_{a}(u_{k})\}. \tag*{(3)}
\end{aligned} { f k } ⊂ L 2 ( R n ; R ) B a ↓ B a { B a ( f k )} ⊂ F a 2 ( C n ) G θ H 2 ( R n ; R ) ⊃ { u k } B a − 1 ↑ B a − 1 G θ F a 2 ( C n ) ⊃ { B a ( u k )} . (3)
Due to the one-to-one unitary correspondence between L 2 ( R 2 ) L^{2}(\mathbb{R}^{2}) L 2 ( R 2 ) and F 2 ( C 2 ) \mathcal{F}^{2}(\mathbb{C}^{2}) F 2 ( C 2 ) , it is equivalent to learn the complexified Neural Operator G ^ θ ^ \widehat{\mathcal{G}}_{\widehat{\theta}} G θ instead such that
( L z , n + 1 ) G ^ θ ^ ≈ I d F a 2 ( C n ) . (L_{z,n}+1)\widehat{\mathcal{G}}_{\widehat{\theta}}\approx Id_{\mathcal{F}_{a}^{2}(\mathbb{C}^{n})}. ( L z , n + 1 ) G θ ≈ I d F a 2 ( C n ) .
The Problem Setting and The Loss of Information of Discrete Segal-Bargmann Transform
Our problem setting is quite general. Consider the time-independent PDEs
L u = f in Ω ⊂ R n , B u = g on ∂ Ω . \mathcal{L}u=f\text{ in }\Omega\subset\mathbb{R}^{n},\quad Bu=g\text{ on }\partial\Omega. L u = f in Ω ⊂ R n , B u = g on ∂ Ω.
Here u , f ∈ L 2 ( R n ; R ) u,f\in L^{2}(\mathbb{R}^{n};\mathbb{R}) u , f ∈ L 2 ( R n ; R ) , g ∈ L 2 ( R n − 1 ; R ) g\in L^{2}(\mathbb{R}^{n-1};\mathbb{R}) g ∈ L 2 ( R n − 1 ; R ) , L \mathcal{L} L is a smooth differential operator, and B B B is a smooth boundary operator. Ω \Omega Ω is a bounded, open, connected subset of R n \mathbb{R}^{n} R n for simplicity, and ∂ Ω \partial\Omega ∂ Ω may have some level of singularity, e.g. non-Lipschitz, etc.
We also consider the time-dependent PDEs
L t u + L x u = f in Ω × [ 0 , T ] ⊂ R n × R t , B x u = g on ∂ Ω × [ 0 , T ] , B t u = u 0 on Ω × { 0 } . \mathcal{L}_{t}u+\mathcal{L}_{x}u=f\text{ in }\Omega\times[0,T]\subset\mathbb{R}^{n}\times\mathbb{R}_{t},\quad B_{x}u=g\text{ on }\partial\Omega\times[0,T],\quad B_{t}u=u_{0}\text{ on }\Omega\times\{0\}. L t u + L x u = f in Ω × [ 0 , T ] ⊂ R n × R t , B x u = g on ∂ Ω × [ 0 , T ] , B t u = u 0 on Ω × { 0 } .
Here u , f ∈ C ( [ 0 , T ] ; L 2 ( R n ; R ) ) u,f\in C([0,T];L^{2}(\mathbb{R}^{n};\mathbb{R})) u , f ∈ C ([ 0 , T ] ; L 2 ( R n ; R )) , g ∈ C ( [ 0 , T ] ; L 2 ( R n − 1 ; R ) ) g\in C([0,T];L^{2}(\mathbb{R}^{n-1};\mathbb{R})) g ∈ C ([ 0 , T ] ; L 2 ( R n − 1 ; R )) , u 0 ∈ L 2 ( R n ; R ) u_{0}\in L^{2}(\mathbb{R}^{n};\mathbb{R}) u 0 ∈ L 2 ( R n ; R ) , L t , L x , B x , B t \mathcal{L}_{t},\mathcal{L}_{x},B_{x},B_{t} L t , L x , B x , B t are smooth differential and boundary operators, respectively.
Under this basic settings, it is legitimate to take a a a -Segal-Bargmann transform on both sides of the PDE, which will result in either a local version as the example shows, or at least a superposition.
In this article, we apply machine learning techniques to estimate the inverse of the differential operators L − 1 \mathcal{L}^{-1} L − 1 or ( L x + L t ) − 1 (\mathcal{L}_x+\mathcal{L}_t)^{-1} ( L x + L t ) − 1 by neural operators G N O \mathcal{G}_{NO} G N O , where the input is f f f , and the output is u u u , provided that the boundary and initial conditions are completely known.
In particular, we only have knowledge of a collection of training set ( u ( k ) , f ( k ) ) k = 1 K (u^{(k)},f^{(k)})_{k=1}^{K} ( u ( k ) , f ( k ) ) k = 1 K where we do not know the functions but instead a finite collection of observations ( u ( k ) ( x i ) , f ( k ) ( x i ) ) i = 1 N (u^{(k)}(x_i),f^{(k)}(x_i))_{i=1}^{N} ( u ( k ) ( x i ) , f ( k ) ( x i ) ) i = 1 N for each k k k only.
When the spatial dimension n n n is high, or when the sampling size N N N or K K K is relatively small, or we have no prior knowledge of how the observations are collected, it is very hard to recover the relation between u u u and f f f efficiently just within the original space R n \mathbb{R}^{n} R n and associated Banach spaces.
For example, f = ∑ β c β h β a ∈ L 2 ( R n ; R ) f=\sum_{\beta}c_{\beta}h_{\beta}^{a}\in L^{2}(\mathbb{R}^{n};\mathbb{R}) f = ∑ β c β h β a ∈ L 2 ( R n ; R ) uniquely. An encoder is necessary to recover { c β } \{c_{\beta}\} { c β } from { f ( x i ) } \{f(x_i)\} { f ( x i )} . However, there is no theoretical guarantee of this step. Nevertheless, the best we can do is to recover c ^ β \hat{c}_{\beta} c ^ β such that f ( x i ) ≈ ∑ β c ^ β h β a ( x i ) f(x_i)\approx\sum_{\beta}\hat{c}_{\beta}h_{\beta}^{a}(x_i) f ( x i ) ≈ ∑ β c ^ β h β a ( x i ) . This indicates the significant demand of observations if we would like to recover f f f everywhere.
However, we may only have very limited observations over { x i } i = 1 N \{x_i\}_{i=1}^{N} { x i } i = 1 N . One of the main purposes of the introduction of the Segal-Bargmann transform and the Bargmann-Fock space is to recover the information of f f f by complex “probes” { z l } \{z_l\} { z l } , which can be chosen arbitrarily (assume that we can afford the cost).
Not only that, we can also give a concrete latent space representation and then decode the data by taking advantage of the RKHS structure of the Bargmann-Fock space. In addition, the elements of F a 2 ( C n ) \mathcal{F}_{a}^{2}(\mathbb{C}^{n}) F a 2 ( C n ) are entire functions, and the rigidity of complex analyticity ensures the uniqueness of our decoder theoretically.
As we said, the purpose of the Segal-Bargmann encoder is due to the lack of information. For the ground truth f ∈ L 2 ( R n ; R ) f\in L^{2}(\mathbb{R}^{n};\mathbb{R}) f ∈ L 2 ( R n ; R ) , the unitary transformation that preserves all information of f f f is
B a ( f ) ( z ) = ∫ R n A a ( z , x ) f ( x ) d x . \mathcal{B}_{a}(f)(z)=\int_{\mathbb{R}^{n}}A_{a}(z,x)f(x)\,dx. B a ( f ) ( z ) = ∫ R n A a ( z , x ) f ( x ) d x .
But in practice, we often only have finitely many observations { f ( x i ) } i = 1 N \{f(x_i)\}_{i=1}^{N} { f ( x i ) } i = 1 N ; in addition, we often have no idea whether the real grids { x i } i = 1 N \{x_i\}_{i=1}^{N} { x i } i = 1 N was deliberately chosen with some pattern or not. Clearly, the naive discrete Segal-Bargmann transform
f ↦ 1 N ∑ i = 1 N A a ( z , x i ) f ( x i ) f\mapsto\frac{1}{N}\sum_{i=1}^{N}A_{a}(z,x_i)f(x_i) f ↦ N 1 i = 1 ∑ N A a ( z , x i ) f ( x i )
is not a good choice. Denote by F = B a ( f ) F=\mathcal{B}_{a}(f) F = B a ( f ) and suppose that we have chosen a complex sampling { z l } l = 1 N \{z_l\}_{l=1}^{N} { z l } l = 1 N , our hope is that we can find good approximations F ~ N \widetilde{F}_{N} F N such that
F ( z l ) ≈ F ~ N ( z l ) , ∀ l . F(z_l)\approx\widetilde{F}_{N}(z_l),\ \forall l. F ( z l ) ≈ F N ( z l ) , ∀ l .
If we can recover F F F , it is equivalent to recover f f f . We therefore introduce an information recovery matrix W = ( w l i ) W=(w_{li}) W = ( w l i ) and denote
F ~ N ( z l ) : = 1 N ∑ i = 1 N w l i A a ( z l , x i ) f ( x i ) . \widetilde{F}_{N}(z_l):=\frac{1}{N}\sum_{i=1}^{N}w_{li}A_{a}(z_l,x_i)f(x_i). F N ( z l ) := N 1 i = 1 ∑ N w l i A a ( z l , x i ) f ( x i ) .
Note that the product of W W W and A = ( A a ( z l , x i ) ) A=(A_{a}(z_l,x_i)) A = ( A a ( z l , x i )) is a Hadamard product. Theoretically, we can apply a nonlinear activation on F ~ N = ( F ~ N ( z 1 ) , … , F ~ N ( z L ) ) t \mathbf{\widetilde{F}}_{N}=(\widetilde{F}_{N}(z_1),\ldots,\widetilde{F}_{N}(z_L))^{t} F N = ( F N ( z 1 ) , … , F N ( z L ) ) t to recover F ( z l ) F(z_l) F ( z l ) with zero error when L L L is large enough by a version of universal approximation theorem [14 ] .
We then use our F ~ N ( z l ) \widetilde{F}_{N}(z_l) F N ( z l ) to construct the encoder. Intuitively, we improve our learning process in (2) via Segal-Bargmann transform as the following diagram shows
{ f k } ⊂ L 2 → G θ L 2 ⊃ { u k } { f k } ⊂ L 2 → G θ L 2 ⊃ { u k } { f k } ⊂ L 2 ↓ B a ↑ B a − 1 ⇒ { f k } ⊂ L 2 ↓ B a + W ↑ B a − 1 { B a ( f k ) } ⊂ F a 2 → G ^ θ ^ F a 2 ⊃ { B a ( u k ) } { B a ( f k ) } ⊂ F a 2 → G ^ θ ^ F a 2 ⊃ { B a ( u k ) } . (4) \begin{array}{ccccc}
\{f_k\}\subset L^2 & \xrightarrow{G_\theta} & L^2\supset\{u_k\} & & \{f_k\}\subset L^2\xrightarrow{G_\theta}L^2\supset\{u_k\}\\
\mathrel{\phantom{\{f_k\}\subset L^2}}\downarrow B_a & & \uparrow B_a^{-1} & \Rightarrow& \mathrel{\phantom{\{f_k\}\subset L^2}}\downarrow B_a+W\qquad\qquad\uparrow B_a^{-1}\\
\{B_a(f_k)\}\subset\mathcal{F}_a^2 & \xrightarrow{\widehat{G}_{\hat{\theta}}} & \mathcal{F}_a^2\supset\{B_a(u_k)\} & & \{B_a(f_k)\}\subset\mathcal{F}_a^2\xrightarrow{\widehat{G}_{\hat{\theta}}}\mathcal{F}_a^2\supset\{B_a(u_k)\}. \tag*{(4)}
\end{array} { f k } ⊂ L 2 { f k } ⊂ L 2 ↓ B a { B a ( f k )} ⊂ F a 2 G θ G θ ^ L 2 ⊃ { u k } ↑ B a − 1 F a 2 ⊃ { B a ( u k )} ⇒ { f k } ⊂ L 2 G θ L 2 ⊃ { u k } { f k } ⊂ L 2 ↓ B a + W ↑ B a − 1 { B a ( f k )} ⊂ F a 2 G θ ^ F a 2 ⊃ { B a ( u k )} . (4)
Therefore, we need to learn both W W W and G ^ θ ^ \widehat{G}_{\hat{\theta}} G θ ^ . We give two algorithms below.
Two-Step Algorithm
In this subsection, we give a twp-step algorithm. The first step is to construct a Segal-Bargmann encoder as shown in (4); the second step is to learn the complex version of the differential operator over the Bargmann-Fock space.
Step 0: Original Input. Assume that we have training set { u ( k ) , f ( k ) } \{u^{(k)},f^{(k)}\} { u ( k ) , f ( k ) } with L u ( k ) = f ( k ) \mathcal{L}u^{(k)}=f^{(k)} L u ( k ) = f ( k ) . Suppose that we have real grids { x i } i = 1 N \{x_i\}_{i=1}^{N} { x i } i = 1 N , either from some distribution { X i } ∼ i . i . d . ρ \{X_i\}\overset{\mathrm{i.i.d.}}{\sim}\rho { X i } ∼ i.i.d. ρ or not, and we only have evaluation representations
{ u ( k ) ( x i ) , f ( k ) ( x i ) } , i = 1 , … , N , k = 1 , … , K . \{u^{(k)}(x_i),f^{(k)}(x_i)\},\ i=1,\ldots,N,\ k=1,\ldots,K. { u ( k ) ( x i ) , f ( k ) ( x i )} , i = 1 , … , N , k = 1 , … , K .
Step 1: Complex Grids and Segal-Bargmann Encoder. Denote by
Λ M = { β ∈ N 0 n : ∣ Λ M ∣ = M } , A a , M ( z , x ) = ∑ ∣ β ∣ ≤ M e β a ( z ) h β a ( x ) , \Lambda_M=\{\beta\in\mathbb{N}_0^n:|\Lambda_M|=M\},\quad A_{a,M}(z,x)=\sum_{|\beta|\leq M}e_\beta^a(z)h_\beta^a(x), Λ M = { β ∈ N 0 n : ∣ Λ M ∣ = M } , A a , M ( z , x ) = ∣ β ∣ ≤ M ∑ e β a ( z ) h β a ( x ) ,
We denote by V M = { h β a : β ∈ Λ M } V_M=\{h_\beta^a:\beta\in\Lambda_M\} V M = { h β a : β ∈ Λ M } the associated finite collection of orthonormal basis with respect to Λ M \Lambda_M Λ M , and B M = sup β ∈ Λ M ∣ β ∣ B_M=\sup_{\beta\in\Lambda_M}|\beta| B M = sup β ∈ Λ M ∣ β ∣ . Set
K M ( z ) : = ∑ β ∈ Λ M ∣ e β a ( z ) ∣ 2 , q M ( z ) = M K M ( z ) , d ρ a ( z ) : = K M ( z ) M d μ a ( z ) = K M ( z ) M ( a π ) n e − a ∣ z ∣ 2 d μ ( z ) . K_M(z):=\sum_{\beta\in\Lambda_M}|e_\beta^a(z)|^2,\quad q_M(z)=\frac{M}{K_M(z)},\quad d\rho_a(z):=\frac{K_M(z)}{M}d\mu_a(z)=\frac{K_M(z)}{M}\left(\frac{a}{\pi}\right)^n e^{-a|z|^2}d\mu(z). K M ( z ) := β ∈ Λ M ∑ ∣ e β a ( z ) ∣ 2 , q M ( z ) = K M ( z ) M , d ρ a ( z ) := M K M ( z ) d μ a ( z ) = M K M ( z ) ( π a ) n e − a ∣ z ∣ 2 d μ ( z ) .
Applying Christoffel-function sampling following Cohen-Migliorati’s [13 ] , we choose complex grids as random sampling of { Z l } l = 1 L / 2 \{Z_l\}_{l=1}^{L/2} { Z l } l = 1 L /2 for even L L L such that
Z 1 , … , Z L / 2 ∼ i . i . d . ρ a . Z_1,\ldots,Z_{L/2}\overset{\mathrm{i.i.d.}}{\sim}\rho_a. Z 1 , … , Z L /2 ∼ i.i.d. ρ a .
Our complex sampling of even size L L L is given by complex conjugacy
Z 1 , … , Z L / 2 , Z ‾ 1 , … , Z ‾ L / 2 = Z 1 , … , Z L / 2 , Z L / 2 + 1 , … , Z L . Z_1,\ldots,Z_{L/2},\overline{Z}_1,\ldots,\overline{Z}_{L/2}=Z_1,\ldots,Z_{L/2},Z_{L/2+1},\ldots,Z_L. Z 1 , … , Z L /2 , Z 1 , … , Z L /2 = Z 1 , … , Z L /2 , Z L /2 + 1 , … , Z L .
Let { z l } l = 1 L \{z_l\}_{l=1}^{L} { z l } l = 1 L be the random sampling from the collection of random variables above. We define
F ~ N k ( z l ) : = 1 N ∑ i = 1 N w l i A a , M ( z l , x i ) f ( k ) ( x i ) . \widetilde{F}_N^k(z_l):=\frac{1}{N}\sum_{i=1}^{N}w_{li}A_{a,M}(z_l,x_i)f^{(k)}(x_i). F N k ( z l ) := N 1 i = 1 ∑ N w l i A a , M ( z l , x i ) f ( k ) ( x i ) .
Here W = ( w l i ) W=(w_{li}) W = ( w l i ) is the information recovery matrix . For general f f f evaluated on { x i } \{x_i\} { x i } , we denote by
g l = ( A a , M ( z l , x 1 ) f ( x 1 ) ⋮ A a , M ( z l , x N ) f ( x N ) ) , w l = ( w l 1 ⋮ w l N ) , μ l = E f [ g l ] , Γ l m = E f [ ( μ l − g l ) ( μ m − g m ) ∗ ] \mathbf{g}_l=\begin{pmatrix}
A_{a,M}(z_l,x_1)f(x_1)\\
\vdots\\
A_{a,M}(z_l,x_N)f(x_N)
\end{pmatrix},\quad
\mathbf{w}_l=\begin{pmatrix}
w_{l1}\\
\vdots\\
w_{lN}
\end{pmatrix},\quad
\mu_l=\mathbb{E}_f[\mathbf{g}_l],\quad
\Gamma_{lm}=\mathbb{E}_f[(\mu_l-\mathbf{g}_l)(\mu_m-\mathbf{g}_m)^*] g l = A a , M ( z l , x 1 ) f ( x 1 ) ⋮ A a , M ( z l , x N ) f ( x N ) , w l = w l 1 ⋮ w l N , μ l = E f [ g l ] , Γ l m = E f [( μ l − g l ) ( μ m − g m ) ∗ ]
where ( ⋅ ) ∗ (\cdot)^* ( ⋅ ) ∗ means Hermitian transpose. Theoretically, W W W is obtained by minimizing the loss
L W , t h e o r y = λ 1 L ∑ l = 1 L ∣ μ l ∣ 2 + λ 2 L ( L − 1 ) ∑ l ≠ m ∣ Γ l m ∣ 2 + λ 3 L ∑ l = 1 L ∣ Γ l l ∣ 2 + λ 4 ∥ W ∥ 2 L_{W,\mathit{theory}}=\frac{\lambda_1}{L}\sum_{l=1}^{L}|\mu_l|^2+\frac{\lambda_2}{L(L-1)}\sum_{l\ne m}|\Gamma_{lm}|^2+\frac{\lambda_3}{L}\sum_{l=1}^{L}|\Gamma_{ll}|^2+\lambda_4\lVert W\rVert^2 L W , theory = L λ 1 l = 1 ∑ L ∣ μ l ∣ 2 + L ( L − 1 ) λ 2 l = m ∑ ∣ Γ l m ∣ 2 + L λ 3 l = 1 ∑ L ∣ Γ l l ∣ 2 + λ 4 ∥ W ∥ 2
with regularization factors λ i \lambda_i λ i , i = 1 , … , 4 i=1,\ldots,4 i = 1 , … , 4 . Using our data { f ( k ) ( x i ) } \{f^{(k)}(x_i)\} { f ( k ) ( x i )} and { z l } \{z_l\} { z l } , we denote by
η l k = 1 N ∑ i = 1 N w l i A a , M ( z l , x i ) f ( k ) ( x i ) − f ( k ) ( z l ) \eta_l^k=\frac{1}{N}\sum_{i=1}^{N}w_{li}A_{a,M}(z_l,x_i)f^{(k)}(x_i)-f^{(k)}(z_l) η l k = N 1 i = 1 ∑ N w l i A a , M ( z l , x i ) f ( k ) ( x i ) − f ( k ) ( z l )
μ ^ l = 1 K ∑ k = 1 K η l k , Γ ^ l m = 1 K ∑ k = 1 K ( u ^ l − η l k ) ( u ^ m − q M k ) ∗ , \hat{\mu}_l=\frac{1}{K}\sum_{k=1}^{K}\eta_l^k,\quad\hat{\Gamma}_{lm}=\frac{1}{K}\sum_{k=1}^{K}(\hat{u}_l-\eta_l^k)(\hat{u}_m-q_M^k)^*, μ ^ l = K 1 k = 1 ∑ K η l k , Γ ^ l m = K 1 k = 1 ∑ K ( u ^ l − η l k ) ( u ^ m − q M k ) ∗ ,
we get our practical loss as
L W , d a t a = λ 1 L ∑ l = 1 L ∣ μ ^ l ∣ 2 + λ 2 L ( L − 1 ) ∑ l ≠ m ∣ Γ ^ l m ∣ 2 + λ 3 L ∑ l = 1 L ∣ Γ ^ l l ∣ 2 + λ 4 ∥ W ∥ 2 . L_{W,data}=\frac{\lambda_1}{L}\sum_{l=1}^{L}|\hat{\mu}_l|^2+\frac{\lambda_2}{L(L-1)}\sum_{l\ne m}|\hat{\Gamma}_{lm}|^2+\frac{\lambda_3}{L}\sum_{l=1}^{L}|\hat{\Gamma}_{ll}|^2+\lambda_4\|W\|^2. L W , d a t a = L λ 1 l = 1 ∑ L ∣ μ ^ l ∣ 2 + L ( L − 1 ) λ 2 l = m ∑ ∣ Γ ^ l m ∣ 2 + L λ 3 l = 1 ∑ L ∣ Γ ^ l l ∣ 2 + λ 4 ∥ W ∥ 2 .
Step 2: Complex Input. Denote by
e a ( z ) = ( e 1 a ( z ) ⋯ e M a ( z ) ) t , \mathbf{e}^a(z)=\left(e_1^a(z)\ \cdots\ e_M^a(z)\right)^t, e a ( z ) = ( e 1 a ( z ) ⋯ e M a ( z ) ) t ,
we then define our complex inputs
c 0 k : = 1 L ∑ l = 1 L q M ( z l ) e a ( z l ) ‾ F ~ N k ( z l ) . (5) \mathbf{c}_0^k:=\frac{1}{L}\sum_{l=1}^{L}q_M(z_l)\overline{\mathbf{e}^a(z_l)}\widetilde{F}_N^k(z_l). \tag*{(5)} c 0 k := L 1 l = 1 ∑ L q M ( z l ) e a ( z l ) F N k ( z l ) . (5)
Note that the holomorphy of e β a e_\beta^a e β a , F ~ N k \widetilde{F}_N^k F N k and the complex conjugacy symmetry of { Z l } \{Z_l\} { Z l } imply that c k \mathbf{c}^k c k are real-valued M M M -vectors.
Step 3: The Neural Operator. Note that ∣ Λ M ∣ = M |\Lambda_M|=M ∣ Λ M ∣ = M , we want to set the latent space representation F N , 0 k F_{N,0}^k F N , 0 k such that
B a ( f ( k ) ) = f ( k ) ≈ F N , 0 k = ∑ β ∈ Λ M c β , 0 k e β a . B_a(f^{(k)})=f^{(k)}\approx F_{N,0}^k=\sum_{\beta\in\Lambda_M}c_{\beta,0}^k e_\beta^a. B a ( f ( k ) ) = f ( k ) ≈ F N , 0 k = β ∈ Λ M ∑ c β , 0 k e β a .
Here the concrete latent space is given by Span V M \operatorname{Span}V_M Span V M . It is equivalent to define the map from evaluation representations of f ( k ) f^{(k)} f ( k ) (and u ( k ) u^{(k)} u ( k ) ) to c k \mathbf{c}^k c k given in Step 2 , which can also considered as a map from L 2 ( R n ; R ) L^2(\mathbb{R}^n;\mathbb{R}) L 2 ( R n ; R ) to F a 2 ( C n ) F_a^2(\mathbb{C}^n) F a 2 ( C n ) ,
B ~ : R N → R M , ( f ( k ) ( x 1 ) ⋮ f ( k ) ( x N ) ) = f k ↦ c 0 k , B ~ : L 2 ( R n ; R ) → F a 2 ( C n ) , f ( k ) ↦ F N , 0 k . \widetilde{B}:\mathbb{R}^N\to\mathbb{R}^M,\quad
\begin{pmatrix}
f^{(k)}(x_1)\\
\vdots\\
f^{(k)}(x_N)
\end{pmatrix}
=f^k\mapsto\mathbf{c}_0^k,\quad
\widetilde{B}:L^2(\mathbb{R}^n;\mathbb{R})\to F_a^2(\mathbb{C}^n),\ f^{(k)}\mapsto F_{N,0}^k. B : R N → R M , f ( k ) ( x 1 ) ⋮ f ( k ) ( x N ) = f k ↦ c 0 k , B : L 2 ( R n ; R ) → F a 2 ( C n ) , f ( k ) ↦ F N , 0 k .
With inverse Segal-Bargmann transform, we would hope without learning process (as we proved in Theorem 5.1 below)
h β a = B a − 1 ( e β a ) ⇒ f ( k ) ( x ) ≈ B a − 1 ( F N , 0 k ) ( x ) = ∑ β ∈ Λ M c β , 0 k h β a ( x ) : = f N , 0 k ( x ) . h_\beta^a=B_a^{-1}(e_\beta^a)\Rightarrow f^{(k)}(x)\approx B_a^{-1}(F_{N,0}^k)(x)=\sum_{\beta\in\Lambda_M}c_{\beta,0}^k h_\beta^a(x):=f_{N,0}^k(x). h β a = B a − 1 ( e β a ) ⇒ f ( k ) ( x ) ≈ B a − 1 ( F N , 0 k ) ( x ) = β ∈ Λ M ∑ c β , 0 k h β a ( x ) := f N , 0 k ( x ) .
This induces a map from R M \mathbb{R}^M R M to R N \mathbb{R}^N R N by
B ~ − 1 : R M → R N , c 0 k ↦ c N , 0 k : = ( f N , 0 k ( x 1 ) ⋮ f N , 0 k ( x N ) ) , \widetilde{B}^{-1}:\mathbb{R}^M\to\mathbb{R}^N,\ \mathbf{c}_0^k\mapsto\mathbf{c}_{N,0}^k:=
\begin{pmatrix}
f_{N,0}^k(x_1)\\
\vdots\\
f_{N,0}^k(x_N)
\end{pmatrix}, B − 1 : R M → R N , c 0 k ↦ c N , 0 k := f N , 0 k ( x 1 ) ⋮ f N , 0 k ( x N ) ,
B ~ − 1 : F a 2 ( C n ) → L 2 ( R n ; R ) , F N , 0 k ( z ) = ∑ β ∈ Λ c β , 0 k e β a ( z ) ↦ ∑ β ∈ Λ M c β , 0 k h β a ( x ) = f N , 0 k ( x ) . \widetilde{B}^{-1}:F_a^2(\mathbb{C}^n)\to L^2(\mathbb{R}^n;\mathbb{R}),\ F_{N,0}^k(z)=\sum_{\beta\in\Lambda}c_{\beta,0}^k e_\beta^a(z)\mapsto\sum_{\beta\in\Lambda_M}c_{\beta,0}^k h_\beta^a(x)=f_{N,0}^k(x). B − 1 : F a 2 ( C n ) → L 2 ( R n ; R ) , F N , 0 k ( z ) = β ∈ Λ ∑ c β , 0 k e β a ( z ) ↦ β ∈ Λ M ∑ c β , 0 k h β a ( x ) = f N , 0 k ( x ) .
Relabeled our real input
v 0 , k = f ( k ) , c 0 k = B ~ ( f k ) = B ~ ( v 0 , k ) = d 0 , k . v^{0,k}=f^{(k)},\ \mathbf{c}_0^k=\widetilde{B}(f^k)=\widetilde{B}(v^{0,k})=\mathbf{d}^{0,k}. v 0 , k = f ( k ) , c 0 k = B ( f k ) = B ( v 0 , k ) = d 0 , k .
For the 1st Segal-Bargmann layer, we apply a m m m -layer MLP G θ 1 G_{\theta_1} G θ 1 to c 0 , k \mathbf{c}^{0,k} c 0 , k such that
R M ∋ η 1 , k = G θ 1 d 0 , k ⇒ η ~ 1 , k = B ~ − 1 ( η 1 , k ) . \mathbb{R}^{M} \ni\eta^{1,k}=G_{\theta_1}\mathbf{d}^{0,k}\Rightarrow\widetilde{\eta}^{1,k}=\widetilde{B}^{-1}(\eta^{1,k}). R M ∋ η 1 , k = G θ 1 d 0 , k ⇒ η 1 , k = B − 1 ( η 1 , k ) .
The map B ~ − 1 \widetilde{B}^{-1} B − 1 is exactly the same as given above, which means
η ~ 1 , k = B ~ − 1 ( η 1 , k ) = ( B a − 1 ( ∑ β ∈ Λ M η β 1 , k e β a ) ( x 1 ) ⋮ B a − 1 ( ∑ β ∈ Λ M η β 1 , k e β a ) ( x N ) ) . \widetilde{\eta}^{1,k}=\widetilde{B}^{-1}(\eta^{1,k})=
\begin{pmatrix}
B_a^{-1}\left(\sum_{\beta\in\Lambda_M}\eta_{\beta}^{1,k}e_{\beta}^{a}\right)(x_1)\\
\vdots\\
B_a^{-1}\left(\sum_{\beta\in\Lambda_M}\eta_{\beta}^{1,k}e_{\beta}^{a}\right)(x_N)
\end{pmatrix}. η 1 , k = B − 1 ( η 1 , k ) = B a − 1 ( ∑ β ∈ Λ M η β 1 , k e β a ) ( x 1 ) ⋮ B a − 1 ( ∑ β ∈ Λ M η β 1 , k e β a ) ( x N ) .
Adding a linear part and a nonlinear activation σ \sigma σ (taking GeLU or ReLU), we have the output of the first Segal-Bargmann layer, or the input of the second Segal-Bargmann layer
v 1 , k = σ ( W 0 v 0 , k + η ~ 1 , k ) = σ ( W 0 v 0 , k + B ~ − 1 G θ 1 d 0 , k ) = σ ( W 0 v 0 , k + η ~ 1 , k ) = σ ( W 0 v 0 , k + B ~ − 1 G θ 1 B ~ v 0 , k ) . \begin{aligned}
\mathbf{v}^{1,k}=\sigma(W_0\mathbf{v}^{0,k}+\widetilde{\eta}^{1,k})=\sigma(W_0\mathbf{v}^{0,k}+\widetilde{B}^{-1}G_{\theta_1}\mathbf{d}^{0,k}) \\
&=\sigma(W_0\mathbf{v}^{0,k}+\widetilde{\eta}^{1,k})=\sigma(W_0\mathbf{v}^{0,k}+\widetilde{B}^{-1}G_{\theta_1}\widetilde{B}\mathbf{v}^{0,k}).
\end{aligned} v 1 , k = σ ( W 0 v 0 , k + η 1 , k ) = σ ( W 0 v 0 , k + B − 1 G θ 1 d 0 , k ) = σ ( W 0 v 0 , k + η 1 , k ) = σ ( W 0 v 0 , k + B − 1 G θ 1 B v 0 , k ) .
Or in the sense of functions
v 1 , k ( x ) = σ ( W 0 v 0 , k ( x ) + B − 1 ( G θ 1 B ~ ( v 0 , k ) ( Z l ) ) ( x ) ) . v^{1,k}(x)=\sigma(W_0v^{0,k}(x)+B^{-1}(G_{\theta_1}\widetilde{B}(v^{0,k})(Z_l))(x)). v 1 , k ( x ) = σ ( W 0 v 0 , k ( x ) + B − 1 ( G θ 1 B ( v 0 , k ) ( Z l )) ( x )) .
Then we can do the second Segal-Bargmann layer, and get either complex output
d 2 , k = G θ 2 B ~ ( v 1 , k ) . \mathbf{d}^{2,k}=G_{\theta_2}\widetilde{B}(\mathbf{v}^{1,k}). d 2 , k = G θ 2 B ( v 1 , k ) .
or we can get another real output
v 2 , k = σ ( W 1 v 1 , k + B ~ − 1 G θ 1 B ~ v 1 , k ) . \mathbf{v}^{2,k}=\sigma(W_1\mathbf{v}^{1,k}+\widetilde{B}^{-1}G_{\theta_1}\widetilde{B}\mathbf{v}^{1,k}). v 2 , k = σ ( W 1 v 1 , k + B − 1 G θ 1 B v 1 , k ) .
or in the sense of function
v 2 , k ( x ) = σ ( W 1 v 1 , k ( x ) + B ~ − 1 ( G θ 2 B ~ ( v 1 , k ) ( Z l ) ( x ) ) . v^{2,k}(x)=\sigma(W_1v^{1,k}(x)+\widetilde{B}^{-1}(G_{\theta_2}\widetilde{B}(v^{1,k})(Z_l)(x)). v 2 , k ( x ) = σ ( W 1 v 1 , k ( x ) + B − 1 ( G θ 2 B ( v 1 , k ) ( Z l ) ( x )) .
And we can keep going to get a total of S S S Segal-Bargmann layers, where the complex output and real output are denoted by
d S , k = G θ S B ~ ( v S − 1 , k ) , \mathbf{d}^{S,k}=G_{\theta_S}\widetilde{B}(\mathbf{v}^{S-1,k}), d S , k = G θ S B ( v S − 1 , k ) ,
v S , k ( x ) = W S − 1 , k v S − 1 , k ( x ) + B ~ − 1 ( G θ S B ~ ( v S − 1 , k ) ( Z l ) ( x ) . (6) v^{S,k}(x)=W_{S-1,k}v^{S-1,k}(x)+\widetilde{B}^{-1}(G_{\theta_S}\widetilde{B}(v^{S-1,k})(Z_l)(x). \tag*{(6)} v S , k ( x ) = W S − 1 , k v S − 1 , k ( x ) + B − 1 ( G θ S B ( v S − 1 , k ) ( Z l ) ( x ) . (6)
Step 4: The Real Relative L 2 L^2 L 2 -Loss. By the expression of outputs in (6), we define a relative L 2 L^2 L 2 -loss in the original real space. Set
u ^ k : = v S , k ⇒ u ^ k : = G Θ v 0 , k = G Θ f ( k ) , \widehat{u}^{k}:=v^{S,k}\Rightarrow\widehat{u}^{k}:=G_{\Theta}v^{0,k}=G_{\Theta}f^{(k)}, u k := v S , k ⇒ u k := G Θ v 0 , k = G Θ f ( k ) ,
we have
L M S E , r e a l = 1 K ∑ k = 1 K ∑ i = 1 N ∣ u ^ k ( x i ) − u ( k ) ( x i ) ∣ 2 ∑ i = 1 N ∣ u ( k ) ( x i ) ∣ 2 . (7) L_{MSE,real}=\frac{1}{K}\sum_{k=1}^{K}\frac{\sum_{i=1}^{N}|\widehat{u}^{k}(x_i)-u^{(k)}(x_i)|^2}{\sum_{i=1}^{N}|u^{(k)}(x_i)|^2}. \tag*{(7)} L M S E , r e a l = K 1 k = 1 ∑ K ∑ i = 1 N ∣ u ( k ) ( x i ) ∣ 2 ∑ i = 1 N ∣ u k ( x i ) − u ( k ) ( x i ) ∣ 2 . (7)
Step 5: Testing.
Suppose that we have testing set { u t e s t k , f t e s t k } \{u_{\mathrm{test}}^{k},f_{\mathrm{test}}^{k}\} { u test k , f test k } . Let’s assume that our complex trained output is U ^ t e s t k \widehat{U}_{\mathrm{test}}^{k} U test k constructed by d ^ t e s t k = G ~ Θ d t e s t 0 , k \widehat{\mathbf{d}}_{\mathrm{test}}^{k}= \widetilde{G}_{\Theta}\mathbf{d}_{\mathrm{test}}^{0,k} d test k = G Θ d test 0 , k where the input is d t e s t 0 , k = c t e s t k \mathbf{d}_{\mathrm{test}}^{0,k}=\mathbf{c}_{\mathrm{test}}^{k} d test 0 , k = c test k coming from (5) with respect to complex grids { Z l } l = 1 N \{Z_l\}_{l=1}^{N} { Z l } l = 1 N as given above.
It follows that
u ^ t e s t , r k = G Θ f t e s t k . \widehat{u}_{\mathrm{test},r}^{k}=G_{\Theta}f_{\mathrm{test}}^{k}. u test , r k = G Θ f test k .
It follows that the testing relative l 2 l^2 l 2 error is given by
E r r r = 1 K ∑ k = 1 K ∑ i = 1 N ∣ u ^ t e s t , r k ( x i ) − u t e s t k ( x i ) ∣ 2 ∑ i = 1 N ∣ u t e s t k ( x i ) ∣ 2 . Err_r=\frac{1}{K}\sum_{k=1}^{K}\frac{\sum_{i=1}^{N}|\widehat{u}_{\mathrm{test},r}^{k}(x_i)-u_{\mathrm{test}}^{k}(x_i)|^2}{\sum_{i=1}^{N}|u_{\mathrm{test}}^{k}(x_i)|^2}. E r r r = K 1 k = 1 ∑ K ∑ i = 1 N ∣ u test k ( x i ) ∣ 2 ∑ i = 1 N ∣ u test , r k ( x i ) − u test k ( x i ) ∣ 2 .
We can replace { x i } \{x_i\} { x i } by { y j } \{y_j\} { y j } to test the error in terms of the generalization of the NO.
One-Step End-to-End Algorithm.
The only change we make is to replace the pre-training encoder W W W by a neural network W θ ′ W_{\theta'} W θ ′ , therefore (5) becomes
c 0 k : = 1 L ∑ l = 1 L q M ( z l ) e a ( z l ) F ~ N , θ ′ k ( z l ) . (8) c_0^k := \frac{1}{L}\sum_{l=1}^{L}q_M(z_l)e^{a}(z_l)\widetilde{F}_{N,\theta'}^k(z_l). \tag*{(8)} c 0 k := L 1 l = 1 ∑ L q M ( z l ) e a ( z l ) F N , θ ′ k ( z l ) . (8)
where
F ~ N , θ ′ k ( z ) : = 1 N ∑ i = 1 N w l i , θ ′ A a , M ( z , x i ) f ( k ) ( x i ) . \widetilde{F}_{N,\theta'}^k(z):=\frac{1}{N}\sum_{i=1}^{N}w_{li,\theta'}A_{a,M}(z,x_i)f^{(k)}(x_i). F N , θ ′ k ( z ) := N 1 i = 1 ∑ N w l i , θ ′ A a , M ( z , x i ) f ( k ) ( x i ) .
Everything else are exactly the same as in §4.3. The encoding neural network only applies to the construction of the initial complex inputs. We then apply the loss (7) to optimize both θ ′ \theta' θ ′ and Θ \Theta Θ .
Summary of Algorithms.
The two algorithms are summarized below:
Algorithm 1 Two-step training of the Segal–Bargmann neural operator
Input: Training observations { f ( k ) ( x i ) , u ( k ) ( x i ) } i , k \{f^{(k)}(x_i),u^{(k)}(x_i)\}_{i,k} { f ( k ) ( x i ) , u ( k ) ( x i ) } i , k ; parameters a a a , M M M , L L L , S S S with even L L L .
Output: Encoder W W W and operator parameters Θ \Theta Θ .
Sample z 1 , … , z L / 2 z_1,\ldots,z_{L/2} z 1 , … , z L /2 independently from ρ a \rho_a ρ a and set z l + L / 2 = z l ‾ z_{l+L/2}=\overline{z_l} z l + L /2 = z l .
Minimize L W , d a t a L_{W,\mathrm{data}} L W , data over W W W , freeze W W W , and construct the encoder B ~ \widetilde{B} B .
Form u ^ k = G Θ f ( k ) \widehat{u}^k=G_\Theta f^{(k)} u k = G Θ f ( k ) using S S S Segal–Bargmann layers and minimize L M S E , r e a l L_{\mathrm{MSE,real}} L MSE , real over Θ \Theta Θ .
Algorithm 2 End-to-end training of the Segal–Bargmann neural operator
Input: Training observations and parameters as above; encoding network W θ ′ W_{\theta'} W θ ′ .
Output: Trained parameters θ ′ \theta' θ ′ and Θ \Theta Θ .
Sample conjugate-paired complex points as in Algorithm 1.
Construct the initial coefficients c 0 k c_0^k c 0 k using W θ ′ W_{\theta'} W θ ′ ; keep the encoders in subsequent layers fixed.
Form u ^ k = G Θ f ( k ) \widehat{u}^k=G_\Theta f^{(k)} u k = G Θ f ( k ) and minimize L M S E , r e a l L_{\mathrm{MSE,real}} L MSE , real jointly over θ ′ \theta' θ ′ and Θ \Theta Θ .
L ∞ L^\infty L ∞ -Convergence and Independence of The Dimension
In this section, we consider the convergence rate between the estimate u ^ \widehat{u} u and the ground truth solution u u u of the PDE L u = f \mathcal{L}u=f L u = f . For notational simplicity, we give proofs only for elliptic smooth differential operator L \mathcal{L} L . By selecting the appropriate functional spaces, the analysis for time-dependent smooth differential operators is essentially the same, expect notationally more complicated. Recall that we only have finitely many observations { f ( x i ) } \{f(x_i)\} { f ( x i )} . In §5.1, we consider whether the unknown ground truth F = B a ( f ) F=\mathcal{B}_a(f) F = B a ( f ) can be approximated by a finite expansion F ^ = ∑ β ∈ Λ M c ^ β e β a \widehat{F}=\sum_{\beta\in\Lambda_M}\widehat{c}_\beta e_\beta^a F = ∑ β ∈ Λ M c β e β a determined by both complex grids { z l } l = 1 L \{z_l\}_{l=1}^{L} { z l } l = 1 L and observations { f ( x i ) } i = 1 N \{f(x_i)\}_{i=1}^{N} { f ( x i ) } i = 1 N . F ^ \widehat{F} F serves as an RKHS approximation of F F F . In §5.2, we give L ∞ L^\infty L ∞ -type and L 2 L^2 L 2 -type convergence rate between u ^ \widehat{u} u and u u u , in terms of the size of the complex grids { z l } l = 1 L \{z_l\}_{l=1}^{L} { z l } l = 1 L . Under the stated hypotheses, the L − 1 / 2 L^{-1/2} L − 1/2 reconstruction errors are dimension-free, and are independent of the size of real grids { x i } i = 1 N \{x_i\}_{i=1}^{N} { x i } i = 1 N .
Let f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) f\in L^2(\mathbb{R}^n;\mathbb{R})\cap C^\infty(\mathbb{R}^n;\mathbb{R}) f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) and suppose that we only know the evaluations of f f f at finitely many grids { x i } i = 1 N \{x_i\}_{i=1}^{N} { x i } i = 1 N . In this section, we show that how to recover f f f from { f ( x i ) } i = 1 N \{f(x_i)\}_{i=1}^{N} { f ( x i ) } i = 1 N by introducing complex sampling grids on C n \mathbb{C}^n C n , provided that a very limited prior knowledge about the evaluation representations { x i } i = 1 N \{x_i\}_{i=1}^{N} { x i } i = 1 N is given.
Theoretically, any f ∈ L 2 ( R n ; R ) f \in L^2(\mathbb{R}^n;\mathbb{R}) f ∈ L 2 ( R n ; R ) can be written as
f = ∑ β c β h β a , ∥ f ∥ L 2 2 = ∑ β ∣ c β ∣ 2 . f = \sum_{\beta} c_\beta h_\beta^a,\quad\|f\|_{L^2}^2 = \sum_{\beta} |c_\beta|^2. f = β ∑ c β h β a , ∥ f ∥ L 2 2 = β ∑ ∣ c β ∣ 2 .
in the L 2 L^2 L 2 -sense where
c β = ⟨ f , h β a ⟩ L 2 ( R n ; R ) , c : = ( c 1 , … , c β , … , c M ) . c_\beta= \langle f,h_\beta^a\rangle_{L^2(\mathbb{R}^n;\mathbb{R})},\quad\mathbf{c}:=(c_1,\ldots,c_\beta,\ldots,c_M). c β = ⟨ f , h β a ⟩ L 2 ( R n ; R ) , c := ( c 1 , … , c β , … , c M ) .
by orthonormality of { h β a } \{h_\beta^a\} { h β a } . Numerically, it is always one of the biggest problems of how to project an infinite dimensional space onto a suitable finite dimensional subspace. Practically, we have to choose finitely many elements to form a finite linear combination of { h β a } \{h_\beta^a\} { h β a } in the sense that
P M f : = ∑ β ∈ Λ M c β h β a , ∥ f − P M f ∥ L 2 ( R n ; R ) → 0 as M → ∞ . P_Mf := \sum_{\beta\in\Lambda_M} c_\beta h_\beta^a,\quad\|f-P_Mf\|_{L^2(\mathbb{R}^n;\mathbb{R})}\to0\text{ as }M\to\infty. P M f := β ∈ Λ M ∑ c β h β a , ∥ f − P M f ∥ L 2 ( R n ; R ) → 0 as M → ∞.
If we have complete information about f f f , then each c β = ⟨ f , h β a ⟩ L 2 c_\beta=\langle f,h_\beta^a\rangle_{L^2} c β = ⟨ f , h β a ⟩ L 2 in the sense of L 2 L^2 L 2 -inner product. But with only { f ( x i ) } i = 1 N \{f(x_i)\}_{i=1}^N { f ( x i ) } i = 1 N , the standard way is to approximate f f f by ∑ β ∈ Λ M c β h β a \sum_{\beta\in\Lambda_M}c_\beta h_\beta^a ∑ β ∈ Λ M c β h β a via the discrete least square
b ~ = argmin d ( 1 N ∑ i = 1 N ( f ( x i ) − ∑ β ∈ Λ M d β h β a ( x i ) ) ) 2 . (9) \widetilde{\mathbf b}=\mathop{\operatorname{argmin}}_{\mathbf d}\left(\frac{1}{N}\sum_{i=1}^N\left(f(x_i)-\sum_{\beta\in\Lambda_M}d_\beta h_\beta^a(x_i)\right)\right)^2. \tag*{(9)} b = argmin d N 1 i = 1 ∑ N f ( x i ) − β ∈ Λ M ∑ d β h β a ( x i ) 2 . (9)
The other way is to estimate c β c_\beta c β via naive discrete sum
c ~ = ( c ~ 1 , … , c ~ β , … , c ~ M ) , c ~ β = 1 N ∑ i = 1 N f ( x i ) h β a ( x i ) (10) \widetilde{\mathbf c}=(\widetilde c_1,\ldots,\widetilde c_\beta,\ldots,\widetilde c_M),\quad\widetilde c_\beta=\frac{1}{N}\sum_{i=1}^N f(x_i)h_\beta^a(x_i) \tag*{(10)} c = ( c 1 , … , c β , … , c M ) , c β = N 1 i = 1 ∑ N f ( x i ) h β a ( x i ) (10)
This unweighted sum does not in general approximate the Lebesgue inner product without suitable sampling or quadrature weights. Set the M × M M\times M M × M matrix H = ( H ) β , β ′ H=(H)_{\beta,\beta'} H = ( H ) β , β ′ as
H = ( ∑ i = 1 N h 1 a ( x i ) h 1 a ( x i ) ⋯ ∑ i = 1 N h 1 a ( x i ) h M a ( x i ) ⋮ ⋱ ⋮ ∑ i = 1 N h M a ( x i ) h 1 a ( x i ) ⋯ ∑ i = 1 N h M a ( x i ) h M a ( x i ) ) . (11) H=\begin{pmatrix}
\sum_{i=1}^N h_1^a(x_i)h_1^a(x_i) & \cdots& \sum_{i=1}^N h_1^a(x_i)h_M^a(x_i)\\
\vdots& \ddots& \vdots\\
\sum_{i=1}^N h_M^a(x_i)h_1^a(x_i) & \cdots& \sum_{i=1}^N h_M^a(x_i)h_M^a(x_i)
\end{pmatrix}. \tag*{(11)} H = ∑ i = 1 N h 1 a ( x i ) h 1 a ( x i ) ⋮ ∑ i = 1 N h M a ( x i ) h 1 a ( x i ) ⋯ ⋱ ⋯ ∑ i = 1 N h 1 a ( x i ) h M a ( x i ) ⋮ ∑ i = 1 N h M a ( x i ) h M a ( x i ) . (11)
If H H H is invertible,
b ~ β = ∑ β ′ ∈ Λ M ( H − 1 ) β , β ′ ∑ i = 1 N f ( x i ) h β ′ a ( x i ) = ∑ β ′ ∈ Λ M N ( H − 1 ) β , β ′ ( 1 N ∑ i = 1 N f ( x i ) h β ′ a ( x i ) ) = ∑ β ′ ∈ Λ M N ( H − 1 ) β , β ′ c ~ β ′ . \widetilde b_\beta=\sum_{\beta'\in\Lambda_M}(H^{-1})_{\beta,\beta'}\sum_{i=1}^N f(x_i)h_{\beta'}^a(x_i)=\sum_{\beta'\in\Lambda_M}N(H^{-1})_{\beta,\beta'}\left(\frac{1}{N}\sum_{i=1}^N f(x_i)h_{\beta'}^a(x_i)\right)=\sum_{\beta'\in\Lambda_M}N(H^{-1})_{\beta,\beta'}\widetilde c_{\beta'}. b β = β ′ ∈ Λ M ∑ ( H − 1 ) β , β ′ i = 1 ∑ N f ( x i ) h β ′ a ( x i ) = β ′ ∈ Λ M ∑ N ( H − 1 ) β , β ′ ( N 1 i = 1 ∑ N f ( x i ) h β ′ a ( x i ) ) = β ′ ∈ Λ M ∑ N ( H − 1 ) β , β ′ c β ′ .
From this, we get
∥ c ~ − c ∥ R M = ∥ H N b ~ − c ∥ R M ≤ ∥ H N ( b ~ − c ) ∥ R M + ∥ ( H N − I ) c ∥ R M . \|\widetilde{\mathbf c}-\mathbf c\|_{\mathbb{R}^M}=\left\|\frac{H}{N}\widetilde{\mathbf b}-\mathbf c\right\|_{\mathbb{R}^M}\leq\left\|\frac{H}{N}(\widetilde{\mathbf b}-\mathbf c)\right\|_{\mathbb{R}^M}+\left\|\left(\frac{H}{N}-I\right)\mathbf c\right\|_{\mathbb{R}^M}. ∥ c − c ∥ R M = N H b − c R M ≤ N H ( b − c ) R M + ( N H − I ) c R M .
If ∥ b ~ − c ∥ = O ( 1 ) \|\widetilde{\mathbf b}-\mathbf c\|=O(1) ∥ b − c ∥ = O ( 1 ) and 1 N ∥ H ∥ = O ( 1 ) \frac{1}{N}\|H\|=O(1) N 1 ∥ H ∥ = O ( 1 ) , then it implies that ∥ c ~ − c ∥ = O ( 1 ) \|\widetilde{\mathbf c}-\mathbf c\|=O(1) ∥ c − c ∥ = O ( 1 ) . The hypothesis of 1 N ∥ H ∥ = O ( 1 ) \frac{1}{N}\|H\|=O(1) N 1 ∥ H ∥ = O ( 1 ) is a plausible assumption. If the empirical inner products approximate the L 2 L^2 L 2 inner product ⟨ h β a , h β ′ a ⟩ L 2 = δ β , β ′ \langle h_\beta^a,h_{\beta'}^a\rangle_{L^2}=\delta_{\beta,\beta'} ⟨ h β a , h β ′ a ⟩ L 2 = δ β , β ′ . Therefore, there are grids { x i } \{x_i\} { x i } such that
1 N ∑ i = 1 N h β a ( x i ) h β ′ a ( x i ) ≈ δ β , β ′ ⇒ H ≈ N I M × M . \frac{1}{N}\sum_{i=1}^N h_\beta^a(x_i)h_{\beta'}^a(x_i)\approx\delta_{\beta,\beta'}\Rightarrow H\approx NI_{M\times M}. N 1 i = 1 ∑ N h β a ( x i ) h β ′ a ( x i ) ≈ δ β , β ′ ⇒ H ≈ N I M × M .
Our hypothesis says that the symmetric matrix H H H is not too far away from an invertible matrix whose largest eigenvalue is O ( N ) O(N) O ( N ) , i.e. H H H is not severely ill-conditioned. Analogously,
∥ b ~ − c ∥ R M = ∥ N H − 1 c ~ − c ∥ R M ≤ ∥ N H − 1 ( c ~ − c ) ∥ R M + ∥ ( N H − 1 − I ) c ∥ R M . \|\widetilde{\mathbf b}-\mathbf c\|_{\mathbb{R}^M}=\|NH^{-1}\widetilde{\mathbf c}-\mathbf c\|_{\mathbb{R}^M}\leq\|NH^{-1}(\widetilde{\mathbf c}-\mathbf c)\|_{\mathbb{R}^M}+\|(NH^{-1}-I)\mathbf c\|_{\mathbb{R}^M}. ∥ b − c ∥ R M = ∥ N H − 1 c − c ∥ R M ≤ ∥ N H − 1 ( c − c ) ∥ R M + ∥ ( N H − 1 − I ) c ∥ R M .
If ∥ c ~ − c ∥ R M = O ( 1 ) \|\widetilde{\mathbf c}-\mathbf c\|_{\mathbb{R}^M}=O(1) ∥ c − c ∥ R M = O ( 1 ) and N ∥ H − 1 ∥ = O ( 1 ) N\|H^{-1}\|=O(1) N ∥ H − 1 ∥ = O ( 1 ) , then ∥ b ~ − c ∥ R M = O ( 1 ) \|\widetilde{\mathbf b}-\mathbf c\|_{\mathbb{R}^M}=O(1) ∥ b − c ∥ R M = O ( 1 ) .
The discussion above indicates that the matrix H H H characterized by the evaluations of orthonormal basis { h β a } \{h_\beta^a\} { h β a } is crucial. But it is more important to have prior knowledge of how grids { x i } \{x_i\} { x i } are distributed as the basic example above shows. However, the best we can say is that { x i } \{x_i\} { x i } are samples of random variables { X i } \{X_i\} { X i } . We cannot say more about the means and variances of { X i } \{X_i\} { X i } .
The above discussion also indicates that the truncation of the orthonormal expansion of f f f is also crucial. These motivate the following two assumption:
Assumption A: Let f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) f \in L^2(\mathbb{R}^n;\mathbb{R}) \cap C^\infty(\mathbb{R}^n;\mathbb{R}) f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) . We say that f f f is accessible if there exists a finite index set Λ M = { β ∈ N 0 n : ∣ Λ M ∣ = M } = { β 1 , … , β M } \Lambda_M = \{\beta\in\mathbb{N}_0^n : |\Lambda_M| = M\} = \{\beta_1,\ldots,\beta_M\} Λ M = { β ∈ N 0 n : ∣ Λ M ∣ = M } = { β 1 , … , β M } and associated finite collection of orthonormal basis V M = { h β a : β ∈ Λ M } V_M = \{h_\beta^a : \beta\in\Lambda_M\} V M = { h β a : β ∈ Λ M } such that
f = ∑ β ∈ Λ M c β h β a ⟺ f ∈ Span V M f = \sum_{\beta\in\Lambda_M} c_\beta h_\beta^a \Longleftrightarrow f \in\operatorname{Span} V_M f = β ∈ Λ M ∑ c β h β a ⟺ f ∈ Span V M
in the L 2 L^2 L 2 -sense. It is plausible e.g. under the low-dimensional sparsity assumption in [7 ] .
Remark 5.1. The Assumption A is very restrictive. However, we will give dimension-free reconstruction error with this assumption for the purpose of notational simplicity. We will then give results of the reconstruction error in the most general setting, especially without this assumption.
Assumption B: Let F = B a ( f ) F = \mathcal{B}_a(f) F = B a ( f ) . We say that the information f f f is recoverable from finite observations { f ( x i ) } i = 1 N \{f(x_i)\}_{i=1}^N { f ( x i ) } i = 1 N if for some collection of complex random variables { Z l } l = 1 N ∼ i . i . d . ρ C ( z ) \{Z_l\}_{l=1}^N \overset{\mathrm{i.i.d.}}{\sim} \rho_{\mathbb{C}}(z) { Z l } l = 1 N ∼ i.i.d. ρ C ( z ) , there exists a random matrix W = ( w l i ) ∈ C L × N W = (w_{li}) \in\mathbb{C}^{L \times N} W = ( w l i ) ∈ C L × N (called the information recovery matrix ) such that for our complex approximation
F ~ N ( Z l ) : = 1 N ∑ i = 1 N w l i A a ( Z l , X i ) f ( X i ) , (12) \widetilde{F}_N(Z_l) := \frac{1}{N}\sum_{i=1}^N w_{li}\mathcal{A}_a(Z_l,X_i)f(X_i), \tag*{(12)} F N ( Z l ) := N 1 i = 1 ∑ N w l i A a ( Z l , X i ) f ( X i ) , (12)
the difference between the approximation F ~ N \widetilde{F}_N F N and unknown ground truth F F F at Z l Z_l Z l
η l : = F ( Z l ) − F ~ N ( Z l ) \eta_l := F(Z_l) - \widetilde{F}_N(Z_l) η l := F ( Z l ) − F N ( Z l )
satisfies the following conditions, where Z = ( Z 1 , … , Z L ) Z = (Z_1,\ldots,Z_L) Z = ( Z 1 , … , Z L ) denotes the full collection of complex samples:
E X ∣ Z [ η l ] = 0 , E X ∣ Z [ η l η m ‾ ] = σ l m 2 δ l m , σ l m 2 ≤ ( C ′ ) 2 < ∞ . (13) \mathbb{E}_{X|Z}[\eta_l] = 0, \mathbb{E}_{X|Z}[\eta_l\overline{\eta_m}] = \sigma_{lm}^2\delta_{lm},\ \sigma_{lm}^2 \le(C')^2 < \infty. \tag*{(13)} E X ∣ Z [ η l ] = 0 , E X ∣ Z [ η l η m ] = σ l m 2 δ l m , σ l m 2 ≤ ( C ′ ) 2 < ∞. (13)
Remark 5.2. For a fixed training set, [14 Theorem 3.2] proves convergence of the total training loss to zero for a sufficiently wide two-layer ReLU network under its stated assumptions. We modify their proof and show that Assumption B can be achieved under some hypotheses. The proof can be found in Appendix. Therefore, it is reasonable to assume Assumption B .
We state several basic cases when Assumption B are satisfied in Remark 5.3 below. We also give two algorithm for the construction of W W W in § 4.3 and § 4.4 . We also explain why (13) is a reasonable goal under the Segal-Bargmann framework in § 4.2 .
More importantly, we can treat W W W as a neural network for the purpose of information recovery from data in the original space to the information in the Bargmann-Fock space by grids on the phase space from physics point of view. We then give an end-to-end algorithm which combines information recovery and operator learning.
Remark 5.3. Suppose that we estimate f f f by ∑ β ∈ Λ M c ^ β h β a \sum_{\beta\in\Lambda_M}\widehat{c}_\beta h_\beta^a ∑ β ∈ Λ M c β h β a from our observations { f ( x i ) } \{f(x_i)\} { f ( x i )} . Recall that c ^ β \widehat{c}_\beta c β is our naive guess of the ground truth c β = ⟨ f , h β a ⟩ L 2 ( R n ; R ) c_\beta= \langle f,h_\beta^a\rangle_{L^2(\mathbb{R}^n;\mathbb{R})} c β = ⟨ f , h β a ⟩ L 2 ( R n ; R ) . Set
f ^ : = ∑ β ∈ Λ M c ^ β h β a , f ~ : = ∑ β ∈ Λ M c ~ β h β a . \widehat{f} := \sum_{\beta\in\Lambda_M}\widehat{c}_\beta h_\beta^a,\quad\widetilde{f} := \sum_{\beta\in\Lambda_M}\widetilde{c}_\beta h_\beta^a. f := β ∈ Λ M ∑ c β h β a , f := β ∈ Λ M ∑ c β h β a .
We have
∣ f − f ^ ∣ ≤ ∣ f − P M f ∣ + ∣ P M f − f ~ ∣ + ∣ f ~ − f ^ ∣ . |f-\widehat{f}| \le|f-P_Mf| + |P_Mf-\widetilde{f}| + |\widetilde{f}-\widehat{f}|. ∣ f − f ∣ ≤ ∣ f − P M f ∣ + ∣ P M f − f ∣ + ∣ f − f ∣.
Either the least square (9) or the approximation of the inner product (10) recovers the information of the projection of f f f onto V M V_M V M . We definitely loss the information of f f f on ( Span ( V M ) ) ⊥ (\operatorname{Span}(V_M))^\perp ( Span ( V M ) ) ⊥ .
The Assumption A is very restrictive, which gives a brutal control of the first term in the inequality above. The convergence ∥ f − P M f ∥ L 2 ( R n ; R ) → M → ∞ 0 \lVert f-P_Mf\rVert_{L^2(\mathbb{R}^n;\mathbb{R})} \xrightarrow{M\to\infty} 0 ∥ f − P M f ∥ L 2 ( R n ; R ) M → ∞ 0 motivates finite-dimensional approximation, although exact finite representability in Assumption A is stronger. But it is not enough since our goal is to give L ∞ L^\infty L ∞ -norm of f − f ^ f-\hat{f} f − f ^ , as the L 2 L^2 L 2 -sense tail estimate usually aligns with the curse of dimensionality.
In Lemma 5.1 below, we show that if f f f has enough regularity, and for any spatial dimension n n n and any truncation M M M , there exists an appropriate a a a such that the sup x ∈ K ∣ f − P M f ∣ \sup_{x\in K}|f-P_Mf| sup x ∈ K ∣ f − P M f ∣ is small enough for any compact set K K K given a priori.
Note that the Gauss-Hermite polynomials are nonlinear eigenfunctions of the Fourier transform and hence the highest order of a Gauss-Hermite polynomial reflects the highest frequency it represents. In practice, we may always assume that the functions that characterize our observable data have finite maximum frequencies. Note that a finite Hermite expansion in Assumption A is not exactly band-limited.
We still need to control the second and the third terms in the inequality above. The last term may be tackled by different machine learning approaches such as neural networks.
The gap is the difference between the ground truth P M f P_Mf P M f and our data-driven recovery f ~ \tilde{f} f ~ . If we know the invertibility of H H H with respect to { x i } \{x_i\} { x i } , then we can recover c c c directly. In this case, we get
w l i = N e a ( Z l ) t H − 1 h a ( X i ) A a , M ( Z l , X i ) , ∀ i , l w_{li}=\frac{N e^{a}(Z_l)^{t}H^{-1}h^{a}(X_i)}{A_{a,M}(Z_l,X_i)},\quad\forall i,l w l i = A a , M ( Z l , X i ) N e a ( Z l ) t H − 1 h a ( X i ) , ∀ i , l
where
e a ( z ) = ( e 1 a ( z ) ⋮ e M a ( z ) ) , h a ( z ) = ( h 1 a ( z ) ⋮ h M a ( z ) ) , A a , M ( z , x ) : = P M A a ( z , x ) = ∑ β ∈ Λ M e β a ( z ) h β a ( x ) . e^{a}(z)=\begin{pmatrix}e^{a}_{1}(z)\\ \vdots\\ e^{a}_{M}(z)\end{pmatrix},\quad h^{a}(z)=\begin{pmatrix}h^{a}_{1}(z)\\ \vdots\\ h^{a}_{M}(z)\end{pmatrix},\quad A_{a,M}(z,x):=P_MA_a(z,x)=\sum_{\beta\in\Lambda_M}e^{a}_{\beta}(z)h^{a}_{\beta}(x). e a ( z ) = e 1 a ( z ) ⋮ e M a ( z ) , h a ( z ) = h 1 a ( z ) ⋮ h M a ( z ) , A a , M ( z , x ) := P M A a ( z , x ) = β ∈ Λ M ∑ e β a ( z ) h β a ( x ) .
When { x i } \{x_i\} { x i } are sampling from i.i.d. random variables { X i } i = 1 N ∼ i . i . d . ρ \{X_i\}_{i=1}^{N}\mathrel{\overset{\mathrm{i.i.d.}}{\sim}}\rho { X i } i = 1 N ∼ i.i.d. ρ , and N N N is large enough, Assumption B holds by taking Monte-Carlo sampling with
w l i = 1 ρ ( X i ) , ∀ l . w_{li}=\frac{1}{\rho(X_i)},\quad\forall l. w l i = ρ ( X i ) 1 , ∀ l .
In practice, these are not the cases in general. However, the two basic examples above illustrate the feasibility of our Assumption B . This is one of our main purposes of the introduction of the α \alpha α -Segal-Bargmann transform and the α \alpha α -Bargmann-Fock space, where we use complex samples to try to construct a latent representation of the original data, in order to obtain dimension reduction and feature compression to some extent. Assumption B states recovery conditions used in our theoretical derivations.
It is easier to get error bounds of ∥ f − f ~ ∥ L 2 ( R n ; R ) \lVert f-\tilde{f}\rVert_{L^2(\mathbb{R}^n;\mathbb{R})} ∥ f − f ~ ∥ L 2 ( R n ; R ) , but our goal is ∥ f − f ~ ∥ L ∞ ( R n ; R ) \lVert f-\tilde{f}\rVert_{L^\infty(\mathbb{R}^n;\mathbb{R})} ∥ f − f ~ ∥ L ∞ ( R n ; R ) . The Segal-Bargmann transform allows us to take advantage of the RKHS structure and the rigidity of complex analyticity from the Bargmann-Fock space.
Later we would also like to discuss how to recover c c c by dropping the Assumption A .
Convergence with Complex L L L -Sampling
The results in this subsection are preliminaries for our neural operator learning process in later sections. In this subsection, we will show that for f f f that satisfies Assumption A and the grids { x i } \{x_i\} { x i } that satisfy Assumption B , we can apply the α \alpha α -Segal-Bargmann transform, and introduce complex L L L -sampling in the α \alpha α -Bargmann-Fock space to obtain an error bound of
sup x ∈ R n ∣ f − f ~ ∣ \sup_{x\in\mathbb{R}^{n}}|f-\tilde{f}| x ∈ R n sup ∣ f − f ~ ∣
for f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) f \in L^2(\mathbb{R}^n;\mathbb{R}) \cap C^\infty(\mathbb{R}^n;\mathbb{R}) f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) . If we assume that f ∈ H s ( R n ; R ) f \in H^s(\mathbb{R}^n;\mathbb{R}) f ∈ H s ( R n ; R ) for s > n 2 s > \frac{n}{2} s > 2 n , we can get the same error bound without Assumption A .
More importantly, we show that with an appropriate choice of a a a that is specific for the spatial dimension n n n , the error bound we will give is independent of the spatial dimension n n n .
The crucial step is to not to consider the error bound in the original space, but in the a a a -Bargmann-Fock space. We see the advantage of the a a a -Segal-Bargmann transform immediately. By Definition 3.2 and Definition 3.3, we have
B a ( f ) = F = ∑ β c β e β a , c β = ⟨ f , h β a ⟩ L 2 ( R n ; R ) = ⟨ F , e β a ⟩ F a 2 ( C n ) \mathcal{B}_a(f)=F=\sum_{\beta}c_\beta e_\beta^a,\quad c_\beta=\langle f,h_\beta^a\rangle_{L^2(\mathbb{R}^n;\mathbb{R})}=\langle F,e_\beta^a\rangle_{\mathcal{F}_a^2(\mathbb{C}^n)} B a ( f ) = F = β ∑ c β e β a , c β = ⟨ f , h β a ⟩ L 2 ( R n ; R ) = ⟨ F , e β a ⟩ F a 2 ( C n )
where the equality holds pointwise! In addition,
F ^ : = ∑ β ∈ Λ M c ^ β e β a = B a ( f ^ ) , ∥ f − f ^ ∥ L 2 ( R n ; R ) = ∥ F − F ^ ∥ F a 2 ( C n ) . \widehat{F}:=\sum_{\beta\in\Lambda_M}\widehat{c}_\beta e_\beta^a=\mathcal{B}_a(\widehat{f}),\quad\|f-\widehat{f}\|_{L^2(\mathbb{R}^n;\mathbb{R})}=\|F-\widehat{F}\|_{\mathcal{F}_a^2(\mathbb{C}^n)}. F := β ∈ Λ M ∑ c β e β a = B a ( f ) , ∥ f − f ∥ L 2 ( R n ; R ) = ∥ F − F ∥ F a 2 ( C n ) .
It follows that we can equally discuss ∥ F − F ^ ∥ F a 2 ( C n ) \|F-\widehat{F}\|_{\mathcal{F}_a^2(\mathbb{C}^n)} ∥ F − F ∥ F a 2 ( C n ) . More importantly, merely with Assumption A and Assumption B , we can give pointwise error bound
sup z ∈ D ∣ F − F ^ ∣ \sup_{z\in D}|F-\widehat{F}| z ∈ D sup ∣ F − F ∣
which is again independent of the dimension, since the Fock space F a 2 \mathcal{F}_a^2 F a 2 is an RKHS. The key point is that we will estimate c ^ β \widehat{c}_\beta c β by using complex grids { z l } ⊂ C n \{z_l\}\subset\mathbb{C}^n { z l } ⊂ C n , which are decoupled from the original grids { x i } \{x_i\} { x i } . The decoupling between real and complex grids allows us to choose complex grids at will.
We need the following technical lemma for the estimate of the projection of f f f onto Span ( V M ) ⊥ \operatorname{Span}(V_M)^\perp Span ( V M ) ⊥ .
Lemma 5.1. Let f ∈ H a s ( R n ; R ) ∩ C ∞ ( R n ; R ) f \in H_a^s(\mathbb{R}^n;\mathbb{R}) \cap C^\infty(\mathbb{R}^n;\mathbb{R}) f ∈ H a s ( R n ; R ) ∩ C ∞ ( R n ; R ) for some s ≥ s 0 > n 2 s \ge s_0 > \frac{n}{2} s ≥ s 0 > 2 n . Then for each M 0 ∈ N M_0 \in\mathbb{N} M 0 ∈ N ,
sup x ∈ R n ∣ f − P M 0 f ∣ = sup x ∈ R n ∣ ∑ ∣ β ∣ > M 0 c β h β a ∣ ≤ ( 4 π ) − n 4 Γ ( s − n 2 ) Γ ( s ) ∥ f ∥ H a s (14) \sup_{x\in\mathbb{R}^n}|f-P_{M_0}f|=\sup_{x\in\mathbb{R}^n}\left|\sum_{|\beta|>M_0}c_\beta h_\beta^a\right|\le(4\pi)^{-\frac{n}{4}}\sqrt{\frac{\Gamma\left(s-\frac{n}{2}\right)}{\Gamma(s)}}\|f\|_{H_a^s} \tag*{(14)} x ∈ R n sup ∣ f − P M 0 f ∣ = x ∈ R n sup ∣ β ∣ > M 0 ∑ c β h β a ≤ ( 4 π ) − 4 n Γ ( s ) Γ ( s − 2 n ) ∥ f ∥ H a s (14) and
sup x ∈ R n ∣ f − P M 0 f ∣ ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − s − s 0 2 ( 4 π ) − n 4 Γ ( s 0 − n 2 ) Γ ( s 0 ) ∥ f ∥ H a s . (15) \sup_{x\in\mathbb{R}^n}|f-P_{M_0}f|\le\left(1+2a(2M_0+n)\right)^{-\frac{s-s_0}{2}}(4\pi)^{-\frac{n}{4}}\sqrt{\frac{\Gamma\left(s_0-\frac{n}{2}\right)}{\Gamma(s_0)}}\|f\|_{H_a^s}. \tag*{(15)} x ∈ R n sup ∣ f − P M 0 f ∣ ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − 2 s − s 0 ( 4 π ) − 4 n Γ ( s 0 ) Γ ( s 0 − 2 n ) ∥ f ∥ H a s . (15) In addition,
∥ f − P M 0 f ∥ L 2 ( R n ; R ) ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − s 2 ∥ f ∥ H a s ( R n ; R ) . (16) \|f-P_{M_0}f\|_{L^2(\mathbb{R}^n;\mathbb{R})}\le\left(1+2a(2M_0+n)\right)^{-\frac{s}{2}}\|f\|_{H_a^s(\mathbb{R}^n;\mathbb{R})}. \tag*{(16)} ∥ f − P M 0 f ∥ L 2 ( R n ; R ) ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − 2 s ∥ f ∥ H a s ( R n ; R ) . (16)
Proof. By Definition 3.5 and the eigenvalue equation of H β a \mathcal{H}_\beta^a H β a , we compute directly that
∣ f ( x ) − P M 0 f ( x ) ∣ = ∣ ∑ ∣ β ∣ > M 0 c β h β a ( x ) ∣ ≤ ∑ ∣ β ∣ > M 0 ( ( 1 + λ β , n ) s 2 ∣ c β ∣ ) ( ( 1 + λ β , n ) − s 2 ∣ h β a ( x ) ∣ ) ≤ ( ∑ ∣ β ∣ > M 0 ( 1 + λ β , n ) s ∣ c β ∣ 2 ) 1 2 ( ∑ ∣ β ∣ > M 0 ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 ) 1 2 ≤ ∥ f ∥ H a s ( R n ; R ) ( ∑ ∣ β ∣ > M 0 ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 ) 1 2 . \begin{aligned}
|f(x)-P_{M_0}f(x)|=\left|\sum_{|\beta|>M_0}c_\beta h_\beta^a(x)\right|\le\sum_{|\beta|>M_0}\left((1+\lambda_{\beta,n})^{\frac{s}{2}}|c_\beta|\right)\left((1+\lambda_{\beta,n})^{-\frac{s}{2}}|h_\beta^a(x)|\right) \\
&\le\left(\sum_{|\beta|>M_0}(1+\lambda_{\beta,n})^s|c_\beta|^2\right)^{\frac{1}{2}}\left(\sum_{|\beta|>M_0}(1+\lambda_{\beta,n})^{-s}|h_\beta^a(x)|^2\right)^{\frac{1}{2}} \\
&\le\|f\|_{H_a^s(\mathbb{R}^n;\mathbb{R})}\left(\sum_{|\beta|>M_0}(1+\lambda_{\beta,n})^{-s}|h_\beta^a(x)|^2\right)^{\frac{1}{2}}.
\end{aligned} ∣ f ( x ) − P M 0 f ( x ) ∣ = ∣ β ∣ > M 0 ∑ c β h β a ( x ) ≤ ∣ β ∣ > M 0 ∑ ( ( 1 + λ β , n ) 2 s ∣ c β ∣ ) ( ( 1 + λ β , n ) − 2 s ∣ h β a ( x ) ∣ ) ≤ ∣ β ∣ > M 0 ∑ ( 1 + λ β , n ) s ∣ c β ∣ 2 2 1 ∣ β ∣ > M 0 ∑ ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 2 1 ≤ ∥ f ∥ H a s ( R n ; R ) ∣ β ∣ > M 0 ∑ ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 2 1 .
Here we applied Cauchy-Schwarz in the second line above.
When s > n 2 s > \frac{n}{2} s > 2 n , we apply the Gamma function identity
∑ β ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 = ∑ β 1 Γ ( s ) ∫ 0 ∞ t s − 1 e − ( 1 + λ β , n ) t ∣ h β a ( x ) ∣ 2 d t = 1 Γ ( s ) ∫ 0 ∞ t s − 1 e − t ( ∑ β e − λ β , n t ∣ h β a ( x ) ∣ 2 ) d t . \begin{aligned}
\sum_{\beta}(1+\lambda_{\beta,n})^{-s}|h_{\beta}^{a}(x)|^{2}
= \sum_{\beta}\frac{1}{\Gamma(s)}\int_{0}^{\infty}t^{s-1}e^{-(1+\lambda_{\beta,n})t}|h_{\beta}^{a}(x)|^{2}\,\mathrm{d}t \\
&= \frac{1}{\Gamma(s)}\int_{0}^{\infty}t^{s-1}e^{-t}\left(\sum_{\beta}e^{-\lambda_{\beta,n}t}|h_{\beta}^{a}(x)|^{2}\right)\,\mathrm{d}t.
\end{aligned} β ∑ ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 = β ∑ Γ ( s ) 1 ∫ 0 ∞ t s − 1 e − ( 1 + λ β , n ) t ∣ h β a ( x ) ∣ 2 d t = Γ ( s ) 1 ∫ 0 ∞ t s − 1 e − t β ∑ e − λ β , n t ∣ h β a ( x ) ∣ 2 d t .
Here the sum and the integration are interchangeable by Tonelli’s theorem. By the one dimensional Mehler’s formula [44 (18.18.28)] and the multi-dimensional derivation [30 (4.9)] , we have
∑ β e − λ β , n t ∣ h β a ( x ) ∣ 2 = ( a π sinh ( 4 a t ) ) n 2 e − 2 a tanh ( 2 a t ) ∣ x ∣ 2 \sum_{\beta}e^{-\lambda_{\beta,n}t}|h_{\beta}^{a}(x)|^{2}
= \left(\frac{a}{\pi\sinh(4at)}\right)^{\frac{n}{2}}e^{-2a\tanh(2at)|x|^{2}} β ∑ e − λ β , n t ∣ h β a ( x ) ∣ 2 = ( π sinh ( 4 a t ) a ) 2 n e − 2 a t a n h ( 2 a t ) ∣ x ∣ 2
which is the Mehler kernel as the diagonal heat kernel with respect to e − t H β a e^{-tH_{\beta}^{a} } e − t H β a . It follows that
sup x ∈ R n ( ∑ ∣ β ∣ > M 0 ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 ) ≤ 1 Γ ( s ) ∫ 0 ∞ t s − 1 e − t ( a π sinh ( 4 a t ) ) n 2 ( sup x ∈ R n e − 2 a tanh ( 2 a t ) ∣ x ∣ 2 ) d t ≤ 1 Γ ( s ) ∫ 0 ∞ t s − 1 e − t ( a π sinh ( 4 a t ) ) n 2 d t ≤ 1 Γ ( s ) ∫ 0 ∞ t s − 1 e − t ( a 4 a t π ) n 2 d t = 1 ( 4 π ) n 2 Γ ( s ) ∫ 0 ∞ t s − n 2 − 1 e − t d t = ( 4 π ) − n 2 Γ ( s − n 2 ) Γ ( s ) . \begin{aligned}
\sup_{x\in\mathbb{R}^{n}}\left(\sum_{|\beta|>M_{0}}(1+\lambda_{\beta,n})^{-s}|h_{\beta}^{a}(x)|^{2}\right)
\leq\frac{1}{\Gamma(s)}\int_{0}^{\infty}t^{s-1}e^{-t}\left(\frac{a}{\pi\sinh(4at)}\right)^{\frac{n}{2}}\left(\sup_{x\in\mathbb{R}^{n}}e^{-2a\tanh(2at)|x|^{2}}\right)\,\mathrm{d}t \\
&\leq\frac{1}{\Gamma(s)}\int_{0}^{\infty}t^{s-1}e^{-t}\left(\frac{a}{\pi\sinh(4at)}\right)^{\frac{n}{2}}\,\mathrm{d}t \\
&\leq\frac{1}{\Gamma(s)}\int_{0}^{\infty}t^{s-1}e^{-t}\left(\frac{a}{4at\pi}\right)^{\frac{n}{2}}\,\mathrm{d}t
= \frac{1}{(4\pi)^{\frac{n}{2}}\Gamma(s)}\int_{0}^{\infty}t^{s-\frac{n}{2}-1}e^{-t}\,\mathrm{d}t \\
&= (4\pi)^{-\frac{n}{2}}\frac{\Gamma\left(s-\frac{n}{2}\right)}{\Gamma(s)}.
\end{aligned} x ∈ R n sup ∣ β ∣ > M 0 ∑ ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 ≤ Γ ( s ) 1 ∫ 0 ∞ t s − 1 e − t ( π sinh ( 4 a t ) a ) 2 n ( x ∈ R n sup e − 2 a t a n h ( 2 a t ) ∣ x ∣ 2 ) d t ≤ Γ ( s ) 1 ∫ 0 ∞ t s − 1 e − t ( π sinh ( 4 a t ) a ) 2 n d t ≤ Γ ( s ) 1 ∫ 0 ∞ t s − 1 e − t ( 4 a t π a ) 2 n d t = ( 4 π ) 2 n Γ ( s ) 1 ∫ 0 ∞ t s − 2 n − 1 e − t d t = ( 4 π ) − 2 n Γ ( s ) Γ ( s − 2 n ) .
The ratio Γ ( s − n 2 ) Γ ( s ) ≤ 1 \frac{\Gamma\left(s-\frac{n}{2}\right)}{\Gamma(s)}\leq1 Γ ( s ) Γ ( s − 2 n ) ≤ 1 if s − n 2 ≥ 2 s-\frac{n}{2}\geq2 s − 2 n ≥ 2 . Therefore we conclude that
sup x ∈ R n ∣ f ( x ) − P M 0 f ( x ) ∣ ≤ ( 4 π ) − n 4 Γ ( s − n 2 ) Γ ( s ) ∥ f ∥ H a s ( R n ; R ) , \sup_{x\in\mathbb{R}^{n}}|f(x)-P_{M_{0}}f(x)|
\leq(4\pi)^{-\frac{n}{4}}\sqrt{\frac{\Gamma\left(s-\frac{n}{2}\right)}{\Gamma(s)}}\|f\|_{\mathcal{H}_{a}^{s}(\mathbb{R}^{n};\mathbb{R})}, x ∈ R n sup ∣ f ( x ) − P M 0 f ( x ) ∣ ≤ ( 4 π ) − 4 n Γ ( s ) Γ ( s − 2 n ) ∥ f ∥ H a s ( R n ; R ) ,
which gives (14). By the same argument as above,
∣ f ( x ) − P M 0 f ( x ) ∣ = ∣ ∑ ∣ β ∣ > M 0 c β h β a ( x ) ∣ ≤ ∑ ∣ β ∣ > M 0 ( ( 1 + λ β , n ) s 2 ∣ c β ∣ ) ( ( 1 + λ β , n ) − s 2 ∣ h β a ( x ) ∣ ) ≤ ( ∑ ∣ β ∣ > M 0 ( 1 + λ β , n ) s ∣ c β ∣ 2 ) 1 2 ( ∑ ∣ β ∣ > M 0 ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 ) 1 2 ≤ ∥ f ∥ H a s ( R n ; R ) ( ∑ ∣ β ∣ > M 0 ( 1 + λ β , n ) − ( s − s 0 ) ( 1 + λ β , n ) − s 0 ∣ h β a ( x ) ∣ 2 ) 1 2 . \begin{aligned}
|f(x)-P_{M_{0}}f(x)|
= \left|\sum_{|\beta|>M_{0}}c_{\beta}h_{\beta}^{a}(x)\right|
\leq\sum_{|\beta|>M_{0}}\left((1+\lambda_{\beta,n})^{\frac{s}{2}}|c_{\beta}|\right)\left((1+\lambda_{\beta,n})^{-\frac{s}{2}}|h_{\beta}^{a}(x)|\right) \\
&\leq\left(\sum_{|\beta|>M_{0}}(1+\lambda_{\beta,n})^{s}|c_{\beta}|^{2}\right)^{\frac{1}{2}}
\left(\sum_{|\beta|>M_{0}}(1+\lambda_{\beta,n})^{-s}|h_{\beta}^{a}(x)|^{2}\right)^{\frac{1}{2}} \\
&\leq\|f\|_{\mathcal{H}_{a}^{s}(\mathbb{R}^{n};\mathbb{R})}
\left(\sum_{|\beta|>M_{0}}(1+\lambda_{\beta,n})^{-(s-s_{0})}(1+\lambda_{\beta,n})^{-s_{0}}|h_{\beta}^{a}(x)|^{2}\right)^{\frac{1}{2}}.
\end{aligned} ∣ f ( x ) − P M 0 f ( x ) ∣ = ∣ β ∣ > M 0 ∑ c β h β a ( x ) ≤ ∣ β ∣ > M 0 ∑ ( ( 1 + λ β , n ) 2 s ∣ c β ∣ ) ( ( 1 + λ β , n ) − 2 s ∣ h β a ( x ) ∣ ) ≤ ∣ β ∣ > M 0 ∑ ( 1 + λ β , n ) s ∣ c β ∣ 2 2 1 ∣ β ∣ > M 0 ∑ ( 1 + λ β , n ) − s ∣ h β a ( x ) ∣ 2 2 1 ≤ ∥ f ∥ H a s ( R n ; R ) ∣ β ∣ > M 0 ∑ ( 1 + λ β , n ) − ( s − s 0 ) ( 1 + λ β , n ) − s 0 ∣ h β a ( x ) ∣ 2 2 1 .
Note that λ β , n = ( 4 a ∣ β ∣ + 2 a n ) \lambda_{\beta,n}=(4a|\beta|+2an) λ β , n = ( 4 a ∣ β ∣ + 2 an ) , it follows that when ∣ β ∣ > M 0 |\beta|>M_{0} ∣ β ∣ > M 0 ,
sup x ∈ R n ∣ f ( x ) − P M 0 f ( x ) ∣ ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − s − s 0 2 ( 4 π ) − n 4 Γ ( s 0 − n 2 ) Γ ( s 0 ) ∥ f ∥ H a s ( R n ; R ) , \sup_{x\in\mathbb{R}^{n}}|f(x)-P_{M_{0}}f(x)|
\leq(1+2a(2M_{0}+n))^{-\frac{s-s_{0}}{2}}(4\pi)^{-\frac{n}{4}}\sqrt{\frac{\Gamma\left(s_{0}-\frac{n}{2}\right)}{\Gamma(s_{0})}}\|f\|_{\mathcal{H}_{a}^{s}(\mathbb{R}^{n};\mathbb{R})}, x ∈ R n sup ∣ f ( x ) − P M 0 f ( x ) ∣ ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − 2 s − s 0 ( 4 π ) − 4 n Γ ( s 0 ) Γ ( s 0 − 2 n ) ∥ f ∥ H a s ( R n ; R ) ,
which gives (15).
The last inequality (16) is more straightforward,
∥ f − P M 0 f ∥ L 2 ( R n ; R ) 2 = ∑ ∣ β ∣ > M 0 ∣ c β ∣ 2 = ∑ ∣ β ∣ > M 0 ( 1 + 4 a ∣ β ∣ + 2 a n ) − s ( 1 + 4 a ∣ β ∣ + 2 a n ) s ∣ c β ∣ 2 ≤ 1 ( 1 + 2 a ( 2 M 0 + n ) ) s ∑ β ( 1 + 4 a ∣ β ∣ + 2 a n ) s ∣ c β ∣ 2 = 1 ( 1 + 2 a ( 2 M 0 + n ) ) s ∥ f ∥ H a s ( R n ; R ) 2 . \left\|f-P_{M_0}f\right\|_{L^2(\mathbb{R}^n;\mathbb{R})}^2=\sum_{|\beta|>M_0}|c_\beta|^2
=\sum_{|\beta|>M_0}(1+4a|\beta|+2an)^{-s}(1+4a|\beta|+2an)^s|c_\beta|^2
\le\frac{1}{(1+2a(2M_0+n))^s}\sum_\beta(1+4a|\beta|+2an)^s|c_\beta|^2
=\frac{1}{(1+2a(2M_0+n))^s}\|f\|_{\mathcal{H}^s_a(\mathbb{R}^n;\mathbb{R})}^2. ∥ f − P M 0 f ∥ L 2 ( R n ; R ) 2 = ∣ β ∣ > M 0 ∑ ∣ c β ∣ 2 = ∣ β ∣ > M 0 ∑ ( 1 + 4 a ∣ β ∣ + 2 an ) − s ( 1 + 4 a ∣ β ∣ + 2 an ) s ∣ c β ∣ 2 ≤ ( 1 + 2 a ( 2 M 0 + n ) ) s 1 β ∑ ( 1 + 4 a ∣ β ∣ + 2 an ) s ∣ c β ∣ 2 = ( 1 + 2 a ( 2 M 0 + n ) ) s 1 ∥ f ∥ H a s ( R n ; R ) 2 .
Denote by
F ^ ( z ) : = ∑ β ∈ Λ M c ^ β e β a ( z ) \widehat{F}(z):=\sum_{\beta\in\Lambda_M}\widehat{c}_\beta e^a_\beta(z) F ( z ) := β ∈ Λ M ∑ c β e β a ( z )
the data-driven approximation of F F F . Let’s focus on sup x ∈ D ∣ F ( z ) − F ^ ( z ) ∣ \sup_{x\in D}|F(z)-\widehat{F}(z)| sup x ∈ D ∣ F ( z ) − F ( z ) ∣ first for some bounded, open subset D ⊂ C n D\subset\mathbb{C}^n D ⊂ C n . By Liouville’s theorem, the error F − F ^ F-\widehat{F} F − F cannot be bounded on the whole space C n \mathbb{C}^n C n unless it is constant. The other error bounds, including our main goal sup x ∈ R n ∣ f ( x ) − f ^ ( x ) ∣ \sup_{x\in\mathbb{R}^n}|f(x)-\widehat{f}(x)| sup x ∈ R n ∣ f ( x ) − f ( x ) ∣ are natural consequences of the following argument.
With Assumption A and Assumption B , we are ready to give a dimension-free error bound for ∣ F ( z ) − F ^ ( z ) ∣ |F(z)-\widehat{F}(z)| ∣ F ( z ) − F ( z ) ∣ for all z ∈ D z\in D z ∈ D .
Theorem 5.1. Let f ∈ L 2 ( R n ; R ) f\in L^2(\mathbb{R}^n;\mathbb{R}) f ∈ L 2 ( R n ; R ) , F = B a ( f ) F=\mathcal{B}_a(f) F = B a ( f ) , ∥ f ∥ L 2 ( R n ; R ) ≤ C < ∞ \|f\|_{L^2(\mathbb{R}^n;\mathbb{R})}\le C<\infty ∥ f ∥ L 2 ( R n ; R ) ≤ C < ∞ and ∥ f ∥ L ∞ ( R n ; R ) = C ′ < ∞ \|f\|_{L^\infty(\mathbb{R}^n;\mathbb{R})}=C'<\infty ∥ f ∥ L ∞ ( R n ; R ) = C ′ < ∞ . Let D ⊂ C n D\subset\mathbb{C}^n D ⊂ C n be a domain such that D ‾ \overline{D} D is compact. If both Assumption A and Assumption B holds, then for each fixed dimension n n n and fixed truncation M = ∣ Λ M ∣ M=|\Lambda_M| M = ∣ Λ M ∣ , the data-driven approximation F ^ ( z ) = ∑ β ∈ Λ M c ^ β e β a ( z ) \widehat{F}(z)=\sum_{\beta\in\Lambda_M}\widehat{c}_\beta e^a_\beta(z) F ( z ) = ∑ β ∈ Λ M c β e β a ( z ) satisfies
E Z , X [ sup z ∈ D ∣ F ( z ) − F ^ ( z ) ∣ ] = O ( L − 1 2 ) , (17) \mathbb{E}_{Z,X}\left[\sup_{z\in D}|F(z)-\widehat{F}(z)|\right]=O\left(L^{-\frac{1}{2}}\right), \tag*{(17)} E Z , X [ z ∈ D sup ∣ F ( z ) − F ( z ) ∣ ] = O ( L − 2 1 ) , (17) where c ^ = ( c ^ 1 , … , c ^ β , … , c ^ M ) \widehat{c}=(\widehat{c}_1,\ldots,\widehat{c}_\beta,\ldots,\widehat{c}_M) c = ( c 1 , … , c β , … , c M ) is determined by { f ( x i ) } \{f(x_i)\} { f ( x i )} and complex grids { z l } l = 1 L \{z_l\}_{l=1}^{L} { z l } l = 1 L sampled random variables Z 1 , … , Z L ∼ i . i . d . ρ a ( z ) = ∑ β ∈ Λ M ∣ e β a ( z ) ∣ 2 M μ a ( z ) Z_1,\ldots,Z_L\mathrel{\mathop{\sim}^{i.i.d.}}\rho_a(z)=\frac{\sum_{\beta\in\Lambda_M}|e^a_\beta(z)|^2}{M}\mu_a(z) Z 1 , … , Z L ∼ i . i . d . ρ a ( z ) = M ∑ β ∈ Λ M ∣ e β a ( z ) ∣ 2 μ a ( z ) . Precisely speaking,
E X , Z [ sup z ∈ D ∣ F ( z ) − F ^ ( z ) ∣ ] ≤ sup z ∈ D ( e a ∣ z ∣ 2 2 ) ⋅ ( M − 1 ) C 2 L + M ( C ′ ) 2 L . (18) \mathbb{E}_{X,Z}\left[\sup_{z\in D}|F(z)-\widehat{F}(z)|\right]\le\sup_{z\in D}\left(e^{\frac{a|z|^2}{2}}\right)\cdot\sqrt{\frac{(M-1)C^2}{L}+\frac{M(C')^2}{L}}. \tag*{(18)} E X , Z [ z ∈ D sup ∣ F ( z ) − F ( z ) ∣ ] ≤ z ∈ D sup ( e 2 a ∣ z ∣ 2 ) ⋅ L ( M − 1 ) C 2 + L M ( C ′ ) 2 . (18)
Proof. By Assumption A , f = P M f f=P_Mf f = P M f . By the linearity of the a a a -Segal-Bargmann transform,
F ( z ) = ∑ β ∈ Λ M c β e β a = P M F , c β = ⟨ F , e β a ⟩ F a 2 ( C n ) = ⟨ f , h β a ⟩ L 2 ( R n ; R ) . F(z)=\sum_{\beta\in\Lambda_M}c_\beta e^a_\beta=P_MF,\ c_\beta=\langle F,e^a_\beta\rangle_{\mathcal{F}^2_a(\mathbb{C}^n)}=\langle f,h^a_\beta\rangle_{L^2(\mathbb{R}^n;\mathbb{R})}. F ( z ) = β ∈ Λ M ∑ c β e β a = P M F , c β = ⟨ F , e β a ⟩ F a 2 ( C n ) = ⟨ f , h β a ⟩ L 2 ( R n ; R ) .
with pointwise identification. For each fixed z z z , A a ( z , x ) A_a(z,x) A a ( z , x ) in Definition 3.1 is an L 2 L^2 L 2 -function. By Definition 3.1 and Definition 3.4), it follows that for fixed z z z ,
A a ( z , x ) = ∑ β ⟨ A a ( z , ⋅ ) , h β a ( ⋅ ) ⟩ L 2 ( R n ; R ) h β a = ∑ β B a ( h β a ) h β a = ∑ β e β a h β a . A_a(z,x)=\sum_\beta\langle A_a(z,\cdot),h^a_\beta(\cdot)\rangle_{L^2(\mathbb{R}^n;\mathbb{R})}h^a_\beta=\sum_\beta\mathcal{B}_a(h^a_\beta)h^a_\beta=\sum_\beta e^a_\beta h^a_\beta. A a ( z , x ) = β ∑ ⟨ A a ( z , ⋅ ) , h β a ( ⋅ ) ⟩ L 2 ( R n ; R ) h β a = β ∑ B a ( h β a ) h β a = β ∑ e β a h β a .
Denote by
A a , M ( z , x ) : = P M A a ( z , x ) = ∑ β ∈ Λ M e β a ( z ) h β a ( x ) . A_{a,M}(z,x):=P_MA_a(z,x)=\sum_{\beta\in\Lambda_M}e^a_\beta(z)h^a_\beta(x). A a , M ( z , x ) := P M A a ( z , x ) = β ∈ Λ M ∑ e β a ( z ) h β a ( x ) .
With F = P M F F=P_MF F = P M F , we get from uniqueness of orthonormal expansion that
∑ β ∈ Λ M c β e β a ( z ) = F ( z ) = ∫ R n A a ( z , x ) f ( x ) d x = ∫ R n ( ∑ β h β a ( x ) e β a ( z ) ) f ( x ) d x = ∫ R n ( ∑ β ∈ Λ M h β a ( x ) e β a ( z ) ) f ( x ) d x = ∫ R n A a , M ( z , x ) f ( x ) d x . \sum_{\beta\in\Lambda_M}c_\beta e_\beta^a(z)=F(z)=\int_{\mathbb{R}^n}A_a(z,x)f(x)\,dx=\int_{\mathbb{R}^n}\left(\sum_\beta h_\beta^a(x)e_\beta^a(z)\right)f(x)\,dx
=\int_{\mathbb{R}^n}\left(\sum_{\beta\in\Lambda_M}h_\beta^a(x)e_\beta^a(z)\right)f(x)\,dx=\int_{\mathbb{R}^n}A_{a,M}(z,x)f(x)\,dx. β ∈ Λ M ∑ c β e β a ( z ) = F ( z ) = ∫ R n A a ( z , x ) f ( x ) d x = ∫ R n β ∑ h β a ( x ) e β a ( z ) f ( x ) d x = ∫ R n β ∈ Λ M ∑ h β a ( x ) e β a ( z ) f ( x ) d x = ∫ R n A a , M ( z , x ) f ( x ) d x .
To construct the estimate F ^ \hat{F} F ^ with coefficients c ^ ∈ C M \hat{c}\in\mathbb{C}^M c ^ ∈ C M , we select complex sampling { Z l } l = 1 L \{Z_l\}_{l=1}^L { Z l } l = 1 L and associated real sampling { X i , l } i = 1 N \{X_{i,l}\}_{i=1}^N { X i , l } i = 1 N for each l l l .
Set
K M ( z ) : = ∑ β ∈ Λ M ∣ e β a ( z ) ∣ 2 , d ρ a ( z ) = K M ( z ) M d μ a ( z ) . K_M(z):=\sum_{\beta\in\Lambda_M}|e_\beta^a(z)|^2,\ d\rho_a(z)=\frac{K_M(z)}{M}d\mu_a(z). K M ( z ) := β ∈ Λ M ∑ ∣ e β a ( z ) ∣ 2 , d ρ a ( z ) = M K M ( z ) d μ a ( z ) .
we pick
Z 1 , … , Z L ∼ i . i . d . ρ a . Z_1,\ldots,Z_L\mathrel{\mathop{\sim}^{i.i.d.}}\rho_a. Z 1 , … , Z L ∼ i . i . d . ρ a .
Such a sampling approach is called Christoffel-function sampling following Cohen-Migliorati’s [13 ] optimal weighted least-squares sampling, as an unbiased sampling with a uniform control of the variance. Our complex grids { z l } l = 1 L \{z_l\}_{l=1}^L { z l } l = 1 L samples from { Z l } l = 1 L \{Z_l\}_{l=1}^L { Z l } l = 1 L .
Let { x i } i = 1 N \{x_i\}_{i=1}^N { x i } i = 1 N be the real grids that are sampling from random variables { X i } i = 1 N \{X_i\}_{i=1}^N { X i } i = 1 N . By Assumption B , (12) and (13), we have a random matrix W W W such that
F = B a ( f ) ≈ F ~ N ( Z l ) : = 1 N ∑ i = 1 N w l i A a , M ( Z l , X i ) f ( X i ) ⇒ F ( Z l ) : = F ~ N ( Z l ) + η l F=B_a(f)\approx\widetilde{F}_N(Z_l):=\frac{1}{N}\sum_{i=1}^Nw_{li}A_{a,M}(Z_l,X_i)f(X_i)\Rightarrow F(Z_l):=\widetilde{F}_N(Z_l)+\eta_l F = B a ( f ) ≈ F N ( Z l ) := N 1 i = 1 ∑ N w l i A a , M ( Z l , X i ) f ( X i ) ⇒ F ( Z l ) := F N ( Z l ) + η l
where
E X ∣ Z [ η l ] = E X ∣ Z [ F ( Z l ) − F ~ N ( Z l ) ] = 0 , E X ∣ Z [ η l η m ‾ ] = σ l m 2 δ l m . \mathbb{E}_{X\mid Z}[\eta_l]=\mathbb{E}_{X\mid Z}[F(Z_l)-\widetilde{F}_N(Z_l)]=0,\mathbb{E}_{X\mid Z}[\eta_l\overline{\eta_m}]=\sigma_{lm}^2\delta_{lm}. E X ∣ Z [ η l ] = E X ∣ Z [ F ( Z l ) − F N ( Z l )] = 0 , E X ∣ Z [ η l η m ] = σ l m 2 δ l m .
Set the weight, the basis vector and the unknown ground truth
q M ( z ) = M K M ( z ) , e a ( z ) = ( e 1 a ( z ) ⋮ e M a ( z ) ) , c = ( c 1 ⋮ c M ) . q_M(z)=\frac{M}{K_M(z)},e^a(z)=\begin{pmatrix}e_1^a(z)\\ \vdots\\ e_M^a(z)\end{pmatrix},c=\begin{pmatrix}c_1\\ \vdots\\ c_M\end{pmatrix}. q M ( z ) = K M ( z ) M , e a ( z ) = e 1 a ( z ) ⋮ e M a ( z ) , c = c 1 ⋮ c M .
We define
c ^ : = 1 L ∑ l = 1 L q M ( Z l ) e a ( Z l ) ‾ F ~ N ( Z l ) (19) \hat{c}:=\frac{1}{L}\sum_{l=1}^Lq_M(Z_l)\overline{e^a(Z_l)}\widetilde{F}_N(Z_l) \tag*{(19)} c ^ := L 1 l = 1 ∑ L q M ( Z l ) e a ( Z l ) F N ( Z l ) (19)
Observe that
F ( Z l ) = e a ( Z l ) t c , F(Z_l)=e^a(Z_l)^tc, F ( Z l ) = e a ( Z l ) t c ,
(19) implies that
c ^ − c = 1 L ∑ l = 1 L ( q M ( Z l ) e a ( Z l ) e a ( Z l ) ‾ t − I M ) c − 1 L ∑ l = 1 L q M ( Z l ) e a ( Z l ) ‾ η l (20) \hat{c}-c=\frac{1}{L}\sum_{l=1}^L\left(q_M(Z_l)e^a(Z_l)\overline{e^a(Z_l)}^t-I_M\right)c-\frac{1}{L}\sum_{l=1}^Lq_M(Z_l)\overline{e^a(Z_l)}\eta_l \tag*{(20)} c ^ − c = L 1 l = 1 ∑ L ( q M ( Z l ) e a ( Z l ) e a ( Z l ) t − I M ) c − L 1 l = 1 ∑ L q M ( Z l ) e a ( Z l ) η l (20)
We now analyze E Z , X [ ∥ c ^ − c ∥ 2 ] \mathbb{E}_{Z,X}\left[\|\hat{c}-c\|^2\right] E Z , X [ ∥ c ^ − c ∥ 2 ] by analyzing two terms in (20), respectively. For the first term of (20), note that
E Z [ q M ( Z l ) e a ( Z l ) e a ( Z l ) ‾ t ] = ∫ C n M K M ( z ) e a ( z ) e a ( z ) ‾ t d ρ a ( z ) = ∫ C n M K M ( z ) e a ( z ) e a ( z ) ‾ t K M ( z ) M d μ a ( z ) = I M \mathbb{E}_Z\left[q_M(Z_l)e^a(Z_l)\overline{e^a(Z_l)}^t\right]=\int_{\mathbb{C}^n}\frac{M}{K_M(z)}e^a(z)\overline{e^a(z)}^t\,d\rho_a(z)
=\int_{\mathbb{C}^n}\frac{M}{K_M(z)}e^a(z)\overline{e^a(z)}^t\frac{K_M(z)}{M}\,d\mu_a(z)=I_M E Z [ q M ( Z l ) e a ( Z l ) e a ( Z l ) t ] = ∫ C n K M ( z ) M e a ( z ) e a ( z ) t d ρ a ( z ) = ∫ C n K M ( z ) M e a ( z ) e a ( z ) t M K M ( z ) d μ a ( z ) = I M
by orthonormality of { e β a } \{e^a_\beta\} { e β a } . It follows that
E Z , X [ 1 L ∑ l = 1 L ( q M ( Z l ) e a ( Z l ‾ ) e a ( Z l ) t − I M ) c ] = 1 L ∑ l = 1 L E Z [ q M ( Z l ) e a ( Z l ‾ ) e a ( Z l ) t − I M ] c = 0. \mathbb{E}_{Z,X}\left[\frac{1}{L}\sum_{l=1}^{L}\left(q_M(Z_l)e^a(\overline{Z_l})e^a(Z_l)^t-I_M\right)c\right]=\frac{1}{L}\sum_{l=1}^{L}\mathbb{E}_Z\left[q_M(Z_l)e^a(\overline{Z_l})e^a(Z_l)^t-I_M\right]c=0. E Z , X [ L 1 l = 1 ∑ L ( q M ( Z l ) e a ( Z l ) e a ( Z l ) t − I M ) c ] = L 1 l = 1 ∑ L E Z [ q M ( Z l ) e a ( Z l ) e a ( Z l ) t − I M ] c = 0.
Since { Z l } ∼ i . i . d . ρ a \{Z_l\}\mathrel{\mathop{\sim}^{i.i.d.}}\rho_a { Z l } ∼ i . i . d . ρ a , it follows that
E Z , X [ ∥ 1 L ∑ l = 1 L ( q M ( Z l ) e a ( Z l ‾ ) e a ( Z l ) t − I M ) c ∥ 2 ] = 1 L ∥ E Z [ ( q M ( Z ) e a ( Z ‾ ) e a ( Z ) t − I M ) c ] 2 ∥ , Z ∼ ρ a ≤ 1 L ∥ E Z [ q M ( Z ) e a ( Z ‾ ) e a ( Z ) t q M ( Z ) e a ( Z ‾ ) e a ( Z ) t ] − 2 E Z [ q M ( Z ) e a ( Z ‾ ) e a ( Z ) t ] I M + I M 2 ∥ ∥ c ∥ 2 = 1 L ∥ E Z [ q M ( Z ) 2 e a ( Z ‾ ) K M ( Z ) e a ( Z ) t ] − I M ∥ ∥ c ∥ 2 = 1 L ∥ M E Z [ q M ( Z ) e a ( Z ‾ ) e a ( Z ) t ] − I M ∥ ∥ c ∥ 2 = 1 L ∥ ( M − 1 ) I M ∥ ∥ c ∥ 2 = M − 1 L ∥ f ∥ L 2 ( R n ; R ) 2 ≤ ( M − 1 ) C 2 L . \begin{aligned}
\mathbb{E}_{Z,X}\left[\left\|\frac{1}{L}\sum_{l=1}^{L}\left(q_M(Z_l)e^a(\overline{Z_l})e^a(Z_l)^t-I_M\right)c\right\|^2\right]
&=\frac{1}{L}\left\|\mathbb{E}_Z\left[\left(q_M(Z)e^a(\overline{Z})e^a(Z)^t-I_M\right)c\right]^2\right\|,\quad Z\sim\rho_a\\
&\leq\frac{1}{L}\left\|\mathbb{E}_Z\left[q_M(Z)e^a(\overline{Z})e^a(Z)^tq_M(Z)e^a(\overline{Z})e^a(Z)^t\right]\right.\\
&\qquad\left.-2\mathbb{E}_Z\left[q_M(Z)e^a(\overline{Z})e^a(Z)^t\right]I_M+I_M^2\right\|\|c\|^2\\
&=\frac{1}{L}\left\|\mathbb{E}_Z\left[q_M(Z)^2e^a(\overline{Z})K_M(Z)e^a(Z)^t\right]-I_M\right\|\|c\|^2\\
&=\frac{1}{L}\left\|M\mathbb{E}_Z\left[q_M(Z)e^a(\overline{Z})e^a(Z)^t\right]-I_M\right\|\|c\|^2\\
&=\frac{1}{L}\left\|(M-1)I_M\right\|\|c\|^2\\
&=\frac{M-1}{L}\|f\|_{L^2(\mathbb{R}^n;\mathbb{R})}^2\leq\frac{(M-1)C^2}{L}.
\end{aligned} E Z , X L 1 l = 1 ∑ L ( q M ( Z l ) e a ( Z l ) e a ( Z l ) t − I M ) c 2 = L 1 E Z [ ( q M ( Z ) e a ( Z ) e a ( Z ) t − I M ) c ] 2 , Z ∼ ρ a ≤ L 1 E Z [ q M ( Z ) e a ( Z ) e a ( Z ) t q M ( Z ) e a ( Z ) e a ( Z ) t ] − 2 E Z [ q M ( Z ) e a ( Z ) e a ( Z ) t ] I M + I M 2 ∥ c ∥ 2 = L 1 E Z [ q M ( Z ) 2 e a ( Z ) K M ( Z ) e a ( Z ) t ] − I M ∥ c ∥ 2 = L 1 M E Z [ q M ( Z ) e a ( Z ) e a ( Z ) t ] − I M ∥ c ∥ 2 = L 1 ∥ ( M − 1 ) I M ∥ ∥ c ∥ 2 = L M − 1 ∥ f ∥ L 2 ( R n ; R ) 2 ≤ L ( M − 1 ) C 2 .
By Assumption B , the second term of (20) satisfies
E Z , X [ 1 L ∑ l = 1 L q M ( Z l ) e a ( Z l ‾ ) η l ] = E Z [ 1 L ∑ l = 1 L [ q M ( Z l ) e a ( Z l ‾ ) E X ∣ Z [ η l ] ] ] = 0. \mathbb{E}_{Z,X}\left[\frac{1}{L}\sum_{l=1}^{L}q_M(Z_l)e^a(\overline{Z_l})\eta_l\right]=\mathbb{E}_Z\left[\frac{1}{L}\sum_{l=1}^{L}\left[q_M(Z_l)e^a(\overline{Z_l})\mathbb{E}_{X|Z}[\eta_l]\right]\right]=0. E Z , X [ L 1 l = 1 ∑ L q M ( Z l ) e a ( Z l ) η l ] = E Z [ L 1 l = 1 ∑ L [ q M ( Z l ) e a ( Z l ) E X ∣ Z [ η l ] ] ] = 0.
The second variation satisfies
E Z , X [ ∥ 1 L ∑ l = 1 L q M ( Z l ) e a ( Z l ‾ ) η l ∥ 2 ] = 1 L 2 E Z [ ∑ l , m = 1 L q M ( Z l ) q M ( Z m ) e a ( Z l ) t e a ( Z m ‾ ) E X ∣ Z [ η l η m ‾ ] ] = 1 L 2 E Z [ ∑ l = 1 L q M ( Z l ) 2 K M ( Z l ) σ l 2 ] ≤ 1 L 2 ⋅ M L ⋅ ( C ′ ) 2 = M ( C ′ ) 2 L . \begin{aligned}
\mathbb{E}_{Z,X}\left[\left\|\frac{1}{L}\sum_{l=1}^{L}q_M(Z_l)e^a(\overline{Z_l})\eta_l\right\|^2\right]
&=\frac{1}{L^2}\mathbb{E}_Z\left[\sum_{l,m=1}^{L}q_M(Z_l)q_M(Z_m)e^a(Z_l)^te^a(\overline{Z_m})\mathbb{E}_{X|Z}[\eta_l\overline{\eta_m}]\right]\\
&=\frac{1}{L^2}\mathbb{E}_Z\left[\sum_{l=1}^{L}q_M(Z_l)^2K_M(Z_l)\sigma_l^2\right]\\
&\leq\frac{1}{L^2}\cdot ML\cdot(C')^2=\frac{M(C')^2}{L}.
\end{aligned} E Z , X L 1 l = 1 ∑ L q M ( Z l ) e a ( Z l ) η l 2 = L 2 1 E Z l , m = 1 ∑ L q M ( Z l ) q M ( Z m ) e a ( Z l ) t e a ( Z m ) E X ∣ Z [ η l η m ] = L 2 1 E Z [ l = 1 ∑ L q M ( Z l ) 2 K M ( Z l ) σ l 2 ] ≤ L 2 1 ⋅ M L ⋅ ( C ′ ) 2 = L M ( C ′ ) 2 .
Therefore,
E Z , X [ ∥ c ^ − c ∥ 2 ] = E Z , X [ ∥ 1 L ∑ l = 1 L ( q M ( Z l ) e a ( Z l ‾ ) e a ( Z l ) t − I M ) c ∥ 2 ] + E Z , X [ ∥ 1 L ∑ l = 1 L q M ( Z l ) e a ( Z l ‾ ) η l ∥ 2 ] ≤ ( M − 1 ) C 2 L + M ( C ′ ) 2 L . \begin{aligned}
\mathbb{E}_{Z,X}\left[\|\hat{c}-c\|^2\right]
&=\mathbb{E}_{Z,X}\left[\left\|\frac{1}{L}\sum_{l=1}^{L}\left(q_M(Z_l)e^a(\overline{Z_l})e^a(Z_l)^t-I_M\right)c\right\|^2\right]\\
&\quad+\mathbb{E}_{Z,X}\left[\left\|\frac{1}{L}\sum_{l=1}^{L}q_M(Z_l)e^a(\overline{Z_l})\eta_l\right\|^2\right]\\
&\leq\frac{(M-1)C^2}{L}+\frac{M(C')^2}{L}.
\end{aligned} E Z , X [ ∥ c ^ − c ∥ 2 ] = E Z , X L 1 l = 1 ∑ L ( q M ( Z l ) e a ( Z l ) e a ( Z l ) t − I M ) c 2 + E Z , X L 1 l = 1 ∑ L q M ( Z l ) e a ( Z l ) η l 2 ≤ L ( M − 1 ) C 2 + L M ( C ′ ) 2 .
Applying the RKHS inequality, it follows immediately that
E X , Z [ sup z ∈ D ∣ F ( z ) − F ^ ( z ) ∣ ] ≤ E X , Z [ sup z ∈ D K a ( z , z ) ∥ F − F ^ ∥ F a 2 ( C n ) ] ≤ sup z ∈ D ( e a ∣ z ∣ 2 ) 1 2 ( E X , Z [ ∥ c − c ^ ∥ 2 ] ) 1 2 ≤ sup z ∈ D ( e a ∣ z ∣ 2 2 ) ⋅ ( M − 1 ) C 2 L + M ( C ′ ) 2 L . \begin{aligned}
\mathbb{E}_{X,Z}\left[\sup_{z\in D}\left|F(z)-\widehat{F}(z)\right|\right]\leq\mathbb{E}_{X,Z}\left[\sup_{z\in D}\sqrt{K_a(z,z)}\left\|F-\widehat{F}\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}\right] \\
&\leq\sup_{z\in D}\left(e^{a|z|^2}\right)^{\frac{1}{2}}\left(\mathbb{E}_{X,Z}\left[\left\|c-\widehat{c}\right\|^2\right]\right)^{\frac{1}{2}} \\
&\leq\sup_{z\in D}\left(e^{\frac{a|z|^2}{2}}\right)\cdot\sqrt{\frac{(M-1)C^2}{L}+\frac{M(C')^2}{L}}.
\end{aligned} E X , Z [ z ∈ D sup F ( z ) − F ( z ) ] ≤ E X , Z [ z ∈ D sup K a ( z , z ) F − F F a 2 ( C n ) ] ≤ z ∈ D sup ( e a ∣ z ∣ 2 ) 2 1 ( E X , Z [ ∥ c − c ∥ 2 ] ) 2 1 ≤ z ∈ D sup ( e 2 a ∣ z ∣ 2 ) ⋅ L ( M − 1 ) C 2 + L M ( C ′ ) 2 .
This gives (18). For fixed M M M (e.g. when Λ M \Lambda_M Λ M is a sparse set in the sense of [7 ] ), we conclude that
E Z , X [ sup z ∈ D ∣ F ( z ) − F ^ ( z ) ∣ ] = O ( L − 1 2 ) , \mathbb{E}_{Z,X}\left[\sup_{z\in D}\left|F(z)-\widehat{F}(z)\right|\right]=O\left(L^{-\frac{1}{2}}\right), E Z , X [ z ∈ D sup F ( z ) − F ( z ) ] = O ( L − 2 1 ) ,
which is (17). □
Remark 5.4. The error bound (18) depends only on f f f , { x i } \{x_i\} { x i } , M M M , L L L , and is independent of the spatial dimension n n n .
More importantly, we achieve the canonical Monte-Carlo rate O ( L − 1 / 2 ) O(L^{-1/2}) O ( L − 1/2 ) on the compact set D ‾ ⊂ C n \overline{D}\subset\mathbb{C}^n D ⊂ C n for the expected uniform error, not just root-mean-square error.
This pointwise error bound on D ⊂ C n D\subset\mathbb{C}^n D ⊂ C n leads to the pointwise error bound of f − f ^ f-\widehat{f} f − f on compact sets K ⊂ R n K\subset\mathbb{R}^n K ⊂ R n , provided that Assumption A and Assumption B hold.
Now we are ready to give a dimension-free pointwise error bound between f f f and f ^ \widehat{f} f within any compact subset K ⊂ R n K\subset\mathbb{R}^n K ⊂ R n . However, if we compute ∣ f − f ^ ∣ \left|f-\widehat{f}\right| f − f without using the Segal-Bargmann transform and our estimate of c ^ \widehat{c} c by complex data, we will have no control, even in the L 2 L^2 L 2 -sense, as Assumption B does not tell us how to recover information merely with data in L 2 ( R n ; R ) L^2(\mathbb{R}^n;\mathbb{R}) L 2 ( R n ; R ) ; or we might get an error bound suffers the curse of dimensionality. Note that Assumption A implies f ∈ C ∞ ( R n ; R ) f\in C^\infty(\mathbb{R}^n;\mathbb{R}) f ∈ C ∞ ( R n ; R ) immediately; therefore we can write f = ∑ β ∈ Λ M c β h β a f=\sum_{\beta\in\Lambda_M}c_\beta h_\beta^a f = ∑ β ∈ Λ M c β h β a in the pointwise sense.
Theorem 5.2. Let f ∈ L 2 ( R n ; R ) f\in L^2(\mathbb{R}^n;\mathbb{R}) f ∈ L 2 ( R n ; R ) , ∥ f ∥ L 2 ( R n ; R ) 2 ≤ C < ∞ \left\|f\right\|_{L^2(\mathbb{R}^n;\mathbb{R})}^2\leq C<\infty ∥ f ∥ L 2 ( R n ; R ) 2 ≤ C < ∞ and ∥ f ∥ L ∞ ( R n ; R ) = C ′ < ∞ \left\|f\right\|_{L^\infty(\mathbb{R}^n;\mathbb{R})}=C'<\infty ∥ f ∥ L ∞ ( R n ; R ) = C ′ < ∞ . Let D = B C n ( 0 ; R ) ⊂ C n D=B_{\mathbb{C}^n}(0;R)\subset\mathbb{C}^n D = B C n ( 0 ; R ) ⊂ C n and K = B R n ( 0 ; r ) ‾ ⊂ Re D = B R n ( 0 ; R ) ⊂ D K=\overline{B_{\mathbb{R}^n}(0;r)}\subset\operatorname{Re}D=B_{\mathbb{R}^n}(0;R)\subset D K = B R n ( 0 ; r ) ⊂ Re D = B R n ( 0 ; R ) ⊂ D . If both Assumption A and Assumption B hold with respect to random variables { Z l } l = 1 L ∼ i . i . d . ρ a ( z ) \{Z_l\}_{l=1}^L\mathrel{\overset{\mathrm{i.i.d.}}{\sim}}\rho_a(z) { Z l } l = 1 L ∼ i.i.d. ρ a ( z ) , then for each fixed dimension n n n and fixed truncation M = ∣ Λ M ∣ M=\left|\Lambda_M\right| M = ∣ Λ M ∣ , there exist an appropriate choice of a a a and a data-driven approximation f ^ ( x ) = ∑ β ∈ Λ M c ^ β h β a ( x ) \widehat{f}(x)=\sum_{\beta\in\Lambda_M}\widehat{c}_\beta h_\beta^a(x) f ( x ) = ∑ β ∈ Λ M c β h β a ( x ) such that the dimension-free algebraic error rate
E Z , X [ sup x ∈ K ∣ f ( x ) − f ^ ( x ) ∣ ] = O ( L − 1 2 ) , (21) \mathbb{E}_{Z,X}\left[\sup_{x\in K}\left|f(x)-\widehat{f}(x)\right|\right]=O\left(L^{-\frac{1}{2}}\right), \tag*{(21)} E Z , X [ x ∈ K sup f ( x ) − f ( x ) ] = O ( L − 2 1 ) , (21) holds, where c ^ = ( c ^ 1 , … , c ^ β , … , c ^ M ) \widehat{c}=(\widehat{c}_1,\ldots,\widehat{c}_\beta,\ldots,\widehat{c}_M) c = ( c 1 , … , c β , … , c M ) is determined by complex grids { z l } l = 1 L \{z_l\}_{l=1}^L { z l } l = 1 L that are sampling from { Z l } l = 1 L \{Z_l\}_{l=1}^L { Z l } l = 1 L . Precisely speaking,
E Z , X [ sup x ∈ K ∣ f ( x ) − f ^ ( x ) ∣ ] ≤ ( 2 a π ) n 4 ( e a R 2 2 ⋅ ( a R 2 ) n n ! + e − a ( R − r ) 2 s − n 2 ( 1 − s ) − n + B M 2 Γ ( n , s a R 2 ) Γ ( n ) ⋅ Γ ( n + B M , ( 1 − s ) a R 2 ) Γ ( n + B M ) ) ( M − 1 ) C 2 L + M ( C ′ ) 2 L . (22) \mathbb{E}_{Z,X}\left[\sup_{x\in K}\left|f(x)-\widehat{f}(x)\right|\right]
\leq
\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}
\left(e^{\frac{aR^2}{2}}\cdot\frac{(aR^2)^n}{n!}
+e^{-a(R-r)^2}s^{-\frac{n}{2}}(1-s)^{-\frac{n+B_M}{2}}
\sqrt{\frac{\Gamma(n,saR^2)}{\Gamma(n)}\cdot\frac{\Gamma(n+B_M,(1-s)aR^2)}{\Gamma(n+B_M)}}\right)
\sqrt{\frac{(M-1)C^2}{L}+\frac{M(C')^2}{L}}. \tag*{(22)} E Z , X [ x ∈ K sup f ( x ) − f ( x ) ] ≤ ( π 2 a ) 4 n ( e 2 a R 2 ⋅ n ! ( a R 2 ) n + e − a ( R − r ) 2 s − 2 n ( 1 − s ) − 2 n + B M Γ ( n ) Γ ( n , s a R 2 ) ⋅ Γ ( n + B M ) Γ ( n + B M , ( 1 − s ) a R 2 ) ) L ( M − 1 ) C 2 + L M ( C ′ ) 2 . (22)
Proof. We apply the inverse Segal-Bargmann transform
∣ f ( x ) − f ^ ( x ) ∣ = ∣ B a − 1 ( F − F ^ ) ∣ ≤ ( 2 a π ) n 4 ( a π ) n ∫ C n ∣ e − a 2 z ‾ ⋅ z ‾ − a ∣ x ∣ 2 + 2 a z ‾ ⋅ x ∣ ∣ F − F ^ ∣ e − a ∣ z ∣ 2 d μ ( z ) = ( 2 a π ) n 4 ( a π ) n ∫ ∣ z ∣ ≤ R ∣ e − a 2 z ‾ ⋅ z ‾ − a ∣ x ∣ 2 + 2 a z ‾ ⋅ x ∣ ∣ F − F ^ ∣ e − a ∣ z ∣ 2 d μ ( z ) + ( 2 a π ) n 4 ( a π ) n ∫ ∣ z ∣ > R ∣ e − a 2 z ‾ ⋅ z ‾ − a ∣ x ∣ 2 + 2 a z ‾ ⋅ x ∣ ∣ F − F ^ ∣ e − a ∣ z ∣ 2 d μ ( z ) = : E 1 ( x ) + E 2 ( x ) . \begin{aligned}
|f(x)-\widehat{f}(x)|=|\mathcal{B}_{a}^{-1}(F-\widehat{F})|\leq\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\left(\frac{a}{\pi}\right)^{n}\int_{\mathbb{C}^{n}}|e^{-\frac{a}{2}\overline{z}\cdot\overline{z}-a|x|^{2}+2a\overline{z}\cdot x}||F-\widehat{F}|e^{-a|z|^{2}}d\mu(z) \\
&=\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\left(\frac{a}{\pi}\right)^{n}\int_{|z|\leq R}|e^{-\frac{a}{2}\overline{z}\cdot\overline{z}-a|x|^{2}+2a\overline{z}\cdot x}||F-\widehat{F}|e^{-a|z|^{2}}d\mu(z) \\
&+\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\left(\frac{a}{\pi}\right)^{n}\int_{|z|>R}|e^{-\frac{a}{2}\overline{z}\cdot\overline{z}-a|x|^{2}+2a\overline{z}\cdot x}||F-\widehat{F}|e^{-a|z|^{2}}d\mu(z) \\
&=:E_{1}(x)+E_{2}(x).
\end{aligned} ∣ f ( x ) − f ( x ) ∣ = ∣ B a − 1 ( F − F ) ∣ ≤ ( π 2 a ) 4 n ( π a ) n ∫ C n ∣ e − 2 a z ⋅ z − a ∣ x ∣ 2 + 2 a z ⋅ x ∣∣ F − F ∣ e − a ∣ z ∣ 2 d μ ( z ) = ( π 2 a ) 4 n ( π a ) n ∫ ∣ z ∣ ≤ R ∣ e − 2 a z ⋅ z − a ∣ x ∣ 2 + 2 a z ⋅ x ∣∣ F − F ∣ e − a ∣ z ∣ 2 d μ ( z ) + ( π 2 a ) 4 n ( π a ) n ∫ ∣ z ∣ > R ∣ e − 2 a z ⋅ z − a ∣ x ∣ 2 + 2 a z ⋅ x ∣∣ F − F ∣ e − a ∣ z ∣ 2 d μ ( z ) =: E 1 ( x ) + E 2 ( x ) .
Set z = u + i v z=u+iv z = u + i v , u , v ∈ R n u,v\in\mathbb{R}^{n} u , v ∈ R n and ω 2 n − 1 = ( 2 π ) n Γ ( n ) \omega^{2n-1}=\frac{(2\pi)^{n}}{\Gamma(n)} ω 2 n − 1 = Γ ( n ) ( 2 π ) n be the surface area of S 2 n \mathbb{S}^{2n} S 2 n , we analyze E 1 ( x ) E_{1}(x) E 1 ( x ) and E 2 ( x ) E_{2}(x) E 2 ( x ) for x ∈ K x\in K x ∈ K . For E 1 ( x ) E_{1}(x) E 1 ( x ) , we use (18) and compute that
sup x ∈ K E 1 ( x ) ≤ sup ∣ z ∣ ≤ R ∣ F ( z ) − F ^ ( z ) ∣ ⋅ ( 2 a π ) n 4 ( a π ) n sup x ∈ K ∫ ∣ u ∣ 2 + ∣ v ∣ 2 ≤ R 2 e − a ∣ u − x ∣ 2 e − a 2 ∣ u ∣ 2 e − a 2 ∣ v ∣ 2 d u d v ≤ sup ∣ z ∣ ≤ R ∣ F ( z ) − F ^ ( z ) ∣ ⋅ ( 2 a π ) n 4 ( a π ) n ω 2 n − 1 ∫ 0 R e − a t 2 2 t 2 n − 1 d t = sup ∣ z ∣ ≤ R ∣ F ( z ) − F ^ ( z ) ∣ ⋅ ( 2 a π ) n 4 ( a π ) n ( 2 π ) n a n Γ ( n ) ⋅ γ ( n , a R 2 2 ) ≤ sup ∣ z ∣ ≤ R ∣ F ( z ) − F ^ ( z ) ∣ ⋅ ( 2 a π ) n 4 ⋅ 2 n ⋅ ( a R 2 ) n 2 n n ! ≤ e a R 2 2 ⋅ ( 2 a π ) n 4 ⋅ ( a R 2 ) n n ! ⋅ ∥ c − c ^ ∥ . \begin{aligned}
\sup_{x\in K}E_{1}(x)\leq\sup_{|z|\leq R}|F(z)-\widehat{F}(z)|\cdot\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\left(\frac{a}{\pi}\right)^{n}\sup_{x\in K}\int_{|u|^{2}+|v|^{2}\leq R^{2}}e^{-a|u-x|^{2}}e^{-\frac{a}{2}|u|^{2}}e^{-\frac{a}{2}|v|^{2}}\,du\,dv \\
&\leq\sup_{|z|\leq R}|F(z)-\widehat{F}(z)|\cdot\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\left(\frac{a}{\pi}\right)^{n}\omega^{2n-1}\int_{0}^{R}e^{-\frac{at^{2}}{2}}t^{2n-1}\,dt \\
&=\sup_{|z|\leq R}|F(z)-\widehat{F}(z)|\cdot\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\left(\frac{a}{\pi}\right)^{n}\frac{(2\pi)^{n}}{a^{n}\Gamma(n)}\cdot\gamma\left(n,\frac{aR^{2}}{2}\right) \\
&\leq\sup_{|z|\leq R}|F(z)-\widehat{F}(z)|\cdot\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\cdot2^{n}\cdot\frac{(aR^{2})^{n}}{2^{n}n!} \\
&\leq e^{\frac{aR^{2}}{2}}\cdot\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\cdot\frac{(aR^{2})^{n}}{n!}\cdot\|\mathbf{c}-\widehat{\mathbf{c}}\|.
\end{aligned} x ∈ K sup E 1 ( x ) ≤ ∣ z ∣ ≤ R sup ∣ F ( z ) − F ( z ) ∣ ⋅ ( π 2 a ) 4 n ( π a ) n x ∈ K sup ∫ ∣ u ∣ 2 + ∣ v ∣ 2 ≤ R 2 e − a ∣ u − x ∣ 2 e − 2 a ∣ u ∣ 2 e − 2 a ∣ v ∣ 2 d u d v ≤ ∣ z ∣ ≤ R sup ∣ F ( z ) − F ( z ) ∣ ⋅ ( π 2 a ) 4 n ( π a ) n ω 2 n − 1 ∫ 0 R e − 2 a t 2 t 2 n − 1 d t = ∣ z ∣ ≤ R sup ∣ F ( z ) − F ( z ) ∣ ⋅ ( π 2 a ) 4 n ( π a ) n a n Γ ( n ) ( 2 π ) n ⋅ γ ( n , 2 a R 2 ) ≤ ∣ z ∣ ≤ R sup ∣ F ( z ) − F ( z ) ∣ ⋅ ( π 2 a ) 4 n ⋅ 2 n ⋅ 2 n n ! ( a R 2 ) n ≤ e 2 a R 2 ⋅ ( π 2 a ) 4 n ⋅ n ! ( a R 2 ) n ⋅ ∥ c − c ∥.
We now control the tail term E 2 ( x ) E_{2}(x) E 2 ( x ) . For any constant s ∈ ( 0 , 1 ) s\in(0,1) s ∈ ( 0 , 1 ) , we have
sup x ∈ K E 2 ( x ) = ( 2 a π ) n 4 ( a π ) n sup x ∈ K ∫ ∣ z ∣ > R ( ∣ e − a 2 z ‾ ⋅ z ‾ − a ∣ x ∣ 2 + 2 a z ‾ ⋅ x ∣ e − ( s + 1 ) a 2 ∣ z ∣ 2 ) ( ∣ F − F ^ ∣ e − − ( s + 1 ) a 2 ∣ z ∣ 2 ) d μ ( z ) ≤ ( 2 a π ) n 4 ( ( a π ) n sup x ∈ K ∫ ∣ z ∣ > R ∣ e − a z ‾ ⋅ z ‾ − 2 a ∣ x ∣ 2 + 4 a z ‾ ⋅ x ∣ e − ( s + 1 ) a ∣ z ∣ 2 d μ ( z ) ) 1 2 × ( ( a π ) n ∫ ∣ z ∣ > R ∣ F − F ^ ∣ 2 e − ( s + 1 ) a ∣ z ∣ 2 d μ ( z ) ) 1 2 = ( 2 a π ) n 4 sup x ∈ K E 2 , 1 ( x ) ⋅ E 2 , 2 . \begin{aligned}
\sup_{x\in K}E_{2}(x)=\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\left(\frac{a}{\pi}\right)^{n}\sup_{x\in K}\int_{|z|>R}\left(|e^{-\frac{a}{2}\overline{z}\cdot\overline{z}-a|x|^{2}+2a\overline{z}\cdot x}|e^{-\frac{(s+1)a}{2}|z|^{2}}\right)\left(|F-\widehat{F}|e^{-\frac{-(s+1)a}{2}|z|^{2}}\right)d\mu(z) \\
&\leq\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\left(\left(\frac{a}{\pi}\right)^{n}\sup_{x\in K}\int_{|z|>R}|e^{-a\overline{z}\cdot\overline{z}-2a|x|^{2}+4a\overline{z}\cdot x}|e^{-(s+1)a|z|^{2}}d\mu(z)\right)^{\frac{1}{2}} \\
&\times\left(\left(\frac{a}{\pi}\right)^{n}\int_{|z|>R}|F-\widehat{F}|^{2}e^{-(s+1)a|z|^{2}}d\mu(z)\right)^{\frac{1}{2}} \\
&=\left(\frac{2a}{\pi}\right)^{\frac{n}{4}}\sup_{x\in K}E_{2,1}(x)\cdot E_{2,2}.
\end{aligned} x ∈ K sup E 2 ( x ) = ( π 2 a ) 4 n ( π a ) n x ∈ K sup ∫ ∣ z ∣ > R ( ∣ e − 2 a z ⋅ z − a ∣ x ∣ 2 + 2 a z ⋅ x ∣ e − 2 ( s + 1 ) a ∣ z ∣ 2 ) ( ∣ F − F ∣ e − 2 − ( s + 1 ) a ∣ z ∣ 2 ) d μ ( z ) ≤ ( π 2 a ) 4 n ( ( π a ) n x ∈ K sup ∫ ∣ z ∣ > R ∣ e − a z ⋅ z − 2 a ∣ x ∣ 2 + 4 a z ⋅ x ∣ e − ( s + 1 ) a ∣ z ∣ 2 d μ ( z ) ) 2 1 × ( ( π a ) n ∫ ∣ z ∣ > R ∣ F − F ∣ 2 e − ( s + 1 ) a ∣ z ∣ 2 d μ ( z ) ) 2 1 = ( π 2 a ) 4 n x ∈ K sup E 2 , 1 ( x ) ⋅ E 2 , 2 .
Set B M = sup β ∈ Λ M ∣ β ∣ B_M=\sup_{\beta\in\Lambda_M}|\beta| B M = sup β ∈ Λ M ∣ β ∣ . For E 2 , 2 E_{2,2} E 2 , 2 , we recall Assumption A and the orthonormality of { e β a } \{e^a_\beta\} { e β a } (which still holds on any radial region), which implies that
E 2 , 2 2 = ( a π ) n ∫ ∣ z ∣ > R ∣ F − F ^ ∣ 2 e − ( s + 1 ) a ∣ z ∣ 2 d μ ( z ) = ∑ β ∈ Λ M ∣ c ^ β − c β ∣ 2 ∫ ∣ z ∣ > R e β a ( z ) e β a ( z ‾ ) e s a ∣ z ∣ 2 d μ a ( z ) ≤ ∑ β ∈ Λ M ∣ c ^ β − c β ∣ 2 ( a π ) n a ∣ β ∣ β ! ∫ S 2 n − 1 ∣ ω β ∣ 2 d σ ( ω ) ∫ R ∞ t 2 n + 2 ∣ β ∣ − 1 e − ( 1 − s ) a t 2 d t = ∑ β ∈ Λ M ∣ c ^ β − c β ∣ 2 ( 1 − s ) − ( n + ∣ β ∣ ) ⋅ Γ ( n + ∣ β ∣ , ( 1 − s ) a R 2 ) Γ ( n + ∣ β ∣ ) ≤ ( 1 − s ) − ( n + B M ) ⋅ Γ ( n + B M , ( 1 − s ) a R 2 ) Γ ( n + B M ) ∑ β ∈ Λ M ∣ c ^ β − c β ∣ 2 = ( 1 − s ) − ( n + B M ) ⋅ Γ ( n + B M , ( 1 − s ) a R 2 ) Γ ( n + B M ) ∥ F − F ^ ∥ F a 2 ( C n ) . \begin{aligned}
E_{2,2}^2=\left(\frac{a}{\pi}\right)^n\int_{|z|>R}|F-\widehat{F}|^2e^{-(s+1)a|z|^2}\,d\mu(z) \\
&=\sum_{\beta\in\Lambda_M}|\widehat{c}_\beta-c_\beta|^2\int_{|z|>R}e^a_\beta(z)e^a_\beta(\overline{z})e^{sa|z|^2}\,d\mu_a(z) \\
&\leq\sum_{\beta\in\Lambda_M}|\widehat{c}_\beta-c_\beta|^2\left(\frac{a}{\pi}\right)^n\frac{a^{|\beta|}}{\beta!}\int_{S^{2n-1}}|\omega^\beta|^2\,d\sigma(\omega)\int_R^\infty t^{2n+2|\beta|-1}e^{-(1-s)at^2}\,dt \\
&=\sum_{\beta\in\Lambda_M}|\widehat{c}_\beta-c_\beta|^2(1-s)^{-(n+|\beta|)}\cdot\frac{\Gamma\left(n+|\beta|,(1-s)aR^2\right)}{\Gamma(n+|\beta|)} \\
&\leq(1-s)^{-(n+B_M)}\cdot\frac{\Gamma\left(n+B_M,(1-s)aR^2\right)}{\Gamma(n+B_M)}\sum_{\beta\in\Lambda_M}|\widehat{c}_\beta-c_\beta|^2 \\
&=(1-s)^{-(n+B_M)}\cdot\frac{\Gamma\left(n+B_M,(1-s)aR^2\right)}{\Gamma(n+B_M)}\|F-\widehat{F}\|_{\mathcal{F}^2_a(\mathbb{C}^n)}.
\end{aligned} E 2 , 2 2 = ( π a ) n ∫ ∣ z ∣ > R ∣ F − F ∣ 2 e − ( s + 1 ) a ∣ z ∣ 2 d μ ( z ) = β ∈ Λ M ∑ ∣ c β − c β ∣ 2 ∫ ∣ z ∣ > R e β a ( z ) e β a ( z ) e s a ∣ z ∣ 2 d μ a ( z ) ≤ β ∈ Λ M ∑ ∣ c β − c β ∣ 2 ( π a ) n β ! a ∣ β ∣ ∫ S 2 n − 1 ∣ ω β ∣ 2 d σ ( ω ) ∫ R ∞ t 2 n + 2∣ β ∣ − 1 e − ( 1 − s ) a t 2 d t = β ∈ Λ M ∑ ∣ c β − c β ∣ 2 ( 1 − s ) − ( n + ∣ β ∣ ) ⋅ Γ ( n + ∣ β ∣ ) Γ ( n + ∣ β ∣ , ( 1 − s ) a R 2 ) ≤ ( 1 − s ) − ( n + B M ) ⋅ Γ ( n + B M ) Γ ( n + B M , ( 1 − s ) a R 2 ) β ∈ Λ M ∑ ∣ c β − c β ∣ 2 = ( 1 − s ) − ( n + B M ) ⋅ Γ ( n + B M ) Γ ( n + B M , ( 1 − s ) a R 2 ) ∥ F − F ∥ F a 2 ( C n ) .
Using the same polar coordinates approach,
sup x ∈ K E 2 , 1 ( x ) 2 ≤ ( a π ) n ∫ ∣ u ∣ 2 + ∣ v ∣ 2 > R 2 e − a s ( ∣ u ∣ 2 + ∣ v ∣ 2 ) ( sup x ∈ K e − 2 a ∣ u − x ∣ 2 ) d u d v ≤ ( a π ) n e − 2 a ( R − r ) 2 ⋅ ω 2 n − 1 ∫ R ∞ e − a s t 2 t 2 n − 1 d t = e − 2 a ( R − r ) 2 s − n Γ ( n , s a R 2 ) Γ ( n ) . \begin{aligned}
\sup_{x\in K}E_{2,1}(x)^2\leq\left(\frac{a}{\pi}\right)^n\int_{|u|^2+|v|^2>R^2}e^{-as(|u|^2+|v|^2)}\left(\sup_{x\in K}e^{-2a|u-x|^2}\right)\,du\,dv \\
&\leq\left(\frac{a}{\pi}\right)^n e^{-2a(R-r)^2}\cdot\omega_{2n-1}\int_R^\infty e^{-ast^2}t^{2n-1}\,dt \\
&=e^{-2a(R-r)^2}s^{-n}\frac{\Gamma(n,saR^2)}{\Gamma(n)}.
\end{aligned} x ∈ K sup E 2 , 1 ( x ) 2 ≤ ( π a ) n ∫ ∣ u ∣ 2 + ∣ v ∣ 2 > R 2 e − a s ( ∣ u ∣ 2 + ∣ v ∣ 2 ) ( x ∈ K sup e − 2 a ∣ u − x ∣ 2 ) d u d v ≤ ( π a ) n e − 2 a ( R − r ) 2 ⋅ ω 2 n − 1 ∫ R ∞ e − a s t 2 t 2 n − 1 d t = e − 2 a ( R − r ) 2 s − n Γ ( n ) Γ ( n , s a R 2 ) .
It follows that
sup x ∈ K E 2 ( x ) ≤ ( 2 a π ) n 4 ⋅ e − a ( R − r ) 2 s − n 2 ( 1 − s ) − n + B M 2 Γ ( n , s a R 2 ) Γ ( n ) ⋅ Γ ( n + B M , ( 1 − s ) a R 2 ) Γ ( n + B M ) ∥ F − F ^ ∥ F a 2 ( C n ) ≤ ( 2 a π ) n 4 ⋅ e − a ( R − r ) 2 s − n 2 ( 1 − s ) − n + B M 2 Γ ( n , s a R 2 ) Γ ( n ) ⋅ Γ ( n + B M , ( 1 − s ) a R 2 ) Γ ( n + B M ) ∥ c − c ^ ∥ . \begin{aligned}
\sup_{x\in K}E_2(x)\leq\left(\frac{2a}{\pi}\right)^{\frac n4}\cdot e^{-a(R-r)^2}s^{-\frac n2}(1-s)^{-\frac{n+B_M}{2}}\sqrt{\frac{\Gamma(n,saR^2)}{\Gamma(n)}\cdot\frac{\Gamma\left(n+B_M,(1-s)aR^2\right)}{\Gamma(n+B_M)}}\|F-\widehat{F}\|_{\mathcal{F}^2_a(\mathbb{C}^n)} \\
&\leq\left(\frac{2a}{\pi}\right)^{\frac n4}\cdot e^{-a(R-r)^2}s^{-\frac n2}(1-s)^{-\frac{n+B_M}{2}}\sqrt{\frac{\Gamma(n,saR^2)}{\Gamma(n)}\cdot\frac{\Gamma\left(n+B_M,(1-s)aR^2\right)}{\Gamma(n+B_M)}}\|c-\widehat{c}\|.
\end{aligned} x ∈ K sup E 2 ( x ) ≤ ( π 2 a ) 4 n ⋅ e − a ( R − r ) 2 s − 2 n ( 1 − s ) − 2 n + B M Γ ( n ) Γ ( n , s a R 2 ) ⋅ Γ ( n + B M ) Γ ( n + B M , ( 1 − s ) a R 2 ) ∥ F − F ∥ F a 2 ( C n ) ≤ ( π 2 a ) 4 n ⋅ e − a ( R − r ) 2 s − 2 n ( 1 − s ) − 2 n + B M Γ ( n ) Γ ( n , s a R 2 ) ⋅ Γ ( n + B M ) Γ ( n + B M , ( 1 − s ) a R 2 ) ∥ c − c ∥.
Taking the expectation of the sum of the error bounds of E 1 E_1 E 1 and E 2 E_2 E 2 gives (22). We can choose a a a small enough, such that the possible dimensional growths in n n n are compensated. It follows that (21) holds. □
Since Assumption A holds, we can use the Cauchy-Schwarz inequality to get another upper bound of ∣ f − f ^ ∣ |f-\widehat{f}| ∣ f − f ∣ on R n \mathbb{R}^n R n . The following error bound is worse than (22) in general because of the extra M \sqrt{M} M term.
Proposition 5.1. Under the same hypotheses of Theorem 5.2, and the same construction of c ^ \widehat{c} c , the data-driven approximation f ^ ( x ) = ∑ β ∈ Λ M c ^ β h β ( x ) \widehat{f}(x)=\sum_{\beta\in\Lambda_M}\widehat{c}_\beta h^\beta(x) f ( x ) = ∑ β ∈ Λ M c β h β ( x ) satisfies the dimension-free algebraic error rate
E Z , X [ sup x ∈ R n ∣ f ( x ) − f ^ ( x ) ∣ ] = O ( L − 1 2 ) . (23) \mathbb{E}_{Z,X}\left[\sup_{x\in\mathbb{R}^n}|f(x)-\widehat{f}(x)|\right]=O(L^{-\frac12}). \tag*{(23)} E Z , X [ x ∈ R n sup ∣ f ( x ) − f ( x ) ∣ ] = O ( L − 2 1 ) . (23)
Precisely speaking,
E X , Z [ sup x ∈ R n ∣ f ( x ) − f ^ ( x ) ∣ ] ≤ C 0 n ( 2 a ) n 4 ( 1 + B M n ) − n 12 ⋅ M ( M − 1 ) C 2 L + M ( C ′ ) 2 L . (24) \mathbb{E}_{X,Z}\left[\sup_{x\in\mathbb{R}^n}|f(x)-\widehat{f}(x)|\right]\leq C_0^n(2a)^{\frac n4}\left(1+\frac{B_M}{n}\right)^{-\frac n{12}}\cdot\sqrt{M}\sqrt{\frac{(M-1)C^2}{L}+\frac{M(C')^2}{L}}. \tag*{(24)} E X , Z [ x ∈ R n sup ∣ f ( x ) − f ( x ) ∣ ] ≤ C 0 n ( 2 a ) 4 n ( 1 + n B M ) − 12 n ⋅ M L ( M − 1 ) C 2 + L M ( C ′ ) 2 . (24)
Proof. By the same complex sampling procedure as in Theorem 5.1, we apply Cauchy-Schwarz and get
E Z , X [ sup x ∈ R n ∣ f ( x ) − f ^ ( x ) ∣ ] = E Z , X [ sup x ∈ R n ∣ ∑ β ∈ Λ M ( c β − c ^ β ) h β a ∣ ] ≤ sup x ∈ R n , β ∈ Λ M ∣ h β a ∣ ⋅ E Z , X [ ∑ β ∈ Λ M ∣ c β − c ^ β ∣ ] ≤ sup x ∈ R n , β ∈ Λ M ∣ h β a ∣ ( ∑ β ∈ Λ M 1 ) 1 2 E ρ a [ ( ∑ β ∈ Λ M ∣ c β − c ^ β ∣ 2 ) 1 2 ] ≤ sup x ∈ R n , β ∈ Λ M ∣ h β a ∣ ⋅ M ( M − 1 ) C 2 L + M ( C ′ ) 2 L \begin{aligned}
\mathbb{E}_{Z,X}\left[\sup_{x\in\mathbb{R}^{n}}|f(x)-\hat{f}(x)|\right]=\mathbb{E}_{Z,X}\left[\sup_{x\in\mathbb{R}^{n}}\left|\sum_{\beta\in\Lambda_{M}}(c_{\beta}-\hat{c}_{\beta})h_{\beta}^{a}\right|\right]\leq\sup_{x\in\mathbb{R}^{n},\beta\in\Lambda_{M}}|h_{\beta}^{a}|\cdot\mathbb{E}_{Z,X}\left[\sum_{\beta\in\Lambda_{M}}|c_{\beta}-\hat{c}_{\beta}|\right] \\
&\leq\sup_{x\in\mathbb{R}^{n},\beta\in\Lambda_{M}}|h_{\beta}^{a}|\left(\sum_{\beta\in\Lambda_{M}}1\right)^{\frac{1}{2}}\mathbb{E}_{\rho_{a}}\left[\left(\sum_{\beta\in\Lambda_{M}}|c_{\beta}-\hat{c}_{\beta}|^{2}\right)^{\frac{1}{2}}\right] \\
&\leq\sup_{x\in\mathbb{R}^{n},\beta\in\Lambda_{M}}|h_{\beta}^{a}|\cdot\sqrt{M}\sqrt{\frac{(M-1)C^{2}}{L}+\frac{M(C')^{2}}{L}}
\end{aligned} E Z , X [ x ∈ R n sup ∣ f ( x ) − f ^ ( x ) ∣ ] = E Z , X x ∈ R n sup β ∈ Λ M ∑ ( c β − c ^ β ) h β a ≤ x ∈ R n , β ∈ Λ M sup ∣ h β a ∣ ⋅ E Z , X β ∈ Λ M ∑ ∣ c β − c ^ β ∣ ≤ x ∈ R n , β ∈ Λ M sup ∣ h β a ∣ β ∈ Λ M ∑ 1 2 1 E ρ a β ∈ Λ M ∑ ∣ c β − c ^ β ∣ 2 2 1 ≤ x ∈ R n , β ∈ Λ M sup ∣ h β a ∣ ⋅ M L ( M − 1 ) C 2 + L M ( C ′ ) 2
by the upper bound of E Z , X [ ∥ c − c ^ ∥ ℓ M 2 2 ] \mathbb{E}_{Z,X}[\|c-\hat{c}\|_{\ell^{2}_{M}}^{2}] E Z , X [ ∥ c − c ^ ∥ ℓ M 2 2 ] in Theorem 5.1. The canonical upper bound of one-dimensional physicists’ Hermitian polynomial says
sup y ∈ R ∣ h m a ( y ) ∣ = sup y ∈ R ∣ ( 2 a ) 1 4 H m ( 2 a y ) ∣ ≤ C 0 ( 2 a ) 1 4 ( m + 1 ) − 1 12 . \sup_{y\in\mathbb{R}}|h_{m}^{a}(y)|=\sup_{y\in\mathbb{R}}|(2a)^{\frac{1}{4}}H_{m}(\sqrt{2a}y)|\leq C_{0}(2a)^{\frac{1}{4}}(m+1)^{-\frac{1}{12}}. y ∈ R sup ∣ h m a ( y ) ∣ = y ∈ R sup ∣ ( 2 a ) 4 1 H m ( 2 a y ) ∣ ≤ C 0 ( 2 a ) 4 1 ( m + 1 ) − 12 1 .
Here C 0 < 1 C_{0}<1 C 0 < 1 is a constant independent of a a a and m m m . Applying this upper bound and the standard AM-GM inequality to h β a h_{\beta}^{a} h β a with β = ( β 1 , … , β n ) \beta=(\beta_{1},\ldots,\beta_{n}) β = ( β 1 , … , β n ) , we get
sup x ∈ R n ∣ h β a ( x ) ∣ ≤ C 0 n ( 2 a ) n 4 ∏ j = 1 n ( β j + 1 ) − 1 12 = C 0 n ( 2 a ) n 4 ( ( ∏ j = 1 n ( β j + 1 ) ) 1 n ) − n 12 ≤ C 0 n ( 2 a ) n 4 ( 1 n ∑ j = 1 n ( β j + 1 ) ) − n 12 = C 0 n ( 2 a ) n 4 ( 1 + ∣ β ∣ n ) − n 12 . \begin{aligned}
\sup_{x\in\mathbb{R}^{n}}|h_{\beta}^{a}(x)|\leq C_{0}^{n}(2a)^{\frac{n}{4}}\prod_{j=1}^{n}(\beta_{j}+1)^{-\frac{1}{12}}=C_{0}^{n}(2a)^{\frac{n}{4}}\left(\left(\prod_{j=1}^{n}(\beta_{j}+1)\right)^{\frac{1}{n}}\right)^{-\frac{n}{12}} \\
&\leq C_{0}^{n}(2a)^{\frac{n}{4}}\left(\frac{1}{n}\sum_{j=1}^{n}(\beta_{j}+1)\right)^{-\frac{n}{12}}=C_{0}^{n}(2a)^{\frac{n}{4}}\left(1+\frac{|\beta|}{n}\right)^{-\frac{n}{12}}.
\end{aligned} x ∈ R n sup ∣ h β a ( x ) ∣ ≤ C 0 n ( 2 a ) 4 n j = 1 ∏ n ( β j + 1 ) − 12 1 = C 0 n ( 2 a ) 4 n ( j = 1 ∏ n ( β j + 1 ) ) n 1 − 12 n ≤ C 0 n ( 2 a ) 4 n ( n 1 j = 1 ∑ n ( β j + 1 ) ) − 12 n = C 0 n ( 2 a ) 4 n ( 1 + n ∣ β ∣ ) − 12 n .
Therefore,
E Z , X [ sup x ∈ R n ∣ f ( x ) − f ^ ( x ) ∣ ] ≤ C 0 n ( 2 a ) n 4 ( 1 + B M n ) − n 12 ⋅ M ( M − 1 ) C 2 L + M ( C ′ ) 2 L , \mathbb{E}_{Z,X}\left[\sup_{x\in\mathbb{R}^{n}}|f(x)-\hat{f}(x)|\right]\leq C_{0}^{n}(2a)^{\frac{n}{4}}\left(1+\frac{B_{M}}{n}\right)^{-\frac{n}{12}}\cdot\sqrt{M}\sqrt{\frac{(M-1)C^{2}}{L}+\frac{M(C')^{2}}{L}}, E Z , X [ x ∈ R n sup ∣ f ( x ) − f ^ ( x ) ∣ ] ≤ C 0 n ( 2 a ) 4 n ( 1 + n B M ) − 12 n ⋅ M L ( M − 1 ) C 2 + L M ( C ′ ) 2 ,
which gives (24). Now we can choose a a a small enough to compensate for any potential growth depending on n n n , which follows that
E ρ a [ sup x ∈ R n ∣ f ( x ) − f ^ ( x ) ∣ ] = O ( L − 1 2 ) , \mathbb{E}_{\rho_{a}}\left[\sup_{x\in\mathbb{R}^{n}}|f(x)-\hat{f}(x)|\right]=O\left(L^{-\frac{1}{2}}\right), E ρ a [ x ∈ R n sup ∣ f ( x ) − f ^ ( x ) ∣ ] = O ( L − 2 1 ) ,
which is exactly the dimension-free error bound in (23). □
Analogous to Remark 5.4, we achieve the canonical Monte-Carlo rate for the pointwise error estimate between f f f and f ^ \hat{f} f ^ , which is again independent of the dimension. The upper bound in (24) may not be sharp once we restrict x ∈ K x\in K x ∈ K for some compact subset K K K . (24) just holds uniformly on R n \mathbb{R}^{n} R n .
Assumption B is essential. We give an error bound between f f f and f ^ \hat{f} f ^ below when we drop the Assumption A , i.e. f = ∑ β c β h β a f=\sum_{\beta}c_{\beta}h_{\beta}^{a} f = ∑ β c β h β a as an infinite series. Merely with Assumption B and f ∈ H a s ( R n ; R ) f\in\mathcal{H}_{a}^{s}(\mathbb{R}^{n};\mathbb{R}) f ∈ H a s ( R n ; R ) when s < n 2 s<\frac{n}{2} s < 2 n , we usually have no L 2 L^{2} L 2 -control between f f f and f ^ \hat{f} f ^ whose coefficients are estimated from data in original space only.
Recall the RKHS kernel K a ( z , w ) = e a z ⋅ w ‾ K_{a}(z,w)=e^{az\cdot\overline{w}} K a ( z , w ) = e a z ⋅ w . Denote the normalized RKHS kernel with complex variable z z z and complex parameter w w w by
k a ( z , w ) = e − a 2 ∣ w ∣ 2 K a ( z , w ) . k_{a}(z,w)=e^{-\frac{a}{2}|w|^{2}}K_{a}(z,w). k a ( z , w ) = e − 2 a ∣ w ∣ 2 K a ( z , w ) .
By the Seip’s sampling theorem for Bargmann-Fock space [43 ] , there exists a deterministic complex sampling set Λ = { z l } l = 1 ∞ \Lambda=\{z_l\}_{l=1}^{\infty} Λ = { z l } l = 1 ∞ such that { k a ( z , z l ) } l = 1 ∞ \{k_a(z,z_l)\}_{l=1}^{\infty} { k a ( z , z l ) } l = 1 ∞ is a frame for F a 2 ( C n ) \mathcal{F}_a^2(\mathbb{C}^n) F a 2 ( C n ) with respect to two positive constants A A A , B B B . Define the frame operator
S F ( z ) = ∑ l ⟨ F , k a ( ⋅ , z l ) ⟩ F a 2 k a ( z , z l ) SF(z)=\sum_l\langle F,k_a(\cdot,z_l)\rangle_{\mathcal{F}_a^2}k_a(z,z_l) S F ( z ) = l ∑ ⟨ F , k a ( ⋅ , z l ) ⟩ F a 2 k a ( z , z l )
By the Seip’s sampling theorem,
A I ≤ S ≤ B I . AI\leq S\leq BI. A I ≤ S ≤ B I .
Hence S − 1 S^{-1} S − 1 is well-defined. We define the normalized dual RKHS kernel
k ~ a ( z , z l ) = S − 1 k a ( z , z l ) ⇒ K ~ a ( z , z l ) = e a 2 ∣ z l ∣ 2 k ~ a ( z , z l ) . \widetilde{k}_a(z,z_l)=S^{-1}k_a(z,z_l)\Rightarrow\widetilde{K}_a(z,z_l)=e^{\frac{a}{2}|z_l|^2}\widetilde{k}_a(z,z_l). k a ( z , z l ) = S − 1 k a ( z , z l ) ⇒ K a ( z , z l ) = e 2 a ∣ z l ∣ 2 k a ( z , z l ) .
We conclude that
F ( z ) = ∑ l ⟨ F , k a ( ⋅ , z l ) ⟩ F a 2 k ~ a ( z , z l ) = ∑ l ⟨ F , K a ( ⋅ , z l ) ⟩ F a 2 K ~ a ( z , z l ) = ∑ l F ( z l ) K ~ a ( z , z l ) . (25) F(z)=\sum_l\langle F,k_a(\cdot,z_l)\rangle_{\mathcal{F}_a^2}\widetilde{k}_a(z,z_l)=\sum_l\langle F,K_a(\cdot,z_l)\rangle_{\mathcal{F}_a^2}\widetilde{K}_a(z,z_l)=\sum_lF(z_l)\widetilde{K}_a(z,z_l). \tag*{(25)} F ( z ) = l ∑ ⟨ F , k a ( ⋅ , z l ) ⟩ F a 2 k a ( z , z l ) = l ∑ ⟨ F , K a ( ⋅ , z l ) ⟩ F a 2 K a ( z , z l ) = l ∑ F ( z l ) K a ( z , z l ) . (25)
Our next result shows that with the Segal-Bargmann transform and the special Seip-type complex grids, we can still get algebraic convergence rate even when s < d 2 s<\frac{d}{2} s < 2 d .
Theorem 5.3. Let f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) ∩ H a s f\in L^2(\mathbb{R}^n;\mathbb{R})\cap C^\infty(\mathbb{R}^n;\mathbb{R})\cap\mathcal{H}_a^s f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) ∩ H a s such that ∥ f ∥ H a s ( R n ; R ) ≤ C 1 \|f\|_{\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R})}\leq C_1 ∥ f ∥ H a s ( R n ; R ) ≤ C 1 . If only Assumption B holds for some complex random sampling { V l } l = 1 L ∼ i . i . d . ρ V \{V_l\}_{l=1}^L\overset{\mathrm{i.i.d.}}{\sim}\rho_V { V l } l = 1 L ∼ i.i.d. ρ V with μ a ≪ ρ V \mu_a\ll\rho_V μ a ≪ ρ V , there exists a data-driven approximation f ^ = ∑ l = 1 L d ^ l B a − 1 ( K ~ a ( ⋅ , z l ) ) ( x ) \widehat{f}=\sum_{l=1}^L\widehat{d}_lB_a^{-1}(\widetilde{K}_a(\cdot,z_l))(x) f = ∑ l = 1 L d l B a − 1 ( K a ( ⋅ , z l )) ( x ) such that
E V , X [ ∥ f − f ^ ∥ L 2 ( R n ; R ) ] ≤ D a , s , Λ L − s 2 n ∥ f ∥ H a s ( R n ; R ) + D a , s , Λ ′ L − 1 2 ∥ f ∥ L 2 ( R n ; R ) , (26) \mathbb{E}_{V,X}\left[\|f-\widehat{f}\|_{L^2(\mathbb{R}^n;\mathbb{R})}\right]\leq D_{a,s,\Lambda}L^{-\frac{s}{2n}}\|f\|_{\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R})}+D'_{a,s,\Lambda}L^{-\frac{1}{2}}\|f\|_{L^2(\mathbb{R}^n;\mathbb{R})}, \tag*{(26)} E V , X [ ∥ f − f ∥ L 2 ( R n ; R ) ] ≤ D a , s , Λ L − 2 n s ∥ f ∥ H a s ( R n ; R ) + D a , s , Λ ′ L − 2 1 ∥ f ∥ L 2 ( R n ; R ) , (26) where d ^ = ( d ^ 1 , … , d ^ L ) \widehat{\mathbf{d}}=(\widehat{d}_1,\ldots,\widehat{d}_L) d = ( d 1 , … , d L ) is determined by random sampling of { V l } l = 1 L ∼ i . i . d . ρ V \{V_l\}_{l=1}^L\overset{\mathrm{i.i.d.}}{\sim}\rho_V { V l } l = 1 L ∼ i.i.d. ρ V and { f ( X i ) } i = 1 N \{f(X_i)\}_{i=1}^N { f ( X i ) } i = 1 N . In particular, we get the following algebraic error bound
E V , X [ ∥ f − f ^ ∥ L 2 ( R n ; R ) ] = O ( L − s 2 n ) . (27) \mathbb{E}_{V,X}\left[\|f-\widehat{f}\|_{L^2(\mathbb{R}^n;\mathbb{R})}\right]=O\left(L^{-\frac{s}{2n}}\right). \tag*{(27)} E V , X [ ∥ f − f ∥ L 2 ( R n ; R ) ] = O ( L − 2 n s ) . (27)
Proof. Let F ~ N ( V l ) \widetilde{F}_N(V_l) F N ( V l ) be given as in 12 such that
η l = F ( V l ) − F ~ N ( V l ) \eta_l=F(V_l)-\widetilde{F}_N(V_l) η l = F ( V l ) − F N ( V l )
satisfies (13). Set q V = d μ a d ρ V q_V=\frac{d\mu_a}{d\rho_V} q V = d ρ V d μ a and define
d ^ l = 1 L ∑ j = 1 L F ~ N ( V j ) q V ( V j ) K ~ a ( V j , z l ) , l = 1 , … , L . (28) \widehat{d}_l=\frac{1}{L}\sum_{j=1}^L\widetilde{F}_N(V_j)q_V(V_j)\widetilde{K}_a(V_j,z_l),\quad l=1,\ldots,L. \tag*{(28)} d l = L 1 j = 1 ∑ L F N ( V j ) q V ( V j ) K a ( V j , z l ) , l = 1 , … , L . (28)
Denote by
P L F ( z ) = ∑ l = 1 L ⟨ F , K a ( ⋅ , z l ) ⟩ F a 2 K ~ a ( z , z l ) , P_LF(z)=\sum_{l=1}^L\langle F,K_a(\cdot,z_l)\rangle_{\mathcal{F}_a^2}\widetilde{K}_a(z,z_l), P L F ( z ) = l = 1 ∑ L ⟨ F , K a ( ⋅ , z l ) ⟩ F a 2 K a ( z , z l ) ,
we consider
E V , X [ ∥ F − F ^ ∥ F a 2 ( C n ) 2 ] ≤ 2 E V , X [ ∥ F − P L F ∥ F a 2 ( C n ) 2 ] + 2 E V , X [ ∥ P L F − F ^ ∥ F a 2 ( C n ) 2 ] : = 2 E 1 + 2 E 2 . \mathbb{E}_{V,X}\left[\|F-\widehat{F}\|_{\mathcal{F}_a^2(\mathbb{C}^n)}^2\right]\leq2\mathbb{E}_{V,X}\left[\|F-P_LF\|_{\mathcal{F}_a^2(\mathbb{C}^n)}^2\right]+2\mathbb{E}_{V,X}\left[\|P_LF-\widehat{F}\|_{\mathcal{F}_a^2(\mathbb{C}^n)}^2\right]:=2E_1+2E_2. E V , X [ ∥ F − F ∥ F a 2 ( C n ) 2 ] ≤ 2 E V , X [ ∥ F − P L F ∥ F a 2 ( C n ) 2 ] + 2 E V , X [ ∥ P L F − F ∥ F a 2 ( C n ) 2 ] := 2 E 1 + 2 E 2 .
Since f ∈ H a s ( R n ; R ) f\in\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R}) f ∈ H a s ( R n ; R ) , it follows from the normalized expansion of F F F in (25) that
∑ l ( 1 + ∥ z l ∥ 2 ) s ∣ ⟨ F , k a ( ⋅ , z l ) ⟩ F a 2 ∣ 2 ≤ C a , s , Λ ∥ f ∥ H a s ( R n ; R ) 2 . \sum_l(1+\|z_l\|^2)^s|\langle F,k_a(\cdot,z_l)\rangle_{\mathcal{F}_a^2}|^2\leq C_{a,s,\Lambda}\|f\|_{\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R})}^2. l ∑ ( 1 + ∥ z l ∥ 2 ) s ∣ ⟨ F , k a ( ⋅ , z l ) ⟩ F a 2 ∣ 2 ≤ C a , s , Λ ∥ f ∥ H a s ( R n ; R ) 2 .
In the Seip sampling, we have ∥ z l ∥ ≤ R l \|z_l\|\leq R_l ∥ z l ∥ ≤ R l , and we may assume that R l ≤ R l + 1 R_l\leq R_{l+1} R l ≤ R l + 1 , ∀ l \forall l ∀ l . It follows that there exist constants D 1 , D 2 D_1,D_2 D 1 , D 2 for a regular sampling lattice, independent of L L L , R L R_L R L such that
D 1 R L 2 n ≤ L ≤ D 2 R L 2 n . D_1R_L^{2n}\leq L\leq D_2R_L^{2n}. D 1 R L 2 n ≤ L ≤ D 2 R L 2 n .
We observe that
E 1 ≤ 1 A E V , X [ ∑ l > L ∣ ⟨ F , K a ( ⋅ , z l ) ⟩ F a 2 ∣ 2 ] ≤ 1 A ( 1 + R L 2 ) − s ∑ l ( 1 + ∥ z l ∥ 2 ) s ∣ ⟨ F , K a ( ⋅ , z l ) ⟩ F a 2 ∣ 2 ≤ C a , s , Λ A ( 1 + R L 2 ) − s ∥ f ∥ H a s ( R n ; R ) 2 ≤ C a , s , Λ A ( 1 + D 2 − 1 n L 1 n ) − s ∥ f ∥ H a s ( R n ; R ) 2 = : D a , s , Λ 2 L − s n ∥ f ∥ H a s ( R n ; R ) 2 . \begin{aligned}
E_1 &\le\frac{1}{A}\mathbb{E}_{V,X}\left[\sum_{l>L}\left|\langle F,K_a(\cdot,z_l)\rangle_{\mathcal{F}_a^2}\right|^2\right]\le\frac{1}{A}(1+R_L^2)^{-s}\sum_l(1+\|z_l\|^2)^s\left|\langle F,K_a(\cdot,z_l)\rangle_{\mathcal{F}_a^2}\right|^2\\
&\le\frac{C_{a,s,\Lambda}}{A}(1+R_L^2)^{-s}\|f\|_{\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R})}^2\le\frac{C_{a,s,\Lambda}}{A}(1+D_2^{-\frac{1}{n}}L^{\frac{1}{n}})^{-s}\|f\|_{\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R})}^2\\
&=:D_{a,s,\Lambda}^2L^{-\frac{s}{n}}\|f\|_{\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R})}^2.
\end{aligned} E 1 ≤ A 1 E V , X [ l > L ∑ ⟨ F , K a ( ⋅ , z l ) ⟩ F a 2 2 ] ≤ A 1 ( 1 + R L 2 ) − s l ∑ ( 1 + ∥ z l ∥ 2 ) s ⟨ F , K a ( ⋅ , z l ) ⟩ F a 2 2 ≤ A C a , s , Λ ( 1 + R L 2 ) − s ∥ f ∥ H a s ( R n ; R ) 2 ≤ A C a , s , Λ ( 1 + D 2 − n 1 L n 1 ) − s ∥ f ∥ H a s ( R n ; R ) 2 =: D a , s , Λ 2 L − n s ∥ f ∥ H a s ( R n ; R ) 2 .
For E 2 E_2 E 2 , we compute that
E 2 ≤ E V , X [ ∥ ∑ l = 1 L ( F ( z l ) − d ^ l ) K ~ a ( ⋅ , z l ) ∥ F a 2 ( C n ) 2 ] E_2\le\mathbb{E}_{V,X}\left[\left\|\sum_{l=1}^{L}(F(z_l)-\hat{d}_l)\widetilde{K}_a(\cdot,z_l)\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}^2\right] E 2 ≤ E V , X l = 1 ∑ L ( F ( z l ) − d ^ l ) K a ( ⋅ , z l ) F a 2 ( C n ) 2
≤ E V , X [ ∥ ∑ l = 1 L ( F ( z l ) − 1 L ∑ j = 1 L F ( V j ) q V ( V j ) K a ( V j , z l ) ‾ ) K ~ a ( ⋅ , z l ) ∥ F a 2 ( C n ) 2 ] + E V , X [ ∥ ∑ l = 1 L ( 1 L ∑ j = 1 L F ( V j ) q V ( V j ) K a ( V j , z l ) ‾ − d ^ l ) K ~ a ( ⋅ , z l ) ∥ F a 2 ( C n ) 2 ] = E V [ E X ∣ V [ ∥ ∑ l = 1 L ( F ( z l ) − 1 L ∑ j = 1 L F ( V j ) q V ( V j ) K a ( V j , z l ) ‾ ) K ~ a ( ⋅ , z l ) ∥ F a 2 ( C n ) 2 ] ] + E V , X [ E X ∣ V [ ∥ ∑ l = 1 L ( 1 L ∑ j = 1 L η j q V ( V j ) K a ( V j , z l ) ‾ ) K ~ a ( ⋅ , z l ) ∥ F a 2 ( C n ) 2 ] ] . \begin{aligned}
&\le\mathbb{E}_{V,X}\left[\left\|\sum_{l=1}^{L}\left(F(z_l)-\frac{1}{L}\sum_{j=1}^{L}F(V_j)q_V(V_j)\overline{K_a(V_j,z_l)}\right)\widetilde{K}_a(\cdot,z_l)\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}^2\right]\\
&\quad+\mathbb{E}_{V,X}\left[\left\|\sum_{l=1}^{L}\left(\frac{1}{L}\sum_{j=1}^{L}F(V_j)q_V(V_j)\overline{K_a(V_j,z_l)}-\hat{d}_l\right)\widetilde{K}_a(\cdot,z_l)\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}^2\right]\\
&=\mathbb{E}_V\left[\mathbb{E}_{X|V}\left[\left\|\sum_{l=1}^{L}\left(F(z_l)-\frac{1}{L}\sum_{j=1}^{L}F(V_j)q_V(V_j)\overline{K_a(V_j,z_l)}\right)\widetilde{K}_a(\cdot,z_l)\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}^2\right]\right]\\
&\quad+\mathbb{E}_{V,X}\left[\mathbb{E}_{X|V}\left[\left\|\sum_{l=1}^{L}\left(\frac{1}{L}\sum_{j=1}^{L}\eta_jq_V(V_j)\overline{K_a(V_j,z_l)}\right)\widetilde{K}_a(\cdot,z_l)\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}^2\right]\right].
\end{aligned} ≤ E V , X l = 1 ∑ L ( F ( z l ) − L 1 j = 1 ∑ L F ( V j ) q V ( V j ) K a ( V j , z l ) ) K a ( ⋅ , z l ) F a 2 ( C n ) 2 + E V , X l = 1 ∑ L ( L 1 j = 1 ∑ L F ( V j ) q V ( V j ) K a ( V j , z l ) − d ^ l ) K a ( ⋅ , z l ) F a 2 ( C n ) 2 = E V E X ∣ V l = 1 ∑ L ( F ( z l ) − L 1 j = 1 ∑ L F ( V j ) q V ( V j ) K a ( V j , z l ) ) K a ( ⋅ , z l ) F a 2 ( C n ) 2 + E V , X E X ∣ V l = 1 ∑ L ( L 1 j = 1 ∑ L η j q V ( V j ) K a ( V j , z l ) ) K a ( ⋅ , z l ) F a 2 ( C n ) 2 .
By the same argument of Monte-Carlo sampling with respect to { V l } l = 1 L ∼ i . i . d . ρ V \{V_l\}_{l=1}^{L}\overset{\mathrm{i.i.d.}}{\sim}\rho_V { V l } l = 1 L ∼ i.i.d. ρ V as in Theorem 5.1, we conclude from (13) that
E 2 ≤ ( D a , s , Λ ′ ) 2 L − 1 ∥ f ∥ L 2 ( R n ; R ) 2 E_2\le(D'_{a,s,\Lambda})^2L^{-1}\|f\|_{L^2(\mathbb{R}^n;\mathbb{R})}^2 E 2 ≤ ( D a , s , Λ ′ ) 2 L − 1 ∥ f ∥ L 2 ( R n ; R ) 2
where D a , s , Λ ′ D'_{a,s,\Lambda} D a , s , Λ ′ depends on the upper bound of the Fock norms of both K a K_a K a and K ~ a \widetilde{K}_a K a .
Note that by definition of f ^ \hat{f} f ^ , we have
B a − 1 ( F − F ^ ) = f − f ^ . \mathcal{B}_a^{-1}(F-\hat{F})=f-\hat{f}. B a − 1 ( F − F ^ ) = f − f ^ .
By the unitary property of the inverse Segal-Bargmann transform, we get (25) by adding estimates of E 1 E_1 E 1 and E 2 E_2 E 2 together, after absorbing fixed numerical factors into the constants. □
L ∞ L^\infty L ∞ -Convergence of Solving Partial Differential Equations by Neural Operators.
Consider L u = f \mathcal{L}u=f L u = f on R n \mathbb{R}^n R n where L \mathcal{L} L is a smooth differential operator. Our goal is to learn L − 1 : f ↦ u \mathcal{L}^{-1}:f\mapsto u L − 1 : f ↦ u . In this section, we show that we can apply complex neural operator G ~ θ ~ \widetilde{\mathcal{G}}_{\tilde{\theta}} G θ ~ to approximate L − 1 \mathcal{L}^{-1} L − 1 , and give the error bounds. The first version requires both Assumption A and Assumption B, while the second version requires merely Assumption B. In addition, we give an algorithm for the information recovery matrix W W W .
The next lemma verifies the diagram (3).
Lemma 5.2. Let L \mathcal{L} L be a differential operator with smooth coefficients. For u , f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) u,f \in L^2(\mathbb{R}^n;\mathbb{R}) \cap C^\infty(\mathbb{R}^n;\mathbb{R}) u , f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) . Suppose that there exists a continuous linear functional G ~ \widetilde{G} G such that B a u = G ~ B a f B_a u=\widetilde{G}B_a f B a u = G B a f , then the following relation holds.
L − 1 f = B a − 1 G ~ B a f \mathcal{L}^{-1}f=B_a^{-1}\widetilde{G}B_a f L − 1 f = B a − 1 G B a f for every u u u such that L u = f \mathcal{L}u=f L u = f .
Proof. The proof is immediate by the linearity of B a B_a B a and B a − 1 B_a^{-1} B a − 1 .
Let’s discuss the simple case where Lemma 5.2 holds. We also need the universal approximation theorem for neural operators on Hilbert space:
Proposition 5.2. [11 ] Let G ~ : F a 2 ( C n ) → F a 2 ( C n ) \widetilde{G}: \mathcal{F}_a^2(\mathbb{C}^n) \to\mathcal{F}_a^2(\mathbb{C}^n) G : F a 2 ( C n ) → F a 2 ( C n ) be a solution operator of the following equation
U = G ~ F U=\widetilde{G}F U = G F on the Hilbert space F a 2 ( C n ) \mathcal{F}_a^2(\mathbb{C}^n) F a 2 ( C n ) , which satisfies
∥ G ~ ( F 1 ) − G ~ ( F 2 ) ∥ F a 2 ( C n ) ≤ C G ~ , K ∞ ∥ F 1 − F 2 ∥ F a 2 ( C n ) , ∀ F 1 , F 2 ∈ K ∞ ⊂ F a 2 ( C n ) (29) \|\widetilde{G}(F_1)-\widetilde{G}(F_2)\|_{\mathcal{F}_a^2(\mathbb{C}^n)} \le C_{\widetilde{G},\mathcal{K}_\infty}\|F_1-F_2\|_{\mathcal{F}_a^2(\mathbb{C}^n)},\ \forall F_1,F_2\in\mathcal{K}_\infty\subset\mathcal{F}_a^2(\mathbb{C}^n) \tag*{(29)} ∥ G ( F 1 ) − G ( F 2 ) ∥ F a 2 ( C n ) ≤ C G , K ∞ ∥ F 1 − F 2 ∥ F a 2 ( C n ) , ∀ F 1 , F 2 ∈ K ∞ ⊂ F a 2 ( C n ) (29) for some compact admissible set K 1 \mathcal{K}_1 K 1 . Then for any ϵ N O > 0 \epsilon_{NO}>0 ϵ N O > 0 , there exists a neural operator G ~ θ ~ \widetilde{G}_{\widetilde{\theta}} G θ parametrized by θ ~ \widetilde{\theta} θ such that
sup F ∈ K 1 ∥ G ~ ( F ) ( z ) − G ~ θ ~ ( F ) ( z ) ∥ F a 2 ( C n ) < ϵ N O . \sup_{F\in\mathcal{K}_1}\|\widetilde{G}(F)(z)-\widetilde{G}_{\widetilde{\theta}}(F)(z)\|_{\mathcal{F}_a^2(\mathbb{C}^n)}<\epsilon_{NO}. F ∈ K 1 sup ∥ G ( F ) ( z ) − G θ ( F ) ( z ) ∥ F a 2 ( C n ) < ϵ N O .
We are now ready to give our first result in neural operator estimate of the operator L − 1 \mathcal{L}^{-1} L − 1 .
Theorem 5.4. Let s ≥ s 0 > n 2 + 2 s\ge s_0>\frac{n}{2}+2 s ≥ s 0 > 2 n + 2 and u , f ∈ H a s ( R n ; R ) ∩ C ∞ ( R n ; R ) u,f\in\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R})\cap C^\infty(\mathbb{R}^n;\mathbb{R}) u , f ∈ H a s ( R n ; R ) ∩ C ∞ ( R n ; R ) satisfying L u = f \mathcal{L}u=f L u = f , where L \mathcal{L} L is a smooth differential operator whose inverse L − 1 \mathcal{L}^{-1} L − 1 is also a smooth operator. Suppose that we only have finitely many observations { f ( x i ) } i = 1 N \{f(x_i)\}_{i=1}^N { f ( x i ) } i = 1 N of the unknown ground truth f f f . If Assumption A holds for f f f , and Assumption B holds for some i.i.d. complex sampling { Z l } l = 1 L ⊂ K ⊂ C n \{Z_l\}_{l=1}^L\subset K\subset\mathbb{C}^n { Z l } l = 1 L ⊂ K ⊂ C n , then for any ϵ > 0 \epsilon>0 ϵ > 0 and a fixed compact admissible set K ⊂ L 2 ( R n ; R ) \mathcal{K}\subset L^2(\mathbb{R}^n;\mathbb{R}) K ⊂ L 2 ( R n ; R ) that contains Span ( V M ) \operatorname{Span}(V_M) Span ( V M ) , there exists a neural operator G ~ N O \widetilde{G}_{NO} G N O parametrized by θ ~ \widetilde{\theta} θ such that
E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( f ) ( x ) − B a − 1 ∘ G ~ N O ∘ B a ( f ^ ) ( x ) ∣ ] ≤ C 0 n ( 2 a ) n 4 ( 1 + B M n ) − n 12 M ( ϵ N O + C G ~ , K ( M − 1 ) ∥ f ∥ L 2 ( R n ; R ) 2 L + M ∥ f ∥ L ∞ ( R n ; R ) 2 L ) . (30) \mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in\mathcal{K},\{Z_l\}\subset K}}\left|\mathcal{L}^{-1}(f)(x)-B_a^{-1}\circ\widetilde{G}_{NO}\circ B_a(\widehat{f})(x)\right|\right]
\le C_0^n(2a)^{\frac n4}\left(1+\frac{B_M}{n}\right)^{-\frac n{12}}\sqrt{M}\left(\epsilon_{NO}+C_{\widetilde{G},\mathcal{K}}\sqrt{\frac{(M-1)\|f\|_{L^2(\mathbb{R}^n;\mathbb{R})}^2}{L}+\frac{M\|f\|_{L^\infty(\mathbb{R}^n;\mathbb{R})}^2}{L}}\right). \tag*{(30)} E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( f ) ( x ) − B a − 1 ∘ G N O ∘ B a ( f ) ( x ) ] ≤ C 0 n ( 2 a ) 4 n ( 1 + n B M ) − 12 n M ϵ N O + C G , K L ( M − 1 ) ∥ f ∥ L 2 ( R n ; R ) 2 + L M ∥ f ∥ L ∞ ( R n ; R ) 2 . (30) where f ^ = ∑ β ∈ Λ M c ^ β a h β a \widehat{f}=\sum_{\beta\in\Lambda_M}\widehat{c}^{\,a}_{\beta}h^a_{\beta} f = ∑ β ∈ Λ M c β a h β a is determined by { Z l } \{Z_l\} { Z l } and { f ( x i ) } \{f(x_i)\} { f ( x i )} as in Theorem 5.2. In particular, there exists an appropriate choice of a a a such that the following dimension-free error bound holds:
E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( f ) − B a − 1 ∘ G ~ N O ∘ B a ( f ^ ) ∣ ] = O ( L − 1 2 ) . (31) \mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in\mathcal{K},\{Z_l\}\subset K}}\left|\mathcal{L}^{-1}(f)-B_a^{-1}\circ\widetilde{G}_{NO}\circ B_a(\widehat{f})\right|\right]=O\left(L^{-\frac12}\right). \tag*{(31)} E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( f ) − B a − 1 ∘ G N O ∘ B a ( f ) ] = O ( L − 2 1 ) . (31)
Proof. Fix some ϵ > 0 \epsilon>0 ϵ > 0 . Since B a B_a B a is a unitary transformation, K ′ : = B a ( K ) \mathcal{K}':=B_a(\mathcal{K}) K ′ := B a ( K ) is a compact admissible set of F a 2 ( C n ) \mathcal{F}_a^2(\mathbb{C}^n) F a 2 ( C n ) . By our hypotheses, B a ( f ^ ) , B a ( f ) ∈ K ′ B_a(\widehat{f}),B_a(f)\in\mathcal{K}' B a ( f ) , B a ( f ) ∈ K ′ . Fix a positive constant ϵ N O < ϵ \epsilon_{NO}<\epsilon ϵ N O < ϵ that will be determined later. By Lemma 5.2 and Proposition 5.2, there exists a neural operator G ~ N O \widetilde{G}_{\mathrm{NO}} G NO such that
E Z , X [ sup B a ( f ) ∈ K ′ , { Z l } ⊂ K ∥ G ~ ( B a ( f ) ) − G ~ N O ∘ B a ( f ^ ) ∥ F a 2 ( C n ) ] ≤ E Z , X [ ∥ G ~ ( B a ( f ) ) − G ~ ∘ B a ( f ^ ) ∥ F a 2 ( C n ) ] + E Z , X [ sup B a ( f ) ∈ K ′ , { Z l } ⊂ K ∥ G ~ ∘ B a ( f ^ ) − G ~ N O ∘ B a ( f ^ ) ∥ F a 2 ( C n ) ] ≤ C G ~ , K ′ E Z , X [ ∥ G ~ ( B a ( f ) ) − G ~ ∘ B a ( f ^ ) ∥ F a 2 ( C n ) ] + ϵ N O ≤ C G ~ , K ′ ( M − 1 ) C 2 L + M ( C ′ ) 2 L + ϵ N O . \begin{aligned}
\mathbb{E}_{Z,X}\left[\sup_{\substack{B_a(f)\in K',\{Z_l\}\subset K}}\left\|\widetilde{G}(B_a(f))-\widetilde{G}_{\mathrm{NO}}\circ B_a(\hat{f})\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}\right]
&\le\mathbb{E}_{Z,X}\left[\left\|\widetilde{G}(B_a(f))-\widetilde{G}\circ B_a(\hat{f})\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}\right] \\
&\quad+\mathbb{E}_{Z,X}\left[\sup_{\substack{B_a(f)\in K',\{Z_l\}\subset K}}\left\|\widetilde{G}\circ B_a(\hat{f})-\widetilde{G}_{\mathrm{NO}}\circ B_a(\hat{f})\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}\right] \\
&\le C_{\widetilde{G},K'}\mathbb{E}_{Z,X}\left[\left\|\widetilde{G}(B_a(f))-\widetilde{G}\circ B_a(\hat{f})\right\|_{\mathcal{F}_a^2(\mathbb{C}^n)}\right]+\epsilon_{\mathrm{NO}} \\
&\le C_{\widetilde{G},K'}\sqrt{\frac{(M-1)C^2}{L}+\frac{M(C')^2}{L}}+\epsilon_{\mathrm{NO}}.
\end{aligned} E Z , X [ B a ( f ) ∈ K ′ , { Z l } ⊂ K sup G ( B a ( f )) − G NO ∘ B a ( f ^ ) F a 2 ( C n ) ] ≤ E Z , X [ G ( B a ( f )) − G ∘ B a ( f ^ ) F a 2 ( C n ) ] + E Z , X [ B a ( f ) ∈ K ′ , { Z l } ⊂ K sup G ∘ B a ( f ^ ) − G NO ∘ B a ( f ^ ) F a 2 ( C n ) ] ≤ C G , K ′ E Z , X [ G ( B a ( f )) − G ∘ B a ( f ^ ) F a 2 ( C n ) ] + ϵ NO ≤ C G , K ′ L ( M − 1 ) C 2 + L M ( C ′ ) 2 + ϵ NO .
Apply Lemma 5.2 and Proposition 5.1, we conclude that
E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( f ) ( x ) − B a − 1 ∘ G ~ N O ∘ B a ( f ^ ) ( x ) ∣ ] = E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ B a − 1 ( G ~ ( B a ( f ) ) − G ~ N O ∘ B a ( f ^ ) ) ( x ) ∣ ] ≤ C 0 n ( 2 a ) n 4 ( 1 + B M n ) − n 12 ⋅ M ( C G ~ , K ( M − 1 ) C 2 L + M ( C ′ ) 2 L + C 1 ( 2 a ) n 4 ϵ N O ) ≤ C 0 n ( 2 a ) n 4 ( 1 + B M n ) − n 12 ⋅ M ( ϵ + C G ~ , K ( M − 1 ) C 2 L + M ( C ′ ) 2 L ) . \begin{aligned}
\mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}}\left|L^{-1}(f)(x)-B_a^{-1}\circ\widetilde{G}_{\mathrm{NO}}\circ B_a(\hat{f})(x)\right|\right]
&=\mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}}\left|B_a^{-1}\left(\widetilde{G}(B_a(f))-\widetilde{G}_{\mathrm{NO}}\circ B_a(\hat{f})\right)(x)\right|\right] \\
&\le C_0^n(2a)^{\frac n4}\left(1+\frac{B_M}{n}\right)^{-\frac n{12}}\cdot\sqrt{M}\left(C_{\widetilde{G},K}\sqrt{\frac{(M-1)C^2}{L}+\frac{M(C')^2}{L}}+C_1(2a)^{\frac n4}\epsilon_{\mathrm{NO}}\right) \\
&\le C_0^n(2a)^{\frac n4}\left(1+\frac{B_M}{n}\right)^{-\frac n{12}}\cdot\sqrt{M}\left(\epsilon+C_{\widetilde{G},K}\sqrt{\frac{(M-1)C^2}{L}+\frac{M(C')^2}{L}}\right).
\end{aligned} E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( f ) ( x ) − B a − 1 ∘ G NO ∘ B a ( f ^ ) ( x ) ] = E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup B a − 1 ( G ( B a ( f )) − G NO ∘ B a ( f ^ ) ) ( x ) ] ≤ C 0 n ( 2 a ) 4 n ( 1 + n B M ) − 12 n ⋅ M ( C G , K L ( M − 1 ) C 2 + L M ( C ′ ) 2 + C 1 ( 2 a ) 4 n ϵ NO ) ≤ C 0 n ( 2 a ) 4 n ( 1 + n B M ) − 12 n ⋅ M ( ϵ + C G , K L ( M − 1 ) C 2 + L M ( C ′ ) 2 ) .
Here the bound C 1 ( 2 a ) n 4 C_1(2a)^{\frac n4} C 1 ( 2 a ) 4 n comes from the integration ∫ C n ∣ A a ( z ˉ , x ) ∣ d μ a ( z ) \int_{\mathbb{C}^n}|A_a(\bar{z},x)|d\mu_a(z) ∫ C n ∣ A a ( z ˉ , x ) ∣ d μ a ( z ) , where C 1 C_1 C 1 is independent of the dimension. We can shrink ϵ N O \epsilon_{\mathrm{NO}} ϵ NO if necessary so that the last inequality above holds. This gives (30). With an appropriate choice of a a a , the a n 4 a^{\frac n4} a 4 n term is small enough to compensate any potential dimensional growth in M M M and C 0 n C_0^n C 0 n . Hence (31) follows. □
Remark 5.5. If we define
G θ : = B a − 1 ∘ G ~ N O ∘ B a , G_\theta:=B_a^{-1}\circ\widetilde{G}_{\mathrm{NO}}\circ B_a, G θ := B a − 1 ∘ G NO ∘ B a ,
we can apply Markov’s inequality for fixed δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) to show that the following error bound holds in probability at least 1 − δ 1-\delta 1 − δ :
sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ u ( x ) − G θ ( f ^ ) ( x ) ∣ = O ( L − 1 2 ) . (32) \sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}}|u(x)-G_\theta(\hat{f})(x)|=O\left(L^{-\frac12}\right). \tag*{(32)} x ∈ R n , f ∈ K , { Z l } ⊂ K sup ∣ u ( x ) − G θ ( f ^ ) ( x ) ∣ = O ( L − 2 1 ) . (32)
It means that with the help of complex grids { z l } l = 1 L \{z_l\}_{l=1}^L { z l } l = 1 L , we can start at a finite set of observations { f ( x i ) } i = 1 N \{f(x_i)\}_{i=1}^N { f ( x i ) } i = 1 N to solve the PDE with O ( L − 1 2 ) O\left(L^{-\frac12}\right) O ( L − 2 1 ) error, provided that Assumption A and Assumption B hold.
If we only know that the observations { f ( x i ) } \{f(x_i)\} { f ( x i )} are evaluated on grids { x i } \{x_i\} { x i } that are sampled from a collection of random variables { X i } \{X_i\} { X i } but nothing else, we cannot reach this conclusion.
If we assume f ∈ H a s ( R n ; R ) f\in\mathcal{H}_a^s(\mathbb{R}^n;\mathbb{R}) f ∈ H a s ( R n ; R ) , we can drop the Assumption A and get the same convergence rate as in (31).
Theorem 5.5. Let u , f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) u,f\in L^2(\mathbb{R}^n;\mathbb{R})\cap C^\infty(\mathbb{R}^n;\mathbb{R}) u , f ∈ L 2 ( R n ; R ) ∩ C ∞ ( R n ; R ) satisfying L u = f Lu=f Lu = f , where L L L is a smooth differential operator whose inverse L − 1 L^{-1} L − 1 is a bounded smooth operator. Suppose that we only have finitely many observations { f ( x i ) } i = 1 N \{f(x_i)\}_{i=1}^N { f ( x i ) } i = 1 N of the unknown ground truth f f f . If only Assumption B holds for P M f P_Mf P M f for some i.i.d. complex sampling { Z l } l = 1 L ⊂ K ⊂ C n \{Z_l\}_{l=1}^{L} \subset K \subset\mathbb{C}^n { Z l } l = 1 L ⊂ K ⊂ C n , then for any ϵ > 0 \epsilon> 0 ϵ > 0 and a fixed compact admissible set K ⊂ L 2 ( R n ; R ) K \subset L^2(\mathbb{R}^n;\mathbb{R}) K ⊂ L 2 ( R n ; R ) that contains Span ( V M ) \operatorname{Span}(V_M) Span ( V M ) , there exists a neural operator G ~ N O \widetilde{G}_{\mathrm{NO}} G NO parametrized by θ ~ \widetilde{\theta} θ such that
E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( f ) ( x ) − B a − 1 ∘ G ~ N O ∘ B a ( f ^ ) ( x ) ∣ ] ≤ C 0 m ( 2 a ) n 4 ( 1 + B M n ) − n 12 M ( ϵ + C G ~ , K ~ ( M − 1 ) ∥ f ∥ L 2 ( R n ; R ) 2 L + C M M ∥ f ∥ L ∞ ( R n ; R ) L ) + ( 1 + 2 a ( 2 M 0 + n ) ) − s − s 0 2 ( 4 π ) − n 4 Γ ( s 0 − n 2 ) Γ ( s 0 ) ∥ L − 1 ∥ ∥ f ∥ H a s (33) \begin{aligned}
\mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}} \left|\mathcal{L}^{-1}(f)(x)-\mathcal{B}_a^{-1}\circ\widetilde{G}_{\mathrm{NO}}\circ\mathcal{B}_a(\widehat{f})(x)\right|\right] \\
&\le C_0^m(2a)^{\frac{n}{4}}\left(1+\frac{B_M}{n}\right)^{-\frac{n}{12}}\sqrt{M}\left(\epsilon+C_{\widetilde{G},\widetilde{K}}\sqrt{\frac{(M-1)\|f\|_{L^2(\mathbb{R}^n;\mathbb{R})}^2}{L}}+\frac{C_M M\|f\|_{L^\infty(\mathbb{R}^n;\mathbb{R})}}{L}\right) \\
&+\left(1+2a(2M_0+n)\right)^{-\frac{s-s_0}{2}}(4\pi)^{-\frac{n}{4}}\sqrt{\frac{\Gamma\left(s_0-\frac{n}{2}\right)}{\Gamma(s_0)}}\|\mathcal{L}^{-1}\|\|f\|_{H_a^s} \tag*{(33)}
\end{aligned} E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( f ) ( x ) − B a − 1 ∘ G NO ∘ B a ( f ) ( x ) ] ≤ C 0 m ( 2 a ) 4 n ( 1 + n B M ) − 12 n M ϵ + C G , K L ( M − 1 ) ∥ f ∥ L 2 ( R n ; R ) 2 + L C M M ∥ f ∥ L ∞ ( R n ; R ) + ( 1 + 2 a ( 2 M 0 + n ) ) − 2 s − s 0 ( 4 π ) − 4 n Γ ( s 0 ) Γ ( s 0 − 2 n ) ∥ L − 1 ∥∥ f ∥ H a s (33)
where f ^ = ∑ β ∈ Λ M c ^ β h β a \widehat{f}=\sum_{\beta\in\Lambda_M}\widehat{c}_{\beta}h_{\beta}^{a} f = ∑ β ∈ Λ M c β h β a determined by { Z l } \{Z_l\} { Z l } and { f ( x i ) } \{f(x_i)\} { f ( x i )} as in Theorem 5.2. In particular, there exists an appropriate choice of a a a such that the following dimension-free error bound holds:
E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( f ) − B a − 1 ∘ G ~ N O ∘ B a ( f ^ ) ∣ ] = O ( L − 1 2 ) . (34) \mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}} \left|\mathcal{L}^{-1}(f)-\mathcal{B}_a^{-1}\circ\widetilde{G}_{\mathrm{NO}}\circ\mathcal{B}_a(\widehat{f})\right|\right]=O\left(L^{-\frac{1}{2}}\right). \tag*{(34)} E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( f ) − B a − 1 ∘ G NO ∘ B a ( f ) ] = O ( L − 2 1 ) . (34)
Proof. Clearly,
E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( f ) ( x ) − B a − 1 ∘ G ~ N O ∘ B a ( f ^ ) ( x ) ∣ ] ≤ E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( f ) ( x ) − L − 1 ( P M f ) ( x ) ∣ ] + E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( P M f ) ( x ) − B a − 1 ∘ G ~ N O ∘ B a ( f ^ ) ( x ) ∣ ] = E 1 + E 2 . \begin{aligned}
\mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}} \left|\mathcal{L}^{-1}(f)(x)-\mathcal{B}_a^{-1}\circ\widetilde{G}_{\mathrm{NO}}\circ\mathcal{B}_a(\widehat{f})(x)\right|\right] \\
&\le\mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}} \left|\mathcal{L}^{-1}(f)(x)-\mathcal{L}^{-1}(P_Mf)(x)\right|\right] \\
&+\mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}} \left|\mathcal{L}^{-1}(P_Mf)(x)-\mathcal{B}_a^{-1}\circ\widetilde{G}_{\mathrm{NO}}\circ\mathcal{B}_a(\widehat{f})(x)\right|\right]
=E_1+E_2.
\end{aligned} E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( f ) ( x ) − B a − 1 ∘ G NO ∘ B a ( f ) ( x ) ] ≤ E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( f ) ( x ) − L − 1 ( P M f ) ( x ) ] + E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( P M f ) ( x ) − B a − 1 ∘ G NO ∘ B a ( f ) ( x ) ] = E 1 + E 2 .
For E 1 E_1 E 1 , we apply (15) in Lemma 5.1,
E 1 = sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( f ) ( x ) − L − 1 ( P M f ) ∣ ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − s − s 0 2 ( 4 π ) − n 4 Γ ( s 0 − n 2 ) Γ ( s 0 ) ∥ L − 1 ( f ) − L − 1 ( P M f ) ∥ H a s ( R n ; R ) ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − s − s 0 2 ( 4 π ) − n 4 Γ ( s 0 − n 2 ) Γ ( s 0 ) ∥ L − 1 ∥ ∥ f ∥ H a s ( R n ; R ) \begin{aligned}
E_1=\sup_{\substack{x\in\mathbb{R}^n,f\in K,\{Z_l\}\subset K}}\left|\mathcal{L}^{-1}(f)(x)-\mathcal{L}^{-1}(P_Mf)\right| \\
&\le\left(1+2a(2M_0+n)\right)^{-\frac{s-s_0}{2}}(4\pi)^{-\frac{n}{4}}\sqrt{\frac{\Gamma\left(s_0-\frac{n}{2}\right)}{\Gamma(s_0)}}\left\|\mathcal{L}^{-1}(f)-\mathcal{L}^{-1}(P_Mf)\right\|_{H_a^s(\mathbb{R}^n;\mathbb{R})} \\
&\le\left(1+2a(2M_0+n)\right)^{-\frac{s-s_0}{2}}(4\pi)^{-\frac{n}{4}}\sqrt{\frac{\Gamma\left(s_0-\frac{n}{2}\right)}{\Gamma(s_0)}}\|\mathcal{L}^{-1}\|\|f\|_{H_a^s(\mathbb{R}^n;\mathbb{R})}
\end{aligned} E 1 = x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( f ) ( x ) − L − 1 ( P M f ) ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − 2 s − s 0 ( 4 π ) − 4 n Γ ( s 0 ) Γ ( s 0 − 2 n ) L − 1 ( f ) − L − 1 ( P M f ) H a s ( R n ; R ) ≤ ( 1 + 2 a ( 2 M 0 + n ) ) − 2 s − s 0 ( 4 π ) − 4 n Γ ( s 0 ) Γ ( s 0 − 2 n ) ∥ L − 1 ∥∥ f ∥ H a s ( R n ; R )
By the hypotheses of the statement with respect to P M f P_Mf P M f , we apply (30) and conclude that E 2 E_2 E 2 satisfies
E Z , X [ sup x ∈ R n , f ∈ K , { Z l } ⊂ K ∣ L − 1 ( P M f ) ( x ) − B a − 1 ∘ G ~ N O ∘ B a ( f ^ ) ( x ) ∣ ] ≤ C 0 n ( 2 a ) n 4 ( 1 + B M n ) − n 12 ⋅ M ( ϵ + C G ~ , K ( M − 1 ) ∥ P M f ∥ L 2 ( R n ; R ) 2 L + M ∥ P M f ∥ L ∞ ( R n ; R ) 2 L ) ≤ C 0 n ( 2 a ) n 4 ( 1 + B M n ) − n 12 ⋅ M ( ϵ + C G ~ , K ( M − 1 ) ∥ f ∥ L 2 ( R n ; R ) 2 L + C M M ∥ f ∥ L ∞ ( R n ; R ) 2 L ) . \begin{aligned}
\mathbb{E}_{Z,X}\left[\sup_{\substack{x\in\mathbb{R}^{n},f\in\mathcal{K},\{Z_l\}\subset\mathcal{K}}}\left|\mathcal{L}^{-1}(P_Mf)(x)-\mathcal{B}_a^{-1}\circ\widetilde{\mathcal{G}}_{NO}\circ\mathcal{B}_a(\hat{f})(x)\right|\right]
\leq C_0^n(2a)^{\frac{n}{4}}\left(1+\frac{B_M}{n}\right)^{-\frac{n}{12}}\cdot\sqrt{M}\left(\epsilon+C_{\widetilde{G},\mathcal{K}}\sqrt{\frac{(M-1)\|P_Mf\|_{L^2(\mathbb{R}^n;\mathbb{R})}^2}{L}+\frac{M\|P_Mf\|_{L^\infty(\mathbb{R}^n;\mathbb{R})}^2}{L}}\right) \\
&\leq C_0^n(2a)^{\frac{n}{4}}\left(1+\frac{B_M}{n}\right)^{-\frac{n}{12}}\cdot\sqrt{M}\left(\epsilon+C_{\widetilde{G},\mathcal{K}}\sqrt{\frac{(M-1)\|f\|_{L^2(\mathbb{R}^n;\mathbb{R})}^2}{L}+\frac{C_M M\|f\|_{L^\infty(\mathbb{R}^n;\mathbb{R})}^2}{L}}\right).
\end{aligned} E Z , X [ x ∈ R n , f ∈ K , { Z l } ⊂ K sup L − 1 ( P M f ) ( x ) − B a − 1 ∘ G N O ∘ B a ( f ^ ) ( x ) ] ≤ C 0 n ( 2 a ) 4 n ( 1 + n B M ) − 12 n ⋅ M ϵ + C G , K L ( M − 1 ) ∥ P M f ∥ L 2 ( R n ; R ) 2 + L M ∥ P M f ∥ L ∞ ( R n ; R ) 2 ≤ C 0 n ( 2 a ) 4 n ( 1 + n B M ) − 12 n ⋅ M ϵ + C G , K L ( M − 1 ) ∥ f ∥ L 2 ( R n ; R ) 2 + L C M M ∥ f ∥ L ∞ ( R n ; R ) 2 .
where C M C_M C M depends only on f f f and M M M . Combining the upper bound of E 1 E_1 E 1 and E 2 E_2 E 2 gives (33) . By the same argument of Theorem 5.4 , there exists an appropriate a ≪ 1 a\ll1 a ≪ 1 such that (34) holds, which is dimension-free. □ \square □
Numerical Experiments
In this section, we study the Segal–Bargmann neural operator (SBNO) through 2D Darcy flow, the 1D Burgers’ equation, and two nonlinear elliptic equations in spatial dimension 5. Based on our best understanding, the state-of-the-art neural operator approach for unknown L \mathcal{L} L is spatial dimension 3. Our goal is to learn the corresponding solution operators from finitely many observations of input and output functions. For the first three PDEs, we compare SBNO with FNO on a regular grid, and with GNOT and BelNet on scattered points. Within each comparison, the methods use the same training and testing functions and the same observations. The Darcy and Burgers’ experiments use 2000 training functions and 200 independent testing functions. For the scattered-point experiments, we also evaluate the same testing functions on a new point set to study how the learned operators depend on the observation locations.
For the Burgers’ equation, the models receive only observations of the initial field and predict solution values throughout 0 ≤ t ≤ 1 0\leq t\leq1 0 ≤ t ≤ 1 in one evaluation. Different from standard benchmark comparisons, no intermediate solution snapshots are given as inputs for our experiments. This requires learning the evolution over the whole interval without recursive time stepping. The new-point test is more demanding than testing only at the training locations, since the operator must generalize to both unseen initial conditions and unseen space–time sampling points. All tests remain within the training time interval.
Two-dimensional Darcy flow
We first consider the Darcy equation
− ∇ ⋅ ( κ ( x ) ∇ u ( x ) ) = g in Ω , u = 0 on ∂ Ω , (35) -\nabla\cdot\left(\kappa(x)\nabla u(x)\right)=g\quad\text{in }\Omega,\qquad u=0\quad\text{on }\partial\Omega, \tag*{(35)} − ∇ ⋅ ( κ ( x ) ∇ u ( x ) ) = g in Ω , u = 0 on ∂ Ω , (35)
where κ \kappa κ is the positive permeability field. The input is the collection { κ ( x i ) } i = 1 N \{\kappa(x_i)\}_{i=1}^{N} { κ ( x i ) } i = 1 N , and the output consists of the solution values { u ( x i ) } i = 1 N \{u(x_i)\}_{i=1}^{N} { u ( x i ) } i = 1 N . Thus, we seek an approximation of the map κ ↦ u \kappa\mapsto u κ ↦ u with the forcing and boundary conditions fixed.
For the comparison with FNO [32 ] , we take Ω = ( − 1 , 1 ) 2 \Omega=(-1,1)^2 Ω = ( − 1 , 1 ) 2 and g = 1 g=1 g = 1 . The permeability fields are drawn from four families: discontinuous mosaics, rotated mosaics, curved channels, and continuous multiscale fields with sharp transitions. Their random parameters are chosen independently for each function, with 0.1 ≤ κ ≤ 10 0.1\leq\kappa\leq10 0.1 ≤ κ ≤ 10 . We solve (35) on a 1024 × 1024 1024\times1024 1024 × 1024 uniform grid by a conservative five-point finite-difference scheme, using harmonic averages of permeability across adjacent nodes. Retaining every sixteenth node along each coordinate gives the 64 × 64 64\times64 64 × 64 grid used by both SBNO and FNO.
For the comparison with GNOT [22 ] and BelNet [57 ] , we use Ω = ( − 1 , 1 ) 2 \Omega=(-1,1)^2 Ω = ( − 1 , 1 ) 2 and g = − 1 g=-1 g = − 1 , which gives the convention ∇ ⋅ ( κ ∇ u ) = 1 \nabla\cdot(\kappa\nabla u)=1 ∇ ⋅ ( κ ∇ u ) = 1 in the scattered-point dataset. Here, κ \kappa κ is a random expansion in 256 normalized Hermite functions, with a positive ground-state component and bounded perturbations chosen to preserve positivity. We draw 1024 2 1024^2 102 4 2 interior nodes from a density proportional to e − 2 ∣ x ∣ 2 e^{-2|x|^2} e − 2∣ x ∣ 2 on D D D , add boundary nodes, and form a Delaunay triangulation. The reference solutions are computed by conforming piecewise-linear finite elements. We then select 32 2 = 1024 32^2=1024 3 2 2 = 1024 interior nodes without replacement and take the permeability and solution values directly from those nodes. In this experiment, 32 × 32 32\times32 32 × 32 denotes the number of observations; the points are scattered rather than arranged as a tensor grid.
We plot the relative L 2 L^2 L 2 test error of both SBNO and FNO on original 64 × 64 64\times64 64 × 64 uniform grids and expanded 128 × 128 128\times128 128 × 128 grids, the E X [ ∣ u ^ ( X ) − u ( X ) ∣ ∣ u ( X ) ∣ ] \operatorname{E}_X\left[\frac{|\hat{u}(X)-u(X)|}{|u(X)|}\right] E X [ ∣ u ( X ) ∣ ∣ u ^ ( X ) − u ( X ) ∣ ] error, and the relative L ∞ L^\infty L ∞ error in Figure 1 (PDF p. 33) .
Figure 1 (PDF p. 33). 2D Darcy Flow FNO vs SBNO
Figure 1 (PDF p. 33) demonstrates that SBNO outperforms FNO for all three types of errors, for both original grids and for generalization; in particular, the typical relative solution error for SBNO is one order better than FNO.
We point out that FNO has a lower expected relative error on the on the 128 × 128 128\times128 128 × 128 grid than the 64 × 64 64\times64 64 × 64 grid. This is a reasonable result since the smaller proportion of grid points in the outermost grid layers are chosen for 128 × 128 128\times128 128 × 128 grids, where the errors are larger for FNO. In contrast, the numerical results illustrates the stability of our SBNO approach everywhere.
We also give the training speed comparison table between a fast algorithm of SBNO and FNO in Table 1 for a 32 × 32 32\times32 32 × 32 original grid test. Note that the fast SBNO algorithm is not used in training the results in Figure 1 (PDF p. 33) above. Table 1 shows that the fast SBNO can reach the same accuracy as FNO, meanwhile 10 times faster than the FNO.
Table 1. Computational benchmark comparison between the Segal–Bargmann Neural Operator (SBNO) and the Fourier Neural Operator (FNO).
On a total of 1024 scattered grids, we also plot the relative L 2 L^2 L 2 test error of SBNO, GNOT and BelNet, the E X [ ∣ u ^ ( X ) − u ( X ) ∣ ∣ u ( X ) ∣ ] \operatorname{E}_X\left[\frac{|\hat{u}(X)-u(X)|}{|u(X)|}\right] E X [ ∣ u ( X ) ∣ ∣ u ^ ( X ) − u ( X ) ∣ ] error, and the relative L ∞ L^\infty L ∞ error in Figure 2 . Figure 2 demonstrates that SBNO is comparable with GNOT and outperforms BelNet for all three types of errors for unseen test functions on original grids; in particular, the testing error of SBNO is one order better than the BelNet. Although the GNOT performs a little better in this scenario, this experiments also demonstrates the wide applicability of our SBNO approach, especially we give experimental evidence of the outstanding behavior of SBNO for other PDEs.
Figure 2. 2D Darcy Flow SBNO vs GNOT vs BelNet
The one-dimensional Burgers' equation
We next consider the viscous Burgers’ equation on [ 0 , 1 ] × ( − 1 , 1 ) [0,1]\times(-1,1) [ 0 , 1 ] × ( − 1 , 1 ) :
∂ t u + u ∂ x u = 0.1 ∂ x x u , u ( 0 , x ) = u 0 ( x ) , (36) \partial_t u+u\partial_xu=0.1\partial_{xx}u,\qquad u(0,x)=u_0(x), \tag*{(36)} ∂ t u + u ∂ x u = 0.1 ∂ xx u , u ( 0 , x ) = u 0 ( x ) , (36)
with periodic boundary conditions u ( t , − 1 ) = u ( t , 1 ) u(t,-1)=u(t,1) u ( t , − 1 ) = u ( t , 1 ) and ∂ x u ( t , − 1 ) = ∂ x u ( t , 1 ) \partial_xu(t,-1)=\partial_xu(t,1) ∂ x u ( t , − 1 ) = ∂ x u ( t , 1 ) . Our goal is to learn the map u 0 ↦ u ( t , x ) u_0 \mapsto u(t,x) u 0 ↦ u ( t , x ) over the whole time interval. At a sampled point ( t i , x i ) (t_i,x_i) ( t i , x i ) , the input value is u 0 ( x i ) u_0(x_i) u 0 ( x i ) and the output value is u ( t i , x i ) u(t_i,x_i) u ( t i , x i ) . The initial condition is therefore extended constantly in time when constructing the input observations.
We generate u 0 u_0 u 0 as a zero-mean Gaussian Fourier series with frequencies 1 , … , 16 1,\ldots,16 1 , … , 16 , using independent Gaussian sine and cosine coefficients whose variances decrease with frequency. A fixed scaling gives an expected spatial mean square of 0.5 2 0.5^2 0. 5 2 . The reference solution is obtained from the Cole–Hopf formula, using 2048 quadrature points for the initial heat potential and 128 positive Fourier frequencies for its evolution. This representation allows direct evaluation at the scattered space–time points. Writing s t = 2 t − 1 s_t=2t-1 s t = 2 t − 1 , we draw 1024 2 1024^2 102 4 2 points ( s t , x ) (s_t,x) ( s t , x ) from the same truncated Gaussian law as in the scattered Darcy experiment and select 32 2 = 1024 32^2=1024 3 2 2 = 1024 of them for training and testing. SBNO is compared with GNOT [22 ] and BelNet [57 ] using these observations.
For each test initial condition, predictions are compared with the reference values at the same 1024 space–time points. We first use the training point set and then a new, independently sampled set { ( t i ′ , x i ′ ) } i = 1 1024 \{(t_i',x_i')\}_{i=1}^{1024} {( t i ′ , x i ′ ) } i = 1 1024 , with inputs u 0 ( x i ′ ) u_0(x_i') u 0 ( x i ′ ) and targets u ( t i ′ , x i ′ ) u(t_i',x_i') u ( t i ′ , x i ′ ) . The same 200 unseen initial conditions are used for both tests. Thus, 32 2 32^2 3 2 2 counts scattered points over the whole interval; it does not denote 32 time slices. The second test additionally measures generalization to new observation locations.
On a total of 1024 scattered grids in [ 0 , 1 ] t × [ − 1 , 1 ] x [0,1]_t \times[-1,1]_x [ 0 , 1 ] t × [ − 1 , 1 ] x , we plot the relative L 2 L^2 L 2 test error of SBNO, GNOT and BelNet, the E X [ ∣ u ^ ( X ) − u ( X ) ∣ ∣ u ( X ) ∣ ] \mathbb{E}_X\left[\frac{|\hat{u}(X)-u(X)|}{|u(X)|}\right] E X [ ∣ u ( X ) ∣ ∣ u ^ ( X ) − u ( X ) ∣ ] error, and the relative L ∞ L^\infty L ∞ error in Figure 3 . Figure 3 demonstrates that SBNO outperforms GNOT and BelNet for all three types of errors for unseen test functions on original grids; in particular, the testing error of SBNO is at least half of the GNOT for all three types of errors, and is one order better than the BelNet.
Figure 3. 2D Darcy Flow SBNO vs GNOT vs BelNet
Five-dimensional nonlinear elliptic equations
Finally, we consider the two equations
− Δ u + u + u 3 = f , and − Δ u + e u + u 3 = f , x ∈ R 5 , (37) -\Delta u + u + u^3 = f, \qquad\text{and} \qquad-\Delta u + e^u + u^3 = f, \qquad x \in\mathbb{R}^5, \tag*{(37)} − Δ u + u + u 3 = f , and − Δ u + e u + u 3 = f , x ∈ R 5 , (37)
with u ( x ) → 0 u(x) \to0 u ( x ) → 0 as ∣ x ∣ → ∞ |x| \to\infty ∣ x ∣ → ∞ . We observe the functions on [ − 1 , 1 ] 5 [-1,1]^5 [ − 1 , 1 ] 5 and learn the map f ↦ u f \mapsto u f ↦ u from the sampled pairs { f ( x i ) , u ( x i ) } i = 1 N \{f(x_i),u(x_i)\}_{i=1}^{N} { f ( x i ) , u ( x i ) } i = 1 N . The boundary values of the restricted solutions are inherited from their whole-space definitions and are generally nonzero on the boundary of the observation cube. These are stationary problems, so no initial condition is prescribed.
To generate the data, we first choose a solution
u ( k ) ( x ) = ∑ β ∈ S k c β k h β 5 ( x ) , S k ⊂ { β ∈ N 0 5 : ∣ β ∣ ≤ 6 } , ∣ S k ∣ = 50 , (38) u^{(k)}(x) = \sum_{\beta\in S_k} c_{\beta}^{k} h_{\beta}^{5}(x), \qquad S_k \subset\{\beta\in\mathbb{N}_0^5 : |\beta| \le6\}, \qquad|S_k| = 50, \tag*{(38)} u ( k ) ( x ) = β ∈ S k ∑ c β k h β 5 ( x ) , S k ⊂ { β ∈ N 0 5 : ∣ β ∣ ≤ 6 } , ∣ S k ∣ = 50 , (38)
where h β 5 h_{\beta}^{5} h β 5 denotes a normalized Hermite function with parameter a = 5 a = 5 a = 5 . For each function, we select the 50 indices uniformly without replacement from the 2002 available modes and draw the coefficients independently from N ( 0 , 1 / 50 ) \mathcal{N}(0,1/50) N ( 0 , 1/50 ) . We then evaluate f ( k ) = − Δ u ( k ) + u ( k ) + ( u ( k ) ) 3 f^{(k)} = -\Delta u^{(k)} + u^{(k)} + (u^{(k)})^3 f ( k ) = − Δ u ( k ) + u ( k ) + ( u ( k ) ) 3 or f ( k ) = − Δ u ( k ) + e u ( k ) + ( u ( k ) ) 3 f^{(k)} = -\Delta u^{(k)} + e^{u^{(k)}} + (u^{(k)})^3 f ( k ) = − Δ u ( k ) + e u ( k ) + ( u ( k ) ) 3 analytically at uniformly sampled points in [ − 1 , 1 ] 5 [-1,1]^5 [ − 1 , 1 ] 5 . Both nonlinear terms are evaluated without truncation. This gives 2000 training pairs and 1000 independent testing pairs for each equation. Only the sampled function values and coordinates are supplied to the models; the Hermite coefficients are used to generate the data. For the exponential equation, f ( x ) → 1 f(x) \to1 f ( x ) → 1 at infinity, while f − 1 f-1 f − 1 and u u u are square-integrable on R 5 \mathbb{R}^{5} R 5 . The observations and reported errors are restricted to the cube. Note that the second PDE is a challenging one: once we take the exponential nonlinearity of u u u , the input f f f encodes infinitely Gauss-Hermitian modes.
We plot the relative L 2 L^{2} L 2 test error, the E X [ ∣ u ^ ( X ) − u ( X ) ∣ ∣ u ( X ) ∣ ] \mathbb{E}_{X}\left[\frac{|\hat{u}(X)-u(X)|}{|u(X)|}\right] E X [ ∣ u ( X ) ∣ ∣ u ^ ( X ) − u ( X ) ∣ ] error, and the relative L ∞ L^{\infty} L ∞ error for SBNO in Figure 4 . We use a total of 1 × 10 5 1 \times10^{5} 1 × 1 0 5 real samplings for both − Δ u + u + u 3 = f -\Delta u+u+u^{3}=f − Δ u + u + u 3 = f and − Δ u + e u + u 3 = f -\Delta u+e^{u}+u^{3}=f − Δ u + e u + u 3 = f . Figure 4 demonstrates that SBNO can achieve a very good testing error for relatively simple nonlinearity, and can still learn the solution operator even when highly nonlinear term e u e^{u} e u . We expect better numerical results for better designs of neural networks. These results suggest that our proposed method has great potential to be applied to more challenging PDEs and to higher spatial dimensions.
Figure 4. 5D Two Nonlinear PDEs SBNO
Data Availability Statement and Conflict of Interest. No external datasets were used in this study. All data used in the numerical experiments were generated computationally using the procedures described in this paper. The authors declare no conflicts of interest related to this work.
Appendix: Validity of Assumption B
In this section, we show that with probability 1 − δ 1-\delta 1 − δ , where δ \delta δ can be arbitrarily small, the Assumption B with η l ≡ 0 \eta_l \equiv0 η l ≡ 0 for all l l l holds with some assumptions that will be stated below. It indicates that the Assumption B is a general and reasonable assumption. The proof follows the construction of [14 Theorem 3.2] .
Model and statement
Fix observation locations x 1 , … , x N x_1,\ldots,x_N x 1 , … , x N and a training set
{ ( f ( k ) , y k ) } k = 1 K , f ( k ) = ( f ( k ) ( x 1 ) , … , f ( k ) ( x N ) ) T ∈ R N , ∥ f ( k ) ∥ 2 = 1 , ∣ y k ∣ ≤ C y . (39) \{(f^{(k)},y_k)\}_{k=1}^{K}, \qquad f^{(k)}=(f^{(k)}(x_1),\ldots,f^{(k)}(x_N))^{\mathsf{T}}\in\mathbb{R}^{N}, \qquad\|f^{(k)}\|_2=1,\qquad|y_k|\le C_y. \tag*{(39)} {( f ( k ) , y k ) } k = 1 K , f ( k ) = ( f ( k ) ( x 1 ) , … , f ( k ) ( x N ) ) T ∈ R N , ∥ f ( k ) ∥ 2 = 1 , ∣ y k ∣ ≤ C y . (39)
The training set does not change with L L L . Let A ( z , x ) A(z,x) A ( z , x ) be fixed; in the intended application, A = A a , M A=A_{a,M} A = A a , M with fixed a a a and M M M . Draw Z 1 , … , Z L Z_1,\ldots,Z_L Z 1 , … , Z L independently from a fixed distribution ρ \rho ρ , and hold them fixed during training. Set
v l ( W , Z l ) = ( w l 1 A ( Z l , x 1 ) , … , w l N A ( Z l , x N ) ) T . (40) v_l(W,Z_l)=\left(w_{l1}A(Z_l,x_1),\ldots,w_{lN}A(Z_l,x_N)\right)^{\mathsf{T}}. \tag*{(40)} v l ( W , Z l ) = ( w l 1 A ( Z l , x 1 ) , … , w l N A ( Z l , x N ) ) T . (40)
To use the real ReLU with a complex kernel, take the real part of the preactivation:
u k ( W ) = 1 L ∑ l = 1 L b l σ ( Re ( v l ( W , Z l ) T f ( k ) ) ) , σ ( s ) = max { s , 0 } , L ( W ) = 1 2 ∑ k = 1 K ( u k ( W ) − y k ) 2 . (41) \begin{aligned}
u_k(W)&=\frac{1}{\sqrt{L}}\sum_{l=1}^{L}b_l\sigma\left(\operatorname{Re}(v_l(W,Z_l)^{\mathsf{T}}\mathbf{f}^{(k)})\right),\\
\sigma(s)&=\max\{s,0\},\qquad\mathcal{L}(W)=\frac{1}{2}\sum_{k=1}^{K}(u_k(W)-y_k)^2. \tag*{(41)}
\end{aligned} u k ( W ) σ ( s ) = L 1 l = 1 ∑ L b l σ ( Re ( v l ( W , Z l ) T f ( k ) ) ) , = max { s , 0 } , L ( W ) = 2 1 k = 1 ∑ K ( u k ( W ) − y k ) 2 . (41)
Write w l = p l + i q l w_l=p_l+iq_l w l = p l + i q l and θ l = ( p l , q l ) ∈ R 2 N \theta_l=(p_l,q_l)\in\mathbb{R}^{2N} θ l = ( p l , q l ) ∈ R 2 N . Initialize θ l ( 0 ) ∼ N ( 0 , I 2 N ) \theta_l(0)\sim\mathcal{N}(0,I_{2N}) θ l ( 0 ) ∼ N ( 0 , I 2 N ) and b l ∼ Unif { − 1 , 1 } b_l\sim\operatorname{Unif}\{-1,1\} b l ∼ Unif { − 1 , 1 } , independently across neurons and independently of the Z l Z_l Z l . Keep b l b_l b l fixed and train only θ l \theta_l θ l by Euclidean gradient flow for L \mathcal{L} L , using the ReLU differential-inclusion convention. Thus the trained parameters are the original w l i w_{li} w l i , through their real and imaginary parts.
For the proof, define
ξ k ( z ) = ( ( Re A ( z , x i ) f i ( k ) ) i = 1 N ( − Im A ( z , x i ) f i ( k ) ) i = 1 N ) , B ( z ) = max 1 ≤ i ≤ N ∣ A ( z , x i ) ∣ . (42) \xi_k(z)=
\begin{pmatrix}
\left(\operatorname{Re}A(z,x_i)f_i^{(k)}\right)_{i=1}^{N}\\
\left(-\operatorname{Im}A(z,x_i)f_i^{(k)}\right)_{i=1}^{N}
\end{pmatrix},
\qquad
B(z)=\max_{1\le i\le N}|A(z,x_i)|. \tag*{(42)} ξ k ( z ) = ( Re A ( z , x i ) f i ( k ) ) i = 1 N ( − Im A ( z , x i ) f i ( k ) ) i = 1 N , B ( z ) = 1 ≤ i ≤ N max ∣ A ( z , x i ) ∣. (42)
Then Re ( v l T f ( k ) ) = θ l T ξ k ( Z l ) \operatorname{Re}(v_l^{\mathsf{T}}\mathbf{f}^{(k)})=\theta_l^{\mathsf{T}}\xi_k(Z_l) Re ( v l T f ( k ) ) = θ l T ξ k ( Z l ) and ∥ ξ k ( z ) ∥ 2 ≤ B ( z ) \|\xi_k(z)\|_2\le B(z) ∥ ξ k ( z ) ∥ 2 ≤ B ( z ) . The vectors ξ k ( Z l ) \xi_k(Z_l) ξ k ( Z l ) are auxiliary expressions for differentiating (3); the training inputs remain f ( k ) \mathbf{f}^{(k)} f ( k ) .
Define
H k j ∞ = E Z , G [ ξ k ( Z ) ⊤ ξ j ( Z ) 1 { G ⊤ ξ k ( Z ) > 0 , G ⊤ ξ j ( Z ) > 0 } ] , G ∼ N ( 0 , I 2 N ) , G ⊥ Z . (43) H_{kj}^{\infty}=\mathbb{E}_{Z,G}\left[\xi_k(Z)^{\top}\xi_j(Z)\mathbf{1}_{\{G^{\top}\xi_k(Z)>0,G^{\top}\xi_j(Z)>0\}}\right],\qquad G\sim\mathcal{N}(0,I_{2N}),\qquad G\perp Z. \tag*{(43)} H k j ∞ = E Z , G [ ξ k ( Z ) ⊤ ξ j ( Z ) 1 { G ⊤ ξ k ( Z ) > 0 , G ⊤ ξ j ( Z ) > 0 } ] , G ∼ N ( 0 , I 2 N ) , G ⊥ Z . (43)
Theorem 7.1. Suppose
μ 4 : = E ρ B ( Z ) 4 < ∞ , λ 0 : = λ min ( H ∞ ) > 0. (44) \mu_4:=\mathbb{E}_{\rho_B(Z)}^4<\infty,\qquad\lambda_0:=\lambda_{\min}(H^\infty)>0. \tag*{(44)} μ 4 := E ρ B ( Z ) 4 < ∞ , λ 0 := λ m i n ( H ∞ ) > 0. (44) Write μ j = E ρ B ( Z ) j \mu_j=\mathbb{E}_{\rho_B(Z)}^j μ j = E ρ B ( Z ) j for j = 2 , 3 j=2,3 j = 2 , 3 and S = C y 2 + μ 2 / 2 S=C_y^2+\mu_2/2 S = C y 2 + μ 2 /2 . There is a universal constant C > 0 C>0 C > 0 such that, for any δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) , if
L ≥ C ( K 2 μ 4 λ 0 2 δ + K 6 μ 2 2 S λ 0 4 δ 3 ) , (45) L\ge C\left(\frac{K^2\mu_4}{\lambda_0^2\delta}+\frac{K^6\mu_2^2S}{\lambda_0^4\delta^3}\right), \tag*{(45)} L ≥ C ( λ 0 2 δ K 2 μ 4 + λ 0 4 δ 3 K 6 μ 2 2 S ) , (45) then, with probability at least 1 − δ 1-\delta 1 − δ over complex sampling and initialization,
∥ u ( t ) − y ∥ 2 2 ≤ e − λ 0 t ∥ u ( 0 ) − y ∥ 2 2 for all t ≥ 0. (46) \|u(t)-y\|_2^2\le e^{-\lambda_0t}\|u(0)-y\|_2^2\qquad\text{for all }t\ge0. \tag*{(46)} ∥ u ( t ) − y ∥ 2 2 ≤ e − λ 0 t ∥ u ( 0 ) − y ∥ 2 2 for all t ≥ 0. (46) In particular, L ( W ( t ) ) → 0 \mathcal{L}(W(t))\to0 L ( W ( t )) → 0 as t → ∞ t\to\infty t → ∞ for each such fixed finite width. The event can be chosen to work for every global admissible gradient-flow trajectory.
Proof. We follow the four steps in §3 of [14 ] : concentration at initialization, stability near initialization, control of weight movement, and continuation for all time. Write ξ k l = ξ k ( Z l ) \xi_{kl}=\xi_k(Z_l) ξ k l = ξ k ( Z l ) and let s k l ( t ) s_{kl}(t) s k l ( t ) be the ReLU derivative at θ l ( t ) ⊤ ξ k l \theta_l(t)^{\top}\xi_{kl} θ l ( t ) ⊤ ξ k l , with a value in [ 0 , 1 ] [0,1] [ 0 , 1 ] permitted at zero. Almost everywhere,
θ ˙ l = b l L ∑ k = 1 K ( y k − u k ) s k l ξ k l , \dot{\theta}_l=\frac{b_l}{\sqrt{L}}\sum_{k=1}^{K}(y_k-u_k)s_{kl}\xi_{kl}, θ ˙ l = L b l k = 1 ∑ K ( y k − u k ) s k l ξ k l ,
u ˙ = H ( t ) ( y − u ) , H k j ( t ) = 1 L ∑ l = 1 L ξ k l ⊤ ξ j l s k l ( t ) s j l ( t ) . (47) \dot{u}=H(t)(y-u),\qquad H_{kj}(t)=\frac{1}{L}\sum_{l=1}^{L}\xi_{kl}^{\top}\xi_{jl}s_{kl}(t)s_{jl}(t). \tag*{(47)} u ˙ = H ( t ) ( y − u ) , H k j ( t ) = L 1 l = 1 ∑ L ξ k l ⊤ ξ j l s k l ( t ) s j l ( t ) . (47)
This is the Gram matrix for training W W W itself, with
ξ k ( z ) ⊤ ξ j ( z ) = ∑ i = 1 N ∣ A ( z , x i ) ∣ 2 f i ( k ) f i ( j ) . \xi_k(z)^{\top}\xi_j(z)=\sum_{i=1}^{N}|A(z,x_i)|^2f_i^{(k)}f_i^{(j)}. ξ k ( z ) ⊤ ξ j ( z ) = i = 1 ∑ N ∣ A ( z , x i ) ∣ 2 f i ( k ) f i ( j ) .
The summands of H ( 0 ) H(0) H ( 0 ) are independent across neurons, have mean H ∞ H^\infty H ∞ , and their entries have second moments at most μ 4 \mu_4 μ 4 . Thus
E ∥ H ( 0 ) − H ∞ ∥ F 2 ≤ K 2 μ 4 L . (48) \mathbb{E}\|H(0)-H^\infty\|_F^2\le\frac{K^2\mu_4}{L}. \tag*{(48)} E ∥ H ( 0 ) − H ∞ ∥ F 2 ≤ L K 2 μ 4 . (48)
Markov’s inequality and the first term of (7) give, with probability at least 1 − δ / 3 1-\delta/3 1 − δ /3 ,
∥ H ( 0 ) − H ∞ ∥ 2 ≤ λ 0 / 4 , λ min ( H ( 0 ) ) ≥ 3 λ 0 / 4. (49) \|H(0)-H^\infty\|_2\le\lambda_0/4,\qquad\lambda_{\min}(H(0))\ge3\lambda_0/4. \tag*{(49)} ∥ H ( 0 ) − H ∞ ∥ 2 ≤ λ 0 /4 , λ m i n ( H ( 0 )) ≥ 3 λ 0 /4. (49)
Put B l = B ( Z l ) B_l=B(Z_l) B l = B ( Z l ) and c 0 = 2 / π c_0=\sqrt{2/\pi} c 0 = 2/ π . For R > 0 R>0 R > 0 , define
E k l ( R ) = { ξ k l ≠ 0 , ∣ θ l ( 0 ) ⊤ ξ k l ∣ ≤ R B l ∥ ξ k l ∥ 2 } . E_{kl}(R)=\left\{\xi_{kl}\ne0,\ \left|\theta_l(0)^{\top}\xi_{kl}\right|\le RB_l\|\xi_{kl}\|_2\right\}. E k l ( R ) = { ξ k l = 0 , θ l ( 0 ) ⊤ ξ k l ≤ R B l ∥ ξ k l ∥ 2 } .
Conditionally on Z l Z_l Z l , the normalized nonzero Gaussian projection has variance one, so
P ( E k l ( R ) ∣ Z l ) ≤ c 0 R B l . (50) \mathbb{P}(E_{kl}(R)\mid Z_l)\le c_0RB_l. \tag*{(50)} P ( E k l ( R ) ∣ Z l ) ≤ c 0 R B l . (50)
If ∥ θ l − θ l ( 0 ) ∥ 2 ≤ R B l \lVert\theta_l-\theta_l(0)\rVert_2\le R B_l ∥ θ l − θ l ( 0 ) ∥ 2 ≤ R B l , the activation on input k k k can change only on E k l ( R ) E_{kl}(R) E k l ( R ) . When ξ k l = 0 \xi_{kl}=0 ξ k l = 0 , its Gram contribution is zero. Uniformly over this neighborhood and over derivative choices at zero,
∥ H − H ( 0 ) ∥ 2 ≤ T R : = 1 L ∑ l = 1 L ∑ k , j = 1 K B l 2 ( 1 E k l ( R ) + 1 E j l ( R ) ) . \lVert H-H(0)\rVert_2\le T_R:=\frac{1}{L}\sum_{l=1}^{L}\sum_{k,j=1}^{K}B_l^2\left(\mathbf{1}_{E_{kl}(R)}+\mathbf{1}_{E_{jl}(R)}\right). ∥ H − H ( 0 ) ∥ 2 ≤ T R := L 1 l = 1 ∑ L k , j = 1 ∑ K B l 2 ( 1 E k l ( R ) + 1 E j l ( R ) ) .
By (12), E T R ≤ 2 c 0 K 2 R μ 3 \mathbb{E}T_R\le2c_0K^2R\mu_3 E T R ≤ 2 c 0 K 2 R μ 3 . Choose
R = δ λ 0 24 c 0 K 2 μ 3 . (51) R=\frac{\delta\lambda_0}{24c_0K^2\mu_3}. \tag*{(51)} R = 24 c 0 K 2 μ 3 δ λ 0 . (51)
Then P ( T R > λ 0 / 4 ) ≤ δ / 3 \mathbb{P}(T_R>\lambda_0/4)\le\delta/3 P ( T R > λ 0 /4 ) ≤ δ /3 . Together with (11), this gives λ min ( H ) ≥ λ 0 / 2 \lambda_{\min}(H)\ge\lambda_0/2 λ m i n ( H ) ≥ λ 0 /2 throughout the neighborhood. The radii R B l R B_l R B l explicitly retain the dependence on Z l Z_l Z l .
The independent output signs give E u k ( 0 ) = 0 \mathbb{E}u_k(0)=0 E u k ( 0 ) = 0 , and the Gaussian ReLU identity gives
E u k ( 0 ) 2 = 1 2 E ∥ ξ k ( Z ) ∥ 2 2 ≤ μ 2 / 2. \mathbb{E}u_k(0)^2=\frac{1}{2}\mathbb{E}\lVert\xi_k(Z)\rVert_2^2\le\mu_2/2. E u k ( 0 ) 2 = 2 1 E ∥ ξ k ( Z ) ∥ 2 2 ≤ μ 2 /2.
Therefore E ∥ u ( 0 ) − y ∥ 2 2 ≤ K S \mathbb{E}\lVert u(0)-y\rVert_2^2\le KS E ∥ u ( 0 ) − y ∥ 2 2 ≤ K S , and with probability at least 1 − δ / 3 1-\delta/3 1 − δ /3 ,
E 0 2 : = ∥ u ( 0 ) − y ∥ 2 2 ≤ 3 K S δ . (52) E_0^2:=\lVert u(0)-y\rVert_2^2\le\frac{3KS}{\delta}. \tag*{(52)} E 0 2 := ∥ u ( 0 ) − y ∥ 2 2 ≤ δ 3 K S . (52)
As long as the trajectory remains in the neighborhood, (9) implies
d d t ∥ u − y ∥ 2 2 = − 2 ( u − y ) T H ( t ) ( u − y ) ≤ − λ 0 ∥ u − y ∥ 2 2 . \frac{d}{dt}\lVert u-y\rVert_2^2=-2(u-y)^{\mathsf T}H(t)(u-y)\le-\lambda_0\lVert u-y\rVert_2^2. d t d ∥ u − y ∥ 2 2 = − 2 ( u − y ) T H ( t ) ( u − y ) ≤ − λ 0 ∥ u − y ∥ 2 2 .
Hence ∥ u ( t ) − y ∥ 2 ≤ e − λ 0 t / 2 E 0 \lVert u(t)-y\rVert_2\le e^{-\lambda_0t/2}E_0 ∥ u ( t ) − y ∥ 2 ≤ e − λ 0 t /2 E 0 and
∥ θ l ( t ) − θ l ( 0 ) ∥ 2 ≤ B l K L ∫ 0 t ∥ u ( s ) − y ∥ 2 d s ≤ B l R ′ , R ′ = 2 K E 0 λ 0 L . (53) \lVert\theta_l(t)-\theta_l(0)\rVert_2\le\frac{B_l\sqrt{K}}{\sqrt{L}}\int_0^t\lVert u(s)-y\rVert_2\,ds
\le B_lR',\qquad R'=\frac{2\sqrt{K}E_0}{\lambda_0\sqrt{L}}. \tag*{(53)} ∥ θ l ( t ) − θ l ( 0 ) ∥ 2 ≤ L B l K ∫ 0 t ∥ u ( s ) − y ∥ 2 d s ≤ B l R ′ , R ′ = λ 0 L 2 K E 0 . (53)
Equations (7), (13), and (14), with a sufficiently large universal C C C , give R ′ ≤ R / 2 R'\le R/2 R ′ ≤ R /2 . The three events above hold jointly with probability at least 1 − δ 1-\delta 1 − δ . On their intersection, there can be no first exit from the neighborhood: up to any proposed exit time, (15) bounds each displacement by half its allowed radius. Neurons with B l = 0 B_l=0 B l = 0 have zero velocity. The bounded displacements also allow continuation for all time. The Gram lower bound therefore holds globally, and integration of the loss differential inequality proves (8). □
Application to the truncated Segal–Bargmann kernel
Corollary 7.1. Let
A ( z , x ) = A a , M ( z , x ) = ∑ β ∈ Λ M e β a ( z ) h β a ( x ) , d ρ a ( z ) = ∑ β ∈ Λ M ∣ e β a ( z ) ∣ 2 M d μ a ( z ) . (54) A(z,x)=A_{a,M}(z,x)=\sum_{\beta\in\Lambda_M}e_\beta^a(z)h_\beta^a(x),\qquad d\rho_a(z)=\frac{\sum_{\beta\in\Lambda_M}|e_\beta^a(z)|^2}{M}\,d\mu_a(z). \tag*{(54)} A ( z , x ) = A a , M ( z , x ) = β ∈ Λ M ∑ e β a ( z ) h β a ( x ) , d ρ a ( z ) = M ∑ β ∈ Λ M ∣ e β a ( z ) ∣ 2 d μ a ( z ) . (54) with fixed a > 0 a>0 a > 0 and finite Λ M \Lambda_M Λ M containing 0 0 0 . If no two distinct training vectors f ( k ) f^{(k)} f ( k ) are parallel, both conditions in (6) hold, so (7) and (8) apply.
Proof. For each sensor, A a , M ( z , x i ) A_{a,M}(z,x_i) A a , M ( z , x i ) is a polynomial in z z z , while ρ a \rho_a ρ a has a polynomial-times-Gaussian density. Thus B ( Z ) B(Z) B ( Z ) has finite moments of every finite order. The constant coefficient of A a , M ( z , x i ) A_{a,M}(z, x_i) A a , M ( z , x i ) is h a 0 ( x i ) > 0 h_a^0(x_i)>0 h a 0 ( x i ) > 0 . Each polynomial is nonzero, and its zero set has Lebesgue measure zero. Absolute continuity of ρ a \rho_a ρ a and finiteness of N N N imply that all A a , M ( Z , x i ) A_{a,M}(Z, x_i) A a , M ( Z , x i ) are nonzero almost surely.
For each such z z z , the real-linear map f ↦ ξ ( z , f ) \mathbf{f}\mapsto\xi(z,\mathbf{f}) f ↦ ξ ( z , f ) is injective and preserves nonparallelness. Apply [14 Theorem 3.1] to the normalized vectors ξ k ( z ) / ∥ ξ k ( z ) ∥ 2 \xi_k(z)/\|\xi_k(z)\|_2 ξ k ( z ) /∥ ξ k ( z ) ∥ 2 . Multiplication by their positive norms shows that the conditional Gaussian Gram matrix on the unnormalized ξ k ( z ) \xi_k(z) ξ k ( z ) is positive definite. Its expectation is H ∞ H^\infty H ∞ and is finite by the moment bound. Every nonzero vector has strictly positive conditional quadratic form almost surely, and hence strictly positive expected quadratic form. Since K K K is finite, λ min ( H ∞ ) > 0 \lambda_{\min}(H^\infty)>0 λ m i n ( H ∞ ) > 0 .
□
References
Department of Technology, Operations and Statistics, New York University, New York, NY, USA Email address: jx479@nyu.edu
Department of Mathematics and Statistics, Boston University, Boston, MA, USA Email address: jcandas@bu.edu
Department of Mathematics and Statistics, Boston University, Boston, MA, USA Email address: mkon@bu.edu
Isara Labs, San Francisco, CA, USA Email address: jerry@isara.io