betaFeedback

WorksComputer Science

Constructive Universal Approximation by Infinite-Dimensional Neural Networks via Gaussian-Weighted Ridgelet Transform

Abstract

Neural networks whose inputs are functions, as in operator learning, are usually shown to be universal by existence arguments, either by reduction to finite-dimensional networks or by Stone–Weierstrass-type density; these prove that approximants exist but say little about how the parameters depend on the target. We take a constructive route. We extend the ridgelet transform, an integral transform related to the Fourier, Radon, and wavelet transforms, to continuous maps on a separable infinite-dimensional Hilbert space, replacing the Lebesgue measure that is unavailable in infinite dimensions by Gaussian weights with a homogeneous mixture on the direction space. The resulting transform has a Plancherel identity and reconstruction formulas incorporating the frame operator, the correction induced by the weights, and it assigns to each target an explicit coefficient measure, one for every admissible analysis filter. From these representations we obtain uniform approximation on compact sets for all continuous non-polynomial activations of polynomial growth, including ReLU, and, by Monte Carlo sampling of the coefficient, finite-width networks with a dimension-free O(N−1/2)O(N^{-1/2}) error in the width NN for globally Lipschitz activations and coefficients with finite variation and second moments. Explicit examples include Gaussian targets and neural-operator layers. A companion Lean~4 formalization provides machine-checked proofs, with its correspondence to the manuscript documented at https://shosonoda.github.io/lean-operator-ridgelet/ .

Introduction

We develop ridgelet analysis for neural networks with infinite-dimensional inputs. The construction connects explicit integral representations to finite-width approximation and uniform approximation on compact sets.

From function inputs to network coefficients

Operator learning seeks to approximate nonlinear maps whose inputs are functions and whose outputs may also be functions. Examples include solution maps of differential equations and layers that combine an integral operator with a pointwise nonlinearity [54, 64, 67]. On a separable real Hilbert space H\mathcal{H}, a basic model with values in a Hilbert space Y\mathcal{Y} is

fN(x)=∑j=1Nvjσ(⟨aj,x⟩−bj),aj∈H, bj∈R, vj∈Y.f_N(x) = \sum_{j=1}^{N} v_j \sigma(\langle a_j, x\rangle- b_j), \qquad a_j \in\mathcal{H},\ b_j \in\mathbb{R},\ v_j \in\mathcal{Y}.

Each direction aja_j defines a linear measurement of the input. Beyond asking whether these networks can approximate a target, we ask how to choose their directions, biases, and output coefficients from that target.

An integral representation addresses this question by replacing the finite sum with a superposition of neurons,

f(x)=Sσ[Γ](x):=∫H×Rσ(⟨a,x⟩−b) Γ(da,db).f(x) = S_\sigma[\Gamma](x) := \int_{\mathcal{H}\times\mathbb{R}} \sigma(\langle a, x\rangle- b)\,\Gamma(\mathrm{d}a, \mathrm{d}b).

Here Γ\Gamma is a finite scalar or vector coefficient measure. Its total variation measures the size of the representation. Once Γ\Gamma is known, its normalized variation supplies a probability law for sampling neurons; its polar density supplies the output weights. Barron’s argument then relates the finite-width approximation error to the variation and moments of this measure [4, 5].

The analytic tool used here is a ridgelet transform: an integral operator that sends a target to coefficients indexed by directions and biases. It is related to the Fourier, Radon, and wavelet transforms [16, 74, 87], but we construct and explain it directly on H\mathcal{H}. No knowledge of ridgelet analysis is needed to read the definitions or the proofs below.

Universality of such networks is usually proved by existence arguments: by reduction to a finite-dimensional approximation theorem, or by Stone–Weierstrass-type density arguments on the infinite-dimensional space [9, 23, 46]; random features give rates on Hilbert inputs [36]. These results establish existence or sampling bounds without an analysis operator. In this paper, we supply that operator: an explicit coefficient formula, a norm identity, and reconstruction, for the full class of continuous non-polynomial activations of polynomial growth, from which universality and finite-width rates follow.

The main technical obstacle is that an infinite-dimensional Hilbert space has no locally finite, translation-invariant analogue of Lebesgue measure, so the finite-dimensional ridgelet integrals cannot be carried over directly. We integrate against a Gaussian input measure and a homogeneous mixture of Gaussian measures on the direction space; the homogeneity parameter of the mixture replaces the dimension-dependent factor of the finite-dimensional admissibility condition. Section 3 constructs these measures and the transform.

Main results

Our main contributions connect transform identities to constructive neural-network approximation; the universality statement itself is of a type already known, and the novelty lies in how the approximants are obtained. The precise assumptions are stated with each theorem.

Transform and reconstruction. We construct a Gaussian-weighted ridgelet transform with a Plancherel identity: the norm of the coefficients equals a constant times the target norm in a Hilbert space defined by weighted Fourier analysis (Theorem 3.14). The transform is injective and has closed range. Reconstruction uses a frame operator, an invertible correction induced by the Gaussian weights, which can be undone after synthesis or before analysis (Theorem 4.8).

Explicit network coefficients. For targets given by a Fourier integral, we derive an explicit coefficient formula and an integral-network representation (Theorem 4.5). Smoothness of the Fourier density along lines through the origin, together with weighted bounds on its derivatives, makes the coefficient measure finite and gives the parameter moments needed for approximation (Theorem 6.6). The representation allows continuous activations of polynomial growth; non-polynomiality permits a nonzero reconstruction constant. In particular, ReLU is admissible when paired with a suitable analysis filter (Corollary 5.5). Different admissible analysis filters give different coefficients for the same target, so the representation is a family indexed by the filter.

Approximation rates and universality. Monte Carlo approximation of the integral representation gives finite-width networks. For globally Lipschitz activations, their expected uniform error on a compact set is of order N−1/2N^{-1/2}, where NN is the width, with no explicit input or output dimension factor (Theorem 6.5). The constant depends on the coefficient variation, parameter moments, and compact set. Density of the represented targets then proves cc-universality, meaning uniform approximation of every continuous target on each compact subset of the input space (Theorem 6.8). These results cover Hilbert-valued outputs, including function-valued operators.

Examples and numerical evidence. We give Gaussian targets, Gaussian-parameter networks, and neural-operator layers built from periodic convolution and a Dirichlet solution operator, with explicit representations (Section 7). Three experiments illustrate the predicted approximation rates across input resolutions, check an explicit transform formula, and separate finite-width error from numerical discretization error (Section 8).

Scope. The coefficient formula is explicit once a spectral density is given; for a general continuous target, the universality proof first chooses a spectral approximation, and the constants VV and M2M_{2} depend on that choice. We do not obtain a uniform width–accuracy relation for all continuous targets, a coefficient-computation algorithm, or a statistical learning guarantee. Universality is proved in the compact-open topology, not globally on weighted non-compact spaces, and the N−1/2N^{-1/2} rate is a Barron-type statement for targets whose coefficient has finite variation and moments, consistent with the lower bounds for Lipschitz operator classes (Section 1.3). Every finite-width network factors through finitely many measurements ⟨aj,x⟩\langle a_j,x\rangle; representing the directions themselves with finitely many numbers is a further approximation problem. Tempered reconstruction is an identity in a dual space, and general nonlinear operator activations are outside the reconstruction theorem.

Related work

Ridgelet analysis originates with Murata [74], Candès [16], and Candès and Donoho [17] as a transform adapted to ridge functions. Its distributional extension admits unbounded activations, including ReLU [87]; weighted Sobolev activation spaces are developed in Sonoda et al. [90]; and the Fourier-slice method yields transforms for group convolutional and non-Euclidean networks [88, 89], where the Hilbert-space construction of Sonoda et al. [89 Section 4] integrates over a finite-dimensional subspace. Complexity bounds derived from integral representations are surveyed in Kůrková [59].

Integration over an infinite-dimensional space with respect to a Gaussian measure is classical: Fourier–Wiener transforms [14, 60], Gaussian Radon transforms [7, 39, 42, 43, 73], and a ridgelet transform for Wiener functionals [68], whose atoms are generated by a functional on the Cameron–Martin space and whose basic identity relates two Gaussian spaces of different variance rather than reconstructing the target. In this paper, the atoms are neurons, the direction measure is homogeneous, and the Plancherel identity holds on a single space; Appendix F makes these comparisons explicit, including the relation of our analysis transform to the Gaussian Radon transform.

Universal approximation with infinite-dimensional inputs goes back to the approximation of functionals [19, 20, 81, 91]. Compact-open universality is known on Fréchet spaces with operator activations [9, 32], in Lp(μ)L^p(\mu) [33], for scalar ridge neurons on locally convex spaces with every continuous non-polynomial activation [46], and globally on weighted spaces [23, 82]; neural operators [53, 54, 64, 65, 67] motivate the examples in § 7. These are existence theorems; here the approximants are obtained from an explicit coefficient, one for each admissible analysis filter.

Finite-width bounds by sampling a parameter distribution follow Barron [5], Bach [4], and E, Ma, and Wu [28], with Hilbert-input versions in Gonon et al. [36, 37], Banach-valued versions in Korolev [52], Neufeld and Schmocker [78], and encoder–decoder versions in Yang and Shi [99]. In this paper, the sampling measure is the homogeneous measure να\nu_\alpha, the coefficient is characterized by an analysis operator with a Plancherel identity and a closed-range projection, and the uniform rate covers Hilbert-valued outputs. Lower bounds [2, 62, 63] exclude such rates for all Lipschitz operators; ours concerns a Barron-type class. Continuous frames [30, 41], Gaussian measures [12, 24, 80], and Wiener chaos [48] supply the general background.

Notation and organization

Unless stated otherwise, H\mathcal{H} is an infinite-dimensional separable real Hilbert space with inner product ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle, B(H)\mathcal{B}(\mathcal{H}) its Borel sigma-algebra, and μQ\mu_Q the centred Gaussian measure with covariance QQ, that is, the Borel probability measure with ∫Hei⟨x,ξ⟩ μQ(dx)=e−⟨Qξ,ξ⟩/2\int_{\mathcal{H}} e^{i\langle x,\xi\rangle}\,\mu_Q(dx)=e^{-\langle Q\xi,\xi\rangle/2}. Throughout, PP and QQ are injective, positive, self-adjoint, trace-class operators on H\mathcal{H}, and Dωa:=ωaD_{\omega}a:=\omega a. Functions on H\mathcal{H} are complex valued unless stated otherwise; Y\mathcal{Y} denotes a separable complex Hilbert space of outputs. On R\mathbb{R} we use

h♯(ω)=∫Rh(t)e−itω dt,h(t)=12π∫Rh♯(ω)eitω dω.(1)h^\sharp(\omega)=\int_{\mathbb{R}}h(t)e^{-it\omega}\,dt,\qquad h(t)=\frac{1}{2\pi}\int_{\mathbb{R}}h^\sharp(\omega)e^{it\omega}\,d\omega. \tag*{(1)}

We use ♯\sharp for one-dimensional Fourier transforms, including tempered distributions and the partial transform in the bias:

γ♯(a,ω):=∫Rγ(a,b)e−ibω db.\gamma^\sharp(a,\omega):=\int_{\mathbb{R}}\gamma(a,b)e^{-ib\omega}\,db.

A hat denotes the spatial Fourier transform, with the same negative exponential convention. For a finite measure MM on H\mathcal{H},

M^(ξ):=∫He−i⟨x,ξ⟩ M(dx).\widehat{M}(\xi):=\int_{\mathcal{H}}e^{-i\langle x,\xi\rangle}\,M(dx).

For square-integrable coefficients the partial transform is understood in L2L^2; see Lemma 3.6. The letter ρ\rho always denotes the analysis filter, σ\sigma a scalar synthesis activation; ω\omega is a frequency or dilation variable, ξ\xi a spectral variable, aa a direction, and bb a bias. Distributional pairings ⟨u,φ⟩\langle u,\varphi\rangle are bilinear.

We write μC\mu_C for a centred Gaussian law with covariance CC, on the ambient Hilbert space specified in context; in one dimension μv\mu_v denotes the Gaussian law of variance vv. When operator notation is needed, FRF_{\mathbb{R}} denotes the one-dimensional Fourier transform and FbF_b its partial version in the bias; FQF_Q denotes the Gaussian-weighted spatial transform. The inverse one-dimensional transform is denoted by hˇ\check{h}. Spectral densities are denoted by gg, parameter densities by γ\gamma, and finite coefficient measures by Γ\Gamma.

§ 2 fixes the network classes. § 3 constructs the transform and proves the Plancherel identity (Theorem 3.14). § 4 proves the representation and reconstruction theorems (Theorems 4.5 and 4.8), describes the frame operator, and records the vector-valued extension. § 5 gives reconstruction formulas for activations that are tempered distributions, including ReLU. § 6 contains the approximation rates and the universality theorem (Theorems 6.6 and 6.8). § 7 presents explicit examples, § 8 reports numerical experiments, and § 9 discusses open questions. The supporting lemmas and proofs needed for these results appear in the main text before their use. The appendices give additional kernel and projection formulas, supplementary approximation bounds, the finite-dimensional comparison and dilation obstruction, numerical methods, and an extended overview of related work (Appendix § 15). The experiment code is supplied as supplementary material.

Every theorem, proposition, lemma, corollary, and example of this paper has been formalized and machine-checked in Lean 4 with Mathlib. The blueprint at https://shosonoda.github.io/lean-operator-ridgelet/ links each item to the Lean statement that carries it and to its proof. Two clauses are stated there in a weaker form than here: the L2L^2 clause in the conclusion of Theorem 5.10, and the closed forms of the reconstruction constants in Proposition 5.12, which the Lean statements carry as the Sobolev pairing itself. Nothing in the present paper depends on the formalization; it is offered as an independent check.

Neural networks with Hilbert-space inputs

We define finite-width networks and integral networks with scalar activations and Hilbert-valued outputs. These definitions fix the coefficient measures and the compact-open topology used in the representation and approximation results.

Finite-width and integral networks

Definition 2.1 (Finite-width network). Let σ:R→C\sigma:\mathbb{R}\to\mathbb{C} be continuous. A width-NN network on H\mathcal{H} with values in Y\mathcal{Y} is

fN(x)=∑j=1Nvj σ(⟨aj,x⟩−bj),vj∈Y,(aj,bj)∈H×R.(2)f_N(x)=\sum_{j=1}^{N}v_j\,\sigma\left(\langle a_j,x\rangle-b_j\right),\qquad v_j\in\mathcal{Y},\qquad(a_j,b_j)\in\mathcal{H}\times\mathbb{R}. \tag*{(2)}

The scalar case is Y=C\mathcal{Y}=\mathbb{C}.

We measure approximation in the compact-open topology of C(H;Y)C(\mathcal{H};\mathcal{Y}), generated by the seminorms

∥f∥C(K;Y):=sup⁡x∈K∥f(x)∥Y,K⊂H compact,\|f\|_{C(K;\mathcal{Y})}:=\sup_{x\in K}\|f(x)\|_{\mathcal{Y}},\qquad K\subset\mathcal{H}\text{ compact},

and write C(K)C(K) when Y=C\mathcal{Y}=\mathbb{C}. Compact-open approximation does not truncate the input: it asks for uniform approximation on each fixed compact subset of H\mathcal{H}. Finite-rank projections converge uniformly on a compact set, but a target need not factor through any one of them, and the examples in § 7 do not.

A vector measure assigns a vector to each Borel set and is countably additive in the output norm. Its variation is the positive measure ∣Γ∣(E)=sup⁡P∑j∥Γ(Ej)∥Y|\Gamma|(E) = \sup_{\mathcal{P}} \sum_j \|\Gamma(E_j)\|_{\mathcal{Y}}, where the supremum is over finite Borel partitions of EE; bounded variation means that its mass on the whole parameter space is finite. Since Y\mathcal{Y} is a Hilbert space, such a measure has a density hh with respect to its variation: Γ(E)=∫Eh d∣Γ∣\Gamma(E) = \int_E h\,\mathrm{d}|\Gamma|, with ∥h∥Y=1\|h\|_{\mathcal{Y}} = 1 almost everywhere. Accordingly, an integral ∫u dΓ\int u\,\mathrm{d}\Gamma of a scalar function means the Bochner integral ∫uh d∣Γ∣\int uh\,\mathrm{d}|\Gamma|. A vector-valued function is Bochner integrable when it is strongly measurable (an almost-everywhere limit of measurable simple functions) and its norm is integrable; its integral is obtained by integrating simple functions that approximate it in L1L^1. We refer to [27] for more details.

Definition 2.2 (Integral network). Let Γ\Gamma be a Y\mathcal{Y}-valued Borel measure of bounded variation on Θ:=H×R\Theta:= \mathcal{H} \times\mathbb{R}, with variation measure ∣Γ∣|\Gamma| and total variation ∥Γ∥TV:=∣Γ∣(Θ)\|\Gamma\|_{\mathrm{TV}} := |\Gamma|(\Theta). Whenever the Bochner integral exists, define

Sσ[Γ](x):=∫H×Rσ(⟨a,x⟩−b) Γ(da,db).(3)S_\sigma[\Gamma](x) := \int_{\mathcal{H}\times\mathbb{R}} \sigma(\langle a,x\rangle-b)\,\Gamma(\mathrm{d}a,\mathrm{d}b). \tag*{(3)}

If σ\sigma is globally Lipschitz, it suffices that

∫H×R(1+∥a∥+∣b∣) d∣Γ∣(a,b)<∞.(4)\int_{\mathcal{H}\times\mathbb{R}} (1+\|a\|+|b|)\,\mathrm{d}|\Gamma|(a,b) < \infty. \tag*{(4)}

When Γ=γλ\Gamma= \gamma\lambda has a density with respect to a sigma-finite reference measure λ\lambda on Θ\Theta, we also write Sσ[γ]S_\sigma[\gamma], and we say that the network has coefficient γ\gamma.

Finite-width networks are the syntheses of finite atomic measures. The total variation ∥Γ∥TV\|\Gamma\|_{\mathrm{TV}} is the infinite-width analogue of the ℓ1\ell^1-norm of the outer weights, and it is the quantity that enters Barron’s discretization in § 6. Scalar coefficients are complex; a real network is recovered by taking real parts.

The Gaussian-weighted ridgelet transform

We construct the transform using a Gaussian input measure and a homogeneous mixture of Gaussian direction measures. The Fourier-slice and Plancherel identities establish its analytic properties, and explicit filters show how the admissibility assumptions can be met.

The input and direction measures

Recall that PP and QQ are injective, positive, self-adjoint, trace-class operators on H\mathcal{H}; μQ\mu_Q is the input Gaussian law. Gaussian weighting makes the input integral available, but a single Gaussian law on directions is not homogeneous under dilation. In finite-dimensional ridgelet analysis, the dilation Jacobian ∣ω∣−m|\omega|^{-m} of Lebesgue measure on Rm\mathbb{R}^m enters the admissibility condition. The following mixture replaces mm by a freely chosen homogeneity parameter α\alpha, so the corresponding condition remains meaningful on H\mathcal{H}. For α>0\alpha> 0, define a Borel measure on the direction space by

να(E):=∫0∞μ2sP(E) sα/2−1 ds,E∈B(H).(5)\nu_\alpha(E) := \int_0^\infty\mu_{2sP}(E)\,s^{\alpha/2-1}\,\mathrm{d}s,\qquad E \in\mathcal{B}(\mathcal{H}). \tag*{(5)}

We first realize the Gaussian components on a common probability space and justify integration against the mixture.

The Gaussian measures can be realized on a common probability space (Ω,F,P)(\Omega,\mathcal{F},\mathbb{P}). Let Pej=pjejPe_j=p_je_j with pj>0p_j>0, ∑jpj<∞\sum_j p_j<\infty, and let ZjZ_j be independent standard normal variables. The series

X=∑j≥1pjZjej,Xs=2s XX=\sum_{j\geq1}\sqrt{p_j}Z_je_j,\qquad X_s=\sqrt{2s}\,X

References

  1. [1]Beatrice Acciaio, Anastasis Kratsios, and Gudmund Pammer. Designing universal causal deep learning models: the geometric (hyper)transformer. Mathematical Finance, 34(2):671–735, 2024.
  2. [2]Ben Adcock, Gregor Maier, and Rahul Parhi. Towards sharp minimax risk bounds for operator learning, 2025. arXiv:2512.17805.arxiv.org/abs/2512.17805
  3. [3]Ben Adcock, Michael Griebel, and Gregor Maier. Near-optimal learning of Gaussian Sobolev operators, 2026. arXiv:2607.11921.arxiv.org/abs/2607.11921
  4. [4]Francis Bach. Breaking the Curse of Dimensionality with Convex Neural Networks. Journal of Machine Learning Research, 18(19):1–53, 2017.
  5. [5]Andrew R. Barron. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory, 39(3):930–945, 1993.DOI
  6. [6]Pau Battle, Matthieu Darcy, Bamdad Hosseini, and Houman Owhadi. Kernel methods are competitive for operator learning. Journal of Computational Physics, 496:112549, 2024.
  7. [7]Jeremy J. Becnel. A limiting process to invert the Gauss–Radon transform. Communications on Stochastic Analysis, 13(2):219–242, 2019.DOI
  8. [8]Jeremy J. Becnel and Ambar N. Sengupta. A support theorem for a Gaussian Radon transform in infinite dimensions. Transactions of the American Mathematical Society, 364(3):1281–1291, 2012.DOI
  9. [9]Fred Espen Benth, Nils Detering, and Luca Galimberti. Neural networks in Fréchet spaces. Annals of Mathematics and Artificial Intelligence, 91(1):75–103, 2023.
  10. [10]Kaushik Bhattacharya, Bamdad Hosseini, Nikola B. Kovachki, and Andrew M. Stuart. Model reduction and neural networks for parametric PDEs. The SMAI Journal of Computational Mathematics, 7:121–157, 2021.arxiv.org/abs/2005.03180
  11. [11]Eugene Bilokopytov and Foivos Xanthos. A universal approximation theorem and its applications to vector lattice theory. Journal of Mathematical Analysis and Applications, 561(1):130632, 2026.
  12. [12]Vladimir I. Bogachev. Gaussian Measures, volume 62 of Mathematical Surveys and Monographs. American Mathematical Society, Providence, RI, 1998.
  13. [13]Vladimir I. Bogachev. Ornstein–Uhlenbeck operators and semigroups. Russian Mathematical Surveys, 73(2):191–260, 2018.DOI
  14. [14]Robert H. Cameron and William T. Martin. Fourier–Wiener transforms of analytic functionals. Duke Mathematical Journal, 12(3):489–507, 1945.DOI
  15. [15]Robert H. Cameron and William T. Martin. Fourier–Wiener transforms of functionals belonging to L₂ over the space C. Duke Mathematical Journal, 14(1):99–107, 1947.DOI
  16. [16]Emmanuel Jean Candès. Ridgelets: theory and applications. PhD thesis, Stanford University, 1998.
  17. [17]Emmanuel Jean Candès and David Leigh Donoho. Ridgelets: a key to higher-dimensional intermittency? Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 357(1760):2495–2509, 1999.
  18. [18]Shuang Chen, Juncai He, and Xue-Cheng Tai. Neural flow operators can approximate any operator: abstract frameworks and universal approximations, 2026. arXiv:2605.22557.arxiv.org/abs/2605.22557
  19. [19]Tianping Chen and Hong Chen. Approximations of continuous functionals by neural networks with application to dynamic systems. IEEE Transactions on Neural Networks, 4(6):910–918, 1993.pubmed.ncbi.nlm.nih.gov/18276521
  20. [20]Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks, 6(4):911–917, 1995.
  21. [21]Tianping Chen and Hong Chen. Approximation capability to functions of several variables, non-linear functionals, and operators by radial basis function neural networks. IEEE Transactions on Neural Networks, 6(4):904–910, 1995.
  22. [22]Sonja Cox, Asma Khedher, and Thijs Maessen. Universal approximation by signatures for infinite-dimensional rough paths, 2026. arXiv:2603.03058.arxiv.org/abs/2603.03058
  23. [23]Christa Cuchiero, Philipp Schmocker, and Josef Teichmann. Global universal approximation of functional input maps on weighted spaces. Constructive Approximation, 63(2):537–612, 2026.arxiv.org/abs/2306.03303
  24. [24]Giuseppe Da Prato and Jerzy Zabczyk. Stochastic Equations in Infinite Dimensions, volume 152 of Encyclopedia of Mathematics and its Applications. Cambridge University Press, Cambridge, 2nd edition, 2014.
  25. [25]Filippo de Feo. Universal approximation of nonlinear operators and their derivatives, 2026. arXiv:2605.15285.arxiv.org/abs/2605.15285
  26. [26]Maarten V. de Hoop, Nikola B. Kovachki, Nicholas H. Nelsen, and Andrew M. Stuart. Convergence rates for learning linear operators from noisy data. SIAM/ASA Journal on Uncertainty Quantification, 11(2):480–513, 2023.DOI
  27. [27]Joseph Diestel and John Jerry Uhl, Jr. Vector Measures, volume 15 of Mathematical Surveys. American Mathematical Society, Providence, RI, 1977. doi: 10.1090/surv/015.
  28. [28]Weinan E, Chao Ma, and Lei Wu. The Barron Space and the Flow-Induced Function Spaces for Neural Network Models. Constructive Approximation, 55(1):369–406, 2022.arxiv.org/abs/1906.08039
  29. [29]Frédéric Ferraty and Philippe Vieu. Nonparametric Functional Data Analysis: Theory and Practice. Springer Series in Statistics. Springer, New York, 2006.DOI
  30. [30]Massimo Fornasier and Holger Rauhut. Continuous Frames, Function Spaces, and the Discretization Problem. Journal of Fourier Analysis and Applications, 11(3):245–287, 2005.DOI
  31. [31]Takashi Furuya, Koichi Taniguchi, and Satoshi Okuda. Quantitative approximation for neural operators in nonlinear parabolic equations. In The Thirteenth International Conference on Learning Representations, 2025.arxiv.org/abs/2410.02151
  32. [32]Luca Galimberti. Neural networks in non-metric spaces. Analysis and Applications, 24(6):1415–1463, 2025.arxiv.org/abs/2406.09310
  33. [33]Luca Galimberti. L^p approximation results for infinite dimensional neural networks, 2026. arXiv:2608.08230.arxiv.org/abs/2608.08230
  34. [34]Luca Galimberti, Anastasis Kratsios, and Giulia Livieri. Designing universal causal deep learning models: the case of infinite-dimensional dynamical systems from stochastic analysis. Constructive Approximation, 64(2):411–477, 2026.
  35. [35]Janek Gödeke and Pascal Fersel. New universal operator approximation theorem for encoder–decoder architectures, 2025. arXiv:2503.24092.arxiv.org/abs/2503.24092
  36. [36]Lukas Gonon, Lyudmila Grigoryeva, and Juan-Pablo Ortega. Approximation bounds for random neural networks and reservoir systems. The Annals of Applied Probability, 33(1):28–69, 2023.arxiv.org/abs/2002.05933
  37. [37]Lukas Gonon, Lyudmila Grigoryeva, and Juan-Pablo Ortega. Infinite-dimensional reservoir computing. Neural Networks, 179:106486, 2024.
  38. [38]Leonard Gross. Potential theory on Hilbert space. Journal of Functional Analysis, 1(2):123–181, 1967.DOI
  39. [39]Alexander Hertle. Gaussian surface measures and the Radon transform on separable Banach spaces. In Dietrich Kölzow, editor, Measure Theory Oberwolfach 1979, volume 794 of Lecture Notes in Mathematics, pages 513–531. Springer, Berlin, Heidelberg, 1980.DOI
  40. [40]Takeyuki Hida, Hui-Hsiung Kuo, Jürgen Potthoff, and Ludwig Streit. White Noise: An Infinite Dimensional Calculus, volume 253 of Mathematics and Its Applications. Kluwer Academic Publishers, Dordrecht, 1993.
  41. [41]Nicki Holighaus and Peter Balazs. Discretization in generalized coorbit spaces: extensions, annotations and errata for “Continuous Frames, Function Spaces and the Discretization Problem” by M. Fornasier and H. Rauhut. ArXiv preprint 1702.06485, 2017.arxiv.org/abs/1702.06485
  42. [42]Irina Holmes. An inversion formula for the Gaussian Radon transform for Banach spaces. Infinite Dimensional Analysis, Quantum Probability and Related Topics, 16(4):1350034, 2013.arxiv.org/abs/1308.1392
  43. [43]Irina Holmes and Ambar N. Sengupta. A Gaussian Radon transform for Banach spaces. Journal of Functional Analysis, 263(11):3689–3706, 2012.arxiv.org/abs/1208.5743
  44. [44]Irina Holmes and Ambar N. Sengupta. The Gaussian Radon transform in classical Wiener space. Communications on Stochastic Analysis, 8(2):247–268, 2014.arxiv.org/abs/1409.5633
  45. [45]Irina Holmes and Ambar N. Sengupta. The Gaussian Radon transform and machine learning. Infinite Dimensional Analysis, Quantum Probability and Related Topics, 18(3):1550019, 2015.arxiv.org/abs/1310.4794
  46. [46]Vugar E. Ismailov. On shallow feedforward neural networks with inputs from a topological space. Annals of Mathematics and Artificial Intelligence, 2026. Published online.DOI
  47. [47]Vugar E. Ismailov. Universality of shallow and deep neural networks on non-Euclidean spaces, 2026. arXiv:2602.23381.arxiv.org/abs/2602.23381
  48. [48]Svante Janson. Gaussian Hilbert Spaces, volume 129 of Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, 1997.DOI
  49. [49]Palle E. T. Jorgensen, Myung-Sin Song, and James Tian. Infinite-dimensional stochastic transforms and reproducing kernel Hilbert space. Sampling Theory, Signal Processing, and Data Analysis, 21(1):12, 2023.arxiv.org/abs/2209.03801
  50. [50]Hachem Kadri, Emmanuel Duflos, Philippe Preux, Stéphane Canu, Alain Rakotomamonjy, and Julien Audiffren. Operator-valued kernels for learning from functional response data. Journal of Machine Learning Research, 17(20):1–54, 2016.arxiv.org/abs/1510.08231
  51. [51]Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic, and Bharath K. Sriperumbudur. Gaussian processes and kernel methods: a review on connections and equivalences, 2018. arXiv:1807.02582.arxiv.org/abs/1807.02582
  52. [52]Yury Korolev. Two-layer neural networks with values in a Banach space. SIAM Journal on Mathematical Analysis, 54(6):6358–6389, 2022.arxiv.org/abs/2105.02095
  53. [53]Nikola Kovachki, Samuel Lanthaler, and Siddhartha Mishra. On universal approximation and error bounds for Fourier neural operators. Journal of Machine Learning Research, 22(290):1–76, 2021.arxiv.org/abs/2107.07562
  54. [54]Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizdadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural Operator: Learning maps between function spaces with applications to PDEs. Journal of Machine Learning Research, 24(89):1–97, 2023.
  55. [55]Nikola B. Kovachki, Samuel Lanthaler, and Hrushikesh Mhaskar. Data complexity estimates for operator learning, 2024. arXiv:2405.15992.arxiv.org/abs/2405.15992
  56. [56]Nikola B. Kovachki, Samuel Lanthaler, and Andrew M. Stuart. Operator learning: algorithms and analysis. In Numerical Analysis Meets Machine Learning, volume 25 of Handbook of Numerical Analysis, pages 419–467. Elsevier, 2024.arxiv.org/abs/2402.15715
  57. [57]Anastasis Kratsios and Ievgen Bilokopytov. Non-Euclidean universal approximation. In Advances in Neural Information Processing Systems, volume 33, pages 10635–10646, 2020.arxiv.org/abs/2006.02341
  58. [58]Anastasis Kratsios, Takashi Furuya, Jose Antonio Lara Benitez, Matti Lassas, and Maarten de Hoop. Mixtures of neural operators reduce active complexity in operator learning, 2024. arXiv:2404.09101.arxiv.org/abs/2404.09101
  59. [59]Věra Kůrková. Complexity estimates based on integral transforms induced by computational units. Neural Networks, 33:160–167, 2012.DOI
  60. [60]Hui-Hsiung Kuo. The Fourier transform in white noise calculus. Journal of Multivariate Analysis, 31(2):311–327, 1989.DOI
  61. [61]Samuel Lanthaler. Operator learning with PCA-Net: upper and lower complexity bounds. Journal of Machine Learning Research, 24(318):1–67, 2023.arxiv.org/abs/2303.16317
  62. [62]Samuel Lanthaler. Operator learning of Lipschitz operators: an information-theoretic perspective, 2024. arXiv:2406.18794.arxiv.org/abs/2406.18794
  63. [63]Samuel Lanthaler and Andrew M. Stuart. The parametric complexity of operator learning. IMA Journal of Numerical Analysis, 46(2):647–712, 2026.arxiv.org/abs/2306.15924
  64. [64]Samuel Lanthaler, Siddhartha Mishra, and George Em Karniadakis. Error estimates for DeepONets: a deep learning framework in infinite dimensions. Transactions of Mathematics and Its Applications, 6(1):tnac001, 2022.arxiv.org/abs/2102.09618
  65. [65]Samuel Lanthaler, Zongyi Li, and Andrew M. Stuart. Nonlocality and nonlinearity implies universality in operator learning. Constructive Approximation, 62(2):261–303, 2025.arxiv.org/abs/2304.13221
  66. [66]Yuh-Jia Lee. Analytic version of test functionals, Fourier transform, and a characterization of measures in white noise calculus. Journal of Functional Analysis, 100(2):359–380, 1991.DOI
  67. [67]Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, 2021.
  68. [68]Claude Martias. La transformée en ridelettes pour les fonctionnelles de Wiener. Comptes Rendus Mathématique, 350(5–6):259–262, 2012.
  69. [69]Claude Martias. On the subexponentiality of the ridgelet transform. Comptes Rendus Mathématique, 352(12):1029–1031, 2014.DOI
  70. [70]Claude Martias. The wavelet transform for Wiener functionals and some applications. Stochastics, 86(5):817–834, 2014.DOI
  71. [71]Claude Martias. Bases of wavelets and multiresolution in analysis on Wiener space. In Francisco Bulnes, editor, Recent Advances in Wavelet Transforms and Their Applications. IntechOpen, London, 2022.DOI
  72. [72]Hrushikesh N. Mhaskar and Nahmwoo Hahm. Neural networks for functional approximation and system identification. Neural Computation, 9(1):143–159, 1997.pubmed.ncbi.nlm.nih.gov/9117896
  73. [73]Vochita Mihai and Ambar N. Sengupta. The Radon–Gauss transform. Soochow Journal of Mathematics, 33(3):415–434, 2007.
  74. [74]Noboru Murata. An integral representation of functions using three-layered networks and their approximation bounds. Neural Networks, 9(6):947–956, 1996.pubmed.ncbi.nlm.nih.gov/12662574
  75. [75]Frank Natterer. The Mathematics of Computerized Tomography. B. G. Teubner, Stuttgart, 1986. doi: 10.1007/978-3-663-01409-6.DOI
  76. [76]Nicholas H. Nelsen and Andrew M. Stuart. The random feature model for input–output maps between Banach spaces. SIAM Journal on Scientific Computing, 43(5):A3212–A3243, 2021.DOI
  77. [77]Nicholas H. Nelsen and Andrew M. Stuart. Operator learning using random features: a tool for scientific computing. SIAM Review, 66(3):535–571, 2024.DOI
  78. [78]Ariel Neufeld and Philipp Schmocker. Universal approximation property of Banach space-valued random feature models including random neural networks, 2023. arXiv:2312.08410.arxiv.org/abs/2312.08410
  79. [79]Ariel Neufeld and Philipp Schmock­er. Universal approximation results for neural networks with non-polynomial activation function over non-compact domains. Analysis and Applications, 24(5):1123–1173, 2026.arxiv.org/abs/2410.14759
  80. [80]Claudia Prévot and Michael Röckner. A Concise Course on Stochastic Partial Differential Equations, volume 1905 of Lecture Notes in Mathematics. Springer, Berlin, 2007.
  81. [81]Fabrice Rossi and Brieuc Conan-Guez. Functional multi-layer perceptron: a non-linear tool for functional data analysis. Neural Networks, 18(1):45–60, 2005.pubmed.ncbi.nlm.nih.gov/15649661
  82. [82]Philipp Schmock­er and Josef Teichmann. Weighted universal approximation of differentiable maps on infinite-dimensional manifolds, 2026. arXiv:2606.09820.arxiv.org/abs/2606.09820
  83. [83]Irving E. Segal. Tensor algebras over Hilbert spaces. I. Transactions of the American Mathematical Society, 81:106–134, 1956.DOI
  84. [84]Jian Qing Shi and Taeryon Choi. Gaussian Process Regression Analysis for Functional Data. Chapman & Hall/CRC, Boca Raton, 2011.DOI
  85. [85]L i n h a o Song, Jun Fan, Di-Rong Chen, and Ding-Xuan Zhou. Approximation of nonlinear functionals using deep ReLU networks. Journal of Fourier Analysis and Applications, 29(4):50, 2023.arxiv.org/abs/2304.04443
  86. [86]L i n h a o Song, Ying Liu, Jun Fan, and Ding-Xuan Zhou. Approximation of smooth functionals using deep ReLU networks. Neural Networks, 166:424–436, 2023.DOI
  87. [87]Sho Sonoda and Noboru Murata. Neural network with unbounded activation functions is universal approximator. Applied and Computational Harmonic Analysis, 43(2):233–268, 2017.
  88. [88]Sho Sonoda, Isao Ishikawa, and Masahiro Ikeda. Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35, pages 38680–38694, New Orleans, Louisiana, USA, 2022. Curran Associates, Inc.arxiv.org/abs/2205.14819
  89. [89]Sho Sonoda, Isao Ishikawa, and Masahiro Ikeda. A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks. Journal of Statistical Planning and Inference, 233:106184, 2024.DOI
  90. [90]Sho Sonoda, Isao Ishikawa, and Masahiro Ikeda. Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space. ArXiv preprint 2106.04770v2, 2026.
  91. [91]Maxwell B. Stinchcombe. Neural network approximation of continuous functionals and continuous functions on compactifications. Neural Networks, 12(3):467–477, 1999.pubmed.ncbi.nlm.nih.gov/12662689
  92. [92]Koichi Taniguchi and Sho Sonoda. Generalization error bounds for Picard-type operator learning in nonlinear parabolic PDEs, 2026. arXiv:2605.10277.arxiv.org/abs/2605.10277
  93. [93]Barinder Thind, Kevin Multani, and Jiguo Cao. Deep learning with functional inputs. Journal of Computational and Graphical Statistics, 32(1):171–180, 2023.DOI
  94. [94]Wilfredo Urbina-Romero. Gaussian Harmonic Analysis. Springer Monographs in Mathematics. Springer, Cham, 2019.
  95. [95]Aad W. van der Vaart and J. Harry van Zanten. Rates of contraction of posterior distributions based on Gaussian process priors. The Annals of Statistics, 36(3):1435–1463, 2008.
  96. [96]Teun D. H. van Nuland. Noncompact uniform universal approximation. Neural Networks, 173:106181, 2024.DOI
  97. [97]Jane-Ling Wang, Jeng-Min Chiou, and Hans-Georg Müller. Functional data analysis. Annual Review of Statistics and Its Application, 3:257–295, 2016.
  98. [98]George Wynne and Andrew B. Duncan. A kernel two-sample test for functional data. Journal of Machine Learning Research, 23(73):1–51, 2022.arxiv.org/abs/2008.11095
  99. [99]Jia-Qi Yang and Lei Shi. Variation Spaces for Encoder–Decoder Neural Operators: Approximation and Generalization. ArXiv preprint 2606.01244v2, 2026.arxiv.org/abs/2606.01244
  100. [100]Boyuan Yao, Dingcheng Luo, Lianghao Cao, Nikola Kovachki, Thomas O’Leary-Roseberry, and Omar Ghattas. Derivative-informed Fourier neural operator: universal approximation and applications to PDE-constrained optimization, 2025. arXiv:2512.14086.arxiv.org/abs/2512.14086
  101. [101]Junwen Yao, Jonas Mueller, and Jane-Ling Wang. Deep learning for functional data analysis with adaptive basis layers. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 11898–11908, 2021.arxiv.org/abs/2106.10414
  102. [102]Emanuele Zappala and Maryam Bagherian. Universal approximation of operators with transformers and neural integral operators. Axioms, 15(9):676, 2026.arxiv.org/abs/2409.00841
  103. [103]Tian-Yi Zhou, Namjoon Suh, Guang Cheng, and Xiaoming Huo. Neural operators for nonlinear functionals on RKHS, 2024. arXiv:2403.12187.arxiv.org/abs/2403.12187

Paper details

vlnc:W_amta8bp9
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents