betaFeedback

WorksMathematics

Neural Network Approximation and Structural Complexity of Semilinear PDEs with Lower Bounds

Abstract

Neural networks are commonly used to learn the solution operators of PDEs such as the Navier-Stokes equations and viscous conservation laws. On standard benchmarks, moderately sized and often shallow networks achieve small errors that decay at algebraic rates with the network size. These observations are not well described by the current approximation theory, which treats the solution operator as a generic map and derives worst-case bounds over large classes of operators. We show that the structure of the equations and their properties explain the observed behavior. For semilinear parabolic equations, the approximation costs separate into two terms, where one is controlled by an invariant of the governing law and the other by the data. The constitutive law can be parameterized as a single block whose size for accuracy ηη is O(η−pχlog⁡η−1)O(η^{-pχ}\logη^{-1}) for every p>pfp>p_f, where the structural exponent pfp_f only depends on the computation graph of ff. This exponent does not depend on the dimension and is in fact zero for polynomial laws, so for the two-dimensional Navier-Stokes equations the nonlinear block costs only O(log⁡η−1)O(\logη^{-1}) parameters. The data enters through encoding layers with a bounded number of learned weights per mode, so neither term grows with the grid resolution or the time step. While the analysis only uses the max-norm error of the pointwise block, we show that one obtains approximation bounds in a stronger norm along the whole trajectory. For viscous Hamilton-Jacobi equations with a CsC^s Hamiltonian, the approximation requires on the order of η−1/sη^{-1/s} parameters, and we prove a matching lower bound, up to logarithms. This shows that solution operators for this class of problems have low structural complexity, even though they are nonlinear maps between infinite-dimensional spaces.

Introduction

Let Ω⊂Rdx\Omega\subset\mathbb{R}^{d_x} be a bounded spatial domain and consider the semilinear evolution equation

ut+Lu=f(u,∇u),u(0)=u0,(1)u_t + Lu = f(u, \nabla u), \qquad u(0) = u_0, \tag*{(1)}

where LL is a linear differential operator and ff the nonlinear constitutive law depending on the state and its derivative. This class of partial differential equations (PDE) covers a wide range of important dynamical models in science and engineering, for example, viscous Burgers and viscous conservation laws, viscous Hamilton–Jacobi equations, reaction-diffusion systems e.g. Allen-Cahn and the diffusive FitzHugh-Nagumo system, Cahn–Hilliard, and the two-dimensional incompressible Navier–Stokes equations [29, 22, 62, 77, 76]. In scientific machine learning (SciML), these systems and their associated solution operators have become benchmark examples to test architectures and algorithms [45, 52, 4, 36, 50, 73, 30, 28].

The goal of these methods is to approximate the solution operator GT:u0↦uG_T : u_0 \mapsto u, which maps the initial data either to the solution at the final time t=Tt = T [45, 35, 10] or to the full trajectory on [0,T][0,T] [50, 48, 73, 26, 75, 49]. We adopt the latter formulation throughout this work, which is a stronger requirement. Since these equations serve as test cases throughout the literature for learning solution operators with neural networks, we study the structural complexity of approximating their solution operators [45, 52, 36, 50]. In particular, we quantify how many trainable parameters are needed to approximate the solution operator to accuracy η\eta, in the sense of the parametric complexity for operator learning [42]. Our aim is to analyze these systems in a way that allows for a nuanced separation between the quantities that control the cost of approximation. This error leads to terms that depend on the complexity of the nonlinear governing law, the derived regularity from the dynamics, the spatial and temporal resolution, the encoding of the input, and the contraction of the iterates. We show that the first two terms require trainable parameters, and that their total number does not grow with the grid size or resolution. The remaining error terms can affect the size of the network; however, they do not change the number of parameters that must be learned. In this way, we show that the solution operators of semilinear PDEs have low structural complexity, which provides a theoretical explanation for why learned neural approximations perform well on them, even though they are seemingly complicated nonlinear operators between infinite-dimensional spaces.

In practice neural operators [52, 45, 36, 4, 91] utilize an encoder to map the inputs into a latent space. For Fourier Neural Operators (FNO), this is the spectral domain given by the fast Fourier transform (FFT), and for PCA-based networks, it is the span of the leading principal components of the data generated by a PCA layer [45, 4]. Most neural operators utilize linear layers (affine or convolutional) and point-wise nonlinear activation functions applied with the same weights at every spatial location [36, 45, 52, 4]. In empirical studies, neural operators are more parameter efficient than what the theory of generic nonlinear operators shows. Specifically, in operator learning theory the rates have a curse of parametric complexity [42, 47] which yields approximation and sampling bounds that are exponentials of power laws. However, simulations show that the errors can decay at algebraic rates, for example in the size of the training set [52, 15, 72], and can be efficient over larger multi-operator settings [50, 49, 58, 30, 28, 9].

One theoretical limitation is that the universal approximation results [13, 83] and the quantitative complexity results for the class of Lipschitz or smooth operators [42, 47, 82] take the solution operator GTG_T to be a generic map between Banach spaces. Since these operators are only characterized by their Lipschitz or CrC^r regularity on infinite-dimensional spaces, the metric entropy grows like exp⁡(η−p)\exp(\eta^{-p}), and the associated cost for standard architectures is the same. In particular, this leads to exponential scalings for the parameter count [42, 39, 47] and generalization rates that are only logarithmic in terms of the number of samples [37, 1, 47, 81, 82]. To get algebraic approximation rates for Lipschitz operators, one needs non-standard architectures such as super-expressive activation functions or non-standard ReLU architectures [88, 69] which is consistent with the metric-entropy bound for encoding information [39, 47].

Utilizing the structure of the operator or data can help to avoid the curse of parametric complexity. One approach is to assume holomorphic dependence on the input, which yields algebraic rates [70, 61, 31, 54, 65]. For elliptic problems, the size of networks that can approximate the solution scales like the size of networks representing the coefficients or the nonlinearity [56, 55]. Another direction is to relate the architecture to the PDE by showing that it can emulate a numerical scheme. Examples include the Picard iteration for scalar reaction laws [26, 75, 24], learned time-stepping operators [11], iterated one-step operators for convex monotone semigroups [6], and emulating stable spectral schemes [10]. For FNO that emulate stable spectral schemes, [10] showed that a polynomial nonlinearity only adds a logarithmic cost, while the number of retained Fourier modes determines the overall rate.

Compared with these works, we take a complementary approach that applies to laws that are nonlinear in (u,∇u)(u,\nabla u). We separate the parameter count into a term that is controlled by the governing equation and a term controlled by the derived regularity of the data. We show that both terms are necessary. We use three properties of semilinear parabolic PDEs that are absent for a generic Lipschitz operator:

  1. Fixed-point Structure: On a short horizon, the solutions are the unique fixed points of the mild map

Φ[u](t)=S(t)u0+∫0tS(t−s)f(u(s),∇u(s)) ds\Phi[u](t) = S(t)u_0 + \int_0^t S(t-s)f(u(s),\nabla u(s))\,\mathrm{d}s

which is a contraction on a ball in C([0,T];Xα)C([0,T];X^\alpha). Thus, a solution operator can be taken as the limit of an iteration whose only nonlinearity comes from ff.

  1. Parabolic Smoothing: The semigroup S(t)=e−tLS(t)=e^{-tL} is analytic and smoothing, so certain high frequencies are damped. Therefore, we can quantify the truncation and quadrature errors with explicit algebraic rates induced directly from the dynamics itself.

  1. Algebraic Parametric Complexity of the Governing Law: The term ff can be written as a finite directed acyclic graph (DAG) made from elementary operations, and is independent of any discretization. The cost of approximating the governing law is determined by the structure of the DAG rather than by the number of variables of ff or the number of space-time points.

Combining these properties allows a standard architecture to use one small point-wise neural network whose parameters are shared across grid points, time levels and iterations, and thus keeps the number of trainable parameters small.

Main result

The analysis uses the following pipeline to chain the approximations (see Figure 1). The Galerkin truncation, temporal quadrature, and spatial sampling are used to form a discrete Picard map that approximates the mild map. The governing law ff is represented by its DAG form and is replaced by a deep ReLU network which approximates each node in the DAG. To apply it over the grid, weight sharing through a 1×11 \times1 convolution (Lemma 3.4) is used. This results in a looped structure that mimics a fixed-point method.

Figure 1. Approximation Pipeline and Associated Errors. All steps are discretizations except the last one, which replaces the governing law by a shared ReLU network and the corresponding trainable parameters. The linear steps utilize fixed transforms, learned spectral weights, and parameter sharing. This leads to a loop, which is illustrated in Figure 3.

Architecture of the neural Picard approximation

Figure 3. The architecture of Definition 4.7, drawn as a loop. One sweep consists of the four layers (2a)–(2d), the loop is applied nn times with the same weights, and the encoded data is connected to every sweep. The trainable parameters are the spectral weights of the shaded layers and those of the single block f^ε\widehat{f}_{\varepsilon}, which acts as a 1×11 \times1 convolution across the MM sample points and is reused at every one of the QQ time levels entering the Duhamel sum and at every sweep.

Let Xs=D(Ls/2)X^{s}=D(L^{s/2}) be the space generated by LL, γs\gamma_s be the spatial regularity of the data, and ζt≤1\zeta_t \le1 be the resulting temporal order of accuracy (see § 1.4 for notation). The main complexity result is informally stated below and summarizes the rigorous results Theorems 5.6 and 5.14.

Theorem 1.1 (Main result, informal). Let uu be the mild solution of (1) on [0,T][0,T] with spatial regularity γs\gamma_s. If ff can be written as a finite DAG whose nonlinear nodes are multilinear or component-wise CsjC^{s_j}, with pmax⁡=1/smin⁡p_{\max}=1/s_{\min}, then for every sufficiently small η>0\eta>0, the architecture from Definition 4.7, with

λN+1≍η−2/(γs−α),h≍η1/(γs−α),Δt≍η1/ζt,n≍log⁡η−1,\lambda_{N+1}\asymp\eta^{-2/(\gamma_s-\alpha)}, \qquad h\asymp\eta^{1/(\gamma_s-\alpha)}, \qquad\Delta t\asymp\eta^{1/\zeta_t}, \qquad n\asymp\log\eta^{-1},

admits a choice of trainable parameters for which ∥u^−u∥C([0,T];Xα)≤η\|\widehat{u}-u\|_{C([0,T];X^\alpha)}\le\eta holds. Its trainable parameters scale as

W(f^ε)≤Cη−pmax⁡χslog⁡η−1+Wspec≤Cη−dx/(γs−α),W(\widehat{f}_{\varepsilon})\le C\eta^{-p_{\max}\chi_s}\log\eta^{-1}+W_{\mathrm{spec}}\le C\eta^{-d_x/(\gamma_s-\alpha)},

and are independent of the number of grid points, time levels, and sweeps. The full network without parameter sharing scales like

W(An)≤Cη−ν(log⁡η−1)2,ν=dxγs−α+1ζt+pmax⁡χs,dep⁡(An)≤C(log⁡η−1)2,W(\mathcal{A}_n)\le C\eta^{-\nu}\left(\log\eta^{-1}\right)^2,\qquad\nu=\frac{d_x}{\gamma_s-\alpha}+\frac{1}{\zeta_t}+p_{\max}\chi_s,\qquad\operatorname{dep}(\mathcal{A}_n)\le C\left(\log\eta^{-1}\right)^2,

where χs≥1\chi_s\ge1 equals $1$ for α<2\alpha<2 and decreases to 11 as the data becomes smoother, with an extra factor (log⁡η−1)pmax⁡(\log\eta^{-1})^{p_{\max}} when $\alpha=2.

If every node is multilinear, then ν=dxγs−α+1ζt\nu=\frac{d_x}{\gamma_s-\alpha}+\frac{1}{\zeta_t} and the complexity is set only by the space-time resolution.

The exponent pmax⁡=1/smin⁡p_{\max}=1/s_{\min} is the classical ReLU rate [86, 18] applied to the least smooth nonlinear node in the DAG form of ff. It is the only place where the smoothness of ff appears explicitly in the bounds. It is multiplied by the stability factor χs\chi_s, which comes from the lack of smoothing by the Duhamel kernel when we measure the error in XαX^\alpha, α>2\alpha>2. This is sharp (see Proposition 7.1), but under some assumptions it can be improved (see § 7.1).

Contributions

We list some of the contributions of this work.

  1. Separation of the Parameter Count (Theorems 5.6 and 5.14). We split the number of trainable parameters needed to obtain an accuracy of η\eta in C([0,T];Xα)C([0,T];X^\alpha) into a term controlled by the governing law, i.e. Cη−pχslog⁡η−1C\eta^{-p\chi_s}\log\eta^{-1}, and a term controlled by the regularity of the data, i.e. Cη−dx/(γs−α)C\eta^{-d_x/(\gamma_s-\alpha)}. The terms do not grow with the grid, the time step, or the number of iterations of the looped network due to parameter sharing. These results also apply to popular operator learning approaches, for example, those based on FNO or DeepONet (see Remarks 4.8 and 4.9).

  1. The Structural Exponent (Section 3). For governing laws whose DAG depends on inputs (u,∇u)(u,\nabla u), the complexity is set by the least smooth nonlinear node (see Theorem 3.7), building on [68, 64]. Taking the infimum over all DAG forms of ff yields the structural exponent pfp_f which is an invariant of the system and does not depend on dd, dxd_x or the number of state components. For polynomial systems, it is equal to zero.

  1. A Derived Error Decomposition (Section 5). All of the sources of error and assumptions on the regularity are derived from the mild formulation. They are bounded separately rather than assumed through the accuracy of a particular scheme (Theorem 5.4). The error is measured in C([0,T];Xα)C([0,T];X^\alpha) which is a stronger norm than is typically used [26, 75, 10] but still achieves a scaling for the depth that is only C(log⁡η−1)2C(\log\eta^{-1})^2 (see Theorem 5.13), the same as in [26, 24]. The decomposition is written in the interpretable form of

κn⏟iteration+Cεξα(N,Δt)⏟network+εtail(u0)+εenc(u0)⏟encoding+CλN+1−(γs−α)/2+Chr⏟spatial+C(Δt)ζ⏟temporal\underbrace{\kappa^n}_{\text{iteration}}+\underbrace{C\varepsilon\xi_\alpha(N,\Delta t)}_{\text{network}}+\underbrace{\varepsilon^{\mathrm{tail}}(u_0)+\varepsilon^{\mathrm{enc}}(u_0)}_{\text{encoding}}+C\lambda_{N+1}^{-(\gamma_s-\alpha)/2}+\underbrace{Ch^r}_{\text{spatial}}+\underbrace{C(\Delta t)^\zeta}_{\text{temporal}}

For α>2\alpha>2 the network error is amplified, which multiplies its exponent by χs\chi_s. This amplification is sharp by Proposition 7.1 in general, but can be removed by certain formulations for gradient-free laws when dx≤3d_x\leq3 (see Section 7.1).

  1. Lower Bounds (Section 7). Any architecture whose weights depend continuously on the governing laws needs at least cη−1/(s+1)c\eta^{-1/(s+1)} parameters for gradient-free governing laws and at least c(ηlog⁡η−1)−1/sc(\eta\log\eta^{-1})^{-1/s} over a class of gradient dependent governing laws. For gradient dependent laws and s≥2s\geq2, the lower bound rates match the upper bound rates up to logs (see Theorems 7.3 and 7.5). If the spectrum of LL is trainable, at least one weight mode is required, so the data term in our analysis is sharp (Proposition 7.6).

In addition, our results are extended to longer time intervals with the same network (Proposition 5.9). For gradient-free laws, the error is controlled by a one-sided rate rather than only using contraction (Lemma 5.10). This also extends to the solution operator using just one set of weights for all data in some fixed ball (Theorem 6.5). We compute the exponents for seven commonly used PDEs and examine the constants in the Allen-Cahn case, where the block has a few hundred parameters on a single window and the trainable count is dominated by the spectral weights (Section 8).

Related work

There are several quantitative results for learning the solutions to PDEs using neural networks. In [40], error estimates are provided for DeepONet, and in [47], approximation and generalization rates are provided over the class of Lipschitz operators. Generic error bounds for operator learning that apply to nonlinear parabolic equations were shown in [17]. Error bounds were derived for FNO in [35], and upper and lower complexity bounds for PCA-Net can be found in [38]. Approximation and generalization bounds for encoder-decoder architectures were analyzed in [85]. In [89], universal approximation was derived for a discretization-invariant extension of DeepONet, where the network has an additional component to resolve the grids, and general universal approximation results appeared in [83]. In [41], they proved lower bounds showing that operator learning approaches that use a linear reconstruction step, e.g. DeepONet and PCA-Net, cannot efficiently approximate discontinuous solutions. In [25], they showed that bijective neural operators between infinite-dimensional Hilbert spaces may not admit continuous finite-dimensional discretizations. In [43], the authors examined the discretization error associated with evaluating an FNO on a grid and proved an algebraic rate with respect to the resolution.

As shown in [42], the curse of parametric complexity over Lipschitz or CsC^{s} classes leads to parameter counts of exp⁡(ε−r)\exp(\varepsilon^{-r}) for operator learning, including DeepONet, PCA-Net, and FNO. However, using structures or other information can alleviate this restriction. FNO can emulate pseudo-spectral methods for periodic domains using C3C^{3} activation functions (not ReLU), and so can approximate the Darcy and Navier-Stokes solution operators with explicit bounds [35]. In [40], they used the Cole-Hopf transformation to obtain explicit DeepONet bounds for viscous Burgers. In our results, the bound follows from the bilinear structure of the law. In [38], it is shown that PCA-Net can avoid the curse of parametric complexity for Darcy flow by using the structure of the parametrized coefficient and avoids it for the Navier-Stokes equations through emulation of a numerical method. Their results are measured in the averaged error.

For the setting of self-adjoint linear operators, [16] showed that the generalization rates are algebraic and depend on the eigenvalue decay and data smoothness. In [10], under the assumption that the solution operator has a stable and accurate Fourier spectral discretization, they proved that FNO learns an accurate approximation with polynomial sample complexity. They show that the complexity for the problem ut+Au+Ds(G(u))=fu_t + Au + D_s(\mathcal{G}(u)) = f on the torus, under mild conditions, leads to algebraic rates in L2L^{2} and provide generalization bounds.

References

  1. [1]Ben Adcock, Gregor Maier, and Rahul Parhi. Towards sharp minimax risk bounds for operator learning. arXiv preprint arXiv:2512.17805, 2025.arxiv.org/abs/2512.17805
  2. [2]Hajer Bahouri, Jean-Yves Chemin, and Raphaël Danchin. Fourier analysis and nonlinear partial differential equations, volume 343. Springer, 2011.DOI
  3. [3]Shaojie Bai, J Zico Kolter, and Vladlen Koltun. Deep equilibrium models. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1909.01377
  4. [4]Kaushik Bhattacharya, Bamdad Hosseini, Nikola B Kovachki, and Andrew M Stuart. Model reduction and neural networks for parametric pdes. The SMAI journal of computational mathematics, 7:121–157, 2021.arxiv.org/abs/2005.03180
  5. [5]Guy E Blelloch. Prefix sums and their applications. 1990.
  6. [6]Jonas Blessing, Philipp Schmock er, and Alessandro Sgarabottolo. Neural operators approximate strongly continuous convex monotone semigroups. arXiv preprint arXiv:2609.02727, 2026.arxiv.org/abs/2609.02727
  7. [7]Helmut Bolcskei, Philipp Grohs, Gitta Kutyniok, and Philipp Petersen. Optimal approximation with sparsely connected deep neural networks. SIAM Journal on Mathematics of Data Science, 1(1):8–45, 2019.
  8. [8]Karol Borsuk. Drei sätze über die n-dimensionale euklidische sphäre. Fundamenta Mathematicae, 20(1):177–190, 1933.DOI
  9. [9]Yadi Cao, Yuxuan Liu, Liu Yang, Rose Yu, Hayden Schaeffer, and Stanley Osher. Vicon: Vision in-context operator networks for multi-physics fluid dynamics prediction. arXiv preprint arXiv:2411.16063, 2024.arxiv.org/abs/2411.16063
  10. [10]Nisha Chandramoorthy, Daniel Sanz-Alonso, and Nathan Waniorek. From spectral methods to sample complexity bounds for fourier neural operators. arXiv preprint arXiv:2607.00320, 2026.arxiv.org/abs/2607.00320
  11. [11]Ke Chen, Meenakshi Krishnan, and Haizhao Yang. Error analysis for learning the time-stepping operator of evolutionary pdes. arXiv preprint arXiv:2509.04256, 2025.arxiv.org/abs/2509.04256
  12. [12]Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.arxiv.org/abs/1806.07366
  13. [13]Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE transactions on neural networks, 6(4):911–917, 1995.
  14. [14]Steven M Cox and Paul C Matthews. Exponential time differencing for stiff systems. Journal of Computational Physics, 176(2):430–455, 2002.DOI
  15. [15]Maarten V de Hoop, Daniel Zhengyu Huang, Elizabeth Qian, and Andrew M Stuart. The cost-accuracy trade-off in operator learning with neural networks. arXiv preprint arXiv:2203.13181, 2022.arxiv.org/abs/2203.13181
  16. [16]Maarten V de Hoop, Nikola B Kovachki, Nicholas H Nelsen, and Andrew M Stuart. Convergence rates for learning linear operators from noisy data. SIAM/ASA Journal on Uncertainty Quantification, 11(2):480–513, 2023.DOI
  17. [17]Tim De Ryck and Siddhartha Mishra. Generic bounds on the approximation error for physics-informed (and) operator learning. Advances in Neural Information Processing Systems, 35:10945–10958, 2022.arxiv.org/abs/2205.11393
  18. [18]Ronald DeVore, Boris Hanin, and Guergana Petrova. Neural network approximation. Acta Numerica, 30:327–444, 2021.arxiv.org/abs/2012.14501
  19. [19]Ronald A DeVore, Ralph Howard, and Charles Micchelli. Optimal nonlinear approximation. Manuscripta mathematica, 63(4):469–478, 1989.DOI
  20. [20]James R Driscoll and Dennis M Healy. Computing fourier transforms and convolutions on the 2-sphere. Advances in applied mathematics, 15(2):202–250, 1994.
  21. [21]Weinan E, Martin Hutzenthaler, Arnulf Jentzen, Thomas Kruse, et al. Multilevel picard iterations for solving smooth semilinear parabolic heat equations. Partial Differential Equations and Applications, 2(6):80, 2021.DOI
  22. [22]Lawrence C Evans. Partial differential equations, volume 19. American mathematical society, 2022.
  23. [23]Michael Feischl, Christoph Schwab, and F Zehetgruber. Neural general operator networks via banach fixed point iterations. SAM Research Report, 2025, 2025.
  24. [24]Takashi Furuya, Ryo Ozawa, and Jenn-Nan Wang. Approximation theory of laplacian-based neural operators for reaction-diffusion system. arXiv preprint arXiv:2605.12025, 2026.arxiv.org/abs/2605.12025
  25. [25]Takashi Furuya, Michael Puthawala, Maarten V de Hoop, and Matti Lassas. Can neural operators always be continuously discretized? Advances in Neural Information Processing Systems, 37:98936–98993, 2024.arxiv.org/abs/2412.03393
  26. [26]Takashi Furuya, Koichi Taniguchi, and Satoshi Okuda. Quantitative approximation for neural operators in nonlinear parabolic equations. In International Conference on Learning Representations, volume 2025, pages 85528–85556, 2025.arxiv.org/abs/2410.02151
  27. [27]Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In Proceedings of the 27th international conference on international conference on machine learning, pages 399–406, 2010.
  28. [28]Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, and Jun Zhu. Dpot: Auto-regressive denoising operator transformer for large-scale pde pre-training. arXiv preprint arXiv:2403.03542, 2024.arxiv.org/abs/2403.03542
  29. [29]Daniel Henry. Geometric theory of semilinear parabolic equations. Springer, 2006.
  30. [30]Maximilian Herde, Bogdan Raonić, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmanuel De Bezenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for pdes. Advances in Neural Information Processing Systems, 37:72525–72624, 2024.arxiv.org/abs/2405.19101
  31. [31]Lukas Herrmann, Christoph Schwab, and Jakob Zech. Neural and spectral operator surrogates: unified construction and expression rate bounds. Advances in Computational Mathematics, 50(4):72, 2024.arxiv.org/abs/2207.04950
  32. [32]Marlis Hochbruck and Alexander Ostermann. Exponential integrators. Acta Numerica, 19:209–286, 2010.DOI
  33. [33]Martin Hutzenthaler, Arnulf Jentzen, Thomas Kruse, and Tuan Anh Nguyen. A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations. SN partial differential equations and applications, 1(2):10, 2020.
  34. [34]Pengzhan Jin, Shuai Meng, and Lu Lu. Mionet: Learning multiple-input operators via tensor product. SIAM Journal on Scientific Computing, 44(6):A3490–A3514, 2022.arxiv.org/abs/2202.06137
  35. [35]Nikola Kovachki, Samuel Lanthaler, and Siddhartha Mishra. On universal approximation and error bounds for fourier neural operators. Journal of Machine Learning Research, 22(290):1–76, 2021.arxiv.org/abs/2107.07562
  36. [36]Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadnesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1–97, 2023.
  37. [37]Nikola B Kovachki, Samuel Lanthaler, and Hrushikesh Mhaskar. Data complexity estimates for operator learning. arXiv preprint arXiv:2405.15992, 2024.arxiv.org/abs/2405.15992
  38. [38]Samuel Lanthaler. Operator learning with pca-net: upper and lower complexity bounds. Journal of Machine Learning Research, 24(318):1–67, 2023.arxiv.org/abs/2303.16317
  39. [39]Samuel Lanthaler. Operator learning of lipschitz operators: An information-theoretic perspective. arXiv preprint arXiv:2406.18794, 2024.arxiv.org/abs/2406.18794
  40. [40]Samuel Lanthaler, Siddhartha Mishra, and George E Karniadakis. Error estimates for deeponets: A deep learning framework in infinite dimensions. Transactions of Mathematics and its Applications, 6(1), 2022.arxiv.org/abs/2102.09618
  41. [41]Samuel Lanthaler, Roberto Molinaro, Patrik Hadorn, and Siddhartha Mishra. Nonlinear reconstruction for operator learning of pdes with discontinuities. arXiv preprint arXiv:2210.01074, 2022.arxiv.org/abs/2210.01074
  42. [42]Samuel Lanthaler and Andrew M Stuart. The parametric complexity of operator learning. IMA Journal of Numerical Analysis, 46(2):647–712, 2026.arxiv.org/abs/2306.15924
  43. [43]Samuel Lanthaler, Andrew M Stuart, and Margaret Trautner. Discretization error of fourier neural operators. arXiv preprint arXiv:2405.02221, 2024.arxiv.org/abs/2405.02221
  44. [44]J Douglas Lawson. Generalized runge-kutta processes for stable systems with large lipschitz constants. SIAM Journal on Numerical Analysis, 4(3):372–380, 1967.DOI
  45. [45]Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.arxiv.org/abs/2010.08895
  46. [46]J-L Lions. Quelques méthodes de résolution des problèmes aux limites non-linéaires. Dunod, 1969.
  47. [47]Hao Liu, Zecheng Zhang, Wenjing Liao, and Hayden Schaeffer. Neural scaling laws of deep relu and deep operator network: A theoretical study. arXiv preprint arXiv:2410.00357, 2024.arxiv.org/abs/2410.00357
  48. [48]Yuxuan Liu, Jingmin Sun, Xinjie He, Griffin Pinney, Zecheng Zhang, and Hayden Schaeffer. Prose-fd: A multimodal pdf foundation model for learning multiple operators for forecasting fluid dynamics. arXiv preprint arXiv:2409.09811, 2024.arxiv.org/abs/2409.09811
  49. [49]Yuxuan Liu, Jingmin Sun, and Hayden Schaeffer. Boat: A block causal transformer for pde foundation models for fluid dynamics. arXiv preprint arXiv:2501.18972, 2025.arxiv.org/abs/2501.18972
  50. [50]Yuxuan Liu, Zecheng Zhang, and Hayden Schaeffer. Prose: Predicting multiple operators and symbolic expressions using multimodal transformers. Neural Networks, 180:106707, 2024.DOI
  51. [51]Gabriel J Lord, Catherine E Powell, and Tony Shardlow. An introduction to computational stochastic pdes. Cambridge Texts Appl. Math, 50, 2014.DOI
  52. [52]Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218–229, 2021.
  53. [53]Alessandra Lunardi. Interpolation theory, volume 16. Edizioni della normale, Pisa, 2018.
  54. [54]Carlo Marcati and Christoph Schwab. Exponential convergence of deep operator networks for elliptic partial differential equations. SIAM Journal on Numerical Analysis, 61(3):1513–1545, 2023.DOI
  55. [55]Tanya Marwah, Zachary Lipton, and Andrej Risteski. Parametric complexity bounds for approximating pdes with neural networks. Advances in Neural Information Processing Systems, 34:15044–15055, 2021.arxiv.org/abs/2103.02138
  56. [56]Tanya Marwah, Zachary Chase Lipton, Jianfeng Lu, and Andrej Risteski. Neural network approximations of pdes beyond linearity: A representational perspective. In international conference on machine learning, pages 24139–24172. PMLR, 2023.arxiv.org/abs/2210.12101
  57. [57]Tanya Marwah, Ashwini Pokle, Jico Kolter, Zachary Lipton, Jianfeng Lu, and Andrej Risteski. Deep equilibrium based neural operators for steady-state pdes. Advances in Neural Information Processing Systems, 36:15716–15737, 2023.arxiv.org/abs/2312.00234
  58. [58]Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawczyk, Francois Lanusse, et al. Multiple physics pretraining for spatiotemporal surrogate models. Advances in Neural Information Processing Systems, 37:119­301–119335, 2024.DOI
  59. [59]Vishal Monga, Yuelong Li, and Yonina C Eldar. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Processing Magazine, 38(2):18–44, 2021.arxiv.org/abs/1912.10557
  60. [60]Ariel Neufeld and Tuan Anh Nguyen. Rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of gradient-dependent semilinear heat equations. arXiv preprint arXiv:2403.09200, 2024.arxiv.org/abs/2403.09200
  61. [61]Joost AA Opschoor, Christoph Schwab, and Jakob Zech. Exponential relu dnn expression of holomorphic maps in high dimension. Constructive Approximation, 55(1):537–582, 2022.DOI
  62. [62]Amnon Pazy. Semigroups of linear operators and applications to partial differential equations. Springer Science & Business Media, 2012.
  63. [63]Philipp Petersen and Felix Voigtlaender. Optimal approximation of piecewise smooth functions using deep relu neural networks. Neural Networks, 108:296–330, 2018.DOI
  64. [64]Tomaso Poggio and Maia Fraser. Compositional sparsity of learnable functions. Bulletin of the American Mathematical Society, 61(3), 2024.DOI
  65. [65]Niklas Reinhardt, Sven Wang, and Jakob Zech. Statistical learning theory for neural operators. Journal of Machine Learning Research, 27(102):1–82, 2026.
  66. [66]Thomas Runst and Winfried Sickel. Sobolev spaces of fractional order, Nemytskii operators, and nonlinear partial differential equations, volume 3. Walter de Gruyter, 2011.
  67. [67]Hayden Schaeffer. Learning partial differential equations via data discovery and sparse optimization. Proceedings. Mathematical, physical, and engineering sciences, 473(2197):20160446, 2017.DOI
  68. [68]Johannes Schmidt-Hieber. Nonparametric regression using deep neural networks with ReLU activation function. Annals of Statistics, 48:1875–1897, 2020.
  69. [69]Christoph Schwab, Andreas Stein, and Jakob Zech. Deep operator network approximation rates for lipschitz operators. Analysis and Applications, 24(01):199–239, 2026.arxiv.org/abs/2307.09835
  70. [70]Christoph Schwab and Jakob Zech. Deep learning in high dimension: Neural network expression rates for generalized polynomial chaos expansions in uq. Analysis and Applications, 17(01):19–55, 2019.DOI
  71. [71]Jonathan W Siegel. Sharp lower bounds on the manifold widths of sobolev and besov spaces. Journal of Complexity, 85:101884, 2024.arxiv.org/abs/2402.04407
  72. [72]Shashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji, Dmitriy Morozov, Michael W Mahoney, and Amir Gholami. Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior. Advances in neural information processing systems, 36:71242–71262, 2023.arxiv.org/abs/2306.00258
  73. [73]Jingmin Sun, Yuxuan Liu, Zecheng Zhang, and Hayden Schaeffer. Towards a foundation model for partial differential equations: Multioperator learning and extrapolation. Physical Review E, 111(3):035304, 2025.DOI
  74. [74]Yifan Sun, Linan Zhang, and Hayden Schaeffer. Neupde: Neural network based ordinary and partial differential equations for modeling time-dependent data. In Mathematical and Scientific Machine Learning, pages 352–372. PMLR, 2020.arxiv.org/abs/1908.03190
  75. [75]Koichi Taniguchi and Sho Sonoda. Generalization error bounds for picard-type operator learning in nonlinear parabolic pdes. arXiv preprint arXiv:2605.10277, 2026.arxiv.org/abs/2605.10277
  76. [76]Roger Temam. Infinite-dimensional dynamical systems in mechanics and physics. Springer Science & Business Media, 2012.
  77. [77]Roger Temam. Navier–Stokes equations: theory and numerical analysis, volume 343. American Mathematical Society, 2024.
  78. [78]Vidar Thomée. Galerkin finite element methods for parabolic problems. Springer Science & Business Media, 2007.
  79. [79]Alex Townsend, Marcus Webb, and Sheehan Olver. Fast polynomial transforms based on toeplitz and hankel matrices. Mathematics of Computation, 87(312):1913–1934, 2018.arxiv.org/abs/1604.07486
  80. [80]Adrien Weihs, Chunyang Liao, Jingmin Sun, and Hayden Schaeffer. Kernel methods for learning operators with multiple inputs and outputs. arXiv preprint arXiv:2608.11831, 2026.arxiv.org/abs/2608.11831
  81. [81]Adrien Weihs and Hayden Schaeffer. Generalization bounds and statistical guarantees for multi-task and multiple operator learning with mno networks. arXiv preprint arXiv:2604.01961, 2026.arxiv.org/abs/2604.01961
  82. [82]Adrien Weihs and Hayden Schaeffer. Multiple neural operators achieve near-optimal rates for multi-task learning. arXiv preprint arXiv:2605.22724, 2026.arxiv.org/abs/2605.22724
  83. [83]Adrien Weihs, Jingmin Sun, Zecheng Zhang, and Hayden Schaeffer. A deep learning framework for multi-operator learning: Architectures and approximation theory. Applied and Computational Harmonic Analysis, page 101925, 2026.arxiv.org/abs/2510.25379
  84. [84]Hermann Weyl. Das asymptotische verteilungsgetsetz der eigenwerte linearer partieller differentialgleichungen (mit einer anwendung auf die theorie der hohlraumstrahlung). Mathematische Annalen, 71(4):441–479, 1912.DOI
  85. [85]Jia-Qi Yang and Lei Shi. Efficient approximation for encoder–decoder neural operators via variation spaces. arXiv preprint arXiv:2606.01244, 2026.arxiv.org/abs/2606.01244
  86. [86]Dmitry Yarotsky. Error bounds for approximations with deep relu networks. Neural networks, 94:103–114, 2017.DOI
  87. [87]Dmitry Yarotsky. Optimal approximation of continuous functions by very deep relu networks. In Conference on learning theory, pages 639–649. PMLR, 2018.arxiv.org/abs/1802.03620
  88. [88]Dmitry Yarotsky. Elementary superexpressive activations. In International conference on machine learning, pages 11932–11940. PMLR, 2021.arxiv.org/abs/2102.10911
  89. [89]Zecheng Zhang, Wing Tat Leung, and Hayden Schaeffer. A discretization-invariant extension and analysis of some deep operator networks. Journal of Computational and Applied Mathematics, 456:116226, 2025.
  90. [90]Zecheng Zhang, Christian Moya, Lu Lu, Guang Lin, and Hayden Schaeffer. D2no: Efficient handling of heterogeneous input function spaces with distributed deep neural operators. Computer Methods in Applied Mechanics and Engineering, 428:117084, 2024.arxiv.org/abs/2310.18888
  91. [91]Zecheng Zhang, Leung Wing Tat, and Hayden Schaeffer. Belnet: Basis enhanced learning, a mesh-free neural operator. Proceedings of the Royal Society A, 479(2276):20230043, 2023.arxiv.org/abs/2212.07336

Paper details

vlnc:W_ms8a5jfg
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents