betaFeedback

WorksMathematics

Parameter Bounds and Inverse Approximation for Linearized ReLUk^k Networks on the Sphere

Abstract

We study direct and inverse approximation for linearized shallow ReLUk^k networks on Sd\mathbb S^d with quasi-uniform hidden parameters. For 0<α<d+2k+12d0<α<\frac{d+2k+1}{2d}, under the coefficient growth condition (n∑j=1naj,n2)12≤nαM\bigl(n\sum_{j=1}^na_{j,n}^2\bigr)^{\frac{1}{2}}\leq n^αM, we prove that approximation at the order n−d+2k+12d+αn^{-\frac{d+2k+1}{2d}+α} is equivalent to membership in the parity-compatible Besov space B2,∞,∗d+2k+12−αd(Sd)\mathcal B_{2,\infty,*}^{\frac{d+2k+1}{2}-αd}(\mathbb S^d). Mere convergence under the same coefficient budget does not imply this regularity. For unconstrained approximation along a nested parameter sequence, a single ridge function gives a counterexample when α<12α<\frac{1}{2}, whereas for α>12α>\frac{1}{2} the endpoint Besov inverse conclusion follows from a Bernstein inequality under antipodal quasi-uniformity. The inverse conclusion is also recovered when the prescribed rate holds for two quasi-uniform parameter sequences whose union is antipodally separated.

Introduction

The approximation theory of shallow neural networks seeks to quantify how the approximation error depends on the network width, the activation function, the regularity of the target, and the size of the representing coefficients. For Barron and variation-type spaces, dimension-independent rates are commonly obtained through probabilistic or greedy discretization arguments [20, 2, 3, 12, 1, 21, 40, 42, 41, 27]. For Sobolev classes, the rate instead reflects both smoothness and dimension [29, 37, 38]. These direct estimates give sufficient conditions for attaining a prescribed rate. The converse question—what regularity is forced by that rate—is subtler and is closely related to Bernstein inequalities, inverse approximation theorems, and the structure of neural-network approximation classes [42, 43, 17, 14, 4, 18].

We focus on the ReLUk^{k} activation σk(t)=(max⁡{0,t})k\sigma_k(t) = (\max\{0,t\})^k, k∈N0k \in\mathbb{N}_0, with the usual Heaviside convention when k=0k = 0, and on linearized shallow networks, in which the hidden parameters are fixed and only the output coefficients vary. By the positive homogeneity of σk\sigma_k, the parameter variables may be normalized to directions on the unit sphere. We consider the approximation problem on

Sd={η∈Rd+1:∣η∣=1},S^{d}=\{\eta\in\mathbb{R}^{d+1}:|\eta|=1\},

where the basic ridge functions take the form σk(θ⋅η)\sigma_k(\theta\cdot\eta) with θ,η∈Sd\theta,\eta\in\mathbb{S}^d.

The spherical setting is useful both as a normalized model for affine ReLUk^k networks and as an approximation problem in its own right. Spherical neural networks arise naturally for data with directional or rotational structure [7, 19, 9], and their approximation and generalization properties have been studied in [15, 16, 44]. Harmonic analysis on the sphere also provides localized polynomial kernels, cubature formulas, and stable discretizations on quasi-uniform point sets [28, 30, 31, 32, 33, 34]. These tools connect the geometry of the hidden parameters with approximation accuracy and coefficient stability.

Let Θn={θj,n∗}j=1n⊂Sd\Theta_n=\{\theta_{j,n}^{*}\}_{j=1}^n\subset\mathbb{S}^d. For α≥0\alpha\geq0 and M>0M>0, define

Lnk,α(Θn)={g∈L2(Sd):g(η)=∑j=1naj,nσk(θj,n∗⋅η), (n∑j=1naj,n2)12≤nαM}.(1)L_n^{k,\alpha}(\Theta_n)=\left\{g\in L^2(\mathbb{S}^d):g(\eta)=\sum_{j=1}^n a_{j,n}\sigma_k(\theta_{j,n}^{*}\cdot\eta),\ \left(n\sum_{j=1}^n a_{j,n}^2\right)^{\frac{1}{2}}\leq n^\alpha M\right\}. \tag*{(1)}

We write Lnk(Θn)L_n^k(\Theta_n) when no constraint is imposed on the coefficients. The constant MM is always independent of nn; in a direct approximation statement for a given ff, it may depend on ff. Schwartz inequality shows the coefficient bound of the Barron space [43] is consistent with the constraint in (1) when α=0\alpha=0:

∑j=1n∣aj,n∣≤(n∑j=1naj,n2)12≤M\sum_{j=1}^n |a_{j,n}|\leq\left(n\sum_{j=1}^n a_{j,n}^2\right)^{\frac{1}{2}}\leq M

The identity

σk(θ⋅η)+(−1)kσk(−θ⋅η)=(θ⋅η)k\sigma_k(\theta\cdot\eta)+(-1)^k\sigma_k(-\theta\cdot\eta)=(\theta\cdot\eta)^k

shows that every element of Lnk(Θn)L_n^k(\Theta_n) has a prescribed parity up to a spherical polynomial of degree at most kk. Therefore, for a function space S(Sd)\mathcal{S}(\mathbb{S}^d) containing Pk(Sd)\mathcal{P}_k(\mathbb{S}^d), we write

S∗(Sd):={f∈S(Sd):Πmf=0 for every m>k with m−k even}.\mathcal{S}_*(\mathbb{S}^d):=\left\{f\in\mathcal{S}(\mathbb{S}^d):\Pi_mf=0\text{ for every }m>k\text{ with }m-k\text{ even}\right\}.

Here Πm\Pi_m denotes the spherical harmonic projection introduced below. Equivalently, up to Pk(Sd)\mathcal{P}_k(\mathbb{S}^d), S∗(Sd)\mathcal{S}_*(\mathbb{S}^d) consists of even functions when kk is odd and odd functions when kk is even.

A sharp direct approximation result was established in [23]. For a quasi-uniform parameter sequence {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} and 0≤α≤d+2k+12d0\leq\alpha\leq\frac{d+2k+1}{2d}, one has, with MM controlled by the Sobolev norm of ff,

inf⁡fn∈Lnk,α(Θn)∥f−fn∥L2(Sd)≲n−d+2k+12d+α∥f∥H∗d+2k+12−dα(Sd).(2)\inf_{f_n\in L_n^{k,\alpha}(\Theta_n)}\|f-f_n\|_{L^2(\mathbb{S}^d)}\lesssim n^{-\frac{d+2k+1}{2d}+\alpha}\|f\|_{\mathcal{H}_*^{\frac{d+2k+1}{2}-d\alpha}(\mathbb{S}^d)}. \tag*{(2)}

The endpoint d+2k+12\frac{d+2k+1}{2} is determined by the decay of the spherical harmonic coefficients of a ReLUk^k ridge function.

Two converse results provide the immediate background for this work. First, the integral characterization in [23] shows that if, for some quasi-uniform sequence {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty},

lim⁡n→∞inf⁡fn∈Lnk,α(Θn)∥f−fn∥L2(Sd)=0,(3)\lim_{n\to\infty}\inf_{f_n\in L_n^{k,\alpha}(\Theta_n)}\|f-f_n\|_{L^2(\mathbb{S}^d)}=0, \tag*{(3)}

then f∈H∗d+2k+12(Sd)f\in\mathcal{H}_*^{\frac{d+2k+1}{2}}(\mathbb{S}^d). Second, the Bernstein inequality established in [24] gives an inverse theorem for the unconstrained spaces. If {Θn}n≥1\{\Theta_n\}_{n\geq1} is nested and antipodally quasi-uniform and 12<α<d+2k+12d\frac{1}{2}<\alpha<\frac{d+2k+1}{2d}, then

inf⁡fn∈Lnk(Θn)∥f−fn∥L2(Sd)≲n−12−2k+12d+α.(4)\inf_{f_n\in L_n^k(\Theta_n)}\|f-f_n\|_{L^2(\mathbb{S}^d)}\lesssim n^{-\frac{1}{2}-\frac{2k+1}{2d}+\alpha}. \tag*{(4)}

implies f∈H∗r(Sd)f \in H_*^r(\mathbb{S}^d) for every 0≤r<d+2k+12−αd0 \le r < \frac{d+2k+1}{2}-\alpha d. We strengthen these subcritical Sobolev conclusions below to the endpoint Besov space.

Direct and inverse estimates of this kind are widespread in classical approximation theory: together they determine whether a rate is sharp and whether improved convergence forces additional regularity of the target [5, 36, 13, 45, 25]. One of the key concepts in this theory is the notion of an approximation space (see [10]) corresponding to a sequence of linear spaces. Specifically, let Vn⊂L2(Sd)V_n \subset L^2(\mathbb{S}^d) be a linear space of dimension nn. For a given decay rate s>0s > 0, the approximation spaces with respect to the sequence of linear spaces VnV_n is defined by the semi-norm

∣f∣As({Vn}n=1∞):=sup⁡n≥1nsE(f,Vn),(5)|f|_{\mathcal{A}_s(\{V_n\}_{n=1}^{\infty})} := \sup_{n\ge1} n^s E(f,V_n), \tag*{(5)}

where

E(f,Vn)=inf⁡fn∈Vn∥f−fn∥L2(Sd)(6)E(f,V_n) = \inf_{f_n\in V_n} \|f-f_n\|_{L^2(\mathbb{S}^d)} \tag*{(6)}

is the error of best approximation by the space VnV_n.

A useful classical comparison to our results is provided by the setting of linear approximation by splines. The simplest case of this is linear approximation by piecewise constants, which is detailed in Section 3.1 in [10], for instance. In this theory, the approximation spaces for a sequence of (non-nested) piecewise constant linear spaces VnV_n are shown to be Besov spaces, but only for decay rates up to n−sn^{-s} with s<1s < 1 (see [11], Chapter 7, Section 7). This is due to the fact that higher-order smoothness cannot be captured by piecewise constants, and indeed for higher order decay rates the approximation spaces cannot be described in a simple form.

When we consider the linear spaces Vn:=Lnk(Θn)V_n := L_n^k(\Theta_n) for a sequence of quasi-uniform points Θn\Theta_n, one might expect that the same characterization of the approximation spaces holds. Our results show that this is indeed the case, up to approximation order s<12+2k+12ds < \frac{1}{2}+\frac{2k+1}{2d}, provided that we additionally constrain the coefficients of the approximant as in the definition (1.1), i.e., we consider

E(f,Ln,Mk,α(Θn)):=inf⁡fn∈Ln,Mk,α(Θn)∥f−fn∥L2(Sd),(7)E(f,L_{n,M}^{k,\alpha}(\Theta_n)) := \inf_{f_n\in L_{n,M}^{k,\alpha}(\Theta_n)} \|f-f_n\|_{L^2(\mathbb{S}^d)}, \tag*{(7)}

instead of E(f,Vn)E(f,V_n) for the linear spaces Vn=Lnk(Θn)V_n=L_n^k(\Theta_n). This additional constraint is required since the ReLUk^k ridge functions are globally supported and highly correlated, in contrast to the spline basis. Motivated by this comparison, we address the following two questions:

  1. Which function classes can be approximated when a prescribed growth bound is imposed on the coefficients?

  1. Which regularity properties can be inferred from a prescribed approximation rate, with or without an explicit coefficient bound?

For α≥0\alpha\ge0, the natural coefficient and approximation scales considered here are

(n∑j=1naj,n2)12≤nαM,∥f−∑j=1naj,nσk(θj∗⋅∘)∥L2(Sd)≲n−12−2k+12d+α.(8)\left(n\sum_{j=1}^{n}a_{j,n}^{2}\right)^{\frac{1}{2}} \le n^\alpha M,\qquad\left\|f-\sum_{j=1}^{n}a_{j,n}\sigma_k(\theta_j^* \cdot\circ)\right\|_{L^2(\mathbb{S}^d)} \lesssim n^{-\frac{1}{2}-\frac{2k+1}{2d}+\alpha}. \tag*{(8)}

For α=0\alpha=0, these are the coefficient scale and endpoint rate in (1.2). Allowing the coefficient norm to grow like nαn^\alpha weakens the corresponding regularity and approximation order by the same factor.

Our first result identifies the exact approximation space at this scale. For 0<α<d+2k+12d0 < \alpha< \frac{d+2k+1}{2d} and every quasi-uniform sequence {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty},

f∈B2,∞,∗d+2k+12d−α(Sd)⟺ there exists M>0 such that inf⁡fn∈Ln,Mk(Θn)∥f−fn∥L2(Sd)≲n−d+2k+12d+α.(9)f \in B_{2,\infty,*}^{\frac{d+2k+1}{2d}-\alpha}(\mathbb{S}^{d}) \Longleftrightarrow\text{ there exists } M > 0 \text{ such that } \inf_{f_n \in L_{n,M}^{k}(\Theta_n)} \lVert f-f_n\rVert_{L^2(\mathbb{S}^{d})} \lesssim n^{-\frac{d+2k+1}{2d}+\alpha}. \tag*{(9)}

Thus there is no loss of smoothness in the inverse conclusion. The rate assumption remains essential: when α>0\alpha> 0, mere convergence by approximants satisfying the same coefficient growth bound does not imply membership in B2,∞,∗d+2k+12d−α(Sd)B_{2,\infty,*}^{\frac{d+2k+1}{2d}-\alpha}(\mathbb{S}^{d}). This contrasts with the endpoint case α=0\alpha= 0 in (3).

We next remove the coefficient constraint and retain only the approximation rate. The resulting picture is more delicate. If 0<α<120 < \alpha< \frac{1}{2} and {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} is nested, the estimate

inf⁡fn∈Lnk(Θn)∥f−fn∥L2(Sd)≲n−d+2k+12d+α(10)\inf_{f_n \in L_n^k(\Theta_n)} \lVert f-f_n\rVert_{L^2(\mathbb{S}^{d})} \lesssim n^{-\frac{d+2k+1}{2d}+\alpha} \tag*{(10)}

does not imply that f∈B2,∞,∗d+2k+12d−α(Sd)f \in B_{2,\infty,*}^{\frac{d+2k+1}{2d}-\alpha}(\mathbb{S}^{d}). Indeed, nestedness preserves at least one ridge function at all levels, and a single ridge has exact Besov smoothness k+12k+\frac{1}{2}. If α>12\alpha> \frac{1}{2} and the nested sequence is antipodally quasi-uniform, the Bernstein inequality yields the converse implication at the endpoint order d+2k+12d−α\frac{d+2k+1}{2d}-\alpha.

The inverse conclusion can nevertheless be recovered from two geometrically independent parameter sequences. Suppose the same target satisfies (10) for quasi-uniform sequences {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} and {Θ~n}n=1∞\{\widetilde{\Theta}_n\}_{n=1}^{\infty}, and that Θn∪Θ~n\Theta_n \cup\widetilde{\Theta}_n is antipodally separated at scale n−1/dn^{-1/d}. Subtracting approximants on the two parameter sets and applying a coefficient lower bound on their union yields

∑j=1naj,n2≲n2α−1.\sum_{j=1}^{n} a_{j,n}^{2} \lesssim n^{2\alpha-1}.

Thus the coefficient growth required above is recovered from the two approximation estimates themselves, and the Besov inverse conclusion follows from (9).

These results separate two pieces of information that are often conflated in neural-network approximation: the size of the approximation error and the stability of the representation producing it. A single parameter sequence may approximate a target rapidly for geometric reasons that reveal little about its smoothness. An explicit coefficient bound, or approximation on two suitably separated parameter sequences, rules out this degeneracy and restores a sharp inverse statement. The result therefore has a saturation-type interpretation once the admissible coefficient growth is fixed, but it is not a classical saturation theorem for a single fixed approximation process. This perspective complements recent work relating neural-network norms, approximation classes, Bernstein inequalities, and generalization [42, 41, 43, 27, 18].

The remainder of the paper is organized as follows. Section 2 states the main theorem. Section 3 introduces the spherical harmonic notation, the ReLUk^k expansion, and the quadrature formula used in the analysis. Section 4 contains the proofs.

Main Results

Theorem 2.1. Let d∈Nd \in\mathbb{N}, k∈N0k \in\mathbb{N}_0, 0<α<d+2k+12d0 < \alpha< \frac{d+2k+1}{2d}, and let {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} be a sequence of quasi-uniform collections on Sd\mathbb{S}^{d}.

  1. Let f∈L2(Sd)f \in L^2(\mathbb{S}^d), then f∈B2,∞,∗d+2k+12−αd(Sd)f \in B_{2,\infty,*}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d) if and only if there exists MM independent of nn, such that

inf⁡fn∈Ln,Mk,α(Θn)∥f−fn∥L2(Sd)≲n−d+2k+12d+α.(11)\inf_{f_n\in L_{n,M}^{k,\alpha}(\Theta_n)} \lVert f-f_n\rVert_{L^2(\mathbb{S}^d)} \lesssim n^{-\frac{d+2k+1}{2d}+\alpha}. \tag*{(11)}
  1. For every M>0M>0 and f∈L∗2(Sd)f\in L_*^2(\mathbb{S}^d),

lim⁡n→∞inf⁡fn∈Ln,Mk,α(Θn)∥f−fn∥L2(Sd)=0.(12)\lim_{n\to\infty}\inf_{f_n\in L_{n,M}^{k,\alpha}(\Theta_n)}\lVert f-f_n\rVert_{L^2(\mathbb{S}^d)}=0. \tag*{(12)}
  1. Suppose further that {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} is antipodally quasi-uniform and nested. If α>12\alpha>\frac{1}{2} and f∈L2(Sd)f\in L^2(\mathbb{S}^d) satisfies

inf⁡fn∈Lnk(Θn)∥f−fn∥L2(Sd)≲n−d+2k+12d+α,(13)\inf_{f_n\in L_n^k(\Theta_n)}\lVert f-f_n\rVert_{L^2(\mathbb{S}^d)}\lesssim n^{-\frac{d+2k+1}{2d}+\alpha}, \tag*{(13)}

then it follows that f∈B2,∞,∗d+2k+12−αd(Sd)f\in B_{2,\infty,*}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d). On the other hand, for every α≤1/2\alpha\leq1/2, this characterization fails, i.e., there exists an f∉B2,∞,∗d+2k+12−αd(Sd)f\notin B_{2,\infty,*}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d) which satisfies (13).

  1. Let f∈L2(Sd)f\in L^2(\mathbb{S}^d), and suppose there exists another sequence of antipodally quasi-uniform collections {Θ~n}n=1∞\{\widetilde{\Theta}_n\}_{n=1}^{\infty}, with Θ~n={θ~j,n∗}j=1n\widetilde{\Theta}_n=\{\widetilde{\theta}_{j,n}^*\}_{j=1}^n, such that #(Θn∪Θ~n)=2n\#(\Theta_n\cup\widetilde{\Theta}_n)=2n,

min⁡ξ,ζ∈Θn∪Θ~nξ≠ζ{ρ(ξ,ζ),ρ(ξ,−ζ)}≳n−1d,(14)\min_{\substack{\xi,\zeta\in\Theta_n\cup\widetilde{\Theta}_n\\ \xi\ne\zeta}}\{\rho(\xi,\zeta),\rho(\xi,-\zeta)\}\gtrsim n^{-\frac{1}{d}}, \tag*{(14)}

and

inf⁡fn∈Lnk(Θn)∥f−fn∥L2(Sd)+inf⁡f~n∈Lnk(Θ~n)∥f−f~n∥L2(Sd)≲n−d+2k+12d+α.(15)\inf_{f_n\in L_n^k(\Theta_n)}\lVert f-f_n\rVert_{L^2(\mathbb{S}^d)}+\inf_{\widetilde{f}_n\in L_n^k(\widetilde{\Theta}_n)}\lVert f-\widetilde{f}_n\rVert_{L^2(\mathbb{S}^d)}\lesssim n^{-\frac{d+2k+1}{2d}+\alpha}. \tag*{(15)}

Then f∈B2,∞,∗d+2k+12−αd(Sd)f\in B_{2,\infty,*}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d).

Preliminaries

Throughout, dηd\eta denotes the normalized rotation-invariant measure on Sd\mathbb{S}^d. We write Pm(Sd)\mathcal{P}_m(\mathbb{S}^d) for the restrictions to Sd\mathbb{S}^d of polynomials on Rd+1\mathbb{R}^{d+1} of total degree at most mm.

Spherical harmonics, Besov spaces, and Sobolev spaces

For m∈N0m\in\mathbb{N}_0, let Ym\mathcal{Y}_m be the space of spherical harmonics of degree mm, and fix an orthonormal basis {Ym,ℓ}ℓ=1N(m)\{Y_{m,\ell}\}_{\ell=1}^{N(m)}. Its dimension is

N(m):=dim⁡(Ym)=dim⁡(Pm(Sd))−dim⁡(Pm−1(Sd))={(2m+d−1)Γ(m+d−1)Γ(m+1)Γ(d),m≠0,1,m=0.(16)N(m):=\dim(\mathcal{Y}_m)=\dim(\mathcal{P}_m(\mathbb{S}^d))-\dim(\mathcal{P}_{m-1}(\mathbb{S}^d))= \begin{cases} (2m+d-1)\dfrac{\Gamma(m+d-1)}{\Gamma(m+1)\Gamma(d)}, & m\ne0,\\ 1, & m=0. \end{cases} \tag*{(16)}

The orthogonal decomposition into spherical harmonics gives, for every f∈L2(Sd)f\in L^2(\mathbb{S}^d),

f(η)=∑m=0∞∑ℓ=1N(m)f^(m,ℓ)Ym,ℓ(η),a.e. η∈Sd,(17)f(\eta)=\sum_{m=0}^{\infty}\sum_{\ell=1}^{N(m)}\widehat{f}(m,\ell)Y_{m,\ell}(\eta),\qquad\text{a.e. }\eta\in\mathbb{S}^d, \tag*{(17)}

where

f^(m,ℓ)=⟨f,Ym,ℓ⟩L2(Sd),m∈N0,1≤ℓ≤N(m).\widehat{f}(m,\ell)=\langle f,Y_{m,\ell}\rangle_{L^2(\mathbb{S}^d)},\qquad m\in\mathbb{N}_0,\quad1\leq\ell\leq N(m).

Let Πm\Pi_m be the L2(Sd)L^2(\mathbb{S}^d)-orthogonal projection onto Ym\mathcal{Y}_m, and for J∈N0J\in\mathbb{N}_0 set

PJf:=∑m=0JΠmf,PJcf:=f−PJf,(18)\mathcal{P}_Jf:=\sum_{m=0}^{J}\Pi_mf,\qquad\mathcal{P}_J^cf:=f-\mathcal{P}_Jf, \tag*{(18)}

as well as

Δ0f:=P1f,Δqf:=(P2q−P2q−1)f,q≥1.(19)\Delta_0f:=\mathcal{P}_1f,\qquad\Delta_qf:=(\mathcal{P}_{2^q}-\mathcal{P}_{2^{q-1}})f,\qquad q\geq1. \tag*{(19)}

Definition 3.1 (Besov and Sobolev spaces). For γ>0\gamma>0, define

B2,∞γ(Sd):={f∈L2(Sd):∥f∥B2,∞γ(Sd)<∞},B_{2,\infty}^{\gamma}(\mathbb{S}^d):=\left\{f\in L^2(\mathbb{S}^d):\|f\|_{B_{2,\infty}^{\gamma}(\mathbb{S}^d)}<\infty\right\},

where

∥f∥B2,∞γ(Sd):=∥Δ0f∥L2(Sd)+sup⁡q≥12qγ∥Δqf∥L2(Sd).(20)\|f\|_{B_{2,\infty}^{\gamma}(\mathbb{S}^d)}:=\|\Delta_0f\|_{L^2(\mathbb{S}^d)}+\sup_{q\geq1}2^{q\gamma}\|\Delta_qf\|_{L^2(\mathbb{S}^d)}. \tag*{(20)}

For γ≥0\gamma\geq0, the Sobolev space Hγ(Sd)H^\gamma(\mathbb{S}^d) is defined by

∥f∥Hγ(Sd)2:=∑m=0∞(1+m)2γ∑ℓ=1N(m)∣f^(m,ℓ)∣2.(21)\|f\|_{H^\gamma(\mathbb{S}^d)}^2:=\sum_{m=0}^{\infty}(1+m)^{2\gamma}\sum_{\ell=1}^{N(m)}|\widehat{f}(m,\ell)|^2. \tag*{(21)}

For γ=0\gamma=0, this is the norm on L2(Sd)L^2(\mathbb{S}^d).

Quasi-uniform parameter collections

Let ρ\rho be the geodesic distance on Sd\mathbb{S}^d. For Θn={θj,n∗}j=1n⊂Sd\Theta_n=\{\theta^*_{j,n}\}_{j=1}^n\subset\mathbb{S}^d, define its fill distance, separation distance, and antipodal separation distance by

hn:=max⁡η∈Sdmin⁡1≤j≤nρ(η,θj,n∗),h~n:=min⁡i≠jρ(θi,n∗,θj,n∗),h_n:=\max_{\eta\in\mathbb{S}^d}\min_{1\leq j\leq n}\rho(\eta,\theta^*_{j,n}),\qquad\widetilde{h}_n:=\min_{i\neq j}\rho(\theta^*_{i,n},\theta^*_{j,n}),

and

h^n:=min⁡{min⁡i≠jρ(θi,n∗,θj,n∗),min⁡i≠jρ(−θi,n∗,θj,n∗)}.\widehat{h}_n:=\min\left\{\min_{i\neq j}\rho(\theta^*_{i,n},\theta^*_{j,n}),\min_{i\neq j}\rho(-\theta^*_{i,n},\theta^*_{j,n})\right\}.

The collection Θn\Theta_n is called quasi-uniform if hn≲h~nh_n\lesssim\widetilde{h}_n, and antipodally quasi-uniform if hn≲h^nh_n\lesssim\widehat{h}_n. A sequence {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} has either property when the corresponding constant is uniform in nn, and it is called nested if Θn⊂Θn+1\Theta_n\subset\Theta_{n+1} for every nn. In particular,

hn≃h~n≃n−1/dh_n\simeq\widetilde{h}_n\simeq n^{-1/d}

for a quasi-uniform sequence, while

hn≃h^n≃n−1/dh_n\simeq\widehat{h}_n\simeq n^{-1/d}

in the antipodally quasi-uniform case. When one value of nn is fixed, we write h=hnh=h_n.

Ultraspherical expansion of σk\sigma_k

Let Lwd2([−1,1])\mathcal{L}^2_{w_d}([-1,1]) be the weighted space with

⟨f,g⟩wd=∫−11f(t)g(t)(1−t2)d−22 dt,∥f∥Lwd2([−1,1])=⟨f,f⟩wd12.\langle f,g\rangle_{w_d}=\int_{-1}^{1}f(t)g(t)(1-t^2)^{\frac{d-2}{2}}\,dt,\qquad\|f\|_{L^2_{w_d}([-1,1])}=\langle f,f\rangle_{w_d}^{\frac{1}{2}}.

The ultraspherical polynomials {pm}m=0∞\{p_m\}_{m=0}^{\infty} form an orthogonal basis of this space and satisfy

⟨pm,pℓ⟩wd=0,m≠ℓ,m,ℓ∈N0.(22)\langle p_m,p_\ell\rangle_{w_d}=0,\qquad m\ne\ell,\quad m,\ell\in\mathbb{N}_0. \tag*{(22)}

We use the normalization for which the spherical addition formula reads

pm(η⋅θ)=∑ℓ=1N(m)Ym,ℓ(η)Ym,ℓ(θ);(23)p_m(\eta\cdot\theta)=\sum_{\ell=1}^{N(m)}Y_{m,\ell}(\eta)Y_{m,\ell}(\theta); \tag*{(23)}

see [8 Theorem 1.2.6]. Here ωd\omega_d denotes the surface area of Sd\mathbb{S}^d. The activation σk∈Lwd2([−1,1])\sigma_k\in L^2_{w_d}([-1,1]) admits the expansion

σk=∑m=0∞σ^k(m)pm,(24)\sigma_k=\sum_{m=0}^{\infty}\widehat{\sigma}_k(m)p_m, \tag*{(24)}

where

σ^k(m)=⟨pm,σk⟩wd∥pm∥Lwd2([−1,1])2.\widehat{\sigma}_k(m)=\frac{\langle p_m,\sigma_k\rangle_{w_d}}{\|p_m\|_{L^2_{w_d}([-1,1])}^2}.

Following [39, 6, 32, 1], the nonzero coefficients are indexed by

Eσk={m≥k+1:m−k odd}∪{0,…,k}.E_{\sigma_k}=\{m\ge k+1:m-k\ \text{odd}\}\cup\{0,\ldots,k\}.

For m≥k+1m\ge k+1 with m−km-k odd, they are given explicitly by

σ^k(m)=ωd−1k!Γ(d2)ωd(−1)m−k−12Γ(m−k)2mΓ(m−k+12)Γ(m+d+k+12).(25)\widehat{\sigma}_k(m)=\frac{\omega_{d-1}k!\Gamma\left(\frac{d}{2}\right)}{\omega_d}\frac{(-1)^{\frac{m-k-1}{2}}\Gamma(m-k)}{2^m\Gamma\left(\frac{m-k+1}{2}\right)\Gamma\left(\frac{m+d+k+1}{2}\right)}. \tag*{(25)}

Then a neuron on Sd\mathbb{S}^d has an expansion

σk(θ⋅η)=∑m∈Eσkσ^k(m)∑ℓ=1N(m)Ym,ℓ(θ)Ym,ℓ(η).(26)\sigma_k(\theta\cdot\eta)=\sum_{m\in E_{\sigma_k}}\widehat{\sigma}_k(m)\sum_{\ell=1}^{N(m)}Y_{m,\ell}(\theta)Y_{m,\ell}(\eta). \tag*{(26)}

Related ultraspherical expansions of ReLU-type zonal functions appear in [33].

Polynomial quadrature on quasi-uniform collections

We shall use the following positive quadrature rule.

Lemma 3.1 ([28 Theorem 4.1], [35 Theorem 3.2]). There exist nonnegative weights τ1,n,…,τn,n\tau_{1,n},\ldots,\tau_{n,n} satisfying τj,n≲hd\tau_{j,n}\lesssim h^d, and a constant c1c_1 independent of nn and hh, such that

∫Sdp(η)q(η) dη=∑j=1nτj,np(θj,n∗)q(θj,n∗),∀p,q∈PJ(Sd),(27)\int_{\mathbb{S}^d}p(\eta)q(\eta)\,d\eta=\sum_{j=1}^{n}\tau_{j,n}p(\theta_{j,n}^{*})q(\theta_{j,n}^{*}),\qquad\forall p,q\in\mathbb{P}_{J}(\mathbb{S}^d), \tag*{(27)}

where J=⌊c1h−1⌋J=\lfloor c_1h^{-1}\rfloor. Moreover, if

min⁡i≠jρ(θi,n∗,θj,n∗)≳h,(28)\min_{i\ne j}\rho(\theta_{i,n}^{*},\theta_{j,n}^{*})\gtrsim h, \tag*{(28)}

then there exists c2>0c_2>0 such that (3.12) holds for J≤c2n1dJ\le c_2n^{\frac{1}{d}} and

τj,n≃hd≃n−1,j=1,…,n.(29)\tau_{j,n}\simeq h^d\simeq n^{-1},\qquad j=1,\ldots,n. \tag*{(29)}

The constants are independent of nn and may depend only on the dimension and the uniform quasi-uniformity constants.

We shall use the endpoint case of the direct theorem in [23] in the following form.

Theorem 3.1 (Theorem 4.1, [23]). Let d≥2d\ge2, k∈N0k\in\mathbb{N}_0, and {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} be quasi-uniform. For every f∈H∗d+2k+12(Sd)f\in\mathcal{H}_{*}^{\frac{d+2k+1}{2}}(\mathbb{S}^d), there is

fn=∑j=1naj,nσk(θj,n∗⋅∘)f_n=\sum_{j=1}^{n}a_{j,n}\sigma_k(\theta_{j,n}^{*}\cdot\circ)

such that

∥f−fn∥L2(Sd)≲n−d+2k+12d∥f∥Hd+2k+12(Sd),(n∑j=1naj,n2)12≲∥f∥Hd+2k+12(Sd).(30)\|f-f_n\|_{L^2(\mathbb{S}^d)}\lesssim n^{-\frac{d+2k+1}{2d}}\|f\|_{\mathcal{H}^{\frac{d+2k+1}{2}}(\mathbb{S}^d)},\qquad\left(n\sum_{j=1}^{n}a_{j,n}^2\right)^{\frac{1}{2}}\lesssim\|f\|_{\mathcal{H}^{\frac{d+2k+1}{2}}(\mathbb{S}^d)}. \tag*{(30)}

The theorem is the endpoint specialization of the spherical direct approximation theorem in [23]. Although its parity condition is commonly stated for the pure even or odd component, the same construction covers the additional Pk(Sd)\mathbb{P}_k(\mathbb{S}^d) component: all degrees 0≤m≤k0\le m\le k belong to EσkE_{\sigma_k} and are reproduced by the quadrature formula.

We also record the stability estimate used in items 3 and 4. Its Bernstein part is the estimate proved in [24]; the same localization argument also gives the coefficient lower bound. We include the short common argument so that the latter is not left implicit.

Theorem 3.2 (Theorem 1, [24]; Theorem 1, [22]). Let d≥2d\ge2, k∈N0k\in\mathbb{N}_0, and {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} be antipodally quasi-uniform. Then

inf⁡an∈Rn∥an∥2=1∥∑j=1naj,nσk(θj,n∗⋅∘)∥L2(Sd)≃n−2k+12d.(31)\inf_{\substack{a_n\in\mathbb{R}^n\\ \|a_n\|_2=1}}\left\|\sum_{j=1}^{n}a_{j,n}\sigma_k(\theta_{j,n}^{*}\cdot\circ)\right\|_{L^2(\mathbb{S}^d)}\simeq n^{-\frac{2k+1}{2d}}. \tag*{(31)}

Moreover, let 0≤s<r<k+120\le s<r<k+\frac{1}{2}, then for fn∈Lnk(Θn)f_n\in L_n^k(\Theta_n),

∥fn∥Hr(Sd)≲nr−sd∥fn∥Hs(Sd),(32)\|f_n\|_{\mathcal{H}^r(\mathbb{S}^d)}\lesssim n^{\frac{r-s}{d}}\|f_n\|_{\mathcal{H}^s(\mathbb{S}^d)}, \tag*{(32)}

where the implied constant is independent of nn, Θn\Theta_n, and fnf_n.

Proof. We write

∥∑j=1naj,nσk(θj,n∗⋅∘)∥L2(Sd)2=∫Sd(∑j=1naj,nσk(θj,n∗⋅η))2dη=∫SdanT(σk(θi,n∗⋅η)σk(θj,n∗⋅η))i,j=1nandη=anTMnan,\begin{aligned} \left\|\sum_{j=1}^{n}a_{j,n}\sigma_k(\theta_{j,n}^{*}\cdot\circ)\right\|_{L^2(\mathbb{S}^d)}^2 = \int_{\mathbb{S}^d}\left(\sum_{j=1}^{n}a_{j,n}\sigma_k(\theta_{j,n}^{*}\cdot\eta)\right)^2\mathrm{d}\eta\\ &= \int_{\mathbb{S}^d}a_n^{\mathsf{T}}\left(\sigma_k(\theta_{i,n}^{*}\cdot\eta)\sigma_k(\theta_{j,n}^{*}\cdot\eta)\right)_{i,j=1}^{n}a_n\mathrm{d}\eta = a_n^{\mathsf{T}}\mathcal{M}_n a_n, \end{aligned}

where

Mn=(∫Sdσk(θi,n∗⋅η)σk(θj,n∗⋅η)dη)i,j=1n.\mathcal{M}_n=\left(\int_{\mathbb{S}^d}\sigma_k(\theta_{i,n}^{*}\cdot\eta)\sigma_k(\theta_{j,n}^{*}\cdot\eta)\mathrm{d}\eta\right)_{i,j=1}^{n}.

By [22 Theorem 1],

λmin⁡(Mn)≃n−2k+1d,\lambda_{\min}(\mathcal{M}_n)\simeq n^{-\frac{2k+1}{d}},

which proves (3.16).

(3.17) is exactly the result in [24].

Proofs of the main theorem

Throughout this section, c2c_2 and τj,n\tau_{j,n} are as in Lemma 3.1. In particular, τj,n≃n−1\tau_{j,n}\simeq n^{-1}.

Proof of item 1

Direct implication. Let f∈B2,∞,∗d+2k+12−αd(Sd)f\in B_{2,\infty,*}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d). Choose Q=Q(n)Q=Q(n) so that 2Q≤n1d<2Q+12^Q\le n^{\frac{1}{d}}<2^{Q+1} and put gn:=P2Qfg_n:=P_{2^Q}f. The dyadic definition of the Besov norm and the orthogonality of the blocks give

∥f−gn∥L2(Sd)≲2−Q(d+2k+12−αd)∥f∥B2,∞d+2k+12−αd(Sd),∥gn∥Hd+2k+12(Sd)≲2Qαd∥f∥B2,∞d+2k+12−αd(Sd).\|f-g_n\|_{L^2(\mathbb{S}^d)}\lesssim2^{-Q\left(\frac{d+2k+1}{2}-\alpha d\right)}\|f\|_{B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d)},\qquad \|g_n\|_{\mathcal{H}^{\frac{d+2k+1}{2}}(\mathbb{S}^d)}\lesssim2^{Q\alpha d}\|f\|_{B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d)}.

Indeed, the first estimate follows by summing the squared high-frequency blocks, while the second follows from

∥gn∥Hd+2k+122≲∑q=0Q2q(d+2k+1)∥Δqf∥22≲∥f∥B2,∞d+2k+12−αd2∑q=0Q22q(αd)≲22Qαd∥f∥B2,∞d+2k+12−αd2,\|g_n\|_{\mathcal{H}^{\frac{d+2k+1}{2}}}^2\lesssim\sum_{q=0}^{Q}2^{q(d+2k+1)}\|\Delta_qf\|_2^2\lesssim\|f\|_{B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}}^2\sum_{q=0}^{Q}2^{2q(\alpha d)}\lesssim2^{2Q\alpha d}\|f\|_{B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}}^2,

where αd>0\alpha d>0. Apply Theorem 3.1 to gng_n. The resulting network, denoted by fnf_n, satisfies

(n∑j=1naj,n2)12≲∥gn∥Hd+2k+12≲nα∥f∥B2,∞d+2k+12−αd,\left(n\sum_{j=1}^{n}a_{j,n}^2\right)^{\frac{1}{2}}\lesssim\|g_n\|_{\mathcal{H}^{\frac{d+2k+1}{2}}}\lesssim n^{\alpha}\|f\|_{B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}},

and hence belongs to Ln,Mk,α(Θn)L_{n,M}^{k,\alpha}(\Theta_n) for an MM independent of nn. Moreover,

∥f−fn∥2≤∥f−gn∥2+∥gn−fn∥2≲2−Q(d+2k+12−αd)∥f∥B2,∞d+2k+12−αd+n−d+2k+12d2Qαd∥f∥B2,∞d+2k+12−αd\|f-f_n\|_2\le\|f-g_n\|_2+\|g_n-f_n\|_2\lesssim2^{-Q\left(\frac{d+2k+1}{2}-\alpha d\right)}\|f\|_{B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}}+n^{-\frac{d+2k+1}{2d}}2^{Q\alpha d}\|f\|_{B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}}
≲n−d+2k+12d+α∥f∥B2,∞d+2k+12−αd.\lesssim n^{-\frac{d+2k+1}{2d}+\alpha}\|f\|_{B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}}.

Inverse implication. For each nn, let J=⌊c2n1d⌋J=\lfloor c2n^{\frac{1}{d}}\rfloor. For 0≤m≤J0\le m\le J and 1≤ℓ≤N(m)1\le\ell\le N(m), define

Vn(m,ℓ):=(τ1,nYm,ℓ(θ1,n∗),…,τn,nYm,ℓ(θn,n∗))T.(33)V_n(m,\ell):=\left(\sqrt{\tau_{1,n}}Y_{m,\ell}(\theta_{1,n}^{*}),\ldots,\sqrt{\tau_{n,n}}Y_{m,\ell}(\theta_{n,n}^{*})\right)^{\mathsf{T}}. \tag*{(33)}

Then

Vn(m,ℓ)TVn(m′,ℓ′)=δ(m,ℓ),(m′,ℓ′),0≤m,m′≤J,(34)V_n(m,\ell)^{\mathsf{T}}V_n(m',\ell')=\delta_{(m,\ell),(m',\ell')},\qquad0\le m,m'\le J, \tag*{(34)}

by applying (3.12) to Ym,ℓYm′,ℓ′Y_{m,\ell}Y_{m',\ell'}.

For a representation fn=∑j=1naj,nσk(θj,n∗⋅)f_n=\sum_{j=1}^{n}a_{j,n}\sigma_k(\theta_{j,n}^{*}\cdot) satisfying the defining constraint of Ln,Mk,αL_{n,M}^{k,\alpha}, set

yn:=(a1,nτ1,n,…,an,nτn,n)T,v^n(m,ℓ):=ynTVn(m,ℓ).(35)y_n:=\left(\frac{a_{1,n}}{\sqrt{\tau_{1,n}}},\ldots,\frac{a_{n,n}}{\sqrt{\tau_{n,n}}}\right)^{\mathsf{T}},\qquad\widehat{v}_n(m,\ell):=y_n^{\mathsf{T}}V_n(m,\ell). \tag*{(35)}

For m≤Jm\le J, the addition formula gives

f^n(m,ℓ)=σ^k(m)v^n(m,ℓ).(36)\widehat{f}_n(m,\ell)=\widehat{\sigma}_k(m)\widehat{v}_n(m,\ell). \tag*{(36)}

Moreover, by (4.3), Bessel’s inequality, and τj,n≃n−1\tau_{j,n}\simeq n^{-1},

∑m=0J∑ℓ=1N(m)∣v^n(m,ℓ)∣2≤∥yn∥22=∑j=1naj,n2τj,n≲n∑j=1naj,n2.(37)\sum_{m=0}^{J}\sum_{\ell=1}^{N(m)}|\widehat{v}_n(m,\ell)|^2\le\|y_n\|_2^2=\sum_{j=1}^{n}\frac{a_{j,n}^2}{\tau_{j,n}}\lesssim n\sum_{j=1}^{n}a_{j,n}^2. \tag*{(37)}

For every sufficiently large qq, choose n=nq:=⌈C2qd⌉n=n_q:=\lceil C2^{qd}\rceil, where the fixed constant CC is large enough that 2q≤⌊c2nq/d⌋=J2^q\le\lfloor c2^{n_q/d}\rfloor=J. Thus nq≃2qdn_q\simeq2^{qd}. By the assumed approximation estimate, there exists fn∈Ln,Mk,αf_n\in L_{n,M}^{k,\alpha} such that

∥f−fn∥L2(Sd)≲n−d+2k+12d+α≲2−q(d+2k+12−αd).(38)\|f-f_n\|_{L^2(\mathbb{S}^d)}\lesssim n^{-\frac{d+2k+1}{2d}+\alpha}\lesssim2^{-q\left(\frac{d+2k+1}{2}-\alpha d\right)}. \tag*{(38)}

The explicit formula (3.10) and the Gamma-ratio asymptotics imply

∣σ^k(m)∣≲m−d+2k+12,m>k,(39)|\widehat{\sigma}_k(m)|\lesssim m^{-\frac{d+2k+1}{2}},\qquad m>k, \tag*{(39)}

where σ^k(m)=0\widehat{\sigma}_k(m)=0 for m>km>k with m−km-k even. Hence, for all sufficiently large qq, (4.5) and (4.6) give

∥(P2q−P2q−1)fn∥L2(Sd)2=∑m=2q−1+12qσ^k(m)2∑ℓ=1N(m)∣v^n(m,ℓ)∣2≤max⁡2q−1<m≤2q∣σ^k(m)∣2∥yn∥22≲2−q(d+2k+1)n2α≲2−2q(d+2k+12−αd).(40)\begin{aligned} \|(P_{2^q}-P_{2^{q-1}})f_n\|_{L^2(\mathbb{S}^d)}^2 &=\sum_{m=2^{q-1}+1}^{2^q}\widehat{\sigma}_k(m)^2\sum_{\ell=1}^{N(m)}|\widehat{v}_n(m,\ell)|^2\\ &\le\max_{2^{q-1}<m\le2^q}|\widehat{\sigma}_k(m)|^2\|y_n\|_2^2\\ &\lesssim2^{-q(d+2k+1)}n^{2\alpha}\lesssim2^{-2q\left(\frac{d+2k+1}{2}-\alpha d\right)}. \tag*{(40)} \end{aligned}

Since P2q−P2q−1P_{2^q}-P_{2^{q-1}} is an orthogonal projection, (4.7) and (4.9) yield

∥(P2q−P2q−1)f∥L2(Sd)2≤2∥(P2q−P2q−1)fn∥L2(Sd)2+2∥(P2q−P2q−1)(f−fn)∥L2(Sd)2≲2−2q(d+2k+12−αd).(41)\begin{aligned} \|(P_{2^q}-P_{2^{q-1}})f\|_{L^2(\mathbb{S}^d)}^2 &\le2\|(P_{2^q}-P_{2^{q-1}})f_n\|_{L^2(\mathbb{S}^d)}^2+2\|(P_{2^q}-P_{2^{q-1}})(f-f_n)\|_{L^2(\mathbb{S}^d)}^2\\ &\lesssim2^{-2q\left(\frac{d+2k+1}{2}-\alpha d\right)}. \tag*{(41)} \end{aligned}

Equation (4.10) and the finitely many low-frequency blocks give

sup⁡q≥02q(d+2k+12−αd)∥Δqf∥L2(Sd)<∞,\sup_{q\ge0}2^{q\left(\frac{d+2k+1}{2}-\alpha d\right)}\|\Delta_qf\|_{L^2(\mathbb{S}^d)}<\infty,

so f∈B2,∞d+2k+12−αd(Sd)f \in B^{\frac{d+2k+1}{2}-\alpha d}_{2,\infty}(S^d). It remains to verify the parity condition. For every m>km>k with m−km-k even, one has σ^k(m)=0\hat{\sigma}_k(m)=0, so Πmfn=0\Pi_m f_n=0. Since d+2k+12−αd>0\frac{d+2k+1}{2}-\alpha d>0, (4.7) implies fn→ff_n \to f in L2(Sd)L^2(S^d) along the above choice of nn. Consequently, Πmf=0\Pi_m f=0 for every such mm, and hence f∈B2,∞,∗d+2k+12−αd(Sd)f \in B^{\frac{d+2k+1}{2}-\alpha d}_{2,\infty,*}(S^d).

Proof of item 2

Fix arbitrary ϵ>0\epsilon>0 and fix LL such that

∥f−ΠLf∥L2(Sd)≤ϵ.(42)\|f-\Pi_Lf\|_{L^2(S^d)} \le\epsilon. \tag*{(42)}

By Theorem 3.1, for each n∈Nn\in\mathbb{N}, there exists M0>0M_0>0 and an,1,…,an,na_{n,1},\ldots,a_{n,n} with

n∑j=1naj,n2≤M0∥ΠLf∥Hd+2k+12(Sd)n\sum_{j=1}^{n}a_{j,n}^2 \le M_0\|\Pi_Lf\|_{\mathcal{H}^{\frac{d+2k+1}{2}}(S^d)}

such that

∥fn−ΠLf∥L2(Sd)≲∥ΠLf∥Hd+2k+12(Sd)n−d+2k+12d.\|f_n-\Pi_Lf\|_{L^2(S^d)} \lesssim\|\Pi_Lf\|_{\mathcal{H}^{\frac{d+2k+1}{2}}(S^d)}n^{-\frac{d+2k+1}{2d}}.

Let nn be sufficiently large such that

Mnα>M0∥ΠLf∥Hd+2k+12(Sd),∥ΠLf∥Hd+2k+12(Sd)n−d+2k+12d≤ϵ,Mn^\alpha>M_0\|\Pi_Lf\|_{\mathcal{H}^{\frac{d+2k+1}{2}}(S^d)},\qquad\|\Pi_Lf\|_{\mathcal{H}^{\frac{d+2k+1}{2}}(S^d)}n^{-\frac{d+2k+1}{2d}}\le\epsilon,

then fn∈Ln,Mk,α(Θn)f_n\in L^{k,\alpha}_{n,M}(\Theta_n) and

∥f−fn∥L2(Sd)≤∥f−ΠLf∥L2(Sd)+∥fn−ΠLf∥L2(Sd)≤2ϵ.(43)\|f-f_n\|_{L^2(S^d)}\le\|f-\Pi_Lf\|_{L^2(S^d)}+\|f_n-\Pi_Lf\|_{L^2(S^d)}\le2\epsilon. \tag*{(43)}

Proof of item 3

Case 1: α<12\alpha<\frac{1}{2}. Since {Θn}n=1∞\{\Theta_n\}_{n=1}^{\infty} is nested, we may fix θ1,1∗∈Θ1\theta^*_{1,1}\in\Theta_1, so that θ1,1∗∈Θn\theta^*_{1,1}\in\Theta_n for every nn. Let

f(η):=σk(θ1,1∗⋅η).f(\eta):=\sigma_k(\theta^*_{1,1}\cdot\eta).

Then f∈Lnk(Θn)f\in L^k_n(\Theta_n) for every nn, so the left-hand side of (2.3) is identically zero. It remains to verify the claimed lack of regularity. By (3.8), (3.9), and (3.10), for m≥k+1m\ge k+1 with m−km-k odd,

∥Πmf∥L2(Sd)2=σ^k(m)2∑ℓ=1N(m)∣Ym,ℓ(θ1,1∗)∣2=σ^k(m)2N(m)≍m−2k−2.(44)\|\Pi_mf\|_{L^2(S^d)}^2=\hat{\sigma}_k(m)^2\sum_{\ell=1}^{N(m)}|Y_{m,\ell}(\theta^*_{1,1})|^2=\hat{\sigma}_k(m)^2N(m)\asymp m^{-2k-2}. \tag*{(44)}

Consequently, for every sufficiently large qq,

∥Δqf∥L2(Sd)2≍∑2q−1<m≤2qm−k oddm−2k−2≍2−q(2k+1).\|\Delta_qf\|_{L^2(S^d)}^2\asymp\sum_{\substack{2^{q-1}<m\le2^q\\m-k\ \mathrm{odd}}}m^{-2k-2}\asymp2^{-q(2k+1)}.

Therefore,

2q(d+2k+12−αd)∥Δqf∥L2(Sd)≍2q(d+2k+12−αd−k−12)=2qd(12−α)⟶∞.2^{q\left(\frac{d+2k+1}{2}-\alpha d\right)}\|\Delta_qf\|_{L^2(S^d)} \asymp2^{q\left(\frac{d+2k+1}{2}-\alpha d-k-\frac{1}{2}\right)} =2^{qd\left(\frac{1}{2}-\alpha\right)}\longrightarrow\infty.

The expansion of ff has the required parity above degree kk, and hence f∈L∗2(Sd)∖B2,∞,∗d+2k+12−αd(Sd)f\in L^2_*(S^d)\setminus B^{\frac{d+2k+1}{2}-\alpha d}_{2,\infty,*}(S^d).

Case 2: α>12\alpha> \frac{1}{2}. Let f∈L2(Sd)f \in L^2(\mathbb{S}^d) satisfy (2.3), for q≥0q \ge0, let nq=2(q+1)dn_q = 2^{(q+1)d} and choose vq∈Lnqk(Θnq)v_q \in L_{n_q}^k(\Theta_{n_q}) such that

∥f−vq∥L2(Sd)≲2−q(d+2k+12−αd).(45)\lVert f-v_q\rVert_{L^2(\mathbb{S}^d)} \lesssim2^{-q\left(\frac{d+2k+1}{2}-\alpha d\right)}. \tag*{(45)}

Set u0:=v0u_0 := v_0 and uq:=vq−vq−1u_q := v_q-v_{q-1} for q≥1q \ge1. By nestedness, uq∈Lnqk(Θnq)u_q \in L_{n_q}^k(\Theta_{n_q}), and (45) gives

∥uq∥L2(Sd)≲2−q(d+2k+12−αd)\lVert u_q\rVert_{L^2(\mathbb{S}^d)} \lesssim2^{-q\left(\frac{d+2k+1}{2}-\alpha d\right)}

after adjusting the first term. Take r∈(d+2k+12−αd,k+12)r \in\left(\frac{d+2k+1}{2}-\alpha d,k+\frac{1}{2}\right), then the antipodal quasi-uniformity assumption and (3.17) yield

∥uq∥Hr(Sd)≲nqrd∥uq∥L2(Sd)≲2q(r−d+2k+12+αd).(46)\lVert u_q\rVert_{H^r(\mathbb{S}^d)} \lesssim n_q^{\frac{r}{d}}\lVert u_q\rVert_{L^2(\mathbb{S}^d)} \lesssim2^{q\left(r-\frac{d+2k+1}{2}+\alpha d\right)}. \tag*{(46)}

As f=∑q=0∞uqf=\sum_{q=0}^{\infty}u_q in L2(Sd)L^2(\mathbb{S}^d), for Q∈NQ \in\mathbb{N}, equations (45)–(46) give

∥ΔQf∥L2(Sd)≤∑q<Q∥ΔQuq∥L2(Sd)+∑q≥Q∥uq∥L2(Sd)≲2−Qr∑q<Q∥ΔQuq∥Hr(Sd)+∑q≥Q∥uq∥L2(Sd)≲2−Qr∑q<Q2q(r−d+2k+12+αd)+∑q≥Q2−q(d+2k+12−αd)≲2−Q(d+2k+12−αd).(47)\begin{aligned} \lVert\Delta_Qf\rVert_{L^2(\mathbb{S}^d)} &\leq\sum_{q<Q}\lVert\Delta_Qu_q\rVert_{L^2(\mathbb{S}^d)}+\sum_{q\geq Q}\lVert u_q\rVert_{L^2(\mathbb{S}^d)} \\ &\lesssim2^{-Qr}\sum_{q<Q}\lVert\Delta_Qu_q\rVert_{H^r(\mathbb{S}^d)}+\sum_{q\geq Q}\lVert u_q\rVert_{L^2(\mathbb{S}^d)} \\ &\lesssim2^{-Qr}\sum_{q<Q}2^{q\left(r-\frac{d+2k+1}{2}+\alpha d\right)}+\sum_{q\geq Q}2^{-q\left(\frac{d+2k+1}{2}-\alpha d\right)} \\ &\lesssim2^{-Q\left(\frac{d+2k+1}{2}-\alpha d\right)}. \tag*{(47)} \end{aligned}

Hence f∈B2,∞d+2k+12−αd(Sd)f \in B_{2,\infty}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d). Finally, vq→fv_q \to f in L2L^2, and every vqv_q has zero projection onto Ym\mathcal{Y}_m whenever m>km>k and m−km-k is even. Therefore f∈B2,∞,∗d+2k+12−αd(Sd)f \in B_{2,\infty,*}^{\frac{d+2k+1}{2}-\alpha d}(\mathbb{S}^d).

Case 3: α=12\alpha=\frac{1}{2}. By nestedness, after relabeling the points if necessary, we may write

Θn={θ1∗,…,θn∗},n=1,2,….\Theta_n=\{\theta_1^*,\ldots,\theta_n^*\},\qquad n=1,2,\ldots.

Since dim⁡PJ(Sd)≍Jd\dim\mathbb{P}_J(\mathbb{S}^d)\asymp J^d, we may choose s0s_0 sufficiently large and integers JsJ_s, s≥s0s\ge s_0, such that

Js+2∈Eσk,Js≍2s+1d,dim⁡PJs(Sd)≤2s−1,(48)J_s+2\in E_{\sigma_k},\qquad J_s\asymp2^{\frac{s+1}{d}},\qquad\dim\mathbb{P}_{J_s}(\mathbb{S}^d)\leq2^{s-1}, \tag*{(48)}

and Js+2≤c22s+1dJ_s+2\leq c_2 2^{\frac{s+1}{d}}, which lies below the quadrature degree associated with Θ2s+1\Theta_{2^{s+1}}.

For each s≥s0s\geq s_0, we follow the notations in [22 (4.28)–(4.34)] and denote the orthogonal complement of the orthonormal vectors in (4.3):

V(Js):=span⁡{V2s+1(m,ℓ):m∈Eσk, m≤Js, 1≤ℓ≤N(m)}⊥.\mathcal{V}(J_s):=\operatorname{span}\{V_{2^{s+1}}(m,\ell):m\in E_{\sigma_k},\ m\leq J_s,\ 1\leq\ell\leq N(m)\}^{\perp}.

By (4.3) and (48),

dim⁡(V(Js)∩{v∈R2s+1:v1=⋯=v2s=0})≥2s−dim⁡PJs(Sd)≥2s−1.\dim\left(\mathcal{V}(J_s)\cap\left\{v\in\mathbb{R}^{2^{s+1}}:v_1=\cdots=v_{2^s}=0\right\}\right)\geq2^s-\dim\mathbb{P}_{J_s}(\mathbb{S}^d)\geq2^{s-1}.

We may therefore choose a unit vector

v(s)=(v1(s),…,v2s+1(s))⊤∈V(Js),v1(s)=⋯=v2s(s)=0.v^{(s)}=\left(v_1^{(s)},\ldots,v_{2^{s+1}}^{(s)}\right)^\top\in\mathcal{V}(J_s),\qquad v_1^{(s)}=\cdots=v_{2^s}^{(s)}=0.

Thus the coefficients of the corresponding ridge combination are supported precisely on

Θ2s+1∖Θ2s={θ2s+1∗,…,θ2s+1∗}.\Theta_{2^{s+1}}\setminus\Theta_{2^s}=\{\theta_{2^s+1}^*,\ldots,\theta_{2^{s+1}}^*\}.

Let

Ws:=diag⁡(τ1,2s+1,…,τ2s+1,2s+1)W_s := \operatorname{diag}(\tau_{1,2^{s+1}},\ldots,\tau_{2^{s+1},2^{s+1}})

and MsM_s be the Gram matrix of the ridge functions associated with Θ2s+1\Theta_{2^{s+1}}. Using h2s+1≃Js−1h_{2^{s+1}}\simeq J_s^{-1}, the one-sided estimate proved in [22 (4.34)] gives

(v(s))⊤Ws1/2MsWs1/2v(s)≲Js−(d+2k+1).(49)\left(v^{(s)}\right)^\top W_s^{1/2}M_sW_s^{1/2}v^{(s)}\lesssim J_s^{-(d+2k+1)}. \tag*{(49)}

Set

ai(s):=τi,2s+1 vi(s),2s<i≤2s+1,a_i^{(s)}:=\sqrt{\tau_{i,2^{s+1}}}\,v_i^{(s)},\qquad2^s<i\leq2^{s+1},

then τi,2s+1≃2−(s+1)\tau_{i,2^{s+1}}\simeq2^{-(s+1)} and ∥v(s)∥2=1\|v^{(s)}\|_2=1 yields

∥ai(s)∥22≃2−(s+1).(50)\|a_i^{(s)}\|_2^2\simeq2^{-(s+1)}. \tag*{(50)}

Denote

gs:=2s+12∑i=2s+12s+1ai(s)σk(θi∗⋅).g_s:=2^{\frac{s+1}{2}}\sum_{i=2^s+1}^{2^{s+1}}a_i^{(s)}\sigma_k(\theta_i^*\cdot).

Then for m∈Eσkm\in E_{\sigma_k}, m≤Jsm\leq J_s and 1≤ℓ≤N(m)1\leq\ell\leq N(m), (4.5) and v(s)∈V(Js)v^{(s)}\in\mathcal{V}(J_s) give

g^s(m,ℓ)=2s+12σ^k(m)∑i=2s+12s+1ai(s)Ym,ℓ(θi∗)=2s+12σ^k(m)(v(s))⊤V2s+1(m,ℓ)=0.\widehat{g}_s(m,\ell)=2^{\frac{s+1}{2}}\widehat{\sigma}_k(m)\sum_{i=2^s+1}^{2^{s+1}}a_i^{(s)}Y_{m,\ell}(\theta_i^*)=2^{\frac{s+1}{2}}\widehat{\sigma}_k(m)\left(v^{(s)}\right)^\top V_{2^{s+1}}(m,\ell)=0.

Recall σ^k(m)\widehat{\sigma}_k(m) is supported in EσkE_{\sigma_k},

PJsgs=0.(51)\mathcal{P}_{J_s}g_s=0. \tag*{(51)}

Moreover, (4.18) yields

∥gs∥22=2s+1(v(s))⊤Ws1/2MsWs1/2v(s)≲2s+1Js−(d+2k+1)≲Js−(2k+1).(52)\|g_s\|_2^2=2^{s+1}\left(v^{(s)}\right)^\top W_s^{1/2}M_sW_s^{1/2}v^{(s)}\lesssim2^{s+1}J_s^{-(d+2k+1)}\lesssim J_s^{-(2k+1)}. \tag*{(52)}

Define

f:=∑s=s0∞gs,Fs:=∑t=s0sgt.f:=\sum_{s=s_0}^{\infty}g_s,\qquad F_s:=\sum_{t=s_0}^{s}g_t.

By (4.21), then f∈L2(Sd)f\in L^2(\mathbb{S}^d) and by nestedness, Fs∈L2s+1k(Θ2s+1)F_s\in L_{2^{s+1}}^k(\Theta_{2^{s+1}}). Now for 2s+1≤n<2s+22^{s+1}\leq n<2^{s+2}, nestedness and (4.21) yield (2.3):

inf⁡vn∈Lnk(Θn)∥f−vn∥2≤∥f−Fs∥2≤∑t>s∥gt∥2≲∑t>s2−(t+1)(2k+1)2d≲2−(s+2)(2k+1)2d≲n−2k+12d.\inf_{v_n\in L_n^k(\Theta_n)}\|f-v_n\|_2\leq\|f-F_s\|_2\leq\sum_{t>s}\|g_t\|_2\lesssim\sum_{t>s}2^{-\frac{(t+1)(2k+1)}{2d}}\lesssim2^{-\frac{(s+2)(2k+1)}{2d}}\lesssim n^{-\frac{2k+1}{2d}}.

It remains to prove that f∉B2,∞r(Sd)f\notin B_{2,\infty}^r(\mathbb{S}^d). When FsF_s is represented over Θ2s+1\Theta_{2^{s+1}}, the disjointness of the coefficient supports and (4.19) imply

∑t=s0s∑i=2t+12t+12t+1∣ai(t)∣2≃s−s0+1≃s.\sum_{t=s_0}^{s}\sum_{i=2^t+1}^{2^{t+1}}2^{t+1}|a_i^{(t)}|^2\simeq s-s_0+1\simeq s.

Let qs=⌈log⁡2(C3h2s+1)⌉q_s=\left\lceil\log_2\left(\frac{C_3}{h_{2s+1}}\right)\right\rceil, applying [26 Lemma 2.1] therefore gives

∥(P2qs+2−P2qs)Fs∥2≳2−qs(k+12)s.(53)\left\|(P_{2q_s+2}-P_{2q_s})F_s\right\|_2\gtrsim2^{-q_s(k+\frac{1}{2})}\sqrt{s}. \tag*{(53)}

On the other hand, antipodally quasi-uniformity yields 2qs≃h−1≃2d~2^{q_s}\simeq h^{-1}\simeq2^{\tilde d}, (52) implies

∥(P2qs+2−P2qs)(f−Fs)∥2≤∥f−Fs∥2≲2−s+2d(k+12)≲2−qs(k+12).\left\|(P_{2q_s+2}-P_{2q_s})(f-F_s)\right\|_2\leq\left\|f-F_s\right\|_2\lesssim2^{-\frac{s+2}{d}(k+\frac{1}{2})}\lesssim2^{-q_s(k+\frac{1}{2})}.

Combining this with (53), we obtain

2qs(k+12)∥(P2qs+2−P2qs)f∥2≳s⟶∞,2^{q_s(k+\frac{1}{2})}\left\|(P_{2q_s+2}-P_{2q_s})f\right\|_2\gtrsim\sqrt{s}\longrightarrow\infty,

which implies f∉B2,∞r(Sd)f\notin B_{2,\infty}^r(S^d).

Finally, every gsg_s has the required parity above degree kk, and this property is preserved under convergence in L2(Sd)L^2(S^d). Therefore

f∈L∗2(Sd)∖B2,∞,∗k+12(Sd).f\in L_*^2(S^d)\setminus B_{2,\infty,*}^{k+\frac{1}{2}}(S^d).

Proof of item 4

By (15), for every nn we can choose

fn=∑j=1naj,nσk(θj,n∗⋅∘),f~n=∑j=1na~j,nσk(θ~j,n∗⋅∘)f_n=\sum_{j=1}^{n}a_{j,n}\sigma_k(\theta_{j,n}^*\cdot\circ),\qquad\widetilde f_n=\sum_{j=1}^{n}\widetilde a_{j,n}\sigma_k(\widetilde\theta_{j,n}^*\cdot\circ)

such that

∥f−fn∥L2(Sd)+∥f−f~n∥L2(Sd)≲n−d+2k+12d+α.(54)\left\|f-f_n\right\|_{L^2(S^d)}+\left\|f-\widetilde f_n\right\|_{L^2(S^d)}\lesssim n^{-\frac{d+2k+1}{2d}+\alpha}. \tag*{(54)}

Applying Theorem 3.2, (14), and the coefficients a1,n,…,an,n,−a~1,n,…,−a~n,na_{1,n},\ldots,a_{n,n},-\widetilde a_{1,n},\ldots,-\widetilde a_{n,n} gives

∥fn−f~n∥L2(Sd)2≳n−2k+1d(∑j=1naj,n2+∑j=1na~j,n2).(55)\left\|f_n-\widetilde f_n\right\|_{L^2(S^d)}^2\gtrsim n^{-\frac{2k+1}{d}}\left(\sum_{j=1}^{n}a_{j,n}^2+\sum_{j=1}^{n}\widetilde a_{j,n}^2\right). \tag*{(55)}

On the other hand, (54) yields

∥fn−f~n∥L2(Sd)≤∥f−fn∥L2(Sd)+∥f−f~n∥L2(Sd)≲n−d+2k+12d+α.\left\|f_n-\widetilde f_n\right\|_{L^2(S^d)}\leq\left\|f-f_n\right\|_{L^2(S^d)}+\left\|f-\widetilde f_n\right\|_{L^2(S^d)}\lesssim n^{-\frac{d+2k+1}{2d}+\alpha}.

Thus we obtain

∑j=1naj,n2+∑j=1na~j,n2≲n2α−1.(56)\sum_{j=1}^{n}a_{j,n}^2+\sum_{j=1}^{n}\widetilde a_{j,n}^2\lesssim n^{2\alpha-1}. \tag*{(56)}

and in particular,

(n∑j=1naj,n2)12≲nα.\left(n\sum_{j=1}^{n}a_{j,n}^2\right)^{\frac{1}{2}}\lesssim n^\alpha.

Therefore, for some M>0M>0 independent of nn, fn∈Ln,Mk,α(Θn)f_n\in L_{n,M}^{k,\alpha}(\Theta_n), while (54) retains the rate in item 1. The inverse implication in item 1 therefore gives f∈B2,∞,∗d+2k+12d−αd(Sd)f\in B_{2,\infty,*}^{\frac{d+2k+1}{2d}-\alpha d}(S^d), proving item 4.

References

  1. [1]F. Bach. Breaking the curse of dimensionality with convex neural networks. The Journal of Machine Learning Research, 18(1):629–681, 2017.
  2. [2]A. R. Barron. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information theory, 39(3):930–945, 1993.DOI
  3. [3]A. R. Barron. Approximation and estimation bounds for artificial neural networks. Machine learning, 14:115–133, 1994.
  4. [4]J. Bernstein, A. Vahdat, Y. Yue, and M.-Y. Liu. On the distance between two neural networks and the stability of learning. Advances in Neural Information Processing Systems, 33:21370–21381, 2020.arxiv.org/abs/2002.03432
  5. [5]S. N. Bernstein. Sur l’ordre de la meilleure approximation des fonctions continues par les polynômes de degré donné. Mem. Cl. Sci. Acad. Roy. Belg., 4:1–103, 1912.
  6. [6]J. Bourgain and J. Lindenstrauss. Projection bodies. In Geometric Aspects of Functional Analysis: Israel Seminar (GAFA) 1986–87, pages 250–270. Springer, 2006.
  7. [7]T. S. Cohen, M. Geiger, J. Köhler, and M. Welling. Spherical CNNs. In International Conference on Learning Representations, 2018.arxiv.org/abs/1801.10130
  8. [8]F. Dai and Y. Xu. Approximation theory and harmonic analysis on spheres and balls. Springer, 2013.
  9. [9]M. Defferrard, N. Perraudin, T. Kacprzak, and R. Sgier. DeepSphere: Towards an equivariant graph-based spherical CNN. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.arxiv.org/abs/1904.05146
  10. [10]R. A. DeVore. Nonlinear approximation. Acta Numerica, 7:51–150, 1998.DOI
  11. [11]R. A. DeVore and G. G. Lorentz. Constructive approximation, volume 303. Springer Science & Business Media, 1993.
  12. [12]R. A. DeVore and V. N. Temlyakov. Some remarks on greedy algorithms. Advances in computational Mathematics, 5(1):173–187, 1996.DOI
  13. [13]R. H. DeVore, Ronald A and C. Michelli. Optimal nonlinear approximation. Manuscripta mathematica, 58(3):295–306, 1989.DOI
  14. [14]W. E and B. Yu. A priori estimates and analysis of a numerical method for machine learning. Science China Mathematics, 62(11):2303–2330, 2019.
  15. [15]Z. Fang, H. Feng, S. Huang, and D.-X. Zhou. Theory of deep convolutional neural networks ii: Spherical analysis. Neural Networks, 131:154–162, 2020.arxiv.org/abs/2007.14285
  16. [16]H. Feng, S. Huang, and D.-X. Zhou. Generalization analysis of CNNs for classification on spheres. IEEE Transactions on Neural Networks and Learning Systems, 2023.
  17. [17]Z.-C. Guo and D.-X. Zhou. Optimal approximation with relu neural networks. Neural Networks, 128:96–109, 2020.
  18. [18]S. Huang, J. Zhou, H. Feng, and D.-X. Zhou. Generalization analysis of pairwise learning for ranking with deep neural networks. Neural Computation, 35(6):1135–1158, 2023.DOI
  19. [19]C. Jiang, J. Huang, K. Kashinath, Prabhat, P. Marcus, and M. Niessner. Spherical CNNs on unstructured grids. In International Conference on Learning Representations, 2019.arxiv.org/abs/1901.02039
  20. [20]L. K. Jones. A simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training. The Annals of Statistics, 20(1):608–613, 1992.DOI
  21. [21]J. M. Klusowski and A. R. Barron. Approximation by combinations of relu and squared relu ridge functions with l1 and l0 controls. IEEE Transactions on Information Theory, 64(12):7649–7656, 2018.arxiv.org/abs/1607.07819
  22. [22]X. Liu, T. Mao, and J. Xu. Condition numbers and eigenvalue spectra of shallow networks on spheres. arXiv preprint arXiv:2511.02625, 2025.arxiv.org/abs/2511.02625
  23. [23]X. Liu, T. Mao, and J. Xu. Integral representations of sobolev spaces via ReLU^k activation function and optimal error estimates for linearized networks. arXiv preprint arXiv:2505.00351, 2025.arxiv.org/abs/2505.00351
  24. [24]X. Liu, T. Mao, and J. Xu. Optimal neural network approximation via empirical least squares with deterministic samples. arXiv preprint arXiv:2608.06687, 2026.arxiv.org/abs/2608.06687
  25. [25]V. Maiorov. Pseudo-dimension and entropy of manifolds formed by affine-invariant dictionary. Advances in Computational Mathematics, 25:435–450, 2006.DOI
  26. [26]T. Mao and J. Xu. Sharp lower bounds for linearized relu^k approximation on the sphere. arXiv preprint arXiv:2510.04060, 2025.
  27. [27]H. Mhaskar and T. Mao. Tractability of approximation by general shallow networks. arXiv preprint arXiv:2308.03230, 2023.arxiv.org/abs/2308.03230
  28. [28]H. Mhaskar, F. Narcowich, and J. Ward. Spherical marcinkiewicz-zygmund inequalities and positive quadrature. Mathematics of computation, 70(235):1113–1130, 2001.DOI
  29. [29]H. N. Mhaskar. Approximation properties of a multilayered feedforward artificial neural network. Advances in Computational Mathematics, 1:61–80, 1993.DOI
  30. [30]H. N. Mhaskar. Polynomial operators and local smoothness classes on the unit interval. Journal of Approximation Theory, 131(2):243–267, 2004.DOI
  31. [31]H. N. Mhaskar. On the representation of smooth functions on the sphere using finitely many bits. Applied and Computational Harmonic Analysis, 18(3):215–233, 2005.DOI
  32. [32]H. N. Mhaskar. Weighted quadrature formulas and approximation by zonal function networks on the sphere. Journal of Complexity, 22(3):348–370, 2006.DOI
  33. [33]H. N. Mhaskar. Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions. Journal of Complexity, 51:1–19, 2019.arxiv.org/abs/1709.08174
  34. [34]H. N. Mhaskar. Kernel-based analysis of massive data. Frontiers in Applied Mathematics and Statistics, 6:30, 2020.arxiv.org/abs/2003.13226
  35. [35]F. J. Narcowich, J. D. Ward, and H. Wendland. Sobolev error estimates and a bernstein inequality for scattered data interpolation via radial basis functions. Constructive Approximation, 24:175–186, 2006.DOI
  36. [36]S. M. Nikolskii. Approximation of Functions of Several Variables and Imbedding Theorems. Springer, Berlin, 1975.DOI
  37. [37]P. P. Petrushev. Approximation by ridge functions and neural networks. SIAM Journal on Mathematical Analysis, 30(1):155–189, 1998.DOI
  38. [38]A. Pinkus. Approximation theory of the mlp model in neural networks. Acta numerica, 8:143–195, 1999.DOI
  39. [39]R. Schneider. Zu einem problem von shephard über die projektionen konvexer körper. Mathematische Zeitschrift, 101:71–82, 1967.DOI
  40. [40]J. W. Siegel and J. Xu. High-order approximation rates for shallow neural networks with cosine and ReLUk activation functions. Applied and Computational Harmonic Analysis, 58:1–26, 2022.
  41. [41]J. W. Siegel and J. Xu. Optimal convergence rates for the orthogonal greedy algorithm. IEEE Transactions on Information Theory, 68(5):3354–3361, 2022.arxiv.org/abs/2106.15000
  42. [42]J. W. Siegel and J. Xu. Sharp bounds on the approximation rates, metric entropy, and n-widths of shallow neural networks. Foundations of Computational Mathematics, pages 1–57, 2022.DOI
  43. [43]J. W. Siegel and J. Xu. Characterization of the variation spaces corresponding to shallow neural networks. Constructive Approximation, 57(3):1109–1132, 2023.arxiv.org/abs/2106.15002
  44. [44]Y. Yang, S. Huang, H. Feng, and D.-X. Zhou. Spherical analysis of learning nonlinear functionals. Journal of Machine Learning Research, 2026.arxiv.org/abs/2410.01047
  45. [45]D.-X. Zhou and K. Jetter. Approximation with polynomial kernels and svm classifiers. Advances in Computational Mathematics, 25(1):323–344, 2006.DOI

Paper details

vlnc:W_q6c5qqt2
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents