Introduction
We seek to estimate a function F F F from n n n independent observations
( X i , Y i ) , Y i = F ( X i ) + ζ i , i = 1 , … , n , (X_i,Y_i), \qquad Y_i=F(X_i)+\zeta_i, \qquad i=1,\ldots,n, ( X i , Y i ) , Y i = F ( X i ) + ζ i , i = 1 , … , n ,
where the predictors X i ∈ R D X_i\in\mathbb{R}^{D} X i ∈ R D are independent copies of a random vector X X X , the responses Y i Y_i Y i take values in R p \mathbb{R}^{p} R p , and the noise vectors ζ i \zeta_i ζ i are centered and independent of each other and of the predictors. We write ( X , Y , ζ ) (X,Y,\zeta) ( X , Y , ζ ) for a generic observation and its noise. Suppose that all we know about F F F is that it is of Hölder smoothness s s s on a full-dimensional region of R D \mathbb{R}^{D} R D . The best achievable accuracy is then governed by a simple bias–variance trade-off. A local estimator F ^ \widehat{F} F of suitable order on a neighborhood of width b b b has a squared bias of order b 2 s b^{2s} b 2 s , and its variance is inversely proportional to the number of observations in the neighborhood, which is of order n b D nb^D n b D at a typical interior point when the predictors have a locally bounded positive ambient density. Balancing the two terms gives
b 2 s ≍ 1 n b D ⟹ b ≍ n − 1 / ( 2 s + D ) , E ∥ F ^ ( X ) − F ( X ) ∥ 2 ≍ n − 2 s / ( 2 s + D ) , (1) b^{2s} \asymp\frac{1}{nb^D} \quad\Longrightarrow\quad b \asymp n^{-1/(2s+D)}, \qquad\mathbb{E}\lVert\widehat{F}(X)-F(X)\rVert^2 \asymp n^{-2s/(2s+D)}, \tag*{(1)} b 2 s ≍ n b D 1 ⟹ b ≍ n − 1/ ( 2 s + D ) , E ∥ F ( X ) − F ( X ) ∥ 2 ≍ n − 2 s / ( 2 s + D ) , (1)
or n − s / ( 2 s + D ) n^{-s/(2s+D)} n − s / ( 2 s + D ) for the root mean squared error. This is the minimax rate over the class of s s s -smooth functions ([94 , 45 , 98 ] ), and kernel, Fourier and multiscale estimators attain it ([45 , 14 , 15 , 16 ] ). The learning rate 2 s / ( 2 s + D ) 2s/(2s+D) 2 s / ( 2 s + D ) degrades quickly as D D D grows. This is an instance of the curse of dimensionality . Since the rate is minimax, it cannot be improved uniformly over all s s s -smooth functions; it can only be improved for regression problems with additional structure. Even when additional assumptions on F F F give a learning rate with exponent independent of D D D , the constants in the bounds on the estimation error may grow exponentially with D D D , so that the sample size sufficient to guarantee a prescribed accuracy still grows exponentially with D D D : this is also part of the curse of dimensionality. When the existence of an estimator that attains a learning rate independent of D D D is proven, the next problem becomes to produce an algorithm for constructing the estimator at computational cost not exponential in D D D .
It is natural to ask which kind of special structure in F F F is both plausible in applications and exploitable by estimators so that the statistical and computational costs do not grow exponentially with D D D . This of course has been subject of much research, and we discuss some of the ideas most related to the present work in § 1.4 .
As we discuss at length in § 1.4 , we are interested in understanding when additional structure in F F F helps avoid the curse of dimensionality; and in particular when F F F has compositional structure. We study the Manifold-Index Model : the predictors fill a tube around an unknown m m m -dimensional manifold in R D \mathbb{R}^D R D , and their mean responses depend only on their nearest points on the manifold. We discuss the detail momentarily in § 1.1 . This model extends multi-index regression by replacing the linear projection onto a fixed linear subspace with the nonlinear nearest-point projection onto the manifold. For curves, this construction is essentially the Nonlinear Single-Variable Model of Wu and Maggioni [102 ] . From n n n observations, Multiscale Anisotropic IOcal Regression (MANOR) attains the rate of m m m -dimensional nonparametric regression without knowledge of the manifold or the link. At fixed model scales, normalized response range and ratio of the manifold’s volume to the m m m -th power of its reach, which are natural scalings of the problem, the sample requirement grows polynomially with D D D ; the computational cost is polynomial in D D D and n n n (§ 1.2 ). MANOR initializes the tangent spaces by multiscale local principal component analysis (PCA) ((Little et al., 2009a,b; Chen et al., 2011; Little et al., 2017)) of predictors with similar responses, blending inverse regression ([69 , 37 , 28 ] ) ideas with geometric analysis of the (approximate) level sets. It refines these estimated tangent spaces by pooling outer products of local regression slopes to form adapted, anisotropic neighborhoods ([52 , 104 , 31 ] ), and finally uses them for prediction with highly anisotropic local kernels.
We next define the Manifold-Index Model and state our main results informally; § 1.4 reviews related models and estimators and puts this work in a broad context.
The Manifold-Index Model
Let M ⊂ R D \mathcal{M} \subset\mathbb{R}^{D} M ⊂ R D be an m m m -dimensional submanifold whose reach reach M \operatorname{reach}_{\mathcal{M}} reach M is positive ([39 , 82 ] ). Every point at distance less than reach M \operatorname{reach}_{\mathcal{M}} reach M from M \mathcal{M} M has a unique nearest point on M \mathcal{M} M , so the nearest-point projection
Π M ( x ) = arg min q ∈ M ∥ x − q ∥ \Pi_{\mathcal{M}}(x) = \underset{q \in\mathcal{M}}{\operatorname{arg\,min}} \lVert x-q \rVert Π M ( x ) = q ∈ M arg min ∥ x − q ∥
is well defined on the tube of radius reach M \operatorname{reach}_{\mathcal{M}} reach M around M \mathcal{M} M . Every X X X in this tube decomposes as
X = Q + Z , Q = Π M X ∈ M , (2) X=Q+Z,\qquad Q=\Pi_{\mathcal{M}}X\in\mathcal{M}, \tag*{(2)} X = Q + Z , Q = Π M X ∈ M , (2)
where the manifold index Q Q Q is the base point , the nearest point of M \mathcal{M} M to X X X , and Z = X − Q Z=X-Q Z = X − Q is the displacement from it (Figure 1 ). Write T Q M T_Q\mathcal{M} T Q M for the tangent space at Q Q Q and N Q M = ( T Q M ) ⊥ N_Q\mathcal{M}=(T_Q\mathcal{M})^\perp N Q M = ( T Q M ) ⊥ for its normal space.
Figure 1. Geometry, response values and gradients in the Manifold-Index Model. All panels use a cylindrical surface patch M \mathcal{M} M with m = p = 2 m=p=2 m = p = 2 , D = 3 D=3 D = 3 , and f ( s , a ) = ( ( s + a ) / 2 , a ) f(s,a)=((s+a)/\sqrt{2},a) f ( s , a ) = (( s + a ) / 2 , a ) , where s s s is circumferential arc length and a a a is the axial coordinate. (a) Normal fibers and nearest-point projection. The arrows project X = Q + Z X=Q+Z X = Q + Z and X ′ = Q − Z X'=Q-Z X ′ = Q − Z to Q Q Q , so F ( X ) = F ( X ′ ) = f ( Q ) F(X)=F(X')=f(Q) F ( X ) = F ( X ′ ) = f ( Q ) . The orange plane is T Q M T_Q\mathcal{M} T Q M ; the dashed grey boundary encloses a tube of diameter O ( σ N D − m ) O(\sigma_N\sqrt{D-m}) O ( σ N D − m ) (Assumption (A2)). (b) Component level sets. The purple sheet F 1 = 0 F_1=0 F 1 = 0 and teal sheet F 2 = 0 F_2=0 F 2 = 0 meet M \mathcal{M} M along the curves s + a = 0 s+a=0 s + a = 0 and a = 0 a=0 a = 0 , respectively, and extend along normal fibers. Their intersection is the normal fiber through Q Q Q , where F = ( 0 , 0 ) F=(0,0) F = ( 0 , 0 ) . The orange, black and red arrows show ∇ F 1 \nabla F_1 ∇ F 1 at X X X , Q Q Q , X ′ X' X ′ , respectively; the grey-green arrow shows ∇ F 2 \nabla F_2 ∇ F 2 . (c) The same gradients translated to a common origin in T Q M T_Q\mathcal{M} T Q M (orange). For a unit-radius cylinder and signed outward normal offset r r r , ∇ F 1 = ( e s / ( 1 + r ) + e a ) / 2 \nabla F_1=(e_s/(1+r)+e_a)/\sqrt{2} ∇ F 1 = ( e s / ( 1 + r ) + e a ) / 2 and ∇ F 2 = e a \nabla F_2=e_a ∇ F 2 = e a , where e s , e a e_s,e_a e s , e a are the unit circumferential and axial tangent vectors at Q Q Q . Thus ∇ F 1 \nabla F_1 ∇ F 1 changes along the fiber, while ∇ F 2 \nabla F_2 ∇ F 2 is unchanged; each gradient is orthogonal to its component level surface in (b), and the two gradients span T Q M T_Q\mathcal{M} T Q M at each location.
Definition 1.1 (Manifold-Index Model). A random pair ( X , Y ) ∈ R D × R p (X,Y)\in\mathbb{R}^{D}\times\mathbb{R}^{p} ( X , Y ) ∈ R D × R p follows the Manifold-Index Model if
E [ Y ∣ X ] = F ( X ) = f ( Π M X ) , \mathbb{E}[Y\mid X]=F(X)=f(\Pi_{\mathcal{M}}X), E [ Y ∣ X ] = F ( X ) = f ( Π M X ) , where M ⊂ R D \mathcal{M}\subset\mathbb{R}^{D} M ⊂ R D is an m m m -dimensional submanifold with reach M > 0 \operatorname{reach}_{\mathcal{M}}>0 reach M > 0 , f : M → R p f:\mathcal{M}\to\mathbb{R}^{p} f : M → R p is the link function , and X X X has distance at most R R R from M \mathcal{M} M , for some R < reach M R<\operatorname{reach}_{\mathcal{M}} R < reach M .
Throughout, the normal fiber over q ∈ M q\in\mathcal{M} q ∈ M is the affine disk { q + z : z ∈ N q M , ∥ z ∥ ≤ R } \{q+z:z\in N_q\mathcal{M},\ \lVert z\rVert\le R\} { q + z : z ∈ N q M , ∥ z ∥ ≤ R } in the predictor tube. Write D f Df D f for the differential of f f f , represented by a p × m p\times m p × m matrix in orthonormal tangent coordinates, and ∇ F ( x ) ∈ R D × p \nabla F(x)\in\mathbb{R}^{D\times p} ∇ F ( x ) ∈ R D × p for the matrix whose columns are the gradients of the p p p components of F F F . We call D f ( q ) Df(q) D f ( q ) rank deficient when rank D f ( q ) < m \operatorname{rank}Df(q)<m rank D f ( q ) < m ; isolation of such points always refers to the base manifold M \mathcal{M} M . It is partially rank deficient if 0 < rank D f ( q ) < m 0<\operatorname{rank}Df(q)<m 0 < rank D f ( q ) < m . For a matrix M M M , Ran M \operatorname{Ran}M Ran M denotes its range and M T M^{T} M T its transpose.
Problem statement. Under the Manifold-Index Model , the manifold M \mathcal{M} M , the link f f f and the distribution of X X X are unknown. Our goal is to estimate F F F from independent observations ( X i , Y i ) (X_i,Y_i) ( X i , Y i ) with Y i = F ( X i ) + ζ i Y_i=F(X_i)+\zeta_i Y i = F ( X i ) + ζ i , where the centered noise vectors are independent of the predictors and of one another; they are sub-Gaussian, and on the initialization sample their norms are at most one eighth of the response-window width ((Assumption (A5))).
Three features of the model deserve emphasis.
The predictors are full-dimensional. The normal displacement Z Z Z ranges over a ( D − m ) (D-m) ( D − m ) -dimensional ball of radius R R R , and the distribution of X X X does not concentrate on M \mathcal{M} M as n n n grows. If X X X has a locally bounded positive ambient density, a Euclidean ball of sufficiently small radius b b b around a typical interior data point contains of order n b D nb^{D} n b D observations, and the covariance of the predictors in such a ball does not reveal the tangent space of M \mathcal{M} M . This contrasts with models in which the data concentrate near a low-dimensional manifold, and it parallels the single-index model, in which the predictors typically have large variance orthogonally to the index.
The regression function is intrinsically m m m -dimensional. It is constant on each normal fiber and is determined by its values on M \mathcal{M} M . The responses reveal tangential directions wherever D f Df D f has full rank.
The model interpolates between index models and general compositions. If M \mathcal{M} M is an affine subspace, Π M \Pi_{\mathcal{M}} Π M is an affine orthogonal projection and the model is a multi-index model: F F F depends only on the projection onto that affine subspace. Its index space is the direction space of M \mathcal{M} M . For a curved M \mathcal{M} M , the inner map Π M \Pi_{\mathcal{M}} Π M is nonlinear as well, but it has a precise geometric form. § 2 shows that this form is what makes the problem learnable: the gradient of F F F at Q + Z Q+Z Q + Z lies in the tangent space T Q M T_Q\mathcal{M} T Q M , so the fixed index space is replaced by a field of tangent spaces that moves with the base point.
Main result
For Q ∈ M Q\in\mathcal{M} Q ∈ M , let P Q P_Q P Q denote the orthogonal projector onto T Q M T_Q\mathcal{M} T Q M . At an independent query X X X , we assess estimates P ^ ( x ) \widehat{P}(x) P ( x ) of P Π M x P_{\Pi_{\mathcal{M}}x} P Π M x and F ^ \widehat{F} F of the regression function by
E t a n ( P ^ ) : = E ∥ P ^ ( X ) − P Π M X ∥ , E p r e d ( F ^ ) : = E ∥ F ^ ( X ) − F ( X ) ∥ 2 E ∥ F ( X ) − E F ( X ) ∥ 2 . (3) \mathcal{E}_{\mathrm{tan}}(\widehat{P}) := \mathbb{E}\left\|\widehat{P}(X)-P_{\Pi_{\mathcal{M}}X}\right\|,\qquad\mathcal{E}_{\mathrm{pred}}(\widehat{F}) := \frac{\mathbb{E}\left\|\widehat{F}(X)-F(X)\right\|^2}{\mathbb{E}\left\|F(X)-\mathbb{E}F(X)\right\|^2}. \tag*{(3)} E tan ( P ) := E P ( X ) − P Π M X , E pred ( F ) := E ∥ F ( X ) − E F ( X ) ∥ 2 E F ( X ) − F ( X ) 2 . (3)
The expectation averages over the training sample, the random split used for initialization, and an independent query. The first error is the mean sine of the largest principal angle between the estimated and true tangent spaces; the second is the mean squared prediction error relative to the variance of the signal.
Theorem (informal). Under ( A 1 ) (A1) ( A 1 ) –( A 5 ) (A5) ( A 5 ) , let p ≥ m p \ge m p ≥ m , D > m D > m D > m , with a tube-to-reach ratio 0 < R / reach M < 2 / 3 0 < R/\operatorname{reach}_{\mathcal{M}} < 2/3 0 < R / reach M < 2/3 . The differential D f Df D f may be rank deficient at finitely many points. With the scales and construction of Theorem 3.5 , and positive signal variance,
E t a n = O ~ ( n − 2 / ( m + 4 ) ) and E p r e d = O ~ ( n − 4 / ( m + 4 ) ) . \mathcal{E}_{\mathrm{tan}} = \widetilde{O}\left(n^{-2/(m+4)}\right) \quad\text{and} \quad\mathcal{E}_{\mathrm{pred}} = \widetilde{O}\left(n^{-4/(m+4)}\right). E tan = O ( n − 2/ ( m + 4 ) ) and E pred = O ( n − 4/ ( m + 4 ) ) .
Here O ~ \widetilde{O} O suppresses fixed powers of log n \log n log n for each fixed model. At fixed intrinsic dimension, model scales and vol ( M ) / reach M m \operatorname{vol}(\mathcal{M})/\operatorname{reach}_{\mathcal{M}}^{m} vol ( M ) / reach M m , the constants grow at most polynomially in D + p D+p D + p . For a fixed grid of scan radii and n ≥ D n \ge D n ≥ D , constructing the estimator costs O ( n 2 D ( D + p ) log n ) O(n^{2D}(D+p)\log n) O ( n 2 D ( D + p ) log n ) operations.
Both exponents are governed by m m m . Constancy on normal fibers allows a neighborhood of tangential width b b b to retain about n b m nb^{m} n b m observations without incurring normal approximation bias. The model itself requires only R < reach M R < \operatorname{reach}_{\mathcal{M}} R < reach M for the nearest-point projection to be defined. The analysis also requires the bound (16) on root mean square tangential stretch and the uniform bound (18) on variation of the tangential derivative along normal fibers near rank-deficient points.
At a regular base point the component gradients span the tangent space. At an isolated point where D f Df D f is rank deficient, slopes at neighboring base points supply the missing directions. The geometric conditions control the typical tangential stretch of normal offsets in the regular region (A2) and the variation of the gradient of f ∘ Π M f \circ\Pi_{\mathcal{M}} f ∘ Π M along each fiber near a point where D f Df D f is rank deficient (A3).
Contributions
The Manifold-Index Model is a nonlinear generalization of the multi-index model in which the index space is replaced by the tangent spaces of an unknown manifold, the predictors fill a tube of radius below the reach, and the link function may be non-injective. It lies between semiparametric index models and general nonparametric compositions.
An estimator. Multiscale Anisotropic lOcal Regression (MANOR) estimates the tangent field by multiscale local PCA on response preimages, the sets of predictors whose responses lie within a small distance of a given response, refines it by pooling outer products of local slopes, and predicts with an anisotropic kernel smoother. These steps use local covariances and affine least squares.
Guarantees. With s = 2 s = 2 s = 2 , Theorem 3.5 gives intrinsic-dimensional rates for mean tangent-space error and squared prediction risk. At fixed intrinsic dimension, model scales and vol ( M ) / reach M m \operatorname{vol}(\mathcal{M})/\operatorname{reach}_{\mathcal{M}}^{m} vol ( M ) / reach M m , the leading risk coefficient and the sample threshold grow at most polynomially in D + p D+p D + p .
Relation to the curve case. For curves and coarsely monotone links of smoothness s ∈ [ 1 / 2 , 1 ] s \in[1/2,1] s ∈ [ 1/2 , 1 ] , Wu and Maggioni [102 ] has a noise-dependent resolution limit. Here the prediction error continues to decrease with sample size at a fixed noise level, for manifolds of any intrinsic dimension and links that may have several preimages. The refinement schedule allows 1 < s ≤ 2 1 < s \le2 1 < s ≤ 2 ; the theorem and experiments use s = 2 s = 2 s = 2 .
(iv) Multiscale initialization. Restricting a response preimage to a small patch of the manifold gives local PCA, performed at the right scale [72 , 74 ] , a spectral gap even when the predictors fill a tube of positive radius. This supplies the initial tangent spaces for anisotropic refinement [52 ] .
(v) Rank-deficient D f Df D f at isolated points. Neighboring slopes can have opposite signs in a direction missing at the central point. Pooling their outer products preserves the signal in that direction; the fiberwise variation bound keeps it aligned with the local tangent space.
(vi) Tangent-space estimation. We consistently estimate tangent planes when predictors fill a tube whose radius is comparable to the reach. The estimator uses the noisy responses and the regularity of the link to recover this geometry.
Experiments on a line segment, a curved index and two surfaces examine the dependence of both errors on sample size. A diffusion example tests prediction when the manifold-index structure is approximate.
Context and related work
We review three kinds of structure that can avoid the curse of dimensionality: low-dimensional data, regularity assumptions, and compositional models. We discuss the last in more detail, distinguishing linear and nonlinear inner maps and the estimators used to learn them.
Low-dimensional data. The distribution of X X X may be concentrated on, or near, a low-dimensional set, say an m m m -dimensional manifold with m ≪ D m \ll D m ≪ D , while F F F is a generic s s s -smooth function on that set. For data supported on the manifold with density bounded above and below, a small Euclidean ball of radius b b b contains about n b m nb^m n b m observations instead of n b D nb^D n b D . With noise in the normal directions, the n b m nb^m n b m count applies only at scales larger than the noise thickness. For manifold-supported data, local estimators adapt to the intrinsic dimension and attain the rate n − 2 s / ( 2 s + m ) n^{-2s/(2s+m)} n − 2 s / ( 2 s + m ) [11 , 58 , 59 , 70 , 71 , 75 , 87 ] ; some of these estimators also come with fast algorithms and adapt to unknown regularity that may vary across locations and scales [70 , 71 ] . Here the reduction comes from the geometry of the distribution of X X X , not from the structure of F F F . Local PCA estimates tangent spaces when the predictors lie on or near a manifold, and regression on manifold-supported data is well developed [1 , 83 , 81 , 5 , 26 ] . Those results do not directly provide the coarse tangent estimate needed in a fixed-radius tube. Proposition 3.1 establishes it from response-preimage local covariances, after which anisotropic neighborhoods aligned with these estimates support finer estimation.
Highly regular functions. One may restrict F F F to classes simpler than s s s -smooth functions on R D \mathbb{R}^D R D : let the smoothness grow proportionally to D D D ; impose mixed smoothness [96 , 78 a] , or anisotropic smoothness with ordinary smoothness along O ( 1 ) O(1) O ( 1 ) coordinate directions and smoothness of order D D D along all others, for which neural networks avoid the curse of dimensionality in approximation [97 ] ; or impose integrability conditions on the Fourier transform, as in Barron spaces [7 , 6 ] . In general, these restrictions can become demanding as the ambient dimension grows. For a fixed index space, Barron-type conditions can exploit concentration of the gradient in that space. Curvature removes this fixed concentration, and Fourier transforms of measures supported on curved sets have different decay behavior [4 , 19 ] . The argument that places index models in Barron spaces therefore does not extend even to a curve M \mathcal{M} M of dimension 1; see Wu and Maggioni [102 ] for discussion.
Compositional structure. A third approach assumes that F F F is a composition
F = f ∘ g , g : R D → R m , f : R m → R p , m < D , (4) F = f \circ g,\qquad g : \mathbb{R}^{D} \to\mathbb{R}^{m},\qquad f : \mathbb{R}^{m} \to\mathbb{R}^{p},\qquad m < D, \tag*{(4)} F = f ∘ g , g : R D → R m , f : R m → R p , m < D , (4)
in which g g g extracts a small number of features and the link function f f f acts on them. Classical instances for which the curse of dimensionality can provably be avoided include additive and generalized additive models [95 , 20 , 48 , 51 ] and the single- and multi-index models discussed below [53 , 47 , 69 , 104 ] . Composition alone, however, does not remove the curse of dimensionality: Juditsky et al. [54 ] show that the smoothness of g g g can improve the minimax rate for general compositions, but the improved rate still depends on D D D . Results for deep networks [88 , 90 ] , for hierarchical interaction models [55 , 8 ] and for architectures inspired by the Kolmogorov–Arnold representation [60 , 76 ] quantify approximation and estimation over classes of compositions of smooth functions. Some of these classes restrict the innermost functions to small subsets of the given coordinates, whereas generalized hierarchical interaction models also allow arbitrary low-dimensional linear projections at the innermost level [8 Definition 2] . On the computational side, these results often concern approximation, or the risk of an estimator, and neither is typically proved to be computable efficiently. Moreover, their constants, if not the rates, often depend exponentially on D D D [89 ] : this is also part of the curse of dimensionality, as the sufficient sample size supplied by such a bound to guarantee a prescribed accuracy can grow exponentially with D D D . Even recent works that provably avoid the curse of dimensionality still have constants, often left implicit, that grow exponentially with D D D . When g g g is a polynomial and f f f belongs to a known finite-dimensional space, specially trained networks provably avoid the curse [25 , 100 , 63 ] , both in the rates and in the constants. For a generic nonlinear g g g , learning g g g is itself a D D D -dimensional nonparametric problem: a small range dimension alone does not remove the ambient degrees of freedom.
Linear inner maps: index models
The best understood compositional model is the one in which g g g is linear: for a matrix G ∈ R D × m G \in\mathbb{R}^{D \times m} G ∈ R D × m of rank m m m ,
F ( x ) = f ( G T x ) , f : R m → R p . (5) F(x) = f(G^{\mathsf{T}}x),\qquad f : \mathbb{R}^{m} \to\mathbb{R}^{p}. \tag*{(5)} F ( x ) = f ( G T x ) , f : R m → R p . (5)
Here F F F depends on x x x only through its projection onto the index space Ran G \operatorname{Ran} G Ran G , and the chain rule shows that every gradient lies in this fixed subspace:
∇ F ( x ) = G D f ( G T x ) T , Ran ∇ F ( x ) ⊆ Ran G for every x . (6) \nabla F(x) = G Df(G^{\mathsf{T}}x)^{\mathsf{T}},\qquad\operatorname{Ran}\nabla F(x) \subseteq\operatorname{Ran} G\quad\text{for every }x. \tag*{(6)} ∇ F ( x ) = G D f ( G T x ) T , Ran ∇ F ( x ) ⊆ Ran G for every x . (6)
If Ran G \operatorname{Ran} G Ran G were known, a neighborhood of width b b b along Ran G \operatorname{Ran} G Ran G , unrestricted orthogonally, would contain about n b m nb^m n b m observations, and (1) would hold with D D D replaced by m m m :
b 2 s ≲ 1 n b m ⟹ b ≍ n − 1 / ( 2 s + m ) , E ∥ F ^ b ( X ) − F ( X ) ∥ 2 ≲ n − 2 s / ( 2 s + m ) . (7) b^{2s} \lesssim\frac{1}{nb^m}\quad\Longrightarrow\quad b \asymp n^{-1/(2s+m)},\qquad\mathbb{E}\lVert\widehat{F}_b(X)-F(X)\rVert^2 \lesssim n^{-2s/(2s+m)}. \tag*{(7)} b 2 s ≲ n b m 1 ⟹ b ≍ n − 1/ ( 2 s + m ) , E ∥ F b ( X ) − F ( X ) ∥ 2 ≲ n − 2 s / ( 2 s + m ) . (7)
Stone [94 ] conjectured that this intrinsic rate is optimal for multi-index models. For the single-index model (m = 1 m=1 m = 1 ) it is attained by kernel estimators ([46 Theorem 3.3] , [50 Section 2.5] , [45 Corollary 22.1] ), and it is minimax ([41 Theorem 2] ); under suitable conditions the index itself can be estimated at the parametric rate n − 1 / 2 n^{-1/2} n − 1/2 . Index models also provide test beds for feature learning by neural networks ([10 , 12 , 13 , 32 , 63 , 80 , 85 ] ); trained networks appear to exploit such linear structure ([84 ] ).
Semiparametric least squares, M M M -estimators and average-derivative methods estimate a fixed index space under varying smoothness and optimization conditions ([53 , 47 , 34 , 35 , 36 , 22 , 93 , 46 ] ). The minimum average variance estimator and outer-product-of-gradients methods use local affine fits and pooled gradient information ([104 , 103 ] ); structural adaptation alternates between estimating an index space and adapting elongated neighborhoods ([52 , 31 ] ). Inverse regression methods, such as sliced inverse regression ([37 , 69 , 29 ] ), sliced average variance estimation ([28 ] ), and contour and directional regression ([67 , 66 ] ), use statistics of X X X within slices of the response. Smallest Vector Regression gives an explicit conditional construction with polynomial dependence on D D D ([62 ] ). These methods provide the index-model precedents for our response preimages, gradient pooling and anisotropic refinement; here the estimated space varies with the base point rather than being global.
Nonlinear inner maps. Compositional models are well understood when g g g is linear, but the corresponding guarantees are more limited when g g g is nonlinear. Nonlinear sufficient dimension reduction, in kernel form ([40 , 101 ] ) or through nonlinear extensions of inverse regression ([68 , 65 ] ), and active manifolds ([21 ] ) seek nonlinear features that capture the dependence of Y Y Y on X X X . Within the settings considered here, they do not provide the intrinsic-dimension guarantees with polynomial dependence on D D D that we seek. In previous work ([102 ] ) we studied the simplest nonlinear case, in which g g g is the nearest-point coordinate on an unknown curve γ ⊂ R D \gamma\subset\mathbb{R}^D γ ⊂ R D , so that F ( x ) = f ( Π γ x ) F(x)=f(\Pi_\gamma x) F ( x ) = f ( Π γ x ) (the Nonlinear Single-Variable Model, originally introduced in Lanteri et al. [61 ] ). Response preimages approximate the preimages of F F F , local principal component analysis of a response preimage recovers the direction of the curve, and a one-dimensional local-polynomial fit estimates f f f . For coarsely monotone link functions of smoothness s ∈ [ 1 / 2 , 1 ] s \in[1/2,1] s ∈ [ 1/2 , 1 ] , that estimator attains the rate n − 2 s / ( 2 s + 1 ) n^{-2s/(2s+1)} n − 2 s / ( 2 s + 1 ) up to logarithmic factors, down to an explicit saturation level determined by curvature, noise and the scale below which f f f may fail to be monotone. The estimator is constructed in time O ( D 2 n log n ) O(D^2n\log n) O ( D 2 n log n ) .
The present paper extends the curve case of Wu and Maggioni [102 ] to manifolds of any intrinsic dimension, with prediction rates whose exponent matches, up to logarithms, the minimax exponent over the broader class of Appendix § 9 , and explicit algorithms for constructing the estimator, and constants with no exponential dependence on the ambient dimension D D D . The sample threshold separates the ambient dimension from the probability of observing a local patch. Both enter polynomially.
Models inspired by the nonlinear single-variable model of Wu and Maggioni [102 ] have also been considered by Shen et al. [91 ] , who establish transformer approximation and empirical risk generalization bounds under a manifold-projection model. Their approximation result relies on an oracle network construction, and they do not provide a computationally tractable learning algorithm whose output satisfies the stated guarantees. Moreover, none of the numerical tests (with transformers) in that work are actually performed for regression with a nonlinear underlying manifold. They consider the case of s s s -Hölder link functions f f f , for s ≤ 1 s \le1 s ≤ 1 : first-order projection accuracy suffices at this regularity, whereas exploiting s = 2 s = 2 s = 2 requires controlling the shape operator and the variation of normal spaces. They also consider completely noiseless responses Y Y Y . For s = 2 s = 2 s = 2 , we give a constructive estimator with noisy responses whose exponent in n n n matches, up to logarithms, the minimax exponent over the broader regularity class in Appendix § 9 , at fixed positive noise.
Also building on Wu and Maggioni [102 ] , Kokot et al. [56 ] analyze a localized expected-gradient- outer-product (EGOP) construction under a supervised noisy-manifold hypothesis and state a learning rate whose exponent depends on the intrinsic dimension. The supplied proofs (in version arXiv:2601.07061v4), however, do not establish this guarantee: the covariance scaling asserted in their Theorem 2 is false under its stated hypotheses, while the proof of their manifold result, Theorem 3, inverts a compressed Hessian that is necessarily singular under the stated geometric and Hessian-rank hypotheses.1 Moreover, even when the manifold geometry is held fixed as D D D increases, Gaussian localization can produce variance factors that grow exponentially in D D D , and a coefficient used in the published bias proof can grow faster than exponentially in D D D . The published analysis therefore does not establish that the constants in the error bounds grow more slowly than exponentially in D D D . We control our rates and constants, both in the theory and the algorithm, showing that they are all controlled by low-degree polynomials in D D D , thereby fully avoiding the curse of dimensionality.
Connections to multi-task learning. Multi-task learning is a framework for learning multiple related tasks simultaneously [23 , 9 , 2 ] . It is a generalization of single-task learning, where the goal is to learn a single task, but the tasks are related and may share a representation [3 , 79 ] . In our setting, the tasks F j = f j ∘ Π M F_j = f_j \circ\Pi_{\mathcal{M}} F j = f j ∘ Π M , j = 1 , … , p j = 1,\ldots,p j = 1 , … , p , share the manifold index Π M X \Pi_{\mathcal{M}}X Π M X . The functions F j F_j F j are defined on the tube, while the links f j f_j f j are defined on M \mathcal{M} M . The predictors fill a D D D -dimensional tube, so their distribution alone need not reveal this representation. Pooling gradients across response components and nearby base points allows us to estimate the common tangent spaces and attain the intrinsic prediction rate.
Connections to the estimator. Phase 1 groups observations with similar responses and uses local predictor covariances to obtain initial tangent-space estimates. Phase 2 pools outer products of local gradients to update an anisotropic metric, connecting the construction to outer-product-of-gradients and structural-adaptation methods [104 , 52 , 31 ] and to recursive feature machines [84 ] . Response selection is used for initialization; subsequent updates use responses through local regression slopes and use the current tangent estimates to select neighborhoods. The ambient-dimension dependence remains explicit.
Organization and notation
Section 2 describes MANOR and its mechanism, ending with simplified pseudocode (Algorithm 1). Section 3 states the assumptions, scales, estimator and sample-size condition; Appendix A.1 gives the exact definitions. Section 4 reports the experiments, and Section 5 discusses what the results imply for prediction and tangent-space recovery. Appendix A proves Theorem 3.5 , Appendix B gives the experiment-specific constructions, and Appendix C establishes the minimax exponent for the broader regularity class defined there and states conditions for its transfer to (A1)–(A5). Table 1 collects the core notation used in the main text; proof-specific symbols are introduced where used in Appendix A.
Table 1. Core notation used in the main text. Proof-specific symbols are introduced where used in Appendix A.
Rate comparisons in Sections 1 and 2 may suppress logarithmic factors in n n n . We use a + = max { a , 0 } a_{+} = \max\{a, 0\} a + = max { a , 0 } , I I I for the identity on the space in question, and P ⊥ = I − P P^{\perp} = I - P P ⊥ = I − P for the complementary orthogonal projector. The ball B ( x , r ) B(x,r) B ( x , r ) is Euclidean; B M ( q , r ) B_{\mathcal{M}}(q,r) B M ( q , r ) uses the geodesic distance d M d_{\mathcal{M}} d M on M \mathcal{M} M . An average written avg is an arithmetic average over the indicated finite set. For a symmetric matrix M M M , the operators P Top m ( M ) P_{\operatorname{Top}m}(M) P Top m ( M ) and P Bot m ( M ) P_{\operatorname{Bot}m}(M) P Bot m ( M ) return the orthogonal projectors onto the eigenspaces of its m m m largest and m m m smallest eigenvalues, respectively; all spectral ties are resolved by a fixed deterministic rule. For a subspace V V V , P V P_V P V denotes its orthogonal projector. The notation O ~ \widetilde{O} O suppresses fixed powers of log n \log n log n .
The core idea: tangential gradients and anisotropic localization
The intrinsic rate in Theorem 3.5 rests on two geometric facts. The compositional structure places every gradient of F F F in a tangent space of M \mathcal{M} M (§ 2.1 ). If these tangent spaces were known, anisotropic neighborhoods would localize at the intrinsic scale (§ 2.2 ). Phase 1 constructs a first approximation of such neighborhoods from the responses (§ 2.3 ); Phase 2 sharpens them over O ( log n ) O(\log n) O ( log n ) steps until they reach the statistically optimal scale (§ 2.4 ); and Phase 3 uses them to predict (§ 2.5 ). Algorithm 1 summarizes the procedure; Section 3 gives its precise definition and finite-sample guarantees. For readability, the formulas in this section omit the spatial restriction (3.10) and the sample splitting specified in Section 3.3 and Appendix A.1.
Gradients of the composition lie in tangent spaces
Fix a point x = Q + z x = Q + z x = Q + z of the tube, with base point Q = Π M x Q = \Pi_{\mathcal{M}}x Q = Π M x and normal displacement z ∈ N Q M z \in N_Q\mathcal{M} z ∈ N Q M . Let E Q ∈ R D × m E_Q \in\mathbb{R}^{D \times m} E Q ∈ R D × m have orthonormal columns spanning T Q M T_Q\mathcal{M} T Q M , so that P Q = E Q E Q T P_Q = E_QE_Q^{\mathsf{T}} P Q = E Q E Q T ; we identify tangent vectors with their coordinates in R m \mathbb{R}^m R m . Let I I Q : T Q M × T Q M → N Q M \mathrm{II}_Q : T_Q\mathcal{M} \times T_Q\mathcal{M} \to N_Q\mathcal{M} II Q : T Q M × T Q M → N Q M be the second fundamental form. The shape operator S Q , z S_{Q,z} S Q , z , abbreviated S z S_z S z at a fixed base, is the symmetric matrix defined by ⟨ S z a , b ⟩ = ⟨ I I Q ( E Q a , E Q b ) , z ⟩ \langle S_z a,b\rangle= \langle\mathrm{II}_Q(E_Qa,E_Qb),z\rangle ⟨ S z a , b ⟩ = ⟨ II Q ( E Q a , E Q b ) , z ⟩ . The Weingarten equation (Proposition 8.4 Lee [64 ] ) gives its meaning: if the base point moves along a tangent direction a a a and the normal vector z z z is carried along, the tangential part of the change in z z z is − E Q S z a -E_QS_za − E Q S z a . The point x = Q + z x=Q+z x = Q + z therefore moves tangentially by E Q ( I − S z ) a E_Q(I-S_z)a E Q ( I − S z ) a . When D f ( Q ) Df(Q) D f ( Q ) acts on ambient vectors, we extend it by zero on N Q M N_Q\mathcal{M} N Q M ; in coordinates this extension is D f ( Q ) E Q T Df(Q)E_Q^{\mathsf T} D f ( Q ) E Q T . The differential of Π M \Pi_{\mathcal{M}} Π M at x x x is (Lemma A.1)
D Π M ( Q + z ) = E Q ( I − S z ) − 1 E Q T . (8) D\Pi_{\mathcal{M}}(Q+z)=E_Q(I-S_z)^{-1}E_Q^{\mathsf T}. \tag*{(8)} D Π M ( Q + z ) = E Q ( I − S z ) − 1 E Q T . (8)
Moving along a normal fiber fixes the base point, so D Π M ( x ) D\Pi_{\mathcal{M}}(x) D Π M ( x ) annihilates N Q M N_Q\mathcal{M} N Q M . Moving the base point along a tangent vector a a a , carrying the normal displacement along, moves x x x to first order by E Q ( I − S z ) a E_Q(I-S_z)a E Q ( I − S z ) a plus a normal vector (the Weingarten equation [64 Proposition 8.4] ). Thus a tangential displacement of x x x is amplified by ( I − S z ) − 1 (I-S_z)^{-1} ( I − S z ) − 1 at the base point. The shape operator satisfies ∥ S z ∥ ≤ ∥ z ∥ / reach M \lVert S_z\rVert\le\lVert z\rVert/\operatorname{reach}_{\mathcal{M}} ∥ S z ∥ ≤ ∥ z ∥ / reach M [82 ] , so I − S z I-S_z I − S z is invertible in the tube; if ∥ z ∥ ≤ R = c reach M \lVert z\rVert\le R=c\operatorname{reach}_{\mathcal{M}} ∥ z ∥ ≤ R = c reach M with the tube ratio c c c of Assumption (A1), then ( 1 + c ) − 1 I ⪯ ( I − S z ) − 1 ⪯ ( 1 − c ) − 1 I (1+c)^{-1}I \preceq(I-S_z)^{-1} \preceq(1-c)^{-1}I ( 1 + c ) − 1 I ⪯ ( I − S z ) − 1 ⪯ ( 1 − c ) − 1 I . The chain rule applied to F = f ∘ Π M F=f\circ\Pi_{\mathcal{M}} F = f ∘ Π M gives
∇ F ( Q + z ) = E Q ( I − S z ) − 1 D f ( Q ) T ∈ R D × p , (9) \nabla F(Q+z)=E_Q(I-S_z)^{-1}Df(Q)^{\mathsf T}\in\mathbb{R}^{D\times p}, \tag*{(9)} ∇ F ( Q + z ) = E Q ( I − S z ) − 1 D f ( Q ) T ∈ R D × p , (9)
where D f ( Q ) ∈ R p × m Df(Q)\in\mathbb{R}^{p\times m} D f ( Q ) ∈ R p × m is the differential of f f f at Q Q Q in the basis E Q E_Q E Q . Since E Q E_Q E Q has full column rank and I − S z I-S_z I − S z is invertible, rank D F ( Q + z ) = rank D f ( Q ) \operatorname{rank}DF(Q+z)=\operatorname{rank}Df(Q) rank D F ( Q + z ) = rank D f ( Q ) , where D F = ( ∇ F ) T DF=(\nabla F)^{\mathsf T} D F = ( ∇ F ) T . Thus each point where D f Df D f is rank deficient gives an entire normal fiber where D F DF D F has rank below m m m .
The geometric point is that the normal displacement changes the coordinates of a tangent derivative without creating a normal derivative. Thus the ambient gradients can vary along a fiber; at a full-rank base point they span the same tangent space at every offset.
Formula (2.2) contains three facts that the rest of the paper uses.
(b) The gradients span the tangent space: if D f ( Q ) Df(Q) D f ( Q ) has rank m m m , then Ran ∇ F ( Q + z ) = T Q M \operatorname{Ran}\nabla F(Q+z)=T_Q\mathcal{M} Ran ∇ F ( Q + z ) = T Q M for every z z z in the fiber. The multi-index inclusion Ran ∇ F ( x ) ⊆ Ran G \operatorname{Ran}\nabla F(x)\subseteq\operatorname{Ran}G Ran ∇ F ( x ) ⊆ Ran G in (6) becomes Ran ∇ F ( x ) = T Π M x M \operatorname{Ran}\nabla F(x)=T_{\Pi_{\mathcal{M}}x}\mathcal{M} Ran ∇ F ( x ) = T Π M x M : the index space becomes a moving tangent space.
(c) The subspace is invariant at full rank. As the normal offset varies, ( I − S z ) − 1 (I-S_z)^{-1} ( I − S z ) − 1 can change the component gradients within T Q M T_Q\mathcal{M} T Q M . When D f ( Q ) Df(Q) D f ( Q ) has rank m m m , their span remains T Q M T_Q\mathcal{M} T Q M at every offset; its projector P Q P_Q P Q is the object we estimate.
On a curve the shape factor is scalar: wherever the derivative is nonzero, one gradient identifies the tangent line along the entire fiber, as in Wu and Maggioni [102 ] . For m > 1 m>1 m > 1 , a single nonzero gradient spans a line that can rotate within the tangent space. With p ≥ m p\ge m p ≥ m response components, the gradients can identify the whole plane at a regular base point (Figure 1).
Localizing along the manifold
To provide intuition and motivation for our construction, we briefly discuss here an idealized scenario. If M \mathcal{M} M were known, base points, geodesic distances and tangent spaces would be available. To estimate F F F near x 0 x_0 x 0 with base point q 0 q_0 q 0 , consider the neighborhood
N b ( x 0 ) = { q + z : q ∈ B M ( q 0 , b ) , z ∈ N q M , ∥ z ∥ ≤ R } , \mathcal{N}_b(x_0)=\{q+z:q\in B_{\mathcal{M}}(q_0,b),\ z\in N_q\mathcal{M},\ \lVert z\rVert\le R\}, N b ( x 0 ) = { q + z : q ∈ B M ( q 0 , b ) , z ∈ N q M , ∥ z ∥ ≤ R } ,
which consists of the entire normal fibers over a geodesic ball of radius b b b in M \mathcal{M} M . A local linear (or local constant) estimator on this neighborhood has two properties:
Bias of order b s b^s b s . On N b ( x 0 ) \mathcal{N}_b(x_0) N b ( x 0 ) , the function F ( x ) = f ( Π M x ) F(x)=f(\Pi_{\mathcal{M}}x) F ( x ) = f ( Π M x ) depends only on the base point, which stays within intrinsic distance b b b of q 0 q_0 q 0 . For 1 < s ≤ 2 1<s\leq2 1 < s ≤ 2 , the first-order expansion of f f f around q 0 q_0 q 0 has remainder O ( b s ) O(b^s) O ( b s ) . A local linear fit in tangent coordinates reproduces the linear term. A local average with an even leading kernel has the same bias order under the density regularity in (A2), since the contribution of the linear term is O ( b 2 ) O(b^2) O ( b 2 ) . The extent of the neighborhood in the normal directions does not contribute to the bias, because F F F does not vary along fibers.
Variance of order 1 / ( n b m ) 1/(nb^m) 1/ ( n b m ) . Observations in N b ( x 0 ) \mathcal{N}_b(x_0) N b ( x 0 ) correspond to base points in a geodesic ball of radius b b b , giving about n P { Π M X ∈ B M ( q 0 , b ) } ≍ n b m n\mathbb{P}\{\Pi_{\mathcal{M}}X\in B_{\mathcal{M}}(q_0,b)\}\asymp nb^m n P { Π M X ∈ B M ( q 0 , b )} ≍ n b m observations if the base points have a density bounded above and below on M \mathcal{M} M . Complete fibers contribute no factor ( b / R ) D − m (b/R)^{D-m} ( b / R ) D − m .
The tube is therefore statistically wide in directions along which F F F is constant and locally narrow only in the m m m directions that affect the response. An effective estimator should use the wide directions for replication and spend its spatial resolution on the tangent directions. Balancing the two gives (7) , with b ≍ n − 1 / ( 2 s + m ) b\asymp n^{-1/(2s+m)} b ≍ n − 1/ ( 2 s + m ) . With a locally bounded positive ambient density, at a typical interior point a Euclidean ball of radius b < R b<R b < R contains about n b D nb^D n b D observations and gives (1) .
Near x 0 x_0 x 0 , the neighborhood N b ( x 0 ) \mathcal{N}_b(x_0) N b ( x 0 ) is approximated by an ellipsoid aligned with T q 0 M T_{q_0}\mathcal{M} T q 0 M ; building the ellipsoid requires only the tangent space T q 0 M T_{q_0}\mathcal{M} T q 0 M , not M \mathcal{M} M itself. For a rank-m m m projector P P P and widths 0 < b ≤ h 0<b\leq h 0 < b ≤ h , define the anisotropic distance
d P , b , h ( v ) 2 = ∥ v ∥ 2 h 2 + ∥ P v ∥ 2 b 2 , v ∈ R D . (10) d_{P,b,h}(v)^2=\frac{\lVert v\rVert^2}{h^2}+\frac{\lVert Pv\rVert^2}{b^2},\qquad v\in\mathbb{R}^D. \tag*{(10)} d P , b , h ( v ) 2 = h 2 ∥ v ∥ 2 + b 2 ∥ P v ∥ 2 , v ∈ R D . (10)
The set { v : d P , b , h ( v ) ≤ 1 } \{v:d_{P,b,h}(v)\leq1\} { v : d P , b , h ( v ) ≤ 1 } is an ellipsoid with m m m short semi-axes of length b / 1 + ρ 2 b/\sqrt{1+\rho^2} b / 1 + ρ 2 along Ran P \operatorname{Ran}P Ran P and D − m D-m D − m long semi-axes of length h h h along Ran P ⊥ \operatorname{Ran}P^\perp Ran P ⊥ , where ρ = b / h ≤ 1 \rho=b/h\leq1 ρ = b / h ≤ 1 . We call b b b the tangential width , h h h the normal width and ρ \rho ρ the aspect ratio . With P = P q 0 P=P_{q_0} P = P q 0 and a normal width h h h slightly larger than the fiber diameter 2 R 2R 2 R , the ellipsoid { x : d P , b , h ( x − x 0 ) ≤ 1 } \{x:d_{P,b,h}(x-x_0)\leq1\} { x : d P , b , h ( x − x 0 ) ≤ 1 } contains the complete normal fibers over a tangential ball of radius comparable to b b b . The diameter, not the radius, is needed because x 0 x_0 x 0 may lie at normal distance R R R from M \mathcal{M} M on one side of its fiber, while the fiber extends to distance R R R on the other side.
Two difficulties separate this ideal picture from an estimator. The first is that P q 0 P_{q_0} P q 0 is unknown and changes with the base point. The second is that an anisotropic neighborhood is only as thin as the tangent estimate is accurate. If ∥ P ^ − P q 0 ∥ = Δ \lVert\widehat{P}-P_{q_0}\rVert=\Delta ∥ P − P q 0 ∥ = Δ , then a vector v v v with ∥ P ^ v ∥ ≤ b \lVert\widehat{P}v\rVert\leq b ∥ P v ∥ ≤ b and ∥ v ∥ ≤ h \lVert v\rVert\leq h ∥ v ∥ ≤ h has a true tangential component ∥ P q 0 v ∥ ≤ b + Δ h = ( ρ + Δ ) h \lVert P_{q_0}v\rVert\leq b+\Delta h=(\rho+\Delta)h ∥ P q 0 v ∥ ≤ b + Δ h = ( ρ + Δ ) h . A neighborhood of aspect ratio ρ \rho ρ therefore requires a tangent estimate with error at most a fixed fraction of ρ \rho ρ when h / R h/R h / R is fixed. More precisely, the proof controls R Δ / b R\Delta/b R Δ/ b , the displacement across a fiber in units of the short width. This coupling drives the estimator. A coarse tangent estimate allows a moderately thin neighborhood; local regression in that neighborhood yields a better tangent estimate; the better estimate allows a thinner neighborhood; and so on. Phase 1 starts this process, Phase 2 runs it for O ( log n ) O(\log n) O ( log n ) steps, and Phase 3 uses the final neighborhoods to predict.
Phase 1: a warm start from response preimages
Phase 1, the pilot , estimates tangent spaces from predictors with similar responses. Since Y = f ( Q ) + ζ Y = f(Q) + \zeta Y = f ( Q ) + ζ , conditioning on the response at a given base point leaves the normal displacement law unchanged (Lemma A.2 ). Fix a response-window width δ Y > 0 \delta_Y > 0 δ Y > 0 (this δ Y \delta_Y δ Y is independent of n n n ). For an observation i i i , called the anchor , the response preimage
U i = { X j : ∥ Y j − Y i ∥ ≤ δ Y } U_i = \{X_j : \lVert Y_j - Y_i \rVert\le\delta_Y\} U i = { X j : ∥ Y j − Y i ∥ ≤ δ Y }
collects predictors with responses close to Y i Y_i Y i . Phase 1 computes local covariances on U i ∩ B ( X i , r ) U_i \cap B(X_i,r) U i ∩ B ( X i , r ) as the spatial radius r r r varies. The base points can form several distinct groups, or packets , when f f f is non-injective. The packet containing the anchor’s base point Q i = Π M X i Q_i = \Pi_{\mathcal{M}}X_i Q i = Π M X i can be close in ambient distance to another packet with similar responses. A useful radius retains the normal spread while confining the selected base points to one small patch; Figure 2 illustrates this choice. The spatial extent of the patch, rather than the number of connected components in it, determines the spread along the manifold. This is a local use of inverse regression (§ 1.4 ).
Figure 2. Phase 1: Multiscale local PCA in a response preimage: schematic with noiseless responses and a locally bounded positive ambient density. The orange regions depict the population set { x : ∥ F ( x ) − Y i ∥ ≤ δ Y } \{x : \lVert F(x) - Y_i\rVert\le\delta_Y\} { x : ∥ F ( x ) − Y i ∥ ≤ δ Y } within the tube; U i U_i U i consists of the sampled predictors in this set. (a–c) A normal section of a curved manifold M \mathcal{M} M , with fixed Y i , X i Y_i, X_i Y i , X i and increasing radius. Dark orange marks the part inside B ( X i , r ) B(X_i,r) B ( X i , r ) , pale orange the rest, and dark blue the retained base points. The small ball truncates fibers, and at this scale the curse of dimensionality applies; the intermediate ball isolates C 1 C_1 C 1 ; the large ball also reaches C 2 C_2 C 2 . (d) A 2-dimensional M ⊂ R 3 \mathcal{M} \subset\mathbb{R}^3 M ⊂ R 3 with p = 2 p=2 p = 2 : the slab { x : ∣ F k ( x ) − ( Y i ) k ∣ ≤ δ Y } \{x : \lvert F_k(x) - (Y_i)_k\rvert\le\delta_Y\} { x : ∣ F k ( x ) − ( Y i ) k ∣ ≤ δ Y } is the union of the normal fibers over a thin strip of M \mathcal{M} M around the level set { f k = ( Y i ) k } \{f_k=(Y_i)_k\} { f k = ( Y i ) k } , two curves for k = 1 k=1 k = 1 (green) and one for k = 2 k=2 k = 2 (purple); U i U_i U i lies in the intersection of these slabs, a union of cylinders (orange) over the packets C 1 C_1 C 1 and C 2 C_2 C 2 around the points of f − 1 ( Y i ) f^{-1}(Y_i) f − 1 ( Y i ) . The dashed curve is the section of (a–c). (e) The eigenvalues λ 1 ≥ ⋯ ≥ λ D \lambda_1 \ge\cdots\ge\lambda_D λ 1 ≥ ⋯ ≥ λ D of the covariance Σ ^ i ( r ) \widehat{\Sigma}_i(r) Σ i ( r ) of the predictors in U i ∩ B ( X i , r ) U_i \cap B(X_i,r) U i ∩ B ( X i , r ) , in the regimes of (a), (b) and (c) (bands): all eigenvalues grow together; the m m m tangential ones λ D − m + 1 , … , λ D \lambda_{D-m+1}, \ldots, \lambda_D λ D − m + 1 , … , λ D saturate at a level set by the packet diameter, while the D − m D-m D − m normal ones keep growing; the direction joining C 1 C_1 C 1 and C 2 C_2 C 2 acquires a large variance, so that λ 1 \lambda_1 λ 1 rises and λ D − m + 1 \lambda_{D-m+1} λ D − m + 1 joins the normal eigenvalues. (f) The score H \mathcal{H} H of (2.4) is maximal at the selected radius r 2 r_2 r 2 and decreases once C 2 C_2 C 2 enters the ball. Dashed lines mark r 1 r_1 r 1 , r 2 r_2 r 2 , r 3 r_3 r 3 .
Multiscale local PCA. Write Σ ^ i ( r ) \widehat{\Sigma}_i(r) Σ i ( r ) for the empirical centered covariance of the predictors in U i ∩ B ( X i , r ) U_i \cap B(X_i,r) U i ∩ B ( X i , r ) . Figure 2 (a–Figure 2 c) illustrates three regimes of the radius scan.
At a typical interior point, a sufficiently small ball lies inside the packet in every direction. With a locally bounded positive ambient density, the rescaled cloud becomes nearly uniform as the radius tends to zero, and its covariance eigenvalues are comparable.
For intermediate radii, the ball isolates one packet and meets no other packet; it may still mildly truncate the outer ends of the normal fibers. The selected predictors have a small spread along T Q i M T_{Q_i}\mathcal{M} T Q i M (the packet diameter) and a large spread across it (the retained fiber length), so the m m m smallest covariance eigenvalues separate from the D − m D-m D − m largest, and their eigenvectors approximately span T Q i M T_{Q_i}\mathcal{M} T Q i M .
For large radii, the ball reaches other packets of the same response preimage. The selected predictors form a mixture, the covariance acquires a large eigenvalue in the direction joining the packets, and the separation deteriorates.
The scan is therefore a three-way balance: a radius that is too small hides the normal spread, an intermediate radius exposes the tangent–normal split, and a large radius mixes packets. The score below rewards the middle regime while penalizing both failures. Figure 2 (e,Figure 2 f) shows the spectral separation and the resulting score. For a covariance matrix Σ \Sigma Σ , let λ 1 ≥ ⋯ ≥ λ D \lambda_1 \ge\cdots\ge\lambda_D λ 1 ≥ ⋯ ≥ λ D be its eigenvalues. To detect the second regime, Phase 1 uses the score
H ( Σ ) = log ( λ 2 ⋯ λ D − m ) 2 / ( D − m − 1 ) λ 1 λ D − m + 1 ( D − m ≥ 2 ) , H ( Σ ) = log λ 1 λ 2 ( D − m = 1 ) , (11) \mathcal{H}(\Sigma) = \log\frac{(\lambda_2 \cdots\lambda_{D-m})^{2/(D-m-1)}}{\lambda_1\lambda_{D-m+1}} \quad(D-m \ge2), \qquad\mathcal{H}(\Sigma) = \log\frac{\lambda_1}{\lambda_2} \quad(D-m = 1), \tag*{(11)} H ( Σ ) = log λ 1 λ D − m + 1 ( λ 2 ⋯ λ D − m ) 2/ ( D − m − 1 ) ( D − m ≥ 2 ) , H ( Σ ) = log λ 2 λ 1 ( D − m = 1 ) , (11)
For D − m ≥ 2 D-m \ge2 D − m ≥ 2 , let λ ˉ = ( λ 2 ⋯ λ D − m ) 1 / ( D − m − 1 ) \bar{\lambda} = (\lambda_2 \cdots\lambda_{D-m})^{1/(D-m-1)} λ ˉ = ( λ 2 ⋯ λ D − m ) 1/ ( D − m − 1 ) be the geometric mean of the eigenvalues λ 2 , … , λ D − m \lambda_2,\ldots,\lambda_{D-m} λ 2 , … , λ D − m ; then
H ( Σ ) = log λ ˉ λ D − m + 1 + log λ ˉ λ 1 . \mathcal{H}(\Sigma) = \log\frac{\bar{\lambda}}{\lambda_{D-m+1}} + \log\frac{\bar{\lambda}}{\lambda_1}. H ( Σ ) = log λ D − m + 1 λ ˉ + log λ 1 λ ˉ .
The first term is large when λ D − m + 1 \lambda_{D-m+1} λ D − m + 1 , the largest of the m m m tangential eigenvalues, is small compared with λ ˉ \bar{\lambda} λ ˉ (the cloud is a thin packet). The second term is never positive, and it is close to zero unless λ 1 \lambda_1 λ 1 is much larger than λ ˉ \bar{\lambda} λ ˉ (a second packet has entered the ball). For D − m = 1 D-m=1 D − m = 1 the score only measures how thin the cloud is. Phase 1 scans a finite grid of radii R \mathcal{R} R , selects the smallest maximizer r i r_i r i of r ↦ H ( Σ ^ i ( r ) ) r \mapsto\mathcal{H}(\widehat{\Sigma}_i(r)) r ↦ H ( Σ i ( r )) , and records an orthonormal basis $V_i of the eigenspace of the m m m smallest eigenvalues of Σ ^ i ( r i ) \widehat{\Sigma}_i(r_i) Σ i ( r i ) . Under (A4), the theoretical scan grid lies in the part of regime (2) that retains complete fibers. The numerical scan may also select a radius that truncates their ends, as in Figure 2 (b).
Aggregation. The scan supplies many local bases, but each basis can be distorted when the selected ball truncates the normal fibers, or by response noise in the response preimage. The ball is centered at X i X_i X i , not at its base point, so the retained part of a fiber is not centered on M \mathcal{M} M and its length varies with the base point; this tilts the eigenvectors of Σ ^ i ( r i ) \widehat{\Sigma}_i(r_i) Σ i ( r i ) away from T Q i M T_{Q_i}\mathcal{M} T Q i M . Aggregation turns these local estimates into one projector field by averaging projectors rather than basis vectors. Phase 1 therefore averages the projectors V j V j T V_jV_j^{\mathsf{T}} V j V j T over the observations j j j in the selected neighborhood of i i i and keeps the dominant m m m -dimensional eigenspace of the average. With Ω i = { j : X j ∈ U i ∩ B ( X i , r i ) } \Omega_i=\{j:X_j\in U_i\cap B(X_i,r_i)\} Ω i = { j : X j ∈ U i ∩ B ( X i , r i )} ,
P ^ p i l ( X i ) = P Top m ( 1 ∣ Ω i ∣ ∑ j ∈ Ω i V j V j T ) . \widehat{P}_{\mathrm{pil}}(X_i)=P_{\operatorname{Top}m}\left(\frac{1}{|\Omega_i|}\sum_{j\in\Omega_i}V_jV_j^{\mathsf{T}}\right). P pil ( X i ) = P Top m ∣ Ω i ∣ 1 j ∈ Ω i ∑ V j V j T .
We average projectors because a basis is only defined up to an orthogonal transformation. The average is stable: if all projectors in the collection are within operator-norm distance Δ \Delta Δ of P Q i P_{Q_i} P Q i , so is their average, and replacing a fraction α \alpha α of them by arbitrary projectors adds at most 2 α 2\alpha 2 α . Once the averaged projector P ‾ \overline{P} P satisfies ∥ P ‾ − P Q i ∥ ≤ 1 / 4 \lVert\overline{P}-P_{Q_i}\rVert\leq1/4 ∥ P − P Q i ∥ ≤ 1/4 , extracting its dominant m m m -dimensional eigenspace cannot inflate the error by more than a fixed factor: ∥ P Top m ( P ‾ ) − P Q i ∥ ≤ C ∥ P ‾ − P Q i ∥ \lVert P_{\operatorname{Top}m}(\overline{P})-P_{Q_i}\rVert\leq C\lVert\overline{P}-P_{Q_i}\rVert ∥ P Top m ( P ) − P Q i ∥ ≤ C ∥ P − P Q i ∥ for an absolute constant C C C (Lemma A.6).
Output. Phase 1 returns the pooled projectors P ^ p i l \widehat{P}_{\mathrm{pil}} P pil , from which the initial projector P ^ 0 \widehat{P}_0 P 0 of Phase 2 is obtained by the assignment of § 3.3 , and the selected radii, whose average, possibly multiplied by a constant factor h r e s c a l e h_{\mathrm{rescale}} h rescale , is the initial normal width h 0 h_0 h 0 of Phase 2. Quantities attached to Phase 1 carry the subscript “pil” (for “pilot”). The analysis uses the same multiscale PCA on a grid within the one-packet interval of (A4). Proposition 3.1 bounds the angle after aggregation at the scale required for the first refinement.
A small Euclidean ball localizes the base point by cutting the normal fibers short (§ 2.2 ). The initial tangent estimate lets Phase 2 choose tangential and normal widths separately.
Phase 2: shrink tangentially, expand normally
Phase 2 performs k ∗ k_\ast k ∗ refinement steps with decreasing tangential widths b k = ρ k h k b_k=\rho_kh_k b k = ρ k h k , where ρ k \rho_k ρ k is the aspect ratio and h k h_k h k the normal width at step k k k . Each center fits an affine function using its incoming projector P ^ k − 1 \widehat{P}_{k-1} P k − 1 . The next tangent field is computed from the outer products of these slopes.
(a) Local gradients. At each center X j X_j X j , fit once using its own incoming projector P ^ k − 1 ( X j ) \widehat{P}_{k-1}(X_j) P k − 1 ( X j ) and the kernel K ( t ) = ( 1 − t ) 2 K(t)=(1-t)^2 K ( t ) = ( 1 − t ) 2 . The optimization variables are an intercept β ^ 0 ∈ R p \widehat{\beta}_0\in\mathbb{R}^p β 0 ∈ R p and a slope matrix G ^ ∈ R D × p \widehat{G}\in\mathbb{R}^{D\times p} G ∈ R D × p .
( β ^ 0 , j , k , G ^ j , k ) ∈ arg min β 0 , G ∑ ℓ K ( d P ^ k − 1 ( X j ) , b k , h k ( X ℓ − X j ) 2 ) ∥ Y ℓ − β 0 − G T ( X ℓ − X j ) ∥ 2 . (\widehat{\beta}_{0,j,k},\widehat{G}_{j,k})\in\underset{\beta_0,G}{\operatorname{arg\,min}}\sum_{\ell}K\left(d_{\widehat{P}_{k-1}(X_j),b_k,h_k}(X_\ell-X_j)^2\right)\lVert Y_\ell-\beta_0-G^{\mathsf{T}}(X_\ell-X_j)\rVert^2. ( β 0 , j , k , G j , k ) ∈ β 0 , G arg min ℓ ∑ K ( d P k − 1 ( X j ) , b k , h k ( X ℓ − X j ) 2 ) ∥ Y ℓ − β 0 − G T ( X ℓ − X j ) ∥ 2 .
The slope matrix G ^ j , k ∈ R D × p \widehat{G}_{j,k}\in\mathbb{R}^{D\times p} G j , k ∈ R D × p estimates the gradients of the response components near X j X_j X j .
(b) Pooling. At a target x x x , use its incoming projector to select
W x , k = { u : d P ^ k − 1 ( x ) , b k , h k ( u − x ) ≤ 1 } . \mathcal{W}_{x,k}=\{u:d_{\widehat{P}_{k-1}(x),b_k,h_k}(u-x)\le1\}. W x , k = { u : d P k − 1 ( x ) , b k , h k ( u − x ) ≤ 1 } .
Average the outer products and retain the leading m m m -dimensional eigenspace:
C ^ k ( x ) = avg j : X j ∈ W x , k G ^ j , k G ^ j , k T , P ^ k ( x ) = P T o p m ( C ^ k ( x ) ) . \widehat{C}_k(x)=\underset{j:X_j\in\mathcal{W}_{x,k}}{\operatorname{avg}}\ \widehat{G}_{j,k}\widehat{G}_{j,k}^{\mathsf{T}},\qquad\widehat{P}_k(x)=P_{\mathrm{Top}_m}\left(\widehat{C}_k(x)\right). C k ( x ) = j : X j ∈ W x , k avg G j , k G j , k T , P k ( x ) = P Top m ( C k ( x ) ) .
Averaging outer products of local gradient estimates is the device of the outer product of gradients method for index models (see § 1.4 and references therein). Here each fit follows the estimated tangent space at its own center, and pooling refines the moving tangent field from step to step.
Figure 3. Phase 2 refinement in the normal section of Figure 2 . The orange fibers mark the noiseless response selection region depicted in Figure 2 ; the black outlines show the predictor neighborhoods about the same anchor X i X_i X i . Dark orange lies inside the displayed neighborhood and pale orange outside. Panel (a) reproduces the intermediate-radius selection in Figure 2 (b), which here truncates the ends of the fibers. In panels (b) and (c), b k + 1 < b k b_{k+1}<b_k b k + 1 < b k narrows the range of base points while h k + 1 > h k h_{k+1}>h_k h k + 1 > h k retains a longer portion of each nearby fiber. Unlike Phase 1, Phase 2 does not restrict a neighborhood to the response preimage: it selects predictors by position alone, and their responses enter the local affine fits.
What a neighborhood contributes. At a regular base point, the responses already distinguish every tangent direction. At a point where D f Df D f is rank deficient, some of those directions are invisible on the central fiber. The surrounding ball matters: a missing direction whose slope grows linearly away from the central point contributes energy of order b k 2 b_k^2 b k 2 to the pooled slope outer products. Taking the outer product before averaging retains this energy even when the slopes at opposite sides of the point have different signs. Meanwhile the tangent planes at neighboring centers rotate. Assumption (A3) limits how much this rotation can change the response differential on the complete fibers near a point where D f Df D f is rank deficient.
Why the next neighborhood retains complete fibers. Across a normal fiber of length 2 R 2R 2 R , a tilt Δ \Delta Δ displaces the estimated tangential coordinates by at most 2 R Δ 2R\Delta 2 R Δ . If this displacement fits within the next tangential width, the neighborhood retains complete fibers above a common ball of base points. Thus refinement must control the angle relative to the next width . Near a point where D f Df D f is rank deficient the local fits recover a missing tangent slope of order b k b_k b k , while their normal slope is of order b k 2 b_k^2 b k 2 in fixed physical units. Cancellation of the first-order tilt in the fit gives the error bound needed to retain complete fibers at the next step (Lemma A.12 ). Away from points where D f Df D f is rank deficient, curvature transfers part of a center’s tangent error to its neighbors. By (A.33), the root mean square stretch in (A2) controls this transfer over two successive updates.
The schedule and where to stop. The aspect ratio shrinks by a ρ = e − 1 / [ 2 ( 2 s + m ) ] a_\rho=e^{-1/[2(2s+m)]} a ρ = e − 1/ [ 2 ( 2 s + m )] , and the normal width grows by a h = exp { 1 / [ 2 ( D + 4 ) k ∗ ] } a_h=\exp\{1/[2(D+4)k_*]\} a h = exp { 1/ [ 2 ( D + 4 ) k ∗ ]} per step, see (23) . Starting from b 0 = ρ 0 h 0 b_0=\rho_0h_0 b 0 = ρ 0 h 0 , b k = b 0 ( a ρ a h ) k b_k=b_0(a_\rho a_h)^k b k = b 0 ( a ρ a h ) k decreases. The statistical results use s = 2 s=2 s = 2 . The warm start supplies the first aligned neighborhood. Through the ensuing rounds the largest displacement due to tilt remains smaller than the next tangential width (Proposition 3.3 ). Narrower neighborhoods reduce tangential bias. At fixed local sampling density, a neighborhood contains order n f b k m n_f b_k^m n f b k m observations from a sample of size n f n_f n f assigned to that round. Refinement stops while these observations still resolve a normal slope of order b k 2 b_k^2 b k 2 . At fixed model scales the terminal tangential width is of order n − 1 / ( m + 4 ) n^{-1/(m+4)} n − 1/ ( m + 4 ) , up to logarithms, with complete normal fibers.
Phase 3: prediction
The regression estimate is an anisotropic kernel smoother. For a fixed positive multiplier c ρ c_\rho c ρ , set h ∗ = h k ∗ h_* = h_{k_*} h ∗ = h k ∗ , ρ ∗ = c ρ ρ k ∗ \rho_* = c_\rho\rho_{k_*} ρ ∗ = c ρ ρ k ∗ and b ∗ = ρ ∗ h ∗ b_*=\rho_*h_* b ∗ = ρ ∗ h ∗ . For a query x x x ,
F ^ ( x ) : = ∑ i ϖ i ( x ) Y i ∑ i ϖ i ( x ) , ϖ i ( x ) : = K ( ∥ x − X i ∥ 2 h ∗ 2 + ∥ P ^ k ∗ ( X i ) ( x − X i ) ∥ 2 b ∗ 2 ) (12) \widehat{F}(x) := \frac{\sum_i \varpi_i(x)Y_i}{\sum_i \varpi_i(x)}, \qquad\varpi_i(x) := K\left(\frac{\lVert x-X_i\rVert^2}{h_*^2}+\frac{\lVert\widehat{P}_{k_*}(X_i)(x-X_i)\rVert^2}{b_*^2}\right) \tag*{(12)} F ( x ) := ∑ i ϖ i ( x ) ∑ i ϖ i ( x ) Y i , ϖ i ( x ) := K ( h ∗ 2 ∥ x − X i ∥ 2 + b ∗ 2 ∥ P k ∗ ( X i ) ( x − X i ) ∥ 2 ) (12)
where the sum is over the observations reserved for prediction; their mean is used if the denominator vanishes. Each predictor supplies its own tangent plane. With the true planes the kernel is even to first order in the base displacement, giving bias of order b ∗ 2 b_*^2 b ∗ 2 . The local count is proportional to the size of the prediction sample times b ∗ m b_*^m b ∗ m at fixed local density. Near a point where D f Df D f vanishes, even an imperfectly aligned neighborhood has little response variation. If D f Df D f is only partially rank deficient, the other responses still vary, but the least accurate planes occupy a shrinking region. Integrating their contribution preserves the m m m -dimensional prediction rate (Proposition 3.4 ).
Algorithm
Algorithm 1 gives simplified pseudocode for the three phases of MANOR. It shows how each fitted slope enters the pools and how the widths change between rounds. § 3.3 specifies the assignment of an anchor star to each target, the local cylinder, the tests for retaining a fit and the independent samples used in the theorem. There, an anchor star S S S consists of an eligible anchor and the eligible anchors linked to it by the response and spatial tests of Appendix § 7 .§ 1 ; P S P_S P S projects orthogonally onto the leading m m m eigenspace of the average of their pooled projectors. Appendix § 7 .§ 1 determines the stopping step k ∗ k_* k ∗ . In the
Algorithm 1: MANOR (simplified pseudocode)
Input: data ( X i , Y i ) i = 1 n ⊂ R D × R p (X_i,Y_i)_{i=1}^n \subset\mathbb{R}^D \times\mathbb{R}^p ( X i , Y i ) i = 1 n ⊂ R D × R p ; intrinsic dimension m m m ; kernel K ( t ) = ( 1 − t ) 2 K(t)=(1-t)^2 K ( t ) = ( 1 − t ) 2 ; response resolution δ Y \delta_Y δ Y ; finite set of radii R \mathcal{R} R ; initial aspect ratio ρ 0 ∈ ( 0 , 1 ) \rho_0 \in(0,1) ρ 0 ∈ ( 0 , 1 ) ; factors a ρ < 1 ≤ a h a_\rho<1\leq a_h a ρ < 1 ≤ a h ; number of steps k ∗ k_* k ∗ ; constants h r e s c a l e , c ρ h_{\mathrm{rescale}},c_\rho h rescale , c ρ
Output: field of tangent spaces x ↦ P ^ k ∗ ( x ) x\mapsto\widehat{P}_{k_*}(x) x ↦ P k ∗ ( x ) ; regression estimate x ↦ F ^ ( x ) x\mapsto\widehat{F}(x) x ↦ F ( x )
// Phase 1: warm start by multiscale local PCA
for i = 1 , … , n i=1,\ldots,n i = 1 , … , n do
U i ← { X j : ∥ Y j − Y i ∥ ≤ δ Y } U_i \leftarrow\{X_j:\|Y_j-Y_i\|\leq\delta_Y\} U i ← { X j : ∥ Y j − Y i ∥ ≤ δ Y } \hfill (response preimage)
for r ∈ R r\in\mathcal{R} r ∈ R do
Ω i ( r ) ← { j : X j ∈ U i , ∥ X j − X i ∥ ≤ r } \Omega_i(r)\leftarrow\{j:X_j\in U_i,\|X_j-X_i\|\leq r\} Ω i ( r ) ← { j : X j ∈ U i , ∥ X j − X i ∥ ≤ r } ;
Σ ^ i ( r ) ← \widehat{\Sigma}_i(r)\leftarrow Σ i ( r ) ← covariance of { X j } j ∈ Ω i ( r ) \{X_j\}_{j\in\Omega_i(r)} { X j } j ∈ Ω i ( r ) ;
r i ← r_i\leftarrow r i ← smallest maximizer of r ↦ H ( Σ ^ i ( r ) ) r\mapsto\mathcal{H}(\widehat{\Sigma}_i(r)) r ↦ H ( Σ i ( r )) ;
V i ← V_i\leftarrow V i ← eigenvectors of the m m m smallest eigenvalues of Σ ^ i ( r i ) \widehat{\Sigma}_i(r_i) Σ i ( r i ) ;
for i = 1 , … , n i=1,\ldots,n i = 1 , … , n do
P ^ p i l ( X i ) ← P T o p m ( ∣ Ω i ( r i ) ∣ − 1 ∑ j ∈ Ω i ( r i ) V j V j T ) \widehat{P}_{\mathrm{pil}}(X_i)\leftarrow P_{\mathrm{Top}m}\left(|\Omega_i(r_i)|^{-1}\sum_{j\in\Omega_i(r_i)}V_jV_j^{\mathsf{T}}\right) P pil ( X i ) ← P Top m ( ∣ Ω i ( r i ) ∣ − 1 ∑ j ∈ Ω i ( r i ) V j V j T ) \hfill (pool at anchor i i i )
h 0 ← h r e s c a l e ⋅ n − 1 ∑ i r i h_0\leftarrow h_{\mathrm{rescale}}\cdot n^{-1}\sum_i r_i h 0 ← h rescale ⋅ n − 1 ∑ i r i \hfill (average selected radius, rescaled)
// Phase 2: centerwise local fits and refinement
For each target x x x , set P ^ 0 ( x ) ← P S \widehat{P}_0(x)\leftarrow P_S P 0 ( x ) ← P S for its assigned anchor star S S S (Appendix § 7 .§ 1 );
for k = 1 , … , k ∗ k=1,\ldots,k_* k = 1 , … , k ∗ do
ρ k ← a ρ ρ k − 1 , h k ← a h h k − 1 , b k ← ρ k h k \rho_k\leftarrow a_\rho\rho_{k-1},\quad h_k\leftarrow a_hh_{k-1},\quad b_k\leftarrow\rho_kh_k ρ k ← a ρ ρ k − 1 , h k ← a h h k − 1 , b k ← ρ k h k ;
for j = 1 , … , n j=1,\ldots,n j = 1 , … , n do
G ^ j , k ← slope \widehat{G}_{j,k}\leftarrow\operatorname{slope} G j , k ← slope of min β 0 , G ∑ ℓ K ( d P ^ k − 1 ( X j ) , b k , h k ( X ℓ − X j ) 2 ) ∥ Y ℓ − β 0 − G T ( X ℓ − X j ) ∥ 2 \min_{\beta_0,G}\sum_\ell K\left(d_{\widehat{P}_{k-1}(X_j),b_k,h_k}(X_\ell-X_j)^2\right)\|Y_\ell-\beta_0-G^{\mathsf{T}}(X_\ell-X_j)\|^2 min β 0 , G ∑ ℓ K ( d P k − 1 ( X j ) , b k , h k ( X ℓ − X j ) 2 ) ∥ Y ℓ − β 0 − G T ( X ℓ − X j ) ∥ 2 ;
A ^ j , k ← G ^ j , k G ^ j , k T \widehat{A}_{j,k}\leftarrow\widehat{G}_{j,k}\widehat{G}_{j,k}^{\mathsf{T}} A j , k ← G j , k G j , k T \hfill (store once per center)
for each target x x x (data points and queries) do
W x , k ← { u : d P ^ k − 1 ( x ) , b k , h k ( u − x ) ≤ 1 } W_{x,k}\leftarrow\{u:d_{\widehat{P}_{k-1}(x),b_k,h_k}(u-x)\leq1\} W x , k ← { u : d P k − 1 ( x ) , b k , h k ( u − x ) ≤ 1 } ;
I x , k ← { j : X j ∈ W x , k } I_{x,k}\leftarrow\{j:X_j\in W_{x,k}\} I x , k ← { j : X j ∈ W x , k } ;
P ^ k ( x ) ← P T o p m ( ∣ I x , k ∣ − 1 ∑ j ∈ I x , k A ^ j , k ) \widehat{P}_k(x)\leftarrow P_{\mathrm{Top}m}\left(|I_{x,k}|^{-1}\sum_{j\in I_{x,k}}\widehat{A}_{j,k}\right) P k ( x ) ← P Top m ( ∣ I x , k ∣ − 1 ∑ j ∈ I x , k A j , k ) ;
// Phase 3: prediction at a query point x x x
ρ ∗ ← c ρ ρ k ∗ , h ∗ ← h k ∗ , b ∗ ← ρ ∗ h ∗ \rho_*\leftarrow c_\rho\rho_{k_*},\quad h_*\leftarrow h_{k_*},\quad b_*\leftarrow\rho_*h_* ρ ∗ ← c ρ ρ k ∗ , h ∗ ← h k ∗ , b ∗ ← ρ ∗ h ∗ ;
ϖ i ( x ) ← K ( h ∗ − 2 ∥ x − X i ∥ 2 + b ∗ − 2 ∥ P ^ k ∗ ( X i ) ( x − X i ) ∥ 2 ) , i = 1 , … , n \varpi_i(x)\leftarrow K\left(h_*^{-2}\|x-X_i\|^2+b_*^{-2}\|\widehat{P}_{k_*}(X_i)(x-X_i)\|^2\right),\quad i=1,\ldots,n ϖ i ( x ) ← K ( h ∗ − 2 ∥ x − X i ∥ 2 + b ∗ − 2 ∥ P k ∗ ( X i ) ( x − X i ) ∥ 2 ) , i = 1 , … , n ;
F ^ ( x ) ← ∑ i ϖ i ( x ) Y i / ∑ i ϖ i ( x ) \widehat{F}(x)\leftarrow\sum_i\varpi_i(x)Y_i/\sum_i\varpi_i(x) F ( x ) ← ∑ i ϖ i ( x ) Y i / ∑ i ϖ i ( x ) ; \hfill (prediction-fold mean if the denominator is zero)
version analysed in the theorem, k ∗ k_* k ∗ , ρ 0 \rho_0 ρ 0 , h r e s c a l e h_{\mathrm{rescale}} h rescale , the Phase 1 filters and the local cylinder are set from m m m , δ Y \delta_Y δ Y , the radii R \mathcal{R} R and the model scales R R R , σ N \sigma_N σ N , r s i n g r_{\mathrm{sing}} r sing , w Y w_Y w Y , p 0 p_0 p 0 , π − \pi_- π − , σ ζ \sigma_\zeta σ ζ and σ f , + \sigma_{f,+} σ f , + of § 3.1 , together with fixed multipliers.
Computational cost. Phase 1 costs O ( n 2 p + n ∣ R ∣ ( n D 2 + D 3 ) ) O(n^2p+n|\mathcal{R}|(nD^2+D^3)) O ( n 2 p + n ∣ R ∣ ( n D 2 + D 3 )) using direct covariance computations and dense eigendecompositions. In Phase 2, each center is fitted once, at cost O ( N ′ ( D 2 + D p ) + D 3 + D 2 p ) O(N'(D^2+Dp)+D^3+D^2p) O ( N ′ ( D 2 + D p ) + D 3 + D 2 p ) for N ′ N' N ′ local observations; its slope outer product is reused across pools. At the terminal scale, N ′ N' N ′ is of order n 2 s / ( 2 s + m ) n^{2s/(2s+m)} n 2 s / ( 2 s + m ) up to logarithms, while earlier fits may use N ′ ≤ n N'\leq n N ′ ≤ n . Direct neighborhood searches and pooling add O ( n 2 D 2 ) O(n^2D^2) O ( n 2 D 2 ) per iteration. Theorem 3.5 gives the full training bound, which for fixed dimensions and k ∗ = O ( log n ) k_*=O(\log n) k ∗ = O ( log n ) is O ( ( ∣ R ∣ + log n ) n 2 ) O((|\mathcal{R}|+\log n)n^2) O (( ∣ R ∣ + log n ) n 2 ) . With tangent bases stored as D × m D\times m D × m matrices, prediction at one test point costs O ( n ( D m + p ) ) O(n(Dm+p)) O ( n ( D m + p )) .
Assumptions and main results
The link differential D f Df D f may be rank deficient at finitely many points. Appendix § 7 gives the proofs in the order of the three phases.
Role of the assumptions. (A1) sets the reach geometry and tube ratio; (A2) specifies how the full normal fibers are sampled and how they affect tangent coordinates; (A3) supplies link smoothness and local rank; (A4) localizes the response preimage within the scan; and (A5) matches the response window to the noise and normal-variance scales.
Conventions. The confidence exponent τ > 0 \tau> 0 τ > 0 is fixed; a quantity is called fixed if it does not depend on n n n . Write ∥ M ∥ F = ( ∑ i , j M i j 2 ) 1 / 2 \lVert M\rVert_{F} = \left(\sum_{i,j} M_{ij}^{2}\right)^{1/2} ∥ M ∥ F = ( ∑ i , j M ij 2 ) 1/2 for the Frobenius norm of a matrix M M M , σ j ( M ) \sigma_j(M) σ j ( M ) for its j j j th largest singular value, σ min ( M ) \sigma_{\min}(M) σ m i n ( M ) for its smallest singular value, and λ min ( M ) \lambda_{\min}(M) λ m i n ( M ) for the smallest eigenvalue of a symmetric matrix. For random vectors, Cov ( U , V ) = E [ ( U − E U ) ( V − E V ) T ] \operatorname{Cov}(U,V) = \mathbb{E}[(U-\mathbb{E}U)(V-\mathbb{E}V)^{\mathsf{T}}] Cov ( U , V ) = E [( U − E U ) ( V − E V ) T ] and Cov ( U ) = Cov ( U , U ) \operatorname{Cov}(U) = \operatorname{Cov}(U,U) Cov ( U ) = Cov ( U , U ) . The symbol 1 E \mathbf{1}_{E} 1 E denotes the indicator of a set or event E E E , and ∣ S ∣ \lvert S\rvert ∣ S ∣ is the cardinality of a finite set S S S . Write B d ( r ) = B R d ( 0 , r ) B_d(r) = B_{\mathbb{R}^{d}}(0,r) B d ( r ) = B R d ( 0 , r ) for the Euclidean ball in R d \mathbb{R}^{d} R d centered at the origin with radius r r r . For a scalar random variable U U U , write ∥ U ∥ ψ 2 = inf { t > 0 : E exp ( U 2 / t 2 ) ≤ 2 } \lVert U\rVert_{\psi_2} = \inf\{t > 0 : \mathbb{E}\exp(U^2/t^2) \le2\} ∥ U ∥ ψ 2 = inf { t > 0 : E exp ( U 2 / t 2 ) ≤ 2 } . The letter C C C denotes positive constants that depend only on the fixed model parameters, the kernel and τ \tau τ , and may change from line to line; it never depends on n n n , D D D , p p p . Lower bounds are written with C − 1 C^{-1} C − 1 . We write a ≲ b a \lesssim b a ≲ b for a ≤ C b a \le Cb a ≤ C b , and a ≍ b a \asymp b a ≍ b when both a ≲ b a \lesssim b a ≲ b and b ≲ a b \lesssim a b ≲ a . Subscripts on C C C , ≲ \lesssim ≲ or O O O indicate additional allowed constant dependence. The geometric constants depend on m m m , s s s , c c c , the dimensionless chart and packet-diameter bounds, the design constants in (A2), and the link bounds in (A3), including c o p c_{\mathrm{op}} c op in (17) . In particular, the population pilot constant and the multipliers used to choose the initial widths are independent of p 0 p_0 p 0 in (3.7), which enters the pilot sample requirement explicitly. The constants in Propositions 3.3 and 3.4 and in Theorem 3.5 may also depend on the ratio of σ N \sigma_N σ N to the pilot base resolution w Y w_Y w Y of (A4). Local mass, response resolution and the units for each statistical bound are specified below.
Assumptions
The five assumptions below specify the conditions for Theorem 3.5 and Propositions 3.1 –3.4 . § 4 also studies link functions and designs beyond these conditions.
The tube ratio c c c , the smoothness exponent, the chart bounds, the design bounds in (A2), packet-diameter bounds and constants in (17) are independent of n n n , D D D , p p p . The dimensional quantities below are in the original predictor and response units.
(A1) Geometry in reach units. D > m ≥ 1 D > m \ge1 D > m ≥ 1 and M ⊂ R D \mathcal{M} \subset\mathbb{R}^{D} M ⊂ R D is a compact C 3 C^{3} C 3 submanifold of dimension m m m without boundary, with reach M > 0 \operatorname{reach}_{\mathcal{M}} > 0 reach M > 0 . Put R = reach M R = \operatorname{reach}_{\mathcal{M}} R = reach M with a fixed 0 < c < 2 / 3 0 < c < 2/3 0 < c < 2/3 . In charts of radius reach M / 5 \operatorname{reach}_{\mathcal{M}}/5 reach M /5 , write the manifold as q + w + g ( w ) q+w+g(w) q + w + g ( w ) for w ∈ T q M w \in T_q\mathcal{M} w ∈ T q M , with g ( w ) ∈ N q M g(w) \in N_q\mathcal{M} g ( w ) ∈ N q M . The graph map g g g satisfies ∥ D g ∥ ≤ 1 / 4 \lVert Dg\rVert\le1/4 ∥ D g ∥ ≤ 1/4 , and the dimensionless chart bound
reach M ∥ D 2 g ∥ + reach M 2 ∥ D 3 g ∥ \operatorname{reach}_{\mathcal{M}}\lVert D^2g\rVert+ \operatorname{reach}_{\mathcal{M}}^2\lVert D^3g\rVert reach M ∥ D 2 g ∥ + reach M 2 ∥ D 3 g ∥
is at most a constant independent of n n n , D D D , p p p .
(A2) Sampling the normal fibers. The base point Q Q Q has a C 1 C^1 C 1 density π \pi π with respect to manifold volume. Write π − = inf M π > 0 \pi_- = \inf_{\mathcal{M}}\pi> 0 π − = inf M π > 0 . Write μ q \mu_q μ q for the conditional law of Z Z Z given Q = q Q=q Q = q . It is centered and has support the full normal ball of radius R R R . Its covariance Σ N ( q ) \Sigma_N(q) Σ N ( q ) , restricted to N q M N_q\mathcal{M} N q M , has eigenvalues in [ σ N 2 , C N σ N 2 ] [\sigma_N^2,C_N\sigma_N^2] [ σ N 2 , C N σ N 2 ] , where σ N > 0 \sigma_N>0 σ N > 0 is the normal standard-deviation scale and C N ≥ 1 C_N\ge1 C N ≥ 1 is fixed, with σ N 2 ≍ R 2 / ( D − m + 2 ) \sigma_N^2\asymp R^2/(D-m+2) σ N 2 ≍ R 2 / ( D − m + 2 ) . As an ambient operator, Σ N ( q ) \Sigma_N(q) Σ N ( q ) is extended by zero on T q M T_q\mathcal{M} T q M . The ratio sup M π / π − \sup_{\mathcal{M}}\pi/\pi_- sup M π / π − and the scaled Riemannian gradient σ N ∥ ∇ M log π ∥ \sigma_N\lVert\nabla_{\mathcal{M}}\log\pi\rVert σ N ∥ ∇ M log π ∥ are bounded. For unit normals e e e and scalar Lipschitz functions ϕ \phi ϕ , with Lipschitz seminorm Lip ( ϕ ) \operatorname{Lip}(\phi) Lip ( ϕ ) ,
E μ q ∣ ⟨ e , Z ⟩ ∣ 4 ≤ C N σ N 4 , Var μ q ϕ ( Z ) ≤ C N σ N 2 Lip ( ϕ ) 2 . (13) \mathbb{E}_{\mu_q}|\langle e,Z\rangle|^4\le C_N\sigma_N^4,\qquad\operatorname{Var}_{\mu_q}\phi(Z)\le C_N\sigma_N^2\operatorname{Lip}(\phi)^2. \tag*{(13)} E μ q ∣ ⟨ e , Z ⟩ ∣ 4 ≤ C N σ N 4 , Var μ q ϕ ( Z ) ≤ C N σ N 2 Lip ( ϕ ) 2 . (13)
The law may vary along the manifold. If d M ( q , q ′ ) ≤ σ N / 8 d_{\mathcal{M}}(q,q')\le\sigma_N/8 d M ( q , q ′ ) ≤ σ N /8 , identify the two normal spaces by the isometry given by the polar factor of orthogonal projection from N q M N_q\mathcal{M} N q M to N q ′ M N_{q'}\mathcal{M} N q ′ M . The law μ q ′ \mu_{q'} μ q ′ pulled back by this isometry, denoted by μ ~ q ′ \widetilde{\mu}_{q'} μ q ′ , satisfies
∥ log d μ ~ q ′ d μ q ∥ L ∞ ( μ q ) ≤ C N d M ( q , q ′ ) / σ N . (14) \left\lVert\log\frac{d\widetilde{\mu}_{q'}}{d\mu_q}\right\rVert_{L^\infty(\mu_q)}\le C_N d_{\mathcal{M}}(q,q')/\sigma_N. \tag*{(14)} log d μ q d μ q ′ L ∞ ( μ q ) ≤ C N d M ( q , q ′ ) / σ N . (14)
Thus local changes in the sampling weights are measured on the scale of the spread in one normal direction. Write λ ( q ) = σ m ( D f ( q ) ) \lambda(q)=\sigma_m(Df(q)) λ ( q ) = σ m ( D f ( q )) for the smallest singular value of the differential of the link and Σ s i n g = { q ∈ M : λ ( q ) = 0 } \Sigma_{\mathrm{sing}}=\{q\in\mathcal{M}:\lambda(q)=0\} Σ sing = { q ∈ M : λ ( q ) = 0 } for the set of points where D f Df D f is rank deficient. Let r s i n g r_{\mathrm{sing}} r sing , the singular-chart radius , be a radius with r s i n g / σ N r_{\mathrm{sing}}/\sigma_N r sing / σ N a fixed positive ratio at most 1 / 8 1/8 1/8 when Σ s i n g \Sigma_{\mathrm{sing}} Σ sing is nonempty. The regular region is { q : d M ( q , Σ s i n g ) ≥ r s i n g / 2 } \{q:d_{\mathcal{M}}(q,\Sigma_{\mathrm{sing}})\ge r_{\mathrm{sing}}/2\} { q : d M ( q , Σ sing ) ≥ r sing /2 } , and the regular condition number κ f \kappa_f κ f of D f Df D f is the supremum of σ 1 ( D f ( q ) ) / σ m ( D f ( q ) ) \sigma_1(Df(q))/\sigma_m(Df(q)) σ 1 ( D f ( q )) / σ m ( D f ( q )) over it. On the regular region, define the root mean square tangential stretch
ω b e n d : = sup q i n t h i s r e g i o n , v ∈ T q M ∥ v ∥ = 1 ( E μ q ⟨ I I q ( v , v ) , Z ⟩ 2 ) 1 / 2 . (15) \omega_{\mathrm{bend}}:=\sup_{\substack{q\ \mathrm{in\ this\ region},\,v\in T_q\mathcal{M}\\ \lVert v\rVert=1}}\left(\mathbb{E}_{\mu_q}\langle\mathrm{II}_q(v,v),Z\rangle^2\right)^{1/2}. \tag*{(15)} ω bend := q in this region , v ∈ T q M ∥ v ∥ = 1 sup ( E μ q ⟨ II q ( v , v ) , Z ⟩ 2 ) 1/2 . (15)
Let C r e g C_{\mathrm{reg}} C reg be an admissible constant in (A.33), depending only on m m m , c c c , C N C_N C N and κ f \kappa_f κ f (Remark A.13), and recall a ρ = e − 1 / [ 2 ( m + 4 ) ] a_\rho=e^{-1/[2(m+4)]} a ρ = e − 1/ [ 2 ( m + 4 )] for s = 2 s=2 s = 2 . Assume that the stretch is small:
c + C r e g ω b e n d a ρ 2 + C r e g ω b e n d a ρ 4 < 1 , (16) \frac{c+C_{\mathrm{reg}}\omega_{\mathrm{bend}}}{a_\rho^2}+\frac{C_{\mathrm{reg}}\omega_{\mathrm{bend}}}{a_\rho^4}<1, \tag*{(16)} a ρ 2 c + C reg ω bend + a ρ 4 C reg ω bend < 1 , (16)
with a gap independent of n n n , D D D , p p p . For each fixed c < 2 / 3 < a 2 2 c<2/3<a_2^2 c < 2/3 < a 2 2 , this permits sufficiently small positive ω b e n d \omega_{\mathrm{bend}} ω bend , which measures curvature times the normal spread.
(A3) Isolated rank deficiency of D f Df D f . The link f : M → R p f:\mathcal{M}\to\mathbb{R}^p f : M → R p , p ≥ m p\ge m p ≥ m , is C 1 , 1 C^{1,1} C 1 , 1 . Its differential satisfies ∥ D f ∥ ≤ σ f , + \lVert Df\rVert\le\sigma_{f,+} ∥ D f ∥ ≤ σ f , + and is Lipschitz in parallel frames, with constant at most C σ f , + / σ N C\sigma_{f,+}/\sigma_N C σ f , + / σ N . The set Σ s i n g \Sigma_{\mathrm{sing}} Σ sing of points where D f Df D f is rank deficient, defined in (A2), is finite, with cardinality bounded independently of n n n , D D D and p p p . The balls B M ( q 0 , 2 r s i n g ) B_{\mathcal{M}}(q_0,2r_{\mathrm{sing}}) B M ( q 0 , 2 r sing ) , q 0 ∈ Σ s i n g q_0\in\Sigma_{\mathrm{sing}} q 0 ∈ Σ sing , which we call the singular charts , are pairwise disjoint. Since r s i n g ≤ σ N / 8 r_{\mathrm{sing}}\le\sigma_N/8 r sing ≤ σ N /8 and σ N ≤ R \sigma_N\le R σ N ≤ R , each singular chart has radius at most R / 4 R/4 R /4 , less than the chart radius reach M / 5 \operatorname{reach}_{\mathcal{M}}/5 reach M /5 of (A1). For a fixed constant c o p > 0 c_{\mathrm{op}}>0 c op > 0 and every q ∈ M q\in\mathcal{M} q ∈ M ,
λ ( q ) ≥ c o p σ f , + min { 1 , d M ( q , Σ s i n g ) σ N } , (17) \lambda(q)\ge c_{\mathrm{op}}\sigma_{f,+}\min\left\{1,\frac{d_{\mathcal{M}}(q,\Sigma_{\mathrm{sing}})}{\sigma_N}\right\}, \tag*{(17)} λ ( q ) ≥ c op σ f , + min { 1 , σ N d M ( q , Σ sing ) } , (17) and at each q 0 ∈ Σ s i n g q_0\in\Sigma_{\mathrm{sing}} q 0 ∈ Σ sing the nonzero singular values of D f ( q 0 ) Df(q_0) D f ( q 0 ) are at least c o p σ f , + c_{\mathrm{op}}\sigma_{f,+} c op σ f , + . Thus λ \lambda λ is at least a fixed multiple of σ f , + \sigma_{f,+} σ f , + outside the balls of radius r s i n g / 2 r_{\mathrm{sing}}/2 r sing /2 , and the directions in
ker D f ( q 0 ) \operatorname{ker} Df(q_0) ker D f ( q 0 ) become visible linearly with distance inside the singular charts. At each q 0 q_0 q 0 the differential has a linear expansion in parallel frames. For a common modulus ω ( t ) → 0 \omega(t) \to0 ω ( t ) → 0 as t ↓ 0 t \downarrow0 t ↓ 0 , its remainder at distance t t t is bounded by ( σ f , + / σ N ) t ω ( t / σ N ) (\sigma_{f,+}/\sigma_N)t\omega(t/\sigma_N) ( σ f , + / σ N ) t ω ( t / σ N ) . This requires a second differential at the singular points only.
In each singular chart the tangential derivative of F = f ∘ Π M F=f\circ\Pi_{\mathcal{M}} F = f ∘ Π M varies along a normal fiber by at most a fixed fraction ε s i n g > 0 \varepsilon_{\mathrm{sing}}>0 ε sing > 0 of the weakest tangent signal:
sup z ∈ N q M , ∥ z ∥ ≤ R ∥ D f ( q ) ( I − S q , z ) − 1 − D f ( q ) ∥ ≤ ε s i n g λ ( q ) . (18) \sup_{z\in N_q\mathcal{M},\,\lVert z\rVert\le R}\left\lVert Df(q)(I-S_{q,z})^{-1}-Df(q)\right\rVert\le\varepsilon_{\mathrm{sing}}\lambda(q). \tag*{(18)} z ∈ N q M , ∥ z ∥ ≤ R sup D f ( q ) ( I − S q , z ) − 1 − D f ( q ) ≤ ε sing λ ( q ) . (18)
At q 0 q_0 q 0 this means D f ( q 0 ) S q 0 , z = 0 Df(q_0)S_{q_0,z}=0 D f ( q 0 ) S q 0 , z = 0 ; at nearby regular points it implies ∥ S q , z ∥ ≤ ( 1 + c ) ε s i n g \lVert S_{q,z}\rVert\le(1+c)\varepsilon_{\mathrm{sing}} ∥ S q , z ∥ ≤ ( 1 + c ) ε sing . Thus (18) restricts the shape operators throughout each singular chart. The constant ε s i n g \varepsilon_{\mathrm{sing}} ε sing depends on the bounds in (17) and the link, chart and design bounds as in Appendix § 7 .§ 4.3 . When D f Df D f has full rank everywhere, Σ s i n g \Sigma_{\mathrm{sing}} Σ sing is empty and (17) reads λ ≥ c o p σ f , + \lambda\ge c_{\mathrm{op}}\sigma_{f,+} λ ≥ c op σ f , + on M \mathcal{M} M ; set r s i n g = reach M r_{\mathrm{sing}}=\operatorname{reach}_{\mathcal{M}} r sing = reach M in the radius r c y l r_{\mathrm{cyl}} r cyl of the local cylinder (A5).
(A4) Response packets visible to the scan. Fix a packet-diameter constant C p k t > 0 C_{\mathrm{pkt}}>0 C pkt > 0 , the response-window width δ Y \delta_Y δ Y of the scan and a base width w Y w_Y w Y . These are the response and base resolutions of the pilot scan: by (20) below, base points within the scan radius whose true responses differ by at most 2 δ Y 2\delta_Y 2 δ Y lie within C p k t w Y C_{\mathrm{pkt}}w_Y C pkt w Y of one another on M \mathcal{M} M . Assume there is p 0 > 0 p_0>0 p 0 > 0 such that, for every base point q q q ,
P { d M ( Q , q ) ≤ w Y , ∥ f ( Q ) − f ( q ) ∥ ≤ δ Y / 4 } ≥ p 0 . (19) \mathbb{P}\left\{d_{\mathcal{M}}(Q,q)\le w_Y,\ \lVert f(Q)-f(q)\rVert\le\delta_Y/4\right\}\ge p_0. \tag*{(19)} P { d M ( Q , q ) ≤ w Y , ∥ f ( Q ) − f ( q )∥ ≤ δ Y /4 } ≥ p 0 . (19)
Each scan radius lies strictly between max { 2.05 R , 2 R + 2 C p k t w Y } \max\{2.05R,2R+2C_{\mathrm{pkt}}w_Y\} max { 2.05 R , 2 R + 2 C pkt w Y } and 3 reach M / 2 3\operatorname{reach}_{\mathcal{M}}/2 3 reach M /2 . For base points visible to the scan assume
\begin{aligned}
\lVert q'-q\rVert&\le\max\{R+2R\},\\
\lVert f(q')-f(q)\rVert&\le2\delta_Y
\end{aligned}
\quad\Longrightarrow\quad d_{\mathcal{M}}(q',q)\le C_{\mathrm{pkt}}w_Y. \tag*{(20)}\end{aligned}
Thus equal responses may occur on spatially separated packets. The diameter constant C p k t C_{\mathrm{pkt}} C pkt is fixed before choosing the resolution. After choosing w Y w_Y w Y and δ Y \delta_Y δ Y , the probability lower bound p 0 p_0 p 0 is determined; in units of the local mass π M − w Y \pi^{-w_Y}_{\mathcal{M}} π M − w Y it need not remain bounded below as the resolution decreases. Proposition 3.1 uses conditional moments, whose bounds are independent of p 0 p_0 p 0 .
(A5) Response noise and pilot resolution. The noise vectors are centered, independent of one another and of the predictors, with ∥ ⟨ v , ζ i ⟩ ∥ ψ 2 ≤ σ ζ \lVert\langle v,\zeta_i\rangle\rVert_{\psi_2}\le\sigma_\zeta ∥⟨ v , ζ i ⟩ ∥ ψ 2 ≤ σ ζ for unit v v v and bounded σ ζ / δ Y \sigma_\zeta/\delta_Y σ ζ / δ Y . Set r c y l = γ c y l r s i n g r_{\mathrm{cyl}}=\gamma_{\mathrm{cyl}}r_{\mathrm{sing}} r cyl = γ cyl r sing , the tangential radius of the local cylinder of § 3.3 , where 0 < γ c y l < 1 0<\gamma_{\mathrm{cyl}}<1 0 < γ cyl < 1 is a small fixed geometric multiplier, and let ε 0 > 0 \varepsilon_0>0 ε 0 > 0 be the fixed resolution margin. At the pilot resolution assume
∥ ζ i ∥ ≤ δ Y / 8 a.s. for all pilot observations , w Y ≤ ε 0 min { σ N , r c y l } . (21) \begin{aligned}
\lVert\zeta_i\rVert&\le\delta_Y/8\quad\text{a.s. for all pilot observations},\\
w_Y&\le\varepsilon_0\min\{\sigma_N,r_{\mathrm{cyl}}\}. \tag*{(21)}
\end{aligned} ∥ ζ i ∥ w Y ≤ δ Y /8 a.s. for all pilot observations , ≤ ε 0 min { σ N , r cyl } . (21)
The resolution margin ε 0 \varepsilon_0 ε 0 is chosen after the chart and kernel bounds, c o p c_{\mathrm{op}} c op in (17) , and pilot multipliers (Appendix § 7 .§ 4.3 ); it also depends on the constants for Phase 2 chosen there, on the gap in (16) and on the modulus ω \omega ω of (A3). Neither noise bound depends on n n n or D D D .
In Phase 1 the pilot noise bound keeps the true responses of an anchor and each observation it selects within 2 δ Y 2\delta_Y 2 δ Y of one another, as required by (A4). The pilot blocks are reserved before sampling, and their random split into anchor and design folds is independent of the data. The response-window width δ Y \delta_Y δ Y is fixed as n n n grows. Concentration for refinement and prediction uses the directional sub-Gaussian bound.
Together with (A4), the almost-sure bound is a signal-to-noise condition. At a regular base point, (3.8) applied to base points at intrinsic distance just above C p k t w Y C_{\mathrm{pkt}}w_Y C pkt w Y forces 2 δ Y ≤ σ f , + C p k t w Y 2\delta_Y \le\sigma_{f,+}C_{\mathrm{pkt}}w_Y 2 δ Y ≤ σ f , + C pkt w Y , so the pilot noise norm is at most a fixed fraction, of order C p k t ϵ 0 C_{\mathrm{pkt}}\epsilon_0 C pkt ϵ 0 , of the tangential response scale σ f , + σ N \sigma_{f,+}\sigma_N σ f , + σ N . Near a point where D f Df D f is rank deficient the admissible fraction is smaller, of order ( C p k t ϵ 0 ) 2 ω ( C p k t ϵ 0 ) (C_{\mathrm{pkt}}\epsilon_0)^2\omega(C_{\mathrm{pkt}}\epsilon_0) ( C pkt ϵ 0 ) 2 ω ( C pkt ϵ 0 ) , as for the cusp model in Remark A.16. For fixed packet and resolution margins and link modulus, these fractions are independent of D D D . The corresponding bounds in response units scale with σ f , + σ N \sigma_{f,+}\sigma_N σ f , + σ N , where σ N ≍ R / D − m + 2 \sigma_N \asymp R/\sqrt{D-m+2} σ N ≍ R / D − m + 2 . At fixed R R R and σ f , + \sigma_{f,+} σ f , + this scale decreases with codimension. For isotropic Gaussian noise the p \sqrt{p} p term in (A.41) also limits the per-coordinate standard deviation at a given response resolution.
Units and local mass. The width σ N \sigma_N σ N measures displacement in one normal direction. Taking R = c reach M ≍ D − m R = c\,\operatorname{reach}_{\mathcal{M}} \asymp\sqrt{D-m} R = c reach M ≍ D − m in physical predictor units keeps this variance of order one as D D D increases. The response windows and a positive sub-Gaussian noise scale may then be fixed in physical response units. If p p p varies, (A5) still controls the norm of the response noise at the chosen resolution. Under X ↦ a X X X \mapsto a_X X X ↦ a X X and Y ↦ a Y Y Y \mapsto a_Y Y Y ↦ a Y Y , the two sides of (3.6) and of (3.5) scale by a Y / a X a_Y/a_X a Y / a X . The fiber-law ratio in (3.2), the noise relative to the response windows, the normalized risk and the local probabilities π − b m \pi_{-b}^m π − b m are unchanged. Since π \pi π integrates to one against manifold volume and sup M π / π − m \sup_{\mathcal{M}}\pi/\pi_-^m sup M π / π − m is bounded, π − σ N m ≍ σ N m / vol ( M ) \pi_{-\sigma_N}^m \asymp\sigma_N^m/\operatorname{vol}(\mathcal{M}) π − σ N m ≍ σ N m / vol ( M ) : the local mass is the inverse volume of M \mathcal{M} M in units of σ N m \sigma_N^m σ N m . The size of M \mathcal{M} M enters the bounds only through this ratio; for connected M \mathcal{M} M the response diameter diam ( f ( M ) ) \operatorname{diam}(f(\mathcal{M})) diam ( f ( M )) in the last term of (3.17) is also bounded by it (Appendix A.7).
Choice of scales
The normal width starts at a fixed multiple of the mean selected radius. Its multiplier keeps h k / R h_k/R h k / R above the minimum normal width (in units of R R R ) in Lemma A.12 and below a fixed upper bound. The initial tangential width b 0 b_0 b 0 is comparable to a sufficiently large fixed multiple of w Y w_Y w Y , so the pilot tilt fits within b 1 b_1 b 1 . The local cylinder of Section 3.3 keeps the base points of the observations used at a target in one graph chart of (A1), even when h k h_k h k exceeds the reach. Thereafter b k b_k b k shrinks geometrically. Refinement stops at the finest width for which the local count n f π − b k m n_f\pi_{-b_k}^m n f π − b k m times ( b k / σ N ) 4 (b_k/\sigma_N)^4 ( b k / σ N ) 4 covers the sampling cost in (A.1). Appendix A.1 gives the integer rule, and Appendix A.6 the sample size needed to start it.
The estimator used in the theorem
The three phases use the scales of Section 3.2. We specify here which observations enter each step and how the pilot restricts a neighborhood to one chart. Appendix A.1 gives the thresholds for the anchor stars and the filters for degenerate fits.
Sample splitting. The Phase 1 observations are split uniformly at random between the design fold S d e s \mathcal{S}_{\mathrm{des}} S des , which selects the scan radii, and the anchor fold S a n c \mathcal{S}_{\mathrm{anc}} S anc , which supplies the projectors to be pooled. Each refinement step k k k uses fresh fit and pool folds S k f i t , S k p o o l S_k^{\mathrm{fit}}, S_k^{\mathrm{pool}} S k fit , S k pool , and prediction uses a separate fold S p r e d S^{\mathrm{pred}} S pred of size n p r e d n_{\mathrm{pred}} n pred (Appendix § 7 .§ 1 ).
Phase 1. For each anchor in the anchor fold, the scan uses the design-fold observations in U i ∩ B ( X i , r ) U_i \cap B(X_i,r) U i ∩ B ( X i , r ) . It retains radii passing the count and covariance filters of Appendix § 7 .§ 1 and chooses the smallest maximizer of (11) . An anchor is eligible if at least one radius is retained. Each eligible anchor stores the mean of the observations selected at its chosen radius, its selected mean , and a pooled projector ((A.2), obtained by averaging the projectors onto the bottom m m m covariance eigenspaces of nearby eligible anchors.
Anchor stars and the local cylinder. Two eligible anchors are linked if their responses and their selected means are close, at the thresholds of Appendix § 7 .§ 1 . An anchor star S S S consists of an eligible anchor and the eligible anchors linked to it. There is one such star for each eligible anchor, and the stars may overlap. Its center x S x_S x S is the average of their selected means, and P S P_S P S projects onto the leading m m m eigenspace of the average of their pooled projectors, so that x S + Ran P S x_S+\operatorname{Ran}P_S x S + Ran P S estimates a tangent plane. Each target x x x is assigned an anchor star minimizing the tangential residual ∥ P S ( x − x S ) ∥ \lVert P_S(x-x_S)\rVert ∥ P S ( x − x S )∥ among the anchor stars with ∥ x − x S ∥ ≤ 2 R \lVert x-x_S\rVert\le2R ∥ x − x S ∥ ≤ 2 R that pass the size threshold of Appendix § 7 .§ 1 , and we set P ^ 0 ( x ) = P S \widehat P_0(x)=P_S P 0 ( x ) = P S .
The neighborhoods of Phases 2 and 3 have normal width at least 3 R 3R 3 R (Appendix § 7 .§ 1 ), which may exceed the reach, so they can contain predictors whose base points lie on a distant part of M \mathcal{M} M (Figure 4 ). They are therefore intersected with the local cylinder of the target x x x , the set of points x ′ x' x ′ with
Figure 4. Local cylinder and pooling neighborhood for a curve in the plane (m = 1 m = 1 m = 1 , D = 2 D = 2 D = 2 ); the drawing is not to scale. Grey band: the predictor tube of radius R R R around M \mathcal{M} M (blue). Orange ellipse: { v : d P ^ k − 1 ( x ) , b k , h k ( v − x ) ≤ 1 } \{v : d_{\widehat{P}_{k-1}(x),b_k,h_k}(v-x) \le1\} { v : d P k − 1 ( x ) , b k , h k ( v − x ) ≤ 1 } , with the metric (10) , centered at the target x x x above q q q . Its normal width h k ≥ 3 R h_k \ge3R h k ≥ 3 R can exceed the reach and admit predictors from a distant part of the tube (red). The dashed rectangle is the local cylinder (22) , oriented by the pilot projector P S P_S P S : its tangential half-width r c y l r_{\mathrm{cyl}} r cyl is measured from x x x , and its normal half-height R + 8 r c y l R + 8r_{\mathrm{cyl}} R + 8 r cyl from x S + Ran P S x_S + \operatorname{Ran} P_S x S + Ran P S . The green region is the pooling neighborhood W x , k W_{x,k} W x , k in (24) , the intersection of the ellipsoid and cylinder; only its intersection with the grey tube contains predictors. It retains complete normal fibers near q q q and excludes the distant red part (Lemma A.11 ). The cylinder is fixed by the pilot, whereas the ellipsoid uses the step-k k k widths and the current projector P ^ k − 1 ( x ) \widehat{P}_{k-1}(x) P k − 1 ( x ) ; their axes are drawn parallel here.
∥ P S ( x ′ − x ) ∥ ≤ r c y l , ∥ P S ⊥ ( x ′ − x S ) ∥ ≤ R + 8 r c y l , ∥ P S ( x ′ − x S ) ∥ ≤ 8 r c y l , (22) \lVert P_S(x'-x)\rVert\le r_{\mathrm{cyl}},\qquad
\lVert P_S^\perp(x'-x_S)\rVert\le R+8r_{\mathrm{cyl}},\qquad
\lVert P_S(x'-x_S)\rVert\le8r_{\mathrm{cyl}}, \tag*{(22)} ∥ P S ( x ′ − x )∥ ≤ r cyl , ∥ P S ⊥ ( x ′ − x S )∥ ≤ R + 8 r cyl , ∥ P S ( x ′ − x S )∥ ≤ 8 r cyl , (22)
where r c y l r_{\mathrm{cyl}} r cyl is the radius fixed in (A5). The first two conditions form a cylinder with tangential radius r c y l r_{\mathrm{cyl}} r cyl measured from the target and normal radius R + 8 r c y l R+8r_{\mathrm{cyl}} R + 8 r cyl measured from the estimated plane x S + Ran P S x_S+\operatorname{Ran}P_S x S + Ran P S . The third bounds the tangential displacement from the star center; on the event of Proposition 3.1 , it follows from the first (Appendix § 7 .§ 3 ). Proposition 3.1 shows that the local cylinder contains the complete normal fibers over B M ( Π M x , r c y l / 4 ) B_{\mathcal{M}}(\Pi_{\mathcal{M}}x,r_{\mathrm{cyl}}/4) B M ( Π M x , r cyl /4 ) and that the points it contains have base points in a single graph chart of (A1).
Phase 2. The normal width h k h_k h k and the aspect ratio ρ k \rho_k ρ k follow the multiplicative schedule, with per-step factors a h a_h a h and a ρ a_\rho a ρ ,
h 0 = h r e s c a l e avg i e l i g i b l e r i , h k = a h h k − 1 , ρ k = a ρ ρ k − 1 , 1 ≤ k ≤ k ∗ , h_0=h_{\mathrm{rescale}}\underset{i\ \mathrm{eligible}}{\operatorname{avg}} r_i,\qquad
h_k=a_hh_{k-1},\qquad
\rho_k=a_\rho\rho_{k-1},\qquad
1\le k\le k_*, h 0 = h rescale i eligible avg r i , h k = a h h k − 1 , ρ k = a ρ ρ k − 1 , 1 ≤ k ≤ k ∗ ,
a ρ = e − 1 / [ 2 ( 3 s + m ) ] , a h = exp { 1 2 ( D + 4 ) k ∗ } . (23) a_\rho=e^{-1/[2(3s+m)]},\qquad
a_h=\exp\left\{\frac{1}{2(D+4)k_*}\right\}. \tag*{(23)} a ρ = e − 1/ [ 2 ( 3 s + m )] , a h = exp { 2 ( D + 4 ) k ∗ 1 } . (23)
The normal width remains comparable to the reach, and b k = ρ k h k b_k=\rho_kh_k b k = ρ k h k is the tangential width. Let χ 0 ( x , v ) \chi_0(x,v) χ 0 ( x , v ) indicate membership of v v v in the local cylinder (22) of x x x . The pooling neighborhood at x x x and the fitting weights at a center u u u are
W x , k = { v : χ 0 ( x , v ) = 1 , d P ^ k − 1 ( x ) , b k , h k ( v − x ) ≤ 1 } , \mathcal{W}_{x,k}=\left\{v:\ \chi_0(x,v)=1,\ d_{\widehat P_{k-1}(x),b_k,h_k}(v-x)\le1\right\}, W x , k = { v : χ 0 ( x , v ) = 1 , d P k − 1 ( x ) , b k , h k ( v − x ) ≤ 1 } ,
ω u , k ( v ) = χ 0 ( u , v ) K ( d P ^ k − 1 ( u ) , b k , h k ( v − u ) 2 ) . (24) \omega_{u,k}(v)=\chi_0(u,v)K\left(d_{\widehat P_{k-1}(u),b_k,h_k}(v-u)^2\right). \tag*{(24)} ω u , k ( v ) = χ 0 ( u , v ) K ( d P k − 1 ( u ) , b k , h k ( v − u ) 2 ) . (24)
Each pool center u u u fits a slope on the fit fold with weights ω u , k \omega_{u,k} ω u , k . Pooling the stored outer products over W x , k \mathcal{W}_{x,k} W x , k gives C ^ k ( x ) \widehat C_k(x) C k ( x ) and hence P ^ k ( x ) = P Top m ( C ^ k ( x ) ) \widehat P_k(x)=P_{\operatorname{Top}_m}(\widehat C_k(x)) P k ( x ) = P Top m ( C k ( x )) on the concentration event.
The effective local count is N k : = n f π − b k m N_k := n_f\pi^{-b_k^m} N k := n f π − b k m , where n f n_f n f is the size of a sample block up to integer rounding (Appendix § 7 .§ 1 ).
Phase 3. The prediction estimator F ^ \widehat{F} F in (A.7) uses the prediction fold, with widths h ∗ h_* h ∗ and b ∗ = ρ ∗ h ∗ b_* = \rho_*h_* b ∗ = ρ ∗ h ∗ , and restricts the kernel to the local cylinder (22) . The widths prescribed in Appendix § 7 .§ 1 ensure that, at a query x = q + z x = q + z x = q + z , the prediction kernel is at least 1 / 16 1/16 1/16 on every normal fiber above B M ( q , b ∗ / 16 ) B_{\mathcal{M}}(q,b_*/16) B M ( q , b ∗ /16 ) , and place the restricted support strictly inside a local base chart (Lemma A.11 ).
Results
At a target x = q + z x = q + z x = q + z , write d ( q ) = min { 1 , d M ( q , Σ s i n g ) / σ N } d(q) = \min\{1,d_{\mathcal{M}}(q,\Sigma_{\mathrm{sing}})/\sigma_N\} d ( q ) = min { 1 , d M ( q , Σ sing ) / σ N } , with d = 1 d = 1 d = 1 if the singular set is empty. This distance distinguishes the tangent directions recovered at the center from those recovered through its neighbors. The following results use (A1)–(A5), the estimator above, and n ≥ n 0 n \ge n_0 n ≥ n 0 , where the sample threshold is
n 0 = C Θ log 2 ( 2 + Θ ) , n_0 = C\Theta\log^2(2+\Theta), n 0 = C Θ log 2 ( 2 + Θ ) ,
Θ = max { ( 1 + D + p ) 4 ( 1 + R / σ N ) 16 π − σ N m ( w Y / σ N ) m + 4 ( 1 + σ ζ σ f , + σ N ) 2 , D + log ( 2 ∣ R ∣ / p 0 ) p 0 ( 1 + max R w Y + ( max R ) 2 reach M σ N 2 w Y ) 2 } . (25) \Theta= \max\left\{\frac{(1+D+p)^4(1+R/\sigma_N)^{16}}{\pi_-\sigma_N^m(w_Y/\sigma_N)^{m+4}}\left(1+\frac{\sigma_\zeta}{\sigma_{f,+\sigma_N}}\right)^2,\frac{D+\log(2|\mathcal{R}|/p_0)}{p_0}\left(1+\frac{\max\mathcal{R}}{w_Y}+\frac{(\max\mathcal{R})^2\operatorname{reach}_{\mathcal{M}}}{\sigma_N^2w_Y}\right)^2\right\}. \tag*{(25)} Θ = max { π − σ N m ( w Y / σ N ) m + 4 ( 1 + D + p ) 4 ( 1 + R / σ N ) 16 ( 1 + σ f , + σ N σ ζ ) 2 , p 0 D + log ( 2∣ R ∣/ p 0 ) ( 1 + w Y max R + σ N 2 w Y ( max R ) 2 reach M ) 2 } . (25)
with a constant C C C that depends on τ \tau τ . Appendix A.6 shows that n ≥ n 0 n \ge n_0 n ≥ n 0 implies the two sample conditions used in the proofs.
Proposition 3.1 (Phase 1: the warm start). With probability at least 1 − C n − τ − 2 1-Cn^{-\tau-2} 1 − C n − τ − 2 every target x = q + z x=q+z x = q + z is assigned an anchor star S S S with center x S x_S x S satisfying ∥ x S − q ∥ ≤ C w Y \lVert x_S-q\rVert\le Cw_Y ∥ x S − q ∥ ≤ C w Y , and ∥ P ^ 0 ( x ) − P q ∥ ≤ C w Y / reach M \lVert\widehat{P}_0(x)-P_q\rVert\le Cw_Y/\operatorname{reach}_{\mathcal{M}} ∥ P 0 ( x ) − P q ∥ ≤ C w Y / reach M .
The local cylinder of x x x contains every normal fiber above B M ( q , r c y l / 4 ) B_{\mathcal{M}}(q,r_{\mathrm{cyl}}/4) B M ( q , r cyl /4 ) , and the points it contains have base points in B M ( q , C r c y l ) B_{\mathcal{M}}(q,Cr_{\mathrm{cyl}}) B M ( q , C r cyl ) , contained in a single graph chart of (A1).
Proposition 3.2 (Phase 2: a refinement neighborhood). Fix 1 ≤ k ≤ k ∗ 1 \le k \le k_* 1 ≤ k ≤ k ∗ , with 0 < η ≤ 1 / 16 0 < \eta\le1/16 0 < η ≤ 1/16 , A ≥ 1 A \ge1 A ≥ 1 and the initial widths h 0 , b 0 h_0,b_0 h 0 , b 0 chosen as in Appendix A.4.3, satisfying (A.37); use the schedule (3.11). With probability at least 1 − C ( k + 1 ) n − τ − 2 1-C(k+1)n^{-\tau-2} 1 − C ( k + 1 ) n − τ − 2 , the following hold simultaneously at every tube point x = q + z x=q+z x = q + z : W x , k W_{x,k} W x , k contains the complete normal fibers above B M ( q , b k / 16 ) B_{\mathcal{M}}(q,b_k/16) B M ( q , b k /16 ) , and ω x , k ≥ 1 / 16 \omega_{x,k}\ge1/16 ω x , k ≥ 1/16 there. Moreover, E ω x , k ( X ) ≍ P { X ∈ W x , k } ≍ π − b k m \mathbb{E}\,\omega_{x,k}(X)\asymp\mathbb{P}\{X\in W_{x,k}\}\asymp\pi_-b_k^m E ω x , k ( X ) ≍ P { X ∈ W x , k } ≍ π − b k m (Lemma A.11), and the updated projector satisfies R ∥ P ^ k ( x ) − P q ∥ < a ρ η b k ≤ η b k + 1 R\lVert\widehat{P}_k(x)-P_q\rVert<a_\rho\eta b_k\le\eta b_{k+1} R ∥ P k ( x ) − P q ∥ < a ρ η b k ≤ η b k + 1 throughout the tube.
The proof combines the concentration bounds (A.34)–(A.36) with estimates in three regions: within distance A b k Ab_k A b k of a point where D f Df D f is rank deficient (Lemma A.12); the surrounding annulus up to distance r s i n g r_{\mathrm{sing}} r sing (A.28); and the regular region, where two successive updates control the error (A.33). Induction in Appendix A.4.3 gives the stated probability.
Proposition 3.3 (Phase 2: all refinement steps). With probability at least 1 − C n − τ − 1 1-Cn^{-\tau-1} 1 − C n − τ − 1 , simultaneously for 1 ≤ k ≤ k ∗ 1\le k\le k_* 1 ≤ k ≤ k ∗ and every tube point x = q + z x=q+z x = q + z , W x , k W_{x,k} W x , k contains every normal fiber above B M ( q , b k / 16 ) B_{\mathcal{M}}(q,b_k/16) B M ( q , b k /16 ) , ω x , k ≥ 1 / 16 \omega_{x,k}\ge1/16 ω x , k ≥ 1/16 on these fibers, R ∥ P ^ k ( x ) − P q ∥ < a ρ η b k R\lVert\widehat{P}_k(x)-P_q\rVert<a_\rho\eta b_k R ∥ P k ( x ) − P q ∥ < a ρ η b k , and
∥ P ^ k ( x ) − P q ∥ ≤ C σ N R ( b k / σ N ) 2 d ( q ) + b k / σ N . (26) \lVert\widehat{P}_k(x)-P_q\rVert\le C\frac{\sigma_N}{R}\frac{(b_k/\sigma_N)^2}{d(q)+b_k/\sigma_N}. \tag*{(26)} ∥ P k ( x ) − P q ∥ ≤ C R σ N d ( q ) + b k / σ N ( b k / σ N ) 2 . (26)
Proposition 3.4 (Phase 3: prediction). Fix the pilot and refinement data and the pilot split on the event of Proposition 3.3 and put u = b ∗ / σ N u=b_*/\sigma_N u = b ∗ / σ N . In units of ( σ f , + σ N ) 2 (\sigma_{f,+\sigma_N})^2 ( σ f , + σ N ) 2 , the squared prediction error of the estimator F ^ \widehat{F} F defined in (A.7), averaged over the prediction fold and an independent query, is at most
C [ u 4 { 1 + 1 { m = 2 , ∃ q ∈ M : rank D f ( q ) = 1 } ∣ log u ∣ } + p { σ ζ / ( σ f , + σ N ) } 2 + u 2 n p r e d π − b ∗ m ] (27) C\left[u^4\left\{1+\mathbf{1}_{\{m=2,\ \exists q\in\mathcal{M}:\operatorname{rank}Df(q)=1\}}|\log u|\right\}+\frac{p\{\sigma_\zeta/(\sigma_{f,+\sigma_N})\}^2+u^2}{n_{\mathrm{pred}}\pi_-b_*^m}\right] \tag*{(27)} C [ u 4 { 1 + 1 { m = 2 , ∃ q ∈ M : rank D f ( q ) = 1 } ∣ log u ∣ } + n pred π − b ∗ m p { σ ζ / ( σ f , + σ N ) } 2 + u 2 ] (27) plus the exponentially small contribution (A.38) from a small prediction denominator. The extra logarithm is needed in this bound only for a surface with such a point.
The estimator in (A.7) uses the averaging formula (12) on the prediction fold S p r e d S_{\mathrm{pred}} S pred , with weights multiplied by the cylinder indicator χ 0 ( x , X i ) \chi_0(x,X_i) χ 0 ( x , X i ) , and projections P ^ k ∗ \widehat{P}_{k_*} P k ∗ constructed on the separate pilot folds S a n c , S d e s S_{\mathrm{anc}},S_{\mathrm{des}} S anc , S des and refinement folds S k f i t , S k p o o l S_k^{\mathrm{fit}},S_k^{\mathrm{pool}} S k fit , S k pool , 1 ≤ k ≤ k ∗ 1 \le k \le k_* 1 ≤ k ≤ k ∗ .
Theorem 3.5 (Learning with rank-deficient D f Df D f ). Let ( X , Y ) (X,Y) ( X , Y ) satisfy (A1)–(A5) with D > m D > m D > m and p ≥ m p \ge m p ≥ m , for the margins ε s i n g \varepsilon_{\mathrm{sing}} ε sing , γ c y l \gamma_{\mathrm{cyl}} γ cyl and ε 0 \varepsilon_0 ε 0 determined in Appendix § 7 .§ 4.3 ; these are positive and depend only on the fixed model parameters. Fix 0 < c < 2 / 3 0 < c < 2/3 0 < c < 2/3 and a confidence exponent τ > 0 \tau> 0 τ > 0 . Let n 0 = C Θ log 2 ( 2 + Θ ) n_0 = C\Theta\log^2(2+\Theta) n 0 = C Θ log 2 ( 2 + Θ ) be the sample threshold (25) ; n 0 n_0 n 0 and the constants below depend on τ \tau τ . For n ≥ n 0 n \ge n_0 n ≥ n 0 , the MANOR estimator with s = 2 s = 2 s = 2 , tuned as in Appendix § 7 .§ 1 with m m m , δ Y \delta_Y δ Y , the radii R \mathcal{R} R and the scales R \mathcal{R} R , σ N \sigma_N σ N , r s i n g r_{\mathrm{sing}} r sing , w Y w_Y w Y , p 0 p_0 p 0 , π − \pi_- π − , σ ζ \sigma_\zeta σ ζ and σ f , + \sigma_{f,+} σ f , + , has mean tangent-space error
E t a n ( P ^ k ∗ ) = O ~ ( n − 2 / ( m + 4 ) ) , (28) \mathcal{E}_{\mathrm{tan}}(\widehat{P}_{k_*}) = \widetilde{O}\left(n^{-2/(m+4)}\right), \tag*{(28)} E tan ( P k ∗ ) = O ( n − 2/ ( m + 4 ) ) , (28) with the expectation taken over the training data, the pilot split and an independent query. Its integrated squared prediction error satisfies
E ∥ F ^ ( X ) − F ( X ) ∥ 2 ( σ f , + σ N ) 2 ≤ C ( D , p ) polylog ( n ) n 4 / ( m + 4 ) + C { diam ( f ( M ) ) 2 + p σ ζ 2 } n τ + 1 ( σ f , + σ N ) 2 . (29) \frac{\mathbb{E}\|\widehat{F}(X)-F(X)\|^2}{(\sigma_{f,+}\sigma_N)^2}
\le
\frac{C(D,p)\operatorname{polylog}(n)}{n^{4/(m+4)}}+
\frac{C\{\operatorname{diam}(f(\mathcal{M}))^2+p\sigma_\zeta^2\}}{n^{\tau+1}(\sigma_{f,+}\sigma_N)^2}. \tag*{(29)} ( σ f , + σ N ) 2 E ∥ F ( X ) − F ( X ) ∥ 2 ≤ n 4/ ( m + 4 ) C ( D , p ) polylog ( n ) + n τ + 1 ( σ f , + σ N ) 2 C { diam ( f ( M ) ) 2 + p σ ζ 2 } . (29) The second term is the contribution of the exceptional events of Propositions 3.1 and 3.3 and of a small prediction denominator, which have total probability at most C n − τ − 1 Cn^{-\tau-1} C n − τ − 1 and on which the prediction error is bounded only by the response diameter and the noise. For fixed intrinsic dimension, dimensionless model margins, scale ratios and vol ( M ) / reach ( M ) m \operatorname{vol}(\mathcal{M})/\operatorname{reach}(\mathcal{M})^m vol ( M ) / reach ( M ) m , the constant in the tangent bound, C ( D , p ) C(D,p) C ( D , p ) and the sample threshold n 0 n_0 n 0 grow at most polynomially in D + p D+p D + p (Appendix § 7 .6). Constructing the estimator requires
O ( n ∣ R ∣ ( n D 2 + D 3 ) + k ∗ { n 2 ( D 2 + D p ) + n ( D 3 + D 2 p ) } ) O\left(n|\mathcal{R}|(nD^2+D^3)+k_*\{n^2(D^2+Dp)+n(D^3+D^2p)\}\right) O ( n ∣ R ∣ ( n D 2 + D 3 ) + k ∗ { n 2 ( D 2 + D p ) + n ( D 3 + D 2 p )} ) operations, where k ∗ = O ( log n ) k_* = O(\log n) k ∗ = O ( log n ) is the number of refinement steps, and each evaluation of F ^ \widehat{F} F requires O ( n ( D m + p ) ) O(n(Dm+p)) O ( n ( D m + p )) operations (§ 2.6 ). For a fixed radius grid R \mathcal{R} R and n ≥ D n \ge D n ≥ D , the construction cost is O ( n 2 ( D 2 + D p ) log n ) O(n^2(D^2+Dp)\log n) O ( n 2 ( D 2 + D p ) log n ) .
The exponent 4 / ( m + 4 ) 4/(m+4) 4/ ( m + 4 ) is that of nonparametric regression with two orders of smoothness in m m m dimensions ([94 ] ). Appendix C establishes this minimax exponent, for fixed positive noise, over the broader class of Manifold-Index regression functions with the regularity of (A1) and (A3). Least squares over a finite net attains it without (A2) and (A4), using of (A5) only the sub-Gaussian noise bound; the net has cardinality bounded exponentially in a power of n n n , but the argument supplies no algorithm for constructing and evaluating its representatives. The transfer of the minimax conclusion to the restricted class (A1)–(A5) is conditional on the subclass conditions stated there. The tube ratio c c c enters Phase 1 only through a chord inequality that requires c < 2 / 3 c < 2/3 c < 2/3 . In Phase 2 it enters jointly with the bound on root mean square tangential stretch in (A2). On a connected manifold the local sampling mass also bounds the normalized response range (Appendix § 7 .7). When the signal variance is positive, dividing the squared prediction error by it gives the relative risk ε p r e d \varepsilon_{\mathrm{pred}} ε pred in (3) .
Scope of the result. Near an isolated rank loss, (A3) and (A4) serve different purposes: neighboring gradients recover the missing tangent directions, while localized response packets provide the warm start. Scalar extrema on closed curves violate the packet condition at sufficiently fine pilot resolution; vector-valued cusp maps can satisfy it (Remark A.16). Appendix § 7 .9 gives a fully specified noisy model and explains the restrictions in the equality case p = m p=m p = m .
The Gaussian-noise extension in Corollary A.14 applies over a finite sample-size range when the noise variance and pilot response resolution are fixed. Its statement and proof are in Appendix § 7 .8.
Uniform tangent-space error with rank-deficient D f Df D f . On the event of Proposition 3.3 , setting d ( q 0 ) = 0 d(q_0)=0 d ( q 0 ) = 0 in (26) bounds the tangent-space error everywhere on the normal fiber above a singular point q 0 q_0 q 0 by C b k ∗ / R Cb_{k_*}/R C b k ∗ / R , or O ~ ( n − 1 / ( m + 4 ) ) \widetilde{O}(n^{-1/(m+4)}) O ( n − 1/ ( m + 4 ) ) at the terminal scale. This is also a uniform bound over the tube. The faster mean rate in Theorem 3.5 comes from integrating the distance-dependent bound over the base points (Appendix § 7 .7). If D f Df D f has full rank throughout M \mathcal{M} M , then d ≡ 1 d\equiv1 d ≡ 1 and O ~ ( n − 2 / ( m + 4 ) ) \widetilde{O}(n^{-2/(m+4)}) O ( n − 2/ ( m + 4 ) ) holds uniformly as well.
How the results fit together
Figure 5 traces the dependencies of the results leading to the main Theorem. The pilot gives a single local chart and the first aligned neighborhood. The fitted slopes keep subsequent neighborhoods aligned, with the weakest tangent signal coming from a whole ball of regular centers near a point where D f Df D f is rank deficient. Prediction uses the stronger angle bound away from the finitely many points where D f Df D f is rank deficient and integrates the remaining error near them.
Figure 5. From a response-selected warm start to prediction. The successive neighborhoods retain complete normal fibers. The terminal angle varies with the distance to the nearest singular point; its integrated effect determines the prediction bound.
Dimension dependence. In the sample conditions (A.1) and (A.39), the ambient and response dimensions enter through explicit polynomial factors, such as ( 1 + D + p ) 4 (1+D+p)^4 ( 1 + D + p ) 4 and ( 1 + R / σ N ) 16 (1+R/\sigma_N)^{16} ( 1 + R / σ N ) 16 with R / σ N ≍ D − m + 2 R/\sigma_N \asymp\sqrt{D-m+2} R / σ N ≍ D − m + 2 . The local mass π − σ N m \pi-\sigma_N^m π − σ N m is of the order of the probability that a base point falls in a geodesic ball of radius σ N \sigma_N σ N on M \mathcal{M} M . It is comparable to σ N m / vol ( M ) \sigma_N^m/\operatorname{vol}(\mathcal{M}) σ N m / vol ( M ) , the inverse of the number of such balls needed to cover M \mathcal{M} M . Since R = reach M R=\operatorname{reach}\mathcal{M} R = reach M ,
( π − σ N m ) − 1 ≲ vol ( M ) reach ( M ) m c − m ( D − m + 2 ) m / 2 . (\pi-\sigma_N^m)^{-1}\lesssim\frac{\operatorname{vol}(\mathcal{M})}{\operatorname{reach}(\mathcal{M})^m}c^{-m}(D-m+2)^{m/2}. ( π − σ N m ) − 1 ≲ reach ( M ) m vol ( M ) c − m ( D − m + 2 ) m /2 .
For fixed m m m and c c c , this contribution is polynomial in D D D provided vol ( M ) / reach ( M ) m \operatorname{vol}(\mathcal{M})/\operatorname{reach}(\mathcal{M})^m vol ( M ) / reach ( M ) m grows at most polynomially. The theorem retains the local mass explicitly, without imposing this volume bound. The probability lower bound in (19) enters through p 0 − 1 p_0^{-1} p 0 − 1 . Since p 0 ≤ C π − w Y m p_0 \le C\pi^{-w_Y^m} p 0 ≤ C π − w Y m , it also grows at least like ( D − m + 2 ) m / 2 (D-m+2)^{m/2} ( D − m + 2 ) m /2 , and it is polynomial in D D D when, in addition, p 0 / ( π − w Y m ) p_0/(\pi^{-w_Y^m}) p 0 / ( π − w Y m ) and σ N / w Y \sigma_N/w_Y σ N / w Y are fixed.
Numerical experiments
Table 2 summarizes five experiments. The line segment and Meyer helix have non-monotone scalar links (p = m = 1 p=m=1 p = m = 1 ) and show how prediction and tangent-space errors vary with sample size and ambient dimension. The bumpy sphere and coupled Meyer helix have links whose differentials have rank m m m . On these surfaces, normal displacements scale with local reach rather than the global reach used in the theorem. The bumpy sphere also shows how the errors change over successive refinement steps. The diffusion experiment tests prediction when first-hit probabilities vary along a normal fiber. Appendix B gives the constructions and sampling distributions, including the non-uniform normal laws.
Table 2. Overview of the numerical experiments.
Implementation and error measures
Implementation. The numerical implementation chooses the scan radii from the data and reuses observations across refinement rounds. Within each scaling experiment, the parameters governing the scan and the refinement schedule are held fixed across sample sizes and repetitions.
Error measures. For prediction we report the test mean squared error relative to the response variance (relMSE). The curve experiments use Var ( Y ) \operatorname{Var}(Y) Var ( Y ) as the denominator. The surface scaling experiments use the total empirical variance of the noiseless test responses, corresponding to the signal variance in (3) . For the diffusion experiment, independent Monte Carlo estimates provide the reference responses (§ 4.6 ).
For tangent estimation the scaling plots report the mean discrepancy over independent test queries (devTan). For a curve γ \gamma γ , the pointwise error is ∥ v ^ ( X ) − γ ′ ∥ \lVert\hat{v}(X)-\gamma' \rVert ∥ v ^ ( X ) − γ ′ ∥ , where v ^ ( X ) \hat{v}(X) v ^ ( X ) is the estimated unit tangent vector and γ ′ \gamma' γ ′ is the unit tangent vector of γ \gamma γ at Π M X \Pi_{\mathcal{M}}X Π M X , oriented so that ⟨ v ^ ( X ) , γ ′ ⟩ ≥ 0 \langle\hat{v}(X),\gamma' \rangle\ge0 ⟨ v ^ ( X ) , γ ′ ⟩ ≥ 0 . For a surface, let θ 1 , θ 2 \theta_1,\theta_2 θ 1 , θ 2 be the principal angles between the estimated and true tangent planes. The pointwise error is
∥ P ^ ( X ) − P Π M X ∥ F 2 = ( sin 2 θ 1 + sin 2 θ 2 ) 1 / 2 , (30) \frac{\lVert\hat{P}(X)-P_{\Pi_{\mathcal{M}}X}\rVert_F}{\sqrt{2}} = \left(\sin^2\theta_1+\sin^2\theta_2\right)^{1/2}, \tag*{(30)} 2 ∥ P ^ ( X ) − P Π M X ∥ F = ( sin 2 θ 1 + sin 2 θ 2 ) 1/2 , (30)
On both curves and surfaces the pointwise discrepancy lies between the projector operator norm in (3) and 2 \sqrt{2} 2 times that norm.
Reference slopes. Theorem 3.5 gives the reference exponents for the two errors plotted against sample size: − 4 / ( m + 4 ) -4/(m + 4) − 4/ ( m + 4 ) for relMSE and − 2 / ( m + 4 ) -2/(m + 4) − 2/ ( m + 4 ) for mean tangent-space error. Thus the reference slopes are − 0.8 -0.8 − 0.8 and − 0.4 -0.4 − 0.4 on curves, and − 2 / 3 -2/3 − 2/3 and − 1 / 3 -1/3 − 1/3 on surfaces, respectively. We fit slopes by least squares on logarithmic axes over the sample-size ranges specified for each experiment.
A single-index model
Set the normal standard deviation σ γ = 0.5 \sigma_\gamma= 0.5 σ γ = 0.5 and the segment length len γ = 12 σ γ D \operatorname{len}_\gamma= \sqrt{12}\sigma_\gamma\sqrt{D} len γ = 12 σ γ D . In the single-index model , the curve γ \gamma γ is the segment { x ∈ R D : 0 ≤ x 1 ≤ len γ , x 2 = ⋯ = x D = 0 } \{x \in\mathbb{R}^{D}: 0 \le x_1 \le\operatorname{len}_\gamma, x_2 = \cdots= x_D = 0\} { x ∈ R D : 0 ≤ x 1 ≤ len γ , x 2 = ⋯ = x D = 0 } , and the predictor has independent coordinates X 1 ∼ Unif ( 0 , len γ ) X_1 \sim\operatorname{Unif}(0,\operatorname{len}_\gamma) X 1 ∼ Unif ( 0 , len γ ) and X 2 , … , X D ∼ N ( 0 , σ γ 2 ) X_2,\ldots,X_D \sim\mathcal{N}(0,\sigma_\gamma^2) X 2 , … , X D ∼ N ( 0 , σ γ 2 ) . Thus Var ( X 1 ) = D σ γ 2 \operatorname{Var}(X_1) = D\sigma_\gamma^2 Var ( X 1 ) = D σ γ 2 is comparable with the total normal variance ( D − 1 ) σ γ 2 (D-1)\sigma_\gamma^2 ( D − 1 ) σ γ 2 , and the regression function F ( X ) = f ( X 1 ) F(X) = f(X_1) F ( X ) = f ( X 1 ) depends on X X X through a single linear index. Writing t ∈ [ 0 , len γ ] t \in[0,\operatorname{len}_\gamma] t ∈ [ 0 , len γ ] for the arc length, the link function is f ( t ) = f ∗ ( t / len γ ) f(t) = f_*(t/\operatorname{len}_\gamma) f ( t ) = f ∗ ( t / len γ ) , where
f ∗ ( t ) = 64 t 3 − 96 t 2 + 36 t − 2 + e t + 0.8 cos ( π t ) + 2 sin ( 5 π t + 0.1 ) , 0 ≤ t ≤ 1. (31) f_*(t) = 64t^3 - 96t^2 + 36t - 2 + e^t + 0.8\cos(\pi t) + 2\sin(5\pi t + 0.1), \qquad0 \le t \le1. \tag*{(31)} f ∗ ( t ) = 64 t 3 − 96 t 2 + 36 t − 2 + e t + 0.8 cos ( π t ) + 2 sin ( 5 π t + 0.1 ) , 0 ≤ t ≤ 1. (31)
A response in the range of f ∗ f_* f ∗ has between one and five preimages (Figure 6 ). The noise standard deviation is 1% of the response standard deviation.
Figure 6. Normalized link f ∗ f_* f ∗ in (4.2) for the line segment and the Meyer helix.
At each n n n we use a 90:10 training–test split and three independent repetitions. We evaluate prediction against the noiseless response, normalize by Var ( Y ) \operatorname{Var}(Y) Var ( Y ) and set δ Y = ( max i Y i − min i Y i ) / 50 \delta_Y = (\max_i Y_i - \min_i Y_i)/50 δ Y = ( max i Y i − min i Y i ) /50 . At each D D D we compare three fixed multipliers c ρ c_\rho c ρ in ρ ∗ = c ρ ρ k ∗ \rho_* = c_\rho\rho_{k_*} ρ ∗ = c ρ ρ k ∗ (Appendix § 8 .§ 1 ).
Results. On the line segment the tangent direction is constant. Its estimated direction has mean error close to the parametric rate n − 1 / 2 n^{-1/2} n − 1/2 in all three ambient dimensions (Figure 7 ). The fitted slopes for relMSE range from − 0.94 -0.94 − 0.94 to − 0.69 -0.69 − 0.69 , around the reference slope − 0.8 -0.8 − 0.8 .
Figure 7. Line segment: relMSE (top row) and mean tangent-direction error (devTan, bottom row) against sample size n n n on logarithmic axes. Columns correspond to D = 18 , 24 , 30 D = 18, 24, 30 D = 18 , 24 , 30 , from left to right. The three curves in each panel use the three fixed c ρ c_\rho c ρ values in Appendix § 8 .§ 1 . Bars show variation across three independent repetitions at each n n n ; legends give slopes fitted over the displayed range.
The Meyer helix
The Meyer helix was introduced in our study of coarsely monotone links [102 ] . With its reach normalized to D \sqrt{D} D , the curve has diameter of order D \sqrt{D} D , length of order D 3 / 2 D^{3/2} D 3/2 and effective linear dimension of order D D D , defined here as the number of singular values of the sampled coordinate matrix exceeding 5% of its largest singular value (Appendix § 8 .§ 1 ). Thus a one-dimensional index moves through an increasing number of ambient directions. We sample the normal displacement from N ( 0 , σ γ 2 I D − 1 ) \mathcal{N}(0,\sigma_\gamma^2 I_{D-1}) N ( 0 , σ γ 2 I D − 1 ) , with σ γ = 0.5 \sigma_\gamma= 0.5 σ γ = 0.5 , truncated at ∥ Z ∥ < 0.9 D \lVert Z\rVert< 0.9\sqrt{D} ∥ Z ∥ < 0.9 D . The link f ( t ) = len γ f ∗ ( t / len γ ) f(t) = \operatorname{len}_\gamma f_*(t/\operatorname{len}_\gamma) f ( t ) = len γ f ∗ ( t / len γ ) has the non-monotone shape of Figure 6 ; its arc-length derivative is independent of the curve length at a fixed relative position. We use the noise level and test protocol of § 4.2 , with response resolution δ Y = ( max i Y i − min i Y i ) / 100 \delta_Y = (\max_i Y_i - \min_i Y_i)/100 δ Y = ( max i Y i − min i Y i ) /100 .
Results. At D = 18 D = 18 D = 18 and 24 24 24 , the fitted prediction slopes lie between − 0.97 -0.97 − 0.97 and − 0.72 -0.72 − 0.72 , and the tangent-error slopes are close to − 0.4 -0.4 − 0.4 (Figure 8 ). At D = 30 D = 30 D = 30 , relMSE falls steeply up to n ≃ 2.4 × 10 5 n \simeq2.4 \times10^{5} n ≃ 2.4 × 1 0 5 and more slowly thereafter; fitting across both segments gives slopes from − 2.38 -2.38 − 2.38 to − 2.05 -2.05 − 2.05 . Mean tangent-direction error declines steadily over the same range, with slopes near − 0.60 -0.60 − 0.60 for all three choices of c ρ c_\rho c ρ .
Figure 8. Meyer helix: relMSE (top row) and mean tangent-direction error (devTan, bottom row) against sample size n n n on logarithmic axes. Columns correspond to D = 18 , 24 , 30 D = 18, 24, 30 D = 18 , 24 , 30 , from left to right. The three curves in each panel use the three fixed c ρ c_{\rho} c ρ values in Appendix § 8 .§ 1 . Bars show variation across three independent repetitions at each n n n ; legends give slopes fitted over the displayed range.
The bumpy sphere
The bumpy sphere is a closed two-dimensional surface in R 20 \mathbb{R}^{20} R 20 with two R 6 \mathbb{R}^{6} R 6 -valued link functions differing in phase complexity and the number of response preimage packets. For both, prediction and tangent-space errors decrease over the sampled range despite normal displacements beyond the global tube of (A1) (Figure 10 and Table 3 ).
Table 3. Least-squares slopes of the logarithm of the arithmetic mean error against log n \log n log n , calculated from the ten independent repetitions at every sample size. The reference slopes are those for squared prediction risk and mean tangent-space error in Theorem 3.5 with m = 2 m=2 m = 2 .
Figure 10. Bumpy sphere in D = 20 D = 20 D = 20 with the linear-phase link function (a) and quadratic-phase link function (b), with 1% response noise. Left: relative mean squared prediction error on noiseless test responses. Right: mean tangent-space error ( sin 2 θ 1 + sin 2 θ 2 ) 1 / 2 (\sin^2\theta_1 + \sin^2\theta_2)^{1/2} ( sin 2 θ 1 + sin 2 θ 2 ) 1/2 on test queries. Points show means of ten independent repetitions; bars show one sample standard deviation. The constants of Table 5 are fixed across all n n n and repetitions. The dash-dotted line marks n = 10 6 n = 10^6 n = 1 0 6 ; the legends give log–log least-squares slopes of the mean errors over n ≥ 10 6 n \ge10^6 n ≥ 1 0 6 .
Manifold and predictors. The surface in Figure 9a is a radial deformation of the unit sphere. Let r r r be the smooth radial function of Appendix § 8 .§ 2 and choose the scale c M c_{\mathcal{M}} c M to give the surface numerically estimated reach 20 \sqrt{20} 20 , as for the Meyer helix. Then
M = { q ( u ) : u ∈ S 2 } , q ( u ) = c M r ( u ) ( u 1 , u 2 , u 3 , 0 , … , 0 ) ∈ R 20 , (32) \mathcal{M} = \{q(u) : u \in\mathbb{S}^{2}\}, \qquad q(u) = c_{\mathcal{M}}r(u)(u_{1},u_{2},u_{3},0,\ldots,0) \in\mathbb{R}^{20}, \tag*{(32)} M = { q ( u ) : u ∈ S 2 } , q ( u ) = c M r ( u ) ( u 1 , u 2 , u 3 , 0 , … , 0 ) ∈ R 20 , (32)
The surface lies in the first three coordinates, while the estimator recovers its tangent field from predictors spread over all 18 normal directions. We sample Q = q ( u ) Q = q(u) Q = q ( u ) with u u u uniform on S 2 \mathbb{S}^{2} S 2 and use centered normal displacements scaled by the local reach. The sampling law and reach calibration are given in Appendix § 8 .§ 2 .
Link functions and noise. Write ξ ( u ) = O r ( u ) u ∈ R 3 \xi(u)=O r(u)\,u \in\mathbb{R}^{3} ξ ( u ) = O r ( u ) u ∈ R 3 , where O O O is a fixed rotation of R 3 \mathbb{R}^{3} R 3 (Appendix B.2); thus ξ ( u ) \xi(u) ξ ( u ) is the first three coordinates of q ( u ) q(u) q ( u ) , rotated by O O O and rescaled by c M − 1 c_{\mathcal{M}}^{-1} c M − 1 . For a phase map ψ : R 3 → R 3 \psi:\mathbb{R}^{3}\to\mathbb{R}^{3} ψ : R 3 → R 3 , the link function has the six components
f 2 i − 1 ( q ( u ) ) = 1 + cos ψ i ( ξ ( u ) ) 2 , f 2 i ( q ( u ) ) = 1 + sin ψ i ( ξ ( u ) ) 2 , i = 1 , 2 , 3. (33) f_{2i-1}(q(u))=\frac{1+\cos\psi_i(\xi(u))}{2},\qquad f_{2i}(q(u))=\frac{1+\sin\psi_i(\xi(u))}{2},\qquad i=1,2,3. \tag*{(33)} f 2 i − 1 ( q ( u )) = 2 1 + cos ψ i ( ξ ( u )) , f 2 i ( q ( u )) = 2 1 + sin ψ i ( ξ ( u )) , i = 1 , 2 , 3. (33)
We consider two trigonometric link functions: the linear-phase link function f l i n f^{\mathrm{lin}} f lin , with ψ i ( ξ ) = 8.75 ξ i \psi_i(\xi)=8.75\,\xi_i ψ i ( ξ ) = 8.75 ξ i , and the quadratic-phase link function f q u a d f^{\mathrm{quad}} f quad , with ψ i ( ξ ) = 15 ξ i + 2 ξ j ξ l \psi_i(\xi)=15\,\xi_i+2\,\xi_j\xi_l ψ i ( ξ ) = 15 ξ i + 2 ξ j ξ l , where { j , l } = { 1 , 2 , 3 } ∖ { i } \{j,l\}=\{1,2,3\}\setminus\{i\} { j , l } = { 1 , 2 , 3 } ∖ { i } (Figure 9b ,Figure 9c )). Both link functions satisfy (A3), as verified in Appendix B.2. The phases make several turns over the surface, so neither link function is injective, and a response preimage can consist of several separated packets; since the phase of f q u a d f^{\mathrm{quad}} f quad varies faster, its packets are more numerous. Training and validation responses carry 1% relative Gaussian noise with the covariance specified in Appendix B.2. Test errors use the noiseless responses.
Protocol and fixed parameters. The constants in Table 5 are selected on validation data at n ∈ { 1 , 5 , 9 } × 10 j n\in\{1,5,9\}\times10^j n ∈ { 1 , 5 , 9 } × 1 0 j , j = 4 , 5 j=4,5 j = 4 , 5 , using the criterion in Appendix B.2, then fixed across sample sizes and repetitions. For each link function and each sample size
Table 5. Constants held fixed in the ten independent repetitions. The response preimage size is n Y = min { n , max { 234 , ⌈ π Y n ⌉ } } n_Y = \min\{n,\max\{234,\lceil\pi_Y n\rceil\}\} n Y = min { n , max { 234 , ⌈ π Y n ⌉}} , and the number of Phase 2 iterations is k ∗ = ⌈ 2 log n + k 0 ⌉ k_* = \lceil2\log n + k_0\rceil k ∗ = ⌈ 2 log n + k 0 ⌉ . The width and aspect-ratio schedules are given in (B.3); prediction uses the normal width c h h k ∗ c_hh_{k_*} c h h k ∗ , with a tuning multiplier c h c_h c h in place of the value one of Section 2.5, and the aspect ratio c ρ ρ k ∗ c_\rho\rho_{k_*} c ρ ρ k ∗ .
n ∈ { 1 , 2 , 3 , 4 , 5 , 7 } × 10 j , j = 4 , 5 , 6 , n\in\{1,2,3,4,5,7\}\times10^j,\qquad j=4,5,6, n ∈ { 1 , 2 , 3 , 4 , 5 , 7 } × 1 0 j , j = 4 , 5 , 6 ,
we perform ten independent repetitions, each with fresh training, validation and test data. Data sets are also independent across sample sizes. Within a repetition, the two link functions share the predictors and standardized Gaussian noise draws. Each estimator is evaluated on 5000 independent test queries.
The relative mean squared prediction error is normalized by the total empirical variance of the noiseless test responses. The tangent-space error is (4.1), evaluated after orthonormalizing the estimated basis. In Figure 10 , markers give the mean of the ten repetition-level errors and bars give one sample standard deviation.
(a) The surface M \mathcal{M} M
(b) The components of f l i n f^{\mathrm{lin}} f lin
(c) The components of f q u a d f^{\mathrm{quad}} f quad
Figure 9. The bumpy sphere (§ 4.4 ). (a) Orthographic view of M \mathcal{M} M in its first three coordinates; the other 17 vanish. (b, c) Components f 1 , … , f 6 f_1,\ldots,f_6 f 1 , … , f 6 of the two link functions versus longitude λ \lambda λ and latitude φ \varphi φ of u ∈ S 2 u \in S^2 u ∈ S 2 .
Results. As shown in Figure 10 , both errors decrease with the sample size for both link functions. Fitting the mean errors over n ≥ 10 6 n \ge10^6 n ≥ 1 0 6 gives prediction-error slopes − 0.815 -0.815 − 0.815 for f l i n f^{\mathrm{lin}} f lin and − 0.844 -0.844 − 0.844 for f q u a d f^{\mathrm{quad}} f quad , and tangent-error slopes − 0.252 -0.252 − 0.252 and − 0.350 -0.350 − 0.350 , respectively (Table 3 ). All four fits have coefficients of determination above 0.99. The tangent-space error slope for f q u a d f^{\mathrm{quad}} f quad is close to the reference exponent − 1 / 3 -1/3 − 1/3 , whereas the tangent-space error for f l i n f^{\mathrm{lin}} f lin decreases more slowly over this range. Both prediction errors decrease faster than the reference slope − 2 / 3 -2/3 − 2/3 over the fitted range.
At n = 7 × 10 6 n = 7 \times10^6 n = 7 × 1 0 6 , the prediction errors are ( 2.19 ± 0.05 ) × 10 − 5 (2.19 \pm0.05) \times10^{-5} ( 2.19 ± 0.05 ) × 1 0 − 5 for f l i n f^{\mathrm{lin}} f lin and ( 3.71 ± 0.35 ) × 10 − 5 (3.71 \pm0.35) \times10^{-5} ( 3.71 ± 0.35 ) × 1 0 − 5 for f q u a d f^{\mathrm{quad}} f quad . The corresponding tangent-space errors are ( 5.96 ± 0.06 ) × 10 − 3 (5.96 \pm0.06) \times10^{-3} ( 5.96 ± 0.06 ) × 1 0 − 3 and ( 4.28 ± 0.16 ) × 10 − 3 (4.28 \pm0.16) \times10^{-3} ( 4.28 ± 0.16 ) × 1 0 − 3 , where each ± \pm ± gives one sample standard deviation across the ten repetitions.
Performance of iterative refinement. Figure 11 follows ten independent quadratic-phase runs at n = 5 × 10 5 n = 5 \times10^5 n = 5 × 1 0 5 , 7 × 10 5 7 \times10^5 7 × 1 0 5 and 10 6 10^6 1 0 6 with the fixed parameters in Table 5 . Iteration k = 0 k = 0 k = 0 is the Phase 1 initialization. Each trajectory uses the same training points for tangent-space error and the same 5000 independent test queries with noiseless responses for prediction error. The prediction bandwidth follows the prescribed schedule.
Figure 11. Iterative refinement for the quadratic-phase bumpy-sphere experiment. Left: mean tangent-space error on the training points. Right: relative mean squared prediction error on 5000 independent test queries with noiseless responses. Iteration k = 0 k = 0 k = 0 is the Phase 1 initialization. Curves show means over ten independent repetitions using the parameters of Table 5; bars give one sample standard deviation.
From initialization to the final iteration k ∗ k_* k ∗ , Phase 2 reduces the mean tangent-space error by factors of 43–53 and the mean relative prediction error by factors of 223–360 (Table 4 ). Holding observations, centers and final prediction bandwidth fixed, replacing the final estimated tangent spaces by the true ones reduces prediction error by 12–14%. At the same bandwidth, the initial tangent-space estimates give errors more than two orders of magnitude larger than the final estimates.
Table 4. Initial and final iterative-refinement errors: means ± \pm ± one sample standard deviation across ten independent repetitions. Prediction uses noiseless test responses.
A coupled Meyer helix surface
Let E i ( d ) E_i^{(d)} E i ( d ) and ϕ i ( d ) \phi_i^{(d)} ϕ i ( d ) be the localized envelopes and oscillatory phases, and μ M ∈ R 20 \mu_M \in\mathbb{R}^{20} μ M ∈ R 20 and c M > 0 c_M > 0 c M > 0 the centering vector and scale, specified in Appendix B.4. Coupling two Meyer helix coordinate families (Section 4.3) gives the two-dimensional surface $\mathcal{M} = {q(s,t) : (s,t) \in [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 in R 20 \mathbb{R}^{20} R 20 , where
q ( s , t ) = c M ( G ( s , t ) − μ M ) , q(s,t) = c_M\left(G(s,t)-\mu_M\right), q ( s , t ) = c M ( G ( s , t ) − μ M ) ,
G i j ( s , t ) = E i ( 5 ) ( s ) E j ( 4 ) ( t ) cos ( ϕ i ( 5 ) ( s ) + ϕ j ( 4 ) ( t ) ) , i = 1 , … , 5 , j = 1 , … , 4. (34) G_{ij}(s,t) = E_i^{(5)}(s)E_j^{(4)}(t)\cos\left(\phi_i^{(5)}(s)+\phi_j^{(4)}(t)\right), \qquad i=1,\ldots,5,\quad j=1,\ldots,4. \tag*{(34)} G ij ( s , t ) = E i ( 5 ) ( s ) E j ( 4 ) ( t ) cos ( ϕ i ( 5 ) ( s ) + ϕ j ( 4 ) ( t ) ) , i = 1 , … , 5 , j = 1 , … , 4. (34)
Both parameters enter all twenty coordinates. Figure 12 shows two surface projections; the common color key and coordinate curves identify corresponding regions.
Figure 12. The coupled Meyer helix surface in R 20 \mathbb{R}^{20} R 20 , projected onto the first three principal components of noisy predictors (left) and a random three-dimensional subspace (right). Only the surface is shown. The central key encodes ( s , t ) (s,t) ( s , t ) ; coordinate curves are s = j / 10 s=j/10 s = j /10 (white solid) and t = j / 10 t=j/10 t = j /10 (dark dashed), j = 1 , … , 9 j=1,\ldots,9 j = 1 , … , 9 .
Predictors and link functions. We sample ( s , t ) (s,t) ( s , t ) uniformly on the parameter square and set X = Q + Z X=Q+Z X = Q + Z , with Q = q ( s , t ) Q=q(s,t) Q = q ( s , t ) and Z Z Z normal to the surface at Q Q Q . As for the bumpy sphere, displacement scales with the numerically estimated local reach; Appendix § 8 .§ 4 gives the sampling law. The first of the two three-component links uses a complex polynomial. Choose fixed a Y > 0 a_Y>0 a Y > 0 and μ Y ∈ R 3 \mu_Y\in\mathbb{R}^3 μ Y ∈ R 3 to center the response and give it unit total variance on the calibration sample (Appendix § 8 .§ 4 ):
z = − 1 + 2 s + i ( 0.25 + 1.5 t ) , p ( z ) = z 7 ( z + 0.6 + 0.2 i ) , z=-1+2s+i(0.25+1.5t), \qquad p(z)=z^7(z+0.6+0.2i), z = − 1 + 2 s + i ( 0.25 + 1.5 t ) , p ( z ) = z 7 ( z + 0.6 + 0.2 i ) ,
f p o l y ( q ( s , t ) ) = a Y [ ( ℜ p ( z ) ∣ p ( z ) ∣ , ℑ p ( z ) ∣ p ( z ) ∣ , log ∣ p ( z ) ∣ ) T − μ Y ] . (35) f^{\mathrm{poly}}(q(s,t))=a_Y\left[\left(\Re\frac{p(z)}{|p(z)|},\Im\frac{p(z)}{|p(z)|},\log|p(z)|\right)^{\mathsf{T}}-\mu_Y\right]. \tag*{(35)} f poly ( q ( s , t )) = a Y [ ( ℜ ∣ p ( z ) ∣ p ( z ) , ℑ ∣ p ( z ) ∣ p ( z ) , log ∣ p ( z ) ∣ ) T − μ Y ] . (35)
The differential has rank two, while a response can have several preimages. Figure 13 shows the components of the link function: the first two encode the phase of the polynomial, and the third its log magnitude.
Figure 13. Components of f p o l y f^{\mathrm{poly}} f poly on the parameter square. All panels use the same diverging color map; the first two share the same color scale.
For the same surface and predictor law, define f r a t f^{\mathrm{rat}} f rat from the rational map
z = − 1 + 2 s + i ( 0.3 + 1.5 t ) , w ( z ) = ( z + 1.3 − 0.1 i z − 1.3 − 1.8 i ) 10 . (36) z=-1+2s+\mathrm{i}(0.3+1.5t),\qquad w(z)=\left(\frac{z+1.3-0.1\mathrm{i}}{z-1.3-1.8\mathrm{i}}\right)^{10}. \tag*{(36)} z = − 1 + 2 s + i ( 0.3 + 1.5 t ) , w ( z ) = ( z − 1.3 − 1.8 i z + 1.3 − 0.1 i ) 10 . (36)
It uses the phase and log-magnitude features in (4.6), with w w w in place of p p p , followed by centering and normalization to unit total variance. This link function also has a ranktwo differential and can have several parameter preimages. Its components are shown in Figure 17 in Appendix § 8 .§ 4 . Both link functions use 1% relative Gaussian noise in training and validation.
Figure 17. Components of f r a t f^{\mathrm{rat}} f rat on the parameter square. The first two panels encode the phase of the rational map and share the same color scale; the third encodes its log magnitude. All panels use the same diverging color map.
Protocol and results. The response-neighbor count and refinement schedule are (B.2)–(B.3), with parameters selected separately for each link function on validation data and held fixed across sample sizes and repetitions (Appendix § 8 .§ 4 ). At each n ∈ { 1 , 2 , 3 , 4 , 5 , 7 } × { 10 4 , 10 5 , 10 6 } n \in\{1,2,3,4,5,7\} \times\{10^{4},10^{5},10^{6}\} n ∈ { 1 , 2 , 3 , 4 , 5 , 7 } × { 1 0 4 , 1 0 5 , 1 0 6 } we perform ten independent repetitions per link, with fresh training data and independent validation and test sets of 5000 queries each. Prediction errors use noiseless test responses. Tangent-space errors compare the estimated planes with the analytic tangent planes at the sampled base points, using (30) .
Figure 14 shows decreasing prediction and tangent-space errors for both link functions across the full sample-size range. Least-squares fits to the mean errors at the eight sample sizes n ≥ 5 × 10 5 n \ge5 \times10^{5} n ≥ 5 × 1 0 5 give prediction slopes − 0.685 -0.685 − 0.685 and − 0.712 -0.712 − 0.712 for f p o l y f^{\mathrm{poly}} f poly and f r a t f^{\mathrm{rat}} f rat , respectively; the corresponding tangent-space slopes are − 0.414 -0.414 − 0.414 and − 0.426 -0.426 − 0.426 . All four coefficients of determination exceed 0.993. The prediction slopes are close to the reference exponent − 2 / 3 -2/3 − 2/3 , while the tangent-space errors decrease faster than the reference exponent − 1 / 3 -1/3 − 1/3 over the fitted range. At n = 7 × 10 6 n = 7 \times10^{6} n = 7 × 1 0 6 , the relative prediction and tangent-space errors are ( 6.72 ± 0.47 ) × 10 − 6 (6.72 \pm0.47) \times10^{-6} ( 6.72 ± 0.47 ) × 1 0 − 6 and 0.03564 ± 0.00058 0.03564 \pm0.00058 0.03564 ± 0.00058 for f p o l y f^{\mathrm{poly}} f poly , and ( 4.33 ± 1.43 ) × 10 − 6 (4.33 \pm1.43) \times10^{-6} ( 4.33 ± 1.43 ) × 1 0 − 6 and 0.07193 ± 0.00059 0.07193 \pm0.00059 0.07193 ± 0.00059 for f r a t f^{\mathrm{rat}} f rat . Each ± \pm ± denotes one sample standard deviation across the ten repetitions.
Figure 14. Prediction and tangent-space errors for the coupled Meyer helix surface with the link functions f p o l y f^{\mathrm{poly}} f poly and f r a t f^{\mathrm{rat}} f rat and 1% response noise. Each marker is the mean of ten independent repetitions for that link function; error bars show one sample standard deviation. The vertical line marks n = 5 × 10 5 n = 5 \times10^{5} n = 5 × 1 0 5 , the lower end of the fitted range. The legends give least-squares slopes of the logarithms of the mean errors. Each link function uses the parameters of Table 8 for all n n n .
First-hit probabilities of an overdamped Langevin diffusion
In molecular dynamics and other stochastic systems with several time scales, trajectories often stay close to a low-dimensional set, and quantities such as committor or first-hit probabilities are approximately functions of a few slow variables, the reaction coordinates [38 , 27 , 86 , 30 ] . Estimating such a probability from simulations started at many initial conditions is a regression problem close to the Manifold-Index Model: the predictor is the initial condition, the manifold is the low-dimensional set, and the response is a vector of probabilities. We consider a synthetic example in which the low-dimensional set is the sphere M = { ( u , 0 , … , 0 ) : u ∈ S 2 } ⊂ R 20 \mathcal{M} = \{(u,0,\ldots,0) : u \in\mathbb{S}^{2}\} \subset\mathbb{R}^{20} M = {( u , 0 , … , 0 ) : u ∈ S 2 } ⊂ R 20 , whose reach is 1.
Diffusion and response. Write x ∈ R 20 x \in\mathbb{R}^{20} x ∈ R 20 as x = ( r u , z ) x = (ru,z) x = ( r u , z ) , with r ≥ 0 r \ge0 r ≥ 0 , u ∈ S 2 u \in\mathbb{S}^{2} u ∈ S 2 and z ∈ R 17 z \in\mathbb{R}^{17} z ∈ R 17 , so that Π M x = ( u , 0 ) \Pi_{\mathcal{M}}x = (u,0) Π M x = ( u , 0 ) when r > 0 r > 0 r > 0 . For an initial condition x x x , let B B B be standard Brownian motion in R 20 \mathbb{R}^{20} R 20 , and let U U U be the potential defined next. Let X x ( t ) X^{x}(t) X x ( t ) solve
d X x ( t ) = − ∇ U ( X x ( t ) ) d t + 0.4 d B ( t ) , X x ( 0 ) = x , (37) dX^{x}(t) = -\nabla U(X^{x}(t))\,dt + 0.4\,dB(t), \qquad X^{x}(0) = x, \tag*{(37)} d X x ( t ) = − ∇ U ( X x ( t )) d t + 0.4 d B ( t ) , X x ( 0 ) = x , (37)
With tangential potential U t a n U_{\mathrm{tan}} U tan and positive normal stiffnesses ν 0 , … , ν 17 \nu_0,\ldots,\nu_{17} ν 0 , … , ν 17 specified in Appendix B.3,
U ( x ) = U t a n ( u ) + ν 0 2 ( r − 1 ) 2 + 1 2 ∑ i = 1 17 ν i z i 2 . (38) U(x) = U_{\mathrm{tan}}(u) + \frac{\nu_0}{2}(r-1)^2 + \frac{1}{2}\sum_{i=1}^{17}\nu_i z_i^2. \tag*{(38)} U ( x ) = U tan ( u ) + 2 ν 0 ( r − 1 ) 2 + 2 1 i = 1 ∑ 17 ν i z i 2 . (38)
The tangential potential is shown in Figure 15 (b), and the quadratic terms confine the normal coordinates. The four targets A 1 , … , A 4 ⊂ R 20 A_1,\ldots,A_4 \subset\mathbb{R}^{20} A 1 , … , A 4 ⊂ R 20 combine the angular regions in Figure 15 (a) with restrictions on the normal coordinates. Appendix B.3 gives the potential, stiffnesses and target boundaries. The regression function F = ( F 1 , … , F 4 ) F = (F_1,\ldots,F_4) F = ( F 1 , … , F 4 ) has the components
Figure 15. The diffusion of § 4.6 . (a) The sphere M \mathcal{M} M in its first three coordinates, seen from the hemispheres u 1 ≥ 0 u_1 \ge0 u 1 ≥ 0 and u 1 ≤ 0 u_1 \le0 u 1 ≤ 0 , with the angular parts of the targets A 1 , … , A 4 A_1,\ldots,A_4 A 1 , … , A 4 . (b) The tangential potential U t a n U_{\mathrm{tan}} U tan as a function of the longitude λ \lambda λ and the latitude φ \varphi φ of u u u ; the white lines bound the angular parts of the targets.
F j ( x ) = P ( X x reaches A j before the other targets ∣ X x reaches a target before time 100 ) , F_j(x) = \mathbb{P}\left(X^{x}\text{ reaches }A_j\text{ before the other targets}\mid X^{x}\text{ reaches a target before time }100\right), F j ( x ) = P ( X x reaches A j before the other targets ∣ X x reaches a target before time 100 ) ,
and the response is a Monte Carlo estimate of F ( x ) F(x) F ( x ) : among the simulated paths of (4.8) that reach a target before time 100, the fractions that reach A 1 , … , A 4 A_1,\ldots,A_4 A 1 , … , A 4 first. Figure 16 shows the resulting response fields. Training responses use 48,000 paths per initial condition, and validation and test responses use 400,000; the only noise is the Monte Carlo error, whose variance depends on F ( x ) F(x) F ( x ) .
Figure 16. Responses of the diffusion of § 4.6 at 60,000 plotted training initial conditions (a subset of the full training set), as functions of the longitude λ \lambda λ and the latitude φ \varphi φ . The panel Y j Y_j Y j shows the j j j -th component of the response, the Monte Carlo estimate of F j F_j F j with 48,000 paths; all panels use the same color scale.
Initial conditions. The initial conditions are X = ( r u , z ) X=(ru,z) X = ( r u , z ) , where u u u is uniform on S 2 \mathbb{S}^2 S 2 , r = 1 + ε r r=1+\varepsilon_r r = 1 + ε r with ε r ∼ N ( 0 , 0.5 2 ) \varepsilon_r\sim\mathcal{N}(0,0.5^2) ε r ∼ N ( 0 , 0. 5 2 ) truncated to [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] , and z ∼ N ( 0 , 0.5 2 I 17 ) z\sim\mathcal{N}(0,0.5^2I_{17}) z ∼ N ( 0 , 0. 5 2 I 17 ) , all independent; as for the curves, the normal coordinates have standard deviation 0.5 before truncation. Their typical distance ( ( r − 1 ) 2 + ∥ z ∥ 2 ) 1 / 2 ((r-1)^2+\lVert z\rVert^2)^{1/2} (( r − 1 ) 2 + ∥ z ∥ 2 ) 1/2 from M \mathcal{M} M is about 2.1, twice the reach, although the base point ( u , 0 ) (u,0) ( u , 0 ) is unique unless r = 0 r=0 r = 0 . The model is approximate: F F F depends on the whole initial condition, but normal relaxation on time scales 1 / ν i ≤ 0.03 1/\nu_i\leq0.03 1/ ν i ≤ 0.03 is much faster than angular motion over distances of order one, so F F F is close to a function of u u u . When u u u lies in the angular part of A j A_j A j , F ( u , 0 ) F(u,0) F ( u , 0 ) is the j j j -th unit vector, violating (A3).
Data and parameters. The data set contains 4 × 10 6 4\times10^6 4 × 1 0 6 initial conditions, of which 5000 are reserved for validation and 5000 for testing; the estimator is fitted to n = 4 × 10 5 n=4\times10^5 n = 4 × 1 0 5 of the others. It uses the parametrization (B.2)–(B.3) in Appendix § 8 .§ 2 , with constants selected on validation data at n = 10 6 n=10^6 n = 1 0 6 and held fixed for the present experiment (Appendix § 8 .§ 3 ).
Results. On the 5000 test conditions, the relative mean squared error is 0.048, and the tangent-space error (30) , computed with the tangent planes of the sphere, is 0.054. Since F F F is not available in closed form, the relative mean squared error is computed against the test responses, which are independent Monte Carlo estimates of F ( X ) F(X) F ( X ) with 400,000 paths, and normalized by the sum of the sample variances of their four components. The estimator recovers the sphere’s tangent planes from points typically about twice its reach away.
Discussion
The Manifold-Index Model separates the dimension of the predictor distribution from the dimension that governs regression. Predictors can fill the tube in all D D D directions, yet complete normal fibers contribute observations without increasing the approximation bias. Multiscale PCA on response preimages identifies the initial tangent planes, and pooling outer products of local slopes refines them. For links with Lipschitz differential (s = 2 s = 2 s = 2 ), Theorem 3.5 gives the m m m -dimensional prediction rate.
MANOR takes the intrinsic dimension m m m as input. At an isolated point where D f Df D f is rank deficient, inferring dimension from the response differential at that point alone would underestimate m m m . Slopes at neighboring base points reveal the missing tangent directions.
In the diffusion experiment of § 4.6 , the reference map u ↦ F ( u , 0 ) u \mapsto F(u, 0) u ↦ F ( u , 0 ) on the sphere equals the j j j th unit vector throughout the angular patch of target A j A_j A j . Predicting this common value does not require identifying the tangent plane: within the patch the reference response is constant in every tangent direction. Away from the sphere, the first-hit probabilities also depend on the normal displacement, so the manifold-index description of the full regression function is approximate.
An open question is whether comparable guarantees hold for p < m p < m p < m , in particular for a scalar response on a manifold of dimension greater than one. Manifolds with boundary require localization near the edge, while anisotropic tubes could exploit longer fibers in directions where the nearest point remains unique. It also remains open whether m m m -dimensional rates can be obtained without inverse regression when the intrinsic geometry is concealed by the normal displacement, and whether fixed response resolution can accommodate unbounded noise as the sample size grows.
The estimated tangent field defines a spatially varying anisotropic metric. This suggests using the same neighborhood geometry for broader regression classes and for metric learning. Models with several layers of composition raise the further question of how to estimate and refine the geometry at each layer. Tree-based search could reduce the computational cost of constructing the neighborhoods.
Acknowledgments and Disclosure of Funding
This research was partially supported by AFOSR grants FA9550-21-1-0317 and FA9550-23-1-0445. Prisma Analytics Inc. provided and managed the computing resources used for the numerical experiments. AI tools (ChatGPT, Claude and Grok) assisted with developing and checking proofs, manuscript preparation, and implementation of the numerical experiments, including CUDA kernels for the GPU implementation of the estimator.
Proof of Theorem 3.5
As in the curve analysis of Wu and Maggioni [102 ] , we first localize the predictors by response. Here the resulting estimates initialize a moving tangent field. After the estimator definitions and geometric tools, Appendices § 7 .§ 3 –§ 7 .§ 5 follow the three phases; Appendices § 7 .6 and § 7 .7 give the sample threshold and the integrated error bounds.
Precise definitions
Number of steps. The number k ∗ k_\ast k ∗ of refinement steps and the block size n f n_f n f are fixed by a deterministic rule that uses no data. Let K 0 > 0 K_0>0 K 0 > 0 be the initial tangential-width multiplier, a fixed constant whose value is chosen in the paragraph on starting widths below. For j ≥ 0 j\geq0 j ≥ 0 put
n f ( j ) = ⌊ n 4 j + 8 ⌋ , b j = K 0 w Y e 1 / [ 2 ( D + 4 ) ] a ρ j , n_f(j)=\left\lfloor\frac{n}{4j+8}\right\rfloor,\qquad b_j=K_0w_Y e^{1/[2(D+4)]}a_\rho^j, n f ( j ) = ⌊ 4 j + 8 n ⌋ , b j = K 0 w Y e 1/ [ 2 ( D + 4 )] a ρ j ,
and choose the largest number of steps j ≥ 1 j\geq1 j ≥ 1 for which
n f ( j ) π − σ N m ( b j / σ N ) m + 4 ≥ C s a m p ( 1 + D + p ) 4 ( 1 + R / σ N ) 16 ( 1 + σ ζ σ f , + σ N ) 2 log ( 2 n ) . (39) n_f(j)\,\pi^{-\sigma_N^m}(b_j/\sigma_N)^{m+4}\geq C_{\mathrm{samp}}(1+D+p)^4(1+R/\sigma_N)^{16}\left(1+\frac{\sigma_\zeta}{\sigma_{f,+}\sigma_N}\right)^2\log(2n). \tag*{(39)} n f ( j ) π − σ N m ( b j / σ N ) m + 4 ≥ C samp ( 1 + D + p ) 4 ( 1 + R / σ N ) 16 ( 1 + σ f , + σ N σ ζ ) 2 log ( 2 n ) . (39)
Set k ∗ = j k_\ast=j k ∗ = j and n f = n f ( k ∗ ) n_f=n_f(k_\ast) n f = n f ( k ∗ ) . This comparison is monotone in j j j . Since a ρ m + 4 = e − 1 / 2 a_\rho^{m+4}=e^{-1/2} a ρ m + 4 = e − 1/2 , k ∗ = O ( log n ) k_\ast=O(\log n) k ∗ = O ( log n ) and n f ≳ n / log ( 2 n ) n_f\gtrsim n/\log(2n) n f ≳ n / log ( 2 n ) . The multiplier C s a m p C_{\mathrm{samp}} C samp , and the multiplier C p i l C_{\mathrm{pil}} C pil of the pilot sample condition (A.39), are fixed constants and are part of the estimator. They depend only on the fixed model parameters, the kernel and τ \tau τ ; the requirements on them are collected in Appendix § 7 .6.
Sample splitting. With k ∗ k_\ast k ∗ as in § 3.2 and n f = ⌊ n / ( 4 k ∗ + 8 ) ⌋ n_f=\lfloor n/(4k_\ast+8)\rfloor n f = ⌊ n / ( 4 k ∗ + 8 )⌋ , the observations are divided into 4 k ∗ + 8 4k_\ast+8 4 k ∗ + 8 disjoint blocks of n f n_f n f or n f + 1 n_f+1 n f + 1 observations. At each step k k k , one block supplies S k f i t S_k^{\mathrm{fit}} S k fit and another S k p o o l S_k^{\mathrm{pool}} S k pool . Of the remaining blocks, reserve at least one for prediction (S p r e d S_{\mathrm{pred}} S pred ) and at least two for the pilot. Fix the two pilot fold sizes in advance, each at least n f n_f n f . Independently of the data, draw S a n c S_{\mathrm{anc}} S anc uniformly from the pooled pilot indices at its prescribed size; the complement is S d e s S_{\mathrm{des}} S des . Conditional on these labels, the observations in disjoint folds are independent. Write n p r e d = ∣ S p r e d ∣ n_{\mathrm{pred}}=|S_{\mathrm{pred}}| n pred = ∣ S pred ∣ . The total pilot size used in Corollary A.14 is N p i l = ∣ S a n c ∣ + ∣ S d e s ∣ N_{\mathrm{pil}}=|S_{\mathrm{anc}}|+|S_{\mathrm{des}}| N pil = ∣ S anc ∣ + ∣ S des ∣ .
Phase 1. For i ∈ S a n c i\in S_{\mathrm{anc}} i ∈ S anc , compute the centered covariance of the observations j ∈ S d e s j\in S_{\mathrm{des}} j ∈ S des with ∥ Y j − Y i ∥ ≤ δ Y \lVert Y_j-Y_i\rVert\leq\delta_Y ∥ Y j − Y i ∥ ≤ δ Y and ∥ X j − X i ∥ ≤ r \lVert X_j-X_i\rVert\leq r ∥ X j − X i ∥ ≤ r . Keep a radius when the number of selected observations is at least ( p 0 / 8 ) ∣ S d e s ∣ (p_0/8)|S_{\mathrm{des}}| ( p 0 /8 ) ∣ S des ∣ and the ( D − m ) (D-m) ( D − m ) -th largest eigenvalue λ D − m \lambda_{D-m} λ D − m of their covariance is at least one eighth of the normal variance lower bound σ N 2 \sigma_N^2 σ N 2 in (A2). A sharper known lower bound for inf q λ min ( Σ N ( q ) ∣ N q M ) \inf_q\lambda_{\min}(\Sigma_N(q)|_{N_qM}) inf q λ m i n ( Σ N ( q ) ∣ N q M ) , when available, replaces σ N 2 \sigma_N^2 σ N 2 in this test. Resolve ties at the boundary of a selected eigenspace by a fixed deterministic rule. An anchor is eligible if at least one radius is kept. For each eligible anchor let r i r_i r i be the smallest kept radius maximizing the score (11) ; store the mean of the observations selected at radius r i r_i r i , its selected mean , and the bottom m m m eigenvectors V i V_i V i of their covariance. We call V i V i ⊤ V_iV_i^\top V i V i ⊤ its raw projector . Write Ω i a n c \Omega_i^{\mathrm{anc}} Ω i anc for the eligible anchors in its response window and spatial ball, its projector pool. The warm start at the anchor is its pooled projector
P ^ p i l ( X i ) = P T o p m ( 1 ∣ Ω i a n c ∣ ∑ j ∈ Ω i a n c V j V j T ) . (40) \widehat{P}_{\mathrm{pil}}(X_i)=P_{\mathrm{Top}_m}\left(\frac{1}{|\Omega_i^{\mathrm{anc}}|}\sum_{j\in\Omega_i^{\mathrm{anc}}}V_jV_j^{\mathsf{T}}\right). \tag*{(40)} P pil ( X i ) = P Top m ∣ Ω i anc ∣ 1 j ∈ Ω i anc ∑ V j V j T . (40)
Self-inclusion makes this pool nonempty. If no anchor is eligible, record a pilot failure and use a fixed rank-m m m projector.
Assignment to a target. Choose C l i n k C_{\mathrm{link}} C link large enough to join anchors satisfying the event in (19) for the same base point q q q (Appendix § 7 .§ 3 ). Link two eligible anchors if their responses differ by at most 3 δ Y / 4 3\delta_Y/4 3 δ Y /4 and their selected means differ by at most C l i n k w Y C_{\mathrm{link}}w_Y C link w Y . An anchor star S S S consists of an eligible anchor, its owner , and the eligible anchors linked to it. Retain the anchor stars with at least ( p 0 / 8 ) ∣ S a n c ∣ (p_0/8)|S_{\mathrm{anc}}| ( p 0 /8 ) ∣ S anc ∣ anchors, a threshold below p 0 ∣ S a n c ∣ p_0|S_{\mathrm{anc}}| p 0 ∣ S anc ∣ , the lower bound on the expected number of anchors satisfying (19) . The center x S x_S x S of an anchor star is the average of the selected means of its anchors, and P S P_S P S is the projector onto the top m m m eigenspace of the average of their pooled projectors. Among the retained anchor stars with ∥ x − x S ∥ ≤ 2 R \lVert x-x_S\rVert\le2R ∥ x − x S ∥ ≤ 2 R , assign to the target x x x one minimizing the tangential residual ∥ P S ( x − x S ) ∥ \lVert P_S(x-x_S)\rVert ∥ P S ( x − x S )∥ , breaking ties by index. Set P ^ 0 ( x ) = P S \widehat{P}_0(x)=P_S P 0 ( x ) = P S . The local cylinder of x x x is the set (22) defined by this anchor star. If no anchor star qualifies, the local cylinder is empty and the initial projector is a fixed rank-m m m projector.
Starting widths. Take H 0 ≥ 3 H_0\ge3 H 0 ≥ 3 from Lemma A.12, let K 0 > 0 K_0>0 K 0 > 0 be the initial tangential-width multiplier chosen below, and set
h r e s c a l e = H 0 R / min R , h 0 = h r e s c a l e avg i e l i g i b l e r i , ρ 0 = K 0 w Y H 0 R . (41) h_{\mathrm{rescale}}=H_0R/\min\mathcal{R},\qquad h_0=h_{\mathrm{rescale}}\underset{i\ \mathrm{eligible}}{\operatorname{avg}}\,r_i,\qquad\rho_0=\frac{K_0w_Y}{H_0R}. \tag*{(41)} h rescale = H 0 R / min R , h 0 = h rescale i eligible avg r i , ρ 0 = H 0 R K 0 w Y . (41)
On a pilot failure, when no anchor is eligible, put h 0 = h r e s c a l e min R = H 0 R h_0=h_{\mathrm{rescale}}\min\mathcal{R}=H_0R h 0 = h rescale min R = H 0 R . Thus h k / R h_k/R h k / R lies between H 0 H_0 H 0 and H 0 ( max R / min R ) e 1 / [ 2 ( D + 4 ) ] H_0(\max\mathcal{R}/\min\mathcal{R})^{e^{1/[2(D+4)]}} H 0 ( max R / min R ) e 1/ [ 2 ( D + 4 )] . The ratio of the two endpoints is bounded at fixed c c c . The local cylinder (22) restricts the base points independently of the normal width h k h_k h k . Choose the fixed K 0 K_0 K 0 large enough that the pilot error is at most η b 1 / ( 2 R ) \eta b_1/(2R) η b 1 / ( 2 R ) and its first regular update in (A.33) is at most a ρ η b 1 / ( 2 R ) a_\rho\eta b_1/(2R) a ρ η b 1 / ( 2 R ) . Then
K 0 w Y ≤ b 0 = ρ 0 h 0 ≤ K 0 w Y max R / min R . (42) K_0w_Y\le b_0=\rho_0h_0\le K_0w_Y\max\mathcal{R}/\min\mathcal{R}. \tag*{(42)} K 0 w Y ≤ b 0 = ρ 0 h 0 ≤ K 0 w Y max R / min R . (42)
By (A5), every width in this interval is at most K 0 ε 0 ( max R / min R ) min { σ N , r c y l } K_0\varepsilon_0(\max\mathcal{R}/\min\mathcal{R})\min\{\sigma_N,r_{\mathrm{cyl}}\} K 0 ε 0 ( max R / min R ) min { σ N , r cyl } . Use the r c y l r_{\mathrm{cyl}} r cyl in (A5), h ∗ = h k ∗ h_*=h_{k_*} h ∗ = h k ∗ , ρ ∗ = c ρ ρ k ∗ \rho_*=c_\rho\rho_{k_*} ρ ∗ = c ρ ρ k ∗ , and b ∗ = ρ ∗ h ∗ b_*=\rho_*h_* b ∗ = ρ ∗ h ∗ . Choose the fixed c ρ ≥ 1 c_\rho\ge1 c ρ ≥ 1 so that both the true and estimated prediction kernels satisfy the lower bound in Lemma A.11, as used in Appendix § 7 .§ 5 ; at this resolution the cylinder-restricted prediction support stays strictly inside the local base chart of Lemma A.11.
Phase 2. Use the neighborhoods W x , k \mathcal{W}_{x,k} W x , k and fitting weights ω u , k \omega_{u,k} ω u , k of (24) . For each u = X j u=X_j u = X j , j ∈ S k p o o l j\in S_k^{\mathrm{pool}} j ∈ S k pool , compute and store the slope
( β ^ 0 , u , k , G ^ u , k ) ∈ arg min β 0 , G ∑ ℓ ∈ S k f i t ω u , k ( X ℓ ) ∥ Y ℓ − β 0 − G T ( X ℓ − u ) ∥ 2 . (\widehat{\beta}_{0,u,k},\widehat{G}_{u,k})\in\underset{\beta_0,G}{\operatorname{arg\,min}}\ \sum_{\ell\in S_k^{\mathrm{fit}}}\omega_{u,k}(X_\ell)\lVert Y_\ell-\beta_0-G^{\mathsf{T}}(X_\ell-u)\rVert^2. ( β 0 , u , k , G u , k ) ∈ β 0 , G arg min ℓ ∈ S k fit ∑ ω u , k ( X ℓ ) ∥ Y ℓ − β 0 − G T ( X ℓ − u ) ∥ 2 .
The fit uses its own local cylinder and metric and does not depend on the target that later uses it. Retain the fit only if at least one observation has positive weight and its weighted Gram matrix, including the intercept, is invertible. The concentration bound (A.34) guarantees this uniformly. Let I x \mathcal{I}_x I x be the pool-fold indices whose centers lie in W x , k \mathcal{W}_{x,k} W x , k and whose fits pass. Define
C ^ k ( x ) = 1 ∣ I x ∣ ∑ j ∈ I x G ^ X j , k G ^ X j , k T , P ^ k ( x ) = P Top m ( C ^ k ( x ) ) . (43) \widehat{C}_k(x) = \frac{1}{|\mathcal{I}_x|} \sum_{j \in\mathcal{I}_x} \widehat{G}_{X_j,k}\widehat{G}_{X_j,k}^{\mathsf{T}}, \qquad\widehat{P}_k(x) = P_{\operatorname{Top}_m}\left(\widehat{C}_k(x)\right). \tag*{(43)} C k ( x ) = ∣ I x ∣ 1 j ∈ I x ∑ G X j , k G X j , k T , P k ( x ) = P Top m ( C k ( x ) ) . (43)
If I x \mathcal{I}_x I x is empty, retain P ^ k − 1 ( x ) \widehat{P}_{k-1}(x) P k − 1 ( x ) . All fits pass on the uniform concentration event, so filtering removes no centers from the pool.
Population counterparts. Let G u , k G_{u,k} G u , k be the slope of the population affine fit with weights ω u , k \omega_{u,k} ω u , k under the full predictor law. The population pooled matrix averages its outer product over the target neighborhood:
C k ( x ) = E [ G X , k G X , k T ∣ X ∈ W x , k ] . (44) C_k(x) = \mathbb{E}\left[G_{X,k}G_{X,k}^{\mathsf{T}} \mid X \in\mathcal{W}_{x,k}\right]. \tag*{(44)} C k ( x ) = E [ G X , k G X , k T ∣ X ∈ W x , k ] . (44)
Both the fitting mass E ω u , k ( X ) \mathbb{E}\,\omega_{u,k}(X) E ω u , k ( X ) and the neighborhood probability P { X ∈ W x , k } \mathbb{P}\{X \in\mathcal{W}_{x,k}\} P { X ∈ W x , k } are of order π k − b m \pi^{-b_m}_k π k − b m by the kernel and probability bounds in Proposition 3.2, so N k = n f π k − b m N_k = n_f\pi^{-b_m}_k N k = n f π k − b m is the effective local count for fitting and pooling.
Phase 3. With the prediction widths h k ∗ h_{k_*} h k ∗ and b k ∗ = ρ ∗ h k ∗ b_{k_*} = \rho_*h_{k_*} b k ∗ = ρ ∗ h k ∗ , for a query x x x and i ∈ S p r e d i \in\mathcal{S}_{\mathrm{pred}} i ∈ S pred let
ϖ i ( x ) = χ 0 ( x , X i ) K ( ∥ x − X i ∥ 2 h k ∗ 2 + ∥ P ^ k ∗ ( X i ) ( x − X i ) ∥ 2 b k ∗ 2 ) , F ^ ( x ) = ∑ i ∈ S p r e d ϖ i ( x ) Y i ∑ i ∈ S p r e d ϖ i ( x ) . (45) \varpi_i(x) = \chi_0(x,X_i)K\left(\frac{\lVert x-X_i\rVert^2}{h_{k_*}^2} + \frac{\lVert\widehat{P}_{k_*}(X_i)(x-X_i)\rVert^2}{b_{k_*}^2}\right), \qquad\widehat{F}(x) = \frac{\sum_{i \in\mathcal{S}_{\mathrm{pred}}}\varpi_i(x)Y_i}{\sum_{i \in\mathcal{S}_{\mathrm{pred}}}\varpi_i(x)}. \tag*{(45)} ϖ i ( x ) = χ 0 ( x , X i ) K ( h k ∗ 2 ∥ x − X i ∥ 2 + b k ∗ 2 ∥ P k ∗ ( X i ) ( x − X i ) ∥ 2 ) , F ( x ) = ∑ i ∈ S pred ϖ i ( x ) ∑ i ∈ S pred ϖ i ( x ) Y i . (45)
with the prediction-fold mean used if the denominator vanishes. The weights depend only on predictors and on the final field of tangent spaces.
Targets. The estimator defines a field on the whole tube by evaluating (A.5) at any point x x x of the tube. Each step uses the same stored center slopes for all such evaluations. Uniform bounds apply in particular at future pool centers and prediction predictors. Conditional arguments use only the Phase 1 and completed refinement folds; the current fit and pool folds and prediction responses remain independent of this information.
Tools
We collect the moment, regression, subspace and integration estimates used by the 3 phases.
Lemma A.1 (Differential of the projection) Under (A1), for x = Q + z x = Q + z x = Q + z with Q = Π M x Q = \Pi_{\mathcal{M}}x Q = Π M x , z ∈ N Q M z \in N_Q\mathcal{M} z ∈ N Q M and ∥ z ∥ ≤ R = c reach M \lVert z\rVert\le R = c\,\operatorname{reach}_{\mathcal{M}} ∥ z ∥ ≤ R = c reach M ,
D Π M ( Q + z ) = E Q ( I − S z ) − 1 E Q T , ( 1 + c ) − 1 I ⪯ ( I − S z ) − 1 ⪯ ( 1 − c ) − 1 I . D\Pi_{\mathcal{M}}(Q + z) = E_Q(I - S_z)^{-1}E_Q^{\mathsf{T}}, \qquad(1+c)^{-1}I \preceq(I-S_z)^{-1} \preceq(1-c)^{-1}I. D Π M ( Q + z ) = E Q ( I − S z ) − 1 E Q T , ( 1 + c ) − 1 I ⪯ ( I − S z ) − 1 ⪯ ( 1 − c ) − 1 I .
Proof Choose an orthonormal normal frame N q ∈ R D × ( D − m ) N_q \in\mathbb{R}^{D \times(D-m)} N q ∈ R D × ( D − m ) near Q Q Q whose normal connection vanishes at Q Q Q , and write z = N Q ξ 0 z = N_Q\xi_0 z = N Q ξ 0 . Differentiate ( q , ξ ) ↦ q + N q ξ (q,\xi) \mapsto q + N_q\xi ( q , ξ ) ↦ q + N q ξ , with ξ ∈ R D − m \xi\in\mathbb{R}^{D-m} ξ ∈ R D − m , at ( Q , ξ 0 ) (Q,\xi_0) ( Q , ξ 0 ) in the base direction E Q a E_Qa E Q a , a ∈ R m a \in\mathbb{R}^m a ∈ R m . The base contributes E Q a E_Qa E Q a ; the Weingarten equation gives the frame contribution − E Q S z a -E_QS_za − E Q S z a , with zero normal component at Q Q Q . Thus the derivative is E Q ( I − S z ) a E_Q(I-S_z)a E Q ( I − S z ) a . Differentiating Π M ( q + N q ξ ) = q \Pi_{\mathcal{M}}(q + N_q\xi) = q Π M ( q + N q ξ ) = q in base and normal directions gives D Π M ( Q + z ) E Q ( I − S z ) = E Q D\Pi_{\mathcal{M}}(Q+z)E_Q(I-S_z)=E_Q D Π M ( Q + z ) E Q ( I − S z ) = E Q and D Π M ( Q + z ) P Q ⊥ = 0 D\Pi_{\mathcal{M}}(Q+z)P_Q^\perp=0 D Π M ( Q + z ) P Q ⊥ = 0 , which is (8) . Finally, the reach bounds the second fundamental form, ∥ S z ∥ ≤ ∥ z ∥ / reach M ≤ c \lVert S_z\rVert\leq\lVert z\rVert/\operatorname{reach}_{\mathcal{M}}\leq c ∥ S z ∥ ≤ ∥ z ∥ / reach M ≤ c [82 ] , so the eigenvalues of the symmetric matrix I − S z I-S_z I − S z lie in [ 1 − c , 1 + c ] [1-c,1+c] [ 1 − c , 1 + c ] . ■
Lemma A.2 (Response events do not select normal displacements). Under (A5), let E E E be an event that depends only on ( Q , ζ ) (Q,\zeta) ( Q , ζ ) , for instance E = { ∥ Y − y ∥ ≤ δ } E=\{\lVert Y-y\rVert\leq\delta\} E = {∥ Y − y ∥ ≤ δ } for fixed y y y and δ \delta δ . Then for every bounded function b b b ,
E [ b ( Z ) 1 E ∣ Q ] = E [ b ( Z ) ∣ Q ] P ( E ∣ Q ) . \mathbb{E}[b(Z)\mathbf{1}_E\mid Q]=\mathbb{E}[b(Z)\mid Q]\mathbb{P}(E\mid Q). E [ b ( Z ) 1 E ∣ Q ] = E [ b ( Z ) ∣ Q ] P ( E ∣ Q ) . Equivalently, conditioning on E E E in addition to Q Q Q leaves the conditional law of Z Z Z unchanged.
Proof. The noise is independent of X = Q + Z X=Q+Z X = Q + Z , so conditionally on Q Q Q the variables Z Z Z and ζ \zeta ζ are independent, and 1 E \mathbf{1}_E 1 E is a function of ( Q , ζ ) (Q,\zeta) ( Q , ζ ) . An independent random anchor (such as Y i Y_i Y i in a response preimage) is handled by conditioning on it. If V ⊂ R D \mathcal{V}\subset\mathbb{R}^D V ⊂ R D is such that, on E E E , the event { X ∈ V } \{X\in\mathcal{V}\} { X ∈ V } is determined by Q Q Q (for instance, when V \mathcal{V} V contains the whole normal fiber over every base point compatible with E E E ), then E ∩ { X ∈ V } E\cap\{X\in\mathcal{V}\} E ∩ { X ∈ V } is again determined by ( Q , ζ ) (Q,\zeta) ( Q , ζ ) and the identity holds with it in place of E E E . ■
Lemma A.3 (Changes of normalized averages). Let μ 0 , μ 1 \mu_0,\mu_1 μ 0 , μ 1 be finite positive measures on the same space, with m j = μ j ( 1 ) > 0 m_j=\mu_j(1)>0 m j = μ j ( 1 ) > 0 , and put h ˉ j = μ j ( h ) / m j \bar{h}_j=\mu_j(h)/m_j h ˉ j = μ j ( h ) / m j for a bounded vector- or matrix-valued function h h h . Write Δ = ∫ ∣ d μ 1 − d μ 0 ∣ \Delta=\int\lvert d\mu_1-d\mu_0\rvert Δ = ∫ ∣ d μ 1 − d μ 0 ∣ and osc ( h ) = sup x , y ∥ h ( x ) − h ( y ) ∥ \operatorname{osc}(h)=\sup_{x,y}\lVert h(x)-h(y)\rVert osc ( h ) = sup x , y ∥ h ( x ) − h ( y )∥ on their joint support. Then
h ˉ 1 − h ˉ 0 = ( μ 1 − μ 0 ) ( h − h ˉ 0 ) m 1 , ∥ h ˉ 1 − h ˉ 0 ∥ ≤ Δ m 1 osc ( h ) . (46) \bar{h}_1-\bar{h}_0=\frac{(\mu_1-\mu_0)(h-\bar{h}_0)}{m_1},\qquad\lVert\bar{h}_1-\bar{h}_0\rVert\leq\frac{\Delta}{m_1}\operatorname{osc}(h). \tag*{(46)} h ˉ 1 − h ˉ 0 = m 1 ( μ 1 − μ 0 ) ( h − h ˉ 0 ) , ∥ h ˉ 1 − h ˉ 0 ∥ ≤ m 1 Δ osc ( h ) . (46) Replacing h h h under μ 1 \mu_1 μ 1 by a function within ϵ \epsilon ϵ uniformly adds at most ϵ \epsilon ϵ to this bound.
Proof. Subtract the two normalized integrals. Since h ˉ 0 \bar{h}_0 h ˉ 0 is an average of h h h , ∥ h − h ˉ 0 ∥ ≤ osc ( h ) \lVert h-\bar{h}_0\rVert\leq\operatorname{osc}(h) ∥ h − h ˉ 0 ∥ ≤ osc ( h ) ; integration against the variation measure gives the bound. ■
Lemma A.4 (Local affine fits). Let ϖ ℓ ≥ 0 \varpi_\ell\geq0 ϖ ℓ ≥ 0 , x ℓ ∈ R D x_\ell\in\mathbb{R}^D x ℓ ∈ R D and a ℓ ∈ R p a_\ell\in\mathbb{R}^p a ℓ ∈ R p , ℓ = 1 , … , N \ell=1,\ldots,N ℓ = 1 , … , N , with at least one positive weight, and put
W = ∑ ℓ ϖ ℓ , a ˉ = W − 1 ∑ ℓ ϖ ℓ a ℓ , x ˉ = W − 1 ∑ ℓ ϖ ℓ x ℓ , M = ∑ ℓ ϖ ℓ ( x ℓ − x ˉ ) ( x ℓ − x ˉ ) T . W=\sum_\ell\varpi_\ell,\qquad\bar{a}=W^{-1}\sum_\ell\varpi_\ell a_\ell,\qquad\bar{x}=W^{-1}\sum_\ell\varpi_\ell x_\ell,\qquad M=\sum_\ell\varpi_\ell(x_\ell-\bar{x})(x_\ell-\bar{x})^{\mathsf{T}}. W = ℓ ∑ ϖ ℓ , a ˉ = W − 1 ℓ ∑ ϖ ℓ a ℓ , x ˉ = W − 1 ℓ ∑ ϖ ℓ x ℓ , M = ℓ ∑ ϖ ℓ ( x ℓ − x ˉ ) ( x ℓ − x ˉ ) T . If M M M is invertible, set γ ℓ = ϖ ℓ M − 1 ( x ℓ − x ˉ ) \gamma_\ell=\varpi_\ell M^{-1}(x_\ell-\bar{x}) γ ℓ = ϖ ℓ M − 1 ( x ℓ − x ˉ ) . The weighted least-squares affine fit of a ℓ a_\ell a ℓ on x ℓ x_\ell x ℓ has intercept β 0 \beta_0 β 0 and slope G G G with
β 0 = a ˉ − G T x ˉ , G = M − 1 ∑ ℓ ϖ ℓ ( x ℓ − x ˉ ) a ℓ T = ∑ ℓ γ ℓ a ℓ T . (47) \beta_0=\bar{a}-G^{\mathsf{T}}\bar{x},\qquad G=M^{-1}\sum_\ell\varpi_\ell(x_\ell-\bar{x})a_\ell^{\mathsf{T}}=\sum_\ell\gamma_\ell a_\ell^{\mathsf{T}}. \tag*{(47)} β 0 = a ˉ − G T x ˉ , G = M − 1 ℓ ∑ ϖ ℓ ( x ℓ − x ˉ ) a ℓ T = ℓ ∑ γ ℓ a ℓ T . (47)
The coefficients satisfy ∑ ℓ γ ℓ = 0 \sum_{\ell}\gamma_{\ell}=0 ∑ ℓ γ ℓ = 0 and ∑ ℓ γ ℓ ( x ℓ − x ˉ ) T = I D \sum_{\ell}\gamma_{\ell}(x_{\ell}-\bar{x})^{\mathsf{T}}=I_D ∑ ℓ γ ℓ ( x ℓ − x ˉ ) T = I D . Consequently constants are removed exactly, an affine response is reproduced exactly, and if a ℓ = c 0 + G 0 T x ℓ + e ℓ a_{\ell}=c_0+G_0^{\mathsf{T}}x_{\ell}+e_{\ell} a ℓ = c 0 + G 0 T x ℓ + e ℓ , then
G = G 0 + ∑ ℓ γ ℓ e ℓ T . (48) G=G_0+\sum_{\ell}\gamma_{\ell}e_{\ell}^{\mathsf{T}}. \tag*{(48)} G = G 0 + ℓ ∑ γ ℓ e ℓ T . (48)
Let Σ a = W − 1 ∑ ℓ ϖ ℓ ( a ℓ − a ˉ ) ( a ℓ − a ˉ ) T \Sigma_a=W^{-1}\sum_{\ell}\varpi_{\ell}(a_{\ell}-\bar{a})(a_{\ell}-\bar{a})^{\mathsf{T}} Σ a = W − 1 ∑ ℓ ϖ ℓ ( a ℓ − a ˉ ) ( a ℓ − a ˉ ) T and osc ( a ) = max ℓ , ℓ ′ ∥ a ℓ − a ℓ ′ ∥ \operatorname{osc}(a)=\max_{\ell,\ell'}\lVert a_{\ell}-a_{\ell'}\rVert osc ( a ) = max ℓ , ℓ ′ ∥ a ℓ − a ℓ ′ ∥ . For every orthogonal projector P P P ,
∥ P G ∥ ≤ ( W ∥ P M − 1 P ∥ ∥ Σ a ∥ ) 1 / 2 ≤ ( W ∥ P M − 1 P ∥ ) 1 / 2 osc ( a ) . (49) \lVert PG\rVert\leq(W\lVert PM^{-1}P\rVert\lVert\Sigma_a\rVert)^{1/2}\leq(W\lVert PM^{-1}P\rVert)^{1/2}\operatorname{osc}(a). \tag*{(49)} ∥ P G ∥ ≤ ( W ∥ P M − 1 P ∥ ∥ Σ a ∥ ) 1/2 ≤ ( W ∥ P M − 1 P ∥ ) 1/2 osc ( a ) . (49)
The same statements hold for weighted integrals against a measure in place of finite sums. Under the normalized weighted law, suppose t = ξ − Γ ( z − y ) t=\xi-\Gamma(z-y) t = ξ − Γ ( z − y ) , with fixed Γ , y \Gamma,y Γ , y , Cov ( ξ , z ) = 0 \operatorname{Cov}(\xi,z)=0 Cov ( ξ , z ) = 0 and invertible coordinate covariances. The fit of a response a a a written as β + B t + C z \beta+Bt+Cz β + B t + C z satisfies
B = Cov ( a , ξ ) Cov ( ξ ) − 1 , C = Cov ( a , z ) Cov ( z ) − 1 + B Γ . (50) B=\operatorname{Cov}(a,\xi)\operatorname{Cov}(\xi)^{-1},\qquad C=\operatorname{Cov}(a,z)\operatorname{Cov}(z)^{-1}+B\Gamma. \tag*{(50)} B = Cov ( a , ξ ) Cov ( ξ ) − 1 , C = Cov ( a , z ) Cov ( z ) − 1 + B Γ. (50)
Proof. The centered normal equations give (A.9); ∑ ℓ γ ℓ = 0 \sum_{\ell}\gamma_{\ell}=0 ∑ ℓ γ ℓ = 0 and ∑ ℓ γ ℓ ( x ℓ − x ˉ ) T = I D \sum_{\ell}\gamma_{\ell}(x_{\ell}-\bar{x})^{\mathsf{T}}=I_D ∑ ℓ γ ℓ ( x ℓ − x ˉ ) T = I D give (A.10). Under the normalized weighted law, Cauchy–Schwarz yields, for unit v ∈ Ran P v\in\operatorname{Ran}P v ∈ Ran P and w ∈ R P w\in\mathbb{R}^{P} w ∈ R P ,
∣ v T G w ∣ 2 ≤ W ( v T M − 1 v ) ( w T Σ a w ) . \lvert v^{\mathsf{T}}Gw\rvert^2\leq W(v^{\mathsf{T}}M^{-1}v)(w^{\mathsf{T}}\Sigma_aw). ∣ v T Gw ∣ 2 ≤ W ( v T M − 1 v ) ( w T Σ a w ) .
Taking suprema and using ∥ Σ a ∥ ≤ osc ( a ) 2 \lVert\Sigma_a\rVert\leq\operatorname{osc}(a)^2 ∥ Σ a ∥ ≤ osc ( a ) 2 proves (A.11). For (A.12), regress on the uncorrelated coordinates ( ξ , z ) (\xi,z) ( ξ , z ) , then substitute ξ = t + Γ ( z − y ) \xi=t+\Gamma(z-y) ξ = t + Γ ( z − y ) .
Lemma A.5 (Perturbation of normal equations). Let Q ⪰ λ I Q\succeq\lambda I Q ⪰ λ I with λ > 0 \lambda>0 λ > 0 , C = Q − 1 B C=Q^{-1}B C = Q − 1 B , and C ^ = Q ^ − 1 B ^ \widehat{C}=\widehat{Q}^{-1}\widehat{B} C = Q − 1 B , where Q ^ \widehat{Q} Q is symmetric and ∥ Q ^ − Q ∥ ≤ λ / 2 \lVert\widehat{Q}-Q\rVert\leq\lambda/2 ∥ Q − Q ∥ ≤ λ /2 . Then
∥ C ^ − C ∥ ≤ 2 λ ( ∥ B ^ − B ∥ + ∥ Q ^ − Q ∥ ∥ C ∥ ) . (51) \lVert\widehat{C}-C\rVert\leq\frac{2}{\lambda}\left(\lVert\widehat{B}-B\rVert+\lVert\widehat{Q}-Q\rVert\lVert C\rVert\right). \tag*{(51)} ∥ C − C ∥ ≤ λ 2 ( ∥ B − B ∥ + ∥ Q − Q ∥ ∥ C ∥ ) . (51) More specifically, let T , N T,N T , N be linear maps from finite-dimensional Euclidean spaces into a Hilbert space, with T T T of full column rank. Let H = ( T ∗ T ) 1 / 2 H=(T^{*}T)^{1/2} H = ( T ∗ T ) 1/2 , C = T † N C=T^{\dagger}N C = T † N , V = N − T C V=N-TC V = N − T C , and C ^ = ( T + E ) † ( N + F ) \widehat{C}=(T+E)^{\dagger}(N+F) C = ( T + E ) † ( N + F ) . If ϵ = ∥ E H − 1 ∥ ≤ 1 / 2 \epsilon=\lVert EH^{-1}\rVert\leq1/2 ϵ = ∥ E H − 1 ∥ ≤ 1/2 , then
∥ H ( C ^ − C ) ∥ ≤ 4 { ( 1 + ϵ ) ∥ F − E C ∥ + ϵ ∥ V ∥ } . (52) \lVert H(\widehat{C}-C)\rVert\leq4\{(1+\epsilon)\lVert F-EC\rVert+\epsilon\lVert V\rVert\}. \tag*{(52)} ∥ H ( C − C )∥ ≤ 4 {( 1 + ϵ ) ∥ F − E C ∥ + ϵ ∥ V ∥} . (52)
Proof. Subtracting the normal equations gives Q ^ ( C ^ − C ) = B ^ − B − ( Q ^ − Q ) C \widehat{Q}(\widehat{C}-C)=\widehat{B}-B-(\widehat{Q}-Q)C Q ( C − C ) = B − B − ( Q − Q ) C , hence (A.13). For the second assertion, T ∗ V = 0 T^{*}V=0 T ∗ V = 0 gives the normal-equation identity
( T + E ) ∗ ( T + E ) ( C ^ − C ) = ( T + E ) ∗ ( F − E C ) + E ∗ V . (T+E)^{*}(T+E)(\widehat{C}-C)=(T+E)^{*}(F-EC)+E^{*}V. ( T + E ) ∗ ( T + E ) ( C − C ) = ( T + E ) ∗ ( F − E C ) + E ∗ V .
The singular values of ( T + E ) H − 1 (T+E)H^{-1} ( T + E ) H − 1 lie between 1 − ϵ 1-\epsilon 1 − ϵ and 1 + ϵ 1+\epsilon 1 + ϵ . Multiply by H − 1 H^{-1} H − 1 , invert the normalized Gram, and use ( 1 − ϵ ) − 2 ≤ 4 (1-\epsilon)^{-2}\leq4 ( 1 − ϵ ) − 2 ≤ 4 . ■
Lemma A.6 (Block eigenspace perturbation). Let M M M be symmetric and P P P an orthogonal projector of rank m m m with
P M P ⪰ a P , P ⊥ M P ⊥ ⪯ b P ⊥ , a − b = g > 0. PMP \succeq aP,\qquad P^\perp MP^\perp\preceq bP^\perp,\qquad a-b=g>0. P M P ⪰ a P , P ⊥ M P ⊥ ⪯ b P ⊥ , a − b = g > 0. If E E E is symmetric with ∥ P E P ∥ ≤ g / 4 \lVert PEP\rVert\leq g/4 ∥ P E P ∥ ≤ g /4 and ∥ P ⊥ E P ⊥ ∥ ≤ g / 4 \lVert P^\perp EP^\perp\rVert\leq g/4 ∥ P ⊥ E P ⊥ ∥ ≤ g /4 , then
∥ P Top m ( M + E ) − P ∥ ≤ 2 g ( ∥ P ⊥ M P ∥ + ∥ P ⊥ E P ∥ ) . (53) \left\lVert P_{\operatorname{Top}_m}(M+E)-P\right\rVert\leq\frac{2}{g}\left(\lVert P^\perp MP\rVert+\lVert P^\perp EP\rVert\right). \tag*{(53)} P Top m ( M + E ) − P ≤ g 2 ( ∥ P ⊥ M P ∥ + ∥ P ⊥ E P ∥ ) . (53) Proof. Write M ′ = M + E M'=M+E M ′ = M + E and let M 0 = P M ′ P + P ⊥ M ′ P ⊥ M_0=PM'P+P^\perp M'P^\perp M 0 = P M ′ P + P ⊥ M ′ P ⊥ be its block-diagonal part. The diagonal perturbations only reduce the gap: P M 0 P ⪰ ( a − g / 4 ) P PM_0P\succeq(a-g/4)P P M 0 P ⪰ ( a − g /4 ) P and P ⊥ M 0 P ⊥ ⪯ ( b + g / 4 ) P ⊥ P^\perp M_0P^\perp\preceq(b+g/4)P^\perp P ⊥ M 0 P ⊥ ⪯ ( b + g /4 ) P ⊥ . By the min–max principle, the m m m -th largest eigenvalue of M ′ M' M ′ is at least the smallest eigenvalue of its compression to Ran P \operatorname{Ran} P Ran P , hence at least a − g / 4 a-g/4 a − g /4 , while M 0 M_0 M 0 restricted to Ran P ⊥ \operatorname{Ran} P^\perp Ran P ⊥ is at most b + g / 4 b+g/4 b + g /4 . These two spectral sets are separated by g / 2 g/2 g /2 , and the sin Θ theorem [33 , 92 ] gives ∥ P Top m ( M ′ ) − P ∥ ≤ 2 ∥ P ⊥ M ′ P ∥ / g \lVert P_{\operatorname{Top}_m}(M')-P\rVert\leq2\lVert P^\perp M'P\rVert/g ∥ P Top m ( M ′ ) − P ∥ ≤ 2 ∥ P ⊥ M ′ P ∥ / g . Since ∥ P ⊥ M ′ P ∥ ≤ ∥ P ⊥ M P ∥ + ∥ P ⊥ E P ∥ \lVert P^\perp M'P\rVert\leq\lVert P^\perp MP\rVert+\lVert P^\perp EP\rVert ∥ P ⊥ M ′ P ∥ ≤ ∥ P ⊥ M P ∥ + ∥ P ⊥ E P ∥ , the claim follows. Only the cross block rotates the eigenspace.
Applied to − Σ -\Sigma − Σ , Lemma A.6 controls the eigenspace of the m m m smallest eigenvalues of a covariance matrix; this is how it is used in Phase 1.
Lemma A.7 (Pooled slope subspaces). Let T : R m → H T:\mathbb{R}^{m}\to\mathcal{H} T : R m → H and N : R k → H N:\mathbb{R}^{k}\to\mathcal{H} N : R k → H be linear, where H \mathcal{H} H is a Hilbert space and s = σ m ( T ) > 0 s=\sigma_m(T)>0 s = σ m ( T ) > 0 . Put G = T † N G=T^\dagger N G = T † N , V = N − T G V=N-TG V = N − T G , and let P G P_G P G project onto the graph of G ∗ G^* G ∗ . If ∥ V ∥ < s \lVert V\rVert<s ∥ V ∥ < s , then
∥ P Top m ( [ T , N ] ∗ [ T , N ] ) − P G ∥ ≤ ∥ V ∥ 2 min ( 1 , ∥ G ∥ ) s 2 − ∥ V ∥ 2 . (54) \left\lVert P_{\operatorname{Top}_m}\left([T,N]^*[T,N]\right)-P_G\right\rVert\leq\frac{\lVert V\rVert^2\min(1,\lVert G\rVert)}{s^2-\lVert V\rVert^2}. \tag*{(54)} P Top m ( [ T , N ] ∗ [ T , N ] ) − P G ≤ s 2 − ∥ V ∥ 2 ∥ V ∥ 2 min ( 1 , ∥ G ∥) . (54) Proof. Since T ∗ V = 0 T^*V=0 T ∗ V = 0 ,
[ T , N ] ∗ [ T , N ] = [ I , G ] ∗ T ∗ T [ I , G ] + diag ( 0 , V ∗ V ) . [T,N]^*[T,N]=[I,G]^*T^*T[I,G]+\operatorname{diag}(0,V^*V). [ T , N ] ∗ [ T , N ] = [ I , G ] ∗ T ∗ T [ I , G ] + diag ( 0 , V ∗ V ) . The first term has range Ran P G \operatorname{Ran}P_G Ran P G and positive spectrum at least s 2 s^2 s 2 ; min–max bounds the bottom spectrum of the sum by ∥ V ∥ 2 \lVert V\rVert^2 ∥ V ∥ 2 . The cross block relative to P G P_G P G has norm at most ∥ V ∥ 2 min ( 1 , ∥ G ∥ ) \lVert V\rVert^2\min(1,\lVert G\rVert) ∥ V ∥ 2 min ( 1 , ∥ G ∥) . The eigenspace perturbation bound used in Lemma A.6 gives (A.16).
Lemma A.8 (Normal moments under changes of measure). Let μ \mu μ be a centered law on R k \mathbb{R}^{k} R k and s > 0 s>0 s > 0 , with s 2 I ⪯ Cov μ ( Z ) ⪯ C N s 2 I s^2I\preceq\operatorname{Cov}_{\mu}(Z)\preceq C_Ns^2I s 2 I ⪯ Cov μ ( Z ) ⪯ C N s 2 I and Var μ g ≤ C N s 2 Lip ( g ) 2 \operatorname{Var}_{\mu}g\leq C_Ns^2\operatorname{Lip}(g)^2 Var μ g ≤ C N s 2 Lip ( g ) 2 for scalar Lipschitz g g g . For a probability law ν \nu ν with a ≤ d ν / d μ ≤ b a\leq d\nu/d\mu\leq b a ≤ d ν / d μ ≤ b , a > 0 a>0 a > 0 ,
a Cov μ ( Z ) ⪯ Cov ν ( Z ) ⪯ b Cov μ ( Z ) , a\operatorname{Cov}_{\mu}(Z)\preceq\operatorname{Cov}_{\nu}(Z)\preceq b\operatorname{Cov}_{\mu}(Z), a Cov μ ( Z ) ⪯ Cov ν ( Z ) ⪯ b Cov μ ( Z ) , ∥ Reg ν g ∥ ≤ ∥ g − E ν g ∥ L 2 ( ν ) a s ≤ b C N a Lip ( g ) . (55) \lVert\operatorname{Reg}_{\nu}g\rVert\leq\frac{\lVert g-\mathbb{E}_{\nu}g\rVert_{L^2(\nu)}}{\sqrt{a}s}\leq\sqrt{\frac{bC_N}{a}}\operatorname{Lip}(g). \tag*{(55)} ∥ Reg ν g ∥ ≤ a s ∥ g − E ν g ∥ L 2 ( ν ) ≤ a b C N Lip ( g ) . (55)
where Reg ν g = Cov ν ( g , Z ) Cov ν ( Z ) − 1 \operatorname{Reg}_{\nu} g=\operatorname{Cov}_{\nu}(g,Z)\operatorname{Cov}_{\nu}(Z)^{-1} Reg ν g = Cov ν ( g , Z ) Cov ν ( Z ) − 1 . The first regression bound holds for every g ∈ L 2 ( ν ) g\in L^{2}(\nu) g ∈ L 2 ( ν ) . The lower covariance bound only requires ν ≥ a μ \nu\ge a\mu ν ≥ a μ . If ∥ ϵ ∥ ∞ ≤ δ \|\epsilon\|_{\infty}\le\delta ∥ ϵ ∥ ∞ ≤ δ and ∫ Z ϵ d μ = 0 \int Z\epsilon\,d\mu=0 ∫ Z ϵ d μ = 0 , then, for every unit vector e e e ,
∣ ∫ ⟨ e , Z ⟩ g ( Z ) ϵ ( Z ) d μ ( Z ) ∣ ≤ C N δ s 2 Lip ( g ) . (56) \left|\int\langle e,Z\rangle g(Z)\epsilon(Z)\,d\mu(Z)\right|\le C_N\delta s^2\operatorname{Lip}(g). \tag*{(56)} ∫ ⟨ e , Z ⟩ g ( Z ) ϵ ( Z ) d μ ( Z ) ≤ C N δ s 2 Lip ( g ) . (56)
Thus (A2) gives the moment bound (A.18) by taking μ = μ q \mu=\mu_q μ = μ q , s = σ N s=\sigma_N s = σ N , ϵ = 1 \epsilon=1 ϵ = 1 ; for ϵ = d μ ~ q ′ / d μ q − 1 \epsilon=d\widetilde{\mu}_{q'}/d\mu_q-1 ϵ = d μ q ′ / d μ q − 1 , its bound gains the factor C d M ( q , q ′ ) / σ N Cd_{\mathcal{M}}(q,q')/\sigma_N C d M ( q , q ′ ) / σ N . Vector responses are tested in one direction.
Proof The identity Var ν h = inf c E ν ( h − c ) 2 \operatorname{Var}_{\nu}h=\inf_c\mathbb{E}_{\nu}(h-c)^2 Var ν h = inf c E ν ( h − c ) 2 proves both covariance comparisons and Var ν g ≤ b Var μ g \operatorname{Var}_{\nu}g\le b\operatorname{Var}_{\mu}g Var ν g ≤ b Var μ g . Equation (A.11) with P = I P=I P = I then proves the regression bounds in (A.17). For (A.18), subtract E μ g \mathbb{E}_{\mu}g E μ g and apply Cauchy–Schwarz under μ \mu μ , followed by the covariance and Lipschitz variance bounds. Both nearby fiber laws are centered, as required. ■
Lemma A.9 (Convex averages on exceptional events) Let G \mathcal{G} G be a sigma-algebra and y ^ = ∑ i a i ( μ i + ζ i ) \widehat{y}=\sum_i a_i(\mu_i+\zeta_i) y = ∑ i a i ( μ i + ζ i ) , where a i ≥ 0 a_i\ge0 a i ≥ 0 , ∑ i a i = 1 \sum_i a_i=1 ∑ i a i = 1 , and a i , μ i , y a_i,\mu_i,y a i , μ i , y are measurable with respect to G \mathcal{G} G . Suppose ζ i \zeta_i ζ i are conditionally independent and centered with E [ ∥ ζ i ∥ 2 ∣ G ] ≤ v 2 \mathbb{E}[\|\zeta_i\|^2\mid\mathcal{G}]\le v^2 E [ ∥ ζ i ∥ 2 ∣ G ] ≤ v 2 , and ∥ μ i − y ∥ ≤ Δ \|\mu_i-y\|\le\Delta ∥ μ i − y ∥ ≤ Δ . For every E ∈ G E\in\mathcal{G} E ∈ G ,
E [ 1 E ∥ y ^ − y ∥ 2 ] ≤ ( Δ 2 + v 2 ) P ( E ) . (57) \mathbb{E}\left[1_E\|\widehat{y}-y\|^2\right]\le(\Delta^2+v^2)\mathbb{P}(E). \tag*{(57)} E [ 1 E ∥ y − y ∥ 2 ] ≤ ( Δ 2 + v 2 ) P ( E ) . (57) Proof Conditioning on G \mathcal{G} G , convexity bounds the squared bias by Δ 2 \Delta^2 Δ 2 , and independence bounds the noise variance by v 2 ∑ i a i 2 ≤ v 2 v^2\sum_i a_i^2\le v^2 v 2 ∑ i a i 2 ≤ v 2 . Multiply by 1 E 1_E 1 E and integrate. ■
Lemma A.10 (Integrating distance singularities) Under (A1)–(A2), let S ⊂ M S\subset\mathcal{M} S ⊂ M be finite and d S ( q ) = min { 1 , d M ( q , S ) / σ N } d_S(q)=\min\{1,d_{\mathcal{M}}(q,S)/\sigma_N\} d S ( q ) = min { 1 , d M ( q , S ) / σ N } , with d S = 1 d_S=1 d S = 1 if S S S is empty. For α > 0 \alpha>0 α > 0 and 0 < u ≤ 1 0<u\le1 0 < u ≤ 1 ,
E ( d S ( Q ) + u ) − α ≤ C [ 1 + 1 { m = α } ∣ log u ∣ + 1 { m < α } u m − α ] . (58) \mathbb{E}\left(d_S(Q)+u\right)^{-\alpha}\le C\left[1+1_{\{m=\alpha\}}|\log u|+1_{\{m<\alpha\}}u^{m-\alpha}\right]. \tag*{(58)} E ( d S ( Q ) + u ) − α ≤ C [ 1 + 1 { m = α } ∣ log u ∣ + 1 { m < α } u m − α ] . (58) where C C C depends only on m , α , ∣ S ∣ m,\alpha,|S| m , α , ∣ S ∣ , the chart bounds and the bound on sup M π / π − \sup_{\mathcal{M}}\pi/\pi_- sup M π / π − .
Proof A ball in a graph chart of (A1), of radius a fixed small multiple of σ N \sigma_N σ N , gives π − σ N m ≤ C \pi_- \sigma_N^m\le C π − σ N m ≤ C , hence σ N m sup M π ≤ C \sigma_N^m\sup_{\mathcal{M}}\pi\le C σ N m sup M π ≤ C . Chart volume bounds and a union bound over S S S imply P { d S ( Q ) ≤ t } ≤ C t m \mathbb{P}\{d_S(Q)\le t\}\le Ct^m P { d S ( Q ) ≤ t } ≤ C t m for 0 < t < 1 0<t<1 0 < t < 1 . Integration of the distribution function bounds the expectation by 1 + C α ∫ 0 1 t m ( t + u ) − α − 1 d t 1+C\alpha\int_0^1t^m(t+u)^{-\alpha-1}\,dt 1 + C α ∫ 0 1 t m ( t + u ) − α − 1 d t . Splitting at t = u t=u t = u proves (A.20). ■
Phase 1: proof of Proposition 3.1
Condition throughout on the pilot split. Recall the probability lower bound p 0 p_0 p 0 in (19) . The argument first conditions on a retained response window; its moments are controlled by its support, without division by p 0 p_0 p 0 . Only the empirical concentration step uses this lower bound on probability.
Conditional population moments. Fix an anchor X i = q i + z i X_i=q_i+z_i X i = q i + z i , its noise, and a scan radius r r r . If an observation is retained, then
∥ f ( Q ) − f ( q i ) ∥ ≤ δ Y + 2 ( δ Y / 8 ) < 2 δ Y , ∥ Q − q i ∥ ≤ r + 2 R . \lVert f(Q)-f(q_i)\rVert\le\delta_Y+2(\delta_Y/8)<2\delta_Y,\qquad\lVert Q-q_i\rVert\le r+2R. ∥ f ( Q ) − f ( q i )∥ ≤ δ Y + 2 ( δ Y /8 ) < 2 δ Y , ∥ Q − q i ∥ ≤ r + 2 R .
Consequently (20) gives d M ( Q , q i ) ≤ C p k t w Y d_{\mathcal{M}}(Q,q_i)\le C_{\mathrm{pkt}}w_Y d M ( Q , q i ) ≤ C pkt w Y . Since r > 2 R + 2 C p k t w Y r>2R+2C_{\mathrm{pkt}}w_Y r > 2 R + 2 C pkt w Y , the spatial ball contains the entire normal fiber above each such base. Conditioning on membership in the response preimage therefore leaves that fiber law centered, with its original covariance (Lemma A.2 ). For j ∈ S d e s j\in S_{\mathrm{des}} j ∈ S des , let I j I_j I j indicate retention and put p j = E I j p_j=\mathbb{E}I_j p j = E I j . Write μ r e t = ( ∑ j p j ) − 1 ∑ j p j L ( Q j ∣ I j = 1 ) \mu_{\mathrm{ret}}=(\sum_jp_j)^{-1}\sum_jp_j\mathcal{L}(Q_j\mid I_j=1) μ ret = ( ∑ j p j ) − 1 ∑ j p j L ( Q j ∣ I j = 1 ) for the retention-weighted mixture of base laws. It is an arbitrary probability measure supported on B M ( q i , C p k t w Y ) B_{\mathcal{M}}(q_i,C_{\mathrm{pkt}}w_Y) B M ( q i , C pkt w Y ) ; in particular,
∫ d M ( q , q i ) j d μ r e t ( q ) ≤ ( C p k t w Y ) j , j = 1 , 2 , 3. \int d_{\mathcal{M}}(q,q_i)^j\,d\mu_{\mathrm{ret}}(q)\le(C_{\mathrm{pkt}}w_Y)^j,\qquad j=1,2,3. ∫ d M ( q , q i ) j d μ ret ( q ) ≤ ( C pkt w Y ) j , j = 1 , 2 , 3.
These bounds do not involve p 0 p_0 p 0 .
Put P = P q i P=P_{q_i} P = P q i . By fiber centering, the corresponding predictor mixture has mean and covariance
q ˉ = ∫ q d μ r e t ( q ) , Σ = Cov μ r e t ( Q ) + ∫ Σ N ( q ) d μ r e t ( q ) . \bar q=\int q\,d\mu_{\mathrm{ret}}(q),\qquad\Sigma=\operatorname{Cov}_{\mu_{\mathrm{ret}}}(Q)+\int\Sigma_N(q)\,d\mu_{\mathrm{ret}}(q). q ˉ = ∫ q d μ ret ( q ) , Σ = Cov μ ret ( Q ) + ∫ Σ N ( q ) d μ ret ( q ) .
Tangent transport and the normal-chord bound imply
∥ P Σ P ∥ ≤ C w Y 2 ( 1 + σ N 2 / reach M 2 ) , \lVert P\Sigma P\rVert\le Cw_Y^2(1+\sigma_N^2/\operatorname{reach}_{\mathcal{M}}^2), ∥ P Σ P ∥ ≤ C w Y 2 ( 1 + σ N 2 / reach M 2 ) ,
P ⊥ Σ P ⊥ ⪰ σ N 2 ( 1 − C w Y 2 / reach M 2 ) P ⊥ , P^\perp\Sigma P^\perp\succeq\sigma_N^2(1-Cw_Y^2/\operatorname{reach}_{\mathcal{M}}^2)P^\perp, P ⊥ Σ P ⊥ ⪰ σ N 2 ( 1 − C w Y 2 / reach M 2 ) P ⊥ ,
∥ P ⊥ Σ P ∥ ≤ C ( w Y 2 + σ N 2 ) w Y / reach M . \lVert P^\perp\Sigma P\rVert\le C(w_Y^2+\sigma_N^2)w_Y/\operatorname{reach}_{\mathcal{M}}. ∥ P ⊥ Σ P ∥ ≤ C ( w Y 2 + σ N 2 ) w Y / reach M .
For the base term, the tangent chord is O ( w Y ) O(w_Y) O ( w Y ) and its normal component is O ( w Y 2 / reach M ) O(w_Y^2/\operatorname{reach}_{\mathcal{M}}) O ( w Y 2 / reach M ) ; for the fiber term use σ N 2 P q ⊥ ⪯ Σ N ( q ) ⪯ C N σ N 2 P q ⊥ \sigma_N^2P_q^\perp\preceq\Sigma_N(q)\preceq C_N\sigma_N^2P_q^\perp σ N 2 P q ⊥ ⪯ Σ N ( q ) ⪯ C N σ N 2 P q ⊥ . Since w Y ≪ σ N w_Y\ll\sigma_N w Y ≪ σ N , Lemma A.6 , applied to − Σ -\Sigma − Σ , gives angle C w Y / reach M Cw_Y/\operatorname{reach}_{\mathcal{M}} C w Y / reach M for the bottom m m m eigenspace, and ∥ q ˉ − q i ∥ ≤ C w Y \lVert\bar q-q_i\rVert\le Cw_Y ∥ q ˉ − q i ∥ ≤ C w Y . Under (A4) these bounds hold at every scan radius and hence at the radius that maximizes the score.
Sampling and averaging. The event in (19) has probability at least p 0 p_0 p 0 . Its observations satisfy the response test, since δ Y / 4 + 2 δ Y / 8 = δ Y / 2 \delta_Y/4+2\delta_Y/8=\delta_Y/2 δ Y /4 + 2 δ Y /8 = δ Y /2 , and the spatial test: by (20) their bases are within C p k t w Y C_{\mathrm{pkt}}w_Y C pkt w Y of the anchor’s base, so their predictor distance is at most 2 R + C p k t w Y < r 2R+C_{\mathrm{pkt}}w_Y<r 2 R + C pkt w Y < r . Conditional on the anchor, let μ ^ , Σ ^ \hat\mu,\hat\Sigma μ ^ , Σ ^ be the empirical mean and covariance of the retained predictors. We show that, under (A.39), with probability at least 1 − C n − τ − 2 1-Cn^{-\tau-2} 1 − C n − τ − 2 , simultaneously for all anchors and scan radii, at least ∣ S d e s ∣ p 0 / 2 |S_{\mathrm{des}}|p_0/2 ∣ S des ∣ p 0 /2 observations are retained and
∥ μ ^ − q ˉ ∥ ≤ C w Y , ∥ Σ ^ − Σ ∥ ≤ C σ N 2 w Y / reach M . \lVert\hat\mu-\bar q\rVert\le Cw_Y,\qquad\lVert\hat\Sigma-\Sigma\rVert\le C\sigma_N^2w_Y/\operatorname{reach}_{\mathcal{M}}. ∥ μ ^ − q ˉ ∥ ≤ C w Y , ∥ Σ ^ − Σ ∥ ≤ C σ N 2 w Y / reach M .
For a fixed anchor and radius put N d = ∣ S d e s ∣ N_d=|S_{\mathrm{des}}| N d = ∣ S des ∣ , p ˉ = N d − 1 ∑ j p j ≥ p 0 \bar p=N_d^{-1}\sum_jp_j\ge p_0 p ˉ = N d − 1 ∑ j p j ≥ p 0 , r max = max R r_{\max}=\max\mathcal{R} r m a x = max R and t = ( τ + 4 ) log ( ( 2 + n ∣ R ∣ ) / p 0 ) t=(\tau+4)\log((2+n|\mathcal{R}|)/p_0) t = ( τ + 4 ) log (( 2 + n ∣ R ∣ ) / p 0 ) . Conditional on the anchor, the variables I j I_j I j , I j ⟨ v , X j − X i ⟩ I_j\langle v,X_j-X_i\rangle I j ⟨ v , X j − X i ⟩ and I j ⟨ v , X j − X i ⟩ 2 I_j\langle v,X_j-X_i\rangle^2 I j ⟨ v , X j − X i ⟩ 2 are independent across j j j and, for unit v v v , bounded by 1 1 1 , r max r_{\max} r m a x , r max 2 r_{\max}^2 r m a x 2 , respectively, with variance sums at most N d p ˉ N_d\bar p N d p ˉ times the squared bounds. Bernstein’s inequality on a 1 / 4 1/4 1/4 -net of at most 9 D 9^D 9 D unit vectors, normalization by N r = ∑ j I j N_r=\sum_j I_j N r = ∑ j I j , and empirical centering give
N r ≥ N d p ˉ / 2 , ∥ μ ^ − q ˉ ∥ ≤ C r max ϵ , ∥ Σ ^ − Σ ∥ ≤ C r max 2 ϵ . N_r\ge N_d\bar p/2,\qquad\lVert\hat\mu-\bar q\rVert\le Cr_{\max}\epsilon,\qquad\lVert\hat\Sigma-\Sigma\rVert\le Cr_{\max}^2\epsilon. N r ≥ N d p ˉ /2 , ∥ μ ^ − q ˉ ∥ ≤ C r m a x ϵ , ∥ Σ ^ − Σ ∥ ≤ C r m a x 2 ϵ .
with ϵ = ( D + t ) / ( N d p ˉ ) + ( D + t ) / ( N d p ˉ ) \epsilon= \sqrt{(D+t)/(N d \bar{p})} + (D+t)/(N d \bar{p}) ϵ = ( D + t ) / ( N d p ˉ ) + ( D + t ) / ( N d p ˉ ) , outside an event of probability C e − t C e^{-t} C e − t . Condition (A.39) makes the last two bounds at most w Y w_Y w Y and σ N 2 w Y / reach M \sigma_N^2 w_Y/\operatorname{reach}_{\mathcal{M}} σ N 2 w Y / reach M and absorbs the linear term in ϵ \epsilon ϵ . A union bound over all anchors and radii leaves failure probability C n ∣ R ∣ e − t ≤ C n − τ − 2 C n|\mathcal{R}|e^{-t} \le C n^{-\tau-2} C n ∣ R ∣ e − t ≤ C n − τ − 2 . On this event the empirical eigenvalue λ D − m ( Σ ^ ) \lambda_{D-m}(\widehat{\Sigma}) λ D − m ( Σ ) exceeds σ N 2 / 8 \sigma_N^2/8 σ N 2 /8 . Thus every anchor is eligible on this event, and its selected mean and raw projector satisfy the same population bounds.
For an anchor i i i , every member j j j of its projector pool has d M ( Q j , Q i ) ≤ C p k t w Y d_{\mathcal{M}}(Q_j,Q_i) \le C_{\mathrm{pkt}}w_Y d M ( Q j , Q i ) ≤ C pkt w Y . Averaging their raw projectors and extracting the top m m m eigenspace therefore preserves error C w Y / reach M Cw_Y/\operatorname{reach}_{\mathcal{M}} C w Y / reach M relative to P Q i P_{Q_i} P Q i . In an anchor star, the selected means differ by at most C l i n k w Y C_{\mathrm{link}}w_Y C link w Y from the owner’s selected mean. Their base points are then at ambient distance C w Y Cw_Y C w Y from the owner’s base. The local graph chart of (A1) converts this to intrinsic distance C w Y Cw_Y C w Y . The second projector average consequently gives, at the owner’s base q ∗ q_* q ∗ ,
∥ x S − q ∗ ∥ ≤ C w Y , ∥ P S − P q ∗ ∥ ≤ C w Y / reach M . (59) \|x_S-q_*\| \le Cw_Y,\qquad\|P_S-P_{q_*}\| \le Cw_Y/\operatorname{reach}_{\mathcal{M}}. \tag*{(59)} ∥ x S − q ∗ ∥ ≤ C w Y , ∥ P S − P q ∗ ∥ ≤ C w Y / reach M . (59)
A w Y w_Y w Y -net of M \mathcal{M} M has at most C / ( π − w Y m ) C/(\pi-w_Y^m) C / ( π − w Y m ) points. At every net point, the number of anchors satisfying the event in (19) is binomial with success probability at least p 0 p_0 p 0 , hence at least ∣ S a n c ∣ p 0 / 2 |S_{\mathrm{anc}}|p_0/2 ∣ S anc ∣ p 0 /2 outside an event of probability e − ∣ S a n c ∣ p 0 / 8 ≤ e − t e^{-|S_{\mathrm{anc}}|p_0/8} \le e^{-t} e − ∣ S anc ∣ p 0 /8 ≤ e − t . Since p 0 ≤ C π − w Y m p_0 \le C\pi-w_Y^m p 0 ≤ C π − w Y m by the bounded density ratio, the net has at most C / p 0 C/p_0 C / p 0 points, and the union bound leaves a failure probability at most C n − τ − 2 Cn^{-\tau-2} C n − τ − 2 . They have pairwise response differences at most 3 δ Y / 4 3\delta_Y/4 3 δ Y /4 and selected means at distance C w Y Cw_Y C w Y , so they form a retained anchor star when C l i n k C_{\mathrm{link}} C link is chosen large enough. This supplies the nearby anchor stars used below. All constants in the population and averaging bounds are independent of p 0 p_0 p 0 ; decreasing p 0 p_0 p 0 increases only the required sample size.
Assignment to a target. The preceding net argument supplies a retained anchor star within C w Y Cw_Y C w Y of each query base. At x = q + z x=q+z x = q + z the tangential residual of a nearby anchor star is O ( w Y ) O(w_Y) O ( w Y ) and it is within 2 R 2R 2 R . For the assigned anchor star in (A.21), put y = q ∗ + P q ∗ ⊥ ( x − q ∗ ) y=q_*+P_{q_*}^{\perp}(x-q_*) y = q ∗ + P q ∗ ⊥ ( x − q ∗ ) . Its smaller measured tangential residual implies ∥ x − y ∥ ≤ C w Y \|x-y\| \le Cw_Y ∥ x − y ∥ ≤ C w Y and ∥ y − q ∗ ∥ ≤ 2 R + C w Y \|y-q_*\| \le2R+Cw_Y ∥ y − q ∗ ∥ ≤ 2 R + C w Y . The reach normal-chord inequality yields
( 1 − ∥ z ∥ + ∥ y − q ∗ ∥ 2 reach M ) ∥ q − q ∗ ∥ ≤ ∥ x − y ∥ . \left(1-\frac{\|z\|+\|y-q_*\|}{2\operatorname{reach}_{\mathcal{M}}}\right)\|q-q_*\| \le\|x-y\|. ( 1 − 2 reach M ∥ z ∥ + ∥ y − q ∗ ∥ ) ∥ q − q ∗ ∥ ≤ ∥ x − y ∥.
It follows by taking the inner product of x − y x-y x − y with q − q ∗ q-q_* q − q ∗ and bounding each normal component of the chord by ∥ q − q ∗ ∥ 2 / ( 2 reach M ) \|q-q_*\|^2/(2\operatorname{reach}_{\mathcal{M}}) ∥ q − q ∗ ∥ 2 / ( 2 reach M ) . The coefficient is positive for each fixed c < 2 / 3 c<2/3 c < 2/3 under (A5). Thus ∥ q − q ∗ ∥ ≲ w Y \|q-q_*\| \lesssim w_Y ∥ q − q ∗ ∥ ≲ w Y . At this distance, projection of the chord onto M \mathcal{M} M gives an intrinsic path of comparable length by Lemma A.1 . Tangent transport proves the asserted angle and center estimates. For base distance at most r c y l / 4 r_{\mathrm{cyl}}/4 r cyl /4 , normal rotation times R R R plus the base displacement and the O ( w Y ) O(w_Y) O ( w Y ) pilot residual stay within the bounds of the first two conditions in (3.10), and the third condition follows from the first, since the tangential residual of the assigned anchor star is O ( w Y ) O(w_Y) O ( w Y ) and w Y ≤ ε 0 r c y l w_Y \le\varepsilon_0 r_{\mathrm{cyl}} w Y ≤ ε 0 r cyl by (A5); all offsets pass. Conversely, applying the same chord bound to an accepted point (its normal distance from the plane x S + R an P S x_S+R\operatorname{an}P_S x S + R an P S is at most R + O ( r c y l ) R+O(r_{\mathrm{cyl}}) R + O ( r cyl ) ) puts its base within C r c y l Cr_{\mathrm{cyl}} C r cyl of q q q , with C C C depending only on c c c . The choice of γ c y l \gamma_{\mathrm{cyl}} γ cyl places this ball inside one chart. This proves Proposition 3.1.
Phase 2: local affine fits and pooled subspaces
For the proof only, put L = σ N L=\sigma_N L = σ N , u = b / L u=b/L u = b / L , ν = L / R \nu=L/R ν = L / R and H = h / R H=h/R H = h / R . We measure tangent lengths in L L L , normal displacements in R R R , and responses in σ f , + L \sigma_{f,+}L σ f , + L . Normals denoted by x , y , z x,y,z x , y , z in a fiber integral below have length at most one. We first bound the inverse covariance of the fitting coordinates, then compare the regression moments under the actual and frozen fiber laws. Subspace perturbation bounds convert these estimates into tangent-space errors, separately in the three regions described after Proposition 3.2.
Population fits and slope outer products
At a target x = q + R z x x=q+Rz_x x = q + R z x , with normal offset z x z_x z x in units of R R R , let P P P project onto the fitted plane, write this plane as the graph of G ∗ : T q M → N q M G^*:T_qM\to N_qM G ∗ : T q M → N q M , and set Δ = ∥ P − P q ∥ \Delta=\lVert P-P_q\rVert Δ = ∥ P − P q ∥ . In physical coordinates, completing the square in (10) translates the tangent coordinate by Γ ( z − R z x ) \Gamma(z-Rz_x) Γ ( z − R z x ) , with
Γ = ( I + ( b / h ) 2 ( I + G G ∗ ) ) − 1 G . \Gamma=\left(I+(b/h)^2(I+G G^*)\right)^{-1}G. Γ = ( I + ( b / h ) 2 ( I + G G ∗ ) ) − 1 G .
The tangent quadratic form in units b b b differs from I I I by O ( Δ 2 + ( b / h ) 2 ) O(\Delta^2+(b/h)^2) O ( Δ 2 + ( b / h ) 2 ) , and the remaining normal quadratic form in units h h h by O ( Δ 2 ) O(\Delta^2) O ( Δ 2 ) from I I I . When Δ = O ( b / R ) \Delta=O(b/R) Δ = O ( b / R ) and h ≍ R h\asymp R h ≍ R , we have ∥ G − Γ ∥ + ∣ ∥ G ∥ − Δ ∣ = O ( ( b / R ) 3 ) \lVert G-\Gamma\rVert+\lvert\lVert G\rVert-\Delta\rvert=O((b/R)^3) ∥ G − Γ ∥ + ∣∥ G ∥ − Δ ∣ = O (( b / R ) 3 ) . Thus a strict bound on ∥ Γ ∥ \lVert\Gamma\rVert ∥ Γ ∥ gives the corresponding bound on Δ \Delta Δ after decreasing the initial resolution.
Lemma A.11 (Localization by the pilot cylinder). On the event of Proposition 3.1, fix x = q + R z x x=q+Rz_x x = q + R z x and put U q = B M ( q , r c y l / 4 ) U_q=B_M(q,r_{\mathrm{cyl}}/4) U q = B M ( q , r cyl /4 ) . Take h ≥ 3 R h\ge3R h ≥ 3 R , 0 < η ≤ 1 / 16 0<\eta\le1/16 0 < η ≤ 1/16 and b ≤ min { R / 8 , r c y l / 64 } b\le\min\{R/8,r_{\mathrm{cyl}}/64\} b ≤ min { R /8 , r cyl /64 } . Let P ~ b \widetilde P_b P b be a field of rank-m m m orthogonal projectors. For a tube point x ′ = q ′ + z ′ x'=q'+z' x ′ = q ′ + z ′ , let t x ( x ′ ) = d P ~ b ( x ) , b , h ( x ′ − x ) 2 t_x(x')=d_{\widetilde P_b(x),b,h}(x'-x)^2 t x ( x ′ ) = d P b ( x ) , b , h ( x ′ − x ) 2 when the window uses the projector at its center x x x , or t x ( x ′ ) = d P ~ b ( x ′ ) , b , h ( x ′ − x ) 2 t_x(x')=d_{\widetilde P_b(x'),b,h}(x'-x)^2 t x ( x ′ ) = d P b ( x ′ ) , b , h ( x ′ − x ) 2 for prediction. Assume R ∥ P ~ b ( x ) − P q ∥ ≤ η b R\lVert\widetilde P_b(x)-P_q\rVert\le\eta b R ∥ P b ( x ) − P q ∥ ≤ η b in the first case, and R ∥ P ~ b ( x ′ ) − P q ′ ∥ ≤ η b R\lVert\widetilde P_b(x')-P_{q'}\rVert\le\eta b R ∥ P b ( x ′ ) − P q ′ ∥ ≤ η b throughout the cylinder in the second. For the fixed sufficiently small γ c y l \gamma_{\mathrm{cyl}} γ cyl in (A5),
χ 0 ( x , x ′ ) φ ( t x ( x ′ ) ) = 1 { q ′ ∈ U q } φ ( t x ( x ′ ) ) , φ ( t ) = K ( t ) or 1 { t ≤ 1 } . (60) \chi_0(x,x')\varphi(t_x(x'))=\mathbf{1}_{\{q'\in U_q\}}\varphi(t_x(x')),
\qquad\varphi(t)=K(t)\ \text{or}\ \mathbf{1}_{\{t\le1\}}. \tag*{(60)} χ 0 ( x , x ′ ) φ ( t x ( x ′ )) = 1 { q ′ ∈ U q } φ ( t x ( x ′ )) , φ ( t ) = K ( t ) or 1 { t ≤ 1 } . (60) The set { x ′ : χ 0 ( x , x ′ ) = 1 , t x ( x ′ ) ≤ 1 } \{x':\chi_0(x,x')=1,\ t_x(x')\le1\} { x ′ : χ 0 ( x , x ′ ) = 1 , t x ( x ′ ) ≤ 1 } has bases at distance less than 8 b 8b 8 b from q q q , and K ( t x ( x ′ ) ) ≥ 1 / 16 K(t_x(x'))\ge1/16 K ( t x ( x ′ )) ≥ 1/16 on every normal fiber above B M ( q , b / 16 ) B_M(q,b/16) B M ( q , b /16 ) . Hence both its probability and the restricted kernel mass are ≍ π − b \asymp\pi^{-b} ≍ π − b .
Proof. The pilot proof puts the base of every point in the cylinder within C r c y l C r_{\mathrm{cyl}} C r cyl of q q q . Choose γ c y l \gamma_{\mathrm{cyl}} γ cyl so this distance is at most reach M / 5 \operatorname{reach}_M/5 reach M /5 , and write s = d M ( q , q ′ ) s=d_M(q,q') s = d M ( q , q ′ ) . Along the minimizing geodesic, the reach bounds the tangent-chord remainder by s 2 / ( 2 reach M ) s^2/(2\operatorname{reach}_M) s 2 / ( 2 reach M ) and tangent transport by s / reach M s/\operatorname{reach}_M s / reach M . Since ∥ R z x ∥ , ∥ z ′ ∥ ≤ R = c reach M \lVert Rz_x\rVert,\lVert z'\rVert\le R=c\operatorname{reach}_M ∥ R z x ∥ , ∥ z ′ ∥ ≤ R = c reach M , both ∥ P q ( x ′ − x ) ∥ \lVert P_q(x'-x)\rVert ∥ P q ( x ′ − x )∥ and ∥ P q ′ ( x ′ − x ) ∥ \lVert P_{q'}(x'-x)\rVert ∥ P q ′ ( x ′ − x )∥ are at least ( 1 − c − s / ( 2 reach M ) ) s ≥ ( 1 − c ) s / 2 (1-c-s/(2\operatorname{reach}_M))s\ge(1-c)s/2 ( 1 − c − s / ( 2 reach M )) s ≥ ( 1 − c ) s /2 . If t x ( x ′ ) ≤ 1 t_x(x')\le1 t x ( x ′ ) ≤ 1 , the relevant estimated projection has norm at most b b b , whereas ∥ x ′ − x ∥ ≤ s + 2 R \lVert x'-x\rVert\le s+2R ∥ x ′ − x ∥ ≤ s + 2 R . Thus, in either case,
( 1 − c 2 − η b R ) s ≤ ( 1 + 2 η ) b . \left(\frac{1-c}{2}-\frac{\eta b}{R}\right)s\le(1+2\eta)b. ( 2 1 − c − R η b ) s ≤ ( 1 + 2 η ) b .
which gives s < 8 b ≤ r c y l / 8 s<8b\le r_{\mathrm{cyl}}/8 s < 8 b ≤ r cyl /8 . Conversely, every fiber above U q U_q U q lies in the cylinder by Proposition 3.1; this proves (A.22). For s ≤ b / 16 s\le b/16 s ≤ b /16 , tangent transport also bounds either true projection by ( 1 + c ) s (1+c)s ( 1 + c ) s . Consequently
t x ( x ′ ) ≤ ( 1 + c 16 + 2 η + η b 16 R ) 2 + ( 2 R + b / 16 h ) 2 < 3 4 . t_x(x') \leq\left(\frac{1+c}{16}+2\eta+\frac{\eta b}{16R}\right)^2+\left(\frac{2R+b/16}{h}\right)^2<\frac{3}{4}. t x ( x ′ ) ≤ ( 16 1 + c + 2 η + 16 R η b ) 2 + ( h 2 R + b /16 ) 2 < 4 3 .
Then K ( t x ( x ′ ) ) ≥ 1 / 16 K(t_x(x'))\geq1/16 K ( t x ( x ′ )) ≥ 1/16 . Integrating the full normal laws over the inner and outer base balls proves the mass bounds under (A2).
The intercept lets us use z z z in place of z − R z x z-Rz_x z − R z x . On a subset of the fibers retained in Lemma A.11, the kernel is bounded below and the translated tangent coordinate in b b b units ranges over a fixed ball for every z z z . Write X ( w , z ) X(w,z) X ( w , z ) for the tube point with base-chart coordinate w w w and transported normal offset z z z . The base slice map w ↦ P ( X ( w , z ) − x ) w\mapsto P(X(w,z)-x) w ↦ P ( X ( w , z ) − x ) has smallest linear singular value at least ( 1 − c ) 1 − Δ 2 (1-c)\sqrt{1-\Delta^2} ( 1 − c ) 1 − Δ 2 at the origin and derivative variation O ( ∥ w ∥ / reach M ) O(\lVert w\rVert/\operatorname{reach}_M) O (∥ w ∥ / reach M ) . Its restriction to the local support therefore has bounded Jacobian and inverse in m m m dimensions. In the normal frame at the target, (3.2) bounds the sampling law on this subset above and below by fixed multiples of the product of horizontal volume and the target fiber law. Under that product the centered horizontal and normal coordinates are orthogonal; the former have covariance comparable to I I I and the latter to ν 2 I \nu^2 I ν 2 I . The observed normal coordinate differs from z z z by O ( ν 2 u 2 ) O(\nu^2u^2) O ( ν 2 u 2 ) in norm. After division by ν \nu ν , this changes any unit linear test’s standard deviation by O ( ν u 2 ) O(\nu u^2) O ( ν u 2 ) . The variational covariance comparison of Lemma A.8, applied on this subset, gives a uniformly bounded inverse covariance in horizontal units b b b and normal units σ N \sigma_N σ N . The argument applies to every center’s own fit.
Pooling outer products. Write T v , N v T_v,N_v T v , N v for the tangent and normal columns of each stored slope’s transpose, and T , N T,N T , N for their stacks in the Hilbert space of responses over centers, under the conditional pool law or its empirical analog. For G = T † N G=T^\dagger N G = T † N in mixed units, the pooled matrix in common length units is [ T , ν N ] ∗ [ T , ν N ] [T,\nu N]^*[T,\nu N] [ T , ν N ] ∗ [ T , ν N ] and its comparison graph is ν G \nu G ν G . If ∥ N ∥ / σ m ( T ) = O ( u ) \lVert N\rVert/\sigma_m(T)=O(u) ∥ N ∥ / σ m ( T ) = O ( u ) and ∥ G ∥ = O ( u ) \lVert G\rVert=O(u) ∥ G ∥ = O ( u ) , (A.16) gives projector error O ( ν 3 u 3 ) O(\nu^3u^3) O ( ν 3 u 3 ) . Graph coordinates are Lipschitz in the projector near the tangent plane; dividing by ν \nu ν gives O ( ν 2 u 3 ) = O ( u 3 ) O(\nu^2u^3)=O(u^3) O ( ν 2 u 3 ) = O ( u 3 ) in mixed units.
Comparison of the actual and frozen normal equations. The population normal equations use f ( Q ) f(Q) f ( Q ) as the response since E [ ζ ∣ X ] = 0 \mathbb{E}[\zeta\mid X]=0 E [ ζ ∣ X ] = 0 . At each fit center, first apply (A.22) to restrict the base integration to its own U q U_q U q . All actual and frozen measures below use this same base chart and common normal coordinates. At a fit center of offset y y y , freeze the tangent frame and curvature at its base q q q . First transport the law μ q \mu_q μ q over the fit support, keeping all observed coordinates and weights. This gives a reference fit with a fixed normal law in the chart. Write A z = ( I − S q , R z ) − 1 A_z=(I-S_{q,Rz})^{-1} A z = ( I − S q , R z ) − 1 and θ = R Γ / b \theta=R\Gamma/b θ = R Γ/ b . At fixed translated horizontal variable s s s in units b b b , the frozen base displacement in L L L units is w = u A z ( s − θ ( z − y ) ) w=uA_z(s-\theta(z-y)) w = u A z ( s − θ ( z − y )) . In ( s , z ) (s,z) ( s , z ) the frozen section is even in s s s . Its base Jacobian includes det A z \det A_z det A z . Since ∥ A z ∥ ≤ ( 1 − c ) − 1 \lVert A_z\rVert\leq(1-c)^{-1} ∥ A z ∥ ≤ ( 1 − c ) − 1 and the section translation is bounded, its bases also lie within C b Cb C b of the center. The resolution choice in Appendix A.4.3 keeps both supports strictly inside U q U_q U q . We may therefore omit the cylinder within this chart and extend the squared kernel by zero; the extension vanishes near the chart boundary. Take s s s in a fixed bounding ball. If g 0 , g g_0,g g 0 , g denote the weights times normalized base densities in the frozen and curved geometries, both with the reference fiber law, and s ′ , n ′ s',n' s ′ , n ′ the observed sheared horizontal and normal design coordinates, use ∥ a ∥ C z 0 , 1 = sup z ∥ a ( z ) ∥ + Lip z ( a ) \|a\|_{C_z^{0,1}}=\sup_z\|a(z)\|+\operatorname{Lip}_z(a) ∥ a ∥ C z 0 , 1 = sup z ∥ a ( z ) ∥ + Lip z ( a ) . Then
∥ g − g 0 ∥ C z 0 , 1 + ∥ s ′ − s ∥ C z 0 , 1 ≤ C u , ∥ g ∥ C z 0 , 1 + ∥ g 0 ∥ C z 0 , 1 ≤ C , ∥ n ′ − z ∥ C z 0 , 1 ≤ C v 2 u 2 . (61) \begin{aligned}
\|g-g_0\|_{C_z^{0,1}}+\|s'-s\|_{C_z^{0,1}}&\le Cu, &
\|g\|_{C_z^{0,1}}+\|g_0\|_{C_z^{0,1}}&\le C,\\
\|n'-z\|_{C_z^{0,1}}&\le Cv^2u^2. \tag*{(61)}
\end{aligned} ∥ g − g 0 ∥ C z 0 , 1 + ∥ s ′ − s ∥ C z 0 , 1 ∥ n ′ − z ∥ C z 0 , 1 ≤ C u , ≤ C v 2 u 2 . ∥ g ∥ C z 0 , 1 + ∥ g 0 ∥ C z 0 , 1 ≤ C , (61)
Writing g g r g_{\mathrm{gr}} g gr for the local graph map, use the isometric normal frame ( − D g g r ∗ , I ) ( I + D g g r D g g r ∗ ) − 1 / 2 (-Dg_{\mathrm{gr}}^*,I)(I+Dg_{\mathrm{gr}}Dg_{\mathrm{gr}}^*)^{-1/2} ( − D g gr ∗ , I ) ( I + D g gr D g gr ∗ ) − 1/2 . This is the polar isometry in (14) : projection of an origin normal onto the graph normal space has this isometric factor. Taylor expansion using (A1) gives horizontal error O ( ( L / reach M ) ∥ w ∥ 2 ) O((L/\operatorname{reach}_{\mathcal{M}})\|w\|^2) O (( L / reach M ) ∥ w ∥ 2 ) in L L L units and vertical error O ( ( L 2 / ( R reach M ) ) ∥ w ∥ 2 ) O((L^2/(R\operatorname{reach}_{\mathcal{M}}))\|w\|^2) O (( L 2 / ( R reach M )) ∥ w ∥ 2 ) in R R R units, also after a normal derivative since D z w = O ( u ) D_zw=O(u) D z w = O ( u ) . The density comparison uses L ∥ ∇ log π ∥ ≤ C L\|\nabla\log\pi\|\le C L ∥∇ log π ∥ ≤ C , and the squared kernel has Lipschitz derivative. Together with the separation from the chart boundary, these facts give (A.23) without a cylinder or chart boundary term.
Subtract the exactly reproduced ambient affine response at the center; this is the affine-reproduction identity of Lemma A.4 . The first remaining term is u D f ( q ) ( A z − I ) ( s − θ ( z − y ) ) u\,Df(q)(A_z-I)(s-\theta(z-y)) u D f ( q ) ( A z − I ) ( s − θ ( z − y )) in normalized link units. The actual response differs from it by O ( u 2 ) O(u^2) O ( u 2 ) in C z 0 , 1 C_z^{0,1} C z 0 , 1 . The notation L z 2 L_z^2 L z 2 refers to integration against the reference normal law, reweighted by the section’s kernel and Jacobian when present; the density ratio is bounded above and below. The first term has Lipschitz norm O ( u ) O(u) O ( u ) and directional L z 2 L_z^2 L z 2 norm O ( ν u ) O(\nu u) O ( ν u ) : the m m m -dimensional entries of S q , R z S_q,R_z S q , R z have second moments O ( ν 2 ) O(\nu^2) O ( ν 2 ) by (A1)–(A2). Its frozen horizontal coefficient in s s s units and intercept are O ( ν u ) O(\nu u) O ( ν u ) by conditional evenness and directional moments. Its normal coefficient is O ( u ) O(u) O ( u ) because regression of a Lipschitz scalar on z z z costs C Lip C\operatorname{Lip} C Lip under this reweighted law (Lemma A.8 ). After subtracting that frozen fit, the residual has zero mean and zero covariance with the fitting coordinates under the frozen law, L z 2 L_z^2 L z 2 norm O ( ν u ) O(\nu u) O ( ν u ) in each response direction, and C z 0 , 1 C_z^{0,1} C z 0 , 1 norm O ( u ) O(u) O ( u ) .
For both comparisons, let r 0 r_0 r 0 be the response minus the reference affine fit, evaluated in the compared coordinates, and let Δ a s , Δ a n \Delta_a^s,\Delta_a^n Δ a s , Δ a n be the changes from the reference fit’s tangent and normal coefficients. The preceding covariance comparison gives Cov ( s ′ ) ≍ I \operatorname{Cov}(s')\asymp I Cov ( s ′ ) ≍ I and Cov ( n ′ ) Cov ( n ′ , s ′ ) Cov ( s ′ ) − 1 Cov ( s ′ , n ′ ) ≍ ν 2 I \operatorname{Cov}(n')\operatorname{Cov}(n',s')\operatorname{Cov}(s')^{-1}\operatorname{Cov}(s',n')\asymp\nu^2I Cov ( n ′ ) Cov ( n ′ , s ′ ) Cov ( s ′ ) − 1 Cov ( s ′ , n ′ ) ≍ ν 2 I ; conditional evenness, (A.23) and Lemma A.8 give ∥ Cov ( s ′ , n ′ ) ∥ ≤ C v 2 u \|\operatorname{Cov}(s',n')\|\le Cv^2u ∥ Cov ( s ′ , n ′ ) ∥ ≤ C v 2 u . The Schur-complement inverse ([18 Appendix A.5.5] ), applied to the centered normal equations of Lemma A.4 , therefore gives
∥ Δ a s ∥ + ∥ Δ a n ∥ ≤ C ( ∥ Cov ( s ′ , r 0 ) ∥ + ν − 2 ∥ Cov ( n ′ , r 0 ) ∥ ) . (62) \|\Delta_a^s\|+\|\Delta_a^n\|\le C\bigl(\|\operatorname{Cov}(s',r_0)\|+\nu^{-2}\|\operatorname{Cov}(n',r_0)\|\bigr). \tag*{(62)} ∥ Δ a s ∥ + ∥ Δ a n ∥ ≤ C ( ∥ Cov ( s ′ , r 0 ) ∥ + ν − 2 ∥ Cov ( n ′ , r 0 ) ∥ ) . (62)
All bounds hold in operator norm by testing scalar response directions. For the geometry comparison, (A.23) gives Cov ( s ′ , r 0 ) = O ( u 2 ) \operatorname{Cov}(s',r_0)=O(u^2) Cov ( s ′ , r 0 ) = O ( u 2 ) and Cov ( n ′ , r 0 ) = O ( ν 2 u 2 ) \operatorname{Cov}(n',r_0)=O(\nu^2u^2) Cov ( n ′ , r 0 ) = O ( ν 2 u 2 ) . Indeed, the frozen normal equations vanish, so both covariances are integrals of r 0 r_0 r 0 against the changes s ′ g − s g 0 s'g-sg_0 s ′ g − s g 0 and n ′ g − z g 0 n'g-zg_0 n ′ g − z g 0 of the weighted design coordinates. Since ∥ r 0 ∥ C z 0 , 1 = O ( u ) \|r_0\|_{C_z^{0,1}}=O(u) ∥ r 0 ∥ C z 0 , 1 = O ( u ) , (A.23) bounds the first by O ( u 2 ) O(u^2) O ( u 2 ) and the part of the second that contains n ′ − z n'-z n ′ − z by O ( ν 2 u 3 ) O(\nu^2u^3) O ( ν 2 u 3 ) . For the remaining part, the function ∫ r 0 ( g − g 0 ) d s \int r_0\,(g-g_0)\,ds ∫ r 0 ( g − g 0 ) d s of z z z has Lipschitz seminorm O ( u 2 ) O(u^2) O ( u 2 ) , and Lemma A.8 with Cov ( z ) ⪯ C ν 2 I \operatorname{Cov}(z)\preceq C\nu^2I Cov ( z ) ⪯ C ν 2 I bounds its covariance with z z z by C ν 2 u 2 C\nu^2u^2 C ν 2 u 2 . The normal mean is O ( ν 2 ) O(\nu^2) O ( ν 2 ) , and replacing z z z by n ′ = z + O C z 0 , 1 ( ν 2 u 2 ) n'=z+O_{C_z^{0,1}}(\nu^2u^2) n ′ = z + O C z 0 , 1 ( ν 2 u 2 ) preserves this order after centering. Thus (A.24) bounds both coefficient changes by O ( u 2 ) O(u^2) O ( u 2 ) .
To restore the varying fiber law, work at fixed base q ′ q' q ′ and put ϵ = d μ ~ q ′ / d μ q ′ − 1 \epsilon=d\widetilde{\mu}_{q'}/d\mu_{q'}-1 ϵ = d μ q ′ / d μ q ′ − 1 . By (14) and centering of both laws, ∥ ϵ ∥ ∞ ≤ C u \|\epsilon\|_\infty\le Cu ∥ ϵ ∥ ∞ ≤ C u and ∫ ϵ d μ q ′ = ∫ z ϵ d μ q ′ = 0 \int\epsilon\,d\mu_{q'}=\int z\epsilon\,d\mu_{q'}=0 ∫ ϵ d μ q ′ = ∫ z ϵ d μ q ′ = 0 . At fixed base, the weighted residual mean and its uncentered cross moment with the tangent coordinates change by O ( u ∥ r 0 ∥ C z 0 , 1 ) O(u\lVert r_0\rVert_{C_z^{0,1}}) O ( u ∥ r 0 ∥ C z 0 , 1 ) , whereas Lemma A.8, applied to K r 0 Kr_0 K r 0 , bounds the change of each normal cross moment by C ν 2 u ∥ r 0 ∥ C z 0 , 1 C\nu^2u\lVert r_0\rVert_{C_z^{0,1}} C ν 2 u ∥ r 0 ∥ C z 0 , 1 . The base restriction to U q U_q U q is independent of the normal variable, and the local kernel is separated from its boundary. Thus the integrands in these weighted moments have the normal Lipschitz bounds used here after zero extension, without differentiating a cylinder indicator. Integration over the base ball, normalization by the kernel mass ≍ π − b m \asymp\pi^{-b}m ≍ π − b m , centering with normal mean O ( ν 2 ) O(\nu^2) O ( ν 2 ) , and replacing z z z by n ′ n' n ′ preserve these orders. Hence (A.24) bounds both coefficient changes by O ( u ∥ r 0 ∥ C z 0 , 1 ) O(u\lVert r_0\rVert_{C_z^{0,1}}) O ( u ∥ r 0 ∥ C z 0 , 1 ) . The preceding residual bounds give ∥ r 0 ∥ C z 0 , 1 = O ( u ) \lVert r_0\rVert_{C_z^{0,1}}=O(u) ∥ r 0 ∥ C z 0 , 1 = O ( u ) . Dividing the horizontal coefficient by u u u and undoing the bounded shear give tangent- and normal-slope errors O ( u ) O(u) O ( u ) and O ( u 2 ) O(u^2) O ( u 2 ) , respectively.
The actual response in a singular chart. Fix an isolated singularity q 0 q_0 q 0 . For a target within A b Ab A b of it, where A A A is fixed, Lemma A.11, applied to the target pool and then to each center’s own fit, puts all contributing bases within C A b CAb C A b of q 0 q_0 q 0 . We express these localized integrals in the common chart at q 0 q_0 q 0 ; the fixed chart boundaries do not cut their supports. In a graph chart centered there write the base as ( L v , g ( L v ) ) (Lv,g(Lv)) ( Lv , g ( Lv )) , Υ ( v ) = D g ( L v ) \Upsilon(v)=Dg(Lv) Υ ( v ) = D g ( Lv ) , Ξ ( v ) = ( I + Υ ( v ) Υ ( v ) ∗ ) − 1 / 2 \Xi(v)=(I+\Upsilon(v)\Upsilon(v)^*)^{-1/2} Ξ ( v ) = ( I + Υ ( v ) Υ ( v ) ∗ ) − 1/2 . Use the orthonormal normal frame ( − Υ ∗ , I ) Ξ (-\Upsilon^*,I)\Xi ( − Υ ∗ , I ) Ξ . In these fixed tangent and normal coordinates the observed predictor is exactly
t = v + ℓ ( v , z ) , ℓ ( v , z ) = − ( R / L ) Υ ( v ) ∗ Ξ ( v ) z , t=v+\ell(v,z),\qquad\ell(v,z)=-(R/L)\Upsilon(v)^*\Xi(v)z, t = v + ℓ ( v , z ) , ℓ ( v , z ) = − ( R / L ) Υ ( v ) ∗ Ξ ( v ) z ,
z ~ = z + g ( L v ) / R + ( Ξ ( v ) − I ) z , ∣ z ∣ ≤ 1. \widetilde z=z+g(Lv)/R+(\Xi(v)-I)z,\qquad|z|\le1. z = z + g ( Lv ) / R + ( Ξ ( v ) − I ) z , ∣ z ∣ ≤ 1.
Use the transported law at q 0 q_0 q 0 as reference throughout this chart; the actual fiber weights differ from it by O A ( u ) O_A(u) O A ( u ) . Set J = D f ( q 0 ) / σ f , + J=Df(q_0)/\sigma_{f,+} J = D f ( q 0 ) / σ f , + , the normalized link differential at the singularity. For ∣ v ∣ ≤ C A u |v|\le C_Au ∣ v ∣ ≤ C A u ,
∥ ℓ ∥ + ∥ D z ℓ ∥ ≤ C ε s i n g ∣ v ∣ , ∥ D v ℓ ∥ ≤ C ε s i n g , \lVert\ell\rVert+\lVert D_z\ell\rVert\le C\varepsilon_{\mathrm{sing}}|v|,\qquad\lVert D_v\ell\rVert\le C\varepsilon_{\mathrm{sing}}, ∥ ℓ ∥ + ∥ D z ℓ ∥ ≤ C ε sing ∣ v ∣ , ∥ D v ℓ ∥ ≤ C ε sing ,
∥ J ℓ ∥ + ∥ J D z ℓ ∥ ≤ C ( ε s i n g + ∣ v ∣ ) ∣ v ∣ 2 , ∥ J D v ℓ ∥ ≤ C ( ε s i n g + ∣ v ∣ ) ∣ v ∣ . \lVert J\ell\rVert+\lVert JD_z\ell\rVert\le C(\varepsilon_{\mathrm{sing}}+|v|)|v|^2,\qquad\lVert JD_v\ell\rVert\le C(\varepsilon_{\mathrm{sing}}+|v|)|v|. ∥ J ℓ ∥ + ∥ J D z ℓ ∥ ≤ C ( ε sing + ∣ v ∣ ) ∣ v ∣ 2 , ∥ J D v ℓ ∥ ≤ C ( ε sing + ∣ v ∣ ) ∣ v ∣.
Here and below z z z -derivatives are in the unit normal ball. To verify the second line, the difference of differentials in (18) multiplied on the right by I − S q , R z I-S_{q,R}z I − S q , R z is D f ( q ) S q , R z Df(q)S_{q,R}z D f ( q ) S q , R z . Thus ∥ D f ( q ) S q , R z ∥ ≤ C ε s i n g ∣ v ∣ \lVert Df(q)S_{q,R}z\rVert\le C\varepsilon_{\mathrm{sing}}|v| ∥ D f ( q ) S q , R z ∥ ≤ C ε sing ∣ v ∣ , since the weakest singular value vanishes at 0 and D f Df D f is Lipschitz. At regular points (18) also implies ∥ S q , R z ∥ ≤ ( 1 + c ) ε s i n g \lVert S_{q,R}z\rVert\le(1+c)\varepsilon_{\mathrm{sing}} ∥ S q , R z ∥ ≤ ( 1 + c ) ε sing , and continuity extends this to the center. Parallel transporting to the origin now gives ∥ J S q , R z ∥ ≤ C ε s i n g ∣ v ∣ \lVert JS_{q,R}z\rVert\le C\varepsilon_{\mathrm{sing}}|v| ∥ J S q , R z ∥ ≤ C ε sing ∣ v ∣ . In graph coordinates the shape matrix is ( I + Υ ∗ Υ ) − 1 R ⟨ D 2 g , Ξ z ⟩ (I+\Upsilon^*\Upsilon)^{-1}R\langle D^2g,\Xi z\rangle ( I + Υ ∗ Υ ) − 1 R ⟨ D 2 g , Ξ z ⟩ ; this follows by pairing the second graph derivative with ( − Υ ∗ , I ) Ξ z (-\Upsilon^*,I)\Xi z ( − Υ ∗ , I ) Ξ z . Metric and tangent-frame corrections are O ( ( L ∣ v ∣ / reach M ) 2 ) O((L|v|/\operatorname{reach}_M)^2) O (( L ∣ v ∣/ reach M ) 2 ) , and differentiating the Ξ \Xi Ξ in Υ ∗ Ξ \Upsilon^*\Xi Υ ∗ Ξ costs O ( ∣ v ∣ 2 ) O(|v|^2) O ( ∣ v ∣ 2 ) after scaling. This proves the derivative bounds. Integrating radially gives the bounds for ℓ \ell ℓ ; it is linear in z z z . Also z ~ − z \widetilde z-z z − z , its z z z -derivative and ∣ v ∣ D v ( z ~ − z ) |v|D_v(\widetilde z-z) ∣ v ∣ D v ( z − z ) are O ( ( L / R ) 2 ∣ v ∣ 2 ) O((L/R)^2|v|^2) O (( L / R ) 2 ∣ v ∣ 2 ) by (A1).
Subtract the ambient affine response f ( q 0 ) + σ f , + L t f(q_0)+\sigma_{f,+}L t f ( q 0 ) + σ f , + L t , which every local fit reproduces exactly by Lemma A.4. Define the symmetric bilinear map B [ v , w ] = ( L / σ f , + ) ( ∇ v D f ) ( q 0 ) w B[v,w]=(L/\sigma_{f,+})(\nabla_vDf)(q_0)w B [ v , w ] = ( L / σ f , + ) ( ∇ v D f ) ( q 0 ) w , where ∇ D f \nabla Df ∇ D f is the covariant derivative in parallel tangent frames, and use ∥ B ∥ = sup ∥ v ∥ , ∥ w ∥ ≤ 1 ∥ B [ v , w ] ∥ \lVert B\rVert=\sup_{\lVert v\rVert,\lVert w\rVert\le1}\lVert B[v,w]\rVert ∥ B ∥ = sup ∥ v ∥ , ∥ w ∥ ≤ 1 ∥ B [ v , w ]∥ . The remaining response in observed horizontal coordinate t t t is 1 2 B [ t , t ] + e ( t , z ) \frac12B[t,t]+e(t,z) 2 1 B [ t , t ] + e ( t , z ) , where
∥ e ∥ + u ∥ D t e ∥ + ∥ D z e ∥ ≤ C A u 2 { ε s i n g + u + ω ( C A u ) } . (63) \lVert e\rVert+u\lVert D_te\rVert+\lVert D_ze\rVert\le C_Au^2\{\varepsilon_{\mathrm{sing}}+u+\omega(C_Au)\}. \tag*{(63)} ∥ e ∥ + u ∥ D t e ∥ + ∥ D z e ∥ ≤ C A u 2 { ε sing + u + ω ( C A u )} . (63)
Indeed v ↦ t v \mapsto t v ↦ t is invertible with D t v = I + O ( ε s i n g ) D_t v = I + O(\varepsilon_{\mathrm{sing}}) D t v = I + O ( ε sing ) and D z v = O ( ε s i n g ∣ v ∣ ) D_z v = O(\varepsilon_{\mathrm{sing}}|v|) D z v = O ( ε sing ∣ v ∣ ) . Subtract the two quadratics at v v v and t t t , apply the second line above to the affine part, and apply (A3) to the intrinsic Taylor remainder. The input plane at each center, expressed in the fixed chart plane rather than its own tangent, changes its shear in units b / R b/R b / R by O A ( ε s i n g ) O_A(\varepsilon_{\mathrm{sing}}) O A ( ε sing ) . Changing the output back costs the same amount; its component tested by J J J costs only O A ( ( ε s i n g + u ) ( L / R ) u 2 ) O_A((\varepsilon_{\mathrm{sing}}+u)(L/R)u^2) O A (( ε sing + u ) ( L / R ) u 2 ) . Thus the three contributions are the frame and chart comparison O A ( ( ε s i n g + u ) u 2 ) O_A((\varepsilon_{\mathrm{sing}}+u)u^2) O A (( ε sing + u ) u 2 ) from J ℓ J\ell J ℓ , the change of quadratic coordinates − B [ ℓ , t ] + 1 2 B [ ℓ , ℓ ] = O A ( ε s i n g u 2 ) -B[\ell,t]+\frac{1}{2}B[\ell,\ell]=O_A(\varepsilon_{\mathrm{sing}}u^2) − B [ ℓ , t ] + 2 1 B [ ℓ , ℓ ] = O A ( ε sing u 2 ) , and the link Taylor remainder O A ( u 2 ω ( C A u ) ) O_A(u^2\omega(C_Au)) O A ( u 2 ω ( C A u )) . Their t t t - and z z z -derivatives have the same scaled order, which gives the full C z 0 , 1 C_z^{0,1} C z 0 , 1 bound in (A.25).
The quadratic response over the full tangent ball. Now measure the base positions in units of b b b . For the quadratic response in (A.25) write x , y , z x,y,z x , y , z for the unit-ball normal offsets of the target, pool center and fitting observation. In the space of square-integrable responses over pool centers, let Π \Pi Π project onto the constant functions with values in Ran J \operatorname{Ran} J Ran J . This projection leaves response variation within Ran J \operatorname{Ran} J Ran J . The input shear is θ = R Γ / b \theta=R\Gamma/b θ = R Γ/ b with Γ \Gamma Γ in physical units; θ x \theta_x θ x and θ c t r \theta_{\mathrm{ctr}} θ ctr refer to the full target and fit center. Let a a a be the target base’s displacement from q 0 q_0 q 0 in b b b units, put v y ( z ) = 1 − ∥ z − y ∥ 2 / H 2 v_y(z)=1-\|z-y\|^2/H^2 v y ( z ) = 1 − ∥ z − y ∥ 2 / H 2 , and let ξ \xi ξ denote the centered horizontal displacement in the hard outer window. In the leading horizontal section the outer base point has coordinates
w = a + θ x ( x − y ) + ξ , E [ ξ ∣ y ] = 0 , Cov ( ξ ∣ y ) = v x ( y ) m + 2 I . w=a+\theta_x(x-y)+\xi,\qquad\mathbb{E}[\xi\mid y]=0,\qquad\operatorname{Cov}(\xi\mid y)=\frac{v_x(y)}{m+2}I. w = a + θ x ( x − y ) + ξ , E [ ξ ∣ y ] = 0 , Cov ( ξ ∣ y ) = m + 2 v x ( y ) I .
Below Z Z Z is the unit-ball normal variable. Subscripts F , y F,y F , y and T , y T,y T , y denote, respectively, its reference fit law (the law at q 0 q_0 q 0 tilted by v y m / 2 + 2 v_y^{m/2+2} v y m /2 + 2 ) and law used for tangent second moments (reweighted by v y m / 2 + 3 v_y^{m/2+3} v y m /2 + 3 ), each normalized to mass one. Expectations and covariances carry the same subscripts. Write Reg F , y ϕ = Cov F , y ( ϕ , Z ) Cov F , y ( Z ) − 1 \operatorname{Reg}_{F,y}\phi=\operatorname{Cov}_{F,y}(\phi,Z)\operatorname{Cov}_{F,y}(Z)^{-1} Reg F , y ϕ = Cov F , y ( ϕ , Z ) Cov F , y ( Z ) − 1 for the regression coefficient with intercept, and let ( e j ) (e_j) ( e j ) be an orthonormal tangent basis. The resulting tangent coefficients restricted to ker J \ker J ker J , divided by u u u , and normal coefficients, divided by u 2 u^2 u 2 , in mixed units, are
U v = B [ w + θ c t r ( y − E T , y Z ) , v ] , v ∈ ker J , (64) U v=B[w+\theta_{\mathrm{ctr}}(y-\mathbb{E}_{T,y}Z),v],\qquad v\in\ker J, \tag*{(64)} U v = B [ w + θ ctr ( y − E T , y Z ) , v ] , v ∈ ker J , (64)
V = ∑ j B [ e j , e j ] 2 ( m + 6 ) H 2 ( 2 y − Reg F , y ∥ Z ∥ 2 ) T + 1 2 Reg F , y B [ θ c t r Z , θ c t r Z ] − B [ θ c t r E T , y Z , θ c t r ⋅ ] . (65) V=\frac{\sum_j B[e_j,e_j]}{2(m+6)H^2}\left(2y-\operatorname{Reg}_{F,y}\|Z\|^2\right)^{\mathsf T}
+\frac{1}{2}\operatorname{Reg}_{F,y}B[\theta_{\mathrm{ctr}}Z,\theta_{\mathrm{ctr}}Z]
-B[\theta_{\mathrm{ctr}}\mathbb{E}_{T,y}Z,\theta_{\mathrm{ctr}}\cdot]. \tag*{(65)} V = 2 ( m + 6 ) H 2 ∑ j B [ e j , e j ] ( 2 y − Reg F , y ∥ Z ∥ 2 ) T + 2 1 Reg F , y B [ θ ctr Z , θ ctr Z ] − B [ θ ctr E T , y Z , θ ctr ⋅ ] . (65)
A scalar regression row is identified with its vector. Translate the inner tangent coordinate by θ c t r ( z − y ) \theta_{\mathrm{ctr}}(z-y) θ ctr ( z − y ) . Its conditional law is even, with covariance v y ( z ) I / ( m + 6 ) v_y(z)I/(m+6) v y ( z ) I / ( m + 6 ) for kernel ( 1 − s ) + 2 (1-s)^2_+ ( 1 − s ) + 2 . Apply (A.12) to the quadratic: expanding ∥ z − y ∥ 2 \|z-y\|^2 ∥ z − y ∥ 2 and canceling the terms linear in θ c t r y \theta_{\mathrm{ctr}}y θ ctr y gives (A.26)–(A.27), separately at each center; the incoming field may vary arbitrarily between centers.
Lemma A.12 (Pooled slopes for a quadratic response) Under (A2), let g B g_B g B be the least value of ( ∑ j = 1 m ∣ B [ e j , v ] ∣ 2 ) 1 / 2 \left(\sum_{j=1}^{m} \lvert B[e_j,v]\rvert^2\right)^{1/2} ( ∑ j = 1 m ∣ B [ e j , v ] ∣ 2 ) 1/2 over unit v ∈ ker J v \in\ker J v ∈ ker J , where ( e j ) (e_j) ( e j ) is an orthonormal tangent basis. If g B > 0 g_B > 0 g B > 0 , one can choose 0 < η ≤ 1 / 16 0 < \eta\le1/16 0 < η ≤ 1/16 and a finite H 0 ≥ 3 H_0 \ge3 H 0 ≥ 3 , depending only on m m m , C N C_N C N , a ρ a_\rho a ρ and ∥ B ∥ / g B \lVert B\rVert/g_B ∥ B ∥ / g B , such that for every H ≥ H 0 H \ge H_0 H ≥ H 0 , every target and every incoming field with ∥ θ ∥ ≤ η \lVert\theta\rVert\le\eta ∥ θ ∥ ≤ η .
∥ [ ( I − Π ) V ] † ( I − Π ) V ∥ ≤ a ρ η / 4. \left\lVert[(I-\Pi)V]^\dagger(I-\Pi)V\right\rVert\le a_\rho\eta/4. [( I − Π ) V ] † ( I − Π ) V ≤ a ρ η /4.
Proof For H ≥ 3 H \ge3 H ≥ 3 , the two reweighted normal laws just defined have density ratios, relative to the original fiber law, bounded above and below in terms of m m m . Lemma A.8 , with s = ν = σ N / R s=\nu=\sigma_N/R s = ν = σ N / R , gives ∥ Reg F , y ϕ ∥ ≤ C Lip ϕ \lVert\operatorname{Reg}_{F,y}\phi\rVert\le C\operatorname{Lip}\phi ∥ Reg F , y ϕ ∥ ≤ C Lip ϕ at every actual center. Equation (A.27) therefore gives ∥ V ∥ ≤ C ∥ B ∥ ( H − 2 + η 2 ) \lVert V\rVert\le C\lVert B\rVert(H^{-2}+\eta^2) ∥ V ∥ ≤ C ∥ B ∥ ( H − 2 + η 2 ) , with C C C depending only on m m m and C N C_N C N . For the term B [ w , v ] B[w,v] B [ w , v ] , subtracting any constant cannot remove its conditional fluctuation B [ ξ , v ] B[\xi,v] B [ ξ , v ] . Thus
σ min ( ( I − Π ) U ) ≥ 1 − 4 / H 2 m + 2 g B − 3 ∥ B ∥ η . \sigma_{\min}((I-\Pi)U) \ge\frac{\sqrt{1-4/H^2}}{m+2}g_B-3\lVert B\rVert\eta. σ m i n (( I − Π ) U ) ≥ m + 2 1 − 4/ H 2 g B − 3 ∥ B ∥ η .
The triangle inequality in L 2 L^2 L 2 permits the incoming field to depend on ξ \xi ξ . First choose η ≤ 1 / 16 \eta\le1/16 η ≤ 1/16 so that the subtracted term is less than half the positive lower bound at H = 3 H=3 H = 3 , and the η 2 \eta^2 η 2 contribution after inversion is at most a ρ η / 8 a_\rho\eta/8 a ρ η /8 . Then choose H 0 H_0 H 0 so the H − 2 H^{-2} H − 2 contribution costs at most another a ρ η / 8 a_\rho\eta/8 a ρ η /8 . The constants used in making these two choices are uniform for H ≥ 3 H \ge3 H ≥ 3 . ■
The signal g B g_B g B is supplied by (3.5). For a unit v ∈ ker J v \in\ker J v ∈ ker J , differentiate along the geodesic from q 0 q_0 q 0 in any unit direction e e e , transporting v v v in parallel. Dividing the lower bound for ∥ D f v ∥ \lVert Df\,v\rVert ∥ D f v ∥ by the geodesic distance and passing to the center gives ∥ B [ e , v ] ∥ ≥ c o p \lVert B[e,v]\rVert\ge c_{\mathrm{op}} ∥ B [ e , v ]∥ ≥ c op in normalized units. Thus g B ≥ m c o p g_B \ge\sqrt{m}\,c_{\mathrm{op}} g B ≥ m c op ; the conditional horizontal variance in Lemma A.12 supplies this signal even after projecting out the constant functions with values in Ran J \operatorname{Ran}J Ran J .
In the translated tangent section the displacement of t t t per unit z z z is O A ( u ) O_A(u) O A ( u ) . Thus the integrated remainder (A.25) costs its displayed order also in the reference normal slope. The base density and horizontal Jacobian in this section differ from their constant values relatively by O A ( ε s i n g + u ) O_A(\varepsilon_{\mathrm{sing}}+u) O A ( ε sing + u ) , including their normal Lipschitz errors: differentiate det ( I + D v ℓ ) − 1 \det(I+D_v\ell)^{-1} det ( I + D v ℓ ) − 1 , using D z D v ℓ = O ( ε s i n g ) D_zD_v\ell=O(\varepsilon_{\mathrm{sing}}) D z D v ℓ = O ( ε sing ) , D ℓ 2 = O ( 1 ) D_\ell^2=O(1) D ℓ 2 = O ( 1 ) by (A1), and D z v = O A ( u ) D_zv=O_A(u) D z v = O A ( u ) . The fit kernel retains a uniform positive lower bound on the subset used in the covariance comparison. Replacing the normal predictor z z z by z ~ \widetilde z z adds at most O A ( ( b / L ) 2 ) O_A((b/L)^2) O A (( b / L ) 2 ) to the scaled normal error: its coordinate discrepancy is O A ( ( L / R ) 2 u 2 ) O_A((L/R)^2u^2) O A (( L / R ) 2 u 2 ) , the residual is O A ( u 2 ) O_A(u^2) O A ( u 2 ) , and the normal covariance is of order ( L / R ) 2 (L/R)^2 ( L / R ) 2 . After removing the common affine response, the remaining response and the reference fit have size O A ( u 2 ) O_A(u^2) O A ( u 2 ) on this chart: the horizontal coefficient in b b b units and the normal coefficient in R R R units are both O A ( u 2 ) O_A(u^2) O A ( u 2 ) by (A.25)–(A.27) and the preceding bound for the normal regression coefficient. At fixed base the response is constant in z z z before subtraction of the affine part, whose remaining normal variation is bounded by the estimate for J D z ℓ JD_z\ell J D z ℓ . The residual therefore has C z 0 , 1 C_z^{0,1} C z 0 , 1 norm O A ( u 2 ) O_A(u^2) O A ( u 2 ) in the preceding comparison of fiber laws. The changing law thus perturbs the tangent slope by O A ( u 2 ) O_A(u^2) O A ( u 2 ) and the normal slope by O A ( u 3 ) O_A(u^3) O A ( u 3 ) . For the hard outer section, only the ellipsoidal boundary moves: both sections stay inside the fixed base chart by the preceding localization. The relative boundary-shell mass is O A ( ε s i n g + u ) O_A(\varepsilon_{\mathrm{sing}}+u) O A ( ε sing + u ) and the fiber density changes by O A ( u ) O_A(u) O A ( u ) on the intersection. Lemma A.3 therefore compares bounded moments at the same centers, without transporting the incoming field.
The section metric correction O ( ( b / h ) 2 + Δ 2 ) O((b/h)^2+\Delta^2) O (( b / h ) 2 + Δ 2 ) is absorbed in the same comparison, since the normal section radius stays positive. Put χ = ε s i n g + u + ω ( C A u ) \chi=\varepsilon_{\mathrm{sing}}+u+\omega(C_Au) χ = ε sing + u + ω ( C A u ) . Divide tangent coefficients in ker J \ker J ker J by u u u and normal coefficients by u 2 u^2 u 2 . For the complementary tangent columns, T + = J ∣ ( ker J ) ⊥ + O A ( u ) T_+=J|_{(\ker J)^\perp}+O_A(u) T + = J ∣ ( k e r J ) ⊥ + O A ( u ) , where the restriction of J J J to ( ker J ) ⊥ (\ker J)^\perp ( ker J ) ⊥ is constant over centers and has nonzero singular values bounded below by (A3). Thus ∥ P Ran T + − Π ∥ ≤ C A u \|P_{\operatorname{Ran}T_+}-\Pi\|\leq C_Au ∥ P Ran T + − Π∥ ≤ C A u in each corresponding pool space; both projections vanish if J = 0 J=0 J = 0 .
Set Q 0 = U ∗ ( I − Π ) U Q_0=U^*(I-\Pi)U Q 0 = U ∗ ( I − Π ) U and B 0 = U ∗ ( I − Π ) V B_0=U^*(I-\Pi)V B 0 = U ∗ ( I − Π ) V . Let Q , B a c t Q,B_{\mathrm{act}} Q , B act be the actual rescaled Gram and cross moment after projecting onto ( Ran T + ) ⊥ (\operatorname{Ran}T_+)^\perp ( Ran T + ) ⊥ . The rescaled columns are bounded, with pointwise errors O A ( χ ) O_A(\chi) O A ( χ ) . Lemma A.3 and the preceding pool-law comparison give ∥ Q − Q 0 ∥ + ∥ B a c t − B 0 ∥ ≤ C A χ \|Q-Q_0\|+\|B_{\mathrm{act}}-B_0\|\leq C_A\chi ∥ Q − Q 0 ∥ + ∥ B act − B 0 ∥ ≤ C A χ . Indeed, both are normalized averages of products of bounded columns. Replacing the actual columns by the model ones changes each product by O A ( χ ) O_A(\chi) O A ( χ ) ; replacing the projection onto ( Ran T + ) ⊥ (\operatorname{Ran}T_+)^\perp ( Ran T + ) ⊥ by I − Π I-\Pi I − Π changes it by O A ( u ) O_A(u) O A ( u ) ; and the change of pool law, of relative mass O A ( ε s i n g + u ) O_A(\varepsilon_{\mathrm{sing}}+u) O A ( ε sing + u ) , changes the average by the same order, by (A.8). Lemma A.12 gives a fixed positive lower bound for the smallest eigenvalue of Q 0 Q_0 Q 0 . Lemma A.5 therefore bounds the change of Q − 1 B a c t Q^{-1}B_{\mathrm{act}} Q − 1 B act by C A χ C_A\chi C A χ when C A χ C_A\chi C A χ is at most half that lower bound. The complementary graph block is O A ( u 2 ) O_A(u^2) O A ( u 2 ) : its right-hand side is O A ( u 2 ) O_A(u^2) O A ( u 2 ) , as is the product of the tangent columns restricted to ker J \ker J ker J , of size O A ( u ) O_A(u) O A ( u ) , and the corresponding graph coefficients, also of size O A ( u ) O_A(u) O A ( u ) , while T + T_+ T + is uniformly invertible on its range. Consequently ∥ N ∥ = O A ( u 2 ) \|N\|=O_A(u^2) ∥ N ∥ = O A ( u 2 ) and σ m ( T ) ≳ A u \sigma_m(T)\gtrsim_A u σ m ( T ) ≳ A u ; Lemma A.7 adds only O A ( u 3 ) O_A(u^3) O A ( u 3 ) in mixed units.
Choose H 0 = C / η H_0=C/\sqrt{\eta} H 0 = C / η . Taking C C C large and then η \eta η small makes H − 2 / η H^{-2}/\eta H − 2 / η small for every H ≥ H 0 H\geq H_0 H ≥ H 0 , as the proof requires. At fixed A A A , the scaled normal graph differs from that in Lemma A.12 by O A ( χ ) O_A(\chi) O A ( χ ) plus sampling error. The rotations back to individual frames are included; no continuity of the estimated projector field is used.
The annulus between the singular and regular regions. Write d = d ( q ) d=d(q) d = d ( q ) . For a tube point v v v , write q v = Π M v q_v=\Pi_Mv q v = Π M v and z v = ( v − q v ) / R z_v=(v-q_v)/R z v = ( v − q v ) / R . From here on, Γ \Gamma Γ denotes R / L R/L R / L times the map defined by completing the square above, so it maps normal coordinates in R R R units to tangent coordinates in L L L units, with a subscript or argument for the point at which it is taken. The frozen normal laws F , y F,y F , y and T , y T,y T , y now also include the base Jacobian det A z \det A_{z} det A z ; the expectation, covariance and regression notation remains the same. For A u ≤ d ≤ r s i n g / L Au\leq d\leq r_{\mathrm{sing}}/L A u ≤ d ≤ r sing / L , (3.5) gives σ m ( D f ( q ) ) / σ f , + ≳ d \sigma_m(Df(q))/\sigma_{f,+}\gtrsim d σ m ( D f ( q )) / σ f , + ≳ d . By (3.6), ∥ D f ( q ) S q , R z ∥ ≲ ε s i n g d \|Df(q)S_{q,Rz}\|\lesssim\varepsilon_{\mathrm{sing}}d ∥ D f ( q ) S q , R z ∥ ≲ ε sing d in normalized units. The coefficient D f ( q ) ( A z − I ) = D f ( q ) S q , R z A z Df(q)(A_z-I)=Df(q)S_{q,Rz}A_z D f ( q ) ( A z − I ) = D f ( q ) S q , R z A z has the same normal Lipschitz bound: differentiate the latter product in z z z and use linearity of the shape operator and the bounded resolvent. Let T , N T,N T , N be the actual stacked tangent and normal fit columns in mixed units, H T = ( T ∗ T ) 1 / 2 H_T=(T^*T)^{1/2} H T = ( T ∗ T ) 1/2 , and δ \delta δ the supremum of ∥ N ∥ \|N\| ∥ N ∥ over the target and its fit centers. The linear frozen tangent coefficient at normal offset y y y is T y 0 = D f ( q ) E T , y ( I − S q , R z ) − 1 T_y^0=Df(q)\mathbb{E}_{T,y}(I-S_{q,Rz})^{-1} T y 0 = D f ( q ) E T , y ( I − S q , R z ) − 1 , where the weight defining E T , y \mathbb{E}_{T,y} E T , y includes det ( I − S q , R z ) − 1 \det(I-S_{q,Rz})^{-1} det ( I − S q , R z ) − 1 . The preceding bound gives ∥ T y 0 − D f ( q ) ∥ ≤ C ε s i n g d \|T_y^0-Df(q)\|\leq C\varepsilon_{\mathrm{sing}}d ∥ T y 0 − D f ( q ) ∥ ≤ C ε sing d . Subtract the reproduced center affine response; for a center v v v of base q v q_v q v the frozen residual is D f ( q v ) ( ( I − S q v , R z ) − 1 − I ) Df(q_v)((I-S_{q_v,Rz})^{-1}-I) D f ( q v ) (( I − S q v , R z ) − 1 − I ) applied to the tangent displacement. Its coefficient and normal Lipschitz seminorm are O ( ε s i n g d ) O(\varepsilon_{\mathrm{sing}}d) O ( ε sing d ) , since d ( q v ) = d + O ( u ) d(q_v)=d+O(u) d ( q v ) = d + O ( u ) . Lemma A.8 bounds its normal regression after the section translation by C ε s i n g d δ C\varepsilon_{\mathrm{sing}}d\delta C ε sing d δ . The nonlinear response remainder and the change from frozen to actual coordinates contribute C u 2 Cu^2 C u 2 by the centered-moment estimate following (A.23). Tangentially the fit differs from T y 0 T_y^0 T y 0 by O ( u ) O(u) O ( u ) : moving its base by O ( b ) O(b) O ( b ) changes the differential and geometry by O ( u ) O(u) O ( u ) , and (3.2) changes the normalized integral of the bounded resolvent by O ( u ) O(u) O ( u ) ; the tilt changes the quadratic form in the normal coordinates and tangent second-moment weight by O ( ∥ Γ ∥ 2 ) O(\lVert\Gamma\rVert^2) O (∥ Γ ∥ 2 ) relative to the aligned fit. Testing a target normal in a center frame adds K q ( w ) n = S q , R n w K_q(w)n=S_{q,R_n}w K q ( w ) n = S q , R n w , with w w w measured in L L L units. Thus
T v = T y 0 + O ( u ) , N v = T v K q ( w ) + O ( u 2 + ε s i n g d δ ) , ∥ N v ∥ ≤ C ( d u + u 2 ) . T_v=T_y^0+O(u),\qquad N_v=T_vK_q(w)+O(u^2+\varepsilon_{\mathrm{sing}}d\delta),\qquad\lVert N_v\rVert\le C(du+u^2). T v = T y 0 + O ( u ) , N v = T v K q ( w ) + O ( u 2 + ε sing d δ ) , ∥ N v ∥ ≤ C ( d u + u 2 ) .
For conditioning uniform in A A A , the fit comparisons give sup v ∥ T v − D f ( q ) ∥ ≤ C 1 ( ε s i n g d + u ) \sup_v\lVert T_v-Df(q)\rVert\le C_1(\varepsilon_{\mathrm{sing}}d+u) sup v ∥ T v − D f ( q )∥ ≤ C 1 ( ε sing d + u ) on C b C_b C b neighborhoods, with C 1 C_1 C 1 independent of A A A . Fix ε s i n g ≤ c o p / ( 8 C 1 ) \varepsilon_{\mathrm{sing}}\le c_{\mathrm{op}}/(8C_1) ε sing ≤ c op / ( 8 C 1 ) . For every A ≥ max { 1 , 8 C 1 / c o p } A\ge\max\{1,8C_1/c_{\mathrm{op}}\} A ≥ max { 1 , 8 C 1 / c op } , σ m ( T ) ≥ c o p d / 2 \sigma_m(T)\ge c_{\mathrm{op}}d/2 σ m ( T ) ≥ c op d /2 and ∥ T v H T − 1 ∥ + ∥ T y 0 H T − 1 ∥ + ∥ D f ( q ) H T − 1 ∥ ≤ C \lVert T_vH_T^{-1}\rVert+\lVert T_y^0H_T^{-1}\rVert+\lVert Df(q)H_T^{-1}\rVert\le C ∥ T v H T − 1 ∥ + ∥ T y 0 H T − 1 ∥ + ∥ D f ( q ) H T − 1 ∥ ≤ C .
Use these estimates before applying the last inverse to the normal equation. Replacing T v H T − 1 T_vH_T^{-1} T v H T − 1 by T y 0 H T − 1 T_y^0H_T^{-1} T y 0 H T − 1 in H T − 1 T ∗ N H_T^{-1}T^{*}N H T − 1 T ∗ N costs O ( ( u / d ) d u ) = O ( u 2 ) O((u/d)du)=O(u^2) O (( u / d ) d u ) = O ( u 2 ) ; replacing the other tangent factor inside T v K q ( w ) T_vK_q(w) T v K q ( w ) costs O ( u 2 ) O(u^2) O ( u 2 ) . Conditional on y y y , the centered part of the frozen w w w integrates to zero, since its multiplier depends only on y y y . Its mean is ( I − S q , R y ) − 1 Γ x ( z x − y ) (I-S_{q,R_y})^{-1}\Gamma_x(z_x-y) ( I − S q , R y ) − 1 Γ x ( z x − y ) in mixed units. Applying T y 0 K q T_y^0K_q T y 0 K q to this mean costs O ( ε s i n g d δ ) O(\varepsilon_{\mathrm{sing}}d\delta) O ( ε sing d δ ) by the product bound on D f ( q ) S q , R n Df(q)S_{q,R_n} D f ( q ) S q , R n at this target. Replacing the true law by the frozen law for this oracle integrand costs O ( u 2 ) O(u^2) O ( u 2 ) , by the section-boundary and density comparison above; the input field is kept at the same centers in the error terms. Consequently for G = T ∗ N G=T^{*}N G = T ∗ N ,
∥ H T G ∥ ≤ C ( u 2 + ε s i n g d δ ) , ( d + u ) ∥ G ∥ ≤ C ( u 2 + ε s i n g d δ ) . (66) \lVert H_TG\rVert\le C(u^2+\varepsilon_{\mathrm{sing}}d\delta),\qquad(d+u)\lVert G\rVert\le C(u^2+\varepsilon_{\mathrm{sing}}d\delta). \tag*{(66)} ∥ H T G ∥ ≤ C ( u 2 + ε sing d δ ) , ( d + u ) ∥ G ∥ ≤ C ( u 2 + ε sing d δ ) . (66)
Only C b C_b C b neighborhoods enter these estimates, with C C C independent of A A A . Thus the coefficient of u 2 u^2 u 2 in (A.28) is fixed before A A A is enlarged in Appendix A.4.3. The second inequality follows from the first because G = H T − 2 T ∗ N G=H_T^{-2}T^{*}N G = H T − 2 T ∗ N and the comparison above gives σ m ( T ) ≳ d + u \sigma_m(T)\gtrsim d+u σ m ( T ) ≳ d + u .
Two successive updates on the regular region. Suppose the target is at least r s i n g r_{\mathrm{sing}} r sing from Σ s i n g \Sigma_{\mathrm{sing}} Σ sing . Its target and center neighborhoods in two successive rounds stay outside radius r s i n g / 2 r_{\mathrm{sing}}/2 r sing /2 since b 0 ≪ r s i n g b_0\ll r_{\mathrm{sing}} b 0 ≪ r sing . The derivation below uses only this property, so it also applies with a step-k k k pool center of the target in place of the target, as it is used for (A.33). For a base displacement w w w in L L L units, let K q ( w ) n = S q , R n w K_q(w)n=S_{q,R_n}w K q ( w ) n = S q , R n w . It maps a unit normal test to the first-order rotation of the tangent frame and has norm at most c ∥ w ∥ c\lVert w\rVert c ∥ w ∥ . Let δ \delta δ be the supremum of ∥ Γ ∥ \lVert\Gamma\rVert ∥ Γ ∥ over the target and contributing centers.
Recall the regular condition number κ f \kappa_f κ f of (A2). In the bounds below, up to (A.33), the constants C C C multiplying ω b e n d \omega_{\mathrm{bend}} ω bend depend only on m m m , c c c , C N C_N C N and κ f \kappa_f κ f ; Remark A.13 also gives uniformity as c c c decreases.
At a center with base r r r and unit normal offset y y y , put A z = ( I − S r , R z ) − 1 A_z=(I-S_{r,R_z})^{-1} A z = ( I − S r , R z ) − 1 and let Γ \Gamma Γ be its incoming mixed shear. Conditional symmetry of the horizontal coordinate gives frozen normal coefficient
D f ( r ) { E T , y [ A z ] Γ − Cov F , y ( A z Γ ( z − y ) , z ) Cov F , y ( z ) − 1 } . Df(r)\left\{\mathbb{E}_{T,y}[A_z]\Gamma-\operatorname{Cov}_{F,y}\bigl(A_z\Gamma(z-y),z\bigr)\operatorname{Cov}_{F,y}(z)^{-1}\right\}. D f ( r ) { E T , y [ A z ] Γ − Cov F , y ( A z Γ ( z − y ) , z ) Cov F , y ( z ) − 1 } .
Indeed, set J = D f ( r ) J=Df(r) J = D f ( r ) and ξ = t + Γ ( z − y ) \xi=t+\Gamma(z-y) ξ = t + Γ ( z − y ) , where t t t is the observed tangent coordinate in L L L units. Conditional symmetry gives E [ ξ ∣ z ] = 0 \mathbb{E}[\xi\mid z]=0 E [ ξ ∣ z ] = 0 . Apply (A.12) to J A z ξ − J A z Γ ( z − y ) JA_z\xi-JA_z\Gamma(z-y) J A z ξ − J A z Γ ( z − y ) : its ξ \xi ξ coefficient is J E T , y A z J\mathbb{E}_{T,y}A_z J E T , y A z and its z z z coefficient is − J Reg F , y [ A z Γ ( z − y ) ] -J\operatorname{Reg}_{F,y}[A_z\Gamma(z-y)] − J Reg F , y [ A z Γ ( z − y )] . Returning to t t t gives the displayed D f ( r ) Df(r) D f ( r ) -normal coefficient; its most transparent form is the stored normal-gradient block. Write S Z = S r , R Z S_Z=S_{r,R_Z} S Z = S r , R Z in the unit normal coordinates. Regression reproduces S Z Γ y = K r ( Γ y ) Z S_Z\Gamma y=\mathcal{K}_r(\Gamma y)Z S Z Γ y = K r ( Γ y ) Z exactly. Thus the center’s normal coefficient is J { K r ( Γ y ) + R y } J\{\mathcal{K}_r(\Gamma y)+\mathcal{R}_y\} J { K r ( Γ y ) + R y } , where
R y = ( E T , y [ A Z ] − I ) Γ − Reg F , y [ ( A Z − I ) Γ Z ] + Reg F , y [ S Z 2 A Z Γ y ] . \mathcal{R}_y=\bigl(\mathbb{E}_{T,y}[A_Z]-I\bigr)\Gamma-\operatorname{Reg}_{F,y}\bigl[(A_Z-I)\Gamma Z\bigr]+\operatorname{Reg}_{F,y}\bigl[S_Z^2A_Z\Gamma y\bigr]. R y = ( E T , y [ A Z ] − I ) Γ − Reg F , y [ ( A Z − I ) Γ Z ] + Reg F , y [ S Z 2 A Z Γ y ] .
Here we used A Z − I = S Z + S Z 2 A Z A_Z-I=S_Z+S_Z^2A_Z A Z − I = S Z + S Z 2 A Z . Under either section tilt, (13) , the covariance lower bound and polarization give
∥ S Z ∥ L 4 ≤ C ω b e n d , ∥ Γ Z ∥ L 4 ≤ C ν ∥ Γ ∥ . \lVert S_Z\rVert_{L^4}\le C\omega_{\mathrm{bend}},\qquad\lVert\Gamma_Z\rVert_{L^4}\le C\nu\lVert\Gamma\rVert. ∥ S Z ∥ L 4 ≤ C ω bend , ∥ Γ Z ∥ L 4 ≤ C ν ∥ Γ ∥ .
Indeed each entry of S Z S_Z S Z is a normal linear functional of norm at most ω b e n d / ν \omega_{\mathrm{bend}}/\nu ω bend / ν ; summing squared entries gives the first bound. The second follows by summing the m m m rows of Γ \Gamma Γ . Since ∥ A Z − I ∥ ≤ ∥ S Z ∥ / ( 1 − c ) \lVert A_Z-I\rVert\le\lVert S_Z\rVert/(1-c) ∥ A Z − I ∥ ≤ ∥ S Z ∥ / ( 1 − c ) , ∥ E T , y A Z − I ∥ ≤ C ω b e n d \lVert\mathbb{E}_{T,y}A_Z-I\rVert\le C\omega_{\mathrm{bend}} ∥ E T , y A Z − I ∥ ≤ C ω bend by (15) . Lemma A.8 bounds regression by C / ν C/\nu C / ν times the response L 2 L^2 L 2 norm. Since ∥ S Z 2 A Z ∥ ≤ ∥ S Z ∥ 2 / ( 1 − c ) \lVert S_Z^2A_Z\rVert\le\lVert S_Z\rVert^2/(1-c) ∥ S Z 2 A Z ∥ ≤ ∥ S Z ∥ 2 / ( 1 − c ) , Cauchy–Schwarz and ω b e n d / ν ≤ C N c \omega_{\mathrm{bend}}/\nu\le C\sqrt{N}c ω bend / ν ≤ C N c give the bound C ω b e n d ∥ Γ ∥ C\omega_{\mathrm{bend}}\lVert\Gamma\rVert C ω bend ∥ Γ ∥ for each of the three terms of R y R_y R y . Hence
∥ R y ∥ ≤ C ω b e n d ∥ Γ ∥ , ∥ E T , y A Z − I ∥ ≤ C ω b e n d . (67) \lVert R_y\rVert\le C\omega_{\mathrm{bend}}\lVert\Gamma\rVert,\qquad\lVert\mathbb{E}_{T,y}A_Z-I\rVert\le C\omega_{\mathrm{bend}}. \tag*{(67)} ∥ R y ∥ ≤ C ω bend ∥ Γ ∥ , ∥ E T , y A Z − I ∥ ≤ C ω bend . (67)
Freeze the outer pool at q q q , and write E 0 \mathbb{E}_0 E 0 for expectation in that pool. Put J = D f ( q ) J=Df(q) J = D f ( q ) , T y 0 = J E T , y A Z T_y^0=J\mathbb{E}_{T,y}A_Z T y 0 = J E T , y A Z and Q y = ( T y 0 ) ∗ T y 0 Q_y=(T_y^0)^*T_y^0 Q y = ( T y 0 ) ∗ T y 0 . The eigenvalues of E T , y A Z \mathbb{E}_{T,y}A_Z E T , y A Z lie in [ ( 1 + c ) − 1 , ( 1 − c ) − 1 ] [(1+c)^{-1},(1-c)^{-1}] [( 1 + c ) − 1 , ( 1 − c ) − 1 ] . Therefore
∥ ( E 0 Q y ) − 1 ∥ ≤ ( 1 + c ) 2 σ m ( J ) 2 , ∥ Q y − J ∗ J ∥ ≤ C ∥ J ∥ 2 ω b e n d , \lVert(\mathbb{E}_0Q_y)^{-1}\rVert\le\frac{(1+c)^2}{\sigma_m(J)^2},\qquad\lVert Q_y-J^*J\rVert\le C\lVert J\rVert^2\omega_{\mathrm{bend}}, ∥( E 0 Q y ) − 1 ∥ ≤ σ m ( J ) 2 ( 1 + c ) 2 , ∥ Q y − J ∗ J ∥ ≤ C ∥ J ∥ 2 ω bend ,
where the second bound follows by subtracting one factor at a time.
Write Γ j ( v ) \Gamma_j(v) Γ j ( v ) for the mixed shear at v v v after step j j j . Let a v a_v a v be the parallel transport of Γ k − 1 ( v ) z v \Gamma_{k-1}(v)z_v Γ k − 1 ( v ) z v to T q M T_qM T q M , extended by zero outside the actual pool in the common base and normal coordinates. Put U v = K q ( w ) + K q ( a v ) U_v=K_q(w)+K_q(a_v) U v = K q ( w ) + K q ( a v ) : the two terms account for transport to the target and the tilt of the center fit, respectively. With normal column T y 0 U v T_y^0U_v T y 0 U v , the frozen normal equation gives G 0 = ( E 0 Q y ) − 1 E 0 [ Q y U v ] G_0=(\mathbb{E}_0Q_y)^{-1}\mathbb{E}_0[Q_yU_v] G 0 = ( E 0 Q y ) − 1 E 0 [ Q y U v ] . Hence
G 0 − E 0 U v = ( E 0 Q y ) − 1 E 0 [ ( Q y − J ∗ J ) ( E 0 [ U v ∣ y ] − E 0 U v ) ] . G_0-\mathbb{E}_0U_v=(\mathbb{E}_0Q_y)^{-1}\mathbb{E}_0\left[(Q_y-J^*J)\left(\mathbb{E}_0[U_v\mid y]-\mathbb{E}_0U_v\right)\right]. G 0 − E 0 U v = ( E 0 Q y ) − 1 E 0 [ ( Q y − J ∗ J ) ( E 0 [ U v ∣ y ] − E 0 U v ) ] .
The centered horizontal section has E 0 [ w ∣ y ] = A y Γ x ( z x − y ) \mathbb{E}_0[w\mid y]=A_y\Gamma_x(z_x-y) E 0 [ w ∣ y ] = A y Γ x ( z x − y ) , whereas ∥ a v ∥ ≤ δ \lVert a_v\rVert\le\delta ∥ a v ∥ ≤ δ pointwise. Hence ∥ E 0 [ U v ∣ y ] ∥ ≤ c ( 2 / ( 1 − c ) + 1 ) δ \lVert\mathbb{E}_0[U_v\mid y]\rVert\le c(2/(1-c)+1)\delta ∥ E 0 [ U v ∣ y ]∥ ≤ c ( 2/ ( 1 − c ) + 1 ) δ . The last three bounds give the absolute error
∥ G 0 − E 0 U v ∥ ≤ C κ f 2 ω b e n d δ . (68) \lVert G_0-\mathbb{E}_0U_v\rVert\le C\kappa_f^2\omega_{\mathrm{bend}}\delta. \tag*{(68)} ∥ G 0 − E 0 U v ∥ ≤ C κ f 2 ω bend δ . (68)
To evaluate the remaining mean, the frozen outer normal marginal has bounded density relative to the unselected law of Z / R Z/R Z / R . Polarization of (15) gives E 0 ∥ S q , R y ∥ ≤ C ω b e n d \mathbb{E}_0\lVert S_{q,R_y}\rVert\le C\omega_{\mathrm{bend}} E 0 ∥ S q , R y ∥ ≤ C ω bend . Moreover ∥ E 0 y ∥ ≤ C ν \lVert\mathbb{E}_0y\rVert\le C\nu ∥ E 0 y ∥ ≤ C ν by testing unit normal directions, and ν ∥ K q ∥ ≤ ω b e n d \nu\lVert K_q\rVert\le\omega_{\mathrm{bend}} ν ∥ K q ∥ ≤ ω bend : the covariance lower bound gives σ N ∥ I I q ( v , v ) ∥ ≤ ω b e n d \sigma_N\lVert\mathrm{II}_q(v,v)\rVert\le\omega_{\mathrm{bend}} σ N ∥ II q ( v , v )∥ ≤ ω bend for each unit tangent v v v . In E 0 [ A y Γ x ( z x − y ) ] \mathbb{E}_0[A_y\Gamma_x(z_x-y)] E 0 [ A y Γ x ( z x − y )] use A y − I = A y S q , R y A_y-I=A_yS_{q,R_y} A y − I = A y S q , R y to obtain
∥ E 0 U v − K q ( Γ x z x + E 0 a v ) ∥ ≤ C ω b e n d δ . (69) \lVert\mathbb{E}_0U_v-K_q(\Gamma_xz_x+\mathbb{E}_0a_v)\rVert\le C\omega_{\mathrm{bend}}\delta. \tag*{(69)} ∥ E 0 U v − K q ( Γ x z x + E 0 a v )∥ ≤ C ω bend δ . (69)
The center formula adds J R y JR_y J R y to the normal column. Writing its leading term as T y 0 K q ( a v ) T_y^0K_q(a_v) T y 0 K q ( a v ) instead of J K q ( a v ) JK_q(a_v) J K q ( a v ) costs at most C ∥ J ∥ ω b e n d δ C\lVert J\rVert\omega_{\mathrm{bend}}\delta C ∥ J ∥ ω bend δ . The two errors together cost at most C ∥ J ∥ ω b e n d δ C\lVert J\rVert\omega_{\mathrm{bend}}\delta C ∥ J ∥ ω bend δ in the column, hence at most
C κ f ω b e n d δ (70) C\kappa_f\omega_{\mathrm{bend}}\delta \tag*{(70)} C κ f ω bend δ (70)
after the pooled inverse T † T^\dagger T † for the stack of T y 0 T_y^0 T y 0 .
The actual and frozen sections differ by relative mass O ( u ) O(u) O ( u ) : within the common local chart, whose boundary is separated from both supports, at each normal offset the rescaled base boundary moves by O ( u ) O(u) O ( u ) , its radius is bounded below, and the density ratio on the intersection is 1 + O ( u ) 1 + O(u) 1 + O ( u ) . The tangent column changes by O ( u ) O(u) O ( u ) ; after transport to the target, the normal column changes by O ( u 2 ) O(u^2) O ( u 2 ) by (A.23) and the comparison of the normal regression coefficients above. Here ∥ w ∥ + ∥ a v ∥ = O ( u ) \lVert w\rVert+\lVert a_v\rVert= O(u) ∥ w ∥ + ∥ a v ∥ = O ( u ) , so the Gram error is O ( u ) O(u) O ( u ) and the cross-moment error is O ( u 2 ) O(u^2) O ( u 2 ) (Lemma A.3 ). The frozen graph is O ( u ) O(u) O ( u ) ; Lemma A.5 gives an O ( u 2 ) O(u^2) O ( u 2 ) graph error. Explicitly, the two Gram matrices differ by O ( u ) O(u) O ( u ) and have bounded inverses, and the two cross moments differ by O ( u 2 ) O(u^2) O ( u 2 ) and have size O ( u ) O(u) O ( u ) ; a graph is the product of an inverse Gram matrix and a cross moment, so the two graphs differ by O ( u 2 ) + O ( u ) O ( u ) O(u^2)+O(u)O(u) O ( u 2 ) + O ( u ) O ( u ) . The same change of measure costs O ( u 2 ) O(u^2) O ( u 2 ) in E 0 a v \mathbb{E}_0a_v E 0 a v , with the incoming field held at the same centers. The three contributions (A.30), (A.31) and (A.32) sum to at most C ω b e n d δ C\omega_{\mathrm{bend}}\delta C ω bend δ . Consequently the outgoing mixed shear has leading term
K q ( Γ k − 1 ( x ) z x + E a v ) , K_q\bigl(\Gamma_{k-1}(x)z_x+\mathbb{E}a_v\bigr), K q ( Γ k − 1 ( x ) z x + E a v ) ,
where the expectation is over the centers v v v of the target’s actual pool, with remainder of norm at most C ω b e n d δ + C u 2 C\omega_{\mathrm{bend}}\delta+Cu^2 C ω bend δ + C u 2 at population level.
Let δ j \delta_j δ j be the supremum of ∥ Γ j ∥ \lVert\Gamma_j\rVert ∥ Γ j ∥ over the neighborhoods used by the update, and put u j = b j / L u_j=b_j/L u j = b j / L . Evaluating the preceding leading term on its own normal offset introduces a shape operator. Its pool average is controlled by the mass bounds in Lemma A.11 and (15) , giving
∥ Γ k ( x ) ∥ ≤ ( c + C r e g ω b e n d ) δ k − 1 + C r e g ω b e n d δ k − 2 + C ( u k 2 + u k − 1 2 ) . (71) \lVert\Gamma_k(x)\rVert\le(c+C_{\mathrm{reg}}\omega_{\mathrm{bend}})\delta_{k-1}+C_{\mathrm{reg}}\omega_{\mathrm{bend}}\delta_{k-2}+C(u_k^2+u_{k-1}^2). \tag*{(71)} ∥ Γ k ( x )∥ ≤ ( c + C reg ω bend ) δ k − 1 + C reg ω bend δ k − 2 + C ( u k 2 + u k − 1 2 ) . (71)
Indeed, the vector inside K q K_q K q in the preceding leading-term formula has norm at most 2 δ k − 1 2\delta_{k-1} 2 δ k − 1 . Evaluation at the center’s own offset gives
∥ Γ k − 1 ( v ) z v ∥ ≤ 2 ∥ S Π M v , R z v ∥ δ k − 2 + C ω b e n d δ k − 2 + C u k − 1 2 . \lVert\Gamma_{k-1}(v)z_v\rVert\le2\lVert S_{\Pi_{M_v,R}z_v}\rVert\delta_{k-2}+C\omega_{\mathrm{bend}}\delta_{k-2}+Cu_{k-1}^2. ∥ Γ k − 1 ( v ) z v ∥ ≤ 2 ∥ S Π M v , R z v ∥ δ k − 2 + C ω bend δ k − 2 + C u k − 1 2 .
At fixed base the selected normal mass is bounded above by the original fiber law. Lemma A.11 bounds the ratio of the mass above B M ( q , 8 b ) B_M(q,8b) B M ( q , 8 b ) to the pool mass by a constant (up to 1 + O ( u ) 1+O(u) 1 + O ( u ) ). The fiber mean of ∥ S Π M v , R z v ∥ \lVert S_{\Pi_{M_v,R}z_v}\rVert ∥ S Π M v , R z v ∥ is at most m ω b e n d m\omega_{\mathrm{bend}} m ω bend by polarization; its pool mean is therefore at most C ω b e n d + O ( u k ) C\omega_{\mathrm{bend}}+O(u_k) C ω bend + O ( u k ) ; the last term multiplies δ k − 2 = O ( u k − 1 ) \delta_{k-2}=O(u_{k-1}) δ k − 2 = O ( u k − 1 ) and is part of the quadratic remainder. Applying ∥ K q ∥ ≤ c \lVert K_q\rVert\le c ∥ K q ∥ ≤ c to that averaged vector and adding the current remainder gives a coefficient c + C ω b e n d c+C\omega_{\mathrm{bend}} c + C ω bend for δ k − 1 \delta_{k-1} δ k − 1 and a coefficient C ω b e n d C\omega_{\mathrm{bend}} C ω bend for δ k − 2 \delta_{k-2} δ k − 2 . Taking C r e g C_{\mathrm{reg}} C reg to be the larger of these two constants proves (A.33); the direct first update has coefficient at most 2 c + C r e g ω b e n d 2c+C_{\mathrm{reg}}\omega_{\mathrm{bend}} 2 c + C reg ω bend . Fresh-fold errors add to the remainders on the concentration events of both rounds (only step 1 for the first update). The pilot multiplier in (A.4) covers the first update.
Remark A.13 (Dependence of the regular-region constant) The preceding proof gives C r e g = C r e g ( m , c , C N , κ f ) C_{\mathrm{reg}}=C_{\mathrm{reg}}(m,c,C_N,\kappa_f) C reg = C reg ( m , c , C N , κ f ) , independent of D D D , p p p , n n n . It can be chosen uniformly for 0 < c ≤ c ˉ < 2 / 3 0<c\le\bar c<2/3 0 < c ≤ c ˉ < 2/3 : the section density ratios and the ratios of volumes of the outer and inner base balls in Lemma A.11 are bounded in terms of m m m , c ˉ \bar c c ˉ , since 1 − 4 / H 2 ≥ 5 / 9 1-4/H^2\ge5/9 1 − 4/ H 2 ≥ 5/9 , η ≤ 1 / 16 \eta\le1/16 η ≤ 1/16 and ( 1 + c ˉ ) − m ≤ det A z ≤ ( 1 − c ˉ ) − m (1+\bar c)^{-m}\le\det A_z\le(1-\bar c)^{-m} ( 1 + c ˉ ) − m ≤ det A z ≤ ( 1 − c ˉ ) − m ; the normal-moment estimates and pooled inverses introduce only C N C_N C N and κ f \kappa_f κ f . For curves, κ f = 1 \kappa_f=1 κ f = 1 even when the nonzero derivative is small; the coefficient of u k 2 + u k − 1 2 u_k^2+u_{k-1}^2 u k 2 + u k − 1 2 in (A.33) may depend on c o p − 1 c_{\mathrm{op}}^{-1} c op − 1 . In the circle example of Appendix § 7 .9, ω b e n d = c / 3 \omega_{\mathrm{bend}}=c/\sqrt{3} ω bend = c / 3 , so this uniform bound gives (16) for all sufficiently small positive c c c . That example also verifies the remaining assumptions.
Sampling the fits and their outer products
Condition on all earlier folds. At width b = b k b=b_k b = b k , with the effective local count N k = n f π − b k m N_k=n f\pi^{-b_k^m} N k = n f π − b k m of Appendix A.1, put
e k = C ( 1 + R / σ N ) 6 ( 1 + σ ζ σ f , + σ N ) ( ( 1 + D + p ) 2 log ( 2 n ) N k + ( 1 + D + p ) 2 log ( 2 n ) N k ) . e_k=C(1+R/\sigma_N)^6\left(1+\frac{\sigma_\zeta}{\sigma_{f,+}\sigma_N}\right)\left(\sqrt{\frac{(1+D+p)^2\log(2n)}{N_k}}+\frac{(1+D+p)^2\log(2n)}{N_k}\right). e k = C ( 1 + R / σ N ) 6 ( 1 + σ f , + σ N σ ζ ) N k ( 1 + D + p ) 2 log ( 2 n ) + N k ( 1 + D + p ) 2 log ( 2 n ) .
Put ℓ = C τ ( 1 + D + p ) 2 log ( 2 n ) \ell=C_\tau(1+D+p)^2\log(2n) ℓ = C τ ( 1 + D + p ) 2 log ( 2 n ) and β k = ℓ / N k + ℓ / N k \beta_k=\sqrt{\ell/N_k}+\ell/N_k β k = ℓ / N k + ℓ / N k . Condition on the earlier folds and pool locations. For each stored fit with weight ω \omega ω from (3.12), use the augmented predictor ψ = ( 1 , s ′ , n ′ / ν ) \psi=(1,s',n'/\nu) ψ = ( 1 , s ′ , n ′ / ν ) and response y 0 = ( f ( Q ) − f ( q ) ) / ( σ f , + L ) y_0=(f(Q)-f(q))/(\sigma_{f,+}L) y 0 = ( f ( Q ) − f ( q )) / ( σ f , + L ) ; on its support, ∥ ψ ∥ ≤ C ( 1 + ν − 1 ) \lVert\psi\rVert\le C(1+\nu^{-1}) ∥ ψ ∥ ≤ C ( 1 + ν − 1 ) and ∣ y 0 ∣ ≤ C u \lvert y_0\rvert\le Cu ∣ y 0 ∣ ≤ C u . Normalize population and empirical moments by E ω ≍ π − b k m \mathbb{E}\omega\asymp\pi^{-b_k^m} E ω ≍ π − b k m , which cancels from the normal equations. Set H ψ = E [ ω ψ ψ ∗ ] / E ω H_\psi=\mathbb{E}[\omega\psi\psi^*]/\mathbb{E}\omega H ψ = E [ ω ψ ψ ∗ ] / E ω . Write Θ \Theta Θ for the population coefficients for y 0 y_0 y 0 and Θ ^ \widehat{\Theta} Θ for the empirical coefficients for ( Y − f ( q ) ) / ( σ f , + L ) (Y-f(q))/(\sigma_{f,+}L) ( Y − f ( q )) / ( σ f , + L ) , both regressed on ψ \psi ψ . Let H ^ ψ \widehat{H}_\psi H ψ be the empirical counterpart of H ψ H_\psi H ψ . The covariance comparison above gives C − 1 I ⪯ H ψ ⪯ C I C^{-1}I\preceq H_\psi\preceq CI C − 1 I ⪯ H ψ ⪯ C I . Least-squares projection bounds the population coefficient matrix by ∥ Θ ∥ ≤ C u \lVert\Theta\rVert\le Cu ∥ Θ ∥ ≤ C u , hence ∥ y 0 − Θ ∗ ψ ∥ ≤ C ( 1 + ν − 1 ) u \lVert y_0-\Theta^*\psi\rVert\le C(1+\nu^{-1})u ∥ y 0 − Θ ∗ ψ ∥ ≤ C ( 1 + ν − 1 ) u . Bernstein’s inequality ([99 Theorem 2.9.5] ), applied on 1 / 4 1/4 1/4 -nets, bounds the difference between H ψ H_\psi H ψ and its empirical counterpart on the fit fold by C ( 1 + ν − 1 ) 2 β k C(1+\nu^{-1})^2\beta_k C ( 1 + ν − 1 ) 2 β k and the empirical average of ω ψ ( y 0 − Θ ∗ ψ ) ∗ / E ω \omega\psi(y_0-\Theta^*\psi)^*/\mathbb{E}\omega ω ψ ( y 0 − Θ ∗ ψ ) ∗ / E ω by C ( 1 + ν − 1 ) 2 u β k C(1+\nu^{-1})^2u\beta_k C ( 1 + ν − 1 ) 2 u β k ; the latter has zero expectation. Condition (A.1) gives
∥ H ^ ψ − H ψ ∥ ≤ C ( 1 + ν − 1 ) 2 β k ≤ λ min ( H ψ ) / 2. (72) \lVert\widehat{H}_\psi-H_\psi\rVert\le C(1+\nu^{-1})^2\beta_k\le\lambda_{\min}(H_\psi)/2. \tag*{(72)} ∥ H ψ − H ψ ∥ ≤ C ( 1 + ν − 1 ) 2 β k ≤ λ m i n ( H ψ ) /2. (72)
Conditional on the predictors, (A5) and ω 2 ≤ ω \omega^2\le\omega ω 2 ≤ ω bound the noise contribution to the coefficients by C { σ ζ / ( σ f , + L ) } β k C\{\sigma_\zeta/(\sigma_{f,+}L)\}\beta_k C { σ ζ / ( σ f , + L )} β k . Lemma A.5, with Q = H ψ Q=H_\psi Q = H ψ , gives
∥ Θ ^ − Θ ∥ ≤ C ( ( 1 + ν − 1 ) 2 u + σ ζ σ f , + L ) β k ≤ e k , (73) \lVert\widehat{\Theta}-\Theta\rVert\le C\left((1+\nu^{-1})^2u+\frac{\sigma_\zeta}{\sigma_{f,+}L}\right)\beta_k\le e_k, \tag*{(73)} ∥ Θ − Θ ∥ ≤ C ( ( 1 + ν − 1 ) 2 u + σ f , + L σ ζ ) β k ≤ e k , (73)
since u ≤ 1 u\le1 u ≤ 1 and 1 + ν − 1 = 1 + R / σ N 1+\nu^{-1}=1+R/\sigma_N 1 + ν − 1 = 1 + R / σ N . A union bound over at most n n n stored centers makes these bounds simultaneous with failure probability at most C n − τ − 2 Cn^{-\tau-2} C n − τ − 2 by the choice of C τ C_\tau C τ , despite reuse of the fit fold. Dividing the tangent and normal coefficients by u u u and ν \nu ν gives errors C e k / u Ce_k/u C e k / u and C e k / ν Ce_k/\nu C e k / ν in mixed slope units. A center-to-target rotation adds O ( u ) O(u) O ( u ) times the tangent error to the mixed normal error.
For the outer sample, first pool the population fits as in (A.6), which are fixed functions of the predictor conditional on the earlier folds. Let H T = ( T ∗ T ) 1 / 2 H_T=(T^*T)^{1/2} H T = ( T ∗ T ) 1/2 under the conditional target window and suppose the normal matrices satisfy sup v ∥ N v ∥ ≤ M N \sup_v\lVert N_v\rVert\le M_N sup v ∥ N v ∥ ≤ M N on this window. In all three regions sup v ∥ T v H T − 1 ∥ ≤ C \sup_v\lVert T_vH_T^{-1}\rVert\le C sup v ∥ T v H T − 1 ∥ ≤ C : use the lower bound (3.5) on the regular region, the annular comparison to D f ( q ) Df(q) D f ( q ) , or, for d ( q ) ≤ A u d(q)\le Au d ( q ) ≤ A u , the comparison to J + O A ( u ) J+O_A(u) J + O A ( u ) and the bound σ m ( T ) ≳ A u \sigma_m(T)\gtrsim_Au σ m ( T ) ≳ A u . In the last case the varying part is O A ( u ) O_A(u) O A ( u ) and for any unit vector a a a the difference between ∥ T v a ∥ \lVert T_va\rVert ∥ T v a ∥ and ∥ T a ∥ L 2 \lVert Ta\rVert_{L^2} ∥ T a ∥ L 2 is O A ( u ) O_A(u) O A ( u ) . For M N > 0 M_N>0 M N > 0 , the normalized matrices H T − 1 T ∗ T v H T − 1 H_T^{-1}T^*T_vH_T^{-1} H T − 1 T ∗ T v H T − 1 and H T − 1 T ∗ N v / M N H_T^{-1}T^*N_v/M_N H T − 1 T ∗ N v / M N have bounded operator norms on the window; when M N = 0 M_N=0 M N = 0 , the cross moment vanishes. Multiplying their scalar directional tests by the window indicator gives variances O ( π − b k m ) O(\pi^{-b_k^m}) O ( π − b k m ) . Their uniform relative deviations are O ( β k ) O(\beta_k) O ( β k ) , as follows. For uniformity, recall the population slope G v , k ∈ R D × p G_{v,k}\in\mathbb{R}^{D\times p} G v , k ∈ R D × p at a center v v v from Appendix A.1. Conditional on earlier folds, M ( v ) = G v , k G v , k ∗ M(v)=G_{v,k}G_{v,k}^* M ( v ) = G v , k G v , k ∗ is fixed. Rescale the scalar tests above by their uniform bound and extend them by zero outside their window W = W x , k W=\mathcal{W}_{x,k} W = W x , k , the target neighborhood (24) , which lies in the local cylinder; call the resulting functions h W h_W h W . We retain these observable windows for the uniform concentration argument; Lemma A.11 supplies their support and mass bounds, without replacing the indexed windows by geodesic balls. Then ∣ h W ∣ ≤ 1 W |h_W|\leq1_W ∣ h W ∣ ≤ 1 W , and h W h_W h W is a linear functional of M ( v ) M(v) M ( v ) on W W W . The subgraphs { ( v , t ) : t < h W ( v ) } \{(v,t):t<h_W(v)\} {( v , t ) : t < h W ( v )} are Boolean combinations of a fixed number of linear inequalities in the O ( D 2 ) O(D^2) O ( D 2 ) coordinates 1 , t , v i , v i v j , M ( v ) i j 1,t,v_i,v_iv_j,M(v)_{ij} 1 , t , v i , v i v j , M ( v ) ij . Thus their Vapnik–Chervonenkis (VC) dimension is at most C D 2 CD^2 C D 2 (Goldberg and Jerrum [44 ] ), without requiring regularity of M M M .
Write P n f \mathbb{P}_{n_f} P n f for the empirical measure of the pool fold. At fixed width b b b , all window masses p W = P ( W ) p_W=\mathbb{P}(W) p W = P ( W ) are comparable to π − b m \pi^{-b_m} π − b m . Applying Giné and Guillou [43 ] and Bousquet [17 ] to the tests and window indicators, whose variances are at most C π − b m C\pi^{-b_m} C π − b m , gives simultaneously
∣ ( P n f − P ) h W ∣ + ∣ P n f ( W ) − p W ∣ ≤ C ( p W ℓ n f + ℓ n f ) , (74) \left|(\mathbb{P}_{n_f}-\mathbb{P})h_W\right|+\left|\mathbb{P}_{n_f}(W)-p_W\right|\leq C\left(\sqrt{\frac{p_W\ell}{n_f}}+\frac{\ell}{n_f}\right), \tag*{(74)} ( P n f − P ) h W + P n f ( W ) − p W ≤ C ( n f p W ℓ + n f ℓ ) , (74)
with ℓ \ell ℓ as above after enlarging C τ C_\tau C τ , since the VC dimension is at most C ( 1 + D + p ) 2 C(1+D+p)^2 C ( 1 + D + p ) 2 . The failure probability is at most C n − τ − 2 Cn^{-\tau-2} C n − τ − 2 . Condition (A.1) gives P n f ( W ) ≥ p W / 2 \mathbb{P}_{n_f}(W)\geq p_W/2 P n f ( W ) ≥ p W /2 . The centering identity in Lemma A.3 , with ∣ P h W ∣ ≤ p W |\mathbb{P}h_W|\leq p_W ∣ P h W ∣ ≤ p W , bounds the conditional-mean error by 2 / p W 2/p_W 2/ p W times the displayed deviation. Condition (A.1) makes this deviation, after the fit and pooling normalizations, a small multiple of u k 2 u_k^2 u k 2 .
Replacing the population fits by the simultaneously estimated ones now perturbs the normalized tangent matrix T v H T − 1 T_vH_T^{-1} T v H T − 1 by O ( e k / ( u ( d + u ) ) ) O(e_k/(u(d+u))) O ( e k / ( u ( d + u ))) in a singular chart, where d = d ( q ) d=d(q) d = d ( q ) . Its product with the normal bound M N = O A ( u ( d + u ) ) M_N=O_A(u(d+u)) M N = O A ( u ( d + u )) costs O A ( e k ) O_A(e_k) O A ( e k ) , and the normal perturbation costs O ( e k / ν ) O(e_k/\nu) O ( e k / ν ) . The tangent Gram normalized on both sides by the population matrix H T − 1 H_T^{-1} H T − 1 stays bounded above and below when e k / ν e_k/\nu e k / ν is a sufficiently small multiple of u 2 u^2 u 2 . Apply (A.14) with E E E and F F F the errors in the stacked tangent and normal columns, and V = N − T G V=N-TG V = N − T G . The preceding bounds give ∥ H T ( G ^ − G ) ∥ = O ( e k / ν ) \|H_T(\widehat{G}-G)\|=O(e_k/\nu) ∥ H T ( G − G ) ∥ = O ( e k / ν ) , the additional term in (A.28). For d ( q ) ≤ A u d(q)\leq Au d ( q ) ≤ A u , the error after division by the weakest singular value is at most C A e k / ( ν u ) C_Ae_k/(\nu u) C A e k / ( ν u ) , which preserves the strict inequality in Lemma A.12 by the third requirement on C s a m p C_{\mathrm{samp}} C samp . On a regular patch the weakest singular value is bounded below; this gives the sampling contribution used in (A.33). The two events for pool sampling and for the common fit fold are intersected, without treating stored slopes as independent. Intersecting with the pilot event and summing the conditional failure probabilities over O ( log n ) O(\log n) O ( log n ) rounds gives the event of Proposition 3.3 .
Preservation of alignment over successive rounds
The constants can be chosen in the following order. The lower bound (3.5) and the fixed scale bounds of (A3) give η , H 0 \eta,H_0 η , H 0 in Lemma A.12 ; the regular constants in (A.33) determine the stretch condition (16) . Choose A A A large enough that dividing the term C u 2 Cu^2 C u 2 in (A.28) by u ( d + u ) u(d+u) u ( d + u ) for d ≥ A u d\geq Au d ≥ A u costs at most a ρ η / 8 a_\rho\eta/8 a ρ η /8 . This coefficient uses the uniform chart bounds and the annular signal lower bound with A A A above a fixed conditioning threshold; it does not grow with A A A . Only the local comparison terms carry the notation C A C_A C A ; the coefficient of u 2 u^2 u 2 used to choose A A A is fixed. Next take ϵ s i n g \epsilon_{\mathrm{sing}} ϵ sing small enough to absorb both the annular term C ϵ s i n g δ C\epsilon_{\mathrm{sing}}\delta C ϵ sing δ and the comparison error C A ϵ s i n g C_A\epsilon_{\mathrm{sing}} C A ϵ sing for d ≤ A u d\leq Au d ≤ A u . Fix K 0 K_0 K 0 for the pilot and its first regular update and c ρ c_\rho c ρ for the kernel lower bound in the prediction step. Then choose ε 0 \varepsilon_0 ε 0 in (A5) so the initial width is a small enough fraction of σ N \sigma_N σ N , r s i n g r_{\mathrm{sing}} r sing and r c y l r_{\mathrm{cyl}} r cyl , with
c ρ b 0 ≤ min { R / 8 , r c y l / 64 } , max { C A , C c ρ } b 0 < min { σ N / 8 , r c y l / 16 } . (75) c_\rho b_0 \le\min\{R/8,r_{\mathrm{cyl}}/64\}, \qquad\max\{C_A,Cc_\rho\}b_0 < \min\{\sigma_N/8,r_{\mathrm{cyl}}/16\}. \tag*{(75)} c ρ b 0 ≤ min { R /8 , r cyl /64 } , max { C A , C c ρ } b 0 < min { σ N /8 , r cyl /16 } . (75)
The constants here bound the actual and frozen support radii. These fixed choices give Lemma A.11 at every refinement and prediction width, separate all compared supports from their chart boundaries, and cover every use of (14) . Finally the third requirement on C s a m p C_{\mathrm{samp}} C samp in Appendix § 7 .6 makes e k / ν e_k/\nu e k / ν a sufficiently small fixed multiple of u k 2 u_k^2 u k 2 .
The pilot then starts with the required incoming angle. At d ≤ A u k d \leq Au_k d ≤ A u k , the strict bound in Lemma A.12 , the chart comparison and the sampling event imply that the outgoing physical angle is less than a ρ η b k / R a_\rho\eta b_k/R a ρ η b k / R . The complementary graph block is already O A ( u k 2 ) O_A(u_k^2) O A ( u k 2 ) by the same chart comparison. On A u k ≤ d ≤ r s i n g / L Au_k \leq d \leq r_{\mathrm{sing}}/L A u k ≤ d ≤ r sing / L , the same bound follows from (A.28) by the choice of A A A and ε s i n g \varepsilon_{\mathrm{sing}} ε sing . On d ≥ r s i n g / L d \geq r_{\mathrm{sing}}/L d ≥ r sing / L , divide (A.33) by the next width. The sum of its two predecessor coefficients is bounded by the stricter squared-width sum in (16) ; the remaining terms divided by u k u_k u k are O ( u k ) O(u_k) O ( u k ) . Its first step is covered by the initial slack in (A.4). For an empty singular set use this regular estimate everywhere, with fixed 0 < η ≤ 1 / 16 0 < \eta\leq1/16 0 < η ≤ 1/16 and H 0 ≥ 3 H_0 \geq3 H 0 ≥ 3 as required by Lemma A.11 . These estimates leave slack for the cubic graph-to-angle conversion. Since b k + 1 / b k ≥ a ρ b_{k+1}/b_k \geq a_\rho b k + 1 / b k ≥ a ρ , each window contains the normal fibers specified in Proposition 3.2 at every round. Induct from the pilot event. On its intersection with the concentration events through step k − 1 k-1 k − 1 , the induction hypothesis supplies the incoming angular bounds. Conditional on these earlier folds, all three concentration bounds (A.34)–(A.36) hold at step k k k with probability at least 1 − C n − τ − 2 1-Cn^{-\tau-2} 1 − C n − τ − 2 ; on this event, the preceding estimates give the updated angle bound. Including the pilot failure, the conclusions hold simultaneously through step k k k with probability at least 1 − C ( k + 1 ) n − τ − 2 1-C(k+1)n^{-\tau-2} 1 − C ( k + 1 ) n − τ − 2 , proving Proposition 3.2 . Taking k = k ∗ = O ( log n ) k=k_* = O(\log n) k = k ∗ = O ( log n ) gives total failure probability at most C n − τ − 1 Cn^{-\tau-1} C n − τ − 1 and the first claim of Proposition 3.3 .
For the sharper bound (26) , take the supremum over targets of ( d + u k ) ∥ Γ k ∥ / u k 2 (d+u_k)\lVert\Gamma_k\rVert/u_k^2 ( d + u k ) ∥ Γ k ∥ / u k 2 . For d ≤ A u k d \leq Au_k d ≤ A u k it is bounded using Proposition 3.2 . In the annular region a predecessor is at base distance O ( b k ) O(b_k) O ( b k ) , so its coefficient after division by squared width is at most C ε s i n g ( 1 + C / A ) / a ρ 2 < 1 C\varepsilon_{\mathrm{sing}}(1+C/A)/a_\rho^2 < 1 C ε sing ( 1 + C / A ) / a ρ 2 < 1 . For a regular target, the sum in (16) acquires a factor at most 1 + C b 0 / r s i n g 1+Cb_0/r_{\mathrm{sing}} 1 + C b 0 / r sing over the two rounds and remains below one. The remaining terms are uniformly bounded. Thus the supremum is at most the maximum of the initial two bounds, the bound for d ≤ A u k d \leq Au_k d ≤ A u k , and a bound for the remaining terms divided by one minus the larger sum of coefficients multiplying previous errors. The resulting constant may depend on L / b 0 L/b_0 L / b 0 . Returning to physical angles gives (26) and proves Proposition 3.3 .
Phase 3: proof of Proposition 3.4
Fix the pilot and refinement data and the pilot split on the event of Proposition 3.3 and set u = b ∗ / σ N u=b_*/\sigma_N u = b ∗ / σ N . Fix a query x = q + R z x x=q+Rz_x x = q + R z x , with z x z_x z x in units of R R R as in Appendix § 7 .§ 4 . Apply Lemma A.11 with the projector evaluated at the predictor, for both the true and estimated kernels; the bound R ∥ P ^ k ∗ ( x ) − P q ∥ < a ρ η b k ∗ R\lVert\widehat{P}_{k_*}(x)-P_q\rVert< a_\rho\eta b_{k_*} R ∥ P k ∗ ( x ) − P q ∥ < a ρ η b k ∗ in Proposition 3.3 and b ∗ = c ρ b k ∗ b_*=c_\rho b_{k_*} b ∗ = c ρ b k ∗ give the hypothesis of the lemma. Both integrals are thereby restricted to the same U q = B M ( q , r c y l / 4 ) U_q=B_M(q,r_{\mathrm{cyl}}/4) U q = B M ( q , r cyl /4 ) , and their supports have base radius O ( b ∗ ) O(b_*) O ( b ∗ ) . All comparisons below are between these localized measures. First use the true projector P Π M χ i P_{\Pi_M\chi_i} P Π M χ i at each predictor X i X_i X i in (A.7). If the query is q + R z x q + Rz_x q + R z x and the predictor base has displacement w w w from q q q , its tangent projection of the query-minus-predictor vector, transported back to T q M T_q\mathcal{M} T q M , is
− ( I − S q , R z x ) w + O ( ∥ w ∥ 2 / reach M ) -(I-S_{q,Rz_x})w+O(\lVert w\rVert^2/\operatorname{reach}_{\mathcal{M}}) − ( I − S q , R z x ) w + O (∥ w ∥ 2 / reach M )
in physical units: the predictor’s own offset drops out exactly; differentiating the moving tangent projection gives the S q , R z x w S_{q,Rz_x}w S q , R z x w term. For a fixed parallel normal offset z z z the Euclidean squared distance has constant term ∥ z − R z x ∥ 2 \lVert z-Rz_x\rVert^2 ∥ z − R z x ∥ 2 and no term linear in w w w . The leading oracle kernel is therefore even in w w w for each z z z . Its support is strictly inside U q U_q U q by the lower singular-value bound for I − S q , R z x I-S_{q,Rz_x} I − S q , R z x and the fixed resolution choice, so the base restriction does not truncate this even section and the frozen average of the linear link term vanishes. On the O ( b ∗ ) O(b_*) O ( b ∗ ) support, chart, density, kernel and fiber-law changes, in the same base and normal coordinates, have total variation O ( u ) O(u) O ( u ) times the kernel mass. Lemma A.3, applied to the O ( u ) O(u) O ( u ) response oscillation, gives bias O ( u 2 ) O(u^2) O ( u 2 ) in units σ f , + σ N \sigma_{f,+}\sigma_N σ f , + σ N . The C 1 , 1 C^{1,1} C 1 , 1 remainder has the same order. No cylinder or chart boundary contributes to this comparison.
Lemma A.11 bounds both kernels below by 1 / 16 1/16 1/16 on every normal fiber above B M ( q , b ∗ / 16 ) B_{\mathcal{M}}(q,b_*/16) B M ( q , b ∗ /16 ) and places their supports above B M ( q , 8 b ∗ ) B_{\mathcal{M}}(q,8b_*) B M ( q , 8 b ∗ ) . On that ball, d ( Π M X i ) + u ≍ d ( q ) + u d(\Pi_{\mathcal{M}}X_i)+u\asymp d(q)+u d ( Π M X i ) + u ≍ d ( q ) + u , and replacing P Π M X i P_{\Pi_{\mathcal{M}}X_i} P Π M X i changes the tangent argument by at most C R ∥ P b ^ k ∗ ( X i ) − P Π M X i ∥ CR\lVert P_{\widehat{b}_{k_*}}(X_i)-P_{\Pi_{\mathcal{M}}X_i}\rVert C R ∥ P b k ∗ ( X i ) − P Π M X i ∥ . Lipschitz continuity of t ↦ ( 1 − a − t 2 ) + 2 t\mapsto(1-a-t^2)_+^2 t ↦ ( 1 − a − t 2 ) + 2 uniformly in a ≥ 0 a\geq0 a ≥ 0 and (3.14) bound the weight difference by C u / ( d ( q ) + u ) Cu/(d(q)+u) C u / ( d ( q ) + u ) . Both masses are ≍ π − b ∗ m \asymp\pi^{-b_*m} ≍ π − b ∗ m . Lemma A.3 therefore bounds the bias change by C u 2 / ( d ( q ) + u ) Cu^2/(d(q)+u) C u 2 / ( d ( q ) + u ) . Near a point where D f = 0 Df=0 D f = 0 , the response oscillation is O ( u ( d ( q ) + u ) ) O(u(d(q)+u)) O ( u ( d ( q ) + u )) , giving bias O ( u 2 ) O(u^2) O ( u 2 ) instead. Outside the singular charts the bias is also O ( u 2 ) O(u^2) O ( u 2 ) . Partial rank deficiency requires m ≥ 2 m\geq2 m ≥ 2 ; applying Lemma A.10 with α = 2 \alpha=2 α = 2 to those points gives integrated squared bias O ( u 4 ) O(u^4) O ( u 4 ) , with an extra 1 + ∣ log u ∣ 1+|\log u| 1 + ∣ log u ∣ only for m = 2 m=2 m = 2 when such points exist.
For a fixed query, Bernstein’s inequality gives a prediction denominator ≳ n p r e d π − b ∗ m \gtrsim n_{\mathrm{pred}}\pi^{-b_*m} ≳ n pred π − b ∗ m except with probability exp ( − c ′ n p r e d π − b ∗ m ) \exp(-c'n_{\mathrm{pred}}\pi^{-b_*m}) exp ( − c ′ n pred π − b ∗ m ) . Center the responses at the population weighted mean and use this denominator in the ratio. Independence of the prediction observations and 0 ≤ K 2 ≤ K 0\leq K^2\leq K 0 ≤ K 2 ≤ K give conditional mean square fluctuation, in normalized units, at most
C { p ( σ ζ σ f , + σ N ) 2 + u 2 } n p r e d π − b ∗ m . \frac{C\left\{p\left(\frac{\sigma_\zeta}{\sigma_{f,+}\sigma_N}\right)^2+u^2\right\}}{n_{\mathrm{pred}}\pi^{-b_*m}}. n pred π − b ∗ m C { p ( σ f , + σ N σ ζ ) 2 + u 2 } .
On denominator failure, including the fallback to the mean of the prediction sample, apply Lemma A.9 conditional on the pilot and refinement data, query and prediction locations, with Δ = diam ( f ( M ) ) \Delta=\operatorname{diam}(f(\mathcal{M})) Δ = diam ( f ( M )) and v 2 = C p σ ζ 2 v^2=Cp\sigma_\zeta^2 v 2 = C p σ ζ 2 . Its normalized risk contribution is at most
C exp ( − c ′ n p r e d π − b ∗ m ) diam ( f ( M ) ) 2 + p σ ζ 2 ( σ f , + σ N ) 2 . (76) C\exp(-c'n_{\mathrm{pred}}\pi^{-b_*m})
\frac{\operatorname{diam}(f(\mathcal{M}))^2+p\sigma_\zeta^2}{(\sigma_{f,+}\sigma_N)^2}. \tag*{(76)} C exp ( − c ′ n pred π − b ∗ m ) ( σ f , + σ N ) 2 diam ( f ( M ) ) 2 + p σ ζ 2 . (76)
Stopping and the sample threshold
The rule (A.1) of Appendix A.1 for the number of steps makes the polynomial sample requirement explicit. By (A.4) the actual terminal width for a run of j j j steps lies between b j b_j b j and ( max R / min R ) b j (\max\mathcal{R}/\min\mathcal{R})b_j ( max R / min R ) b j . The width decreases within a run because a ρ a h < 1 a_\rho a_h<1 a ρ a h < 1 for D > m D>m D > m . At each preceding round, the left-hand side of (A.1) with the actual width b k b_k b k is no smaller. The multiplier C s a m p C_{\mathrm{samp}} C samp is taken large enough for four requirements: the Gram perturbations in Appendix A.4.2 are at most half the smallest eigenvalue; the empirical window masses there are at least half their population values; substitution into the definition of e k e_k e k makes e k / ν e_k/\nu e k / ν at most the fixed multiple of u k 2 u_k^2 u k 2 required in Appendix § 7 .§ 4.3 ; and the prediction count satisfies the bound at the end of this subsection. The third requirement also makes the pool deviation in Appendix § 7 .§ 4.2 a small multiple of u k 2 u_k^2 u k 2 , since β k ≤ e k / ν \beta_k \le e_k/\nu β k ≤ e k / ν . The multiplier C p i l C_{\mathrm{pil}} C pil in (A.39) is taken large enough for the Bernstein bounds of Appendix § 7 .§ 3 .
It suffices that (A.1) hold for j = 1 j=1 j = 1 , and that the smaller of the two pilot folds satisfy
min { ∣ S a n c ∣ , ∣ S d e s ∣ } p 0 ≥ C p i l [ D + log 2 + n ∣ R ∣ p 0 ] ( 1 + max R w Y + ( max R ) 2 reach M σ N 2 w Y ) 2 . (77) \min\{|S_{\mathrm{anc}}|,|S_{\mathrm{des}}|\}p_0 \ge C_{\mathrm{pil}}\left[D+\log\frac{2+n|\mathcal{R}|}{p_0}\right]\left(1+\frac{\max\mathcal{R}}{w_Y}+\frac{(\max\mathcal{R})^2\operatorname{reach}_{\mathcal{M}}}{\sigma_N^2w_Y}\right)^2. \tag*{(77)} min { ∣ S anc ∣ , ∣ S des ∣ } p 0 ≥ C pil [ D + log p 0 2 + n ∣ R ∣ ] ( 1 + w Y max R + σ N 2 w Y ( max R ) 2 reach M ) 2 . (77)
Since each pilot fold has at least n f n_f n f observations, both requirements are conditions on n n n alone. Both hold for every n ≥ n 0 n\ge n_0 n ≥ n 0 , with the threshold n 0 = C Θ log 2 ( 2 + Θ ) n_0=C\Theta\log^2(2+\Theta) n 0 = C Θ log 2 ( 2 + Θ ) of (3.13), when the constant there is large. Indeed, b 1 / σ N b_1/\sigma_N b 1 / σ N is a fixed multiple of w Y / σ N w_Y/\sigma_N w Y / σ N and n f ( 1 ) ≥ n / 24 n_f(1)\ge n/24 n f ( 1 ) ≥ n /24 , so (A.1) with j = 1 j=1 j = 1 follows from n ≥ C Θ log ( 2 n ) n\ge C\Theta\log(2n) n ≥ C Θ log ( 2 n ) . Since π N − σ m \pi_N^{-\sigma_m} π N − σ m and b 0 / σ N b_0/\sigma_N b 0 / σ N are bounded, the rule for the number of steps gives k ∗ ≤ 2 log n + C k_*\le2\log n+C k ∗ ≤ 2 log n + C , hence n f ≥ n / ( C log ( 2 n ) ) n_f\ge n/(C\log(2n)) n f ≥ n / ( C log ( 2 n )) . Because D + log ( ( 2 + n ∣ R ∣ ) / p 0 ) ≤ ( D + log ( 2 ∣ R ∣ / p 0 ) ) ( 1 + log ( 2 n ) ) D+\log((2+n|\mathcal{R}|)/p_0)\le(D+\log(2|\mathcal{R}|/p_0))(1+\log(2n)) D + log (( 2 + n ∣ R ∣ ) / p 0 ) ≤ ( D + log ( 2∣ R ∣/ p 0 )) ( 1 + log ( 2 n )) , (A.39) with n f n_f n f in place of min { ∣ S a n c ∣ , ∣ S d e s ∣ } \min\{|S_{\mathrm{anc}}|,|S_{\mathrm{des}}|\} min { ∣ S anc ∣ , ∣ S des ∣ } follows from n ≥ C Θ log 2 ( 2 n ) n\ge C\Theta\log^2(2n) n ≥ C Θ log 2 ( 2 n ) . Both inequalities hold for n ≥ C Θ log 2 ( 2 + Θ ) n\ge C\Theta\log^2(2+\Theta) n ≥ C Θ log 2 ( 2 + Θ ) . The threshold depends on τ \tau τ through the multipliers C s a m p C_{\mathrm{samp}} C samp and C p i l C_{\mathrm{pil}} C pil . Up to logarithmic factors it is polynomial in D + p D+p D + p , in the inverse local masses ( π N − σ m ) − 1 (\pi_N^{-\sigma_m})^{-1} ( π N − σ m ) − 1 and p 0 − 1 p_0^{-1} p 0 − 1 , and in the scale ratios R / σ N R/\sigma_N R / σ N , σ N / w Y \sigma_N/w_Y σ N / w Y , σ ζ / ( σ f , + σ N ) \sigma_\zeta/(\sigma_{f,+}\sigma_N) σ ζ / ( σ f , + σ N ) , max R / w Y \max R/w_Y max R / w Y and ( max R ) 2 reach M / ( σ N 2 w Y ) (\max R)^2\operatorname{reach}_{\mathcal{M}}/(\sigma_N^2w_Y) ( max R ) 2 reach M / ( σ N 2 w Y ) . Since b ∗ ≥ b k ∗ b_*\ge b_{k_*} b ∗ ≥ b k ∗ , b k ∗ ≤ σ N b_{k_*}\le\sigma_N b k ∗ ≤ σ N and n p r e d ≥ n f n_{\mathrm{pred}}\ge n_f n pred ≥ n f , (A.1) gives n p r e d π N − b ∗ ≥ C log ( 2 n ) n_{\mathrm{pred}}\pi_N^{-b_*}\ge C\log(2n) n pred π N − b ∗ ≥ C log ( 2 n ) . By the fourth requirement on C s a m p C_{\mathrm{samp}} C samp , the constant here is as large as the prescribed confidence exponent τ \tau τ requires, so the denominator failure probability on the good training event is at most n − τ − 1 n^{-\tau-1} n − τ − 1 . Failure of (A.1) at k ∗ + 1 k_*+1 k ∗ + 1 , the bounded ratio of neighboring widths and n f ( k ∗ + 1 ) ≍ n f n_f(k_*+1)\asymp n_f n f ( k ∗ + 1 ) ≍ n f give
( b ∗ σ N ) m + 4 ≲ ( 1 + D + p ) 4 ( 1 + R / σ N ) 16 ( 1 + σ ζ / ( σ f , + σ N ) ) 2 log 2 ( 2 n ) n π N − σ m . (78) \left(\frac{b_*}{\sigma_N}\right)^{m+4}\lesssim\frac{(1+D+p)^4(1+R/\sigma_N)^{16}(1+\sigma_\zeta/(\sigma_{f,+}\sigma_N))^2\log^2(2n)}{n\pi_N^{-\sigma_m}}. \tag*{(78)} ( σ N b ∗ ) m + 4 ≲ n π N − σ m ( 1 + D + p ) 4 ( 1 + R / σ N ) 16 ( 1 + σ ζ / ( σ f , + σ N ) ) 2 log 2 ( 2 n ) . (78)
All displayed dimension costs are polynomial since R / σ N ≍ D − m + 2 R/\sigma_N\asymp\sqrt{D-m+2} R / σ N ≍ D − m + 2 . The local probability and the initial resolution remain explicit. The localization requirements of Lemma A.11 constrain only the fixed initial resolution in Appendix § 7 .§ 4.3 ; they introduce no additional power of n n n or dimension factor into these sample bounds.
Proof of Theorem 3.5
Fix the pilot and refinement data and the pilot split on the event of Proposition 3.3 , and put u = b k ∗ / σ N u=b_{k_*}/\sigma_N u = b k ∗ / σ N . Equation (26) and Lemma A.10 , with S = Σ s i n g S=\Sigma_{\mathrm{sing}} S = Σ sing and α = 1 \alpha=1 α = 1 , give conditional mean tangent-space error C ( σ N / R ) u 2 [ 1 + 1 { m = 1 } ∣ log u ∣ ] C(\sigma_N/R)u^2[1+\mathbf{1}_{\{m=1\}}|\log u|] C ( σ N / R ) u 2 [ 1 + 1 { m = 1 } ∣ log u ∣ ] . On the complementary event the projector error is at most one, contributing at most C n − τ − 1 Cn^{-\tau-1} C n − τ − 1 to its expectation. The terminal width bound (A.40) applies since b ∗ = c ρ b k ∗ b_*=c_\rho b_{k_*} b ∗ = c ρ b k ∗ in the theorem; it gives the stated rate for E t a n \mathcal{E}_{\mathrm{tan}} E tan .
For prediction, (A.1) bounds the variance in (3.15) by a constant times ( b ∗ / σ N ) 4 (b_*/\sigma_N)^4 ( b ∗ / σ N ) 4 on the good training event. A point with partially rank-deficient D f D_f D f in dimension m = 2 m=2 m = 2 contributes the logarithm in the integrated bias. Equation (A.40) then gives the first term of (3.17).
Training failure and conditional denominator failure each have probability at most C n − τ − 1 Cn^{-\tau-1} C n − τ − 1 by Proposition 3.3 and Appendix § 7 .6. Lemma A.9 gives the exceptional-event term of (29) .
If M \mathcal{M} M is connected, place points σ N / 8 \sigma_N/8 σ N /8 apart along a minimizing geodesic. Their disjoint intrinsic balls of radius σ N / 32 \sigma_N/32 σ N /32 each have probability at least C − 1 π − σ N m C^{-1}\pi^{-\sigma_N^m} C − 1 π − σ N m by (A1)–(A2). The length of the geodesic is therefore at most C σ N [ 1 + ( π − σ N m ) − 1 ] C\sigma_N[1+(\pi^{-\sigma_N^m})^{-1}] C σ N [ 1 + ( π − σ N m ) − 1 ] . Integrating D f Df D f along it yields diam ( f ( M ) ) / ( σ f , + σ N ) ≤ C [ 1 + ( π − σ N m ) − 1 ] \operatorname{diam}(f(\mathcal{M}))/(\sigma_{f,+}\sigma_N)\le C[1+(\pi^{-\sigma_N^m})^{-1}] diam ( f ( M )) / ( σ f , + σ N ) ≤ C [ 1 + ( π − σ N m ) − 1 ] .
Gaussian response noise
Corollary A.14 (Gaussian response noise). Assume (A1)–(A4) and the spatial resolution condition in (21) , and let the response noises be independent N ( 0 , σ ‾ ζ 2 I p ) N(0,\overline{\sigma}_{\zeta}^{2}I_p) N ( 0 , σ ζ 2 I p ) vectors, independent of the predictors, with σ ‾ ζ > 0 \overline{\sigma}_{\zeta}>0 σ ζ > 0 . Use the estimator, scales and sample-size requirements of Theorem 3.5 with σ ζ = 2 σ ‾ ζ \sigma_{\zeta}=2\overline{\sigma}_{\zeta} σ ζ = 2 σ ζ . Let N p i l = ∣ S a n c ∣ + ∣ S d e s ∣ N_{\mathrm{pil}}=|\mathcal{S}_{\mathrm{anc}}|+|\mathcal{S}_{\mathrm{des}}| N pil = ∣ S anc ∣ + ∣ S des ∣ be the total number of observations in the pilot anchor and design folds (Appendix § 7 .§ 1 ). If
δ Y ≥ 8 σ ‾ ζ ( p + 2 log ( N p i l n τ + 1 ) ) , (79) \delta_Y \ge8\overline{\sigma}_{\zeta}\left(\sqrt{p}+\sqrt{2\log(N_{\mathrm{pil}}n^{\tau+1})}\right), \tag*{(79)} δ Y ≥ 8 σ ζ ( p + 2 log ( N pil n τ + 1 ) ) , (79) then (28) and (29) hold, with possibly larger constants.
Remark A.15 (Fixed Gaussian variance and response resolution). With p p p , σ ‾ ζ > 0 \overline{\sigma}_{\zeta}>0 σ ζ > 0 and δ Y \delta_Y δ Y fixed, the probability that every pilot noise vector has norm at most δ Y / 8 \delta_Y/8 δ Y /8 tends to zero as N p i l → ∞ N_{\mathrm{pil}}\to\infty N pil → ∞ . At this variance and resolution, condition (A.41) holds only over a finite range of sample sizes.
Proof of Corollary A.14. Set ϵ = n − τ − 1 \epsilon=n^{-\tau-1} ϵ = n − τ − 1 and let E G = { ∥ ζ i ∥ ≤ δ Y / 8 for all i ∈ S a n c ∪ S d e s } E_G=\{\lVert\zeta_i\rVert\le\delta_Y/8\text{ for all }i\in\mathcal{S}_{\mathrm{anc}}\cup\mathcal{S}_{\mathrm{des}}\} E G = {∥ ζ i ∥ ≤ δ Y /8 for all i ∈ S anc ∪ S des } . For G ∼ N ( 0 , I p ) G\sim N(0,I_p) G ∼ N ( 0 , I p ) , the Gaussian norm bound P { ∥ G ∥ > p + 2 t } ≤ e − t \mathbb{P}\{\lVert G\rVert>\sqrt{p}+\sqrt{2t}\}\le e^{-t} P {∥ G ∥ > p + 2 t } ≤ e − t , t ≥ 0 t\ge0 t ≥ 0 , and a union bound give, by (A.41),
P ( E G c ) ≤ N p i l exp [ − 1 2 ( δ Y 8 σ ‾ ζ − p ) 2 ] ≤ ϵ . \mathbb{P}(E_G^c)\le N_{\mathrm{pil}}\exp\left[-\frac{1}{2}\left(\frac{\delta_Y}{8\overline{\sigma}_{\zeta}}-\sqrt{p}\right)^2\right]\le\epsilon. P ( E G c ) ≤ N pil exp [ − 2 1 ( 8 σ ζ δ Y − p ) 2 ] ≤ ϵ .
The union of the pilot blocks is reserved before sampling, so E G E_G E G is independent of their random anchor/design split. Conditional on E G E_G E G , the pilot noises remain independent of one another and of all predictors, and each has its Gaussian law conditioned on a centered ball. These laws are centered by symmetry. Radial truncation cannot increase E exp ( ⟨ v , ζ i ⟩ 2 / ( 4 σ ‾ ζ 2 ) ) ≤ 2 \mathbb{E}\exp(\langle v,\zeta_i\rangle^2/(4\overline{\sigma}_{\zeta}^{2}))\le\sqrt{2} E exp (⟨ v , ζ i ⟩ 2 / ( 4 σ ζ 2 )) ≤ 2 for a unit v v v : Gaussian radius and direction are independent, and the integrand is nondecreasing in the radius for each direction. Thus their directional ψ 2 \psi_2 ψ 2 norms are at most 2 σ ‾ ζ 2\overline{\sigma}_{\zeta} 2 σ ζ . The remaining folds retain their original independent Gaussian laws. All hypotheses of Theorem 3.5 therefore hold conditionally with the same geometric and sampling constants; in particular 2 σ ‾ ζ / δ Y 2\overline{\sigma}_{\zeta}/\delta_Y 2 σ ζ / δ Y is bounded by (A.41).
On E G c E_G^c E G c , the projector error is at most one, and Lemma A.9 bounds the prediction estimator’s conditional squared risk by diam ( f ( M ) ) 2 + p σ ‾ ζ 2 \operatorname{diam}(f(\mathcal{M}))^2+p\overline{\sigma}_{\zeta}^{2} diam ( f ( M ) ) 2 + p σ ζ 2 . E G E_G E G and the weights are independent of the prediction noises. Since E G E_G E G depends only on pilot noises, splitting each expectation over E G E_G E G and E G c E_G^c E G c adds at most ϵ \epsilon ϵ to the mean tangent error and at most
ϵ diam ( f ( M ) ) 2 + p σ ‾ ζ 2 ( σ f , + σ N ) 2 \epsilon\frac{\operatorname{diam}(f(\mathcal{M}))^2+p\overline{\sigma}_{\zeta}^{2}}{(\sigma_{f,+}\sigma_N)^2} ϵ ( σ f , + σ N ) 2 diam ( f ( M ) ) 2 + p σ ζ 2
to the normalized prediction risk. Since τ + 1 > 2 / ( m + 4 ) \tau+ 1 > 2/(m + 4) τ + 1 > 2/ ( m + 4 ) , the first term is absorbed into the stated tangent rate. With ϵ = n − τ − 1 \epsilon= n^{-\tau-1} ϵ = n − τ − 1 and σ ζ = 2 σ ‾ ζ \sigma_{\zeta} = 2\overline{\sigma}_{\zeta} σ ζ = 2 σ ζ , the second is absorbed into the exceptional-event term of (29) by enlarging its constant.
Examples and response dimension
Example 1 (A noisy annulus with rank-deficient D f Df D f ) Let m = 1 m = 1 m = 1 , D = p = 2 D = p = 2 D = p = 2 , M = { q ( θ ) = ( cos θ , sin θ ) : 0 ≤ θ < 2 π } \mathcal{M} = \{q(\theta) = (\cos\theta,\sin\theta) : 0 \le\theta< 2\pi\} M = { q ( θ ) = ( cos θ , sin θ ) : 0 ≤ θ < 2 π } , and σ = c / 3 \sigma= c/\sqrt{3} σ = c / 3 . Define
f ( q ( θ ) ) = ( arcsin cos θ 1 + σ 2 , sin 3 θ σ 2 + sin 2 θ ) . (80) f(q(\theta)) = \left(\arcsin\frac{\cos\theta}{\sqrt{1+\sigma^2}}, \frac{\sin^3\theta}{\sigma^2+\sin^2\theta}\right). \tag*{(80)} f ( q ( θ )) = ( arcsin 1 + σ 2 cos θ , σ 2 + sin 2 θ sin 3 θ ) . (80)
Take independent Θ ∼ Unif [ 0 , 2 π ) \Theta\sim\operatorname{Unif}[0,2\pi) Θ ∼ Unif [ 0 , 2 π ) , U ∼ Unif [ − c , c ] U \sim\operatorname{Unif}[-c,c] U ∼ Unif [ − c , c ] , and ζ ∼ Unif B R 2 ( 0 , δ Y / 16 ) \zeta\sim\operatorname{Unif} B_{\mathbb{R}^2}(0,\delta_Y/16) ζ ∼ Unif B R 2 ( 0 , δ Y /16 ) , and observe
X = ( 1 + U ) q ( Θ ) , Y = f ( q ( Θ ) ) + ζ . X = (1+U)q(\Theta), \qquad Y = f(q(\Theta))+\zeta. X = ( 1 + U ) q ( Θ ) , Y = f ( q ( Θ )) + ζ .
The predictor law fills the annulus of radius c c c around the circle, and both noise variables have positive variance. The link is injective and smooth, with derivative of rank zero exactly at q ( 0 ) q(0) q ( 0 ) and q ( π ) q(\pi) q ( π ) . For the strictly positive choices of c c c , w Y w_Y w Y , δ Y \delta_Y δ Y specified in the verification below, all assumptions of Theorem 3.5 hold. In particular the theorem yields mean tangent error O ~ ( n − 2 / 5 ) \widetilde{O}(n^{-2/5}) O ( n − 2/5 ) and squared prediction error O ~ ( n − 4 / 5 ) \widetilde{O}(n^{-4/5}) O ( n − 4/5 ) for this model.
Verification of Example 1. All constants in this verification are uniform for 0 < c ≤ 1 / 16 0 < c \le1/16 0 < c ≤ 1/16 . The circle has reach one and bounded C 3 C^3 C 3 graph charts, and its base density is constant, π = π − = 1 / ( 2 π ) \pi= \pi_- = 1/(2\pi) π = π − = 1/ ( 2 π ) . The normal law is uniform on the full interval [ − c , c ] [-c,c] [ − c , c ] , so
σ N = σ = c / 3 , E U 4 = c 4 / 5 = ( 9 / 5 ) σ 4 , Var g ( U ) ≤ σ 2 Lip ( g ) 2 . \sigma_N = \sigma= c/\sqrt{3}, \qquad\mathbb{E}U^4 = c^4/5 = (9/5)\sigma^4, \qquad\operatorname{Var} g(U) \le\sigma^2 \operatorname{Lip}(g)^2. σ N = σ = c / 3 , E U 4 = c 4 /5 = ( 9/5 ) σ 4 , Var g ( U ) ≤ σ 2 Lip ( g ) 2 .
The last inequality follows by applying the Lipschitz bound to two independent copies of U U U . Polar transport carries one radial normal line to the next and preserves this law; (14) therefore has left side zero. We may take C N = 2 C_N = 2 C N = 2 .
Write s = sin θ s = \sin\theta s = sin θ . Differentiation of (A.42) gives, in the unit arclength coordinate,
f ′ ( θ ) = ( − s σ 2 + s 2 , cos θ s 2 ( 3 σ 2 + s 2 ) ( σ 2 + s 2 ) 2 ) . f'(\theta) = \left(-\frac{s}{\sqrt{\sigma^2+s^2}}, \cos\theta\frac{s^2(3\sigma^2+s^2)}{(\sigma^2+s^2)^2}\right). f ′ ( θ ) = ( − σ 2 + s 2 s , cos θ ( σ 2 + s 2 ) 2 s 2 ( 3 σ 2 + s 2 ) ) .
The second scalar fraction has maximum 9 / 8 9/8 9/8 , so ∥ f ′ ∥ ≤ 2 \lVert f' \rVert\le2 ∥ f ′ ∥ ≤ 2 . The first coordinate vanishes only at 0 , π 0,\pi 0 , π ; at those points the second coordinate also vanishes and f ′ ′ ( 0 ) = ( − 1 / σ , 0 ) f''(0) = (-1/\sigma,0) f ′′ ( 0 ) = ( − 1/ σ , 0 ) , f ′ ′ ( π ) = ( 1 / σ , 0 ) f''(\pi) = (1/\sigma,0) f ′′ ( π ) = ( 1/ σ , 0 ) . On each open semicircle the first component of f f f is strictly monotone, and the sign of the second distinguishes the semicircles. Thus f f f is injective. Direct differentiation gives ∥ f ′ ′ ∥ ≤ 10 / σ \lVert f'' \rVert\le10/\sigma ∥ f ′′ ∥ ≤ 10/ σ and, within distance σ / 4 \sigma/4 σ /4 of a point where D f Df D f is rank deficient, ∥ f ′ ′ ′ ∥ ≤ 100 / σ 2 \lVert f''' \rVert\le100/\sigma^2 ∥ f ′′′ ∥ ≤ 100/ σ 2 . Take σ f , + = 2 \sigma_{f,+} = 2 σ f , + = 2 , r s i n g = σ / 8 r_{\mathrm{sing}} = \sigma/8 r sing = σ /8 and c o p = 1 / 64 c_{\mathrm{op}} = 1/64 c op = 1/64 . For distance t ≤ σ / 8 t \le\sigma/8 t ≤ σ /8 from a point where D f Df D f is rank deficient, the first derivative component has magnitude at least t / ( 2 σ ) t/(2\sigma) t / ( 2 σ ) ; outside distance σ / 16 \sigma/16 σ /16 its magnitude is at least 1 / 32 1/32 1/32 . These inequalities give (17) with c o p = 1 / 64 c_{\mathrm{op}} = 1/64 c op = 1/64 . The normalized Taylor modulus can be taken as ω ( t ) = 100 t \omega(t) = 100t ω ( t ) = 100 t . In particular all normalized link bounds are independent of c c c , and at either point where D f Df D f is rank deficient ∥ B ∥ g B = 1 \lVert B\rVert_{g_B} = 1 ∥ B ∥ g B = 1 .
The geometric margins can be fixed uniformly before choosing the response resolution. For m = 1 m = 1 m = 1 , C N = 2 C_N = 2 C N = 2 and κ f = 1 \kappa_f = 1 κ f = 1 , Remark A.13 gives a finite bound C ‾ r e g \overline{C}_{\mathrm{reg}} C reg valid for every 0 < c ≤ 1 / 4 0 < c \le1/4 0 < c ≤ 1/4 . Use the uniform chart, design and link bounds just verified to choose η \eta η , H 0 H_0 H 0 , A A A , ε s i n g \varepsilon_{\mathrm{sing}} ε sing as in Appendix § 7 .§ 4.3 , using 1 / 4 1/4 1/4 as an upper bound on the tube ratio. Fix any
0 < c ≤ min { 1 16 , ε s i n g 2 , a ρ 4 8 ( 1 + C ‾ r e g ) } . (81) 0 < c \le\min\left\{\frac{1}{16},\frac{\varepsilon_{\mathrm{sing}}}{2},\frac{a_\rho^4}{8(1+\overline{C}_{\mathrm{reg}})}\right\}. \tag*{(81)} 0 < c ≤ min { 16 1 , 2 ε sing , 8 ( 1 + C reg ) a ρ 4 } . (81)
This is a nonempty interval, since C ‾ r e g \overline{C}_{\mathrm{reg}} C reg and the margins used to choose its endpoints are independent of c c c . The shape operator in a unit radial direction has magnitude one. Hence the left side of (18) is at most c ∥ D f ( q ) ∥ / ( 1 − c ) ≤ ε s i n g λ ( q ) c\lVert Df(q)\rVert/(1-c) \le\varepsilon_{\mathrm{sing}}\lambda(q) c ∥ D f ( q )∥ / ( 1 − c ) ≤ ε sing λ ( q ) , since m = 1 m = 1 m = 1 . Moreover ω b e n d = σ = c / 3 \omega_{\mathrm{bend}} = \sigma= c/\sqrt{3} ω bend = σ = c / 3 , and (A.43) gives (16) . This verifies both curvature restrictions.
Choose a finite scan grid in [ 2.1 c , 2.2 c ] [2.1c, 2.2c] [ 2.1 c , 2.2 c ] , containing both endpoints, and set C p k t = 2 C_{\mathrm{pkt}} = 2 C pkt = 2 . After fixing K 0 K_0 K 0 and the multiplier γ c y l \gamma_{\mathrm{cyl}} γ cyl of the local cylinder, choose ϵ > 0 \epsilon> 0 ϵ > 0 sufficiently small for (A5) and Appendix § 7 .§ 4.3 , with additionally ϵ < 1 / ( 100 C p k t ) \epsilon< 1/(100C_{\mathrm{pkt}}) ϵ < 1/ ( 100 C pkt ) . Put
w Y = w = ϵ σ , δ Y = 2 − 13 w 3 / σ 2 , p 0 = π − δ Y / 4. (82) w_Y = w = \epsilon\sigma,\qquad\delta_Y = 2^{-13}w^3/\sigma^2,\qquad p_0 = \pi-\delta_Y/4. \tag*{(82)} w Y = w = ϵ σ , δ Y = 2 − 13 w 3 / σ 2 , p 0 = π − δ Y /4. (82)
All scan radii satisfy the strict bounds in (A4). Since ∥ D f ∥ ≤ 2 \lVert Df\rVert\le2 ∥ D f ∥ ≤ 2 , every point in the intrinsic interval of radius δ Y / 8 \delta_Y/8 δ Y /8 about a base point satisfies the two inequalities in (19) . Its probability is π − δ Y / 4 = p 0 \pi-\delta_Y/4 = p_0 π − δ Y /4 = p 0 , proving (19) . For the diameter condition, any pair visible to the scan has intrinsic distance d ≤ 8 σ d \le8\sigma d ≤ 8 σ , by the circle chord formula and c ≤ 1 / 16 c \le1/16 c ≤ 1/16 . For these pairs we claim
∥ f ( q ) − f ( q ′ ) ∥ ≥ d 3 / ( 1024 σ 2 ) . (83) \lVert f(q)-f(q')\rVert\ge d^3/(1024\sigma^2). \tag*{(83)} ∥ f ( q ) − f ( q ′ )∥ ≥ d 3 / ( 1024 σ 2 ) . (83)
If the shorter arc avoids the σ \sigma σ -neighborhoods of the two points where D f Df D f is rank deficient, the first response coordinate has derivative of fixed sign and magnitude at least 1 / 2 1/2 1/2 , which proves the claim. Otherwise both endpoints are within 9 σ 9\sigma 9 σ of the same point where D f Df D f is rank deficient. On the same side of that point, integration of ∣ f 1 ′ ( t ) ∣ ≥ ∣ t ∣ / ( 10 σ ) \lvert f_1'(t)\rvert\ge\lvert t\rvert/(10\sigma) ∣ f 1 ′ ( t )∣ ≥ ∣ t ∣ / ( 10 σ ) gives ∣ f 1 ( t ) − f 1 ( t ′ ) ∣ ≥ d 2 / ( 20 σ ) ≥ d 3 / ( 160 σ 2 ) \lvert f_1(t)-f_1(t')\rvert\ge d^2/(20\sigma) \ge d^3/(160\sigma^2) ∣ f 1 ( t ) − f 1 ( t ′ )∣ ≥ d 2 / ( 20 σ ) ≥ d 3 / ( 160 σ 2 ) . On opposite sides, ∣ f 2 ( t ) ∣ ≥ ∣ t ∣ 3 / ( 100 σ 2 ) \lvert f_2(t)\rvert\ge\lvert t\rvert^3/(100\sigma^2) ∣ f 2 ( t )∣ ≥ ∣ t ∣ 3 / ( 100 σ 2 ) and the two signs are opposite; the inequality a 3 + b 3 ≥ ( a + b ) 3 / 4 a^3+b^3 \ge(a+b)^3/4 a 3 + b 3 ≥ ( a + b ) 3 /4 gives d 3 / ( 400 σ 2 ) d^3/(400\sigma^2) d 3 / ( 400 σ 2 ) . These establish (A.45). Thus response distance at most 2 δ Y = 2 − 12 w 3 / σ 2 2\delta_Y = 2^{-12}w^3/\sigma^2 2 δ Y = 2 − 12 w 3 / σ 2 implies d ≤ w ≤ C p k t w d \le w \le C_{\mathrm{pkt}}w d ≤ w ≤ C pkt w , proving (20) .
Finally the independent response noise in Example 1 is centered, bounded by δ Y / 16 \delta_Y/16 δ Y /16 , and sub-Gaussian, with positive covariance ( δ Y / 16 ) 2 I 2 / 4 (\delta_Y/16)^2 I_2/4 ( δ Y /16 ) 2 I 2 /4 . It satisfies (A5). At the chosen resolution, (A.44) gives p 0 = 2 − 15 ϵ 2 π − w > 0 p_0 = 2^{-15}\epsilon^2\pi-w > 0 p 0 = 2 − 15 ϵ 2 π − w > 0 . In units of the local mass π − w \pi-w π − w , the lower bound p 0 p_0 p 0 in (19) decreases as the resolution is refined. It enters the pilot sample-size requirement (A.39); the angular and width multipliers are already fixed. For this fixed model both sample-size conditions hold for all sufficiently large n n n .
pilot resolution, there are such pairs visible to the scan whose intrinsic separation exceeds C p k t w Y C_{\mathrm{pkt}}w_Y C pkt w Y , contradicting (20) for every positive response-window width. Thus ordinary scalar minima and maxima, locally of type f ( 0 ) ± t 2 f(0) \pm t^2 f ( 0 ) ± t 2 , are not covered, including when higher-order terms make f f f asymmetric.
Other response components can distinguish the branches while preserving rank deficiency of D f Df D f . For the local cusp model f ( t ) = ( t 2 , t 3 ) f(t)=(t^2,t^3) f ( t ) = ( t 2 , t 3 ) ,
∥ D f ( t ) ∥ ≲ ∣ t ∣ , ∥ f ( t ) − f ( s ) ∥ ≥ ∣ t − s ∣ 3 / 4. \lVert Df(t)\rVert\lesssim\lvert t\rvert,\qquad\lVert f(t)-f(s)\rVert\ge\lvert t-s\rvert^3/4. ∥ D f ( t )∥ ≲ ∣ t ∣ , ∥ f ( t ) − f ( s )∥ ≥ ∣ t − s ∣ 3 /4.
The second inequality follows from t 2 + t s + s 2 ≥ ( t − s ) 2 / 4 t^2+ts+s^2\ge(t-s)^2/4 t 2 + t s + s 2 ≥ ( t − s ) 2 /4 . Consequently a sufficiently small fixed multiple of w Y 3 w_Y^3 w Y 3 for δ Y \delta_Y δ Y gives the local packet bound, although D f ( 0 ) = 0 Df(0)=0 D f ( 0 ) = 0 . Example 1 verifies all the hypotheses for this type of rank deficiency of D f Df D f on a closed curve with noise. For f ( t ) = ( t 2 , t ) f(t)=(t^2,t) f ( t ) = ( t 2 , t ) , D f ( 0 ) Df(0) D f ( 0 ) instead has full rank: a critical point of one component need not make D f Df D f rank deficient.
Nonisolated points where D f Df D f is rank deficient and higher-order vanishing of its smallest singular value are excluded by (A3); for example, the injective local map t ↦ ( t 3 , t 4 ) t\mapsto(t^3,t^4) t ↦ ( t 3 , t 4 ) has ∥ D f ( t ) ∥ ≲ ∣ t ∣ 2 \lVert Df(t)\rVert\lesssim\lvert t\rvert^2 ∥ D f ( t )∥ ≲ ∣ t ∣ 2 and fails the linear condition (17) .
Remark A.17 (The equality case p = m p=m p = m ) Theorem 3.5 includes p = m p=m p = m formally, but the other assumptions impose the following obstructions. For p = m ≥ 2 p=m\ge2 p = m ≥ 2 , put V = f ( Σ s i n g ) V=f(\Sigma_{\mathrm{sing}}) V = f ( Σ sing ) , a finite set by (A3). The inverse function theorem and compactness imply that f ( M ) ∖ V f(\mathcal{M})\setminus V f ( M ) ∖ V is nonempty and both open and closed in R m ∖ V \mathbb{R}^m\setminus V R m ∖ V . Since the latter is connected, the two sets must coincide, contradicting boundedness of f ( M ) f(\mathcal{M}) f ( M ) . Thus this case is vacuous under (A1) and (A3). For p = m = 1 p=m=1 p = m = 1 , topology permits finitely many critical points, but a scalar link on a closed curve must have extrema, which (A3) makes nondegenerate. Their two local inverse branches have equal responses and violate (A4) at sufficiently fine pilot resolution (Remark A.16), so the scalar equality case is also vacuous in that regime. This conclusion depends on the fixed resolution in (A4), not on topology alone. Both arguments use the absence of boundary in (A1).
Details on the setup of the numerical experiments
The curve experiments
Sample sizes and tuning. In order of dimension D = 18 , 24 , 30 D=18,24,30 D = 18 , 24 , 30 , the single-index experiment uses 11, 9 and 8 sample sizes, with n / 10 5 n/10^5 n /1 0 5 in [ 0.8 , 3.5 ] [0.8,3.5] [ 0.8 , 3.5 ] , [ 1.2 , 4 ] [1.2,4] [ 1.2 , 4 ] and [ 1.6 , 4 ] [1.6,4] [ 1.6 , 4 ] , respectively. The Meyer helix uses 10, 9 and 11 sample sizes with n / 10 5 n/10^5 n /1 0 5 in [ 0.8 , 3 ] [0.8,3] [ 0.8 , 3 ] , [ 1 , 3.5 ] [1,3.5] [ 1 , 3.5 ] and [ 1 , 4 ] [1,4] [ 1 , 4 ] . For k ∗ = ⌈ 2 log n + k 0 ⌉ k_*=\lceil2\log n+k_0\rceil k ∗ = ⌈ 2 log n + k 0 ⌉ , the offsets k 0 k_0 k 0 are ( − 10 , − 10 , − 7 ) (-10,-10,-7) ( − 10 , − 10 , − 7 ) for the segment and ( − 4 , − 4 , − 1 ) (-4,-4,-1) ( − 4 , − 4 , − 1 ) for the helix. At each dimension, the three choices of the prediction multiplier c ρ c_\rho c ρ are spaced by a factor of 2 1 / 3 2^{1/3} 2 1/3 . The middle values at the three dimensions are ( 2 − 5 / 2 , 2 − 14 / 3 , 2 − 14 / 3 ) (2^{-5/2},2^{-14/3},2^{-14/3}) ( 2 − 5/2 , 2 − 14/3 , 2 − 14/3 ) for the segment and ( 1 , 2 2 / 3 , 2 4 / 3 ) (1,2^{2/3},2^{4/3}) ( 1 , 2 2/3 , 2 4/3 ) for the helix.
Meyer helix. We use the construction of Wu and Maggioni (2026, Appendix C). The Meyer helix combines localized envelopes with oscillations whose frequencies and phases vary across coordinates. Set a = 10 a=10 a = 10 , ℓ D , k = ( 1 + 0.3 cos ( a k ) ) / D \ell_{D,k}=(1+0.3\cos(ak))/D ℓ D , k = ( 1 + 0.3 cos ( ak )) / D , ℓ D , k ′ = ( 1 + 0.3 sin ( a k ) ) / D \ell'_{D,k}=(1+0.3\sin(ak))/D ℓ D , k ′ = ( 1 + 0.3 sin ( ak )) / D , and G ( z ) = exp ( − z 2 / ( 1 + z ) ) G(z)=\exp(-z^2/(1+z)) G ( z ) = exp ( − z 2 / ( 1 + z )) . Then
x k ( t ) = 1 2 π 4 ℓ D , k cos ( a k + t − k / D ℓ D , k ′ ) G ( ∣ k / D − t ∣ ℓ D , k ) , k = 1 , … , D , t ∈ [ 0 , 1 ] . (84) x_k(t)=\frac{1}{\sqrt[4]{2\pi}\sqrt{\ell_{D,k}}}\cos\left(ak+\frac{t-k/D}{\ell'_{D,k}}\right)G\left(\frac{\lvert k/D-t\rvert}{\ell_{D,k}}\right),\qquad k=1,\ldots,D,\quad t\in[0,1]. \tag*{(84)} x k ( t ) = 4 2 π ℓ D , k 1 cos ( ak + ℓ D , k ′ t − k / D ) G ( ℓ D , k ∣ k / D − t ∣ ) , k = 1 , … , D , t ∈ [ 0 , 1 ] . (84)
The oscillatory factors produce the curve’s winding geometry. For u = t − k / D u=t-k/D u = t − k / D and ℓ = ℓ D , k \ell=\ell_{D,k} ℓ = ℓ D , k , the envelope satisfies
exp ( − u 2 ℓ ( ℓ + ∣ u ∣ ) ) = 1 − u 2 ℓ 2 + ∣ u ∣ 3 ℓ 3 + O ( u 4 ) . \exp\left(-\frac{u^2}{\ell(\ell+|u|)}\right)=1-\frac{u^2}{\ell^2}+\frac{|u|^3}{\ell^3}+O(u^4). exp ( − ℓ ( ℓ + ∣ u ∣ ) u 2 ) = 1 − ℓ 2 u 2 + ℓ 3 ∣ u ∣ 3 + O ( u 4 ) .
It is C 2 , 1 C^{2,1} C 2 , 1 , but its third derivative jumps at u = 0 u=0 u = 0 . The resulting third-derivative jump of the curve generally has a nonzero normal component, so the curve need not be a C 3 C^3 C 3 manifold, as required by (A1). By scale invariance of the regression problem, we normalize its reach to D \sqrt{D} D ; normal displacements of order one per coordinate remain in the tube, and curvature does not grow with D D D .
After this normalization, the length grows like D 1.5 D^{1.5} D 1.5 , the diameter and the reach like D 0.5 D^{0.5} D 0.5 , the curvature decays like D − 0.5 D^{-0.5} D − 0.5 , and the effective linear dimension grows like D D D . These scaling properties, including the singular-value count defined in § 4.3 , are documented in our earlier paper [102 Appendix C.0.2, Eq. (27) and Fig. 13] .
The bumpy sphere
Neighborhood counts and refinement schedule. For the surface experiments, Phase 1 selects the n Y n_Y n Y observations whose responses are closest to the anchor response in Euclidean distance. Let π Y \pi_Y π Y be the prescribed fraction of responses to retain, k 0 k_0 k 0 the iteration offset, and k h k_h k h the cap on the number of normal-width growth factors. We set
n Y = min { n , max { 234 , ⌈ π Y n ⌉ } } , k ∗ = ⌈ 2 log n + k 0 ⌉ . (85) n_Y=\min\{n,\max\{234,\lceil\pi_Y n\rceil\}\},\qquad k_*=\lceil2\log n+k_0\rceil. \tag*{(85)} n Y = min { n , max { 234 , ⌈ π Y n ⌉}} , k ∗ = ⌈ 2 log n + k 0 ⌉ . (85)
Phase 2 uses
ρ 0 a ρ k − 1 and h 0 a h min { k − 1 , k h } , a ρ = e − 1 / 12 , a h = e 1 / 40 . (86) \rho_0 a_\rho^{k-1}\qquad\text{and}\qquad h_0a_h^{\min\{k-1,k_h\}},\qquad a_\rho=e^{-1/12},\qquad a_h=e^{1/40}. \tag*{(86)} ρ 0 a ρ k − 1 and h 0 a h m i n { k − 1 , k h } , a ρ = e − 1/12 , a h = e 1/40 . (86)
Table 5 gives the parameters selected by validation for each link.
Radial function. Let u ˉ j \bar{u}_j u ˉ j , a j a_j a j , w j w_j w j denote the centers, amplitudes and widths of six broad bumps, and u ^ i \hat{u}_i u ^ i , A i A_i A i , v i v_i v i , o i o_i o i those of 32 smaller bumps, with o i o_i o i their offsets; the values are specified immediately below. For u ∈ S 2 u\in S^2 u ∈ S 2 , the radial function in (4.3) is
r ( u ) = 1 + 0.75 [ 0.035 u 1 − 0.025 u 2 + 0.045 u 3 + ∑ j = 1 6 a j exp ( u T u ˉ j − 1 w j 2 ) + ∑ i = 0 31 A i [ exp ( u T u ^ i − 1 v i 2 ) − o i ] ] . (87) \begin{aligned}
r(u)&=1+0.75\left[0.035u_1-0.025u_2+0.045u_3+\sum_{j=1}^{6}a_j\exp\left(\frac{u^\mathsf{T}\bar{u}_j-1}{w_j^2}\right)\right.\\
&\left.\qquad+\sum_{i=0}^{31}A_i\left[\exp\left(\frac{u^\mathsf{T}\hat{u}_i-1}{v_i^2}\right)-o_i\right]\right]. \tag*{(87)}
\end{aligned} r ( u ) = 1 + 0.75 [ 0.035 u 1 − 0.025 u 2 + 0.045 u 3 + j = 1 ∑ 6 a j exp ( w j 2 u T u ˉ j − 1 ) + i = 0 ∑ 31 A i [ exp ( v i 2 u T u ^ i − 1 ) − o i ] ] . (87)
The six broad terms have the centers u ˉ j = u ~ j / ∥ u ~ j ∥ \bar{u}_j=\tilde{u}_j/\|\tilde{u}_j\| u ˉ j = u ~ j /∥ u ~ j ∥ , amplitudes a j a_j a j and widths w j w_j w j of Table 6 . The 32 smaller terms have centers on a spiral, widths, amplitudes and offsets
Table 6. The six broad terms in (B.4): unnormalized centers, amplitudes and widths.
u ^ i = ( 1 − t i 2 cos ϕ i , 1 − t i 2 sin ϕ i , t i ) , t i = 1 − 2 i + 1 32 , ϕ i = i π ( 3 − 5 ) + 0.73 , \hat{u}_i=\left(\sqrt{1-t_i^2}\cos\phi_i,\sqrt{1-t_i^2}\sin\phi_i,t_i\right),\qquad t_i=1-\frac{2i+1}{32},\qquad\phi_i=i\pi(3-\sqrt{5})+0.73, u ^ i = ( 1 − t i 2 cos ϕ i , 1 − t i 2 sin ϕ i , t i ) , t i = 1 − 32 2 i + 1 , ϕ i = iπ ( 3 − 5 ) + 0.73 ,
v i = 0.26 + 0.055 ( 5 i + 1 ) m o d 8 7 , o i = v i 2 2 ( 1 − e − 2 / v i 2 ) , v_i=0.26+0.055\frac{(5i+1)\mathbin{\bmod}8}{7},\qquad o_i=\frac{v_i^2}{2}\left(1-e^{-2/v_i^2}\right), v i = 0.26 + 0.055 7 ( 5 i + 1 ) mod 8 , o i = 2 v i 2 ( 1 − e − 2/ v i 2 ) ,
A i = 0.36 a ~ i − a ˉ max 0 ≤ k ≤ 31 ∣ a ~ k − a ˉ ∣ , a ~ i = 0.72 sin ( 1.73 i + 0.21 ) + 0.28 cos ( 2.91 i + 0.47 ) , (88) A_i=0.36\frac{\tilde{a}_i-\bar{a}}{\max_{0\leq k\leq31}|\tilde{a}_k-\bar{a}|},\qquad\tilde{a}_i=0.72\sin(1.73i+0.21)+0.28\cos(2.91i+0.47), \tag*{(88)} A i = 0.36 max 0 ≤ k ≤ 31 ∣ a ~ k − a ˉ ∣ a ~ i − a ˉ , a ~ i = 0.72 sin ( 1.73 i + 0.21 ) + 0.28 cos ( 2.91 i + 0.47 ) , (88)
for i = 0 , … , 31 i=0,\ldots,31 i = 0 , … , 31 , where a ˉ = 32 − 1 ∑ i = 0 31 a ~ i \bar a=32^{-1}\sum_{i=0}^{31}\tilde a_i a ˉ = 3 2 − 1 ∑ i = 0 31 a ~ i . The offset o i o_i o i is the mean of the i i i -th exponential over the sphere, so all 32 terms have mean zero. On a fine sphere grid, r r r ranges from 0.688 to 1.401. Each exponential in (B.4) lies in ( 0 , 1 ] (0,1] ( 0 , 1 ] , so the coefficient signs give the uniform bound r ( u ) ≤ 4.26 r(u)\le4.26 r ( u ) ≤ 4.26 for all u u u , used below to verify the full rank of D f q u a d D f^{\mathrm{quad}} D f quad .
Scale, reach and normal displacement. The numerically estimated reach of the unscaled surface { r ( u ) u : u ∈ S 2 } \{r(u)u:u\in S^2\} { r ( u ) u : u ∈ S 2 } is 0.09475149, and the factor c M = 47.19858183760044 c_M=47.19858183760044 c M = 47.19858183760044 in (32) sets it to 20 \sqrt{20} 20 . The scaled surface has radius between 32.5 and 66.1. The reach estimate combines a screen of 65,536 sphere points with continuous searches for the smallest radius of curvature. The local reach reach M ( Q ) \operatorname{reach}_{\mathcal M}(Q) reach M ( Q ) is the largest radius of a ball around Q Q Q in which every point has a unique nearest point on M \mathcal M M , so that reach M = inf Q reach M ( Q ) \operatorname{reach}_{\mathcal M}=\inf_Q\operatorname{reach}_{\mathcal M}(Q) reach M = inf Q reach M ( Q ) . It is estimated from 50,000 scrambled Sobol points of the sphere, using pairwise normal-projection distances and the analytic radii of curvature; at a new point, the estimate interpolates the 32 nearest of these points with weights proportional to the inverse squared distance, and it is capped by the analytic radius of curvature at that point. Let V V V be uniform on S 19 S^{19} S 19 and independent of a χ 2 \chi^2 χ 2 variable G G G with 20 degrees of freedom, conditioned on G ≤ 64.8 G\le64.8 G ≤ 64.8 . The normal displacement is
W = 0.49 reach M ( Q ) 20 G V , Z = P Q ⊥ W . W=\frac{0.49\operatorname{reach}_{\mathcal M}(Q)}{\sqrt{20}}\sqrt{G}V,\qquad Z=P_Q^\perp W. W = 20 0.49 reach M ( Q ) G V , Z = P Q ⊥ W .
Rotation. The rotation in the definition of ξ ( u ) \xi(u) ξ ( u ) is
O = ( 0.04400330497343763 0.23930491044768210 − 0.96994683822725110 − 0.30239263248978127 0.92853374787354440 0.21536892736838617 0.95216721481434900 0.28382783319388520 0.11322259552381468 ) . O=
\begin{pmatrix}
0.04400330497343763 & 0.23930491044768210 & -0.96994683822725110\\
-0.30239263248978127 & 0.92853374787354440 & 0.21536892736838617\\
0.95216721481434900 & 0.28382783319388520 & 0.11322259552381468
\end{pmatrix}. O = 0.04400330497343763 − 0.30239263248978127 0.95216721481434900 0.23930491044768210 0.92853374787354440 0.28382783319388520 − 0.96994683822725110 0.21536892736838617 0.11322259552381468 .
Regularity of the link function. Here σ f , − = inf Q ∈ M σ m ( D f ( Q ) ) \sigma_{f,-}=\inf_{Q\in\mathcal M}\sigma_m(Df(Q)) σ f , − = inf Q ∈ M σ m ( D f ( Q )) and σ f , + = sup Q ∈ M ∥ D f ( Q ) ∥ \sigma_{f,+}=\sup_{Q\in\mathcal M}\|Df(Q)\| σ f , + = sup Q ∈ M ∥ D f ( Q ) ∥ . Since each pair of components in (33) traces a circle of radius 1 / 2 1/2 1/2 , ∥ D f ( Q ) v ∥ = 1 2 ∥ D ( ψ ∘ ξ ) v ∥ \|D f(Q)v\|=\frac{1}{2}\|D(\psi\circ\xi)v\| ∥ D f ( Q ) v ∥ = 2 1 ∥ D ( ψ ∘ ξ ) v ∥ for tangent vectors v v v , where ψ ∘ ξ \psi\circ\xi ψ ∘ ξ is viewed as a function on M \mathcal{M} M . For f l i n f^{\mathrm{lin}} f lin the differential D f ( Q ) Df(Q) D f ( Q ) is therefore a multiple of an isometry, with σ f , − = σ f , + = 8.75 / ( 2 c M ) ≈ 0.0927 \sigma_{f,-}=\sigma_{f,+}=8.75/(2c_{\mathcal{M}})\approx0.0927 σ f , − = σ f , + = 8.75/ ( 2 c M ) ≈ 0.0927 and σ f , + / σ f , − = 1 \sigma_{f,+}/\sigma_{f,-}=1 σ f , + / σ f , − = 1 . For f q u a d f^{\mathrm{quad}} f quad , the derivative matrix of the phase map ψ \psi ψ is 15 I + 2 A ( ξ ) 15I+2A(\xi) 15 I + 2 A ( ξ ) , where A ( ξ ) A(\xi) A ( ξ ) is symmetric with zero diagonal and A i j ( ξ ) = ξ l A_{ij}(\xi)=\xi_l A ij ( ξ ) = ξ l for distinct i , j , l ∈ { 1 , 2 , 3 } i,j,l\in\{1,2,3\} i , j , l ∈ { 1 , 2 , 3 } , so that ∥ A ( ξ ) ∥ ≤ 2 ∥ ξ ∥ \lVert A(\xi)\rVert\le\sqrt{2}\lVert\xi\rVert ∥ A ( ξ )∥ ≤ 2 ∥ ξ ∥ ; since ∥ ξ ( u ) ∥ = r ( u ) ≤ 4.26 \lVert\xi(u)\rVert=r(u)\le4.26 ∥ ξ ( u )∥ = r ( u ) ≤ 4.26 by the bound above, its eigenvalues are at least 15 − 2 2 ⋅ 4.26 > 2.9 15-2\sqrt{2}\cdot4.26>2.9 15 − 2 2 ⋅ 4.26 > 2.9 , and D f q u a d ( Q ) Df^{\mathrm{quad}}(Q) D f quad ( Q ) has full rank. Numerically, σ f , − ≈ 0.131 \sigma_{f,-}\approx0.131 σ f , − ≈ 0.131 , σ f , + ≈ 0.187 \sigma_{f,+}\approx0.187 σ f , + ≈ 0.187 and σ f , + / σ f , − ≈ 1.43 \sigma_{f,+}/\sigma_{f,-}\approx1.43 σ f , + / σ f , − ≈ 1.43 for f q u a d f^{\mathrm{quad}} f quad .
Response noise. Let Σ ^ \widehat{\Sigma} Σ be the covariance of the noiseless responses of the first min { n , 10 6 } \min\{n,10^6\} min { n , 1 0 6 } training samples. Training and validation responses carry Gaussian noise ζ ∼ N ( 0 , 10 − 4 Σ ^ ) \zeta\sim\mathcal{N}(0,10^{-4}\widehat{\Sigma}) ζ ∼ N ( 0 , 1 0 − 4 Σ ) .
Phase 1. The response-neighbor count and the number of Phase 2 steps are (B.2). These observations are sorted by predictor distance to the anchor. The scan uses 128 truncation positions, from 3 D 3D 3 D to ⌊ 0.8 n Y ⌋ \lfloor0.8n_Y\rfloor ⌊ 0.8 n Y ⌋ . For D = 20 D=20 D = 20 , the lower bound 234 = max { 9 D , ⌈ ( 3 D + 127 ) / 0.8 ⌉ } 234=\max\{9D,\lceil(3D+127)/0.8\rceil\} 234 = max { 9 D , ⌈( 3 D + 127 ) /0.8 ⌉} ensures 128 distinct positions. For surfaces, candidate subspaces are compared on two halves of the Phase 1 data. The returned radius is the root-mean-square scale of the selected covariance. The initial width is h 0 = h r e s c a l e r m e d h_0=h_{\mathrm{rescale}}r_{\mathrm{med}} h 0 = h rescale r med , where r m e d r_{\mathrm{med}} r med is the median of the positive returned radii over the retained anchors, floored at 10 − 6 10^{-6} 1 0 − 6 .
Refinement and prediction. The aspect ratio and normal width at step k k k are (B.3), with the constants of Table 5 .
Tangent-space error. We orthonormalize each estimated basis with two passes of modified Gram–Schmidt in double precision before computing its tangent-space error.
Selection of the parameters. The validation search evaluated 78 parameter vectors for f l i n f^{\mathrm{lin}} f lin and 66 for f q u a d f^{\mathrm{quad}} f quad at the sample sizes A = { 1 , 5 , 9 } × 10 k \mathcal{A}=\{1,5,9\}\times10^k A = { 1 , 5 , 9 } × 1 0 k , k = 3 , 4 , 5 k=3,4,5 k = 3 , 4 , 5 . Let L n ( p ) L_n(\mathbf{p}) L n ( p ) be the validation relative mean squared error of the parameter vector p \mathbf{p} p at sample size n n n , and L n ∗ L_n^* L n ∗ the smallest value over the tested vectors. The selected vector minimizes
max n ∈ A { L n ( p ) L n ∗ − 1 } \max_{n\in\mathcal{A}}\left\{\frac{L_n(\mathbf{p})}{L_n^*}-1\right\} n ∈ A max { L n ∗ L n ( p ) − 1 }
with ties broken by the geometric mean of L n ( p ) / L n ∗ L_n(\mathbf{p})/L_n^* L n ( p ) / L n ∗ over A \mathcal{A} A .
The diffusion
Potential. Let q 1 , … , q 4 q_1,\ldots,q_4 q 1 , … , q 4 be the vectors in Table 7 . The tangential term in (38) is
Table 7. The vectors of the tangential potential U t a n U_{\mathrm{tan}} U tan in (B.5).
U t a n ( u ) = − 1 10 ∑ j = 1 4 e ( 0.425 + 0.17 j ) q j T u , (89) U_{\mathrm{tan}}(u)=-\frac{1}{10}\sum_{j=1}^{4}e^{(0.425+0.17j)q_j^{\mathsf{T}}u}, \tag*{(89)} U tan ( u ) = − 10 1 j = 1 ∑ 4 e ( 0.425 + 0.17 j ) q j T u , (89)
The first three vectors have length 31 / 6 ≈ 0.928 \sqrt{31}/6\approx0.928 31 /6 ≈ 0.928 and are not normalized. With z i = x i + 3 z_i=x_{i+3} z i = x i + 3 , the stiffnesses in (38) are
ν 0 = 736 15 , ν 1 = ⋯ = ν 4 = 3680 81 , ν 5 = ⋯ = ν 10 = 115 3 , ν 11 = ⋯ = ν 17 = 14720 343 , \nu_0=\frac{736}{15},\qquad\nu_1=\cdots=\nu_4=\frac{3680}{81},\qquad\nu_5=\cdots=\nu_{10}=\frac{115}{3},\qquad\nu_{11}=\cdots=\nu_{17}=\frac{14720}{343}, ν 0 = 15 736 , ν 1 = ⋯ = ν 4 = 81 3680 , ν 5 = ⋯ = ν 10 = 3 115 , ν 11 = ⋯ = ν 17 = 343 14720 ,
approximately 49.07, 45.43, 38.33 and 42.92. The normal coordinates therefore relax on time scales between 1 / ν 0 ≈ 0.020 1/\nu_0\approx0.020 1/ ν 0 ≈ 0.020 and 1 / ν 5 ≈ 0.026 1/\nu_5\approx0.026 1/ ν 5 ≈ 0.026 .
Targets. For u ∈ S 2 u\in S^2 u ∈ S 2 , let λ ( u ) ∈ ( − π , π ] \lambda(u)\in(-\pi,\pi] λ ( u ) ∈ ( − π , π ] and φ ( u ) ∈ [ − π / 2 , π / 2 ] \varphi(u)\in[-\pi/2,\pi/2] φ ( u ) ∈ [ − π /2 , π /2 ] denote longitude and latitude, so u = ( cos φ cos λ , cos φ sin λ , sin φ ) u=(\cos\varphi\cos\lambda,\cos\varphi\sin\lambda,\sin\varphi) u = ( cos φ cos λ , cos φ sin λ , sin φ ) . For x = ( r u , z ) x=(ru,z) x = ( r u , z ) (§ 4.6 ), the targets are
A j = { x : u ∈ D j , ∣ r − 1 ∣ < 0.8 , ∥ z ∥ < 0.8 } , j = 1 , … , 4 , A_j=\left\{x:u\in D_j,\ \lvert r-1\rvert<0.8,\ \lVert z\rVert<0.8\right\},\qquad j=1,\ldots,4, A j = { x : u ∈ D j , ∣ r − 1 ∣ < 0.8 , ∥ z ∥ < 0.8 } , j = 1 , … , 4 ,
with the angular parts
D j = { u : ∣ λ ( u ) − 2 π ( j − 2 ) 3 ∣ ≤ π 6 , − arctan 1 2 ≤ φ ( u ) ≤ 0 } , j = 1 , 2 , 3 , D_j=\left\{u:\left\lvert\lambda(u)-\frac{2\pi(j-2)}{3}\right\rvert\leq\frac{\pi}{6},\ -\arctan\frac{1}{2}\leq\varphi(u)\leq0\right\},\qquad j=1,2,3, D j = { u : λ ( u ) − 3 2 π ( j − 2 ) ≤ 6 π , − arctan 2 1 ≤ φ ( u ) ≤ 0 } , j = 1 , 2 , 3 ,
D 4 = { u : u 3 > cos 0.2 } . D_4=\{u:u_3>\cos0.2\}. D 4 = { u : u 3 > cos 0.2 } .
Simulation. The paths of (37) are simulated by the Euler–Maruyama scheme with time step 0.002 and 50,000 steps, which gives the horizon 100.
Selection of the parameters. Validation at n = 10 6 n=10^6 n = 1 0 6 selected the constants in two rounds. The first compared the 24 combinations of π Y ∈ { 0.005 , 0.010 , 0.020 } \pi_Y\in\{0.005,0.010,0.020\} π Y ∈ { 0.005 , 0.010 , 0.020 } , h r e s c a l e ∈ { 2.45 , 3.50 } h_{\mathrm{rescale}}\in\{2.45,3.50\} h rescale ∈ { 2.45 , 3.50 } , ρ 0 ∈ { 0.5 , 1 } \rho_0\in\{0.5,1\} ρ 0 ∈ { 0.5 , 1 } and k h ∈ { 2 , 4 } k_h\in\{2,4\} k h ∈ { 2 , 4 } , with k 0 = − 16 k_0=-16 k 0 = − 16 and c ρ = c h = 0.25 c_\rho=c_h=0.25 c ρ = c h = 0.25 . The second round kept the selected values of these four constants and compared the 12 combinations of k 0 ∈ { − 20 , − 16 , − 12 } k_0\in\{-20,-16,-12\} k 0 ∈ { − 20 , − 16 , − 12 } , c ρ ∈ { 0.25 , 0.5 } c_\rho\in\{0.25,0.5\} c ρ ∈ { 0.25 , 0.5 } and c h ∈ { 0.25 , 0.5 } c_h\in\{0.25,0.5\} c h ∈ { 0.25 , 0.5 } . The selected constants are
( π Y , k 0 , h r e s c a l e , ρ 0 , k h , c ρ , c h ) = ( 0.005 , − 12 , 2.45 , 0.5 , 2 , 0.5 , 0.25 ) , (\pi_Y,k_0,h_{\mathrm{rescale}},\rho_0,k_h,c_\rho,c_h)=(0.005,-12,2.45,0.5,2,0.5,0.25), ( π Y , k 0 , h rescale , ρ 0 , k h , c ρ , c h ) = ( 0.005 , − 12 , 2.45 , 0.5 , 2 , 0.5 , 0.25 ) ,
and are used without change at n = 4 × 10 5 n=4\times10^5 n = 4 × 1 0 5 , giving n Y = 2000 n_Y=2000 n Y = 2000 observations per response preimage and k ∗ = 14 k_*=14 k ∗ = 14 Phase 2 steps.
The coupled Meyer helix surface
Surface. For d ∈ { 4 , 5 } d \in\{4,5\} d ∈ { 4 , 5 } and i = 1 , … , d i=1,\ldots,d i = 1 , … , d , the envelopes and phases in (34) are
c i ( d ) = i − 1 d − 1 , w i ( d ) = 1 + 0.3 cos ( 10 i ) d , v i ( d ) = 1 + 0.3 sin ( 10 i ) d , c_i^{(d)}=\frac{i-1}{d-1},\qquad w_i^{(d)}=\frac{1+0.3\cos(10i)}{d},\qquad v_i^{(d)}=\frac{1+0.3\sin(10i)}{d}, c i ( d ) = d − 1 i − 1 , w i ( d ) = d 1 + 0.3 cos ( 10 i ) , v i ( d ) = d 1 + 0.3 sin ( 10 i ) ,
E i ( d ) ( u ) = exp { − ( u − c i ( d ) ) 2 2 w i ( d ) ( 2 w i ( d ) + ∣ u − c i ( d ) ∣ ) } ( 2 π ) 1 / 4 ( w i ( d ) ) 1 / 4 , ϕ i ( d ) ( u ) = 10 i + u − c i ( d ) v i ( d ) . E_i^{(d)}(u)=\frac{\exp\left\{-\frac{(u-c_i^{(d)})^2}{2w_i^{(d)}(2w_i^{(d)}+\lvert u-c_i^{(d)}\rvert)}\right\}}{(2\pi)^{1/4}(w_i^{(d)})^{1/4}},\qquad\phi_i^{(d)}(u)=10i+\frac{u-c_i^{(d)}}{v_i^{(d)}}. E i ( d ) ( u ) = ( 2 π ) 1/4 ( w i ( d ) ) 1/4 exp { − 2 w i ( d ) ( 2 w i ( d ) + ∣ u − c i ( d ) ∣) ( u − c i ( d ) ) 2 } , ϕ i ( d ) ( u ) = 10 i + v i ( d ) u − c i ( d ) .
At an envelope center, writing w = w i ( d ) w=w_i^{(d)} w = w i ( d ) and c i = c i ( d ) c_i=c_i^{(d)} c i = c i ( d ) ,
E i ( d ) ( c i + u ) = E i ( d ) ( c i ) [ 1 − u 2 4 w 2 + ∣ u ∣ 3 8 w 3 + O ( u 4 ) ] . E_i^{(d)}(c_i+u)=E_i^{(d)}(c_i)\left[1-\frac{u^2}{4w^2}+\frac{\lvert u\rvert^3}{8w^3}+O(u^4)\right]. E i ( d ) ( c i + u ) = E i ( d ) ( c i ) [ 1 − 4 w 2 u 2 + 8 w 3 ∣ u ∣ 3 + O ( u 4 ) ] .
As for the curve in Appendix B.1, these envelopes are C 2 , 1 C^{2,1} C 2 , 1 and generally fail to give the C 3 C^3 C 3 geometry in (A1). The coordinates are ordered first by i i i , then by j j j . The vector μ M \mu_M μ M is the mean of G G G on a fixed 100,000-point calibration sample, and c M = 74.10524209977933 c_M=74.10524209977933 c M = 74.10524209977933 . Sampling ( s , t ) (s,t) ( s , t ) uniformly on [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 gives uniform parameter area. The resulting patch has boundary.
Local reach calibration and normal displacement. At calibration point Q Q Q with analytic P Q P_Q P Q , use the minimum of the curvature radius and tangent–chord quotients
∥ Q ′ − Q ∥ 2 2 ∥ ( I − P Q ) ( Q ′ − Q ) ∥ , \frac{\lVert Q'-Q\rVert^2}{2\lVert(I-P_Q)(Q'-Q)\rVert}, 2 ∥( I − P Q ) ( Q ′ − Q )∥ ∥ Q ′ − Q ∥ 2 ,
over calibration points Q ′ Q' Q ′ at parameter distance at least 0.02 from $Q. In an orthonormal tangent frame, let ∥ I I Q ∥ F \lVert\mathrm{II}_Q\rVert_F ∥ II Q ∥ F be the Frobenius norm of the second fundamental form and put e j = ( cos ( j π / 128 ) , sin ( j π / 128 ) ) e_j=(\cos(j\pi/128),\sin(j\pi/128)) e j = ( cos ( j π /128 ) , sin ( j π /128 )) , $0\le j<128$. The curvature radius is estimated by
[ max 0 ≤ j < 128 ∥ I I Q ( e j , e j ) ∥ + π 128 ∥ I I Q ∥ F ] − 1 . \left[\max_{0\le j<128}\lVert\mathrm{II}_Q(e_j,e_j)\rVert+\frac{\pi}{128}\lVert\mathrm{II}_Q\rVert_F\right]^{-1}. [ 0 ≤ j < 128 max ∥ II Q ( e j , e j )∥ + 128 π ∥ II Q ∥ F ] − 1 .
The scaling makes the smallest calibrated value equal to 20 \sqrt{20} 20 ; the minimum, median and maximum are 4.472, 30.448 and 63.784. At a new parameter point, we interpolate the 32 nearest calibration values with weights proportional to inverse squared parameter distance, using distance floor 10 − 12 10^{-12} 1 0 − 12 , and cap the result by the curvature radius there. Write ℓ ( Q ) \ell(Q) ℓ ( Q ) for this numerical local-reach estimate. The calibration is fixed across all repetitions.
Draw W ∼ N ( 0 , I 20 ) W\sim\mathcal{N}(0,I_{20}) W ∼ N ( 0 , I 20 ) conditional on ∥ W ∥ ≤ 1.8 20 \lVert W\rVert\le1.8\sqrt{20} ∥ W ∥ ≤ 1.8 20 , independently of Q Q Q , and set
Z = 0.49 ℓ ( Q ) 20 ( I − P Q ) W , X = Q + Z . (90) Z=\frac{0.49\ell(Q)}{\sqrt{20}}(I-P_Q)W,\qquad X=Q+Z. \tag*{(90)} Z = 20 0.49 ℓ ( Q ) ( I − P Q ) W , X = Q + Z . (90)
Thus ∥ Z ∥ ≤ 0.882 ℓ ( Q ) \lVert Z\rVert\le0.882\ell(Q) ∥ Z ∥ ≤ 0.882 ℓ ( Q ) .
Polynomial link function and noise. The normalization constants in (4.6) are
a Y = 0.31029722321272146 , μ Y = ( − 0.058782804805174 0.013768230206499 0.916843578667272 ) . a_Y = 0.31029722321272146,\qquad\mu_Y = \begin{pmatrix} -0.058782804805174 \\ 0.013768230206499 \\ 0.916843578667272 \end{pmatrix}. a Y = 0.31029722321272146 , μ Y = − 0.058782804805174 0.013768230206499 0.916843578667272 .
On the fixed calibration sample, the covariance of the normalized response is
Σ f ≃ ( 0.0478891 0.000157187 − 0.0283818 0.000157187 0.0480453 0.00918279 − 0.0283818 0.00918279 0.9040656 ) , tr ( Σ f ) = 1. \Sigma_f \simeq\begin{pmatrix} 0.0478891 & 0.000157187 & -0.0283818 \\ 0.000157187 & 0.0480453 & 0.00918279 \\ -0.0283818 & 0.00918279 & 0.9040656 \end{pmatrix},\qquad\operatorname{tr}(\Sigma_f)=1. Σ f ≃ 0.0478891 0.000157187 − 0.0283818 0.000157187 0.0480453 0.00918279 − 0.0283818 0.00918279 0.9040656 , tr ( Σ f ) = 1.
Training and validation responses are Y = f p o l y ( Q ) + ζ Y=f^{\mathrm{poly}}(Q)+\zeta Y = f poly ( Q ) + ζ , where ζ ∼ N ( 0 , 10 − 4 Σ f ) \zeta\sim\mathcal{N}(0,10^{-4}\Sigma_f) ζ ∼ N ( 0 , 1 0 − 4 Σ f ) is independent of X X X ; testing uses the noiseless response. Both zeros of p p p and all zeros of p ′ ( z ) = z 6 ( 8 z + 4.2 + 1.4 i ) p'(z)=z^6(8z+4.2+1.4i) p ′ ( z ) = z 6 ( 8 z + 4.2 + 1.4 i ) lie outside the parameter rectangle, so the feature differential in (4.6) has rank two. Inverting p p p at calibration responses yields one to four parameter preimages.
Link function constructed from a rational map. For f r a t f^{\mathrm{rat}} f rat , replace a Y , μ Y a_Y,\mu_Y a Y , μ Y in (4.6) by
a r a t = 0.1353656776435608 , μ r a t = ( − 0.00308426830608437 − 0.00465022983878112 0.8002766360491473 ) . a_{\mathrm{rat}}=0.1353656776435608,\qquad\mu_{\mathrm{rat}}=\begin{pmatrix} -0.00308426830608437 \\ -0.00465022983878112 \\ 0.8002766360491473 \end{pmatrix}. a rat = 0.1353656776435608 , μ rat = − 0.00308426830608437 − 0.00465022983878112 0.8002766360491473 .
On the same fixed calibration sample, the normalized response has covariance
Σ r a t ≃ ( 0.00920537 0.0000133033 0.00123311 0.0000133033 0.00911811 0.000516827 0.00123311 0.000516827 0.98167652 ) , tr ( Σ r a t ) = 1. \Sigma_{\mathrm{rat}}\simeq\begin{pmatrix} 0.00920537 & 0.0000133033 & 0.00123311 \\ 0.0000133033 & 0.00911811 & 0.000516827 \\ 0.00123311 & 0.000516827 & 0.98167652 \end{pmatrix},\qquad\operatorname{tr}(\Sigma_{\mathrm{rat}})=1. Σ rat ≃ 0.00920537 0.0000133033 0.00123311 0.0000133033 0.00911811 0.000516827 0.00123311 0.000516827 0.98167652 , tr ( Σ rat ) = 1.
Training and validation responses are Y = f r a t ( Q ) + ζ Y=f^{\mathrm{rat}}(Q)+\zeta Y = f rat ( Q ) + ζ , with ζ ∼ N ( 0 , 10 − 4 Σ r a t ) \zeta\sim\mathcal{N}(0,10^{-4}\Sigma_{\mathrm{rat}}) ζ ∼ N ( 0 , 1 0 − 4 Σ rat ) independent of X X X ; testing uses the noiseless response. On the parameter rectangle, w w w in (4.7) has no zero or pole and w ′ ( z ) ≠ 0 w'(z)\ne0 w ′ ( z ) = 0 , so the response differential has rank two. Inverting w w w at calibration responses gives one to four parameter preimages. Figure 17 shows the response components.
Parameters and validation. The implementation uses counts and schedule (B.2)–(B.3) and the constants in Table 8 . The initial normal width is h r e s c a l e h_{\mathrm{rescale}} h rescale times the median positive Phase 1 radius; prediction scales are c ρ , c h c_\rho,c_h c ρ , c h times the final aspect ratio and normal width.
Table 8. Selected parameters for the two link functions on the coupled Meyer helix surface. Each set is fixed across all sample sizes and repetitions.
For each link function, the final comparison used six parameter vectors and independent validation sets of 5000 queries at each n ∈ { 1 , 2 , 3 , 4 , 5 , 7 } × { 10 3 , 10 4 , 10 5 } n \in\{1,2,3,4,5,7\} \times\{10^3,10^4,10^5\} n ∈ { 1 , 2 , 3 , 4 , 5 , 7 } × { 1 0 3 , 1 0 4 , 1 0 5 } . For each sample size, we divide a candidate’s validation relative mean squared error by the smallest error among the six candidates. The selected candidate minimizes the largest ratio across sample sizes, with ties broken by the geometric mean of the ratios.
Evaluation. Prediction error is the mean squared Euclidean error divided by the sum of the component sample variances of the 5000 noiseless test responses (divisor 4999). Estimated tangent bases are orthonormalized before computing (30) . Error bars are sample standard deviations across ten repetitions (divisor 9). The slopes are unweighted least-squares fits of log arithmetic mean error against log n n n over the eight displayed sample sizes n ≥ 5 × 10 5 n \ge5 \times10^5 n ≥ 5 × 1 0 5 .
Statistical learnability and minimax rate
We show that regression functions F = f ∘ Π M F = f \circ\Pi_{\mathcal{M}} F = f ∘ Π M , with M \mathcal{M} M as regular as in (A1) and f f f as regular as in (A3), are learnable at the rate n − 4 / ( m + 4 ) n^{-4/(m+4)} n − 4/ ( m + 4 ) of Theorem 3.5 , with constants linear in D + p D+p D + p , and that its exponent in n n n is minimax for the class below, at fixed positive noise. The upper bound uses neither (A2) nor (A4), of (A5) only the sub-Gaussian noise bound, and no rank condition on D f D f D f . Its estimator is a least-squares fit over a finite family whose cardinality is bounded exponentially in a power of n n n ; the construction is an existence argument, and it uses standard arguments from learning theory, borrowing from standard complexity arguments for learning in the function class ([45 , 105 ] ), and perhaps slightly less standard but rather well-established arguments for studying the complexity of having to also estimate the manifold well enough to be able to attain the desired rate for regression ([42 , 1 ] ). We do not know whether an estimator computable in polynomial time attains the same bound under these assumptions alone; MANOR attains the rate in polynomial time under (A1)–(A5).
The class and the two bounds
Fix integers m < D m < D m < D and p p p , a tube radius R > 0 R > 0 R > 0 , a ratio 0 < c < 1 0 < c < 1 0 < c < 1 and the unit σ N = R / D − m + 2 \sigma_N = R/\sqrt{D-m+2} σ N = R / D − m + 2 , of the order prescribed in (A2). Fix a regularity constant C F ≥ 1 C_F \ge1 C F ≥ 1 , a volume bound v max v_{\max} v m a x , an ambient radius ϱ \varrho ϱ and a response bound B f B_f B f . Let M \mathfrak{M} M be the set of compact m m m -dimensional C 3 C^3 C 3 submanifolds M ⊂ B D ( ϱ ) \mathcal{M} \subset B_D(\varrho) M ⊂ B D ( ϱ ) without boundary, with reach M ≥ R / c \operatorname{reach}_{\mathcal{M}} \ge R/c reach M ≥ R / c and vol ( M ) ≤ v max \operatorname{vol}(\mathcal{M}) \le v_{\max} vol ( M ) ≤ v m a x , whose graph charts of radius R / ( 5 c ) R/(5c) R / ( 5 c ) , written as in (A1), satisfy ∥ D g ∥ ≤ 1 / 4 \lVert Dg\rVert\le1/4 ∥ D g ∥ ≤ 1/4 and
( R / c ) ∥ D 2 g ∥ + ( R / c ) 2 ∥ D 3 g ∥ ≤ C F . (R/c)\lVert D^2g\rVert+ (R/c)^2\lVert D^3g\rVert\le C_F. ( R / c ) ∥ D 2 g ∥ + ( R / c ) 2 ∥ D 3 g ∥ ≤ C F .
For M ∈ M \mathcal{M} \in\mathfrak{M} M ∈ M let L ( M ) \mathcal{L}(\mathcal{M}) L ( M ) be the set of C 1 , 1 C^{1,1} C 1 , 1 links f : M → R p f:\mathcal{M} \to\mathbb{R}^p f : M → R p with sup M ∥ f ∥ ≤ B f \sup_{\mathcal{M}}\lVert f\rVert\le B_f sup M ∥ f ∥ ≤ B f and ∥ D f ∥ ≤ σ f , + \lVert Df\rVert\le\sigma_{f,+} ∥ D f ∥ ≤ σ f , + whose differential is Lipschitz in parallel frames with constant at most C F σ f , + / σ N C_F\sigma_{f,+}/\sigma_N C F σ f , + / σ N . A manifold as in (A1) with R = c reach M R = c\operatorname{reach}_{\mathcal{M}} R = c reach M and a link as in (A3) satisfy these conditions for suitable values of C F C_F C F , v max v_{\max} v m a x , ϱ \varrho ϱ and B f B_f B f . Write M R = { x : dist ( x , M ) ≤ R } \mathcal{M}_R = \{x : \operatorname{dist}(x,\mathcal{M}) \le R\} M R = { x : dist ( x , M ) ≤ R } for the tube and
F = { ( M , F ) : M ∈ M , F = f ∘ Π M on M R , f ∈ L ( M ) } . \mathcal{F} = \{(\mathcal{M},F): \mathcal{M} \in\mathfrak{M},\ F = f \circ\Pi_{\mathcal{M}}\ \text{on}\ \mathcal{M}_R,\ f \in\mathcal{L}(\mathcal{M})\}. F = {( M , F ) : M ∈ M , F = f ∘ Π M on M R , f ∈ L ( M )} .
We assume M ≠ ∅ \mathfrak{M} \ne\varnothing M = ∅ . Then ϱ ≥ R / c \varrho\ge R/c ϱ ≥ R / c , since a compact submanifold of B D ( ϱ ) B_D(\varrho) B D ( ϱ ) has reach at most ϱ \varrho ϱ , and v max ≥ c m ( R / c ) m ≥ c m σ N m v_{\max} \ge c_m(R/c)^m \ge c_m\sigma_N^m v m a x ≥ c m ( R / c ) m ≥ c m σ N m , since a compact manifold without boundary and with reach at least R / c R/c R / c has volume at least c m ( R / c ) m c_m(R/c)^m c m ( R / c ) m [82 ] . We observe ( X i , Y i ) (X_i,Y_i) ( X i , Y i ) , i ≤ n i \le n i ≤ n , where X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n are independent copies of a random vector X ∈ M R X \in\mathcal{M}_R X ∈ M R with otherwise arbitrary law, Y i = F ( X i ) + ζ i Y_i = F(X_i) + \zeta_i Y i = F ( X i ) + ζ i , and the noise vectors ζ i \zeta_i ζ i are centered, independent of one another and of the predictors, and satisfy ∥ ⟨ v , ζ i ⟩ ∥ ψ 2 ≤ σ ζ \lVert\langle v,\zeta_i\rangle\rVert_{\psi_2} \le\sigma_\zeta ∥⟨ v , ζ i ⟩ ∥ ψ 2 ≤ σ ζ for unit v v v . Put
Λ = 1 + σ ζ 2 + B f 2 ( σ f , + σ N ) 2 , ℓ n = log ( e n max { 1 , ϱ σ N , B f σ f , + σ N } ) . \Lambda= 1 + \frac{\sigma_\zeta^2 + B_f^2}{(\sigma_{f,+}\sigma_N)^2},\qquad\ell_n = \log\left(e n \max\left\{1,\frac{\varrho}{\sigma_N},\frac{B_f}{\sigma_{f,+}\sigma_N}\right\}\right). Λ = 1 + ( σ f , + σ N ) 2 σ ζ 2 + B f 2 , ℓ n = log ( e n max { 1 , σ N ϱ , σ f , + σ N B f } ) .
Proposition C.1 (Upper bound) There are an estimator F ^ L S \widehat{F}_{\mathrm{LS}} F LS and a constant C C C depending only on m m m , c c c and C F C_F C F such that, for every ( M , F ) ∈ F (\mathcal{M},F) \in\mathcal{F} ( M , F ) ∈ F , every law of X X X on M R \mathcal{M}_R M R and every n ≥ Λ ( D + p ) v max / σ N m n \ge\Lambda(D+p)v_{\max}/\sigma_N^m n ≥ Λ ( D + p ) v m a x / σ N m ,
E ∥ F ^ L S ( X ) − F ( X ) ∥ 2 ( σ f , + σ N ) 2 ≤ C ( Λ ( D + p ) v max / σ N m n ) 4 / ( m + 4 ) ℓ n . \frac{\mathbb{E}\lVert\widehat{F}_{\mathrm{LS}}(X)-F(X)\rVert^2}{(\sigma_{f,+}\sigma_N)^2}
\le C\left(\frac{\Lambda(D+p)v_{\max}/\sigma_N^m}{n}\right)^{4/(m+4)}\ell_n. ( σ f , + σ N ) 2 E ∥ F LS ( X ) − F ( X ) ∥ 2 ≤ C ( n Λ ( D + p ) v m a x / σ N m ) 4/ ( m + 4 ) ℓ n .
Proposition C.2 (Lower bound) Let M 0 ∈ M \mathcal{M}_0 \in\mathfrak{M} M 0 ∈ M have volume v 0 v_0 v 0 , and suppose B f ≥ σ f , + σ N B_f \ge\sigma_{f,+}\sigma_N B f ≥ σ f , + σ N . There are constants c 0 , n 1 > 0 c_0,n_1 > 0 c 0 , n 1 > 0 depending only on m m m , c c c and C F C_F C F such that, for n ≥ n 1 ( σ ζ / ( σ f , + σ N ) ) 2 v 0 / σ N m n \ge n_1(\sigma_\zeta/(\sigma_{f,+}\sigma_N))^2v_0/\sigma_N^m n ≥ n 1 ( σ ζ / ( σ f , + σ N ) ) 2 v 0 / σ N m , every estimator F ^ \widehat{F} F satisfies
sup E ∥ F ^ ( X ) − F ( X ) ∥ 2 ( σ f , + σ N ) 2 ≥ c 0 ( σ ζ 2 ( σ f , + σ N ) 2 v 0 / σ N m n ) 4 / ( m + 4 ) , \sup\frac{\mathbb{E}\lVert\widehat{F}(X)-F(X)\rVert^2}{(\sigma_{f,+}\sigma_N)^2}
\ge c_0\left(\frac{\sigma_\zeta^2}{(\sigma_{f,+}\sigma_N)^2}\frac{v_0/\sigma_N^m}{n}\right)^{4/(m+4)}, sup ( σ f , + σ N ) 2 E ∥ F ( X ) − F ( X ) ∥ 2 ≥ c 0 ( ( σ f , + σ N ) 2 σ ζ 2 n v 0 / σ N m ) 4/ ( m + 4 ) , where the supremum is over ( M , F ) ∈ F (\mathcal{M},F) \in\mathcal{F} ( M , F ) ∈ F , the laws of X X X on M R \mathcal{M}_R M R and the admissible noise laws.
For fixed positive noise, the two bounds match in their exponents in n n n and in the inverse local mass v max / σ N m v_{\max}/\sigma_N^m v m a x / σ N m , up to the logarithm ℓ n \ell_n ℓ n , when v 0 ≍ v max v_0 \asymp v_{\max} v 0 ≍ v m a x . The upper bound contains Λ \Lambda Λ in place of σ ζ 2 / ( σ f , + σ N ) 2 \sigma_{\zeta}^{2}/(\sigma_{f,+}\sigma_{N})^{2} σ ζ 2 / ( σ f , + σ N ) 2 ; the noise dependence therefore matches when σ ζ 2 ≳ ( σ f , + σ N ) 2 + B f 2 \sigma_{\zeta}^{2} \gtrsim(\sigma_{f,+}\sigma_{N})^{2} + B_{f}^{2} σ ζ 2 ≳ ( σ f , + σ N ) 2 + B f 2 , but not uniformly as σ ζ → 0 \sigma_{\zeta} \to0 σ ζ → 0 . In particular, the lower bound is vacuous at zero noise. The factor p p p is matched in the noise-dominated regime: the construction of Appendix C.3, with independent bumps in each of the p p p response coordinates and amplitudes divided by p \sqrt{p} p , gives the lower bound with p σ ζ 2 p\sigma_{\zeta}^{2} p σ ζ 2 in place of σ ζ 2 \sigma_{\zeta}^{2} σ ζ 2 , provided the sample threshold in Proposition C.2 is also multiplied by p p p . The factor D 4 / ( m + 4 ) D^{4/(m+4)} D 4/ ( m + 4 ) remains unmatched: a fixed manifold cannot detect the ambient dimension, so Proposition C.2 does not address the sharpness of the dependence on D D D . Compared with Theorem 3.5, Proposition C.1 has the same exponent in n n n and in the inverse local mass, since vol ( M ) / σ N m ≤ 1 / ( π − σ N m ) \operatorname{vol}(\mathcal{M})/\sigma_{N}^{m} \leq1/(\pi-\sigma_{N}^{m}) vol ( M ) / σ N m ≤ 1/ ( π − σ N m ) . Inside that power it has the factor D + p D+p D + p where (A.40) has ( 1 + D + p ) 4 ( 1 + R / σ N ) 16 (1+D+p)^{4}(1+R/\sigma_{N})^{16} ( 1 + D + p ) 4 ( 1 + R / σ N ) 16 , and its sample-size requirement is linear in D + p D+p D + p . The response bound enters Λ \Lambda Λ here, whereas in (3.17) the response diameter enters only the exceptional-event term.
The same exponent is minimax under (A1)–(A5) conditionally on the existence of a subclass with fixed strict margins that admits the small bump perturbations used below on a regular region of volume comparable to vol ( M ) \operatorname{vol}(\mathcal{M}) vol ( M ) . The perturbations must preserve all five assumptions with the same constants, and the subclass must admit a fixed compactly supported response-noise density satisfying (A5), whose square root has a square-integrable derivative in the perturbation direction. Under these conditions, a Hellinger version of Assouad’s argument gives the same exponent with bounded noise. We do not construct such a subclass here; the unconditional minimax conclusion of this appendix concerns the broader class F \mathcal{F} F .
Proof of the upper bound
Reduction to entropy. Call a finite family G ϵ \mathcal{G}_{\epsilon} G ϵ of maps R D → B p ( B f ) \mathbb{R}^{D} \to B_{p}(B_{f}) R D → B p ( B f ) an ϵ \epsilon ϵ -net of F \mathcal{F} F if every ( M , F ) ∈ F (\mathcal{M},F) \in\mathcal{F} ( M , F ) ∈ F has a G ∈ G ϵ G \in\mathcal{G}_{\epsilon} G ∈ G ϵ with sup x ∈ M R ∥ F ( x ) − G ( x ) ∥ ≤ ϵ \sup_{x\in\mathcal{M}_{R}}\|F(x)-G(x)\| \leq\epsilon sup x ∈ M R ∥ F ( x ) − G ( x ) ∥ ≤ ϵ . The least-squares estimator F ^ L S \widehat{F}_{\mathrm{LS}} F LS , a minimizer of ∑ i ∥ Y i − G ( X i ) ∥ 2 \sum_{i}\|Y_{i}-G(X_{i})\|^{2} ∑ i ∥ Y i − G ( X i ) ∥ 2 over G ∈ G ϵ G \in\mathcal{G}_{\epsilon} G ∈ G ϵ , satisfies
E ∥ F ^ L S ( X ) − F ( X ) ∥ 2 ≤ C [ ϵ 2 + ( σ ζ 2 + B f 2 ) 1 + log ∣ G ϵ ∣ n ] . (91) \mathbb{E}\|\widehat{F}_{\mathrm{LS}}(X)-F(X)\|^{2} \leq C\left[\epsilon^{2}+(\sigma_{\zeta}^{2}+B_{f}^{2})\frac{1+\log|\mathcal{G}_{\epsilon}|}{n}\right]. \tag*{(91)} E ∥ F LS ( X ) − F ( X ) ∥ 2 ≤ C [ ϵ 2 + ( σ ζ 2 + B f 2 ) n 1 + log ∣ G ϵ ∣ ] . (91)
Indeed, let G ∘ ∈ G ϵ G^{\circ} \in\mathcal{G}_{\epsilon} G ∘ ∈ G ϵ be within ϵ \epsilon ϵ of F F F . Conditionally on the predictors, ∑ i ⟨ ζ i , G ( X i ) − G ∘ ( X i ) ⟩ \sum_{i}\langle\zeta_{i},G(X_{i})-G^{\circ}(X_{i})\rangle ∑ i ⟨ ζ i , G ( X i ) − G ∘ ( X i )⟩ has ψ 2 \psi_{2} ψ 2 norm at most C σ ζ ( ∑ i ∥ G ( X i ) − G ∘ ( X i ) ∥ 2 ) 1 / 2 C\sigma_{\zeta}(\sum_{i}\|G(X_{i})-G^{\circ}(X_{i})\|^{2})^{1/2} C σ ζ ( ∑ i ∥ G ( X i ) − G ∘ ( X i ) ∥ 2 ) 1/2 for each G G G . A union bound over G ϵ \mathcal{G}_{\epsilon} G ϵ in the basic inequality of least squares bounds the empirical mean squared error of F ^ L S \widehat{F}_{\mathrm{LS}} F LS by C [ ϵ 2 + σ ζ 2 ( 1 + log ∣ G ϵ ∣ + t ) / n ] C[\epsilon^{2}+\sigma_{\zeta}^{2}(1+\log|\mathcal{G}_{\epsilon}|+t)/n] C [ ϵ 2 + σ ζ 2 ( 1 + log ∣ G ϵ ∣ + t ) / n ] with probability at least 1 − e − t 1-e^{-t} 1 − e − t . Bernstein’s inequality for the variables ∥ G ( X ) − F ( X ) ∥ 2 ≤ 4 B f 2 \|G(X)-F(X)\|^{2} \leq4B_{f}^{2} ∥ G ( X ) − F ( X ) ∥ 2 ≤ 4 B f 2 , with a union bound over G ϵ \mathcal{G}_{\epsilon} G ϵ , passes to the population mean squared error at the cost of C B f 2 ( 1 + log ∣ G ϵ ∣ + t ) / n CB_{f}^{2}(1+\log|\mathcal{G}_{\epsilon}|+t)/n C B f 2 ( 1 + log ∣ G ϵ ∣ + t ) / n , and integrating over t t t gives (C.1) (compare [45 Chapter 11] ). As in [105 ] , the rate is then governed by the metric entropy of F \mathcal{F} F . It remains to bound log ∣ G ϵ ∣ \log|\mathcal{G}_{\epsilon}| log ∣ G ϵ ∣ .
Stability of the nearest-point projection.
Lemma C.3 Let M , M ~ \mathcal{M},\widetilde{\mathcal{M}} M , M be compact manifolds without boundary and with reach at least R / c R/c R / c . Assume every q ∈ M q \in\mathcal{M} q ∈ M has a corresponding q ′ ∈ M ~ q' \in\widetilde{\mathcal{M}} q ′ ∈ M satisfying
∥ q − q ′ ∥ ≤ δ , ∥ P q − P ~ q ′ ∥ ≤ θ , δ ≤ ( 1 − c ) R / ( 2 c ) , \|q-q'\| \leq\delta,\qquad\|P_{q}-\widetilde{P}_{q'}\| \leq\theta,\qquad\delta\leq(1-c)R/(2c), ∥ q − q ′ ∥ ≤ δ , ∥ P q − P q ′ ∥ ≤ θ , δ ≤ ( 1 − c ) R / ( 2 c ) , :::`
where P q P_q P q , P ~ q ′ \widetilde{P}_{q'} P q ′ project onto the respective tangent spaces. For every x ∈ M R x \in\mathcal{M}_R x ∈ M R , with q = Π M x q = \Pi_{\mathcal{M}}x q = Π M x and corresponding q ′ q' q ′ ,
∥ Π M ~ x − q ′ ∥ ≤ 2 ( δ + R θ ) 1 − c . \left\|\Pi_{\widetilde{\mathcal{M}}}x-q'\right\| \leq\frac{2(\delta+R\theta)}{1-c}. Π M x − q ′ ≤ 1 − c 2 ( δ + R θ ) .
If moreover f ∈ L ( M ) f \in\mathcal{L}(\mathcal{M}) f ∈ L ( M ) , f ~ ∈ L ( M ~ ) \widetilde{f} \in\mathcal{L}(\widetilde{\mathcal{M}}) f ∈ L ( M ) , ∥ f ( q ) − f ~ ( q ′ ) ∥ ≤ ϵ 1 \|f(q)-\widetilde{f}(q')\| \leq\epsilon_1 ∥ f ( q ) − f ( q ′ ) ∥ ≤ ϵ 1 for all corresponding pairs q , q ′ q,q' q , q ′ , and the right side of the display is at most R / ( 2 c ) R/(2c) R / ( 2 c ) , then on M R \mathcal{M}_R M R
∥ f ( Π M x ) − f ~ ( Π M ~ x ) ∥ ≤ ϵ 1 + 4 σ f , + ( δ + R θ ) 1 − c . \left\|f(\Pi_{\mathcal{M}}x)-\widetilde{f}(\Pi_{\widetilde{\mathcal{M}}}x)\right\| \leq\epsilon_1+\frac{4\sigma_{f,+}(\delta+R\theta)}{1-c}. f ( Π M x ) − f ( Π M x ) ≤ ϵ 1 + 1 − c 4 σ f , + ( δ + R θ ) .
Proof. Since ∥ x − q ′ ∥ ≤ R + δ < R / c \|x-q'\| \leq R+\delta<R/c ∥ x − q ′ ∥ ≤ R + δ < R / c , the point q ~ = Π M ~ x \widetilde{q}=\Pi_{\widetilde{\mathcal{M}}}x q = Π M x is defined and ∥ x − q ~ ∥ ≤ R + δ \|x-\widetilde{q}\|\leq R+\delta ∥ x − q ∥ ≤ R + δ . Put u = q ~ − q ′ u=\widetilde{q}-q' u = q − q ′ . For a set of reach at least R / c R/c R / c , points a , b a,b a , b of the set and a vector ν \nu ν normal at a a a ,
⟨ ν , b − a ⟩ ≤ c 2 R ∥ ν ∥ ∥ b − a ∥ 2 . \langle\nu,b-a\rangle\leq\frac{c}{2R}\|\nu\|\|b-a\|^2. ⟨ ν , b − a ⟩ ≤ 2 R c ∥ ν ∥∥ b − a ∥ 2 .
([39 Theorems 4.8(7) and 4.18] ). Decompose x − q ′ = ν ′ + t ′ x-q'=\nu'+t' x − q ′ = ν ′ + t ′ into its normal and tangential parts at q ′ q' q ′ . Since P q ( x − q ) = 0 P_q(x-q)=0 P q ( x − q ) = 0 ,
∥ t ′ ∥ ≤ ∥ ( P ~ q ′ − P q ) ( x − q ) ∥ + ∥ q − q ′ ∥ ≤ R θ + δ . \|t'\| \leq\|(\widetilde{P}_{q'}-P_q)(x-q)\|+\|q-q'\|\leq R\theta+\delta. ∥ t ′ ∥ ≤ ∥ ( P q ′ − P q ) ( x − q ) ∥ + ∥ q − q ′ ∥ ≤ R θ + δ .
Applying the inequality at q ′ q' q ′ with ν ′ \nu' ν ′ and at q ~ \widetilde{q} q with x − q ~ x-\widetilde{q} x − q ,
∥ u ∥ 2 = ⟨ x − q ′ , u ⟩ + ⟨ x − q ~ , q ′ − q ~ ⟩ ≤ c ( R + δ ) R ∥ u ∥ 2 + ∥ t ′ ∥ ∥ u ∥ , \|u\|^2=\langle x-q',u\rangle+\langle x-\widetilde{q},q'-\widetilde{q}\rangle\leq\frac{c(R+\delta)}{R}\|u\|^2+\|t'\|\|u\|, ∥ u ∥ 2 = ⟨ x − q ′ , u ⟩ + ⟨ x − q , q ′ − q ⟩ ≤ R c ( R + δ ) ∥ u ∥ 2 + ∥ t ′ ∥∥ u ∥ ,
which gives the first claim because c ( R + δ ) / R ≤ ( 1 + c ) / 2 c(R+\delta)/R\leq(1+c)/2 c ( R + δ ) / R ≤ ( 1 + c ) /2 . For the second, ∥ f ( q ) − f ~ ( q ~ ) ∥ ≤ ϵ 1 + ∥ f ~ ( q ′ ) − f ~ ( q ~ ) ∥ \|f(q)-\widetilde{f}(\widetilde{q})\|\leq\epsilon_1+\|\widetilde{f}(q')-\widetilde{f}(\widetilde{q})\| ∥ f ( q ) − f ( q ) ∥ ≤ ϵ 1 + ∥ f ( q ′ ) − f ( q ) ∥ , and the last term is at most σ f , + \sigma_{f,+} σ f , + times the geodesic distance from q ′ q' q ′ to q ~ \widetilde{q} q on M ~ \widetilde{\mathcal{M}} M , which is at most 2 ∥ u ∥ 2\|u\| 2∥ u ∥ when ∥ u ∥ ≤ R / ( 2 c ) \|u\|\leq R/(2c) ∥ u ∥ ≤ R / ( 2 c ) ([82 Proposition 6.3] ).
Degrees of freedom. Put ϵ r e l = ϵ / ( σ f , + σ N ) ≤ 1 \epsilon_{\mathrm{rel}}=\epsilon/(\sigma_{f,+}\sigma_N)\leq1 ϵ rel = ϵ / ( σ f , + σ N ) ≤ 1 . By Lemma C.3 it suffices to describe ( M , f ) (\mathcal{M},f) ( M , f ) up to δ ≍ ϵ / σ f , + \delta\asymp\epsilon/\sigma_{f,+} δ ≍ ϵ / σ f , + in position, up to θ ≍ ϵ / ( σ f , + R ) \theta\asymp\epsilon/(\sigma_{f,+}R) θ ≍ ϵ / ( σ f , + R ) in tangent planes, and up to ϵ \epsilon ϵ in the values of f f f . Cover M \mathcal{M} M by N ≤ C v max / R m N\leq C v_{\max}/R^m N ≤ C v m a x / R m charts as in (A1), centered at a maximal R / ( 10 c ) R/(10c) R / ( 10 c ) -separated subset of M \mathcal{M} M ; the bound on N N N is the volume lower bound for balls in manifolds with positive reach ([82 Lemma 5.3] ). Quantize each center to accuracy δ \delta δ and each tangent plane to accuracy θ \theta θ , and fix for every quantized plane an orthonormal basis of it and of its orthogonal complement. Over the quantized plane M \mathcal{M} M is still a graph, of a map g g g with ∥ D g ∥ ≤ 1 / 4 + C θ \|Dg\|\leq1/4+C\theta ∥ D g ∥ ≤ 1/4 + C θ and the same bounds on D 2 g D^2g D 2 g and D 3 g D^3g D 3 g up to constants; in the fixed bases g g g has values in R D − m \mathbb{R}^{D-m} R D − m , and we write φ ( w ) = f ( q + w + g ( w ) ) \varphi(w)=f(q+w+g(w)) φ ( w ) = f ( q + w + g ( w )) for the link in the chart. The pair ( M , f ) (\mathcal{M},f) ( M , f ) is then described by the following data, all in classical classes on Euclidean m m m -balls; a count of this kind, for the Hausdorff distance and without tracking the ambient dimension, is in Genovese et al. [42 ] , and the chart model is that of Aamari and Levrard [1 ] . Write ℓ ( ϵ ) = log ( e max { 1 , ϱ / σ N , B f / ( σ f , + σ N ) } / ϵ r e l ) \ell(\epsilon)=\log(e\max\{1,\varrho/\sigma_N,B_f/(\sigma_{f,+}\sigma_N)\}/\epsilon_{\mathrm{rel}}) ℓ ( ϵ ) = log ( e max { 1 , ϱ / σ N , B f / ( σ f , + σ N )} / ϵ rel ) . The counts below bound the logarithm of the number of quantized values.
The N N N centers in B D ( ϱ ) B_D(\varrho) B D ( ϱ ) and tangent planes, to accuracy δ \delta δ and θ \theta θ : at most C N D ℓ ( ϵ ) CND\,\ell(\epsilon) C N D ℓ ( ϵ ) , by volumetric bounds.
The N N N graph maps g g g , which lie in a ball of C 3 C^3 C 3 maps with values in R D − m \mathbb{R}^{D-m} R D − m , to accuracy θ \theta θ in C 1 C^1 C 1 . Degree-two Taylor polynomials on a grid of mesh R θ R\sqrt{\theta} R θ suffice, as in Kolmogorov and Tikhomirov [57 ] for smoothness 3 − 1 = 2 3-1=2 3 − 1 = 2 ; quantizing their coefficient vectors in R D − m \mathbb{R}^{D-m} R D − m by volumetric nets costs at most C N ( D − m ) θ − m / 2 log ( e / θ ) CN(D-m)\theta^{-m/2}\log(e/\theta) C N ( D − m ) θ − m /2 log ( e / θ ) .
The N N N link maps φ \varphi φ , which lie in a ball of C 1 , 1 C^{1,1} C 1 , 1 maps with values in R p \mathbb{R}^p R p , to accuracy ε \varepsilon ε in the uniform norm. Degree-one Taylor polynomials on a grid of mesh σ N ε r e l \sigma_N\sqrt{\varepsilon_{\mathrm{rel}}} σ N ε rel suffice, with coefficient vectors in R p \mathbb{R}^p R p quantized in the same way, at a cost of at most C N p ( R / σ N ) m ε r e l − m / 2 ℓ ( ε ) CNp\left(R/\sigma_N\right)^{m}\varepsilon_{\mathrm{rel}}^{-m/2}\ell(\varepsilon) C N p ( R / σ N ) m ε rel − m /2 ℓ ( ε ) .
If two pairs have the same quantized data, corresponding points with the same chart coordinate satisfy the position and tangent-projector bounds of Lemma C.3 with C δ , C θ C\delta,C\theta C δ , C θ in place of δ , θ \delta,\theta δ , θ ; their link values differ by at most C ε C\varepsilon C ε . Choose one pair ( M ~ , f ~ ) (\widetilde{M},\widetilde{f}) ( M , f ) of F \mathcal{F} F for each value of the quantized data, and let its net function be f ~ ∘ Π M ~ \widetilde{f}\circ\Pi_{\widetilde{M}} f ∘ Π M on the tube of radius ( 1 + c ) R / ( 2 c ) (1+c)R/(2c) ( 1 + c ) R / ( 2 c ) around M ~ \widetilde{M} M , where the projection is still defined, and zero elsewhere. By Lemma C.3 these functions form a C ε C\varepsilon C ε -net. Since ϱ ≥ R / c \varrho\ge R/c ϱ ≥ R / c , log ( e / θ ) ≤ C ℓ ( ε ) \log(e/\theta)\le C\ell(\varepsilon) log ( e / θ ) ≤ C ℓ ( ε ) . Since R ≥ σ N R\ge\sigma_N R ≥ σ N and
N θ − m / 2 ≲ v max σ N ( σ N R ) m / 2 ε r e l − m / 2 , N\theta^{-m/2}\lesssim\frac{v_{\max}}{\sigma_N}\left(\frac{\sigma_N}{R}\right)^{m/2}\varepsilon_{\mathrm{rel}}^{-m/2}, N θ − m /2 ≲ σ N v m a x ( R σ N ) m /2 ε rel − m /2 ,
we obtain
log ∣ G ε ∣ ≤ C ( D + p ) v max σ N m ε r e l − m / 2 ℓ ( ε ) . (92) \log|\mathcal{G}_{\varepsilon}|\le C(D+p)\frac{v_{\max}}{\sigma_N^m}\varepsilon_{\mathrm{rel}}^{-m/2}\ell(\varepsilon). \tag*{(92)} log ∣ G ε ∣ ≤ C ( D + p ) σ N m v m a x ε rel − m /2 ℓ ( ε ) . (92)
The manifold and the link have the same exponent m / 2 m/2 m /2 : the manifold is needed only through its tangent planes, and for a C 3 C^3 C 3 manifold these are as smooth as a C 1 , 1 C^{1,1} C 1 , 1 link. Inserting (C.2) in (C.1) with
ε r e l ( m + 4 ) / 2 = Λ ( D + p ) v max σ N m n \varepsilon_{\mathrm{rel}}^{(m+4)/2}=\frac{\Lambda(D+p)v_{\max}}{\sigma_N^m n} ε rel ( m + 4 ) /2 = σ N m n Λ ( D + p ) v m a x
proves Proposition C.1. At this resolution, (C.2) gives
log ∣ G ε ∣ ≤ C ( ( D + p ) v max / σ N m ) 4 / ( m + 4 ) ( n / Λ ) m / ( m + 4 ) ℓ n . \log|\mathcal{G}_{\varepsilon}|\le C\left((D+p)v_{\max}/\sigma_N^m\right)^{4/(m+4)}(n/\Lambda)^{m/(m+4)}\ell_n. log ∣ G ε ∣ ≤ C ( ( D + p ) v m a x / σ N m ) 4/ ( m + 4 ) ( n /Λ ) m / ( m + 4 ) ℓ n .
This bounds the logarithm of the number of candidates. Direct evaluation of all empirical losses uses n ∣ G ε ∣ n|\mathcal{G}_{\varepsilon}| n ∣ G ε ∣ pointwise function evaluations, assuming access to the representatives. A runtime bound would additionally require an algorithm to construct these representatives and evaluate their nearest-point projections; the entropy argument provides only their existence.
Proof of the lower bound
Restrict the supremum to M = M 0 \mathcal{M}=\mathcal{M}_0 M = M 0 , to Q Q Q uniform on M 0 \mathcal{M}_0 M 0 with Z Z Z uniform on the normal ball of radius R R R given Q Q Q , and to Gaussian noise N ( 0 , σ 2 I p ) N(0,\sigma^2 I_p) N ( 0 , σ 2 I p ) with σ \sigma σ a fixed fraction of σ ζ \sigma_{\zeta} σ ζ . In this subproblem M 0 \mathcal{M}_0 M 0 is known, so Q = Π M 0 X Q=\Pi_{\mathcal{M}_0}X Q = Π M 0 X is observed and Z Z Z carries no information about f f f : the subproblem is regression of f f f on M 0 \mathcal{M}_0 M 0 with uniform design, since averaging an estimator over the normal fiber does not increase its risk, by Jensen’s inequality. For b ≤ σ N / 6 b\le\sigma_N/6 b ≤ σ N /6 , which is less than R / ( 10 c ) R/(10c) R / ( 10 c ) , take N 0 ≍ v 0 / b m N_0\asymp v_0/b^m N 0 ≍ v 0 / b m disjoint geodesic balls of radius b b b in M 0 \mathcal{M}_0 M 0 with centers q k q_k q k , a fixed smooth bump ψ \psi ψ on the unit ball, a unit vector e ∈ R p e \in\mathbb{R}^p e ∈ R p , a small fixed λ 0 > 0 \lambda_0 > 0 λ 0 > 0 , and
f ϑ = ∑ k ≤ N 0 ϑ k Φ k , Φ k ( q ) = λ 0 σ f , + b 2 σ N ψ ( 2 P q k ( q − q k ) / b ) e , ϑ ∈ { 0 , 1 } N 0 , f_{\vartheta}=\sum_{k\leq N_0}\vartheta_k\Phi_k,\qquad\Phi_k(q)=\lambda_0\frac{\sigma_{f,+}b^2}{\sigma_N}\psi(2P_{q_k}(q-q_k)/b)e,\qquad\vartheta\in\{0,1\}^{N_0}, f ϑ = k ≤ N 0 ∑ ϑ k Φ k , Φ k ( q ) = λ 0 σ N σ f , + b 2 ψ ( 2 P q k ( q − q k ) / b ) e , ϑ ∈ { 0 , 1 } N 0 ,
with Φ k \Phi_k Φ k given by this formula on the k k k -th ball and zero elsewhere. For λ 0 \lambda_0 λ 0 small, depending only on m m m , c c c and C F C_F C F , every f ϑ f_{\vartheta} f ϑ belongs to L ( M 0 ) \mathcal{L}(\mathcal{M}_0) L ( M 0 ) , by the chart bound. If ϑ \vartheta ϑ and ϑ ′ \vartheta' ϑ ′ differ in one coordinate, ∥ f ϑ − f ϑ ′ ∥ L 2 \|f_{\vartheta}-f_{\vartheta'}\|_{L^2} ∥ f ϑ − f ϑ ′ ∥ L 2 under the uniform law is of order
( λ 0 σ f , + b 2 / σ N ) 2 b m / v 0 \left(\lambda_0\sigma_{f,+}b^2/\sigma_N\right)^2b^m/v_0 ( λ 0 σ f , + b 2 / σ N ) 2 b m / v 0
and the Kullback–Leibler divergence of the samples is n / ( 2 σ 2 ) n/(2\sigma^2) n / ( 2 σ 2 ) times this quantity. Choose b b b so that this divergence is one, that is
b m + 4 ≍ σ 2 σ N 2 v 0 n λ 0 2 σ f , + 2 ; b^{m+4}\asymp\frac{\sigma^2\sigma_N^2v_0}{n\lambda_0^2\sigma_{f,+}^2}; b m + 4 ≍ n λ 0 2 σ f , + 2 σ 2 σ N 2 v 0 ;
the condition on n n n ensures b ≤ σ N / 6 b\leq\sigma_N/6 b ≤ σ N /6 . Assouad’s lemma ([98 Theorem 2.12(iv] ), applied to the bumps of disjoint supports, bounds the risk below by a constant times N 0 N_0 N 0 such squared distances, that is by a constant times ( λ 0 σ f , + / σ N ) 2 b 4 (\lambda_0\sigma_{f,+}/\sigma_N)^2b^4 ( λ 0 σ f , + / σ N ) 2 b 4 , which is the claim. This is the argument of Stone [94 ] and of Györfi et al. [45 ] in a chart.
Remark C.4 (Known manifold and anisotropic classes) If M \mathcal{M} M is known, the degrees of freedom are those of the link alone. In tubular coordinates x = q + z x=q+z x = q + z the function F ( q + z ) = f ( q ) F(q+z)=f(q) F ( q + z ) = f ( q ) does not depend on the D − m D-m D − m normal variables, so it belongs to an anisotropic Hölder class with smoothness s = 2 s=2 s = 2 in m m m variables and arbitrary smoothness in the others. Its effective smoothness s ˉ \bar{s} s ˉ , given by 1 / s ˉ = ∑ i 1 / s i = m / s 1/\bar{s}=\sum_i1/s_i=m/s 1/ s ˉ = ∑ i 1/ s i = m / s with s i s_i s i the smoothness in the i i i th variable, yields the classical anisotropic rate n − 2 s ˉ / ( 2 s ˉ + 1 ) = n − 2 s / ( 2 s + m ) n^{-2\bar{s}/(2\bar{s}+1)}=n^{-2s/(2s+m)} n − 2 s ˉ / ( 2 s ˉ + 1 ) = n − 2 s / ( 2 s + m ) ([49 ] ). The content of Lemma C.3 and of the count above is that the unknown change of coordinates costs no more entropy than the link. For general smoothness, a C s C^s C s link with s ≥ 1 s\geq1 s ≥ 1 on a C s + 1 C^{s+1} C s + 1 manifold gives the exponent 2 s / ( 2 s + m ) 2s/(2s+m) 2 s / ( 2 s + m ) by the same argument.
Remark C.5 (Flat pieces can lose one order) Pieces that are polynomial in the tangent coordinates P q j ( x − q j ) P_{q_j}(x-q_j) P q j ( x − q j ) at points q j q_j q j of a net of M \mathcal{M} M , and constant along the normal space at q j q_j q j , in general approximate F F F only to first order in the spacing b b b of the net. For x = q + z x=q+z x = q + z with z z z normal at q q q ,
P q j ( x − q j ) = P q j ( q − q j ) + ( P q j − P q ) z , P_{q_j}(x-q_j)=P_{q_j}(q-q_j)+(P_{q_j}-P_q)z, P q j ( x − q j ) = P q j ( q − q j ) + ( P q j − P q ) z ,
and the last term has size up to c d M ( q , q j ) c\,d_{\mathcal{M}}(q,q_j) c d M ( q , q j ) : the normal fibers over nearby base points are tilted against the normal space at q j q_j q j . This is the amplification ( I − S z ) − 1 (I-S_z)^{-1} ( I − S z ) − 1 of the tangential gradient in § 2.1 . In this worst case, a construction with a fixed normal space on each patch needs b ≍ ε b\asymp\varepsilon b ≍ ε , hence ε − m \varepsilon^{-m} ε − m pieces instead of ε − m / 2 \varepsilon^{-m/2} ε − m /2 , and gives only the rate n − 2 / ( m + 2 ) n^{-2/(m+2)} n − 2/ ( m + 2 ) . Projecting onto an approximating manifold, as in Lemma C.3 , follows the variation of the normal spaces with the base point; each normal fiber itself is an affine disk.