betaFeedback

WorksMathematics

Adaptive Sparsity-Aware Projection-Assisted Spectral Embedding for High-Dimensional Noisy Data with Nonlinear Structures

Abstract

In this paper, we propose SPASE, an adaptive, sparsity-aware, projection-assisted spectral embedding method for feature extraction from high-dimensional noisy data with underlying nonlinear structures and latent sparsity patterns. We first construct a data-driven projection that adapts to the hidden sparsity patterns and the signal-to-noise ratio, reducing noise-induced distortion of pairwise distances. We then apply kernel-based spectral embedding to the projected data to explore the underlying geometric structure. The key methodological contribution is an energy-based projection selection algorithm that is adaptive to the signal-to-noise ratio and sparsity of the signal representation, while mitigating the impact of noise. On the theoretical side, we show that the resulting embeddings converge to the eigenfunctions of an integral operator associated with the latent signal, and the method can remain effective when the signal-to-noise ratio is low. We demonstrate the empirical performance of the proposed method through simulation studies and real-data applications, showing improved accuracy in downstream tasks such as classification and clustering.

Introduction

Feature extraction is a fundamental problem in modern data science, particularly in the analysis of high-dimensional data. Classical linear methods such as principal component analysis (PCA) [38] are natural starting points when the dominant variability of the data can be well represented by a low-dimensional linear subspace. However, many modern datasets exhibit intrinsically nonlinear geometric structure that cannot be adequately captured by linear representations alone. To address this challenge, a large class of manifold learning and nonlinear spectral embedding methods has been developed, including kernel PCA [57], Isomap [63], locally linear embedding [55], Laplacian eigenmaps [6], diffusion maps [17], vector diffusion maps [61], t-SNE [66], and UMAP [47]. More recently, neural-network-based approaches such as autoencoders [37], denoising autoencoders [69], and variational autoencoders [41] have provided additional flexible frameworks for nonlinear representation learning. For downstream tasks such as classification, clustering, and visualization, informative low-dimensional representations are often essential for revealing meaningful structure and enabling reliable statistical analysis.

Existing manifold learning and spectral embedding methods are typically most effective when the observed geometry still reliably reflects the underlying signal structure, such as in clean or signal-dominated settings. In many modern high-dimensional applications, however, the ambient noise can substantially distort pairwise distances, affinities, and neighborhood relations before the final embedding step is even performed. As a result, the observed geometry itself may no longer provide a reliable representation of the latent nonlinear structure. For example, in biomedical imaging such as cryo-electron microscopy, low-dose acquisition often leads to extremely low signal-to-noise ratios [5]. At the same time, signals in such applications often possess additional low-complexity structure, such as sparse or compressible representations [51]. This additional structure creates an opportunity to partially recover or stabilize the underlying geometry before applying nonlinear spectral embedding methods.

Motivated by these observations, in this paper, we study feature extraction under the signal-plus-noise model

xi=si+zi∈Rp,1≤i≤n,(1)\mathbf{x}_i = \mathbf{s}_i + \mathbf{z}_i \in\mathbb{R}^{p}, \qquad1 \le i \le n, \tag*{(1)}

where si\mathbf{s}_i denotes the latent signal component and zi\mathbf{z}_i denotes the additive noise component. We focus on the setting where the latent signal has nonlinear geometric structure and admits a sparse or compressible representation. We propose an adaptive sparsity-aware projection-assisted spectral embedding method for this problem. The main idea is to first perform an adaptive projection step to denoise the pairwise distances and then apply spectral embedding to the projected data. We first summarize some relevant related works in § 1.1 and then provide an overview of our method and contribution in § 1.2.

Some Related Works

The behavior of manifold learning and spectral embedding methods has been extensively studied in fixed-dimensional settings. In the noiseless case, a substantial theoretical literature establishes the convergence of graph- and kernel-based constructions to their population or continuum counterparts; see, for example, [7, 36, 70, 64, 34, 65, 26, 15, 43]. Related analyses have also been developed for specific methods and models, including continuum Isomap [8], locally linear embedding [74], and vector diffusion maps and connection Laplacians [61, 62]. In parallel, a number of robust variants have been proposed in fixed-dimensional signal-plus-noise settings, including perturbation analysis of graph-Laplacian eigenvectors for denoising [48], robust locally linear embedding [13], and robust kernel Isomap [16]. Collectively, these works show that moderate noise can often be accommodated when the observed geometry still reflects the latent signal structure with reasonable fidelity.

The situation becomes substantially more challenging in high-dimensional noisy settings, where the main difficulty is not only the perturbation of the final embedding step, but also the distortion of the geometry itself. High-dimensional noise may induce concentration effects and geometric artifacts, so that distances, affinities, and neighborhood relations constructed from the observations become much less informative [1, 28, 27, 10]. Nevertheless, when the signal dominates the noise, recent works have shown that direct graph- and kernel-based spectral methods can still recover meaningful low-dimensional structure under suitable conditions. These include kernel spectral embedding with adaptive bandwidth selection [20], robustness analysis for connection graph Laplacians under additive noise [29], graph-Laplacian analysis under high-dimensional noise [22], landmark-based spectral embedding [59], and multiview sensor-fusion methods [23, 21]. Taken together, these results provide theoretical justification for nonlinear spectral embedding in certain high-dimensional signal-plus-noise regimes, provided that the underlying geometry remains sufficiently informative and that the bandwidth is properly chosen.

Much less is understood in low-SNR regimes, where the noise dominates the signal and the observed geometry may no longer provide a reliable basis for direct graph or kernel construction. In this setting, additional low-complexity structure becomes particularly important. For linear methods, sparsity has been widely used to identify informative directions in high-dimensional noisy data through sparse PCA and related approaches. Early sparse PCA methods include penalized regression, regularized low-rank approximation, and penalized matrix decomposition [77, 60, 73]. Related transform-domain reduction and feature-selection ideas before downstream low-dimensional analysis appear in [71, 32, 49], while selection-based methods for sparse or compressible signals in prescribed bases are studied in [40]. Subsequent developments include augmented sparse PCA [52], two-stage coordinate-selection procedures [9], and adaptive sparse principal subspace estimation [12]. However, these methods are primarily designed for linear covariance structures, and much less is known about how sparsity can be used to stabilize nonlinear geometric structure for manifold learning and spectral embedding in high-dimensional low-SNR settings.

To address this gap, we propose an adaptive sparsity-aware projection-assisted spectral embedding method for feature extraction from high-dimensional noisy data with nonlinear structures under sparse or compressible representations. The key idea is to use adaptive projection to denoise the pairwise geometry before applying nonlinear spectral embedding. Our approach applies across a broad range of SNR levels, while being particularly useful when the noise dominates the signal. An overview of our method and contributions is given in § 1.2.

An Overview of Main Results and Contributions

Figure 1.1 illustrates the overall workflow of the proposed framework. As in [57, 6, 17, 61, 20], our algorithm relies on pairwise distances to construct an affinity matrix for spectral embedding. In high-dimensional noisy settings, however, the main challenge is that these pairwise distances may already become unreliable before the final eigendecomposition is performed. In particular, high-dimensional asymptotics reveal rigid geometric effects in which observations tend to become nearly equidistant or nearly orthogonal [35, 2], while nearest-neighbor structure may become increasingly unreliable through hubness and related distortions [54, 3]. Moreover, high-dimensional noise can substantially alter the effective kernel geometry extracted from the observations [27]. Consequently, the main issue is not only whether the spectral embedding step itself is stable, but also whether the input geometry still faithfully reflects the latent signal structure. The goal of our method is thus to denoise the pairwise geometry before applying spectral embedding.

Overview of the proposed adaptive sparsity-aware projection-assisted spectral embedding framework

Figure 1.1. Overview of the proposed adaptive sparsity-aware projection-assisted spectral embedding framework. The method first performs adaptive projection to denoise pairwise geometry in high-dimensional noisy observations and then applies spectral embedding to the projected data to recover the latent nonlinear structure.

To address this difficulty, we exploit additional sparsity structure in the latent signal. When the signal admits a sparse representation in a prescribed basis, most of its energy is concentrated on a relatively small number of directions, while the ambient noise is more diffusely spread across coordinates. This separation between concentrated signal energy and diffuse noise motivates an adaptive geometry-denoising step prior to manifold learning. Specifically, we perform a preliminary data-adaptive projection step in which empirically energetic directions are retained and noise-dominated ones are discarded, similar in spirit to earlier transform-domain reduction and feature-screening ideas; see, for example, [71, 32, 49, 40]. The resulting projection stabilizes the pairwise geometry before subsequent spectral analysis.

Based on this idea, we propose an adaptive sparsity-aware projection-assisted spectral embedding method in § 2. The overall procedure consists of two main steps. First, we construct a data-driven projection by selecting directions in a prescribed basis so as to reduce the effect of noise before pairwise distances are computed. The key methodological contribution lies in this step: we introduce an empirical-energy-based thresholding rule (Algorithm 1) that is fully data-driven and, as shown in Proposition 3.4, adapts to the unknown underlying SNR and sparsity structure of the data. Intuitively, the procedure retains directions whose empirical energies remain distinguishable from the ambient noise level, while filtering out directions dominated by noise. As shown in Theorem rem 3.6, the projection denoises the pairwise geometry of the data and makes subsequent spectral analysis more reliable.

Second, after this denoising projection step, we apply a spectral embedding method with bandwidth selection in the spirit of [20] to the projected data. This produces a unified project-then-embed framework that combines adaptive geometry recovery with nonlinear spectral embedding. We show in Theorem 3.8 that, after normalization, the resulting embeddings converge to the eigenfunctions of an integral operator associated with the clean latent signal. Although we adopt [20] here for concreteness and tractability, the second step can in principle be replaced by other spectral embedding methods, for example, diffusion-based procedures such as Laplacian eigenmaps [6], diffusion maps [17], and vector diffusion maps [61]; we leave these extensions for future work.

Our theoretical analysis in § 3 shows that the proposed framework applies across a broad range of SNR levels and remains effective even in low-SNR regimes where the noise dominates the signal and direct geometric construction may fail. In addition to the theoretical analysis, § 4 and § 5 compare the empirical performance of the proposed method with [55, 4, 17, 37, 69, 20] through simulation studies and real data examples. The results demonstrate that the proposed geometry-denoising framework substantially improves the robustness of nonlinear feature extraction in high-dimensional noisy settings, particularly in low-SNR regimes.

Organization and Notations

The rest of the paper is organized as follows. In § 2, we introduce the proposed method. In § 3, we state the main assumptions and establish the theoretical guarantees. § 4 and § 5 present simulation studies and real data analyses. Additional simulation and real data results are provided in Appendices § 6 and § 7, respectively. Supplementary algorithms are given in Appendix § 9, and technical proofs are deferred to Appendix § 8.

Conventions. We write CC, C0C_0, etc. for generic large positive constants whose values may change from line to line, and ε\varepsilon, δ\delta, etc. for generic small positive constants. We use ⌊a⌋\lfloor a \rfloor for the greatest integer less than or equal to aa. When a constant depends on a parameter aa, we indicate this by C(a)C(a) or CaC_a. For sequences {an}\{a_n\} and {bn}\{b_n\} depending on nn, an=O(bn)a_n = O(b_n) means ∣an∣≤C∣bn∣|a_n| \le C|b_n| for some constant C>0C > 0, while an=o(bn)a_n = o(b_n) means ∣an∣≤cn∣bn∣|a_n| \le c_n|b_n| for some positive sequence cn↓0c_n \downarrow0 as n→∞n \to\infty. We also use an≲bna_n \lesssim b_n to mean an=O(bn)a_n = O(b_n), an≳bna_n \gtrsim b_n to mean bn=O(an)b_n = O(a_n), and an≍bna_n \asymp b_n to mean both an=O(bn)a_n = O(b_n) and bn=O(an)b_n = O(a_n). The probabilistic analogues are an=OP(bn)a_n = O_{\mathbb{P}}(b_n) and an=oP(bn)a_n = o_{\mathbb{P}}(b_n).

Bold lowercase letters such as x,y\mathbf{x}, \mathbf{y} denote vectors, while bold uppercase letters such as K\mathbf{K} denote matrices. Calligraphic letters such as K\mathcal{K} and L\mathcal{L} denote operators or function spaces, and plain letters such as f,gf, g denote generic functions. For norms, ∥x∥2=(∑j=1p∣xj∣2)1/2\|\mathbf{x}\|_2 = \left(\sum_{j=1}^{p}|x_j|^2\right)^{1/2} denotes the ℓ2\ell_2-norm of x=(x1,…,xp)T∈Rp\mathbf{x} = (x_1,\ldots,x_p)^{\mathsf{T}} \in\mathbb{R}^p. We denote by ∥L∥\|\mathcal{L}\| and ∥A∥\|\mathbf{A}\| the operator norms of an operator L\mathcal{L} and a matrix A\mathbf{A}, respectively. For a matrix A\mathbf{A}, let ΠA\mathbf{\Pi}_{\mathbf{A}} denote the orthogonal projection onto Col⁡(A)\operatorname{Col}(\mathbf{A}).

Methodology

In this section, we introduce the proposed methodology. The overall framework consists of two main stages: an adaptive projection step for denoising pairwise geometry, followed by nonlinear spectral embedding on the projected data. We first construct an adaptive sparse projection to denoise the pairwise geometry of the observed high-dimensional data in § 2.1. We then apply a spectral embedding method to the projected samples in § 2.2. Finally, in § 2.3, we describe how the resulting embeddings can be used for downstream tasks such as clustering and prediction. For notational convenience, throughout the paper we adopt the convention that for a positive integer nn, [n]={1,2,…,n}[n]=\{1,2,\ldots,n\}.

Adaptive Sparse Projection

To denoise the pairwise geometry, we first construct a data-driven projection from the observed samples {xi}i∈[n]⊂Rp\{x_i\}_{i\in[n]}\subset\mathbb{R}^p, which is the key step in our methodology. Let W={wj}j∈[p]\mathbf{W}=\{w_j\}_{j\in[p]} be a prescribed orthonormal basis of Rp\mathbb{R}^p, for example, the Haar wavelet basis [18]. In this paper, we may take W\mathbf{W} to be a wavelet basis, since many structured signals have local features and are therefore represented by a relatively small number of large coefficients in wavelet bases [46, 24, 25]. For each 1≤i≤n1\leq i\leq n and 1≤j≤p1\leq j\leq p, define

yij=(xi−xˉ)Twj,(2)y_{ij}=(x_i-\bar{x})^{\mathsf{T}}w_j, \tag*{(2)}

where xˉ=n−1∑ixi\bar{x}=n^{-1}\sum_i x_i, and compute the empirical energy of the jj-th direction by

Ej=1n∑i=1nyij2.(3)E_j=\frac{1}{n}\sum_{i=1}^{n}y_{ij}^{2}. \tag*{(3)}

We then order {Ej}j∈[p]\{E_j\}_{j\in[p]} as E(1)≥E(2)≥⋯≥E(p)E_{(1)}\geq E_{(2)}\geq\cdots\geq E_{(p)}. The basic idea is to retain the basis directions corresponding to the KK largest empirical energies.

A central issue in this step is the choice of the retained dimension KK. If KK is too large, we retain too many noise-dominated directions; if KK is too small, we may discard directions that still contain useful signal information. To address this issue, we choose KK in a data-driven way based on the ordered empirical energies. More specifically, we use σ^2\hat{\sigma}^{2}, computed as the average of the smallest ss empirical energies, as an estimate of the noise variance, and choose KK as the largest jj such that E(j)≥(1+τ)σ^2E_{(j)}\geq(1+\tau)\hat{\sigma}^{2}, where τ\tau is a prespecified small positive constant. The role of ss is to ensure that σ^2\hat{\sigma}^{2} provides a reliable estimate of the noise level. In practice, ss can be chosen as a small fixed integer. As long as ss is fixed, it does not affect the theoretical order of KK under the conditions of Proposition 3.4. The parameter τ\tau controls how conservative the cutoff is: a larger τ\tau leads to a smaller KK, while a smaller τ\tau leads to a larger KK. However, changing τ\tau mainly affects directions whose signal energies are comparable to the noise level, which typically carry only a small fraction of the signal information. By contrast, clearly signal-dominated directions remain selected and clearly noise-dominated directions remain excluded. Theoretically, each fixed τ>0\tau>0 affects the order of KK only through multiplicative constants under the conditions of Proposition 3.4. In this paper, we fix τ=0.5\tau=0.5 in our experiments. The selection of KK is summarized in Algorithm 1.

Remark 2.1. The selected projection dimension KK is controlled jointly by the relative strength between the signal and the noise and by how sparse the signal is in the chosen basis. When the relative strength between the signal and the noise is larger, more basis directions have empirical Algorithm 1 Adaptive selection of the projection dimension KK

Inputs: Ordered empirical energies {E(j)}j=1p\{E_{(j)}\}_{j=1}^{p}, a small positive fixed number τ\tau, and a small fixed integer ss.

Step one: Compute the estimated noise variance

σ^2=1s∑j=1sE(p+1−j).\widehat{\sigma}^{2}=\frac{1}{s}\sum_{j=1}^{s}E_{(p+1-j)}.

Step two: Compute

K=max⁡{j:E(j)≥(1+τ)σ^2}.(4)K=\max\left\{j:E_{(j)}\geq(1+\tau)\widehat{\sigma}^{2}\right\}. \tag*{(4)}

Output: The selected projection dimension KK.

energies that remain clearly above the noise level, and the selected dimension becomes larger. By contrast, when relative strength between the signal and the noise is smaller, fewer directions can be reliably distinguished and the selected dimension becomes smaller. Likewise, when the signal is more sparse in the basis, its energy is concentrated on fewer directions, which creates a sharper separation between informative and uninformative directions and makes the choice of KK smaller. We will illustrate these effects on KK in Figure 4.6 in § 4.

Figure 4.6. Selected projection dimension KK versus sparsity parameter qq under different σ\sigma in the two-disjoint-spheres model. The reported results are based on 250 repetitions.

References

  1. [1]C. C. Aggarwal, A. Hinneburg, and D. A. Keim. On the surprising behavior of distance metrics in high dimensional space. In J. Van den Bussche and V. Vianu, editors, Database Theory — ICDT 2001, volume 1973 of Lecture Notes in Computer Science, pages 420–434, Berlin, Heidelberg, 2001. Springer.
  2. [2]J. Ahn, J. S. Marron, K. M. Müller, and Y.-Y. Chi. The high-dimension, low-sample-size geometric representation holds under mild conditions. Biometrika, 94(3):760–766, 2007.DOI
  3. [3]F. Angiulli. On the behavior of intrinsically high-dimensional spaces: Distances, direct and reverse nearest neighbors, and hubness. Journal of Machine Learning Research, 18(170):1–60, 2018.DOI
  4. [4]M. Balasubramanian and E. L. Schwartz. The Isomap algorithm and topological stability. Science, 295(5552):7, Jan. 2002.pubmed.ncbi.nlm.nih.gov/11778013
  5. [5]W. T. Baxter, R. A. Grassucci, H. Gao, and J. Frank. Determination of signal-to-noise ratios and spectral snrs in cryo-em low-dose imaging of molecules. Journal of Structural Biology, 166(2):126–132, 2009.DOI
  6. [6]M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation, 15(6):1373–1396, 2003.DOI
  7. [7]M. Belkin and P. Niyogi. Convergence of Laplacian eigenmaps. Advances in neural information processing systems, 19, 2006.DOI
  8. [8]M. Bernstein, V. de Silva, J. C. Langford, and J. B. Tenenbaum. Graph approximations to geodesics on embedded manifolds and their statistical properties. The Annals of Statistics, 28(5):1241–1260, 2000.
  9. [9]A. Birnbaum, I. M. Johnstone, B. Nadler, and D. Paul. Minimax bounds for sparse PCA with noisy high-dimensional data. The Annals of Statistics, 41(3):1055–1084, 2013.pubmed.ncbi.nlm.nih.gov/25324581
  10. [10]C. Bordenave. On Euclidean random matrices in high dimension. Electronic Communications in Probability, 18(25):1–8, 2013.arxiv.org/abs/1209.5888
  11. [11]S. Boucheron, G. Lugosi, and P. Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, Oxford, 2013.
  12. [12]T. T. Cai, Z. Ma, and Y. Wu. Sparse PCA: Optimal rates and adaptive estimation. The Annals of Statistics, 41(6):3074–3110, 2013.DOI
  13. [13]H. Chang and D.-Y. Yeung. Robust locally linear embedding. Pattern Recognition, 39(6):1053–1065, 2006.DOI
  14. [14]M.-Y. Cheng and H.-t. Wu. Local linear regression on manifolds and its geometric interpretation. Journal of the American Statistical Association, 108(504):1421–1434, 2013.arxiv.org/abs/1201.0327
  15. [15]X. Cheng and N. Wu. Eigen-convergence of Gaussian kernelized graph Laplacian by manifold heat interpolation. Applied and Computational Harmonic Analysis, 61:132–190, 2022.DOI
  16. [16]H. Choi and S. Choi. Robust kernel Isomap. Pattern Recognition, 40(3):853–862, 2007.DOI
  17. [17]R. R. Coifman and S. Lafon. Diffusion maps. Applied and computational harmonic analysis, 21(1):5–30, 2006.DOI
  18. [18]I. Daubechies. Ten Lectures on Wavelets, volume 61 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, 1992.
  19. [20]X. Ding and R. Ma. Learning low-dimensional nonlinear structures from high-dimensional noisy data: An integral operator approach. The Annals of Statistics, 51(4):1744–1769, 2023.arxiv.org/abs/2203.00126
  20. [21]X. Ding, C. Shen, and H.-T. Wu. Generalized robust adaptive-bandwidth multi-view manifold learning in high dimensions with noise. arXiv preprint arXiv:2602.10530, 2026.arxiv.org/abs/2602.10530
  21. [22]X. Ding and H.-T. Wu. Impact of signal-to-noise ratio and bandwidth on graph laplacian spectrum from high-dimensional noisy point cloud. IEEE Transactions on Information Theory, 69(3):1899–1931, 2023.arxiv.org/abs/2011.10725
  22. [23]X. Ding and H.-T. Wu. How do kernel-based sensor fusion algorithms behave under high-dimensional noise? Information and Inference: A Journal of the IMA, 13(1):iaad051, 2024.arxiv.org/abs/2111.10940
  23. [24]D. L. Donoho and I. M. Johnstone. Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81(3):425–455, 1994.DOI
  24. [25]D. L. Donoho and I. M. Johnstone. Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association, 90(432):1200–1224, 1995.DOI
  25. [26]D. B. Dunson, H.-T. Wu, and N. Wu. Spectral convergence of graph Laplacian and heat kernel reconstruction in l∞ from random samples. Applied and Computational Harmonic Analysis, 55:282–336, 2021.
  26. [27]N. El Karoui. On information plus noise kernel random matrices. The Annals of Statistics, 38(5):3191–3216, 2010.
  27. [28]N. El Karoui. The spectrum of kernel random matrices. The Annals of Statistics, 38(1):1–50, 2010.
  28. [29]N. El Karoui and H. Wu. Graph connection Laplacian methods can be made robust to noise. The Annals of Statistics, 44(1):346–372, 2016.
  29. [30]L. Erdős and H.-T. Yau. A Dynamical Approach to Random Matrix Theory, volume 28 of Courant Lecture Notes in Mathematics. American Mathematical Society, Providence, RI, 2017.DOI
  30. [32]G.-C. Feng, P. C. Yuen, and D. Q. Dai. Human face recognition using PCA on wavelet subband. Journal of Electronic Imaging, 9(2):226–233, 2000.DOI
  31. [33]R. A. Fisher. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2):179–188, 1936.DOI
  32. [34]N. García Trillos, M. Gerlach, M. Hein, and D. Slepčev. Error estimates for spectral convergence of the graph Laplacian on random geometric graphs toward the laplace–beltrami operator. Foundations of Computational Mathematics, 20(4):827–887, 2020.DOI
  33. [35]P. Hall, J. S. Marron, and A. Neeman. Geometric representation of high dimension, low sample size data. Journal of the Royal Statistical Society: Series B, 67(3):427–444, 2005.
  34. [36]M. Hein, J.-Y. Audibert, and U. v. Luxburg. Graph Laplacians and their convergence on random neighborhood graphs. Journal of Machine Learning Research, 8(6), 2007.arxiv.org/abs/math/0608522
  35. [37]G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504–507, 2006.pubmed.ncbi.nlm.nih.gov/16873662
  36. [38]H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6):417–441, 1933.
  37. [39]L. Hubert and P. Arabie. Comparing partitions. Journal of Classification, 2(1):193–218, 1985.DOI
  38. [40]I. M. Johnstone and A. Y. Lu. On consistency and sparsity for principal components analysis in high dimensions. Journal of the American Statistical Association, 104(486):682–693, 2009.pubmed.ncbi.nlm.nih.gov/20617121
  39. [41]D. P. Kingma and M. Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, 2014.arxiv.org/abs/1312.6114
  40. [43]C. Li, N. G. Trillos, H. Li, and L. Suchan. Central limit theorems for the eigenvalues of graph Laplacians on data clouds. arXiv preprint arXiv:2507.18803, 2025.arxiv.org/abs/2507.18803
  41. [44]J. MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics, pages 281–297. University of California Press, 1967.
  42. [45]S. Mallat. A Wavelet Tour of Signal Processing: The Sparse Way. Elsevier, 3 edition, 2009.
  43. [46]S. G. Mallat. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7):674–693, 1989.
  44. [47]L. McInnes, J. Healy, and J. Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018.arxiv.org/abs/1802.03426
  45. [48]F. G. Meyer and X. Shen. Perturbation of the eigenvectors of the graph Laplacian: Application to image denoising. Applied and Computational Harmonic Analysis, 36(2):326–334, 2014.arxiv.org/abs/1202.6666
  46. [49]B. Nadler and R. R. Coifman. The prediction error in CLS and PLS: The importance of feature selection prior to multivariate calibration. Journal of Chemometrics, 19(2):107–118, 2005.DOI
  47. [51]B. A. Olshausen and D. J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583):607–609, 1996.pubmed.ncbi.nlm.nih.gov/8637596
  48. [52]D. Paul and I. M. Johnstone. Augmented sparse principal component analysis for high dimensional data, 2007. Technical report, December 11, 2007.
  49. [53]F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É. Duchesnay. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830, 2011.arxiv.org/abs/1201.0490
  50. [54]M. Radovanovic, A. Nanopoulos, and M. Ivanović. Hubs in space: Popular nearest neighbors in high-dimensional data. Journal of Machine Learning Research, 11(86):2487–2531, 2010.DOI
  51. [55]S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000.pubmed.ncbi.nlm.nih.gov/11125150
  52. [56]D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Nature, 323:533–536, 1986.
  53. [57]B. Schölkopf, A. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10(5):1299–1319, 1998.DOI
  54. [58]R. J. Serfling. Approximation Theorems of Mathematical Statistics. John Wiley & Sons, New York, 1980.
  55. [59]C. Shen and H.-T. Wu. Scalability and robustness of spectral embedding: landmark diffusion is all you need. Information and Inference: A Journal of the IMA, 11(4):1527–1595, 2022.arxiv.org/abs/2001.00801
  56. [60]H. Shen and J. Z. Huang. Sparse principal component analysis via regularized low rank matrix approximation. Journal of Multivariate Analysis, 99(6):1015–1034, 2008.DOI
  57. [61]A. Singer and H.-T. Wu. Vector diffusion maps and the connection Laplacian. Communications on Pure and Applied Mathematics, 65(8):1067–1144, 2012.DOI
  58. [62]A. Singer and H.-T. Wu. Spectral convergence of the connection laplacian from random samples. Information and Inference: A Journal of the IMA, 6(1):58–123, 2017.arxiv.org/abs/1306.1587
  59. [63]J. B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, 2000.pubmed.ncbi.nlm.nih.gov/11125149
  60. [64]D. Ting, L. Huang, and M. I. Jordan. An analysis of the convergence of graph Laplacians. In Proceedings of the 27th International Conference on Machine Learning (ICML), 2010. Also available as UC Berkeley Statistics Technical Report 792.arxiv.org/abs/1101.5435
  61. [65]N. G. Trillos, F. Hoffmann, and B. Hosseini. Geometric structure of graph laplacian embeddings. Journal of Machine Learning Research, 22(63):1–55, 2021.
  62. [66]L. Van der Maaten and G. Hinton. Visualizing data using t-SNE. Journal of machine learning research, 9(11), 2008.
  63. [67]A. W. van der Vaart. Asymptotic Statistics. Cambridge University Press, Cambridge, 1998.
  64. [68]R. Vershynin. High-dimensional probability: An introduction with applications in data science. Cambridge University Press, 2018.
  65. [69]P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096–1103, 2008.DOI
  66. [70]U. Von Luxburg, M. Belkin, and O. Bousquet. Consistency of spectral clustering. The Annals of Statistics, 36(2):555–586, 2008.DOI
  67. [71]M. V. Wickerhauser. Large-rank approximate principal component analysis with wavelets for signal feature discrimination and the inversion of complicated maps. Journal of Chemical Information and Computer Sciences, 34(5):1036–1046, 1994.DOI
  68. [72]C. K. I. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems 13 (N\̈eurIPS), pages 682–688, 2001.
  69. [73]D. M. Witten, R. Tibshirani, and T. Hastie. A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics, 10(3):515–534, 2009.DOI
  70. [74]H.-T. Wu and N. Wu. Think globally, fit locally under the manifold setup: Asymptotic analysis of locally linear embedding. The Annals of Statistics, 46(6B):3805–3837, 2018.arxiv.org/abs/1703.04058
  71. [75]J. Yang, R. Shi, and B. Ni. MedMNIST classification decathlon: A lightweight automl benchmark for medical image analysis. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 191–195. IEEE, 2021.DOI
  72. [76]J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni. MedMNIST v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
  73. [77]H. Zou, T. Hastie, and R. Tibshirani. Sparse principal component analysis. Journal of Computational and Graphical Statistics, 15(2):265–286, 2006.

Paper details

vlnc:W_h239esug
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents