betaFeedback

WorksMathematics

Scalable and Adaptive Distribution Regression via Random Features

Abstract

Distribution regression seeks to learn mappings from probability distributions to response variables and has emerged as a unifying framework for kernel methods, multi-instance learning, and set-valued learning. Despite its success, kernel-based distribution regression remains computationally expensive and sensitive to kernel choice, motivating scalable and adaptive representations. Building on random feature approximations, we develop a finite-sample analysis of Random Feature Distribution Regression (RaDR) under two-stage sampling. Our perturbation analysis separates the effects of distribution-level sampling, finite within-bag sampling, and random feature approximation. We establish sufficient feature-budget conditions under which RaDR preserves the order of our excess-risk upper bound for exact two-stage kernel distribution regression while maintaining computational efficiency. We further propose Adaptive Random Feature Distribution Regression (A-RaDR), a unified score-guided framework that adaptively allocates random features according to downstream learning objectives. Experiments on synthetic benchmarks and 41 real-world datasets demonstrate that RaDR delivers substantial computational savings while maintaining competitive predictive performance, whereas A-RaDR further improves learning accuracy in distribution regression while retaining the computational speed up. Together, these results establish a practical and theoretically grounded framework for adaptive, scalable distribution learning

Introduction

Machine learning has long been built upon a simple but powerful abstraction: data as vectors [61, 21]. From classical linear models and support vector machines to modern deep neural networks, the dominant paradigm in machine learning has traditionally assumed that each observation can be represented as a fixed-dimensional point in a Euclidean space. This vector-centric viewpoint has driven decades of progress in computer vision, speech recognition, scientific computing, and predictive modeling. Even as machine learning architectures have grown increasingly sophisticated, much of modern learning theory and algorithm design remains fundamentally rooted in vector-valued representations [29, 7].

Yet many real-world learning problems are fundamentally not vector-valued. In numerous scientific and engineering domains, each observation is more naturally represented as a collection, set, bag, or probability distribution rather than a single feature vector [16, 35]. A medical diagnosis may rely on an entire collection of pathology images or patient measurements [9, 23]; a manufacturing process may be characterized by particle size distributions evolving over time [59]; a sensor network may generate streams of stochastic observations [15]; and modern language models themselves process increasingly long contexts that are intrinsically set- or sequence-valued. In such settings, the statistical object of interest is no longer an individual sample, but the underlying distribution generating those samples. This shift has fueled growing interest in learning paradigms capable of operating directly on distributions, where the input to the learning system is itself a probability measure estimated through finite observations [48, 19, 40].

Existing approaches to learning from bags, sets, and distributions can be broadly organized into two major paradigms. The first is the multi-instance learning (MIL) paradigm, which treats each input as a finite unordered bag of instances [16, 35, 19], is increasingly dominated by deep neural architectures [23, 66, 30]. The second is distribution regression, which explicitly formulates the problem as supervised learning from probability distributions to responses [41, 57, 40]. MIL is algorithmically flexible but often statistically opaque and data hungry, whereas distribution regression is principled, stable, and theoretically well understood, but limited by computation.

Distribution regression formulates the set-valued learning problems as supervised learning problems in which each input is an underlying probability distribution observed through a finite bag of samples. Early nonparametric formulations treated the problem directly as regression on probability measures and established consistency under distribution-free assumptions, often through density-based or kernel-kernel constructions [41]. A more influential line of work later represented distributions through kernel mean embeddings (KME), which map probability measures into a reproducing kernel Hilbert space (RKHS) [48, 52, 53, 40, 36]. This embedding viewpoint provides a natural way to compare distributions through inner products, distances, and kernels on embedded measures, and has become the dominant foundation for kernel-based distribution regression. Within this framework, support measure machines introduced a discriminative learning formulation over probability measures [38], while subsequent two-stage learning theory established、】【 consistency and convergence guarantees when both the distributions and the samples from each distribution are random [56, 57, 6]. These developments make kernel mean embedding-based distribution regression especially attractive in scientific and safety-critical settings.

Despite these theoretical guarantee, computation remains a major obstacle in large scale applications. Kernel mean embedding-based distribution regression typically requires constructing and manipulating large kernel matrices over both instances and bags, so the cost grows rapidly with the number of distributions and the number of samples per distribution. As a result, the very framework that is arguably the most principled default for distributional supervised learning becomes difficult to deploy in large-scale settings. Moreover, as in conventional kernel learning, performance remains tightly coupled to the choice of kernel [20, 13, 12]. A poorly chosen kernel may yield a representation that is statistically ineffective, while adaptive kernel selection or learning procedures can introduce additional optimization overhead.

Previous studies on kernel approximations for distribution regression have primarily emphasized computational efficiency and practical implementation [18, 37, 62, 1], while recent work on adaptive distribution kernels has focused on learning representations suited to downstream tasks [24]. These developments have largely pursued efficiency and adaptivity along separate lines, the integration of both scalability and adaptivity into a common framework receives much less attention. Building on these efforts, we develop a finite-sample analysis of random feature distribution regression to theoretically justify its computation and performance trade-off and extend the framework through score-guided feature selection to address scalability and adaptivity together. The paper is organized around three main contributions.

  1. In § 3, we develop a finite-sample analysis of Random Feature Distribution Regression (RaDR) in the classical two-stage sampling setting. Our perturbation analysis separates the effects of distribution-level sampling, finite within-bag sampling, and random feature approximation. For kernel ridge regression with a linear distribution kernel, we establish sufficient feature-budget conditions under which RaDR preserves the order of the excess-risk upper bound of exact two-stage kernel distribution regression, while reducing the computational cost.

  1. In § 4, we propose Adaptive Random Feature Distribution Regression (A-RaDR), a unified score-guided framework for achieving adaptive kernels in distribution regression. Our formulation encompasses several important adaptive sampling strategies, including kernel alignment, leverage scores, and distribution variance, within a single framework.

  1. In § 5, we conduct extensive experiments on synthetic benchmarks and 41 real-world datasets. The results demonstrate that RaDR substantially improves computational efficiency while maintaining competitive predictive performance, and that A-RaDR further improves prediction accuracy in the low-feature regime through adaptive feature selection.

Problem Setup and State-of-the-Art

In this section, we review the mathematical foundations of kernel mean embeddings and two-stage kernel distribution regression. After introducing their definitions and formulations, we highlight the computational bottlenecks that motivate the scalable approach introduced in the following sections.

Notation used throughout the paper

Before proceeding, we summarize the main notation used in the remainder of the paper. Scalars or functions are denoted by non-bold letters such as xx, yy, and kk. Vectors are denoted by bold lower-case letters such as x\mathbf{x} and y\mathbf{y}, and matrices by bold upper-case letters such as X\mathbf{X} and K\mathbf{K}. Calligraphic letters such as X\mathcal{X}, Y\mathcal{Y}, and Hk\mathcal{H}_k denote spaces, sets, or function classes. Probability measures on a space X\mathcal{X} are written as P\mathbb{P}, Q\mathbb{Q}, while empirical bags are written as Xi={xij}j=1Ni\mathcal{X}_i = \{\mathbf{x}_{ij}\}_{j=1}^{N_i}. We denote inner product with ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle, norm with ∥⋅∥\lVert\cdot\rVert, expectation with E[⋅]\mathbb{E}[\cdot], with appropriate subscripts specifying their associated space or measure.

We use nn for the number of bags or distributions in the training set. The quantity NiN_i denotes the size of the iith training bag, and NtN_t denotes the test bag size. The embedding kernel is written as kk, with RKHS Hk\mathcal{H}_k, and the distribution kernel is written as KK. The population embedding of P\mathbb{P} is μP\mu_{\mathbb{P}}, while its empirical embedding from bag Xi\mathcal{X}_i is μ^i\hat{\mu}_i.

Distribution regression as learning over probability measures

Distribution regression considers a supervised learning problem with inputs drawn from a collection of probability measures on an instance space X\mathcal{X} and outputs taking values in Y⊆R\mathcal{Y} \subseteq\mathbb{R} for regression, or in a finite label space for classification. Let

(Pi,yi)∈M1+(X)×Y,i=1,…,n,(\mathbb{P}_i,y_i) \in\mathcal{M}_1^+(\mathcal{X}) \times\mathcal{Y}, \qquad i=1,\ldots,n,

denote the population-level training pairs, where M1+(X)\mathcal{M}_1^+(\mathcal{X}) is a suitable class of Borel probability measures on X\mathcal{X}. The target is to learn a function

f⋆:M1+(X)→Yf^\star:\mathcal{M}_1^+(\mathcal{X}) \to\mathcal{Y}

from the observed training data and then predict the response associated with a new input distribution P\mathbb{P}.

In practice, however, the distributions Pi\mathbb{P}_i are not directly observed. Instead, for each ii, one only observes a bag of samples

Xi={xi1,…,xiNi},xij∼i.i.d.Pi,\mathcal{X}_i = \{\mathbf{x}_{i1},\ldots,\mathbf{x}_{iN_i}\}, \qquad\mathbf{x}_{ij} \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \mathbb{P}_i,

where NiN_i is the bag size for the iith distribution. The actual training data therefore take the form

Dn={(Xi,yi)}i=1n.\mathcal{D}_n = \{(\mathcal{X}_i,y_i)\}_{i=1}^{n}.

As mentioned in the beginning of this Section, we reserve nn for the number of distributions, and NiN_i for the number of instances in bag ii, since the distinction between these two sampling levels becomes important both computationally and statistically. In particular, later theoretical results will separately track the effect of finite bag sampling and finite random feature approximation.

Distribution regression formulation can therefore be stated as follows: given finitely many bags X1,…,Xn\mathcal{X}_1,\ldots,\mathcal{X}_n and corresponding responses y1,…,yny_1,\ldots,y_n, estimate a predictor f^\hat{f} such that f^(X)\hat{f}(\mathcal{X}) approximates the response associated with the underlying distribution that generated the bag X\mathcal{X}. In a nutshell, the covariates are latent probability measures, and the observed bags are noisy finite-sample surrogates for those measures.

Kernel mean embeddings of distributions

A standard nonparametric way to represent probability distributions is through kernel mean embeddings [48, 40]. Let k:X×X→Rk : \mathcal{X} \times\mathcal{X} \to\mathbb{R} be a positive definite kernel with associated RKHS Hk\mathcal{H}_k. For a distribution P∈M1+(X)\mathbb{P} \in\mathcal{M}^{+}_{1}(\mathcal{X}) satisfying the usual integrability condition, its kernel mean embedding is defined as

μP=∫Xk(x,⋅) dP(x)∈Hk.\mu_{\mathbb{P}} = \int_{\mathcal{X}} k(\mathbf{x},\cdot)\,\mathrm{d}\mathbb{P}(\mathbf{x}) \in\mathcal{H}_k.

The embedding μP\mu_{\mathbb{P}} is the RKHS element that averages the feature map k(x,⋅)k(\mathbf{x},\cdot) under P\mathbb{P}. Equivalently, for any h∈Hkh \in\mathcal{H}_k,

⟨h,μP⟩Hk=∫Xh(x) dP(x),\langle h,\mu_{\mathbb{P}}\rangle_{\mathcal{H}_k} = \int_{\mathcal{X}} h(\mathbf{x})\,\mathrm{d}\mathbb{P}(\mathbf{x}),

where ⟨⋅,⋅⟩Hk\langle\cdot,\cdot\rangle_{\mathcal{H}_k} denotes the inner product in RKHS Hk\mathcal{H}_k, so the mean embedding acts as a linear functional encoding expectations of RKHS test functions. When kk is characteristic [53, 46], the map P↦μP\mathbb{P} \mapsto\mu_{\mathbb{P}} is injective, meaning that the embedding uniquely determines the distribution. In another word, kernel mean embedding is lossless. This lossless embedding property is one of the key reasons behind the popularity of KME in distribution regression.

Since the population distributions are often not directly available in practice, one replaces μPi\mu_{\mathbb{P}_i} by its empirical counterpart

μ^i=1Ni∑j=1Nik(xij,⋅).\hat{\mu}_i = \frac{1}{N_i}\sum_{j=1}^{N_i} k(\mathbf{x}_{ij},\cdot).

This is simply the average feature map over the bag Xi\mathcal{X}_i. The empirical embedding is unbiased for the population embedding and converges in RKHS norm under standard assumptions. Accordingly, the first stage of kernel distribution regression consists of replacing each latent distribution Pi\mathbb{P}_i by the empirical element μ^i∈Hk\hat{\mu}_i \in\mathcal{H}_k.

Kernels on embedded distributions

It is worth noting that in the distribution regression pipeline, there are usually two kernels involved. In the previous subsection, we have introduced the embedding kernel kk acting on instances x∈X\mathbf{x} \in\mathcal{X} to represent distributions in Hk\mathcal{H}_k. A second kernel, defined on these embedded objects, is then used for the downstream supervised task, we term the downstream supervised learning kernel as distribution kernel to differentiate it from embedding kernel kk. Let K:Hk×Hk→RK : \mathcal{H}_k \times\mathcal{H}_k \to\mathbb{R} be a positive definite kernel on the embedding space. Then the induced distribution kernel is K(P,Q)=K(μP,μQ)K(P, Q) = K(\mu_P, \mu_Q). In practice one uses the empirical approximation K^ij=K(μ^i,μ^j)\widehat{K}_{ij} = K(\widehat{\mu}_i, \widehat{\mu}_j). A common and important example is the Gaussian kernel on the RKHS:

K(μ,ν)=exp⁡(−∥μ−ν∥Hk22τ2),K(\mu, \nu) = \exp\left(-\frac{\lVert\mu-\nu\rVert_{\mathcal{H}_k}^{2}}{2\tau^{2}}\right),

where τ>0\tau> 0 is a bandwidth parameter. Since

∥μP−μQ∥Hk2=Ex,x′∼Pk(x,x′)+Ez,z′∼Qk(z,z′)−2Ex∼P,z∼Qk(x,z),\lVert\mu_P-\mu_Q\rVert_{\mathcal{H}_k}^{2} = \mathbb{E}_{x,x'\sim P}k(x,x') + \mathbb{E}_{z,z'\sim Q}k(z,z') - 2\mathbb{E}_{x\sim P,z\sim Q}k(x,z),

the induced kernel depends on pairwise embedding kernel evaluations across and within bags. Its empirical version is obtained by replacing expectations with bag averages:

∥μ^i−μ^j∥Hk2=1Ni2∑a=1Ni∑b=1Nik(xia,xib)+1Nj2∑a=1Nj∑b=1Njk(xja,xjb)\lVert\widehat{\mu}_i-\widehat{\mu}_j\rVert_{\mathcal{H}_k}^{2} = \frac{1}{N_i^{2}}\sum_{a=1}^{N_i}\sum_{b=1}^{N_i}k(x_{ia},x_{ib}) + \frac{1}{N_j^{2}}\sum_{a=1}^{N_j}\sum_{b=1}^{N_j}k(x_{ja},x_{jb})
−2NiNj∑a=1Ni∑b=1Njk(xia,xjb).-\frac{2}{N_iN_j}\sum_{a=1}^{N_i}\sum_{b=1}^{N_j}k(x_{ia},x_{jb}).

This expression already reveals the main computational difficulty, that even a single distribution kernel evaluation requires summing over all cross-bag instance pairs.

More generally, the second-stage kernel KK may be chosen according to the downstream task. Linear kernels, radial kernels, and others are all possible. In this paper, we will focus on the linear kernel K(μ,ν)=⟨μ,ν⟩HKK(\mu,\nu) = \langle\mu,\nu\rangle_{\mathcal{H}_K}, for theoretical analysis, and include Gaussian distribution kernel for numerical studies.

Two-stage distribution regression

The preceding construction leads to a natural two-stage learning paradigm (Szabó et al. [57]). In the first stage, each input distribution is estimated from a finite bag through its empirical embedding, and transformed into an potentially infinite-dimensional RKHS representation via the embedding kernel kk. In the second stage, a supervised kernel learning method is applied to the resulting embeddings via the distribution kernel KK. For kernel ridge regression, this yields the predictor

f^(P)=∑i=1nαiK(μ^i,μP),\widehat{f}(\mathbb{P}) = \sum_{i=1}^{n}\alpha_iK(\widehat{\mu}_i,\mu_{\mathbb{P}}),

where the coefficient vector α=(α1,…,αn)⊤\alpha= (\alpha_1,\ldots,\alpha_n)^{\top} solves

(KN+λnIn)α=y,(K_N+\lambda n I_n)\alpha= y,

with

References

  1. [1]Jose E Adsuara, Adrián Pérez-Suay, Jordi Muñoz-Marí, Anna Mateo-Sanchis, Maria Piles, and Gustau Camps-Valls. Nonlinear distribution regression for remote sensing applications. IEEE Transactions on Geoscience and Remote Sensing, 57(12):10025–10035, 2019.arxiv.org/abs/2012.06377
  2. [2]Stuart Andrews, Ioannis Tsochantaridis, and Thomas Hofmann. Support vector machines for multiple-instance learning. In The 2003 Advances in Neural Information Processing Systems, volume 15, Vancouver, Canada, Dec 2003.
  3. [3]Haim Avron, Michael Kapralov, Cameron Musco, Christopher Musco, Ameya Velingker, and Amir Zandieh. Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees. In The 2017 International Conference on Machine Learning, Sydney, Australia, Aug 2017. PMLR.arxiv.org/abs/1804.09893
  4. [4]Francis Bach. On the equivalence between kernel quadrature rules and random feature expansions. Journal of Machine Learning Research, 18(21):1–38, 2017.
  5. [5]Francis R Bach, Gert RG Lanckriet, and Michael I Jordan. Multiple kernel learning, conic duality, and the SMO algorithm. In The 2004 International Conference on Machine Learning, page 6, Banff, Canada, Jul 2004.DOI
  6. [6]François Bachoc, Louis Béthune, Alberto González-Sanz, and Jean-Michel Loubes. Improved learning theory for kernel distribution regression with two-stage sampling. The Annals of Statistics, 53(4):1753–1782, 2025.arxiv.org/abs/2308.14335
  7. [7]Yoshua Bengio, Ian Goodfellow, Aaron Courville, et al. Deep learning, volume 1. MIT press Cambridge, MA, USA, 2017.DOI
  8. [8]Forrest Briggs, Balaji Lakshminarayanan, Lawrence Neal, Xiaoli Z. Fern, Raviv Raich, Sarah J. K. Hadley, Adam S. Hadley, and Matthew G. Betts. Acoustic classification of multiple simultaneous bird species: A multi-instance multi-label approach. The Journal of the Acoustical Society of America, 131(6):4640–4650, 2012.DOI
  9. [9]Gabriele Campanella, Matthew G Hanna, Luke Geneslaw, Allen Miraflor, Vitor Werneck Krauss Silva, Klaus J Busam, Edi Brogi, Victor E Reuter, David S Klimstra, and Thomas J Fuchs. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine, 25(8):1301–1309, 2019.DOI
  10. [10]Andrea Caponnetto and Ernesto De Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7(3):331–368, 2007. doi: 10.1007/s10208-006-0196-8.DOI
  11. [11]Yixin Chen, Jinbo Bi, and James Z. Wang. MILES: Multiple-instance learning via embedded instance selection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12):1931–1947, 2006.pubmed.ncbi.nlm.nih.gov/17108368
  12. [12]Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Two-stage learning kernel algorithms. In The 2010 International Conference on Machine Learning, Haifa, Israel, Jun 2010.
  13. [13]Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Algorithms for learning kernels based on centered alignment. Journal of Machine Learning Research, 13:795–828, 2012.arxiv.org/abs/1203.0550
  14. [14]Nello Cristianini, John Shawe-Taylor, Andre Elisseeff, and Jaz Kandola. On kernel-target alignment. In The 2001 Advances in Neural Information Processing Systems, volume 14, British Columbia, Canada, Dec 2001.
  15. [15]Don Dennis, Chirag Pabbaraju, Harsha Vardhan Simhadri, and Prateek Jain. Multiple instance learning for efficient sequential data classification on resource-constrained devices. In The 2018 Advances in Neural Information Processing Systems, volume 31, Montréal, canada, Dec 2018.
  16. [16]Thomas G Dietterich, Richard H Lathrop, and Tomás Lozano-Pérez. Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1-2):31–71, 1997.DOI
  17. [17]Petros Drineas, Malik Magdon-Ismail, Michael W Mahoney, and David P Woodruff. Fast approximation of matrix coherence and statistical leverage. Journal of Machine Learning Research, 13 (1):3475–3506, 2012.
  18. [18]Seth R Flaxman, Yu-Xiang Wang, and Alexander J Smola. Who supported Obama in 2012? Ecological inference through distribution regression. In The 2015 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 289–298, Sydney, Australia, Aug 2015.
  19. [19]James Foulds and Eibe Frank. A review of multi-instance learning assumptions. The Knowledge Engineering Review, 25(1):1–25, 2010.DOI
  20. [20]Mehmet Gönen and Ethem Alpaydın. Multiple kernel learning algorithms. Journal of Machine Learning Research, 12:2211–2268, 2011.DOI
  21. [21]Trevor Hastie, Robert Tibshirani, Jerome Friedman, et al. The elements of statistical learning: Data mining, inference, and prediction, 2009.
  22. [22]Thomas Hofmann, Bernhard Schölkopf, and Alexander J Smola. Kernel methods in machine learning. The Annals of Statistics, 36(3):1171–1220, 2008.
  23. [23]Maximilian Ilse, Jakub Tomczak, and Max Welling. Attention-based deep multiple instance learning. In The 2018 International Conference on Machine Learning, Stockholm, Sweden, Jul 2018. PMLR.
  24. [24]Oleksii Kachaiev and Stefano Recanatesi. Learning to embed distributions via maximum kernel entropy. In The 2024 Advances in Neural Information Processing Systems, volume 37, Vancouver, Canada, Dec 2024.arxiv.org/abs/2408.00549
  25. [25]Melih Kandemir, Chong Zhang, and Fred A. Hamprecht. Empowering multiple instance histopathology cancer diagnosis by cell graphs. In The 2014 International Conference on Medical Image Computing and Computer-Assisted Intervention, Boston, USA, Sep 2014. Springer.pubmed.ncbi.nlm.nih.gov/25485383
  26. [26]Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.arxiv.org/abs/1412.6980
  27. [27]Gert RG Lanckriet, Nello Cristianini, Peter Bartlett, Laurent El Ghaoui, and Michael I Jordan. Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research, 5(Jan):27–72, 2004.
  28. [28]Ho Chung Leon Law, Danica J Sutherland, Dino Sejdinovic, and Seth Flaxman. Bayesian approaches to distribution regression. In The 2018 International Conference on Artificial Intelligence and Statistics, pages 1167–1176, Canary Islands, Spain, Apr 2018. PMLR.arxiv.org/abs/1705.04293
  29. [29]Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436–444, 2015.DOI
  30. [30]Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In The 2019 International Conference on Machine Learning, Long Beach, USA, Jun 2019. PMLR.arxiv.org/abs/1810.00825
  31. [32]Zhu Li, Jean-Francois Ton, Dino Oglic, and Dino Sejdinovic. Towards a unified analysis of random Fourier features. In The 2019 International Conference on Machine Learning, pages 3905–3914, Long Beach, USA, Jun 2019. PMLR.
  32. [33]Zhu Li, Jean-Francois Ton, Dino Oglic, and Dino Sejdinovic. Towards a unified analysis of random Fourier features. Journal of Machine Learning Research, 22(108):1–51, 2021.
  33. [34]Michael W Mahoney. Randomized algorithms for matrices and data. Foundations and Trends in Machine Learning, 3(2):123–224, 2011.
  34. [35]Oded Maron and Tomás Lozano-Pérez. A framework for multiple-instance learning. In The 1997 Advances in Neural Information Processing Systems, volume 10, Denver, USA, Dec 1997.
  35. [36]Dimitri Meunier, Massimiliano Pontil, and Carlo Ciliberto. Distribution regression with sliced Wasserstein kernels. In The 2022 International Conference on Machine Learning, Baltimore, USA, Jul 2022. PMLR.arxiv.org/abs/2202.03926
  36. [37]Jovana Mitrovic, Dino Sejdinovic, and Yee-Whye Teh. DR-ABC: Approximate Bayesian computation with kernel-based distribution regression. In The 2016 International Conference on Machine Learning, pages 1482–1491, New York, USA, Jun 2016. PMLR.arxiv.org/abs/1602.04805
  37. [38]Krikamol Muandet, Kenji Fukumizu, Francesco Dinuzzo, and Bernhard Schölkopf. Learning from distributions via support measure machines. In The 2012 Advances in Neural Information Processing Systems, volume 25, Long Beach, USA, Dec 2012.arxiv.org/abs/1202.6504
  38. [39]Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In The 2013 International Conference on Machine Learning, Atlanta, USA, Jun 2013. PMLR.arxiv.org/abs/1301.2115
  39. [40]Kr ïkamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, and Bernhard Schölkopf. Kernel mean embedding of distributions: A review and beyond. Foundations and Trends in Machine Learning, 10(1-2):1–141, 2017.DOI
  40. [41]Barnabás Póczos, Alessandro Rinaldo, Aarti Singh, and Larry Wasserman. Distribution-free distribution regression. In The 2013 International Conference on Artificial Intelligence and Statistics, Scottsdale, USA, Apr 2013. PMLR.arxiv.org/abs/1302.0082
  41. [42]Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In The 2007 Advances in Neural Information Processing Systems, volume 20, Vancouver, Canada, Dec 2007.
  42. [43]Alessandro Rudi and Lorenzo Rosasco. Generalization properties of learning with random features. In The 2017 Advances in Neural Information Processing Systems, volume 30, Long Beach, USA, Dec 2017.arxiv.org/abs/1602.04474
  43. [44]Walter Rudin. Fourier analysis on groups. Courier Dover Publications, 2017.
  44. [46]Carl-Johann Simon-Gabriel and Bernhard Schölkopf. Kernel distribution embeddings: Universal kernels, characteristic kernels and kernel metrics on distributions. Journal of Machine Learning Research, 19(44):1–29, 2018.
  45. [47]Aman Sinha and John C Duchi. Learning kernels with random features. In The 2016 Advances in Neural Information Processing Systems, volume 29, Barcelona, Spain, Dec 2016.
  46. [48]Alex Smola, Arthur Gretton, Le Song, and Bernhard Schölkopf. A Hilbert space embedding for distributions. In The 2007 International Conference on Algorithmic Learning Theory, Sendai, Japan, Oct 2007. Springer.
  47. [49]Alexander J Smola and Bernhard Schölkopf. Learning with kernels, volume 4. GMD-Forschungszentrum Informationstechnik Berlin, Germany, 1998.
  48. [50]Sören Sonnenburg, Gunnar Rätsch, Christin Schäfer, and Bernhard Schölkopf. Large scale multiple kernel learning. The Journal of Machine Learning Research, 7:1531–1565, 2006.DOI
  49. [51]Ashwin Srinivasan, Stephen Muggleton, and Ross D. King. Comparing the use of background knowledge by inductive logic programming systems. In The 1995 International Workshop on Inductive Logic Programming, Leuven, Belgium, Sep 1995.
  50. [52]Bharath K Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf, and Gert RG Lanckriet. Hilbert space embeddings and metrics on probability measures. Journal of Machine Learning Research, 11:1517–1561, 2010.arxiv.org/abs/0907.5309
  51. [53]Bharath K Sriperumbudur, Kenji Fukumizu, and Gert RG Lanckriet. Universality, characteristic kernels and RKHS embedding of measures. Journal of Machine Learning Research, 12(7):2389–2410, 2011.arxiv.org/abs/1003.0887
  52. [54]Yitong Sun, Anna Gilbert, and Ambuj Tewari. But how does it work in theory? Linear SVM with random features. In The 2018 Advances in Neural Information Processing Systems, volume 31, Montréal, canada, Dec 2018.
  53. [55]Danica J. Sutherland and Jeff Schneider. On the error of random fourier features. In The 2015 Conference on Uncertainty in Artificial Intelligence, page 862–871, Arlington, Virginia, USA, 2015.arxiv.org/abs/1506.02785
  54. [56]Zoltán Szabó, Arthur Gretton, Barnabás Póczos, and Bharath Sriperumbudur. Two-stage sampled learning theory on distributions. In The 2015 International Conference on Artificial Intelligence and Statistics, San Diego, USA, May 2015. PMLR.arxiv.org/abs/1402.1754
  55. [57]Zoltán Szabó, Bharath K Sriperumbudur, Barnabás Póczos, and Arthur Gretton. Learning theory for distribution regression. Journal of Machine Learning Research, 17(152):1–40, 2016.
  56. [58]Qingping Tao, Stephen Scott, N. V. Vinodchandran, and Thomas Takeo Osugi. SVM-based generalized multiple-instance learning via approximate box counting. In The 2004 International Conference on Machine Learning, page 101, Banff, Canada, Jul 2004. ACM.DOI
  57. [59]Daan Van Hauwermeiren, Michiel Stock, Thomas De Beer, and Ingmar Nopens. Predicting pharmaceutical particle size distributions using kernel mean embedding. Pharmaceutics, 12(3):271, 2020.DOI
  58. [60]Vladimir N Vapnik. The support vector method. In The 1997 International Conference on Artificial Neural Networks, Lausanne, Switzerland, Oct 1997. Springer.DOI
  59. [61]Vladimir N Vapnik. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5):988–999, 1999.DOI
  60. [62]Di Wang, Ping Wang, Cong Wang, Shuo Zhuang, and Junzhi Shi. A conformal prediction inspired approach for distribution regression with random Fourier features. Applied Soft Computing, 97:106807, 2020.DOI
  61. [63]Yinsong Wang and Shahin Shahrampour. Orcca: Optimal randomized canonical correlation analysis. IEEE Transactions on Neural Networks and Learning Systems, 34(8):5024–5036, 2021.DOI
  62. [64]Christopher KI Williams and Matthias Seeger. Using the Nyström method to speed up kernel machines. In The 2000 Advances in Neural Information Processing Systems, Denver, USA, Dec 2001. MIT Press.
  63. [65]Tianbao Yang, Yu-Feng Li, Mehrdad Mahdavi, Rong Jin, and Zhi-Hua Zhou. Nystrom method vs random Fourier features: A theoretical and empirical comparison. In The 2012 Advances in Neural Information Processing Systems, volume 25, Lake Tahoe, USA, Dec 2012.
  64. [66]Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. In The 2017 Advances in Neural Information Processing Systems, volume 30, Long Beach, USA, Dec 2017.
  65. [67]Kai Zhang, Ivor W Tsang, and James T Kwok. Improved Nyström low-rank approximation and error analysis. In The 2008 International Conference on Machine Learning, Helsinki, Finland, Jul 2008.DOI
  66. [68]Tong Zhang. Learning bounds for kernel regression using effective data dimensionality. Neural Computation, 17(9):2077–2098, 2005. doi: 10.1162/0899766054323008.DOI
  67. [69]Xiao Zhang and Shizhong Liao. Incremental randomized sketching for online kernel learning. In The 2019 International Conference on Machine Learning, Long Beach, USA, Jun 2019. PMLR.

Paper details

vlnc:W_n6e64uak
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents