betaFeedback

WorksMathematics

R2-GP: Variance-Calibrated Priors for Additive Gaussian Processes

Abstract

We develop R2-GP, an R2D2-type prior for Gaussian process (GP) generalised additive models (GAMs) that calibrates explained variance (R2R^2) for Matérn covariance kernels. The function variance for standard Matérn kernels depends jointly on the effect magnitude and length-scale. As such, smooth and rough components with identical amplitudes contribute very different amounts of variation. With independent priors on component amplitudes, this variation results in variance overestimation, especially in higher dimensions. We overcome this by taking direct control over each component's expected contribution to prior explained variance. We do this by rescaling each component based on its length scale and observed inputs, and modelling each rescaled variance as a share of the function variance induced by a global R2R^2 prior. This allows R2-GP to scale well to many components. To also scale to large sample sizes, we extend R2-GP to Hilbert space approximation methods, a framework for kernels exploiting their spectral decompositions for fast and efficient inference. Our simulations and real-world case studies demonstrate superior generalisation and parameter recovery over independent variance priors.

Introduction

Gaussian processes (GPs) are a class of probabilistic methods for modelling non-linear functions. They do so by assuming a multivariate Gaussian prior distribution over a finite set of these functions and are specified through a mean function and a covariance kernel (Williams and Rasmussen, [27]). Common choices of covariance kernel involve the Matérn class of functions (Rasmussen and Williams, [19]), defined using a length scale parameter and a magnitude parameter (also called GP marginal variance). The former controls the correlation decay rate between two arbitrary inputs while the latter controls prior variance of the functions at a single input.

GP priors have been specifically used in generalised additive models (GAMs; Hastie and Tibshirani, [8, 9]) as an approach to model smooth functions (Wood, [28]). In GAMs, a response y∈RNy \in\mathbb{R}^{N} is modelled using multiple covariates X∈RN×DX \in\mathbb{R}^{N \times D} such that

yi=μ0+∑d=1Dfd(xid)+ϵi,ϵi∼N(0,σ2),(1)y_i = \mu_0 + \sum_{d=1}^{D} f_d(x_{id}) + \epsilon_i,\qquad\epsilon_i \sim N(0,\sigma^2), \tag*{(1)}

for i=1,…,Ni = 1,\ldots,N, where μ0\mu_0 is the intercept, ϵi\epsilon_i is the residual error and fdf_d is a GP specific to the dd-th covariate. Additive GPs arise as a special case of the general class of models presented in Duvenaud et al. [4].

In GAMs, modelling a response with many covariates introduces a variance explosion problem familiar from high-dimensional linear regression. If each of the DD components receives an independent variance prior with positive mass on non-zero values, the total signal variance accumulates as DD grows. Consequently, the implied prior on the proportion of explained variance R2R^2 pathologically concentrates near one. This may lead to overfitting and contradict the modeller’s domain knowledge about R2R^2, rendering the model predictively inconsistent [20].

Preventing this variance accumulation requires explicitly controlling the model’s total variance. In regression contexts, a natural approach is to place a prior on R2R^2 and partition it across the components, analogous to the R2D2 prior for linear models [31]. The mapping between a component’s magnitude parameter and the observed function variance one wishes to control is however blurred by the simultaneous impact of the length scale and the input design. Because of this entanglement, smooth and rough components with identical magnitudes contribute vastly different amounts to observed function variation (Figure 1).

GP realisations and normaliser

Figure 1. GP Realisations and Normaliser Q(ρ)Q(\rho). Realisations of a zero-mean GP with an SE kernel and unit marginal variance on x∈[−1,1]x \in[-1,1]. The shaded band is ±1\pm1 marginal standard deviation. Panels (a) to (c) hold the marginal variance fixed and vary the length scale, and report the sample standard deviation averaged over 4000 realisations. As ρ\rho grows, the signal variance falls even though the marginal variance is unchanged. Panel (d) shows the resulting normaliser Q(ρ)Q(\rho) of (5).

We overcome this structural issue with our R2-GP prior. By rescaling each component with a normaliser which is derived from its length scale and the input design, R2-GP forces the component’s magnitude to exactly match its expected variation across the data. This calibration allows a global R2R^2 prior which is partitioned with a simplex prior that forces components to compete for a share of the variance budget.

A drawback of this normalisation is the computational burden it adds to posterior inference. Since the normaliser depends on the length scale, it must be recomputed at every gradient evaluation when using MCMC methods like Hamiltonian Monte Carlo [16, 10]. For an exact GP, this requires O(N2)O(N^2), added to the standard O(N3)O(N^3) cost of factorising the exact covariance matrix.

We solve this scalability issue by extending the R2-GP prior to Hilbert space approximation methods (HSGPs) [24, 21, 15]. HSGPs exploit the spectral decomposition of the kernel using basis functions MM for large sample scalability and have been extended to GAMs efficiently [25]. With the extension to HSGPs based on M≪NM \ll N basis functions, computing the normaliser simplifies to a single pre-computation of O(NM)O(NM) and O(M)O(M).

Code to replicate the experiments is available at https://github.com/davkoh/r2-gp.

Contributions.

  1. We develop R2-GP by extending the R2D2 family of priors to GPs for GAMs. We derive the necessary normalising approach of the kernel such that a prior over the GP’s R2R^2 maps analytically to its expected sample variance. A Dirichlet decomposition of R2R^2 into contributions of its GAM components allows its application to high dimensions.

  1. We derive R2-GP prior for the Hilbert space approximation and show that the normaliser admits an exact finite-sample expression requiring O(NM)O(NM) preparation and O(M)O(M) and scales to high dimensions.

  1. Through simulations and real-world data, we show that the R2-GP prior outperforms independent priors for additive HSGPs in prediction and parameter recovery and sampling efficiency, and is competitive with alternate shrinkage priors.

Related Work. Priors placed on R2R^2, since the original Zhang et al. [31], have been extended to multilevel models [1], generalised linear mixed models [30], autoregressions [11], spatial models [29] and grouped predictors ([2]. The present paper takes the variance allocation mechanism from that literature and addresses the question it leaves on how to extend this to GPs.

Closest to our normalisation approach is the standardisation procedure of Ferrari and Ventrucci [5], who scale variance partitioning priors in latent Gaussian models by the variance of a zero-mean-constrained effect over an assumed covariate distribution, evaluated at fixed values of unknown kernel parameters such as the range of a Matérn process. Our normaliser is instead a function of the length scale and is recomputed at every evaluation inside the sampler, which is what makes the O(M)O(M) identity of Lemma A.1 particularly efficient. Penalised complexity priors for Gaussian random fields ([22, 6]) shrink towards a simpler base model, whereas the normalisation here fixes the units in which the scale parameter is expressed, and the two could in principle be combined.

The remainder of the paper is organised as follows. § 2 derives the normalised additive prior and its properties, among them the implied prior on explained variance. § 3 presents our simulation studies and § 4 demonstrates a real-world application. Proofs and additional results are collected in Appendix.

Methods

In our framework, the dd-th column of the design matrix XX holds the observed values x1d,…,xNdx_{1d}, \ldots, x_{Nd} of covariate dd. We use GAM GP prior with independent processes per covariate,

fd=αdgd,gd∼GP(0,k(⋅,⋅;ρd)),(2)f_d = \alpha_d g_d,\qquad g_d \sim\mathcal{GP}(0, k(\cdot,\cdot;\rho_d)), \tag*{(2)}

where kk is a stationary kernel of the Matérn family, which includes the squared exponential (SE) kernel as a limiting case, with k(x,x;ρd)=1k(x,x;\rho_d)=1, so that gdg_d has unit marginal variance. Parameter αd2\alpha_d^2 is the magnitude of the component, which equals its pointwise marginal variance, Var⁡(fd(x))=αd2\operatorname{Var}(f_d(x))=\alpha_d^2 at every input.1 In contrast, the signal variance is the expected variation that the function exhibits across the inputs, Efd[Var⁡x(fd(x)∣fd)]\mathbb{E}_{f_d}[\operatorname{Var}_x(f_d(x)\mid f_d)]. We denote by Ex\mathbb{E}_x and Var⁡x\operatorname{Var}_x the expectation and variance operators with respect to xx. In the following, we first derive the mathematical disconnect between these two variances, then provide the necessary concepts of the Hilbert space (HS) approximation to finally define an efficient finite-sample normaliser, which we ultimately use to construct the R2-GP prior.

Calibrating GPs for Explained Variance

In linear regression, placing a prior directly on the proportion of explained variance R2=Var⁡(f(x))/Var⁡(y)R^2=\operatorname{Var}(f(x))/\operatorname{Var}(y) provides an interpretable way to specify regularisation on variance parameters of regression coefficients. For GAM GPs the R2R^2 identity is,

Rmarg2=∑d=1Dαd2∑d=1Dαd2+σ2,(3)R^2_{\mathrm{marg}}=\frac{\sum_{d=1}^{D}\alpha_d^2}{\sum_{d=1}^{D}\alpha_d^2+\sigma^2}, \tag*{(3)}

where a prior πR2\pi_{R^2} maps to prior παd\pi_{\alpha_d}, conditional on σ2\sigma^2. We define Equation (3) as the marginal R2R^2, since it’s marginal of the effect of the design and any other kernel parameters. The magnitude αd2\alpha_d^2, however is not equal to the signal variance. To see this, consider the total variance of component

fd(x)f_d(x),

Var⁡(fd(x))=αd2=Efd[Var⁡x(fd(x)∣fd)]+Var⁡fd(Ex[fd(x)∣fd]).(4)\operatorname{Var}(f_d(x)) = \alpha_d^2 = \mathbb{E}_{f_d}\left[\operatorname{Var}_x(f_d(x) \mid f_d)\right] + \operatorname{Var}_{f_d}\left(\mathbb{E}_x[f_d(x) \mid f_d]\right). \tag*{(4)}

The first term is the expected variance of the function across the observed inputs, while the second term captures the variance of the average height of function realisations (See Appendix § 8.§ 1).

The second term is precisely the point of departure between a correlated GP and a linear regression. If f(x)f(x) were a linear term with mean-centred inputs, the level, Ex[fd(x)∣fd]\mathbb{E}_x[f_d(x) \mid f_d], is deterministically zero, such that marginal variance equals signal variance. In contrast, the average level of a realised GP acts as a random constant shift across all input observations. Because this shift is constant with respect to xx, it does not contribute to the function’s variation over the inputs (the signal variance), and the level itself is completely absorbed by the intercept μ0\mu_0 in (1).

The size of the level variance depends on the length scale ρd\rho_d and on the input design. Denoting by kˉd=Ex,x′[k(x,x′;ρd)]\bar{k}_d = \mathbb{E}_{x,x'}[k(x,x';\rho_d)] the average kernel correlation between two independent inputs xx and x′x', the level variance reduces to αd2kˉd\alpha_d^2\bar{k}_d, and the signal variance is therefore

Efd[Var⁡x(fd(x)∣fd)]=αd2Qd(ρd),Qd(ρd)=1−kˉd.(5)\mathbb{E}_{f_d}\left[\operatorname{Var}_x(f_d(x) \mid f_d)\right] = \alpha_d^2 Q_d(\rho_d), \qquad Q_d(\rho_d) = 1 - \bar{k}_d. \tag*{(5)}

For the Matérn family, whose correlations are positive and increase with the length scale, Qd(ρd)∈(0,1)Q_d(\rho_d) \in(0,1) (unless all inputs coincide). As Figure 1 illustrates, smoother realisations (larger ρd\rho_d) carry a larger part of their variance in the level, so that their signal variance falls well below the magnitude αd2\alpha_d^2.

Hence, we define the realised R2R^2 as the modelling target:

Rreal2=∑d=1Dαd2Qd(ρd)∑d=1Dαd2Qd(ρd)+σ2,(6)R^2_{\mathrm{real}} = \frac{\sum_{d=1}^{D} \alpha_d^2 Q_d(\rho_d)}{\sum_{d=1}^{D} \alpha_d^2 Q_d(\rho_d) + \sigma^2}, \tag*{(6)}

which, whenever Qd(ρd)<1Q_d(\rho_d) < 1, implies Rreal2<Rmarg2R^2_{\mathrm{real}} < R^2_{\mathrm{marg}}. Furthermore, because QdQ_d depends on ρd\rho_d, the implied prior on explained variance depends on the length scales, and so does the share of each component in the total signal variance.

This motivates replacing (2) with a normalised prior,

fd=αdQd(ρd)gd.(7)f_d = \frac{\alpha_d}{\sqrt{Q_d(\rho_d)}}g_d. \tag*{(7)}

Given ρd\rho_d, this is a deterministic rescaling that leaves fdf_d Gaussian and its correlation structure unchanged. By (5), it makes the signal variance of fdf_d equal to αd2\alpha_d^2 for every length scale, while the marginal variance becomes αd2/Qd(ρd)\alpha_d^2/Q_d(\rho_d). Consequently, Rmarg2R^2_{\mathrm{marg}} and Rreal2R^2_{\mathrm{real}} agree, and, as in the linear case, we can place a prior on a global R2R^2 and partition it across components with a Dirichlet distribution,

R2∼Beta⁡(μRϕR,(1−μR)ϕR),α2=σ2R21−R2,R^2 \sim\operatorname{Beta}\left(\mu_R\phi_R, (1-\mu_R)\phi_R\right), \qquad\alpha^2 = \sigma^2\frac{R^2}{1-R^2},
αd2=ψdα2,ψ∼Dirichlet⁡(a1D),(8)\alpha_d^2 = \psi_d\alpha^2, \qquad\psi\sim\operatorname{Dirichlet}(a\mathbf{1}_D), \tag*{(8)}

where μR∈(0,1)\mu_R \in(0,1) and ϕR>0\phi_R > 0 parameterise the Beta prior through its mean and precision [1]. We refer to (7) together with (8) as the R2-GP prior. Its total signal variance is α2\alpha^2, and the Dirichlet weight ψd\psi_d is the share of component dd, for any length scale value. Since the proportions in a Dirichlet distribution have strictly negative pairwise correlation, the GAM components intrinsically compete for their share of the total explained variance.

The Hilbert Space Representation

The Hilbert space (HS) approximation methods is based on the eigenvalue decomposition of the Gram matrix for a chosen kernel using its spectral density along with eigenvalues and eigenfunctions based on the GP inputs. Following Solin and Särkkä [24], we use a compact cover [−Ld,Ld][-L_d,L_d] containing the observed values of covariate dd. Based on the Dirichlet boundary conditions, we get closed form solutions for eigenvalues ω\omega and eigenfunctions ϕ\phi of the kernel such that

ϕdm(x)=1Ldsin⁡(ωdm(x+Ld)),\phi_{dm}(x) = \frac{1}{\sqrt{L_d}}\sin\left(\omega_{dm}(x+L_d)\right),
ωdm=mπ2Ld,m=1,…,M,(9)\omega_{dm} = \frac{m\pi}{2L_d}, \qquad m=1,\ldots,M, \tag*{(9)}

where MM is the number of basis functions. The matrix Φd∈RN×M\Phi_d \in\mathbb{R}^{N\times M} contains the eigenfunction values at the observed inputs, such that [Φd]im=ϕdm(xid)[\Phi_d]_{im}=\phi_{dm}(x_{id}), with ϕdm∈RN\phi_{dm}\in\mathbb{R}^N for its mm-th column. We obtain the spectral density Sdm=S(ωdm;ρd)>0S_{dm}=S(\omega_{dm};\rho_d)>0 of our chosen kernel evaluated at the mm-th frequency, collected in Sd=(Sd1,…,SdM)TS_d=(S_{d1},\ldots,S_{dM})^\mathsf{T}. For example, the spectral density of the SE kernel k(r;ρ)=exp⁡{−r2/(2ρ2)}k(r;\rho)=\exp\{-r^2/(2\rho^2)\} is of the form S(ω;ρ)=2πρexp⁡(−ρ2ω2/2)S(\omega;\rho)=\sqrt{2\pi}\rho\exp(-\rho^2\omega^2/2) obtained by solving the Fourier integral with respect to the inputs. The spectral forms of other Matérn kernels are provided in Riutort-Mayol et al. [21].

Lemma 2.1 (Finite-design covariance). Let zd∼N(0,IM)z_d \sim N(0,I_M) and gd=Φddiag⁡(Sd)zd∈RNg_d=\Phi_d\operatorname{diag}(\sqrt{S_d})z_d\in\mathbb{R}^N. Then, conditionally on XX and ρd\rho_d,

gd∼N(0,K~d),K~d=Φddiag⁡(Sd)ΦdT,(10)g_d \sim N(0,\widetilde{K}_d), \qquad\widetilde{K}_d=\Phi_d\operatorname{diag}(S_d)\Phi_d^\mathsf{T}, \tag*{(10)}

and K~d\widetilde{K}_d is symmetric positive semi-definite with rank K~d≤min⁡(N,M)\widetilde{K}_d\leq\min(N,M).

The matrix K~d\widetilde{K}_d is the finite-basis approximation to the exact kernel matrix KdK_d with [Kd]ij=k(xid−xjd;ρd)[K_d]_{ij}=k(x_{id}-x_{jd};\rho_d). Under a sufficiently large boundary and increasing MM it converges to KdK_d. We use this convergence as established and refer to Solin and Särkkä [24] for the underlying expansion. For practical guidance on choosing LdL_d and MM, we follow Riutort-Mayol et al. [21]. Since k(0)=1k(0)=1, the diagonal of K~d\widetilde{K}_d is approximately one, which is the marginal normalisation that § 2.1 identified as insufficient for modelling the signal variance of the function.

The Finite-sample Normaliser

In HS form, we define the normaliser of component dd as the expected sample variance of the unit-magnitude component gdg_d of Lemma 2.1, evaluated at the observed inputs,

Qd(ρd)≔Ez[Var⁡^N(gd)∣X,ρd],(11)Q_d(\rho_d) \coloneqq\mathbb{E}_z\left[\widehat{\operatorname{Var}}_N(g_d)\mid X,\rho_d\right], \tag*{(11)}

where Var⁡^N(v)=(N−1)−1∑i=1N(vi−vˉ)2\widehat{\operatorname{Var}}_N(v)=(N-1)^{-1}\sum_{i=1}^{N}(v_i-\bar{v})^2 is the sample variance of a vector v∈RNv\in\mathbb{R}^N and Ez\mathbb{E}_z is the expectation over the coefficients zdz_d. This is the finite-sample counterpart of (5), with the sample variance over the observed inputs in place of the variance operator Var⁡x\operatorname{Var}_x and the HSGP covariance K~d\widetilde{K}_d in place of the exact kernel. Evaluating (11) through the N×NN\times N matrix K~d\widetilde{K}_d would cost O(N2M)O(N^2M) operations each time ρd\rho_d changes. However, because the coefficients in zdz_d are independent with unit variance, the cross terms between different basis functions have expectation zero, and the normaliser simplifies to the sum of the sample variances of the basis functions, weighted by the spectral density (Lemma A.1 in Appendix § 8.§ 2),

Qd(ρd)=∑m=1MSdmVar⁡^N(ϕdm).(12)Q_d(\rho_d)=\sum_{m=1}^{M}S_{dm}\widehat{\operatorname{Var}}_N(\phi_{dm}). \tag*{(12)}

This identity allows the R2-GP prior to scale well with NN. The sample variances Var⁡^N(ϕdm)\widehat{\operatorname{Var}}_N(\phi_{dm}) of the basis columns depend on the design alone and not on any parameter, so they are precomputed once at cost O(NM)O(NM). Inside the sampler, the normaliser then reduces to an inner product of length MM between the spectral density SdS_d and these precomputed variances, which requires only O(M)O(M) work each time the length scale ρd\rho_d changes.2

Expanding the sample variance in (11) gives the HSGP counterpart of the decomposition (4),

Qd(ρd)=1N−1∑i=1N[K~d]ii−NN−1Var⁡z(gˉd),(13)Q_d(\rho_d)=\frac{1}{N-1}\sum_{i=1}^{N}[\widetilde{K}_d]_{ii}-\frac{N}{N-1}\operatorname{Var}_z(\bar{g}_d), \tag*{(13)}

where gˉd\bar{g}_d is the sample mean of the realisation gdg_d. The first term aggregates the marginal variances, each approximately one, and the second is the prior variance of the realisation’s level, which the intercept absorbs. Together they equal the average marginal variance minus the average covariance between two distinct observed inputs, the sample analogue of 1−kˉd1-\bar{k}_d in (5).

The Normalised Additive Prior in HS Form

We construct the normalised HSGPs, using αd2\alpha_d^2 from (8) as follows

fd=αdQd(ρd)gd=Φd(cd⊙zd),cd=αdSdQd(ρd),f=μ0+∑d=1Dfd,(14)\begin{aligned} f_d&=\frac{\alpha_d}{\sqrt{Q_d(\rho_d)}}g_d=\Phi_d(c_d\odot z_d),\\ c_d&=\alpha_d\sqrt{\frac{S_d}{Q_d(\rho_d)}},\qquad f=\mu_0+\sum_{d=1}^{D}f_d, \tag*{(14)} \end{aligned}

where ⊙\odot is the elementwise product. The square root and division are elementwise. Together with (8), (14) is the HSGP form of the R2-GP prior.

As for the exact GP in § 2.1, the signal variance of each component is then its allocated variance,

Ez[Var⁡^N(fd)∣X,ρd,αd]=αd2,d=1,…,D,(15)\mathbb{E}_{z}\left[\widehat{\operatorname{Var}}_N(f_d) \mid X,\rho_d,\alpha_d\right] = \alpha_d^2,\qquad d=1,\ldots,D, \tag*{(15)}

and, for independent z1,…,zDz_1,\ldots,z_D, the signal variances add up to the total budget,

Ez[Var⁡^N(∑d=1Dfd)∣X,ρ,α]=∑d=1Dαd2=α2.(16)\mathbb{E}_{z}\left[\widehat{\operatorname{Var}}_N\left(\sum_{d=1}^{D} f_d\right)\mid X,\rho,\alpha\right] = \sum_{d=1}^{D}\alpha_d^2 = \alpha^2. \tag*{(16)}

Three comments are in order that highlight the usefulness of the R2 prior for GPs. Firstly, by writing fd=Φdβdefff_d = \Phi_d\beta_d^{\mathrm{eff}} with βdeff=cd⊙zd\beta_d^{\mathrm{eff}} = c_d \odot z_d, the proposed prior is equivalent to independent centred priors on the basis weights

βdmeff∼N(0,αd2SdmQd(ρd)),m=1,…,M.(17)\beta_{dm}^{\mathrm{eff}} \sim\mathcal{N}\left(0,\frac{\alpha_d^2 S_{dm}}{Q_d(\rho_d)}\right),\qquad m=1,\ldots,M. \tag*{(17)}

Computationally, the zdz_d remains standard normal and the scale is applied through a deterministic transformation, and thus, equivalent to the non-centred parameterisation ([17]) which is well known to keep the posterior geometry well conditioned when a signal variance is small.

Second, because the deterministic rescaling in (7) applies a single global scalar 1/Qd(ρd)1/\sqrt{Q_d(\rho_d)} to all basis functions of component dd equally, it preserves the relative weighting of the spectral densities SdmS_{dm} that govern the smoothness of the function. Furthermore, because this scalar multiplier is independent of the basis index mm, it does not interact with the truncation of the series. Consequently, the normalised HSGP inherits the same convergence guarantees as the standard HSGP: as the number of basis functions MM grows, the approximate prior converges to the exact normalised GP prior.

Third, the prior on the total signal variance does not change as components are added. By (16) and (8), it equals σ2R2/(1−R2)\sigma^2R^2/(1-R^2) for every DD, so that its prior is the one placed on R2R^2. If instead each αd\alpha_d has a prior second moment s2s^2 that is free of DD, as under independent priors, it has prior mean Ds2Ds^2. Figure 2 shows the consequence for prior draws for the empirical proportion of explained variance, Remp2=Var⁡^N(f)/(Var⁡^N(f)+σ2)R^2_{\mathrm{emp}} = \widehat{\operatorname{Var}}_N(f)/(\widehat{\operatorname{Var}}_N(f) + \sigma^2) under different priors. Under the R2-GP prior with R2∼Beta⁡(1,2)R^2 \sim\operatorname{Beta}(1,2) and a=1a = 1 (panel R2-GP), its implied prior stays close to the target for every DD between 1 and 100. Under independent half-normal priors with unit scale, the median rises from 0.260.26 at D=1D = 1 to 0.990.99 at D=100D = 100. As competing shrinkage priors, which are further introduced in § 3, the regularised horseshoe (Piironen and Vehtari [18]) and normal-gamma (Griffin and Brown [7]) may shrink each αd\alpha_d but leave the prior on their total to shrink or grow with DD.

Implied prior on empirical explained variance

Figure 2. Implied Prior on Remp2R^2_{\mathrm{emp}}. Empirical CDF of Remp2=Var⁡^N(f)/(Var⁡^N(f)+σ2)R^2_{\mathrm{emp}} = \widehat{\operatorname{Var}}_N(f)/(\widehat{\operatorname{Var}}_N(f) + \sigma^2) for additive HSGP GAMs under four priors on αd\alpha_d, as the number of components DD grows. CDFs are generated from 2000 prior draws: N=200N = 200, M=100M = 100, σ=1\sigma= 1 and ρd∼InvGamma⁡(5,4)\rho_d \sim\operatorname{InvGamma}(5, 4). These figures are insensitive to the prior ρd\rho_d. RHS and normal-gamma are specified as in § 3, with p0=5p_0 = 5. The dashed curve is the target Beta⁡(1,2)\operatorname{Beta}(1, 2). Only R2-GP is stable in DD.

Notice that calibration holds for Var⁡^N(f)\widehat{\operatorname{Var}}_N(f) but not for the empirical proportion of explained variance, which is a nonlinear function of it. The map t↦t/(t+σ2)t \mapsto t/(t+\sigma^2) is strictly concave on t>0t > 0, so Jensen’s inequality gives

E[Remp2∣X,σ,R2]≤α2α2+σ2=R2,(18)\mathbb{E}\left[R^2_{\mathrm{emp}} \mid X,\sigma,R^2\right] \le\frac{\alpha^2}{\alpha^2+\sigma^2} = R^2, \tag*{(18)}

with strict inequality whenever Var⁡^N(f)\widehat{\operatorname{Var}}_N(f) is non-degenerate. This accounts for the slight shift to the left for small DD in Figure 2 for the R2-GP prior. This is not a deficiency of the proposed prior, since no choice of QdQ_d calibrates both E[Var⁡^N(f)]\mathbb{E}[\widehat{\operatorname{Var}}_N(f)] and E[h(Var⁡^N(f))]\mathbb{E}[h(\widehat{\operatorname{Var}}_N(f))] for a nonlinear hh.

Finally, to ensure the component’s prior expected marginal variance remains finite, the prior on the length scale ρd\rho_d must be sufficiently light-tailed. As ρd→∞\rho_d \to\infty, the normaliser for the exact kernel decays such that 1/Qd(ρd)≍ρdγ1/Q_d(\rho_d) \asymp\rho_d^\gamma, where γ=1\gamma= 1 for the exponential kernel and γ=2\gamma= 2 for smoother Matérn variants (Rasmussen and Williams [19]). Therefore, the exact marginal variance αd2/Qd(ρd)\alpha_d^2/Q_d(\rho_d) has a finite prior expectation if and only if E[ρdγ]<∞\mathbb{E}[\rho_d^\gamma] < \infty. While the HSGP approximation inherently bounds this variance for finite MM, preserving the properties of the exact-kernel limit rules out heavy-tailed priors, such as an Inverse-Gamma(0.1,0.1)(0.1, 0.1) distribution.

Simulation study

We assess finite-sample predictive and parameter recovery performance of the R2-GP prior, that is, {(8)} together with {(14)}, as the number of GAM components DD increases. The competitors are priors on the component scales that carry popular shrinkage approaches over to {(1)}. These have not yet been considered for HSGP GAMs to our knowledge, but pose straightforward extensions based off our methodology.

Setup

Data Generation. We generate data from (1) with μ0=0\mu_0 = 0 and σ=1\sigma= 1. Each fdf_d is a draw from the normalised prior (14) with a common length scale ρd=ρ0\rho_d = \rho_0 and signal variance

αd2=σ2DR021−R02,R02=13,(19)\alpha_d^2 = \frac{\sigma^2}{D}\frac{R_0^2}{1-R_0^2}, \qquad R_0^2 = \frac{1}{3}, \tag*{(19)}

for d=1,…,Dd = 1,\ldots,D. By (16), the total signal variance is σ2R02/(1−R02)=1/2\sigma^2R_0^2/(1-R_0^2) = 1/2 for every DD, while the signal variance of a single component decays at rate D−1D^{-1}. We call (19) the dense data-generating process (DGP). We also consider a sparse DGP in Appendix § 9.§ 2, in which the first five components share the same total signal variance and αd=0\alpha_d = 0 for d>5d > 5. We vary D∈{5,10,30,50,100}D \in\{5,10,30,50,100\}, ρ0∈{0.3,0.5,0.7}\rho_0 \in\{0.3,0.5,0.7\} and the kernel choice that includes SE, Matérn 3/2 and Matérn 5/2. For each condition, we simulate 50 replications. A data set has N=120N = 120 training and 500 test observations. Inputs are iid uniform on [−1,1][-1,1]. Following Riutort-Mayol et al. [21], we set Ld=max⁡(1.5,4.5ρ0)L_d = \max(1.5,4.5\rho_0) and M=⌈1.2κLd/ρ0⌉M = \lceil1.2\kappa L_d/\rho_0\rceil with κ=1.75\kappa= 1.75, 3.42 and 2.65 for the three kernels, so that 10≤M≤2110 \le M \le21. The data are generated with 4M4M basis functions, so that the ff is generated close as a close representation to the desired GP. Since (16) holds in expectation over zz, the realised value Rtrue2R^2_{\mathrm{true}}, that is, Remp2R^2_{\mathrm{emp}} at the generating ff and σ\sigma, varies across data sets around the nominal R02R_0^2.

Priors. All models use the normalised components (14) and differ only in the prior on α1,…,αD\alpha_1,\ldots,\alpha_D. R2-GP is (8) with μR=1/3\mu_R = 1/3, ϕR=3\phi_R = 3 and a=1a = 1, so that R2∼Beta⁡(1,2)R^2 \sim\operatorname{Beta}(1,2), as in § 2.4. We compare it with three alternatives, where a superscript ++ denotes truncation to (0,∞)(0,\infty). (i) The normal prior, αd∼N+(0,1)\alpha_d \sim\mathcal{N}^{+}(0,1) independently, is the analogue of ridge regression and represents independent priors on the component scales. (ii) The regularised horseshoe (RHS; Piironen and Vehtari, [18]), αd∣λd,τ,ζ∼N+(0,τ2λ~d2)\alpha_d \mid\lambda_d,\tau,\zeta\sim\mathcal{N}^{+}(0,\tau^2\widetilde{\lambda}_d^2) with λ~d2=ζ2λd2/(ζ2+τ2λd2)\widetilde{\lambda}_d^2 = \zeta^2\lambda_d^2/(\zeta^2+\tau^2\lambda_d^2), λd∼t3(0,1)\lambda_d \sim t_3(0,1), τ∼C+(0,τ0)\tau\sim\mathcal{C}^{+}(0,\tau_0) and ζ2∼InvGamma⁡(2,8)\zeta^2 \sim\operatorname{InvGamma}(2,8), represents sparsity-favouring priors. We set τ0=p0(D−p0)−1σN−1/2\tau_0 = p_0(D-p_0)^{-1}\sigma N^{-1/2} as in Piironen and Vehtari [18], with a prior guess of p0=min⁡(5,D−1)p_0 = \min(5,D-1) active components. While p0=5p_0 = 5 is miscalibrated for the dense DGP, τ\tau is nevertheless learned from the data, and p0p_0 is well calibrated for the sparse DGP. (iii) The normal-gamma prior (NG; Griffin and Brown, [7]), αd∣ξd∼N+(0,σ2ξd)\alpha_d \mid\xi_d \sim\mathcal{N}^{+}(0,\sigma^2\xi_d) with ξd∼Gamma⁡(ϑ,ϑ/η)\xi_d \sim\operatorname{Gamma}(\vartheta,\vartheta/\eta) in shape-rate form, ϑ∼Exp⁡(1)\vartheta\sim\operatorname{Exp}(1) and η∼InvGamma⁡(2,1)\eta\sim\operatorname{InvGamma}(2,1), represents global-local shrinkage priors that are popular in other domains. The remaining priors are common to all models, namely ρd∼N+(ρ0,0.12)\rho_d \sim\mathcal{N}^{+}(\rho_0,0.1^2), σ∼N+(0,1)\sigma\sim\mathcal{N}^{+}(0,1) and μ0∼N(0,22)\mu_0 \sim\mathcal{N}(0,2^2). The comparisons presented here isolate the effect of the prior on αd\alpha_d. We show in Appendix § 9.§ 3 that the qualitative behaviour of the competing priors is the same also without the normalisation.

Expected Behaviour. The total signal variance is independent of DD under R2-GP, whereas its mean is DD under normal, and Dσ2D\sigma^2 under NG, respectively. Since the generating value is 1/21/2 throughout, expect normal and NG to overestimate the signal variance and to underestimate σ\sigma by a margin that grows with DD, in line with Figure 2. RHS is a sparsity favouring prior which for p0=5p_0 = 5 has τ0=O(D−1)\tau_0 = O(D^{-1}), hence, decays faster than the generating αd=O(D−1/2)\alpha_d = O(D^{-1/2}) which is expected to lead to excessive shrinkage for large DD.

Evaluation. Predictive performance is measured by the log predictive density summed over the test observations, which estimates the expected log predictive density (elpd; Vehtari et al., [26]), relative to R2-GP,

Δelpd⁡m(s)=elpd⁡m(s)−elpd⁡R2-GP(s),(20)\Delta\operatorname{elpd}^{(s)}_m = \operatorname{elpd}^{(s)}_m - \operatorname{elpd}^{(s)}_{\mathrm{R2\text{-}GP}}, \tag*{(20)}

for prior mm and replication ss, so that negative values favour R2-GP. We report its mean over replications and, following Sivula et al. [23] and McLatchie et al. [13], regard ∣Δelpd∣>4|\Delta\mathrm{elpd}| > 4 as a meaningful difference. For θ∈{α,ρ,σ}\theta\in\{\alpha, \rho, \sigma\} we report the coverage of the central 90% posterior intervals (Table 2, Table 5 and Table 8) and the relative root-mean-squared error, that is relative to the RMSE of the R2-GP (RMSE; Table 3, Table 6 and Table 9).

ρ\rhoPrior\multicolumn{5}{c}{α\alpha}\multicolumn{5}{c}{ρ\rho}\multicolumn{5}{c}{σ\sigma}
510305010051030501005103050100
0.3R2-GP97100100991001001001001001009282929498
Normal9596959610010010010010010090803820
NG9197807799100100100100100928072682
RHS (p0p_0=5)84856347181001001001001009274645442
0.5R2-GP99991001001001001001001001009486889092
Normal959594949910010010010010092846080
NG94979064991001001001001009484726634
RHS (p0p_0=5)88888252331001001001001009484705848
0.7R2-GP9899100100991001001001001009290888896
Normal9595949596100100100100100908472240
NG98989368881001001001001009286747258
RHS (p0p_0=5)87898557401001001001001009286746854

Table 2. Interval Coverage (%), Dense DGP, SE Kernel. Bold marks the coverage closest to 90%.

ρ\rhoPrior\multicolumn{5}{c}{α\alpha}\multicolumn{5}{c}{ρ\rho}\multicolumn{5}{c}{σ\sigma}
510305010051030501005103050100
0.3R2-GP0.0970.0620.0330.0250.0170.0220.0160.0080.0060.0030.0540.0840.0790.0820.078
Normal1.572.304.947.5911.611.051.221.842.503.130.980.952.657.739.97
NG1.521.882.742.923.130.940.981.121.303.131.011.021.662.309.26
RHS (p0p_0=5)1.641.972.642.933.460.870.760.570.540.581.111.301.721.902.22
0.5R2-GP0.0920.0610.0300.0290.0180.0160.0120.0060.0040.0030.0530.0650.0770.0780.097
Normal1.962.856.658.0513.261.011.071.602.503.650.981.021.834.797.92
NG1.662.023.052.893.350.930.951.061.132.701.001.031.491.655.26
RHS (p0p_0=5)1.752.022.612.513.080.880.810.680.660.811.071.211.481.711.88
0.7R2-GP0.0950.0610.0310.0270.0180.0110.0080.0040.0030.0020.0530.0650.0740.0750.082
Normal2.073.117.128.9815.520.941.081.451.986.320.981.001.413.239.53
NG1.712.053.023.043.500.920.961.011.271.821.001.031.411.523.34
RHS (p0p_0=5)1.792.062.482.553.040.870.840.700.710.781.051.171.421.721.97

Table 3. Relative RMSE, Dense DGP, SE Kernel. The R2-GP rows give its RMSE in italics.

ρ\rhoPriorα\alpha: 5103050100ρ\rho: 5103050100σ\sigma: 5103050100
0.3R2-GP989910010010010010010010010092889690100
Normal949695981001001001001001009884200
NG92958791100100100100100100968284520
RHS (p0=5p_0=5)87786042161001001001001008282705046
0.5R2-GP989998100981001001001001008894948892
Normal9494959610010010010010010086883600
NG93958277991001001001001008686806614
RHS (p0=5p_0=5)89816352291001001001001008680684454
0.7R2-GP979798100981001001001001008892988894
Normal939495949910010010010010090845000
NG95968475991001001001001008886827030
RHS (p0=5p_0=5)87876761351001001001001008686746058

Table 5. Interval Coverage (%), Dense DGP, Matérn 3/2 Kernel. As in Table 2.

ρ\rhoPriorα\alpha: 5103050100ρ\rho: 5103050100σ\sigma: 5103050100
0.3R2-GP0.0870.0650.0340.0250.0160.0190.0120.0060.0050.0030.0570.0680.0670.0940.087
Normal1.601.904.926.2611.181.071.172.092.183.420.931.005.677.548.93
NG1.581.752.732.822.990.960.931.221.702.440.991.051.933.138.62
RHS (p0=5p_0=5)1.701.952.723.013.650.870.710.550.570.741.161.392.031.782.13
0.5R2-GP0.0960.0660.0350.0240.0160.0120.0080.0040.0030.0020.0650.0680.0640.0890.098
Normal1.772.194.858.6212.480.971.031.502.604.451.020.943.406.667.85
NG1.601.842.613.353.380.930.911.051.392.461.021.011.762.096.61
RHS (p0=5p_0=5)1.671.932.523.073.470.880.770.630.670.961.091.311.911.981.74
0.7R2-GP0.0970.0620.0360.0240.0170.0070.0050.0030.0020.0020.0640.0660.0590.0790.093
Normal1.812.625.099.3913.070.921.001.322.086.920.960.972.755.928.48
NG1.581.922.573.313.310.890.911.151.272.110.990.991.601.895.85
RHS (p0=5p_0=5)1.661.992.402.963.260.840.770.750.911.111.081.241.862.061.75

Table 6. Relative RMSE, Dense DGP, Matérn 3/2 Kernel. As in Table 3.

ρ\rhoPriorα=5\alpha=5103050100ρ=5\rho=5103050100σ=5\sigma=5103050100
0.3R2-GP92971001001001001001001001008288969298
Normal9397959710010010010010010088862000
NG89958584100100100100100100848686482
RHS (p0=5p_0=5)81805949221001001001001008286686458
0.5R2-GP989898991001001001001001009092969294
Normal9495959410010010010010010090884820
NG94978276991001001001001008890807014
RHS (p0=5p_0=5)88806356271001001001001008688626050
0.7R2-GP9898981001001001001001001008894909494
Normal939595949910010010010010088906080
NG95978477951001001001001008892787644
RHS (p0=5p_0=5)90856661381001001001001008688666654

Table 8. Interval Coverage (%), Dense DGP, Matérn 5/2 Kernel. As in Table 2.

ρ\rhoPriorα=5\alpha=5103050100ρ=5\rho=5103050100σ=5\sigma=5103050100
0.3R2-GP0.1000.0660.0310.0250.0170.0200.0160.0070.0050.0030.0660.0710.0560.0820.078
Normal1.582.035.166.7910.641.031.141.942.372.891.001.115.118.499.87
NG1.531.762.813.022.780.930.991.041.782.631.021.061.663.299.56
RHS (p0=5p_0=5)1.621.892.912.963.360.870.810.540.570.801.091.152.551.881.86
0.5R2-GP0.0990.0670.0340.0240.0180.0130.0090.0040.0030.0020.0540.0630.0700.0860.087
Normal1.812.205.188.9611.940.951.091.582.444.711.021.032.316.078.97
NG1.591.802.673.373.250.920.930.991.333.001.021.021.581.907.14
RHS (p0=5p_0=5)1.671.892.553.013.190.850.780.640.670.851.041.131.821.731.75
0.7R2-GP0.1010.0640.0360.0240.0170.0090.0060.0030.0020.0020.0530.0630.0720.0790.087
Normal1.902.525.499.8013.470.910.991.451.875.341.021.001.674.588.83
NG1.621.892.623.423.380.890.901.071.362.011.011.031.401.754.77
RHS (p0=5p_0=5)1.711.922.422.913.150.860.790.710.730.951.021.101.631.791.81

Table 9. Relative RMSE, Dense DGP, Matérn 5/2 Kernel. As in Table 3.

Values above one indicate worse recovery than under R2-GP. Lastly, we compare the posterior of Remp2R^2_{\mathrm{emp}} with Rtrue2R^2_{\mathrm{true}}. All models are implemented in Stan (Carpenter et al. [3]) and fitted with four chains of 2000 iterations, with the half as warm-up at a target acceptance rate of 0.95. Figure 3 shows the predictive performance and the recovery of Rtrue2R^2_{\mathrm{true}} for the SE kernel with ρ0=0.3\rho_0 = 0.3. We show figures for all other conditions in Appendix § 9. See Table 4, Table 7 and Table 10 for sampling diagnostics.

Dense DGP, SE Kernel, $\rho_0 = 0.3$

Figure 3. Dense DGP, SE Kernel, ρ0=0.3\rho_0 = 0.3. Left: mean Δelpd⁡\Delta\operatorname{elpd} over 50 simulations (solid) with ±1\pm1 Monte Carlo standard error (dotted). Dashed lines mark ±4\pm4 and negative values favour R2-GP. Right: posterior means (points) and central 90% intervals (bars) of Remp2R^2_{\mathrm{emp}} for each simulation. Crosses mark the realised Rtrue2R^2_{\mathrm{true}}.

ρ\rhoPrior\multicolumn{5}{c}{R^\hat{R}}\multicolumn{5}{c}{ESS}\multicolumn{5}{c}{Divergences}\multicolumn{5}{c}{Time (s)}
5103050100510305010051030501005103050100
0.3R2-GP1.001.001.011.011.0110789988146614980.00.00.20.61.95.79.12651129
Normal1.001.011.011.261.3795186848043181.21.45.82832357.716764581634
NG1.021.041.171.401.125692223925621410116116781232204350903
RHS (p0p_0=5)1.001.011.011.011.018967705515787082.33.67.215128.4143663134
0.5R2-GP1.001.001.011.011.011176114810199466070.00.00.00.31.47.2123256191
Normal1.001.001.011.051.371043973756217192.01.61.1802339.320973491292
NG1.011.031.151.251.436053285124328.91869199491331175395961
RHS (p0p_0=5)1.001.011.011.011.0110359115786245684.14.617163211164569171
0.7R2-GP1.001.001.001.011.011289126610999996840.70.40.00.01.78.1143865196
Normal1.001.001.011.011.4811111078919531142.91.90.88.224713261283241457
NG1.011.031.151.231.616093345422202834891935817412054391125
RHS (p0p_0=5)1.001.011.011.011.0110939306326365876.61012182113215782199

Table 4. Sampling Diagnostics, Dense DGP, SE Kernel. Time is the sampling time in seconds.

ρ\rhoPriorR^\hat{R}: 5103050100ESS: 5103050100Divergences: 5103050100Time (s): 5103050100
0.3R2-GP1.001.001.011.011.0110539907165814750.00.00.12.46.17.7133872215
Normal1.001.011.071.251.3792580717123160.60.41142432479.1201777382401
NG1.021.051.191.391.085121973728582.31269665314492724501678
RHS (p0=5p_0=5)1.011.011.011.011.028617485415796781.22.36.9182811195499273
0.5R2-GP1.001.001.011.011.01117911209667755400.00.10.10.47.28.7154273265
Normal1.001.011.011.201.39101698854542171.70.76.523927312251386902543
NG1.031.041.181.331.216502763823506.1281101187316472944831760
RHS (p0=5p_0=5)1.001.011.011.011.029408336145596342.28.48.11740142259112317
0.7R2-GP1.001.001.011.011.011228117910998566101.00.00.10.42.011174990282
Normal1.001.001.011.091.4510571026754112162.11.02.313023814301485742364
NG1.011.031.151.281.3767931741213118241221185920492945681924
RHS (p0=5p_0=5)1.001.011.011.021.0210189216435305614.25.5143337162772127349

Table 7. Sampling Diagnostics, Dense DGP, Matérn 3/2 Kernel. As in Table 4.

ρ\rhoPriorR^\hat{R} 5103050100ESS 5103050100Divergences 5103050100Time (s) 5103050100
0.3R2-GP1.001.001.011.011.0110539847656354480.00.00.20.44.67.2123359178
Normal1.001.011.021.281.3493283932030170.80.6382332639.2181356622012
NG1.011.031.211.321.1054726527295815151041026516402753971400
RHS (p0=5p_0=5)1.011.011.011.011.038607365845585752.12.77.3124711184984242
0.5R2-GP1.001.001.011.011.01116911589969096180.00.00.10.71.69.0143969207
Normal1.001.011.011.141.3899097567784171.82.01.913621212231285292034
NG1.011.031.171.281.226442663823487.2281061048416442835251315
RHS (p0=5p_0=5)1.001.011.011.011.019719366545836572.34.015222913215799238
0.7R2-GP1.001.001.011.011.011241123111149026590.20.10.10.62.59.9174885239
Normal1.001.001.011.031.3611021069862279152.21.42.15322815311505002106
NG1.011.031.151.251.5066633037192912351191226918512815421488
RHS (p0=5p_0=5)1.011.011.011.011.011010964717565593103.9141628152468128310

Table 10. Sampling Diagnostics, Dense DGP, Matérn 5/2 Kernel. As in Table 4.

Results

Predictive Performance. R2-GP attains the highest mean elpd in each of the 45 conditions, against every competitor. At D=5D = 5 the priors are close, however normal and NG deteriorate as DD grows. This is as a consequence of the lack of joint shrinkage of the GAM components, which leads to overestimation of Rtrue2R^2_{\mathrm{true}}. RHS stays closest to R2-GP at higher DD, with mean Δelpd∈[−13.5,−4.0]\Delta\mathrm{elpd} \in[-13.5, -4.0] for all D≥10D \ge10. As expected, with p0=5p_0 = 5 RHS overshrinks as DD increases, which can be seen as underestimation of Rtrue2R^2_{\mathrm{true}} in Figure 3. Due to the small true variance shares as DD increases, this results in less of a penalty in predictive performance than the overestimation of Rtrue2R^2_{\mathrm{true}} seen for normal and NG.

Recovery. Recovery mirrors the predictive results for the dense DGP. R2-GP has the lowest or close to the lowest RMSE for (α,σ)(\alpha, \sigma) in all conditions. Interval coverage is close to or above the nominal level, with its advantage widening with DD. (Table 2 and Table 3 for the SE kernel, Appendix § 9 for the Matérn kernels).

Sampling Efficiency. R2-GP also improves the sampling efficiency markedly. On average over replications, its largest R^\hat{R} is at most 1.01 and its smallest bulk effective sample size is at least 448 in every condition. It has the fewest divergent transitions in all conditions and the shortest sampling time in all but one. At D=100D = 100, by contrast, the normal prior and NG fail to converge. The results of the normal and NG priors at large DD therefore reflect the prior and the sampler jointly. The RHS prior converges, with R^≤1.03\hat{R} \le1.03, but needs up to 1.6 times the sampling time.

Sparse Allocation. Under the sparse DGP (Appendix § 9.§ 2), RHS with the calibration p0=5p_0 = 5 attains a higher mean elpd than R2-GP for D≥30D \ge30, however by small margins. The normal prior underperforms as in the dense DGP, and NG does so at D=100D = 100. Relative to RHS, R2-GP’s RMSE is slightly lower for the non-zero αd\alpha_d in all conditions, however higher for the true zero αd\alpha_d. This is as expected, since for a=1a = 1 the allocation ψ\psi is uniform on the simplex, whereas concentration near zero requires a small aa ([31, 1, 11]).

Summary. For dense signals R2-GP performs at least as well as every competing prior in terms of prediction and recovers component scales most accurately. In particular, our method recovers the component scales most accurately in all conditions, with a margin over the normal and NG priors that grows with DD. Sampling wise, our prior is the most reliable prior to sample. The patterns are almost identical across the three kernels and the three length scales.

Real-world Case Study

Data. In addition to our simulation scenarios, we demonstrate our proposed methods in analysing the BloodBrain data [14] from the R package caret [12]. The dataset describes 208 chemical compounds using 134 molecular descriptors based on the compounds structure. The response yy is the logarithm of the ratio of the compound’s concentration in the brain to the blood, a standard procedure to measure how readily a chemical compound crosses the blood-brain barrier (with logBBB\mathit{logBBB} as response). For simplicity, we consider only continuous covariates. The data has N=208N = 208 samples and D=101D = 101 covariates.

Model Fit. We estimate the same set of models as in § 3, except for the RHS prior for which p0=10p_{0} = 10. We consider the SE and Matérn 3/2 kernels. We set boundary conditions with c=2c = 2 and compute the number of basis functions accordingly. To compare predictive performance, the data is split into 80/20 train/test ratio with 10 random replications. Each fit uses 4 chains with 1000 warm-up and 1000 sample draws.

Results. The covariate clogp describing lipophilicity is a biologically relevant covariate for modelling logBBB\mathit{logBBB} [14] in the BloodBrain data. Additionally, we also investigate here lumo describing the energy component. The effects of these two covariates on response logBBB\mathit{logBBB} for all fitted models are shown in Figure 4. Across both kernels and all models, logBBB\mathit{logBBB} increases with an increase in clogp, indicating lipophilic compounds cross the blood-brain-barrier readily. For lumo, we see a decreasing effect on logBBB\mathit{logBBB}. The shrinkage priors agree while maintaining tight posterior credible intervals while the Normal model remains unstable. See Appendix § 10 for the effect of more covariates. Our proposed R2-GP prior provides superior predictive performance as well as sampling speed and diagnostics (Appendix Table 20) compared to the Normal model. RHS and NG provide comparable predictive performance R2-GPs, but worse MCMC convergence, the latter also reflected in the simulation study. Overall, using R2-GP is computationally and inferentially more robust for this data while providing the benefit of a prior based off of first principles.

Covariate effects for clogp and lumo under SE and Matérn 3/2 kernels

Figure 4. Covariate Effects, BloodBrain Data. Top 2 covariates are shown based on their contribution to explained variance.

Discussion

We introduce the R2-GP prior for GAM GPs under which the magnitude of a component equals its signal variance, the expected variation across the inputs. We model magnitudes based off of an interpretable prior on the model’s proportion of explained variance and a Dirichlet partition of that variance across components. Thus, components compete for explained variance, preventing variance explosion with many components. In HS form, the normaliser costs O(M)O(M) per gradient evaluation, so the approach also scales well for large sample sizes. Our simulations and real-data case study demonstrate superior predictive performance of R2-GP over independent normal priors. Although competitive in predictive performance with other shrinkage priors, R2-GP outperforms in terms of sampling efficiency, especially when many components are active. Future works should include richer kernel structures, such as interactions between covariates and as well as applications in other GP extensions.

AI Use Statement

In this work, we used generative AI tools for code prototyping of parts of the code (Claude Opus 5 High from Anthropic). Further, we used the ChatGPT 5.6 model from OpenAI for minor language edits, aiming to enhance readability. LLM-generated code and text was verified, modified and tested for correctness by the authors. We take responsibility for the final content of this work, including text, claims, or artifacts produced with the aid of generative AI.

References

Overview. This supplement has four parts. Appendix A gives the derivations and proofs for § 2. Appendix B gives the full results of the simulation study of § 3. Appendix C gives further results for the case study of § 4. Appendix D describes the computing infrastructure. Table 1 lists the content of each part and the section of the main text that it supports.

AppendixContentFiguresTablesSection
A.1Variance decomposition (4) and normaliser (5) for the exact GP2.1
A.2Proofs of Lemmas 2.1 and A.1, of the decomposition (13) and of the calibration results (15) and (16)2.2–2.4
B.1Dense DGP: Δelpd⁡\Delta\operatorname{elpd} and posterior of Remp2R^2_{\mathrm{emp}} (figures), interval coverage, relative RMSE and sampling diagnostics (tables), for all kernels and length scales5–102–103.2
B.2Sparse DGP: the same figures and tables11–1611–193.2
B.3Competing priors without the normalisation: Δelpd⁡\Delta\operatorname{elpd} for both DGPs17–223.1
CBloodBrain data: effects of the 12 covariates with the largest share of explained variance, predictive performance and sampling diagnostics23–24204
DHardware, software versions and run times3, 4

Table 1. Overview of the Supplementary Material

Proofs

Derivations for Section 2.1

Throughout, ExE_x and Var⁡x\operatorname{Var}_x are the expectation and variance operators with respect to random variable xx, and xx and x′x' are independent inputs.

Decomposition (4). Apply the law of total variance to fd(x)f_d(x) under the joint distribution of xx and fdf_d. Conditioning on xx gives

Var⁡(fd(x))=Ex[Var⁡fd(fd(x))]+Var⁡x(Efd[fd(x)])=αd2,(21)\operatorname{Var}(f_d(x)) = E_x\left[\operatorname{Var}_{f_d}(f_d(x))\right] + \operatorname{Var}_x\left(E_{f_d}[f_d(x)]\right) = \alpha_d^2, \tag*{(21)}

since under (2) the prior mean is zero and Var⁡fd(fd(x))=αd2k(x,x;ρd)=αd2\operatorname{Var}_{f_d}(f_d(x)) = \alpha_d^2 k(x,x;\rho_d) = \alpha_d^2 at every input. Conditioning on fdf_d instead gives Var⁡(fd(x))=Efd[Var⁡x(fd(x)∣fd)]+Var⁡fd(Ex[fd(x)∣fd])\operatorname{Var}(f_d(x)) = E_{f_d}[\operatorname{Var}_x(f_d(x) \mid f_d)] + \operatorname{Var}_{f_d}(E_x[f_d(x) \mid f_d]), and the two expressions together are (4).

Linear Term. The same two steps hold for any zero-mean prior on a function ff of xx, with Ex[Var⁡f(f(x))]E_x[\operatorname{Var}_f(f(x))] on the left-hand side. For f(x)=xβf(x) = x\beta with centred inputs, Ex[x]=0E_x[x] = 0, the level is Ex[x]β=0E_x[x]\beta= 0 for every β\beta. Its variance is therefore zero, and the marginal variance, averaged over the inputs, equals the signal variance, Var⁡(β)Ex[x2]\operatorname{Var}(\beta) E_x[x^2].

Variance of the Level and QdQ_d. Under (2), the level has mean zero, and xx, x′x' and fdf_d are independent, so that

Var⁡fd[Ex[fd(x)∣fd]]=Efd[Ex[fd(x)∣fd]Ex′[fd(x′)∣fd]]=Ex,x′[Efd[fd(x)fd(x′)]]=αd2Ex,x′[k(x,x′;ρd)],\operatorname{Var}_{f_d}\left[\mathbb{E}_x[f_d(x)\mid f_d]\right]=\mathbb{E}_{f_d}\left[\mathbb{E}_x[f_d(x)\mid f_d]\mathbb{E}_{x'}[f_d(x')\mid f_d]\right]=\mathbb{E}_{x,x'}\left[\mathbb{E}_{f_d}[f_d(x)f_d(x')]\right]=\alpha_d^2\mathbb{E}_{x,x'}[k(x,x';\rho_d)],

which is αd2kˉd\alpha_d^2\bar{k}_d and, with (4), gives (5). A Matérn kernel has the form k(x,x′;ρd)=κ(∣x−x′∣/ρd)k(x,x';\rho_d)=\kappa(|x-x'|/\rho_d), where κ\kappa is positive and strictly decreasing with κ(0)=1\kappa(0)=1. Hence 0<k(x,x′;ρd)<10<k(x,x';\rho_d)<1 for x≠x′x\ne x', and k(x,x′;ρd)k(x,x';\rho_d) increases with ρd\rho_d. Unless all inputs coincide, it follows that kˉd∈(0,1)\bar{k}_d\in(0,1) increases with ρd\rho_d, so that Qd(ρd)∈(0,1)Q_d(\rho_d)\in(0,1) decreases with ρd\rho_d. For a general stationary kernel, αd2kˉd\alpha_d^2\bar{k}_d is still a variance that is at most αd2\alpha_d^2, so that Qd∈[0,1]Q_d\in[0,1] without any assumption on the sign of the correlations.

Finite-sample Counterpart. Section 2.3 replaces Var⁡x\operatorname{Var}_x by the sample variance Var⁡^N\widehat{\operatorname{Var}}_N over the observed inputs x1d,…,xNdx_{1d},\ldots,x_{Nd}. The proof of (26) holds for any covariance matrix. For the exact kernel matrix KdK_d in place of K~d\widetilde{K}_d, with [Kd]ii=1[K_d]_{ii}=1, it gives

Efd[Var⁡^N(fd)]=αd2{1−1N(N−1)∑i≠jk(xid,xjd;ρd)}\mathbb{E}_{f_d}\left[\widehat{\operatorname{Var}}_N(f_d)\right]=\alpha_d^2\left\{1-\frac{1}{N(N-1)}\sum_{i\ne j}k(x_{id},x_{jd};\rho_d)\right\}

under (2), so that the expectation over two independent inputs in kˉd\bar{k}_d becomes the average over pairs of distinct observed inputs.

Proofs for Sections 2.2 to 2.4

Throughout, Ez\mathbb{E}_z denotes expectation over the coefficient vectors z1,…,zDz_1,\ldots,z_D only, holding XX, ρ\rho and α\alpha fixed. Two elementary identities are used repeatedly and are stated here once. Let PN=IN−N−11N1NTP_N=I_N-N^{-1}1_N1_N^{\mathsf T} be the centring matrix, which is symmetric and idempotent with PN1N=0P_N1_N=0 and tr⁡PN=N−1\operatorname{tr}P_N=N-1. Subtracting the sample mean is an orthogonal projection, so for any v∈RNv\in\mathbb{R}^N

Var⁡^N(v)=1N−1∥PNv∥2=1N−1vTPNv,(22)\widehat{\operatorname{Var}}_N(v)=\frac{1}{N-1}\lVert P_Nv\rVert^2=\frac{1}{N-1}v^{\mathsf T}P_Nv, \tag*{(22)}

and for a random vector uu with mean zero and covariance Σ\Sigma and a fixed matrix AA, writing the scalar uTAuu^{\mathsf T}Au as tr⁡(AuuT)\operatorname{tr}(Auu^{\mathsf T}) and using linearity of the trace gives

E[uTAu]=tr⁡(AΣ).(23)\mathbb{E}[u^{\mathsf T}Au]=\operatorname{tr}(A\Sigma). \tag*{(23)}

Proof of Lemma 2.1. Write gd=Azdg_d=Az_d with the fixed matrix A=Φddiag⁡(Sd)A=\Phi_d\operatorname{diag}(\sqrt{S_d}). A linear image of a Gaussian vector is Gaussian, with Ez[gd]=AEz[zd]=0\mathbb{E}_z[g_d]=A\mathbb{E}_z[z_d]=0 and

Ez[gdgdT]=AEz[zdzdT]AT=AAT=Φddiag⁡(Sd)diag⁡(Sd)ΦdT=Φddiag⁡(Sd)ΦdT,\mathbb{E}_z[g_dg_d^{\mathsf T}]=A\mathbb{E}_z[z_dz_d^{\mathsf T}]A^{\mathsf T}=AA^{\mathsf T}=\Phi_d\operatorname{diag}(\sqrt{S_d})\operatorname{diag}(\sqrt{S_d})\Phi_d^{\mathsf T}=\Phi_d\operatorname{diag}(S_d)\Phi_d^{\mathsf T},

using diag⁡(Sd)2=diag⁡(Sd)\operatorname{diag}(\sqrt{S_d})^2=\operatorname{diag}(S_d), valid since the matrices are diagonal and Sdm>0S_{dm}>0. Symmetry is immediate, and vTK~dv=∥diag⁡(Sd)ΦdTv∥2≥0v^{\mathsf T}\widetilde{K}_dv=\lVert\operatorname{diag}(\sqrt{S_d})\Phi_d^{\mathsf T}v\rVert^2\ge0 gives positive semi-definiteness. The rank bound follows from rank⁡(AAT)=rank⁡(A)≤min⁡(N,M)\operatorname{rank}(AA^{\mathsf T})=\operatorname{rank}(A)\le\min(N,M). □

Lemma A.1 (Equivalent expressions for the normaliser). Let gdg_d and K~d\widetilde{K}_d be as in Lemma 2.1. Then the normaliser (11) satisfies

Qd(ρd)=1N−1tr⁡(PNK~d)(24)Q_d(\rho_d)=\frac{1}{N-1}\operatorname{tr}(P_N\widetilde{K}_d) \tag*{(24)}
=∑m=1MSdmVar⁡^N(ϕdm)(25)=\sum_{m=1}^{M}S_{dm}\widehat{\operatorname{Var}}_N(\phi_{dm}) \tag*{(25)}
=1N−1{tr⁡(K~d)−1N1NTK~d1N}.(26)=\frac{1}{N-1}\left\{\operatorname{tr}(\widetilde{K}_d)-\frac{1}{N}1_N^{\mathsf T}\widetilde{K}_d1_N\right\}. \tag*{(26)}

Moreover, Qd(ρd)>0Q_d(\rho_d)>0 whenever M≥2M\ge2 and the observed inputs of covariate dd lie in (−Ld,Ld)(-L_d,L_d) and are not all equal. Proof. By (22), Var⁡^N(gd)=(N−1)−1gdTPNgd\widehat{\operatorname{Var}}_{N}(g_d)=(N-1)^{-1}g_d^{\mathsf{T}}P_Ng_d. Conditionally on XX and ρd\rho_d the matrix PNP_N is fixed and gdg_d has mean zero with covariance K~d\widetilde{K}_d by Lemma 2.1, so (23) gives (24).

For (12), substitute K~d=Φddiag⁡(Sd)ΦdT\widetilde{K}_d=\Phi_d\operatorname{diag}(S_d)\Phi_d^{\mathsf{T}} and use cyclicity together with the fact that tr⁡(diag⁡(S)B)=∑mSmBmm\operatorname{tr}(\operatorname{diag}(S)B)=\sum_m S_mB_{mm} for any square BB:

tr⁡(PNΦddiag⁡(Sd)ΦdT)=tr⁡(diag⁡(Sd)ΦdTPNΦd)=∑m=1MSdm[ΦdTPNΦd]mm=∑m=1MSdmϕdmTPNϕdm.\begin{aligned} \operatorname{tr}(P_N\Phi_d\operatorname{diag}(S_d)\Phi_d^{\mathsf{T}})=\operatorname{tr}(\operatorname{diag}(S_d)\Phi_d^{\mathsf{T}}P_N\Phi_d) \\ &=\sum_{m=1}^{M}S_{dm}\left[\Phi_d^{\mathsf{T}}P_N\Phi_d\right]_{mm}=\sum_{m=1}^{M}S_{dm}\phi_{dm}^{\mathsf{T}}P_N\phi_{dm}. \end{aligned}

Dividing by N−1N-1 and applying (22) to each column turns ϕdmTPNϕdm/(N−1)\phi_{dm}^{\mathsf{T}}P_N\phi_{dm}/(N-1) into Var⁡^N(ϕdm)\widehat{\operatorname{Var}}_N(\phi_{dm}). The off-diagonal entries of ΦdTPNΦd\Phi_d^{\mathsf{T}}P_N\Phi_d, which are the sample covariances between different basis columns, do not enter, because the coefficients in zdz_d are uncorrelated.

For (26), expand PN=IN−N−11N1NTP_N=I_N-N^{-1}\mathbf{1}_N\mathbf{1}_N^{\mathsf{T}} inside the trace and use tr⁡(1N1NTK~d)=1NTK~d1N\operatorname{tr}(\mathbf{1}_N\mathbf{1}_N^{\mathsf{T}}\widetilde{K}_d)=\mathbf{1}_N^{\mathsf{T}}\widetilde{K}_d\mathbf{1}_N.

For strict positivity, Sdm>0S_{dm}>0 and Var⁡^N(ϕdm)≥0\widehat{\operatorname{Var}}_N(\phi_{dm})\ge0, so that (12) is zero only if every basis column is constant across the observed inputs. Let uid=(xid+Ld)/(2Ld)u_i^d=(x_{id}+L_d)/(2L_d), which lies in (0,1)(0,1) by assumption, so that ϕdm(xid)=Ld−1/2sin⁡(mπuid)\phi_{dm}(x_{id})=L_d^{-1/2}\sin(m\pi u_i^d) by (9). The first column is constant only if every uiu_i equals u∗u^* or 1−u∗1-u^* for some u∗∈(0,1/2]u^*\in(0,1/2]. If both values occur and u∗<1/2u^*<1/2, the second column takes the two distinct values ±Ld−1/2sin⁡(2πu∗)\pm L_d^{-1/2}\sin(2\pi u^*) and is not constant. Otherwise all inputs are equal. The condition M≥2M\ge2 cannot be dropped: for M=1M=1, a design with two distinct input values that are symmetric about the centre of the interval gives Qd=0Q_d=0.

Proof of the decomposition (13). Write Var⁡^N(gd)=(N−1)−1{∑i=1N[gd]i2−Ngˉd2}\widehat{\operatorname{Var}}_N(g_d)=(N-1)^{-1}\left\{\sum_{i=1}^{N}[g_d]_i^2-N\bar{g}_d^2\right\} with gˉd=N−11NTgd\bar{g}_d=N^{-1}\mathbf{1}_N^{\mathsf{T}}g_d. By Lemma 2.1, [gd]i[g_d]_i and gˉd\bar{g}_d have mean zero, with Ez[[gd]i2]=[K~d]ii\mathbb{E}_z[[g_d]_i^2]=[\widetilde{K}_d]_{ii} and Ez[gˉd2]=Var⁡z(gˉd)=N−21NTK~d1N\mathbb{E}_z[\bar{g}_d^2]=\operatorname{Var}_z(\bar{g}_d)=N^{-2}\mathbf{1}_N^{\mathsf{T}}\widetilde{K}_d\mathbf{1}_N. Taking the expectation term by term gives (13), in agreement with (26). Splitting 1NTK~d1N\mathbf{1}_N^{\mathsf{T}}\widetilde{K}_d\mathbf{1}_N into its diagonal and off-diagonal sums gives the equivalent form

Qd(ρd)=1N∑i=1N[K~d]ii−1N(N−1)∑i≠j[K~d]ij.Q_d(\rho_d)=\frac{1}{N}\sum_{i=1}^{N}[\widetilde{K}_d]_{ii}-\frac{1}{N(N-1)}\sum_{i\ne j}[\widetilde{K}_d]_{ij}.

the average marginal variance minus the average covariance between two distinct observed inputs. When the diagonal of K~d\widetilde{K}_d is one, Qd<1Q_d<1 if and only if this average covariance is positive. Without a condition on the sign of the covariances, positive semi-definiteness of K~d\widetilde{K}_d gives only Qd≤tr⁡(K~d)/(N−1)Q_d\le\operatorname{tr}(\widetilde{K}_d)/(N-1), which is N/(N−1)N/(N-1) for a unit diagonal.

Proof of the component calibration (15). Conditionally on XX, ρd\rho_d and αd\alpha_d, the factor αd/Qd\alpha_d/\sqrt{Q_d} is a deterministic scalar, and Var⁡^N\widehat{\operatorname{Var}}_N is homogeneous of degree two, since Var⁡^N(cv)=(N−1)−1(cv)TPN(cv)=c2Var⁡^N(v)\widehat{\operatorname{Var}}_N(cv)=(N-1)^{-1}(cv)^{\mathsf{T}}P_N(cv)=c^2\widehat{\operatorname{Var}}_N(v). Hence

Ez[Var⁡^N(fd)]=αd2QdEz[Var⁡^N(gd)]=αd2Qd⋅Qd=αd2\mathbb{E}_z\left[\widehat{\operatorname{Var}}_N(f_d)\right]=\frac{\alpha_d^2}{Q_d}\mathbb{E}_z\left[\widehat{\operatorname{Var}}_N(g_d)\right]=\frac{\alpha_d^2}{Q_d}\cdot Q_d=\alpha_d^2

by the definition (11). Strict positivity of QdQ_d holds under the conditions of Lemma A.1.

Proof of the additive calibration (16). Let Cov⁡^N(u,v)=(N−1)−1uTPNv\widehat{\operatorname{Cov}}_N(u,v)=(N-1)^{-1}u^{\mathsf{T}}P_Nv, which is bilinear and symmetric with Var⁡^N(u)=Cov⁡^N(u,u)\widehat{\operatorname{Var}}_N(u)=\widehat{\operatorname{Cov}}_N(u,u). Bilinearity gives the exact decomposition

Var⁡^N(∑dfd)=∑d=1DVar⁡^N(fd)+2∑1≤d<e≤DCov⁡^N(fd,fe).\widehat{\operatorname{Var}}_N\left(\sum_d f_d\right)=\sum_{d=1}^{D}\widehat{\operatorname{Var}}_N(f_d)+2\sum_{1\le d<e\le D}\widehat{\operatorname{Cov}}_N(f_d,f_e).

For d≠ed\ne e, cyclicity of the trace as in (23) gives

Ez[Cov⁡^N(fd,fe)]=1N−1tr⁡(PNEz[fefdT])=0,\mathbb{E}_z\left[\widehat{\operatorname{Cov}}_N(f_d,f_e)\right]=\frac{1}{N-1}\operatorname{tr}\left(P_N\mathbb{E}_z[f_ef_d^{\mathsf{T}}]\right)=0,

since given XX, ρ\rho and α\alpha the vectors fdf_d and fef_e are functions of the independent zdz_d and zez_e alone and each has mean zero, so Ez[fefdT]=Ez[fe]Ez[fd]T=0\mathbb{E}_z[f_ef_d^{\mathsf{T}}]=\mathbb{E}_z[f_e]\mathbb{E}_z[f_d]^{\mathsf{T}}=0. Applying (15) to each diagonal term gives ∑dαd2\sum_d\alpha_d^2, which equals α2∑dψd=α2\alpha^2\sum_d\psi_d=\alpha^2 because ψ\psi lies in the simplex.

Additional Simulation Results

This appendix gives the full results of the simulation study of § 3. The settings are those of § 3.1. In the figures and tables, ρ\rho in a panel title or in the first column of a table is the generating length scale ρ0\rho_0, and RHS without a qualifier and RHS (p0=5p_0=5) are the calibration with p0=min⁡(5,D−1)p_0=\min(5,D-1).

Tables. For each DGP and kernel, three tables give the coverage of the central 90% posterior intervals, the RMSE of the posterior mean relative to R2-GP, and the sampling diagnostics. All entries are means over 50 replications, with one column for each DD. For α\alpha and ρ\rho, coverage and RMSE are taken across the components. A relative RMSE above one indicates worse recovery than under R2-GP. For each fit, R^\hat{R} is the largest value and ESS is the smallest bulk effective sample size over all αd\alpha_d, ρd\rho_d, σ\sigma and the log density, and divergences are counted among the 4000 post-warm-up draws. Bold marks the best value in each column for each ρ0\rho_0.

Dense DGP

Figures 5 to 7 show the mean Δelpd⁡\Delta\operatorname{elpd} and Figures 8 to 10 the posterior of Remp2R^2_{\mathrm{emp}}, for the three kernels and the three length scales. Figure 3 is the condition with the SE kernel and ρ0=0.3\rho_0=0.3. Tables 2 to 10 give the parameter recovery and the sampling diagnostics. Averaged over replications, the posterior mean of Remp2R^2_{\mathrm{emp}} minus Rtrue2R^2_{\mathrm{true}} lies in [−0.05,0.00][-0.05,0.00] under R2-GP in all conditions. At D=100D=100, it lies in [0.62,0.64][0.62,0.64] under the normal prior, in [0.23,0.62][0.23,0.62] under NG and in [−0.27,−0.21][-0.27,-0.21] under RHS (p0=5p_0=5). The recovery of ρ\rho is governed by its informative prior: its coverage is 100% after rounding and its RMSE is below 0.03 under all priors.

Predictive performance for the dense DGP with the Matérn 3/2 kernel

Figure 6. Predictive Performance, Dense DGP, Matérn 3/2 Kernel. As in Figure 5.

Predictive performance for the dense DGP with the SE kernel

Figure 5. Predictive Performance, Dense DGP, SE Kernel. Mean Δelpd⁡\Delta\operatorname{elpd} over 50 replications (solid) with ±1\pm1 Monte Carlo standard error (dotted), for each generating length scale ρ0\rho_0 (panels). Dashed lines mark ±4\pm4 and negative values favour R2-GP.

Predictive performance for a dense DGP with a Matérn 5/2 kernel

Figure 7. Predictive Performance, Dense DGP, Matérn 5/2 Kernel. As in Figure 5.

Figure 9. Explained Variance, Dense DGP, Matérn 3/2 Kernel. As in Figure 8.

Figure 8. Explained Variance, Dense DGP, SE Kernel. Posterior means (points) and central 90% intervals (bars) of Remp2R^{2}_{\mathrm{emp}} for each replication, for each generating length scale ρ0\rho_{0} (rows). Crosses mark Rtrue2R^{2}_{\mathrm{true}}.

Figure 10. Explained Variance, Dense DGP, Matérn 5/2 Kernel. As in Figure 8.

Sparse DGP

In the sparse DGP only the first five components are active, with αd2=σ2R02/{5(1−R02)}\alpha_d^2 = \sigma^2 R_0^2/\{5(1-R_0^2)\} for d≤5d \le5 and αd=0\alpha_d = 0 otherwise, so that the total signal variance equals that of the dense DGP (19). All other settings are those of § 3.1. The priors are R2-GP, normal, NG and RHS with p0=min⁡(5,D−1)p_0 = \min(5,D-1), which equals the number of active components for D≥10D \ge10. Figure 11 to Figure 16 and Table 11 to Table 19 give the results. The tables report the RMSE of α\alpha separately for the active and the inactive components. For D≥30D \ge30, the intervals of R2-GP contain the generating value of an active αd\alpha_d in at most 27% of cases, and those of RHS in at most 49%. This agrees with the uniform allocation for a=1a = 1 that § 3.2 discusses. The recovery of σ\sigma and of Rtrue2R^2_{\mathrm{true}} under R2-GP stays accurate.

Figure 11. Predictive Performance, Sparse DGP, SE Kernel. As in Figure 5.

Figure 16. Explained Variance, Sparse DGP, Matérn 5/25/2 Kernel. As in Figure 8.

ρ\rhoPriorα\alpha5103050100ρ\rho5103050100σ\sigma5103050100
0.3R2-GP9777231111001001001001009282849494
Normal9596959510010010010010010090783020
NG9285646760100100100100100928080808
RHS (p0=5p_0=5)85664441231001001001001009278809084
0.5R2-GP998421901001001001001009484929090
Normal959597969810010010010010092785280
NG94906663641001001001001009480888046
RHS (p0=5p_0=5)88754740251001001001001009484948294
0.7R2-GP988218611001001001001009288829090
Normal9495979499100100100100100928268180
NG98916656551001001001001009282869076
RHS (p0=5p_0=5)88764538261001001001001009284888690

Table 11. Interval Coverage (%), Sparse DGP, SE Kernel. As in Table 2, for the active components. The interval of a positive parameter cannot contain the zero of an inactive component.

ρ\rhoPriorR^\widehat{R} 5R^\widehat{R} 10R^\widehat{R} 30R^\widehat{R} 50R^\widehat{R} 100ESS 5ESS 10ESS 30ESS 50ESS 100Divergences 5Divergences 10Divergences 30Divergences 50Divergences 100Time (s) 5Time (s) 10Time (s) 30Time (s) 50Time (s) 100
0.3R2-GP1.001.001.011.011.02103310327455823860.00.00.12.1207.6123775234
Normal1.001.011.021.221.3493086736830181.00.5322282317.9201307682202
NG1.011.041.141.311.13536239422346101365814320473044841628
RHS (p0=5p_0=5)1.011.011.011.021.028797684824323762.21.89.7142611184895295
0.5R2-GP1.001.001.011.011.011163117110368125700.20.00.10.3128.9144181236
Normal1.001.011.011.161.381004102670677171.81.32.712918913251316062350
NG1.011.041.141.221.396232334120256.7431371347417533645901465
RHS (p0=5p_0=5)1.001.011.011.011.029779126955484252.65.57.91531131956110323
0.7R2-GP1.001.001.011.011.0112341262116810067010.20.00.10.11.111185195311
Normal1.001.011.011.051.5610691083898298125.02.11.44818515301615732116
NG1.011.051.111.191.6565327044251413511641465619673807161724
RHS (p0=5p_0=5)1.001.011.011.011.02101010188276664967.07.99.21430152564131374

Table 19. Sampling Diagnostics, Sparse DGP, Matérn 5/2 Kernel. As in Table Table 4.

Figure 12. Predictive Performance, Sparse DGP, Matérn 3/2 Kernel. As in Figure 5.

Figure 13. Predictive Performance, Sparse DGP, Matérn 5/2 Kernel. As in Figure 5.

Figure 14. Explained Variance, Sparse DGP, SE Kernel. As in Figure 8.

Figure 15. Explained Variance, Sparse DGP, Matérn 3/23/2 Kernel. As in Figure 8.

ρ\rhoPriorR^\hat{R}5103050100ESS5103050100Divergences5103050100Time (s)5103050100
0.3R2-GP1.001.001.011.011.01104010038436454630.00.00.00.5116.29.12649124
Normal1.001.011.011.181.2994690150351201.41.73.92512538.015754371377
NG1.011.031.151.321.245242433417435.72895110811238212356904
RHS (p0=5p_0=5)1.001.011.011.021.029048326274174873.03.45.012218.6133565156
0.5R2-GP1.001.001.001.011.011191118610879606480.00.00.00.11.97.3113253137
Normal1.001.001.011.021.5110541047836250122.41.52.3702221019873031331
NG1.031.021.111.201.595942934223191120121146921333224435867
RHS (p0=5p_0=5)1.001.011.011.011.0310109256896255476.54.211164010164072169
0.7R2-GP1.001.001.011.011.0112901266120411158110.70.10.10.11.48.3133764167
Normal1.001.001.011.011.5010941108949562114.01.72.79.218112261233081303
NG1.011.031.111.221.596742924622925531611758916442604671047
RHS (p0=5p_0=5)1.001.011.011.011.01106710278287636129.15.816112112185085199

Table 13. Sampling Diagnostics, Sparse DGP, SE Kernel. As in Table 4.

ρ\rhoPrior\multicolumn{5}{c}{α\alpha}\multicolumn{5}{c}{ρ\rho}\multicolumn{5}{c}{σ\sigma}
510305010051030501005103050100
0.3R2-GP9880271101001001001001009488988894
Normal959594971001001001001001009886200
NG9584706964100100100100100988686640
RHS (p0=5p_0=5)88684933241001001001001008286906680
0.5R2-GP978018601001001001001008890889290
Normal9393969810010010010010010084862600
NG94856059621001001001001008486927818
RHS (p0=5p_0=5)88703833301001001001001008688948282
0.7R2-GP988019701001001001001008886969686
Normal9493969610010010010010010088824000
NG94885958641001001001001008884969022
RHS (p0=5p_0=5)87714034321001001001001008484968082

Table 14. Interval Coverage (%), Sparse DGP, Matérn 3/2 Kernel. As in Table 11.

ρ\rhoPrior\multicolumn{5}{c}{α\alpha, active}\multicolumn{5}{c}{α\alpha, inactive}\multicolumn{5}{c}{ρ\rho}\multicolumn{5}{c}{σ\sigma}
5103050100510305010051030501005103050100
0.3R2-GP0.0870.1210.1760.2070.244—0.1630.1090.0850.0600.0190.0160.0150.0120.0100.0560.0660.0790.1060.103
Normal1.611.140.930.660.27—1.352.212.724.151.071.151.361.381.150.941.034.796.917.46
NG1.561.341.110.940.70—0.770.720.931.620.970.981.091.191.680.971.051.492.457.20
RHS (p0=5p_0=5)1.701.511.241.171.10—0.550.480.430.360.870.830.820.720.831.171.201.081.211.36
0.5R2-GP0.0960.1160.1870.2130.235—0.1630.1000.0810.0640.0120.0110.0090.0070.0080.0650.0670.0660.0790.123
Normal1.781.430.790.700.38—1.362.453.354.130.980.941.161.281.251.021.003.547.646.29
NG1.601.441.141.060.73—0.750.630.631.540.940.900.931.051.401.020.991.301.485.17
RHS (p0=5p_0=5)1.681.551.201.131.07—0.550.490.430.420.890.800.780.790.891.081.131.101.421.02
0.7R2-GP0.0970.1170.1870.2130.235—0.1620.0990.0790.0630.0070.0070.0060.0050.0050.0630.0660.0540.0680.114
Normal1.821.530.860.810.48—1.452.573.674.490.900.951.121.281.790.970.993.306.836.83
NG1.591.451.151.110.74—0.780.600.551.490.870.911.000.981.471.000.971.131.244.51
RHS (p0=5p_0=5)1.661.531.181.111.06—0.550.490.440.430.840.820.830.830.881.091.111.001.370.96

Table 15. Relative RMSE, Sparse DGP, Matérn 3/2 Kernel. As in Table 12.

ρ\rhoPriorα\alpha, active5103050100α\alpha, inactive5103050100ρ\rho5103050100σ\sigma5103050100
0.3R2-GP0.0970.1180.1790.2030.242—0.1580.1030.0860.0600.0220.0220.0170.0160.0130.0530.0820.0840.0770.102
Normal1.571.300.950.800.35—1.402.342.984.351.061.171.391.420.950.980.932.667.947.56
NG1.521.391.211.010.72—0.740.560.701.650.931.001.071.181.671.001.051.241.976.88
RHS (p0=5p_0=5)1.641.491.231.141.08—0.540.470.440.390.880.870.820.780.831.111.151.181.231.15
0.5R2-GP0.0920.1120.1800.2050.243—0.1630.1030.0830.0580.0160.0140.0120.0110.0080.0530.0670.0730.0750.081
Normal1.961.641.000.900.52—1.492.603.555.090.991.021.111.162.080.971.032.034.929.85
NG1.651.411.171.130.77—0.820.620.481.440.920.910.920.951.701.001.041.001.045.30
RHS (p0=5p_0=5)1.761.501.171.111.06—0.580.520.460.420.870.810.830.810.921.071.111.011.021.04
0.7R2-GP0.0950.1180.1830.2080.243—0.1600.1000.0800.0570.0110.0100.0090.0080.0060.0530.0660.0690.0690.074
Normal2.071.641.070.970.69—1.672.893.905.740.950.981.051.042.520.981.001.643.5610.69
NG1.701.341.181.180.94—0.850.600.460.950.910.910.890.931.361.001.011.020.912.52
RHS (p0=5p_0=5)1.791.471.141.091.05—0.590.530.470.440.860.800.880.810.891.051.110.970.980.95

Table 12. Relative RMSE, Sparse DGP, SE Kernel. As in Table 3, with the RMSE of α\alpha taken separately across the active and the inactive components, and the RMSE of ρ\rho across the active components. At D=5D=5 all components are active.

ρ\rhoPrior\multicolumn{5}{c}{R^\hat{R}}\multicolumn{5}{c}{ESS}\multicolumn{5}{c}{Divergences}\multicolumn{5}{c}{Time (s)}
5103050100510305010051030501005103050100
0.3R2-GP1.001.001.011.011.01104910297595554260.00.00.14.16.27.9144586282
Normal1.001.011.061.281.3792584020925140.50.6592482339.8211888473109
NG1.011.031.151.361.155182404023412.82961957716533125591934
RHS (p0=5p_0=5)1.011.011.011.011.038497504584754162.00.9161141111858107345
0.5R2-GP1.001.001.011.011.011181115610207504830.10.00.10.16.09.1154889324
Normal1.001.001.011.191.331041100057640180.81.49.220122312251467382664
NG1.011.041.141.241.325992714722355.6351141518818533336541844
RHS (p0=5p_0=5)1.001.011.011.021.029848887045424192.33.59.12839132360107323
0.7R2-GP1.001.001.011.011.011239123911219045810.50.00.10.43.4111853101372
Normal1.001.011.011.071.3710691057782112172.11.02.417020415321626662537
NG1.011.021.121.201.41693310432521174010616510822653827201967
RHS (p0=5p_0=5)1.001.011.011.011.0310149647486334635.15.78.21543152568130466

Table 16. Sampling Diagnostics, Sparse DGP, Matérn 3/2 Kernel. As in Table 4.

ρ\rhoPriorα\alpha 5α\alpha 10α\alpha 30α\alpha 50α\alpha 100ρ\rho 5ρ\rho 10ρ\rho 30ρ\rho 50ρ\rho 100σ\sigma 5σ\sigma 10σ\sigma 30σ\sigma 50σ\sigma 100
0.3R2-GP9282231411001001001001008292949892
Normal9296969610010010010010010088861200
NG9086666865100100100100100848484662
RHS (p0=5p_0=5)81674240261001001001001008286909274
0.5R2-GP9874181301001001001001009092949086
Normal9493979210010010010010010088884640
NG94846364641001001001001008692908610
RHS (p0=5p_0=5)88664344281001001001001008888809478
0.7R2-GP9876191001001001001001008894949086
Normal9394959399100100100100100909052100
NG94876261701001001001001008890888634
RHS (p0=5p_0=5)89704245321001001001001008692869088

Table 17. Interval Coverage (%), Sparse DGP, Matérn 5/2 Kernel. As in Table Table 11.

ρ\rhoPriorα\alpha, active 5α\alpha, active 10α\alpha, active 30α\alpha, active 50α\alpha, active 100α\alpha, inactive 5α\alpha, inactive 10α\alpha, inactive 30α\alpha, inactive 50α\alpha, inactive 100ρ\rho 5ρ\rho 10ρ\rho 30ρ\rho 50ρ\rho 100σ\sigma 5σ\sigma 10σ\sigma 30σ\sigma 50σ\sigma 100
0.3R2-GP0.1000.1140.1830.2010.240—0.1620.1030.0870.0620.0200.0180.0150.0140.0110.0670.0700.0690.0850.124
Normal1.571.330.840.790.30—1.322.312.764.041.011.171.261.411.051.001.124.318.106.19
NG1.541.391.120.990.69—0.770.670.831.570.921.010.981.331.771.021.051.432.475.98
RHS (p0=5p_0=5)1.621.511.221.161.09—0.570.460.440.400.880.910.780.850.801.091.081.251.211.03
0.5R2-GP0.0990.1250.1860.1950.237—0.1580.0980.0890.0610.0130.0120.0100.0100.0080.0550.0600.0640.0770.114
Normal1.831.280.840.990.39—1.462.553.134.420.941.021.171.321.431.021.062.686.896.87
NG1.601.341.151.160.72—0.770.520.621.590.900.900.931.081.481.011.031.191.405.28
RHS (p0=5p_0=5)1.681.471.191.111.07—0.540.460.510.420.850.810.740.870.851.041.091.230.970.99
0.7R2-GP0.1000.1260.1880.1970.234—0.1560.0960.0870.0620.0090.0080.0070.0070.0060.0530.0610.0630.0740.102
Normal1.921.340.921.050.55—1.562.763.364.740.900.941.060.942.261.011.002.074.937.88
NG1.621.321.161.200.80—0.800.520.561.270.890.890.920.871.161.001.011.171.153.79
RHS (p0=5p_0=5)1.721.451.171.101.04—0.550.470.510.440.860.830.760.810.861.011.051.200.920.98

Table 18. Relative RMSE, Sparse DGP, Matérn 5/2 Kernel. As in Table Table 12.

Non-normalised Competing Priors

All comparisons above apply the competing priors to the normalised components (14). Here we place the normal, NG and RHS priors on αd\alpha_d in the usual HSGP parameterisation, fd=αdgdf_d = \alpha_d g_d, in which αd2\alpha_d^2 is the marginal variance. R2-GP remains the reference, and all other settings are unchanged. Since αd\alpha_d is then on a different scale from the generating value, we compare predictive performance. Figure 17 to Figure 22 show that the qualitative behaviour of the competing priors is the same as with the normalisation.

Non-normalised competing priors for the dense DGP with the SE kernel

Figure 17. Non-normalised Competing Priors, Dense DGP, SE Kernel. As in Figure 5, with the normal, NG and RHS priors on αd\alpha_d in fd=αdgdf_d = \alpha_d g_d.

Non-normalised competing priors, sparse DGP, Matérn 5/2 kernel

Figure 22. Non-normalised Competing Priors, Sparse DGP, Matérn 5/2 Kernel. As in Figure 17.

In the dense DGP, R2-GP attains the highest mean elpd in each condition against every non-normalised prior. At D=100D = 100 the mean Δ\Deltaelpd lies between −149-149 and −105-105 for the normal prior, between −47-47 and −33-33 for NG, and between −9.1-9.1 and −4.5-4.5 for RHS. Since Qd<1Q_d < 1 for the kernels considered, a given prior on αd\alpha_d implies less signal variance without the normalisation. This lessens the overestimation under the normal and NG priors, whose mean elpd improves by up to 37 and 24, but does not remove it. For RHS the mean elpd is lower by at most 1.4 in both DGPs. In the sparse DGP, RHS again attains a higher mean elpd than R2-GP for D≥30D \ge30, by at most 12.1, and the normal prior again underperforms, with a mean Δ\Deltaelpd between −150-150 and −106-106 at D=100D = 100. At D=100D = 100 the bias in Remp2R^2_{\mathrm{emp}} has the same sign and a similar range for each prior as with the normalisation, and the normal and NG priors again fail to converge (not shown).

Non-normalised competing priors for the dense DGP with the Matérn 3/2 kernel

Figure 18. Non-normalised Competing Priors, Dense DGP, Matérn 3/2 Kernel. As in Figure 17.

Non-normalised competing priors, dense DGP, Matérn 5/2 kernel

Figure 19. Non-normalised Competing Priors, Dense DGP, Matérn 5/2 Kernel. As in Figure 17.

Non-normalised competing priors, sparse DGP, SE kernel

Figure 20. Non-normalised Competing Priors, Sparse DGP, SE Kernel. As in Figure 17.

Non-normalised competing priors, sparse DGP, Matérn 3/2 kernel

Figure 21. Non-normalised Competing Priors, Sparse DGP, Matérn 3/2 Kernel. As in Figure 17.

Additional Case Study Results

Figures 23 and 24 extend Figure 4 to 12 covariates, for the SE and the Matérn 3/2 kernel. The covariates are ranked by their posterior mean share of explained variance under R2-GP with the SE kernel, and both figures show the same covariates. For these figures, the models of § 4 are fitted to all 208 compounds. Table 20 gives the predictive performance and the sampling diagnostics over the 10 train/test splits.

PriorSEMatérn 3/2
ELPDR2R^2Roos2R^2_{\mathrm{oos}}R^\hat{R}ESSDiv.TimeELPDR2R^2Roos2R^2_{\mathrm{oos}}R^\hat{R}ESSDiv.Time
R2-GP−40.90.730.561.0085290.051−40.20.770.561.0164790.0126
RHS−41.40.720.551.0231780.272−41.00.760.551.0231500.0176
NG−41.30.700.551.232140.061−40.70.740.551.037710.0126
Normal−46.00.950.562.11752.5100−48.91.000.553.462423.8257

Table 20. Case Study Predictive and Sampling Diagnostics

Covariate effects for the BloodBrain data across twelve covariates, comparing R2-GP, RHS, NG, and Normal models

Figure 24. Covariate Effects, BloodBrain Data, Matérn 3/2 Kernel. As in Figure 23, for the same covariates.

Covariate effects for 12 covariates in the BloodBrain data using the SE kernel

Figure 23. Covariate Effects, BloodBrain Data, SE Kernel. Posterior means (lines) and pointwise 90% intervals (bands) of the centred effects fd(x)−fˉdf_d(x) - \bar{f}_d for the 12 covariates with the largest share of explained variance. Ticks mark the observed covariate values.

Note: Predictive performance and sampling behaviour of fitted models on the BloodBrain data over 10 random train/test split replications. ELPD and posterior R2R^2 (in-sample and out-of-sample) are reported as means over the replications. R^\hat{R} and ESS are reported as the worst case over all model parameters across replications; Div. is the mean number of divergent transitions per fit and time the median sampling time per fit in minutes.

Computing Infrastructure

All computations ran on CPUs; no GPUs were used. The simulation study was run on an institutional high-performance computing cluster managed by Slurm, on nodes with Intel Xeon (Haswell to Cascade Lake) and AMD EPYC (Milan) processors. Each fit was allocated four cores, one per chain, and 3 GB of memory, and the workflow was orchestrated with Snakemake 8.18. Models were fitted with CmdStan 2.35.0 [3] through the R package cmdstanr 0.9.0 in R 4.5.3, compiled with GCC 12.3.

The simulation comprises 57 750 model fits, all of which completed. Sampling took 1.3 minutes per fit at the median and 5.9 minutes on average; 99% of the fits finished within about an hour and the slowest took 4.7 hours. In total this is about 5 700 hours of four-core time, or 23 000 core-hours. The case study adds 80 fits on the data splits, which took between 27 minutes and 4.5 hours each (median 1.7 hours), about 650 core-hours in total, and eight fits to the full data.

References

  1. [1]Javier Enrique Aguilar and Paul-Christian BürKner. Intuitive joint priors for Bayesian linear multilevel models: The R2D2M2 prior. Electronic Journal of Statistics, 17(1):1711–1767, 2023. doi: 10.1214/23-EJS2136.DOI
  2. [2]Javier Enrique Aguilar, David Kohns, Aki Vehtari, and Paul-Christian BürKner. R2 priors for grouped variance decomposition in high-dimensional regression. arXiv preprint arXiv:2507.11833, 2025.arxiv.org/abs/2507.11833
  3. [3]Bob Carpenter, Andrew Gelman, Matthew D. Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, and Allen Riddell. Stan: A probabilistic programming language. Journal of Statistical Software, 76(1):1–32, 2017. doi: 10.18637/jss.v076.i01.DOI
  4. [4]David Duvenaud, Hannes Nickisch, and Carl Edward Rasmussen. Additive Gaussian processes. In Advances in Neural Information Processing Systems, volume 24, 2011. Page range to be confirmed.arxiv.org/abs/1112.4394
  5. [5]Luisa Ferrari and Massimo Ventrucci. A standardization procedure to incorporate variance partitioning based priors in latent Gaussian models. arXiv preprint arXiv:2501.16057, 2025.arxiv.org/abs/2501.16057
  6. [6]Geir-Arne Fuglstad, Daniel Simpson, Finn Lindgren, and Håvard Rue. Constructing priors that penalize the complexity of Gaussian random fields. Journal of the American Statistical Association, 114(525):445–452, 2019. doi: 10.1080/01621459.2017.1415907.DOI
  7. [7]Jim E. Griffin and Philip J. Brown. Inference with normal-gamma prior distributions in regression problems. Bayesian Analysis, 5(1):171–188, 2010. doi: 10.1214/10-BA507.DOI
  8. [8]Trevor Hastie and Robert Tibshirani. Generalized additive models. Statistical Science, 1(3):297–310, 1986.
  9. [9]Trevor J. Hastie and Robert J. Tibshirani. Generalized Additive Models. Chapman and Hall, London, 1990.
  10. [10]Matthew D. Hoffman and Andrew Gelman. The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15:1593–1623, 2014.DOI
  11. [11]David Kohns, Noa Kallioinen, Yann McLatchie, and Aki Vehtari. The ARR2 prior: Flexible predictive prior definition for Bayesian auto-regressions. Bayesian Analysis, 2025. doi: 10.1214/25-BA1512. Advance publication.DOI
  12. [12]Max Kuhn. Building predictive models in R using the caret package. Journal of Statistical Software, 28(5):1–26, 2008.
  13. [13]Yann McLatchie, Sölvi Rögnvaldsson, Frank Weber, and Aki Vehtari. Advances in projection predictive inference. Statistical Science, 40(1):128–147, 2025. doi: 10.1214/24-STS949.DOI
  14. [14]Scot R. Mente and Franco Lombardo. A recursive-partitioning model for blood–brain barrier permeation. Journal of Computer-Aided Molecular Design, 19(7):465–481, 2005.DOI
  15. [15]Soham Mukherjee, Manfred Claassen, and Paul-Christian Bürkner. Hilbert space methods for approximating multi-output latent variable gaussian processes. Statistics and Computing, 36:110, 2026. doi: https://doi.org/10.1007/s11222-026-10869-x.
  16. [16]Radford M. Neal. MCMC using Hamiltonian dynamics. In Steve Brooks, Andrew Gelman, Galin L. Jones, and Xiao-Li Meng, editors, Handbook of Markov Chain Monte Carlo, pages 113–162. Chapman and Hall/CRC, 2011.DOI
  17. [17]Omiros Papaspiliopoulos, Gareth O. Roberts, and Martin Sköld. A general framework for the parametrization of hierarchical models. Statistical Science, 22(1):59–73, 2007. doi: 10.1214/088342307000000014.DOI
  18. [18]Juho Piironen and Aki Vehtari. Sparsity information and regularization in the horseshoe and other shrinkage priors. Electronic Journal of Statistics, 11(2):5018–5051, 2017. doi: 10.1214/17-EJS1337SI.DOI
  19. [19]Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA, 2006.DOI
  20. [20]Anna Elisabeth Riha, Leevi Lindgren, David Kohns, Paul-Christian Bürkner, and Aki Vehtari. To select or not to select: predictively consistent priors instead of model selection. arXiv preprint arXiv:2606.22850, 2026.arxiv.org/abs/2606.22850
  21. [21]Gabriel Riutort-Mayol, Paul-Christian Bürkner, Michael R. Andersen, Arno Solin, and Aki Vehtari. Practical Hilbert space approximate Bayesian Gaussian processes for probabilistic programming. Statistics and Computing, 33(1):17, 2023. doi: 10.1007/s11222-022-10167-2.
  22. [22]Daniel Simpson, Håvard Rue, Andrea Riebler, Thiago G. Martins, and Sigrunn H. Sørbye. Penalising model component complexity: A principled, practical approach to constructing priors. Statistical Science, 32(1):1–28, 2017. doi: 10.1214/16-STS576.DOI
  23. [23]Tuomas Sivula, Måns Magnusson, Asael Alonzo Matamoros, and Aki Vehtari. Uncertainty in Bayesian leave-one-out cross-validation based model comparison. Bayesian Analysis, 2025. doi: 10.1214/25-BA1569. Advance publication.DOI
  24. [24]Arno Solin and Simo Särkkä. Hilbert space methods for reduced-rank Gaussian process regression. Statistics and Computing, 30(2):419–446, 2020. doi: 10.1007/s11222-019-09886-w.DOI
  25. [25]Juho Timonen and Harri Lähdesmäki. Scalable Mixed-Domain Gaussian Process Modeling and Model Reduction for Longitudinal Data. Bayesian Analysis, pages 1 – 28, 2025. doi: 10.1214/25-BA1539. URL https://doi.org/10.1214/25-BA1539.
  26. [26]Aki Vehtari, Andrew Gelman, and Jonah Gabry. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27(5):1413–1432, 2017. doi: 10.1007/s11222-016-9696-4.DOI
  27. [27]Christopher Williams and Carl Rasmussen. Gaussian processes for regression. In D. Touretzky, M. C. Mozer, and M. Hasselmo, editors, Advances in Neural Information Processing Systems, volume 8. MIT Press, 1995.
  28. [28]Simon N. Wood. Generalized Additive Models, An Introduction with R, Second Edition. Chapman and Hall/CRC, 2017.
  29. [29]Eric Yanchenko, Howard D Bondell, and Brian J Reich. Spatial regression modeling via the r2d2 framework. Environmetrics, 35(2):e2829, 2024.arxiv.org/abs/2301.09951
  30. [30]Eric Yanchenko, Howard D Bondell, and Brian J Reich. The r2d2 prior for generalized linear mixed models. The American Statistician, 79(1):40–49, 2025.DOI
  31. [31]Yan Dora Zhang, Brian P. Naughton, Howard D. Bondell, and Brian J. Reich. Bayesian regression using a prior on the model fit: The R2-D2 shrinkage prior. Journal of the American Statistical Association, 117(538):862–874, 2022. doi: 10.1080/01621459.2020.1825449.

Paper details

vlnc:W_zh2nc7m9
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Claude Opus 5 High, ChatGPT 5.6

View statement
Looking for related papers…
Contents