betaFeedback

WorksMathematics

Higher Order Risk Comparisons of Oracle Calibrated Global Local Priors

Abstract

First-order minimaxity offers limited guidance for choosing among global-local shrinkage priors, since different local-tail distributions can yield the same leading risk. Comparisons at a common global scale may also be misleading, as the scale interacts with the shape of the local prior. We study the worst-case squared-error risks of posterior means in sparse Gaussian normal-means estimation, allowing each prior its own risk-minimizing global scale. For local-variance priors with polynomial tails and logarithmic corrections, we characterize the calibrations that retain first- and second-order minimaxity and derive the asymptotic form of the unrestricted oracle scale. Even after this optimization, differences in risk remain. The polynomial tail index, logarithmic tail exponent and a scale-invariant higher-order correction distinguish priors at successively finer orders. In particular, the horseshoe has smaller asymptotic oracle risk than the horseshoe+, despite the latter's additional logarithmic tail factor. We also establish the calibration accuracy needed to preserve these differences and sufficient conditions for their stability under high-dimensional Gaussian regression designs.

Introduction

Sparse Bayesian estimation has developed through both priors that explicitly represent sparse model structure, such as spike-and-slab constructions, and continuous shrinkage priors that approximate sparsity without assigning positive mass to exact zeros ([14, 34]). Among the latter, global–local priors combine strong shrinkage of small coefficients with heavy tails that protect large signals, while retaining a convenient continuous parameterization ([12, 30]). Constructions including the normal–gamma, generalized beta, horseshoe, Dirichlet–Laplace and horseshoe+ priors exhibit substantial variation in local-tail behaviour and global shrinkage ([21, 2, 12, 8, 7]). This diversity raises a prior-selection question that leading-order optimality alone cannot resolve: how should one distinguish continuous shrinkage priors that are already equivalent at the sparse minimax order?

Classical sparse-estimation theory distinguishes minimax rates from sharper risk benchmarks. For nearly black normal-mean classes and broader ℓp\ell_p-type spaces, the leading minimax risks were characterized in ([17, 16]), while Johnstone [24] obtained a refined minimax expansion showing that first-order-equivalent procedures can differ at the next asymptotic order. Adaptation to unknown sparsity has also been studied extensively: empirical-Bayes thresholding and shrinkage can recover sharp minimax benchmarks [25, 42, 23], as can procedures based on false-discovery-rate thresholding and sorted ℓ1\ell_1 regularization [1] (Su and Candès 2016). These developments establish the relevance of sharper risk comparisons beyond first-order rates, but do not determine how the tail structures of different global–local priors affect their higher-order risks.

For continuous global–local shrinkage, the sparse normal-means model has provided the principal setting for first-order optimality theory. The horseshoe posterior mean attains the minimax squared-error rate under suitable global-scale calibration [39], and general conditions on local-scale priors ensure minimax posterior contraction [40]. For broader classes of regularly varying one-group priors, Ghosh and Chakrabarti [19] established asymptotic minimaxity of posterior means, including the exact leading constant for an important horseshoe-type subclass, while Qin and Ghosh [31] extended exact asymptotic minimax point-estimation results to additional global–local priors and the Dirichlet–Laplace construction. Empirical-Bayes and hierarchical global-scale specifications further permit adaptation to unknown sparsity [38, 27]. Collectively, these results provide a substantial first-order theory, but leave unresolved the risk ordering among priors after their leading minimax performance has been matched.

Finer aspects of shrinkage-prior behaviour have received increasing attention. Regular variation characterizes the interaction between concentration near zero and heavy tails [6], while tail modifications can affect large-signal robustness and testing performance [7, 22]. Polynomial tail order also influences sharp contraction properties [35], and tail-adaptive priors permit data-dependent adjustment of tail behaviour [26]. Global-scale calibration provides another source of regularization, since incorporating sparsity information can substantially change shrinkage even for a fixed local prior [29]. Related multiple-testing theory establishes asymptotic Bayes and sharp minimax properties for the horseshoe and broader one-group procedures under testing losses [10, 15, 20, 28]. These results show that prior tails and global scales are statistically consequential, but improvements under large-signal, contraction or testing criteria need not imply an ordering under sparse worst-case squared-error loss. To our knowledge, existing theory does not identify the first separating order of the worst-case risks of different global–local posterior means after each prior has been placed on its own risk-optimal global scale, or characterize the successive tail features governing that separation. The central question is therefore which prior characteristics remain distinguishable after oracle calibration, at what risk resolutions, and with what calibration accuracy.

Extending such comparisons to high-dimensional regression introduces an additional difficulty through design dependence. Restricted-isometry and restricted-eigenvalue conditions control sparse geometry [11, 9, 32] and underpin minimax regression theory [33]. Bayesian regression results similarly establish concentration, prediction and estimation guarantees under sparse-design conditions for discrete and continuous shrinkage priors [13, 5, 36] (Armگان et al. 2013). Our objective differs from these first-order guarantees: we ask how closely a regression design must approximate orthogonality for a finer, prior-specific risk separation from the Gaussian sequence model to remain visible. Sparse design geometry thus becomes a resolution requirement for transferring higher-order comparisons.

We study posterior-mean estimation under squared-error loss over nearly black normal-mean classes and a broad log-regular family of global–local priors whose local-variance tails combine polynomial decay with logarithmic modification. The analysis gives a four-part answer. First, we derive a sharp calibration phase diagram identifying how the polynomial tail index and global scale must be coupled to attain first- and second-order minimaxity, separating second-order-optimal, first-order-only and sub-optimal regimes. Second, optimizing over all positive global scales yields the unrestricted prior-specific oracle calibration, together with a quadratic excess-risk penalty 4α2st2{1+o(1)}4\alpha^2st^2\{1+o(1)\} for local multiplicative perturbations τ=πroet\tau=\pi_r^o e^t. Third, after oracle calibration removes leading scale differences, the remaining risks are resolved successively by the polynomial tail index α\alpha, logarithmic tail exponent γ\gamma and scale-invariant next-order coefficient ζπ\zeta_\pi, at first separating orders s/Ls/L, s/L2s/L^2 and s/L3s/L^3, respectively. This lexicographic hierarchy implies, in particular, that the horseshoe has smaller oracle worst-case risk than the horseshoe+ at their first asymptotically distinguishable order. Fourth, we transfer the hierarchy to high-dimensional Gaussian linear regression: under sparse near-orthogonality, the regression risk differs from its sequence-model counterpart by O[sL2{γn+δn2}]O[sL^2\{\gamma_n+\delta_n^2\}], so a separation of order s/Lks/L^k is preserved under the sufficient design-resolution condition γn+δn2=o(L−k−2)\gamma_n+\delta_n^2=o(L^{-k-2}).

These results identify first-order minimaxity as a screening principle rather than a complete basis for prior choice. Once leading performance has been matched, prior shape determines the first separating risk order, while calibration and design perturbations determine whether that separation remains visible. This resolution principle is specific to posterior means under sparse worst-case squared-error loss and does not imply a universal prior ranking across other inferential criteria.

Sparse normal means and log-regular shrinkage priors

Sparse normal means and risk

We observe independent coordinates from the Gaussian sequence model

Yi=θi+εi,εi∼N(0,1)(i=1,…,p),Y_i = \theta_i + \varepsilon_i,\quad\varepsilon_i \sim N(0,1)\quad(i=1,\ldots,p),

where the unknown mean vector θ=(θ1,…,θp)T\theta=(\theta_1,\ldots,\theta_p)^{\mathsf T} is sparse. We consider the nearly black parameter class

Θp,s={θ∈Rp:∥θ∥0≤s},s/p→0,L=log⁡(p/s)→∞.\Theta_{p,s}=\{\theta\in\mathbb{R}^p:\|\theta\|_0\le s\},\quad s/p\to0,\quad L=\log(p/s)\to\infty.

Thus ss bounds the number of nonzero coordinates, while LL is the logarithmic sparsity scale used throughout the asymptotic analysis.

For an estimator T=T(Y)T=T(Y), let Rp,s(T)R_{p,s}(T) denote its worst-case squared-error risk over Θp,s\Theta_{p,s}. The corresponding minimax risk satisfies

Rp,s(T)=sup⁡θ∈Θp,sEθ{∥T−θ∥22},Rp,s∗=inf⁡TRp,s(T)=2sL{1+o(1)};(1)R_{p,s}(T)=\sup_{\theta\in\Theta_{p,s}}E_\theta\{\|T-\theta\|_2^2\},\quad R^*_{p,s}=\inf_T R_{p,s}(T)=2sL\{1+o(1)\}; \tag*{(1)}

see Donoho et al. [17]; Johnstone [24]. We study posterior-mean estimators under continuous global–local shrinkage priors and compare their risks with the benchmark in (1). The leading term 2sL2sL determines first-order minimaxity, while the subsequent analysis resolves distinctions that remain after this dominant sparse-estimation difficulty has been matched.

Log-regular global–local priors

We place independent global–local priors on the coordinates of θ\theta. Writing uiu_i for the squared local scale, the hierarchy is

θi∣ui,τ∼N(0,τ2ui),ui∼π(i=1,…,p),(2)\theta_i\mid u_i,\tau\sim N(0,\tau^2u_i),\quad u_i\sim\pi\quad(i=1,\ldots,p), \tag*{(2)}

where τ>0\tau>0 is the global scale and π\pi is a density on (0,∞)(0,\infty). Throughout, tail conditions are stated for the local variance uiu_i, rather than for its square root. For a given π\pi and τ\tau, let θ^π,τ=E(θ∣Y)\widehat{\theta}_{\pi,\tau}=E(\theta\mid Y) denote the posterior mean under (2), and write Rp,s(π,τ)=Rp,s(θ^π,τ)R_{p,s}(\pi,\tau)=R_{p,s}(\widehat{\theta}_{\pi,\tau}).

For a scalar observation yy, write θ^π,τ(y)=y w(y,τ)\widehat{\theta}_{\pi,\tau}(y)=y\,w(y,\tau), where w(y,τ)=1−E{(1+τ2u)−1∣y}w(y,\tau)=1-E\{(1+\tau^2u)^{-1}\mid y\} is the posterior nonshrinkage weight, with dependence on π\pi suppressed in the notation. In the small-τ\tau regimes considered below, the equation w(aτ,τ)=1/2w(a_\tau,\tau)=1/2 has a unique positive solution for all sufficiently large pp; we call aτa_\tau the intrinsic transition threshold. It marks the point at which the posterior mean retains one half of the observed signal and provides the natural coordinate for the calibration and oracle-risk expansions in § 3.1.

Our analysis distinguishes priors not only through their polynomial tail order but also through logarithmic components that are invisible to first-order regular-variation theory. We use the following log-regular class.

Assumption 1 (Log-regular local scales). The density π\pi is positive, proper and three times continuously differentiable on (0,∞)(0,\infty). For some Cπ>0C_\pi> 0, α>0\alpha> 0 and γ,β,χ∈R\gamma,\beta,\chi\in\mathbb{R}

π(u)=Cπu−α−1(log⁡u)γ{1+βlog⁡u+χ(log⁡u)2+ρ(u)}(u→∞).(3)\pi(u) = C_\pi u^{-\alpha-1}(\log u)^\gamma\left\{1 + \frac{\beta}{\log u} + \frac{\chi}{(\log u)^2} + \rho(u)\right\} \qquad(u \to\infty). \tag*{(3)}

Writing D=u d/duD = u\,d/du, the remainder satisfies

Djρ(u)=O{(log⁡u)−3−j}(j=0,1,2,3).D^j\rho(u) = O\{(\log u)^{-3-j}\} \qquad(j = 0,1,2,3).

At the origin, π\pi is regularly varying with index δ0−1\delta_0 - 1 for some δ0>0\delta_0 > 0; equivalently,

π(u)=uδ0−1ℓ0(1/u)(u↓0),\pi(u) = u^{\delta_0-1}\ell_0(1/u) \qquad(u \downarrow0),

where ℓ0\ell_0 is eventually positive and slowly varying at infinity. The comparisons below are made over a fixed class for which α∈[α‾,α‾]⊂(0,2)\alpha\in[\underline{\alpha},\overline{\alpha}] \subset(0,2), δ0∈[δ‾,δ‾]⊂(0,∞)\delta_0 \in[\underline{\delta},\overline{\delta}] \subset(0,\infty), ∣γ∣+∣β∣+∣χ∣|\gamma| + |\beta| + |\chi| is uniformly bounded, CπC_\pi is bounded away from zero and infinity, and the tail-remainder bounds and origin regular variation above hold uniformly.

The parameter α\alpha determines the polynomial upper-tail order, whereas γ\gamma records the leading logarithmic modification and β\beta and χ\chi describe finer slowly varying corrections. At the opposite end of the scale, δ0\delta_0 determines the regular-variation index at the origin, while ℓ0\ell_0 allows slowly varying modifications there. The upper-tail specification therefore retains distinctions that disappear if the slowly varying component is replaced by an asymptotically constant factor, whereas the origin condition rules out more singular behaviour that could generate competing large-global-scale regimes; the higher-order comparisons below are governed by the upper-tail parameters. The familiar horseshoe prior provides a basic example: its local-variance density is π(u)={πu1/2(1+u)}−1\pi(u) = \{\pi u^{1/2}(1+u)\}^{-1}, and hence α=1/2\alpha= 1/2, γ=0\gamma= 0 and δ0=1/2\delta_0 = 1/2. The horseshoe+ has the same polynomial indices at both ends but an additional logarithmic factor in its upper tail, corresponding to α=1/2\alpha= 1/2, γ=1\gamma= 1 and δ0=1/2\delta_0 = 1/2 at leading order [12, 7]. Thus the two priors are indistinguishable by polynomial tail order alone but are separated by the logarithmic upper-tail structure retained in Assumption 1.

To characterize how tail shape interacts with global shrinkage, let ϵ=s/p\epsilon= s/p and consider the scale family

τ=cϵrLb,(4)\tau= c\epsilon^r L^b, \tag*{(4)}

where c>0c > 0, r>0r > 0 and b∈Rb \in\mathbb{R} are fixed as p→∞p \to\infty. The exponent rr controls the polynomial calibration to sparsity, bb gives its logarithmic refinement and cc determines the remaining constant scale. This parameterization is used to characterize the calibration phase diagram in § 3.1; the oracle analysis in § 3.2 subsequently removes this restriction and minimizes the worst-case risk over all τ>0\tau> 0.

Orders of optimality and risk comparison

The leading benchmark in (1) distinguishes procedures only at the dominant sparse-minimax order. We call a prior–scale specification (π,τ)(\pi,\tau) first-order minimax if

Rp,s(π,τ)=2sL{1+o(1)}.R_{p,s}(\pi,\tau) = 2sL\{1+o(1)\}.

The classical minimax risk admits the sharper expansion

References

  1. [1]Abramovich, F., Y. Benjamini, D. L. Donoho, and I. M. Johnstone (2006). Adapting to unknown sparsity by controlling the false discovery rate. Ann. Statist. 34(2), 584–653.arxiv.org/abs/math/0505374
  2. [2]Armagan, A., D. B. Dunson, and M. A. Clyde (2011). Generalized beta mixtures of Gaussians. In Advances in Neural Information Processing Systems, Volume 24, pp. 523–531.arxiv.org/abs/1107.4976
  3. [4]Bai, R. and M. Ghosh (2021). On the beta prime prior for scale parameters in high-dimensional Bayesian regression models. Statist. Sinica 31(2), 843–865.
  4. [5]Bhadra, A., J. Datta, Y. Li, N. G. Polson, and B. Willard (2019). Prediction risk for the horseshoe regression. J. Mach. Learn. Res. 20(78), 1–39.
  5. [6]Bhadra, A., J. Datta, N. G. Polson, and B. Willard (2016). Default Bayesian analysis with global-local shrinkage priors. Biometrika 103(4), 955–969.arxiv.org/abs/1510.03516
  6. [7]Bhadra, A., J. Datta, N. G. Polson, and B. Willard (2017). The horseshoe+ estimator of ultra-sparse signals. Bayesian Anal. 12(4), 1105–1131.arxiv.org/abs/1502.00560
  7. [8]Bhattacharya, A., D. Pati, N. S. Pillai, and D. B. Dunson (2015). Dirichlet–laplace priors for optimal shrinkage. J. Am. Statist. Assoc. 110(512), 1479–1490.arxiv.org/abs/1401.5398
  8. [9]Bickel, P. J., Y. Ritov, and A. B. Tsybakov (2009). Simultaneous analysis of Lasso and Dantzig selector. Ann. Statist. 37(4), 1705–1732.DOI
  9. [10]Bogdan, M., A. Chakrabarti, F. Frommlet, and J. K. Ghosh (2011). Asymptotic Bayes-optimality under sparsity of some multiple testing procedures. Ann. Statist. 39(3), 1551–1579.DOI
  10. [11]Candès, E. J. and T. Tao (2005). Decoding by linear programming. IEEE Trans. Inform. Theory 51(12), 4203–4215.DOI
  11. [12]Carvalho, C. M., N. G. Polson, and J. G. Scott (2010). The horseshoe estimator for sparse signals. Biometrika 97(2), 465–480.
  12. [13]Castillo, I., J. Schmidt-Hieber, and A. van der Vaart (2015). Bayesian linear regression with sparse priors. Ann. Statist. 43(5), 1986–2018.DOI
  13. [14]Castillo, I. and A. W. van der Vaart (2012). Needles and straw in a haystack: Posterior concentration for possibly sparse sequences. Ann. Statist. 40(4), 2069–2101.DOI
  14. [15]Datta, J. and J. K. Ghosh (2013). Asymptotic properties of Bayes risk for the horseshoe prior. Bayesian Anal. 8(1), 111–132.DOI
  15. [16]Donoho, D. L. and I. M. Johnstone (1994). Minimax risk over ℓp-balls for ℓq-error. Probab. Theory Relat. Fields 99(2), 277–303.
  16. [17]Donoho, D. L., I. M. Johnstone, J. C. Hoch, and A. S. Stern (1992). Maximum entropy and the nearly black object. J. R. Statist. Soc. B 54(1), 41–67.DOI
  17. [18]Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Anal. 1(3), 515–534.
  18. [19]Ghosh, P. and A. Chakrabarti (2017). Asymptotic optimality of one-group shrinkage priors in sparse high-dimensional problems. Bayesian Anal. 12(4), 1133–1161.
  19. [20]Ghosh, P., X. Tang, M. Ghosh, and A. Chakrabarti (2016). Asymptotic properties of Bayes risk of a general class of shrinkage priors in multiple hypothesis testing under sparsity. Bayesian Anal. 11(3), 753–796.arxiv.org/abs/1310.7462
  20. [21]Griffin, J. E. and P. J. Brown (2010). Inference with normal-gamma prior distributions in regression problems. Bayesian Anal. 5(1), 171–188.DOI
  21. [22]Hamura, Y., K. Irie, and S. Sugasawa (2026). Shrinkage with robustness: Log-adjusted priors for sparse signals. Ann. Inst. Statist. Math.. Accepted for publication.arxiv.org/abs/2001.08465
  22. [23]Jiang, W. and C.-H. Zhang (2009). General maximum likelihood empirical Bayes estimation of normal means. Ann. Statist. 37(4), 1647–1684.DOI
  23. [24]Johnstone, I. M. (1994). On minimax estimation of a sparse normal mean vector. Ann. Statist. 22(1), 271–289.DOI
  24. [25]Johnstone, I. M. and B. W. Silverman (2004). Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32(4), 1594–1649.
  25. [26]Lee, S. Y., P. Zhao, D. Pati, and B. K. Mallick (2024). Tail-adaptive Bayesian shrinkage. Electron. J. Statist. 18(2), 4667–4723.arxiv.org/abs/2007.02192
  26. [27]Paul, S. and A. Chakrabarti (2025). Posterior contraction rate and asymptotic Bayes optimality for one group global–local shrinkage priors in sparse normal means problem. Ann. Inst. Statist. Math. 77(5), 787–819.DOI
  27. [28]Paul, S., P. Ghosh, and A. Chakrabarti (2025). Sharp asymptotic minimaxity for multiple testing using one-group shrinkage priors. arXiv:2505.16428.arxiv.org/abs/2505.16428
  28. [29]Piironen, J. and A. Vehtari (2017). Sparsity information and regularization in the horseshoe and other shrinkage priors. Electron. J. Statist. 11(2), 5018–5051.DOI
  29. [30]Polson, N. G. and J. G. Scott (2011). Shrink globally, act locally: Sparse Bayesian regularization and prediction. In J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West (Eds.), Bayesian Statistics 9, pp. 501–538. Oxford: Oxford University Press.DOI
  30. [31]Qin, Z. and M. Ghosh (2024). Global-local shrinkage priors for asymptotic point and interval estimation of normal means under sparsity. Sankhya A 86(1), 93–137.DOI
  31. [32]Raskutti, G., M. J. Wainwright, and B. Yu (2010). Restricted eigenvalue properties for correlated Gaussian designs. J. Mach. Learn. Res. 11, 2241–2259.DOI
  32. [33]Raskutti, G., M. J. Wainwright, and B. Yu (2011). Minimax rates of estimation for high-dimensional linear regression over ℓ_q-balls. IEEE Trans. Inform. Theory 57(10), 6976–6994.arxiv.org/abs/0910.2042
  33. [34]Rocková, V. (2018). Bayesian estimation of sparse signals with a continuous spike-and-slab prior. Ann. Statist. 46(1), 401–437.DOI
  34. [35]Song, Q. (2020). Bayesian shrinkage towards sharp minimaxity. Electron. J. Statist. 14(2), 2714–2741.arxiv.org/abs/2004.05307
  35. [36]Song, Q. and F. Liang (2023). Nearly optimal Bayesian shrinkage for high-dimensional regression. Sci. China Math. 66(2), 409–442.arxiv.org/abs/1712.08964
  36. [38]van der Pas, S., B. Szabó, and A. van der Vaart (2017). Adaptive posterior contraction rates for the horseshoe. Electron. J. Statist. 11(2), 3196–3225.arxiv.org/abs/1702.03698
  37. [39]van der Pas, S. L., B. J. K. Kleijn, and A. W. van der Vaart (2014). The horseshoe estimator: Posterior concentration around nearly black vectors. Electron. J. Statist. 8(2), 2585–2618.DOI
  38. [40]van der Pas, S. L., J.-B. Salomond, and J. Schmidt-Hieber (2016). Conditions for posterior contraction in the sparse normal means problem. Electron. J. Statist. 10(1), 976–1000.DOI
  39. [41]Vershynin, R. (2018). High-Dimensional Probability: An Introduction with Applications in Data Sci- ence. Cambridge University Press.
  40. [42]Zhang, C.-H. (2005). General empirical Bayes wavelet methods and exactly adaptive minimax estima- tion. Ann. Statist. 33(1), 54–100.

Paper details

vlnc:W_nknvbfmt
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC0-1.0
Usage

views · 5

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents