betaFeedback

WorksMathematics

A uniform improvement to conformal risk control via a generalization of e-closure

Abstract

We extend the recently introduced e-closure principle for multiple testing to general state and action spaces and to arbitrary, including continuously indexed, families of risks. A collection of e-values indexed by states and risks certifies a menu of actions, and every data-dependent selection from this menu controls each expected loss at its prescribed level. Conversely, every valid procedure is certified by its own normalized losses, which often leave part of the risk budget unused. We introduce a method called recertification, which spends this unused budget. Using known distributional structure, it enlarges the certifying e-values while preserving validity and selects a less conservative action from the enlarged menu. One particularly compelling application is to the popular framework of conformal risk control (CRC) with a known loss family. Here, recertification redistributes unused risk budget among the held-out points of each hypothetical augmented sample. The resulting controller is valid under every finite exchangeable law and less (usually strictly) conservative than CRC. For convex losses on an interval of actions, repeated recertification converges geometrically to an admissible controller. As a second example involving false discovery rate control with e-values, recertifying randomized e-BH yields a closed-form procedure that rejects a superset of randomized e-BH for every realization of the randomization and maximizes the expected number of rejections under the same constraints. Recertifying with the larger e-collection of closed e-BH gives a second closed-form procedure that rejects a superset of the first. Experiments on CIFAR-100 hierarchical classification and simulations demonstrate improved budget utilization and gains in informativeness and power over the corresponding baselines.

Introduction

Many statistical procedures turn data into an informed action, while guaranteeing that some expected loss stays below a prescribed level. Statistical decision theory formalizes this task through a decision rule mapping data to actions, a loss depending on the action and an unknown state of nature, and the resulting statewise expected loss, or risk [47, 9]. The bound imposes a risk budget that must hold in every state. Among rules satisfying this constraint, one seeks the most useful decisions, as in the constrained problems of Hodges and Lehmann [ [23].

Multiple testing is a decision problem of exactly this kind. With KK hypotheses, the action is a rejection set R⊆[K]R \subseteq[K], so the action space is the collection 2[K]2^{[K]} of all subsets of [K][K], and the unknown state is the set T⊆[K]T \subseteq[K] of true null hypotheses. The false discovery proportion ∣T∩R∣/(∣R∣∨1)\lvert T \cap R\rvert/(\lvert R\rvert\vee1) is the loss, and false discovery rate (FDR) control bounds its expectation by α\alpha. The procedure of Benjamini and Hochberg [ [8] achieves this by comparing the ordered p-values with linearly increasing thresholds; its e-value analogue, the e-BH procedure of Wang and Ramdas [ [48], applies the reciprocal thresholds to e-values, which are nonnegative statistics with expectation at most one under their null hypotheses [45, 38]. The recent work of Xu et al. [2026b] introduces the e-closure principle, a general principle that uniformly improves many testing procedures based on e-values, including e-BH. Specifically, given an e-value ETE_T for each candidate true-null set TT, valid when TT is the set of true nulls, the principle certifies every rejection set RR that satisfies

∣T∩R∣∣R∣∨1≤αETfor every T⊆[K].(1)\frac{|T \cap R|}{|R| \vee1} \le\alpha E_T \qquad\text{for every } T \subseteq[K]. \tag*{(1)}

Under the actual null set, the left side is the false discovery proportion and the right side has expectation at most α\alpha. The false discovery rate is therefore controlled even when the analyst chooses among the certified rejection sets after seeing the data. Conversely, Xu et al. [2026b] show that a procedure controls the false discovery rate simultaneously over the rejection sets it may report if and only if all of them are certified in this way by some family of e-values. The principle applies equally to other error rates that are expected losses, and it contains the classical closed testing principle for the familywise error rate [36] as the case of e-values taking only the values 0 and 1/α1/\alpha.

Our first contribution is to note that nothing in this argument depends on the actions being rejection sets, so e-closure extends well beyond multiple testing. In a general decision problem, a state ss specifies a class of distributions for the data XX, an action aa is chosen from an arbitrary set, and each of several losses Lt(s,a;X)L_t(s,a;X), indexed by tt, carries its own budget αt(s)\alpha_t(s). A family of e-values, one for each state and risk, certifies the actions whose losses fit within all the budgets it allows. Every data-dependent choice among the certified actions then controls every risk, and every procedure with this guarantee is certified by some such family [21, 22, 1]. In § 2 we develop the e-closure principle at this level of generality, with menus of arbitrary actions, possibly continuously indexed states and risks, and losses that may depend on the data.

The correspondence can be read in both directions, and the direction from procedures to e-values suggests how to improve a procedure. Improving a decision rule directly is difficult: a more liberal rule may have higher power or utility, but it is rarely clear how far a given rule can be pushed before its guarantee fails. By contrast, improving an e-value is easier. If an e-value has expectation strictly below one, and the shortfall can be computed, for instance conditionally on a statistic whose conditional law is known under the null, then adding the shortfall back yields a larger e-value. Through the e-closure correspondence, larger e-values translate back into decisions. Starting from a valid procedure, we apply this construction to its canonical e-values: its losses divided by their risk budgets. Inverting the enlarged certificate yields a menu containing the original actions and potentially more liberal ones, all with the same guarantee. We call this step recertification. It generalizes the anchoring idea of Ramdas [ [37], which keeps an existing false discovery rate procedure and spends the slack in its validity proof, and it can be repeated.

We apply recertification to two problems. The first is conformal risk control (CRC) [3], which extends conformal prediction from coverage to general monotone losses. Given nn exchangeable calibration points and a known family of losses indexed by a conservativeness parameter λ\lambda, CRC selects the least conservative λ\lambda for which the calibration losses, together with a bound on the loss of the unseen test point, fit within the budget (n+1)α(n+1)\alpha. The selected controller typically leaves part of this budget unused. Recertification computes this unused budget on every such sample, redistributes it among the n+1n+1 held-out positions, and inverts the resulting e-values. The new controller is valid under every finite exchangeable law, and for every calibration sample it is no more conservative than CRC. In this sense, for a fixed loss family and observation domain, it improves CRC uniformly. This recertification step can be repeated, with diminishing returns. For convex losses, the iterates converge geometrically to a controller that admits no further pointwise improvement, and on finite outcome spaces a coordinatewise sweep reaches such a controller without convexity (§ 3).

Figure 1 illustrates these gains in hierarchical classification on CIFAR-100 (§ 3.6), using rules that give more specific predictions when the classifier is more confident. At a target of 15%, CRC attains a mean test risk of 14.21%. Repeated recertification raises it to 14.95%, close to the target, and reduces the mean prediction-set size from 10.77 to 8.92 labels, a reduction of 17.2%.

Figure 1. Risk-budget use and mean prediction-set size on CIFAR-100 with confidence-adaptive prediction rules (n=50n = 50, α=0.15\alpha= 0.15). Round 0 is CRC, and later rounds apply recertification. Results are averaged over 300 paired splits, each with 16 test images.

The second application returns to multiple testing. The e-BH procedure is valid under arbitrary dependence. dence but can be conservative, and Xu and Ramdas [ [49] showed that external randomization recovers part of the lost power. Their U-eBH procedure boosts the e-values by dividing them by a single uniform random variable and then applies e-BH. In this setting, we treat the conditional law of the rejection set given the e-values as a randomized action. In these terms, U-eBH is certified by the family of e-values behind e-BH, through a version of the self-consistency condition of Blanchard and Roquain [ [10] that holds on average over the randomization, and it typically leaves part of the budget allowed by this family unused. Recertification spends this budget and yields a closed-form randomized rule, recertified U-eBH (RU-eBH), which rejects at least as many hypotheses as U-eBH for every realization of the randomization and has the largest expected number of rejections among all randomized procedures certified by the same family.

The same principle applies to closed e-BH, the improvement of e-BH by e-closure [Xu et al., 2026b], whose family of e-values is larger because it adapts to the number of true nulls. Recertifying with this family gives a second closed-form rule, recertified closed e-BH (RC-eBH), whose rejection set contains that of RU-eBH and which has the largest expected number of rejections among all randomized procedures certified by this family, closed e-BH included (§ 4). In simulations, both rules are more powerful than e-BH and U-eBH in every setting, and RC-eBH is the most powerful of the eight procedures we compare in nearly every setting.

Related work

E-values and decisions. E-values have become a standard tool for testing and multiple testing; they remain valid when the significance level is chosen after seeing the data Ramdas and Wang, [38], Chapter 4. This property was identified by Wang and Ramdas [ [48], Katsevich and Ramdas [ [26] and Xu et al. [2024] in multiple testing, and in particular when testing a single hypothesis. Grünwald [ [22] developed this into a generalization of the Neyman–Pearson framework to data-dependent levels in an expected-loss setting. Koning [ [28] then showed that e-values are necessary for post hoc testing and introduced the post hoc p-value, Chugg et al. [ [12] studied admissibility in this framework, and Koning [ [29] derived its validity criterion from axioms. For a single risk, our certified menu is the set of actions whose e-posterior risk bound Grünwald, [21], with each state’s loss divided by its budget, is at most one. Andrews and Chen [ [1] study decisions reported together with certificates of their loss, and Koning and van Meer [ [32] give evidence-weighted loss bounds for decisions based on conformal e-prediction sets. Decision-theoretic views of conformal prediction also study how prediction sets should inform downstream actions Leukauf et al., (2024); Kiyani et al. [27]. These works allow general states and actions; our focus is the constructive use of the correspondence, which keeps the error budget fixed and recertifies an existing procedure.

Closure principles. Closed testing [36] controls the familywise error rate, and every coherent procedure for that error rate is a closed testing procedure [40]. Closed testing also yields simultaneous bounds on the false discovery proportion of every rejection set [18], and only closed testing procedures are admissible for that purpose [19]. E-closure [Xu et al., 2026b] extends the closure principle to error rates defined by expected losses. Its follow-ups include the uniform improvement of the Benjamini–Hochberg procedure [17], anchoring [37], online versions [Xu et al., 2026a], complete-class and admissibility results for e-value procedures [42], and a continuous closure for allocating resources among strategic agents through statistical contracts [11]. Koning [2026b] closes evidence over classes of hypotheses through e-measures, which yield evidence bounds holding uniformly over data-informed decisions and connect to e-closure in multiple testing. We note that representation and admissibility are separate questions. Every valid procedure is certified by some family of e-values, while admissibility requires further conditions, and our admissibility results for CRC rely on the structure of that problem.

Randomized multiple testing. Randomization is classical in testing, where it makes tests based on discrete statistics exact [34]. For false discovery rate control with e-values under arbitrary dependence, Xu and Ramdas [ [49] showed that stochastic rounding of the e-values and U-eBH increase the power of e-BH, and Xu et al. [2026b] combine stochastic rounding with e-closure. These methods randomize the e-values. Our procedures instead choose the distribution of the rejection set directly, subject either to per-hypothesis budgets that form a randomized version of the self-consistency condition of Blanchard and Roquain [ [10] or to the constraints that certify closed e-BH. Other uniform improvements of e-BH include its minimally adaptive version [24], closed e-BH [Xu et al., 2026b], and conditional calibration [15, 33], which recalibrates the e-values given a sufficient statistic. All of these improvements fit our framework. The first two supply larger certifying e-collections, and recertifying with them using randomization improves upon both (§ 4.4 and Appendix § 8.§ 5); conditional calibration is itself a form of recertification (Remark 4.3).

Conformal risk control. CRC already handles several, even infinitely many, risks that share a monotone action parameter [3 Section 4.3]. Risk-Controlling Prediction Sets (RCPS) and Learn then Test (LTT) instead provide high-probability statements about population risks [7, 4]. Extensions of CRC treat non-exchangeable data [14], cross-validation [13], non-monotonic losses [2], and end-to-end training [53]. The leave-one-out characterization of validity that we use (Lemma 3.1) is the risk-control counterpart of the universality of conformal prediction [46 Proposition 2.9]; see also [43], and Angelopoulos [ [2] for its sufficiency direction. Conformal prediction with e-values is developed by Vovk [ [44], Koning [ [31], Gauthier et al. [ [16], and Koning and van Meer [ [32]. Apart from the sharper bound on the unseen test loss in envelope CRC [53], which serves as our starting point, we are not aware of work on improving CRC controllers pointwise or on their admissibility. We use CRC to show concretely how large such improvements can be and what they cost in computation.

Organization

Section 2 formulates e-closure for general decision problems, including randomized actions, and introduces recertification. Section 3 develops the improvement of CRC, including its admissibility theory, worked examples and a real-data study. Section 4 returns to multiple testing, recertifies randomized e-BH with the e-collections of e-BH and closed e-BH, and reports simulations on Gaussian mean testing. Section 5 comments on the choice among certified actions and concludes.

Generalized e-closure and recertification

This section sets up the decision framework used throughout the paper. We first describe states, actions and losses, then establish the correspondence between e-values and valid decisions, and finally introduce recertification, which turns this correspondence into a way of improving existing procedures.

States, actions, and a family of risks

Suppose that a statistician observes data XX taking values in a space X\mathcal{X}. The distribution of XX is governed by an unknown state s∈Ss \in\mathcal{S}, and each state specifies a nonempty class Ps\mathcal{P}_s of distributions of XX. After observing XX, the statistician takes an action a∈Aa \in\mathcal{A}, whose quality is measured by a family of known nonnegative losses

Lt(s,a;x),t∈T.L_t(s,a;x), \qquad t \in\mathcal{T}.

The index tt ranges over the kinds of error that the statistician wishes to control, such as several error rates or a continuum of loss functions, and each loss may depend on the observation xx as well as on the state and the action. The sets S\mathcal{S}, A\mathcal{A} and T\mathcal{T} are arbitrary and may be uncountable.

Each loss comes with a budget. Given fixed positive levels αt(s)\alpha_t(s), a decision rule A:X→AA:\mathcal{X} \to\mathcal{A} is valid if

EP[Lt(s,A(X);X)]≤αt(s)for every s∈S, P∈Ps, t∈T.(2)\mathbb{E}_P[L_t(s,A(X);X)] \leq\alpha_t(s) \quad\text{for every } s \in\mathcal{S},\ P \in\mathcal{P}_s,\ t \in\mathcal{T}. \tag*{(2)}

In words, whatever the state and whichever distribution within it generates the data, the expected loss of the selected action stays within its budget, separately for every loss in the family. The expectation is unconditional: it averages over the data and hence over the action that the rule selects. Throughout, we assume that all decision rules and all losses that we evaluate are measurable, and when we discuss sets of actions we also assume that their statewise loss envelopes, defined below, are measurable. Appendix § 6 gives sufficient conditions and discusses the uncountable intersections that arise.

Validity is certified by e-values. Recall that an e-value for a class of distributions is a nonnegative statistic whose expectation is at most one under every distribution in the class. Since (2) involves every state and every loss, we use one e-value for each such pair. An ee-collection is a family of nonnegative measurable functions Es,tE_{s,t} satisfying

EPEs,t(X)≤1(s∈S, P∈Ps, t∈T).(3)\mathbb{E}_P E_{s,t}(X) \leq1 \quad(s \in\mathcal{S},\ P \in\mathcal{P}_s,\ t \in\mathcal{T}). \tag*{(3)}

In other words, an e-collection is a set of e-values, one for each combination of state and loss. Because an e-collection will be used to certify actions, we also refer to it as a certificate.

From e-collections to decisions and back

We now make precise the correspondence between e-collections and valid decisions described in the introduction. It is convenient to work with sets of actions rather than with single rules. A menu assigns to each observation xx a nonempty set M(x)⊆A\mathcal{M}(x) \subseteq\mathcal{A} of actions that the statistician is prepared to take. After seeing the data, the statistician may choose an action from M(x)\mathcal{M}(x), for instance by maximizing a utility or following a scientific preference. A decision rule is the special case of a menu with a single action at each observation. We first show how an e-collection produces a menu, then how a menu produces an e-collection, and finally that the two constructions are compatible.

From e-collections to menus. An e-collection certifies an action at the observation xx when every loss of that action fits within its budget, scaled by the corresponding e-value. The certified menu of EE is therefore

Cα(E;x)={a∈A:Lt(s,a;x)≤αt(s)Es,t(x) ∀ s,t}.(4)\mathcal{C}_\alpha(E;x) = \{a \in\mathcal{A}: L_t(s,a;x) \leq\alpha_t(s)E_{s,t}(x)\ \forall\,s,t\}. \tag*{(4)}

The constraint for state ss caps the loss an action may incur if ss is true, in proportion to Es,t(x)E_{s,t}(x). Intuitively, a large e-value indicates that the data are unlikely under ss, so the menu may include actions that would perform poorly under ss; a small one restricts the menu to actions that perform well under ss.

From menus to e-collections. Conversely, a menu determines the largest loss it can incur in each state. For a menu M\mathcal{M}, define its loss envelopes and their normalized versions by

gs,tM(x)=sup⁡a∈M(x)Lt(s,a;x),Es,tM(x)=gs,tM(x)αt(s).(5)g^{\mathcal{M}}_{s,t}(x)=\sup_{a\in\mathcal{M}(x)} L_t(s,a;x),\qquad E^{\mathcal{M}}_{s,t}(x)=\frac{g^{\mathcal{M}}_{s,t}(x)}{\alpha_t(s)}. \tag*{(5)}

The menu is valid if EPgs,tM(X)≤αt(s)\mathbb{E}_{P}g^{\mathcal{M}}_{s,t}(X)\leq\alpha_t(s) for every s∈Ss\in\mathcal{S}, P∈PsP\in\mathcal{P}_s and t∈Tt\in\mathcal{T}. This requires (2) to hold for every selection from the menu simultaneously, and it is equivalent to EME^{\mathcal{M}} being an e-collection, which we call the canonical e-collection of M\mathcal{M}. Applying the first construction to it gives the closure of M\mathcal{M},

M‾(x)=Cα(EM;x)={a:Lt(s,a;x)≤gs,tM(x) ∀s,t},(6)\overline{\mathcal{M}}(x)=C_\alpha(E^{\mathcal{M}};x)=\{a:L_t(s,a;x)\leq g^{\mathcal{M}}_{s,t}(x)\ \forall s,t\}, \tag*{(6)}

which consists of all actions whose losses never exceed those already incurred by the menu.

The following theorem, which we call generalized e-closure, states that the two constructions describe the same objects: certified menus are valid, and every valid menu is certified by its canonical e-collection.

Theorem 2.1. Let EE be an e-collection. Every measurable selection A(X)∈Cα(E;X)A(X)\in C_\alpha(E;X) satisfies (2). More strongly, for every state–loss pair (s,t)(s,t) and every P∈PsP\in\mathcal{P}_s,

EP[sup⁡a∈Cα(E;X)Lt(s,a;X)]≤αt(s).(7)\mathbb{E}_{P}\left[\sup_{a\in C_\alpha(E;X)}L_t(s,a;X)\right]\leq\alpha_t(s). \tag*{(7)}

Further, a menu M\mathcal{M} is valid if and only if M⊆Cα(E)\mathcal{M}\subseteq C_\alpha(E) for some e-collection EE. The loss closure (2.5) is extensive, monotone under inclusion of menus and idempotent, and it preserves every loss envelope. In particular,

M=Cα(E)⟹M=Cα(EM).(8)\mathcal{M}=C_\alpha(E)\quad\Longrightarrow\quad\mathcal{M}=C_\alpha(E^{\mathcal{M}}). \tag*{(8)}

Proof. For every a∈Cα(E;X)a\in C_\alpha(E;X), definition (4) gives Lt(s,a;X)≤αt(s)Es,t(X)L_t(s,a;X)\leq\alpha_t(s)E_{s,t}(X), so the supremum in (2.6) is at most $\alpha_t(s)E_{s,t}(X)$. Taking expectations under $P\in\mathcal{P}_s$ and using (2.2) proves {xref}(2.6)`, and the validity of every measurable selection follows. If M⊆Cα(E)\mathcal{M}\subseteq C_\alpha(E), then gs,tM≤αt(s)Es,tg^{\mathcal{M}}_{s,t}\leq\alpha_t(s)E_{s,t}, so M\mathcal{M} is valid. Conversely, if M\mathcal{M} is valid, then EME^{\mathcal{M}} is an e-collection, and every action in M(x)\mathcal{M}(x) satisfies the constraints defining Cα(EM;x)C_\alpha(E^{\mathcal{M}};x), so M⊆M‾\mathcal{M}\subseteq\overline{\mathcal{M}}. Every action in M‾(x)\overline{\mathcal{M}}(x) has losses at most gs,tM(x)g^{\mathcal{M}}_{s,t}(x), so M‾\overline{\mathcal{M}} has the same envelopes as M\mathcal{M}; idempotence follows, and monotonicity follows from the monotonicity of the envelopes. Finally, if M=Cα(E)\mathcal{M}=C_\alpha(E), then M⊆Cα(EM)\mathcal{M}\subseteq C_\alpha(E^{\mathcal{M}}), and gs,tM≤αt(s)Es,tg^{\mathcal{M}}_{s,t}\leq\alpha_t(s)E_{s,t} gives the reverse inclusion Cα(EM)⊆Cα(E)=MC_\alpha(E^{\mathcal{M}})\subseteq C_\alpha(E)=\mathcal{M}. □

By (7), the guarantee holds simultaneously over all actions in a certified menu. The statistician may therefore choose an action after seeing the data, for instance by maximizing a utility or applying a scientific filter, and every such choice from the menu of a fixed e-collection controls every risk. This is closely related to the post hoc validity of e-values in hypothesis testing, where the significance level, and more generally the loss, may be chosen after the data are observed [22, 28, 12]. We also note that the e-values may be arbitrarily dependent, because for each loss the proof uses only the corresponding e-value under the true state.

Multiple testing as a special case. The e-closure principle for multiple testing is recovered as follows. Take both the state space and the action space to be 2[K]2^{[K]}, let PT\mathcal{P}_T contain the distributions under which TT is the set of true null hypotheses, and use the loss L(T,R)=∣T∩R∣/(∣R∣∨1)L(T,R)=|T\cap R|/(|R|\vee1). The certified menu (4) then consists of the rejection sets certified by (1), which is the e-closure principle for the false discovery rate [Xu et al., 2026b], and the loss L(T,R)=1{T∩R≠∅}L(T,R)=\mathbf{1}\{T\cap R\neq\varnothing\} gives its familywise error rate version. E-values for intersection hypotheses, which are valid whenever TT is contained in the set of true nulls, are in particular valid for the exact-state classes PT\mathcal{P}_T used here. § 4 reviews this setting in more detail. The general framework requires no structure on the actions, such as unions or an ordering, and no monotonicity of the losses. For instance, the false discovery proportion can decrease when a rejection is added.

References

  1. [1]I. Andrews and J. Chen. Certified decisions. arXiv preprint arXiv:2502.17830, 2025.arxiv.org/abs/2502.17830
  2. [2]A. N. Angelopoulos. Conformal risk control for non-monotonic losses. arXiv preprint arXiv:2602.20151, 2026.arxiv.org/abs/2602.20151
  3. [3]A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster. Conformal risk control. In International Conference on Learning Representations, 2024.
  4. [4]A. N. Angelopoulos, S. Bates, E. J. Candès, M. I. Jordan, and L. Lei. Learn then test: Calibrating predictive algorithms to achieve risk control. The Annals of Applied Statistics, 19(2):1641–1662, 2025.arxiv.org/abs/2110.01052
  5. [5]T. Bai and Y. Jin. Optimized conformal selection: Powerful selective inference after conformity score optimization. arXiv preprint arXiv:2411.17983, 2024.arxiv.org/abs/2411.17983
  6. [6]T. Bai and Y. Jin. Conformal selective prediction with general risk control. arXiv preprint arXiv:2603.24704, 2026.arxiv.org/abs/2603.24704
  7. [7]S. Bates, A. N. Angelopoulos, L. Lei, J. Malik, and M. I. Jordan. Distribution-free, risk-controlling prediction sets. Journal of the ACM, 68(6):Article 43, 2021.arxiv.org/abs/2101.02703
  8. [8]Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57(1):289–300, 1995.DOI
  9. [9]J. O. Berger. Statistical Decision Theory and Bayesian Analysis. Springer, second edition, 1985.DOI
  10. [10]G. Blanchard and E. Roquain. Two simple sufficient conditions for FDR control. Electronic Journal of Statistics, 2:963–992, 2008.
  11. [11]B. Chugg, M. I. Jordan, and A. Ramdas. Strategic multi-agent resource allocation via statistical contracts. arXiv preprint arXiv:2610.18336, 2026a.arxiv.org/abs/2610.18336
  12. [12]B. Chugg, T. Lardy, A. Ramdas, and P. D. Grünwald. On admissibility in post-hoc hypothesis testing. International Journal of Approximate Reasoning, 191:109634, 2026b.arxiv.org/abs/2508.00770
  13. [13]K. M. Cohen, S. Park, O. Simeone, and S. Shamai. Cross-validation conformal risk control. In IEEE International Symposium on Information Theory, 2024.arxiv.org/abs/2401.11974
  14. [14]A. Farinhas, C. Zerva, D. Ulmer, and A. F. T. Martins. Non-exchangeable conformal risk control. In International Conference on Learning Representations, 2024.arxiv.org/abs/2310.01262
  15. [15]W. Fithian and L. Lei. Conditional calibration for false discovery rate control under dependence. The Annals of Statistics, 50(6):3091–3118, 2022.arxiv.org/abs/2007.10438
  16. [16]E. Gauthier, F. Bach, and M. I. Jordan. E-values expand the scope of conformal prediction. arXiv preprint arXiv:2503.13050, 2025.arxiv.org/abs/2503.13050
  17. [17]J. J. Goeman. A uniform improvement of the Benjamini–Hochberg procedure via e-closure. arXiv preprint arXiv:2606.01854, 2026.arxiv.org/abs/2606.01854
  18. [18]J. J. Goeman and A. Solari. Multiple testing for exploratory research. Statistical Science, 26(4):584–597, 2011.DOI
  19. [19]J. J. Goeman, J. Hemerik, and A. Solari. Only closed testing procedures are admissible for controlling false discovery proportions. The Annals of Statistics, 49(2):1218–1238, 2021.DOI
  20. [20]P. González-González. Embedded representations of popular image datasets (CIFAR10, CIFAR100, CIFAR100coarse, MNIST, FashionMNIST, SVHN). Zenodo dataset, version 1, https://doi.org/10.5281/zenodo.21131944, 2026.
  21. [21]P. D. Grünwald. The e-posterior. Philosophical Transactions of the Royal Society A, 381(2247):20220146, 2023.DOI
  22. [22]P. D. Grünwald. Beyond Neyman–Pearson: E-values enable hypothesis testing with a data-driven alpha. Proceedings of the National Academy of Sciences, 121(39):e2302098121, 2024.DOI
  23. [23]J. L. Hodges, Jr. and E. L. Lehmann. The use of previous experience in reaching statistical decisions. The Annals of Mathematical Statistics, 23(3):396–407, 1952.
  24. [24]N. Ignatiadis, R. Wang, and A. Ramdas. E-values as unnormalized weights in multiple testing. Biometrika, 111(2):417–439, 2024.arxiv.org/abs/2204.12447
  25. [25]Y. Jin and E. J. Candès. Model-free selective inference under covariate shift via weighted conformal p-values. arXiv preprint arXiv:2307.09291, 2023.arxiv.org/abs/2307.09291
  26. [26]E. Katsevich and A. Ramdas. Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings. The Annals of Statistics, 48(6):3465–3487, 2020.DOI
  27. [27]S. Kiyani, G. J. Pappas, A. Roth, and H. Hassani. Decision theoretic foundations for conformal prediction: Optimal uncertainty quantification for risk-averse agents. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 30943–30965, 2025.arxiv.org/abs/2502.02561
  28. [28]N. W. Koning. Post-hoc α hypothesis testing and the post-hoc p-value. arXiv preprint arXiv:2312.08040, 2025.arxiv.org/abs/2312.08040
  29. [29]N. W. Koning. Axioms for testing with data-dependent levels, e-values and p-values. arXiv preprint arXiv:2605.28429, 2026a.arxiv.org/abs/2605.28429
  30. [31]N. W. Koning. Measuring evidence against exchangeability and group invariance with e-values. The Annals of Statistics, 54(4):1843–1869, 2026c.arxiv.org/abs/2310.01153
  31. [32]N. W. Koning and S. van Meer. Fuzzy prediction sets: Conformal prediction with e-values. arXiv preprint arXiv:2509.13130, 2026.arxiv.org/abs/2509.13130
  32. [33]J. Lee and Z. Ren. Boosting e-BH via conditional calibration. arXiv preprint arXiv:2404.17562, 2024.arxiv.org/abs/2404.17562
  33. [34]E. L. Lehmann and J. P. Romano. Testing Statistical Hypotheses. Springer, fourth edition, 2022.
  34. [36]R. Marcus, E. Peritz, and K. R. Gabriel. On closed testing procedures with special reference to ordered analysis of variance. Biometrika, 63(3):655–660, 1976.DOI
  35. [37]A. Ramdas. Anchored multiple testing: A transparent use of e-closure to improve FDR procedures. arXiv preprint arXiv:2610.11279, 2026.arxiv.org/abs/2610.11279
  36. [38]A. Ramdas and R. Wang. Hypothesis testing with e-values. Foundations and Trends in Statistics, 1(1–2):1–390, 2025.DOI
  37. [39]A. Solari and J. J. Goeman. Minimally adaptive BH: A tiny but uniform improvement of the procedure of Benjamini and Hochberg. Biometrical Journal, 59(4):776–780, 2017.DOI
  38. [40]E. Sonnemann. General solutions to multiple testing problems. Biometrical Journal, 50(5):641–656, 2008.DOI
  39. [41]S. M. Srivastava. A Course on Borel Sets, volume 180 of Graduate Texts in Mathematics. Springer, 1998.
  40. [42]L. Sun and R. Wang. Admissibility and complete classes for false discovery rate control with e-values. arXiv preprint arXiv:2607.14380, 2026.arxiv.org/abs/2607.14380
  41. [43]R. J. Tibshirani, R. F. Barber, and A. Ramdas. Conformal prediction through the lens of hypothesis testing: Universality, impossibility, and optimality. arXiv preprint arXiv:2608.27310, 2026.arxiv.org/abs/2608.27310
  42. [44]V. Vovk. Conformal e-prediction. Pattern Recognition, 166:111674, 2025.DOI
  43. [45]V. Vovk and R. Wang. E-values: Calibration, combination and applications. The Annals of Statistics, 49 (3):1736–1754, 2021.DOI
  44. [46]V. Vovk, A. Gammerman, and G. Shafer. Algorithmic Learning in a Random World. Springer, second edition, 2022.
  45. [47]A. Wald. Statistical Decision Functions. John Wiley & Sons, 1950.
  46. [48]R. Wang and A. Ramdas. False discovery rate control with e-values. Journal of the Royal Statistical Society: Series B, 84(3):822–852, 2022.arxiv.org/abs/2009.02824
  47. [49]Z. Xu and A. Ramdas. More powerful multiple testing under dependence via randomization. arXiv preprint arXiv:2305.11126, 2023.arxiv.org/abs/2305.11126
  48. [53]C. Yeh, N. Christianson, A. Wierman, and Y. Yue. Conformal risk training: End-to-end optimization of conformal risk control. In Advances in Neural Information Processing Systems, 2025.arxiv.org/abs/2510.08748

Paper details

vlnc:W_upthgsg8
Version
v2 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

AI

View statement
Looking for related papers…
Contents