Introduction
Conformal p-values make a simple comparison: where does a test point’s score fall among the scores of a held-out calibration set? As long as the calibration and test data are exchangeable, these p-values are valid in finite samples [1 , 11 ] . But in many applications, the data aren’t exchangeable. The test units come from a covariate distribution Q Q Q that differs from the calibration distribution P P P , and the plain comparison loses its guarantee. Weighted conformal p-values [20 , 12 ] correct for this covariate shift by reweighting each calibration point by the likelihood ratio w ( x ) ∝ d Q / d P ( x ) w(x) \propto dQ/dP(x) w ( x ) ∝ d Q / d P ( x ) . They’re used for two tasks. In selection, we’re given m m m test units and want the ones whose unobserved outcome clears a threshold. Outlier detection is about flagging test units that don’t come from the calibration distribution. Either way, we form a p-value for each test unit, and we want to control the false discovery rate (FDR). The default tool is the Benjamini–Hochberg (BH) procedure [2 ] .
For unweighted conformal p-values, this is safe. The p-values are positively dependent, and BH controls the FDR [1 , 11 ] . Weights break that argument. Jin & Candès [12 ] note that weighted conformal p-values “cannot be passed on to classical multiple testing procedures since they may not obey a well-known positive dependence property”, and they propose weighted conformalized selection, which has exact finite-sample control. Does plain BH on weighted conformal p-values also control the FDR? They left that open. In their words, “it remains unclear whether applying existing multiple testing procedures controls the FDR in finite samples” (§1.2), even though BH “empirically controls the FDR in all of our numerical experiments” (§2.3). Recent work still treats the question as open (Section 1.1).
Contributions. This paper answers the question and measures the failure. Throughout, n n n is the number of calibration points, m m m the number of test units, q q q the target level and B B B the ratio of the largest to the smallest weight; κ : = B / n \kappa:= B/n κ := B / n .
BH does not control the FDR in finite samples (Theorem 5 ). We give two configurations that satisfy Jin & Candès’s assumptions and have FDR > q >q > q , certified in exact rational arithmetic: one in which every test unit is null, and one in strict selection, where the threshold is independent of the outcome given the covariate. In the second, with n = 80 n=80 n = 80 , m = 8 m=8 m = 8 and q = 1 / 10 q=1/10 q = 1/10 , the FDR is 1.136 q 1.136q 1.136 q .
With non-null units, the worst excess is of order B m / ( n q ) \sqrt{Bm/(nq)} B m / ( n q ) . In every configuration with B ≤ n B\le n B ≤ n and at every n n n , F D R / q − 1 ≤ κ + 1.612 ( 1 + κ ) B m / ( n q ) + O ( B m / ( n q ) ) \mathrm{FDR}/q-1\le\kappa+1.612\sqrt{(1+\kappa)Bm/(nq)}+O(Bm/(nq)) FDR / q − 1 ≤ κ + 1.612 ( 1 + κ ) B m / ( n q ) + O ( B m / ( n q )) (Theorem 6 ). Conversely, in a Poisson-limit model a construction with non-null “decoy” units that share a threshold has excess c ( q ) κ m / q c(q)\sqrt{\kappa m/q} c ( q ) κm / q to first order, as κ → 0 \kappa\to0 κ → 0 and then m → ∞ m\to\infty m → ∞ , with c ( q ) = ( 1 − q ) e − 1 − 2 q / ( 2 2 π ) c(q)=(1-q)e^{-1-2q}/(2\sqrt{2\pi}) c ( q ) = ( 1 − q ) e − 1 − 2 q / ( 2 2 π ) (Theorems 9 , 11 and 15 ). For some fixed distributions the excess decays only like n − 1 / 2 n^{-1/2} n − 1/2 (Theorem 12 and Example 13 ).
When every test unit is null, the excess is smaller. It is O ( B / n ) O(B/n) O ( B / n ) at fixed m m m (Theorem 17 ), and there are configurations where F D R − q \mathrm{FDR}-q FDR − q is, to first order, at least 1 2 q ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) B / n \frac{1}{2}q(1-q)(m-1)(1-q+q/m)B/n 2 1 q ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) B / n , linear in m m m ; as m → ∞ m\to\infty m → ∞ the constant per unit of q m B / n qmB/n q m B / n is at least 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) (Theorem 16 ). Whether the upper bound is also linear in m m m is supported numerically and by a sketch, but not proved.
Practical remedies (§ 7 ): a computable envelope for the FDR, a level deflation with proved finite-sample control that uses only n n n , m m m , q q q and B B B , and a numerical rule of thumb, about ten times less conservative, that covered every worst case we found.
How large is the excess? The certified strict-selection construction uses a weight ratio comparable to n n n (B ≈ 1.5 n B\approx1.5n B ≈ 1.5 n to 1.7 n 1.7n 1.7 n ). If we re-tune it for each n n n , it gives FDR = 1.069 q =1.069q = 1.069 q , 1.103 q 1.103q 1.103 q and 1.136 q 1.136q 1.136 q at n = 20 n=20 n = 20 , 40 40 40 and 80 80 80 (Remark 88). When B ≪ n B\ll n B ≪ n , the rates above govern the excess. Across the Poisson-limit configurations our searches found, the relative excess stayed below 0.17 B m / ( n q ) 0.17\sqrt{Bm/(nq)} 0.17 B m / ( n q ) whenever B / n ≤ 0.1 B/n\le0.1 B / n ≤ 0.1 . Among those configurations, the largest excess at any B / n B/n B / n was 51%, and it needed extreme weights (B B B about 10 n 10n 10 n ). But in the Poisson-limit model, the coefficient of B / n \sqrt{B/n} B / n in the excess grows like m \sqrt{m} m (Theorem 11 , as B / n → 0 B/n\to0 B / n → 0 ).
Mechanisms and techniques. Given the calibration data D D D , the test p-values are independent, so all the dependence that BH sees passes through D D D . Lemma 2 makes this exact. When every null is an oracle null (its threshold equals its outcome), F D R − q \mathrm{FDR}-q FDR − q is a weighted sum of covariances minus a non-negative term. Each covariance is between the null mass below a BH threshold and the probability that the other units drive BH to exactly that many rejections. The non-negative term comes from non-null units with small oracle p-values, and in our constructions it is zero. This covariance behaves very differently in two regimes (§ 3 ). If every test unit is an oracle null, the conditional law of the test p-value is uniform on average over D D D , so a fluctuation of the calibration data has no first-order effect. What’s left is a second-order effect of the curvature of Simes’ test, of order B / n B/n B / n . Things are different when non-null units share a threshold. Their rejection switches on and off with D D D , and the switch can be correlated with the null mass at first order, which gives an excess of order B / n \sqrt{B/n} B / n . For the upper bound we use a cruder reduction: the FDR is at most q q q times the expected largest ratio, over the BH thresholds, of the conditional null mass below a threshold to the threshold itself (Proposition 3 ). We then bound the worst relative error of a weighted empirical distribution function on the BH grid.
Some of our lower bounds are proved in a Poisson-limit model, in which a few heavily weighted covariate cells have Poisson calibration counts, and some statements rest on numerical evidence. We say so wherever this is the case.
Outline. § 2 sets up the notation and two reductions, and § 3 gives the counterexamples and the two mechanisms. § 4 proves, with non-null units, an upper bound of order B m / ( n q ) \sqrt{Bm/(nq)} B m / ( n q ) and, in the Poisson-limit model, a lower bound of the same order. § 6 treats the global null. § 7 collects what a practitioner can compute, and § 8 discusses open problems. All proofs are in the appendices; Appendix § 14 gives the certificates for the counterexamples.
Related work
Multiple testing under dependence. The BH procedure [2 ] controls the FDR for independent p-values and, as Jin & Candès [12 ] recall, “when applied to PRDS p-values” [3 ] . For all other forms of dependence, Benjamini & Yekutieli [3 ] show that a conservative modification of BH controls the FDR: BH at level q / ∑ k ≤ m 1 / k q/\sum_{k\le m}1/k q / ∑ k ≤ m 1/ k . Hommel [10 ] gives tests of the global (intersection) null hypothesis under arbitrary dependence, with sharp bounds on their error probability, and Sarkar [18 ] extends FDR control under positive dependence from the BH step-up procedure to more general stepwise procedures with the same critical values. Fithian & Lei [5 ] calibrate a data-dependent rejection threshold for each hypothesis to target exact FDR control when the dependence is known; their dBH procedure dominates BH under positive regression dependence and BY in general.
Conformal p-values and selection. For unweighted conformal p-values, Bates et al. [1 ] prove that the p-values are positively dependent and that BH controls the FDR exactly, in a marginal sense. Jin & Candès [11 ] use conformal p-values with BH-type thresholds for selection under exchangeability. Under covariate shift, weighted conformal p-values [20 , 12 ] “cannot be passed on to classical multiple testing procedures since they may not obey a well-known positive dependence property” [12 ] . This motivates weighted conformalized selection (WCS), which comes with exact finite-sample control. With e-values, Lee & Ren [15 ] show that e-BH on weighted conformal e-values controls the FDR, “but unlike before, this is no longer equivalent to applying BH to the weighted conformal p-values” (their §6.2). Lee et al. [16 ] develop full-conformal e-values with finite-sample control.
The question remains open in recent work. Lee et al. [16 ] compare with BH on weighted conformal p-values, which “is not necessarily FDR-controlling”. Hennhöfer & Preisach (2026, §5.4) write that applying BH directly to discrete weighted conformal p-values “is not theoretically justified in general”, and Hennhöfer et al. (2026, §2.8) note that “exact finite-sample FDR control generally requires procedures tailored to the weighted setting”. None of these works proves or disproves finite-sample control, or bounds the excess. Neither does any other work we found.
Setting and two reductions
Data and p-values. We observe calibration data ( X i , Y i ) (X_i,Y_i) ( X i , Y i ) , i = 1 , … , n i=1,\ldots,n i = 1 , … , n , drawn i.i.d. from P P P , and m m m test units ( X n + j , Y n + j ) (X_{n+j},Y_{n+j}) ( X n + j , Y n + j ) drawn from Q Q Q , where d Q / d P = w ( X ) / E P w \mathrm{d}Q/\mathrm{d}P=w(X)/\mathbb{E}_P w d Q / d P = w ( X ) / E P w for a known weight function with 1 ≤ w ≤ B 1\le w\le B 1 ≤ w ≤ B . We write B : = sup w / E P w ≤ B B:=\sup w/\mathbb{E}_P w\le B B := sup w / E P w ≤ B and κ : = B / n \kappa:=B/n κ := B / n . Test unit j j j has a threshold c n + j c_{n+j} c n + j and is null iff Y n + j ≤ c n + j Y_{n+j}\le c_{n+j} Y n + j ≤ c n + j . With a score V ( x , y ) V(x,y) V ( x , y ) that is monotone in y y y , write V i : = V ( X i , Y i ) V_i:=V(X_i,Y_i) V i := V ( X i , Y i ) and V ^ n + j : = V ( X n + j , c n + j ) \widehat V_{n+j}:=V(X_{n+j},c_{n+j}) V n + j := V ( X n + j , c n + j ) . The weighted conformal p-value of Jin & Candès [12 ] is
p j = ∑ i ≤ n w ( X i ) 1 { V i < V ^ n + j } + U j ( w ( X n + j ) + ∑ i ≤ n w ( X i ) 1 { V i = V ^ n + j } ) ∑ i ≤ n w ( X i ) + w ( X n + j ) , p_j=\frac{\sum_{i\le n}w(X_i)\mathbf{1}\{V_i<\widehat V_{n+j}\}+U_j\left(w(X_{n+j})+\sum_{i\le n}w(X_i)\mathbf{1}\{V_i=\widehat V_{n+j}\}\right)}{\sum_{i\le n}w(X_i)+w(X_{n+j})}, p j = ∑ i ≤ n w ( X i ) + w ( X n + j ) ∑ i ≤ n w ( X i ) 1 { V i < V n + j } + U j ( w ( X n + j ) + ∑ i ≤ n w ( X i ) 1 { V i = V n + j } ) ,
with U j ∼ Unif ( 0 , 1 ) U_j\sim\operatorname{Unif}(0,1) U j ∼ Unif ( 0 , 1 ) for tie-breaking. The oracle p-value p j ∗ p_j^* p j ∗ is the same expression with V ^ n + j \widehat V_{n+j} V n + j replaced by the true score V ( X n + j , Y n + j ) V(X_{n+j},Y_{n+j}) V ( X n + j , Y n + j ) . A null unit with c n + j = Y n + j c_{n+j}=Y_{n+j} c n + j = Y n + j (an oracle null) has p j = p j ∗ p_j=p_j^* p j = p j ∗ , and every null unit has p j ≥ p j ∗ p_j\ge p_j^* p j ≥ p j ∗ . BH at level q q q uses the grid t k : = q k / m t_k:=qk/m t k := q k / m ; it finds the largest k k k such that at least k k k p-values are at most t k t_k t k , and rejects the units whose p-values are at most that t k t_k t k . We write D D D for the calibration data. The assumptions of Jin & Candès [12 ] (i.i.d. test tuples and U j U_j U j , independent of D D D ) hold throughout, and some results need only the weaker assumptions (A1)–(A3) of Assumption 61 (Appendix E). Strict selection means that c n + j c_{n+j} c n + j is independent of Y n + j Y_{n+j} Y n + j given X n + j X_{n+j} X n + j ; in the global null every test unit is null. In outlier detection the inliers are drawn from Q Q Q and are the nulls, and the outliers come from another distribution.
Two exact facts organise the analysis. Oracle p-values are exactly uniform, ties included; and, given D D D , the test units are i.i.d., which leads to an exact identity for the excess and to two reductions.
Lemma 1 (Exact uniformity). P ( p j ∗ ≤ s ) = s \mathbb{P}(p_j^*\le s)=s P ( p j ∗ ≤ s ) = s for all s ∈ [ 0 , 1 ] s\in[0,1] s ∈ [ 0 , 1 ] .
Intuition. Given the multiset of the n + 1 n+1 n + 1 points, the test point is each of them with probability proportional to its weight. The randomised weighted CDF maps each tie group to an interval whose length is exactly that group’s probability. So the intervals tile [ 0 , 1 ] [0,1] [ 0 , 1 ] and p ∗ p^* p ∗ is uniform, ties included.
Lemma 2 (Exact covariance identity). Let A k ( D ) : = P ( unit 1 null , p 1 ≤ t k ∣ D ) A_k(D) := \mathbb{P}(\text{unit 1 null},\ p_1 \le t_k \mid D) A k ( D ) := P ( unit 1 null , p 1 ≤ t k ∣ D ) and r k ( D ) : = P ( R 1 = k ∣ D ) r_k(D) := \mathbb{P}(R_1 = k \mid D) r k ( D ) := P ( R 1 = k ∣ D ) , where R 1 R_1 R 1 is the number of BH rejections among ( 0 , p 2 , … , p m ) (0,p_2,\ldots,p_m) ( 0 , p 2 , … , p m ) . If every null unit is an oracle null and ϵ k : = P ( unit 1 non-null , p 1 ∗ ≤ t k ) \epsilon_k := \mathbb{P}(\text{unit 1 non-null},\ p_1^* \le t_k) ϵ k := P ( unit 1 non-null , p 1 ∗ ≤ t k ) , then for every n n n ,
FDR − q = m ∑ k = 1 m 1 k Cov ( A k ( D ) , r k ( D ) ) − m ∑ k = 1 m 1 k ϵ k E r k ( D ) . \operatorname{FDR}-q = m\sum_{k=1}^{m}\frac{1}{k}\operatorname{Cov}(A_k(D),r_k(D))-m\sum_{k=1}^{m}\frac{1}{k}\epsilon_k\mathbb{E}r_k(D). FDR − q = m k = 1 ∑ m k 1 Cov ( A k ( D ) , r k ( D )) − m k = 1 ∑ m k 1 ϵ k E r k ( D ) .
Intuition. Given the calibration data, BH sees m m m i.i.d. p-values, and the leave-one-out identity turns the FDR into an average of the null mass below the threshold the other units set. If that null mass did not move with what the other units do, the average would be q q q (minus the ϵ \epsilon ϵ -term). So the whole excess is a covariance between the null mass below t k t_k t k and the chance that the others drive BH to exactly k k k rejections.
Lemma 2 is the basis of the fixed-distribution analysis in § 4 . For upper bounds we use a cruder reduction that needs no oracle assumption.
Proposition 3 (Calibration-conditional reduction). Let F D ∗ ( t ) : = P ( p ∗ ≤ t ∣ D ) F_D^*(t) := \mathbb{P}(p^* \le t \mid D) F D ∗ ( t ) := P ( p ∗ ≤ t ∣ D ) for a test point from Q Q Q and M D : = max k ≤ m F D ∗ ( t k ) / t k M_D := \max_{k\le m} F_D^*(t_k)/t_k M D := max k ≤ m F D ∗ ( t k ) / t k . Then FDR ≤ q E [ M D ] \operatorname{FDR} \le q\mathbb{E}[M_D] FDR ≤ q E [ M D ] , for every configuration under JC’s assumptions (non-null units, shared thresholds and score atoms allowed).
Intuition. Conditionally on D D D the test p-values are independent, so BH behaves as under independence, except that a null’s conditional law may put more than t t t of its mass below t t t . BH can exploit that excess at whichever grid point it is largest. So the FDR is at most q q q times the expected worst ratio F D ∗ ( t k ) / t k F_D^*(t_k)/t_k F D ∗ ( t k ) / t k .
In the global null, BH makes a false discovery as soon as it rejects anything, so the FDR is the probability that Simes’ test rejects when applied to m m m i.i.d. draws from the conditional law of a test p-value.
Lemma 4 (Global-null reduction). If every test unit is null, FDR = E ψ m ( F D ) \operatorname{FDR} = \mathbb{E}\psi_m(F_D) FDR = E ψ m ( F D ) , where ψ m \psi_m ψ m is the rejection probability of Simes’ test for m m m i.i.d. draws with CDF F D ( t ) : = P ( p 1 ≤ t ∣ D ) F_D(t) := \mathbb{P}(p_1 \le t \mid D) F D ( t ) := P ( p 1 ≤ t ∣ D ) . If moreover every null is an oracle null (c = Y c=Y c = Y ), then E F D ( t ) = t \mathbb{E}F_D(t)=t E F D ( t ) = t ; in general E F D ( t ) ≤ t \mathbb{E}F_D(t)\le t E F D ( t ) ≤ t .
Heavy cells and the Poisson limit. Our constructions combine many light covariate values of weight 1 with a few heavy cells of weight κ n = B \kappa n = B κn = B and P P P -mass λ / n \lambda/n λ / n , where λ = ω / κ \lambda=\omega/\kappa λ = ω / κ . If N N N is the number of calibration points in a heavy cell, its normalised calibration weight K : = κ N K:=\kappa N K := κ N has mean ω \omega ω and, in the Poisson-limit model, standard deviation κ ω \sqrt{\kappa\omega} κω ; this fluctuation drives all our lower-bound constructions. In the Poisson-limit model the heavy calibration counts are independent Poisson( λ ) (\lambda) ( λ ) variables and the light calibration scores have exactly the uniform CDF; this is the limit n → ∞ n\to\infty n → ∞ at fixed κ \kappa κ . Lemma 44 proves that the global-null FDR converges to its value in this model; for the non-null structures of Section 4 that convergence is not proved, and the results there are stated in the model itself.
Counterexamples and the two mechanisms
We first settle the question.
Theorem 5 (No finite-sample control). There are configurations satisfying JC’s assumptions with FDR > q \operatorname{FDR}>q FDR > q , certified by computer in exact rational arithmetic: (i) global null, n = 5 n=5 n = 5 , m = 2 m=2 m = 2 , q = 4 / 5 q=4/5 q = 4/5 : FDR = 393728107849 / 492032000000 ≈ 0.80021 \operatorname{FDR}=393728107849/492032000000\approx0.80021 FDR = 393728107849/492032000000 ≈ 0.80021 ; (ii) strict selection (c ⊥ Y ∣ X c\perp Y\mid X c ⊥ Y ∣ X ; Y ∣ X Y\mid X Y ∣ X is non-degenerate on the light covariate and fixed on the two heavy ones), n = 80 n=80 n = 80 , m = 8 m=8 m = 8 , q = 1 / 10 q=1/10 q = 1/10 : FDR = 1.13585 q \operatorname{FDR}=1.13585q FDR = 1.13585 q (an exact rational; Appendix § 14 ).
Instance (i) uses a large level (q = 4 / 5 q=4/5 q = 4/5 ) and tiny sizes, so that the excess FDR − q \operatorname{FDR}-q FDR − q , about 2 × 10 − 4 2\times10^{-4} 2 × 1 0 − 4 , can be certified by full enumeration. Instance (ii) is a genuine strict-selection problem at q = 1 / 10 q=1/10 q = 1/10 : the threshold depends only on the covariate, and the outcome is random on the light covariate; its weight ratio, B = 137 B=137 B = 137 , exceeds n = 80 n=80 n = 80 . Both instances use a few covariate cells with large weights, and their excesses come from the two mechanisms below, respectively.
Mechanism 1 (global null, second order). When every test unit is an oracle null, the conditional law of a test p-value given D D D is uniform on average over D D D (Lemma 4 ), so a fluctuation of the heavy calibration weight has no first-order effect. What remains is second order: Simes’ rejection probability is a curved function of the conditional distribution function on the grid, and averaging a curved function over the fluctuation produces an excess of order Var ( K ) = κ ω \operatorname{Var}(K)=\kappa\omega Var ( K ) = κω , that is, of order B / n B/n B / n .
Mechanism 2 (non-null selection, first order). Suppose a block of non-null units (decoys ) shares one threshold, so that their p-values coincide and sit near a BH threshold. Whether BH counts them then depends on the calibration draw. At the mean draw the null law is exactly uniform below q q q (saturation, Lemma 8 ), so the FDR is exactly q q q on both sides of the switch. Away from the mean, each side’s FDR moves linearly in the heavy weight, with different slopes, and the switch pairs a lighter heavy total with the side on which the decoys are counted, which is where the null units gain (Figure 1 ). The average of such a kinked function exceeds its value at the mean by an amount proportional to the standard deviation κ ω \sqrt{\kappa\omega} κω : a first-order effect.
Figure 1. Left: the FDR given the heavy calibration total K K K (the configuration of Example 10 , m = 8 m=8 m = 8 , q = 0.1 q=0.1 q = 0.1 , Poisson-limit model; exact evaluation). Each branch is linear through q q q at K = ω K=\omega K = ω ; the switch keeps the counted branch to the left and the other to the right, so the conditional FDR has a kink at the mean and its average exceeds q q q by order κ \sqrt{\kappa} κ . Right: the worst-case excess against κ = B / n \kappa=B/n κ = B / n at m = 8 m=8 m = 8 : exact Poisson-limit FDR of the decoy construction (∝ κ \propto\sqrt{\kappa} ∝ κ ), the global-null linear lower bound (Theorem 16 , ∝ κ \propto\kappa ∝ κ ), and the upper bound of Theorem 6 with B ˉ = κ n / ( 1 + ω ) \bar{B}=\kappa n/(1+\omega) B ˉ = κn / ( 1 + ω ) .
Non-null selection: a worst-case excess of order B m / ( n q ) \sqrt{Bm/(nq)} B m / ( n q )
This section shows that, when non-null units can share a threshold, the worst excess is of order B m / ( n q ) \sqrt{Bm/(nq)} B m / ( n q ) . We give an upper bound valid in every configuration with B ≤ n B\le n B ≤ n at every n n n (Theorem 6 ), a lower bound of the same order in the Poisson-limit model (Theorems 9 and 11 ), and a fixed-distribution result as n → ∞ n\to\infty n → ∞ at fixed B B B (Theorem 12 ). Theorem 15 collects them.
Upper bound.
Theorem 6 (Upper bound, every configuration). Let κ = B / n ≤ 1 \kappa=B/n\le1 κ = B / n ≤ 1 , x : = B ˉ m / ( n q ) x:=\bar{B}m/(nq) x := B ˉ m / ( n q ) and c : = 2 + ( 1 + κ ) q / 3 c:=2+(1+\kappa)q/3 c := 2 + ( 1 + κ ) q /3 . Then
FDR q − 1 ≤ E [ M D ] − 1 ≤ κ + 1.612 ( 1 + κ ) x + 2.7726 c x , \frac{\operatorname{FDR}}{q}-1 \le\mathbb{E}[M_D]-1 \le\kappa+1.612\sqrt{(1+\kappa)x}+2.7726cx, q FDR − 1 ≤ E [ M D ] − 1 ≤ κ + 1.612 ( 1 + κ ) x + 2.7726 c x , and E [ M D ] − 1 ≤ κ + J \mathbb{E}[M_D]-1\le\kappa+J E [ M D ] − 1 ≤ κ + J with J J J an explicit one-dimensional integral (computable, slightly tighter).
Intuition. The reduction turns the FDR into the worst relative error of a weighted empirical CDF on the grid. At t k t_k t k that relative error is about B ‾ m / ( n q k ) \sqrt{\overline{B}m/(nqk)} B m / ( n q k ) , so the first grid points dominate. One-sided Bernstein tails summed over the grid and integrated exactly give the constant 2 ∫ 0 ∞ min ( 1 , 1 / ( e v 2 − 1 ) ) d v = 1.612 \sqrt{2}\int_0^\infty\min(1,1/(e^{v^2}-1))\,dv=1.612 2 ∫ 0 ∞ min ( 1 , 1/ ( e v 2 − 1 )) d v = 1.612 .
The constant 1.612 is far above the lower-bound constant c ( q ) c(q) c ( q ) below. Bounds proved through the reduction cannot go below about 0.399:
Proposition 7 (No better constant through the reduction). In the Poisson-limit model, a heavy atom placed below every light score with ω = t 1 / ( 1 − t 1 ) \omega=t_1/(1-t_1) ω = t 1 / ( 1 − t 1 ) gives, as κ → 0 \kappa\to0 κ → 0 , E [ M D ] − 1 = 1 − 1 / m 2 π B ‾ m / ( n q ) ( 1 + o ( 1 ) ) \mathbb{E}[M_D]-1=\frac{1-1/m}{\sqrt{2\pi}}\sqrt{\overline{B}m/(nq)}\,(1+o(1)) E [ M D ] − 1 = 2 π 1 − 1/ m B m / ( n q ) ( 1 + o ( 1 )) . So no bound proved through Proposition 3 has a leading constant below ( 1 − 1 / m ) / 2 π (1-1/m)/\sqrt{2\pi} ( 1 − 1/ m ) / 2 π , which tends to 1 / 2 π ≈ 0.399 1/\sqrt{2\pi}\approx0.399 1/ 2 π ≈ 0.399 .
Intuition. With all heavy mass at the very bottom, filling [ 0 , t 1 ] [0,t_1] [ 0 , t 1 ] on average, every ratio F D ∗ ( t k ) / t k F_D^*(t_k)/t_k F D ∗ ( t k ) / t k equals 1 at the mean draw and falls linearly in the heavy total, steepest at k = 1 k=1 k = 1 (slope about m / q m/q m / q ). A light draw pushes M D M_D M D up through k = 1 k=1 k = 1 . The Gaussian mean of that positive part, σ / 2 π \sigma/\sqrt{2\pi} σ / 2 π , is the whole effect.
Lower bounds: decoys in the Poisson limit
The lower bounds use Mechanism 2 in the Poisson-limit model. There is one heavy cell, at score quantile g g g , with normalised calibration weight K K K of mean ω \omega ω . Given K K K , a light unit with score quantile u u u has p-value f K ( u ) : = ( u + K 1 { g < u } ) / ( 1 + K ) f_K(u):=(u+K\mathbf{1}\{g<u\})/(1+K) f K ( u ) := ( u + K 1 { g < u }) / ( 1 + K ) , and heavy test units have p-values spread over the heavy window [ g / ( 1 + K ) , ( g + K ) / ( 1 + K ) ] [g/(1+K),(g+K)/(1+K)] [ g / ( 1 + K ) , ( g + K ) / ( 1 + K )] . A fraction ζ \zeta ζ of the light units, those with u > 1 − ζ u>1-\zeta u > 1 − ζ , are decoys: they share the threshold quantile v 0 v_0 v 0 , so all of them have the p-value f K ( v 0 ) f_K(v_0) f K ( v 0 ) . We write A k ( K ) A_k(K) A k ( K ) for the null mass below t k t_k t k given K K K . The deterministic-K K K FDR is the FDR of the limiting model at a fixed value of K K K , with the decoy atom counted from t k 0 t_{k_0} t k 0 on (side − - − ) or from t k 0 + 1 t_{k_0+1} t k 0 + 1 on (side + + + ), where t k 0 t_{k_0} t k 0 is the grid point it crosses (Appendix § 11 gives the details).
Lemma 8 (Saturation at the mean). In the decoy structure (one heavy cell of mean weight ω \omega ω ; light nulls; a non-null decoy block with a common threshold; f ω ( 1 − ζ ) ≥ q f_\omega(1-\zeta)\ge q f ω ( 1 − ζ ) ≥ q ), the deterministic-K K K FDR equals q q q at K = ω K=\omega K = ω on both sides of the switch.
Theorem 9 (The κ \sqrt{\kappa} κ expansion). In the Poisson-limit model, let the decoy atom cross the grid point t k 0 t_{k_0} t k 0 (1 ≤ k 0 ≤ m 1\le k_0\le m 1 ≤ k 0 ≤ m ) at the switch point k ∗ = ω + z 0 κ ω k^*=\omega+z_0\sqrt{\kappa\omega} k ∗ = ω + z 0 κω , and assume strict saturation (f ω ( 1 − ζ ) > q f_\omega(1-\zeta)>q f ω ( 1 − ζ ) > q ), v 0 > g v_0>g v 0 > g , and genericity at K = ω K=\omega K = ω (no grid point at an edge of the heavy window or at a jump of the light map). Then, uniformly in z 0 z_0 z 0 in compacts, as κ → 0 \kappa\to0 κ → 0 , F D R = q + κ ω ( h + − h − ) φ ( z 0 ) + O ( κ ) \mathrm{FDR}=q+\sqrt{\kappa\omega}(h_+-h_-)\varphi(z_0)+O(\kappa) FDR = q + κω ( h + − h − ) φ ( z 0 ) + O ( κ ) , with h s = m ∑ k 1 k A k ( ω ) P s ( R 1 = k ) h_s=m\sum_k\frac{1}{k}A_k(\omega)P_s(R_1=k) h s = m ∑ k k 1 A k ( ω ) P s ( R 1 = k ) , where under P − P_- P − (P + P_+ P + ) the decoy atom is counted from t k 0 t_{k_0} t k 0 (from t k 0 + 1 t_{k_0+1} t k 0 + 1 ) on.
Intuition. At the mean heavy total the FDR is exactly q q q on both sides of the switch. Each side moves linearly in the heavy total, with different slopes. The switch selects the steeper side exactly when the total is low, so the average picks up the slope difference times E ( K − ω ) + ≈ κ ω φ ( z 0 ) \mathbb{E}(K-\omega)^+\approx\sqrt{\kappa\omega}\,\varphi(z_0) E ( K − ω ) + ≈ κω φ ( z 0 ) .
A concrete instance at m = 8 m=8 m = 8 :
Example 10. q = 0.1 q=0.1 q = 0.1 , m = 8 m=8 m = 8 , one heavy cell at g = 0.018418 g=0.018418 g = 0.018418 with ω = 0.086271 \omega=0.086271 ω = 0.086271 , decoy fraction ζ = 0.91847 \zeta=0.91847 ζ = 0.91847 : F D R / q − 1 = 0.6707 κ + 1.68 κ \mathrm{FDR}/q-1=0.6707\sqrt{\kappa}+1.68\kappa FDR / q − 1 = 0.6707 κ + 1.68 κ (the coefficient is the formula of Theorem 9 , evaluated numerically; the κ \kappa κ term is fitted to exact Poisson-limit evaluations down to κ = 10 − 6 \kappa=10^{-6} κ = 1 0 − 6 ).
The coefficient is large: at κ = 10 − 3 \kappa=10^{-3} κ = 1 0 − 3 this gives F D R / q − 1 ≈ 0.023 \mathrm{FDR}/q-1\approx0.023 FDR / q − 1 ≈ 0.023 . It also grows with m m m . Placing the decoys at the second BH threshold gives a family whose coefficient has an explicit limit as m → ∞ m\to\infty m → ∞ .
Theorem 11 (m \sqrt{m} m growth with an explicit constant). In the Poisson-limit model, for a decoy switching at t 2 t_2 t 2 with ω = q / m \omega=q/m ω = q / m and decoy fraction 1 / m 1/m 1/ m (the structure of Theorem 35), the coefficient C m : = lim κ → 0 ( F D R / q − 1 ) / κ C_m:=\lim_{\kappa\to0}(\mathrm{FDR}/q-1)/\sqrt{\kappa} C m := lim κ → 0 ( FDR / q − 1 ) / κ satisfies lim m → ∞ C m / m / q = c ( q ) : = ( 1 − q ) e 1 − 2 q / ( 2 2 π ) \lim_{m\to\infty}C_m/\sqrt{m/q}=c(q):=(1-q)e^{1-2q}/(2\sqrt{2\pi}) lim m → ∞ C m / m / q = c ( q ) := ( 1 − q ) e 1 − 2 q / ( 2 2 π ) (= 0.054 at q = 0.1 q=0.1 q = 0.1 ), an iterated limit (κ → 0 \kappa\to0 κ → 0 first).
Intuition. The decoy p-value sits on t 2 t_2 t 2 , so the calibration draw decides whether BH counts it at the second threshold. A light heavy draw lowers every null p-value and pulls the decoy below t 2 t_2 t 2 , so BH stops at R = 2 R=2 R = 2 instead of R = 1 R=1 R = 1 exactly when the nulls are favoured. The fluctuation is κ ω \sqrt{\kappa\omega} κω with ω ≈ q / m \omega\approx q/m ω ≈ q / m , but its effect is summed over m m m units, which leaves κ m / q \sqrt{\kappa m/q} κm / q .
Fixed distributions. The Poisson-limit model lets the distribution change with n n n . For a single fixed distribution the same mechanism can still give an excess of order n − 1 / 2 n^{-1/2} n − 1/2 , but only if the decoy threshold sits exactly at the population switching point (condition (F2) in Appendix § 9 ); otherwise the switch freezes for large n n n , and Remark 26 outlines why the excess is then o ( n − 1 / 2 ) o(n^{-1/2}) o ( n − 1/2 ) . Under (F1)–(F4) the limit c 0 c_0 c 0 below can still be zero (for example when the decoys sit at t 1 t_1 t 1 ); c 0 > 0 c_0>0 c 0 > 0 is certified for the distributions of Example 13.
Theorem 12 (Fixed distribution, finite n n n ). For a fixed P P P with one heavy atom of weight B B B and P P P -mass π H \pi_H π H , uniform light scores, and the decoy threshold at the population switching point (conditions (F1)–(F4)), n ( F D R n − q ) → c 0 \sqrt{n}(\mathrm{FDR}_n-q)\to c_0 n ( FDR n − q ) → c 0 as n → ∞ n\to\infty n → ∞ , with c 0 c_0 c 0 in closed form (Appendix § 9 ).
Example 13 (The parameters of Example 10). For the parameters of Example 10 with a fixed weight B B B , the sign c 0 > 0 c_0>0 c 0 > 0 is certified in exact rational arithmetic at B = 2 , 5 , 10 , 100 B=2,5,10,100 B = 2 , 5 , 10 , 100 . Numerically, c 0 / ( q B ) = 0.316 , 0.522 , 0.595 , 0.663 c_0/(q\sqrt{B})=0.316, 0.522, 0.595, 0.663 c 0 / ( q B ) = 0.316 , 0.522 , 0.595 , 0.663 at these B B B , approaching that example’s Poisson-limit coefficient 0.6707 as B B B grows (numerically; the limit in B B B is not proved).
Intuition. For a fixed distribution each conditional null mass fluctuates by about 1 / n 1/\sqrt{n} 1/ n , so smooth dependence on the calibration data contributes only O ( 1 / n ) O(1/n) O ( 1/ n ) to the covariance identity (Lemma 2). The decoys’ rejection is an on/off switch of the same fluctuation, and E [ X 1 { X > 0 } ] \mathbb{E}[X\mathbf{1}\{X>0\}] E [ X 1 { X > 0 }] is of order sd ( X ) \operatorname{sd}(X) sd ( X ) . So the excess is of order 1 / n 1/\sqrt{n} 1/ n , with a constant that, numerically, tends to the Poisson-limit coefficient as B B B grows.
Remark 14 (Outlier detection). In outlier detection, in the Poisson-limit model, the first-order κ \sqrt{\kappa} κ term cannot produce an excess once κ \kappa κ is below a small threshold (Proposition 41). At moderate B / n B/n B / n a switching outlier block does exceed the largest global-null FDR our searches found (numerically; Appendix § 11 ).
The rate. The next theorem combines the upper and lower bounds and states exactly what is proved where.
Theorem 15 (The rate). Write x : = B m ˉ / ( n q ) x:=B\bar{m}/(nq) x := B m ˉ / ( n q ) .
(i) Upper bound, every n n n . For every configuration under JC’s assumptions with weight ratio at most B B B and κ = B / n ≤ 1 \kappa=B/n\le1 κ = B / n ≤ 1 , F D R / q − 1 ≤ κ + 1.612 ( 1 + κ ) x + 2.776 c x \mathrm{FDR}/q-1\le\kappa+1.612\sqrt{(1+\kappa)x}+2.776\,c\,x FDR / q − 1 ≤ κ + 1.612 ( 1 + κ ) x + 2.776 c x (Theorem 6).
(ii) Lower bound, Poisson-limit model, iterated limit. For the structure of Theorem 11 in the Poisson-limit model, take κ → 0 \kappa\to0 κ → 0 first and then m → ∞ m\to\infty m → ∞ :
lim m → ∞ lim κ → 0 F D R / q − 1 κ m / q = c ( q ) = ( 1 − q ) e − 1 − 2 q 2 2 π . \lim_{m\to\infty}\lim_{\kappa\to0}\frac{\mathrm{FDR}/q-1}{\sqrt{\kappa m/q}}=c(q)=\frac{(1-q)e^{-1-2q}}{2\sqrt{2\pi}}. m → ∞ lim κ → 0 lim κm / q FDR / q − 1 = c ( q ) = 2 2 π ( 1 − q ) e − 1 − 2 q . The limits are not shown to commute, and no rate is given for the inner one. That the finite-n n n FDR converges to its Poisson-limit value as n → ∞ n\to\infty n → ∞ at fixed κ = B / n \kappa=B/n κ = B / n is not proved for this structure.
(iii) Finite-n n n support. For one fixed distribution (fixed B B B , n → ∞ n \to\infty n → ∞ ), Theorem 12 gives n ( F D R n − q ) → c 0 \sqrt{n}(\mathrm{FDR}_n-q) \to c_0 n ( FDR n − q ) → c 0 , and c 0 > 0 c_0 > 0 c 0 > 0 for the distributions certified in Example 13 . So for those distributions the excess is of order n − 1 / 2 n^{-1/2} n − 1/2 for all large n n n , with no explicit n 0 n_0 n 0 . Evaluating c 0 c_0 c 0 for the structures of Theorem 11 at fixed B B B shows the same m \sqrt{m} m growth (numerically).
So the worst excess is of order B m / ( n q ) \sqrt{Bm/(nq)} B m / ( n q ) : the upper side holds at every n n n , and the lower side holds in the Poisson-limit model in the iterated limit (ii), with (iii) as finite-n n n support.
Search for a larger lower constant (numerical). With a saturation margin (f ω ( 1 − ζ ) ≥ q + 0.005 f_\omega(1-\zeta) \ge q+0.005 f ω ( 1 − ζ ) ≥ q + 0.005 ), structures with bottom-placed or stacked heavy cells and up to three decoy blocks do not beat Theorem 11 ’s family: the best coefficients found are 0.0553, 0.0547, 0.0544 and 0.0542 (each ± 0.0005 \pm0.0005 ± 0.0005 ) in units of m / q \sqrt{m/q} m / q at m = 32 , 64 , 128 , 256 m=32,64,128,256 m = 32 , 64 , 128 , 256 , against 0.0552, 0.0546, 0.0544 and 0.0542 for the structure of Theorem 11 . Without the margin the optimiser finds a spurious gain at a degenerate boundary, where the non-null-oracle term of Theorem 2 becomes first order; exact Poisson-limit sums rule it out. So we have no evidence that the lower constant c ( q ) c(q) c ( q ) can be improved within these families; the gap to the upper constant 1.612 is discussed in § 8 .
Global null: an O ( B / n ) O(B/n) O ( B / n ) excess, with a lower bound linear in m m m
When every test unit is null, all rejections are false, and the FDR is the probability that Simes’ test rejects (Lemma 4 ). Mechanism 1 is then the only effect, and it is second order: the excess is of order B / n B/n B / n rather than B / n \sqrt{B/n} B / n . We give a lower bound that is linear in m m m , an upper bound that is O ( B / n ) O(B/n) O ( B / n ) at every fixed m m m , and a conditional result on the constant.
Theorem 16 (Lower bound). For every m ≥ 2 m \ge2 m ≥ 2 , q ∈ ( 0 , 1 ) q \in(0,1) q ∈ ( 0 , 1 ) and ϵ > 0 \epsilon> 0 ϵ > 0 , and all small κ \kappa κ with n n n large, there is a global-null configuration with F D R ≥ q + ( L m ∗ ( q ) − ϵ ) B / n \mathrm{FDR} \ge q + (L_m^*(q)-\epsilon)B/n FDR ≥ q + ( L m ∗ ( q ) − ϵ ) B / n , where L m ∗ ( q ) L_m^*(q) L m ∗ ( q ) , the best first-order coefficient over the Poisson window configurations of Appendix § 12 , satisfies L m ∗ ( q ) ≥ 1 2 q ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) L_m^*(q) \ge\frac{1}{2}q(1-q)(m-1)(1-q+q/m) L m ∗ ( q ) ≥ 2 1 q ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) and lim inf m → ∞ L m ∗ ( q ) / ( q m ) ≥ 1 2 ( 1 − q ) \liminf_{m\to\infty} L_m^*(q)/(qm) \ge\frac{1}{2}(1-q) lim inf m → ∞ L m ∗ ( q ) / ( q m ) ≥ 2 1 ( 1 − q ) . So the excess is linear in m m m , and as m → ∞ m \to\infty m → ∞ its constant per unit of q m B / n qmB/n q m B / n is at least 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) .
Intuition. Heavy cells in different bands fluctuate independently, so their contributions add. The band curvatures of Simes’ test sum to exactly m ( m − 1 ) ( 1 − q ) m(m-1)(1-q) m ( m − 1 ) ( 1 − q ) , because Simes stays exact when a p-value is duplicated (∑ k = 1 m − 1 D k = m ( m − 1 ) ( 1 − q ) \sum_{k=1}^{m-1}D_k=m(m-1)(1-q) ∑ k = 1 m − 1 D k = m ( m − 1 ) ( 1 − q ) ). Spreading cells over all bands collects that sum.
So some global-null configurations reach an excess of order m B / n mB/n m B / n , against B m / ( n q ) \sqrt{Bm/(nq)} B m / ( n q ) with non-null units, which is much smaller when B m / n Bm/n B m / n is small. Theorem 17 gives O ( B / n ) O(B/n) O ( B / n ) at fixed m m m ; an upper bound linear in m m m is not proved. The upper bound comes from the envelope of Theorem 21 , combined with a high-probability bound on the conditional distribution function of a test p-value; it holds for every score law, ties included.
Theorem 17 (Upper bound, every score law). In the global null, for ρ > 1 + κ \rho> 1+\kappa ρ > 1 + κ , F D R ≤ q [ 1 + 2 ( m − 1 ) β m ( ρ ) B n + 1 + e − 2 a ′ 1 − e − a ′ ] \mathrm{FDR} \le q[1+2(m-1)\beta_m(\rho)\frac{B}{n+1}+\frac{e^{-2a'}}{1-e^{-a'}}] FDR ≤ q [ 1 + 2 ( m − 1 ) β m ( ρ ) n + 1 B + 1 − e − a ′ e − 2 a ′ ] , with a ′ = n q ( ρ − 1 − κ ) 2 / ( 2 B m ˉ ( 3 ρ − 2 − 2 κ ) ) a'=nq(\rho-1-\kappa)^2/(2B\bar{m}(3\rho-2-2\kappa)) a ′ = n q ( ρ − 1 − κ ) 2 / ( 2 B m ˉ ( 3 ρ − 2 − 2 κ )) and β m ( ρ ) \beta_m(\rho) β m ( ρ ) explicit. Also F D R ≤ q [ 1 + m ( m − 1 ) 2 B / ( n + 1 ) ] \mathrm{FDR} \le q[1+m(m-1)^2B/(n+1)] FDR ≤ q [ 1 + m ( m − 1 ) 2 B / ( n + 1 )] for all n n n and B B B ; in particular, at fixed m m m the excess is O ( B / n ) O(B/n) O ( B / n ) . Both bounds hold for non-oracle nulls too, since then p j ≥ p j ∗ p_j \ge p_j^* p j ≥ p j ∗ and B H BH B H is monotone.
Remark 18 (Linearity in m m m is not proved). The bound is linear in m m m wherever β m ( ρ ) \beta_m(\rho) β m ( ρ ) stays bounded in m m m . Numerically it does (β m ( 1 ) ≈ 1.78 \beta_m(1)\approx1.78 β m ( 1 ) ≈ 1.78 at m = 1000 m=1000 m = 1000 , q = 0.1 q=0.1 q = 0.1 , T ˉ B = 0 \bar{T}_B=0 T ˉ B = 0 ), and an m m m -free majorant via a Chernoff bound is sketched but not written out; it also shows ρ ( q + m / ( n + 1 ) ) < 1 \rho(q+\!m/(n+1))<1 ρ ( q + m / ( n + 1 )) < 1 . So linearity of the upper bound in m m m is supported numerically and by a sketch, but not proved.
Is 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) the right constant? Within the family of configurations used for the lower bound it is, under a condition that we have verified in exact arithmetic for many ( m , q ) (m,q) ( m , q ) but have not proved.
Proposition 19 (Conditional sharpness). Simes’ Hessian at the uniform point has non-positive off-diagonal entries and, for 0 < q ≤ 1 / e 0 < q \le1/e 0 < q ≤ 1/ e , is positive semidefinite on grid points 2 , … , m 2,\ldots,m 2 , … , m . If all band curvatures are positive (the positive-curvature condition of Appendix § 12 , which is open), then within the Poisson window family of Theorem 16 (windows strictly inside one band each) the first-order constant per unit of q m B / n qmB/n q m B / n is at most 1 2 ( 1 − q ) ( m − 1 ) / m \frac{1}{2}(1-q)(m-1)/m 2 1 ( 1 − q ) ( m − 1 ) / m ; so the constant 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) is attained as m → ∞ m \to\infty m → ∞ and is not exceeded in that
family. For 0 < q ≤ 1 / e 0 < q \le1/e 0 < q ≤ 1/ e the Hessian result extends this bound on the quadratic form to windows above t 1 t_1 t 1 that contain grid points, but the first-order expansion is not proved for such windows.
We conjecture that the answer is the same over all global-null configurations.
Conjecture 20. In the global null the linear-regime constant equals 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) .
What a practitioner can compute
The counterexamples show that BH on weighted conformal p-values can exceed its level, and Theorem 6 shows that the excess is small when B m / ( n q ) Bm/(nq) B m / ( n q ) is small. We give three tools: an envelope that bounds the FDR by the expectation of a quantity computable from data, a deflated level with proved finite-sample control, and a much less conservative numerical rule. When the thresholds are shared across units, weighted conformal selection controls the FDR exactly and is the safe choice (end of this section).
Theorem 21 (Envelope). FDR ≤ q m ∑ j E Γ j ∗ ≤ q m ∑ j E Γ j sup ≤ q m ∑ j E Γ ~ j \operatorname{FDR} \le\frac{q}{m}\sum_j \mathbb{E}\Gamma_j^* \le\frac{q}{m}\sum_j \mathbb{E}\Gamma_j^{\operatorname{sup}} \le\frac{q}{m}\sum_j \mathbb{E}\widetilde{\Gamma}_j FDR ≤ m q ∑ j E Γ j ∗ ≤ m q ∑ j E Γ j sup ≤ m q ∑ j E Γ j , where Γ ~ j = R 0 ( p − j − τ ˉ j ) / R 0 ( p − j + τ ˉ j ) \widetilde{\Gamma}_j = R^0(p_{-j}-\bar{\tau}_j)/R^0(p_{-j}+\bar{\tau}_j) Γ j = R 0 ( p − j − τ ˉ j ) / R 0 ( p − j + τ ˉ j ) (R 0 R^0 R 0 : BH count with an appended zero) and τ ˉ j ≤ B / ( n + 1 ) \bar{\tau}_j \le B/(n+1) τ ˉ j ≤ B / ( n + 1 ) ; Γ ~ j \widetilde{\Gamma}_j Γ j and Γ j sup \Gamma_j^{\operatorname{sup}} Γ j sup are computable from data.
Each Γ ~ j \widetilde{\Gamma}_j Γ j is at least 1, and it equals 1 unless shifting the other p-values by ± τ ˉ j \pm\bar{\tau}_j ± τ ˉ j changes the BH count. So the envelope is close to q q q when few p-values lie within B / ( n + 1 ) B/(n+1) B / ( n + 1 ) of a BH threshold, and it flags the data sets on which the weights could matter.
Provable level deflation (conservative). By Theorem 6 , BH at level q ′ q' q ′ with q ′ ( 1 + ε ( q ′ ) ) ≤ q q'(1+\varepsilon(q')) \le q q ′ ( 1 + ε ( q ′ )) ≤ q controls the FDR at q q q in finite samples, where ε \varepsilon ε uses only n n n , m m m , q q q and a weight bound B B B (Figure 3 ). At q = 0.1 q=0.1 q = 0.1 , ε \varepsilon ε is at most 0.3 once n / B ≳ 3.8 ⋅ 10 4 n/B \gtrsim3.8 \cdot10^4 n / B ≳ 3.8 ⋅ 1 0 4 (m = 100 m=100 m = 100 ) or 3.8 ⋅ 10 5 3.8 \cdot10^5 3.8 ⋅ 1 0 5 (m = 1000 m=1000 m = 1000 ). The level has to be solved from this inequality rather than set to q / ( 1 + ε ( q ) ) q/(1+\varepsilon(q)) q / ( 1 + ε ( q )) , because ε \varepsilon ε grows as the level falls: at n = 10 5 n=10^5 n = 1 0 5 , m = 100 m=100 m = 100 , B = 2 B=2 B = 2 and q = 0.1 q=0.1 q = 0.1 the solution is q ′ = 0.0770 q'=0.0770 q ′ = 0.0770 , while q / ( 1 + ε ( q ) ) = 0.0795 q/(1+\varepsilon(q))=0.0795 q / ( 1 + ε ( q )) = 0.0795 would only guarantee FDR ≤ 0.1028 \le0.1028 ≤ 0.1028 .
Figure 3. Provable excess ε \varepsilon ε (Theorem 6 , integral form J J J ) against n n n , q = 0.1 q = 0.1 q = 0.1 , for m = 10 , 100 , 1000 m = 10, 100, 1000 m = 10 , 100 , 1000 and weight ratios B = 2 B = 2 B = 2 (solid) and B = 10 B = 10 B = 10 (dashed). BH at the level q ′ q' q ′ solving q ′ ( 1 + ε ( q ′ ) ) = q q'(1 + \varepsilon(q')) = q q ′ ( 1 + ε ( q ′ )) = q controls the FDR at q q q .
Rule of thumb (numerical, not a guarantee). Across every configuration our searches found (909 optimised Poisson-limit configurations over selection with decoys, strict selection, outlier detection, non-null signal families and global-null families, q ∈ { 0.05 , 0.1 , 0.2 } q \in\{0.05,0.1,0.2\} q ∈ { 0.05 , 0.1 , 0.2 } , m m m from 2 to 1024; plus the bottom-placed and stacked structures of § 4 ), with κ = B / n ≤ 0.1 \kappa=B/n \le0.1 κ = B / n ≤ 0.1 ,
FDR q − 1 ≤ 0.17 B m n q , \frac{\operatorname{FDR}}{q}-1 \le0.17\sqrt{\frac{Bm}{nq}}, q FDR − 1 ≤ 0.17 n q B m ,
and at any κ \kappa κ the FDR never exceeded 1.52 q 1.52q 1.52 q . The scaled worst case falls with m m m and then levels off (0.167 at m = 2 m=2 m = 2 , 0.115 at m = 8 m=8 m = 8 , 0.057 at m = 32 m=32 m = 32 , and close to 0.054 for m m m from 64 to 256, from the structures of § 4 ). Used as a deflation, it suggests running BH at the q ′ q' q ′ that solves q ′ ( 1 + 0.17 B m / ( n q ′ ) ) = q q'(1+0.17\sqrt{Bm/(nq')})=q q ′ ( 1 + 0.17 B m / ( n q ′ ) ) = q . This is a heuristic: we did not rerun the worst cases at lowered levels. Its constant is about ten times smaller than the proved one (Table 1).
Table 1. Bounds on the relative excess FDR/q − 1 q-1 q − 1 when BH is run at level q q q . The proved values are for q = 0.1 q=0.1 q = 0.1 (they depend on ( m , B , n ) (m,B,n) ( m , B , n ) almost only through x x x ). The rule of thumb covers every worst case found, but carries no guarantee.
Shared thresholds in selection. Use weighted conformal selection with pruning, which has exact finite-sample control ([12 Thm. 3.1] ).
Discussion and open problems
BH on weighted conformal p p p -values does not control the FDR in finite samples. When non-null units can share a threshold the worst excess is of order B m / ( n q ) \sqrt{Bm/(nq)} B m / ( n q ) (the upper bound at every n n n with B ≤ n B \le n B ≤ n , the lower bound in the Poisson-limit model); in the global null it is O ( B / n ) O(B/n) O ( B / n ) at fixed m m m , with a lower bound that grows linearly in m m m . In practice the excess is small when B m / ( n q ) Bm/(nq) B m / ( n q ) is small, and a deflated level restores provable control. Several questions remain.
The constants. The upper and lower constants of Theorem 15 are 1.612 and c ( q ) c(q) c ( q ) , which is 0.054 at q = 0.1 q = 0.1 q = 0.1 . Of the factor of about 30 between them, at most about 3.1 is lost in evaluating the reduction FDR ≤ q E [ M D ] \le q\,\mathbb{E}[M_D] ≤ q E [ M D ] (about 1.64 from the Chernoff tails and at most about 1.9 from the union over grid points). The rest separates the reduction from the best constructions we found: numerically, the reduction’s own constant is at least about 9.6 times c ( q ) c(q) c ( q ) . So either the true constant is well above c ( q ) c(q) c ( q ) , or closing the gap needs an upper bound that does not go through E [ M D ] \mathbb{E}[M_D] E [ M D ] (Proposition 7 ).
Finite n n n . Theorem 12 gives no explicit n 0 n_0 n 0 , and the convergence of the finite-n n n FDR to its Poisson-limit value is proved only for the global null (Lemma 44), not for the non-null structures of § 4 . An explicit n 0 n_0 n 0 would make Theorem 15 (iii) hold at explicit n n n ; that convergence, ideally with a rate, would carry Theorem 15 (ii) to large finite n n n at fixed κ \kappa κ , as Lemma 44 does for the global null.
The global null. The positive-curvature condition (Appendix § 12 ) and Conjecture 20 are open, and so is an m m m -free bound on β m ( ρ ) \beta_m(\rho) β m ( ρ ) , which would make the upper bound of Theorem 17 linear in m m m .
Proofs for the exact identities and the fixed-distribution theorem
This appendix proves Lemma 1, Lemma 2 and Theorem 12 . It uses the weighted exchangeability of Lemma 67 and the leave-one-out identity of Lemma 66 .
Under (A1), the calibration pairs ( X i , Y i ) i ≤ n (X_i,Y_i)_{i\le n} ( X i , Y i ) i ≤ n are i.i.d. P P P and the test pair ( X n + 1 , Y n + 1 ) (X_{n+1},Y_{n+1}) ( X n + 1 , Y n + 1 ) has joint law Q Q Q , with d Q / d P = w ( X ) / E P w \mathrm{d}Q/\mathrm{d}P = w(X)/\mathbb{E}_P w d Q / d P = w ( X ) / E P w . Write V i : = V ( X i , Y i ) V_i := V(X_i,Y_i) V i := V ( X i , Y i ) , so that V n + 1 V_{n+1} V n + 1 is the true score of the test point. The randomised oracle p-value of Jin & Candès [12 ] (see eq. (4)) is
p ∗ = ∑ i ≤ n w ( X i ) 1 { V i < V n + 1 } + U ( w ( X n + 1 ) + ∑ i ≤ n w ( X i ) 1 { V i = V n + 1 } ) ∑ i ≤ n w ( X i ) + w ( X n + 1 ) . (1) p^* = \frac{\sum_{i\le n}w(X_i)1\{V_i<V_{n+1}\}+U\left(w(X_{n+1})+\sum_{i\le n}w(X_i)1\{V_i=V_{n+1}\}\right)}{\sum_{i\le n}w(X_i)+w(X_{n+1})}. \tag*{(1)} p ∗ = ∑ i ≤ n w ( X i ) + w ( X n + 1 ) ∑ i ≤ n w ( X i ) 1 { V i < V n + 1 } + U ( w ( X n + 1 ) + ∑ i ≤ n w ( X i ) 1 { V i = V n + 1 } ) . (1)
where U ∼ U ( 0 , 1 ) U\sim U(0,1) U ∼ U ( 0 , 1 ) is independent. Lemma 1 states that P ( p ∗ ≤ s ) = s \mathbb{P}(p^*\le s)=s P ( p ∗ ≤ s ) = s for every s ∈ [ 0 , 1 ] s\in[0,1] s ∈ [ 0 , 1 ] .
Proof of Lemma 1. Condition on the multiset of the n + 1 n+1 n + 1 pairs ( X i , Y i ) (X_i,Y_i) ( X i , Y i ) , i ≤ n + 1 i\le n+1 i ≤ n + 1 ; Lemma 67 covers exactly this conditioning. By Lemma 67, given the multiset, the test point is pair i i i with probability w ( X i ) / ∑ l ≤ n + 1 w ( X l ) w(X_i)/\sum_{l\le n+1}w(X_l) w ( X i ) / ∑ l ≤ n + 1 w ( X l ) , and U U U is uniform and independent of that choice. Lemma 67 is self-contained and handles ties; it is the covariate-shift case of the weighted exchangeability of Tibshirani et al. [20 ] (§3.2, Lemma 2).
Group the n + 1 n+1 n + 1 pairs by distinct score value a a a . Let W a W_a W a be the total weight of group a a a , W < a W_{<a} W < a the total weight of the pairs with smaller scores, and Σ w : = ∑ l ≤ n + 1 w ( X l ) \Sigma_w:=\sum_{l\le n+1}w(X_l) Σ w := ∑ l ≤ n + 1 w ( X l ) .
The test point lands in group a a a with probability W a / Σ w W_a/\Sigma_w W a / Σ w .
Given that, (1) reads p ∗ = ( W < a + U W a ) / Σ w p^*=(W_{<a}+UW_a)/\Sigma_w p ∗ = ( W < a + U W a ) / Σ w , so p ∗ p^* p ∗ is uniform on [ W < a / Σ w , ( W < a + W a ) / Σ w ] [W_{<a}/\Sigma_w,(W_{<a}+W_a)/\Sigma_w] [ W < a / Σ w , ( W < a + W a ) / Σ w ] .
These intervals tile [ 0 , 1 ] [0,1] [ 0 , 1 ] , and each has length equal to its probability. So, given the multiset, p ∗ p^* p ∗ is exactly uniform on [ 0 , 1 ] [0,1] [ 0 , 1 ] , ties included, and hence P ( p ∗ ≤ s ) = s \mathbb{P}(p^*\le s)=s P ( p ∗ ≤ s ) = s for every s ∈ [ 0 , 1 ] s\in[0,1] s ∈ [ 0 , 1 ] .
The exact covariance identity
Assume JC’s independence: the m m m test units are i.i.d. and independent of the calibration data D D D ; and, given D D D , the test p-values are i.i.d., each a function of its own unit, its own U j U_j U j and D D D . Let A k ( D ) A_k(D) A k ( D ) , r k ( D ) r_k(D) r k ( D ) , R 1 R_1 R 1 and ε k \varepsilon_k ε k be as in Lemma 2. We prove two statements; (b) is the identity of Lemma 2.
(a) At every n n n ,
F D R = m ∑ k = 1 m 1 k E [ A k ( D ) r k ( D ) ] . (2) \mathrm{FDR}=m\sum_{k=1}^{m}\frac{1}{k}\mathbb{E}[A_k(D)r_k(D)]. \tag*{(2)} FDR = m k = 1 ∑ m k 1 E [ A k ( D ) r k ( D )] . (2)
(b) Suppose every null unit is an oracle null (p = p ∗ p=p^* p = p ∗ ), and put ε k : = P ( unit 1 non-null , p 1 ∗ ≤ t k ) \varepsilon_k:=\mathbb{P}(\text{unit 1 non-null},p_1^*\le t_k) ε k := P ( unit 1 non-null , p 1 ∗ ≤ t k ) . Then E A k ( D ) = t k − ε k \mathbb{E}A_k(D)=t_k-\varepsilon_k E A k ( D ) = t k − ε k and
F D R − q = m ∑ k = 1 m 1 k Cov ( A k ( D ) , r k ( D ) ) − m ∑ k = 1 m 1 k ε k E r k ( D ) . (3) \mathrm{FDR}-q=m\sum_{k=1}^{m}\frac{1}{k}\operatorname{Cov}(A_k(D),r_k(D))-m\sum_{k=1}^{m}\frac{1}{k}\varepsilon_k\mathbb{E}r_k(D). \tag*{(3)} FDR − q = m k = 1 ∑ m k 1 Cov ( A k ( D ) , r k ( D )) − m k = 1 ∑ m k 1 ε k E r k ( D ) . (3)
Proof of Lemma 2. (a) By the leave-one-out identity (Lemma 66), unit j j j is rejected iff p j ≤ t R j p_j\le t_{R_j} p j ≤ t R j , and then R = R j R=R_j R = R j ; here R j R_j R j is the number of BH rejections when p j p_j p j is replaced by 0. So
F D R = ∑ j = 1 m E [ 1 { j n u l l } 1 { p j ≤ t R j } R j ] . \mathrm{FDR}=\sum_{j=1}^{m}\mathbb{E}\left[\frac{1\{j\ \mathrm{null}\}1\{p_j\le t_{R_j}\}}{R_j}\right]. FDR = j = 1 ∑ m E [ R j 1 { j null } 1 { p j ≤ t R j } ] .
Given D D D , unit j j j is independent of p − j p_{-j} p − j , and R j R_j R j depends only on p − j p_{-j} p − j . So the conditional expectation of 1 { j n u l l , p j ≤ t k } 1\{j\ \mathrm{null},p_j\le t_k\} 1 { j null , p j ≤ t k } given ( D , p − j ) (D,p_{-j}) ( D , p − j ) is A k ( D ) A_k(D) A k ( D ) . Exchangeability of the units then gives
F D R = m E [ A R 1 ( D ) R 1 ] = m ∑ k = 1 m 1 k E [ A k ( D ) 1 { R 1 = k } ] . (4) \mathrm{FDR}=m\mathbb{E}\left[\frac{A_{R_1}(D)}{R_1}\right]=m\sum_{k=1}^{m}\frac{1}{k}\mathbb{E}\left[A_k(D)1\{R_1=k\}\right]. \tag*{(4)} FDR = m E [ R 1 A R 1 ( D ) ] = m k = 1 ∑ m k 1 E [ A k ( D ) 1 { R 1 = k } ] . (4)
Conditioning on D D D gives (2) .
(b) Since every null unit is an oracle null, Lemma 1 gives
E A k ( D ) = P ( p 1 ∗ ≤ t k ) − P ( unit 1 non-null , p 1 ∗ ≤ t k ) = t k − ϵ k . \mathbb{E}A_k(D)=\mathbb{P}(p_1^*\le t_k)-\mathbb{P}(\text{unit 1 non-null},p_1^*\le t_k)=t_k-\epsilon_k. E A k ( D ) = P ( p 1 ∗ ≤ t k ) − P ( unit 1 non-null , p 1 ∗ ≤ t k ) = t k − ϵ k .
Write E [ A k r k ] = E A k E r k + Cov ( A k , r k ) \mathbb{E}[A_kr_k]=\mathbb{E}A_k\mathbb{E}r_k+\operatorname{Cov}(A_k,r_k) E [ A k r k ] = E A k E r k + Cov ( A k , r k ) in (2) . Since t k = q k / m t_k=qk/m t k = q k / m and R 1 ∈ { 1 , … , m } R_1\in\{1,\ldots,m\} R 1 ∈ { 1 , … , m } ,
m ∑ k = 1 m 1 k t k E r k ( D ) = q ∑ k = 1 m E r k ( D ) = q , m\sum_{k=1}^{m}\frac{1}{k}t_k\mathbb{E}r_k(D)=q\sum_{k=1}^{m}\mathbb{E}r_k(D)=q, m k = 1 ∑ m k 1 t k E r k ( D ) = q k = 1 ∑ m E r k ( D ) = q ,
which gives (3) .
Example 22. (A two-state illustration of Lemma 2.) Take m = 2 m=2 m = 2 and q = 0.2 q=0.2 q = 0.2 , so t 1 = 0.1 t_1=0.1 t 1 = 0.1 and t 2 = 0.2 t_2=0.2 t 2 = 0.2 . Let D D D take two states, a a a and b b b , with probability 1 2 \frac{1}{2} 2 1 each. Units are null with probability 1 2 \frac{1}{2} 2 1 , with oracle p-values; the other half are decoys whose oracle p-value exceeds q q q . The null mass below the thresholds is A 1 = 0.10 A_1=0.10 A 1 = 0.10 , A 2 = 0.25 A_2=0.25 A 2 = 0.25 in state a a a and A 1 = 0.10 A_1=0.10 A 1 = 0.10 , A 2 = 0.15 A_2=0.15 A 2 = 0.15 in state b b b ; the means 0.10 and 0.20 and are as Lemma 1 requires. The decoys’ reported p-value is 0.18 in state a a a (below t 2 t_2 t 2 ) and 0.5 in state b b b . With m = 2 m=2 m = 2 , R 1 = 2 R_1=2 R 1 = 2 iff the other unit’s p-value is at most t 2 t_2 t 2 , so r 2 ( a ) = 0.25 + 0.5 = 0.75 r_2(a)=0.25+0.5=0.75 r 2 ( a ) = 0.25 + 0.5 = 0.75 and r 2 ( b ) = 0.15 r_2(b)=0.15 r 2 ( b ) = 0.15 .
Directly: the FDR given the state is 2 [ 0.10 ⋅ 0.25 + 0.25 ⋅ 0.75 / 2 ] = 0.2375 2[0.10\cdot0.25+0.25\cdot0.75/2]=0.2375 2 [ 0.10 ⋅ 0.25 + 0.25 ⋅ 0.75/2 ] = 0.2375 in state a a a and 2 [ 0.10 ⋅ 0.85 + 0.15 ⋅ 0.15 / 2 ] = 0.1925 2[0.10\cdot0.85+0.15\cdot0.15/2]=0.1925 2 [ 0.10 ⋅ 0.85 + 0.15 ⋅ 0.15/2 ] = 0.1925 in state b b b ; the mean is 0.2150.
By the identity: Cov ( A 1 , r 1 ) = 0 \operatorname{Cov}(A_1,r_1)=0 Cov ( A 1 , r 1 ) = 0 and Cov ( A 2 , r 2 ) = 1 2 ( 0.25 ⋅ 0.75 + 0.15 ⋅ 0.15 ) − 0.20 ⋅ 0.45 = 0.015 \operatorname{Cov}(A_2,r_2)=\frac{1}{2}(0.25\cdot0.75+0.15\cdot0.15)-0.20\cdot0.45=0.015 Cov ( A 2 , r 2 ) = 2 1 ( 0.25 ⋅ 0.75 + 0.15 ⋅ 0.15 ) − 0.20 ⋅ 0.45 = 0.015 , so F D R − q = 2 ⋅ 1 2 ⋅ 0.015 = 0.015 \mathrm{FDR}-q=2\cdot\frac{1}{2}\cdot0.015=0.015 FDR − q = 2 ⋅ 2 1 ⋅ 0.015 = 0.015 .
In state a a a , more null mass sits below q q q and the decoys sit below q q q , so BH reaches 2 rejections exactly when the threshold is generous to the nulls. If the decoys’ p-value did not depend on the state (0.5 in both), then r 2 = A 2 r_2=A_2 r 2 = A 2 , so Cov ( A 2 , r 2 ) = Var ( A 2 ) = 0.0025 \operatorname{Cov}(A_2,r_2)=\operatorname{Var}(A_2)=0.0025 Cov ( A 2 , r 2 ) = Var ( A 2 ) = 0.0025 , while Cov ( A 1 , r 1 ) = 0 \operatorname{Cov}(A_1,r_1)=0 Cov ( A 1 , r 1 ) = 0 because A 1 A_1 A 1 is constant. This gives F D R = q + 0.0025 \mathrm{FDR}=q+0.0025 FDR = q + 0.0025 (directly, 1 2 ( 0.2125 + 0.1925 ) = 0.2025 \frac{1}{2}(0.2125+0.1925)=0.2025 2 1 ( 0.2125 + 0.1925 ) = 0.2025 ): smaller, and of second order in the fluctuation.
The fixed-distribution theorem
Setting S ( B , π H , g , ζ , v 0 ; q , m , k 0 ) S(B,\pi_H,g,\zeta,v_0;q,m,k_0) S ( B , π H , g , ζ , v 0 ; q , m , k 0 ) . Selection with thresholds c = c ( X ) c=c(X) c = c ( X ) ; a unit is null iff Y ≤ c Y\le c Y ≤ c . The covariate X X X is either light , with a latent u ∼ U ( 0 , 1 ) u\sim U(0,1) u ∼ U ( 0 , 1 ) (P P P -probability 1 − π H 1-\pi_H 1 − π H , weight w = 1 w=1 w = 1 ), or the heavy atom (P P P -probability π H \pi_H π H , weight w = B w=B w = B ). Y = u Y=u Y = u for light units and Y = g Y=g Y = g for the heavy atom, and the score is V = Y V=Y V = Y . The thresholds are:
heavy: c = g = Y c=g=Y c = g = Y (null);
light with u ≤ 1 − ζ u\le1-\zeta u ≤ 1 − ζ : c = u = Y c=u=Y c = u = Y (null);
light with u > 1 − ζ u>1-\zeta u > 1 − ζ : c = v 0 < Y c=v_0<Y c = v 0 < Y (non-null; the decoys).
Write w ˉ : = E P w = 1 − π H + B π H \bar{w}:=\mathbb{E}_P w=1-\pi_H+B\pi_H w ˉ := E P w = 1 − π H + B π H ; γ : = ( 1 − π H ) / w ˉ \gamma:=(1-\pi_H)/\bar{w} γ := ( 1 − π H ) / w ˉ , the Q Q Q -density of light units in u u u ; Q H : = B π H / w ˉ Q_H:=B\pi_H/\bar{w} Q H := B π H / w ˉ ; and Q D : = γ ζ Q_D:=\gamma\zeta Q D := γ ζ . The law P P P does not depend on n n n .
Conditions. Throughout, 0 < π H < 1 0<\pi_H<1 0 < π H < 1 , B ≥ 1 B\ge1 B ≥ 1 , m ≥ 2 m\ge2 m ≥ 2 and 0 < q < 1 0<q<1 0 < q < 1 . (The formulas below divide by π H 2 \pi_H^2 π H 2 , and the bound W ≥ n W\ge n W ≥ n used in the proof needs B ≥ 1 B\ge1 B ≥ 1 .)
(F1) 0 < g < v 0 < 1 − ζ < 1 0<g<v_0<1-\zeta<1 0 < g < v 0 < 1 − ζ < 1 .
(F2) Zero offset: p ˉ D : = ( ( 1 − π H ) v 0 + B π H ) / w ˉ = t k 0 \bar{p}_D:=((1-\pi_H)v_0+B\pi_H)/\bar{w}=t_{k_0} p ˉ D := (( 1 − π H ) v 0 + B π H ) / w ˉ = t k 0 for some k 0 ∈ { 1 , … , m } k_0\in\{1,\ldots,m\} k 0 ∈ { 1 , … , m } .
(F3) Decoys above q: ( ( 1 − π H ) ( 1 − ζ ) + B π H ) / w ˉ > q ((1-\pi_H)(1-\zeta)+B\pi_H)/\bar{w}>q (( 1 − π H ) ( 1 − ζ ) + B π H ) / w ˉ > q . This also keeps every light level u k u_k u k (defined below) below 1 − ζ 1-\zeta 1 − ζ .
(F4) Genericity: no t k t_k t k equals b H : = ( 1 − π H ) g / w ˉ b_H:=(1-\pi_H)g/\bar{w} b H := ( 1 − π H ) g / w ˉ or T H : = ( ( 1 − π H ) g + B π H ) / w ˉ T_H:=((1-\pi_H)g+B\pi_H)/\bar{w} T H := (( 1 − π H ) g + B π H ) / w ˉ .
By (F1), p ˉ D > T H \bar{p}_D>T_H p ˉ D > T H .
Population quantities. For G = ( G 1 , … , G m ) ∈ [ 0 , 1 ] m G = (G_1,\ldots,G_m) \in[0,1]^m G = ( G 1 , … , G m ) ∈ [ 0 , 1 ] m , let Π k ( G ) : = P ( R 1 = k ) \Pi_k(G) := \mathbb{P}(R_1 = k) Π k ( G ) := P ( R 1 = k ) when the m − 1 m-1 m − 1 other p-values are i.i.d. with CDF values G G G on the grid ( t 1 , … , t m ) (t_1,\ldots,t_m) ( t 1 , … , t m ) . The others’ CDF on the grid when the decoys are counted from t k 0 + 1 t_{k_0+1} t k 0 + 1 is G ˉ j 0 : = t j + Q D 1 { j > k 0 } \bar{G}^{0}_j := t_j + Q_D \mathbf{1}\{j > k_0\} G ˉ j 0 := t j + Q D 1 { j > k 0 } ; when they are counted from t k 0 t_{k_0} t k 0 it is G ˉ 1 : = G ˉ 0 + Q D e k 0 \bar{G}^{1} := \bar{G}^{0} + Q_D e_{k_0} G ˉ 1 := G ˉ 0 + Q D e k 0 , where e k 0 e_{k_0} e k 0 is the k 0 k_0 k 0 -th unit vector. The jump of r k r_k r k at the switch is
Δ r k : = Π k ( G ˉ 1 ) − Π k ( G ˉ 0 ) . \Delta r_k := \Pi_k(\bar{G}^{1}) - \Pi_k(\bar{G}^{0}). Δ r k := Π k ( G ˉ 1 ) − Π k ( G ˉ 0 ) .
Gaussian limit. Let e ∼ N ( 0 , π H ( 1 − π H ) ) e \sim\mathcal{N}(0,\pi_H(1-\pi_H)) e ∼ N ( 0 , π H ( 1 − π H )) , and let b b b , independent of e e e , be a Brownian bridge divided by 1 − π H \sqrt{1-\pi_H} 1 − π H :
Cov ( b ( x ) , b ( y ) ) = min ( x , y ) − x y 1 − π H . \operatorname{Cov}(b(x),b(y)) = \frac{\min(x,y)-xy}{1-\pi_H}. Cov ( b ( x ) , b ( y )) = 1 − π H min ( x , y ) − x y .
Define
η D : = ( B − v 0 − t k 0 ( B − 1 ) ) e + ( 1 − π H ) b ( v 0 ) w ˉ , (5) \eta_D := \frac{(B-v_0-t_{k_0}(B-1))e+(1-\pi_H)b(v_0)}{\bar{w}}, \tag*{(5)} η D := w ˉ ( B − v 0 − t k 0 ( B − 1 )) e + ( 1 − π H ) b ( v 0 ) , (5)
and define ξ k \xi_k ξ k by the region of t k t_k t k :
ξ k : = γ [ ( t k ( B − 1 ) + u k ) e 1 − π H − b ( u k ) ] ; \xi_k := \gamma\left[\frac{(t_k(B-1)+u_k)e}{1-\pi_H}-b(u_k)\right]; ξ k := γ [ 1 − π H ( t k ( B − 1 ) + u k ) e − b ( u k ) ] ;
ξ k : = Q H [ ( t k ( B − 1 ) + g ) e − ( 1 − π H ) b ( g ) ] B π H − Q H N k e B π H 2 ; \xi_k := \frac{Q_H\left[(t_k(B-1)+g)e-(1-\pi_H)b(g)\right]}{B\pi_H}-\frac{Q_HN_ke}{B\pi_H^2}; ξ k := B π H Q H [ ( t k ( B − 1 ) + g ) e − ( 1 − π H ) b ( g ) ] − B π H 2 Q H N k e ;
ξ k : = γ [ ( t k ( B − 1 ) + u k − B ) e 1 − π H − b ( u k ) ] . \xi_k := \gamma\left[\frac{(t_k(B-1)+u_k-B)e}{1-\pi_H}-b(u_k)\right]. ξ k := γ [ 1 − π H ( t k ( B − 1 ) + u k − B ) e − b ( u k ) ] .
Theorem 23 (Precise form of Theorem 12). In the setting S ( B , π H , g , ζ , v 0 ; q , m , k 0 ) S(B,\pi_H,g,\zeta,v_0;q,m,k_0) S ( B , π H , g , ζ , v 0 ; q , m , k 0 ) , under ( F 1 ) (F1) ( F 1 ) –( F 4 ) (F4) ( F 4 ) , as n → ∞ n \to\infty n → ∞ ,
n ( F D R n − q ) ⟶ c 0 : = − m ∑ k = 1 m 1 k Δ r k Cov ( ξ k , η D ) 2 π sd ( η D ) . (6) \sqrt{n}\,(\mathrm{FDR}_n-q)\longrightarrow c_0:=-m\sum_{k=1}^{m}\frac{1}{k}\frac{\Delta r_k\operatorname{Cov}(\xi_k,\eta_D)}{\sqrt{2\pi}\operatorname{sd}(\eta_D)}. \tag*{(6)} n ( FDR n − q ) ⟶ c 0 := − m k = 1 ∑ m k 1 2 π sd ( η D ) Δ r k Cov ( ξ k , η D ) . (6) In particular , if c 0 > 0 c_0>0 c 0 > 0 , then F D R n ≥ q + c 0 / ( 2 n ) \mathrm{FDR}_n \ge q+c_0/(2\sqrt{n}) FDR n ≥ q + c 0 / ( 2 n ) for every n ≥ n 0 n\ge n_0 n ≥ n 0 , for some n 0 n_0 n 0 .
The threshold n 0 n_0 n 0 and the o ( n − 1 / 2 ) o(n^{-1/2}) o ( n − 1/2 ) term are not explicit: the proof uses the central limit theorem and a uniform-integrability step. An explicit-rate version would need Berry–Esseen-type bounds for the multinomial counts and for the empirical quantile; these are not given here.
Proof of Theorem 12 (Theorem 23). Let n → ∞ n \to\infty n → ∞ with P P P fixed. C C C , c c c , c ′ c' c ′ denote positive constants that depend on P P P , q q q , m m m but not on n n n .
Step 1 (identity). Every null unit has c = Y c=Y c = Y , so it is an oracle null, and Lemma 2 gives
F D R n − q = m ∑ k = 1 m 1 k Cov ( A k , r k ) − m ∑ k = 1 m 1 k ε k E r k . (7) \mathrm{FDR}_n-q=m\sum_{k=1}^{m}\frac{1}{k}\operatorname{Cov}(A_k,r_k)-m\sum_{k=1}^{m}\frac{1}{k}\varepsilon_k\mathbb{E}r_k. \tag*{(7)} FDR n − q = m k = 1 ∑ m k 1 Cov ( A k , r k ) − m k = 1 ∑ m k 1 ε k E r k . (7)
A decoy’s oracle p-value is at least S ( 1 − ζ ) / ( W + 1 ) S(1-\zeta)/(W+1) S ( 1 − ζ ) / ( W + 1 ) , where S ( x ) S(x) S ( x ) is the calibration weight below x x x and W W W the total calibration weight. By (F3) and Hoeffding’s inequality [9 ] , ε k ≤ e − c n \varepsilon_k\le e^{-cn} ε k ≤ e − c n for some c > 0 c>0 c > 0 .
Step 2 (exact formulas given D D D ). Let N N N be the number of heavy calibration points, L = n − N L=n-N L = n − N , G ^ \widehat{G} G the empirical CDF of the L L L light scores, and W = L + B N W=L+BN W = L + B N . Eq. (4) of Jin & Candès [12 ] gives:
light null with latent u u u : p = ( L G ^ ( u ) + B N 1 { g < u } + U ) / ( W + 1 ) p = (L\widehat{G}(u) + BN1\{g < u\} + U)/(W + 1) p = ( L G ( u ) + B N 1 { g < u } + U ) / ( W + 1 ) ;
heavy test unit: p = ( L G ^ ( g ) + U ( B N + B ) ) / ( W + B ) p = (L\widehat{G}(g) + U(BN + B))/(W + B) p = ( L G ( g ) + U ( B N + B )) / ( W + B ) ;
decoy: p D = ( L G ^ ( v 0 ) + B N + U ) / ( W + 1 ) p_D = (L\widehat{G}(v_0) + BN + U)/(W + 1) p D = ( L G ( v 0 ) + B N + U ) / ( W + 1 ) . There are almost surely no ties at v 0 v_0 v 0 .
So, with clip ( x , 0 , 1 ) : = min { 1 , max { 0 , x } } \operatorname{clip}(x,0,1) := \min\{1,\max\{0,x\}\} clip ( x , 0 , 1 ) := min { 1 , max { 0 , x }} ,
A k = γ ∫ 0 1 − ζ clip ( t k ( W + 1 ) − L G ^ ( u ) − B N 1 { g < u } , 0 , 1 ) d u + Q H clip ( t k ( W + B ) − L G ^ ( g ) B N + B , 0 , 1 ) . A_k = \gamma\int_0^{1-\zeta} \operatorname{clip}\left(t_k(W+1)-L\widehat{G}(u)-BN1\{g<u\},0,1\right)\,du + Q_H\operatorname{clip}\left(\frac{t_k(W+B)-L\widehat{G}(g)}{BN+B},0,1\right). A k = γ ∫ 0 1 − ζ clip ( t k ( W + 1 ) − L G ( u ) − B N 1 { g < u } , 0 , 1 ) d u + Q H clip ( B N + B t k ( W + B ) − L G ( g ) , 0 , 1 ) .
The decoys add Q D d k Q_Dd_k Q D d k to the others’ CDF at t k t_k t k , where d k : = clip ( t k ( W + 1 ) − L G ^ ( v 0 ) − B N , 0 , 1 ) d_k := \operatorname{clip}(t_k(W+1)-L\widehat{G}(v_0)-BN,0,1) d k := clip ( t k ( W + 1 ) − L G ( v 0 ) − B N , 0 , 1 ) . So r k = Π k ( A + Q D d ) r_k = \Pi_k(A+Q_Dd) r k = Π k ( A + Q D d ) , with A = ( A 1 , … , A m ) A=(A_1,\ldots,A_m) A = ( A 1 , … , A m ) and d = ( d 1 , … , d m ) d=(d_1,\ldots,d_m) d = ( d 1 , … , d m ) , and Π k \Pi_k Π k is a polynomial with bounded derivatives on [ 0 , 1 ] m [0,1]^m [ 0 , 1 ] m .
Step 3 (good event). Let x ( 1 ) < ⋯ < x ( L ) x_{(1)} < \cdots< x_{(L)} x ( 1 ) < ⋯ < x ( L ) be the light calibration scores in increasing order. Let E n E_n E n be the intersection of the following events:
∣ N / n − π H ∣ ≤ n − 1 / 2 log n |N/n-\pi_H| \le n^{-1/2}\log n ∣ N / n − π H ∣ ≤ n − 1/2 log n ;
L ≥ n ( 1 − π H ) / 2 L \ge n(1-\pi_H)/2 L ≥ n ( 1 − π H ) /2 ;
sup x ∣ G ^ ( x ) − x ∣ ≤ n − 1 / 2 log n \sup_x|\widehat{G}(x)-x| \le n^{-1/2}\log n sup x ∣ G ( x ) − x ∣ ≤ n − 1/2 log n ;
the largest spacing between consecutive light scores, including the gaps at 0 and 1, is at most log 2 n / n \log^2 n/n log 2 n / n .
By Hoeffding’s inequality [9 ] , and by the Dvoretzky–Kiefer–Wolfowitz inequality [4 ] with Massart’s constant [17 ] applied conditionally on N N N , the first three events fail with probability at most C e − c log 2 n Ce^{-c\log^2 n} C e − c l o g 2 n . For the spacing: a fixed interval of length log 2 n / ( 2 n ) \log^2 n/(2n) log 2 n / ( 2 n ) is empty with probability ( 1 − log 2 n / ( 2 n ) ) L ≤ e − c ′ log 2 n (1-\log^2 n/(2n))^L \le e^{-c'\log^2 n} ( 1 − log 2 n / ( 2 n ) ) L ≤ e − c ′ l o g 2 n . If some spacing exceeds log 2 n / n \log^2 n/n log 2 n / n , then one of about 2 n / log 2 n 2n/\log^2 n 2 n / log 2 n fixed such intervals is empty. So P ( E n c ) ≤ C n e − c log 2 n \mathbb{P}(E_n^c) \le Cn e^{-c\log^2 n} P ( E n c ) ≤ C n e − c l o g 2 n , which is still super-polynomially small. On E n E_n E n , for n ≥ n 1 n \ge n_1 n ≥ n 1 :
(i) d j = 0 d_j=0 d j = 0 for j < k 0 j<k_0 j < k 0 and d j = 1 d_j=1 d j = 1 for j > k 0 j>k_0 j > k 0 , because p ‾ D = t k 0 \overline{p}_D=t_{k_0} p D = t k 0 sits strictly between t k 0 − 1 t_{k_0-1} t k 0 − 1 and t k 0 + 1 t_{k_0+1} t k 0 + 1 ;
(ii) each t k t_k t k stays in its population region, by (F4);
(iii) A k A_k A k is within C log 2 n / n C\log^2 n/n C log 2 n / n of the following expression. Below g g g it is γ x ( j ) \gamma x_{(j)} γ x ( j ) . In the heavy region it is γ g + Q H ( t k W − L G ^ ( g ) ) / ( B N ) \gamma g+Q_H(t_kW-L\widehat{G}(g))/(BN) γ g + Q H ( t k W − L G ( g )) / ( B N ) . Above g g g it is γ x ( j ′ ) + Q H \gamma x_{(j')}+Q_H γ x ( j ′ ) + Q H ; here γ x ( j ′ ) \gamma x_{(j')} γ x ( j ′ ) already includes the light mass below g g g . The indices j j j and j ′ j' j ′ are the order-statistic indices fixed by t k t_k t k , W W W and N N N . The exact light part is γ [ x ( j ) + f ( x ( j + 1 ) − x ( j ) ) ] \gamma[x_{(j)}+f(x_{(j+1)}-x_{(j)})] γ [ x ( j ) + f ( x ( j + 1 ) − x ( j ) )] for some f ∈ [ 0 , 1 ] f\in[0,1] f ∈ [ 0 , 1 ] , so the error includes one spacing: the bound C log 2 n / n C\log^2 n/n C log 2 n / n absorbs one spacing, the + 1 +1 + 1 and + B +B + B terms in the denominators, the U U U -smoothing, and the integer parts.
Step 4 does not use (iii). Step 5 needs only the sandwich γ x ( j ) ≤ ( light part ) ≤ γ x ( j + 1 ) \gamma x_{(j)}\le(\text{light part})\le\gamma x_{(j+1)} γ x ( j ) ≤ ( light part ) ≤ γ x ( j + 1 ) , and that ∣ x ( j ) − u k ∣ ≤ sup ∣ G ^ − id ∣ + ∣ j / L − u k ∣ + 1 / L |x_{(j)}-u_k|\le\sup|\widehat{G}-\operatorname{id}|+|j/L-u_k|+1/L ∣ x ( j ) − u k ∣ ≤ sup ∣ G − id ∣ + ∣ j / L − u k ∣ + 1/ L is sub-Gaussian at scale n − 1 / 2 n^{-1/2} n − 1/2 . Step 6 needs the error to be o ( n − 1 / 2 ) o(n^{-1/2}) o ( n − 1/2 ) , which log 2 n / n \log^2 n/n log 2 n / n is. Off E n E_n E n , every quantity lies in [ 0 , 1 ] [0,1] [ 0 , 1 ] .
Step 4 (smooth part plus switch). Put
δ D : = L G ^ ( v 0 ) + B N W + 1 − t k 0 . \delta_D := \frac{L\widehat{G}(v_0)+BN}{W+1}-t_{k_0}. δ D := W + 1 L G ( v 0 ) + B N − t k 0 .
Then d k 0 = clip ( − ( W + 1 ) δ D , 0 , 1 ) d_{k_0}=\operatorname{clip}(-(W+1)\delta_D,0,1) d k 0 = clip ( − ( W + 1 ) δ D , 0 , 1 ) . Let G 0 : = A + Q D 1 { ⋅ > k 0 } G^0:=A+Q_D1\{\cdot>k_0\} G 0 := A + Q D 1 { ⋅ > k 0 } and G 1 : = G 0 + Q D e k 0 G^1:=G^0+Q_De_{k_0} G 1 := G 0 + Q D e k 0 ; by Step 3(i), A + Q D d = G 0 + d k 0 Q D e k 0 A+Q_Dd=G^0+d_{k_0}Q_De_{k_0} A + Q D d = G 0 + d k 0 Q D e k 0 on E n E_n E n . On E n E_n E n :
Π k \Pi_k Π k is a polynomial in its k 0 k_0 k 0 -th coordinate. So for d ∈ [ 0 , 1 ] d\in[0,1] d ∈ [ 0 , 1 ] ,
Π k ( G 0 + d Q D e k 0 ) = ( 1 − d ) Π k ( G 0 ) + d Π k ( G 1 ) + O ( d ( 1 − d ) ) . \Pi_k(G^0+dQ_De_{k_0})=(1-d)\Pi_k(G^0)+d\Pi_k(G^1)+O(d(1-d)). Π k ( G 0 + d Q D e k 0 ) = ( 1 − d ) Π k ( G 0 ) + d Π k ( G 1 ) + O ( d ( 1 − d )) .
r k = Π k ( G 0 ( A ) ) + 1 { δ D ≤ 0 } Δ k ( A ) + O ( 1 { ∣ δ D ∣ ≤ 2 / n } ) , (8) r_k=\Pi_k(G^0(A))+1\{\delta_D\le0\}\Delta_k(A)+O(1\{|\delta_D|\le2/n\}), \tag*{(8)} r k = Π k ( G 0 ( A )) + 1 { δ D ≤ 0 } Δ k ( A ) + O ( 1 { ∣ δ D ∣ ≤ 2/ n }) , (8)
where Δ k ( A ) : = Π k ( G 1 ( A ) ) − Π k ( G 0 ( A ) ) \Delta_k(A):=\Pi_k(G^1(A))-\Pi_k(G^0(A)) Δ k ( A ) := Π k ( G 1 ( A )) − Π k ( G 0 ( A )) is a polynomial in A A A with Δ k ( A ˉ ) = Δ r k \Delta_k(\bar A)=\Delta r_k Δ k ( A ˉ ) = Δ r k , and A ˉ : = ( t 1 , … , t m ) \bar A:=(t_1,\ldots,t_m) A ˉ := ( t 1 , … , t m ) .
Step 5 (covariances). Write Δ A k : = A k − E A k \Delta A_k:=A_k-\mathbb{E}A_k Δ A k := A k − E A k . Then E ∣ Δ A k ∣ p ≤ C p n − p / 2 \mathbb{E}|\Delta A_k|^p\le C_p n^{-p/2} E ∣Δ A k ∣ p ≤ C p n − p /2 for p ≤ 4 p\le4 p ≤ 4 : on E n E_n E n , A k A_k A k is Lipschitz in N / n N/n N / n , G ^ ( g ) \widehat G(g) G ( g ) and one light order statistic, each with sub-Gaussian deviations at scale n − 1 / 2 n^{-1/2} n − 1/2 ; off E n E_n E n , A k A_k A k is bounded. Also ∣ E A − A ˉ ∣ ≤ e − c n |\mathbb{E}A-\bar A|\le e^{-c n} ∣ E A − A ˉ ∣ ≤ e − c n by Step 1. Hence:
Cov ( A k , Π k ( G 0 ( A ) ) ) = ∑ i ∂ i Π k Cov ( A k , A i ) + O ( E [ ∣ Δ A k ∣ ∣ A − A ˉ ∣ 2 ] ) = O ( 1 / n ) \operatorname{Cov}(A_k,\Pi_k(G^0(A)))=\sum_i\partial_i\Pi_k\operatorname{Cov}(A_k,A_i)+O(\mathbb{E}[|\Delta A_k||A-\bar A|^2])=O(1/n) Cov ( A k , Π k ( G 0 ( A ))) = ∑ i ∂ i Π k Cov ( A k , A i ) + O ( E [ ∣Δ A k ∣∣ A − A ˉ ∣ 2 ]) = O ( 1/ n ) , with ∂ i Π k \partial_i\Pi_k ∂ i Π k taken at G 0 ( A ˉ ) G^0(\bar A) G 0 ( A ˉ ) ;
∣ Cov ( A k , 1 { δ D ≤ 0 } ( Δ k ( A ) − Δ r k ) ) ∣ ≤ C E [ ∣ Δ A k ∣ ∣ A − A ˉ ∣ ] = O ( 1 / n ) |\operatorname{Cov}(A_k,1\{\delta_D\le0\}(\Delta_k(A)-\Delta r_k))|\le C\mathbb{E}[|\Delta A_k||A-\bar A|]=O(1/n) ∣ Cov ( A k , 1 { δ D ≤ 0 } ( Δ k ( A ) − Δ r k )) ∣ ≤ C E [ ∣Δ A k ∣∣ A − A ˉ ∣ ] = O ( 1/ n ) ;
∣ Cov ( A k , O ( 1 { ∣ δ D ∣ ≤ 2 / n } ) ) ∣ ≤ C ( E Δ A k 2 ) 1 / 2 P ( ∣ δ D ∣ ≤ 2 / n ) 1 / 2 = O ( n − 1 / 2 ) ⋅ o ( 1 ) |\operatorname{Cov}(A_k,O(1\{|\delta_D|\le2/n\}))|\le C(\mathbb{E}\Delta A_k^2)^{1/2}\mathbb{P}(|\delta_D|\le2/n)^{1/2}=O(n^{-1/2})\cdot o(1) ∣ Cov ( A k , O ( 1 { ∣ δ D ∣ ≤ 2/ n })) ∣ ≤ C ( E Δ A k 2 ) 1/2 P ( ∣ δ D ∣ ≤ 2/ n ) 1/2 = O ( n − 1/2 ) ⋅ o ( 1 ) . Here P ( ∣ n δ D ∣ ≤ 2 / n ) → 0 \mathbb{P}(|\sqrt n\delta_D|\le2/\sqrt n)\to0 P ( ∣ n δ D ∣ ≤ 2/ n ) → 0 , because n δ D \sqrt n\delta_D n δ D converges to the continuous law of η D \eta_D η D (Step 6);
the contribution of E n c E_n^c E n c is O ( n e − c log 2 n ) O(ne^{-c\log^2 n}) O ( n e − c l o g 2 n ) , still super-polynomially small.
So
Cov ( A k , r k ) = Δ r k E [ Δ A k 1 { δ D ≤ 0 } ] + o ( n − 1 / 2 ) . (9) \operatorname{Cov}(A_k,r_k)=\Delta r_k\mathbb{E}[\Delta A_k1\{\delta_D\le0\}]+o(n^{-1/2}). \tag*{(9)} Cov ( A k , r k ) = Δ r k E [ Δ A k 1 { δ D ≤ 0 }] + o ( n − 1/2 ) . (9)
Step 6 (joint CLT). We show
( n Δ A 1 , … , n Δ A m , n δ D ) ⇒ ( ξ 1 , … , ξ m , η D ) . (10) (\sqrt n\Delta A_1,\ldots,\sqrt n\Delta A_m,\sqrt n\delta_D)\Rightarrow(\xi_1,\ldots,\xi_m,\eta_D). \tag*{(10)} ( n Δ A 1 , … , n Δ A m , n δ D ) ⇒ ( ξ 1 , … , ξ m , η D ) . (10)
Up to O ( log 2 n / n ) O(\log^2 n/n) O ( log 2 n / n ) on E n E_n E n , each coordinate is a smooth function of: N / n N/n N / n ; G ^ \widehat G G at the fixed points g g g and v 0 v_0 v 0 ; and at most one light order statistic x ( j ) x_{(j)} x ( j ) per k k k , with j / L → u k j/L\to u_k j / L → u k . The vector formed by N N N and the light counts in a fixed finite partition is multinomial. Given N N N , the light process satisfies L ( G ^ − id ) → \sqrt L(\widehat G-\operatorname{id})\to L ( G − id ) → Brownian bridge at fixed points, whatever the value of N N N . This gives the limits e e e and b b b , and they are independent. Both statements are the multivariate central limit theorem ([21 Ch. 2] ): for the multinomial counts, and, at finitely many fixed points, for the i.i.d. indicator vectors of the light scores, whose limit is the Brownian bridge at those points ([21 Ch. 19] ).
Order statistics. The claim is joint, and j / L j/L j / L is random through N N N .
The index. Below g g g , j : = ⌊ t k ( W + 1 ) ⌋ j:=\lfloor t_k(W+1)\rfloor j := ⌊ t k ( W + 1 )⌋ ; above g g g , j : = ⌊ t k ( W + 1 ) − B N ⌋ j:=\lfloor t_k(W+1)-BN\rfloor j := ⌊ t k ( W + 1 ) − B N ⌋ . In both cases j / L = h k ( N / n ) + O ( 1 / n ) j/L=h_k(N/n)+O(1/n) j / L = h k ( N / n ) + O ( 1/ n ) for a smooth function h k h_k h k with h k ( π H ) = u k h_k(\pi_H)=u_k h k ( π H ) = u k .
Call a value of N N N admissible if it satisfies E n E_n E n ’s conditions on N N N , and condition on an admissible N N N . Then L L L and j j j are fixed, and ϑ n : = j / L \vartheta_n:=j/L ϑ n := j / L is within C n − 1 / 2 log n Cn^{-1/2}\log n C n − 1/2 log n of u k u_k u k .
Put X n : = n ( x ( j ) − ϑ n ) X_n:=\sqrt n(x_{(j)}-\vartheta_n) X n := n ( x ( j ) − ϑ n ) and Y n : = n ( G ^ ( ϑ n ) − ϑ n ) Y_n:=\sqrt n(\widehat G(\vartheta_n)-\vartheta_n) Y n := n ( G ( ϑ n ) − ϑ n ) . For each fixed z z z ,
{ X n ≤ z } = { L G ^ ( ϑ n + z / n ) ≥ j } = { n ( G ^ ( ϑ n + z / n ) − ϑ n ) ≥ 0 } . \{X_n\le z\}=\{L\widehat G(\vartheta_n+z/\sqrt n)\ge j\}=\{\sqrt n(\widehat G(\vartheta_n+z/\sqrt n)-\vartheta_n)\ge0\}. { X n ≤ z } = { L G ( ϑ n + z / n ) ≥ j } = { n ( G ( ϑ n + z / n ) − ϑ n ) ≥ 0 } .
This holds for any sample, since G ^ \widehat G G is right-continuous.
Given N N N , n [ G ^ ( ϑ n + z / n ) − G ^ ( ϑ n ) ] \sqrt n[\widehat G(\vartheta_n+z/\sqrt n)-\widehat G(\vartheta_n)] n [ G ( ϑ n + z / n ) − G ( ϑ n )] has mean exactly z z z and variance at most n ∣ z ∣ / L = O ( n − 1 / 2 ) \sqrt n|z|/L=O(n^{-1/2}) n ∣ z ∣/ L = O ( n − 1/2 ) . So, by Chebyshev’s inequality, for every ϵ > 0 \epsilon>0 ϵ > 0 both P ( X n ≤ z , Y n < − z − ϵ ∣ N ) \mathbb{P}(X_n\le z,Y_n<-z-\epsilon\mid N) P ( X n ≤ z , Y n < − z − ϵ ∣ N ) and P ( X n > z , Y n > − z + ϵ ∣ N ) \mathbb{P}(X_n>z,Y_n>-z+\epsilon\mid N) P ( X n > z , Y n > − z + ϵ ∣ N ) tend to 0 0 0 , uniformly over admissible N N N , for z z z in a compact range.
ϑ n \vartheta_n ϑ n is tight uniformly in N N N . Var ( ϑ n ∣ N ) ≤ n / ( 4 L ) ≤ 1 / ( 2 ( 1 − π H ) ) \operatorname{Var}(\vartheta_n\mid N)\le n/(4L)\le1/(2(1-\pi_H)) Var ( ϑ n ∣ N ) ≤ n / ( 4 L ) ≤ 1/ ( 2 ( 1 − π H )) . So for any α > 0 \alpha>0 α > 0 there is z ˉ \bar z z ˉ with P ( ∣ Y n ∣ ≥ z ˉ ∣ N ) ≤ α \mathbb{P}(|Y_n|\ge\bar z\mid N)\le\alpha P ( ∣ Y n ∣ ≥ z ˉ ∣ N ) ≤ α . Take an ϵ \epsilon ϵ -mesh grid of z z z in [ − z ˉ , z ˉ + 2 ] [-\bar z, \bar z+2] [ − z ˉ , z ˉ + 2 ] , with ϵ ∈ ( 0 , 1 ) \epsilon\in(0,1) ϵ ∈ ( 0 , 1 ) .
If X n + Y n > 2 ϵ X_n+Y_n>2\epsilon X n + Y n > 2 ϵ and ∣ Y n ∣ ≤ z ˉ |Y_n|\le\bar z ∣ Y n ∣ ≤ z ˉ , pick a grid point z ∈ ( − Y n + ϵ , − Y n + 2 ϵ ] z\in(-Y_n+\epsilon,-Y_n+2\epsilon] z ∈ ( − Y n + ϵ , − Y n + 2 ϵ ] . Then X n > z X_n>z X n > z and Y n > − z + ϵ Y_n>-z+\epsilon Y n > − z + ϵ .
If X n + Y n < − 2 ϵ X_n+Y_n<-2\epsilon X n + Y n < − 2 ϵ (and ∣ Y n ∣ ≤ z ˉ |Y_n|\le\bar z ∣ Y n ∣ ≤ z ˉ ), pick z ∈ [ − Y n − 2 ϵ , − Y n − ϵ ) z\in[-Y_n-2\epsilon,-Y_n-\epsilon) z ∈ [ − Y n − 2 ϵ , − Y n − ϵ ) . Then X n < z X_n<z X n < z and Y n < − z − ϵ Y_n<-z-\epsilon Y n < − z − ϵ .
This is the lemma in Ghosh [6 ] 's proof of the Bahadur representation, written out.
Also, given N N N , n [ ( G ^ ( ϑ n ) − ϑ n ) − ( G ^ ( u k ) − u k ) ] \sqrt{n}\left[(\widehat{G}(\vartheta_n)-\vartheta_n)-(\widehat{G}(u_k)-u_k)\right] n [ ( G ( ϑ n ) − ϑ n ) − ( G ( u k ) − u k ) ] has variance at most n ∣ ϑ n − u k ∣ / L → 0 n\lvert\vartheta_n-u_k\rvert/L\to0 n ∣ ϑ n − u k ∣ / L → 0 . Hence
n ( x ( j ) − ϑ n ) = − n ( G ^ ( u k ) − u k ) + o P ( 1 ) , \sqrt{n}(x_{(j)}-\vartheta_n)=-\sqrt{n}\left(\widehat{G}(u_k)-u_k\right)+o_P(1), n ( x ( j ) − ϑ n ) = − n ( G ( u k ) − u k ) + o P ( 1 ) ,
where u k u_k u k is a fixed point.
Finally, n ( ϑ n − u k ) = h k ′ ( π H ) n ( N / n − π H ) + o ( 1 ) \sqrt{n}(\vartheta_n-u_k)=h_k'(\pi_H)\sqrt{n}(N/n-\pi_H)+o(1) n ( ϑ n − u k ) = h k ′ ( π H ) n ( N / n − π H ) + o ( 1 ) , uniformly on E n E_n E n ; at this scale it is linear. Here h k ′ ( π H ) = t k B / ( 1 − π H ) 2 h_k'(\pi_H)=t_kB/(1-\pi_H)^2 h k ′ ( π H ) = t k B / ( 1 − π H ) 2 below g g g and ( t k − 1 ) B / ( 1 − π H ) 2 (t_k-1)B/(1-\pi_H)^2 ( t k − 1 ) B / ( 1 − π H ) 2 above g g g . These match the coefficients of e e e in ξ k \xi_k ξ k .
So every coordinate of (9) is, up to o P ( 1 ) o_P(1) o P ( 1 ) , a smooth function of n ( N / n − π H ) \sqrt{n}(N/n-\pi_H) n ( N / n − π H ) and of n ( G ^ − i d ) \sqrt{n}(\widehat{G}-\mathrm{id}) n ( G − id ) at the fixed points g g g , v 0 v_0 v 0 and u k u_k u k . The joint CLT for the multinomial counts of (heavy, light in the fixed partition cells) gives joint convergence.
The delta method gives the linear forms ξ k \xi_k ξ k and η D \eta_D η D defined above. As a numerical check, the covariances they imply agree with simulated covariances at n = 10 7 n=10^7 n = 1 0 7 on eight random parameter sets, within 1.5 standard errors. For the delta method see van der Vaart [21 ] .
Step 7 (the kink). sup n E [ n Δ A k 2 ] < ∞ \sup_n\mathbb{E}[n\Delta A_k^2]<\infty sup n E [ n Δ A k 2 ] < ∞ gives uniform integrability. Also P ( η D = 0 ) = 0 \mathbb{P}(\eta_D=0)=0 P ( η D = 0 ) = 0 , since sd ( η D ) > 0 \operatorname{sd}(\eta_D)>0 sd ( η D ) > 0 : the b ( v 0 ) b(v_0) b ( v 0 ) term alone has positive variance. So, by continuous mapping and convergence of expectations under uniform integrability (van der Vaart [21 ] ),
n E [ Δ A k 1 { δ D ≤ 0 } ] → E [ ξ k 1 { η D ≤ 0 } ] = − Cov ( ξ k , η D ) φ ( 0 ) sd ( η D ) , φ ( 0 ) = 1 2 π . \sqrt{n}\mathbb{E}\left[\Delta A_k\mathbf{1}\{\delta_D\leq0\}\right]\to\mathbb{E}\left[\xi_k\mathbf{1}\{\eta_D\leq0\}\right]=-\frac{\operatorname{Cov}(\xi_k,\eta_D)\varphi(0)}{\operatorname{sd}(\eta_D)},\qquad\varphi(0)=\frac{1}{\sqrt{2\pi}}. n E [ Δ A k 1 { δ D ≤ 0 } ] → E [ ξ k 1 { η D ≤ 0 } ] = − sd ( η D ) Cov ( ξ k , η D ) φ ( 0 ) , φ ( 0 ) = 2 π 1 .
The last equality holds for a centred Gaussian pair: write ξ k = ( Cov ( ξ k , η D ) / Var ( η D ) ) η D \xi_k=\left(\operatorname{Cov}(\xi_k,\eta_D)/\operatorname{Var}(\eta_D)\right)\eta_D ξ k = ( Cov ( ξ k , η D ) / Var ( η D ) ) η D plus a part independent of η D \eta_D η D .
Steps 1, 5 and 7 together give n ( F D R n − q ) → c 0 \sqrt{n}(\mathrm{FDR}_n-q)\to c_0 n ( FDR n − q ) → c 0 ; the bound for n ≥ n 0 n\geq n_0 n ≥ n 0 follows from the limit. □
Remark 24 (Two points of care). Step 4’s O ( d ( 1 − d ) ) O(d(1-d)) O ( d ( 1 − d )) bound needs Π k \Pi_k Π k to be a polynomial of bounded degree in one coordinate. That holds, with degree at most m − 1 m-1 m − 1 . Step 6 is the only place where the joint law matters. The covariance formulas for ξ k \xi_k ξ k and η D \eta_D η D come from it, and the exact-arithmetic evaluations of c 0 c_0 c 0 use exactly those formulas.
Remark 25 (Two checks on c 0 c_0 c 0 ). (i) Under (F2), u k 0 = v 0 u_{k_0}=v_0 u k 0 = v 0 (t k 0 = p ˉ D > T H t_{k_0}=\bar{p}_D>T_H t k 0 = p ˉ D > T H , so the third region applies), and therefore η D = − ξ k 0 \eta_D=-\xi_{k_0} η D = − ξ k 0 coefficient by coefficient (also confirmed numerically, to 10 − 14 10^{-14} 1 0 − 14 ). The k 0 k_0 k 0 -term of c 0 c_0 c 0 is therefore m Δ r k 0 sd ( η D ) / ( k 0 2 π ) m\,\Delta r_{k_0}\operatorname{sd}(\eta_D)/(k_0\sqrt{2\pi}) m Δ r k 0 sd ( η D ) / ( k 0 2 π ) , which is ≥ 0 \geq0 ≥ 0 , and > 0 >0 > 0 when k 0 ≥ 2 k_0\geq2 k 0 ≥ 2 . Indeed, couple the m − 1 m-1 m − 1 draws under G ~ 0 \widetilde{G}^{0} G 0 and G ~ 1 \widetilde{G}^{1} G 1 so that each decoy draw sits in band k 0 + 1 k_0+1 k 0 + 1 , respectively band k 0 k_0 k 0 , and all other draws agree. The counts at the grid points then agree except at index k 0 k_0 k 0 , where they are larger under G ~ 1 \widetilde{G}^{1} G 1 . So the two BH counts with a forced zero, R 0 R^0 R 0 and R 1 R^1 R 1 , agree unless R 0 < k 0 R^0<k_0 R 0 < k 0 and R 1 = k 0 R^1=k_0 R 1 = k 0 , and Δ r k 0 = P ( R 0 < k 0 , R 1 = k 0 ) ≥ 0 \Delta r_{k_0}=\mathbb{P}(R^0<k_0,R^1=k_0)\geq0 Δ r k 0 = P ( R 0 < k 0 , R 1 = k 0 ) ≥ 0 . For k 0 ≥ 2 k_0\geq2 k 0 ≥ 2 this is positive: if k 0 − 1 k_0-1 k 0 − 1 of the draws are decoys and the other m − k 0 m-k_0 m − k 0 lie above t m t_m t m (which has positive probability: Q D > 0 Q_D>0 Q D > 0 by (F1), and the mass above t m t_m t m is positive by (F3)), then R 0 = 1 R^0=1 R 0 = 1 and R 1 = k 0 R^1=k_0 R 1 = k 0 . For k 0 = 1 k_0=1 k 0 = 1 , R 0 = R 1 R^0=R^1 R 0 = R 1 always, so every Δ r k \Delta r_k Δ r k is 0 0 0 and c 0 = 0 c_0=0 c 0 = 0 .
(ii) At B = 1 B=1 B = 1 (no weighting), c 0 = 0 c_0=0 c 0 = 0 in every configuration checked (60 configurations; numerical). This is consistent with exact FDR control in the unweighted case (Jin & Candès, [11 ] ).
Remark 26 (Without the tuning (F2)). If p ˉ D \bar{p}_D p ˉ D is not a grid point, the switch is frozen on E n E_n E n , and n ( F D R n − q ) → 0 \sqrt{n}(\mathrm{FDR}_n-q)\to0 n ( FDR n − q ) → 0 . This is recorded with only that one-line reason, as an outline rather than a separate proof. So a fixed distribution shows the n \sqrt{n} n excess only when its decoy threshold sits exactly at a switching point. The worst case over distributions with weight ratio at most B B B still has the B / n \sqrt{B/n} B / n order, because the tuning is available to the adversary.
The values for the parameters of Example 10. Example 13 evaluates (5) at q = 0.1 q = 0.1 q = 0.1 , m = 8 m = 8 m = 8 , k 0 = 8 k_0 = 8 k 0 = 8 , g = 0.018418 g = 0.018418 g = 0.018418 , π H = 0.086271 / B \pi_H = 0.086271/B π H = 0.086271/ B and ζ = 0.91847 \zeta= 0.91847 ζ = 0.91847 , with v 0 v_0 v 0 from (F2) (v 0 = 0.021680 v_0 = 0.021680 v 0 = 0.021680 at B = 10 B = 10 B = 10 ); this is the structure of Example 10. For B ∈ { 2 , 5 , 10 , 100 } B \in\{2, 5, 10, 100\} B ∈ { 2 , 5 , 10 , 100 } , (F1)–(F4) hold and − m ∑ k Δ r k Cov ( ξ k , η D ) / k > 0 -m \sum_k \Delta r_k\operatorname{Cov}(\xi_k,\eta_D)/k > 0 − m ∑ k Δ r k Cov ( ξ k , η D ) / k > 0 in exact rational arithmetic, with an independent enumeration of the BH count; the values of c 0 c_0 c 0 themselves involve one square root in floating point (for example c 0 = 0.18806 c_0 = 0.18806 c 0 = 0.18806 at B = 10 B = 10 B = 10 , so FDR n ≈ 0.1 + 0.188 / n \operatorname{FDR}_n \approx0.1 + 0.188/\sqrt{n} FDR n ≈ 0.1 + 0.188/ n ). We have not proved that c 0 / ( q B ) c_0/(q\sqrt{B}) c 0 / ( q B ) tends to 0.6707, the Poisson-limit coefficient of Example 10, as B → ∞ B \to\infty B → ∞ . The evidence is numerical only: in floating point, c 0 / ( q B ) c_0/(q\sqrt{B}) c 0 / ( q B ) is 0.6699 at B = 10 3 B = 10^3 B = 1 0 3 and 0.6707 at B = 10 5 B = 10^5 B = 1 0 5 . Figure 2 compares n ( FDR n − q ) \sqrt{n}(\operatorname{FDR}_n - q) n ( FDR n − q ) at finite n n n with c 0 c_0 c 0 .
Figure 2. Theorem 12 at finite n n n : n ( F D R n − q ) \sqrt{n}(\mathrm{FDR}_n-q) n ( FDR n − q ) divided by the predicted limit c 0 c_0 c 0 , for one fixed distribution per B B B (the parameters of Example 10). Rao–Blackwellised simulation of the calibration data with E [ F D P ∣ D ] \mathbb{E}[\mathrm{FDP}\mid D] E [ FDP ∣ D ] computed exactly; bars are one standard error.
Proofs for the calibration-conditional upper bound
This appendix proves Proposition 3 , Theorem 6 and Proposition 7 . It does not use the envelope of § 7 .
Setting. We assume (A1) and (A3) of § 2 , together with the independence assumption of Jin & Candès [12 ] : the m m m test tuples ( X n + j , Y n + j , c n + j , U j ) (X_{n+j}, Y_{n+j}, c_{n+j}, U_j) ( X n + j , Y n + j , c n + j , U j ) are mutually independent and independent of the calibration data D : = ( Z 1 , … , Z n ) D := (Z_1, \ldots, Z_n) D := ( Z 1 , … , Z n ) , Z i = ( X i , Y i ) Z_i = (X_i, Y_i) Z i = ( X i , Y i ) . In selection, ( X n + j , Y n + j ) ∼ Q (X_{n+j}, Y_{n+j}) \sim Q ( X n + j , Y n + j ) ∼ Q , and c n + j c_{n+j} c n + j may be any function of X n + j X_{n+j} X n + j and of noise independent of everything else. In outlier detection, the null units (inliers) have ( X , Y ) ∼ Q (X, Y) \sim Q ( X , Y ) ∼ Q and the outliers have any law. The set of nulls may be fixed or random; if it is random, it must be independent of D D D , and the tuples must be independent given the null set.
Notation. V i : = V ( X i , Y i ) V_i := V(X_i, Y_i) V i := V ( X i , Y i ) , V ^ n + j : = V ( X n + j , c n + j ) \widehat{V}_{n+j} := V(X_{n+j}, c_{n+j}) V n + j := V ( X n + j , c n + j ) and w i : = w ( X i ) w_i := w(X_i) w i := w ( X i ) . For v ∈ R v \in\mathbb{R} v ∈ R let
W : = ∑ i ≤ n w i , S < ( v ) : = ∑ i ≤ n w i 1 { V i < v } , S = ( v ) : = ∑ i ≤ n w i 1 { V i = v } . W := \sum_{i \le n} w_i,\qquad S_{<}(v) := \sum_{i \le n} w_i\mathbf{1}\{V_i < v\},\qquad S_{=}(v) := \sum_{i \le n} w_i\mathbf{1}\{V_i = v\}. W := i ≤ n ∑ w i , S < ( v ) := i ≤ n ∑ w i 1 { V i < v } , S = ( v ) := i ≤ n ∑ w i 1 { V i = v } .
Eq. (4) of Jin & Candès [12 ] reads p j = ( S < ( V ^ n + j ) + U j ( w n + j + S = ( V ^ n + j ) ) ) / ( W + w n + j ) p_j = \bigl(S_{<}(\widehat{V}_{n+j}) + U_j(w_{n+j} + S_{=}(\widehat{V}_{n+j}))\bigr)/(W + w_{n+j}) p j = ( S < ( V n + j ) + U j ( w n + j + S = ( V n + j )) ) / ( W + w n + j ) . The oracle p-value p j ∗ p_j^* p j ∗ is the same expression with the true score V n + j V_{n+j} V n + j in place of V ^ n + j \widehat{V}_{n+j} V n + j and the same U j U_j U j . For a test point ( X , Y ) ∼ Q (X, Y) \sim Q ( X , Y ) ∼ Q let F D ∗ ( t ) : = P ( p j ∗ ≤ t ∣ D ) F_D^*(t) := \mathbb{P}(p_j^* \le t \mid D) F D ∗ ( t ) := P ( p j ∗ ≤ t ∣ D ) ; it does not depend on j j j or on the thresholds c c c . Let M D : = max 1 ≤ k ≤ m F D ∗ ( t k ) / t k M_D := \max_{1 \le k \le m} F_D^*(t_k)/t_k M D := max 1 ≤ k ≤ m F D ∗ ( t k ) / t k , with t k = q k / m t_k = qk/m t k = q k / m , and B ‾ : = ( ess sup w ) / E P [ w ] \overline{B} := (\operatorname{ess\,sup} w)/\mathbb{E}_P[w] B := ( ess sup w ) / E P [ w ] ; with 1 ≤ w ≤ B 1 \le w \le B 1 ≤ w ≤ B we have 1 ≤ B ‾ ≤ B 1 \le\overline{B} \le B 1 ≤ B ≤ B . Write κ : = B / n \kappa:= B/n κ := B / n . We assume q ∈ ( 0 , 1 ) q \in(0, 1) q ∈ ( 0 , 1 ) throughout.
Remark 27 (The assumption q < 1 q < 1 q < 1 ). In the proofs of Proposition 3 , Lemma 30 and Theorem 6 no step needs q < 1 q < 1 q < 1 . When s k = t k ( 1 + κ ) ≥ 1 s_k = t_k(1 + \kappa) \ge1 s k = t k ( 1 + κ ) ≥ 1 , the tail event { F D ∗ ( t k ) > s k + d } \{F_D^*(t_k) > s_k + d\} { F D ∗ ( t k ) > s k + d } is empty because F D ∗ ≤ 1 F_D^* \le1 F D ∗ ≤ 1 . The assumption q ∈ ( 0 , 1 ) q \in(0, 1) q ∈ ( 0 , 1 ) is kept only as a harmless standing assumption.
A one-sided Bernstein inequality
Lemma 28 (One-sided Bernstein). Let ξ 1 , … , ξ n \xi_1, \ldots, \xi_n ξ 1 , … , ξ n be independent with E ξ i = 0 \mathbb{E}\xi_i = 0 E ξ i = 0 and ξ i ≤ M \xi_i \le M ξ i ≤ M almost surely, for some M > 0 M > 0 M > 0 , and let ∑ i Var ξ i ≤ σ 2 \sum_i \operatorname{Var}\xi_i \le\sigma^2 ∑ i Var ξ i ≤ σ 2 . Then for every ϱ > 0 \varrho> 0 ϱ > 0 ,
P ( ∑ i = 1 n ξ i ≥ ϱ ) ≤ exp ( − ϱ 2 2 ( σ 2 + M ϱ / 3 ) ) . \mathbb{P}\left(\sum_{i=1}^n \xi_i \ge\varrho\right) \le\exp\left(-\frac{\varrho^2}{2(\sigma^2 + M\varrho/3)}\right). P ( i = 1 ∑ n ξ i ≥ ϱ ) ≤ exp ( − 2 ( σ 2 + M ϱ /3 ) ϱ 2 ) .
Proof. If σ 2 = 0 \sigma^2 = 0 σ 2 = 0 , every ξ i \xi_i ξ i is 0 almost surely and the claim is trivial, so assume σ 2 > 0 \sigma^2 > 0 σ 2 > 0 . The condition M > 0 M > 0 M > 0 is no restriction: with mean zero, M < 0 M < 0 M < 0 is impossible and M = 0 M = 0 M = 0 forces ξ i ≡ 0 \xi_i \equiv0 ξ i ≡ 0 .
Step 1. For τ > 0 \tau> 0 τ > 0 put g τ ( ϑ ) : = ( e τ ϑ − 1 − τ ϑ ) / ϑ 2 g_\tau(\vartheta) := (e^{\tau\vartheta} - 1 - \tau\vartheta)/\vartheta^2 g τ ( ϑ ) := ( e τ ϑ − 1 − τ ϑ ) / ϑ 2 , with g τ ( 0 ) : = τ 2 / 2 g_\tau(0) := \tau^2/2 g τ ( 0 ) := τ 2 /2 . Then
g τ ( ϑ ) = τ 2 ∫ 0 1 ( 1 − r ) e τ ϑ r d r g_\tau(\vartheta) = \tau^2 \int_0^1 (1-r)e^{\tau\vartheta r}\,dr g τ ( ϑ ) = τ 2 ∫ 0 1 ( 1 − r ) e τ ϑ r d r
(integrate by parts to check the identity), which is non-decreasing in ϑ \vartheta ϑ because each integrand is. So for ϑ ≤ M \vartheta\le M ϑ ≤ M , e τ ϑ ≤ 1 + τ ϑ + ϑ 2 g τ ( M ) e^{\tau\vartheta} \le1 + \tau\vartheta+ \vartheta^2 g_\tau(M) e τ ϑ ≤ 1 + τ ϑ + ϑ 2 g τ ( M ) . Step 2. Let Ψ ( r ) : = e r − 1 − r \Psi(r) := e^r - 1 - r Ψ ( r ) := e r − 1 − r , so that g τ ( M ) = Ψ ( τ M ) / M 2 g_\tau(M) = \Psi(\tau M)/M^2 g τ ( M ) = Ψ ( τ M ) / M 2 . Taking expectations,
E e τ ξ i ≤ 1 + Var ( ξ i ) Ψ ( τ M ) M 2 ≤ exp ( Var ( ξ i ) Ψ ( τ M ) M 2 ) . \mathbb{E}e^{\tau\xi_i} \le1 + \operatorname{Var}(\xi_i)\frac{\Psi(\tau M)}{M^2} \le\exp\left(\operatorname{Var}(\xi_i)\frac{\Psi(\tau M)}{M^2}\right). E e τ ξ i ≤ 1 + Var ( ξ i ) M 2 Ψ ( τ M ) ≤ exp ( Var ( ξ i ) M 2 Ψ ( τ M ) ) .
Step 3. For 0 ≤ r < 3 0 \le r < 3 0 ≤ r < 3 , Ψ ( r ) = ∑ j ≥ 2 r j / j ! ≤ ( r 2 / 2 ) ∑ j ≥ 0 ( r / 3 ) j = r 2 / ( 2 ( 1 − r / 3 ) ) \Psi(r) = \sum_{j\ge2}r^j/j! \le(r^2/2)\sum_{j\ge0}(r/3)^j = r^2/(2(1-r/3)) Ψ ( r ) = ∑ j ≥ 2 r j / j ! ≤ ( r 2 /2 ) ∑ j ≥ 0 ( r /3 ) j = r 2 / ( 2 ( 1 − r /3 )) , because j ! ≥ 2 ⋅ 3 j − 2 j! \ge2\cdot3^{j-2} j ! ≥ 2 ⋅ 3 j − 2 .
Step 4. Chernoff’s bound (Markov’s inequality applied to e τ ∑ i ξ i e^{\tau\sum_i \xi_i} e τ ∑ i ξ i , and independence) gives, for 0 < τ < 3 / M 0 < \tau< 3/M 0 < τ < 3/ M ,
P ( ∑ i ξ i ≥ ϱ ) ≤ exp ( − τ ϱ + σ 2 τ 2 2 ( 1 − τ M / 3 ) ) . \mathbb{P}\left(\sum_i \xi_i \ge\varrho\right) \le\exp\left(-\tau\varrho+ \frac{\sigma^2\tau^2}{2(1-\tau M/3)}\right). P ( i ∑ ξ i ≥ ϱ ) ≤ exp ( − τ ϱ + 2 ( 1 − τ M /3 ) σ 2 τ 2 ) .
Step 5. Take τ : = ϱ / ( σ 2 + M ϱ / 3 ) \tau:= \varrho/(\sigma^2+M\varrho/3) τ := ϱ / ( σ 2 + M ϱ /3 ) . Then τ M < 3 \tau M < 3 τ M < 3 and 1 − τ M / 3 = σ 2 / ( σ 2 + M ϱ / 3 ) 1-\tau M/3 = \sigma^2/(\sigma^2+M\varrho/3) 1 − τ M /3 = σ 2 / ( σ 2 + M ϱ /3 ) . So σ 2 τ 2 / ( 2 ( 1 − τ M / 3 ) ) = τ ϱ / 2 \sigma^2\tau^2/(2(1-\tau M/3)) = \tau\varrho/2 σ 2 τ 2 / ( 2 ( 1 − τ M /3 )) = τ ϱ /2 , and the exponent is − τ ϱ / 2 = − ϱ 2 / ( 2 ( σ 2 + M ϱ / 3 ) ) -\tau\varrho/2 = -\varrho^2/(2(\sigma^2+M\varrho/3)) − τ ϱ /2 = − ϱ 2 / ( 2 ( σ 2 + M ϱ /3 )) .
Chernoff bounds for Poisson and binomial counts
The proofs in Appendices B–E use the following standard tail bounds.
Lemma 29 (Chernoff bounds). Let X X X be Poisson with mean μ > 0 \mu> 0 μ > 0 , or a sum of independent Bernoulli variables with total mean μ > 0 \mu> 0 μ > 0 (for example a binomial variable). Let h ( u ) : = u log u − u + 1 h(u) := u\log u-u+1 h ( u ) := u log u − u + 1 for u > 0 u > 0 u > 0 and h ( 0 ) : = 1 h(0) := 1 h ( 0 ) := 1 . Then
P ( X ≥ x ) ≤ e − μ h ( x / μ ) ( x ≥ μ ) , P ( X ≤ x ) ≤ e − μ h ( x / μ ) ( 0 ≤ x ≤ μ ) . \mathbb{P}(X \ge x) \le e^{-\mu h(x/\mu)} \qquad(x \ge\mu), \qquad\mathbb{P}(X \le x) \le e^{-\mu h(x/\mu)} \qquad(0 \le x \le\mu). P ( X ≥ x ) ≤ e − μ h ( x / μ ) ( x ≥ μ ) , P ( X ≤ x ) ≤ e − μ h ( x / μ ) ( 0 ≤ x ≤ μ ) .
In particular, for every c > 0 c > 0 c > 0 , P ( ∣ X − μ ∣ ≥ c μ ) ≤ 2 e − μ h c \mathbb{P}(|X-\mu| \ge c\mu) \le2e^{-\mu h_c} P ( ∣ X − μ ∣ ≥ c μ ) ≤ 2 e − μ h c with h c : = min { h ( 1 + c ) , h ( max { 1 − c , 0 } ) } > 0 h_c := \min\{h(1+c),h(\max\{1-c,0\})\} > 0 h c := min { h ( 1 + c ) , h ( max { 1 − c , 0 })} > 0 .
Proof. For real θ \theta θ , E e θ X ≤ exp ( μ ( e θ − 1 ) ) \mathbb{E}e^{\theta X} \le\exp(\mu(e^\theta-1)) E e θ X ≤ exp ( μ ( e θ − 1 )) : with equality for the Poisson law, and, for a sum of Bernoulli( p i ) (p_i) ( p i ) variables, from 1 + p i ( e θ − 1 ) ≤ exp ( p i ( e θ − 1 ) ) 1+p_i(e^\theta-1) \le\exp(p_i(e^\theta-1)) 1 + p i ( e θ − 1 ) ≤ exp ( p i ( e θ − 1 )) . By Markov’s inequality applied to e θ X e^{\theta X} e θ X , P ( X ≥ x ) ≤ exp ( − θ x + μ ( e θ − 1 ) ) \mathbb{P}(X \ge x) \le\exp(-\theta x+\mu(e^\theta-1)) P ( X ≥ x ) ≤ exp ( − θ x + μ ( e θ − 1 )) for θ ≥ 0 \theta\ge0 θ ≥ 0 , and P ( X ≤ x ) ≤ exp ( − θ x + μ ( e θ − 1 ) ) \mathbb{P}(X \le x) \le\exp(-\theta x+\mu(e^\theta-1)) P ( X ≤ x ) ≤ exp ( − θ x + μ ( e θ − 1 )) for θ ≤ 0 \theta\le0 θ ≤ 0 . For x > 0 x > 0 x > 0 take θ = log ( x / μ ) \theta= \log(x/\mu) θ = log ( x / μ ) , which has the required sign in each case; the exponent is then − x log ( x / μ ) + x − μ = − μ h ( x / μ ) -x\log(x/\mu)+x-\mu=-\mu h(x/\mu) − x log ( x / μ ) + x − μ = − μ h ( x / μ ) . For x = 0 x=0 x = 0 , let θ → − ∞ \theta\to-\infty θ → − ∞ : P ( X ≤ 0 ) ≤ e − μ = e − μ h ( 0 ) \mathbb{P}(X \le0) \le e^{-\mu}=e^{-\mu h(0)} P ( X ≤ 0 ) ≤ e − μ = e − μ h ( 0 ) . For the last claim, { ∣ X − μ ∣ ≥ c μ } \{|X-\mu| \ge c\mu\} { ∣ X − μ ∣ ≥ c μ } is contained in { X ≥ ( 1 + c ) μ } ∪ { X ≤ ( 1 − c ) μ } \{X \ge(1+c)\mu\}\cup\{X \le(1-c)\mu\} { X ≥ ( 1 + c ) μ } ∪ { X ≤ ( 1 − c ) μ } , where the second event is empty if c > 1 c>1 c > 1 , and h > 0 h>0 h > 0 on [ 0 , ∞ ) ∖ { 1 } [0,\infty)\setminus\{1\} [ 0 , ∞ ) ∖ { 1 } because h h h is strictly convex with h ( 1 ) = h ′ ( 1 ) = 0 h(1)=h'(1)=0 h ( 1 ) = h ′ ( 1 ) = 0 .
The reduction
Proof of Proposition 3. Under the setting above we show FDR ≤ q E [ M D ] \operatorname{FDR} \le q\mathbb{E}[M_D] FDR ≤ q E [ M D ] , and in outlier detection with m 0 m_0 m 0 nulls FDR ≤ ( m 0 / m ) q E [ M D ] \operatorname{FDR} \le(m_0/m)q\mathbb{E}[M_D] FDR ≤ ( m 0 / m ) q E [ M D ] ; with a random null set, FDR ≤ ( E [ m 0 ] / m ) q E [ M D ] \operatorname{FDR} \le(\mathbb{E}[m_0]/m)q\mathbb{E}[M_D] FDR ≤ ( E [ m 0 ] / m ) q E [ M D ] .
Step 1. By the leave-one-out identity (Lemma 66), unit j j j is rejected iff p j ≤ q R j / m p_j \le qR_j/m p j ≤ q R j / m , where R j : = R 0 ( p − j ) R_j := R^0(p_{-j}) R j := R 0 ( p − j ) is the number of BH rejections when p j p_j p j is replaced by 0; on that event the number of rejections is R j R_j R j . So
FDR = ∑ j = 1 m E [ 1 { p j ≤ q R j / m , j n u l l } R j ] . \operatorname{FDR}=\sum_{j=1}^{m}\mathbb{E}\left[\frac{\mathbf{1}\{p_j\le qR_j/m,\ j\ \mathrm{null}\}}{R_j}\right]. FDR = j = 1 ∑ m E [ R j 1 { p j ≤ q R j / m , j null } ] .
Step 2. Given D D D , the p-value p j p_j p j and the null indicator of unit j j j are functions of unit j j j ’s tuple, while R j R_j R j is a function of the other units. So they are conditionally independent given D D D , and
E [ 1 { p j ≤ q R j / m , j n u l l } R j ∣ D ] = E [ G D , j ( q R j / m ) R j ∣ D ] , G D , j ( t ) : = P ( p j ≤ t , j n u l l ∣ D ) . \mathbb{E}\left[\left.\frac{\mathbf{1}\{p_j\le qR_j/m,\ j\ \mathrm{null}\}}{R_j}\right|D\right]
=
\mathbb{E}\left[\left.\frac{G_{D,j}(qR_j/m)}{R_j}\right|D\right],
\qquad
G_{D,j}(t):=\mathbb{P}(p_j\le t,\ j\ \mathrm{null}\mid D). E [ R j 1 { p j ≤ q R j / m , j null } D ] = E [ R j G D , j ( q R j / m ) D ] , G D , j ( t ) := P ( p j ≤ t , j null ∣ D ) .
Step 3. On { j n u l l } \{j\ \mathrm{null}\} { j null } in selection, Y n + j ≤ c n + j Y_{n+j}\le c_{n+j} Y n + j ≤ c n + j , so V : = V n + j ≤ V ^ : = V ^ n + j V:=V_{n+j}\le\widehat V:=\widehat V_{n+j} V := V n + j ≤ V := V n + j by (A3). Then p j ≥ p j ∗ p_j\ge p_j^* p j ≥ p j ∗ , in two cases.
( W + w n + j ) ( p j − p j ∗ ) ≥ ( 1 − U j ) S = ( V ) + U j S = ( V ^ ) ≥ 0 ; (W + w_{n+j})(p_j - p_j^*) \ge(1 - U_j) S_{=}(V) + U_j S_{= }(\hat{V}) \ge0; ( W + w n + j ) ( p j − p j ∗ ) ≥ ( 1 − U j ) S = ( V ) + U j S = ( V ^ ) ≥ 0 ;
the U j w n + j U_j w_{n+j} U j w n + j terms cancel.
In outlier detection p j = p j ∗ p_j = p_j^* p j = p j ∗ for nulls. So G D , j ( t ) ≤ F D ∗ ( t ) G_{D,j}(t) \le F_D^*(t) G D , j ( t ) ≤ F D ∗ ( t ) .
Step 4. R j R_j R j takes values r ∈ { 1 , … , m } r \in\{1,\ldots,m\} r ∈ { 1 , … , m } , and F D ∗ ( t r ) / r = ( q / m ) F D ∗ ( t r ) / t r ≤ ( q / m ) M D F_D^*(t_r)/r = (q/m) F_D^*(t_r)/t_r \le(q/m) M_D F D ∗ ( t r ) / r = ( q / m ) F D ∗ ( t r ) / t r ≤ ( q / m ) M D . Summing over j j j gives FDR ≤ q E [ M D ] \operatorname{FDR} \le q \mathbb{E}[M_D] FDR ≤ q E [ M D ] . In outlier detection only the m 0 m_0 m 0 nulls contribute; with a random null set, condition on it first.
Lemma 30 (Explicit forms). In the setting above:
(i) For all n n n , m m m , q q q , B B B and every score law (atoms allowed),
E [ M D ] ≤ 1 + κ + 2 m π ( 1 + κ ) B ‾ n q + 12 ( 1 + log m ) m B ‾ n q . \mathbb{E}[M_D] \le1 + \kappa+ 2m\sqrt{\frac{\pi(1+\kappa)\overline{B}}{nq}} + 12(1+\log m)\frac{m\overline{B}}{nq}. E [ M D ] ≤ 1 + κ + 2 m n q π ( 1 + κ ) B + 12 ( 1 + log m ) n q m B .
(ii) Let θ : = 5 B ‾ m L n q \theta:= \sqrt{\frac{5\overline{B}mL}{nq}} θ := n q 5 B m L with L : = log ( 0.8 n m / ( B ‾ q ) ) L := \log(0.8nm/(\overline{B}q)) L := log ( 0.8 nm / ( B q )) . If L ≥ 1 L \ge1 L ≥ 1 , θ ≤ 1 \theta\le1 θ ≤ 1 and κ ≤ 1 \kappa\le1 κ ≤ 1 , then
E [ M D ] ≤ 1 + κ + 2 θ . \mathbb{E}[M_D] \le1+\kappa+2\theta. E [ M D ] ≤ 1 + κ + 2 θ .
(iii) If κ ≤ 1 \kappa\le1 κ ≤ 1 , then
E [ M D ] ≤ 1 + κ + 7.4 B ‾ m n q + 33 B ‾ m n q . \mathbb{E}[M_D] \le1+\kappa+7.4\sqrt{\frac{\overline{B}m}{nq}}+33\frac{\overline{B}m}{nq}. E [ M D ] ≤ 1 + κ + 7.4 n q B m + 33 n q B m .
With Proposition 3 , (i) gives FDR ≤ q [ 1 + O ( m B ‾ / n ) ] \operatorname{FDR} \le q[1+O(m\sqrt{\overline{B}/n})] FDR ≤ q [ 1 + O ( m B / n )] for every configuration covered by the assumptions of Jin & Candès [12 ] : non-null units, decoys, clustered thresholds and score atoms included. No anti-concentration of the test p p p -values is needed. Bound (iii) is smaller than (i) for every m ≥ 6 m \ge6 m ≥ 6 (at m = 5 m=5 m = 5 only where both bounds exceed 5), and it has no logarithmic factor, unlike (ii).
Proof of Lemma 30(i). Fix t = t k t=t_k t = t k and write s : = t ( 1 + κ ) s:=t(1+\kappa) s := t ( 1 + κ ) .
Step 1. From p ∗ p^* p ∗ to a weighted empirical CDF. For u ∈ [ 0 , 1 ] u \in[0,1] u ∈ [ 0 , 1 ] let Λ ^ ( v , u ) : = ( S < ( v ) + u S = ( v ) ) / W \widehat{\Lambda}(v,u):=(S_{<}(v)+uS_{=}(v))/W Λ ( v , u ) := ( S < ( v ) + u S = ( v )) / W . For a test point ( V , w , U ) (V,w,U) ( V , w , U ) ,
p ∗ ≥ S < ( V ) + U S = ( V ) W + w = Λ ^ ( V , U ) W W + w . p^* \ge\frac{S_{<}(V)+US_{=}(V)}{W+w}=\widehat{\Lambda}(V,U)\frac{W}{W+w}. p ∗ ≥ W + w S < ( V ) + U S = ( V ) = Λ ( V , U ) W + w W .
Since w / W ≤ B / n = κ w/W \le B/n=\kappa w / W ≤ B / n = κ , p ∗ ≤ t p^* \le t p ∗ ≤ t implies Λ ^ ( V , U ) ≤ s \widehat{\Lambda}(V,U)\le s Λ ( V , U ) ≤ s .
Step 2. Population version. Let Λ ( v , u ) : = Q ( V < v ) + u Q ( V = v ) \Lambda(v,u):=Q(V<v)+uQ(V=v) Λ ( v , u ) := Q ( V < v ) + u Q ( V = v ) . With U U U uniform and independent, Λ ( V , U ) ∼ Unif ( 0 , 1 ) \Lambda(V,U)\sim\operatorname{Unif}(0,1) Λ ( V , U ) ∼ Unif ( 0 , 1 ) under Q Q Q (the randomised probability integral transform). For completeness: fix s ∈ ( 0 , 1 ) s\in(0,1) s ∈ ( 0 , 1 ) and let v s : = inf { v : Q ( V ≤ v ) ≥ s } v_s:=\inf\{v:Q(V\le v)\ge s\} v s := inf { v : Q ( V ≤ v ) ≥ s } , so that Q ( V < v s ) ≤ s ≤ Q ( V ≤ v s ) Q(V<v_s)\le s\le Q(V\le v_s) Q ( V < v s ) ≤ s ≤ Q ( V ≤ v s ) . Then Λ ( v , u ) ≤ Q ( V ≤ v ) < s \Lambda(v,u)\le Q(V\le v)<s Λ ( v , u ) ≤ Q ( V ≤ v ) < s for v < v s v<v_s v < v s , and, if Q ( V = v s ) > 0 Q(V=v_s)>0 Q ( V = v s ) > 0 , Λ ( v s , u ) < s \Lambda(v_s,u)<s Λ ( v s , u ) < s for u < u s : = ( s − Q ( V < v s ) ) / Q ( V = v s ) u<u_s:=(s-Q(V<v_s))/Q(V=v_s) u < u s := ( s − Q ( V < v s )) / Q ( V = v s ) . So P ( Λ ( V , U ) < s ) ≥ Q ( V < v s ) + u s Q ( V = v s ) = s \mathbb{P}(\Lambda(V,U)<s)\ge Q(V<v_s)+u_sQ(V=v_s)=s P ( Λ ( V , U ) < s ) ≥ Q ( V < v s ) + u s Q ( V = v s ) = s (and ≥ Q ( V < v s ) = s \ge Q(V<v_s)=s ≥ Q ( V < v s ) = s if Q ( V = v s ) = 0 Q(V=v_s)=0 Q ( V = v s ) = 0 ). Hence Λ ( V , U ) \Lambda(V,U) Λ ( V , U ) is stochastically smaller than Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) ; it also has mean P ( V ′ < V ) + 1 2 P ( V ′ = V ) = 1 2 \mathbb{P}(V'<V)+\frac{1}{2}\mathbb{P}(V'=V)=\frac{1}{2} P ( V ′ < V ) + 2 1 P ( V ′ = V ) = 2 1 for an independent copy V ′ V' V ′ , by symmetry, so it is Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) . Also P ( ( V , U ) < l e x ( v , u ) ) = Λ ( v , u ) \mathbb{P}((V,U)<_{\mathrm{lex}}(v,u))=\Lambda(v,u) P (( V , U ) < lex ( v , u )) = Λ ( v , u ) in the lexicographic order. Both Λ ^ \widehat{\Lambda} Λ and Λ \Lambda Λ are non-decreasing in that order, and as ( v , u ) (v,u) ( v , u ) increases, Λ \Lambda Λ takes every value in [ 0 , 1 ) [0,1) [ 0 , 1 ) .
Step 3. Localising. Fix d > 0 d>0 d > 0 with Λ \Lambda Λ -level ℓ : = s + d < 1 \ell:=s+d<1 ℓ := s + d < 1 , and pick ( v + , u + ) (v^+,u^+) ( v + , u + ) with Λ ( v + , u + ) = ℓ \Lambda(v^+,u^+)=\ell Λ ( v + , u + ) = ℓ . If Λ ^ ( v + , u + ) > s \widehat{\Lambda}(v^+,u^+)>s Λ ( v + , u + ) > s , then every ( v , u ) ≥ l e x ( v + , u + ) (v,u)\ge_{\mathrm{lex}}(v^+,u^+) ( v , u ) ≥ lex ( v + , u + ) has Λ ^ > s \widehat{\Lambda}>s Λ > s , so { Λ ^ ( V , U ) ≤ s } ⊂ { ( V , U ) < l e x ( v + , u + ) } \{\widehat{\Lambda}(V,U)\le s\}\subset\{(V,U)<_{\mathrm{lex}}(v^+,u^+)\} { Λ ( V , U ) ≤ s } ⊂ {( V , U ) < lex ( v + , u + )} and F D ∗ ( t ) ≤ ℓ = s + d F_D^*(t)\le\ell=s+d F D ∗ ( t ) ≤ ℓ = s + d . Hence
P ( F D ∗ ( t ) > s + d ) ≤ P ( Λ ^ ( v + , u + ) ≤ s ) . \mathbb{P}(F_D^*(t)>s+d)\le\mathbb{P}(\widehat{\Lambda}(v^+,u^+)\le s). P ( F D ∗ ( t ) > s + d ) ≤ P ( Λ ( v + , u + ) ≤ s ) .
If s + d ≥ 1 s+d\ge1 s + d ≥ 1 , the probability on the left is 0 0 0 , since F D ∗ ≤ 1 F_D^*\le1 F D ∗ ≤ 1 . Step 4. Bernstein. Let w i ′ : = w i / E P [ w ] w_i' := w_i/\mathbb{E}_{P}[w] w i ′ := w i / E P [ w ] (so E P w ′ = 1 \mathbb{E}_{P}w'=1 E P w ′ = 1 and w ′ < B ˉ w'<\bar{B} w ′ < B ˉ ) and χ i : = 1 { V i < v + } + u + 1 { V i = v + } ∈ [ 0 , 1 ] \chi_i := \mathbb{1}\{V_i<v^+\}+u^+\mathbb{1}\{V_i=v^+\}\in[0,1] χ i := 1 { V i < v + } + u + 1 { V i = v + } ∈ [ 0 , 1 ] . Then E P [ w ′ χ ] = Λ ( v + , u + ) = ℓ \mathbb{E}_{P}[w'\chi]=\Lambda(v^+,u^+)=\ell E P [ w ′ χ ] = Λ ( v + , u + ) = ℓ , because d Q / d P = w / E P w \mathrm{d}Q/\mathrm{d}P=w/\mathbb{E}_{P}w d Q / d P = w / E P w . The bad event { Λ ^ ( v + , u + ) ≤ s } \{\widehat{\Lambda}(v^+,u^+)\le s\} { Λ ( v + , u + ) ≤ s } is { ∑ i w i ′ ( χ i − s ) ≤ 0 } \{\sum_i w_i'(\chi_i-s)\le0\} { ∑ i w i ′ ( χ i − s ) ≤ 0 } . Put η i : = w i ′ ( χ i − s ) \eta_i:=w_i'(\chi_i-s) η i := w i ′ ( χ i − s ) , which has mean d d d , and η ~ i : = η i − d \widetilde{\eta}_i:=\eta_i-d η i := η i − d .
∣ η ~ i ∣ ≤ B ˉ |\widetilde{\eta}_i|\le\bar{B} ∣ η i ∣ ≤ B ˉ : η i ∈ [ − B ˉ s , B ˉ ( 1 − s ) ] \eta_i\in[-\bar{B}s,\bar{B}(1-s)] η i ∈ [ − B ˉ s , B ˉ ( 1 − s )] and d ≤ B ˉ d d\le\bar{B}d d ≤ B ˉ d , so η ~ i ≥ − B ˉ ( s + d ) ≥ − B ˉ \widetilde{\eta}_i\ge-\bar{B}(s+d)\ge-\bar{B} η i ≥ − B ˉ ( s + d ) ≥ − B ˉ and η ~ i ≤ B ˉ ( 1 − s ) ≤ B ˉ \widetilde{\eta}_i\le\bar{B}(1-s)\le\bar{B} η i ≤ B ˉ ( 1 − s ) ≤ B ˉ .
E η ~ i 2 ≤ E [ w 2 ( χ − s ) 2 ] ≤ B ˉ E P [ w ′ ( χ − s ) 2 ] = B ˉ E Q [ ( χ − s ) 2 ] = B ˉ ( Var Q χ + d 2 ) ≤ B ˉ ( ℓ ( 1 − ℓ ) + d 2 ) \mathbb{E}\widetilde{\eta}_i^2\le\mathbb{E}[w^2(\chi-s)^2]\le\bar{B}\mathbb{E}_{P}[w'(\chi-s)^2]=\bar{B}\mathbb{E}_{Q}[(\chi-s)^2]=\bar{B}(\operatorname{Var}_{Q}\chi+d^2)\le\bar{B}(\ell(1-\ell)+d^2) E η i 2 ≤ E [ w 2 ( χ − s ) 2 ] ≤ B ˉ E P [ w ′ ( χ − s ) 2 ] = B ˉ E Q [( χ − s ) 2 ] = B ˉ ( Var Q χ + d 2 ) ≤ B ˉ ( ℓ ( 1 − ℓ ) + d 2 ) . (Under Q Q Q , χ ∈ [ 0 , 1 ] \chi\in[0,1] χ ∈ [ 0 , 1 ] has mean ℓ \ell ℓ .)
Lemma 28, applied to ξ i : = − η ~ i \xi_i:=-\widetilde{\eta}_i ξ i := − η i with M = B ˉ M=\bar{B} M = B ˉ (only the bound η ~ i ≥ − B ˉ \widetilde{\eta}_i\ge-\bar{B} η i ≥ − B ˉ is used here), σ 2 : = n B ˉ ( ℓ + d 2 ) ≥ ∑ i Var ξ i \sigma^2:=n\bar{B}(\ell+d^2)\ge\sum_i\operatorname{Var}\xi_i σ 2 := n B ˉ ( ℓ + d 2 ) ≥ ∑ i Var ξ i and ϱ = n d \varrho=nd ϱ = n d , gives
P ( ∑ i η ~ i ≤ − n d ) ≤ exp ( − n d 2 2 [ B ˉ ( ℓ + d 2 ) + B ˉ d / 3 ] ) . \mathbb{P}\left(\sum_i\widetilde{\eta}_i\le-nd\right)\le\exp\left(-\frac{nd^2}{2[\bar{B}(\ell+d^2)+\bar{B}d/3]}\right). P ( i ∑ η i ≤ − n d ) ≤ exp ( − 2 [ B ˉ ( ℓ + d 2 ) + B ˉ d /3 ] n d 2 ) .
P ( F D ∗ ( t ) > s + d ) ≤ exp ( − n d 2 2 B ˉ ( s + 3 d ) ) . \mathbb{P}\left(F_D^*(t)>s+d\right)\le\exp\left(-\frac{nd^2}{2\bar{B}(s+3d)}\right). P ( F D ∗ ( t ) > s + d ) ≤ exp ( − 2 B ˉ ( s + 3 d ) n d 2 ) .
Step 5. Integrating. E ( F D ∗ ( t ) − s ) + = ∫ 0 1 P ( F D ∗ ( t ) > s + d ) d d \mathbb{E}(F_D^*(t)-s)^+=\int_0^1\mathbb{P}(F_D^*(t)>s+d)\,\mathrm{d}d E ( F D ∗ ( t ) − s ) + = ∫ 0 1 P ( F D ∗ ( t ) > s + d ) d d . For d ≤ s / 3 d\le s/3 d ≤ s /3 the exponent is at least n d 2 / ( 4 B ˉ s ) nd^2/(4\bar{B}s) n d 2 / ( 4 B ˉ s ) , which contributes at most π B ˉ s / n \sqrt{\pi\bar{B}s/n} π B ˉ s / n (a half-Gaussian integral). For d ≥ s / 3 d\ge s/3 d ≥ s /3 it is at least n d / ( 12 B ˉ ) nd/(12\bar{B}) n d / ( 12 B ˉ ) , which contributes at most 12 B ˉ / n 12\bar{B}/n 12 B ˉ / n . So
E ( F D ∗ ( t ) − s ) + ≤ π B ˉ s / n + 12 B ˉ / n . \mathbb{E}(F_D^*(t)-s)^+\le\sqrt{\pi\bar{B}s/n}+12\bar{B}/n. E ( F D ∗ ( t ) − s ) + ≤ π B ˉ s / n + 12 B ˉ / n .
Step 6. Summing over the grid. With s k : = t k ( 1 + κ ) s_k:=t_k(1+\kappa) s k := t k ( 1 + κ ) , M D ≤ ( 1 + κ ) + ∑ k ( F D ∗ ( t k ) − s k ) + / t k M_D\le(1+\kappa)+\sum_{k}(F_D^*(t_k)-s_k)^+/t_k M D ≤ ( 1 + κ ) + ∑ k ( F D ∗ ( t k ) − s k ) + / t k . Taking expectations, and using ∑ k ≤ m t k − 1 / 2 ≤ 2 m / q \sum_{k\le m}t_k^{-1/2}\le2m/\sqrt{q} ∑ k ≤ m t k − 1/2 ≤ 2 m / q and ∑ k ≤ m 1 / t k ≤ ( m / q ) ( 1 + log m ) \sum_{k\le m}1/t_k\le(m/q)(1+\log m) ∑ k ≤ m 1/ t k ≤ ( m / q ) ( 1 + log m ) , gives (i). □
Proof of Lemma 30(ii). Take d k : = θ t k d_k:=\theta t_k d k := θ t k in Step 4 of the proof of (i). Under the stated conditions ℓ = t k ( 1 + κ + θ ) ≤ 3 t k \ell=t_k(1+\kappa+\theta)\le3t_k ℓ = t k ( 1 + κ + θ ) ≤ 3 t k , d k 2 ≤ t k d_k^2\le t_k d k 2 ≤ t k and d k / 3 ≤ t k / 3 d_k/3\le t_k/3 d k /3 ≤ t k /3 , so ℓ + d k 2 + d k / 3 ≤ 5 t k \ell+d_k^2+d_k/3\le5t_k ℓ + d k 2 + d k /3 ≤ 5 t k and
P ( F D ∗ ( t k ) > s k + θ t k ) ≤ exp ( − n θ 2 t k / ( 10 B ˉ ) ) = e − c θ k , c θ : = n θ 2 q 10 B ˉ m . \mathbb{P}(F_D^*(t_k)>s_k+\theta t_k)\le\exp(-n\theta^2t_k/(10\bar{B}))=e^{-c_\theta k},\qquad c_\theta:=\frac{n\theta^2q}{10\bar{B}m}. P ( F D ∗ ( t k ) > s k + θ t k ) ≤ exp ( − n θ 2 t k / ( 10 B ˉ )) = e − c θ k , c θ := 10 B ˉ m n θ 2 q .
Now M D ≤ 1 + κ + θ + ∑ k 1 { F D ∗ ( t k ) > s k + θ t k } / t k M_D\le1+\kappa+\theta+\sum_k\mathbb{1}\{F_D^*(t_k)>s_k+\theta t_k\}/t_k M D ≤ 1 + κ + θ + ∑ k 1 { F D ∗ ( t k ) > s k + θ t k } / t k , and
E ∑ k 1 { F D ∗ ( t k ) > s k + θ t k } t k ≤ m q ∑ k e − c θ k ≤ m q e − c θ 1 − e − c θ . \mathbb{E}\sum_k\frac{\mathbb{1}\{F_D^*(t_k)>s_k+\theta t_k\}}{t_k}
\le\frac{m}{q}\sum_k e^{-c_\theta k}
\le\frac{m}{q}\frac{e^{-c_\theta}}{1-e^{-c_\theta}}. E k ∑ t k 1 { F D ∗ ( t k ) > s k + θ t k } ≤ q m k ∑ e − c θ k ≤ q m 1 − e − c θ e − c θ .
If e − c θ ≤ q θ / ( 2 m ) e^{-c_\theta}\le q\theta/(2m) e − c θ ≤ q θ / ( 2 m ) (which is ≤ 1 2 \le\frac12 ≤ 2 1 ), this is at most θ \theta θ . With θ 2 = 5 B ˉ m L / ( n q ) \theta^2=5\bar{B}mL/(nq) θ 2 = 5 B ˉ m L / ( n q ) we get c θ = L / 2 c_\theta=L/2 c θ = L /2 , and e − L / 2 ≤ q θ / ( 2 m ) e^{-L/2}\le q\theta/(2m) e − L /2 ≤ q θ / ( 2 m ) is equivalent to L ≥ log ( 0.8 n m / ( B ˉ q ) ) − log L L\ge\log(0.8nm/(\bar{B}q))-\log L L ≥ log ( 0.8 nm / ( B ˉ q )) − log L . So the choice L : = log ( 0.8 n m / ( B ˉ q ) ) L:=\log(0.8nm/(\bar{B}q)) L := log ( 0.8 nm / ( B ˉ q )) works whenever L ≥ 1 L\ge1 L ≥ 1 . Hence E [ M D ] ≤ 1 + κ + 2 θ \mathbb{E}[M_D]\le1+\kappa+2\theta E [ M D ] ≤ 1 + κ + 2 θ . □
Proof of Lemma 30(iii). With κ ≤ 1 \kappa\le1 κ ≤ 1 and s k : = t k ( 1 + κ ) s_k:=t_k(1+\kappa) s k := t k ( 1 + κ ) , let M D ′ : = max k ( F D ∗ ( t k ) − s k ) + / t k M_D':=\max_k(F_D^*(t_k)-s_k)^+/t_k M D ′ := max k ( F D ∗ ( t k ) − s k ) + / t k , so that M D ≤ 1 + κ + M D ′ M_D\le1+\kappa+M_D' M D ≤ 1 + κ + M D ′ .
Step 1. Tail at one grid point. Step 4 of the proof of (i), with d : = u t k d:=ut_k d := u t k , gives for every u > 0 ( t h e p r o b a b i l i t y i s $ 0 u>0 (the probability is \$0 u > 0 ( t h e p r o babi l i t y i s $0 when s k + d ≥ 1 s_k+d\ge1 s k + d ≥ 1 , so d ≤ 1 d\le1 d ≤ 1 and s k < 1 s_k<1 s k < 1 there)
P ( F D ∗ ( t k ) > s k + u t k ) ≤ exp ( − n u 2 t k 2 2 B ˉ ( s k + 3 u t k ) ) . \mathbb{P}(F_D^*(t_k)>s_k+ut_k)\le\exp\left(-\frac{nu^2t_k^2}{2\bar{B}(s_k+3ut_k)}\right). P ( F D ∗ ( t k ) > s k + u t k ) ≤ exp ( − 2 B ˉ ( s k + 3 u t k ) n u 2 t k 2 ) .
Since s k ≤ 2 t k s_k\le2t_k s k ≤ 2 t k , the exponent is at least k a g ( u ) ka\,g(u) k a g ( u ) , with a : = n q / ( 2 B ˉ m ) a:=nq/(2\bar{B}m) a := n q / ( 2 B ˉ m ) and g ( u ) : = u 2 / ( 2 + 3 u ) g(u):=u^2/(2+3u) g ( u ) := u 2 / ( 2 + 3 u ) . Step 2. Union over the grid. P ( M D ′ > u ) ≤ ∑ k ≥ 1 e − k a g ( u ) = e − a g ( u ) 1 − e − a g ( u ) \mathbb{P}(M'_D > u) \le\sum_{k\ge1} e^{-kag(u)} = \frac{e^{-ag(u)}}{1-e^{-ag(u)}} P ( M D ′ > u ) ≤ ∑ k ≥ 1 e − k a g ( u ) = 1 − e − a g ( u ) e − a g ( u ) . The tails decay geometrically in k k k , because relative fluctuations shrink like k − 1 / 2 k^{-1/2} k − 1/2 ; this is why no log m \log m log m appears.
Step 3. Integrating. g g g is increasing. Let u 0 u_0 u 0 solve a g ( u 0 ) = log 2 ag(u_0)=\log2 a g ( u 0 ) = log 2 . For u ≥ u 0 u\ge u_0 u ≥ u 0 the tail is at most 2 e − a g ( u ) 2e^{-ag(u)} 2 e − a g ( u ) . Also g ( u ) ≥ min ( u 2 / 4 , u / 6 ) g(u)\ge\min(u^2/4,u/6) g ( u ) ≥ min ( u 2 /4 , u /6 ) (split at u = 2 / 3 u=2/3 u = 2/3 ). So
E [ M D ′ ] ≤ u 0 + 2 ∫ 0 ∞ e − a min ( u 2 / 4 , u / 6 ) d u ≤ u 0 + 2 π / a + 12 / a , \mathbb{E}[M'_D]\le u_0+2\int_0^\infty e^{-a\min(u^2/4,u/6)}\,du\le u_0+2\sqrt{\pi/a}+12/a, E [ M D ′ ] ≤ u 0 + 2 ∫ 0 ∞ e − a m i n ( u 2 /4 , u /6 ) d u ≤ u 0 + 2 π / a + 12/ a ,
and u 0 ≤ 4 log 2 / a + 6 log 2 / a ≤ 1.666 / a + 4.16 / a u_0\le\sqrt{4\log2/a}+6\log2/a\le1.666/\sqrt{a}+4.16/a u 0 ≤ 4 log 2/ a + 6 log 2/ a ≤ 1.666/ a + 4.16/ a . Hence
E [ M D ′ ] ≤ 5.211 a + 16.16 a ≤ 7.37 B ˉ m n q + 32.4 B ˉ m n q . \mathbb{E}[M'_D]\le\frac{5.211}{\sqrt{a}}+\frac{16.16}{a}\le7.37\sqrt{\frac{\bar{B}m}{nq}}+32.4\frac{\bar{B}m}{nq}. E [ M D ′ ] ≤ a 5.211 + a 16.16 ≤ 7.37 n q B ˉ m + 32.4 n q B ˉ m .
Remark 31 (The bound does not vanish as B → 1 B\to1 B → 1 ). E [ M D ] − 1 \mathbb{E}[M_D]-1 E [ M D ] − 1 does not vanish as B → 1 B\to1 B → 1 . Even with constant weights it is about m / ( n q ) \sqrt{m/(nq)} m / ( n q ) : in simulation, E [ M D ] = 1.18 \mathbb{E}[M_D]=1.18 E [ M D ] = 1.18 (m = 10 m=10 m = 10 ) and 1.44 1.44 1.44 (m = 50 m=50 m = 50 ) at n = 1000 n=1000 n = 1000 , q = 0.1 q=0.1 q = 0.1 , where BH controls the FDR at exactly q q q . So the bounds of this appendix are informative only when m ≪ n q / B ˉ m\ll nq/\bar{B} m ≪ n q / B ˉ , and “O ( m B / n ) O(m\sqrt{B/n}) O ( m B / n ) ” hides a factor 1 / q 1/\sqrt{q} 1/ q . For weights near 1 1 1 the envelope bounds (Theorem 21 ), which vanish at B = 1 B=1 B = 1 , are better.
Tighter constants: proof of Theorem 6
We prove Theorem 6 in the following form. In the setting above let κ = B / n ≤ 1 \kappa=B/n\le1 κ = B / n ≤ 1 , x : = B ˉ m / ( n q ) x:=\bar{B}m/(nq) x := B ˉ m / ( n q ) and c : = 2 + ( 1 + κ ) q / 3 c:=2+(1+\kappa)q/3 c := 2 + ( 1 + κ ) q /3 (so c ≤ 2 + 2 q / 3 c\le2+2q/3 c ≤ 2 + 2 q /3 ). With s k : = t k ( 1 + κ ) s_k:=t_k(1+\kappa) s k := t k ( 1 + κ ) , define the per-point tail bounds
T k ( u ) : = exp ( − n u 2 t k 2 2 B ˉ [ s k ( 1 + u t k / 3 ) + u t k ( 1 + u t k ) ] ) if u t k < 1 − s k , T k ( u ) : = 0 otherwise . T_k(u):=\exp\left(-\frac{nu^2t_k^2}{2\bar{B}\left[s_k(1+ut_k/3)+ut_k(1+ut_k)\right]}\right)\quad\text{if }ut_k<1-s_k,\qquad T_k(u):=0\text{ otherwise}. T k ( u ) := exp ( − 2 B ˉ [ s k ( 1 + u t k /3 ) + u t k ( 1 + u t k ) ] n u 2 t k 2 ) if u t k < 1 − s k , T k ( u ) := 0 otherwise .
Then
E [ M D ] ≤ 1 + κ + J , J : = ∫ 0 ∞ min ( 1 , ∑ k = 1 m T k ( u ) ) d u , \mathbb{E}[M_D]\le1+\kappa+J,\qquad J:=\int_0^\infty\min\left(1,\sum_{k=1}^{m}T_k(u)\right)\,du, E [ M D ] ≤ 1 + κ + J , J := ∫ 0 ∞ min ( 1 , k = 1 ∑ m T k ( u ) ) d u ,
and, in closed form,
E [ M D ] ≤ 1 + κ + 1.6120 ( 1 + κ ) x + 2.7726 c x . \mathbb{E}[M_D]\le1+\kappa+1.6120\sqrt{(1+\kappa)x}+2.7726cx. E [ M D ] ≤ 1 + κ + 1.6120 ( 1 + κ ) x + 2.7726 c x .
Together with Proposition 3, F D R / q − 1 ≤ E [ M D ] − 1 \mathrm{FDR}/q-1\le\mathbb{E}[M_D]-1 FDR / q − 1 ≤ E [ M D ] − 1 , this is Theorem 6.
Proof of Theorem 6. The proof changes Step 4 of the proof of Lemma 30(i) and the proof of Lemma 30(iii).
Step 1. One-sided Bernstein. In Step 4 of the proof of Lemma 30(i), the bad event is ∑ i η i ≤ 0 \sum_i\eta_i\le0 ∑ i η i ≤ 0 , with η i = w i ′ ( χ i − s ) \eta_i=w_i'(\chi_i-s) η i = w i ′ ( χ i − s ) and E η i = d \mathbb{E}\eta_i=d E η i = d .
Put η ~ i : = η i − d \widetilde{\eta}_i:=\eta_i-d η i := η i − d , which has mean zero. Since η i ≥ − w i ′ s ≥ − B ˉ s \eta_i\ge-w_i's\ge-\bar{B}s η i ≥ − w i ′ s ≥ − B ˉ s , we have − η ~ i ≤ M : = d + B ˉ s -\widetilde{\eta}_i\le M:=d+\bar{B}s − η i ≤ M := d + B ˉ s . Only this upper bound is needed; the two-sided ∣ η ~ i ∣ ≤ B ˉ |\widetilde{\eta}_i|\le\bar{B} ∣ η i ∣ ≤ B ˉ used in Lemma 30(i) is wasteful.
Var ( η ~ i ) ≤ E η i 2 − d 2 ≤ B ˉ ( ℓ ( 1 − ℓ ) + d 2 ) − d 2 = : σ d 2 \operatorname{Var}(\widetilde{\eta}_i)\le\mathbb{E}\eta_i^2-d^2\le\bar{B}(\ell(1-\ell)+d^2)-d^2=:\sigma_d^2 Var ( η i ) ≤ E η i 2 − d 2 ≤ B ˉ ( ℓ ( 1 − ℓ ) + d 2 ) − d 2 =: σ d 2 , as in Lemma 30(i).
Lemma 28, applied to ξ i : = − η ~ i \xi_i:=-\widetilde{\eta}_i ξ i := − η i with σ 2 = n σ d 2 \sigma^2=n\sigma_d^2 σ 2 = n σ d 2 and ϱ = n d \varrho=nd ϱ = n d (σ d 2 > 0 \sigma_d^2>0 σ d 2 > 0 whenever 0 < ℓ < 1 0<\ell<1 0 < ℓ < 1 ), gives
P ( F D ∗ ( t ) > s + d ) ≤ exp ( − n d 2 2 ( σ d 2 + M d / 3 ) ) . \mathbb{P}(F_D^*(t)>s+d)\le\exp\left(-\frac{nd^2}{2(\sigma_d^2+Md/3)}\right). P ( F D ∗ ( t ) > s + d ) ≤ exp ( − 2 ( σ d 2 + M d /3 ) n d 2 ) .
Step 2. Exact s k s_k s k . Write d = u t k d=ut_k d = u t k and s k = t k ( 1 + κ ) s_k=t_k(1+\kappa) s k = t k ( 1 + κ ) . The probability is 0 0 0 once s k + d ≥ 1 s_k+d\ge1 s k + d ≥ 1 , so otherwise u t k < 1 ut_k<1 u t k < 1 . Then s k ( 1 + d / 3 ) + d ( 1 + d ) ≤ t k [ ( 1 + κ ) + c u ] s_k(1+d/3)+d(1+d)\le t_k[(1+\kappa)+cu] s k ( 1 + d /3 ) + d ( 1 + d ) ≤ t k [( 1 + κ ) + c u ] . The exponent is therefore at least k a h ( u ) kah(u) k ah ( u ) , with a : = n q / ( 2 B ˉ m ) a:=nq/(2\bar{B}m) a := n q / ( 2 B ˉ m ) and h ( u ) : = u 2 / ( ( 1 + κ ) + c u ) h(u):=u^2/((1+\kappa)+cu) h ( u ) := u 2 / (( 1 + κ ) + c u ) . Step 3. Union and integral. With M D ′ : = max k ( F D ∗ ( t k ) − s k ) + / t k M'_D := \max_k(F_D^*(t_k)-s_k)^+/t_k M D ′ := max k ( F D ∗ ( t k ) − s k ) + / t k (so M D ≤ 1 + κ + M D ′ M_D \le1+\kappa+M'_D M D ≤ 1 + κ + M D ′ ),
P ( M D ′ > u ) ≤ min ( 1 , ∑ k ≥ 1 e − k a h ( u ) ) = φ 0 ( a h ( u ) ) , φ 0 ( r ) : = min ( 1 , 1 e r − 1 ) . \mathbb{P}(M'_D>u)\le\min\left(1,\sum_{k\ge1}e^{-kah(u)}\right)=\varphi_0(ah(u)),\qquad\varphi_0(r):=\min\left(1,\frac{1}{e^r-1}\right). P ( M D ′ > u ) ≤ min ( 1 , k ≥ 1 ∑ e − k ah ( u ) ) = φ 0 ( ah ( u )) , φ 0 ( r ) := min ( 1 , e r − 1 1 ) .
So E [ M D ′ ] ≤ ∫ 0 ∞ φ 0 ( a h ( u ) ) d u \mathbb{E}[M'_D]\le\int_0^\infty\varphi_0(ah(u))\,du E [ M D ′ ] ≤ ∫ 0 ∞ φ 0 ( ah ( u )) d u . The same union bound with the sharper per-point tails T k ( u ) T_k(u) T k ( u ) of Step 1 (with d = u t k d=ut_k d = u t k ) gives the computable form E [ M D ′ ] ≤ J \mathbb{E}[M'_D]\le J E [ M D ′ ] ≤ J .
Step 4. Closed form. Substitute ν = a h ( u ) \nu=\sqrt{ah(u)} ν = ah ( u ) . Since h h h is increasing from 0 0 0 to ∞ \infty ∞ , u = u ( ν ) u=u(\nu) u = u ( ν ) is a bijection, and solving the quadratic gives u ( ν ) ≤ u ˉ ( ν ) : = ν ( 1 + κ ) / a + c ν 2 / a u(\nu)\le\bar u(\nu):=\nu\sqrt{(1+\kappa)/a}+c\nu^2/a u ( ν ) ≤ u ˉ ( ν ) := ν ( 1 + κ ) / a + c ν 2 / a .
Write φ 0 ( ν 2 ) = ∫ ν ∞ ( − d d r φ 0 ( r 2 ) ) d r \varphi_0(\nu^2)=\int_\nu^\infty\left(-\frac{d}{dr}\varphi_0(r^2)\right)\,dr φ 0 ( ν 2 ) = ∫ ν ∞ ( − d r d φ 0 ( r 2 ) ) d r , with a non-negative integrand. Then by Tonelli,
∫ 0 ∞ φ 0 ( ν 2 ) u ′ ( ν ) d ν = ∫ 0 ∞ u ( r ) ( − d d r φ 0 ( r 2 ) ) d r ≤ ∫ 0 ∞ u ˉ ( r ) ( − d d r φ 0 ( r 2 ) ) d r = ∫ 0 ∞ φ 0 ( ν 2 ) u ˉ ′ ( ν ) d ν , \begin{aligned}
\int_0^\infty\varphi_0(\nu^2)u'(\nu)\,d\nu
=\int_0^\infty u(r)\left(-\frac{d}{dr}\varphi_0(r^2)\right)\,dr \\
&\le\int_0^\infty\bar u(r)\left(-\frac{d}{dr}\varphi_0(r^2)\right)\,dr
=\int_0^\infty\varphi_0(\nu^2)\bar u'(\nu)\,d\nu,
\end{aligned} ∫ 0 ∞ φ 0 ( ν 2 ) u ′ ( ν ) d ν = ∫ 0 ∞ u ( r ) ( − d r d φ 0 ( r 2 ) ) d r ≤ ∫ 0 ∞ u ˉ ( r ) ( − d r d φ 0 ( r 2 ) ) d r = ∫ 0 ∞ φ 0 ( ν 2 ) u ˉ ′ ( ν ) d ν ,
using u ( 0 ) = u ˉ ( 0 ) = 0 u(0)=\bar u(0)=0 u ( 0 ) = u ˉ ( 0 ) = 0 . The left side is ∫ 0 ∞ φ 0 ( a h ( u ) ) d u \int_0^\infty\varphi_0(ah(u))\,du ∫ 0 ∞ φ 0 ( ah ( u )) d u .
The last integral is ( 1 + κ ) / a ⋅ I + ( c / a ) ∫ 0 ∞ φ 0 ( r ) d r \sqrt{(1+\kappa)/a}\cdot I+(c/a)\int_0^\infty\varphi_0(r)\,dr ( 1 + κ ) / a ⋅ I + ( c / a ) ∫ 0 ∞ φ 0 ( r ) d r , with I : = ∫ 0 ∞ min ( 1 , 1 / ( e ν − 1 ) ) d ν I:=\int_0^\infty\min(1,1/(e^\nu-1))\,d\nu I := ∫ 0 ∞ min ( 1 , 1/ ( e ν − 1 )) d ν .
∫ 0 ∞ φ 0 = ln 2 + [ ln ( 1 − e − r ) ] ln 2 ∞ = 2 ln 2 \int_0^\infty\varphi_0=\ln2+[\ln(1-e^{-r})]_{\ln2}^\infty=2\ln2 ∫ 0 ∞ φ 0 = ln 2 + [ ln ( 1 − e − r ) ] l n 2 ∞ = 2 ln 2 .
With 1 / a = 2 B ˉ m / ( n ˉ q ) = 2 x 1/a=2\bar Bm/(\bar n q)=2x 1/ a = 2 B ˉ m / ( n ˉ q ) = 2 x , this is 2 I ( 1 + κ ) x + 4 ln 2 ⋅ c x \sqrt{2}I\sqrt{(1+\kappa)x}+4\ln2\cdot cx 2 I ( 1 + κ ) x + 4 ln 2 ⋅ c x , the closed form.
Remark 32 (Constants and variants). The constants are 2 I \sqrt{2}I 2 I and 4 ln 2 4\ln2 4 ln 2 , with I = 1.139773 I=1.139773 I = 1.139773 (numerical quadrature). The exact values are 2 I = 1.611883 \sqrt{2}I=1.611883 2 I = 1.611883 and 4 ln 2 = 2.772589 4\ln2=2.772589 4 ln 2 = 2.772589 , so 1.6120 and 2.7726 are upward roundings. For q = 0.1 q=0.1 q = 0.1 and small κ \kappa κ the closed form reads 1 + κ + 1.612 x + 5.64 x 1+\kappa+1.612\sqrt{x}+5.64x 1 + κ + 1.612 x + 5.64 x , against 7.4 x + 33 x 7.4\sqrt{x}+33x 7.4 x + 33 x in Lemma 30(iii). For all κ ≤ 1 \kappa\le1 κ ≤ 1 and q ≤ 1 q\le1 q ≤ 1 it is at most 1 + κ + 2.28 x + 7.4 x 1+\kappa+2.28\sqrt{x}+7.4x 1 + κ + 2.28 x + 7.4 x . By Step 2, T k ( u ) ≤ e − k a h ( u ) T_k(u)\le e^{-kah(u)} T k ( u ) ≤ e − k ah ( u ) , so J J J never exceeds the closed form. Using B ˉ = B \bar B=B B ˉ = B (or any upper bound on B ˉ \bar B B ˉ ) is valid: every T k ( u ) T_k(u) T k ( u ) , and hence J J J , is increasing in B ˉ \bar B B ˉ .
The reduction cannot do better: proof of Proposition 7
Proposition 7 is a lower bound for the reduction quantity E [ M D ] \mathbb{E}[M_D] E [ M D ] of Proposition 3, not for the FDR. It says how good any bound obtained through Proposition 3 can possibly be.
Model. We use the Poisson-limit calibration model (§ 2 , Lemma 44):
light units have weight 1 and continuous scores, with latent u ∼ U ( 0 , 1 ) u\sim U(0,1) u ∼ U ( 0 , 1 ) ;
one heavy atom has weight κ n \kappa n κn and P P P -mass λ / n \lambda/n λ / n , with λ = ω / κ \lambda=\omega/\kappa λ = ω / κ ;
the heavy atom’s score lies below every light score;
ω : = t 1 / ( 1 − t 1 ) \omega:=t_1/(1-t_1) ω := t 1 / ( 1 − t 1 ) . Then at K = ω − κ K=\omega-\kappa K = ω − κ the heavy window, which includes the test point’s own weight κ \kappa κ , is exactly [ 0 , t 1 ] [0,t_1] [ 0 , t 1 ] ; at K = ω K=\omega K = ω it is [ 0 , t 1 ] [0,t_1] [ 0 , t 1 ] up to O ( κ ) O(\kappa) O ( κ ) . (Numerically, among bottom-placed atoms this ω \omega ω is optimal to leading order.)
Precise statement. Fix q ∈ ( 0 , 1 ) q\in(0,1) q ∈ ( 0 , 1 ) and m ≥ 2 m\ge2 m ≥ 2 . Then, as κ → 0 \kappa\to0 κ → 0 ,
E [ M D ] − 1 = C m r e d κ + O m ( κ ) , C m r e d : = m 2 π q ( 1 − q m ) 3 / 2 = [ ( 1 − q ) ( 1 − q m ) ] 1 / 2 m . \mathbb{E}[M_D]-1=C_m^{\mathrm{red}}\sqrt{\kappa}+O_m(\kappa),\qquad C_m^{\mathrm{red}}:=\sqrt{\frac{m}{2\pi q}}\left(1-\frac{q}{m}\right)^{3/2}=\frac{\left[(1-q)\left(1-\frac{q}{m}\right)\right]^{1/2}}{m}. E [ M D ] − 1 = C m red κ + O m ( κ ) , C m red := 2 π q m ( 1 − m q ) 3/2 = m [ ( 1 − q ) ( 1 − m q ) ] 1/2 .
In the units x : = B ˉ m / ( n ˉ q ) x:=\bar Bm/(\bar n q) x := B ˉ m / ( n ˉ q ) of Theorem 6 , with B ˉ = κ n / E P w \bar B=\kappa n/\mathbb{E}_P w B ˉ = κn / E P w and E P w → 1 + ω \mathbb{E}_P w\to1+\omega E P w → 1 + ω , this reads
E [ M D ] − 1 = 1 − 1 / m 2 π x ( 1 + o ( 1 ) ) . \mathbb{E}[M_D]-1=\frac{1-1/m}{\sqrt{2\pi}}\sqrt{x}\,(1+o(1)). E [ M D ] − 1 = 2 π 1 − 1/ m x ( 1 + o ( 1 )) .
So for every m m m , the supremum of E [ M D ] − 1 \mathbb{E}[M_D]-1 E [ M D ] − 1 over laws with weight ratio B ˉ \bar B B ˉ is at least ( 1 − 1 / m ) / 2 π B ˉ m / ( n ˉ q ) (1-1/m)/\sqrt{2\pi}\sqrt{\bar Bm/(\bar n q)} ( 1 − 1/ m ) / 2 π B ˉ m / ( n ˉ q ) in the Poisson limit. Any bound proved through Proposition 3 therefore has leading constant at least 0.399 ( 1 − 1 / m ) 0.399(1-1/m) 0.399 ( 1 − 1/ m ) .
From finite n n n to the Poisson limit. For fixed κ \kappa κ , the finite-n n n laws converge to the Poisson-limit law (the coupling of Lemma 44). M D ≤ m / q M_D \le m/q M D ≤ m / q is bounded and is a continuous function of ( F D ∗ ( t k ) ) k (F_D^*(t_k))_k ( F D ∗ ( t k ) ) k , so E n [ M D ] \mathbb{E}_n[M_D] E n [ M D ] converges to its Poisson-limit value. This uses the convergence F D ∗ ( t k ) → F N ( t k ) F_D^*(t_k) \to F_N(t_k) F D ∗ ( t k ) → F N ( t k ) in probability established in Steps 1–5 of the proof of Lemma 44; that lemma is stated for the FWER and for g h ∈ ( 0 , 1 ) g_h \in(0,1) g h ∈ ( 0 , 1 ) , but those steps apply unchanged to an atom at g = 0 g=0 g = 0 . The rest is the continuous mapping theorem and bounded convergence [21 Ch. 2] . A bound valid at every finite n n n therefore applies to the limit. This uses only the convergence in distribution that Lemma 44 provides.
Proof of Proposition 7. Let K : = κ N K := \kappa N K := κ N , with N ∼ Poisson ( λ ) N \sim\operatorname{Poisson}(\lambda) N ∼ Poisson ( λ ) . In the Poisson limit, write Q H = ω / ( 1 + ω ) Q_H = \omega/(1+\omega) Q H = ω / ( 1 + ω ) for the heavy Q Q Q -mass and Q L = 1 / ( 1 + ω ) Q_L = 1/(1+\omega) Q L = 1/ ( 1 + ω ) for the light Q Q Q -density, and write F K F_K F K for F D ∗ F_D^* F D ∗ , which depends on the calibration data through K K K .
Step 1. Conditional law.
A heavy test point carries weight K + κ K+\kappa K + κ , including its own, and nothing lies below it. So p ∗ ∼ Unif [ 0 , h K ] p^* \sim\operatorname{Unif}[0,h_K] p ∗ ∼ Unif [ 0 , h K ] , with h K : = ( K + κ ) / ( 1 + K + κ ) h_K := (K+\kappa)/(1+K+\kappa) h K := ( K + κ ) / ( 1 + K + κ ) .
A light test point with latent u u u has all heavy weight below it, so p ∗ = ( K + u ) / ( 1 + K ) p^*=(K+u)/(1+K) p ∗ = ( K + u ) / ( 1 + K ) .
Hence, for t ≤ q t \le q t ≤ q ,
F K ( t ) = Q H min ( 1 , t / h K ) + Q L ( t ( 1 + K ) − K ) + . F_K(t)=Q_H\min(1,t/h_K)+Q_L\left(t(1+K)-K\right)^+. F K ( t ) = Q H min ( 1 , t / h K ) + Q L ( t ( 1 + K ) − K ) + .
Step 2. Near K = ω K=\omega K = ω . Fix δ 0 > 0 \delta_0>0 δ 0 > 0 small, so that K + κ ≥ ω / 2 K+\kappa\ge\omega/2 K + κ ≥ ω /2 and h K < t 2 h_K<t_2 h K < t 2 when ∣ K − ω ∣ ≤ δ 0 |K-\omega|\le\delta_0 ∣ K − ω ∣ ≤ δ 0 and κ \kappa κ is small. Put ρ k ( K ) : = F K ( t k ) / t k − 1 \rho_k(K):=F_K(t_k)/t_k-1 ρ k ( K ) := F K ( t k ) / t k − 1 .
For k ≥ 2 k\ge2 k ≥ 2 : t k > h K > K / ( 1 + K ) t_k>h_K>K/(1+K) t k > h K > K / ( 1 + K ) , so F K ( t k ) = Q H + Q L ( t k ( 1 + K ) − K ) F_K(t_k)=Q_H+Q_L(t_k(1+K)-K) F K ( t k ) = Q H + Q L ( t k ( 1 + K ) − K ) . This gives ρ k ( K ) = − a k ( K − ω ) \rho_k(K)=-a_k(K-\omega) ρ k ( K ) = − a k ( K − ω ) exactly, with a k : = Q L ( 1 − t k ) / t k = ( 1 − t 1 ) ( 1 − t k ) / t k a_k:=Q_L(1-t_k)/t_k=(1-t_1)(1-t_k)/t_k a k := Q L ( 1 − t k ) / t k = ( 1 − t 1 ) ( 1 − t k ) / t k . (Note 1 / ( 1 + ω ) = 1 − t 1 1/(1+\omega)=1-t_1 1/ ( 1 + ω ) = 1 − t 1 , and ρ k ( ω ) = 0 \rho_k(\omega)=0 ρ k ( ω ) = 0 .)
For k = 1 k=1 k = 1 there are three cases.
K ≤ ω − κ K\le\omega-\kappa K ≤ ω − κ : then h K ≤ t 1 h_K\le t_1 h K ≤ t 1 , and ρ 1 = − ( 1 − t 1 ) 2 ( K − ω ) / t 1 \rho_1=-(1-t_1)^2(K-\omega)/t_1 ρ 1 = − ( 1 − t 1 ) 2 ( K − ω ) / t 1 exactly.
K ≥ ω K\ge\omega K ≥ ω : then ρ 1 = Q H / h K − 1 \rho_1=Q_H/h_K-1 ρ 1 = Q H / h K − 1 . This equals − κ / ( ( 1 + ω ) ( ω + κ ) ) = O ( κ ) -\kappa/((1+\omega)(\omega+\kappa))=O(\kappa) − κ / (( 1 + ω ) ( ω + κ )) = O ( κ ) at K = ω K=\omega K = ω , has derivative − Q H / ( ω + κ ) 2 = − ( 1 − t 1 ) 2 / t 1 + O ( κ ) -Q_H/(\omega+\kappa)^2=-(1-t_1)^2/t_1+O(\kappa) − Q H / ( ω + κ ) 2 = − ( 1 − t 1 ) 2 / t 1 + O ( κ ) there, and has a bounded second derivative on the window.
On [ ω − κ , ω ] [\omega-\kappa,\omega] [ ω − κ , ω ] , ρ 1 \rho_1 ρ 1 interpolates between the two cases within O ( κ ) O(\kappa) O ( κ ) .
So with a 1 : = ( 1 − t 1 ) 2 / t 1 a_1:=(1-t_1)^2/t_1 a 1 := ( 1 − t 1 ) 2 / t 1 , ρ k ( K ) = − a k ( K − ω ) + e k ( K ) \rho_k(K)=-a_k(K-\omega)+e_k(K) ρ k ( K ) = − a k ( K − ω ) + e k ( K ) with ∣ e k ( K ) ∣ ≤ C ′ ( κ + ( K − ω ) 2 ) |e_k(K)|\le C'(\kappa+(K-\omega)^2) ∣ e k ( K ) ∣ ≤ C ′ ( κ + ( K − ω ) 2 ) , uniformly in k k k .
Step 3. The maximum. a 1 / a k = k ( 1 − t 1 ) / ( 1 − t k ) ≥ 1 a_1/a_k=k(1-t_1)/(1-t_k)\ge1 a 1 / a k = k ( 1 − t 1 ) / ( 1 − t k ) ≥ 1 , and a k a_k a k decreases in k k k . So max k ( − a k ( K − ω ) ) = a 1 ( ω − K ) + − a m ( K − ω ) + \max_k(-a_k(K-\omega))=a_1(\omega-K)^+-a_m(K-\omega)^+ max k ( − a k ( K − ω )) = a 1 ( ω − K ) + − a m ( K − ω ) + . The maximum is 1-Lipschitz in the sup norm, so on ∣ K − ω ∣ ≤ δ 0 |K-\omega|\le\delta_0 ∣ K − ω ∣ ≤ δ 0 ,
M D − 1 = a 1 ( ω − K ) + − a m ( K − ω ) + + O ( κ + ( K − ω ) 2 ) . M_D-1=a_1(\omega-K)^+-a_m(K-\omega)^++O\left(\kappa+(K-\omega)^2\right). M D − 1 = a 1 ( ω − K ) + − a m ( K − ω ) + + O ( κ + ( K − ω ) 2 ) .
Step 4. Expectation.
Off the window, M D − 1 ∈ [ − 1 , m / q ] M_D-1\in[-1,m/q] M D − 1 ∈ [ − 1 , m / q ] , and P ( ∣ K − ω ∣ > δ 0 ) = P ( ∣ N − λ ∣ > δ 0 / κ ) ≤ 2 e − c 1 / κ \mathbb{P}(|K-\omega|>\delta_0)=\mathbb{P}(|N-\lambda|>\delta_0/\kappa)\le2e^{-c_1/\kappa} P ( ∣ K − ω ∣ > δ 0 ) = P ( ∣ N − λ ∣ > δ 0 / κ ) ≤ 2 e − c 1 / κ for some c 1 > 0 c_1>0 c 1 > 0 , by Lemma 29 with c = δ 0 / ω c=\delta_0/\omega c = δ 0 / ω .
E ( K − ω ) = 0 \mathbb{E}(K-\omega)=0 E ( K − ω ) = 0 and E ( K − ω ) 2 = κ ω \mathbb{E}(K-\omega)^2=\kappa\omega E ( K − ω ) 2 = κω , so
E [ M D ] − 1 = ( a 1 − a m ) E ( K − ω ) + + O ( κ ) . \mathbb{E}[M_D]-1=(a_1-a_m)\mathbb{E}(K-\omega)^++O(\kappa). E [ M D ] − 1 = ( a 1 − a m ) E ( K − ω ) + + O ( κ ) .
The exact Poisson identity E [ ( N − λ ) 1 { N ≥ j } ] = λ P ( N = j − 1 ) \mathbb{E}[(N-\lambda)1\{N\ge j\}]=\lambda\mathbb{P}(N=j-1) E [( N − λ ) 1 { N ≥ j }] = λ P ( N = j − 1 ) , with j = ⌊ λ ⌋ + 1 j=\lfloor\lambda\rfloor+1 j = ⌊ λ ⌋ + 1 (as in the proof of Theorem 9), gives E ( K − ω ) + = κ λ P ( N = ⌊ λ ⌋ ) = ω P ( N = ⌊ λ ⌋ ) \mathbb{E}(K-\omega)^+=\kappa\lambda\mathbb{P}(N=\lfloor\lambda\rfloor)=\omega\mathbb{P}(N=\lfloor\lambda\rfloor) E ( K − ω ) + = κλ P ( N = ⌊ λ ⌋) = ω P ( N = ⌊ λ ⌋) .
By Stirling’s formula, j ! = 2 π j ( j / e ) j ( 1 + O ( 1 / j ) ) j!=\sqrt{2\pi j}(j/e)^j(1+O(1/j)) j ! = 2 π j ( j / e ) j ( 1 + O ( 1/ j )) . With j = ⌊ λ ⌋ = λ − f j=\lfloor\lambda\rfloor=\lambda-f j = ⌊ λ ⌋ = λ − f , f ∈ [ 0 , 1 ) f\in[0,1) f ∈ [ 0 , 1 ) , this gives P ( N = j ) = ( 2 π j ) − 1 / 2 exp { j − λ + j log ( λ / j ) } ( 1 + O ( 1 / λ ) ) \mathbb{P}(N=j)=(2\pi j)^{-1/2}\exp\{j-\lambda+j\log(\lambda/j)\}(1+O(1/\lambda)) P ( N = j ) = ( 2 π j ) − 1/2 exp { j − λ + j log ( λ / j )} ( 1 + O ( 1/ λ )) , and j log ( λ / j ) = f + O ( 1 / λ ) j\log(\lambda/j)=f+O(1/\lambda) j log ( λ / j ) = f + O ( 1/ λ ) . So P ( N = ⌊ λ ⌋ ) = ( 2 π λ ) − 1 / 2 ( 1 + O ( 1 / λ ) ) \mathbb{P}(N=\lfloor\lambda\rfloor)=(2\pi\lambda)^{-1/2}(1+O(1/\lambda)) P ( N = ⌊ λ ⌋) = ( 2 π λ ) − 1/2 ( 1 + O ( 1/ λ )) , and E ( K − ω ) + = κ ω / 2 π + O ( κ 3 / 2 ) \mathbb{E}(K-\omega)^+=\sqrt{\kappa\omega/\sqrt{2\pi}}+O(\kappa^{3/2}) E ( K − ω ) + = κω / 2 π + O ( κ 3/2 ) .
Step 5. Algebra. With ω = t 1 / ( 1 − t 1 ) \omega=t_1/(1-t_1) ω = t 1 / ( 1 − t 1 ) and t 1 = q / m t_1=q/m t 1 = q / m ,
( a 1 − a m ) ω = ( 1 − t 1 ) 3 / 2 t 1 − 1 / 2 − ( 1 − q ) ( 1 − t 1 ) 1 / 2 t 1 1 / 2 / q (a_1-a_m)\sqrt{\omega}=(1-t_1)^{3/2}t_1^{-1/2}-(1-q)(1-t_1)^{1/2}t_1^{1/2}/q ( a 1 − a m ) ω = ( 1 − t 1 ) 3/2 t 1 − 1/2 − ( 1 − q ) ( 1 − t 1 ) 1/2 t 1 1/2 / q
which, divided by 2 π \sqrt{2\pi} 2 π , gives C m r e d C_m^{\mathrm{red}} C m red .
Units: κ = ( 1 + ω ) B ‾ / n ⋅ ( 1 + o ( 1 ) ) \kappa=(1+\omega)\overline{B}/n\cdot(1+o(1)) κ = ( 1 + ω ) B / n ⋅ ( 1 + o ( 1 )) and 1 + ω = ( 1 − t 1 ) − 1 1+\omega=(1-t_1)^{-1} 1 + ω = ( 1 − t 1 ) − 1 .
So C m r e d κ = x / ( 2 π ) [ ( 1 − t 1 ) − ( 1 − q ) / m ] = x / ( 2 π ) ( 1 − 1 / m ) C_m^{\mathrm{red}}\sqrt{\kappa}=\sqrt{x/(2\pi)}[(1-t_1)-(1-q)/m]=\sqrt{x/(2\pi)}(1-1/m) C m red κ = x / ( 2 π ) [( 1 − t 1 ) − ( 1 − q ) / m ] = x / ( 2 π ) ( 1 − 1/ m ) .
Remark 33 (Scope of Proposition 7). This is a Poisson-limit statement. Heuristically, the O m ( κ ) O_m(\kappa) O m ( κ ) remainder is small once κ m / q ≪ 1 \kappa m/q\ll1 κm / q ≪ 1 , since the spread of K K K is κ ω \sqrt{\kappa\omega} κω (numerically, at m = 512 m=512 m = 512 , q = 0.05 q=0.05 q = 0.05 , κ = 10 − 6 \kappa=10^{-6} κ = 1 0 − 6 the ratio to the asymptotic value is already 0.997 0.997 0.997 ). It is a lower bound only: the single bottom atom is not the worst case for E [ M D ] \mathbb{E}[M_D] E [ M D ] . Several heavy cells just above the bottom, stacked so that each fills one grid step, do better numerically: at m = 8 m=8 m = 8 , two cells give 0.412 0.412 0.412 –0.422 0.422 0.422 against 0.349 0.349 0.349 here; at m = 32 m=32 m = 32 , four cells give 0.518 ± 0.0005 0.518\pm0.0005 0.518 ± 0.0005 (Poisson Monte Carlo) against 0.387 0.387 0.387 , and the value still rises with more cells. Each cell puts its kink at a different grid point, so M D M_D M D becomes the maximum of a random walk rather than one positive part. So the true supremum of the reduction’s coefficient is unknown: it is at least ( 1 − 1 / m ) / 2 π (1-1/m)/\sqrt{2\pi} ( 1 − 1/ m ) / 2 π (proved here) and, numerically, at least 0.518 0.518 0.518 at m = 32 m=32 m = 32 .
Proofs for the κ \sqrt{\kappa} κ regime in the Poisson limit
All results in this appendix are statements about the Poisson-limit model: the heavy calibration count is exactly Poisson, and the light calibration scores have exactly the uniform CDF. The expansions are as κ → 0 \kappa\to0 κ → 0 within that model. That the finite-n n n FDR converges to its Poisson-limit value as n → ∞ n\to\infty n → ∞ at fixed κ = B / n \kappa=B/n κ = B / n is proved for the global null (Lemma 44), not for the decoy structures of this appendix; Theorem 12 gives finite-n n n support. Numerical evaluations of the formulas below (the configuration of Example 10 , finite-n n n simulations, and optimised structures switching at t 2 t_2 t 2 ) are not reproduced here.
Setting
We work in the Poisson-limit selection model with one heavy cell.
Light covariates have weight 1 1 1 and latent score quantile u ∼ U ( 0 , 1 ) u\sim U(0,1) u ∼ U ( 0 , 1 ) , with Y = y ( X ) Y=y(X) Y = y ( X ) deterministic and score V = Y V=Y V = Y .
Light units with u ≤ 1 − ζ u\le1-\zeta u ≤ 1 − ζ get oracle thresholds c = Y c=Y c = Y . They are null, and their p-value is f K ( u ) f_K(u) f K ( u ) .
Light units with u > 1 − ζ u>1-\zeta u > 1 − ζ are decoys: their threshold quantile is v 0 < u v_0<u v 0 < u , so they are non-null, and all of them get the same p-value p D = f K ( v 0 ) p_D=f_K(v_0) p D = f K ( v 0 ) .
One heavy cell at score quantile g < v 0 g<v_0 g < v 0 : weight κ n \kappa n κn , P P P -mass λ / n \lambda/n λ / n with λ = ω / κ \lambda=\omega/\kappa λ = ω / κ , Y = g Y=g Y = g and c = Y c=Y c = Y (null). Its calibration total is K = κ N K=\kappa N K = κ N with N ∼ Poisson ( ω / κ ) N\sim\operatorname{Poisson}(\omega/\kappa) N ∼ Poisson ( ω / κ ) , so K = ω + κ ω Z + O ( κ ) K=\omega+\sqrt{\kappa\omega}Z+O(\kappa) K = ω + κω Z + O ( κ ) in law, Z Z Z standard normal.
f K ( u ) : = ( u + K 1 { g < u } ) / ( 1 + K ) f_K(u):=(u+K\mathbf{1}\{g<u\})/(1+K) f K ( u ) := ( u + K 1 { g < u }) / ( 1 + K ) .
Under Q Q Q the light units have total mass 1 / Ξ 1/\Xi 1/Ξ and the heavy cell has mass ω / Ξ \omega/\Xi ω /Ξ , where Ξ : = 1 + ω \Xi:=1+\omega Ξ := 1 + ω . In the deterministic-K K K quantities below, a heavy test unit’s p-value is spread uniformly over the heavy window [ g / ( 1 + K ) , ( g + K ) / ( 1 + K ) ] [g/(1+K),(g+K)/(1+K)] [ g / ( 1 + K ) , ( g + K ) / ( 1 + K )] , the gap of the light map u ↦ f K ( u ) u\mapsto f_K(u) u ↦ f K ( u ) at u = g u=g u = g ; the exact window at K K K is this window at K + κ K+\kappa K + κ (Step 1 of the proof of Theorem 9 ).
Given K K K , the test units are i.i.d. (of three types: light null, decoy, heavy), and the decoys’ p-value is the deterministic atom p D = f K ( v 0 ) p_D=f_K(v_0) p D = f K ( v 0 ) . Let k 0 ∈ { 1 , … , m } k_0\in\{1,\ldots,m\} k 0 ∈ { 1 , … , m } be the grid index that the decoy atom crosses at the switch point k ∗ k^* k ∗ defined below. For s ∈ { − , + } s\in\{-,+\} s ∈ { − , + } define the deterministic-K K K FDR on each side of the switch,
Φ det ( K , s ) : = m ∑ k = 1 m 1 k A k ( K ) P K , s ( R 1 = k ) , \Phi_{\det}(K,s):=m\sum_{k=1}^{m}\frac{1}{k}A_k(K)\mathbb{P}_{K,s}(R_1=k), Φ d e t ( K , s ) := m k = 1 ∑ m k 1 A k ( K ) P K , s ( R 1 = k ) ,
where
A k ( K ) : = P ( n u l l , p ≤ t k ∣ K ) A_k(K) := \mathbb{P}(\mathrm{null},\ p \le t_k \mid K) A k ( K ) := P ( null , p ≤ t k ∣ K ) (the Poisson-limit counterpart of A k ( D ) A_k(D) A k ( D ) );
R 1 R_1 R 1 is the BH count with a forced zero among m − 1 m-1 m − 1 i.i.d. units, as in Lemma 2 ;
under P K , s \mathbb{P}_{K,s} P K , s , the decoy atom is counted in the BH counts at t k t_k t k for k ≥ k 0 k \ge k_0 k ≥ k 0 (s = − s=- s = − ), or for k ≥ k 0 + 1 k \ge k_0+1 k ≥ k 0 + 1 (s = + s=+ s = + ; for k 0 = m k_0=m k 0 = m it is then not counted at all).
We write P s : = P ω , s \mathbb{P}_s := \mathbb{P}_{\omega,s} P s := P ω , s and E s \mathbb{E}_s E s for the corresponding expectation, and φ \varphi φ for the standard normal density.
Saturation at the mean
Proof of Lemma 8. Assume f ω ( 1 − ζ ) ≥ q f_\omega(1-\zeta) \ge q f ω ( 1 − ζ ) ≥ q , so all decoys’ oracle p-values lie above q q q . We show first that A k ( ω ) = t k A_k(\omega) = t_k A k ( ω ) = t k for every k ≤ m k \le m k ≤ m .
At K = ω K=\omega K = ω the full oracle law (all light u u u , plus the heavy cell) is exactly uniform on [ 0 , 1 ] [0,1] [ 0 , 1 ] . This is the deterministic-K K K version of exact uniformity (Lemma 1), as in the global-null multi-cell expansion of the Poisson-limit model at Δ K = 0 \Delta K=0 Δ K = 0 . For the structure of Theorem 35 the computation is written out in Step 1 of its proof.
Below q q q only null units contribute.
So A k ( ω ) = P ( p ∗ ≤ t k ∣ K = ω ) = t k A_k(\omega) = \mathbb{P}(p^* \le t_k \mid K=\omega) = t_k A k ( ω ) = P ( p ∗ ≤ t k ∣ K = ω ) = t k for k ≤ m k \le m k ≤ m . Given K K K the units are i.i.d., so the leave-one-out identity (Lemma 66 ) applies conditionally on K K K and gives, for both s s s ,
Φ d e t ( ω , s ) = m ∑ k = 1 m 1 k q k m P s ( R 1 = k ) = q ∑ k = 1 m P s ( R 1 = k ) = q . \Phi_{\mathrm{det}}(\omega,s) = m \sum_{k=1}^{m} \frac{1}{k}\frac{qk}{m}\mathbb{P}_s(R_1=k) = q\sum_{k=1}^{m}\mathbb{P}_s(R_1=k) = q. Φ det ( ω , s ) = m k = 1 ∑ m k 1 m q k P s ( R 1 = k ) = q k = 1 ∑ m P s ( R 1 = k ) = q .
The κ \sqrt{\kappa} κ expansion
Full statement of Theorem 9. Write FDR ( κ ) \operatorname{FDR}(\kappa) FDR ( κ ) for the FDR of the Poisson-limit model at κ \kappa κ . Assume:
(a) 1 ≤ k 0 ≤ m 1 \le k_0 \le m 1 ≤ k 0 ≤ m (Theorem 35 writes out the case k 0 = 2 k_0=2 k 0 = 2 , z 0 = 0 z_0=0 z 0 = 0 , with explicit derivatives);
(b) Lemma 8 ’s condition, strengthened to f ω ( 1 − ζ ) > q f_\omega(1-\zeta)>q f ω ( 1 − ζ ) > q ; in particular f ω ( 1 − ζ ) ≠ t k f_\omega(1-\zeta)\ne t_k f ω ( 1 − ζ ) = t k for every k k k . (With equality, A m A_m A m has a kink at K = ω K=\omega K = ω , its one-sided slopes differ by ( 1 − q ) / ( 1 + ω ) (1-q)/(1+\omega) ( 1 − q ) / ( 1 + ω ) , and h s h_s h s below is undefined.)
(c) genericity at K = ω K=\omega K = ω for the other breakpoints: no grid point is an edge of the heavy window or a jump of the light map;
(d) v 0 > g v_0>g v 0 > g .
Let the decoy threshold depend on κ \kappa κ , v 0 = v 0 ( κ ) v_0=v_0(\kappa) v 0 = v 0 ( κ ) , so that the switch point k ∗ k^* k ∗ (the solution of f k ∗ ( v 0 ) = t k 0 f_{k^*}(v_0)=t_{k_0} f k ∗ ( v 0 ) = t k 0 ) satisfies k ∗ − ω = z 0 κ ω k^*-\omega=z_0\sqrt{\kappa\omega} k ∗ − ω = z 0 κω , with z 0 z_0 z 0 in a compact set. In particular z 0 = 0 z_0=0 z 0 = 0 is allowed: then v 0 = t k 0 ( 1 + ω ) − ω v_0=t_{k_0}(1+\omega)-\omega v 0 = t k 0 ( 1 + ω ) − ω , which does not depend on κ \kappa κ . No assumption on the position of k ∗ k^* k ∗ relative to K w K_w K w (the value of K K K at which the heavy window’s top edge crosses t k 0 t_{k_0} t k 0 ) is needed: the proof uses only K w ≠ ω K_w\ne\omega K w = ω , which is part of (c). Then, as κ → 0 \kappa\to0 κ → 0 , uniformly in z 0 z_0 z 0 ,
FDR ( κ ) = q + κ ω ( h + − h − ) φ ( z 0 ) + O ( κ ) , h s : = ∂ K Φ d e t ( K , s ) ∣ K = ω . (11) \operatorname{FDR}(\kappa)=q+\sqrt{\kappa\omega}\,(h_+-h_-)\varphi(z_0)+O(\kappa),\qquad h_s:=\left.\partial_K\Phi_{\mathrm{det}}(K,s)\right|_{K=\omega}. \tag*{(11)} FDR ( κ ) = q + κω ( h + − h − ) φ ( z 0 ) + O ( κ ) , h s := ∂ K Φ det ( K , s ) ∣ K = ω . (11)
By Lemma 8 the terms containing ∂ K P K , s ( R 1 = k ) \partial_K\mathbb{P}_{K,s}(R_1=k) ∂ K P K , s ( R 1 = k ) cancel at K = ω K=\omega K = ω : since A k ( ω ) = t k A_k(\omega)=t_k A k ( ω ) = t k , they add up to m ∑ k 1 k t k ∂ K P K , s ( R 1 = k ) = q ∂ K ∑ k P K , s ( R 1 = k ) = 0 m\sum_k \frac{1}{k}t_k\partial_K\mathbb{P}_{K,s}(R_1=k)=q\partial_K\sum_k\mathbb{P}_{K,s}(R_1=k)=0 m ∑ k k 1 t k ∂ K P K , s ( R 1 = k ) = q ∂ K ∑ k P K , s ( R 1 = k ) = 0 . Hence
h s = m ∑ k = 1 m 1 k A k ′ ( ω ) P s ( R 1 = k ) . h_s=m\sum_{k=1}^{m}\frac{1}{k}A_k'(\omega)\mathbb{P}_s(R_1=k). h s = m k = 1 ∑ m k 1 A k ′ ( ω ) P s ( R 1 = k ) .
Proof of Theorem 9. Since v 0 > g v_0 > g v 0 > g , p D = f K ( v 0 ) = ( v 0 + K ) / ( 1 + K ) p_D = f_K(v_0) = (v_0 + K)/(1 + K) p D = f K ( v 0 ) = ( v 0 + K ) / ( 1 + K ) , which is increasing in K K K . Let k ∗ ∗ < k ∗ k^{**} < k^* k ∗∗ < k ∗ be the value of K K K at which p D = t k 0 − 1 p_D = t_{k_0-1} p D = t k 0 − 1 (if k 0 ≥ 2 k_0 \ge2 k 0 ≥ 2 and this value is positive), and K ↑ > k ∗ K^{\uparrow} > k^* K ↑ > k ∗ the value at which p D = t k 0 + 1 p_D = t_{k_0+1} p D = t k 0 + 1 (if k 0 < m k_0 < m k 0 < m ). Then the decoy atom is counted from t k 0 t_{k_0} t k 0 on when k ∗ ∗ < K ≤ k ∗ k^{**} < K \le k^* k ∗∗ < K ≤ k ∗ , and from t k 0 + 1 t_{k_0+1} t k 0 + 1 on (for k 0 = m k_0 = m k 0 = m : not at all) when k ∗ < K ≤ K ↑ k^* < K \le K^{\uparrow} k ∗ < K ≤ K ↑ . Both ω − k ∗ ∗ \omega- k^{**} ω − k ∗∗ and K ↑ − ω K^{\uparrow} - \omega K ↑ − ω are bounded below by positive constants for small κ \kappa κ , because k ∗ − ω → 0 k^* - \omega\to0 k ∗ − ω → 0 while k ∗ − k ∗ ∗ k^* - k^{**} k ∗ − k ∗∗ and K ↑ − k ∗ K^{\uparrow} - k^* K ↑ − k ∗ stay bounded away from 0.
Step 1 (conditioning on K K K ). Let FDR ( K ) \operatorname{FDR}(K) FDR ( K ) denote the FDR given K K K . For ∣ K − ω ∣ ≤ ω / 2 |K - \omega| \le\omega/2 ∣ K − ω ∣ ≤ ω /2 with k ∗ ∗ < K ≤ K ↑ k^{**} < K \le K^{\uparrow} k ∗∗ < K ≤ K ↑ (drop the first condition if k ∗ ∗ k^{**} k ∗∗ is not defined or not positive, and the second if k 0 = m k_0 = m k 0 = m ),
FDR ( K ) = Φ d e t ( K , side ( K ) ) + O ( κ ) uniformly , side ( K ) = − ⟺ K ≤ k ∗ . \operatorname{FDR}(K) = \Phi_{\mathrm{det}}(K,\operatorname{side}(K)) + O(\kappa) \quad\text{uniformly}, \qquad\operatorname{side}(K) = - \Longleftrightarrow K \le k^*. FDR ( K ) = Φ det ( K , side ( K )) + O ( κ ) uniformly , side ( K ) = − ⟺ K ≤ k ∗ .
(At K = k ∗ K = k^* K = k ∗ , p D = t k 0 p_D = t_{k_0} p D = t k 0 is counted, because BH rejects p ≤ t p \le t p ≤ t .) The exact heavy window at K K K is the limiting window at K + κ K + \kappa K + κ , and the test masses agree exactly. The restriction ∣ K − ω ∣ ≤ ω / 2 |K - \omega| \le\omega/2 ∣ K − ω ∣ ≤ ω /2 is needed: as K → 0 K \to0 K → 0 the heavy window shrinks and the Lipschitz constant grows like 1 / K 1/K 1/ K . The event { ∣ K − ω ∣ > ω / 2 } \{|K - \omega| > \omega/2\} { ∣ K − ω ∣ > ω /2 } has exponentially small Poisson mass, by Lemma 29. (A numerical check in the configuration of Example 10 gives sup ∣ FDR ( K ) − Φ d e t ( K , side ( K ) ) ∣ / κ = 0.054 \sup|\operatorname{FDR}(K) - \Phi_{\mathrm{det}}(K,\operatorname{side}(K))|/\kappa= 0.054 sup ∣ FDR ( K ) − Φ det ( K , side ( K )) ∣/ κ = 0.054 over ± 6 \pm6 ± 6 standard deviations around ω \omega ω .)
The relation fails for K < k ∗ ∗ K < k^{**} K < k ∗∗ , where k ∗ ∗ < k ∗ k^{**} < k^* k ∗∗ < k ∗ is the point at which the decoy atom drops into bin k 0 − 1 k_0 - 1 k 0 − 1 ; there further switches follow. On that region ∣ FDR ( K ) − Φ d e t ( K , − ) ∣ ≤ C ∣ K − ω ∣ |\operatorname{FDR}(K) - \Phi_{\mathrm{det}}(K,-)| \le C|K - \omega| ∣ FDR ( K ) − Φ det ( K , − ) ∣ ≤ C ∣ K − ω ∣ , and the event { K < k ∗ ∗ } \{K < k^{**}\} { K < k ∗∗ } sits at distance ω − k ∗ ∗ > 0 \omega- k^{**} > 0 ω − k ∗∗ > 0 from ω \omega ω , so Step 4 absorbs it. For k 0 < m k_0 < m k 0 < m the relation also fails for K > K ↑ K > K^{\uparrow} K > K ↑ , where the atom is counted only from t k 0 + 2 t_{k_0+2} t k 0 + 2 on, or later. There FDR ( K ) \operatorname{FDR}(K) FDR ( K ) and Φ d e t ( K , + ) \Phi_{\mathrm{det}}(K,+) Φ det ( K , + ) both lie in [ 0 , 1 ] [0,1] [ 0 , 1 ] (each is an FDR), and { K > K ↑ } \{K > K^{\uparrow}\} { K > K ↑ } sits at distance K ↑ − ω > 0 K^{\uparrow} - \omega> 0 K ↑ − ω > 0 from ω \omega ω , so Step 4 absorbs it in the same way.
Step 2 (smoothness). Each Φ d e t ( ⋅ , s ) \Phi_{\mathrm{det}}(\mathord\cdot,s) Φ det ( ⋅ , s ) is smooth near ω \omega ω under the remaining genericity (b)–(c). Since Φ d e t \Phi_{\mathrm{det}} Φ det is bounded and Φ d e t ( ω , s ) = q \Phi_{\mathrm{det}}(\omega,s) = q Φ det ( ω , s ) = q (Lemma 8 ),
∣ Φ d e t ( K , s ) − q − h s ( K − ω ) ∣ ≤ C ( K − ω ) 2 for all K , \left|\Phi_{\mathrm{det}}(K,s) - q - h_s(K - \omega)\right| \le C(K - \omega)^2 \quad\text{for all } K, ∣ Φ det ( K , s ) − q − h s ( K − ω ) ∣ ≤ C ( K − ω ) 2 for all K ,
with C C C of order 1 / δ 2 1/\delta^2 1/ δ 2 , where δ \delta δ is the distance from ω \omega ω to the nearest breakpoint.
Step 3 (averaging; an exact identity). E ( K − ω ) = 0 \mathbb{E}(K - \omega) = 0 E ( K − ω ) = 0 , and for N ∼ Poisson ( λ ) N \sim\operatorname{Poisson}(\lambda) N ∼ Poisson ( λ ) , E [ ( N − λ ) 1 { N ≥ j } ] = λ P ( N = j − 1 ) \mathbb{E}[(N - \lambda)1\{N \ge j\}] = \lambda\mathbb{P}(N = j - 1) E [( N − λ ) 1 { N ≥ j }] = λ P ( N = j − 1 ) (from n P ( N = n ) = λ P ( N = n − 1 ) n\mathbb{P}(N = n) = \lambda\mathbb{P}(N = n - 1) n P ( N = n ) = λ P ( N = n − 1 ) ). Taking j = ⌊ k ∗ / κ ⌋ + 1 j = \lfloor k^*/\kappa\rfloor+ 1 j = ⌊ k ∗ / κ ⌋ + 1 , so that { K > k ∗ } = { N ≥ j } \{K > k^*\} = \{N \ge j\} { K > k ∗ } = { N ≥ j } ,
E [ ( K − ω ) 1 { K > k ∗ } ] = ω P ( N = ⌊ k ∗ / κ ⌋ ) = κ ω φ ( z 0 ) + O ( κ ) , \mathbb{E}[(K - \omega)1\{K > k^*\}] = \omega\mathbb{P}(N = \lfloor k^*/\kappa\rfloor) = \sqrt{\kappa\omega}\,\varphi(z_0) + O(\kappa), E [( K − ω ) 1 { K > k ∗ }] = ω P ( N = ⌊ k ∗ / κ ⌋) = κω φ ( z 0 ) + O ( κ ) ,
uniformly in z 0 z_0 z 0 , by the local central limit theorem for the Poisson law: λ P ( N = j ) = φ ( z ) + O ( λ − 1 / 2 ) \sqrt{\lambda}\mathbb{P}(N = j) = \varphi(z) + O(\lambda^{-1/2}) λ P ( N = j ) = φ ( z ) + O ( λ − 1/2 ) uniformly for z : = ( j − λ ) / λ z := (j-\lambda)/\sqrt{\lambda} z := ( j − λ ) / λ in a compact set. (By Stirling’s formula log j ! = j log j − j + 1 2 log ( 2 π j ) + O ( 1 / j ) \log j! = j\log j - j + \frac{1}{2}\log(2\pi j) + O(1/j) log j ! = j log j − j + 2 1 log ( 2 π j ) + O ( 1/ j ) , so log P ( N = j ) = j − λ − j log ( j / λ ) − 1 2 log ( 2 π j ) + O ( 1 / λ ) \log\mathbb{P}(N = j) = j - \lambda- j\log(j/\lambda) - \frac{1}{2}\log(2\pi j) + O(1/\lambda) log P ( N = j ) = j − λ − j log ( j / λ ) − 2 1 log ( 2 π j ) + O ( 1/ λ ) ; with j / λ = 1 + z / λ j/\lambda= 1 + z/\sqrt{\lambda} j / λ = 1 + z / λ , j log ( j / λ ) = z λ + 1 2 z 2 + O ( λ − 1 / 2 ) j\log(j/\lambda) = z\sqrt{\lambda} + \frac{1}{2}z^2 + O(\lambda^{-1/2}) j log ( j / λ ) = z λ + 2 1 z 2 + O ( λ − 1/2 ) and j − λ = z λ j - \lambda= z\sqrt{\lambda} j − λ = z λ , which gives log P ( N = j ) = − 1 2 log ( 2 π λ ) − 1 2 z 2 + O ( λ − 1 / 2 ) \log\mathbb{P}(N = j) = -\frac{1}{2}\log(2\pi\lambda) - \frac{1}{2}z^2 + O(\lambda^{-1/2}) log P ( N = j ) = − 2 1 log ( 2 π λ ) − 2 1 z 2 + O ( λ − 1/2 ) .) Here ( j − λ ) / λ = z 0 + O ( λ − 1 / 2 ) (j-\lambda)/\sqrt{\lambda} = z_0 + O(\lambda^{-1/2}) ( j − λ ) / λ = z 0 + O ( λ − 1/2 ) and φ \varphi φ is Lipschitz.
Step 4 (remainders). E ( K − ω ) 2 = κ ω \mathbb{E}(K - \omega)^2 = \kappa\omega E ( K − ω ) 2 = κω , and the far region { ∣ K − ω ∣ > δ ′ } \{|K - \omega| > \delta'\} { ∣ K − ω ∣ > δ ′ } has exponentially small mass once κ < δ ′ 2 / ω \kappa< \delta'^2/\omega κ < δ ′2 / ω (the Poisson tail bound of Step 1). Combining Steps 1–4, and using E ( K − ω ) = 0 \mathbb{E}(K - \omega) = 0 E ( K − ω ) = 0 in the form E [ ( K − ω ) 1 { K ≤ k ∗ } ] = − E [ ( K − ω ) 1 { K > k ∗ } ] \mathbb{E}[(K - \omega)1\{K \le k^*\}] = -\mathbb{E}[(K - \omega)1\{K > k^*\}] E [( K − ω ) 1 { K ≤ k ∗ }] = − E [( K − ω ) 1 { K > k ∗ }] ,
FDR ( κ ) = E Φ d e t ( K , side ( K ) ) + O ( κ ) = q + h − E [ ( K − ω ) 1 { K ≤ k ∗ } ] + h + E [ ( K − ω ) 1 { K > k ∗ } ] + O ( κ ) = q + ( h + − h − ) E [ ( K − ω ) 1 { K > k ∗ } ] + O ( κ ) , \begin{aligned}
\operatorname{FDR}(\kappa) = \mathbb{E}\Phi_{\mathrm{det}}(K,\operatorname{side}(K)) + O(\kappa) = q + h_-\mathbb{E}[(K - \omega)1\{K \le k^*\}] + h_+\mathbb{E}[(K - \omega)1\{K > k^*\}] + O(\kappa) \\
&= q + (h_+ - h_-)\mathbb{E}[(K - \omega)1\{K > k^*\}] + O(\kappa),
\end{aligned} FDR ( κ ) = E Φ det ( K , side ( K )) + O ( κ ) = q + h − E [( K − ω ) 1 { K ≤ k ∗ }] + h + E [( K − ω ) 1 { K > k ∗ }] + O ( κ ) = q + ( h + − h − ) E [( K − ω ) 1 { K > k ∗ }] + O ( κ ) ,
and Step 3 gives (10). □
Remark 34 (Scope). For k 0 < m k_0 < m k 0 < m , side + + + means that the decoy atom is counted from t k 0 + 1 t_{k_0+1} t k 0 + 1 on, not that it is not counted; the upper switch at K ↑ K^{\uparrow} K ↑ (p D p_D p D crossing t k 0 + 1 t_{k_0+1} t k 0 + 1 ) is handled in Step 1 like k ∗ ∗ k^{**} k ∗∗ . Theorem 35 is the case k 0 = 2 k_0 = 2 k 0 = 2 , z 0 = 0 z_0 = 0 z 0 = 0 , with the derivatives A k ′ ( ω ) A_k'(\omega) A k ′ ( ω ) computed explicitly. For k 0 = 1 k_0 = 1 k 0 = 1 the theorem holds but is empty: the two sides differ only in the count at t 1 t_1 t 1 , which never changes R 1 R_1 R 1 (k = 1 k = 1 k = 1 is always feasible), so h + = h − h_+ = h_- h + = h − .
A switch at t 2 t_2 t 2
Theorem 35 (Switch at t 2 t_2 t 2 ). Consider the Poisson-limit selection model with one heavy cell (score g g g , mean weight ω \omega ω ) and one decoy block of light units (fraction ζ \zeta ζ , threshold quantile v 0 v_0 v 0 ), and let q < 1 q < 1 q < 1 . Assume: (i) t 1 ( 1 + ω ) < g < t 2 ( 1 + ω ) − ω t_1(1+\omega)<g<t_2(1+\omega)-\omega t 1 ( 1 + ω ) < g < t 2 ( 1 + ω ) − ω , so that the heavy window [ g / ( 1 + ω ) , ( g + ω ) / ( 1 + ω ) ] [g/(1+\omega),(g+\omega)/(1+\omega)] [ g / ( 1 + ω ) , ( g + ω ) / ( 1 + ω )] lies strictly inside ( t 1 , t 2 ) (t_1,t_2) ( t 1 , t 2 ) at K = ω K=\omega K = ω . Writing a : = ω m / q a:=\omega m/q a := ω m / q , this needs a < m / ( m − q ) a<m/(m-q) a < m / ( m − q ) . The second inequality is g < v 0 g<v_0 g < v 0 ;
(ii) v 0 : = t 2 ( 1 + ω ) − ω v_0:=t_2(1+\omega)-\omega v 0 := t 2 ( 1 + ω ) − ω (so v 0 > g v_0>g v 0 > g , and the decoy p p p -value equals t 2 t_2 t 2 at K = ω K=\omega K = ω );
(iii) 1 − ζ > v 0 1-\zeta>v_0 1 − ζ > v 0 , and f ω ( 1 − ζ ) > q f_\omega(1-\zeta)>q f ω ( 1 − ζ ) > q .
Then, as κ → 0 \kappa\to0 κ → 0 ,
F D R q − 1 = κ C m + O ( κ ) , C m = ω 2 π h + − h − q , h s = m ∑ k = 1 m 1 k A k ′ ( ω ) P s ( R 1 = k ) , \frac{\mathrm{FDR}}{q}-1=\sqrt{\kappa}\,C_m+O(\kappa),\qquad C_m=\sqrt{\frac{\omega}{2\pi}}\frac{h_+-h_-}{q},\qquad h_s=m\sum_{k=1}^{m}\frac{1}{k}A'_k(\omega)\mathbb{P}_s(R_1=k), q FDR − 1 = κ C m + O ( κ ) , C m = 2 π ω q h + − h − , h s = m k = 1 ∑ m k 1 A k ′ ( ω ) P s ( R 1 = k ) ,
with A 1 ′ ( ω ) = t 1 / Ξ A'_1(\omega)=t_1/\Xi A 1 ′ ( ω ) = t 1 /Ξ and A k ′ ( ω ) = ( t k − 1 ) / Ξ A'_k(\omega)=(t_k-1)/\Xi A k ′ ( ω ) = ( t k − 1 ) /Ξ for k ≥ 2 k\ge2 k ≥ 2 (Ξ = 1 + ω \Xi=1+\omega Ξ = 1 + ω ), and P s ( R 1 = k ) \mathbb{P}_s(R_1=k) P s ( R 1 = k ) the law of BH's leave-one-out count when each other unit's p p p -value has CDF t k t_k t k at grid point k k k , plus a decoy atom of mass ζ / Ξ \zeta/\Xi ζ /Ξ counted from k = 2 k=2 k = 2 (s = − s=- s = − ) or from k = 3 k=3 k = 3 (s = + s=+ s = + ).
Proof. We list only the differences from the proof of Theorem 9 .
Step 1 (saturation, as in Lemma 8 ). At K = ω K=\omega K = ω the null p p p -values are exactly uniform on [ 0 , 1 − ζ / Ξ ] [0,1-\zeta/\Xi] [ 0 , 1 − ζ /Ξ ] : light nulls below g g g map to [ 0 , g / ( 1 + ω ) ) [0,g/(1+\omega)) [ 0 , g / ( 1 + ω )) , the heavy test units fill [ g / ( 1 + ω ) , ( g + ω ) / ( 1 + ω ) ] [g/(1+\omega),(g+\omega)/(1+\omega)] [ g / ( 1 + ω ) , ( g + ω ) / ( 1 + ω )] with mass ω / Ξ \omega/\Xi ω /Ξ , and light nulls above g g g map on with density 1. Since 1 − ζ / Ξ = f ω ( 1 − ζ ) > q 1-\zeta/\Xi=f_\omega(1-\zeta)>q 1 − ζ /Ξ = f ω ( 1 − ζ ) > q , A k ( ω ) = t k A_k(\omega)=t_k A k ( ω ) = t k for all k k k , so Φ d e t ( ω , s ) = m ∑ k 1 k t k P s ( R 1 = k ) = q \Phi_{\mathrm{det}}(\omega,s)=m\sum_k\frac{1}{k}t_k\mathbb{P}_s(R_1=k)=q Φ det ( ω , s ) = m ∑ k k 1 t k P s ( R 1 = k ) = q for both sides, and the ∂ P s \partial\mathbb{P}_s ∂ P s terms cancel in h s h_s h s because ∑ k ∂ P s ( R 1 = k ) = 0 \sum_k\partial\mathbb{P}_s(R_1=k)=0 ∑ k ∂ P s ( R 1 = k ) = 0 .
Step 2 (derivatives). Under assumption (i) no grid point is in the heavy window or the light-map jump near K = ω K=\omega K = ω , so A k A_k A k is affine in K K K there. For k = 1 k=1 k = 1 only light nulls below g g g count (u ≤ t 1 ( 1 + K ) u\le t_1(1+K) u ≤ t 1 ( 1 + K ) ), which gives A 1 ′ = t 1 / Ξ A'_1=t_1/\Xi A 1 ′ = t 1 /Ξ . For k ≥ 2 k\ge2 k ≥ 2 the heavy cell is fully counted (constant mass ω / Ξ \omega/\Xi ω /Ξ ), and light nulls above g g g satisfy u ≤ t k ( 1 + K ) − K u\le t_k(1+K)-K u ≤ t k ( 1 + K ) − K , which gives A k ′ = ( t k − 1 ) / Ξ A'_k=(t_k-1)/\Xi A k ′ = ( t k − 1 ) /Ξ .
Step 3 (switches). p D ( K ) = ( v 0 + K ) / ( 1 + K ) p_D(K)=(v_0+K)/(1+K) p D ( K ) = ( v 0 + K ) / ( 1 + K ) is increasing. Side − - − (decoy counted from k = 2 k=2 k = 2 ) holds iff K ≤ k ∗ K\le k^* K ≤ k ∗ , with k ∗ = ω k^*=\omega k ∗ = ω ; side + + + (counted from k = 3 k=3 k = 3 ) holds for k ∗ < K < K 3 k^*<K<K_3 k ∗ < K < K 3 , where p D ( K 3 ) = t 3 p_D(K_3)=t_3 p D ( K 3 ) = t 3 , i.e. K 3 − ω = ( t 3 − t 2 ) ( 1 + ω ) / ( 1 − t 3 ) > 0 K_3-\omega=(t_3-t_2)(1+\omega)/(1-t_3)>0 K 3 − ω = ( t 3 − t 2 ) ( 1 + ω ) / ( 1 − t 3 ) > 0 . This upper switch is at a fixed distance from ω \omega ω , so it is handled exactly like k ∗ ∗ k^{**} k ∗∗ in the proof of Theorem 9 (a bounded difference times an exponentially small Poisson tail). Below k ∗ k^* k ∗ , p D p_D p D would cross t 1 t_1 t 1 only at K − ω = − ( t 2 − t 1 ) ( 1 + ω ) / ( 1 − t 1 ) < − ω K-\omega=-(t_2-t_1)(1+\omega)/(1-t_1)<-\omega K − ω = − ( t 2 − t 1 ) ( 1 + ω ) / ( 1 − t 1 ) < − ω , i.e. at K < 0 K<0 K < 0 , so this is never reached (p D ( 0 ) = v 0 > t 1 p_D(0)=v_0>t_1 p D ( 0 ) = v 0 > t 1 ).
Step 4. Steps 1–4 of the proof of Theorem 9 then apply verbatim with z 0 = 0 z_0=0 z 0 = 0 ; since φ ( 0 ) = 1 / 2 π \varphi(0)=1/\sqrt{2\pi} φ ( 0 ) = 1/ 2 π , (10) divided by q q q is the display.
Remark 36. C m C_m C m does not depend on g g g : neither A k ′ A'_k A k ′ nor P s \mathbb{P}_s P s involves it. The constant in the O ( κ ) O(\kappa) O ( κ ) remainder is of order 1 / δ 2 1/\delta^2 1/ δ 2 , where δ \delta δ is the distance in K K K from ω \omega ω to the nearest breakpoint (Step 2 of the proof of Theorem 9 ). So for each generic structure the expansion holds, and C m C_m C m is the coefficient.
m \sqrt{m} m growth with an explicit constant
Theorem 37 (General form of Theorem 11 ). Fix q ∈ ( 0 , 1 ) q\in(0,1) q ∈ ( 0 , 1 ) , b > 0 b>0 b > 0 and a ∈ ( 0 , 1 ] a\in(0,1] a ∈ ( 0 , 1 ] . For each m m m , take the structure of Theorem 35 with ω = a q / m \omega=aq/m ω = a q / m and ζ = b / m \zeta=b/m ζ = b / m (g g g generic, v 0 = t 2 ( 1 + ω ) − ω v_0=t_2(1+\omega)-\omega v 0 = t 2 ( 1 + ω ) − ω ), and let C m C_m C m be its coefficient: F D R / q − 1 = C m κ + O ( κ ) \mathrm{FDR}/q-1=C_m\sqrt{\kappa}+O(\kappa) FDR / q − 1 = C m κ + O ( κ ) . Then
C m = a m 2 π q Ξ − 1 Δ m ( q , b ) , Δ m : = E − [ 1 { R 1 ≥ 2 } R 1 ] − E + [ 1 { R 1 ≥ 2 } R 1 ] , C_m=\sqrt{\frac{am}{2\pi q}}\Xi^{-1}\Delta_m(q,b),\qquad\Delta_m:=\mathbb{E}_-\left[\frac{\mathbf{1}\{R_1\ge2\}}{R_1}\right]-\mathbb{E}_+\left[\frac{\mathbf{1}\{R_1\ge2\}}{R_1}\right], C m = 2 π q am Ξ − 1 Δ m ( q , b ) , Δ m := E − [ R 1 1 { R 1 ≥ 2 } ] − E + [ R 1 1 { R 1 ≥ 2 } ] , and Δ m → Δ ( q , b ) : = E [ 1 { R − ≥ 2 } / R − ] − E [ 1 { R + ≥ 2 } / R + ] \Delta_m\to\Delta(q,b):=\mathbb{E}[\mathbf{1}\{R^-\ge2\}/R^-]-\mathbb{E}[\mathbf{1}\{R^+\ge2\}/R^+] Δ m → Δ ( q , b ) := E [ 1 { R − ≥ 2 } / R − ] − E [ 1 { R + ≥ 2 } / R + ] as m → ∞ m\to\infty m → ∞ . Here
R s : = max { k ≥ 1 : 1 + Π ( k ) + D 1 { k ≥ k s } ≥ k } , R^s:=\max\{k\ge1:1+\Pi(k)+D\mathbf{1}\{k\ge k_s\}\ge k\}, R s := max { k ≥ 1 : 1 + Π ( k ) + D 1 { k ≥ k s } ≥ k } , with Π \Pi Π a Poisson process of rate q q q on the integers' scale (Π ( k ) ∼ Poisson ( q k ) \Pi(k)\sim\operatorname{Poisson}(qk) Π ( k ) ∼ Poisson ( q k ) ), D ∼ Poisson ( b ) D\sim\operatorname{Poisson}(b) D ∼ Poisson ( b ) independent of Π \Pi Π , k − = 2 k_-=2 k − = 2 and k + = 3 k_+=3 k + = 3 . Hence C m / m → a / ( 2 π q ) Δ ( q , b ) C_m/\sqrt{m}\to\sqrt{a/(2\pi q)}\Delta(q,b) C m / m → a / ( 2 π q ) Δ ( q , b ) .
Proof of Theorem 37 . Step 1 (exact identity). By Theorem 35, h s = m ∑ k k A k ′ P s ( R 1 = k ) h_s=m\sum_k kA'_k\mathbb{P}_s(R_1=k) h s = m ∑ k k A k ′ P s ( R 1 = k ) with A 1 ′ = t 1 / Ξ A'_1=t_1/\Xi A 1 ′ = t 1 /Ξ and A k ′ = ( t k − 1 ) / Ξ A'_k=(t_k-1)/\Xi A k ′ = ( t k − 1 ) /Ξ for k ≥ 2 k\ge2 k ≥ 2 . Writing ( t k − 1 ) / k = t k / k − 1 / k (t_k-1)/k=t_k/k-1/k ( t k − 1 ) / k = t k / k − 1/ k and t k / k = q / m t_k/k=q/m t k / k = q / m ,
h s m = − Ξ − 1 E s [ 1 { R 1 ≥ 2 } R 1 ] + q m Ξ . \frac{h_s}{m}=-\Xi^{-1}\mathbb{E}_s\left[\frac{\mathbf{1}\{R_1\ge2\}}{R_1}\right]+\frac{q}{m\Xi}. m h s = − Ξ − 1 E s [ R 1 1 { R 1 ≥ 2 } ] + m Ξ q .
The last term is the same for both sides, so ( h + − h − ) / m = Ξ − 1 Δ m (h_+-h_-)/m=\Xi^{-1}\Delta_m ( h + − h − ) / m = Ξ − 1 Δ m , and C m = ω / ( 2 π ) ( h + − h − ) / q C_m=\sqrt{\omega/(2\pi)}(h_+-h_-)/q C m = ω / ( 2 π ) ( h + − h − ) / q with ω = a q / m \omega=aq/m ω = a q / m gives the display.
Step 2 (law of R 1 R_1 R 1 ). The other m − 1 m-1 m − 1 p-values are i.i.d. with grid CDF t k + ( ζ / Ξ ) 1 { k ≥ k s } t_k+(\zeta/\Xi)\mathbf{1}\{k\ge k_s\} t k + ( ζ /Ξ ) 1 { k ≥ k s } , and R 1 = max { k : 1 + N m ( k ) ≥ k } R_1=\max\{k:1+N_m(k)\ge k\} R 1 = max { k : 1 + N m ( k ) ≥ k } , where N m ( k ) N_m(k) N m ( k ) counts those at or below t k t_k t k . Fix an integer cutoff k c u t k_{\mathrm{cut}} k cut . The increments of ( N m ( 1 ) , … , N m ( k c u t ) ) (N_m(1),\ldots,N_m(k_{\mathrm{cut}})) ( N m ( 1 ) , … , N m ( k cut )) are multinomial with cell probabilities q / m q/m q / m (and the decoy cell b / ( m Ξ ) b/(m\Xi) b / ( m Ξ ) ), so they converge to independent Poisson variables. (For fixed counts c 1 , … , c r c_1,\ldots,c_r c 1 , … , c r with total c c c and cell probabilities p i , m p_{i,m} p i , m with m p i , m → μ i mp_{i,m}\to\mu_i m p i , m → μ i , the multinomial probability
( m − 1 ) ! ( m − 1 − c ) ! ∏ i p i , m c i c i ! ( 1 − ∑ i p i , m ) m − 1 − c \frac{(m-1)!}{(m-1-c)!}\prod_i\frac{p_{i,m}^{c_i}}{c_i!}\left(1-\sum_i p_{i,m}\right)^{m-1-c} ( m − 1 − c )! ( m − 1 )! i ∏ c i ! p i , m c i ( 1 − i ∑ p i , m ) m − 1 − c
tends to ∏ i e − μ i μ i c i / c i ! \prod_i e^{-\mu_i}\mu_i^{c_i}/c_i! ∏ i e − μ i μ i c i / c i ! , since ( m − 1 ) ! / ( m − 1 − c ) ! ∼ m c (m-1)!/(m-1-c)!\sim m^c ( m − 1 )! / ( m − 1 − c )! ∼ m c and ( 1 − ∑ i p i , m ) m − 1 − c → e − ∑ i μ i (1-\sum_i p_{i,m})^{m-1-c}\to e^{-\sum_i\mu_i} ( 1 − ∑ i p i , m ) m − 1 − c → e − ∑ i μ i .)
( N m ( k ) ) k ≤ k c u t ⇒ ( Π ( k ) + D 1 { k ≥ k s } ) k ≤ k c u t . (N_m(k))_{k\le k_{\mathrm{cut}}}\Rightarrow(\Pi(k)+D\mathbf{1}\{k\ge k_s\})_{k\le k_{\mathrm{cut}}}. ( N m ( k ) ) k ≤ k cut ⇒ ( Π ( k ) + D 1 { k ≥ k s } ) k ≤ k cut .
Step 3 (uniform tail). N m ( k ) N_m(k) N m ( k ) is binomial with mean μ k ≤ q k + b \mu_k\le qk+b μ k ≤ q k + b . By the Chernoff bound P ( X ≥ x ) ≤ exp ( − μ h ( x / μ ) ) \mathbb{P}(X\ge x)\le\exp(-\mu h(x/\mu)) P ( X ≥ x ) ≤ exp ( − μ h ( x / μ )) for x ≥ μ x\ge\mu x ≥ μ , h ( u ) = u log u − u + 1 h(u)=u\log u-u+1 h ( u ) = u log u − u + 1 (Lemma 29), applied with x = k − 1 x=k-1 x = k − 1 ,
P ( 1 + N m ( k ) ≥ k ) ≤ exp ( − c q , b k ) for all k > k min ( q , b ) and all m , \mathbb{P}(1+N_m(k)\ge k)\le\exp(-c_{q,b}k)\qquad\text{for all }k>k_{\min}(q,b)\text{ and all }m, P ( 1 + N m ( k ) ≥ k ) ≤ exp ( − c q , b k ) for all k > k m i n ( q , b ) and all m ,
with c q , b > 0 c_{q,b}>0 c q , b > 0 because q < 1 q<1 q < 1 . Two conditions are used here: μ h ( x / μ ) \mu h(x/\mu) μ h ( x / μ ) decreases in μ \mu μ for μ < x \mu<x μ < x , so μ k \mu_k μ k may be replaced by q k + b qk+b q k + b ; and this needs k > ( 1 + b ) / ( 1 − q ) k>(1+b)/(1-q) k > ( 1 + b ) / ( 1 − q ) , strictly (at k = ( 1 + b ) / ( 1 − q ) k=(1+b)/(1-q) k = ( 1 + b ) / ( 1 − q ) the exponent is 0). The constant c q , b c_{q,b} c q , b depends on b b b as well as on q q q , and the exponent per k k k tends to log ( 1 / q ) − 1 + q > 0 \log(1/q)-1+q>0 log ( 1/ q ) − 1 + q > 0 . So
P ( R 1 ≥ k c u t ) ≤ ∑ k ≥ k c u t e − c q , b k \mathbb{P}(R_1\ge k_{\mathrm{cut}})\le\sum_{k\ge k_{\mathrm{cut}}}e^{-c_{q,b}k} P ( R 1 ≥ k cut ) ≤ k ≥ k cut ∑ e − c q , b k
uniformly in m m m , once k c u t > k min k_{\mathrm{cut}}>k_{\min} k cut > k m i n .
Step 4 (conclusion). Steps 2 and 3 give P s ( R 1 = k ) → P ( R s = k ) \mathbb{P}_s(R_1=k)\to\mathbb{P}(R^s=k) P s ( R 1 = k ) → P ( R s = k ) for every k k k (on { R 1 < k c u t } \{R_1<k_{\mathrm{cut}}\} { R 1 < k cut } the event { R 1 = k } \{R_1=k\} { R 1 = k } is a function of ( N m ( j ) ) j ≤ k c u t (N_m(j))_{j\le k_{\mathrm{cut}}} ( N m ( j ) ) j ≤ k cut ), and tightness, so R 1 ⇒ R s R_1\Rightarrow R^s R 1 ⇒ R s . Since 1 { R ≥ 2 } / R \mathbf{1}\{R\ge2\}/R 1 { R ≥ 2 } / R is bounded, Δ m → Δ \Delta_m\to\Delta Δ m → Δ . Finally Ξ = 1 + a q / m → 1 \Xi=1+aq/m\to1 Ξ = 1 + a q / m → 1 .
Lemma 39 (A ballot probability). Let Π \Pi Π be a Poisson process of rate q ∈ ( 0 , 1 ) q\in(0,1) q ∈ ( 0 , 1 ) . Then P ( Π ( j ) ≤ j − 1 for every integer j ≥ 1 ) = 1 − q \mathbb{P}(\Pi(j)\le j-1\text{ for every integer }j\ge1)=1-q P ( Π ( j ) ≤ j − 1 for every integer j ≥ 1 ) = 1 − q .
Proof. Let X i : = Π ( i ) − Π ( i − 1 ) X_i:=\Pi(i)-\Pi(i-1) X i := Π ( i ) − Π ( i − 1 ) , i.i.d. Pois ( q ) \operatorname{Pois}(q) Pois ( q ) , and Y 0 : = 0 Y_0:=0 Y 0 := 0 , Y j : = ∑ i ≤ j ( X i − 1 ) Y_j:=\sum_{i\le j}(X_i-1) Y j := ∑ i ≤ j ( X i − 1 ) . The event is E : = { Y j < 0 for all j ≥ 1 } E:=\{Y_j<0\text{ for all }j\ge1\} E := { Y j < 0 for all j ≥ 1 } ; write p : = P ( E ) p:=\mathbb{P}(E) p := P ( E ) . The steps X i − 1 X_i-1 X i − 1 are integers ≥ − 1 \ge-1 ≥ − 1 .
Weak records. Call j ≥ 0 j\ge0 j ≥ 0 a weak record if Y j ≥ Y i Y_j\ge Y_i Y j ≥ Y i for all i < j i<j i < j ; j = 0 j=0 j = 0 is one. Each weak record is followed by a next one, namely the first later time at which Y Y Y is at least its current value, with probability P ( ∃ j ≥ 1 : Y j ≥ 0 ) = 1 − p \mathbb{P}(\exists j\ge1:Y_j\ge0)=1-p P ( ∃ j ≥ 1 : Y j ≥ 0 ) = 1 − p , independently of the past (strong Markov property). So the number L L L of weak records has E L = ∑ ℓ ≥ 1 ( 1 − p ) ℓ − 1 = 1 / p \mathbb{E}L=\sum_{\ell\ge1}(1-p)^{\ell-1}=1/p E L = ∑ ℓ ≥ 1 ( 1 − p ) ℓ − 1 = 1/ p (with 1 / 0 : = ∞ 1/0:=\infty 1/0 := ∞ ).
Duality. Let σ : = inf { j ≥ 1 : Y j < 0 } \sigma:=\inf\{j\ge1:Y_j<0\} σ := inf { j ≥ 1 : Y j < 0 } . For n ≥ 1 n\ge1 n ≥ 1 , reversing the order of the first n n n steps preserves their joint law and turns Y j Y_j Y j into Y n − Y n − j Y_n-Y_{n-j} Y n − Y n − j , so it maps { Y 1 ≥ 0 , … , Y n ≥ 0 } \{Y_1\ge0,\ldots,Y_n\ge0\} { Y 1 ≥ 0 , … , Y n ≥ 0 } onto { n is a weak record } \{n\text{ is a weak record}\} { n is a weak record } . Hence P ( σ > n ) = P ( n is a weak record ) \mathbb{P}(\sigma>n)=\mathbb{P}(n\text{ is a weak record}) P ( σ > n ) = P ( n is a weak record ) for all n ≥ 0 n\ge0 n ≥ 0 , and E σ = ∑ n ≥ 0 P ( σ > n ) = E L = 1 / p \mathbb{E}\sigma=\sum_{n\ge0}\mathbb{P}(\sigma>n)=\mathbb{E}L=1/p E σ = ∑ n ≥ 0 P ( σ > n ) = E L = 1/ p .
Wald. The steps are at least − 1 -1 − 1 , so Y σ = − 1 Y_\sigma=-1 Y σ = − 1 on { σ < ∞ } \{\sigma<\infty\} { σ < ∞ } , and Y j ≥ 0 Y_j\ge0 Y j ≥ 0 for j < σ j<\sigma j < σ . Since Y j + ( 1 − q ) j Y_j+(1-q)j Y j + ( 1 − q ) j is a martingale, ( 1 − q ) E [ σ ∧ n ] = − E Y σ ∧ n ≤ 1 (1-q)\mathbb{E}[\sigma\wedge n]=-\mathbb{E}Y_{\sigma\wedge n}\le1 ( 1 − q ) E [ σ ∧ n ] = − E Y σ ∧ n ≤ 1 , so E σ ≤ 1 / ( 1 − q ) < ∞ \mathbb{E}\sigma\le1/(1-q)<\infty E σ ≤ 1/ ( 1 − q ) < ∞ by monotone convergence. Then Y σ = ∑ i ≥ 1 ( X i − 1 ) 1 { σ ≥ i } Y_\sigma=\sum_{i\ge1}(X_i-1)\mathbf{1}\{\sigma\ge i\} Y σ = ∑ i ≥ 1 ( X i − 1 ) 1 { σ ≥ i } with { σ ≥ i } \{\sigma\ge i\} { σ ≥ i } independent of X i X_i X i , and E ∑ i X i 1 { σ ≥ i } = E X 1 E σ < ∞ \mathbb{E}\sum_iX_i\mathbf{1}\{\sigma\ge i\}=\mathbb{E}X_1\mathbb{E}\sigma<\infty E ∑ i X i 1 { σ ≥ i } = E X 1 E σ < ∞ , so by Fubini (Wald’s identity) − 1 = E Y σ = ( q − 1 ) E σ -1=\mathbb{E}Y_\sigma=(q-1)\mathbb{E}\sigma − 1 = E Y σ = ( q − 1 ) E σ , that is, E σ = 1 / ( 1 − q ) \mathbb{E}\sigma=1/(1-q) E σ = 1/ ( 1 − q ) .
Hence p = 1 / E σ = 1 − q p = 1/\mathbb{E}\sigma= 1-q p = 1/ E σ = 1 − q .
Lemma 40 (Closed form and positivity). For every m m m , Δ m = 1 2 [ P − ( R 1 = 2 ) − P + ( R 1 = 2 ) ] > 0 \Delta_m = \frac{1}{2}\left[\mathbb{P}_{-}(R_1=2)-\mathbb{P}_{+}(R_1=2)\right]>0 Δ m = 2 1 [ P − ( R 1 = 2 ) − P + ( R 1 = 2 ) ] > 0 . Moreover
Δ ( q , b ) = b 2 ( 1 − q ) e − ( 2 q + b ) > 0 for all q ∈ ( 0 , 1 ) , b > 0. \Delta(q,b)=\frac{b}{2}(1-q)e^{-(2q+b)}>0 \qquad\text{for all }q\in(0,1),\ b>0. Δ ( q , b ) = 2 b ( 1 − q ) e − ( 2 q + b ) > 0 for all q ∈ ( 0 , 1 ) , b > 0.
Proof. Coupling. The two sides differ only at k = 2 k=2 k = 2 : for k ≥ 3 k\ge3 k ≥ 3 both count the decoys. Coupling the two sides on the same draws, R − = R + R_{-}=R_{+} R − = R + unless R + = 1 R_{+}=1 R + = 1 and R − = 2 R_{-}=2 R − = 2 . On that event 1 { R ≥ 2 } / R \mathbf{1}\{R\ge2\}/R 1 { R ≥ 2 } / R equals 1 2 \frac{1}{2} 2 1 on side − - − and 0 0 0 on side + + + , so exactly, for every m m m , Δ m = 1 2 [ P − ( R 1 = 2 ) − P + ( R 1 = 2 ) ] > 0 \Delta_m=\frac{1}{2}\left[\mathbb{P}_{-}(R_1=2)-\mathbb{P}_{+}(R_1=2)\right]>0 Δ m = 2 1 [ P − ( R 1 = 2 ) − P + ( R 1 = 2 ) ] > 0 . By Step 4 of the proof of Theorem 37 , letting m → ∞ m\to\infty m → ∞ gives Δ ( q , b ) = 1 2 [ P ( R − = 2 ) − P ( R + = 2 ) ] \Delta(q,b)=\frac{1}{2}\left[\mathbb{P}(R^{-}=2)-\mathbb{P}(R^{+}=2)\right] Δ ( q , b ) = 2 1 [ P ( R − = 2 ) − P ( R + = 2 ) ] .
Limit. Write c ( k ) : = Π ( k ) + D 1 { k ≥ k s } c(k):=\Pi(k)+D\mathbf{1}\{k\ge k_s\} c ( k ) := Π ( k ) + D 1 { k ≥ k s } , so that R s = max { k : 1 + c ( k ) ≥ k } R^{s}=\max\{k:1+c(k)\ge k\} R s = max { k : 1 + c ( k ) ≥ k } . If c ( 2 ) ≥ 2 c(2)\ge2 c ( 2 ) ≥ 2 then 1 + c ( 3 ) ≥ 3 1+c(3)\ge3 1 + c ( 3 ) ≥ 3 and R s ≥ 3 R^{s}\ge3 R s ≥ 3 , while k = 2 k=2 k = 2 is feasible only if c ( 2 ) ≥ 1 c(2)\ge1 c ( 2 ) ≥ 1 ; so R s = 2 R^{s}=2 R s = 2 requires c ( 2 ) = 1 c(2)=1 c ( 2 ) = 1 , and then R s = 2 R^{s}=2 R s = 2 holds iff 1 + c ( k ) ≤ k − 1 1+c(k)\le k-1 1 + c ( k ) ≤ k − 1 for every k ≥ 3 k\ge3 k ≥ 3 . On side − - − (k − = 2 k_{-}=2 k − = 2 ) this reads Π ′ ( j ) ≤ j − 1 \Pi'(j)\le j-1 Π ′ ( j ) ≤ j − 1 for every integer j ≥ 1 j\ge1 j ≥ 1 , where Π ′ ( j ) : = Π ( j + 2 ) − Π ( 2 ) \Pi'(j):=\Pi(j+2)-\Pi(2) Π ′ ( j ) := Π ( j + 2 ) − Π ( 2 ) is again a rate-q q q Poisson process, independent of ( Π ( 2 ) , D ) (\Pi(2),D) ( Π ( 2 ) , D ) ; by Lemma 39 this has probability 1 − q 1-q 1 − q .
Side + + + (k + = 3 k_{+}=3 k + = 3 ) additionally needs D = 0 D=0 D = 0 , else 1 + c ( 3 ) ≥ 3 1+c(3)\ge3 1 + c ( 3 ) ≥ 3 . Hence, with Pois ( 1 ; μ ) : = μ e − μ \operatorname{Pois}(1;\mu):=\mu e^{-\mu} Pois ( 1 ; μ ) := μ e − μ ,
P ( R − = 2 ) = ( 1 − q ) Pois ( 1 ; 2 q + b ) , P ( R + = 2 ) = ( 1 − q ) Pois ( 1 ; 2 q ) e − b , \mathbb{P}(R^{-}=2)=(1-q)\operatorname{Pois}(1;2q+b), \qquad\mathbb{P}(R^{+}=2)=(1-q)\operatorname{Pois}(1;2q)e^{-b}, P ( R − = 2 ) = ( 1 − q ) Pois ( 1 ; 2 q + b ) , P ( R + = 2 ) = ( 1 − q ) Pois ( 1 ; 2 q ) e − b ,
and
Δ ( q , b ) = 1 2 ( 1 − q ) [ ( 2 q + b ) e − ( 2 q + b ) − 2 q e − 2 q e − b ] = b 2 ( 1 − q ) e − ( 2 q + b ) > 0. \Delta(q,b)=\frac{1}{2}(1-q)\left[(2q+b)e^{-(2q+b)}-2q e^{-2q}e^{-b}\right]=\frac{b}{2}(1-q)e^{-(2q+b)}>0. Δ ( q , b ) = 2 1 ( 1 − q ) [ ( 2 q + b ) e − ( 2 q + b ) − 2 q e − 2 q e − b ] = 2 b ( 1 − q ) e − ( 2 q + b ) > 0.
As a numerical check of the closed form, an exact backward recursion of the limiting model that does not use the closed form matches it to 2 ⋅ 10 − 15 2\cdot10^{-15} 2 ⋅ 1 0 − 15 for q ∈ { 0.05 , 0.1 , 0.2 } q\in\{0.05,0.1,0.2\} q ∈ { 0.05 , 0.1 , 0.2 } and b ∈ { 0.5 , 1 , 2 } b\in\{0.5,1,2\} b ∈ { 0.5 , 1 , 2 } , and positivity was scanned for q q q from 0.01 0.01 0.01 to 0.9 0.9 0.9 .
Proof of Theorem 11 . The structure of Theorem 11 (ω = q / m \omega=q/m ω = q / m , decoy fraction 1 / m 1/m 1/ m ) is the case a = b = 1 a=b=1 a = b = 1 of Theorem 37 , whose conditions hold for m ≥ m 0 ( q , 1 ) m\ge m_0(q,1) m ≥ m 0 ( q , 1 ) (Remark 38). By Theorem 37 , C m / m / q = q C m / m → a / ( 2 π ) Δ ( q , b ) C_m/\sqrt{m/q}=\sqrt{q}\,C_m/\sqrt{m}\to\sqrt{a/(2\pi)}\Delta(q,b) C m / m / q = q C m / m → a / ( 2 π ) Δ ( q , b ) , in the iterated limit of Remark 38 (κ → 0 \kappa\to0 κ → 0 at fixed m m m , then m → ∞ m\to\infty m → ∞ ). By Lemma 40 , Δ ( q , b ) = b 2 ( 1 − q ) e − ( 2 q + b ) \Delta(q,b)=\frac{b}{2}(1-q)e^{-(2q+b)} Δ ( q , b ) = 2 b ( 1 − q ) e − ( 2 q + b ) , which is maximised over b b b at b = 1 b=1 b = 1 . With a = b = 1 a=b=1 a = b = 1 ,
lim m → ∞ C m m / q = ( 1 − q ) e − 1 − 2 q 2 2 π = c ( q ) , \lim_{m\to\infty}\frac{C_m}{\sqrt{m/q}}=\frac{(1-q)e^{-1-2q}}{2\sqrt{2\pi}}=c(q), m → ∞ lim m / q C m = 2 2 π ( 1 − q ) e − 1 − 2 q = c ( q ) ,
that is 0.0631 0.0631 0.0631 , 0.0541 0.0541 0.0541 and 0.0394 0.0394 0.0394 at q = 0.05 q=0.05 q = 0.05 , 0.1 0.1 0.1 and 0.2 0.2 0.2 (C m / m → 0.1710 C_m/\sqrt{m}\to0.1710 C m / m → 0.1710 at q = 0.1 q=0.1 q = 0.1 ).
Outlier detection
Proposition 41 (Outlier detection, asymptotic). Consider the Poisson-limit model of Theorem 9 in outlier detection:
inliers are i.i.d. Q Q Q , are all null, and have oracle p p p -values with conditional CDF F K F_K F K given K K K ;
each test unit is independently an outlier with probability π 1 \pi_1 π 1 (the labels are i.i.d. Bernoulli, not a fixed count; the leave-one-out identity needs this);
outliers are not in the calibration data, and their p p p -values f K ( v o ) f_K(v_o) f K ( v o ) , with v o > g v_o>g v o > g , form a switching block at t k o t_{k_o} t k o , with switch point k ∗ − ω = z 0 κ ω k^*-\omega=z_0\sqrt{\kappa\omega} k ∗ − ω = z 0 κω as in Theorem 9 .
Then, under the genericity assumptions of Theorem 9 , and uniformly over z 0 z_0 z 0 in a compact set,
FDR ( κ ) ≤ ( 1 − π 1 ) q + 2 ( 1 − π 1 ) π 1 m ( m − 1 ) M F κ ω φ ( z 0 ) + O ( κ ) , M F : = max k ∣ ∂ K F K ( t k ) ∣ K = ω ∣ . \operatorname{FDR}(\kappa)\le(1-\pi_1)q+2(1-\pi_1)\pi_1m(m-1)M_F\sqrt{\kappa\omega}\,\varphi(z_0)+O(\kappa), \qquad M_F:=\max_k\left|\partial_KF_K(t_k)\big|_{K=\omega}\right|. FDR ( κ ) ≤ ( 1 − π 1 ) q + 2 ( 1 − π 1 ) π 1 m ( m − 1 ) M F κω φ ( z 0 ) + O ( κ ) , M F := k max ∂ K F K ( t k ) K = ω .
So
FDR ( κ ) ≤ q − π 1 [ q − 2 ( 1 − π 1 ) m ( m − 1 ) M F κ ω 2 π ] + O ( κ ) , \operatorname{FDR}(\kappa) \le q-\pi_{1}\left[q-2(1-\pi_{1})m(m-1)M_{F}\frac{\sqrt{\kappa\omega}}{\sqrt{2\pi}}\right]+O(\kappa), FDR ( κ ) ≤ q − π 1 [ q − 2 ( 1 − π 1 ) m ( m − 1 ) M F 2 π κω ] + O ( κ ) ,
and for κ ω < 2 π ( q / ( 2 ( 1 − π 1 ) m ( m − 1 ) M F ) ) 2 \kappa\omega<2\pi\left(q/(2(1-\pi_{1})m(m-1)M_{F})\right)^{2} κω < 2 π ( q / ( 2 ( 1 − π 1 ) m ( m − 1 ) M F ) ) 2 the first-order (κ \sqrt{\kappa} κ ) term cannot produce an excess.
Proof. Zeroth order. At K = ω K=\omega K = ω the inlier law is exactly uniform. The leave-one-out identity (Lemma 66) with null mass 1 − π 1 1-\pi_{1} 1 − π 1 , i.e. A k ( ω ) = ( 1 − π 1 ) t k A_{k}(\omega)=(1-\pi_{1})t_{k} A k ( ω ) = ( 1 − π 1 ) t k , gives Φ d e t ( ω , s ) = ( 1 − π 1 ) q \Phi_{\mathrm{det}}(\omega,s)=(1-\pi_{1})q Φ det ( ω , s ) = ( 1 − π 1 ) q for either s s s .
First order. Here A k ( K ) = ( 1 − π 1 ) F K ( t k ) A_{k}(K)=(1-\pi_{1})F_{K}(t_{k}) A k ( K ) = ( 1 − π 1 ) F K ( t k ) , so h s = ( 1 − π 1 ) m ∑ k 1 k ∂ K F K ( t k ) P s ( R 1 = k ) h_{s}=(1-\pi_{1})m\sum_{k}\frac{1}{k}\partial_{K}F_{K}(t_{k})\mathbb{P}_{s}(R_{1}=k) h s = ( 1 − π 1 ) m ∑ k k 1 ∂ K F K ( t k ) P s ( R 1 = k ) . The ∂ P \partial\mathbb{P} ∂ P terms cancel because A k ( ω ) m / k = ( 1 − π 1 ) q A_{k}(\omega)m/k=(1-\pi_{1})q A k ( ω ) m / k = ( 1 − π 1 ) q is constant in k k k and ∑ k P s ( R 1 = k ) = 1 \sum_{k}\mathbb{P}_{s}(R_{1}=k)=1 ∑ k P s ( R 1 = k ) = 1 .
Total variation. By coupling (the two sides differ only on the event that some other unit is an outlier),
∑ k ∣ P + ( R 1 = k ) − P − ( R 1 = k ) ∣ ≤ 2 [ 1 − ( 1 − π 1 ) m − 1 ] ≤ 2 ( m − 1 ) π 1 . \sum_{k}\left|\mathbb{P}_{+}(R_{1}=k)-\mathbb{P}_{-}(R_{1}=k)\right|\le2\left[1-(1-\pi_{1})^{m-1}\right]\le2(m-1)\pi_{1}. k ∑ ∣ P + ( R 1 = k ) − P − ( R 1 = k ) ∣ ≤ 2 [ 1 − ( 1 − π 1 ) m − 1 ] ≤ 2 ( m − 1 ) π 1 .
Bounding 1 / k 1/k 1/ k by 1 then gives the crude bound ∣ h + − h − ∣ ≤ 2 ( 1 − π 1 ) π 1 m ( m − 1 ) M F |h_{+}-h_{-}|\le2(1-\pi_{1})\pi_{1}m(m-1)M_{F} ∣ h + − h − ∣ ≤ 2 ( 1 − π 1 ) π 1 m ( m − 1 ) M F .
Averaging. This is Step 3 of the proof of Theorem 9, within that proof’s framework: FDR ( κ ) = ( 1 − π 1 ) q + κ ω ( h + − h − ) φ ( z 0 ) + O ( κ ) \operatorname{FDR}(\kappa)=(1-\pi_{1})q+\sqrt{\kappa\omega}(h_{+}-h_{-})\varphi(z_{0})+O(\kappa) FDR ( κ ) = ( 1 − π 1 ) q + κω ( h + − h − ) φ ( z 0 ) + O ( κ ) , and the bound on ∣ h + − h − ∣ |h_{+}-h_{-}| ∣ h + − h − ∣ gives the first display. The second uses φ ( z 0 ) ≤ 1 / 2 π \varphi(z_{0})\le1/\sqrt{2\pi} φ ( z 0 ) ≤ 1/ 2 π . For k 0 < m k_{0}<m k 0 < m the upper switch at t k 0 + 1 t_{k_{0}+1} t k 0 + 1 is handled as in Step 1 of the proof of Theorem 9. □
Remark 42 (What Proposition 41 does and does not say). It is an asymptotic statement only. With the heavy structure of Example 10 (ω = 0.0863 \omega=0.0863 ω = 0.0863 ) and M F = 0.832 M_{F}=0.832 M F = 0.832 (the exact supremum over t t t of ∣ ∂ K F K ( t ) ∣ |\partial_{K}F_{K}(t)| ∣ ∂ K F K ( t ) ∣ , computed numerically), and bounding 1 − π 1 1-\pi_{1} 1 − π 1 by 1, the condition reads κ < 2.7 ⋅ 10 − 7 \kappa<2.7\cdot10^{-7} κ < 2.7 ⋅ 1 0 − 7 at q = 0.1 q=0.1 q = 0.1 , m = 32 m=32 m = 32 , and κ < 1.1 ⋅ 10 − 6 \kappa<1.1\cdot10^{-6} κ < 1.1 ⋅ 1 0 − 6 at q = 0.2 q=0.2 q = 0.2 . At practical B / n B/n B / n it says nothing: there a switching outlier block does produce an excess (numerically, FDR / q = 1.110 \operatorname{FDR}/q=1.110 FDR / q = 1.110 in the limit at q = 0.1 q=0.1 q = 0.1 , m = 32 m=32 m = 32 , κ ≈ 0.033 \kappa\approx0.033 κ ≈ 0.033 , against 1.045 for the largest global-null value our searches found at the same q q q and m m m with κ ≤ 0.1 \kappa\le0.1 κ ≤ 0.1 , and 1.090 with unbounded weights). The proposition inherits the genericity assumptions of Theorem 9, and the O ( κ ) O(\kappa) O ( κ ) term is not uniform in π 1 \pi_{1} π 1 or in the structure.
Proofs for the global-null lower bound and Simes’ Hessian
This appendix proves Lemma 4, Theorem 16 and Proposition 19. Throughout, m ≥ 2 m\ge2 m ≥ 2 and q ∈ ( 0 , 1 ) q\in(0,1) q ∈ ( 0 , 1 ) are fixed, t k : = q k / m t_{k}:=qk/m t k := q k / m for k = 1 , … , m k=1,\ldots,m k = 1 , … , m , t 0 : = 0 t_{0}:=0 t 0 := 0 , and t : = ( t 1 , … , t m ) ∈ R m \mathbf{t}:=(t_{1},\ldots,t_{m})\in\mathbb{R}^{m} t := ( t 1 , … , t m ) ∈ R m .
Configurations and the global-null reduction
Simes’ rejection probability. For a CDF F F F on [ 0 , 1 ] [0,1] [ 0 , 1 ] , let ψ m ( F ) \psi_{m}(F) ψ m ( F ) be the probability that BH (equivalently, Simes’ test) at level q q q makes at least one rejection among m m m i.i.d. draws with CDF F F F .
For any F F F , ψ m \psi_{m} ψ m depends on F F F only through ( F ( t 1 ) , … , F ( t m ) ) (F(t_{1}),\ldots,F(t_{m})) ( F ( t 1 ) , … , F ( t m )) : the band counts are multinomial whatever F F F is.
ψ m \psi_{m} ψ m is a polynomial in these values: a multinomial sum over band counts, with band k = ( t k − 1 , t k ] k=(t_{k-1},t_{k}] k = ( t k − 1 , t k ] for k ≤ m k\le m k ≤ m and band m + 1 = ( t m , 1 ] m+1=(t_{m},1] m + 1 = ( t m , 1 ] . We use the same polynomial for every vector x ∈ R m x\in\mathbb{R}^{m} x ∈ R m and write ψ m ( x ) \psi_{m}(x) ψ m ( x ) .
∇ 2 ψ \nabla^{2}\psi ∇ 2 ψ denotes its Hessian at t \mathbf{t} t .
Definition 43 (The family C n ( κ , g , λ ) \mathcal{C}^{n}(\kappa,g,\lambda) C n ( κ , g , λ ) ). Let H ≥ 1 H\ge1 H ≥ 1 , let κ h , λ h > 0 \kappa_{h},\lambda_{h}>0 κ h , λ h > 0 for h ≤ H h\le H h ≤ H , and let g 1 , … , g H ∈ ( 0 , 1 ) g_{1},\ldots,g_{H}\in(0,1) g 1 , … , g H ∈ ( 0 , 1 ) be distinct. The covariate X X X takes values in [ 0 , 1 ] ∪ { x 1 , … , x H } [0,1]\cup\{x_{1},\ldots,x_{H}\} [ 0 , 1 ] ∪ { x 1 , … , x H } .
(i) P P P puts mass 1 − ∑ h λ h / n 1-\sum_{h}\lambda_{h}/n 1 − ∑ h λ h / n on [ 0 , 1 ] [0,1] [ 0 , 1 ] , spread uniformly (“light” points, weight w = 1 w=1 w = 1 ), and mass λ h / n \lambda_{h}/n λ h / n on the atom x h x_{h} x h (“heavy” points, weight w = κ h n w=\kappa_{h}n w = κ h n ).
(ii) Y = y ( X ) Y=y(X) Y = y ( X ) is deterministic: y ( x ) = x y(x)=x y ( x ) = x on [ 0 , 1 ] [0,1] [ 0 , 1 ] and y ( x h ) = g h y(x_{h})=g_{h} y ( x h ) = g h . The score is V ( x , y ) = y V(x,y)=y V ( x , y ) = y .
(iii) d Q / d P ∝ w dQ/dP\propto w d Q / d P ∝ w . The weight ratio is B : = max w / min w = n max h κ h B:=\max w/\min w=n\max_{h}\kappa_{h} B := max w / min w = n max h κ h , once that is at least 1; the sup norm of d Q / d P dQ/dP d Q / d P is B ‾ : = B / E P w \overline{B}:=B/\mathbb{E}_{P}w B := B / E P w .
(iv) Selection setting: thresholds c n + j = y ( X n + j ) c_{n+j}=y(X_{n+j}) c n + j = y ( X n + j ) . This is a function of X X X , so (A2) holds, and every test unit is null with Y = c Y=c Y = c .
(v) Outlier-detection setting: every test unit is an inlier from Q Q Q .
P P P depends on n n n (the atoms have mass λ h / n \lambda_h/n λ h / n ). The bounds below are therefore worst-case bounds over the class of P P P with a given weight ratio, not statements about a fixed P P P . In both settings p j p_j p j equals the oracle p j ∗ p_j^* p j ∗ .
Proof of Lemma 4. Steps 1–3 use only that every test unit is null. Step 4 uses p j = p j ∗ p_j=p_j^* p j = p j ∗ , which holds in both settings of Definition 43; with non-oracle nulls, p j ≥ p j ∗ p_j\ge p_j^* p j ≥ p j ∗ for a monotone score, so EFD ( t ) ≤ t \operatorname{EFD}(t)\le t EFD ( t ) ≤ t instead.
Every test unit is null, so FDP = 1 { at least one rejection } \operatorname{FDP}=\mathbf{1}\{\text{at least one rejection}\} FDP = 1 { at least one rejection } .
Given the calibration data D D D , the test tuples and their U U U 's are i.i.d., so the test p-values are i.i.d. with CDF F D ( t ) : = P ( p 1 ≤ t ∣ D ) F_D(t):=\mathbb{P}(p_1\le t\mid D) F D ( t ) := P ( p 1 ≤ t ∣ D ) .
Hence FDR = FWER = E [ ψ m ( F D ) ] \operatorname{FDR}=\operatorname{FWER}=\mathbb{E}[\psi_m(F_D)] FDR = FWER = E [ ψ m ( F D )] .
By Lemma 67, conditional on the multiset of the n + 1 n+1 n + 1 points formed by the calibration data and test point 1, p 1 ∗ p_1^* p 1 ∗ is uniform on [ 0 , 1 ) [0,1) [ 0 , 1 ) (cf. Lemma 1). So E F D ( t ) = P ( p 1 ≤ t ) = t \mathbb{E}F_D(t)=\mathbb{P}(p_1\le t)=t E F D ( t ) = P ( p 1 ≤ t ) = t .
The Poisson limit
The Poisson-limit model. Let N h ∼ Poisson ( λ h ) N_h\sim\operatorname{Poisson}(\lambda_h) N h ∼ Poisson ( λ h ) be independent, K : = ∑ h κ h N h K:=\sum_h\kappa_hN_h K := ∑ h κ h N h , K < h : = ∑ h ′ : g h ′ < g h κ h ′ N h ′ K_{<h}:=\sum_{h':g_{h'}<g_h}\kappa_{h'}N_{h'} K < h := ∑ h ′ : g h ′ < g h κ h ′ N h ′ and Ξ : = 1 + ∑ h κ h λ h \Xi:=1+\sum_h\kappa_h\lambda_h Ξ := 1 + ∑ h κ h λ h . Define
F N ( t ) : = 1 Ξ Leb { u ∈ [ 0 , 1 ] : u + ∑ h : g h < u κ h N h 1 + K ≤ t } + ∑ h κ h λ h Ξ clip ( t − α h β h , 0 , 1 ) , F_N(t):=\frac{1}{\Xi}\operatorname{Leb}\left\{u\in[0,1]:\frac{u+\sum_{h:g_h<u}\kappa_hN_h}{1+K}\le t\right\}+\sum_h\frac{\kappa_h\lambda_h}{\Xi}\operatorname{clip}\left(\frac{t-\alpha_h}{\beta_h},0,1\right), F N ( t ) := Ξ 1 Leb { u ∈ [ 0 , 1 ] : 1 + K u + ∑ h : g h < u κ h N h ≤ t } + h ∑ Ξ κ h λ h clip ( β h t − α h , 0 , 1 ) ,
α h : = g h + K < h 1 + K + κ h , β h : = κ h ( 1 + N h ) 1 + K + κ h , \alpha_h:=\frac{g_h+K_{<h}}{1+K+\kappa_h},\qquad\beta_h:=\frac{\kappa_h(1+N_h)}{1+K+\kappa_h}, α h := 1 + K + κ h g h + K < h , β h := 1 + K + κ h κ h ( 1 + N h ) ,
where clip ( x , 0 , 1 ) : = min { max { x , 0 } , 1 } \operatorname{clip}(x,0,1):=\min\{\max\{x,0\},1\} clip ( x , 0 , 1 ) := min { max { x , 0 } , 1 } , and let Φ : = E ψ m ( F N ) \Phi:=\mathbb{E}\psi_m(F_N) Φ := E ψ m ( F N ) .
Lemma 44 (Poisson limit). Fix ( m , q , g , κ , λ ) (m,q,g,\kappa,\lambda) ( m , q , g , κ , λ ) and let FWER n \operatorname{FWER}_n FWER n be the FWER \operatorname{FWER} FWER of BH in the configuration C n ( κ , g , λ ) C_n(\kappa,g,\lambda) C n ( κ , g , λ ) (either setting). Then FWER n → Φ \operatorname{FWER}_n\to\Phi FWER n → Φ as n → ∞ n\to\infty n → ∞ , and E F N ( t ) = t \mathbb{E}F_N(t)=t E F N ( t ) = t for every t t t .
Proof. Build all n n n on one probability space:
N = ( N h ) h ≤ H N=(N_h)_{h\le H} N = ( N h ) h ≤ H , with independent Poisson( λ h ) (\lambda_h) ( λ h ) coordinates;
an i.i.d. Unif( 0 , 1 ) (0,1) ( 0 , 1 ) sequence, which supplies the light calibration scores;
the heavy calibration counts N ( n ) ∼ Multinomial ( n ; λ 1 / n , … , λ H / n ) N^{(n)}\sim\operatorname{Multinomial}(n;\lambda_1/n,\ldots,\lambda_H/n) N ( n ) ∼ Multinomial ( n ; λ 1 / n , … , λ H / n ) , coupled to N N N so that P ( N ( n ) ≠ N ) ≤ ( ∑ h λ h ) 2 / n \mathbb{P}(N^{(n)}\ne N)\le(\sum_h\lambda_h)^2/n P ( N ( n ) = N ) ≤ ( ∑ h λ h ) 2 / n . Such a coupling exists. Write Λ : = ∑ h λ h \Lambda:=\sum_h\lambda_h Λ := ∑ h λ h and N ( n ) = ∑ i ≤ n ϵ i N^{(n)}=\sum_{i\le n}\epsilon_i N ( n ) = ∑ i ≤ n ϵ i , where the ϵ i \epsilon_i ϵ i are i.i.d. and equal the h h h -th unit vector with probability λ h / n \lambda_h/n λ h / n and 0 0 0 otherwise. Couple each ϵ i \epsilon_i ϵ i maximally with a vector π i \pi_i π i of independent Poisson( λ h / n ) (\lambda_h/n) ( λ h / n ) coordinates, independently over i i i : the total variation distance between their laws is ( Λ / n ) ( 1 − e − Λ / n ) ≤ ( Λ / n ) 2 (\Lambda/n)(1-e^{-\Lambda/n})\le(\Lambda/n)^2 ( Λ/ n ) ( 1 − e − Λ/ n ) ≤ ( Λ/ n ) 2 . Then ∑ i π i \sum_i\pi_i ∑ i π i has the law of N N N and differs from N ( n ) N^{(n)} N ( n ) with probability at most n ( Λ / n ) 2 = Λ 2 / n n(\Lambda/n)^2=\Lambda^2/n n ( Λ/ n ) 2 = Λ 2 / n . (This is the coupling behind Le Cam's inequality ([14 ] ).) For each n n n , realise it by drawing N ( n ) N^{(n)} N ( n ) from its conditional law given N N N .
Then:
Let F ~ n \widetilde F_n F n be the conditional CDF of a test p-value in the configuration whose heavy calibration counts are N N N and whose n L : = n − ∑ h N h n_L:=n-\sum_hN_h n L := n − ∑ h N h light calibration scores are the first n L n_L n L terms of the sequence; on { N ( n ) = N } \{N^{(n)}=N\} { N ( n ) = N } , F ~ n = F D \widetilde F_n=F_D F n = F D . Steps 2–5 concern F ~ n \widetilde F_n F n . The empirical CDF G ^ n \widehat G_n G n of these light scores satisfies sup u ∣ G ^ n ( u ) − u ∣ → 0 \sup_u|\widehat G_n(u)-u|\to0 sup u ∣ G n ( u ) − u ∣ → 0 almost surely, by the Glivenko–Cantelli theorem (van der Vaart, [21 ] , Ch. 19).
A light test unit with score u u u has
p = n L G ^ n ( u − ) + n ∑ g h < u κ h N h + U ( 1 + ties ) n L + n K + 1 . p=\frac{n_L\widehat G_n(u-)+n\sum_{g_h<u}\kappa_hN_h+U(1+\text{ties})}{n_L+nK+1}. p = n L + n K + 1 n L G n ( u − ) + n ∑ g h < u κ h N h + U ( 1 + ties ) .
Light ties have probability 0 0 0 , and n L / n → 1 n_L/n\to1 n L / n → 1 . So p → ( u + ∑ g h < u κ h N h ) / ( 1 + K ) p\to(u+\sum_{g_h<u}\kappa_hN_h)/(1+K) p → ( u + ∑ g h < u κ h N h ) / ( 1 + K ) uniformly in u u u .
A heavy test unit of cell h h h has
p = n L G ^ n ( g h ) + n K < h + U ( κ h n + κ h n N h ) n L + n K + κ h n ⟶ α h + β h U . p=\frac{n_L\widehat G_n(g_h)+nK_{<h}+U(\kappa_hn+\kappa_hnN_h)}{n_L+nK+\kappa_hn}\longrightarrow\alpha_h+\beta_hU. p = n L + n K + κ h n n L G n ( g h ) + n K < h + U ( κ h n + κ h n N h ) ⟶ α h + β h U .
The test-type masses are ( 1 − ∑ h λ h / n ) / Ξ n (1-\sum_h\lambda_h/n)/\Xi_n ( 1 − ∑ h λ h / n ) / Ξ n (light) and κ h λ h / Ξ n \kappa_h\lambda_h/\Xi_n κ h λ h / Ξ n (cell h h h ), where Ξ n : = 1 − ∑ h λ h / n + ∑ h κ h λ h → Ξ \Xi_n:=1-\sum_h\lambda_h/n+\sum_h\kappa_h\lambda_h\to\Xi Ξ n := 1 − ∑ h λ h / n + ∑ h κ h λ h → Ξ .
The limiting light map u ↦ p ( u ) u\mapsto p(u) u ↦ p ( u ) is strictly increasing, so { u : p ( u ) = t } \{u:p(u)=t\} { u : p ( u ) = t } is Lebesgue-null; and β h > 0 \beta_h>0 β h > 0 . Hence F ~ n ( t j ) → F N ( t j ) \widetilde F_n(t_j)\to F_N(t_j) F n ( t j ) → F N ( t j ) almost surely. Since F D = F ~ n F_D=\widetilde F_n F D = F n outside an event of probability at most Λ 2 / n \Lambda^2/n Λ 2 / n , F D ( t j ) → F N ( t j ) F_D(t_j)\to F_N(t_j) F D ( t j ) → F N ( t j ) in probability.
ψ m \psi_m ψ m is bounded and continuous on [ 0 , 1 ] m [0,1]^m [ 0 , 1 ] m , so bounded convergence gives FWER n = E ψ m ( F D ) → Φ \operatorname{FWER}_n=\mathbb{E}\,\psi_m(F_D)\to\Phi FWER n = E ψ m ( F D ) → Φ (the equality is Lemma 4). Likewise E F D ( t ) = t \mathbb{E}F_D(t)=t E F D ( t ) = t (Lemma 4) passes to the limit. (Bounded convergence: convergence in probability of bounded variables implies convergence of their expectations; see van der Vaart, 1998, Ch. 2.)
Remark 45 (No rate). Lemma 44 gives no rate. The crude coupling bound needs ( ∑ h λ h ) 2 / n ≪ κ (\sum_h\lambda_h)^2/n\ll\kappa ( ∑ h λ h ) 2 / n ≪ κ ; in the scaling λ = ω / κ \lambda=\omega/\kappa λ = ω / κ of Section D.3 that is n ≫ κ − 3 n\gg\kappa^{-3} n ≫ κ − 3 . Finite-n n n Monte Carlo suggests much faster convergence than this bound: for m = 2 m=2 m = 2 and κ = 0.01 \kappa=0.01 κ = 0.01 , ( FWER n − q ) / κ = 0.0351 , 0.0361 , 0.0364 (\operatorname{FWER}_n-q)/\kappa=0.0351,\ 0.0361,\ 0.0364 ( FWER n − q ) / κ = 0.0351 , 0.0361 , 0.0364 at n = 2 ⋅ 10 3 , 2 ⋅ 10 4 , 2 ⋅ 10 5 n=2\cdot10^3,\ 2\cdot10^4,\ 2\cdot10^5 n = 2 ⋅ 1 0 3 , 2 ⋅ 1 0 4 , 2 ⋅ 1 0 5 , against the limit 0.03688 0.03688 0.03688 .
The multi-cell expansion
Window configurations.
Choose band indices 1 ≤ k 1 ≤ ⋯ ≤ k H ≤ m − 1 1\le k_1\le\cdots\le k_H\le m-1 1 ≤ k 1 ≤ ⋯ ≤ k H ≤ m − 1 .
Choose disjoint open intervals J h = ( a h , a h + ℓ h ) J_h=(a_h,a_h+\ell_h) J h = ( a h , a h + ℓ h ) , each with closure inside ( t k h , t k h + 1 ) (t_{k_h},t_{k_h+1}) ( t k h , t k h + 1 ) , and with a 1 < a 2 < ⋯ a_1<a_2<\cdots a 1 < a 2 < ⋯ .
Let ℓ + : = ∑ h ℓ h < 1 \ell_+:=\sum_h\ell_h<1 ℓ + := ∑ h ℓ h < 1 and Ω : = ℓ + / ( 1 − ℓ + ) \Omega:=\ell_+/(1-\ell_+) Ω := ℓ + / ( 1 − ℓ + ) , and set ω h : = ℓ h ( 1 + Ω ) , g h : = a h ( 1 + Ω ) − ∑ h ′ < h ω h ′ , κ h : = κ , λ h : = ω h / κ \omega_h:=\ell_h(1+\Omega),\qquad g_h:=a_h(1+\Omega)-\sum_{h'<h}\omega_{h'},\qquad\kappa_h:=\kappa,\qquad\lambda_h:=\omega_h/\kappa ω h := ℓ h ( 1 + Ω ) , g h := a h ( 1 + Ω ) − ∑ h ′ < h ω h ′ , κ h := κ , λ h := ω h / κ .
Then 0 < g 1 < ⋯ < g H < 1 0<g_1<\cdots<g_H<1 0 < g 1 < ⋯ < g H < 1 :
J h J_h J h is the p-scale window of cell h h h when K K K equals its mean Ω \Omega Ω . The test mass of cell h h h is κ λ h / Ξ = ω h / ( 1 + Ω ) = ℓ h \kappa\lambda_h/\Xi=\omega_h/(1+\Omega)=\ell_h κ λ h /Ξ = ω h / ( 1 + Ω ) = ℓ h , and the light test mass is 1 − ℓ + 1-\ell_+ 1 − ℓ + .
For k = 1 , … , m − 1 k=1,\ldots,m-1 k = 1 , … , m − 1 write d k ∈ R m d_k\in\mathbb{R}^m d k ∈ R m for the vector d k ( j ) : = t j − 1 { j ≥ k + 1 } d_k(j):=t_j-\mathbf{1}\{j\ge k+1\} d k ( j ) := t j − 1 { j ≥ k + 1 } , and
D k : = d k T ∇ 2 ψ d k . \mathcal{D}_k:=d_k^{\mathsf T}\nabla^2\psi\,d_k. D k := d k T ∇ 2 ψ d k .
Theorem 46 (Multi-cell expansion). As κ → 0 \kappa\to0 κ → 0 , with the window configuration ( k h , a h , ℓ h ) h ≤ H (k_h,a_h,\ell_h)_{h\le H} ( k h , a h , ℓ h ) h ≤ H fixed,
Φ ( κ ) − q = 1 2 κ ( 1 − ℓ + ) ∑ h ℓ h D k h + O ( κ 2 ) . \Phi(\kappa)-q=\frac{1}{2}\kappa(1-\ell_+)\sum_h\ell_h\mathcal{D}_{k_h}+O(\kappa^2). Φ ( κ ) − q = 2 1 κ ( 1 − ℓ + ) h ∑ ℓ h D k h + O ( κ 2 ) .
Proof. Let K h : = κ N h K_h := \kappa N_h K h := κ N h and Δ K h : = K h − ω h \Delta K_h := K_h-\omega_h Δ K h := K h − ω h . Then E Δ K h = 0 \mathbb{E}\Delta K_h=0 E Δ K h = 0 , Var K h = κ ω h \operatorname{Var}K_h=\kappa\omega_h Var K h = κ ω h , E Δ K h 3 = κ 2 ω h \mathbb{E}\Delta K_h^3=\kappa^2\omega_h E Δ K h 3 = κ 2 ω h and E Δ K h 4 = 3 κ 2 ω h 2 + κ 3 ω h \mathbb{E}\Delta K_h^4=3\kappa^2\omega_h^2+\kappa^3\omega_h E Δ K h 4 = 3 κ 2 ω h 2 + κ 3 ω h , and the Δ K h \Delta K_h Δ K h are independent. (All cumulants of Poisson ( λ ) \operatorname{Poisson}(\lambda) Poisson ( λ ) equal λ \lambda λ , so its third and fourth central moments are λ \lambda λ and λ + 3 λ 2 \lambda+3\lambda^2 λ + 3 λ 2 ; here Δ K h = κ ( N h − λ h ) \Delta K_h=\kappa(N_h-\lambda_h) Δ K h = κ ( N h − λ h ) with λ h = ω h / κ \lambda_h=\omega_h/\kappa λ h = ω h / κ .)
Step 1: the good event G G G . Choose δ > 0 \delta>0 δ > 0 and κ 0 > 0 \kappa_0>0 κ 0 > 0 so that, whenever ∣ Δ K h ∣ ≤ δ |\Delta K_h|\le\delta ∣Δ K h ∣ ≤ δ for all h h h and κ ≤ κ 0 \kappa\le\kappa_0 κ ≤ κ 0 , two kinds of interval lie strictly inside ( t k h , t k h + 1 ) (t_{k_h},t_{k_h+1}) ( t k h , t k h + 1 ) :
every heavy window [ α h , α h + β h ] [\alpha_h,\alpha_h+\beta_h] [ α h , α h + β h ] of F N F_N F N ;
every jump of the light map, [ ( g h + K < h ) / ( 1 + K ) , ( g h + K < h + K h ) / ( 1 + K ) ] [(g_h+K_{<h})/(1+K),(g_h+K_{<h}+K_h)/(1+K)] [( g h + K < h ) / ( 1 + K ) , ( g h + K < h + K h ) / ( 1 + K )] .
This is possible: all these endpoints are continuous in ( ( K h ) h , κ ) ((K_h)_h,\kappa) (( K h ) h , κ ) , and at ( ( K h ) h , κ ) = ( ( ω h ) h , 0 ) ((K_h)_h,\kappa)=((\omega_h)_h,0) (( K h ) h , κ ) = (( ω h ) h , 0 ) both intervals equal the closure of J h J_h J h . Call this event G G G . By Lemma 29 applied to each N h N_h N h (with c = δ / ω h c=\delta/\omega_h c = δ / ω h ), P ( G c ) ≤ 2 H e − γ / κ \mathbb{P}(G^c)\le2H e^{-\gamma/\kappa} P ( G c ) ≤ 2 H e − γ / κ for some γ > 0 \gamma>0 γ > 0 .
Step 2: on G G G , F N F_N F N is exactly affine in K K K at every grid point.
No t j t_j t j lies in a heavy window, so the heavy term of (11) at t j t_j t j is ℓ h 1 { k h < j } \ell_h\mathbf{1}\{k_h<j\} ℓ h 1 { k h < j } .
The light term is u j / Ξ u_j/\Xi u j /Ξ , where u j u_j u j solves u + ∑ g h < u K h = t j ( 1 + K ) u+\sum_{g_h<u}K_h=t_j(1+K) u + ∑ g h < u K h = t j ( 1 + K ) .
The cells with g h < u j g_h<u_j g h < u j are exactly those with k h < j k_h<j k h < j , so
F N ( t j ) = ( 1 − ℓ + ) [ t j ( 1 + ∑ h K h ) − ∑ k h < j K h ] + ∑ k h < j ℓ h . F_N(t_j)=(1-\ell_+)\left[t_j\left(1+\sum_h K_h\right)-\sum_{k_h<j}K_h\right]+\sum_{k_h<j}\ell_h. F N ( t j ) = ( 1 − ℓ + ) t j ( 1 + h ∑ K h ) − k h < j ∑ K h + k h < j ∑ ℓ h .
F N − t = ( 1 − ℓ + ) ∑ h Δ K h d k h . (12) F_N-t=(1-\ell_+)\sum_h\Delta K_h\,d_{k_h}. \tag*{(12)} F N − t = ( 1 − ℓ + ) h ∑ Δ K h d k h . (12)
Step 3: expansion. On G G G , ψ m ( F N ) \psi_m(F_N) ψ m ( F N ) is a polynomial of degree at most m m m in ( Δ K h ) h (\Delta K_h)_h ( Δ K h ) h , by (12) . Expand it around Δ K = 0 \Delta K=0 Δ K = 0 , using ψ m ( t ) = q \psi_m(t)=q ψ m ( t ) = q : Simes’ test has level exactly q q q for independent uniform p-values [19 ] .
First order: ( 1 − ℓ + ) ∑ h ∇ ψ ⋅ d k h E [ Δ K h 1 G ] (1-\ell_+)\sum_h\nabla\psi\cdot d_{k_h}\,\mathbb{E}[\Delta K_h\mathbf{1}_G] ( 1 − ℓ + ) ∑ h ∇ ψ ⋅ d k h E [ Δ K h 1 G ] . Since E Δ K h = 0 \mathbb{E}\Delta K_h=0 E Δ K h = 0 , ∣ E [ Δ K h 1 G ] ∣ = ∣ E [ Δ K h 1 G c ] ∣ ≤ ( κ ω h P ( G c ) ) 1 / 2 = O ( e − γ / ( 2 κ ) ) |\mathbb{E}[\Delta K_h\mathbf{1}_G]|=|\mathbb{E}[\Delta K_h\mathbf{1}_{G^c}]|\le(\kappa\omega_h\mathbb{P}(G^c))^{1/2}=O(e^{-\gamma/(2\kappa)}) ∣ E [ Δ K h 1 G ] ∣ = ∣ E [ Δ K h 1 G c ] ∣ ≤ ( κ ω h P ( G c ) ) 1/2 = O ( e − γ / ( 2 κ ) ) .
Second order: 1 2 ( 1 − ℓ + ) 2 ∑ h , h ′ E [ Δ K h Δ K h ′ 1 G ] d k h T ∇ 2 ψ d k h ′ \frac{1}{2}(1-\ell_+)^2\sum_{h,h'}\mathbb{E}[\Delta K_h\Delta K_{h'}\mathbf{1}_G]d_{k_h}^{\mathsf T}\nabla^2\psi\,d_{k_{h'}} 2 1 ( 1 − ℓ + ) 2 ∑ h , h ′ E [ Δ K h Δ K h ′ 1 G ] d k h T ∇ 2 ψ d k h ′ . By independence this is 1 2 ( 1 − ℓ + ) 2 ∑ h κ ω h D k h + O ( e − γ / ( 2 κ ) ) \frac{1}{2}(1-\ell_+)^2\sum_h\kappa\omega_hD_{k_h}+O(e^{-\gamma/(2\kappa)}) 2 1 ( 1 − ℓ + ) 2 ∑ h κ ω h D k h + O ( e − γ / ( 2 κ ) ) .
Third and higher order: every mixed moment of degree 3 vanishes except E Δ K h 3 = κ 2 ω h \mathbb{E}\Delta K_h^3=\kappa^2\omega_h E Δ K h 3 = κ 2 ω h . Moments of degree at least 4 are O ( κ 2 ) O(\kappa^2) O ( κ 2 ) (central moments of κ ⋅ Poisson ( ω / κ ) \kappa\cdot\operatorname{Poisson}(\omega/\kappa) κ ⋅ Poisson ( ω / κ ) ). So the remainder is O ( κ 2 ) O(\kappa^2) O ( κ 2 ) .
On G c G^c G c , ∣ ψ m − q ∣ ≤ 1 |\psi_m-q|\le1 ∣ ψ m − q ∣ ≤ 1 , so the contribution of G c G^c G c is O ( e − γ / κ ) O(e^{-\gamma/\kappa}) O ( e − γ / κ ) .
Finally ( 1 − ℓ + ) 2 ω h = ( 1 − ℓ + ) ℓ h (1-\ell_+)^2\omega_h=(1-\ell_+)\ell_h ( 1 − ℓ + ) 2 ω h = ( 1 − ℓ + ) ℓ h .
Remark 47 (How small κ \kappa κ must be (numerical)). For m = 2 m=2 m = 2 , ψ m \psi_m ψ m is quadratic, so the expansion is exact on G G G . In general κ 0 \kappa_0 κ 0 must make G G G typical: δ \delta δ must be several standard deviations κ ω h \sqrt{\kappa\omega_h} κ ω h , so κ 0 \kappa_0 κ 0 shrinks like ( margin ) 2 / ω (\text{margin})^2/\omega ( margin ) 2 / ω as the windows approach the band edges. Example (exact Poisson-limit values): q = 0.2 q=0.2 q = 0.2 , m = 8 m=8 m = 8 , bands { 1 , 2 } \{1,2\} { 1 , 2 } , centred windows of width 0.8 q / m 0.8q/m 0.8 q / m ; Theorem 46 predicts ( Φ − q ) / κ → 0.401400 (\Phi-q)/\kappa\to0.401400 ( Φ − q ) / κ → 0.401400 , and the exact values are 0.3488 0.3488 0.3488 , 0.3754 0.3754 0.3754 , 0.3942 0.3942 0.3942 at κ = 3 ⋅ 10 − 3 \kappa=3\cdot10^{-3} κ = 3 ⋅ 1 0 − 3 , 10 − 3 10^{-3} 1 0 − 3 , 3 ⋅ 10 − 4 3\cdot10^{-4} 3 ⋅ 1 0 − 4 and 0.400477 0.400477 0.400477 , 0.401411 0.401411 0.401411 , 0.401410 0.401410 0.401410 , 0.401403 0.401403 0.401403 at κ = 10 − 4 \kappa=10^{-4} κ = 1 0 − 4 , 3 ⋅ 10 − 5 3\cdot10^{-5} 3 ⋅ 1 0 − 5 , 10 − 5 10^{-5} 1 0 − 5 , …
The band curvatures
For s ∈ ( 0 , 1 ) s \in(0,1) s ∈ ( 0 , 1 ) not a grid point, let e s ∈ R m e_s \in\mathbb{R}^m e s ∈ R m , e s ( j ) : = 1 { s ≤ t j } e_s(j) := 1\{s \le t_j\} e s ( j ) := 1 { s ≤ t j } , and d s : = t − e s d_s := \mathbf{t} - e_s d s := t − e s . Then
d s = d k d_s = d_k d s = d k for s ∈ ( t k , t k + 1 ) s \in(t_k,t_{k+1}) s ∈ ( t k , t k + 1 ) , k = 1 , … , m − 1 k = 1,\ldots,m-1 k = 1 , … , m − 1 ;
d s = d 0 : = t − ( 1 , … , 1 ) d_s = d_0 := \mathbf{t} - (1,\ldots,1) d s = d 0 := t − ( 1 , … , 1 ) for s < t 1 s < t_1 s < t 1 ;
d s = t d_s = \mathbf{t} d s = t for s > t m s > t_m s > t m .
Write D ( s ) : = d s T ∇ 2 ψ d s \mathcal{D}(s) := d_s^\mathsf{T}\nabla^2\psi\,d_s D ( s ) := d s T ∇ 2 ψ d s and D 0 : = d 0 T ∇ 2 ψ d 0 \mathcal{D}_0 := d_0^\mathsf{T}\nabla^2\psi\,d_0 D 0 := d 0 T ∇ 2 ψ d 0 . Let a r ( s ) a_r(s) a r ( s ) be the probability that Simes’ test rejects when r r r of the m m m p p p -values equal s s s and the other m − r m-r m − r are i.i.d. Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) .
Lemma 48 (Curvature as a duplicate second difference). d s T ∇ 2 ψ d s = m ( m − 1 ) [ a 2 ( s ) − 2 a 1 ( s ) + q ] d_s^\mathsf{T}\nabla^2\psi\,d_s = m(m-1)\left[a_2(s)-2a_1(s)+q\right] d s T ∇ 2 ψ d s = m ( m − 1 ) [ a 2 ( s ) − 2 a 1 ( s ) + q ] .
Proof. For ϵ ∈ [ 0 , 1 ] \epsilon\in[0,1] ϵ ∈ [ 0 , 1 ] , t + ϵ ( e s − t ) \mathbf{t}+\epsilon(e_s-\mathbf{t}) t + ϵ ( e s − t ) is the grid vector of the CDF of the mixture ( 1 − ϵ ) Unif ( 0 , 1 ) + ϵ δ s (1-\epsilon)\operatorname{Unif}(0,1)+\epsilon\delta_s ( 1 − ϵ ) Unif ( 0 , 1 ) + ϵ δ s . A draw from this mixture equals s s s with probability ϵ \epsilon ϵ , independently across draws. Simes’ test depends only on the band counts, so
ψ m ( t + ϵ ( e s − t ) ) = ∑ r = 0 m ( m r ) ϵ r ( 1 − ϵ ) m − r a r ( s ) , \psi_m\left(\mathbf{t}+\epsilon(e_s-\mathbf{t})\right)=\sum_{r=0}^{m}\binom{m}{r}\epsilon^r(1-\epsilon)^{m-r}a_r(s), ψ m ( t + ϵ ( e s − t ) ) = r = 0 ∑ m ( r m ) ϵ r ( 1 − ϵ ) m − r a r ( s ) ,
an identity of polynomials in ϵ \epsilon ϵ . Take the second derivative at ϵ = 0 \epsilon=0 ϵ = 0 . The left side gives ( e s − t ) T ∇ 2 ψ ( e s − t ) = d s T ∇ 2 ψ d s (e_s-\mathbf{t})^\mathsf{T}\nabla^2\psi(e_s-\mathbf{t})=d_s^\mathsf{T}\nabla^2\psi\,d_s ( e s − t ) T ∇ 2 ψ ( e s − t ) = d s T ∇ 2 ψ d s . The right side gives m ( m − 1 ) ( a 2 − 2 a 1 + a 0 ) m(m-1)(a_2-2a_1+a_0) m ( m − 1 ) ( a 2 − 2 a 1 + a 0 ) , and a 0 = q a_0=q a 0 = q .
Lemma 49 (Duplicate exactness). For S ∼ Unif ( 0 , 1 ) S \sim\operatorname{Unif}(0,1) S ∼ Unif ( 0 , 1 ) and every r ∈ { 0 , … , m } r \in\{0,\ldots,m\} r ∈ { 0 , … , m } , E a r ( S ) = q \mathbb{E}a_r(S)=q E a r ( S ) = q . Hence
∫ 0 1 d s T ∇ 2 ψ d s d s = 0. \int_0^1 d_s^\mathsf{T}\nabla^2\psi\,d_s\,\mathrm{d}s=0. ∫ 0 1 d s T ∇ 2 ψ d s d s = 0.
Proof. For r = 0 r=0 r = 0 this is Simes exactness. Let r ≥ 1 r \ge1 r ≥ 1 , let r r r copies equal S S S , and let the other m − r m-r m − r values be independent uniforms p i p_i p i . All hypotheses are null, so FWER = E ∑ i 1 { i ∈ R } / ∣ R ∣ \operatorname{FWER}=\mathbb{E}\sum_i 1\{i\in\mathcal{R}\}/|\mathcal{R}| FWER = E ∑ i 1 { i ∈ R } /∣ R ∣ , where R \mathcal{R} R is the rejection set, the sum runs over all m m m values, and 0 / 0 : = 0 0/0:=0 0/0 := 0 .
A single p i p_i p i . By Lemma 66, 1 { i ∈ R } / ∣ R ∣ = 1 { p i ≤ q R i / m } / R i 1\{i\in\mathcal{R}\}/|\mathcal{R}|=1\{p_i\le qR_i/m\}/R_i 1 { i ∈ R } /∣ R ∣ = 1 { p i ≤ q R i / m } / R i , where R i R_i R i , the BH count with p i p_i p i replaced by 0, depends only on the other values. p i p_i p i is independent of them, so the expectation is E [ ( q R i / m ) / R i ] = q / m \mathbb{E}[(qR_i/m)/R_i]=q/m E [( q R i / m ) / R i ] = q / m .
The r r r copies of S S S . They are rejected together or not at all. Write N ( k ) N(k) N ( k ) for the number of the m m m values that are at most t k t_k t k , and N S ( k ) N_S(k) N S ( k ) for the same count with all r r r copies set to 0; so N ≤ N S N\le N_S N ≤ N S . Let R S R_S R S be the BH count with all r r r copies set to 0. Then R S ≥ r R_S\ge r R S ≥ r since r ≥ 1 r\ge1 r ≥ 1 , and R S R_S R S is independent of S S S .
If S ≤ q R S / m S\le qR_S/m S ≤ q R S / m , then N ( R S ) = N S ( R S ) N(R_S)=N_S(R_S) N ( R S ) = N S ( R S ) , so the BH count is R S R_S R S .
Otherwise the copies do not count at any k ≤ R S k\le R_S k ≤ R S , and N ≤ N S N\le N_S N ≤ N S gives a BH count at most R S R_S R S , with threshold < S <S < S .
So the copies are rejected iff S ≤ q R S / m S\le qR_S/m S ≤ q R S / m , and then ∣ R ∣ = R S |\mathcal{R}|=R_S ∣ R ∣ = R S . Their total is E [ r 1 { S ≤ q R S / m } / R S ] = r q / m \mathbb{E}[r1\{S\le qR_S/m\}/R_S]=rq/m E [ r 1 { S ≤ q R S / m } / R S ] = r q / m .
Sum: ( m − r ) q / m + r q / m = q (m-r)q/m+rq/m=q ( m − r ) q / m + r q / m = q .
Averaging Lemma 48 over s s s then gives ∫ 0 1 d s T ∇ 2 ψ d s d s = m ( m − 1 ) ( q − 2 q + q ) = 0 \int_0^1 d_s^\mathsf{T}\nabla^2\psi\,d_s\,\mathrm{d}s=m(m-1)(q-2q+q)=0 ∫ 0 1 d s T ∇ 2 ψ d s d s = m ( m − 1 ) ( q − 2 q + q ) = 0 .
Lemma 50 (The curvature budget). D 0 = − m ( m − 1 ) ( 1 − q ) \mathcal{D}_0=-m(m-1)(1-q) D 0 = − m ( m − 1 ) ( 1 − q ) , t T ∇ 2 ψ t = 0 \mathbf{t}^\mathsf{T}\nabla^2\psi\,\mathbf{t}=0 t T ∇ 2 ψ t = 0 , and
∑ k = 1 m − 1 D k = m ( m − 1 ) ( 1 − q ) . \sum_{k=1}^{m-1}\mathcal{D}_k=m(m-1)(1-q). k = 1 ∑ m − 1 D k = m ( m − 1 ) ( 1 − q ) .
Proof. 1. For s < t 1 s<t_1 s < t 1 , one copy at s s s already gives a rejection, so a 1 ( s ) = a 2 ( s ) = 1 a_1(s)=a_2(s)=1 a 1 ( s ) = a 2 ( s ) = 1 , and Lemma 48 gives D 0 = m ( m − 1 ) ( q − 1 ) \mathcal{D}_0=m(m-1)(q-1) D 0 = m ( m − 1 ) ( q − 1 ) .
For s > t m s > t_m s > t m , the copies never count. So a r ( s ) a_r(s) a r ( s ) is Simes’ rejection probability for m − r m-r m − r uniforms at thresholds q k / m qk/m q k / m , which is a Simes test at level q ( m − r ) / m q(m-r)/m q ( m − r ) / m and exact: a r ( s ) = q ( m − r ) / m a_r(s) = q(m-r)/m a r ( s ) = q ( m − r ) / m . Then Lemma 48 gives
t T ∇ 2 ψ t = m ( m − 1 ) q m [ ( m − 2 ) − 2 ( m − 1 ) + m ] = 0. \mathbf{t}^{\mathsf{T}}\nabla^2\psi\,\mathbf{t} = m(m-1)\frac{q}{m}\left[(m-2)-2(m-1)+m\right] = 0. t T ∇ 2 ψ t = m ( m − 1 ) m q [ ( m − 2 ) − 2 ( m − 1 ) + m ] = 0.
d s d_s d s is constant on each band, and bands 1 , … , m 1,\ldots,m 1 , … , m have length q / m q/m q / m while band m + 1 m+1 m + 1 has length 1 − q 1-q 1 − q . So Lemma 49 reads
q m ∑ k = 0 m − 1 D k + ( 1 − q ) ⋅ 0 = 0 , \frac{q}{m}\sum_{k=0}^{m-1}\mathcal{D}_k+(1-q)\cdot0=0, m q k = 0 ∑ m − 1 D k + ( 1 − q ) ⋅ 0 = 0 ,
that is, ∑ k = 1 m − 1 D k = − D 0 = m ( m − 1 ) ( 1 − q ) \sum_{k=1}^{m-1}\mathcal{D}_k=-\mathcal{D}_0=m(m-1)(1-q) ∑ k = 1 m − 1 D k = − D 0 = m ( m − 1 ) ( 1 − q ) .
Lemma 51 (The first interior band). D 1 = m ( m − 1 ) ( 1 − q ) [ 2 ( 1 − q / m ) m − 2 − 1 ] \mathcal{D}_1=m(m-1)(1-q)\left[2(1-q/m)^{m-2}-1\right] D 1 = m ( m − 1 ) ( 1 − q ) [ 2 ( 1 − q / m ) m − 2 − 1 ] .
Proof. Take s ∈ ( t 1 , t 2 ) s\in(t_1,t_2) s ∈ ( t 1 , t 2 ) in Lemma 48 .
Two copies give two values at most t 2 t_2 t 2 , so a 2 ( s ) = 1 a_2(s)=1 a 2 ( s ) = 1 .
With one copy at s s s and m − 1 m-1 m − 1 uniforms with order statistics U ( i ) U_{(i)} U ( i ) , there is no rejection iff # { U ≤ t i + 1 } ≤ i − 1 \#\{U\le t_{i+1}\}\le i-1 # { U ≤ t i + 1 } ≤ i − 1 for i = 1 , … , m − 1 i=1,\ldots,m-1 i = 1 , … , m − 1 , that is, iff U ( i ) > q / m + i q / m U_{(i)}>q/m+iq/m U ( i ) > q / m + i q / m for all i i i .
For M M M uniforms and b 0 , b 1 ≥ 0 b_0,b_1\ge0 b 0 , b 1 ≥ 0 with b 0 + M b 1 ≤ 1 b_0+Mb_1\le1 b 0 + M b 1 ≤ 1 ,
P ( U ( i ) > b 0 + i b 1 ∀ i ≤ M ) = ( 1 − b 0 ) M ( 1 − M b 1 1 − b 0 ) = ( 1 − b 0 − M b 1 ) ( 1 − b 0 ) M − 1 . \mathbb{P}(U_{(i)}>b_0+ib_1\ \forall i\le M)=(1-b_0)^M\left(1-\frac{Mb_1}{1-b_0}\right)=(1-b_0-Mb_1)(1-b_0)^{M-1}. P ( U ( i ) > b 0 + i b 1 ∀ i ≤ M ) = ( 1 − b 0 ) M ( 1 − 1 − b 0 M b 1 ) = ( 1 − b 0 − M b 1 ) ( 1 − b 0 ) M − 1 .
To see this: all points must exceed b 0 b_0 b 0 ; rescale to ( b 0 , 1 ] (b_0,1] ( b 0 , 1 ] ; then apply Daniels’ identity P ( U ( i ) > i b ′ ∀ i ≤ M ) = 1 − M b ′ \mathbb{P}(U_{(i)}>ib'\ \forall i\le M)=1-Mb' P ( U ( i ) > i b ′ ∀ i ≤ M ) = 1 − M b ′ . That identity is Simes exactness, which is Lemma 49 with r = 0 r=0 r = 0 .
With M = m − 1 M=m-1 M = m − 1 and b 0 = b 1 = q / m b_0=b_1=q/m b 0 = b 1 = q / m : 1 − a 1 ( s ) = ( 1 − q ) ( 1 − q / m ) m − 2 1-a_1(s)=(1-q)(1-q/m)^{m-2} 1 − a 1 ( s ) = ( 1 − q ) ( 1 − q / m ) m − 2 .
So D 1 = m ( m − 1 ) [ 1 − 2 + 2 ( 1 − q ) ( 1 − q / m ) m − 2 + q ] \mathcal{D}_1=m(m-1)\left[1-2+2(1-q)(1-q/m)^{m-2}+q\right] D 1 = m ( m − 1 ) [ 1 − 2 + 2 ( 1 − q ) ( 1 − q / m ) m − 2 + q ] .
Lemmas 50 and 51 have also been checked in exact arithmetic, with q q q symbolic, for m = 2 , … , 9 m=2,\ldots,9 m = 2 , … , 9 , by building ψ m \psi_m ψ m as an explicit polynomial; for example, at m = 8 m=8 m = 8 , q = 1 / 5 q=1/5 q = 1/5 , D 1 = 10295206327 / 320000000 ≈ 32.17 \mathcal{D}_1=10295206327/320000000\approx32.17 D 1 = 10295206327/320000000 ≈ 32.17 .
Lemma 52 (Uniform exponential tail). For 2 ≤ k ≤ m − 1 2\le k\le m-1 2 ≤ k ≤ m − 1 , ∣ D k ∣ ≤ 2 m ( m − 1 ) T k |\mathcal{D}_k|\le2m(m-1)T_k ∣ D k ∣ ≤ 2 m ( m − 1 ) T k , where T k : = ∑ i = k + 1 m P ( Bin ( m , q i / m ) ≥ i − 2 ) T_k:=\sum_{i=k+1}^{m}\mathbb{P}(\operatorname{Bin}(m,qi/m)\ge i-2) T k := ∑ i = k + 1 m P ( Bin ( m , q i / m ) ≥ i − 2 ) . Let i q : = ⌈ 2 / ( 1 − q ) ⌉ i_q:=\lceil2/(1-q)\rceil i q := ⌈ 2/ ( 1 − q )⌉ . Then
T k ≤ # { i : k + 1 ≤ i < i q } + ∑ i ≥ max ( k + 1 , i q ) exp ( − [ ( i − 2 ) log i − 2 q i − ( i − 2 ) + q i ] ) . T_k\le\#\{i:k+1\le i<i_q\}+\sum_{i\ge\max(k+1,i_q)}\exp\left(-\left[(i-2)\log\frac{i-2}{qi}-(i-2)+qi\right]\right). T k ≤ # { i : k + 1 ≤ i < i q } + i ≥ m a x ( k + 1 , i q ) ∑ exp ( − [ ( i − 2 ) log q i i − 2 − ( i − 2 ) + q i ] ) .
The right side does not depend on m m m . For k ≥ i q k\ge i_q k ≥ i q it decays like e − k ( log ( 1 / q ) − 1 + q ) + O ( 1 ) e^{-k(\log(1/q)-1+q)+O(1)} e − k ( l o g ( 1/ q ) − 1 + q ) + O ( 1 ) .
Proof. Take s ∈ ( t k , t k + 1 ) s\in(t_k,t_{k+1}) s ∈ ( t k , t k + 1 ) and r ∈ { 1 , 2 } r\in\{1,2\} r ∈ { 1 , 2 } , and let N ′ ( t ) N'(t) N ′ ( t ) count the m − r m-r m − r uniforms that are at most t t t .
No rejection means N ′ ( t i ) ≤ i − 1 N'(t_i)\le i-1 N ′ ( t i ) ≤ i − 1 for i ≤ k i\le k i ≤ k and N ′ ( t i ) ≤ i − 1 − r N'(t_i)\le i-1-r N ′ ( t i ) ≤ i − 1 − r for i ≥ k + 1 i\ge k+1 i ≥ k + 1 .
This event is contained in E 0 : = { N ′ ( t i ) ≤ i − 1 ∀ i } E_0:=\{N'(t_i)\le i-1\ \forall i\} E 0 := { N ′ ( t i ) ≤ i − 1 ∀ i } , which has probability 1 − q ( m − r ) / m 1-q(m-r)/m 1 − q ( m − r ) / m by Simes exactness.
E 0 E_0 E 0 minus the event is contained in { ∃ i ≥ k + 1 : N ′ ( t i ) ≥ i − r } \{\exists i\ge k+1:N'(t_i)\ge i-r\} { ∃ i ≥ k + 1 : N ′ ( t i ) ≥ i − r } , which has probability at most T k T_k T k , since N ′ ( t i ) N'(t_i) N ′ ( t i ) is stochastically at most Bin ( m , q i / m ) \operatorname{Bin}(m,qi/m) Bin ( m , q i / m ) .
So a r ( s ) = q ( m − r ) / m + θ r a_r(s)=q(m-r)/m+\theta_r a r ( s ) = q ( m − r ) / m + θ r with θ r ∈ [ 0 , T k ] \theta_r\in[0,T_k] θ r ∈ [ 0 , T k ] , and a 2 − 2 a 1 + q = θ 2 − 2 θ 1 ∈ [ − 2 T k , T k ] a_2-2a_1+q=\theta_2-2\theta_1\in[-2T_k,T_k] a 2 − 2 a 1 + q = θ 2 − 2 θ 1 ∈ [ − 2 T k , T k ] ; with Lemma 48 this gives ∣ D k ∣ ≤ 2 m ( m − 1 ) T k |\mathcal{D}_k|\le2m(m-1)T_k ∣ D k ∣ ≤ 2 m ( m − 1 ) T k .
Each term of T k T_k T k with i < i q i<i_q i < i q is bounded by 1. For i ≥ i q i\ge i_q i ≥ i q we have i − 2 ≥ q i i-2\ge qi i − 2 ≥ q i , and P ( Bin ( m , p ) ≥ x ) ≤ exp ( − [ x log ( x / ( m p ) ) − x + m p ] ) \mathbb{P}(\operatorname{Bin}(m,p)\ge x)\le\exp(-[x\log(x/(mp))-x+mp]) P ( Bin ( m , p ) ≥ x ) ≤ exp ( − [ x log ( x / ( m p )) − x + m p ]) for x ≥ m p x\ge mp x ≥ m p (Lemma 29); here x = i − 2 x=i-2 x = i − 2 and m p = q i mp=qi m p = q i .
The lower-bound theorems
Let
L m ∗ ( q ) : = sup 1 2 ( 1 − ℓ + ) ∑ h ℓ h D k h , L_m^*(q):=\sup\ \frac{1}{2}(1-\ell_+)\sum_h \ell_h\mathcal{D}_{k_h}, L m ∗ ( q ) := sup 2 1 ( 1 − ℓ + ) h ∑ ℓ h D k h ,
the supremum over all window configurations of Section D.3.
Theorem 53 (Bounds on L m ∗ L_m^* L m ∗ ). (1) (Single band.) L m ∗ ( q ) ≥ 1 2 q ( 1 − q / m ) ( m − 1 ) ( 1 − q ) [ 2 ( 1 − q / m ) m − 2 − 1 ] L_m^*(q) \ge\frac{1}{2}q(1-q/m)(m-1)(1-q)[2(1-q/m)^{m-2}-1] L m ∗ ( q ) ≥ 2 1 q ( 1 − q / m ) ( m − 1 ) ( 1 − q ) [ 2 ( 1 − q / m ) m − 2 − 1 ] .
This is vacuous when D 1 < 0 \mathcal{D}_1<0 D 1 < 0 , that is (for m ≥ 3 m\ge3 m ≥ 3 ) when q > m ( 1 − 2 − 1 / ( m − 2 ) ) q>m(1-2^{-1/(m-2)}) q > m ( 1 − 2 − 1/ ( m − 2 ) ) , which tends to ln 2 \ln2 ln 2 .
(2) (All positive bands.) L m ∗ ( q ) ≥ 1 2 q ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) L_m^*(q) \ge\frac{1}{2}q(1-q)(m-1)(1-q+q/m) L m ∗ ( q ) ≥ 2 1 q ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) .
(3) (Asymptotic.) lim inf m → ∞ L m ∗ ( q ) / ( q m ) ≥ 1 2 ( 1 − q ) \liminf_{m\to\infty} L_m^*(q)/(qm) \ge\frac{1}{2}(1-q) lim inf m → ∞ L m ∗ ( q ) / ( q m ) ≥ 2 1 ( 1 − q ) .
Proof. (1) Take one window of width θ q / m \theta q/m θ q / m in the band ( t 1 , t 2 ) (t_1,t_2) ( t 1 , t 2 ) , with θ ↑ 1 \theta\uparrow1 θ ↑ 1 . Its value 1 2 ( 1 − θ q / m ) ( θ q / m ) D 1 \frac{1}{2}(1-\theta q/m)(\theta q/m)\mathcal{D}_1 2 1 ( 1 − θ q / m ) ( θ q / m ) D 1 tends to 1 2 ( 1 − q / m ) ( q / m ) D 1 \frac{1}{2}(1-q/m)(q/m)\mathcal{D}_1 2 1 ( 1 − q / m ) ( q / m ) D 1 ; now use Lemma 51.
(2) Let K : = { k ≥ 1 : D k > 0 } \mathcal{K}:=\{k\ge1:\mathcal{D}_k>0\} K := { k ≥ 1 : D k > 0 } . Fill each band in K \mathcal{K} K with a window of width θ q / m \theta q/m θ q / m , θ ↑ 1 \theta\uparrow1 θ ↑ 1 . Then ℓ + ≤ ( m − 1 ) q / m \ell_+\le(m-1)q/m ℓ + ≤ ( m − 1 ) q / m , and by Lemma 50, ∑ k ∈ K D k ≥ ∑ k = 1 m − 1 D k = m ( m − 1 ) ( 1 − q ) \sum_{k\in\mathcal{K}}\mathcal{D}_k\ge\sum_{k=1}^{m-1}\mathcal{D}_k=m(m-1)(1-q) ∑ k ∈ K D k ≥ ∑ k = 1 m − 1 D k = m ( m − 1 ) ( 1 − q ) . So, in the limit θ ↑ 1 \theta\uparrow1 θ ↑ 1 , 1 2 ( 1 − ℓ + ) ( q / m ) ∑ k ∈ K D k ≥ 1 2 ( 1 − q + q / m ) q ( m − 1 ) ( 1 − q ) \frac{1}{2}(1-\ell_+)(q/m)\sum_{k\in\mathcal{K}}\mathcal{D}_k\ge\frac{1}{2}(1-q+q/m)q(m-1)(1-q) 2 1 ( 1 − ℓ + ) ( q / m ) ∑ k ∈ K D k ≥ 2 1 ( 1 − q + q / m ) q ( m − 1 ) ( 1 − q ) .
(3) Use windows of width θ q / m \theta q/m θ q / m , θ ↑ 1 \theta\uparrow1 θ ↑ 1 , in bands 1 , … , k 0 1,\ldots,k_0 1 , … , k 0 , with k 0 = k 0 ( m ) → ∞ k_0=k_0(m)\to\infty k 0 = k 0 ( m ) → ∞ and k 0 / m → 0 k_0/m\to0 k 0 / m → 0 , so ℓ + ≤ k 0 q / m → 0 \ell_+\le k_0q/m\to0 ℓ + ≤ k 0 q / m → 0 .
By Lemmas 50 and 52, ∑ k ≤ k 0 D k ≥ m ( m − 1 ) ( 1 − q ) − 2 m ( m − 1 ) ∑ k > k 0 T k \sum_{k\le k_0}\mathcal{D}_k\ge m(m-1)(1-q)-2m(m-1)\sum_{k>k_0}T_k ∑ k ≤ k 0 D k ≥ m ( m − 1 ) ( 1 − q ) − 2 m ( m − 1 ) ∑ k > k 0 T k .
∑ k > k 0 T k → 0 \sum_{k>k_0}T_k\to0 ∑ k > k 0 T k → 0 uniformly in m m m , by the m m m -free bound of Lemma 52.
Divide 1 2 ( 1 − ℓ + ) ( q / m ) ∑ k ≤ k 0 D k \frac{1}{2}(1-\ell_+)(q/m)\sum_{k\le k_0}\mathcal{D}_k 2 1 ( 1 − ℓ + ) ( q / m ) ∑ k ≤ k 0 D k by q m qm q m .
Parts (2) and (3) do not need the positive-curvature condition below. Per unit of q m qm q m , part (2) tends to 1 2 ( 1 − q ) 2 \frac{1}{2}(1-q)^2 2 1 ( 1 − q ) 2 as m → ∞ m\to\infty m → ∞ and part (3) gives 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) .
Theorem 54 (Finite-n n n lower bound). Fix m ≥ 2 m\ge2 m ≥ 2 , q ∈ ( 0 , 1 ) q\in(0,1) q ∈ ( 0 , 1 ) and ϵ > 0 \epsilon>0 ϵ > 0 . There is κ 0 > 0 \kappa_0>0 κ 0 > 0 such that, for each κ ∈ ( 0 , κ 0 ] \kappa\in(0,\kappa_0] κ ∈ ( 0 , κ 0 ] , there is n 0 ( κ ) n_0(\kappa) n 0 ( κ ) with the following property. For every n ≥ n 0 n\ge n_0 n ≥ n 0 there is a configuration in C n \mathcal{C}_n C n that
satisfies (A1)–(A3), in the selection setting with c = c ( X ) c=c(X) c = c ( X ) and in the outlier-detection setting;
has all m m m test units null;
has weight ratio B = κ n B=\kappa n B = κn ;
has
FDR ( BH ) ≥ q + ( L m ∗ ( q ) − ϵ ) B n ≥ q [ 1 + ( 1 2 ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) − ϵ / q ) B n ] . \operatorname{FDR}(\operatorname{BH}) \ge q+(L_m^*(q)-\epsilon)\frac{B}{n}
\ge q\left[1+\left(\frac{1}{2}(1-q)(m-1)(1-q+q/m)-\epsilon/q\right)\frac{B}{n}\right]. FDR ( BH ) ≥ q + ( L m ∗ ( q ) − ϵ ) n B ≥ q [ 1 + ( 2 1 ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) − ϵ / q ) n B ] .
Proof of Theorem 54, and hence of Theorem 16. Pick a window configuration with 1 2 ( 1 − ℓ + ) ∑ h ℓ h D k h ≥ L m ∗ − ϵ / 3 \frac{1}{2}(1-\ell_+)\sum_h\ell_h\mathcal{D}_{k_h}\ge L_m^*-\epsilon/3 2 1 ( 1 − ℓ + ) ∑ h ℓ h D k h ≥ L m ∗ − ϵ /3 . By Theorem 46, pick κ 0 \kappa_0 κ 0 so that the O ( κ 2 ) O(\kappa^2) O ( κ 2 ) term is at most ϵ κ / 3 \epsilon\kappa/3 ϵ κ /3 for κ ≤ κ 0 \kappa\le\kappa_0 κ ≤ κ 0 . By Lemma 44, pick n 0 ( κ ) n_0(\kappa) n 0 ( κ ) so that l v e r t FDR n − Φ ( κ ) ∣ ≤ ϵ κ / 3 lvert\operatorname{FDR}_n-\Phi(\kappa)\rvert\le\epsilon\kappa/3 l v er t FDR n − Φ ( κ )∣ ≤ ϵ κ /3 for n ≥ n 0 n\ge n_0 n ≥ n 0 (here FDR n = FWER n \operatorname{FDR}_n=\operatorname{FWER}_n FDR n = FWER n by Lemma 4). Then FDR n ≥ q + κ ( L m ∗ − ϵ / 3 ) − 2 ϵ κ / 3 = q + ( L m ∗ − ϵ ) B / n \operatorname{FDR}_n\ge q+\kappa(L_m^*-\epsilon/3)-2\epsilon\kappa/3=q+(L_m^*-\epsilon)B/n FDR n ≥ q + κ ( L m ∗ − ϵ /3 ) − 2 ϵ κ /3 = q + ( L m ∗ − ϵ ) B / n , since κ h = κ \kappa_h=\kappa κ h = κ gives B = κ n B=\kappa n B = κn . The second inequality is Theorem 53(2).
Theorem 16 is this statement, with the bounds on L m ∗ ( q ) L_m^*(q) L m ∗ ( q ) from Theorem 53(2) and (3).
Simes’ Hessian and conditional sharpness
Write H : = ∇ 2 ψ m ( t ) \mathbf{H}:=\nabla^2\psi_m(t) H := ∇ 2 ψ m ( t ) , and H r \mathbf{H}_r H r for H \mathbf{H} H restricted to the coordinates 2 , … , m 2,\ldots,m 2 , … , m . Write u j \mathbf{u}_j u j for the j j j -th coordinate vector of R m \mathbb{R}^m R m .
Lemma 56 (Z-matrix). H j k ≤ 0 \mathbf{H}_{jk}\le0 H j k ≤ 0 for all j ≠ k j\ne k j = k in { 1 , … , m } \{1,\ldots,m\} { 1 , … , m } .
Proof. 1. Moving F ( t j ) F(t_j) F ( t j ) up by ϵ \epsilon ϵ moves mass ϵ \epsilon ϵ from band j + 1 j+1 j + 1 to band j j j : one extra point A sits in band j j j instead of band j + 1 j+1 j + 1 . Similarly for k k k , with an extra point B.
Two-point formula. For band probabilities π \pi π , ∂ 2 ψ / ∂ π a ∂ π b = m ( m − 1 ) ⋅ P ( reject ∣ m − 2 uniforms plus one point in band a and one in band b ) \partial^2\psi/\partial\pi_a\partial\pi_b = m(m-1)\cdot\mathbb{P}(\text{reject}\mid m-2\text{ uniforms plus one point in band }a\text{ and one in band }b) ∂ 2 ψ / ∂ π a ∂ π b = m ( m − 1 ) ⋅ P ( reject ∣ m − 2 uniforms plus one point in band a and one in band b ) , and ∂ / ∂ F j = ∂ / ∂ π j − ∂ / ∂ π j + 1 \partial/\partial F_j=\partial/\partial\pi_j-\partial/\partial\pi_{j+1} ∂ / ∂ F j = ∂ / ∂ π j − ∂ / ∂ π j + 1 . So
H j k = m ( m − 1 ) E [ f ( 1 , 1 ) − f ( 1 , 0 ) − f ( 0 , 1 ) + f ( 0 , 0 ) ] , H_{jk}=m(m-1)\mathbb{E}[f(1,1)-f(1,0)-f(0,1)+f(0,0)], H j k = m ( m − 1 ) E [ f ( 1 , 1 ) − f ( 1 , 0 ) − f ( 0 , 1 ) + f ( 0 , 0 )] ,
where f ( a , b ) f(a,b) f ( a , b ) is the rejection indicator with A in band j j j (a = 1 a=1 a = 1 ) or j + 1 j+1 j + 1 (a = 0 a=0 a = 0 ), B in band k k k (b = 1 b=1 b = 1 ) or k + 1 k+1 k + 1 (b = 0 b=0 b = 0 ), and the other m − 2 m-2 m − 2 values uniform.
Let j < k j<k j < k (H is symmetric). The position of A changes the count only at index j j j , and that of B only at index k k k . So f ( a , b ) = 1 − ( 1 − o ) ( 1 − χ j ( a ) ) ( 1 − χ k ( b ) ) f(a,b)=1-(1-o)(1-\chi_j(a))(1-\chi_k(b)) f ( a , b ) = 1 − ( 1 − o ) ( 1 − χ j ( a )) ( 1 − χ k ( b )) , where o o o indicates a crossing at some index other than j j j and k k k , and χ j , χ k \chi_j,\chi_k χ j , χ k (crossing at index j j j , resp. k k k ) are non-decreasing 0/1 functions.
The mixed difference is − ( 1 − o ) ( χ j ( 1 ) − χ j ( 0 ) ) ( χ k ( 1 ) − χ k ( 0 ) ) ≤ 0 -(1-o)(\chi_j(1)-\chi_j(0))(\chi_k(1)-\chi_k(0))\leq0 − ( 1 − o ) ( χ j ( 1 ) − χ j ( 0 )) ( χ k ( 1 ) − χ k ( 0 )) ≤ 0 , pointwise. □
Lemma 57. (Row sums in closed form.) Let S l ′ S'_l S l ′ be the number of m − 2 m-2 m − 2 i.i.d. uniforms that are at most t l t_l t l , and Y l : = S l ′ − l Y_l:=S'_l-l Y l := S l ′ − l . For 2 ≤ j ≤ m 2\leq j\leq m 2 ≤ j ≤ m let r j : = ∑ k = 2 m H j k r_j:=\sum_{k=2}^{m}H_{jk} r j := ∑ k = 2 m H j k , and for 2 ≤ j ≤ m − 1 2\leq j\leq m-1 2 ≤ j ≤ m − 1 let c j : = q / ( m − q j ) c_j:=q/(m-qj) c j := q / ( m − q j ) and
E 1 : = { S 1 ′ = 0 ; Y l ≤ − 1 ( 2 ≤ l < j ) ; Y j = − 1 ; Y l ≤ − 2 ( l > j ) } , E_1:=\{S'_1=0;\ Y_l\leq-1\ (2\leq l<j);\ Y_j=-1;\ Y_l\leq-2\ (l>j)\}, E 1 := { S 1 ′ = 0 ; Y l ≤ − 1 ( 2 ≤ l < j ) ; Y j = − 1 ; Y l ≤ − 2 ( l > j )} , E 2 : = { S 1 ′ = 0 ; Y l ≤ − 2 ( 2 ≤ l < j ) ; Y j = − 2 ; Y l ≤ − 3 ( l > j ) } . E_2:=\{S'_1=0;\ Y_l\leq-2\ (2\leq l<j);\ Y_j=-2;\ Y_l\leq-3\ (l>j)\}. E 2 := { S 1 ′ = 0 ; Y l ≤ − 2 ( 2 ≤ l < j ) ; Y j = − 2 ; Y l ≤ − 3 ( l > j )} . Then, for 2 ≤ j ≤ m − 1 2\leq j\leq m-1 2 ≤ j ≤ m − 1 ,
r j = m ( m − 1 ) [ P ( E 2 ) − P ( E 1 ) ] , r_j=m(m-1)\left[\mathbb{P}(E_2)-\mathbb{P}(E_1)\right], r j = m ( m − 1 ) [ P ( E 2 ) − P ( E 1 ) ] , P ( E 1 ) = ( m − 2 j − 1 ) t j j − 1 ( 1 − t j ) m − 1 − j ⋅ 1 j ⋅ ( 1 − ( m − 1 − j ) c j ) , \mathbb{P}(E_1)=\binom{m-2}{j-1}t_j^{j-1}(1-t_j)^{m-1-j}\cdot\frac{1}{j}\cdot\left(1-(m-1-j)c_j\right), P ( E 1 ) = ( j − 1 m − 2 ) t j j − 1 ( 1 − t j ) m − 1 − j ⋅ j 1 ⋅ ( 1 − ( m − 1 − j ) c j ) , P ( E 2 ) = ( m − 2 j − 2 ) t j j − 2 ( 1 − t j ) m − j ⋅ 1 j ( j − 1 j ) j − 3 ⋅ ( 1 − ( m − j ) c j ) , \mathbb{P}(E_2)=\binom{m-2}{j-2}t_j^{j-2}(1-t_j)^{m-j}\cdot\frac{1}{j}\left(\frac{j-1}{j}\right)^{j-3}\cdot\left(1-(m-j)c_j\right), P ( E 2 ) = ( j − 2 m − 2 ) t j j − 2 ( 1 − t j ) m − j ⋅ j 1 ( j j − 1 ) j − 3 ⋅ ( 1 − ( m − j ) c j ) , and r m = ( m − 1 ) q m − 2 ( ( m − 1 ) / m ) m − 3 > 0 r_m=(m-1)q^{m-2}((m-1)/m)^{m-3}>0 r m = ( m − 1 ) q m − 2 (( m − 1 ) / m ) m − 3 > 0 .
Proof. 1. r j r_j r j is the mixed second derivative along u j \mathbf{u}_j u j (A moves from band j + 1 j+1 j + 1 to band j j j ) and along ∑ k ≥ 2 u k \sum_{k\geq2}\mathbf{u}_k ∑ k ≥ 2 u k (B moves from the top band m + 1 m+1 m + 1 to band 2). B in band 2 adds 1 to the count at every index ≥ 2 \geq2 ≥ 2 .
Write f ˉ ( a , b ) \bar f(a,b) f ˉ ( a , b ) for the indicator of “no rejection”, with a = 1 a=1 a = 1 if A is in band j j j and b = 1 b=1 b = 1 if B is in band 2. Every case needs S 1 ′ = 0 S'_1=0 S 1 ′ = 0 , and then:
for 2 ≤ l < j 2\leq l<j 2 ≤ l < j : Y l ≤ − 1 − b Y_l\leq-1-b Y l ≤ − 1 − b ;
at l = j l=j l = j : Y j ≤ − 1 − a − b Y_j\leq-1-a-b Y j ≤ − 1 − a − b ;
for l > j l>j l > j : Y l ≤ − 2 − b Y_l\leq-2-b Y l ≤ − 2 − b .
So f ˉ ( 0 , 0 ) − f ˉ ( 1 , 0 ) = 1 { E 1 } \bar f(0,0)-\bar f(1,0)=1\{E_1\} f ˉ ( 0 , 0 ) − f ˉ ( 1 , 0 ) = 1 { E 1 } and f ˉ ( 0 , 1 ) − f ˉ ( 1 , 1 ) = 1 { E 2 } \bar f(0,1)-\bar f(1,1)=1\{E_2\} f ˉ ( 0 , 1 ) − f ˉ ( 1 , 1 ) = 1 { E 2 } , and r j / ( m ( m − 1 ) ) = − ( mixed difference of f ˉ ) = P ( E 2 ) − P ( E 1 ) r_j/(m(m-1))=-(\text{mixed difference of }\bar f)=\mathbb{P}(E_2)-\mathbb{P}(E_1) r j / ( m ( m − 1 )) = − ( mixed difference of f ˉ ) = P ( E 2 ) − P ( E 1 ) .
Condition on the number of uniforms below t j t_j t j , which is j − 1 j-1 j − 1 for E 1 E_1 E 1 and j − 2 j-2 j − 2 for E 2 E_2 E 2 .
The points below t j t_j t j are uniform on [ 0 , t j ] [0,t_j] [ 0 , t j ] . After rescaling by t j t_j t j , their constraint is U ( i ) > i / j U_{(i)}>i/j U ( i ) > i / j for E 1 E_1 E 1 (Daniels: probability 1 / j 1/j 1/ j ), and U ( i ) > ( i + 1 ) / j U_{(i)}>(i+1)/j U ( i ) > ( i + 1 ) / j for E 2 E_2 E 2 (Abel–Daniels, the identity in the proof of Lemma 51: probability ( 1 / j ) ( ( j − 1 ) / j ) j − 3 (1/j)((j-1)/j)^{j-3} ( 1/ j ) (( j − 1 ) / j ) j − 3 ).
The points above t j t_j t j are uniform on ( t j , 1 ] (t_j,1] ( t j , 1 ] . In both cases the constraint is N ( c j i ) ≤ i − 1 N(c_ji)\leq i-1 N ( c j i ) ≤ i − 1 after rescaling, so Daniels gives 1 − M c j 1-Mc_j 1 − M c j , with M = m − 1 − j M=m-1-j M = m − 1 − j or M = m − j M=m-j M = m − j points. Daniels needs M c j ≤ 1 Mc_j\leq1 M c j ≤ 1 , which is equivalent to q ≤ 1 q\leq1 q ≤ 1 here. For E 1 E_1 E 1 the constraint at i = m − j i=m-j i = m − j is vacuous. At j = 2 j=2 j = 2 , the identity of Lemma 51 with M = 0 M=0 M = 0 gives 1.
For j = m j=m j = m , E 1 E_1 E 1 is impossible (it needs m − 1 m-1 m − 1 of the m − 2 m-2 m − 2 uniforms below q q q ), and E 2 E_2 E 2 is the Abel–Daniels event for m − 2 m-2 m − 2 points on [ 0 , q ] [0,q] [ 0 , q ] . □
Theorem 58 (Simes’ Hessian is PSD away from the first grid point). For every m ≥ 2 m \ge2 m ≥ 2 and every 0 < q ≤ 1 / e 0<q\le1/e 0 < q ≤ 1/ e , H r \mathbf{H}_r H r is diagonally dominant with non-negative diagonal, hence positive semidefinite.
Proof. By Lemma 56, the off-diagonal entries are non-positive, so H j j − ∑ k ≥ 2 , k ≠ j ∣ H j k ∣ = r j H_{jj}-\sum_{k\ge2,\,k\ne j}|H_{jk}|=r_j H j j − ∑ k ≥ 2 , k = j ∣ H j k ∣ = r j , and it suffices that every row sum r j r_j r j is non-negative. (A symmetric matrix A A A with A j j ≥ ∑ k ≠ j ∣ A j k ∣ A_{jj}\ge\sum_{k\ne j}|A_{jk}| A j j ≥ ∑ k = j ∣ A j k ∣ for every j j j is positive semidefinite: x ⊤ A x ≥ ∑ j A j j x j 2 − ∑ j ≠ k ∣ A j k ∣ ∣ x j ∣ ∣ x k ∣ ≥ ∑ j ( A j j − ∑ k ≠ j ∣ A j k ∣ ) x j 2 ≥ 0 x^\top A x\ge\sum_j A_{jj}x_j^2-\sum_{j\ne k}|A_{jk}||x_j||x_k|\ge\sum_j\left(A_{jj}-\sum_{k\ne j}|A_{jk}|\right)x_j^2\ge0 x ⊤ A x ≥ ∑ j A j j x j 2 − ∑ j = k ∣ A j k ∣∣ x j ∣∣ x k ∣ ≥ ∑ j ( A j j − ∑ k = j ∣ A j k ∣ ) x j 2 ≥ 0 , by 2 ∣ x j x k ∣ ≤ x j 2 + x k 2 2|x_jx_k|\le x_j^2+x_k^2 2∣ x j x k ∣ ≤ x j 2 + x k 2 and symmetry.) By Lemma 57, r m > 0 r_m>0 r m > 0 . For 2 ≤ j ≤ m − 1 2\le j\le m-1 2 ≤ j ≤ m − 1 , the ratio P ( E 2 ) / P ( E 1 ) \mathbb{P}(E_2)/\mathbb{P}(E_1) P ( E 2 ) / P ( E 1 ) simplifies to
ρ j = ( j − 1 j ) j − 2 ⋅ 1 q ⋅ Λ j , Λ j : = ( m − q j ) m ( 1 − q ) ( m − j ) ( m − q ( m − 1 ) ) . \rho_j=\left(\frac{j-1}{j}\right)^{j-2}\cdot\frac{1}{q}\cdot\Lambda_j,\qquad\Lambda_j:=\frac{(m-qj)m(1-q)}{(m-j)(m-q(m-1))}. ρ j = ( j j − 1 ) j − 2 ⋅ q 1 ⋅ Λ j , Λ j := ( m − j ) ( m − q ( m − 1 )) ( m − q j ) m ( 1 − q ) .
Λ j ≥ 1 \Lambda_j\ge1 Λ j ≥ 1 iff j m ( 1 − q ) 2 ≥ q ( m − j ) jm(1-q)^2\ge q(m-j) j m ( 1 − q ) 2 ≥ q ( m − j ) , which holds for all j ≥ 2 j\ge2 j ≥ 2 when 2 ( 1 − q ) 2 ≥ q 2(1-q)^2\ge q 2 ( 1 − q ) 2 ≥ q , that is, when q ≤ 1 / 2 q\le1/2 q ≤ 1/2 .
( ( j − 1 ) / j ) j − 2 ≥ ( 1 − 1 / j ) j − 1 ≥ 1 / e ((j-1)/j)^{j-2}\ge(1-1/j)^{j-1}\ge1/e (( j − 1 ) / j ) j − 2 ≥ ( 1 − 1/ j ) j − 1 ≥ 1/ e .
So ρ j ≥ 1 / ( e q ) ≥ 1 \rho_j\ge1/(eq)\ge1 ρ j ≥ 1/ ( e q ) ≥ 1 for q ≤ 1 / e q\le1/e q ≤ 1/ e , and r j ≥ 0 r_j\ge0 r j ≥ 0 .
Remark 59 (Sharpness of 1 / e 1/e 1/ e (numerical)). From the exact closed form, ρ j \rho_j ρ j is strictly decreasing in q q q , so diagonal dominance holds exactly on an interval [ 0 , q m ∗ ] [0,q_m^*] [ 0 , q m ∗ ] , with q m ∗ = 0.612 , 0.511 , 0.444 , 0.391 , 0.375 , 0.370 , 0.369 q_m^*=0.612,\ 0.511,\ 0.444,\ 0.391,\ 0.375,\ 0.370,\ 0.369 q m ∗ = 0.612 , 0.511 , 0.444 , 0.391 , 0.375 , 0.370 , 0.369 at m = 10 , 30 , 100 , 10 3 , 10 4 , 10 5 , 10 6 m=10,\ 30,\ 100,\ 10^3,\ 10^4,\ 10^5,\ 10^6 m = 10 , 30 , 100 , 1 0 3 , 1 0 4 , 1 0 5 , 1 0 6 ; these tend to 1 / e 1/e 1/ e , at a rate of about m − 1 / 2 m^{-1/2} m − 1/2 . Beyond q m ∗ q_m^* q m ∗ , H r \mathbf{H}_r H r can still be PSD without diagonal dominance: exact L D L ⊤ LDL^\top L D L ⊤ decompositions confirm this at q = 2 / 3 q=2/3 q = 2/3 for m = 7 , … , 13 m=7,\ldots,13 m = 7 , … , 13 and at q = 3 / 5 q=3/5 q = 3/5 for m = 12 , 13 m=12,13 m = 12 , 13 . PSD fails at q = 9 / 10 q=9/10 q = 9/10 for every m m m from 3 to 13. At m = 2 m=2 m = 2 , H r = [ 2 ] \mathbf{H}_r=[2] H r = [ 2 ] is PSD for every q q q .
Check ( m = 3 , q = 1 / 5 ) (m=3,\ q=1/5) ( m = 3 , q = 1/5 ) . Here t = ( 1 / 15 , 2 / 15 , 1 / 5 ) \mathbf{t}=(1/15,2/15,1/5) t = ( 1/15 , 2/15 , 1/5 ) and there is one free uniform U U U . Row j = 2 j=2 j = 2 : E 1 = { U ∈ ( t 1 , t 2 ] } E_1=\{U\in(t_1,t_2]\} E 1 = { U ∈ ( t 1 , t 2 ]} has probability 1 / 15 1/15 1/15 , E 2 = { U > t 3 } E_2=\{U>t_3\} E 2 = { U > t 3 } has probability 4 / 5 4/5 4/5 , and r 2 = 3 ⋅ 2 ⋅ ( 4 / 5 − 1 / 15 ) = 22 / 5 r_2=3\cdot2\cdot(4/5-1/15)=22/5 r 2 = 3 ⋅ 2 ⋅ ( 4/5 − 1/15 ) = 22/5 .
Row 3: r 3 = ( m − 1 ) q m − 2 ( ( m − 1 ) / m ) 0 = 2 / 5 r_3=(m-1)q^{m-2}((m-1)/m)^0=2/5 r 3 = ( m − 1 ) q m − 2 (( m − 1 ) / m ) 0 = 2/5 . The exact restricted Hessian is
H r = ( 24 / 5 − 2 / 5 − 2 / 5 4 / 5 ) \mathbf{H}_r=\begin{pmatrix}24/5&-2/5\\-2/5&4/5\end{pmatrix} H r = ( 24/5 − 2/5 − 2/5 4/5 )
its row sums are 22 / 5 22/5 22/5 and 2 / 5 2/5 2/5 , the off-diagonal entry is negative, and the matrix is positive definite.
Corollary 60 (Within the window family, windows above t 1 t_1 t 1 ). Let 0 < q ≤ 1 / e 0<q\le1/e 0 < q ≤ 1/ e . For an interval J ⊂ ( t 1 , 1 ) J\subset(t_1,1) J ⊂ ( t 1 , 1 ) of length ∣ J ∣ |J| ∣ J ∣ , let d ˉ J : = ∣ J ∣ − 1 ∫ J d s d s \bar d_J:=|J|^{-1}\int_J d_s\,ds d ˉ J := ∣ J ∣ − 1 ∫ J d s d s and D ( J ) : = d ˉ J ⊤ ∇ 2 ψ d ˉ J \mathcal{D}(J):=\bar d_J^\top\nabla^2\psi\,\bar d_J D ( J ) := d ˉ J ⊤ ∇ 2 ψ d ˉ J , the quadratic form at the average of d s d_s d s over J J J . Then ∣ J ∣ D ( J ) ≤ ∫ J D ( s ) d s |J|\mathcal{D}(J)\le\int_J\mathcal{D}(s)\,ds ∣ J ∣ D ( J ) ≤ ∫ J D ( s ) d s . Hence every Poisson window configuration whose windows J h J_h J h (pairwise disjoint, of lengths ℓ h \ell_h ℓ h , ℓ + = ∑ h ℓ h < 1 \ell_+=\sum_h\ell_h<1 ℓ + = ∑ h ℓ h < 1 ) lie in ( t 1 , 1 ) (t_1,1) ( t 1 , 1 ) satisfies
1 2 ( 1 − ℓ + ) ∑ h ℓ h D ( J h ) ≤ 1 2 q m ∑ k = 1 m − 1 D k + , D k + : = max { D k , 0 } . \frac{1}{2}(1-\ell_+)\sum_h\ell_h\mathcal{D}(J_h)\le\frac{1}{2}\frac{q}{m}\sum_{k=1}^{m-1}\mathcal{D}_k^+,\qquad\mathcal{D}_k^+:=\max\{\mathcal{D}_k,0\}. 2 1 ( 1 − ℓ + ) h ∑ ℓ h D ( J h ) ≤ 2 1 m q k = 1 ∑ m − 1 D k + , D k + := max { D k , 0 } . Proof. 1. For s ∈ J s\in J s ∈ J we have s > t 1 s>t_1 s > t 1 , so d s ( 1 ) = t 1 d_s(1)=t_1 d s ( 1 ) = t 1 : the vectors d s d_s d s differ only in coordinates 2 , … , m 2,\ldots,m 2 , … , m . By Theorem 58, the quadratic form x ↦ x ⊤ ∇ 2 ψ x x\mapsto x^\top\nabla^2\psi\,x x ↦ x ⊤ ∇ 2 ψ x is convex on their affine hull.
Jensen’s inequality then gives ∣ J ∣ D ( J ) ≤ ∫ J D ( s ) d s |J|\mathcal{D}(J)\le\int_J\mathcal{D}(s)\,ds ∣ J ∣ D ( J ) ≤ ∫ J D ( s ) d s .
The windows are pairwise disjoint and 0 ≤ 1 − ℓ + ≤ 1 0\le1-\ell_+\le1 0 ≤ 1 − ℓ + ≤ 1 , so 1 2 ( 1 − ℓ + ) ∑ h ℓ h D ( J h ) ≤ 1 2 ∫ ( t 1 , 1 ) D ( s ) + d s \frac{1}{2}(1-\ell_+)\sum_h\ell_h\mathcal{D}(J_h)\le\frac{1}{2}\int_{(t_1,1)}\mathcal{D}(s)^+\,ds 2 1 ( 1 − ℓ + ) ∑ h ℓ h D ( J h ) ≤ 2 1 ∫ ( t 1 , 1 ) D ( s ) + d s . On band k + 1 k+1 k + 1 , D ( s ) = D k \mathcal{D}(s)=\mathcal{D}_k D ( s ) = D k , and the top band has D ( s ) = t ⊤ ∇ 2 ψ t = 0 \mathcal{D}(s)=\mathbf{t}^\top\nabla^2\psi\,\mathbf{t}=0 D ( s ) = t ⊤ ∇ 2 ψ t = 0 (Lemma 50), so it contributes nothing. The integral is therefore q m ∑ k = 1 m − 1 D k + \frac{q}{m}\sum_{k=1}^{m-1}\mathcal{D}_k^+ m q ∑ k = 1 m − 1 D k + .
This inequality does not need the positive-curvature condition below. With it (D k ≥ 0 \mathcal{D}_k\ge0 D k ≥ 0 for all k k k ), the right-hand side equals 1 2 q m m ( m − 1 ) ( 1 − q ) = 1 2 q ( 1 − q ) ( m − 1 ) \frac{1}{2}\frac{q}{m}m(m-1)(1-q)=\frac{1}{2}q(1-q)(m-1) 2 1 m q m ( m − 1 ) ( 1 − q ) = 2 1 q ( 1 − q ) ( m − 1 ) by Lemma 50. That matches the lower bound of Theorem 53(2) up to the factor 1 − q + q / m 1-q+q/m 1 − q + q / m , and is asymptotically sharp: per unit of q m qm q m it is 1 2 ( 1 − q ) ( m − 1 ) / m → 1 2 ( 1 − q ) \frac{1}{2}(1-q)(m-1)/m\to\frac{1}{2}(1-q) 2 1 ( 1 − q ) ( m − 1 ) / m → 2 1 ( 1 − q ) , the constant of Theorem 53(3).
For the window configurations of Section D.3, whose windows lie strictly inside one band each, the left-hand side is the coefficient of Theorem 46, and the same bound is immediate without Theorem 58: windows in the same band have total length at most q / m q/m q / m , so 1 2 ( 1 − ℓ + ) ∑ h ℓ h D k h ≤ 1 2 q m ∑ k D k + \frac{1}{2}(1-\ell_+)\sum_h\ell_h\mathcal{D}_{k_h}\le\frac{1}{2}\frac{q}{m}\sum_k\mathcal{D}_k^+ 2 1 ( 1 − ℓ + ) ∑ h ℓ h D k h ≤ 2 1 m q ∑ k D k + . Corollary 60 adds windows above t 1 t_1 t 1 that contain grid points. We have proved Theorem 46 only for windows strictly inside one band, because its good event needs this. For windows in ( t 1 , 1 ) (t_1,1) ( t 1 , 1 ) that contain a grid point, we have not
proved that 1 2 κ ( 1 − ℓ + ) ∑ h ℓ h D ( J h ) \frac{1}{2}\kappa(1-\ell_+)\sum_h \ell_h\mathcal{D}(J_h) 2 1 κ ( 1 − ℓ + ) ∑ h ℓ h D ( J h ) is the first-order term of Φ − q \Phi-q Φ − q . Windows that reach below t 1 t_1 t 1 are covered only numerically (a scan over windows straddling many grid points), and configurations outside the Poisson window family are open.
The positive-curvature condition. D k > 0 \mathcal{D}_k>0 D k > 0 for all 1 ≤ k ≤ m − 1 1\leq k\leq m-1 1 ≤ k ≤ m − 1 . It is false for large q q q : by Lemma 51, D 1 < 0 \mathcal{D}_1<0 D 1 < 0 when q > m ( 1 − 2 − 1 / ( m − 2 ) ) q>m(1-2^{-1/(m-2)}) q > m ( 1 − 2 − 1/ ( m − 2 ) ) , for example when q > 0.9546 q>0.9546 q > 0.9546 at m = 6 m=6 m = 6 . It is open in general; it has been verified in exact rational arithmetic for all k k k at q ∈ { 1 / 20 , 1 / 10 , 1 / 5 , 1 / 2 , 7 / 10 } q\in\{1/20,1/10,1/5,1/2,7/10\} q ∈ { 1/20 , 1/10 , 1/5 , 1/2 , 7/10 } and m ∈ { 10 , 16 , 24 , 32 } m\in\{10,16,24,32\} m ∈ { 10 , 16 , 24 , 32 } and for all q ∈ ( 0 , 1 ) q\in(0,1) q ∈ ( 0 , 1 ) and k ≥ 2 k\geq2 k ≥ 2 when m ≤ 9 m\leq9 m ≤ 9 , and numerically for m ≤ 1024 m\leq1024 m ≤ 1024 at q ∈ { 0.05 , 0.1 , 0.2 } q\in\{0.05,0.1,0.2\} q ∈ { 0.05 , 0.1 , 0.2 } (up to finite-difference noise) and in the m → ∞ m\to\infty m → ∞ limit for k ≤ 25 k\leq25 k ≤ 25 . Writing a 2 − 2 a 1 + a 0 a_2-2a_1+a_0 a 2 − 2 a 1 + a 0 as a second difference of the two-point rejection kernel does not prove it, because that kernel is not conditionally positive semidefinite over all bands (a value in band 1 always rejects).
Proof of Proposition 19. The off-diagonal entries of Simes’ Hessian at the uniform point are non-positive by Lemma 56 (all j ≠ k j\ne k j = k in { 1 , … , m } \{1,\ldots,m\} { 1 , … , m } ), and for 0 < q ≤ 1 / e 0<q\leq1/e 0 < q ≤ 1/ e its restriction to grid points 2 , … , m 2,\ldots,m 2 , … , m is positive semidefinite by Theorem 58 . For the conditional claim, assume the positive-curvature condition. For every window configuration of Section D.3, the coefficient of Theorem 46 satisfies 1 2 ( 1 − ℓ + ) ∑ h ℓ h D k h ≤ 1 2 q ∑ k D k + = 1 2 q ( 1 − q ) ( m − 1 ) \frac{1}{2}(1-\ell_+)\sum_h \ell_h\mathcal{D}_{k_h}\leq\frac{1}{2}q\sum_k\mathcal{D}_k^+=\frac{1}{2}q(1-q)(m-1) 2 1 ( 1 − ℓ + ) ∑ h ℓ h D k h ≤ 2 1 q ∑ k D k + = 2 1 q ( 1 − q ) ( m − 1 ) (the remark above and Lemma 50 ), that is, at most 1 2 ( 1 − q ) ( m − 1 ) / m \frac{1}{2}(1-q)(m-1)/m 2 1 ( 1 − q ) ( m − 1 ) / m per unit of q m qm q m ; this uses neither Theorem 58 nor q ≤ 1 / e q\leq1/e q ≤ 1/ e . The configurations in the proofs of Theorem 53 (2) and Theorem 53 (3) are window configurations; they reach 1 2 q ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) \frac{1}{2}q(1-q)(m-1)(1-q+q/m) 2 1 q ( 1 − q ) ( m − 1 ) ( 1 − q + q / m ) and, per unit of q m qm q m , 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) as m → ∞ m\to\infty m → ∞ . So within this family the constant per unit of q m B / n qmB/n q m B / n is below 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) at every m m m and tends to 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) as m → ∞ m\to\infty m → ∞ .
Proofs for the envelope theorem and the global-null upper bound
Setting and assumptions
Calibration points are Z i = ( X i , Y i ) Z_i=(X_i,Y_i) Z i = ( X i , Y i ) , i = 1 , … , n i=1,\ldots,n i = 1 , … , n . Test unit j ∈ { 1 , … , m } j\in\{1,\ldots,m\} j ∈ { 1 , … , m } has data Z n + j = ( X n + j , Y n + j ) Z_{n+j}=(X_{n+j},Y_{n+j}) Z n + j = ( X n + j , Y n + j ) , a threshold c n + j c_{n+j} c n + j and a tie-breaking variable U j U_j U j ; it is null iff Y n + j ≤ c n + j Y_{n+j}\leq c_{n+j} Y n + j ≤ c n + j .
Assumption 61 ((A1)–(A3)).
(A1) The calibration points Z 1 , … , Z n Z_1,\ldots,Z_n Z 1 , … , Z n are i.i.d. from P P P on X × Y \mathcal{X}\times\mathcal{Y} X × Y . For each test unit j j j , Z n + j ∼ Q Z_{n+j}\sim Q Z n + j ∼ Q , where
d Q d P ( x , y ) = w ( x ) E P [ w ( X ) ] and 1 ≤ w ( x ) ≤ B . \frac{\mathrm{d}Q}{\mathrm{d}P}(x,y)=\frac{w(x)}{\mathbb{E}_P[w(X)]}
\qquad\text{and}\qquad
1\leq w(x)\leq B. d P d Q ( x , y ) = E P [ w ( X )] w ( x ) and 1 ≤ w ( x ) ≤ B .
(A2) For every j j j , with H j : = ( X n + ℓ , c n + ℓ , U ℓ ) ℓ ≠ j H_j:=(X_{n+\ell},c_{n+\ell},U_\ell)_{\ell\ne j} H j := ( X n + ℓ , c n + ℓ , U ℓ ) ℓ = j :
(i) Z 1 , … , Z n , Z n + j Z_1,\ldots,Z_n,Z_{n+j} Z 1 , … , Z n , Z n + j are mutually independent;
(ii) H j H_j H j is independent of ( Z 1 , … , Z n , Z n + j ) (Z_1,\ldots,Z_n,Z_{n+j}) ( Z 1 , … , Z n , Z n + j ) ;
(iii) U j ∼ Unif ( 0 , 1 ) U_j\sim\operatorname{Unif}(0,1) U j ∼ Unif ( 0 , 1 ) is independent of ( Z 1 , … , Z n , Z n + j , H j ) (Z_1,\ldots,Z_n,Z_{n+j},H_j) ( Z 1 , … , Z n , Z n + j , H j ) .
(A3) The score V : X × Y → R V:\mathcal{X}\times\mathcal{Y}\to\mathbb{R} V : X × Y → R is monotone: y ≤ y ′ y\leq y' y ≤ y ′ implies V ( x , y ) ≤ V ( x , y ′ ) V(x,y)\leq V(x,y') V ( x , y ) ≤ V ( x , y ′ ) .
Comments on the assumptions. The p p p -values below are invariant to rescaling w w w , so the normalisation 1 ≤ w 1\leq w 1 ≤ w loses nothing. Parts (ii) and (iii) of (A2) together say that the law of ( Z 1 , … , Z n , Z n + j , H j , U j ) (Z_1,\ldots,Z_n,Z_{n+j},H_j,U_j) ( Z 1 , … , Z n , Z n + j , H j , U j ) is the product of the law of the Z Z Z ’s, the law P H j \mathbb{P}_{H_j} P H j of H j H_j H j , and Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) ; this is what Lemma 67 states. The other test units may depend on each other arbitrarily, and their responses Y n + ℓ Y_{n+\ell} Y n + ℓ and the threshold c n + ℓ c_{n+\ell} c n + ℓ play no role below. Jin & Candès [12 ] assume i.i.d. test tuples and independent U U U ’s, which implies (A2). What (A2) excludes are covariates X n + ℓ X_{n+\ell} X n + ℓ or thresholds c n + ℓ c_{n+\ell} c n + ℓ that depend on Z n + j Z_{n+j} Z n + j or on the calibration data (for example, thresholds tuned on the data).
Notation. V i : = V ( X i , Y i ) V_i := V(X_i,Y_i) V i := V ( X i , Y i ) for i ≤ n + m i \le n+m i ≤ n + m , V ^ n + ℓ : = V ( X n + ℓ , c n + ℓ ) \widehat{V}_{n+\ell} := V(X_{n+\ell},c_{n+\ell}) V n + ℓ := V ( X n + ℓ , c n + ℓ ) and w i : = w ( X i ) w_i := w(X_i) w i := w ( X i ) . For a calibration multiset C C C of n n n points and a test unit ℓ \ell ℓ , write
p ℓ [ C ] : = ∑ e ∈ C w e 1 { V e < V ^ n + ℓ } + U ℓ ( w n + ℓ + ∑ e ∈ C w e 1 { V e = V ^ n + ℓ } ) ∑ e ∈ C w e + w n + ℓ p_\ell[C] := \frac{\sum_{e\in C} w_e 1\{V_e < \widehat{V}_{n+\ell}\} + U_\ell\left(w_{n+\ell}+\sum_{e\in C}w_e1\{V_e=\widehat{V}_{n+\ell}\}\right)}{\sum_{e\in C}w_e+w_{n+\ell}} p ℓ [ C ] := ∑ e ∈ C w e + w n + ℓ ∑ e ∈ C w e 1 { V e < V n + ℓ } + U ℓ ( w n + ℓ + ∑ e ∈ C w e 1 { V e = V n + ℓ } )
where w e w_e w e and V e V_e V e are the weight and score of the point e e e . The reported p-value is p ℓ : = p ℓ [ { Z 1 , … , Z n } ] p_\ell:= p_\ell[\{Z_1,\ldots,Z_n\}] p ℓ := p ℓ [{ Z 1 , … , Z n }] , which is eq. (4) of Jin & Candès [12 ] . BH at level q q q rejects R : = { ℓ : p ℓ ≤ q k ∗ / m } R := \{\ell: p_\ell\le qk^*/m\} R := { ℓ : p ℓ ≤ q k ∗ / m } , with k ∗ : = max { k : # { ℓ : p ℓ ≤ q k / m } ≥ k } k^* := \max\{k : \#\{\ell: p_\ell\le qk/m\} \ge k\} k ∗ := max { k : # { ℓ : p ℓ ≤ q k / m } ≥ k } (k ∗ : = 0 k^* := 0 k ∗ := 0 if there is no such k k k ); the grid is t k = q k / m t_k=qk/m t k = q k / m . For x ∈ R m − 1 x\in\mathbb{R}^{m-1} x ∈ R m − 1 , R 0 ( x ) R^0(x) R 0 ( x ) denotes the number of BH rejections at level q q q among the m m m numbers ( x 1 , … , x m − 1 , 0 ) (x_1,\ldots,x_{m-1},0) ( x 1 , … , x m − 1 , 0 ) . Then R 0 ≥ 1 R^0\ge1 R 0 ≥ 1 , because the forced 0 0 0 is ≤ q / m \le q/m ≤ q / m , and R 0 R^0 R 0 is non-increasing in each coordinate. For x ∈ R m − 1 x\in\mathbb{R}^{m-1} x ∈ R m − 1 and s ∈ R s\in\mathbb{R} s ∈ R , x ± s x\pm s x ± s is x x x with s s s added to or subtracted from every coordinate.
Statements
Fix j j j . Define:
S j S_j S j : the multiset { Z 1 , … , Z n , Z n + j } \{Z_1,\ldots,Z_n,Z_{n+j}\} { Z 1 , … , Z n , Z n + j } of n + 1 n+1 n + 1 points;
for e ∈ S j e\in S_j e ∈ S j , p − j ( e ) : = ( p ℓ [ S j ∖ { e } ] ) ℓ ≠ j ∈ R m − 1 p_{-j}(e):=(p_\ell[S_j\setminus\{e\}])_{\ell\ne j}\in\mathbb{R}^{m-1} p − j ( e ) := ( p ℓ [ S j ∖ { e }] ) ℓ = j ∈ R m − 1 ;
ϱ j ( e ) : = R 0 ( p − j ( e ) ) \varrho_j(e):=R^0(p_{-j}(e)) ϱ j ( e ) := R 0 ( p − j ( e )) (we write ϱ \varrho ϱ to keep ρ \rho ρ free for the parameter of Sections E.6 and E.7);
e ⪯ e ′ e\preceq e' e ⪯ e ′ iff V e ≤ V e ′ V_e\le V_{e'} V e ≤ V e ′ ; a tie group is a class of equal scores. For a tie group G G G with common score v G v_G v G , write e ⪰ G e\succeq G e ⪰ G if V e ≥ v G V_e\ge v_G V e ≥ v G and e ⪯ G e\preceq G e ⪯ G if V e ≤ v G V_e\le v_G V e ≤ v G .
Copies of the same atom give the same multiset S j ∖ { e } S_j\setminus\{e\} S j ∖ { e } , so ϱ j \varrho_j ϱ j and the tie group are functions of the value of e e e , not of its label. Define
Γ j ∗ : = max G max { ϱ j ( e ) : e ⪰ G } min { ϱ j ( e ) : e ⪯ G } , \Gamma_j^*:=\max_G\frac{\max\{\varrho_j(e):e\succeq G\}}{\min\{\varrho_j(e):e\preceq G\}}, Γ j ∗ := G max min { ϱ j ( e ) : e ⪯ G } max { ϱ j ( e ) : e ⪰ G } ,
the maximum being over the tie groups G G G of S j S_j S j . For ℓ ≠ j \ell\ne j ℓ = j let W ℓ : = ∑ e ∈ S j w e + w n + ℓ W_\ell:=\sum_{e\in S_j}w_e+w_{n+\ell} W ℓ := ∑ e ∈ S j w e + w n + ℓ and ϕ ℓ ( w ) : = w / ( W ℓ − w ) \phi_\ell(w):=w/(W_\ell-w) ϕ ℓ ( w ) := w / ( W ℓ − w ) ; let w max w_{\max} w m a x and w min w_{\min} w m i n be the largest and smallest weights in S j S_j S j , and
δ j : = max ℓ ≠ j [ ϕ ℓ ( w max ) − ϕ ℓ ( w min ) ] . \delta_j:=\max_{\ell\ne j}\left[\phi_\ell(w_{\max})-\phi_\ell(w_{\min})\right]. δ j := ℓ = j max [ ϕ ℓ ( w m a x ) − ϕ ℓ ( w m i n ) ] .
Theorem 62 (Envelope, full form). Under (A1)–(A3),
FDR ( BH ) ≤ q m ∑ j = 1 m E [ Γ j ∗ ] ≤ q m ∑ j = 1 m E [ Γ j ] , Γ j : = max e 0 ∈ S j R 0 ( p − j ( e 0 ) − δ j ) R 0 ( p − j ( e 0 ) + δ j ) \operatorname{FDR}(\operatorname{BH})\le\frac{q}{m}\sum_{j=1}^{m}\mathbb{E}[\Gamma_j^*]\le\frac{q}{m}\sum_{j=1}^{m}\mathbb{E}[\Gamma_j],\qquad\Gamma_j:=\max_{e_0\in S_j}\frac{R^0(p_{-j}(e_0)-\delta_j)}{R^0(p_{-j}(e_0)+\delta_j)} FDR ( BH ) ≤ m q j = 1 ∑ m E [ Γ j ∗ ] ≤ m q j = 1 ∑ m E [ Γ j ] , Γ j := e 0 ∈ S j max R 0 ( p − j ( e 0 ) + δ j ) R 0 ( p − j ( e 0 ) − δ j ) and deterministically
δ j ≤ δ n ( B ) : = ( B − 1 ) ( B + n + 1 ) ( n + 1 ) ( B + n ) . \delta_j\le\delta_n(B):=\frac{(B-1)(B+n+1)}{(n+1)(B+n)}. δ j ≤ δ n ( B ) := ( n + 1 ) ( B + n ) ( B − 1 ) ( B + n + 1 ) . If w w w is constant, then δ j = 0 \delta_j=0 δ j = 0 , so Γ j = 1 \Gamma_j=1 Γ j = 1 and FDR ≤ q \operatorname{FDR}\le q FDR ≤ q .
Corollary 63 (Computable envelopes). Let p − j o b s : = ( p ℓ ) ℓ ≠ j p^{\mathrm{obs}}_{-j}:=(p_\ell)_{\ell\ne j} p − j obs := ( p ℓ ) ℓ = j be the observed p-values of the other test units, and put τ ˉ j : = max ℓ ≠ j ϕ ℓ ( w max ) \bar{\tau}_j:=\max_{\ell\ne j}\phi_\ell(w_{\max}) τ ˉ j := max ℓ = j ϕ ℓ ( w m a x ) and η j : = δ j + τ ˉ j \eta_j:=\delta_j+\bar{\tau}_j η j := δ j + τ ˉ j . Define
Γ ~ j : = R 0 ( p − j o b s − τ ˉ j ) R 0 ( p − j o b s + τ ˉ j ) , Γ ^ j : = R 0 ( p − j o b s − η j ) R 0 ( p − j o b s + η j ) . \widetilde{\Gamma}_j:=\frac{R^0(p^{\mathrm{obs}}_{-j}-\bar{\tau}_j)}{R^0(p^{\mathrm{obs}}_{-j}+\bar{\tau}_j)},\qquad\widehat{\Gamma}_j:=\frac{R^0(p^{\mathrm{obs}}_{-j}-\eta_j)}{R^0(p^{\mathrm{obs}}_{-j}+\eta_j)}. Γ j := R 0 ( p − j obs + τ ˉ j ) R 0 ( p − j obs − τ ˉ j ) , Γ j := R 0 ( p − j obs + η j ) R 0 ( p − j obs − η j ) . Under (A1)–(A3), Γ j ∗ ≤ Γ ~ j ≤ Γ ^ j \Gamma_j^*\le\widetilde{\Gamma}_j\le\widehat{\Gamma}_j Γ j ∗ ≤ Γ j ≤ Γ j pointwise, and hence
FDR ( BH ) ≤ q m ∑ j E [ Γ ~ j ] ≤ q m ∑ j E [ Γ ^ j ] . \operatorname{FDR}(\operatorname{BH})\le\frac{q}{m}\sum_j\mathbb{E}[\widetilde{\Gamma}_j]\le\frac{q}{m}\sum_j\mathbb{E}[\widehat{\Gamma}_j]. FDR ( BH ) ≤ m q j ∑ E [ Γ j ] ≤ m q j ∑ E [ Γ j ] . Γ ~ j \widetilde{\Gamma}_j Γ j and Γ ^ j \widehat{\Gamma}_j Γ j use only observed quantities: the calibration data, the test covariates and the other test units’ p-values. So they are computable in both the selection and the outlier-detection setting. The shifts satisfy
τ ˉ j ≤ τ ^ j / ( 1 − τ ^ j ) \bar{\tau}_j \le\hat{\tau}_j/(1-\hat{\tau}_j) τ ˉ j ≤ τ ^ j / ( 1 − τ ^ j ) , where τ ^ j : = w max / ∑ e ∈ S j w e \hat{\tau}_j := w_{\max}/\sum_{e\in S_j} w_e τ ^ j := w m a x / ∑ e ∈ S j w e ;
τ ˉ j ≤ B / ( n + 1 ) \bar{\tau}_j \le B/(n+1) τ ˉ j ≤ B / ( n + 1 ) deterministically;
η j ≤ 2 τ ^ j / ( 1 − τ ^ j ) \eta_j \le2\hat{\tau}_j/(1-\hat{\tau}_j) η j ≤ 2 τ ^ j / ( 1 − τ ^ j ) .
Γ ~ j \tilde{\Gamma}_j Γ ~ j does not bound Γ j \Gamma_j Γ j in general: exact enumeration finds configurations with Γ j > Γ ~ j \Gamma_j > \tilde{\Gamma}_j Γ j > Γ ~ j . The chain is Γ j ∗ ≤ Γ ~ j ≤ Γ ^ j \Gamma_j^* \le\tilde{\Gamma}_j \le\hat{\Gamma}_j Γ j ∗ ≤ Γ ~ j ≤ Γ ^ j , with Γ j \Gamma_j Γ j a separate bound that is not computable (Section E.5).
Proposition 64 (The tightest computable envelope). Γ j ∗ \Gamma_j^* Γ j ∗ depends on the unobserved Y n + j Y_{n+j} Y n + j only through v = V n + j v=V_{n+j} v = V n + j , and only through how v v v is ordered relative to the calibration scores V 1 , … , V n V_1,\ldots,V_n V 1 , … , V n and the V ^ n + ℓ \hat{V}_{n+\ell} V ^ n + ℓ , ℓ ≠ j \ell\ne j ℓ = j . Write Γ j ∗ ( v ) \Gamma_j^*(v) Γ j ∗ ( v ) for Γ j ∗ \Gamma_j^* Γ j ∗ computed with test score v v v , and let r r r be the number of distinct values among those scores. Then
Γ j s u p : = sup v Γ j ∗ ( v ) \Gamma_j^{\mathrm{sup}}:=\sup_v\Gamma_j^*(v) Γ j sup := v sup Γ j ∗ ( v ) is a maximum over at most 2 r + 1 2r+1 2 r + 1 cases, it is computable, and Γ j ∗ ≤ Γ j s u p ≤ Γ ~ j \Gamma_j^*\le\Gamma_j^{\mathrm{sup}}\le\tilde{\Gamma}_j Γ j ∗ ≤ Γ j sup ≤ Γ ~ j .
Remark 65. In 40 random exact examples with m = 2 m=2 m = 2 , the excess of q m ∑ j E [ Γ j s u p ] \frac{q}{m}\sum_j\mathbb{E}[\Gamma_j^{\mathrm{sup}}] m q ∑ j E [ Γ j sup ] over q q q had median 2.9% (maximum 35%) of the excess of q m ∑ j E [ Γ ^ j ] \frac{q}{m}\sum_j\mathbb{E}[\hat{\Gamma}_j] m q ∑ j E [ Γ ^ j ] (numerical).
The leave-one-out identity and the conditional law of the test point
Lemma 66 (Leave-one-out identity). Let p 1 , … , p m p_1,\ldots,p_m p 1 , … , p m be real numbers, let R R R be the BH rejection set at level q q q , fix j j j , and let R j : = R 0 ( ( p ℓ ) ℓ ≠ j ) R_j:=R^0((p_\ell)_{\ell\ne j}) R j := R 0 (( p ℓ ) ℓ = j ) , the BH count when p j p_j p j is replaced by 0. Then j ∈ R j\in R j ∈ R iff p j ≤ q R j / m p_j\le qR_j/m p j ≤ q R j / m , and on that event ∣ R ∣ = R j |R|=R_j ∣ R ∣ = R j .
Proof. Let N ( k ) : = # { ℓ : p ℓ ≤ q k / m } N(k):=\#\{\ell:p_\ell\le qk/m\} N ( k ) := # { ℓ : p ℓ ≤ q k / m } , and N j ( k ) N_j(k) N j ( k ) the same count with p j : = 0 p_j:=0 p j := 0 ; so k ∗ = max { k : N ( k ) ≥ k } k^*=\max\{k:N(k)\ge k\} k ∗ = max { k : N ( k ) ≥ k } , R j = max { k : N j ( k ) ≥ k } R_j=\max\{k:N_j(k)\ge k\} R j = max { k : N j ( k ) ≥ k } , N ≤ N j N\le N_j N ≤ N j , and ∣ R ∣ = N ( k ∗ ) = k ∗ |R|=N(k^*)=k^* ∣ R ∣ = N ( k ∗ ) = k ∗ when k ∗ ≥ 1 k^*\ge1 k ∗ ≥ 1 . If p j ≤ q k / m p_j\le qk/m p j ≤ q k / m , then N ( k ) = N j ( k ) N(k)=N_j(k) N ( k ) = N j ( k ) .
If p j ≤ q R j / m p_j\le qR_j/m p j ≤ q R j / m , then N ( R j ) = N j ( R j ) ≥ R j N(R_j)=N_j(R_j)\ge R_j N ( R j ) = N j ( R j ) ≥ R j , so k ∗ ≥ R j k^*\ge R_j k ∗ ≥ R j . For any k > R j k>R_j k > R j , N ( k ) ≤ N j ( k ) < k N(k)\le N_j(k)<k N ( k ) ≤ N j ( k ) < k , so k ∗ = R j k^*=R_j k ∗ = R j . Since p j ≤ q k ∗ / m p_j\le qk^*/m p j ≤ q k ∗ / m , j ∈ R j\in R j ∈ R and ∣ R ∣ = R j |R|=R_j ∣ R ∣ = R j .
If j ∈ R j\in R j ∈ R , then p j ≤ q k ∗ / m p_j\le qk^*/m p j ≤ q k ∗ / m , so N j ( k ∗ ) = N ( k ∗ ) ≥ k ∗ N_j(k^*)=N(k^*)\ge k^* N j ( k ∗ ) = N ( k ∗ ) ≥ k ∗ and R j ≥ k ∗ R_j\ge k^* R j ≥ k ∗ . For k ≥ k ∗ k\ge k^* k ≥ k ∗ , p j ≤ q k / m p_j\le qk/m p j ≤ q k / m , so N ( k ) = N j ( k ) N(k)=N_j(k) N ( k ) = N j ( k ) , and therefore R j = k ∗ R_j=k^* R j = k ∗ .
Lemma 67 (Conditional law of the test point; weighted exchangeability). Fix j j j and assume (A1) and (A2) for this j j j , with X × Y \mathcal{X}\times\mathcal{Y} X × Y a standard Borel space. In this lemma and its proof, relabel the test point as Z n + 1 : = Z n + j Z_{n+1}:=Z_{n+j} Z n + 1 := Z n + j , and write S : = S j S:=S_j S := S j (the multiset, that is, the empirical measure ∑ i = 1 n + 1 δ Z i \sum_{i=1}^{n+1}\delta_{Z_i} ∑ i = 1 n + 1 δ Z i ), H : = H j H:=H_j H := H j and U : = U j U:=U_j U := U j . Then for every bounded measurable f f f ,
E [ f ( Z n + 1 , S , H , U ) ∣ S , H ] = ∑ k = 1 n + 1 π k ∫ 0 1 f ( z k , S , H , u ) d u , π k : = w ( z k ) ∑ ℓ = 1 n + 1 w ( z ℓ ) , \mathbb{E}[f(Z_{n+1},S,H,U)\mid S,H]=\sum_{k=1}^{n+1}\pi_k\int_0^1f(z_k,S,H,u)\,du,\qquad\pi_k:=\frac{w(z_k)}{\sum_{\ell=1}^{n+1}w(z_\ell)}, E [ f ( Z n + 1 , S , H , U ) ∣ S , H ] = k = 1 ∑ n + 1 π k ∫ 0 1 f ( z k , S , H , u ) d u , π k := ∑ ℓ = 1 n + 1 w ( z ℓ ) w ( z k ) , where z 1 , … , z n + 1 z_1,\ldots,z_{n+1} z 1 , … , z n + 1 is any enumeration of S S S (with repeats).
The right-hand side is a symmetric function of ( z 1 , … , z n + 1 ) (z_1,\ldots,z_{n+1}) ( z 1 , … , z n + 1 ) , so it is σ ( S , H ) \sigma(S,H) σ ( S , H ) -measurable. Symmetric implies σ ( S ) \sigma(S) σ ( S ) -measurable: take a Borel isomorphism of the space onto a Borel subset of [ 0 , 1 ] [0,1] [ 0 , 1 ] [13 ] ; a symmetric Borel function f f f then equals f ~ ∘ \tilde{f}\circ f ~ ∘ (order statistics of the images), and the order statistics are functions of S S S .
Proof. Let P ⊗ ( n + 1 ) P^{\otimes(n+1)} P ⊗ ( n + 1 ) be the law under which all n + 1 n+1 n + 1 points are i.i.d. P P P , and let w ˉ : = E P w \bar{w}:=\mathbb{E}_P w w ˉ := E P w . By (A1) and the independence in (A2)(i), the true law of ( Z 1 , … , Z n + 1 ) (Z_1,\ldots,Z_{n+1}) ( Z 1 , … , Z n + 1 ) has density w ( z n + 1 ) / w ˉ w(z_{n+1})/\bar{w} w ( z n + 1 ) / w ˉ with respect to P ⊗ ( n + 1 ) P^{\otimes(n+1)} P ⊗ ( n + 1 ) (here w ( z ) : = w ( x ) w(z):=w(x) w ( z ) := w ( x ) for z = ( x , y ) z=(x,y) z = ( x , y ) ).
First part. Take a bounded measurable χ ( z , S ) \chi(z,S) χ ( z , S ) . For any bounded symmetric g ( S ) g(S) g ( S ) ,
E [ χ ( Z n + 1 , S ) g ( S ) ] = E P ⊗ ( n + 1 ) [ w ( Z n + 1 ) w ˉ χ ( Z n + 1 , S ) g ( S ) ] = 1 n + 1 ∑ k = 1 n + 1 E P ⊗ ( n + 1 ) [ w ( Z k ) w ˉ χ ( Z k , S ) g ( S ) ] . \mathbb{E}[\chi(Z_{n+1},S)g(S)]=\mathbb{E}_{P^{\otimes(n+1)}}\left[\frac{w(Z_{n+1})}{\bar{w}}\chi(Z_{n+1},S)g(S)\right]=\frac{1}{n+1}\sum_{k=1}^{n+1}\mathbb{E}_{P^{\otimes(n+1)}}\left[\frac{w(Z_k)}{\bar{w}}\chi(Z_k,S)g(S)\right]. E [ χ ( Z n + 1 , S ) g ( S )] = E P ⊗ ( n + 1 ) [ w ˉ w ( Z n + 1 ) χ ( Z n + 1 , S ) g ( S ) ] = n + 1 1 k = 1 ∑ n + 1 E P ⊗ ( n + 1 ) [ w ˉ w ( Z k ) χ ( Z k , S ) g ( S ) ] .
by exchangeability under P ⊗ ( n + 1 ) P^{\otimes(n+1)} P ⊗ ( n + 1 ) . Write Σ w : = ∑ l w ( Z l ) \Sigma w := \sum_l w(Z_l) Σ w := ∑ l w ( Z l ) and χ ˉ ( S ) : = ∑ k ( w ( Z k ) / Σ w ) χ ( Z k , S ) \bar{\chi}(S) := \sum_k (w(Z_k)/\Sigma w)\chi(Z_k,S) χ ˉ ( S ) := ∑ k ( w ( Z k ) /Σ w ) χ ( Z k , S ) , which is symmetric. The last display equals
E P ⊗ [ Σ w ( n + 1 ) w ˉ χ ˉ ( S ) g ( S ) ] . \mathbb{E}_{P^{\otimes}}\left[\frac{\Sigma w}{(n+1)\bar{w}}\bar{\chi}(S)g(S)\right]. E P ⊗ [ ( n + 1 ) w ˉ Σ w χ ˉ ( S ) g ( S ) ] .
Applying the same two equalities to the symmetric function χ ˉ ( S ) g ( S ) \bar{\chi}(S)g(S) χ ˉ ( S ) g ( S ) in place of χ ⋅ g \chi\cdot g χ ⋅ g gives
E [ χ ˉ ( S ) g ( S ) ] = E P ⊗ [ Σ w ( n + 1 ) w ˉ χ ˉ ( S ) g ( S ) ] . \mathbb{E}[\bar{\chi}(S)g(S)] = \mathbb{E}_{P^{\otimes}}\left[\frac{\Sigma w}{(n+1)\bar{w}}\bar{\chi}(S)g(S)\right]. E [ χ ˉ ( S ) g ( S )] = E P ⊗ [ ( n + 1 ) w ˉ Σ w χ ˉ ( S ) g ( S ) ] .
Hence E [ χ ( Z n + 1 , S ) g ( S ) ] = E [ χ ˉ ( S ) g ( S ) ] \mathbb{E}[\chi(Z_{n+1},S)g(S)] = \mathbb{E}[\bar{\chi}(S)g(S)] E [ χ ( Z n + 1 , S ) g ( S )] = E [ χ ˉ ( S ) g ( S )] for all bounded symmetric g g g , which means E [ χ ( Z n + 1 , S ) ∣ S ] = χ ˉ ( S ) \mathbb{E}[\chi(Z_{n+1},S)\mid S] = \bar{\chi}(S) E [ χ ( Z n + 1 , S ) ∣ S ] = χ ˉ ( S ) .
Second part. By (A2) the law of ( Z 1 , … , Z n + 1 , H , U ) (Z_1,\ldots,Z_{n+1},H,U) ( Z 1 , … , Z n + 1 , H , U ) is a product: the law of the Z Z Z ’s, times P H P_H P H , times Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) . The test functions for conditioning on ( S , H ) (S,H) ( S , H ) are bounded g ( S , H ) g(S,H) g ( S , H ) with g ( ⋅ , h ) g(\cdot,h) g ( ⋅ , h ) symmetric for each h h h .
For fixed ( h , u ) (h,u) ( h , u ) , apply the first part to χ h , u ( z , S ) : = f ( z , S , h , u ) \chi_{h,u}(z,S) := f(z,S,h,u) χ h , u ( z , S ) := f ( z , S , h , u ) and the symmetric function g ( ⋅ , h ) g(\cdot,h) g ( ⋅ , h ) .
Then integrate over the law of ( H , U ) (H,U) ( H , U ) . Fubini applies because f f f and g g g are bounded and the law is a product.
Remark 68 (Relation to weighted exchangeability). Lemma 67 is the covariate-shift case of the weighted exchangeability of Tibshirani et al. [20 ] : their Lemma 2 (§3.2) states that independent draws Z i ∼ P i Z_i\sim P_i Z i ∼ P i are weighted exchangeable with weight functions w 1 ≡ 1 w_1\equiv1 w 1 ≡ 1 and w i = d P i / d P 1 w_i=dP_i/dP_1 w i = d P i / d P 1 . Their proof of Lemma 3 (§3.3) conditions on the event that the multiset equals { z 1 , … , z n + 1 } \{z_1,\ldots,z_{n+1}\} { z 1 , … , z n + 1 } and assumes, for simplicity, that the scores are almost surely distinct, “but the result holds in the general case as well”. Their §3.5 reduces the weights to w ( x i ) / ∑ j w ( x j ) w(x_i)/\sum_j w(x_j) w ( x i ) / ∑ j w ( x j ) , which is our π k \pi_k π k . Lemma 67 writes the tie and duplicate case out in full. We need that case for heavy covariate cells with point-mass scores.
Example 69 (Lemma 67 with a duplicated atom). Take n = 2 n=2 n = 2 , P P P uniform on { x , y } \{x,y\} { x , y } , w ( x ) = 1 w(x)=1 w ( x ) = 1 and w ( y ) = 3 w(y)=3 w ( y ) = 3 , so Q ( x ) = 1 / 4 Q(x)=1/4 Q ( x ) = 1/4 and Q ( y ) = 3 / 4 Q(y)=3/4 Q ( y ) = 3/4 . Condition on the multiset { x , y , y } \{x,y,y\} { x , y , y } . It arises from ( Z 1 , Z 2 , Z 3 ) = ( x , y , y ) (Z_1,Z_2,Z_3)=(x,y,y) ( Z 1 , Z 2 , Z 3 ) = ( x , y , y ) with probability 1 2 ⋅ 3 4 ⋅ 3 4 = 3 / 16 \frac{1}{2}\cdot\frac{3}{4}\cdot\frac{3}{4}=3/16 2 1 ⋅ 4 3 ⋅ 4 3 = 3/16 (test point y y y ), from ( y , x , y ) (y,x,y) ( y , x , y ) with probability $3/16( t e s t p o i n t (test point ( t es tp o in t y), and from ( y , y , x ) (y,y,x) ( y , y , x ) with probability 1 2 ⋅ 1 4 ⋅ 1 4 = 1 / 16 \frac{1}{2}\cdot\frac{1}{4}\cdot\frac{1}{4}=1/16 2 1 ⋅ 4 1 ⋅ 4 1 = 1/16 (test point x x x ). So P ( test = x ∣ multiset ) = 1 / 7 P(\text{test}=x\mid\text{multiset})=1/7 P ( test = x ∣ multiset ) = 1/7 . The lemma gives π x = 1 / ( 1 + 3 + 3 ) = 1 / 7 \pi_x=1/(1+3+3)=1/7 π x = 1/ ( 1 + 3 + 3 ) = 1/7 , and the two copies of y y y get 3 / 7 3/7 3/7 each, 6 / 7 6/7 6/7 in total. Under P ⊗ ( n + 1 ) P^{\otimes(n+1)} P ⊗ ( n + 1 ) all points are exchangeable, and the true law reweights each configuration by the weight of whichever point is the test point; copies of an atom are interchangeable, so a sum over an enumeration with repeats is all that is needed.
Proof of the envelope theorem
Proof of Theorem 62. Step 1 (leave-one-out). Let R j : = R 0 ( p − j o b s ) R_j:=R^0(p^{\mathrm{obs}}_{-j}) R j := R 0 ( p − j obs ) , the BH count when p j p_j p j is replaced by 0. By Lemma 66 , j ∈ R j\in\mathcal{R} j ∈ R iff p j ≤ q R j / m p_j\le qR_j/m p j ≤ q R j / m , and on that event ∣ R ∣ = R j |\mathcal{R}|=R_j ∣ R ∣ = R j . So
FDR = ∑ j = 1 m E [ 1 { Y n + j ≤ c n + j } 1 { p j ≤ q R j / m } R j ] . \operatorname{FDR}=\sum_{j=1}^{m}\mathbb{E}\left[\frac{\mathbf{1}\{Y_{n+j}\le c_{n+j}\}\mathbf{1}\{p_j\le qR_j/m\}}{R_j}\right]. FDR = j = 1 ∑ m E [ R j 1 { Y n + j ≤ c n + j } 1 { p j ≤ q R j / m } ] .
Step 2 (oracle p p p -value). Let p j ∗ p_j^* p j ∗ be eq. (4) with V ^ n + j \hat{V}_{n+j} V ^ n + j replaced by V n + j V_{n+j} V n + j (same U j U_j U j ). On { Y n + j ≤ c n + j } \{Y_{n+j}\le c_{n+j}\} { Y n + j ≤ c n + j } , (A3) gives V n + j ≤ V ^ n + j V_{n+j}\le\hat{V}_{n+j} V n + j ≤ V ^ n + j .
If V n + j = V ^ n + j V_{n+j}=\hat{V}_{n+j} V n + j = V ^ n + j , then p j = p j ∗ p_j=p_j^* p j = p j ∗ .
If V n + j < V ^ n + j V_{n+j}<\hat{V}_{n+j} V n + j < V ^ n + j , write L ∗ L^* L ∗ and T ∗ T^* T ∗ for the calibration weight strictly below and tied with V n + j V_{n+j} V n + j , and L ^ \hat{L} L ^ and T ^ \hat{T} T ^ for the same at V ^ n + j \hat{V}_{n+j} V ^ n + j . Let W : = ∑ i ≤ n w i W:=\sum_{i\le n}w_i W := ∑ i ≤ n w i . Then L ^ ≥ L ∗ + T ∗ \hat{L}\ge L^*+T^* L ^ ≥ L ∗ + T ∗ , and
( W + w n + j ) ( p j − p j ∗ ) ≥ ( L ∗ + T ∗ ) + ( w n + j + T ^ ) U j − L ∗ − ( w n + j + T ∗ ) U j = T ∗ ( 1 − U j ) + T ^ U j ≥ 0. (13) (W+w_{n+j})(p_j-p_j^*)\ge(L^*+T^*)+(w_{n+j}+\hat{T})U_j-L^*-(w_{n+j}+T^*)U_j=T^*(1-U_j)+\hat{T}U_j\ge0. \tag*{(13)} ( W + w n + j ) ( p j − p j ∗ ) ≥ ( L ∗ + T ∗ ) + ( w n + j + T ^ ) U j − L ∗ − ( w n + j + T ∗ ) U j = T ∗ ( 1 − U j ) + T ^ U j ≥ 0. (13)
R j R_j R j does not involve p j p_j p j . The null indicator is used here only pointwise and is then dropped, so
FDR ≤ ∑ j = 1 m E [ 1 { p j ∗ ≤ q R j / m } R j ] . (14) \operatorname{FDR} \le\sum_{j=1}^{m} \mathbb{E}\left[\frac{\mathbf{1}\{p_j^* \le qR_j/m\}}{R_j}\right]. \tag*{(14)} FDR ≤ j = 1 ∑ m E [ R j 1 { p j ∗ ≤ q R j / m } ] . (14)
Step 3 (conditional law of the test point). Fix j j j and abbreviate Z n + 1 : = Z n + j Z_{n+1}:=Z_{n+j} Z n + 1 := Z n + j , S : = S j S:=S_j S := S j , H : = H j H:=H_j H := H j , as in Lemma 67. For ℓ ≠ j \ell\ne j ℓ = j , p ℓ p_\ell p ℓ depends on the calibration multiset only symmetrically, and on ( X n + ℓ , c n + ℓ , U ℓ ) (X_{n+\ell},c_{n+\ell},U_\ell) ( X n + ℓ , c n + ℓ , U ℓ ) . The calibration multiset of the observed p-values is S ∖ { Z n + 1 } S\setminus\{Z_{n+1}\} S ∖ { Z n + 1 } . So R j = ϱ ( Z n + 1 , S , H ) R_j=\varrho(Z_{n+1},S,H) R j = ϱ ( Z n + 1 , S , H ) , where ϱ ( z k , S , H ) : = R 0 ( p − j \varrho(z_k,S,H):=R^0(p_{-j} ϱ ( z k , S , H ) := R 0 ( p − j computed from S ∖ { z k } S\setminus\{z_k\} S ∖ { z k } and H ) H) H ) ; this is the ϱ j ( e ) \varrho_j(e) ϱ j ( e ) of Section E.2 with e = z k e=z_k e = z k . Similarly, p j ∗ = Π < ( Z n + 1 ) + Π = ( Z n + 1 ) U j p_j^*=\Pi_{<}(Z_{n+1})+\Pi_{=}(Z_{n+1})U_j p j ∗ = Π < ( Z n + 1 ) + Π = ( Z n + 1 ) U j , where for z ∈ S z\in S z ∈ S , Π < ( z ) \Pi_{<}(z) Π < ( z ) and Π = ( z ) \Pi_{=}(z) Π = ( z ) are the π \pi π -masses of the elements of S S S with score strictly below and equal to V ( z ) V(z) V ( z ) ; the tied mass includes z z z itself.
Apply Lemma 67 to f = 1 { p j ∗ ≤ q R j / m } / R j f=\mathbf{1}\{p_j^*\le qR_j/m\}/R_j f = 1 { p j ∗ ≤ q R j / m } / R j . Let the tie groups be G 1 ≺ G 2 ≺ ⋯ G_1\prec G_2\prec\cdots G 1 ≺ G 2 ≺ ⋯ , with masses Π G > 0 \Pi_G>0 Π G > 0 (positive because w ≥ 1 w\ge1 w ≥ 1 ). They define consecutive half-open intervals I G = [ Π < G , Π < G + Π G ) I_G=[\Pi_{<G},\Pi_{<G}+\Pi_G) I G = [ Π < G , Π < G + Π G ) , which partition [ 0 , 1 ) [0,1) [ 0 , 1 ) . For u ∈ [ 0 , 1 ) u\in[0,1) u ∈ [ 0 , 1 ) write G ( u ) G(u) G ( u ) for the group with u ∈ I G u\in I_G u ∈ I G . Substituting u ′ = Π < ( z k ) + Π = ( z k ) u u'=\Pi_{<}(z_k)+\Pi_{=}(z_k)u u ′ = Π < ( z k ) + Π = ( z k ) u group by group gives
E [ 1 { p j ∗ ≤ q R j / m } R j ∣ S , H ] = ∫ 0 1 ∑ z k ∈ G ( u ) π k Π G ( u ) 1 { u ≤ q ϱ k / m } ϱ k d u , (15) \mathbb{E}\left[\left.\frac{\mathbf{1}\{p_j^*\le qR_j/m\}}{R_j}\right|S,H\right]
=\int_0^1\sum_{z_k\in G(u)}\frac{\pi_k}{\Pi_{G(u)}}\frac{\mathbf{1}\{u\le q\varrho_k/m\}}{\varrho_k}\,du, \tag*{(15)} E [ R j 1 { p j ∗ ≤ q R j / m } S , H ] = ∫ 0 1 z k ∈ G ( u ) ∑ Π G ( u ) π k ϱ k 1 { u ≤ q ϱ k / m } d u , (15)
with ϱ k : = ϱ ( z k , S , H ) \varrho_k:=\varrho(z_k,S,H) ϱ k := ϱ ( z k , S , H ) . The point u = 1 u=1 u = 1 has measure zero.
Step 4 (envelopes). Set ϱ ‾ ( u ) : = max { ϱ k : z k ⪰ G ( u ) } \overline{\varrho}(u):=\max\{\varrho_k:z_k\succeq G(u)\} ϱ ( u ) := max { ϱ k : z k ⪰ G ( u )} and ϱ ‾ ( u ) : = min { ϱ k : z k ⪯ G ( u ) } \underline{\varrho}(u):=\min\{\varrho_k:z_k\preceq G(u)\} ϱ ( u ) := min { ϱ k : z k ⪯ G ( u )} .
Both are non-increasing step functions on [ 0 , 1 ) [0,1) [ 0 , 1 ) , constant on each I G I_G I G : as u u u grows, ϱ ‾ \overline{\varrho} ϱ is a maximum over a shrinking set and ϱ ‾ \underline{\varrho} ϱ is a minimum over a growing set.
Every z k ∈ G ( u ) z_k\in G(u) z k ∈ G ( u ) satisfies both z k ⪰ G ( u ) z_k\succeq G(u) z k ⪰ G ( u ) and z k ⪯ G ( u ) z_k\preceq G(u) z k ⪯ G ( u ) , so ϱ ‾ ( u ) ≤ ϱ k ≤ ϱ ‾ ( u ) \underline{\varrho}(u)\le\varrho_k\le\overline{\varrho}(u) ϱ ( u ) ≤ ϱ k ≤ ϱ ( u ) .
The weights π k / Π G ( u ) \pi_k/\Pi_{G(u)} π k / Π G ( u ) over z k ∈ G ( u ) z_k\in G(u) z k ∈ G ( u ) sum to 1, so the integrand in (15) is at most 1 { u ≤ q ϱ ‾ ( u ) / m } / ϱ ‾ ( u ) \mathbf{1}\{u\le q\overline{\varrho}(u)/m\}/\underline{\varrho}(u) 1 { u ≤ q ϱ ( u ) / m } / ϱ ( u ) .
Step 5 (monotone integral).
A : = { u ∈ [ 0 , 1 ) : u ≤ q ϱ ‾ ( u ) / m } \mathcal{A}:=\{u\in[0,1):u\le q\overline{\varrho}(u)/m\} A := { u ∈ [ 0 , 1 ) : u ≤ q ϱ ( u ) / m } is downward-closed: if u ∈ A u\in\mathcal{A} u ∈ A and u ′ < u u'<u u ′ < u , then u ′ < u ≤ q ϱ ‾ ( u ) / m ≤ q ϱ ‾ ( u ′ ) / m u'<u\le q\overline{\varrho}(u)/m\le q\overline{\varrho}(u')/m u ′ < u ≤ q ϱ ( u ) / m ≤ q ϱ ( u ′ ) / m . So A \mathcal{A} A is an interval with left end 0. Let u ∗ : = sup A u^*:=\sup\mathcal{A} u ∗ := sup A .
ϱ ‾ ≥ 1 \underline{\varrho}\ge1 ϱ ≥ 1 , so [ 0 , min ( q / m , 1 ) ) ⊂ A [0,\min(q/m,1))\subset\mathcal{A} [ 0 , min ( q / m , 1 )) ⊂ A and u ∗ ≥ min ( q / m , 1 ) > 0 u^*\ge\min(q/m,1)>0 u ∗ ≥ min ( q / m , 1 ) > 0 .
Since the I G I_G I G are half-open and finitely many, there is ϵ > 0 \epsilon>0 ϵ > 0 with ( u ∗ − ϵ , u ∗ ) (u^*-\epsilon,u^*) ( u ∗ − ϵ , u ∗ ) contained in a single interval I G ∗ I_{G^*} I G ∗ . G ∗ G^* G ∗ is the group just left of u ∗ u^* u ∗ : if u ∗ u^* u ∗ is interior to some I G I_G I G , then G ∗ = G G^*=G G ∗ = G ; if u ∗ u^* u ∗ is a boundary point, G ∗ G^* G ∗ is the group to its left; if u ∗ = 1 u^*=1 u ∗ = 1 , G ∗ G^* G ∗ is the top group. G ∗ G^* G ∗ is not in general G ( u ∗ ) G(u^*) G ( u ∗ ) . (Using G ( u ∗ ) G(u^*) G ( u ∗ ) would break the argument: in an example with n = 2 n=2 n = 2 , m = 2 m=2 m = 2 and u ∗ u^* u ∗ on a group boundary, the right-hand group’s ϱ ‾ \overline{\varrho} ϱ violates u ∗ ≤ q ϱ ‾ / m u^*\le q\overline{\varrho}/m u ∗ ≤ q ϱ / m .)
On ( u ∗ − ϵ , u ∗ ) (u^*-\epsilon,u^*) ( u ∗ − ϵ , u ∗ ) , ϱ ‾ = ϱ ‾ G ∗ : = max { ϱ k : z k ⪰ G ∗ } \overline{\varrho}=\overline{\varrho}_{G^*}:=\max\{\varrho_k:z_k\succeq G^*\} ϱ = ϱ G ∗ := max { ϱ k : z k ⪰ G ∗ } and ϱ ‾ = ϱ ‾ G ∗ : = min { ϱ k : z k ⪯ G ∗ } \underline{\varrho}=\underline{\varrho}_{G^*}:=\min\{\varrho_k:z_k\preceq G^*\} ϱ = ϱ G ∗ := min { ϱ k : z k ⪯ G ∗ } .
Letting u ↑ u ∗ u\uparrow u^* u ↑ u ∗ inside A \mathcal{A} A in u ≤ q ϱ ‾ ( u ) / m u\le q\overline{\varrho}(u)/m u ≤ q ϱ ( u ) / m gives u ∗ ≤ q ϱ ‾ G ∗ / m u^*\le q\overline{\varrho}_{G^*}/m u ∗ ≤ q ϱ G ∗ / m .
ϱ ‾ \underline{\varrho} ϱ is non-increasing, so ϱ ‾ ( u ) ≥ ϱ ‾ G ∗ \underline{\varrho}(u)\ge\underline{\varrho}_{G^*} ϱ ( u ) ≥ ϱ G ∗ for all u < u ∗ u<u^* u < u ∗ . Hence
∫ A d u ϱ ‾ ( u ) ≤ u ∗ ϱ ‾ G ∗ ≤ q m ϱ ‾ G ∗ ϱ ‾ G ∗ ≤ q m Γ j ∗ . \int_{\mathcal{A}}\frac{du}{\underline{\varrho}(u)}
\le\frac{u^*}{\underline{\varrho}_{G^*}}
\le\frac{q}{m}\frac{\overline{\varrho}_{G^*}}{\underline{\varrho}_{G^*}}
\le\frac{q}{m}\Gamma_j^*. ∫ A ϱ ( u ) d u ≤ ϱ G ∗ u ∗ ≤ m q ϱ G ∗ ϱ G ∗ ≤ m q Γ j ∗ .
Γ j ∗ \Gamma_j^* Γ j ∗ is a symmetric measurable function of ( S , H ) (S,H) ( S , H ) . Taking expectations in (15) and summing over j j j in (14) gives FDR ≤ q m ∑ j E [ Γ j ∗ ] \operatorname{FDR}\le\frac{q}{m}\sum_j\mathbb{E}[\Gamma_j^*] FDR ≤ m q ∑ j E [ Γ j ∗ ] .
Step 6 (monotonicity defect). Fix ℓ ≠ j \ell\ne j ℓ = j . Let N ℓ N_\ell N ℓ and W ℓ W_\ell W ℓ be the numerator and denominator of p ℓ [ ⋅ ] p_\ell[\cdot] p ℓ [ ⋅ ] when the calibration multiset is all of S j S_j S j , and let p ˉ ℓ : = N ℓ / W ℓ \bar p_\ell:=N_\ell/W_\ell p ˉ ℓ := N ℓ / W ℓ . Removing e e e subtracts w e θ e w_e\theta_e w e θ e from N ℓ N_\ell N ℓ and w e w_e w e from W ℓ W_\ell W ℓ , where θ e = 1 \theta_e=1 θ e = 1 , U ℓ U_\ell U ℓ or 0 0 0 according as V e < V ~ n + ℓ V_e<\widetilde{V}_{n+\ell} V e < V n + ℓ , V e = V ~ n + ℓ V_e=\widetilde{V}_{n+\ell} V e = V n + ℓ or V e > V ~ n + ℓ V_e>\widetilde{V}_{n+\ell} V e > V n + ℓ . Hence
p ℓ [ S j ∖ { e } ] = N ℓ − w e θ e W ℓ − w e = p ˉ ℓ + h ( w e , θ e ) , h ( w , θ ) : = w ( p ˉ ℓ − θ ) W ℓ − w . p_\ell[S_j\setminus\{e\}]=\frac{N_\ell-w_e\theta_e}{W_\ell-w_e}=\bar p_\ell+h(w_e,\theta_e),
\qquad
h(w,\theta):=\frac{w(\bar p_\ell-\theta)}{W_\ell-w}. p ℓ [ S j ∖ { e }] = W ℓ − w e N ℓ − w e θ e = p ˉ ℓ + h ( w e , θ e ) , h ( w , θ ) := W ℓ − w w ( p ˉ ℓ − θ ) .
θ e \theta_e θ e is non-increasing in V e V_e V e (since 0 ≤ U ℓ ≤ 1 0 \le U_\ell\le1 0 ≤ U ℓ ≤ 1 ), and h h h is decreasing in θ \theta θ . So for e ⪯ e ′ e \preceq e' e ⪯ e ′ we have θ e ′ ≤ θ e \theta_{e'} \le\theta_e θ e ′ ≤ θ e and, writing p ℓ ( e ) p_\ell(e) p ℓ ( e ) for p ℓ [ S j ∖ { e } ] p_\ell[S_j \setminus\{e\}] p ℓ [ S j ∖ { e }] ,
p ℓ ( e ′ ) − p ℓ ( e ) = h ( w e ′ , θ e ′ ) − h ( w e , θ e ) ≥ h ( w e ′ , θ e ) − h ( w e , θ e ) = ( p ˉ ℓ − θ e ) ( ϕ ℓ ( w e ′ ) − ϕ ℓ ( w e ) ) . p_\ell(e') - p_\ell(e) = h(w_{e'},\theta_{e'}) - h(w_e,\theta_e) \ge h(w_{e'},\theta_e) - h(w_e,\theta_e) = (\bar p_\ell- \theta_e)(\phi_\ell(w_{e'}) - \phi_\ell(w_e)). p ℓ ( e ′ ) − p ℓ ( e ) = h ( w e ′ , θ e ′ ) − h ( w e , θ e ) ≥ h ( w e ′ , θ e ) − h ( w e , θ e ) = ( p ˉ ℓ − θ e ) ( ϕ ℓ ( w e ′ ) − ϕ ℓ ( w e )) .
∣ p ˉ ℓ − θ e ∣ ≤ 1 |\bar p_\ell-\theta_e| \le1 ∣ p ˉ ℓ − θ e ∣ ≤ 1 and ϕ ℓ \phi_\ell ϕ ℓ is increasing on [ w min , w max ] [w_{\min},w_{\max}] [ w m i n , w m a x ] , so p ℓ ( e ′ ) ≥ p ℓ ( e ) − δ j p_\ell(e') \ge p_\ell(e)-\delta_j p ℓ ( e ′ ) ≥ p ℓ ( e ) − δ j . R 0 R^0 R 0 is non-increasing in each coordinate, so for any tie group G G G and any e 0 ∈ G e_0 \in G e 0 ∈ G :
every e ⪰ G e \succeq G e ⪰ G has e ⪰ e 0 e \succeq e_0 e ⪰ e 0 , so ϱ j ( e ) ≤ R 0 ( p − j ( e 0 ) − δ j ) \varrho_j(e) \le R^0(p_{-j}(e_0)-\delta_j) ϱ j ( e ) ≤ R 0 ( p − j ( e 0 ) − δ j ) ;
every e ⪯ G e \preceq G e ⪯ G has e ⪯ e 0 e \preceq e_0 e ⪯ e 0 , so p − j ( e ) ≤ p − j ( e 0 ) + δ j p_{-j}(e) \le p_{-j}(e_0)+\delta_j p − j ( e ) ≤ p − j ( e 0 ) + δ j and ϱ j ( e ) ≥ R 0 ( p − j ( e 0 ) + δ j ) \varrho_j(e) \ge R^0(p_{-j}(e_0)+\delta_j) ϱ j ( e ) ≥ R 0 ( p − j ( e 0 ) + δ j ) .
Apply this to the maximising group in the definition of Γ j ∗ \Gamma_j^* Γ j ∗ . Hence Γ j ∗ ≤ Γ j \Gamma_j^* \le\Gamma_j Γ j ∗ ≤ Γ j .
Step 7 (deterministic δ \delta δ ). Write a : = w max a := w_{\max} a := w m a x , b : = w min b := w_{\min} b := w m i n and Δ ( b , W ℓ ) : = a / ( W ℓ − a ) − b / ( W ℓ − b ) \Delta(b,W_\ell) := a/(W_\ell-a)-b/(W_\ell-b) Δ ( b , W ℓ ) := a / ( W ℓ − a ) − b / ( W ℓ − b ) , so δ j = max ℓ ≠ j Δ ( b , W ℓ ) \delta_j = \max_{\ell\ne j}\Delta(b,W_\ell) δ j = max ℓ = j Δ ( b , W ℓ ) .
W ℓ ≥ a + n b + 1 W_\ell\ge a+nb+1 W ℓ ≥ a + nb + 1 : one element of S j S_j S j has weight a a a , the other n n n have weight ≥ b \ge b ≥ b , and w n + ℓ ≥ 1 w_{n+\ell} \ge1 w n + ℓ ≥ 1 .
∂ Δ / ∂ W ℓ = − a / ( W ℓ − a ) 2 + b / ( W ℓ − b ) 2 ≤ 0 \partial\Delta/\partial W_\ell= -a/(W_\ell-a)^2+b/(W_\ell-b)^2 \le0 ∂ Δ/ ∂ W ℓ = − a / ( W ℓ − a ) 2 + b / ( W ℓ − b ) 2 ≤ 0 , because a ≥ b a \ge b a ≥ b and W ℓ − a ≤ W ℓ − b W_\ell-a \le W_\ell-b W ℓ − a ≤ W ℓ − b . So Δ \Delta Δ is largest at W ℓ = a + n b + 1 W_\ell=a+nb+1 W ℓ = a + nb + 1 , where it equals Δ 0 ( b ) : = a / ( n b + 1 ) − b / ( a + ( n − 1 ) b + 1 ) \Delta_0(b):=a/(nb+1)-b/(a+(n-1)b+1) Δ 0 ( b ) := a / ( nb + 1 ) − b / ( a + ( n − 1 ) b + 1 ) .
Δ 0 ′ ( b ) = − a n / ( n b + 1 ) 2 − ( a + 1 ) / ( a + ( n − 1 ) b + 1 ) 2 < 0 \Delta_0'(b)=-an/(nb+1)^2-(a+1)/(a+(n-1)b+1)^2<0 Δ 0 ′ ( b ) = − an / ( nb + 1 ) 2 − ( a + 1 ) / ( a + ( n − 1 ) b + 1 ) 2 < 0 , so Δ 0 \Delta_0 Δ 0 is largest at b = 1 b=1 b = 1 : Δ 0 ( 1 ) = ( a − 1 ) ( a + n + 1 ) / ( ( n + 1 ) ( a + n ) ) \Delta_0(1)=(a-1)(a+n+1)/((n+1)(a+n)) Δ 0 ( 1 ) = ( a − 1 ) ( a + n + 1 ) / (( n + 1 ) ( a + n )) .
Δ 0 ( 1 ) = ( a − 1 ) ( 1 + 1 / ( a + n ) ) / ( n + 1 ) \Delta_0(1)=(a-1)(1+1/(a+n))/(n+1) Δ 0 ( 1 ) = ( a − 1 ) ( 1 + 1/ ( a + n )) / ( n + 1 ) . It is increasing in a a a , because d d a [ ( a − 1 ) ( a + n + 1 ) / ( a + n ) ] = 1 + ( n + 1 ) / ( a + n ) 2 > 0 \frac{d}{da}[(a-1)(a+n+1)/(a+n)]=1+(n+1)/(a+n)^2>0 d a d [( a − 1 ) ( a + n + 1 ) / ( a + n )] = 1 + ( n + 1 ) / ( a + n ) 2 > 0 . So a ≤ B a\le B a ≤ B gives δ j ≤ δ n ( B ) \delta_j\le\delta_n(B) δ j ≤ δ n ( B ) .
Finally, if w w w is constant then w max = w min w_{\max}=w_{\min} w m a x = w m i n , so δ j = 0 \delta_j=0 δ j = 0 , every ratio in Γ j \Gamma_j Γ j equals 1 1 1 , and FDR ≤ q \le q ≤ q .
Proof of Corollary 63. (Step 8.) The observed p-values of the other units use the calibration multiset S j ∖ { Z n + j } S_j\setminus\{Z_{n+j}\} S j ∖ { Z n + j } , so p − j o b s = p − j ( Z n + j ) p_{-j}^{\mathrm{obs}}=p_{-j}(Z_{n+j}) p − j obs = p − j ( Z n + j ) . From Step 6, p ℓ ( e ) = p ˉ ℓ + w e ( p ˉ ℓ − θ e ) / ( W ℓ − w e ) p_\ell(e)=\bar p_\ell+w_e(\bar p_\ell-\theta_e)/(W_\ell-w_e) p ℓ ( e ) = p ˉ ℓ + w e ( p ˉ ℓ − θ e ) / ( W ℓ − w e ) , with p ˉ ℓ − θ e ∈ [ p ˉ ℓ − 1 , p ˉ ℓ ] \bar p_\ell-\theta_e\in[\bar p_\ell-1,\bar p_\ell] p ˉ ℓ − θ e ∈ [ p ˉ ℓ − 1 , p ˉ ℓ ] . So
p ˉ ℓ − ( 1 − p ˉ ℓ ) ϕ ℓ ( w max ) ≤ p ℓ ( e ) ≤ p ˉ ℓ + p ˉ ℓ ϕ ℓ ( w max ) \bar p_\ell-(1-\bar p_\ell)\phi_\ell(w_{\max}) \le p_\ell(e) \le\bar p_\ell+\bar p_\ell\phi_\ell(w_{\max}) p ˉ ℓ − ( 1 − p ˉ ℓ ) ϕ ℓ ( w m a x ) ≤ p ℓ ( e ) ≤ p ˉ ℓ + p ˉ ℓ ϕ ℓ ( w m a x )
for every e ∈ S j e\in S_j e ∈ S j .
Hence ∣ p ℓ ( e ) − p ℓ o b s ∣ ≤ ϕ ℓ ( w max ) ≤ τ ˉ j |p_\ell(e)-p_\ell^{\mathrm{obs}}|\le\phi_\ell(w_{\max})\le\bar\tau_j ∣ p ℓ ( e ) − p ℓ obs ∣ ≤ ϕ ℓ ( w m a x ) ≤ τ ˉ j for all e ∈ S j e\in S_j e ∈ S j and ℓ ≠ j \ell\ne j ℓ = j . By monotonicity of R 0 R^0 R 0 , every ϱ j ( e ) \varrho_j(e) ϱ j ( e ) lies in [ R 0 ( p − j o b s + τ ˉ j ) , R 0 ( p − j o b s − τ ˉ j ) ] [R^0(p_{-j}^{\mathrm{obs}}+\bar\tau_j),R^0(p_{-j}^{\mathrm{obs}}-\bar\tau_j)] [ R 0 ( p − j obs + τ ˉ j ) , R 0 ( p − j obs − τ ˉ j )] . So
Γ j ∗ ≤ max e ϱ j ( e ) min e ϱ j ( e ) ≤ Γ ~ j ≤ Γ ^ j , \Gamma_j^* \le\frac{\max_e\varrho_j(e)}{\min_e\varrho_j(e)} \le\widetilde\Gamma_j \le\widehat\Gamma_j, Γ j ∗ ≤ min e ϱ j ( e ) max e ϱ j ( e ) ≤ Γ j ≤ Γ j ,
where the last step uses η j ≥ τ ˉ j \eta_j\ge\bar\tau_j η j ≥ τ ˉ j and the monotonicity of R 0 R^0 R 0 . (Going through Γ j \Gamma_j Γ j instead costs an extra shift δ j \delta_j δ j for nothing; that route gives Γ j ≤ Γ ^ j \Gamma_j\le\widehat\Gamma_j Γ j ≤ Γ j , since p − j ( e 0 ) ∓ δ j p_{-j}(e_0)\mp\delta_j p − j ( e 0 ) ∓ δ j lies within η j \eta_j η j of p − j o b s p_{-j}^{\mathrm{obs}} p − j obs in each coordinate.) The FDR bounds follow from Theorem 62 .
Every quantity in Γ ^ j \widehat\Gamma_j Γ j is observed: the weights of S j S_j S j are w ( X 1 ) , … , w ( X n ) , w ( X n + j ) w(X_1),\ldots,w(X_n),w(X_{n+j}) w ( X 1 ) , … , w ( X n ) , w ( X n + j ) ; W ℓ W_\ell W ℓ also uses w ( X n + ℓ ) w(X_{n+\ell}) w ( X n + ℓ ) ; and p − j o b s p_{-j}^{\mathrm{obs}} p − j obs are the reported p-values. For the shift bounds:
ϕ ℓ ( w max ) = w max / ( W ℓ − w max ) ≤ w max / ( ∑ S j w − w max ) = τ ^ j / ( 1 − τ ^ j ) \phi_\ell(w_{\max})=w_{\max}/(W_\ell-w_{\max})\le w_{\max}/(\sum_{S_j}w-w_{\max})=\widehat\tau_j/(1-\widehat\tau_j) ϕ ℓ ( w m a x ) = w m a x / ( W ℓ − w m a x ) ≤ w m a x / ( ∑ S j w − w m a x ) = τ j / ( 1 − τ j ) , which gives (i);
W ℓ − w max ≥ n ⋅ 1 + 1 W_\ell-w_{\max}\ge n\cdot1+1 W ℓ − w m a x ≥ n ⋅ 1 + 1 , since w ≥ 1 w\ge1 w ≥ 1 , and w max ≤ B w_{\max}\le B w m a x ≤ B , so τ ˉ j ≤ B / ( n + 1 ) \bar\tau_j\le B/(n+1) τ ˉ j ≤ B / ( n + 1 ) , which is (ii);
δ j ≤ max ℓ ϕ ℓ ( w max ) = τ ˉ j \delta_j\le\max_\ell\phi_\ell(w_{\max})=\bar\tau_j δ j ≤ max ℓ ϕ ℓ ( w m a x ) = τ ˉ j , so η j ≤ 2 τ ˉ j ≤ 2 τ ^ j / ( 1 − τ ^ j ) \eta_j\le2\bar\tau_j\le2\widehat\tau_j/(1-\widehat\tau_j) η j ≤ 2 τ ˉ j ≤ 2 τ j / ( 1 − τ j ) , which is (iii).
Proof of Proposition 64. In S j S_j S j only the score of Z n + j Z_{n+j} Z n + j is unobserved; its covariate, hence its weight, is observed. For e ≠ Z n + j e\ne Z_{n+j} e = Z n + j , p ℓ [ S j ∖ { e } ] p_\ell[S_j\setminus\{e\}] p ℓ [ S j ∖ { e }] depends on v = V n + j v=V_{n+j} v = V n + j only through 1 { v < V n + ℓ } \mathbf{1}\{v<V_{n+\ell}\} 1 { v < V n + ℓ } and 1 { v = V n + ℓ } \mathbf{1}\{v=V_{n+\ell}\} 1 { v = V n + ℓ } (for e = Z n + j e=Z_{n+j} e = Z n + j it does not depend on v v v ), and the order ⪯ \preceq ⪯ on S j S_j S j depends on v v v only through its comparisons with the calibration scores. So Γ j ∗ ( v ) \Gamma_j^*(v) Γ j ∗ ( v ) is constant as v v v ranges over each of the r r r distinct values and each of the r + 1 r+1 r + 1 open gaps between them: at most 2 r + 1 2r+1 2 r + 1 cases. Taking v = V n + j v=V_{n+j} v = V n + j gives Γ j ∗ ≤ Γ j s u p \Gamma_j^*\le\Gamma_j^{\mathrm{sup}} Γ j ∗ ≤ Γ j sup . The bound Γ j ∗ ( v ) ≤ Γ ~ j \Gamma_j^*(v)\le\widetilde\Gamma_j Γ j ∗ ( v ) ≤ Γ j of Step 8 holds for every v v v , and Γ ~ j \widetilde\Gamma_j Γ j does not depend on v v v ; hence Γ j s u p ≤ Γ ~ j \Gamma_j^{\mathrm{sup}}\le\widetilde\Gamma_j Γ j sup ≤ Γ j .
Proof of Theorem 21. Assume (A1)–(A3). The first inequality FDR ≤ q m ∑ j E Γ j ∗ \operatorname{FDR} \le\frac{q}{m}\sum_j \mathbb{E}\Gamma_j^* FDR ≤ m q ∑ j E Γ j ∗ is Theorem 62 . The pointwise bounds Γ j ∗ ≤ Γ j s u p ≤ Γ ~ j \Gamma_j^* \le\Gamma_j^{\mathrm{sup}} \le\widetilde{\Gamma}_j Γ j ∗ ≤ Γ j sup ≤ Γ j are Proposition 64 , with Γ ~ j \widetilde{\Gamma}_j Γ j as in Corollary 63 (there p − j p_{-j} p − j is written p − j o b s p_{-j}^{\mathrm{obs}} p − j obs ). The bound τ ˉ j ≤ B / ( n + 1 ) \bar{\tau}_j \le B/(n+1) τ ˉ j ≤ B / ( n + 1 ) is Corollary 63(ii), and computability of Γ ~ j \widetilde{\Gamma}_j Γ j and Γ j s u p \Gamma_j^{\mathrm{sup}} Γ j sup is stated there and in Proposition 64 .
Example 70 (Size of the shifts). Take n = 4 n=4 n = 4 calibration points with weights ( 1 , 1 , 1 , 2 ) (1,1,1,2) ( 1 , 1 , 1 , 2 ) and a test unit j j j with weight 1, so ∑ S j w = 6 \sum_{S_j} w=6 ∑ S j w = 6 and w max = 2 w_{\max}=2 w m a x = 2 . Another test unit ℓ \ell ℓ with weight 1 has W ℓ = 7 W_\ell=7 W ℓ = 7 , so ϕ ℓ ( 2 ) = 2 / 5 \phi_\ell(2)=2/5 ϕ ℓ ( 2 ) = 2/5 and ϕ ℓ ( 1 ) = 1 / 6 \phi_\ell(1)=1/6 ϕ ℓ ( 1 ) = 1/6 . Then δ j = 2 / 5 − 1 / 6 = 7 / 30 \delta_j=2/5-1/6=7/30 δ j = 2/5 − 1/6 = 7/30 , τ ˉ j = 2 / 5 \bar{\tau}_j=2/5 τ ˉ j = 2/5 and η j = 19 / 30 \eta_j=19/30 η j = 19/30 ; the bound 2 τ ^ j / ( 1 − τ ^ j ) 2\widehat{\tau}_j/(1-\widehat{\tau}_j) 2 τ j / ( 1 − τ j ) with τ ^ j = 1 / 3 \widehat{\tau}_j=1/3 τ j = 1/3 gives 1 ≥ 19 / 30 1\ge19/30 1 ≥ 19/30 . With such a small n n n , η j \eta_j η j is large and Γ ^ j \widehat{\Gamma}_j Γ j is uninformative; Γ ~ j \widetilde{\Gamma}_j Γ j uses the shift τ ˉ j = 2 / 5 \bar{\tau}_j=2/5 τ ˉ j = 2/5 . Heuristically, τ ˉ j \bar{\tau}_j τ ˉ j is about the largest weight share, so Γ ~ j ≈ 1 \widetilde{\Gamma}_j\approx1 Γ j ≈ 1 when m τ ˉ j ≪ q m\bar{\tau}_j\ll q m τ ˉ j ≪ q .
Remarks: what is computable, and outlier detection
Remark 71 (The deterministic shift). δ n ( B ) ≈ ( B − 1 ) / ( n + 1 ) \delta_n(B)\approx(B-1)/(n+1) δ n ( B ) ≈ ( B − 1 ) / ( n + 1 ) when B ≪ n B\ll n B ≪ n . When B ≥ n + 2 B\ge n+2 B ≥ n + 2 the bound is at least about 1; this is the Poisson regime, where the excess is O ( 1 ) O(1) O ( 1 ) .
Remark 72 (What can be computed). Γ j ∗ \Gamma_j^* Γ j ∗ and Γ j s u p \Gamma_j^{\mathrm{sup}} Γ j sup are not computable in the selection setting: S j S_j S j contains Z n + j Z_{n+j} Z n + j , and every ϱ j ( e ) \varrho_j(e) ϱ j ( e ) with e e e a calibration point uses V ( X n + j , Y n + j ) V(X_{n+j},Y_{n+j}) V ( X n + j , Y n + j ) , with Y n + j Y_{n+j} Y n + j unobserved.
In outlier detection, Z n + j Z_{n+j} Z n + j is the observed test point, so Γ j ∗ \Gamma_j^* Γ j ∗ is computable there, provided the randomisations U ℓ U_\ell U ℓ are recorded.
δ j \delta_j δ j , Γ j s u p \Gamma_j^{\mathrm{sup}} Γ j sup , Γ ~ j \widetilde{\Gamma}_j Γ j and Γ ^ j \widehat{\Gamma}_j Γ j use only covariates, reported p-values and recorded U U U 's, so they are computable in both settings.
All of these bound an expectation . The realised value q m ∑ j Γ ~ j \frac{q}{m}\sum_j\widetilde{\Gamma}_j m q ∑ j Γ j is an observable random variable whose mean is at least the FDR. It is not a high-probability bound on the realised FDP.
At small n n n , q m ∑ j E [ Γ ^ j ] \frac{q}{m}\sum_j\mathbb{E}[\widehat{\Gamma}_j] m q ∑ j E [ Γ j ] often exceeds 1, which is vacuous; values 1.27, 1.49 and 2.29 occur in exact examples.
Proposition 73 (Outlier detection). In outlier detection, let H 0 H_0 H 0 be the set of inliers: for j ∈ H 0 j\in H_0 j ∈ H 0 , Z n + j ∼ Q Z_{n+j}\sim Q Z n + j ∼ Q as in (A1), while the outliers need not follow Q Q Q . Put the outliers’ data ( X , score and U ) (X,\text{ score and }U) ( X , score and U ) for ℓ ≠ j \ell\ne j ℓ = j into H j H_j H j , and require (A2) as stated for every j ∈ H 0 j\in H_0 j ∈ H 0 , so that H j H_j H j is independent of the calibration data and Z n + j Z_{n+j} Z n + j . Then
FDR ≤ q m ∑ j ∈ H 0 E [ Γ j ∗ ] ≤ q m ∑ j E [ Γ j ∗ ] , \operatorname{FDR}\le\frac{q}{m}\sum_{j\in H_0}\mathbb{E}[\Gamma_j^*]\le\frac{q}{m}\sum_j\mathbb{E}[\Gamma_j^*], FDR ≤ m q j ∈ H 0 ∑ E [ Γ j ∗ ] ≤ m q j ∑ E [ Γ j ∗ ] ,
and likewise with Γ ~ j \widetilde{\Gamma}_j Γ j and Γ ^ j \widehat{\Gamma}_j Γ j in place of Γ j ∗ \Gamma_j^* Γ j ∗ .
Proof. (A1) puts every test unit under Q Q Q , but outliers do not follow Q Q Q . Restrict the FDR sum of Step 1 to the inliers j ∈ H 0 j\in H_0 j ∈ H 0 . For these p j = p j ∗ p_j=p_j^* p j = p j ∗ , so Step 2 is trivial. With the outliers’ data in H j H_j H j and (A2) as stated, Steps 3–8 go through unchanged.
Remark 74 (Heuristic, not a result). In a global-null configuration with a single heavy calibration unit of weight B B B and n n n light units, τ ˉ j ≈ B / n \bar{\tau}_j\approx B/n τ ˉ j ≈ B / n . Then Γ ~ j − 1 \widetilde{\Gamma}_j-1 Γ j − 1 is driven by how many other p-values fall within B / n B/n B / n of a BH threshold. That suggests order m B / n mB/n m B / n in the linear regime, matching Theorem 16 . Explicit bounds of this kind, under anti-concentration and sub-uniformity conditions, are proved in Sections E.6 and E.7.
Explicit bounds from the envelope
Notation. τ B : = B / ( n + 1 ) \tau_B:=B/(n+1) τ B := B / ( n + 1 ) , which bounds the shift τ ˉ j \bar{\tau}_j τ ˉ j of Γ ~ j \widetilde{\Gamma}_j Γ j (Corollary 63(ii)); I k : = ( t k − τ B , t k + τ B ] I_k:=(t_k-\tau_B,t_k+\tau_B] I k := ( t k − τ B , t k + τ B ] ; x ( j ) : = p − j o b s x^{(j)}:=p_{-j}^{\mathrm{obs}} x ( j ) := p − j obs , the other m − 1 m-1 m − 1 reported p-values, with x ( i ) ( j ) x_{(i)}^{(j)} x ( i ) ( j ) (or x ( i ) x_{(i)} x ( i ) ) the i i i -th smallest.
Lemma 75 (Window lemma). For x ∈ R m − 1 x\in\mathbb{R}^{m-1} x ∈ R m − 1 and s ≥ 0 s\ge0 s ≥ 0 , let r − : = R 0 ( x − s ) r^-:=R^0(x-s) r − := R 0 ( x − s ) and r + : = R 0 ( x + s ) r^+:=R^0(x+s) r + := R 0 ( x + s ) . Then
r − r + − 1 ≤ ( r − − 1 ) 1 { r − > r + } ≤ ∑ k = 2 m ( k − 1 ) 1 { x ( k − 1 ) ∈ ( t k − s , t k + s ] } . \frac{r^-}{r^+}-1\le(r^--1)\mathbf{1}\{r^->r^+\}\le\sum_{k=2}^{m}(k-1)\mathbf{1}\{x_{(k-1)}\in(t_k-s,t_k+s]\}. r + r − − 1 ≤ ( r − − 1 ) 1 { r − > r + } ≤ k = 2 ∑ m ( k − 1 ) 1 { x ( k − 1 ) ∈ ( t k − s , t k + s ]} .
Proof. • If r − = r + r^{-}=r^{+} r − = r + , the left side is 0.
Otherwise r − > r + ≥ 1 r^{-}>r^{+}\ge1 r − > r + ≥ 1 , so r − ≥ 2 r^{-}\ge2 r − ≥ 2 and the ratio minus 1 is at most r − − 1 r^{-}-1 r − − 1 .
Write R 0 ( y ) = max { k : 1 + # { ℓ : y ℓ ≤ t k } ≥ k } R^{0}(y)=\max\{k:1+\#\{\ell:y_{\ell}\le t_{k}\}\ge k\} R 0 ( y ) = max { k : 1 + # { ℓ : y ℓ ≤ t k } ≥ k } .
Since r − r^{-} r − is feasible for x − s x-s x − s , # { ℓ : x ℓ ≤ t r − + s } ≥ r − − 1 \#\{\ell:x_{\ell}\le t_{r^{-}}+s\}\ge r^{-}-1 # { ℓ : x ℓ ≤ t r − + s } ≥ r − − 1 , so x ( r − − 1 ) ≤ t r − + s x_{(r^{-}-1)}\le t_{r^{-}}+s x ( r − − 1 ) ≤ t r − + s .
Since r − > r + r^{-}>r^{+} r − > r + , r − r^{-} r − is not feasible for x + s x+s x + s , so # { ℓ : x ℓ ≤ t r − − s } ≤ r − − 2 \#\{\ell:x_{\ell}\le t_{r^{-}}-s\}\le r^{-}-2 # { ℓ : x ℓ ≤ t r − − s } ≤ r − − 2 , so x ( r − − 1 ) > t r − − s x_{(r^{-}-1)}>t_{r^{-}}-s x ( r − − 1 ) > t r − − s .
So the indicator for k = r − k=r^{-} k = r − equals 1.
Monotonicity in s s s . R 0 ( x − s ) / R 0 ( x + s ) R^{0}(x-s)/R^{0}(x+s) R 0 ( x − s ) / R 0 ( x + s ) is non-decreasing in s s s , since R 0 R^{0} R 0 is non-increasing in each coordinate. Since τ ˉ j ≤ τ B \bar{\tau}_{j}\le\tau_{B} τ ˉ j ≤ τ B , we get
Γ ~ j ≤ Υ j : = R 0 ( x ( j ) − τ B ) R 0 ( x ( j ) + τ B ) . \widetilde{\Gamma}_{j}\le\Upsilon_{j}:=\frac{R^{0}(x^{(j)}-\tau_{B})}{R^{0}(x^{(j)}+\tau_{B})}. Γ j ≤ Υ j := R 0 ( x ( j ) + τ B ) R 0 ( x ( j ) − τ B ) .
Theorem 76 (A bound for every configuration). Under (A1)–(A3), including non-null test units, and in outlier detection (Proposition 73),
FDR ≤ q m ∑ j E [ Γ ~ j ] ≤ q + q m ∑ j = 1 m ∑ k = 2 m ( k − 1 ) P ( x ( k − 1 ) ( j ) ∈ I k ) . \operatorname{FDR}\le\frac{q}{m}\sum_{j}\mathbb{E}[\widetilde{\Gamma}_{j}]\le q+\frac{q}{m}\sum_{j=1}^{m}\sum_{k=2}^{m}(k-1)\mathbb{P}\left(x^{(j)}_{(k-1)}\in I_{k}\right). FDR ≤ m q j ∑ E [ Γ j ] ≤ q + m q j = 1 ∑ m k = 2 ∑ m ( k − 1 ) P ( x ( k − 1 ) ( j ) ∈ I k ) . Proof. Use Corollary 63 (or Proposition 73), then Γ ~ j ≤ Υ j \widetilde{\Gamma}_{j}\le\Upsilon_{j} Γ j ≤ Υ j and Lemma 75 with s = τ B s=\tau_{B} s = τ B , then take expectations.
Corollary 77 (Anti-concentration). Suppose that for every test unit ℓ \ell ℓ and every interval I I I of length 2 τ B 2\tau_{B} 2 τ B inside [ t 2 − τ B , t m + τ B ] [t_{2}-\tau_{B},t_{m}+\tau_{B}] [ t 2 − τ B , t m + τ B ] , P ( p ℓ ∈ I ) ≤ L ∣ I ∣ \mathbb{P}(p_{\ell}\in I)\le L|I| P ( p ℓ ∈ I ) ≤ L ∣ I ∣ . Then, under the assumptions of Theorem 76,
FDR ≤ q [ 1 + L m ( m − 1 ) 2 B n + 1 ] . \operatorname{FDR}\le q\left[1+Lm(m-1)^{2}\frac{B}{n+1}\right]. FDR ≤ q [ 1 + L m ( m − 1 ) 2 n + 1 B ] . Proof. { x ( k − 1 ) ∈ I k } \{x_{(k-1)}\in I_{k}\} { x ( k − 1 ) ∈ I k } implies that some x ℓ ∈ I k x_{\ell}\in I_{k} x ℓ ∈ I k , so P ( x ( k − 1 ) ∈ I k ) ≤ ( m − 1 ) ⋅ 2 L τ B \mathbb{P}(x_{(k-1)}\in I_{k})\le(m-1)\cdot2L\tau_{B} P ( x ( k − 1 ) ∈ I k ) ≤ ( m − 1 ) ⋅ 2 L τ B . Also ∑ k = 2 m ( k − 1 ) = m ( m − 1 ) / 2 \sum_{k=2}^{m}(k-1)=m(m-1)/2 ∑ k = 2 m ( k − 1 ) = m ( m − 1 ) /2 . Insert both into Theorem 76.
When L = 1 L=1 L = 1 holds exactly. An oracle null p-value ( V ^ = V ) (\widehat{V}=V) ( V = V ) is exactly Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) marginally (Lemma 67; this is Lemma 1), so P ( p ℓ ∈ I ) = ∣ I ∣ \mathbb{P}(p_{\ell}\in I)=|I| P ( p ℓ ∈ I ) = ∣ I ∣ . So in every global-null configuration with oracle thresholds (every threshold equal to Y Y Y , or outlier detection with all units inliers),
FDR ≤ q [ 1 + m ( m − 1 ) 2 B n + 1 ] for all n , B . \operatorname{FDR}\le q\left[1+m(m-1)^{2}\frac{B}{n+1}\right]\qquad\text{for all }n,B. FDR ≤ q [ 1 + m ( m − 1 ) 2 n + 1 B ] for all n , B .
This bound is fully explicit, but cubic in m m m , against the linear lower bound of Theorem 16.
Conditional sub-uniformity. Assume, as Jin & Candès [12 ] do, that the test tuples ( X n + ℓ , Y n + ℓ , c n + ℓ ) (X_{n+\ell},Y_{n+\ell},c_{n+\ell}) ( X n + ℓ , Y n + ℓ , c n + ℓ ) and the U ℓ U_{\ell} U ℓ are i.i.d. across units and independent of the calibration data D D D (this implies (A2)). Then, given D D D , the coordinates of x ( j ) x^{(j)} x ( j ) are i.i.d. with some CDF G D G_{D} G D : the conditional law of one unit’s reported p-value, mixing null and non-null types.
Theorem 78 (Linear in m m m under conditional sub-uniformity). Assume (A1), (A3) and the i.i.d. structure just described. Fix ρ ≥ 1 \rho\ge1 ρ ≥ 1 and define
G : = { D : G D ( t k + τ B ) ≤ ρ ( t k + τ B ) for k = 2 , … , m } \mathcal{G}:=\{D:G_{D}(t_{k}+\tau_{B})\le\rho(t_{k}+\tau_{B})\text{ for }k=2,\ldots,m\} G := { D : G D ( t k + τ B ) ≤ ρ ( t k + τ B ) for k = 2 , … , m } ;
L : = max 2 ≤ k ≤ m P ( p ℓ ∈ I k ) / ∣ I k ∣ L:=\max_{2\le k\le m}\mathbb{P}(p_{\ell}\in I_{k})/|I_{k}| L := max 2 ≤ k ≤ m P ( p ℓ ∈ I k ) /∣ I k ∣ , the marginal anti-concentration at the windows;
β m ( ρ ) : = ∑ k = 2 m ( k − 1 ) P ( Bin ( m − 2 , min { 1 , ρ ( t k + τ B ) } ) ≥ k − 2 ) \beta_{m}(\rho):=\sum_{k=2}^{m}(k-1)\mathbb{P}\left(\operatorname{Bin}(m-2,\min\{1,\rho(t_{k}+\tau_{B})\})\ge k-2\right) β m ( ρ ) := ∑ k = 2 m ( k − 1 ) P ( Bin ( m − 2 , min { 1 , ρ ( t k + τ B )}) ≥ k − 2 ) .
Then for every j j j ,
E [ Γ ~ j ] ≤ 1 + 2 L ( m − 1 ) τ B β m ( ρ ) + ( m − 1 ) P ( G c ) , \mathbb{E}[\widetilde{\Gamma}_j] \le1 + 2L(m-1)\tau_B\beta_m(\rho) + (m-1)\mathbb{P}(\mathcal{G}^{c}), E [ Γ j ] ≤ 1 + 2 L ( m − 1 ) τ B β m ( ρ ) + ( m − 1 ) P ( G c ) ,
and
FDR ≤ q [ 1 + 2 L ( m − 1 ) β m ( ρ ) B n + 1 + ( m − 1 ) P ( G c ) ] . \operatorname{FDR} \le q\left[1+2L(m-1)\beta_m(\rho)\frac{B}{n+1}+(m-1)\mathbb{P}(\mathcal{G}^{c})\right]. FDR ≤ q [ 1 + 2 L ( m − 1 ) β m ( ρ ) n + 1 B + ( m − 1 ) P ( G c ) ] .
Proof. By Γ ~ j ≤ Υ j \widetilde{\Gamma}_j \le\Upsilon_j Γ j ≤ Υ j and Lemma 75, Υ j − 1 ≤ r − − 1 ≤ m − 1 \Upsilon_j-1 \le r^{-}-1 \le m-1 Υ j − 1 ≤ r − − 1 ≤ m − 1 always; use this on G c \mathcal{G}^{c} G c . On G \mathcal{G} G :
The event { x ( k − 1 ) ∈ I k } \{x_{(k-1)} \in I_k\} { x ( k − 1 ) ∈ I k } needs some x ℓ ∈ I k x_\ell\in I_k x ℓ ∈ I k and at least k − 2 k-2 k − 2 of the other m − 2 m-2 m − 2 coordinates ≤ t k + τ B \le t_k+\tau_B ≤ t k + τ B .
Given D D D the coordinates are i.i.d. G D G_D G D . By a union bound,
P ( x ( k − 1 ) ∈ I k ∣ D ) ≤ ( m − 1 ) G D ( I k ) P ( Bin ( m − 2 , G D ( t k + τ B ) ) ≥ k − 2 ) . \mathbb{P}(x_{(k-1)} \in I_k \mid D) \le(m-1)G_D(I_k)\mathbb{P}\left(\operatorname{Bin}(m-2,G_D(t_k+\tau_B)) \ge k-2\right). P ( x ( k − 1 ) ∈ I k ∣ D ) ≤ ( m − 1 ) G D ( I k ) P ( Bin ( m − 2 , G D ( t k + τ B )) ≥ k − 2 ) .
On G \mathcal{G} G the binomial factor is at most its value at success probability min { 1 , ρ ( t k + τ B ) } \min\{1,\rho(t_k+\tau_B)\} min { 1 , ρ ( t k + τ B )} , because the binomial tail is increasing in the success probability.
Take E [ ⋅ 1 G ] \mathbb{E}[\cdot\mathbf{1}_{\mathcal{G}}] E [ ⋅ 1 G ] (G \mathcal{G} G is D D D -measurable) and use E [ G D ( I k ) ] = P ( p ℓ ∈ I k ) ≤ 2 L τ B \mathbb{E}[G_D(I_k)]=\mathbb{P}(p_\ell\in I_k)\le2L\tau_B E [ G D ( I k )] = P ( p ℓ ∈ I k ) ≤ 2 L τ B .
Summing over k k k with the weights k − 1 k-1 k − 1 of Lemma 75 gives the bound on E [ Γ ~ j ] \mathbb{E}[\widetilde{\Gamma}_j] E [ Γ j ] ; the FDR bound follows from Corollary 63 with τ B = B / ( n + 1 ) \tau_B=B/(n+1) τ B = B / ( n + 1 ) .
Remark 79 (Size of β m ( ρ ) \beta_m(\rho) β m ( ρ ) ). The values below are a numerical evaluation of the formula for β m ( ρ ) \beta_m(\rho) β m ( ρ ) , computed with τ B = 0 \tau_B=0 τ B = 0 (the limit n / ( m B ) → ∞ n/(mB)\to\infty n / ( m B ) → ∞ ):
Table 79. Values of β m ( ρ ) \beta_m(\rho) β m ( ρ ) for τ B = 0 \tau_B=0 τ B = 0 .
With τ B > 0 \tau_B>0 τ B > 0 , β m ( ρ ) \beta_m(\rho) β m ( ρ ) grows with m τ B m\tau_B m τ B : at q = 0.1 q=0.1 q = 0.1 , ρ = 1 \rho=1 ρ = 1 , B = 3 B=3 B = 3 , n = 1000 n=1000 n = 1000 it is 2.61 (m = 100 m=100 m = 100 ), 15.4 (m = 1000 m=1000 m = 1000 ) and 638 (m = 10 4 m=10^4 m = 1 0 4 ). We have only sketched an m m m -free majorant of β m ( ρ ) \beta_m(\rho) β m ( ρ ) . Since t k + τ B ≤ ( q + m τ B ) k / m t_k+\tau_B\le(q+m\tau_B)k/m t k + τ B ≤ ( q + m τ B ) k / m , whenever ρ ( q + m τ B ) < 1 \rho(q+m\tau_B)<1 ρ ( q + m τ B ) < 1 a Chernoff bound on the binomial tails (Lemma 29) should bound β m ( ρ ) \beta_m(\rho) β m ( ρ ) uniformly in m m m , but we have not written that step out.
Remark 80 (The event G \mathcal{G} G at ρ = 1 \rho=1 ρ = 1 ). E [ G D ( t ) ] = t \mathbb{E}[G_D(t)]=t E [ G D ( t )] = t in the global null, so at ρ = 1 \rho=1 ρ = 1 the event G D ( t k ) > t k G_D(t_k)>t_k G D ( t k ) > t k has probability about 1 2 \frac{1}{2} 2 1 at each grid point. A global-null simulation gives P ( G c ) ≈ 0.78 \mathbb{P}(\mathcal{G}^{c})\approx0.78 P ( G c ) ≈ 0.78 at ρ = 1 \rho=1 ρ = 1 for n = 2000 n=2000 n = 2000 and n = 20000 n=20000 n = 20000 , and the term ( m − 1 ) P ( G c ) (m-1)\mathbb{P}(\mathcal{G}^{c}) ( m − 1 ) P ( G c ) then makes the bound vacuous. The useful regime is ρ = 1 + ε n \rho=1+\varepsilon_n ρ = 1 + ε n with ε n ↓ 0 \varepsilon_n\downarrow0 ε n ↓ 0 slowly; Section E.7 bounds P ( G c ) \mathbb{P}(\mathcal{G}^{c}) P ( G c ) for ρ > 1 + κ \rho>1+\kappa ρ > 1 + κ . In the global null (L = 1 L=1 L = 1 ) the leading coefficient 2 β m ( ρ ) 2\beta_m(\rho) 2 β m ( ρ ) is about 3.6 at q = 0.1 q=0.1 q = 0.1 , ρ = 1 \rho=1 ρ = 1 (τ B = 0 \tau_B=0 τ B = 0 , Remark 79), against the constant 1 2 ( 1 − q ) = 0.45 \frac{1}{2}(1-q)=0.45 2 1 ( 1 − q ) = 0.45 of Theorem 16 : a factor of about 8.
The global-null upper bound
In this subsection every test unit is an oracle null (p ℓ = p ℓ ∗ p_\ell=p_\ell^{*} p ℓ = p ℓ ∗ : every threshold equal to Y Y Y , or outlier detection with all units inliers), and the test units are i.i.d. and independent of D D D as in Theorem 78 . Then G D ( t ) = P ( p ∗ ≤ t ∣ D ) = F D ∗ ( t ) G_D(t)=\mathbb{P}(p^{*}\le t\mid D)=F_D^{*}(t) G D ( t ) = P ( p ∗ ≤ t ∣ D ) = F D ∗ ( t ) . Recall κ = B / n \kappa=B/n κ = B / n and B ‾ = ( ess sup w ) / E P [ w ] ≤ B \overline{B}=(\operatorname{ess\,sup} w)/\mathbb{E}_P[w]\le B B = ( ess sup w ) / E P [ w ] ≤ B .
Lemma 81 ( P ( G c ) \mathbb{P}(\mathcal{G}^{c}) P ( G c ) , continuous scores). Assume in addition that the score V V V has a continuous law under P P P (hence under Q Q Q ). Let ρ > 1 + κ \rho>1+\kappa ρ > 1 + κ and 0 < ε < ε max : = ( ρ − 1 − κ ) / ( ρ + 1 + κ ) 0<\varepsilon<\varepsilon_{\max}:=(\rho-1-\kappa)/(\rho+1+\kappa) 0 < ε < ε m a x := ( ρ − 1 − κ ) / ( ρ + 1 + κ ) . Then
P ( G c ) ≤ e − 2 a 1 − e − a + exp ( − n ε 2 2 B ( 1 + ε / 3 ) ) , a : = n ε 2 ρ q 2 B m ( 1 + ε / 3 ) . \mathbb{P}(\mathcal{G}^{c}) \le\frac{e^{-2a}}{1-e^{-a}}+\exp\left(-\frac{n\varepsilon^2}{2B(1+\varepsilon/3)}\right),\qquad a:=\frac{n\varepsilon^2\rho q}{2Bm(1+\varepsilon/3)}. P ( G c ) ≤ 1 − e − a e − 2 a + exp ( − 2 B ( 1 + ε /3 ) n ε 2 ) , a := 2 B m ( 1 + ε /3 ) n ε 2 ρq . :::`
Proof. Fix b = t k + τ B b=t_k+\tau_B b = t k + τ B with ρ b ≤ 1 \rho b\le1 ρ b ≤ 1 .
Step 1: from G D G_D G D to the weighted empirical CDF. Let W : = ∑ i ≤ n w i ≥ n W:=\sum_{i\le n}w_i\ge n W := ∑ i ≤ n w i ≥ n and G ^ ( v ) : = ∑ i w i 1 { V i < v } / W \widehat{G}(v):=\sum_i w_i\mathbf{1}\{V_i<v\}/W G ( v ) := ∑ i w i 1 { V i < v } / W . A test point with score v v v and weight w z ≤ B w_z\le B w z ≤ B has p = ( W G ^ ( v ) + U w z ) / ( W + w z ) p=(W\widehat{G}(v)+Uw_z)/(W+w_z) p = ( W G ( v ) + U w z ) / ( W + w z ) . So p ≤ b p\le b p ≤ b implies G ^ ( v ) ≤ b ( 1 + B / W ) ≤ b ′ : = b ( 1 + κ ) \widehat{G}(v)\le b(1+B/W)\le b^\prime:=b(1+\kappa) G ( v ) ≤ b ( 1 + B / W ) ≤ b ′ := b ( 1 + κ ) .
Step 2: localising. Let v ∗ v^* v ∗ satisfy Q < ( v ∗ ) : = Q ( V < v ∗ ) = ρ b Q_<(v^*):=Q(V<v^*)=\rho b Q < ( v ∗ ) := Q ( V < v ∗ ) = ρ b ; it exists by continuity. Grid points with ρ b > 1 \rho b>1 ρ b > 1 are trivially in G \mathcal{G} G . If G ^ ( v ∗ ) > b ′ \widehat{G}(v^*)>b^\prime G ( v ∗ ) > b ′ , then every v ≥ v ∗ v\ge v^* v ≥ v ∗ has G ^ ( v ) > b ′ \widehat{G}(v)>b^\prime G ( v ) > b ′ (G ^ \widehat{G} G is non-decreasing), so G D ( b ) ≤ Q < ( v ∗ ) = ρ b G_D(b)\le Q_<(v^*)=\rho b G D ( b ) ≤ Q < ( v ∗ ) = ρ b . Hence { G D ( b ) > ρ b } ⊂ { G ^ ( v ∗ ) ≤ b ′ } \{G_D(b)>\rho b\}\subset\{\widehat{G}(v^*)\le b^\prime\} { G D ( b ) > ρ b } ⊂ { G ( v ∗ ) ≤ b ′ } .
Step 3: Bernstein. Write G ^ ( v ∗ ) = S 1 / S 0 \widehat{G}(v^*)=S_1/S_0 G ( v ∗ ) = S 1 / S 0 , with S 1 : = n − 1 ∑ i w i 1 { V i < v ∗ } S_1:=n^{-1}\sum_i w_i\mathbf{1}\{V_i<v^*\} S 1 := n − 1 ∑ i w i 1 { V i < v ∗ } , of mean μ 1 = ρ b E P w \mu_1=\rho b\mathbb{E}_P w μ 1 = ρ b E P w (this uses d Q / d P = w / E P w dQ/dP=w/\mathbb{E}_P w d Q / d P = w / E P w exactly), and S 0 : = n − 1 ∑ i w i S_0:=n^{-1}\sum_i w_i S 0 := n − 1 ∑ i w i , of mean μ 0 = E P w ≥ 1 \mu_0=\mathbb{E}_P w\ge1 μ 0 = E P w ≥ 1 .
If S 1 > ( 1 − ε ) μ 1 S_1>(1-\varepsilon)\mu_1 S 1 > ( 1 − ε ) μ 1 and S 0 < ( 1 + ε ) μ 0 S_0<(1+\varepsilon)\mu_0 S 0 < ( 1 + ε ) μ 0 , then G ^ ( v ∗ ) > ρ b ( 1 − ε ) / ( 1 + ε ) ≥ b ′ \widehat{G}(v^*)>\rho b(1-\varepsilon)/(1+\varepsilon)\ge b^\prime G ( v ∗ ) > ρ b ( 1 − ε ) / ( 1 + ε ) ≥ b ′ , because ε < ε max \varepsilon<\varepsilon_{\max} ε < ε m a x .
The summands lie in [ 0 , B ] [0,B] [ 0 , B ] (respectively [ 1 , B ] [1,B] [ 1 , B ] ), with variance at most B B B times their mean. The one-sided Bernstein inequality, Lemma 28, with range constant B B B (applied to μ 1 − w i 1 { V i < v ∗ } \mu_1-w_i\mathbf{1}\{V_i<v^*\} μ 1 − w i 1 { V i < v ∗ } for the lower tail of S 1 S_1 S 1 and to w i − μ 0 w_i-\mu_0 w i − μ 0 for the upper tail of S 0 S_0 S 0 ), gives
P ( S 1 ≤ ( 1 − ε ) μ 1 ) ≤ exp ( − n ε 2 μ 1 2 B ( 1 + ε / 3 ) ) ≤ exp ( − n ε 2 ρ q k 2 B m ( 1 + ε / 3 ) ) , \mathbb{P}(S_1\le(1-\varepsilon)\mu_1)\le\exp\left(-\frac{n\varepsilon^2\mu_1}{2B(1+\varepsilon/3)}\right)\le\exp\left(-\frac{n\varepsilon^2\rho qk}{2Bm(1+\varepsilon/3)}\right), P ( S 1 ≤ ( 1 − ε ) μ 1 ) ≤ exp ( − 2 B ( 1 + ε /3 ) n ε 2 μ 1 ) ≤ exp ( − 2 B m ( 1 + ε /3 ) n ε 2 ρq k ) ,
P ( S 0 ≥ ( 1 + ε ) μ 0 ) ≤ exp ( − n ε 2 2 B ( 1 + ε / 3 ) ) . \mathbb{P}(S_0\ge(1+\varepsilon)\mu_0)\le\exp\left(-\frac{n\varepsilon^2}{2B(1+\varepsilon/3)}\right). P ( S 0 ≥ ( 1 + ε ) μ 0 ) ≤ exp ( − 2 B ( 1 + ε /3 ) n ε 2 ) .
(Here μ 1 ≥ ρ b ≥ ρ t k = ρ q k / m \mu_1\ge\rho b\ge\rho t_k=\rho qk/m μ 1 ≥ ρ b ≥ ρ t k = ρq k / m , since E P w ≥ 1 \mathbb{E}_P w\ge1 E P w ≥ 1 and b ≥ t k b\ge t_k b ≥ t k ; and μ 0 ≥ 1 \mu_0\ge1 μ 0 ≥ 1 .)
Step 4: union. The S 0 S_0 S 0 event does not depend on k k k . The union over k = 2 , … , m k=2,\ldots,m k = 2 , … , m of the S 1 S_1 S 1 events has probability at most ∑ k ≥ 2 e − a k = e − 2 a / ( 1 − e − a ) \sum_{k\ge2}e^{-ak}=e^{-2a}/(1-e^{-a}) ∑ k ≥ 2 e − ak = e − 2 a / ( 1 − e − a ) .
With L = 1 L=1 L = 1 (oracle nulls are exactly uniform), in the global null with continuous scores,
FDR ≤ q [ 1 + 2 ( m − 1 ) β m ( ρ ) B n + 1 + ( m − 1 ) ( e − 2 a 1 − e − a + e − n ε 2 / ( 2 B ( 1 + ε / 3 ) ) ) ] . \operatorname{FDR}\le q\left[1+2(m-1)\beta_m(\rho)\frac{B}{n+1}+(m-1)\left(\frac{e^{-2a}}{1-e^{-a}}+e^{-n\varepsilon^2/(2B(1+\varepsilon/3))}\right)\right]. FDR ≤ q [ 1 + 2 ( m − 1 ) β m ( ρ ) n + 1 B + ( m − 1 ) ( 1 − e − a e − 2 a + e − n ε 2 / ( 2 B ( 1 + ε /3 )) ) ] .
This is fully explicit, and useful only when n ≫ B m / ( q ε 2 ) n\gg Bm/(q\varepsilon^2) n ≫ B m / ( q ε 2 ) . For example, at q = 0.1 q=0.1 q = 0.1 , m = 10 m=10 m = 10 , B = 2 B=2 B = 2 , ρ = 1.2 \rho=1.2 ρ = 1.2 , ε = 0.09 \varepsilon=0.09 ε = 0.09 : ( m − 1 ) P ( G c ) ≤ 0.089 (m-1)\mathbb{P}(\mathcal{G}^c)\le0.089 ( m − 1 ) P ( G c ) ≤ 0.089 at n = 10 5 n=10^5 n = 1 0 5 , and < 10 − 19 <10^{-19} < 1 0 − 19 at n = 10 6 n=10^6 n = 1 0 6 . A Monte Carlo check at n = 10 5 n=10^5 n = 1 0 5 found no failure of G \mathcal{G} G , against the bound P ( G c ) ≤ 0.0099 \mathbb{P}(\mathcal{G}^c)\le0.0099 P ( G c ) ≤ 0.0099 : the bound is valid but loose (the ρ \rho ρ actually needed is about 1.07 1.07 1.07 , not 1.2 1.2 1.2 ). The crude step Υ j − 1 ≤ m − 1 \Upsilon_j-1\le m-1 Υ j − 1 ≤ m − 1 on G c \mathcal{G}^c G c is what costs here. With score atoms, Step 2 needs the U U U -randomisation; that case is Corollary 83.
Corollary 83 (P ( G c ) \mathbb{P}(\mathcal{G}^c) P ( G c ) for every score law). Assume (A1) and (A3), with weights 1 ≤ w ≤ B 1\le w\le B 1 ≤ w ≤ B , and q ∈ ( 0 , 1 ) q\in(0,1) q ∈ ( 0 , 1 ) ; every test unit is an oracle null; the test units are i.i.d. given D D D (JC’s independence); and ρ > 1 + κ \rho>1+\kappa ρ > 1 + κ . Put
a ′ : = n q ( ρ − 1 − κ ) 2 2 B m ( 3 ρ − 2 − 2 κ ) . a^\prime:=\frac{nq(\rho-1-\kappa)^2}{2Bm(3\rho-2-2\kappa)}. a ′ := 2 B m ( 3 ρ − 2 − 2 κ ) n q ( ρ − 1 − κ ) 2 . Then, with no assumption on the score law (atoms allowed),
P ( G c ) ≤ e − 2 a ′ 1 − e − a ′ , \mathbb{P}(\mathcal{G}^c)\le\frac{e^{-2a^\prime}}{1-e^{-a^\prime}}, P ( G c ) ≤ 1 − e − a ′ e − 2 a ′ , and therefore, by Theorem 78 with L = 1 L=1 L = 1 ,
FDR ≤ q [ 1 + 2 ( m − 1 ) β m ( ρ ) B n + 1 + ( m − 1 ) e − 2 a ′ 1 − e − a ′ ] . \operatorname{FDR}\le q\left[1+2(m-1)\beta_m(\rho)\frac{B}{n+1}+(m-1)\frac{e^{-2a^\prime}}{1-e^{-a^\prime}}\right]. FDR ≤ q [ 1 + 2 ( m − 1 ) β m ( ρ ) n + 1 B + ( m − 1 ) 1 − e − a ′ e − 2 a ′ ] .
Proof. • In the global null with oracle nulls, the test p-values are the p ∗ p^* p ∗ , so G D = F D ∗ G_D=F_D^* G D = F D ∗ .
Steps 1–4 of the proof of the explicit bound on E [ M D ] \mathbb{E}[M_D] E [ M D ] (Theorem 6 ; Appendix B), namely the passage from p ∗ p^* p ∗ to a weighted empirical CDF, its population version, the localisation in the lexicographic order of ( score , U ) (\text{score},U) ( score , U ) , and the Bernstein step, are written there for t = t k t=t_k t = t k but hold for any t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) . They handle score atoms through the lexicographic order and the U U U -randomisation, and they give, for b ∈ ( 0 , 1 ) b\in(0,1) b ∈ ( 0 , 1 ) and d > 0 d>0 d > 0 ,
P ( F D ∗ ( b ) > b ( 1 + κ ) + d ) ≤ exp ( − n d 2 2 B ‾ ( b ( 1 + κ ) + 3 d ) ) , (16) \mathbb{P}\left(F_D^*(b)>b(1+\kappa)+d\right)\leq\exp\left(-\frac{nd^2}{2\overline{B}\left(b(1+\kappa)+3d\right)}\right), \tag*{(16)} P ( F D ∗ ( b ) > b ( 1 + κ ) + d ) ≤ exp ( − 2 B ( b ( 1 + κ ) + 3 d ) n d 2 ) , (16)
with probability 0 once b ( 1 + κ ) + d ≥ 1 b(1+\kappa)+d\geq1 b ( 1 + κ ) + d ≥ 1 . We do not repeat these steps here.
Take b : = b k : = t k + τ B b:=b_k:=t_k+\tau_B b := b k := t k + τ B and d : = ( ρ − 1 − κ ) b k > 0 d:=(\rho-1-\kappa)b_k>0 d := ( ρ − 1 − κ ) b k > 0 . Then b k ( 1 + κ ) + d = ρ b k b_k(1+\kappa)+d=\rho b_k b k ( 1 + κ ) + d = ρ b k , so { G D ( b k ) > ρ b k } \{G_D(b_k)>\rho b_k\} { G D ( b k ) > ρ b k } is the event on the left of (16) (it is empty if ρ b k ≥ 1 \rho b_k\geq1 ρ b k ≥ 1 ). – Since b ( 1 + κ ) + 3 d = b k ( 3 ρ − 2 − 2 κ ) b(1+\kappa)+3d=b_k(3\rho-2-2\kappa) b ( 1 + κ ) + 3 d = b k ( 3 ρ − 2 − 2 κ ) , the exponent is n ( ρ − 1 − κ ) 2 b k / ( 2 B ‾ ( 3 ρ − 2 − 2 κ ) ) n(\rho-1-\kappa)^2b_k/(2\overline{B}(3\rho-2-2\kappa)) n ( ρ − 1 − κ ) 2 b k / ( 2 B ( 3 ρ − 2 − 2 κ )) . – Since b k ≥ t k = q k / m b_k\geq t_k=qk/m b k ≥ t k = q k / m , the exponent is at least k a ′ ka' k a ′ .
A union over k = 2 , … , m k=2,\ldots,m k = 2 , … , m gives P ( G c ) ≤ ∑ k ≥ 2 e − k a ′ = e − 2 a ′ / ( 1 − e − a ′ ) \mathbb{P}(G^c)\leq\sum_{k\geq2}e^{-ka'}=e^{-2a'}/(1-e^{-a'}) P ( G c ) ≤ ∑ k ≥ 2 e − k a ′ = e − 2 a ′ / ( 1 − e − a ′ ) .
The FDR bound is Theorem 78 with L = 1 L=1 L = 1 : with oracle nulls, p ℓ = p ℓ ∗ p_\ell=p_\ell^* p ℓ = p ℓ ∗ is exactly Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) for every score law (Lemma 1), so P ( p ℓ ∈ I k ) = ∣ I k ∣ \mathbb{P}(p_\ell\in I_k)=|I_k| P ( p ℓ ∈ I k ) = ∣ I k ∣ .
Remark 84 (Comparison with Lemma 81 ). Corollary 83 has no S 0 S_0 S 0 term and uses B ‾ ≤ B \overline{B}\leq B B ≤ B . Its exponent is better than that of Lemma 81 only for ρ ≲ 2.5 \rho\lesssim2.5 ρ ≲ 2.5 (with κ = 0 \kappa=0 κ = 0 and B ‾ = B \overline{B}=B B = B ): at ρ = 3 \rho=3 ρ = 3 the ratio a ′ / a a'/a a ′ / a is 0.89 0.89 0.89 , and at ρ = 10 \rho=10 ρ = 10 it is 0.55 0.55 0.55 . The regime that matters is ρ \rho ρ close to 1, where Corollary 83 is better: at ρ = 1.2 \rho=1.2 ρ = 1.2 and κ ≈ 0 \kappa\approx0 κ ≈ 0 , a ′ = 0.0125 n q / ( B ‾ m ) a'=0.0125\,nq/(\overline{B}m) a ′ = 0.0125 n q / ( B m ) , against a ≈ 0.0048 n q / ( B m ) a\approx0.0048\,nq/(Bm) a ≈ 0.0048 n q / ( B m ) . For q = 0.1 q=0.1 q = 0.1 , m = 10 m=10 m = 10 , B = B ‾ = 2 B=\overline{B}=2 B = B = 2 , ρ = 1.2 \rho=1.2 ρ = 1.2 and n = 10 5 n=10^5 n = 1 0 5 : a ′ = 6.25 a'=6.25 a ′ = 6.25 , so ( m − 1 ) P ( G c ) ≤ 9 e − 12.5 / ( 1 − e − 6.25 ) = 3.4 ⋅ 10 − 5 (m-1)\mathbb{P}(G^c)\leq9e^{-12.5}/(1-e^{-6.25})=3.4\cdot10^{-5} ( m − 1 ) P ( G c ) ≤ 9 e − 12.5 / ( 1 − e − 6.25 ) = 3.4 ⋅ 1 0 − 5 , against 0.089 0.089 0.089 from Lemma 81 .
Proof of Theorem 17. Assume (A1) and (A3) with 1 ≤ w ≤ B 1\leq w\leq B 1 ≤ w ≤ B , q ∈ ( 0 , 1 ) q\in(0,1) q ∈ ( 0 , 1 ) , test units i.i.d. and independent of D D D , every test unit an oracle null (the global null), and ρ > 1 + κ \rho>1+\kappa ρ > 1 + κ . By Theorem 78 ,
FDR ≤ q [ 1 + 2 L ( m − 1 ) β m ( ρ ) B n + 1 + ( m − 1 ) P ( G c ) ] . \operatorname{FDR}\leq q\left[1+2L(m-1)\beta_m(\rho)\frac{B}{n+1}+(m-1)\mathbb{P}(G^c)\right]. FDR ≤ q [ 1 + 2 L ( m − 1 ) β m ( ρ ) n + 1 B + ( m − 1 ) P ( G c ) ] .
With oracle nulls, p ℓ = p ℓ ∗ p_\ell=p_\ell^* p ℓ = p ℓ ∗ is exactly Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) for every score law (Lemma 1; the U U U -randomisation handles ties), so P ( p ℓ ∈ I k ) = ∣ I k ∣ \mathbb{P}(p_\ell\in I_k)=|I_k| P ( p ℓ ∈ I k ) = ∣ I k ∣ and L = 1 L=1 L = 1 . Theorem 78 itself makes no continuity assumption: its ingredients are Lemma 75, the envelope (which handles ties through Lemma 67 ) and the monotonicity of binomial tails. Corollary 83 gives P ( G c ) ≤ e − 2 a ′ / ( 1 − e − a ′ ) \mathbb{P}(G^c)\leq e^{-2a'}/(1-e^{-a'}) P ( G c ) ≤ e − 2 a ′ / ( 1 − e − a ′ ) with a ′ a' a ′ as in the statement. Substituting gives the displayed bound, with β m ( ρ ) \beta_m(\rho) β m ( ρ ) explicit (Theorem 78 ).
Fixed m m m . The paragraph “When L = 1 L=1 L = 1 holds exactly” (Corollary 77 with L = 1 L=1 L = 1 ) gives FDR − q ≤ q m ( m − 1 ) 2 B / ( n + 1 ) \operatorname{FDR}-q\leq qm(m-1)2B/(n+1) FDR − q ≤ q m ( m − 1 ) 2 B / ( n + 1 ) for all n n n and B B B , so at fixed m m m the excess is O ( B / n ) O(B/n) O ( B / n ) .
Non-oracle nulls. If every test unit is null but some have c n + j > Y n + j c_{n+j}>Y_{n+j} c n + j > Y n + j , then p j ≥ p j ∗ p_j\geq p_j^* p j ≥ p j ∗ for every j j j (the score is monotone, and the p p p -value of eq. (4) is non-decreasing in the test score). Whenever BH rejects at least one hypothesis with the p j p_j p j , it also does so with the smaller p j ∗ p_j^* p j ∗ , so the FWER, which is the FDR here, is at most the FWER of the p j ∗ p_j^* p j ∗ . These are the p-values of the configuration with every threshold replaced by Y Y Y , an oracle-null configuration with the same P P P , Q Q Q and weights. Both bounds above therefore hold for every global-null configuration.
Remark 85 (Linearity in m m m ). The bound of Theorem 17 is linear in m m m wherever β m ( ρ ) \beta_m(\rho) β m ( ρ ) stays bounded in m m m . Numerically it does at τ B = 0 \tau_B=0 τ B = 0 (β m ( 1 ) ≈ 1.78 \beta_m(1)\approx1.78 β m ( 1 ) ≈ 1.78 at m = 1000 m=1000 m = 1000 , q = 0.1 q=0.1 q = 0.1 ; Remark 79). The m m m -free majorant of β m ( ρ ) \beta_m(\rho) β m ( ρ ) is only a sketch (Remark 79), and it also needs ρ ( q + m τ B ) < 1 \rho(q+m\tau_B)<1 ρ ( q + m τ B ) < 1 . So we have proved the bound with β m ( ρ ) \beta_m(\rho) β m ( ρ ) explicit, but we have not proved that it is linear in m m m : that rests on the sketch. The bound is informative only once n ≫ B ‾ m / ( q ( ρ − 1 ) ) n\gg\overline{B}m/(q(\rho-1)) n ≫ B m / ( q ( ρ − 1 )) , and its constant is still about 8 times the constant 1 2 ( 1 − q ) \frac{1}{2}(1-q) 2 1 ( 1 − q ) of Theorem 16 .
Certificates for the counterexamples
This appendix documents the two computations behind Theorem 5 . The proof of that theorem is computer-assisted. In each instance the covariate takes finitely many values, the FDR is a finite sum of rational numbers, and that sum was evaluated in exact rational arithmetic (no floating point at any stage), so the comparison with q q q is exact. What has to be trusted is that the programs implement the formulas below. For each instance we state the configuration, the finite formula that was evaluated, and which independent programs were run on which instances.
Common reductions
Setting. Calibration ( X i , Y i ) i ≤ n (X_i,Y_i)_{i\le n} ( X i , Y i ) i ≤ n i.i.d. P P P ; test tuples ( X n + j , Y n + j , c n + j ) j ≤ m (X_{n+j},Y_{n+j},c_{n+j})_{j\le m} ( X n + j , Y n + j , c n + j ) j ≤ m i.i.d., with ( X n + j , Y n + j ) ∼ Q (X_{n+j},Y_{n+j})\sim Q ( X n + j , Y n + j ) ∼ Q , d Q / d P ( x , y ) = w ( x ) / E P w \mathrm{d}Q/\mathrm{d}P(x,y)=w(x)/\mathbb{E}_P w d Q / d P ( x , y ) = w ( x ) / E P w , independent of the calibration data D D D ; tie-breaking variables U j U_j U j i.i.d. Unif [ 0 , 1 ] \operatorname{Unif}[0,1] Unif [ 0 , 1 ] , independent of everything else. Unit j j j is null iff Y n + j ≤ c n + j Y_{n+j}\le c_{n+j} Y n + j ≤ c n + j . With V ^ n + j : = V ( X n + j , c n + j ) \widehat V_{n+j}:=V(X_{n+j},c_{n+j}) V n + j := V ( X n + j , c n + j ) and V i : = V ( X i , Y i ) V_i:=V(X_i,Y_i) V i := V ( X i , Y i ) , the weighted conformal p-value is Jin & Candès [12 ] ’s eq. (4),
p j = ∑ i = 1 n w ( X i ) 1 { V i < V ^ n + j } + ( w ( X n + j ) + ∑ i = 1 n w ( X i ) 1 { V i = V ^ n + j } ) U j ∑ i = 1 n w ( X i ) + w ( X n + j ) . p_j=\frac{\sum_{i=1}^{n}w(X_i)\mathbf{1}\{V_i<\widehat V_{n+j}\}+\left(w(X_{n+j})+\sum_{i=1}^{n}w(X_i)\mathbf{1}\{V_i=\widehat V_{n+j}\}\right)U_j}{\sum_{i=1}^{n}w(X_i)+w(X_{n+j})}. p j = ∑ i = 1 n w ( X i ) + w ( X n + j ) ∑ i = 1 n w ( X i ) 1 { V i < V n + j } + ( w ( X n + j ) + ∑ i = 1 n w ( X i ) 1 { V i = V n + j } ) U j .
BH at level q q q rejects R = { j : p j ≤ t k ∗ } \mathcal{R}=\{j:p_j\le t_{k^*}\} R = { j : p j ≤ t k ∗ } , k ∗ = max { k : # { j : p j ≤ t k } ≥ k } k^*=\max\{k:\#\{j:p_j\le t_k\}\ge k\} k ∗ = max { k : # { j : p j ≤ t k } ≥ k } (R = ∅ \mathcal{R}=\varnothing R = ∅ if no such k k k ), and FDR = E [ # { j ∈ R : Y n + j ≤ c n + j } / max ( 1 , ∣ R ∣ ) ] \operatorname{FDR}=\mathbb{E}\left[\#\{j\in\mathcal{R}:Y_{n+j}\le c_{n+j}\}/\max(1,|\mathcal{R}|)\right] FDR = E [ # { j ∈ R : Y n + j ≤ c n + j } / max ( 1 , ∣ R ∣ ) ] . Since p j p_j p j is unchanged when w w w is multiplied by a positive constant, weights below are given up to a common factor, and the weight ratio is B = max w / min w B=\max w/\min w B = max w / min w .
(a) Per-cell laws. Let S : = ∑ i ≤ n w ( X i ) S:=\sum_{i\le n}w(X_i) S := ∑ i ≤ n w ( X i ) . If the threshold is a function of the covariate, c n + j = c ( X n + j ) c_{n+j}=c(X_{n+j}) c n + j = c ( X n + j ) , then given D D D and X n + j = x X_{n+j}=x X n + j = x , eq. (4) is affine in U j U_j U j , so
p j ∣ D , X n + j = x ∼ Unif [ S < ( x ) S + w ( x ) , S < ( x ) + w ( x ) + S = ( x ) S + w ( x ) ] . (17) p_j\mid D,X_{n+j}=x\sim\operatorname{Unif}\left[\frac{S_{<}(x)}{S+w(x)},\frac{S_{<}(x)+w(x)+S_{=}(x)}{S+w(x)}\right]. \tag*{(17)} p j ∣ D , X n + j = x ∼ Unif [ S + w ( x ) S < ( x ) , S + w ( x ) S < ( x ) + w ( x ) + S = ( x ) ] . (17)
where S < ( x ) S_{<}(x) S < ( x ) and S = ( x ) S_{=}(x) S = ( x ) are the total calibration weights with score below, respectively equal to, V ^ ( x ) = V ( x , c ( x ) ) \widehat V(x)=V(x,c(x)) V ( x ) = V ( x , c ( x )) . Write G x ( t ) = G x ( t ; D ) G_x(t)=G_x(t;D) G x ( t ) = G x ( t ; D ) for the CDF of this uniform law.
(b) Conditional i.i.d. structure. The pair ( p j , 1 { j null } ) (p_j,\mathbf{1}\{j\text{ null}\}) ( p j , 1 { j null }) is a fixed function of D D D and of the j j j -th tuple ( X n + j , Y n + j , c n + j , U j ) (X_{n+j},Y_{n+j},c_{n+j},U_j) ( X n + j , Y n + j , c n + j , U j ) . These tuples are i.i.d. and independent of D D D , so given D D D the pairs ( p j , 1 { j null } ) (p_j,\mathbf{1}\{j\text{ null}\}) ( p j , 1 { j null }) , j ≤ m j\le m j ≤ m , are i.i.d. Write F D ( t ) : = P ( p 1 ≤ t ∣ D ) F_D(t):=\mathbb{P}(p_1\le t\mid D) F D ( t ) := P ( p 1 ≤ t ∣ D ) and, as in Lemma 2 , A k ( D ) : = P ( unit 1 null , p 1 ≤ t k ∣ D ) A_k(D):=\mathbb{P}(\text{unit 1 null},p_1\le t_k\mid D) A k ( D ) := P ( unit 1 null , p 1 ≤ t k ∣ D ) . All these quantities depend on D D D only through the vector of calibration counts of the support points of ( X , Y ) (X,Y) ( X , Y ) , which is multinomial; the sums over D D D below are sums over these count vectors.
(c) Two identities for E [ FDP ∣ D ] \mathbb{E}[\operatorname{FDP}\mid D] E [ FDP ∣ D ] . For M ≥ 0 M\ge0 M ≥ 0 and k ≤ m k\le m k ≤ m let
Φ M , k ( D ) : = ∑ c ( M c k + 1 , … , c m , M − ∣ c ∣ ) ∏ j = k + 1 m ( F D ( t j ) − F D ( t j − 1 ) ) c j ( 1 − F D ( t m ) ) M − ∣ c ∣ , (18) \Phi_{M,k}(D):=\sum_{\mathbf{c}}\binom{M}{c_{k+1},\ldots,c_m,M-|\mathbf{c}|}\prod_{j=k+1}^{m}\left(F_D(t_j)-F_D(t_{j-1})\right)^{c_j}\left(1-F_D(t_m)\right)^{M-|\mathbf{c}|}, \tag*{(18)} Φ M , k ( D ) := c ∑ ( c k + 1 , … , c m , M − ∣ c ∣ M ) j = k + 1 ∏ m ( F D ( t j ) − F D ( t j − 1 ) ) c j ( 1 − F D ( t m ) ) M − ∣ c ∣ , (18)
the sum running over c = ( c k + 1 , … , c m ) ∈ Z ≥ 0 m − k \mathbf{c}=(c_{k+1},\ldots,c_m)\in\mathbb{Z}_{\ge0}^{m-k} c = ( c k + 1 , … , c m ) ∈ Z ≥ 0 m − k with c k + 1 + ⋯ + c j ≤ j − k − 1 c_{k+1}+\cdots+c_j\le j-k-1 c k + 1 + ⋯ + c j ≤ j − k − 1 for every j ∈ { k + 1 , … , m } j\in\{k+1,\ldots,m\} j ∈ { k + 1 , … , m } , and ∣ c ∣ = c k + 1 + ⋯ + c m |\mathbf{c}|=c_{k+1}+\cdots+c_m ∣ c ∣ = c k + 1 + ⋯ + c m (Φ 0 , k : = 1 \Phi_{0,k}:=1 Φ 0 , k := 1 ). Given D D D , Φ M , k ( D ) \Phi_{M,k}(D) Φ M , k ( D ) is the probability that M M M i.i.d. draws from the conditional law of p 1 p_1 p 1 all exceed t k t_k t k and that, for every j > k j>k j > k , at most j − k − 1 j-k-1 j − k − 1 of them are ≤ t j \le t_j ≤ t j ; the multinomial sum assigns the draws to the bands ( t j − 1 , t j ] (t_{j-1},t_j] ( t j − 1 , t j ] , j > k j>k j > k , and to ( t m , 1 ] (t_m,1] ( t m , 1 ] . The engines evaluate it by a dynamic programme over j = k + 1 , … , m j=k+1,\ldots,m j = k + 1 , … , m whose state is the running total c k + 1 + ⋯ + c j c_{k+1}+\cdots+c_j c k + 1 + ⋯ + c j .
Leave-one-out form. Let R j R_j R j be the number of BH rejections when p j p_j p j is replaced by 0 0 0 . By the leave-one-out identity (Lemma 66), j ∈ R j\in\mathcal{R} j ∈ R iff p j ≤ t R j p_j\le t_{R_j} p j ≤ t R j , and then ∣ R ∣ = R j |\mathcal{R}|=R_j ∣ R ∣ = R j ; hence FDP = ∑ j ≤ m 1 { j null } 1 { p j ≤ t R j } / R j \operatorname{FDP}=\sum_{j\le m}\mathbf{1}\{j\text{ null}\}\mathbf{1}\{p_j\le t_{R_j}\}/R_j FDP = ∑ j ≤ m 1 { j null } 1 { p j ≤ t R j } / R j . Given D D D , the m m m summands have the same law by (b), and R 1 R_1 R 1 is a function of the units 2 , … , m 2,\ldots,m 2 , … , m , hence independent of unit 1 given D D D . So, with r k ( D ) = P ( R 1 = k ∣ D ) r_k(D)=\mathbb{P}(R_1=k\mid D) r k ( D ) = P ( R 1 = k ∣ D ) as in Lemma 2 ,
E [ FDP ∣ D ] = m ∑ k = 1 m 1 k A k ( D ) r k ( D ) , r k ( D ) = ( m − 1 k − 1 ) F D ( t k ) k − 1 Φ m − k , k ( D ) . (19) \mathbb{E}[\operatorname{FDP}\mid D]=m\sum_{k=1}^{m}\frac{1}{k}A_k(D)r_k(D),\qquad r_k(D)=\binom{m-1}{k-1}F_D(t_k)^{k-1}\Phi_{m-k,k}(D). \tag*{(19)} E [ FDP ∣ D ] = m k = 1 ∑ m k 1 A k ( D ) r k ( D ) , r k ( D ) = ( k − 1 m − 1 ) F D ( t k ) k − 1 Φ m − k , k ( D ) . (19)
For the second formula: R 1 = k R_1 = k R 1 = k iff at least k k k of ( 0 , p 2 , … , p m ) (0,p_2,\ldots,p_m) ( 0 , p 2 , … , p m ) are ≤ t k \leq t_k ≤ t k and fewer than j j j are ≤ t j \leq t_j ≤ t j for every j > k j>k j > k ; taking j = k + 1 j=k+1 j = k + 1 forces exactly k k k of them ≤ t k \leq t_k ≤ t k , i.e. exactly k − 1 k-1 k − 1 of p 2 , … , p m p_2,\ldots,p_m p 2 , … , p m , and the other m − k m-k m − k satisfy the event defining Φ m − k , k \Phi_{m-k,k} Φ m − k , k .
Cutoff form. By the same counting, P ( k ∗ = k ∣ D ) = ( m k ) F D ( t k ) k Φ m − k , k ( D ) \mathbb{P}(k^* = k \mid D) = \binom{m}{k} F_D(t_k)^k \Phi_{m-k,k}(D) P ( k ∗ = k ∣ D ) = ( k m ) F D ( t k ) k Φ m − k , k ( D ) for k ≥ 1 k \geq1 k ≥ 1 . Given D D D and k ∗ = k k^* = k k ∗ = k , the rejected set is { j : p j ≤ t k } \{j : p_j \leq t_k\} { j : p j ≤ t k } , and the event { k ∗ = k } \{k^* = k\} { k ∗ = k } depends on the rejected units only through { p j ≤ t k } \{p_j \leq t_k\} { p j ≤ t k } . So, by (b), each rejected unit is null with conditional probability A k ( D ) / F D ( t k ) A_k(D)/F_D(t_k) A k ( D ) / F D ( t k ) , and
E [ F D P ∣ D ] = ∑ k ≥ 1 : F D ( t k ) > 0 ( m k ) F D ( t k ) k Φ m − k , k ( D ) A k ( D ) F D ( t k ) . (20) \mathbb{E}[\mathrm{FDP} \mid D] =
\sum_{\substack{k \geq1: F_D(t_k)>0}}
\binom{m}{k} F_D(t_k)^k \Phi_{m-k,k}(D)
\frac{A_k(D)}{F_D(t_k)}. \tag*{(20)} E [ FDP ∣ D ] = k ≥ 1 : F D ( t k ) > 0 ∑ ( k m ) F D ( t k ) k Φ m − k , k ( D ) F D ( t k ) A k ( D ) . (20)
In both forms, F D R = ∑ D P ( D ) E [ F D P ∣ D ] \mathrm{FDR} = \sum_D \mathbb{P}(D)\mathbb{E}[\mathrm{FDP} \mid D] FDR = ∑ D P ( D ) E [ FDP ∣ D ] .
(d) The independent checker. A separate program, written directly from the definitions above and sharing no code with the engines of Section F.3, evaluates
F D R = ∑ z ∈ Z n ∑ z ′ ∈ Z ′ m ∏ i = 1 n P ( z i ) ∏ j = 1 m Q ′ ( z j ′ ) ∫ [ 0 , 1 ] m F D P ( z , z ′ , u ) d u , (21) \mathrm{FDR} =
\sum_{z\in\mathcal{Z}^n}
\sum_{z'\in\mathcal{Z}'^m}
\prod_{i=1}^{n}\mathbb{P}(z_i)
\prod_{j=1}^{m}Q'(z'_j)
\int_{[0,1]^m}\mathrm{FDP}(z,z',u)\,du, \tag*{(21)} FDR = z ∈ Z n ∑ z ′ ∈ Z ′ m ∑ i = 1 ∏ n P ( z i ) j = 1 ∏ m Q ′ ( z j ′ ) ∫ [ 0 , 1 ] m FDP ( z , z ′ , u ) d u , (21)
where Z \mathcal{Z} Z is the (finite) support of ( X , Y ) (X,Y) ( X , Y ) under P P P , Z ′ \mathcal{Z}' Z ′ the support of ( X , Y , c ) (X,Y,c) ( X , Y , c ) for a test unit, Q ′ Q' Q ′ its law, and F D P ( z , z ′ , u ) \mathrm{FDP}(z,z',u) FDP ( z , z ′ , u ) is the BH false discovery proportion for calibration sequence z z z , test sequence z ′ z' z ′ and tie-breakers u u u . It enumerates every ordered calibration and test sequence. For each, p j = α j + β j u j p_j=\alpha_j+\beta_j u_j p j = α j + β j u j with α j , β j \alpha_j,\beta_j α j , β j read off eq. (4); it cuts [ 0 , 1 ] [0,1] [ 0 , 1 ] for each u j u_j u j at the points where α j + β j u j \alpha_j+\beta_j u_j α j + β j u j equals a grid point t k t_k t k , so that on every box of the product partition each p j p_j p j stays in one BH bin and BH’s rejection set is constant; it evaluates BH at the box centre and adds (box volume)× F D P \times\mathrm{FDP} × FDP . The integral in (21) is therefore computed exactly. The same program computes P ( p 1 ≤ t , unit 1 null ) \mathbb{P}(p_1\leq t,\ \text{unit 1 null}) P ( p 1 ≤ t , unit 1 null ) exactly, to check eq. (3) of Jin & Candès [12 ] and, for oracle nulls, the exact uniformity of Lemma 1.
Instance (i): global null, n = 5 n=5 n = 5 , m = 2 m=2 m = 2 , q = 4 / 5 q=4/5 q = 4/5
Configuration. Covariate X ∈ { 0 , 1 , 2 } X \in\{0,1,2\} X ∈ { 0 , 1 , 2 } ; response Y ≡ 0 Y \equiv0 Y ≡ 0 ; pre-determined threshold c n + j = 0 c_{n+j}=0 c n + j = 0 for every test unit, so every test unit is null and F D R = P ( R ≠ ∅ ) \mathrm{FDR}=\mathbb{P}(R\neq\varnothing) FDR = P ( R = ∅ ) ; score V ( x , y ) = y − μ ^ ( x ) V(x,y)=y-\hat{\mu}(x) V ( x , y ) = y − μ ^ ( x ) , monotone in y y y , with μ ^ = ( − 2 , − 1 , 0 ) \hat{\mu}=(-2,-1,0) μ ^ = ( − 2 , − 1 , 0 ) . Since c = Y c=Y c = Y , V ^ n + j = V ( X n + j , Y n + j ) \hat{V}_{n+j}=V(X_{n+j},Y_{n+j}) V ^ n + j = V ( X n + j , Y n + j ) : every p-value is an oracle p-value.
Table 1 (PDF p. 52).
Here E P w = 31 / 5 \mathbb{E}_P w=31/5 E P w = 31/5 and Q ( X = x ) = P ( X = x ) w ( x ) / E P w Q(X=x)=P(X=x)w(x)/\mathbb{E}_P w Q ( X = x ) = P ( X = x ) w ( x ) / E P w . The weight ratio is B = 50 B=50 B = 50 , and the weight is strictly decreasing in the score. BH’s grid is t 1 = q / 2 = 2 / 5 t_1=q/2=2/5 t 1 = q /2 = 2/5 , t 2 = q = 4 / 5 t_2=q=4/5 t 2 = q = 4/5 .
Per-cell laws. Let ( N 0 , N 1 , N 2 ) (N_0,N_1,N_2) ( N 0 , N 1 , N 2 ) be the calibration counts of the three cells, so
P ( D ) = 5 ! N 0 ! N 1 ! N 2 ! ( 3 4 ) N 0 ( 3 20 ) N 1 ( 1 10 ) N 2 , \mathbb{P}(D)=\frac{5!}{N_0!N_1!N_2!}
\left(\frac{3}{4}\right)^{N_0}
\left(\frac{3}{20}\right)^{N_1}
\left(\frac{1}{10}\right)^{N_2}, P ( D ) = N 0 ! N 1 ! N 2 ! 5 ! ( 4 3 ) N 0 ( 20 3 ) N 1 ( 10 1 ) N 2 ,
and S = N 0 + 3 N 1 + 50 N 2 S=N_0+3N_1+50N_2 S = N 0 + 3 N 1 + 50 N 2 . By (16), given D D D a test unit in cell x x x has
x = 2 : Unif [ 0 , 50 ( 1 + N 2 ) S + 50 ] , x = 1 : Unif [ 50 N 2 S + 3 , 50 N 2 + 50 N 2 + 3 ( 1 + N 1 ) S + 3 ] , x = 0 : Unif [ 50 N 2 + 3 N 1 S + 1 , 1 ] . x=2:\quad\operatorname{Unif}\left[0,\frac{50(1+N_2)}{S+50}\right],\qquad
x=1:\quad\operatorname{Unif}\left[\frac{50N_2}{S+3},\frac{50N_2+50N_2+3(1+N_1)}{S+3}\right],\qquad
x=0:\quad\operatorname{Unif}\left[\frac{50N_2+3N_1}{S+1},1\right]. x = 2 : Unif [ 0 , S + 50 50 ( 1 + N 2 ) ] , x = 1 : Unif [ S + 3 50 N 2 , S + 3 50 N 2 + 50 N 2 + 3 ( 1 + N 1 ) ] , x = 0 : Unif [ S + 1 50 N 2 + 3 N 1 , 1 ] .
Formula evaluated. The certified value was computed by the checker, i.e. by (20) with ∣ Z ∣ = ∣ Z ′ ∣ = 3 |\mathcal{Z}|=|\mathcal{Z}'|=3 ∣ Z ∣ = ∣ Z ′ ∣ = 3 : 3 5 3^5 3 5 ordered calibration sequences times 3 2 3^2 3 2 ordered test sequences, each with the exact box integral over ( U 1 , U 2 ) (U_1,U_2) ( U 1 , U 2 ) and F D P = 1 { R ≠ ∅ } \mathrm{FDP}=1\{R\neq\varnothing\} FDP = 1 { R = ∅ } .
Analytic form (m = 2 m = 2 m = 2 ). The following two lemmas reduce the FDR of an m = 2 m = 2 m = 2 global-null instance to a sum over the count vectors ( N 0 , N 1 , N 2 ) (N_0, N_1, N_2) ( N 0 , N 1 , N 2 ) and show what an excess over q q q means.
Lemma 86 (Two tests, global null). Let m = 2 m = 2 m = 2 , let both test units be null, and let each p j p_j p j be exactly Unif ( 0 , 1 ) \operatorname{Unif}(0,1) Unif ( 0 , 1 ) . Then BH at level q q q has
FDR = FWER = q + P ( p 1 , p 2 ∈ ( q / 2 , q ] ) − P ( p 1 , p 2 ∈ ( 0 , q / 2 ] ) . \operatorname{FDR} = \operatorname{FWER} = q + \mathbb{P}(p_1,p_2 \in(q/2,q]) - \mathbb{P}(p_1,p_2 \in(0,q/2]). FDR = FWER = q + P ( p 1 , p 2 ∈ ( q /2 , q ]) − P ( p 1 , p 2 ∈ ( 0 , q /2 ]) .
Proof. BH rejects something iff min ( p 1 , p 2 ) ≤ q / 2 \min(p_1,p_2) \le q/2 min ( p 1 , p 2 ) ≤ q /2 or max ( p 1 , p 2 ) ≤ q \max(p_1,p_2) \le q max ( p 1 , p 2 ) ≤ q . Write A j = { p j ≤ q / 2 } A_j = \{p_j \le q/2\} A j = { p j ≤ q /2 } and B j = { q / 2 < p j ≤ q } B_j = \{q/2 < p_j \le q\} B j = { q /2 < p j ≤ q } . The rejection event is ( A 1 ∪ A 2 ) ∪ ( B 1 ∩ B 2 ) (A_1 \cup A_2) \cup(B_1 \cap B_2) ( A 1 ∪ A 2 ) ∪ ( B 1 ∩ B 2 ) , and the two pieces are disjoint. Exact uniformity gives P ( A 1 ∪ A 2 ) = q / 2 + q / 2 − P ( A 1 ∩ A 2 ) \mathbb{P}(A_1 \cup A_2) = q/2 + q/2 - \mathbb{P}(A_1 \cap A_2) P ( A 1 ∪ A 2 ) = q /2 + q /2 − P ( A 1 ∩ A 2 ) .
For independent uniform p-values both probabilities equal q 2 / 4 q^2/4 q 2 /4 , so the FWER is exactly q q q ; BH overshoots q q q exactly when “both moderate” coincidences are more likely than “both very small” ones.
Lemma 87 (Conditioning on the calibration data). In the setting of Lemma 86, suppose in addition that p 1 , p 2 p_1,p_2 p 1 , p 2 are i.i.d. given D D D , and let a ( D ) : = P ( p 1 ≤ q / 2 ∣ D ) a(D) := \mathbb{P}(p_1 \le q/2 \mid D) a ( D ) := P ( p 1 ≤ q /2 ∣ D ) , b ( D ) : = P ( q / 2 < p 1 ≤ q ∣ D ) b(D) := \mathbb{P}(q/2 < p_1 \le q \mid D) b ( D ) := P ( q /2 < p 1 ≤ q ∣ D ) . Then E a ( D ) = E b ( D ) = q / 2 \mathbb{E}\,a(D) = \mathbb{E}\,b(D) = q/2 E a ( D ) = E b ( D ) = q /2 and
FDR − q = E [ b ( D ) 2 ] − E [ a ( D ) 2 ] = Var b ( D ) − Var a ( D ) . \operatorname{FDR} - q = \mathbb{E}[b(D)^2] - \mathbb{E}[a(D)^2] = \operatorname{Var} b(D) - \operatorname{Var} a(D). FDR − q = E [ b ( D ) 2 ] − E [ a ( D ) 2 ] = Var b ( D ) − Var a ( D ) .
Proof. By conditional independence, P ( p 1 , p 2 ∈ ( 0 , q / 2 ] ) = E [ a ( D ) 2 ] \mathbb{P}(p_1,p_2 \in(0,q/2]) = \mathbb{E}[a(D)^2] P ( p 1 , p 2 ∈ ( 0 , q /2 ]) = E [ a ( D ) 2 ] and P ( p 1 , p 2 ∈ ( q / 2 , q ] ) = E [ b ( D ) 2 ] \mathbb{P}(p_1,p_2 \in(q/2,q]) = \mathbb{E}[b(D)^2] P ( p 1 , p 2 ∈ ( q /2 , q ]) = E [ b ( D ) 2 ] . Exact uniformity gives E a ( D ) = P ( p 1 ≤ q / 2 ) = q / 2 \mathbb{E}\,a(D) = \mathbb{P}(p_1 \le q/2) = q/2 E a ( D ) = P ( p 1 ≤ q /2 ) = q /2 and E b ( D ) = q / 2 \mathbb{E}\,b(D) = q/2 E b ( D ) = q /2 . Substitute into Lemma 86.
Instance (i) satisfies both lemmas: every unit is null; the p-values are oracle p-values, hence exactly uniform by Lemma 1; and they are i.i.d. given D D D by (b). With the per-cell laws above, a ( D ) = ∑ x Q ( X = x ) G x ( 2 / 5 ) a(D) = \sum_x Q(X = x)G_x(2/5) a ( D ) = ∑ x Q ( X = x ) G x ( 2/5 ) and b ( D ) = ∑ x Q ( X = x ) ( G x ( 4 / 5 ) − G x ( 2 / 5 ) ) b(D) = \sum_x Q(X = x)(G_x(4/5) - G_x(2/5)) b ( D ) = ∑ x Q ( X = x ) ( G x ( 4/5 ) − G x ( 2/5 )) , so
FDR = 4 5 + ∑ N 0 + N 1 + N 2 = 5 P ( D ) ( b ( D ) 2 − a ( D ) 2 ) , (22) \operatorname{FDR} = \frac{4}{5} + \sum_{N_0+N_1+N_2=5} \mathbb{P}(D)\bigl(b(D)^2-a(D)^2\bigr), \tag*{(22)} FDR = 5 4 + N 0 + N 1 + N 2 = 5 ∑ P ( D ) ( b ( D ) 2 − a ( D ) 2 ) , (22)
a sum of 21 rational terms. The interactions between the two test p-values pass only through the shared calibration data, and Lemma 87 says that BH exceeds q q q exactly when the calibration draw moves the chance of a “moderate” p-value, b ( D ) b(D) b ( D ) , more than it moves the chance of a “very small” one, a ( D ) a(D) a ( D ) . For instance (i) the certified value below is therefore equivalent to Var b ( D ) − Var a ( D ) = 102507849 / 492032000000 > 0 \operatorname{Var} b(D)-\operatorname{Var} a(D)=102507849/492032000000>0 Var b ( D ) − Var a ( D ) = 102507849/492032000000 > 0 .
Value and certification. The checker returns, in exact rational arithmetic,
FDR = 393728107849 492032000000 ≈ 0.8002083 , FDR − q = 102507849 492032000000 ≈ 2.083 × 10 − 4 , FDR / q ≈ 1.00026 , \operatorname{FDR} = \frac{393728107849}{492032000000} \approx0.8002083,\qquad\operatorname{FDR}-q = \frac{102507849}{492032000000} \approx2.083 \times10^{-4},\qquad\operatorname{FDR}/q \approx1.00026, FDR = 492032000000 393728107849 ≈ 0.8002083 , FDR − q = 492032000000 102507849 ≈ 2.083 × 1 0 − 4 , FDR / q ≈ 1.00026 ,
so FDR > q \operatorname{FDR} > q FDR > q . The same program verified exactly that P ( p 1 ≤ t ) = t \mathbb{P}(p_1 \le t) = t P ( p 1 ≤ t ) = t at t = 2 / 5 t = 2/5 t = 2/5 and t = 4 / 5 t = 4/5 t = 4/5 , which a missing own weight or a wrong tie term in eq. (4) would break. A second program, written separately and sharing no code with the checker, evaluates the 21-term sum (22) from the per-cell laws above, in exact rational arithmetic. It returns the same fraction, with total mass exactly 1 and E a ( D ) = E b ( D ) = 2 / 5 \mathbb{E}\,a(D) = \mathbb{E}\,b(D) = 2/5 E a ( D ) = E b ( D ) = 2/5 exactly. It relies on the per-cell laws and on Lemma 87; the checker relies on neither. The checker itself was validated on other instances: it agrees exactly with the first strict-selection engine on ten small instances (Section F.3), and on a smaller global-null instance (n = 3 n = 3 n = 3 , m = 2 m = 2 m = 2 ) it agrees with a separate search engine in floating point and with a direct Monte Carlo simulation.
Instance (ii): strict selection, n = 80 n=80 n = 80 , m = 8 m=8 m = 8 , q = 1 / 10 q=1/10 q = 1/10
Configuration. Three covariate values: a light value ℓ \ell ℓ and two heavy values h 1 , h 2 h_1,h_2 h 1 , h 2 . The score is V ( x , y ) = y V(x,y)=y V ( x , y ) = y (monotone), and the threshold is a function of the covariate.
Light covariate ℓ \ell ℓ . P-mass π ℓ = 1 − π 1 − π 2 \pi_\ell= 1-\pi_1-\pi_2 π ℓ = 1 − π 1 − π 2 , weight 1; Y ∈ { 0 , 1 } Y \in\{0,1\} Y ∈ { 0 , 1 } with P ( Y = 0 ∣ X = ℓ ) = ν \mathbb{P}(Y=0 \mid X=\ell)=\nu P ( Y = 0 ∣ X = ℓ ) = ν ; threshold c ( ℓ ) = 1 / 2 c(\ell)=1/2 c ( ℓ ) = 1/2 . So V ^ = 1 / 2 \widehat{V}=1/2 V = 1/2 for every light test unit, and a light unit is null iff Y = 0 Y=0 Y = 0 .
Heavy covariate h 1 h_1 h 1 : P P P -mass π 1 \pi_1 π 1 , weight W 1 W_1 W 1 , Y = − 2 Y=-2 Y = − 2 deterministically, threshold c ( h 1 ) = − 2 c(h_1)=-2 c ( h 1 ) = − 2 (= Y =Y = Y , null).
Heavy covariate h 2 h_2 h 2 : P P P -mass π 2 \pi_2 π 2 , weight W 2 W_2 W 2 , Y = − 1 Y=-1 Y = − 1 deterministically, threshold c ( h 2 ) = − 1 c(h_2)=-1 c ( h 2 ) = − 1 (= Y =Y = Y , null).
The certified parameters are
ν = 1 25 , π 1 = 3 10 , π 2 = 7 400 , W 1 = 1 22 , W 2 = 137 22 . \nu=\frac{1}{25},\qquad\pi_1=\frac{3}{10},\qquad\pi_2=\frac{7}{400},\qquad W_1=\frac{1}{22},\qquad W_2=\frac{137}{22}. ν = 25 1 , π 1 = 10 3 , π 2 = 400 7 , W 1 = 22 1 , W 2 = 22 137 .
so π ℓ = 273 / 400 \pi_\ell=273/400 π ℓ = 273/400 , E P w = 1417 / 1760 \mathbb{E}_P w=1417/1760 E P w = 1417/1760 , and Q ( X = ℓ ) = 462 / 545 Q(X=\ell)=462/545 Q ( X = ℓ ) = 462/545 , Q ( X = h 1 ) = 24 / 1417 Q(X=h_1)=24/1417 Q ( X = h 1 ) = 24/1417 , Q ( X = h 2 ) = 959 / 7085 Q(X=h_2)=959/7085 Q ( X = h 2 ) = 959/7085 . The calibration support points ( ℓ , 0 ) , ( ℓ , 1 ) , ( h 1 , − 2 ) , ( h 2 , − 1 ) (\ell,0),(\ell,1),(h_1,-2),(h_2,-1) ( ℓ , 0 ) , ( ℓ , 1 ) , ( h 1 , − 2 ) , ( h 2 , − 1 ) have P P P -masses 273 / 10000 , 819 / 1250 , 3 / 10 , 7 / 400 273/10000,819/1250,3/10,7/400 273/10000 , 819/1250 , 3/10 , 7/400 . Multiplying all weights by 22 (which leaves every p j p_j p j unchanged) gives w ( h 1 ) = 1 w(h_1)=1 w ( h 1 ) = 1 , w ( ℓ ) = 22 w(\ell)=22 w ( ℓ ) = 22 , w ( h 2 ) = 137 w(h_2)=137 w ( h 2 ) = 137 , so 1 ≤ w ≤ B = 137 1\le w\le B=137 1 ≤ w ≤ B = 137 . The threshold is independent of Y Y Y given X X X (it is a function of X X X ), and Y ∣ X Y\mid X Y ∣ X is non-degenerate on the light covariate. BH’s grid is t k = k / 80 t_k=k/80 t k = k /80 , k = 1 , … , 8 k=1,\ldots,8 k = 1 , … , 8 . The light units play the role of the block of non-null “decoys” with a shared threshold in Mechanism 2 (§ 3 ).
Per-type laws. Let ( L 0 , L 1 , H 1 , H 2 ) (L_0,L_1,H_1,H_2) ( L 0 , L 1 , H 1 , H 2 ) be the calibration counts of the four support points, so
P ( D ) = n ! L 0 ! L 1 ! H 1 ! H 2 ! ( π ℓ ν ) L 0 ( π ℓ ( 1 − ν ) ) L 1 π 1 H 1 π 2 H 2 , \mathbb{P}(D)=\frac{n!}{L_0!L_1!H_1!H_2!}(\pi_\ell\nu)^{L_0}\bigl(\pi_\ell(1-\nu)\bigr)^{L_1}\pi_1^{H_1}\pi_2^{H_2}, P ( D ) = L 0 ! L 1 ! H 1 ! H 2 ! n ! ( π ℓ ν ) L 0 ( π ℓ ( 1 − ν ) ) L 1 π 1 H 1 π 2 H 2 ,
S = L 0 + L 1 + W 1 H 1 + W 2 H 2 S=L_0+L_1+W_1H_1+W_2H_2 S = L 0 + L 1 + W 1 H 1 + W 2 H 2 , and S < : = L 0 + W 1 H 1 + W 2 H 2 S_{<}:=L_0+W_1H_1+W_2H_2 S < := L 0 + W 1 H 1 + W 2 H 2 is the calibration weight with score below 1 / 2 1/2 1/2 . The calibration scores are − 2 -2 − 2 (h 1 h_1 h 1 ), − 1 -1 − 1 (h 2 h_2 h 2 ), and 0, 1 (light); the test scores are V ^ = 1 / 2 , − 2 , − 1 \widehat{V}=1/2,-2,-1 V = 1/2 , − 2 , − 1 for ℓ , h 1 , h 2 \ell,h_1,h_2 ℓ , h 1 , h 2 . So ((17) ) gives, given D D D ,
ℓ : Unif [ S S + 1 , S < + 1 S + 1 ] , h 1 : Unif [ 0 , W 1 ( 1 + H 1 ) S + W 1 ] , h 2 : Unif [ W 1 H 1 S + W 2 , W 1 H 1 + W 2 ( 1 + H 2 ) S + W 2 ] . \ell:\ \operatorname{Unif}\left[\frac{S}{S+1},\frac{S_{<}+1}{S+1}\right],\qquad h_1:\ \operatorname{Unif}\left[0,\frac{W_1(1+H_1)}{S+W_1}\right],\qquad h_2:\ \operatorname{Unif}\left[\frac{W_1H_1}{S+W_2},\frac{W_1H_1+W_2(1+H_2)}{S+W_2}\right]. ℓ : Unif [ S + 1 S , S + 1 S < + 1 ] , h 1 : Unif [ 0 , S + W 1 W 1 ( 1 + H 1 ) ] , h 2 : Unif [ S + W 2 W 1 H 1 , S + W 2 W 1 H 1 + W 2 ( 1 + H 2 ) ] .
With G ℓ , G 1 , G 2 G_\ell,G_1,G_2 G ℓ , G 1 , G 2 the corresponding CDFs, and since a light unit’s null indicator 1 { Y = 0 } \mathbb{1}\{Y=0\} 1 { Y = 0 } does not enter its p-value (V ^ = 1 / 2 \widehat{V}=1/2 V = 1/2 does not involve Y Y Y ),
F D ( t ) = Q ( ℓ ) G ℓ ( t ) + Q ( h 1 ) G 1 ( t ) + Q ( h 2 ) G 2 ( t ) , A k ( D ) = ν Q ( ℓ ) G ℓ ( t k ) + Q ( h 1 ) G 1 ( t k ) + Q ( h 2 ) G 2 ( t k ) . F_D(t)=Q(\ell)G_\ell(t)+Q(h_1)G_1(t)+Q(h_2)G_2(t),\qquad A_k(D)=\nu Q(\ell)G_\ell(t_k)+Q(h_1)G_1(t_k)+Q(h_2)G_2(t_k). F D ( t ) = Q ( ℓ ) G ℓ ( t ) + Q ( h 1 ) G 1 ( t ) + Q ( h 2 ) G 2 ( t ) , A k ( D ) = ν Q ( ℓ ) G ℓ ( t k ) + Q ( h 1 ) G 1 ( t k ) + Q ( h 2 ) G 2 ( t k ) .
Formula evaluated. F D R = ∑ D P ( D ) E [ F D P ∣ D ] \mathrm{FDR}=\sum_D\mathbb{P}(D)\mathbb{E}[\mathrm{FDP}\mid D] FDR = ∑ D P ( D ) E [ FDP ∣ D ] , summed over all ( n + 3 3 ) = 91 , 881 \binom{n+3}{3}=91,881 ( 3 n + 3 ) = 91 , 881 count vectors ( L 0 , L 1 , H 1 , H 2 ) (L_0,L_1,H_1,H_2) ( L 0 , L 1 , H 1 , H 2 ) with L 0 + L 1 + H 1 + H 2 = 80 L_0+L_1+H_1+H_2=80 L 0 + L 1 + H 1 + H 2 = 80 , by two engines:
Engine 1 uses the leave-one-out form ((19) ), with Φ m − k , k \Phi_{m-k,k} Φ m − k , k computed by the dynamic programme for ((18) );
Engine 2 uses the cutoff form ((20) ). It was written separately, shares no code with engine 1, and uses a different identity for E [ F D P ∣ D ] \mathbb{E}[\mathrm{FDP}\mid D] E [ FDP ∣ D ] .
Both run in exact rational arithmetic, and each also checks that ∑ D P ( D ) = 1 \sum_D\mathbb{P}(D)=1 ∑ D P ( D ) = 1 exactly.
Value. Both engines return the same rational number, in lowest terms F D R = N 80 / ( d 80 ⋅ 10 336 ) \mathrm{FDR}=N_{80}/(d_{80}\cdot10^{336}) FDR = N 80 / ( d 80 ⋅ 1 0 336 ) , with numerator (569 digits)
N 80 = 1044564337482314073024813881132458603392898299247231873373586760314483547300861 278232133158150616583312427966146342866115850113946318950238406798026425384486384976721839 192108426659992504290584914773281822107585438012374954023727760784442571591013318019528009 22047391494467878328904548630158065817622850751867151481525050488842759054211461964035123 2095619518082666595783265633987818990209252306760395372182682224706340240456472121398681223 260710592965723695533482807042834130826415589365869765566866028125230192845249339195544581 13681596294344224828236615786831147681427. \begin{aligned}
N_{80}={}&1044564337482314073024813881132458603392898299247231873373586760314483547300861\\
&278232133158150616583312427966146342866115850113946318950238406798026425384486384976721839\\
&192108426659992504290584914773281822107585438012374954023727760784442571591013318019528009\\
&22047391494467878328904548630158065817622850751867151481525050488842759054211461964035123\\
&2095619518082666595783265633987818990209252306760395372182682224706340240456472121398681223\\
&260710592965723695533482807042834130826415589365869765566866028125230192845249339195544581\\
&13681596294344224828236615786831147681427.
\end{aligned} N 80 = 1044564337482314073024813881132458603392898299247231873373586760314483547300861 278232133158150616583312427966146342866115850113946318950238406798026425384486384976721839 192108426659992504290584914773281822107585438012374954023727760784442571591013318019528009 22047391494467878328904548630158065817622850751867151481525050488842759054211461964035123 2095619518082666595783265633987818990209252306760395372182682224706340240456472121398681223 260710592965723695533482807042834130826415589365869765566866028125230192845249339195544581 13681596294344224828236615786831147681427.
and, in the denominator, (233 digits)
d 80 = 9196331503758140790804144629877636764967009302451975267024776468369927234919750918 05039118919156747741225486894368138182655888546834987883541747593790952158140149085109841 113624729385338618381517587244550929574460897754643035062272. \begin{aligned}
d_{80}={}&9196331503758140790804144629877636764967009302451975267024776468369927234919750918\\
&05039118919156747741225486894368138182655888546834987883541747593790952158140149085109841\\
&113624729385338618381517587244550929574460897754643035062272.
\end{aligned} d 80 = 9196331503758140790804144629877636764967009302451975267024776468369927234919750918 05039118919156747741225486894368138182655888546834987883541747593790952158140149085109841 113624729385338618381517587244550929574460897754643035062272.
(digits grouped in tens from the right). Numerically F D R = 0.1135848938 … \mathrm{FDR}=0.1135848938\ldots FDR = 0.1135848938 … , F D R / q = 1.1358489383 … \mathrm{FDR}/q=1.1358489383\ldots FDR / q = 1.1358489383 … , and F D R − q = 0.0135848938 … > 0 \mathrm{FDR}-q=0.0135848938\ldots>0 FDR − q = 0.0135848938 … > 0 .
Certification. The following checks were run.
Two exact engines. Engines 1 and 2 return exactly the same rational number at n = 80 n = 80 n = 80 (above) and on the certified n = 40 n = 40 n = 40 instance of Remark 88. They were also compared at n = 3 , 6 , 10 , 20 n = 3, 6, 10, 20 n = 3 , 6 , 10 , 20 (m = 8 m = 8 m = 8 , q = 1 / 10 q = 1/10 q = 1/10 ) on five parameter sets ( ν , π 1 , π 2 , W 1 , W 2 ) (\nu, \pi_1, \pi_2, W_1, W_2) ( ν , π 1 , π 2 , W 1 , W 2 ) : the three certified sets of Remark 88, and ( 1 / 5 , 1 / 5 , 1 / 5 , 3 / 2 , 4 ) (1/5, 1/5, 1/5, 3/2, 4) ( 1/5 , 1/5 , 1/5 , 3/2 , 4 ) and ( 3 / 10 , 1 / 10 , 3 / 10 , 17 / 9 , 2 ) (3/10, 1/10, 3/10, 17/9, 2) ( 3/10 , 1/10 , 3/10 , 17/9 , 2 ) , drawn pseudo-randomly with a fixed seed. On all 20 instances the two engines return the same rational number, and each finds total mass exactly 1. At n = 20 n = 20 n = 20 with the certified n = 20 n = 20 n = 20 parameters both give FDR/q = 1.0694867 … q = 1.0694867\ldots q = 1.0694867 … , the value in Remark 88.
Independent checker. Engine 1 equals the checker of Section F.1(d), as exact rationals, on ten small instances of the same model: ( n , m ) ∈ { ( 2 , 2 ) , ( 3 , 2 ) , ( 2 , 3 ) , ( 3 , 3 ) , ( 4 , 2 ) } (n, m) \in\{(2, 2), (3, 2), (2, 3), (3, 3), (4, 2)\} ( n , m ) ∈ {( 2 , 2 ) , ( 3 , 2 ) , ( 2 , 3 ) , ( 3 , 3 ) , ( 4 , 2 )} , two each, with q ∈ { 1 / 5 , 3 / 10 } q \in\{1/5, 3/10\} q ∈ { 1/5 , 3/10 } and the other parameters pseudo-random rationals. Since the checker evaluates eq. (4) and BH directly, this tests the model reduction (the per-type laws, (b), and (18)), not only the arithmetic.
Direct simulation. A separate simulation of the actual units (eq. (4) and BH, no use of the reductions) at the certified n = 80 n = 80 n = 80 parameters, 2 × 10 5 2 \times10^5 2 × 1 0 5 replications, gives FDR/q = 1.1338 ± 0.0062 q = 1.1338 \pm0.0062 q = 1.1338 ± 0.0062 , against the exact 1.13585. The formula (18), evaluated in floating point, was also compared with this simulation on two other configurations (differences at most 1.4 standard errors).
What this does not cover: the checker was not run on the certified n = 80 n = 80 n = 80 instance itself (it would enumerate 4 80 ⋅ 4 8 4^{80} \cdot4^{8} 4 80 ⋅ 4 8 ordered sequences), and both engines rely on the same per-type laws above, which are tested against the checker only on small instances.
Remark 88 (The same construction at smaller n n n ). With the parameters re-chosen for each n n n (m = 8 m = 8 m = 8 , q = 1 / 10 q = 1/10 q = 1/10 ), the same model was certified in exact arithmetic at n = 20 n = 20 n = 20 and n = 40 n = 40 n = 40 :
Table 88.
At n = 20 n = 20 n = 20 the sum runs over all 1,771 count vectors, with total mass exactly 1. At n = 40 n = 40 n = 40 the two engines agree exactly. The simulations use 2 × 10 5 2 \times10^5 2 × 1 0 5 replications. The excess grows with n n n : 6.9%, 10.3% and 13.6%.
Proof of the theorem
Proof of Theorem 5. The proof is by exact computation. Both instances satisfy Jin & Candès [12 ] ’s assumptions: covariate shift with a known weight function bounded above and below (B = 50 B = 50 B = 50 and B = 137 B = 137 B = 137 ); i.i.d. test units independent of the calibration data; thresholds that are fixed functions of the test unit’s own covariate (constant in (i)), so that their independence condition holds; and a score monotone in y y y . In instance (i) every test unit is null with c = Y c = Y c = Y (global null). In instance (ii) the threshold is independent of Y Y Y given X X X and Y ∣ X Y \mid X Y ∣ X is non-degenerate on the light covariate (strict selection). (i) By Section F.2, FDR = 393728107849 / 49203200000 > 4 / 5 = q = 393728107849/49203200000 > 4/5 = q = 393728107849/49203200000 > 4/5 = q . (ii) By Section F.3, FDR = N 80 / ( d 80 ⋅ 10 336 ) = 1.13585 q = N_{80}/(d_{80} \cdot10^{336}) = 1.13585q = N 80 / ( d 80 ⋅ 1 0 336 ) = 1.13585 q to five decimals, > q > q > q . In both cases the comparison with q q q is between exact rationals. □
Use of AI tools
Large language models were used throughout the research process, including to help write the code for the numerical computations and to revise the manuscript. We checked all results, code and claims, and take full responsibility for the paper.