Introduction
Density estimation given independent and identically distributed (i.i.d.) observations from a distribution P P P is of key importance in statistics, where downstream tasks such as prediction or functional estimation depend on having an accurate density estimate. As datasets grow in size, it is important to identify flexible methods which remain scalable in high dimensions or with large sample sizes, while providing uncertainty quantification.
Bayesian nonparametric methods are popular for density estimation due to their flexibility, regularization of the density estimates, and built-in uncertainty quantification. A classic example is the Dirichlet process mixture model (DPMM) [9 , 20 ] which models the density as an infinite mixture model, and typically relies on Markov chain Monte Carlo (MCMC) for inference. Besides mixture modeling, another popular class of methods is based on Pólya trees (PT; Lavine [ [17 ] ) which recursively partition the sample space. While PTs are conjugate and computationally scalable, their performance relies on a fixed partition which limits adaptivity, leading to extensions such as Optional Pólya Trees (OPT; Wong and Ma [ [35 ] ) which allow adaptive choice of the splitting variable and stopping. However, exact posterior inference for OPTs requires recursive marginalization over possible partition trees, making computation prohibitively expensive in higher dimensions. More recent extensions, such as hidden Markov Pólya trees (HMPT; Awaya and Ma [ [2 ] ), additionally allow adaptive splitting locations and employ sequential Monte Carlo (SMC) to approximate the posterior over partitions to improve scalability in high dimensions.
While the more adaptive Bayesian nonparametric methods offer considerable flexibility, they face various computational challenges due to their reliance on MCMC, SMC or expensive recursive marginalization. In addition to their computational demands, existing Bayesian nonparametric methods often require careful specification of prior hyperparameters and algorithm settings, which can influence both estimation accuracy and computational efficiency. Outside the context of density estimation, but motivated by similar challenges in computation and hyperparameter tuning, recent work has revisited the Bayesian bootstrap [32 , 24 ] as a means for default and scalable nonparametric inference on Euclidean functionals. Fong et al. [ [11 ] elicit a simple Dirichlet process (DP) prior on the probability measure and leverage the conjugacy of the DP to obtain a highly scalable posterior bootstrap algorithm for inference on parameters which does not require MCMC. Another key advantage is that only a scalar prior count hyperparameter and base measure are required to be specified. The scalability and ease of the posterior bootstrap have led to applications in robust inference [7 ] , high-dimensional regression [25 , 26 ] , and the use of generative models for prior construction [27 ] . Connections to the Pólya urn have also spurred recent advances in predictive Bayes [13 , 12 ] . However, due to the reliance on the DP, the posterior bootstrap cannot naturally handle the case where the target functional requires a density, or the density itself is of interest.
Our contribution
In this work, we follow in the spirit of the posterior bootstrap, and consider a simple default prior on the density by smoothing a DP over a partition of the sample space, yielding a closed-form posterior we term the partition posterior bootstrap (PPB). The key to maintaining scalability is to treat the partition of the sample space as a nonparametric hyperparameter, whose efficient selection forms the focus of the paper. This avoids the need for sampling or integration over nonparametric latent structures, such as the mixture allocations in DPMMs or the partitions of the sample space in OPTs and HMPTs. Our approach is a natural generalization of Bayesian histogram approaches as introduced in Hjort [1996], where Dirichlet conjugacy is also exploited. In particular, Simensen et al. [ [34 ] proposed a univariate Bayesian irregular histogram estimator which adaptively selects the number and locations of bins through approximate maximum a posteriori model selection, using a combination of search heuristics and dynamic programming. However, their optimization procedure exploits the natural ordering of intervals in one dimension and does not readily extend to multivariate settings.
Our main methodological contribution is a scalable selection procedure for the partition: we exploit a natural recursive factorization of the marginal likelihood under the smoothed DP prior, yielding closed-form Bayes factors for local splitting decisions. This motivates a deterministic greedy empirical Bayes (GEB) algorithm that recursively constructs a binary tree partition of the space. At each node, we evaluate the Bayes factors associated with potential splitting variables and locations, selecting the split with the largest Bayes factor and stopping if none exceeds one. Thus, both the partition structure and stopping decisions are automatically determined through local Bayesian model comparisons. Crucially, our GEB algorithm remains computationally scalable in higher dimensions and with large sample sizes, and is orders of magnitude faster than competing Bayesian methods while maintaining excellent performance in practice.
Our main theoretical contribution is to establish guarantees for the PPB posterior based on the partition actually selected by the GEB algorithm. This contrasts with existing results which study either globally optimal estimators (sieve MLE or MAP; [19 , 34 ] or full posteriors over partitions [Castillo and Randrianarisoa, 2022, Liu et al. [19 ] , both of which are computationally challenging or infeasible to obtain in higher dimensions. Our results can also be viewed as extending the asymptotic theory of empirical Bayes [29 , 30 ] to the setting of data-adaptive recursive partitions. To handle the greedy nature of our algorithm, we introduce a family of admissible trees and show that the GEB partition belongs to this family with high probability, capturing all identifiable signals while avoiding unnecessary splitting in regions dominated by noise. We then establish contraction rates for the resulting PPB posterior, demonstrating adaptation to β \beta β -Hölder continuity for 0 < β ≤ 1 0 < \beta\le1 0 < β ≤ 1 , anisotropic smoothness, axis-aligned discontinuities, and variable sparsity.
We conclude the introduction with a visualization of the partition obtained by our GEB algorithm in a simple simulation.
Example 1 (GEB partition). Consider the two-dimensional density formed as a product of independent marginals, each equal to the mixture 0.5 0.5 0.5 Beta( 2 , 2 ) + 0.5 (2, 2) + 0.5 ( 2 , 2 ) + 0.5 Uniform( 0 , 1 ) (0, 1) ( 0 , 1 ) . Figure 1 compares the true density and three histogram estimates based on n = 100,000 n = 100{,}000 n = 100 , 000 samples and different partitions. Panel (b) shows a standard equal-width histogram with a 7 × 7 = 49 7 \times7 = 49 7 × 7 = 49 grid, which struggles to capture the density’s geometry. Panel (c) depicts a partition guided by the true density’s contour level sets, which acts as an ideal but infeasible benchmark. Panel (d) then illustrates the GEB partition, which chooses 49 rectangular bins and approximates the contours of the true density well. The estimation quality is verified in Table 1 , where the average test log-likelihood on held-out data is noticeably higher for the GEB partition than for the equal-width partition and close to the contour partition. Importantly, the number of bins is chosen automatically, and the total run-time is only 0.1 seconds.
Figure 1. Density estimates for a two-dimensional product mixture.
Table 1. Held-out average test log-likelihood.
We now outline the structure of the paper. § 2 introduces the smoothed DP prior and the resulting PPB algorithm. § 3 then addresses the central challenge of choosing the partition with our GEB algorithm, and § 4 establishes the theoretical guarantees of the approach. Finally, § 5 presents simulation and real-data experiments demonstrating the method’s performance and substantial computational savings over existing Bayesian nonparametric alternatives. We conclude with a discussion in § 6 .
Partition posterior bootstrap
Let P P P be the data-generating distribution with Lebesgue density p p p from which we observe n n n i.i.d. observations x 1 : n = ( x 1 , … , x n ) x_{1:n} = (x_1, \ldots, x_n) x 1 : n = ( x 1 , … , x n ) where x i ∈ A ⊂ R D x_i \in\mathcal{A} \subset\mathbb{R}^{D} x i ∈ A ⊂ R D . For a probability measure μ \mu μ and measurable sets A A A , B B B with μ ( A ) > 0 \mu(A) > 0 μ ( A ) > 0 , we adopt the shorthand μ A : = μ ( A ) \mu_A := \mu(A) μ A := μ ( A ) and μ B ∣ A : = μ A ∩ B / μ A \mu_{B \mid A} := \mu_{A \cap B}/\mu_A μ B ∣ A := μ A ∩ B / μ A throughout this paper. Let P = { A 1 , … , A M } \mathcal{P} = \{A_1, \ldots, A_M\} P = { A 1 , … , A M } denote a measurable partition of the sample space A \mathcal{A} A , and let N A = ∑ i = 1 n 1 ( x i ∈ A ) N_A = \sum_{i=1}^{n} \mathbb{1}(x_i \in A) N A = ∑ i = 1 n 1 ( x i ∈ A ) .
The posterior bootstrap begins by specifying the nonparametric prior DP ( α , H ) \operatorname{DP}(\alpha, H) DP ( α , H ) , where α \alpha α is the concentration parameter and H H H is a base measure representing our prior guess of P P P . It is well known that the posterior given x 1 : n x_{1:n} x 1 : n is another Dirichlet process. However, a draw from a DP is almost surely discrete, making it unsuitable for modeling a continuous density p p p directly. To address this, we consider augmenting the DP prior with a smoothing partition P \mathcal{P} P , which preserves tractability in posterior sampling and admits a natural marginal likelihood which we exploit in § 3 .
We now augment the DP prior of the posterior bootstrap with a partition P \mathcal{P} P , where we assume H A > 0 H_A > 0 H A > 0 for every A ∈ P A \in\mathcal{P} A ∈ P . Let H H H have a Lebesgue density h h h , and assume that P ≪ H P \ll H P ≪ H . We then construct our smoothed DP prior on the density p p p as
f ( x ) = ∑ A ∈ P G A h ( x ∣ A ) , G ∼ DP ( α , H ) , (1) f(x) = \sum_{A \in\mathcal{P}} G_A h(x \mid A), \qquad G \sim\operatorname{DP}(\alpha, H), \tag*{(1)} f ( x ) = A ∈ P ∑ G A h ( x ∣ A ) , G ∼ DP ( α , H ) , (1)
where h ( x ∣ A ) h(x \mid A) h ( x ∣ A ) is the conditional density of the form
h ( x ∣ A ) = 1 ( x ∈ A ) h ( x ) H A . h(x \mid A) = \mathbb{1}(x \in A) \frac{h(x)}{H_A}. h ( x ∣ A ) = 1 ( x ∈ A ) H A h ( x ) .
The partition P \mathcal{P} P thus controls the resolution at which the density f f f can depart from the base density, where the shape of h h h is retained locally within each A A A .
As in the posterior bootstrap, the underlying random probability measure is specified through a single DP ( α , H ) \operatorname{DP}(\alpha, H) DP ( α , H ) prior. The additional ingredient is the partition P \mathcal{P} P used to smooth this measure, which will be learned using a GEB algorithm in § 3 . Our prior still contains simple and interpretable elements, and another useful feature of our specific prior is that it preserves the base density as the prior mean for any choice of P \mathcal{P} P , that is
E [ f ( x ) ∣ P ] = ∑ A ∈ P H A h ( x ∣ A ) = h ( x ) . \mathbb{E}[f(x) \mid\mathcal{P}] = \sum_{A \in\mathcal{P}} H_A h(x \mid A) = h(x). E [ f ( x ) ∣ P ] = A ∈ P ∑ H A h ( x ∣ A ) = h ( x ) .
Our construction can be viewed as a natural extension of the Bayesian histogram formulations of Hjort [ [15 ] and Simensen et al. [ [34 ] by replacing the uniform density within each cell with the conditional base density h ( ⋅ ∣ A ) h(\cdot\mid A) h ( ⋅ ∣ A ) , giving the above prior mean property. When H H H is uniform, the construction coincides with the usual Dirichlet histogram model, and more generally our prior belongs to the class of smoothed Dirichlet process priors discussed by Hjort [1996]. The particular partition smoothing used here enables a simple posterior bootstrap algorithm for inference on the density or its functionals, which we outline in the next subsection. As we will see in § 3 , the above prior has a marginal likelihood structure that is particularly suitable for our GEB algorithm for the partition.
Posterior computation
Since ( G A : A ∈ P ) ∼ Dirichlet ( α H A : A ∈ P ) (G_A : A \in\mathcal{P}) \sim\operatorname{Dirichlet}(\alpha H_A : A \in\mathcal{P}) ( G A : A ∈ P ) ∼ Dirichlet ( α H A : A ∈ P ) , posterior inference conditioned on P \mathcal{P} P follows from well-known properties of Dirichlet conjugacy which is usually leveraged for Bayesian histograms. A draw from the posterior distribution of p p p given x 1 : n x_{1:n} x 1 : n can be represented as
f ( x ) = ∑ A ∈ P G ~ A h ( x ∣ A ) , G ~ ∼ DP ( α + n , α H + ∑ i = 1 n δ x i α + n ) . f(x) = \sum_{A \in\mathcal{P}} \widetilde{G}_A h(x \mid A), \qquad\widetilde{G} \sim\operatorname{DP}\left(\alpha+n,\frac{\alpha H+\sum_{i=1}^{n}\delta_{x_i}}{\alpha+n}\right). f ( x ) = A ∈ P ∑ G A h ( x ∣ A ) , G ∼ DP ( α + n , α + n α H + ∑ i = 1 n δ x i ) .
We choose the above representation for the posterior as it retains the smoothed DP interpretation. Unlike the posterior bootstrap, our prior induces a posterior predictive density and marginal likelihood. The posterior predictive density given P \mathcal{P} P , which is the posterior mean of f f f given P \mathcal{P} P , is simply a reweighted version of the base density h h h :
f ˉ ( x ) = ∑ A ∈ P G ˉ A h ( x ∣ A ) , G ˉ A = α H A + N A α + n . (2) \bar{f}(x) = \sum_{A \in\mathcal{P}} \bar{G}_A h(x \mid A), \qquad\bar{G}_A = \frac{\alpha H_A + N_A}{\alpha+n}. \tag*{(2)} f ˉ ( x ) = A ∈ P ∑ G ˉ A h ( x ∣ A ) , G ˉ A = α + n α H A + N A . (2)
The marginal likelihood for P \mathcal{P} P , which will be the main focus of § 3 , is then
M ( P ) = Γ ( α ) Γ ( α + n ) ∏ A ∈ P ( Γ ( α H A + N A ) Γ ( α H A ) ∏ i : x i ∈ A h ( x i ∣ A ) ) . (3) \mathcal{M}(\mathcal{P}) = \frac{\Gamma(\alpha)}{\Gamma(\alpha+n)} \prod_{A \in\mathcal{P}} \left(\frac{\Gamma(\alpha H_A+N_A)}{\Gamma(\alpha H_A)} \prod_{i:x_i \in A} h(x_i \mid A)\right). \tag*{(3)} M ( P ) = Γ ( α + n ) Γ ( α ) A ∈ P ∏ ( Γ ( α H A ) Γ ( α H A + N A ) i : x i ∈ A ∏ h ( x i ∣ A ) ) . (3)
Conditional on P \mathcal{P} P , posterior sampling involves a partition posterior bootstrap as outlined in Algorithm 1. Functionals with closed-form expressions can be evaluated directly from each density draw. For a risk minimizer θ ( f ) = arg min θ ∈ Θ ∫ ℓ ( x , θ ) f ( x ) d x \theta(f) = \arg\min_{\theta\in\Theta} \int\ell(x,\theta) f(x)\,\mathrm{d}x θ ( f ) = arg min θ ∈ Θ ∫ ℓ ( x , θ ) f ( x ) d x , the expected loss can instead be approximated by Monte Carlo through S − 1 ∑ s = 1 S ℓ ( x s , θ ) S^{-1}\sum_{s=1}^{S} \ell(x_s,\theta) S − 1 ∑ s = 1 S ℓ ( x s , θ ) , where x s x_s x s is drawn by first selecting a cell A A A with probability G A ( b ) G_A^{(b)} G A ( b ) and then drawing a sample from H ( ⋅ ∣ A ) H(\cdot\mid A) H ( ⋅ ∣ A ) .
Algorithm 1: Partition posterior bootstrap
Compute cell counts N A = ∑ i = 1 n 1 ( x i ∈ A ) N_A = \sum_{i=1}^{n} \mathbb{1}(x_i \in A) N A = ∑ i = 1 n 1 ( x i ∈ A ) for all A ∈ P A \in\mathcal{P} A ∈ P
for b ← 1 b \leftarrow1 b ← 1 to B B B do
Draw ( G A ( b ) : A ∈ P ) ∼ Dirichlet ( α H A + N A : A ∈ P ) (G_A^{(b)} : A \in\mathcal{P}) \sim\operatorname{Dirichlet}(\alpha H_A + N_A : A \in\mathcal{P}) ( G A ( b ) : A ∈ P ) ∼ Dirichlet ( α H A + N A : A ∈ P )
Set f ( b ) ( x ) = ∑ A ∈ P G A ( b ) h ( x ∣ A ) f^{(b)}(x) = \sum_{A \in\mathcal{P}} G_A^{(b)} h(x \mid A) f ( b ) ( x ) = ∑ A ∈ P G A ( b ) h ( x ∣ A )
Compute θ ( b ) = θ ( f ( b ) ) \theta^{(b)} = \theta(f^{(b)}) θ ( b ) = θ ( f ( b ) )
end
return { f ( 1 ) , … , f ( B ) } \{f^{(1)},\ldots,f^{(B)}\} { f ( 1 ) , … , f ( B ) } and { θ ( 1 ) , … , θ ( B ) } \{\theta^{(1)},\ldots,\theta^{(B)}\} { θ ( 1 ) , … , θ ( B ) }
Greedy empirical Bayes for partition selection
In this section, we view P \mathcal{P} P as a nonparametric structural hyperparameter to be selected using the marginal likelihood M ( P ) \mathcal{M}(\mathcal{P}) M ( P ) , which is a natural criterion due to connections to prediction [10 ] . Since the space of partitions is enormous, global optimization is infeasible in practice. We thus seek a greedy alternative, as the PPB marginal likelihood lends itself to a natural scalable algorithm based on recursive partitioning.
Split Bayes factors
We begin by studying the key quantity related to the marginal likelihood defined in (3) , which drives our greedy empirical Bayes algorithm. Consider a partition P \mathcal{P} P , and define the refined partition P ′ \mathcal{P}' P ′ which splits a cell A ∈ P A \in\mathcal{P} A ∈ P into disjoint cells A 0 , A 1 A_0,A_1 A 0 , A 1 so A = A 0 ∪ A 1 A = A_0 \cup A_1 A = A 0 ∪ A 1 , that is P ′ = P ∖ { A } ∪ { A 0 , A 1 } \mathcal{P}' = \mathcal{P} \setminus\{A\} \cup\{A_0,A_1\} P ′ = P ∖ { A } ∪ { A 0 , A 1 } . We now define the ratio of marginal likelihoods B A ( A 0 , A 1 ) : = M ( P ′ ) / M ( P ) \mathcal{B}_A(A_0,A_1) := \mathcal{M}(\mathcal{P}')/\mathcal{M}(\mathcal{P}) B A ( A 0 , A 1 ) := M ( P ′ ) / M ( P ) as the split Bayes factor at A A A , which measures the evidence in favor of splitting A A A into A 0 , A 1 A_0,A_1 A 0 , A 1 relative to leaving A A A unsplit. Crucially, the split Bayes factor satisfies the identity
B A ( A 0 , A 1 ) = B ( α H A 0 + N A 0 , α H A 1 + N A 1 ) B ( α H A 0 , α H A 1 ) ( 1 H A 0 ∣ A ) N A 0 ( 1 H A 1 ∣ A ) N A 1 , (4) \mathcal{B}_A(A_0,A_1) = \frac{B(\alpha H_{A_0}+N_{A_0},\alpha H_{A_1}+N_{A_1})}{B(\alpha H_{A_0},\alpha H_{A_1})}\left(\frac{1}{H_{A_0\mid A}}\right)^{N_{A_0}}\left(\frac{1}{H_{A_1\mid A}}\right)^{N_{A_1}}, \tag*{(4)} B A ( A 0 , A 1 ) = B ( α H A 0 , α H A 1 ) B ( α H A 0 + N A 0 , α H A 1 + N A 1 ) ( H A 0 ∣ A 1 ) N A 0 ( H A 1 ∣ A 1 ) N A 1 , (4)
where B ( h 1 , h 2 ) B(h_1,h_2) B ( h 1 , h 2 ) is the beta function. The above identity is the cornerstone of our greedy approach: B A ( A 0 , A 1 ) \mathcal{B}_A(A_0,A_1) B A ( A 0 , A 1 ) depends solely on the counts and base probabilities of the proposed child cells A 0 , A 1 A_0,A_1 A 0 , A 1 , and is unaffected by the observations or partition outside A A A . Furthermore, B A \mathcal{B}_A B A has a natural interpretation as a binomial Bayes factor: conditional on N A N_A N A , we are comparing a binomial model for N A 0 N_{A_0} N A 0 with a Beta( α H A 0 , α H A 1 ) (\alpha H_{A_0},\alpha H_{A_1}) ( α H A 0 , α H A 1 ) prior on the success probability against the null model where this probability is fixed at H A 0 ∣ A H_{A_0|A} H A 0 ∣ A .
Before moving on to the GEB algorithm, we illustrate how the above interpretable form of the split Bayes factor arises from our prior, and more specifically from well-known properties of the Dirichlet distribution. We can express the marginal likelihood as
M ( P ) = [ ∏ i = 1 n h ( x i ) ] E [ ∏ A ∈ P ( G A H A ) N A ] . \mathcal{M}(\mathcal{P})=\left[\prod_{i=1}^{n}h(x_i)\right]\mathbb{E}\left[\prod_{A\in\mathcal{P}}\left(\frac{G_A}{H_A}\right)^{N_A}\right]. M ( P ) = [ i = 1 ∏ n h ( x i ) ] E [ A ∈ P ∏ ( H A G A ) N A ] .
where the expectation is over ( G A : A ∈ P ) ∼ Dirichlet ( α H A : A ∈ P ) (G_A:A\in\mathcal{P})\sim\operatorname{Dirichlet}(\alpha H_A:A\in\mathcal{P}) ( G A : A ∈ P ) ∼ Dirichlet ( α H A : A ∈ P ) . Now the split of A A A into A 0 A_0 A 0 and A 1 A_1 A 1 would introduce the following term into the expectation:
∏ j = 0 1 ( G A j H A j ) N A j = ( G A H A ) N A ∏ j = 0 1 ( G A j ∣ A H A j ∣ A ) N A j . \prod_{j=0}^{1}\left(\frac{G_{A_j}}{H_{A_j}}\right)^{N_{A_j}}=\left(\frac{G_A}{H_A}\right)^{N_A}\prod_{j=0}^{1}\left(\frac{G_{A_j|A}}{H_{A_j|A}}\right)^{N_{A_j}}. j = 0 ∏ 1 ( H A j G A j ) N A j = ( H A G A ) N A j = 0 ∏ 1 ( H A j ∣ A G A j ∣ A ) N A j .
By the aggregation property, replacing G A 0 G_{A_0} G A 0 and G A 1 G_{A_1} G A 1 by their sum G A G_A G A recovers the original Dirichlet distribution on P \mathcal{P} P , since α H A 0 + α H A 1 = α H A \alpha H_{A_0}+\alpha H_{A_1}=\alpha H_A α H A 0 + α H A 1 = α H A . By neutrality, the conditional probabilities ( G A 0 ∣ A , G A 1 ∣ A ) (G_{A_0|A},G_{A_1|A}) ( G A 0 ∣ A , G A 1 ∣ A ) are independent of the aggregated vector ( G C : C ∈ P ) ∼ Dirichlet ( α H C : C ∈ P ) (G_C:C\in\mathcal{P})\sim\operatorname{Dirichlet}(\alpha H_C:C\in\mathcal{P}) ( G C : C ∈ P ) ∼ Dirichlet ( α H C : C ∈ P ) , with G A 0 ∣ A ∼ Beta ( α H A 0 , α H A 1 ) G_{A_0|A}\sim\operatorname{Beta}(\alpha H_{A_0},\alpha H_{A_1}) G A 0 ∣ A ∼ Beta ( α H A 0 , α H A 1 ) and G A 1 ∣ A = 1 − G A 0 ∣ A G_{A_1|A}=1-G_{A_0|A} G A 1 ∣ A = 1 − G A 0 ∣ A . Consequently, the factor ( G A / H A ) N A (G_A/H_A)^{N_A} ( G A / H A ) N A restores the contribution of A A A to M ( P ) \mathcal{M}(\mathcal{P}) M ( P ) , while the expectation of the remaining product factors out by independence, yielding the stated split Bayes factor.
Finally, we highlight some connections to related ideas in the PT literature. The leveraged independence property is closely connected to tail-freeness [14 Definition 3.11] , and our prior can be regarded as a partially specified Pólya tree [14 Section 3.7.3] . Exploiting this tail-free structure to obtain a nodewise factorization is a common starting point for inference in PT models. For example, OPT uses local factors to recursively marginalize over stopping and splitting configurations [35 ] , while Ma and Bruni [2024] factorize a partial likelihood into nodewise terms. However, PT models typically use nodewise factors as computational steps for global inferential quantities, whereas GEB will directly use the local evidence for constructing P \mathcal{P} P .
Greedy empirical Bayes procedure
We have just shown that once a partition is held fixed, any further refinement of the partition contributes to the marginal likelihood only locally through the split Bayes factor. The core of our GEB algorithm is thus to recursively partition the sample space by sequentially maximizing the split Bayes factor until we reach a partition where it is no longer advantageous to refine further. To cement the idea, let T T T be a binary tree whose root node is A \mathcal{A} A and leaves form the partition P \mathcal{P} P . We will subsequently refer to A \mathcal{A} A as a node of the tree T T T , and restrict splits of A \mathcal{A} A to be axis-aligned, so that its child nodes must take the form
A 0 = { x ∈ A : x ( d ) ≤ q } , A 1 = { x ∈ A : x ( d ) > q } , A_0 = \{x \in A : x^{(d)} \le q\}, \qquad A_1 = \{x \in A : x^{(d)} > q\}, A 0 = { x ∈ A : x ( d ) ≤ q } , A 1 = { x ∈ A : x ( d ) > q } ,
where q ∈ R q \in\mathbb{R} q ∈ R is the split location, d ∈ { 1 , … , D } d \in\{1,\ldots,D\} d ∈ { 1 , … , D } is the split dimension, and x ( d ) x^{(d)} x ( d ) denotes the d d d -th element of the vector x x x . In addition, for regularity, we restrict attention to eligible splits, meaning those producing two nonempty children with N A 0 ≥ 1 N_{A_0} \ge1 N A 0 ≥ 1 and N A 1 ≥ 1 N_{A_1} \ge1 N A 1 ≥ 1 . We call a node A A A eligible if it admits at least one eligible split. As a result, we will henceforth write the split Bayes factor as B A ( d , q ) \mathcal{B}_A(d,q) B A ( d , q ) to highlight the potential splits. Finally, for a constant c ∈ R c \in\mathbb{R} c ∈ R , let the choice q = c − q = c_- q = c − denote a split of the form A 0 = { x ∈ A : x ( d ) < c } A_0 = \{x \in A : x^{(d)} < c\} A 0 = { x ∈ A : x ( d ) < c } and A 1 = { x ∈ A : x ( d ) ≥ c } A_1 = \{x \in A : x^{(d)} \ge c\} A 1 = { x ∈ A : x ( d ) ≥ c } with B A ( d , c − ) = lim q ↑ c B A ( d , q ) \mathcal{B}_A(d,c_-) = \lim_{q \uparrow c} \mathcal{B}_A(d,q) B A ( d , c − ) = lim q ↑ c B A ( d , q ) .
Our GEB algorithm is then illustrated in Algorithm 2. At each eligible node A A A , we maximize the split Bayes factor, which in turn greedily maximizes the marginal likelihood M ( P ) \mathcal{M}(\mathcal{P}) M ( P ) , hence the GEB name. Importantly, the split Bayes factor provides an interpretable stopping criterion based on the binomial Bayes factor, as observing B A ( d ^ , q ^ ) ≤ 1 \mathcal{B}_A(\widehat{d},\widehat{q}) \le1 B A ( d , q ) ≤ 1 indicates that splitting cannot further increase the marginal likelihood. A desirable property of the GEB algorithm is that both the split and stopping decisions are fully governed by the marginal likelihood, without the need for heuristic stopping rules as is common in tree algorithms.
Table 2.
A useful sanity check for the GEB algorithm is that under axis-aligned splits that produce non-empty child nodes, the maximum of the split Bayes factor can be computed using finitely many evaluations, and the GEB tree will always have finite depth.
Theorem 1 (Split Bayes factor). Consider axis-aligned splits producing non-empty children. If N A ≤ 2 N_A \le2 N A ≤ 2 , no eligible split has B A ( d , q ) > 1 \mathcal{B}_A(d,q) > 1 B A ( d , q ) > 1 . Otherwise, the maximum is almost surely of the form B A ( d , x i ( d ) ) \mathcal{B}_A(d,x_i^{(d)}) B A ( d , x i ( d ) ) or B A ( d , ( x i ( d ) ) − ) \mathcal{B}_A(d,(x_i^{(d)})_-) B A ( d , ( x i ( d ) ) − ) for some d d d and observation x i ∈ A x_i \in A x i ∈ A , excluding the sample extrema in coordinate d d d within A A A . Thus, at most 2 D ( N A − 2 ) 2D(N_A-2) 2 D ( N A − 2 ) evaluations suffice.
The detailed proof is given in Appendix § 9 . In practice however, it is convenient both computationally and theoretically to restrict candidate splits to a deterministic m m m -grid along each dimension, that is
q r = l ( d ) + r w ( d ) m for r = 1 , … , m − 1 , (5) q_r = l^{(d)} + r\frac{w^{(d)}}{m} \quad\text{for } r=1,\ldots,m-1, \tag*{(5)} q r = l ( d ) + r m w ( d ) for r = 1 , … , m − 1 , (5)
where m ≥ 2 m \ge2 m ≥ 2 is a chosen integer, and l ( d ) l^{(d)} l ( d ) and w ( d ) w^{(d)} w ( d ) are the lower bound and width of A A A along dimension d d d . Note that if A A A is a rectangle, every node A A A is also a rectangle. Again, splits that leave either child empty are not eligible, and a node is eligible if at least one candidate is eligible. In particular, the computation is reduced to at most D ( m − 1 ) D(m-1) D ( m − 1 ) eligible choices at each node, and our theory in § 4 requires a fixed and finite m m m .
Intuitively, the split Bayes factor balances evidence of local departures from h ( ⋅ ∣ A ) h(\cdot\mid A) h ( ⋅ ∣ A ) against the penalty for further refinement. By choosing both the split dimension and location, GEB can place split boundaries near potential discontinuities, vary node sizes across regions, and avoid unnecessary splits along uninformative coordinates or in regions where a coarse approximation already suffices. This suggests adaptation to the density’s discontinuities, smoothness, and effective dimension, and § 4 formalizes this intuition under structural assumptions and establishes adaptive posterior contraction rates.
Practical considerations
In this section, we discuss some practical choices for the hyperparameters in the PPB model. We begin with a brief discussion on the choice of the base measure. In many cases, it is convenient to map data to the unit cube [ 0 , 1 ] D [0,1]^D [ 0 , 1 ] D using coordinate-wise differentiable transformations with strictly positive derivative, and take H H H to be uniform on A = [ 0 , 1 ] D A=[0,1]^D A = [ 0 , 1 ] D . The density on the original scale can be computed using the Jacobian, and the choice of H H H is therefore replaced by the choice of the transform. Section 5 describes the specific transforms used in our experiments.
In addition to H H H , GEB requires only two additional hyperparameters: the concentration parameter α > 0 \alpha> 0 α > 0 and the candidate-grid size m m m , where we recommend α = 1 \alpha= 1 α = 1 and m = 32 m = 32 m = 32 as robust, computationally efficient defaults. The parameter α \alpha α adds pseudo-observations from H H H , where a larger α \alpha α supports finer partitions while increasing shrinkage towards h h h . As a result, increasing α \alpha α balances two opposing effects: performance may improve as finer partitions capture structure, but may deteriorate when shrinkage and sampling variability due to fine partitions become excessive. Small values such as α = 1 \alpha= 1 α = 1 mitigate this deterioration and provide a robust default. Increasing m m m permits more precise split locations at greater computational cost, with diminishing gains once the grid resolves the relevant structure of the density. For example, densities with sharp local features may prefer larger m m m . Following Awaya and Ma [ [2 ] , we use m = 32 m = 32 m = 32 to balance resolution and computation. Note that Theorem 1 allows for finite computation even if we take m → ∞ m \to\infty m → ∞ . Finally, while our defaults work well in practice, one can run GEB for different values of ( α , m ) (\alpha, m) ( α , m ) and select the pair yielding the highest marginal likelihood; see Appendix § 15 .§ 3 for a sensitivity study.
Theory
In this section, we study the asymptotic properties of the GEB algorithm, and in particular establish adaptive posterior contraction rates for the induced PPB around the true density. As discussed earlier, unlike analyses based on globally optimal trees, our theory applies directly to the tree selected by GEB which can actually be obtained in practice. As a result, we require a proof technique based on local optimality and a set of greedy-specific assumptions, the exposition of which forms the focus of this section. For convenience, we assume for the theoretical results that A = [ 0 , 1 ] D \mathcal{A} = [0, 1]^D A = [ 0 , 1 ] D , H H H is the uniform distribution, and splitting is based on the m m m -grid as outlined in (5) . We now consider X i ∼ P X_i \sim P X i ∼ P for i = 1 , … , n i = 1, \ldots, n i = 1 , … , n , where statements in probability are with respect to the product measure.
Signal and bias-variance tradeoff
The core of our GEB algorithm is the split Bayes factor, which drives both splitting and stopping decisions. To understand the nature of the algorithm, we will need to introduce the notion of signal at a given node A A A . Let K ( p 1 ; p 2 ) K(p_1; p_2) K ( p 1 ; p 2 ) denote the Kullback-Leibler (KL) divergence between densities p 1 p_1 p 1 and p 2 p_2 p 2 . We will define the signal of a node A A A as
S A : = P A K { p ( ⋅ ∣ A ) ; h ( ⋅ ∣ A ) } . (6) S_A := P_A K\{p(\cdot\mid A); h(\cdot\mid A)\}. \tag*{(6)} S A := P A K { p ( ⋅ ∣ A ) ; h ( ⋅ ∣ A )} . (6)
The signal is thus larger if h ( ⋅ ∣ A ) h(\cdot\mid A) h ( ⋅ ∣ A ) is a poor local approximation of p ( ⋅ ∣ A ) p(\cdot\mid A) p ( ⋅ ∣ A ) , but this is down-weighted by the probability mass of A A A for smaller nodes. To see why S A S_A S A is a useful notion of signal, for a given partition P \mathcal{P} P , consider the noiseless partition density
p † ( x ) = ∑ A ∈ P P A h ( x ∣ A ) . p^\dagger(x) = \sum_{A \in\mathcal{P}} P_A h(x \mid A). p † ( x ) = A ∈ P ∑ P A h ( x ∣ A ) .
The KL divergence between p p p and p † p^\dagger p † can be shown to satisfy
K ( p ; p † ) = ∑ A ∈ P S A , (7) K(p; p^\dagger) = \sum_{A \in\mathcal{P}} S_A, \tag*{(7)} K ( p ; p † ) = A ∈ P ∑ S A , (7)
using the fact p ( x ) = P A p ( x ∣ A ) p(x) = P_A p(x \mid A) p ( x ) = P A p ( x ∣ A ) for x ∈ A x \in A x ∈ A for a measure P P P with density p p p . Each node A ∈ P A \in\mathcal{P} A ∈ P thus contributes S A S_A S A to the overall KL divergence, balancing local fit with node size.
A counterpart to the signal is the split gain at a node A A A for a split ( d , q ) (d,q) ( d , q ) , defined as
s A ( d , q ) : = P A [ P A 0 ∣ A log P A 0 ∣ A H A 0 ∣ A + ( 1 − P A 0 ∣ A ) log ( 1 − P A 0 ∣ A 1 − H A 0 ∣ A ) ] . (8) s_A(d,q) := P_A \left[P_{A_0 \mid A} \log\frac{P_{A_0 \mid A}}{H_{A_0 \mid A}} + (1-P_{A_0 \mid A}) \log\left(\frac{1-P_{A_0 \mid A}}{1-H_{A_0 \mid A}}\right)\right]. \tag*{(8)} s A ( d , q ) := P A [ P A 0 ∣ A log H A 0 ∣ A P A 0 ∣ A + ( 1 − P A 0 ∣ A ) log ( 1 − H A 0 ∣ A 1 − P A 0 ∣ A ) ] . (8)
Analogous to the signal, the term in the square brackets is the KL divergence between the Bernoulli distributions with success probabilities P A 0 ∣ A P_{A_0 \mid A} P A 0 ∣ A and H A 0 ∣ A H_{A_0 \mid A} H A 0 ∣ A . Crucially, the split gain and signal are related by the lemma below.
Lemma 1 (Split gain). Consider splitting a node A A A at ( d , q ) (d,q) ( d , q ) into children A 0 A_0 A 0 and A 1 A_1 A 1 . The split gain then satisfies
S A 0 + S A 1 = S A − s A ( d , q ) . S_{A_0} + S_{A_1} = S_A - s_A(d,q). S A 0 + S A 1 = S A − s A ( d , q ) .
Splitting a node A A A at ( d , q ) (d,q) ( d , q ) thus reduces the KL divergence in (7) exactly by s A ( d , q ) s_A(d,q) s A ( d , q ) ; further details are provided in Appendix C.§ 1 . A key assumption, which we outline in § 4.3 , is that the maximum split gain is sufficiently large relative to S A S_A S A , so that greedy splits can substantially reduce the node’s contribution to the KL divergence.
Finally, we connect the split gain to the split Bayes factor. For sufficiently large n n n and for nodes with N A > 0 N_A > 0 N A > 0 , we show in Appendix § 12 that the split Bayes factor satisfies
P A [ log B A ( d , q ) N A ] = s A ( d , q ) − 1 2 n log n H A + ε A , n ( d , q ) , (9) P_A \left[\frac{\log B_A(d,q)}{N_A}\right] = s_A(d,q) - \frac{1}{2n}\log\frac{n}{H_A} + \varepsilon_{A,n}(d,q), \tag*{(9)} P A [ N A log B A ( d , q ) ] = s A ( d , q ) − 2 n 1 log H A n + ε A , n ( d , q ) , (9)
where the second negative term is the Bayesian penalty, and the remainder ε A , n ( d , q ) \varepsilon_{A,n}(d,q) ε A , n ( d , q ) accounts for sampling and approximation errors. The above decomposition reveals a competition between the split gain and the complexity penalty for a given n n n . For strong signals, the split gain s A ( d , q ) s_A(d,q) s A ( d , q ) generally dominates (9) , leading to B A ( d , q ) > 1 B_A(d,q) > 1 B A ( d , q ) > 1 which drives splitting. On the other hand, for smaller nodes and/or weak signals, the negative penalty will tend to drive B A ( d , q ) < 1 B_A(d,q) < 1 B A ( d , q ) < 1 which favors stopping. A key technical step (Appendix E.2.2) is to control the remainder terms under both the strong and weak signal regimes, so that the split gain or penalty dominates respectively with high probability.
Importantly, strong and weak signals must be understood relative to the sample size n n n . Although the population signals are defined using the true node probabilities P A P_A P A , these are estimated in practice (e.g. with N A / n N_A/n N A / n ), and the data must support both splitting and the estimation of the resulting density. Further splitting reduces the approximation error in (7) , but leaves smaller nodes with fewer observations in each child for estimating node probabilities. The split Bayes factor thus reflects this bias–variance tradeoff: for a fixed node and split, larger n n n reduces both the penalty and the remainder with high probability, allowing weaker signals to justify further splitting. This motivates the n n n -dependent signal threshold introduced in the next section.
Near-optimal and admissible trees
This section introduces the key oracle machinery behind our proof, leveraging the intuition from the split Bayes factor decomposition in (9) . In particular, in order to handle the greedy nature of our GEB algorithm, we will need to define an oracle class of trees based on local criteria without assuming any kind of global optimality.
We begin by defining a useful class of trees which captures a local version of near-optimality. Let δ ∈ ( 0 , 1 ) \delta\in(0, 1) δ ∈ ( 0 , 1 ) be a slack parameter which governs the near-optimality of our oracle class. In most cases, the value of δ \delta δ is unimportant and we can take δ = 1 / 2 \delta= 1/2 δ = 1/2 for convenience. One exception where δ \delta δ must be selected appropriately is if p p p is discontinuous as in Example 6, but we will tend to omit the dependence on δ \delta δ for convenience. We now define the class of near-optimal trees denoted as T ∗ \mathcal{T}^{*} T ∗ , which is similar to the tree growing rules defined in Chi et al. [ [6 ] .
Definition 1 (Near-optimal trees). Fix δ ∈ ( 0 , 1 ) \delta\in(0, 1) δ ∈ ( 0 , 1 ) . Let T ∗ \mathcal{T}^{*} T ∗ consist of all finite trees rooted at A A A such that, at every internal node A A A , the chosen split ( d ^ , q ^ ) (\hat{d}, \hat{q}) ( d ^ , q ^ ) satisfies
s A ( d ^ , q ^ ) ≥ ( 1 − δ ) max d , q s A ( d , q ) , s_A(\hat{d}, \hat{q}) \ge(1-\delta)\max_{d,q} s_A(d,q), s A ( d ^ , q ^ ) ≥ ( 1 − δ ) d , q max s A ( d , q ) , where the maximum is taken over the candidate m m m -grid splits of A A A .
The class T ∗ \mathcal{T}^{*} T ∗ is defined entirely at the population level, depending only on p p p , m m m , and δ \delta δ , which we will use to construct our assumptions in § 4.3 . While T ∗ \mathcal{T}^{*} T ∗ constrains the choice of splits, it allows for arbitrarily deep trees, which does not take into account the aforementioned bias–variance tradeoff. In order to handle this, we will define a threshold to indicate that the signal S A S_A S A of a node is large enough relative to n n n .
Specifically, a node A A A is deemed identifiable if its signal satisfies
S A ≥ C 0 log n n , (10) S_A \ge\frac{C_0 \log n}{n}, \tag*{(10)} S A ≥ n C 0 log n , (10)
where C 0 > 0 C_0 > 0 C 0 > 0 is a sufficiently large constant that only depends on p p p , m m m and δ \delta δ as defined in Appendix § 12 . To see why this is a suitable threshold, we refer back to the decomposition of the split Bayes factor in (9) . Since s A ( d , q ) ≤ S A s_A(d, q) \le S_A s A ( d , q ) ≤ S A by Lemma 1, no split gain can overcome the negative penalty unless S A S_A S A is at least of order ( log n ) / n (\log n)/n ( log n ) / n . Furthermore, since the split gain is nonnegative, each child has signal no greater than its parent, so once a node falls below the identifiability threshold, all its descendants remain non-identifiable.
We are now ready to introduce the class of admissible trees T n \mathcal{T}_n T n based on the above threshold.
Definition 2 (Admissible trees). Let k max k_{\max} k m a x be the fixed positive integer depending only on p p p , m m m and δ \delta δ as defined in Appendix § 12 . Let T n \mathcal{T}_n T n consist of all trees constructed as follows:
Start with a near-optimal tree T ∈ T ∗ T \in\mathcal{T}^* T ∈ T ∗ whose internal nodes are identifiable and whose leaves are non-identifiable at n n n .
Refine each leaf of T T T using fewer than k max k_{\max} k m a x additional m m m -grid splits.
Figure 2 illustrates an admissible tree: all identifiable nodes are split near-optimally, while subsequent non-identifiable subtrees have limited size. This reflects the desired bias-variance tradeoff, where nodes with strong signals are split to reduce approximation bias, and limited splitting on nodes with weak signals controls the estimation variance. In § 4.4 , we show that under our assumptions, GEB selects an admissible tree with high probability and the resulting PPB posterior thus attains adaptive contraction rates.
Assumptions
We now outline the assumptions required for our main results. Throughout, we assume that p p p , m m m and δ \delta δ are fixed.
Assumption 1 (Bounded density). There exist constants 0 < C l ≤ C u < ∞ 0 < C_l \le C_u < \infty 0 < C l ≤ C u < ∞ such that C l ≤ p ( x ) ≤ C u C_l \le p(x) \le C_u C l ≤ p ( x ) ≤ C u for all x ∈ [ 0 , 1 ] D x \in[0,1]^D x ∈ [ 0 , 1 ] D .
Assumption 2 (Greedy identifiability). There exists a constant κ ∈ ( 0 , 1 ] \kappa\in(0,1] κ ∈ ( 0 , 1 ] such that, for every tree T ∈ T ∗ T \in\mathcal{T}^* T ∈ T ∗ and every node A ∈ T A \in T A ∈ T with S A > 0 S_A > 0 S A > 0 ,
max d , q s A ( d , q ) ≥ κ S A . \max_{d,q} s_A(d,q) \ge\kappa S_A. d , q max s A ( d , q ) ≥ κ S A . where the maximum is over the candidate m m m -grid splits of $A.
Assumption 3 (Signal decay). There exist constants C 1 > 0 C_1 > 0 C 1 > 0 and γ ∈ ( 1 , ∞ ] \gamma\in(1,\infty] γ ∈ ( 1 , ∞ ] such that, for every tree T ∈ T ∗ T \in\mathcal{T}^* T ∈ T ∗ and every node A ∈ T A \in T A ∈ T with S A > 0 S_A > 0 S A > 0 ,
S A ≤ ( C 1 P A ) γ . S_A \le(C_1P_A)^\gamma. S A ≤ ( C 1 P A ) γ . For γ = ∞ \gamma= \infty γ = ∞ , we interpret the condition as P A ≥ C 1 − 1 P_A \ge C_1^{-1} P A ≥ C 1 − 1 for every such node.
Assumption 1 ensures node probabilities are comparable to volumes, i.e. C l H A ≤ P A ≤ C u H A C_lH_A \le P_A \le C_uH_A C l H A ≤ P A ≤ C u H A . As Assumption 1 is standard, we will not discuss it further. Assumption 2 addresses the greedy nature of the algorithm, while Assumption 3 controls approximation difficulty.
Assumption 2 requires that a single candidate split can capture a fixed fraction of the signal at each node under consideration. A similar ‘sufficient impurity decrease’ assumption is made in the random forest literature [6 , 23 ] .
We see from Lemma 1 that if Assumption 2 holds, a near-optimal split would give
S A 0 + S A 1 ≤ [ 1 − ( 1 − δ ) κ ] S A , S_{A_0} + S_{A_1} \le[1 - (1-\delta)\kappa]S_A, S A 0 + S A 1 ≤ [ 1 − ( 1 − δ ) κ ] S A ,
so repeated near-optimal splitting of nodes with S A > 0 S_A > 0 S A > 0 can drive the signals, and thus the KL divergence from (7) , to 0. To illustrate how Assumption 2 can fail, consider a non-uniform univariate density symmetric about x = 0.5 x = 0.5 x = 0.5 . When m = 2 m = 2 m = 2 , the only candidate split of A A A is at q = 0.5 q = 0.5 q = 0.5 , for which P A 0 ∣ A = H A 0 ∣ A = 1 / 2 P_{A_0|A} = H_{A_0|A} = 1/2 P A 0 ∣ A = H A 0 ∣ A = 1/2 . Consequently, s A ( 1 , 0.5 ) = 0 s_A(1,0.5) = 0 s A ( 1 , 0.5 ) = 0 despite S A > 0 S_A > 0 S A > 0 . Fortunately, under additional regularity conditions, a sufficiently fine grid can ensure greedy identifiability, as we illustrate shortly in the examples.
For 1 < γ < ∞ 1 < \gamma< \infty 1 < γ < ∞ , Assumption 3 bounds the signal by ( P A ) γ (P_A)^\gamma ( P A ) γ up to a multiplicative constant. For 0 < P A < 1 0 < P_A < 1 0 < P A < 1 , a larger exponent γ > 1 \gamma> 1 γ > 1 gives an upper bound on S A S_A S A that decays faster. as P A P_A P A approaches zero. Since the signals sum to the KL approximation error in (7) , a larger γ \gamma γ thus intuitively corresponds to faster decay of approximation bias as P \mathcal{P} P is refined, indicating an easier approximation problem. As we will see shortly, for Hölder continuous densities with exponent β ∈ ( 0 , 1 ] \beta\in(0,1] β ∈ ( 0 , 1 ] , we can take γ = 1 + 2 β \gamma= 1 + 2\beta γ = 1 + 2 β , directly linking smoothness to signal decay. The endpoint γ = ∞ \gamma= \infty γ = ∞ includes suitably aligned piecewise-constant densities for which a finite partition achieves zero approximation error. Assumption 3 is similar in flavor to the global approximation decay rate of Liu et al. [ [19 ] .
Next, we illustrate examples of verifying Assumptions 2 and 3 in the D = 1 D = 1 D = 1 case, with derivations in Appendix D.1. We defer the more nuanced multivariate case to § 4.5 .
Hölder continuous with 0 < β ≤ 1 0 < \beta\le1 0 < β ≤ 1
In the univariate case, we have the useful sufficient condition for Assumption 3.
Proposition 1. Suppose D = 1 D = 1 D = 1 , Assumption 1 holds and H = Unif [ 0 , 1 ] H = \operatorname{Unif}[0,1] H = Unif [ 0 , 1 ] . If p p p is β \beta β -Hölder continuous for some β ∈ ( 0 , 1 ] \beta\in(0,1] β ∈ ( 0 , 1 ] , then Assumption 3 holds with γ = 1 + 2 β \gamma= 1 + 2\beta γ = 1 + 2 β .
In the univariate case in particular, we can verify the signal decay condition over all intervals A ⊆ [ 0 , 1 ] A \subseteq[0,1] A ⊆ [ 0 , 1 ] , so we need not consider the class of near-optimal trees T ∗ \mathcal{T}^* T ∗ . However, in the multivariate case, the near-optimal tree plays a role, as we will see in § 4.5 .
While Assumption 3 essentially holds for free in the univariate Hölder continuous case (given Assumption 1), Assumption 2 does not. Below, we provide a sufficient condition which can be checked more easily than Assumption 2, based on the locally reverse Poincaré inequality (LRP) condition introduced by Mazumder and Wang [ [23 ] in the regression context.
Proposition 2. Let D = 1 D = 1 D = 1 , H = Unif [ 0 , 1 ] H = \operatorname{Unif}[0,1] H = Unif [ 0 , 1 ] , and p p p be absolutely continuous on [ 0 , 1 ] [0,1] [ 0 , 1 ] . Suppose there exists C 2 > 0 C_2 > 0 C 2 > 0 such that, for every interval A ⊆ [ 0 , 1 ] A \subseteq[0,1] A ⊆ [ 0 , 1 ] with H A > 0 H_A > 0 H A > 0 ,
( ∫ A ∣ p ′ ( x ) ∣ d x ) 2 ≤ C 2 H A ∫ A ( p ( x ) − P A H A ) 2 d x . \left(\int_A |p'(x)| \, \mathrm{d}x\right)^2 \le\frac{C_2}{H_A}\int_A \left(p(x) - \frac{P_A}{H_A}\right)^2 \, \mathrm{d}x. ( ∫ A ∣ p ′ ( x ) ∣ d x ) 2 ≤ H A C 2 ∫ A ( p ( x ) − H A P A ) 2 d x . Then Assumption 2 holds if m ≥ max { 2 , ⌈ C 2 ⌉ } m \ge\max\{2, \lceil C_2\rceil\} m ≥ max { 2 , ⌈ C 2 ⌉} .
In the above, P A / H A P_A/H_A P A / H A is the mean of the density p ( X ) p(X) p ( X ) under X ∼ Unif ( A ) X \sim\operatorname{Unif}(A) X ∼ Unif ( A ) , so the right-hand side is a scaled variance term. The LRP condition thus controls the total variation of p p p relative to its variance under the uniform distribution, thereby limiting excessive oscillations or highly localized changes. While the above is still technical, we can verify the conditions of Proposition 2 for a few examples that are Hölder continuous for β ∈ ( 0 , 1 ] \beta\in(0,1] β ∈ ( 0 , 1 ] .
Example 2 (Monotonic). Let p p p be a monotone, absolutely continuous density satisfying 0 < c ≤ ∣ p ′ ( x ) ∣ ≤ C < ∞ 0 < c \leq\lvert p'(x)\rvert\leq C < \infty 0 < c ≤ ∣ p ′ ( x )∣ ≤ C < ∞ almost everywhere. Then the LRP condition of Proposition 2 holds. If Assumption 1 holds, then Assumption 3 holds with γ = 3 \gamma= 3 γ = 3 since p p p is Lipschitz. The LRP condition for this class was also verified in Mazumder and Wang [2023, Example 3.1].
Example 3 (Densities with a cusp). Consider p ( x ) ∝ 1 + c ∣ x − x 0 ∣ β p(x) \propto1+c\lvert x-x_0\rvert^\beta p ( x ) ∝ 1 + c ∣ x − x 0 ∣ β for c ∈ ( − 1 , ∞ ) ∖ { 0 } c \in(-1,\infty)\setminus\{0\} c ∈ ( − 1 , ∞ ) ∖ { 0 } , x 0 ∈ [ 0 , 1 ] x_0 \in[0,1] x 0 ∈ [ 0 , 1 ] and β ∈ ( 0 , 1 ] \beta\in(0,1] β ∈ ( 0 , 1 ] . Clearly p p p is bounded above and away from 0 0 0 , is absolutely continuous and has Hölder exponent β \beta β . It thus satisfies Assumptions 1 and 3 with γ = 1 + 2 β \gamma= 1+2\beta γ = 1 + 2 β , and one can further show that p p p also satisfies the LRP condition of Proposition 2.
C k [ 0 , 1 ] C^k[0,1] C k [ 0 , 1 ] with k ≥ 1 k \geq1 k ≥ 1
We now consider the smoother case p ∈ C k [ 0 , 1 ] p \in C^k[0,1] p ∈ C k [ 0 , 1 ] with k ≥ 1 k \geq1 k ≥ 1 , which is always Lipschitz. Under Assumption 1, applying Proposition 1 gives Assumption 3 with decay exponent γ = 3 \gamma= 3 γ = 3 . Fortunately in this setting, we can also establish Assumption 2 using an interpretable non-vanishing derivative condition which implies the above LRP condition.
Proposition 3. Let D = 1 D = 1 D = 1 , H = Unif [ 0 , 1 ] H = \operatorname{Unif}[0,1] H = Unif [ 0 , 1 ] , and p ∈ C k [ 0 , 1 ] p \in C^k[0,1] p ∈ C k [ 0 , 1 ] for some integer k ≥ 1 k \geq1 k ≥ 1 . If additionally for every x ∈ [ 0 , 1 ] x \in[0,1] x ∈ [ 0 , 1 ] , we have
max 1 ≤ j ≤ k ∣ p ( j ) ( x ) ∣ > 0 , \max_{1 \leq j \leq k} \lvert p^{(j)}(x)\rvert> 0, 1 ≤ j ≤ k max ∣ p ( j ) ( x )∣ > 0 , then Assumption 2 holds for sufficiently large m m m , where the threshold depends on p p p and k k k .
For k = 1 k = 1 k = 1 , the condition implies that p p p is strictly monotone, with derivative either strictly positive or negative. For larger k k k , stationary points are permitted provided they are not flat through order k k k , where all derivatives of orders 1 , … , k 1,\ldots,k 1 , … , k vanish. It nevertheless covers a broad class of densities, including C 2 C^2 C 2 strongly convex densities and multimodal densities.
Example 4 (Polynomial). Let p ( x ) p(x) p ( x ) be a polynomial density of degree r ≥ 1 r \geq1 r ≥ 1 on [ 0 , 1 ] [0,1] [ 0 , 1 ] . Since p ( r ) ( x ) = r ! a r ≠ 0 p^{(r)}(x) = r!a_r \ne0 p ( r ) ( x ) = r ! a r = 0 , where a r a_r a r is the leading coefficient, Proposition 3 holds with k = r k = r k = r . Polynomials were also shown to satisfy LRP in Mazumder and Wang [2023, Example 3.3].
Example 5 (Sinusoidal). Let p ( x ) = 1 + a sin ( 2 π x ) p(x) = 1 + a\sin(2\pi x) p ( x ) = 1 + a sin ( 2 π x ) for 0 < ∣ a ∣ < 1 0 < \lvert a\rvert< 1 0 < ∣ a ∣ < 1 . We can then apply Proposition 3 with k = 2 k = 2 k = 2 since max { ∣ cos ( 2 π x ) ∣ , ∣ sin ( 2 π x ) ∣ } > 0 \max\{\lvert\cos(2\pi x)\rvert,\lvert\sin(2\pi x)\rvert\} > 0 max {∣ cos ( 2 π x )∣ , ∣ sin ( 2 π x )∣} > 0 for all x ∈ [ 0 , 1 ] x \in[0,1] x ∈ [ 0 , 1 ] .
More generally, every non-constant density that is real analytic on a neighborhood of [ 0 , 1 ] [0,1] [ 0 , 1 ] satisfies the condition of Proposition 3 for some finite k k k ; this covers many examples including (truncated) Gaussian mixtures and exponentials of non-constant polynomials. Finally, while Assumption 2 is much simpler to verify in the C k [ 0 , 1 ] C^k[0,1] C k [ 0 , 1 ] setting, higher smoothness alone does not improve the γ = 3 \gamma=3 γ = 3 exponent, as we are ultimately still limited by the piecewise-constant approximation (e.g. see Castillo and Randrianarisoa [2022]).
Step density
Finally, a step density example illustrates the role of choosing δ \delta δ appropriately and restricting the assumptions to near-optimal trees, which allows for γ = ∞ \gamma=\infty γ = ∞ .
Example 6 (Step density). Let m ≥ 2 m \ge2 m ≥ 2 , and consider the univariate step density given by p ( x ) = p 1 1 ( x ∈ [ 0 , 1 m ] ) + p 2 1 ( x ∈ ( 1 m , 1 ] ) p(x)=p_1\mathbb{1}\left(x\in\left[0,\frac{1}{m}\right]\right)+p_2\mathbb{1}\left(x\in\left(\frac{1}{m},1\right]\right) p ( x ) = p 1 1 ( x ∈ [ 0 , m 1 ] ) + p 2 1 ( x ∈ ( m 1 , 1 ] ) , where p 1 , p 2 > 0 p_1,p_2>0 p 1 , p 2 > 0 , p 1 ≠ p 2 p_1\ne p_2 p 1 = p 2 , and p 1 + p 2 ( m − 1 ) = m p_1+p_2(m-1)=m p 1 + p 2 ( m − 1 ) = m so p p p is a density. In this case, one can show that choosing δ = 1 / m \delta=1/m δ = 1/ m ensures that any split of the root in a near-optimal tree must occur at q = 1 / m q=1/m q = 1/ m . All descendants then lie in constant-density regions, so the root is the only node with S A > 0 S_A>0 S A > 0 . Consequently, Assumptions 2 and 3 hold with γ = ∞ \gamma=\infty γ = ∞ . Unlike in the univariate smooth examples, Assumption 3 cannot hold uniformly over all intervals here, illustrating the role of the near-optimal tree restriction.
Main results
We are now ready to state our main result, which guarantees that the GEB algorithm selects a member of the admissible tree family T n \mathcal{T}_n T n with high probability.
Theorem 2 (Admissibility of the GEB tree). Fix m ≥ 2 m \ge2 m ≥ 2 and α > 0 \alpha>0 α > 0 , and suppose Assumptions 1, 2, and 3 hold. Let T n \mathcal{T}_n T n denote the GEB tree obtained from Algorithm 2 with m m m -grid splitting as in (5) . Then we have P ( T n ∈ T n ) ⟶ 1 \mathbb{P}(\mathcal{T}_n\in\mathcal{T}_n)\longrightarrow1 P ( T n ∈ T n ) ⟶ 1 as n → ∞ n\to\infty n → ∞ .
Proof outline. We control the error in the log split Bayes factor expansion (9) and bound the complexity of the two parts of an admissible tree shown in Figure 2 . Locally, we bound the probabilities of failing to split an identifiable node near-optimally and of making at least k max k_{\max} k m a x splits below a first non-identifiable node. A union bound over the possible nodes then turns these local bounds into the global admissibility guarantee above. □ \square □
Figure 2. An illustration of an admissible tree, where the boxed subtree contains 2 splits.
We highlight that the Bayesian penalty in the split Bayes factor is what discourages excess splitting once the remaining signal is weak, so the resulting tree remains within the admissible tree class with high probability. A technical point is that it may be possible to relax Assumption 3 for the above, although it greatly simplifies the proof and is necessary for the next result. We will now see that the admissible tree class allows for adaptive posterior contraction rates to be achieved by the PPB given the GEB partition.
Theorem 3 (Posterior contraction). Fix m ≥ 2 m \ge2 m ≥ 2 and α > 0 \alpha> 0 α > 0 , and suppose Assumptions 1, 2, and 3 hold. Let f f f be a density drawn from the PPB posterior Π ( ⋅ ∣ X 1 : n ) \Pi(\cdot\mid X_{1:n}) Π ( ⋅ ∣ X 1 : n ) based on the leaf partition of the GEB tree T n T_n T n from Theorem 2. For a sufficiently large constant M > 0 M > 0 M > 0 ,
Π { f : ∥ p − f ∥ L 1 ≥ M ( log n n ) ( 1 − 1 / γ ) / 2 | X 1 : n } → 0 in P -probability . \Pi\left\{f : \lVert p-f\rVert_{L_1} \ge M\left(\frac{\log n}{n}\right)^{(1-1/\gamma)/2} \,\middle|\, X_{1:n}\right\} \to0 \quad\text{in } \mathbb{P}\text{-probability}. Π { f : ∥ p − f ∥ L 1 ≥ M ( n log n ) ( 1 − 1/ γ ) /2 X 1 : n } → 0 in P -probability .
Proof outline. We leverage the two requirements of admissibility to control the bias–variance tradeoff. The identifiability threshold limits the signal left in each leaf and hence the approximation bias, while k max k_{\max} k m a x limits the extra splitting and thus the number of additional leaf probabilities to estimate, helping control the sampling error. Together these bounds give the stated rate for the posterior mean in (2). Finally, we use the Dirichlet posterior variances of the leaf probabilities to show that the posterior concentrates around the mean.
We see the bias-variance tradeoff of the admissible tree class gives an adaptive posterior contraction rate of n − ( γ − 1 ) / ( 2 γ ) n^{-(\gamma-1)/(2\gamma)} n − ( γ − 1 ) / ( 2 γ ) up to logarithmic factors, where a larger exponent γ > 1 \gamma> 1 γ > 1 improves the rate. For example, with γ = 1 + 2 β \gamma= 1 + 2\beta γ = 1 + 2 β in the Hölder case in § 4.3.1 , we have a posterior contraction rate of n − β / ( 1 + 2 β ) n^{-\beta/(1+2\beta)} n − β / ( 1 + 2 β ) , which agrees with the minimax rate for Hölder-smooth densities with β ∈ ( 0 , 1 ] \beta\in(0,1] β ∈ ( 0 , 1 ] . We note that this may not be the minimax rate for the class of Hölder-smooth densities which satisfy Assumption 2, as this restricted class is smaller. Likewise, the step function case in Example 6 gives a parametric rate of n − 1 / 2 n^{-1/2} n − 1/2 . In the next section, we demonstrate multivariate examples of adaptation.
Verifying assumptions for D > 1 D > 1 D > 1
As alluded to earlier, Assumptions 2 and 3 are less straightforward to verify in the multivariate case due to pathologies that can occur even when D = 2 D = 2 D = 2 . For Assumption 2, interactions between the coordinates can cancel out marginal trends, preventing detection via axis-aligned splits. As an example, if p p p is a bivariate copula density, which has uniform marginals, and H = Unif [ 0 , 1 ] 2 H = \operatorname{Unif}[0,1]^2 H = Unif [ 0 , 1 ] 2 , the split gain s A ( d , q ) s_A(d,q) s A ( d , q ) is equal to 0 for all root splits even if p ≠ h p \ne h p = h . A similar counterexample for random forests in regression is considered in [2022, Remark 3]. For Assumption 3, the challenge is that an elongated rectangle can have small P A P_A P A while retaining large variation along an informative coordinate. Thus, unlike the univariate case, smoothness alone does not ensure the required signal decay over all rectangles, and restricting to near-optimal nodes when verifying Assumption 3 is needed.
Despite the above challenges, we are able to provide general, verifiable sufficient conditions for Assumptions 2 and 3, albeit with slightly less generality than in the D = 1 D = 1 D = 1 case. In particular, we can rely on a multivariate version of the LRP condition (from Proposition 2), and an additional marginal visibility condition unique to the multivariate case. The latter condition ensures that averaging over other coordinates does not hide trends, which combats examples like the bivariate copula density. In the interest of space, we defer a detailed discussion of the general assumptions to Appendix D.§ 2 , and instead demonstrate the wide range of examples covered by our conditions below.
In all the below, we assume D ≥ 2 D \ge2 D ≥ 2 , H = Unif [ 0 , 1 ] D H = \operatorname{Unif}[0,1]^D H = Unif [ 0 , 1 ] D , and Assumption 1 holds. Some of the below examples, such as the additive, monotonic and convex examples, are similar to those considered in [2022], [2023] for regression.
Example 7 (Product and additive densities). Suppose the density has either an additive structure, p ( x ) ∝ c + ∑ d = 1 D g d ( x ( d ) ) p(x) \propto c + \sum_{d=1}^{D} g_d(x^{(d)}) p ( x ) ∝ c + ∑ d = 1 D g d ( x ( d ) ) , or a product structure, p ( x ) ∝ ∏ d = 1 D g d ( x ( d ) ) p(x) \propto\prod_{d=1}^{D} g_d(x^{(d)}) p ( x ) ∝ ∏ d = 1 D g d ( x ( d ) ) . If each component g d g_d g d is absolutely continuous and satisfies the LRP condition in Proposition 2, then Assumption 2 holds for sufficiently large m m m . Furthermore, if each g d g_d g d is β d \beta_d β d -Hölder continuous for β d ∈ ( 0 , 1 ] \beta_d \in(0,1] β d ∈ ( 0 , 1 ] , then Assumption 3 holds with γ = 1 + 2 β 0 / D \gamma= 1 + 2\beta_0/D γ = 1 + 2 β 0 / D where β 0 = ( D − 1 ∑ d = 1 D β d − 1 ) − 1 \beta_0 = \left(D^{-1}\sum_{d=1}^{D}\beta_d^{-1}\right)^{-1} β 0 = ( D − 1 ∑ d = 1 D β d − 1 ) − 1 . Note that β 0 \beta_0 β 0 governs the minimax rate for the anisotropic Hölder class [2013].
Example 8 (Coordinate-wise monotonicity). Suppose p ∈ C 1 ( [ 0 , 1 ] D ) p \in C^1([0,1]^D) p ∈ C 1 ([ 0 , 1 ] D ) and ∣ ∂ p ( x ) / ∂ x ( d ) ∣ > 0 \left|\partial p(x)/\partial x^{(d)}\right| > 0 ∂ p ( x ) / ∂ x ( d ) > 0 for every x ∈ [ 0 , 1 ] D x \in[0,1]^D x ∈ [ 0 , 1 ] D and every d = 1 , … , D d = 1,\ldots,D d = 1 , … , D . By continuity, each partial derivative has a fixed sign, so p p p is strictly monotone in each coordinate with the others fixed. Then, for sufficiently large m m m , Assumptions 2 and 3 hold with γ = 1 + 2 / D \gamma= 1 + 2/D γ = 1 + 2/ D .
Example 9 (Coordinate-wise convexity). Suppose p ∈ C 2 ( [ 0 , 1 ] D ) p \in C^2([0,1]^D) p ∈ C 2 ([ 0 , 1 ] D ) and ∣ ∂ 2 p ( x ) / ∂ ( x ( d ) ) 2 ∣ > 0 \left|\partial^2p(x)/\partial(x^{(d)})^2\right| > 0 ∂ 2 p ( x ) / ∂ ( x ( d ) ) 2 > 0 for every x ∈ [ 0 , 1 ] D x \in[0,1]^D x ∈ [ 0 , 1 ] D and every d = 1 , … , D d = 1,\ldots,D d = 1 , … , D . Then, for sufficiently large m m m , Assumptions 2 and 3 hold with γ = 1 + 2 / D \gamma= 1 + 2/D γ = 1 + 2/ D . Note that this allows for non-additive examples, and contains strong convexity as a special case.
Example 10 (Variable sparsity). Suppose p ( x ) = g ( x ( 1 : D 0 ) ) p(x) = g(x^{(1:D_0)}) p ( x ) = g ( x ( 1 : D 0 ) ) for some 1 ≤ D 0 < D 1 \le D_0 < D 1 ≤ D 0 < D , so the remaining coordinates are independent uniform noise. If g g g satisfies Assumptions 1, 2, and 3 in D 0 D_0 D 0 dimensions, then p p p satisfies the assumptions with the same m m m , κ \kappa κ and γ \gamma γ .
In Appendix D.2.6, we include an illustrative D = 2 D = 2 D = 2 quadratic example showing how interactions affect marginal visibility, and how the copula pathology can be alleviated with a coordinate transformation. Importantly, Examples 7 and 10 illustrate multivariate adaptation of the PPB posterior based on the GEB partition to anisotropic smoothness and variable sparsity. These features need not be specified to the algorithm, provided the stated assumptions hold for the chosen m m m . In fact, the anisotropic decay exponent extends beyond additive and product densities under appropriate assumptions, as we show in Corollary D.1 in the Appendix. In the variable sparsity case, if g g g is Lipschitz and satisfies additional assumptions, the decay exponent improves from γ = 1 + 2 / D \gamma= 1 + 2/D γ = 1 + 2/ D to γ = 1 + 2 / D 0 \gamma= 1 + 2/D_0 γ = 1 + 2/ D 0 , yielding a faster contraction rate. We thus see that the GEB partition enables adaptation even in challenging multivariate settings.
Experiments
Our experiments consider three separate structural regimes in the spirit of the examples in § 4.3 and § 4.5 , namely densities with axis-aligned discontinuities, smooth densities, and low-dimensional densities embedded in noise. The simulations are intentionally designed to exhibit these features, while the real datasets provide corresponding examples of the respective features in practice. For the density estimation experiments, we compare our approach to three baselines over 20 replicates, reporting the average test log score, n t e s t − 1 ∑ i = 1 n t e s t log p ^ ( x i t e s t ) n_{\mathrm{test}}^{-1}\sum_{i=1}^{n_{\mathrm{test}}} \log\hat{p}(x_i^{\mathrm{test}}) n test − 1 ∑ i = 1 n test log p ^ ( x i test ) , where larger values indicate better density estimation. A final Laplace distribution simulation illustrates uncertainty quantification for the integrated squared density as the functional of interest with the PPB. All experiments are run on the CPU of a Windows 11 machine with an AMD Ryzen 7 H 260 processor and 32GB of RAM.
Candidate models
We compare the following estimators: (1) GEB (ours), with shrinkage α = 1 \alpha= 1 α = 1 and grid size m = 32 m = 32 m = 32 ; (2) HMPT [2 ] , a Bayesian nonparametric tree model that accommodates both flexible partitioning and adaptive local smoothness, using SMC for inference; (3) PT, a conventional Pólya tree with midpoint splits; and (4) DPMM, a Dirichlet process mixture of Gaussian location–scale kernels. For the DPMM, we use diagonal covariance matrices and truncated variational inference for the density estimation experiments, and MCMC for more accurate uncertainty quantification in the later posterior uncertainty simulation. Full hyperparameter settings for all methods are provided in Appendix § 15 .§ 1 . For all examples, we use the posterior predictive density as the estimate.
Since the tree methods are defined on [ 0 , 1 ] D [0,1]^D [ 0 , 1 ] D , we map each coordinate to the unit interval using coordinate-wise transforms before fitting GEB, HMPT, and PT. We consider two transforms, the first of which is the Gaussian transform T j G ( x ( j ) ) = Φ ( ( x ( j ) − μ ( j ) ) / σ ( j ) ) T_j^{G}(x^{(j)}) = \Phi((x^{(j)}-\mu^{(j)})/\sigma^{(j)}) T j G ( x ( j ) ) = Φ (( x ( j ) − μ ( j ) ) / σ ( j ) ) , where Φ \Phi Φ is the standard Gaussian cumulative distribution function (CDF), and μ ( j ) \mu^{(j)} μ ( j ) and σ ( j ) \sigma^{(j)} σ ( j ) are the training sample mean and standard deviation of coordinate j j j . The second transform we consider is the mixture transform T j M ( x ( j ) ) = ( 1 − ϵ ) F U n i f ( x ( j ) ; a ( j ) , b ( j ) ) + ϵ Φ ( ( x ( j ) − μ ( j ) ) / σ ( j ) ) T_j^{M}(x^{(j)}) = (1-\epsilon)F_{\mathrm{Unif}}(x^{(j)};a^{(j)},b^{(j)}) + \epsilon\Phi((x^{(j)}-\mu^{(j)})/\sigma^{(j)}) T j M ( x ( j ) ) = ( 1 − ϵ ) F Unif ( x ( j ) ; a ( j ) , b ( j ) ) + ϵ Φ (( x ( j ) − μ ( j ) ) / σ ( j ) ) , where a ( j ) , b ( j ) a^{(j)}, b^{(j)} a ( j ) , b ( j ) are the training sample minimum and maximum of coordinate j j j , F U n i f ( x ; a , b ) F_{\mathrm{Unif}}(x;a,b) F Unif ( x ; a , b ) denotes the CDF of U n i f ( a , b ) \mathrm{Unif}(a,b) Unif ( a , b ) , and ϵ ∈ ( 0 , 1 ) \epsilon\in(0,1) ϵ ∈ ( 0 , 1 ) is a small number. We set ϵ = 0.001 \epsilon= 0.001 ϵ = 0.001 , and the small Gaussian component in T j M T_j^{M} T j M allows the fitted density to assign positive density to future observations outside the training sample range. We use the suffixes -G and -M to indicate the Gaussian and the mixture transforms respectively in the simulation results. For our results, we map the fitted densities back to the untransformed data scale using the Jacobian of the coordinate-wise transformation.
Simulation study
For our simulation datasets, we let n = n t e s t = 5,000 n = n_{\mathrm{test}} = 5{,}000 n = n test = 5 , 000 , and consider three data-generating distributions. The first setting is 0.9 U n i f ( [ 0.2 , 0.8 ] D ) + 0.1 U n i f ( [ 0 , 1 ] D ) 0.9\,\mathrm{Unif}([0.2,0.8]^D) + 0.1\,\mathrm{Unif}([0,1]^D) 0.9 Unif ([ 0.2 , 0.8 ] D ) + 0.1 Unif ([ 0 , 1 ] D ) with D = 5 D = 5 D = 5 , which contains axis-aligned discontinuities along the boundary of a high-density central rectangle. The second setting is a smooth D = 2 D = 2 D = 2 Gaussian mixture with density 0.5 N 2 ( μ 0 , Σ 0 ) + 0.5 N 2 ( μ 1 , Σ 1 ) 0.5\,\mathcal{N}_2(\mu_0,\Sigma_0) + 0.5\,\mathcal{N}_2(\mu_1,\Sigma_1) 0.5 N 2 ( μ 0 , Σ 0 ) + 0.5 N 2 ( μ 1 , Σ 1 ) , where μ 0 = ( − 2 , − 1 ) ⊤ \mu_0 = (-2,-1)^\top μ 0 = ( − 2 , − 1 ) ⊤ , μ 1 = ( 2 , 1 ) ⊤ \mu_1 = (2,1)^\top μ 1 = ( 2 , 1 ) ⊤ , and Σ 0 = ( 0.70 0.35 0.35 0.80 ) \Sigma_0 = \begin{pmatrix} 0.70 & 0.35 \\ 0.35 & 0.80 \end{pmatrix} Σ 0 = ( 0.70 0.35 0.35 0.80 ) and Σ 1 = ( 0.60 − 0.25 − 0.25 0.90 ) \Sigma_1 = \begin{pmatrix} 0.60 & -0.25 \\ -0.25 & 0.90 \end{pmatrix} Σ 1 = ( 0.60 − 0.25 − 0.25 0.90 ) . Finally, we consider a sparse setting with D = 50 D = 50 D = 50 , where the first D 0 = 2 D_0 = 2 D 0 = 2 coordinates follow the same Gaussian mixture and the remaining 48 coordinates are independent standard Gaussian noise, independent of the first two coordinates.
Table 2 reports the average test log scores and run-times for the three simulation settings. In the axis-aligned setting, GEB-M achieves the highest score, followed by HMPT-M. The mixture transform largely preserves the sharp density changes at the boundary of the central rectangle, yielding a setting similar to Example 6, which can be captured by a small number of informative splits. For the two-dimensional Gaussian mixture, DPMM achieves the highest score, while the tree methods remain competitive. The Gaussian transform provides modest improvements for GEB and HMPT relative to the mixture transform. In the sparse 50-dimensional setting, the Gaussian transform makes the independent noise coordinates approximately uniform, yielding a setting similar to Example 10 . GEB-G achieves the highest score, closely followed by HMPT-G. All three tree methods perform substantially better with the Gaussian transform than with the mixture transform, consistent with the benefit of adapting to active coordinates.
Across the three settings, GEB combines competitive or superior predictive accuracy with substantially shorter run-times than both HMPT and DPMM, and all GEB configurations take less than one second on average to run. While the PT is faster still, it generally achieves lower test log scores. As an aside, DPMM’s shorter run-time in the sparse setting reflects faster variational convergence to a simpler mixture when the noise coordinates are included.
Table 2 (PDF p. 23). Simulation average test log scores and wall-clock run-times. Standard errors are given in parentheses. Best results are in bold.
Real-data experiments
We next apply the methods to the POWER, Dry Bean, and ISOLET datasets from the UCI Machine Learning Repository [16 ] , chosen to reflect the three structural regimes considered above. After preprocessing, POWER contains 2,049,280 total observations with D = 6 D = 6 D = 6 . In each replication, we subsample 100,000 observations without replacement and split them equally into training and test sets. For Dry Bean (D = 16 D = 16 D = 16 ) and ISOLET (D = 100 D = 100 D = 100 ), we use all 13,611 and 7,797 observations respectively, dividing each dataset into approximately equal training and test sets.
Figure 3 (top) illustrates parallels between the real datasets and the three simulation settings using two representative coordinates from each dataset. In POWER, “Voltage” and “Sub-metering 3” form pronounced horizontal bands, which likely can be captured efficiently by axis-aligned splits. In Dry Bean, “Perimeter” and “Aspect ratio” show several overlapping clusters, consistent with the mixture structure suggested by the seven bean varieties. In ISOLET, the Gaussian-transformed observations show pronounced variation in density along PC1 but are spread approximately uniformly along PC100. This contrast is reminiscent of the sparse simulation setting, where some coordinates contain little structure.
Figure 3. Bivariate scatterplots of two representative coordinates from POWER, Dry Bean, and ISOLET. The top row shows the training data, and the bottom row shows samples of the same size from the PPB posterior predictive density based on the GEB partition. POWER is shown on the mixture-transformed scale, while Dry Bean and ISOLET are shown on the Gaussian-transformed scale.
For the tree-based methods, we use the mixture transform for POWER, which preserves its banded structure, and the Gaussian transform for Dry Bean and ISOLET. In particular, the Gaussian transform maps approximately Gaussian coordinates to approximately uniform ones, allowing GEB to leave these coordinates largely unsplit. Before applying these transforms, POWER is preprocessed following Papamakarios et al. [2017], while ISOLET is reduced to 100 dimensions using PCA fitted to the standardized training data. Further preprocessing and evaluation details are provided in Appendix § 15 .§ 2 .
Figure 3 (bottom) shows that samples from the PPB posterior predictive density closely match the main visual patterns in the training data, and the rectangular structure induced by GEB’s axis-aligned partition is also visible in the generated samples. This visual agreement is accompanied by competitive performance on held-out data, as shown by the average test log scores in Table 3 . On POWER, GEB achieves the highest score, consistent with its strong performance in the axis-aligned simulation setting. On Dry Bean, DPMM achieves the highest score, while GEB is the strongest tree-based method, substantially outperforming both HMPT and PT. On the 100-dimensional ISOLET dataset, GEB again achieves the highest score, followed by HMPT, extending its favorable predictive performance to the high-dimensional setting. Across all three datasets, GEB combines this predictive accuracy with run-times of less than one second on average, making it substantially faster than HMPT and DPMM. PT is once again the fastest method, but achieves noticeably lower test log scores than GEB on every dataset. As in the sparse simulation, fast variational convergence to a simple mixture explains DPMM’s short run-time on ISOLET.
Table 3. Real-data average test log scores and wall-clock run-times. Standard errors are given in parentheses. Best results are in bold.
Posterior uncertainty simulation
After obtaining a partition with the GEB procedure, we can carry out posterior inference using the PPB. To assess the quality of its uncertainty quantification, we carry out a final simulation experiment under a D = 1 D = 1 D = 1 standard Laplace density p ( x ) = 1 2 exp ( − ∣ x ∣ ) p(x) = \frac{1}{2}\exp(-|x|) p ( x ) = 2 1 exp ( − ∣ x ∣ ) , with the functional of interest being the integrated squared density θ ( f ) = ∫ R f ( x ) 2 d x \theta(f) = \int_{\mathbb{R}} f(x)^2 \, dx θ ( f ) = ∫ R f ( x ) 2 d x . The true parameter value can be shown to be θ ( p ) = 1 / 4 \theta(p) = 1/4 θ ( p ) = 1/4 .
We run 1,000 repeats, each with n = 5,000 n = 5{,}000 n = 5 , 000 training observations and 1,000 retained posterior draws. The PPB samples of θ \theta θ are obtained using Algorithm 1, whereas posterior density draws for the DPMM are generated using the MCMC implementation in the BNPmix package. For both methods, we construct 95% central credible intervals for θ \theta θ from the posterior samples, and we apply the Gaussian transform to map to [ 0 , 1 ] [0,1] [ 0 , 1 ] for PPB.
Table 4 shows that PPB achieves similar coverage of θ \theta θ with similar interval lengths, and noticeably smaller bias of the posterior mean estimate for θ \theta θ . The downward bias observed for both models is consistent with the discussion in Yiu et al. [2025]. The reported PPB run-time, which includes GEB partition selection, is substantially lower than that of DPMM, reflecting the efficiency of posterior sampling without MCMC. Finally, Figure 4 visually compares the two posterior distributions for one dataset, illustrating their similar spread.
Table 4. Coverage of 95% credible intervals, mean credible interval length, bias of the posterior mean, and run-times over 1,000 replications. Standard errors are given in parentheses.
Figure 4. Posterior distribution of the integrated squared density under PPB + GEB and the DPMM for one simulated dataset. The dashed line indicates the true θ ( p ) = 1 / 4 \theta(p) = 1/4 θ ( p ) = 1/4 .
Discussion
In this paper, we have introduced the partition posterior bootstrap (PPB) based on a smoothed Dirichlet process, which avoids the need for expensive MCMC and SMC for posterior computation. We introduce a greedy empirical Bayes (GEB) algorithm for selecting the partition, which scales very well to large samples and high-dimensional data. Empirically, GEB is orders of magnitude faster than existing Bayesian nonparametric alternatives, and remains competitive in prediction accuracy across challenging data regimes with minimal tuning. We thus view the PPB with the GEB partition as a practical off-the-shelf choice in settings where scalability and automatic tuning are paramount. Importantly, we are able to provide theoretical guarantees for the actual GEB algorithm and its associated PPB, demonstrating adaptive posterior contraction rates in challenging multivariate settings like anisotropic smoothness and variable sparsity, which may help explain its excellent practical performance.
A natural direction for future work is to extend the associated greedy empirical Bayes principle beyond density estimation. The interpretable local Bayes factors exploited here arise from the Dirichlet aggregation property, which may not be specific to density trees. Other models admitting a tractable tail-free factorization could benefit from the same structure, yielding scalable GEB procedures. Using GEB trees as part of a larger algorithm, such as boosting, may also form an interesting direction of future research.
Acknowledgments
Edwin Fong receives funding from the Research Grants Council of Hong Kong through the Early Career Scheme (Grant 27304424) and the General Research Fund (Grant 17306925). During the preparation of this manuscript, ChatGPT Astra was used to assist with coding, proofreading, and some of the technical arguments for verifying the assumptions in the examples. The authors have reviewed, verified and take full responsibility for all content. All datasets used in this paper are publicly available. Code for reproducing the results can be found at https://github.com/stlllll/GEB_density .
Preliminaries and notation
To handle the many constants involved throughout the Appendix, we write x ≲ y x \lesssim y x ≲ y to mean that there exists a constant C C C , independent of node choice A A A and sample size n n n , such that x ≤ C y x \le Cy x ≤ C y . Likewise, we use x ≳ y x \gtrsim y x ≳ y for the reverse direction. When both directions hold, for possibly different constants, we write x ≍ y x \asymp y x ≍ y . Unless otherwise stated, A A A will be a positive-volume rectangle in [ 0 , 1 ] D [0,1]^D [ 0 , 1 ] D . With the exception of Section B, H H H will be assumed to be Unif [ 0 , 1 ] D \operatorname{Unif}[0,1]^D Unif [ 0 , 1 ] D .
We also introduce some useful shorthand for our later proofs. For node A A A , we will write
p ˉ A = P A H A , σ A 2 = 1 H A ∫ A [ p ( x ) − p ˉ A ] 2 d x . (11) \bar{p}_A = \frac{P_A}{H_A}, \qquad\sigma_A^2 = \frac{1}{H_A}\int_A [p(x)-\bar{p}_A]^2 \, dx. \tag*{(11)} p ˉ A = H A P A , σ A 2 = H A 1 ∫ A [ p ( x ) − p ˉ A ] 2 d x . (11)
Note that the above correspond to mean and variance of p ( X ) p(X) p ( X ) under H ( ⋅ ∣ A ) H(\cdot\mid A) H ( ⋅ ∣ A ) . A useful property that we use repeatedly is the following: under Assumption 1, we have
P A ≍ H A , p ˉ A ≍ 1. P_A \asymp H_A, \qquad\bar{p}_A \asymp1. P A ≍ H A , p ˉ A ≍ 1.
Proof of Theorem 1
To prove Theorem 1, we first show that nodes containing at most two observations stop automatically. For larger nodes, we study the split Bayes factor between consecutive observed coordinates and show that its supremum is attained at, or immediately to the left of, an observed coordinate.
Nodes with N A ≤ 2 N_A \le2 N A ≤ 2
We first verify that the stopping rule is automatic for nodes with N A ≤ 2 N_A \le2 N A ≤ 2 . If N A ≤ 1 N_A \le1 N A ≤ 1 , then node A A A is not eligible. Now suppose N A = 2 N_A = 2 N A = 2 . Every eligible split has N A 0 = N A 1 = 1 N_{A_0} = N_{A_1} = 1 N A 0 = N A 1 = 1 . Using the beta function form of the split Bayes factor,
B A ( d , q ) = B ( α H A 0 + 1 , α H A 1 + 1 ) B ( α H A 0 , α H A 1 ) H A 2 H A 0 H A 1 = α 2 H A 0 H A 1 α H A ( α H A + 1 ) H A 2 H A 0 H A 1 = α H A α H A + 1 < 1. \begin{aligned}
\mathcal{B}_A(d,q) &= \frac{\mathrm{B}(\alpha H_{A_0}+1,\alpha H_{A_1}+1)}{\mathrm{B}(\alpha H_{A_0},\alpha H_{A_1})}\frac{H_A^2}{H_{A_0}H_{A_1}} \\
&= \frac{\alpha^2 H_{A_0}H_{A_1}}{\alpha H_A(\alpha H_A+1)}\frac{H_A^2}{H_{A_0}H_{A_1}} = \frac{\alpha H_A}{\alpha H_A+1}<1.
\end{aligned} B A ( d , q ) = B ( α H A 0 , α H A 1 ) B ( α H A 0 + 1 , α H A 1 + 1 ) H A 0 H A 1 H A 2 = α H A ( α H A + 1 ) α 2 H A 0 H A 1 H A 0 H A 1 H A 2 = α H A + 1 α H A < 1.
Thus every eligible split has B A ( d , q ) < 1 \mathcal{B}_A(d,q)<1 B A ( d , q ) < 1 , and the stopping rule therefore retains A A A as a terminal node.
Derivatives
Decomposing the beta functions into gamma functions and using Γ ( z + 1 ) = z Γ ( z ) \Gamma(z + 1) = z\Gamma(z) Γ ( z + 1 ) = z Γ ( z ) for z > 0 z > 0 z > 0 , we obtain
log B A ( d , q ) = ∑ j = 0 N A 0 − 1 log ( α H A 0 + j ) + ∑ j = 0 N A 1 − 1 log ( α H A 1 + j ) − N A 0 log H A 0 − N A 1 log H A 1 + log ( Γ ( α H A ) Γ ( α H A + N A ) H A N A ) . \log B_A(d,q) = \sum_{j=0}^{N_{A_0}-1} \log(\alpha H_{A_0} + j) + \sum_{j=0}^{N_{A_1}-1} \log(\alpha H_{A_1} + j) - N_{A_0} \log H_{A_0} - N_{A_1} \log H_{A_1} + \log\left(\frac{\Gamma(\alpha H_A)}{\Gamma(\alpha H_A + N_A)}H_A^{N_A}\right). log B A ( d , q ) = j = 0 ∑ N A 0 − 1 log ( α H A 0 + j ) + j = 0 ∑ N A 1 − 1 log ( α H A 1 + j ) − N A 0 log H A 0 − N A 1 log H A 1 + log ( Γ ( α H A + N A ) Γ ( α H A ) H A N A ) .
The last term is independent of H A 0 H_{A_0} H A 0 . The function above may be discontinuous at observed data coordinates, since crossing such a value changes N A 0 N_{A_0} N A 0 and N A 1 N_{A_1} N A 1 by one. For each d d d , moving q q q between consecutive observed coordinates leaves the child counts unchanged, while the base probabilities may vary. On each such interval, we may regard log B A ( d , q ) \log B_A(d,q) log B A ( d , q ) as a function of H A 0 H_{A_0} H A 0 , keeping the child counts and H A H_A H A fixed and setting H A 1 = H A − H A 0 H_{A_1} = H_A - H_{A_0} H A 1 = H A − H A 0 . Since H H H has a density, H A 0 H_{A_0} H A 0 is continuous and nondecreasing in q q q . Differentiating with respect to H A 0 H_{A_0} H A 0 gives
∂ log B A ∂ H A 0 = ∑ j = 1 N A 0 − 1 − j ( α H A 0 + j ) H A 0 + ∑ j = 1 N A 1 − 1 j ( α H A 1 + j ) H A 1 . (12) \frac{\partial\log B_A}{\partial H_{A_0}} = \sum_{j=1}^{N_{A_0}-1}\frac{-j}{(\alpha H_{A_0}+j)H_{A_0}} + \sum_{j=1}^{N_{A_1}-1}\frac{j}{(\alpha H_{A_1}+j)H_{A_1}}. \tag*{(12)} ∂ H A 0 ∂ log B A = j = 1 ∑ N A 0 − 1 ( α H A 0 + j ) H A 0 − j + j = 1 ∑ N A 1 − 1 ( α H A 1 + j ) H A 1 j . (12)
Differentiating again yields
∂ 2 log B A ∂ H A 0 2 = ∑ j = 1 N A 0 − 1 j ( 2 α H A 0 + j ) ( α H A 0 + j ) 2 H A 0 2 + ∑ j = 1 N A 1 − 1 j ( 2 α H A 1 + j ) ( α H A 1 + j ) 2 H A 1 2 . (13) \frac{\partial^2 \log B_A}{\partial H_{A_0}^2} = \sum_{j=1}^{N_{A_0}-1}\frac{j(2\alpha H_{A_0}+j)}{(\alpha H_{A_0}+j)^2H_{A_0}^2} + \sum_{j=1}^{N_{A_1}-1}\frac{j(2\alpha H_{A_1}+j)}{(\alpha H_{A_1}+j)^2H_{A_1}^2}. \tag*{(13)} ∂ H A 0 2 ∂ 2 log B A = j = 1 ∑ N A 0 − 1 ( α H A 0 + j ) 2 H A 0 2 j ( 2 α H A 0 + j ) + j = 1 ∑ N A 1 − 1 ( α H A 1 + j ) 2 H A 1 2 j ( 2 α H A 1 + j ) . (13)
Boundary intervals
We first handle the boundary cases, where one of the child nodes contains exactly one datum.
Lemma B.1. Let N A ≥ 3 N_A \ge3 N A ≥ 3 and a < b a < b a < b , and suppose the child counts remain constant for q ∈ [ a , b ) q \in[a,b) q ∈ [ a , b ) . Then we have sup q ∈ [ a , b ) log B A ( d , q ) = log B A ( d , b − ) \sup_{q\in[a,b)}\log B_A(d,q) = \log B_A(d,b-) sup q ∈ [ a , b ) log B A ( d , q ) = log B A ( d , b − ) if N A 0 = 1 N_{A_0} = 1 N A 0 = 1 , and sup q ∈ [ a , b ) log B A ( d , q ) = log B A ( d , a ) \sup_{q\in[a,b)}\log B_A(d,q) = \log B_A(d,a) sup q ∈ [ a , b ) log B A ( d , q ) = log B A ( d , a ) if N A 1 = 1 N_{A_1} = 1 N A 1 = 1 .
Proof. For the case N A 0 = 1 N_{A_0} = 1 N A 0 = 1 , we have N A 1 = N A − 1 N_{A_1} = N_A - 1 N A 1 = N A − 1 . Substituting into (B.1), the first sum is empty and hence
∂ log B A ∂ H A 0 = ∑ j = 1 N A − 2 j ( α H A 1 + j ) H A 1 > 0. \frac{\partial\log B_A}{\partial H_{A_0}} = \sum_{j=1}^{N_A-2}\frac{j}{(\alpha H_{A_1}+j)H_{A_1}} > 0. ∂ H A 0 ∂ log B A = j = 1 ∑ N A − 2 ( α H A 1 + j ) H A 1 j > 0.
Consequently, log B A ( d , q ) \log\mathcal{B}_A(d,q) log B A ( d , q ) is nondecreasing in q q q on [ a , b ) [a,b) [ a , b ) , and
sup q ∈ [ a , b ) log B A ( d , q ) = log B A ( d , b − ) . \sup_{q\in[a,b)} \log\mathcal{B}_A(d,q)=\log\mathcal{B}_A(d,b_-). q ∈ [ a , b ) sup log B A ( d , q ) = log B A ( d , b − ) .
This value is attained by the allowed boundary split q = b − q=b_- q = b − . If instead N A 1 = 1 N_{A_1}=1 N A 1 = 1 , then N A 0 = N A − 1 N_{A_0}=N_A-1 N A 0 = N A − 1 . By symmetry, ∂ log B A / ∂ H A 0 < 0 \partial\log\mathcal{B}_A/\partial H_{A_0}<0 ∂ log B A / ∂ H A 0 < 0 , so log B A ( d , q ) \log\mathcal{B}_A(d,q) log B A ( d , q ) is nonincreasing in q q q and its maximum on [ a , b ) [a,b) [ a , b ) is attained at a a a .
Interior intervals
We next consider intervals where both children contain at least two observations. The result then follows from convexity in H A 0 H_{A_0} H A 0 .
Lemma B.2. Let N A ≥ 4 N_A \ge4 N A ≥ 4 and a < b a < b a < b , and suppose the child counts remain constant for q ∈ [ a , b ) q \in[a,b) q ∈ [ a , b ) . Additionally assume min { N A 0 , N A 1 } ≥ 2 \min\{N_{A_0},N_{A_1}\} \ge2 min { N A 0 , N A 1 } ≥ 2 . We then have
sup q ∈ [ a , b ) log B A ( d , q ) = max { log B A ( d , a ) , log B A ( d , b − ) } . \sup_{q\in[a,b)} \log\mathcal{B}_A(d,q)=\max\{\log\mathcal{B}_A(d,a),\log\mathcal{B}_A(d,b_-)\}. q ∈ [ a , b ) sup log B A ( d , q ) = max { log B A ( d , a ) , log B A ( d , b − )} .
Proof. On the interval [ a , b ) [a,b) [ a , b ) , the child counts remain constant. By (B.2), log B A \log\mathcal{B}_A log B A is strictly convex as a function of H A 0 H_{A_0} H A 0 . Since H A 0 H_{A_0} H A 0 is continuous and nondecreasing in q q q , the supremum over [ a , b ) [a,b) [ a , b ) equals the larger of the endpoint values log B A ( d , a ) \log\mathcal{B}_A(d,a) log B A ( d , a ) and log B A ( d , b − ) \log\mathcal{B}_A(d,b_-) log B A ( d , b − ) . This proves the claim.
Main proof
For N A ≥ 3 N_A \ge3 N A ≥ 3 , the observations almost surely have distinct values in each coordinate, since P P P has a Lebesgue density. For each dimension d d d , consider the intervals between consecutive observed coordinates within A A A . Lemma B.1 shows that the sample minimum and the left-hand value at the sample maximum can be discarded from the search. Lemma B.2 reduces each remaining interval to its endpoint values, including the left-hand value at its right endpoint. Thus, the maximum is attained at an observed coordinate excluding the sample extrema, using one of the two boundary assignments. There are at most 2 ( N A − 2 ) 2(N_A-2) 2 ( N A − 2 ) such choices per dimension, giving 2 D ( N A − 2 ) 2D(N_A-2) 2 D ( N A − 2 ) evaluations in total.
Properties of the KL divergence
We present two useful results about the KL divergence which serve as tools for the later main proofs.
Proof of Lemma 1
We take 0 log 0 = 0 0 \log0 = 0 0 log 0 = 0 and set S A = s A ( d , q ) = 0 S_A = s_A(d,q) = 0 S A = s A ( d , q ) = 0 whenever P A = 0 P_A = 0 P A = 0 . For any node A A A , we have
S A = ∫ A p ( x ) log p ( x ) d x − P A log p ˉ A . S_A = \int_A p(x) \log p(x)\,\mathrm{d}x - P_A \log\bar{p}_A. S A = ∫ A p ( x ) log p ( x ) d x − P A log p ˉ A .
If P A = 0 P_A = 0 P A = 0 , both children also have zero mass, so the claim follows. For P A > 0 P_A > 0 P A > 0 , since A 0 A_0 A 0 and A 1 A_1 A 1 partition A A A , additivity of the integral and the definition of the split gain give
S A − s A ( d , q ) = ∫ A p ( x ) log p ( x ) d x − ∑ i = 0 1 P A i log p ˉ A i = ∑ i = 0 1 [ ∫ A i p ( x ) log p ( x ) d x − P A i log p ˉ A i ] = S A 0 + S A 1 . \begin{aligned}
S_A - s_A(d,q) &= \int_A p(x) \log p(x)\,\mathrm{d}x - \sum_{i=0}^{1} P_{A_i} \log\bar{p}_{A_i} \\
&= \sum_{i=0}^{1} \left[\int_{A_i} p(x) \log p(x)\,\mathrm{d}x - P_{A_i} \log\bar{p}_{A_i}\right] \\
&= S_{A_0} + S_{A_1}.
\end{aligned} S A − s A ( d , q ) = ∫ A p ( x ) log p ( x ) d x − i = 0 ∑ 1 P A i log p ˉ A i = i = 0 ∑ 1 [ ∫ A i p ( x ) log p ( x ) d x − P A i log p ˉ A i ] = S A 0 + S A 1 .
KL and variance
Our next result relates the KL divergence, and thus the signal, to the variance of the density under the base measure. This will be particularly useful when verifying Assumptions 2 and 3.
Lemma C.1. Under Assumption 1, we have
K { p ( ⋅ ∣ A ) ; h ( ⋅ ∣ A ) } ≍ σ A 2 . K\{p(\cdot\mid A); h(\cdot\mid A)\} \asymp\sigma_A^2. K { p ( ⋅ ∣ A ) ; h ( ⋅ ∣ A )} ≍ σ A 2 .
Proof. The standard inequalities between the KL divergence, the Hellinger metric and the χ 2 \chi^2 χ 2 -divergence give
∫ [ f ( x ) − g ( x ) ] 2 d x ≤ K ( f ; g ) ≤ ∫ [ f ( x ) − g ( x ) ] 2 g ( x ) d x . \int\left[\sqrt{f(x)}-\sqrt{g(x)}\right]^2\,\mathrm{d}x \le K(f;g) \le\int\frac{[f(x)-g(x)]^2}{g(x)}\,\mathrm{d}x. ∫ [ f ( x ) − g ( x ) ] 2 d x ≤ K ( f ; g ) ≤ ∫ g ( x ) [ f ( x ) − g ( x ) ] 2 d x .
Plugging in the conditional densities, the upper bound becomes
∫ A [ p ( x ∣ A ) − 1 / H A ] 2 1 / H A d x = σ A 2 ( p ‾ A ) 2 ≲ σ A 2 . (14) \int_A \frac{\left[p(x\mid A)-1/H_A\right]^2}{1/H_A}\,dx=\frac{\sigma_A^2}{(\overline{p}_A)^2}\lesssim\sigma_A^2. \tag*{(14)} ∫ A 1/ H A [ p ( x ∣ A ) − 1/ H A ] 2 d x = ( p A ) 2 σ A 2 ≲ σ A 2 . (14)
The lower bound becomes
∫ A [ p ( x ∣ A ) − 1 / H A ] 2 d x = 1 P A ∫ A [ p ( x ) − p ‾ A ] 2 d x . \int_A\left[\sqrt{p(x\mid A)}-\sqrt{1/H_A}\right]^2\,dx=\frac{1}{P_A}\int_A\left[\sqrt{p(x)}-\sqrt{\overline{p}_A}\right]^2\,dx. ∫ A [ p ( x ∣ A ) − 1/ H A ] 2 d x = P A 1 ∫ A [ p ( x ) − p A ] 2 d x .
For any x ∈ A x\in A x ∈ A , we have
[ p ( x ) − p ‾ A ] 2 = [ p ( x ) − p ‾ A p ( x ) + p ‾ A ] 2 ≳ [ p ( x ) − p ‾ A ] 2 . \left[\sqrt{p(x)}-\sqrt{\overline{p}_A}\right]^2=\left[\frac{p(x)-\overline{p}_A}{\sqrt{p(x)}+\sqrt{\overline{p}_A}}\right]^2\gtrsim[p(x)-\overline{p}_A]^2. [ p ( x ) − p A ] 2 = [ p ( x ) + p A p ( x ) − p A ] 2 ≳ [ p ( x ) − p A ] 2 .
We thus have
∫ A [ p ( x ∣ A ) − 1 / H A ] 2 d x ≳ 1 p ‾ A σ A 2 ≳ σ A 2 . \int_A\left[\sqrt{p(x\mid A)}-\sqrt{1/H_A}\right]^2\,dx\gtrsim\frac{1}{\overline{p}_A}\sigma_A^2\gtrsim\sigma_A^2. ∫ A [ p ( x ∣ A ) − 1/ H A ] 2 d x ≳ p A 1 σ A 2 ≳ σ A 2 .
This proves the result.
Structural assumptions
In this section, we provide detailed proofs and further examples for the results surrounding Assumptions 2 and 3. As in the main paper, we begin with the univariate case, before generalizing to the multivariate.
Univariate case
Proof of Proposition 1
Let A ⊆ [ 0 , 1 ] A\subseteq[0,1] A ⊆ [ 0 , 1 ] be any interval with H A > 0 H_A>0 H A > 0 . Using Lemma C.1, we have
S A = P A K { p ( ⋅ ∣ A ) ; h ( ⋅ ∣ A ) } ≲ P A σ A 2 . S_A=P_AK\{p(\cdot\mid A);h(\cdot\mid A)\}\lesssim P_A\sigma_A^2. S A = P A K { p ( ⋅ ∣ A ) ; h ( ⋅ ∣ A )} ≲ P A σ A 2 .
Next, for x ∈ A x\in A x ∈ A ,
∣ p ( x ) − p ‾ A ∣ = 1 H A ∣ ∫ A [ p ( x ) − p ( y ) ] d y ∣ ≤ 1 H A ∫ A ∣ x − y ∣ β d y \left|p(x)-\overline{p}_A\right|=\frac{1}{H_A}\left|\int_A[p(x)-p(y)]\,dy\right|
\leq\frac{1}{H_A}\int_A|x-y|^\beta\,dy ∣ p ( x ) − p A ∣ = H A 1 ∫ A [ p ( x ) − p ( y )] d y ≤ H A 1 ∫ A ∣ x − y ∣ β d y
where the second line follows from Hölder continuity. Since ∣ x − y ∣ ≤ H A |x-y| \le H_A ∣ x − y ∣ ≤ H A , the above gives ∣ p ( x ) − p ˉ A ∣ ≲ H A β |p(x)-\bar{p}_A| \lesssim H_A^\beta ∣ p ( x ) − p ˉ A ∣ ≲ H A β , which then implies
σ A 2 ≲ H A 2 β H A ∫ A d x = H A 2 β . \sigma_A^2 \lesssim\frac{H_A^{2\beta}}{H_A}\int_A dx = H_A^{2\beta}. σ A 2 ≲ H A H A 2 β ∫ A d x = H A 2 β .
Under Assumption 1, we note that H A 2 β ≲ ( P A ) 2 β H_A^{2\beta} \lesssim(P_A)^{2\beta} H A 2 β ≲ ( P A ) 2 β , so
S A ≲ ( P A ) 1 + 2 β , S_A \lesssim(P_A)^{1+2\beta}, S A ≲ ( P A ) 1 + 2 β ,
which gives Assumption 3 with γ = 1 + 2 β \gamma= 1 + 2\beta γ = 1 + 2 β .
Proof of Proposition 2
Our proof uses an integration argument analogous to Lemma B.2 of Mazumder and Wang [2023], and additional steps to handle the KL-based split gain and the m m m -grid. As p p p is absolutely continuous and D = 1 D = 1 D = 1 , it is almost everywhere (a.e.) differentiable. Let A = [ a , b ] ⊆ [ 0 , 1 ] A = [a,b] \subseteq[0,1] A = [ a , b ] ⊆ [ 0 , 1 ] where b > a b > a b > a and S A > 0 S_A > 0 S A > 0 .
We begin by applying Pinsker’s inequality to the KL divergence in the split gain:
s A ( 1 , q ) ≥ 2 P A ( P A 0 ∣ A − H A 0 ∣ A ) 2 = 2 P A ( P A 0 − H A 0 p ˉ A ) 2 (15) s_A(1,q) \ge2P_A\left(P_{A_0|A}-H_{A_0|A}\right)^2 = \frac{2}{P_A}\left(P_{A_0}-H_{A_0}\bar{p}_A\right)^2 \tag*{(15)} s A ( 1 , q ) ≥ 2 P A ( P A 0 ∣ A − H A 0 ∣ A ) 2 = P A 2 ( P A 0 − H A 0 p ˉ A ) 2 (15)
for q ∈ ( a , b ) q \in(a,b) q ∈ ( a , b ) , where we can write
F A ( q ) : = P A 0 − H A 0 p ˉ A = ∫ a q [ p ( x ) − p ˉ A ] d x . F_A(q) := P_{A_0}-H_{A_0}\bar{p}_A = \int_a^q [p(x)-\bar{p}_A]\,dx. F A ( q ) := P A 0 − H A 0 p ˉ A = ∫ a q [ p ( x ) − p ˉ A ] d x .
Note that F A ( a ) = F A ( b ) = 0 F_A(a)=F_A(b)=0 F A ( a ) = F A ( b ) = 0 from the definition of p ˉ A \bar{p}_A p ˉ A . We thus want to lower bound ∣ F A ( q ) ∣ |F_A(q)| ∣ F A ( q ) ∣ for an appropriately chosen q q q in order to lower bound s A ( 1 , q ) s_A(1,q) s A ( 1 , q ) for Assumption 2.
Since p p p is continuous, F A F_A F A is continuously differentiable, with F A ′ ( x ) = p ( x ) − p ˉ A F_A'(x)=p(x)-\bar{p}_A F A ′ ( x ) = p ( x ) − p ˉ A , so we have
H A σ A 2 = ∫ a b [ F A ′ ( x ) ] 2 d x . H_A\sigma_A^2 = \int_a^b [F_A'(x)]^2\,dx. H A σ A 2 = ∫ a b [ F A ′ ( x ) ] 2 d x .
As p p p is absolutely continuous by assumption, so is F A ′ F_A' F A ′ with F A ′ ′ ( x ) = p ′ ( x ) F_A''(x)=p'(x) F A ′′ ( x ) = p ′ ( x ) a.e., and this term arises in the left-hand side of the LRP inequality. We now use integration by parts to recover both the F A F_A F A and p ′ p' p ′ terms from the above equality:
∫ a b [ F A ′ ( x ) ] 2 d x = [ F A ( x ) F A ′ ( x ) ] a b − ∫ a b F A ( x ) F A ′ ′ ( x ) d x = − ∫ a b F A ( x ) p ′ ( x ) d x . \begin{aligned}
\int_a^b [F_A'(x)]^2\,dx
&= [F_A(x)F_A'(x)]_a^b-\int_a^b F_A(x)F_A''(x)\,dx \\
&= -\int_a^b F_A(x)p'(x)\,dx.
\end{aligned} ∫ a b [ F A ′ ( x ) ] 2 d x = [ F A ( x ) F A ′ ( x ) ] a b − ∫ a b F A ( x ) F A ′′ ( x ) d x = − ∫ a b F A ( x ) p ′ ( x ) d x .
where the boundary term vanishes. Together, we have the key identity
H A σ A 2 = − ∫ A F A ( x ) p ′ ( x ) d x . (16) H_A\sigma_A^2=-\int_A F_A(x)\,p'(x)\,\mathrm{d}x. \tag*{(16)} H A σ A 2 = − ∫ A F A ( x ) p ′ ( x ) d x . (16)
Similar arguments connecting variances to derivatives through integration by parts are used in the study of Stein-type covariance identities, e.g. Ernst et al. [ [8 ] .
Next, since F A ( x ) F_A(x) F A ( x ) is continuous and A A A is compact, there must exist some q q q such that ∣ F A ( q ) ∣ = max x ∈ A ∣ F A ( x ) ∣ |F_A(q)|=\max_{x\in A}|F_A(x)| ∣ F A ( q ) ∣ = max x ∈ A ∣ F A ( x ) ∣ by the extreme value theorem. From (D.2), we then have
H A σ A 2 ≤ ∣ F A ( q ) ∣ ∫ A ∣ p ′ ( x ) ∣ d x . H_A\sigma_A^2\leq|F_A(q)|\int_A |p'(x)|\,\mathrm{d}x. H A σ A 2 ≤ ∣ F A ( q ) ∣ ∫ A ∣ p ′ ( x ) ∣ d x .
Rearranging and applying the assumed LRP inequality then gives
∣ F A ( q ) ∣ ≥ H A σ A C 2 . (17) |F_A(q)|\geq\frac{H_A\sigma_A}{\sqrt{C_2}}. \tag*{(17)} ∣ F A ( q ) ∣ ≥ C 2 H A σ A . (17)
Next, we will round q q q to the nearest grid point, and use Lipschitz continuity to bound the change in F A F_A F A . Since p p p is continuous on A A A , the mean value theorem for integrals ensures there exists c ∈ A c\in A c ∈ A such that p ˉ A = p ( c ) \bar{p}_A=p(c) p ˉ A = p ( c ) . For any x ∈ A x\in A x ∈ A , we then have
∣ p ( x ) − p ˉ A ∣ = ∣ ∫ c x p ′ ( y ) d y ∣ ≤ ∫ A ∣ p ′ ( y ) ∣ d y ≤ C 2 σ A . (18) |p(x)-\bar{p}_A|=\left|\int_c^x p'(y)\,\mathrm{d}y\right|\leq\int_A |p'(y)|\,\mathrm{d}y\leq\sqrt{C_2}\sigma_A. \tag*{(18)} ∣ p ( x ) − p ˉ A ∣ = ∫ c x p ′ ( y ) d y ≤ ∫ A ∣ p ′ ( y ) ∣ d y ≤ C 2 σ A . (18)
where the last inequality follows again from LRP. As a result, ∣ F A ′ ( x ) ∣ ≤ C 2 σ A |F'_A(x)|\leq\sqrt{C_2}\sigma_A ∣ F A ′ ( x ) ∣ ≤ C 2 σ A , so F A F_A F A is Lipschitz with constant C 2 σ A \sqrt{C_2}\sigma_A C 2 σ A . Let q ^ \hat{q} q ^ denote the nearest grid point to q q q , including the endpoints a , b a,b a , b for now, so that ∣ q ^ − q ∣ ≤ H A / ( 2 m ) |\hat{q}-q|\leq H_A/(2m) ∣ q ^ − q ∣ ≤ H A / ( 2 m ) . The Lipschitz constant gives
∣ F A ( q ^ ) − F A ( q ) ∣ ≤ H A C 2 σ A 2 m ≤ C 2 2 m ∣ F A ( q ) ∣ |F_A(\hat{q})-F_A(q)|\leq\frac{H_A\sqrt{C_2}\sigma_A}{2m}\leq\frac{C_2}{2m}|F_A(q)| ∣ F A ( q ^ ) − F A ( q ) ∣ ≤ 2 m H A C 2 σ A ≤ 2 m C 2 ∣ F A ( q ) ∣
where the last inequality follows from (D.3). Since m ≥ C 2 m\geq C_2 m ≥ C 2 by assumption, we have
∣ F A ( q ^ ) ∣ ≥ 1 2 H A σ A C 2 . |F_A(\hat{q})|\geq\frac{1}{2}\frac{H_A\sigma_A}{\sqrt{C_2}}. ∣ F A ( q ^ ) ∣ ≥ 2 1 C 2 H A σ A .
Note that the weaker condition m > C 2 / 2 m>C_2/2 m > C 2 /2 also suffices for greedy identifiability, but our condition is convenient. Since F A ( a ) = F A ( b ) = 0 F_A(a)=F_A(b)=0 F A ( a ) = F A ( b ) = 0 but ∣ F A ( q ^ ) ∣ > 0 |F_A(\hat{q})|>0 ∣ F A ( q ^ ) ∣ > 0 , we must have q ^ ∈ ( a , b ) \hat{q}\in(a,b) q ^ ∈ ( a , b ) as a valid candidate split.
Finally, we connect the above to the split gain. Applying the above inequality to (D.1) gives
s A ( 1 , q ^ ) ≥ H A σ A 2 2 p ‾ A C 2 2 = P A σ A 2 2 ( p ‾ A ) 2 C 2 2 . s_A(1,\widehat{q}) \ge\frac{H_A\sigma_A^2}{2\overline{p}_A C_2^2} = \frac{P_A\sigma_A^2}{2(\overline{p}_A)^2 C_2^2}. s A ( 1 , q ) ≥ 2 p A C 2 2 H A σ A 2 = 2 ( p A ) 2 C 2 2 P A σ A 2 .
Using the χ 2 \chi^2 χ 2 -divergence upper bound on the KL and the equality statement in (C.1), which does not require Assumption 1, we have P A σ A 2 / ( p ‾ A ) 2 ≥ S A P_A\sigma_A^2/(\overline{p}_A)^2 \ge S_A P A σ A 2 / ( p A ) 2 ≥ S A , so
s A ( 1 , q ^ ) ≥ 1 2 C 2 2 S A s_A(1,\widehat{q}) \ge\frac{1}{2C_2^2}S_A s A ( 1 , q ) ≥ 2 C 2 2 1 S A
which gives Assumption 2 with κ = min { 1 , 1 / ( 2 C 2 ) } \kappa=\min\{1,1/(2C_2)\} κ = min { 1 , 1/ ( 2 C 2 )} .
Proof of Proposition 3
To verify LRP, we will use the lower bound on the derivatives to ensure p p p still has a trend at each x x x in order to lower bound the variance. By continuity of the derivatives, compactness of [ 0 , 1 ] [0,1] [ 0 , 1 ] , and the nonvanishing-derivative assumption, we have
t : = min x ∈ [ 0 , 1 ] max 1 ≤ j ≤ k ∣ p ( j ) ( x ) ∣ > 0. t := \min_{x\in[0,1]} \max_{1\le j\le k} \lvert p^{(j)}(x)\rvert> 0. t := x ∈ [ 0 , 1 ] min 1 ≤ j ≤ k max ∣ p ( j ) ( x )∣ > 0.
For each x x x , we have at least one j x j_x j x such that ∣ p ( j x ) ( x ) ∣ ≥ t \lvert p^{(j_x)}(x)\rvert\ge t ∣ p ( j x ) ( x )∣ ≥ t , where j x j_x j x may change with x x x . By continuity, we can choose a relatively open interval I x ⊆ [ 0 , 1 ] I_x\subseteq[0,1] I x ⊆ [ 0 , 1 ] containing x x x such that
∣ p ( j x ) ( y ) ∣ ≥ t 2 for y ∈ I x . \lvert p^{(j_x)}(y)\rvert\ge\frac{t}{2}\quad\text{for }y\in I_x. ∣ p ( j x ) ( y )∣ ≥ 2 t for y ∈ I x .
The intervals { I x : x ∈ [ 0 , 1 ] } \{I_x:x\in[0,1]\} { I x : x ∈ [ 0 , 1 ]} cover [ 0 , 1 ] [0,1] [ 0 , 1 ] , and by compactness, they admit a finite subcover { I x l } l = 1 L \{I_{x_l}\}_{l=1}^L { I x l } l = 1 L depending only on p p p and k k k .
For each interval I x l I_{x_l} I x l , we now leverage a sublevel set estimate given a lower bound on a derivative, which is used in Fourier analysis. Let g : [ a , b ] → R g:[a,b]\to\mathbb{R} g : [ a , b ] → R be j j j -times differentiable on ( a , b ) (a,b) ( a , b ) for j ≥ 1 j\ge1 j ≥ 1 , satisfying ∣ g ( j ) ( x ) ∣ ≥ t / 2 > 0 \lvert g^{(j)}(x)\rvert\ge t/2>0 ∣ g ( j ) ( x )∣ ≥ t /2 > 0 . We then have from Rogers [ [31 ] that for any u > 0 u>0 u > 0 ,
∣ { y ∈ [ a , b ] : ∣ g ( y ) ∣ ≤ u } ∣ ≤ 2 j ( 2 u t ) 1 / j . \left\lvert\{y\in[a,b]:\lvert g(y)\rvert\le u\}\right\rvert\le2j\left(\frac{2u}{t}\right)^{1/j}. ∣ { y ∈ [ a , b ] : ∣ g ( y )∣ ≤ u } ∣ ≤ 2 j ( t 2 u ) 1/ j .
The above states that if a function has some derivative uniformly bounded away from 0 by assumption, it cannot remain close to 0 for too much of the interval. Since subtracting a constant from p p p does not change its derivatives, for any interval A ⊆ [ 0 , 1 ] A\subseteq[0,1] A ⊆ [ 0 , 1 ] with $H_A>0, we can substitute g = p − p ˉ A g=p-\bar{p}_A g = p − p ˉ A and j = j x l j=j_{x_l} j = j x l , giving
∣ { x ∈ [ 0 , 1 ] : ∣ p ( x ) − p ˉ A ∣ ≤ u } ∣ ≤ ∑ l = 1 L ∣ { x ∈ I x l : ∣ p ( x ) − p ˉ A ∣ ≤ u } ∣ ≤ ∑ l = 1 L 2 j x l ( 2 u t ) 1 / j x l . \begin{aligned}
\left|\{x\in[0,1]:|p(x)-\bar{p}_A|\le u\}\right|\le\sum_{l=1}^{L}\left|\{x\in I_{x_l}:|p(x)-\bar{p}_A|\le u\}\right| \\
&\le\sum_{l=1}^{L}2j_{x_l}\left(\frac{2u}{t}\right)^{1/j_{x_l}}.
\end{aligned} ∣ { x ∈ [ 0 , 1 ] : ∣ p ( x ) − p ˉ A ∣ ≤ u } ∣ ≤ l = 1 ∑ L ∣ { x ∈ I x l : ∣ p ( x ) − p ˉ A ∣ ≤ u } ∣ ≤ l = 1 ∑ L 2 j x l ( t 2 u ) 1/ j x l .
Since j x l ≤ k j_{x_l}\le k j x l ≤ k , choosing 0 < u ≤ t / 2 0<u\le t/2 0 < u ≤ t /2 gives
∣ { x ∈ [ 0 , 1 ] : ∣ p ( x ) − p ˉ A ∣ ≤ u } ∣ ≤ 2 k L ( 2 u t ) 1 / k . \left|\{x\in[0,1]:|p(x)-\bar{p}_A|\le u\}\right|\le2kL\left(\frac{2u}{t}\right)^{1/k}. ∣ { x ∈ [ 0 , 1 ] : ∣ p ( x ) − p ˉ A ∣ ≤ u } ∣ ≤ 2 k L ( t 2 u ) 1/ k .
Note that the above bound holds uniformly over A A A . Next, we choose
u = t 2 ( H A 4 k L ) k u=\frac{t}{2}\left(\frac{H_A}{4kL}\right)^k u = 2 t ( 4 k L H A ) k
which satisfies u ≤ t / 2 u\le t/2 u ≤ t /2 since H A ≤ 1 H_A\le1 H A ≤ 1 , thus giving
∣ { x ∈ [ 0 , 1 ] : ∣ p ( x ) − p ˉ A ∣ ≤ u } ∣ ≤ H A 2 . \left|\{x\in[0,1]:|p(x)-\bar{p}_A|\le u\}\right|\le\frac{H_A}{2}. ∣ { x ∈ [ 0 , 1 ] : ∣ p ( x ) − p ˉ A ∣ ≤ u } ∣ ≤ 2 H A .
This then gives
∫ A [ p ( x ) − p ˉ A ] 2 d x ≥ H A u 2 2 , \int_A[p(x)-\bar{p}_A]^2\,dx\ge\frac{H_Au^2}{2}, ∫ A [ p ( x ) − p ˉ A ] 2 d x ≥ 2 H A u 2 ,
which then gives the key bound
σ A 2 ≥ u 2 2 ≳ H A 2 k . (19) \sigma_A^2\ge\frac{u^2}{2}\gtrsim H_A^{2k}. \tag*{(19)} σ A 2 ≥ 2 u 2 ≳ H A 2 k . (19)
We now use a Sobolev interpolation inequality to bound the left-hand side of the LRP condition as a function of H A k H_A^k H A k and σ A \sigma_A σ A . As p − p ˉ A ∈ C k [ 0 , 1 ] p-\bar{p}_A\in C^k[0,1] p − p ˉ A ∈ C k [ 0 , 1 ] , it lies in the Sobolev space W k , 1 ( A ) W^{k,1}(A) W k , 1 ( A ) , so we can apply Leoni [2017, Theorem 7.40, Equation (7.23)] for k ≥ 2 k\ge2 k ≥ 2 giving
∫ A ∣ p ′ ( x ) ∣ d x ≲ ( H A 2 ) − 1 ∫ A ∣ p ( x ) − p ˉ A ∣ d x + ( H A 2 ) k − 1 ∫ A ∣ p ( k ) ( x ) ∣ d x \int_A|p'(x)|\,dx\lesssim\left(\frac{H_A}{2}\right)^{-1}\int_A|p(x)-\bar{p}_A|\,dx+\left(\frac{H_A}{2}\right)^{k-1}\int_A|p^{(k)}(x)|\,dx ∫ A ∣ p ′ ( x ) ∣ d x ≲ ( 2 H A ) − 1 ∫ A ∣ p ( x ) − p ˉ A ∣ d x + ( 2 H A ) k − 1 ∫ A ∣ p ( k ) ( x ) ∣ d x
where we have chosen the length as H A / 2 H_A/2 H A /2 (written as ℓ \ell ℓ in their notation). Since ∣ p ( k ) ( x ) ∣ |p^{(k)}(x)| ∣ p ( k ) ( x ) ∣ is continuous, it is bounded on [ 0 , 1 ] [0,1] [ 0 , 1 ] . Cauchy-Schwarz applied to the first term then gives
∫ A ∣ p ′ ( x ) ∣ d x ≲ σ A + H A k . \int_A|p'(x)|\,dx\lesssim\sigma_A+H_A^k. ∫ A ∣ p ′ ( x ) ∣ d x ≲ σ A + H A k .
For k = 1 k = 1 k = 1 , continuity and compactness again give bounded p ′ p' p ′ , so ∫ A ∣ p ′ ( x ) ∣ d x ≲ H A \int_A |p'(x)|dx \lesssim H_A ∫ A ∣ p ′ ( x ) ∣ d x ≲ H A . Finally, (D.5) gives ∫ A ∣ p ′ ( x ) ∣ d x ≲ σ A \int_A |p'(x)|dx \lesssim\sigma_A ∫ A ∣ p ′ ( x ) ∣ d x ≲ σ A in both cases. Squaring establishes the LRP condition, so Proposition 2 completes the proof for k ≥ 1 k \ge1 k ≥ 1 .
Hölder examples
In this section, we verify the LRP condition for the two examples in the main paper. Again, let A = [ a , b ] A = [a,b] A = [ a , b ] with b > a b > a b > a throughout.
Monotonic We begin with the monotonic example which is slightly easier. By the upper bound on the absolute value of the derivative, we immediately have
( ∫ A ∣ p ′ ( x ) ∣ d x ) 2 ≤ C 2 H A 2 . (20) \left(\int_A |p'(x)|dx\right)^2 \le C^2 H_A^2. \tag*{(20)} ( ∫ A ∣ p ′ ( x ) ∣ d x ) 2 ≤ C 2 H A 2 . (20)
Monotonicity, absolute continuity and the bounds on the absolute value of the derivative give c ∣ x − y ∣ ≤ ∣ p ( x ) − p ( y ) ∣ ≤ C ∣ x − y ∣ c|x-y| \le|p(x)-p(y)| \le C|x-y| c ∣ x − y ∣ ≤ ∣ p ( x ) − p ( y ) ∣ ≤ C ∣ x − y ∣ . Next, we have the useful variance identity, Var [ X ] = 1 2 E [ ( X − Y ) 2 ] \operatorname{Var}[X] = \frac{1}{2}\mathbb{E}[(X-Y)^2] Var [ X ] = 2 1 E [( X − Y ) 2 ] , when Y Y Y is an independent copy of X X X . Applying this to σ A 2 \sigma_A^2 σ A 2 with X X X and Y Y Y drawn independently from Unif [ a , b ] \operatorname{Unif}[a,b] Unif [ a , b ] , we have
σ A 2 = 1 2 E [ { p ( X ) − p ( Y ) } 2 ] ≥ c 2 2 E [ ( X − Y ) 2 ] = c 2 H A 2 12 . \sigma_A^2 = \frac{1}{2}\mathbb{E}\left[\{p(X)-p(Y)\}^2\right] \ge\frac{c^2}{2}\mathbb{E}\left[(X-Y)^2\right] = \frac{c^2H_A^2}{12}. σ A 2 = 2 1 E [ { p ( X ) − p ( Y ) } 2 ] ≥ 2 c 2 E [ ( X − Y ) 2 ] = 12 c 2 H A 2 .
Combining with (D.6) gives
( ∫ A ∣ p ′ ( x ) ∣ d x ) 2 ≤ 12 ( C c ) 2 σ A 2 . \left(\int_A |p'(x)|dx\right)^2 \le12\left(\frac{C}{c}\right)^2\sigma_A^2. ( ∫ A ∣ p ′ ( x ) ∣ d x ) 2 ≤ 12 ( c C ) 2 σ A 2 .
The same constant was obtained in Mazumder and Wang [2023, Example 3.1].
Cusp For the cusp example, first consider an interval I = [ u , v ] I = [u,v] I = [ u , v ] lying to the left of x 0 x_0 x 0 . The density gives, for almost every x ∈ I x \in I x ∈ I ,
∣ p ′ ( x ) ∣ ∣ p ( v ) − p ( u ) ∣ = β ( x 0 − x ) β − 1 ( x 0 − u ) β − ( x 0 − v ) β ≥ β H I , \frac{|p'(x)|}{|p(v)-p(u)|} = \frac{\beta(x_0-x)^{\beta-1}}{(x_0-u)^\beta-(x_0-v)^\beta} \ge\frac{\beta}{H_I}, ∣ p ( v ) − p ( u ) ∣ ∣ p ′ ( x ) ∣ = ( x 0 − u ) β − ( x 0 − v ) β β ( x 0 − x ) β − 1 ≥ H I β ,
as the denominator satisfies
( x 0 − u ) β [ 1 − ( x 0 − v x 0 − u ) β ] ≤ ( x 0 − u ) β − 1 H I ≤ ( x 0 − x ) β − 1 H I , (x_0-u)^\beta\left[1-\left(\frac{x_0-v}{x_0-u}\right)^\beta\right] \le(x_0-u)^{\beta-1}H_I \le(x_0-x)^{\beta-1}H_I, ( x 0 − u ) β [ 1 − ( x 0 − u x 0 − v ) β ] ≤ ( x 0 − u ) β − 1 H I ≤ ( x 0 − x ) β − 1 H I ,
where the first inequality uses r β ≥ r r^\beta\ge r r β ≥ r for r ∈ [ 0 , 1 ] r \in[0,1] r ∈ [ 0 , 1 ] , and the second uses x ≥ u x \ge u x ≥ u and β − 1 ≤ 0 \beta- 1 \le0 β − 1 ≤ 0 . Since p p p is monotone on I I I , the same variance argument as above yields
σ I 2 ≥ β 2 12 ∣ p ( v ) − p ( u ) ∣ 2 . \sigma_I^2 \ge\frac{\beta^2}{12}|p(v)-p(u)|^2. σ I 2 ≥ 12 β 2 ∣ p ( v ) − p ( u ) ∣ 2 .
Moreover, ∫ I ∣ p ′ ( x ) ∣ d x = ∣ p ( v ) − p ( u ) ∣ \int_I |p'(x)|\,dx = |p(v)-p(u)| ∫ I ∣ p ′ ( x ) ∣ d x = ∣ p ( v ) − p ( u ) ∣ , so
( ∫ I ∣ p ′ ( x ) ∣ d x ) 2 ≤ 12 β 2 σ I 2 . (21) \left(\int_I |p'(x)|\,dx\right)^2 \le\frac{12}{\beta^2}\sigma_I^2. \tag*{(21)} ( ∫ I ∣ p ′ ( x ) ∣ d x ) 2 ≤ β 2 12 σ I 2 . (21)
The argument for I I I on the right of x 0 x_0 x 0 is identical, so the above bound holds there too.
Now suppose a < x 0 < b a < x_0 < b a < x 0 < b , and let I I I be the longer of [ a , x 0 ] [a,x_0] [ a , x 0 ] and [ x 0 , b ] [x_0,b] [ x 0 , b ] . Since we have
∫ a x 0 β ( x 0 − x ) β − 1 d x = ( x 0 − a ) β \int_a^{x_0} \beta(x_0-x)^{\beta-1}\,dx=(x_0-a)^\beta ∫ a x 0 β ( x 0 − x ) β − 1 d x = ( x 0 − a ) β
and similarly for [ x 0 , b ] [x_0,b] [ x 0 , b ] , it follows that
∫ A ∣ p ′ ( x ) ∣ d x ≤ 2 ∫ I ∣ p ′ ( x ) ∣ d x . \int_A |p'(x)|\,dx \le2\int_I |p'(x)|\,dx. ∫ A ∣ p ′ ( x ) ∣ d x ≤ 2 ∫ I ∣ p ′ ( x ) ∣ d x .
Conditioning a uniform draw from A A A on whether it lies in I I I , the law of total variance gives
σ A 2 ≥ H I H A σ I 2 + H A ∖ I H A σ A ∖ I 2 ≥ H I H A σ I 2 ≥ 1 2 σ I 2 , \sigma_A^2 \ge\frac{H_I}{H_A}\sigma_I^2+\frac{H_{A\setminus I}}{H_A}\sigma_{A\setminus I}^2 \ge\frac{H_I}{H_A}\sigma_I^2 \ge\frac{1}{2}\sigma_I^2, σ A 2 ≥ H A H I σ I 2 + H A H A ∖ I σ A ∖ I 2 ≥ H A H I σ I 2 ≥ 2 1 σ I 2 ,
where we used H I ≥ H A / 2 H_I \ge H_A/2 H I ≥ H A /2 in the last inequality. Applying (D.7) therefore yields
( ∫ A ∣ p ′ ( x ) ∣ d x ) 2 ≤ 4 ( ∫ I ∣ p ′ ( x ) ∣ d x ) 2 ≤ 48 β 2 σ I 2 ≤ 96 β 2 σ A 2 . \left(\int_A |p'(x)|\,dx\right)^2 \le4\left(\int_I |p'(x)|\,dx\right)^2 \le\frac{48}{\beta^2}\sigma_I^2 \le\frac{96}{\beta^2}\sigma_A^2. ( ∫ A ∣ p ′ ( x ) ∣ d x ) 2 ≤ 4 ( ∫ I ∣ p ′ ( x ) ∣ d x ) 2 ≤ β 2 48 σ I 2 ≤ β 2 96 σ A 2 .
This establishes the LRP condition with C 2 = 96 / β 2 C_2 = 96/\beta^2 C 2 = 96/ β 2 .
C k [ 0 , 1 ] C^k[0,1] C k [ 0 , 1 ] examples
For any non-constant density p p p that is real analytic on an open interval containing [ 0 , 1 ] [0,1] [ 0 , 1 ] , at each x ∈ [ 0 , 1 ] x \in[0,1] x ∈ [ 0 , 1 ] some derivative of positive order must be nonzero. By the same continuity and finite-subcover argument used in the proof of Proposition 3, these derivative orders can be chosen uniformly bounded by some finite k k k . Thus the condition of Proposition 3 holds.
Step density
For the step density in Example 6, one can verify that the split gain at q ≥ 1 / m q \ge1/m q ≥ 1/ m is
s A ( 1 , q ) = [ 1 − ( 1 − q ) p 2 ] log 1 − ( 1 − q ) p 2 q + ( 1 − q ) p 2 log p 2 s_A(1,q) = [1-(1-q)p_2]\log\frac{1-(1-q)p_2}{q} + (1-q)p_2\log p_2 s A ( 1 , q ) = [ 1 − ( 1 − q ) p 2 ] log q 1 − ( 1 − q ) p 2 + ( 1 − q ) p 2 log p 2
which follows as P A 1 = ( 1 − q ) p 2 P_{A_1}=(1-q)p_2 P A 1 = ( 1 − q ) p 2 . One can verify that
d 2 d q 2 s A ( 1 , q ) = ( 1 − p 2 ) 2 q 2 [ 1 − ( 1 − q ) p 2 ] ≥ 0 , \frac{d^2}{dq^2}s_A(1,q)=\frac{(1-p_2)^2}{q^2[1-(1-q)p_2]}\ge0, d q 2 d 2 s A ( 1 , q ) = q 2 [ 1 − ( 1 − q ) p 2 ] ( 1 − p 2 ) 2 ≥ 0 ,
so s A ( 1 , q ) s_A(1,q) s A ( 1 , q ) is convex in q q q . At the optimal split q = 1 / m q=1/m q = 1/ m , we have s A ( 1 , 1 / m ) = S A s_A(1,1/m)=S_A s A ( 1 , 1/ m ) = S A , and a continuous extension gives s A ( 1 , 1 ) = 0 s_A(1,1)=0 s A ( 1 , 1 ) = 0 . Since p 1 ≠ p 2 p_1\ne p_2 p 1 = p 2 , we must have S A > 0 S_A>0 S A > 0 . By convexity, for q = t / m + ( 1 − t ) q=t/m+(1-t) q = t / m + ( 1 − t ) for 0 ≤ t ≤ 1 0\le t\le1 0 ≤ t ≤ 1 , we have
s A ( 1 , q ) ≤ t S A . s_A(1,q)\le tS_A. s A ( 1 , q ) ≤ t S A .
Since q ≥ 2 / m q\ge2/m q ≥ 2/ m corresponds to t ≤ ( m − 2 ) / ( m − 1 ) t\le(m-2)/(m-1) t ≤ ( m − 2 ) / ( m − 1 ) , we thus have
s A ( 1 , q ) ≤ m − 2 m − 1 S A < ( 1 − 1 m ) S A for q ≥ 2 / m . s_A(1,q)\le\frac{m-2}{m-1}S_A<\left(1-\frac{1}{m}\right)S_A\quad\text{for }q\ge2/m. s A ( 1 , q ) ≤ m − 1 m − 2 S A < ( 1 − m 1 ) S A for q ≥ 2/ m .
Thus, δ = 1 / m \delta=1/m δ = 1/ m ensures that q = 1 / m q=1/m q = 1/ m is the only near-optimal root split among the candidate grid points.
Multivariate case
In the multivariate setting, we require two new ingredients. The first is to define the multivariate version of the LRP condition, which is relatively natural and plays the same role as the univariate case. The second new ingredient is a notion of marginal visibility , where differences between p p p and the base density h h h are visible to axis-aligned splits in order to combat the copula counterexample. We assume H = Unif [ 0 , 1 ] D H=\operatorname{Unif}[0,1]^D H = Unif [ 0 , 1 ] D throughout.
Fortunately, we can borrow much of the univariate machinery. To that end, let A = ∏ d = 1 D I d A=\prod_{d=1}^{D}I_d A = ∏ d = 1 D I d be a rectangle with H A > 0 H_A>0 H A > 0 . For each dimension d d d , we define the coordinate average
p ˉ A , d ( t ) = 1 H A − d ∫ A − d p ( t , x − d ) d x − d , t ∈ I d , \bar{p}_{A,d}(t)=\frac{1}{H_{A_{-d}}}\int_{A_{-d}}p(t,x_{-d})\,dx_{-d},\qquad t\in I_d, p ˉ A , d ( t ) = H A − d 1 ∫ A − d p ( t , x − d ) d x − d , t ∈ I d ,
where x − d x_{-d} x − d denotes x x x with its d d d -th element removed, and A − d = ∏ j ≠ d I j A_{-d}=\prod_{j\ne d}I_j A − d = ∏ j = d I j . Since p p p is integrable, Fubini’s theorem ensures that p ˉ A , d ( t ) \bar{p}_{A,d}(t) p ˉ A , d ( t ) is well-defined a.e. Note that the above is again the conditional mean of p ( t , X − d ) p(t, X_{-d}) p ( t , X − d ) under H ( ⋅ ∣ X − d ∈ A − d ) H(\cdot\mid X_{-d} \in A_{-d}) H ( ⋅ ∣ X − d ∈ A − d ) . Thus, p ˉ A , d ( t ) \bar{p}_{A,d}(t) p ˉ A , d ( t ) describes how the density varies along coordinate d d d , after averaging uniformly over the remaining coordinates within A − d A_{-d} A − d .
A nice property satisfied by the above coordinate average is
p ˉ A = 1 H I d ∫ I d p ˉ A , d ( t ) d t , (22) \bar{p}_A = \frac{1}{H_{I_d}} \int_{I_d} \bar{p}_{A,d}(t)\,\mathrm{d}t, \tag*{(22)} p ˉ A = H I d 1 ∫ I d p ˉ A , d ( t ) d t , (22)
so we can naturally define the coordinate-averaged variance of p ˉ A , d ( X ( d ) ) \bar{p}_{A,d}(X^{(d)}) p ˉ A , d ( X ( d ) ) under H ( ⋅ ∣ A ) H(\cdot\mid A) H ( ⋅ ∣ A ) :
σ A , d 2 = 1 H I d ∫ I d [ p ˉ A , d ( t ) − p ˉ A ] 2 d t . \sigma^2_{A,d} = \frac{1}{H_{I_d}} \int_{I_d} \left[\bar{p}_{A,d}(t) - \bar{p}_A\right]^2\,\mathrm{d}t. σ A , d 2 = H I d 1 ∫ I d [ p ˉ A , d ( t ) − p ˉ A ] 2 d t .
We can then simply define the LRP condition using p ˉ A , d ′ ( t ) \bar{p}'_{A,d}(t) p ˉ A , d ′ ( t ) and σ A , d 2 \sigma^2_{A,d} σ A , d 2 as below.
Assumption D.1 (Coordinate-averaged LRP). Let H = Unif [ 0 , 1 ] D H = \operatorname{Unif}[0,1]^D H = Unif [ 0 , 1 ] D and p ∈ W 1 , 1 ( ( 0 , 1 ) D ) p \in W^{1,1}((0,1)^D) p ∈ W 1 , 1 (( 0 , 1 ) D ) . There exists C 2 > 0 C_2 > 0 C 2 > 0 such that, for every rectangle A ⊆ [ 0 , 1 ] D A \subseteq[0,1]^D A ⊆ [ 0 , 1 ] D with H A > 0 H_A > 0 H A > 0 and each d ∈ { 1 , … , D } d \in\{1,\ldots,D\} d ∈ { 1 , … , D } ,
( ∫ I d ∣ p ˉ A , d ′ ( t ) ∣ d t ) 2 ≤ C 2 σ A , d 2 . \left(\int_{I_d} \left|\bar{p}'_{A,d}(t)\right|\,\mathrm{d}t\right)^2 \le C_2 \sigma^2_{A,d}. ( ∫ I d p ˉ A , d ′ ( t ) d t ) 2 ≤ C 2 σ A , d 2 .
The Sobolev condition ensures that p p p has a representative that is absolutely continuous on lines, with integrable partial derivatives [18 Theorem 11.45] . These properties together with Fubini’s theorem ensure that each coordinate average p ˉ A , d \bar{p}_{A,d} p ˉ A , d admits an absolutely continuous representative using standard arguments.
Our new marginal visibility condition then requires that the coordinate-averaged variance uniformly captures a fraction of the total variance σ A 2 \sigma^2_A σ A 2 .
Assumption D.2 (Marginal visibility). There exists η ∈ ( 0 , 1 ] \eta\in(0,1] η ∈ ( 0 , 1 ] such that for every rectangle A ⊆ [ 0 , 1 ] D A \subseteq[0,1]^D A ⊆ [ 0 , 1 ] D with H A > 0 H_A > 0 H A > 0 , we have
max d σ A , d 2 ≥ η σ A 2 . \max_d \sigma^2_{A,d} \ge\eta\sigma^2_A. d max σ A , d 2 ≥ η σ A 2 .
In the case where p p p is a non-constant bivariate copula density, taking A = A A = \mathcal{A} A = A violates the above, as σ A , 1 2 = σ A , 2 2 = 0 \sigma^2_{\mathcal{A},1} = \sigma^2_{\mathcal{A},2} = 0 σ A , 1 2 = σ A , 2 2 = 0 , while σ A 2 > 0 \sigma^2_{\mathcal{A}} > 0 σ A 2 > 0 . Note that when D = 1 D = 1 D = 1 , marginal visibility holds automatically with η = 1 \eta= 1 η = 1 , since σ A , 1 2 = σ A 2 \sigma^2_{\mathcal{A},1} = \sigma^2_{\mathcal{A}} σ A , 1 2 = σ A 2 .
Main results
We then have our main results below, from which we derive our examples.
Proposition D.1. Suppose p p p satisfies Assumptions D.1 and D.2, and let m ≥ max { 2 , ⌈ C 2 ⌉ } m \ge\max\{2,\lceil C_2\rceil\} m ≥ max { 2 , ⌈ C 2 ⌉} . Then Assumption 2 holds.
Proof. Let A A A be a rectangle with S A > 0 S_A > 0 S A > 0 . For this A A A , choose d d d to be the coordinate satisfying the marginal visibility inequality, and let I d = [ a , b ] I_d = [a,b] I d = [ a , b ] . We can then apply almost the exact same proof as Proposition 2. Starting from (D.1) with s A ( d , q ) s_A(d,q) s A ( d , q ) and q ∈ ( a , b ) q \in(a,b) q ∈ ( a , b ) instead, we additionally require the identities
P A 0 = ∫ a q H A − d p ˉ A , d ( t ) d t , H A 0 = H A − d ( q − a ) . P_{A_0} = \int_a^q H_{A_{-d}}\bar{p}_{A,d}(t)\,\mathrm{d}t,\qquad H_{A_0} = H_{A_{-d}}(q-a). P A 0 = ∫ a q H A − d p ˉ A , d ( t ) d t , H A 0 = H A − d ( q − a ) . which together give
F A , d ( q ) : = P A 0 − H A 0 p ˉ A = H A − d ∫ a q [ p ˉ A , d ( t ) − p ˉ A ] d t . F_{A,d}(q) := P_{A_0} - H_{A_0}\bar{p}_A = H_{A_{-d}}\int_a^q [\bar{p}_{A,d}(t)-\bar{p}_A]\,\mathrm{d}t. F A , d ( q ) := P A 0 − H A 0 p ˉ A = H A − d ∫ a q [ p ˉ A , d ( t ) − p ˉ A ] d t . Again F A , d ( a ) = F A , d ( b ) = 0 F_{A,d}(a) = F_{A,d}(b) = 0 F A , d ( a ) = F A , d ( b ) = 0 from (D.8). Since p ˉ A , d \bar{p}_{A,d} p ˉ A , d admits an absolutely continuous representative and satisfies Assumption D.1, we can apply the exact same argument from Proposition 2 to F A , d / H A − d F_{A,d}/H_{A_{-d}} F A , d / H A − d . Together with Pinsker’s inequality, this gives the existence of a candidate grid point q ^ ∈ ( a , b ) \hat{q} \in(a,b) q ^ ∈ ( a , b ) such that
s A ( d , q ^ ) ≥ H I d 2 H A − d 2 σ A , d 2 2 P A C 2 = H A σ A , d 2 2 p ˉ A C 2 ≥ η H A σ A 2 2 p ˉ A C 2 . s_A(d,\hat{q}) \ge\frac{H_{I_d}^2 H_{A_{-d}}^2 \sigma_{A,d}^2}{2P_A C_2} = \frac{H_A\sigma_{A,d}^2}{2\bar{p}_A C_2} \ge\frac{\eta H_A\sigma_A^2}{2\bar{p}_A C_2}. s A ( d , q ^ ) ≥ 2 P A C 2 H I d 2 H A − d 2 σ A , d 2 = 2 p ˉ A C 2 H A σ A , d 2 ≥ 2 p ˉ A C 2 η H A σ A 2 . where the final inequality is from marginal visibility. The same final step from the equality in (C.1) gives
s A ( d , q ^ ) ≥ η 2 C 2 S A s_A(d,\hat{q}) \ge\frac{\eta}{2C_2}S_A s A ( d , q ^ ) ≥ 2 C 2 η S A so Assumption 2 holds with κ = min { 1 , η / ( 2 C 2 ) } \kappa= \min\{1,\eta/(2C_2)\} κ = min { 1 , η / ( 2 C 2 )} .
The above proposition highlights that marginal visibility is the extra ingredient that allows us to extend the univariate LRP argument to establish greedy identifiability as in Assumption 2 when D > 1 D > 1 D > 1 . We will demonstrate verification of the marginal visibility condition shortly.
We now proceed to show sufficient conditions for signal decay as in Assumption 3. To begin, we need a technical lemma which controls the variance under nested rectangles, which will help combat the issue of elongated rectangles.
Lemma D.1. Let p p p be continuous, and suppose Assumption D.1 holds. For any nested rectangles B ⊆ A B \subseteq A B ⊆ A with H A , H B > 0 H_A, H_B > 0 H A , H B > 0 , we have
σ B 2 ≲ σ A 2 . \sigma_B^2 \lesssim\sigma_A^2. σ B 2 ≲ σ A 2 .
Proof. First, we note that
σ B 2 = 1 H B ∫ B [ p ( x ) − p ˉ B ] 2 d x ≤ 1 H B ∫ B [ p ( x ) − p ˉ A ] 2 d x , (23) \sigma_B^2 = \frac{1}{H_B}\int_B [p(x)-\bar{p}_B]^2\,\mathrm{d}x \leq\frac{1}{H_B}\int_B [p(x)-\bar{p}_A]^2\,\mathrm{d}x, \tag*{(23)} σ B 2 = H B 1 ∫ B [ p ( x ) − p ˉ B ] 2 d x ≤ H B 1 ∫ B [ p ( x ) − p ˉ A ] 2 d x , (23)
which follows as p ˉ B \bar{p}_B p ˉ B minimizes ∫ B [ p ( x ) − c ] 2 d x \int_B [p(x)-c]^2\,\mathrm{d}x ∫ B [ p ( x ) − c ] 2 d x over constants c ∈ R c \in\mathbb{R} c ∈ R . We will now show that sup x ∈ A [ p ( x ) − p ˉ A ] 2 ≲ σ A 2 \sup_{x\in A}[p(x)-\bar{p}_A]^2 \lesssim\sigma_A^2 sup x ∈ A [ p ( x ) − p ˉ A ] 2 ≲ σ A 2 under Assumption D.1, which is sufficient for σ B 2 ≲ σ A 2 \sigma_B^2 \lesssim\sigma_A^2 σ B 2 ≲ σ A 2 .
Write A = ∏ d = 1 D I d A = \prod_{d=1}^{D} I_d A = ∏ d = 1 D I d , and fix a coordinate d d d . Under Assumption D.1, we can apply the proof of Proposition 2 to p ˉ A , d \bar{p}_{A,d} p ˉ A , d again as we did in Proposition D.1. The bound (D.4) thus gives
∣ p ˉ A , d ( t ) − p ˉ A ∣ ≤ C 2 σ A , d \lvert\bar{p}_{A,d}(t)-\bar{p}_A\rvert\leq\sqrt{C_2}\,\sigma_{A,d} ∣ p ˉ A , d ( t ) − p ˉ A ∣ ≤ C 2 σ A , d
for any t ∈ I d t \in I_d t ∈ I d . For any constant c ∈ R c \in\mathbb{R} c ∈ R , the triangle inequality and AM-GM then give
∣ p ˉ A , d ( t ) − c ∣ 2 ≲ [ C 2 σ A , d + ∣ p ˉ A − c ∣ ] 2 \lvert\bar{p}_{A,d}(t)-c\rvert^2 \lesssim\left[\sqrt{C_2}\,\sigma_{A,d}+\lvert\bar{p}_A-c\rvert\right]^2 ∣ p ˉ A , d ( t ) − c ∣ 2 ≲ [ C 2 σ A , d + ∣ p ˉ A − c ∣ ] 2
≲ σ A , d 2 + [ p ˉ A − c ] 2 = 1 H I d ∫ I d [ p ˉ A , d ( u ) − c ] 2 d u , (24) \lesssim\sigma_{A,d}^2+[\bar{p}_A-c]^2=\frac{1}{H_{I_d}}\int_{I_d}[\bar{p}_{A,d}(u)-c]^2\,\mathrm{d}u, \tag*{(24)} ≲ σ A , d 2 + [ p ˉ A − c ] 2 = H I d 1 ∫ I d [ p ˉ A , d ( u ) − c ] 2 d u , (24)
where the final equality follows from E [ ( Y − c ) 2 ] = Var [ Y ] + ( E [ Y ] − c ) 2 \mathbb{E}[(Y-c)^2]=\operatorname{Var}[Y]+(\mathbb{E}[Y]-c)^2 E [( Y − c ) 2 ] = Var [ Y ] + ( E [ Y ] − c ) 2 and (D.8).
We now convert the above coordinate-averaged bound to a section-wise bound. Fix a coordinate d d d , a point x − d ∈ A − d x_{-d}\in A_{-d} x − d ∈ A − d , and a constant c ∈ R c\in\mathbb{R} c ∈ R . For ε > 0 \varepsilon>0 ε > 0 , define
Q ε = A − d ∩ ∏ j ≠ d [ x ( j ) − ε , x ( j ) + ε ] , A ε = I d × Q ε . Q_\varepsilon=A_{-d}\cap\prod_{j\neq d}[x^{(j)}-\varepsilon,x^{(j)}+\varepsilon],\qquad A_\varepsilon=I_d\times Q_\varepsilon. Q ε = A − d ∩ j = d ∏ [ x ( j ) − ε , x ( j ) + ε ] , A ε = I d × Q ε .
The rectangles A ε A_\varepsilon A ε have positive volume and remain inside A A A , and Q ε Q_\varepsilon Q ε shrinks to x − d x_{-d} x − d as ε → 0 \varepsilon\to0 ε → 0 . Since p p p is continuous, for each fixed t ∈ I d t\in I_d t ∈ I d ,
lim ε → 0 p ˉ A ε , d ( t ) = p ( t , x − d ) . \lim_{\varepsilon\to0}\bar{p}_{A_\varepsilon,d}(t)=p(t,x_{-d}). ε → 0 lim p ˉ A ε , d ( t ) = p ( t , x − d ) .
Moreover, continuity and compact support imply that p p p is bounded, so p ˉ A ε , d \bar{p}_{A_\varepsilon,d} p ˉ A ε , d is uniformly bounded. We now apply (D.10) to A ε A_\varepsilon A ε at a fixed t t t and take ε → 0 \varepsilon\to0 ε → 0 , where dominated convergence for the right-hand side gives
∣ p ( t , x − d ) − c ∣ 2 ≲ 1 H I d ∫ I d ∣ p ( u , x − d ) − c ∣ 2 d u (25) |p(t,x_{-d})-c|^2 \lesssim\frac{1}{H_{I_d}}\int_{I_d}|p(u,x_{-d})-c|^2\,du \tag*{(25)} ∣ p ( t , x − d ) − c ∣ 2 ≲ H I d 1 ∫ I d ∣ p ( u , x − d ) − c ∣ 2 d u (25)
for every t ∈ I d t\in I_d t ∈ I d , x − d ∈ A − d x_{-d}\in A_{-d} x − d ∈ A − d and c ∈ R c\in\mathbb{R} c ∈ R , with an implicit constant depending only on C 2 C_2 C 2 .
We now apply (D.11) successively for each d d d . For any x ∈ A x\in A x ∈ A , applying it to x ( 1 ) x^{(1)} x ( 1 ) gives
∣ p ( x ) − c ∣ 2 ≲ 1 H I 1 ∫ I 1 ∣ p ( u 1 , x ( 2 : D ) ) − c ∣ 2 d u 1 . |p(x)-c|^2 \lesssim\frac{1}{H_{I_1}}\int_{I_1}|p(u_1,x^{(2:D)})-c|^2\,du_1. ∣ p ( x ) − c ∣ 2 ≲ H I 1 1 ∫ I 1 ∣ p ( u 1 , x ( 2 : D ) ) − c ∣ 2 d u 1 .
For each u 1 u_1 u 1 , applying (D.11) to x ( 2 ) x^{(2)} x ( 2 ) in the integrand gives
∣ p ( x ) − c ∣ 2 ≲ 1 H I 1 H I 2 ∫ I 1 ∫ I 2 ∣ p ( u 1 , u 2 , x ( 3 : D ) ) − c ∣ 2 d u 2 d u 1 . |p(x)-c|^2 \lesssim\frac{1}{H_{I_1}H_{I_2}}\int_{I_1}\int_{I_2}|p(u_1,u_2,x^{(3:D)})-c|^2\,du_2\,du_1. ∣ p ( x ) − c ∣ 2 ≲ H I 1 H I 2 1 ∫ I 1 ∫ I 2 ∣ p ( u 1 , u 2 , x ( 3 : D ) ) − c ∣ 2 d u 2 d u 1 .
Repeating this argument through all D D D coordinates gives
∣ p ( x ) − c ∣ 2 ≲ 1 H A ∫ A ∣ p ( u ) − c ∣ 2 d u , |p(x)-c|^2 \lesssim\frac{1}{H_A}\int_A|p(u)-c|^2\,du, ∣ p ( x ) − c ∣ 2 ≲ H A 1 ∫ A ∣ p ( u ) − c ∣ 2 d u ,
with an implicit constant depending only on C 2 C_2 C 2 and D D D . Setting c = p ˉ A c=\bar p_A c = p ˉ A , we have
sup x ∈ A ∣ p ( x ) − p ˉ A ∣ 2 ≲ σ A 2 . \sup_{x\in A}|p(x)-\bar p_A|^2 \lesssim\sigma_A^2. x ∈ A sup ∣ p ( x ) − p ˉ A ∣ 2 ≲ σ A 2 .
As discussed earlier, the above is then sufficient for σ B 2 ≲ σ A 2 \sigma_B^2\lesssim\sigma_A^2 σ B 2 ≲ σ A 2 from (D.9).
We are now equipped to show the multivariate version of Proposition 1.
Proposition D.2. Suppose Assumption 1 and the conditions of Proposition D.1 hold. If p p p is additionally β \beta β -Hölder continuous for some β ∈ ( 0 , 1 ] \beta\in(0,1] β ∈ ( 0 , 1 ] , then Assumption 3 holds with γ = 1 + 2 β / D \gamma=1+2\beta/D γ = 1 + 2 β / D .
Proof. We first illustrate why the proof of Proposition 1 faces difficulties when D > 1 D>1 D > 1 . Starting with S A ≲ P A σ A 2 S_A\lesssim P_A\sigma_A^2 S A ≲ P A σ A 2 , a direct extension of the Hölder argument would require ∥ x − y ∥ ≲ H A 1 / D \lVert x-y\rVert\lesssim H_A^{1/D} ∥ x − y ∥ ≲ H A 1/ D for all x , y ∈ A x,y\in A x , y ∈ A . However, this bound fails for general rectangles, as an elongated rectangle can have arbitrarily small volume while ∥ x − y ∥ \lVert x-y\rVert ∥ x − y ∥ remains large. We must thus bound the variance another way.
We begin with some general properties under axis-aligned splitting. Let A A A be an arbitrary rectangle with S A > 0 S_A>0 S A > 0 , where again we write A = ∏ d = 1 D I d A=\prod_{d=1}^D I_d A = ∏ d = 1 D I d . The χ 2 \chi^2 χ 2 -divergence upper bound on the KL applied to the split gain gives
s A ( d , q ) ≤ P A ∑ i = 0 1 ( P A i ∣ A − H A i ∣ A ) 2 H A i ∣ A = 1 p ˉ A ∑ i = 0 1 H A i ( p ˉ A i − p ˉ A ) 2 , s_A(d,q) \le P_A \sum_{i=0}^{1} \frac{(P_{A_i|A}-H_{A_i|A})^2}{H_{A_i|A}} = \frac{1}{\bar p_A}\sum_{i=0}^{1} H_{A_i}(\bar p_{A_i}-\bar p_A)^2, s A ( d , q ) ≤ P A i = 0 ∑ 1 H A i ∣ A ( P A i ∣ A − H A i ∣ A ) 2 = p ˉ A 1 i = 0 ∑ 1 H A i ( p ˉ A i − p ˉ A ) 2 ,
where we used P A i ∣ A = ( H A i p ˉ A i ) / ( H A p ˉ A ) P_{A_i|A}=(H_{A_i}\bar p_{A_i})/(H_A\bar p_A) P A i ∣ A = ( H A i p ˉ A i ) / ( H A p ˉ A ) . Since the split only changes coordinate d d d , let J 0 , J 1 J_0,J_1 J 0 , J 1 be the subintervals of I d I_d I d induced by the split, so A i = J i × A − d A_i=J_i\times A_{-d} A i = J i × A − d . We then have
p ˉ A i = 1 H J i H A − d ∫ J i ∫ A − d p ( t , x − d ) d x − d d t = 1 H J i ∫ J i p ˉ A , d ( t ) d t . \bar p_{A_i}=\frac{1}{H_{J_i}H_{A_{-d}}}\int_{J_i}\int_{A_{-d}}p(t,x_{-d})\,dx_{-d}\,dt=\frac{1}{H_{J_i}}\int_{J_i}\bar p_{A,d}(t)\,dt. p ˉ A i = H J i H A − d 1 ∫ J i ∫ A − d p ( t , x − d ) d x − d d t = H J i 1 ∫ J i p ˉ A , d ( t ) d t .
Consequently, we have
p ˉ A i − p ˉ A = 1 H J i ∫ J i [ p ˉ A , d ( t ) − p ˉ A ] d t . \bar p_{A_i}-\bar p_A=\frac{1}{H_{J_i}}\int_{J_i}[\bar p_{A,d}(t)-\bar p_A]\,dt. p ˉ A i − p ˉ A = H J i 1 ∫ J i [ p ˉ A , d ( t ) − p ˉ A ] d t .
Next, for any d d d and t ∈ I d t\in I_d t ∈ I d , the Hölder assumption gives
∣ p ˉ A , d ( t ) − p ˉ A ∣ = ∣ 1 H A ∫ A − d ∫ I d [ p ( t , x − d ) − p ( u , x − d ) ] d u d x − d ∣ ≲ ∣ I d ∣ β , (26) \left|\bar p_{A,d}(t)-\bar p_A\right|=\left|\frac{1}{H_A}\int_{A_{-d}}\int_{I_d}[p(t,x_{-d})-p(u,x_{-d})]\,du\,dx_{-d}\right|\lesssim|I_d|^\beta, \tag*{(26)} ∣ p ˉ A , d ( t ) − p ˉ A ∣ = H A 1 ∫ A − d ∫ I d [ p ( t , x − d ) − p ( u , x − d )] d u d x − d ≲ ∣ I d ∣ β , (26)
which follows from ∣ t − u ∣ ≤ ∣ I d ∣ |t-u|\le|I_d| ∣ t − u ∣ ≤ ∣ I d ∣ . Together, these identities and the Hölder bound give ∣ p ˉ A i − p ˉ A ∣ ≲ ∣ I d ∣ β |\bar p_{A_i}-\bar p_A|\lesssim|I_d|^\beta ∣ p ˉ A i − p ˉ A ∣ ≲ ∣ I d ∣ β , which then gives
s A ( d , q ) ≲ 1 p ˉ A ∑ i = 0 1 H A i ∣ I d ∣ 2 β = H A p ˉ A ∣ I d ∣ 2 β ≲ P A ∣ I d ∣ 2 β , s_A(d,q)\lesssim\frac{1}{\bar p_A}\sum_{i=0}^{1}H_{A_i}|I_d|^{2\beta}=\frac{H_A}{\bar p_A}|I_d|^{2\beta}\lesssim P_A|I_d|^{2\beta}, s A ( d , q ) ≲ p ˉ A 1 i = 0 ∑ 1 H A i ∣ I d ∣ 2 β = p ˉ A H A ∣ I d ∣ 2 β ≲ P A ∣ I d ∣ 2 β ,
where the final inequality uses Assumption 1. Suppose now that ( d ^ , q ^ ) (\hat d,\hat q) ( d ^ , q ^ ) is a near-optimal split of A A A . By Proposition D.1, Assumption 2 holds, so the above inequality gives
S A ≲ s A ( d ^ , q ^ ) ≲ P A ∣ I d ^ ∣ 2 β . (27) S_A\lesssim s_A(\hat d,\hat q)\lesssim P_A|I_{\hat d}|^{2\beta}. \tag*{(27)} S A ≲ s A ( d ^ , q ^ ) ≲ P A ∣ I d ^ ∣ 2 β . (27)
The above inequality illustrates that when the split coordinate is narrow, Hölder continuity limits the split gain, and the remaining signal must be small.
We now leverage the above property of near-optimal splits to show Assumption 3. Fix a near-optimal tree T ∈ T ∗ T\in\mathcal{T}^* T ∈ T ∗ and a node B ∈ T B\in T B ∈ T , where B = ∏ d = 1 D J d B=\prod_{d=1}^{D}J_d B = ∏ d = 1 D J d , S B > 0 S_B>0 S B > 0 , and B ≠ A B\ne A B = A . Let d ^ \hat d d ^ be a coordinate of minimum width, so that ∣ J d ^ ∣ = min d ∣ J d ∣ |J_{\hat d}|=\min_d|J_d| ∣ J d ^ ∣ = min d ∣ J d ∣ . Naturally, we then have
H B ≥ ∣ J d ^ ∣ D . (28) H_B\ge|J_{\hat d}|^D. \tag*{(28)} H B ≥ ∣ J d ^ ∣ D . (28)
Along the path from the root to B B B in the tree T T T , let A = ∏ d = 1 D I d A=\prod_{d=1}^{D}I_d A = ∏ d = 1 D I d be the last ancestor whose near-optimal split was along coordinate d ^ \hat d d ^ . Such an ancestor exists because $B\ne A implies ∣ J d ^ ∣ < 1 |J_{\hat d}|<1 ∣ J d ^ ∣ < 1 , so this coordinate must have been split along this path. While A A A need not be the parent of B B B , the interval J d ^ J_{\hat d} J d ^ coincides with the subinterval of I d ^ I_{\hat d} I d ^ corresponding to the child of A A A that contains B B B . By the m m m -grid restriction,
∣ J d ^ ∣ ≥ ∣ I d ^ ∣ m . |J_{\hat d}| \ge\frac{|I_{\hat d}|}{m}. ∣ J d ^ ∣ ≥ m ∣ I d ^ ∣ .
Since the near-optimal split of A A A is along coordinate d ^ \hat d d ^ , (D.13) gives
∣ J d ^ ∣ ≳ ( S A P A ) 1 / ( 2 β ) . |J_{\hat d}| \gtrsim\left(\frac{S_A}{P_A}\right)^{1/(2\beta)}. ∣ J d ^ ∣ ≳ ( P A S A ) 1/ ( 2 β ) .
and Lemma C.1 and the nested variance result from Lemma D.1 (since B ⊆ A B \subseteq A B ⊆ A ) then give
S A P A ≳ σ A 2 ≳ σ B 2 ≳ S B P B . \frac{S_A}{P_A} \gtrsim\sigma_A^2 \gtrsim\sigma_B^2 \gtrsim\frac{S_B}{P_B}. P A S A ≳ σ A 2 ≳ σ B 2 ≳ P B S B .
Combining the above with (D.14) and P B ≳ H B P_B \gtrsim H_B P B ≳ H B from Assumption 1 gives
S B ≲ ( P B ) 1 + 2 β / D . S_B \lesssim(P_B)^{1+2\beta/D}. S B ≲ ( P B ) 1 + 2 β / D .
Finally, we handle the edge cases. The case S B = 0 S_B=0 S B = 0 is immediate, and the root also satisfies the desired bound, since P A = 1 P_A=1 P A = 1 and S A < ∞ S_A<\infty S A < ∞ .
Finally, we consider a minor extension to the anisotropic Hölder setting where each coordinate may have different smoothness [33 ] . Following the coordinatewise formulation of Barron et al. [ [3 ] and Bhattacharya et al. [ [4 ] , we assume that there exist a constant L ∈ ( 0 , ∞ ) L \in(0,\infty) L ∈ ( 0 , ∞ ) and exponents ( β 1 , … , β D ) ∈ ( 0 , 1 ] D (\beta_1,\ldots,\beta_D) \in(0,1]^D ( β 1 , … , β D ) ∈ ( 0 , 1 ] D such that
∣ p ( t , x − d ) − p ( u , x − d ) ∣ ≤ L ∣ t − u ∣ β d (29) |p(t,x_{-d})-p(u,x_{-d})| \le L|t-u|^{\beta_d} \tag*{(29)} ∣ p ( t , x − d ) − p ( u , x − d ) ∣ ≤ L ∣ t − u ∣ β d (29)
for every d ∈ { 1 , … , D } d \in\{1,\ldots,D\} d ∈ { 1 , … , D } , t , u ∈ [ 0 , 1 ] t,u \in[0,1] t , u ∈ [ 0 , 1 ] , and x − d ∈ [ 0 , 1 ] D − 1 x_{-d} \in[0,1]^{D-1} x − d ∈ [ 0 , 1 ] D − 1 . Fortunately, the above condition fits naturally into our proof technique, and we recover the familiar harmonic mean exponent.
Corollary D.1. Suppose Assumption 1 and the conditions of Proposition D.1 hold. If p p p additionally satisfies the anisotropic Hölder condition (D.15) with exponents ( β 1 , … , β D ) ∈ ( 0 , 1 ] D (\beta_1,\ldots,\beta_D) \in(0,1]^D ( β 1 , … , β D ) ∈ ( 0 , 1 ] D , then Assumption 3 holds with γ = 1 + 2 β 0 / D \gamma=1+2\beta_0/D γ = 1 + 2 β 0 / D , where β 0 = ( D − 1 ∑ d = 1 D β d − 1 ) − 1 \beta_0=\left(D^{-1}\sum_{d=1}^{D}\beta_d^{-1}\right)^{-1} β 0 = ( D − 1 ∑ d = 1 D β d − 1 ) − 1 .
Proof. The proof is identical to that of Proposition D.2, except for a few minor changes. First, following the section-wise definition of anisotropic Hölder smoothness, the argument in (D.12) can be tightened to have ∣ I d ∣ β d |I_d|^{\beta_d} ∣ I d ∣ β d on the right hand side, and similarly for (D.13). The other difference is the final step when choosing the coordinate d ^ \widehat{d} d for the ancestor argument. Instead of selecting d ^ \widehat{d} d such that we have minimum width, we instead choose d ^ \widehat{d} d to minimize ∣ J d ∣ β d |J_d|^{\beta_d} ∣ J d ∣ β d . This allows for coordinates with larger exponents to have wider intervals. Finally, the volume bound in (D.14) is replaced with a tighter argument. Since d ^ \widehat{d} d minimizes ∣ J d ∣ β d |J_d|^{\beta_d} ∣ J d ∣ β d , we have ∣ J d ∣ ≥ ( ∣ J d ^ ∣ β d ^ ) 1 / β d |J_d| \ge\left(|J_{\widehat{d}}|^{\beta_{\widehat{d}}}\right)^{1/\beta_d} ∣ J d ∣ ≥ ( ∣ J d ∣ β d ) 1/ β d for every d d d . Multiplying over coordinates gives
H B = ∏ d = 1 D ∣ J d ∣ ≥ ( ∣ J d ^ ∣ β d ^ ) ∑ d = 1 D 1 / β d , H_B = \prod_{d=1}^{D} |J_d| \ge\left(|J_{\widehat{d}}|^{\beta_{\widehat{d}}}\right)^{\sum_{d=1}^{D}1/\beta_d}, H B = d = 1 ∏ D ∣ J d ∣ ≥ ( ∣ J d ∣ β d ) ∑ d = 1 D 1/ β d ,
which then gives the tighter bound
min d ∣ J d ∣ β d = ∣ J d ^ ∣ β d ^ ≤ H B β 0 / D . \min_d |J_d|^{\beta_d} = |J_{\widehat{d}}|^{\beta_{\widehat{d}}} \le H_B^{\beta_0/D}. d min ∣ J d ∣ β d = ∣ J d ∣ β d ≤ H B β 0 / D .
The remainder of the proof is then straightforward with β 0 \beta_0 β 0 in place of β \beta β .
Product and additive densities
Additive case We consider the additive form first with p ( x ) = Z − 1 [ c + ∑ d = 1 D g d ( x ( d ) ) ] p(x) = Z^{-1}[c+\sum_{d=1}^{D}g_d(x^{(d)})] p ( x ) = Z − 1 [ c + ∑ d = 1 D g d ( x ( d ) )] , where Z Z Z is the normalizing constant. We use an additive variance decomposition similar to that in Mazumder and Wang [2023, Lemma B.3]. Since σ A 2 \sigma_A^2 σ A 2 is the variance under H ( ⋅ ∣ A ) H(\cdot\mid A) H ( ⋅ ∣ A ) , and the coordinates are independent under H ( ⋅ ∣ A ) H(\cdot\mid A) H ( ⋅ ∣ A ) , additivity of variance gives
σ A 2 = 1 Z 2 ∑ d = 1 D Var H [ g d ( X ( d ) ) ∣ A ] . \sigma_A^2 = \frac{1}{Z^2}\sum_{d=1}^{D}\operatorname{Var}_H[g_d(X^{(d)}) \mid A]. σ A 2 = Z 2 1 d = 1 ∑ D Var H [ g d ( X ( d ) ) ∣ A ] .
where the above notation indicates X ∼ H ( ⋅ ∣ A ) X \sim H(\cdot\mid A) X ∼ H ( ⋅ ∣ A ) . Next, we can see that
p ‾ A , d ( t ) = Z − 1 [ c A , d + g d ( t ) ] , \overline{p}_{A,d}(t) = Z^{-1}[c_{A,d}+g_d(t)], p A , d ( t ) = Z − 1 [ c A , d + g d ( t )] ,
where c A , d c_{A,d} c A , d is a constant depending on A A A and d d d , so we have
σ A , d 2 = Var H [ p ‾ A , d ( X ( d ) ) ∣ A ] = 1 Z 2 Var H [ g d ( X ( d ) ) ∣ A ] , \sigma_{A,d}^2 = \operatorname{Var}_H[\overline{p}_{A,d}(X^{(d)}) \mid A] = \frac{1}{Z^2}\operatorname{Var}_H[g_d(X^{(d)}) \mid A], σ A , d 2 = Var H [ p A , d ( X ( d ) ) ∣ A ] = Z 2 1 Var H [ g d ( X ( d ) ) ∣ A ] ,
as adding a constant to a random variable does not change its variance. We thus have σ A 2 = ∑ d = 1 D σ A , d 2 \sigma_A^2 = \sum_{d=1}^{D}\sigma_{A,d}^2 σ A 2 = ∑ d = 1 D σ A , d 2 . The above then gives max d σ A , d 2 ≥ σ A 2 / D \max_d \sigma_{A,d}^2 \ge\sigma_A^2/D max d σ A , d 2 ≥ σ A 2 / D , so Assumption D.2 is satisfied.
Next, we verify Assumption D.1. Since each g d g_d g d is absolutely continuous, the additive form immediately gives p ∈ W 1 , 1 ( ( 0 , 1 ) D ) p \in W^{1,1}((0,1)^D) p ∈ W 1 , 1 (( 0 , 1 ) D ) . Each coordinate average is also absolutely continuous, as it is an affine transformation of g d g_d g d . Next, the LRP condition is invariant under rescaling and addition of constants, so it carries over from g d g_d g d to p ‾ A , d \overline{p}_{A,d} p A , d with the same constant.
Finally, for anisotropic Hölder condition, we note that
∣ p ( t , x − d ) − p ( u , x − d ) ∣ = Z − 1 ∣ g d ( t ) − g d ( u ) ∣ , |p(t,x_{-d})-p(u,x_{-d})|=Z^{-1}|g_d(t)-g_d(u)|, ∣ p ( t , x − d ) − p ( u , x − d ) ∣ = Z − 1 ∣ g d ( t ) − g d ( u ) ∣ ,
so Corollary D.1 holds with exponents ( β 1 , … , β D ) (\beta_1,\ldots,\beta_D) ( β 1 , … , β D ) .
Product case For the product case, it is simpler to assume g d ( t ) g_d(t) g d ( t ) are densities so p ( x ) = ∏ d = 1 D g d ( x ( d ) ) p(x)=\prod_{d=1}^{D}g_d(x^{(d)}) p ( x ) = ∏ d = 1 D g d ( x ( d ) ) , and naturally Assumption 1 holds for each component. Here, it is more convenient to work with the KL divergence then use Lemma C.1 to connect it to σ A 2 \sigma_A^2 σ A 2 . Since both p ( ⋅ ∣ A ) p(\cdot\mid A) p ( ⋅ ∣ A ) and h ( ⋅ ∣ A ) h(\cdot\mid A) h ( ⋅ ∣ A ) are product densities, KL additivity gives
σ A 2 ≳ ∑ d = 1 D K { g d ( ⋅ ∣ I d ) ; Unif ( I d ) } ≳ ∑ d = 1 D Var H [ g d ( X ( d ) ) ∣ A ] , \sigma_A^2\gtrsim\sum_{d=1}^{D}K\{g_d(\cdot\mid I_d);\operatorname{Unif}(I_d)\}\gtrsim\sum_{d=1}^{D}\operatorname{Var}_{H}[g_d(X^{(d)})\mid A], σ A 2 ≳ d = 1 ∑ D K { g d ( ⋅ ∣ I d ) ; Unif ( I d )} ≳ d = 1 ∑ D Var H [ g d ( X ( d ) ) ∣ A ] ,
which follows from applications of Lemma C.1 . We can also verify that
p ˉ A , d ( t ) = c A , d g d ( t ) , \bar{p}_{A,d}(t)=c_{A,d}g_d(t), p ˉ A , d ( t ) = c A , d g d ( t ) ,
where c A , d c_{A,d} c A , d is bounded uniformly above and away from 0 from Assumption 1, since it is an average of ∏ j ≠ d D g j ( x ( j ) ) \prod_{j\ne d}^D g_j(x^{(j)}) ∏ j = d D g j ( x ( j ) ) . We thus have
σ A , d 2 = Var H [ p ˉ A , d ( X ( d ) ) ∣ A ] ≍ Var H [ g d ( X ( d ) ) ∣ A ] . \sigma_{A,d}^2=\operatorname{Var}_{H}[\bar{p}_{A,d}(X^{(d)})\mid A]\asymp\operatorname{Var}_{H}[g_d(X^{(d)})\mid A]. σ A , d 2 = Var H [ p ˉ A , d ( X ( d ) ) ∣ A ] ≍ Var H [ g d ( X ( d ) ) ∣ A ] .
We thus have σ A 2 ≳ ∑ d = 1 D σ A , d 2 \sigma_A^2\gtrsim\sum_{d=1}^{D}\sigma_{A,d}^2 σ A 2 ≳ ∑ d = 1 D σ A , d 2 , so again max d σ A , d 2 ≳ σ A 2 / D \max_d\sigma_{A,d}^2\gtrsim\sigma_A^2/D max d σ A , d 2 ≳ σ A 2 / D and Assumption D.2 holds.
Since each g d g_d g d is absolutely continuous, the product form gives
∂ p ( x ) ∂ x ( d ) = g d ′ ( x ( d ) ) ∏ j ≠ d g j ( x ( j ) ) a.e. \frac{\partial p(x)}{\partial x^{(d)}}=g_d'(x^{(d)})\prod_{j\ne d}g_j(x^{(j)})\quad\text{a.e.} ∂ x ( d ) ∂ p ( x ) = g d ′ ( x ( d ) ) j = d ∏ g j ( x ( j ) ) a.e.
The above is integrable, so p ∈ W 1 , 1 ( ( 0 , 1 ) D ) p\in W^{1,1}((0,1)^D) p ∈ W 1 , 1 (( 0 , 1 ) D ) , and each coordinate average is absolutely continuous and inherits the LRP condition from g d g_d g d again by scale invariance. Assumption D.1 thus holds.
Finally, boundedness of the components gives
∣ p ( t , x − d ) − p ( u , x − d ) ∣ = ∣ g d ( t ) − g d ( u ) ∣ ∏ j ≠ d g j ( x ( j ) ) ≲ ∣ t − u ∣ β d , |p(t,x_{-d})-p(u,x_{-d})|=|g_d(t)-g_d(u)|\prod_{j\ne d}g_j(x^{(j)})\lesssim|t-u|^{\beta_d}, ∣ p ( t , x − d ) − p ( u , x − d ) ∣ = ∣ g d ( t ) − g d ( u ) ∣ j = d ∏ g j ( x ( j ) ) ≲ ∣ t − u ∣ β d ,
where we again use Assumption 1 to handle ∏ j ≠ d g j ( x ( j ) ) \prod_{j\ne d}g_j(x^{(j)}) ∏ j = d g j ( x ( j ) ) . The anisotropic Hölder condition (D.15) thus holds with the component exponents.
Coordinate-wise monotonicity
From continuity of the partial derivative and the compact support, there exist constants such that 0 < c ≤ ∣ ∂ p ( x ) / ∂ x ( d ) ∣ ≤ C < ∞ 0 < c \le\left|\partial p(x)/\partial x^{(d)}\right| \le C < \infty 0 < c ≤ ∂ p ( x ) / ∂ x ( d ) ≤ C < ∞ for all x x x and d d d . We first show that the coordinate averages inherit these derivative bounds. Since p ∈ C 1 ( [ 0 , 1 ] D ) p \in C^1([0,1]^D) p ∈ C 1 ([ 0 , 1 ] D ) , Leibniz’s rule gives
p ˉ A , d ′ ( t ) = 1 H A − d ∫ A − d ∂ p ( t , x − d ) ∂ t d x − d \bar{p}'_{A,d}(t) = \frac{1}{H_{A-d}} \int_{A-d} \frac{\partial p(t,x_{-d})}{\partial t}\,dx_{-d} p ˉ A , d ′ ( t ) = H A − d 1 ∫ A − d ∂ t ∂ p ( t , x − d ) d x − d
for every t t t in the interior of I d I_d I d . Since the partial derivative has a fixed sign, averaging over x − d x_{-d} x − d preserves the bounds c ≤ ∣ p ˉ A , d ′ ( t ) ∣ ≤ C c \le\left|\bar{p}'_{A,d}(t)\right| \le C c ≤ p ˉ A , d ′ ( t ) ≤ C . The coordinate average is therefore monotone and continuously differentiable, so the argument in Section D.1.4 gives
σ A , d 2 ≳ ∣ I d ∣ 2 . \sigma^2_{A,d} \gtrsim|I_d|^2. σ A , d 2 ≳ ∣ I d ∣ 2 .
Next, since p ∈ C 1 ( [ 0 , 1 ] D ) p \in C^1([0,1]^D) p ∈ C 1 ([ 0 , 1 ] D ) has bounded partial derivatives, the mean value theorem gives ∣ p ( x ) − p ( y ) ∣ ≲ ∥ x − y ∥ |p(x)-p(y)| \lesssim\|x-y\| ∣ p ( x ) − p ( y ) ∣ ≲ ∥ x − y ∥ . Thus, following the argument of Proposition 1,
∣ p ( x ) − p ˉ A ∣ = 1 H A ∣ ∫ A [ p ( x ) − p ( y ) ] d y ∣ ≲ 1 H A ∫ A ∥ x − y ∥ d y ≤ ( ∑ d = 1 D ∣ I d ∣ 2 ) 1 / 2 . |p(x)-\bar{p}_A| = \frac{1}{H_A}\left|\int_A [p(x)-p(y)]\,dy\right| \lesssim\frac{1}{H_A}\int_A \|x-y\|\,dy \le\left(\sum_{d=1}^D |I_d|^2\right)^{1/2}. ∣ p ( x ) − p ˉ A ∣ = H A 1 ∫ A [ p ( x ) − p ( y )] d y ≲ H A 1 ∫ A ∥ x − y ∥ d y ≤ ( d = 1 ∑ D ∣ I d ∣ 2 ) 1/2 .
The above then implies
σ A 2 ≲ ∑ d = 1 D ∣ I d ∣ 2 ≲ ∑ d = 1 D σ A , d 2 , \sigma_A^2 \lesssim\sum_{d=1}^D |I_d|^2 \lesssim\sum_{d=1}^D \sigma^2_{A,d}, σ A 2 ≲ d = 1 ∑ D ∣ I d ∣ 2 ≲ d = 1 ∑ D σ A , d 2 ,
so we again have max d σ A , d 2 ≳ σ A 2 / D \max_d \sigma^2_{A,d} \gtrsim\sigma_A^2/D max d σ A , d 2 ≳ σ A 2 / D , giving Assumption D.2.
The LRP condition also holds for p ˉ A , d \bar{p}_{A,d} p ˉ A , d following Section D.1.4 with a constant uniform over A A A and d d d , so we have Assumption D.1. Finally, Lipschitz continuity implies Hölder continuity with exponent β = 1 \beta= 1 β = 1 .
Coordinate-wise convexity
Since the second order partial derivatives are continuous and nonzero, they must have fixed sign. As in the preceding monotonicity proof, continuity and compactness give uniform bounds on the second-order partial derivatives, which coordinate-averaging preserves as their signs are fixed. Thus, 0 < c ≤ ∣ p ˉ A , d ′ ′ ( t ) ∣ ≤ C < ∞ 0 < c \le\left|\bar{p}''_{A,d}(t)\right| \le C < \infty 0 < c ≤ p ˉ A , d ′′ ( t ) ≤ C < ∞ . We may use a simplified version of the proof of Proposition 3 for k = 2 k = 2 k = 2 , omitting the finite subcover because the second derivative is bounded away from zero uniformly, which gives
σ A , d 2 ≳ ∣ I d ∣ 4 . \sigma_{A,d}^{2} \gtrsim|I_d|^{4}. σ A , d 2 ≳ ∣ I d ∣ 4 .
The coordinate-wise convexity thus prevents interactions from canceling out all of the variation when coordinate-averaging.
Next, we require a tighter argument for connecting σ A 2 \sigma_A^2 σ A 2 to ∑ d = 1 D σ A , d 2 \sum_{d=1}^{D} \sigma_{A,d}^2 ∑ d = 1 D σ A , d 2 based on smoothness due to the above quartic lower bound, as the bound used in the previous monotonic case is too weak. We carry out a linear approximation of p ( x ) p(x) p ( x ) to connect p p p with p ˉ A , d \bar{p}_{A,d} p ˉ A , d , and use the boundedness of the Hessian to control the error. Let z z z be the centroid of A A A , which will be useful under averaging shortly. Taylor’s theorem gives
p ( x ) = p ( z ) + ∇ p ( z ) T ( x − z ) + R ( x ) , (30) p(x) = p(z) + \nabla p(z)^{\mathsf{T}}(x-z) + R(x), \tag*{(30)} p ( x ) = p ( z ) + ∇ p ( z ) T ( x − z ) + R ( x ) , (30)
where boundedness of the Hessian from p ∈ C 2 ( [ 0 , 1 ] D ) p \in C^2([0,1]^D) p ∈ C 2 ([ 0 , 1 ] D ) implies
sup x ∈ A ∣ R ( x ) ∣ ≲ ∑ d = 1 D ∣ I d ∣ 2 . \sup_{x\in A} |R(x)| \lesssim\sum_{d=1}^{D} |I_d|^2. x ∈ A sup ∣ R ( x ) ∣ ≲ d = 1 ∑ D ∣ I d ∣ 2 .
Since z z z is the center of A A A , for every coordinate d d d ,
1 ∣ I d ∣ ∫ I d ( x ( d ) − z ( d ) ) d x ( d ) = 0. \frac{1}{|I_d|}\int_{I_d}\left(x^{(d)}-z^{(d)}\right)\,dx^{(d)}=0. ∣ I d ∣ 1 ∫ I d ( x ( d ) − z ( d ) ) d x ( d ) = 0.
Thus, averaging (D.16) over x ∈ A x \in A x ∈ A removes the linear term, and averaging over x − d ∈ A − d x_{-d} \in A_{-d} x − d ∈ A − d removes all but the d d d -th linear term. Consequently, averaging over A A A and A − d A_{-d} A − d gives
p ˉ A = p ( z ) + R ˉ A , p ˉ A , d ( t ) = p ( z ) + ∂ p ( z ) ∂ z ( d ) ( t − z ( d ) ) + R d ( t ) , \bar{p}_A = p(z) + \bar{R}_A,\qquad\bar{p}_{A,d}(t) = p(z) + \frac{\partial p(z)}{\partial z^{(d)}}\left(t-z^{(d)}\right) + R_d(t), p ˉ A = p ( z ) + R ˉ A , p ˉ A , d ( t ) = p ( z ) + ∂ z ( d ) ∂ p ( z ) ( t − z ( d ) ) + R d ( t ) ,
where
R ˉ A = 1 H A ∫ A R ( x ) d x , R d ( t ) = 1 H A − d ∫ A − d R ( t , x − d ) d x − d , \bar{R}_A = \frac{1}{H_A}\int_A R(x)\,dx,\qquad R_d(t) = \frac{1}{H_{A_{-d}}}\int_{A_{-d}} R(t,x_{-d})\,dx_{-d}, R ˉ A = H A 1 ∫ A R ( x ) d x , R d ( t ) = H A − d 1 ∫ A − d R ( t , x − d ) d x − d ,
and both ∣ R ˉ A ∣ |\bar{R}_A| ∣ R ˉ A ∣ and ∣ R d ( t ) ∣ |R_d(t)| ∣ R d ( t ) ∣ are bounded by sup x ∈ A ∣ R ( x ) ∣ \sup_{x\in A}|R(x)| sup x ∈ A ∣ R ( x ) ∣ . The above then implies
p ( x ) − p ˉ A = ∑ d = 1 D [ p ˉ A , d ( x ( d ) ) − p ˉ A ] + R ( x ) − ∑ d = 1 D R d ( x ( d ) ) + ( D − 1 ) R ˉ A = ∑ d = 1 D [ p ˉ A , d ( x ( d ) ) − p ˉ A ] + R ~ A ( x ) \begin{aligned}
p(x)-\bar{p}_A = \sum_{d=1}^{D}\left[\bar{p}_{A,d}(x^{(d)})-\bar{p}_A\right] + R(x)-\sum_{d=1}^{D}R_d(x^{(d)})+(D-1)\bar{R}_A \\
&= \sum_{d=1}^{D}\left[\bar{p}_{A,d}(x^{(d)})-\bar{p}_A\right]+\widetilde{R}_A(x)
\end{aligned} p ( x ) − p ˉ A = d = 1 ∑ D [ p ˉ A , d ( x ( d ) ) − p ˉ A ] + R ( x ) − d = 1 ∑ D R d ( x ( d ) ) + ( D − 1 ) R ˉ A = d = 1 ∑ D [ p ˉ A , d ( x ( d ) ) − p ˉ A ] + R A ( x )
where the combined remainder satisfies sup x ∈ A ∣ R ~ A ( x ) ∣ ≲ ∑ d = 1 D ∣ I d ∣ 2 \sup_{x\in A}|\widetilde{R}_A(x)|\lesssim\sum_{d=1}^{D}|I_d|^2 sup x ∈ A ∣ R A ( x ) ∣ ≲ ∑ d = 1 D ∣ I d ∣ 2 by the triangle inequality. Squaring both sides, repeated application of Cauchy-Schwarz gives
[ p ( x ) − p ˉ A ] 2 ≲ ∑ d = 1 D [ p ˉ A , d ( x ( d ) ) − p ˉ A ] 2 + ∑ d = 1 D ∣ I d ∣ 4 . [p(x)-\bar p_A]^2 \lesssim\sum_{d=1}^{D}[\bar p_{A,d}(x^{(d)})-\bar p_A]^2+\sum_{d=1}^{D}|I_d|^4. [ p ( x ) − p ˉ A ] 2 ≲ d = 1 ∑ D [ p ˉ A , d ( x ( d ) ) − p ˉ A ] 2 + d = 1 ∑ D ∣ I d ∣ 4 .
Integrating the above over x ∈ A x \in A x ∈ A and dividing by H A H_A H A gives
σ A 2 ≲ ∑ d = 1 D σ A , d 2 + ∑ d = 1 D ∣ I d ∣ 4 . \sigma_A^2 \lesssim\sum_{d=1}^{D}\sigma_{A,d}^2+\sum_{d=1}^{D}|I_d|^4. σ A 2 ≲ d = 1 ∑ D σ A , d 2 + d = 1 ∑ D ∣ I d ∣ 4 .
Combining this with σ A , d 2 ≳ ∣ I d ∣ 4 \sigma_{A,d}^2 \gtrsim|I_d|^4 σ A , d 2 ≳ ∣ I d ∣ 4 yields
σ A 2 ≲ ∑ d = 1 D σ A , d 2 ≤ D max d σ A , d 2 , \sigma_A^2 \lesssim\sum_{d=1}^{D}\sigma_{A,d}^2 \leq D\max_d \sigma_{A,d}^2, σ A 2 ≲ d = 1 ∑ D σ A , d 2 ≤ D d max σ A , d 2 ,
thus establishing Assumption D.2.
Continuing with the argument of Proposition 3 gives that p ˉ A , d \bar p_{A,d} p ˉ A , d satisfies LRP with a constant uniform over A , d A,d A , d , so we have Assumption D.1. Finally, since p ∈ C 2 ( [ 0 , 1 ] D ) p \in C^2([0,1]^D) p ∈ C 2 ([ 0 , 1 ] D ) has bounded gradient, it is Lipschitz and hence Hölder continuous with exponent β = 1 \beta=1 β = 1 .
Variable sparsity
Suppose the marginal density g ( x ( 1 : D 0 ) ) g(x^{(1:D_0)}) g ( x ( 1 : D 0 ) ) satisfies Assumptions 1, 2 and 3 with some κ \kappa κ and γ \gamma γ (and some m ≥ 2 m \geq2 m ≥ 2 and δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) ). Let A = B × [ 0 , 1 ] D − D 0 A=B \times[0,1]^{D-D_0} A = B × [ 0 , 1 ] D − D 0 where B B B is a positive volume subrectangle in [ 0 , 1 ] D 0 [0,1]^{D_0} [ 0 , 1 ] D 0 . Here, quantities indexed by B B B or its descendants are computed under g g g , and near-optimal trees for p p p and g g g use the same m m m and δ \delta δ .
For x ∈ A x \in A x ∈ A , we have p ( x ) = g ( x ( 1 : D 0 ) ) p(x)=g(x^{(1:D_0)}) p ( x ) = g ( x ( 1 : D 0 ) ) and p ( x ∣ A ) = g ( x ( 1 : D 0 ) ∣ B ) p(x \mid A)=g(x^{(1:D_0)} \mid B) p ( x ∣ A ) = g ( x ( 1 : D 0 ) ∣ B ) , which then give
P A = P B , S A = S B . P_A=P_B,\qquad S_A=S_B. P A = P B , S A = S B .
Next, we consider splitting A A A at some ( d , q ) (d,q) ( d , q ) . If d ≤ D 0 d \leq D_0 d ≤ D 0 , let B 0 B_0 B 0 and B 1 B_1 B 1 be the children obtained by splitting B B B at the same ( d , q ) (d,q) ( d , q ) . The corresponding children of A A A are then A i = B i × [ 0 , 1 ] D − D 0 A_i=B_i \times[0,1]^{D-D_0} A i = B i × [ 0 , 1 ] D − D 0 for i = 0 , 1 i=0,1 i = 0 , 1 . In this case, one can similarly show that
P A i = P B i , P A i ∣ A = P B i ∣ B , H A i ∣ A = H B i ∣ B , P_{A_i}=P_{B_i},\qquad P_{A_i\mid A}=P_{B_i\mid B},\qquad H_{A_i\mid A}=H_{B_i\mid B}, P A i = P B i , P A i ∣ A = P B i ∣ B , H A i ∣ A = H B i ∣ B ,
which results in s A ( d , q ) = s B ( d , q ) s_A(d,q)=s_B(d,q) s A ( d , q ) = s B ( d , q ) . If d > D 0 d>D_0 d > D 0 , then conditional on X ∈ A X \in A X ∈ A , the independent noise X ( d ) X^{(d)} X ( d ) remains uniform on [ 0 , 1 ] [0,1] [ 0 , 1 ] under both P P P and H H H . Hence $P_{A_i\mid A}=H_{A_i\mid A} giving s A ( d , q ) = 0 s_A(d,q)=0 s A ( d , q ) = 0 . Summarizing, we have
s A ( d , q ) = { s B ( d , q ) if d ≤ D 0 , 0 otherwise. (31) s_A(d,q)=
\begin{cases}
s_B(d,q) & \text{if } d \le D_0,\\
0 & \text{otherwise.}
\end{cases} \tag*{(31)} s A ( d , q ) = { s B ( d , q ) 0 if d ≤ D 0 , otherwise. (31)
From the above, if additionally B B B belongs to a near-optimal tree for g g g , then we have
max d , q s A ( d , q ) = max d ≤ D 0 , q s B ( d , q ) ≥ κ S B = κ S A . \max_{d,q} s_A(d,q)=\max_{d\le D_0,q} s_B(d,q)\ge\kappa S_B=\kappa S_A. d , q max s A ( d , q ) = d ≤ D 0 , q max s B ( d , q ) ≥ κ S B = κ S A .
Furthermore, for γ < ∞ \gamma<\infty γ < ∞ , we have
S A = S B ≤ ( C 1 P B ) γ = ( C 1 P A ) γ . S_A=S_B\le(C_1P_B)^\gamma=(C_1P_A)^\gamma. S A = S B ≤ ( C 1 P B ) γ = ( C 1 P A ) γ .
The same argument works for γ = ∞ \gamma=\infty γ = ∞ . For a node A A A of the aforementioned form, κ \kappa κ and γ \gamma γ are thus valid for the inequalities in Assumptions 2 and 3 to hold for p p p .
We now proceed by induction to show that every node A A A in a near-optimal tree for p p p with S A > 0 S_A>0 S A > 0 has the form A = B × [ 0 , 1 ] D − D 0 A=B\times[0,1]^{D-D_0} A = B × [ 0 , 1 ] D − D 0 , where B B B belongs to a near-optimal tree for g g g . This holds at the root, with B = [ 0 , 1 ] D 0 B=[0,1]^{D_0} B = [ 0 , 1 ] D 0 . For the induction step, suppose the claim holds for a node A A A in a near-optimal tree of p p p with S A > 0 S_A>0 S A > 0 , and let ( d ^ , q ^ ) (\hat{d},\hat{q}) ( d ^ , q ^ ) be any near-optimal split of A A A . The assumption on B B B and (D.17) give
s A ( d ^ , q ^ ) ≥ ( 1 − δ ) max d , q s A ( d , q ) ≥ ( 1 − δ ) κ S A > 0. s_A(\hat{d},\hat{q})\ge(1-\delta)\max_{d,q}s_A(d,q)\ge(1-\delta)\kappa S_A>0. s A ( d ^ , q ^ ) ≥ ( 1 − δ ) d , q max s A ( d , q ) ≥ ( 1 − δ ) κ S A > 0.
Note that we must have d ^ ≤ D 0 \hat{d}\le D_0 d ^ ≤ D 0 , as s A ( d ^ , q ^ ) > 0 s_A(\hat{d},\hat{q})>0 s A ( d ^ , q ^ ) > 0 , so the children also take the form A i = B i × [ 0 , 1 ] D − D 0 A_i=B_i\times[0,1]^{D-D_0} A i = B i × [ 0 , 1 ] D − D 0 . Since (D.17) implies the split of B B B is also near-optimal for g g g , we have that B i B_i B i must belong to a near-optimal tree for g g g . Finally, by Lemma 1, every node with S A > 0 S_A>0 S A > 0 has only ancestors with S A ′ > 0 S_{A'}>0 S A ′ > 0 , so the induction covers all nodes relevant to Assumptions 2 and 3. Assumptions 2 and 3 thus hold for p p p with the same constants as for g g g , and Assumption 1 is obvious.
Quadratic examples
In this section, we discuss an illustrative quadratic example for D = 2 D=2 D = 2 . Let p p p be a general quadratic polynomial in 2 dimensions, where p > 0 p>0 p > 0 on [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 and p p p integrates to 1. From Example 9, if the coefficients for the terms ( x ( 1 ) ) 2 (x^{(1)})^2 ( x ( 1 ) ) 2 and ( x ( 2 ) ) 2 (x^{(2)})^2 ( x ( 2 ) ) 2 are non-zero, we then have greedy identifiability for sufficiently large m m m . This example shows that quadratic densities can include interactions while still satisfying marginal visibility.
A quadratic counterexample for marginal visibility is
p ( x ) = 1 + θ ( 1 − 2 x ( 1 ) ) ( 1 − 2 x ( 2 ) ) p(x) = 1 + \theta\left(1 - 2x^{(1)}\right)\left(1 - 2x^{(2)}\right) p ( x ) = 1 + θ ( 1 − 2 x ( 1 ) ) ( 1 − 2 x ( 2 ) )
for θ ∈ ( − 1 , 1 ) ∖ { 0 } \theta\in(-1,1) \setminus\{0\} θ ∈ ( − 1 , 1 ) ∖ { 0 } which is the FGM bivariate copula density. The above quadratic has coefficients zero for ( x ( 1 ) ) 2 (x^{(1)})^2 ( x ( 1 ) ) 2 and ( x ( 2 ) ) 2 (x^{(2)})^2 ( x ( 2 ) ) 2 , so does not satisfy the coordinate-wise convexity condition of Example 9 . In fact, as discussed in the main paper, the above has uniform marginals despite not being uniform on [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 , so Assumption 2 fails.
Fortunately, in this example, we can apply a transform coordinate-wise to repair this issue. Consider y ( j ) = [ exp ( λ x ( j ) ) − 1 ] / [ exp ( λ ) − 1 ] y^{(j)} = [\exp(\lambda x^{(j)}) - 1]/[\exp(\lambda) - 1] y ( j ) = [ exp ( λ x ( j ) ) − 1 ] / [ exp ( λ ) − 1 ] for λ > 0 \lambda> 0 λ > 0 , which maps each coordinate from [ 0 , 1 ] → [ 0 , 1 ] [0,1] \to[0,1] [ 0 , 1 ] → [ 0 , 1 ] . Let p Y p_Y p Y be the transformed density which is no longer a copula, and suppose H H H is chosen to still be uniform on the y y y scale. One can then verify with the chain rule that ∂ 2 p Y ( y ) / ∂ ( y ( 1 ) ) 2 \partial^2p_Y(y)/\partial(y^{(1)})^2 ∂ 2 p Y ( y ) / ∂ ( y ( 1 ) ) 2 is a positive term times
λ + θ ( 1 − 2 x ( 2 ) ) [ λ ( 1 − 2 x ( 1 ) ) + 3 ] \lambda+ \theta(1 - 2x^{(2)})[\lambda(1 - 2x^{(1)}) + 3] λ + θ ( 1 − 2 x ( 2 ) ) [ λ ( 1 − 2 x ( 1 ) ) + 3 ]
where x ( j ) x^{(j)} x ( j ) is the suitable inverse transform of y ( j ) y^{(j)} y ( j ) . The above term is lower bounded by λ − ∣ θ ∣ ( 3 + λ ) \lambda- |\theta|(3 + \lambda) λ − ∣ θ ∣ ( 3 + λ ) , and is positive for all y y y if λ > 3 ∣ θ ∣ / ( 1 − ∣ θ ∣ ) \lambda> 3|\theta|/(1 - |\theta|) λ > 3∣ θ ∣/ ( 1 − ∣ θ ∣ ) . Symmetry gives the same bound for y ( 2 ) y^{(2)} y ( 2 ) , which is then sufficient for the coordinate-wise convexity condition of Example 9 .
Technical results
This section develops the complexity and probability bounds, together with the log marginal likelihood expansion, used in the proofs of the main theorems. Throughout this section, we assume that Assumptions 1, 2, and 3 hold, with m ≥ 2 m \ge2 m ≥ 2 , δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) and all constants in the assumptions fixed independently of n n n .
Admissible family complexity
In this section, we analyze the complexity of the admissible family by considering the contributions from the identifiable and non-identifiable parts separately. For the identifiable part, we account for the multiple possible near-optimal splitting paths. For the non-identifiable part, we use the gap between strong- and weak-signal nodes to choose a fixed threshold k max k_{\max} k m a x for the number of excess splits. Combining these results with the boundedness of k max k_{\max} k m a x , we obtain a bound on the complexity of admissible partitions.
For the remainder of the proof, it is convenient to write the identifiability threshold as
τ n = C 0 log n n , \tau_n = \frac{C_0 \log n}{n}, τ n = n C 0 log n ,
where C 0 > 0 C_0 > 0 C 0 > 0 is a sufficiently large constant. We will initially regard C 0 C_0 C 0 and the positive integer k max k_{\max} k m a x as fixed constants and specify their specific choices shortly below.
Oracle nodes
We begin by defining the set of oracle nodes below which will be useful for later parts of the proof, before analyzing their complexity in Proposition E.1.
Definition E.1 (Oracle nodes). For all admissible trees T ∈ T n T \in\mathcal{T}_n T ∈ T n , truncate each branch at its first non-identifiable node, and let T n ∗ \mathcal{T}_n^* T n ∗ and L n ∗ \mathcal{L}_n^* L n ∗ be the collections of all internal nodes and leaves respectively of all resulting trees. We refer to T n ∗ \mathcal{T}_n^* T n ∗ as the identifiable oracle nodes, and L n ∗ \mathcal{L}_n^* L n ∗ as the non-identifiable oracle leaves.
Figure E.1 shows the oracle nodes contributed by one truncated admissible tree, as described in the first part of Definition 2. We now have the complexity bound.
Figure E.1. Oracle nodes contributed by a truncated admissible tree. Shaded internal nodes belong to T n ∗ \mathcal{T}_n^* T n ∗ , and unshaded leaves belong to L n ∗ \mathcal{L}_n^* L n ∗ .
Proposition E.1 (Complexity exponents). For n ≥ 2 n \ge2 n ≥ 2 , the oracle nodes satisfy the inequality
∣ T n ∗ ∣ + ∣ L n ∗ ∣ ≲ τ n − t 0 , \lvert\mathcal{T}_n^* \rvert+ \lvert\mathcal{L}_n^* \rvert\lesssim\tau_n^{-t_0}, ∣ T n ∗ ∣ + ∣ L n ∗ ∣ ≲ τ n − t 0 , where the exponent is
t 0 = log [ 2 D ( m − 1 ) ] log [ m / ( m − 1 ) ] (32) t_0 = \frac{\log[2D(m-1)]}{\log[m/(m-1)]} \tag*{(32)} t 0 = log [ m / ( m − 1 )] log [ 2 D ( m − 1 )] (32)
Proof. Every grid split reduces the volume by a factor at most ( m − 1 ) / m (m-1)/m ( m − 1 ) / m , so a node A A A at depth ℓ \ell ℓ satisfies H A ≤ [ ( m − 1 ) / m ] ℓ H_A \le[(m-1)/m]^\ell H A ≤ [( m − 1 ) / m ] ℓ . Assumption 1 gives S A ≤ P A log ( C u / C l ) ≲ H A S_A \le P_A \log(C_u/C_l) \lesssim H_A S A ≤ P A log ( C u / C l ) ≲ H A . Hence, if A A A is identifiable, we have τ n ≤ S A ≤ [ ( m − 1 ) / m ] ℓ \tau_n \le S_A \le[(m-1)/m]^\ell τ n ≤ S A ≤ [( m − 1 ) / m ] ℓ . Taking logarithms yields
ℓ ≤ log τ n − 1 log [ m / ( m − 1 ) ] + C , \ell\le\frac{\log\tau_n^{-1}}{\log[m/(m-1)]} + C, ℓ ≤ log [ m / ( m − 1 )] log τ n − 1 + C ,
where C > 0 C > 0 C > 0 is independent of n n n . Every non-identifiable oracle leaf A ∈ L n ∗ A \in\mathcal{L}_n^* A ∈ L n ∗ that is not the root has an identifiable parent in I n ∗ \mathcal{I}_n^* I n ∗ , so the same bound holds for oracle leaves after increasing C C C by one. The bound also holds for the root, since τ n ≤ C 0 / e ≤ 1 \tau_n \le C_0/e \le1 τ n ≤ C 0 / e ≤ 1 for n ≥ 2 n \ge2 n ≥ 2 . Thus, the maximum depth ℓ n \ell_n ℓ n attained by an oracle node is finite and satisfies
ℓ n ≤ log τ n − 1 log [ m / ( m − 1 ) ] + C . \ell_n \le\frac{\log\tau_n^{-1}}{\log[m/(m-1)]} + C. ℓ n ≤ log [ m / ( m − 1 )] log τ n − 1 + C .
Across all candidate trees, there are at most [ 2 D ( m − 1 ) ] ℓ [2D(m-1)]^\ell [ 2 D ( m − 1 ) ] ℓ distinct nodes at depth ℓ \ell ℓ . Summing over depths and applying the geometric series formula therefore gives
∣ I n ∗ ∣ + ∣ L n ∗ ∣ ≤ ∑ ℓ = 0 ℓ n [ 2 D ( m − 1 ) ] ℓ ≲ [ 2 D ( m − 1 ) ] ℓ n . |\mathcal{I}_n^*| + |\mathcal{L}_n^*| \le\sum_{\ell=0}^{\ell_n} [2D(m-1)]^\ell\lesssim[2D(m-1)]^{\ell_n}. ∣ I n ∗ ∣ + ∣ L n ∗ ∣ ≤ ℓ = 0 ∑ ℓ n [ 2 D ( m − 1 ) ] ℓ ≲ [ 2 D ( m − 1 ) ] ℓ n .
Applying the upper bound on ℓ n \ell_n ℓ n then gives the result. □ \square □
While the above bound works for any C 0 C_0 C 0 , we now specify a specific value for the remainder of the proofs. We will see that a sufficiently large choice is
C 0 = 864 C u C l t 0 κ δ 2 (33) C_0 = 864\frac{C_u}{C_l}\frac{t_0}{\kappa\delta^2} \tag*{(33)} C 0 = 864 C l C u κ δ 2 t 0 (33)
where ( C l , C u ) (C_l,C_u) ( C l , C u ) and κ \kappa κ are from Assumptions 1 and 2 respectively, and δ \delta δ is from Definition 1. Note that since t 0 ≥ 1 t_0 \ge1 t 0 ≥ 1 , our specific choice satisfies C 0 > 1 C_0 > 1 C 0 > 1 .
Excess splits
Next, we handle the additional complexity induced by the excess splitting on non-identifiable nodes in the admissible trees. We first introduce the notion of a stopping split , which mirrors the definition of identifiable nodes in (10), but is an upper threshold for weak split gains.
Definition E.2 (Stopping split). A split ( d , q ) (d,q) ( d , q ) on A A A is a stopping split if
s A ( d , q ) ≤ C s log n n , s_A(d,q) \le\frac{C_s\log n}{n}, s A ( d , q ) ≤ n C s log n ,
where the constant takes the form
C s = 1 40 C l C u . C_s = \frac{1}{40}\frac{C_l}{C_u}. C s = 40 1 C u C l .
Intuitively, a stopping split is one where the split gain is too weak relative to the penalty at a given n n n , so the penalty dominates. Again, we have chosen a specific value for convenience which will be used later. For now, we highlight that C s < 1 < C 0 C_s < 1 < C_0 C s < 1 < C 0 , so the stopping signal threshold is below the identifiable one.
We now leverage the limited amount of splitting that occurs between the identifiability and stopping thresholds to control the non-identifiable part of admissible trees.
Lemma E.1 (Splitting beyond identifiability). Let k max k_{\max} k m a x be the integer defined as
k max = ⌈ C 0 C s ⌉ + ⌈ 15 t 0 ⌉ (34) k_{\max} = \left\lceil\frac{C_0}{C_s} \right\rceil+ \left\lceil15t_0 \right\rceil \tag*{(34)} k m a x = ⌈ C s C 0 ⌉ + ⌈ 15 t 0 ⌉ (34) where t 0 t_0 t 0 is defined in (E.1). Then any subtree rooted at a non-identifiable node A A A with k max k_{\max} k m a x splits contains at least ⌈ 15 t 0 ⌉ \left\lceil15t_0 \right\rceil ⌈ 15 t 0 ⌉ stopping splits.
Proof. Since A A A is non-identifiable, (10) upper bounds its signal with
S A < C 0 log n n . S_A < \frac{C_0 \log n}{n}. S A < n C 0 log n .
For any non-stopping split within a node B ⊆ A B \subseteq A B ⊆ A , Definition E.2 guarantees that the split gain satisfies
s B ( d , q ) > C s log n n . s_B(d,q) > \frac{C_s \log n}{n}. s B ( d , q ) > n C s log n .
By the signal decomposition of Lemma 1, the sum of all split gains in the subtree is at most S A S_A S A . Hence the number of non-stopping splits is strictly less than C 0 / C s C_0/C_s C 0 / C s . The remaining splits must thus be stopping splits.
The choice ⌈ 15 t 0 ⌉ \left\lceil15t_0 \right\rceil ⌈ 15 t 0 ⌉ allows the later probability bounds to offset the oracle node complexity τ n − t 0 \tau_n^{-t_0} τ n − t 0 . Finally, we bound the number of leaves of an admissible tree.
Lemma E.2 (Leaf count for admissible trees). Let T ∈ T n T \in\mathcal{T}_n T ∈ T n be any admissible tree, and let P \mathcal{P} P be its leaf partition. For n ≥ 2 n \ge2 n ≥ 2 , we have
∣ P ∣ ≲ τ n − 1 / γ , (35) |\mathcal{P}| \lesssim\tau_n^{-1/\gamma}, \tag*{(35)} ∣ P ∣ ≲ τ n − 1/ γ , (35) where the implicit constant is independent of T T T and n n n .
Proof. Consider truncating each branch of T T T at its first non-identifiable node, as in Definition E.1. Let L L L be the leaf partition of the resulting tree. If the root is identifiable, every B ∈ L B \in L B ∈ L has an identifiable parent A A A . The grid restriction and Assumption 1 give H B ≥ H A / m ≳ P A H_B \ge H_A/m \gtrsim P_A H B ≥ H A / m ≳ P A . For γ < ∞ \gamma< \infty γ < ∞ , identifiability and Assumption 3 imply τ n ≤ S A ≤ ( C 1 P A ) γ \tau_n \le S_A \le(C_1P_A)^\gamma τ n ≤ S A ≤ ( C 1 P A ) γ , so P A ≳ τ n 1 / γ P_A \gtrsim\tau_n^{1/\gamma} P A ≳ τ n 1/ γ . For γ = ∞ \gamma= \infty γ = ∞ , the same conclusion follows from P A ≥ C 1 − 1 P_A \ge C_1^{-1} P A ≥ C 1 − 1 . Thus every B ∈ L B \in L B ∈ L satisfies
H B ≳ τ n 1 / γ . H_B \gtrsim\tau_n^{1/\gamma}. H B ≳ τ n 1/ γ .
If the root is non-identifiable, then L = { A } L = \{A\} L = { A } , and this bound again holds trivially for n ≥ 2 n \ge2 n ≥ 2 . Since L L L partitions A \mathcal{A} A , we have
1 = ∑ B ∈ L H B ≳ ∣ L ∣ τ n 1 / γ , 1 = \sum_{B \in L} H_B \gtrsim|L|\tau_n^{1/\gamma}, 1 = B ∈ L ∑ H B ≳ ∣ L ∣ τ n 1/ γ ,
and hence ∣ L ∣ ≲ τ n − 1 / γ |L| \lesssim\tau_n^{-1/\gamma} ∣ L ∣ ≲ τ n − 1/ γ . By admissibility, each B ∈ L B \in L B ∈ L undergoes fewer than k max k_{\max} k m a x additional splits, producing at most k max k_{\max} k m a x leaves. Since k max k_{\max} k m a x is fixed, we have
∣ P ∣ ≤ k max ∣ L ∣ ≲ τ n − 1 / γ . |\mathcal{P}| \le k_{\max}|L| \lesssim\tau_n^{-1/\gamma}. ∣ P ∣ ≤ k m a x ∣ L ∣ ≲ τ n − 1/ γ .
Partition complexity
Lemma E.2 controls the leaf count uniformly over all admissible trees. We now control the complexity of the possible partitions across these trees.
Lemma E.3 (Partition complexity). Let B n \mathcal{B}_n B n be the collection of all sets that can be expressed as a union of leaves of a single admissible tree T ∈ T n T \in\mathcal{T}_n T ∈ T n . For n ≥ 2 n \ge2 n ≥ 2 , we have
log ∣ B n ∣ ≲ τ n − 1 / γ log n . (36) \log|\mathcal{B}_n| \lesssim\tau_n^{-1/\gamma}\log n. \tag*{(36)} log ∣ B n ∣ ≲ τ n − 1/ γ log n . (36) Proof. Starting from a node A A A , consider an ordered sequence of k k k grid splits, each acting on a current leaf. Before the j j j -th split, there are exactly j j j leaves, each admitting D ( m − 1 ) D(m-1) D ( m − 1 ) candidate splits. Hence the number of such sequences is at most
∏ j = 1 k j D ( m − 1 ) = k ! [ D ( m − 1 ) ] k . (37) \prod_{j=1}^{k} jD(m-1) = k![D(m-1)]^k. \tag*{(37)} j = 1 ∏ k j D ( m − 1 ) = k ! [ D ( m − 1 ) ] k . (37) This also holds for k = 0 k = 0 k = 0 , when there is one empty sequence.
For the global bound, let k n k_n k n be the maximum number of leaves of an admissible tree, where Lemma E.2 gives k n ≲ τ n − 1 / γ k_n \lesssim\tau_n^{-1/\gamma} k n ≲ τ n − 1/ γ if n ≥ 2 n \ge2 n ≥ 2 . Each admissible tree has at most $k_n - 1
internal nodes. For each k = 0 , … , k n − 1 k = 0,\ldots,k_n - 1 k = 0 , … , k n − 1 , there are at most k ! [ D ( m − 1 ) ] k k![D(m - 1)]^k k ! [ D ( m − 1 ) ] k ordered split sequences, each producing k + 1 k + 1 k + 1 leaves and 2 k + 1 2^{k+1} 2 k + 1 cell unions. Therefore,
∣ B n ∣ ≤ ∑ k = 0 k n − 1 k ! [ D ( m − 1 ) ] k 2 k + 1 ≤ k n ! [ D ( m − 1 ) ] k n 2 k n , |\mathcal{B}_n| \le\sum_{k=0}^{k_n-1} k![D(m-1)]^k2^{k+1} \le k_n![D(m-1)]^{k_n}2^{k_n}, ∣ B n ∣ ≤ k = 0 ∑ k n − 1 k ! [ D ( m − 1 ) ] k 2 k + 1 ≤ k n ! [ D ( m − 1 ) ] k n 2 k n ,
where the last inequality follows by bounding each of the k n k_n k n summands by the largest term and using D ( m − 1 ) ≥ 1 D(m - 1) \ge1 D ( m − 1 ) ≥ 1 . Taking logarithms gives
log ∣ B n ∣ ≤ k n log { 2 D ( m − 1 ) k n } ≲ τ n − 1 / γ log n , \log|\mathcal{B}_n| \le k_n \log\{2D(m-1)k_n\} \lesssim\tau_n^{-1/\gamma}\log n, log ∣ B n ∣ ≤ k n log { 2 D ( m − 1 ) k n } ≲ τ n − 1/ γ log n ,
where the first inequality used log ( k n ! ) ≤ k n log k n \log(k_n!) \le k_n \log k_n log ( k n !) ≤ k n log k n .
Nodewise decision conditions
This section contains a technical study of the decomposition (9), where we formally control the size of the error. We then summarize the sufficient conditions for near-optimal splitting and stopping based on the decomposition of the split Bayes factor.
Key events
To control the error term in (9), we will need to define useful events on the counts within a node A A A . To that end, for a node A A A with split ( d , q ) (d,q) ( d , q ) and left child A 0 A_0 A 0 , we write the empirical estimates of the node probabilities for N A > 0 N_A > 0 N A > 0 as
P ^ A = N A n , P ^ A 0 ∣ A = N A 0 N A . \widehat{P}_A = \frac{N_A}{n}, \qquad\widehat{P}_{A_0\mid A} = \frac{N_{A_0}}{N_A}. P A = n N A , P A 0 ∣ A = N A N A 0 .
where N A = ∑ i = 1 n 1 ( X i ∈ A ) N_A = \sum_{i=1}^{n}\mathbf{1}(X_i \in A) N A = ∑ i = 1 n 1 ( X i ∈ A ) with X i ∼ i . i . d . P X_i \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} P X i ∼ i.i.d. P . For convenience, let P ^ A = P ^ A 0 ∣ A = 0 \widehat{P}_A = \widehat{P}_{A_0\mid A} = 0 P A = P A 0 ∣ A = 0 if N A = 0 N_A = 0 N A = 0 . For a given A A A and ( d , q ) (d,q) ( d , q ) , in the events below, N A N_A N A , N A 0 N_{A_0} N A 0 , P ^ A \widehat{P}_A P A and P ^ A 0 ∣ A \widehat{P}_{A_0\mid A} P A 0 ∣ A are random variables.
Definition E.3 (Node count event). For a node A A A with P A > 0 P_A > 0 P A > 0 , let
E A , n : = { ∣ P ^ A P A − 1 ∣ ≤ 1 log n } . \mathcal{E}_{A,n} := \left\{\left|\frac{\widehat{P}_A}{P_A}-1\right| \le\frac{1}{\log n}\right\}. E A , n := { P A P A − 1 ≤ log n 1 } .
Definition E.4 (Split count event). Let A A A be a node with P A > 0 P_A > 0 P A > 0 , and ( d , q ) (d,q) ( d , q ) be a split of A A A resulting in left child A 0 A_0 A 0 . For a finite constant t > 0 t > 0 t > 0 , let
D A , n ( t , d , q ) : = { N A > 0 , ( P ^ A 0 ∣ A − P A 0 ∣ A ) 2 P A 0 ∣ A ( 1 − P A 0 ∣ A ) < 6 t log n N A } , D ~ A , n ( t ) : = ⋂ d , q D A , n ( t , d , q ) , \mathcal{D}_{A,n}(t,d,q) := \left\{N_A > 0,\ \frac{\left(\widehat{P}_{A_0\mid A}-P_{A_0\mid A}\right)^2}{P_{A_0\mid A}(1-P_{A_0\mid A})} < 6t\frac{\log n}{N_A}\right\},\qquad\widetilde{\mathcal{D}}_{A,n}(t) := \bigcap_{d,q}\mathcal{D}_{A,n}(t,d,q), D A , n ( t , d , q ) := ⎩ ⎨ ⎧ N A > 0 , P A 0 ∣ A ( 1 − P A 0 ∣ A ) ( P A 0 ∣ A − P A 0 ∣ A ) 2 < 6 t N A log n ⎭ ⎬ ⎫ , D A , n ( t ) := d , q ⋂ D A , n ( t , d , q ) , where the intersection ranges over all candidate m m m -grid splits of A A A .
The first event controls the accuracy of the empirical node probability, while the split count events control the empirical conditional child probabilities. Note that D ~ A , n ( t ) \widetilde{\mathcal{D}}_{A,n}(t) D A , n ( t ) corresponds to the inequality holding uniformly over all candidate splits. The choice of t t t will aid with technical proofs later, with splitting using t = t 0 t=t_0 t = t 0 and stopping using the smaller t = 1 / 15 t=1/15 t = 1/15 . Section E.3 studies the probability of the above events.
We now state a lemma summarizing some regularity properties under the above events, which is useful for later proofs.
Lemma E.4. Let t > 0 t > 0 t > 0 be a fixed constant. There exists a constant b ∈ ( 0 , 1 / 2 ) b \in(0,1/2) b ∈ ( 0 , 1/2 ) such that, for sufficiently large n n n , the following properties hold uniformly over nodes A A A of admissible trees and candidate splits ( d , q ) (d,q) ( d , q ) of $A.
The node probabilities satisfy the deterministic bounds
P A ≳ ( log n ) 5 n , P A 0 ∣ A ∈ [ b , 1 − b ] . (38) P_A \gtrsim\frac{(\log n)^5}{n},\qquad P_{A_0\mid A}\in[b,1-b]. \tag*{(38)} P A ≳ n ( log n ) 5 , P A 0 ∣ A ∈ [ b , 1 − b ] . (38) On the event E A , n \mathcal{E}_{A,n} E A , n , we then have
P ^ A ≳ ( log n ) 5 n . (39) \widehat{P}_A \gtrsim\frac{(\log n)^5}{n}. \tag*{(39)} P A ≳ n ( log n ) 5 . (39) Finally, on E A , n ∩ D A , n ( t , d , q ) \mathcal{E}_{A,n}\cap\mathcal{D}_{A,n}(t,d,q) E A , n ∩ D A , n ( t , d , q ) , we have
∣ P ^ A 0 ∣ A − P A 0 ∣ A ∣ ≲ ( log n ) − 2 , P ^ A 0 ∣ A ∈ [ b , 1 − b ] . (40) \left|\widehat{P}_{A_0\mid A}-P_{A_0\mid A}\right|\lesssim(\log n)^{-2},\qquad\widehat{P}_{A_0\mid A}\in[b,1-b]. \tag*{(40)} P A 0 ∣ A − P A 0 ∣ A ≲ ( log n ) − 2 , P A 0 ∣ A ∈ [ b , 1 − b ] . (40)
Proof. We begin with the deterministic results. Let B B B be the last identifiable node on the path from the root to A A A , including A A A itself. If there is no such node, take B B B to be the root. In either case, P B ≳ τ n 1 / γ P_B\gtrsim\tau_n^{1/\gamma} P B ≳ τ n 1/ γ , as for an identifiable B B B this follows from S B ≥ τ n S_B\geq\tau_n S B ≥ τ n and Assumption 3, while for the root P B = 1 P_B=1 P B = 1 .
By Definition 2, the path from B B B to A A A contains at most k max k_{\max} k m a x splits. Each split retains at least 1 / m 1/m 1/ m of the parent volume, so Assumption 1 gives
P A ≥ C l C u m − k max P B ≳ τ n 1 / γ . P_A \ge\frac{C_l}{C_u}m^{-k_{\max}}P_B \gtrsim\tau_n^{1/\gamma}. P A ≥ C u C l m − k m a x P B ≳ τ n 1/ γ .
Since τ n = C 0 log n / m \tau_n=C_0\log n/m τ n = C 0 log n / m and 1 / γ < 1 1/\gamma<1 1/ γ < 1 , we have n τ n 1 / γ / ( log n ) 5 → ∞ n\tau_n^{1/\gamma}/(\log n)^5\to\infty n τ n 1/ γ / ( log n ) 5 → ∞ . Hence, for sufficiently large n n n , we have P A ≳ ( log n ) 5 / n P_A\gtrsim(\log n)^5/n P A ≳ ( log n ) 5 / n uniformly over admissible nodes. Next, the m m m -grid restriction and Assumption 1 imply
C l m C u ≤ P A 0 ∣ A ≤ 1 − C l m C u . \frac{C_l}{mC_u}\le P_{A_0\mid A}\le1-\frac{C_l}{mC_u}. m C u C l ≤ P A 0 ∣ A ≤ 1 − m C u C l .
We now proceed to the implications of the events. On E A , n \mathcal{E}_{A,n} E A , n , we have P ^ A / P A ∈ [ 1 − 1 / log n , 1 + 1 / log n ] \widehat{P}_A/P_A\in[1-1/\log n,1+1/\log n] P A / P A ∈ [ 1 − 1/ log n , 1 + 1/ log n ] , so for n ≥ 3 n\ge3 n ≥ 3 , we have P ^ A ≍ P A \widehat{P}_A\asymp P_A P A ≍ P A . Combining this with the preceding probability bound gives (E.8) uniformly over A A A for sufficiently large n n n .
If D A , n ( t , d , q ) \mathcal{D}_{A,n}(t,d,q) D A , n ( t , d , q ) also holds, P A 0 ∣ A ( 1 − P A 0 ∣ A ) ≤ 1 / 4 P_{A_0\mid A}(1-P_{A_0\mid A})\le1/4 P A 0 ∣ A ( 1 − P A 0 ∣ A ) ≤ 1/4 gives
∣ P ^ A 0 ∣ A − P A 0 ∣ A ∣ ≤ 3 t log n 2 N A ≲ ( log n ) − 2 \left|\widehat{P}_{A_0\mid A}-P_{A_0\mid A}\right|
\le\sqrt{\frac{3t\log n}{2N_A}}\lesssim(\log n)^{-2} P A 0 ∣ A − P A 0 ∣ A ≤ 2 N A 3 t log n ≲ ( log n ) − 2
since N A = n P ^ A ≳ ( log n ) 5 N_A=n\widehat{P}_A\gtrsim(\log n)^5 N A = n P A ≳ ( log n ) 5 on E A , n \mathcal{E}_{A,n} E A , n by (E.8), and t > 0 t>0 t > 0 is fixed. Finally, ∣ P ^ A 0 ∣ A − P A 0 ∣ A ∣ \left|\widehat{P}_{A_0\mid A}-P_{A_0\mid A}\right| P A 0 ∣ A − P A 0 ∣ A will eventually be sufficiently small uniformly over A A A and ( d , q ) (d,q) ( d , q ) , so we can choose b = C l / ( 2 m C u ) b=C_l/(2mC_u) b = C l / ( 2 m C u ) .
Decomposition error control
We now turn to the key technical lemma for controlling the error term in the split Bayes factor expansion from (9).
Lemma E.5. Fix a constant 0 < t ≤ t 0 0<t\le t_0 0 < t ≤ t 0 . For sufficiently large n n n , the following bound holds uniformly over nodes A A A of an admissible tree T ∈ T n T\in\mathcal{T}_n T ∈ T n and candidate splits ( d , q ) (d,q) ( d , q ) whenever the event E A , n ∩ D A , n ( t , d , q ) \mathcal{E}_{A,n}\cap\mathcal{D}_{A,n}(t,d,q) E A , n ∩ D A , n ( t , d , q ) holds:
∣ ε A , n ( d , q ) ∣ ≤ 2 3 ( C u C l ) 1 / 2 s A ( d , q ) t P A log n N A + 3 t P A log n N A + C n , |\varepsilon_{A,n}(d,q)|\le2\sqrt{3}\left(\frac{C_u}{C_l}\right)^{1/2}\sqrt{s_A(d,q)}\sqrt{\frac{tP_A\log n}{N_A}}+3\frac{tP_A\log n}{N_A}+\frac{C}{n}, ∣ ε A , n ( d , q ) ∣ ≤ 2 3 ( C l C u ) 1/2 s A ( d , q ) N A t P A log n + 3 N A t P A log n + n C , where C l , C u C_l,C_u C l , C u are from Assumption 1, and C C C is independent of A A A , ( d , q ) (d,q) ( d , q ) and n n n .
Proof. Throughout the proof, we work on E A , n ∩ D A , n ( t , d , q ) \mathcal{E}_{A,n}\cap\mathcal{D}_{A,n}(t,d,q) E A , n ∩ D A , n ( t , d , q ) . All thresholds on n n n below can be chosen independently of A A A and ( d , q ) (d,q) ( d , q ) , since the bounds and constants used are uniform.
To recover the split gain and penalty in (9) , we first approximate the two beta functions in (4) , which will also yield the expression for ε A , n ( d , q ) \varepsilon_{A,n}(d,q) ε A , n ( d , q ) . We then bound its components separately under the count events and apply the triangle inequality to give the target bound.
For the prior contribution term − log B ( α H A 0 , α H A 1 ) -\log B(\alpha H_{A_0},\alpha H_{A_1}) − log B ( α H A 0 , α H A 1 ) , we use properties of the gamma function. Writing z = α H A z=\alpha H_A z = α H A and r = H A 0 ∣ A r=H_{A_0|A} r = H A 0 ∣ A , the identity Γ ( 1 + z ) = z Γ ( z ) \Gamma(1+z)=z\Gamma(z) Γ ( 1 + z ) = z Γ ( z ) gives
log Γ ( z ) Γ ( z r ) Γ ( z ( 1 − r ) ) = log H A + log { α r ( 1 − r ) Γ ( 1 + z ) Γ ( 1 + z r ) Γ ( 1 + z ( 1 − r ) ) } . \log\frac{\Gamma(z)}{\Gamma(zr)\Gamma(z(1-r))}=\log H_A+\log\left\{\frac{\alpha r(1-r)\,\Gamma(1+z)}{\Gamma(1+zr)\Gamma(1+z(1-r))}\right\}. log Γ ( z r ) Γ ( z ( 1 − r )) Γ ( z ) = log H A + log { Γ ( 1 + z r ) Γ ( 1 + z ( 1 − r )) α r ( 1 − r ) Γ ( 1 + z ) } .
The first term is our approximation, while the second term is continuous on the compact set [ 0 , α ] × [ 1 / m , 1 − 1 / m ] [0,\alpha]\times[1/m,1-1/m] [ 0 , α ] × [ 1/ m , 1 − 1/ m ] as a function of ( z , r ) (z,r) ( z , r ) , so its absolute value is bounded by a constant depending only on m m m and α \alpha α .
For the numerator beta function, Stirling’s formula for the gamma function gives
log B ( α H A 0 + N A 0 , α H A 1 + N A 1 ) = [ ∑ i = 0 1 N A i log N A i − N A log N A ] + [ ∑ i = 0 1 ( α H A i − 1 2 ) log N A i − ( α H A − 1 2 ) log N A ] + R A , n ( d , q ) . \begin{aligned}
\log B(\alpha H_{A_0}+N_{A_0},\alpha H_{A_1}+N_{A_1})
={}&\left[\sum_{i=0}^{1}N_{A_i}\log N_{A_i}-N_A\log N_A\right]\\
&+\left[\sum_{i=0}^{1}\left(\alpha H_{A_i}-\frac{1}{2}\right)\log N_{A_i}-\left(\alpha H_A-\frac{1}{2}\right)\log N_A\right]+R_{A,n}(d,q).
\end{aligned} log B ( α H A 0 + N A 0 , α H A 1 + N A 1 ) = [ i = 0 ∑ 1 N A i log N A i − N A log N A ] + [ i = 0 ∑ 1 ( α H A i − 2 1 ) log N A i − ( α H A − 2 1 ) log N A ] + R A , n ( d , q ) .
Henceforth for convenience, the notation R A , n ( d , q ) R_{A,n}(d,q) R A , n ( d , q ) is used to denote a remainder that is bounded uniformly in A A A , ( d , q ) (d,q) ( d , q ) , and n n n , where its meaning may change between lines. In the above, the remainder absorbs the constant 1 2 log ( 2 π ) \frac{1}{2}\log(2\pi) 2 1 log ( 2 π ) , the errors in Stirling’s approximation, and the errors from replacing log ( N B + α H B ) \log(N_B+\alpha H_B) log ( N B + α H B ) with log N B \log N_B log N B for B ∈ { A 0 , A 1 , A } B\in\{A_0,A_1,A\} B ∈ { A 0 , A 1 , A } .
We now proceed to show the remainder is indeed uniformly bounded. First, (E.8) and (E.9) ensure that N B ≳ ( log n ) 5 N_B\gtrsim(\log n)^5 N B ≳ ( log n ) 5 uniformly for sufficiently large n n n . The absolute error in Stirling’s expansion of log Γ ( N B + α H B ) \log\Gamma(N_B+\alpha H_B) log Γ ( N B + α H B ) is bounded by
1 12 ( N B + α H B ) ≲ ( log n ) − 5 , \frac{1}{12(N_B+\alpha H_B)}\lesssim(\log n)^{-5}, 12 ( N B + α H B ) 1 ≲ ( log n ) − 5 ,
and is therefore uniformly bounded. Additionally, we now show that replacing log ( N B + α H B ) \log(N_B+\alpha H_B) log ( N B + α H B ) with log N B \log N_B log N B incurs a uniformly bounded error. For sufficiently large n n n , the uniform count lower bound ensures N B ≥ max { 1 , α } N_B\ge\max\{1,\alpha\} N B ≥ max { 1 , α } . Since H B ≤ 1 H_B\le1 H B ≤ 1 , we have N B + α H B − 1 2 > 0 N_B+\alpha H_B-\frac{1}{2}>0 N B + α H B − 2 1 > 0 and α H B ≤ N B \alpha H_B\le N_B α H B ≤ N B . Using log ( 1 + x ) ≤ x \log(1+x)\le x log ( 1 + x ) ≤ x for x ≥ 0 x\ge0 x ≥ 0 , we therefore obtain
0 ≤ ( N B + α H B − 1 2 ) log ( 1 + α H B N B ) ≤ ( N B + α H B ) α H B N B ≤ 2 α H B ≤ 2 α . 0\le\left(N_B+\alpha H_B-\frac{1}{2}\right)\log\left(1+\frac{\alpha H_B}{N_B}\right)\le(N_B+\alpha H_B)\frac{\alpha H_B}{N_B}\le2\alpha H_B\le2\alpha. 0 ≤ ( N B + α H B − 2 1 ) log ( 1 + N B α H B ) ≤ ( N B + α H B ) N B α H B ≤ 2 α H B ≤ 2 α .
Next, we seek to combine the two terms in the square brackets. The first part equals a log-likelihood:
∑ i = 0 1 N A i log N A i − N A log N A = ∑ i = 0 1 N A i log P ^ A i ∣ A . \sum_{i=0}^{1} N_{A_i}\log N_{A_i}-N_A\log N_A=\sum_{i=0}^{1}N_{A_i}\log\widehat{P}_{A_i\mid A}. i = 0 ∑ 1 N A i log N A i − N A log N A = i = 0 ∑ 1 N A i log P A i ∣ A .
In the second bracket, E A , n \mathcal{E}_{A,n} E A , n gives n P A / 2 ≤ N A ≤ 2 n P A nP_A/2\le N_A\le2nP_A n P A /2 ≤ N A ≤ 2 n P A for sufficiently large n n n , while (E.9) gives b N A ≤ N A i ≤ N A bN_A\le N_{A_i}\le N_A b N A ≤ N A i ≤ N A . Together with C l H A ≤ P A ≤ C u H A C_lH_A\le P_A\le C_uH_A C l H A ≤ P A ≤ C u H A , these imply
b C l 2 ≤ N B n H A ≤ 2 C u , B ∈ { A 0 , A 1 , A } . \frac{bC_l}{2}\le\frac{N_B}{nH_A}\le2C_u,\qquad B\in\{A_0,A_1,A\}. 2 b C l ≤ n H A N B ≤ 2 C u , B ∈ { A 0 , A 1 , A } .
Thus log N B − log ( n H A ) = log { N B / ( n H A ) } \log N_B-\log(nH_A)=\log\{N_B/(nH_A)\} log N B − log ( n H A ) = log { N B / ( n H A )} is uniformly bounded. Since the coefficients α H B − 1 / 2 \alpha H_B-1/2 α H B − 1/2 are also uniformly bounded, replacing each log N B \log N_B log N B in the second bracket with log ( n H A ) \log(nH_A) log ( n H A ) incurs only a uniformly bounded error. The resulting coefficient of log ( n H A ) \log(nH_A) log ( n H A ) is then
∑ i = 0 1 ( α H A i − 1 2 ) − ( α H A − 1 2 ) = − 1 2 . \sum_{i=0}^{1}\left(\alpha H_{A_i}-\frac{1}{2}\right)-\left(\alpha H_A-\frac{1}{2}\right)=-\frac{1}{2}. i = 0 ∑ 1 ( α H A i − 2 1 ) − ( α H A − 2 1 ) = − 2 1 .
Combining the two beta function approximations with the remaining term in (4) gives
log B A ( d , q ) = ∑ i = 0 1 N A i log P ^ A i ∣ A H A i ∣ A + log H A − 1 2 log ( n H A ) + R A , n ( d , q ) = N A K ( P ^ A 0 ∣ A ) − 1 2 log n H A + R A , n ( d , q ) , \begin{aligned}
\log B_A(d,q)=\sum_{i=0}^{1}N_{A_i}\log\frac{\widehat{P}_{A_i\mid A}}{H_{A_i\mid A}}+\log H_A-\frac{1}{2}\log(nH_A)+R_{A,n}(d,q) \\
&=N_AK(\widehat{P}_{A_0\mid A})-\frac{1}{2}\log\frac{n}{H_A}+R_{A,n}(d,q),
\end{aligned} log B A ( d , q ) = i = 0 ∑ 1 N A i log H A i ∣ A P A i ∣ A + log H A − 2 1 log ( n H A ) + R A , n ( d , q ) = N A K ( P A 0 ∣ A ) − 2 1 log H A n + R A , n ( d , q ) ,
where K ( u ) K(u) K ( u ) for u ∈ ( 0 , 1 ) u\in(0,1) u ∈ ( 0 , 1 ) is the Bernoulli KL divergence defined as
K ( u ) = u log u H A 0 ∣ A + ( 1 − u ) log 1 − u 1 − H A 0 ∣ A . K(u)=u\log\frac{u}{H_{A_0\mid A}}+(1-u)\log\frac{1-u}{1-H_{A_0\mid A}}. K ( u ) = u log H A 0 ∣ A u + ( 1 − u ) log 1 − H A 0 ∣ A 1 − u .
Finally, (E.7) and Assumption 1 give H A ≥ 1 / n H_A\ge1/n H A ≥ 1/ n for sufficiently large n n n , which implies log ( n / H A ) ≤ 2 log n \log(n/H_A)\le2\log n log ( n / H A ) ≤ 2 log n . On E A , n \mathcal{E}_{A,n} E A , n , inserting the factor P ^ A / P A \widehat{P}_A/P_A P A / P A into the penalty therefore changes it by at most
1 2 ∣ P ^ A P A − 1 ∣ log n H A ≤ 1. \frac{1}{2}\left|\frac{\widehat{P}_A}{P_A}-1\right|\log\frac{n}{H_A}\le1. 2 1 P A P A − 1 log H A n ≤ 1.
Absorbing this uniformly bounded error into R A , n ( d , q ) R_{A,n}(d,q) R A , n ( d , q ) gives
log B A ( d , q ) = N A K ( P ^ A 0 ∣ A ) − P ^ A 2 P A log n H A + R A , n ( d , q ) , (41) \log B_A(d,q)=N_AK(\widehat{P}_{A_0\mid A})-\frac{\widehat{P}_A}{2P_A}\log\frac{n}{H_A}+R_{A,n}(d,q), \tag*{(41)} log B A ( d , q ) = N A K ( P A 0 ∣ A ) − 2 P A P A log H A n + R A , n ( d , q ) , (41)
which yields the desired penalty in (9) after multiplication by P A / N A P_A/N_A P A / N A .
Armed with the above, we next expand K ( P ^ A 0 ∣ A ) K(\widehat{P}_{A_0\mid A}) K ( P A 0 ∣ A ) about P A 0 ∣ A P_{A_0\mid A} P A 0 ∣ A . Let Δ n = P ^ A 0 ∣ A − P A 0 ∣ A \Delta_n=\widehat{P}_{A_0\mid A}-P_{A_0\mid A} Δ n = P A 0 ∣ A − P A 0 ∣ A , suppressing its dependence on A A A and ( d , q ) (d,q) ( d , q ) , and note that K K K has the derivatives
K ′ ( u ) = log u ( 1 − H A 0 ∣ A ) ( 1 − u ) H A 0 ∣ A , K ′ ′ ( u ) = 1 u ( 1 − u ) , K ′ ′ ′ ( u ) = 2 u − 1 u 2 ( 1 − u ) 2 . (42) K'(u)=\log\frac{u(1-H_{A_0|A})}{(1-u)H_{A_0|A}}, \qquad K''(u)=\frac{1}{u(1-u)}, \qquad K'''(u)=\frac{2u-1}{u^2(1-u)^2}. \tag*{(42)} K ′ ( u ) = log ( 1 − u ) H A 0 ∣ A u ( 1 − H A 0 ∣ A ) , K ′′ ( u ) = u ( 1 − u ) 1 , K ′′′ ( u ) = u 2 ( 1 − u ) 2 2 u − 1 . (42)
Since P ^ A 0 ∣ A = P A 0 ∣ A + Δ n \widehat{P}_{A_0|A}=P_{A_0|A}+\Delta_n P A 0 ∣ A = P A 0 ∣ A + Δ n and K ( P A 0 ∣ A ) = s A ( d , q ) / P A K(P_{A_0|A})=s_A(d,q)/P_A K ( P A 0 ∣ A ) = s A ( d , q ) / P A , we can write the quadratic expansion:
K ( P ^ A 0 ∣ A ) = s A ( d , q ) P A + K ′ ( P A 0 ∣ A ) Δ n + 1 2 K ′ ′ ( P A 0 ∣ A ) Δ n 2 + R n . K(\widehat{P}_{A_0|A})=\frac{s_A(d,q)}{P_A}+K'(P_{A_0|A})\Delta_n+\frac{1}{2}K''(P_{A_0|A})\Delta_n^2+R_n. K ( P A 0 ∣ A ) = P A s A ( d , q ) + K ′ ( P A 0 ∣ A ) Δ n + 2 1 K ′′ ( P A 0 ∣ A ) Δ n 2 + R n .
By (E.7) and (E.9), the segment between P A 0 ∣ A P_{A_0|A} P A 0 ∣ A and P ^ A 0 ∣ A \widehat{P}_{A_0|A} P A 0 ∣ A is away from the boundaries, so K ′ ′ ′ ( u ) K'''(u) K ′′′ ( u ) is uniformly bounded on this interval. Taylor’s theorem then gives ∣ R n ∣ ≲ ∣ Δ n ∣ 3 |R_n|\lesssim|\Delta_n|^3 ∣ R n ∣ ≲ ∣ Δ n ∣ 3 . Moreover, Definition E.4 gives N A Δ n 2 ≲ log n N_A\Delta_n^2\lesssim\log n N A Δ n 2 ≲ log n , while (E.9) gives ∣ Δ n ∣ ≲ ( log n ) − 2 |\Delta_n|\lesssim(\log n)^{-2} ∣ Δ n ∣ ≲ ( log n ) − 2 . Hence
N A ∣ R n ∣ ≲ N A Δ n 2 ∣ Δ n ∣ ≲ 1 log n , N_A|R_n|\lesssim N_A\Delta_n^2|\Delta_n|\lesssim\frac{1}{\log n}, N A ∣ R n ∣ ≲ N A Δ n 2 ∣ Δ n ∣ ≲ log n 1 ,
and we can absorb N A R n N_AR_n N A R n into R A , n ( d , q ) R_{A,n}(d,q) R A , n ( d , q ) in (E.10).
Multiplying both sides of (E.10) by P A / N A P_A/N_A P A / N A and substituting the Taylor expansion gives the decomposition in (9) , with error
ε A , n ( d , q ) = P A [ K ′ ( P A 0 ∣ A ) Δ n + 1 2 K ′ ′ ( P A 0 ∣ A ) Δ n 2 + R A , n ( d , q ) N A ] . \varepsilon_{A,n}(d,q)=P_A\left[K'(P_{A_0|A})\Delta_n+\frac{1}{2}K''(P_{A_0|A})\Delta_n^2+\frac{R_{A,n}(d,q)}{N_A}\right]. ε A , n ( d , q ) = P A [ K ′ ( P A 0 ∣ A ) Δ n + 2 1 K ′′ ( P A 0 ∣ A ) Δ n 2 + N A R A , n ( d , q ) ] .
We proceed to bound the three distinct terms above. On E A , n \mathcal{E}_{A,n} E A , n for sufficiently large n n n , the earlier bound N A ≥ n P A / 2 N_A\ge nP_A/2 N A ≥ n P A /2 and the uniform boundedness of R A , n ( d , q ) R_{A,n}(d,q) R A , n ( d , q ) give
∣ P A R A , n ( d , q ) N A ∣ ≲ 1 n . (43) \left|\frac{P_AR_{A,n}(d,q)}{N_A}\right|\lesssim\frac{1}{n}. \tag*{(43)} N A P A R A , n ( d , q ) ≲ n 1 . (43)
The quadratic noise is directly controlled by the definition of D A , n ( t , d , q ) \mathcal{D}_{A,n}(t,d,q) D A , n ( t , d , q ) :
P A 2 K ′ ′ ( P A 0 ∣ A ) Δ n 2 < 3 t P A log n N A . (44) \frac{P_A}{2}K''(P_{A_0|A})\Delta_n^2<3\frac{tP_A\log n}{N_A}. \tag*{(44)} 2 P A K ′′ ( P A 0 ∣ A ) Δ n 2 < 3 N A t P A log n . (44)
We now control the linear term. By Assumption 1, any u u u between H A 0 ∣ A H_{A_0|A} H A 0 ∣ A and P A 0 ∣ A P_{A_0|A} P A 0 ∣ A satisfies
u ≥ min { H A 0 ∣ A , P A 0 ∣ A } ≥ C l C u P A 0 ∣ A , u\ge\min\{H_{A_0|A},P_{A_0|A}\}\ge\frac{C_l}{C_u}P_{A_0|A}, u ≥ min { H A 0 ∣ A , P A 0 ∣ A } ≥ C u C l P A 0 ∣ A ,
and similarly 1 − u ≥ ( C l / C u ) ( 1 − P A 0 ∣ A ) 1-u\ge(C_l/C_u)(1-P_{A_0|A}) 1 − u ≥ ( C l / C u ) ( 1 − P A 0 ∣ A ) . Hence (E.11) gives
K ′ ′ ( u ) = 1 u + 1 1 − u ≤ C u C l ( 1 P A 0 ∣ A + 1 1 − P A 0 ∣ A ) = C u C l K ′ ′ ( P A 0 ∣ A ) . K''(u)=\frac{1}{u}+\frac{1}{1-u}\le\frac{C_u}{C_l}\left(\frac{1}{P_{A_0|A}}+\frac{1}{1-P_{A_0|A}}\right)=\frac{C_u}{C_l}K''(P_{A_0|A}). K ′′ ( u ) = u 1 + 1 − u 1 ≤ C l C u ( P A 0 ∣ A 1 + 1 − P A 0 ∣ A 1 ) = C l C u K ′′ ( P A 0 ∣ A ) .
Recall that K ( H A 0 ∣ A ) = K ′ ( H A 0 ∣ A ) = 0 K(H_{A_0|A}) = K'(H_{A_0|A}) = 0 K ( H A 0 ∣ A ) = K ′ ( H A 0 ∣ A ) = 0 . If P A 0 ∣ A = H A 0 ∣ A P_{A_0|A} = H_{A_0|A} P A 0 ∣ A = H A 0 ∣ A , the linear term therefore vanishes. Otherwise, Cauchy’s mean value theorem applied to ( K ′ ) 2 (K')^2 ( K ′ ) 2 and K K K gives some ξ \xi ξ strictly between P A 0 ∣ A P_{A_0|A} P A 0 ∣ A and H A 0 ∣ A H_{A_0|A} H A 0 ∣ A such that
∣ K ′ ( P A 0 ∣ A ) ∣ 2 − 0 K ( P A 0 ∣ A ) − 0 = 2 K ′ ( ξ ) K ′ ′ ( ξ ) K ′ ( ξ ) = 2 K ′ ′ ( ξ ) ≤ 2 C u C l K ′ ′ ( P A 0 ∣ A ) . \frac{\lvert K'(P_{A_0|A})\rvert^2 - 0}{K(P_{A_0|A}) - 0} = \frac{2K'(\xi)K''(\xi)}{K'(\xi)} = 2K''(\xi) \le2\frac{C_u}{C_l}K''(P_{A_0|A}). K ( P A 0 ∣ A ) − 0 ∣ K ′ ( P A 0 ∣ A ) ∣ 2 − 0 = K ′ ( ξ ) 2 K ′ ( ξ ) K ′′ ( ξ ) = 2 K ′′ ( ξ ) ≤ 2 C l C u K ′′ ( P A 0 ∣ A ) .
Here K ′ ( ξ ) ≠ 0 K'(\xi) \ne0 K ′ ( ξ ) = 0 because K ′ ′ > 0 K'' > 0 K ′′ > 0 and K ′ ( H A 0 ∣ A ) = 0 K'(H_{A_0|A}) = 0 K ′ ( H A 0 ∣ A ) = 0 . Using K ( P A 0 ∣ A ) = s A ( d , q ) / P A K(P_{A_0|A}) = s_A(d,q)/P_A K ( P A 0 ∣ A ) = s A ( d , q ) / P A , rearranging the above gives
∣ K ′ ( P A 0 ∣ A ) ∣ ≤ 2 ( C u C l ) 1 / 2 s A ( d , q ) P A K ′ ′ ( P A 0 ∣ A ) . \lvert K'(P_{A_0|A})\rvert\le\sqrt{2}\left(\frac{C_u}{C_l}\right)^{1/2}\sqrt{\frac{s_A(d,q)}{P_A}}\sqrt{K''(P_{A_0|A})}. ∣ K ′ ( P A 0 ∣ A )∣ ≤ 2 ( C l C u ) 1/2 P A s A ( d , q ) K ′′ ( P A 0 ∣ A ) .
Multiplying the above by P A ∣ Δ n ∣ P_A\lvert\Delta_n\rvert P A ∣ Δ n ∣ and applying (E.13) gives
∣ P A K ′ ( P A 0 ∣ A ) Δ n ∣ ≤ 2 3 ( C u C l ) 1 / 2 s A ( d , q ) t P A log n N A . \lvert P_AK'(P_{A_0|A})\Delta_n\rvert\le2\sqrt{3}\left(\frac{C_u}{C_l}\right)^{1/2}\sqrt{s_A(d,q)}\sqrt{\frac{tP_A\log n}{N_A}}. ∣ P A K ′ ( P A 0 ∣ A ) Δ n ∣ ≤ 2 3 ( C l C u ) 1/2 s A ( d , q ) N A t P A log n .
The triangle inequality completes the proof.
Lemma E.5 crucially controls the error in the split Bayes factor decomposition from (9) under the events on the node and split counts. For fixed t t t , we see that if s A ( d , q ) s_A(d,q) s A ( d , q ) is large relative to log n / n \log n/n log n / n , it will dominate this error. Conversely, when s A ( d , q ) = 0 s_A(d,q) = 0 s A ( d , q ) = 0 , the linear term vanishes and the remaining error is O ( t log n / n ) O(t\log n/n) O ( t log n / n ) , allowing the stopping penalty to dominate for sufficiently small t t t .
Near-optimal splitting and stopping
We can now leverage Lemma E.5 to connect the events E A , n \mathcal{E}_{A,n} E A , n and D A , n \mathcal{D}_{A,n} D A , n to splitting and stopping of the GEB algorithm. Throughout this subsection, for any node A A A , let ( d ^ , q ^ ) (\hat d,\hat q) ( d ^ , q ^ ) maximize B A ( d , q ) \mathcal{B}_A(d,q) B A ( d , q ) over all D ( m − 1 ) D(m-1) D ( m − 1 ) candidate splits. The maximizer need not be unique, so we fix a deterministic tie-breaking rule (the choice is unimportant), such as choosing the smallest d d d and then the smallest q q q among maximizers. For fixed A A A , the selected pair ( d ^ , q ^ ) (\hat d,\hat q) ( d ^ , q ^ ) is then a random discrete vector. We also highlight that B A ( d , q ) \mathcal{B}_A(d,q) B A ( d , q ) is random and depends on n n n , but we omit the n n n dependence to match the main paper.
We highlight that this definition of ( d ^ , q ^ ) (\hat d,\hat q) ( d ^ , q ^ ) is slightly different to Algorithm 2, where the algorithm only splits eligible nodes A A A at eligible splits. Instead, we handle eligibility separately using the event defined below,
Q A , n = ⋂ d , q { N A 0 ≥ 1 , N A 1 ≥ 1 } , Q_{A,n} = \bigcap_{d,q}\{N_{A_0} \ge1,\ N_{A_1} \ge1\}, Q A , n = d , q ⋂ { N A 0 ≥ 1 , N A 1 ≥ 1 } ,
where the intersection is over all D ( m − 1 ) D(m-1) D ( m − 1 ) candidate splits of A A A . It is clear that if Q A , n \mathcal{Q}_{A,n} Q A , n holds and B A ( d ^ , q ^ ) > 1 \mathcal{B}_{A}(\hat{d},\hat{q})>1 B A ( d ^ , q ^ ) > 1 , then ( d ^ , q ^ ) (\hat{d},\hat{q}) ( d ^ , q ^ ) defined above is equivalent to the split chosen by the GEB algorithm at node A A A , as all splits are eligible.
Lemma E.6 (Sufficient condition for near-optimal splitting). For each identifiable oracle node A ∈ T n ∗ A\in\mathcal{T}_{n}^{*} A ∈ T n ∗ , define the event G A , n \mathcal{G}_{A,n} G A , n as
G A , n : = { s A ( d ^ , q ^ ) ≥ ( 1 − δ ) max d , q s A ( d , q ) } ∩ { log B A ( d ^ , q ^ ) > 0 } ∩ Q A , n , \mathcal{G}_{A,n}:=\left\{s_{A}(\hat{d},\hat{q})\geq(1-\delta)\max_{d,q}s_{A}(d,q)\right\}\cap\left\{\log\mathcal{B}_{A}(\hat{d},\hat{q})>0\right\}\cap\mathcal{Q}_{A,n}, G A , n := { s A ( d ^ , q ^ ) ≥ ( 1 − δ ) d , q max s A ( d , q ) } ∩ { log B A ( d ^ , q ^ ) > 0 } ∩ Q A , n , where the maximum is over all candidate m m m -grid splits at A A A . Then for sufficiently large n n n , the following holds uniformly over the identifiable oracle nodes $A\in\mathcal{T}_{n}^{ }:*
E A , n ∩ D ~ A , n ( t 0 ) ⊆ G A , n . \mathcal{E}_{A,n}\cap\widetilde{\mathcal{D}}_{A,n}(t_{0})\subseteq\mathcal{G}_{A,n}. E A , n ∩ D A , n ( t 0 ) ⊆ G A , n .
Proof. All thresholds on n n n in this proof can be chosen independently of A A A and ( d , q ) (d,q) ( d , q ) , since the bounds and constants used below are uniform. Fix A ∈ T n ∗ A\in\mathcal{T}_{n}^{*} A ∈ T n ∗ and work on E A , n ∩ D ~ A , n ( t 0 ) \mathcal{E}_{A,n}\cap\widetilde{\mathcal{D}}_{A,n}(t_{0}) E A , n ∩ D A , n ( t 0 ) , which in particular ensures N A > 0 N_{A}>0 N A > 0 .
We begin with the first near-optimal condition. Let ( d ∗ , q ∗ ) (d^{*},q^{*}) ( d ∗ , q ∗ ) maximize the population split gain s A ( d , q ) s_{A}(d,q) s A ( d , q ) , with the same deterministic tie-breaking rule. The penalty in (9) is common to all candidate splits, so we have the following split gain gap for any split ( d , q ) (d,q) ( d , q ) :
P A N A { log B A ( d ∗ , q ∗ ) − log B A ( d , q ) } = s A ( d ∗ , q ∗ ) − s A ( d , q ) + ε A , n ( d ∗ , q ∗ ) − ε A , n ( d , q ) . \frac{P_{A}}{N_{A}}\left\{\log\mathcal{B}_{A}(d^{*},q^{*})-\log\mathcal{B}_{A}(d,q)\right\}=s_{A}(d^{*},q^{*})-s_{A}(d,q)+\varepsilon_{A,n}(d^{*},q^{*})-\varepsilon_{A,n}(d,q). N A P A { log B A ( d ∗ , q ∗ ) − log B A ( d , q ) } = s A ( d ∗ , q ∗ ) − s A ( d , q ) + ε A , n ( d ∗ , q ∗ ) − ε A , n ( d , q ) .
We now show that for sufficiently large n n n , and uniformly over A ∈ T n ∗ A\in\mathcal{T}_{n}^{*} A ∈ T n ∗ , any candidate split that is not near-optimal will not be a maximizer. Specifically, suppose ( d ′ , q ′ ) (d',q') ( d ′ , q ′ ) is not near-optimal, so s A ( d ′ , q ′ ) < ( 1 − δ ) s A ( d ∗ , q ∗ ) s_{A}(d',q')<(1-\delta)s_{A}(d^{*},q^{*}) s A ( d ′ , q ′ ) < ( 1 − δ ) s A ( d ∗ , q ∗ ) . Then we have
P A N A { log B A ( d ∗ , q ∗ ) − log B A ( d ′ , q ′ ) } > δ s A ( d ∗ , q ∗ ) − 2 sup d , q ∣ ε A , n ( d , q ) ∣ . \frac{P_{A}}{N_{A}}\left\{\log\mathcal{B}_{A}(d^{*},q^{*})-\log\mathcal{B}_{A}(d',q')\right\}>\delta s_{A}(d^{*},q^{*})-2\sup_{d,q}\left|\varepsilon_{A,n}(d,q)\right|. N A P A { log B A ( d ∗ , q ∗ ) − log B A ( d ′ , q ′ ) } > δ s A ( d ∗ , q ∗ ) − 2 d , q sup ∣ ε A , n ( d , q ) ∣ .
From the above, it is clear that if
sup d , q ∣ ε A , n ( d , q ) ∣ ≤ δ 2 s A ( d ∗ , q ∗ ) , (45) \sup_{d,q}\left|\varepsilon_{A,n}(d,q)\right|\leq\frac{\delta}{2}s_{A}(d^{*},q^{*}), \tag*{(45)} d , q sup ∣ ε A , n ( d , q ) ∣ ≤ 2 δ s A ( d ∗ , q ∗ ) , (45)
then log B A ( d ∗ , q ∗ ) − log B A ( d ′ , q ′ ) > 0 \log\mathcal{B}_{A}(d^{*},q^{*})-\log\mathcal{B}_{A}(d',q')>0 log B A ( d ∗ , q ∗ ) − log B A ( d ′ , q ′ ) > 0 , so ( d ′ , q ′ ) (d',q') ( d ′ , q ′ ) cannot be chosen. We now proceed to show that the above holds true for sufficiently large n n n under E A , n ∩ D ~ A , n ( t 0 ) \mathcal{E}_{A,n}\cap\widetilde{\mathcal{D}}_{A,n}(t_{0}) E A , n ∩ D A , n ( t 0 ) .
We will rely on Assumption 2 and identifiability which give
s A ( d ∗ , q ∗ ) ≥ κ S A ≥ κ C 0 log n n . (46) s_{A}(d^{*},q^{*})\geq\kappa S_{A}\geq\frac{\kappa C_{0}\log n}{n}. \tag*{(46)} s A ( d ∗ , q ∗ ) ≥ κ S A ≥ n κ C 0 log n . (46)
On E A , n ∩ D ~ A , n ( t 0 ) \mathcal{E}_{A,n} \cap\widetilde{\mathcal{D}}_{A,n}(t_0) E A , n ∩ D A , n ( t 0 ) , Lemma E.5 with t = t 0 t=t_0 t = t 0 upper bounds ∣ ε A , n ( d , q ) ∣ |\varepsilon_{A,n}(d,q)| ∣ ε A , n ( d , q ) ∣ , and we now allocate δ s A ( d ∗ , q ∗ ) / 6 \delta s_A(d^*,q^*)/6 δ s A ( d ∗ , q ∗ ) /6 to each of the bound’s three terms. For the third term, choosing n ≥ exp { 6 C / ( δ κ C 0 ) } n \ge\exp\{6C/(\delta\kappa C_0)\} n ≥ exp { 6 C / ( δ κ C 0 )} together with (E.15) gives the following bound uniformly over identifiable A A A :
C n ≤ δ 6 s A ( d ∗ , q ∗ ) . \frac{C}{n} \le\frac{\delta}{6}s_A(d^*,q^*). n C ≤ 6 δ s A ( d ∗ , q ∗ ) .
For the second error term, under E A , n \mathcal{E}_{A,n} E A , n , we can choose n n n sufficiently large so that P A / N A ≤ 2 / n P_A/N_A \le2/n P A / N A ≤ 2/ n . The second term in the error then satisfies
3 t 0 P A log n N A ≤ 6 t 0 log n n ≤ δ 6 s A ( d ∗ , q ∗ ) , 3t_0\frac{P_A\log n}{N_A} \le6t_0\frac{\log n}{n} \le\frac{\delta}{6}s_A(d^*,q^*), 3 t 0 N A P A log n ≤ 6 t 0 n log n ≤ 6 δ s A ( d ∗ , q ∗ ) ,
where the last inequality follows from (E.15) and κ C 0 ≥ 36 t 0 / δ \kappa C_0 \ge36t_0/\delta κ C 0 ≥ 36 t 0 / δ from the choice of C 0 C_0 C 0 in (E.2). For the first term, we use s A ( d , q ) ≤ s A ( d ∗ , q ∗ ) s_A(d,q) \le s_A(d^*,q^*) s A ( d , q ) ≤ s A ( d ∗ , q ∗ ) and log n / n ≤ s A ( d ∗ , q ∗ ) / ( κ C 0 ) \log n/n \le s_A(d^*,q^*)/(\kappa C_0) log n / n ≤ s A ( d ∗ , q ∗ ) / ( κ C 0 ) from (E.15) which gives
2 3 ( C u C l ) 1 / 2 s A ( d , q ) t 0 P A log n N A ≤ 2 6 ( C u C l ) 1 / 2 t 0 κ C 0 s A ( d ∗ , q ∗ ) = δ 6 s A ( d ∗ , q ∗ ) , 2\sqrt{3}\left(\frac{C_u}{C_l}\right)^{1/2}\sqrt{s_A(d,q)}\sqrt{\frac{t_0P_A\log n}{N_A}} \le2\sqrt{6}\left(\frac{C_u}{C_l}\right)^{1/2}\sqrt{\frac{t_0}{\kappa C_0}}s_A(d^*,q^*) = \frac{\delta}{6}s_A(d^*,q^*), 2 3 ( C l C u ) 1/2 s A ( d , q ) N A t 0 P A log n ≤ 2 6 ( C l C u ) 1/2 κ C 0 t 0 s A ( d ∗ , q ∗ ) = 6 δ s A ( d ∗ , q ∗ ) ,
again using P A / N A ≤ 2 / n P_A/N_A \le2/n P A / N A ≤ 2/ n and the choice of C 0 C_0 C 0 in (E.2). The above equality motivates our exact choice of C 0 C_0 C 0 . The three bounds prove (E.14), so every maximizer of log B A ( d , q ) \log\mathcal{B}_A(d,q) log B A ( d , q ) must be near-optimal.
We now handle the event { log B A ( d ^ , q ^ ) > 0 } \{\log\mathcal{B}_A(\hat d,\hat q)>0\} { log B A ( d ^ , q ^ ) > 0 } , where n n n is sufficiently large so all the above bounds hold. Assumption 1 and (E.7) give H A ≳ ( log n ) 5 / n H_A \gtrsim(\log n)^5/n H A ≳ ( log n ) 5 / n , so H A > 1 / n H_A>1/n H A > 1/ n for sufficiently large n n n . Using (E.15) and κ C 0 > 2 / δ \kappa C_0>2/\delta κ C 0 > 2/ δ from (E.2), we obtain
1 2 n log n H A < log n n ≤ s A ( d ∗ , q ∗ ) κ C 0 ≤ δ 2 s A ( d ∗ , q ∗ ) . \frac{1}{2n}\log\frac{n}{H_A} < \frac{\log n}{n} \le\frac{s_A(d^*,q^*)}{\kappa C_0} \le\frac{\delta}{2}s_A(d^*,q^*). 2 n 1 log H A n < n log n ≤ κ C 0 s A ( d ∗ , q ∗ ) ≤ 2 δ s A ( d ∗ , q ∗ ) .
Therefore, the split Bayes factor expansion in (9) and (E.14) give
P A N A log B A ( d ^ , q ^ ) ≥ P A N A log B A ( d ∗ , q ∗ ) ≥ s A ( d ∗ , q ∗ ) − 1 2 n log n H A − δ 2 s A ( d ∗ , q ∗ ) > ( 1 − δ ) s A ( d ∗ , q ∗ ) > 0. \begin{aligned}
\frac{P_A}{N_A}\log\mathcal{B}_A(\hat d,\hat q) \ge\frac{P_A}{N_A}\log\mathcal{B}_A(d^*,q^*) \\
&\ge s_A(d^*,q^*)-\frac{1}{2n}\log\frac{n}{H_A}-\frac{\delta}{2}s_A(d^*,q^*) \\
&> (1-\delta)s_A(d^*,q^*)>0.
\end{aligned} N A P A log B A ( d ^ , q ^ ) ≥ N A P A log B A ( d ∗ , q ∗ ) ≥ s A ( d ∗ , q ∗ ) − 2 n 1 log H A n − 2 δ s A ( d ∗ , q ∗ ) > ( 1 − δ ) s A ( d ∗ , q ∗ ) > 0.
Finally, Lemma E.4 gives P ^ A 0 ∣ A ∈ [ b , 1 − b ] \widehat{P}_{A_0|A}\in[b,1-b] P A 0 ∣ A ∈ [ b , 1 − b ] for every candidate split, ensuring that both children contain observations, so Q A , n \mathcal{Q}_{A,n} Q A , n holds. □ \square □
The next lemma gives a sufficient condition for ruling out a candidate split under the GEB algorithm, either because it is ineligible or because its Bayes factor is at most one.
Lemma E.7 (Sufficient condition for stopping). For any node A A A of an admissible tree T ∈ T n T \in\mathcal{T}_n T ∈ T n and candidate split ( d , q ) (d,q) ( d , q ) , define the event
W A ( d , q ) : = { log B A ( d , q ) ≤ 0 } ∪ { N A 0 ≥ 1 , N A 1 ≥ 1 } c . \mathcal{W}_A(d,q) := \{\log\mathcal{B}_A(d,q) \le0\} \cup\{N_{A_0} \ge1, N_{A_1} \ge1\}^{c}. W A ( d , q ) := { log B A ( d , q ) ≤ 0 } ∪ { N A 0 ≥ 1 , N A 1 ≥ 1 } c . Then, for sufficiently large n n n , the following holds uniformly over such nodes and stopping splits ( d , q ) (d,q) ( d , q ) satisfying s A ( d , q ) ≤ C s log n / n s_A(d,q) \le C_s \log n/n s A ( d , q ) ≤ C s log n / n :
E A , n ∩ D A , n ( 1 / 15 , d , q ) ⊆ W A ( d , q ) . \mathcal{E}_{A,n} \cap\mathcal{D}_{A,n}(1/15,d,q) \subseteq\mathcal{W}_A(d,q). E A , n ∩ D A , n ( 1/15 , d , q ) ⊆ W A ( d , q ) . Proof. As before, all thresholds on n n n in this proof can be chosen independently of A A A and ( d , q ) (d,q) ( d , q ) , and we work on E A , n ∩ D A , n ( 1 / 15 , d , q ) \mathcal{E}_{A,n} \cap\mathcal{D}_{A,n}(1/15,d,q) E A , n ∩ D A , n ( 1/15 , d , q ) , which in particular ensures N A > 0 N_A > 0 N A > 0 . Since H A ≤ 1 H_A \le1 H A ≤ 1 , the penalty in (9) is at least log n / ( 2 n ) \log n/(2n) log n / ( 2 n ) in absolute value, so
P A N A log B A ( d , q ) ≤ − log n 2 n + s A ( d , q ) + ∣ ε A , n ( d , q ) ∣ . \frac{P_A}{N_A}\log\mathcal{B}_A(d,q) \le-\frac{\log n}{2n} + s_A(d,q) + \lvert\varepsilon_{A,n}(d,q)\rvert. N A P A log B A ( d , q ) ≤ − 2 n log n + s A ( d , q ) + ∣ ε A , n ( d , q )∣ . We thus seek to show that the above term is non-positive under our assumptions. We now carry out a similar exercise as in Lemma E.6 and study the split gain and the three error terms in Lemma E.5. For the third term, we can choose n n n sufficiently large so that
C n ≤ log n 40 n . \frac{C}{n} \le\frac{\log n}{40n}. n C ≤ 40 n log n . For the second term, on E A , n \mathcal{E}_{A,n} E A , n , we can again choose n n n sufficiently large so that P A / N A ≤ 5 / ( 4 n ) P_A/N_A \le5/(4n) P A / N A ≤ 5/ ( 4 n ) . This then gives the second term as
3 15 P A log n N A ≤ 1 4 log n n . \frac{3}{15}\frac{P_A\log n}{N_A} \le\frac{1}{4}\frac{\log n}{n}. 15 3 N A P A log n ≤ 4 1 n log n . We then handle the first term, where the stopping condition in Definition E.2 gives s A ( d , q ) ≤ C s log n / n s_A(d,q) \le C_s\log n/n s A ( d , q ) ≤ C s log n / n . Applying Lemma E.5 with t = 1 / 15 t=1/15 t = 1/15 , the above bounds on s A ( d , q ) s_A(d,q) s A ( d , q ) and P A / N A P_A/N_A P A / N A control its first term with
2 3 ( C u C l ) 1 / 2 s A ( d , q ) P A log n 15 N A ≤ ( C u C l ) 1 / 2 C s log n n = 1 40 log n n ≤ 1 5 log n n . 2\sqrt{3}\left(\frac{C_u}{C_l}\right)^{1/2}\sqrt{s_A(d,q)}\sqrt{\frac{P_A\log n}{15N_A}} \le\left(\frac{C_u}{C_l}\right)^{1/2}\sqrt{C_s}\frac{\log n}{n} = \frac{1}{\sqrt{40}}\frac{\log n}{n} \le\frac{1}{5}\frac{\log n}{n}. 2 3 ( C l C u ) 1/2 s A ( d , q ) 15 N A P A log n ≤ ( C l C u ) 1/2 C s n log n = 40 1 n log n ≤ 5 1 n log n . where the final equality uses the definition of C s C_s C s . Consequently, the above error bound and the fact that s A ( d , q ) ≤ log n / ( 40 n ) s_A(d,q) \le\log n/(40n) s A ( d , q ) ≤ log n / ( 40 n ) since C s ≤ 1 / 40 C_s \le1/40 C s ≤ 1/40 give
P A N A log B A ( d , q ) ≤ − [ 1 2 − 1 40 − ( 1 40 + 1 5 + 1 4 ) ] log n n ≤ 0 \frac{P_A}{N_A}\log\mathcal{B}_A(d,q) \le-\left[\frac{1}{2}-\frac{1}{40}-\left(\frac{1}{40}+\frac{1}{5}+\frac{1}{4}\right)\right]\frac{\log n}{n} \le0 N A P A log B A ( d , q ) ≤ − [ 2 1 − 40 1 − ( 40 1 + 5 1 + 4 1 ) ] n log n ≤ 0 for sufficiently large n n n . Since P A / N A > 0 P_A/N_A > 0 P A / N A > 0 , this implies log B A ( d , q ) ≤ 0 \log\mathcal{B}_A(d,q) \le0 log B A ( d , q ) ≤ 0 , so its enlargement W A ( d , q ) \mathcal{W}_A(d,q) W A ( d , q ) also holds.
Nodewise probabilistic properties
We have derived interpretable sufficient conditions for both splitting and stopping in the previous section, which ultimately depend on the two count events from Definitions E.3 and E.4. We now proceed to obtain probabilistic bounds on near-optimal splitting and stopping.
Probabilistic bounds for count events
We begin with a technical lemma on a probabilistic bound for the count events in Definitions E.3 and E.4.
Lemma E.8. Fix a constant t > 0 t > 0 t > 0 . For sufficiently large n n n , the following probabilistic bounds hold uniformly over nodes A A A of admissible trees and candidate splits ( d , q ) (d,q) ( d , q ) . First,
P ( E A , n c ) ≤ n − log n . \mathbb{P}\left(\mathcal{E}_{A,n}^{c}\right) \le n^{-\log n}. P ( E A , n c ) ≤ n − l o g n . Second, for every integer k ∈ { 1 , … , n } k \in\{1,\ldots,n\} k ∈ { 1 , … , n } with P ( N A = k ) > 0 \mathbb{P}(N_A = k) > 0 P ( N A = k ) > 0 satisfying the condition of E A , n \mathcal{E}_{A,n} E A , n , namely
∣ k n P A − 1 ∣ ≤ 1 log n , \left|\frac{k}{nP_A}-1\right| \le\frac{1}{\log n}, n P A k − 1 ≤ log n 1 , we have
P ( D A , n ( t , d , q ) c ∣ N A = k ) ≤ 2 n t , P ( D ~ A , n ( t ) c ∣ N A = k ) ≤ 2 D m n t . \mathbb{P}\left(\mathcal{D}_{A,n}(t,d,q)^{c}\mid N_A=k\right) \le\frac{2}{n^t}, \qquad\mathbb{P}\left(\widetilde{\mathcal{D}}_{A,n}(t)^{c}\mid N_A=k\right) \le\frac{2Dm}{n^t}. P ( D A , n ( t , d , q ) c ∣ N A = k ) ≤ n t 2 , P ( D A , n ( t ) c ∣ N A = k ) ≤ n t 2 D m .
Proof. All thresholds on n n n below can be chosen independently of A A A and ( d , q ) (d,q) ( d , q ) . For the first result, we apply the binomial Chernoff bound to N A ∼ Binomial ( n , P A ) N_A \sim\operatorname{Binomial}(n,P_A) N A ∼ Binomial ( n , P A ) . For 0 < λ < 1 0 < \lambda< 1 0 < λ < 1 ,
P ( ∣ P ^ A P A − 1 ∣ ≥ λ ) = P ( ∣ N A − n P A ∣ ≥ λ n P A ) ≤ 2 exp ( − λ 2 n P A 3 ) . \mathbb{P}\left(\left|\frac{\widehat{P}_A}{P_A}-1\right|\ge\lambda\right)=\mathbb{P}\left(\left|N_A-nP_A\right|\ge\lambda nP_A\right)\le2\exp\left(-\frac{\lambda^2 nP_A}{3}\right). P ( P A P A − 1 ≥ λ ) = P ( ∣ N A − n P A ∣ ≥ λn P A ) ≤ 2 exp ( − 3 λ 2 n P A ) .
Take λ = 1 / log n \lambda=1/\log n λ = 1/ log n , which lies in ( 0 , 1 ) (0,1) ( 0 , 1 ) for n ≥ 3 n\ge3 n ≥ 3 . By (E.7), n P A / ( log n ) 4 → ∞ nP_A/(\log n)^4\to\infty n P A / ( log n ) 4 → ∞ , so for sufficiently large n n n , we have n P A ≥ 6 ( log n ) 4 nP_A\ge6(\log n)^4 n P A ≥ 6 ( log n ) 4 for every such A A A . Substituting this lower bound into the Chernoff bound gives
P ( E A , n c ) ≤ 2 exp { − n P A 3 ( log n ) 2 } ≤ 2 e − 2 ( log n ) 2 ≤ e − ( log n ) 2 = n − log n . \mathbb{P}\left(\mathcal{E}_{A,n}^{c}\right)\le2\exp\left\{-\frac{nP_A}{3(\log n)^2}\right\}\le2e^{-2(\log n)^2}\le e^{-(\log n)^2}=n^{-\log n}. P ( E A , n c ) ≤ 2 exp { − 3 ( log n ) 2 n P A } ≤ 2 e − 2 ( l o g n ) 2 ≤ e − ( l o g n ) 2 = n − l o g n .
We now turn to the split count events. The event D A , n ( t , d , q ) \mathcal{D}_{A,n}(t,d,q) D A , n ( t , d , q ) is unchanged when A 0 A_0 A 0 and A 1 A_1 A 1 are interchanged, so we may assume without loss of generality that P A 0 ∣ A ≤ 1 / 2 P_{A_0\mid A}\le1/2 P A 0 ∣ A ≤ 1/2 . Since ( N A 0 , N A 1 , n − N A ) ∼ Multinomial ( n ; P A 0 , P A 1 , 1 − P A ) (N_{A_0}, N_{A_1}, n-N_A) \sim\operatorname{Multinomial}(n; P_{A_0}, P_{A_1}, 1-P_A) ( N A 0 , N A 1 , n − N A ) ∼ Multinomial ( n ; P A 0 , P A 1 , 1 − P A ) , conditioning gives N A 0 ∣ N A = k ∼ Binomial ( k , P A 0 ∣ A ) N_{A_0} \mid N_A=k \sim\operatorname{Binomial}(k, P_{A_0\mid A}) N A 0 ∣ N A = k ∼ Binomial ( k , P A 0 ∣ A ) . For the next conditional Chernoff bound, we set
η = 6 t log n k 1 − P A 0 ∣ A P A 0 ∣ A . \eta= \sqrt{\frac{6t\log n}{k}\frac{1-P_{A_0\mid A}}{P_{A_0\mid A}}}. η = k 6 t log n P A 0 ∣ A 1 − P A 0 ∣ A .
With this choice, on { N A = k } \{N_A=k\} { N A = k } , the event D A , n ( t , d , q ) c \mathcal{D}_{A,n}(t,d,q)^{c} D A , n ( t , d , q ) c is equivalent to
∣ N A 0 − k P A 0 ∣ A ∣ ≥ η k P A 0 ∣ A . \lvert N_{A_0}-kP_{A_0\mid A}\rvert\ge\eta kP_{A_0\mid A}. ∣ N A 0 − k P A 0 ∣ A ∣ ≥ η k P A 0 ∣ A .
The binomial Chernoff bound therefore applies once we verify 0 < η < 1 0<\eta<1 0 < η < 1 . By the condition on k k k and (E.7),
k ≥ ( 1 − 1 log n ) n P A ≳ ( log n ) 5 . k \ge\left(1-\frac{1}{\log n}\right)nP_A \gtrsim(\log n)^5. k ≥ ( 1 − log n 1 ) n P A ≳ ( log n ) 5 .
As (E.7) also directly gives P A 0 ∣ A ∈ [ b , 1 − b ] P_{A_0\mid A}\in[b,1-b] P A 0 ∣ A ∈ [ b , 1 − b ] , we have for sufficiently large n n n ,
0 < η 2 = 6 t log n k 1 − P A 0 ∣ A P A 0 ∣ A ≲ ( log n ) − 4 < 1 , 0<\eta^2=\frac{6t\log n}{k}\frac{1-P_{A_0\mid A}}{P_{A_0\mid A}}\lesssim(\log n)^{-4}<1, 0 < η 2 = k 6 t log n P A 0 ∣ A 1 − P A 0 ∣ A ≲ ( log n ) − 4 < 1 ,
uniformly over admissible A A A and ( d , q ) (d,q) ( d , q ) . Thus the Chernoff bound applies and gives
P ( D A , n ( t , d , q ) c ∣ N A = k ) ≤ 2 exp ( − η 2 k P A 0 ∣ A 3 ) = 2 exp [ − 2 t ( 1 − P A 0 ∣ A ) log n ] ≤ 2 n − t , \mathbb{P}\left(\mathcal{D}_{A,n}(t,d,q)^c\mid N_A=k\right)
\le2\exp\left(-\frac{\eta^2 kP_{A_0\mid A}}{3}\right)
=2\exp\left[-2t(1-P_{A_0\mid A})\log n\right]
\le2n^{-t}, P ( D A , n ( t , d , q ) c ∣ N A = k ) ≤ 2 exp ( − 3 η 2 k P A 0 ∣ A ) = 2 exp [ − 2 t ( 1 − P A 0 ∣ A ) log n ] ≤ 2 n − t ,
where the last inequality uses 1 − P A 0 ∣ A ≥ 1 / 2 1-P_{A_0\mid A}\ge1/2 1 − P A 0 ∣ A ≥ 1/2 .
For the simultaneous event D ~ A , n ( t ) \widetilde{\mathcal{D}}_{A,n}(t) D A , n ( t ) , its complement is the union of the individual complements D A , n ( t , d , q ) c \mathcal{D}_{A,n}(t,d,q)^c D A , n ( t , d , q ) c over all candidate splits. There are D ( m − 1 ) D(m-1) D ( m − 1 ) such splits, so the union bound simply gives
P ( D ~ A , n ( t ) c ∣ N A = k ) ≤ ∑ d , q P ( D A , n ( t , d , q ) c ∣ N A = k ) ≤ 2 D ( m − 1 ) n − t ≤ 2 D m n − t . \mathbb{P}\left(\widetilde{\mathcal{D}}_{A,n}(t)^c\mid N_A=k\right)
\le\sum_{d,q}\mathbb{P}\left(\mathcal{D}_{A,n}(t,d,q)^c\mid N_A=k\right)
\le2D(m-1)n^{-t}\le2Dm n^{-t}. P ( D A , n ( t ) c ∣ N A = k ) ≤ d , q ∑ P ( D A , n ( t , d , q ) c ∣ N A = k ) ≤ 2 D ( m − 1 ) n − t ≤ 2 D m n − t .
Splitting and stopping probabilities
Lemmas E.6 and E.7 express the splitting/stopping decisions in terms of the count events. We now use this connection and the just shown probabilistic bounds for the two requirements of admissibility in Definition 2, namely near-optimal splitting at identifiable oracle nodes and bounded splitting below non-identifiable oracle leaves. We first control the probability of a near-optimal split at an identifiable oracle node.
Lemma E.9 (Near-optimal splitting probability). Let G A , n \mathcal{G}_{A,n} G A , n denote the near-optimal splitting event in Lemma E.6. For sufficiently large n n n , uniformly over identifiable oracle nodes A ∈ T n ∗ A \in\mathcal{T}_n^* A ∈ T n ∗ , we have
P ( G A , n c ) ≤ 2 D m n t 0 + 1 n log n . \mathbb{P}\left(\mathcal{G}_{A,n}^{c}\right) \leq\frac{2Dm}{n^{t_0}}+\frac{1}{n^{\log n}}. P ( G A , n c ) ≤ n t 0 2 D m + n l o g n 1 . where t 0 t_0 t 0 is defined in (E.1).
Proof. By Lemma E.6, E A , n ∩ D ~ A , n ( t 0 ) ⊆ G A , n \mathcal{E}_{A,n} \cap\widetilde{\mathcal{D}}_{A,n}(t_0) \subseteq\mathcal{G}_{A,n} E A , n ∩ D A , n ( t 0 ) ⊆ G A , n . Thus, a failure of near-optimal splitting requires either the node count event to fail, or the node count event to hold while the simultaneous split count event fails:
G A , n c ⊆ E A , n c ∪ { E A , n ∩ D ~ A , n ( t 0 ) c } . \mathcal{G}_{A,n}^{c} \subseteq\mathcal{E}_{A,n}^{c} \cup\left\{\mathcal{E}_{A,n} \cap\widetilde{\mathcal{D}}_{A,n}(t_0)^{c}\right\}. G A , n c ⊆ E A , n c ∪ { E A , n ∩ D A , n ( t 0 ) c } .
Lemma E.8 bounds the first probability by n − log n n^{-\log n} n − l o g n . On E A , n \mathcal{E}_{A,n} E A , n , the observed count N A N_A N A satisfies the condition for the conditional bound in Lemma E.8. Since E A , n \mathcal{E}_{A,n} E A , n is determined by N A N_A N A , conditioning on N A N_A N A and applying the tower property gives
P ( E A , n ∩ D ~ A , n ( t 0 ) c ) = E [ 1 E A , n P ( D ~ A , n ( t 0 ) c ∣ N A ) ] ≤ 2 D m n t 0 . \mathbb{P}\left(\mathcal{E}_{A,n} \cap\widetilde{\mathcal{D}}_{A,n}(t_0)^{c}\right)=\mathbb{E}\left[\mathbb{1}_{\mathcal{E}_{A,n}}\mathbb{P}\left(\widetilde{\mathcal{D}}_{A,n}(t_0)^{c}\mid N_A\right)\right]\leq\frac{2Dm}{n^{t_0}}. P ( E A , n ∩ D A , n ( t 0 ) c ) = E [ 1 E A , n P ( D A , n ( t 0 ) c ∣ N A ) ] ≤ n t 0 2 D m .
Combining the above with the union bound proves the claim. □
We next control splitting below a non-identifiable oracle leaf. Admissibility allows fewer than k max k_{\max} k m a x refinement splits, so the required guarantee concerns the total number of splits rather than the decision at a single node. By Lemma E.1, any sequence reaching k max k_{\max} k m a x splits must contain several stopping splits. Lemma E.7 then relates continued splitting to failures of the count events. The following result bounds the probability of these repeated failures.
Lemma E.10 (Excess splitting probability). For each non-identifiable oracle leaf A ∈ L n ∗ A \in\mathcal{L}_n^* A ∈ L n ∗ , consider applying the GEB splitting and stopping rules starting at A A A , and let H A , n \mathcal{H}_{A,n} H A , n denote the event that this recursion makes at least k max k_{\max} k m a x splits.
For sufficiently large n n n , uniformly over non-identifiable oracle leaves A ∈ L n ∗ A \in\mathcal{L}_n^* A ∈ L n ∗ , we have
P ( H A , n ) ≤ k max ! { D ( m − 1 ) } k max { ( 2 n − 1 / 15 ) ⌈ 15 t 0 ⌉ + ⌈ 15 t 0 ⌉ n − log n } , \mathbb{P}(\mathcal{H}_{A,n}) \leq k_{\max}!\{D(m-1)\}^{k_{\max}}\left\{(2n^{-1/15})^{\lceil15t_0\rceil}+\lceil15t_0\rceil n^{-\log n}\right\}, P ( H A , n ) ≤ k m a x ! { D ( m − 1 ) } k m a x { ( 2 n − 1/15 ) ⌈ 15 t 0 ⌉ + ⌈ 15 t 0 ⌉ n − l o g n } , where t 0 t_0 t 0 is defined in (E.1) and k max k_{\max} k m a x is given in (E.3).
Proof. Throughout this proof, we consider a fixed node A ∈ L n ∗ A \in\mathcal{L}_n^* A ∈ L n ∗ and take A A A as the root of the GEB recursion. We will now proceed to upper bound P ( H A , n ) \mathbb{P}(\mathcal{H}_{A,n}) P ( H A , n ) using a conditional independence argument.
Fix a deterministic sequence of k max k_{\max} k m a x splits rooted at A A A , where at least k s : = ⌈ 15 t 0 ⌉ k_s := \lceil15t_0\rceil k s := ⌈ 15 t 0 ⌉ of these are stopping splits by Lemma E.1. For i = 1 , … , k s i = 1,\ldots,k_s i = 1 , … , k s , let ( d i , q i ) (d_i,q_i) ( d i , q i ) denote the i i i -th stopping split in the sequence and B i B_i B i the node at which it occurs. Since A A A is a leaf of a truncated admissible tree, applying the first k max − 1 k_{\max}-1 k m a x − 1 splits at A A A produces an admissible tree by Definition 2. Every node at which one of the k max k_{\max} k m a x splits occurs already belongs to this tree, including the one for the last split. Thus all B i B_i B i belong to a common admissible tree.
Next, we write the count events for the nodes B i B_i B i with stopping splits as
E i = E B i , n , D i = D B i , n ( 1 / 15 , d i , q i ) . \mathcal{E}_i = \mathcal{E}_{B_i,n}, \qquad\mathcal{D}_i = \mathcal{D}_{B_i,n}(1/15,d_i,q_i). E i = E B i , n , D i = D B i , n ( 1/15 , d i , q i ) .
Suppose the GEB recursion begins with the fixed sequence of k max k_{\max} k m a x splits, where it may continue to split further. It must thus select ( d i , q i ) (d_i,q_i) ( d i , q i ) at B i B_i B i for every i = 1 , … , k s i = 1,\ldots,k_s i = 1 , … , k s . If all the node count events E i \mathcal{E}_i E i hold, Lemma E.7 implies that every D i \mathcal{D}_i D i must fail, otherwise GEB could not have selected the corresponding split. The only other possibility is that at least one E i \mathcal{E}_i E i fails. For j = 1 , … , k s j = 1,\ldots,k_s j = 1 , … , k s , define the event
F j = ⋂ i = 1 j ( D i c ∩ E i ) , F 0 = Ω . F_j = \bigcap_{i=1}^{j}(\mathcal{D}_i^c \cap\mathcal{E}_i), \qquad F_0 = \Omega. F j = i = 1 ⋂ j ( D i c ∩ E i ) , F 0 = Ω.
Thus, the event that the GEB recursion begins with the fixed sequence of k max k_{\max} k m a x splits is contained in the event
F k s ∪ ⋃ i = 1 k s E i c . (47) F_{k_s} \cup\bigcup_{i=1}^{k_s}\mathcal{E}_i^c. \tag*{(47)} F k s ∪ i = 1 ⋃ k s E i c . (47)
The above form will allow us to leverage our probabilistic bounds. The node count failures E i c \mathcal{E}_i^c E i c have small probabilities guaranteed by Lemma E.8, which are easy to handle. The harder task is to control F k s F_{k_s} F k s probabilistically, which will require conditional independence along the split sequence, which we show now.
For a fixed node B B B with P B > 0 P_B > 0 P B > 0 and children B 0 B_0 B 0 , B 1 B_1 B 1 , and any finite collection of fixed measurable sets C 1 , … , C r ⊆ B c C_1,\ldots,C_r \subseteq B^c C 1 , … , C r ⊆ B c , we have
( N B 0 , N B 1 ) ⊥ ⊥ ( N C 1 , … , N C r ) ∣ N B , (48) (N_{B_0},N_{B_1}) \perp\!\!\!\perp(N_{C_1},\ldots,N_{C_r}) \mid N_B, \tag*{(48)} ( N B 0 , N B 1 ) ⊥ ⊥ ( N C 1 , … , N C r ) ∣ N B , (48)
N B 0 ∣ N B ∼ Binomial ( N B , P B 0 ∣ B ) . N_{B_0} \mid N_B \sim\operatorname{Binomial}(N_B,P_{B_0\mid B}). N B 0 ∣ N B ∼ Binomial ( N B , P B 0 ∣ B ) .
To see this, first suppose that 0 < P B < 1 0 < P_B < 1 0 < P B < 1 and that C 1 , … , C r C_1,\ldots,C_r C 1 , … , C r form a partition of B c B^c B c . The conclusion then follows directly from the properties of the multinomial distribution, e.g. [1 Section 2.1.5] . For arbitrary C 1 , … , C r ⊆ B c C_1,\ldots,C_r \subseteq B^c C 1 , … , C r ⊆ B c , choose a finite partition of B c B^c B c such that each C i C_i C i is a union of cells. Since each N C i N_{C_i} N C i is a sum of cell counts, the same conditional independence holds for ( N C 1 , … , N C r ) (N_{C_1},\ldots,N_{C_r}) ( N C 1 , … , N C r ) . If P B = 1 P_B = 1 P B = 1 , all external counts are zero a.s., and the conclusion is immediate.
We now leverage the above for a fixed stopping split j j j . For each i < j i < j i < j , the events E i \mathcal{E}_i E i and D i D_i D i depend only on the counts in B i B_i B i and B i 0 B_{i0} B i 0 , where B i 0 B_{i0} B i 0 is the left child of B i B_i B i . Since no earlier split cuts through B j B_j B j , each such region C C C either contains B j B_j B j or is disjoint from it. Hence
N C = N C ∖ B j + 1 { B j ⊆ C } N B j . N_C = N_{C\setminus B_j} + \mathbb{1}\{B_j \subseteq C\}N_{B_j}. N C = N C ∖ B j + 1 { B j ⊆ C } N B j .
The indicator is deterministic because the split sequence is fixed. Thus, given N B j N_{B_j} N B j , counts on fixed subsets of B j c B_j^c B j c determine F j − 1 F_{j-1} F j − 1 , while the child count N B j 0 N_{B_{j0}} N B j 0 determines D j D_j D j . By (E.17),
D j ⊥ ⊥ F j − 1 ∣ N B j . (49) D_j \perp\!\!\!\perp F_{j-1} \mid N_{B_j}. \tag*{(49)} D j ⊥ ⊥ F j − 1 ∣ N B j . (49)
We will now bound P ( F j ) \mathbb{P}(F_j) P ( F j ) recursively, using F j = F j − 1 ∩ E j ∩ D j c F_j = F_{j-1} \cap\mathcal{E}_j \cap D_j^c F j = F j − 1 ∩ E j ∩ D j c . Since N B j N_{B_j} N B j determines E j \mathcal{E}_j E j , conditioning on N B j N_{B_j} N B j and 1 F j − 1 \mathbb{1}_{F_{j-1}} 1 F j − 1 gives
P ( F j ) = E [ 1 F j − 1 1 E j P ( D j c ∣ N B j , 1 F j − 1 ) ] = E [ 1 F j − 1 1 E j P ( D j c ∣ N B j ) ] , \mathbb{P}(F_j) = \mathbb{E}\left[\mathbb{1}_{F_{j-1}}\mathbb{1}_{\mathcal{E}_j}\mathbb{P}\left(D_j^c \mid N_{B_j},\mathbb{1}_{F_{j-1}}\right)\right]
= \mathbb{E}\left[\mathbb{1}_{F_{j-1}}\mathbb{1}_{\mathcal{E}_j}\mathbb{P}\left(D_j^c \mid N_{B_j}\right)\right], P ( F j ) = E [ 1 F j − 1 1 E j P ( D j c ∣ N B j , 1 F j − 1 ) ] = E [ 1 F j − 1 1 E j P ( D j c ∣ N B j ) ] ,
where the first equality follows from the tower property, and the second equality follows from the conditional independence in (E.18).
On E j \mathcal{E}_j E j , the observed count k = N B j k = N_{B_j} k = N B j satisfies ∣ k / ( n P B j ) − 1 ∣ ≤ 1 / log n \left|k/(nP_{B_j}) - 1\right| \le1/\log n k / ( n P B j ) − 1 ≤ 1/ log n . Lemma E.8 with t = 1 / 15 t = 1/15 t = 1/15 therefore bounds P ( D j c ∣ N B j ) \mathbb{P}(D_j^c \mid N_{B_j}) P ( D j c ∣ N B j ) by 2 n − 1 / 15 2n^{-1/15} 2 n − 1/15 on this event. Hence, for sufficiently large n n n , uniformly over A A A , the candidate sequences, and j j j ,
P ( F j ) ≤ 2 n − 1 / 15 P ( F j − 1 ∩ E j ) ≤ 2 n − 1 / 15 P ( F j − 1 ) . \mathbb{P}(F_j) \le2n^{-1/15}\mathbb{P}(F_{j-1} \cap\mathcal{E}_j) \le2n^{-1/15}\mathbb{P}(F_{j-1}). P ( F j ) ≤ 2 n − 1/15 P ( F j − 1 ∩ E j ) ≤ 2 n − 1/15 P ( F j − 1 ) .
For j = 1 j = 1 j = 1 , the same calculation holds with F 0 = Ω F_0 = \Omega F 0 = Ω . Iteration yields
P ( F k s ) ≤ ( 2 n − 1 / 15 ) k s . \mathbb{P}(F_{k_s}) \le(2n^{-1/15})^{k_s}. P ( F k s ) ≤ ( 2 n − 1/15 ) k s .
Lemma E.8 also gives P ( E i c ) ≤ n − log n \mathbb{P}(\mathcal{E}_i^c) \le n^{-\log n} P ( E i c ) ≤ n − l o g n for every i i i . By (E.16), the probability that the GEB recursion begins with the fixed sequence of k max k_{\max} k m a x splits is at most
( 2 n − 1 / 15 ) k s + k s n − log n . (2n^{-1/15})^{k_s} + k_s n^{-\log n}. ( 2 n − 1/15 ) k s + k s n − l o g n .
By (E.6), there are at most k max ! { D ( m − 1 ) } k max k_{\max}! \{D(m-1)\}^{k_{\max}} k m a x ! { D ( m − 1 ) } k m a x candidate sequences of length k max k_{\max} k m a x . The event H A , n \mathcal{H}_{A,n} H A , n occurs exactly when the recursion begins with one of these sequences, so taking a union bound over all possible k max k_{\max} k m a x split sequences completes the proof.
The two probabilistic bounds in this section correspond to the two requirements of admissible trees, which we now leverage to prove Theorem 2 in the next section.
Proof of Theorem 2
By Definition 2, an admissible tree is obtained by splitting identifiable nodes near-optimally until the leaves are non-identifiable, followed by fewer than k max k_{\max} k m a x additional splits below each leaf. We show that GEB satisfies both requirements with probability tending to one.
Suppose the event ⋂ A ∈ I n ∗ G A , n \bigcap_{A\in\mathcal{I}_n^*}\mathcal{G}_{A,n} ⋂ A ∈ I n ∗ G A , n holds, with G A , n \mathcal{G}_{A,n} G A , n defined in Lemma E.6. For each A ∈ I n ∗ A\in\mathcal{I}_n^* A ∈ I n ∗ , the event G A , n \mathcal{G}_{A,n} G A , n ensures that all candidate splits are eligible and that the selected split is near-optimal with a positive log split Bayes factor. If the root is identifiable, it belongs to I n ∗ \mathcal{I}_n^* I n ∗ , so GEB splits it near-optimally. After a near-optimal split on a node A ∈ I n ∗ A\in\mathcal{I}_n^* A ∈ I n ∗ , each identifiable child also belongs to I n ∗ \mathcal{I}_n^* I n ∗ . Induction therefore shows that under ⋂ A ∈ I n ∗ G A , n \bigcap_{A\in\mathcal{I}_n^*}\mathcal{G}_{A,n} ⋂ A ∈ I n ∗ G A , n , GEB starting at the root splits every identifiable node it reaches near-optimally without stopping.
By Lemma E.9 and Proposition E.1, a union bound over I n ∗ \mathcal{I}_n^* I n ∗ gives
P ( ⋃ A ∈ I n ∗ G A , n c ) ≲ ( n C 0 log n ) t 0 ( 2 D m n t 0 + 1 n log n ) ≲ ( log n ) − t 0 ( 1 + n t 0 − log n ) → 0 , \mathbb{P}\left(\bigcup_{A\in\mathcal{I}_n^*}\mathcal{G}_{A,n}^{c}\right)\lesssim\left(\frac{n}{C_0\log n}\right)^{t_0}\left(\frac{2D m}{n^{t_0}}+\frac{1}{n^{\log n}}\right)
\lesssim(\log n)^{-t_0}\left(1+n^{t_0-\log n}\right)\to0, P A ∈ I n ∗ ⋃ G A , n c ≲ ( C 0 log n n ) t 0 ( n t 0 2 D m + n l o g n 1 ) ≲ ( log n ) − t 0 ( 1 + n t 0 − l o g n ) → 0 ,
since t 0 t_0 t 0 is a positive constant. Thus, with probability tending to one, GEB splits every identifiable node it reaches near-optimally, establishing the first requirement of admissibility.
For the second requirement, suppose the event ⋂ A ∈ L n ∗ H A , n c \bigcap_{A\in\mathcal{L}_n^*}\mathcal{H}_{A,n}^{c} ⋂ A ∈ L n ∗ H A , n c holds, with H A , n \mathcal{H}_{A,n} H A , n defined in Lemma E.10. On this event, applying the GEB splitting and stopping rules starting at any A ∈ L n ∗ A\in\mathcal{L}_n^* A ∈ L n ∗ produces a subtree with fewer than k max k_{\max} k m a x splits. When the first requirement of admissibility holds, the first non-identifiable node A A A on each branch of T n T_n T n belongs to L n ∗ \mathcal{L}_n^* L n ∗ . For each such A A A , the event H A , n c \mathcal{H}_{A,n}^{c} H A , n c holds, so the subtree of T n T_n T n rooted at A A A contains fewer than k max k_{\max} k m a x splits in total, establishing the second requirement of admissibility. By Lemma E.10 and Proposition E.1, a union bound gives
P ( ⋃ A ∈ L n ∗ H A , n ) ≲ ( n C 0 log n ) t 0 k max ! { D ( m − 1 ) } k max { ( 2 n − 1 / 15 ) ⌈ 15 t 0 ⌉ + ⌈ 15 t 0 ⌉ n − log n } \mathbb{P}\left(\bigcup_{A\in\mathcal{L}_n^*}\mathcal{H}_{A,n}\right) \lesssim\left(\frac{n}{C_0\log n}\right)^{t_0} k_{\max}! \{D(m-1)\}^{k_{\max}} \left\{(2n^{-1/15})^{\lceil15t_0\rceil}+\lceil15t_0\rceil n^{-\log n}\right\} P A ∈ L n ∗ ⋃ H A , n ≲ ( C 0 log n n ) t 0 k m a x ! { D ( m − 1 ) } k m a x { ( 2 n − 1/15 ) ⌈ 15 t 0 ⌉ + ⌈ 15 t 0 ⌉ n − l o g n }
≲ ( log n ) − t 0 { n t 0 − ⌈ 15 t 0 ⌉ / 15 + n t 0 − log n } → 0 , \lesssim(\log n)^{-t_0}\left\{n^{t_0-\lceil15t_0\rceil/15}+n^{t_0-\log n}\right\}\to0, ≲ ( log n ) − t 0 { n t 0 − ⌈ 15 t 0 ⌉ /15 + n t 0 − l o g n } → 0 ,
since k max k_{\max} k m a x is a constant and ⌈ 15 t 0 ⌉ / 15 ≥ t 0 \lceil15t_0\rceil/15\ge t_0 ⌈ 15 t 0 ⌉ /15 ≥ t 0 .
Both requirements of an admissible tree from Definition 2 thus hold on the event
( ⋂ A ∈ I n ∗ G A , n ) ∩ ( ⋂ A ∈ L n ∗ H A , n c ) . (50) \left(\bigcap_{A\in\mathcal{I}_n^*}\mathcal{G}_{A,n}\right)\cap\left(\bigcap_{A\in\mathcal{L}_n^*}\mathcal{H}_{A,n}^{c}\right). \tag*{(50)} A ∈ I n ∗ ⋂ G A , n ∩ A ∈ L n ∗ ⋂ H A , n c . (50)
Additionally, if the root A A A is non-identifiable, the first requirement of admissibility holds automatically. By the two preceding probabilistic bounds and a union bound argument, the event (F.1) has probability tending to one, so
P ( T n ∈ T n ) → 1. \mathbb{P}(T_n\in\mathcal{T}_n)\to1. P ( T n ∈ T n ) → 1.
Posterior contraction
By Theorem 2, the GEB tree is admissible with probability tending to one. Admissibility controls both the remaining signal in its leaves and the number of leaf probabilities to be estimated. We first use these properties to establish convergence of the posterior mean.
Let P n \mathcal{P}_n P n be the leaf partition of the GEB tree T n T_n T n . Recall from (2) that the posterior mean density on this partition is
f ˉ n ( x ) = ∑ A ∈ P n G ˉ A h ( x ∣ A ) , G ˉ A = α H A + N A n + α . \bar{f}_n(x)=\sum_{A\in\mathcal{P}_n}\bar{G}_A h(x\mid A),\qquad\bar{G}_A=\frac{\alpha H_A+N_A}{n+\alpha}. f ˉ n ( x ) = A ∈ P n ∑ G ˉ A h ( x ∣ A ) , G ˉ A = n + α α H A + N A .
To separate the prior contribution from the error due to partitioning and estimating the cell probabilities, define
f ^ n ( x ) = ∑ A ∈ P n P ^ A h ( x ∣ A ) , P ^ A = N A n , \widehat{f}_n(x)=\sum_{A\in\mathcal{P}_n}\widehat{P}_A h(x\mid A),\qquad\widehat{P}_A=\frac{N_A}{n}, f n ( x ) = A ∈ P n ∑ P A h ( x ∣ A ) , P A = n N A ,
and its population counterpart
f n ( x ) = ∑ A ∈ P n P A h ( x ∣ A ) . f_n(x)=\sum_{A\in\mathcal{P}_n}P_A h(x\mid A). f n ( x ) = A ∈ P n ∑ P A h ( x ∣ A ) .
Since H H H is uniform, both are constant on each leaf. The triangle inequality gives
∥ p − f ˉ n ∥ L 1 ≤ ∥ f ^ n − f ˉ n ∥ L 1 + ∥ p − f n ∥ L 1 + ∥ f n − f ^ n ∥ L 1 . (51) \lVert p-\bar f_n\rVert_{L_1}\le\lVert\hat f_n-\bar f_n\rVert_{L_1}+\lVert p-f_n\rVert_{L_1}+\lVert f_n-\hat f_n\rVert_{L_1}. \tag*{(51)} ∥ p − f ˉ n ∥ L 1 ≤ ∥ f ^ n − f ˉ n ∥ L 1 + ∥ p − f n ∥ L 1 + ∥ f n − f ^ n ∥ L 1 . (51)
The first term measures the effect of the prior on the cell probabilities. The second is the approximation bias of the partition, and the third is the sampling error in estimating its cell probabilities. With the prior-bias-variance decomposition, we are able to show the posterior mean convergence based on the admissibility of the GEB tree.
Lemma G.1 (Posterior mean convergence). Fix m ≥ 2 m\ge2 m ≥ 2 and α > 0 \alpha>0 α > 0 , and suppose Assumptions 1, 2, and 3 hold. There exists a constant C > 0 C>0 C > 0 , independent of n n n , such that
P { ∥ p − f ˉ n ∥ L 1 ≤ C τ n ( 1 − 1 / γ ) / 2 } → 1. \mathbb{P}\left\{\lVert p-\bar f_n\rVert_{L_1}\le C\tau_n^{(1-1/\gamma)/2}\right\}\to1. P { ∥ p − f ˉ n ∥ L 1 ≤ C τ n ( 1 − 1/ γ ) /2 } → 1.
Proof. We bound the three terms in (G.1) in order. For the prior contribution, observe that
G ˉ A − P ^ A = α n + α ( H A − P ^ A ) . \bar G_A-\hat P_A=\frac{\alpha}{n+\alpha}(H_A-\hat P_A). G ˉ A − P ^ A = n + α α ( H A − P ^ A ) .
The cells are disjoint and each conditional base density integrates to one, so
∥ f ^ n − f ˉ n ∥ L 1 = ∑ A ∈ P n ∣ G ˉ A − P ^ A ∣ ≤ α n + α ∑ A ∈ P n ( H A + P ^ A ) = 2 α n + α . (52) \lVert\hat f_n-\bar f_n\rVert_{L_1}=\sum_{A\in\mathcal{P}_n}\left|\bar G_A-\hat P_A\right|\le\frac{\alpha}{n+\alpha}\sum_{A\in\mathcal{P}_n}(H_A+\hat P_A)=\frac{2\alpha}{n+\alpha}. \tag*{(52)} ∥ f ^ n − f ˉ n ∥ L 1 = A ∈ P n ∑ G ˉ A − P ^ A ≤ n + α α A ∈ P n ∑ ( H A + P ^ A ) = n + α 2 α . (52)
This bound holds uniformly over any P n \mathcal{P}_n P n .
We next work on the admissibility event { T n ∈ T n } \{T_n\in\mathcal{T}_n\} { T n ∈ T n } to control the bias and variance. By Definition 2, each leaf is non-identifiable or is a descendant of a non-identifiable node. Since Lemma 1 gives S A 0 + S A 1 ≤ S A S_{A_0}+S_{A_1}\le S_A S A 0 + S A 1 ≤ S A , signal cannot increase along a path, so S A < τ n S_A<\tau_n S A < τ n for every A ∈ P n A\in\mathcal{P}_n A ∈ P n . Applying the signal identity (7) and the admissible leaf count (E.4) gives
K ( p ; f n ) = ∑ A ∈ P n S A ≤ ∣ P n ∣ τ n ≲ τ n 1 − 1 / γ . K(p;f_n)=\sum_{A\in\mathcal{P}_n}S_A\le|\mathcal{P}_n|\tau_n\lesssim\tau_n^{1-1/\gamma}. K ( p ; f n ) = A ∈ P n ∑ S A ≤ ∣ P n ∣ τ n ≲ τ n 1 − 1/ γ .
Pinsker’s inequality therefore yields
∥ p − f n ∥ L 1 ≤ 2 K ( p ; f n ) ≲ τ n ( 1 − 1 / γ ) / 2 . (53) \lVert p-f_n\rVert_{L_1}\le\sqrt{2K(p;f_n)}\lesssim\tau_n^{(1-1/\gamma)/2}. \tag*{(53)} ∥ p − f n ∥ L 1 ≤ 2 K ( p ; f n ) ≲ τ n ( 1 − 1/ γ ) /2 . (53)
For the variance term, following the proof of Lugosi and Nobel [1996, Theorem 1], we first express the sampling error through cell probability differences:
∥ f n − f ^ n ∥ L 1 = ∑ A ∈ P n ∣ P ^ A − P A ∣ . \lVert f_n-\hat f_n\rVert_{L_1}=\sum_{A\in\mathcal{P}_n}\left|\hat P_A-P_A\right|. ∥ f n − f ^ n ∥ L 1 = A ∈ P n ∑ P ^ A − P A .
Since the empirical and population cell probabilities both sum to one, the total excess probability on cells with P ^ A > P A \widehat{P}_A>P_A P A > P A equals the total missing probability on cells with P ^ A < P A \widehat{P}_A<P_A P A < P A . As in the proof of Lugosi and Nobel [1996, Lemma 1], this gives
∑ A ∈ P n ∣ P ^ A − P A ∣ = 2 ∑ A ∈ P n P ^ A > P A ( P ^ A − P A ) . \sum_{A\in\mathcal{P}_n}\left|\widehat{P}_A-P_A\right|=2\sum_{\substack{A\in\mathcal{P}_n\\\widehat{P}_A>P_A}}\left(\widehat{P}_A-P_A\right). A ∈ P n ∑ P A − P A = 2 A ∈ P n P A > P A ∑ ( P A − P A ) .
On { T n ∈ T n } \{T_n\in\mathcal{T}_n\} { T n ∈ T n } , the union of the cells in the last sum belongs to the deterministic collection B n \mathcal{B}_n B n from Lemma E.3, so
∥ f n − f ^ n ∥ L 1 ≤ 2 sup B ∈ B n ∣ P ^ B − P B ∣ , (54) \left\|f_n-\widehat{f}_n\right\|_{L_1}\leq2\sup_{B\in\mathcal{B}_n}\left|\widehat{P}_B-P_B\right|, \tag*{(54)} f n − f n L 1 ≤ 2 B ∈ B n sup P B − P B , (54)
where P ^ B = n − 1 ∑ i = 1 n 1 ( X i ∈ B ) \widehat{P}_B=n^{-1}\sum_{i=1}^{n}\mathbf{1}(X_i\in B) P B = n − 1 ∑ i = 1 n 1 ( X i ∈ B ) . Thus it suffices to control empirical probability deviations uniformly over B n \mathcal{B}_n B n .
We now bound the above supremum under the unconditional sampling law. For each fixed B ∈ B n B\in\mathcal{B}_n B ∈ B n , Hoeffding’s inequality gives, for any fixed L > 0 L>0 L > 0 ,
P ( ∣ P ^ B − P B ∣ > L τ n ( 1 − 1 / γ ) / 2 ) ≤ 2 exp ( − 2 n L 2 τ n 1 − 1 / γ ) = 2 exp ( − 2 L 2 C 0 τ n − 1 / γ log n ) . \mathbb{P}\left(\left|\widehat{P}_B-P_B\right|>L\tau_n^{(1-1/\gamma)/2}\right)\leq2\exp\left(-2nL^2\tau_n^{1-1/\gamma}\right)=2\exp\left(-2L^2C_0\tau_n^{-1/\gamma}\log n\right). P ( P B − P B > L τ n ( 1 − 1/ γ ) /2 ) ≤ 2 exp ( − 2 n L 2 τ n 1 − 1/ γ ) = 2 exp ( − 2 L 2 C 0 τ n − 1/ γ log n ) .
By the complexity bound (E.5), there exists a constant C B > 0 C_{\mathcal{B}}>0 C B > 0 , independent of n n n , such that
log ∣ B n ∣ ≤ C B τ n − 1 / γ log n . \log|\mathcal{B}_n|\leq C_{\mathcal{B}}\tau_n^{-1/\gamma}\log n. log ∣ B n ∣ ≤ C B τ n − 1/ γ log n .
Choose L L L sufficiently large that 2 L 2 C 0 > C B 2L^2C_0>C_{\mathcal{B}} 2 L 2 C 0 > C B . A union bound then gives
P ( sup B ∈ B n ∣ P ^ B − P B ∣ > L τ n ( 1 − 1 / γ ) / 2 ) ≤ 2 exp { − ( 2 L 2 C 0 − C B ) τ n − 1 / γ log n } → 0. \mathbb{P}\left(\sup_{B\in\mathcal{B}_n}\left|\widehat{P}_B-P_B\right|>L\tau_n^{(1-1/\gamma)/2}\right)\leq2\exp\left\{-(2L^2C_0-C_{\mathcal{B}})\tau_n^{-1/\gamma}\log n\right\}\to0. P ( B ∈ B n sup P B − P B > L τ n ( 1 − 1/ γ ) /2 ) ≤ 2 exp { − ( 2 L 2 C 0 − C B ) τ n − 1/ γ log n } → 0.
By the above bound and Theorem 2, we have that the following joint event holds with probability tending to 1:
{ T n ∈ T n , sup B ∈ B n ∣ P ^ B − P B ∣ ≤ L τ n ( 1 − 1 / γ ) / 2 } . \left\{T_n\in\mathcal{T}_n,\ \sup_{B\in\mathcal{B}_n}\left|\widehat{P}_B-P_B\right|\leq L\tau_n^{(1-1/\gamma)/2}\right\}. { T n ∈ T n , B ∈ B n sup P B − P B ≤ L τ n ( 1 − 1/ γ ) /2 } .
On this event, (G.4) gives
∥ f n − f ^ n ∥ L 1 ≤ 2 L τ n ( 1 − 1 / γ ) / 2 . \left\|f_n-\widehat{f}_n\right\|_{L_1}\leq2L\tau_n^{(1-1/\gamma)/2}. f n − f n L 1 ≤ 2 L τ n ( 1 − 1/ γ ) /2 .
Moreover, (G.2) and (G.3) show that the prior contribution is of smaller order and the approximation bias is of order at most τ n ( 1 − 1 / γ ) / 2 \tau_n^{(1-1/\gamma)/2} τ n ( 1 − 1/ γ ) /2 . Combining the three bounds in (G.1) gives, on this event and for sufficiently large n n n ,
∥ p − f ‾ n ∥ L 1 ≲ τ n ( 1 − 1 / γ ) / 2 . \left\|p-\overline{f}_n\right\|_{L_1}\lesssim\tau_n^{(1-1/\gamma)/2}. p − f n L 1 ≲ τ n ( 1 − 1/ γ ) /2 .
Since this event has probability tending to one, the result follows.
Proof of Theorem 3
Let ε n = τ n ( 1 − 1 / γ ) / 2 \varepsilon_n=\tau_n^{(1-1/\gamma)/2} ε n = τ n ( 1 − 1/ γ ) /2 , let C C C be the constant in Lemma G.1, and fix M > C M>C M > C .
Conditional on the data, P n \mathcal{P}_n P n is fixed, and the posterior cell probabilities satisfy
( G A : A ∈ P n ) ∣ x 1 : n ∼ Dirichlet ( α H A + N A : A ∈ P n ) . (G_A:A\in\mathcal{P}_n)\mid x_{1:n}\sim\operatorname{Dirichlet}(\alpha H_A+N_A:A\in\mathcal{P}_n). ( G A : A ∈ P n ) ∣ x 1 : n ∼ Dirichlet ( α H A + N A : A ∈ P n ) .
Their conditional means and variances are
E Π [ G A ∣ x 1 : n ] = G ‾ A , Var Π ( G A ∣ x 1 : n ) = G ‾ A ( 1 − G ‾ A ) n + α + 1 . \mathbb{E}_{\Pi}[G_A\mid x_{1:n}]=\overline{G}_A,\qquad
\operatorname{Var}_{\Pi}(G_A\mid x_{1:n})=
\frac{\overline{G}_A(1-\overline{G}_A)}{n+\alpha+1}. E Π [ G A ∣ x 1 : n ] = G A , Var Π ( G A ∣ x 1 : n ) = n + α + 1 G A ( 1 − G A ) .
Since the cells are disjoint and h ( ⋅ ∣ A ) h(\cdot\mid A) h ( ⋅ ∣ A ) integrates to 1 over A A A , Cauchy–Schwarz gives
∥ f − f ‾ n ∥ L 1 2 = ( ∑ A ∈ P n ∣ G A − G ‾ A ∣ ) 2 ≤ ∣ P n ∣ ∑ A ∈ P n ( G A − G ‾ A ) 2 . \|f-\overline{f}_n\|_{L_1}^2=
\left(\sum_{A\in\mathcal{P}_n}|G_A-\overline{G}_A|\right)^2
\leq|\mathcal{P}_n|\sum_{A\in\mathcal{P}_n}(G_A-\overline{G}_A)^2. ∥ f − f n ∥ L 1 2 = ( A ∈ P n ∑ ∣ G A − G A ∣ ) 2 ≤ ∣ P n ∣ A ∈ P n ∑ ( G A − G A ) 2 .
Taking posterior expectations therefore yields
E Π [ ∥ f − f ‾ n ∥ L 1 2 ∣ x 1 : n ] ≤ ∣ P n ∣ ∑ A ∈ P n Var Π ( G A ∣ x 1 : n ) = ∣ P n ∣ n + α + 1 ( 1 − ∑ A ∈ P n G ‾ A 2 ) ≤ ∣ P n ∣ n + α + 1 . (55) \begin{aligned}
\mathbb{E}_{\Pi}[\|f-\overline{f}_n\|_{L_1}^2\mid x_{1:n}]
\leq|\mathcal{P}_n|\sum_{A\in\mathcal{P}_n}\operatorname{Var}_{\Pi}(G_A\mid x_{1:n}) \\
&= \frac{|\mathcal{P}_n|}{n+\alpha+1}\left(1-\sum_{A\in\mathcal{P}_n}\overline{G}_A^2\right) \\
&\leq\frac{|\mathcal{P}_n|}{n+\alpha+1}. \tag*{(55)}
\end{aligned} E Π [ ∥ f − f n ∥ L 1 2 ∣ x 1 : n ] ≤ ∣ P n ∣ A ∈ P n ∑ Var Π ( G A ∣ x 1 : n ) = n + α + 1 ∣ P n ∣ ( 1 − A ∈ P n ∑ G A 2 ) ≤ n + α + 1 ∣ P n ∣ . (55)
For a singleton partition, its posterior cell probability is one almost surely, and the same bound holds. Thus (G.5) holds for every observed sample.
Next, fix an observed sample x 1 : n x_{1:n} x 1 : n . If ∥ p − f ‾ n ∥ L 1 ≤ C ε n \|p-\overline{f}_n\|_{L_1}\leq C\varepsilon_n ∥ p − f n ∥ L 1 ≤ C ε n , then by the triangle inequality, every posterior draw of f f f satisfying ∥ p − f ∥ L 1 ≥ M ε n \|p-f\|_{L_1}\geq M\varepsilon_n ∥ p − f ∥ L 1 ≥ M ε n also satisfies
∥ f − f ‾ n ∥ L 1 ≥ ∥ p − f ∥ L 1 − ∥ p − f ‾ n ∥ L 1 ≥ ( M − C ) ε n . \|f-\overline{f}_n\|_{L_1}\geq\|p-f\|_{L_1}-\|p-\overline{f}_n\|_{L_1}\geq(M-C)\varepsilon_n. ∥ f − f n ∥ L 1 ≥ ∥ p − f ∥ L 1 − ∥ p − f n ∥ L 1 ≥ ( M − C ) ε n .
If instead ∥ p − f ‾ n ∥ L 1 > C ε n \|p-\overline{f}_n\|_{L_1}>C\varepsilon_n ∥ p − f n ∥ L 1 > C ε n , we bound the posterior tail probability by one. Combining the two cases, the below holds for P \mathbb{P} P -almost all X 1 : n X_{1:n} X 1 : n :
Π { ∥ p − f ∥ L 1 ≥ M ε n ∣ X 1 : n } ≤ 1 { ∥ p − f ‾ n ∥ L 1 > C ε n } + Π { ∥ f − f ‾ n ∥ L 1 ≥ ( M − C ) ε n ∣ X 1 : n } ≤ 1 { ∥ p − f ‾ n ∥ L 1 > C ε n } + E Π [ ∥ f − f ‾ n ∥ L 1 2 ∣ X 1 : n ] ( M − C ) 2 ε n 2 ≤ 1 { ∥ p − f ‾ n ∥ L 1 > C ε n } + ∣ P n ∣ ( n + α + 1 ) ( M − C ) 2 ε n 2 . \begin{aligned}
\Pi\{\|p-f\|_{L_1}\geq M\varepsilon_n\mid X_{1:n}\}
\leq\mathbf{1}_{\{\|p-\overline{f}_n\|_{L_1}>C\varepsilon_n\}}
+\Pi\{\|f-\overline{f}_n\|_{L_1}\geq(M-C)\varepsilon_n\mid X_{1:n}\} \\
&\leq\mathbf{1}_{\{\|p-\overline{f}_n\|_{L_1}>C\varepsilon_n\}}
+\frac{\mathbb{E}_{\Pi}[\|f-\overline{f}_n\|_{L_1}^2\mid X_{1:n}]}{(M-C)^2\varepsilon_n^2} \\
&\leq\mathbf{1}_{\{\|p-\overline{f}_n\|_{L_1}>C\varepsilon_n\}}
+\frac{|\mathcal{P}_n|}{(n+\alpha+1)(M-C)^2\varepsilon_n^2}.
\end{aligned} Π { ∥ p − f ∥ L 1 ≥ M ε n ∣ X 1 : n } ≤ 1 { ∥ p − f n ∥ L 1 > C ε n } + Π { ∥ f − f n ∥ L 1 ≥ ( M − C ) ε n ∣ X 1 : n } ≤ 1 { ∥ p − f n ∥ L 1 > C ε n } + ( M − C ) 2 ε n 2 E Π [ ∥ f − f n ∥ L 1 2 ∣ X 1 : n ] ≤ 1 { ∥ p − f n ∥ L 1 > C ε n } + ( n + α + 1 ) ( M − C ) 2 ε n 2 ∣ P n ∣ .
The second inequality applies Markov’s inequality to ∥ f − f ˉ n ∥ L 1 2 \lVert f-\bar{f}_n\rVert_{L_1}^2 ∥ f − f ˉ n ∥ L 1 2 under the posterior, and the third uses (G.5).
By Lemma G.1, the indicator converges to zero in probability. By Theorem 2 and Lemma E.2, ∣ P n ∣ ≲ τ n − 1 / γ |\mathcal{P}_n| \lesssim\tau_n^{-1/\gamma} ∣ P n ∣ ≲ τ n − 1/ γ with probability tending to one, so the second term is of order O P { ( log n ) − 1 } O_{\mathbb{P}}\{(\log n)^{-1}\} O P {( log n ) − 1 } which also converges to zero in P \mathbb{P} P -probability. Substituting τ n = C 0 log n / n \tau_n=C_0\log n/n τ n = C 0 log n / n and absorbing C 0 ( 1 − 1 / γ ) / 2 C_0^{(1-1/\gamma)/2} C 0 ( 1 − 1/ γ ) /2 into M M M gives the stated rate.
Experimental details
Model hyperparameters
For completeness, we list the hyperparameter settings of all methods used in the experiments.
GEB: We set the shrinkage parameter to α = 1 \alpha=1 α = 1 and divide each coordinate into m = 32 m=32 m = 32 grid subintervals. For the density-estimation experiments, we use the PPB posterior mean given the GEB selected partition. For the posterior-uncertainty experiment, we generate B = 1,000 B=1{,}000 B = 1 , 000 posterior draws of θ \theta θ using Algorithm 1.
HMPT: We use maximum resolution 15, partition-location penalty η = 0.1 \eta=0.1 η = 0.1 , 32 grid subintervals, and 1,000 SMC particles as suggested by Awaya and Ma [2024]. The code is available at https://github.com/MaStatLab/HMPT_experiments.
DPMM: In the density-estimation experiments, we use the variational inference implementation of sklearn, with truncation level 50, concentration parameter 1, at most 1,000 iterations, and five initializations. Note that the DP concentration of 1 matches our GEB choice of α = 1 \alpha=1 α = 1 . For the posterior uncertainty experiment, we use the MCMC sampler implemented in BNPmix::PYdensity. We again set the concentration parameter to 1 and run each chain for 2,000 iterations, discarding the first 1,000 as burn-in and retaining 1,000 posterior draws.
Real-data preprocessing
We outline dataset-specific preprocessing below.
POWER: We carry out the same preprocessing steps as the MAF benchmark Papamakarios et al. [ [28 ] (https://github.com/gpapamak/maf ). In each replication, 100,000 observations are sampled without replacement from the 2,049,280 observations, with 50,000 assigned to training and 50,000 to testing.
ISOLET: In each replication, the 7,797 observations are randomly divided into 3,898 training and 3,899 test observations. To stay within the D D D range reported for HMPT in Awaya and Ma [ [2 ] , PCA is then fitted to the standardized training sample, and the first 100 principal components of the 617 original dimensions are used for analysis.
Hyperparameter sensitivity
We examine the sensitivity of GEB to the shrinkage parameter α \alpha α and grid size m m m using the Gaussian mixture simulation with D = 2 D = 2 D = 2 and the Dry Bean data. For each dataset, we use the same 20 train–test splits as in the main experiments and apply the Gaussian marginal transformation fitted to the training sample. We evaluate model fit by the log marginal likelihood per training observation and predictive performance by the average test log score:
1 n t r a i n log M ( P n ) , 1 n t e s t ∑ i = 1 n t e s t log p ^ ( X i t e s t ) . \frac{1}{n_{\mathrm{train}}}\log\mathcal{M}(\mathcal{P}_n), \qquad\frac{1}{n_{\mathrm{test}}}\sum_{i=1}^{n_{\mathrm{test}}}\log\widehat{p}(X_i^{\mathrm{test}}). n train 1 log M ( P n ) , n test 1 i = 1 ∑ n test log p ( X i test ) .
We vary α = 10 ℓ \alpha= 10^{\ell} α = 1 0 ℓ for integer ℓ \ell ℓ from − 2 -2 − 2 to 4 4 4 with m = 32 m = 32 m = 32 fixed, and m = 2 j m = 2^j m = 2 j for integer j j j from 1 to 9 with α = 1 \alpha= 1 α = 1 fixed. Figure H.1 shows the means and interquartile ranges of both criteria across the 20 splits.
At m = 32 m = 32 m = 32 , the test log score and log marginal likelihood both initially increase with α \alpha α and then decline. This pattern is consistent with the competing effects of partition resolution and posterior shrinkage discussed in the main text. We retain α = 1 \alpha= 1 α = 1 as a default for robustness, while the similar trends of the two criteria suggest that additional marginal-likelihood tuning may improve predictive performance.
At α = 1 \alpha= 1 α = 1 , both criteria generally increase with m m m , with diminishing gains beyond a moderate grid size. The resolution needed depends on the density structure, as smooth densities may require fewer candidate splits, whereas sharp local features can benefit from a denser grid. Increasing m m m also raises the candidate-search cost (but see Theorem 1). Following the grid choice in Awaya and Ma [2024], we use m = 32 m = 32 m = 32 in the main paper to balance resolution and computation. Across both datasets, the agreement between the marginal likelihood and test log score supports the use of empirical Bayes tuning for α \alpha α and m m m as well, with more noticeable improvements for Dry Bean.
Figure H.1. Sensitivity of GEB to α \alpha α and m m m for the Gaussian mixture simulation with D = 2 D = 2 D = 2 (top) and Dry Bean (bottom). The left panels vary α \alpha α from 10 − 2 10^{-2} 1 0 − 2 to 10 4 10^{4} 1 0 4 with m = 32 m = 32 m = 32 ; the right panels vary m m m from 2 to 512 with α = 1 \alpha= 1 α = 1 . Lines show the mean test log score and log marginal likelihood per training observation across 20 train–test splits, and shaded regions show the corresponding interquartile ranges.