Introduction
Estimating the effect of a policy or treatment is a key question in econometrics and biostatistics. Randomization is the gold standard to avoid confounding, but cannot always be attained due to ethical or practical concerns. In the observational setting, we need to account for treatment assignment and adjust for confounding to obtain valid inference [63 ] . Besides the average treatment effect (ATE), we might also be interested in the conditional average treatment effect (CATE). The CATE indicates the expected benefit that an individual might have from a given treatment, given their observed covariates. It is especially of interest for making cost-effective policy choices and avoiding harm when treating a population. A medicine that might benefit the population on average might severely harm a certain subgroup [41 ] .
Treatment heterogeneity is notoriously hard to estimate, as many randomized controlled trials (RCTs) are not powered to detect it [36 , 41 ] . Furthermore, treatment heterogeneity can be sparse or nonexistent, calling for methods that can deal with this setting. Causal machine learning methods like R-, T-, and S-learners can struggle with smaller sample sizes, depending on the structure of the treatment effect and the underlying learner, as learning relationships becomes harder in data sets where signal is hard to find [53 , 45 ] . Bayesian Causal Forests (BCF) have been shown to perform well in settings with weak treatment signals, as they are flexible tree ensemble methods whose priors on the tree structure and leaf values act as regularizers [34 ] . The choice of these priors is therefore central to controlling the estimated treatment heterogeneity.
BCF use separate prognostic and treatment-effect ensembles, which helps prevent the prognostic signal from distorting regularization of the treatment effect [33 , 34 ] . The leaf prior in the treatment ensemble then determines how strongly regional effects are regularized. The BART prior assigns the same variance to every leaf, while the recently proposed Horseshoe Forests consider independent local scales [38 ] . These choices fix the relative variances to be equal or leave them unrelated, hence, they do not model how treatment-effect variation is distributed across the regions of a tree.
Joint variance allocation has proved useful in high-dimensional regression, where a total prior variance is distributed across coefficients to shrink negligible signals while retaining substantial ones [13 , 77 , 2 , 76 ] . We adapt this principle to treatment trees through the Dirichlet-Tree Bayesian Causal Forest, DT-BCF. Our contributions are as follows:
We construct a topology-indexed family of priors that separates the total leaf variance of each treatment tree from its allocation across regions. The family contains equal division as a limiting allocation and Dirichlet-tree priors with random unequal shares.
We characterize the marginal shrinkage of the leaf values and derive a theoretical limit on how many regions can express substantial treatment-effect heterogeneity. We also establish posterior consistency.
We benchmark DT-BCF against Bayesian tree priors and external learners in controlled simulations, a causal-inference benchmark, and a real-data application. DT-BCF performs comparably to the Bayesian alternatives across a broad range of settings, with the largest gains when heterogeneity is concentrated in a few regions.
Related work
Bayesian tree models have been used for causal inference and treatment effect estimation [36 , 31 ] . BCF separates the prognostic and treatment-effect functions so that each can receive its own regularization and includes the estimated propensity score in the prognostic function to mitigate regularization-induced confounding [33 , 34 ] . Later extensions modify different parts of this construction. Targeted smooth BCF imposes smoothness along a designated effect modifier [66 ] . Shrinkage BCF places sparsity-inducing priors on the splitting probabilities to identify covariates that drive prognosis and treatment heterogeneity [17 , 47 ] . Horseshoe Forests consider using the Horseshoe (HS) prior on the leaf nodes to allow different amounts of shrinkage across the covariate space [38 ] . DT-BCF changes the dependence among the leaf variances within each treatment tree, jointly regularizing regional contributions.
The closest prior constructions arise in joint shrinkage and variance decomposition. The Dirichlet–Laplace prior distributes a global scale across regression coefficients, while R 2 R^2 R 2 based priors allocate explained variance across model terms [13 , 77 , 2 , 4 , 76 , 43 ] . Hierarchical decomposition priors allocate variance along a tree of additive model components specified by the analyst (Fuglstad et al., 2020). These approaches separate the overall amount of variation from its relative allocation among a fixed collection of model components. In DT-BCF, we allocate the variance by using the discovered topology of the tree recursively.
Outside Bayesian tree ensembles, the R-, X-learners, causal forests, multi-task Gaussian processes, and neural networks provide alternative estimators of heterogeneous effects [53 , 45 , 40 , 11 , 8 , 65 ] . Our experiments emphasize comparisons among Bayesian tree priors and use the X- and R-learners as external references.
Methods
We first review the BCF treatment ensemble and identify the role of the leaf prior. We then show how to specify a tree prior which separates the total leaf variance of tree from its allocation across regions. To construct the allocation, we make use of the topology of tree. We also study the prior’s properties. Proofs are shown in Appendix § 11 .
Bayesian Causal Forests
For unit i i i , let A i ∈ { 0 , 1 } A_i \in\{0,1\} A i ∈ { 0 , 1 } denote treatment, x i ∈ R p \boldsymbol{x}_i \in\mathbb{R}^p x i ∈ R p the observed covariates, and y i ( a ) y_i(a) y i ( a ) the potential outcome under treatment a ∈ { 0 , 1 } a \in\{0,1\} a ∈ { 0 , 1 } . The observed response is y i = y i ( A i ) y_i = y_i(A_i) y i = y i ( A i ) . We assume consistency and no interference, conditional exchangeability { y i ( 0 ) , y i ( 1 ) } ⊥ ⊥ A i ∣ x i \{y_i(0), y_i(1)\} \perp\!\!\!\perp A_i \mid\boldsymbol{x}_i { y i ( 0 ) , y i ( 1 )} ⊥ ⊥ A i ∣ x i , and positivity 0 < e ( x ) < 1 0 < e(\boldsymbol{x}) < 1 0 < e ( x ) < 1 on the covariate support, where e ( x ) = Pr ( A i = 1 ∣ x i = x ) e(\boldsymbol{x}) = \Pr(A_i = 1 \mid\boldsymbol{x}_i = \boldsymbol{x}) e ( x ) = Pr ( A i = 1 ∣ x i = x ) [37 ] . Under these conditions, the Conditional Average Treatment Effect (CATE)
τ ( x ) = E { y i ( 1 ) − y i ( 0 ) ∣ x i = x } = E ( y i ∣ x i = x , A i = 1 ) − E ( y i ∣ x i = x , A i = 0 ) (1) \tau(\boldsymbol{x}) = \mathbb{E}\{y_i(1)-y_i(0) \mid\boldsymbol{x}_i = \boldsymbol{x}\}
= \mathbb{E}(y_i \mid\boldsymbol{x}_i = \boldsymbol{x}, A_i = 1) - \mathbb{E}(y_i \mid\boldsymbol{x}_i = \boldsymbol{x}, A_i = 0) \tag*{(1)} τ ( x ) = E { y i ( 1 ) − y i ( 0 ) ∣ x i = x } = E ( y i ∣ x i = x , A i = 1 ) − E ( y i ∣ x i = x , A i = 0 ) (1)
is identified by the difference between the treated and control outcome regressions. Bayesian Causal Forests (BCF) model the observed outcome as
y i = μ ( x i , e ^ i ) + A i { α + g τ ( x i ) } + ε i , ε i ∼ N ( 0 , σ 2 ) , (2) y_i = \mu(\boldsymbol{x}_i,\widehat{e}_i) + A_i\{\alpha+ g_\tau(\boldsymbol{x}_i)\} + \varepsilon_i,\qquad\varepsilon_i \sim N(0,\sigma^2), \tag*{(2)} y i = μ ( x i , e i ) + A i { α + g τ ( x i )} + ε i , ε i ∼ N ( 0 , σ 2 ) , (2)
where e ^ i \widehat{e}_i e i estimates e ( x i ) e(\boldsymbol{x}_i) e ( x i ) , μ \mu μ is the prognostic function, and τ ( x ) = α + g τ ( x ) \tau(\boldsymbol{x}) = \alpha+ g_\tau(\boldsymbol{x}) τ ( x ) = α + g τ ( x ) is the treatment-effect function [34 ] . Including the estimated propensity score in μ \mu μ helps protect the treatment-effect estimate from regularization-induced confounding [33 ] . We retain the BCF prior for μ \mu μ and modify only the leaf prior in the treatment ensemble g τ g_\tau g τ .
To define leaf priors we make use of the representation of a tree by its leaves. Let treatment tree T j T_j T j partition the covariate space into L j L_j L j leaves L ( T j ) \mathcal{L}(T_j) L ( T j ) . Let z j ( x ) ∈ { 0 , 1 } L j \boldsymbol{z}_j(\boldsymbol{x}) \in\{0,1\}^{L_j} z j ( x ) ∈ { 0 , 1 } L j be the one-hot vector indicating the leaf that contains x \boldsymbol{x} x , and let h j = ( h j ℓ ) ℓ ∈ L ( T j ) \boldsymbol{h}_j = (h_{j\ell})_{\ell\in\mathcal{L}(T_j)} h j = ( h j ℓ ) ℓ ∈ L ( T j ) collect the leaf values. Tree j j j contributes
g ( x ; T j , h j ) = z j ( x ) T h j , (3) g(\boldsymbol{x};T_j,\boldsymbol{h}_j) = \boldsymbol{z}_j(\boldsymbol{x})^\mathsf{T}\boldsymbol{h}_j, \tag*{(3)} g ( x ; T j , h j ) = z j ( x ) T h j , (3)
and g τ ( x ) = ∑ j = 1 m τ g ( x ; T j , h j ) g_\tau(\boldsymbol{x}) = \sum_{j=1}^{m_\tau} g(\boldsymbol{x};T_j,\boldsymbol{h}_j) g τ ( x ) = ∑ j = 1 m τ g ( x ; T j , h j ) . We consider Chipman et al. (2010)’s prior on the topology of a tree, in which a node at depth d d d splits with probability ν ( 1 + d ) − η \nu(1+d)^{-\eta} ν ( 1 + d ) − η . We select splitting variables using a discrete uniform distributions. Given T j T_j T j , BART assigns independent N ( 0 , σ h 2 ) N(0,\sigma_h^2) N ( 0 , σ h 2 ) distributions to the elements of h j h_j h j [21 ] . This leaves the choice of scale prior to determine how variation is distributed across the leaves. We consider common and leaf-specific scale priors next.
Problem Statement
Under BART, given that x x x belongs to exactly one leaf, the prior variance of g ( x ; T j , h j ) g(x; T_j, h_j) g ( x ; T j , h j ) is σ h 2 \sigma_h^2 σ h 2 . While considering further splits leaves the variance unchanged, the total variance the leaves represent increases. For a treatment tree, we seek two properties. 1) Leaf variances should be able to differ across covariate space, so that a leaf is shrunk strongly toward zero where the data supports no heterogeneity. 2) Second, the sum of the leaf variances of a tree should have a prior that does not depend on L j L_j L j , so that a split redistributes prior variance within a tree (i.e the total variability of a tree) instead of adding to it.
Neither property holds under the BART leaf prior. Priors on the leafs with conditionally independent local scales, such as the Horseshoe [19 , 57 , 54 ] , meet the first but not the second requirement. Under any such prior, the sum of the leaf variances is increasing in L j L_j L j , implying that the second property cannot be satisfied. Scaling BART leaf variances by L j L_j L j could endow the prior with the second property, but every leaf would receive the same fixed variance, offering no adaptability. We meet both by letting the law of the leaf variances depend on the topology of the tree.
To this end, we propose a prior that separates the total variance of a tree from its allocation across leaves. Tree j j j receives a positive tree scale ψ j 2 \psi_j^2 ψ j 2 and an allocation vector ϕ j = ( ϕ j ℓ ) ℓ ∈ L ( T j ) \phi_j=(\phi_{j\ell})_{\ell\in\mathcal{L}(T_j)} ϕ j = ( ϕ j ℓ ) ℓ ∈ L ( T j ) satisfying ϕ j ℓ > 0 \phi_{j\ell}>0 ϕ j ℓ > 0 and ∑ ℓ ϕ j ℓ = 1 \sum_{\ell}\phi_{j\ell}=1 ∑ ℓ ϕ j ℓ = 1 . The prior on ψ j 2 \psi_j^2 ψ j 2 is independent of the tree, while the law of ϕ j \phi_j ϕ j is constructed using information from the topology. Given T j T_j T j , ψ j 2 \psi_j^2 ψ j 2 and ϕ j \phi_j ϕ j , the leaf values are independent with h j ℓ ∼ N ( 0 , V j ℓ ) h_{j\ell}\sim\mathrm{N}(0,V_{j\ell}) h j ℓ ∼ N ( 0 , V j ℓ ) , with V j ℓ = V j ϕ j ℓ V_{j\ell}=V_j\phi_{j\ell} V j ℓ = V j ϕ j ℓ and V j = σ 2 ω ψ j 2 V_j=\sigma^2\omega\psi_j^2 V j = σ 2 ω ψ j 2 , where ω > 0 \omega>0 ω > 0 is a shared constant of the forest. Since, ∑ ℓ ∈ L ( T j ) V j ℓ = V j \sum_{\ell\in\mathcal{L}(T_j)}V_{j\ell}=V_j ∑ ℓ ∈ L ( T j ) V j ℓ = V j for every tree j j j and every value of L j L_j L j , we immediately satisfy the second property we seek. We are therefore able to jointly control the total amount of signal that the tree can represent. The compositional nature of V j ℓ V_{j\ell} V j ℓ implies that a larger share for one leaf leaves less total variance V j V_j V j for the others. This is precisely the joint regularization of leaf values we aim to achieve. To achieve this we need to specify prior distributions for ψ j 2 \psi_j^2 ψ j 2 and ϕ j \phi_j ϕ j .
Dirichlet-Tree Allocation
We now specify the law of the shares ϕ j \phi_j ϕ j given the topology of the tree. The random topology of T j T_j T j requires a law for the shares on every possible tree. A law chosen separately for each topology would not be able to express how the shares of a tree relate to those of a tree obtained by splitting one of its leaves. Therefore, we define the shares ϕ j \phi_j ϕ j through the same binary recursion that constructs the partition. The main idea is that each internal node divides the share of total variance V j V_j V j it receives between its two children, while the rest of the tree remains unchanged.
Let I ( T j ) \mathcal{I}(T_j) I ( T j ) be the internal nodes of T j T_j T j , and ϕ j v \phi_{jv} ϕ j v the share reaching node v v v , with ϕ j v = 1 \phi_{jv}=1 ϕ j v = 1 at the root. Let the branch fraction U j v ∈ ( 0 , 1 ) U_{jv}\in(0,1) U j v ∈ ( 0 , 1 ) at node v ∈ I ( T j ) v\in\mathcal{I}(T_j) v ∈ I ( T j ) send
ϕ j v L = U j v ϕ j v , ϕ j v R = ( 1 − U j v ) ϕ j v (4) \phi_{jv_L}=U_{jv}\phi_{jv},\qquad\phi_{jv_R}=(1-U_{jv})\phi_{jv} \tag*{(4)} ϕ j v L = U j v ϕ j v , ϕ j v R = ( 1 − U j v ) ϕ j v (4)
to the left and right children v L v_L v L and v R v_R v R . Therefore, each leaf share is the product of the branch fractions along its root-to-leaf path, using U j v U_{jv} U j v on a left branch and 1 − U j v 1-U_{jv} 1 − U j v on a right branch.
If the branch fractions U j v U_{jv} U j v are independent with
U j v ∣ T j ∼ Beta { c L ( v ) , c R ( v ) } , v ∈ I ( T j ) , (5) U_{jv}\mid T_j\sim\operatorname{Beta}\{c_L(v),c_R(v)\},\qquad v\in\mathcal{I}(T_j), \tag*{(5)} U j v ∣ T j ∼ Beta { c L ( v ) , c R ( v )} , v ∈ I ( T j ) , (5)
the shares ϕ j \phi_j ϕ j follow a Dirichlet-Tree distribution [50 ] (Dennis 1991). The name refers to the recursive beta construction, and the terminal shares need not to have an ordinary Dirichlet distribution. For example, the symmetric choice c L ( v ) = c R ( v ) = a c_L(v) = c_R(v) = a c L ( v ) = c R ( v ) = a divides shares evenly on average. A leaf at depth d ℓ d_\ell d ℓ has mean share 2 − d ℓ 2^{-d_\ell} 2 − d ℓ for every value of a a a , so the shares cannot be exchangeable unless all leaves have the same depth. The share is a product of d ℓ d_\ell d ℓ independent Beta( a , a ) (a, a) ( a , a ) variables, which is not beta distributed for d ℓ ≥ 2 d_\ell\ge2 d ℓ ≥ 2 .
Instead, we aim to control how V j V_j V j is allocated across the leaves by specifying the mean shares directly. For this purpose, the terminal law should separate two aspects of the allocation. Relative leaf weights should determine the mean shares, while a total concentration should determine how strongly the realized shares can depart from those means. We obtain this separation by choosing the branch parameters so that the induced terminal law is an ordinary Dirichlet distribution. Hence, each share then has a beta marginal whose mean and concentration are explicit in the parameters. The Dirichlet aggregation property allows us to compare the allocation laws before and after a leaf splits [7 , 46 ] . Within the Dirichlet-Tree construction, the desired leaf law results from setting the beta parameters at each internal node as the sums of the leaf parameters below its two children [50 ] . We call the result weighted allocation.
Specifically, we derive the branch parameters from positive leaf weights ( w ℓ ) ℓ ∈ L ( T j ) (w_\ell)_{\ell\in\mathcal{L}(T_j)} ( w ℓ ) ℓ ∈ L ( T j ) and a concentration a > 0 a > 0 a > 0 . If W v L W_v^L W v L and W v R W_v^R W v R are the total weights of the leaves below v L v^L v L and v R v^R v R , we set
c L ( v ) = a W v L , c R ( v ) = a W v R . (6) c_L(v) = aW_v^L, \qquad c_R(v) = aW_v^R. \tag*{(6)} c L ( v ) = a W v L , c R ( v ) = a W v R . (6)
The shares are then ϕ j ∣ T j ∼ Dirichlet { ( a w ℓ ) ℓ ∈ L ( T j ) } \phi_j \mid T_j \sim\operatorname{Dirichlet}\{(aw_\ell)_{\ell\in\mathcal{L}(T_j)}\} ϕ j ∣ T j ∼ Dirichlet {( a w ℓ ) ℓ ∈ L ( T j ) } . Hence,
ϕ j ℓ ∣ T j ∼ Beta { a w ℓ , a ( W j − w ℓ ) } , E ( ϕ j ℓ ∣ T j ) = w ℓ W j , (7) \phi_{j\ell} \mid T_j \sim\operatorname{Beta}\{aw_\ell, a(W_j - w_\ell)\}, \qquad\mathbb{E}(\phi_{j\ell} \mid T_j) = \frac{w_\ell}{W_j}, \tag*{(7)} ϕ j ℓ ∣ T j ∼ Beta { a w ℓ , a ( W j − w ℓ )} , E ( ϕ j ℓ ∣ T j ) = W j w ℓ , (7)
where W j = ∑ ℓ w ℓ W_j = \sum_\ell w_\ell W j = ∑ ℓ w ℓ . The weights w ℓ w_\ell w ℓ set the mean allocation, while the total concentration a W j aW_j a W j controls dispersion around it.
Since splits can be considered local refinements of variance allocation, we impose the requirement that a split on a share between children will not change the law of the other shares. Let T ′ T' T ′ be obtained from T T T by splitting leaf ℓ \ell ℓ into ℓ L \ell_L ℓ L and ℓ R \ell_R ℓ R , and index the other leaves by 1 , … , L − 1 1, \ldots, L-1 1 , … , L − 1 under both trees. We call the allocation consistent under splits when adding the two shares of the two new leaves recovers the law of the shares before the split. Namely, ( ϕ 1 ′ , … , ϕ L − 1 ′ , ϕ ℓ L ′ + ϕ ℓ R ′ ) = d ( ϕ 1 , … , ϕ L − 1 , ϕ ℓ ) (\phi'_1, \ldots, \phi'_{L-1}, \phi'_{\ell_L} + \phi'_{\ell_R}) \mathrel{\overset{d}{=}} (\phi_1, \ldots, \phi_{L-1}, \phi_\ell) ( ϕ 1 ′ , … , ϕ L − 1 ′ , ϕ ℓ L ′ + ϕ ℓ R ′ ) = d ( ϕ 1 , … , ϕ L − 1 , ϕ ℓ ) . By the aggregation property of the Dirichlet distribution, consistency is a condition on the weights. Hence we set the depth weights w ℓ = ρ d ℓ w_\ell= \rho^{d_\ell} w ℓ = ρ d ℓ , where ρ > 0 \rho> 0 ρ > 0 and d ℓ d_\ell d ℓ is the depth of leaf ℓ \ell ℓ . The value of ρ \rho ρ determines how a leaf’s contribution changes when it is replaced by its children.
Proposition 1 (Consistency under splits). The weighted allocation is consistent under a split of leaf ℓ \ell ℓ if and only if w ℓ L + w ℓ R = w ℓ w_{\ell_L} + w_{\ell_R} = w_\ell w ℓ L + w ℓ R = w ℓ . For the depth weights w ℓ = ρ d ℓ w_\ell= \rho^{d_\ell} w ℓ = ρ d ℓ , consistency under every split holds if and only if ρ = 1 / 2 \rho= 1/2 ρ = 1/2 . Additionally, the symmetric choice is consistent under every split.
At ρ = 1 / 2 \rho= 1/2 ρ = 1/2 , each child inherits half of its parent’s weight, so W j = 1 W_j = 1 W j = 1 for every topology. Hence, the Dirichlet concentration is always a a a regardless of the topology, making it comparable across the different trees of the forest. In this case, a leaf at depth d ℓ d_\ell d ℓ has mean share 2 − d ℓ 2^{-d_\ell} 2 − d ℓ , and each child receives, on average, half of its parent’s share. At ρ = 1 \rho= 1 ρ = 1 , all weights equal one. Thus, the shares have a symmetric Dirichlet distribution with parameter a a a , and every leaf has mean share 1 / L j 1/L_j 1/ L j irrespective of its depth.
Tree Scale Prior
The tree scale ψ j 2 \psi_j^2 ψ j 2 controls all leaf values in tree j j j through their total variance V j = σ 2 ω ψ j 2 V_j = \sigma^2\omega\psi_j^2 V j = σ 2 ω ψ j 2 . When ψ j 2 \psi_j^2 ψ j 2 is small, every leaf value is concentrated near zero and the tree contributes little throughout the covariate space. As ψ j 2 \psi_j^2 ψ j 2 increases, the tree can make progressively larger regional contributions. Its prior should therefore concentrate near zero and decay slowly enough to keep large scales plausible. To achieve both behaviors we consider the scaled beta-prime family
ψ j 2 ∼ BetaPrime ( a ψ , b ψ , s ψ ) , (8) \psi_j^2 \sim\operatorname{BetaPrime}(a_\psi, b_\psi, s_\psi), \tag*{(8)} ψ j 2 ∼ BetaPrime ( a ψ , b ψ , s ψ ) , (8)
with density proportional to ( t / s ψ ) a ψ − 1 ( 1 + t / s ψ ) − ( a ψ + b ψ ) (t/s_\psi)^{a_\psi-1}(1+t/s_\psi)^{-(a_\psi+b_\psi)} ( t / s ψ ) a ψ − 1 ( 1 + t / s ψ ) − ( a ψ + b ψ ) for t > 0 t > 0 t > 0 . The tree scales are independent across trees and independent of the topologies and shares. Since ∑ ℓ ∈ L ( T j ) V j ℓ = V j = σ 2 ω ψ j 2 \sum_{\ell\in\mathcal{L}(T_j)} V_{j\ell}=V_j=\sigma^2\omega\psi_j^2 ∑ ℓ ∈ L ( T j ) V j ℓ = V j = σ 2 ω ψ j 2 , the induced prior on the total leaf variance is the same for every tree size and topology.
We set a ψ = b ψ = 1 / 2 a_\psi=b_\psi=1/2 a ψ = b ψ = 1/2 and s ψ = 1 s_\psi=1 s ψ = 1 , for which ψ j \psi_j ψ j is standard half-Cauchy. Conditioned on ϕ j \phi_j ϕ j , integrating over ψ j \psi_j ψ j implies that h j ℓ / ( σ ω ϕ j ℓ ) h_{j\ell}/(\sigma\sqrt{\omega\phi_{j\ell}}) h j ℓ / ( σ ω ϕ j ℓ ) has a Horseshoe marginal, with a logarithmic spike at zero and ∣ h ∣ − 2 |h|^{-2} ∣ h ∣ − 2 tails [55 , 56 ] . It remains to integrate over the Dirichlet-Tree prior on ϕ j \phi_j ϕ j . Proposition 2 shows that, under the weighted allocation, this integration preserves the spike and the horseshoe tail order, and can sharpen the spike from logarithmic to polynomial.
Proposition 2 (Marginal behavior of a leaf value). Let T j T_j T j have at least two leaves, and let c ℓ c_\ell c ℓ be the smallest branch parameter on the path to leaf ℓ \ell ℓ , taking c L ( v ) c_L(v) c L ( v ) at a left branch and c R ( v ) c_R(v) c R ( v ) at a right branch. Under the general Dirichlet-Tree distribution, conditionally on ψ j 2 \psi_j^2 ψ j 2 , the density of h j ℓ h_{j\ell} h j ℓ is bounded at zero if and only if c ℓ > 1 / 2 c_\ell> 1/2 c ℓ > 1/2 .
Under the weighted allocation and Equation (8) , let c ℓ ∗ = min ( a ψ , a w ℓ ) c_\ell^*=\min(a_\psi, a_{w\ell}) c ℓ ∗ = min ( a ψ , a w ℓ ) and let p p p be the density of h j ℓ ∣ T j h_{j\ell}\mid T_j h j ℓ ∣ T j . Then p p p is bounded at zero if and only if c ℓ ∗ > 1 / 2 c_\ell^*>1/2 c ℓ ∗ > 1/2 . If c ℓ ∗ < 1 / 2 c_\ell^*<1/2 c ℓ ∗ < 1/2 and a ψ ≠ a w ℓ a_\psi\ne aw_\ell a ψ = a w ℓ , then p ( h ) ≍ ∣ h ∣ 2 c ℓ ∗ − 1 p(h)\asymp|h|^{2c_\ell^*-1} p ( h ) ≍ ∣ h ∣ 2 c ℓ ∗ − 1 as h → 0 h\to0 h → 0 . In all cases, p ( h ) ≍ ∣ h ∣ − ( 2 b ψ + 1 ) p(h)\asymp|h|^{-(2b_\psi+1)} p ( h ) ≍ ∣ h ∣ − ( 2 b ψ + 1 ) as ∣ h ∣ → ∞ |h|\to\infty ∣ h ∣ → ∞ .
With a = ρ = a ψ = b ψ = 1 / 2 a=\rho=a_\psi=b_\psi=1/2 a = ρ = a ψ = b ψ = 1/2 , a leaf at depth d ≥ 1 d\ge1 d ≥ 1 has a w ℓ = 2 − d − 1 a_{w\ell}=2^{-d-1} a w ℓ = 2 − d − 1 , so its density is of order ∣ h ∣ 2 − d − 1 |h|^{2^{-d-1}} ∣ h ∣ 2 − d − 1 near zero and ∣ h ∣ − 2 |h|^{-2} ∣ h ∣ − 2 in the tails. This polynomial spike sharpens with depth and is stronger than the logarithmic spike of the horseshoe prior. The Dirichlet-Tree allocation therefore concentrates deeper leaf values more strongly near zero while preserving the horseshoe tail order available to large values. This is attractive when most regional contributions should be negligible and a few may be large.
Hyperparameter Specification
We specify the remaining components through interpretable prior quantities. We begin with ω \omega ω , which rescales every leaf variance and hence the prior of g τ g_\tau g τ . We express its effect through S τ = sd { g τ ( x i ) } S_\tau=\operatorname{sd}\{g_\tau(x_i)\} S τ = sd { g τ ( x i )} , the across-unit standard deviation of the treatment effect at the observed covariates. Following a prior-predictive approach to hyperparameter specification, [29 , 49 ] , we choose ω \omega ω through the induced prior for S τ S_\tau S τ . Since V j V_j V j is proportional to σ 2 \sigma^2 σ 2 , the prior of S τ / σ S_\tau/\sigma S τ / σ does not depend on σ \sigma σ . We set ω \omega ω so that the prior median of S τ / σ S_\tau/\sigma S τ / σ targets ζ = 1 / 2 \zeta=1/2 ζ = 1/2 .
Once ω \omega ω sets the overall spread, a a a controls how V j V_j V j is distributed among the leaves. We measure the concentration of this allocation by ∑ ℓ ϕ j ℓ 2 \sum_\ell\phi_{j\ell}^2 ∑ ℓ ϕ j ℓ 2 , which equals one when a single leaf receives all of V j V_j V j and 1 / L j 1/L_j 1/ L j under equal division. Since ϕ j \phi_j ϕ j is random, we define the prior effective allocation size as A e f f ( T j ) = { E ( ∑ ℓ ϕ j ℓ 2 ∣ T j ) } − 1 A_{\mathrm{eff}}(T_j)=\{\mathbb{E}(\sum_\ell\phi_{j\ell}^2\mid T_j)\}^{-1} A eff ( T j ) = { E ( ∑ ℓ ϕ j ℓ 2 ∣ T j ) } − 1 . If ρ = 1 / 2 \rho=1/2 ρ = 1/2 , then ( 1 + a ) / ( 1 + a / 2 ) ≤ A e f f ( T j ) < 1 + a (1+a)/(1+a/2)\le A_{\mathrm{eff}}(T_j)<1+a ( 1 + a ) / ( 1 + a /2 ) ≤ A eff ( T j ) < 1 + a (See Appendix § 11 .§ 4 ). Thus, 1 + a 1+a 1 + a provides an interpretable upper bound for choosing a a a . We set a = 1 / 2 a=1/2 a = 1/2 , giving A e f f ( T j ) ∈ [ 1.2 , 1.5 ] A_{\mathrm{eff}}(T_j)\in[1.2,1.5] A eff ( T j ) ∈ [ 1.2 , 1.5 ] , thus favoring allocations in which a few leaves receive most of V j V_j V j .
Theoretical Properties
By construction, the Dirichlet-Tree prior meets the two requirements of § 3.2 . We now show the implications for the shrinkage of the leaf values and state posterior consistency.
Within-tree Shrinkage Conditional on the rest of the model, let n j ℓ n_{j\ell} n j ℓ be the number of treated observations in leaf ℓ \ell ℓ and R ˉ j ℓ \bar{R}_{j\ell} R ˉ j ℓ their mean partial residual, the local estimate of tree j j j ’s contribution. Given ψ j 2 \psi_j^2 ψ j 2 and ϕ j \phi_j ϕ j , the conditional posterior mean of h j ℓ h_{j\ell} h j ℓ is ( 1 − κ j ℓ ) R ˉ j ℓ (1-\kappa_{j\ell})\bar{R}_{j\ell} ( 1 − κ j ℓ ) R ˉ j ℓ , where
κ j ℓ = ( 1 + n j ℓ ω ψ j 2 ϕ j ℓ ) − 1 (9) \kappa_{j\ell} = \left(1+n_{j\ell}\omega\psi_j^2\phi_{j\ell}\right)^{-1} \tag*{(9)} κ j ℓ = ( 1 + n j ℓ ω ψ j 2 ϕ j ℓ ) − 1 (9)
is its shrinkage factor [57 , 12 , 3 ] . For q ∈ ( 0 , 1 ) q \in(0,1) q ∈ ( 0 , 1 ) , define N j , q = # { ℓ : κ j ℓ ≤ 1 − q } N_{j,q} = \#\{\ell: \kappa_{j\ell} \le1-q\} N j , q = # { ℓ : κ j ℓ ≤ 1 − q } , the number of leaves that retain at least the fraction q q q of their local mean.
Proposition 3. Let every leaf of T j T_j T j contain between 1 and n n n treated observations. Under the Dirichlet-Tree prior,
N j , q ≤ ( 1 − q ) n ω ψ j 2 q (10) N_{j,q} \le\frac{(1-q)n\omega\psi_j^2}{q} \tag*{(10)} N j , q ≤ q ( 1 − q ) nω ψ j 2 (10) for every ψ j 2 > 0 \psi_j^2 > 0 ψ j 2 > 0 and ϕ j \phi_j ϕ j . If the shares in (9) are instead i.i.d. local scales, independent of ψ j 2 \psi_j^2 ψ j 2 , whose distribution has unbounded support independent of L j L_j L j , then E ( N j , q ∣ ψ j 2 ) ≥ p q L j \mathbb{E}(N_{j,q} \mid\psi_j^2) \ge p_q L_j E ( N j , q ∣ ψ j 2 ) ≥ p q L j , where p q > 0 p_q > 0 p q > 0 .
The proposition bounds the number of regions in which a tree can report a substantial estimated effect. With independent local scales, the expected count instead grows at least linearly in L j L_j L j (See Appendix § 11 .§ 4 ). Under the BART leaf prior, the shrinkage factor κ j ℓ \kappa_{j\ell} κ j ℓ depends only on n j ℓ n_{j\ell} n j ℓ , so the prior cannot reallocate variance between regions.
Treatment Forest At the forest level, different trees may assign large allocations of variance to different regions. We can extend the bound in Equation (10) to the forest of m τ m_\tau m τ trees, showing that under our prior the forest can consider at most ( 1 − q ) n ω ∑ j ψ j 2 / q (1-q)n\omega\sum_j\psi_j^2/q ( 1 − q ) nω ∑ j ψ j 2 / q regions of substantial effect.
We can additionally consider the size of the effects the forest can attain. Conditioned on the topologies T j T_j T j and σ 2 \sigma^2 σ 2 , for a fixed x x x we have
Pr { g τ ( x ) > t } ∼ ∑ j = 1 m τ Pr { g ( x ; T j , h j ) > t } (11) \Pr\{g_\tau(x)>t\} \sim\sum_{j=1}^{m_\tau}\Pr\{g(x;T_j,h_j)>t\} \tag*{(11)} Pr { g τ ( x ) > t } ∼ j = 1 ∑ m τ Pr { g ( x ; T j , h j ) > t } (11)
as t → ∞ t \to\infty t → ∞ , since the tree contributions g ( x ; T j , h j ) g(x;T_j,h_j) g ( x ; T j , h j ) are independent and the tail of each tree is t − 2 b ψ t^{-2b_\psi} t − 2 b ψ asymptotically (Feller [25 ] ). Therefore our prior allows for a large effect at x x x to arise from at least one large tree contribution, while allowing the others to remain small. This shows that the forest prior limits the number of regions with a substantial effect and still allows a large effect in any one of them.
Posterior Consistency Concentrating a tree’s variance in a few regions through the joint prior raises the concern that we may rule out treatment effects that vary throughout the covariate space. Consistency results for BART cannot settle this, since they assume independent priors on the leaf values (Rocková and Saha [62 ] ). Let P 0 P_0 P 0 denote the data-generating law of ( x , A , y ) (x,A,y) ( x , A , y ) and let P μ , τ , σ P_{\mu,\tau,\sigma} P μ , τ , σ denote the law induced by the fitted regression surfaces and residual scale.
Theorem 1 (Weak consistency). Under Conditions (i)–(iv) of Appendix § 11 .§ 5 , the prior of DT-BCF assigns positive probability to every Kullback–Leibler neighborhood of P 0 P_0 P 0 . For every weak neighborhood W \mathcal{W} W of P 0 P_0 P 0 ,
Π ( P μ , τ , σ ∉ W ∣ D n ) ⟶ 0 P 0 ∞ -almost surely . (12) \Pi(P_{\mu,\tau,\sigma} \notin\mathcal{W} \mid D_n) \longrightarrow0 \quad P_0^\infty\text{-almost surely}. \tag*{(12)} Π ( P μ , τ , σ ∈ / W ∣ D n ) ⟶ 0 P 0 ∞ -almost surely . (12) If P μ , τ , σ = P 0 P_{\mu,\tau,\sigma}=P_0 P μ , τ , σ = P 0 , then σ = σ 0 \sigma=\sigma_0 σ = σ 0 , μ { x , e 0 ( x ) } = μ 0 ( x ) \mu\{x,e_0(x)\}=\mu_0(x) μ { x , e 0 ( x )} = μ 0 ( x ) , and τ ( x ) = τ 0 ( x ) \tau(x)=\tau_0(x) τ ( x ) = τ 0 ( x ) for Q Q Q -almost every x x x .
In practical terms, Theorem 1 shows that even if DT-BCF induces a preference for few regions of heterogeneity, this does not exclude effects that vary throughout the covariate space. With enough data, the posterior concentrates around the true distribution of the data, P 0 P_0 P 0 , which determines the treatment effect. This holds when the true surfaces μ 0 \mu_0 μ 0 and τ 0 \tau_0 τ 0 are continuous, treated and control units overlap, and the propensity score is known.
At first sight, Proposition 3 seems to say otherwise, since it bounds the number of regions in which a tree can report a substantial estimated effect. That bound, however, grows with the tree scale ψ j 2 \psi_j^2 ψ j 2 and with the number n n n of treated observations in a leaf, so it loosens as the data support larger tree scales and as leaves gather more observations. Hence, the prior shrinks spurious heterogeneity in small samples without ruling out widespread heterogeneity in large ones.
Computation
Computation follows standard BART/BCF backfitting proposals (Hastie and Tibshirani, [35 ] ; Hahn et al., [34 ] ; Pratola, [58 ] ), with the full sampler described in Appendix § 10 . A treatment-tree grow move additionally draws a branch fraction U j v U_{jv} U j v from its beta prior, while the reverse prune move removes it. The Metropolis–Hastings ratio includes the corresponding Dirichlet-Tree density and topology-induced leaf variances.
We sample the continuous parameters of a treatment tree in one of two ways. The first integrates the leaf values out and updates each branch fraction and the tree scale by univariate slice sampling (Neal, [52 ] ), for any a a a and ρ \rho ρ . The second applies when s ψ = 1 s_\psi=1 s ψ = 1 and a ψ = a W j a_\psi=a_{W_j} a ψ = a W j , which holds at our default ρ = 1 / 2 \rho=1/2 ρ = 1/2 . Writing B j ℓ = ψ j 2 ϕ j ℓ B_{j\ell}=\psi_j^2\phi_{j\ell} B j ℓ = ψ j 2 ϕ j ℓ , we use the hierarchical gamma representation of Zhang et al., [77 ] , which makes their full conditionals given the leafs generalized inverse Gaussian. We proceed to draw and normalize them, which gives a blocked Gibbs update of the tree scale and the shares (Aguilar and Bürkner, [2 ] ). The intercept and residual variance retain their standard full conditional updates.
Experiments
We now present experiments designed to evaluate the empirical performance of DT-BCF across different patterns of treatment-effect heterogeneity and covariate dimensions, relative to alternative priors and external learners.
Models and Evaluation
We compare DT-BCF with three groups of models, namely variants of DT-BCF a (DT-BCF + DART , DT-BCF (ρ = 1 \rho=1 ρ = 1 ), and Equal ), priors with independent leaf variances (Horseshoe HS and Fixed ), and external references (BCF , the X-learner and the R-learner ). All Bayesian models are the BCF model (2) with the prognostic forest of § 3.1 .
DT-BCF uses the weighted allocation with a = ρ = 1 / 2 a=\rho=1/2 a = ρ = 1/2 and the tree scale ψ j 2 ∼ BetaPrime ( 1 / 2 , 1 / 2 , 1 ) \psi_j^2 \sim\operatorname{BetaPrime}(1/2,1/2,1) ψ j 2 ∼ BetaPrime ( 1/2 , 1/2 , 1 ) . DT-BCF + DART adds, in both forests, the DART prior on the probability that a split uses each covariate ([47 ] . DT-BCF (ρ = 1 \rho= 1 ρ = 1 ) and Equal keep the tree scale and change the allocation. DT-BCF (ρ = 1 \rho= 1 ρ = 1 ) the shares are symmetric Dirichlet with mean 1 / L j 1/L_j 1/ L j , and Equal , the limit as a → ∞ a \to\infty a → ∞ , gives every leaf the variance V j / L j V_j/L_j V j / L j . HS is the horseshoe prior, which considers shrinkage but no joint regularization of the scales, within each tree ([18 ] ). Fixed is the BART prior with constant variance σ h 2 = σ 2 ω \sigma_h^2 = \sigma^2\omega σ h 2 = σ 2 ω . In these six models, ω \omega ω is set so that the prior median of S τ / σ S_\tau/\sigma S τ / σ is ς = 1 / 2 \varsigma= 1/2 ς = 1/2 , which makes them comparable at the same prior spread of the effect. In the application, HS is instead the Horseshoe Forest of Jacobs et al. [38 ] , fitted with its authors’ package at its defaults.
Finally, BCF ([34 ] ), the baseline, is fitted at its published defaults, without matching ς \varsigma ς . We fit BCF with the authors’ package, and with our sampler in the designs only with p ≥ n p \ge n p ≥ n , where the package fails. The X-learner and the R-learner , both with gradient-boosted trees provide loss-based references ([20 , 45 , 53 ] ). Bayesian models run 4 Markov chains, with the first half of each discarded as warmup ([72 ] ).
We measure accuracy by the root mean squared error (RMSE), over the n n n units, of the estimate of the centered effect δ i = τ ( x i ) − n − 1 ∑ i ′ τ ( x i ′ ) \delta_i = \tau(\mathbf{x}_i) - n^{-1}\sum_{i'}\tau(\mathbf{x}_{i'}) δ i = τ ( x i ) − n − 1 ∑ i ′ τ ( x i ′ ) , which is the posterior mean for the Bayesian models. Centering removes the error in the average effect, so the comparison focuses on the variation
of the effect across units, which the leaf prior regularizes. For the Bayesian models we also report the coverage (Cov.) and length of the 90% intervals for δ i \delta_i δ i . In designs where some units have no effect, the false positive rate (FPR) and the true positive rate (TPR) are the proportions of units with τ ( x i ) = 0 \tau(x_i) = 0 τ ( x i ) = 0 and with τ ( x i ) ≠ 0 \tau(x_i) \ne0 τ ( x i ) = 0 whose 90% interval for τ ( x i ) \tau(x_i) τ ( x i ) excludes zero. The log predictive density of held-out data (lpd) and complete tables are in Appendix § 12 .
Leaf Priors
To isolate how the leaf priors allocate variance once a partition has been found, we fix a balanced tree with L L L leaves. Each leaf holds m = 16 m = 16 m = 16 observations, summarized by their mean R ˉ ℓ ∣ h ℓ ∼ N ( h ℓ , σ 2 / m ) \bar{R}_{\ell} \mid h_{\ell} \sim N(h_{\ell}, \sigma^2/m) R ˉ ℓ ∣ h ℓ ∼ N ( h ℓ , σ 2 / m ) with σ = 1 \sigma= 1 σ = 1 . Of the L L L leaves, k k k have effects ± Δ \pm\Delta ± Δ , with signal-to-noise ratio (SNR) Δ m 1 / 2 / σ = 3 \Delta m^{1/2}/\sigma= 3 Δ m 1/2 / σ = 3 , and the others have none (null leaves). We take k = 2 k = 2 k = 2 with L ∈ { 8 , 16 , 32 , 64 } L \in\{8, 16, 32, 64\} L ∈ { 8 , 16 , 32 , 64 } and k ∈ { 8 , 16 } k \in\{8, 16\} k ∈ { 8 , 16 } with L = 32 L = 32 L = 32 , with 200 data sets per condition and chains of 8,000 iterations. We match ω \omega ω at every L L L for every model, including the leaf prior of BCF, so that all share the prior median spread of the leaf values.
On null leaves, DT-BCF has the smallest RMSE in every condition (Figure 1 ). Equal, BCF and Fixed give every leaf a common variance. Equal and BCF differ only in its prior and nearly coincide. Keeping the null leaves near zero then requires a small common variance, also shrinking the non-null leaves. Fixed has the smallest RMSE on leaves with an effect and the largest on the others. This is expected since its matched variance is proportional to σ 2 / m \sigma^2/m σ 2 / m , thus shrinking every leaf mean only slightly. DT-BCF instead can give most of V j V_j V j to the two leaves with large ∣ R ˉ ℓ ∣ |\bar{R}_{\ell}| ∣ R ˉ ℓ ∣ . It is less accurate than BCF on leaves with an effect at L ≤ 16 L \le16 L ≤ 16 and more accurate than BCF and Equal at L ≥ 32 L \ge32 L ≥ 32 , where their common variance is informed mainly by the leaves without an effect. Within the Dirichlet-Tree family, the RMSE on null leaves is smallest for DT-BCF and largest for Equal, whose shares are the least concentrated. HS also shrinks null leaves less than DT-BCF. With k = 8 k = 8 k = 8 or 16, DT-BCF keeps the smallest RMSE on null leaves but has the largest on leaves with an effect. With a = 1 / 2 a = 1/2 a = 1/2 , the prior effective allocation size satisfies 1.2 ≤ A e f f ( T j ) < 1.5 1.2 \le A_{\mathrm{eff}}(T_j) < 1.5 1.2 ≤ A eff ( T j ) < 1.5 , so the prior concentrates V V V on few leaves. This behavior suits effects confined to a few regions of the covariate space.
Figure 1. Fixed Tree. log-RMSE of the posterior mean segmented over leaves without (top) and with (bottom) an effect, mean and two standard errors over 200 data sets. Left, k = 2 k = 2 k = 2 , against the number of leaves L L L . Right, L = 32 L = 32 L = 32 , against the number k k k of leaves with an effect. Dotted, the RMSE σ / m 1 / 2 \sigma/m^{1/2} σ / m 1/2 of the leaf means.
Forest Simulations
We next consider causal forest simulations, in which the tree search must also find the partition. Each data set has n = 200 n = 200 n = 200 units with covariates x i ∼ U ( 0 , 1 ) p x_i \sim U(0,1)^p x i ∼ U ( 0 , 1 ) p and a propensity score that depends only on x 1 x_1 x 1 to x 5 x_5 x 5 . We consider four scenarios that differ in how the treatment effect is distributed over the covariate space. In 2 Regions , the effect equals 2 on two rectangles defined by x 1 x_{1} x 1 to x 5 x_{5} x 5 and zero elsewhere. In 4 Regions Graded , it equals 4 ⋅ 0.6 k − 1 4 \cdot0.6^{k-1} 4 ⋅ 0. 6 k − 1 on rectangle k = 1 , … , 4 k = 1, \ldots, 4 k = 1 , … , 4 and zero elsewhere, so that the regions carry effects of different sizes. In Smooth Nonlinear , every unit has an effect, given by the Friedman function τ 0 ( x ) = 10 sin ( π x 1 x 2 ) + 20 ( x 3 − 1 / 2 ) 2 + 10 x 4 + 5 x 5 \tau_{0}(x) = 10 \sin(\pi x_{1}x_{2}) + 20(x_{3} - 1/2)^{2} + 10x_{4} + 5x_{5} τ 0 ( x ) = 10 sin ( π x 1 x 2 ) + 20 ( x 3 − 1/2 ) 2 + 10 x 4 + 5 x 5 . Smooth Graded multiplies τ 0 \tau_{0} τ 0 by 1.65 − q ( x ) 1.65^{-q(x)} 1.6 5 − q ( x ) , where q ( x ) ∈ { 0 , 1 , 2 , 3 } q(x) \in\{0, 1, 2, 3\} q ( x ) ∈ { 0 , 1 , 2 , 3 } indexes the quadrant of ( x 6 , x 7 ) (x_{6}, x_{7}) ( x 6 , x 7 ) , so that the amplitude of the effect varies across regions. Smooth scenarios have σ = 3 \sigma= \sqrt{3} σ = 3 .
We perform two studies: In the tree-size study we fix p = 50 p = 50 p = 50 and vary η ∈ { 0.25 , 0.5 , 1 , 2 } \eta\in\{0.25, 0.5, 1, 2\} η ∈ { 0.25 , 0.5 , 1 , 2 } in the split probability 0.95 ( 1 + d ) − η 0.95(1 + d)^{-\eta} 0.95 ( 1 + d ) − η of the treatment trees, where smaller η \eta η favors larger trees. In the dimension study we fix η = 2 \eta= 2 η = 2 and vary p ∈ { 20 , 50 , 200 , 500 , 1000 } p \in\{20, 50, 200, 500, 1000\} p ∈ { 20 , 50 , 200 , 500 , 1000 } , of which at most seven covariates modify the effect. Each case has 40 data sets, and every chain runs 4,000 iterations.
Tree-size Prior. On the region surfaces, DT-BCF has lower RMSE than BCF and HS at every η \eta η , with a smaller margin over BCF on 4 Regions Graded; DT-BCF + DART has the lowest RMSE in Table 1 . The gain comes from null regions, where leaves receive small shares of a fixed total variance V j V_{j} V j . On 2 Regions, DT-BCF reports an effect for about half as many null units as BCF; on both surfaces its FPR is lower than that of HS, at the cost of lower TPR. Its intervals also cover more often than those of HS and, on 2 Regions, of BCF. As η \eta η decreases, the RMSE and FPR of DT-BCF remain nearly constant. Fixed and HS deteriorate on both region surfaces. This pattern is consistent with the fixed total variance V j V_{j} V j . A new leaf receives part of V j V_{j} V j under DT-BCF, while the independent priors in HS and Fixed introduce another leaf-specific source of variance. On the smooth surfaces, where every unit has an effect, BCF is more accurate than DT-BCF. The X-learner can be more accurate than DT-BCF on the region surfaces but less accurate than DT-BCF + DART.
Table 1. Tree-Size Study. p = 50 p = 50 p = 50 over 40 data sets (max s.e ≤ 0.06 \le0.06 ≤ 0.06 ). Bold, the best RMSE, FPR and coverage. BCF and the X-learner do not depend on η \eta η .
Dimension The Bayesian models with uniform splitting probabilities lose accuracy as p p p grows because most proposed splits use covariates that do not modify the effect (Figure 2 ). Up to p = 50 p = 50 p = 50 , DT-BCF is the most accurate of these models on the region surfaces, or is as accurate as BCF. At larger p p p , it remains close to BCF on 2 Regions and ahead of HS on 4 Regions Graded, although BCF has lower RMSE there when p ≥ 500 p \ge500 p ≥ 500 . The leaf prior regularizes a partition after it has been formed, so it cannot recover an effect region that the tree search misses. DART addresses this search problem by concentrating splits on effect modifiers. With DART, DT-BCF has lower RMSE than BCF on both region surfaces at every p p p . Holding DART fixed also gives DT-BCF + DART lower RMSE than HS + DART at every p p p , while Equal + DART performs similarly. These comparisons provide more consistent evidence for a tree-level total variance than for an additional benefit from unequal allocation in these designs.
Figure 2. Dimension study (n = 200 n = 200 n = 200 , η = 2 \eta= 2 η = 2 ), RMSE of the centered effect against the number of covariates p p p , mean and two standard errors over 40 data sets, on a log scale.
ACIC 2016 Benchmark
We next evaluate the models on the data analysis competition of the 2016 Atlantic Causal Inference Conference (ACIC 2016), a widely used benchmark for heterogeneous effects [23 ] . BART-based methods were among the most accurate, so BCF is a strong reference. The benchmark simulates treatment and outcome from the 58 correlated, mixed-type covariates of 4,802 units of a perinatal study.
The competition defines 77 settings, which differ in how treatment and outcome are generated. Before fitting any model, we drew 20 of them at random so that they cover the competition’s range of heterogeneity (Appendix § 12 .§ 4 ). One has a constant effect and is reported there. In the other 19, the effect varies across nearly all units, as in the smooth scenarios.
Models are fitted to the first 10 of the 100 data sets (replicates) of every setting, with chains of 20,000 iterations. Since the RMSE ranges over more than an order of magnitude across data sets, we report it relative to BCF on the same dataset in Table 2 . DT-BCF and HS have a geometric-mean RMSE close to that of BCF. DT-BCF + DART has the lowest, with an interval below one and a ratio below one in 18 of the 19 settings (Table 2 ), consistent with DART concentrating the splits on the covariates that modify the effect. The intervals of DT-BCF cover more often than those of BCF and HS, at similar mean length.
Table 2. ACIC 2016 Benchmark. 19 settings, 10 replicates each. Geometric-mean RMSE ratio to BCF with a two-s.e. interval, number of settings with a ratio below one, coverage and length of 90% intervals.
Application to Pancreatic Cancer Survival
We also study how our models perform in real life data in the context of censored survival outcomes. We consider the pancreatic ductal adenocarcinoma cohort of The Cancer Genome Atlas [15 , 38 ] dataset. Of the 130 patients, 36 received adjuvant radiotherapy after resection. We model overall survival on the log scale with an accelerated failure time model that imputes censored survival times and uses 3,027 clinical and gene-expression covariates. Each model fit 4 chains of 40,000 iterations. See Appendix § 12 .§ 5 for more details.
DT-BCF and BCF are in accordance (Table 3 ). Every model estimates a positive ATE, with posterior means between 0.95 and 1.04 on the log survival scale and 95 % intervals above zero. The estimated leave-one-out predictive densities of DT-BCF and BCF are similar, and that of HS is lower. To judge whether the cohort is informative about heterogeneity, we compare the prior and posterior of the standard deviation of the effect across patients. Under the matched priors, its posterior median stays at or below the prior median, with wide intervals. Hence, the cohort carries little information about how much the effect varies across patients. This is also reflected in the 95% CI of the departure τ ( x i ) − τ ˉ \tau(x_i) - \bar{\tau} τ ( x i ) − τ ˉ from the average effect which contains zero for every model and patient, meaning that the data does not identify patients who benefit more or less than average. Under every model, the estimated effects of the 130 patients lie close together with their wide 95%
Table 3. Pancreatic Cohort. Effect of adjuvant radiotherapy on log survival time. ATE, posterior mean with 95% CI. SD across patients, prior median and posterior median with 95% interval. PSIS-LOO elpd (s.e.).
Figure 3. Pancreatic cohort. Effect τ ( x i ) \tau(x_i) τ ( x i ) of adjuvant radiotherapy on log survival time for each patient under the models of Table 3 . The horizontal lines are the 95% intervals. The black line connects the posterior means, with the patients ordered by posterior mean within each model. The vertical line marks zero.
intervals all overlapping (Figure 9 ). From a practical perspective, the estimates give no basis for choosing which patients receive radiotherapy based on their clinical or genomic profile.
Figure 9. Pancreatic Cohort. Effect τ ( x i ) \tau(x_i) τ ( x i ) of adjuvant radiotherapy on log survival time for each patient under the models of Table 3 . The horizontal lines are the 95% intervals and the black line connects the posterior means, with the patients ordered by posterior mean within each model. The vertical line marks zero.
We show in Figure 10 that every model predicts higher survival with radiotherapy than without it throughout follow-up. At two years, the predicted survival is about 0.7 with radiotherapy and about 0.4 without it. DT–BCF, DT–BCF + DART, BCF and Fixed give nearly identical curves, while HS gives a somewhat smaller difference between the two curves. For this dataset, the estimated benefit of radiotherapy therefore depends little on the choice of prior.
Figure 10. Pancreatic cohort. Predicted survival of the 130 patients under each model of Table 3 if every patient received adjuvant radiotherapy (solid) or none did (dashed). The bands show pointwise 95% credible intervals and the steps show Kaplan–Meier curves of the 36 treated and the 94 untreated patients.
Figure 4. Pancreatic cohort. Predicted survival of the 130 patients under each model of Table 3 if every patient received adjuvant radiotherapy (solid) or none did (dashed). The bands show pointwise 95% credible intervals and the steps show Kaplan–Meier curves of the 36 treated and the 94 untreated patients.
Discussion
We have introduced DT-BCF, which gives the leaves of each treatment tree a joint variance prior informed by the tree topology. Our construction separates the overall contribution of a tree from its distribution across regions and combines strong shrinkage of negligible regional effects with support for substantial ones. Since standard leaf priors set the variance of each region independently, the number of regions in which a tree reports a substantial effect is allowed to grow with the size of the tree. We have shown that our prior controls this by sharing a total variance among the regions of each tree, which compete for this variance. Regions whose data support an effect claim a large share of it, leaving the others strongly shrunk toward zero. Joint shrinkage priors in regression apply the same principle, distributing a total variance or global scale over model components fixed in advance [28 , 77 , 2 ] . In DT-BCF, the components are the regions of a random tree, so the prior allocates variance over a partition that is itself learned from the data.
Across our empirical studies, DT-BCF performs competitively over diverse effect surfaces, providing its clearest gains when heterogeneity is concentrated in a few regions. In high dimensions, performance also depends on discovering which covariates are important, since the leaf prior regularizes a partition only after the search has formed it. Adding DART produces consistent gains in high dimensions and in the benchmarks, indicating that joint leaf allocation and adaptive splitting address complementary regularization problems.
The Dirichlet-Tree construction restricts the dependence structures available among the leaf variances. More flexible compositional laws, including logistic-normal allocations [6 , 4 ] , could represent richer relationships among regions, although they would require new posterior computation. Our prior also acts within each tree, while the trees of the treatment forest remain independent a priori. Allocating a forest-level total variance across the trees, and then across the leaves of each tree, would extend the joint regularization to the whole ensemble. This approach would reflect the spirit of grouped shrinkage priors that regularize both groups and their members [14 , 1 , 67 , 5 ] . The survival application shows that the method extends to censored outcomes. Adapting it to other BART variants, such as targeted smooth BCF and shrinkage BCF [66 , 17 ] , and to other outcome models is another direction for future work.
To adjust for confounding, we build on the work of Hahn et al. (2020). The estimated propensity score enters the prognostic function, which guards against regularization-induced confounding [33 ] , while identification rests on the observed covariates being sufficient to adjust for treatment assignment. Because DT-BCF changes only the leaf prior of the treatment trees, this adjustment remains unchanged. Our prior could also be paired with other adjustments such as Robinson’s orthogonalization. This approach underlies the R-learner [45 ] , separating the estimation of the treatment effect from that of nuisance functions [61 , 53 ] . Within such a decomposition, the Dirichlet-Tree prior could be used to regularize the effect function. When unmeasured confounding is a concern, instrumental variables can identify local average treatment effects under different assumptions [10 ] . We can apply our joint leaf prior to the tree-based models of that effect.
Code availability
An implementation of DT-BCF and the code to reproduce the experiments are available at https://github.com/causality-lab/bcf-dirichlet-trees .
Bayesian Causal Forest recap
This section writes down the BART and BCF specification that the rest of the paper starts from. The tree prior, the prognostic forest, and the backfitting update are the usual ones. What changes, in § 3.3 , is the prior on the leaf values of the treatment trees.
BART
Bayesian additive regression trees (BART) model a continuous outcome by a sum of regression trees (Chipman et al. [21 ] ). For observation i i i ,
y i = f ( x i ) + ϵ i , f ( x i ) = ∑ j = 1 m g ( x i ; T j , h j ) , ϵ i ∼ N ( 0 , σ 2 ) . (13) y_i = f(x_i) + \epsilon_i,\qquad f(x_i) = \sum_{j=1}^{m} g(x_i;T_j,\mathbf{h}_j),\qquad\epsilon_i \sim\mathcal{N}(0,\sigma^2). \tag*{(13)} y i = f ( x i ) + ϵ i , f ( x i ) = j = 1 ∑ m g ( x i ; T j , h j ) , ϵ i ∼ N ( 0 , σ 2 ) . (13)
The tree T j T_j T j is a binary partition of the covariate space, and h j = ( h j ℓ ) ℓ ∈ L ( T j ) \mathbf{h}_j = (h_{j\ell})_{\ell\in\mathcal{L}(T_j)} h j = ( h j ℓ ) ℓ ∈ L ( T j ) holds the value in each leaf. Let z j ( x ) z_j(x) z j ( x ) indicate the leaf that contains x x x . The contribution of the tree is then the piecewise constant function g ( x ; T j , h j ) = z j ( x ) T h j g(x;T_j,\mathbf{h}_j) = z_j(x)^\mathsf{T}\mathbf{h}_j g ( x ; T j , h j ) = z j ( x ) T h j . Regularizing the depth and the leaf values prevents overfitting. The sum allows the forest to model nonlinearities.
We put a priors on how a tree is grown and on the leaf values. Following Chipman et al. [21 ] , a node at depth d d d , counting the root as depth zero, splits with probability
p s p l i t ( d ) = ν ( 1 + d ) − η , 0 < ν < 1 , η ≥ 0. (14) p_{\mathrm{split}}(d) = \nu(1+d)^{-\eta},\qquad0 < \nu< 1,\qquad\eta\ge0. \tag*{(14)} p split ( d ) = ν ( 1 + d ) − η , 0 < ν < 1 , η ≥ 0. (14)
The parameters are set as ν = 0.95 \nu= 0.95 ν = 0.95 and η = 2 \eta= 2 η = 2 as defaults. This equals a splitting probability of 0.95 at the root and 0.95/4 at the first split, so the prior prefers shallow trees and places less mass on deep partitions. When a split is drawn, one variable is drawn uniformly from the covariates, and the cut-point is drawn uniformly from the cut-points that the observations reaching that node still permit.
Conditional on the tree structure, the leaf values have independent conjugate normal priors,
h j ℓ ∣ T j , σ h 2 ∼ N ( 0 , σ h 2 ) , ℓ ∈ L ( T j ) . (15) h_{j\ell} \mid T_j,\sigma_h^2 \sim\mathcal{N}(0,\sigma_h^2),\qquad\ell\in\mathcal{L}(T_j). \tag*{(15)} h j ℓ ∣ T j , σ h 2 ∼ N ( 0 , σ h 2 ) , ℓ ∈ L ( T j ) . (15)
The zero mean pulls each trees contribution to zero. The scale is set from the number of trees. At a fixed x x x , each tree contributes exactly one leaf, so the prior variance of f ( x ) f(x) f ( x ) given the partitions equals m σ h 2 m\sigma_h^2 m σ h 2 . Taking σ h = σ 0 / m \sigma_h = \sigma_0/\sqrt{m} σ h = σ 0 / m makes that variance σ 0 2 \sigma_0^2 σ 0 2 however many trees are used. Chipman et al. [21 ] first shift and rescale the observed y i y_i y i into [ − 0.5 , 0.5 ] [-0.5, 0.5] [ − 0.5 , 0.5 ] , and they set σ 0 = 0.5 / k \sigma_0 = 0.5/k σ 0 = 0.5/ k with k = 2 k = 2 k = 2 . Then f ( x ) f(x) f ( x ) lands outside the range of the rescaled outcome with small prior probability.
Each leaf is given its own variance σ h 2 \sigma_h^2 σ h 2 , and the sum of the leaf-parameter variances in the tree rises by σ h 2 \sigma_h^2 σ h 2 . The prior on the tree depths means that the number of leafs, and the variance remains limited. Thus, the normal prior regularizes the leaf given the tree structure.
The noise variance has a conjugate inverse-gamma prior, σ 2 ∼ InvGamma ( a σ , b σ ) \sigma^2 \sim\operatorname{InvGamma}(a_\sigma,b_\sigma) σ 2 ∼ InvGamma ( a σ , b σ ) , with density proportional to ( σ 2 ) − a σ − 1 exp ( − b σ / σ 2 ) (\sigma^2)^{-a_\sigma-1}\exp(-b_\sigma/\sigma^2) ( σ 2 ) − a σ − 1 exp ( − b σ / σ 2 ) . The scaled inverse-χ 2 \chi^2 χ 2 prior of Chipman et al. [21 ] is the special case a σ = ν / 2 a_\sigma= \nu/2 a σ = ν /2 and b σ = ν λ / 2 b_\sigma= \nu\lambda/2 b σ = ν λ /2 . They take ν = 3 \nu= 3 ν = 3 and choose λ \lambda λ so that the 0.9 quantile of the prior on σ \sigma σ equals the residual standard deviation from a linear regression of the rescaled outcome.
Sampling in BART
We draw the posterior by Bayesian backfitting (Hastie and Tibshirani [35 ] ; Chipman et al. [21 ] ). One tree is updated at a time, on the part of the outcome the other trees do not already explain, and σ 2 \sigma^2 σ 2 is drawn after the trees. In one iteration the steps are the following.
Partial residual. To update tree j j j , remove the other trees from the outcome,
R i j = y i − ∑ k ≠ j g ( x i ; T k , h k ) . (16) R_{ij} = y_i - \sum_{k \ne j} g(x_i; T_k, h_k). \tag*{(16)} R ij = y i − k = j ∑ g ( x i ; T k , h k ) . (16)
A tree that has been updated in at this iteration is subtracted from the outcome.
Tree structure. Conditional on the other trees, and residual R i j R_{ij} R ij updating tree j j j simplifies to updating one regression tree. As (15) is conjugate normal, the leaf values integrate out of the likelihood of a proposed partition, and the Metropolis–Hastings step only has to compare partitions. The candidate is built with one of four moves (Chipman et al., [21 ] ; Pratola, [58 ] ). The grow move picks a leaf and splits it. The variable and the cut-point are drawn uniformly from those that the observations in the leaf still allow. The prune move picks an internal node whose two children are both leaves and collapses them back into that node. The change replaces the splitting rule at an internal node. The swap move exchanges the rule at a node with the rule at one of its children. To ensure sparsity, the splits that would put fewer than five observations in a child are discarded. The move is accepted or rejected with the Metropolis–Hastings probability given by the marginal likelihood, the tree prior (14) , and the probability of proposing the move in each direction.
Leaf values. Once a move has been selected, the leaves are drawn independently conditional on the tree structure. Let N j ℓ N_{j\ell} N j ℓ be the observations that fall in leaf ℓ \ell ℓ , let n j ℓ = ∣ N j ℓ ∣ n_{j\ell} = |N_{j\ell}| n j ℓ = ∣ N j ℓ ∣ , and let S j ℓ S_{j\ell} S j ℓ be the sum of R i j R_{ij} R ij over those observations. Then
v j ℓ = ( n j ℓ σ 2 + 1 σ h 2 ) − 1 , h j ℓ ∣ − ∼ N ( v j ℓ S j ℓ σ 2 , v j ℓ ) . (17) v_{j\ell} = \left(\frac{n_{j\ell}}{\sigma^2} + \frac{1}{\sigma_h^2}\right)^{-1}, \qquad h_{j\ell} \mid- \sim\mathrm{N}\left(\frac{v_{j\ell}S_{j\ell}}{\sigma^2}, v_{j\ell}\right). \tag*{(17)} v j ℓ = ( σ 2 n j ℓ + σ h 2 1 ) − 1 , h j ℓ ∣ − ∼ N ( σ 2 v j ℓ S j ℓ , v j ℓ ) . (17)
The draw is pulled toward the residual mean in the leaf and toward zero. The two precisions are n j ℓ / σ 2 n_{j\ell}/\sigma^2 n j ℓ / σ 2 and 1 / σ h 2 1/\sigma_h^2 1/ σ h 2 . A leaf with either few observations, or a large σ 2 \sigma^2 σ 2 , is the one that stays near zero.
Residual variance. Once every tree has been updated, draw
σ 2 ∣ − ∼ InvGamma ( a σ + n 2 , b σ + 1 2 ∑ i = 1 n { y i − f ( x i ) } 2 ) , (18) \sigma^2 \mid- \sim\operatorname{InvGamma}\left(a_\sigma+ \frac{n}{2}, b_\sigma+ \frac{1}{2}\sum_{i=1}^{n}\{y_i - f(x_i)\}^2\right), \tag*{(18)} σ 2 ∣ − ∼ InvGamma ( a σ + 2 n , b σ + 2 1 i = 1 ∑ n { y i − f ( x i ) } 2 ) , (18)
using the same parameterization as above.
Draws of f ( x ) f(x) f ( x ) from this sampler are posterior draws of the conditional mean, and the credible intervals are quantiles of those draws.
Bayesian Causal Forests (BCF)
Bayesian causal forests do not run a single BART of y y y on ( x , A ) (x, A) ( x , A ) (Hahn et al., [34 ] ). Prognosis and treatment effect are given two forests, with different priors. The outcome model (2) is
y i = μ ( x i , e ^ i ) + A i τ ( x i ) + ϵ i , ϵ i ∼ N ( 0 , σ 2 ) , (19) y_i = \mu(x_i, \widehat{e}_i) + A_i \tau(x_i) + \epsilon_i, \qquad\epsilon_i \sim\mathrm{N}(0, \sigma^2), \tag*{(19)} y i = μ ( x i , e i ) + A i τ ( x i ) + ϵ i , ϵ i ∼ N ( 0 , σ 2 ) , (19)
where A i ∈ { 0 , 1 } A_i \in\{0,1\} A i ∈ { 0 , 1 } and e ^ i \hat{e}_i e ^ i estimates the propensity score e ( x ) = Pr ( A i = 1 ∣ x i = x ) e(\mathbf{x}) = \Pr(A_i = 1 \mid\mathbf{x}_i = \mathbf{x}) e ( x ) = Pr ( A i = 1 ∣ x i = x ) . Under the assumptions in § 3.1 , τ ( x ) \tau(\mathbf{x}) τ ( x ) equals the CATE. As in the main text we write
μ ( x , e ^ ) = ∑ j = 1 m μ g ( x , e ^ ; T j μ , h j μ ) , (20) \mu(\mathbf{x},\hat{e}) = \sum_{j=1}^{m_\mu} g(\mathbf{x},\hat{e};T_j^\mu,h_j^\mu), \tag*{(20)} μ ( x , e ^ ) = j = 1 ∑ m μ g ( x , e ^ ; T j μ , h j μ ) , (20)
τ ( x ) = α + ∑ j = 1 m τ g ( x ; T j τ , h j τ ) = α + g τ ( x ) . (21) \tau(\mathbf{x}) = \alpha+ \sum_{j=1}^{m_\tau} g(\mathbf{x};T_j^\tau,h_j^\tau) = \alpha+ g_\tau(\mathbf{x}). \tag*{(21)} τ ( x ) = α + j = 1 ∑ m τ g ( x ; T j τ , h j τ ) = α + g τ ( x ) . (21)
The two functions do not receive the same regularization. The prognostic function has to follow the baseline outcome, including non-linearities and interactions, so it gets a flexible BART prior. Treatment signal is often less pronounced than the prognostic signal. BCF therefore regularize τ \tau τ more strongly than μ \mu μ . If the treatment trees never split, g τ ( x ) g_\tau(\mathbf{x}) g τ ( x ) is constant in x \mathbf{x} x and the CATE does not vary. In our parameterization that constant can also sit in α \alpha α , with g τ g_\tau g τ shrunk toward zero.
For μ \mu μ , Hahn et al., [34 ] use m μ = 200 m_\mu= 200 m μ = 200 trees and splitting probability prior 0.95 ( 1 + d ) − 2 0.95(1+d)^{-2} 0.95 ( 1 + d ) − 2 . The leaf standard deviation is σ h = σ 0 / m μ \sigma_h = \sigma_0/\sqrt{m_\mu} σ h = σ 0 / m μ , so σ 0 \sigma_0 σ 0 is the prior standard deviation of μ ( x ) \mu(\mathbf{x}) μ ( x ) at a given x \mathbf{x} x . They put a half-Cauchy prior on σ 0 \sigma_0 σ 0 with median twice the marginal standard deviation of y y y . For τ \tau τ they use m τ = 50 m_\tau= 50 m τ = 50 trees and splitting probability 0.25 ( 1 + d ) − 3 0.25(1+d)^{-3} 0.25 ( 1 + d ) − 3 , hence 0.25 at the root, and a half-normal prior on the analogous standard deviation of τ ( x ) \tau(\mathbf{x}) τ ( x ) , with median equal to the marginal standard deviation of y y y . Conditional on the scale, the leaf values remain independent N ( 0 , σ h 2 ) \mathrm{N}(0,\sigma_h^2) N ( 0 , σ h 2 ) draws. A new leaf gets its own σ h 2 \sigma_h^2 σ h 2 . It does not receive a share of a total that is held fixed when the tree grows. That is the leaf prior replaced in § 3.3 .
Coding A A A as 0/1 or as ± 1 / 2 \pm1/2 ± 1/2 changes how the prior variance is divided between the two conditional outcome means, and the BCF software estimates this coding [34 ] . Equation (2) keeps the 0/1 coding fixed.
The propensity score is in μ \mu μ to mitigate regularization-induced confounding [33 , 34 ] . When the prior on μ \mu μ shrinks too hard, some of the covariate–outcome association is left over, and the treatment contrast can absorb it. This is easiest to see under targeted selection, where the probability of treatment depends on the expected outcome under control, and μ ( x ) \mu(\mathbf{x}) μ ( x ) and e ( x ) e(\mathbf{x}) e ( x ) then describe much of the same variation. Approximating that surface with axis-aligned splits takes a deep tree, which (14) penalizes, while a split on A A A is penalized less. Supplying e ^ \hat{e} e ^ gives the prognostic trees a single covariate that tracks this dependence. We estimate it once and keep it fixed while the outcome model is updated. Hahn et al. [34 ] treat the estimated score as a covariate transformation, not as an extra unknown with its own posterior. A related point is made by Linero [48 ] : taking the outcome regression and the treatment assignment mechanism to be a priori independent is already an uninformative prior, and one way to break this independence is to let the propensity score enter the mean function.
Sampling alternates between the two forests on partial residuals using Bayesian backfitting [35 ] . With the latest draw of the other function we set
R i μ = y i − A i τ ( x i ) , (22) R_i^\mu= y_i - A_i\tau(\mathbf{x}_i), \tag*{(22)} R i μ = y i − A i τ ( x i ) , (22)
R i τ = y i − μ ( x i , e ^ i ) , (23) R_i^\tau= y_i - \mu(\mathbf{x}_i,\hat{e}_i), \tag*{(23)} R i τ = y i − μ ( x i , e ^ i ) , (23)
and we update the prognostic trees against R i μ R_i^\mu R i μ . The treatment update uses the conditional model R i τ = A i τ ( x i ) + ε i R_i^\tau= A_i\tau(\mathbf{x}_i) + \varepsilon_i R i τ = A i τ ( x i ) + ε i . Under the fixed 0/1 coding, a control observation has conditional mean μ ( x i , e ^ i ) \mu(\mathbf{x}_i,\hat{e}_i) μ ( x i , e ^ i ) whatever τ \tau τ is, and the treatment trees are updated from the treated observations. The prognostic scale and the treatment scale are then drawn from their full conditionals under the half-Cauchy and half-normal priors, given the current leaf values. The noise variance is drawn from its inverse-gamma full conditional given the residuals y i − μ ( x i , e ^ i ) − A i τ ( x i ) y_i-\mu(x_i,\hat e_i)-A_i\tau(x_i) y i − μ ( x i , e ^ i ) − A i τ ( x i ) . The stored draws of τ ( x i ) \tau(x_i) τ ( x i ) are the posterior of the CATE. Appendix § 10 gives the sampler for our model.
Other BCF variants change a different part of this model and still take the leaf values as independent given their scales. Targeted smooth BCF makes the treatment effect smooth along one chosen covariate (Starling et al., [66 ] ). Shrinkage BCF learns the splitting probabilities rather than fixing (14) (Caron et al., [17 ] ; Linero, [47 ] ). Horseshoe forests put a separate half-Cauchy scale on each leaf, in a survival model (Jacobs et al., [38 ] ). We use a BCF-type prior for μ \mu μ , with a half-Cauchy scale on the prognostic forest (Appendix § 12 .§ 1 ). On the treatment leaves we use the joint Dirichlet-Tree allocation of § 3.3 instead of independent normal variances.
Meta-Learners for Heterogeneous Treatment Effects
Meta-learners are estimation strategies for heterogeneous treatment effects that utilize arbitrary supervised learning methods as so-called base learners. Prominent examples include the S-, T-, X-, R and DR-learner [45 , 53 , 40 ] . The S- and T-learners constitute the most basic approaches. Under the identifiability assumptions (consistency, no interference, conditional exchangeability, positivity), the CATE can be written as
τ ( x ) = E { y i ( 1 ) − y i ( 0 ) ∣ x i = x } = E { y i ( a i ) ∣ x i = x , a i = 1 } − E { y i ( a i ) ∣ x i = x , a i = 0 } \tau(x) = \mathbb{E}\{y_i(1) - y_i(0) \mid x_i = x\}
= \mathbb{E}\{y_i(a_i) \mid x_i = x, a_i = 1\} - \mathbb{E}\{y_i(a_i) \mid x_i = x, a_i = 0\} τ ( x ) = E { y i ( 1 ) − y i ( 0 ) ∣ x i = x } = E { y i ( a i ) ∣ x i = x , a i = 1 } − E { y i ( a i ) ∣ x i = x , a i = 0 }
The S-learner fits a single regression model m ^ \hat{m} m ^ to
m ( x i , a i ) = E { y i ( a i ) ∣ x i = x , a i = a i } m(x_i, a_i) = \mathbb{E}\{y_i(a_i) \mid x_i = x, a_i = a_i\} m ( x i , a i ) = E { y i ( a i ) ∣ x i = x , a i = a i }
and predicts the CATE as m ^ ( x , 1 ) − m ^ ( x , 0 ) \hat{m}(x, 1) - \hat{m}(x, 0) m ^ ( x , 1 ) − m ^ ( x , 0 ) . A limitation of this approach is that the treatment is treated as a regular covariate that can be overlooked by the model.
In comparison, the T-learner improves upon this issue by fitting two separate regression models to
m a ( x i ) = E { y i ( a i ) ∣ x i = x , a i = a } , a ∈ { 0 , 1 } , m_a(x_i) = \mathbb{E}\{y_i(a_i) \mid x_i = x, a_i = a\}, \quad a \in\{0, 1\}, m a ( x i ) = E { y i ( a i ) ∣ x i = x , a i = a } , a ∈ { 0 , 1 } ,
using only observations from the respective treatment arm. The T-learner estimates the CATE by taking the difference m ^ 1 ( x ) − m ^ 0 ( x ) \hat{m}_1(x) - \hat{m}_0(x) m ^ 1 ( x ) − m ^ 0 ( x ) . This approach has the disadvantage that it is susceptible to regularization bias [45 ] , particularly when the true CATE is close to zero.
Since the true CATE is unobserved in real-data settings, simulation studies are a valuable tool to assist practitioners with choosing a suitable meta-learner. Based on previous simulation studies [16 , 42 ] (Xu et al. 2023), we consider the X- and R-learners as strong choices and therefore include them as baselines within our experiments. For the nuisance functions learned within these algorithms, including the propensity score, we use XGBoost [20 ] as the base learner which is reliable default choice on tabular data [32 ] .
X-Learner
The X-learner [45 ] is a meta-learner that estimates the CATE by computing a weighted average of imputed treatment effects that requires an estimate of the propensity score. Analogously to the T-learner, the X-learner begins by learning a regression model for m 1 m_1 m 1 and m 0 m_0 m 0 stratified by treatment. Then, for each observation i i i , the imputed treatment effect constructed as
d ~ i ( 1 ) = y i ( 1 ) − m ^ 0 ( x i ) if a i = 1 , or (24) \tilde{d}_i(1) = y_i(1) - \hat{m}_0(x_i) \quad\text{if } a_i = 1,\ \text{or} \tag*{(24)} d ~ i ( 1 ) = y i ( 1 ) − m ^ 0 ( x i ) if a i = 1 , or (24)
d ~ i ( 0 ) = m ^ 1 ( x i ) − y i ( 0 ) if a i = 0. (25) \tilde{d}_i(0) = \hat{m}_1(x_i) - y_i(0) \quad\text{if } a_i = 0. \tag*{(25)} d ~ i ( 0 ) = m ^ 1 ( x i ) − y i ( 0 ) if a i = 0. (25)
Under m ^ a = m a \hat{m}_a = m_a m ^ a = m a , we have that τ ( x ) = E [ d ~ i ( a ) ∣ x i = x ] = : τ a ( x ) \tau(x) = \mathbb{E}[\tilde{d}_i(a)\mid x_i = x] =: \tau_a(x) τ ( x ) = E [ d ~ i ( a ) ∣ x i = x ] =: τ a ( x ) for a ∈ { 0 , 1 } a \in\{0,1\} a ∈ { 0 , 1 } . In the second step, the X-learner fits regression models to τ 1 ( x ) \tau_1(x) τ 1 ( x ) and τ 0 ( x ) \tau_0(x) τ 0 ( x ) using the data from the respective treatment arm. We obtain the final CATE estimate as the propensity-score-weighted average of the two CATE models
τ ^ ( x ) = e ^ ( x ) τ 0 ( x ) + ( 1 − e ^ ( x ) ) τ 1 ( x ) . \hat{\tau}(x) = \hat{e}(x)\tau_0(x) + (1 - \hat{e}(x))\tau_1(x). τ ^ ( x ) = e ^ ( x ) τ 0 ( x ) + ( 1 − e ^ ( x )) τ 1 ( x ) .
This construction is especially beneficial under unbalanced treatment designs. Without loss of generality, assume that group 0 is substantially smaller than group 1. Then, m ^ 1 \hat{m}_1 m ^ 1 can be estimated accurately using the large treatment arm 1. Hence, the resulting imputed treatment effects d ~ i ( 0 ) \tilde{d}_i(0) d ~ i ( 0 ) in the small group can be considered highly informative. In the final estimate, the corresponding CATE estimate τ 0 ( x ) \tau_0(x) τ 0 ( x ) receives more weight than the model τ 1 ( x ) \tau_1(x) τ 1 ( x ) based on the less informative imputed treatment effects, given that treatment 1 is generally more common than treatment 0. In summary, the X-learner allows to construct accurate CATE estimates in unbalanced settings by efficiently using information from the larger treatment group. In comparison, small treatment groups can directly harm the T-learner as it requires reliable estimates of both m 1 m_1 m 1 and m 0 m_0 m 0 to obtain an accurate CATE model.
R-Learner
The R-learner [53 ] is a meta-learner that estimates the CATE by minimizing a loss function using residuals of the treatment a a a and the conditional mean outcome m ∗ ( x ) = E [ y i ∣ x i = x ] m^*(x) = \mathbb{E}[y_i \mid x_i = x] m ∗ ( x ) = E [ y i ∣ x i = x ] . Specifically, the CATE function can be characterized by
τ ( ⋅ ) = argmin τ ~ ( ⋅ ) E [ ( ( y i − m ∗ ( x i ) ) − ( a i − e ( x i ) ) τ ~ ( x i ) ) 2 ] = argmin τ ~ ( ⋅ ) E [ ( a i − e ( x i ) ) 2 ( y i − m ∗ ( x i ) a i − e ( x i ) − τ ~ ( x i ) ) 2 ] . \begin{aligned}
\tau(\cdot) &= \underset{\tilde{\tau}(\cdot)}{\operatorname{argmin}}\ \mathbb{E}\left[\left((y_i-m^*(x_i))-(a_i-e(x_i))\tilde{\tau}(x_i)\right)^2\right] \\
&= \underset{\tilde{\tau}(\cdot)}{\operatorname{argmin}}\ \mathbb{E}\left[(a_i-e(x_i))^2\left(\frac{y_i-m^*(x_i)}{a_i-e(x_i)}-\tilde{\tau}(x_i)\right)^2\right].
\end{aligned} τ ( ⋅ ) = τ ~ ( ⋅ ) argmin E [ ( ( y i − m ∗ ( x i )) − ( a i − e ( x i )) τ ~ ( x i ) ) 2 ] = τ ~ ( ⋅ ) argmin E [ ( a i − e ( x i ) ) 2 ( a i − e ( x i ) y i − m ∗ ( x i ) − τ ~ ( x i ) ) 2 ] .
Since m ∗ m^* m ∗ and e e e are generally unknown, these need to be replaced by estimates to obtain the CATE based on the above loss-based representation.
The R-learner algorithm starts by dividing the training data into q q q folds to obtain cross-fitted estimates of m ^ ∗ \widehat{m}^* m ∗ and e ^ \widehat{e} e for each observation. More precisely, for observation i i i , the estimate is obtained from a model fitted on the q − 1 q-1 q − 1 folds that do not contain observation i i i . Let m ^ ∗ − q ( i ) ( x i ) \widehat{m}^{*-q(i)}(x_i) m ∗− q ( i ) ( x i ) and e ^ − q ( i ) ( x i ) \widehat{e}^{-q(i)}(x_i) e − q ( i ) ( x i ) denote the corresponding out-of-fold predictions. Using these estimates, the pseudo-outcomes
h i ∗ = y i − m ^ ∗ − q ( i ) ( x i ) a i − e ^ − q ( i ) ( x i ) h_i^* = \frac{y_i-\widehat{m}^{*-q(i)}(x_i)}{a_i-\widehat{e}^{-q(i)}(x_i)} h i ∗ = a i − e − q ( i ) ( x i ) y i − m ∗− q ( i ) ( x i )
are constructed. Then, the R-learner fits a weighted regression model of h i ∗ h_i^* h i ∗ on x x x , using ( a i − e ^ − q ( i ) ( x i ) ) 2 (a_i-\widehat{e}^{-q(i)}(x_i))^2 ( a i − e − q ( i ) ( x i ) ) 2 as regression weights, to obtain the CATE estimate τ ^ ( x ) \widehat{\tau}(x) τ ( x ) .
An advantageous property of the R-learner is its quasi-oracle property. Specifically, Nie and Wager (2021) show that, when penalized kernel regression is used to obtain the regression model for τ ^ ( x ) \widehat{\tau}(x) τ ( x ) , the asymptotic error bounds are unaffected by replacing the ground-truth functions m ∗ m^* m ∗ and e e e by their estimated versions.
Accelerated Failure Time Models
Under the accelerated failure time (AFT) model [39 ] , the logarithm of the survival time is linked to covariates through an additive regression model. Under the CATE formulation in Eq. (1) used by the Bayesian causal forest [34 ] , this can be written as
log ( T ( a ) ) = μ ( x , e ^ ) + a τ ( x ) + ε , \log(T(a)) = \mu(x,\widehat{e}) + a\tau(x) + \varepsilon, log ( T ( a )) = μ ( x , e ) + a τ ( x ) + ε ,
where ε ∼ N ( 0 , σ 2 ) \varepsilon\sim\mathcal{N}(0,\sigma^2) ε ∼ N ( 0 , σ 2 ) and τ ( x ) \tau(x) τ ( x ) is the CATE on the log-survival-time scale. In our experiments in § 5 , we consider the right-censored survival data setting, where the survival time is observed only through min { T , C } \min\{T,C\} min { T , C } , with C C C denoting the censoring time. In addition to the causal identification assumptions (SUTVA and strong ignorability), identification from right-censored data requires a conditional non-informative censoring assumption. Specifically, we assume that C ( a ) C(a) C ( a ) and T ( a ) T(a) T ( a ) are independent given treatment a ∈ 0 , 1 a \in0,1 a ∈ 0 , 1 and covariates x x x , such that the distribution of T ( a ) T(a) T ( a ) can be recovered based on right-censored data [38 ] .
Posterior Computation
We sample from the posterior by Bayesian backfitting [21 , 34 ] . The trees are updated one at a time, each conditional on its partial residuals. A structural proposal integrates out the leaf values. Conditional on the resulting tree, these values are drawn from their normal full conditionals. For a treatment tree, the branch fractions and tree scale are updated by one of two procedures. The collapsed procedure uses slice sampling with the leaf values integrated out and applies for all a a a and ρ \rho ρ . When the tree-scale prior is coupled to the allocation, the blocked procedure jointly updates the tree scale and all terminal shares.
Sections D.1 to D.7 describe the sampler for the model of § 3 under the settings of Appendix § 12 .§ 1 . Section D.8 gives the additional update required for the accelerated failure time model with censored outcomes. The implementation uses R, with the leaf likelihoods and branch-fraction updates compiled in C++.
One Iteration
One iteration of the sampler performs the following steps in order, conditional on the most recent values of all remaining quantities.
Prognostic forest. Each prognostic tree is updated by one structural move followed by a draw of its leaf values. For unit i i i , the partial residual is y i − A i { α + g τ ( x i ) } y_i-A_i\{\alpha+g_\tau(x_i)\} y i − A i { α + g τ ( x i )} minus the contribution of the other prognostic trees. All n n n units contribute to this update. After the last tree, the forest scale ψ μ 2 \psi_\mu^2 ψ μ 2 is drawn. With DART, the split probabilities of the prognostic forest are drawn next.
Intercept. The intercept α \alpha α is drawn from its normal full conditional.
Treatment forest. Each treatment tree j j j is updated by one structural move. Its branch fractions, tree scale ψ j 2 \psi_j^2 ψ j 2 and leaf values are then updated in the order specified by the procedure in Appendix § 10 .§ 5 . For a treated unit i i i , the partial residual is R i j = y i − μ ( x i , e ^ i ) − α − ∑ j ′ ≠ j g ( x i ; T j ′ , h j ′ ) R_{ij}=y_i-\mu(x_i,\widehat e_i)-\alpha-\sum_{j'\ne j}g(x_i;T_{j'},h_{j'}) R ij = y i − μ ( x i , e i ) − α − ∑ j ′ = j g ( x i ; T j ′ , h j ′ ) . Only treated units contribute to this update. With DART, the split probabilities of the treatment forest are drawn after the last tree.
Residual variance. The variance σ 2 \sigma^2 σ 2 is drawn from its inverse-gamma full conditional.
Under the 0/1 coding of (2) , the likelihood contribution of a control unit does not depend on g τ g_\tau g τ . The treatment trees are therefore fitted to the treated units, and their splitting rules are formed from the covariates of those units. The treatment effect τ ( x i ) = α + g τ ( x i ) \tau(x_i)=\alpha+g_\tau(x_i) τ ( x i ) = α + g τ ( x i ) is nevertheless evaluated at every unit, including the control units.
Leaf Values
Consider one tree with partial residuals R i R_i R i . Let b ℓ b_\ell b ℓ denote the prior variance of its leaf value h ℓ h_\ell h ℓ divided by σ 2 \sigma^2 σ 2 . For a treatment leaf, b ℓ = ω ψ j 2 ϕ j ℓ b_\ell=\omega\psi_j^2\phi_{j\ell} b ℓ = ω ψ j 2 ϕ j ℓ , so its value depends on the terminal share ϕ j ℓ \phi_{j\ell} ϕ j ℓ . For a prognostic leaf, b ℓ = ω μ ψ μ 2 b_\ell=\omega\mu\psi_\mu^2 b ℓ = ω μ ψ μ 2 is common to all leaves of the forest. Let n ℓ n_\ell n ℓ be the number of units in leaf ℓ \ell ℓ and S ℓ S_\ell S ℓ the sum of their partial residuals. Integrating out h ℓ h_\ell h ℓ gives the leaf factor
log M ℓ ( b ℓ ) = − 1 2 log ( 1 + n ℓ b ℓ ) + S ℓ 2 b ℓ 2 σ 2 ( 1 + n ℓ b ℓ ) , (26) \log M_\ell(b_\ell)=-\frac{1}{2}\log(1+n_\ell b_\ell)+\frac{S_\ell^2 b_\ell}{2\sigma^2(1+n_\ell b_\ell)}, \tag*{(26)} log M ℓ ( b ℓ ) = − 2 1 log ( 1 + n ℓ b ℓ ) + 2 σ 2 ( 1 + n ℓ b ℓ ) S ℓ 2 b ℓ , (26)
up to terms that do not depend on the partition or on b ℓ b_\ell b ℓ . Whenever the leaf values are integrated out, the acceptance ratios and conditional densities below depend on products and ratios of these factors. Conditional on the tree and b ℓ b_\ell b ℓ , the leaf value is drawn from
h ℓ ∣ ⋅ ∼ N ( S ℓ b ℓ 1 + n ℓ b ℓ , σ 2 b ℓ 1 + n ℓ b ℓ ) . h_\ell\mid\cdot\sim\mathcal{N}\left(\frac{S_\ell b_\ell}{1+n_\ell b_\ell},\frac{\sigma^2 b_\ell}{1+n_\ell b_\ell}\right). h ℓ ∣ ⋅ ∼ N ( 1 + n ℓ b ℓ S ℓ b ℓ , 1 + n ℓ b ℓ σ 2 b ℓ ) .
The conditional mean can equivalently be written as ( 1 − κ ℓ ) S ℓ / n ℓ (1-\kappa_\ell)S_\ell/n_\ell ( 1 − κ ℓ ) S ℓ / n ℓ , where κ ℓ = ( 1 + n ℓ b ℓ ) − 1 \kappa_\ell=(1+n_\ell b_\ell)^{-1} κ ℓ = ( 1 + n ℓ b ℓ ) − 1 is the shrinkage factor in (9) .
Splitting Rules and DART
A splitting rule at node v v v consists of a covariate k k k and a cutpoint c c c . Let q q q be the number of covariates available to a forest. The treatment forest has q = p q=p q = p , whereas the prognostic forest has q = p + 1 q=p+1 q = p + 1 because it may also split on e ^ i \hat{e}_i e ^ i . Given the units that reach v v v , covariate k k k is selected with probability s k s_k s k . Conditional on k k k , the cutpoint is drawn uniformly from the K v k K_{vk} K v k valid cutpoints, defined as the midpoints between consecutive distinct values of x k x_k x k among these units that send at least one unit to each child. The resulting rule has probability s k / K v k s_k/K_{vk} s k / K v k . If the selected covariate has no valid cutpoint at v v v , the tree remains unchanged. These rule probabilities, together with the split probability in (14) , define the prior on the topology and its rules. We set ν = 0.95 \nu=0.95 ν = 0.95 and use the value of η \eta η specified in Appendix § 12 .§ 1 .
Without DART, s k = 1 / q s_k=1/q s k = 1/ q . Under DART (Linero, [47 ] ), the split probabilities s = ( s 1 , … , s q ) \boldsymbol{s}=(s_1,\ldots,s_q) s = ( s 1 , … , s q ) of a forest have the prior Dirichlet ( θ / q , … , θ / q ) \operatorname{Dirichlet}(\theta/q,\ldots,\theta/q) Dirichlet ( θ / q , … , θ / q ) with θ = 1 \theta=1 θ = 1 . After the last tree of the forest, they are drawn from s ∣ ⋅ ∼ Dirichlet ( θ / q + c 1 , … , θ / q + c q ) \boldsymbol{s}\mid\cdot\sim\operatorname{Dirichlet}(\theta/q+c_1,\ldots,\theta/q+c_q) s ∣ ⋅ ∼ Dirichlet ( θ / q + c 1 , … , θ / q + c q ) , where c k c_k c k is the number of internal nodes that split on covariate k k k .
Structural Moves
Each tree update proposes one of four moves, GROW, PRUNE, CHANGE and SWAP, with probabilities 0.3, 0.3, 0.2 and 0.2 (Chipman et al., [21 ] ). Its Metropolis–Hastings acceptance probability integrates out the leaf values through the factors in Equation (26) .
GROW and PRUNE. GROW selects a terminal node P P P uniformly from the L L L leaves, draws a rule at P P P as in Appendix § 10 .§ 3 , and splits P P P into the children P L P_L P L and P R P_R P R . In a treatment tree with a random allocation, GROW also draws the new branch fraction from its prior, U P ∼ Beta { c L ( P ) , c R ( P ) } U_P\sim\operatorname{Beta}\{c_L(P),c_R(P)\} U P ∼ Beta { c L ( P ) , c R ( P )} in (6) . The resulting child shares are ϕ P L = U P ϕ P \phi_{P_L}=U_P\phi_P ϕ P L = U P ϕ P and ϕ P R = ( 1 − U P ) ϕ P \phi_{P_R}=(1-U_P)\phi_P ϕ P R = ( 1 − U P ) ϕ P . The proposal is accepted with probability min ( 1 , r ) \min(1,r) min ( 1 , r ) , where
r = M P L M P R M P ⋅ p s p l i t ( d P ) { 1 − p s p l i t ( d P + 1 ) } 2 1 − p s p l i t ( d P ) ⋅ π ( U ′ ∣ T ′ ) π ( U ∣ T ) Beta { U P ; c L ( P ) , c R ( P ) } ⋅ L N ′ . r=\frac{M_{P_L}M_{P_R}}{M_P}\cdot\frac{p_{\mathrm{split}}(d_P)\{1-p_{\mathrm{split}}(d_P+1)\}^2}{1-p_{\mathrm{split}}(d_P)}\cdot\frac{\pi(U' \mid T')}{\pi(U\mid T)\operatorname{Beta}\{U_P;c_L(P),c_R(P)\}}\cdot\frac{L}{N'}. r = M P M P L M P R ⋅ 1 − p split ( d P ) p split ( d P ) { 1 − p split ( d P + 1 ) } 2 ⋅ π ( U ∣ T ) Beta { U P ; c L ( P ) , c R ( P )} π ( U ′ ∣ T ′ ) ⋅ N ′ L .
Here d P d_P d P is the depth of P P P , N ′ N' N ′ is the number of internal nodes of the proposed tree T ′ T' T ′ whose two children are terminal, and π ( U ∣ T ) \pi(U\mid T) π ( U ∣ T ) is the product of the beta densities of the branch fractions under T T T . The rule probability cancels because the rule is drawn from its prior, and the move probabilities cancel because GROW and PRUNE are proposed with the same probability. At ρ = 1 / 2 \rho=1/2 ρ = 1/2 , splitting P P P leaves the parameters c L ( v ) c_L(v) c L ( v ) and c R ( v ) c_R(v) c R ( v ) of every ancestor v v v unchanged, so the third factor equals one. At ρ = 1 \rho=1 ρ = 1 , the third factor accounts for the corresponding changes in the ancestor parameters. PRUNE selects uniformly one of the N N N internal nodes whose two children are terminal and removes those children together with their branch fraction. Its acceptance probability uses the reciprocal of the corresponding GROW ratio. Prognostic trees and treatment-tree priors without random branch fractions omit the branch-fraction draw. Their factors M ℓ M_\ell M ℓ use the leaf variances implied by the corresponding prior.
CHANGE and SWAP. CHANGE selects an internal node v v v uniformly and draws a replacement rule conditional on the units that reach it. SWAP selects an internal node v v v with at least one internal child, selects one such child uniformly, and exchanges the two rules. Neither proposal changes the topology or the branch fractions. Both proposals can redistribute units among the descendants of v v v , thereby changing the valid cutpoints and rule probabilities at those nodes. To preserve rule validity and reversibility, each internal node below v v v retains its covariate and the rank of its cutpoint among the valid cutpoints for that covariate. After the proposal, its cutpoint is placed at the same rank among the cutpoints that are valid for the units that now reach the node. The proposal is rejected if any retained rank exceeds the number of available cutpoints. Otherwise, it is accepted with probability min ( 1 , r ) \min(1,r) min ( 1 , r ) . The ratio r r r contains the ratio of the products of M ℓ M_\ell M ℓ over the leaves below v v v and the ratio of the products of the rule probabilities over the internal nodes below v v v . For SWAP, the latter product also includes v v v . For CHANGE, the rule probability at v v v cancels with its proposal probability.
Tree Scales and Branch fractions
After the structural move of treatment tree j j j , the sampler uses one of two procedures to update its branch fractions, tree scale and leaf values. Under the collapsed procedure, the branch fractions and tree scale are updated with the leaf values integrated out, after which the leaf values are redrawn. The blocked procedure first applies the collapsed branch updates and redraws the leaf values, then jointly updates the tree scale and terminal shares conditional on those values. The collapsed procedure applies for all a a a and ρ \rho ρ . The blocked procedure requires the beta-prime prior on the tree scale to be coupled to the Dirichlet-Tree allocation through s ψ = 1 s_\psi= 1 s ψ = 1 and a ψ = a W j a_\psi= aW_j a ψ = a W j ([44 , 77 , 2 ] ). This condition holds at the default ρ = 1 / 2 \rho= 1/2 ρ = 1/2 .
Collapsed Update. Each branch fraction is updated in turn with the leaf values integrated out. Write η v = log { U j v / ( 1 − U j v ) } \eta_v = \log\{U_{jv}/(1-U_{jv})\} η v = log { U j v / ( 1 − U j v )} and expit ( x ) = 1 / ( 1 + e − x ) \operatorname{expit}(x) = 1/(1+e^{-x}) expit ( x ) = 1/ ( 1 + e − x ) . Conditional on the remaining branch fractions, the tree scale and the partial residuals, the density of η v \eta_v η v is
p ( η v ∣ ⋅ ) ∝ exp ( η v ) c L ( v ) expit ( − η v ) c R ( v ) ∏ ℓ ∈ L v M ℓ ( ω ψ j 2 ϕ j ℓ ) , p(\eta_v \mid\cdot) \propto\exp(\eta_v)^{c_L(v)}\operatorname{expit}(-\eta_v)^{c_R(v)} \prod_{\ell\in\mathcal{L}_v} M_\ell(\omega\psi_j^2\phi_{j\ell}), p ( η v ∣ ⋅ ) ∝ exp ( η v ) c L ( v ) expit ( − η v ) c R ( v ) ℓ ∈ L v ∏ M ℓ ( ω ψ j 2 ϕ j ℓ ) ,
where L v \mathcal{L}_v L v is the set of leaves below v v v , whose shares depend on η v \eta_v η v . The first two factors give the beta prior of U j v U_{jv} U j v on the logit scale, including its Jacobian. We draw each η v \eta_v η v by univariate slice sampling with stepping out (Neal, [52 ] ), using the initial width max { 1 , 1 / min ( c L ( v ) , c R ( v ) ) } \max\{1,1/\min(c_L(v),c_R(v))\} max { 1 , 1/ min ( c L ( v ) , c R ( v ))} and at most 50 steps. The tree scale is then drawn by slice sampling with stepping out on log ψ j 2 \log\psi_j^2 log ψ j 2 . The initial width is one, and the target density is
p ( ψ j 2 ∣ ⋅ ) ∝ π ( ψ j 2 ) ∏ ℓ ∈ L ( T j ) M ℓ ( ω ψ j 2 ϕ j ℓ ) , p(\psi_j^2 \mid\cdot) \propto\pi(\psi_j^2) \prod_{\ell\in\mathcal{L}(T_j)} M_\ell(\omega\psi_j^2\phi_{j\ell}), p ( ψ j 2 ∣ ⋅ ) ∝ π ( ψ j 2 ) ℓ ∈ L ( T j ) ∏ M ℓ ( ω ψ j 2 ϕ j ℓ ) ,
where π \pi π is the beta-prime density in (8) . The leaf values are drawn last, conditional on the updated branch fractions and tree scale. Marginalizing the leaf values from the preceding updates addresses the posterior dependence induced by weak identification of a single tree. When the data contain little information about h j h_j h j , the leaf values and ψ j 2 \psi_j^2 ψ j 2 are strongly dependent a posteriori. An update of ψ j 2 \psi_j^2 ψ j 2 conditional on h j h_j h j can consequently mix slowly, whereas the collapsed procedure updates the tree scale under its marginal conditional distribution.
Blocked Update. Define B j ℓ = ψ j 2 ϕ j ℓ B_{j\ell} = \psi_j^2\phi_{j\ell} B j ℓ = ψ j 2 ϕ j ℓ , let w ℓ w_\ell w ℓ be the weights in (6), and write W j = ∑ ℓ w ℓ W_j = \sum_\ell w_\ell W j = ∑ ℓ w ℓ . When s ψ = 1 s_\psi= 1 s ψ = 1 and a ψ = a W j a_\psi= aW_j a ψ = a W j , the prior admits the hierarchical gamma representation used in the R2D2 prior (Zhang et al. [77 ] ). Introduce the auxiliary variable ξ ∼ Gamma ( b ψ , 1 ) \xi\sim\operatorname{Gamma}(b_\psi,1) ξ ∼ Gamma ( b ψ , 1 ) , where gamma distributions are parameterized by shape and rate. Conditional on ξ \xi ξ , the variables B j ℓ B_{j\ell} B j ℓ are independently distributed as Gamma ( a w ℓ , ξ ) \operatorname{Gamma}(aw_\ell,\xi) Gamma ( a w ℓ , ξ ) . It follows that ψ j 2 = ∑ ℓ B j ℓ ∼ BetaPrime ( a W j , b ψ , 1 ) \psi_j^2 = \sum_\ell B_{j\ell} \sim\operatorname{BetaPrime}(aW_j,b_\psi,1) ψ j 2 = ∑ ℓ B j ℓ ∼ BetaPrime ( a W j , b ψ , 1 ) and that ϕ j = B j / ψ j 2 ∼ Dirichlet { ( a w ℓ ) ℓ } \phi_j = B_j/\psi_j^2 \sim\operatorname{Dirichlet}\{(aw_\ell)_\ell\} ϕ j = B j / ψ j 2 ∼ Dirichlet {( a w ℓ ) ℓ } is independent of ψ j 2 \psi_j^2 ψ j 2 . Under the coupling condition, this representation recovers the prior of § 3 . Conditional on the leaf values, the B j ℓ B_{j\ell} B j ℓ are independent with densities
p ( B j ℓ ∣ h j , ξ , ⋅ ) ∝ B j ℓ a w ℓ − 3 / 2 exp { − ξ B j ℓ − h j ℓ 2 2 σ 2 ω B j ℓ } . p(B_{j\ell}\mid h_j,\xi,\cdot) \propto B_{j\ell}^{aw_\ell-3/2}\exp\left\{-\xi B_{j\ell}-\frac{h_{j\ell}^2}{2\sigma^2\omega B_{j\ell}}\right\}. p ( B j ℓ ∣ h j , ξ , ⋅ ) ∝ B j ℓ a w ℓ − 3/2 exp { − ξ B j ℓ − 2 σ 2 ω B j ℓ h j ℓ 2 } .
which are generalized inverse Gaussian densities. Conditional on the B j ℓ B_{j\ell} B j ℓ , the auxiliary variable is drawn from ξ ∣ ⋅ ∼ Gamma ( a W j + b ψ , 1 + ψ j 2 ) \xi\mid\cdot\sim\operatorname{Gamma}(aW_j+b_\psi,1+\psi_j^2) ξ ∣ ⋅ ∼ Gamma ( a W j + b ψ , 1 + ψ j 2 ) . Drawing and normalizing the B j ℓ B_{j\ell} B j ℓ updates the tree scale and all terminal shares in one block, following the compositional strategy of Aguilar and Bürkner [2 ] . The updated values are ψ j 2 = ∑ ℓ B j ℓ \psi_j^2=\sum_\ell B_{j\ell} ψ j 2 = ∑ ℓ B j ℓ and ϕ j ℓ = B j ℓ / ψ j 2 \phi_{j\ell}=B_{j\ell}/\psi_j^2 ϕ j ℓ = B j ℓ / ψ j 2 . Each branch fraction is reconstructed as the proportion of the share below its node that belongs to the left subtree.
At ρ = 1 / 2 \rho=1/2 ρ = 1/2 , every topology has W j = 1 W_j=1 W j = 1 , so the coupling condition holds with a ψ = a = 1 / 2 a_\psi=a=1/2 a ψ = a = 1/2 . At ρ = 1 \rho=1 ρ = 1 , W j = L j W_j=L_j W j = L j changes with the topology. No fixed value of a ψ a_\psi a ψ then satisfies the condition for every tree, and the blocked update would target a different prior. At ρ = 1 / 2 \rho=1/2 ρ = 1/2 , the collapsed and blocked procedures target the same posterior. The reported fits use the collapsed update.
Prognostic Forest, Intercept and Residual Variance
All prognostic leaves share the variance σ ω 2 ψ μ 2 \sigma^2_\omega\psi_\mu^2 σ ω 2 ψ μ 2 . The forest has one scale ψ μ 2 ∼ BetaPrime ( a μ , b μ , s μ ) \psi_\mu^2\sim\operatorname{BetaPrime}(a_\mu,b_\mu,s_\mu) ψ μ 2 ∼ BetaPrime ( a μ , b μ , s μ ) , with a μ = b μ = 1 / 2 a_\mu=b_\mu=1/2 a μ = b μ = 1/2 and s μ = 1 s_\mu=1 s μ = 1 . After the last prognostic tree, this scale is drawn through the gamma–inverse-gamma representation of the beta-prime distribution using an auxiliary variable ζ \zeta ζ ,
ψ μ 2 ∣ ζ , ⋅ ∼ InvGamma ( b μ + L μ 2 , ζ + ∑ ℓ h ℓ 2 2 σ 2 ω μ ) , ζ ∣ ψ μ 2 ∼ Gamma ( a μ + b μ , 1 ψ μ 2 + 1 s μ ) . \psi_\mu^2\mid\zeta,\cdot\sim\operatorname{InvGamma}\left(b_\mu+\frac{L_\mu}{2},\zeta+\frac{\sum_\ell h_\ell^2}{2\sigma^2\omega_\mu}\right),\qquad
\zeta\mid\psi_\mu^2\sim\operatorname{Gamma}\left(a_\mu+b_\mu,\frac{1}{\psi_\mu^2}+\frac{1}{s_\mu}\right). ψ μ 2 ∣ ζ , ⋅ ∼ InvGamma ( b μ + 2 L μ , ζ + 2 σ 2 ω μ ∑ ℓ h ℓ 2 ) , ζ ∣ ψ μ 2 ∼ Gamma ( a μ + b μ , ψ μ 2 1 + s μ 1 ) .
where L μ L_\mu L μ is the total number of leaves in the prognostic forest and the sum runs over those leaves. The intercept has the prior α ∼ N ( 0 , 100 σ 2 ) \alpha\sim N(0,100\sigma^2) α ∼ N ( 0 , 100 σ 2 ) . For each treated unit, define r i = y i − μ ( x i , e ^ i ) − g τ ( x i ) r_i=y_i-\mu(x_i,\hat e_i)-g_\tau(x_i) r i = y i − μ ( x i , e ^ i ) − g τ ( x i ) , and let n 1 n_1 n 1 denote the number of treated units. Its full conditional is
α ∣ ⋅ ∼ N ( ∑ i : A i = 1 r i n 1 + 1 / 100 , σ 2 n 1 + 1 / 100 ) . \alpha\mid\cdot\sim N\left(\frac{\sum_{i:A_i=1}r_i}{n_1+1/100},\frac{\sigma^2}{n_1+1/100}\right). α ∣ ⋅ ∼ N ( n 1 + 1/100 ∑ i : A i = 1 r i , n 1 + 1/100 σ 2 ) .
Because the prior variances of the leaf values and α \alpha α are proportional to σ 2 \sigma^2 σ 2 , their quadratic forms enter the full conditional of σ 2 \sigma^2 σ 2 . Let RSS = ∑ i [ y i − μ ( x i , e ^ i ) − A i { α + g τ ( x i ) } ] 2 \operatorname{RSS}=\sum_i[y_i-\mu(x_i,\hat e_i)-A_i\{\alpha+g_\tau(x_i)\}]^2 RSS = ∑ i [ y i − μ ( x i , e ^ i ) − A i { α + g τ ( x i )} ] 2 be the residual sum of squares, let L L L be the total number of leaves in both forests, and define Q = ∑ ℓ h ℓ 2 / b ℓ Q=\sum_\ell h_\ell^2/b_\ell Q = ∑ ℓ h ℓ 2 / b ℓ , where the sum runs over all leaves. Under the inverse-gamma prior with shape a σ a_\sigma a σ and scale b σ b_\sigma b σ given in Appendix F.1, the full conditional is
σ 2 ∣ ⋅ ∼ InvGamma ( a σ + n + 1 + L 2 , b σ + RSS 2 + α 2 200 + Q 2 ) . \sigma^2\mid\cdot\sim\operatorname{InvGamma}\left(a_\sigma+\frac{n+1+L}{2},b_\sigma+\frac{\operatorname{RSS}}{2}+\frac{\alpha^2}{200}+\frac{Q}{2}\right). σ 2 ∣ ⋅ ∼ InvGamma ( a σ + 2 n + 1 + L , b σ + 2 RSS + 200 α 2 + 2 Q ) .
Initialization, Chains and Output
Each chain is initialized with single-leaf trees, tree scales drawn from their priors, α = 0 \alpha= 0 α = 0 and σ 2 \sigma^2 σ 2 equal to the sample variance of the centered outcome. The chains run independently under different seeds. Appendix § 12 gives the numbers of chains, iterations and warmup iterations, together with the thinning interval, for each study. At every retained draw, the sampler records τ ( x i ) = α + g τ ( x i ) \tau(x_i) = \alpha+ g_{\tau}(x_i) τ ( x i ) = α + g τ ( x i ) for each unit, the average effect n − 1 ∑ i τ ( x i ) n^{-1}\sum_i \tau(x_i) n − 1 ∑ i τ ( x i ) and the centered effects δ i \delta_i δ i defined in Appendix § 12 .§ 1 . Posterior summaries pool the retained draws across chains.
Censored Outcomes
Right censoring adds a latent-data update to the accelerated failure time model of Appendix § 9 . For patient i i i , y i y_i y i is the log event time when the event is observed and the log censoring time otherwise. For each censored patient, introduce a latent log event time z i > y i z_i > y_i z i > y i . Conditional on the latent times, the model reduces to the Gaussian model in (2) . The updates in Sections D.1 to D.6 therefore apply with z i z_i z i in place of y i y_i y i . Each iteration additionally draws every latent time from
z i ∣ ⋅ ∼ N { μ ( x i , e ~ i ) + A i τ ( x i ) , σ 2 } truncated to ( y i , ∞ ) . z_i \mid\cdot\sim\mathcal{N}\left\{\mu(x_i,\tilde{e}_i) + A_i\tau(x_i), \sigma^2\right\}\text{ truncated to }(y_i,\infty). z i ∣ ⋅ ∼ N { μ ( x i , e ~ i ) + A i τ ( x i ) , σ 2 } truncated to ( y i , ∞ ) .
Each latent time is drawn by exact rejection sampling. When y i y_i y i lies at or below the conditional mean, the proposal is the untruncated normal distribution. Otherwise, the sampler uses the exponential proposal of Robert [60 ] . The log event times of patients with an observed event remain fixed. A censored latent time is initialized at the median of the observed log event times that exceed y i y_i y i . If none exceeds y i y_i y i , it is initialized at y i y_i y i plus half the standard deviation of the observed log event times.
For the leave-one-out estimates in Appendix § 12 .§ 5 , define m i = μ ( x i , e ~ i ) + A i τ ( x i ) m_i = \mu(x_i,\tilde{e}_i) + A_i\tau(x_i) m i = μ ( x i , e ~ i ) + A i τ ( x i ) at each retained draw. For a patient with an observed event, the pointwise likelihood is the normal density with mean m i m_i m i and variance σ 2 \sigma^2 σ 2 evaluated at y i y_i y i . For a censored patient, it is the survival probability 1 − Φ { ( y i − m i ) / σ } 1 - \Phi\{(y_i - m_i)/\sigma\} 1 − Φ {( y i − m i ) / σ } , where Φ \Phi Φ denotes the standard normal distribution function.
Proofs
Split Consistency
We begin by showing consistency under splits.
Proposition 1. For a tree with at least two leaves, and with the other leaf weights unchanged, the weighted allocation is consistent under a split of leaf ℓ \ell ℓ if and only if w ℓ L + w ℓ R = w ℓ w_{\ell_L}+w_{\ell_R}=w_\ell w ℓ L + w ℓ R = w ℓ . For the depth weights w ℓ = ρ d ℓ w_\ell=\rho^{d_\ell} w ℓ = ρ d ℓ , consistency under every split holds if and only if ρ = 1 / 2 \rho=1/2 ρ = 1/2 . Additionally, the symmetric choice is consistent under every split.
Proof. For each r ∈ L ( T ) r\in\mathcal{L}(T) r ∈ L ( T ) , let G r ∼ Gamma ( a w r , 1 ) G_r\sim\operatorname{Gamma}(a w_r,1) G r ∼ Gamma ( a w r , 1 ) independently and define Φ r = G r / ∑ s G s \Phi_r=G_r/\sum_s G_s Φ r = G r / ∑ s G s . Then Φ ∼ Dirichlet { ( a w r ) r } \boldsymbol{\Phi}\sim\operatorname{Dirichlet}\{(a w_r)_r\} Φ ∼ Dirichlet {( a w r ) r } [44 , 46 ] . At an internal node v v v , the sums of the G r G_r G r below its left and right children are independent gamma variables with respective shapes a W v L aW_v^L a W v L and a W v R aW_v^R a W v R . Their normalized ratio has distribution Beta ( a W v L , a W v R ) \operatorname{Beta}(aW_v^L,aW_v^R) Beta ( a W v L , a W v R ) and is independent of the normalized compositions within the two subtrees. Recursive application of this gamma decomposition proves that Equation (6) induces the terminal law Dirichlet { ( a w r ) r } \operatorname{Dirichlet}\{(a w_r)_r\} Dirichlet {( a w r ) r } .
Let T ′ T' T ′ be obtained from a tree T T T with at least two leaves by splitting ℓ \ell ℓ , leaving every other leaf weight unchanged. Dirichlet aggregation under T ′ T' T ′ gives
( ϕ 1 ′ , … , ϕ L − 1 ′ , ϕ ℓ L ′ + ϕ ℓ R ′ ) ∼ Dirichlet { a w 1 , … , a w L − 1 , a ( w ℓ L + w ℓ R ) } . (27) (\phi'_1,\ldots,\phi'_{L-1},\phi'_{\ell_L}+\phi'_{\ell_R})\sim\operatorname{Dirichlet}\{a w_1,\ldots,a w_{L-1},a(w_{\ell_L}+w_{\ell_R})\}. \tag*{(27)} ( ϕ 1 ′ , … , ϕ L − 1 ′ , ϕ ℓ L ′ + ϕ ℓ R ′ ) ∼ Dirichlet { a w 1 , … , a w L − 1 , a ( w ℓ L + w ℓ R )} . (27)
whereas the corresponding vector under T T T has distribution Dirichlet ( a w 1 , … , a w L − 1 , a w ℓ ) \operatorname{Dirichlet}(a w_1,\ldots,a w_{L-1},a w_\ell) Dirichlet ( a w 1 , … , a w L − 1 , a w ℓ ) . Dirichlet parameter vectors are identifiable in dimension at least two [46 ] . The two laws coincide precisely when w ℓ L + w ℓ R = w ℓ w_{\ell_L}+w_{\ell_R}=w_\ell w ℓ L + w ℓ R = w ℓ .
For w ℓ = ρ d ℓ w_\ell=\rho^{d_\ell} w ℓ = ρ d ℓ , the children of a leaf at depth d ℓ d_\ell d ℓ have combined weight
w ℓ L + w ℓ R = 2 ρ d ℓ + 1 = 2 ρ w ℓ . (28) w_{\ell_L}+w_{\ell_R}=2\rho^{d_\ell+1}=2\rho w_\ell. \tag*{(28)} w ℓ L + w ℓ R = 2 ρ d ℓ + 1 = 2 ρ w ℓ . (28)
The required equality holds for every split when 2 ρ = 1 2\rho=1 2 ρ = 1 .
For the symmetric recursion, retain the branch fractions at all internal nodes of T T T and introduce an independent Beta ( a , a ) \operatorname{Beta}(a,a) Beta ( a , a ) fraction at ℓ \ell ℓ . The shares of the unchanged leaves retain their values. The two new shares are U ϕ ℓ U\phi_\ell U ϕ ℓ and ( 1 − U ) ϕ ℓ (1-U)\phi_\ell ( 1 − U ) ϕ ℓ , whose sum equals ϕ ℓ \phi_\ell ϕ ℓ . Aggregation under T ′ T' T ′ recovers the terminal shares under T T T for every realization of the branch fractions, and hence the two aggregated laws agree.
Marginal Distributions
We now show the marginal behavior of the leaf values.
Proposition 2. Let T j T_j T j have at least two leaves, and let c ℓ c_\ell c ℓ be the smallest branch parameter on the path to leaf ℓ \ell ℓ , taking c L ( v ) c_L(v) c L ( v ) at a left branch and c R ( v ) c_R(v) c R ( v ) at a right branch. Under the general Dirichlet-Tree distribution, conditionally on ψ j 2 \psi_j^2 ψ j 2 , the density of h j ℓ h_{j\ell} h j ℓ is bounded at zero if and only if c ℓ > 1 / 2 c_\ell>1/2 c ℓ > 1/2 .
Under the weighted allocation and Equation (8) , let c ℓ ⋆ = min ( a ψ j , a w ℓ ) c_\ell^\star=\min(a\psi_j,a w_\ell) c ℓ ⋆ = min ( a ψ j , a w ℓ ) and let p p p be the density of h j ℓ ∣ T j h_{j\ell}\mid T_j h j ℓ ∣ T j . Then p p p is bounded at zero if and only if c ℓ ⋆ > 1 / 2 c_\ell^\star>1/2 c ℓ ⋆ > 1/2 . If c ℓ ⋆ < 1 / 2 c_\ell^\star<1/2 c ℓ ⋆ < 1/2 and a ψ j ≠ a w ℓ a\psi_j\ne a w_\ell a ψ j = a w ℓ , then p ( h ) ≍ ∣ h ∣ 2 c ℓ ⋆ − 1 p(h)\asymp|h|^{2c_\ell^\star-1} p ( h ) ≍ ∣ h ∣ 2 c ℓ ⋆ − 1 as h → 0 h\to0 h → 0 . In all cases, p ( h ) ≍ ∣ h ∣ − ( 2 b ℓ + 1 ) p(h)\asymp|h|^{-(2b_\ell+1)} p ( h ) ≍ ∣ h ∣ − ( 2 b ℓ + 1 ) as ∣ h ∣ → ∞ |h|\to\infty ∣ h ∣ → ∞ .
Proof. Write κ = σ 2 ω \kappa=\sigma^2\omega κ = σ 2 ω . For a node v v v on the path to leaf ℓ \ell ℓ , let B v B_v B v denote the selected branch fraction, meaning U v U_v U v on a left branch and 1 − U v 1-U_v 1 − U v on a right branch. If c ℓ ( v ) c_\ell(v) c ℓ ( v ) is the parameter on the selected side and c ˉ ℓ ( v ) \bar{c}_\ell(v) c ˉ ℓ ( v ) the parameter on the other side, then the B v B_v B v are independent with B v ∼ Beta { c ℓ ( v ) , c ˉ ℓ ( v ) } B_v\sim\operatorname{Beta}\{c_\ell(v),\bar{c}_\ell(v)\} B v ∼ Beta { c ℓ ( v ) , c ˉ ℓ ( v )} and ϕ ℓ = ∏ v ≻ ℓ B v \phi_\ell=\prod_{v\succ\ell}B_v ϕ ℓ = ∏ v ≻ ℓ B v . Fix ψ j 2 = t > 0 \psi_j^2=t>0 ψ j 2 = t > 0 . We can write the conditional density of h j ℓ h_{j\ell} h j ℓ as
p t ( h ) = E [ ( 2 π κ t ϕ ℓ ) − 1 / 2 exp { − h 2 2 κ t ϕ ℓ } | T j ] . (29) p_t(h)=\mathbb{E}\left[(2\pi\kappa t\phi_\ell)^{-1/2}\exp\left\{-\frac{h^2}{2\kappa t\phi_\ell}\right\}\middle|T_j\right]. \tag*{(29)} p t ( h ) = E [ ( 2 π κ t ϕ ℓ ) − 1/2 exp { − 2 κ t ϕ ℓ h 2 } T j ] . (29)
As ∣ h ∣ ↓ 0 |h|\downarrow0 ∣ h ∣ ↓ 0 , the integrand increases to ( 2 π κ t ) − 1 / 2 ϕ ℓ − 1 / 2 (2\pi\kappa t)^{-1/2}\phi_\ell^{-1/2} ( 2 π κ t ) − 1/2 ϕ ℓ − 1/2 . Monotone convergence and independence along the path give
p t ( 0 ) = 1 2 π κ t ∏ v ≺ ℓ B { c ℓ ( v ) − 1 / 2 , c ‾ ℓ ( v ) } B { c ℓ ( v ) , c ‾ ℓ ( v ) } , (30) p_t(0)=\frac{1}{\sqrt{2\pi\kappa t}}\prod_{v\prec\ell}\frac{B\{c_\ell(v)-1/2,\overline{c}_\ell(v)\}}{B\{c_\ell(v),\overline{c}_\ell(v)\}}, \tag*{(30)} p t ( 0 ) = 2 π κ t 1 v ≺ ℓ ∏ B { c ℓ ( v ) , c ℓ ( v )} B { c ℓ ( v ) − 1/2 , c ℓ ( v )} , (30)
whenever every selected parameter exceeds 1 / 2 1/2 1/2 .
For each v ≺ ℓ v\prec\ell v ≺ ℓ , the moment E ( B v − 1 / 2 ) \mathbb{E}(B_v^{-1/2}) E ( B v − 1/2 ) is finite if and only if c ℓ ( v ) > 1 / 2 c_\ell(v)>1/2 c ℓ ( v ) > 1/2 . Hence, the product in (30) is finite when this condition holds at every node on the path to ℓ \ell ℓ , or equivalently,
p t ( 0 ) < ∞ ⟺ c ℓ > 1 / 2. (31) p_t(0)<\infty\quad\Longleftrightarrow\quad c_\ell>1/2. \tag*{(31)} p t ( 0 ) < ∞ ⟺ c ℓ > 1/2. (31)
Since p t ( h ) ↑ p t ( 0 ) p_t(h)\uparrow p_t(0) p t ( h ) ↑ p t ( 0 ) as ∣ h ∣ ↓ 0 |h|\downarrow0 ∣ h ∣ ↓ 0 , the same condition characterizes boundedness of the conditional density at zero.
Under the weighted allocation, set
α = a w ℓ , β = a ( W j − w ℓ ) , T = ψ j 2 , Φ = ϕ j ℓ , S = T Φ . (32) \alpha=aw_\ell,\qquad\beta=a(W_j-w_\ell),\qquad T=\psi_j^2,\qquad\Phi=\phi_{j\ell},\qquad S=T\Phi. \tag*{(32)} α = a w ℓ , β = a ( W j − w ℓ ) , T = ψ j 2 , Φ = ϕ j ℓ , S = T Φ. (32)
Since T j T_j T j has at least two leaves, β > 0 \beta>0 β > 0 . Hence, the terminal Dirichlet law is Φ ∼ Beta ( α , β ) \Phi\sim\operatorname{Beta}(\alpha,\beta) Φ ∼ Beta ( α , β ) , independently of T ∼ BetaPrime ( a ψ , b ψ , s ψ ) T\sim\operatorname{BetaPrime}(a_\psi,b_\psi,s_\psi) T ∼ BetaPrime ( a ψ , b ψ , s ψ ) . At the origin, monotone convergence gives
p ( 0 ) = ( 2 π κ ) − 1 / 2 E ( T − 1 / 2 ) E ( Φ − 1 / 2 ) . (33) p(0)=(2\pi\kappa)^{-1/2}\mathbb{E}(T^{-1/2})\mathbb{E}(\Phi^{-1/2}). \tag*{(33)} p ( 0 ) = ( 2 π κ ) − 1/2 E ( T − 1/2 ) E ( Φ − 1/2 ) . (33)
The beta-prime moment E ( T r ) \mathbb{E}(T^r) E ( T r ) is finite for − a ψ < r < b ψ -a_\psi<r<b_\psi − a ψ < r < b ψ , and the beta moment E ( Φ r ) \mathbb{E}(\Phi^r) E ( Φ r ) is finite when r > − α r>-\alpha r > − α . The density p p p is bounded at zero if and only if a ψ > 1 / 2 a_\psi>1/2 a ψ > 1/2 and α > 1 / 2 \alpha>1/2 α > 1/2 , equivalently c ℓ ∗ = min ( a ψ , a w ℓ ) > 1 / 2 c_\ell^*=\min(a_\psi,aw_\ell)>1/2 c ℓ ∗ = min ( a ψ , a w ℓ ) > 1/2 .
Let f S f_S f S denote the density of S S S , and suppose a ψ ≠ α a_\psi\ne\alpha a ψ = α . The product-density identity and dominated convergence give
f S ( s ) = C S s c ℓ ∗ − 1 { 1 + o ( 1 ) } , s ↓ 0 , (34) f_S(s)=C_Ss^{c_\ell^*-1}\{1+o(1)\},\qquad s\downarrow0, \tag*{(34)} f S ( s ) = C S s c ℓ ∗ − 1 { 1 + o ( 1 )} , s ↓ 0 , (34)
where
C S = { s ψ − a ψ B ( a ψ , b ψ ) B ( α − a ψ , β ) B ( α , β ) , a ψ < α , s ψ − α B ( a ψ − α , b ψ + α ) B ( a ψ , b ψ ) B ( α , β ) , α < a ψ . (35) C_S=
\begin{cases}
\dfrac{s_\psi^{-a_\psi}}{B(a_\psi,b_\psi)}\dfrac{B(\alpha-a_\psi,\beta)}{B(\alpha,\beta)}, & a_\psi<\alpha,\\[6pt]
\dfrac{s_\psi^{-\alpha}B(a_\psi-\alpha,b_\psi+\alpha)}{B(a_\psi,b_\psi)B(\alpha,\beta)}, & \alpha<a_\psi.
\end{cases} \tag*{(35)} C S = ⎩ ⎨ ⎧ B ( a ψ , b ψ ) s ψ − a ψ B ( α , β ) B ( α − a ψ , β ) , B ( a ψ , b ψ ) B ( α , β ) s ψ − α B ( a ψ − α , b ψ + α ) , a ψ < α , α < a ψ . (35)
When a ψ < α a_\psi<\alpha a ψ < α , the identity
s 1 − a ψ f S ( s ) = s ψ − a ψ B ( a ψ , b ψ ) ∫ 0 1 u − a ψ ( 1 + s s ψ u ) − ( a ψ + b ψ ) f Φ ( u ) d u (36) s^{1-a_\psi}f_S(s)=\frac{s_\psi^{-a_\psi}}{B(a_\psi,b_\psi)}\int_0^1u^{-a_\psi}\left(1+\frac{s}{s_\psi u}\right)^{-(a_\psi+b_\psi)}f_\Phi(u)\,du \tag*{(36)} s 1 − a ψ f S ( s ) = B ( a ψ , b ψ ) s ψ − a ψ ∫ 0 1 u − a ψ ( 1 + s ψ u s ) − ( a ψ + b ψ ) f Φ ( u ) d u (36)
converges by domination because E ( Φ − a ψ ) < ∞ \mathbb{E}(\Phi^{-a_\psi})<\infty E ( Φ − a ψ ) < ∞ . The same domination bounds s 1 − a ψ f S ( s ) s^{1-a_\psi}f_S(s) s 1 − a ψ f S ( s ) uniformly on a neighborhood of zero.
When α < a ψ \alpha<a_\psi α < a ψ , conditioning on T T T gives
s 1 − α f S ( s ) = 1 B ( α , β ) ∫ s ∞ t − α ( 1 − s t ) β − 1 f T ( t ) d t . (37) s^{1-\alpha}f_S(s)=\frac{1}{B(\alpha,\beta)}\int_s^\infty t^{-\alpha}\left(1-\frac{s}{t}\right)^{\beta-1}f_T(t)\,dt. \tag*{(37)} s 1 − α f S ( s ) = B ( α , β ) 1 ∫ s ∞ t − α ( 1 − t s ) β − 1 f T ( t ) d t . (37)
Split the integral at 2 s 2s 2 s . On [ 2 s , ∞ ) [2s,\infty) [ 2 s , ∞ ) , the factor involving β \beta β is bounded by max ( 1 , 2 1 − β ) \max(1,2^{1-\beta}) max ( 1 , 2 1 − β ) , and E ( T − α ) < ∞ \mathbb{E}(T^{-\alpha})<\infty E ( T − α ) < ∞ because α < a ψ \alpha<a_\psi α < a ψ . The contribution from this interval converges to E ( T − α ) / B ( α , β ) \mathbb{E}(T^{-\alpha})/B(\alpha,\beta) E ( T − α ) / B ( α , β ) . On [ s , 2 s ] [s,2s] [ s , 2 s ] , the bound f T ( t ) ≤ C t a ψ − 1 f_T(t)\le Ct^{a_\psi-1} f T ( t ) ≤ C t a ψ − 1 and the substitution t = s u t=su t = s u give an upper bound proportional to
s a ψ − α ∫ 1 2 u a ψ − α − 1 ( 1 − 1 / u ) β − 1 d u . (38) s^{a_\psi-\alpha}\int_1^2 u^{a_\psi-\alpha-1}(1-1/u)^{\beta-1}\,du. \tag*{(38)} s a ψ − α ∫ 1 2 u a ψ − α − 1 ( 1 − 1/ u ) β − 1 d u . (38)
The last equation converges to zero because a ψ > α a_\psi>\alpha a ψ > α and β > 0 \beta>0 β > 0 . We can use these bounds to control s 1 − α f S ( s ) s^{1-\alpha}f_S(s) s 1 − α f S ( s ) uniformly near zero.
The beta-prime density also satisfies the global bound
sup t > 0 t b ψ + 1 f T ( t ) ≤ s ψ b ψ B ( a ψ , b ψ ) . (39) \sup_{t>0}t^{b_\psi+1}f_T(t)\le\frac{s_\psi^{b_\psi}}{B(a_\psi,b_\psi)}. \tag*{(39)} t > 0 sup t b ψ + 1 f T ( t ) ≤ B ( a ψ , b ψ ) s ψ b ψ . (39)
The product-density formula gives
f S ( s ) ≤ s ψ b ψ E ( Φ b ψ ) B ( a ψ , b ψ ) s − b ψ − 1 , s > 0. (40) f_S(s)\le\frac{s_\psi^{b_\psi}\mathbb{E}(\Phi^{b_\psi})}{B(a_\psi,b_\psi)}s^{-b_\psi-1},\qquad s>0. \tag*{(40)} f S ( s ) ≤ B ( a ψ , b ψ ) s ψ b ψ E ( Φ b ψ ) s − b ψ − 1 , s > 0. (40)
Together with the small-s s s estimates we present above, this bound yields a finite M 0 M_0 M 0 such that f S ( s ) ≤ M 0 s c ℓ ∗ − 1 f_S(s)\le M_0s^{c_\ell^*-1} f S ( s ) ≤ M 0 s c ℓ ∗ − 1 for every s > 0 s>0 s > 0 when a ψ ≠ α a_\psi\ne\alpha a ψ = α .
The normal scale-mixture density can be written as [9 , 74 ]
p ( h ) = ∣ h ∣ ∫ 0 ∞ ( 2 π κ r ) − 1 / 2 exp { − 1 / ( 2 κ r ) } f S ( h 2 r ) d r . (41) p(h)=|h|\int_0^\infty(2\pi\kappa r)^{-1/2}\exp\{-1/(2\kappa r)\}f_S(h^2r)\,dr. \tag*{(41)} p ( h ) = ∣ h ∣ ∫ 0 ∞ ( 2 π κ r ) − 1/2 exp { − 1/ ( 2 κ r )} f S ( h 2 r ) d r . (41)
If c ℓ ∗ < 1 / 2 c_\ell^*<1/2 c ℓ ∗ < 1/2 , substitution of Equation (34) and dominated convergence yield
lim h → 0 ∣ h ∣ 1 − 2 c ℓ ∗ p ( h ) = C 0 , ℓ , C 0 , ℓ = C S ( 2 π κ ) − 1 / 2 ( 2 κ ) 1 / 2 − c ℓ ∗ Γ ( 1 / 2 − c ℓ ∗ ) , (42) \lim_{h\to0}|h|^{1-2c_\ell^*}p(h)=C_{0,\ell},\qquad C_{0,\ell}=C_S(2\pi\kappa)^{-1/2}(2\kappa)^{1/2-c_\ell^*}\Gamma(1/2-c_\ell^*), \tag*{(42)} h → 0 lim ∣ h ∣ 1 − 2 c ℓ ∗ p ( h ) = C 0 , ℓ , C 0 , ℓ = C S ( 2 π κ ) − 1/2 ( 2 κ ) 1/2 − c ℓ ∗ Γ ( 1/2 − c ℓ ∗ ) , (42)
and C 0 , ℓ C_{0,\ell} C 0 , ℓ is positive and finite. The global bound above gives a dominating function proportional to r c ℓ ∗ − 3 / 2 exp { − 1 / ( 2 κ r ) } r^{c_\ell^*-3/2}\exp\{-1/(2\kappa r)\} r c ℓ ∗ − 3/2 exp { − 1/ ( 2 κ r )} , which is integrable when c ℓ ∗ < 1 / 2 c_\ell^*<1/2 c ℓ ∗ < 1/2 . Set b = b ψ b=b_\psi b = b ψ . The beta-prime density satisfies
f T ( t ) ∼ s ψ b B ( a ψ , b ) t − b − 1 , t → ∞ . (43) f_T(t)\sim\frac{s_\psi^b}{B(a_\psi,b)}t^{-b-1},\qquad t\to\infty. \tag*{(43)} f T ( t ) ∼ B ( a ψ , b ) s ψ b t − b − 1 , t → ∞. (43)
Since 0 < Φ < 1 0<\Phi<1 0 < Φ < 1 and E ( Φ b ) < ∞ \mathbb{E}(\Phi^b)<\infty E ( Φ b ) < ∞ , dominated convergence in the product density gives
f S ( s ) ∼ D s − b − 1 , D = s ψ b B ( a ψ , b ) E ( Φ b ) . (44) f_S(s)\sim Ds^{-b-1},\qquad D=\frac{s_\psi^b}{B(a_\psi,b)}\mathbb{E}(\Phi^b). \tag*{(44)} f S ( s ) ∼ D s − b − 1 , D = B ( a ψ , b ) s ψ b E ( Φ b ) . (44)
After multiplication by ∣ h ∣ 2 b + 1 |h|^{2b+1} ∣ h ∣ 2 b + 1 , the global tail bound dominates the integrand in Equation (41) by a constant multiple of r − b − 3 / 2 exp { − 1 / ( 2 κ r ) } r^{-b-3/2}\exp\{-1/(2\kappa r)\} r − b − 3/2 exp { − 1/ ( 2 κ r )} , which is integrable for b > 0 b>0 b > 0 . Dominated convergence in Equation (41) gives
lim ∣ h ∣ → ∞ ∣ h ∣ 2 b + 1 p ( h ) = K ℓ , K ℓ = ( 2 κ s ψ ) b Γ ( b + 1 / 2 ) π B ( a ψ , b ) B ( α + b , β ) B ( α , β ) , (45) \lim_{|h|\to\infty}|h|^{2b+1}p(h)=K_\ell,\qquad K_\ell=\frac{(2\kappa s_\psi)^b\Gamma(b+1/2)}{\sqrt{\pi}B(a_\psi,b)}\frac{B(\alpha+b,\beta)}{B(\alpha,\beta)}, \tag*{(45)} ∣ h ∣ → ∞ lim ∣ h ∣ 2 b + 1 p ( h ) = K ℓ , K ℓ = π B ( a ψ , b ) ( 2 κ s ψ ) b Γ ( b + 1/2 ) B ( α , β ) B ( α + b , β ) , (45)
where K ℓ K_\ell K ℓ is positive and finite.
Heterogeneity Regions
We proceed to bound the number of regions that express heterogeneity.
Proposition 3 (Number of regions expressing heterogeneity). Let every leaf of T j T_j T j contain between 1 and n n n treated observations. Under the Dirichlet-Tree prior,
N j , q ≤ ( 1 − q ) n ω ψ j 2 q (46) N_{j,q} \le\frac{(1-q)n\omega\psi_j^2}{q} \tag*{(46)} N j , q ≤ q ( 1 − q ) nω ψ j 2 (46) for every ψ j 2 > 0 \psi_j^2 > 0 ψ j 2 > 0 and ϕ j \phi_j ϕ j .
Proof. For each leaf counted by N j , q N_{j,q} N j , q ,
n j ℓ ω ψ j 2 ϕ j ℓ ≥ q 1 − q . (47) n_{j\ell}\omega\psi_j^2\phi_{j\ell} \ge\frac{q}{1-q}. \tag*{(47)} n j ℓ ω ψ j 2 ϕ j ℓ ≥ 1 − q q . (47)
The bound n j ℓ ≤ n n_{j\ell} \le n n j ℓ ≤ n implies
ϕ j ℓ ≥ c q , (48) \phi_{j\ell} \ge c_q, \tag*{(48)} ϕ j ℓ ≥ c q , (48)
where c q = q ( 1 − q ) n ω ψ j 2 c_q = \frac{q}{(1-q)n\omega\psi_j^2} c q = ( 1 − q ) nω ψ j 2 q . Since ∑ ℓ ϕ j ℓ = 1 \sum_{\ell}\phi_{j\ell}=1 ∑ ℓ ϕ j ℓ = 1 ,
N j , q c q ≤ ∑ ℓ : κ j ℓ ≤ 1 − q ϕ j ℓ ≤ 1. (49) N_{j,q}c_q \le\sum_{\ell:\,\kappa_{j\ell}\le1-q}\phi_{j\ell} \le1. \tag*{(49)} N j , q c q ≤ ℓ : κ j ℓ ≤ 1 − q ∑ ϕ j ℓ ≤ 1. (49)
Rearranging the last inequality gives the claimed bound. □
Comparison with Independent Local Scales. Let λ j ℓ ≥ 0 \lambda_{j\ell} \ge0 λ j ℓ ≥ 0 be i.i.d. local scales, independent of T j T_j T j and ψ j 2 \psi_j^2 ψ j 2 , whose common law has unbounded support and does not depend on L j L_j L j . Replace the Dirichlet-Tree variance σ 2 ω ψ j 2 ϕ j ℓ \sigma^2\omega\psi_j^2\phi_{j\ell} σ 2 ω ψ j 2 ϕ j ℓ by σ 2 ω ψ j 2 λ j ℓ 2 \sigma^2\omega\psi_j^2\lambda_{j\ell}^2 σ 2 ω ψ j 2 λ j ℓ 2 . Conditional on T j T_j T j and ψ j 2 \psi_j^2 ψ j 2 , define
κ j ℓ l o c = ( 1 + n j ℓ ω ψ j 2 λ j ℓ 2 ) − 1 , N j , q l o c = # { ℓ : κ j ℓ l o c ≤ 1 − q } . (50) \kappa_{j\ell}^{\mathrm{loc}}=\left(1+n_{j\ell}\omega\psi_j^2\lambda_{j\ell}^2\right)^{-1}, \qquad N_{j,q}^{\mathrm{loc}}=\#\{\ell:\kappa_{j\ell}^{\mathrm{loc}}\le1-q\}. \tag*{(50)} κ j ℓ loc = ( 1 + n j ℓ ω ψ j 2 λ j ℓ 2 ) − 1 , N j , q loc = # { ℓ : κ j ℓ loc ≤ 1 − q } . (50)
Let
p q ( ψ j 2 ) = Pr { λ j ℓ 2 ≥ q ( 1 − q ) ω ψ j 2 } . (51) p_q(\psi_j^2)=\Pr\left\{\lambda_{j\ell}^2\ge\frac{q}{(1-q)\omega\psi_j^2}\right\}. \tag*{(51)} p q ( ψ j 2 ) = Pr { λ j ℓ 2 ≥ ( 1 − q ) ω ψ j 2 q } . (51)
The unbounded support of the local-scale distribution implies p q ( ψ j 2 ) > 0 p_q(\psi_j^2)>0 p q ( ψ j 2 ) > 0 . Moreover, since n j ℓ ≥ 1 n_{j\ell}\ge1 n j ℓ ≥ 1 , we have
{ λ j ℓ 2 ≥ q ( 1 − q ) ω ψ j 2 } ⊆ { κ j ℓ l o c ≤ 1 − q } . (52) \left\{\lambda_{j\ell}^2\ge\frac{q}{(1-q)\omega\psi_j^2}\right\}\subseteq\left\{\kappa_{j\ell}^{\mathrm{loc}}\le1-q\right\}. \tag*{(52)} { λ j ℓ 2 ≥ ( 1 − q ) ω ψ j 2 q } ⊆ { κ j ℓ loc ≤ 1 − q } . (52)
Consequently,
E ( N j , q l o c ∣ T j , ψ j 2 ) = ∑ ℓ = 1 L j Pr ( κ j ℓ l o c ≤ 1 − q ∣ T j , ψ j 2 ) ≥ L j p q ( ψ j 2 ) . (53) \mathbb{E}\left(N_{j,q}^{\mathrm{loc}}\mid T_j,\psi_j^2\right)=\sum_{\ell=1}^{L_j}\Pr\left(\kappa_{j\ell}^{\mathrm{loc}}\le1-q\mid T_j,\psi_j^2\right)\ge L_jp_q(\psi_j^2). \tag*{(53)} E ( N j , q loc ∣ T j , ψ j 2 ) = ℓ = 1 ∑ L j Pr ( κ j ℓ loc ≤ 1 − q ∣ T j , ψ j 2 ) ≥ L j p q ( ψ j 2 ) . (53)
Since the local-scale distribution does not depend on L j L_j L j , neither does p q ( ψ j 2 ) p_q(\psi_j^2) p q ( ψ j 2 ) . Thus, the expected number of regions satisfying the criterion grows at least linearly with L j L_j L j .
For example, consider standard half-Cauchy local scales, as in the horseshoe prior [19 ] . In this case,
p q ( ψ j 2 ) = 1 − 2 π arctan { q ( 1 − q ) ω ψ j 2 } 1 / 2 . (54) p_q(\psi_j^2)=1-\frac{2}{\pi}\arctan\left\{\frac{q}{(1-q)\omega\psi_j^2}\right\}^{1/2}. \tag*{(54)} p q ( ψ j 2 ) = 1 − π 2 arctan { ( 1 − q ) ω ψ j 2 q } 1/2 . (54)
Substitution into the preceding bound gives
E ( N j , q l o c ∣ T j , ψ j 2 ) ≥ L j [ 1 − 2 π arctan { q ( 1 − q ) ω ψ j 2 } 1 / 2 ] . (55) \mathbb{E}\left(N^{\mathrm{loc}}_{j,q}\mid T_j,\psi_j^2\right)\ge L_j\left[1-\frac{2}{\pi}\arctan\left\{\frac{q}{(1-q)\omega\psi_j^2}\right\}^{1/2}\right]. \tag*{(55)} E ( N j , q loc ∣ T j , ψ j 2 ) ≥ L j 1 − π 2 arctan { ( 1 − q ) ω ψ j 2 q } 1/2 . (55)
For fixed q q q , ω \omega ω , and ψ j 2 > 0 \psi_j^2>0 ψ j 2 > 0 , the term in brackets is strictly positive and does not depend on L j L_j L j . Thus, the expected number of regions satisfying the criterion grows at least linearly with the number of leaves. In contrast, Proposition 3 bounds the count under the Dirichlet-Tree prior independently of L j L_j L j .
Effective Allocation Size
We show how to derive the bounds presented in the main text for the effective allocation size A e f f ( T j ) A_{\mathrm{eff}}(T_j) A eff ( T j ) . Under the weighted allocation,
E ( ∑ ℓ ϕ j ℓ 2 ∣ T j ) = a ∑ ℓ w ℓ 2 + W j W j ( a W j + 1 ) . (56) \mathbb{E}\left(\sum_{\ell}\phi_{j\ell}^2\mid T_j\right)=\frac{a\sum_{\ell}w_{\ell}^2+W_j}{W_j(aW_j+1)}. \tag*{(56)} E ( ℓ ∑ ϕ j ℓ 2 ∣ T j ) = W j ( a W j + 1 ) a ∑ ℓ w ℓ 2 + W j . (56)
For ρ = 1 / 2 \rho=1/2 ρ = 1/2 , splitting a leaf of weight 2 − d 2^{-d} 2 − d replaces it by two leaves of weight 2 − ( d + 1 ) 2^{-(d+1)} 2 − ( d + 1 ) , so W j = 1 W_j=1 W j = 1 for every finite binary tree. If the tree has at least two leaves, then 0 < ∑ ℓ w ℓ 2 ≤ ( max ℓ w ℓ ) ∑ ℓ w ℓ ≤ 1 / 2 0<\sum_{\ell}w_{\ell}^2\leq(\max_{\ell}w_{\ell})\sum_{\ell}w_{\ell}\leq1/2 0 < ∑ ℓ w ℓ 2 ≤ ( max ℓ w ℓ ) ∑ ℓ w ℓ ≤ 1/2 . The preceding bounds give
1 + a 1 + a / 2 ≤ A e f f ( T j ) = 1 + a 1 + a ∑ ℓ w ℓ 2 < 1 + a . (57) \frac{1+a}{1+a/2}\leq A_{\mathrm{eff}}(T_j)=\frac{1+a}{1+a\sum_{\ell}w_{\ell}^2}<1+a. \tag*{(57)} 1 + a /2 1 + a ≤ A eff ( T j ) = 1 + a ∑ ℓ w ℓ 2 1 + a < 1 + a . (57)
The lower bound is attained by the two-leaf tree. The upper bound is not attained by a finite tree, but complete trees approach it as their depth tends to infinity.
Posterior Consistency
The proof follows the classical Kullback–Leibler support argument of Schwartz [64 ] and Ghosal et al. [30 ] ; see also Ghosal et al. [30 ] for a broader treatment of posterior convergence. We first establish that the prior assigns positive probability to uniform neighborhoods of the true prognostic and treatment-effect surfaces. We then show that these neighborhoods are contained in arbitrarily small Kullback–Leibler neighborhoods of the true observable law. Weak posterior consistency follows from Schwartz’s theorem.
Consider a fixed number p p p of covariates, x ∼ Q \boldsymbol{x}\sim Q x ∼ Q on a compact rectangle X ⊂ R p \mathcal{X}\subset\mathbb{R}^{p} X ⊂ R p , A ∣ x ∼ Bernoulli { e 0 ( x ) } A\mid\boldsymbol{x}\sim\operatorname{Bernoulli}\{e_0(\boldsymbol{x})\} A ∣ x ∼ Bernoulli { e 0 ( x )} , and
y ∣ x , A ∼ N ( μ 0 ( x ) + A τ 0 ( x ) , σ 0 2 ) . (58) y\mid\boldsymbol{x},A\sim\mathcal{N}\left(\mu_0(\boldsymbol{x})+A\tau_0(\boldsymbol{x}),\sigma_0^2\right). \tag*{(58)} y ∣ x , A ∼ N ( μ 0 ( x ) + A τ 0 ( x ) , σ 0 2 ) . (58)
The fitted model uses the true propensity e 0 e_0 e 0 , and τ ( x ) = α + g τ ( x ) \tau(\boldsymbol{x})=\alpha+g_{\tau}(\boldsymbol{x}) τ ( x ) = α + g τ ( x ) denotes its complete treatment-effect surface.
Conditions. Assume the following:
(i) The surfaces μ 0 \mu_0 μ 0 and τ 0 \tau_0 τ 0 are continuous on X \mathcal{X} X , σ 0 > 0 \sigma_0>0 σ 0 > 0 , and ϵ ≤ e 0 ( x ) ≤ 1 − ϵ \epsilon\leq e_0(\boldsymbol{x})\leq1-\epsilon ϵ ≤ e 0 ( x ) ≤ 1 − ϵ for some ϵ ∈ ( 0 , 1 / 2 ] \epsilon\in(0,1/2] ϵ ∈ ( 0 , 1/2 ] and Q Q Q -almost every x \boldsymbol{x} x .
(ii) Each forest contains a fixed positive number of trees. The topology prior assigns positive mass to every finite binary topology, every covariate has positive splitting probability, and each conditional cutpoint law has full support on the interior of its coordinate range. (iii) Treatment-tree branch fractions are independent beta variables with positive parameters. The tree scale is independent of the branch fractions and has a positive density on ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) , and ω > 0 \omega> 0 ω > 0 . Conditional on a prognostic topology and any σ > 0 \sigma> 0 σ > 0 , the prognostic terminal-value density is positive everywhere on its Euclidean space.
(iv) Conditional on σ \sigma σ , the tree-specific states have product form across the fixed collection of trees. The conditional density of α \alpha α is positive in a neighborhood of zero, the prior for σ \sigma σ assigns positive mass to every neighborhood of σ 0 \sigma_0 σ 0 , and the induced joint densities are measurable.
Theorem 1 (Weak consistency). Under these conditions, the prior of D T DT D T -BCF assigns positive probability to every Kullback–Leibler neighborhood of the law P 0 P_0 P 0 of ( x , A , y ) (x,A,y) ( x , A , y ) . Moreover, for every weak neighborhood W \mathcal{W} W of P 0 P_0 P 0 ,
Π ( P μ , τ , σ ∉ W ∣ D n ) ⟶ 0 P 0 ∞ -almost surely . (59) \Pi(P_{\mu,\tau,\sigma} \notin\mathcal{W} \mid D_n) \longrightarrow0 \quad P_0^\infty\text{-almost surely}. \tag*{(59)} Π ( P μ , τ , σ ∈ / W ∣ D n ) ⟶ 0 P 0 ∞ -almost surely . (59) If P μ , τ , σ = P 0 P_{\mu,\tau,\sigma}=P_0 P μ , τ , σ = P 0 , then σ = σ 0 \sigma=\sigma_0 σ = σ 0 , μ { x , e 0 ( x ) } = μ 0 ( x ) \mu\{x,e_0(x)\}=\mu_0(x) μ { x , e 0 ( x )} = μ 0 ( x ) , and τ ( x ) = τ 0 ( x ) \tau(x)=\tau_0(x) τ ( x ) = τ 0 ( x ) for Q Q Q -almost every x x x .
Proof. In the following let μ ~ ( x ) = μ { x , e 0 ( x ) } \widetilde{\mu}(x)=\mu\{x,e_0(x)\} μ ( x ) = μ { x , e 0 ( x )} . Consider a finite treatment tree T T T with L L L leaves. For t = ψ j 2 > 0 t=\psi_j^2>0 t = ψ j 2 > 0 , σ > 0 \sigma>0 σ > 0 , and a realized allocation ϕ \phi ϕ , the covariance matrix of its terminal values is
Σ T ( t , ϕ , σ ) = σ 2 ω t diag ( ϕ ) . (60) \Sigma_T(t,\phi,\sigma)=\sigma^2\omega t\operatorname{diag}(\phi). \tag*{(60)} Σ T ( t , ϕ , σ ) = σ 2 ω t diag ( ϕ ) . (60)
For L ≥ 2 L\ge2 L ≥ 2 , positivity of the beta branch parameters gives ϕ ℓ ∈ ( 0 , 1 ) \phi_\ell\in(0,1) ϕ ℓ ∈ ( 0 , 1 ) almost surely for every leaf. When L = 1 L=1 L = 1 , ϕ 1 = 1 \phi_1=1 ϕ 1 = 1 . In either case, Σ T ( t , ϕ , σ ) \Sigma_T(t,\phi,\sigma) Σ T ( t , ϕ , σ ) is positive definite.
Let h = ( h 1 , … , h L ) ⊤ \mathbf{h}=(h_1,\ldots,h_L)^\top h = ( h 1 , … , h L ) ⊤ denote the vector of terminal values. Conditional on t t t , ϕ \phi ϕ , and σ \sigma σ , h ∼ N L { 0 , Σ T ( t , ϕ , σ ) } \mathbf{h}\sim N_L\{0,\Sigma_T(t,\phi,\sigma)\} h ∼ N L { 0 , Σ T ( t , ϕ , σ )} . Since Σ T ( t , ϕ , σ ) \Sigma_T(t,\phi,\sigma) Σ T ( t , ϕ , σ ) is positive definite, the conditional density of h \mathbf{h} h is strictly positive on R L \mathbb{R}^L R L . Hence, for every h 0 ∈ R L \mathbf{h}_0\in\mathbb{R}^L h 0 ∈ R L and η > 0 \eta>0 η > 0 ,
Π ( ∥ h − h 0 ∥ ∞ < η ∣ T , σ ) > 0. (61) \Pi(\lVert\mathbf{h}-\mathbf{h}_0\rVert_\infty<\eta\mid T,\sigma)>0. \tag*{(61)} Π (∥ h − h 0 ∥ ∞ < η ∣ T , σ ) > 0. (61)
Assumption (iii) gives the same conclusion for a prognostic tree.
Fix f ∈ C ( X ) f\in C(\mathcal{X}) f ∈ C ( X ) and η > 0 \eta>0 η > 0 . Since X \mathcal{X} X is compact, f f f is uniformly continuous. Hence, there exists δ > 0 \delta>0 δ > 0 such that
∥ x − x ′ ∥ ∞ < δ ⟹ ∣ f ( x ) − f ( x ′ ) ∣ < η 2 . (62) \lVert x-x'\rVert_\infty<\delta\quad\Longrightarrow\quad|f(x)-f(x')|<\frac{\eta}{2}. \tag*{(62)} ∥ x − x ′ ∥ ∞ < δ ⟹ ∣ f ( x ) − f ( x ′ ) ∣ < 2 η . (62)
Choose a finite axis-aligned recursive partition { R k } k = 1 K \{R_k\}_{k=1}^K { R k } k = 1 K whose cells have diameter smaller than δ / 2 \delta/2 δ /2 , and select x k ∈ R k \mathbf{x}_k\in R_k x k ∈ R k for each cell. Let T f T_f T f be a binary tree representing this partition, with terminal value f ( x k ) f(\mathbf{x}_k) f ( x k ) assigned to leaf R k R_k R k .
Since T f T_f T f contains finitely many cutpoints, we can chose sufficiently small open neighborhoods of these cutpoints such that every perturbed partition has corresponding leaves { R k ′ } k = 1 K \{R'_k\}_{k=1}^K { R k ′ } k = 1 K satisfying
sup x ∈ R k ′ ∥ x − x k ∥ ∞ < δ , k = 1 , … , K . (63) \sup_{\mathbf{x}\in R'_k}\lVert\mathbf{x}-\mathbf{x}_k\rVert_\infty<\delta,\qquad k=1,\ldots,K. \tag*{(63)} x ∈ R k ′ sup ∥ x − x k ∥ ∞ < δ , k = 1 , … , K . (63)
It follows from the choice of δ \delta δ that
sup x ∈ R k ′ ∣ f ( x ) − f ( x k ) ∣ < η 2 , k = 1 , … , K . (64) \sup_{\mathbf{x}\in R'_k}|f(\mathbf{x})-f(\mathbf{x}_k)|<\frac{\eta}{2},\qquad k=1,\ldots,K. \tag*{(64)} x ∈ R k ′ sup ∣ f ( x ) − f ( x k ) ∣ < 2 η , k = 1 , … , K . (64)
Assumption (ii) assigns positive probability to the topology of T f T_f T f , its split variables, and the specified cutpoint neighborhoods. Conditional on this event, (61) assigns positive probability to
max 1 ≤ k ≤ K ∣ h k − f ( x k ) ∣ < η 2 . (65) \max_{1\le k\le K}|h_k-f(\mathbf{x}_k)|<\frac{\eta}{2}. \tag*{(65)} 1 ≤ k ≤ K max ∣ h k − f ( x k ) ∣ < 2 η . (65)
For every x ∈ R k l x \in R_k^l x ∈ R k l , the triangle inequality gives
∣ g ( x ) − f ( x ) ∣ ≤ ∣ h k − f ( x k ) ∣ + ∣ f ( x k ) − f ( x ) ∣ < η . (66) |g(x)-f(x)| \leq|h_k-f(x_k)|+|f(x_k)-f(x)|<\eta. \tag*{(66)} ∣ g ( x ) − f ( x ) ∣ ≤ ∣ h k − f ( x k ) ∣ + ∣ f ( x k ) − f ( x ) ∣ < η . (66)
Consequently,
Π ( ∥ g − f ∥ ∞ < η ∣ σ ) > 0. (67) \Pi(\lVert g-f\rVert_\infty<\eta\mid\sigma)>0. \tag*{(67)} Π (∥ g − f ∥ ∞ < η ∣ σ ) > 0. (67)
Apply Equation (67) to one tree in each forest. In the prognostic forest, the approximating tree can split only on the original covariates, so continuity of e 0 e_0 e 0 is not required. Assign zero target values to the remaining trees and to α \alpha α . The product-support conditions in assumptions (iii)–(iv), together with the prior support for σ \sigma σ , imply that every δ > 0 \delta>0 δ > 0 has positive prior probability of the event
{ ∥ μ ~ − μ 0 ∥ ∞ < δ , ∥ τ − τ 0 ∥ ∞ < δ , ∣ σ − σ 0 ∣ < δ } . (68) \{\lVert\tilde{\mu}-\mu_0\rVert_\infty<\delta,\quad\lVert\tau-\tau_0\rVert_\infty<\delta,\quad|\sigma-\sigma_0|<\delta\}. \tag*{(68)} {∥ μ ~ − μ 0 ∥ ∞ < δ , ∥ τ − τ 0 ∥ ∞ < δ , ∣ σ − σ 0 ∣ < δ } . (68)
Relative to the product of Q Q Q , counting measure on { 0 , 1 } \{0,1\} { 0 , 1 } , and Lebesgue measure, the Kullback–Leibler divergence is
K ( P 0 , P μ ~ , τ , σ ) = log σ σ 0 + σ 0 2 2 σ 2 − 1 2 + 1 2 σ 2 ∫ X [ ( 1 − e 0 ) ( μ ~ − μ 0 ) 2 + e 0 { μ ~ + τ − μ 0 − τ 0 } 2 ] d Q , (69) \begin{aligned}
K(P_0,P_{\tilde{\mu},\tau,\sigma})={}&\log\frac{\sigma}{\sigma_0}+\frac{\sigma_0^2}{2\sigma^2}-\frac{1}{2}\\
&+\frac{1}{2\sigma^2}\int_{\mathcal{X}}\left[(1-e_0)(\tilde{\mu}-\mu_0)^2+e_0\{\tilde{\mu}+\tau-\mu_0-\tau_0\}^2\right]\,dQ, \tag*{(69)}
\end{aligned} K ( P 0 , P μ ~ , τ , σ ) = log σ 0 σ + 2 σ 2 σ 0 2 − 2 1 + 2 σ 2 1 ∫ X [ ( 1 − e 0 ) ( μ ~ − μ 0 ) 2 + e 0 { μ ~ + τ − μ 0 − τ 0 } 2 ] d Q , (69)
with every function in the integrand evaluated at x x x . On the event in Equation (68) ,
( 1 − e 0 ) ( μ ~ − μ 0 ) 2 + e 0 ( μ ~ + τ − μ 0 − τ 0 ) 2 ≤ 4 δ 2 . (70) (1-e_0)(\tilde{\mu}-\mu_0)^2+e_0(\tilde{\mu}+\tau-\mu_0-\tau_0)^2\leq4\delta^2. \tag*{(70)} ( 1 − e 0 ) ( μ ~ − μ 0 ) 2 + e 0 ( μ ~ + τ − μ 0 − τ 0 ) 2 ≤ 4 δ 2 . (70)
For σ ≥ σ 0 / 2 \sigma\geq\sigma_0/2 σ ≥ σ 0 /2 , this yields
K ( P 0 , P μ ~ , τ , σ ) ≤ r ( σ ) + 8 δ 2 σ 0 2 , r ( σ ) = log σ σ 0 + σ 0 2 2 σ 2 − 1 2 . (71) K(P_0,P_{\tilde{\mu},\tau,\sigma})\leq r(\sigma)+\frac{8\delta^2}{\sigma_0^2},\qquad r(\sigma)=\log\frac{\sigma}{\sigma_0}+\frac{\sigma_0^2}{2\sigma^2}-\frac{1}{2}. \tag*{(71)} K ( P 0 , P μ ~ , τ , σ ) ≤ r ( σ ) + σ 0 2 8 δ 2 , r ( σ ) = log σ 0 σ + 2 σ 2 σ 0 2 − 2 1 . (71)
The function r r r is continuous at σ 0 \sigma_0 σ 0 and r ( σ 0 ) = 0 r(\sigma_0)=0 r ( σ 0 ) = 0 . Given ϵ > 0 \epsilon>0 ϵ > 0 , choose δ < σ 0 / 2 \delta<\sigma_0/2 δ < σ 0 /2 so that 8 δ 2 / σ 0 2 < ϵ / 2 8\delta^2/\sigma_0^2<\epsilon/2 8 δ 2 / σ 0 2 < ϵ /2 , and restrict the σ \sigma σ -neighborhood further so that r ( σ ) < ϵ / 2 r(\sigma)<\epsilon/2 r ( σ ) < ϵ /2 . Equations (68) and (71) give
Π ( P : K ( P 0 , P ) < ϵ ) > 0. (72) \Pi(P:K(P_0,P)<\epsilon)>0. \tag*{(72)} Π ( P : K ( P 0 , P ) < ϵ ) > 0. (72)
The prior has Kullback–Leibler support at P 0 P_0 P 0 . Schwartz’s theorem (Schwartz, [64 ] ) establishes weak consistency.
It remains to verify the identification statement. If P μ ~ , τ , σ = P 0 P_{\tilde{\mu},\tau,\sigma}=P_0 P μ ~ , τ , σ = P 0 , overlap gives equality of the conditional Gaussian laws at both treatment levels for Q Q Q -almost every x x x . The equality at A = 0 A=0 A = 0 gives σ = σ 0 \sigma=\sigma_0 σ = σ 0 and μ ~ = μ 0 \tilde{\mu}=\mu_0 μ ~ = μ 0 . The equality at A = 1 A=1 A = 1 then gives μ ~ + τ = μ 0 + τ 0 \tilde{\mu}+\tau=\mu_0+\tau_0 μ ~ + τ = μ 0 + τ 0 , and hence τ = τ 0 \tau=\tau_0 τ = τ 0 Q Q Q -almost everywhere.
Experimental details and further results
This appendix first gives the settings that the studies of § 5 share. The design and the complete results of each study follow, in the order of that section.
Common setup
Models. We consider the six models of § 5.1 whose ω \omega ω is matched run in our sampler. They share the BCF model (2) and, apart from DART, differ only in the prior variance V j ℓ V_{j\ell} V j ℓ of the leaf value h j ℓ h_{j\ell} h j ℓ of a treatment tree. § 5.1 defines DT-BCF, its variants and Fixed. HS sets V j ℓ = σ 2 ω ψ j ℓ 2 V_{j\ell} = \sigma^{2}\omega\psi_{j\ell}^{2} V j ℓ = σ 2 ω ψ j ℓ 2 , where the local scales λ j ℓ \lambda_{j\ell} λ j ℓ are independent and half-Cauchy. Its tree scale has the prior ψ j 2 ∼ BetaPrime ( 1 / 2 , 1 / 2 , 1 ) \psi_{j}^{2} \sim\operatorname{BetaPrime}(1/2,1/2,1) ψ j 2 ∼ BetaPrime ( 1/2 , 1/2 , 1 ) of DT-BCF, under which ψ j \psi_{j} ψ j is also half-Cauchy. In the application, HS is instead the Horseshoe Forest of Jacobs et al. [38 ] . It is fitted with the function CausalHorseForest of the authors’ package at its defaults, including the value k = 1.5 k = 1.5 k = 1.5 of the parameter that sets its prior width, with e ^ i \hat{e}_{i} e ^ i added to the covariates of its prognostic forest. We do not match its prior to ς \varsigma ς . The models marked + DART place the prior s ∼ Dirichlet ( θ / q , … , θ / q ) \boldsymbol{s} \sim\operatorname{Dirichlet}(\theta/q,\ldots,\theta/q) s ∼ Dirichlet ( θ / q , … , θ / q ) of Linero [47 ] on the split probabilities of each forest, where q q q is the number of covariates of that forest and θ = 1 \theta= 1 θ = 1 is held fixed. HS + DART and Equal + DART, which appear in Appendix § 12 .§ 3 , are defined in the same way as DT-BCF + DART.
Prior Settings. We use m τ = 50 m_{\tau} = 50 m τ = 50 treatment trees with split probability 0.95 ( 1 + d ) − η 0.95(1 + d)^{-\eta} 0.95 ( 1 + d ) − η , where η = 2 \eta= 2 η = 2 outside the tree-size study, and m μ = 200 m_{\mu} = 200 m μ = 200 prognostic trees with split probability 0.95 ( 1 + d ) − 2 0.95(1 + d)^{-2} 0.95 ( 1 + d ) − 2 . The prognostic leaf values have variance σ 2 ω μ ψ μ 2 \sigma^{2}\omega_{\mu}\psi_{\mu}^{2} σ 2 ω μ ψ μ 2 , with one scale ψ μ 2 ∼ BetaPrime ( 1 / 2 , 1 / 2 , 1 ) \psi_{\mu}^{2} \sim\operatorname{BetaPrime}(1/2,1/2,1) ψ μ 2 ∼ BetaPrime ( 1/2 , 1/2 , 1 ) for the whole forest and ω μ = 1 / m μ \omega_{\mu} = 1/m_{\mu} ω μ = 1/ m μ , so that given ψ μ 2 \psi_{\mu}^{2} ψ μ 2 the prior variance of μ ( x , e ^ ) \mu(x,\hat{e}) μ ( x , e ^ ) at any covariate value is σ 2 ψ μ 2 \sigma^{2}\psi_{\mu}^{2} σ 2 ψ μ 2 . As in BCF, e ^ i \hat{e}_{i} e ^ i enters μ \mu μ as a covariate.
The outcome is centered at its sample mean. The intercept has the prior α ∼ N ( 0 , 100 σ 2 ) \alpha\sim\mathcal{N}(0,100\sigma^{2}) α ∼ N ( 0 , 100 σ 2 ) . The residual variance has an inverse-gamma prior with shape 3 and scale v v v , so its prior mean is v / 2 v/2 v /2 . The value of v v v is the sample variance of y y y in the forest simulations, the residual variance of a lasso regression of y y y on ( A i , x i ) (A_{i},x_{i}) ( A i , x i ) with a cross-validated penalty in ACIC 2016 (Friedman et al. [26 ] ), and the squared scale of an intercept-only Gaussian accelerated failure time model in the application (Therneau, [68 ] ). In every reported fit the leaf variances carry the factor σ 2 \sigma^{2} σ 2 , as in § 3 .
Specification of ω \omega ω . § 3.5 sets ω \omega ω so that the prior median of S τ / σ S_{\tau}/\sigma S τ / σ equals ς = 1 / 2 \varsigma= 1/2 ς = 1/2 . We compute this value of ω \omega ω by simulation with σ = 1 \sigma= 1 σ = 1 . Each of 1,000 prior draws grows m τ m_{\tau} m τ treatment trees from the tree prior, in which a node splits with the split probability of its depth. The split variable is drawn uniformly, and the cut is the midpoint of the range of that variable among the units in the node. The shares, tree scales and leaf values are then drawn from their priors, which gives one draw of S τ S_{\tau} S τ over the units of a reference design. Because S τ S_{\tau} S τ is proportional to ω 1 / 2 \omega^{1/2} ω 1/2 , a run at ω 0 = 1 / m τ \omega_{0} = 1/m_{\tau} ω 0 = 1/ m τ whose median is M 0 M_{0} M 0 gives ω = ω 0 ( ς / M 0 ) 2 \omega= \omega_{0}(\varsigma/M_{0})^{2} ω = ω 0 ( ς / M 0 ) 2 . A second, independent run of 1,000 draws at this ω \omega ω checks the median. The reference design is a sample of 500 units from U ( 0 , 1 ) 5 U(0,1)^{5} U ( 0 , 1 ) 5 in the forest simulations, the 4,802 units of the competition in ACIC 2016, and the 130 patients in the application. Each model is matched separately, and in the tree-size study separately at each η \eta η .
BCF. BCF is fitted with the bcf package of Hahn et al. [34 ] when p < n p < n p < n , which covers the tree-size study, the dimension study at p ∈ { 20 , 50 } p \in\{20,50\} p ∈ { 20 , 50 } and ACIC 2016. The package runs at its defaults, with the prior of Appendix § 7 .§ 2 , and receives the same propensity score as the other models. Because the package does not run when p ≥ n p \ge n p ≥ n , BCF is then fitted in our sampler. This version keeps the numbers of trees and the split probabilities of the package, with a half-normal prior on the scale of the treatment forest and a half-Cauchy prior on the scale of the prognostic forest. In our sampler the treatment indicator enters with the fixed coding 0 and 1, without the coefficients b 0 b_0 b 0 and b 1 b_1 b 1 of Appendix § 7 .§ 2 . Its ω \omega ω is matched to ς = 1 \varsigma= 1 ς = 1 , which approximates the width of the package’s default prior for the treatment forest. This is the only version of BCF whose prior is matched. On the fixed partition, BCF denotes its leaf prior on the given tree, with one half-normal scale shared by all leaves (Appendix § 12 .§ 2 ).
Loss-based Models. The X-learner (Künzel et al., [45 ] ) and the R-learner (Nie and Wager, [53 ] ) are fitted with the functions xboost and rboost of the rlearner package, which estimate each regression of Appendix § 8 by gradient-boosted trees (Chen and Guestrin, [20 ] ). Each regression is tuned over ten random configurations of the learning rate, tree depth, subsampling and regularization parameters. Every configuration is evaluated by ten-fold cross-validation with up to 1,000 boosting rounds and early stopping after ten rounds without improvement. The best configuration is then refitted to the full sample of that regression. The outcome regression of the R-learner is cross-fitted over the same ten folds. Both learners otherwise run at the package defaults and receive the same propensity score as the other models. They return point estimates, so coverage and interval length are not reported for them. The learners are not fitted in the application.
Propensity Scores. In the forest simulations every model, including the loss-based ones, receives the true propensity score, e ^ i = e ( x i ) \hat e_i = e(x_i) e ^ i = e ( x i ) . In ACIC 2016 and in the application every model receives the same estimate, from a five-fold cross-fitted logistic lasso (Friedman et al., [26 ] ) whose penalty minimizes the deviance of an inner five-fold cross-validation. The estimate is computed once per data set and truncated to [ 10 − 4 , 1 − 10 − 4 ] [10^{-4}, 1 - 10^{-4}] [ 1 0 − 4 , 1 − 1 0 − 4 ] .
Metrics. Let δ i = τ ( x i ) − n − 1 ∑ i ′ = 1 n τ ( x i ′ ) \delta_i = \tau(x_i) - n^{-1}\sum_{i'=1}^{n}\tau(x_{i'}) δ i = τ ( x i ) − n − 1 ∑ i ′ = 1 n τ ( x i ′ ) be the centered effect of unit i i i in a data set of size n n n . For the Bayesian models its estimate δ ^ i \hat\delta_i δ ^ i is the posterior mean of the same quantity computed from each draw of τ \tau τ , and for the loss-based models it is the centered point estimate. The root mean squared error
RMSE = 1 n ∑ i = 1 n ( δ ^ i − δ i ) 2 \operatorname{RMSE} = \sqrt{\frac{1}{n}\sum_{i=1}^{n}(\hat\delta_i-\delta_i)^2} RMSE = n 1 i = 1 ∑ n ( δ ^ i − δ i ) 2
is the precision in estimation of heterogeneous effects of Hill (Hill [36 ] ), applied to the centered effect. The 90% interval for δ i \delta_i δ i runs from the 5% to the 95% quantile of the draws pooled over the chains. Coverage is the proportion of units whose δ i \delta_i δ i lies in its interval, and length is the average width of the intervals. In scenarios where some units have no effect, a unit is reported as having an effect when the 90% interval for τ ( x i ) \tau(x_i) τ ( x i ) excludes zero. The false positive rate (FPR) is the proportion of units with τ ( x i ) = 0 \tau(x_i)=0 τ ( x i ) = 0 that are reported, and the true positive rate (TPR) is the proportion of units with τ ( x i ) ≠ 0 \tau(x_i)\ne0 τ ( x i ) = 0 that are reported. In the forest simulations, predictive accuracy is measured by the log predictive density (lpd) of n t e s t = 500 n_{\mathrm{test}}=500 n test = 500 test units drawn from the same process (Vehtari et al., [71 ] ),
lpd = 1 n t e s t ∑ k = 1 n t e s t log [ 1 S ∑ s = 1 S N { y k ; μ ( s ) ( x k , e k ) + A k τ ( s ) ( x k ) , σ ( s ) 2 } ] , \operatorname{lpd}=\frac{1}{n_{\mathrm{test}}}\sum_{k=1}^{n_{\mathrm{test}}}\log\left[\frac{1}{S}\sum_{s=1}^{S}\mathcal{N}\{y_k;\mu^{(s)}(x_k,e_k)+A_k\tau^{(s)}(x_k),\sigma^{(s)2}\}\right], lpd = n test 1 k = 1 ∑ n test log [ S 1 s = 1 ∑ S N { y k ; μ ( s ) ( x k , e k ) + A k τ ( s ) ( x k ) , σ ( s ) 2 } ] ,
where s s s indexes the S S S posterior draws. In the application, the expected log-pointwise predictive density (elpd) for a new patient is estimated by Pareto-smoothed importance-sampling leave-one-out cross-validation (PSIS-LOO),with its standard error (Vehtari et al., [71 ] ). Its pointwise likelihood is that of the censored model on the log-time scale, the normal density of log Y i \log Y_i log Y i for a patient with an observed event and the normal survival function at log Y i \log Y_i log Y i for a censored patient (Appendix § 10 .8). In the figures, every uncertainty band for a mean over data sets is the mean plus or minus two standard errors. In the figures and tables, the standard error of such a mean is the standard deviation over data sets divided by the square root of their number, computed separately for each model. The ratios of Table 2 are treated in the same way on the log scale (Appendix § 12 .§ 4 ).
Software, Data and Computing. Table 4 lists the software and data with their versions and licenses. Our sampler is written in R ([59 ] ), with C++ kernels compiled through Rcpp ([24 ] ). The ACIC 2016 data sets are generated by the aciccomp2016 package of Dorie et al. ((2019)) from covariates of the Collaborative Perinatal Project. The pancreatic cohort is the TCGA PAAD object curated by pdacR ([69 ] ) from the data of Cancer Genome Atlas Research Network ((2017)), which are open-access data of the Genomic Data Commons of the National Cancer Institute. The fits of the simulations and of ACIC 2016 ran on a shared server with two AMD EPYC 9655 processors (192 cores) and 2.3 TB of memory, under Ubuntu 24.04 and R 4.5.3. The fits of the application ran on that server and on a laptop (MacBook Pro, Apple M5 Pro, 24 GB of memory, macOS, R 4.6.0). The tables and figures were produced on the laptop. Each fit ran in its own R process, with single-threaded linear algebra.
Table 4. Software and Data Used.
Leaf Priors on a Fixed Partition
This study separates the leaf prior from the tree search of the forest. § 5.2 reports its conditions at signal-to-noise ratio 3. This subsection gives its design and its complete results.
Data. The tree is the balanced binary tree with L = 2 D L = 2^{D} L = 2 D leaves, D ∈ { 3 , 4 , 5 , 6 } D \in\{3, 4, 5, 6\} D ∈ { 3 , 4 , 5 , 6 } , held fixed. For each data set, k k k leaves are drawn uniformly without replacement. Half of them receive h ℓ = Δ h_{\ell} = \Delta h ℓ = Δ and half h ℓ = − Δ h_{\ell} = -\Delta h ℓ = − Δ . The other leaves have h ℓ = 0 h_{\ell} = 0 h ℓ = 0 , so the leaf values average to zero. Each leaf holds m = 16 m = 16 m = 16 observations with residual standard deviation σ = 1 \sigma= 1 σ = 1 , and the effect size is Δ = SNR ⋅ σ / m 1 / 2 \Delta= \operatorname{SNR} \cdot\sigma/ m^{1/2} Δ = SNR ⋅ σ / m 1/2 . The signal-to-noise ratio SNR \operatorname{SNR} SNR is therefore Δ \Delta Δ in units of the standard error of a leaf mean. The mean residual of leaf ℓ \ell ℓ is R ℓ = h ℓ + ε ℓ R_{\ell} = h_{\ell} + \varepsilon_{\ell} R ℓ = h ℓ + ε ℓ , with independent ε ℓ ∼ N ( 0 , σ 2 / m ) \varepsilon_{\ell} \sim\mathrm{N}(0, \sigma^{2}/m) ε ℓ ∼ N ( 0 , σ 2 / m ) . The sampler receives the count m m m and the residual sum R ˉ ℓ \bar{R}_{\ell} R ˉ ℓ of each leaf. There are 200 data sets of a condition dependent only on ( L , k , Δ ) (L, k, \Delta) ( L , k , Δ ) and their seeds, so every model and every value of ζ \zeta ζ is fitted to the same data sets. Besides the conditions of Figure 1 , the design varies the signal-to-noise ratio over { 2 , 3 , 4 } \{2, 3, 4\} { 2 , 3 , 4 } at k = 2 k = 2 k = 2 , adds k = 32 k = 32 k = 32 at L = 32 L = 32 L = 32 , where every leaf has an effect, and varies ζ \zeta ζ over { 1 / 4 , 1 / 2 , 1 } \{1/4, 1/2, 1\} { 1/4 , 1/2 , 1 } at L = 32 L = 32 L = 32 , k = 2 k = 2 k = 2 and SNR 3. In all, the design has 17 conditions.
Priors. Given its scales and shares, each leaf value has the prior h ℓ ∼ N ( 0 , V ℓ ) h_\ell\sim\mathrm{N}(0,V_\ell) h ℓ ∼ N ( 0 , V ℓ ) , independently across leaves. DT-BCF and DT-BCF ( ρ = 1 ) (\rho= 1) ( ρ = 1 ) set V ℓ = σ 2 ω ψ 2 ϕ ℓ V_\ell= \sigma^2\omega\psi^2\phi_\ell V ℓ = σ 2 ω ψ 2 ϕ ℓ with ϕ ∼ Dirichlet ( a ρ D , … , a ρ D ) \phi\sim\operatorname{Dirichlet}(a\rho^D,\ldots,a\rho^D) ϕ ∼ Dirichlet ( a ρ D , … , a ρ D ) and a = 1 / 2 a = 1/2 a = 1/2 . Every leaf of a balanced tree has depth D D D , so both values of ρ \rho ρ give a symmetric Dirichlet law, with concentration a / L a/L a / L per leaf at ρ = 1 / 2 \rho= 1/2 ρ = 1/2 and a a a at ρ = 1 \rho= 1 ρ = 1 . Equal sets V ℓ = σ 2 ω ψ 2 / L V_\ell= \sigma^2\omega\psi^2/L V ℓ = σ 2 ω ψ 2 / L . HS sets V ℓ = σ 2 ω ψ 2 λ ℓ 2 V_\ell= \sigma^2\omega\psi^2\lambda_\ell^2 V ℓ = σ 2 ω ψ 2 λ ℓ 2 with independent standard half-Cauchy local scales λ ℓ \lambda_\ell λ ℓ . In these four models ψ 2 ∼ BetaPrime ( 1 / 2 , 1 / 2 , 1 ) \psi^2 \sim\operatorname{BetaPrime}(1/2,1/2,1) ψ 2 ∼ BetaPrime ( 1/2 , 1/2 , 1 ) , so ψ \psi ψ is standard half-Cauchy. BCF sets V ℓ = σ 2 ω s 2 V_\ell= \sigma^2\omega s^2 V ℓ = σ 2 ω s 2 , with one standard half-normal scale s s s shared by all leaves. Fixed sets V ℓ = σ 2 ω V_\ell= \sigma^2\omega V ℓ = σ 2 ω . The sampler runs the updates of Appendix § 10 for the shares, the scales and the leaf values of one tree, without structural moves and with σ \sigma σ known.
Prior Specification. For every model, every L L L and every value of ς \varsigma ς , ω \omega ω is chosen so that the prior median of { L − 1 ∑ ℓ ( h ℓ − h ˉ ) 2 } 1 / 2 \{L^{-1}\sum_\ell(h_\ell-\bar h)^2\}^{1/2} { L − 1 ∑ ℓ ( h ℓ − h ˉ ) 2 } 1/2 equals ς \varsigma ς , where h ˉ \bar h h ˉ is the mean of the leaf values. Because every leaf holds m m m observations, this quantity is the across-unit standard deviation S τ S_\tau S τ of § 3.5 , up to its divisor. The median is estimated from 250,000 prior draws and checked on an independent set of 250,000. Every checked median lies within one percent of its target (Table 7 ). BCF (our version) is matched to the same ς \varsigma ς here, so that the comparison concerns the form of its leaf prior. In the forest experiments it keeps its own prior width (Appendix § 12 .§ 1 ).
Table 7. Fixed Partition. The scaling constant ω \omega ω of each model, chosen by prior-predictive simulation so that the median over prior draws of the standard deviation of the L L L leaf values, with divisor L L L , equals ς \varsigma ς . Each value is solved from 250,000 prior draws and checked on an independent set of 250,000. The lower block gives the relative error of the checked median. No data enter the matching. For Fixed, ω \omega ω is the leaf variance itself.
Table 7.
Losses. With h ^ ℓ \hat h_\ell h ^ ℓ the posterior mean of h ℓ h_\ell h ℓ , let M S E 1 \mathrm{MSE}_1 MSE 1 and M S E 0 \mathrm{MSE}_0 MSE 0 be the means of ( h ^ ℓ − h ℓ ) 2 (\hat h_\ell-h_\ell)^2 ( h ^ ℓ − h ℓ ) 2 over the leaves with and without an effect in one data set. Figure 1 reports M S E 0 1 / 2 \mathrm{MSE}_0^{1/2} MSE 0 1/2 and M S E 1 1 / 2 \mathrm{MSE}_1^{1/2} MSE 1 1/2 , averaged over data sets. Two combined losses summarize both kinds of leaves with different weights. The balanced RMSE { ( M S E 0 + M S E 1 ) / 2 } 1 / 2 \{(\mathrm{MSE}_0+\mathrm{MSE}_1)/2\}^{1/2} {( MSE 0 + MSE 1 ) /2 } 1/2 gives the two kinds equal weight and was fixed before the experiment. The per-leaf RMSE { [ k M S E 1 + ( L − k ) M S E 0 ] / L } 1 / 2 \{[k\mathrm{MSE}_1+(L-k)\mathrm{MSE}_0]/L\}^{1/2} {[ k MSE 1 + ( L − k ) MSE 0 ] / L } 1/2 gives every leaf equal weight and is therefore also the error per observation. It was defined after the results were seen. Standard errors are over the 200 data sets, computed separately for each model.
Results. Figure 5 and Table 5 extend Figure 1 to signal-to-noise ratios 2 and 4. On leaves without an effect, DT-BCF has the smallest RMSE in every condition and HS the second smallest. The difference between the two exceeds two standard errors in every condition. On leaves with an effect, Fixed has the smallest RMSE whenever some leaves have no effect. At ς = 1 / 2 \varsigma= 1/2 ς = 1/2 its matched leaf variance is four to five times σ 2 / m \sigma^2/m σ 2 / m (Table 7 ), so its posterior mean keeps about four fifths of every leaf mean. The comparison of DT-BCF with BCF and Equal on these leaves depends on the signal. At SNR 2, BCF is more accurate than DT-BCF at L ≤ 16 L \le16 L ≤ 16 , and the two are within two standard errors at L ≥ 32 L \ge32 L ≥ 32 . At SNR 3 and 4, the RMSE of DT-BCF falls below those of BCF and Equal as L L L grows, by more than two standard errors from L = 32 L = 32 L = 32 . BCF and Equal give every leaf a common variance. As L L L grows, the leaves without an effect pull this variance toward zero, and the two leaves with an effect are shrunk with them. The shares of DT-BCF can instead place a large part of the tree’s variance on the two leaves with large ∣ R ˉ ℓ ∣ \lvert\bar{R}_{\ell}\rvert ∣ R ˉ ℓ ∣ .
Figure 5. Fixed Partition. RMSE of the posterior means of the leaf values on leaves without (top) and with (bottom) an effect, on a log scale, mean and two standard errors over 200 data sets. The first three columns vary L L L at k = 2 k = 2 k = 2 and signal-to-noise ratios 2, 3 and 4, and the last column varies k k k at L = 32 L = 32 L = 32 and SNR 3. Dotted, the RMSE σ / m 1 / 2 \sigma/m^{1/2} σ / m 1/2 of the leaf means.
Table 5. Fixed Partition. Here k = 2 k = 2 k = 2 leaves with an effect and ς = 1 / 2 \varsigma= 1/2 ς = 1/2 . Mean over the 200 data sets of each error of the posterior means of the leaf values, with its standard error in parentheses. Bold, the smallest mean in each row. The balanced RMSE is { ( M S E 1 + M S E 0 ) / 2 } 1 / 2 \{(\mathrm{MSE}_1 + \mathrm{MSE}_0)/2\}^{1/2} {( MSE 1 + MSE 0 ) /2 } 1/2 and was fixed before the experiment. The per-leaf RMSE is { [ k M S E 1 + ( L − k ) M S E 0 ] / L } 1 / 2 \{[k\mathrm{MSE}_1 + (L-k)\mathrm{MSE}_0]/L\}^{1/2} {[ k MSE 1 + ( L − k ) MSE 0 ] / L } 1/2 and was defined after the results were seen.
The combined losses order the models differently. The balanced RMSE is smallest for Fixed in every condition except L = 8 L = 8 L = 8 at SNR 4, because M S E 1 \mathrm{MSE}_1 MSE 1 is much larger than M S E 0 \mathrm{MSE}_0 MSE 0 and Fixed has the smallest M S E 1 \mathrm{MSE}_1 MSE 1 . The per-leaf RMSE is smallest for DT-BCF, or within 0.001 of the smallest, in every condition with k = 2 k = 2 k = 2 .
When more leaves carry an effect, the ordering on those leaves reverses (Table 6 ). From k = 8 k = 8 k = 8 at L = 32 L = 32 L = 32 , DT-BCF has the largest RMSE on leaves with an effect and the largest values of both combined losses. Its RMSE on leaves without an effect stays at or below its value at k = 2 k = 2 k = 2 . This is consistent with its prior, which at a = 1 / 2 a = 1/2 a = 1/2 favors allocations in which a few leaves receive most of the tree’s variance (§ 3.5 ). Changing ς \varsigma ς from 1 / 4 1/4 1/4 to 1 leaves the errors of DT-BCF, DT-BCF (ρ = 1 \rho= 1 ρ = 1 ), Equal, BCF and HS nearly unchanged. Fixed instead trades error on the leaves with an effect against error on the others, since its leaf variance is ω \omega ω itself.
Table 6. Fixed Partition. Here L = 32 L = 32 L = 32 and SNR 3. The first four rows of each block vary the number k k k of leaves with an effect at ς = 1 / 2 \varsigma= 1/2 ς = 1/2 , half of them at + Δ +\Delta + Δ and half at − Δ -\Delta − Δ . The last two rows vary ς \varsigma ς at k = 2 k = 2 k = 2 , whose row at ς = 1 / 2 \varsigma= 1/2 ς = 1/2 is shared. At k = 32 k = 32 k = 32 every leaf has an effect, so the error on leaves without an effect and the balanced RMSE are undefined, and the per-leaf RMSE equals the RMSE on leaves with an effect. Entries and bold as in Table 5 .
Table 6.
Forest Simulations
This subsection gives the data-generating processes of § 5.3 and the complete results of the tree-size and dimension studies, for every model and metric.
Data. Each data set has n = 200 n = 200 n = 200 training units and 500 test units drawn from the same process. The covariates are x i ∼ U ( 0 , 1 ) p \mathbf{x}_i \sim\mathrm{U}(0,1)^p x i ∼ U ( 0 , 1 ) p , the treatment is A i ∼ Bernoulli { e ( x i ) } A_i \sim\operatorname{Bernoulli}\{e(\mathbf{x}_i)\} A i ∼ Bernoulli { e ( x i )} , and y i = μ ( x i ) + A i τ ( x i ) + ε i y_i = \mu(\mathbf{x}_i) + A_i\tau(\mathbf{x}_i) + \varepsilon_i y i = μ ( x i ) + A i τ ( x i ) + ε i with ε i ∼ N ( 0 , σ 2 ) \varepsilon_i \sim\mathrm{N}(0,\sigma^2) ε i ∼ N ( 0 , σ 2 ) . In the two region scenarios, μ ( x ) = 1.5 + 2 x 1 + x 2 − 1.5 x 3 + sin ( 2 π x 4 ) \mu(\mathbf{x}) = 1.5 + 2x_1 + x_2 - 1.5x_3 + \sin(2\pi x_4) μ ( x ) = 1.5 + 2 x 1 + x 2 − 1.5 x 3 + sin ( 2 π x 4 ) , e ( x ) = logit − 1 { − 0.4 + 0.5 ( 1.3 x 1 − x 2 ) } e(\mathbf{x}) = \operatorname{logit}^{-1}\{-0.4 + 0.5(1.3x_1 - x_2)\} e ( x ) = logit − 1 { − 0.4 + 0.5 ( 1.3 x 1 − x 2 )} and σ = 1 / 2 \sigma= 1/2 σ = 1/2 . The rectangles are R 1 = { x 1 > 0.6 , x 2 > 0.5 } R_1 = \{x_1 > 0.6, x_2 > 0.5\} R 1 = { x 1 > 0.6 , x 2 > 0.5 } , R 2 = { x 1 < 0.3 , x 3 > 0.7 } R_2 = \{x_1 < 0.3, x_3 > 0.7\} R 2 = { x 1 < 0.3 , x 3 > 0.7 } , R 3 = { x 2 < 0.25 , x 4 < 0.35 } R_3 = \{x_2 < 0.25, x_4 < 0.35\} R 3 = { x 2 < 0.25 , x 4 < 0.35 } and R 4 = { x 3 > 0.65 , x 5 < 0.4 } R_4 = \{x_3 > 0.65, x_5 < 0.4\} R 4 = { x 3 > 0.65 , x 5 < 0.4 } . The 2 Regions scenario sets τ ( x ) = 2 \tau(\mathbf{x}) = 2 τ ( x ) = 2 on R 1 ∪ R 2 R_1 \cup R_2 R 1 ∪ R 2 . The 4 Regions Graded scenario sets τ ( x ) = 4 ⋅ 0.6 r − 1 \tau(\mathbf{x}) = 4 \cdot0.6^{r-1} τ ( x ) = 4 ⋅ 0. 6 r − 1 on the part of R r R_r R r that lies outside R 1 , … , R r − 1 R_1,\ldots,R_{r-1} R 1 , … , R r − 1 , for r = 1 , … , 4 r = 1,\ldots,4 r = 1 , … , 4 . Both set τ ( x ) = 0 \tau(\mathbf{x}) = 0 τ ( x ) = 0 elsewhere. Smooth Nonlinear sets τ = τ 0 \tau= \tau_0 τ = τ 0 of § 5.3 and σ = 3 \sigma= \sqrt{3} σ = 3 . Its prognostic function is linear, μ ( x ) = x T β \mu(\mathbf{x}) = \mathbf{x}^{\mathsf{T}}\boldsymbol{\beta} μ ( x ) = x T β , with coefficients drawn for each data set that are zero with probability 0.9 and standard normal otherwise. Its propensity score is the probit e ( x ) = Φ ( − 0.5 + 0.4 x 1 − 0.1 x 3 + 0.3 x 5 ) e(\mathbf{x}) = \Phi(-0.5 + 0.4x_1 - 0.1x_3 + 0.3x_5) e ( x ) = Φ ( − 0.5 + 0.4 x 1 − 0.1 x 3 + 0.3 x 5 ) , where Φ \Phi Φ is the standard normal distribution function. Smooth Graded sets τ ( x ) = 1.65 − G ( x ) τ 0 ( x ) \tau(\mathbf{x}) = 1.65^{-G(\mathbf{x})}\tau_0(\mathbf{x}) τ ( x ) = 1.6 5 − G ( x ) τ 0 ( x ) , with G ( x ) = I ( x 6 > 1 / 2 ) + 2 I ( x 7 > 1 / 2 ) G(\mathbf{x}) = \mathbb{I}(x_6 > 1/2) + 2\mathbb{I}(x_7 > 1/2) G ( x ) = I ( x 6 > 1/2 ) + 2 I ( x 7 > 1/2 ) , and σ = 3 \sigma= \sqrt{3} σ = 3 . Its μ \mu μ and e e e are those of the region scenarios. Each setting has 40 data sets with seeds 1 to 40. A scenario and a seed give the same data set at every η \eta η , so every model is fitted to the same data sets at every η \eta η . The dimension study at p = 50 p = 50 p = 50 reuses the fits of the tree-size study at η = 2 \eta= 2 η = 2 .
Models. Both studies fit every model of Appendix § 12 .§ 1 , including the DART versions of DT-BCF, Equal and HS. The tree-size exponent η \eta η changes only the split probability of the treatment trees in our sampler. BCF keeps its own split probability 0.25 ( 1 + d ) − 3 0.25(1+d)^{-3} 0.25 ( 1 + d ) − 3 , and the loss-based models have no tree prior. Their fits therefore do not depend on η \eta η . BCF is fitted with its package in the tree-size study and at p ∈ { 20 , 50 } p \in\{20,50\} p ∈ { 20 , 50 } , and in our sampler at p ≥ 200 p \ge200 p ≥ 200 .
Errors on Units. Figure 7 splits the RMSE of the centered effect in the region scenarios into the units with and without an effect, as Figure 1 does for leaves. It also reports the spurious SD, the standard deviation of δ ^ i \hat{\delta}_i δ ^ i over the units without an effect. Because δ i \delta_i δ i takes one value on these units, the spurious SD is zero for a model that estimates no heterogeneity among them.
Figure 7. Dimension Study. n = 200 n = 200 n = 200 and η = 2 \eta= 2 η = 2 , region scenarios. RMSE of the centered effect on units without (top) and with (middle) an effect, and the spurious SD (bottom), against the number of covariates p p p , mean and two standard errors over 40 data sets.
Results. Table 8 to Table 13 give the tree-size study, and Table 14 to Table 19 the dimension study. They contain the values behind Table 1 and Figure 2 and add the models and metrics that the main text does not show. On the region scenarios, Equal has a smaller mean RMSE than DT-BCF in all eight settings of the tree-size study, by more than two standard errors in four. Equal + DART has a smaller mean RMSE than DT-BCF + DART in all eight, by more than two standard errors in two. In the dimension study, Equal has a smaller mean RMSE than DT-BCF in all ten settings, by more than two standard errors in four. Equal + DART has a smaller mean RMSE than DT-BCF + DART in eight of the ten, in none by more than two standard errors. Equal + DART also has the smallest FPR in most region settings of the tree-size study. A DART version has the largest lpd in every region setting, Equal + DART in all of those of the tree-size study. In the dimension study, the X-learner has the smallest RMSE on 2 Regions from p = 200 p = 200 p = 200 and on 4 Regions Graded from p = 500 p = 500 p = 500 .
Table 8. Table 8. Tree-size Study. n = 200 n = 200 n = 200 and p = 50 p = 50 p = 50 . RMSE of the centered effect by scenario and the tree-size exponent η \eta η . Mean over 40 data sets with its standard error in parentheses. Bold, the smallest mean in each column of a scenario. BCF and the loss-based models do not depend on η \eta η , so their entries repeat the same fits.
Table 13. Tree-size Study. n = 200 n = 200 n = 200 and p = 50 p = 50 p = 50 . Log predictive density of 500 test units, averaged over the units. Entries as in Table 8 . Bold, the largest mean in each column of a scenario.
Table 14. Dimension Study. n = 200 n = 200 n = 200 and η = 2 \eta= 2 η = 2 . RMSE of the centered effect by scenario and the number of covariates p p p . Mean over 40 data sets with its standard error in parentheses. Bold, the smallest mean in each column of a scenario.
Table 19. Table 19. Dimension Study. n = 200 n = 200 n = 200 and η = 2 \eta= 2 η = 2 . Log predictive density of 500 test units, averaged over the units. Entries as in Table 14 . Bold , the largest mean in each column of a scenario.
On the smooth scenarios, BCF has the smallest RMSE in every setting except Smooth Nonlinear at p = 50 p = 50 p = 50 , where Equal + DART is smallest at every η \eta η . The coverage of BCF is also the closest to 0.90 in most of these settings. The lpd of the models other than Fixed spans at most 0.09 in any smooth setting.
Coverage lies below 0.90 for most models and settings. Fixed has the lowest coverage and the smallest lpd in every setting.
Equal Division on Region Scenarios. Equal is the limit a → ∞ a \to\infty a → ∞ of the weighted allocation at ρ = 1 \rho= 1 ρ = 1 , so Equal, DT-BCF (ρ = 1 \rho= 1 ρ = 1 ) and DT-BCF belong to one family. Each of the three has a smaller mean RMSE than BCF and HS on both region scenarios at every η \eta η . The three give the tree-level total variance V j V_j V j the same form and differ in how unequally a tree divides it among its leaves. The posterior mean of ( ∑ j ϕ i j 2 ) − 1 \left(\sum_j \phi_{ij}^{2}\right)^{-1} ( ∑ j ϕ ij 2 ) − 1 , averaged over the treatment trees and the data sets of a setting, is the realized counterpart of A e f f ( T j ) A_{\mathrm{eff}}(T_j) A eff ( T j ) in § 3.5 . It lies between 1.27 and 1.38 for DT-BCF, between 1.61 and 2.42 for DT-BCF (ρ = 1 \rho= 1 ρ = 1 ), and between 2.50 and 4.67 for Equal, for which it equals the number of leaves. Without DART, the mean RMSE on the region scenarios follows the reverse order of this quantity, largest for DT-BCF and smallest for Equal, in seven of the eight settings of the tree-size study.
We attribute this ordering to three features of the designs, the small treatment trees, the shape of the regions and the independence of the covariates. Each rectangle is defined by two of the independent uniform covariates, so both leaves of any single split contain units with and without an effect. The split at x 1 = 0.6 x_1 = 0.6 x 1 = 0.6 in 2 Regions, for instance, leaves an effect on half of the units on one side and on 15 percent of them on the other. Because the trees are this small, their leaves typically contain units of both kinds. Such leaves call for comparable shares, whereas under DT-BCF the shares of a two-leaf tree follow Beta( 1 / 4 , 1 / 4 ) (1/4, 1/4) ( 1/4 , 1/4 ) instead, a prior that favors giving nearly all of V j V_j V j to one of the two leaves. The fixed partition of Appendix § 12 .§ 2 , where every leaf either carries an effect or does not, shows when this concentration helps. There DT-BCF has the smallest RMSE on the leaves without an effect in every condition. With 8 or more of the 32 leaves carrying an effect, its RMSE on those leaves is the largest. When all 32 leaves carry an effect, Equal and BCF have the smallest (Table 6 ). The concentrated shares also affect detection. In all eight region settings of the tree-size study, DT-BCF has a larger mean TPR than Equal, by more than two standard errors in one, and a larger mean FPR. The forest fits cannot separate the shape of the regions from the independence of the covariates. With correlated covariates, or with regions defined by a single covariate, one split could separate most units with an effect from those without, as the leaves of the fixed partition do.
Figure 6. Tree-size Study. n = 200 n = 200 n = 200 and p = 50 p = 50 p = 50 . RMSE of the centered effect against the tree-size exponent η \eta η , where smaller η \eta η favors larger treatment trees, mean and two standard errors over 40 data sets, on a log scale. BCF and the loss-based models do not depend on η \eta η .
Tree-size Study. n = 200 n = 200 n = 200 and p = 50 p = 50 p = 50 . Average length of the 90% intervals for the centered effect. Entries as in Table 8 . No entry is bold, because Fixed has the shortest intervals in every column together with the lowest coverage (Table 9 ), so length is read together with coverage.
Table 10.
Table 9. Table 9. Tree-size Study. n = 200 n = 200 n = 200 and p = 50 p = 50 p = 50 . Coverage of the 90% intervals for the centered effect, averaged over units. Entries as in Table 8 . Bold, the coverage closest to 0.90 in each column of a scenario.
Table 12. Tree-size Study. n = 200 n = 200 n = 200 and p = 50 p = 50 p = 50 . True positive rate, the share of units with an effect whose 90% interval for τ ( x i ) \tau(x_i) τ ( x i ) excludes zero. Entries as in Table 8 . No entry is bold, because Fixed has the largest rate in every column together with the largest FPR (Table 11 ), so the TPR is read together with the FPR.
Table 12.
Table 11. Tree-size Study. n = 200 n = 200 n = 200 and p = 50 p = 50 p = 50 . False positive rate, the share of units without an effect whose 90% interval for τ ( x i ) \tau(x_i) τ ( x i ) excludes zero. Entries as in Table 8 . Bold, the smallest rate in each column of a scenario.
Table 11.
Table 16. Dimension Study. n = 200 n = 200 n = 200 and η = 2 \eta= 2 η = 2 . Average length of the 90% intervals for the centered effect. Entries as in Table 14 . No entry is bold, because Fixed has the shortest intervals in every column together with the lowest coverage (Table 15 ), so length is read together with coverage.
Table 15. Dimension Study. n = 200 n = 200 n = 200 and η = 2 \eta= 2 η = 2 . Coverage of the 90% intervals for the centered effect, averaged over units. Entries as in Table 14 . Bold, the coverage closest to 0.90 in each column of a scenario.
Table 15.
Table 18. Dimension Study. n = 200 n = 200 n = 200 and η = 2 \eta= 2 η = 2 . True positive rate, the share of units with an effect whose 90% interval for τ ( x i ) \tau(x_i) τ ( x i ) excludes zero. Entries as in Table 14 . No entry is bold, because Fixed has the largest rate in every column together with the largest FPR (Table 17 ), so the TPR is read together with the FPR.
Table 18.
Table 17. Dimension Study. n = 200 n = 200 n = 200 and η = 2 \eta= 2 η = 2 . False positive rate, the share of units without an effect whose 90% interval for τ ( x i ) \tau(x_i) τ ( x i ) excludes zero. Entries as in Table 14 . Bold, the smallest rate in each column of a scenario.
Table 17.
ACIC 2016 Benchmark
The simulated designs of Appendix § 12 .§ 3 draw independent uniform covariates. To assess the models on an empirical covariate distribution with known unit-level effects, § 5.4 uses the data of the 2016 Atlantic Causal Inference Conference competition [23 ] . This subsection gives its design and its complete results.
Data. The competition fixes a design matrix of n = 4,802 n = 4{,}802 n = 4 , 802 units and 58 covariates from the Collaborative Perinatal Project, of which 3 are categorical, 5 binary, 27 counts and 23 continuous [23 ] . We code the three categorical covariates, with 6, 16 and 5 levels, as 25 indicator columns, one for every level of the first and for every level but one of the other two, which gives 80 columns. On this design the competition simulates the treatment assignment and the outcome under 77 settings with 100 replicates each. A setting is defined by six parameters, the functional form of the assignment model (linear, polynomial or step), a parameter of the baseline treatment probability (0.35 or 0.65), the overlap (full, or restricted by one term), the functional form of the response model (linear, exponential or step), the alignment of the confounders between the two models (0, 0.25 or 0.75), and the heterogeneity of the treatment effect (none, medium or high). For every unit the competition supplies the conditional means μ 0 ( x i ) \mu_0(x_i) μ 0 ( x i ) and μ 1 ( x i ) \mu_1(x_i) μ 1 ( x i ) of the outcome under control and treatment, so the effect τ ( x i ) = μ 1 ( x i ) − μ 0 ( x i ) \tau(x_i) = \mu_1(x_i) - \mu_0(x_i) τ ( x i ) = μ 1 ( x i ) − μ 0 ( x i ) and the centered effect δ i \delta_i δ i are known. We take σ \sigma σ to be the standard deviation of y i − μ A i ( x i ) y_i - \mu_{A_i}(x_i) y i − μ A i ( x i ) over the units of a data set.
Setting Selection. We define the active fraction
F ϵ = 1 n # { i : ∣ δ i ∣ > ϵ σ } , ϵ = 0.2 , F_{\epsilon} = \frac{1}{n}\#\{i : \lvert\delta_i\rvert> \epsilon\sigma\}, \qquad\epsilon= 0.2, F ϵ = n 1 # { i : ∣ δ i ∣ > ϵ σ } , ϵ = 0.2 ,
as the share of units whose centered effect exceeds a fifth of the residual standard deviation in absolute value. Before any model was fitted, the 77 settings were ordered by F ϵ F_{\epsilon} F ϵ , averaged over replicates 1 to 3, and divided into 20 groups of consecutive settings of nearly equal size. One setting was drawn from each group with a recorded seed. Table 20 lists the 20 settings with their parameters. Two of the 77 settings have a constant effect and F ϵ = 0 F_{\epsilon} = 0 F ϵ = 0 , and one of them, setting 3, was drawn. The other 19 drawn settings have F ϵ F_{\epsilon} F ϵ between 0.889 and 0.979, and the replicates of one setting differ in F ϵ F_{\epsilon} F ϵ by up to 0.16. The selected settings therefore vary little in this fraction. They vary more in the spread of the effect, whose standard deviation across units ranges from 2.6 to 6.1 times σ \sigma σ in replicates 1 to 3. We fit replicates 1 to 10 of every selected setting, 200 data sets in all.
Table 20. Table 20. ACIC 2016 Settings. The 20 settings drawn before any model was fitted, in increasing order of the active fraction F ϵ F_{\epsilon} F ϵ (ϵ = 0.2 \epsilon= 0.2 ϵ = 0.2 , mean over replicates 1 to 3, as used for the selection). The next six columns are the competition’s parameters of the setting ([23 ] ). The range of F ϵ F_{\epsilon} F ϵ , the number of treated units and the ratio of the standard deviation of the centered effect to the residual standard deviation σ \sigma σ are computed over replicates 1 to 3, the last as a mean. Every data set has n = 4,802 n = 4{,}802 n = 4 , 802 units.
Preprocessing and Models. In each data set, the outcome is centered and scaled to unit standard deviation before fitting, and τ \tau τ , δ i \delta_i δ i and σ \sigma σ are scaled with it. Every error below is therefore in units of the standard deviation of the outcome of its data set. On this scale σ \sigma σ lies between 0.13 and 0.26. Every model receives the same cross-fitted estimate of the propensity score (Appendix § 12 .§ 1 ), and the true assignment probabilities are not used. BCF is fitted with its package at its defaults. In our sampler we fit DT-BCF, DT-BCF (ρ = 1 \rho= 1 ρ = 1 ), the DART version of each, HS and Fixed, with ω \omega ω chosen on the 80 columns of the competition design before any outcome was read. On an independent set of 1,000 prior draws, the median of S τ / σ S_\tau/\sigma S τ / σ lies between 0.46 and 0.52 for every model. Equal and the DART version of HS were not fitted in this study.
Losses. The RMSE of the centered effect ranges over more than an order of magnitude across data sets, so § 5.4 compares the models through its ratio to that of BCF on the same data set. With ℓ ˉ \bar{\ell} ℓ ˉ and s s s the mean and standard deviation of the log ratio over the N N N data sets, the reported ratio is exp ( ℓ ˉ ) \exp(\bar{\ell}) exp ( ℓ ˉ ) with the interval exp ( ℓ ˉ ± 2 s / N 1 / 2 ) \exp(\bar{\ell} \pm2s/N^{1/2}) exp ( ℓ ˉ ± 2 s / N 1/2 ) . The BCF package failed on two data sets, replicate 2 of setting 4 and replicate 9 of setting 48, so N = 188 N = 188 N = 188 . A setting counts as below one when the geometric mean of the ratio over its replicates is below one.
Results. Table Table 21 extends Table 2 with DT-BCF (ρ = 1 \rho= 1 ρ = 1 ) and its DART version, whose ratios are lower than those of DT-BCF and DT-BCF + DART by 0.007 and 0.011. The ratio of DT-BCF to BCF is below one in 10 of the 19 settings (Table 22 ), so the comparison between the two changes sign across settings. The ratio of DT-BCF + DART is below one in 18. Because every setting with a non-constant effect has F ϵ F_{\epsilon} F ϵ between 0.889 and 0.979, the benchmark does not show how the comparison depends on F ϵ F_{\epsilon} F ϵ . On setting 3, where the effect is constant, the models in our sampler have an RMSE of the centered effect between 0.011 and 0.021, and their intervals contain the centered effect of almost every unit (Table 23 ). BCF has an RMSE of 0.022 and a coverage of 0.91, and the X-learner and the R-learner have RMSEs of 0.060 and 0.146.
Table 21. Table 21. ACIC 2016. The 19 settings with a non-constant effect. The RMSE ratio is the geometric mean over the 188 data sets with a BCF fit of the RMSE of the centered effect divided by that of BCF on the same data set, with its two-standard-error interval. Settings is the number of settings whose geometric-mean ratio is below one. Coverage and length are those of the 90% intervals for the centered effect, averaged over the 190 data sets, or 188 for BCF. Bold, the smallest ratio, the largest number of settings and the coverage closest to 0.90.
Table 22. Table 22. ACIC 2016. Settings, in increasing order of F ϵ F_{\epsilon} F ϵ . The third column gives the mean RMSE of the centered effect of BCF on the standardized outcome, and the others the geometric mean over the replicates of a setting of the RMSE of each model divided by that of BCF on the same data set. Every setting has 10 replicates, except settings 4 and 48 with 9. Bold, the smallest ratio in each row when it is below one.
Table 23. Table 23. ACIC 2016. Constant setting, 10 replicates. RMSE of the centered effect on the standardized outcome, mean over data sets with its standard error in parentheses, and the coverage of the 90% intervals for the centered effect. Because the centered effect is zero for every unit, the RMSE measures spurious heterogeneity. Bold, the smallest RMSE.
t
Pancreatic Cohort
This subsection describes the cohort, the covariates and the models of § 5.5 , gives the results of every model fitted to the cohort, and reports the diagnostics of the leave-one-out estimates.
Cohort. The data are the TCGA PAAD object of pdacR Torre-Healy et al. [69 ] , which holds clinical records and gene expression for 181 tumors of the pancreatic ductal adenocarcinoma study of Cancer Genome Atlas Research Network (2017). Following Jacobs et al. [38 ] , we exclude the 51 patients with a missing survival time, vital status or radiotherapy status, which leaves 130 patients. The treatment is adjuvant radiotherapy, received by 36 patients, and the outcome is overall survival in days. Of the 130 patients, 71 died during follow-up and 59 are censored. The median follow-up, estimated by the reverse Kaplan–Meier method, is 23.2 months, against 23.3 months in Jacobs et al. [38 ] . Our censoring rate of 45 percent is below their 47 percent, so the two analyses differ in the vital status of at least two patients. TCGA required every contributing site to document that its institutional review board had approved the use of the specimens and data or did not regard it as human-subjects research. Its open-access tier holds only de-identified clinical data National Cancer Institute, [51 ] . We use only these open-access data (Appendix § 12 .§ 1 ).
Covariates. The clinical block contains sex, grade, pathological T and N stage, residual tumor, history of chronic pancreatitis, KRAS mutation status, the number of lymph nodes, the surgical procedure, race and histological type. Missing categories are coded as an additional level, and missing numeric values are imputed by the median with an indicator of missingness. The clinical block thus has 27 columns. The expression block applies log 2 ( x + 1 ) \log_{2}(x+1) log 2 ( x + 1 ) to the expression values, keeps the 3000 genes with the largest median absolute deviation, as in Jacobs et al. [38 ] , and centers and scales each gene. Jacobs et al. [38 ] also include genes selected from the literature, which we omit because their list was not available to us. With p = 3,027 p = 3{,}027 p = 3 , 027 covariates and n = 130 n = 130 n = 130 , BCF is fitted in our sampler (Appendix § 12 .§ 1 ). The propensity score is the cross-fitted logistic lasso of Appendix § 12 .§ 1 and ranges from 0.19 to 0.42.
Models. Every model uses the accelerated failure time model of Appendix § 9 for the log survival time. In our sampler, the log time of a censored patient is a latent variable, drawn in each iteration from its normal full conditional truncated below at the log censoring time (Appendix § 10 .8). Given the latent times, the model is the BCF model with a normal outcome. Besides the models of Table 3 , we fit DT-BCF (ρ = 1 \rho= 1 ρ = 1 ), its DART version. In this subsection, HS denotes the CausalHorseForest fit of Table 3 . The priors of the models in our sampler are those of Appendix § 12 .§ 1 , with ω \omega ω matched on the 130 patients. Each of them runs 4 chains of 40000 iterations, of which the first 10000 are warmup, and keeps every third draw. A fit took 25 to 39 minutes with one core per chain. HS runs 4 chains of 10000 iterations, of which the first 5000 are warmup.
Results. Table 24 extends Table 3 with the two additional models. The posterior means of the average effect lie between 0.95 and 1.15, and every 95% interval lies above zero (Figure 8 ). DT-BCF (ρ = 1 \rho= 1 ρ = 1 ) has the longest upper tails, with 95% intervals that reach 4.42 for the average effect and 5.21 for the SD across patients. Its DART version reaches 2.26 and 3.07, and DT-BCF reaches 1.70 and 1.87. Under every matched prior, the posterior median of the SD across patients lies below its prior median. For no model and no patient does the 95% interval of the departure τ ( x i ) − τ ˉ \tau(x_i) - \bar{\tau} τ ( x i ) − τ ˉ exclude zero. Under each model of Table 3 , the 95% intervals for τ ( x i ) \tau(x_i) τ ( x i ) of all 130 patients overlap in a common range of values (Figure 9 ).
Table 24. Pancreatic Cohort. Reported Models. ATE, posterior mean of the average effect on log survival time with its 95% interval. SD across patients, the standard deviation of τ ( x i ) \tau(x_i) τ ( x i ) over the 130 patients, with its prior median at σ = 1 \sigma= 1 σ = 1 and its posterior median with its 95% interval. The prior median comes from 4,000 prior draws that grow the treatment trees as the sampler does, on the treated patients with cut points drawn uniformly and, for the DART versions, with DART split probabilities. σ \sigma σ , posterior median of the residual standard deviation on the log time scale. HS is the CausalHorseForest fit of Table 3 , whose prior is not matched. No entry is bold, because the cohort has no known effect against which the models could be ranked.
Figure 8. Pancreatic Cohort. Posterior densities of the average effect on log survival time under the models of Table 3 . The shaded region is the 95% interval, the vertical line the posterior mean, and the dashed line marks zero. The axis ends at 6, beyond which lie 8 of the 40,000 draws of DT-BCF + DART.
The survival curves of Figure 10 average over all 130 patients, while each Kaplan–Meier curve describes only the patients who received that treatment. The Kaplan–Meier curve of the 94 untreated patients lies within the 95% bands of the models at nearly every time shown. That of the 36 treated patients lies above the bands until shortly after the first treated death at 7.7 months, and below them around 25 and 36 months. At these times few treated patients remain at risk, 8 at 24 months and 3 at 36 months, so each death is a large step.
Leave-one-out Estimates. The elpd of Table 3 is estimated by PSIS-LOO [71 ] . For each patient, the posterior draws θ ( s ) \theta^{(s)} θ ( s ) are reweighted by importance ratios proportional to 1 / p ( y i ∣ θ ( s ) ) 1/p(y_i \mid\theta^{(s)}) 1/ p ( y i ∣ θ ( s ) ) , and the largest ratios are smoothed by a fitted generalized Pareto distribution. The estimated shape k ^ i \hat{k}_i k ^ i of that distribution measures how reliable the reweighting is. Above 0.7, the estimate of the term of patient i i i can have a large error [73 ] . Table 25 reports the diagnostics. The number of patients above 0.7 ranges from one for each DART version to 24 for Fixed and 36 for DT-BCF.
Table 25. Pancreatic Cohort. PSIS-LOO estimates and their Pareto diagnostics. elpd, expected log pointwise predictive density on the log time scale with its standard error, where the Jacobian of the log transformation is common to all models and omitted. p l o o p_{\mathrm{loo}} p loo , the estimated effective number of parameters. The last three columns give the number of the 130 patients with k ^ i > 0.5 \hat{k}_i > 0.5 k ^ i > 0.5 and k ^ i > 0.7 \hat{k}_i > 0.7 k ^ i > 0.7 , and the largest k ^ i \hat{k}_i k ^ i . No entry is bold, because the elpd of the models in our sampler differ by less than half of one standard error.
The patients flagged under DT-BCF and under Fixed are largely the same, nearly all of them untreated, and most of them died during follow-up. They differ from the observations that usually produce a large k ^ i \hat{k}_i k ^ i , which are fitted poorly or have a large influence on the fit. Under DT-BCF, the posterior mean of μ ( x i , e ^ i ) + A i τ ( x i ) \mu(x_i, \hat{e}_i) + A_i\tau(x_i) μ ( x i , e ^ i ) + A i τ ( x i ) lies close to their observed log times, within a median of 0.19 residual standard deviations against 0.82 for the other patients. Their contributions lpd i − elpd l o o , i \operatorname{lpd}_i - \operatorname{elpd}_{\mathrm{loo},i} lpd i − elpd loo , i to the effective number of parameters p l o o p_{\mathrm{loo}} p loo are small, with a median of 0.11 against 0.20, and together they account for 9.3 of the 35.0 effective parameters of DT-BCF. The patients flagged under HS follow the usual pattern instead, lying a median of 1.31 residual standard deviations from their fitted values and contributing a median of 0.95 to p l o o p_{\mathrm{loo}} p loo .
We are nonetheless conservative in the use of these estimates and read them only for the ordering of the models. The elpd of the models in our sampler lie within 4.5 of each other, less than half of one standard error, and that of HS lies between 23 and 28 below them. The ordering is unchanged on the 85 patients whose k ^ i \hat{k}_i k ^ i is at most 0.7 under DT-BCF, BCF, HS and Fixed, where the sums of the pointwise estimates are − 78.4 -78.4 − 78.4 , − 78.0 -78.0 − 78.0 , − 92.0 -92.0 − 92.0 and − 78.9 -78.9 − 78.9 . Exact refits that leave out each flagged patient would replace the approximation where it is least reliable.