betaFeedback

WorksMathematics

Outcome-dependent density combination by mixture Bayesian predictive synthesis

Abstract

In the problem of density combination, the decision maker receives multiple density functions and is tasked to learn which density is used most in which region of the outcome space, and to combine them into a single density. The outcome-dependent density combination can be cast as the Bayesian predictive synthesis, where the decision maker's uncertainty is naturally integrated. The models for the outcome-dependent weights have been considered in Johnson and West (2025), but limited to the Gaussian density kernel. In this study, we point out the limitation of the model flexibility of the Gaussian weights, and theoretically and numerically verify that the estimated weights can be least outcome-dependent. As an alternative, we propose the use of the softmax (logistic-normal) functions as the models for the weights, together with the bias-adjustment to the submitted densities. The posterior computation is feasible by using the custom Metropolis-Hastings method with the Pólya-gamma augmentation. The utility of the proposed method is illustrated in the real data analysis of movie theatre revenues, where a human expert reports the point quantile forecasts without any uncertainty quantification.

Introduction

In predictive analysis, it is common to have multiple predictive distributions of the quantity of interest from different sources of information, such as models, computer algorithms, and/or forecasts by human experts. For example, macroeconomic indices such as GDP can be forecast by multiple macroeconometric models [e.g. 5], in addition that the expectations of professionals on those indices are surveyed and integrated into the model-based predictions [e.g. 1]. Since various models and experts provide different forecasts, it is reasonable to assume that one forecast describes the typical values, while another is informative about outliers. Considering the diversity of the provided predictive distributions, we, as the decision maker, are then tasked to combine these distributions into a single distribution in an outcome-dependent way, while learning which distribution is relevant in which region of the outcome space.

Standard density-combination methods offer only a partial answer to this problem of outcome-dependent density combination. Linear and logarithmic pools, together with score-based prediction pools, are widely used to construct aggregate density forecasts [8, 12, 9, 11]. However, these methods consider the mixture of the densities using the constant, outcome-independent weights. To address the outcome-dependent weights, the generalized density forecast combination of Kapetanios et al. [15] partitions the outcome space and assigns region-specific weights to the candidate densities. However, such pooling rules are not, by themselves, Bayesian updating: they construct an aggregate density directly rather than deriving a posterior predictive distribution conditional on their information set.

A natural Bayesian solution to the problem of density combination is the Bayesian model averaging, where the prior over the models, algorithms and experts is assumed in order to derive the combined density as the posterior. However, it is well known that the predictive density based on the model averaging is outcome-independent. In this context, Bayesian predictive synthesis (BPS) considers the prior over the densities/distributions, providing an outcome-dependent but coherent way of combining the densities. Originating in Bayesian agent opinion analysis [7, 22, 23], BPS introduces a custom latent variable for each density and combines them through a synthesis function. The influential dynamic BPS model of McAlinn and West (2019) develops this idea through a dynamic regression synthesis function. Subsequent work extends the BPS framework in several directions, including multivariate synthesis [18], sequential computation [16], and application-specific synthesis models [4, 21]. However, modeling of the synthesis function for the outcome-dependent density combination has not been explored extensively.

In this study, we consider the BPS-based approach to the outcome-dependent density combination, including the models for weight functions, posterior computation and illustration by applications. One key development in the BPS-based density combination is the outcome-dependent generalized opinion pools of Johnson and West (2025), covering issues related to uncertainty quantification such as model-set incompleteness. The outcome-dependent weights are modeled by using the Gaussian density kernel, so that each candidate density is used most in some neighborhood, but less outside. Although the Gaussian weights were reported to provide reasonable predictions in time series analysis, their direct application to density combination was not studied. In fact, we have confirmed in simulation studies that the estimates of the Gaussian weights can be constant, hence outcome-independent. We mathematically verify this phenomenon using the theory of posterior concentration in a large sample situation.

We then consider an alternative BPS-based model for density combination, beyond the Gaussian models. First, we enable the synthesis function to account for our uncertainty on the reported forecast densities, as considered in Johnson (2017). This modification not only improves the model flexibility but also simplifies the posterior computation by the Markov chain Monte Carlo method. Second, we propose to use softmax functions, or logistic-normal density functions, as the model for outcome-dependent weights. Unlike the Gaussian weights, the softmax weights are unbounded, allowing the sharp switch from one candidate density to another. The posterior computation is straightforward using the Pólya-gamma augmentation technique [19]. Taken together, the flexibility of the proposed model can handle a challenging problem of density combination, such as the combination of point forecasts about density quantiles, as illustrated later using the dataset of movie theatre revenue.

The remainder of the paper is organized as follows. § 2 reviews density forecast combinations, BPS, and outcome-dependent mixture synthesis. § 3 studies Gaussian-kernel outcome-dependent weights and their undesirable behavior in a large sample situation. § 4 defines the softmax weight models and discusses their difference from the Gaussian weights. § 5 develops custom Markov chain Monte Carlo algorithms for posterior computation. § 7 presents the results of numerical experiments for comparative studies. § 8 is an illustration of the proposed method in a real data analysis. § 9 concludes with possible future research.

Bayesian predictive synthesis

In this section, we briefly review the framework of Bayesian predictive synthesis (BPS). The density combination is viewed as the BPS of a certain type and called mixture BPS. Specifically, we consider two models of the mixture BPS: Dirac and bias-adjusted synthesis functions.

Density combination as BPS

Suppose that we are interested in the prediction of univariate quantity yy and, for this predictive purpose, given KK probability densities, h1(⋅),…,hK(⋅)h_1(\cdot), \ldots, h_K(\cdot). These densities define the information set, denoted by H={h1,…,hK}\mathcal{H} = \{h_1, \ldots, h_K\}. As noted in the introduction, these densities can be provided from different sources, including statistical/econometric models, deterministic algorithms and/or human experts, which are all called agents. Since H\mathcal{H} is known, the uncertainty about yy should be quantified by its conditional distribution, or “posterior”, as p(y∣H)p(y \mid\mathcal{H}). In modeling this distribution, however, one must specify the joint distribution of yy and H\mathcal{H}, which is often a difficult task. Instead, a series of research on predictive synthesis by Genest and Schervish [7] and West and Crosse [23] considered a situation where such distribution is specified only partially. It has been proven that, for such a model to be coherent in uncertainty quantification, the posterior distribution of interest must be of the form,

p(y∣H)=∫α(y∣x)h(x) dxp(y \mid\mathcal{H}) = \int\alpha(y \mid x)h(x)\,\mathrm{d}x

where x=(x1,…,xK)⊤x = (x_1,\ldots,x_K)^\top, α(y∣x)\alpha(y \mid x) is a conditional density called the synthesis function, and

h(x)=∏k=1Khk(xk).h(x) = \prod_{k=1}^{K} h_k(x_k).

This representation implies that modeling of (y,H)(y,\mathcal{H}) is equivalent to specifying α(y∣x)\alpha(y \mid x). McAlinn [17], McAlinn and West (2019) and other recent research on BPS have studied various models for α(y∣x)\alpha(y \mid x) and the functional forms of the implied predictive density p(y∣H)p(y \mid\mathcal{H}).

Modeling of synthesis functions

By choosing synthesis function α(y∣x)\alpha(y \mid x) appropriately, typical models for density combination can be viewed as BPS, hence justified as a coherent way of quantifying predictive uncertainty. In modeling the synthesis function, we specify a class of functions indexed by unknown quantities, θ\theta, and parametrize the synthesis function as α(y∣x,θ)\alpha(y \mid x,\theta). Since θ\theta is unknown to us, we quantify our uncertainty on θ\theta via our posterior distribution using some test data. In this process of posterior computation, we evaluate the predictive performance of each agent in H\mathcal{H}.

The predictive analysis in the BPS framework is formulated as follows. The BPS equation is rewritten with explicit dependence on parameter θ\theta as

p(y∣H,θ)=∫α(y∣x,θ)h(x) dxp(y \mid\mathcal{H},\theta) = \int\alpha(y \mid x,\theta)h(x)\,\mathrm{d}x

Then, test data of size nn, or y1,…,yny_1,\ldots,y_n, are assumed to be generated independently from this distribution. With prior Π(θ)\Pi(\theta), the posterior distribution of θ\theta is obtained as

p(θ∣y1:n,H)∝Π(θ)∏i=1n∫α(yi∣xi,θ)h(xi) dxi,p(\theta\mid y_{1:n},\mathcal{H}) \propto\Pi(\theta) \prod_{i=1}^{n} \int\alpha(y_i \mid x_i,\theta)h(x_i)\,\mathrm{d}x_i,

where xi=(xi1,…,xiK)Tx_i = (x_{i1}, \ldots, x_{iK})^{\mathsf{T}} for i=1,…,ni = 1, \ldots, n. This density is often numerically approximated using a Markov chain Monte Carlo (MCMC) method with sampling of latent variables x1:nx_{1:n} and parameter θ\theta. Finally, the predictive distribution of our interest is expressed as

p(y∣y1:n,H)=∬α(y∣x,θ)h(x)p(θ∣y1:n,H) dx dθ,p(y \mid y_{1:n}, \mathcal{H}) = \iint\alpha(y \mid x, \theta)h(x)p(\theta\mid y_{1:n}, \mathcal{H}) \, \mathrm{d}x \, \mathrm{d}\theta,

which is also evaluated by simulation.

For synthesis function α(y∣x,θ)\alpha(y \mid x, \theta), we consider two models that correspond to the outcome-dependent density combination.

Dirac synthesis: Following Johnson and West [14], we first consider the synthesis function of the form,

α(y∣x,θ)=w0(x;θ)h0(y)+∑k=1Kwk(x;θ)δxk(y),\alpha(y \mid x, \theta) = w_0(x; \theta)h_0(y) + \sum_{k=1}^{K} w_k(x; \theta)\delta_{x_k}(y),

where δxk(y)\delta_{x_k}(y) is the degenerate distribution of yy at xkx_k, and w0(x;θ)w_0(x; \theta), w1(x;θ)w_1(x; \theta), …\ldots, wK(x;θ)w_K(x; \theta) are the weight functions; they are non-negative and sum to unity for any xx and θ\theta. Here, we have added an extra density, h0(y)h_0(y), to the mixture form. This density is called a baseline density, represents our own prior belief on yy and complements the incompleteness of H\mathcal{H} as the model set. The functional form of wk(⋅;θ)w_k(\cdot; \theta) will be given later in § 3 and § 4. We call this model Dirac synthesis.

Note that, in general, the agent kk’s weight function can depend on not only xkx_k but also the other xk′x_{k'} (k′≠kk' \ne k). If one assumes the independence of the agents in the weights and restricts the weight function as wk(xk;θ)w_k(x_k; \theta), the integral in p(y∣H,θ)p(y \mid\mathcal{H}, \theta) is simplified as

p(y∣H,θ)=w0(θ)h0(y)+∑k=1Kwk(y;θ)hk(y),(1)p(y \mid\mathcal{H}, \theta) = w_0(\theta)h_0(y) + \sum_{k=1}^{K} w_k(y; \theta)h_k(y), \tag*{(1)}

where w0(θ)w_0(\theta) is the integral of w0(x;θ)w_0(x; \theta) with respect to h(x)h(x). This is exactly the outcome-dependent density combination of H={h1,…,hK}\mathcal{H} = \{h_1, \ldots, h_K\} (and baseline h0h_0) as desired. Note also that this form can be viewed as the outcome-independent combination of tilted agent forecasts, whose density kernels are given by wk(y;θ)hk(y)w_k(y; \theta)h_k(y).

Bias-adjusted synthesis: The synthesis function can be parametrized further to increase its predictive flexibility by replacing δxk(y)\delta_{x_k}(y) in the Dirac synthesis with some continuous distribution as

α(y∣x,θ)=w0(x;θ)h0(y)+∑k=1Kwk(x;θ)N(y∣xk−βk,τk2),(2)\alpha(y|x,\theta) = w_0(x;\theta)h_0(y) + \sum_{k=1}^{K} w_k(x;\theta)\mathcal{N}(y|x_k-\beta_k,\tau_k^2), \tag*{(2)}

where N(y∣μ,σ2)\mathcal{N}(y|\mu,\sigma^2) is the density of the normal distribution with mean μ\mu and variance σ2\sigma^2 evaluated at yy. Compared to the Dirac synthesis, this functional form reflects our intention not to take the reported prediction xkx_k from agent hkh_k as-is, but to account for potential bias by βk\beta_k and extra uncertainty by additive noise with variance τk2\tau_k^2. The synthesis with location bias was considered in Johnson and West [14] and extra uncertainty in Johnson [13] in the case of the Gaussian weight specification discussed later. We name this formulation as Bias-adjusted synthesis.

The utility of this formulation is evident when the forecasts are reported as point predictions. Suppose that the kk-th agent reports its forecast not in the form of a probability density, but a point forecast value, y^k\hat{y}_k, with no information on its uncertainty. In the BPS framework, this scenario translates as hk(⋅)=δy^k(⋅)h_k(\cdot) = \delta_{\hat{y}_k}(\cdot). In the Dirac synthesis, the combined density in (1) is simply a mixture of those point masses (although it could be outcome-dependent). However, in the bias-adjusted synthesis, we have

p(y∣H,θ,β,τ2)=w0(y^;θ)h0(y)+∑k=1Kwk(y^;θ)N(y∣y^k−βk,τk2),(3)p(y|\mathcal{H},\theta,\beta,\tau^2) = w_0(\hat{y};\theta)h_0(y) + \sum_{k=1}^{K} w_k(\hat{y};\theta)\mathcal{N}(y|\hat{y}_k-\beta_k,\tau_k^2), \tag*{(3)}

where y^=(y^1,…,y^K)⊤\hat{y} = (\hat{y}_1,\ldots,\hat{y}_K)^\top. In this expression, the point mass distribution is smoothed, similarly to the density estimation with the Gaussian kernels.

Although (β,τ2)(\beta,\tau^2) is supposed to address the bias in point forecast and uncertainty quantification of each agent predictive density, the role of those parameters in fitting data yy overlaps with that of the other parameter θ\theta in the weight function. Put differently, θ\theta and (β,τ2)(\beta,\tau^2) could be unidentifiable. This conflict of the parameters is clarified by marginalizing xx out and rewriting the model in the outcome-independent form:

p(y∣H,θ,β,τ2)=w0(θ)h0(y)+∑k=1K∫wk(x;θ)N(xk∣y+βk,τk2)h(x) dx.p(y|\mathcal{H},\theta,\beta,\tau^2) = w_0(\theta)h_0(y) + \sum_{k=1}^{K} \int w_k(x;\theta)\mathcal{N}(x_k|y+\beta_k,\tau_k^2)h(x)\,dx.

If the weight functions are modeled carefully, both Dirac and bias-adjusted synthesis models should fit to the data equivalently. Nevertheless, we consider the bias-adjusted synthesis for computational reasons; as pointed out later in § 5, the posterior computation under the Dirac synthesis by the MCMC method is complicated due to the degenerate distributions. To mitigate the problem of the identifiability, we use an informative (not diffuse) prior for (β,τ2)(\beta, \tau^2) and maintain the interpretability of the parameter θ\theta and weight functions.

Gaussian-kernel outcome-dependent weights

In this section, we review the Gaussian-kernel specification for outcome-dependent BPS weights proposed by Johnson and West [14], then point out its potentially-undesirable behavior in a large-sample situation.

Gaussian weight functions

For each agent k=1,…,Kk = 1, \ldots, K, we specify the weight function by the Gaussian kernel as

wk(xk;θ)=qkexp⁡{−(xk−μk)22σk2},(4)w_k(x_k;\theta) = q_k \exp\left\{-\frac{(x_k-\mu_k)^2}{2\sigma_k^2}\right\}, \tag*{(4)}

References

  1. [1]Adrian, M. T., D. Giannone, M. Luciani, and M. West (2025). Scenario synthesis and macroeconomic risk. International Monetary Fund Working Paper.arxiv.org/abs/2505.05193
  2. [2]Adrian, T., N. Boyarchenko, and D. Giannone (2019). Vulnerable growth. American Economic Review 109(4), 1263–1289.
  3. [3]Byrd, R. H., P. Lu, J. Nocedal, and C. Zhu (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing 16(5), 1190–1208.
  4. [4]Cabel, D., S. Sugasawa, M. Kato, K. Takanashi, and K. McAlinn (2022). Bayesian spatial predictive synthesis. arXiv preprint arXiv:2203.05197.arxiv.org/abs/2203.05197
  5. [5]Chernis, T. and T. Webley (2022). Nowcasting canadian gdp with density combinations. Technical report, Bank of Canada Staff Discussion Paper.DOI
  6. [6]Davis, P. J. and P. Rabinowitz (1984). Methods of Numerical Integration (2 ed.). Academic Press.
  7. [7]Genest, C. and M. J. Schervish (1985). Modeling expert judgments for Bayesian updating. The Annals of Statistics, 1198–1212.DOI
  8. [8]Genest, C. and J. V. Zidek (1986). Combining probability distributions: A critique and an annotated bibliography. Statistical Science 1(1), 114–135.DOI
  9. [9]Geweke, J. and G. Amisano (2011). Optimal prediction pools. Journal of Econometrics 164(1), 130–141.
  10. [10]Ghosal, S. and A. van der Vaart (2017). Fundamentals of Nonparametric Bayesian Inference. Cambridge: Cambridge University Press.
  11. [11]Gneiting, T. and R. Ranjan (2013). Combining predictive distributions. Electronic Journal of Statistics 7, 1747–1782.arxiv.org/abs/1106.1638
  12. [12]Hall, S. G. and J. Mitchell (2007). Combining density forecasts. International Journal of Forecasting 23(1), 1–13.DOI
  13. [13]Johnson, M. C. (2017). Bayesian predictive synthesis: Forecast calibration and combination. Ph. D. thesis, Duke University.
  14. [14]Johnson, M. C. and M. West (2025). Bayesian Predictive Synthesis with outcome-dependent pools. Statistical Science 40(1), 109–127.arxiv.org/abs/1803.01984
  15. [15]Kapetanios, G., J. Mitchell, S. Price, and N. Fawcett (2015). Generalised density forecast combinations. Journal of Econometrics 188(1), 150–165.
  16. [16]Masuda, R. and K. Irie (2023). Sequential Bayesian predictive synthesis. arXiv preprint arXiv:2308.15910.arxiv.org/abs/2308.15910
  17. [17]McAlinn, K. (2017). Dynamic modeling and Bayesian predictive synthesis. Ph. D. thesis, Duke University.
  18. [18]McAlinn, K., K. A. Aastveit, J. Nakajima, and M. West (2020). Multivariate Bayesian Predictive Synthesis in macroeconomic forecasting. Journal of the American Statistical Association 115(531), 1092–1110.
  19. [19]Polson, N. G., J. G. Scott, and J. Windle (2013). Bayesian inference for logistic models using Pólya-Gamma latent variables. Journal of the American Statistical Association 108(504), 1339–1349.
  20. [20]Pratt, J. W. and R. Schlaifer (1985). Repetitive assessment of judgmental probability distributions: A case study. In J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith (Eds.), Bayesian Statistics 2, pp. 393–424. Amsterdam: North-Holland.
  21. [21]Sugasawa, S., K. Takanashi, K. McAlinn, and E. M. Airoldi (2023). Bayesian causal synthesis for meta-inference on heterogeneous treatment effects. arXiv preprint arXiv:2304.07726.arxiv.org/abs/2304.07726
  22. [22]West, M. (1992). Modelling agent forecast distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology 54, 553–567.DOI
  23. [23]West, M. and J. Crosse (1992). Modelling of probabilistic agent opinion. Journal of the Royal Statistical Society: Series B (Methodological) 54(1), 285–299.

Paper details

vlnc:W_gj86uuu2
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 2

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents