WorksComputer Science
Robust Aggregation of Quantile Statistics: from Median to Interval
Abstract
Quantile-based summaries, such as prediction intervals and medians, are commonly used in economic forecasting, climate science, and public health. For highly skewed samples, experimental evidence suggests that estimates of the mean can be biased toward the median, whereas estimates of the median are relatively accurate. This motivates aggregating explicitly elicited quantiles rather than reported means. We study robust aggregation of quantile reports for bounded outcomes under squared loss, minimizing worst-case regret relative to an omniscient Bayesian aggregator who knows the information structure and observes all private signals. Without restrictions on the information structure, no rule improves on the constant forecast at the range midpoint in worst-case regret. We therefore consider weak substitutes, a diminishing-returns condition on the value of information, and characterize optimal aggregation rules under this condition. For central prediction intervals with common nominal coverage , the optimal rule, which we call regularized simple average for prediction intervals, shrinks the average reported interval midpoint toward the range midpoint, with a weight determined by and the number of experts. This optimal rule strictly improves on the constant baseline in worst-case regret when ; at or above this threshold, the constant rule is optimal. The optimal worst-case regret is minimized at a unique interior coverage level, revealing a nonmonotonic relationship between coverage and aggregation performance. For median reports, regularized simple average for medians similarly shrinks the average reported median toward the range midpoint, assigning weight to the reported average and strictly improving on the constant baseline in worst-case regret for every finite .
Introduction
Suppose you wish to forecast how much U.S. real GDP will grow next year. You consult several experts, each of whom is uncertain about the outcome and communicates this uncertainty through lower and upper forecasts. In the fourth-quarter 2018 Duke CFO survey, for example, respondents were asked for “worst-case” and “best-case” GDP growth estimates associated with a ten-percent chance in each tail [11] (Figure 1). The two tail estimates define a central prediction interval with nominal coverage (the 10th and 90th percentiles). If only these interval reports are available, how should we combine them into a single point forecast? And would asking for narrower or wider intervals lead to a more reliable aggregate? These questions are particularly relevant to online platforms that elicit and aggregate quantile forecasts.

Figure 1. The GDP-growth question from the fourth-quarter 2018 Duke CFO survey [11]. The highlighted ten-percent tail probabilities define the 10th and 90th percentiles, forming a central eighty-percent prediction interval.
Quantile-based summaries, including prediction intervals and medians, are natural ways to communicate subjective uncertainty. They are used in finance [5], climate science [4], public health [6], and energy forecasting [19]. Behavioral evidence provides an additional motivation for studying quantiles directly. As reviewed by Garthwaite et al. [14], experimental evidence on skewed samples suggests that participants’ estimates of sample means were biased toward the corresponding sample medians, whereas their estimates of sample medians were relatively accurate.
However, quantile reports might not contain enough information to yield a point forecast that outperforms even the best constant predictor. Three challenges arise. First, quantile reports can obscure information relevant to prediction under squared loss, for which the posterior mean is optimal. For a binary outcome, for example, posterior probabilities of 0.6 and 0.9 both imply a median report of 1, despite corresponding to different posterior means. Second, the information structure—the joint distribution of the outcome and the experts’ private signals—is unknown. Without this knowledge, we can neither apply the Bayesian combiner nor determine whether experts’ information is redundant or complementary, leaving it unclear how their reports should be combined. Third, when eliciting prediction intervals, the choice of coverage level determines which quantiles are reported and thus what information is available for prediction; how to choose this level is itself an open question.
We study the above challenges under a robust aggregation framework. We consider a bounded outcome, normalized to , and experts who each observe a private signal. Each expert truthfully reports a central posterior prediction interval at a common nominal coverage level . The aggregator observes these reports and knows the reporting format, but not the underlying information structure. Its task is to produce a point forecast evaluated by squared loss. We measure its performance by regret relative to an omniscient Bayesian aggregator who knows the information structure and observes all private signals, and therefore predicts the full-signal posterior mean. We seek an aggregation rule that minimizes the worst-case regret over a class of possible information structures, following the robust forecast aggregation framework [1, 25].
Existing work on robust forecast aggregation has primarily considered posterior probability reports [1, 18, 22] or posterior mean reports [25]. However, these results do not directly determine how to aggregate prediction intervals or medians, nor do they answer which interval coverage level should be elicited.
As in these prior settings, obtaining a nontrivial worst-case guarantee requires restrictions on the information structure. When arbitrary dependence among private signals is allowed, no aggregation rule based on quantile reports can achieve lower worst-case regret than the constant forecast . This is because the general case allows the XOR construction (Example 3.1) which makes each expert’s posterior distribution given their own signal—and hence every quantile report—uninformative even though the signals jointly reveal the outcome.
We focus on information structures satisfying weak substitutes, a diminishing-returns condition on the value of information introduced by Chen and Waggoner [8] and used in forecast aggregation by Neyman and Roughgarden [25]. Informally, this condition implies that the improvement in expected squared loss from observing an additional signal cannot increase as more signals become available. This assumption rules out the pure XOR example while permitting experts’ information to overlap. Under weak substitutes, we derive explicit expressions for the optimal aggregators of interval and median reports, and identify the best interval coverage levels at which the optimal worst-case regret achieves its minimum.
Summary of Results
Our main findings are as follows:
Optimal aggregation of prediction intervals. We characterize an optimal rule, which we call regularized simple average for prediction intervals (Theorem 4.1). It combines the average of the reported interval midpoints with the midpoint of the outcome range, placing a weight on the reports that depends only on the number of experts and the nominal coverage level . Figure 2 compares its worst-case regret with simple averaging of interval midpoints (Corollary 4.3) and with the optimal worst-case regret attainable from posterior-mean reports (Proposition C.1).

Figure 2. Worst-case regret under weak substitutes. SA is robust optimal for posterior-mean reports, and RSA for intervals and medians. Interval bands span the infimum and supremum over ; SA averages interval midpoints. Median SA coincides with the upper orange boundary. The constant 1/2 has worst-case regret 0.25.
A phase transition and an optimal interval reporting level. We show that regularized simple average strictly improves upon the optimal constant forecast in worst-case regret if , whereas the constant forecast is robust optimal when . Moreover, the optimal worst-case regret is minimized at a unique interior coverage level , characterized by , strictly below the phase threshold. Both lower and higher reporting levels yield worse guarantees. In particular, for bounded outcomes, the eighty-percent coverage level used in the motivating survey lies in the constant-optimal regime for any . Figure 3 illustrates the threshold and the optimal reporting level as the number of experts varies (Theorem 4.2).

Figure 3. The phase threshold and the regret-minimizing interval reporting level both decrease with . Pale blue allows strict improvement over the constant rule in worst-case regret; pale orange marks levels where the constant is robust optimal, including the threshold. The regret-minimizing level lies strictly between zero and the threshold.
Optimal aggregation of medians. For median reports, regularized simple average for medians is also optimal. It combines the average reported median with the midpoint of the outcome range, assigning weight to the reported average and the remaining weight to the constant midpoint. This rule strictly improves upon the optimal constant forecast in worst-case regret for every finite . Its optimal weight and worst-case regret agree with the corresponding limits of the interval results as , although central
prediction intervals need not shrink to a single point when the posterior has multiple medians (Theorem 5.1). Figure 2 compares its worst-case regret with that of simple averaging of medians (Corollary 5.2).
Together, these results show that robust aggregation involves not only how to combine experts’ quantile reports, but also which quantiles to ask them to report.
Related Work
Aggregation of quantile statistics. Quantile-based summaries, including prediction intervals and medians, are natural ways to communicate subjective uncertainty. Proper quantile and interval scoring rules provide truthful mechanisms for eliciting such statistics [6, 17]. Recent studies adopt such mechanisms and investigate how to aggregate the elicited quantile statistics and interval forecasts [17, 29]. We focus on aggregation in this paper, and for simplicity, assume truthful reports without specifying an elicitation mechanism.
Given the experts’ quantile reports, existing studies either aggregate them into a point estimate [24, 29], or aggregate them into distributional summaries represented by quantiles, intervals, or full distributions [13, 15, 16, 17, 20, 23, 28]. Regarding the form of aggregation rules, existing studies combine reported values or corresponding quantiles through weighted average [15, 17, 24, 29] or use linear combinations of quantile forecasts with coefficients estimated from historical data [16, 28]. Other approaches pool predictive distributions through pointwise averages or medians of cumulative distribution functions [13, 20, 23] or logarithmic pooling of densities [7]. We characterize a robust optimal point prediction rule, regularized simple average, that shrinks the average of interval midpoints or reported medians toward the fixed midpoint of the outcome range, with a weight given explicitly in terms of the number of experts and, for interval reports, the nominal coverage level.
For theoretical guarantees, existing theoretical work characterizes when quantile aggregation improves on random selection under proper scoring rules [17] and assesses gains from combining forecasts through encompassing tests [16]. Other work derives maximum-likelihood aggregation rules under specific signal models [29]. In this paper, we characterize the robust optimal aggregation rule with minimum worst-case regret relative to the omniscient aggregator under weak substitutes [8, 25], optimizing over all possible aggregation rules.
Robust Aggregation Frameworks. The robust aggregation framework, introduced by Arieli et al. [1], seeks rules with worst-case performance guarantees over a specified class of information structures. In their work, they consider a binary outcome scenario where two Bayesian experts truthfully report their posterior beliefs.
Subsequent work varies the assumptions on information structures and the information available to the aggregator. Several studies assume that experts’ signals are conditionally independent [2, 9, 18]. Other approaches study unknown dependence with known individual information structures [3, 10, 22]. Neyman and Roughgarden Neyman and Roughgarden [25] study posterior-mean reports for general real-valued outcomes under weak substitutes and projective substitutes.
Another line of work varies the information elicited from experts. Second-order reports can improve aggregation [21, 26], while Frongillo et al. [12] develop a general framework of elicitable queries and relate query complexity to aggregation accuracy. To our knowledge, our work is the first to characterize robust optimal aggregation of prediction intervals and medians for general bounded outcomes under squared loss and weak substitutes.
Model
Fix an integer and a nominal coverage level . Write and . The outcome is a random variable taking values in . Each expert observes a private signal taking values in . We use lowercase letters and for their realizations, and write and . An information structure is a joint distribution of the outcome and signals. Expert forms a Bayesian posterior and truthfully reports the central prediction interval
We use this lower generalized quantile convention throughout. The two quantile levels assign the same nominal tail probability on each side. The resulting interval has posterior coverage at least , with possibly larger coverage when the posterior has atoms. A realized interval report is denoted by .
For median reports, we set , using the same symbol for the random report and for its realization. This replaces the two quantile levels by their common limit as . Figure 4 illustrates both reporting formats.

Figure 4. Quantile reports summarize an expert’s posterior belief. For this continuous example, covers the central probability, and the dashed green line is the median . With atoms, the interval’s coverage can exceed .
The aggregator knows and the reporting format, including for interval reports, but not the information structure or the private signals. After observing the report vector , it gives a single point prediction of the outcome. Let denote the space of a single expert’s report under the chosen reporting format. An aggregation rule is a measurable map , with output at a realized report vector .
Benchmark. We evaluate predictions using squared loss. An omniscient aggregator knows both and all private signals, so its optimal prediction is the full-signal posterior mean
Given aggregation rule , its regret relative to this benchmark is
We seek a rule minimizing worst-case regret, which we call a robust optimal rule. For interval reports with nominal coverage level , define the optimal worst-case regret over a class of information structures by
If is unrestricted, no aggregation rule has lower worst-case regret than the constant rule , as the following example shows.
Example 3.1 (XOR information structure). Let be uniformly distributed on and set , where denotes addition modulo two. For , let the remaining signals be constant.
Every expert’s posterior is Bernoulli, so all interval reports are at every ; all lower median reports are 0. Thus the report vector is constant, while the full signals reveal exactly. If a rule outputs at this report vector, then
The constant rule satisfies for every information structure. Together with the XOR lower bound, this shows that the optimal worst-case regret over unrestricted information structures is for either reporting format.
In the XOR example, the first two signals are information complements: neither is informative alone, but together they determine the outcome. To exclude such scenarios, we introduce the weak substitutes condition [8, 25].
Definition 3.2 (Weak substitutes). For , write and define the information value of this coalition by
This is the improvement in expected squared loss from observing , relative to the uninformed prediction ; in particular, . An information structure satisfies weak substitutes if
for every and .
Weak substitutes means that the ex ante marginal value of an additional signal is nonincreasing as the observed coalition grows. Figure 5 illustrates this property in a symmetric information structure, plotting the information value of a coalition’s signals against its size. The following equivalent variance representation justifies this measure of information value and will be used throughout the paper.
Proposition 3.3 (Variance representation of weak substitutes). For , let , so and . Then . Consequently, the weak substitutes condition is equivalent to
caption: Weak substitutes in a symmetric example: each signal independently reveals a fair binary outcome with probability and otherwise reveals nothing, giving .
for every and . In particular, it implies
The proof is given in Appendix A. Let be the class of information structures satisfying weak substitutes, and abbreviate as . When is fixed, we write and for the individual and full-signal posterior means.
Aggregation of Prediction Intervals
For interval reports , write for each reported midpoint and for their simple average.
Theorem 4.1 (Regularized Simple Average for Prediction Intervals). For every and , a robust optimal aggregation rule for in (2) is
Let , so that . The weight is
We call this rule regularized simple average for prediction intervals. Its worst-case regret equals , where
The third branch in both formulas applies only when or . In particular, if and only if .
The rule moves the average reported midpoint toward the center , placing weight on the reports and the remaining weight on . This adjustment accounts for the fact that an interval midpoint need not equal the expert’s posterior mean. At the threshold , every weight in is optimal, and in Equation (5) we select the constant rule. The adjacent formulas for worst-case regret agree at every boundary.
Optimal reporting level. Before proving Theorem 4.1 and explaining the weights , we use the formulas to identify the nominal coverage level that minimizes the worst-case regret.
Theorem 4.2 (Optimal reporting level). For each integer , the optimal worst-case regret has a unique minimizer over , denoted by . This minimizer satisfies
and strictly decreases as increases, with as . Moreover,
The infimum is attained only at , and the supremum is attained exactly when .
Proof. The second branch of Equation (6) has derivative
Its numerator is strictly increasing and changes sign from negative to positive between and . The first branch strictly decreases and the third (when present) strictly increases, so continuity gives the unique minimizer. The supremum follows directly from Theorem 4.1. Substituting Equation (7) gives the minimum value. Finally, is strictly increasing, so its solution at strictly decreases with . Equation (7) gives and , yielding the stated asymptotics.
Low reporting levels can pool weak and strong evidence into the same extreme report, whereas high levels can pool beliefs on opposite sides of into the same wide interval. The optimal reporting level balances these two losses of information. The minimum worst-case regret increases toward as grows. Thus, more expert reports yield a larger worst-case regret relative to the corresponding omniscient aggregator benchmark.
Simple-average comparisons. Taking gives simple averaging of interval midpoints, . Its exact worst-case regret determines the orange band in Figure 2.
Corollary 4.3 (Simple average for prediction intervals). For every and , write for the worst-case regret achieved by the simple average. Then
Consequently,
Appendix C.2 proves the corollary using a method similar to that for Theorem 4.1. Under posterior mean reports, simple average is robust optimal with worst-case regret . The argument of Neyman and Roughgarden [25] implies this optimality result in our bounded-outcome setting. Appendix C.1 gives a direct proof.
We prove Theorem 4.1 in three steps. Section 4.1 considers binary outcomes, deriving the upper bound and explaining the weights in Equation (5). Section 4.2 extends this upper bound to all bounded outcomes in through Lemma 4.6, which also applies to median reports. Section 4.3 then gives matching lower bounds using binary information structures, completing the proof.
Upper Bound for Binary Outcomes
For now suppose . For a weight , we can write the resulting regularized simple average as
where denotes the contribution of expert ’s report. We also write for .
For binary outcomes, each expert must hold a Bernoulli posterior. When expert ’s posterior mean is , using and the lower-quantile convention in Equation (1) we have:
| posterior mean | report | |
{#eq:4.7}
Fix an information structure . Write , , and for the prior mean and the variances of the individual and full-signal posterior means. Jensen’s inequality and give
The weak substitutes condition implies by Proposition 3.3, while . To control the remaining reporting error, we bound it in terms of the posterior variance :
The last term comes from degenerate Bernoulli posteriors or .
Lemma 4.4. For and , inequality (4.9) holds for every Bernoulli posterior with mean and its corresponding contribution if and only if
The two terms come from the extreme-report and central-report regions, respectively. Appendix § 8.§ 1 proves the lemma. For any valid , substituting inequality (4.9) into inequality (4.8) gives
Here ; the second inequality uses both and , with .
For , the coefficient lies in for every . Choosing yields
The two requirements coincide at , apart from , so
For , the first piece decreases with , and the second is a quadratic minimized at . Minimizing these two pieces gives the positive weights in Equation (5). At small levels the quadratic minimum determines the weight. It meets when , giving the transition . At intermediate levels the best weight lies where the two constraints meet; if reaches one before the phase threshold, the best weight is one. These are the three positive-weight cases in Equation (5).
When , both branches in are no less than , indicating that this upper bound no longer yields a strict improvement over the constant rule. The matching lower bound in § 4.3 shows that no aggregation rule can achieve worst-case regret below at or above this threshold. This explains the phase threshold .
Proposition 4.5 (Binary upper bound). For every and , regularized simple average for prediction intervals satisfies
for every with , where the right-hand side is the expression in Equation (6).
Proof. For , substitute the three positive weights in Equation (5) into inequality (4.10). They give the first three branches of Equation (6), respectively. For , the proposed rule is , whose regret is .
Extension to Bounded Outcomes
The core idea is that we can decompose the general posterior over into a family of binary random variables. Specifically, given expert 's posterior , we can construct
for every threshold . Moreover, we have a similar decomposition for the quantiles:
For each threshold , is a binary random variable, and the thresholded endpoints and are the lower quantiles of at levels and , respectively. Then we have
The following lemma transfers a binary posterior inequality to arbitrary posterior distributions supported on through such representations.
Lemma 4.6 (From binary posterior bounds to regret). Fix , , and . Let take values in , set , and suppose a measurable family of predictions satisfies and, for almost every ,
Then
For any , suppose these hypotheses hold for each expert’s conditional posterior and almost every private-signal realization, with report-based prediction and the same and . Then the aggregation rule satisfies
Proposition 4.7 (Upper bound for interval aggregation). For every , , with bounded outcome , and ,
Consequently, regularized simple average for prediction intervals satisfies for every , where the right-hand side is the expression in Equation (6).
For interval reports, Lemma 4.4 supplies inequality (4.13) with , , and . When , we have , so Lemma 4.6 yields upper bound . Above this threshold, the selected rule is constant and its regret is at most . Appendix § 8.§ 3 gives the detailed proof.
Matching Lower Bound
Theorem 4.8 (Secret sharing with a common posterior). For every , , and , there is a finite information structure with a binary outcome such that every aggregation rule satisfies
where the right-hand side uses the expression in Equation (6). If , one such structure satisfies for every .
The construction builds on Shamir’s secret-sharing scheme, where sufficiently many private signals recover the outcome while fewer shares reveal nothing [27]. Neyman and Roughgarden [25] randomize the recovery threshold uniformly to satisfy the weak substitutes condition and obtain the lower bound . In that construction, interval reports introduce no additional information loss relative to posterior means, so the bound falls strictly below .
To create that additional ambiguity and obtain a matching lower bound, we add an additional common signal . Each expert observes , the recovery threshold , and a private share. This construction ensures that each expert’s posterior is Bernoulli, while all signals together reveal . We choose the distribution of to hide as much information as possible within each report category. Conditional on and , the private shares can exhibit threshold complementarities, while the ex ante coalition information values satisfy weak substitutes. The aggregator cannot distinguish different posterior means that induce the same reports, indicating a lower bound on the regret relative to the omniscient benchmark which predicts exactly in this construction. By choosing the distribution of appropriately, we can match every branch of Equation (6). Appendix B.4 gives the full construction.
Combining Theorem 4.8 with Proposition 4.7 proves Theorem 4.1. Appendix B.5 gives the complete argument.
Aggregation of Medians
For median reports , write for their simple average. Denote the robust optimal worst-case regret by
Theorem 5.1 (Regularized Simple Average for Medians). For every , a robust optimal aggregation rule for median reports under weak substitutes is
where . We call this rule regularized simple average for medians. Its worst-case regret is
The same rule and worst-case regret apply under any method for selecting a median based on each expert’s signal, including when the worst case also ranges over such selections.
As for prediction intervals, regularized simple average combines the average report with the center . The rule improves on the optimal constant prediction in worst-case regret by for every finite .
Relation to prediction intervals. The optimal weight and worst-case regret agree with the corresponding limits in Theorem 4.1:
Nevertheless, taking the limit in the interval theorem does not by itself prove the median result. For a fair Bernoulli posterior, every positive-level central interval is , whereas the lower generalized median is . If every expert has this posterior, regularized simple average for prediction intervals always outputs , while regularized simple average for medians outputs . We therefore establish the median result directly.
Simple-average comparisons. Taking gives simple averaging of medians, . Its exact worst-case regret determines the median SA curve in Figure 2, which coincides with the upper interval-SA boundary in Equation (9).
Corollary 5.2 (Simple averaging for medians). For every , simple averaging has exact worst-case regret
under any median-selection method.
Appendix § 9.§ 3 proves the corollary.
The proof of Theorem 5.1 follows the same structure as the interval analysis in § 4. § 5.1 first establishes a binary posterior inequality analogous to inequality (11), then applies Lemma 4.6 to extend it to bounded outcomes and derive the upper bound. § 5.2 obtains a matching lower bound by choosing a different distribution for the common signal in the secret-sharing construction of § 4.3. § 5.3 combines the two bounds to prove Theorem 5.1.
Upper Bound
Fix and . Define the contribution of expert ’s median report and the resulting aggregation rule by
Binary posterior inequality. First focus on binary outcomes. For a Bernoulli posterior with mean , the unique median is zero if and one if ; when , every is a median. For every such median report, we have
This is the median counterpart of inequality (11) with .
Extension to bounded outcomes. Lemma 4.6 transfers inequality (21) to bounded outcomes with and .
Proposition 5.3 (Upper bound for median aggregation). For every with bounded outcome and every ,
This bound holds for any median selection method.
Proof. Fix . For each expert’s posterior and any reported median , define
If , then ; if , then . Hence is a median of , including when that binary posterior has multiple medians. Threshold integration and the affine form of give
Applying Lemma 4.6 with inequality (5.3), , , and gives inequality (5.4). The argument applies to any signal-dependent median selection, so the bound holds for every such method.
Matching Lower Bound
Proposition 5.4 (Lower bound for median aggregation). For every and , there is a finite binary information structure with unique posterior medians such that every aggregation rule satisfies
Proof. We use the common-posterior construction of § 4.3, choosing the following law of to hide different posterior probabilities behind identical median reports.
Fix , and put and . Define
This common posterior mean has expectation and variance
Applying the common-posterior construction of Appendix § 8.§ 4.1 to this law yields a finite information structure with for every expert and . Since , Lemma B.1 ensures that .
Every singleton posterior has a unique median: zero when and one when . Consequently, the only report vectors are and , each with probability . Denote and for an arbitrary aggregation rule . Since , the regret satisfies
The lower bound is independent of . For every , sufficiently small therefore gives the required finite structure in simultaneously for all rules. Its unique medians make the construction independent of the median selection method.
Proof of Theorem 5.1
Proof. Choosing in Proposition 5.3 gives the rule in (19) and the upper bound
The rule lies in by convexity. Proposition 5.4 implies that every rule has worst-case regret at least for every . Letting and taking the infimum over rules yields the matching lower bound for . Together these bounds prove Equation (20) and optimality of the regularized simple average in Equation (19). The upper bound allows any signal-dependent median selection, while the lower-bound structures have unique medians. Thus the result holds for each median selection method and when the worst case also ranges over such methods.
Discussion and Future Work
We characterize robust optimal rules for interval and median reports and show how interval coverage affects optimal worst-case regret.
There are several future directions. First, we assume truthful reports and central prediction intervals at a common nominal coverage level. Future work could study asymmetric intervals or heterogeneous reporting levels across experts. Second, we mainly focus on the weak substitutes condition. Considering more general information structures could provide a richer representation of the aggregator’s knowledge. Finally, the proposed rules have not been evaluated empirically. Robust optimality concerns worst-case regret over the class of information structures, while other rules may achieve lower regret in particular structures. Experiments with real-world data could evaluate how the proposed rules compare with alternatives in practice.
Proof of Proposition 3.3
Proof. Fix and write . For every , the tower property gives . Since is determined by and , we have
Expanding therefore yields
By the definition of and , it follows that
To obtain inequality (3), let for and . Apply weak substitutes with and :
Summing these inequalities and using , we obtain
The variance representation now gives inequality (3).
Proofs for Optimal Interval Aggregation
Proof of Lemma 4.4
Proof. Fix expert ’s Bernoulli posterior with mean and corresponding interval report . In the low-report region , we have . Subtracting the right-hand side of inequality (11) from the left-hand side gives
This is nonpositive throughout the region exactly when . Necessity follows by taking . The high-report region gives the same condition after replacing by .
For the central report , we have , and the corresponding difference is
The smallest posterior variance in this region is , attained at . Hence the equivalent condition for the central-report region is . Combining these two conditions gives .
Proof of Lemma 4.6
Proof. On a fixed probability space, define the residual
The second expression is jointly convex in because . Since and , inequality (15) and Jensen’s inequality yield
which proves inequality (16). For the regret bound, fix and write , , , , and . The decomposition in inequality (10) also holds for bounded outcomes and contributions :
Apply inequality (16) to each expert’s conditional posterior, take expectations over that expert’s signal realizations, and then average over the experts. By the law of total variance, the averaged posterior variance is , while the averaged term equals . Thus
The second inequality uses from weak substitutes and . The third uses and , and the last uses .
Proof of Proposition 4.7
Proof. Fix , , , and . Set and . For each expert’s posterior, the thresholded endpoints in Section 4.2 are valid binary interval reports, so Lemma 4.4 verifies inequality (15) with . Equation (14) gives the required representation of . Lemma 4.6 therefore yields
which is inequality (18). For , substituting the weights in Equation (5) gives the first three branches of Equation (6). For , the selected rule is constant and has regret because . Thus the proposed rule has the stated upper bound for every .
Proof of Theorem 4.8
We first define a common-posterior information structure and characterize when it satisfies weak substitutes. We then give the specific choice of the distribution of for the low and intermediate reporting levels and for the midpoint and constant regions, respectively. Combining these constructions proves Theorem 4.8.
A common-posterior information structure
We adapt the construction of Neyman and Roughgarden [25, Proposition 2.5] by adding a common signal . Let have finite support and , and draw . Independently of , draw uniformly from . Fix a prime and, conditional on , draw independent uniform coefficients , independently of . Following Shamir’s scheme [27], define
Let be the resulting joint distribution of . Every expert observes and together with a private share ; the aggregator observes only the reports.
Lemma B.1 (Weak substitutes condition). In the common-posterior construction above, , each expert’s posterior is , and the full set of signals reveals . Hence for every and .
For every nonempty , its information value is
with . Therefore, the information structure defined above satisfies weak substitutes if and only if
Proof. Any distinct shares recover by polynomial interpolation. For fewer than shares, the map from the uniform coefficients to the evaluations has full row rank, so they provide no further information about conditional on . Hence, for every nonempty coalition of size ,
Since , each expert’s posterior is , and since , . Both and have mean , and . Independence of from and therefore give Equation (B.1) by Proposition 3.3.
The information value depends only on coalition size. Its first marginal increment is , and every subsequent increment is . Thus the weak substitutes condition holds if and only if
The low and intermediate reporting levels
Lemma B.2. If , then for every there is a finite information structure with a binary outcome such that for every aggregation rule .
Proof. Set
Since , these choices give , , and
For sufficiently small , put and , and choose to satisfy
Equation (B.2) implies , so for all sufficiently small . Let have probabilities , , on low, mid, high, respectively, and define
Zero-probability categories are omitted. Symmetry gives , and Equation (B.3) gives
for every such .
Use this law of in the construction of Appendix § 8.§ 4.1, and denote the resulting structure by . Lemma B.1 gives , , , and . Since , every expert reports , , or according to low, mid, high. The conditional outcome means are , , and , respectively. Since , every aggregation rule satisfies
As , . The limiting lower bound is
These are the first two branches of Equation (6). For any , one sufficiently small therefore gives simultaneously for every .
The midpoint and constant regions
Lemma B.3. For every , , and , there is a finite information structure with a binary outcome such that every aggregation rule satisfies
If , one such structure gives for every . Consequently,
Proof. Choose and define
where
Then and . The construction of Appendix § 8.§ 4.1 and Lemma B.1 give a finite structure with and . All experts report [0, 1] if and only if . This event happens with probability , and conditional on this event, is Bernoulli. Thus, writing , every rule satisfies
As , this lower bound tends to
For any , choosing sufficiently close to , independently of , gives the stated lower bound for every rule. If , some permits , so one structure gives regret at least $1/4. □
Proof of Theorem 4.8. If , Lemma B.2 supplies the required structure for the first two branches of Equation (6). If , Lemma B.3 gives the third branch. For , the same lemma gives the last branch, including the approximating sequence at equality and the single structure above the threshold. These cases cover every , and each chosen structure works simultaneously for all aggregation rules. □
Proof of Theorem 4.1
Proof. Proposition 4.7 bounds the worst-case regret of by the expression in Equation (6). Theorem 4.8 bounds every rule’s worst-case regret below by the same expression minus any , using binary structures within . Letting and taking the infimum over rules proves Equation (6) and optimality of the rule in Equation (4).
The first three branches of Equation (6) are strictly below , and adjacent expressions agree at their boundaries. At , Proposition 4.7 and Equation (13) give the upper bound for every ; the matching lower bound makes each of these weights optimal. □
Simple Average Comparisons
This appendix establishes the simple-average benchmarks in Figure 2. All worst cases range over the bounded-outcome class , and regret is measured against the omniscient aggregator predicting the full-signal posterior mean .
Simple Average of Posterior Means
Proposition C.1 (Simple average for posterior means). When each expert reports , the simple average is robust optimal, with
Here ranges over all measurable rules for aggregating the reported posterior means.
Proof. For each expert’s posterior , set and . Then , and the binary certificate in Lemma 4.6 holds with and . Its regret bound gives
For the matching lower bound, apply the common-posterior information structure in Appendix § 8.§ 4.1 with common posterior mean distributed as follows:
Then and , so Lemma B.1 gives a finite information structure with and . On the event , the report vector is and the conditional outcome variance is . Thus every aggregation rule incurs regret at least .
Simple Average of Interval Midpoints
Proof of Corollary 4.3. Write for simple averaging of interval midpoints, and set .
Upper bound. Taking in Lemma 4.4 gives
Whenever , the binary certificate and threshold representation in § 4.2 satisfy Lemma 4.6 with and . Hence, for every with bounded outcome,
For , the required condition holds because . For , it also holds when .
A second certificate covers the remaining reporting levels. For a Bernoulli posterior with mean and the midpoint of the reported interval, the three report categories in Equation (4.7) give
Thus the binary squared loss is at most . Equivalently, its squared prediction bias is at most , which is the certificate in Lemma 4.6 with . The midpoint threshold representation in Equation (14) with lets us apply that lemma directly, yielding
for every with bounded outcome.
For , combining the two certificates gives
For , the second certificate gives . The first certificate improves this bound exactly when , where it is valid, so
Matching lower bounds. Reuse the distribution of in Equation (B.4) from the proof of Lemma B.3 with
Then and . The construction of Appendix § 8.§ 4.1 and Lemma B.1 therefore give a finite structure with and .
If , choose . All experts report when , and report when . Thus, as ,
If , choose . All experts report when , and report when . Thus, as ,
Together with the upper bounds, the two limits prove Equation (8).
Range over reporting levels. Both expressions inside the minimum in the first branch of Equation (8) strictly decrease on . The second branch is nondecreasing on , increasing until it reaches . The branches agree at , with value . This proves the infimum in Equation (9).
For the supremum, the first branch has limit as and lies strictly below this limit, while the second is at most . This proves the supremum in Equation (9). □
Simple Average of Medians
Proof of Corollary 5.2. Taking in inequality gives the upper bound for simple averaging:
for every .
For the matching lower bound, use the finite structures from the proof of Proposition 5.4. Under these information structures, the only median-report vectors are and , each with probability . Simple average sets and in each case. Hence
as .
The unique medians make this matching lower bound valid for every selection method. □
References
- [1]Itai Arieli, Yakov Babichenko, and Rann Smorodinsky. Robust forecast aggregation. Proceedings of the National Academy of Sciences, 115(52):E12135–E12143, 2018. doi: 10.1073/pnas.1813934115. URL https://www.pnas.org/doi/abs/10.1073/pnas.1813934115.DOI
- [2]Itai Arieli, Yakov Babichenko, and Rann Smorodinsky. Identifiable information structures. Games and Economic Behavior, 120:16–27, 2020. ISSN 0899-8256. doi: https://doi.org/10.1016/j.geb.2019.12.006. URL https://www.sciencedirect.com/science/article/pii/S0899825619301824.
- [3]Itai Arieli, Yakov Babichenko, Inbal Talgam-Cohen, and Konstantin Zabarnyi. A random dictator is all you need. American Economic Journal: Microeconomics, 17(1):66–96, February 2025. doi: 10.1257/mic.20230255. URL https://www.aeaweb.org/articles?id=10.1257/mic.20230255.DOI
- [4]J. L. Bamber, M. Oppenheimer, R. E. Kopp, W. P. Aspinall, and Roger M. Cooke. Ice sheet and climate processes driving the uncertainty in projections of future sea level rise: Findings from a structured expert judgement approach. Earth’s Future, 10(10):e2022EF002772, 2022. doi: https://doi.org/10.1029/2022EF002772. URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2022EF002772.
- [5]Itzhak Ben-David, John R. Graham, and Campbell R. Harvey. Managerial miscalibration*. The Quarterly Journal of Economics, 128(4):1547–1584, 11 2013. ISSN 0033-5533. doi: 10.1093/qje/qjt023. URL https://doi.org/10.1093/qje/qjt023.
- [6]Johannes Bracher, Evan L. Ray, Tilmann Gneiting, and Nicholas G. Reich. Evaluating epidemic forecasts in an interval format. PLOS Computational Biology, 17(2):1–15, 02 2021. doi: 10.1371/journal.pcbi.1008618. URL https://doi.org/10.1371/journal.pcbi.1008618.
- [7]Fabio Busetti. Quantile aggregation of density forecasts. Oxford Bulletin of Economics and Statistics, 79(4):495–512, 2017. doi: https://doi.org/10.1111/obes.12163. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/obes.12163.
- [8]Yiling Chen and Bo Waggoner. Informational substitutes. In 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), pages 239–247, 2016. doi: 10.1109/FOCS.2016.33.DOI
- [9]Zhi Chen, Cheng Peng, and Wei Tang. Prior-agnostic robust forecast aggregation, 2026. URL https://arxiv.org/abs/2604.24517.
- [10]Henrique de Oliveira, Yuhta Ishii, and Xiao Lin. Robust aggregation of correlated information. The Review of Economic Studies, page rdag117, 10 2026. ISSN 0034-6527. doi: 10.1093/restud/rdag117. URL https://doi.org/10.1093/restud/rdag117.
- [11]Duke University’s Fuqua School of Business. Duke CFO Global Business Outlook Survey: U.S. questionnaire, fourth quarter 2018, 2018. URL https://cfosurvey.fuqua.duke.edu/2018q4/survey/index.htm. Question 6.
- [12]Rafael Frongillo, Mary Monroe, Eric Neyman, and Bo Waggoner. Robust forecast aggregation via additional queries, 2025. URL https://arxiv.org/abs/2512.05271.
- [13]Anil Gaba, Ilia Tsetlin, and Robert L. Winkler. Combining interval forecasts. Decision Analysis, 14(1):1–20, 2017. doi: 10.1287/deca.2016.0340. URL https://doi.org/10.1287/deca.2016.0340.
- [14]Paul H Garthwaite, Joseph B Kadane, and Anthony O’Hagan. Statistical methods for eliciting probability distributions. Journal of the American Statistical Association, 100 (470):680–701, 2005. doi: 10.1198/016214505000000105. URL https://doi.org/10.1198/016214505000000105.
- [15]Christian Genest. Vincentization Revisited. The Annals of Statistics, 20(2):1137 – 1142, 1992. doi: 10.1214/aos/1176348676. URL https://doi.org/10.1214/aos/1176348676.
- [16]Raffaella Giacomini and Ivana Komunjer. Evaluation and combination of conditional quantile forecasts. Journal of Business & Economic Statistics, 23(4):416–431, 2005. doi: 10.1198/073500105000000018. URL https://doi.org/10.1198/073500105000000018.
- [17]Yael Grushka-Cockayne, Kenneth C. Lichtendahl, Victor Richmond R. Jose, and Robert L. Winkler. Quantile evaluation, sensitivity to bracketing, and sharing business payoffs. Operations Research, 65(3):712–728, 2017. doi: 10.1287/opre.2017.1588. URL https://doi.org/10.1287/opre.2017.1588.
- [18]Yongkang Guo, Jason D. Hartline, Zhihuan Huang, Yuqing Kong, Anant Shah, and Fang-Yi Yu. Algorithmic robust forecast aggregation. In Proceedings of the 26th ACM Conference on Economics and Computation, EC ’25, page 1110–1129, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400719431. doi: 10.1145/3736252.3742674. URL https://doi.org/10.1145/3736252.3742674.
- [19]Tao Hong, Pierre Pinson, Shu Fan, Hamidreza Zareipour, Alberto Troccoli, and Rob J. Hyndman. Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond. International Journal of Forecasting, 32(3):896–913, 2016. ISSN 0169-2070. doi: https://doi.org/10.1016/j.ijforecast.2016.02.001. URL https://www.sciencedirect.com/science/article/pii/S0169207016000133.
- [20]Stephen C. Hora, Benjamin R. Fransen, Natasha Hawkins, and Irving Susel. Median aggregation of distribution functions. Decision Analysis, 10(4):279–291, 2013. doi: 10.1287/deca.2013.0282. URL https://doi.org/10.1287/deca.2013.0282.
- [21]Yuqing Kong. Peer expectation in robust forecast aggregation: The possibility/impossibility, 2024. URL https://arxiv.org/abs/2402.06062.
- [22]Gilat Levy and Ronny Razin. Combining forecasts in the presence of ambiguity over correlation structures. Journal of Economic Theory, 199:105075, 2022. ISSN 0022-0531. doi: https://doi.org/10.1016/j.jet.2020.105075. URL https://www.sciencedirect.com/science/article/pii/S0022053118303144. Symposium Issue on Ambiguity, Robustness, and Model Uncertainty.
- [23]Kenneth C. Lichtendahl, Yael Grushka-Cockayne, and Robert L. Winkler. Is it better to average probabilities or quantiles? Management Science, 59(7):1594–1611, 2013. doi: 10.1287/mnsc.1120.1667. URL https://doi.org/10.1287/mnsc.1120.1667.
- [24]Aidan Lyon, Bonnie C. Wintle, and Mark Burgman. Collective wisdom: Methods of confidence interval aggregation. Journal of Business Research, 68(8):1759–1767, 2015. ISSN 0148-2963. doi: https://doi.org/10.1016/j.jbusres.2014.08.012. URL https://www.sciencedirect.com/science/article/pii/S0148296315001526. Special Issue on Simple Versus Complex Forecasting.
- [25]Eric Neyman and Tim Roughgarden. Are you smarter than a random expert? the robust aggregation of substitutable signals. In Proceedings of the 23rd ACM Conference on Economics and Computation, EC ’22, page 990–1012, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450391504. doi: 10.1145/3490486.3538243. URL https://doi.org/10.1145/3490486.3538243.
- [26]Yuqi Pan, Zhaohua Chen, and Yuqing Kong. Robust decision aggregation with second-order information. In Proceedings of the ACM Web Conference 2024, WWW ’24, page 77–88, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400701719. doi: 10.1145/3589334.3645384. URL https://doi.org/10.1145/3589334.3645384.
- [27]Adi Shamir. How to share a secret. Commun. ACM, 22(11):612–613, November 1979. ISSN 0001-0782. doi: 10.1145/359168.359176. URL https://doi.org/10.1145/359168.359176.
- [28]James W. Taylor and Derek W. Bunn. Combining forecast quantiles using quantile regression: Investigating the derived weights, estimator bias and imposing constraints. Journal of Applied Statistics, 25(2):193–206, 1998. doi: 10.1080/02664769823188. URL https://doi.org/10.1080/02664769823188.
- [29]Johan Ugander, Ryan Drapeau, and Carlos Guestrin. The wisdom of multiple guesses. In Proceedings of the Sixteenth ACM Conference on Economics and Computation, EC ’15, page 643–660, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450334105. doi: 10.1145/2764468.2764529. URL https://doi.org/10.1145/2764468.2764529.