WorksComputer Science
A Matching Lower Bound for Online Smooth Calibration
Abstract
We study smooth calibration error in sequential binary prediction, a measure of how far probabilistic forecasts are from being calibrated that is robust to small perturbations of the forecasts. The minimax smooth calibration error is known to lie between and (Qiao and Zheng, 2024), where is the number of rounds. We close this gap for both adaptive and oblivious adversaries. We construct an adaptive adversary whose outcome mean moves with the forecaster's cumulative residual, which makes compensating for past residuals costly and forces every forecaster to incur expected smooth calibration error. Simulating this adversary against the forecaster's average prediction yields an oblivious adversary with the same guarantee. These lower bounds match the known upper bound up to universal constants, and are therefore minimax optimal. Our results also imply matching bounds for calibration distance and lower calibration distance. The proof ideas are partially developed by GPT-6 Astra under the authors' guidance.
Introduction
We study the sequential binary forecasting problem of Foster and Vohra [9]. Over rounds, a forecaster and an adversary interact as follows. At each round , the forecaster predicts a probability that the next binary outcome equals 1, and the adversary simultaneously selects the outcome . Both and are then revealed, and may factor into the subsequent actions of both players. This problem abstracts many applications in which probabilistic predictions are issued sequentially, including weather forecasting ([3, 6]), uncertainty quantification of machine learning models ([12]), fair risk prediction ([16]), and forecasting for downstream decision makers ([19]).
A natural requirement for the predictions to be interpretable as probabilities is calibration (Dawid, [6]): among the rounds on which the forecaster predicts a value , the outcome should equal 1 in an fraction of them. Formally, the forecaster is perfectly calibrated if for every . The most widely used measure of the violation of this condition is the calibration error, also known as the expected calibration error (ECE),
where only the values contribute to the sum. However, ECE is discontinuous in the predictions. Suppose that contains equally many 0s and 1s. Then predicting in every round gives zero ECE. If each is instead perturbed to for tiny, distinct, nonzero , then every round falls into its own bin, and ECE jumps to , although the predictions are essentially unchanged.
This sensitivity motivates the smooth calibration error [18, 2], which tests the residuals against all bounded Lipschitz functions of the predictions,
where is the class of 1-Lipschitz functions from to . Since nearby predictions receive nearby weights, smCE is robust to small perturbations of the predictions, and in the example above it remains small.
For smooth calibration error, however, a gap remains in the sequential setting. Qiao and Zheng [22] gave a forecaster that achieves smooth calibration error against any adversary. It might appear obvious that this rate is tight, since independent fair outcomes make the cumulative residual of order when the forecaster predicts . Surprisingly, Qiao and Zheng [22] showed that this argument is incorrect: against independent fair outcomes, a forecaster can compensate for past residuals with slightly biased predictions, and achieve only polylogarithmic smooth calibration error. Their adversary circumvents this by generating fair outcomes until the error is large and then stopping early with constant outcomes, which yields an lower bound. This leaves the following question:
In this paper, we answer this question affirmatively, against both adaptive and oblivious adversaries. We first construct a new adaptive adversary whose outcome mean moves with the forecaster’s cumulative residual, which makes compensating for past residuals costly, and show that it forces every forecaster to incur expected smooth calibration error, improving the lower bound of Qiao and Zheng [22] and matching the known upper bound. We further convert this adversary into an oblivious one, which simulates the interaction against the forecaster’s average prediction before the game, and show that the same lower bound holds even when the outcome sequence is fixed in advance. Since smooth calibration error lower bounds both calibration distance and lower calibration distance up to a constant factor, our results also imply matching bounds for these two measures in both settings. A comparison of our results with directly related previous results is summarized in Table 1.
Table 1. Comparison of upper and lower bounds on the minimax expected calibration error in the simultaneous forecasting protocol. All of the upper bounds and our lower bounds hold against both adaptive and oblivious adversaries. In this table, is the total number of interactions, smCE stands for smooth calibration error, LowerCalDist stands for lower calibration distance, and CalDist stands for calibration distance. The lower bounds on lower calibration distance and calibration distance are direct consequences of the lower bound on smooth calibration error, as detailed in Remark 3.2.
Notation. For a positive integer , let . We write and for expectation and probability, and for an indicator. We use calligraphic letters for sets and strategies. The notation and hides universal positive constants, and denotes matching bounds in both directions. We write if for a universal constant , and if and .
Related Work
Sequential Calibration. Sequential calibration studies probabilistic forecasting of binary outcomes, where the forecaster issues a prediction before observing each outcome and the outcomes may depend on previously revealed predictions. Foster and Vohra [9] established the existence of randomized forecasters that achieve asymptotic calibration. Foster [8] subsequently gave an algorithm based on Blackwell’s approachability theorem. For the classical calibration error, these approaches yield an expected error, and Hart [14] gave a direct minimax proof of this rate. Dagan et al. [5] improved the upper bound to for a universal constant . For smooth calibration error and calibration distance, Qiao and Zheng [22] established an expected error. Arunachaleswaran et al. [1] subsequently proposed the Almost-One-Step-Ahead algorithm, an explicit deterministic forecaster that guarantees calibration distance at most on every outcome sequence. Calibration has also been studied across multiple subpopulations ([16, 13]), in omniprediction ([10]), and for forecasts used by downstream decision makers ([19, 17]), including guarantees based on smooth calibration ([15, 11]).
Calibration Lower Bounds and Adversary Design. For the classical calibration error, independent fair outcomes already yield an lower bound. Qiao and Valiant [21] strengthened this to via early stopping and sidestepping. Dagan et al. [5] further obtained an lower bound with an oblivious adversary, whose outcomes are generated independently of the forecaster’s predictions. For smooth calibration error and calibration distance, Qiao and Zheng [22] proved an lower bound with an adversary that generates independent fair bits until a history-dependent stopping time and then repeats a fixed outcome. Our adversary retains the constant continuation after stopping, but replaces the fair bits with Bernoulli outcomes whose mean depends on the cumulative forecasting residual. The extension of this lower bound to oblivious adversaries follows the standard reduction from adaptive to oblivious adversaries in online learning ([4]), which does not apply to calibration error in general, since no deterministic forecaster achieves sublinear ECE ([20, 7]).
Preliminaries
We consider an online binary forecasting game between a forecaster and an adversary over rounds. At each round , the forecaster predicts a probability and the adversary simultaneously selects an outcome , after which both are revealed. The prediction may be randomized and may depend on the history , but not on the current outcome . Formally, a forecaster strategy consists of an internal random variable , drawn before the game independently of the adversary, and deterministic maps , such that the prediction at round is , where and ; in particular, by induction, is a deterministic function of and . Therefore, for simplicity, we drop the dependence on the previous predictions and write in place of . Depending on how the adversary generates the revealed outcome , we consider two settings.
In the adaptive setting, may be randomized and may depend on the history , but not on the current prediction .
In the oblivious setting, the adversary fixes the entire outcome sequence before the game, without observing the forecaster’s predictions, and reveals one outcome per round. In particular, cannot depend on any of the forecaster’s predictions.
We write and for the outcome and forecast sequences. The forecaster aims to keep the smooth calibration error ([18, 2]) of the realized sequences small. Our goal is to characterize the minimax smooth calibration error in both settings,
where ranges over all forecaster strategies and denote the sets of adaptive and oblivious adversary strategies, respectively, and is generated by the interaction between and . That is, is taken over both the internal randomness of and the randomness of . In particular, for a fixed outcome sequence , we write for the oblivious adversary that reveals , in which case is taken over only. Since every oblivious adversary is a special case of an adaptive one, i.e., , we have . In both settings, is the smooth calibration error defined as follows.
Definition 3.1 (Smooth calibration error). Let be the class of all 1-Lipschitz functions from to . The smooth calibration error of is
Definition 3.1 requires the forecaster to be calibrated around every value , while treating nearby forecasts as essentially the same. Specifically, for any fixed , among the rounds in which the forecaster predicts approximately , the outcome should equal 1 in approximately a fraction of them. Suppose this fails around some , say the residuals add up to a large positive amount. Then a test that is a small positive bump centered around and vanishes elsewhere collects this residual while ignoring all other rounds, so is large. On the other hand, the Lipschitz constraint prevents from fluctuating rapidly near . Consequently, the residuals for all are effectively grouped together, and an over-prediction at one value can be offset by an under-prediction at a nearby value. This makes robust to small perturbations of the forecasts, in contrast to the classical expected calibration error (ECE), which treats every forecast value as a separate bin and can change drastically under arbitrarily small perturbations. In short, smCE asks the forecaster to be calibrated at every scale coarser than the resolution of its forecasts, which is the property that matters when forecasts are used by downstream decision makers with smooth responses.
Remark 3.2 (Distance-based measures). Besides smCE, two distance-based calibration measures have been studied: the calibration distance and the lower calibration distance [22] (Błasiok et al. 2023). The calibration distance is the distance from to the nearest perfectly calibrated forecast sequence,
where is the set of perfectly calibrated forecast sequences. The lower calibration distance relaxes this by allowing randomized repairs,
where is the set of all such that each is a finitely supported distribution on and for every . Since every corresponds to the point masses in , we have . Moreover, Błasiok et al. (2023, Theorem 7.3) implies . Therefore, any lower bound on smCE yields a lower bound of the same order on both LowerCalDist and CalDist.
Adversaries and Lower Bounds
In this section, we show that the upper bound is tight against both adaptive and oblivious adversaries. In Section 4.1, we construct a single adaptive adversary that forces every forecaster to incur expected smooth calibration error. In Section 4.2, we further extend this adversary, together with the induced lower bound, to oblivious adversaries.
Adaptive Adversary
We summarize the adversary in Algorithm 1, which follows the early-stopping design of Qiao and Zheng [22]. In particular, the adversary keeps track of the cumulative residual
and operates in two phases separated by the threshold : an active phase, in which the outcomes are random, and a frozen phase, in which the outcomes are constant. We explain the design of each phase in turn.
Freezing a Large Residual. We start with the easy case. Note that is the score of the constant test and is the score of , so that smCE. Hence, once the residual reaches the threshold, i.e., at some round , the adversary only needs to keep it there until the end. This is achieved by raising the flag frozen and repeating the constant outcome in all remaining rounds. If , every later increment is nonnegative whatever the forecaster predicts, and symmetrically for . Therefore never decreases after the crossing, and smCE.
Random Outcomes Force the Forecaster to Adjust. It remains to handle forecasters that keep the residual below the threshold. In the active phase, the adversary draws using fresh randomness, so that the residual behaves like a random walk. To see this, let be the conditional variance of . Ignoring the frozen phase for the moment and summing the expected increments of over all rounds, we obtain
When is bounded away from 0 and 1, is bounded away from zero, so the first sum is of order . Hence, if the forecaster makes no adjustment, for example, , then the residual grows to order and reaches the threshold with constant probability. The only way to keep the residual small is to make the cross term in (3) negative, that is, to choose such that has the opposite sign of , which pushes the residual back toward zero.
Penalizing the Adjustment. Such adjustment is detected by the affine test . Indeed, conditioning on the history and the current prediction, we have
In Qiao and Zheng [22], the outcomes in the active phase are fair bits with , so the adjustment is penalized only by the quadratic term . Being second order, this penalty is mild, and the forecaster can control the residual through many small adjustments at little cost. To make the penalty harsher, we let the mean depend on the residual,
so that has the opposite sign of . The second term in (4) then becomes , which is positive precisely when the forecaster adjusts. In other words, the cross term that the forecaster must make negative to control the residual in (3) reappears with the opposite sign in the affine score. This intensified penalty is the key to the sharper lower bound.
Lower Bounds under Adaptive Adversary
The following theorem presents the lower bound induced by the adaptive adversary in Algorithm 1. Its full proof is deferred to Appendix § 7.§ 1.
Algorithm 1 Residual-Dependent Adversary
1: Input: horizon ; set , 2: ; frozen false 3: for do 4: if frozen then 5: 6: else 7: Draw 8: end if 9: Reveal , observe the forecaster’s prediction , and set 10: if frozen = false and then 11: frozen true; 12: end if 13: end for
Theorem 4.1 (Lower bound against adaptive adversaries). For every integer , let denote the adversary in Algorithm 1. Then every randomized forecasting strategy satisfies
Consequently, the minimax smooth calibration error against adaptive adversaries satisfies .
Theorem 4.1 shows that the adversary in Algorithm 1 forces every forecaster to incur expected smooth calibration error. This lower bound matches the upper bounds of Qiao and Zheng [22]; Arunachaleswaran et al. [1] up to a universal constant, and thus characterizes the minimax rate of smooth calibration error against adaptive adversaries.
Proof Overview. We now give a brief overview of why this adversary achieves the lower bound. Adding times (3) to the sum of (4) over all rounds cancels the cross terms and roughly leaves
whatever the forecaster does, where we drop the nonnegative terms involving . Consequently, every forecaster faces a dichotomy: either the residual reaches the threshold and the frozen phase forces error at least , or the game stays in the active phase for most of the rounds and the affine test collects a score of order . As a result, the forecaster always incurs an error of order at least , and choosing yields the desired rate.
Oblivious Adversary
In this subsection, we extend the adaptive adversary of Section 4.1 to the oblivious setting. At a high level, the adversary simulates the entire interaction between Algorithm 1 and the forecaster Algorithm 2 Oblivious Residual-Dependent Adversary
1: Input: horizon and forecasting strategy ; set , 2: // Simulation before the game 3: ; frozen false 4: for do 5: Set if frozen, and draw otherwise 6: Set , where 7: if frozen = false and then 8: frozen true; 9: end if 10: end for
Ensure:
before the game, with the forecaster’s actual predictions replaced by their expectations. The detailed mechanism is summarized in Algorithm 2, whose adversary we denote by . In the following, we fix a forecasting strategy with internal random variable , and recall from § 3 that its prediction at round is .
In particular, Algorithm 2 simulates the interaction between the adversary in Algorithm 1 and a deterministic forecaster , which predicts the average of over its internal randomness. Formally, given the past outcome , predicts
at each round . Following Algorithm 1, as long as the residual between the simulated outcomes and the predictions of stays below the threshold , the adversary draws , whose mean moves with the residual. Once the threshold is reached, the adversary repeats the constant outcome that preserves the sign of the residual. Since the simulation uses only the description of but not its realized predictions, the outcome sequence is independent of the internal randomness , so that is a valid oblivious adversary. After the simulation, the adversary reveals the simulated outcomes one per round in the actual game, regardless of the forecaster’s actual predictions. Note that is a randomized oblivious adversary, i.e., a distribution over outcome sequences , rather than a single sequence. As we show below, replacing the forecaster’s predictions with their expectations does not help the forecaster. Consequently, this distribution forces a large expected smooth calibration error on at least one sequence in its support, which yields the fixed sequence in Theorem 4.2, so that the lower bound of Theorem 4.1 carries over to oblivious adversaries.
Lower Bounds under Oblivious Adversary
The following theorem presents the lower bound induced by the oblivious adversary in Algorithm 2. Its full proof is deferred to Appendix § 7.§ 2.
Theorem 4.2 (Lower bound against oblivious adversaries). For every integer and every randomized forecasting strategy , there exists an outcome sequence such that the oblivious adversary satisfies
where is the forecast sequence of on . Consequently, the minimax smooth calibration error against oblivious adversaries satisfies .
Theorem 4.2 shows that the lower bound continues to hold against oblivious adversaries, with the same constant as Theorem 4.1. Since , combining it with the upper bounds of Qiao and Zheng [22]; Arunachaleswaran et al. [1] gives , which settles the minimax rate of smooth calibration error in both settings. Moreover, recall from Remark 3.2 that for every outcome sequence and forecast sequence . Combining this with Theorem 4.2 immediately leads to the following corollary.
Corollary 4.3 (Calibration distance lower bound). For every integer and every randomized forecasting strategy , the oblivious adversary in Theorem 4.2 satisfies
Corollary 4.3 matches, up to a universal constant, the upper bounds on both distance-based measures in Qiao and Zheng [22], which hold against adaptive and hence oblivious adversaries.
Proof Overview. We now give a brief overview of why this adversary achieves the lower bound. The proof of Theorem 4.1 only uses the four tests , whose maximal score
is convex in , since is affine and is a convex quadratic in . Hence, for every fixed outcome sequence , since and is convex, Jensen’s inequality gives , where is exactly the forecast sequence of on . That is, the forecaster performs no better than its deterministic average . Meanwhile, since Algorithm 2 simulates Algorithm 1 against , the proof of Theorem 4.1 gives , where is the simulated forecast sequence. Combining the two bounds yields Theorem 4.2.
Conclusion
In this paper, we study the minimax smooth calibration error in sequential binary prediction against both adaptive and oblivious adversaries. In particular, we construct a new adaptive adversary whose outcome mean moves with the forecaster’s cumulative residual and which switches to constant outcomes once the residual becomes large, so that any attempt to compensate for past residuals is penalized by an affine test. We further convert it into an oblivious adversary that simulates this interaction against the forecaster’s average prediction before the game, which loses nothing since the tests used in our analysis have a convex maximal score. We show that both adversaries force every forecaster to incur expected smooth calibration error, which improves the lower bound of Qiao and Zheng [22] and matches the known upper bound, thereby settling the minimax rates of smooth calibration error, as well as of calibration distance and its relaxation, in both settings up to universal constants.
Acknowledgment
The proof ideas are partially developed by GPT-6 Astra under the authors’ guidance. The authors re-organized the proof, verified the proof details, prepared the draft, and take full responsibility for the contents.
Proofs of Main Results
In this section, we provide the full proofs of Theorems 4.1 and 4.2. We first set up the notation and establish a lemma on the interaction with the adaptive adversary, which is shared by both proofs.
Fix an arbitrary, possibly randomized, forecasting strategy , let be its internal random variable, and let for . Since , the predictions are -measurable. Moreover, on the active rounds of Algorithm 1, the outcome is drawn from using fresh randomness that is independent of and of the history. We lower bound the calibration error through two kinds of tests, namely the constant tests and , and the affine test , which lead to the following scores:
Moreover, we track the first round at which the residual reaches the threshold ,
and let be the event that the threshold is reached, so that on . Since is -measurable, is a stopping time with respect to .
Lemma A.1. For every integer and every randomized forecasting strategy , let be the adaptive adversary in Algorithm 1. Then the quantity satisfies
Proof. We first compute as follows:
For each fixed , the indicator is -measurable, since , and so are and . Moreover, on , round is active, so that . Therefore,
where the last equality uses and . Consequently, we see that
Similarly, we write as the telescoping sum , where each increment satisfies . Therefore,
For each fixed , all of , , , and are -measurable. Moreover, on , we have and , where is the variance of Bernoulli. Therefore, writing and expanding,
since all terms that are linear in vanish in conditional expectation. Consequently, we see that
Adding times (A.2) to (A.1) cancels the cross terms, which yields
Now, taking expectations in the bound and applying (A.3), we obtain
where we drop the nonnegative term . We now show that both remaining terms are well behaved. First, since and each increment of the residual has magnitude at most one, we have . Since , we further have , so that and the last term is nonnegative. Second, on every active round , we have , so that by the choice . Hence , which implies . Therefore, writing , we obtain
That is, a long active phase forces a large affine score. On the other hand, we have , so that , i.e., reaching the threshold often forces a large constant score. Since on , it follows that
Substituting this into (A.4) gives , i.e., . This completes the proof of Lemma A.1.
Proof of Theorem 4.1
Proof of Theorem 4.1. By Lemma A.1, it suffices to show that is lower bounded by . If , the threshold is reached at round , and every later outcome equals . Hence every later increment has the same sign as . If , there are no later rounds. Therefore, the constant test if , and otherwise, has score , which gives the first lower bound. For the second lower bound, suppose and consider . Its score over the first rounds is , and its score in every later round is . Similarly, if , the test has score over the first rounds and score in every later round. All these tests belong to the class of the four tests in (9). In summary, we have
Taking expectations and applying Lemma A.1 completes the proof of Theorem 4.1.
Proof of Theorem 4.2
We first show that, for a fixed outcome sequence, randomizing the predictions cannot decrease the restricted objective in (9) below its value at the averaged predictions.
Lemma A.2 (Convexity). For every fixed , the map is convex on . Consequently, every random vector satisfies
Proof. The score is affine in , so is convex in . Moreover, expanding each summand of gives
which is a sum of convex quadratic functions of . Hence is convex in . Since the pointwise maximum of convex functions is convex, is convex by (9). The first inequality in (A.6) follows from , since , and the second inequality follows from Jensen’s inequality.
We are now ready to prove Theorem 4.2.
Proof of Theorem 4.2. Fix a randomized forecasting strategy with internal random variable , and let be the deterministic forecaster in Algorithm 2, which predicts at each round . For every , define the expected smooth calibration error of on and the forecast sequence of on , respectively, as
where is the forecast sequence of on . For every fixed , the outcome sequence does not depend on , so that by definition. Applying Lemma A.2 with , we obtain
Next, let be the random outcome sequence generated by in Algorithm 2. By construction, the simulation in Algorithm 2 is exactly the interaction between Algorithm 1 and the deterministic forecaster , whose prediction at round is . Consequently, has the same distribution as the outcome and forecast sequences generated by the interaction between and . Applying (A.5) and Lemma A.1 to the deterministic forecaster , we obtain
Since the left-hand side of (A.8) is an average of over , there exists such that . Plugging this into (A.7), we obtain that, for this specific ,
which proves (8). Finally, since and is arbitrary, (1) gives , which completes the proof of Theorem 4.2.
References
- [1]Arunachaleswaran, E. R., Collina, N., Roth, A. and Shi, M. (2025). An elementary predictor obtaining distance to calibration. In Proceedings of the 2025 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). SIAM.DOI
- [2]Blasiok, J., Gopalan, P., Hu, L. and Nakkiran, P. (2023). A unifying theory of distance from calibration. In Proceedings of the 55th Annual ACM Symposium on Theory of Computing.arxiv.org/abs/2211.16886
- [3]Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly weather review 78 1–3.doi.org/10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2
- [4]Cesa-Bianchi, N., Lugosi, G. et al. (2006). Prediction, learning, and games, vol. 1. Cambridge university press, Cambridge.
- [5]Dagan, Y., Daskalakis, C., Fishelson, M., Golowich, N., Kleinberg, R. and Okoroafor, P. (2025). Breaking the t^(2/3) barrier for sequential calibration. In Proceedings of the 57th Annual ACM Symposium on Theory of Computing.
- [6]Dawid, A. P. (1982). The well-calibrated bayesian. Journal of the American statistical Association 77 605–610.
- [7]Dawid, A. P. (1985). Comment: The impossibility of inductive inference. Journal of the American Statistical Association 80 340–341.DOI
- [8]Foster, D. P. (1999). A proof of calibration via blackwell’s approachability theorem. Games and Economic Behavior 29 73–78.
- [9]Foster, D. P. and Vohra, R. V. (1998). Asymptotic calibration. Biometrika 85 379–390.DOI
- [10]Garg, S., Jung, C., Reingold, O. and Roth, A. (2024). Oracle efficient online multicalibration and omniprediction. In Proceedings of the 2024 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). SIAM.arxiv.org/abs/2307.08999
- [11]Gopalan, P., Stavropoulos, K., Talwar, K. and Tankala, P. (2026). The importance of being smoothly calibrated. In 7th Symposium on Foundations of Responsible Computing (FORC 2026).
- [12]Guo, C., Pleiss, G., Sun, Y. and Weinberger, K. Q. (2017). On calibration of modern neural networks. In International conference on machine learning. PMLR.arxiv.org/abs/1706.04599
- [13]Gupta, V., Jung, C., Noarov, G., Pai, M. M. and Roth, A. (2022). Online multivalid learning: Means, moments, and prediction intervals. In 13th Innovations in Theoretical Computer Science Conference (ITCS 2022).DOI
- [14]Hart, S. (2025). Calibrated forecasts: The minimax proof. In Matching, Dynamics and Games for the Allocation of Resources: Essays in Celebration of David Gale’s 100th Birthday. Springer, 153–159.arxiv.org/abs/2209.05863
- [15]Hartline, J., Wu, Y. and Yang, Y. (2025). Smooth calibration and decision making. In 6th Symposium on Foundations of Responsible Computing (FORC 2025).
- [16]Hébert-Johnson, U., Kim, M., Reingold, O. and Rothblum, G. (2018). Multicalibration: Calibration for the (computationally-identifiable) masses. In International conference on machine learning. PMLR.
- [17]Hu, L. and Wu, Y. (2024). Predict to minimize swap regret for all payoff-bounded tasks. In 2024 IEEE 65th Annual Symposium on Foundations of Computer Science (FOCS). IEEE.DOI
- [18]Kakade, S. M. and Foster, D. P. (2008). Deterministic calibration and nash equilibrium. Journal of Computer and System Sciences 74 115–130.
- [19]Kleinberg, R., Paes Leme, R., Schneider, J. and Teng, Y. (2023). U-calibration: Forecasting for an unknown agent. In Proceedings of Thirty Sixth Conference on Learning Theory. PMLR.arxiv.org/abs/2307.00168
- [20]Oakes, D. (1985). Self-calibrating priors do not exist. Journal of the American Statistical Association 80 339–339.DOI
- [21]Qiao, M. and Valiant, G. (2021). Stronger calibration lower bounds via sidestepping. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing.arxiv.org/abs/2012.03454
- [22]Qiao, M. and Zheng, L. (2024). On the distance from calibration in sequential prediction. In The Thirty Seventh Annual Conference on Learning Theory. PMLR.arxiv.org/abs/2402.07458