Introduction
In information theory, prediction concerns the problem of estimating the continuation of a sequence of symbols, or tokens , based on the sequence’s past and on any additional information available about the data distribution. The estimate is produced by a predictor , namely a probability distribution on the possible continuations. Under logarithmic loss, the best predictor for a known source is the source distribution itself, and the excess loss incurred by a universal predictor is called regret . Universal prediction studies predictors whose regret is small uniformly over a prescribed class of sources.
Universal prediction is a classical topic, with roots that can be traced at least to Laplace’s rule of succession [1 ] . Its connection to modern machine learning is nevertheless direct: language models are trained as next-token predictors, and their performance is usually evaluated through variants of logarithmic loss. However, in the classical formulation of universal prediction, one usually predicts a single long sequence generated by one unknown source. Instead, in many data-rich prediction problems such as large language model training, information about the source is available through a collection of independent training sequences, and performance is then measured on an independent test sequence. This connection motivates revisiting universal prediction in settings where a predictor has access to many previously observed examples before being evaluated on a fresh test sequence. In this work, we therefore introduce batch regret , a conditional regret measure in which the predictor is evaluated on a test sequence of ℓ \ell ℓ tokens, conditioned on the observation of a batch of n n n training sequences.
The main focus of this paper is the single-source batch setting, where all training sequences and the test sequence are drawn independently from the same unknown distribution ρ θ \rho_{\theta} ρ θ . This setting captures an idealized form of in-distribution prediction: the training batch provides information about the unknown source, and the predictor uses this information when assigning probabilities to the test sequence. More specifically, we will focus on additive smoothing estimators, in which the predictive probability at any given moment is obtained by adding a constant bias β > 0 \beta> 0 β > 0 to the empirical frequencies of the tokens. This class includes the Laplacian [1 ] and the Krichevsky–Trofimov [2 ] estimators and has been shown to achieve near-optimal asymptotic average regret on a single sequence. Here we will study their conditional extension, in which the training batch is used to improve the estimates. In particular, we will analyze their batch regret when the target classes are binary memoryless and first-order Markov sources. Under this setting, the theoretical limits derived in this work complement a more practical line of research that studies large language models’ capability to optimally predict tokens generated from controlled data distributions such as Markov sources [3 , 4 , 5 , 6 , 7 , 8 ] .
Main contributions
Our contributions are organized as follows. In Section III we formulate single-source batch regret and relate it to the Kullback-Leibler divergence between the test sequence’s parametric distribution and a predictor, conditioned on the training batch. In Section IV we prove a conditional regret-capacity theorem, which equates minimax batch regret to a maximal conditional mutual information between the distribution parameter and the test sequence given the training batch. As the maximal mutual information can be lower-bounded by choosing any prior distribution on the parameter space, this result can be used to lower bounds on minimax batch regret, which we do in the subsequent sections. In Section V, we introduce conditional additive smoothing estimators, or add-β \beta β estimators, for binary memoryless sources and we prove general estimates of their batch regret in the interior and at the border of the simplex, valid whenever n ℓ → ∞ n\ell\to\infty n ℓ → ∞ . The overall behavior over the entire simplex depends on the regime in which the batch size n n n and the sequence length ℓ \ell ℓ interact, and is discussed in the following two sections: in Section VI we consider the large-n n n regime, while in Section VII we study the complementary fixed-n n n , large-ℓ \ell ℓ regime. In Section VIII we discuss the optimal batch regret over the class of all predictors instead. In Section IX, we extend our analysis to first-order binary Markov sources with uniformly positive transitions. Under this setting, assumptions on the initial distribution of the Markov source are fundamental: in fact, we show how the leading coefficient of batch regret changes whenever the initial law is known, free, or stationary. Finally, in Section X we consider a generalized batch regret measure based on conditional Rényi divergence, which we call batch α \alpha α -regret, and we prove its relation with a conditional extension of Sibson’s α \alpha α -mutual information, thus providing an operational interpretation to it. A preliminary version of some of the results of this paper was first presented in [9 , 10 ] .
Universal prediction under cross-entropy loss and universal lossless compression are essentially mathematically equivalent [11 , 12 ] , so that literature on either topic is relevant to the other. Universal compression-prediction for memoryless sources was studied by Xie and Barron under both average [13 ] and worst-case regrets [14 ] . Their asymptotic techniques are closely related to the compact-interior expansion in Theorem 4. The worst-case regret for Markov sources was studied in [14 , 15 , 16 ] . The average case was more recently characterized in [17 , 18 ] . The techniques used in [18 ] , in particular Paulin’s concentration inequalities for Markov chains [19 ] , are instrumental to proving our Theorem 14.
A batch regret with one test sample is the average next-token regret defined in Eq. (28) and first studied by Krichevsky [20 ] . This regret measure has since been studied in several works [21 , 22 , 23 , 24 ] ; in particular, [24 ] predicts one test symbol after a training sample and therefore belongs to this next-token setting. A form of conditional average regret for more than one token was introduced in [25 ] , where it is studied for location families. This metric is also studied by [22 ] and [26 ] . In particular, [26 ] focuses on memoryless sources, under which their metric is equivalent to our definition of batch regret. However, in this work we study a different regime of n n n and ℓ \ell ℓ and a different class of predictors. The equivalence between the two definitions of conditional regret does not extend to other classes of distributions, e.g., the Markov class studied in Section IX: according to our definition, the training batch is composed of n n n independent Markov sequences of length ℓ \ell ℓ , instead of one sequence of n ℓ n\ell n ℓ i.i.d. samples. A closer setting is the one studied in [22 ] , whose general formulation includes independent sequences; however, it does not study the regret as a function of the batch size n n n and the sequence length ℓ \ell ℓ , which is the focus of this work. Nonetheless, both works include a conditional version of the regret-capacity theorem, similar to our Theorem 2.
Sibson’s α \alpha α -mutual information was introduced by Sibson in [27 ] as an information radius. A revisitation of it as a generalization of Shannon’s mutual information was proposed in [28 ] , which also reviews many of its properties. A more recent discussion can be found in [29 ] . Proposals for conditional versions of Sibson’s mutual information, including the one used in Eq. (142) , appear in [30 , 31 ] . The maximization of Sibson’s mutual information and related measures is studied in [32 ] , including a proof of a regret-capacity theorem for α \alpha α -mutual information. The work [33 ] discusses Sibson’s mutual information in a universal-compression context while studying a closely related redundancy measure called Sundaresan divergence. The predictors introduced in [34 ] are based on Sibson’s mutual information; in Theorem 18, we obtain conditional analogues of these predictors as optimal predictors under α \alpha α -batch regret.
Background
In this section we recall basic information-theoretic definitions and results that will be useful in the sequel. The alphabets are finite and all logarithms are in base e e e , unless stated otherwise.
Given two conditional distributions p U ∣ Z p_{U|Z} p U ∣ Z and q U ∣ Z q_{U|Z} q U ∣ Z and a distribution p Z p_Z p Z , we use the conditional Kullback–Leibler (KL) divergence
D ( p U ∣ Z ∥ q U ∣ Z ∣ p Z ) ≜ ∑ z ∈ Z p Z ( z ) ∑ u ∈ U p U ∣ Z ( u ∣ z ) log p U ∣ Z ( u ∣ z ) q U ∣ Z ( u ∣ z ) . (1) D(p_{U|Z}\|q_{U|Z}\mid p_Z) \triangleq\sum_{z\in\mathcal{Z}} p_Z(z)\sum_{u\in\mathcal{U}}p_{U|Z}(u\mid z)\log\frac{p_{U|Z}(u\mid z)}{q_{U|Z}(u\mid z)}. \tag*{(1)} D ( p U ∣ Z ∥ q U ∣ Z ∣ p Z ) ≜ z ∈ Z ∑ p Z ( z ) u ∈ U ∑ p U ∣ Z ( u ∣ z ) log q U ∣ Z ( u ∣ z ) p U ∣ Z ( u ∣ z ) . (1)
Equivalently, when p U Z p_{UZ} p U Z and q U Z q_{UZ} q U Z have the same Z Z Z -marginal p Z p_Z p Z , this quantity is equal to the classical KL divergence D ( p U Z ∥ q U Z ) D(p_{UZ}\|q_{UZ}) D ( p U Z ∥ q U Z ) . We also write it as D ( U ∥ U ^ ∣ Z ) D(U\|\widehat{U}\mid Z) D ( U ∥ U ∣ Z ) when the underlying conditional laws are clear. Conditional mutual information is the corresponding information radius,
I ( X ; Y ∣ Z ) = D ( p X Y Z ∥ p X ∣ Z p Y ∣ Z ∣ p Z ) . (2) I(X;Y\mid Z)=D(p_{XYZ}\|p_{X|Z}p_{Y|Z}\mid p_Z). \tag*{(2)} I ( X ; Y ∣ Z ) = D ( p X Y Z ∥ p X ∣ Z p Y ∣ Z ∣ p Z ) . (2)
In cases where the distribution of X X X varies, we may also write I P ( X ; Y ∣ Z ) I_P(X;Y\mid Z) I P ( X ; Y ∣ Z ) to highlight the fact that X X X is distributed according to P P P .
For α > 0 \alpha>0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 , the Rényi divergence [35 ] is
D α ( p X ∥ q X ) ≜ 1 α − 1 log ∑ x ∈ X p X ( x ) α q X ( x ) 1 − α . (3) D_\alpha(p_X\|q_X)\triangleq\frac{1}{\alpha-1}\log\sum_{x\in\mathcal{X}}p_X(x)^\alpha q_X(x)^{1-\alpha}. \tag*{(3)} D α ( p X ∥ q X ) ≜ α − 1 1 log x ∈ X ∑ p X ( x ) α q X ( x ) 1 − α . (3)
Its limiting cases include D 1 ( p X ∥ q X ) = D ( p X ∥ q X ) D_1(p_X\|q_X)=D(p_X\|q_X) D 1 ( p X ∥ q X ) = D ( p X ∥ q X ) and
D ∞ ( p X ∥ q X ) = sup x ∈ X log p X ( x ) q X ( x ) . (4) D_\infty(p_X\|q_X)=\sup_{x\in\mathcal{X}}\log\frac{p_X(x)}{q_X(x)}. \tag*{(4)} D ∞ ( p X ∥ q X ) = x ∈ X sup log q X ( x ) p X ( x ) . (4)
We will also use the conditional version
D α ( p U ∣ Z ∥ q U ∣ Z ∣ p Z ) ≜ 1 α − 1 log ∑ z ∈ Z p Z ( z ) ( ∑ u ∈ U p U ∣ Z ( u ∣ z ) α q U ∣ Z ( u ∣ z ) 1 − α ) . (5) D_{\alpha}(p_{U|Z}\|q_{U|Z}\mid p_Z) \triangleq\frac{1}{\alpha-1}\log\sum_{z\in\mathcal{Z}}p_Z(z)\left(\sum_{u\in\mathcal{U}}p_{U|Z}(u\mid z)^\alpha q_{U|Z}(u\mid z)^{1-\alpha}\right). \tag*{(5)} D α ( p U ∣ Z ∥ q U ∣ Z ∣ p Z ) ≜ α − 1 1 log z ∈ Z ∑ p Z ( z ) ( u ∈ U ∑ p U ∣ Z ( u ∣ z ) α q U ∣ Z ( u ∣ z ) 1 − α ) . (5)
Finally, Sibson’s α \alpha α -mutual information [27 ] is
I α ( X ; Y ) ≜ α α − 1 log ∑ y ∈ Y ( ∑ x ∈ X p X ( x ) p Y ∣ X ( y ∣ x ) α ) 1 / α . (6) I_{\alpha}(X;Y)\triangleq\frac{\alpha}{\alpha-1}\log\sum_{y\in\mathcal{Y}}\left(\sum_{x\in\mathcal{X}}p_X(x)p_{Y|X}(y\mid x)^\alpha\right)^{1/\alpha}. \tag*{(6)} I α ( X ; Y ) ≜ α − 1 α log y ∈ Y ∑ ( x ∈ X ∑ p X ( x ) p Y ∣ X ( y ∣ x ) α ) 1/ α . (6)
It admits the variational representation
I α ( X ; Y ) = min Y ^ : X ⊥ Y ^ D α ( Y ∥ Y ^ ∣ X ) , (7) I_{\alpha}(X;Y)=\min_{\hat{Y}:X\perp\hat{Y}}D_{\alpha}(Y\|\hat{Y}\mid X), \tag*{(7)} I α ( X ; Y ) = Y ^ : X ⊥ Y ^ min D α ( Y ∥ Y ^ ∣ X ) , (7)
for the α \alpha α -orders considered in this paper. Conditional analogues are introduced in Section X.
Classical universal prediction
Fix a sequence length ℓ \ell ℓ . A predictor assigns probabilities p ^ ( x ℓ ) \hat{p}(x^\ell) p ^ ( x ℓ ) to sequences x ℓ = ( x 1 , … , x ℓ ) ∈ X ℓ x^\ell=(x_1,\ldots,x_\ell)\in\mathcal{X}^\ell x ℓ = ( x 1 , … , x ℓ ) ∈ X ℓ . When the predictor is used sequentially, it is specified by one-step conditionals and induces the joint distribution
p ^ ( x ℓ ) = ∏ i = 0 ℓ − 1 p ^ ( x i + 1 ∣ x i ) . (8) \hat{p}(x^\ell)=\prod_{i=0}^{\ell-1}\hat{p}(x_{i+1}\mid x^i). \tag*{(8)} p ^ ( x ℓ ) = i = 0 ∏ ℓ − 1 p ^ ( x i + 1 ∣ x i ) . (8)
Universal prediction benchmarks p ^ \hat{p} p ^ against a class P = { p θ : θ ∈ Θ } \mathcal{P}=\{p_\theta:\theta\in\Theta\} P = { p θ : θ ∈ Θ } . Under logarithmic loss,
L ( p ^ , x ℓ ) ≜ − log p ^ ( x ℓ ) , (9) L(\hat{p},x^\ell)\triangleq-\log\hat{p}(x^\ell), \tag*{(9)} L ( p ^ , x ℓ ) ≜ − log p ^ ( x ℓ ) , (9)
the pointwise regret relative to p θ p_\theta p θ is
R n , ℓ ( p ^ , θ , x ℓ ) ≜ L ( p ^ , x ℓ ) − L ( p θ , x ℓ ) = log p θ ( x ℓ ) p ^ ( x ℓ ) . (10) R_{n,\ell}(\hat{p},\theta,x^\ell)\triangleq L(\hat{p},x^\ell)-L(p_\theta,x^\ell)=\log\frac{p_\theta(x^\ell)}{\hat{p}(x^\ell)}. \tag*{(10)} R n , ℓ ( p ^ , θ , x ℓ ) ≜ L ( p ^ , x ℓ ) − L ( p θ , x ℓ ) = log p ^ ( x ℓ ) p θ ( x ℓ ) . (10)
The two classical aggregate criteria are the average regret
R a v , ℓ ( p ^ , θ ) ≜ E X ℓ ∼ p θ R n , ℓ ( p ^ , θ , X ℓ ) (11) R_{\mathrm{av},\ell}(\hat{p},\theta)\triangleq\mathbb{E}_{X^\ell\sim p_\theta}R_{n,\ell}(\hat{p},\theta,X^\ell) \tag*{(11)} R av , ℓ ( p ^ , θ ) ≜ E X ℓ ∼ p θ R n , ℓ ( p ^ , θ , X ℓ ) (11)
= ∑ x ℓ ∈ X ℓ p θ ( x ℓ ) log p θ ( x ℓ ) p ^ ( x ℓ ) (12) =\sum_{x^\ell\in\mathcal{X}^\ell}p_\theta(x^\ell)\log\frac{p_\theta(x^\ell)}{\hat{p}(x^\ell)} \tag*{(12)} = x ℓ ∈ X ℓ ∑ p θ ( x ℓ ) log p ^ ( x ℓ ) p θ ( x ℓ ) (12)
= D ( p θ ∥ p ^ ) , (13) =D(p_\theta\|\hat{p}), \tag*{(13)} = D ( p θ ∥ p ^ ) , (13)
and the worst-case regret
R max , ℓ ( p ^ , θ ) ≜ sup x ℓ ∈ X ℓ R n , ℓ ( p ^ , θ , x ℓ ) (14) R_{\max,\ell}(\hat{p},\theta)\triangleq\sup_{x^\ell\in\mathcal{X}^\ell}R_{n,\ell}(\hat{p},\theta,x^\ell) \tag*{(14)} R m a x , ℓ ( p ^ , θ ) ≜ x ℓ ∈ X ℓ sup R n , ℓ ( p ^ , θ , x ℓ ) (14)
= sup x ℓ ∈ X ℓ log p θ ( x ℓ ) p ^ ( x ℓ ) (15) =\sup_{x^\ell\in\mathcal{X}^\ell}\log\frac{p_\theta(x^\ell)}{\hat{p}(x^\ell)} \tag*{(15)} = x ℓ ∈ X ℓ sup log p ^ ( x ℓ ) p θ ( x ℓ ) (15)
= D ∞ ( p θ ∥ p ^ ) . (16) =D_\infty(p_\theta\|\hat{p}). \tag*{(16)} = D ∞ ( p θ ∥ p ^ ) . (16)
Equivalently,
R a v , ℓ ( p ^ , θ ) = E θ L ( p ^ , X ℓ ) − E θ L ( p θ , X ℓ ) , (17) R_{\mathrm{av},\ell}(\hat{p},\theta)=\mathbb{E}_{\theta}L(\hat{p},X^\ell)-\mathbb{E}_{\theta}L(p_\theta,X^\ell), \tag*{(17)} R av , ℓ ( p ^ , θ ) = E θ L ( p ^ , X ℓ ) − E θ L ( p θ , X ℓ ) , (17)
so average regret is the excess expected logarithmic loss over the source-matched predictor.
Since the parameter is unknown, the usual minimax criteria maximize over the class:
R a v , ℓ ( p ^ ) ≜ sup θ ∈ Θ R a v , ℓ ( p ^ , θ ) , (18) R_{\mathrm{av},\ell}(\hat{p})\triangleq\sup_{\theta\in\Theta}R_{\mathrm{av},\ell}(\hat{p},\theta), \tag*{(18)} R av , ℓ ( p ^ ) ≜ θ ∈ Θ sup R av , ℓ ( p ^ , θ ) , (18)
R max , ℓ ( p ^ ) ≜ sup θ ∈ Θ R max , ℓ ( p ^ , θ ) . (19) R_{\max,\ell}(\hat{p})\triangleq\sup_{\theta\in\Theta}R_{\max,\ell}(\hat{p},\theta). \tag*{(19)} R m a x , ℓ ( p ^ ) ≜ θ ∈ Θ sup R m a x , ℓ ( p ^ , θ ) . (19)
Whenever it exists, the normalized maximum-likelihood (NML) predictor minimizes R max R_{\max} R m a x [36 ] :
p ^ N M L ( x ℓ ) = sup θ ∈ Θ p θ ( x ℓ ) ∑ x ℓ sup θ ∈ Θ p θ ( x ℓ ) , (20) \hat{p}_{\mathrm{NML}}(x^\ell)=\frac{\sup_{\theta\in\Theta}p_\theta(x^\ell)}{\sum_{x^\ell}\sup_{\theta\in\Theta}p_\theta(x^\ell)}, \tag*{(20)} p ^ NML ( x ℓ ) = ∑ x ℓ sup θ ∈ Θ p θ ( x ℓ ) sup θ ∈ Θ p θ ( x ℓ ) , (20)
with minimax value
inf p ^ R max , ℓ ( p ^ ) = log ∑ x ℓ sup θ ∈ Θ p θ ( x ℓ ) . (21) \inf_{\hat{p}}R_{\max,\ell}(\hat{p})=\log\sum_{x^\ell}\sup_{\theta\in\Theta}p_\theta(x^\ell). \tag*{(21)} p ^ inf R m a x , ℓ ( p ^ ) = log x ℓ ∑ θ ∈ Θ sup p θ ( x ℓ ) . (21)
For average regret, the corresponding result is the redundancy-or regret-capacity theorem. Let W ( Θ ) \mathcal{W}(\Theta) W ( Θ ) be the set of all prior distributions on Θ \Theta Θ . Then,
inf p ^ R a v , ℓ ( p ^ ) = sup W ∈ W ( Θ ) I W ( θ ; X ℓ ) , (22) \inf_{\hat{p}}R_{\mathrm{av},\ell}(\hat{p})=\sup_{W\in\mathcal{W}(\Theta)}I_W(\theta;X^\ell), \tag*{(22)} p ^ inf R av , ℓ ( p ^ ) = W ∈ W ( Θ ) sup I W ( θ ; X ℓ ) , (22)
where ( θ , X ℓ ) ∼ W ( θ ) p θ ( X ℓ ) (\theta,X^\ell)\sim W(\theta)p_\theta(X^\ell) ( θ , X ℓ ) ∼ W ( θ ) p θ ( X ℓ ) . For a fixed prior W W W , the Bayes-optimal predictor under average regret is the mixture
p ^ W ( x ℓ ) = ∫ Θ p θ ( x ℓ ) d W ( θ ) . (23) \hat{p}_W(x^\ell)=\int_{\Theta}p_\theta(x^\ell)\,dW(\theta). \tag*{(23)} p ^ W ( x ℓ ) = ∫ Θ p θ ( x ℓ ) d W ( θ ) . (23)
When a capacity-achieving prior exists, its mixture is minimax optimal. Such priors are rarely available in closed form, but they can often be approximated numerically by Blahut–Arimoto-type algorithms [37 , 38 ] .
Memoryless sources
For an alphabet X = { 0 , 1 , … , M − 1 } \mathcal{X}=\{0,1,\ldots,M-1\} X = { 0 , 1 , … , M − 1 } , the memoryless class consists of i.i.d. distributions p θ p_\theta p θ with p θ ( X i = x ) = θ x p_\theta(X_i=x)=\theta_x p θ ( X i = x ) = θ x , where
θ = ( θ 0 , … , θ M − 1 ) , θ x ≥ 0 , ∑ x = 0 M − 1 θ x = 1. (24) \theta=(\theta_0,\ldots,\theta_{M-1}),\qquad\theta_x\ge0,\qquad\sum_{x=0}^{M-1}\theta_x=1. \tag*{(24)} θ = ( θ 0 , … , θ M − 1 ) , θ x ≥ 0 , x = 0 ∑ M − 1 θ x = 1. (24)
A central online predictor for this class is the additive smoothing estimator
p ^ β ( X ℓ + 1 = x ∣ x ℓ ) = n x ( x ℓ ) + β ℓ + M β , β > 0 , (25) \hat{p}_{\beta}(X_{\ell+1}=x\mid x^\ell)=\frac{n_x(x^\ell)+\beta}{\ell+M\beta},\qquad\beta>0, \tag*{(25)} p ^ β ( X ℓ + 1 = x ∣ x ℓ ) = ℓ + M β n x ( x ℓ ) + β , β > 0 , (25)
where n x ( x ℓ ) n_x(x^\ell) n x ( x ℓ ) is the number of occurrences of symbol x x x in x ℓ x^\ell x ℓ . The special case β = 1 / 2 \beta=1/2 β = 1/2 is the Krichevsky–Trofimov (KT) predictor,
p ^ K T ( X ℓ + 1 = x ∣ x ℓ ) = n x ( x ℓ ) + 1 2 ℓ + M 2 . (26) \hat{p}_{\mathrm{KT}}(X_{\ell+1}=x\mid x^\ell)=\frac{n_x(x^\ell)+\frac{1}{2}}{\ell+\frac{M}{2}}. \tag*{(26)} p ^ KT ( X ℓ + 1 = x ∣ x ℓ ) = ℓ + 2 M n x ( x ℓ ) + 2 1 . (26)
The add-β \beta β predictor is the posterior predictive distribution under the symmetric Dirichlet prior W β ∼ Dir ( β ) W_\beta\sim\operatorname{Dir}(\beta) W β ∼ Dir ( β ) ; equivalently, the joint distribution induced by Eq. (8) satisfies
p ^ β ( x ℓ ) = ∫ Θ p θ ( x ℓ ) d W β ( θ ) . (27) \hat{p}_{\beta}(x^\ell)=\int_{\Theta}p_\theta(x^\ell)\,dW_\beta(\theta). \tag*{(27)} p ^ β ( x ℓ ) = ∫ Θ p θ ( x ℓ ) d W β ( θ ) . (27)
The KT predictor is asymptotically minimax for memoryless sources, up to an O ( 1 ) O(1) O ( 1 ) term, under both average and worst-case regret [13 , 14 ] .
We will also refer to the average next-token regret studied by Krichevsky [20 ] ,
R n , ℓ ( p ^ ) = sup θ ∈ Θ E θ log p θ ( X ℓ + 1 ∣ X ℓ ) p ^ ( X ℓ + 1 ∣ X ℓ ) . (28) R_{n,\ell}(\hat{p}) = \sup_{\theta\in\Theta} \mathbb{E}_{\theta}\log\frac{p_{\theta}(X_{\ell+1}\mid X^{\ell})}{\hat{p}(X_{\ell+1}\mid X^{\ell})}. \tag*{(28)} R n , ℓ ( p ^ ) = θ ∈ Θ sup E θ log p ^ ( X ℓ + 1 ∣ X ℓ ) p θ ( X ℓ + 1 ∣ X ℓ ) . (28)
For binary memoryless sources, the best add-β \beta β predictor for this next-token criterion has β = β 0 ≈ 0.50922 \beta=\beta_0\approx0.50922 β = β 0 ≈ 0.50922 , and asymptotic optimality among all next-token predictors is achieved by the rule of [21 Eq. (5.18)] .
Batch regret
In the universal prediction literature, it is customary to consider the regret for the prediction of an entire sequence of n n n symbols, generated by a single source. That is the setting that we described in Section II-B. However, many prediction settings distinguish between samples available before prediction and samples revealed during prediction. This distinction is especially natural when modeling in-distribution learning: a predictor is given a finite training batch from an unknown source and must then predict a fresh sequence from the same source. We therefore introduce a new form of regret, batch regret , which measures the excess logarithmic loss on the test sequence after conditioning on the training batch.
In the sequel, we compactly denote each sequence of the training batch as x i ∈ X ℓ x_i\in\mathcal{X}^{\ell} x i ∈ X ℓ , so that x n = ( x 1 , x 2 , … , x n ) x^n=(x_1,x_2,\ldots,x_n) x n = ( x 1 , x 2 , … , x n ) is a set of n n n training sequences, each consisting of ℓ \ell ℓ samples. Similarly, the test sequence is denoted by y ∈ X ℓ y\in\mathcal{X}^{\ell} y ∈ X ℓ . Each sequence is independent of the others, each possibly generated from a different distribution. Each distribution is determined by a parameter belonging to a parameter set Θ \Theta Θ , i.e., each distribution belongs to a certain parametric class, which we denote by P = { p θ : θ ∈ Θ } \mathcal{P}=\{p_{\theta}:\theta\in\Theta\} P = { p θ : θ ∈ Θ } . More specifically, let ϕ i ∈ Θ \phi_i\in\Theta ϕ i ∈ Θ be the parameter of the distribution of the sequence x i x_i x i , so that x i ∼ p ϕ i x_i\sim p_{\phi_i} x i ∼ p ϕ i . We compactly denote by ϕ = ( ϕ 1 , ϕ 2 , … , ϕ n ) \phi=(\phi_1,\phi_2,\ldots,\phi_n) ϕ = ( ϕ 1 , ϕ 2 , … , ϕ n ) the set of parameters of the training sequences. We also let θ ∈ Θ \theta\in\Theta θ ∈ Θ be the parameter of the test sequence y y y , i.e., y ∼ p θ y\sim p_{\theta} y ∼ p θ . We also compactly denote, with some abuse of notation, p ϕ ( x n ) = ∏ i = 1 n p ϕ i ( x i ) p_{\phi}(x^n)=\prod_{i=1}^{n}p_{\phi_i}(x_i) p ϕ ( x n ) = ∏ i = 1 n p ϕ i ( x i ) . We now define batch regret as follows.
Definition 1 (Batch regret): Let x n = ( x 1 , x 2 , … , x n ) x^n=(x_1,x_2,\ldots,x_n) x n = ( x 1 , x 2 , … , x n ) be a batch of n n n training sequences, where each x i ∈ X ℓ x_i\in\mathcal{X}^{\ell} x i ∈ X ℓ is distributed according to p ϕ i p_{\phi_i} p ϕ i independently of the others. Let p ^ ( y ∣ x n ) \hat{p}(y\mid x^n) p ^ ( y ∣ x n ) be a predictor that, given x n x^n x n , estimates the probability of a test sequence y ∈ X ℓ y\in\mathcal{X}^{\ell} y ∈ X ℓ , generated independently of x n x^n x n according to p θ p_{\theta} p θ . Batch regret is then defined as
R n , ℓ ( p ^ , ϕ , θ ) ≜ E X n ∼ p ϕ Y ∼ p θ log p θ ( Y ) p ^ ( Y ∣ X n ) . (29) R_{n,\ell}(\hat{p},\phi,\theta)\triangleq\mathbb{E}_{\substack{X^n\sim p_{\phi}\\Y\sim p_{\theta}}}\log\frac{p_{\theta}(Y)}{\hat{p}(Y\mid X^n)}. \tag*{(29)} R n , ℓ ( p ^ , ϕ , θ ) ≜ E X n ∼ p ϕ Y ∼ p θ log p ^ ( Y ∣ X n ) p θ ( Y ) . (29)
In a more general multi-source version, the training and test distributions could be different but related, for instance by belonging to a common subset Θ ~ ⊆ Θ \widetilde{\Theta}\subseteq\Theta Θ ⊆ Θ . In the present paper we restrict attention to the single-source case, in which all training sequences and the test sequence are drawn independently from the same distribution p θ p_{\theta} p θ . We study this setting for binary memoryless sources and first-order binary Markov sources.
When all sequences are drawn independently from the same distribution p θ p_{\theta} p θ , the batch regret takes the form
R n , ℓ ( p ^ , θ ) ≜ E X n ∼ p θ Y ∼ p θ log p θ ( Y ) p ^ ( Y ∣ X n ) . (30) R_{n,\ell}(\hat{p},\theta)\triangleq\mathbb{E}_{\substack{X^n\sim p_{\theta}\\Y\sim p_{\theta}}}\log\frac{p_{\theta}(Y)}{\hat{p}(Y\mid X^n)}. \tag*{(30)} R n , ℓ ( p ^ , θ ) ≜ E X n ∼ p θ Y ∼ p θ log p ^ ( Y ∣ X n ) p θ ( Y ) . (30)
Similarly to the classical average regret analyzed in Section II-B, the objective is to find a predictor p ^ \hat{p} p ^ that is universal , i.e., that performs well for all distributions in a given set P = { p θ : θ ∈ Θ } \mathcal{P}=\{p_{\theta}:\theta\in\Theta\} P = { p θ : θ ∈ Θ } . Hence, we are interested in the maximal batch regret
R n , ℓ ( p ^ ) ≜ sup θ ∈ Θ R n , ℓ ( p ^ , θ ) . (31) R_{n,\ell}(\hat{p})\triangleq\sup_{\theta\in\Theta}R_{n,\ell}(\hat{p},\theta). \tag*{(31)} R n , ℓ ( p ^ ) ≜ θ ∈ Θ sup R n , ℓ ( p ^ , θ ) . (31)
The objective of this section is to study theoretical limits of the single-distribution maximal batch regret when P \mathcal{P} P is either the set of all Bernoulli distributions, or the set of all first-order binary Markov distributions, asymptotically as ℓ \ell ℓ becomes large.
Remark. We note here that if n = 0 n=0 n = 0 , then we fall back into the classical full-sequence average regret setting discussed in Section II-B. If instead ℓ = 1 \ell=1 ℓ = 1 , we fall into the next-token average regret of Eq. (28) studied by Krichevsky [20 ] . Our setting can be thought as unifying and generalizing those two perspectives.
Regime. The interplay between ℓ \ell ℓ and n n n is fundamental, and different results below use different limits. For the class of memoryless sources, we consider both the regime n → ∞ n\to\infty n → ∞ with arbitrary ℓ \ell ℓ , and the complementary regime where n n n is fixed and ℓ → ∞ \ell\to\infty ℓ → ∞ . For the class of first-order Markov sources, we consider the regime where n , ℓ → ∞ n,\ell\to\infty n , ℓ → ∞ simultaneously. Additional assumptions are stated explicitly when relevant.
A conditional regret-capacity theorem
In Section II we noted that the minimax value of average regret is equal to a maximal mutual information, via the regret-capacity theorem in Eq. (22) . An equivalent result also holds for batch regret, in the single-distribution case introduced in Eq. (30) . More specifically, batch regret can be interpreted as a conditional generalization of average regret; in this section we prove a conditional regret-capacity theorem, connecting minimax batch regret to a maximal conditional mutual information. We first state the result in its general form, and later specialize it to the batch case introduced in § 3 .
Let X X X and Y Y Y be two finite alphabets, and let Q ( Y ∣ X ) \mathcal{Q}(Y\mid X) Q ( Y ∣ X ) denote the set of all stochastic kernels from X X X to Y Y Y . For any Q ∈ Q ( Y ∣ X ) Q\in\mathcal{Q}(Y\mid X) Q ∈ Q ( Y ∣ X ) , define the conditional Kullback–Leibler divergence
r θ ( Q ) ≜ D ( P θ Y ∣ X ∥ Q | P θ X ) , (32) r_{\theta}(Q)\triangleq D\left(P_{\theta}^{Y\mid X}\middle\|Q\middle|P_{\theta}^{X}\right), \tag*{(32)} r θ ( Q ) ≜ D ( P θ Y ∣ X Q P θ X ) , (32)
where P θ X , Y P_{\theta}^{X,Y} P θ X , Y is a joint probability distribution on X × Y p a r a m e t r i z e d b y X\times Y parametrized by X × Y p a r am e t r i z e d b y \theta$ belonging to a parameter set Θ \Theta Θ . Also assume that Θ \Theta Θ is equipped with a σ \sigma σ -algebra and that θ ↦ P θ X , Y ( x , y ) \theta\mapsto P_{\theta}^{X,Y}(x,y) θ ↦ P θ X , Y ( x , y ) is measurable for every ( x , y ) (x,y) ( x , y ) . First note the following useful conditional Bayes decomposition for the conditional KL divergence in Eq. (32) . For any measurable prior W W W for which the mixture is defined, let
P W ( x , y ) = ∫ Θ P θ X , Y ( x , y ) d W ( θ ) . (33) P_W(x,y)=\int_{\Theta}P_{\theta}^{X,Y}(x,y)\,dW(\theta). \tag*{(33)} P W ( x , y ) = ∫ Θ P θ X , Y ( x , y ) d W ( θ ) . (33)
Then, for every stochastic kernel Q Q Q ,
∫ Θ r θ ( Q ) d W ( θ ) = I W ( Θ ; Y ∣ X ) + D ( P W Y ∣ X ∥ Q | P W X ) . (34) \int_{\Theta} r_{\theta}(Q)\,\mathrm{d}W(\theta)=I_W(\Theta;Y\mid X)
+D\left(P_W^{Y\mid X}\middle\|Q\middle|P_W^X\right). \tag*{(34)} ∫ Θ r θ ( Q ) d W ( θ ) = I W ( Θ ; Y ∣ X ) + D ( P W Y ∣ X Q P W X ) . (34)
With the help of this general identity, we can prove the main result of this section.
Theorem 2 (Conditional regret-capacity theorem): Under the assumptions on Θ \Theta Θ and P θ X , Y P_\theta^{X,Y} P θ X , Y above, we have
min Q ∈ Q ( Y ∣ X ) sup θ ∈ Θ r θ ( Q ) = sup W ∈ W ( Θ ) I W ( Θ ; Y ∣ X ) . (35) \min_{Q\in\mathcal{Q}(\mathcal{Y}\mid\mathcal{X})}\sup_{\theta\in\Theta}r_\theta(Q)
=
\sup_{W\in\mathcal{W}(\Theta)} I_W(\Theta;Y\mid X). \tag*{(35)} Q ∈ Q ( Y ∣ X ) min θ ∈ Θ sup r θ ( Q ) = W ∈ W ( Θ ) sup I W ( Θ ; Y ∣ X ) . (35) Furthermore, if there exists a prior W ⋆ W^\star W ⋆ attaining the right-hand side supremum and a minimax Q ⋆ Q^\star Q ⋆ , then Q ⋆ Q^\star Q ⋆ is the conditional distribution of Y Y Y given X X X induced by a mixture of the form in Eq. (33) with W = W ⋆ W=W^\star W = W ⋆ , i.e.,
Q ⋆ ( ⋅ ∣ x ) = P W ⋆ Y ∣ X ( ⋅ ∣ x ) , (36) Q^\star(\cdot\mid x)=P_{W^\star}^{Y\mid X}(\cdot\mid x), \tag*{(36)} Q ⋆ ( ⋅ ∣ x ) = P W ⋆ Y ∣ X ( ⋅ ∣ x ) , (36) and r θ ( Q ⋆ ) r_\theta(Q^\star) r θ ( Q ⋆ ) is constant W ⋆ W^\star W ⋆ -almost surely. Furthermore, W ⋆ W^\star W ⋆ may be chosen to have finite support satisfying
∣ supp ( W ⋆ ) ∣ ≤ 1 + ∣ X ∣ ( ∣ Y ∣ − 1 ) . (37) \left|\operatorname{supp}(W^\star)\right|\leq1+|\mathcal{X}|(|\mathcal{Y}|-1). \tag*{(37)} ∣ supp ( W ⋆ ) ∣ ≤ 1 + ∣ X ∣ ( ∣ Y ∣ − 1 ) . (37)
The proof of the regret-capacity equivalence above is essentially a conditional generalization of [39 ] . The proof of the existence of a capacity-achieving input distribution with finite support is a conditional generalization of the cardinality bound in [40 ] , which is retrieved when ∣ X ∣ = 1 |\mathcal{X}|=1 ∣ X ∣ = 1 . We defer the details to Appendix A. Other versions of this general result were also proved in [22 ] and [26 ] under different assumptions.
We now show how this general result applies to batch universal prediction. Let X \mathcal{X} X be the common alphabet of the source sequences, let
X n = ( X 1 , … , X n ) ∈ ( X ℓ ) n and Y ∈ X ℓ (38) X^n=(X_1,\ldots,X_n)\in(\mathcal{X}^{\ell})^n
\quad\text{and}\quad
Y\in\mathcal{X}^{\ell} \tag*{(38)} X n = ( X 1 , … , X n ) ∈ ( X ℓ ) n and Y ∈ X ℓ (38)
be the training batch and the test sequence, respectively, and set X = X n X=X^n X = X n in Theorem 2. In the single-source model, conditionally on θ \theta θ , the n + 1 n+1 n + 1 sequences are independent and have common law p θ p_\theta p θ on X ℓ \mathcal{X}^{\ell} X ℓ , so
P θ X n , Y ( x n , y ) = ( ∏ i = 1 n p θ ( x i ) ) p θ ( y ) . (39) P_\theta^{X^n,Y}(x^n,y)=\left(\prod_{i=1}^{n}p_\theta(x_i)\right)p_\theta(y). \tag*{(39)} P θ X n , Y ( x n , y ) = ( i = 1 ∏ n p θ ( x i ) ) p θ ( y ) . (39)
A batch predictor p ^ ( y ∣ x n ) \hat p(y\mid x^n) p ^ ( y ∣ x n ) is precisely a stochastic kernel from ( X ℓ ) n (\mathcal{X}^{\ell})^n ( X ℓ ) n to X ℓ \mathcal{X}^{\ell} X ℓ , and its conditional information radius is the batch regret:
r θ ( p ^ ) = D ( P θ Y ∣ X n ∥ p ^ | P θ X n ) = ∑ x n , y p θ ( x n ) p θ ( y ) log p θ ( y ) p ^ ( y ∣ x n ) = R n , ℓ ( p ^ , θ ) . (40) \begin{aligned}
r_\theta(\hat p)=D\left(P_\theta^{Y\mid X^n}\middle\|\hat p\middle|P_\theta^{X^n}\right) \\
&=\sum_{x^n,y}p_\theta(x^n)p_\theta(y)\log\frac{p_\theta(y)}{\hat p(y\mid x^n)}
=R_{n,\ell}(\hat p,\theta). \tag*{(40)}
\end{aligned} r θ ( p ^ ) = D ( P θ Y ∣ X n p ^ P θ X n ) = x n , y ∑ p θ ( x n ) p θ ( y ) log p ^ ( y ∣ x n ) p θ ( y ) = R n , ℓ ( p ^ , θ ) . (40)
Consequently, Theorem 2 specializes to the following result.
Corollary 3 (Conditional regret-capacity theorem for batch regret): Under the same assumptions of Theorem 2, the minimax value of single-distribution batch regret as defined in Eq. (30) satisfies
min p ^ sup θ ∈ Θ R n , ℓ ( p ^ , θ ) = sup W ∈ W ( Θ ) I W ( Θ ; Y ∣ X n ) , (41) \min_{\hat p}\sup_{\theta\in\Theta}R_{n,\ell}(\hat p,\theta)
=
\sup_{W\in\mathcal{W}(\Theta)}I_W(\Theta;Y\mid X^n), \tag*{(41)} p ^ min θ ∈ Θ sup R n , ℓ ( p ^ , θ ) = W ∈ W ( Θ ) sup I W ( Θ ; Y ∣ X n ) , (41) where the infimum is taken over all predictors p ^ ∈ Q ( X ℓ ∣ ( X ℓ ) n ) \hat p\in\mathcal{Q}(\mathcal{X}^{\ell}\mid(\mathcal{X}^{\ell})^n) p ^ ∈ Q ( X ℓ ∣ ( X ℓ ) n ) . In particular, every chosen prior W W W gives the lower bound
min p ^ sup θ ∈ Θ R n , ℓ ( p ^ , θ ) ≥ I W ( Θ ; Y ∣ X n ) . (42) \min_{\hat p}\sup_{\theta\in\Theta}R_{n,\ell}(\hat p,\theta)\geq I_W(\Theta;Y\mid X^n). \tag*{(42)} p ^ min θ ∈ Θ sup R n , ℓ ( p ^ , θ ) ≥ I W ( Θ ; Y ∣ X n ) . (42)
This is how the theorem will be used later: a convenient prior turns finding a lower bound to the minimax batch regret into the computation or estimation of a conditional mutual information.
Conditional add-β \beta β estimators for binary memoryless sources
We now focus on the class of memoryless sources that we introduced in Section II, in particular with binary alphabet X = { 0 , 1 } \mathcal{X}=\{0,1\} X = { 0 , 1 } , i.e., the class of Bernoulli distributions. Under any fixed distribution in this class, a batch of n n n sequences X n X^n X n , each consisting of ℓ \ell ℓ samples, is equivalent to one single sequence of length n ℓ n\ell n ℓ , due to the i.i.d. nature of the data. Hence, under this setting the average batch regret can be written as the sum of ℓ \ell ℓ one-step conditional regrets. Let the one-step regret of the next bit given m m m previous samples be
ρ m ( p ^ , θ ) ≜ E θ log p θ ( X m + 1 ) p ^ ( X m + 1 ∣ X m ) . (43) \rho_m(\hat p,\theta)\triangleq\mathbb{E}_\theta\log\frac{p_\theta(X_{m+1})}{\hat p(X_{m+1}\mid X^m)}. \tag*{(43)} ρ m ( p ^ , θ ) ≜ E θ log p ^ ( X m + 1 ∣ X m ) p θ ( X m + 1 ) . (43)
= E θ [ θ log θ p ^ ( X m + 1 = 1 ∣ X m ) + ( 1 − θ ) log 1 − θ p ^ ( X m + 1 = 0 ∣ X m ) ] . (44) =\mathbb{E}_\theta\left[
\theta\log\frac{\theta}{\hat p(X_{m+1}=1\mid X^m)}
+(1-\theta)\log\frac{1-\theta}{\hat p(X_{m+1}=0\mid X^m)}
\right]. \tag*{(44)} = E θ [ θ log p ^ ( X m + 1 = 1 ∣ X m ) θ + ( 1 − θ ) log p ^ ( X m + 1 = 0 ∣ X m ) 1 − θ ] . (44)
Then, batch regret is equal to the cumulative regret
R n , ℓ ( p ^ , θ ) = ∑ m = t t + ℓ − 1 ρ m ( p ^ , θ ) , t = n ℓ . (45) R_{n,\ell}(\hat p,\theta)=\sum_{m=t}^{t+\ell-1}\rho_m(\hat p,\theta),
\qquad t=n\ell. \tag*{(45)} R n , ℓ ( p ^ , θ ) = m = t ∑ t + ℓ − 1 ρ m ( p ^ , θ ) , t = n ℓ . (45)
We now introduce an important predictor, that can be considered a conditional version of the classical additive smoothing estimator for the batch regret case. Crucially, in the batch case the predictor needs to account for both the training sequences and, in an online fashion, the test sequence as well. Formally, the predictor is defined as
p ^ β ( y ℓ ∣ x n ) = ∏ i = 0 ℓ − 1 p ^ β ( y i + 1 ∣ x n , y i ) , (46) \hat p_\beta(y^\ell\mid x^n)=\prod_{i=0}^{\ell-1}\hat p_\beta(y_{i+1}\mid x^n,y^i), \tag*{(46)} p ^ β ( y ℓ ∣ x n ) = i = 0 ∏ ℓ − 1 p ^ β ( y i + 1 ∣ x n , y i ) , (46)
where
p ^ β ( Y i + 1 = 1 ∣ x n , y i ) = n 1 ( x n ) + n 1 ( y i ) + β t + i + 2 β (47) \hat p_\beta(Y_{i+1}=1\mid x^n,y^i)=\frac{n_1(x^n)+n_1(y^i)+\beta}{t+i+2\beta} \tag*{(47)} p ^ β ( Y i + 1 = 1 ∣ x n , y i ) = t + i + 2 β n 1 ( x n ) + n 1 ( y i ) + β (47)
for a chosen additive smoothing constant β > 0 \beta>0 β > 0 , and n 1 ( z ) n_1(z) n 1 ( z ) is the number of ones in the sequence z z z . We call this predictor the conditional additive smoothing estimator, or conditional add-β \beta β estimator . As we shall see, this predictor has the correct leading-order regret on compact subsets of the Bernoulli parameter interval, even though it is not asymptotically optimal over the full interval, just like the next-token additive smoothing estimator studied by Krichevsky [20 ] .
Remark. It is useful to compare the conditional add-β \beta β estimator above with possible alternatives. One such alternative is an additive smoothing predictor that estimates the probability of the test sequence y ℓ y^\ell y ℓ ignoring the past. This predictor is defined, for a chosen parameter β > 0 \beta> 0 β > 0 , by
p ^ β ( y ℓ ∣ x n ) = ∏ i = 0 ℓ − 1 p ^ β ( y i + 1 ∣ x n , y i ) , (48) \hat{p}_{\beta}(y^\ell\mid x^n)=\prod_{i=0}^{\ell-1}\hat{p}_{\beta}(y_{i+1}\mid x^n,y^i), \tag*{(48)} p ^ β ( y ℓ ∣ x n ) = i = 0 ∏ ℓ − 1 p ^ β ( y i + 1 ∣ x n , y i ) , (48)
where
p ^ β ( Y i + 1 = 1 ∣ x n , y i ) = n 1 ( y i ) + β i + 2 β . (49) \hat{p}_{\beta}(Y_{i+1}=1\mid x^n,y^i)=\frac{n_1(y^i)+\beta}{i+2\beta}. \tag*{(49)} p ^ β ( Y i + 1 = 1 ∣ x n , y i ) = i + 2 β n 1 ( y i ) + β . (49)
Its corresponding batch regret is just the average regret for the full-sequence of length ℓ \ell ℓ . In this case, β = 1 / 2 \beta=1/2 β = 1/2 asymptotically achieves a regret with the optimal leading coefficient, R n , ℓ ( p ^ 1 / 2 ) = 1 2 log ℓ + O ( 1 ) R_{n,\ell}(\hat{p}_{1/2})=\frac{1}{2}\log\ell+O(1) R n , ℓ ( p ^ 1/2 ) = 2 1 log ℓ + O ( 1 ) . A second alternative estimates the probability of a sequence only considering the training batch x t x^t x t and none of the past symbols of y ℓ y^\ell y ℓ . Such a predictor is
p ^ β ( y ℓ ∣ x n ) = ( n 1 ( x t ) + β t + 2 β ) n 1 ( y ℓ ) ( n 0 ( x t ) + β t + 2 β ) n 0 ( y ℓ ) , (50) \hat{p}_{\beta}(y^\ell\mid x^n)=\left(\frac{n_1(x^t)+\beta}{t+2\beta}\right)^{n_1(y^\ell)}\left(\frac{n_0(x^t)+\beta}{t+2\beta}\right)^{n_0(y^\ell)}, \tag*{(50)} p ^ β ( y ℓ ∣ x n ) = ( t + 2 β n 1 ( x t ) + β ) n 1 ( y ℓ ) ( t + 2 β n 0 ( x t ) + β ) n 0 ( y ℓ ) , (50)
where n 0 ( z ) n_0(z) n 0 ( z ) is the number of zeros in z z z . In this case batch regret is equal to ℓ \ell ℓ times the Krichevsky next-token regret. The best constant is then β = β 0 ≈ 0.50922 … \beta=\beta_0\approx0.50922\ldots β = β 0 ≈ 0.50922 … , and the asymptotic batch regret equals R n , ℓ ( p ^ β 0 ) = β 0 n + o ( 1 n ) R_{n,\ell}(\hat{p}_{\beta_0})=\frac{\beta_0}{n}+o\left(\frac{1}{n}\right) R n , ℓ ( p ^ β 0 ) = n β 0 + o ( n 1 ) . One can think of the conditional add-β \beta β predictor in Eq. (46) as a refined version of the one in Eq. (50) , where the additive smoothing constant is updated every time a new token from y ℓ y^\ell y ℓ is revealed. The simpler predictor in Eq. (50) will be useful in proving results on the optimal batch regret for Markov distributions in § 9 .
Importantly, the predictor in Eq. (46) is also the posterior predictive distribution associated with a Beta( β , β ) (\beta,\beta) ( β , β ) prior after observing the t = n ℓ t=n\ell t = n ℓ training symbols. That is, the predictor can be represented via the integral formula
p ^ β ( y ℓ ∣ x n ) = ∫ 0 1 p θ ( y ℓ ) d W β ( θ ∣ x n ) , (51) \hat{p}_{\beta}(y^\ell\mid x^n)=\int_0^1 p_\theta(y^\ell)\,dW_{\beta}(\theta\mid x^n), \tag*{(51)} p ^ β ( y ℓ ∣ x n ) = ∫ 0 1 p θ ( y ℓ ) d W β ( θ ∣ x n ) , (51)
where
W β ( θ ∣ x n ) ∝ θ n 1 ( x n ) + β − 1 ( 1 − θ ) n 0 ( x n ) + β − 1 (52) W_{\beta}(\theta\mid x^n)\propto\theta^{n_1(x^n)+\beta-1}(1-\theta)^{n_0(x^n)+\beta-1} \tag*{(52)} W β ( θ ∣ x n ) ∝ θ n 1 ( x n ) + β − 1 ( 1 − θ ) n 0 ( x n ) + β − 1 (52)
is a prior (Beta) distribution on the parameter space Θ \Theta Θ that depends on the training sequences x n x^n x n . This last formulation reveals how the conditional additive smoothing predictor can also be interpreted as a mixture predictor , such that the resulting probability of the test sequence y ℓ y^\ell y ℓ is an average of the probabilities assigned to the sequence by the distributions in the class P \mathcal{P} P . The particular feature of this predictor is that the weighting function w w w changes depending on the training set, i.e., all the information obtained from the training set is encoded into the weighting function.
We now turn our attention to determining asymptotic properties of batch regret for conditional additive smoothing estimators over the class of binary memoryless sources. The next two theorems give an asymptotic expansion of batch regret under two specific parameter regimes: when the parameter belongs to the interior of the simplex, i.e., θ ∈ [ δ , 1 − δ ] \theta\in[\delta,1-\delta] θ ∈ [ δ , 1 − δ ] for some δ ∈ ( 0 , 1 2 ) \delta\in(0,\frac{1}{2}) δ ∈ ( 0 , 2 1 ) , and when the parameter lies at the endpoints, i.e., θ ∈ { 0 , 1 } \theta\in\{0,1\} θ ∈ { 0 , 1 } . Interestingly, both results hold at the same time for two complementary regimes: the large-batch-size regime, when n → ∞ n\to\infty n → ∞ and ℓ \ell ℓ may or may not go to infinity, and the finite-batch-size regime, when n n n is a fixed integer and ℓ → ∞ \ell\to\infty ℓ → ∞ . Their limitation is that they are not, by themselves, sufficient to determine the asymptotic batch regret over the full interval [ 0 , 1 ] [0,1] [ 0 , 1 ] . As we will see in the following sections, this requires a more sophisticated Poisson analysis, that depends on whether the batch size n n n is finite or not. That is, while the two complementary regimes above share the same asymptotics in the interior and endpoint parameter regimes, this is no longer true when the entire parameter space is taken into account.
We first present the interior regime asymptotics, which shows that, when only compact subsets of the parameter space are considered, all add-β \beta β estimators asymptotically have the same batch regret.
Theorem 4 (Simplex interior expansion). Let β > 0 \beta>0 β > 0 , let 0 < δ ≤ 1 2 0<\delta\le\frac{1}{2} 0 < δ ≤ 2 1 , and let t = n ℓ t=n\ell t = n ℓ . Then
sup θ ∈ [ δ , 1 − δ ] ∣ R n , ℓ ( p ^ β , θ ) − 1 2 ∑ m = t t + ℓ − 1 1 m ∣ = O β , δ ( ∑ m = t t + ℓ − 1 1 m 2 ) . (53) \sup_{\theta\in[\delta,1-\delta]}\left|R_{n,\ell}(\hat{p}_{\beta},\theta)-\frac{1}{2}\sum_{m=t}^{t+\ell-1}\frac{1}{m}\right|=O_{\beta,\delta}\left(\sum_{m=t}^{t+\ell-1}\frac{1}{m^2}\right). \tag*{(53)} θ ∈ [ δ , 1 − δ ] sup R n , ℓ ( p ^ β , θ ) − 2 1 m = t ∑ t + ℓ − 1 m 1 = O β , δ ( m = t ∑ t + ℓ − 1 m 2 1 ) . (53) Consequently,
sup θ ∈ [ δ , 1 − δ ] ∣ R n , ℓ ( p ^ β , θ ) − 1 2 log ( 1 + 1 n ) ∣ = O β , δ ( 1 n 2 ℓ ) . (54) \sup_{\theta\in[\delta,1-\delta]}\left|R_{n,\ell}(\hat{p}_{\beta},\theta)-\frac{1}{2}\log\left(1+\frac{1}{n}\right)\right|=O_{\beta,\delta}\left(\frac{1}{n^2\ell}\right). \tag*{(54)} θ ∈ [ δ , 1 − δ ] sup R n , ℓ ( p ^ β , θ ) − 2 1 log ( 1 + n 1 ) = O β , δ ( n 2 ℓ 1 ) . (54)
We defer the proof to Appendix B. The next theorem, instead, concerns the interval endpoints.
Theorem 5 (Simplex endpoints expansion). Let β > 0 \beta>0 β > 0 , let θ ∈ { 0 , 1 } \theta\in\{0,1\} θ ∈ { 0 , 1 } , and let t = n ℓ t=n\ell t = n ℓ . Then, as t → ∞ t\to\infty t → ∞ ,
R n , ℓ ( p ^ β , θ ) = β log ( 1 + 1 n ) + O β ( 1 n 2 ℓ ) . (55) R_{n,\ell}(\hat{p}_{\beta},\theta)=\beta\log\left(1+\frac{1}{n}\right)+O_{\beta}\left(\frac{1}{n^2\ell}\right). \tag*{(55)} R n , ℓ ( p ^ β , θ ) = β log ( 1 + n 1 ) + O β ( n 2 ℓ 1 ) . (55)
Proof. We prove the case θ = 1 \theta=1 θ = 1 . The case θ = 0 \theta=0 θ = 0 follows by symmetry. Define
f β ( x ) = log Γ ( x + 2 β ) − log Γ ( x + β ) , x > 0. (56) f_{\beta}(x)=\log\Gamma(x+2\beta)-\log\Gamma(x+\beta),\qquad x>0. \tag*{(56)} f β ( x ) = log Γ ( x + 2 β ) − log Γ ( x + β ) , x > 0. (56)
Then
R n , ℓ ( p ^ β , θ = 1 ) = f β ( t + ℓ ) − f β ( t ) . (57) R_{n,\ell}(\hat{p}_{\beta},\theta=1)=f_{\beta}(t+\ell)-f_{\beta}(t). \tag*{(57)} R n , ℓ ( p ^ β , θ = 1 ) = f β ( t + ℓ ) − f β ( t ) . (57)
The standard digamma expansion, uniformly for x ≥ 1 x\ge1 x ≥ 1 and fixed c > 0 c>0 c > 0 , gives
ψ ( x + c ) = log x + c − 1 / 2 x + O c ( 1 x 2 ) , (58) \psi(x+c)=\log x+\frac{c-1/2}{x}+O_c\left(\frac{1}{x^2}\right), \tag*{(58)} ψ ( x + c ) = log x + x c − 1/2 + O c ( x 2 1 ) , (58)
where ψ \psi ψ is the digamma function. Hence
f β ′ ( x ) = ψ ( x + 2 β ) − ψ ( x + β ) = β x + O β ( 1 x 2 ) . (59) f_{\beta}'(x)=\psi(x+2\beta)-\psi(x+\beta)=\frac{\beta}{x}+O_{\beta}\left(\frac{1}{x^2}\right). \tag*{(59)} f β ′ ( x ) = ψ ( x + 2 β ) − ψ ( x + β ) = x β + O β ( x 2 1 ) . (59)
Therefore,
R n , ℓ ( p ^ β , θ = 1 ) = ∫ t t + ℓ f β ′ ( x ) d x (60) R_{n,\ell}(\hat{p}_{\beta},\theta=1)=\int_t^{t+\ell}f_{\beta}'(x)\,dx \tag*{(60)} R n , ℓ ( p ^ β , θ = 1 ) = ∫ t t + ℓ f β ′ ( x ) d x (60)
= β log t + ℓ t + O β ( ∫ t t + ℓ d x x 2 ) (61) =\beta\log\frac{t+\ell}{t}+O_{\beta}\left(\int_t^{t+\ell}\frac{dx}{x^2}\right) \tag*{(61)} = β log t t + ℓ + O β ( ∫ t t + ℓ x 2 d x ) (61)
= β log ( 1 + 1 n ) + O β ( ℓ t 2 ) . (62) =\beta\log\left(1+\frac{1}{n}\right)+O_{\beta}\left(\frac{\ell}{t^2}\right). \tag*{(62)} = β log ( 1 + n 1 ) + O β ( t 2 ℓ ) . (62)
Since t = n ℓ t=n\ell t = n ℓ , the last error term is O β ( 1 / ( n ℓ ) ) O_{\beta}(1/(n\ell)) O β ( 1/ ( n ℓ )) , completing the proof.
As the next sections will clarify, it turns out that in the large-n n n regime, the regret at the interval endpoints is actually the dominant term of the regret over the full interval whenever β ≥ β 0 \beta\ge\beta_0 β ≥ β 0 . When β < β 0 \beta< \beta_0 β < β 0 , neither the interior regret nor the endpoint regret are dominant: the largest regret is achieved for parameters in the so-called Poisson regime , where θ \theta θ is of order 1 / t 1/t 1/ t , with t = n ℓ t=n\ell t = n ℓ . The phenomenon is the same that also occurs for next-token regret, which was first observed and analyzed by Krichevsky [20 ] . Krichevsky’s one-step proof already establishes the two uniform ingredients needed for the proof of the batch case: a Poisson approximation for bounded normalized counts [20 Lemma 5] and a large-count estimate for unbounded normalized counts [20 Lemma 2] . Our contribution is to insert those one-step estimates into the conditional batch setting, where the regret of the test block is a sum of one-step regrets over the ℓ \ell ℓ symbols of Y ℓ Y^\ell Y ℓ . Depending on whether n n n is finite or not, this accumulation of one-step regrets produces a different cumulative batch regret: in the large-n n n regime, the ℓ \ell ℓ regret terms are maximized by approximately the same θ t \theta_t θ t ; in the finite-n n n regime, the maximizing θ t \theta_t θ t changes along the test sequence, and a more sophisticated analysis is required.
Asymptotic Batch Regret in the Large-Batch-Size Regime
We first analyze the case when the batch size n n n is large. The next theorem gives an asymptotic expansion of batch regret for binary memoryless sources over the full interval [ 0 , 1 ] [0,1] [ 0 , 1 ] , valid whenever n → ∞ n \to\infty n → ∞ . For m ≥ 1 m \ge1 m ≥ 1 , let
ρ m ( β ) ( θ ) = E K ∼ Binomial ( m , θ ) [ θ log θ ( K + β ) / ( m + 2 β ) + ( 1 − θ ) log 1 − θ ( m − K + β ) / ( m + 2 β ) ] (63) \rho_m^{(\beta)}(\theta)=\mathbb{E}_{K\sim\operatorname{Binomial}(m,\theta)}\left[\theta\log\frac{\theta}{(K+\beta)/(m+2\beta)}+(1-\theta)\log\frac{1-\theta}{(m-K+\beta)/(m+2\beta)}\right] \tag*{(63)} ρ m ( β ) ( θ ) = E K ∼ Binomial ( m , θ ) [ θ log ( K + β ) / ( m + 2 β ) θ + ( 1 − θ ) log ( m − K + β ) / ( m + 2 β ) 1 − θ ] (63)
be the one-step regret of the add-β \beta β predictor after m m m past samples. Define, for λ > 0 \lambda>0 λ > 0 ,
Δ β ( λ ) ≜ λ [ log λ − E Z ∼ Pois ( λ ) log ( Z + β ) ] , (64) \Delta_\beta(\lambda)\triangleq\lambda\left[\log\lambda-\mathbb{E}_{Z\sim\operatorname{Pois}(\lambda)}\log(Z+\beta)\right], \tag*{(64)} Δ β ( λ ) ≜ λ [ log λ − E Z ∼ Pois ( λ ) log ( Z + β ) ] , (64)
and set Δ β ( 0 ) = 0 \Delta_\beta(0)=0 Δ β ( 0 ) = 0 . Finally, let
C β ≜ β + sup λ ≥ 0 Δ β ( λ ) . (65) C_\beta\triangleq\beta+\sup_{\lambda\ge0}\Delta_\beta(\lambda). \tag*{(65)} C β ≜ β + λ ≥ 0 sup Δ β ( λ ) . (65)
Theorem 6 (Full-simplex asymptotics for add-β \beta β estimators): Let β > 0 \beta>0 β > 0 , let t = n ℓ t=n\ell t = n ℓ , and let n → ∞ n\to\infty n → ∞ , with ℓ = ℓ ( n ) \ell=\ell(n) ℓ = ℓ ( n ) any positive integer sequence. Then
max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β , θ ) = C β log ( 1 + 1 n ) + o ( 1 n ) (66) \max_{\theta\in[0,1]}R_{n,\ell}(\hat p_\beta,\theta)=C_\beta\log\left(1+\frac{1}{n}\right)+o\left(\frac{1}{n}\right) \tag*{(66)} θ ∈ [ 0 , 1 ] max R n , ℓ ( p ^ β , θ ) = C β log ( 1 + n 1 ) + o ( n 1 ) (66) = C β n + o ( 1 n ) . (67) =\frac{C_\beta}{n}+o\left(\frac{1}{n}\right). \tag*{(67)} = n C β + o ( n 1 ) . (67)
We defer the proof to Appendix C. The theorem shows that the full-simplex maximum may not be attained either in the simplex interior or at the endpoints. More specifically, the proof of Theorem 6 distinguishes between three regimes depending on the asymptotic order of θ t \theta_t θ t . If either t θ t → 0 t\theta_t\to0 t θ t → 0 or t ( 1 − θ t ) → 0 t(1-\theta_t)\to0 t ( 1 − θ t ) → 0 , i.e., the endpoint regime , then
R n , ℓ ( p ^ β , θ t ) = β log ( 1 + 1 n ) + o ( 1 n ) , (68) R_{n,\ell}(\hat p_\beta,\theta_t)=\beta\log\left(1+\frac{1}{n}\right)+o\left(\frac{1}{n}\right), \tag*{(68)} R n , ℓ ( p ^ β , θ t ) = β log ( 1 + n 1 ) + o ( n 1 ) , (68)
that is, the leading coefficient of batch regret for θ t \theta_t θ t is β \beta β . If instead t θ t → λ t\theta_t\to\lambda t θ t → λ or t ( 1 − θ t ) → λ t(1-\theta_t)\to\lambda t ( 1 − θ t ) → λ , 0 < λ < ∞ 0<\lambda<\infty 0 < λ < ∞ , i.e., the Poisson regime , then
R n , ℓ ( p ^ β , θ t ) = ( β + Δ β ( λ ) ) log ( 1 + 1 n ) + o ( 1 n ) . (69) R_{n,\ell}(\hat p_\beta,\theta_t)=(\beta+\Delta_\beta(\lambda))\log\left(1+\frac{1}{n}\right)+o\left(\frac{1}{n}\right). \tag*{(69)} R n , ℓ ( p ^ β , θ t ) = ( β + Δ β ( λ )) log ( 1 + n 1 ) + o ( n 1 ) . (69)
Finally if both t θ t → ∞ t\theta_t\to\infty t θ t → ∞ and t ( 1 − θ t ) → ∞ t(1-\theta_t)\to\infty t ( 1 − θ t ) → ∞ , i.e., the interior regime , then
R n , ℓ ( p ^ β , θ t ) = 1 2 log ( 1 + 1 n ) + o ( 1 n ) . (70) R_{n,\ell}(\hat p_\beta,\theta_t)=\frac{1}{2}\log\left(1+\frac{1}{n}\right)+o\left(\frac{1}{n}\right). \tag*{(70)} R n , ℓ ( p ^ β , θ t ) = 2 1 log ( 1 + n 1 ) + o ( n 1 ) . (70)
This shows that the regimes treated by Theorems 4 and 5 are retrieved as special cases of Theorem 6. However, Theorems 4 and 5 give sharper reminders.
The preceding theorem reduces the full-simplex performance of the add-β \beta β rule to the single function Δ β \Delta_\beta Δ β . In fact, Δ β \Delta_\beta Δ β accounts for all the three regimes described above. For λ = 0 \lambda=0 λ = 0 , one has β + Δ β ( 0 ) = β \beta+\Delta_\beta(0)=\beta β + Δ β ( 0 ) = β , which is the leading regret coefficient in the endpoint regime. On the other hand, a Taylor expansion of log ( Z + β ) \log(Z+\beta) log ( Z + β ) around λ \lambda λ gives, for fixed β > 0 \beta>0 β > 0 ,
E log ( Z + β ) = log λ + β − 1 2 λ + O β ( 1 λ 2 ) , (71) \mathbb{E}\log(Z+\beta)=\log\lambda+\frac{\beta-\frac{1}{2}}{\lambda}+O_\beta\left(\frac{1}{\lambda^2}\right), \tag*{(71)} E log ( Z + β ) = log λ + λ β − 2 1 + O β ( λ 2 1 ) , (71)
where Z ∼ Pois ( λ ) Z\sim\operatorname{Pois}(\lambda) Z ∼ Pois ( λ ) , and hence
β + Δ β ( λ ) → 1 2 , λ → ∞ , (72) \beta+\Delta_\beta(\lambda)\to\frac{1}{2},\qquad\lambda\to\infty, \tag*{(72)} β + Δ β ( λ ) → 2 1 , λ → ∞ , (72)
which corresponds to the interior regime. The Poisson regime is obtained for λ ∈ ( 0 , ∞ ) \lambda\in(0,\infty) λ ∈ ( 0 , ∞ ) .
Krichevsky obtained the optimal coefficient β 0 \beta_0 β 0 for the next-token regret by finding the value of β \beta β for which the endpoint coefficient and the largest coefficient in the Poisson regime were equal [20 Fig. 1 and Lemma 6] . In our notation it can be characterized as
β 0 = inf { β > 0 : Δ β ( λ ) ≤ 0 for every λ > 0 } . (73) \beta_0=\inf\{\beta>0:\Delta_\beta(\lambda)\le0\text{ for every }\lambda>0\}. \tag*{(73)} β 0 = inf { β > 0 : Δ β ( λ ) ≤ 0 for every λ > 0 } . (73)
For the sake of completeness, the following result establishes that the coefficient characterized by Eq. (73) is indeed the same as Krichevsky’s optimal β 0 \beta_0 β 0 [20 ] .
Proposition 7 (Optimal add-β \beta β parameter): Let β 0 \beta_0 β 0 be defined by (73). Then the defining set is nonempty and
1 2 ≤ β 0 ≤ 1. (74) \frac{1}{2}\le\beta_0\le1. \tag*{(74)} 2 1 ≤ β 0 ≤ 1. (74) Furthermore,
min β > 0 C β = C β 0 = β 0 . (75) \min_{\beta>0}C_\beta=C_{\beta_0}=\beta_0. \tag*{(75)} β > 0 min C β = C β 0 = β 0 . (75) We defer the proof to Appendix D. Numerical evaluation gives β 0 ≈ 0.509224 \beta_0\approx0.509224 β 0 ≈ 0.509224 , in agreement with Krichevsky’s analysis. The proof of Proposition 7 also clarifies that the optimal bias β 0 \beta_0 β 0 is the threshold point after which the endpoint regret becomes dominant over the largest Poisson regret. In fact, the function C β C_\beta C β can be represented as the upper envelope of
three functions, each corresponding to the largest regret in the interior, Poisson, and endpoint regimes. If we define the Poisson branch as
P β = β + sup λ ∈ M β Δ β ( λ ) , (76) P_\beta= \beta+ \sup_{\lambda\in\mathcal{M}_\beta} \Delta_\beta(\lambda), \tag*{(76)} P β = β + λ ∈ M β sup Δ β ( λ ) , (76)
where
M β = { λ > 0 : λ is a local maximum of Δ β ( λ ) } , (77) \mathcal{M}_\beta= \{\lambda> 0 : \lambda\text{ is a local maximum of }\Delta_\beta(\lambda)\}, \tag*{(77)} M β = { λ > 0 : λ is a local maximum of Δ β ( λ )} , (77)
with the convention that P β = − ∞ P_\beta= -\infty P β = − ∞ if M β \mathcal{M}_\beta M β is empty, then we have that the optimal regret coefficient satisfies
C β = max { 1 2 , β , P β } . (78) C_\beta= \max\left\{\frac{1}{2}, \beta, P_\beta\right\}. \tag*{(78)} C β = max { 2 1 , β , P β } . (78)
Indeed, if the supremum of Δ β \Delta_\beta Δ β over ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) exceeds both boundary values 0 0 0 and 1 / 2 − β 1/2-\beta 1/2 − β , then, because Δ β ( 0 ) = 0 \Delta_\beta(0)=0 Δ β ( 0 ) = 0 and Δ β ( λ ) → 1 / 2 − β \Delta_\beta(\lambda)\to1/2-\beta Δ β ( λ ) → 1/2 − β as λ → ∞ \lambda\to\infty λ → ∞ , the supremum is attained at some finite positive point, which must belong to M β \mathcal{M}_\beta M β . The structure is illustrated numerically in Figure 1 and Figure 2 . Figure 1 shows representative shapes of Δ β \Delta_\beta Δ β , while Figure 2 plots the three branches and their upper envelope.
Figure 1. The Poisson layer function β + Δ β ( λ ) \beta+ \Delta_\beta(\lambda) β + Δ β ( λ ) for three choices of β \beta β . The endpoint value is β + Δ β ( 0 ) = β \beta+ \Delta_\beta(0) = \beta β + Δ β ( 0 ) = β , while all curves converge to the compact-interior value 1 / 2 1/2 1/2 as λ → ∞ \lambda\to\infty λ → ∞ .
Figure 2. The full-simplex add-β \beta β coefficient C β C_\beta C β as the upper envelope of the compact-interior branch 1 / 2 1/2 1/2 , the endpoint branch β \beta β , and the Poisson branch P β P_\beta P β . The crossing point is Krichevsky’s value β 0 ≈ 0.509224 \beta_0 \approx0.509224 β 0 ≈ 0.509224 , where C β 0 = β 0 C_{\beta_0} = \beta_0 C β 0 = β 0 .
Asymptotic Batch Regret in the Finite-Batch-Size Regime
Theorem 6 assumes that the number of training sequences grows, so that the test block is short relative to the training data. There is also a complementary fixed-n n n limit. In this case the optimal parameter θ t \theta_t θ t changes by a non-negligible multiplicative factor over the test block, and the one-step Poisson boundary function has to be integrated. For a fixed positive integer n n n and λ ≥ 0 \lambda\ge0 λ ≥ 0 , define the normalized finite-n n n boundary function
Δ β , n ( λ ) = 1 log ( 1 + 1 / n ) ∫ 1 1 + 1 / n Δ β ( u λ ) u d u (79) \Delta_{\beta,n}(\lambda) = \frac{1}{\log(1+1/n)}\int_{1}^{1+1/n}\frac{\Delta_\beta(u\lambda)}{u}\,du \tag*{(79)} Δ β , n ( λ ) = log ( 1 + 1/ n ) 1 ∫ 1 1 + 1/ n u Δ β ( u λ ) d u (79)
= 1 log ( 1 + 1 / n ) ∫ λ ( 1 + 1 / n ) λ Δ β ( μ ) μ d μ , (80) = \frac{1}{\log(1+1/n)}\int_{\lambda}^{(1+1/n)\lambda}\frac{\Delta_\beta(\mu)}{\mu}\,d\mu, \tag*{(80)} = log ( 1 + 1/ n ) 1 ∫ λ ( 1 + 1/ n ) λ μ Δ β ( μ ) d μ , (80)
with the value at λ = 0 \lambda=0 λ = 0 defined by continuity. Geometrically, Δ β , n \Delta_{\beta,n} Δ β , n can be interpreted as a logarithmic moving average of Δ β \Delta_\beta Δ β over the interval from λ \lambda λ to ( 1 + 1 / n ) λ (1+1/n)\lambda ( 1 + 1/ n ) λ , as it is clear by substituting u = e s u=e^s u = e s ,
Δ β , n ( λ ) = 1 log ( 1 + 1 / n ) ∫ 0 log ( 1 + 1 / n ) Δ β ( λ e s ) d s . (81) \Delta_{\beta,n}(\lambda) = \frac{1}{\log(1+1/n)}\int_{0}^{\log(1+1/n)}\Delta_\beta(\lambda e^s)\,ds. \tag*{(81)} Δ β , n ( λ ) = log ( 1 + 1/ n ) 1 ∫ 0 l o g ( 1 + 1/ n ) Δ β ( λ e s ) d s . (81)
This function is the equivalent of Δ β \Delta_\beta Δ β for the finite-n n n regime, in that it is the leading coefficient of the batch regret of conditional add-β \beta β estimators for binary memoryless sources when the batch size n n n is fixed and the sequence length ℓ \ell ℓ is large. The following theorem formalizes this fact.
Theorem 8 (Fixed-n n n full-simplex limit for add-β \beta β estimators): Let β > 0 \beta>0 β > 0 and fix a positive integer n n n . Then, as ℓ → ∞ \ell\to\infty ℓ → ∞ ,
max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β , θ ) ⟶ C β , n log ( 1 + 1 n ) , (82) \max_{\theta\in[0,1]}R_{n,\ell}(\hat{p}_\beta,\theta)\longrightarrow C_{\beta,n}\log\left(1+\frac{1}{n}\right), \tag*{(82)} θ ∈ [ 0 , 1 ] max R n , ℓ ( p ^ β , θ ) ⟶ C β , n log ( 1 + n 1 ) , (82)
where
C β , n ≜ β + sup λ ≥ 0 Δ β , n ( λ ) . (83) C_{\beta,n}\triangleq\beta+\sup_{\lambda\ge0}\Delta_{\beta,n}(\lambda). \tag*{(83)} C β , n ≜ β + λ ≥ 0 sup Δ β , n ( λ ) . (83)
The proof is given in Appendix § 16 . Similar to the large-n n n case, Δ β , n ( λ ) \Delta_{\beta,n}(\lambda) Δ β , n ( λ ) includes the interior and endpoint regimes as its extreme values: β + Δ β , n ( 0 ) = β \beta+\Delta_{\beta,n}(0)=\beta β + Δ β , n ( 0 ) = β is the regret coefficient at the endpoints, while β + lim λ → ∞ Δ β , n ( λ ) = 1 / 2 \beta+\lim_{\lambda\to\infty}\Delta_{\beta,n}(\lambda)=1/2 β + lim λ → ∞ Δ β , n ( λ ) = 1/2 is the common coefficient in the interior of the simplex. Also notice that as n → ∞ n\to\infty n → ∞ the average in Eq. (81) converges to the single point Δ β ( λ ) \Delta_\beta(\lambda) Δ β ( λ ) , retrieving the large-n n n coefficient; it can be shown that the convergence can be made uniform, so that lim n → ∞ C β , n = C β \lim_{n\to\infty}C_{\beta,n}=C_\beta lim n → ∞ C β , n = C β .
Parallel to Eq. (78) , also the fixed-n n n coefficient C β , n C_{\beta,n} C β , n has an upper-envelope structure. If we let
M β , n = { λ > 0 : λ is a local maximum of Δ β , n } (84) \mathcal{M}_{\beta,n}=\{\lambda>0:\lambda\text{ is a local maximum of }\Delta_{\beta,n}\} \tag*{(84)} M β , n = { λ > 0 : λ is a local maximum of Δ β , n } (84)
and define the Poisson branch by
P β , n = β + sup λ ∈ M β , n Δ β , n ( λ ) , (85) P_{\beta,n}=\beta+\sup_{\lambda\in\mathcal{M}_{\beta,n}}\Delta_{\beta,n}(\lambda), \tag*{(85)} P β , n = β + λ ∈ M β , n sup Δ β , n ( λ ) , (85)
with the convention P β , n = − ∞ P_{\beta,n}=-\infty P β , n = − ∞ if M β , n \mathcal{M}_{\beta,n} M β , n is empty, then we have that
C β , n = max { 1 2 , β , P β , n } . (86) C_{\beta,n}=\max\left\{\frac{1}{2},\beta,P_{\beta,n}\right\}. \tag*{(86)} C β , n = max { 2 1 , β , P β , n } . (86)
The natural finite-n n n analogue of Krichevsky’s threshold β 0 \beta_0 β 0 is
β n ⋆ = inf { β > 0 : Δ β , n ( λ ) ≤ 0 for every λ > 0 } . (87) \beta_n^\star= \inf\{\beta> 0 : \Delta_{\beta,n}(\lambda) \le0 \text{ for every } \lambda> 0\}. \tag*{(87)} β n ⋆ = inf { β > 0 : Δ β , n ( λ ) ≤ 0 for every λ > 0 } . (87)
Thus β n ⋆ \beta_n^\star β n ⋆ is the smallest bias for which the endpoint branch dominates the Poisson branch. The following theorem shows that β n ⋆ \beta_n^\star β n ⋆ is indeed the optimal bias for the finite-n n n regime and, just like in the large-n n n regime, the optimal regret coefficient and the optimal bias are equal. Furthermore, β n ⋆ \beta_n^\star β n ⋆ is strictly increasing with n n n and converges to β 0 \beta_0 β 0 as n → ∞ n \to\infty n → ∞ .
Theorem 9 (Optimal fixed-n n n add-β \beta β parameters): For every fixed positive integer n n n , there exist finite β n ⋆ > 1 / 2 \beta_n^\star> 1/2 β n ⋆ > 1/2 and λ n ⋆ > 1 \lambda_n^\star> 1 λ n ⋆ > 1 such that
Δ β n ⋆ , n ( λ n ⋆ ) = 0 (88) \Delta_{\beta_n^\star,n}(\lambda_n^\star) = 0 \tag*{(88)} Δ β n ⋆ , n ( λ n ⋆ ) = 0 (88) and β n ⋆ \beta_n^\star β n ⋆ satisfies
min β > 0 C β , n = C β n ⋆ , n = β n ⋆ . (89) \min_{\beta>0} C_{\beta,n} = C_{\beta_n^\star,n} = \beta_n^\star. \tag*{(89)} β > 0 min C β , n = C β n ⋆ , n = β n ⋆ . (89) Furthermore, as function of n n n , the optimal bias satisfies
1 2 < β 1 ⋆ < β 2 ⋆ < ⋯ < β 0 and lim n → ∞ β n ⋆ = β 0 . (90) \frac{1}{2} < \beta_1^\star< \beta_2^\star< \cdots< \beta_0
\quad\text{and}\quad
\lim_{n\to\infty} \beta_n^\star= \beta_0. \tag*{(90)} 2 1 < β 1 ⋆ < β 2 ⋆ < ⋯ < β 0 and n → ∞ lim β n ⋆ = β 0 . (90)
The proof is given in Appendix F. The equations
Δ β , n ( λ ) = 0 , ∂ ∂ λ Δ β , n ( λ ) = 0 (91) \Delta_{\beta,n}(\lambda) = 0,\qquad
\frac{\partial}{\partial\lambda}\Delta_{\beta,n}(\lambda) = 0 \tag*{(91)} Δ β , n ( λ ) = 0 , ∂ λ ∂ Δ β , n ( λ ) = 0 (91)
allow us to numerically characterize ( β n ⋆ , λ n ⋆ ) (\beta_n^\star,\lambda_n^\star) ( β n ⋆ , λ n ⋆ ) . Solving them for increasing n n n gives Figure 3 , in agreement with Theorem 9. Finally, Figure 4 compares the limiting prediction in Theorems 8 and 9 with the actual batch regret for finite ℓ \ell ℓ , by tracking the value of
Figure 3. Numerically computed optimal fixed-n n n parameters β n ⋆ \beta_n^\star β n ⋆ , obtained from Eq. (91) . Theorem 9 proves that they increase strictly to the one-step Krichevsky value β 0 ≈ 0.509224 \beta_0 \approx0.509224 β 0 ≈ 0.509224 , shown by the dashed line.
Figure 4. Finite-ℓ \ell ℓ validation of the asymptotic fixed-n n n batch regret limit. The vertical axis is the worst-case regret divided by its fixed-n n n asymptotic limit, namely V n , ℓ V_{n,\ell} V n , ℓ from Eq. (92) .
V n , ℓ = max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β n ⋆ , θ ) β n ⋆ log ( 1 + 1 / n ) . (92) V_{n,\ell} = \frac{\max_{\theta\in[0,1]} R_{n,\ell}(\hat{p}_{\beta_n^\star},\theta)}{\beta_n^\star\log(1+1/n)}. \tag*{(92)} V n , ℓ = β n ⋆ log ( 1 + 1/ n ) max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β n ⋆ , θ ) . (92)
As ℓ → ∞ \ell\to\infty ℓ → ∞ , V n , ℓ V_{n,\ell} V n , ℓ converges to one as expected.
Optimal batch regret for binary memoryless sources
Having studied the asymptotic batch regret of conditional additive smoothing estimators, we can now use Theorem 2 to sharpen the converse for binary memoryless sources over all predictors. The first-order asymptotic minimax batch regret for memoryless sources was first obtained by Fogel and Feder [26 Theorem 2] , using the sequential Braess–Sauer predictor [21 ] for achievability. Our lower bound below identifies the same leading constant while improving the converse remainder to O ( 1 / ( n 2 ℓ ) ) O(1/(n^2\ell)) O ( 1/ ( n 2 ℓ )) .
Theorem 10 (Minimax lower bound for Bernoulli batch prediction): For the class of i.i.d. Bernoulli distributions, for any positive integer sequences n n n and ℓ \ell ℓ such that t = n ℓ → ∞ t=n\ell\to\infty t = n ℓ → ∞ , we have that
min p ^ max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ , θ ) ≥ 1 2 log ( 1 + 1 n ) − O ( 1 n 2 ℓ ) . (93) \min_{\hat{p}}\max_{\theta\in[0,1]} R_{n,\ell}(\hat{p},\theta)
\ge\frac{1}{2}\log\left(1+\frac{1}{n}\right)-O\left(\frac{1}{n^2\ell}\right). \tag*{(93)} p ^ min θ ∈ [ 0 , 1 ] max R n , ℓ ( p ^ , θ ) ≥ 2 1 log ( 1 + n 1 ) − O ( n 2 ℓ 1 ) . (93)
The proof is given in Appendix G. Let now
R n , ℓ ⋆ ≜ inf p ^ sup θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ , θ ) (94) R_{n,\ell}^\star\triangleq\inf_{\hat{p}}\sup_{\theta\in[0,1]} R_{n,\ell}(\hat{p},\theta) \tag*{(94)} R n , ℓ ⋆ ≜ p ^ inf θ ∈ [ 0 , 1 ] sup R n , ℓ ( p ^ , θ ) (94)
be the unrestricted batch regret for binary memoryless sources. Then, [26 Theorem 2] shows that the preceding lower bound can be combined with the optimal one-step predictor defined by Braess and Sauer [21 ] to obtain a tight first-order characterization of R n , ℓ ⋆ R_{n,\ell}^\star R n , ℓ ⋆ .
Corollary 11 (First-order minimax Bernoulli batch regret [26 ] ): For any positive integer sequences n n n and ℓ \ell ℓ such that t = n ℓ → ∞ t=n\ell\to\infty t = n ℓ → ∞ ,
R n , ℓ ⋆ = ( 1 2 + o ( 1 ) ) log ( 1 + 1 n ) . (95) R_{n,\ell}^\star= \left(\frac{1}{2}+o(1)\right)\log\left(1+\frac{1}{n}\right). \tag*{(95)} R n , ℓ ⋆ = ( 2 1 + o ( 1 ) ) log ( 1 + n 1 ) . (95)
Consequently, for every fixed n n n ,
lim ℓ → ∞ R n , ℓ ⋆ = 1 2 log ( 1 + 1 n ) , (96) \lim_{\ell\to\infty} R_{n,\ell}^\star= \frac{1}{2}\log\left(1+\frac{1}{n}\right), \tag*{(96)} ℓ → ∞ lim R n , ℓ ⋆ = 2 1 log ( 1 + n 1 ) , (96)
whereas, if n → ∞ n\to\infty n → ∞ ,
R n , ℓ ⋆ = 1 2 n + o ( 1 n ) . (97) R_{n,\ell}^\star= \frac{1}{2n}+o\left(\frac{1}{n}\right). \tag*{(97)} R n , ℓ ⋆ = 2 n 1 + o ( n 1 ) . (97)
Proof. The proof follows the one of [26 Theorem 2] . We include it here for the sake of completeness. Let q ˉ m B S \bar{q}^{\mathrm{BS}}_m q ˉ m BS be the binary one-step predictor of Braess and Sauer [21 Sec. 5, Eqs. (5.18)–(5.21)] , and let
ρ ˉ m B S ≜ sup θ ∈ [ 0 , 1 ] E θ [ D ( Bernoulli ( θ ) ∥ q m B S ( ⋅ ∣ X m ) ) ] . (98) \bar{\rho}^{\mathrm{BS}}_m \triangleq\sup_{\theta\in[0,1]} \mathbb{E}_{\theta}\left[D\left(\operatorname{Bernoulli}(\theta) \mathbin{\|} q^{\mathrm{BS}}_m(\cdot\mid X^m)\right)\right]. \tag*{(98)} ρ ˉ m BS ≜ θ ∈ [ 0 , 1 ] sup E θ [ D ( Bernoulli ( θ ) ∥ q m BS ( ⋅ ∣ X m ) ) ] . (98)
Their uniform one-step analysis gives
ρ ˉ m B S = 1 2 m + o ( 1 m ) . (99) \bar{\rho}^{\mathrm{BS}}_m = \frac{1}{2m} + o\left(\frac{1}{m}\right). \tag*{(99)} ρ ˉ m BS = 2 m 1 + o ( m 1 ) . (99)
Write
ρ ˉ m B S ≤ 1 2 + ε m m , ε m → 0 , (100) \bar{\rho}^{\mathrm{BS}}_m \le\frac{\frac{1}{2}+\varepsilon_m}{m}, \qquad\varepsilon_m \to0, \tag*{(100)} ρ ˉ m BS ≤ m 2 1 + ε m , ε m → 0 , (100)
and set ε ˉ t = sup m ≥ t ∣ ε m ∣ \bar{\varepsilon}_t=\sup_{m\ge t}|\varepsilon_m| ε ˉ t = sup m ≥ t ∣ ε m ∣ , so that ε ˉ t → 0 \bar{\varepsilon}_t\to0 ε ˉ t → 0 . Sequentially compose these one-step rules across the test block:
p ^ B S ( y ℓ ∣ x t ) = ∏ i = 0 ℓ − 1 q t + i B S ( y i + 1 ∣ x t , y i ) . (101) \widehat{p}_{\mathrm{BS}}(y^\ell\mid x^t)=\prod_{i=0}^{\ell-1}q^{\mathrm{BS}}_{t+i}(y_{i+1}\mid x^t,y^i). \tag*{(101)} p BS ( y ℓ ∣ x t ) = i = 0 ∏ ℓ − 1 q t + i BS ( y i + 1 ∣ x t , y i ) . (101)
The chain rule for conditional KL divergence gives
sup θ R n , ℓ ( p ^ B S , θ ) ≤ ∑ m = t t + ℓ − 1 ρ ˉ m B S (102) \sup_{\theta}R_{n,\ell}(\widehat{p}_{\mathrm{BS}},\theta)\le\sum_{m=t}^{t+\ell-1}\bar{\rho}^{\mathrm{BS}}_m \tag*{(102)} θ sup R n , ℓ ( p BS , θ ) ≤ m = t ∑ t + ℓ − 1 ρ ˉ m BS (102)
≤ ( 1 2 + ε ˉ t ) ∑ m = t t + ℓ − 1 1 m . (103) \le\left(\frac{1}{2}+\bar{\varepsilon}_t\right)\sum_{m=t}^{t+\ell-1}\frac{1}{m}. \tag*{(103)} ≤ ( 2 1 + ε ˉ t ) m = t ∑ t + ℓ − 1 m 1 . (103)
Using the classical harmonic-sum estimate,
∑ m = t t + ℓ − 1 1 m = log ( 1 + 1 n ) + O ( 1 n 2 ℓ ) , (104) \sum_{m=t}^{t+\ell-1}\frac{1}{m}=\log\left(1+\frac{1}{n}\right)+O\left(\frac{1}{n^2\ell}\right), \tag*{(104)} m = t ∑ t + ℓ − 1 m 1 = log ( 1 + n 1 ) + O ( n 2 ℓ 1 ) , (104)
and the fact that
1 / ( n 2 ℓ ) log ( 1 + 1 / n ) = O ( 1 n ℓ ) = o ( 1 ) , (105) \frac{1/(n^2\ell)}{\log(1+1/n)}=O\left(\frac{1}{n\ell}\right)=o(1), \tag*{(105)} log ( 1 + 1/ n ) 1/ ( n 2 ℓ ) = O ( n ℓ 1 ) = o ( 1 ) , (105)
we obtain the upper bound in Eq. (95) . The lower bound follows directly from Theorem 10.
For fixed n n n and ℓ \ell ℓ , Theorem 2 shows that there is a prior with finite support achieving the optimal batch regret. However, if applied directly to the Bernoulli batch problem, it gives an exponential support upper bound. Thankfully, the following direct conditional counterpart of the classical Witsenhausen bound for the binomial channel [40 ] provides a much smaller bound.
Corollary 12 (Finite-support Bernoulli least-favorable prior). For the binary memoryless batch problem with t = n ℓ t=n\ell t = n ℓ training symbols and a test block of length ℓ \ell ℓ , there exists a capacity-achieving prior on [ 0 , 1 ] [0,1] [ 0 , 1 ] with at most
( t + ℓ ) + 1 = ( n + 1 ) ℓ + 1 (106) (t+\ell)+1=(n+1)\ell+1 \tag*{(106)} ( t + ℓ ) + 1 = ( n + 1 ) ℓ + 1 (106) support points.
Proof. The parameter interval is compact and the finite-output channel law is continuous in θ \theta θ , so a capacity-achieving prior exists in Theorem 2. Due to the i.i.d. nature of the data, a minimax predictor may be taken to depend only on the training count K ∈ { 0 , … , t } K\in\{0,\ldots,t\} K ∈ { 0 , … , t } and the test count J ∈ { 0 , … , ℓ } J\in\{0,\ldots,\ell\} J ∈ { 0 , … , ℓ } . Let Q ⋆ ( j ∣ k ) Q^\star(j\mid k) Q ⋆ ( j ∣ k ) be such a minimax count-based kernel and let M = t + ℓ M=t+\ell M = t + ℓ . The vectors appearing in Eq. (177) have coordinates
v θ ( k , j ) = ( t k ) ( ℓ j ) θ k + j ( 1 − θ ) M − k − j − ( t k ) θ k ( 1 − θ ) t − k Q ⋆ ( j ∣ k ) . (107) v_\theta(k,j)=\binom{t}{k}\binom{\ell}{j}\theta^{k+j}(1-\theta)^{M-k-j}-\binom{t}{k}\theta^k(1-\theta)^{t-k}Q^\star(j\mid k). \tag*{(107)} v θ ( k , j ) = ( k t ) ( j ℓ ) θ k + j ( 1 − θ ) M − k − j − ( k t ) θ k ( 1 − θ ) t − k Q ⋆ ( j ∣ k ) . (107)
Every coordinate in Eq. (107) is a polynomial in θ \theta θ of degree at most M M M . Equivalently, the vector-valued map θ ↦ v θ \theta\mapsto v_\theta θ ↦ v θ is an affine image of the degree-M M M Bernstein vector
( ( M s ) θ s ( 1 − θ ) M − s ) s = 0 M . (108) \left(\binom{M}{s}\theta^s(1-\theta)^{M-s}\right)_{s=0}^{M}. \tag*{(108)} ( ( s M ) θ s ( 1 − θ ) M − s ) s = 0 M . (108)
Because these M + 1 M+1 M + 1 coordinates sum to one, their affine hull has dimension at most M M M . The second part of the proof of Theorem 2 therefore shows that the support points are at most M + 1 = ( n + 1 ) ℓ + 1 M+1=(n+1)\ell+1 M + 1 = ( n + 1 ) ℓ + 1 .
We believe that the previous bound can be improved further by adapting existing results such as [41 ] to the conditional-batch case.
Markov Sources
We now shift our attention to binary first-order Markov sources, i.e., distributions of the form
p ν , p , q ( x ℓ ) = π ν ( x 1 ) ∏ j = 2 ℓ P p , q ( x j ∣ x j − 1 ) , (109) p_{\nu,p,q}(x^\ell)=\pi_\nu(x_1)\prod_{j=2}^{\ell}P_{p,q}(x_j\mid x_{j-1}), \tag*{(109)} p ν , p , q ( x ℓ ) = π ν ( x 1 ) j = 2 ∏ ℓ P p , q ( x j ∣ x j − 1 ) , (109)
where π ν ( 1 ) = ν \pi_\nu(1)=\nu π ν ( 1 ) = ν , and
P p , q = ( 1 − p p q 1 − q ) . (110) P_{p,q}=\begin{pmatrix}1-p & p\\ q & 1-q\end{pmatrix}. \tag*{(110)} P p , q = ( 1 − p q p 1 − q ) . (110)
Following [18 ] , we fix 0 < δ < 1 / 2 0<\delta<1/2 0 < δ < 1/2 and restrict the transition parameters to
( p , q ) ∈ Θ δ ≜ [ δ , 1 − δ ] 2 . (111) (p,q)\in\Theta_\delta\triangleq[\delta,1-\delta]^2. \tag*{(111)} ( p , q ) ∈ Θ δ ≜ [ δ , 1 − δ ] 2 . (111)
This allows us to avoid complex regret behavior near the boundary of the simplex. Contrary to the Bernoulli case, where the entire training batch can be represented as a single i.i.d. sequence of n ℓ n\ell n ℓ tokens, in the Markov case this is no longer true, as every training sequence in the batch is independently re-initialized from the initial distribution of the source. Because of this, in the batch case the parametrization of the initial distribution of the Markov source becomes fundamental, and the optimal batch regret is different for different parametrizations. In this work we focus on three canonical cases, namely if the initial distribution is known, if it is a free parameter, or if it is equal to the stationary distribution of the source. The three scenarios are represented by the following three classes of distributions:
M δ , ν 0 k n o w n ≜ { ( ν 0 , p , q ) : ( p , q ) ∈ Θ δ } , (112) \mathcal{M}^{\mathrm{known}}_{\delta,\nu_0}\triangleq\{(\nu_0,p,q):(p,q)\in\Theta_\delta\}, \tag*{(112)} M δ , ν 0 known ≜ {( ν 0 , p , q ) : ( p , q ) ∈ Θ δ } , (112)
M δ s t a t ≜ { ( p p + q , p , q ) : ( p , q ) ∈ Θ δ } , (113) \mathcal{M}^{\mathrm{stat}}_{\delta}\triangleq\left\{\left(\frac{p}{p+q},p,q\right):(p,q)\in\Theta_\delta\right\}, \tag*{(113)} M δ stat ≜ { ( p + q p , p , q ) : ( p , q ) ∈ Θ δ } , (113)
M δ f r e e ≜ { ( ν , p , q ) : ν ∈ [ 0 , 1 ] , ( p , q ) ∈ Θ δ } . (114) \mathcal{M}^{\mathrm{free}}_{\delta}\triangleq\{(\nu,p,q):\nu\in[0,1],(p,q)\in\Theta_\delta\}. \tag*{(114)} M δ free ≜ {( ν , p , q ) : ν ∈ [ 0 , 1 ] , ( p , q ) ∈ Θ δ } . (114)
When the result is independent of the parametrization of the initial distribution, we simply write M δ \mathcal{M}_\delta M δ to mean any of the classes above.
We now focus on studying the asymptotic batch regret for Markov sources under the three parametrizations above. In this section we limit ourselves to the regime where n → ∞ n \to\infty n → ∞ and ℓ → ∞ \ell\to\infty ℓ → ∞ jointly. The n n n training sequences X 1 , … , X n \mathbf{X}_1,\ldots,\mathbf{X}_n X 1 , … , X n and the test sequence Y \mathbf{Y} Y are independent conditional on the source parameter. For a predictor factored as
p ^ ( y ℓ ∣ x n ) = π ^ ( y 1 ∣ x n ) P ^ ( y 2 ℓ ∣ x n , y 1 ) , (115) \widehat{p}(y^\ell\mid x^n)=\widehat{\pi}(y_1\mid x^n)\widehat{P}(y_2^\ell\mid x^n,y_1), \tag*{(115)} p ( y ℓ ∣ x n ) = π ( y 1 ∣ x n ) P ( y 2 ℓ ∣ x n , y 1 ) , (115)
the chain rule for KL divergence applied to the batch regret in Eq. (30) gives the decomposition
R n , ℓ ( p , θ ^ ) = R n , ℓ ( π ) ( π ^ , θ ) + R n , ℓ ( P ) ( P ^ , θ ) , (116) R_{n,\ell}(p,\widehat{\theta})=R_{n,\ell}^{(\pi)}(\widehat{\pi},\theta)+R_{n,\ell}^{(P)}(\widehat{P},\theta), \tag*{(116)} R n , ℓ ( p , θ ) = R n , ℓ ( π ) ( π , θ ) + R n , ℓ ( P ) ( P , θ ) , (116)
where
R n , ℓ ( π ) ( π ^ , θ ) ≜ E θ log π ν ( Y 1 ) π ^ ( Y 1 ∣ X n ) , (117) R_{n,\ell}^{(\pi)}(\widehat{\pi},\theta)\triangleq\mathbb{E}_{\theta}\log\frac{\pi_{\nu}(Y_1)}{\widehat{\pi}(Y_1\mid X^n)}, \tag*{(117)} R n , ℓ ( π ) ( π , θ ) ≜ E θ log π ( Y 1 ∣ X n ) π ν ( Y 1 ) , (117)
R n , ℓ ( P ) ( P ^ , θ ) ≜ E θ log P p , q ( Y 2 ℓ ∣ Y 1 ) P ^ ( Y 2 ℓ ∣ X n , Y 1 ) , (118) R_{n,\ell}^{(P)}(\widehat{P},\theta)\triangleq\mathbb{E}_{\theta}\log\frac{P_{p,q}(Y_2^\ell\mid Y_1)}{\widehat{P}(Y_2^\ell\mid X^n,Y_1)}, \tag*{(118)} R n , ℓ ( P ) ( P , θ ) ≜ E θ log P ( Y 2 ℓ ∣ X n , Y 1 ) P p , q ( Y 2 ℓ ∣ Y 1 ) , (118)
which can be identified as the regrets of the initial distribution and of the transition kernel respectively. Regarding the former, the following result holds.
Theorem 13 (Initial-state contribution): As n , ℓ → ∞ n,\ell\to\infty n , ℓ → ∞ jointly, the following statements on R n , ℓ ( π ) R_{n,\ell}^{(\pi)} R n , ℓ ( π ) hold.
Under the class M δ , ν 0 k n o w n \mathcal{M}_{\delta,\nu_0}^{\mathrm{known}} M δ , ν 0 known , i.e., if the initial law is known, then
inf π ^ sup θ ∈ M δ , ν 0 k n o w n R n , ℓ ( π ) ( π ^ , θ ) = 0. (119) \inf_{\widehat{\pi}}\sup_{\theta\in\mathcal{M}_{\delta,\nu_0}^{\mathrm{known}}}R_{n,\ell}^{(\pi)}(\widehat{\pi},\theta)=0. \tag*{(119)} π inf θ ∈ M δ , ν 0 known sup R n , ℓ ( π ) ( π , θ ) = 0. (119) Under the class M δ s t a t \mathcal{M}_{\delta}^{\mathrm{stat}} M δ stat , i.e., if the initial law is stationary, then there is a predictor π ^ \widehat{\pi} π such that
sup θ ∈ M δ s t a t R n , ℓ ( π ) ( π ^ , θ ) = O δ ( 1 n ℓ ) = o ( 1 n ) . (120) \sup_{\theta\in\mathcal{M}_{\delta}^{\mathrm{stat}}}R_{n,\ell}^{(\pi)}(\widehat{\pi},\theta)=O_{\delta}\left(\frac{1}{n\ell}\right)=o\left(\frac{1}{n}\right). \tag*{(120)} θ ∈ M δ stat sup R n , ℓ ( π ) ( π , θ ) = O δ ( n ℓ 1 ) = o ( n 1 ) . (120) Under the class M δ f r e e \mathcal{M}_{\delta}^{\mathrm{free}} M δ free , i.e., if the initial law is a free parameter independent of the transition kernel, then
inf π ^ sup θ ∈ M δ f r e e R n , ℓ ( π ) ( π ^ , θ ) = 1 2 n + o ( 1 n ) . (121) \inf_{\widehat{\pi}}\sup_{\theta\in\mathcal{M}_{\delta}^{\mathrm{free}}}R_{n,\ell}^{(\pi)}(\widehat{\pi},\theta)=\frac{1}{2n}+o\left(\frac{1}{n}\right). \tag*{(121)} π inf θ ∈ M δ free sup R n , ℓ ( π ) ( π , θ ) = 2 n 1 + o ( n 1 ) . (121)
Proof. The first statement follows directly by taking π ^ = ν \widehat{\pi}=\nu π = ν . For the second statement, write
π ≜ P θ ( X i , s = 1 ) = p p + q ∈ [ δ , 1 − δ ] (122) \pi\triangleq\mathbb{P}_{\theta}(X_{i,s}=1)=\frac{p}{p+q}\in[\delta,1-\delta] \tag*{(122)} π ≜ P θ ( X i , s = 1 ) = p + q p ∈ [ δ , 1 − δ ] (122)
and define
X ‾ n , ℓ ≜ 1 n ℓ ∑ i = 1 n ∑ s = 1 ℓ X i , s , π ^ ≜ Π [ δ , 1 − δ ] ( X ‾ n , ℓ ) , (123) \overline{X}_{n,\ell}\triangleq\frac{1}{n\ell}\sum_{i=1}^{n}\sum_{s=1}^{\ell}X_{i,s},\qquad\widehat{\pi}\triangleq\Pi_{[\delta,1-\delta]}(\overline{X}_{n,\ell}), \tag*{(123)} X n , ℓ ≜ n ℓ 1 i = 1 ∑ n s = 1 ∑ ℓ X i , s , π ≜ Π [ δ , 1 − δ ] ( X n , ℓ ) , (123)
where Π I \Pi_I Π I denotes Euclidean projection onto I I I . Predict Y 1 = 1 Y_1=1 Y 1 = 1 with probability π ^ \widehat{\pi} π . Let ρ = 1 − p − q \rho=1-p-q ρ = 1 − p − q . The centered state function x ↦ x − π x\mapsto x-\pi x ↦ x − π is an eigenfunction of P p , q P_{p,q} P p , q with eigenvalue ρ \rho ρ . For a stationary Markov chain, correlations of an eigenfunction with eigenvalue ρ \rho ρ decay as ρ k \rho^k ρ k ; see, e.g., [42 Lemma 12.22] . Hence,
Cov θ ( X i , s , X i , s + k ) = π ( 1 − π ) ρ k . (124) \operatorname{Cov}_{\theta}(X_{i,s},X_{i,s+k})=\pi(1-\pi)\rho^k. \tag*{(124)} Cov θ ( X i , s , X i , s + k ) = π ( 1 − π ) ρ k . (124)
Since ∣ ρ ∣ ≤ 1 − 2 δ |\rho|\le1-2\delta ∣ ρ ∣ ≤ 1 − 2 δ ,
Var θ ( ∑ s = 1 ℓ X i , s ) = π ( 1 − π ) { ℓ + 2 ∑ k = 1 ℓ − 1 ( ℓ − k ) ρ k } ≤ ℓ 4 { 1 + 2 ∑ k = 1 ∞ ( 1 − 2 δ ) k } ≤ C δ ℓ . (125) \operatorname{Var}_{\theta}\left(\sum_{s=1}^{\ell}X_{i,s}\right)=\pi(1-\pi)\left\{\ell+2\sum_{k=1}^{\ell-1}(\ell-k)\rho^k\right\}
\le\frac{\ell}{4}\left\{1+2\sum_{k=1}^{\infty}(1-2\delta)^k\right\}\le C_{\delta}\ell. \tag*{(125)} Var θ ( s = 1 ∑ ℓ X i , s ) = π ( 1 − π ) { ℓ + 2 k = 1 ∑ ℓ − 1 ( ℓ − k ) ρ k } ≤ 4 ℓ { 1 + 2 k = 1 ∑ ∞ ( 1 − 2 δ ) k } ≤ C δ ℓ . (125)
Since projection onto [ δ , 1 − δ ] [\delta,1-\delta] [ δ , 1 − δ ] does not increase squared error, and since the n n n training sequences are independent, we have
E θ ( π ^ − π ) 2 ≤ E θ ( X ‾ n , ℓ − π ) 2 ≤ C δ n ℓ . (126) \mathbb{E}_{\theta}(\widehat{\pi}-\pi)^2\le\mathbb{E}_{\theta}(\overline{X}_{n,\ell}-\pi)^2\le\frac{C_{\delta}}{n\ell}. \tag*{(126)} E θ ( π − π ) 2 ≤ E θ ( X n , ℓ − π ) 2 ≤ n ℓ C δ . (126)
For a , b ∈ [ δ , 1 − δ ] a,b\in[\delta,1-\delta] a , b ∈ [ δ , 1 − δ ] , the inequality D ( Bern ( a ) ∥ Bern ( b ) ) ≤ ( a − b ) 2 / [ b ( 1 − b ) ] D(\operatorname{Bern}(a)\mathbin\Vert\operatorname{Bern}(b))\le(a-b)^2/[b(1-b)] D ( Bern ( a ) ∥ Bern ( b )) ≤ ( a − b ) 2 / [ b ( 1 − b )] gives Eq. (120) .
For the third statement, notice that the n n n initial training states are i.i.d. Bernoulli(ν \nu ν ). Applying the one-step rule of Braess and Sauer [21 Eq. (5.18)] on the sequence of n n n initial states gives the upper bound 1 / ( 2 n ) + o ( 1 / n ) 1/(2n)+o(1/n) 1/ ( 2 n ) + o ( 1/ n ) uniformly over ν ∈ [ 0 , 1 ] \nu\in[0,1] ν ∈ [ 0 , 1 ] [21 Sec. 5] . For the converse, restrict the transition kernel to p = q = 1 / 2 p=q=1/2 p = q = 1/2 . Under this submodel, all coordinates after the first one are independent fair bits and carry no information about ν \nu ν . Write the training data as ( S , Z ) (S,Z) ( S , Z ) , where S = ( X 1 , 1 , … , X n , 1 ) S=(X_{1,1},\ldots,X_{n,1}) S = ( X 1 , 1 , … , X n , 1 ) , and for any full-data predictor define p ‾ ( S ) = E [ π ^ ( 1 ∣ S , Z ) ∣ S ] \overline{p}(S)=\mathbb{E}[\widehat{\pi}(1\mid S,Z)\mid S] p ( S ) = E [ π ( 1 ∣ S , Z ) ∣ S ] . Since u ↦ D ( Bern ( ν ) ∥ Bern ( u ) ) u\mapsto D(\operatorname{Bern}(\nu)\mathbin\Vert\operatorname{Bern}(u)) u ↦ D ( Bern ( ν ) ∥ Bern ( u )) is convex,
E D ( Bern ( ν ) ∥ Bern ( π ^ ( 1 ∣ S , Z ) ) ) ≥ E D ( Bern ( ν ) ∥ Bern ( p ‾ ( S ) ) ) . (127) \mathbb{E}D(\operatorname{Bern}(\nu)\mathbin\Vert\operatorname{Bern}(\widehat{\pi}(1\mid S,Z)))
\ge\mathbb{E}D(\operatorname{Bern}(\nu)\mathbin\Vert\operatorname{Bern}(\overline{p}(S))). \tag*{(127)} E D ( Bern ( ν ) ∥ Bern ( π ( 1 ∣ S , Z ))) ≥ E D ( Bern ( ν ) ∥ Bern ( p ( S ))) . (127)
Thus the additional coordinates cannot improve the minimax initial-state prediction problem, and Krichevsky’s arbitrary-predictor lower bound [20 Theorem 2] gives the matching lower bound. ■
We next analyze the regret of the transition kernel. Let n h k ( x i ) n_{hk}(x_i) n hk ( x i ) be the number of transitions h → k h\to k h → k in the sequence x i x_i x i , and let n h ( x i ) ≜ ∑ k n h k ( x i ) n_h(x_i)\triangleq\sum_k n_{hk}(x_i) n h ( x i ) ≜ ∑ k n hk ( x i ) be the number of departures from h h h . Also define the cumulative counts
n h k ( x n ) ≜ ∑ i = 1 n n h k ( x i ) , n h ( x n ) ≜ ∑ i = 1 n n h ( x i ) . (128) n_{hk}(x^n)\triangleq\sum_{i=1}^{n}n_{hk}(x_i),\qquad n_h(x^n)\triangleq\sum_{i=1}^{n}n_h(x_i). \tag*{(128)} n hk ( x n ) ≜ i = 1 ∑ n n hk ( x i ) , n h ( x n ) ≜ i = 1 ∑ n n h ( x i ) . (128)
Consider the non-updating Krichevsky–Trofimov predictor
p ^ K T ( y 2 ℓ ∣ x n , y 1 ) = ∏ h , k ∈ { 0 , 1 } ( n h k ( x n ) + 1 2 n h ( x n ) + 1 ) n h k ( y ) . (129) \widehat{p}_{\mathrm{KT}}(y_2^\ell\mid x^n,y_1)=\prod_{h,k\in\{0,1\}}\left(\frac{n_{hk}(x^n)+\frac{1}{2}}{n_h(x^n)+1}\right)^{n_{hk}(y)}. \tag*{(129)} p KT ( y 2 ℓ ∣ x n , y 1 ) = h , k ∈ { 0 , 1 } ∏ ( n h ( x n ) + 1 n hk ( x n ) + 2 1 ) n hk ( y ) . (129)
where n h k ( y ) n_{hk}(y) n hk ( y ) is the number of transitions h → k h\to k h → k in the test sequence. Then, the following result holds.
Theorem 14 (Transition upper bound): Uniformly over the initial distribution and over ( p , q ) ∈ Θ δ (p,q)\in\Theta_{\delta} ( p , q ) ∈ Θ δ ,
sup θ ∈ M δ ∣ R n , ℓ ( P ) ( p ^ K T , θ ) − 1 n ∣ = O δ ( 1 n n ℓ ) , n , ℓ → ∞ . (130) \sup_{\theta\in\mathcal{M}_{\delta}}\left|R_{n,\ell}^{(P)}(\widehat{p}_{\mathrm{KT}},\theta)-\frac{1}{n}\right|=O_{\delta}\left(\frac{1}{n\sqrt{n\ell}}\right),\qquad n,\ell\to\infty. \tag*{(130)} θ ∈ M δ sup R n , ℓ ( P ) ( p KT , θ ) − n 1 = O δ ( n n ℓ 1 ) , n , ℓ → ∞. (130)
Consequently,
inf P ^ sup θ ∈ M δ R n , ℓ ( P ) ( P ^ , θ ) ≤ 1 n + o ( 1 n ) . (131) \inf_{\widehat{P}}\sup_{\theta\in\mathcal{M}_{\delta}}R_{n,\ell}^{(P)}(\widehat{P},\theta)\le\frac{1}{n}+o\left(\frac{1}{n}\right). \tag*{(131)} P inf θ ∈ M δ sup R n , ℓ ( P ) ( P , θ ) ≤ n 1 + o ( n 1 ) . (131)
We defer the proof to Appendix H. The proof adapts [18 Theorem 2] to the batch case. More specifically, we apply Paulin’s bounded-differences inequality for Markov chains [19 Corollary 2.11] separately to the token counts of each independent training sequence and then we combine the resulting bounds to the full training batch counts via standard sub-Gaussianity properties.
Matching lower bounds can be obtained for the three parametrization classes, once again via the conditional regret-capacity theorem. For each of the three initial distribution parametrizations above, the following result is instrumental in bounding the maximal mutual information in Theorem 2 .
Lemma 15: Under either of the three classes M δ , ν 0 k n o w n \mathcal{M}_{\delta,\nu_0}^{\mathrm{known}} M δ , ν 0 known , M δ s t a t \mathcal{M}_{\delta}^{\mathrm{stat}} M δ stat or M δ f r e e \mathcal{M}_{\delta}^{\mathrm{free}} M δ free , there is a sequence of priors W n , ℓ W_{n,\ell} W n , ℓ such that
I W n , ℓ ( ( P , Q ) ; Y 2 ℓ ∣ X n , Y 1 ) = log ( 1 + 1 n ) + o ( 1 n ) . (132) I_{W_{n,\ell}}\left((P,Q);Y_2^\ell\mid X^n,Y_1\right)=\log\left(1+\frac{1}{n}\right)+o\left(\frac{1}{n}\right). \tag*{(132)} I W n , ℓ ( ( P , Q ) ; Y 2 ℓ ∣ X n , Y 1 ) = log ( 1 + n 1 ) + o ( n 1 ) . (132)
We defer the proof to Appendix I. For each of the three classes, define the minimax batch regret as
R n , ℓ ( M ) ≜ inf p ^ sup θ ∈ M R n , ℓ ( p ^ , θ ) . (133) R_{n,\ell}(\mathcal{M}) \triangleq\inf_{\hat{p}}\sup_{\theta\in\mathcal{M}}R_{n,\ell}(\hat{p},\theta). \tag*{(133)} R n , ℓ ( M ) ≜ p ^ inf θ ∈ M sup R n , ℓ ( p ^ , θ ) . (133)
Then, the following tight asymptotic characterizations hold.
Theorem 16 (Minimax binary Markov batch regret): As n , ℓ → ∞ n,\ell\to\infty n , ℓ → ∞ jointly, we have
R n , ℓ ( M δ , ν 0 k n o w n ) = 1 n + o ( 1 n ) , (134) R_{n,\ell}(\mathcal{M}_{\delta,\nu_0}^{\mathrm{known}})=\frac{1}{n}+o\left(\frac{1}{n}\right), \tag*{(134)} R n , ℓ ( M δ , ν 0 known ) = n 1 + o ( n 1 ) , (134) R n , ℓ ( M δ s t a t ) = 1 n + o ( 1 n ) , (135) R_{n,\ell}(\mathcal{M}_{\delta}^{\mathrm{stat}})=\frac{1}{n}+o\left(\frac{1}{n}\right), \tag*{(135)} R n , ℓ ( M δ stat ) = n 1 + o ( n 1 ) , (135) R n , ℓ ( M δ f r e e ) = 3 2 n + o ( 1 n ) . (136) R_{n,\ell}(\mathcal{M}_{\delta}^{\mathrm{free}})=\frac{3}{2n}+o\left(\frac{1}{n}\right). \tag*{(136)} R n , ℓ ( M δ free ) = 2 n 3 + o ( n 1 ) . (136)
Proof: The upper bounds follow from the decomposition in Eq. (116) , Theorems 13 and 14 . For the lower bounds, we apply Theorem 2 . In the known model, Lemma 15 gives
R n , ℓ ( M δ , ν 0 k n o w n ) ≥ log ( 1 + 1 n ) + o ( 1 n ) = 1 n + o ( 1 n ) . (137) R_{n,\ell}(\mathcal{M}_{\delta,\nu_0}^{\mathrm{known}})\ge\log\left(1+\frac{1}{n}\right)+o\left(\frac{1}{n}\right)=\frac{1}{n}+o\left(\frac{1}{n}\right). \tag*{(137)} R n , ℓ ( M δ , ν 0 known ) ≥ log ( 1 + n 1 ) + o ( n 1 ) = n 1 + o ( n 1 ) . (137)
For the stationary model, the mutual-information chain rule only adds the non-negative term I ( ( P , Q ) ; Y 1 ∣ X n ) I((P,Q);Y_1\mid X^n) I (( P , Q ) ; Y 1 ∣ X n ) , while Lemma 15 gives the same transition lower bound. For the free model, let V V V denote the random initial probability and use a product prior on ( V , P , Q ) (V,P,Q) ( V , P , Q ) . The likelihood and posterior factorize between the free initial parameter and the transition parameters, and the mutual-information chain rule gives
I ( ( V , P , Q ) ; Y ℓ ∣ X n ) = I ( V ; Y 1 ∣ X 1 , 1 , … , X n , 1 ) + I ( ( P , Q ) ; Y 2 ℓ ∣ X n , Y 1 ) . (138) I((V,P,Q);Y^\ell\mid X^n)=I(V;Y_1\mid X_{1,1},\ldots,X_{n,1})+I((P,Q);Y_2^\ell\mid X^n,Y_1). \tag*{(138)} I (( V , P , Q ) ; Y ℓ ∣ X n ) = I ( V ; Y 1 ∣ X 1 , 1 , … , X n , 1 ) + I (( P , Q ) ; Y 2 ℓ ∣ X n , Y 1 ) . (138)
Choose for V V V any fixed beta prior with parameters larger than one. Lemma 29 makes the first term 1 / ( 2 n ) + O ( 1 / n 2 ) 1/(2n)+O(1/n^2) 1/ ( 2 n ) + O ( 1/ n 2 ) , while Lemma 15 makes the second term 1 / n + o ( 1 / n ) 1/n+o(1/n) 1/ n + o ( 1/ n ) .
The leading regret coefficients have a simple dimensional interpretation. The transition kernel has two free scalar parameters, each contributing 1 / ( 2 n ) 1/(2n) 1/ ( 2 n ) to the total regret. A known initial law contributes no additional regret, and therefore the total regret for the known-initial-law model is 2 / ( 2 n ) = 1 / n 2/(2n)=1/n 2/ ( 2 n ) = 1/ n . A stationary initial law introduces no additional free parameter and can be estimated from all n ℓ n\ell n ℓ training states, so its regret contribution is negligible compared to the transition regret, which is then equal to 1 / n 1/n 1/ n again. A free initial law introduces one additional parameter observed through only the n n n independently restarted initial states, and therefore contributes another 1 / ( 2 n ) 1/(2n) 1/ ( 2 n ) to the regret, that sums up to a total of 3 / ( 2 n ) 3/(2n) 3/ ( 2 n ) .
Remark. The achievability result in Theorem 14 is obtained via the non-updating predictor in Eq. (129) . A sequential version of the transition predictor may also update the counts using test transitions revealed so far, just like the predictor in Eq. (46) does for the Bernoulli case. The updating version may improve lower-order terms, but Theorem 16 shows that no predictor can improve the leading regret coefficients. Thus the simpler non-updating predictor is already first-order minimax.
Batch α \alpha α -regret
We finally extend the fixed-distribution batch criterion from KL divergence to conditional Rényi divergence. For the sake of simplicity, in this section we restrict ourselves to the case in which all distributions in Θ \Theta Θ have full support, i.e., p θ ( x ) > 0 p_\theta(x)>0 p θ ( x ) > 0 for all θ \theta θ and x x x . We also restrict attention to α > 1 \alpha>1 α > 1 , as the case 0 < α < 1 0<\alpha<1 0 < α < 1 would require separate minimax arguments. Under these assumptions define, for α > 1 \alpha>1 α > 1 ,
R n , ℓ ( α ) ( p ^ , θ ) ≜ D α ( P θ Y L ∣ X n ∥ p ^ ∥ P θ X n ) (139) R_{n,\ell}^{(\alpha)}(\hat{p},\theta)\triangleq D_\alpha\left(P_\theta^{Y^L\mid X^n}\middle\|\hat{p}\middle\|P_\theta^{X^n}\right) \tag*{(139)} R n , ℓ ( α ) ( p ^ , θ ) ≜ D α ( P θ Y L ∣ X n p ^ P θ X n ) (139)
= 1 α − 1 log { ∑ x n p θ ( x n ) ∑ y p θ ( y ) ( p θ ( y ) p ^ ( y ∣ x n ) ) α − 1 } . (140) =\frac{1}{\alpha-1}\log\left\{\sum_{x^n}p_\theta(x^n)\sum_y p_\theta(y)\left(\frac{p_\theta(y)}{\hat{p}(y\mid x^n)}\right)^{\alpha-1}\right\}. \tag*{(140)} = α − 1 1 log { x n ∑ p θ ( x n ) y ∑ p θ ( y ) ( p ^ ( y ∣ x n ) p θ ( y ) ) α − 1 } . (140)
As α → 1 \alpha\to1 α → 1 , this expression converges to the average batch regret in Eq. (30) . As α → ∞ \alpha\to\infty α → ∞ , it converges to the conditional worst-case batch regret:
R n , ℓ ( ∞ ) ( p ^ , θ ) ≜ lim α → ∞ R n , ℓ ( α ) ( p ^ , θ ) = max x n max y log p θ ( y ) p ^ ( y ∣ x n ) . (141) R_{n,\ell}^{(\infty)}(\hat{p},\theta)\triangleq\lim_{\alpha\to\infty}R_{n,\ell}^{(\alpha)}(\hat{p},\theta)=\max_{x^n}\max_y\log\frac{p_\theta(y)}{\hat{p}(y\mid x^n)}. \tag*{(141)} R n , ℓ ( ∞ ) ( p ^ , θ ) ≜ α → ∞ lim R n , ℓ ( α ) ( p ^ , θ ) = x n max y max log p ^ ( y ∣ x n ) p θ ( y ) . (141)
This last quantity is generally not equal to the ordinary non-batch worst-case regret, because the predictor may depend on the training batch. Nonetheless, their minimax values match, since
inf p ^ sup θ R n , ℓ ( ∞ ) ( p ^ , θ ) = max x n inf p ^ ( ⋅ ∣ x n ) sup θ max y log p θ ( y ) p ^ ( y ∣ x n ) = max x n inf p ^ ( ⋅ ∣ x n ) sup θ max y log p θ ( y ) p ^ ( y ∣ x n ) = inf p ^ sup θ R max , ℓ ( p ^ , θ ) . \begin{aligned}
\inf_{\hat{p}}\sup_\theta R_{n,\ell}^{(\infty)}(\hat{p},\theta)=\max_{x^n}\inf_{\hat{p}(\cdot\mid x^n)}\sup_\theta\max_y\log\frac{p_\theta(y)}{\hat{p}(y\mid x^n)} \\
&=\max_{x^n}\inf_{\hat{p}(\cdot\mid x^n)}\sup_\theta\max_y\log\frac{p_\theta(y)}{\hat{p}(y\mid x^n)} \\
&=\inf_{\hat{p}}\sup_\theta R_{\max,\ell}(\hat{p},\theta).
\end{aligned} p ^ inf θ sup R n , ℓ ( ∞ ) ( p ^ , θ ) = x n max p ^ ( ⋅ ∣ x n ) inf θ sup y max log p ^ ( y ∣ x n ) p θ ( y ) = x n max p ^ ( ⋅ ∣ x n ) inf θ sup y max log p ^ ( y ∣ x n ) p θ ( y ) = p ^ inf θ sup R m a x , ℓ ( p ^ , θ ) .
A regret-capacity theorem can also be proved for α \alpha α -regret, where the role of conditional mutual information is taken over by a conditional version of Sibson’s mutual information [30 , 31 ] .
Definition 17. Let α > 1 \alpha>1 α > 1 . Let θ , X , Y , Y ^ \theta,X,Y,\hat{Y} θ , X , Y , Y ^ be random variables such that ( θ , Y ) − X − Y ^ (\theta,Y)-X-\hat{Y} ( θ , Y ) − X − Y ^ , and suppose that Y Y Y and Y ^ \hat{Y} Y ^ take values in the same discrete alphabet. If p ^ \hat{p} p ^ is the conditional distribution of Y ^ \hat{Y} Y ^ given X X X , define
I α ( θ ; Y ∣ X ) ≜ inf p ^ D α ( Y ∥ Y ^ ∣ θ , X ) . (142) I_\alpha(\theta;Y\mid X)\triangleq\inf_{\hat{p}}D_\alpha(Y\|\hat{Y}\mid\theta,X). \tag*{(142)} I α ( θ ; Y ∣ X ) ≜ p ^ inf D α ( Y ∥ Y ^ ∣ θ , X ) . (142)
For the batch model considered in this paper, this quantity has the following closed form.
Theorem 18. Let X \mathcal{X} X and Y \mathcal{Y} Y be discrete alphabets. Let also ( θ , X n , Y ) ∼ W ( θ ) p θ ( X n ) p θ ( Y ) (\theta, X^n, Y) \sim W(\theta)p_\theta(X^n)p_\theta(Y) ( θ , X n , Y ) ∼ W ( θ ) p θ ( X n ) p θ ( Y ) , where X n ∈ X n X^n \in\mathcal{X}^n X n ∈ X n and Y ∈ Y Y \in\mathcal{Y} Y ∈ Y . Then, under the assumptions of this section,
I α ( θ ; Y ∣ X n ) = 1 α − 1 log ∑ x n { ∑ y ( ∫ Θ p θ ( x n ) p θ ( y ) α d W ( θ ) ) 1 / α } α . (143) I_\alpha(\theta;Y \mid X^n) = \frac{1}{\alpha-1}\log\sum_{x^n}\left\{\sum_y\left(\int_\Theta p_\theta(x^n)p_\theta(y)^\alpha dW(\theta)\right)^{1/\alpha}\right\}^\alpha. \tag*{(143)} I α ( θ ; Y ∣ X n ) = α − 1 1 log x n ∑ { y ∑ ( ∫ Θ p θ ( x n ) p θ ( y ) α d W ( θ ) ) 1/ α } α . (143) The minimizing conditional distribution in Eq. (142) is
p ^ α ( y ∣ x n ) = { ∫ Θ p θ ( y ) α d W ( θ ∣ x n ) } 1 / α ∑ y ′ { ∫ Θ p θ ( y ′ ) α d W ( θ ∣ x n ) } 1 / α , (144) \hat{p}_\alpha(y \mid x^n) = \frac{\left\{\int_\Theta p_\theta(y)^\alpha dW(\theta\mid x^n)\right\}^{1/\alpha}}{\sum_{y'}\left\{\int_\Theta p_\theta(y')^\alpha dW(\theta\mid x^n)\right\}^{1/\alpha}}, \tag*{(144)} p ^ α ( y ∣ x n ) = ∑ y ′ { ∫ Θ p θ ( y ′ ) α d W ( θ ∣ x n ) } 1/ α { ∫ Θ p θ ( y ) α d W ( θ ∣ x n ) } 1/ α , (144) where
W ( θ ∣ x n ) ≜ W ( θ ) p θ ( x n ) ∫ Θ W ( ϑ ) p ϑ ( x n ) d ϑ . (145) W(\theta\mid x^n) \triangleq\frac{W(\theta)p_\theta(x^n)}{\int_\Theta W(\vartheta)p_\vartheta(x^n)d\vartheta}. \tag*{(145)} W ( θ ∣ x n ) ≜ ∫ Θ W ( ϑ ) p ϑ ( x n ) d ϑ W ( θ ) p θ ( x n ) . (145) Proof. For a fixed predictor q ( y ∣ x n ) q(y \mid x^n) q ( y ∣ x n ) , the exponential of ( α − 1 ) D α ( Y ∥ Y ^ ∣ θ , X n ) (\alpha-1)D_\alpha(Y\|\hat{Y}\mid\theta,X^n) ( α − 1 ) D α ( Y ∥ Y ^ ∣ θ , X n ) is
∑ x n ∑ y A x n , y q ( y ∣ x n ) 1 − α , (146) \sum_{x^n}\sum_y A_{x^n,y}q(y\mid x^n)^{1-\alpha}, \tag*{(146)} x n ∑ y ∑ A x n , y q ( y ∣ x n ) 1 − α , (146) where
A x n , y = ∫ Θ p θ ( x n ) p θ ( y ) α d W ( θ ) . (147) A_{x^n,y}=\int_\Theta p_\theta(x^n)p_\theta(y)^\alpha dW(\theta). \tag*{(147)} A x n , y = ∫ Θ p θ ( x n ) p θ ( y ) α d W ( θ ) . (147) The minimization over q ( ⋅ ∣ x n ) q(\cdot\mid x^n) q ( ⋅ ∣ x n ) separates for each value of x n x^n x n . For fixed x n x^n x n , we must minimize
∑ y A x n , y q ( y ∣ x n ) 1 − α (148) \sum_y A_{x^n,y}q(y\mid x^n)^{1-\alpha} \tag*{(148)} y ∑ A x n , y q ( y ∣ x n ) 1 − α (148) over all probability vectors q ( ⋅ ∣ x n ) q(\cdot\mid x^n) q ( ⋅ ∣ x n ) . The Lagrange equations give
q ( y ∣ x n ) ∝ A x n , y 1 / α , (149) q(y\mid x^n)\propto A_{x^n,y}^{1/\alpha}, \tag*{(149)} q ( y ∣ x n ) ∝ A x n , y 1/ α , (149) and the minimum value is ( ∑ y A x n , y 1 / α ) α (\sum_y A_{x^n,y}^{1/\alpha})^\alpha ( ∑ y A x n , y 1/ α ) α . Summing this minimum over x n x^n x n and taking the logarithm gives Eq. (143) . Rewriting A x n , y A_{x^n,y} A x n , y using the posterior W ( θ ∣ x n ) W(\theta\mid x^n) W ( θ ∣ x n ) gives Eq. (144) . ■
The closed-form formula in Eq. (143) is a special case of the formula derived in [30 ] when X n − θ − Y X^n-\theta-Y X n − θ − Y . The predictor in Eq. (144) is a conditional version of the α \alpha α -NML predictor introduced in [34 ] . With the help of Theorem 18, we can prove the following generalization of the conditional regret-capacity theorem to batch α \alpha α -regret.
Theorem 19. Let X \mathcal{X} X be a finite alphabet, let X n = ( X 1 , … , X n ) ∈ ( X ) n X^n=(X_1,\ldots,X_n)\in(\mathcal{X})^n X n = ( X 1 , … , X n ) ∈ ( X ) n and Y ∈ X ℓ Y\in\mathcal{X}^{\ell} Y ∈ X ℓ be n n n training sequences and one test sequence of tokens from X \mathcal{X} X of length ℓ \ell ℓ , and let P = { p θ : θ ∈ Θ } \mathcal{P}=\{p_\theta:\theta\in\Theta\} P = { p θ : θ ∈ Θ } be a parametric class such that ( θ , X n , Y ) ∼ W ( θ ) p θ ( X n ) p θ ( Y ) (\theta,X^n,Y)\sim W(\theta)p_\theta(X^n)p_\theta(Y) ( θ , X n , Y ) ∼ W ( θ ) p θ ( X n ) p θ ( Y ) . Denote by I α , W ( θ ; Y ∣ X n ) I_{\alpha,W}(\theta;Y\mid X^n) I α , W ( θ ; Y ∣ X n ) the conditional Sibson information induced by the prior W W W . Suppose that all quantities below are finite and that there exists a prior W ⋆ W^\star W ⋆ such that
I α , W ⋆ ( θ ; Y ∣ X n ) = sup W I α , W ( θ ; Y ∣ X n ) . (150) I_{\alpha,W^\star}(\theta;Y\mid X^n)=\sup_W I_{\alpha,W}(\theta;Y\mid X^n). \tag*{(150)} I α , W ⋆ ( θ ; Y ∣ X n ) = W sup I α , W ( θ ; Y ∣ X n ) . (150) Then, under the assumptions of this section,
inf p ^ sup θ ∈ Θ R n , ℓ ( α ) ( p ^ , θ ) = I α , W ⋆ ( θ ; Y ∣ X n ) , (151) \inf_{\hat{p}}\sup_{\theta\in\Theta}R_{n,\ell}^{(\alpha)}(\hat{p},\theta)=I_{\alpha,W^\star}(\theta;Y\mid X^n), \tag*{(151)} p ^ inf θ ∈ Θ sup R n , ℓ ( α ) ( p ^ , θ ) = I α , W ⋆ ( θ ; Y ∣ X n ) , (151) and the infimum is attained by the conditional α \alpha α -NML predictor
p ^ α ( y ∣ x n ) = { ∫ Θ p θ ( y ) α d W ⋆ ( θ ∣ x n ) } 1 / α ∑ y ′ { ∫ Θ p θ ( y ′ ) α d W ⋆ ( θ ∣ x n ) } 1 / α , (152) \hat{p}_\alpha(y\mid x^n)=\frac{\left\{\int_\Theta p_\theta(y)^\alpha dW^\star(\theta\mid x^n)\right\}^{1/\alpha}}{\sum_{y'}\left\{\int_\Theta p_\theta(y')^\alpha dW^\star(\theta\mid x^n)\right\}^{1/\alpha}}, \tag*{(152)} p ^ α ( y ∣ x n ) = ∑ y ′ { ∫ Θ p θ ( y ′ ) α d W ⋆ ( θ ∣ x n ) } 1/ α { ∫ Θ p θ ( y ) α d W ⋆ ( θ ∣ x n ) } 1/ α , (152) where
W ⋆ ( θ ∣ x n ) ≜ W ⋆ ( θ ) p θ ( x n ) ∫ Θ W ⋆ ( ϑ ) p ϑ ( x n ) d ϑ . (153) W^\star(\theta\mid x^n)\triangleq\frac{W^\star(\theta)p_\theta(x^n)}{\int_\Theta W^\star(\vartheta)p_\vartheta(x^n)d\vartheta}. \tag*{(153)} W ⋆ ( θ ∣ x n ) ≜ ∫ Θ W ⋆ ( ϑ ) p ϑ ( x n ) d ϑ W ⋆ ( θ ) p θ ( x n ) . (153) Proof. Let C α ≜ sup W I α , W ( θ ; Y ∣ X n ) C_\alpha\triangleq\sup_W I_{\alpha,W}(\theta;Y\mid X^n) C α ≜ sup W I α , W ( θ ; Y ∣ X n ) and define, for a predictor p ^ \hat{p} p ^ ,
A θ ( p ^ ) ≜ E X n , Y ∼ p θ [ ( p θ ( Y ) p ^ ( Y ∣ X n ) ) α − 1 ] . (154) A_\theta(\hat{p})\triangleq\mathbb{E}_{X^n,Y\sim p_\theta}\left[\left(\frac{p_\theta(Y)}{\hat{p}(Y\mid X^n)}\right)^{\alpha-1}\right]. \tag*{(154)} A θ ( p ^ ) ≜ E X n , Y ∼ p θ [ ( p ^ ( Y ∣ X n ) p θ ( Y ) ) α − 1 ] . (154) Since α > 1 \alpha>1 α > 1 ,
inf p ^ sup θ R n , ℓ ( α ) ( p ^ , θ ) = 1 α − 1 log inf p ^ sup θ A θ ( p ^ ) ≥ sup W inf p ^ 1 α − 1 log E W [ A θ ( p ^ ) ] = sup W I α , W ( θ ; Y ∣ X n ) = C α , (155) \inf_{\hat{p}}\sup_\theta R_{n,\ell}^{(\alpha)}(\hat{p},\theta)=\frac{1}{\alpha-1}\log\inf_{\hat{p}}\sup_\theta A_\theta(\hat{p})
\geq\sup_W\inf_{\hat{p}}\frac{1}{\alpha-1}\log\mathbb{E}_W[A_\theta(\hat{p})]
=\sup_W I_{\alpha,W}(\theta;Y\mid X^n)=C_\alpha, \tag*{(155)} p ^ inf θ sup R n , ℓ ( α ) ( p ^ , θ ) = α − 1 1 log p ^ inf θ sup A θ ( p ^ ) ≥ W sup p ^ inf α − 1 1 log E W [ A θ ( p ^ )] = W sup I α , W ( θ ; Y ∣ X n ) = C α , (155) where the last equality follows from Theorem 18.
It remains to prove the matching upper bound. Let p ^ α ⋆ \hat{p}_\alpha^\star p ^ α ⋆ be the predictor induced by W ⋆ W^\star W ⋆ in Eq. (152) . Suppose, toward a contradiction, that there exists θ ~ ∈ Θ \tilde{\theta}\in\Theta θ ~ ∈ Θ such that
R n , ℓ ( α ) ( p ^ α ⋆ , θ ~ ) > C α . (156) R_{n,\ell}^{(\alpha)}(\hat{p}_\alpha^\star,\tilde{\theta})>C_\alpha. \tag*{(156)} R n , ℓ ( α ) ( p ^ α ⋆ , θ ~ ) > C α . (156) For t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] , let W t = ( 1 − t ) W ⋆ + t δ θ ~ W_t=(1-t)W^\star+t\delta_{\tilde{\theta}} W t = ( 1 − t ) W ⋆ + t δ θ ~ and set f α ( t ) = I α , W t ( θ ; Y ∣ X n ) f_\alpha(t)=I_{\alpha,W_t}(\theta;Y\mid X^n) f α ( t ) = I α , W t ( θ ; Y ∣ X n ) . Since W ⋆ W^\star W ⋆ maximizes I α , W ( θ ; Y ∣ X n ) I_{\alpha,W}(\theta;Y\mid X^n) I α , W ( θ ; Y ∣ X n ) , the right derivative satisfies f α ′ ( 0 ) ≤ 0 f_\alpha'(0)\leq0 f α ′ ( 0 ) ≤ 0 .
Let
F α ( W ) = exp { ( α − 1 ) I α , W ( θ ; Y ∣ X n ) } . (157) F_\alpha(W)=\exp\{(\alpha-1)I_{\alpha,W}(\theta;Y\mid X^n)\}. \tag*{(157)} F α ( W ) = exp {( α − 1 ) I α , W ( θ ; Y ∣ X n )} . (157) Differentiating the expression in Eq. (143) along W t W_t W t gives
d d t F α ( W t ) ∣ t = 0 = A θ ~ ( p ^ α ⋆ ) − F α ( W ⋆ ) . (158) \left.\frac{d}{dt}F_\alpha(W_t)\right|_{t=0}=A_{\tilde{\theta}}(\hat{p}_\alpha^\star)-F_\alpha(W^\star). \tag*{(158)} d t d F α ( W t ) t = 0 = A θ ~ ( p ^ α ⋆ ) − F α ( W ⋆ ) . (158) Therefore
f α ′ ( 0 ) = 1 α − 1 ( exp { ( α − 1 ) R n , ℓ ( α ) ( p ^ α ⋆ , θ ~ ) } exp { ( α − 1 ) C α } − 1 ) > 0 , (159) f_\alpha'(0)=\frac{1}{\alpha-1}\left(\frac{\exp\{(\alpha-1)R_{n,\ell}^{(\alpha)}(\hat{p}_\alpha^\star,\tilde{\theta})\}}{\exp\{(\alpha-1)C_\alpha\}}-1\right)>0, \tag*{(159)} f α ′ ( 0 ) = α − 1 1 ( exp {( α − 1 ) C α } exp {( α − 1 ) R n , ℓ ( α ) ( p ^ α ⋆ , θ ~ )} − 1 ) > 0 , (159) contradicting maximality of W ⋆ W^\star W ⋆ . Hence R n , ℓ ( α ) ( p ^ α ⋆ , θ ) ≤ C α R_{n,\ell}^{(\alpha)}(\hat{p}_\alpha^\star,\theta)\leq C_\alpha R n , ℓ ( α ) ( p ^ α ⋆ , θ ) ≤ C α for every θ \theta θ , and the lower bound already proved is tight. ■
Conclusion
In this work we introduced batch regret as a conditional analogue of average regret under logarithmic loss, for the prediction of a fresh test sequence of length ℓ \ell ℓ after observing a training batch of n n n sequences. Under the assumption that all sequences are i.i.d. from the same unknown distribution, we proved a conditional generalization of the classical regret-capacity theorem, which is instrumental in deriving lower bounds for batch regret. Under the same metric, we studied conditional additive smoothing estimators as universal predictors for binary memoryless and first-order Markov sources. For binary memoryless sources, we extended Krichevsky’s next-token prediction analysis to derive sharp asymptotic estimates of batch regret in the two complementary regimes of large batch size n n n and of large sequence length ℓ \ell ℓ , identifying the optimal smoothing bias β \beta β for all those instances. For binary first-order Markov sources, analysis diverges between classical and batch universal prediction, as in the latter case a more prominent role is taken by the Markov source’s initial distribution. Therefore, we analyzed minimax batch regret under fixed, free, and stationary initial distribution assumptions, providing sharp asymptotic estimates in all the three cases. Finally, we introduced batch α \alpha α -regret as an extension of batch regret through conditional Rényi divergence. Under this setting, we proved a generalization of the regret-capacity theorem using a conditional version of Sibson’s α \alpha α -mutual information, and we identified conditional α \alpha α -NML estimators as the predictors achieving the minimal regret.
Proof of Theorem 2
The proof adapts to conditional kernels the standard finite-subfamily reduction used in [39 ] to prove a general unconditional relative-entropy minimax theorem. Following their proof, we first prove a result for finite-support priors, that holds for an arbitrary parameter set Θ \Theta Θ ; then we show that, under the stated assumptions on Θ \Theta Θ , the result extends to arbitrary priors.
Let W f ( Θ ) \mathcal{W}_{f}(\Theta) W f ( Θ ) denote the set of finitely supported probability distributions on Θ \Theta Θ . Let us prove that, for any arbitrary parameter set Θ \Theta Θ ,
min Q ∈ Q ( Y ∣ X ) sup θ ∈ Θ r θ ( Q ) = sup W ∈ W f ( Θ ) I W ( Θ ; Y ∣ X ) . (160) \min_{Q \in\mathcal{Q}(\mathcal{Y}\mid\mathcal{X})}\sup_{\theta\in\Theta} r_{\theta}(Q)
=
\sup_{W \in\mathcal{W}_{f}(\Theta)} I_{W}(\Theta;Y\mid X). \tag*{(160)} Q ∈ Q ( Y ∣ X ) min θ ∈ Θ sup r θ ( Q ) = W ∈ W f ( Θ ) sup I W ( Θ ; Y ∣ X ) . (160)
Write Q = Q ( Y ∣ X ) \mathcal{Q}=\mathcal{Q}(Y\mid X) Q = Q ( Y ∣ X ) . Since the alphabets are finite, Q \mathcal{Q} Q is a compact convex subset of a finite-dimensional Euclidean space. Moreover, the uniform kernel Q u ( y ∣ x ) = 1 / ∣ Y ∣ Q_{u}(y\mid x)=1/|\mathcal{Y}| Q u ( y ∣ x ) = 1/∣ Y ∣ satisfies
sup θ ∈ Θ r θ ( Q u ) ≤ log ∣ Y ∣ , (161) \sup_{\theta\in\Theta} r_{\theta}(Q_{u}) \le\log|\mathcal{Y}|, \tag*{(161)} θ ∈ Θ sup r θ ( Q u ) ≤ log ∣ Y ∣ , (161)
so all minimax values below are finite.
We first solve the problem on a finite parameter subset F = { θ 1 , … , θ m } ⊂ Θ F=\{\theta_{1},\ldots,\theta_{m}\}\subset\Theta F = { θ 1 , … , θ m } ⊂ Θ . Define
C F ≜ min Q ∈ Q max θ ∈ F r θ ( Q ) . (162) C_{F}\triangleq\min_{Q\in\mathcal{Q}}\max_{\theta\in F}r_{\theta}(Q). \tag*{(162)} C F ≜ Q ∈ Q min θ ∈ F max r θ ( Q ) . (162)
For fixed Q Q Q ,
max θ ∈ F r θ ( Q ) = max W ∈ Δ ( F ) ∑ θ ∈ F W ( θ ) r θ ( Q ) , (163) \max_{\theta\in F}r_{\theta}(Q)
=
\max_{W\in\Delta(F)}\sum_{\theta\in F}W(\theta)r_{\theta}(Q), \tag*{(163)} θ ∈ F max r θ ( Q ) = W ∈ Δ ( F ) max θ ∈ F ∑ W ( θ ) r θ ( Q ) , (163)
because a convex combination cannot exceed its largest entry and equality is obtained by a point mass at a maximizing parameter. To interchange the two optimizations, consider the finite-dimensional convex problem
minimize z over ( Q , z ) ∈ Q × R subject to r θ ( Q ) ≤ z , θ ∈ F . (164) \begin{aligned}
&\text{minimize}\quad z\\
&\text{over}\quad(Q,z)\in\mathcal{Q}\times\mathbb{R}\\
&\text{subject to}\quad r_{\theta}(Q)\le z,\quad\theta\in F. \tag*{(164)}
\end{aligned} minimize z over ( Q , z ) ∈ Q × R subject to r θ ( Q ) ≤ z , θ ∈ F . (164)
Each constraint is convex in Q Q Q . The pair ( Q u , log ∣ Y ∣ + 1 ) (Q_{u},\log|\mathcal{Y}|+1) ( Q u , log ∣ Y ∣ + 1 ) is strictly feasible, so Slater’s condition holds and strong Lagrange duality applies [43 Sec. 5.3] . The dual multipliers form the simplex Δ ( F ) \Delta(F) Δ ( F ) , and hence
C F = max W ∈ Δ ( F ) min Q ∈ Q ∑ θ ∈ F W ( θ ) r θ ( Q ) . (165) C_{F}=\max_{W\in\Delta(F)}\min_{Q\in\mathcal{Q}}\sum_{\theta\in F}W(\theta)r_{\theta}(Q). \tag*{(165)} C F = W ∈ Δ ( F ) max Q ∈ Q min θ ∈ F ∑ W ( θ ) r θ ( Q ) . (165)
For a fixed prior W W W on Θ \Theta Θ , let
P W X , Y ( x , y ) = ∑ θ ∈ F W ( θ ) P θ X , Y ( x , y ) . (166) P_{W}^{X,Y}(x,y)=\sum_{\theta\in F}W(\theta)P_{\theta}^{X,Y}(x,y). \tag*{(166)} P W X , Y ( x , y ) = θ ∈ F ∑ W ( θ ) P θ X , Y ( x , y ) . (166)
The conditional Bayes decomposition in Eq. (34) gives
∑ θ ∈ F W ( θ ) r θ ( Q ) = I W ( Θ ; Y ∣ X ) + D ( P W Y ∣ X ∥ Q | P W X ) , (167) \sum_{\theta\in F}W(\theta)r_{\theta}(Q)
=
I_{W}(\Theta;Y\mid X)
+
D\left(P_{W}^{Y\mid X}\middle\|Q\middle|P_{W}^{X}\right), \tag*{(167)} θ ∈ F ∑ W ( θ ) r θ ( Q ) = I W ( Θ ; Y ∣ X ) + D ( P W Y ∣ X Q P W X ) , (167)
where P W X P_{W}^{X} P W X and P W Y ∣ X P_{W}^{Y\mid X} P W Y ∣ X are induced by P W X , Y P_{W}^{X,Y} P W X , Y . The second term is non-negative and vanishes for Q = P W Y ∣ X Q=P_{W}^{Y\mid X} Q = P W Y ∣ X on the support of P W X P_{W}^{X} P W X . Therefore
C F = max W ∈ Δ ( F ) I W ( Θ ; Y ∣ X ) . (168) C_{F}=\max_{W\in\Delta(F)}I_{W}(\Theta;Y\mid X). \tag*{(168)} C F = W ∈ Δ ( F ) max I W ( Θ ; Y ∣ X ) . (168)
We now pass from finite F F F to the arbitrary index set Θ \Theta Θ . Set
C f ≜ sup F ⊂ Θ F f i n i t e C F . (169) C_{f}\triangleq\sup_{\substack{F\subset\Theta\\F\ \mathrm{finite}}}C_{F}. \tag*{(169)} C f ≜ F ⊂ Θ F finite sup C F . (169)
Clearly,
min Q sup θ ∈ Θ r θ ( Q ) ≥ C f . (170) \min_{Q}\sup_{\theta\in\Theta}r_{\theta}(Q)\ge C_{f}. \tag*{(170)} Q min θ ∈ Θ sup r θ ( Q ) ≥ C f . (170)
For each finite F F F , define the sublevel set
K F ≜ { Q ∈ Q : r θ ( Q ) ≤ C f for all θ ∈ F } . (171) K_{F}\triangleq\{Q\in\mathcal{Q}:r_{\theta}(Q)\le C_{f}\text{ for all }\theta\in F\}. \tag*{(171)} K F ≜ { Q ∈ Q : r θ ( Q ) ≤ C f for all θ ∈ F } . (171)
The function Q ↦ r θ ( Q ) Q\mapsto r_{\theta}(Q) Q ↦ r θ ( Q ) is lower semicontinuous, with the usual convention that a positive mass divided by zero gives infinite divergence. Hence K F K_{F} K F is closed. It is also nonempty: the lower-semicontinuous function Q ↦ max θ ∈ F r θ ( Q ) Q\mapsto\max_{\theta\in F}r_{\theta}(Q) Q ↦ max θ ∈ F r θ ( Q ) attains its minimum on compact Q \mathcal{Q} Q , and a minimizer Q F Q_{F} Q F satisfies
r θ ( Q F ) ≤ C F ≤ C f , θ ∈ F . (172) r_{\theta}(Q_{F})\le C_{F}\le C_{f},\qquad\theta\in F. \tag*{(172)} r θ ( Q F ) ≤ C F ≤ C f , θ ∈ F . (172)
Furthermore,
K F 1 ∩ ⋯ ∩ K F s = K F 1 ∪ ⋯ ∪ F s , (173) K_{F_{1}}\cap\cdots\cap K_{F_{s}}=K_{F_{1}\cup\cdots\cup F_{s}}, \tag*{(173)} K F 1 ∩ ⋯ ∩ K F s = K F 1 ∪ ⋯ ∪ F s , (173)
which is nonempty because the union is finite. Thus the closed sets { K F } \{K_{F}\} { K F } have the finite-intersection property. Compactness of Q \mathcal{Q} Q therefore yields
⋂ F ⊂ Θ , F f i n i t e K F ≠ ∅ . (174) \bigcap_{F\subset\Theta,\ F\ \mathrm{finite}}K_{F}\ne\varnothing. \tag*{(174)} F ⊂ Θ , F finite ⋂ K F = ∅ . (174)
Any Q ∗ Q^{*} Q ∗ in this intersection obeys r θ ( Q ∗ ) ≤ C f r_{\theta}(Q^{*})\le C_{f} r θ ( Q ∗ ) ≤ C f for every θ \theta θ , and therefore the reverse inequality to Eq. (170) holds. Combining this observation with Eq. (168) proves Eq. (160) and the existence of a minimax kernel.
Now suppose arbitrary measurable priors are allowed. For every prior W W W and every kernel Q Q Q , the decomposition in Eq. (34) gives
I W ( Θ ; Y ∣ X ) ≤ ∫ r θ ( Q ) d W ( θ ) ≤ sup θ r θ ( Q ) . (175) I_{W}(\Theta;Y\mid X)\le\int r_{\theta}(Q)\,\mathrm{d}W(\theta)\le\sup_{\theta}r_{\theta}(Q). \tag*{(175)} I W ( Θ ; Y ∣ X ) ≤ ∫ r θ ( Q ) d W ( θ ) ≤ θ sup r θ ( Q ) . (175)
Taking the infimum over Q Q Q shows that no arbitrary prior can exceed the common value in Eq. (160) ; the opposite inequality is immediate since finitely-supported priors belong to the set of arbitrary priors. Hence the supremum remains unchanged when arbitrary priors are considered.
Finally, let C C C denote the common value in Eq. (160) , assume that W ⋆ W^\star W ⋆ attains the supremum, and let Q ⋆ Q^\star Q ⋆ be minimax. Applying the Bayes decomposition once more,
C ≥ ∫ r θ ( Q ⋆ ) d W ⋆ ( θ ) = C + D ( P W ⋆ Y ∣ X ∥ Q ⋆ | P W ⋆ X ) ≥ C . (176) C \ge\int r_\theta(Q^\star)\,dW^\star(\theta)
= C + D\left(P^{Y|X}_{W^\star}\middle\|Q^\star\middle|P^X_{W^\star}\right) \ge C. \tag*{(176)} C ≥ ∫ r θ ( Q ⋆ ) d W ⋆ ( θ ) = C + D ( P W ⋆ Y ∣ X Q ⋆ P W ⋆ X ) ≥ C . (176)
Therefore, all inequalities are equalities. This proves Eq. (36) ; since r θ ( Q ⋆ ) ≤ C r_\theta(Q^\star) \le C r θ ( Q ⋆ ) ≤ C for every θ \theta θ and its W ⋆ W^\star W ⋆ -average is C C C , the equalizer property also holds W ⋆ W^\star W ⋆ -almost surely, concluding the first part of the proof.
We now prove the finite-support upper bound. Let E ( Q ⋆ ) = { θ : r θ ( Q ⋆ ) = C } \mathcal{E}(Q^\star)=\{\theta:r_\theta(Q^\star)=C\} E ( Q ⋆ ) = { θ : r θ ( Q ⋆ ) = C } , where C C C is the common value of r θ ( Q ⋆ ) r_\theta(Q^\star) r θ ( Q ⋆ ) . Fix one y 0 ∈ Y y_0\in\mathcal{Y} y 0 ∈ Y and define
v θ ( x , y ) = P θ X , Y ( x , y ) − P θ X ( x ) Q ⋆ ( y ∣ x ) , y ≠ y 0 . (177) v_\theta(x,y)=P_\theta^{X,Y}(x,y)-P_\theta^X(x)Q^\star(y\mid x),\qquad y\ne y_0. \tag*{(177)} v θ ( x , y ) = P θ X , Y ( x , y ) − P θ X ( x ) Q ⋆ ( y ∣ x ) , y = y 0 . (177)
By the last part of Theorem 2, W ⋆ W^\star W ⋆ is concentrated on E ( Q ⋆ ) \mathcal{E}(Q^\star) E ( Q ⋆ ) up to a null set, and Q ⋆ Q^\star Q ⋆ agrees with the W ⋆ W^\star W ⋆ -mixture conditional law. Consequently,
∫ v θ d W ⋆ ( θ ) = 0. (178) \int v_\theta\,dW^\star(\theta)=0. \tag*{(178)} ∫ v θ d W ⋆ ( θ ) = 0. (178)
Therefore, by Caratheodory’s theorem there exist parameters θ 1 , … , θ s ∈ E ( Q ⋆ ) \theta_1,\ldots,\theta_s\in\mathcal{E}(Q^\star) θ 1 , … , θ s ∈ E ( Q ⋆ ) and weights a 1 , … , a s a_1,\ldots,a_s a 1 , … , a s such that
s ≤ d ∗ + 1 , ∑ i = 1 s a i v θ i = 0 , (179) s\le d_\ast+1,\qquad\sum_{i=1}^{s}a_i v_{\theta_i}=0, \tag*{(179)} s ≤ d ∗ + 1 , i = 1 ∑ s a i v θ i = 0 , (179)
where d ∗ d_\ast d ∗ is the dimension of the affine hull of { v θ : θ ∈ E ( Q ⋆ ) } \{v_\theta:\theta\in\mathcal{E}(Q^\star)\} { v θ : θ ∈ E ( Q ⋆ )} . Let W ⋆ = ∑ i a i δ θ i W^\star=\sum_i a_i\delta_{\theta_i} W ⋆ = ∑ i a i δ θ i . The last identity gives
P W ⋆ ( x , y ) = P W ⋆ X ( x ) Q ⋆ ( y ∣ x ) (180) P^{}_{W^\star}(x,y)=P^X_{W^\star}(x)Q^\star(y\mid x) \tag*{(180)} P W ⋆ ( x , y ) = P W ⋆ X ( x ) Q ⋆ ( y ∣ x ) (180)
for every ( x , y ) (x,y) ( x , y ) . Hence the divergence term in Eq. (34) vanishes. Since every θ i \theta_i θ i belongs to E ( Q ⋆ ) \mathcal{E}(Q^\star) E ( Q ⋆ ) ,
∑ i a i r θ i ( Q ⋆ ) = C , (181) \sum_i a_i r_{\theta_i}(Q^\star)=C, \tag*{(181)} i ∑ a i r θ i ( Q ⋆ ) = C , (181)
and Eq. (34) gives I W ⋆ ( Θ ; Y ∣ X ) = C I_{W^\star}(\Theta;Y\mid X)=C I W ⋆ ( Θ ; Y ∣ X ) = C . Finally, the vector in Eq. (177) has at most ∣ X ∣ ( ∣ Y ∣ − 1 ) |\mathcal{X}|(|\mathcal{Y}|-1) ∣ X ∣ ( ∣ Y ∣ − 1 ) coordinates, which proves Eq. (37) .
Proof of Theorem 4
We use the sequential form of the conditional add-β \beta β estimator. Let ρ m ( β ) ( θ ) \rho_m^{(\beta)}(\theta) ρ m ( β ) ( θ ) be the one-step regret defined in Eq. (63) . Since the predictor updates after each test symbol, the chain rule for KL divergence gives
R n , ℓ ( p ^ β , θ ) = ∑ m = t t + ℓ − 1 ρ m ( β ) ( θ ) . (182) R_{n,\ell}(\hat p_\beta,\theta)=\sum_{m=t}^{t+\ell-1}\rho_m^{(\beta)}(\theta). \tag*{(182)} R n , ℓ ( p ^ β , θ ) = m = t ∑ t + ℓ − 1 ρ m ( β ) ( θ ) . (182)
Thus it is enough to prove a compact-interior one-step expansion.
Lemma 20. For every β > 0 \beta>0 β > 0 and every 0 < δ < 1 / 2 0<\delta<1/2 0 < δ < 1/2 ,
sup θ ∈ [ δ , 1 − δ ] ∣ ρ m ( β ) ( θ ) − 1 2 m ∣ = O β , δ ( 1 m 2 ) . (183) \sup_{\theta\in[\delta,1-\delta]}\left|\rho_m^{(\beta)}(\theta)-\frac{1}{2m}\right|=O_{\beta,\delta}\left(\frac{1}{m^2}\right). \tag*{(183)} θ ∈ [ δ , 1 − δ ] sup ρ m ( β ) ( θ ) − 2 m 1 = O β , δ ( m 2 1 ) . (183)
Proof. Let K ∼ Binomial ( m , θ ) K\sim\operatorname{Binomial}(m,\theta) K ∼ Binomial ( m , θ ) and set
Q = K + β m + 2 β . (184) Q=\frac{K+\beta}{m+2\beta}. \tag*{(184)} Q = m + 2 β K + β . (184)
For fixed θ \theta θ , define
d θ ( q ) = θ log θ q + ( 1 − θ ) log 1 − θ 1 − q , 0 < q < 1. (185) d_\theta(q)=\theta\log\frac{\theta}{q}+(1-\theta)\log\frac{1-\theta}{1-q},\qquad0<q<1. \tag*{(185)} d θ ( q ) = θ log q θ + ( 1 − θ ) log 1 − q 1 − θ , 0 < q < 1. (185)
Then ρ m ( β ) ( θ ) = E d θ ( Q ) \rho_m^{(\beta)}(\theta)=\mathbb{E}\,d_\theta(Q) ρ m ( β ) ( θ ) = E d θ ( Q ) . Let
V = Q − θ = K − m θ + β ( 1 − 2 θ ) m + 2 β . (186) V=Q-\theta=\frac{K-m\theta+\beta(1-2\theta)}{m+2\beta}. \tag*{(186)} V = Q − θ = m + 2 β K − m θ + β ( 1 − 2 θ ) . (186)
We first remove a negligible tail. Let
A m = { ∣ K − m θ ∣ ≤ m δ / 4 } . (187) A_m=\{|K-m\theta|\le m\delta/4\}. \tag*{(187)} A m = { ∣ K − m θ ∣ ≤ m δ /4 } . (187)
By Hoeffding’s inequality, P ( A m c ) ≤ 2 exp ( − c δ m ) \mathbb{P}(A_m^c)\le2\exp(-c_\delta m) P ( A m c ) ≤ 2 exp ( − c δ m ) uniformly over θ ∈ [ δ , 1 − δ ] \theta\in[\delta,1-\delta] θ ∈ [ δ , 1 − δ ] . On A m A_m A m , for all sufficiently large m m m , Q ∈ [ δ / 2 , 1 − δ / 2 ] Q\in[\delta/2,1-\delta/2] Q ∈ [ δ /2 , 1 − δ /2 ] . On A m c A_m^c A m c , using Q ∈ [ β / ( m + 2 β ) , ( m + β ) / ( m + 2 β ) ] Q\in[\beta/(m+2\beta),(m+\beta)/(m+2\beta)] Q ∈ [ β / ( m + 2 β ) , ( m + β ) / ( m + 2 β )] and θ ∈ [ δ , 1 − δ ] \theta\in[\delta,1-\delta] θ ∈ [ δ , 1 − δ ] , we have d θ ( Q ) = O β , δ ( log m ) d_\theta(Q)=O_{\beta,\delta}(\log m) d θ ( Q ) = O β , δ ( log m ) . Hence
E [ d θ ( Q ) 1 A m c ] = O β , δ ( m − 2 ) (188) \mathbb{E}[d_\theta(Q)\mathbb{1}_{A_m^c}]=O_{\beta,\delta}(m^{-2}) \tag*{(188)} E [ d θ ( Q ) 1 A m c ] = O β , δ ( m − 2 ) (188)
uniformly in θ \theta θ .
On A m A_m A m , the Taylor expansion around q = θ q=\theta q = θ gives
d θ ( θ + V ) = d θ ′ ′ ( θ ) 2 V 2 + d θ ( 3 ) ( θ ) 6 V 3 + R 4 , θ ( V ) , (189) d_\theta(\theta+V)=\frac{d_\theta''(\theta)}{2}V^2+\frac{d_\theta^{(3)}(\theta)}{6}V^3+R_{4,\theta}(V), \tag*{(189)} d θ ( θ + V ) = 2 d θ ′′ ( θ ) V 2 + 6 d θ ( 3 ) ( θ ) V 3 + R 4 , θ ( V ) , (189)
where d θ ( θ ) = d θ ′ ( θ ) = 0 d_\theta(\theta)=d_\theta'(\theta)=0 d θ ( θ ) = d θ ′ ( θ ) = 0 and, uniformly over θ ∈ [ δ , 1 − δ ] \theta\in[\delta,1-\delta] θ ∈ [ δ , 1 − δ ] and on A m A_m A m ,
∣ R 4 , θ ( V ) ∣ ≤ C δ ∣ V ∣ 4 . (190) |R_{4,\theta}(V)|\le C_\delta|V|^4. \tag*{(190)} ∣ R 4 , θ ( V ) ∣ ≤ C δ ∣ V ∣ 4 . (190)
Since
d θ ′ ′ ( θ ) = 1 θ ( 1 − θ ) , (191) d_\theta''(\theta)=\frac{1}{\theta(1-\theta)}, \tag*{(191)} d θ ′′ ( θ ) = θ ( 1 − θ ) 1 , (191)
and
E V 2 = m θ ( 1 − θ ) + β 2 ( 1 − 2 θ ) 2 ( m + 2 β ) 2 = θ ( 1 − θ ) m + O β ( 1 m 2 ) , (192) \mathbb{E}\,V^2=\frac{m\theta(1-\theta)+\beta^2(1-2\theta)^2}{(m+2\beta)^2}
=\frac{\theta(1-\theta)}{m}+O_\beta\left(\frac{1}{m^2}\right), \tag*{(192)} E V 2 = ( m + 2 β ) 2 m θ ( 1 − θ ) + β 2 ( 1 − 2 θ ) 2 = m θ ( 1 − θ ) + O β ( m 2 1 ) , (192)
we also obtain
d θ ′ ′ ( θ ) 2 E V 2 = 1 2 m + O β , δ ( 1 m 2 ) . (193) \frac{d_\theta''(\theta)}{2}\mathbb{E}\,V^2=\frac{1}{2m}+O_{\beta,\delta}\left(\frac{1}{m^2}\right). \tag*{(193)} 2 d θ ′′ ( θ ) E V 2 = 2 m 1 + O β , δ ( m 2 1 ) . (193)
Furthermore, the centered binomial moments satisfy
E ( K − m θ ) 3 = O ( m ) , E ( K − m θ ) 4 = O ( m 2 ) (194) \mathbb{E}(K-m\theta)^3=O(m),\qquad\mathbb{E}(K-m\theta)^4=O(m^2) \tag*{(194)} E ( K − m θ ) 3 = O ( m ) , E ( K − m θ ) 4 = O ( m 2 ) (194)
uniformly over θ ∈ [ δ , 1 − δ ] \theta\in[\delta,1-\delta] θ ∈ [ δ , 1 − δ ] . Hence
E V 3 = O β , δ ( 1 m 2 ) , E ∣ V ∣ 4 = E V 4 = O β , δ ( 1 m 2 ) . (195) \mathbb{E}\,V^3=O_{\beta,\delta}\left(\frac{1}{m^2}\right),\qquad
\mathbb{E}\,|V|^4=\mathbb{E}\,V^4=O_{\beta,\delta}\left(\frac{1}{m^2}\right). \tag*{(195)} E V 3 = O β , δ ( m 2 1 ) , E ∣ V ∣ 4 = E V 4 = O β , δ ( m 2 1 ) . (195)
The derivatives d θ ( 3 ) ( θ ) d_\theta^{(3)}(\theta) d θ ( 3 ) ( θ ) are uniformly bounded for θ ∈ [ δ , 1 − δ ] \theta\in[\delta,1-\delta] θ ∈ [ δ , 1 − δ ] , and (190) gives
E R 4 , θ ( V ) = O β , δ ( 1 m 2 ) . (196) \mathbb{E}\,R_{4,\theta}(V)=O_{\beta,\delta}\left(\frac{1}{m^2}\right). \tag*{(196)} E R 4 , θ ( V ) = O β , δ ( m 2 1 ) . (196)
Combining these estimates proves Eq. (183) .■
Using (183) and Lemma 20, uniformly over θ ∈ [ δ , 1 − δ ] \theta\in[\delta,1-\delta] θ ∈ [ δ , 1 − δ ] ,
R n , ℓ ( p ^ β , θ ) = 1 2 ∑ m = t t + ℓ − 1 1 m + O β , δ ( ∑ m = t t + ℓ − 1 1 m 2 ) , (197) R_{n,\ell}(\hat{p}_{\beta},\theta)=\frac{1}{2}\sum_{m=t}^{t+\ell-1}\frac{1}{m}+O_{\beta,\delta}\left(\sum_{m=t}^{t+\ell-1}\frac{1}{m^2}\right), \tag*{(197)} R n , ℓ ( p ^ β , θ ) = 2 1 m = t ∑ t + ℓ − 1 m 1 + O β , δ ( m = t ∑ t + ℓ − 1 m 2 1 ) , (197)
which proves Eq. (53) . Finally, we use the following elementary integral comparison for the harmonic sum. For completeness, we include the short proof. For each m m m ,
0 ≤ 1 m − ∫ m m + 1 d x x = ∫ 0 1 ( 1 m − 1 m + u ) d u = ∫ 0 1 u m ( m + u ) d u ≤ ∫ 0 1 u m 2 d u = 1 2 m 2 . (198) \begin{aligned}
0\leq\frac{1}{m}-\int_m^{m+1}\frac{dx}{x} \\
&=\int_0^1\left(\frac{1}{m}-\frac{1}{m+u}\right)du \\
&=\int_0^1\frac{u}{m(m+u)}du\leq\int_0^1\frac{u}{m^2}du=\frac{1}{2m^2}. \tag*{(198)}
\end{aligned} 0 ≤ m 1 − ∫ m m + 1 x d x = ∫ 0 1 ( m 1 − m + u 1 ) d u = ∫ 0 1 m ( m + u ) u d u ≤ ∫ 0 1 m 2 u d u = 2 m 2 1 . (198)
Summing (199) from m = t m=t m = t to t + ℓ − 1 t+\ell-1 t + ℓ − 1 and using ∫ t t + ℓ x − 1 d x = log ( ( t + ℓ ) / t ) \int_t^{t+\ell}x^{-1}dx=\log((t+\ell)/t) ∫ t t + ℓ x − 1 d x = log (( t + ℓ ) / t ) gives
∑ m = t t + ℓ − 1 1 m = log t + ℓ t + O ( ∑ m = t t + ℓ − 1 1 m 2 ) , (199) \sum_{m=t}^{t+\ell-1}\frac{1}{m}=\log\frac{t+\ell}{t}+O\left(\sum_{m=t}^{t+\ell-1}\frac{1}{m^2}\right), \tag*{(199)} m = t ∑ t + ℓ − 1 m 1 = log t t + ℓ + O ( m = t ∑ t + ℓ − 1 m 2 1 ) , (199)
and
∑ m = t t + ℓ − 1 1 m 2 = O ( ℓ t 2 ) = O ( 1 n 2 ℓ ) . (200) \sum_{m=t}^{t+\ell-1}\frac{1}{m^2}=O\left(\frac{\ell}{t^2}\right)=O\left(\frac{1}{n^2\ell}\right). \tag*{(200)} m = t ∑ t + ℓ − 1 m 2 1 = O ( t 2 ℓ ) = O ( n 2 ℓ 1 ) . (200)
Since t = n ℓ t=n\ell t = n ℓ , log ( ( t + ℓ ) / t ) = log ( 1 + 1 / n ) \log((t+\ell)/t)=\log(1+1/n) log (( t + ℓ ) / t ) = log ( 1 + 1/ n ) , and Eq. (54) follows.
Proof of Theorem 6
The proof is based on the uniform one-step analysis of Krichevsky [20 ] . Throughout this appendix, β > 0 \beta>0 β > 0 is fixed. For 0 ≤ a ≤ m 0\leq a\leq m 0 ≤ a ≤ m , let S m , a ∼ Binomial ( m , a / m ) S_{m,a}\sim\operatorname{Binomial}(m,a/m) S m , a ∼ Binomial ( m , a / m ) and define
Δ m , β ( a ) = a ( log a − E log ( S m , a + β ) ) , (201) \Delta_{m,\beta}(a)=a\left(\log a-\mathbb{E}\log(S_{m,a}+\beta)\right), \tag*{(201)} Δ m , β ( a ) = a ( log a − E log ( S m , a + β ) ) , (201)
with the convention 0 log 0 = 0 0\log0=0 0 log 0 = 0 , so that Δ m , β ( 0 ) = 0 \Delta_{m,\beta}(0)=0 Δ m , β ( 0 ) = 0 . Similarly, for a ≥ 0 a\geq0 a ≥ 0 , let Z a ∼ Pois ( a ) Z_a\sim\operatorname{Pois}(a) Z a ∼ Pois ( a ) and define
Δ β ( a ) = a ( log a − E log ( Z a + β ) ) , (202) \Delta_{\beta}(a)=a\left(\log a-\mathbb{E}\log(Z_a+\beta)\right), \tag*{(202)} Δ β ( a ) = a ( log a − E log ( Z a + β ) ) , (202)
again with Δ β ( 0 ) = 0 \Delta_{\beta}(0)=0 Δ β ( 0 ) = 0 . Let also
G β ( a ) = β + Δ β ( a ) . (203) G_{\beta}(a)=\beta+\Delta_{\beta}(a). \tag*{(203)} G β ( a ) = β + Δ β ( a ) . (203)
We first record an exact decomposition of the one-step regret. If μ = m θ \mu=m\theta μ = m θ and ν = m ( 1 − θ ) \nu=m(1-\theta) ν = m ( 1 − θ ) , then, by expanding the definition of ρ m ( β ) \rho_m^{(\beta)} ρ m ( β ) , we get
ρ m ( β ) ( θ ) = log ( m + 2 β ) + θ log θ + ( 1 − θ ) log ( 1 − θ ) − θ E log ( K + β ) − ( 1 − θ ) E log ( m − K + β ) , (204) \rho_m^{(\beta)}(\theta)=\log(m+2\beta)+\theta\log\theta+(1-\theta)\log(1-\theta)-\theta\mathbb{E}\log(K+\beta)-(1-\theta)\mathbb{E}\log(m-K+\beta), \tag*{(204)} ρ m ( β ) ( θ ) = log ( m + 2 β ) + θ log θ + ( 1 − θ ) log ( 1 − θ ) − θ E log ( K + β ) − ( 1 − θ ) E log ( m − K + β ) , (204)
where K ∼ Binomial ( m , θ ) K\sim\operatorname{Binomial}(m,\theta) K ∼ Binomial ( m , θ ) . Multiplying by m m m and using m − K ∼ Binomial ( m , 1 − θ ) m-K\sim\operatorname{Binomial}(m,1-\theta) m − K ∼ Binomial ( m , 1 − θ ) gives
m ρ m ( β ) ( θ ) = m log ( 1 + 2 β m ) + Δ m , β ( μ ) + Δ m , β ( ν ) . (205) m\rho_m^{(\beta)}(\theta)=m\log\left(1+\frac{2\beta}{m}\right)+\Delta_{m,\beta}(\mu)+\Delta_{m,\beta}(\nu). \tag*{(205)} m ρ m ( β ) ( θ ) = m log ( 1 + m 2 β ) + Δ m , β ( μ ) + Δ m , β ( ν ) . (205)
We now need the following two approximation results by Krichevsky [20 ] , for Δ m , β \Delta_{m,\beta} Δ m , β in the small- and large-a a a regimes respectively. For the sake of completeness, we also include our version of their proofs following our notation.
Lemma 21 (Uniform Poisson approximation [20 ] ). For every L < ∞ L<\infty L < ∞ ,
sup 0 ≤ a ≤ L ∣ Δ m , β ( a ) − Δ β ( a ) ∣ → 0. (206) \sup_{0\leq a\leq L}\left|\Delta_{m,\beta}(a)-\Delta_{\beta}(a)\right|\to0. \tag*{(206)} 0 ≤ a ≤ L sup ∣ Δ m , β ( a ) − Δ β ( a ) ∣ → 0. (206)
Proof. Let g ( k ) = log ( k + β ) g(k)=\log(k+\beta) g ( k ) = log ( k + β ) and, for an integer M M M , let
g M ( k ) = g ( k ) 1 { k ≤ M } . (207) g_M(k)=g(k)\mathbf{1}_{\{k\leq M\}}. \tag*{(207)} g M ( k ) = g ( k ) 1 { k ≤ M } . (207)
The truncation is necessary because Le Cam’s inequality, which is our main tool, controls expectations of bounded functions, whereas g g g is unbounded. For fixed M M M , the function g M g_M g M is bounded. Le Cam’s inequality then gives
d T V ( Binomial ( m , a m ) , Pois ( a ) ) ≤ a 2 m , 0 ≤ a ≤ L , (208) d_{\mathrm{TV}}\left(\operatorname{Binomial}\left(m,\frac{a}{m}\right),\operatorname{Pois}(a)\right)\leq\frac{a^2}{m},\qquad0\leq a\leq L, \tag*{(208)} d TV ( Binomial ( m , m a ) , Pois ( a ) ) ≤ m a 2 , 0 ≤ a ≤ L , (208)
for all sufficiently large m m m . Hence, uniformly over 0 ≤ a ≤ L 0\leq a\leq L 0 ≤ a ≤ L ,
∣ E g M ( S m , a ) − E g M ( Z a ) ∣ ≤ 2 ∥ g M ∥ ∞ a 2 m ≤ 2 ∥ g M ∥ ∞ L 2 m , (209) \left|\mathbb{E}g_M(S_{m,a})-\mathbb{E}g_M(Z_a)\right|\leq2\lVert g_M\rVert_{\infty}\frac{a^2}{m}\leq2\lVert g_M\rVert_{\infty}\frac{L^2}{m}, \tag*{(209)} ∣ E g M ( S m , a ) − E g M ( Z a ) ∣ ≤ 2 ∥ g M ∥ ∞ m a 2 ≤ 2 ∥ g M ∥ ∞ m L 2 , (209)
which tends to zero as m → ∞ m\to\infty m → ∞ for each fixed M M M .
It remains to remove the truncation uniformly. Choose c β < ∞ c_\beta<\infty c β < ∞ such that
∣ log ( k + β ) ∣ ≤ c β + k , k ≥ 0. (210) |\log(k+\beta)|\leq c_\beta+k,\qquad k\geq0. \tag*{(210)} ∣ log ( k + β ) ∣ ≤ c β + k , k ≥ 0. (210)
If X X X denotes either S m , a S_{m,a} S m , a or Z a Z_a Z a , then, uniformly over 0 ≤ a ≤ L 0\leq a\leq L 0 ≤ a ≤ L ,
E X = a ≤ L , E X 2 ≤ L + L 2 . (211) \mathbb{E}X=a\leq L,\qquad\mathbb{E}X^2\leq L+L^2. \tag*{(211)} E X = a ≤ L , E X 2 ≤ L + L 2 . (211)
Consequently,
E ∣ g ( X ) − g M ( X ) ∣ ≤ E [ ( c β + X ) 1 { X > M } ] ≤ c β E X M + E X 2 M ≤ c β L + L + L 2 M . (212) \begin{aligned}
\mathbb{E}|g(X)-g_M(X)|\leq\mathbb{E}\left[(c_\beta+X)\mathbf{1}_{\{X>M\}}\right] \\
&\leq c_\beta\frac{\mathbb{E}X}{M}+\frac{\mathbb{E}X^2}{M}\leq\frac{c_\beta L+L+L^2}{M}. \tag*{(212)}
\end{aligned} E ∣ g ( X ) − g M ( X ) ∣ ≤ E [ ( c β + X ) 1 { X > M } ] ≤ c β M E X + M E X 2 ≤ M c β L + L + L 2 . (212)
The bound is uniform in m m m for X = S m , a X=S_{m,a} X = S m , a and uniform in a ∈ [ 0 , L ] a\in[0,L] a ∈ [ 0 , L ] for both choices of X X X .
Define
R M ( S ) = sup m ≥ ⌈ L ⌉ sup 0 ≤ a ≤ L ∣ E g ( S m , a ) − g M ( S m , a ) ∣ . (213) R_M^{(S)}=\sup_{m\geq\lceil L\rceil}\sup_{0\leq a\leq L}|\mathbb{E}g(S_{m,a})-g_M(S_{m,a})|. \tag*{(213)} R M ( S ) = m ≥ ⌈ L ⌉ sup 0 ≤ a ≤ L sup ∣ E g ( S m , a ) − g M ( S m , a ) ∣. (213)
and
R M ( Z ) = sup 0 ≤ a ≤ L ∣ E g ( Z a ) − g M ( Z a ) ∣ . (214) R_M^{(Z)}=\sup_{0\leq a\leq L}|\mathbb{E}g(Z_a)-g_M(Z_a)|. \tag*{(214)} R M ( Z ) = 0 ≤ a ≤ L sup ∣ E g ( Z a ) − g M ( Z a ) ∣. (214)
By (213), R M ( S ) + R M ( Z ) → 0 R_M^{(S)}+R_M^{(Z)}\to0 R M ( S ) + R M ( Z ) → 0 as M → ∞ M\to\infty M → ∞ . The triangle inequality and (210) imply, for every fixed M M M ,
lim sup m → ∞ sup 0 ≤ a ≤ L ∣ E g ( S m , a ) − E g ( Z a ) ∣ ≤ R M ( S ) + R M ( Z ) . (215) \limsup_{m\to\infty}\sup_{0\leq a\leq L}|\mathbb{E}g(S_{m,a})-\mathbb{E}g(Z_a)|\leq R_M^{(S)}+R_M^{(Z)}. \tag*{(215)} m → ∞ lim sup 0 ≤ a ≤ L sup ∣ E g ( S m , a ) − E g ( Z a ) ∣ ≤ R M ( S ) + R M ( Z ) . (215)
Letting M → ∞ M\to\infty M → ∞ gives
sup 0 ≤ a ≤ L ∣ E log ( S m , a + β ) − E log ( Z a + β ) ∣ → 0. (216) \sup_{0\leq a\leq L}|\mathbb{E}\log(S_{m,a}+\beta)-\mathbb{E}\log(Z_a+\beta)|\to0. \tag*{(216)} 0 ≤ a ≤ L sup ∣ E log ( S m , a + β ) − E log ( Z a + β ) ∣ → 0. (216)
Multiplication by a ≤ L a\leq L a ≤ L proves the claim. The lemma above is [20 Lemma 5] , rewritten using our notation, with a = m p a=mp a = m p . Indeed, the functions in the original lemma satisfy F K ( m , β , a / m ) = β + Δ m , β ( a ) F^{K}(m,\beta,a/m)=\beta+\Delta_{m,\beta}(a) F K ( m , β , a / m ) = β + Δ m , β ( a ) and Φ ( β , a ) = β + Δ β ( a ) \Phi(\beta,a)=\beta+\Delta_{\beta}(a) Φ ( β , a ) = β + Δ β ( a ) .
Lemma 22 (Uniform large-count expansion [20 ] ). There exists a function η β ( L ) \eta_\beta(L) η β ( L ) with η β ( L ) → 0 \eta_\beta(L) \to0 η β ( L ) → 0 as L → ∞ L \to\infty L → ∞ such that
lim sup m → ∞ sup L ≤ a ≤ m ∣ Δ m , β ( a ) + β − 1 2 ( 1 − a m ) ∣ ≤ η β ( L ) . (217) \limsup_{m\to\infty}\sup_{L\le a\le m}\left|\Delta_{m,\beta}(a)+\beta-\frac{1}{2}\left(1-\frac{a}{m}\right)\right|\le\eta_\beta(L). \tag*{(217)} m → ∞ lim sup L ≤ a ≤ m sup Δ m , β ( a ) + β − 2 1 ( 1 − m a ) ≤ η β ( L ) . (217) Moreover,
Δ β ( a ) = − β + 1 2 + o ( 1 ) , a → ∞ , (218) \Delta_\beta(a)=-\beta+\frac{1}{2}+o(1),\qquad a\to\infty, \tag*{(218)} Δ β ( a ) = − β + 2 1 + o ( 1 ) , a → ∞ , (218) and hence G β ( a ) → 1 / 2 G_\beta(a)\to1/2 G β ( a ) → 1/2 as a → ∞ a\to\infty a → ∞ .
Proof. Let S = S m , a S=S_{m,a} S = S m , a , p = a / m p=a/m p = a / m , and Δ = S − a \Delta=S-a Δ = S − a . Then
E Δ = 0 , E Δ 2 = a ( 1 − a / m ) . (219) \mathbb{E}\Delta=0,\qquad\mathbb{E}\Delta^2=a(1-a/m). \tag*{(219)} E Δ = 0 , E Δ 2 = a ( 1 − a / m ) . (219)
We use the Taylor expansion of f ( x ) = log ( x + β ) f(x)=\log(x+\beta) f ( x ) = log ( x + β ) around a a a . On the event
A = { ∣ S − a ∣ ≤ a / 2 } , (220) A=\{|S-a|\le a/2\}, \tag*{(220)} A = { ∣ S − a ∣ ≤ a /2 } , (220)
the third derivative of f f f is bounded by C β / a 3 C_\beta/a^3 C β / a 3 , and therefore
∣ f ( S ) − f ( a ) − f ′ ( a ) Δ − 1 2 f ′ ′ ( a ) Δ 2 ∣ ≤ C β ∣ Δ ∣ 3 a 3 . (221) \left|f(S)-f(a)-f'(a)\Delta-\frac{1}{2}f''(a)\Delta^2\right|\le C_\beta\frac{|\Delta|^3}{a^3}. \tag*{(221)} f ( S ) − f ( a ) − f ′ ( a ) Δ − 2 1 f ′′ ( a ) Δ 2 ≤ C β a 3 ∣Δ ∣ 3 . (221)
For binomial random variables, the standard moment bound
E ∣ S − a ∣ 3 ≤ C ( ( a ( 1 − a / m ) ) 3 / 2 + a ( 1 − a / m ) ) (222) \mathbb{E}|S-a|^3\le C\left((a(1-a/m))^{3/2}+a(1-a/m)\right) \tag*{(222)} E ∣ S − a ∣ 3 ≤ C ( ( a ( 1 − a / m ) ) 3/2 + a ( 1 − a / m ) ) (222)
≤ C ( a 3 / 2 + a ) . (223) \le C(a^{3/2}+a). \tag*{(223)} ≤ C ( a 3/2 + a ) . (223)
holds with a universal constant C C C . Hence the contribution of the Taylor remainder on A A A is bounded, after multiplication by a a a , by
C β ( a − 1 / 2 + a − 1 ) . (224) C_\beta\left(a^{-1/2}+a^{-1}\right). \tag*{(224)} C β ( a − 1/2 + a − 1 ) . (224)
On A c A^c A c , a Chernoff bound gives P ( A c ) ≤ 2 e − c a \mathbb{P}(A^c)\le2e^{-ca} P ( A c ) ≤ 2 e − c a for a universal c > 0 c>0 c > 0 . Together with log ( S + β ) ≤ c β + S \log(S+\beta)\le c_\beta+S log ( S + β ) ≤ c β + S and the binomial moment bounds, this implies that the contribution of A c A^c A c to the expectation, after multiplication by a a a , is bounded by a term that is exponentially small in a a a , uniformly in m m m and a ≤ m a\le m a ≤ m . Consequently,
E log ( S + β ) = log ( a + β ) − a ( 1 − a / m ) 2 ( a + β ) 2 + r m , a , (225) \mathbb{E}\log(S+\beta)=\log(a+\beta)-\frac{a(1-a/m)}{2(a+\beta)^2}+r_{m,a}, \tag*{(225)} E log ( S + β ) = log ( a + β ) − 2 ( a + β ) 2 a ( 1 − a / m ) + r m , a , (225)
where
lim sup m → ∞ sup L ≤ a ≤ m a ∣ r m , a ∣ ≤ C β L − 1 / 2 + o m ( 1 ) . (226) \limsup_{m\to\infty}\sup_{L\le a\le m}a|r_{m,a}|\le C_\beta L^{-1/2}+o_m(1). \tag*{(226)} m → ∞ lim sup L ≤ a ≤ m sup a ∣ r m , a ∣ ≤ C β L − 1/2 + o m ( 1 ) . (226)
Using
a ( log a − log ( a + β ) ) = − β + O β ( a − 1 ) (227) a(\log a-\log(a+\beta))=-\beta+O_\beta(a^{-1}) \tag*{(227)} a ( log a − log ( a + β )) = − β + O β ( a − 1 ) (227)
and
a 2 ( 1 − a / m ) 2 ( a + β ) 2 = 1 2 ( 1 − a m ) + O β ( a − 1 ) , (228) \frac{a^2(1-a/m)}{2(a+\beta)^2}=\frac{1}{2}\left(1-\frac{a}{m}\right)+O_\beta(a^{-1}), \tag*{(228)} 2 ( a + β ) 2 a 2 ( 1 − a / m ) = 2 1 ( 1 − m a ) + O β ( a − 1 ) , (228)
we obtain
Δ m , β ( a ) = − β + 1 2 ( 1 − a m ) + ε m , a , (229) \Delta_{m,\beta}(a)=-\beta+\frac{1}{2}\left(1-\frac{a}{m}\right)+\varepsilon_{m,a}, \tag*{(229)} Δ m , β ( a ) = − β + 2 1 ( 1 − m a ) + ε m , a , (229)
with
lim sup m → ∞ sup L ≤ a ≤ m ∣ ε m , a ∣ → 0 (230) \limsup_{m\to\infty}\sup_{L\le a\le m}|\varepsilon_{m,a}|\to0 \tag*{(230)} m → ∞ lim sup L ≤ a ≤ m sup ∣ ε m , a ∣ → 0 (230)
as L → ∞ L\to\infty L → ∞ . This proves (218). The Poisson expansion (219) is obtained by the same Taylor argument with Z a ∼ Pois ( a ) Z_a\sim\operatorname{Pois}(a) Z a ∼ Pois ( a ) , for which E ( Z a − a ) 2 = a \mathbb{E}(Z_a-a)^2=a E ( Z a − a ) 2 = a . In that case the factor 1 − a / m 1-a/m 1 − a / m is replaced by 1 1 1 , yielding Δ β ( a ) = − β + 1 / 2 + o ( 1 ) \Delta_\beta(a)=-\beta+1/2+o(1) Δ β ( a ) = − β + 1/2 + o ( 1 ) .
The lemma above combines two results from [20 ] . The upper bound is given in [20 Lemma 2] , while the lower bound follows from [20 Eq. (60)] . Combining the two preceding lemmas gives the following binary one-step uniform asymptotics.
Lemma 23 (Uniform one-step asymptotics [20 ] ). Let
C β = sup λ ≥ 0 G β ( λ ) . (231) C_\beta=\sup_{\lambda\ge0}G_\beta(\lambda). \tag*{(231)} C β = λ ≥ 0 sup G β ( λ ) . (231) Then
sup θ ∈ [ 0 , 1 ] m ρ m ( β ) ( θ ) → C β . (232) \sup_{\theta\in[0,1]}m\rho_m^{(\beta)}(\theta)\to C_\beta. \tag*{(232)} θ ∈ [ 0 , 1 ] sup m ρ m ( β ) ( θ ) → C β . (232)
Proof. By symmetry it is enough to consider 0 ≤ θ ≤ 1 / 2 0\le\theta\le1/2 0 ≤ θ ≤ 1/2 . Put μ = m θ \mu=m\theta μ = m θ and ν = m ( 1 − θ ) \nu=m(1-\theta) ν = m ( 1 − θ ) . Fix L < ∞ L<\infty L < ∞ . If μ ≤ L \mu\le L μ ≤ L , then Lemma 21 gives
Δ m , β ( μ ) = Δ β ( μ ) + o ( 1 ) (233) \Delta_{m,\beta}(\mu)=\Delta_\beta(\mu)+o(1) \tag*{(233)} Δ m , β ( μ ) = Δ β ( μ ) + o ( 1 ) (233)
uniformly over μ ∈ [ 0 , L ] \mu\in[0,L] μ ∈ [ 0 , L ] . Also ν = m − μ → ∞ \nu=m-\mu\to\infty ν = m − μ → ∞ uniformly over this interval, and Lemma 22 gives
Δ m , β ( ν ) = − β + o ( 1 ) . (234) \Delta_{m,\beta}(\nu)=-\beta+o(1). \tag*{(234)} Δ m , β ( ν ) = − β + o ( 1 ) . (234)
Since m log ( 1 + 2 β / m ) → 2 β m\log(1+2\beta/m)\to2\beta m log ( 1 + 2 β / m ) → 2 β , the decomposition (205) yields
sup 0 ≤ μ ≤ L ∣ m ρ m ( β ) ( μ / m ) − G β ( μ ) ∣ → 0. (235) \sup_{0\le\mu\le L}\left|m\rho_m^{(\beta)}(\mu/m)-G_\beta(\mu)\right|\to0. \tag*{(235)} 0 ≤ μ ≤ L sup m ρ m ( β ) ( μ / m ) − G β ( μ ) → 0. (235)
Therefore
lim sup m → ∞ sup 0 ≤ θ ≤ 1 / 2 m θ ≤ L m ρ m ( β ) ( θ ) ≤ sup 0 ≤ λ ≤ L G β ( λ ) . (236) \limsup_{m\to\infty}\sup_{\substack{0\le\theta\le1/2\\m\theta\le L}}m\rho_m^{(\beta)}(\theta)\le\sup_{0\le\lambda\le L}G_\beta(\lambda). \tag*{(236)} m → ∞ lim sup 0 ≤ θ ≤ 1/2 m θ ≤ L sup m ρ m ( β ) ( θ ) ≤ 0 ≤ λ ≤ L sup G β ( λ ) . (236)
If m θ ≥ L m\theta\ge L m θ ≥ L , then both μ \mu μ and ν \nu ν are at least L L L , for all sufficiently large m m m . Applying Lemma 22 to both terms in (205) gives
Δ m , β ( μ ) = − β + 1 − θ 2 + o L ( 1 ) + o m ( 1 ) , (237) \Delta_{m,\beta}(\mu)=-\beta+\frac{1-\theta}{2}+o_L(1)+o_m(1), \tag*{(237)} Δ m , β ( μ ) = − β + 2 1 − θ + o L ( 1 ) + o m ( 1 ) , (237)
Δ m , β ( ν ) = − β + θ 2 + o L ( 1 ) + o m ( 1 ) . (238) \Delta_{m,\beta}(\nu)=-\beta+\frac{\theta}{2}+o_L(1)+o_m(1). \tag*{(238)} Δ m , β ( ν ) = − β + 2 θ + o L ( 1 ) + o m ( 1 ) . (238)
where o L ( 1 ) o_L(1) o L ( 1 ) denotes a quantity tending to zero as L → ∞ L\to\infty L → ∞ , uniformly in m m m . Hence
lim sup m → ∞ sup 0 ≤ θ ≤ 1 / 2 m θ ≥ L m ρ m ( β ) ( θ ) ≤ 1 2 + o L ( 1 ) . (239) \limsup_{m\to\infty}\sup_{\substack{0\le\theta\le1/2\\m\theta\ge L}}m\rho_m^{(\beta)}(\theta)\le\frac{1}{2}+o_L(1). \tag*{(239)} m → ∞ lim sup 0 ≤ θ ≤ 1/2 m θ ≥ L sup m ρ m ( β ) ( θ ) ≤ 2 1 + o L ( 1 ) . (239)
Taking first m → ∞ m\to\infty m → ∞ and then L → ∞ L\to\infty L → ∞ , and using G β ( λ ) → 1 / 2 G_\beta(\lambda)\to1/2 G β ( λ ) → 1/2 from Lemma 22, gives
lim sup m → ∞ sup θ ∈ [ 0 , 1 ] m ρ m ( β ) ( θ ) ≤ C β . (240) \limsup_{m\to\infty}\sup_{\theta\in[0,1]}m\rho_m^{(\beta)}(\theta)\le C_\beta. \tag*{(240)} m → ∞ lim sup θ ∈ [ 0 , 1 ] sup m ρ m ( β ) ( θ ) ≤ C β . (240)
For the reverse inequality, fix any λ ≥ 0 \lambda\ge0 λ ≥ 0 and take θ m = λ / m \theta_m=\lambda/m θ m = λ / m . Equation (235) gives
lim m → ∞ m ρ m ( β ) ( θ m ) = G β ( λ ) . (241) \lim_{m\to\infty}m\rho_m^{(\beta)}(\theta_m)=G_\beta(\lambda). \tag*{(241)} m → ∞ lim m ρ m ( β ) ( θ m ) = G β ( λ ) . (241)
Moreover, for any fixed θ ∈ ( 0 , 1 ) \theta\in(0,1) θ ∈ ( 0 , 1 ) , applying Lemma 22 to m θ m\theta m θ and m ( 1 − θ ) m(1-\theta) m ( 1 − θ ) gives
lim m → ∞ m ρ m ( β ) ( θ ) = 1 2 . (242) \lim_{m\to\infty}m\rho_m^{(\beta)}(\theta)=\frac{1}{2}. \tag*{(242)} m → ∞ lim m ρ m ( β ) ( θ ) = 2 1 . (242)
Thus the liminf of the left-hand side of (233) is at least C β C_\beta C β , completing the proof.
We now return to the batch regret. The conditional add-β \beta β predictor can be written sequentially as
p ^ β ( y ℓ ∣ x t ) = ∏ i = 1 ℓ q t + i − 1 ( β ) ( y i ∣ x t , y i − 1 ) , (243) \hat{p}_{\beta}(y^{\ell}\mid x^t)=\prod_{i=1}^{\ell}q_{t+i-1}^{(\beta)}(y_i\mid x^t,y^{i-1}), \tag*{(243)} p ^ β ( y ℓ ∣ x t ) = i = 1 ∏ ℓ q t + i − 1 ( β ) ( y i ∣ x t , y i − 1 ) , (243)
where
q m ( β ) ( 1 ∣ z m ) = n ( z m ) + β m + 2 β (244) q_m^{(\beta)}(1\mid z^m)=\frac{n(z^m)+\beta}{m+2\beta} \tag*{(244)} q m ( β ) ( 1 ∣ z m ) = m + 2 β n ( z m ) + β (244)
and n ( z m ) n(z^m) n ( z m ) is the number of ones in z m z^m z m . Therefore, by the chain rule for KL divergence and by the fact that the total number of ones in X t Y i X^tY^i X t Y i is binomial with parameters t + i t+i t + i and θ \theta θ ,
R n , ℓ ( p ^ β , θ ) = ∑ i = 0 ℓ − 1 ρ t + i ( β ) ( θ ) . (245) R_{n,\ell}(\hat{p}_{\beta},\theta)=\sum_{i=0}^{\ell-1}\rho_{t+i}^{(\beta)}(\theta). \tag*{(245)} R n , ℓ ( p ^ β , θ ) = i = 0 ∑ ℓ − 1 ρ t + i ( β ) ( θ ) . (245)
We now prove Theorem 6. By Lemma 23, there exists a sequence ε m ↓ 0 \varepsilon_m\downarrow0 ε m ↓ 0 such that
sup θ ∈ [ 0 , 1 ] ρ m ( β ) ( θ ) ≤ C β + ε m m . (246) \sup_{\theta\in[0,1]}\rho_m^{(\beta)}(\theta)\leq\frac{C_{\beta}+\varepsilon_m}{m}. \tag*{(246)} θ ∈ [ 0 , 1 ] sup ρ m ( β ) ( θ ) ≤ m C β + ε m . (246)
Let ε ˉ t = sup m ≥ t ε m \bar{\varepsilon}_t=\sup_{m\geq t}\varepsilon_m ε ˉ t = sup m ≥ t ε m , so that ε ˉ t → 0 \bar{\varepsilon}_t\to0 ε ˉ t → 0 . Then
sup θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β , θ ) ≤ ( C β + ε ˉ t ) ∑ i = 0 ℓ − 1 1 t + i . (247) \sup_{\theta\in[0,1]}R_{n,\ell}(\hat{p}_{\beta},\theta)\leq(C_{\beta}+\bar{\varepsilon}_t)\sum_{i=0}^{\ell-1}\frac{1}{t+i}. \tag*{(247)} θ ∈ [ 0 , 1 ] sup R n , ℓ ( p ^ β , θ ) ≤ ( C β + ε ˉ t ) i = 0 ∑ ℓ − 1 t + i 1 . (247)
The harmonic sum satisfies
∑ i = 0 ℓ − 1 1 t + i = log t + ℓ t + O ( ℓ t 2 ) (248) \sum_{i=0}^{\ell-1}\frac{1}{t+i}=\log\frac{t+\ell}{t}+O\left(\frac{\ell}{t^2}\right) \tag*{(248)} i = 0 ∑ ℓ − 1 t + i 1 = log t t + ℓ + O ( t 2 ℓ ) (248)
= log ( 1 + 1 n ) + O ( 1 n 2 ℓ ) . (249) =\log\left(1+\frac{1}{n}\right)+O\left(\frac{1}{n^2\ell}\right). \tag*{(249)} = log ( 1 + n 1 ) + O ( n 2 ℓ 1 ) . (249)
Since n → ∞ n\to\infty n → ∞ , the last error is o ( 1 / n ) o(1/n) o ( 1/ n ) , and (248) gives the desired upper bound.
It remains to prove the matching lower bound. Let ε > 0 \varepsilon>0 ε > 0 . By the definition C β = sup λ ≥ 0 G β ( λ ) C_{\beta}=\sup_{\lambda\geq0}G_{\beta}(\lambda) C β = sup λ ≥ 0 G β ( λ ) and by the limit G β ( λ ) → 1 / 2 G_{\beta}(\lambda)\to1/2 G β ( λ ) → 1/2 , we may choose a finite λ ≥ 0 \lambda\geq0 λ ≥ 0 such that G β ( λ ) ≥ C β − ε G_{\beta}(\lambda)\geq C_{\beta}-\varepsilon G β ( λ ) ≥ C β − ε . Set θ t = λ / t \theta_t=\lambda/t θ t = λ / t . For m = t + i m=t+i m = t + i , 0 ≤ i ≤ ℓ − 1 0\leq i\leq\ell-1 0 ≤ i ≤ ℓ − 1 , we have
m θ t = λ ( 1 + i t ) , (250) m\theta_t=\lambda\left(1+\frac{i}{t}\right), \tag*{(250)} m θ t = λ ( 1 + t i ) , (250)
and this quantity converges to λ \lambda λ uniformly in i i i , because i / t ≤ ℓ / t = 1 / n → 0 i/t\leq\ell/t=1/n\to0 i / t ≤ ℓ / t = 1/ n → 0 . The uniform Poisson approximation in (236) then gives
ρ t + i ( β ) ( θ t ) = G β ( λ ) + o ( 1 ) t + i , (251) \rho_{t+i}^{(\beta)}(\theta_t)=\frac{G_{\beta}(\lambda)+o(1)}{t+i}, \tag*{(251)} ρ t + i ( β ) ( θ t ) = t + i G β ( λ ) + o ( 1 ) , (251)
uniformly over 0 ≤ i ≤ ℓ − 1 0\leq i\leq\ell-1 0 ≤ i ≤ ℓ − 1 . Consequently,
R n , ℓ ( p ^ β , θ t ) = ( G β ( λ ) + o ( 1 ) ) ∑ i = 0 ℓ − 1 1 t + i . (252) R_{n,\ell}(\hat{p}_{\beta},\theta_t)=(G_{\beta}(\lambda)+o(1))\sum_{i=0}^{\ell-1}\frac{1}{t+i}. \tag*{(252)} R n , ℓ ( p ^ β , θ t ) = ( G β ( λ ) + o ( 1 )) i = 0 ∑ ℓ − 1 t + i 1 . (252)
Using (250), this is at least
( C β − ε ) log ( 1 + 1 n ) + o ( 1 n ) . (253) (C_{\beta}-\varepsilon)\log\left(1+\frac{1}{n}\right)+o\left(\frac{1}{n}\right). \tag*{(253)} ( C β − ε ) log ( 1 + n 1 ) + o ( n 1 ) . (253)
Since ε > 0 \varepsilon>0 ε > 0 is arbitrary, the lower bound matches the upper bound, completing the proof of Theorem 6.
Proof of Proposition 7
We first prove that the threshold in (73) is well defined and lies in [ 1 / 2 , 1 ] [1/2,1] [ 1/2 , 1 ] . The value β = 1 \beta=1 β = 1 belongs to the defining set: by Jensen’s inequality applied to the convex function − log x -\log x − log x ,
E log ( Z + 1 ) = E [ − log 1 Z + 1 ] (254) \mathbb{E}\log(Z+1)=\mathbb{E}\left[-\log\frac{1}{Z+1}\right] \tag*{(254)} E log ( Z + 1 ) = E [ − log Z + 1 1 ] (254)
≥ − log E 1 Z + 1 (255) \geq-\log\mathbb{E}\frac{1}{Z+1} \tag*{(255)} ≥ − log E Z + 1 1 (255)
= log λ 1 − e − λ > log λ , (256) =\log\frac{\lambda}{1-e^{-\lambda}}>\log\lambda, \tag*{(256)} = log 1 − e − λ λ > log λ , (256)
where we used the Poisson identity
E 1 Z + 1 = e − λ ∑ k ≥ 0 λ k k ! ( k + 1 ) = 1 − e − λ λ . (257) \mathbb{E}\frac{1}{Z+1}=e^{-\lambda}\sum_{k\geq0}\frac{\lambda^k}{k!(k+1)}=\frac{1-e^{-\lambda}}{\lambda}. \tag*{(257)} E Z + 1 1 = e − λ k ≥ 0 ∑ k ! ( k + 1 ) λ k = λ 1 − e − λ . (257)
Thus Δ 1 ( λ ) < 0 \Delta_1(\lambda)<0 Δ 1 ( λ ) < 0 for every λ > 0 \lambda>0 λ > 0 . Conversely, if β < 1 / 2 \beta<1/2 β < 1/2 , then (71) implies
Δ β ( λ ) = 1 2 − β + O β ( 1 λ ) > 0 (258) \Delta_{\beta}(\lambda)=\frac{1}{2}-\beta+O_{\beta}\left(\frac{1}{\lambda}\right)>0 \tag*{(258)} Δ β ( λ ) = 2 1 − β + O β ( λ 1 ) > 0 (258)
for all sufficiently large λ \lambda λ , so no β < 1 / 2 \beta<1/2 β < 1/2 belongs to the defining set.
For each fixed λ > 0 \lambda>0 λ > 0 , the map β ↦ Δ β ( λ ) \beta\mapsto\Delta_{\beta}(\lambda) β ↦ Δ β ( λ ) is continuous and strictly decreasing, since
∂ ∂ β Δ β ( λ ) = − λ E 1 Z + β < 0. (259) \frac{\partial}{\partial\beta}\Delta_{\beta}(\lambda)=-\lambda\mathbb{E}\frac{1}{Z+\beta}<0. \tag*{(259)} ∂ β ∂ Δ β ( λ ) = − λ E Z + β 1 < 0. (259)
Taking a decreasing sequence of admissible values converging to β 0 \beta_0 β 0 and passing to the limit therefore gives
Δ β 0 ( λ ) ≤ 0 , λ > 0. (260) \Delta_{\beta_0}(\lambda)\leq0,\qquad\lambda>0. \tag*{(260)} Δ β 0 ( λ ) ≤ 0 , λ > 0. (260)
Since Δ β 0 ( 0 ) = 0 \Delta_{\beta_0}(0)=0 Δ β 0 ( 0 ) = 0 , it follows that
C β 0 = sup λ ≥ 0 { β 0 + Δ β 0 ( λ ) } = β 0 . (261) C_{\beta_0}=\sup_{\lambda\geq0}\{\beta_0+\Delta_{\beta_0}(\lambda)\}=\beta_0. \tag*{(261)} C β 0 = λ ≥ 0 sup { β 0 + Δ β 0 ( λ )} = β 0 . (261)
The same monotonicity shows that Δ β ( λ ) ≤ 0 \Delta_{\beta}(\lambda)\leq0 Δ β ( λ ) ≤ 0 for every λ > 0 \lambda>0 λ > 0 and every β ≥ β 0 \beta\geq\beta_0 β ≥ β 0 . Hence C β = β C_{\beta}=\beta C β = β throughout this range.
It remains to show that no β < β 0 \beta<\beta_0 β < β 0 gives a smaller value. For each fixed λ > 0 \lambda>0 λ > 0 , let b ( λ ) > 0 b(\lambda)>0 b ( λ ) > 0 be the unique number satisfying
Δ b ( λ ) ( λ ) = 0. (262) \Delta_{b(\lambda)}(\lambda)=0. \tag*{(262)} Δ b ( λ ) ( λ ) = 0. (262)
Uniqueness follows from (260). Existence follows because Δ β ( λ ) → + ∞ \Delta_{\beta}(\lambda)\to+\infty Δ β ( λ ) → + ∞ as β ↓ 0 \beta\downarrow0 β ↓ 0 , due to the event Z = 0 Z=0 Z = 0 , whereas Δ 1 ( λ ) < 0 \Delta_1(\lambda)<0 Δ 1 ( λ ) < 0 . The threshold definition is therefore equivalent to
β 0 = sup λ > 0 b ( λ ) . (263) \beta_0=\sup_{\lambda>0}b(\lambda). \tag*{(263)} β 0 = λ > 0 sup b ( λ ) . (263)
Fix β < β 0 \beta<\beta_0 β < β 0 and ε > 0 \varepsilon>0 ε > 0 , and choose λ \lambda λ such that
b ( λ ) > max { β , β 0 − ε } . (264) b(\lambda)>\max\{\beta,\beta_0-\varepsilon\}. \tag*{(264)} b ( λ ) > max { β , β 0 − ε } . (264)
For every s < b ( λ ) s<b(\lambda) s < b ( λ ) , one has Δ s ( λ ) > 0 \Delta_s(\lambda)>0 Δ s ( λ ) > 0 , or equivalently
E log λ Z + s > 0. (265) \mathbb{E}\log\frac{\lambda}{Z+s}>0. \tag*{(265)} E log Z + s λ > 0. (265)
By Jensen’s inequality,
log E λ Z + s ≥ E log λ Z + s > 0. (266) \log\mathbb{E}\frac{\lambda}{Z+s}\geq\mathbb{E}\log\frac{\lambda}{Z+s}>0. \tag*{(266)} log E Z + s λ ≥ E log Z + s λ > 0. (266)
so λ E [ 1 / ( Z + s ) ] > 1 \lambda\mathbb{E}[1/(Z+s)] > 1 λ E [ 1/ ( Z + s )] > 1 . Since (with G s ( ⋅ ) G_s(\cdot) G s ( ⋅ ) as defined in Equation (203) )
∂ ∂ s G s ( λ ) = 1 − λ E 1 Z + s , (267) \frac{\partial}{\partial s}G_s(\lambda)=1-\lambda\mathbb{E}\frac{1}{Z+s}, \tag*{(267)} ∂ s ∂ G s ( λ ) = 1 − λ E Z + s 1 , (267)
we obtain ∂ ∂ s G s ( λ ) < 0 \frac{\partial}{\partial s}G_s(\lambda)<0 ∂ s ∂ G s ( λ ) < 0 for s < b ( λ ) s<b(\lambda) s < b ( λ ) . Consequently,
G β ( λ ) > G b ( λ ) ( λ ) = b ( λ ) > β 0 − ε . (268) G_\beta(\lambda)>G_{b(\lambda)}(\lambda)=b(\lambda)>\beta_0-\varepsilon. \tag*{(268)} G β ( λ ) > G b ( λ ) ( λ ) = b ( λ ) > β 0 − ε . (268)
Thus C β > β 0 − ε C_\beta>\beta_0-\varepsilon C β > β 0 − ε . Letting ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 gives C β ≥ β 0 C_\beta\geq\beta_0 C β ≥ β 0 for every β < β 0 \beta<\beta_0 β < β 0 . Together with C β = β ≥ β 0 C_\beta=\beta\geq\beta_0 C β = β ≥ β 0 for β ≥ β 0 \beta\geq\beta_0 β ≥ β 0 , this proves (75).
Proof of Theorem 8
The proof uses the same one-step decomposition as the proof of Theorem 6 , but now n n n is fixed while ℓ → ∞ \ell\to\infty ℓ → ∞ . Thus t = n ℓ → ∞ t=n\ell\to\infty t = n ℓ → ∞ , whereas the ratio ℓ / t = 1 / n \ell/t=1/n ℓ / t = 1/ n remains fixed. Put
A n = 1 + 1 n . (269) A_n=1+\frac{1}{n}. \tag*{(269)} A n = 1 + n 1 . (269)
We will use that Δ β \Delta_\beta Δ β is continuous on [ 0 , ∞ ) [0,\infty) [ 0 , ∞ ) , which follows from dominated convergence for Poisson expectations.
Let θ ℓ ∈ [ 0 , 1 ] \theta_\ell\in[0,1] θ ℓ ∈ [ 0 , 1 ] be any sequence and write t = n ℓ t=n\ell t = n ℓ . First assume that
t θ ℓ → λ < ∞ . (270) t\theta_\ell\to\lambda<\infty. \tag*{(270)} t θ ℓ → λ < ∞. (270)
For 0 ≤ i ≤ ℓ − 1 0\leq i\leq\ell-1 0 ≤ i ≤ ℓ − 1 , set
u i , t = 1 + i t , m i = t + i . (271) u_{i,t}=1+\frac{i}{t},\qquad m_i=t+i. \tag*{(271)} u i , t = 1 + t i , m i = t + i . (271)
Then u i , t ∈ [ 1 , A n ] u_{i,t}\in[1,A_n] u i , t ∈ [ 1 , A n ] up to an o ( 1 ) o(1) o ( 1 ) endpoint error, and
m i θ ℓ = u i , t t θ ℓ = λ u i , t + o ( 1 ) (272) m_i\theta_\ell=u_{i,t}\,t\theta_\ell=\lambda u_{i,t}+o(1) \tag*{(272)} m i θ ℓ = u i , t t θ ℓ = λ u i , t + o ( 1 ) (272)
uniformly over i i i . Choose L > A n λ + 1 L>A_n\lambda+1 L > A n λ + 1 . For all sufficiently large ℓ \ell ℓ , the quantities m i θ ℓ m_i\theta_\ell m i θ ℓ all lie in [ 0 , L ] [0,L] [ 0 , L ] . The uniform Poisson approximation in (236), applied with m = m i m=m_i m = m i and a = m i θ ℓ a=m_i\theta_\ell a = m i θ ℓ , gives (with G β ( ⋅ ) G_\beta(\cdot) G β ( ⋅ ) as defined in Equation (203) )
m i ρ m i ( β ) ( θ ℓ ) = G β ( m i θ ℓ ) + o ( 1 ) = G β ( λ u i , t ) + o ( 1 ) , (273) m_i\rho_{m_i}^{(\beta)}(\theta_\ell)=G_\beta(m_i\theta_\ell)+o(1)=G_\beta(\lambda u_{i,t})+o(1), \tag*{(273)} m i ρ m i ( β ) ( θ ℓ ) = G β ( m i θ ℓ ) + o ( 1 ) = G β ( λ u i , t ) + o ( 1 ) , (273)
uniformly over 0 ≤ i ≤ ℓ − 1 0\leq i\leq\ell-1 0 ≤ i ≤ ℓ − 1 . In the last equality we used the continuity of G β G_\beta G β on the compact interval [ 0 , L ] [0,L] [ 0 , L ] . By the batch decomposition (246),
R n , ℓ ( p ^ β , θ ℓ ) = ∑ i = 0 ℓ − 1 G β ( λ u i , t ) + o ( 1 ) t + i = 1 t ∑ i = 0 ℓ − 1 G β ( λ u i , t ) u i , t + o ( 1 ) , (274) R_{n,\ell}(\hat p_\beta,\theta_\ell)=\sum_{i=0}^{\ell-1}\frac{G_\beta(\lambda u_{i,t})+o(1)}{t+i}
=\frac{1}{t}\sum_{i=0}^{\ell-1}\frac{G_\beta(\lambda u_{i,t})}{u_{i,t}}+o(1), \tag*{(274)} R n , ℓ ( p ^ β , θ ℓ ) = i = 0 ∑ ℓ − 1 t + i G β ( λ u i , t ) + o ( 1 ) = t 1 i = 0 ∑ ℓ − 1 u i , t G β ( λ u i , t ) + o ( 1 ) , (274)
because ∑ i = 0 ℓ − 1 ( t + i ) − 1 = O n ( 1 ) \sum_{i=0}^{\ell-1}(t+i)^{-1}=O_n(1) ∑ i = 0 ℓ − 1 ( t + i ) − 1 = O n ( 1 ) . The last formula is a Riemann sum with mesh size 1 / t 1/t 1/ t over the interval [ 1 , A n ] [1,A_n] [ 1 , A n ] . Therefore
R n , ℓ ( p ^ β , θ ℓ ) → ∫ 1 A n G β ( λ u ) u d u = ( β + Δ β , n ( λ ) ) log A n . (275) R_{n,\ell}(\hat p_\beta,\theta_\ell)\to\int_1^{A_n}\frac{G_\beta(\lambda u)}{u}\,du=(\beta+\Delta_{\beta,n}(\lambda))\log A_n. \tag*{(275)} R n , ℓ ( p ^ β , θ ℓ ) → ∫ 1 A n u G β ( λ u ) d u = ( β + Δ β , n ( λ )) log A n . (275)
By symmetry of the binary add-β \beta β predictor, the same conclusion holds if t ( 1 − θ ℓ ) → λ < ∞ t(1-\theta_\ell)\to\lambda<\infty t ( 1 − θ ℓ ) → λ < ∞ .
Now assume that
t θ ℓ → ∞ , t ( 1 − θ ℓ ) → ∞ . (276) t\theta_\ell\to\infty,\qquad t(1-\theta_\ell)\to\infty. \tag*{(276)} t θ ℓ → ∞ , t ( 1 − θ ℓ ) → ∞. (276)
Then, uniformly over 0 ≤ i ≤ ℓ − 1 0\leq i\leq\ell-1 0 ≤ i ≤ ℓ − 1 ,
m i θ ℓ → ∞ , m i ( 1 − θ ℓ ) → ∞ . (277) m_i\theta_\ell\to\infty,\qquad m_i(1-\theta_\ell)\to\infty. \tag*{(277)} m i θ ℓ → ∞ , m i ( 1 − θ ℓ ) → ∞. (277)
Applying Lemma 22 to the two terms in (206) gives, uniformly over i i i ,
m i ρ m i ( β ) ( θ ℓ ) = 1 2 + o ( 1 ) . (278) m_i\rho_{m_i}^{(\beta)}(\theta_\ell)=\frac{1}{2}+o(1). \tag*{(278)} m i ρ m i ( β ) ( θ ℓ ) = 2 1 + o ( 1 ) . (278)
Consequently,
R n , ℓ ( p ^ β , θ ℓ ) = ( 1 2 + o ( 1 ) ) ∑ i = 0 ℓ − 1 1 t + i → 1 2 log A n . (279) R_{n,\ell}(\hat p_\beta,\theta_\ell)=\left(\frac{1}{2}+o(1)\right)\sum_{i=0}^{\ell-1}\frac{1}{t+i}
\to\frac{1}{2}\log A_n. \tag*{(279)} R n , ℓ ( p ^ β , θ ℓ ) = ( 2 1 + o ( 1 ) ) i = 0 ∑ ℓ − 1 t + i 1 → 2 1 log A n . (279)
We can now prove the limit formula. For any fixed λ ≥ 0 \lambda\geq0 λ ≥ 0 , choosing θ ℓ = λ / t \theta_\ell=\lambda/t θ ℓ = λ / t and using (276) gives
lim inf ℓ → ∞ max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β , θ ) ≥ ( β + Δ β , n ( λ ) ) log A n . (280) \liminf_{\ell\to\infty}\max_{\theta\in[0,1]}R_{n,\ell}(\hat p_\beta,\theta)\geq(\beta+\Delta_{\beta,n}(\lambda))\log A_n. \tag*{(280)} ℓ → ∞ lim inf θ ∈ [ 0 , 1 ] max R n , ℓ ( p ^ β , θ ) ≥ ( β + Δ β , n ( λ )) log A n . (280)
Taking the supremum over λ \lambda λ yields
lim inf ℓ → ∞ max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β , θ ) ≥ ( β + sup λ ≥ 0 Δ β , n ) log A n . (281) \liminf_{\ell\to\infty}\max_{\theta\in[0,1]}R_{n,\ell}(\hat p_\beta,\theta)\geq\left(\beta+\sup_{\lambda\geq0}\Delta_{\beta,n}\right)\log A_n. \tag*{(281)} ℓ → ∞ lim inf θ ∈ [ 0 , 1 ] max R n , ℓ ( p ^ β , θ ) ≥ ( β + λ ≥ 0 sup Δ β , n ) log A n . (281)
It remains to prove the corresponding upper bound. For each ℓ \ell ℓ , choose θ ℓ \theta_\ell θ ℓ such that
R n , ℓ ( p ^ β , θ ℓ ) ≥ max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β , θ ) − 1 ℓ . (282) R_{n,\ell}(\hat p_\beta,\theta_\ell)\geq\max_{\theta\in[0,1]}R_{n,\ell}(\hat p_\beta,\theta)-\frac{1}{\ell}. \tag*{(282)} R n , ℓ ( p ^ β , θ ℓ ) ≥ θ ∈ [ 0 , 1 ] max R n , ℓ ( p ^ β , θ ) − ℓ 1 . (282)
Consider any subsequence along which the left-hand side converges to the limsup. Passing to a further subsequence if necessary, t θ ℓ t\theta_\ell t θ ℓ converges in the extended interval [ 0 , ∞ ] [0,\infty] [ 0 , ∞ ] . If t θ ℓ → λ < ∞ t\theta_\ell\to\lambda<\infty t θ ℓ → λ < ∞ , then (276) gives
lim sup R n , ℓ ( p ^ β , θ ℓ ) = ( β + Δ β , n ( λ ) ) log A n (283) \limsup R_{n,\ell}(\hat p_\beta,\theta_\ell)=(\beta+\Delta_{\beta,n}(\lambda))\log A_n \tag*{(283)} lim sup R n , ℓ ( p ^ β , θ ℓ ) = ( β + Δ β , n ( λ )) log A n (283)
≤ ( β + sup λ ≥ 0 Δ β , n ) log A n . (284) \leq\left(\beta+\sup_{\lambda\geq0}\Delta_{\beta,n}\right)\log A_n. \tag*{(284)} ≤ ( β + λ ≥ 0 sup Δ β , n ) log A n . (284)
If t θ ℓ → ∞ t\theta_\ell\to\infty t θ ℓ → ∞ , we pass to a further subsequence along which t ( 1 − θ ℓ ) t(1-\theta_\ell) t ( 1 − θ ℓ ) also converges in [ 0 , ∞ ] [0,\infty] [ 0 , ∞ ] . If t ( 1 − θ ℓ ) → λ < ∞ t(1-\theta_\ell)\to\lambda<\infty t ( 1 − θ ℓ ) → λ < ∞ , the symmetric boundary limit gives the same upper bound. If instead both t θ ℓ t\theta_\ell t θ ℓ and t ( 1 − θ ℓ ) t(1-\theta_\ell) t ( 1 − θ ℓ ) diverge, then the large-count limit (280) gives
lim sup R n , ℓ ( p ^ β , θ ℓ ) = 1 2 log A n ≤ ( β + sup λ ≥ 0 Δ β , n ) log A n . (285) \limsup R_{n,\ell}(\hat p_\beta,\theta_\ell)=\frac{1}{2}\log A_n\leq\left(\beta+\sup_{\lambda\geq0}\Delta_{\beta,n}\right)\log A_n. \tag*{(285)} lim sup R n , ℓ ( p ^ β , θ ℓ ) = 2 1 log A n ≤ ( β + λ ≥ 0 sup Δ β , n ) log A n . (285)
because β + Δ β , n ( λ ) → 1 2 \beta+\Delta_{\beta,n}(\lambda)\to\frac{1}{2} β + Δ β , n ( λ ) → 2 1 as λ → ∞ \lambda\to\infty λ → ∞ . Thus every subsequential limsup of almost-maximal regrets is bounded by ( β + sup λ Δ β , n ) log A n (\beta+\sup_\lambda\Delta_{\beta,n})\log A_n ( β + sup λ Δ β , n ) log A n , and hence
lim sup ℓ → ∞ max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ β , θ ) ≤ ( β + sup λ ≥ 0 Δ β , n ) log A n . (286) \limsup_{\ell\to\infty}\max_{\theta\in[0,1]}R_{n,\ell}(\hat p_\beta,\theta)\leq\left(\beta+\sup_{\lambda\geq0}\Delta_{\beta,n}\right)\log A_n. \tag*{(286)} ℓ → ∞ lim sup θ ∈ [ 0 , 1 ] max R n , ℓ ( p ^ β , θ ) ≤ ( β + λ ≥ 0 sup Δ β , n ) log A n . (286)
Together with the lower bound, this proves Eq. (82) and completes the proof of Theorem 8.
Proof of Theorem 9
Throughout this appendix, n n n is fixed and A n = 1 + 1 / n A_n = 1 + 1/n A n = 1 + 1/ n .
Write
h β ( μ ) = log μ − E Z ∼ Pois ( μ ) log ( Z + β ) , μ > 0. (287) h_\beta(\mu) = \log\mu- \mathbb{E}_{Z\sim\operatorname{Pois}(\mu)} \log(Z+\beta), \qquad\mu>0. \tag*{(287)} h β ( μ ) = log μ − E Z ∼ Pois ( μ ) log ( Z + β ) , μ > 0. (287)
Then
Δ β , n ( λ ) = 1 log A n ∫ λ A n λ h β ( μ ) d μ . (288) \Delta_{\beta,n}(\lambda) = \frac{1}{\log A_n}\int_\lambda^{A_n\lambda} h_\beta(\mu)\,d\mu. \tag*{(288)} Δ β , n ( λ ) = log A n 1 ∫ λ A n λ h β ( μ ) d μ . (288)
We first prove a few auxiliary lemmas, in the same spirit as those in the one-step analyses of Krichevsky and Braess–Sauer [20 , 21 ] . Recall the definition of G β ( ⋅ ) G_\beta(\cdot) G β ( ⋅ ) from Equation (203) .
Lemma 24: Let K ⊂ ( 0 , ∞ ) K\subset(0,\infty) K ⊂ ( 0 , ∞ ) be compact. Uniformly over β ∈ K \beta\in K β ∈ K ,
G β ( λ ) = 1 2 + β 2 / 2 − β + 5 / 12 λ + O K ( λ − 3 / 2 ) , λ → ∞ . (289) G_\beta(\lambda)=\frac{1}{2}+\frac{\beta^2/2-\beta+5/12}{\lambda}+O_K(\lambda^{-3/2}),\qquad\lambda\to\infty. \tag*{(289)} G β ( λ ) = 2 1 + λ β 2 /2 − β + 5/12 + O K ( λ − 3/2 ) , λ → ∞. (289) Proof: Let Z ∼ Pois ( λ ) Z\sim\operatorname{Pois}(\lambda) Z ∼ Pois ( λ ) . A fourth-order Taylor expansion of log ( Z + β ) \log(Z+\beta) log ( Z + β ) around Z = λ Z=\lambda Z = λ , the first four centered Poisson moments, and a Chernoff bound on ∣ Z − λ ∣ > λ / 2 \lvert Z-\lambda\rvert>\lambda/2 ∣ Z − λ ∣ > λ /2 give
E log ( Z + β ) = log λ + β − 1 / 2 λ + − β 2 / 2 + β − 5 / 12 λ 2 + O K ( λ − 5 / 2 ) . (290) \mathbb{E}\log(Z+\beta)=\log\lambda+\frac{\beta-1/2}{\lambda}+\frac{-\beta^2/2+\beta-5/12}{\lambda^2}+O_K(\lambda^{-5/2}). \tag*{(290)} E log ( Z + β ) = log λ + λ β − 1/2 + λ 2 − β 2 /2 + β − 5/12 + O K ( λ − 5/2 ) . (290) Substitution into G β G_\beta G β proves the claim. The coefficient agrees with Krichevsky’s one-step expansion [20 Lemma 3] .
We also need the following two lemmas. For the sake of simplifying their proofs, we make use of numerical calculations obtained with Mathematica. Nonetheless, the same results can be proved analytically via Taylor expansion and error bounds.
Lemma 25: For every β ≥ 1 / 2 \beta\ge1/2 β ≥ 1/2 and 0 < μ ≤ 2 0<\mu\le2 0 < μ ≤ 2 , we have
h β ( μ ) < 0. (291) h_\beta(\mu)<0. \tag*{(291)} h β ( μ ) < 0. (291) Proof: Direct numerical evaluation with Mathematica shows that
sup 0 < μ ≤ 2 h 1 / 2 ( μ ) ≈ − 0.0259 < 0. (292) \sup_{0<\mu\le2} h_{1/2}(\mu)\approx-0.0259<0. \tag*{(292)} 0 < μ ≤ 2 sup h 1/2 ( μ ) ≈ − 0.0259 < 0. (292) The lemma follows since h β ( μ ) h_\beta(\mu) h β ( μ ) is decreasing in β \beta β .
Lemma 26 (A Poisson reciprocal-moment bound): If 0 < β ≤ β 0 0<\beta\le\beta_0 0 < β ≤ β 0 and μ ≥ 1 \mu\ge1 μ ≥ 1 , then
q β ( μ ) : = μ E Z ∼ Pois ( μ ) 1 Z + β > 1. (293) q_\beta(\mu):=\mu\mathbb{E}_{Z\sim\operatorname{Pois}(\mu)}\frac{1}{Z+\beta}>1. \tag*{(293)} q β ( μ ) := μ E Z ∼ Pois ( μ ) Z + β 1 > 1. (293) Proof: For fixed μ \mu μ , q β ( μ ) q_\beta(\mu) q β ( μ ) decreases in β \beta β , so it is enough to take β = β 0 \beta=\beta_0 β = β 0 . Numerical evaluation with Mathematica shows that q β 0 ( 1 ) > 1 q_{\beta_0}(1)>1 q β 0 ( 1 ) > 1 . By standard Poisson differentiation, we have
q β ′ ( μ ) = 1 − q β ( μ ) + 1 − β μ q β ( μ ) . (294) q_\beta'(\mu)=1-q_\beta(\mu)+\frac{1-\beta}{\mu}q_\beta(\mu). \tag*{(294)} q β ′ ( μ ) = 1 − q β ( μ ) + μ 1 − β q β ( μ ) . (294) At a point where q β 0 ( μ ) = 1 q_{\beta_0}(\mu)=1 q β 0 ( μ ) = 1 , the right-hand side equals ( 1 − β 0 ) / μ > 0 (1-\beta_0)/\mu>0 ( 1 − β 0 ) / μ > 0 . Thus q β 0 q_{\beta_0} q β 0 cannot cross the value one downward on [ 1 , ∞ ) [1,\infty) [ 1 , ∞ ) , and monotonicity in β \beta β completes the proof.
Lemma 27: For every 1 / 2 < β < β 0 1/2<\beta<\beta_0 1/2 < β < β 0 , there exist unique numbers 0 < λ − ( β ) < λ + ( β ) < ∞ 0<\lambda_-(\beta)<\lambda_+(\beta)<\infty 0 < λ − ( β ) < λ + ( β ) < ∞ such that
G β ( λ ) > β ⟺ λ ∈ ( λ − ( β ) , λ + ( β ) ) . (295) G_\beta(\lambda)>\beta\quad\Longleftrightarrow\quad\lambda\in(\lambda_-(\beta),\lambda_+(\beta)). \tag*{(295)} G β ( λ ) > β ⟺ λ ∈ ( λ − ( β ) , λ + ( β )) . (295)
Proof: By Frullani’s identity and the Poisson probability-generating function, for λ > 0 \lambda>0 λ > 0 ,
h β ( λ ) = ∫ 0 ∞ e − β t − λ ( 1 − e − t ) − e − λ t t d t = ∫ 0 ∞ e − λ x w β ( x ) d x , (296) h_\beta(\lambda)=\int_0^\infty\frac{e^{-\beta t-\lambda(1-e^{-t})}-e^{-\lambda t}}{t}\,dt
=\int_0^\infty e^{-\lambda x}w_\beta(x)\,dx, \tag*{(296)} h β ( λ ) = ∫ 0 ∞ t e − β t − λ ( 1 − e − t ) − e − λ t d t = ∫ 0 ∞ e − λ x w β ( x ) d x , (296)
where
w β ( x ) = { ( 1 − x ) β − 1 − log ( 1 − x ) − 1 x , 0 < x < 1 , − 1 x , x > 1. (297) w_\beta(x)=
\begin{cases}
\dfrac{(1-x)^{\beta-1}}{-\log(1-x)}-\dfrac{1}{x}, & 0<x<1,\\
-\dfrac{1}{x}, & x>1.
\end{cases} \tag*{(297)} w β ( x ) = ⎩ ⎨ ⎧ − log ( 1 − x ) ( 1 − x ) β − 1 − x 1 , − x 1 , 0 < x < 1 , x > 1. (297)
The integral is absolutely convergent. For 0 < x < 1 0<x<1 0 < x < 1 , put t = − log ( 1 − x ) t=-\log(1-x) t = − log ( 1 − x ) . Then w β ( x ) > 0 w_\beta(x)>0 w β ( x ) > 0 is equivalent to
ϕ β ( t ) : = e ( 1 − β ) t − e − β t − t > 0. (298) \phi_\beta(t):=e^{(1-\beta)t}-e^{-\beta t}-t>0. \tag*{(298)} ϕ β ( t ) := e ( 1 − β ) t − e − β t − t > 0. (298)
Now ϕ β ( 0 ) = ϕ β ′ ( 0 ) = 0 \phi_\beta(0)=\phi_\beta'(0)=0 ϕ β ( 0 ) = ϕ β ′ ( 0 ) = 0 , while
ϕ β ′ ′ ( t ) = ( 1 − β ) 2 e ( 1 − β ) t − β 2 e − β t (299) \phi_\beta''(t)=(1-\beta)^2e^{(1-\beta)t}-\beta^2e^{-\beta t} \tag*{(299)} ϕ β ′′ ( t ) = ( 1 − β ) 2 e ( 1 − β ) t − β 2 e − β t (299)
has exactly one positive zero when 1 / 2 < β < 1 1/2<\beta<1 1/2 < β < 1 . Hence ϕ β ′ \phi_\beta' ϕ β ′ first decreases and then increases, and ϕ β \phi_\beta ϕ β first decreases below zero and then increases to infinity, i.e., it has exactly one positive zero. Consequently w β w_\beta w β has the sign pattern − , + , − -,+,- − , + , − , and therefore exactly two sign changes on ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) . Therefore, by [44 Theorem 4] , — which holds even though w β ( x ) w_\beta(x) w β ( x ) has a singularity in x = 1 x=1 x = 1 , — h β h_\beta h β has at most two positive zeros.
Finally, h β ( λ ) → − ∞ h_\beta(\lambda)\to-\infty h β ( λ ) → − ∞ as λ ↓ 0 \lambda\downarrow0 λ ↓ 0 , and
λ h β ( λ ) = G β ( λ ) − β ⟶ 1 2 − β < 0 (300) \lambda h_\beta(\lambda)=G_\beta(\lambda)-\beta\longrightarrow\frac{1}{2}-\beta<0 \tag*{(300)} λ h β ( λ ) = G β ( λ ) − β ⟶ 2 1 − β < 0 (300)
as λ → ∞ \lambda\to\infty λ → ∞ . Since β < β 0 \beta<\beta_0 β < β 0 , the definition of β 0 \beta_0 β 0 implies that h β h_\beta h β is positive somewhere. It must therefore have exactly two positive zeros, and its positive set is the single interval asserted in Eq. (295) .
:::
Lemma 28: Let f : R → R f:\mathbb{R}\to\mathbb{R} f : R → R be continuous, with { f > 0 } \{f>0\} { f > 0 } a nonempty bounded interval and f < 0 f<0 f < 0 outside its closure. If
∫ x x + L f ( y ) d y = 0 (301) \int_x^{x+L} f(y)\,dy=0 \tag*{(301)} ∫ x x + L f ( y ) d y = 0 (301) for some L > 0 L>0 L > 0 , then, for every 0 < l < L 0<l<L 0 < l < L , there is a z ∈ R z\in\mathbb{R} z ∈ R such that
∫ z z + l f ( y ) d y > 0. (302) \int_z^{z+l}f(y)\,dy>0. \tag*{(302)} ∫ z z + l f ( y ) d y > 0. (302) Proof: Let I = [ x , x + L ] I=[x,x+L] I = [ x , x + L ] and P = I ∩ { f > 0 } P=I\cap\{f>0\} P = I ∩ { f > 0 } ; the latter is a nonempty interval. If l ≤ ∣ P ∣ l\le\lvert P\rvert l ≤ ∣ P ∣ , take a length-l l l interval inside the closure of P P P . If l > ∣ P ∣ l>\lvert P\rvert l > ∣ P ∣ , take a length-l l l interval J ⊂ I J\subset I J ⊂ I containing P P P . Then I ∖ J I\setminus J I ∖ J has positive length and lies where f < 0 f<0 f < 0 , so
∫ J f = ∫ I f − ∫ I ∖ J f > 0. (303) \int_J f=\int_I f-\int_{I\setminus J}f>0. \tag*{(303)} ∫ J f = ∫ I f − ∫ I ∖ J f > 0. (303)
With the help of the previous lemmas, we can now prove the main result.
Proof of Theorem 9: For every fixed λ > 0 \lambda>0 λ > 0 , differentiation under the Poisson expectation gives
∂ ∂ β Δ β , n ( λ ) = − 1 log A n ∫ λ A n λ E Z μ 1 Z μ + β d μ < 0. (304) \frac{\partial}{\partial\beta}\Delta_{\beta,n}(\lambda)
=-\frac{1}{\log A_n}\int_\lambda^{A_n\lambda}\mathbb{E}_{Z_\mu}\frac{1}{Z_\mu+\beta}\,d\mu<0. \tag*{(304)} ∂ β ∂ Δ β , n ( λ ) = − log A n 1 ∫ λ A n λ E Z μ Z μ + β 1 d μ < 0. (304)
Thus the admissible set in Eq. (87) is an upper interval. It is nonempty because β 0 \beta_0 β 0 is admissible: by Eq. (73) ,
h β 0 ( μ ) = Δ β 0 ( μ ) μ ≤ 0 , (305) h_{\beta_0}(\mu)=\frac{\Delta_{\beta_0}(\mu)}{\mu}\le0, \tag*{(305)} h β 0 ( μ ) = μ Δ β 0 ( μ ) ≤ 0 , (305)
so Eq. (288) gives Δ β 0 , n ( λ ) ≤ 0 \Delta_{\beta_0,n}(\lambda)\le0 Δ β 0 , n ( λ ) ≤ 0 . Hence β n ⋆ ≤ β 0 \beta_n^\star\le\beta_0 β n ⋆ ≤ β 0 , and continuity in β \beta β shows that β n ⋆ \beta_n^\star β n ⋆ itself is admissible.
At β = 1 / 2 \beta=1/2 β = 1/2 , Lemma 24 gives
Δ 1 / 2 , n ( λ ) = 1 log A n ∫ 1 A n G 1 / 2 ( u λ ) − 1 / 2 u d u = 1 − A n − 1 24 λ log A n + O ( λ − 3 / 2 ) , (306) \Delta_{1/2,n}(\lambda)=\frac{1}{\log A_n}\int_1^{A_n}\frac{G_{1/2}(u\lambda)-1/2}{u}\,du
=\frac{1-A_n^{-1}}{24\lambda\log A_n}+O(\lambda^{-3/2}), \tag*{(306)} Δ 1/2 , n ( λ ) = log A n 1 ∫ 1 A n u G 1/2 ( u λ ) − 1/2 d u = 24 λ log A n 1 − A n − 1 + O ( λ − 3/2 ) , (306)
which is positive for all sufficiently large λ \lambda λ . Thus β n ⋆ > 1 / 2 \beta_n^\star>1/2 β n ⋆ > 1/2 . Choose β j ↑ β n ⋆ \beta_j\uparrow\beta_n^\star β j ↑ β n ⋆ with β j < β n ⋆ \beta_j<\beta_n^\star β j < β n ⋆ and λ j > 0 \lambda_j>0 λ j > 0 such that Δ β j , n ( λ j ) > 0 \Delta_{\beta_j,n}(\lambda_j)>0 Δ β j , n ( λ j ) > 0 . For large j j j , β j ≥ 1 / 2 \beta_j\ge1/2 β j ≥ 1/2 . If λ j ≤ 1 \lambda_j\le1 λ j ≤ 1 , then [ λ j , A n λ j ] ⊂ ( 0 , 2 ] [\lambda_j,A_n\lambda_j]\subset(0,2] [ λ j , A n λ j ] ⊂ ( 0 , 2 ] , contradicting Lemma 25 ; hence λ j > 1 \lambda_j>1 λ j > 1 . The sequence is bounded above because, uniformly for β ∈ [ 1 / 2 + η , β 0 ] \beta\in[1/2+\eta,\beta_0] β ∈ [ 1/2 + η , β 0 ] ,
Δ β , n ( λ ) = 1 2 − β + O ( λ − 1 ) , (307) \Delta_{\beta,n}(\lambda)=\frac{1}{2}-\beta+O(\lambda^{-1}), \tag*{(307)} Δ β , n ( λ ) = 2 1 − β + O ( λ − 1 ) , (307)
and therefore Δ β j , n ( λ ) ≤ − η + O ( λ − 1 ) \Delta_{\beta_j,n}(\lambda)\le-\eta+O(\lambda^{-1}) Δ β j , n ( λ ) ≤ − η + O ( λ − 1 ) . Passing to a subsequence gives λ j → λ n ⋆ ∈ [ 1 , ∞ ) \lambda_j\to\lambda_n^\star\in[1,\infty) λ j → λ n ⋆ ∈ [ 1 , ∞ ) . Continuity and admissibility imply
Δ β n ⋆ , n ( λ n ⋆ ) = 0 , (308) \Delta_{\beta_n^\star,n}(\lambda_n^\star)=0, \tag*{(308)} Δ β n ⋆ , n ( λ n ⋆ ) = 0 , (308)
and Lemma 25 excludes λ n ⋆ = 1 \lambda_n^\star=1 λ n ⋆ = 1 . This proves the first part of the theorem.
Next, since admissibility excludes larger values, we have
β n ⋆ + sup λ ≥ 0 Δ β n ⋆ , n = β n ⋆ . (309) \beta_n^\star+\sup_{\lambda\ge0}\Delta_{\beta_n^\star,n}=\beta_n^\star. \tag*{(309)} β n ⋆ + λ ≥ 0 sup Δ β n ⋆ , n = β n ⋆ . (309)
Let G β , n ( λ ) = β + Δ β , n G_{\beta,n}(\lambda)=\beta+\Delta_{\beta,n} G β , n ( λ ) = β + Δ β , n . For fixed λ > 0 \lambda>0 λ > 0 ,
∂ ∂ β G β , n ( λ ) = 1 log A n ∫ 1 A n 1 − q β ( u λ ) u d u . (310) \frac{\partial}{\partial\beta}G_{\beta,n}(\lambda)=\frac{1}{\log A_n}\int_1^{A_n}\frac{1-q_\beta(u\lambda)}{u}\,du. \tag*{(310)} ∂ β ∂ G β , n ( λ ) = log A n 1 ∫ 1 A n u 1 − q β ( u λ ) d u . (310)
At λ = λ n ⋆ \lambda=\lambda_n^\star λ = λ n ⋆ , Lemma 26 makes this derivative negative for every 0 < β ≤ β n ⋆ 0<\beta\le\beta_n^\star 0 < β ≤ β n ⋆ . Therefore, if β < β n ⋆ \beta<\beta_n^\star β < β n ⋆ ,
G β , n ( λ n ⋆ ) > G β n ⋆ , n ( λ n ⋆ ) = β n ⋆ , (311) G_{\beta,n}(\lambda_n^\star)>G_{\beta_n^\star,n}(\lambda_n^\star)=\beta_n^\star, \tag*{(311)} G β , n ( λ n ⋆ ) > G β n ⋆ , n ( λ n ⋆ ) = β n ⋆ , (311)
so β + sup λ Δ β , n > β n ⋆ \beta+\sup_\lambda\Delta_{\beta,n}>\beta_n^\star β + sup λ Δ β , n > β n ⋆ . If β > β n ⋆ \beta>\beta_n^\star β > β n ⋆ , then
β + sup λ Δ β , n ≥ G β , n ( 0 ) = β > β n ⋆ . (312) \beta+\sup_\lambda\Delta_{\beta,n}\ge G_{\beta,n}(0)=\beta>\beta_n^\star. \tag*{(312)} β + λ sup Δ β , n ≥ G β , n ( 0 ) = β > β n ⋆ . (312)
This proves the unique optimum. Equation (304) also shows that β + sup λ Δ β , n ( λ ) = β \beta+\sup_\lambda\Delta_{\beta,n}(\lambda)=\beta β + sup λ Δ β , n ( λ ) = β for every β ≥ β n ⋆ \beta\ge\beta_n^\star β ≥ β n ⋆ , and the value at λ n ⋆ \lambda_n^\star λ n ⋆ gives C β n ⋆ , n = β n ⋆ C_{\beta_n^\star,n}=\beta_n^\star C β n ⋆ , n = β n ⋆ .
It remains to compare different values of n n n . Let L n = log A n L_n=\log A_n L n = log A n . First, the inequality β n ⋆ < β 0 \beta_n^\star<\beta_0 β n ⋆ < β 0 is strict, otherwise Eq. (308) would give an interval on which h β 0 h_{\beta_0} h β 0 has zero integral, which is impossible since h β 0 h_{\beta_0} h β 0 is non-positive, continuous, and not identically zero on that interval. For x ∈ R x\in\mathbb{R} x ∈ R , define
f β ( x ) = G β ( e x ) − β = e x h β ( e x ) . (313) f_\beta(x)=G_\beta(e^x)-\beta=e^xh_\beta(e^x). \tag*{(313)} f β ( x ) = G β ( e x ) − β = e x h β ( e x ) . (313)
Changing variables in Eq. (288) yields
Δ β , n ( e x ) = 1 L n ∫ x x + L n f β ( y ) d y . (314) \Delta_{\beta,n}(e^x)=\frac{1}{L_n}\int_x^{x+L_n}f_\beta(y)\,dy. \tag*{(314)} Δ β , n ( e x ) = L n 1 ∫ x x + L n f β ( y ) d y . (314)
At β = β n ⋆ \beta=\beta_n^\star β = β n ⋆ and x n ⋆ = log λ n ⋆ x_n^\star=\log\lambda_n^\star x n ⋆ = log λ n ⋆ , the integral in Eq. (314) is zero. Since 1 / 2 < β n ⋆ < β 0 1/2<\beta_n^\star<\beta_0 1/2 < β n ⋆ < β 0 , Lemma 27 shows that the positive set of f β n ⋆ f_{\beta_n^\star} f β n ⋆ is one bounded interval. Because L n + 1 < L n L_{n+1}<L_n L n + 1 < L n , Lemma 28 produces a window of length L n + 1 L_{n+1} L n + 1 with positive integral. Hence β n ⋆ \beta_n^\star β n ⋆ is not admissible for the ( n + 1 ) (n+1) ( n + 1 ) -length case. Continuity and the strict decrease in Eq. (304) then give
β n + 1 ⋆ > β n ⋆ . (315) \beta_{n+1}^\star>\beta_n^\star. \tag*{(315)} β n + 1 ⋆ > β n ⋆ . (315)
The increasing sequence is bounded above by β 0 \beta_0 β 0 . To identify its limit, fix any β < β 0 \beta<\beta_0 β < β 0 and choose λ \lambda λ with G β ( λ ) > β G_\beta(\lambda)>\beta G β ( λ ) > β . Since L n ↓ 0 L_n\downarrow0 L n ↓ 0 , Eq. (81) gives
β + Δ β , n ( λ ) ⟶ G β ( λ ) > β . (316) \beta+\Delta_{\beta,n}(\lambda)\longrightarrow G_\beta(\lambda)>\beta. \tag*{(316)} β + Δ β , n ( λ ) ⟶ G β ( λ ) > β . (316)
Thus β \beta β is inadmissible for all sufficiently large n n n , so lim inf n β n ⋆ ≥ β 0 \liminf_n\beta_n^\star\ge\beta_0 lim inf n β n ⋆ ≥ β 0 . This proves Eq. (90) .
Proof of Theorem 10
The proof applies the conditional regret-capacity theorem with a beta prior. Fix constants a , b > 1 a,b>1 a , b > 1 and let Θ ∼ Beta ( a , b ) \Theta\sim\operatorname{Beta}(a,b) Θ ∼ Beta ( a , b ) . Conditionally on Θ = θ \Theta=\theta Θ = θ , let Z 1 , Z 2 , … Z_1,Z_2,\ldots Z 1 , Z 2 , … be i.i.d. Bernoulli ( θ ) \operatorname{Bernoulli}(\theta) Bernoulli ( θ ) . The n n n training sequences of length ℓ \ell ℓ can be flattened into Z t = ( Z 1 , … , Z t ) Z^t=(Z_1,\ldots,Z_t) Z t = ( Z 1 , … , Z t ) , where t = n ℓ t=n\ell t = n ℓ , and the test sequence can be identified with Z t + 1 t + ℓ Z_{t+1}^{t+\ell} Z t + 1 t + ℓ .
By Theorem 2, for this prior,
min p ^ max θ ∈ [ 0 , 1 ] R n , ℓ ( p ^ , θ ) ≥ I ( Θ ; Z t + 1 t + ℓ ∣ Z t ) (317) \min_{\hat p}\max_{\theta\in[0,1]}R_{n,\ell}(\hat p,\theta)\ge I(\Theta;Z_{t+1}^{t+\ell}\mid Z^t) \tag*{(317)} p ^ min θ ∈ [ 0 , 1 ] max R n , ℓ ( p ^ , θ ) ≥ I ( Θ ; Z t + 1 t + ℓ ∣ Z t ) (317)
= ∑ m = t t + ℓ − 1 I ( Θ ; Z m + 1 ∣ Z m ) , (318) =\sum_{m=t}^{t+\ell-1}I(\Theta;Z_{m+1}\mid Z^m), \tag*{(318)} = m = t ∑ t + ℓ − 1 I ( Θ ; Z m + 1 ∣ Z m ) , (318)
where the equality is the chain rule for mutual information. We now evaluate the one-step terms.
Lemma 29. Let Θ ∼ Beta ( a , b ) \Theta\sim\operatorname{Beta}(a,b) Θ ∼ Beta ( a , b ) with a , b > 1 a,b>1 a , b > 1 , and let Z 1 , Z 2 , … Z_1,Z_2,\ldots Z 1 , Z 2 , … be conditionally i.i.d. Bernoulli ( Θ ) \operatorname{Bernoulli}(\Theta) Bernoulli ( Θ ) . Then
I ( Θ ; Z m + 1 ∣ Z m ) = 1 2 m + O a , b ( 1 m 2 ) . (319) I(\Theta;Z_{m+1}\mid Z^m)=\frac{1}{2m}+O_{a,b}\left(\frac{1}{m^2}\right). \tag*{(319)} I ( Θ ; Z m + 1 ∣ Z m ) = 2 m 1 + O a , b ( m 2 1 ) . (319)
Proof. Let S m = ∑ i = 1 m Z i S_m=\sum_{i=1}^m Z_i S m = ∑ i = 1 m Z i . Since S m S_m S m is a sufficient statistic for Θ \Theta Θ given Z m Z^m Z m , it is enough to condition on S m S_m S m . Given S m = s S_m=s S m = s , the posterior is
Θ ∣ S m = s ∼ Beta ( A , B ) , A = a + s , B = b + m − s , (320) \Theta\mid S_m=s\sim\operatorname{Beta}(A,B),\qquad A=a+s,\quad B=b+m-s, \tag*{(320)} Θ ∣ S m = s ∼ Beta ( A , B ) , A = a + s , B = b + m − s , (320)
and we write N = A + B = m + a + b N=A+B=m+a+b N = A + B = m + a + b . Let h ( u ) = − u log u − ( 1 − u ) log ( 1 − u ) h(u)=-u\log u-(1-u)\log(1-u) h ( u ) = − u log u − ( 1 − u ) log ( 1 − u ) . The predictive probability of Z m + 1 = 1 Z_{m+1}=1 Z m + 1 = 1 is A / N A/N A / N , and therefore
I ( Θ ; Z m + 1 ∣ S m = s ) = h ( A N ) − E [ h ( Θ ) ∣ S m = s ] . (321) I(\Theta;Z_{m+1}\mid S_m=s)=h\left(\frac{A}{N}\right)-\mathbb{E}[h(\Theta)\mid S_m=s]. \tag*{(321)} I ( Θ ; Z m + 1 ∣ S m = s ) = h ( N A ) − E [ h ( Θ ) ∣ S m = s ] . (321)
Writing E s [ ⋅ ] = E [ ⋅ ∣ S m = s ] \mathbb{E}_s[\cdot]=\mathbb{E}[\cdot\mid S_m=s] E s [ ⋅ ] = E [ ⋅ ∣ S m = s ] , the beta identities give
E s [ Θ log Θ ] = A N { ψ ( A + 1 ) − ψ ( N + 1 ) } , (322) \mathbb{E}_s[\Theta\log\Theta]=\frac{A}{N}\{\psi(A+1)-\psi(N+1)\}, \tag*{(322)} E s [ Θ log Θ ] = N A { ψ ( A + 1 ) − ψ ( N + 1 )} , (322)
E s [ ( 1 − Θ ) log ( 1 − Θ ) ] = B N { ψ ( B + 1 ) − ψ ( N + 1 ) } . (323) \mathbb{E}_s[(1-\Theta)\log(1-\Theta)]=\frac{B}{N}\{\psi(B+1)-\psi(N+1)\}. \tag*{(323)} E s [( 1 − Θ ) log ( 1 − Θ )] = N B { ψ ( B + 1 ) − ψ ( N + 1 )} . (323)
where ψ \psi ψ is the digamma function. Then, we get
I ( Θ ; Z m + 1 ∣ S m = s ) = A N { ψ ( A + 1 ) − log A } (324) I(\Theta; Z_{m+1} \mid S_m=s)=\frac{A}{N}\{\psi(A+1)-\log A\} \tag*{(324)} I ( Θ ; Z m + 1 ∣ S m = s ) = N A { ψ ( A + 1 ) − log A } (324)
+ B N { ψ ( B + 1 ) − log B } (325) +\frac{B}{N}\{\psi(B+1)-\log B\} \tag*{(325)} + N B { ψ ( B + 1 ) − log B } (325)
+ log N − ψ ( N + 1 ) . (326) +\log N-\psi(N+1). \tag*{(326)} + log N − ψ ( N + 1 ) . (326)
Since A ≥ a A\ge a A ≥ a , B ≥ b B\ge b B ≥ b , and N = m + a + b N=m+a+b N = m + a + b , using the standard expansion
ψ ( x + 1 ) = log x + 1 2 x + O ( 1 x 2 ) , (327) \psi(x+1)=\log x+\frac{1}{2x}+O\left(\frac{1}{x^2}\right), \tag*{(327)} ψ ( x + 1 ) = log x + 2 x 1 + O ( x 2 1 ) , (327)
which holds uniformly for x x x bounded away from zero, and Eq. (326) , we obtain
I ( Θ ; Z m + 1 ∣ S m = s ) = 1 2 N + O a , b ( 1 N 2 + 1 N A + 1 N B ) . (328) I(\Theta; Z_{m+1}\mid S_m=s)=\frac{1}{2N}+O_{a,b}\left(\frac{1}{N^2}+\frac{1}{NA}+\frac{1}{NB}\right). \tag*{(328)} I ( Θ ; Z m + 1 ∣ S m = s ) = 2 N 1 + O a , b ( N 2 1 + N A 1 + N B 1 ) . (328)
It remains to average the last two reciprocal terms. Under the beta-binomial law of S m S_m S m ,
P ( S m = s ) = ( m s ) B ( a + s , b + m − s ) B ( a , b ) . (329) \mathbb{P}(S_m=s)=\binom{m}{s}\frac{B(a+s,b+m-s)}{B(a,b)}. \tag*{(329)} P ( S m = s ) = ( s m ) B ( a , b ) B ( a + s , b + m − s ) . (329)
For a > 1 a>1 a > 1 , using
B ( a + s , b + m − s ) = a + s − 1 m + a + b − 1 B ( a + s − 1 , b + m − s ) , (330) B(a+s,b+m-s)=\frac{a+s-1}{m+a+b-1}B(a+s-1,b+m-s), \tag*{(330)} B ( a + s , b + m − s ) = m + a + b − 1 a + s − 1 B ( a + s − 1 , b + m − s ) , (330)
we get
E [ 1 a + S m ] ≤ 1 ( m + a + b − 1 ) B ( a , b ) (331) \mathbb{E}\left[\frac{1}{a+S_m}\right]\leq\frac{1}{(m+a+b-1)B(a,b)} \tag*{(331)} E [ a + S m 1 ] ≤ ( m + a + b − 1 ) B ( a , b ) 1 (331)
⋅ ∑ s = 0 m ( m s ) B ( a + s − 1 , b + m − s ) (332) \cdot\sum_{s=0}^{m}\binom{m}{s}B(a+s-1,b+m-s) \tag*{(332)} ⋅ s = 0 ∑ m ( s m ) B ( a + s − 1 , b + m − s ) (332)
= B ( a − 1 , b ) ( m + a + b − 1 ) B ( a , b ) = O a , b ( 1 m ) . (333) =\frac{B(a-1,b)}{(m+a+b-1)B(a,b)}=O_{a,b}\left(\frac{1}{m}\right). \tag*{(333)} = ( m + a + b − 1 ) B ( a , b ) B ( a − 1 , b ) = O a , b ( m 1 ) . (333)
The same argument, with the roles of a a a and b b b exchanged, gives
E [ 1 b + m − S m ] = O a , b ( 1 m ) . (334) \mathbb{E}\left[\frac{1}{b+m-S_m}\right]=O_{a,b}\left(\frac{1}{m}\right). \tag*{(334)} E [ b + m − S m 1 ] = O a , b ( m 1 ) . (334)
Taking expectations in Eq. (328) therefore yields
I ( Θ ; Z m + 1 ∣ Z m ) = 1 2 ( m + a + b ) + O a , b ( 1 m 2 ) (335) I(\Theta; Z_{m+1}\mid Z^m)=\frac{1}{2(m+a+b)}+O_{a,b}\left(\frac{1}{m^2}\right) \tag*{(335)} I ( Θ ; Z m + 1 ∣ Z m ) = 2 ( m + a + b ) 1 + O a , b ( m 2 1 ) (335)
= 1 2 m + O a , b ( 1 m 2 ) , (336) =\frac{1}{2m}+O_{a,b}\left(\frac{1}{m^2}\right), \tag*{(336)} = 2 m 1 + O a , b ( m 2 1 ) , (336)
which proves the lemma. ■
Combining (319) with Lemma 29, we get
I ( Θ ; Z t + 1 t + ℓ ∣ Z t ) = 1 2 ∑ m = t t + ℓ − 1 1 m + O ( ∑ m = t t + ℓ − 1 1 m 2 ) . (337) I(\Theta; Z_{t+1}^{t+\ell}\mid Z^t)=\frac{1}{2}\sum_{m=t}^{t+\ell-1}\frac{1}{m}+O\left(\sum_{m=t}^{t+\ell-1}\frac{1}{m^2}\right). \tag*{(337)} I ( Θ ; Z t + 1 t + ℓ ∣ Z t ) = 2 1 m = t ∑ t + ℓ − 1 m 1 + O ( m = t ∑ t + ℓ − 1 m 2 1 ) . (337)
Using the harmonic-sum estimate (200), we have
∑ m = t t + ℓ − 1 1 m = log t + ℓ t + O ( ∑ m = t t + ℓ − 1 1 m 2 ) (338) \sum_{m=t}^{t+\ell-1}\frac{1}{m}=\log\frac{t+\ell}{t}+O\left(\sum_{m=t}^{t+\ell-1}\frac{1}{m^2}\right) \tag*{(338)} m = t ∑ t + ℓ − 1 m 1 = log t t + ℓ + O ( m = t ∑ t + ℓ − 1 m 2 1 ) (338)
and
∑ m = t t + ℓ − 1 1 m 2 = O ( ℓ t 2 ) = O ( 1 n 2 ℓ ) . (339) \sum_{m=t}^{t+\ell-1}\frac{1}{m^2}=O\left(\frac{\ell}{t^2}\right)=O\left(\frac{1}{n^2\ell}\right). \tag*{(339)} m = t ∑ t + ℓ − 1 m 2 1 = O ( t 2 ℓ ) = O ( n 2 ℓ 1 ) . (339)
Since t = n ℓ t=n\ell t = n ℓ , this gives
I ( Θ ; Z t + 1 t + ℓ ∣ Z t ) = 1 2 log ( 1 + 1 n ) + O ( 1 n 2 ℓ ) . (340) I(\Theta; Z_{t+1}^{t+\ell}\mid Z^t)=\frac{1}{2}\log\left(1+\frac{1}{n}\right)+O\left(\frac{1}{n^2\ell}\right). \tag*{(340)} I ( Θ ; Z t + 1 t + ℓ ∣ Z t ) = 2 1 log ( 1 + n 1 ) + O ( n 2 ℓ 1 ) . (340)
The lower bound in Eq. (93) follows from Theorem 2.
Proof of Theorem 14
We prove an upper bound on the batch regret of the non-updating predictor in Eq. (129) . Following the idea of Hao et al. [18 ] , the main concentration result that we need is Paulin’s McDiarmid-type inequality for Markov chains [19 Corollary 2.11] . Fix a source θ = ( ν , p , q ) ∈ M δ \theta=(\nu,p,q)\in\mathcal{M}_{\delta} θ = ( ν , p , q ) ∈ M δ . For h ∈ { 0 , 1 } h\in\{0,1\} h ∈ { 0 , 1 } , write
r h ≜ P p , q ( 1 ∣ h ) ∈ [ δ , 1 − δ ] (341) r_h\triangleq P_{p,q}(1\mid h)\in[\delta,1-\delta] \tag*{(341)} r h ≜ P p , q ( 1 ∣ h ) ∈ [ δ , 1 − δ ] (341)
and define the expected number of departures from state h h h in one length-ℓ \ell ℓ sequence by
a h ≜ ∑ s = 1 ℓ − 1 P θ ( X s = h ) , N h ≜ n a h . (342) a_h\triangleq\sum_{s=1}^{\ell-1}P_{\theta}(X_s=h),\qquad N_h\triangleq na_h. \tag*{(342)} a h ≜ s = 1 ∑ ℓ − 1 P θ ( X s = h ) , N h ≜ n a h . (342)
Since every entry of the transition matrix is at least δ \delta δ , for every s ≥ 2 s\ge2 s ≥ 2 ,
P θ ( X s = h ) = ∑ j = 0 1 P θ ( X s − 1 = j ) P p , q ( h ∣ j ) ≥ δ . (343) P_{\theta}(X_s=h)=\sum_{j=0}^{1}P_{\theta}(X_{s-1}=j)P_{p,q}(h\mid j)\ge\delta. \tag*{(343)} P θ ( X s = h ) = j = 0 ∑ 1 P θ ( X s − 1 = j ) P p , q ( h ∣ j ) ≥ δ . (343)
Consequently, for ℓ ≥ 3 \ell\ge3 ℓ ≥ 3 ,
δ ( ℓ − 2 ) ≤ a h ≤ ℓ − 1 , δ n ( ℓ − 2 ) ≤ N h ≤ n ( ℓ − 1 ) . (344) \delta(\ell-2)\le a_h\le\ell-1,\qquad\delta n(\ell-2)\le N_h\le n(\ell-1). \tag*{(344)} δ ( ℓ − 2 ) ≤ a h ≤ ℓ − 1 , δ n ( ℓ − 2 ) ≤ N h ≤ n ( ℓ − 1 ) . (344)
In particular, N h → ∞ N_h\to\infty N h → ∞ uniformly over the class whenever n ℓ → ∞ n\ell\to\infty n ℓ → ∞ .
Let n h ( X n ) n_h(X^n) n h ( X n ) and n h 1 ( X n ) n_{h1}(X^n) n h 1 ( X n ) be the (random) training counts corresponding to the random training batch X n X^n X n , and set
r ^ h ≜ n h 1 ( X n ) + 1 / 2 n h ( X n ) + 1 . (345) \widehat r_h\triangleq\frac{n_{h1}(X^n)+1/2}{n_h(X^n)+1}. \tag*{(345)} r h ≜ n h ( X n ) + 1 n h 1 ( X n ) + 1/2 . (345)
The expected test counts satisfy
E n h ( Y ) = a h , E n h 1 ( Y ) = a h r h , (346) \mathbb{E}\,n_h(Y)=a_h,\qquad\mathbb{E}n_{h1}(Y)=a_hr_h, \tag*{(346)} E n h ( Y ) = a h , E n h 1 ( Y ) = a h r h , (346)
where Y Y Y is the random test sequence. Since the training batch and the test sequence are independent, taking the conditional expectation over the test sequence in the log-likelihood ratio gives
R n , ℓ ( P ) ( p ^ K T , θ ) = ∑ h = 0 1 a h E [ d r h ( r ^ h ) ] , (347) R_{n,\ell}^{(P)}(\widehat p_{\mathrm{KT}},\theta)=\sum_{h=0}^{1}a_h\,\mathbb{E}\left[d_{r_h}(\widehat r_h)\right], \tag*{(347)} R n , ℓ ( P ) ( p KT , θ ) = h = 0 ∑ 1 a h E [ d r h ( r h ) ] , (347)
where
d r ( q ) ≜ r log r q + ( 1 − r ) log 1 − r 1 − q . (348) d_r(q)\triangleq r\log\frac{r}{q}+(1-r)\log\frac{1-r}{1-q}. \tag*{(348)} d r ( q ) ≜ r log q r + ( 1 − r ) log 1 − q 1 − r . (348)
It remains to estimate the two row risks uniformly.
Lemma 30. For every ℓ ≥ 3 \ell\ge3 ℓ ≥ 3 , there are constants C δ , C δ ′ > 0 C_\delta,C'_\delta>0 C δ , C δ ′ > 0 such that, for every h ∈ { 0 , 1 } h\in\{0,1\} h ∈ { 0 , 1 } and every u ≥ 0 u\ge0 u ≥ 0 ,
P θ ( ∣ n h ( X n ) − N h ∣ > u ) ≤ C δ exp ( − C δ u 2 n ℓ ) . (349) P_\theta\left(\left|n_h(X^n)-N_h\right|>u\right)\le C_\delta\exp\left(-C_\delta\frac{u^2}{n\ell}\right). \tag*{(349)} P θ ( ∣ n h ( X n ) − N h ∣ > u ) ≤ C δ exp ( − C δ n ℓ u 2 ) . (349) Moreover, for every fixed integer m ≥ 1 m\ge1 m ≥ 1 ,
E ∣ n h ( X n ) − N h ∣ m ≤ C m , δ ( n ℓ ) m / 2 ≤ C m , δ ′ N h m / 2 . (350) \mathbb{E}\left|n_h(X^n)-N_h\right|^m\le C_{m,\delta}(n\ell)^{m/2}\le C'_{m,\delta}N_h^{m/2}. \tag*{(350)} E ∣ n h ( X n ) − N h ∣ m ≤ C m , δ ( n ℓ ) m /2 ≤ C m , δ ′ N h m /2 . (350)
All constants are uniform over the initial distribution and over θ ∈ M δ \theta\in\mathcal{M}_{\delta} θ ∈ M δ .
Proof: For the i i i th training sequence,
n h ( X i ) = ∑ s = 1 ℓ − 1 1 { X i , s = h } . (351) n_h(X_i)=\sum_{s=1}^{\ell-1}1\{X_{i,s}=h\}. \tag*{(351)} n h ( X i ) = s = 1 ∑ ℓ − 1 1 { X i , s = h } . (351)
We apply Paulin’s McDiarmid inequality for Markov chains [19 Corollary 2.11 and Remark 2.12] . Here d T V d_{\mathrm{TV}} d TV denotes total variation distance. For a transition kernel K K K on { 0 , 1 } \{0,1\} { 0 , 1 } , write
α ( K ) ≜ sup x , x ′ ∈ { 0 , 1 } d T V ( K ( x , ⋅ ) , K ( x ′ , ⋅ ) ) . (352) \alpha(K)\triangleq\sup_{x,x'\in\{0,1\}}d_{\mathrm{TV}}\left(K(x,\cdot),K(x',\cdot)\right). \tag*{(352)} α ( K ) ≜ x , x ′ ∈ { 0 , 1 } sup d TV ( K ( x , ⋅ ) , K ( x ′ , ⋅ ) ) . (352)
For the binary matrix in Eq. (110) , α ( P θ ) = ∣ 1 − p − q ∣ ≤ 1 − 2 δ \alpha(P_{\theta})=|1-p-q|\le1-2\delta α ( P θ ) = ∣1 − p − q ∣ ≤ 1 − 2 δ . Hence the Dobrushin contraction inequality gives α ( P θ r ) ≤ ( 1 − 2 δ ) r \alpha(P_{\theta}^{r})\le(1-2\delta)^{r} α ( P θ r ) ≤ ( 1 − 2 δ ) r . If
m δ ≜ max { 1 , ⌈ log 2 − log ( 1 − 2 δ ) ⌉ } , (353) m_{\delta}\triangleq\max\left\{1,\left\lceil\frac{\log2}{-\log(1-2\delta)}\right\rceil\right\}, \tag*{(353)} m δ ≜ max { 1 , ⌈ − log ( 1 − 2 δ ) log 2 ⌉ } , (353)
then, whenever ℓ − 1 ≥ m δ \ell-1\ge m_{\delta} ℓ − 1 ≥ m δ , the mixing quantities in Paulin’s notation satisfy
α ( P θ r ) ≤ ( 1 − 2 δ ) r , τ min , θ ≤ 9 τ θ ( 1 / 2 ) ≤ 9 m δ . (354) \alpha(P_{\theta}^{r})\le(1-2\delta)^{r},\qquad
\tau_{\min,\theta}\le9\tau_{\theta}(1/2)\le9m_{\delta}. \tag*{(354)} α ( P θ r ) ≤ ( 1 − 2 δ ) r , τ m i n , θ ≤ 9 τ θ ( 1/2 ) ≤ 9 m δ . (354)
Moreover, the counting function has bounded-difference constants c s = 1 c_s=1 c s = 1 : for any two sequences x x x and x ~ \widetilde{x} x ,
∣ n h ( x 1 ℓ − 1 ) − n h ( x ~ 1 ℓ − 1 ) ∣ ≤ ∑ s = 1 ℓ − 1 1 { x s ≠ x ~ s } , ∥ c ∥ 2 2 = ℓ − 1. (355) \left|n_h(x_1^{\ell-1})-n_h(\widetilde{x}_1^{\ell-1})\right|
\le\sum_{s=1}^{\ell-1}1\{x_s\ne\widetilde{x}_s\},\qquad\lVert c\rVert_2^2=\ell-1. \tag*{(355)} n h ( x 1 ℓ − 1 ) − n h ( x 1 ℓ − 1 ) ≤ s = 1 ∑ ℓ − 1 1 { x s = x s } , ∥ c ∥ 2 2 = ℓ − 1. (355)
For ℓ − 1 ≥ m δ \ell-1\ge m_{\delta} ℓ − 1 ≥ m δ , [19 Corollary 2.11] gives
P θ ( ∣ n h ( X i ) − E n h ( X i ) ∣ > u ) ≤ 2 exp ( − 2 u 2 ( ℓ − 1 ) τ min , θ ) ≤ 2 exp ( − c δ u 2 ℓ ) . (356) \mathbb{P}_{\theta}\left(|n_h(X_i)-\mathbb{E}\,n_h(X_i)|>u\right)
\le2\exp\left(-\frac{2u^2}{(\ell-1)\tau_{\min,\theta}}\right)
\le2\exp\left(-c_{\delta}\frac{u^2}{\ell}\right). \tag*{(356)} P θ ( ∣ n h ( X i ) − E n h ( X i ) ∣ > u ) ≤ 2 exp ( − ( ℓ − 1 ) τ m i n , θ 2 u 2 ) ≤ 2 exp ( − c δ ℓ u 2 ) . (356)
For smaller values of ℓ \ell ℓ , set L ≜ ℓ − 1 L\triangleq\ell-1 L ≜ ℓ − 1 and suppose that 1 ≤ L < m δ 1\le L<m_{\delta} 1 ≤ L < m δ . Since 0 ≤ n h ( X i ) ≤ L 0\le n_h(X_i)\le L 0 ≤ n h ( X i ) ≤ L , Hoeffding’s lemma gives
E exp { λ ( n h ( X i ) − E n h ( X i ) ) } ≤ exp ( λ 2 L 2 8 ) , λ ∈ R . (357) \mathbb{E}\exp\{\lambda(n_h(X_i)-\mathbb{E}\,n_h(X_i))\}
\le\exp\left(\frac{\lambda^2L^2}{8}\right),\qquad\lambda\in\mathbb{R}. \tag*{(357)} E exp { λ ( n h ( X i ) − E n h ( X i ))} ≤ exp ( 8 λ 2 L 2 ) , λ ∈ R . (357)
Chernoff’s bound, optimized at λ = 4 u / L 2 \lambda=4u/L^2 λ = 4 u / L 2 , therefore yields
P θ ( ∣ n h ( X i ) − E n h ( X i ) ∣ > u ) ≤ 2 exp ( − 2 u 2 L 2 ) . (358) \mathbb{P}_{\theta}\left(|n_h(X_i)-\mathbb{E}\,n_h(X_i)|>u\right)
\le2\exp\left(-\frac{2u^2}{L^2}\right). \tag*{(358)} P θ ( ∣ n h ( X i ) − E n h ( X i ) ∣ > u ) ≤ 2 exp ( − L 2 2 u 2 ) . (358)
Because L < m δ L<m_{\delta} L < m δ and L ≤ ℓ L\le\ell L ≤ ℓ , we have L 2 ≤ m δ ℓ L^2\le m_{\delta}\ell L 2 ≤ m δ ℓ , and hence
P θ ( ∣ n h ( X i ) − E n h ( X i ) ∣ > u ) ≤ 2 exp ( − 2 m δ u 2 ℓ ) . (359) \mathbb{P}_{\theta}\left(|n_h(X_i)-\mathbb{E}\,n_h(X_i)|>u\right)
\le2\exp\left(-\frac{2}{m_{\delta}}\frac{u^2}{\ell}\right). \tag*{(359)} P θ ( ∣ n h ( X i ) − E n h ( X i ) ∣ > u ) ≤ 2 exp ( − m δ 2 ℓ u 2 ) . (359)
After decreasing c δ c_{\delta} c δ if necessary, Eq. (356) therefore holds uniformly for every ℓ ≥ 2 \ell\ge2 ℓ ≥ 2 .
By the standard tail-to-moment-generating-function equivalence for sub-Gaussian variables, after another adjustment of constants, we have
E exp { λ ( n h ( X i ) − E n h ( X i ) ) } ≤ exp ( C δ λ 2 ℓ ) , λ ∈ R . (360) \mathbb{E}\exp\{\lambda(n_h(X_i)-\mathbb{E}\,n_h(X_i))\}
\le\exp(C_{\delta}\lambda^2\ell),\qquad\lambda\in\mathbb{R}. \tag*{(360)} E exp { λ ( n h ( X i ) − E n h ( X i ))} ≤ exp ( C δ λ 2 ℓ ) , λ ∈ R . (360)
Indeed, integrating Eq. (356) gives the quantitative moment growth
{ E ∣ n h ( X i ) − E n h ( X i ) ∣ m } 1 / m ≤ C δ m ℓ , m ≥ 1 , (361) \{\mathbb{E}|n_h(X_i)-\mathbb{E}\,n_h(X_i)|^m\}^{1/m}
\le C_{\delta}\sqrt{m\ell},\qquad m\ge1, \tag*{(361)} { E ∣ n h ( X i ) − E n h ( X i ) ∣ m } 1/ m ≤ C δ m ℓ , m ≥ 1 , (361)
which is equivalent, for a centered random variable, to the bound on the moment generating function above. The n n n training sequences are independent, and
n h ( X n ) − N h = ∑ i = 1 n ( n h ( X i ) − E n h ( X i ) ) , (362) n_h(X^n)-N_h=\sum_{i=1}^{n}(n_h(X_i)-\mathbb{E}\,n_h(X_i)), \tag*{(362)} n h ( X n ) − N h = i = 1 ∑ n ( n h ( X i ) − E n h ( X i )) , (362)
because E n h ( X n ) = n a h = N h \mathbb{E}\,n_h(X^n)=n a_h=N_h E n h ( X n ) = n a h = N h . The standard moment-generating-function bound for a sum of independent centered sub-Gaussian variables therefore proves Eq. (349) . Integrating its tail gives the first inequality in Eq. (350) . The second follows from Eq. (344) , since n ℓ ≤ ( 3 / δ ) N h n\ell\le(3/\delta)N_h n ℓ ≤ ( 3/ δ ) N h for ℓ ≥ 3 \ell\ge3 ℓ ≥ 3 .
For a fixed row h ∈ { 0 , 1 } h\in\{0,1\} h ∈ { 0 , 1 } , define the centered transition count
D h ≜ n h 1 ( X n ) − r h n h ( X n ) . (363) D_h\triangleq n_{h1}(X^n)-r_hn_h(X^n). \tag*{(363)} D h ≜ n h 1 ( X n ) − r h n h ( X n ) . (363)
Then, the following lemma holds.
Lemma 31: For every h ∈ { 0 , 1 } h\in\{0,1\} h ∈ { 0 , 1 } and ℓ ≥ 3 \ell\ge3 ℓ ≥ 3 ,
E D h = 0 , E D h 2 = r h ( 1 − r h ) N h . (364) \mathbb{E}\,D_h=0,\qquad\mathbb{E}\,D_h^2=r_h(1-r_h)N_h. \tag*{(364)} E D h = 0 , E D h 2 = r h ( 1 − r h ) N h . (364) Furthermore, there are constants c δ , C δ > 0 c_{\delta},C_{\delta}>0 c δ , C δ > 0 such that
P θ ( ∣ D h ∣ > u ) ≤ 2 exp ( − c δ u 2 N h ) , u ≥ 0 , (365) \mathbb{P}_{\theta}(|D_h|>u)\le2\exp\left(-c_{\delta}\frac{u^2}{N_h}\right),\qquad u\ge0, \tag*{(365)} P θ ( ∣ D h ∣ > u ) ≤ 2 exp ( − c δ N h u 2 ) , u ≥ 0 , (365) and, for every fixed integer $m\ge 1,
E ∣ D h ∣ m ≤ C m , δ N h m / 2 . (366) \mathbb{E}|D_h|^m\le C_{m,\delta}N_h^{m/2}. \tag*{(366)} E ∣ D h ∣ m ≤ C m , δ N h m /2 . (366) Proof: Write
D h = ∑ i = 1 n ∑ s = 1 ℓ − 1 ξ i , s , (367) D_h=\sum_{i=1}^{n}\sum_{s=1}^{\ell-1}\xi_{i,s}, \tag*{(367)} D h = i = 1 ∑ n s = 1 ∑ ℓ − 1 ξ i , s , (367) ξ i , s ≜ 1 { X i , s = h } × ( 1 { X i , s + 1 = 1 } − r h ) . (368) \xi_{i,s}\triangleq1\{X_{i,s}=h\}\times(1\{X_{i,s+1}=1\}-r_h). \tag*{(368)} ξ i , s ≜ 1 { X i , s = h } × ( 1 { X i , s + 1 = 1 } − r h ) . (368) Order the summands sequence by sequence and let the filtration reveal each sequence sequentially. Immediately before X i , s + 1 X_{i,s+1} X i , s + 1 is revealed,
E [ ξ i , s ∣ F i , s ] = 1 { X i , s = h } ( P p , q ( 1 ∣ h ) − r h ) = 0. (369) \mathbb{E}[\xi_{i,s}\mid\mathcal{F}_{i,s}]
=1\{X_{i,s}=h\}(P_{p,q}(1\mid h)-r_h)=0. \tag*{(369)} E [ ξ i , s ∣ F i , s ] = 1 { X i , s = h } ( P p , q ( 1 ∣ h ) − r h ) = 0. (369) Thus the summands are martingale differences. Cross terms have zero expectation, while
E [ ξ i , s 2 ∣ F i , s ] = r h ( 1 − r h ) 1 { X i , s = h } . (370) \mathbb{E}[\xi_{i,s}^2\mid\mathcal{F}_{i,s}]
=r_h(1-r_h)1\{X_{i,s}=h\}. \tag*{(370)} E [ ξ i , s 2 ∣ F i , s ] = r h ( 1 − r h ) 1 { X i , s = h } . (370) Summation then proves Eq. (364) . Each martingale difference lies in an interval of length one. Azuma–Hoeffding therefore gives
P θ ( ∣ D h ∣ > u ) ≤ 2 exp ( − u 2 2 n ( ℓ − 1 ) ) . (371) \mathbb{P}_{\theta}(|D_h|>u)\le2\exp\left(-\frac{u^2}{2n(\ell-1)}\right). \tag*{(371)} P θ ( ∣ D h ∣ > u ) ≤ 2 exp ( − 2 n ( ℓ − 1 ) u 2 ) . (371) By Eq. (344) , n ( ℓ − 1 ) ≤ ( 2 / δ ) N h n(\ell-1)\le(2/\delta)N_h n ( ℓ − 1 ) ≤ ( 2/ δ ) N h for ℓ ≥ 3 \ell\ge3 ℓ ≥ 3 , which proves Eq. (365) after adjusting the constant. Integrating the tail gives Eq. (366) .
Lemma 32. Uniformly for r h ∈ [ δ , 1 − δ ] r_h \in[\delta,1-\delta] r h ∈ [ δ , 1 − δ ] , for every initial distribution, and whenever N h → ∞ N_h \to\infty N h → ∞ ,
E d r h ( r ^ h ) = 1 2 N h + O δ ( N h − 3 / 2 ) . (372) \mathbb{E}\,d_{r_h}(\widehat r_h)=\frac{1}{2N_h}+O_\delta(N_h^{-3/2}). \tag*{(372)} E d r h ( r h ) = 2 N h 1 + O δ ( N h − 3/2 ) . (372)
Proof. To ease the notation, let us drop the subscript h h h and write r = r h r=r_h r = r h , T = n h ( X n ) T=n_h(X^n) T = n h ( X n ) , D = D h D=D_h D = D h , N = N h N=N_h N = N h , b = 1 / 2 − r b=1/2-r b = 1/2 − r . By Eqs. (345) and (363) ,
U ≜ r ^ h − r = D + b T + 1 . (373) U \triangleq\widehat r_h-r=\frac{D+b}{T+1}. \tag*{(373)} U ≜ r h − r = T + 1 D + b . (373)
Consider the event
G = { ∣ T − N ∣ ≤ N 2 } ∩ { ∣ D ∣ ≤ δ N 8 } . (374) \mathcal{G}=\left\{|T-N|\leq\frac{N}{2}\right\}\cap\left\{|D|\leq\frac{\delta N}{8}\right\}. \tag*{(374)} G = { ∣ T − N ∣ ≤ 2 N } ∩ { ∣ D ∣ ≤ 8 δ N } . (374)
The preceding two lemmas and Eq. (344) imply
P θ ( G c ) ≤ C δ e − c δ N . (375) \mathbb{P}_{\theta}(\mathcal{G}^{c})\leq C_{\delta}e^{-c_{\delta}N}. \tag*{(375)} P θ ( G c ) ≤ C δ e − c δ N . (375)
For all sufficiently large N N N , on G \mathcal{G} G we have T + 1 ≥ N / 2 T+1\geq N/2 T + 1 ≥ N /2 and hence
∣ U ∣ ≤ δ N / 8 + 1 / 2 N / 2 ≤ δ 2 . (376) |U|\leq\frac{\delta N/8+1/2}{N/2}\leq\frac{\delta}{2}. \tag*{(376)} ∣ U ∣ ≤ N /2 δ N /8 + 1/2 ≤ 2 δ . (376)
Thus r + U ∈ [ δ / 2 , 1 − δ / 2 ] r+U\in[\delta/2,1-\delta/2] r + U ∈ [ δ /2 , 1 − δ /2 ] . Taylor’s expansion applied to Eq. (348) , uniformly on this compact interval, gives
d r ( r + U ) = U 2 2 r ( 1 − r ) + O δ ( ∣ U ∣ 3 ) on G . (377) d_r(r+U)=\frac{U^2}{2r(1-r)}+O_\delta(|U|^3)\quad\text{on }\mathcal{G}. \tag*{(377)} d r ( r + U ) = 2 r ( 1 − r ) U 2 + O δ ( ∣ U ∣ 3 ) on G . (377)
We next evaluate the second moment in Eq. (378). On G \mathcal{G} G ,
∣ 1 ( T + 1 ) 2 − 1 N 2 ∣ ≤ C ∣ T − N ∣ + 1 N 3 . (378) \left|\frac{1}{(T+1)^2}-\frac{1}{N^2}\right|\leq C\frac{|T-N|+1}{N^3}. \tag*{(378)} ( T + 1 ) 2 1 − N 2 1 ≤ C N 3 ∣ T − N ∣ + 1 . (378)
By Cauchy–Schwarz and Eqs. (350) and (366) ,
E [ ( D + b ) 2 ( ∣ T − N ∣ + 1 ) ] ≤ { E ( D + b ) 4 } 1 / 2 { E ( ∣ T − N ∣ + 1 ) 2 } 1 / 2 = O δ ( N 3 / 2 ) . (379) \begin{aligned}
\mathbb{E}\left[(D+b)^2(|T-N|+1)\right]
&\leq\left\{\mathbb{E}(D+b)^4\right\}^{1/2}
\left\{\mathbb{E}(|T-N|+1)^2\right\}^{1/2}\\
&=O_\delta(N^{3/2}). \tag*{(379)}
\end{aligned} E [ ( D + b ) 2 ( ∣ T − N ∣ + 1 ) ] ≤ { E ( D + b ) 4 } 1/2 { E ( ∣ T − N ∣ + 1 ) 2 } 1/2 = O δ ( N 3/2 ) . (379)
Moreover, the contribution of G c \mathcal{G}^c G c to any of the polynomial moments below is exponentially small, because ∣ D ∣ ≤ n ( ℓ − 1 ) |D|\leq n(\ell-1) ∣ D ∣ ≤ n ( ℓ − 1 ) , ∣ T − N ∣ ≤ n ( ℓ − 1 ) + N |T-N|\leq n(\ell-1)+N ∣ T − N ∣ ≤ n ( ℓ − 1 ) + N , and Eq. (376) holds. Using Eqs. (364) , (379) and (380), we obtain
E [ U 2 1 G ] = 1 N 2 E ( D + b ) 2 + O δ ( N − 3 / 2 ) = r ( 1 − r ) N + O δ ( N − 3 / 2 ) , (380) \begin{aligned}
\mathbb{E}[U^2\mathbf{1}_{\mathcal{G}}]
&=\frac{1}{N^2}\mathbb{E}(D+b)^2+O_\delta(N^{-3/2})\\
&=\frac{r(1-r)}{N}+O_\delta(N^{-3/2}), \tag*{(380)}
\end{aligned} E [ U 2 1 G ] = N 2 1 E ( D + b ) 2 + O δ ( N − 3/2 ) = N r ( 1 − r ) + O δ ( N − 3/2 ) , (380)
where E D = 0 \mathbb{E}D=0 E D = 0 and the term b 2 / N 2 b^2/N^2 b 2 / N 2 is absorbed into the remainder. Similarly, on G \mathcal{G} G the denominator is at least N / 2 N/2 N /2 , so Eq. (366) gives
E [ ∣ U ∣ 3 1 G ] ≤ C N 3 E ( ∣ D ∣ + 1 ) 3 = O δ ( N − 3 / 2 ) . (381) \mathbb{E}[|U|^3\mathbf{1}_{\mathcal{G}}]\leq\frac{C}{N^3}\mathbb{E}(|D|+1)^3=O_\delta(N^{-3/2}). \tag*{(381)} E [ ∣ U ∣ 3 1 G ] ≤ N 3 C E ( ∣ D ∣ + 1 ) 3 = O δ ( N − 3/2 ) . (381)
Combining Eqs. (378), (381) and (382) yields
E [ d r ( r ^ h ) 1 G ] = 1 2 N + O δ ( N − 3 / 2 ) . (382) \mathbb{E}[d_r(\widehat r_h)\mathbf{1}_{\mathcal{G}}]=\frac{1}{2N}+O_\delta(N^{-3/2}). \tag*{(382)} E [ d r ( r h ) 1 G ] = 2 N 1 + O δ ( N − 3/2 ) . (382)
It remains to control the complement. Since 0 ≤ T ≤ n ( ℓ − 1 ) 0\leq T\leq n(\ell-1) 0 ≤ T ≤ n ( ℓ − 1 ) and 0 ≤ n h 1 ( X n ) ≤ T 0\leq n_{h1}(X^n)\leq T 0 ≤ n h 1 ( X n ) ≤ T ,
1 2 ( n ( ℓ − 1 ) + 1 ) ≤ r ^ h ≤ 1 − 1 2 ( n ( ℓ − 1 ) + 1 ) . (383) \frac{1}{2(n(\ell-1)+1)}\leq\widehat r_h\leq1-\frac{1}{2(n(\ell-1)+1)}. \tag*{(383)} 2 ( n ( ℓ − 1 ) + 1 ) 1 ≤ r h ≤ 1 − 2 ( n ( ℓ − 1 ) + 1 ) 1 . (383)
Therefore
d r ( r ^ h ) ≤ C δ + log ( 2 ( n ( ℓ − 1 ) + 1 ) ) . (384) d_r(\widehat r_h)\leq C_\delta+\log(2(n(\ell-1)+1)). \tag*{(384)} d r ( r h ) ≤ C δ + log ( 2 ( n ( ℓ − 1 ) + 1 )) . (384)
Together with Eq. (376) and the fact that n ℓ = O δ ( N ) n\ell=O_\delta(N) n ℓ = O δ ( N ) , this shows
E [ d r ( r ^ h ) 1 G c ] = O δ ( N − 3 / 2 ) , (385) \mathbb{E}[d_r(\widehat r_h)\mathbf{1}_{\mathcal{G}^c}]=O_\delta(N^{-3/2}), \tag*{(385)} E [ d r ( r h ) 1 G c ] = O δ ( N − 3/2 ) , (385)
and proves Eq. (373).
We now complete the proof of Theorem 14. Substituting Eq. (373) into Eq. (347) and recalling that N h = n a h N_h=na_h N h = n a h gives
R n , ℓ ( P ) ( p ^ K T , θ ) = ∑ h = 0 1 a h ( 1 2 n a h + O δ ( ( n a h ) − 3 / 2 ) ) = 1 n + O δ ( 1 n 3 / 2 ∑ h = 0 1 1 a h ) . (386) \begin{aligned}
R_{n,\ell}^{(P)}(\widehat p_{\mathrm{KT}},\theta)
=\sum_{h=0}^{1}a_h\left(\frac{1}{2na_h}+O_\delta((na_h)^{-3/2})\right) \\
&=\frac{1}{n}+O_\delta\left(\frac{1}{n^{3/2}}\sum_{h=0}^{1}\frac{1}{\sqrt{a_h}}\right). \tag*{(386)}
\end{aligned} R n , ℓ ( P ) ( p KT , θ ) = h = 0 ∑ 1 a h ( 2 n a h 1 + O δ (( n a h ) − 3/2 ) ) = n 1 + O δ ( n 3/2 1 h = 0 ∑ 1 a h 1 ) . (386)
By Eq. (344) ,
∑ h = 0 1 1 a h ≤ 2 δ ( ℓ − 2 ) = O δ ( ℓ − 1 / 2 ) . (387) \sum_{h=0}^{1}\frac{1}{\sqrt{a_h}}\leq\frac{2}{\sqrt{\delta(\ell-2)}}=O_\delta(\ell^{-1/2}). \tag*{(387)} h = 0 ∑ 1 a h 1 ≤ δ ( ℓ − 2 ) 2 = O δ ( ℓ − 1/2 ) . (387)
Consequently,
R n , ℓ ( P ) ( p ^ K T , θ ) = 1 n + O δ ( 1 n 3 / 2 ℓ ) = 1 n + O δ ( 1 n n ℓ ) , (388) R_{n,\ell}^{(P)}(\widehat p_{\mathrm{KT}},\theta)
=\frac{1}{n}+O_\delta\left(\frac{1}{n^{3/2}\sqrt{\ell}}\right)
=\frac{1}{n}+O_\delta\left(\frac{1}{n\sqrt{n\ell}}\right), \tag*{(388)} R n , ℓ ( P ) ( p KT , θ ) = n 1 + O δ ( n 3/2 ℓ 1 ) = n 1 + O δ ( n n ℓ 1 ) , (388)
uniformly over the initial law and over θ ∈ M δ \theta\in\mathcal{M}_\delta θ ∈ M δ . This is Eq. (130) ; the minimax upper bound in Eq. (131) follows because Eq. (129) is an admissible predictor.
Proof of Lemma 15
The proof relies on conjugate transition priors and the mutual-information chain rule along the test sequence. First write
r 0 = p , r 1 = q , T = n ( ℓ − 1 ) , m = n + 1 , (389) r_0=p,\qquad r_1=q,\qquad T=n(\ell-1),\qquad m=n+1, \tag*{(389)} r 0 = p , r 1 = q , T = n ( ℓ − 1 ) , m = n + 1 , (389)
so that r h r_h r h is the probability of switching from state h h h to state 1 − h 1-h 1 − h . Fix δ < δ 0 < 1 / 2 \delta<\delta_0<1/2 δ < δ 0 < 1/2 and set
K 0 = [ δ 0 , 1 − δ 0 ] 2 ⊂ Θ δ . (390) \mathcal{K}_0=[\delta_0,1-\delta_0]^2\subset\Theta_\delta. \tag*{(390)} K 0 = [ δ 0 , 1 − δ 0 ] 2 ⊂ Θ δ . (390)
All lower-bound priors below are ultimately conditioned on K 0 \mathcal{K}_0 K 0 .
Reference priors and exact beta posteriors
Let
a T = A log T , (391) a_T=A\log T, \tag*{(391)} a T = A log T , (391)
where A > 0 A>0 A > 0 will be chosen sufficiently large. For the known-initial-distribution model, and for the transition component of the free-initial-distribution model, use the reference prior
W ~ T K = Beta ( a T , a T ) ⊗ Beta ( a T , a T ) . (392) \widetilde W_T^{\mathrm K}=\operatorname{Beta}(a_T,a_T)\otimes\operatorname{Beta}(a_T,a_T). \tag*{(392)} W T K = Beta ( a T , a T ) ⊗ Beta ( a T , a T ) . (392)
For the stationary-initial-distribution model, set
b T = a T + m 2 . (393) b_T=a_T+\frac{m}{2}. \tag*{(393)} b T = a T + 2 m . (393)
@page 24/28 starts_mid=0 ends_mid=1
Lemma 32. Uniformly for r h ∈ [ δ , 1 − δ ] r_h \in[\delta,1-\delta] r h ∈ [ δ , 1 − δ ] , for every initial distribution, and whenever N h → ∞ N_h \to\infty N h → ∞ ,
E d r h ( r ^ h ) = 1 2 N h + O δ ( N h − 3 / 2 ) . (394) \mathbb{E}\,d_{r_h}(\widehat r_h)=\frac{1}{2N_h}+O_\delta(N_h^{-3/2}). \tag*{(394)} E d r h ( r h ) = 2 N h 1 + O δ ( N h − 3/2 ) . (394)
Proof. To ease the notation, let us drop the subscript h h h and write r = r h r=r_h r = r h , T = n h ( X n ) T=n_h(X^n) T = n h ( X n ) , D = D h D=D_h D = D h , N = N h N=N_h N = N h , b = 1 / 2 − r b=1/2-r b = 1/2 − r . By Eqs. (345) and (363) ,
U ≜ r ^ h − r = D + b T + 1 . (395) U \triangleq\widehat r_h-r=\frac{D+b}{T+1}. \tag*{(395)} U ≜ r h − r = T + 1 D + b . (395)
Consider the event
G = { ∣ T − N ∣ ≤ N 2 } ∩ { ∣ D ∣ ≤ δ N 8 } . (396) \mathcal{G}=\left\{|T-N|\leq\frac{N}{2}\right\}\cap\left\{|D|\leq\frac{\delta N}{8}\right\}. \tag*{(396)} G = { ∣ T − N ∣ ≤ 2 N } ∩ { ∣ D ∣ ≤ 8 δ N } . (396)
The preceding two lemmas and Eq. (344) imply
P θ ( G c ) ≤ C δ e − c δ N . (397) \mathbb{P}_{\theta}(\mathcal{G}^{c})\leq C_{\delta}e^{-c_{\delta}N}. \tag*{(397)} P θ ( G c ) ≤ C δ e − c δ N . (397)
For all sufficiently large N N N , on G \mathcal{G} G we have T + 1 ≥ N / 2 T+1\geq N/2 T + 1 ≥ N /2 and hence
∣ U ∣ ≤ δ N / 8 + 1 / 2 N / 2 ≤ δ 2 . (398) |U|\leq\frac{\delta N/8+1/2}{N/2}\leq\frac{\delta}{2}. \tag*{(398)} ∣ U ∣ ≤ N /2 δ N /8 + 1/2 ≤ 2 δ . (398)
Thus r + U ∈ [ δ / 2 , 1 − δ / 2 ] r+U\in[\delta/2,1-\delta/2] r + U ∈ [ δ /2 , 1 − δ /2 ] . Taylor’s expansion applied to Eq. (348) , uniformly on this compact interval, gives
d r ( r + U ) = U 2 2 r ( 1 − r ) + O δ ( ∣ U ∣ 3 ) on G . (399) d_r(r+U)=\frac{U^2}{2r(1-r)}+O_\delta(|U|^3)\quad\text{on }\mathcal{G}. \tag*{(399)} d r ( r + U ) = 2 r ( 1 − r ) U 2 + O δ ( ∣ U ∣ 3 ) on G . (399)
We next evaluate the second moment in Eq. (378). On G \mathcal{G} G ,
∣ 1 ( T + 1 ) 2 − 1 N 2 ∣ ≤ C ∣ T − N ∣ + 1 N 3 . (400) \left|\frac{1}{(T+1)^2}-\frac{1}{N^2}\right|\leq C\frac{|T-N|+1}{N^3}. \tag*{(400)} ( T + 1 ) 2 1 − N 2 1 ≤ C N 3 ∣ T − N ∣ + 1 . (400)
By Cauchy–Schwarz and Eqs. (350) and (366) ,
E [ ( D + b ) 2 ( ∣ T − N ∣ + 1 ) ] ≤ { E ( D + b ) 4 } 1 / 2 { E ( ∣ T − N ∣ + 1 ) 2 } 1 / 2 = O δ ( N 3 / 2 ) . (401) \begin{aligned}
\mathbb{E}\left[(D+b)^2(|T-N|+1)\right]
&\leq\left\{\mathbb{E}(D+b)^4\right\}^{1/2}
\left\{\mathbb{E}(|T-N|+1)^2\right\}^{1/2}\\
&=O_\delta(N^{3/2}). \tag*{(401)}
\end{aligned} E [ ( D + b ) 2 ( ∣ T − N ∣ + 1 ) ] ≤ { E ( D + b ) 4 } 1/2 { E ( ∣ T − N ∣ + 1 ) 2 } 1/2 = O δ ( N 3/2 ) . (401)
Moreover, the contribution of G c \mathcal{G}^c G c to any of the polynomial moments below is exponentially small, because ∣ D ∣ ≤ n ( ℓ − 1 ) |D|\leq n(\ell-1) ∣ D ∣ ≤ n ( ℓ − 1 ) , ∣ T − N ∣ ≤ n ( ℓ − 1 ) + N |T-N|\leq n(\ell-1)+N ∣ T − N ∣ ≤ n ( ℓ − 1 ) + N , and Eq. (376) holds. Using Eqs. (364) , (379) and (380), we obtain
E [ U 2 1 G ] = 1 N 2 E ( D + b ) 2 + O δ ( N − 3 / 2 ) = r ( 1 − r ) N + O δ ( N − 3 / 2 ) , (402) \begin{aligned}
\mathbb{E}[U^2\mathbf{1}_{\mathcal{G}}]
&=\frac{1}{N^2}\mathbb{E}(D+b)^2+O_\delta(N^{-3/2})\\
&=\frac{r(1-r)}{N}+O_\delta(N^{-3/2}), \tag*{(402)}
\end{aligned} E [ U 2 1 G ] = N 2 1 E ( D + b ) 2 + O δ ( N − 3/2 ) = N r ( 1 − r ) + O δ ( N − 3/2 ) , (402)
where E D = 0 \mathbb{E}D=0 E D = 0 and the term b 2 / N 2 b^2/N^2 b 2 / N 2 is absorbed into the remainder. Similarly, on G \mathcal{G} G the denominator is at least N / 2 N/2 N /2 , so Eq. (366) gives
E [ ∣ U ∣ 3 1 G ] ≤ C N 3 E ( ∣ D ∣ + 1 ) 3 = O δ ( N − 3 / 2 ) . (403) \mathbb{E}[|U|^3\mathbf{1}_{\mathcal{G}}]\leq\frac{C}{N^3}\mathbb{E}(|D|+1)^3=O_\delta(N^{-3/2}). \tag*{(403)} E [ ∣ U ∣ 3 1 G ] ≤ N 3 C E ( ∣ D ∣ + 1 ) 3 = O δ ( N − 3/2 ) . (403)
Combining Eqs. (378), (381) and (382) yields
E [ d r ( r ^ h ) 1 G ] = 1 2 N + O δ ( N − 3 / 2 ) . (404) \mathbb{E}[d_r(\widehat r_h)\mathbf{1}_{\mathcal{G}}]=\frac{1}{2N}+O_\delta(N^{-3/2}). \tag*{(404)} E [ d r ( r h ) 1 G ] = 2 N 1 + O δ ( N − 3/2 ) . (404)
It remains to control the complement. Since 0 ≤ T ≤ n ( ℓ − 1 ) 0\leq T\leq n(\ell-1) 0 ≤ T ≤ n ( ℓ − 1 ) and 0 ≤ n h 1 ( X n ) ≤ T 0\leq n_{h1}(X^n)\leq T 0 ≤ n h 1 ( X n ) ≤ T ,
1 2 ( n ( ℓ − 1 ) + 1 ) ≤ r ^ h ≤ 1 − 1 2 ( n ( ℓ − 1 ) + 1 ) . (405) \frac{1}{2(n(\ell-1)+1)}\leq\widehat r_h\leq1-\frac{1}{2(n(\ell-1)+1)}. \tag*{(405)} 2 ( n ( ℓ − 1 ) + 1 ) 1 ≤ r h ≤ 1 − 2 ( n ( ℓ − 1 ) + 1 ) 1 . (405)
Therefore
d r ( r ^ h ) ≤ C δ + log ( 2 ( n ( ℓ − 1 ) + 1 ) ) . (406) d_r(\widehat r_h)\leq C_\delta+\log(2(n(\ell-1)+1)). \tag*{(406)} d r ( r h ) ≤ C δ + log ( 2 ( n ( ℓ − 1 ) + 1 )) . (406)
Together with Eq. (376) and the fact that n ℓ = O δ ( N ) n\ell=O_\delta(N) n ℓ = O δ ( N ) , this shows
E [ d r ( r ^ h ) 1 G c ] = O δ ( N − 3 / 2 ) , (407) \mathbb{E}[d_r(\widehat r_h)\mathbf{1}_{\mathcal{G}^c}]=O_\delta(N^{-3/2}), \tag*{(407)} E [ d r ( r h ) 1 G c ] = O δ ( N − 3/2 ) , (407)
and proves Eq. (373).
We now complete the proof of Theorem 14. Substituting Eq. (373) into Eq. (347) and recalling that N h = n a h N_h=na_h N h = n a h gives
R n , ℓ ( P ) ( p ^ K T , θ ) = ∑ h = 0 1 a h ( 1 2 n a h + O δ ( ( n a h ) − 3 / 2 ) ) = 1 n + O δ ( 1 n 3 / 2 ∑ h = 0 1 1 a h ) . (408) \begin{aligned}
R_{n,\ell}^{(P)}(\widehat p_{\mathrm{KT}},\theta)
=\sum_{h=0}^{1}a_h\left(\frac{1}{2na_h}+O_\delta((na_h)^{-3/2})\right) \\
&=\frac{1}{n}+O_\delta\left(\frac{1}{n^{3/2}}\sum_{h=0}^{1}\frac{1}{\sqrt{a_h}}\right). \tag*{(408)}
\end{aligned} R n , ℓ ( P ) ( p KT , θ ) = h = 0 ∑ 1 a h ( 2 n a h 1 + O δ (( n a h ) − 3/2 ) ) = n 1 + O δ ( n 3/2 1 h = 0 ∑ 1 a h 1 ) . (408)
By Eq. (344) ,
∑ h = 0 1 1 a h ≤ 2 δ ( ℓ − 2 ) = O δ ( ℓ − 1 / 2 ) . (409) \sum_{h=0}^{1}\frac{1}{\sqrt{a_h}}\leq\frac{2}{\sqrt{\delta(\ell-2)}}=O_\delta(\ell^{-1/2}). \tag*{(409)} h = 0 ∑ 1 a h 1 ≤ δ ( ℓ − 2 ) 2 = O δ ( ℓ − 1/2 ) . (409)
Consequently,
R n , ℓ ( P ) ( p ^ K T , θ ) = 1 n + O δ ( 1 n 3 / 2 ℓ ) = 1 n + O δ ( 1 n n ℓ ) , (410) R_{n,\ell}^{(P)}(\widehat p_{\mathrm{KT}},\theta)
=\frac{1}{n}+O_\delta\left(\frac{1}{n^{3/2}\sqrt{\ell}}\right)
=\frac{1}{n}+O_\delta\left(\frac{1}{n\sqrt{n\ell}}\right), \tag*{(410)} R n , ℓ ( P ) ( p KT , θ ) = n 1 + O δ ( n 3/2 ℓ 1 ) = n 1 + O δ ( n n ℓ 1 ) , (410)
uniformly over the initial law and over θ ∈ M δ \theta\in\mathcal{M}_\delta θ ∈ M δ . This is Eq. (130) ; the minimax upper bound in Eq. (131) follows because Eq. (129) is an admissible predictor.
Proof of Lemma 15
The proof relies on conjugate transition priors and the mutual-information chain rule along the test sequence. First write
r 0 = p , r 1 = q , T = n ( ℓ − 1 ) , m = n + 1 , (411) r_0=p,\qquad r_1=q,\qquad T=n(\ell-1),\qquad m=n+1, \tag*{(411)} r 0 = p , r 1 = q , T = n ( ℓ − 1 ) , m = n + 1 , (411)
so that r h r_h r h is the probability of switching from state h h h to state 1 − h 1-h 1 − h . Fix δ < δ 0 < 1 / 2 \delta<\delta_0<1/2 δ < δ 0 < 1/2 and set
K 0 = [ δ 0 , 1 − δ 0 ] 2 ⊂ Θ δ . (412) \mathcal{K}_0=[\delta_0,1-\delta_0]^2\subset\Theta_\delta. \tag*{(412)} K 0 = [ δ 0 , 1 − δ 0 ] 2 ⊂ Θ δ . (412)
All lower-bound priors below are ultimately conditioned on K 0 \mathcal{K}_0 K 0 .
Reference priors and exact beta posteriors
Let
a T = A log T , (413) a_T=A\log T, \tag*{(413)} a T = A log T , (413)
where A > 0 A>0 A > 0 will be chosen sufficiently large. For the known-initial-distribution model, and for the transition component of the free-initial-distribution model, use the reference prior
W ~ T K = Beta ( a T , a T ) ⊗ Beta ( a T , a T ) . (414) \widetilde W_T^{\mathrm K}=\operatorname{Beta}(a_T,a_T)\otimes\operatorname{Beta}(a_T,a_T). \tag*{(414)} W T K = Beta ( a T , a T ) ⊗ Beta ( a T , a T ) . (414)
For the stationary-initial-distribution model, set
b T = a T + m 2 . (415) b_T=a_T+\frac{m}{2}. \tag*{(415)} b T = a T + 2 m . (415)
and use the reference density
W ~ T S ( p , q ) = 1 Z T ( p + q ) m p a T − 1 ( 1 − p ) b T − 1 (416) \widetilde{W}^{S}_{T}(p,q)=\frac{1}{Z_{T}}(p+q)^{m}p^{a_{T}-1}(1-p)^{b_{T}-1} \tag*{(416)} W T S ( p , q ) = Z T 1 ( p + q ) m p a T − 1 ( 1 − p ) b T − 1 (416)
× q a T − 1 ( 1 − q ) b T − 1 , ( p , q ) ∈ ( 0 , 1 ) 2 . (417) \times q^{a_{T}-1}(1-q)^{b_{T}-1},\qquad(p,q)\in(0,1)^{2}. \tag*{(417)} × q a T − 1 ( 1 − q ) b T − 1 , ( p , q ) ∈ ( 0 , 1 ) 2 . (417)
Then, the following lemma holds.
Lemma 33: For every fixed B > 0 B>0 B > 0 , A A A can be chosen so that, for j ∈ { K , S } j\in\{K,S\} j ∈ { K , S } ,
W ~ T j ( K 0 c ) ≤ C B T − B . (418) \widetilde{W}^{j}_{T}(\mathcal{K}_{0}^{c})\leq C_{B}T^{-B}. \tag*{(418)} W T j ( K 0 c ) ≤ C B T − B . (418)
Proof: Let
F ( p , q ) = 16 p ( 1 − p ) q ( 1 − q ) , (419) F(p,q)=16p(1-p)q(1-q), \tag*{(419)} F ( p , q ) = 16 p ( 1 − p ) q ( 1 − q ) , (419)
H ( p , q ) = 2 ( p + q ) ( 1 − p ) ( 1 − q ) . (420) H(p,q)=2(p+q)\sqrt{(1-p)(1-q)}. \tag*{(420)} H ( p , q ) = 2 ( p + q ) ( 1 − p ) ( 1 − q ) . (420)
For j ∈ { K , S } j\in\{K,S\} j ∈ { K , S } , denote by f T j f^{j}_{T} f T j the corresponding density divided by its value at ( 1 / 2 , 1 / 2 ) (1/2,1/2) ( 1/2 , 1/2 ) , i.e.,
f T K ( p , q ) = F ( p , q ) a T − 1 , (421) f^{K}_{T}(p,q)=F(p,q)^{a_{T}-1}, \tag*{(421)} f T K ( p , q ) = F ( p , q ) a T − 1 , (421)
f T S ( p , q ) = F ( p , q ) a T − 1 H ( p , q ) m , (422) f^{S}_{T}(p,q)=F(p,q)^{a_{T}-1}H(p,q)^{m}, \tag*{(422)} f T S ( p , q ) = F ( p , q ) a T − 1 H ( p , q ) m , (422)
where F , H ≤ 1 F,H\leq1 F , H ≤ 1 on [ 0 , 1 ] 2 [0,1]^{2} [ 0 , 1 ] 2 . Since ( p , q ) ∉ K 0 (p,q)\notin\mathcal{K}_{0} ( p , q ) ∈ / K 0 implies that at least one coordinate lies outside [ δ 0 , 1 − δ 0 ] [\delta_{0},1-\delta_{0}] [ δ 0 , 1 − δ 0 ] ,
F ( p , q ) ≤ η ≔ 4 δ 0 ( 1 − δ 0 ) < 1 , ( p , q ) ∈ K 0 c . (423) F(p,q)\leq\eta\coloneqq4\delta_{0}(1-\delta_{0})<1,\qquad(p,q)\in\mathcal{K}_{0}^{c}. \tag*{(423)} F ( p , q ) ≤ η : = 4 δ 0 ( 1 − δ 0 ) < 1 , ( p , q ) ∈ K 0 c . (423)
Consequently, for all sufficiently large T T T ,
∫ K 0 c f T j ( p , q ) d p d q ≤ η a T − 1 = O ( e − c a T ) , (424) \int_{\mathcal{K}_{0}^{c}}f^{j}_{T}(p,q)\,dp\,dq\leq\eta^{a_{T}-1}=O(e^{-ca_{T}}), \tag*{(424)} ∫ K 0 c f T j ( p , q ) d p d q ≤ η a T − 1 = O ( e − c a T ) , (424)
where c = − log η > 0 c=-\log\eta>0 c = − log η > 0 . It remains to lower-bound the normalizing integral. Write p = 1 / 2 + x p=1/2+x p = 1/2 + x and q = 1 / 2 + y q=1/2+y q = 1/2 + y . Then
F ( p , q ) = ( 1 − 4 x 2 ) ( 1 − 4 y 2 ) (425) F(p,q)=(1-4x^{2})(1-4y^{2}) \tag*{(425)} F ( p , q ) = ( 1 − 4 x 2 ) ( 1 − 4 y 2 ) (425)
and
H ( p , q ) = ( 1 + x + y ) ( 1 − 2 x ) ( 1 − 2 y ) . (426) H(p,q)=(1+x+y)\sqrt{(1-2x)(1-2y)}. \tag*{(426)} H ( p , q ) = ( 1 + x + y ) ( 1 − 2 x ) ( 1 − 2 y ) . (426)
For ∣ x ∣ , ∣ y ∣ ≤ 1 / 8 |x|,|y|\leq1/8 ∣ x ∣ , ∣ y ∣ ≤ 1/8 , the elementary inequalities log ( 1 − u ) ≥ − 2 u \log(1-u)\geq-2u log ( 1 − u ) ≥ − 2 u for 0 ≤ u ≤ 1 / 2 0\leq u\leq1/2 0 ≤ u ≤ 1/2 and log ( 1 + u ) ≥ u − u 2 \log(1+u)\geq u-u^{2} log ( 1 + u ) ≥ u − u 2 for ∣ u ∣ ≤ 1 / 2 |u|\leq1/2 ∣ u ∣ ≤ 1/2 give
log F ( p , q ) ≥ − 8 ( x 2 + y 2 ) (427) \log F(p,q)\geq-8(x^{2}+y^{2}) \tag*{(427)} log F ( p , q ) ≥ − 8 ( x 2 + y 2 ) (427)
and
log H ( p , q ) = log ( 1 + x + y ) + 1 2 log ( 1 − 2 x ) + 1 2 log ( 1 − 2 y ) ≥ − ( x + y ) 2 − 2 ( x 2 + y 2 ) ≥ − 4 ( x 2 + y 2 ) . (428) \begin{aligned}
\log H(p,q)
&=\log(1+x+y)+\frac{1}{2}\log(1-2x)+\frac{1}{2}\log(1-2y)\\
&\geq-(x+y)^{2}-2(x^{2}+y^{2})\\
&\geq-4(x^{2}+y^{2}). \tag*{(428)}
\end{aligned} log H ( p , q ) = log ( 1 + x + y ) + 2 1 log ( 1 − 2 x ) + 2 1 log ( 1 − 2 y ) ≥ − ( x + y ) 2 − 2 ( x 2 + y 2 ) ≥ − 4 ( x 2 + y 2 ) . (428)
Thus, for both j ∈ { K , S } j\in\{K,S\} j ∈ { K , S } , provided a T ≥ 2 a_{T}\geq2 a T ≥ 2 ,
f T j ( 1 / 2 + x , 1 / 2 + y ) ≥ exp { − 8 ( a T + m ) ( x 2 + y 2 ) } . (429) f^{j}_{T}(1/2+x,1/2+y)\geq\exp\{-8(a_{T}+m)(x^{2}+y^{2})\}. \tag*{(429)} f T j ( 1/2 + x , 1/2 + y ) ≥ exp { − 8 ( a T + m ) ( x 2 + y 2 )} . (429)
Fix
κ = min { 1 / 8 , 1 / 2 − δ 0 } > 0 (430) \kappa=\min\{1/8,1/2-\delta_{0}\}>0 \tag*{(430)} κ = min { 1/8 , 1/2 − δ 0 } > 0 (430)
and consider the square
Q T = { ( 1 / 2 + x , 1 / 2 + y ) : ∣ x ∣ , ∣ y ∣ ≤ κ a T + m } ⊆ K 0 . (431) Q_{T}=\left\{(1/2+x,1/2+y): |x|,|y|\leq\frac{\kappa}{\sqrt{a_{T}+m}}\right\}\subseteq\mathcal{K}_{0}. \tag*{(431)} Q T = { ( 1/2 + x , 1/2 + y ) : ∣ x ∣ , ∣ y ∣ ≤ a T + m κ } ⊆ K 0 . (431)
For every point in Q T Q_{T} Q T ,
( a T + m ) ( x 2 + y 2 ) ≤ 2 κ 2 , (432) (a_{T}+m)(x^{2}+y^{2})\leq2\kappa^{2}, \tag*{(432)} ( a T + m ) ( x 2 + y 2 ) ≤ 2 κ 2 , (432)
so f T j ( p , q ) ≥ e − 16 κ 2 f^{j}_{T}(p,q)\geq e^{-16\kappa^{2}} f T j ( p , q ) ≥ e − 16 κ 2 throughout Q T Q_{T} Q T . Since
∣ Q T ∣ = 4 κ 2 a T + m , (433) |Q_{T}|=\frac{4\kappa^{2}}{a_{T}+m}, \tag*{(433)} ∣ Q T ∣ = a T + m 4 κ 2 , (433)
we obtain the explicit lower bound
∫ ( 0 , 1 ) 2 f T j ( p , q ) d p d q ≥ ∫ Q T f T j ( p , q ) d p d q ≥ e − 16 κ 2 ∣ Q T ∣ = 4 κ 2 e − 16 κ 2 a T + m . (434) \begin{aligned}
\int_{(0,1)^{2}}f^{j}_{T}(p,q)\,dp\,dq
&\geq\int_{Q_{T}}f^{j}_{T}(p,q)\,dp\,dq\\
&\geq e^{-16\kappa^{2}}|Q_{T}|\\
&=\frac{4\kappa^{2}e^{-16\kappa^{2}}}{a_{T}+m}. \tag*{(434)}
\end{aligned} ∫ ( 0 , 1 ) 2 f T j ( p , q ) d p d q ≥ ∫ Q T f T j ( p , q ) d p d q ≥ e − 16 κ 2 ∣ Q T ∣ = a T + m 4 κ 2 e − 16 κ 2 . (434)
Combining the numerator and denominator estimates,
W ~ T j ( K 0 c ) = ∫ K 0 c f T j ( p , q ) d p d q ∫ ( 0 , 1 ) 2 f T j ( p , q ) d p d q ≤ C ( a T + m ) e − c a T . (435) \widetilde{W}^{j}_{T}(\mathcal{K}_{0}^{c})=\frac{\int_{\mathcal{K}_{0}^{c}}f^{j}_{T}(p,q)\,dp\,dq}{\int_{(0,1)^{2}}f^{j}_{T}(p,q)\,dp\,dq}
\leq C(a_{T}+m)e^{-ca_{T}}. \tag*{(435)} W T j ( K 0 c ) = ∫ ( 0 , 1 ) 2 f T j ( p , q ) d p d q ∫ K 0 c f T j ( p , q ) d p d q ≤ C ( a T + m ) e − c a T . (435)
Finally, a T = A log T a_{T}=A\log T a T = A log T , while m = n + 1 = O ( T ) m=n+1=O(T) m = n + 1 = O ( T ) for ℓ ≥ 2 \ell\geq2 ℓ ≥ 2 . Therefore,
W ~ T j ( K 0 c ) ≤ C A T 1 − c A . (436) \widetilde{W}^{j}_{T}(\mathcal{K}_{0}^{c})\leq CAT^{1-cA}. \tag*{(436)} W T j ( K 0 c ) ≤ C A T 1 − c A . (436)
For any prescribed B > 0 B>0 B > 0 , choosing A A A so that c A ≥ B + 1 cA\geq B+1 c A ≥ B + 1 establishes (418) .
Let now
F t = σ ( X 1 , … , X n , Y 1 , … , Y t ) , 1 ≤ t ≤ ℓ − 1. (437) \mathcal{F}_{t}=\sigma(X_{1},\ldots,X_{n},Y_{1},\ldots,Y_{t}),\qquad1\leq t\leq\ell-1. \tag*{(437)} F t = σ ( X 1 , … , X n , Y 1 , … , Y t ) , 1 ≤ t ≤ ℓ − 1. (437)
For h ∈ { 0 , 1 } h\in\{0,1\} h ∈ { 0 , 1 } , let N h , t N_{h,t} N h , t be the total number of observed departures from state h h h in the training sequences and in the partially revealed test sequence up to time t t t , and let S h , t S_{h,t} S h , t be the corresponding number of switches from h h h to 1 − h 1-h 1 − h .
Under (414) , the posterior row parameters are independent and
r h ∣ F t ∼ Beta ( a T + S h , t , a T + N h , t − S h , t ) . (438) r_{h}\mid\mathcal{F}_{t}\sim\operatorname{Beta}(a_{T}+S_{h,t},a_{T}+N_{h,t}-S_{h,t}). \tag*{(438)} r h ∣ F t ∼ Beta ( a T + S h , t , a T + N h , t − S h , t ) . (438)
The same identity holds for the transition posterior in the free model after conditioning on the initial states.
For the stationary model, let K K K be the number of ones among the n n n training initial states and Y 1 Y_{1} Y 1 . Their likelihood is
( p p + q ) K ( q p + q ) m − K = p K q m − K ( p + q ) m . (439) \left(\frac{p}{p+q}\right)^{K}\left(\frac{q}{p+q}\right)^{m-K}=\frac{p^{K}q^{m-K}}{(p+q)^{m}}. \tag*{(439)} ( p + q p ) K ( p + q q ) m − K = ( p + q ) m p K q m − K . (439)
Multiplication by (417) cancels the denominator exactly. Consequently,
p ∣ F t ∼ Beta ( a T + K + S 0 , t , b T + N 0 , t − S 0 , t ) , (440) p\mid\mathcal{F}_{t}\sim\operatorname{Beta}(a_{T}+K+S_{0,t},b_{T}+N_{0,t}-S_{0,t}), \tag*{(440)} p ∣ F t ∼ Beta ( a T + K + S 0 , t , b T + N 0 , t − S 0 , t ) , (440)
q ∣ F t ∼ Beta ( a T + m − K + S 1 , t , b T + N 1 , t − S 1 , t ) , (441) q\mid\mathcal{F}_{t}\sim\operatorname{Beta}(a_{T}+m-K+S_{1,t},b_{T}+N_{1,t}-S_{1,t}), \tag*{(441)} q ∣ F t ∼ Beta ( a T + m − K + S 1 , t , b T + N 1 , t − S 1 , t ) , (441)
and the two coordinates are independent. Thus, in every case considered here, the row posterior is Beta ( A h , t , B h , t ) \operatorname{Beta}(A_{h,t},B_{h,t}) Beta ( A h , t , B h , t ) , with
A h , t , B h , t ≥ a T , A h , t + B h , t = N h , t + C h , T , (442) A_{h,t},B_{h,t}\geq a_{T},\qquad A_{h,t}+B_{h,t}=N_{h,t}+C_{h,T}, \tag*{(442)} A h , t , B h , t ≥ a T , A h , t + B h , t = N h , t + C h , T , (442)
where
2 a T ≤ C h , T ≤ 2 a T + 3 m 2 . (443) 2a_T \le C_{h,T} \le2a_T + \frac{3m}{2}. \tag*{(443)} 2 a T ≤ C h , T ≤ 2 a T + 2 3 m . (443)
For each h h h and T T T , the offsets C h , T C_{h,T} C h , T are constant in the known and free cases and are functions only of the observed initial states in the stationary case.
For R ∼ Beta ( A , B ) R \sim\operatorname{Beta}(A,B) R ∼ Beta ( A , B ) and Z ∣ R ∼ Bernoulli ( R ) Z \mid R \sim\operatorname{Bernoulli}(R) Z ∣ R ∼ Bernoulli ( R ) , write
J ( A , B ) ≜ I ( R ; Z ) . (444) J(A,B) \triangleq I(R;Z). \tag*{(444)} J ( A , B ) ≜ I ( R ; Z ) . (444)
Lemma 34: If A , B ≥ a ≥ 2 A,B \ge a \ge2 A , B ≥ a ≥ 2 and M = A + B M=A+B M = A + B , then there exists a universal constant C > 0 C>0 C > 0 such that
∣ J ( A , B ) − 1 2 M ∣ ≤ C a M . (445) \left|J(A,B)-\frac{1}{2M}\right| \le\frac{C}{aM}. \tag*{(445)} J ( A , B ) − 2 M 1 ≤ a M C . (445)
Proof: The posterior-predictive success probability is A / M A/M A / M . The beta identities used in Lemma 29 give the exact expression
J ( A , B ) = h ( A M ) + A M ψ ( A + 1 ) + B M ψ ( B + 1 ) − ψ ( M + 1 ) , (446) \begin{aligned}
J(A,B)
&=h\left(\frac{A}{M}\right)+\frac{A}{M}\psi(A+1)+\frac{B}{M}\psi(B+1)\\
&\quad-\psi(M+1), \tag*{(446)}
\end{aligned} J ( A , B ) = h ( M A ) + M A ψ ( A + 1 ) + M B ψ ( B + 1 ) − ψ ( M + 1 ) , (446)
where h h h is binary entropy and ψ \psi ψ is the digamma function. Using
ψ ( x + 1 ) = log x + 1 2 x + O ( x − 2 ) , x ≥ 2 , (447) \psi(x+1)=\log x+\frac{1}{2x}+O(x^{-2}),\qquad x\ge2, \tag*{(447)} ψ ( x + 1 ) = log x + 2 x 1 + O ( x − 2 ) , x ≥ 2 , (447)
we have that the logarithmic terms cancel out, the terms of order x − 1 x^{-1} x − 1 give 1 / ( 2 M ) 1/(2M) 1/ ( 2 M ) , while the remaining terms are
O ( 1 M 2 + 1 M A + 1 M B ) = O ( 1 a M ) . (448) O\left(\frac{1}{M^2}+\frac{1}{MA}+\frac{1}{MB}\right)=O\left(\frac{1}{aM}\right). \tag*{(448)} O ( M 2 1 + M A 1 + M B 1 ) = O ( a M 1 ) . (448)
Conditionally on F t \mathcal{F}_t F t and Y t = h Y_t=h Y t = h , the next transition is Bernoulli with parameter r h r_h r h . Hence the mutual-information chain rule gives, under either reference prior,
I W ~ T ( ( P , Q ) ; Y 2 ℓ ∣ X n , Y 1 ) = ∑ t = 1 ℓ − 1 ∑ h = 0 1 E W ~ T [ 1 { Y t = h } J ( A h , t , B h , t ) ] . (449) I_{\widetilde{W}_T}\left((P,Q);Y_2^\ell\mid X^n,Y_1\right)
=\sum_{t=1}^{\ell-1}\sum_{h=0}^{1}\mathbb{E}_{\widetilde{W}_T}\left[1\{Y_t=h\}J(A_{h,t},B_{h,t})\right]. \tag*{(449)} I W T ( ( P , Q ) ; Y 2 ℓ ∣ X n , Y 1 ) = t = 1 ∑ ℓ − 1 h = 0 ∑ 1 E W T [ 1 { Y t = h } J ( A h , t , B h , t ) ] . (449)
Reciprocal counts estimate
For a fixed parameter θ = ( ν , p , q ) \theta=(\nu,p,q) θ = ( ν , p , q ) , define
a h ( θ ) = ∑ s = 1 ℓ − 1 P θ ( Y s = h ) , b h , t ( θ ) = ∑ s = 1 t − 1 P θ ( Y s = h ) . (450) a_h(\theta)=\sum_{s=1}^{\ell-1}\mathbb{P}_{\theta}(Y_s=h),\qquad
b_{h,t}(\theta)=\sum_{s=1}^{t-1}\mathbb{P}_{\theta}(Y_s=h). \tag*{(450)} a h ( θ ) = s = 1 ∑ ℓ − 1 P θ ( Y s = h ) , b h , t ( θ ) = s = 1 ∑ t − 1 P θ ( Y s = h ) . (450)
Then
b h , 1 = 0 , b h , ℓ = a h , b h , t + 1 − b h , t = P θ ( Y t = h ) , (451) b_{h,1}=0,\qquad b_{h,\ell}=a_h,\qquad b_{h,t+1}-b_{h,t}=\mathbb{P}_{\theta}(Y_t=h), \tag*{(451)} b h , 1 = 0 , b h , ℓ = a h , b h , t + 1 − b h , t = P θ ( Y t = h ) , (451)
and
m h , t ( θ ) ≜ E θ N h , t = n a h ( θ ) + b h , t ( θ ) . (452) m_{h,t}(\theta)\triangleq\mathbb{E}_{\theta}N_{h,t}=na_h(\theta)+b_{h,t}(\theta). \tag*{(452)} m h , t ( θ ) ≜ E θ N h , t = n a h ( θ ) + b h , t ( θ ) . (452)
Furthermore, we have that c δ T ≤ m h , t ( θ ) ≤ C δ T c_\delta T\le m_{h,t}(\theta)\le C_\delta T c δ T ≤ m h , t ( θ ) ≤ C δ T for ( p , q ) ∈ K 0 (p,q)\in\mathcal{K}_0 ( p , q ) ∈ K 0 and all sufficiently large ℓ \ell ℓ . Lemma 30, applied to the training batch, and the same Paulin bound applied to the partial test sequence give
P θ { ∣ N h , t − m h , t ( θ ) ∣ > u } ≤ C δ exp ( − c δ u 2 T ) , (453) \mathbb{P}_{\theta}\{|N_{h,t}-m_{h,t}(\theta)|>u\}\le C_\delta\exp\left(-c_\delta\frac{u^2}{T}\right), \tag*{(453)} P θ { ∣ N h , t − m h , t ( θ ) ∣ > u } ≤ C δ exp ( − c δ T u 2 ) , (453)
and therefore
E θ ∣ N h , t − m h , t ( θ ) ∣ ≤ C δ T , (454) \mathbb{E}_{\theta}|N_{h,t}-m_{h,t}(\theta)|\le C_\delta\sqrt{T}, \tag*{(454)} E θ ∣ N h , t − m h , t ( θ ) ∣ ≤ C δ T , (454)
uniformly over ( p , q ) ∈ K 0 (p,q)\in\mathcal{K}_0 ( p , q ) ∈ K 0 , h h h , and t t t .
Lemma 35: Let C h , T C_{h,T} C h , T be any possibly random offset satisfying
2 a T ≤ C h , T ≤ C ( n + log T ) . (455) 2a_T\le C_{h,T}\le C(n+\log T). \tag*{(455)} 2 a T ≤ C h , T ≤ C ( n + log T ) . (455)
Then, uniformly for ( p , q ) ∈ K 0 (p,q)\in\mathcal{K}_0 ( p , q ) ∈ K 0 and h ∈ { 0 , 1 } h\in\{0,1\} h ∈ { 0 , 1 } ,
∑ t = 1 ℓ − 1 E θ [ 1 { Y t = h } N h , t + C h , T ] = log ( 1 + 1 n ) + O δ ( 1 n 3 / 2 ℓ + 1 n ℓ + log T n 2 ℓ ) . (456) \sum_{t=1}^{\ell-1}\mathbb{E}_{\theta}\left[\frac{1\{Y_t=h\}}{N_{h,t}+C_{h,T}}\right]
=\log\left(1+\frac{1}{n}\right)+O_\delta\left(\frac{1}{n^{3/2}\sqrt{\ell}}+\frac{1}{n\ell}+\frac{\log T}{n^2\ell}\right). \tag*{(456)} t = 1 ∑ ℓ − 1 E θ [ N h , t + C h , T 1 { Y t = h } ] = log ( 1 + n 1 ) + O δ ( n 3/2 ℓ 1 + n ℓ 1 + n 2 ℓ log T ) . (456)
Proof: Let p h , t = P θ ( Y t = h ) p_{h,t}=\mathbb{P}_{\theta}(Y_t=h) p h , t = P θ ( Y t = h ) and m h , t = n a h + b h , t m_{h,t}=na_h+b_{h,t} m h , t = n a h + b h , t . On the event ∣ N h , t − m h , t ∣ ≤ m h , t / 2 |N_{h,t}-m_{h,t}|\le m_{h,t}/2 ∣ N h , t − m h , t ∣ ≤ m h , t /2 ,
∣ 1 N h , t + C h , T − 1 m h , t + C h , T ∣ ≤ 2 ∣ N h , t − m h , t ∣ m h , t 2 . (457) \left|\frac{1}{N_{h,t}+C_{h,T}}-\frac{1}{m_{h,t}+C_{h,T}}\right|
\le\frac{2|N_{h,t}-m_{h,t}|}{m_{h,t}^2}. \tag*{(457)} N h , t + C h , T 1 − m h , t + C h , T 1 ≤ m h , t 2 2∣ N h , t − m h , t ∣ . (457)
Using Eqs. (453) and (454) and C h , T ≥ 2 a T C_{h,T}\ge2a_T C h , T ≥ 2 a T , the complementary event contributes exponentially little, and hence
∣ E θ [ 1 { Y t = h } N h , t + C h , T ] − E θ [ 1 { Y t = h } m h , t + C h , T ] ∣ ≤ C T − 3 / 2 + C e − c T / a T . (458) \left|\mathbb{E}_{\theta}\left[\frac{1\{Y_t=h\}}{N_{h,t}+C_{h,T}}\right]
-\mathbb{E}_{\theta}\left[\frac{1\{Y_t=h\}}{m_{h,t}+C_{h,T}}\right]\right|
\le CT^{-3/2}+Ce^{-cT/a_T}. \tag*{(458)} E θ [ N h , t + C h , T 1 { Y t = h } ] − E θ [ m h , t + C h , T 1 { Y t = h } ] ≤ C T − 3/2 + C e − c T / a T . (458)
Since m h , t m_{h,t} m h , t is deterministic for fixed θ \theta θ and C h , T ≤ C ( n + log T ) C_{h,T}\le C(n+\log T) C h , T ≤ C ( n + log T ) , we also have
∣ E θ [ 1 { Y t = h } m h , t + C h , T ] − p h , t m h , t ∣ ≤ C ( n + log T ) T 2 . (459) \left|\mathbb{E}_{\theta}\left[\frac{1\{Y_t=h\}}{m_{h,t}+C_{h,T}}\right]-\frac{p_{h,t}}{m_{h,t}}\right|
\le\frac{C(n+\log T)}{T^2}. \tag*{(459)} E θ [ m h , t + C h , T 1 { Y t = h } ] − m h , t p h , t ≤ T 2 C ( n + log T ) . (459)
Finally, Eq. (451) gives
∑ t = 1 ℓ − 1 p h , t n a h + b h , t = ∑ t = 1 ℓ − 1 b h , t + 1 − b h , t n a h + b h , t . (460) \sum_{t=1}^{\ell-1}\frac{p_{h,t}}{na_h+b_{h,t}}
=\sum_{t=1}^{\ell-1}\frac{b_{h,t+1}-b_{h,t}}{na_h+b_{h,t}}. \tag*{(460)} t = 1 ∑ ℓ − 1 n a h + b h , t p h , t = t = 1 ∑ ℓ − 1 n a h + b h , t b h , t + 1 − b h , t . (460)
Since x ↦ ( n a h + x ) − 1 x\mapsto(na_h+x)^{-1} x ↦ ( n a h + x ) − 1 is decreasing and the increments lie in [ 0 , 1 ] [0,1] [ 0 , 1 ] , we have
0 ≤ ∑ t = 1 ℓ − 1 b h , t + 1 − b h , t n a h + b h , t − ∫ 0 a h d x n a h + x ≤ 1 2 n 2 a h . (461) 0\le\sum_{t=1}^{\ell-1}\frac{b_{h,t+1}-b_{h,t}}{na_h+b_{h,t}}
-\int_0^{a_h}\frac{dx}{na_h+x}
\le\frac{1}{2n^2a_h}. \tag*{(461)} 0 ≤ t = 1 ∑ ℓ − 1 n a h + b h , t b h , t + 1 − b h , t − ∫ 0 a h n a h + x d x ≤ 2 n 2 a h 1 . (461)
The integral equals log ( 1 + 1 / n ) \log(1+1/n) log ( 1 + 1/ n ) and a h ≥ δ ( ℓ − 2 ) a_h\ge\delta(\ell-2) a h ≥ δ ( ℓ − 2 ) . Summing the preceding errors over t t t proves the lemma. ■
By Eqs. (442) , (443) and (445) and Lemma 35, for every fixed ( p , q ) ∈ K 0 (p,q)\in\mathcal{K}_0 ( p , q ) ∈ K 0 each row contributes
1 2 log ( 1 + 1 n ) + o ( 1 / n ) (462) \frac{1}{2}\log\left(1+\frac{1}{n}\right)+o(1/n) \tag*{(462)} 2 1 log ( 1 + n 1 ) + o ( 1/ n ) (462)
uniformly. Indeed, the accumulated remainder from Eq. (445) is O ( 1 / ( n a T ) ) = o ( 1 / n ) O(1/(na_T))=o(1/n) O ( 1/ ( n a T )) = o ( 1/ n ) .
Conditioning onto the Markov class
We finish by passing from the reference priors to priors supported in K 0 \mathcal{K}_0 K 0 . The following elementary observation is useful.
Lemma 36. Let W ~ \widetilde{W} W be a prior distribution, let C \mathcal{C} C be a parameter event with W ~ ( C ) = ε \widetilde{W}(\mathcal{C}) = \varepsilon W ( C ) = ε , and let W = W ~ ( ⋅ ∣ C ) W = \widetilde{W}(\cdot\mid\mathcal{C}) W = W ( ⋅ ∣ C ) . If Y Y Y takes at most M M M values, then
( 1 − ε ) I W ( Θ ; Y ∣ X ) − I W ~ ( Θ ; Y ∣ X ) ≤ h 2 ( ε ) + ε log M . (463) (1-\varepsilon)I_W(\Theta;Y\mid X)-I_{\widetilde{W}}(\Theta;Y\mid X)\le h_2(\varepsilon)+\varepsilon\log M. \tag*{(463)} ( 1 − ε ) I W ( Θ ; Y ∣ X ) − I W ( Θ ; Y ∣ X ) ≤ h 2 ( ε ) + ε log M . (463) Proof. Let C = 1 { Θ ∈ C } C = 1\{\Theta\in\mathcal{C}\} C = 1 { Θ ∈ C } . Since C C C is a function of Θ \Theta Θ , the chain rule gives
I W ~ ( Θ ; Y ∣ X ) = I ( C ; Y ∣ X ) + ( 1 − ε ) I W ( Θ ; Y ∣ X ) + ε I W ~ ( ⋅ ∣ C c ) ( Θ ; Y ∣ X ) . (464) I_{\widetilde{W}}(\Theta;Y\mid X)=I(C;Y\mid X)+(1-\varepsilon)I_W(\Theta;Y\mid X)+\varepsilon I_{\widetilde{W}(\cdot\mid\mathcal{C}^c)}(\Theta;Y\mid X). \tag*{(464)} I W ( Θ ; Y ∣ X ) = I ( C ; Y ∣ X ) + ( 1 − ε ) I W ( Θ ; Y ∣ X ) + ε I W ( ⋅ ∣ C c ) ( Θ ; Y ∣ X ) . (464) Now I ( C ; Y ∣ X ) ≤ h 2 ( ε ) I(C;Y\mid X)\le h_2(\varepsilon) I ( C ; Y ∣ X ) ≤ h 2 ( ε ) and the last conditional mutual information is at most log M \log M log M .
For j ∈ { K , S } j\in\{\mathrm{K},\mathrm{S}\} j ∈ { K , S } , let ε ~ T j = W ~ T j ( K 0 c ) \widetilde{\varepsilon}_T^j=\widetilde{W}_T^j(\mathcal{K}_0^c) ε T j = W T j ( K 0 c ) . Under the reference joint law, the expression in Eq. (449) is nonnegative and is at most ( ℓ − 1 ) log 2 (\ell-1)\log2 ( ℓ − 1 ) log 2 . Therefore Eqs. (418) and (462) imply
I W ~ T j ( ( P , Q ) ; Y 2 ℓ ∣ X n , Y 1 ) = log ( 1 + 1 n ) + o ( 1 / n ) . (465) I_{\widetilde{W}_T^j}((P,Q);Y_2^\ell\mid X^n,Y_1)=\log\left(1+\frac{1}{n}\right)+o(1/n). \tag*{(465)} I W T j (( P , Q ) ; Y 2 ℓ ∣ X n , Y 1 ) = log ( 1 + n 1 ) + o ( 1/ n ) . (465)
Lemma 33 with B > 1 B>1 B > 1 and Lemma 36 with M = 2 ℓ − 1 M=2^{\ell-1} M = 2 ℓ − 1 show that the conditioned priors
W T j = W ~ T j ( ⋅ ∣ K 0 ) (466) W_T^j=\widetilde{W}_T^j(\cdot\mid\mathcal{K}_0) \tag*{(466)} W T j = W T j ( ⋅ ∣ K 0 ) (466)
satisfy the same expansion above, because
h 2 ( ε ~ T j ) + ε ~ T j ( ℓ − 1 ) log 2 = o ( 1 / n ) . (467) h_2(\widetilde{\varepsilon}_T^j)+\widetilde{\varepsilon}_T^j(\ell-1)\log2=o(1/n). \tag*{(467)} h 2 ( ε T j ) + ε T j ( ℓ − 1 ) log 2 = o ( 1/ n ) . (467)
Furthermore, they are supported in the fixed compact subrectangle K 0 ⊂ Θ δ \mathcal{K}_0\subset\Theta_\delta K 0 ⊂ Θ δ . This proves the known and stationary cases of Lemma 15 . In the free model, combine W T K W_T^{\mathrm{K}} W T K with an independent prior on the free initial probability; after conditioning on the initial states, the transition posterior is exactly the known-initial posterior, so the same expansion above applies.
Acknowledgments
This research was supported by the Swiss National Science Foundation Grant No. 10004381.