Introduction
Bilateral trade studies the interaction of two agents—a seller and a buyer—interested in trading a good [e.g., 38]. Each agent holds a private valuation for the good, and they strive to maximize their own utility. Designing “good” mechanisms for bilateral trade is a fundamental problem in economics and has been studied extensively in the computer science community [e.g., 16, 9, 4, 8, 22, 18, 29, 35, 10, 6, 25]. There are two natural objectives that a broker tasked with designing a mechanism for bilateral trade may be interested in maximizing: social welfare, often measured in terms of its increase, the gains from trade, and profit. The predominant approach considered in prior work is Bayesian , as it assumes that both agents’ valuations are drawn from known and typically independent distributions. In many practical applications, however, knowledge of these distributions is a non-trivial assumption, and valuations may not be independent.
In this work, we study profit maximization in repeated bilateral trade through the lens of online learning [e.g., 7, 31, 13]. While a substantial literature studies social welfare maximization [e.g., 14, 15, 3, 6, 36], only recently has the literature started investigating profit from a regret-minimizing perspective, with Di Gregorio et al. [19 , 20 ] providing a complete characterization of the minimax regret regimes for profit maximization when the learner is allowed to post any mechanism which ensures dominant-strategy incentive compatibility and individual rationality . While these general mechanisms for bilateral trade are very expressive, they may be fairly complex and, crucially, require agents to submit their actual valuations to the learner.
In this work, we focus on the important and natural family of posted-price mechanisms. In a posted-price mechanism, the broker just posts two prices, one for the seller and one for the buyer, and the trade happens if and only if both agents agree to trade for the proposed price. These simple mechanisms are specified by two numbers and require each agent only to accept or reject an offer.
We consider two families of data-generation models for the online valuations: the adversarial model, where an oblivious adversary fixes the agents’ valuations before play, and the stochastic model, where valuation pairs are drawn i.i.d. from a fixed unknown distribution. Our benchmark is the best fixed pair of prices, evaluated on the input sequence in the adversarial setting and in expectation in the stochastic setting. We seek algorithms whose expected cumulative profit falls short of this benchmark by a quantity sublinear in the time horizon T T T .
Our Results
We characterize the minimax regret rates for the data-generation models below, up to polylogarithmic factors. See Table 1 for a summary. All upper bounds use one-bit feedback (whether both offers were accepted), whereas the lower bounds hold even with two-bit feedback (the seller’s and buyer’s acceptance decisions observed separately).
For an oblivious adversary, our algorithm guarantees O ( T 2 / 3 log 7 / 3 T ) O(T^{2/3}\log^{7/3} T) O ( T 2/3 log 7/3 T ) regret (Thm. 4.1). Given the lower bound of Ω ( T 2 / 3 ) \Omega(T^{2/3}) Ω ( T 2/3 ) for one-sided pricing [31 ] , our algorithm is nearly tight even when the agents’ valuations are drawn i.i.d. from generic independent distributions. Indeed, setting the seller’s value identically to zero embeds one-sided pricing: the seller always accepts, and a positive seller price only reduces profit without providing additional information.
For valuations drawn i.i.d. from independent regular distributions, we obtain O ( T 3 / 5 log 5 T ) O(T^{3/5}\log^5 T) O ( T 3/5 log 5 T ) regret (Thm. 6.1), complemented by a matching Ω ( T 3 / 5 ) \Omega(T^{3/5}) Ω ( T 3/5 ) lower bound, up to logarithmic factors (Thm. 7.1 ).
First, it is interesting to compare the bilateral trade regret rates with those for the one-dimensional problem of (one-sided) pricing [31 ] , where the learner only interacts with the buyer. There, the adversarial rate is the same, but when it comes to regularity there is a clear differentiation: regularity and MHR yield the same T \sqrt{T} T rate in one-sided pricing [40 , 34 ] , while bilateral trade exhibits two intriguing rates, i.e., T \sqrt{T} T for MHR and the novel (and surprising) rate of T 3 / 5 T^{3/5} T 3/5 for regular distributions.
Table 1. Minimax regret rates for profit maximization with posted prices in bilateral trade. Entries with the same color share the same minimax rate. The adversarial positive result applies to all other settings, while the lower bound for i.i.d. distributions with independent marginals also applies to the generic stochastic setting and the adversarial one.
Second, a comparison with previous work on regret minimization in bilateral trade is also instructive. While for gain-from-trade maximization (and profit maximization with generic mechanisms) there is typically a separation between the adversarial and the stochastic settings, where the former allows for no-regret only under additional assumptions (such as σ \sigma σ -smoothness) [5 , 19 , 20 ] , this does not happen for our problem. In particular, faster rates are here allowed by classical mechanism design concepts, namely regularity and monotone hazard rate. This is partly due to an exciting and novel connection between online bilateral trade and online convex optimization: MHR makes the negative logarithm of expected profit convex, while regularity yields a family of auxiliary profit functions with convex reciprocals.
Challenges
Finding profit-maximizing posted prices poses two challenges: exploiting the structure of the profit function and learning from one-bit feedback. Two generic approaches illustrate why additional ideas are needed.
Structure of Profit. The profit function exhibits a convenient one-sided Lipschitzness property: raising the seller’s price and lowering the buyer’s price preserves every previously accepted trade while lowering the profit proportionally to the displacement. Therefore, we can effectively discretize the action space [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 : restricting the algorithm’s attention to the uniform grid G K G_K G K of mesh 1 / K 1/K 1/ K , with Θ ( K 2 ) \Theta(K^2) Θ ( K 2 ) pairs of prices, induces a discretization error of at most ≈ T / K \approx T/K ≈ T / K (more on this in Lemma A.6 ). Treating these pairs as unrelated arms and running a finite-armed bandit algorithm yields
R T ≲ K 2 T + 2 T K ≈ T 3 / 4 , R_T \lesssim\sqrt{K^2T}+\frac{2T}{K}\approx T^{3/4}, R T ≲ K 2 T + K 2 T ≈ T 3/4 ,
for the right tuning of K ≈ T 1 / 4 K\approx T^{1/4} K ≈ T 1/4 . This is the generic argument for Lipschitz bandits in R 2 \mathbb{R}^2 R 2 [30 ] . This “naive” approach of using a uniform grid and then treating each point independently is tight for (adversarial) one-sided pricing but can be improved for certain classes of objectives using more refined notions of dimension , both in the stochastic and adversarial worlds [30 , 39 ] . Unfortunately, these off-the-shelf approaches do not help for bilateral trade, since the zooming dimension of our problem is still 2 (see Appendix § 11 ).
One-Bit Feedback. The one-bit feedback is at least as informative as the bandit feedback (if you know the outcome of the trade and the prices posted, you can reconstruct exactly your profit), but is also somehow richer, as the prices are not isolated arms but share an underlying structure (so that agents accepting price ( p , q ) (p,q) ( p , q ) allow you to reconstruct the profit for any ( p ′ , q ′ ) (p',q') ( p ′ , q ′ ) with p ′ ≥ p p'\ge p p ′ ≥ p ). This idea is exploited in a recent work of Castiglioni et al. [12 ] for the stochastic version of our problem. There, they show how to uniformly ϵ \epsilon ϵ -estimate the two-dimensional CDF, and thus the profit, of the underlying distribution on the uniform grid of mesh 1 / ϵ 1/\epsilon 1/ ϵ , using as few as O ~ ( 1 / ϵ 3 ) \widetilde{O}(1/\epsilon^3) O ( 1/ ϵ 3 ) pricing queries (which are equivalent to our one-bit feedback). For context, estimating the reward of 1 / ϵ 2 1/\epsilon^2 1/ ϵ 2 independent arms in the stochastic multi-armed bandit problem requires Ω ( 1 / ϵ 4 ) \Omega(1/\epsilon^4) Ω ( 1/ ϵ 4 ) queries. It follows directly from Castiglioni et al. [12 ] that an explore-then-commit approach for our problem would satisfy:
R T ≲ 1 / ϵ 3 exploration + ϵ T exploitation ≈ T 3 / 4 , R_T \lesssim\underset{\text{exploration}}{1/\epsilon^3}+\underset{\text{exploitation}}{\epsilon T}\approx T^{3/4}, R T ≲ exploration 1/ ϵ 3 + exploitation ϵ T ≈ T 3/4 ,
by fine-tuning the precision ε ≈ T − 1 / 4 \varepsilon\approx T^{-1/4} ε ≈ T − 1/4 . Although the query complexity of Castiglioni et al. [12 ] is optimal up to logarithmic factors, this does not imply a T 3 / 4 T^{3/4} T 3/4 lower bound in the online setting. Indeed, it may be possible to quickly rule out prices that are clearly suboptimal and focus the exploration on “good” candidate prices. Indeed, we show that such fine-grained estimation is possible: we propose a localized estimation procedure which allows us to acquire useful information on possibly optimal prices by only querying prices which are not “too” suboptimal.
Related work
Learning in bilateral trade. Regret minimization for bilateral trade has largely focused on gains from trade, with a range of feedback models, stochastic assumptions, and budget constraints [14 , 15 , 3 , 6 , 36 ] . Profit differs from gains from trade both in its objective and in the useful action space, since a broker benefits from a positive spread between the buyer and seller prices. Recent work studies profit maximization over general dominant-strategy incentive-compatible and individually rational mechanisms [19 , 20 ] , with these mechanisms having a much richer allocation rule and a different benchmark; moreover, their learning protocols can use reported valuations. We instead focus on fixed pairs of posted prices and on the single bit that reports whether the proposed trade succeeds. Accordingly, the rates are not directly comparable across these mechanism and feedback classes. For more details on online learning and bilateral trade, we refer to the survey by Bernasconi and Fusco [5 ] .
Learning Posted Prices in Bilateral Trade. Castiglioni et al. [12 ] study uniform approximation of multidimensional CDFs on finite uniform grids from one-bit queries, and are the first to explicitly mention online learning of posted prices for bilateral trade as a direct application of their results, although with a non-tight O ( T 3 / 4 ) O(T^{3/4}) O ( T 3/4 ) rate. We build on their use of monotonicity and shared probability estimates, but manage to achieve nearly optimal rates by localizing the one-bit queries needed to estimate profit. In subsequent work, Castiglioni et al. [11 ] obtain O ~ ( T 7 / 10 ) \widetilde{O}(T^{7/10}) O ( T 7/10 ) regret for a class of stochastic i.i.d. CDF-based objectives that includes posted prices in bilateral trade. Their approach is based on an initial coarse pruning phase, complemented by ideas from the feedback graph literature [1 ] . Both Castiglioni et al. [12 ] and Castiglioni et al. [11 ] only study the stochastic i.i.d. setting with possibly correlated seller and buyer valuations, and do not investigate the adversarial setting or MHR and regular distributions.
Pricing and distributional structure. One-sided online pricing is a classical connection between online learning and economics [7 , 31 , 13 ] ; see also Slivkins [41 ] for the bandit perspective. Pricing-query complexity studies how much can be learned when values are accessible only through acceptance decisions [34 ] . For regular distributions, Singla and Wang [40 ] exploit half-concavity to obtain square-root regret up to logarithmic factors for sequential posted pricing. Our separation shows that applying this intuition independently on the two sides of a market is insufficient. Our MHR algorithm instead uses concavity of the logarithm of expected bilateral profit as a function of both prices.
The Learning Model
In this paper, we study profit maximization in bilateral trade in an online learning setting. In particular, we focus our attention on the family of posted-price mechanisms. At each time step t t t , a new seller and buyer arrive with private valuations ( s t , b t ) ∈ [ 0 , 1 ] 2 (s_t,b_t) \in[0,1]^2 ( s t , b t ) ∈ [ 0 , 1 ] 2 . The learner posts prices ( p t , q t ) ∈ [ 0 , 1 ] 2 (p_t,q_t) \in[0,1]^2 ( p t , q t ) ∈ [ 0 , 1 ] 2 , offering p t p_t p t to the seller and charging q t q_t q t to the buyer. Trade occurs if s t ≤ p t s_t \leq p_t s t ≤ p t and q t ≤ b t q_t \leq b_t q t ≤ b t , with ties resolved in favor of acceptance. The learner receives profit
Profit t ( p t , q t ) = ( q t − p t ) ⋅ 1 { s t ≤ p t } 1 { q t ≤ b t } . \operatorname{Profit}_t(p_t,q_t) = (q_t-p_t) \cdot\mathbf{1}_{\{s_t\leq p_t\}}\mathbf{1}_{\{q_t\leq b_t\}}. Profit t ( p t , q t ) = ( q t − p t ) ⋅ 1 { s t ≤ p t } 1 { q t ≤ b t } .
We consider two data-generation models: the adversarial setting, where an oblivious adversary fixes an arbitrary sequence ( s 1 , b 1 ) , … , ( s T , b T ) (s_1,b_1),\ldots,(s_T,b_T) ( s 1 , b 1 ) , … , ( s T , b T ) before play, and the stochastic setting, where the pairs ( s t , b t ) (s_t,b_t) ( s t , b t ) are sampled i.i.d. from a fixed unknown joint distribution D \mathcal{D} D . Note that, in the latter model, independence across rounds does not require independence between the two agents in a round.
Learning Protocol for Repeated Bilateral Trade
for t = 1 , 2 , … t = 1, 2, \ldots t = 1 , 2 , … do The environment supplies the next private valuation pair ( s t , b t ) ∈ [ 0 , 1 ] 2 (s_t, b_t) \in[0,1]^2 ( s t , b t ) ∈ [ 0 , 1 ] 2 The learner posts prices ( p t , q t ) ∈ [ 0 , 1 ] 2 (p_t, q_t) \in[0,1]^2 ( p t , q t ) ∈ [ 0 , 1 ] 2 The learner observes z t = 1 { s t ≤ p t } ⋅ 1 { q t ≤ b t } z_t = \mathbf{1}_{\{s_t \le p_t\}} \cdot\mathbf{1}_{\{q_t \le b_t\}} z t = 1 { s t ≤ p t } ⋅ 1 { q t ≤ b t } and is awarded Profit t ( p t , q t ) ∈ [ − 1 , 1 ] \operatorname{Profit}_t(p_t, q_t) \in[-1,1] Profit t ( p t , q t ) ∈ [ − 1 , 1 ]
In each round, the learner receives only the trade bit z t = 1 { s t ≤ p t } 1 { q t ≤ b t } z_t = \mathbf{1}_{\{s_t \le p_t\}}\mathbf{1}_{\{q_t \le b_t\}} z t = 1 { s t ≤ p t } 1 { q t ≤ b t } . Its future prices may depend on its past prices, observed bits, and private randomness. We allow all pairs in [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 , including p t > q t p_t > q_t p t > q t , which may produce negative profit. Given an algorithm A \mathcal{A} A and a valuation-generating environment S \mathcal{S} S , the corresponding regret is defined as
R T ( A , S ) = sup ( p , q ) ∈ [ 0 , 1 ] 2 ∑ t = 1 T E [ Profit t ( p , q ) − Profit t ( p t , q t ) ] . R_T(\mathcal{A}, \mathcal{S}) = \sup_{(p,q)\in[0,1]^2} \sum_{t=1}^{T} \mathbb{E}\left[\operatorname{Profit}_t(p,q) - \operatorname{Profit}_t(p_t,q_t)\right]. R T ( A , S ) = ( p , q ) ∈ [ 0 , 1 ] 2 sup t = 1 ∑ T E [ Profit t ( p , q ) − Profit t ( p t , q t ) ] .
Note that the expectation is over the learner’s randomness and, in the stochastic setting, the sampled valuations. An optimal comparator can always be chosen in the compact triangle T = { ( p , q ) ∈ [ 0 , 1 ] 2 : p ≤ q } \mathcal{T} = \{(p,q) \in[0,1]^2 : p \le q\} T = {( p , q ) ∈ [ 0 , 1 ] 2 : p ≤ q } . Outside T \mathcal{T} T , profit is nonpositive, while diagonal prices yield zero. On T \mathcal{T} T , each realized profit function is bounded, nonnegative, and upper semicontinuous, so the comparator objective attains a maximum ( p ⋆ , q ⋆ ) (p^\star,q^\star) ( p ⋆ , q ⋆ ) on T \mathcal{T} T . For a class S \mathcal{S} S of environments, the worst-case regret is sup S ∈ S R T ( A , S ) \sup_{\mathcal{S}\in\mathcal{S}} R_T(\mathcal{A},\mathcal{S}) sup S ∈ S R T ( A , S ) , and the minimax regret is its infimum over learning algorithms. The learning goal is no regret : a worst-case regret bound o ( T ) o(T) o ( T ) , so the expected shortfall from the benchmark, divided by T T T , vanishes.
Expected profit and distributional assumptions. In the stochastic setting, we denote by μ \mu μ the expected profit function, so that μ ( p , q ) = E [ Profit ( p , q ) ] \mu(p,q) = \mathbb{E}[\operatorname{Profit}(p,q)] μ ( p , q ) = E [ Profit ( p , q )] , where Profit \operatorname{Profit} Profit is the profit function when the underlying valuations are drawn according to D \mathcal{D} D . Let P ( p , q ) = P ( s , b ) ∼ D ( s ≤ p , b ≥ q ) P(p,q) = \mathbb{P}_{(s,b)\sim\mathcal{D}}(s \le p, b \ge q) P ( p , q ) = P ( s , b ) ∼ D ( s ≤ p , b ≥ q ) be the trade probability and μ ⋆ = μ ( p ⋆ , q ⋆ ) \mu^\star= \mu(p^\star,q^\star) μ ⋆ = μ ( p ⋆ , q ⋆ ) the optimal expected profit. Then μ ( p , q ) = ( q − p ) ⋅ P ( p , q ) \mu(p,q) = (q-p) \cdot P(p,q) μ ( p , q ) = ( q − p ) ⋅ P ( p , q ) . In particular, the regret in the stochastic setting can equivalently be written as
R T ( A , D ) = T μ ⋆ − ∑ t = 1 T E [ μ ( p t , q t ) ] . R_T(\mathcal{A},\mathcal{D}) = T\mu^\star- \sum_{t=1}^{T}\mathbb{E}[\mu(p_t,q_t)]. R T ( A , D ) = T μ ⋆ − t = 1 ∑ T E [ μ ( p t , q t )] .
Our improved stochastic guarantees concern independent seller and buyer distributions, D = D s × D b \mathcal{D} = \mathcal{D}_s \times\mathcal{D}_b D = D s × D b , satisfying regularity or the monotone-hazard-rate property [37 , 38 , 26 ] . For these two classes, assume strictly positive probability densities f s f_s f s and f b f_b f b on [ 0 , 1 ] [0,1] [ 0 , 1 ] , and let F s F_s F s and F b F_b F b denote their cumulative distribution functions.
Definition 2.1 (Virtual Valuations). The seller’s virtual cost ψ \psi ψ and the buyer’s virtual value ϕ \phi ϕ are
ϕ ( b ) = b − 1 − F b ( b ) f b ( b ) , ∀ b ∈ [ 0 , 1 ] , ψ ( s ) = s + F s ( s ) f s ( s ) , ∀ s ∈ [ 0 , 1 ] . \phi(b) = b - \frac{1-F_b(b)}{f_b(b)}, \ \forall b \in[0,1], \qquad\psi(s) = s + \frac{F_s(s)}{f_s(s)}, \ \forall s \in[0,1]. ϕ ( b ) = b − f b ( b ) 1 − F b ( b ) , ∀ b ∈ [ 0 , 1 ] , ψ ( s ) = s + f s ( s ) F s ( s ) , ∀ s ∈ [ 0 , 1 ] .
Definition 2.2 (Regular Distribution). A seller, respectively buyer, distribution is regular if ψ \psi ψ , respectively ϕ \phi ϕ , is nondecreasing on [ 0 , 1 ] [0,1] [ 0 , 1 ] .
Definition 2.3 (MHR). A seller distribution satisfies the monotone hazard rate (MHR) property if its hazard rate f s / F s f_s/F_s f s / F s is nonincreasing on ( 0 , 1 ] (0,1] ( 0 , 1 ] . A buyer distribution satisfies MHR if its hazard rate f b / ( 1 − F b ) f_b/(1-F_b) f b / ( 1 − F b ) is nondecreasing on [ 0 , 1 ) [0,1) [ 0 , 1 ) .
These monotonicity conditions imply that ψ \psi ψ and ϕ \phi ϕ are nondecreasing, so MHR implies regularity.
Technical Overview
In this section, we provide a technical overview of our results, whose detailed proofs are deferred to the technical sections. We start by presenting in § 3.1 an algorithm which enjoys a T 2 / 3 T^{2/3} T 2/3 regret bound in the generic stochastic setting. While this result is subsumed by our adversarial algorithm with the same rate, its clean structure and analysis provide a first application of our “localized sampling lemma”, which is then used as a building block for our adversarial result (see § 3.2 ). We then highlight the T \sqrt{T} T result for monotone-hazard-rate distributions (§ 3.3 ) and finally the matching upper (Section 3.4) and lower bounds (§ 3.5 ) for regular distributions.
A Simple Discretization Lemma. Before moving to these results, we make a simple observation which follows directly from the one-sided Lipschitzness of the profit function, and is formally stated and proved as Lemma A.6 in Appendix § 10 .§ 3 : uniform grids are a good discretization of the action space [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 . Indeed, for any integer K K K , consider the uniform grid G K = { ( i / K , j / K ) , for i , j = 0 , … , K } G_K = \{(i/K,j/K),\ \text{for } i,j = 0,\ldots,K\} G K = {( i / K , j / K ) , for i , j = 0 , … , K } of step-size 1 / K 1/K 1/ K . For any valuation pair and prices ( p , q ) ∈ T (p,q) \in\mathcal{T} ( p , q ) ∈ T , rounding to ( ⌈ p K ⌉ / K , ⌊ q K ⌋ / K ) (\lceil pK\rceil/K,\lfloor qK\rfloor/K) (⌈ p K ⌉ / K , ⌊ q K ⌋ / K ) loses at most 2 / K 2/K 2/ K in profit; if the rounded seller price exceeds the rounded buyer price, use ( 0 , 0 ) (0,0) ( 0 , 0 ) instead. This implies that all our algorithms may just play in the discretized action space A = T ∩ G K \mathcal{A} = \mathcal{T} \cap G_K A = T ∩ G K (for suitable values of K K K ), as long as they account for an overall discretization error of 2 T / K 2T/K 2 T / K in the regret. Another useful consequence of this discretization result is that, if we fix K K K upfront, we can imagine all agents’ valuations to belong to G K G_K G K , at the cost of the same additive 2 T / K 2T/K 2 T / K term in the regret.
Warm-up: O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) Regret for the Stochastic Setting
We first sketch a T 2 / 3 T^{2/3} T 2/3 algorithm for the stochastic i.i.d. setting, in which the valuation pairs ( s t , b t ) (s_t,b_t) ( s t , b t ) are drawn independently across rounds from a fixed, unknown distribution D \mathcal{D} D . This result is less general than the one we prove in the adversarial setting in § 4 , but already contains the crucial Localized Sampling Lemma (Main Result 1) as an ingredient. In § 3.2 we sketch how to use the Localized Sampling Lemma in the adversarial setting. As discussed in § 1.2 , treating grid prices as unrelated arms or estimating all trade probabilities before committing gives only a O ~ ( T 3 / 4 ) \widetilde{O}(T^{3/4}) O ( T 3/4 ) guarantee. To improve it, exploration must become cheaper as estimates improve. In particular, we would like to estimate profits at promising price pairs using queries whose profit gaps are of the same order as the estimation accuracy. To formalize this intuition, let’s introduce some notation: for a generic pair of prices a ∈ A = G K ∩ T a \in\mathcal{A} = G_K \cap\mathcal{T} a ∈ A = G K ∩ T (a stands for action) and a generic valuation distribution Q \mathcal{Q} Q over [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 , we denote by μ Q ( a ) \mu_{\mathcal{Q}}(a) μ Q ( a ) the expected profit obtained by posting a a a when the underlying valuations are drawn according to Q \mathcal{Q} Q . Similarly, we can define the suboptimality gap Δ Q ( a ) \Delta_{\mathcal{Q}}(a) Δ Q ( a ) of a a a under Q \mathcal{Q} Q , which, for any precision η \eta η , identifies the family of η \eta η -suboptimal actions A η ( Q ) A_\eta(\mathcal{Q}) A η ( Q ) :
Δ Q ( a ) = max a ′ ∈ A μ Q ( a ′ ) − μ Q ( a ) , A η ( Q ) = { a ∈ A : Δ Q ( a ) ≤ η } . \Delta_{\mathcal{Q}}(a) = \max_{a' \in\mathcal{A}} \mu_{\mathcal{Q}}(a') - \mu_{\mathcal{Q}}(a), \qquad A_\eta(\mathcal{Q}) = \{a \in\mathcal{A} : \Delta_{\mathcal{Q}}(a) \le\eta\}. Δ Q ( a ) = a ′ ∈ A max μ Q ( a ′ ) − μ Q ( a ) , A η ( Q ) = { a ∈ A : Δ Q ( a ) ≤ η } .
To formally introduce our “localized sampling lemma”, we need two valuation distributions D \mathcal{D} D and D 0 \mathcal{D}_0 D 0 . The former is the unknown distribution that we need to learn from one-bit pricing queries, while the latter is a known reference distribution . For now, we do not make any further assumption on these two distributions. We have all the ingredients to sketch our sampling routine and its properties. We refer to Lemma 4.8 in § 4 for the precise statement and proof.
Main Result 1 (Localized Sampling Lemma). Fix A = G K ∩ T \mathcal{A} = G_K \cap\mathcal{T} A = G K ∩ T , a known reference distribution D 0 \mathcal{D}_0 D 0 , a precision ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] , and a failure probability δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) . There is a routine, depending only on these inputs, that for every unknown distribution D \mathcal{D} D uses at most
N ( ε , δ ) ∈ O ~ ( K ε 2 + 1 ε 3 ) N(\varepsilon,\delta) \in\widetilde{O}\left(\frac{K}{\varepsilon^2} + \frac{1}{\varepsilon^3}\right) N ( ε , δ ) ∈ O ( ε 2 K + ε 3 1 ) one-bit pricing queries, all in A 2 ε ( D 0 ) A_{2\varepsilon}(\mathcal{D}_0) A 2 ε ( D 0 ) , and returns estimates μ ^ ( ⋅ ) \widehat{\mu}(\cdot) μ ( ⋅ ) satisfying, with probability at least 1 − δ 1-\delta 1 − δ ,
∣ μ ^ ( a ) − μ D ( a ) ∣ ≤ ε / 64 for every a ∈ A ε ( D 0 ) . |\widehat{\mu}(a)-\mu_{\mathcal{D}}(a)| \le\varepsilon/64 \qquad\text{for every } a \in A_{\varepsilon}(\mathcal{D}_0). ∣ μ ( a ) − μ D ( a ) ∣ ≤ ε /64 for every a ∈ A ε ( D 0 ) .
For any precision ε \varepsilon ε , the reference distribution D 0 \mathcal{D}_0 D 0 then individuates two sets A ε ( D 0 ) ⊆ A 2 ε ( D 0 ) ⊆ G K A_{\varepsilon}(\mathcal{D}_0) \subseteq A_{2\varepsilon}(\mathcal{D}_0) \subseteq G_K A ε ( D 0 ) ⊆ A 2 ε ( D 0 ) ⊆ G K , while we would like to estimate up to precision ε \varepsilon ε all prices in A ε ( D 0 ) A_{\varepsilon}(\mathcal{D}_0) A ε ( D 0 ) with respect to the unknown distribution D \mathcal{D} D . This result tells us that we can do it using only O ~ ( T 1 + ε / K + ε 2 ) \widetilde{O}(T^{1+\varepsilon}/K+\varepsilon^2) O ( T 1 + ε / K + ε 2 ) one-bit pricing queries by *only issuing these queries in A 2 ε ( D 0 ) A_{2\varepsilon}(\mathcal{D}_0) A 2 ε ( D 0 ) . Stated differently, our target prices are in A ε ( D 0 ) A_{\varepsilon}(\mathcal{D}_0) A ε ( D 0 ) , while our allowed queries are in A 2 ε ( D 0 ) A_{2\varepsilon}(\mathcal{D}_0) A 2 ε ( D 0 ) . Before providing a proof sketch of the sampling result, we showcase its applicability by arguing that it immediately induces an optimal algorithm for the stochastic setting.
Warm-up Result (Stochastic Setting). Consider the problem of learning the profit-maximizing posted prices in bilateral trade in the stochastic setting. There exists a learning algorithm which enjoys O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret.
Proof sketch. Set K ≈ T 1 / 3 K \approx T^{1/3} K ≈ T 1/3 , H ≈ log K H \approx\log K H ≈ log K , and initialize precision scales ε h = 4 − h \varepsilon_h = 4^{-h} ε h = 4 − h and failure probability δ h = 1 / T 2 H \delta_h = 1/T^2H δ h = 1/ T 2 H for h = 0 , … , H h = 0, \ldots, H h = 0 , … , H . The algorithm follows a successive-elimination approach and proceeds in phases. In the initial phase, it issues Θ ~ ( 1 ) \widetilde{\Theta}(1) Θ ( 1 ) queries for each price in G K G_K G K , to obtain a uniform 1 / 256 1/256 1/256 -approximation of all profits. At the generic phase h h h , it chooses any valuation distribution Q h Q_h Q h consistent with all previous estimates and uses it as a reference for the Localized Sampling Lemma (Main Result 1) to estimate profits on A ε h ( Q h ) A_{\varepsilon_h}(Q_h) A ε h ( Q h ) with precision ε h / 64 \varepsilon_h/64 ε h /64 , only posting prices in A 2 ε h ( Q h ) A_{2\varepsilon_h}(Q_h) A 2 ε h ( Q h ) .
The crucial observation now is that, conditioning on the good event that all calls to the Localized Sampling Lemma and the initial concentrations yield correct estimates, in each phase h h h the algorithm only plays prices in A 2 ε h ( Q h ) ⊆ A C ⋅ ε h ( D ) A_{2\varepsilon_h}(Q_h) \subseteq A_{C\cdot\varepsilon_h}(\mathcal{D}) A 2 ε h ( Q h ) ⊆ A C ⋅ ε h ( D ) , for some constant C C C , thus suffering instantaneous regret which is O ~ ( ε h ) \widetilde{O}(\varepsilon_h) O ( ε h ) at each time step. Moreover, the clean event also tells us that “good prices” do not get discarded along the way, so that A ε h + 1 ( Q h + 1 ) ⊇ A c ⋅ ε h ( D ) A_{\varepsilon_{h+1}}(Q_{h+1}) \supseteq A_{c\cdot\varepsilon_h}(\mathcal{D}) A ε h + 1 ( Q h + 1 ) ⊇ A c ⋅ ε h ( D ) for some constant c ≤ C c \le C c ≤ C .
Conditioning on the clean event (which, for our choice of the δ h \delta_h δ h , has probability at least 1 − 1 / T 1 - 1/T 1 − 1/ T ), we have the following bound on the regret (omitting poly-logarithmic terms):
∑ h = 0 H ε h ⋅ N ( ε h , δ h ) ⏟ Regret in Phase h + T K ⏟ Discretization Error ≈ ∑ h = 0 H ( K ε h + 1 ε h 2 ) + T K ≈ K 2 + T K ≈ T 2 / 3 . \sum_{h=0}^{H} \underbrace{\varepsilon_h \cdot N(\varepsilon_h,\delta_h)}_{\text{Regret in Phase }h}+\underbrace{\frac{T}{K}}_{\text{Discretization Error}} \approx\sum_{h=0}^{H}\left(\frac{K}{\varepsilon_h}+\frac{1}{\varepsilon_h^2}\right)+\frac{T}{K}\approx K^2+\frac{T}{K}\approx T^{2/3}. h = 0 ∑ H Regret in Phase h ε h ⋅ N ( ε h , δ h ) + Discretization Error K T ≈ h = 0 ∑ H ( ε h K + ε h 2 1 ) + K T ≈ K 2 + K T ≈ T 2/3 .
where in the last step we set K ≈ T 1 / 3 K \approx T^{1/3} K ≈ T 1/3 . The complement of the clean event gives constant expected regret, thus concluding the proof. For simplicity, the full confidence-set construction and analysis appear in Appendix Appendix § 12 .
Proof sketch of Main Result 1. Write μ 0 = μ D 0 \mu_0 = \mu_{\mathcal{D}_0} μ 0 = μ D 0 for the expected profit under the known reference distribution. Let D 0 , s \mathcal{D}_{0,s} D 0 , s and D 0 , b \mathcal{D}_{0,b} D 0 , b denote the marginal distributions of D 0 \mathcal{D}_0 D 0 for the seller and buyer, respectively. With L \mathcal{L} L denoting Lebesgue measure, define the auxiliary measure ν \nu ν over [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 as ν = L ⊗ D 0 , b + D 0 , s ⊗ L + D 0 \nu= \mathcal{L} \otimes\mathcal{D}_{0,b} + \mathcal{D}_{0,s} \otimes\mathcal{L} + \mathcal{D}_0 ν = L ⊗ D 0 , b + D 0 , s ⊗ L + D 0 , so that ν ( [ 0 , 1 ] 2 ) = 3 \nu([0,1]^2) = 3 ν ([ 0 , 1 ] 2 ) = 3 . This measure controls how the relative profit of two seller prices changes as the buyer price varies: for any ℓ ≤ r \ell\le r ℓ ≤ r and q 0 ≤ q 1 q_0 \le q_1 q 0 ≤ q 1 , it indeed holds that
∣ [ μ 0 ( r , q 1 ) − μ 0 ( ℓ , q 1 ) ] − [ μ 0 ( r , q 0 ) − μ 0 ( ℓ , q 0 ) ] ∣ ≤ ν ( ( ℓ , r ] × [ q 0 , q 1 ] ) . (1) \left|[\mu_0(r,q_1)-\mu_0(\ell,q_1)]-[\mu_0(r,q_0)-\mu_0(\ell,q_0)]\right|\leq\nu((\ell,r]\times[q_0,q_1]). \tag*{(1)} ∣ [ μ 0 ( r , q 1 ) − μ 0 ( ℓ , q 1 )] − [ μ 0 ( r , q 0 ) − μ 0 ( ℓ , q 0 )] ∣ ≤ ν (( ℓ , r ] × [ q 0 , q 1 ]) . (1)
A seller-price tree. First, we can organize the seller grid { 0 , 1 / K , 2 / K , … , 1 } \{0,1/K,2/K,\ldots,1\} { 0 , 1/ K , 2/ K , … , 1 } into a balanced binary tree, where each node corresponds to an interval: the root corresponds to { 0 , … , 1 } \{0,\ldots,1\} { 0 , … , 1 } , its two children to { 0 , … , 1 / 2 } \{0,\ldots,1/2\} { 0 , … , 1/2 } , { 1 / 2 + 1 / K , … , 1 } \{1/2+1/K,\ldots,1\} { 1/2 + 1/ K , … , 1 } , and so on, up to the K + 1 K+1 K + 1 leaves which correspond to singletons of the form { i / K } \{i/K\} { i / K } . For each node I I I of the tree, we can use the auxiliary measure ν \nu ν to partition the buyer grid into maximal consecutive blocks D 1 I , D 2 I , … D^I_1,D^I_2,\ldots D 1 I , D 2 I , … , satisfying ν ( ( min I , max I ] × [ min D j I , max D j I ] ) ≤ ε \nu((\min I,\max I]\times[\min D^I_j,\max D^I_j])\leq\varepsilon ν (( min I , max I ] × [ min D j I , max D j I ]) ≤ ε . With this construction, each pair of the form ( I , D j I ) (I,D^I_j) ( I , D j I ) corresponds to a rectangle I × D j I ⊆ [ 0 , 1 ] 2 I\times D^I_j\subseteq[0,1]^2 I × D j I ⊆ [ 0 , 1 ] 2 , and can be associated with a representative pair of prices as follows: choose the smallest seller price p ( I , D j I ) ∈ I p_{(I,D^I_j)}\in I p ( I , D j I ) ∈ I maximizing reference profit at q ( I , D j I ) = min D j I q_{(I,D^I_j)}=\min D^I_j q ( I , D j I ) = min D j I . Inequality (1) ensures that this seller stays within ε \varepsilon ε of the best seller in I I I throughout D j I D^I_j D j I . Thus every ε \varepsilon ε -optimal target price pair ( p , q ) ∈ I × D j I (p,q)\in I\times D^I_j ( p , q ) ∈ I × D j I has a 2 ε 2\varepsilon 2 ε -optimal queried pair ( p ( I , D j I ) , q ) (p_{(I,D^I_j)},q) ( p ( I , D j I ) , q ) . Figure 1 shows a target’s seller-tree path, from I 0 = { 0 , 1 / K , 2 / K , … , 1 } I_0=\{0,1/K,2/K,\ldots,1\} I 0 = { 0 , 1/ K , 2/ K , … , 1 } to { p } \{p\} { p } , along with the associated buyer blocks.
Figure 1. A possible sequence of seller and buyer blocks involved in the estimation of the probability function of trade for two specific targets in A \mathcal{A} A , according to Equation (2) .
Sharing estimates. Let P P P be the unknown probability function of both seller and buyer accepting a trade. For a target ( p , q ) (p,q) ( p , q ) in the action space A = G K ∩ T \mathcal{A}=G_K\cap\mathcal{T} A = G K ∩ T , follow its seller-tree path to the leaf { p } \{p\} { p } , and let p j p_j p j be the representative seller in the block containing q q q at depth j j j . We have a convenient telescoping decomposition
P ( p , q ) = P ( p 0 , q ) + ∑ j = 1 h [ P ( p j , q ) − P ( p j − 1 , q ) ] . (2) P(p,q)=P(p_0,q)+\sum_{j=1}^{h}[P(p_j,q)-P(p_{j-1},q)]. \tag*{(2)} P ( p , q ) = P ( p 0 , q ) + j = 1 ∑ h [ P ( p j , q ) − P ( p j − 1 , q )] . (2)
Each root buyer block defines one function q ↦ P ( p 0 , q ) q\mapsto P(p_0,q) q ↦ P ( p 0 , q ) . For each parent–child edge, consider the overlap D j − 1 ∩ D j D_{j-1}\cap D_j D j − 1 ∩ D j of the buyer blocks illustrated in Figure 1 : both representatives are fixed on this overlap, so defines one function q ↦ P ( p j , q ) − P ( p j − 1 , q ) q \mapsto P(p_j,q)-P(p_{j-1},q) q ↦ P ( p j , q ) − P ( p j − 1 , q ) . We call these shared functions : targets using the same root block or the same parent–child block overlap reuse the same estimates. Each difference is a signed seller-strip probability, monotone in q q q , and can be observed without bias using two fresh trade bits. When estimating a shared function, retain only buyer coordinates q q q for which some seller price p p p in the child interval satisfies ( p , q ) ∈ A ε ( D 0 ) (p,q)\in A_\varepsilon(D_0) ( p , q ) ∈ A ε ( D 0 ) . This target also belongs to the parent interval, so the representative guarantee makes both queried pairs 2 ε 2\varepsilon 2 ε -optimal under D 0 D_0 D 0 . Apply the analogous restriction at the root. Thus all queries lie in the permitted set, and every desired target can still be reconstructed using the telescoping decomposition in Equation (2) .
Controlling the sample count. The constant mass of ν \nu ν bounds the total number of blocks, and hence the number of shared functions, by M = O ( K + ε − 1 log K ) M=O(K+\varepsilon^{-1}\log K) M = O ( K + ε − 1 log K ) . So far, we have only used explicitly the reference distribution D 0 D_0 D 0 , while D D D has only been used in the telescopic decomposition. Let W W W denote the sum of the variations of all shared and root functions induced by the decomposition, where the variation of a monotone function is just the absolute difference between its first and last values. Under D D D , the sum of variations W W W belongs to O ( log K ) O(\log K) O ( log K ) : at each tree depth, each unit of probability mass contributes to at most two child corrections. Thus the reference law controls query profitability and the number of functions, while the unknown law controls their total variation. Monotonicity allows us to estimate a given shared function by sampling endpoints and refining only intervals with appreciable variation. For M M M functions of total variation W W W , this costs O ~ ( M / α 2 + W / α 3 ) \widetilde{O}(M/\alpha^2+W/\alpha^3) O ( M / α 2 + W / α 3 ) observations at accuracy α \alpha α (Claim 4.10). Taking α ∈ Θ ( ε / log K ) \alpha\in\Theta(\varepsilon/\log K) α ∈ Θ ( ε / log K ) small enough makes the errors along every path sum to at most ε / 64 \varepsilon/64 ε /64 , at the claimed cost. Multiplication by the known spread q − p ∈ [ 0 , 1 ] q-p\in[0,1] q − p ∈ [ 0 , 1 ] gives the profit estimates. The full construction is in Lemma 4.8 ; the smaller error constant and confidence 1 − δ 1-\delta 1 − δ are obtained by increasing sampling precision, without changing query locality.
O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) Regret for the Adversarial Setting
In this section we sketch the adversarial O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) algorithm, whose full description and analysis are deferred to § 4 . Note that here the sequence of valuations ( s 1 , b 1 ) , … , ( s T , b T ) (s_1,b_1),\ldots,(s_T,b_T) ( s 1 , b 1 ) , … , ( s T , b T ) is fixed in advance, so that there is no stationary distribution to learn. Instead, our algorithm maintains weights on possible prices to compare against, uses them to choose the price to post, and updates them using each trade bit. We describe how these weights are computed. To this end, we first introduce an auxiliary one-round game, between the learner and an adversary. Then, we analyze this game using the Localized Sampling Lemma, and finally we show how the resulting insight can be used to update the actual adversarial algorithm.
An Auxiliary One-Round Game. We present here a one-round game between the learner and an adversary. This adversary chooses a valuation pair in G K G_K G K and a comparator a ′ ∈ A a' \in\mathcal{A} a ′ ∈ A , while the learner receives in input probability weights w w w on A \mathcal{A} A and makes a proposal : it chooses a posted price a ∈ A a \in\mathcal{A} a ∈ A and two possible update vectors Q 0 , Q 1 Q_0,Q_1 Q 0 , Q 1 , one for each possible feedback it may receive. Observing the trade bit z z z then sets the next weights to Q z Q_z Q z . We denote with z a ( v ) z_a(v) z a ( v ) the trade bit for valuation v v v and action a a a . The goal of the learner is to make a randomized proposal which respects the following inequality, uniformly over all possible choices of the adversary:
max v ∈ G K , a ′ ∈ A E [ Profit ( v , a ′ ) − Profit ( v , a ) − β log Q z a ( v ) ( a ′ ) w ( a ′ ) ] ≤ c K . (3) \max_{v\in G_K,\ a'\in\mathcal{A}} \mathbb{E}\left[\operatorname{Profit}(v,a')-\operatorname{Profit}(v,a)-\beta\log\frac{Q_{z_a(v)}(a')}{w(a')}\right]\leq\frac{c}{K}. \tag*{(3)} v ∈ G K , a ′ ∈ A max E [ Profit ( v , a ′ ) − Profit ( v , a ) − β log w ( a ′ ) Q z a ( v ) ( a ′ ) ] ≤ K c . (3)
with c c c universal constant and β ∈ Θ ~ ( K 2 ) \beta\in\widetilde{\Theta}(K^2) β ∈ Θ ( K 2 ) . Note, the left-hand-side of the inequality is a variant of the Algorithmic Information Ratio from Xu and Zeevi [44 ] , while the expectation is only over the learner’s randomized proposal.
The Information Gain bounds the Logarithmic Term. To show that there always exists a learner proposal respecting Equation (3) , we adopt a dual perspective on the game. There, the adversary first chooses a distribution ρ \rho ρ over valuation–comparator pairs in G K × A G_K \times\mathcal{A} G K × A , and the learner responds with a deterministic proposal ( a , Q 0 , Q 1 ) (a,Q_0,Q_1) ( a , Q 0 , Q 1 ) that minimizes the expected profit gap minus the weighted logarithmic update in (3) , evaluated under ρ \rho ρ .
Let’s focus on the logarithmic term in the objective. For a fixed action a a a , consider the ideal posterior updates Q z ( a ′ ) = P ρ ( A = a ′ ∣ z a ( V ) = z ) Q_z(a')=\mathbb{P}_{\rho}(A=a'\mid z_a(V)=z) Q z ( a ′ ) = P ρ ( A = a ′ ∣ z a ( V ) = z ) . Thus Q z Q_z Q z is the conditional distribution of the comparator after observing feedback z z z . For this choice,
E ( V , A ) ∼ ρ [ log Q z a ( V ) ( A ) w ( A ) ] = I ρ ( A ; z a ( V ) ) ⏟ I ρ ( a ) + D K L ( ρ A ∥ w ) ≥ I ρ ( a ) , (4) \mathbb{E}_{(V,A)\sim\rho}\left[\log\frac{Q_{z_a(V)}(A)}{w(A)}\right]
=
\underbrace{I_{\rho}(A;z_a(V))}_{I_{\rho}(a)}
+D_{\mathrm{KL}}(\rho_A\mathbin{\|}w)
\geq I_{\rho}(a), \tag*{(4)} E ( V , A ) ∼ ρ [ log w ( A ) Q z a ( V ) ( A ) ] = I ρ ( a ) I ρ ( A ; z a ( V )) + D KL ( ρ A ∥ w ) ≥ I ρ ( a ) , (4)
where ρ A \rho_A ρ A is the marginal distribution of A A A , and I ρ ( a ) I_{\rho}(a) I ρ ( a ) is the mutual information between A A A and the feedback bit z a ( V ) z_a(V) z a ( V ) under ρ \rho ρ . Crucially, the inequality in (4) shows that mutual information measures a part of the expected logarithmic weight increase. It quantifies how much the trade bit distinguishes the possible comparison prices under this auxiliary law. All in all, to argue that Equation (3) holds, it remains to show that for any distribution ρ \rho ρ , there exists a ∈ A a\in\mathcal{A} a ∈ A such that Δ ρ ( a ) − β I ρ ( a ) ≤ c / K \Delta_{\rho}(a)-\beta I_{\rho}(a)\leq c/K Δ ρ ( a ) − β I ρ ( a ) ≤ c / K , where the quantity being bounded is the parameterized information ratio from Foster et al. [23 ] and Δ ρ ( a ) \Delta_{\rho}(a) Δ ρ ( a ) is a shorthand for E ρ [ Profit ( V , A ) − Profit ( V , a ) ] \mathbb{E}_{\rho}[\operatorname{Profit}(V,A)-\operatorname{Profit}(V,a)] E ρ [ Profit ( V , A ) − Profit ( V , a )] . Indeed, here we are essentially using a known relationship between the parameterized and the algorithmic information ratios (see Lemma 2.7 in Xu and Zeevi [44 ] ).
Inexpensive Detection by Local Sampling. We first show how to detect an increase in optimal profit using queries that are inexpensive under a reference distribution. Fix ρ \rho ρ and let D 0 \mathcal{D}_0 D 0 be its marginal valuation distribution. We supply D 0 \mathcal{D}_0 D 0 as the known reference to the Localized Sampling Lemma. For an improvement threshold x ∈ ( 0 , 1 ] x\in(0,1] x ∈ ( 0 , 1 ] , our goal is to distinguish D 0 \mathcal{D}_0 D 0 from every distribution D \mathcal{D} D satisfying
max a ∈ A μ D ( a ) ≥ max a ∈ A μ D 0 ( a ) + x . \max_{a\in\mathcal{A}}\mu_{\mathcal{D}}(a)\geq\max_{a\in\mathcal{A}}\mu_{\mathcal{D}_0}(a)+x. a ∈ A max μ D ( a ) ≥ a ∈ A max μ D 0 ( a ) + x .
Apply Main Result 1 at precisions x , 2 x , 4 x , … x,2x,4x,\ldots x , 2 x , 4 x , … , capped at 1 1 1 , with failure probability 1 / 10 1/10 1/10 per call. The threshold x x x remains fixed throughout; only the sampling precision increases. The first call estimates profits on A x ( D 0 ) A_x(\mathcal{D}_0) A x ( D 0 ) using queries in A 2 x ( D 0 ) A_{2x}(\mathcal{D}_0) A 2 x ( D 0 ) , and subsequent calls use the corresponding larger sets. In every call, the reference is D 0 \mathcal{D}_0 D 0 . When feedback comes from D 0 \mathcal{D}_0 D 0 , the estimates approximate μ D 0 \mu_{\mathcal{D}_0} μ D 0 ; when feedback comes from D \mathcal{D} D , they approximate μ D \mu_{\mathcal{D}} μ D . We compare these two executions with the same reference, target sets, and querying rules. Consider all sampling experiments with feedback generated from D 0 \mathcal{D}_0 D 0 and randomized as in Remark 4.12, and define
n x ( a ) = 2 E D 0 [ total queries at a across all precision scales ] . n_x(a)=2\mathbb{E}_{\mathcal{D}_0}\left[\text{total queries at }a\text{ across all precision scales}\right]. n x ( a ) = 2 E D 0 [ total queries at a across all precision scales ] .
We want to bound ∑ a ∈ A n x ( a ) ( Δ D 0 ( a ) + x ) \sum_{a \in\mathcal{A}} n_x(a)(\Delta_{\mathcal{D}_0}(a)+x) ∑ a ∈ A n x ( a ) ( Δ D 0 ( a ) + x ) . By linearity of expectation, this is twice the expected sum of Δ D 0 ( a ) + x \Delta_{\mathcal{D}_0}(a)+x Δ D 0 ( a ) + x over all queries. We bound these contributions one call at a time. In the first call, Main Result 1 guarantees that every queried price has reference gap at most 2 x 2x 2 x , so each contributes at most 3 x 3x 3 x . More generally, the additional term remains x x x , and each query contributes at most three times its call’s precision. Multiplying the first call’s sample bound from Main Result 1 by 3 x 3x 3 x gives, with logarithmic factors as in its full proof,
3 x ⋅ O ( log 3 K ( K x 2 + log 3 K x 3 ) ) = O ( log 3 K ( K x + log 3 K x 2 ) ) . 3x \cdot O\left(\log^3 K\left(\frac{K}{x^2}+\frac{\log^3 K}{x^3}\right)\right)=O\left(\log^3 K\left(\frac{K}{x}+\frac{\log^3 K}{x^2}\right)\right). 3 x ⋅ O ( log 3 K ( x 2 K + x 3 log 3 K ) ) = O ( log 3 K ( x K + x 2 log 3 K ) ) .
The corresponding bounds at the larger precisions decrease geometrically. We obtain
∑ a ∈ A n x ( a ) ( Δ D 0 ( a ) + x ) ≤ C log 3 K ( K x + log 3 K x 2 ) . \sum_{a\in\mathcal{A}}n_x(a)(\Delta_{\mathcal{D}_0}(a)+x)\le C\log^3 K\left(\frac{K}{x}+\frac{\log^3 K}{x^2}\right). a ∈ A ∑ n x ( a ) ( Δ D 0 ( a ) + x ) ≤ C log 3 K ( x K + x 2 log 3 K ) .
Because the expected query counts are taken under D 0 \mathcal{D}_0 D 0 , these weights depend only on D 0 \mathcal{D}_0 D 0 and x x x , and are the same for every alternative distribution.
Next, fix an improving distribution D \mathcal{D} D and let a ′ a' a ′ maximize its grid profit. Although a ′ a' a ′ is optimal under D \mathcal{D} D , it may lie outside A x ( D 0 ) A_x(\mathcal{D}_0) A x ( D 0 ) , where the first call guarantees accurate estimates. We therefore choose the first call whose precision is at least Δ D 0 ( a ′ ) \Delta_{\mathcal{D}_0}(a') Δ D 0 ( a ′ ) , so that its target set includes a ′ a' a ′ . The improvement assumption gives
μ D ( a ′ ) − μ D 0 ( a ′ ) ≥ x + Δ D 0 ( a ′ ) . \mu_{\mathcal{D}}(a')-\mu_{\mathcal{D}_0}(a')\ge x+\Delta_{\mathcal{D}_0}(a'). μ D ( a ′ ) − μ D 0 ( a ′ ) ≥ x + Δ D 0 ( a ′ ) .
Because we choose the first such call, this increase exceeds twice its estimation error. By Main Result 1, the returned estimate is close to μ D 0 ( a ′ ) \mu_{\mathcal{D}_0}(a') μ D 0 ( a ′ ) when feedback comes from D 0 \mathcal{D}_0 D 0 , and close to μ D ( a ′ ) \mu_{\mathcal{D}}(a') μ D ( a ′ ) when feedback comes from D \mathcal{D} D , each with probability at least 9 / 10 9/10 9/10 . These two accuracy intervals are disjoint. Hence the event that the estimate lies within the prescribed error of μ D 0 ( a ′ ) \mu_{\mathcal{D}_0}(a') μ D 0 ( a ′ ) has probability at least 9 / 10 9/10 9/10 under D 0 \mathcal{D}_0 D 0 , but at most 1 / 10 1/10 1/10 under D \mathcal{D} D .
Write P 0 P_0 P 0 and P P P for the trade probabilities under D 0 \mathcal{D}_0 D 0 and D \mathcal{D} D . Since the returned estimates are built from trade observations, their different behavior must come from differences in trade probabilities at the sampled prices. The proof quantifies these differences, giving greater weight to prices queried more often under D 0 \mathcal{D}_0 D 0 , and obtains
∑ a ∈ A n x ( a ) ( P ( a ) − P 0 ( a ) ) 2 ≥ 1. \sum_{a\in\mathcal{A}}n_x(a)(P(a)-P_0(a))^2\ge1. a ∈ A ∑ n x ( a ) ( P ( a ) − P 0 ( a ) ) 2 ≥ 1.
Together with the sampling-cost bound, this proves Lemma 4.5 in § 4 : the same inexpensive weights detect every distribution whose optimal profit exceeds the reference optimum by at least x x x .
From Detection to the Information Inequality. We seek a price whose information gain compensates for its regret. Let a ⋆ a^\star a ⋆ maximize profit under D 0 \mathcal{D}_0 D 0 , and let X A X_A X A be the positive increase in optimal grid profit after conditioning on A A A : X A = max a ( E ρ [ Profit ( V , a ) ∣ A ] − E ρ [ Profit ( V , a ⋆ ) ] ) + X_A=\max_a(\mathbb{E}_{\rho}[\operatorname{Profit}(V,a)\mid A]-\mathbb{E}_{\rho}[\operatorname{Profit}(V,a^\star)])_+ X A = max a ( E ρ [ Profit ( V , a ) ∣ A ] − E ρ [ Profit ( V , a ⋆ )] ) + . Since the conditional optimum is at least the profit of playing A A A itself,
Δ ρ ( a ⋆ ) ≤ E A [ X A ] . \Delta_{\rho}(a^\star)\le\mathbb{E}_A[X_A]. Δ ρ ( a ⋆ ) ≤ E A [ X A ] .
Thus a large comparator advantage requires substantial conditional profit improvements. Detection forces these improvements to produce differences in trade probabilities. Averaging over A A A with the same sampling weights, and applying Pinsker’s inequality, gives
P ( X A ≥ x ) ≤ 1 2 ∑ a ∈ A n x ( a ) I ρ ( a ) . \mathbb{P}(X_A\ge x)\le\frac{1}{2}\sum_{a\in\mathcal{A}}n_x(a)\mathcal{I}_{\rho}(a). P ( X A ≥ x ) ≤ 2 1 a ∈ A ∑ n x ( a ) I ρ ( a ) .
Suppose every action violates the desired bound, so Δ ρ ( a ) − β I ρ ( a ) > c / K \Delta_{\rho}(a)-\beta\mathcal{I}_{\rho}(a)>c/K Δ ρ ( a ) − β I ρ ( a ) > c / K . Then I ρ ( a ) < Δ ρ ( a ) / β \mathcal{I}_{\rho}(a)<\Delta_{\rho}(a)/\beta I ρ ( a ) < Δ ρ ( a ) / β : information is small relative to regret at every price. Since Δ ρ ( a ) = Δ D 0 ( a ) + Δ ρ ( a ⋆ ) \Delta_{\rho}(a)=\Delta_{\mathcal{D}_0}(a)+\Delta_{\rho}(a^\star) Δ ρ ( a ) = Δ D 0 ( a ) + Δ ρ ( a ⋆ ) , for x ≥ Δ ρ ( a ⋆ ) / 2 x\ge\Delta_{\rho}(a^\star)/2 x ≥ Δ ρ ( a ⋆ ) /2 the sampling charge Δ D 0 ( a ) + x \Delta_{\mathcal{D}_0}(a)+x Δ D 0 ( a ) + x controls Δ ρ ( a ) \Delta_{\rho}(a) Δ ρ ( a ) within a factor of two. Integrating the resulting tail bound above this cutoff, and using probability at most one below it, gives E A [ X A ] < Δ ρ ( a ⋆ ) \mathbb{E}_A[X_A]<\Delta_{\rho}(a^\star) E A [ X A ] < Δ ρ ( a ⋆ ) for β = K 2 log 6 K \beta=K^2\log^6 K β = K 2 log 6 K and sufficiently large c c c , a contradiction. Hence
min a ∈ A { Δ ρ ( a ) − β I ρ ( a ) } ≤ c K . \min_{a\in\mathcal{A}}\{\Delta_{\rho}(a)-\beta\mathcal{I}_{\rho}(a)\}\le\frac{c}{K}. a ∈ A min { Δ ρ ( a ) − β I ρ ( a )} ≤ K c .
Final Regret Bound. At this point, we have all the ingredients to present a sketch of the result for the adversarial setting. Note, the one-round game we have analyzed so far is just an analytic tool used in guiding the design and analysis of our algorithm, but it is not (directly) how the algorithm itself works.
Main Result 2 (Adversarial Setting). Consider the problem of learning the profit-maximizing posted prices in bilateral trade in the adversarial setting. There exists a learning algorithm which enjoys O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret.
Proof Sketch. In the adversarial setting there is no notion of distribution ρ \rho ρ over valuation-comparator pairs which we can optimize against. However, we can maintain the bound in Equation (3) by using multiplicative weight updates [2 ] to find a good (randomized) action/price against any possible adversarial response (see Lemma 4.6 for the formal statement). The adversarial algorithm maintains weights w t w_t w t over the prices in the action space A \mathcal{A} A , starting from uniform at time 1. At each time step, it calls the multiplicative weight updates routine underlying the auxiliary one-round game with weights w t w_t w t , in order to obtain a randomized proposal ( a t , Q 0 , Q 1 ) (a_t,Q_0,Q_1) ( a t , Q 0 , Q 1 ) . It then posts a t a_t a t , observes feedback z t z_t z t , and sets w t + 1 = Q z t w_{t+1}=Q_{z_t} w t + 1 = Q z t . Fix now any comparator action a ′ ∈ A a' \in\mathcal{A} a ′ ∈ A . For any time step t t t , and any fixed history up to that point, the bound in Equation (3) thus can be rewritten as:
E [ Profit t ( a ′ ) − Profit t ( a t ) ] ≲ β E [ log w t + 1 ( a ′ ) w t ( a ′ ) ] + c K . \mathbb{E}\left[\operatorname{Profit}_t(a')-\operatorname{Profit}_t(a_t)\right]\lesssim\beta\mathbb{E}\left[\log\frac{w_{t+1}(a')}{w_t(a')}\right]+\frac{c}{K}. E [ Profit t ( a ′ ) − Profit t ( a t ) ] ≲ β E [ log w t ( a ′ ) w t + 1 ( a ′ ) ] + K c .
Both expectations are over the proposal’s randomness, conditional on the current history. By summing up the contributions of each time step, and by the law of total probability, we then get that:
∑ t = 1 T E [ Profit t ( a ′ ) − Profit t ( a t ) ] ≲ β E [ ∑ t = 1 T log w t + 1 ( a ′ ) w t ( a ′ ) ] + T K = β E [ log w T + 1 ( a ′ ) w 1 ( a ′ ) ] + T K (Telescopic sum) ≤ β log ∣ A ∣ + T K . \begin{aligned}
\sum_{t=1}^{T}\mathbb{E}\left[\operatorname{Profit}_t(a')-\operatorname{Profit}_t(a_t)\right]\lesssim\beta\mathbb{E}\left[\sum_{t=1}^{T}\log\frac{w_{t+1}(a')}{w_t(a')}\right]+\frac{T}{K} \\
&=\beta\mathbb{E}\left[\log\frac{w_{T+1}(a')}{w_1(a')}\right]+\frac{T}{K}\qquad\text{(Telescopic sum)} \\
&\leq\beta\log|\mathcal{A}|+\frac{T}{K}.
\end{aligned} t = 1 ∑ T E [ Profit t ( a ′ ) − Profit t ( a t ) ] ≲ β E [ t = 1 ∑ T log w t ( a ′ ) w t + 1 ( a ′ ) ] + K T = β E [ log w 1 ( a ′ ) w T + 1 ( a ′ ) ] + K T (Telescopic sum) ≤ β log ∣ A ∣ + K T .
where the last inequality follows from the uniform initialization of w 1 w_1 w 1 , and w T + 1 ( a ′ ) ≤ 1 w_{T+1}(a')\leq1 w T + 1 ( a ′ ) ≤ 1 for all a ′ a' a ′ . Consequently, even the best price in hindsight has expected cumulative advantage at most O ( T / K ) + β log ∣ A ∣ O(T/K)+\beta\log|\mathcal{A}| O ( T / K ) + β log ∣ A ∣ . Adding discretization and taking K ≈ T 1 / 3 K\approx T^{1/3} K ≈ T 1/3 gives R T = O ~ ( T 2 / 3 ) R_T=\widetilde{O}(T^{2/3}) R T = O ( T 2/3 ) .
O ~ ( T ) \widetilde{O}(\sqrt{T}) O ( T ) Regret for Monotone-Hazard-Rate Distributions
We present and sketch the analysis of our learning algorithm for the special case of independent and MHR distributions. A formal presentation and regret analysis can be found in the main body, in Section 5. Crucially, our approach is based on a stochastic convex bandit routine [32 ] , applied to a suitable proxy for the profit objective, and employs an initial filtering procedure to “well condition” the convex problem. We start by introducing the stochastic convex bandit problem.
Stochastic Convex Bandits. In stochastic convex bandits, a learner is presented with a convex set K ⊆ [ 0 , 1 ] 2 \mathcal{K}\subseteq[0,1]^2 K ⊆ [ 0 , 1 ] 2 and a number n n n of interactions. For each one of these interactions, the learner selects a point x τ ∈ K x_\tau\in\mathcal{K} x τ ∈ K and suffers a loss L ( x τ ) \mathcal{L}(x_\tau) L ( x τ ) , while only observing noisy feedback L ( x τ ) + ξ τ \mathcal{L}(x_\tau)+\xi_\tau L ( x τ ) + ξ τ . The (unknown) function L \mathcal{L} L is continuous and convex, while the noise variables ξ 1 , ξ 2 , … \xi_1,\xi_2,\ldots ξ 1 , ξ 2 , … form a sequence of conditionally centered random variables (i.e., E [ ξ τ ∣ F τ ] = 0 \mathbb{E}[\xi_\tau\mid\mathcal{F}_\tau]=0 E [ ξ τ ∣ F τ ] = 0 ) with bounded ψ 2 \psi_2 ψ 2 -norm ∥ ξ τ ∥ ψ 2 = inf { t > 0 : E [ exp ( ξ τ 2 / t 2 ) ∣ F τ ] < 2 } 2 \|\xi_\tau\|_{\psi_2}=\inf\{t>0:\mathbb{E}[\exp(\xi_\tau^2/t^2)\mid\mathcal{F}_\tau]<2\}^2 ∥ ξ τ ∥ ψ 2 = inf { t > 0 : E [ exp ( ξ τ 2 / t 2 ) ∣ F τ ] < 2 } 2 . For this problem, we know that there exists a learning algorithm which enjoys the following properties, as an immediate corollary of Theorem 1 of Lattimore and György [32 ] (for a proof, we refer to Appendix § 10 .5).
Corollary 3.1 (Stochastic convex bandit). *Let C \mathcal{C} C be a known convex body of constant dimension, and let L : C → R L:\mathcal{C}\to\mathbb{R} L : C → R be continuous and convex, with values in a known interval of constant length. Suppose that querying x τ ∈ C x_\tau\in\mathcal{C} x τ ∈ C returns L ( x τ ) + ξ τ L(x_\tau)+\xi_\tau L ( x τ ) + ξ τ . Conditional on previous queries and observations, and on x τ x_\tau x τ , assume that ξ τ \xi_\tau ξ τ has mean zero and ψ 2 \psi_2 ψ 2 norm at most a known γ ≥ 0 \gamma\ge0 γ ≥ 0 . Then, for every n ≥ 1 n \ge1 n ≥ 1 , there is an n n n -query algorithm such that
E [ ∑ τ = 1 n ( L ( x τ ) − min x ∈ C L ( x ) ) ] ∈ O ( ( 1 + γ ) n log 2 ( n ) ) . \mathbb{E}\left[\sum_{\tau=1}^{n}\left(L(x_\tau)-\min_{x\in\mathcal{C}}L(x)\right)\right]\in O\left((1+\gamma)\sqrt{n}\log^2(n)\right). E [ τ = 1 ∑ n ( L ( x τ ) − x ∈ C min L ( x ) ) ] ∈ O ( ( 1 + γ ) n log 2 ( n ) ) .
where the expectation is taken over noise and randomness of the algorithm’s actions. The hidden constant depends only on the affine dimension and the length of the specified interval.
To translate bilateral trade into the language of stochastic convex optimization, we need to find a convex loss which relates to the expected profit and a suitable γ \gamma γ !
A Convex Objective. While the expected profit function μ ( p , q ) \mu(p,q) μ ( p , q ) is critically not concave or convex, the MHR condition implies that the log of the expected profit exhibits concavity. Indeed, we have the following claim, whose proof is deferred to Section 5:
Proposition 3.2. Under the MHR assumption, the loss function L ( p , q ) = − log μ ( p , q ) L(p,q)=-\log\mu(p,q) L ( p , q ) = − log μ ( p , q ) is continuous and convex in the interior of T \mathcal{T} T .
The loss L ( p , q ) L(p,q) L ( p , q ) is convex, so we can flip the problem from maximizing profit to minimizing this notion of loss. If we look at the bandit convex optimization setup, we still have two critical gaps to fill. First, this L L L has the disadvantage of being unbounded on its natural domain, as it diverges close to the boundary of T \mathcal{T} T . Moreover, it is not clear how to use the actual profit obtained by posting prices ( p , q ) (p,q) ( p , q ) to estimate the corresponding loss: the trivial estimator − log [ 1 { t r a d e } ( q − p ) ] -\log[\mathbb{1}_{\{\mathrm{trade}\}}(q-p)] − log [ 1 { trade } ( q − p )] is clearly out of the question.
A filtering procedure. To control the range of the loss function L L L , we start by pruning the action space T \mathcal{T} T with a first coarse uniform estimate of the profit function μ \mu μ . We can safely restrict our attention to the prices in the uniform grid G K G_K G K of stepsize 2 − H = 1 / K 2^{-H}=1/K 2 − H = 1/ K that belong to the triangle T \mathcal{T} T (we know that this only costs us an extra T / K T/K T / K term in the regret by one-sided Lipschitzianity of the profit function, as in Lemma A.6 ), and then proceed through decreasing guesses 1 , 1 / 2 , 1 / 4 , … , 2 − H 1,1/2,1/4,\ldots,2^{-H} 1 , 1/2 , 1/4 , … , 2 − H of the actual range of the optimal profit μ ⋆ \mu^\star μ ⋆ . For the generic guess ε h = 2 − h \varepsilon_h=2^{-h} ε h = 2 − h , the filtering procedure probes O ~ ( K ) \widetilde{O}(K) O ( K ) strategic prices, each one approximately 1 / ε h 1/\varepsilon_h 1/ ε h times, which are then used to provide estimates for all the prices in G K ∩ T G_K\cap\mathcal{T} G K ∩ T . Note that 1 / ε h 1/\varepsilon_h 1/ ε h samples are not enough to guarantee an additive precision of ε h \varepsilon_h ε h , but suffice for a constant multiplicative approximation as soon as the expected profit is approximately ε h \varepsilon_h ε h . If there exists at least one price on the grid whose current estimate μ ^ ( p , q ) \widehat{\mu}(p,q) μ ( p , q ) is safely comparable to the current guess ε h \varepsilon_h ε h , then the routine terminates and returns the convex hull K \mathcal{K} K of all the prices on the grid which are “close” to the guess. Note that, by convexity of L L L , we are sure that all prices in the convex hull are comparable to ε h \varepsilon_h ε h .
If all the guesses fail up to level H H H , corresponding to precision 1 / K 1/K 1/ K , then on the concentration event we have μ ⋆ = O ( 1 / K ) \mu^\star=O(1/K) μ ⋆ = O ( 1/ K ) . By contrast, if the filtering procedure terminates at the h h h -th precision, i.e., it triggers, then the regret incurred in this first phase is at most O ( ε h ) O(\varepsilon_h) O ( ε h ) times the number of probes exploited so far. Denoting by h ⋆ h^\star h ⋆ this last phase, we can then upper bound the regret suffered in this filtering phase as follows:
∑ h = 1 h ⋆ ε h ⋅ K ε h = K ⋅ 2 − h ⋆ ⋅ ∑ h = 1 h ⋆ 2 h ≈ K . \sum_{h=1}^{h^\star}\varepsilon_h\cdot\frac{K}{\varepsilon_h}=K\cdot2^{-h^\star}\cdot\sum_{h=1}^{h^\star}2^h\approx K. h = 1 ∑ h ⋆ ε h ⋅ ε h K = K ⋅ 2 − h ⋆ ⋅ h = 1 ∑ h ⋆ 2 h ≈ K .
The properties of the filtering procedure are summarized in the following qualitative lemma. For the exact statement and proof, we refer to Lemma 5.3.
Lemma 3.3. There exist constants 0 < c < C 0<c<C 0 < c < C such that, with high probability, the filtering procedure either does not trigger and μ ⋆ ≤ c / K \mu^\star\le c/K μ ⋆ ≤ c / K , or returns ε h ⋆ = 2 − h ⋆ ≥ 1 / K \varepsilon_{h^\star}=2^{-h^\star}\ge1/K ε h ⋆ = 2 − h ⋆ ≥ 1/ K and a convex body K ⊆ int ( T ) \mathcal{K}\subseteq\operatorname{int}(\mathcal{T}) K ⊆ int ( T ) satisfying:
(i) Every price ( p , q ) ∈ K (p,q)\in\mathcal{K} ( p , q ) ∈ K has expected profit μ ( p , q ) ≥ c ⋅ ε h ⋆ \mu(p,q)\ge c\cdot\varepsilon_{h^\star} μ ( p , q ) ≥ c ⋅ ε h ⋆ .
(ii) The returned scale approximates the optimal profit: ε h ⋆ ≤ μ ⋆ ≤ C ⋅ ε h ⋆ \varepsilon_{h^\star}\le\mu^\star\le C\cdot\varepsilon_{h^\star} ε h ⋆ ≤ μ ⋆ ≤ C ⋅ ε h ⋆ .
(iii) The set K \mathcal{K} K contains every maximizer of μ \mu μ over G K ∩ T G_K\cap\mathcal{T} G K ∩ T and therefore μ ⋆ ≤ max ( p , q ) ∈ K μ ( p , q ) + C / K \mu^\star\le\max_{(p,q)\in\mathcal{K}}\mu(p,q)+C/K μ ⋆ ≤ max ( p , q ) ∈ K μ ( p , q ) + C / K .
On the same high-probability event, if the procedure triggers, the regret incurred during filtering is O ~ ( K ) \widetilde{O}(K) O ( K ) .
Sierpinski Sampling. Before moving on to the next task, we highlight a cute property of our filtering procedure. Indeed, trivially estimating the expected profit of all points on the grid for a certain multiplicative precision would entail probing separately each one of those O ( K 2 ) O(K^2) O ( K 2 ) prices, while we are able to perform this task by probing only O ( K log K ) O(K \log K) O ( K log K ) prices. To decrease the number of prices to probe, we perform a tree decomposition of G K ∩ T G_K \cap T G K ∩ T such that for all prices ( i / K , j / K ) (i/K,j/K) ( i / K , j / K ) there exists an ℓ \ell ℓ such that the three points ( i / K , ℓ / K ) (i/K,\ell/K) ( i / K , ℓ / K ) , ( ℓ / K , ℓ / K ) (\ell/K,\ell/K) ( ℓ / K , ℓ / K ) , and ( ℓ / K , j / K ) (\ell/K,j/K) ( ℓ / K , j / K ) have been sampled, so that estimates P ^ \hat{P} P ^ are available for their corresponding probabilities of trade (see the two left-most plots in Figure 2 ). By independence, whenever P ( ℓ / K , ℓ / K ) > 0 P(\ell/K,\ell/K) > 0 P ( ℓ / K , ℓ / K ) > 0 , we have the following decomposition of the probability P ( i / K , j / K ) P(i/K,j/K) P ( i / K , j / K ) of trading (see the right-most plot of Figure 2 ):
Figure 2. Visualization of Sierpinski sampling. Left: sampled pairs for K = 16 K = 16 K = 16 ; color and shape identify the first sampling node in the interval tree (middle), and gray pairs are sampled later. Right: the first split separating i i i and j j j supplies three sampled pairs that determine P ( i / K , j / K ) P(i/K,j/K) P ( i / K , j / K ) by independence. Here ( i , j ) = ( 3 , 13 ) (i,j) = (3,13) ( i , j ) = ( 3 , 13 ) and the splitting midpoint is ℓ = 8 \ell= 8 ℓ = 8 .
P ( i / K , j / K ) = P ( i / K , ℓ / K ) ⋅ P ( ℓ / K , j / K ) P ( ℓ / K , ℓ / K ) ≈ P ^ ( i / K , ℓ / K ) ⋅ P ^ ( ℓ / K , j / K ) P ^ ( ℓ / K , ℓ / K ) = P ^ ( i / K , j / K ) . P(i/K,j/K) = \frac{P(i/K,\ell/K) \cdot P(\ell/K,j/K)}{P(\ell/K,\ell/K)} \approx\frac{\hat{P}(i/K,\ell/K) \cdot\hat{P}(\ell/K,j/K)}{\hat{P}(\ell/K,\ell/K)} = \hat{P}(i/K,j/K). P ( i / K , j / K ) = P ( ℓ / K , ℓ / K ) P ( i / K , ℓ / K ) ⋅ P ( ℓ / K , j / K ) ≈ P ^ ( ℓ / K , ℓ / K ) P ^ ( i / K , ℓ / K ) ⋅ P ^ ( ℓ / K , j / K ) = P ^ ( i / K , j / K ) .
Note that overflow due to a small denominator may be an issue. The analysis in Section 5 handles this by combining the concentration guarantees with a conveniently tweaked denominator.
A Suitable Estimator. At the end of the filtering procedure, assuming it did trigger, we have located a convex set of good prices whose expected profit is at least c ⋅ ε h ⋆ c \cdot\varepsilon h_\star c ⋅ ε h ⋆ (point (i) of Lemma 3.3 ) and at most C ⋅ ε h ⋆ C \cdot\varepsilon h_\star C ⋅ ε h ⋆ (point (ii) of Lemma 3.3 ), for suitable constants (or a certificate that μ ⋆ = O ( 1 / K ) \mu^\star= O(1/K) μ ⋆ = O ( 1/ K ) ). The logarithmic loss L ( p , q ) L(p,q) L ( p , q ) thus maps to an interval of constant length. To apply the bandit convex theorem above, we just need to find a feasible estimator for L L L and show that the relative “noise” ξ τ \xi_\tau ξ τ has small ψ 2 \psi_2 ψ 2 -norm.
Here is the trick: we do not use one single round of bilateral trade to simulate one round of the convex problem, but wait for the first success! To estimate L ( p , q ) L(p,q) L ( p , q ) , we repeatedly post prices ( p , q ) (p,q) ( p , q ) until we observe the first trade. Let N p , q N_{p,q} N p , q be the geometric random variable with parameter P ( p , q ) P(p,q) P ( p , q ) which counts the number of attempts needed to observe a trade. We then have the following unbiased estimator [17 ] :
L ^ ( p , q ) = H N p , q − 1 − log ( q − p ) , \hat{L}(p,q) = H_{N_{p,q}-1} - \log(q-p), L ^ ( p , q ) = H N p , q − 1 − log ( q − p ) ,
where H i H_i H i is the i i i th harmonic number (with H 0 = 0 H_0 = 0 H 0 = 0 ). At this point, we are just left with bounding the ψ 2 \psi_2 ψ 2 -norm of the “noise” random variable L ( p , q ) − L ^ ( p , q ) L(p,q) - \hat{L}(p,q) L ( p , q ) − L ^ ( p , q ) , which can be done by explicitly using the geometric nature of N p , q N_{p,q} N p , q . In particular, it is possible to show that ∥ L ( p , q ) − L ^ ( p , q ) ∥ ψ 2 ∈ O ( log K ) \lVert L(p,q) - \hat{L}(p,q)\rVert_{\psi_2} \in O(\sqrt{\log K}) ∥ L ( p , q ) − L ^ ( p , q ) ∥ ψ 2 ∈ O ( log K ) , using the fact that the parameter of the geometric distribution (i.e., the probability of observing a trade) is uniformly at least ε h ⋆ \varepsilon h_\star ε h ⋆ due to the guarantees of the filtering round. All in all, we can use the remaining rounds of bilateral trade (i.e., the ones left after the filtering procedure) to simulate this bandit convex problem, obtaining a bound on the (convex-problem) regret of the following type:
E [ ∑ τ = 1 n ( L ( p τ , q τ ) − min ( p , q ) ∈ K L ( p , q ) ) ] ∈ O ( n log K log 2 n ) . \mathbb{E}\left[\sum_{\tau=1}^{n}\left(L(p_\tau,q_\tau)-\min_{(p,q)\in\mathcal{K}}L(p,q)\right)\right] \in O(\sqrt{n\log K}\log^2 n). E [ τ = 1 ∑ n ( L ( p τ , q τ ) − ( p , q ) ∈ K min L ( p , q ) ) ] ∈ O ( n log K log 2 n ) .
We have all the ingredients for the main result of the section, which appears as Theorem 5.1 in Section 5.
Main Result 3 (MHR distributions). Consider the problem of learning the best prices in bilateral trade when the agents’ valuations are drawn from independent and MHR distributions. There exists a learning algorithm which enjoys O ~ ( T ) \widetilde{O}(\sqrt{T}) O ( T ) regret.
Proof Sketch. We are then left with relating the notion of regret for the convex problem above to the actual regret for the original bilateral trade problem we have been playing all along. In particular, we show how to relate the instantaneous regret L ( p τ , q τ ) − min ( p , q ) ∈ K L ( p , q ) L(p_\tau,q_\tau)-\min_{(p,q)\in\mathcal{K}}L(p,q) L ( p τ , q τ ) − min ( p , q ) ∈ K L ( p , q ) for the generic time step τ \tau τ of the convex problem to the regret suffered during the corresponding N p τ , q τ N_{p_\tau,q_\tau} N p τ , q τ rounds in the bilateral trade world, with respect to the optimal profit μ ∗ \mu^* μ ∗ . Consider the generic interaction τ \tau τ in the bandit convex optimization world, which corresponds to some random t τ t_\tau t τ in the bilateral trade world, and in which the bandit convex routine chooses to play prices ( p τ , q τ ) (p_\tau,q_\tau) ( p τ , q τ ) . This means that the bilateral trade algorithm plays ( p τ , q τ ) (p_\tau,q_\tau) ( p τ , q τ ) from time t τ t_\tau t τ to the stopping time t τ + N p τ , q τ − 1 t_\tau+N_{p_\tau,q_\tau}-1 t τ + N p τ , q τ − 1 . Conditioning on the past, we have the following, for ( p , q ) ∈ K (p,q)\in\mathcal{K} ( p , q ) ∈ K and letting μ K ∗ = max ( p , q ) ∈ K μ ( p , q ) \mu^*_{\mathcal{K}}=\max_{(p,q)\in\mathcal{K}}\mu(p,q) μ K ∗ = max ( p , q ) ∈ K μ ( p , q ) :
E [ ∑ t = t τ t τ + N p τ , q τ − 1 ( Profit ( p , q ) − Profit ( p t , q t ) ) ∣ F τ ] ≤ E [ N p τ , q τ ∣ F τ ] ( μ K ∗ − μ ( p τ , q τ ) ) (By Wald’s equality) = μ K ∗ − μ ( p τ , q τ ) P ( p τ , q τ ) ( N p τ , q τ ∼ Geom ( P ( p τ , q τ ) ) ) ≤ μ K ∗ − μ ( p τ , q τ ) μ ( p τ , q τ ) (As μ ( p , q ) ≤ P ( p , q ) ) ≲ log ( μ K ∗ μ ( p τ , q τ ) ) (As z − 1 ≲ log z for z bounded) = L ( p τ , q τ ) − min ( p , q ) ∈ K L ( p , q ) . \begin{aligned}
\mathbb{E}\left[\left.\sum_{t=t_\tau}^{t_\tau+N_{p_\tau,q_\tau}-1}\left(\operatorname{Profit}(p,q)-\operatorname{Profit}(p_t,q_t)\right)\right|\mathcal{F}_\tau\right]
&\leq\mathbb{E}\left[N_{p_\tau,q_\tau}\mid\mathcal{F}_\tau\right]\left(\mu^*_{\mathcal{K}}-\mu(p_\tau,q_\tau)\right) &&\text{(By Wald’s equality)}\\
&=\frac{\mu^*_{\mathcal{K}}-\mu(p_\tau,q_\tau)}{P(p_\tau,q_\tau)} &&\text{($N_{p_\tau,q_\tau}\sim\operatorname{Geom}(P(p_\tau,q_\tau))$)}\\
&\leq\frac{\mu^*_{\mathcal{K}}-\mu(p_\tau,q_\tau)}{\mu(p_\tau,q_\tau)} &&\text{(As $\mu(p,q)\leq P(p,q)$)}\\
&\lesssim\log\left(\frac{\mu^*_{\mathcal{K}}}{\mu(p_\tau,q_\tau)}\right) &&\text{(As $z-1\lesssim\log z$ for $z$ bounded)}\\
&=L(p_\tau,q_\tau)-\min_{(p,q)\in\mathcal{K}}L(p,q).
\end{aligned} E t = t τ ∑ t τ + N p τ , q τ − 1 ( Profit ( p , q ) − Profit ( p t , q t ) ) F τ ≤ E [ N p τ , q τ ∣ F τ ] ( μ K ∗ − μ ( p τ , q τ ) ) = P ( p τ , q τ ) μ K ∗ − μ ( p τ , q τ ) ≤ μ ( p τ , q τ ) μ K ∗ − μ ( p τ , q τ ) ≲ log ( μ ( p τ , q τ ) μ K ∗ ) = L ( p τ , q τ ) − ( p , q ) ∈ K min L ( p , q ) . (By Wald’s equality) ( N p τ , q τ ∼ Geom ( P ( p τ , q τ )) ) (As μ ( p , q ) ≤ P ( p , q ) ) (As z − 1 ≲ log z for z bounded)
Fix a filtering transcript satisfying the stated guarantees, and let T 0 T_0 T 0 be the remaining horizon. For analysis, continue the same ConOpt execution until all n = T 0 n=T_0 n = T 0 loss observations are complete. Each observation uses at least one trading round, so the actual execution is a prefix of this run. The regional mean-profit gaps μ K ∗ − μ ( p t , q t ) \mu^*_{\mathcal{K}}-\mu(p_t,q_t) μ K ∗ − μ ( p t , q t ) are nonnegative, so their sum on this prefix is at most their sum over the full run. Applying the preceding calculation and the fixed-n n n guarantee, then adding filtering regret and the approximation error in Lemma 3.3 (iii), gives
R T ≲ K + T K + T log K log 2 T ≈ T log 5 / 2 T , R_T\lesssim K+\frac{T}{K}+\sqrt{T\log K\log^2T}\approx\sqrt{T}\log^{5/2}T, R T ≲ K + K T + T log K log 2 T ≈ T log 5/2 T ,
which follows by choosing K ≈ T K\approx\sqrt{T} K ≈ T .
O ~ ( T 3 / 5 ) \widetilde{O}(T^{3/5}) O ( T 3/5 ) Regret for Regular Distributions
We now extend the approach of § 3.3 to independent regular distributions. The same stochastic convex bandit routine remains useful, and we still require an initial filtering procedure to learn the approximate scale of the optimal expected profit, as well as a safe region for the convex bandit routine to query. The main difference is that regularity no longer guarantees convexity of the negative log-profit. We instead construct a small family of convex problems, one of which approximates the original optimum, and progressively discard the less profitable candidates. The formal construction is given in Section 6.
A family of convex objectives. Under regularity, the reciprocals of the seller’s and buyer’s acceptance probabilities are convex. For each λ > 0 \lambda>0 λ > 0 and ( p , q ) ∈ int ( T ) (p,q)\in\operatorname{int}(T) ( p , q ) ∈ int ( T ) , define the auxiliary profit
r λ ( p , q ) = 4 ( q − p ) ( λ / F s ( p ) + λ − 1 / ( 1 − F b ( q ) ) ) 2 . r_\lambda(p,q)=\frac{4(q-p)}{\left(\lambda/F_s(p)+\lambda^{-1}/(1-F_b(q))\right)^2}. r λ ( p , q ) = ( λ / F s ( p ) + λ − 1 / ( 1 − F b ( q )) ) 2 4 ( q − p ) .
First, it is straightforward to show that 1 / r λ 1/r_\lambda 1/ r λ is convex, and r λ ≤ μ r_\lambda\le\mu r λ ≤ μ everywhere. Moreover, at any interior price pair x = ( p , q ) x=(p,q) x = ( p , q ) , choosing the balancing parameter λ x = F s ( p ) / ( 1 − F b ( q ) ) \lambda_x=\sqrt{F_s(p)/(1-F_b(q))} λ x = F s ( p ) / ( 1 − F b ( q )) gives r λ x ( x ) = μ ( x ) r_{\lambda_x}(x)=\mu(x) r λ x ( x ) = μ ( x ) . Thus optimizing the whole family recovers the optimal profit. Crucially, a small error in the logarithm of this parameter causes only a quadratic profit loss (see Lemma 6.2 ):
0 ≤ μ ( x ) − r λ ( x ) ≤ μ ( x ) log 2 λ / λ x . 0 \le\mu(x)-r_\lambda(x) \le\mu(x)\log^2 \lambda/\lambda_x. 0 ≤ μ ( x ) − r λ ( x ) ≤ μ ( x ) log 2 λ / λ x .
Filtering and choosing candidates. As in the MHR case, we first estimate the optimal profit scale, denoted by δ \delta δ , using the sparse grid reconstruction based on Sierpinski sampling. For a target accuracy η \eta η , take K ≈ 1 / η K \approx1/\eta K ≈ 1/ η . If the filtering procedure certifies that μ ⋆ = O ( η ) \mu^\star=O(\eta) μ ⋆ = O ( η ) , posting ( 0 , 0 ) (0,0) ( 0 , 0 ) incurs only O ( T η ) O(T\eta) O ( T η ) regret. Otherwise, we group promising grid prices with similar balancing parameters and take each group’s convex hull. Convexity of the reciprocal ensures that each hull remains profitable for its associated objectives. Discretizing logarithmic balancing parameters at a spacing of order η / δ \sqrt{\eta/\delta} η / δ produces only N = O ~ ( δ / η ) N=\widetilde{O}(\sqrt{\delta/\eta}) N = O ( δ / η ) candidates, including one whose auxiliary optimum is within O ( η ) O(\eta) O ( η ) of μ ⋆ \mu^\star μ ⋆ (Lemma 6.4 ).
Observing the convex losses. The MHR algorithm estimates its loss by repeatedly posting a single price pair until trade occurs. Here, the auxiliary loss depends on both marginal acceptance probabilities, while one-bit feedback reveals only their product. We overcome this by combining trade bits at a carefully chosen reference pair and its two crosses with the queried pair: for query ( p , q ) (p,q) ( p , q ) and reference ( u , v ) (u,v) ( u , v ) , the crossed prices are ( p , v ) (p,v) ( p , v ) and ( u , q ) (u,q) ( u , q ) . Independence supplies the required identities, and randomized sampling routines based on Bernoulli factories turn these bits into bounded, unbiased observations of the reciprocal auxiliary profit (Lemma 6.5). A key geometric property is that, when the reference and query have high auxiliary profit for the same parameter, their crossed pairs also have high actual profit (Lemma 6.3 ).
Refinement and elimination. We cannot, however, simply apply the stochastic convex bandits bound in Corollary 3.1 independently to every candidate. First, some candidates may have auxiliary optima too far below μ ⋆ \mu^\star μ ⋆ , so we must discard them before incurring too much regret. Second, producing loss observations also requires posting the reference pair and crossed prices. The reference must therefore improve as refinement proceeds, so that obtaining more accurate observations does not itself incur excessive regret. Third, the candidate parameter λ \lambda λ depends on unknown marginal probabilities: using a new reference price z z z requires estimating the multiplier ρ \rho ρ relating its balancing parameter λ z \lambda_z λ z to the candidate’s parameter, by λ = ρ λ z \lambda=\rho\lambda_z λ = ρ λ z . This calibration also costs trading rounds and cannot be repeated at every query. Hence, we run successive elimination over the candidate convex problems. In phase j j j , at accuracy ε j = δ 2 − j \varepsilon_j=\delta2^{-j} ε j = δ 2 − j , we use Corollary 3.1 to find a nearly optimal price pair within each surviving candidate’s region and estimate its best auxiliary profit. We eliminate candidates whose estimates are too small compared to the empirical maximum and update each survivor’s reference to the newly found price pair. At the start of each phase, calibration adapts the loss-observation procedure to each candidate’s current reference while keeping the auxiliary parameter λ \lambda λ close to its fixed intended value. The elimination tolerance is of order η + ε \eta+\varepsilon η + ε , ensuring that a nearly optimal candidate survives (Lemma 6.6). Calibration costs O ~ ( 1 / η ) \widetilde{O}(1/\eta) O ( 1/ η ) per candidate per phase.
Main Result 4 (Regular distributions). Consider the problem of learning the best prices in bilateral trade when the agents’ valuations are drawn from independent and regular distributions. There exists a learning algorithm which enjoys O ~ ( T 3 / 5 ) \widetilde{O}(T^{3/5}) O ( T 3/5 ) regret.
Proof Sketch. Initialization costs O ~ ( K ) \widetilde{O}(K) O ( K ) regret. Since only O ( log T ) O(\log T) O ( log T ) phases can start, calibration costs O ~ ( N / η ) \widetilde{O}(N/\eta) O ( N / η ) in total. Summing the optimizer bounds over the runs started before T T T gives O ~ ( N T ) \widetilde{O}(\sqrt{NT}) O ( N T ) . The additional O ( η ) O(\eta) O ( η ) per-round term during refinement is charged only over actual trading rounds, contributing O ( T η ) O(T\eta) O ( T η ) . Combining these bounds gives
R T ≲ K + N η + N T + T η ≲ η − 3 / 2 + T η − 1 / 4 + T η . R_T \lesssim K+\frac{N}{\eta}+\sqrt{NT}+T\eta\lesssim\eta^{-3/2}+\sqrt{T}\,\eta^{-1/4}+T\eta. R T ≲ K + η N + N T + T η ≲ η − 3/2 + T η − 1/4 + T η .
using K ≈ 1 / η K\approx1/\eta K ≈ 1/ η and N = O ~ ( δ / η ) N=\widetilde{O}(\sqrt{\delta/\eta}) N = O ( δ / η ) . Choosing η ≈ T − 2 / 5 \eta\approx T^{-2/5} η ≈ T − 2/5 yields the O ~ ( T 3 / 5 ) \widetilde{O}(T^{3/5}) O ( T 3/5 ) guarantee. In conclusion, we see that relative to MHR, the additional cost comes from identifying a suitable member of the family of convex objectives.
Separation between Regular and MHR Distributions
Our last result concerns a separation between MHR and regular distributions. In particular, we prove that no learning algorithm can hope to achieve a regret smaller than T 3 / 5 T^{3/5} T 3/5 , even when the valuations are drawn from independent and regular distributions. We provide here a sketch of the lower bound construction, while all the details can be found in § 7 ; in particular, the following result is proved as Theorem 7.1 .
Main Result 5 (Regular distributions). Consider the problem of learning the best prices in bilateral trade when the agents’ valuations are drawn from independent and regular distributions. For any learning algorithm A A A , there exists an independent and regular distribution D D D such that R T ( A , D ) ∈ Ω ( T 3 / 5 ) R_T(A,D) \in\Omega(T^{3/5}) R T ( A , D ) ∈ Ω ( T 3/5 ) .
To construct a stochastic lower bound, it is enough to construct a randomized instance (i.e., a mixture of independent and regular distributions) which forces any deterministic algorithm to suffer the desired regret, in expectation with respect to the randomization of the instance. We prove the lower bound even when the learner observes the seller’s and buyer’s acceptance decisions separately.
For an integer parameter K K K , we construct a base distribution D 0 D_0 D 0 and K K K auxiliary distributions D 1 , … , D K D_1,\ldots,D_K D 1 , … , D K over [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 . Crucially, these distributions are all independent and regular, and for each buyer price q q q the optimal seller price is always q / 2 q/2 q /2 . Moreover, under all these distributions the seller valuation is drawn independently from some fixed and known distribution, so that there is no point in deviating from posting q / 2 q/2 q /2 to the seller to “gain information”. This property simplifies the argument, so that we can focus our attention — without loss of generality — on algorithms that will always post prices of that form. We introduce the functions h j ( q ) h^j(q) h j ( q ) , which specify the expected profit of posting prices ( q / 2 , q ) (q/2,q) ( q /2 , q ) when the valuations ( S , B ) (S,B) ( S , B ) are drawn according to D j D_j D j :
h j ( q ) = E j [ Profit ( q / 2 , q ) ] j ∈ { 0 , 1 , … , K } . h^j(q) = \mathbb{E}^j[\operatorname{Profit}(q/2,q)] \qquad j \in\{0,1,\ldots,K\}. h j ( q ) = E j [ Profit ( q /2 , q )] j ∈ { 0 , 1 , … , K } .
The “baseline” profit is maximized on a plateau for q ∈ [ 1 / 4 , 1 / 2 ] q \in[1/4,1/2] q ∈ [ 1/4 , 1/2 ] . This interval is partitioned uniformly into non-overlapping intervals I j I_j I j such that each h j h^j h j exhibits a little “bump” on the corresponding I j I_j I j . Each one of these bumps has magnitude Δ ≈ 1 / K 2 \Delta\approx1/K^2 Δ ≈ 1/ K 2 . These properties are formalized in the following Claim (whose proof is deferred to § 7 ), and are visualized in Figure 3 .
Figure 3. Expected profit h j ( q ) h^j(q) h j ( q ) at prices ( q / 2 , q ) (q/2,q) ( q /2 , q ) for the lower-bound construction. The baseline h 0 h^0 h 0 has a plateau of maximizers. Two alternatives, h 1 h^1 h 1 and h 3 h^3 h 3 (here K = 4 K=4 K = 4 ), coincide with h 0 h^0 h 0 outside I 1 I_1 I 1 and I 3 I_3 I 3 , respectively.
Claim 3.4. For any j = 1 , … , K j=1,\ldots,K j = 1 , … , K , it holds that:
(i) The function h 0 h^0 h 0 has a plateau of maximizers in [ 1 / 4 , 1 / 2 ] [1/4,1/2] [ 1/4 , 1/2 ] , with maximum value 1 / 192 1/192 1/192 .
(ii) Letting q j ∗ q_j^* q j ∗ be the midpoint of I j I_j I j , we have h j ( q j ∗ ) − max q ∉ I j h j ( q ) ≥ Δ h^j(q_j^*)-\max_{q\notin I_j}h^j(q)\geq\Delta h j ( q j ∗ ) − max q ∈ / I j h j ( q ) ≥ Δ .
Remark 3.5. The crux of the argument resides in finding independent and regular distributions which exhibit a structure like the one described in the above claim. In particular, regularity impacts on the magnitude of the “bumps” that we can force on the expected profit function.
For any deterministic algorithm A \mathcal{A} A , we prove that it fails against the uniform mixture of { D j } j = 1 K \{\mathcal{D}_j\}_{j=1}^{K} { D j } j = 1 K . To this end, denote by N j N_j N j the random variable counting the number of times that A \mathcal{A} A posts a buyer price in I j I_j I j . The crucial ingredient to analyze the regret of A \mathcal{A} A under the mixture resides in arguing that the algorithm manages to differentiate between the generic D j \mathcal{D}_j D j and the baseline D 0 \mathcal{D}_0 D 0 only by posting buyer prices in the “right” interval I j I_j I j . In particular, to discern whether there exists a bump in a given I j I_j I j (i.e., if the underlying distribution is actually D j \mathcal{D}_j D j or not, see Claim 3.4), any learner has to post at least 1 / Δ 2 ≈ K 4 1/\Delta^2 \approx K^4 1/ Δ 2 ≈ K 4 prices in I j I_j I j . This intuition is formalized in § 7 , where we prove the following claim.
Claim 3.6. We have the following inequality:
E j [ N j ] − E 0 [ N j ] ≤ T 22 K 2 E 0 [ N j ] . \mathbb{E}^{j}[N_j]-\mathbb{E}^{0}[N_j]\leq\frac{T}{22K^2}\sqrt{\mathbb{E}^{0}[N_j]}. E j [ N j ] − E 0 [ N j ] ≤ 22 K 2 T E 0 [ N j ] .
By Claim 3.4, the regret of the deterministic algorithm A \mathcal{A} A against the generic distribution D j \mathcal{D}_j D j satisfies
R T ( A , D j ) ≥ ( T − E j [ N j ] ) Δ . R_T(\mathcal{A},\mathcal{D}_j)\geq(T-\mathbb{E}^{j}[N_j])\Delta. R T ( A , D j ) ≥ ( T − E j [ N j ]) Δ.
so that if we take the expectation with respect to the uniform choice of j ∈ [ K ] j\in[K] j ∈ [ K ] , we get:
1 K ∑ j = 1 K R T ( A , D j ) ≥ 1 K ∑ j = 1 K ( T − E j [ N j ] ) Δ ≥ 1 K ∑ j = 1 K ( T − E 0 [ N j ] − T 22 K 2 E 0 [ N j ] ) Δ (By Claim 3.6) ≥ K − 1 K T Δ − 1 K ∑ j = 1 K T Δ 22 K 2 E 0 [ N j ] (As ∑ j E 0 [ N j ] ≤ T ) ≥ K − 1 K T Δ − T Δ 22 K 2 1 K ∑ j = 1 K E 0 [ N j ] (By Jensen’s Inequality) ≥ K − 1 K T Δ − T Δ 22 K 2 T K (As ∑ j E 0 [ N j ] ≤ T ) = T Δ ( 1 − 1 K − 1 22 T K 5 ) ≥ T Δ 2 , (By choosing K = ⌈ T 1 / 5 ⌉ ) \begin{aligned}
\frac{1}{K}\sum_{j=1}^{K}R_T(\mathcal{A},\mathcal{D}_j)
&\geq\frac{1}{K}\sum_{j=1}^{K}(T-\mathbb{E}^{j}[N_j])\Delta\\
&\geq\frac{1}{K}\sum_{j=1}^{K}\left(T-\mathbb{E}^{0}[N_j]-\frac{T}{22K^2}\sqrt{\mathbb{E}^{0}[N_j]}\right)\Delta&&\text{(By Claim 3.6)}\\
&\geq\frac{K-1}{K}T\Delta-\frac{1}{K}\sum_{j=1}^{K}\frac{T\Delta}{22K^2}\sqrt{\mathbb{E}^{0}[N_j]} &&\text{(As }\sum_j\mathbb{E}^{0}[N_j]\leq T\text{)}\\
&\geq\frac{K-1}{K}T\Delta-\frac{T\Delta}{22K^2}\sqrt{\frac{1}{K}\sum_{j=1}^{K}\mathbb{E}^{0}[N_j]} &&\text{(By Jensen’s Inequality)}\\
&\geq\frac{K-1}{K}T\Delta-\frac{T\Delta}{22K^2}\sqrt{\frac{T}{K}} &&\text{(As }\sum_j\mathbb{E}^{0}[N_j]\leq T\text{)}\\
&=T\Delta\left(1-\frac{1}{K}-\frac{1}{22}\sqrt{\frac{T}{K^5}}\right)\geq\frac{T\Delta}{2}, &&\text{(By choosing }K=\lceil T^{1/5}\rceil\text{)}
\end{aligned} K 1 j = 1 ∑ K R T ( A , D j ) ≥ K 1 j = 1 ∑ K ( T − E j [ N j ]) Δ ≥ K 1 j = 1 ∑ K ( T − E 0 [ N j ] − 22 K 2 T E 0 [ N j ] ) Δ ≥ K K − 1 T Δ − K 1 j = 1 ∑ K 22 K 2 T Δ E 0 [ N j ] ≥ K K − 1 T Δ − 22 K 2 T Δ K 1 j = 1 ∑ K E 0 [ N j ] ≥ K K − 1 T Δ − 22 K 2 T Δ K T = T Δ ( 1 − K 1 − 22 1 K 5 T ) ≥ 2 T Δ , (By Claim 3.6) (As j ∑ E 0 [ N j ] ≤ T ) (By Jensen’s Inequality) (As j ∑ E 0 [ N j ] ≤ T ) (By choosing K = ⌈ T 1/5 ⌉ )
where the last inequality follows for all T T T such that ⌈ T 1 / 5 ⌉ ≥ 3 \lceil T^{1/5}\rceil\geq3 ⌈ T 1/5 ⌉ ≥ 3 . To get the desired Ω ( T 3 / 5 ) \Omega(T^{3/5}) Ω ( T 3/5 ) regret lower bound, we just need to tune the parameter K K K , setting K ≈ T 1 / 5 K\approx T^{1/5} K ≈ T 1/5 (recall that Δ ≈ 1 K 2 \Delta\approx\frac{1}{K^2} Δ ≈ K 2 1 , so that it is proportional to T − 2 / 5 T^{-2/5} T − 2/5 ).
O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) Regret for the Adversarial Setting
In this section, we provide the complete proof of the regret bound in the adversarial setting. Recall that here we consider an oblivious adversary which fixes a sequence of valuations v 1 , … , v T ∈ [ 0 , 1 ] 2 v_1,\ldots,v_T\in[0,1]^2 v 1 , … , v T ∈ [ 0 , 1 ] 2 before play. We want to prove the following theorem.
Theorem 4.1 (Regret bound). There exists a polynomial time algorithm A \mathcal{A} A such that, for every valuation sequence S S S fixed in advance, the sequence of prices posted by the algorithm satisfies the following regret bound:
R T ( A , S ) ≤ O ( T 2 / 3 log 7 / 3 ( T ) ) . R_T(\mathcal{A},S)\leq O(T^{2/3}\log^{7/3}(T)). R T ( A , S ) ≤ O ( T 2/3 log 7/3 ( T )) .
We show the above by first defining and analyzing the algorithm in § 4.1 and § 4.2 , proving the regret bound conditional on an estimation lemma which relates to the general i.i.d. setting, see Lemma 4.5. We then dedicate § 4.3 to prove the lemma via a delicate estimation procedure, which directly implies a O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret rate in the stochastic i.i.d. via a successive elimination routine, as explained in § 3.1 .
The adversarial algorithm
The learner maintains positive probability weights w t w_t w t on some prices we compare profit against, initially uniform over the action set A \mathcal{A} A . In each round, the update of the weights as well as the price to play by the algorithm are given by subroutine OneRound, which returns a price a t a_t a t and two probability vectors Q 0 , Q 1 Q_0,Q_1 Q 0 , Q 1 on A \mathcal{A} A . Q z ( ⋅ ) Q_z(\cdot) Q z ( ⋅ ) is the updated comparison weight vector that the learner will assign upon trade outcome z = z a t ( v t ) = 1 { s t ≤ p t } ⋅ 1 { b t ≥ q t } z=z_{a_t}(v_t)=\mathbf{1}\{s_t\leq p_t\}\cdot\mathbf{1}\{b_t\geq q_t\} z = z a t ( v t ) = 1 { s t ≤ p t } ⋅ 1 { b t ≥ q t } , recalling that v t = ( s t , b t ) v_t=(s_t,b_t) v t = ( s t , b t ) and a t = ( p t , q t ) a_t=(p_t,q_t) a t = ( p t , q t ) . We call the price and its two potential weight updates a proposal .
Table 1 (PDF p. 19).
Constructing candidate posted prices and updates. Each call to OneRound holds w t w_t w t fixed and iterates over a generic probability distribution ρ \rho ρ on G K × A G_K\times\mathcal{A} G K × A , a local variable to this subroutine that is updated at every iteration. Let ( V , A ) ∼ ρ (V,A)\sim\rho ( V , A ) ∼ ρ be a valuation/action random vector independently sampled according to this table. At an iteration inside OneRound, for each candidate price a a a and trade/no trade bit z z z , let β = K 2 log 6 K \beta=K^2\log^6 K β = K 2 log 6 K and define
Q z ρ ( a ′ ) = β K P ρ ( A = a ′ ∣ z a ( V ) = z ) + 1 / ∣ A ∣ 1 + β K (5) Q_z^\rho(a')=\frac{\beta K\mathbb{P}_\rho(A=a'\mid z_a(V)=z)+1/|\mathcal{A}|}{1+\beta K} \tag*{(5)} Q z ρ ( a ′ ) = 1 + β K β K P ρ ( A = a ′ ∣ z a ( V ) = z ) + 1/∣ A ∣ (5)
where we leave dependence on a a a implicit. Moreover, for a proposal ( a , Q 0 , Q 1 ) (a,Q_0,Q_1) ( a , Q 0 , Q 1 ) and weights w w w , define the score of the proposal against comparison price a ′ a' a ′ and valuation v v v by
h a ( v , a ′ ) = Profit ( v , a ′ ) − Profit ( v , a ) − β log Q z a ( v ) ( a ′ ) w ( a ′ ) . (6) h_a(v,a')=\operatorname{Profit}(v,a')-\operatorname{Profit}(v,a)-\beta\log\frac{Q_{z_a(v)}(a')}{w(a')}. \tag*{(6)} h a ( v , a ′ ) = Profit ( v , a ′ ) − Profit ( v , a ) − β log w ( a ′ ) Q z a ( v ) ( a ′ ) . (6)
In words, the score balances exploration with exploitation, with the first term bounding regret with respect to a ′ a' a ′ and thus representing exploitation, and the second representing posterior information gain and thus exploration, if Q z Q_z Q z is taken to be Q z ρ Q_z^\rho Q z ρ .
In the above, β \beta β is a fixed parameter that controls how much an increase in a comparison price’s weight can compensate for the profit gap of the posted price to the comparison price. In particular, the first two terms of h a ( v , a ′ ) h_a(v,a') h a ( v , a ′ ) are the profit gap of candidate price a a a against a ′ a' a ′ , and if the update Q z a ( v ) ( a ′ ) Q_{z_a(v)}(a') Q z a ( v ) ( a ′ ) increases the weight of a ′ a' a ′ , the logarithm is positive and offsets this loss. This score and idea is exactly the concept of Algorithmic Information Ratio from Xu and Zeevi [44 ] , when taking expectations over ρ \rho ρ with Q z = Q z ρ Q_z=Q_z^\rho Q z = Q z ρ , and after appropriately translating notation.
Inside one step j j j of OneRound, we then choose a price arg min a ∈ A E ρ [ h a ( V , A ) ] \arg\min_{a\in\mathcal{A}}\mathbb{E}_\rho[h_a(V,A)] arg min a ∈ A E ρ [ h a ( V , A )] minimizing the expected score with respect to ρ \rho ρ , and store its associated updates Q 0 ρ , Q 1 ρ Q_0^\rho,Q_1^\rho Q 0 ρ , Q 1 ρ . OneRound only uses this data to update the table ρ \rho ρ . Only after the final randomly chosen step j ∗ j^* j ∗ , is the current proposal, i.e. ( arg min a ∈ A E ρ [ h a ( V , A ) ] , Q 0 ρ , Q 1 ρ ) (\arg\min_{a\in\mathcal{A}}\mathbb{E}_\rho[h_a(V,A)],Q_0^\rho,Q_1^\rho) ( arg min a ∈ A E ρ [ h a ( V , A )] , Q 0 ρ , Q 1 ρ ) returned to Algorithm 1.
Algorithm 2 OneRound(w w w )
Require: Probability vector w w w with w ( a ′ ) ≥ 1 / ( ∣ A ∣ ( 1 + β K ) ) w(a') \ge1/(|\mathcal{A}|(1+\beta K)) w ( a ′ ) ≥ 1/ ( ∣ A ∣ ( 1 + β K )) ; parameters from Equation (8)
Draw j ⋆ j^\star j ⋆ uniformly from { 1 , … , J } \{1,\ldots,J\} { 1 , … , J }
ρ ( v , a ′ ) ← 1 / ( ∣ A ∣ ∣ G K ∣ ) \rho(v,a') \leftarrow1/(|\mathcal{A}||G_K|) ρ ( v , a ′ ) ← 1/ ( ∣ A ∣∣ G K ∣ ) for every ( v , a ′ ) ∈ G K × A (v,a') \in G_K \times\mathcal{A} ( v , a ′ ) ∈ G K × A
for j = 1 , … , j ⋆ j=1,\ldots,j^\star j = 1 , … , j ⋆ do
For each a ∈ A a \in\mathcal{A} a ∈ A , form Q 0 ρ , Q 1 ρ Q_0^\rho,Q_1^\rho Q 0 ρ , Q 1 ρ by Equation (5) and compute E ρ [ h a ( V , A ) ] \mathbb{E}_{\rho}[h_a(V,A)] E ρ [ h a ( V , A )] using Equation (6)
Choose a price a a a minimizing E ρ [ h a ( V , A ) ] \mathbb{E}_{\rho}[h_a(V,A)] E ρ [ h a ( V , A )] , breaking ties by a fixed order, and keep its associated updates Q 0 , Q 1 Q_0,Q_1 Q 0 , Q 1
Simultaneously update all entries by
ρ ( v , a ′ ) ← ρ ( v , a ′ ) ( 1 + η h a ( v , a ′ ) ) 1 + η E ρ [ h a ( V , A ) ] \rho(v,a') \leftarrow\frac{\rho(v,a')(1+\eta h_a(v,a'))}{1+\eta\mathbb{E}_{\rho}[h_a(V,A)]} ρ ( v , a ′ ) ← 1 + η E ρ [ h a ( V , A )] ρ ( v , a ′ ) ( 1 + η h a ( v , a ′ ))
return The last selected price a a a and its associated updates Q 0 , Q 1 Q_0,Q_1 Q 0 , Q 1
Interpretation as MWU. Algorithm 2 is a standard application — see e.g. Theorem 3.1 of Arora et al. [2 ] — of the Multiplicative Weight Update algorithm for finding an upper bound of a minimax problem, namely
max v ∈ G K E [ h a ( v , a ′ ) ] . (7) \underset{v \in G_K}{\max}\ \mathbb{E}\left[h_a(v,a')\right]. \tag*{(7)} v ∈ G K max E [ h a ( v , a ′ ) ] . (7)
The above minimax game can be shown to be equivalent — in terms of value — to the inner game from Algorithm 3 in Xu and Zeevi [44 ] , so that the upper bound we give on Equation (7) immediately implies an analogous adversarial regret bound via their Theorem 3.1 or Theorem 3.3, assuming the possibility of exactly solving their game. The choice of a slightly different game together with a MWU inner routine is to avoid issues when specifying a constructive algorithm: numerically solving their inner game up to some precision does not yield the same precision in regret terms, due to the gradient term appearing in their Theorem 3.4.
To see how MWU provides this direct control, we describe its application to the game from Equation (7) , proving guarantees formally in Lemma 4.6 . The MWU decisions are the pairs ( v , a ′ ) ∈ G K × A (v,a') \in G_K \times\mathcal{A} ( v , a ′ ) ∈ G K × A , and ρ \rho ρ is a distribution over these decisions. At each iteration, the routine constructs a proposal for every candidate price and selects one minimizing E ρ [ h a ( V , A ) ] \mathbb{E}_{\rho}[h_a(V,A)] E ρ [ h a ( V , A )] . The selected proposal then assigns a score to every decision, via Equation (6) . Viewing the negative scores as costs, MWU, which plays the role of the adversary in the minimax game, updates
ρ ( v , a ′ ) ∝ ρ ( v , a ′ ) ( 1 + η h a ( v , a ′ ) ) , \rho(v,a') \propto\rho(v,a')(1+\eta h_a(v,a')), ρ ( v , a ′ ) ∝ ρ ( v , a ′ ) ( 1 + η h a ( v , a ′ )) ,
followed by normalization. Intuitively, this interaction mimics a maximin game over successive inner iterations: the player responds to the current distribution ρ \rho ρ by selecting a proposal, while MWU adjusts ρ \rho ρ to place more weight on pairs ( v , a ′ ) (v,a') ( v , a ′ ) that have received large scores under past proposals. Outputting a proposal by uniformly mixing over the rounds in { 1 , … , J } \{1,\ldots,J\} { 1 , … , J } then provides a bound on Equation (7) .
In Algorithm 2 and for the following proofs, we set the following hyperparameters:
H = 1 + β log ( ∣ A ∣ ( 1 + β K ) ) , η = 1 2 K H 2 , J = ⌈ 4 K 2 H 2 log ( ∣ A ∣ ∣ G K ∣ ) ⌉ . (8) \begin{aligned}
H &= 1+\beta\log(|\mathcal{A}|(1+\beta K)), & \eta&= \frac{1}{2KH^2},\\
J &= \left\lceil4K^2H^2\log(|\mathcal{A}||G_K|)\right\rceil. \tag*{(8)}
\end{aligned} H J = 1 + β log ( ∣ A ∣ ( 1 + β K )) , = ⌈ 4 K 2 H 2 log ( ∣ A ∣∣ G K ∣ ) ⌉ . η = 2 K H 2 1 , (8)
Here H H H bounds the absolute value of the score, η \eta η is the MWU learning rate, and J J J is a sufficient number of iterations to have a bound of at most O ( 1 / K ) O(1/K) O ( 1/ K ) on Equation (7) .
Analysis of OneRound
A tradeoff between profit and information
To analyze OneRound, we first consider the dual formulation of the learning game Equation (7) : a deterministic strategy against a fixed randomized distribution ρ \rho ρ over G K × A G_K \times\mathcal{A} G K × A . The analysis of OneRound then allows relating the maximin game to the minimax one, yielding a bound on Equation (7) , for any considered probability vector w w w , and thus score h h h definition from Equation (6) .
Before proceeding, we introduce a few relevant concepts. First, given a reference action a ∈ A a \in\mathcal{A} a ∈ A , we can define the expected gap Δ ρ ( a ) \Delta_{\rho}(a) Δ ρ ( a ) when the valuation V V V and the action A A A are drawn jointly from ρ \rho ρ :
Δ ρ ( a ) = E ρ [ Profit ( V , A ) − Profit ( V , a ) ] . (9) \Delta_{\rho}(a) = \mathbb{E}_{\rho}\left[\operatorname{Profit}(V,A)-\operatorname{Profit}(V,a)\right]. \tag*{(9)} Δ ρ ( a ) = E ρ [ Profit ( V , A ) − Profit ( V , a ) ] . (9)
Similarly, let Δ ρ ( a ′ , a ) = E ρ [ Profit ( V , a ′ ) − Profit ( V , a ) ] \Delta_{\rho}(a',a)=\mathbb{E}_{\rho}\left[\operatorname{Profit}(V,a')-\operatorname{Profit}(V,a)\right] Δ ρ ( a ′ , a ) = E ρ [ Profit ( V , a ′ ) − Profit ( V , a ) ] , the expected difference between fixed action a ′ , a ∈ A a',a \in\mathcal{A} a ′ , a ∈ A in terms of profit when the valuations are drawn according to the marginal probability of ρ \rho ρ on the valuation space.
The quantity in Equation (9) may well be positive if the dependence between valuation V V V and action A A A under ρ \rho ρ can be exploited to gain more profit. In the following, we will argue that there must exist an action a a a which has either small gap Δ ρ ( a ) \Delta_{\rho}(a) Δ ρ ( a ) or which reveals a lot of dependency between z a ( V ) z_a(V) z a ( V ) and A A A , or both. In other words, not all actions in A \mathcal{A} A can show substantial sub-optimality with respect to ρ \rho ρ while not revealing much about the “strength” of the coupling between V V V and A A A .
To quantify this dependency, we consider the notion of mutual information, i.e. the KL divergence between the joint distribution of { z a ( V ) , A } \{z_a(V),A\} { z a ( V ) , A } under ρ \rho ρ and their joint distribution if we assumed it was the product of the corresponding marginals. More formally:
I ρ ( a ) = ∑ a ′ ∈ A ∑ z ∈ { 0 , 1 } P ρ ( A = a ′ , z a ( V ) = z ) log P ρ ( A = a ′ , z a ( V ) = z ) P ρ ( A = a ′ ) ⋅ P ρ ( z a ( V ) = z ) \mathcal{I}_{\rho}(a)=\sum_{a'\in\mathcal{A}}\sum_{z\in\{0,1\}}\mathbb{P}_{\rho}(A=a',z_a(V)=z)\log\frac{\mathbb{P}_{\rho}(A=a',z_a(V)=z)}{\mathbb{P}_{\rho}(A=a')\cdot\mathbb{P}_{\rho}(z_a(V)=z)} I ρ ( a ) = a ′ ∈ A ∑ z ∈ { 0 , 1 } ∑ P ρ ( A = a ′ , z a ( V ) = z ) log P ρ ( A = a ′ ) ⋅ P ρ ( z a ( V ) = z ) P ρ ( A = a ′ , z a ( V ) = z )
In the rest of this subsection, we prove the following statement, which bounds the parameterized information ratio [23 ] for our problem.
Theorem 4.2 (Information Inequality). There exists a universal constant C 0 C_0 C 0 such that, for any distribution ρ \rho ρ over G K × A G_K \times\mathcal{A} G K × A , the following inequality holds for β = K 2 log 6 K \beta=K^2\log^6 K β = K 2 log 6 K :
min a ∈ A { Δ ρ ( a ) − β ⋅ I ρ ( a ) } ≤ C 0 K . \min_{a\in\mathcal{A}}\left\{\Delta_{\rho}(a)-\beta\cdot\mathcal{I}_{\rho}(a)\right\}\leq\frac{C_0}{K}. a ∈ A min { Δ ρ ( a ) − β ⋅ I ρ ( a ) } ≤ K C 0 .
Proof. Assume towards contradiction that there exists a distribution ρ \rho ρ which violates the statement, i.e., such that Δ ρ ( a ) − β ⋅ I ρ ( a ) > C 0 / K \Delta_{\rho}(a)-\beta\cdot\mathcal{I}_{\rho}(a)>C_0/K Δ ρ ( a ) − β ⋅ I ρ ( a ) > C 0 / K for all actions a ∈ A a\in\mathcal{A} a ∈ A , and for some constant C 0 ∈ R + C_0\in\mathbb{R}^{+} C 0 ∈ R + . We focus on such ρ \rho ρ . We denote with a ⋆ a^{\star} a ⋆ the optimal fixed action under the marginal valuation distribution induced by ρ \rho ρ .
We are under the assumption that Δ ρ ( a ) − β ⋅ I ρ ( a ) > C 0 / K \Delta_{\rho}(a)-\beta\cdot\mathcal{I}_{\rho}(a)>C_0/K Δ ρ ( a ) − β ⋅ I ρ ( a ) > C 0 / K for all actions a ∈ A a\in\mathcal{A} a ∈ A , in particular for a = a ⋆ a=a^{\star} a = a ⋆ , this implies that Δ ρ ( a ⋆ ) \Delta_{\rho}(a^{\star}) Δ ρ ( a ⋆ ) is at least C 0 / K C_0/K C 0 / K (as the information term is non-negative). Note, this does not violate the optimality of a ⋆ a^{\star} a ⋆ under ρ \rho ρ , as the random A A A may adapt to the conditional behavior of V V V , as explained at the beginning of the subsection. We have then proved the following claim.
Claim 4.3. Under the assumption towards contradiction, the following inequality holds: Δ ρ ( a ⋆ ) ∈ ( C 0 / K , 1 ] \Delta_{\rho}(a^{\star})\in(C_0/K,1] Δ ρ ( a ⋆ ) ∈ ( C 0 / K , 1 ] .
For each action a a a , we can introduce an auxiliary distribution over G K G_K G K with the following associated probability trade function, recalling that v = ( s , b ) v=(s,b) v = ( s , b ) :
P a ( v ) = P ρ ( V ∈ [ 0 , s ] × [ b , 1 ] ∣ A = a ) . (10) P_a(v)=\mathbb{P}_{\rho}\left(V\in[0,s]\times[b,1]\mid A=a\right). \tag*{(10)} P a ( v ) = P ρ ( V ∈ [ 0 , s ] × [ b , 1 ] ∣ A = a ) . (10)
The marginal distribution is then the corresponding mixture of all these auxiliary distributions:
P 0 ( v ) = ∑ a ∈ A P ρ ( A = a ) P a ( v ) = P ρ ( V ∈ [ 0 , s ] × [ b , 1 ] ) . (11) P_0(v)=\sum_{a\in\mathcal{A}}\mathbb{P}_{\rho}(A=a)P_a(v)=\mathbb{P}_{\rho}\left(V\in[0,s]\times[b,1]\right). \tag*{(11)} P 0 ( v ) = a ∈ A ∑ P ρ ( A = a ) P a ( v ) = P ρ ( V ∈ [ 0 , s ] × [ b , 1 ] ) . (11)
For any action a a a , we can conveniently decompose the information I ρ ( a ) \mathcal{I}_{\rho}(a) I ρ ( a ) accordingly. In particular, the random feedback z a ( V ) z_a(V) z a ( V ) under ρ \rho ρ is a Bernoulli random variable whose probability of success is P 0 ( a ) P_0(a) P 0 ( a ) under ρ \rho ρ , or P a ′ ( a ) P_{a'}(a) P a ′ ( a ) conditioning on A = a ′ A=a' A = a ′ .
I ρ ( a ) = ∑ a ′ ∈ A ∑ z ∈ { 0 , 1 } P ρ ( A = a ′ , z a ( V ) = z ) log P ρ ( A = a ′ , z a ( V ) = z ) P ρ ( A = a ′ ) ⋅ P ρ ( z a ( V ) = z ) = ∑ a ′ ∈ A P ρ ( A = a ′ ) [ ∑ z ∈ { 0 , 1 } P ρ ( z a ( V ) = z ∣ A = a ′ ) log P ρ ( z a ( V ) = z ∣ A = a ′ ) P ρ ( z a ( V ) = z ) ] . = ∑ a ′ ∈ A P ρ ( A = a ′ ) D K L ( Bern ( P a ′ ( a ) ) ∥ Bern ( P 0 ( a ) ) ) (12) \begin{aligned}
\mathcal{I}_{\rho}(a)
&=\sum_{a'\in\mathcal{A}}\sum_{z\in\{0,1\}}\mathbb{P}_{\rho}(A=a',z_a(V)=z)\log\frac{\mathbb{P}_{\rho}(A=a',z_a(V)=z)}{\mathbb{P}_{\rho}(A=a')\cdot\mathbb{P}_{\rho}(z_a(V)=z)}\\
&=\sum_{a'\in\mathcal{A}}\mathbb{P}_{\rho}(A=a')\left[\sum_{z\in\{0,1\}}\mathbb{P}_{\rho}(z_a(V)=z\mid A=a')\log\frac{\mathbb{P}_{\rho}(z_a(V)=z\mid A=a')}{\mathbb{P}_{\rho}(z_a(V)=z)}\right]. \\
&= \sum_{a'\in\mathcal{A}} \mathbb{P}_{\rho}(A=a')D_{\mathrm{KL}}\left(\operatorname{Bern}(P_{a'}(a)) \mathbin\Vert\operatorname{Bern}(P_0(a))\right) \tag*{(12)}
\end{aligned} I ρ ( a ) = a ′ ∈ A ∑ z ∈ { 0 , 1 } ∑ P ρ ( A = a ′ , z a ( V ) = z ) log P ρ ( A = a ′ ) ⋅ P ρ ( z a ( V ) = z ) P ρ ( A = a ′ , z a ( V ) = z ) = a ′ ∈ A ∑ P ρ ( A = a ′ ) z ∈ { 0 , 1 } ∑ P ρ ( z a ( V ) = z ∣ A = a ′ ) log P ρ ( z a ( V ) = z ) P ρ ( z a ( V ) = z ∣ A = a ′ ) . = a ′ ∈ A ∑ P ρ ( A = a ′ ) D KL ( Bern ( P a ′ ( a )) ∥ Bern ( P 0 ( a )) ) (12)
≥ 2 ∑ a ′ ∈ A P ρ ( A = a ′ ) ( P a ′ ( a ) − P 0 ( a ) ) 2 \ge2\sum_{a'\in\mathcal{A}}\mathbb{P}_{\rho}(A=a')\left(P_{a'}(a)-P_0(a)\right)^2 ≥ 2 a ′ ∈ A ∑ P ρ ( A = a ′ ) ( P a ′ ( a ) − P 0 ( a ) ) 2
We can then combine the above inequality with the fact that we are under the assumption that Δ ρ ( a ) − β I ρ ( a ) > 0 \Delta_{\rho}(a)-\beta\mathcal{I}_{\rho}(a)>0 Δ ρ ( a ) − β I ρ ( a ) > 0 to derive the following claim.
Claim 4.4. Under the assumption towards contradiction, the following inequalities hold:
Δ ρ ( a ) ≥ 2 β ∑ a ′ ∈ A P ρ ( A = a ′ ) ( P a ′ ( a ) − P 0 ( a ) ) 2 , ∀ a ∈ A . \Delta_{\rho}(a)\ge2\beta\sum_{a'\in\mathcal{A}}\mathbb{P}_{\rho}(A=a')\left(P_{a'}(a)-P_0(a)\right)^2,\qquad\forall a\in\mathcal{A}. Δ ρ ( a ) ≥ 2 β a ′ ∈ A ∑ P ρ ( A = a ′ ) ( P a ′ ( a ) − P 0 ( a ) ) 2 , ∀ a ∈ A .
To conclude the argument, we need the following delicate lemma, which is the actual enabler of our claimed regret bound. We defer its proof to a dedicated part of the paper, § 4.3 , where we explain the estimation routine behind the statement.
Lemma 4.5 (Inexpensive Detection). For every distribution ρ \rho ρ on G K × A G_K\times\mathcal{A} G K × A , and x ∈ ( 0 , 1 ] x\in(0,1] x ∈ ( 0 , 1 ] , there are non-negative weights n x ( a ) n_x(a) n x ( a ) (which depend only on ρ \rho ρ and x x x ) such that the following inequality is verified, for some constant C C C :
∑ a ∈ A n x ( a ) ( Δ ρ ( a ∗ , a ) + x ) ≤ C log 3 K ( K x + log 3 K x 2 ) . \sum_{a\in\mathcal{A}}n_x(a)\left(\Delta_{\rho}(a^*,a)+x\right)\le C\log^3 K\left(\frac{K}{x}+\frac{\log^3 K}{x^2}\right). a ∈ A ∑ n x ( a ) ( Δ ρ ( a ∗ , a ) + x ) ≤ C log 3 K ( x K + x 2 log 3 K ) . where a ∗ a^* a ∗ is optimal according to ρ \rho ρ . Moreover, for any random valuation V ~ \widetilde{V} V with probability function of trade P P P such that max a ∈ A E [ Profit ( V ~ , a ) ] ≥ E ρ [ Profit ( V , a ∗ ) ] + x \max_{a\in\mathcal{A}}\mathbb{E}\left[\operatorname{Profit}(\widetilde{V},a)\right]\ge\mathbb{E}_{\rho}\left[\operatorname{Profit}(V,a^*)\right]+x max a ∈ A E [ Profit ( V , a ) ] ≥ E ρ [ Profit ( V , a ∗ ) ] + x , then the same weights respect, with P 0 P_0 P 0 induced by ρ \rho ρ :
∑ a ∈ A n x ( a ) ( P ( a ) − P 0 ( a ) ) 2 ≥ 1. \sum_{a\in\mathcal{A}}n_x(a)\left(P(a)-P_0(a)\right)^2\ge1. a ∈ A ∑ n x ( a ) ( P ( a ) − P 0 ( a ) ) 2 ≥ 1.
Crucially, the choice of weights { n x ( a ) } a ∈ A \{n_x(a)\}_{a\in\mathcal{A}} { n x ( a ) } a ∈ A works for any other considered probability function of trade P P P , once ρ \rho ρ and x x x are fixed.
Now fix any value x ∈ ( Δ ρ ( a ∗ ) / 2 , 1 ] x\in(\Delta_{\rho}(a^*)/2,1] x ∈ ( Δ ρ ( a ∗ ) /2 , 1 ] , and let n x ( a ) n_x(a) n x ( a ) be the weights as in the lemma above. For each action a a a , denote with X a X_a X a the indicator variable on A \mathcal{A} A of whether it holds that max a ∈ A E [ Profit ( V ~ , a ) ] ≥ E ρ [ Profit ( V , a ∗ ) ] + x \max_{a\in\mathcal{A}}\mathbb{E}[\operatorname{Profit}(\widetilde{V},a)]\ge\mathbb{E}_{\rho}[\operatorname{Profit}(V,a^*)]+x max a ∈ A E [ Profit ( V , a )] ≥ E ρ [ Profit ( V , a ∗ )] + x , under V ~ \widetilde{V} V following the (conditional) probability trade function P a P_a P a . Note, we can apply the second part of the lemma if and only if X a X_a X a is equal to 1. We then have:
P A ( max a ∈ A E P A [ Profit ( V ~ , a ) ] ≥ E ρ [ Profit ( V , a ∗ ) ] + x ) = ∑ a ∈ A P ρ ( A = a ) X a ≤ ∑ a ∈ A P ρ ( A = a ) X a ∑ a ′ ∈ A n x ( a ′ ) ( P a ( a ′ ) − P 0 ( a ′ ) ) 2 (By xref‘Lemma 4.5‘) ≤ 1 2 β ∑ a ′ ∈ A n x ( a ′ ) Δ ρ ( a ′ ) (By xref‘Claim 4.4‘) ≤ 1 2 β ∑ a ′ ∈ A n x ( a ′ ) ( Δ ρ ( a ∗ , a ′ ) + 2 x ) (As Δ ρ ( a ) = Δ ρ ( a ∗ ) + Δ ρ ( a ∗ , a ) and Δ ρ ( a ∗ ) / 2 < x ) ≤ C β log 3 K ( K x + log 3 K x 2 ) , (By xref‘Lemma 4.5‘) \begin{aligned}
\mathbb{P}_A\left(\max_{a\in\mathcal{A}}\mathbb{E}_{P_A}\left[\operatorname{Profit}(\widetilde{V},a)\right]\ge\mathbb{E}_{\rho}\left[\operatorname{Profit}(V,a^*)\right]+x\right) \\
&= \sum_{a\in\mathcal{A}}\mathbb{P}_{\rho}(A=a)X_a \\
&\le\sum_{a\in\mathcal{A}}\mathbb{P}_{\rho}(A=a)X_a\sum_{a'\in\mathcal{A}}n_x(a')\left(P_a(a')-P_0(a')\right)^2
\qquad\text{(By {xref}`Lemma 4.5`)} \\
&\le\frac{1}{2\beta}\sum_{a'\in\mathcal{A}}n_x(a')\Delta_{\rho}(a')
\qquad\text{(By {xref}`Claim 4.4`)} \\
&\le\frac{1}{2\beta}\sum_{a'\in\mathcal{A}}n_x(a')\left(\Delta_{\rho}(a^*,a')+2x\right)\qquad\text{(As }\Delta_{\rho}(a)=\Delta_{\rho}(a^*)+\Delta_{\rho}(a^*,a)\text{ and }\Delta_{\rho}(a^*)/2<x\text{)} \\
&\le\frac{C}{\beta}\log^3 K\left(\frac{K}{x}+\frac{\log^3 K}{x^2}\right),
\qquad\text{(By {xref}`Lemma 4.5`)}
\end{aligned} P A ( a ∈ A max E P A [ Profit ( V , a ) ] ≥ E ρ [ Profit ( V , a ∗ ) ] + x ) = a ∈ A ∑ P ρ ( A = a ) X a ≤ a ∈ A ∑ P ρ ( A = a ) X a a ′ ∈ A ∑ n x ( a ′ ) ( P a ( a ′ ) − P 0 ( a ′ ) ) 2 (By xref ‘Lemma 4.5‘) ≤ 2 β 1 a ′ ∈ A ∑ n x ( a ′ ) Δ ρ ( a ′ ) (By xref ‘Claim 4.4‘) ≤ 2 β 1 a ′ ∈ A ∑ n x ( a ′ ) ( Δ ρ ( a ∗ , a ′ ) + 2 x ) (As Δ ρ ( a ) = Δ ρ ( a ∗ ) + Δ ρ ( a ∗ , a ) and Δ ρ ( a ∗ ) /2 < x ) ≤ β C log 3 K ( x K + x 2 log 3 K ) , (By xref ‘Lemma 4.5‘)
Finally, we can use the fact that the expectation of a positive-valued random variable is the tail integral of its survival function, to obtain:
Δ ρ ( a ∗ ) = E A [ E ρ [ Profit ( V , A ) ∣ A ] − E ρ [ Profit ( V , a ∗ ) ] ] ≤ E A [ ( max a ∈ A E P A [ Profit ( V , a ) ] − E ρ [ Profit ( V , a ∗ ) ] ) + ] = ∫ 0 1 P A ( max a ∈ A E V [ Profit ( V , a ) ∣ A ] ≥ E ρ [ Profit ( V , a ⋆ ) ] + x ) d x ≤ Δ ρ ( a ⋆ ) 2 + ∫ Δ ρ ( a ⋆ ) / 2 1 C β log 3 K ( K x + log 3 K x 2 ) d x = Δ ρ ( a ⋆ ) 2 + C β log 3 K [ K log ( x ) − log 3 K x ] Δ ρ ( a ⋆ ) / 2 1 \begin{aligned}
\Delta_{\rho}(a^*)=\mathbb{E}_A\left[\mathbb{E}_{\rho}\left[\operatorname{Profit}(V,A)\mid A\right]-\mathbb{E}_{\rho}\left[\operatorname{Profit}(V,a^*)\right]\right] \\
&\le\mathbb{E}_A\left[\left(\max_{a\in\mathcal{A}}\mathbb{E}_{P_A}\left[\operatorname{Profit}(V,a)\right]-\mathbb{E}_{\rho}\left[\operatorname{Profit}(V,a^*)\right]\right)_+\right] \\
&= \int_{0}^{1} \mathbb{P}_{A}\left(\max_{a\in\mathcal{A}} \mathbb{E}_{V}\left[\operatorname{Profit}(V,a)\mid A\right] \ge\mathbb{E}_{\rho}\left[\operatorname{Profit}(V,a^\star)\right]+x\right)\,dx \\
&\le\frac{\Delta_{\rho}(a^\star)}{2}+\int_{\Delta_{\rho}(a^\star)/2}^{1}\frac{C}{\beta}\log^{3}K\left(\frac{K}{x}+\frac{\log^{3}K}{x^{2}}\right)\,dx \\
&= \frac{\Delta_{\rho}(a^\star)}{2}+\frac{C}{\beta}\log^{3}K\left[K\log(x)-\frac{\log^{3}K}{x}\right]_{\Delta_{\rho}(a^\star)/2}^{1}
\end{aligned} Δ ρ ( a ∗ ) = E A [ E ρ [ Profit ( V , A ) ∣ A ] − E ρ [ Profit ( V , a ∗ ) ] ] ≤ E A [ ( a ∈ A max E P A [ Profit ( V , a ) ] − E ρ [ Profit ( V , a ∗ ) ] ) + ] = ∫ 0 1 P A ( a ∈ A max E V [ Profit ( V , a ) ∣ A ] ≥ E ρ [ Profit ( V , a ⋆ ) ] + x ) d x ≤ 2 Δ ρ ( a ⋆ ) + ∫ Δ ρ ( a ⋆ ) /2 1 β C log 3 K ( x K + x 2 log 3 K ) d x = 2 Δ ρ ( a ⋆ ) + β C log 3 K [ K log ( x ) − x log 3 K ] Δ ρ ( a ⋆ ) /2 1
Rearranging the above, we get, assuming C 0 ≥ 1 C_{0}\ge1 C 0 ≥ 1 :
1 2 ≤ C β ( K log 3 K log ( 2 / Δ ρ ( a ⋆ ) ) Δ ρ ( a ⋆ ) + 2 log 6 K Δ ρ 2 ( a ⋆ ) ) ≤ C β ( K 2 log 3 K log ( 2 K / C 0 ) + 2 K 2 log 6 K C 0 ) \frac{1}{2}\le\frac{C}{\beta}\left(\frac{K\log^{3}K\log(2/\Delta_{\rho}(a^\star))}{\Delta_{\rho}(a^\star)}+\frac{2\log^{6}K}{\Delta_{\rho}^{2}(a^\star)}\right)\le\frac{C}{\beta}\left(\frac{K^{2}\log^{3}K\log(2K/C_{0})+2K^{2}\log^{6}K}{C_{0}}\right) 2 1 ≤ β C ( Δ ρ ( a ⋆ ) K log 3 K log ( 2/ Δ ρ ( a ⋆ )) + Δ ρ 2 ( a ⋆ ) 2 log 6 K ) ≤ β C ( C 0 K 2 log 3 K log ( 2 K / C 0 ) + 2 K 2 log 6 K )
The second inequality follows from Claim 4.3. With β = K 2 log 6 K \beta=K^{2}\log^{6}K β = K 2 log 6 K , its right-hand side is at most
C C 0 ( log ( 2 K / C 0 ) log 3 K + 2 ) ≤ 7 C C 0 ≤ 1 4 \frac{C}{C_{0}}\left(\frac{\log(2K/C_{0})}{\log^{3}K}+2\right)\le\frac{7C}{C_{0}}\le\frac{1}{4} C 0 C ( log 3 K log ( 2 K / C 0 ) + 2 ) ≤ C 0 7 C ≤ 4 1
when C 0 ≥ 32 ( C + 1 ) C_{0}\ge32(C+1) C 0 ≥ 32 ( C + 1 ) , since K ≥ 2 K\ge2 K ≥ 2 , C 0 ≥ 1 C_{0}\ge1 C 0 ≥ 1 , and log ( 2 K / C 0 ) / log 3 K ≤ 2 / ( log 2 ) 2 < 5 \log(2K/C_{0})/\log^{3}K\le2/(\log2)^{2}<5 log ( 2 K / C 0 ) / log 3 K ≤ 2/ ( log 2 ) 2 < 5 . This contradicts the lower bound 1 / 2 1/2 1/2 .
Proving the regret bound
With Theorem 4.2 proven, we can shift our focus to bounding the value of the minimax game from Equation (7) . The Multiplicative Weight Update strategy behind OneRound designs a randomized strategy for proposals which works well against the worst-case choice of a pair ( v , a ′ ) ∈ G K × A (v,a')\in G_{K}\times\mathcal{A} ( v , a ′ ) ∈ G K × A , in terms of the score definition from Equation (6) . Specifically, we show the following, recalling Algorithm 2 and Eq. (5) .
Lemma 4.6 (OneRound guarantee). Let w w w be a probability vector with w ( a ′ ) ≥ 1 / ( ∣ A ∣ ( 1 + β K ) ) w(a')\ge1/(|\mathcal{A}|(1+\beta K)) w ( a ′ ) ≥ 1/ ( ∣ A ∣ ( 1 + β K )) for all a ′ ∈ A a'\in\mathcal{A} a ′ ∈ A and let ( a , Q 0 , Q 1 ) (a,Q_{0},Q_{1}) ( a , Q 0 , Q 1 ) be the proposal returned by OneRound( w ) (w) ( w ) . Its score satisfies
max a ′ ∈ A E v ∈ G K [ h a ( v , a ′ ) ] ≤ C 0 + 2 K . (13) \max_{a'\in\mathcal{A}}\mathbb{E}_{v\in G_{K}}[h_{a}(v,a')]\le\frac{C_{0}+2}{K}. \tag*{(13)} a ′ ∈ A max E v ∈ G K [ h a ( v , a ′ )] ≤ K C 0 + 2 . (13)
Proof of Lemma 4.6. Consider the sequence Algorithm 2 would generate over all J J J iterations. At j ∈ [ J ] j\in[J] j ∈ [ J ] , write ρ j \rho_{j} ρ j for its table and a j a_{j} a j for its selected price, with h a j h_{a_{j}} h a j denoting the score using that iteration’s updates, i.e. Q 0 ρ j , Q 1 ρ j Q_{0}^{\rho_{j}},Q_{1}^{\rho_{j}} Q 0 ρ j , Q 1 ρ j .
We apply the MWU bound from e.g. Arora et al. [2 ] , which we report below, with decisions ( v , a ′ ) ∈ G K × A (v,a')\in G_{K}\times\mathcal{A} ( v , a ′ ) ∈ G K × A , costs − h a j ( v , a ′ ) / H -h_{a_{j}}(v,a')/H − h a j ( v , a ′ ) / H , and learning rate η H \eta H η H .
Proposition 4.7 (Theorem 2.1 from Arora et al. [2 ] ). Let { ℓ i , t } i ∈ X , t ∈ [ T ] \{\ell_{i,t}\}_{i\in\mathcal{X},t\in[T]} { ℓ i , t } i ∈ X , t ∈ [ T ] be a set of losses over an action space X \mathcal{X} X and time horizon T T T , bounded in [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] . Then the set of probability distributions { p t } t ∈ [ T ] \{p_{t}\}_{t\in[T]} { p t } t ∈ [ T ] generated by the Multiplicative Weights Algorithm, with learning rate γ ≤ 1 / 2 \gamma\le1/2 γ ≤ 1/2 and against the sequence { ℓ i , t } i , t \{\ell_{i,t}\}_{i,t} { ℓ i , t } i , t , satisfies, for any fixed j ∈ X j\in\mathcal{X} j ∈ X :
∑ t = 1 T ⟨ p t , { ℓ i , t } i ⟩ ≤ ∑ t = 1 T ℓ j , t + γ ∑ t = 1 T ∣ ℓ j , t ∣ + log ∣ X ∣ γ \sum_{t=1}^{T}\langle p_{t},\{\ell_{i,t}\}_{i}\rangle\le\sum_{t=1}^{T}\ell_{j,t}+\gamma\sum_{t=1}^{T}|\ell_{j,t}|+\frac{\log|\mathcal{X}|}{\gamma} t = 1 ∑ T ⟨ p t , { ℓ i , t } i ⟩ ≤ t = 1 ∑ T ℓ j , t + γ t = 1 ∑ T ∣ ℓ j , t ∣ + γ log ∣ X ∣
Considering Equation (8) and the bounds on w w w and Q z Q_{z} Q z , together with Profit ∈ [ 0 , 1 ] \operatorname{Profit}\in[0,1] Profit ∈ [ 0 , 1 ] , imply ∣ h a j ( v , a ′ ) ∣ ≤ H |h_{a_{j}}(v,a')|\le H ∣ h a j ( v , a ′ ) ∣ ≤ H . Thus the costs lie in [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] , and our choice of η \eta η ensures η H ≤ 1 / 2 \eta H\le1/2 η H ≤ 1/2 , as required by the theorem.
With these choices, the standard MWU rule multiplies each weight by
1 − ( η H ) ( − h a j ( v , a ′ ) H ) = 1 + η h a j ( v , a ′ ) . 1-(\eta H)\left(-\frac{h_{a_{j}}(v,a')}{H}\right)=1+\eta h_{a_{j}}(v,a'). 1 − ( η H ) ( − H h a j ( v , a ′ ) ) = 1 + η h a j ( v , a ′ ) .
After normalization, this is precisely our update of ρ j \rho_{j} ρ j , starting from the uniform distribution as required. Moreover, each factor is at least 1 − η H ≥ 1 / 2 1-\eta H\ge1/2 1 − η H ≥ 1/2 , so the tables remain positive and all conditional updates are well defined.
Substituting these costs in Proposition 4.7 and rescaling gives, for every fixed ( v , a ′ ) (v,a') ( v , a ′ ) :
1 J ∑ j = 1 J h a j ( v , a ′ ) ≤ 1 J ∑ j = 1 J E ρ j [ h a j ( V , A ) ] + log ( ∣ A ∣ ∣ G K ∣ ) η J + η H 2 ≤ 1 J ∑ j = 1 J E ρ j [ h a j ( V , A ) ] + 1 K . (14) \frac{1}{J}\sum_{j=1}^{J}h_{a_j}(v,a')\leq\frac{1}{J}\sum_{j=1}^{J}\mathbb{E}_{\rho_j}\left[h_{a_j}(V,A)\right]+\frac{\log(|\mathcal{A}||\mathcal{G}_K|)}{\eta J}+\eta H^2
\leq\frac{1}{J}\sum_{j=1}^{J}\mathbb{E}_{\rho_j}\left[h_{a_j}(V,A)\right]+\frac{1}{K}. \tag*{(14)} J 1 j = 1 ∑ J h a j ( v , a ′ ) ≤ J 1 j = 1 ∑ J E ρ j [ h a j ( V , A ) ] + η J log ( ∣ A ∣∣ G K ∣ ) + η H 2 ≤ J 1 j = 1 ∑ J E ρ j [ h a j ( V , A ) ] + K 1 . (14)
where the last inequality only uses our choices in Equation (8) . Therefore, in order to get the claimed bound, it suffices to bound each weighted average by ( C 0 + 1 ) / K (C_0+1)/K ( C 0 + 1 ) / K .
By Equation (5) , for every a ′ ∈ A a'\in\mathcal{A} a ′ ∈ A and z ∈ { 0 , 1 } z\in\{0,1\} z ∈ { 0 , 1 } , Q z ρ j ( a ′ ) ≥ β K 1 + β K P ρ j ( A = a ′ ∣ z a j ( V ) = z ) Q_z^{\rho_j}(a')\geq\frac{\beta K}{1+\beta K}\mathbb{P}_{\rho_j}(A=a'\mid z_{a_j}(V)=z) Q z ρ j ( a ′ ) ≥ 1 + β K β K P ρ j ( A = a ′ ∣ z a j ( V ) = z ) . To identify the mutual-information term, insert the marginal probability of A A A into the conditional log ratio:
log P ρ j ( A = a ′ ∣ z a j ( V ) = z ) w ( a ′ ) = log P ρ j ( A = a ′ ∣ z a j ( V ) = z ) P ρ j ( A = a ′ ) + log P ρ j ( A = a ′ ) w ( a ′ ) . \log\frac{\mathbb{P}_{\rho_j}(A=a'\mid z_{a_j}(V)=z)}{w(a')}=\log\frac{\mathbb{P}_{\rho_j}(A=a'\mid z_{a_j}(V)=z)}{\mathbb{P}_{\rho_j}(A=a')}+\log\frac{\mathbb{P}_{\rho_j}(A=a')}{w(a')}. log w ( a ′ ) P ρ j ( A = a ′ ∣ z a j ( V ) = z ) = log P ρ j ( A = a ′ ) P ρ j ( A = a ′ ∣ z a j ( V ) = z ) + log w ( a ′ ) P ρ j ( A = a ′ ) .
Averaging the first term over A A A and its feedback gives I ρ j ( a j ) \mathcal{I}_{\rho_j}(a_j) I ρ j ( a j ) , using the alternative definition for mutual information considered also in Equation (12) . Combining this identity with the preceding lower bound on Q z Q_z Q z thus yields:
E ρ j [ log Q z a j ( V ) ρ j ( A ) w ( A ) ] ≥ I ρ j ( a j ) + ∑ a ′ ∈ A P ρ j ( A = a ′ ) log P ρ j ( A = a ′ ) w ( a ′ ) − log ( 1 + 1 β K ) ≥ I ρ j ( a j ) − 1 β K , \begin{aligned}
\mathbb{E}_{\rho_j}\left[\log\frac{Q_{z_{a_j}(V)}^{\rho_j}(A)}{w(A)}\right]\geq\mathcal{I}_{\rho_j}(a_j)+\sum_{a'\in\mathcal{A}}\mathbb{P}_{\rho_j}(A=a')\log\frac{\mathbb{P}_{\rho_j}(A=a')}{w(a')}-\log\left(1+\frac{1}{\beta K}\right) \\
&\geq\mathcal{I}_{\rho_j}(a_j)-\frac{1}{\beta K},
\end{aligned} E ρ j log w ( A ) Q z a j ( V ) ρ j ( A ) ≥ I ρ j ( a j ) + a ′ ∈ A ∑ P ρ j ( A = a ′ ) log w ( a ′ ) P ρ j ( A = a ′ ) − log ( 1 + β K 1 ) ≥ I ρ j ( a j ) − β K 1 ,
where the second inequality follows because the sum is nonnegative and log ( 1 + 1 / ( β K ) ) ≤ 1 / ( β K ) \log(1+1/(\beta K))\leq1/(\beta K) log ( 1 + 1/ ( β K )) ≤ 1/ ( β K ) . Consequently,
E ρ j [ h a j ( V , A ) ] ≤ Δ ρ j ( a j ) − β I ρ j ( a j ) + 1 K . \mathbb{E}_{\rho_j}\left[h_{a_j}(V,A)\right]\leq\Delta_{\rho_j}(a_j)-\beta\mathcal{I}_{\rho_j}(a_j)+\frac{1}{K}. E ρ j [ h a j ( V , A ) ] ≤ Δ ρ j ( a j ) − β I ρ j ( a j ) + K 1 .
Therefore, Theorem 4.2 and the algorithm’s minimization over candidate prices imply E ρ j [ h a j ( V , A ) ] ≤ ( C 0 + 1 ) / K \mathbb{E}_{\rho_j}\left[h_{a_j}(V,A)\right]\leq(C_0+1)/K E ρ j [ h a j ( V , A ) ] ≤ ( C 0 + 1 ) / K for all j ∈ [ J ] j\in[J] j ∈ [ J ] , for some constant C 0 C_0 C 0 . The actual algorithm stops at a uniformly at random chosen index j ⋆ ∈ [ J ] j^\star\in[J] j ⋆ ∈ [ J ] . Therefore, for every fixed ( v , a ′ ) (v,a') ( v , a ′ ) , Equation (14) yields, taking the expectation over the random sampling of an index in { 1 , … , J } \{1,\ldots,J\} { 1 , … , J } :
E a , Q 0 , Q 1 [ h a ( v , a ′ ) ] = 1 J ∑ j = 1 J h a j ( v , a ′ ) ≤ C 0 + 1 K + 1 K = C 0 + 2 K . \mathbb{E}_{a,Q_0,Q_1}[h_a(v,a')]=\frac{1}{J}\sum_{j=1}^{J}h_{a_j}(v,a')\leq\frac{C_0+1}{K}+\frac{1}{K}=\frac{C_0+2}{K}. E a , Q 0 , Q 1 [ h a ( v , a ′ )] = J 1 j = 1 ∑ J h a j ( v , a ′ ) ≤ K C 0 + 1 + K 1 = K C 0 + 2 .
Using the guarantee for a single round, we can complete the proof for the adversarial regret bound, in a way identical to Theorem 3.1 in Xu and Zeevi [44 ] .
Proof of Theorem 4.1. We use Lemma 4.6 to bound h a h_a h a in each round: the expected profit gap plus the change in log weights. The changes in weight then telescope when summed over rounds. The lower-bound condition on w t w_t w t of Lemma 4.6 holds initially and is preserved by each update prescribed as in Equation (5) . Now fix a comparison price a ′ ∈ A a'\in\mathcal{A} a ′ ∈ A .
Condition on all observations and internal random choices from rounds ( 1 , … , t − 1 ) (1,\ldots,t-1) ( 1 , … , t − 1 ) . This fixes w t w_t w t , while v t v_t v t is fixed in advance. Applying the lemma to the fresh randomness used by OᴎᴇRᴏᴜɴᴅ in round t t t , we obtain:
E [ Profit ( v t , a ′ ) − Profit ( v t , a t ) − β log w t + 1 ( a ′ ) w t ( a ) ∣ History from rounds 1 , … , t − 1 ] ≤ C 0 + 2 K . \mathbb{E}\left[\left.\operatorname{Profit}(v_t,a')-\operatorname{Profit}(v_t,a_t)-\beta\log\frac{w_{t+1}(a')}{w_t(a)}\right|\text{History from rounds }1,\ldots,t-1\right]\leq\frac{C_0+2}{K}. E [ Profit ( v t , a ′ ) − Profit ( v t , a t ) − β log w t ( a ) w t + 1 ( a ′ ) History from rounds 1 , … , t − 1 ] ≤ K C 0 + 2 .
Taking expectations and summing over t t t , the logarithms telescope:
∑ t = 1 T Profit ( v t , a ′ ) − E [ ∑ t = 1 T Profit ( v t , a t ) ] ≤ ( C 0 + 2 ) T K + β E [ log ( w T + 1 ( a ′ ) w 1 ( a ′ ) ) ] ≤ ( C 0 + 2 ) T K + β log ∣ A ∣ . \begin{aligned}
\sum_{t=1}^{T} \operatorname{Profit}(v_t,a')-\mathbb{E}\left[\sum_{t=1}^{T}\operatorname{Profit}(v_t,a_t)\right]\leq\frac{(C_0+2)T}{K}+\beta\mathbb{E}\left[\log\left(\frac{w_{T+1}(a')}{w_1(a')}\right)\right] \\
&\leq\frac{(C_0+2)T}{K}+\beta\log|\mathcal{A}|.
\end{aligned} t = 1 ∑ T Profit ( v t , a ′ ) − E [ t = 1 ∑ T Profit ( v t , a t ) ] ≤ K ( C 0 + 2 ) T + β E [ log ( w 1 ( a ′ ) w T + 1 ( a ′ ) ) ] ≤ K ( C 0 + 2 ) T + β log ∣ A ∣.
where the last inequality uses w 1 ( a ′ ) = 1 / ∣ A ∣ w_1(a')=1/|\mathcal{A}| w 1 ( a ′ ) = 1/∣ A ∣ and w T + 1 ( a ′ ) ≤ 1 w_{T+1}(a')\leq1 w T + 1 ( a ′ ) ≤ 1 .
Restricting to v t ∈ G K v_t\in G_K v t ∈ G K is without loss of generality since rounding seller values up and buyer values down to the grid preserves the profit and feedback for any action in A \mathcal{A} A . Moreover, comparing against a price ( p , q ) ∈ T ∖ G K (p,q)\in\mathcal{T}\setminus G_K ( p , q ) ∈ T ∖ G K can add at most 2 T / K 2T/K 2 T / K regret, by Lemma A.6 . This proves the regret bound from Theorem 4.1, factoring in the choice of β \beta β , and setting K = Θ ( ( T / log 7 ( T ) ) 1 / 3 ) K=\Theta((T/\log^7(T))^{1/3}) K = Θ (( T / log 7 ( T ) ) 1/3 ) .
To conclude, we prove that the algorithm runs in polynomial time. Notice that, for each of its iterations, OneRound needs to scan over the whole grid for the candidate posted price to use for the updates in Equation (5) , and for each candidate price it needs to scan G K × A G_K\times\mathcal{A} G K × A again to compute the conditional probabilities in the updates, for a O ( K 6 ) O(K^6) O ( K 6 ) complexity. Factoring in the values of J J J and H H H , which determine the maximum number of iterations of OneRound, and the fact that OneRound is called once for each time step, we obtain a crude O ~ ( T K 12 ) \widetilde{O}(TK^{12}) O ( T K 12 ) bound.
Low Regret Estimation in the i.i.d. Setting
This section is dedicated to proving the crucial estimation lemma of the regret bound, namely Lemma 4.5. For this section, we consider the stochastic i.i.d. setting, meaning that the distribution of the valuation pairs satisfies a specific and stationary probability function of trade P P P , on [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 . Given a choice of distribution ρ \rho ρ on G K × A G_K\times\mathcal{A} G K × A , we also have a reference marginal probability of trade P 0 P_0 P 0 on G K G_K G K , according to Equation (11) .
Even though we assign it a specific meaning in § 4.2 , from now on and for the rest of the section, ρ \rho ρ can be regarded as any valid distribution and P 0 P_0 P 0 as its probability trade function. Indeed, the estimation routine presented here can be immediately used to obtain O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret rate in the stochastic i.i.d. setting via successive elimination, as explained in § 3.1 , setting P 0 P_0 P 0 to some probability function of trade compatible with the confidence intervals considered at a specific stage.
At a very high level, what we want to prove is that there exists an estimation routine for the stochastic i.i.d. setting satisfying the following two properties:
The regret of the routine is small with respect to the probability function of trade P 0 P_0 P 0 .
For any other probability function of trade P P P with maximum achievable profit substantially larger than P 0 P_0 P 0 , the probability of trade under the two distributions must differ significantly at the prices posted by the routine (when receiving feedback drawn from P 0 P_0 P 0 ).
Lemma 4.5 (Inexpensive Detection). For every distribution ρ \rho ρ on G K × A G_K\times\mathcal{A} G K × A , and x ∈ ( 0 , 1 ) x\in(0,1) x ∈ ( 0 , 1 ) , there are non-negative weights n x ( a ) n_x(a) n x ( a ) (which depend only on ρ \rho ρ and x x x ) such that the following inequality is verified, for some constant C C C :
∑ a ∈ A n x ( a ) ( Δ ρ ( a ∗ , a ) + x ) ≤ C log 3 K ( K x + log 3 K x 2 ) . \sum_{a\in\mathcal{A}}n_x(a)\left(\Delta_\rho(a^*,a)+x\right)\leq C\log^3K\left(\frac{K}{x}+\frac{\log^3K}{x^2}\right). a ∈ A ∑ n x ( a ) ( Δ ρ ( a ∗ , a ) + x ) ≤ C log 3 K ( x K + x 2 log 3 K ) . where a ∗ a^* a ∗ is optimal according to ρ \rho ρ . Moreover, for any random valuation V ~ \widetilde{V} V with probability function of trade P P P such that max a ∈ A E [ Profit ( V ~ , a ) ] ≥ E ρ [ Profit ( V , a ∗ ) ] + x \max_{a\in\mathcal{A}}\mathbb{E}[\operatorname{Profit}(\widetilde{V},a)]\geq\mathbb{E}_{\rho}[\operatorname{Profit}(V,a^*)]+x max a ∈ A E [ Profit ( V , a )] ≥ E ρ [ Profit ( V , a ∗ )] + x , then the same weights respect, with P 0 P_0 P 0 induced by ρ \rho ρ :
∑ a ∈ A n x ( a ) ( P ( a ) − P 0 ( a ) ) 2 ≥ 1. \sum_{a\in\mathcal{A}}n_x(a)(P(a)-P_0(a))^2\geq1. a ∈ A ∑ n x ( a ) ( P ( a ) − P 0 ( a ) ) 2 ≥ 1.
To claim the above, we need a delicate construction that lets us estimate P P P while minimizing regret induced under P 0 P_0 P 0 . We can do this by designing an algorithm which fully exploits the access to P 0 P_0 P 0 itself. This — together with more standard concentration arguments — gives the first bound of the claim, while an information theoretic argument gives the second once we have the first. More specifically, the first bound of the claim follows by the fundamental lemma below, which we prove in a sequence of steps.
Lemma 4.8. For every precision ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] , there exists a routine estimating E P 0 [ Profit ( V ~ , a ) ] \mathbb{E}_{P_0}[\operatorname{Profit}(\tilde{V},a)] E P 0 [ Profit ( V ~ , a )] up to precision ε / 4 \varepsilon/4 ε /4 , with probability at least 9 / 10 9/10 9/10 and simultaneously for all actions in A \mathcal{A} A satisfying Δ ρ ( a ∗ , a ) ≤ ε \Delta_\rho(a^*,a) \le\varepsilon Δ ρ ( a ∗ , a ) ≤ ε . Moreover, for some constant C C C , this routine makes at most C log 3 ( K ) ( K ε 2 + log 3 K ε 3 ) C\log^3(K)\left(\frac{K}{\varepsilon^2}+\frac{\log^3 K}{\varepsilon^3}\right) C log 3 ( K ) ( ε 2 K + ε 3 l o g 3 K ) pricing queries in total, and each played action a ∈ A a \in\mathcal{A} a ∈ A satisfies Δ ρ ( a ∗ , a ) ≤ 2 ε \Delta_\rho(a^*,a) \le2\varepsilon Δ ρ ( a ∗ , a ) ≤ 2 ε .
Proof. Step 1: Build a tree over the seller domain We start by arranging the seller domain G K , s = { 0 , 1 / K , … , 1 } G_{K,s} = \{0,1/K,\ldots,1\} G K , s = { 0 , 1/ K , … , 1 } in a binary tree. We do this in the usual recursive fashion: starting from the whole G K , s G_{K,s} G K , s , split a node I I I if ∣ I ∣ ≥ 2 |I| \ge2 ∣ I ∣ ≥ 2 , and assign to the left child the first ⌊ ∣ I ∣ / 2 ⌋ \lfloor|I|/2 \rfloor ⌊ ∣ I ∣/2 ⌋ elements in increasing order, while leaving the rest to the other child.
Step 2: Control profit differences We now control the variation of profit when fixing seller coordinates and varying buyer coordinates. Let l l l and r r r be two seller prices in a seller node I I I with l ≤ r l \le r l ≤ r and let q 0 q_0 q 0 and q q q be two buyer prices with q 0 ≤ q q_0 \le q q 0 ≤ q . We want to control the following difference:
Δ ρ ( ( r , q ) , ( l , q ) ) − Δ ρ ( ( r , q 0 ) , ( l , q 0 ) ) \Delta_\rho((r,q),(l,q))-\Delta_\rho((r,q_0),(l,q_0)) Δ ρ (( r , q ) , ( l , q )) − Δ ρ (( r , q 0 ) , ( l , q 0 ))
We analyze the above by decomposing the space where trade is allowed to happen under the considered posted prices, and conditioning correspondingly. First, if the valuation V = ( S , B ) V=(S,B) V = ( S , B ) belongs to [ 0 , l ] × [ q , 1 ] [0,l]\times[q,1] [ 0 , l ] × [ q , 1 ] , all four considered pairs of prices trade, and the above difference can be shown to be null. If S ∈ [ 0 , l ] S\in[0,l] S ∈ [ 0 , l ] and B ∈ [ q 0 , q ) B\in[q_0,q) B ∈ [ q 0 , q ) , then posting q q q prevents trade. Only the second term is nonzero: it equals l − r l-r l − r , so subtracting it gives the overall difference r − l r-l r − l . If S ∈ ( l , r ] S\in(l,r] S ∈ ( l , r ] and B ∈ [ q 0 , q ) B\in[q_0,q) B ∈ [ q 0 , q ) , only ( r , q 0 ) (r,q_0) ( r , q 0 ) trades, and the overall difference is − ( q 0 − r ) = r − q 0 -(q_0-r)=r-q_0 − ( q 0 − r ) = r − q 0 . If S ∈ ( l , r ] S\in(l,r] S ∈ ( l , r ] and B ∈ [ q , 1 ] B\in[q,1] B ∈ [ q , 1 ] , posting l l l gives null profit, and the above difference evaluates to q − q 0 q-q_0 q − q 0 . Averaging, we thus obtain that the absolute value of the difference is bounded by:
∣ r − l ∣ ⋅ P 0 ( [ 0 , l ] × [ q 0 , q ) ) + ∣ q − q 0 ∣ ⋅ P 0 ( ( l , r ] × [ q , 1 ] ) + ∣ q 0 − r ∣ ⋅ P 0 ( ( l , r ] × [ q 0 , q ) ) ≤ ∣ r − l ∣ ⋅ P 0 ( [ 0 , 1 ] × [ q 0 , q ) ) + ∣ q − q 0 ∣ ⋅ P 0 ( ( l , r ] × [ 0 , 1 ] ) + P 0 ( ( l , r ] × [ q 0 , q ) ) (15) \begin{aligned}
&|r-l|\cdot\mathbb{P}_0([0,l]\times[q_0,q))+|q-q_0|\cdot\mathbb{P}_0((l,r]\times[q,1])\\
&\qquad+|q_0-r|\cdot\mathbb{P}_0((l,r]\times[q_0,q))\\
&\le|r-l|\cdot\mathbb{P}_0([0,1]\times[q_0,q))+|q-q_0|\cdot\mathbb{P}_0((l,r]\times[0,1])\\
&\qquad+\mathbb{P}_0((l,r]\times[q_0,q)) \tag*{(15)}
\end{aligned} ∣ r − l ∣ ⋅ P 0 ([ 0 , l ] × [ q 0 , q )) + ∣ q − q 0 ∣ ⋅ P 0 (( l , r ] × [ q , 1 ]) + ∣ q 0 − r ∣ ⋅ P 0 (( l , r ] × [ q 0 , q )) ≤ ∣ r − l ∣ ⋅ P 0 ([ 0 , 1 ] × [ q 0 , q )) + ∣ q − q 0 ∣ ⋅ P 0 (( l , r ] × [ 0 , 1 ]) + P 0 (( l , r ] × [ q 0 , q )) (15)
Accordingly, define the positive measure ν \nu ν on [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 in the following way:
ν = L ⊗ P 0 , b + P 0 , s ⊗ L + P 0 (16) \nu=\mathcal{L}\otimes\mathbb{P}_{0,b}+\mathbb{P}_{0,s}\otimes\mathcal{L}+\mathbb{P}_0 \tag*{(16)} ν = L ⊗ P 0 , b + P 0 , s ⊗ L + P 0 (16)
where L \mathcal{L} L is the Lebesgue measure and P 0 , b \mathbb{P}_{0,b} P 0 , b , P 0 , s \mathbb{P}_{0,s} P 0 , s are the two marginals stemming from the distribution P 0 \mathbb{P}_0 P 0 induced by the trade probability function P 0 P_0 P 0 . Combining Equations (15) and (16) , we then get that:
∣ Δ ρ ( ( r , q ) , ( l , q ) ) − Δ ρ ( ( r , q 0 ) , ( l , q 0 ) ) ∣ ≤ ν ( ( l , r ] × [ q 0 , q ) ) (17) \left|\Delta_\rho((r,q),(l,q))-\Delta_\rho((r,q_0),(l,q_0))\right|\le\nu((l,r]\times[q_0,q)) \tag*{(17)} ∣ Δ ρ (( r , q ) , ( l , q )) − Δ ρ (( r , q 0 ) , ( l , q 0 )) ∣ ≤ ν (( l , r ] × [ q 0 , q )) (17)
Importantly, ν \nu ν is finite, and its total mass is equal to 3.
Step 3: Split the buyer coordinates into blocks In this step, for each seller node I I I , we partition the buyer grid into a small number of blocks and choose a representative seller price p I , D ∈ I p_{I,D}\in I p I , D ∈ I for each block D D D . Our goal is to ensure that
E P 0 [ Profit ( V , ( p I , D , q ) ) ] ≥ max p ∈ I E P 0 [ Profit ( V , ( p , q ) ) ] − ε ∀ q ∈ D \mathbb{E}_{P_0}[\operatorname{Profit}(V,(p_{I,D},q))]\ge\max_{p\in I}\mathbb{E}_{P_0}[\operatorname{Profit}(V,(p,q))]-\varepsilon\quad\forall q\in D E P 0 [ Profit ( V , ( p I , D , q ))] ≥ p ∈ I max E P 0 [ Profit ( V , ( p , q ))] − ε ∀ q ∈ D
We use the definition of ν \nu ν to partition the set of buyer coordinates G K , b G_{K,b} G K , b , for any given node I ⊆ G K , s I\subseteq G_{K,s} I ⊆ G K , s of seller coordinates. Fixing I I I , the partition { D ℓ , I } ℓ \{D_{\ell,I}\}_{\ell} { D ℓ , I } ℓ is then defined iteratively, according to the following simple algorithm.
Algorithm 3 Partition buyer coordinates
Require: Node I I I , ordered grid G K , b G_{K,b} G K , b , measure ν \nu ν , threshold ε \varepsilon ε
D 1 , I ← { min G K , b } D_{1,I}\leftarrow\{\min G_{K,b}\} D 1 , I ← { min G K , b } , ℓ ← 1 \ell\leftarrow1 ℓ ← 1
for each q ∈ G K , b q\in G_{K,b} q ∈ G K , b in increasing order do
if ν ( ( min I , max I ] × [ min D ℓ , I , q ) ) ≤ ε \nu((\min I,\max I]\times[\min D_{\ell,I},q))\le\varepsilon ν (( min I , max I ] × [ min D ℓ , I , q )) ≤ ε then
D ℓ , I ← D ℓ , I ∪ { q } D_{\ell,I}\leftarrow D_{\ell,I}\cup\{q\} D ℓ , I ← D ℓ , I ∪ { q }
else
ℓ ← ℓ + 1 \ell\leftarrow\ell+1 ℓ ← ℓ + 1 , D ℓ , I ← { q } D_{\ell,I}\leftarrow\{q\} D ℓ , I ← { q }
return ( D 1 , I , … , D ℓ , I ) (D_{1,I},\ldots,D_{\ell,I}) ( D 1 , I , … , D ℓ , I )
Figure 4. The decomposition considered in Step 2 of the proof of Lemma 4.8 : the area filled with squares is the region of the space when no trade happens under any of the four posted prices; each area where some trade happens and thus considered in the decomposition is filled with a distinct pattern.
It follows that we are splitting G K , b G_{K,b} G K , b in contiguous blocks which are maximal according to ν \nu ν : adding the next coordinate in increasing order to any of the blocks would violate the condition ν ( ( min I , max I ] × [ min D ℓ , I , max D ℓ , I ) ) ≤ ε \nu((\min I,\max I]\times[\min D_{\ell,I},\max D_{\ell,I}))\leq\varepsilon ν (( min I , max I ] × [ min D ℓ , I , max D ℓ , I )) ≤ ε . Notice that we can bound the cardinality of ∣ { D ℓ , I } ℓ ∣ \lvert\{D_{\ell,I}\}_{\ell}\rvert ∣{ D ℓ , I } ℓ ∣ according to ν \nu ν . Specifically, each element of the partition is maximal with respect to ν \nu ν , i.e. we add a new block only when adding one more coordinate to the current block would make mass greater than ε \varepsilon ε . Therefore the cardinality of the partition cannot exceed one plus the total mass of ν \nu ν on ( min I , max I ] × [ 0 , 1 ] (\min I,\max I]\times[0,1] ( min I , max I ] × [ 0 , 1 ] divided by ε \varepsilon ε . In other words, we can have at most
1 + ν ( ( min I , max I ] × [ 0 , 1 ] ) ε (18) 1+\frac{\nu((\min I,\max I]\times[0,1])}{\varepsilon} \tag*{(18)} 1 + ε ν (( min I , max I ] × [ 0 , 1 ]) (18)
buyer coordinate blocks for each I I I .
Now, for each seller node I I I and buyer block D D D , choose the smallest maximizer at buyer price min D \min D min D :
p I , D = min arg max p ∈ I { ( min D − p ) P 0 ( p , min D ) } . (19) p_{I,D}=\min\mathop{\arg\max}_{p\in I}\{(\min D-p)P_0(p,\min D)\}. \tag*{(19)} p I , D = min arg max p ∈ I {( min D − p ) P 0 ( p , min D )} . (19)
The crucial property of this construction is that, taking any p ∈ I p\in I p ∈ I and q ∈ D q\in D q ∈ D and assuming p ≤ p I , D p\leq p_{I,D} p ≤ p I , D , we have ( p , p I , D ] × [ min D , q ) ⊆ ( min I , max I ] × [ min D , max D ) (p,p_{I,D}]\times[\min D,q)\subseteq(\min I,\max I]\times[\min D,\max D) ( p , p I , D ] × [ min D , q ) ⊆ ( min I , max I ] × [ min D , max D ) , so that ν ( ( p , p I , D ] × [ min D , q ) ) ≤ ε \nu((p,p_{I,D}]\times[\min D,q))\leq\varepsilon ν (( p , p I , D ] × [ min D , q )) ≤ ε by definition of the blocks. In turn, using Equation (17) , this implies the following:
Δ ρ ( ( p I , D , q ) , ( p , q ) ) ≥ Δ ρ ( ( p I , D , min D ) , ( p , min D ) ) − ε ≥ − ε , (20) \Delta_{\rho}((p_{I,D},q),(p,q))\geq\Delta_{\rho}((p_{I,D},\min D),(p,\min D))-\varepsilon\geq-\varepsilon, \tag*{(20)} Δ ρ (( p I , D , q ) , ( p , q )) ≥ Δ ρ (( p I , D , min D ) , ( p , min D )) − ε ≥ − ε , (20)
where the last inequality follows by the definition of p I , D p_{I,D} p I , D as maximizer seller price in I I I at buyer price min D \min D min D . An identical reasoning goes through if p > p I , D p>p_{I,D} p > p I , D , by reversing the interval extremes accordingly and using that Δ ρ ( ( p , q ) , ( p I , D , q ) ) = − Δ ρ ( ( p I , D , q ) , ( p , q ) ) \Delta_{\rho}((p,q),(p_{I,D},q))=-\Delta_{\rho}((p_{I,D},q),(p,q)) Δ ρ (( p , q ) , ( p I , D , q )) = − Δ ρ (( p I , D , q ) , ( p , q )) . Therefore, each block D D D has a single seller price p I , D p_{I,D} p I , D which is approximately optimal — under P 0 P_0 P 0 and over I I I — for any considered buyer price q ∈ D q\in D q ∈ D .
Step 4: Reconstruct the probability of trade We want to use the previous constructions to efficiently recover P ( p , q ) P(p,q) P ( p , q ) for any ( p , q ) (p,q) ( p , q ) satisfying Δ ρ ( a ∗ , ( p , q ) ) ≤ ε \Delta_{\rho}(a^*,(p,q))\leq\varepsilon Δ ρ ( a ∗ , ( p , q )) ≤ ε , while satisfying the constraint of playing only 2 ε 2\varepsilon 2 ε -optimal actions, i.e. prices in { a ∈ A : Δ ρ ( a ∗ , a ) ≤ 2 ε } \{a\in A:\Delta_{\rho}(a^*,a)\leq2\varepsilon\} { a ∈ A : Δ ρ ( a ∗ , a ) ≤ 2 ε } .
Due to the partitioning from Step 1, there must exist a leaf in the binary tree with p p p as its value; correspondingly, we must have a path in the tree I 0 ⊇ I 1 ⊇ … ⊇ I h I_0\supseteq I_1\supseteq\ldots\supseteq I_h I 0 ⊇ I 1 ⊇ … ⊇ I h with I 0 = G K , s I_0=G_{K,s} I 0 = G K , s , I h = { p } I_h=\{p\} I h = { p } and h ≤ 1 + log 2 ( K + 1 ) h\leq1+\log_2(K+1) h ≤ 1 + log 2 ( K + 1 ) . We must also have a sequence of non-disjoint blocks D 0 , D 1 , … , D h D_0,D_1,\ldots,D_h D 0 , D 1 , … , D h of buyer coordinates, one for each node in the root-to-leaf path, for which q ∈ D j q \in D_j q ∈ D j for any j ∈ { 0 , 1 , … , h } j \in\{0,1,\ldots,h\} j ∈ { 0 , 1 , … , h } . See Figure 1 in § 3.1 for an illustration. We can then consider the following telescoping decomposition of P ( p , q ) P(p,q) P ( p , q ) :
P ( p , q ) = P ( p I 0 , D 0 , q ) + ∑ j = 1 h ( P ( p I j , D j , q ) − P ( p I j − 1 , D j − 1 , q ) ) (21) P(p,q) = P(p_{I_0,D_0},q) + \sum_{j=1}^{h} \left(P(p_{I_j,D_j},q) - P(p_{I_{j-1},D_{j-1}},q)\right) \tag*{(21)} P ( p , q ) = P ( p I 0 , D 0 , q ) + j = 1 ∑ h ( P ( p I j , D j , q ) − P ( p I j − 1 , D j − 1 , q ) ) (21)
Claim 4.9. Any of the prices considered in the telescopic decomposition are contained in A \mathcal{A} A and are 2 ε 2\varepsilon 2 ε -optimal according to P 0 P_0 P 0 , if ( p , q ) (p,q) ( p , q ) is ε \varepsilon ε -optimal. Moreover, each term highlighted with (#) is monotone in q q q , for q ∈ D j ∩ D j − 1 q \in D_j \cap D_{j-1} q ∈ D j ∩ D j − 1 for the j j j -th parent to child difference, and for q ∈ D 0 q \in D_0 q ∈ D 0 for the root node value.
Proof. Notice that ( p , q ) (p,q) ( p , q ) is assumed to be ε \varepsilon ε -optimal according to P 0 P_0 P 0 , meaning that Equation (20) immediately yields 2 ε 2\varepsilon 2 ε -optimality, since p ∈ I j p \in I_j p ∈ I j and q ∈ D j q \in D_j q ∈ D j by construction. To prove that each belongs to A \mathcal{A} A , we proceed by contradiction, using that in Equation (19) ties are broken by taking minima.
First, if ( p I , D , q ) ∉ A (p_{I,D},q) \notin\mathcal{A} ( p I , D , q ) ∈ / A , then p I , D > q p_{I,D} > q p I , D > q . By assumption then there exists p ∈ I p \in I p ∈ I with ( p , q ) ∈ A (p,q) \in\mathcal{A} ( p , q ) ∈ A and thus p ≤ q < p I , D p \le q < p_{I,D} p ≤ q < p I , D , with the profit obtained at ( p , min D ) (p,\min D) ( p , min D ) lower bounded by the one obtained at ( p I , D , min D ) (p_{I,D},\min D) ( p I , D , min D ) : this follows since posting ( p I , D , min D ) (p_{I,D},\min D) ( p I , D , min D ) always results in negative profit when trading, due to min D ≤ q \min D \le q min D ≤ q and ( p I , D , q ) ∉ A (p_{I,D},q) \notin\mathcal{A} ( p I , D , q ) ∈ / A . Therefore, p p p is also a seller price maximizer at buyer price min D \min D min D , and since p < p I , D p < p_{I,D} p < p I , D , this contradicts the fact that p I , D p_{I,D} p I , D should be the smallest maximizer.
Finally, monotonicity follows by definition of the probability function of trade, see e.g. Equation (10) , and since p I j , D j p_{I_j,D_j} p I j , D j and p I j − 1 , D j − 1 p_{I_{j-1},D_{j-1}} p I j − 1 , D j − 1 — or p I 0 , D 0 p_{I_0,D_0} p I 0 , D 0 , for the root node value — remain fixed, provided q q q is only allowed to vary in the ranges mentioned in the claim. □
Therefore, in order to use Equation (21) for the routine guarantee, we need to estimate one monotone function in q q q for any parent to child edge { I j − 1 , I j } \{I_{j-1},I_j\} { I j − 1 , I j } and intersection of the type D j ∩ D j − 1 D_j \cap D_{j-1} D j ∩ D j − 1 , together with one for each buyer block of the root node. To satisfy the constraint of considering only 2 ε 2\varepsilon 2 ε -optimal actions, it then suffices to restrict the domain of each function to D j ∩ D j − 1 ∩ { q ∈ G K , b : ∃ p ∈ I j such that ( p , q ) ∈ A , Δ ρ ( a ∗ , ( p , q ) ) ≤ ε } D_j \cap D_{j-1} \cap\{q \in G_{K,b} : \exists p \in I_j \text{ such that } (p,q) \in\mathcal{A}, \Delta_\rho(a^*,(p,q)) \le\varepsilon\} D j ∩ D j − 1 ∩ { q ∈ G K , b : ∃ p ∈ I j such that ( p , q ) ∈ A , Δ ρ ( a ∗ , ( p , q )) ≤ ε } , and analogously for the root functions. This guarantees that for every q q q in the resulting domain we have the same certificate of optimality granted by Claim 4.9. Since we are interested in estimating P P P only for ε \varepsilon ε -optimal points of the grid, this restriction does not impact the telescoping decomposition in Equation (21) .
We now count the number of monotone functions in the class resulting from this procedure, which we denote as F \mathcal{F} F . By the bound on the number of buyer coordinates blocks from Equation (18) and considering that at each depth of the seller tree the ranges of the nodes are pairwise disjoint, we get that the overall number of possible pairs ( I , D ) (I,D) ( I , D ) of seller node and buyer block is bounded by, for K ≥ 40 K \ge40 K ≥ 40 :
2 K + 1 + ν ( [ 0 , 1 ] 2 ) ε ⋅ 2 log K ≤ 3 K + 3 ε ⋅ 2 log K . (22) 2K + 1 + \frac{\nu([0,1]^2)}{\varepsilon} \cdot2\log K \le3K + \frac{3}{\varepsilon}\cdot2\log K. \tag*{(22)} 2 K + 1 + ε ν ([ 0 , 1 ] 2 ) ⋅ 2 log K ≤ 3 K + ε 3 ⋅ 2 log K . (22)
In the above we have used the bound on the total mass of ν \nu ν and the fact that the number of nodes in a full binary tree with K + 1 K+1 K + 1 leaves is 2 K + 1 2K+1 2 K + 1 , over a number of levels bounded by 2 + log 2 ( K + 1 ) 2+\log_2(K+1) 2 + log 2 ( K + 1 ) , considering the halving rule from Step 1.
It remains to factor in that we have to estimate a different function for every distinct and non-empty buyer block intersection of the type D j ∩ D j − 1 D_j \cap D_{j-1} D j ∩ D j − 1 , for any edge connecting a parent node I j − 1 I_{j-1} I j − 1 to a child node I j I_j I j . To do this, notice that the buyer blocks of I j − 1 I_{j-1} I j − 1 and I j I_j I j form two different partitions of G K , b G_{K,b} G K , b , with contiguous coordinates in each block, so that the number of nonempty intersections is at most the total number of blocks in the two partitions. Since each node in the tree has at most two children, we can then conclude that its block count must appear once in its own contribution and at most twice in those of its children, when counting all the possible parent-child buyer block intersections and thus when bounding ∣ F ∣ |\mathcal{F}| ∣ F ∣ . Therefore, we need to estimate a number of functions which is at most three times the bound from Equation (22) :
∣ F ∣ ≤ min ( 9 K + 18 log K ε , 3 K 2 ) , (23) |\mathcal{F}| \le\min\left(9K+\frac{18\log K}{\varepsilon},3K^2\right), \tag*{(23)} ∣ F ∣ ≤ min ( 9 K + ε 18 log K , 3 K 2 ) , (23)
where ∣ F ∣ ≤ ( 2 K + 1 ) ( K + 1 ) ≤ 3 K 2 |\mathcal{F}| \le(2K+1)(K+1) \le3K^2 ∣ F ∣ ≤ ( 2 K + 1 ) ( K + 1 ) ≤ 3 K 2 , for K ≥ 4 K \ge4 K ≥ 4 , follows trivially by counting the number of edges in the tree and the number of possible buyer coordinates.
Step 5: Bound total variation and estimate We approach the conclusion of the proof of the lemma by bounding the total variation of each function f ∈ F f \in\mathcal{F} f ∈ F . Intuitively, each function considered is monotone, meaning that we can provide a uniform estimation of the function values by querying and estimating only a number of points which is logarithmic in the size of the domain. We can then use the fact that the function is non-increasing or non-decreasing to “fill the gaps” between points. Lower variation then further decreases the number of points in the domain of the functions that need to be queried to obtain uniform control. Formally, we have the following claim, whose proof is deferred to Appendix § 10 .§ 4 .
Claim 4.10. Let G \mathcal{G} G be a set of monotone functions mapping from some subset of finite domain Y \mathcal{Y} Y to [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] . For every g ∈ G g \in\mathcal{G} g ∈ G and each point y ∈ Y y \in\mathcal{Y} y ∈ Y in the domain of g g g , suppose we have a sequence of mutually independent random variables
Z g , y ( 1 ) , Z g , y ( 2 ) , … , Z_{g,y}^{(1)}, Z_{g,y}^{(2)}, \ldots, Z g , y ( 1 ) , Z g , y ( 2 ) , … , with expectation g ( y ) g(y) g ( y ) and taking values in a common interval of length at most 4. Each query at g ( y ) g(y) g ( y ) reveals the next unused variable in its sequence. Suppose that ∑ g ∈ G TV ( g ) ≤ W \sum_{g \in\mathcal{G}} \operatorname{TV}(g) \le W ∑ g ∈ G TV ( g ) ≤ W , where W W W is a known constant and TV ( g ) \operatorname{TV}(g) TV ( g ) stands for the total variation of g g g .
Then, for α , δ ′ ∈ ( 0 , 1 ) \alpha, \delta' \in(0,1) α , δ ′ ∈ ( 0 , 1 ) , there exists a procedure which estimates every function uniformly to error α \alpha α , simultaneously with probability at least 1 − δ ′ 1-\delta' 1 − δ ′ , using at most
C log ( 2 ∣ G ∣ ∣ Y ∣ / δ ′ ) α 2 ( ∣ G ∣ + W α log ( 2 ∣ Y ∣ ) ) C\frac{\log(2|\mathcal{G}||\mathcal{Y}|/\delta')}{\alpha^2}\left(|\mathcal{G}|+\frac{W}{\alpha}\log(2|\mathcal{Y}|)\right) C α 2 log ( 2∣ G ∣∣ Y ∣/ δ ′ ) ( ∣ G ∣ + α W log ( 2∣ Y ∣ ) ) queries, where C C C is an absolute constant.
In the above claim, we can plug in Equation (23) as the cardinality bound, and use K + 1 K+1 K + 1 as a trivial upper bound on the size of the domain of each function. It remains to bound ∑ f ∈ F TV ( f ) \sum_{f\in\mathcal{F}}\operatorname{TV}(f) ∑ f ∈ F TV ( f ) . To do this, importantly, we now use the actual underlying probability function of trade P P P , consistently with Equation (21) . We use P 0 P_0 P 0 to define each function in F \mathcal{F} F and guarantee approximate optimality with respect to P 0 P_0 P 0 , but every actual evaluated term refers to P P P , which is our estimation target.
A function f ∈ F f \in\mathcal{F} f ∈ F that comes from the combination of a root node I I I and one of its buyer blocks D D D is just f ( q ) = P ( p I , D , q ) f(q)=P(p_{I,D},q) f ( q ) = P ( p I , D , q ) , for q q q in some subset of D D D , with p I , D p_{I,D} p I , D the block representative seller price. It follows that:
TV ( f ) ≤ P P ( [ 0 , p I , D ] × [ min D , max D ] ) ≤ P P ( [ 0 , 1 ] × [ min D , max D ] ) , \operatorname{TV}(f)\le P_P([0,p_{I,D}]\times[\min D,\max D])\le P_P([0,1]\times[\min D,\max D]), TV ( f ) ≤ P P ([ 0 , p I , D ] × [ min D , max D ]) ≤ P P ([ 0 , 1 ] × [ min D , max D ]) ,
by definition of the trade probability function, see e.g. Equation (10) . Moreover, the blocks of buyer coordinates are disjoint, so that the sum of the total variations over the functions associated to the root node must be bounded by 1.
For a function f ∈ F f\in\mathcal{F} f ∈ F that resulted from a tree edge between parent node I j − 1 I_{j-1} I j − 1 and child node I j I_j I j , we instead have f ( q ) = P ( p I j , D j , q ) − P ( p I j − 1 , D j − 1 , q ) f(q)=P(p_{I_j,D_j},q)-P(p_{I_{j-1},D_{j-1}},q) f ( q ) = P ( p I j , D j , q ) − P ( p I j − 1 , D j − 1 , q ) , with D j D_j D j and D j − 1 D_{j-1} D j − 1 as the corresponding buyer blocks and q q q in some subset of their intersection. It follows that:
TV ( f ) ≤ P P ( [ min I j − 1 , max I j − 1 ] × [ min ( D j ∩ D j − 1 ) , max ( D j ∩ D j − 1 ) ] ) . \operatorname{TV}(f)\le P_P([\min I_{j-1},\max I_{j-1}]\times[\min(D_j\cap D_{j-1}),\max(D_j\cap D_{j-1})]). TV ( f ) ≤ P P ([ min I j − 1 , max I j − 1 ] × [ min ( D j ∩ D j − 1 ) , max ( D j ∩ D j − 1 )]) .
Moreover, the collection of non-empty intersections D j ∩ D j − 1 D_j\cap D_{j-1} D j ∩ D j − 1 associated with the edge { I j − 1 , I j } \{I_{j-1},I_j\} { I j − 1 , I j } is pairwise disjoint, and each intersection is a contiguous subset of G K , b G_{K,b} G K , b , since the partitioning of the buyer coordinate always consists of contiguous blocks, by Algorithm 3. Therefore, the sum of the total variations of the functions associated to a specific edge { I j − 1 , I j } \{I_{j-1},I_j\} { I j − 1 , I j } must be bounded by P P ( [ min I j − 1 , max I j − 1 ] × [ 0 , 1 ] ) P_P([\min I_{j-1},\max I_{j-1}]\times[0,1]) P P ([ min I j − 1 , max I j − 1 ] × [ 0 , 1 ]) . To conclude, each parent has two children and the seller nodes are disjoint at each depth, so that each parent covers a different subset of G K , s G_{K,s} G K , s ; since the maximum depth is bounded by 1 + log 2 ( K + 1 ) 1+\log_2(K+1) 1 + log 2 ( K + 1 ) , we all in all get, for K ≥ 10 K\ge10 K ≥ 10 and factoring in the bound for the root functions:
∑ f ∈ F TV ( f ) ≤ 1 + 4 log K ≤ 8 log K (24) \sum_{f\in\mathcal{F}}\operatorname{TV}(f)\le1+4\log K\le8\log K \tag*{(24)} f ∈ F ∑ TV ( f ) ≤ 1 + 4 log K ≤ 8 log K (24)
We can then use the above and apply Claim 4.10, with α = ε / 4.2 log K \alpha=\varepsilon/4.2\log K α = ε /4.2 log K and δ ′ = 1 / 10 \delta'=1/10 δ ′ = 1/10 , so as to target error ε / 4 \varepsilon/4 ε /4 and compensate the accumulation of the estimation error over the telescoping decomposition in Equation (21) , when targeting P ( p , q ) P(p,q) P ( p , q ) , for some ( p , q ) (p,q) ( p , q ) satisfying Δ p ( a ∗ , ( p , q ) ) ≤ ε \Delta_p(a^*,(p,q))\le\varepsilon Δ p ( a ∗ , ( p , q )) ≤ ε .
Step 6: Wrapping up At this point, with probability at least 9 / 10 9/10 9/10 , for each ε \varepsilon ε -optimal target ( p , q ) (p,q) ( p , q ) , we obtain an estimator P ^ ( p , q ) \widehat{P}(p,q) P ( p , q ) satisfying ∣ P ^ ( p , q ) − P ( p , q ) ∣ ≤ ε / 4 |\widehat{P}(p,q)-P(p,q)|\leq\varepsilon/4 ∣ P ( p , q ) − P ( p , q ) ∣ ≤ ε /4 . Now, for any such target ( p , q ) (p,q) ( p , q ) , let μ ( p , q ) = E V ~ ∼ P [ Profit ( V ~ , ( p , q ) ) ] \mu(p,q)=\mathbb{E}_{\widetilde{V}\sim P}[\operatorname{Profit}(\widetilde{V},(p,q))] μ ( p , q ) = E V ∼ P [ Profit ( V , ( p , q ))] , with associated estimator μ ^ ( p , q ) = ( q − p ) P ^ ( p , q ) \widehat{\mu}(p,q)=(q-p)\widehat{P}(p,q) μ ( p , q ) = ( q − p ) P ( p , q ) . We immediately get:
∣ μ ( p , q ) − μ ^ ( p , q ) ∣ = ∣ ( q − p ) ( P ( p , q ) − P ^ ( p , q ) ) ∣ ≤ ∣ P ( p , q ) − P ^ ( p , q ) ∣ ≤ ε 4 |\mu(p,q)-\widehat{\mu}(p,q)|=\left|(q-p)\left(P(p,q)-\widehat{P}(p,q)\right)\right|\leq\left|P(p,q)-\widehat{P}(p,q)\right|\leq\frac{\varepsilon}{4} ∣ μ ( p , q ) − μ ( p , q ) ∣ = ( q − p ) ( P ( p , q ) − P ( p , q ) ) ≤ P ( p , q ) − P ( p , q ) ≤ 4 ε
The above finally proves the claimed statement, since by the construction of F \mathcal{F} F and of the function domains in Steps 3 and 4, every queried action is 2 ε 2\varepsilon 2 ε -optimal under P 0 P_0 P 0 .
Remark 4.11. For any δ ′ ∈ ( 0 , 1 ) \delta^{\prime}\in(0,1) δ ′ ∈ ( 0 , 1 ) , the same construction gives error ε / 64 \varepsilon/64 ε /64 with probability at least 1 − δ ′ 1-\delta^{\prime} 1 − δ ′ without changing the target or query regions. Keep the tree and blocks at precision ε \varepsilon ε , and apply Claim 4.10 with α = ε / [ 64 ( 2 + log 2 ( K + 1 ) ) ] \alpha=\varepsilon/[64(2+\log_2(K+1))] α = ε / [ 64 ( 2 + log 2 ( K + 1 ))] . Each telescoping sum has at most 2 + log 2 ( K + 1 ) 2+\log_2(K+1) 2 + log 2 ( K + 1 ) terms, so its error is at most ε / 64 \varepsilon/64 ε /64 . The resulting query bound is
C log 2 ( K ) log ( K / δ ′ ) ( K ε 2 + log 3 ( K ) ε 3 ) C\log^2(K)\log(K/\delta^{\prime})\left(\frac{K}{\varepsilon^2}+\frac{\log^3(K)}{\varepsilon^3}\right) C log 2 ( K ) log ( K / δ ′ ) ( ε 2 K + ε 3 log 3 ( K ) )
Remark 4.12. The routine of Lemma 4.8 applies invariably, and with the same guarantees, if the feedback received is not the random trade bit, but a generic random variable with the same average and taking values in a fixed range of bounded length. More specifically, for the above estimation guarantee, we only needed Claim 4.10 for random variables with range of length 2 2 2 , since we were estimating either the average of a random bit, or the average difference of two random bits.
For the proof of Lemma 4.5 which follows, we need that the success probability of the queried bits are bounded away from 0 0 0 , so we add some bias to each and rescale the feedback to preserve the original average, resulting in a random variable which takes values in { − 1 / 2 , 3 / 2 } \{-1/2,3/2\} { − 1/2 , 3/2 } . This does not impact in any way the guarantees of Lemma 4.8 or the procedure of its proof, since it just suffices to apply Claim 4.10 with random variables over a common range of length at most 4 4 4 .
Proof of Lemma 4.5. In this proof, we need to consider profit gaps Δ ρ ( a ⋆ , a ) \Delta_{\rho}(a^{\star},a) Δ ρ ( a ⋆ , a ) at different scales, so we run separate instances of Lemma 4.8 , at precisions
ε j = min { 2 j x , 1 } , j = 0 , … , ⌈ log 2 ( 1 / x ) ⌉ \varepsilon_j=\min\{2^j x,1\},\qquad j=0,\ldots,\left\lceil\log_2(1/x)\right\rceil ε j = min { 2 j x , 1 } , j = 0 , … , ⌈ log 2 ( 1/ x ) ⌉
We modify the observations so as to prevent arbitrarily small success probabilities, which would be problematic when bounding from above the KL divergences of the associated Bernoulli variables. At each queried action a a a , we add a small bias to the success probability by independently flipping the trade bit with probability 1 / 4 1/4 1/4 , obtaining a bit Z Z Z with
P P ( Z = 1 ) = 1 4 ( 1 − P ( a ) ) + ( 1 − 1 4 ) P ( a ) = 1 4 + 1 2 P ( a ) . \mathbb{P}_P(Z=1)=\frac{1}{4}(1-P(a))+\left(1-\frac{1}{4}\right)P(a)=\frac{1}{4}+\frac{1}{2}P(a). P P ( Z = 1 ) = 4 1 ( 1 − P ( a )) + ( 1 − 4 1 ) P ( a ) = 4 1 + 2 1 P ( a ) .
Supply the estimation routine with 2 Z − 1 2 2Z-\frac{1}{2} 2 Z − 2 1 . This observation has mean P ( a ) P(a) P ( a ) and takes values in { − 1 / 2 , 3 / 2 } \{-1/2,3/2\} { − 1/2 , 3/2 } ; notice that Lemma 4.8 immediately applies to these tweaked observations, by Remark 4.12.
Now, let n j ( a ) n_j(a) n j ( a ) be the expected number of queries at a a a in experiment j j j under P 0 P_0 P 0 . Then set:
n x ( a ) = 2 ∑ j n j ( a ) n_x(a)=2\sum_j n_j(a) n x ( a ) = 2 j ∑ n j ( a )
which depends only on ρ \rho ρ and x x x . In the following, we bound the cost of each experiment by multiplying its sample bound by the largest cost of one query. Specifically, by the guarantee on actions played of Lemma 4.8 and the definition of ε j \varepsilon_j ε j , every query in experiment j j j satisfies:
Δ ρ ( a ⋆ , a ) + x ≤ 2 ε j + x ≤ 3 ε j . \Delta_{\rho}(a^{\star},a)+x\leq2\varepsilon_j+x\leq3\varepsilon_j. Δ ρ ( a ⋆ , a ) + x ≤ 2 ε j + x ≤ 3 ε j .
Hence
∑ a ∈ A n x ( a ) ( Δ ρ ( a ⋆ , a ) + x ) ≤ 6 ∑ j ε j ∑ a ∈ A n j ( a ) \sum_{a\in\mathcal{A}}n_x(a)\left(\Delta_{\rho}(a^{\star},a)+x\right)\leq6\sum_j\varepsilon_j\sum_{a\in\mathcal{A}}n_j(a) a ∈ A ∑ n x ( a ) ( Δ ρ ( a ⋆ , a ) + x ) ≤ 6 j ∑ ε j a ∈ A ∑ n j ( a )
≤ 6 C log 3 K ∑ j ( K ε j + log 3 K ε j 2 ) ≤ C ′ log 3 K ( K x + log 3 K x 2 ) , \begin{aligned}
\le6C\log^{3} K \sum_j \left(\frac{K}{\varepsilon_j}+\frac{\log^{3} K}{\varepsilon_j^{2}}\right) \\
&\le C'\log^{3} K\left(\frac{K}{x}+\frac{\log^{3} K}{x^{2}}\right),
\end{aligned} ≤ 6 C log 3 K j ∑ ( ε j K + ε j 2 log 3 K ) ≤ C ′ log 3 K ( x K + x 2 log 3 K ) ,
for some other constant C ′ C' C ′ , where the last inequality sums two geometric series. This proves the cost bound.
Next, we show that, for every improving distribution P P P , there is a precision ε j \varepsilon_j ε j at which the estimation routine distinguishes P P P from P 0 P_0 P 0 through its returned profit estimates. Fix any P P P satisfying the improvement condition.
Choose an action a ′ ∈ A a' \in\mathcal{A} a ′ ∈ A maximizing E P [ Profit ( V ~ , a ) ] \mathbb{E}_P[\operatorname{Profit}(\widetilde{V},a)] E P [ Profit ( V , a )] and choose j j j as the first scale with ε j ≥ Δ ρ ( a ∗ , a ′ ) \varepsilon_j \ge\Delta_\rho(a^*,a') ε j ≥ Δ ρ ( a ∗ , a ′ ) . We get
E P [ Profit ( V ~ , a ′ ) ] − E ρ [ Profit ( V , a ′ ) ] ≥ x + Δ ρ ( a ∗ , a ′ ) > ε j 2 . (25) \mathbb{E}_P[\operatorname{Profit}(\widetilde{V},a')] - \mathbb{E}_\rho[\operatorname{Profit}(V,a')] \ge x+\Delta_\rho(a^*,a') > \frac{\varepsilon_j}{2}. \tag*{(25)} E P [ Profit ( V , a ′ )] − E ρ [ Profit ( V , a ′ )] ≥ x + Δ ρ ( a ∗ , a ′ ) > 2 ε j . (25)
The first inequality follows from the improvement assumption and the second inequality follows by minimality of j j j , when j ≠ 0 j \ne0 j = 0 . If j = 0 j=0 j = 0 , then Δ ρ ( a ∗ , a ′ ) ≤ x \Delta_\rho(a^*,a') \le x Δ ρ ( a ∗ , a ′ ) ≤ x and ε j = x \varepsilon_j=x ε j = x , and the same inequality holds. Thus the difference in profit exceeds twice the estimation error ε j / 4 \varepsilon_j/4 ε j /4 . Write P P 0 ( ⋅ ) \mathbb{P}_{P_0}(\cdot) P P 0 ( ⋅ ) and P P ( ⋅ ) \mathbb{P}_{P}(\cdot) P P ( ⋅ ) for probabilities when the routine’s trade observations are generated under P 0 P_0 P 0 and P P P , respectively, including the independent bit-flipping coins. In both cases, the routine uses the same reference P 0 P_0 P 0 and precision ε j \varepsilon_j ε j . Let E E E be the event that the returned estimate at a ′ a' a ′ is within ε j / 4 \varepsilon_j/4 ε j /4 of E ρ [ Profit ( V , a ′ ) ] \mathbb{E}_\rho[\operatorname{Profit}(V,a')] E ρ [ Profit ( V , a ′ )] . Then Equation (25) and the estimation guarantee of Lemma 4.8 give
P P 0 ( E ) ≥ 0.9 , P P ( E ) ≤ 0.1. \mathbb{P}_{P_0}(E) \ge0.9,\qquad\mathbb{P}_{P}(E) \le0.1. P P 0 ( E ) ≥ 0.9 , P P ( E ) ≤ 0.1.
Finally, we analyze the KL divergence between the distributions of the estimation procedure’s transcript under P 0 P_0 P 0 and P P P to prove the weighted separation bound. Let H H H be the sequence of processed bits received by the estimation routine at precision ε j \varepsilon_j ε j . Write L P 0 ( H ) \mathcal{L}_{P_0}(H) L P 0 ( H ) and L P ( H ) \mathcal{L}_{P}(H) L P ( H ) for its laws under P 0 P_0 P 0 and P P P , respectively, keeping the routine and its reference fixed. Since the returned estimates are determined by H H H , so is the event E E E . Pinsker’s inequality therefore gives
KL ( L P 0 ( H ) ∥ L P ( H ) ) ≥ 2 ( P P 0 ( E ) − P P ( E ) ) 2 ≥ 2 ( 0.8 ) 2 > 1. (26) \operatorname{KL}\left(\mathcal{L}_{P_0}(H)\mathbin\Vert\mathcal{L}_{P}(H)\right) \ge2\left(\mathbb{P}_{P_0}(E)-\mathbb{P}_{P}(E)\right)^2 \ge2(0.8)^2>1. \tag*{(26)} KL ( L P 0 ( H ) ∥ L P ( H ) ) ≥ 2 ( P P 0 ( E ) − P P ( E ) ) 2 ≥ 2 ( 0.8 ) 2 > 1. (26)
For the upper bound, we express this divergence as the sum of the contributions from individual observations. Write Bern ( u ) \operatorname{Bern}(u) Bern ( u ) for the Bernoulli law with mean u u u . Conditional on any processed history, the query and stopping rules are identical under both laws, so only the next observation contributes to KL divergence. The chain rule consequently gives, with τ \tau τ being the random number of queries effectively performed by the procedure, N N N being the deterministic cap on them given by Lemma 4.8 and a t a_t a t the action played at round t t t :
KL ( L P 0 ( H ) ∥ L P ( H ) ) = ∑ t = 1 N ∑ a ∈ A P P 0 ( t ≤ τ , a t = a ) KL ( Bern ( 1 4 + 1 2 P 0 ( a ) ) ∥ Bern ( 1 4 + 1 2 P ( a ) ) ) = ∑ a ∈ A n j ( a ) KL ( Bern ( 1 4 + 1 2 P 0 ( a ) ) ∥ Bern ( 1 4 + 1 2 P ( a ) ) ) ≤ ∑ a ∈ A n j ( a ) 1 4 ⋅ ( P ( a ) − P 0 ( a ) ) 2 ( 1 4 + 1 2 ⋅ P ( a ) ) ⋅ ( 3 4 − 1 2 ⋅ P ( a ) ) ≤ 4 3 ∑ a ∈ A n j ( a ) ( P ( a ) − P 0 ( a ) ) 2 . (27) \begin{aligned}
\operatorname{KL}\left(\mathcal{L}_{P_0}(H)\mathbin\Vert\mathcal{L}_{P}(H)\right)
&=\sum_{t=1}^{N}\sum_{a\in\mathcal{A}}\mathbb{P}_{P_0}(t\le\tau,a_t=a)\operatorname{KL}\left(\operatorname{Bern}\left(\frac{1}{4}+\frac{1}{2}P_0(a)\right)\mathbin\Vert\operatorname{Bern}\left(\frac{1}{4}+\frac{1}{2}P(a)\right)\right)\\
&=\sum_{a\in\mathcal{A}}n_j(a)\operatorname{KL}\left(\operatorname{Bern}\left(\frac{1}{4}+\frac{1}{2}P_0(a)\right)\mathbin\Vert\operatorname{Bern}\left(\frac{1}{4}+\frac{1}{2}P(a)\right)\right)\\
&\le\sum_{a\in\mathcal{A}}n_j(a)\frac{\frac{1}{4}\cdot(P(a)-P_0(a))^2}{(\frac{1}{4}+\frac{1}{2}\cdot P(a))\cdot(\frac{3}{4}-\frac{1}{2}\cdot P(a))}\\
&\le\frac{4}{3}\sum_{a\in\mathcal{A}}n_j(a)(P(a)-P_0(a))^2. \tag*{(27)}
\end{aligned} KL ( L P 0 ( H ) ∥ L P ( H ) ) = t = 1 ∑ N a ∈ A ∑ P P 0 ( t ≤ τ , a t = a ) KL ( Bern ( 4 1 + 2 1 P 0 ( a ) ) ∥ Bern ( 4 1 + 2 1 P ( a ) ) ) = a ∈ A ∑ n j ( a ) KL ( Bern ( 4 1 + 2 1 P 0 ( a ) ) ∥ Bern ( 4 1 + 2 1 P ( a ) ) ) ≤ a ∈ A ∑ n j ( a ) ( 4 1 + 2 1 ⋅ P ( a )) ⋅ ( 4 3 − 2 1 ⋅ P ( a )) 4 1 ⋅ ( P ( a ) − P 0 ( a ) ) 2 ≤ 3 4 a ∈ A ∑ n j ( a ) ( P ( a ) − P 0 ( a ) ) 2 . (27)
where the second to last inequality uses the chi-squared bound, see Proposition A.1 .
Finally, first using n x ( a ) ≥ 2 n j ( a ) n_x(a)\ge2n_j(a) n x ( a ) ≥ 2 n j ( a ) , and then combining Equation (26) and Equation (27) , we obtain
∑ a ∈ A n x ( a ) ( P ( a ) − P 0 ( a ) ) 2 ≥ 2 ∑ a ∈ A n j ( a ) ( P ( a ) − P 0 ( a ) ) 2 > 3 2 ≥ 1. \sum_{a\in\mathcal{A}}n_x(a)(P(a)-P_0(a))^2\ge2\sum_{a\in\mathcal{A}}n_j(a)(P(a)-P_0(a))^2>\frac{3}{2}\ge1. a ∈ A ∑ n x ( a ) ( P ( a ) − P 0 ( a ) ) 2 ≥ 2 a ∈ A ∑ n j ( a ) ( P ( a ) − P 0 ( a ) ) 2 > 2 3 ≥ 1.
O-tilde(sqrt(T)) Regret for Monotone-Hazard-Rate Distributions
In this section, we show that the additional structure supplied by the monotone-hazard-rate assumption improves the regret from O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) to O ~ ( T ) \widetilde{O}(\sqrt{T}) O ( T ) . Notably, the algorithm still requires only one-bit feedback.
Theorem 5.1 (Regret for MHR distributions). There exists a learning algorithm A A A with one-bit feedback such that, for every T T T and every pair D D D of independent MHR buyer and seller distributions on [ 0 , 1 ] [0,1] [ 0 , 1 ] with strictly positive densities,
R T ( A , D ) ≤ O ( T ( log ( T ) ) 5 / 2 ) . R_T(A,D) \le O\left(\sqrt{T}(\log(T))^{5/2}\right). R T ( A , D ) ≤ O ( T ( log ( T ) ) 5/2 ) .
To prove the above, the first fundamental fact is the convexity of the surrogate loss L ( p , q ) = − log μ ( p , q ) L(p,q)=-\log\mu(p,q) L ( p , q ) = − log μ ( p , q ) , which is established in the following proposition.
Proposition 3.2. Under the MHR assumption, the loss function L ( p , q ) = − log μ ( p , q ) L(p,q)=-\log\mu(p,q) L ( p , q ) = − log μ ( p , q ) is continuous and convex in the interior of T \mathcal{T} T .
Proof. Notice that we can write explicitly the function L ( p , q ) = − log ( q − p ) − log F s ( p ) − log ( 1 − F b ( q ) ) L(p,q)=-\log(q-p)-\log F_s(p)-\log(1-F_b(q)) L ( p , q ) = − log ( q − p ) − log F s ( p ) − log ( 1 − F b ( q )) , which is well defined and continuous in the interior of T \mathcal{T} T (indeed, the probability density functions are strictly positive by the MHR assumption). The first term is jointly convex on p < q p<q p < q . The remaining two terms depend on separate coordinates and have nondecreasing derivatives − f s ( p ) / F s ( p ) -f_s(p)/F_s(p) − f s ( p ) / F s ( p ) and f b ( q ) / ( 1 − F b ( q ) ) f_b(q)/(1-F_b(q)) f b ( q ) / ( 1 − F b ( q )) , respectively, by MHR. Thus all three terms are convex, and so is their sum. □
The convexity of L ( p , q ) L(p,q) L ( p , q ) suggests an algorithm based on stochastic convex optimization, which we unravel in the next few subsections. Specifically, § 5.1 presents the underlying stochastic convex bandit result, and § 5.2 describes our approach and algorithm. Next, § 5.3 analyzes the construction of a safe convex region, and § 5.4 explains how to estimate L L L from one-bit feedback. Finally, § 5.5 proves the regret bound in Theorem 5.1 .
Stochastic Convex Bandits
We introduce the central statement for the proof of Theorem 5.1 , which is a corollary of a known theorem on stochastic convex bandits by Lattimore and György [32 ] and gives us an algorithm we denote as ConOpt. We defer the proof of the following claim to Appendix § 10 .5.
Corollary 3.1 (Stochastic convex bandit). Let K K K be a known convex body of constant dimension, and let L : K → R L:K\to\mathbb{R} L : K → R be continuous and convex, with values in a known interval of constant length. Suppose that querying x τ ∈ K x_\tau\in K x τ ∈ K returns L ( x τ ) + ξ τ L(x_\tau)+\xi_\tau L ( x τ ) + ξ τ . Conditional on previous queries and observations, and on x τ x_\tau x τ , assume that ξ τ \xi_\tau ξ τ has mean zero and ψ 2 \psi_2 ψ 2 norm at most a known γ ≥ 0 \gamma\ge0 γ ≥ 0 . Then, for every n ≥ 1 n\ge1 n ≥ 1 , there is an n n n -query algorithm such that
E [ ∑ τ = 1 n ( L ( x τ ) − min x ∈ K L ( x ) ) ] ∈ O ( ( 1 + γ ) n log 2 ( n ) ) , \mathbb{E}\left[\sum_{\tau=1}^{n}\left(L(x_\tau)-\min_{x\in K}L(x)\right)\right]\in O\left((1+\gamma)\sqrt{n}\log^2(n)\right), E [ τ = 1 ∑ n ( L ( x τ ) − x ∈ K min L ( x ) ) ] ∈ O ( ( 1 + γ ) n log 2 ( n ) ) , where the expectation is taken over noise and randomness of the algorithm’s actions. The hidden constant depends only on the affine dimension and the length of the specified interval.
Convex Optimization Algorithm
Given the result of the previous section, we describe our online algorithm, namely how to use stochastic convex bandit routine ConOpt to minimize the convex loss L ( p , q ) = − log μ ( p , q ) L(p,q)=-\log\mu(p,q) L ( p , q ) = − log μ ( p , q ) using one-bit feedback. In order to reduce our problem to a setting where the ConOpt applies, several conditions need to be met. We require a convex region containing a nearly optimal price pair, on which − log μ -\log\mu − log μ takes a known range of constant size. We must learn this region with high probability and O ~ ( T ) \widetilde{O}(\sqrt{T}) O ( T ) regret. We also need unbiased loss observations of − log μ -\log\mu − log μ with bounded noise, obtained from 1-bit feedback. In this subsection, we give a detailed description of our algorithm.
Initialize MHR takes as input the horizon T T T and grid resolution K K K . It builds a sparse set of grid pairs by recursive bisection and samples them at successively halved scales δ ≥ 1 / K \delta\ge1/K δ ≥ 1/ K , using independence to estimate every grid profit from three empirical trade frequencies, as explained in the proof of Proposition 5.4. At the first scale with max e μ ^ ( e ) ≥ 3 δ / 2 \max_{e}\widehat{\mu}(e) \ge3\delta/2 max e μ ( e ) ≥ 3 δ /2 , the procedure returns the convex hull of the grid pairs with μ ^ ( e ) ≥ 3 δ / 4 \widehat{\mu}(e) \ge3\delta/4 μ ( e ) ≥ 3 δ /4 , with these thresholds leaving slack for the estimation error.
Algorithm 4 INITIALIZE MHR
Input: Horizon T T T and grid resolution 2 ≤ K ≤ T 2 \le K \le T 2 ≤ K ≤ T . Output: A scale δ \delta δ and a convex region K \mathcal{K} K , or “no trigger”. G K ← { ( i / K , j / K ) : 0 ≤ i ≤ j ≤ K } G_K \leftarrow\{(i/K,j/K):0 \le i \le j \le K\} G K ← {( i / K , j / K ) : 0 ≤ i ≤ j ≤ K } . Recursively bisect { 0 , … , K } \{0,\ldots,K\} { 0 , … , K } down to singletons and for every resulting discrete interval I I I , let k I = ⌊ max I + min I 2 ⌋ k_I=\left\lfloor\frac{\max I+\min I}{2}\right\rfloor k I = ⌊ 2 m a x I + m i n I ⌋ . Let S I = { ( r , k I ) : r ∈ I , k I ≥ r } ∪ { ( k I , r ) : r ∈ I , k I < r } S_I=\{(r,k_I):r\in I,k_I\ge r\}\cup\{(k_I,r):r\in I,k_I<r\} S I = {( r , k I ) : r ∈ I , k I ≥ r } ∪ {( k I , r ) : r ∈ I , k I < r } and S = ⋃ I S I S=\bigcup_I S_I S = ⋃ I S I for δ = 1 , 1 / 2 , 1 / 4 , … \delta=1,1/2,1/4,\ldots δ = 1 , 1/2 , 1/4 , … with δ ≥ 1 / K \delta\ge1/K δ ≥ 1/ K do N ← ⌈ 2 16 log ( 2 T ) / δ ⌉ N\leftarrow\left\lceil2^{16\log(2T)/\delta}\right\rceil N ← ⌈ 2 16 l o g ( 2 T ) / δ ⌉ . for ( i , j ) ∈ S (i,j)\in S ( i , j ) ∈ S do Post ( i / K , j / K ) (i/K,j/K) ( i / K , j / K ) for N N N fresh rounds; let P ‾ i j \overline{P}_{ij} P ij be the fraction with trade. Set μ ^ ( e ) ← 0 \widehat{\mu}(e)\leftarrow0 μ ( e ) ← 0 for every e ∈ G K e\in G_K e ∈ G K . for 1 ≤ i < j ≤ K − 1 1\le i<j\le K-1 1 ≤ i < j ≤ K − 1 do Let k k k be the midpoint of the first recursive split separating i i i and j j j . μ ^ ( i / K , j / K ) ← j − i K P ‾ i k P ‾ k j max { P ‾ i k , P ‾ k j , P ‾ k k } \widehat{\mu}(i/K,j/K)\leftarrow\frac{j-i}{K}\frac{\overline{P}_{ik}\overline{P}_{kj}}{\max\{\overline{P}_{ik},\overline{P}_{kj},\overline{P}_{kk}\}} μ ( i / K , j / K ) ← K j − i m a x { P ik , P k j , P k k } P ik P k j , with 0 / 0 = 0 0/0=0 0/0 = 0 . if max e ∈ G K μ ^ ( e ) ≥ 3 δ / 2 \max_{e\in G_K}\widehat{\mu}(e)\ge3\delta/2 max e ∈ G K μ ( e ) ≥ 3 δ /2 then K ← conv { e ∈ G K : μ ^ ( e ) ≥ 3 δ / 4 } \mathcal{K}\leftarrow\operatorname{conv}\{e\in G_K:\widehat{\mu}(e)\ge3\delta/4\} K ← conv { e ∈ G K : μ ( e ) ≥ 3 δ /4 } . return ( δ , K ) (\delta,\mathcal{K}) ( δ , K ) . return “no trigger”.
Optimize receives the output ( δ , K ) (\delta,\mathcal{K}) ( δ , K ) of Initialize mhr, grid resolution K K K and T 0 T_0 T 0 , the number of remaining trading rounds after Initialize mhr. By MHR, we have the additional property that the loss function L ( p , q ) = − log ( μ ( p , q ) ) L(p,q)=-\log(\mu(p,q)) L ( p , q ) = − log ( μ ( p , q )) is bounded on K \mathcal{K} K . Moreover, it is possible to obtain an unbiased estimator of L ( p , q ) L(p,q) L ( p , q ) by querying ( p , q ) (p,q) ( p , q ) until the first time trade occurs, and this estimator can be shown to have ψ 2 \psi_2 ψ 2 noise at most 4 1 + log ( 2 K ) 4\sqrt{1+\log(2K)} 4 1 + log ( 2 K ) , where ψ 2 \psi_2 ψ 2 is the standard subgaussian norm, defined below for completeness.
Definition 5.2 (Subgaussian norm, see e.g. Definition 2.6.4 in Vershynin [42 ] ). A random variable X X X is called subgaussian if ∃ K > 0 \exists K>0 ∃ K > 0 such that E [ exp ( X 2 / K 2 ) ] ≤ 2 \mathbb{E}[\exp(X^2/K^2)]\le2 E [ exp ( X 2 / K 2 )] ≤ 2 . ∥ X ∥ ψ 2 \lVert X\rVert_{\psi_2} ∥ X ∥ ψ 2 is the infimum value of K K K satisfying the condition, which defines a norm on the space of subgaussian random variables.
Since the conditions explained above are satisfied, Optimize can run the convex stochastic bandit subroutine ConOpt with respect to the loss function L ( p , q ) L(p,q) L ( p , q ) on K \mathcal{K} K , noise bound 4 1 + log ( 2 K ) 4\sqrt{1+\log(2K)} 4 1 + log ( 2 K ) and loss range [ − log 6 δ , log 2 / δ ] [-\log6\delta,\log2/\delta] [ − log 6 δ , log 2/ δ ] , using our unbiased estimator for each query. The mechanism runs ConOpt for at most T 0 T_0 T 0 query rounds, and the actual query rounds performed correspond to a stochastic number of bilateral trade trading rounds, due to the randomness of the unbiased estimator. The procedure does anyway halt when the number of trading rounds reaches T 0 T_0 T 0 , in order to respect the time horizon.
Algorithm 5 Optimize
Input: Initialize mhr output ( δ , K ) (\delta,\mathcal{K}) ( δ , K ) , grid resolution K K K , and T 0 T_0 T 0 remaining trading rounds.
Subroutine: ConOpt
Input: K K K , n n n , loss range, ψ 2 \psi_2 ψ 2 noise bound, and past query–observation pairs.
Output: Next query point in K \mathcal{K} K .
n ← T 0 n \leftarrow T_0 n ← T 0 ; b ← T 0 b \leftarrow T_0 b ← T 0 .
Initialize ConOpt on K \mathcal{K} K for n n n queries, with loss range [ − log 6 δ , log 2 / δ ] [-\log6\delta,\log2/\delta] [ − log 6 δ , log 2/ δ ] and ψ 2 \psi_2 ψ 2 noise bound 4 1 + log ( 2 K ) 4\sqrt{1+\log(2K)} 4 1 + log ( 2 K ) .
for τ = 1 , … , n \tau=1,\ldots,n τ = 1 , … , n do
if b = 0 b=0 b = 0 then stop
Receive the next query x τ = ( p τ , q τ ) x_\tau=(p_\tau,q_\tau) x τ = ( p τ , q τ ) from ConOpt.
Post x τ x_\tau x τ until the first trade or until b b b rounds have been used.
Let N τ N_\tau N τ be the number of rounds used; set b ← b − N τ b\leftarrow b-N_\tau b ← b − N τ .
if no trade occurred then
Post ( 0 , 0 ) (0,0) ( 0 , 0 ) in every remaining trading round; stop.
Send ∑ k = 1 N τ − 1 1 k − log ( q τ − p τ ) \sum_{k=1}^{N_\tau-1}\frac{1}{k}-\log(q_\tau-p_\tau) ∑ k = 1 N τ − 1 k 1 − log ( q τ − p τ ) to ConOpt.
Bilateral mhr combines Initialize mhr and Optimize within a horizon of T T T trading rounds. It sets
K = ⌊ c 0 T log ( 2 T ) ⌋ , K=\left\lfloor\frac{c_0\sqrt{T}}{\log(2T)}\right\rfloor, K = ⌊ log ( 2 T ) c 0 T ⌋ ,
where c 0 ∈ ( 0 , 1 / 2 ] c_0\in(0,1/2] c 0 ∈ ( 0 , 1/2 ] is a sufficiently small universal constant, with this choice balancing the approximation error of the grid against the cost of the initial phase. First, it runs Initialize mhr with inputs T T T and K K K . If no scale triggers, it posts ( 0 , 0 ) (0,0) ( 0 , 0 ) in every remaining round. When Initialize mhr returns ( δ , K ) (\delta,\mathcal{K}) ( δ , K ) , the mechanism passes this output, together with K K K and the number T 0 T_0 T 0 of remaining trading rounds, to Optimize .
Algorithm 6 Bilateral mhr
Input: Horizon T T T and access to one-bit feedback.
Output: Price pairs ( p t , q t ) (p_t,q_t) ( p t , q t ) posted in rounds t = 1 , … , T t=1,\ldots,T t = 1 , … , T .
Fix a sufficiently small universal constant c 0 ∈ ( 0 , 1 / 2 ] c_0\in(0,1/2] c 0 ∈ ( 0 , 1/2 ] .
K ← ⌊ c 0 T / log ( 2 T ) ⌋ K\leftarrow\left\lfloor c_0\sqrt{T}/\log(2T)\right\rfloor K ← ⌊ c 0 T / log ( 2 T ) ⌋ .
Run Initialize mhr ( T , K ) (T,K) ( T , K ) .
if the procedure returns “no trigger” then
Post ( 0 , 0 ) (0,0) ( 0 , 0 ) in every remaining round; stop.
Let ( δ , K ) (\delta,\mathcal{K}) ( δ , K ) be the returned pair and T 0 T_0 T 0 the number of remaining rounds.
Optimize ( δ , K , K , T 0 ) (\delta,\mathcal{K},K,T_0) ( δ , K , K , T 0 ) .
Learning a Safe Convex Region
In this subsection, we show that K \mathcal{K} K contains a near optimal price, that the expected profit of all prices in K \mathcal{K} K is at a constant factor of the current scale δ \delta δ , and give an upper bound on the regret incurred by Initialize mhr .
Lemma 5.3 (Initialize mhr guarantees). Let 2 ≤ K ≤ T 2\le K\le T 2 ≤ K ≤ T . Whenever Initialize mhr triggers, the region K \mathcal{K} K returned is nonempty, compact, convex, and contained in the interior of T \mathcal{T} T . With probability at least 1 − 1 / T 1-1/T 1 − 1/ T , the following hold:
(i) If Initialize mhr returns “no trigger”, then μ ⋆ < 6 / K \mu^\star<6/K μ ⋆ < 6/ K ; otherwise, its returned scale satisfies δ ≤ μ ⋆ < 6 δ \delta\le\mu^\star<6\delta δ ≤ μ ⋆ < 6 δ .
(ii) Every returned region K \mathcal{K} K satisfies
P ( p , q ) ≥ μ ( p , q ) ≥ δ / 2 for all ( p , q ) ∈ K . P(p,q)\ge\mu(p,q)\ge\delta/2\qquad\text{for all }(p,q)\in\mathcal{K}. P ( p , q ) ≥ μ ( p , q ) ≥ δ /2 for all ( p , q ) ∈ K . (iii) Every returned region K \mathcal{K} K contains all grid maximizers, implying, via Lemma A.6 :
μ K ⋆ = max ( p , q ) ∈ K μ ( p , q ) ≥ μ ⋆ − 2 / K . \mu^\star_{\mathcal{K}}=\max_{(p,q)\in\mathcal{K}}\mu(p,q)\ge\mu^\star-2/K. μ K ⋆ = ( p , q ) ∈ K max μ ( p , q ) ≥ μ ⋆ − 2/ K .
Moreover, for a universal constant c 0 c_0 c 0 , choosing
K = ⌊ c 0 T log ( 2 T ) ⌋ (28) K=\left\lfloor c_0\frac{\sqrt{T}}{\log(2T)}\right\rfloor \tag*{(28)} K = ⌊ c 0 log ( 2 T ) T ⌋ (28)
ensures that Initialize MHR uses at most T T T rounds on every history and incurs O ( T log ( T ) ) O(\sqrt{T}\log(T)) O ( T log ( T )) expected regret. If it returns “no trigger”, posting ( 0 , 0 ) (0,0) ( 0 , 0 ) thereafter gives the same regret bound over the entire horizon.
The following proposition deals with the concentration and reconstruction guarantees of estimators in Initialize MHR. We defer its proof to Appendix A.5.
Proposition 5.4 (Simultaneous concentration). Let 2 ≤ K ≤ T 2\le K\le T 2 ≤ K ≤ T . With probability at least 1 − 1 / T 1-1/T 1 − 1/ T , simultaneously at every scale δ \delta δ reached by Initialize MHR, the estimates computed at that scale satisfy
∣ μ ^ ( e ) − μ ( e ) ∣ ≤ μ ( e ) + δ 16 for every e ∈ G K . (29) \left|\widehat{\mu}(e)-\mu(e)\right|\le\frac{\mu(e)+\delta}{16}\qquad\text{for every }e\in G_K. \tag*{(29)} ∣ μ ( e ) − μ ( e ) ∣ ≤ 16 μ ( e ) + δ for every e ∈ G K . (29)
For the remainder of this section, let E \mathcal{E} E be the event that Equation (29) holds for every e ∈ G K e\in G_K e ∈ G K at every scale reached by Initialize MHR. Using the associated accuracy guarantee, we can now prove the main statement.
Proof of Lemma 5.3. Whenever a scale triggers, at least one grid pair has estimate at least 3 δ / 2 3\delta/2 3 δ /2 , so the set selected at 3 δ / 4 3\delta/4 3 δ /4 is nonempty. Boundary and diagonal estimates are zero, so every selected pair lies in { ( p , q ) : 0 < p < q < 1 } \{(p,q):0<p<q<1\} {( p , q ) : 0 < p < q < 1 } . The convex hull of this finite set is therefore nonempty, compact, convex, and contained in the same open triangle, on every history. For the remaining conclusions, assume E \mathcal{E} E holds and write μ K ⋆ = max e ∈ G K μ ( e ) \mu_K^\star=\max_{e\in G_K}\mu(e) μ K ⋆ = max e ∈ G K μ ( e ) .
Suppose Initialize MHR triggers at scale δ \delta δ . If μ K ⋆ < δ \mu_K^\star<\delta μ K ⋆ < δ , then E \mathcal{E} E gives
max e ∈ G K μ ^ ( e ) ≤ μ K ⋆ + μ K ⋆ + δ 16 < 9 δ 8 < 3 δ 2 , \max_{e\in G_K}\widehat{\mu}(e)\le\mu_K^\star+\frac{\mu_K^\star+\delta}{16}<\frac{9\delta}{8}<\frac{3\delta}{2}, e ∈ G K max μ ( e ) ≤ μ K ⋆ + 16 μ K ⋆ + δ < 8 9 δ < 2 3 δ ,
contradicting the trigger and thus implying μ K ⋆ ≥ δ \mu_K^\star\ge\delta μ K ⋆ ≥ δ . If δ < 1 \delta<1 δ < 1 , the preceding scale 2 δ 2\delta 2 δ did not trigger. If μ K ⋆ ≥ 4 δ \mu_K^\star\ge4\delta μ K ⋆ ≥ 4 δ , the estimate of any grid maximizer at that preceding scale needs to be at least
μ K ⋆ − μ K ⋆ + 2 δ 16 ≥ 29 δ 8 > 3 δ 2 , \mu_K^\star-\frac{\mu_K^\star+2\delta}{16}\ge\frac{29\delta}{8}>\frac{3\delta}{2}, μ K ⋆ − 16 μ K ⋆ + 2 δ ≥ 8 29 δ > 2 3 δ ,
which contradicts the fact that the algorithm did not trigger at scale 2 δ 2\delta 2 δ . For δ = 1 \delta=1 δ = 1 , the bound μ K ⋆ < 4 δ \mu_K^\star<4\delta μ K ⋆ < 4 δ follows from μ K ⋆ ≤ 1 \mu_K^\star\le1 μ K ⋆ ≤ 1 . Hence δ ≤ μ K ⋆ < 4 δ \delta\le\mu_K^\star<4\delta δ ≤ μ K ⋆ < 4 δ . By the grid approximation bound Lemma A.6 and δ ≥ 1 / K \delta\ge1/K δ ≥ 1/ K ,
δ ≤ μ K ⋆ ≤ μ ⋆ ≤ μ K ⋆ + 2 K < 6 δ , \delta\le\mu_K^\star\le\mu^\star\le\mu_K^\star+\frac{2}{K}<6\delta, δ ≤ μ K ⋆ ≤ μ ⋆ ≤ μ K ⋆ + K 2 < 6 δ ,
proving (i).
At the triggering scale, E \mathcal{E} E also gives
μ ( e ) ≥ δ ⟹ μ ^ ( e ) ≥ 15 μ ( e ) − δ 16 ≥ 7 δ 8 > 3 δ 4 , \mu(e)\ge\delta\quad\Longrightarrow\quad\widehat{\mu}(e)\ge\frac{15\mu(e)-\delta}{16}\ge\frac{7\delta}{8}>\frac{3\delta}{4}, μ ( e ) ≥ δ ⟹ μ ( e ) ≥ 16 15 μ ( e ) − δ ≥ 8 7 δ > 4 3 δ ,
μ ( e ) < δ / 2 ⟹ μ ^ ( e ) ≤ 17 μ ( e ) + δ 16 < 19 δ 32 < 3 δ 4 . \mu(e)<\delta/2\quad\Longrightarrow\quad\widehat{\mu}(e)\le\frac{17\mu(e)+\delta}{16}<\frac{19\delta}{32}<\frac{3\delta}{4}. μ ( e ) < δ /2 ⟹ μ ( e ) ≤ 16 17 μ ( e ) + δ < 32 19 δ < 4 3 δ .
Consequently,
{ e ∈ G K : μ ( e ) ≥ δ } ⊆ { e ∈ G K : μ ^ ( e ) ≥ 3 δ / 4 } ⊆ { e ∈ G K : μ ( e ) ≥ δ / 2 } ⊆ { ( p , q ) ∈ [ 0 , 1 ] 2 : 0 < p < q < 1 , μ ( p , q ) ≥ δ / 2 } . (30) \{e\in G_K:\mu(e)\ge\delta\}\subseteq\{e\in G_K:\widehat{\mu}(e)\ge3\delta/4\}
\subseteq\{e\in G_K:\mu(e)\ge\delta/2\}
\subseteq\{(p,q)\in[0,1]^2:0<p<q<1,\ \mu(p,q)\ge\delta/2\}. \tag*{(30)} { e ∈ G K : μ ( e ) ≥ δ } ⊆ { e ∈ G K : μ ( e ) ≥ 3 δ /4 } ⊆ { e ∈ G K : μ ( e ) ≥ δ /2 } ⊆ {( p , q ) ∈ [ 0 , 1 ] 2 : 0 < p < q < 1 , μ ( p , q ) ≥ δ /2 } . (30)
By log-concavity of μ \mu μ , the superlevel set { ( p , q ) : 0 < p < q < 1 , μ ( p , q ) ≥ δ / 2 } \{(p,q):0<p<q<1,\ \mu(p,q)\ge\delta/2\} {( p , q ) : 0 < p < q < 1 , μ ( p , q ) ≥ δ /2 } is convex. Taking the convex hull over the above chain of inclusions therefore proves that the convex hull K \mathcal{K} K of selected prices is contained in the superlevel set: together with P ( p , q ) ≥ μ ( p , q ) P(p,q)\ge\mu(p,q) P ( p , q ) ≥ μ ( p , q ) , this proves (ii). Since μ K ⋆ ≥ δ \mu_K^\star\ge\delta μ K ⋆ ≥ δ , the first inclusion in Equation (30) shows that K \mathcal{K} K contains every grid maximizer. Thus μ K ⋆ ≥ μ K ⋆ ≥ μ ⋆ − 2 / K \mu_K^\star\ge\mu_K^\star\ge\mu^\star-2/K μ K ⋆ ≥ μ K ⋆ ≥ μ ⋆ − 2/ K , via Lemma A.6, proving (iii).
Finally, if no scale triggers, let δ \delta δ be the last scale tested. Then 1 / K ≤ δ < 2 / K 1/K\le\delta<2/K 1/ K ≤ δ < 2/ K . If μ K ⋆ ≥ 2 δ \mu_K^\star\ge2\delta μ K ⋆ ≥ 2 δ , a grid maximizer would have estimate at least
μ K ⋆ − μ K ⋆ + δ 16 ≥ 29 δ 16 > 3 δ 2 , \mu_K^\star-\frac{\mu_K^\star+\delta}{16}\ge\frac{29\delta}{16}>\frac{3\delta}{2}, μ K ⋆ − 16 μ K ⋆ + δ ≥ 16 29 δ > 2 3 δ ,
forcing a trigger. Therefore, μ K ⋆ < 2 δ < 4 / K \mu_K^\star< 2\delta< 4/K μ K ⋆ < 2 δ < 4/ K , and Lemma A.6 gives μ ⋆ < 6 / K \mu^\star< 6/K μ ⋆ < 6/ K .
It remains to prove the round and regret bounds for the grid choice in Equation (28) . For the round bound, consider an arbitrary observation history and restrict c 0 ≤ 1 / 2 c_0 \le1/2 c 0 ≤ 1/2 , so that K ≤ T K \le T K ≤ T . Each interval in the recursive construction of S \mathcal{S} S contributes at most as many pairs as it has indices. At any fixed depth the intervals are disjoint, so they contribute at most K + 1 K+1 K + 1 pairs in total. There are at most ⌈ log 2 ( K + 1 ) ⌉ + 1 \lceil\log_2(K+1) \rceil+ 1 ⌈ log 2 ( K + 1 )⌉ + 1 depths. Thus
∣ S ∣ ≤ ( K + 1 ) ( ⌈ log 2 ( K + 1 ) ⌉ + 1 ) ≤ 6 K log ( 2 T ) . |\mathcal{S}| \le(K+1)\left(\lceil\log_2(K+1) \rceil+1\right) \le6K\log(2T). ∣ S ∣ ≤ ( K + 1 ) ( ⌈ log 2 ( K + 1 )⌉ + 1 ) ≤ 6 K log ( 2 T ) .
At each tested scale δ \delta δ , Initialize mhr posts every pair in S \mathcal{S} S for
⌈ 2 16 log ( 2 T ) δ ⌉ \left\lceil\frac{2^{16}\log(2T)}{\delta} \right\rceil ⌈ δ 2 16 log ( 2 T ) ⌉
fresh rounds. Constructing S \mathcal{S} S , reconstructing the grid estimates, and checking the trigger require no additional trading rounds. Letting δ \delta δ denote the last tested scale, notice that the reciprocals of the tested scales sum to 1 + 2 + ⋯ + 1 / δ = 2 / δ − 1 < 2 / δ 1+2+\cdots+1/\delta= 2/\delta-1 < 2/\delta 1 + 2 + ⋯ + 1/ δ = 2/ δ − 1 < 2/ δ . Therefore, writing C = 12 ( 2 16 + 1 ) C=12(2^{16}+1) C = 12 ( 2 16 + 1 ) , we get that the total number of rounds is bounded by at most:
C K log 2 ( 2 T ) δ ≤ C K 2 log 2 ( 2 T ) ≤ C c 0 2 T , (31) C\frac{K\log^2(2T)}{\delta} \le CK^2\log^2(2T) \le Cc_0^2T, \tag*{(31)} C δ K log 2 ( 2 T ) ≤ C K 2 log 2 ( 2 T ) ≤ C c 0 2 T , (31)
where we used δ ≥ 1 / K \delta\ge1/K δ ≥ 1/ K and the choice of K K K from Equation (28) . Choosing c 0 = 1 / C c_0=1/\sqrt{C} c 0 = 1/ C gives a bound of T T T on every history.
For the regret bound, assume E \mathcal{E} E holds again. The bounds already proved give μ ⋆ < 6 δ \mu^\star< 6\delta μ ⋆ < 6 δ if Initialize mhr triggers at δ \delta δ . Equation (31) then yields a cumulative regret bound of
6 C δ K log 2 ( 2 T ) δ ≤ 6 C K log 2 ( 2 T ) ≤ 6 C c 0 T log ( 2 T ) . 6C\delta\frac{K\log^2(2T)}{\delta} \le6CK\log^2(2T) \le6Cc_0\sqrt{T}\log(2T). 6 C δ δ K log 2 ( 2 T ) ≤ 6 C K log 2 ( 2 T ) ≤ 6 C c 0 T log ( 2 T ) .
Similarly, if no scale triggers and ( 0 , 0 ) (0,0) ( 0 , 0 ) is posted thereafter, the regret over the entire horizon is at most
∑ t = 1 T ( μ ⋆ − μ ( p t , q t ) ) ≤ T μ ⋆ < 6 T K ≤ 12 c 0 T log ( 2 T ) , \sum_{t=1}^{T}\left(\mu^\star-\mu(p_t,q_t)\right) \le T\mu^\star< \frac{6T}{K} \le\frac{12}{c_0}\sqrt{T}\log(2T), t = 1 ∑ T ( μ ⋆ − μ ( p t , q t ) ) ≤ T μ ⋆ < K 6 T ≤ c 0 12 T log ( 2 T ) ,
where the second to last inequality follows by μ ⋆ ≤ 6 / K \mu^\star\le6/K μ ⋆ ≤ 6/ K if Initialize mhr does not trigger. We conclude the proof by noticing that P ( E ) ≥ 1 − 1 / T \mathbb{P}(\mathcal{E}) \ge1-1/T P ( E ) ≥ 1 − 1/ T , by Proposition 5.4, which also implies that the expected regret associated to E c \mathcal{E}^c E c can be at most constant. □
For the remainder of the section, consistently with Algorithm 6, we use the choice of K K K in Equation (28) .
A One-Bit Log-Profit Oracle
The previous subsection proved that assuming E \mathcal{E} E , if Initialize mhr triggers at some scale δ \delta δ , it returns a convex set K \mathcal{K} K which approximately contains the optimum and on which expected profit is always within a constant factor of the optimum. We now need to show that Optimize efficiently finds max ( p , q ) ∈ K μ ( p , q ) \max_{(p,q)\in\mathcal{K}}\mu(p,q) max ( p , q ) ∈ K μ ( p , q ) with one-bit feedback.
The first ingredient is building an estimator of the convex loss L ( p , q ) L(p,q) L ( p , q ) from 1-bit observations. For m ≥ 1 m\ge1 m ≥ 1 , let H m = ∑ k = 1 m 1 / k H_m=\sum_{k=1}^{m}1/k H m = ∑ k = 1 m 1/ k , and set H 0 = 0 H_0=0 H 0 = 0 . Recall that for a query x = ( p , q ) ∈ K x=(p,q)\in\mathcal{K} x = ( p , q ) ∈ K , Optimize posts x x x until the first trade. Let N N N be the number of rounds used, then
Z = H N − 1 − log ( q − p ) (32) Z=H_{N-1}-\log(q-p) \tag*{(32)} Z = H N − 1 − log ( q − p ) (32)
is the input given to Conopt in Optimize. We now show that Z Z Z is indeed an unbiased estimator of the loss L ( p , q ) = − log ( μ ( p , q ) ) L(p,q)=-\log(\mu(p,q)) L ( p , q ) = − log ( μ ( p , q )) , with the needed tail control. While the unbiasedness of this estimator is a well-known fact (see Theorem 4.1 in DeGroot [17 ] for a more general result), we provide a proof in Appendix § 10 .5 for completeness and to establish the subgaussian norm bound required to apply Corollary 3.1 .
Claim 5.5 (Estimating a logarithm from a geometric waiting time). Let Q ∈ ( 0 , 1 ] Q \in(0,1] Q ∈ ( 0 , 1 ] and let N ∼ Geom ( Q ) N \sim\operatorname{Geom}(Q) N ∼ Geom ( Q ) . Then
E [ H N − 1 ] = − log Q , ∥ H N − 1 + log Q ∥ ψ 2 ≤ 4 1 − log Q . \mathbb{E}[H_{N-1}] = -\log Q,\qquad\lVert H_{N-1}+\log Q\rVert_{\psi_2} \le4\sqrt{1-\log Q}. E [ H N − 1 ] = − log Q , ∥ H N − 1 + log Q ∥ ψ 2 ≤ 4 1 − log Q .
We now have all the ingredients to verify that the region and loss observations used by Optimize satisfy the conditions of Corollary 3.1 . The next lemma gives a regret bound for the queries, assuming each query is allowed to run until trade.
Lemma 5.6 (Convex optimization from trade bits). Fix a complete transcript compatible with the clean event E \mathcal{E} E on which Initialize mhr triggers and returns ( δ , K ) (\delta, K) ( δ , K ) . For any integer n ≥ 1 n \ge1 n ≥ 1 , initialize ConOpt as in Optimize for n n n queries and answer each query using the estimator in Equation (32) , without a trading-round limit. Conditional on this transcript, the resulting query points x 1 , … , x n ∈ K x_1,\ldots,x_n \in K x 1 , … , x n ∈ K satisfy
E [ ∑ τ = 1 n ( L ( x τ ) − min x ∈ K L ( x ) ) ] = E [ ∑ τ = 1 n log μ K ∗ μ ( x τ ) ] = O ( n log ( T ) log 2 ( 2 n ) ) . (33) \mathbb{E}\left[\sum_{\tau=1}^{n}\left(L(x_\tau)-\min_{x\in K}L(x)\right)\right]
=
\mathbb{E}\left[\sum_{\tau=1}^{n}\log\frac{\mu_K^*}{\mu(x_\tau)}\right]
=
O\left(\sqrt{n\log(T)}\log^2(2n)\right). \tag*{(33)} E [ τ = 1 ∑ n ( L ( x τ ) − x ∈ K min L ( x ) ) ] = E [ τ = 1 ∑ n log μ ( x τ ) μ K ∗ ] = O ( n log ( T ) log 2 ( 2 n ) ) . (33) Proof. By Lemma 5.3, if Initialize mhr triggers, the set K K K is nonempty, compact, and convex. By Proposition 3.2, L L L is continuous and convex on K K K , since K K K is a subset of the interior of T \mathcal{T} T . Moreover, Lemma 5.3 with E \mathcal{E} E gives
μ ( p , q ) ∈ [ δ / 2 , 6 δ ] ( ( p , q ) ∈ K ) , \mu(p,q) \in[\delta/2,6\delta]\qquad((p,q)\in K), μ ( p , q ) ∈ [ δ /2 , 6 δ ] (( p , q ) ∈ K ) , which implies that L L L on K K K takes values in a fixed (given δ \delta δ ) interval of constant length:
L ( p , q ) ∈ [ − log 6 δ , log 2 / δ ] ( ( p , q ) ∈ K ) . (34) L(p,q)\in[-\log6\delta,\log2/\delta]\qquad((p,q)\in K). \tag*{(34)} L ( p , q ) ∈ [ − log 6 δ , log 2/ δ ] (( p , q ) ∈ K ) . (34) We next verify the observation and noise bounds. Fix a query ( p , q ) (p,q) ( p , q ) . The algorithm holds these prices fixed until a trade occurs. Since future valuation pairs are i.i.d. given the choice of ( p , q ) (p,q) ( p , q ) , the waiting time N N N is geometric with success probability which can be lower bounded by, using δ ≥ 1 / K \delta\ge1/K δ ≥ 1/ K :
P ( p , q ) = μ ( p , q ) q − p ≥ μ ( p , q ) ≥ δ 2 ≥ 1 2 K . (35) P(p,q)=\frac{\mu(p,q)}{q-p}\ge\mu(p,q)\ge\frac{\delta}{2}\ge\frac{1}{2K}. \tag*{(35)} P ( p , q ) = q − p μ ( p , q ) ≥ μ ( p , q ) ≥ 2 δ ≥ 2 K 1 . (35) By Claim 5.5, the estimator Z Z Z in Equation (32) is unbiased for L ( p , q ) L(p,q) L ( p , q ) , and we have:
∥ Z − L ( p , q ) ∥ ψ 2 ≤ 4 1 − log P ( p , q ) ≤ 4 1 + log ( 2 K ) , \lVert Z-L(p,q)\rVert_{\psi_2}\le4\sqrt{1-\log P(p,q)}\le4\sqrt{1+\log(2K)}, ∥ Z − L ( p , q ) ∥ ψ 2 ≤ 4 1 − log P ( p , q ) ≤ 4 1 + log ( 2 K ) , where the inequality follows from Equation (35) . Finally, K K K has affine dimension at most two. Applying Corollary 3.1 with the above guarantees yields:
E [ ∑ τ = 1 n ( L ( x τ ) − min x ∈ K L ( x ) ) ] = O ( n ( 1 + log ( 2 K ) ) log 2 ( 2 n ) ) . \mathbb{E}\left[\sum_{\tau=1}^{n}\left(L(x_\tau)-\min_{x\in K}L(x)\right)\right]
=
O\left(\sqrt{n(1+\log(2K))}\log^2(2n)\right). E [ τ = 1 ∑ n ( L ( x τ ) − x ∈ K min L ( x ) ) ] = O ( n ( 1 + log ( 2 K )) log 2 ( 2 n ) ) . Since K ≤ T K\le T K ≤ T and
L ( x τ ) − min x ∈ K L ( x ) = log μ K ∗ μ ( x τ ) , L(x_\tau)-\min_{x\in K}L(x)=\log\frac{\mu_K^*}{\mu(x_\tau)}, L ( x τ ) − x ∈ K min L ( x ) = log μ ( x τ ) μ K ∗ , this proves Equation (33) .
Proving the Regret Bound
Using the guarantees of Lemma 5.6 , we can now proceed to the proof of the main result of this section, the regret bound in Theorem 5.1 . As in Algorithms 5 and 6, let T 0 T_0 T 0 be the number of bilateral trade rounds remaining after the execution of Initialize mhr. The main idea is that we can pretend that ConOpt always runs until the maximum number of convex optimization query rounds T 0 T_0 T 0 , even though we may have to stop earlier due to the number of bilateral trade pricing rounds having reached T 0 T_0 T 0 . Indeed, the regret of the bandit routine restricted to T 0 T_0 T 0 bilateral trade rounds is for sure bounded by the regret of the unrestricted routine which stops after T 0 T_0 T 0 convex bandits queries.
Proof of Theorem 5.1 . Recall that the regret is given by
E [ ∑ t = 1 T ( μ ⋆ − μ ( p t , q t ) ) ] . (36) \mathbb{E}\left[\sum_{t=1}^{T}(\mu^\star-\mu(p_t,q_t))\right]. \tag*{(36)} E [ t = 1 ∑ T ( μ ⋆ − μ ( p t , q t )) ] . (36)
where every summand belongs to [ 0 , 1 ] [0,1] [ 0 , 1 ] . We bound this sum assuming the clean event E \mathcal{E} E from Initialize MHR. This is possible because the contribution of E c \mathcal{E}^{c} E c to the regret is at most T P ( E c ) T\mathbb{P}(\mathcal{E}^{c}) T P ( E c ) , and Lemma 5.3 gives that P ( E c ) ≤ 1 / T \mathbb{P}(\mathcal{E}^{c})\le1/T P ( E c ) ≤ 1/ T . Conditionally on E \mathcal{E} E , if Initialize MHR did not trigger, Lemma 5.3 gives total regret over the horizon of O ( T log T ) O(\sqrt{T}\log T) O ( T log T ) .
Now, fix a complete Initialize MHR transcript on which E \mathcal{E} E holds and the procedure triggers. This implies the remaining horizon T 0 T_0 T 0 , scale δ \delta δ , and region K K K are fixed. Recall that Lemma 5.3 bounds the regret of Initialize MHR by O ( T log T ) O(\sqrt{T}\log T) O ( T log T ) . It remains to bound the regret of Optimize over T 0 T_0 T 0 rounds. For the analysis, continue the same ConOpt execution until all n = T 0 n=T_0 n = T 0 convex optimization queries finish, and let N τ N_\tau N τ be the number of trading rounds used by query τ \tau τ . Since N τ ≥ 1 N_\tau\ge1 N τ ≥ 1 , the actual remaining T 0 T_0 T 0 bilateral trade rounds form a prefix of this execution. Splitting off the approximation error gives
∑ t = T − T 0 + 1 T ( μ ⋆ − μ ( p t , q t ) ) = T 0 ( μ ⋆ − μ K ⋆ ) + ∑ t = T − T 0 + 1 T ( μ K ⋆ − μ ( p t , q t ) ) ≤ 2 T 0 K + ∑ τ = 1 T 0 N τ ( μ K ⋆ − μ ( x τ ) ) . (37) \begin{aligned}
\sum_{t=T-T_0+1}^{T}(\mu^\star-\mu(p_t,q_t))=T_0(\mu^\star-\mu_K^\star)+\sum_{t=T-T_0+1}^{T}(\mu_K^\star-\mu(p_t,q_t)) \\
&\le\frac{2T_0}{K}+\sum_{\tau=1}^{T_0}N_\tau(\mu_K^\star-\mu(x_\tau)). \tag*{(37)}
\end{aligned} t = T − T 0 + 1 ∑ T ( μ ⋆ − μ ( p t , q t )) = T 0 ( μ ⋆ − μ K ⋆ ) + t = T − T 0 + 1 ∑ T ( μ K ⋆ − μ ( p t , q t )) ≤ K 2 T 0 + τ = 1 ∑ T 0 N τ ( μ K ⋆ − μ ( x τ )) . (37)
The inequality uses Lemma 5.3(iii) for the approximation error.
Now, let F τ − 1 \mathcal{F}_{\tau-1} F τ − 1 be the sigma algebra generated by the history before query τ \tau τ (waiting times for Equation (32) and randomization of ConOpt, including the choice of x τ x_\tau x τ ); also let N τ N_\tau N τ be the waiting time for the estimator associated to query τ \tau τ . Conditional on F τ − 1 \mathcal{F}_{\tau-1} F τ − 1 , N τ N_\tau N τ is a geometric random variable with success probability P ( x τ ) P(x_\tau) P ( x τ ) , and therefore, using (ii) from Lemma 5.3:
E [ N τ ∣ F τ − 1 ] = 1 P ( x τ ) ≤ 2 δ . \mathbb{E}[N_\tau\mid\mathcal{F}_{\tau-1}]=\frac{1}{P(x_\tau)}\le\frac{2}{\delta}. E [ N τ ∣ F τ − 1 ] = P ( x τ ) 1 ≤ δ 2 .
Taking the outer expectation and using that x τ x_\tau x τ is measurable according to F τ − 1 \mathcal{F}_{\tau-1} F τ − 1 (see e.g. Chapter 9 in Williams [43 ] ), the instantaneous bilateral trade regret suffered for query τ \tau τ is at most:
E [ E [ N τ ( μ K ⋆ − μ ( x τ ) ) ∣ F τ − 1 ] ] = E [ ( μ K ⋆ − μ ( x τ ) ) E [ N τ ∣ F τ − 1 ] ] ( ( μ K ⋆ − μ ( x τ ) ) bounded and E [ N τ ] < ∞ ) = E [ μ K ⋆ − μ ( x τ ) P ( x τ ) ] ≤ E [ μ K ⋆ μ ( x τ ) − 1 ] ( P ( x τ ) ≥ μ ( x τ ) ) \begin{aligned}
\mathbb{E}\left[\mathbb{E}[N_\tau(\mu_K^\star-\mu(x_\tau))\mid\mathcal{F}_{\tau-1}]\right]
=\mathbb{E}\left[(\mu_K^\star-\mu(x_\tau))\mathbb{E}[N_\tau\mid\mathcal{F}_{\tau-1}]\right]
\quad((\mu_K^\star-\mu(x_\tau))\text{ bounded and }\mathbb{E}[N_\tau]<\infty) \\
&=\mathbb{E}\left[\frac{\mu_K^\star-\mu(x_\tau)}{P(x_\tau)}\right] \\
&\le\mathbb{E}\left[\frac{\mu_K^\star}{\mu(x_\tau)}-1\right]
\quad(P(x_\tau)\ge\mu(x_\tau))
\end{aligned} E [ E [ N τ ( μ K ⋆ − μ ( x τ )) ∣ F τ − 1 ] ] = E [ ( μ K ⋆ − μ ( x τ )) E [ N τ ∣ F τ − 1 ] ] (( μ K ⋆ − μ ( x τ )) bounded and E [ N τ ] < ∞ ) = E [ P ( x τ ) μ K ⋆ − μ ( x τ ) ] ≤ E [ μ ( x τ ) μ K ⋆ − 1 ] ( P ( x τ ) ≥ μ ( x τ ))
≤ 5 E [ log μ K ⋆ μ ( x τ ) ] , \le5\mathbb{E}\left[\log\frac{\mu_K^\star}{\mu(x_\tau)}\right], ≤ 5 E [ log μ ( x τ ) μ K ⋆ ] ,
(Point (i)-(ii) of Lemma 5.3, z − 1 ≤ 5 log z z-1\le5\log z z − 1 ≤ 5 log z for z ∈ [ 1 , 12 ] z\in[1,12] z ∈ [ 1 , 12 ] )
Finally, we can combine the above with Equation (37) and Lemma 5.6 to get a regret bound of:
2 T K + ∑ τ = 1 T 0 E [ N τ ( μ K ⋆ − μ ( x τ ) ) ] = O ( T log T ) + O ( T 0 log ( 2 T ) log 2 ( 2 T 0 ) ) ≤ O ( T log 5 / 2 T ) , \frac{2T}{K}+\sum_{\tau=1}^{T_0}\mathbb{E}[N_\tau(\mu_K^\star-\mu(x_\tau))]
=O(\sqrt{T}\log T)+O\left(\sqrt{T_0\log(2T)}\log^2(2T_0)\right)
\le O(\sqrt{T}\log^{5/2}T), K 2 T + τ = 1 ∑ T 0 E [ N τ ( μ K ⋆ − μ ( x τ ))] = O ( T log T ) + O ( T 0 log ( 2 T ) log 2 ( 2 T 0 ) ) ≤ O ( T log 5/2 T ) ,
where we used the value of K K K from Equation (28) .
O ~ ( T 3 / 5 ) \widetilde{O}(T^{3/5}) O ( T 3/5 ) Regret for Regular Distributions
Theorem 6.1 (Regret for regular distributions). There exists a learning algorithm A \mathcal{A} A with one-bit feedback such that, for every T T T and every pair D \mathcal{D} D of independent regular buyer and seller distributions on [ 0 , 1 ] [0,1] [ 0 , 1 ] with strictly positive densities,
R T ( A , D ) ≤ O ( T 3 / 5 log 5 ( T ) ) . R_T(\mathcal{A},\mathcal{D})\le O\left(T^{3/5}\log^5(T)\right). R T ( A , D ) ≤ O ( T 3/5 log 5 ( T ) ) .
Throughout the proof, let η ∈ ( 0 , 1 ) \eta\in(0,1) η ∈ ( 0 , 1 ) denote the target additive profit accuracy we aim for. Set the discretization parameter K = ⌈ 1 / η ⌉ K=\lceil1/\eta\rceil K = ⌈ 1/ η ⌉ and assume K ≤ T K\le T K ≤ T . We choose η \eta η as a function of T T T at the end.
A family of convex auxiliary objectives
As opposed to Section 5, regularity need not make − log μ -\log\mu − log μ convex. We instead define auxiliary profits with convex reciprocals that match profit at suitable parameters. For x = ( p , q ) ∈ int ( T ) x=(p,q)\in\operatorname{int}(\mathcal{T}) x = ( p , q ) ∈ int ( T ) and λ > 0 \lambda>0 λ > 0 , let
r λ ( p , q ) = 4 ( q − p ) ( λ / F s ( p ) + λ − 1 / ( 1 − F b ( q ) ) ) 2 , λ x = F s ( p ) 1 − F b ( q ) . (38) r_{\lambda}(p,q)=\frac{4(q-p)}{\left(\lambda/F_s(p)+\lambda^{-1}/(1-F_b(q))\right)^2},\qquad\lambda_x=\sqrt{\frac{F_s(p)}{1-F_b(q)}}. \tag*{(38)} r λ ( p , q ) = ( λ / F s ( p ) + λ − 1 / ( 1 − F b ( q )) ) 2 4 ( q − p ) , λ x = 1 − F b ( q ) F s ( p ) . (38)
We call λ x \lambda_x λ x the balancing parameter of x x x . The following lemma justifies optimizing profit through this family.
Lemma 6.2 (Auxiliary profits). For every λ > 0 \lambda>0 λ > 0 , the function x ↦ 1 / r λ ( x ) x\mapsto1/r_\lambda(x) x ↦ 1/ r λ ( x ) is convex on int ( T ) \operatorname{int}(\mathcal{T}) int ( T ) . Writing sech ( t ) = 2 / ( e t + e − t ) \operatorname{sech}(t)=2/(e^t+e^{-t}) sech ( t ) = 2/ ( e t + e − t ) , we have for every x ∈ int ( T ) x\in\operatorname{int}(\mathcal{T}) x ∈ int ( T ) ,
r λ ( x ) = μ ( x ) sech 2 ( log λ λ x ) , 0 ≤ μ ( x ) − r λ ( x ) ≤ μ ( x ) ( log λ λ x ) 2 . (39) r_\lambda(x)=\mu(x)\operatorname{sech}^2\left(\log\frac{\lambda}{\lambda_x}\right),\qquad0\le\mu(x)-r_\lambda(x)\le\mu(x)\left(\log\frac{\lambda}{\lambda_x}\right)^2. \tag*{(39)} r λ ( x ) = μ ( x ) sech 2 ( log λ x λ ) , 0 ≤ μ ( x ) − r λ ( x ) ≤ μ ( x ) ( log λ x λ ) 2 . (39) In particular, 0 < r λ ( x ) ≤ μ ( x ) 0<r_\lambda(x)\le\mu(x) 0 < r λ ( x ) ≤ μ ( x ) and r λ x ( x ) = μ ( x ) r_{\lambda_x}(x)=\mu(x) r λ x ( x ) = μ ( x ) . Moreover, for every θ ∈ R \theta\in\mathbb{R} θ ∈ R ,
μ ⋆ − r λ e θ ( x ) ≤ 2 ( μ ⋆ − r λ ( x ) ) + 2 μ ⋆ θ 2 . (40) \mu^\star-r_{\lambda e^\theta}(x)\le2(\mu^\star-r_\lambda(x))+2\mu^\star\theta^2. \tag*{(40)} μ ⋆ − r λ e θ ( x ) ≤ 2 ( μ ⋆ − r λ ( x )) + 2 μ ⋆ θ 2 . (40)
Proof. Regularity implies that 1 / ( 1 − F b ) 1/(1-F_b) 1/ ( 1 − F b ) is convex [24 Lemma 3.4] ; applying the same result to the reflected seller value 1 − s 1-s 1 − s then gives convexity of 1 / F s 1/F_s 1/ F s . The map ( u , t ) ↦ u 2 / t (u,t)\mapsto u^2/t ( u , t ) ↦ u 2 / t is convex for t > 0 t>0 t > 0 and non-decreasing in u ≥ 0 u\ge0 u ≥ 0 . Composing it with the positive convex sum λ / F s ( p ) + λ − 1 / ( 1 − F b ( q ) ) \lambda/F_s(p)+\lambda^{-1}/(1-F_b(q)) λ / F s ( p ) + λ − 1 / ( 1 − F b ( q )) and the positive affine function 4 ( q − p ) 4(q-p) 4 ( q − p ) proves convexity of 1 / r λ 1/r_\lambda 1/ r λ .
Now set ξ = log ( λ / λ x ) \xi=\log(\lambda/\lambda_x) ξ = log ( λ / λ x ) . By definition, λ = e ξ F s ( p ) / ( 1 − F b ( q ) ) \lambda=e^\xi\sqrt{F_s(p)/(1-F_b(q))} λ = e ξ F s ( p ) / ( 1 − F b ( q )) , so that:
λ F s ( p ) = e ξ F s ( p ) ( 1 − F b ( q ) ) , λ − 1 1 − F b ( q ) = e − ξ F s ( p ) ( 1 − F b ( q ) ) , \frac{\lambda}{F_s(p)}=\frac{e^\xi}{\sqrt{F_s(p)(1-F_b(q))}},\qquad\frac{\lambda^{-1}}{1-F_b(q)}=\frac{e^{-\xi}}{\sqrt{F_s(p)(1-F_b(q))}}, F s ( p ) λ = F s ( p ) ( 1 − F b ( q )) e ξ , 1 − F b ( q ) λ − 1 = F s ( p ) ( 1 − F b ( q )) e − ξ ,
which implies r λ ( x ) = μ ( x ) sech 2 ( ξ ) r_\lambda(x)=\mu(x)\operatorname{sech}^2(\xi) r λ ( x ) = μ ( x ) sech 2 ( ξ ) , by definition of sech \operatorname{sech} sech .
Now let tanh ( t ) = ( e t − e − t ) / ( e t + e − t ) \tanh(t)=(e^t-e^{-t})/(e^t+e^{-t}) tanh ( t ) = ( e t − e − t ) / ( e t + e − t ) be the hyperbolic tangent. By the standard identity tanh 2 ( t ) = 1 − sech 2 ( t ) \tanh^2(t)=1-\operatorname{sech}^2(t) tanh 2 ( t ) = 1 − sech 2 ( t ) , we immediately also get:
μ ( x ) − r λ ( x ) = μ ( x ) tanh 2 ( ξ ) . (41) \mu(x)-r_\lambda(x)=\mu(x)\tanh^2(\xi). \tag*{(41)} μ ( x ) − r λ ( x ) = μ ( x ) tanh 2 ( ξ ) . (41)
Finally, since tanh ( 0 ) = 0 \tanh(0)=0 tanh ( 0 ) = 0 and 0 ≤ tanh ′ ≤ 1 0\le\tanh'\le1 0 ≤ tanh ′ ≤ 1 everywhere, the mean value theorem implies that ∣ tanh ( ξ ) ∣ ≤ ∣ ξ ∣ |\tanh(\xi)|\le|\xi| ∣ tanh ( ξ ) ∣ ≤ ∣ ξ ∣ , which in turn yields μ ( x ) − r λ ( x ) ≤ μ ( x ) ξ 2 \mu(x)-r_\lambda(x)\le\mu(x)\xi^2 μ ( x ) − r λ ( x ) ≤ μ ( x ) ξ 2 as claimed.
To conclude, notice that Equation (41) and tanh \tanh tanh being Lipschitz give:
μ ⋆ − r λ e θ ( x ) = μ ⋆ − μ ( x ) + μ ( x ) tanh 2 ( log λ e θ λ x ) = μ ⋆ − μ ( x ) + μ ( x ) tanh 2 ( ξ + θ ) ≤ μ ⋆ − μ ( x ) + 2 μ ( x ) tanh 2 ( ξ ) + 2 μ ( x ) θ 2 ≤ 2 ( μ ⋆ − r λ ( x ) ) + 2 μ ⋆ θ 2 , \begin{aligned}
\mu^\star-r_{\lambda e^\theta}(x)
&=\mu^\star-\mu(x)+\mu(x)\tanh^2\left(\log\frac{\lambda e^\theta}{\lambda_x}\right)\\
&=\mu^\star-\mu(x)+\mu(x)\tanh^2(\xi+\theta)\\
&\le\mu^\star-\mu(x)+2\mu(x)\tanh^2(\xi)+2\mu(x)\theta^2\\
&\le2(\mu^\star-r_\lambda(x))+2\mu^\star\theta^2,
\end{aligned} μ ⋆ − r λ e θ ( x ) = μ ⋆ − μ ( x ) + μ ( x ) tanh 2 ( log λ x λ e θ ) = μ ⋆ − μ ( x ) + μ ( x ) tanh 2 ( ξ + θ ) ≤ μ ⋆ − μ ( x ) + 2 μ ( x ) tanh 2 ( ξ ) + 2 μ ( x ) θ 2 ≤ 2 ( μ ⋆ − r λ ( x )) + 2 μ ⋆ θ 2 ,
where the last step uses μ ( x ) ≤ μ ⋆ \mu(x)\le\mu^\star μ ( x ) ≤ μ ⋆ .
What the above lemma tells us is essentially that, if we did know λ x ⋆ \lambda_{x^\star} λ x ⋆ , where x ⋆ ∈ arg max ( p , q ) ∈ [ 0 , 1 ] 2 μ ( p , q ) x^\star\in\arg\max_{(p,q)\in[0,1]^2}\mu(p,q) x ⋆ ∈ arg max ( p , q ) ∈ [ 0 , 1 ] 2 μ ( p , q ) , then the corresponding auxiliary profit r λ x ⋆ r_{\lambda_{x^\star}} r λ x ⋆ would be exactly the right objective to optimize. Moreover, an approximation to λ x ⋆ \lambda_{x^\star} λ x ⋆ in log-scale would suffice for the same purpose, due to Equation (39) . Equation (40) instead certifies that if a price has auxiliary profit close to μ ⋆ \mu^\star μ ⋆ for a certain λ \lambda λ , then its auxiliary profit cannot decrease too much if λ \lambda λ gets a small additive perturbation in log-scale.
We now need to find a way to estimate r λ r_\lambda r λ . Notice that one-bit feedback means that the learner observes only the joint trade bit, whereas r λ r_\lambda r λ depends on the two marginal acceptance probabilities. To address this, let x = ( p , q ) x=(p,q) x = ( p , q ) and z = ( u , v ) z=(u,v) z = ( u , v ) be two prices in int ( T ) \operatorname{int}(\mathcal{T}) int ( T ) , and call ( p , v ) (p,v) ( p , v ) and ( u , q ) (u,q) ( u , q ) their crossed pairs. We describe how knowing the joint trade probabilities at z z z and its two crosses lets us compute r λ r_\lambda r λ in x x x . Specifically, for a known multiplier ρ > 0 \rho>0 ρ > 0 and λ = ρ λ z \lambda=\rho\lambda_z λ = ρ λ z , we can estimate the auxiliary profit and its convex reciprocal using queries at z z z and its two crosses, since independence implies
1 r λ ( x ) = P ( z ) 4 ( q − p ) ( ρ P ( p , v ) + ρ − 1 P ( u , q ) ) 2 . (42) \frac{1}{r_\lambda(x)}=\frac{P(z)}{4(q-p)}\left(\frac{\rho}{P(p,v)}+\frac{\rho^{-1}}{P(u,q)}\right)^2. \tag*{(42)} r λ ( x ) 1 = 4 ( q − p ) P ( z ) ( P ( p , v ) ρ + P ( u , q ) ρ − 1 ) 2 . (42)
Using this identity for estimation requires trading at the crossed prices, and the following lemma ensures that this is compatible with low regret. When x x x and z z z have high auxiliary profit for the same λ \lambda λ , their crosses belong to T \mathcal{T} T and also have high profit, with comparable trade probabilities.
Lemma 6.3 (Crossed prices). If x = ( p , q ) x=(p,q) x = ( p , q ) and z = ( u , v ) z=(u,v) z = ( u , v ) are two pairs of prices in int ( T ) \operatorname{int}(\mathcal{T}) int ( T ) satisfying r λ ( x ) , r λ ( z ) ≥ 0.99 μ ⋆ r_\lambda(x),r_\lambda(z)\geq0.99\mu^\star r λ ( x ) , r λ ( z ) ≥ 0.99 μ ⋆ , then ( p , v ) (p,v) ( p , v ) and ( u , q ) (u,q) ( u , q ) belong to T \mathcal{T} T , and either crossed pair y y y satisfies
μ ⋆ − μ ( y ) ≤ 4 ( μ ⋆ − r λ ( x ) + μ ⋆ − r λ ( z ) ) . \mu^\star-\mu(y)\leq4\left(\mu^\star-r_\lambda(x)+\mu^\star-r_\lambda(z)\right). μ ⋆ − μ ( y ) ≤ 4 ( μ ⋆ − r λ ( x ) + μ ⋆ − r λ ( z ) ) . Moreover, the four trade probabilities are within a factor of 3 of each other.
Proof. Let
h x = 1 2 ( λ F s ( p ) + λ − 1 1 − F b ( q ) ) , h z = 1 2 ( λ F s ( u ) + λ − 1 1 − F b ( v ) ) . h_x=\frac{1}{2}\left(\frac{\lambda}{F_s(p)}+\frac{\lambda^{-1}}{1-F_b(q)}\right),\qquad h_z=\frac{1}{2}\left(\frac{\lambda}{F_s(u)}+\frac{\lambda^{-1}}{1-F_b(v)}\right). h x = 2 1 ( F s ( p ) λ + 1 − F b ( q ) λ − 1 ) , h z = 2 1 ( F s ( u ) λ + 1 − F b ( v ) λ − 1 ) .
First, we obtain a bound between each term in the average defining h x h_x h x and h x h_x h x :
∣ λ F s ( p ) − h x ∣ = ∣ λ − 1 1 − F b ( q ) − h x ∣ (the two terms sum to 2 h x ) = h x 2 − 1 P ( x ) ( ( λ F s ( p ) − h x ) 2 = h x 2 − 1 P ( x ) ) = h x 2 − q − p μ ( x ) ( μ ( x ) = ( q − p ) P ( x ) ) = h x 1 − r λ ( x ) μ ( x ) ( q − p = r λ ( x ) h x 2 , by xref‘Equation (39)‘ ) ≤ h x μ ⋆ − r λ ( x ) μ ⋆ ( μ ( x ) ≤ μ ⋆ ) \begin{aligned}
\left|\frac{\lambda}{F_s(p)}-h_x\right|=\left|\frac{\lambda^{-1}}{1-F_b(q)}-h_x\right|\qquad\text{(the two terms sum to }2h_x\text{)} \\
&=\sqrt{h_x^2-\frac{1}{P(x)}}\qquad\left(\left(\frac{\lambda}{F_s(p)}-h_x\right)^2=h_x^2-\frac{1}{P(x)}\right) \\
&=\sqrt{h_x^2-\frac{q-p}{\mu(x)}}\qquad\left(\mu(x)=(q-p)P(x)\right) \\
&=h_x\sqrt{1-\frac{r_\lambda(x)}{\mu(x)}}\qquad\left(q-p=r_\lambda(x)h_x^2,\ \text{by {xref}`Equation (39)`}\right) \\
&\leq h_x\sqrt{\frac{\mu^\star-r_\lambda(x)}{\mu^\star}}\qquad\left(\mu(x)\leq\mu^\star\right)
\end{aligned} F s ( p ) λ − h x = 1 − F b ( q ) λ − 1 − h x (the two terms sum to 2 h x ) = h x 2 − P ( x ) 1 ( ( F s ( p ) λ − h x ) 2 = h x 2 − P ( x ) 1 ) = h x 2 − μ ( x ) q − p ( μ ( x ) = ( q − p ) P ( x ) ) = h x 1 − μ ( x ) r λ ( x ) ( q − p = r λ ( x ) h x 2 , by xref ‘Equation (39)‘ ) ≤ h x μ ⋆ μ ⋆ − r λ ( x ) ( μ ( x ) ≤ μ ⋆ )
≤ 0.1 h x ( r λ ( x ) ≥ 0.99 μ ⋆ ) . (43) \leq0.1h_x\qquad\left(r_\lambda(x)\geq0.99\mu^\star\right). \tag*{(43)} ≤ 0.1 h x ( r λ ( x ) ≥ 0.99 μ ⋆ ) . (43)
In particular, this implies a bound on the crossed price probabilities
P ( p , v ) = ( λ F s ( p ) λ − 1 1 − F b ( v ) ) − 1 ≤ 1 ( 0.9 h x ) ( 0.9 h z ) = 1 0.81 h x h z . (44) P(p,v)=\left(\frac{\lambda}{F_s(p)}\frac{\lambda^{-1}}{1-F_b(v)}\right)^{-1}\leq\frac{1}{(0.9h_x)(0.9h_z)}=\frac{1}{0.81h_xh_z}. \tag*{(44)} P ( p , v ) = ( F s ( p ) λ 1 − F b ( v ) λ − 1 ) − 1 ≤ ( 0.9 h x ) ( 0.9 h z ) 1 = 0.81 h x h z 1 . (44)
The same holds for P ( u , q ) P(u,q) P ( u , q ) .
Now write Δ = ( μ ⋆ − r λ ( x ) ) + ( μ ⋆ − r λ ( z ) ) ≤ 0.02 μ ⋆ \Delta=(\mu^\star-r_\lambda(x))+(\mu^\star-r_\lambda(z))\leq0.02\mu^\star Δ = ( μ ⋆ − r λ ( x )) + ( μ ⋆ − r λ ( z )) ≤ 0.02 μ ⋆ . Then
0 ≤ μ ⋆ − μ ( p , v ) P ( p , v ) + μ ⋆ − μ ( u , q ) P ( u , q ) = μ ⋆ ( 1 P ( p , v ) + 1 P ( u , q ) ) − ( μ ( p , v ) P ( p , v ) + μ ( u , q ) P ( u , q ) ) = μ ⋆ ( λ λ − 1 F s ( p ) ( 1 − F b ( v ) ) + λ λ − 1 F s ( u ) ( 1 − F b ( q ) ) ) − ( ( v − p ) + ( q − u ) ) (Independence) = μ ⋆ ( λ F s ( p ) ( 2 h z − λ F s ( u ) ) + λ F s ( u ) ( 2 h x − λ F s ( p ) ) ) − ( ( q − p ) + ( v − u ) ) = 2 μ ⋆ h x h z − 2 μ ⋆ ( λ F s ( p ) − h x ) ( λ F s ( u ) − h z ) − r λ ( x ) h x 2 − r λ ( z ) h z 2 ( q − p = r λ ( x ) h x 2 ) (45) \begin{aligned}
0\leq\frac{\mu^\star-\mu(p,v)}{P(p,v)}+\frac{\mu^\star-\mu(u,q)}{P(u,q)} \\
&=\mu^\star\left(\frac{1}{P(p,v)}+\frac{1}{P(u,q)}\right)-\left(\frac{\mu(p,v)}{P(p,v)}+\frac{\mu(u,q)}{P(u,q)}\right) \\
&=\mu^\star\left(\frac{\lambda\lambda^{-1}}{F_s(p)(1-F_b(v))}+\frac{\lambda\lambda^{-1}}{F_s(u)(1-F_b(q))}\right)-((v-p)+(q-u))\qquad\text{(Independence)} \\
&=\mu^\star\left(\frac{\lambda}{F_s(p)}\left(2h_z-\frac{\lambda}{F_s(u)}\right)+\frac{\lambda}{F_s(u)}\left(2h_x-\frac{\lambda}{F_s(p)}\right)\right)-((q-p)+(v-u)) \\
&=2\mu^\star h_xh_z-2\mu^\star\left(\frac{\lambda}{F_s(p)}-h_x\right)\left(\frac{\lambda}{F_s(u)}-h_z\right)-r\lambda(x)h_x^2-r\lambda(z)h_z^2 \qquad(q-p=r\lambda(x)h_x^2) \tag*{(45)}
\end{aligned} 0 ≤ P ( p , v ) μ ⋆ − μ ( p , v ) + P ( u , q ) μ ⋆ − μ ( u , q ) = μ ⋆ ( P ( p , v ) 1 + P ( u , q ) 1 ) − ( P ( p , v ) μ ( p , v ) + P ( u , q ) μ ( u , q ) ) = μ ⋆ ( F s ( p ) ( 1 − F b ( v )) λ λ − 1 + F s ( u ) ( 1 − F b ( q )) λ λ − 1 ) − (( v − p ) + ( q − u )) (Independence) = μ ⋆ ( F s ( p ) λ ( 2 h z − F s ( u ) λ ) + F s ( u ) λ ( 2 h x − F s ( p ) λ ) ) − (( q − p ) + ( v − u )) = 2 μ ⋆ h x h z − 2 μ ⋆ ( F s ( p ) λ − h x ) ( F s ( u ) λ − h z ) − r λ ( x ) h x 2 − r λ ( z ) h z 2 ( q − p = r λ ( x ) h x 2 ) (45)
≤ 2 μ ⋆ h x h z − r λ ( x ) h x 2 − r λ ( z ) h z 2 + 2 h x h z μ ⋆ − r λ ( x ) μ ⋆ − r λ ( z ) = ( h x μ ⋆ − r λ ( x ) + h z μ ⋆ − r λ ( z ) ) 2 − μ ⋆ ( h x − h z ) 2 (Completing the square) ≤ Δ ( h x 2 + h z 2 ) − μ ⋆ ( h x − h z ) 2 (Cauchy–Schwarz) (46) \begin{aligned}
\le2\mu^\star h_xh_z-r\lambda(x)h_x^2-r\lambda(z)h_z^2+2h_xh_z\sqrt{\mu^\star-r\lambda(x)}\sqrt{\mu^\star-r\lambda(z)} \\
&=\left(h_x\sqrt{\mu^\star-r\lambda(x)}+h_z\sqrt{\mu^\star-r\lambda(z)}\right)^2-\mu^\star(h_x-h_z)^2
\qquad\text{(Completing the square)} \\
&\le\Delta(h_x^2+h_z^2)-\mu^\star(h_x-h_z)^2
\qquad\text{(Cauchy--Schwarz)} \tag*{(46)}
\end{aligned} ≤ 2 μ ⋆ h x h z − r λ ( x ) h x 2 − r λ ( z ) h z 2 + 2 h x h z μ ⋆ − r λ ( x ) μ ⋆ − r λ ( z ) = ( h x μ ⋆ − r λ ( x ) + h z μ ⋆ − r λ ( z ) ) 2 − μ ⋆ ( h x − h z ) 2 (Completing the square) ≤ Δ ( h x 2 + h z 2 ) − μ ⋆ ( h x − h z ) 2 (Cauchy–Schwarz) (46)
= 2 Δ h x h z − ( μ ⋆ − Δ ) ( h x − h z ) 2 . (47) =2\Delta h_xh_z-(\mu^\star-\Delta)(h_x-h_z)^2. \tag*{(47)} = 2Δ h x h z − ( μ ⋆ − Δ ) ( h x − h z ) 2 . (47)
Since μ ⋆ − Δ > 0 \mu^\star-\Delta>0 μ ⋆ − Δ > 0 , applying first the bound above and then Equation (44) verifies our claim, with y y y as one of the crossed pairs:
μ ⋆ − μ ( y ) ≤ P ( y ) 2 Δ h x h z ≤ 2 Δ 0.81 ≤ 4 Δ . (48) \mu^\star-\mu(y)\le P(y)2\Delta h_xh_z\le\frac{2\Delta}{0.81}\le4\Delta. \tag*{(48)} μ ⋆ − μ ( y ) ≤ P ( y ) 2Δ h x h z ≤ 0.81 2Δ ≤ 4Δ. (48)
Thus both crossed profits are at least 0.92 μ ⋆ > 0 0.92\mu^\star>0 0.92 μ ⋆ > 0 , proving that both crossed prices are in T \mathcal{T} T .
For the probability comparison, the penultimate line of Equation (45) also gives
∣ h x − h z ∣ ≤ 0.02 ( h x 2 + h z 2 ) ≤ 0.2 max { h x , h z } . |h_x-h_z|\le\sqrt{0.02(h_x^2+h_z^2)}\le0.2\max\{h_x,h_z\}. ∣ h x − h z ∣ ≤ 0.02 ( h x 2 + h z 2 ) ≤ 0.2 max { h x , h z } .
Hence min { h x , h z } ≥ 0.8 max { h x , h z } \min\{h_x,h_z\}\ge0.8\max\{h_x,h_z\} min { h x , h z } ≥ 0.8 max { h x , h z } . Consider the original pairs x = ( p , q ) x=(p,q) x = ( p , q ) , z = ( u , v ) z=(u,v) z = ( u , v ) and their crosses ( p , v ) (p,v) ( p , v ) , ( u , q ) (u,q) ( u , q ) . For each of these four pairs, the reciprocal of the trade probability is a product of two factors bounded in Equation (43). Each factor is thus at least 0.9 min { h x , h z } 0.9\min\{h_x,h_z\} 0.9 min { h x , h z } and at most 1.1 max { h x , h z } 1.1\max\{h_x,h_z\} 1.1 max { h x , h z } , so that the reciprocal of the probability of trade is in [ ( 0.9 min { h x , h z } ) 2 , ( 1.1 max { h x , h z } ) 2 ] [(0.9\min\{h_x,h_z\})^2,(1.1\max\{h_x,h_z\})^2] [( 0.9 min { h x , h z } ) 2 , ( 1.1 max { h x , h z } ) 2 ] . Taking maxima and minima over the four pairs therefore gives
max P min P = max ( 1 / P ) min ( 1 / P ) ≤ ( 1.1 max { h x , h z } 0.9 min { h x , h z } ) 2 ≤ ( 1.1 0.9 ⋅ 0.8 ) 2 < 3. \frac{\max P}{\min P}=\frac{\max(1/P)}{\min(1/P)}
\le\left(\frac{1.1\max\{h_x,h_z\}}{0.9\min\{h_x,h_z\}}\right)^2
\le\left(\frac{1.1}{0.9\cdot0.8}\right)^2<3. min P max P = min ( 1/ P ) max ( 1/ P ) ≤ ( 0.9 min { h x , h z } 1.1 max { h x , h z } ) 2 ≤ ( 0.9 ⋅ 0.8 1.1 ) 2 < 3.
Constructing the convex candidate problems
In Section 5, Initialize MHR finds a suitable convex region K K K for the single loss function L L L . In this section, Initialize regular must also select candidate objectives 1 / r λ 1/r_\lambda 1/ r λ , which is nontrivial since the balancing parameters λ x \lambda_x λ x depend on unknown marginal distributions. Recalling Lemma 6.2 , we seek a small family containing a candidate λ \lambda λ whose auxiliary profit nearly attains μ ⋆ \mu^\star μ ⋆ , together with convex regions on which crossed queries (see Lemma 6.3 ) have controlled regret.
Initialization starts from empirical estimates P ^ ( e ) \widehat{P}(e) P ( e ) of the trade probabilities P ( e ) P(e) P ( e ) for e ∈ A = G K ∩ T e\in\mathcal{A}=G_K\cap\mathcal{T} e ∈ A = G K ∩ T , constructed in Step 2, and retains pairs with high estimated profit. It groups these pairs around representative retained pairs, called roots , by comparing their balancing parameters. Although the parameters are unknown, their ratios can be estimated from trade bits at crossed pairs: for x = ( p , q ) x=(p,q) x = ( p , q ) and z = ( u , v ) z=(u,v) z = ( u , v ) , independence gives
log λ x λ z = 1 2 log P ( p , v ) P ( u , q ) . (49) \log\frac{\lambda_x}{\lambda_z}=\frac{1}{2}\log\frac{P(p,v)}{P(u,q)}. \tag*{(49)} log λ z λ x = 2 1 log P ( u , q ) P ( p , v ) . (49)
Each group contains pairs with high profit and similar balancing parameters, so they have a common high auxiliary-profit bound at the root’s parameter. Convexity of the reciprocal in Lemma 6.2 extends this bound to the group’s convex hull. Lemma 6.3 then controls the regret of crossed queries throughout that hull.
For each hull, we discretize a short interval of logarithmic offsets around the root’s balancing parameter. The quadratic approximation bound in Lemma 6.2 ensures that this small mesh suffices to preserve a nearly optimal choice. Each hull–offset pair defines one candidate: if K i K_i K i is its hull, w i w_i w i its root, and θ i \theta_i θ i its offset, its parameter is λ i = e θ i λ w i \lambda_i=e^{\theta_i}\lambda_{w_i} λ i = e θ i λ w i . Each candidate stores K i K_i K i , w i w_i w i , θ i \theta_i θ i , and the root estimate P ^ ( w i ) \widehat{P}(w_i) P ( w i ) , all of which remain fixed. In the following, we use again the routine Initialize MHR from Section 5.
The proof of Lemma 6.4 , which summarizes the properties of Initialize regular, is deferred to Appendix § 10 .6.
Algorithm 7 Initialize regular
Input: Horizon T T T , target accuracy η \eta η with K = ⌈ 1 / η ⌉ ≤ T K = \lceil1/\eta\rceil\le T K = ⌈ 1/ η ⌉ ≤ T , and access to one-bit feedback.
Output: A scale δ \delta δ and candidates, or “no trigger”.
Stop at trading round T T T , even if initialization is unfinished.
Find the profit scale. Run Initialize mhr unchanged, with inputs T T T , K K K , and retain its scale δ \delta δ and sampled index set S \mathcal{S} S . Return “no trigger” if Initialize mhr does so or if δ < 2 16 / K \delta< 2^{16}/K δ < 2 16 / K .
Sample more accurately at the fixed scale. After Initialize mhr returns, post each ( i / K , j / K ) (i/K,j/K) ( i / K , j / K ) with ( i , j ) ∈ S (i,j) \in\mathcal{S} ( i , j ) ∈ S for n = ⌈ 2 38 log ( 2 T ) / δ ⌉ n = \lceil2^{38}\log(2T)/\delta\rceil n = ⌈ 2 38 log ( 2 T ) / δ ⌉ new rounds, and let P ‾ i j \overline{P}_{ij} P ij be the empirical trade frequency. Use the same reconstruction as Initialize mhr: for 1 ≤ i < j ≤ K − 1 1 \le i < j \le K-1 1 ≤ i < j ≤ K − 1 , set
P ^ ( i / K , j / K ) = P ‾ i k P ‾ k j max { P ‾ i k , P ‾ k j , P ‾ k k } , 0 / 0 = 0 , (50) \widehat{P}(i/K,j/K) = \frac{\overline{P}_{ik}\overline{P}_{kj}}{\max\{\overline{P}_{ik},\overline{P}_{kj},\overline{P}_{kk}\}}, \qquad0/0=0, \tag*{(50)} P ( i / K , j / K ) = max { P ik , P k j , P k k } P ik P k j , 0/0 = 0 , (50)
where k k k is the midpoint of the first recursive split separating i , j i,j i , j . Use P ^ ( i / K , i / K ) = P ‾ i i \widehat{P}(i/K,i/K)=\overline{P}_{ii} P ( i / K , i / K ) = P ii for 1 ≤ i ≤ K − 1 1 \le i \le K-1 1 ≤ i ≤ K − 1 , and set probabilities with i = 0 i=0 i = 0 or j = K j=K j = K to zero.
Retain promising pairs. Set μ ^ ( p , q ) = ( q − p ) P ^ ( p , q ) \widehat{\mu}(p,q)=(q-p)\widehat{P}(p,q) μ ( p , q ) = ( q − p ) P ( p , q ) and let μ ^ K ⋆ \widehat{\mu}^{\star}_{K} μ K ⋆ be its maximum over A \mathcal{A} A . Keep pairs with μ ^ ≥ ( 1 − 2 − 13 ) μ ^ K ⋆ \widehat{\mu}\ge(1-2^{-13})\widehat{\mu}^{\star}_{K} μ ≥ ( 1 − 2 − 13 ) μ K ⋆ .
Group prices. Process retained pairs in a fixed order. Assign x = ( p , q ) x=(p,q) x = ( p , q ) to an existing root w = ( u , v ) w=(u,v) w = ( u , v ) if both crosses ( p , v ) (p,v) ( p , v ) and ( u , q ) (u,q) ( u , q ) lie in T \mathcal{T} T , have estimated profit at least 0.99 μ ^ K ⋆ 0.99\widehat{\mu}_{K}^{\star} 0.99 μ K ⋆ , and satisfy
∣ 1 2 log P ^ ( p , v ) P ^ ( u , q ) ∣ ≤ 0.01. \left|\frac{1}{2}\log\frac{\widehat{P}(p,v)}{\widehat{P}(u,q)}\right|\le0.01. 2 1 log P ( u , q ) P ( p , v ) ≤ 0.01.
If no root passes all tests, start a new group with x x x as its root.
Take convex hulls. Use each group’s convex hull as a region.
Discretize the parameter. For D = arctanh ( 1 / 2000 ) D=\operatorname{arctanh}(1/\sqrt{2000}) D = arctanh ( 1/ 2000 ) , divide [ − D , D ] [-D,D] [ − D , D ] into the fewest equal intervals of length at most 1 2 η / δ \frac{1}{2}\sqrt{\eta/\delta} 2 1 η / δ . For each hull, use these endpoints as offsets and create one candidate per offset, saving its root and the root’s probability estimate.
return δ \delta δ and the candidates ( K i , w i , θ i , P ^ ( w i ) ) i = 1 N (K_i,w_i,\theta_i,\widehat{P}(w_i))_{i=1}^{N} ( K i , w i , θ i , P ( w i ) ) i = 1 N .
Lemma 6.4 (Initialization). Consider Initialize regular running until it is finished, without stopping at T T T . With probability at least 1 − 1 / T 1-1/T 1 − 1/ T , it either returns no trigger, with μ ⋆ = O ( 1 / K ) \mu^{\star}=O(1/K) μ ⋆ = O ( 1/ K ) , or returns a scale δ > η \delta>\eta δ > η satisfying δ ≤ μ ⋆ < 6 δ \delta\le\mu^{\star}<6\delta δ ≤ μ ⋆ < 6 δ and a list of N = O ( δ / η log ( 2 T ) ) N=O(\sqrt{\delta/\eta}\log(2T)) N = O ( δ / η log ( 2 T )) fixed candidates ( K i , w i , θ i , P ^ ( w i ) ) (K_i,w_i,\theta_i,\widehat{P}(w_i)) ( K i , w i , θ i , P ( w i )) . Each K i ⊂ int ( T ) K_i\subset\operatorname{int}(\mathcal{T}) K i ⊂ int ( T ) is compact and convex, contains its root w i w_i w i , and has a stored empirical estimate P ^ ( w i ) \widehat{P}(w_i) P ( w i ) of P ( w i ) P(w_i) P ( w i ) . The candidates satisfy:
(i) Auxiliary-profit guarantees. For every candidate i i i and ∣ u ∣ ≤ 10 − 3 |u|\le10^{-3} ∣ u ∣ ≤ 1 0 − 3 , r λ i e u ≥ 0.99 μ ⋆ r_{\lambda_i e^u}\ge0.99\mu^{\star} r λ i e u ≥ 0.99 μ ⋆ throughout K i K_i K i . Some fixed candidate i ⋆ i^{\star} i ⋆ satisfies
max x ∈ K i ⋆ r λ i ⋆ e u ( x ) ≥ μ ⋆ − 3 η ( ∣ u ∣ ≤ 10 − 3 η / δ ) . \max_{x\in K_{i^{\star}}}r_{\lambda_{i^{\star}}e^u}(x)\ge\mu^{\star}-3\eta\qquad(|u|\le10^{-3}\sqrt{\eta/\delta}). x ∈ K i ⋆ max r λ i ⋆ e u ( x ) ≥ μ ⋆ − 3 η ( ∣ u ∣ ≤ 1 0 − 3 η / δ ) .
(ii) Crossed-query bounds. For any two points of K i K_i K i , both crosses belong to T \mathcal{T} T . Their probabilities of trade, and the probabilities at the original points, belong to [ P ^ ( w i ) / 10 , 10 P ^ ( w i ) ] [\widehat{P}(w_i)/10,10\widehat{P}(w_i)] [ P ( w i ) /10 , 10 P ( w i )] . Moreover,
δ ≤ P ^ ( w i ) ≤ 1 , δ q − p ≤ 10 P ^ ( w i ) ( ( p , q ) ∈ K i ) . \delta\le\widehat{P}(w_i)\le1,\qquad\frac{\delta}{q-p}\le10\widehat{P}(w_i)\quad((p,q)\in K_i). δ ≤ P ( w i ) ≤ 1 , q − p δ ≤ 10 P ( w i ) (( p , q ) ∈ K i ) .
On a valid triggered transcript, initialization incurs O ( K log 2 ( T ) ) O(K\log^2(T)) O ( K log 2 ( T )) regret.
Successive elimination
Consider a candidate from the list constructed by Algorithm 7 with parameter λ i = e θ i λ w i \lambda_i=e^{\theta_i}\lambda_{w_i} λ i = e θ i λ w i . We know the offset θ i \theta_i θ i , but the root’s balancing parameter λ w i \lambda_{w_i} λ w i depends on unknown marginal probabilities. Equation (42) shows that, for any multiplier ρ > 0 \rho> 0 ρ > 0 , a reference pair of prices z z z can be used to produce observations of 1 / r ρ λ z 1/r_{\rho\lambda_z} 1/ r ρ λ z . To approximate the candidate’s intended objective 1 / r λ i 1/r_{\lambda_i} 1/ r λ i , we thus choose ρ \rho ρ by estimating λ i / λ z = e θ i λ w i / λ z \lambda_i/\lambda_z = e^{\theta_i\lambda_{w_i}/\lambda_z} λ i / λ z = e θ i λ w i / λ z . Initially z = w i z = w_i z = w i , so this multiplier is simply e θ i e^{\theta_i} e θ i . The following successive elimination algorithm repeatedly refines active candidates and eliminates those with low estimated auxiliary profit, up until the final trading round T T T .
Algorithm 8 Successive Opt: Refinement and Successive Elimination
Input: Horizon T T T , target accuracy η \eta η , the output of initialization, and access to one-bit feedback.
Output: Price pairs posted in the remaining trading rounds.
Stop at trading round T T T , even while producing a loss observation.
if Initialize regular returned “no trigger” then
Post ( 0 , 0 ) (0,0) ( 0 , 0 ) in every remaining round; stop .
Mark all candidates active and initialize their references z i ← w i z_i \leftarrow w_i z i ← w i .
for j = 0 , 1 , … j = 0,1,\ldots j = 0 , 1 , … while trading rounds remain do
Set the accuracy ε j ← δ 2 − j \varepsilon_j \leftarrow\delta2^{-j} ε j ← δ 2 − j and run length n ← 4 j ⌈ C 0 log 4 ( 2 T ) ⌉ n \leftarrow4^j\lceil C_0\log^4(2T)\rceil n ← 4 j ⌈ C 0 log 4 ( 2 T )⌉ , for a sufficiently large universal constant C 0 C_0 C 0 .
for each active candidate i i i do
Calibrate. Using offset θ i \theta_i θ i , fresh trade bits at crossed prices formed by w i w_i w i and z i z_i z i , estimate ρ i ≈ λ i / λ z i \rho_i \approx\lambda_i/\lambda_{z_i} ρ i ≈ λ i / λ z i .
Refine. Keep z i z_i z i and ρ i \rho_i ρ i fixed throughout the following independent runs.
for each of ⌈ C 0 log ( 2 T ) ⌉ \lceil C_0\log(2T)\rceil ⌈ C 0 log ( 2 T )⌉ runs do
Run ConOpt on K i \mathcal{K}_i K i to minimize 1 / r ρ i λ z i 1/r_{\rho_i\lambda_{z_i}} 1/ r ρ i λ z i for n n n loss observations. Construct each observation from fresh trade bits at z i z_i z i and its crossed prices with the optimizer’s query.
Record the average query point and the average loss observation from this run.
Select. Choose the run with the smallest empirical average loss L ^ \widehat{L} L . Let y i y_i y i be its average query point and set r ^ i ← 1 / max { L ^ , 1 / ( 10 δ ) } \widehat{r}_i \leftarrow1/\max\{\widehat{L},1/(10\delta)\} r i ← 1/ max { L , 1/ ( 10 δ )} .
Eliminate. Retain precisely the candidates satisfying r ^ i ≥ max ℓ a c t i v e r ^ ℓ − 10 ( η + ε j ) \widehat{r}_i \ge\max_{\ell\ \mathrm{active}}\widehat{r}_\ell- 10(\eta+\varepsilon_j) r i ≥ max ℓ active r ℓ − 10 ( η + ε j ) .
Update references. Set z i ← y i z_i \leftarrow y_i z i ← y i for every surviving candidate, ready for calibration in the next phase.
Unbiased estimator. After each elimination phase, surviving references are updated to match the candidates’ improved profits. Calibration estimates λ i / λ z \lambda_i/\lambda_z λ i / λ z from the crosses between the root w i w_i w i and the updated reference z z z . The following lemma bounds the calibration error, the observations’ number of rounds and regret.
Lemma 6.5 (Convex loss observations). Assume the guarantees of the high probability event in Lemma 6.4 hold, with Initialize regular triggering before time T T T . Fix a candidate i i i and a reference z ∈ K i z \in\mathcal{K}_i z ∈ K i .
(i) Calibration. Calibration can be implemented so as to incur O ( log ( T ) / η ) O(\log(T)/\eta) O ( log ( T ) / η ) regret, returning a multiplier ρ \rho ρ such that, except with probability at most 1 / T 1/T 1/ T ,
∣ log ρ λ z λ i ∣ ≤ 10 − 3 η / δ . (51) \left|\log\frac{\rho\lambda_z}{\lambda_i}\right| \le10^{-3}\sqrt{\eta/\delta}. \tag*{(51)} log λ i ρ λ z ≤ 1 0 − 3 η / δ . (51)
(ii) Loss observations. For a constant C C C and for ρ \rho ρ satisfying Equation (51) , set λ = ρ λ z \lambda=\rho\lambda_z λ = ρ λ z and L = 1 / r λ L=1/r_{\lambda} L = 1/ r λ . Each query x ∈ K i x\in\mathcal{K}_i x ∈ K i admits an observation Y x Y_x Y x using at least one trading round at prices in T \mathcal{T} T , with
0 ≤ Y x ≤ C / δ , E [ Y x ∣ p a s t , x ] = L ( x ) , (52) 0 \le Y_x \le C/\delta,\qquad\mathbb{E}[Y_x\mid\mathrm{past},x]=L(x), \tag*{(52)} 0 ≤ Y x ≤ C / δ , E [ Y x ∣ past , x ] = L ( x ) , (52)
with the expected number of trading rounds to produce Y x Y_x Y x bounded by C / δ C/\delta C / δ , again conditionally on the choice of x x x and on the history before generating Y x Y_x Y x . Every price pair y y y posted to produce Y x Y_x Y x moreover satisfies
μ ∗ − μ ( y ) ≤ C ( μ ∗ − r λ ( z ) ) + C δ 2 ( L ( x ) − min x ∈ K i L ) . (53) \mu^*-\mu(y)\le C(\mu^*-r_{\lambda}(z))+C\delta^2\left(L(x)-\min_{x\in\mathcal{K}_i}L\right). \tag*{(53)} μ ∗ − μ ( y ) ≤ C ( μ ∗ − r λ ( z )) + C δ 2 ( L ( x ) − x ∈ K i min L ) . (53)
Proof. Throughout this proof, write w = ( a , b ) w=(a,b) w = ( a , b ) for the fixed candidate’s root and z = ( u , v ) z=(u,v) z = ( u , v ) for the reference. For part (i), sample each crossed pair y ∈ { ( u , b ) , ( a , v ) } y\in\{(u,b),(a,v)\} y ∈ {( u , b ) , ( a , v )} for ⌈ C δ log ( 2 T ) / ( η P ~ ( w ) ) ⌉ \lceil C\delta\log(2T)/(\eta\widetilde{P}(w))\rceil ⌈ C δ log ( 2 T ) / ( η P ( w ))⌉ fresh rounds. Let P ‾ ( y ) \overline{P}(y) P ( y ) be the fraction of these new rounds in which a trade occurs, and set
d ^ = 1 2 log P ‾ ( u , b ) P ‾ ( a , v ) , \widehat{d}=\frac{1}{2}\log\frac{\overline{P}(u,b)}{\overline{P}(a,v)}, d = 2 1 log P ( a , v ) P ( u , b ) ,
with d ^ = 0 \widehat d=0 d = 0 if either frequency is zero. By Equation (49) , this estimates 1 2 log ( P ( u , b ) / P ( a , v ) ) = log ( λ z / λ w ) \frac{1}{2}\log(P(u,b)/P(a,v))=\log(\lambda_z/\lambda_w) 2 1 log ( P ( u , b ) / P ( a , v )) = log ( λ z / λ w ) .
Both trade probabilities are at least P ^ ( w ) / 10 \widehat P(w)/10 P ( w ) /10 by Lemma 6.4 (ii), so the expected number of trades at each cross y y y is at least C δ log ( 2 T ) / ( 10 η ) C\delta\log(2T)/(10\eta) C δ log ( 2 T ) / ( 10 η ) . The multiplicative Chernoff bound (see Proposition A.5 ) and a union bound therefore give, for a sufficiently large universal C C C ,
∣ P ‾ ( y ) P ( y ) − 1 ∣ ≤ 10 − 3 2 η / δ ( y ∈ { ( u , b ) , ( a , v ) } ) , \left|\frac{\overline P(y)}{P(y)}-1\right|\leq\frac{10^{-3}}{2}\sqrt{\eta/\delta}\qquad(y\in\{(u,b),(a,v)\}), P ( y ) P ( y ) − 1 ≤ 2 1 0 − 3 η / δ ( y ∈ {( u , b ) , ( a , v )}) ,
except with probability 1 / T 1/T 1/ T . Since η < δ \eta<\delta η < δ , both frequencies are then positive, and ∣ log ( 1 + t ) ∣ ≤ 2 ∣ t ∣ |\log(1+t)|\leq2|t| ∣ log ( 1 + t ) ∣ ≤ 2∣ t ∣ for ∣ t ∣ ≤ 1 / 2 |t|\leq1/2 ∣ t ∣ ≤ 1/2 gives
∣ d ^ − log λ z λ w ∣ = ∣ 1 2 ( log P ‾ ( u , b ) P ( u , b ) − log P ‾ ( a , v ) P ( a , v ) ) ∣ ≤ 1 2 ∑ y ∈ { ( u , b ) , ( a , v ) } ∣ log P ‾ ( y ) P ( y ) ∣ ≤ 10 − 3 η / δ . \left|\widehat d-\log\frac{\lambda_z}{\lambda_w}\right|
=\left|\frac{1}{2}\left(\log\frac{\overline P(u,b)}{P(u,b)}-\log\frac{\overline P(a,v)}{P(a,v)}\right)\right|
\leq\frac{1}{2}\sum_{y\in\{(u,b),(a,v)\}}\left|\log\frac{\overline P(y)}{P(y)}\right|
\leq10^{-3}\sqrt{\eta/\delta}. d − log λ w λ z = 2 1 ( log P ( u , b ) P ( u , b ) − log P ( a , v ) P ( a , v ) ) ≤ 2 1 y ∈ {( u , b ) , ( a , v )} ∑ log P ( y ) P ( y ) ≤ 1 0 − 3 η / δ .
Since λ i = e θ i λ w \lambda_i=e^{\theta_i}\lambda_w λ i = e θ i λ w , choosing ρ = e θ i − d ^ \rho=e^{\theta_i-\widehat d} ρ = e θ i − d gives log ( ρ λ z / λ i ) = log ( λ z / λ w ) − d ^ \log(\rho\lambda_z/\lambda_i)=\log(\lambda_z/\lambda_w)-\widehat d log ( ρ λ z / λ i ) = log ( λ z / λ w ) − d , so the preceding bound proves Equation (51) .
By Lemma 6.4 (ii), P ^ ( w ) ≥ δ \widehat P(w)\geq\delta P ( w ) ≥ δ , so the total number of rounds is at most 2 ⌈ C log ( 2 T ) / η ⌉ = O ( log ( T ) / η ) 2\lceil C\log(2T)/\eta\rceil=O(\log(T)/\eta) 2 ⌈ C log ( 2 T ) / η ⌉ = O ( log ( T ) / η ) . The same point forces both crosses in T \mathcal T T , so that their expected profit is non-negative, immediately implying an O ( log ( T ) / η ) O(\log(T)/\eta) O ( log ( T ) / η ) regret bound.
For part (ii), fix a calibration satisfying Equation (51) and set λ = ρ λ z \lambda=\rho\lambda_z λ = ρ λ z . Since η < δ \eta<\delta η < δ , its error satisfies ∣ log ( λ / λ i ) ∣ ≤ 10 − 3 η / δ ≤ 10 − 3 |\log(\lambda/\lambda_i)|\leq10^{-3}\sqrt{\eta/\delta}\leq10^{-3} ∣ log ( λ / λ i ) ∣ ≤ 1 0 − 3 η / δ ≤ 1 0 − 3 , so the perturbed-parameter guarantee in Lemma 6.4 (i) applies. Together with r λ ( x ) ≤ μ ( x ) r_\lambda(x)\leq\mu(x) r λ ( x ) ≤ μ ( x ) from Lemma 6.2 , it gives
0.99 δ ≤ 0.99 μ ⋆ ≤ r λ ( x ) ≤ μ ( x ) ≤ μ ⋆ ≤ 6 δ ( x ∈ K i ) . (54) 0.99\delta\leq0.99\mu^\star\leq r_\lambda(x)\leq\mu(x)\leq\mu^\star\leq6\delta\qquad(x\in\mathcal K_i). \tag*{(54)} 0.99 δ ≤ 0.99 μ ⋆ ≤ r λ ( x ) ≤ μ ( x ) ≤ μ ⋆ ≤ 6 δ ( x ∈ K i ) . (54)
The latter lemma also gives convexity of L = 1 / r λ L=1/r_\lambda L = 1/ r λ ; continuity follows from continuity of the CDFs and positivity of the denominators on K i \mathcal K_i K i .
We now construct an observation of L ( x ) L(x) L ( x ) for a query x = ( p , q ) ∈ K i x=(p,q)\in\mathcal K_i x = ( p , q ) ∈ K i . Equation (42) expresses L ( x ) L(x) L ( x ) as a known multiple of P ( z ) P(z) P ( z ) times the square of a weighted sum of reciprocal crossed probabilities. We represent these factors by one bit B 0 B_0 B 0 and two independent copies B , B ′ B,B' B , B ′ : multiplying the copies will produce the square in expectation. All queried prices are z z z , ( p , v ) (p,v) ( p , v ) , or ( u , q ) (u,q) ( u , q ) : by Lemma 6.4 (ii), they are in T \mathcal T T and their probabilities lie between P ^ ( w ) / 10 \widehat P(w)/10 P ( w ) /10 and 10 P ^ ( w ) 10\widehat P(w) 10 P ( w ) .
First, represent the reference probability by a bit
B 0 ∼ Ber ( P ( z ) 20 P ^ ( w ) ) . B_0\sim\operatorname{Ber}\left(\frac{P(z)}{20\widehat P(w)}\right). B 0 ∼ Ber ( 20 P ( w ) P ( z ) ) .
Its mean is at most 1 / 2 1/2 1/2 by the probability bounds above. If 20 P ^ ( w ) < 1 20\widehat P(w)<1 20 P ( w ) < 1 , the Bernoulli factory from Theorem 1 in Huber [27 ] generates this bit from trade bits at z z z in O ( 1 / P ^ ( w ) ) O(1/\widehat P(w)) O ( 1/ P ( w )) expected trading rounds. Otherwise, simply retain a trade bit’s success with independent probability 1 / ( 20 P ^ ( w ) ) 1/(20\widehat P(w)) 1/ ( 20 P ( w )) , with the same cost bound.
Next, represent the weighted reciprocal sum. To generate each of B , B ′ B,B' B , B ′ , independently choose y = ( p , v ) y=(p,v) y = ( p , v ) with probability ρ / ( ρ + ρ − 1 ) \rho/(\rho+\rho^{-1}) ρ / ( ρ + ρ − 1 ) and y = ( u , q ) y=(u,q) y = ( u , q ) otherwise. An adaptation of the same procedure from Huber [27 ] we have used above can generate a bit of mean P ^ ( w ) / ( 20 P ( y ) ) ≤ 1 / 2 \widehat P(w)/(20P(y))\leq1/2 P ( w ) / ( 20 P ( y )) ≤ 1/2 from trade bits at y y y (see Propositions 1, 2, and 4 in Lee et al. [33 ] ). It uses O ( 1 / P ( y ) ) = O ( 1 / P ^ ( w ) ) O(1/P(y))=O(1/\widehat P(w)) O ( 1/ P ( y )) = O ( 1/ P ( w )) expected rounds, since P ( y ) ≥ P ^ ( w ) / 10 P(y)\geq\widehat P(w)/10 P ( y ) ≥ P ( w ) /10 . Averaging over the two crossed price pairs gives
E [ B ] = E [ B ′ ] = P ^ ( w ) 20 ( ρ + ρ − 1 ) ( ρ P ( p , v ) + ρ − 1 P ( u , q ) ) . \mathbb E[B]=\mathbb E[B']=\frac{\widehat P(w)}{20(\rho+\rho^{-1})}\left(\frac{\rho}{P(p,v)}+\frac{\rho^{-1}}{P(u,q)}\right). E [ B ] = E [ B ′ ] = 20 ( ρ + ρ − 1 ) P ( w ) ( P ( p , v ) ρ + P ( u , q ) ρ − 1 ) .
Use fresh independent randomness and trade bits for all three bits. If the factories request no trade bits, post z z z once and discard its bit, so each observation uses at least one trading round. Their product now has the required factors in its mean; rescale it by returning
Y x = 2000 ( ρ + ρ − 1 ) 2 ( q − p ) P ^ ( w ) B 0 B B ′ . (55) Y_x=\frac{2000(\rho+\rho^{-1})^2}{(q-p)\widehat P(w)}B_0BB'. \tag*{(55)} Y x = ( q − p ) P ( w ) 2000 ( ρ + ρ − 1 ) 2 B 0 B B ′ . (55)
To verify unbiasedness, independence of the bits gives
E [ Y x ] = 2000 ( ρ + ρ − 1 ) 2 ( q − p ) P ^ ( w ) P ( z ) 20 P ^ ( w ) ( E [ B ] ) 2 = 2000 P ( z ) 20 3 1 q − p ( ρ P ( p , v ) + ρ − 1 P ( u , q ) ) 2 = L ( x ) , \begin{aligned}
\mathbb{E}[Y_x] = \frac{2000(\rho+\rho^{-1})^2}{(q-p)\widehat{P}(w)}\frac{P(z)}{20\widehat{P}(w)}\left(\mathbb{E}[B]\right)^2 \\
&= \frac{2000P(z)}{20^3}\frac{1}{q-p}\left(\frac{\rho}{P(p,v)}+\frac{\rho^{-1}}{P(u,q)}\right)^2 = L(x),
\end{aligned} E [ Y x ] = ( q − p ) P ( w ) 2000 ( ρ + ρ − 1 ) 2 20 P ( w ) P ( z ) ( E [ B ] ) 2 = 2 0 3 2000 P ( z ) q − p 1 ( P ( p , v ) ρ + P ( u , q ) ρ − 1 ) 2 = L ( x ) ,
where 2000 / 20 3 = 1 / 4 2000/20^3=1/4 2000/2 0 3 = 1/4 and the last equality is Equation (42) .
It remains to bound the observation size and expected duration. For the size, we first control the multiplier in Equation (55) . Evaluating Equation (39) at z z z and using λ = ρ λ z \lambda=\rho\lambda_z λ = ρ λ z gives r λ ( z ) = μ ( z ) sech 2 ( log ρ ) = 4 μ ( z ) / ( ρ + ρ − 1 ) 2 r_\lambda(z)=\mu(z)\operatorname{sech}^2(\log\rho)=4\mu(z)/(\rho+\rho^{-1})^2 r λ ( z ) = μ ( z ) sech 2 ( log ρ ) = 4 μ ( z ) / ( ρ + ρ − 1 ) 2 . Rearranging and using Equation (54) yields
( ρ + ρ − 1 ) 2 = 4 μ ( z ) r λ ( z ) ≤ 4 0.99 . (56) (\rho+\rho^{-1})^2=\frac{4\mu(z)}{r_\lambda(z)}\le\frac{4}{0.99}. \tag*{(56)} ( ρ + ρ − 1 ) 2 = r λ ( z ) 4 μ ( z ) ≤ 0.99 4 . (56)
Combining this with B 0 B B ′ ≤ 1 B_0BB'\le1 B 0 B B ′ ≤ 1 and δ / ( q − p ) ≤ 10 P ^ ( w ) \delta/(q-p)\le10\widehat{P}(w) δ / ( q − p ) ≤ 10 P ( w ) from Lemma 6.4 (ii) gives, picking C ≥ 81000 C\ge81000 C ≥ 81000 :
0 ≤ Y x ≤ 2000 ( ρ + ρ − 1 ) 2 ( q − p ) P ^ ( w ) ≤ 2000 ⋅ 4 0.99 ⋅ 10 δ ≤ C δ . 0\le Y_x\le\frac{2000(\rho+\rho^{-1})^2}{(q-p)\widehat{P}(w)}\le2000\cdot\frac{4}{0.99}\cdot\frac{10}{\delta}\le\frac{C}{\delta}. 0 ≤ Y x ≤ ( q − p ) P ( w ) 2000 ( ρ + ρ − 1 ) 2 ≤ 2000 ⋅ 0.99 4 ⋅ δ 10 ≤ δ C .
For the duration, sum the three O ( 1 / P ^ ( w ) ) O(1/\widehat{P}(w)) O ( 1/ P ( w )) factory costs, then use P ^ ( w ) ≥ δ \widehat{P}(w)\ge\delta P ( w ) ≥ δ from the same initialization guarantee. This proves all the observation bounds in Equation (52) .
Finally, to bound the regret of each trading round, L = 1 / r λ L=1/r_\lambda L = 1/ r λ gives
max K i r λ − r λ ( x ) = r λ ( x ) ( max K i r λ ) ( L ( x ) − min K i L ) \max_{K_i}r_\lambda-r_\lambda(x)=r_\lambda(x)\left(\max_{K_i}r_\lambda\right)\left(L(x)-\min_{K_i}L\right) K i max r λ − r λ ( x ) = r λ ( x ) ( K i max r λ ) ( L ( x ) − K i min L )
≤ 36 δ 2 ( L ( x ) − min K i L ) , \le36\delta^2\left(L(x)-\min_{K_i}L\right), ≤ 36 δ 2 ( L ( x ) − K i min L ) ,
using r λ ≤ 6 δ r_\lambda\le6\delta r λ ≤ 6 δ from Equation (54) . Also, μ ⋆ − max K i r λ ≤ μ ⋆ − r λ ( z ) \mu^\star-\max_{K_i}r_\lambda\le\mu^\star-r_\lambda(z) μ ⋆ − max K i r λ ≤ μ ⋆ − r λ ( z ) . Thus Lemma 6.3 gives, for either cross y y y and using again Equation (54) ,
μ ⋆ − μ ( y ) ≤ 4 ( μ ⋆ − r λ ( x ) + μ ⋆ − r λ ( z ) ) \mu^\star-\mu(y)\le4\left(\mu^\star-r_\lambda(x)+\mu^\star-r_\lambda(z)\right) μ ⋆ − μ ( y ) ≤ 4 ( μ ⋆ − r λ ( x ) + μ ⋆ − r λ ( z ) )
≤ 8 ( μ ⋆ − r λ ( z ) ) + 144 δ 2 ( L ( x ) − min K i L ) . \le8\left(\mu^\star-r_\lambda(z)\right)+144\delta^2\left(L(x)-\min_{K_i}L\right). ≤ 8 ( μ ⋆ − r λ ( z ) ) + 144 δ 2 ( L ( x ) − K i min L ) .
The reference z z z has gap μ ⋆ − μ ( z ) ≤ μ ⋆ − r λ ( z ) \mu^\star-\mu(z)\le\mu^\star-r_\lambda(z) μ ⋆ − μ ( z ) ≤ μ ⋆ − r λ ( z ) , so the same bound holds for every round used to produce Y x Y_x Y x , proving Equation (53) .
Refining and elimination In Successive Opt, ConOpt is invoked several times in order to successively refine our estimates while minimizing the regret. Here, refinement means to find a point with nearly maximal auxiliary profit within one candidate’s region and to estimate the auxiliary profit of that point. More precisely, fix a candidate i i i , its current reference z i z_i z i , and a calibrated multiplier ρ i \rho_i ρ i , and write λ = ρ i λ z i \lambda=\rho_i\lambda_{z_i} λ = ρ i λ z i . We keep z i z_i z i and ρ i \rho_i ρ i fixed throughout and minimize the convex loss L = 1 / r λ L=1/r_\lambda L = 1/ r λ on K i K_i K i using ConOpt from Section 5.
The loss is convex by Lemma 6.2 , continuous by continuity of the CDFs, and defined on a compact convex region K i K_i K i by Lemma 6.4 . Lemma 6.5(ii) gives conditionally unbiased observations bounded by C / δ C/\delta C / δ . Multiplying the loss L ( x ) L(x) L ( x ) and the observation Y x Y_x Y x by δ \delta δ bounds the absolute value of the two by a constant, by Lemmata 6.2 and 6.4 and Lemma 6.5(ii). This then trivially implies a constant upper bound on the subgaussian norm (see Definition 5.2) of δ ( Y x − L ( x ) ) \delta(Y_x-L(x)) δ ( Y x − L ( x )) , so that we can safely apply Corollary 3.1 in the affine hull of K i K_i K i , similarly to what we did in Section 5.
Dividing the associated regret bound by δ \delta δ yields, for the non-scaled loss L L L with τ \tau τ indexing the “virtual” convex optimization time like in Section 5:
E [ ∑ τ = 1 n ( L ( x τ ) − min K i L ) ] ≤ C δ n log 2 ( 2 n ) . (57) \mathbb{E}\left[\sum_{\tau=1}^{n}\left(L(x_\tau)-\min_{K_i}L\right)\right]\le\frac{C}{\delta}\sqrt{n}\log^2(2n). \tag*{(57)} E [ τ = 1 ∑ n ( L ( x τ ) − K i min L ) ] ≤ δ C n log 2 ( 2 n ) . (57)
Figure 5. One cycle for candidate i i i . With z i , ρ i z_i,\rho_i z i , ρ i fixed, refinement optimizes 1 / r ρ i λ z i 1/r_{\rho_i\lambda_{z_i}} 1/ r ρ i λ z i and returns the selected run’s average query y i y_i y i and estimate r ^ i \hat r_i r ^ i . If i i i survives comparison with the other candidates, y i y_i y i becomes its reference for the next phase. The region, root, and intended parameter remain fixed.
We then use independent repetitions to turn this expected guarantee into a reliable point and profit estimate, via Markov’s inequality: in phase j j j , at accuracy ε j = δ 2 − j \varepsilon_j=\delta2^{-j} ε j = δ 2 − j , run ⌈ C 0 log ( 2 T ) ⌉ \lceil C_0\log(2T)\rceil ⌈ C 0 log ( 2 T )⌉ independent copies of ConOpt, each for
n = 4 j ⌈ C 0 log 4 ( 2 T ) ⌉ (58) n=4^j\lceil C_0\log^4(2T)\rceil \tag*{(58)} n = 4 j ⌈ C 0 log 4 ( 2 T )⌉ (58)
observations, where C 0 C_0 C 0 is a sufficiently large universal constant, chosen after the constants in the observation and convex-optimization bounds are fixed. For each copy, average its queries and its loss observations. Select the copy with the smallest empirical average L ^ \hat L L ^ , and return its query average y i y_i y i and the corresponding auxiliary-profit estimate r ^ i = 1 / max { L ^ , 1 / ( 10 δ ) } \hat r_i=1/\max\{\hat L,1/(10\delta)\} r ^ i = 1/ max { L ^ , 1/ ( 10 δ )} . y i y_i y i becomes the new reference only if candidate i i i survives the subsequent elimination step.
We discard candidates whose estimates are more than 10 ( η + ε j ) 10(\eta+\varepsilon_j) 10 ( η + ε j ) below the largest estimate, then set z i = y i z_i=y_i z i = y i for each survivor. Starting with ε 0 = δ \varepsilon_0=\delta ε 0 = δ , we halve the accuracy after each phase and continue refinement and elimination until trading round T T T .
The following lemma supplies guarantees on the refinement procedure and elimination rule. Part (i) bounds the accuracy of one refinement, while part (ii) ensures that a nearly optimal candidate remains available, i.e. the i ⋆ i^\star i ⋆ from Lemma 6.4 satisfying
max x ∈ K i ⋆ r λ i ⋆ e u ( x ) ≥ μ ⋆ − 3 η ( ∣ u ∣ ≤ 10 − 3 η / δ ) . \max_{x\in\mathcal{K}_{i^\star}} r_{\lambda_{i^\star}e^u}(x)\ge\mu^\star-3\eta\qquad(|u|\le10^{-3}\sqrt{\eta/\delta}). x ∈ K i ⋆ max r λ i ⋆ e u ( x ) ≥ μ ⋆ − 3 η ( ∣ u ∣ ≤ 1 0 − 3 η / δ ) .
When needed to disambiguate, we use the superscript to link each iteration of the loop in Algorithm 8 with the value of the considered parameters at its beginning.
Lemma 6.6 (Refinement and successive elimination). Assume initialization triggers with the guarantees of Lemma 6.4 .
(i) Refinement. Fix a candidate i i i and associated reference and valid calibrated parameter λ \lambda λ . In any phase j j j started before T T T , refinement returns y ∈ K i y\in\mathcal{K}_i y ∈ K i and r ^ i \hat r_i r ^ i satisfying, except with probability 1 / T 1/T 1/ T ,
max K i r λ − ε j ≤ r ^ i ≤ r λ ( y ) + ε j . (59) \max_{\mathcal{K}_i}r_\lambda-\varepsilon_j\le\hat r_i\le r_\lambda(y)+\varepsilon_j. \tag*{(59)} K i max r λ − ε j ≤ r ^ i ≤ r λ ( y ) + ε j . (59) This guarantee refers to refinement run to completion.
(ii) Elimination. If all calibrations and refinements through a completed phase j j j satisfy Equations (51) and (59) , then i ⋆ i^\star i ⋆ survives to phase j + 1 j+1 j + 1 and every retained and updated reference z i ( j + 1 ) z_i^{(j+1)} z i ( j + 1 ) satisfies
μ ⋆ − r ρ i ( j + 1 ) λ z i ( j + 1 ) ( z i ( j + 1 ) ) ≤ 65 ( η + ε j + 1 ) . (60) \mu^\star-r_{\rho_i^{(j+1)}\lambda_{z_i^{(j+1)}}}\left(z_i^{(j+1)}\right)\le65(\eta+\varepsilon_{j+1}). \tag*{(60)} μ ⋆ − r ρ i ( j + 1 ) λ z i ( j + 1 ) ( z i ( j + 1 ) ) ≤ 65 ( η + ε j + 1 ) . (60)
where ρ i ( j + 1 ) \rho_i^{(j+1)} ρ i ( j + 1 ) is the output of the next valid calibration for candidate i i i . The same bound holds after a valid calibration at the start of phase 0.
Proof. For part (i), fix the starting history and write ε = ε j \varepsilon= \varepsilon_j ε = ε j . In one optimizer copy, let x τ ∈ K i x_\tau\in\mathcal{K}_i x τ ∈ K i denote its queries and Y τ Y_\tau Y τ the corresponding loss observations constructed as in Lemma 6.5(ii). Since this phase starts before round T T T , a run of length n / 4 n/4 n /4 has already finished if j ≥ 1 j \ge1 j ≥ 1 , so n ≤ 4 T n \le4T n ≤ 4 T . For j = 0 j = 0 j = 0 , n = ⌈ C 0 log 4 ( 2 T ) ⌉ n = \lceil C_0 \log^4(2T) \rceil n = ⌈ C 0 log 4 ( 2 T )⌉ . By Equation (57) and the choice of n n n ,
E [ 1 n ∑ τ = 1 n L ( x τ ) − min x ∈ K i L ( x ) ] ≤ C δ n log 2 ( T ) ≤ ε 800 δ 2 . (61) \mathbb{E}\left[\frac{1}{n}\sum_{\tau=1}^{n}L(x_\tau)-\min_{x\in\mathcal{K}_i}L(x)\right]\leq\frac{C}{\delta\sqrt{n}}\log^2(T)\leq\frac{\varepsilon}{800\delta^2}. \tag*{(61)} E [ n 1 τ = 1 ∑ n L ( x τ ) − x ∈ K i min L ( x ) ] ≤ δ n C log 2 ( T ) ≤ 800 δ 2 ε . (61)
for sufficiently large C 0 C_0 C 0 in Equation (58) . Applying Markov’s inequality to this nonnegative excess loss gives
1 n ∑ τ = 1 n L ( x τ ) ≤ min x ∈ K i L ( x ) + ε 200 δ 2 \frac{1}{n}\sum_{\tau=1}^{n}L(x_\tau)\leq\min_{x\in\mathcal{K}_i}L(x)+\frac{\varepsilon}{200\delta^2} n 1 τ = 1 ∑ n L ( x τ ) ≤ x ∈ K i min L ( x ) + 200 δ 2 ε
with probability at least 3 / 4 3/4 3/4 . Call a copy satisfying this inequality good . Since the ⌈ C 0 log ( 2 T ) ⌉ \lceil C_0\log(2T)\rceil ⌈ C 0 log ( 2 T )⌉ copies are independent, the probability that none is good is at most
( 1 / 4 ) ⌈ C 0 log ( 2 T ) ⌉ ≤ ( 2 T ) − C 0 log 4 ≤ 1 2 T , (1/4)^{\lceil C_0\log(2T)\rceil}\leq(2T)^{-C_0\log4}\leq\frac{1}{2T}, ( 1/4 ) ⌈ C 0 l o g ( 2 T )⌉ ≤ ( 2 T ) − C 0 l o g 4 ≤ 2 T 1 ,
for sufficiently large universal C 0 C_0 C 0 .
We cannot compute the average true loss of a copy directly. Instead, we use its empirical average loss L ^ = n − 1 ∑ τ = 1 n Y τ \widehat{L}=n^{-1}\sum_{\tau=1}^{n}Y_\tau L = n − 1 ∑ τ = 1 n Y τ , computed from the observations collected during that run. Although each query x τ x_\tau x τ may depend on earlier observations, Equation (52) gives E [ Y τ − L ( x τ ) ∣ p a s t , x τ ] = 0 \mathbb{E}[Y_\tau-L(x_\tau)\mid\mathrm{past},x_\tau]=0 E [ Y τ − L ( x τ ) ∣ past , x τ ] = 0 and ∣ Y τ − L ( x τ ) ∣ ≤ C / δ \lvert Y_\tau-L(x_\tau)\rvert\leq C/\delta ∣ Y τ − L ( x τ )∣ ≤ C / δ , so that { ∑ τ ≤ n ( Y τ − L ( x τ ) ) } n \{\sum_{\tau\leq n}(Y_\tau-L(x_\tau))\}_{n} { ∑ τ ≤ n ( Y τ − L ( x τ )) } n is a martingale with bounded increments. Azuma-Hoeffding inequality (see Proposition A.4 ) and a union bound therefore give, simultaneously for all copies, for sufficiently large C 0 C_0 C 0 :
∣ L ^ − 1 n ∑ τ = 1 n L ( x τ ) ∣ ≤ ε 400 δ 2 , \left|\widehat{L}-\frac{1}{n}\sum_{\tau=1}^{n}L(x_\tau)\right|\leq\frac{\varepsilon}{400\delta^2}, L − n 1 τ = 1 ∑ n L ( x τ ) ≤ 400 δ 2 ε ,
except with probability 1 / ( 2 T ) 1/(2T) 1/ ( 2 T ) .
Assume both events hold, which happens with probability at least 1 − 1 / T 1-1/T 1 − 1/ T . There is a good copy, and its empirical average exceeds its true average by at most ε / ( 400 δ 2 ) \varepsilon/(400\delta^2) ε / ( 400 δ 2 ) . Consequently, its empirical average is at most
min x ∈ K i L ( x ) + ε 200 δ 2 + ε 400 δ 2 = min x ∈ K i L ( x ) + 3 ε 400 δ 2 . \min_{x\in\mathcal{K}_i}L(x)+\frac{\varepsilon}{200\delta^2}+\frac{\varepsilon}{400\delta^2}=\min_{x\in\mathcal{K}_i}L(x)+\frac{3\varepsilon}{400\delta^2}. x ∈ K i min L ( x ) + 200 δ 2 ε + 400 δ 2 ε = x ∈ K i min L ( x ) + 400 δ 2 3 ε .
As specified in the refinement procedure, select the copy with the smallest empirical average. From now on, x τ x_\tau x τ and L ^ \widehat{L} L refer to this selected copy. Hence L ^ ≤ min x ∈ K i L ( x ) + 3 ε / ( 400 δ 2 ) \widehat{L}\leq\min_{x\in\mathcal{K}_i}L(x)+3\varepsilon/(400\delta^2) L ≤ min x ∈ K i L ( x ) + 3 ε / ( 400 δ 2 ) .
Next we relate the selected copy’s empirical average to the loss at the returned point. Since the concentration bound holds for every copy, it also holds for the selected one, giving 1 n ∑ τ = 1 n L ( x τ ) ≤ L ^ + ε / ( 400 δ 2 ) \frac{1}{n}\sum_{\tau=1}^{n}L(x_\tau)\leq\widehat{L}+\varepsilon/(400\delta^2) n 1 ∑ τ = 1 n L ( x τ ) ≤ L + ε / ( 400 δ 2 ) . Recall that the returned point in Successive Opt is y = n − 1 ∑ τ = 1 n x τ y=n^{-1}\sum_{\tau=1}^{n}x_\tau y = n − 1 ∑ τ = 1 n x τ and belongs to K i \mathcal{K}_i K i by convexity of the region. Convexity of L L L then gives
L ( y ) ≤ 1 n ∑ τ = 1 n L ( x τ ) ≤ L ^ + ε 400 δ 2 . L(y)\leq\frac{1}{n}\sum_{\tau=1}^{n}L(x_\tau)\leq\widehat{L}+\frac{\varepsilon}{400\delta^2}. L ( y ) ≤ n 1 τ = 1 ∑ n L ( x τ ) ≤ L + 400 δ 2 ε .
Combining the two bounds on the selected empirical average yields
L ( y ) − ε 400 δ 2 ≤ L ^ ≤ min x ∈ K i L ( x ) + 3 ε 400 δ 2 . L(y)-\frac{\varepsilon}{400\delta^2}\leq\widehat{L}\leq\min_{x\in\mathcal{K}_i}L(x)+\frac{3\varepsilon}{400\delta^2}. L ( y ) − 400 δ 2 ε ≤ L ≤ x ∈ K i min L ( x ) + 400 δ 2 3 ε .
The map a ↦ 1 / max { a , 1 / ( 10 δ ) } a\mapsto1/\max\{a,1/(10\delta)\} a ↦ 1/ max { a , 1/ ( 10 δ )} is decreasing and 100 δ 2 100\delta^2 100 δ 2 -Lipschitz. Since true losses are at least 1 / ( 6 δ ) 1/(6\delta) 1/ ( 6 δ ) by Equation (54) (specifically, Lemma 6.2 and Lemma 6.4 ), applying this map on L ^ \widehat{L} L to define r ^ \widehat{r} r yields
max x ∈ K i r ( x ) − 3 ε / 4 ≤ r ^ ≤ r ( y ) + ε / 4. (62) \max_{x\in\mathcal{K}_i}r(x)-3\varepsilon/4\leq\widehat{r}\leq r(y)+\varepsilon/4. \tag*{(62)} x ∈ K i max r ( x ) − 3 ε /4 ≤ r ≤ r ( y ) + ε /4. (62)
This proves Equation (59) , with total failure probability at most 1 / T 1/T 1/ T .
For part (ii), assume inductively that i ⋆ i^\star i ⋆ is active at the start of phase j j j . For each active candidate i i i , let r ^ i \hat r_i r ^ i denote the estimate returned by refinement in this phase (i.e. the procedure described above), using the calibrated parameter ρ i ( j ) λ z i ( j ) \rho_i^{(j)}{}_{\lambda_{z_i^{(j)}}} ρ i ( j ) λ z i ( j ) ; similarly let r i r_i r i be r ρ i ( j ) λ z i ( j ) r_{\rho_i^{(j)}{}_{\lambda_{z_i^{(j)}}}} r ρ i ( j ) λ z i ( j ) . Here z i ( j ) z_i^{(j)} z i ( j ) denotes the reference at the start of phase j j j . Then Lemma 6.4 (i), the logarithmic calibration bound in Lemma 6.5(i), and the lower bound in Equation (59) give
r ^ i ⋆ ≥ μ ⋆ − 3 η − ε j . (63) \hat r_{i^\star} \ge\mu^\star- 3\eta- \varepsilon_j. \tag*{(63)} r ^ i ⋆ ≥ μ ⋆ − 3 η − ε j . (63)
Every candidate has r ^ i ≤ μ ⋆ + ε j \hat r_i \le\mu^\star+ \varepsilon_j r ^ i ≤ μ ⋆ + ε j by Equation (59) and Lemma 6.2 . Thus the largest estimate exceeds r ^ i ⋆ \hat r_{i^\star} r ^ i ⋆ by at most 3 η + 2 ε j 3\eta+ 2\varepsilon_j 3 η + 2 ε j , so the threshold 10 ( η + ε j ) 10(\eta+\varepsilon_j) 10 ( η + ε j ) preserves i ⋆ i^\star i ⋆ .
Now assume that i i i is retained. Since i i i is retained, then r ^ i ≥ r ^ i ⋆ − 10 ( η + ε j ) ≥ μ ⋆ − 3 η − ε j − 10 ( η + ε j ) \hat r_i \ge\hat r_{i^\star} - 10(\eta+\varepsilon_j) \ge\mu^\star- 3\eta- \varepsilon_j - 10(\eta+\varepsilon_j) r ^ i ≥ r ^ i ⋆ − 10 ( η + ε j ) ≥ μ ⋆ − 3 η − ε j − 10 ( η + ε j ) , where the second inequality follows from Equation (63) . Together with the refinement upper bound from Equation (59) , this yields, with the updated reference z i ( j + 1 ) = y i z_i^{(j+1)}=y_i z i ( j + 1 ) = y i :
μ ⋆ − r ρ i ( j ) λ z i ( j + 1 ) ( z i ( j + 1 ) ) ≤ μ ⋆ − r ^ i + ε j ≤ 13 η + 12 ε j ≤ 16 ( η + ε j ) . (64) \mu^\star-r_{\rho_i^{(j)}{}_{\lambda_{z_i^{(j+1)}}}}(z_i^{(j+1)}) \le\mu^\star-\hat r_i+\varepsilon_j \le13\eta+12\varepsilon_j \le16(\eta+\varepsilon_j). \tag*{(64)} μ ⋆ − r ρ i ( j ) λ z i ( j + 1 ) ( z i ( j + 1 ) ) ≤ μ ⋆ − r ^ i + ε j ≤ 13 η + 12 ε j ≤ 16 ( η + ε j ) . (64)
Finally, notice that this bound evaluates the updated reference — associated to iteration j + 1 j+1 j + 1 — using the auxiliary-profit function from phase j j j . We must transfer it to the parameter obtained by the next calibration. Let ρ i ( j + 1 ) \rho_i^{(j+1)} ρ i ( j + 1 ) be the output of that calibration, using z i ( j + 1 ) z_i^{(j+1)} z i ( j + 1 ) . Both valid calibrations approximate the same fixed intended parameter λ i \lambda_i λ i . Thus, by Equation (51) in Lemma 6.5, we get:
∣ log ρ i ( j + 1 ) λ z i ( j + 1 ) ρ i ( j ) λ z i ( j ) ∣ ≤ ∣ log ρ i ( j + 1 ) λ z i ( j + 1 ) λ i ∣ + ∣ log ρ i ( j ) λ z i ( j ) λ i ∣ ≤ 2 ⋅ 10 − 3 η / δ . \left|\log\frac{\rho_i^{(j+1)}{}_{\lambda_{z_i^{(j+1)}}}}{\rho_i^{(j)}{}_{\lambda_{z_i^{(j)}}}}\right|
\le
\left|\log\frac{\rho_i^{(j+1)}{}_{\lambda_{z_i^{(j+1)}}}}{\lambda_i}\right|
+
\left|\log\frac{\rho_i^{(j)}{}_{\lambda_{z_i^{(j)}}}}{\lambda_i}\right|
\le2\cdot10^{-3}\sqrt{\eta/\delta}. log ρ i ( j ) λ z i ( j ) ρ i ( j + 1 ) λ z i ( j + 1 ) ≤ log λ i ρ i ( j + 1 ) λ z i ( j + 1 ) + log λ i ρ i ( j ) λ z i ( j ) ≤ 2 ⋅ 1 0 − 3 η / δ .
The stability bound from Equation (40) in Lemma 6.2 , applied at the updated reference, therefore gives
μ ⋆ − r ρ i ( j + 1 ) λ z i ( j + 1 ) ( z i ( j + 1 ) ) ≤ 2 ( μ ⋆ − r ρ i ( j ) λ z i ( j + 1 ) ( z i ( j + 1 ) ) ) + 2 μ ⋆ ( 2 ⋅ 10 − 3 η / δ ) 2 ≤ 32 ( η + ε j ) + 8 ⋅ 10 − 6 μ ⋆ η δ (By Equation (62)) ≤ ( 32 + 48 ⋅ 10 − 6 ) η + 64 ε j + 1 ≤ 65 ( η + ε j + 1 ) , \begin{aligned}
\mu^\star-r_{\rho_i^{(j+1)}{}_{\lambda_{z_i^{(j+1)}}}}(z_i^{(j+1)})
&\le2\left(\mu^\star-r_{\rho_i^{(j)}{}_{\lambda_{z_i^{(j+1)}}}}(z_i^{(j+1)})\right)
+2\mu^\star\left(2\cdot10^{-3}\sqrt{\eta/\delta}\right)^2 \\
&\le32(\eta+\varepsilon_j)+8\cdot10^{-6}\mu^\star\frac{\eta}{\delta}
\qquad\text{(By Equation (62))}\\
&\le(32+48\cdot10^{-6})\eta+64\varepsilon_{j+1}\\
&\le65(\eta+\varepsilon_{j+1}),
\end{aligned} μ ⋆ − r ρ i ( j + 1 ) λ z i ( j + 1 ) ( z i ( j + 1 ) ) ≤ 2 ( μ ⋆ − r ρ i ( j ) λ z i ( j + 1 ) ( z i ( j + 1 ) ) ) + 2 μ ⋆ ( 2 ⋅ 1 0 − 3 η / δ ) 2 ≤ 32 ( η + ε j ) + 8 ⋅ 1 0 − 6 μ ⋆ δ η (By Equation (62)) ≤ ( 32 + 48 ⋅ 1 0 − 6 ) η + 64 ε j + 1 ≤ 65 ( η + ε j + 1 ) ,
using μ ⋆ ≤ 6 δ \mu^\star\le6\delta μ ⋆ ≤ 6 δ and ε j = 2 ε j + 1 \varepsilon_j=2\varepsilon_{j+1} ε j = 2 ε j + 1 .
Initially, all candidates are active, z i ( 0 ) = w i z_i^{(0)}=w_i z i ( 0 ) = w i , and ε 0 = δ \varepsilon_0=\delta ε 0 = δ . Using μ ⋆ ≤ 6 δ \mu^\star\le6\delta μ ⋆ ≤ 6 δ from Lemma 6.4 , we get:
μ ⋆ − r ρ i ( 0 ) λ w i ( w i ) ≤ μ ⋆ ≤ 6 δ ≤ 65 ( η + ε 0 ) , \mu^\star-r_{\rho_i^{(0)}{}_{\lambda_{w_i}}}(w_i)\le\mu^\star\le6\delta\le65(\eta+\varepsilon_0), μ ⋆ − r ρ i ( 0 ) λ w i ( w i ) ≤ μ ⋆ ≤ 6 δ ≤ 65 ( η + ε 0 ) ,
so this same bound holds at the start of phase 0. Induction then concludes the proof of (ii). □
Regret bound
We now combine the preceding lemmata to bound the regret from initialization, calibration, and refinement, accounting also for failed guarantees. Choosing η = T − 2 / 5 \eta=T^{-2/5} η = T − 2/5 then yields the bound in Theorem 6.1. In the following, let C C C be a sufficiently large universal constant which upper bounds all the other constants we have considered in the lemmata of this section.
Proof of Theorem 6.1. Fix η ∈ ( 0 , 1 ) \eta\in(0,1) η ∈ ( 0 , 1 ) with K = ⌈ 1 / η ⌉ ≤ T K=\lceil1/\eta\rceil\le T K = ⌈ 1/ η ⌉ ≤ T . Recall that we stop at round T T T and post only prices in T \mathcal{T} T , using ( 0 , 0 ) (0,0) ( 0 , 0 ) for the remaining rounds if an expression or price is invalid or no candidates remain. Thus every round has regret gap in [ 0 , 1 ] [0,1] [ 0 , 1 ] . Initialization. Analyze Initialize Regular as if it were allowed to run all iterations. Its actual execution may be a prefix of this full execution, as it stops at trading round T T T . By Lemma 6.4, initialization fails with probability at most 1 / T 1/T 1/ T . Charging the entire horizon on this event contributes at most T ⋅ ( 1 / T ) = 1 T \cdot(1/T) = 1 T ⋅ ( 1/ T ) = 1 to expected regret. If its eventual valid output is no trigger , then μ ⋆ ≤ C / K ≤ C η \mu^\star\le C/K \le C\eta μ ⋆ ≤ C / K ≤ C η , so the entire horizon costs at most C T η CT\eta C T η , even if initialization is interrupted before returning that output. If instead it returns a scale and candidates satisfying the guarantees of Lemma 6.4, its full regret is at most O ( K log 2 ( T ) ) O(K\log^2(T)) O ( K log 2 ( T )) . Non-negativity of the instantaneous regret gives the same bound for any prefix; we do not need initialization to actually finish before T T T .
It remains to consider the case of Initialize Regular triggering when initialization completes before T T T . We thus now proceed to the analysis of Successive Opt. Fix the completed transcript of Initialize Regular, so δ \delta δ , the candidates, and their number N N N are fixed, with
N ≤ O ( δ / η log ( T ) ) . (65) N \le O\left(\sqrt{\delta/\eta}\log(T)\right). \tag*{(65)} N ≤ O ( δ / η log ( T ) ) . (65)
All subsequent expectations and probabilities are conditional on this transcript. In the following, we call a calibration or refinement output valid when it satisfies Equations (49) and (57), respectively.
Regret during one optimizer run. We first count the costs of single optimizer calls that start with all preceding outputs valid, including the call producing the first failed output. We will bound the cost of all rounds after that first failure separately.
At the start of phase j j j , a valid calibration gives, via Lemma 6.6(ii):
μ ⋆ − r λ ( z i ) ≤ 65 ( η + ε j ) . \mu^\star-r_\lambda(z_i) \le65(\eta+\varepsilon_j). μ ⋆ − r λ ( z i ) ≤ 65 ( η + ε j ) .
Therefore, for a query x x x and every price pair y y y posted to produce its observation, substituting this bound into (53) from Lemma 6.5(ii) gives:
μ ⋆ − μ ( y ) ≤ 65 C η + 65 C ε j + C δ 2 ( L ( x ) − min a ∈ K i L ( a ) ) . (66) \mu^\star-\mu(y) \le65C\eta+65C\varepsilon_j+C\delta^2\left(L(x)-\min_{a\in\mathcal{K}_i}L(a)\right). \tag*{(66)} μ ⋆ − μ ( y ) ≤ 65 C η + 65 C ε j + C δ 2 ( L ( x ) − a ∈ K i min L ( a ) ) . (66)
Charge the 65 C η 65C\eta 65 C η term only over actual trading rounds, giving at most 65 C T η 65CT\eta 65 C T η in total. To bound the last two terms of (66) , allow each started optimizer run to finish all its n n n observations, including any observation interrupted at time T T T . Their contribution on the actual prefix is at most their contribution on this full run.
Conditional on the past history and query x x x , the other two terms in (66) are fixed, and producing the observation takes at most C / δ C/\delta C / δ expected trading rounds by (52) in Lemma 6.5(ii). Their expected contribution for this query is therefore at most
65 C 2 ε j δ + C 2 δ ( L ( x ) − min a ∈ K i L ( a ) ) . \frac{65C^2\varepsilon_j}{\delta}+C^2\delta\left(L(x)-\min_{a\in\mathcal{K}_i}L(a)\right). δ 65 C 2 ε j + C 2 δ ( L ( x ) − a ∈ K i min L ( a ) ) .
Summing over the n n n queries and applying (57) from the stochastic convex bandits regret bound in Corollary 3.1 bounds the expected contribution of these terms, conditional on the run’s starting history, by
65 C 2 n ε j δ + C 2 δ E [ ∑ τ = 1 n ( L ( x τ ) − min x ∈ K i L ( x ) ) ] ≤ 65 C 2 n ε j δ + O ( n log 2 ( 2 n ) ) . 65C^2\frac{n\varepsilon_j}{\delta}+C^2\delta\mathbb{E}\left[\sum_{\tau=1}^{n}\left(L(x_\tau)-\min_{x\in\mathcal{K}_i}L(x)\right)\right]\le65C^2\frac{n\varepsilon_j}{\delta}+O\left(\sqrt{n}\log^2(2n)\right). 65 C 2 δ n ε j + C 2 δ E [ τ = 1 ∑ n ( L ( x τ ) − x ∈ K i min L ( x ) ) ] ≤ 65 C 2 δ n ε j + O ( n log 2 ( 2 n ) ) .
The choice of n n n from (58) then gives n / 4 j ≤ [ C log 4 ( 2 T ) ] ≤ 2 C log 4 ( 2 T ) n/4^j \le[C\log^4(2T)] \le2C\log^4(2T) n / 4 j ≤ [ C log 4 ( 2 T )] ≤ 2 C log 4 ( 2 T ) . Since ε j = δ 2 − j \varepsilon_j=\delta2^{-j} ε j = δ 2 − j , this implies n ε j / δ = n n / 4 j ≤ 2 C n log 2 ( 2 T ) n\varepsilon_j/\delta=\sqrt{n}\sqrt{n/4^j}\le\sqrt{2C}\sqrt{n}\log^2(2T) n ε j / δ = n n / 4 j ≤ 2 C n log 2 ( 2 T ) . Thus the expected contribution of the last two terms of (66) over one run is at most
65 C 2 2 C n log 2 ( 2 T ) + O ( n log 2 ( 2 n ) ) . (67) 65C^2\sqrt{2C}\sqrt{n}\log^2(2T)+O\left(\sqrt{n}\log^2(2n)\right). \tag*{(67)} 65 C 2 2 C n log 2 ( 2 T ) + O ( n log 2 ( 2 n ) ) . (67)
Summing the optimizer runs. Next, we bound the sum of the numbers of prescribed observations over all optimizer runs started before trading round T T T . Notice that, after fixing a valid initialization transcript, the following counting argument holds for every realization of the subsequent execution, regardless of whether calibrations or refinements fail.
Every completed run uses at least n n n trading rounds, since each observation uses at least one round. Hence the prescribed lengths of completed runs sum to at most T T T . There is at most one unfinished run. If it is in phase j ≥ 1 j \ge1 j ≥ 1 , using Equation (58) , a run of length n / 4 n/4 n /4 must have completed in the preceding phase, so n ≤ 4 T n \le4T n ≤ 4 T . In phase 0, its length is at most 2 C log 4 ( 2 T ) 2C\log^4(2T) 2 C log 4 ( 2 T ) . Using log 4 ( 2 T ) ≤ 10 T \log^4(2T) \le10T log 4 ( 2 T ) ≤ 10 T , we then get, using n k n_k n k as the prescribed number of loss observations in optimizer run k k k :
∑ k is a started run n k ≤ T + 4 T + 2 C log 4 ( 2 T ) ≤ ( 5 + 20 C ) T ≤ 25 C T . (68) \sum_{\text{$k$ is a started run}} n_k \le T + 4T + 2C\log^4(2T) \le(5 + 20C)T \le25CT. \tag*{(68)} k is a started run ∑ n k ≤ T + 4 T + 2 C log 4 ( 2 T ) ≤ ( 5 + 20 C ) T ≤ 25 C T . (68)
In particular, n k ≤ 25 C T n_k \le25CT n k ≤ 25 C T for every started run, so log ( n k ) = O ( log ( T ) ) \log(n_k) = O(\log(T)) log ( n k ) = O ( log ( T )) , which can be used to bound the logarithm in Equation (67) . Choosing C C C sufficiently large, the expected contribution of the last two terms of Equation (66) over run k k k is at most
( 65 C 2 2 C + C 3 ) n k log 2 ( 2 T ) ≤ 100 C 3 n k log 2 ( 2 T ) . (65C^2\sqrt{2C}+C^3)\sqrt{n_k}\log^2(2T) \le100C^3\sqrt{n_k}\log^2(2T). ( 65 C 2 2 C + C 3 ) n k log 2 ( 2 T ) ≤ 100 C 3 n k log 2 ( 2 T ) .
Starting phase j ≥ 1 j \ge1 j ≥ 1 requires completing a run of at least 4 j − 1 4^{j-1} 4 j − 1 observations in the preceding phase. Hence 4 j − 1 ≤ T 4^{j-1} \le T 4 j − 1 ≤ T , so
number of started phases ≤ 2 + log 4 T ≤ 3 log ( T ) . (69) \text{number of started phases} \le2 + \log_4 T \le3\log(T). \tag*{(69)} number of started phases ≤ 2 + log 4 T ≤ 3 log ( T ) . (69)
Each refinement uses at most ⌈ C log ( 2 T ) ⌉ ≤ 2 C log ( T ) \lceil C\log(2T)\rceil\le2C\log(T) ⌈ C log ( 2 T )⌉ ≤ 2 C log ( T ) optimizer copies. With at most N N N active candidates per phase, there are therefore at most 6 C N log 2 ( T ) 6CN\log^2(T) 6 C N log 2 ( T ) started runs. Summing this single-run bound, Cauchy–Schwarz and the run-length bound Equation (68) give
∑ k is a started run 100 C 3 n k log 2 ( 2 T ) ≤ 100 C 3 log 2 ( 2 T ) ∑ started runs 1 ∑ started runs n k \sum_{\text{$k$ is a started run}} 100C^3\sqrt{n_k}\log^2(2T)
\le100C^3\log^2(2T)\sqrt{\sum_{\text{started runs}}1}\sqrt{\sum_{\text{started runs}}n_k} k is a started run ∑ 100 C 3 n k log 2 ( 2 T ) ≤ 100 C 3 log 2 ( 2 T ) started runs ∑ 1 started runs ∑ n k
≤ 100 C 3 log 2 ( 2 T ) 6 C N log 2 ( T ) 25 C T \le100C^3\log^2(2T)\sqrt{6CN\log^2(T)}\sqrt{25CT} ≤ 100 C 3 log 2 ( 2 T ) 6 C N log 2 ( T ) 25 C T
≤ 100 150 C 4 N T log 3 ( 2 T ) \le100\sqrt{150}\,C^4\sqrt{NT}\log^3(2T) ≤ 100 150 C 4 N T log 3 ( 2 T )
≤ 1300 C 4 N T log 3 ( 2 T ) . (70) \le1300C^4\sqrt{NT}\log^3(2T). \tag*{(70)} ≤ 1300 C 4 N T log 3 ( 2 T ) . (70)
With Equation (67) we have bounded the expected regret contribution of a single optimizer run, conditional on the run’s starting history, whenever preceding outputs are valid. We now want to drop the conditioning, while preserving the one on the Initialize regular transcript and while still restricting the expectation to runs started with valid preceding outputs. We can do this by using standard properties5 of conditional expectations (see Chapter 9 in Williams [43 ] ); we then get the following bound, with E k \mathcal{E}_k E k being the clean event that all preceding outputs are valid for run k k k and with R k R_k R k being the actual regret contribution of run k k k before time T T T :
E [ ∑ k is a started run R k 1 E k ] ≤ E [ ∑ k is a started run 100 C 3 log 2 ( 2 T ) n k 1 E k + 65 C T η ] \mathbb{E}\left[\sum_{\text{$k$ is a started run}} R_k1_{\mathcal{E}_k}\right]
\le\mathbb{E}\left[\sum_{\text{$k$ is a started run}} 100C^3\log^2(2T)\sqrt{n_k}1_{\mathcal{E}_k}+65CT\eta\right] E k is a started run ∑ R k 1 E k ≤ E k is a started run ∑ 100 C 3 log 2 ( 2 T ) n k 1 E k + 65 C T η
≤ 1300 C 4 N T log 3 ( 2 T ) + 65 C T η , \le1300C^4\sqrt{NT}\log^3(2T)+65CT\eta, ≤ 1300 C 4 N T log 3 ( 2 T ) + 65 C T η ,
where we considered the extra term from Equation (66) and where we used that the counting argument holds for any realization.
Now, every trading round used by refinement before the first failed output or the end of the time horizon belongs to an optimizer run that started with all preceding outputs valid. The fact that regret is non-negative then immediately gives:
E [ refinement regret up to first failure or time T ] ≤ E [ ∑ k is a started run R k 1 E k ] ≤ 1300 C 4 N T log 3 ( 2 T ) + 65 C T η . (71) \mathbb{E}\left[\text{refinement regret up to first failure or time }T\right] \le\mathbb{E}\left[\sum_{k\text{ is a started run}} R_{k1}\mathcal{E}_k\right]
\le1300C^{4}\sqrt{NT}\log^{3}(2T)+65CT\eta. \tag*{(71)} E [ refinement regret up to first failure or time T ] ≤ E [ k is a started run ∑ R k 1 E k ] ≤ 1300 C 4 N T log 3 ( 2 T ) + 65 C T η . (71)
Calibration and failures. By Equation (69) , there are at most 3 N log ( T ) 3N\log(T) 3 N log ( T ) calibration calls, since each phase calibrates at most N N N candidates. Lemma 6.5(i) bounds each call’s regret by C log ( T ) / η C\log(T)/\eta C log ( T ) / η , including any interrupted prefix. Thus their total regret up to the first failed output or time T T T is at most
3 N log ( T ) ⋅ C log ( T ) η = 3 C N log 2 ( T ) η . (72) 3N\log(T)\cdot\frac{C\log(T)}{\eta}=3CN\frac{\log^{2}(T)}{\eta}. \tag*{(72)} 3 N log ( T ) ⋅ η C log ( T ) = 3 C N η log 2 ( T ) . (72)
There are also at most 3 N log ( 2 T ) 3N\log(2T) 3 N log ( 2 T ) refinement calls, hence at most 6 N log ( 2 T ) 6N\log(2T) 6 N log ( 2 T ) calibration and refinement calls combined. For each calibration or refinement started with valid preceding outputs, the conditional probability that its output fails is at most 1 / T 1/T 1/ T , by Lemmata 6.5 and 6.6. Taking the union over which call produces the first failed output therefore gives:
P ( a completed calibration or refinement fails ) ≤ 6 N log ( 2 T ) T . (73) \mathbb{P}\left(\text{a completed calibration or refinement fails}\right)\le\frac{6N\log(2T)}{T}. \tag*{(73)} P ( a completed calibration or refinement fails ) ≤ T 6 N log ( 2 T ) . (73)
After the first failed output, charge every remaining trading round at the worst-case gap of one. Thus by Equation (73) , the additional expected regret is at most
T ⋅ 6 N log ( 2 T ) T = 6 N log ( 2 T ) . (74) T\cdot\frac{6N\log(2T)}{T}=6N\log(2T). \tag*{(74)} T ⋅ T 6 N log ( 2 T ) = 6 N log ( 2 T ) . (74)
Combining the bounds. Combining the initialization regret, Equation (71) , Equation (72) and Equation (74) on any completed valid triggered initialization transcript, the total expected regret is at most
O ( K log 2 ( T ) ) + O ( N log 2 ( T ) η ) + O ( N T log 3 ( T ) ) + O ( T η ) + O ( N log ( T ) ) . (75) O(K\log^{2}(T))+O\left(N\frac{\log^{2}(T)}{\eta}\right)+O(\sqrt{NT}\log^{3}(T))+O(T\eta)+O(N\log(T)). \tag*{(75)} O ( K log 2 ( T )) + O ( N η log 2 ( T ) ) + O ( N T log 3 ( T )) + O ( T η ) + O ( N log ( T )) . (75)
Choose η = T − 2 / 5 \eta=T^{-2/5} η = T − 2/5 , implying K = ⌈ T 2 / 5 ⌉ K=\lceil T^{2/5}\rceil K = ⌈ T 2/5 ⌉ . Equation (65) then gives N = O ( T 1 / 5 log ( T ) ) N=O(T^{1/5}\log(T)) N = O ( T 1/5 log ( T )) uniformly over all valid triggered transcripts. The terms in (75) therefore satisfy:
O ( T 2 / 5 log 2 ( T ) ) + O ( T 3 / 5 log 3 ( T ) ) + O ( T 3 / 5 log 7 / 2 ( T ) ) + O ( T 3 / 5 ) + O ( T 1 / 5 log 2 T ) ≤ O ( T 3 / 5 log 7 / 2 ( T ) ) , O(T^{2/5}\log^{2}(T))+O(T^{3/5}\log^{3}(T))+O(T^{3/5}\log^{7/2}(T))+O(T^{3/5})+O(T^{1/5}\log^{2}T)\le O(T^{3/5}\log^{7/2}(T)), O ( T 2/5 log 2 ( T )) + O ( T 3/5 log 3 ( T )) + O ( T 3/5 log 7/2 ( T )) + O ( T 3/5 ) + O ( T 1/5 log 2 T ) ≤ O ( T 3/5 log 7/2 ( T )) ,
which proves the regret bound.
Separation Between MHR and Regular Distributions
In this section, we prove a separation between MHR and regular distributions in bilateral trade. More precisely, we prove the following result.
Theorem 7.1. There exists a universal constant c > 0 c>0 c > 0 such that, for every sufficiently large horizon T T T and every learning algorithm A \mathcal{A} A , there exists a product distribution D \mathcal{D} D with regular seller and buyer marginals satisfying
R T ( A , D ) ≥ c T 3 / 5 . R_T(\mathcal{A},\mathcal{D})\ge cT^{3/5}. R T ( A , D ) ≥ c T 3/5 .
Proof. We prove the result for an even stronger class of algorithms, i.e., algorithms that receive two bits of information at each step, indicating separately whether the seller and the buyer accepted the corresponding prices. Since we are proving a lower bound for the stochastic setting, where the agents’ valuations are drawn i.i.d. from the same fixed distribution, it is enough to construct a mixture of i.i.d. instances which is hard for any deterministic algorithm. Indeed, the hardness then extends to any (possibly randomized) algorithm by conditioning on any realization of the algorithm and averaging, so that the algorithm has to fail against at least one of the instances in the mixture.
We first define a baseline distribution D 0 \mathcal{D}_0 D 0 (for simplicity, we keep the same name for the random variables, but change the underlying measure, so that with E 0 \mathbb{E}^{0} E 0 we mean the expectation when ( S , B ) (S,B) ( S , B ) is sampled according to D 0 \mathcal{D}_0 D 0 ). Under D 0 \mathcal{D}_0 D 0 , the seller distribution is uniform in [ 0 , 1 ] [0,1] [ 0 , 1 ] and is thus regular and MHR. The buyer distribution is instead defined by the following (differentiable) survival function S 0 ( b ) = 1 − F 0 ( b ) S_0(b)=1-F_0(b) S 0 ( b ) = 1 − F 0 ( b ) , which induces the virtual value function ϕ 0 ( b ) \phi_0(b) ϕ 0 ( b ) as follows:
S 0 ( b ) = { 1 − 8 3 b , b ∈ [ 0 , 1 / 4 ] 1 48 b 2 , b ∈ [ 1 / 4 , 1 / 2 ] 1 − b 12 b , b ∈ [ 1 / 2 , 1 ] , ϕ 0 ( b ) = b − S 0 ( b ) − S 0 ′ ( b ) = { 2 b − 3 8 , b ∈ [ 0 , 1 / 4 ] b 2 , b ∈ [ 1 / 4 , 1 / 2 ] b 2 , b ∈ [ 1 / 2 , 1 ] . (76) \begin{aligned}
S_0(b) &=
\begin{cases}
1-\frac{8}{3}b, & b\in[0,1/4]\\
\frac{1}{48b^2}, & b\in[1/4,1/2]\\
\frac{1-b}{12b}, & b\in[1/2,1],
\end{cases}
&
\phi_0(b)=b-\frac{S_0(b)}{-S_0'(b)} &=
\begin{cases}
2b-\frac{3}{8}, & b\in[0,1/4]\\
\frac{b}{2}, & b\in[1/4,1/2]\\
b^2, & b\in[1/2,1].
\end{cases} \tag*{(76)}
\end{aligned} S 0 ( b ) = ⎩ ⎨ ⎧ 1 − 3 8 b , 48 b 2 1 , 12 b 1 − b , b ∈ [ 0 , 1/4 ] b ∈ [ 1/4 , 1/2 ] b ∈ [ 1/2 , 1 ] , ϕ 0 ( b ) = b − − S 0 ′ ( b ) S 0 ( b ) = ⎩ ⎨ ⎧ 2 b − 8 3 , 2 b , b 2 , b ∈ [ 0 , 1/4 ] b ∈ [ 1/4 , 1/2 ] b ∈ [ 1/2 , 1 ] . (76)
The virtual valuation is increasing over the domain, and the distribution is thus regular (but not MHR). We now want to construct a family of distributions D j \mathcal{D}_j D j which coincide with D 0 \mathcal{D}_0 D 0 on all the support, with the exception of the values of b b b in certain intervals. To this end, we introduce the following function ρ ( u ) = 16 u 2 ( 1 − u ) 2 ⋅ 1 { u ∈ [ 0 , 1 ] } \rho(u)=16u^2(1-u)^2\cdot\mathbf{1}_{\{u\in[0,1]\}} ρ ( u ) = 16 u 2 ( 1 − u ) 2 ⋅ 1 { u ∈ [ 0 , 1 ]} . Its properties are summarized in the following, immediate, claim.
Claim 7.2. The function ρ \rho ρ is twice continuously differentiable in ( 0 , 1 ) (0,1) ( 0 , 1 ) , has unique maximum 1 1 1 at u = 1 / 2 u=1/2 u = 1/2 and its first derivative vanishes at 0 0 0 and 1 1 1 . Moreover, ∥ ρ ′ ∥ ∞ ≤ 4 \lVert\rho'\rVert_{\infty}\leq4 ∥ ρ ′ ∥ ∞ ≤ 4 , ∥ ρ ′ ′ ∥ ∞ ≤ 32 \lVert\rho''\rVert_{\infty}\leq32 ∥ ρ ′′ ∥ ∞ ≤ 32 .
Fix an integer parameter K K K we set later, and partition uniformly the interval [ 1 / 4 , 1 / 2 ] [1/4,1/2] [ 1/4 , 1/2 ] into K K K sub-intervals I 1 , I 2 , … , I K I_1,I_2,\ldots,I_K I 1 , I 2 , … , I K . We would like to add small local “bumps” to the baseline buyer distribution, one for each interval in { I j } j = 1 K \{I_j\}_{j=1}^K { I j } j = 1 K . Specifically, let Y 0 ( b ) = 1 / S 0 ( b ) Y_0(b)=1/S_0(b) Y 0 ( b ) = 1/ S 0 ( b ) for any b ∈ [ 0 , 1 ) b\in[0,1) b ∈ [ 0 , 1 ) and w = 1 / 4 K w=1/4K w = 1/4 K ; for any j j j , define Y j Y_j Y j as follows:
Y j ( b ) = Y 0 ( b ) − 1 2 w 2 ρ ( b − a j w ) , (77) Y_j(b)=Y_0(b)-\frac{1}{2}w^2\rho\left(\frac{b-a_j}{w}\right), \tag*{(77)} Y j ( b ) = Y 0 ( b ) − 2 1 w 2 ρ ( w b − a j ) , (77)
where a j a_j a j is the left-endpoint of I j I_j I j , and b ∈ [ 0 , 1 ) b\in[0,1) b ∈ [ 0 , 1 ) .
We have all the ingredients to define the distributions D j \mathcal{D}_j D j for j ∈ [ K ] j\in[K] j ∈ [ K ] . The seller distribution is uniform in [ 0 , 1 ] [0,1] [ 0 , 1 ] , while the buyer distribution is characterized by the survival function S j ( b ) S_j(b) S j ( b ) , defined as S j ( b ) = S 0 ( b ) S_j(b)=S_0(b) S j ( b ) = S 0 ( b ) for b ∉ I j b\notin I_j b ∈ / I j and S j ( b ) = 1 / Y j ( b ) S_j(b)=1/Y_j(b) S j ( b ) = 1/ Y j ( b ) for b ∈ I j b\in I_j b ∈ I j .
Before continuing, we show that the buyer marginal for any perturbation is regular.
Claim 7.3. The buyer distribution under D j \mathcal{D}_j D j is well-defined and regular.
Proof. To prove that the distribution is well-defined, it is enough to prove that S j S_j S j is non-negative, continuous, and decreasing in I j I_j I j . For the latter, it suffices proving that Y j ′ Y_j' Y j ′ is strictly positive. Since I j ⊆ [ 1 / 4 , 1 / 2 ] I_j\subseteq[1/4,1/2] I j ⊆ [ 1/4 , 1/2 ] , it follows by Equation (76) that Y 0 ′ ( b ) ≥ 24 Y_0'(b)\geq24 Y 0 ′ ( b ) ≥ 24 for any b ∈ I j b\in I_j b ∈ I j ; together with d ρ ( b − a j / w ) d b ≤ 1 / w ⋅ ∥ ρ ′ ∥ ∞ \frac{d\rho(b-a_j/w)}{db}\leq1/w\cdot\lVert\rho'\rVert_{\infty} d b d ρ ( b − a j / w ) ≤ 1/ w ⋅ ∥ ρ ′ ∥ ∞ , this suffices to show that S j S_j S j is decreasing, recalling that ∥ ρ ′ ∥ ∞ ≤ 12 \lVert\rho'\rVert_{\infty}\leq12 ∥ ρ ′ ∥ ∞ ≤ 12 by Claim 7.2 and noticing that w = 1 / 4 K ≤ 1 w=1/4K\leq1 w = 1/4 K ≤ 1 . Non-negativity then follows simply by continuity of S j S_j S j .
For regularity, we only need to show that ϕ j ′ ≥ 0 \phi_j'\geq0 ϕ j ′ ≥ 0 in the interior of I j I_j I j and that ϕ j \phi_j ϕ j converges to ϕ 0 \phi_0 ϕ 0 at the boundaries of I j I_j I j . To prove the first sufficient condition, notice that differentiating yields:
ϕ j ′ ( b ) = Y j ( b ) Y j ′ ′ ( b ) ( Y j ′ ( b ) ) 2 , \phi_j'(b)=\frac{Y_j(b)Y_j''(b)}{(Y_j'(b))^2}, ϕ j ′ ( b ) = ( Y j ′ ( b ) ) 2 Y j ( b ) Y j ′′ ( b ) ,
for any b b b in the interior of I j I_j I j , so that proving non-negativity of Y j ′ ′ Y_j'' Y j ′′ suffices. This can be shown analogously to the previous lower bound on Y j ′ Y_j' Y j ′ , using that d 2 ρ ( b − a j / w ) d b 2 ≤ 1 / w 2 ⋅ ∥ ρ ′ ′ ∥ ∞ \frac{d^2\rho(b-a_j/w)}{db^2}\leq1/w^2\cdot\lVert\rho''\rVert_{\infty} d b 2 d 2 ρ ( b − a j / w ) ≤ 1/ w 2 ⋅ ∥ ρ ′′ ∥ ∞ and that ∥ ρ ′ ′ ∥ ∞ ≤ 32 \lVert\rho''\rVert_{\infty}\leq32 ∥ ρ ′′ ∥ ∞ ≤ 32 . To prove continuity, notice that we may write ϕ j ( b ) = b − Y j ( b ) / Y j ′ ( b ) \phi_j(b)=b-Y_j(b)/Y_j'(b) ϕ j ( b ) = b − Y j ( b ) / Y j ′ ( b ) : focus can thus be placed on the difference ∣ Y 0 ( b ) / Y 0 ′ ( b ) − Y j ( b ) / Y j ′ ( b ) ∣ \lvert Y_0(b)/Y_0'(b)-Y_j(b)/Y_j'(b)\rvert ∣ Y 0 ( b ) / Y 0 ′ ( b ) − Y j ( b ) / Y j ′ ( b )∣ , which vanishes since ρ \rho ρ and ρ ′ \rho' ρ ′ vanish approaching 0 0 0 or 1 1 1 , by Claim 7.2. □ \square □
Now that we have introduced the baseline distribution D 0 \mathcal{D}_0 D 0 and the family of distributions D j \mathcal{D}_j D j parameterized for j = 1 , … , K j=1,\ldots,K j = 1 , … , K , we can study the corresponding expected profit:
E j [ Profit ( p , q ) ] = ( q − p ) p S j ( q ) . \mathbb{E}^{j}[\operatorname{Profit}(p,q)] = (q-p)pS_j(q). E j [ Profit ( p , q )] = ( q − p ) p S j ( q ) .
From this formula, it is easy to see that for any buyer price q q q , the best seller price comes from setting p = q / 2 p=q/2 p = q /2 , for an expected profit of h j ( q ) = q 2 / 4 ⋅ S j ( q ) h^j(q)=q^2/4\cdot S_j(q) h j ( q ) = q 2 /4 ⋅ S j ( q ) . This observation implies that the underlying optimization problem of finding the best posted prices boils down to finding the best q q q for h j h^j h j , for the considered instances. Moreover, the generic h j h^j h j functions only differ from h 0 h^0 h 0 in the corresponding interval I j I_j I j . If we introduce the suboptimality gap Δ = w 2 / ( 384 ⋅ 12 ) \Delta=w^2/(384\cdot12) Δ = w 2 / ( 384 ⋅ 12 ) , we can get the following result.
Claim 3.4. For any j = 1 , … , K j=1,\ldots,K j = 1 , … , K , it holds that:
The function h 0 h^0 h 0 has a plateau of maximizers in [ 1 / 4 , 1 / 2 ] [1/4,1/2] [ 1/4 , 1/2 ] , with maximum value 1 / 192 1/192 1/192 .
Letting q j ∗ q_j^* q j ∗ be the midpoint of I j I_j I j , we have h j ( q j ∗ ) − max q ∉ I j h j ( q ) ≥ Δ h^j(q_j^*)-\max_{q\notin I_j}h^j(q)\geq\Delta h j ( q j ∗ ) − max q ∈ / I j h j ( q ) ≥ Δ .
Proof. Point (i) follows directly from Equation (76) . Indeed, we have:
h 0 ( q ) = { q 2 4 ( 1 − 8 3 q ) , q ∈ [ 0 , 1 / 4 ] 1 192 , q ∈ [ 1 / 4 , 1 / 2 ] q ( 1 − q ) 48 , q ∈ [ 1 / 2 , 1 ] h^0(q)=
\begin{cases}
\frac{q^2}{4}\left(1-\frac{8}{3}q\right), & q\in[0,1/4] \\
\frac{1}{192}, & q\in[1/4,1/2] \\
\frac{q(1-q)}{48}, & q\in[1/2,1]
\end{cases} h 0 ( q ) = ⎩ ⎨ ⎧ 4 q 2 ( 1 − 3 8 q ) , 192 1 , 48 q ( 1 − q ) , q ∈ [ 0 , 1/4 ] q ∈ [ 1/4 , 1/2 ] q ∈ [ 1/2 , 1 ]
Differentiating, one easily checks that the first branch is indeed increasing and the last is indeed decreasing.
To prove point (ii), notice that by Claim 7.2 , we have ρ ( ( q j ∗ − a j ) / w ) = 1 \rho((q_j^*-a_j)/w)=1 ρ (( q j ∗ − a j ) / w ) = 1 . Therefore, Y j ( q j ∗ ) = 48 ( q j ∗ ) 2 − w 2 / 2 Y_j(q_j^*)=48(q_j^*)^2-w^2/2 Y j ( q j ∗ ) = 48 ( q j ∗ ) 2 − w 2 /2 . Using the definition of S j S_j S j for j ∈ [ K ] j\in[K] j ∈ [ K ] , we immediately get:
h j ( q j ∗ ) − max q ∉ I j h j ( q ) ≥ h j ( q j ∗ ) − 1 192 = ( q j ∗ ) 2 4 ⋅ ( 48 ( q j ∗ ) 2 − w 2 / 2 ) − 1 192 = w 2 384 ⋅ ( 48 ( q j ∗ ) 2 − w 2 / 2 ) ≥ w 2 384 ⋅ 12 . (78) h^j(q_j^*)-\max_{q\notin I_j}h^j(q)\geq h^j(q_j^*)-\frac{1}{192}=\frac{(q_j^*)^2}{4\cdot(48(q_j^*)^2-w^2/2)}-\frac{1}{192}=\frac{w^2}{384\cdot(48(q_j^*)^2-w^2/2)}\geq\frac{w^2}{384\cdot12}. \tag*{(78)} h j ( q j ∗ ) − q ∈ / I j max h j ( q ) ≥ h j ( q j ∗ ) − 192 1 = 4 ⋅ ( 48 ( q j ∗ ) 2 − w 2 /2 ) ( q j ∗ ) 2 − 192 1 = 384 ⋅ ( 48 ( q j ∗ ) 2 − w 2 /2 ) w 2 ≥ 384 ⋅ 12 w 2 . (78)
using that q j ∗ ≤ 1 / 2 q_j^*\leq1/2 q j ∗ ≤ 1/2 since I j ⊆ [ 1 / 4 , 1 / 2 ] I_j\subseteq[1/4,1/2] I j ⊆ [ 1/4 , 1/2 ] .
Fix any deterministic algorithm A \mathcal{A} A , we prove that it fails against the uniform mixture of { D j } j = 1 K \{\mathcal{D}_j\}_{j=1}^K { D j } j = 1 K . To this end, denote with N j N_j N j the random variable counting the number of times that A \mathcal{A} A posts a buyer price in I j I_j I j . The crucial ingredient to analyze the regret of A \mathcal{A} A onto the mixture resides in arguing that the algorithm manages to differentiate between the generic D j \mathcal{D}_j D j and the baseline D 0 \mathcal{D}_0 D 0 only by posting buyer prices in the “right” interval I j I_j I j . Formally, we have:
Claim 3.6. We have the following inequality: E j [ N j ] − E 0 [ N j ] ≤ T 22 K 2 E 0 [ N j ] \mathbb{E}^{j}[N_j]-\mathbb{E}^{0}[N_j]\leq\frac{T}{22K^2}\sqrt{\mathbb{E}^{0}[N_j]} E j [ N j ] − E 0 [ N j ] ≤ 22 K 2 T E 0 [ N j ] .
Proof of Claim 3.6. We denote with z t z_t z t the feedback received by the algorithm at time t t t , and with P z 1 , … , z t − 1 j \mathbb{P}^{j}_{z_1,\ldots,z_{t-1}} P z 1 , … , z t − 1 j the distribution on the feedback random variables up to time t − 1 t-1 t − 1 , induced by the deterministic algorithm and the valuation random variables up to time t − 1 t-1 t − 1 , under the distribution D j \mathcal{D}_j D j . To be clear, P z 1 , … , z t − 1 j \mathbb{P}^{j}_{z_1,\ldots,z_{t-1}} P z 1 , … , z t − 1 j is a push-forward measure on { 0 , 1 } 2 t − 1 \{0,1\}^{2^{t-1}} { 0 , 1 } 2 t − 1 . Moreover, notice that, with q t q_t q t being the posted buyer price at time t t t , 1 { q t ∈ I j } \mathbb{1}_{\{q_t\in I_j\}} 1 { q t ∈ I j } is a measurable function of { z t } t = 1 T \{z_t\}_{t=1}^T { z t } t = 1 T , since the algorithm is deterministic. We then have the following chain of inequalities:
E j [ N j ] − E 0 [ N j ] = ∑ t = 2 T P z 1 , … , z t − 1 j ( q t ∈ I j ) − P z 1 , … , z t − 1 0 ( q t ∈ I j ) (By linearity) ≤ ∑ t = 2 T ∥ P z 1 , … , z t − 1 j − P z 1 , … , z t − 1 0 ∥ T V (By definition of Total Variation) ≤ ∑ t = 2 T 1 2 D K L ( P z 1 , … , z t − 1 0 ∥ P z 1 , … , z t − 1 j ) (By Pinsker Inequality) = ∑ t = 2 T 1 2 D K L ( P z 1 0 ∥ P z 1 j ) + ∑ τ = 2 t − 1 E 0 [ D K L ( P z τ ∣ z 1 , … , z τ − 1 0 ∥ P z τ ∣ z 1 , … , z τ − 1 j ) ] \begin{aligned}
\mathbb{E}^{j}[N_j]-\mathbb{E}^{0}[N_j]
=\sum_{t=2}^{T}\mathbb{P}^{j}_{z_1,\ldots,z_{t-1}}(q_t\in I_j)-\mathbb{P}^{0}_{z_1,\ldots,z_{t-1}}(q_t\in I_j)
\qquad\text{(By linearity)} \\
&\leq\sum_{t=2}^{T}\left\|\mathbb{P}^{j}_{z_1,\ldots,z_{t-1}}-\mathbb{P}^{0}_{z_1,\ldots,z_{t-1}}\right\|_{\mathrm{TV}}
\qquad\text{(By definition of Total Variation)} \\
&\leq\sum_{t=2}^{T}\sqrt{\frac{1}{2}D_{\mathrm{KL}}\left(\mathbb{P}^{0}_{z_1,\ldots,z_{t-1}}\middle\|\mathbb{P}^{j}_{z_1,\ldots,z_{t-1}}\right)}
\qquad\text{(By Pinsker Inequality)} \\
&= \sum_{t=2}^{T} \frac{1}{\sqrt{2}}\sqrt{D_{\mathrm{KL}}\left(\mathbb{P}^{0}_{z_1}\mathbin{\|}\mathbb{P}^{j}_{z_1}\right)+\sum_{\tau=2}^{t-1}\mathbb{E}^{0}\left[D_{\mathrm{KL}}\left(\mathbb{P}^{0}_{z_\tau\mid z_1,\ldots,z_{\tau-1}}\mathbin{\|}\mathbb{P}^{j}_{z_\tau\mid z_1,\ldots,z_{\tau-1}}\right)\right]}
\end{aligned} E j [ N j ] − E 0 [ N j ] = t = 2 ∑ T P z 1 , … , z t − 1 j ( q t ∈ I j ) − P z 1 , … , z t − 1 0 ( q t ∈ I j ) (By linearity) ≤ t = 2 ∑ T P z 1 , … , z t − 1 j − P z 1 , … , z t − 1 0 TV (By definition of Total Variation) ≤ t = 2 ∑ T 2 1 D KL ( P z 1 , … , z t − 1 0 P z 1 , … , z t − 1 j ) (By Pinsker Inequality) = t = 2 ∑ T 2 1 D KL ( P z 1 0 ∥ P z 1 j ) + τ = 2 ∑ t − 1 E 0 [ D KL ( P z τ ∣ z 1 , … , z τ − 1 0 ∥ P z τ ∣ z 1 , … , z τ − 1 j ) ]
(Chain Rule of KL Divergence)
We focus on the single divergence terms, starting from the first one. The algorithm observes some difference in the distribution of the buyer only if it posts q 1 q_1 q 1 in the interval I j I_j I j . The distribution of the seller feedback is instead invariant, since the seller distribution is uniform under both D 0 \mathcal{D}_0 D 0 and D j \mathcal{D}_j D j . We thus have:
D K L ( P z 1 0 ∥ P z 1 j ) = 1 { q 1 ∈ I j } ⋅ D K L ( Bern ( S 0 ( q 1 ) ) ∥ Bern ( S j ( q 1 ) ) ) ≤ 1 { q 1 ∈ I j } ( S 0 ( q 1 ) − S j ( q 1 ) ) 2 S j ( q 1 ) ( 1 − S j ( q 1 ) ) . (79) \begin{aligned}
D_{\mathrm{KL}}\left(\mathbb{P}^{0}_{z_1}\mathbin{\|}\mathbb{P}^{j}_{z_1}\right)=\mathbf{1}_{\{q_1\in I_j\}}\cdot D_{\mathrm{KL}}\left(\operatorname{Bern}(S_0(q_1))\mathbin{\|}\operatorname{Bern}(S_j(q_1))\right) \\
&\leq\mathbf{1}_{\{q_1\in I_j\}}\frac{(S_0(q_1)-S_j(q_1))^2}{S_j(q_1)(1-S_j(q_1))}. \tag*{(79)}
\end{aligned} D KL ( P z 1 0 ∥ P z 1 j ) = 1 { q 1 ∈ I j } ⋅ D KL ( Bern ( S 0 ( q 1 )) ∥ Bern ( S j ( q 1 )) ) ≤ 1 { q 1 ∈ I j } S j ( q 1 ) ( 1 − S j ( q 1 )) ( S 0 ( q 1 ) − S j ( q 1 ) ) 2 . (79)
where the last inequality follows by the fact that log x ≤ x − 1 \log x\leq x-1 log x ≤ x − 1 , see Proposition A.1 in Appendix § 10 .§ 1 for a formal proof.
We now proceed to bound the above expression by upper and lower bounding Y 0 Y_0 Y 0 and Y j Y_j Y j . For the former, since I j ⊆ [ 1 / 4 , 1 / 2 ] I_j\subseteq[1/4,1/2] I j ⊆ [ 1/4 , 1/2 ] , Equation (76) implies 3 ≤ Y 0 ( q ) ≤ 12 3\leq Y_0(q)\leq12 3 ≤ Y 0 ( q ) ≤ 12 for any q ∈ I j q\in I_j q ∈ I j . For the latter, ∥ ρ ∥ ∞ ≤ 1 \lVert\rho\rVert_\infty\leq1 ∥ ρ ∥ ∞ ≤ 1 by Claim 7.2 and w 2 ≤ 1 w^2\leq1 w 2 ≤ 1 imply that 2 ≤ Y j ( q ) ≤ 12 2\leq Y_j(q)\leq12 2 ≤ Y j ( q ) ≤ 12 . Taking reciprocals, we get S j ( q 1 ) ∈ [ 1 / 12 , 1 / 2 ] S_j(q_1)\in[1/12,1/2] S j ( q 1 ) ∈ [ 1/12 , 1/2 ] , so that S j ( q 1 ) ( 1 − S j ( q 1 ) ) ≥ 1 / 24 S_j(q_1)(1-S_j(q_1))\geq1/24 S j ( q 1 ) ( 1 − S j ( q 1 )) ≥ 1/24 . Moreover, using the same bounds, we also get:
∣ S 0 ( q 1 ) − S j ( q 1 ) ∣ = ∣ 1 Y 0 ( q 1 ) − 1 Y j ( q 1 ) ∣ = ∣ Y j ( q 1 ) − Y 0 ( q 1 ) ∣ Y 0 ( q 1 ) Y j ( q 1 ) ≤ w 2 6 . \left|S_0(q_1)-S_j(q_1)\right|=\left|\frac{1}{Y_0(q_1)}-\frac{1}{Y_j(q_1)}\right|=\frac{\left|Y_j(q_1)-Y_0(q_1)\right|}{Y_0(q_1)Y_j(q_1)}\leq\frac{w^2}{6}. ∣ S 0 ( q 1 ) − S j ( q 1 ) ∣ = Y 0 ( q 1 ) 1 − Y j ( q 1 ) 1 = Y 0 ( q 1 ) Y j ( q 1 ) ∣ Y j ( q 1 ) − Y 0 ( q 1 ) ∣ ≤ 6 w 2 .
Plugging the above derivation into Equation (79) , we get:
D K L ( P z 1 0 ∥ P z 1 j ) ≤ w 4 1 { q 1 ∈ I j } = ( 1 4 K ) 4 1 { q 1 ∈ I j } . D_{\mathrm{KL}}\left(\mathbb{P}^{0}_{z_1}\mathbin{\|}\mathbb{P}^{j}_{z_1}\right)\leq w^4\mathbf{1}_{\{q_1\in I_j\}}=\left(\frac{1}{4K}\right)^4\mathbf{1}_{\{q_1\in I_j\}}. D KL ( P z 1 0 ∥ P z 1 j ) ≤ w 4 1 { q 1 ∈ I j } = ( 4 K 1 ) 4 1 { q 1 ∈ I j } .
An identical argument holds also for the generic term, so that we have:
E j [ N j ] − E 0 [ N j ] ≤ ∑ t = 2 T 1 2 D K L ( P z 1 0 ∥ P z 1 j ) + ∑ τ = 2 t − 1 E 0 [ D K L ( P z τ ∣ z 1 , … , z τ − 1 0 ∥ P z τ ∣ z 1 , … , z τ − 1 j ) ] ≤ T 22 K 2 E 0 [ N j ] . \begin{aligned}
\mathbb{E}^{j}[N_j]-\mathbb{E}^{0}[N_j]\leq\sum_{t=2}^{T}\frac{1}{\sqrt{2}}\sqrt{D_{\mathrm{KL}}\left(\mathbb{P}^{0}_{z_1}\mathbin{\|}\mathbb{P}^{j}_{z_1}\right)+\sum_{\tau=2}^{t-1}\mathbb{E}^{0}\left[D_{\mathrm{KL}}\left(\mathbb{P}^{0}_{z_\tau\mid z_1,\ldots,z_{\tau-1}}\mathbin{\|}\mathbb{P}^{j}_{z_\tau\mid z_1,\ldots,z_{\tau-1}}\right)\right]} \\
&\leq\frac{T}{22K^2}\sqrt{\mathbb{E}^{0}[N_j]}.
\end{aligned} E j [ N j ] − E 0 [ N j ] ≤ t = 2 ∑ T 2 1 D KL ( P z 1 0 ∥ P z 1 j ) + τ = 2 ∑ t − 1 E 0 [ D KL ( P z τ ∣ z 1 , … , z τ − 1 0 ∥ P z τ ∣ z 1 , … , z τ − 1 j ) ] ≤ 22 K 2 T E 0 [ N j ] .
By Claim 3.4 and recalling that Δ = w 2 / ( 384 ⋅ 12 ) \Delta=w^2/(384\cdot12) Δ = w 2 / ( 384 ⋅ 12 ) is the “suboptimality gap” of playing outside the right interval, the regret of the deterministic algorithm A \mathcal{A} A against the generic distribution D j \mathcal{D}_j D j is
R T ( A , D j ) ≥ ( T − E j [ N j ] ) Δ , R_T(\mathcal{A},\mathcal{D}_j)\geq(T-\mathbb{E}^{j}[N_j])\Delta, R T ( A , D j ) ≥ ( T − E j [ N j ]) Δ ,
so that if we take the expectation with respect to the uniform choice of j ∈ [ K ] j\in[K] j ∈ [ K ] , we get:
1 K ∑ j = 1 K R T ( A , D j ) ≥ 1 K ∑ j = 1 K ( T − E j [ N j ] ) Δ ≥ 1 K ∑ j = 1 K ( T − E 0 [ N j ] − T 22 K 2 E 0 [ N j ] ) Δ (By xref‘Claim 3.6‘) ≥ K − 1 K T Δ − 1 K ∑ j = 1 K T Δ 22 K 2 E 0 [ N j ] (As ∑ j E 0 [ N j ] ≤ T ) ≥ K − 1 K T Δ − T Δ 22 K 2 1 K ∑ j = 1 K E 0 [ N j ] (By Jensen’s Inequality) ≥ K − 1 K T Δ − T Δ 22 K 2 T K \begin{aligned}
\frac{1}{K}\sum_{j=1}^{K}R_T(\mathcal{A},\mathcal{D}_j)\geq\frac{1}{K}\sum_{j=1}^{K}(T-\mathbb{E}^{j}[N_j])\Delta\\
&\geq\frac{1}{K}\sum_{j=1}^{K}\left(T-\mathbb{E}^{0}[N_j]-\frac{T}{22K^2}\sqrt{\mathbb{E}^{0}[N_j]}\right)\Delta
\qquad\text{(By {xref}`Claim 3.6`)} \\
&\geq\frac{K-1}{K}T\Delta-\frac{1}{K}\sum_{j=1}^{K}\frac{T\Delta}{22K^2}\sqrt{\mathbb{E}^{0}[N_j]}\qquad\text{(As }\sum_j\mathbb{E}^{0}[N_j]\leq T\text{)} \\
&\geq\frac{K-1}{K}T\Delta-\frac{T\Delta}{22K^2}\sqrt{\frac{1}{K}\sum_{j=1}^{K}\mathbb{E}^{0}[N_j]}
\qquad\text{(By Jensen’s Inequality)} \\
&\ge\frac{K-1}{K}T\Delta-\frac{T\Delta}{22K^2}\sqrt{\frac{T}{K}}
\end{aligned} K 1 j = 1 ∑ K R T ( A , D j ) ≥ K 1 j = 1 ∑ K ( T − E j [ N j ]) Δ ≥ K 1 j = 1 ∑ K ( T − E 0 [ N j ] − 22 K 2 T E 0 [ N j ] ) Δ (By xref ‘Claim 3.6‘) ≥ K K − 1 T Δ − K 1 j = 1 ∑ K 22 K 2 T Δ E 0 [ N j ] (As j ∑ E 0 [ N j ] ≤ T ) ≥ K K − 1 T Δ − 22 K 2 T Δ K 1 j = 1 ∑ K E 0 [ N j ] (By Jensen’s Inequality) ≥ K K − 1 T Δ − 22 K 2 T Δ K T
(As ∑ j E 0 [ N j ] ≤ T \sum_j \mathbb{E}^0[N_j]\le T ∑ j E 0 [ N j ] ≤ T )
= T Δ ( 1 − 1 K − 1 22 T K 5 ) ≥ T Δ 2 , =T\Delta\left(1-\frac{1}{K}-\frac{1}{22}\sqrt{\frac{T}{K^5}}\right)\ge\frac{T\Delta}{2}, = T Δ ( 1 − K 1 − 22 1 K 5 T ) ≥ 2 T Δ ,
(By choosing K = ⌊ T 1 / 5 ⌋ K=\lfloor T^{1/5}\rfloor K = ⌊ T 1/5 ⌋ )
where the last inequality follows for all T T T such that ⌊ T 1 / 5 ⌋ ≥ 3 \lfloor T^{1/5}\rfloor\ge3 ⌊ T 1/5 ⌋ ≥ 3 . To conclude the proof, recall that Δ ∝ w 2 ∝ 1 K 2 ∝ 1 T 2 / 5 \Delta\propto w^2\propto\frac{1}{K^2}\propto\frac{1}{T^{2/5}} Δ ∝ w 2 ∝ K 2 1 ∝ T 2/5 1 .
Acknowledgment
The authors acknowledge OpenAI for providing complimentary access through the “ChatGPT for Academic Researchers” program. The work is partly supported by AI-PHOQUS PNRR IC 2021-2027, co-financed by EU Italian Ministry MIMIT and MUR. S.D.G. was also supported in part by the Institute for Complex Systems (Italian National Research Council).
AI Disclosure and Methodology
The authors used several versions of ChatGPT through the “Plus”, “Pro”, and “Academic Researchers” account tiers. In particular, the MHR regret upper bound (Section 5) and the lower bound for regular distributions (§ 7 ) were discovered through repeated interactions with ChatGPT 5.6 Sol. The authors then considerably simplified, shortened, and polished the original proofs.
After Castiglioni et al. [11 ] broke the T 3 / 4 T^{3/4} T 3/4 barrier, the authors tried interacting with ChatGPT 6 Astra Pro through the chat interface to understand whether a O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret upper bound was possible. These conversations did not lead to any nontrivial results. One of the authors then used the agentic “Work” interface of ChatGPT 6 Astra to coordinate multiple Astra agents, providing the existing proofs of the MHR upper bound and the regular lower bound (with a proof of O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret assuming independence), together with a long Markdown document produced during previous “Work” sessions summarizing the unsuccessful approaches considered so far. At that point, the model proposed the first version of what is now the Localized Sampling Lemma (Main Result 1 and Lemma 4.8 ), together with a Bayesian minimax argument for obtaining the O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret bound in the stochastic i.i.d. setting. The observation that successive elimination could be used directly to obtain the same rate came later from the authors (§ 3.1 ).
From the form of the minimax argument, the authors realized that it could also yield an adversarial regret bound. They subsequently simplified the proof of the Localized Sampling Lemma with LLM assistance, approximately halving its length. They also asked whether the adversarial result could be made constructive, and further interactions with the AI assistant led to the algorithm presented in § 4 .
Finally, the authors obtained the regret upper bound for regular distributions (Section 6) by trying to generalize the techniques used for the MHR upper bound, again with help from ChatGPT 6 Astra through the “Work” interface.
The authors also used ChatGPT 6 Astra to help polish and write the text, and to enforce correctness. Astra also provided first drafts of the technical overview for the stochastic i.i.d. and adversarial settings (§ 3.1 and § 3.2 ), the formal description of the successive-elimination algorithm (see Appendix § 12 ), and the comparison with dimension-based bandit bounds in Appendix § 11 .
The authors verified the correctness of all mathematical arguments, checked the references and the originality of the results, and take full responsibility for the content of the paper.
Appendix
Useful Inequalities
Proposition A.1. For any choice of parameters p , q ∈ ( 0 , 1 ) p,q \in(0,1) p , q ∈ ( 0 , 1 ) , we have the following bound on the KL divergence of two independent Bernoulli random variables of parameters p p p and q q q :
D K L ( Bern ( p ) ∥ Bern ( q ) ) ≤ ( p − q ) 2 q ( 1 − q ) . D_{\mathrm{KL}}(\operatorname{Bern}(p) \mathbin{\|} \operatorname{Bern}(q)) \le\frac{(p-q)^2}{q(1-q)}. D KL ( Bern ( p ) ∥ Bern ( q )) ≤ q ( 1 − q ) ( p − q ) 2 .
Proof. The proof follows immediately from the definition of divergence and the fact that log ( x ) ≤ x − 1 \log(x) \le x-1 log ( x ) ≤ x − 1 for x > 0 x>0 x > 0 :
D K L ( Bern ( p ) ∥ Bern ( q ) ) = p log p q + ( 1 − p ) log 1 − p 1 − q ≤ p ( p q − 1 ) + ( 1 − p ) ( 1 − p 1 − q − 1 ) = ( p − q ) 2 q ( 1 − q ) . \begin{aligned}
D_{\mathrm{KL}}(\operatorname{Bern}(p) \mathbin{\|} \operatorname{Bern}(q))
&= p\log\frac{p}{q} + (1-p)\log\frac{1-p}{1-q}\\
&\le p\left(\frac{p}{q}-1\right)+(1-p)\left(\frac{1-p}{1-q}-1\right)\\
&= \frac{(p-q)^2}{q(1-q)}.
\end{aligned} D KL ( Bern ( p ) ∥ Bern ( q )) = p log q p + ( 1 − p ) log 1 − q 1 − p ≤ p ( q p − 1 ) + ( 1 − p ) ( 1 − q 1 − p − 1 ) = q ( 1 − q ) ( p − q ) 2 .
Concentration Inequalities
To prove Proposition 5.4, claim 4.10, and Lemmata 6.5 and 6.6, we use Bernstein’s inequality, Hoeffding’s inequality and the stronger Azuma-Hoeffding’s inequality, see e.g., Theorem 1.2, Problem 1.9 and Theorem 5.1 of Dubhashi and Panconesi [21 ] .
Proposition A.2 (Bernstein’s inequality). Let the random variables X 1 , X 2 , … , X n X_1,X_2,\ldots,X_n X 1 , X 2 , … , X n be independent with X i − E [ X i ] < b X_i-\mathbb{E}[X_i]<b X i − E [ X i ] < b for each i = 1 , 2 , … , n i=1,2,\ldots,n i = 1 , 2 , … , n . Let X = ∑ i X i X=\sum_i X_i X = ∑ i X i , and let σ 2 = ∑ i σ i 2 \sigma^2=\sum_i\sigma_i^2 σ 2 = ∑ i σ i 2 be the variance of X X X . Then, for any t > 0 t>0 t > 0 ,
P ( X > E [ X ] + t ) ≤ exp ( − t 2 2 σ 2 + 2 3 b t ) . \mathbb{P}\left(X>\mathbb{E}[X]+t\right)\le\exp\left(-\frac{t^2}{2\sigma^2+\frac{2}{3}bt}\right). P ( X > E [ X ] + t ) ≤ exp ( − 2 σ 2 + 3 2 b t t 2 ) .
Proposition A.3 (Hoeffding’s Inequality). Let the random variables X 1 , X 2 , … , X n X_1,X_2,\ldots,X_n X 1 , X 2 , … , X n be independent with X i ∈ [ a , b ] X_i\in[a,b] X i ∈ [ a , b ] for a , b ∈ R a,b\in\mathbb{R} a , b ∈ R and for each i = 1 , 2 , … , n i=1,2,\ldots,n i = 1 , 2 , … , n . Letting X = ∑ i X i X=\sum_i X_i X = ∑ i X i , for any t > 0 t>0 t > 0 ,
P ( ∣ X − E [ X ] ∣ > t ) ≤ 2 ⋅ exp ( − 2 t 2 n ( b − a ) 2 ) . \mathbb{P}\left(\lvert X-\mathbb{E}[X]\rvert>t\right)\le2\cdot\exp\left(-\frac{2t^2}{n(b-a)^2}\right). P ( ∣ X − E [ X ]∣ > t ) ≤ 2 ⋅ exp ( − n ( b − a ) 2 2 t 2 ) .
Proposition A.4 (Azuma-Hoeffding inequality). Let { X 0 , X 1 , … } \{X_0,X_1,\ldots\} { X 0 , X 1 , … } be a martingale satisfying, for any i ≥ 1 i\ge1 i ≥ 1 , X i − X i − 1 ∈ [ a , b ] X_i-X_{i-1}\in[a,b] X i − X i − 1 ∈ [ a , b ] almost surely, for two fixed values a , b ∈ R a,b\in\mathbb{R} a , b ∈ R . Then we have the following concentration inequality, for any t > 0 t>0 t > 0 :
P ( X n > X 0 + t ) , P ( X n < X 0 − t ) ≤ exp ( − 2 t 2 n ( b − a ) 2 ) . \mathbb{P}(X_n>X_0+t),\mathbb{P}(X_n<X_0-t)\le\exp\left(-\frac{2t^2}{n(b-a)^2}\right). P ( X n > X 0 + t ) , P ( X n < X 0 − t ) ≤ exp ( − n ( b − a ) 2 2 t 2 ) .
We also use the following version of the multiplicative Chernoff–Hoeffding bound (restatement of Theorem 1.1 of Dubhashi and Panconesi [21 ] ).
Proposition A.5 (Multiplicative Chernoff bound). Let X X X be defined as the sum of n n n random variables X i X_i X i , independently distributed in [ 0 , 1 ] [0,1] [ 0 , 1 ] . Then the following inequalities hold, for ε ∈ ( 0 , 1 ) \varepsilon\in(0,1) ε ∈ ( 0 , 1 ) :
P ( X < ( 1 − ε ) E [ X ] ) ≤ exp ( − 1 2 ε 2 E [ X ] ) \mathbb{P}\left(X<(1-\varepsilon)\mathbb{E}[X]\right)\le\exp\left(-\frac{1}{2}\varepsilon^2\mathbb{E}[X]\right) P ( X < ( 1 − ε ) E [ X ] ) ≤ exp ( − 2 1 ε 2 E [ X ] ) P ( X > ( 1 + ε ) E [ X ] ) ≤ exp ( − 1 3 ε 2 E [ X ] ) \mathbb{P}\left(X>(1+\varepsilon)\mathbb{E}[X]\right)\le\exp\left(-\frac{1}{3}\varepsilon^2\mathbb{E}[X]\right) P ( X > ( 1 + ε ) E [ X ] ) ≤ exp ( − 3 1 ε 2 E [ X ] )
The Discretization Lemma
In this lemma, we formalize the fact that the action space A = G K ∩ T \mathcal{A} = G_K \cap\mathcal{T} A = G K ∩ T yields a discretization error of order T / K T/K T / K .
Lemma A.6 (Discretization Error). For every integer K ≥ 1 K \ge1 K ≥ 1 and every ( p , q ) ∈ T (p,q) \in\mathcal{T} ( p , q ) ∈ T , there is a pair ( p ′ , q ′ ) ∈ A (p',q') \in\mathcal{A} ( p ′ , q ′ ) ∈ A such that, for every valuation pair ( s t , b t ) (s_t,b_t) ( s t , b t ) ,
Profit t ( p , q ) ≤ Profit t ( p ′ , q ′ ) + 2 / K . \operatorname{Profit}_t(p,q) \le\operatorname{Profit}_t(p',q') + 2/K. Profit t ( p , q ) ≤ Profit t ( p ′ , q ′ ) + 2/ K . Consequently, restricting the comparator to A \mathcal{A} A adds at most 2 T / K 2T/K 2 T / K to regret.
Proof. Fix ( p , q ) ∈ T (p,q) \in\mathcal{T} ( p , q ) ∈ T and put p ˉ = ⌈ K p ⌉ / K \bar{p} = \lceil Kp\rceil/K p ˉ = ⌈ K p ⌉ / K and q ˉ = ⌊ K q ⌋ / K \bar{q} = \lfloor Kq\rfloor/K q ˉ = ⌊ K q ⌋ / K . If p ˉ ≤ q ˉ \bar{p} \le\bar{q} p ˉ ≤ q ˉ , use ( p ′ , q ′ ) = ( p ˉ , q ˉ ) (p',q') = (\bar{p},\bar{q}) ( p ′ , q ′ ) = ( p ˉ , q ˉ ) . Every trade accepted at ( p , q ) (p,q) ( p , q ) is still accepted, and its profit decreases by at most 2 / K 2/K 2/ K . Any additional trade has nonnegative profit, so the inequality holds for every valuation pair.
If p ˉ > q ˉ \bar{p} > \bar{q} p ˉ > q ˉ , then q − p < 2 / K q-p < 2/K q − p < 2/ K . In this case use ( p ′ , q ′ ) = ( 0 , 0 ) (p',q') = (0,0) ( p ′ , q ′ ) = ( 0 , 0 ) , whose zero profit is within 2 / K 2/K 2/ K of the profit at ( p , q ) (p,q) ( p , q ) . The chosen grid pair depends only on ( p , q ) (p,q) ( p , q ) , so summing over rounds and taking expectations proves the regret claim.
Missing Result from § 4
Claim 4.10. Let G \mathcal{G} G be a set of monotone functions mapping from some subset of finite domain Y \mathcal{Y} Y to [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] . For every g ∈ G g \in\mathcal{G} g ∈ G and each point y ∈ Y y \in\mathcal{Y} y ∈ Y in the domain of g g g , suppose we have a sequence of mutually independent random variables
Z g , y ( 1 ) , Z g , y ( 2 ) , … , Z_{g,y}^{(1)}, Z_{g,y}^{(2)}, \ldots, Z g , y ( 1 ) , Z g , y ( 2 ) , … , with expectation g ( y ) g(y) g ( y ) and taking values in a common interval of length at most 4. Each query at g ( y ) g(y) g ( y ) reveals the next unused variable in its sequence. Suppose that ∑ g ∈ G TV ( g ) ≤ W \sum_{g\in\mathcal{G}}\operatorname{TV}(g) \le W ∑ g ∈ G TV ( g ) ≤ W , where W W W is a known constant and TV ( g ) \operatorname{TV}(g) TV ( g ) stands for the total variation of g g g .
Then, for α , δ ′ ∈ ( 0 , 1 ) \alpha,\delta' \in(0,1) α , δ ′ ∈ ( 0 , 1 ) , there exists a procedure which estimates every function uniformly to error α \alpha α , simultaneously with probability at least 1 − δ ′ 1-\delta' 1 − δ ′ , using at most
C log ( 2 ∣ G ∣ ∣ Y ∣ / δ ′ ) α 2 ( ∣ G ∣ + W α log ( 2 ∣ Y ∣ ) ) . C\frac{\log(2|\mathcal{G}||\mathcal{Y}|/\delta')}{\alpha^2}\left(|\mathcal{G}|+\frac{W}{\alpha}\log(2|\mathcal{Y}|)\right). C α 2 log ( 2∣ G ∣∣ Y ∣/ δ ′ ) ( ∣ G ∣ + α W log ( 2∣ Y ∣ ) ) . queries, where C C C is an absolute constant.
Proof. Let g ∈ G g \in\mathcal{G} g ∈ G with domain Y g = { y 1 , … , y N } ⊆ Y \mathcal{Y}_g = \{y_1,\ldots,y_N\} \subseteq\mathcal{Y} Y g = { y 1 , … , y N } ⊆ Y . We want to construct an estimator g ^ \hat{g} g ^ for g g g . Let the integer n n n be a sample complexity parameter to be set later. We first start the estimation from the boundary of the domain, assuming that N ≥ 2 N \ge2 N ≥ 2 , and let
g ^ ( y 1 ) = 1 n ∑ i = 1 n Z g , y 1 ( i ) g ^ ( y N ) = 1 n ∑ i = 1 n Z g , y N ( i ) . (80) \hat{g}(y_1)=\frac{1}{n}\sum_{i=1}^{n}Z_{g,y_1}^{(i)}
\qquad
\hat{g}(y_N)=\frac{1}{n}\sum_{i=1}^{n}Z_{g,y_N}^{(i)}. \tag*{(80)} g ^ ( y 1 ) = n 1 i = 1 ∑ n Z g , y 1 ( i ) g ^ ( y N ) = n 1 i = 1 ∑ n Z g , y N ( i ) . (80)
Let E \mathcal{E} E be the clean event in which ∣ 1 n ∑ i = 1 n Z g , y ( i ) − g ( y ) ∣ ≤ α / 8 \left|\frac{1}{n}\sum_{i=1}^{n}Z_{g,y}^{(i)}-g(y)\right|\le\alpha/8 n 1 ∑ i = 1 n Z g , y ( i ) − g ( y ) ≤ α /8 , for any g ∈ G g\in\mathcal{G} g ∈ G and y ∈ Y g y\in\mathcal{Y}_g y ∈ Y g . For now, assume E \mathcal{E} E . Then, if ∣ g ^ ( y 1 ) − g ^ ( y N ) ∣ ≤ α / 2 |\hat{g}(y_1)-\hat{g}(y_N)|\le\alpha/2 ∣ g ^ ( y 1 ) − g ^ ( y N ) ∣ ≤ α /2 , the triangle inequality implies ∣ g ( y 1 ) − g ( y N ) ∣ ≤ α / 2 + α / 4 = 3 α / 4 |g(y_1)-g(y_N)|\le\alpha/2+\alpha/4=3\alpha/4 ∣ g ( y 1 ) − g ( y N ) ∣ ≤ α /2 + α /4 = 3 α /4 , and we can set g ^ ( y ) = g ^ ( y 1 ) \hat{g}(y)=\hat{g}(y_1) g ^ ( y ) = g ^ ( y 1 ) ∀ y ∈ Y g ∖ { y N } \forall y\in\mathcal{Y}_g\setminus\{y_N\} ∀ y ∈ Y g ∖ { y N } , guaranteeing the following, by monotonicity:
∣ g ^ ( y ) − g ( y ) ∣ = ∣ g ^ ( y 1 ) − g ( y ) ∣ ≤ ∣ g ^ ( y 1 ) − g ( y 1 ) ∣ + ∣ g ( y 1 ) − g ( y ) ∣ ≤ α 8 + 3 α 4 < α (81) |\hat{g}(y)-g(y)|=|\hat{g}(y_1)-g(y)|\le|\hat{g}(y_1)-g(y_1)|+|g(y_1)-g(y)|\le\frac{\alpha}{8}+\frac{3\alpha}{4}<\alpha \tag*{(81)} ∣ g ^ ( y ) − g ( y ) ∣ = ∣ g ^ ( y 1 ) − g ( y ) ∣ ≤ ∣ g ^ ( y 1 ) − g ( y 1 ) ∣ + ∣ g ( y 1 ) − g ( y ) ∣ ≤ 8 α + 4 3 α < α (81)
In other words, assuming that the estimated variation is small enough, this simple procedure is enough to guarantee ∥ g ^ − g ∥ ∞ ≤ α \|\hat{g}-g\|_\infty\le\alpha ∥ g ^ − g ∥ ∞ ≤ α , under E \mathcal{E} E .
If ∣ g ^ ( y 1 ) − g ^ ( y N ) ∣ > α / 2 |\hat{g}(y_1)-\hat{g}(y_N)|>\alpha/2 ∣ g ^ ( y 1 ) − g ^ ( y N ) ∣ > α /2 and N > 2 N>2 N > 2 , split the domain in two sets { y 1 , y 2 , … , y ⌈ N / 2 ⌉ } \{y_1,y_2,\ldots,y_{\lceil N/2\rceil}\} { y 1 , y 2 , … , y ⌈ N /2 ⌉ } , { y ⌈ N / 2 ⌉ , … , y N } \{y_{\lceil N/2\rceil},\ldots,y_N\} { y ⌈ N /2 ⌉ , … , y N } and recurse over them with the same procedure explained above and the same halving rule, retaining the estimates on already estimated boundary points. The recursion stops when N ≤ 2 N\le2 N ≤ 2 or when the estimated variation is under α / 2 \alpha/2 α /2 . Under E \mathcal{E} E , both termination conditions enforce ∥ g − g ^ ∥ ∞ ≤ α \|g-\hat{g}\|_\infty\le\alpha ∥ g − g ^ ∥ ∞ ≤ α . Indeed, if the variation is small enough, then an argument identical to Equation (81) yields the guarantee; if N ≤ 2 N \le2 N ≤ 2 , we either have one point or two, so that each can be directly estimated to precision α / 8 \alpha/8 α /8 .
We now need to count the number of points in Y \mathcal{Y} Y whose value of g g g we directly estimate using this procedure, assuming E \mathcal{E} E . First, notice that the recursion tree is a binary tree with maximum depth bounded by 1 + log 2 ( ∣ Y ∣ ) 1+\log_2(|\mathcal{Y}|) 1 + log 2 ( ∣ Y ∣ ) , due to the fact that each pair of children halves the parent node set. Let us focus on a specific depth. If an interval at that depth is split, then the estimated variation must be at least α / 2 \alpha/2 α /2 and the reverse triangle inequality together with E \mathcal{E} E implies that the real variation must be at least α / 4 \alpha/4 α /4 . Moreover, the considered intervals have disjoint interiors, so that any split at that level of the tree consumes at least α / 4 \alpha/4 α /4 of TV ( g ) \operatorname{TV}(g) TV ( g ) . This means that, at any given depth, there can only be 4 TV ( g ) / α 4\operatorname{TV}(g)/\alpha 4 TV ( g ) / α splits.
Any split adds at most one more mean to estimate, and the start of the recursion at the root requires the sample means of at most two points. Therefore, summing over the number of functions in G \mathcal{G} G , we need to estimate the average of a number of random variables bounded by:
2 ∣ G ∣ + 4 ∑ g ∈ G TV ( g ) α ⋅ ( 1 + ⌈ log 2 ∣ Y ∣ ⌉ ) ≤ 2 ∣ G ∣ + 4 W α ⋅ ( 1 + ⌈ log 2 ∣ Y ∣ ⌉ ) (82) 2|\mathcal{G}|+\frac{4\sum_{g\in\mathcal{G}}\operatorname{TV}(g)}{\alpha}\cdot\left(1+\left\lceil\log_2|\mathcal{Y}|\right\rceil\right)\leq2|\mathcal{G}|+\frac{4W}{\alpha}\cdot\left(1+\left\lceil\log_2|\mathcal{Y}|\right\rceil\right) \tag*{(82)} 2∣ G ∣ + α 4 ∑ g ∈ G TV ( g ) ⋅ ( 1 + ⌈ log 2 ∣ Y ∣ ⌉ ) ≤ 2∣ G ∣ + α 4 W ⋅ ( 1 + ⌈ log 2 ∣ Y ∣ ⌉ ) (82)
Clearly, this bound holds only under E \mathcal{E} E , but we can force the procedure to deterministically halt after having estimated enough averages since W W W is known, so the cap in Equation (82) holds independently of the occurrence of the failure event E \mathcal{E} E .
Since each observation lies in an interval of length at most 4 4 4 , Proposition A.3 gives
P ( ∣ 1 n ∑ i = 1 n Z g , y ( i ) − g ( y ) ∣ > α / 8 ) ≤ 2 exp ( − n α 2 512 ) . \mathbb{P}\left(\left|\frac{1}{n}\sum_{i=1}^{n}Z_{g,y}^{(i)}-g(y)\right|>\alpha/8\right)\leq2\exp\left(-\frac{n\alpha^2}{512}\right). P ( n 1 i = 1 ∑ n Z g , y ( i ) − g ( y ) > α /8 ) ≤ 2 exp ( − 512 n α 2 ) .
Choose n = ⌈ 512 α 2 log 2 ∣ G ∣ ∣ Y ∣ δ ′ ⌉ n=\left\lceil\frac{512}{\alpha^2}\log\frac{2|\mathcal{G}||\mathcal{Y}|}{\delta'}\right\rceil n = ⌈ α 2 512 log δ ′ 2∣ G ∣∣ Y ∣ ⌉ . A union bound over all function–point pairs gives P ( E ) ≥ 1 − δ ′ \mathbb{P}(\mathcal{E})\geq1-\delta' P ( E ) ≥ 1 − δ ′ . Multiplying this sample size by the bound on the number of estimated points proves the claim.
Missing Results from Section 5
Corollary 3.1 (Stochastic convex bandit). Let K \mathcal{K} K be a known convex body of constant dimension, and let L : K → R L:\mathcal{K}\to\mathbb{R} L : K → R be continuous and convex, with values in a known interval of constant length. Suppose that querying x τ ∈ K x_\tau\in\mathcal{K} x τ ∈ K returns L ( x τ ) + ξ τ L(x_\tau)+\xi_\tau L ( x τ ) + ξ τ . Conditional on previous queries and observations, and on x τ x_\tau x τ , assume that ξ τ \xi_\tau ξ τ has mean zero and ψ 2 \psi_2 ψ 2 norm at most a known γ ≥ 0 \gamma\geq0 γ ≥ 0 . Then, for every n ≥ 1 n\geq1 n ≥ 1 , there is an n n n -query algorithm such that
E [ ∑ τ = 1 n ( L ( x τ ) − min x ∈ K L ( x ) ) ] ∈ O ( ( 1 + γ ) n log 2 ( n ) ) . \mathbb{E}\left[\sum_{\tau=1}^{n}\left(L(x_\tau)-\min_{x\in\mathcal{K}}L(x)\right)\right]\in O\left((1+\gamma)\sqrt{n}\log^2(n)\right). E [ τ = 1 ∑ n ( L ( x τ ) − x ∈ K min L ( x ) ) ] ∈ O ( ( 1 + γ ) n log 2 ( n ) ) .
where the expectation is taken over noise and randomness of the algorithm’s actions. The hidden constant depends only on the affine dimension and the length of the specified interval.
Proof. The result is a corollary of Theorem 1 from Lattimore and György [32 ] , which requires the domain to be a convex body containing the unit radius Euclidean ball. To enforce this, let d d d be the affine dimension of K \mathcal{K} K . If d = 0 d=0 d = 0 , then K \mathcal{K} K is a singleton and the regret is zero. Otherwise, John’s ellipsoid theorem [28 ] , applied in the affine hull of K \mathcal{K} K , allows an affine and invertible change of coordinates such that
B 2 d ⊆ K ⊆ d B 2 d . \mathbb{B}_2^d\subseteq\mathcal{K}\subseteq d\mathbb{B}_2^d. B 2 d ⊆ K ⊆ d B 2 d .
where B 2 d \mathbb{B}_2^d B 2 d is the Euclidean unit ball. This domain change preserves convexity and regret, due to the map being affine and bijective. In these coordinates, K \mathcal{K} K contains the unit ball and has diameter at most 2 d 2d 2 d .
Write the known interval containing the image of L L L as [ b , b + B ] [b,b+B] [ b , b + B ] , and set
S = max { 1 , B , γ } , L ~ ( x ) = L ( x ) − b S . S=\max\{1,B,\gamma\},\qquad\widetilde{L}(x)=\frac{L(x)-b}{S}. S = max { 1 , B , γ } , L ( x ) = S L ( x ) − b .
The normalized loss is continuous, convex, and takes values in [ 0 , 1 ] [0,1] [ 0 , 1 ] . Run the algorithm of Lattimore and György [32 Theorem 1] for n n n rounds on this normalized problem. In its round τ \tau τ , when it selects x τ x_\tau x τ , perform one query and obtain the observation L ( x τ ) + ξ τ L(x_\tau)+\xi_\tau L ( x τ ) + ξ τ . We compute
L ( x τ ) + ξ τ − b S = L ~ ( x τ ) + ξ τ S , \frac{L(x_\tau)+\xi_\tau-b}{S}=\widetilde{L}(x_\tau)+\frac{\xi_\tau}{S}, S L ( x τ ) + ξ τ − b = L ( x τ ) + S ξ τ ,
and return it to the algorithm. Conditional on the previous queries and observations and on x τ x_\tau x τ , the returned noise ξ τ / S \xi_\tau/S ξ τ / S has mean zero and ψ 2 \psi_2 ψ 2 norm at most one. These are precisely the noise assumptions of [32 Theorem 1] . Since d d d is a constant and K K K after the change of coordinates has diameter at most 2 d 2d 2 d , the theorem gives
E [ ∑ τ = 1 n ( L ~ ( x τ ) − min x ∈ K L ~ ( x ) ) ] = O ( n log 2 ( n ) ) , \mathbb{E}\left[\sum_{\tau=1}^{n}\left(\widetilde{L}(x_\tau)-\min_{x\in\mathcal{K}}\widetilde{L}(x)\right)\right]=O\left(\sqrt{n}\log^2(n)\right), E [ τ = 1 ∑ n ( L ( x τ ) − x ∈ K min L ( x ) ) ] = O ( n log 2 ( n ) ) ,
with a constant depending only on d d d . Every original loss difference is exactly S S S times its normalized counterpart. Multiplying by S ≤ max { 1 , B } ( 1 + γ ) S\leq\max\{1,B\}(1+\gamma) S ≤ max { 1 , B } ( 1 + γ ) proves the claim.
Proposition 5.4 (Simultaneous concentration). Let 2 ≤ K ≤ T 2\leq K\leq T 2 ≤ K ≤ T . With probability at least 1 − 1 / T 1-1/T 1 − 1/ T , simultaneously at every scale δ \delta δ reached by Initialize MHR , the estimates computed at that scale satisfy
∣ μ ^ ( e ) − μ ( e ) ∣ ≤ μ ( e ) + δ 16 for every e ∈ G K . (83) \lvert\widehat{\mu}(e)-\mu(e)\rvert\leq\frac{\mu(e)+\delta}{16}\qquad\text{for every }e\in G_K. \tag*{(83)} ∣ μ ( e ) − μ ( e )∣ ≤ 16 μ ( e ) + δ for every e ∈ G K . (83)
Proof. Fix a scale δ \delta δ and a sampled pair ( i , j ) ∈ S (i,j)\in S ( i , j ) ∈ S . Write P i j = P ( i / K , j / K ) P_{ij}=P(i/K,j/K) P ij = P ( i / K , j / K ) , and let P ‾ i j \overline{P}_{ij} P ij be the average of N N N fresh trade indicators at that pair. Since each indicator is Bernoulli with mean P i j P_{ij} P ij , its second moment equals P i j P_{ij} P ij . This second-order bound enables Bernstein’s inequality (see Proposition A.2): for any ε > 0 \varepsilon>0 ε > 0 , we have
P ( ∣ P ‾ i j − P i j ∣ > ε ) ≤ 2 exp ( − N ε 2 2 P i j + 2 3 ε ) . \mathbb{P}\left(\lvert\overline{P}_{ij}-P_{ij}\rvert>\varepsilon\right)\leq2\exp\left(\frac{-N\varepsilon^2}{2P_{ij}+\frac{2}{3}\varepsilon}\right). P ( ∣ P ij − P ij ∣ > ε ) ≤ 2 exp ( 2 P ij + 3 2 ε − N ε 2 ) .
In order for the claimed statement to hold, we bound the right-hand side by 2 ( 2 T ) − 4 = 2 e − 4 log ( 2 T ) 2(2T)^{-4}=2e^{-4\log(2T)} 2 ( 2 T ) − 4 = 2 e − 4 l o g ( 2 T ) . Therefore, a sufficient condition is:
N ε 2 2 P i j + 2 3 ε ≥ 4 log ( 2 T ) ⟺ ε ≥ 4 log ( 2 T ) 3 N + 8 P i j log ( 2 T ) N + 16 log 2 ( 2 T ) 9 N 2 . \frac{N\varepsilon^2}{2P_{ij}+\frac{2}{3}\varepsilon}\geq4\log(2T)\qquad\Longleftrightarrow\qquad\varepsilon\geq\frac{4\log(2T)}{3N}+\sqrt{\frac{8P_{ij}\log(2T)}{N}+\frac{16\log^2(2T)}{9N^2}}. 2 P ij + 3 2 ε N ε 2 ≥ 4 log ( 2 T ) ⟺ ε ≥ 3 N 4 log ( 2 T ) + N 8 P ij log ( 2 T ) + 9 N 2 16 log 2 ( 2 T ) .
We set ε \varepsilon ε equal to the expression on the right. Thus, with probability at least 1 − 2 ( 2 T ) − 4 1-2(2T)^{-4} 1 − 2 ( 2 T ) − 4 ,
∣ P ‾ i j − P i j ∣ ≤ 4 log ( 2 T ) 3 N + 8 P i j log ( 2 T ) N + 16 log 2 ( 2 T ) 9 N 2 ≤ 8 P i j log ( 2 T ) N + 8 log ( 2 T ) 3 N ≤ P i j δ 2 13 + δ 2 8 (by our choice of N ) ≤ P i j 128 + 2 δ 2 8 ( 2 a b ≤ a + b with a = P i j 2 7 and b = δ 2 8 ) = P i j + δ 128 . \begin{aligned}
\lvert\overline{P}_{ij}-P_{ij}\rvert\leq\frac{4\log(2T)}{3N}+\sqrt{\frac{8P_{ij}\log(2T)}{N}+\frac{16\log^2(2T)}{9N^2}} \\
&\leq\sqrt{\frac{8P_{ij}\log(2T)}{N}}+\frac{8\log(2T)}{3N} \\
&\leq\sqrt{\frac{P_{ij}\delta}{2^{13}}}+\frac{\delta}{2^8}\qquad\text{(by our choice of }N\text{)} \\
&\leq\frac{P_{ij}}{128}+\frac{2\delta}{2^8}\qquad\text{(}2\sqrt{ab}\leq a+b\text{ with }a=\frac{P_{ij}}{2^7}\text{ and }b=\frac{\delta}{2^8}\text{)} \\
&=\frac{P_{ij}+\delta}{128}.
\end{aligned} ∣ P ij − P ij ∣ ≤ 3 N 4 log ( 2 T ) + N 8 P ij log ( 2 T ) + 9 N 2 16 log 2 ( 2 T ) ≤ N 8 P ij log ( 2 T ) + 3 N 8 log ( 2 T ) ≤ 2 13 P ij δ + 2 8 δ (by our choice of N ) ≤ 128 P ij + 2 8 2 δ ( 2 ab ≤ a + b with a = 2 7 P ij and b = 2 8 δ ) = 128 P ij + δ .
There are at most ∣ S ∣ ≤ ( K + 1 ) 2 ≤ 4 T 2 \lvert S\rvert\leq(K+1)^2\leq4T^2 ∣ S ∣ ≤ ( K + 1 ) 2 ≤ 4 T 2 sampled pairs and 1 + ⌊ log 2 ( K ) ⌋ ≤ T 1+\lfloor\log_2(K)\rfloor\leq T 1 + ⌊ log 2 ( K )⌋ ≤ T possible scales. A union bound over all pairs and scales gives total failure probability at most 2 ∣ S ∣ T ( 2 T ) − 4 ≤ 1 / T 2\lvert S\rvert T(2T)^{-4}\leq1/T 2 ∣ S ∣ T ( 2 T ) − 4 ≤ 1/ T . Thus, with probability at least 1 − 1 / T 1-1/T 1 − 1/ T , at every reached scale,
∣ P ‾ i j − P i j ∣ ≤ P i j + δ 128 for all ( i , j ) ∈ S . (84) \lvert\overline{P}_{ij}-P_{ij}\rvert\leq\frac{P_{ij}+\delta}{128}\qquad\text{for all }(i,j)\in S. \tag*{(84)} ∣ P ij − P ij ∣ ≤ 128 P ij + δ for all ( i , j ) ∈ S . (84)
We prove the claimed profit bound deterministically on this high probability event.
We first show that for each interior grid pair, the reconstruction formula estimates its trade probability P i j P_{ij} P ij with error at most ( P i j + δ ) / 16 (P_{ij}+\delta)/16 ( P ij + δ ) /16 . Fix any reached scale δ \delta δ and indices 1 ≤ i < j ≤ K − 1 1\leq i<j\leq K-1 1 ≤ i < j ≤ K − 1 , and let k k k be the midpoint of the first split separating i i i and j j j , as explained in Algorithm 4. The construction includes ( i , k ) (i,k) ( i , k ) , ( k , j ) (k,j) ( k , j ) , and ( k , k ) (k,k) ( k , k ) in S S S . Independence and monotonicity give
P i k P k j = P i j P k k 0 ≤ P i k , P k j , P i j ≤ P k k (85) P_{ik}P_{kj}=P_{ij}P_{kk}\qquad0\leq P_{ik},P_{kj},P_{ij}\leq P_{kk} \tag*{(85)} P ik P k j = P ij P k k 0 ≤ P ik , P k j , P ij ≤ P k k (85)
Denote the estimator’s denominator by D = max { P ‾ i k , P ‾ k j , P ‾ k k } D=\max\{\overline{P}_{ik},\overline{P}_{kj},\overline{P}_{kk}\} D = max { P ik , P k j , P k k } , and notice that 0 ≤ P ‾ i k P ‾ k j / D ≤ min { P ‾ i k , P ‾ k j } 0\leq\overline{P}_{ik}\overline{P}_{kj}/D\leq\min\{\overline{P}_{ik},\overline{P}_{kj}\} 0 ≤ P ik P k j / D ≤ min { P ik , P k j } , with 0 / 0 = 0 0/0=0 0/0 = 0 . Therefore, if P k k < δ / 32 P_{kk}<\delta/32 P k k < δ /32 , then
∣ P ‾ i k P ‾ k j D − P i j ∣ ≤ P k k + P k k + δ 128 < δ 16 , \left|\frac{\overline{P}_{ik}\overline{P}_{kj}}{D}-P_{ij}\right|
\leq P_{kk}+\frac{P_{kk}+\delta}{128}<\frac{\delta}{16}, D P ik P k j − P ij ≤ P k k + 128 P k k + δ < 16 δ ,
where the first inequality follows from P i k ≤ P k k P_{ik}\leq P_{kk} P ik ≤ P k k and Equation (84) , so that both P i j P_{ij} P ij and P ‾ i k P ‾ k j / D \overline{P}_{ik}\overline{P}_{kj}/D P ik P k j / D belong to [ 0 , P k k + ( P k k + δ ) / 128 ] [0,P_{kk}+(P_{kk}+\delta)/128] [ 0 , P k k + ( P k k + δ ) /128 ] .
To handle P k k ≥ δ / 32 P_{kk}\geq\delta/32 P k k ≥ δ /32 , Equation (84) implies, via Equation (85) :
∣ D − P k k ∣ ≤ P k k + δ 128 . (86) |D-P_{kk}|\leq\frac{P_{kk}+\delta}{128}. \tag*{(86)} ∣ D − P k k ∣ ≤ 128 P k k + δ . (86)
Adding and subtracting and applying the error bounds from Equation (84) yields:
∣ P ‾ i k P ‾ k j − P i j D ∣ = ∣ P i k P k j + P k j ( P ‾ i k − P i k ) + P i k ( P ‾ k j − P k j ) + ( P ‾ k j − P k j ) ( P ‾ i k − P i k ) − P i j P k k − P i j ( D − P k k ) ∣ = ∣ P k j ( P ‾ i k − P i k ) + P i k ( P ‾ k j − P k j ) + ( P ‾ k j − P k j ) ( P ‾ i k − P i k ) − P i j ( D − P k k ) ∣ \begin{aligned}
|\overline{P}_{ik}\overline{P}_{kj}-P_{ij}D|
&=\big|P_{ik}P_{kj}+P_{kj}(\overline{P}_{ik}-P_{ik})+P_{ik}(\overline{P}_{kj}-P_{kj})\\
&\qquad+(\overline{P}_{kj}-P_{kj})(\overline{P}_{ik}-P_{ik})-P_{ij}P_{kk}-P_{ij}(D-P_{kk})\big|\\
&=\big|P_{kj}(\overline{P}_{ik}-P_{ik})+P_{ik}(\overline{P}_{kj}-P_{kj})\\
&\qquad+(\overline{P}_{kj}-P_{kj})(\overline{P}_{ik}-P_{ik})-P_{ij}(D-P_{kk})\big|
\end{aligned} ∣ P ik P k j − P ij D ∣ = P ik P k j + P k j ( P ik − P ik ) + P ik ( P k j − P k j ) + ( P k j − P k j ) ( P ik − P ik ) − P ij P k k − P ij ( D − P k k ) = P k j ( P ik − P ik ) + P ik ( P k j − P k j ) + ( P k j − P k j ) ( P ik − P ik ) − P ij ( D − P k k )
(By Equation (85) )
≤ P k j ( P i k + δ ) + P i k ( P k j + δ ) + P i j ( P k k + δ ) 128 + ( P i k + δ ) ( P k j + δ ) 128 2 (Triangle Inequality and Equations (81) and (83)) ≤ ( 3 128 + 34 128 2 ) P k k ( P i j + δ ) ≤ P k k ( P i j + δ ) 32 . \begin{aligned}
&\leq\frac{P_{kj}(P_{ik}+\delta)+P_{ik}(P_{kj}+\delta)+P_{ij}(P_{kk}+\delta)}{128}
+\frac{(P_{ik}+\delta)(P_{kj}+\delta)}{128^2}\\
&\qquad\text{(Triangle Inequality and Equations (81) and (83))}\\
&\leq\left(\frac{3}{128}+\frac{34}{128^2}\right)P_{kk}(P_{ij}+\delta)
\leq\frac{P_{kk}(P_{ij}+\delta)}{32}.
\end{aligned} ≤ 128 P k j ( P ik + δ ) + P ik ( P k j + δ ) + P ij ( P k k + δ ) + 12 8 2 ( P ik + δ ) ( P k j + δ ) (Triangle Inequality and Equations (81) and (83)) ≤ ( 128 3 + 12 8 2 34 ) P k k ( P ij + δ ) ≤ 32 P k k ( P ij + δ ) .
(By Equation (85) and δ 2 ≤ δ ≤ 32 P k k \delta^2\leq\delta\leq32P_{kk} δ 2 ≤ δ ≤ 32 P k k )
To conclude, Equation (86) and δ ≤ 32 P k k \delta\leq32P_{kk} δ ≤ 32 P k k give D ≥ P k k − ( P k k + δ ) / 128 ≥ 95 P k k / 128 ≥ P k k / 2 > 0 D\geq P_{kk}-(P_{kk}+\delta)/128\geq95P_{kk}/128\geq P_{kk}/2>0 D ≥ P k k − ( P k k + δ ) /128 ≥ 95 P k k /128 ≥ P k k /2 > 0 , so that dividing the chain of inequalities above by D D D bounds the trade-probability error by ( P i j + δ ) / 16 (P_{ij}+\delta)/16 ( P ij + δ ) /16 .
In both cases, multiplying by ( j − i ) / K ≤ 1 (j-i)/K\leq1 ( j − i ) / K ≤ 1 gives
∣ μ ^ ( i / K , j / K ) − μ ( i / K , j / K ) ∣ ≤ j − i K P i j + δ 16 ≤ μ ( i / K , j / K ) + δ 16 . |\widehat{\mu}(i/K,j/K)-\mu(i/K,j/K)|
\leq\frac{j-i}{K}\frac{P_{ij}+\delta}{16}
\leq\frac{\mu(i/K,j/K)+\delta}{16}. ∣ μ ( i / K , j / K ) − μ ( i / K , j / K ) ∣ ≤ K j − i 16 P ij + δ ≤ 16 μ ( i / K , j / K ) + δ .
For i = 0 i=0 i = 0 , j = K j=K j = K , or i = j i=j i = j , both the true profit and its estimate are zero. Thus the bound holds for every e ∈ G K e\in G_K e ∈ G K at every reached scale on the same event, proving the proposition.
Claim 5.5 (Estimating a logarithm from a geometric waiting time). Let Q ∈ ( 0 , 1 ] Q\in(0,1] Q ∈ ( 0 , 1 ] and let N ∼ Geom ( Q ) N\sim\operatorname{Geom}(Q) N ∼ Geom ( Q ) . Then
E [ H N − 1 ] = − log Q , ∥ H N − 1 + log Q ∥ ψ 2 ≤ 4 1 − log Q . \mathbb{E}[H_{N-1}]=-\log Q,\qquad\|H_{N-1}+\log Q\|_{\psi_2}\leq4\sqrt{1-\log Q}. E [ H N − 1 ] = − log Q , ∥ H N − 1 + log Q ∥ ψ 2 ≤ 4 1 − log Q .
Proof. The case Q = 1 Q=1 Q = 1 is immediate, so assume Q < 1 Q<1 Q < 1 . First, taking the expectation yields:
E [ H N − 1 ] = ∑ k ≥ 1 P ( N > k ) k = ∑ k ≥ 1 ( 1 − Q ) k k = − log Q . \mathbb{E}[H_{N-1}]
=\sum_{k\geq1}\frac{\mathbb{P}(N>k)}{k}
=\sum_{k\geq1}\frac{(1-Q)^k}{k}
=-\log Q. E [ H N − 1 ] = k ≥ 1 ∑ k P ( N > k ) = k ≥ 1 ∑ k ( 1 − Q ) k = − log Q .
It remains to bound the centered noise ξ = H N − 1 + log Q \xi=H_{N-1}+\log Q ξ = H N − 1 + log Q . The integral bounds for harmonic sums give
log N ≤ H N − 1 ≤ 1 + log N . \log N\leq H_{N-1}\leq1+\log N. log N ≤ H N − 1 ≤ 1 + log N .
We use these inequalities to control the two tails separately. For the upper tail, H N − 1 ≤ 1 + log N H_{N-1} \le1+\log N H N − 1 ≤ 1 + log N gives, for every u ≥ 0 u \ge0 u ≥ 0 ,
P ( ξ ≥ u ) ≤ P ( N ≥ e u − 1 / Q ) = ( 1 − Q ) ⌈ e u − 1 / Q ⌉ − 1 ≤ e − Q ( ⌈ e u − 1 / Q ⌉ − 1 ) ≤ e Q − e u − 1 ≤ e 1 − e u − 1 . \mathbb{P}(\xi\ge u) \le\mathbb{P}\left(N \ge e^{u-1}/Q\right)
= (1-Q)^{\lceil e^{u-1}/Q\rceil-1}
\le e^{-Q(\lceil e^{u-1}/Q\rceil-1)}
\le e^{Q-e^{u-1}} \le e^{1-e^{u-1}}. P ( ξ ≥ u ) ≤ P ( N ≥ e u − 1 / Q ) = ( 1 − Q ) ⌈ e u − 1 / Q ⌉ − 1 ≤ e − Q (⌈ e u − 1 / Q ⌉ − 1 ) ≤ e Q − e u − 1 ≤ e 1 − e u − 1 .
Here we used 1 − Q ≤ e − Q 1-Q \le e^{-Q} 1 − Q ≤ e − Q , ⌈ e u − 1 / Q ⌉ ≥ e u − 1 / Q \lceil e^{u-1}/Q\rceil\ge e^{u-1}/Q ⌈ e u − 1 / Q ⌉ ≥ e u − 1 / Q , and Q ≤ 1 Q \le1 Q ≤ 1 . For u ≥ 2 u \ge2 u ≥ 2 , the inequality e u − 1 − 1 ≥ u 2 / 4 e^{u-1}-1 \ge u^2/4 e u − 1 − 1 ≥ u 2 /4 bounds the last expression by e − u 2 / 4 e^{-u^2/4} e − u 2 /4 . For 0 ≤ u < 2 0 \le u < 2 0 ≤ u < 2 , we use the trivial probability bound 1. Thus
P ( ξ ≥ u ) ≤ { 1 , 0 ≤ u < 2 , e − u 2 / 4 , u ≥ 2. (87) \mathbb{P}(\xi\ge u) \le
\begin{cases}
1, & 0 \le u < 2,\\
e^{-u^2/4}, & u \ge2.
\end{cases} \tag*{(87)} P ( ξ ≥ u ) ≤ { 1 , e − u 2 /4 , 0 ≤ u < 2 , u ≥ 2. (87)
For the lower tail, H N − 1 ≥ log N H_{N-1} \ge\log N H N − 1 ≥ log N gives, for 0 ≤ u ≤ − log Q 0 \le u \le-\log Q 0 ≤ u ≤ − log Q :
P ( ξ ≤ − u ) ≤ P ( N ≤ e − u / Q ) ≤ Q e − u / Q = e − u . \mathbb{P}(\xi\le-u) \le\mathbb{P}\left(N \le e^{-u}/Q\right) \le Q^{e^{-u}/Q} = e^{-u}. P ( ξ ≤ − u ) ≤ P ( N ≤ e − u / Q ) ≤ Q e − u / Q = e − u .
The second inequality is a union bound over the first ⌈ e − u / Q ⌉ \lceil e^{-u}/Q\rceil ⌈ e − u / Q ⌉ trials, each of which succeeds with probability Q Q Q . In this range, u ≤ 1 − log Q u \le1-\log Q u ≤ 1 − log Q , so e − u ≤ e − u 2 / ( 1 − log Q ) e^{-u} \le e^{-u^2/(1-\log Q)} e − u ≤ e − u 2 / ( 1 − l o g Q ) . For u > − log Q u > -\log Q u > − log Q , we have ξ = H N − 1 + log Q ≥ log Q > − u \xi= H_{N-1}+\log Q \ge\log Q > -u ξ = H N − 1 + log Q ≥ log Q > − u , so P ( ξ ≤ − u ) = 0 \mathbb{P}(\xi\le-u)=0 P ( ξ ≤ − u ) = 0 . Consequently,
P ( ξ ≤ − u ) ≤ e − u 2 / ( 1 − log Q ) ( u ≥ 0 ) . (88) \mathbb{P}(\xi\le-u) \le e^{-u^2/(1-\log Q)}
\qquad(u \ge0). \tag*{(88)} P ( ξ ≤ − u ) ≤ e − u 2 / ( 1 − l o g Q ) ( u ≥ 0 ) . (88)
We now combine Equations (87) and (88) . Since 1 − log Q ≥ 1 1-\log Q \ge1 1 − log Q ≥ 1 , for u ≥ 2 u \ge2 u ≥ 2 , summing over disjoint events gives
P ( ∣ ξ ∣ ≥ u ) ≤ e − u 2 / 4 + e − u 2 / ( 1 − log Q ) ≤ 2 e − u 2 / [ 4 ( 1 − log Q ) ] . \mathbb{P}(|\xi| \ge u) \le e^{-u^2/4}+e^{-u^2/(1-\log Q)}
\le2e^{-u^2/[4(1-\log Q)]}. P ( ∣ ξ ∣ ≥ u ) ≤ e − u 2 /4 + e − u 2 / ( 1 − l o g Q ) ≤ 2 e − u 2 / [ 4 ( 1 − l o g Q )] .
For 0 ≤ u < 2 0 \le u < 2 0 ≤ u < 2 , the above probability is trivially at most 1, whereas 3 e − u 2 / [ 4 ( 1 − log Q ) ] ≥ 3 / e > 1 3e^{-u^2/[4(1-\log Q)]} \ge3/e > 1 3 e − u 2 / [ 4 ( 1 − l o g Q )] ≥ 3/ e > 1 . Hence, for every u ≥ 0 u \ge0 u ≥ 0 we can claim:
P ( ∣ ξ ∣ ≥ u ) ≤ 3 e − u 2 / [ 4 ( 1 − log Q ) ] . \mathbb{P}(|\xi| \ge u) \le3e^{-u^2/[4(1-\log Q)]}. P ( ∣ ξ ∣ ≥ u ) ≤ 3 e − u 2 / [ 4 ( 1 − l o g Q )] .
Applying this tail bound with u = 4 ( 1 − log Q ) log t u=4\sqrt{(1-\log Q)\log t} u = 4 ( 1 − log Q ) log t gives
E [ e ξ 2 / [ 16 ( 1 − log Q ) ] ] = 1 + ∫ 1 ∞ P ( ∣ ξ ∣ > 4 ( 1 − log Q ) log t ) d t \mathbb{E}\left[e^{\xi^2/[16(1-\log Q)]}\right]
=1+\int_1^\infty\mathbb{P}\left(|\xi|>4\sqrt{(1-\log Q)\log t}\right)\,dt E [ e ξ 2 / [ 16 ( 1 − l o g Q )] ] = 1 + ∫ 1 ∞ P ( ∣ ξ ∣ > 4 ( 1 − log Q ) log t ) d t
≤ 1 + 3 ∫ 1 ∞ exp ( − 16 ( 1 − log Q ) log t 4 ( 1 − log Q ) ) d t = 1 + 3 ∫ 1 ∞ t − 4 d t = 2. \le1+3\int_1^\infty\exp\left(-\frac{16(1-\log Q)\log t}{4(1-\log Q)}\right)\,dt
=1+3\int_1^\infty t^{-4}\,dt=2. ≤ 1 + 3 ∫ 1 ∞ exp ( − 4 ( 1 − log Q ) 16 ( 1 − log Q ) log t ) d t = 1 + 3 ∫ 1 ∞ t − 4 d t = 2.
By the definition of the ψ 2 \psi_2 ψ 2 norm, this proves ∥ ξ ∥ ψ 2 ≤ 4 1 − log Q \|\xi\|_{\psi_2}\le4\sqrt{1-\log Q} ∥ ξ ∥ ψ 2 ≤ 4 1 − log Q .
Missing Results from Section 6
Lemma 6.4 (Initialization). Consider Initialize regular running until it is finished, without stopping at T T T . With probability at least 1 − 1 / T 1-1/T 1 − 1/ T , it either returns no trigger, with μ ⋆ = O ( 1 / K ) \mu^\star=O(1/K) μ ⋆ = O ( 1/ K ) , or returns a scale δ > η \delta>\eta δ > η satisfying δ ≤ μ ⋆ < 6 δ \delta\le\mu^\star<6\delta δ ≤ μ ⋆ < 6 δ and a list of N = O ( δ / η log ( 2 T ) ) N=O(\sqrt{\delta/\eta}\log(2T)) N = O ( δ / η log ( 2 T )) fixed candidates ( K i , w i , θ i , P ^ ( w i ) ) (K_i,w_i,\theta_i,\widehat{P}(w_i)) ( K i , w i , θ i , P ( w i )) . Each K i ⊂ int ( T ) K_i\subset\operatorname{int}(\mathcal{T}) K i ⊂ int ( T ) is compact and convex, contains its root w i w_i w i , and has a stored empirical estimate P ^ ( w i ) \widehat{P}(w_i) P ( w i ) of P ( w i ) P(w_i) P ( w i ) . The candidates satisfy:
(i) Auxiliary-profit guarantees. For every candidate i i i and ∣ u ∣ ≤ 10 − 3 |u|\le10^{-3} ∣ u ∣ ≤ 1 0 − 3 , r λ i e u ≥ 0.99 μ ⋆ r_{\lambda_i e^u}\ge0.99\mu^\star r λ i e u ≥ 0.99 μ ⋆ throughout K i K_i K i . Some fixed candidate i ⋆ i^\star i ⋆ satisfies
max x ∈ K i ⋆ r λ i ⋆ e u ( x ) ≥ μ ⋆ − 3 η ( ∣ u ∣ ≤ 10 − 3 η / δ ) . \max_{x\in K_{i^\star}} r_{\lambda_{i^\star}e^u}(x)\ge\mu^\star-3\eta
\qquad(|u|\le10^{-3}\sqrt{\eta/\delta}). x ∈ K i ⋆ max r λ i ⋆ e u ( x ) ≥ μ ⋆ − 3 η ( ∣ u ∣ ≤ 1 0 − 3 η / δ ) .
(ii) Crossed-query bounds. For any two points of K i K_i K i , both crosses belong to T \mathcal{T} T . Their probabilities of trade, and the probabilities at the original points, belong to [ P ^ ( w i ) / 10 , 10 P ^ ( w i ) ] [\widehat{P}(w_i)/10,10\widehat{P}(w_i)] [ P ( w i ) /10 , 10 P ( w i )] . Moreover,
δ ≤ P ^ ( w i ) ≤ 1 , δ q − p ≤ 10 P ^ ( w i ) ( ( p , q ) ∈ K i ) . \delta\le\widehat{P}(w_i) \le1,\qquad\frac{\delta}{q-p} \le10\widehat{P}(w_i)\quad((p,q)\in K_i). δ ≤ P ( w i ) ≤ 1 , q − p δ ≤ 10 P ( w i ) (( p , q ) ∈ K i ) .
On a valid triggered transcript, initialization incurs O ( K log 2 ( T ) ) O(K\log^2(T)) O ( K log 2 ( T )) regret.
Proof. Initialize Regular calls Initialize MHR unchanged to find the profit scale, then performs a separate estimation step at that scale before constructing the candidates. We analyze the steps of Algorithm 7 in order, allowing initialization to run to completion.
Step 1: the profit scale. We give a guarantee on the scale δ \delta δ in relation to the maximal expected profit. For the call to Initialize MHR, the concentration event of Proposition 5.4 has failure probability at most 8 T 3 ( 2 T ) − 4 = 1 / ( 2 T ) 8T^3(2T)^{-4}=1/(2T) 8 T 3 ( 2 T ) − 4 = 1/ ( 2 T ) , by the union bound in its proof. On this event, the scale argument in the proof of Lemma 5.3 gives μ ⋆ < 6 / K \mu^\star<6/K μ ⋆ < 6/ K if there is no trigger, and δ ≤ μ ⋆ < 6 δ \delta\le\mu^\star<6\delta δ ≤ μ ⋆ < 6 δ otherwise.
For the later retention and hull bounds, we also compare the grid optimum μ K ⋆ = max e ∈ G K ∩ T μ ( e ) \mu^\star_K=\max_{e\in G_K\cap\mathcal{T}}\mu(e) μ K ⋆ = max e ∈ G K ∩ T μ ( e ) with μ ⋆ \mu^\star μ ⋆ and δ \delta δ . Lemma A.6 (Discretization Error) gives
μ K ⋆ ≥ μ ⋆ − 2 / K . (89) \mu^\star_K\ge\mu^\star-2/K. \tag*{(89)} μ K ⋆ ≥ μ ⋆ − 2/ K . (89)
At a trigger, the 3 δ / 2 3\delta/2 3 δ /2 threshold and the error bound in Proposition 5.4 give
3 δ 2 ≤ 17 μ K ⋆ + δ 16 ⟹ μ K ⋆ ≥ 23 17 δ . (90) \frac{3\delta}{2}\le\frac{17\mu^\star_K+\delta}{16}\quad\Longrightarrow\quad\mu^\star_K\ge\frac{23}{17}\delta. \tag*{(90)} 2 3 δ ≤ 16 17 μ K ⋆ + δ ⟹ μ K ⋆ ≥ 17 23 δ . (90)
The additional no trigger test checks whether δ < 2 16 / K \delta<2^{16}/K δ < 2 16 / K ; then μ ⋆ < 6 ⋅ 2 16 / K = O ( 1 / K ) \mu^\star<6\cdot2^{16}/K=O(1/K) μ ⋆ < 6 ⋅ 2 16 / K = O ( 1/ K ) . Henceforth assume δ ≥ 2 16 / K ≥ 2 15 η > η \delta\ge2^{16}/K\ge2^{15}\eta>\eta δ ≥ 2 16 / K ≥ 2 15 η > η .
Step 2: more accurate empirical estimates. We use the estimates of Initialize MHR with a smaller error tolerance. Keep its sampled Sierpinski index set S S S , with ∣ S ∣ = O ( K log ( K ) ) |S|=O(K\log(K)) ∣ S ∣ = O ( K log ( K )) , and fix the returned scale δ \delta δ . Post each sampled pair for n = ⌈ 2 38 log ( 2 T ) / δ ⌉ n=\lceil2^{38}\log(2T)/\delta\rceil n = ⌈ 2 38 log ( 2 T ) / δ ⌉ fresh rounds. Write P i j = P ( i / K , j / K ) P_{ij}=P(i/K,j/K) P ij = P ( i / K , j / K ) and let P ‾ i j \overline{P}_{ij} P ij be its empirical trade frequency.
For this concentration calculation, condition on the completed transcript of Initialize MHR, so that δ \delta δ and S S S are fixed; all additional samples are fresh. The Bernstein and union-bound calculation in the proof of Proposition 5.4, with this sample size, gives
∣ P ‾ i j − P i j ∣ ≤ P i j + δ 2 18 ( ( i , j ) ∈ S ) . \left|\overline{P}_{ij}-P_{ij}\right|\le\frac{P_{ij}+\delta}{2^{18}}\qquad((i,j)\in S). P ij − P ij ≤ 2 18 P ij + δ (( i , j ) ∈ S ) .
with conditional failure probability at most 1 / ( 2 T ) 1/(2T) 1/ ( 2 T ) . Let Q Q Q be the event that both the concentration guarantee for the initial call and the displayed bound for the additional samples hold. The first guarantee fails with probability at most 1 / ( 2 T ) 1/(2T) 1/ ( 2 T ) . Averaging the conditional failure bound for the additional samples over the initial transcript gives the same bound 1 / ( 2 T ) 1/(2T) 1/ ( 2 T ) for the second guarantee. A union bound therefore yields
P ( Q c ) ≤ 1 2 T + 1 2 T = 1 T . \mathbb{P}(Q^c)\le\frac{1}{2T}+\frac{1}{2T}=\frac{1}{T}. P ( Q c ) ≤ 2 T 1 + 2 T 1 = T 1 .
Assume Q Q Q for the remainder of the proof.
For each grid pair ( i / K , j / K ) ∈ T (i/K,j/K)\in\mathcal{T} ( i / K , j / K ) ∈ T with 0 < i < j < K 0<i<j<K 0 < i < j < K , independence and monotonicity give
P i j = P i k P k j P k k , P i j ≤ P i k , P k j ≤ P k k , (91) P_{ij}=\frac{P_{ik}P_{kj}}{P_{kk}},\qquad P_{ij}\le P_{ik},P_{kj}\le P_{kk}, \tag*{(91)} P ij = P k k P ik P k j , P ij ≤ P ik , P k j ≤ P k k , (91)
where all three pairs on the right are sampled. We use exactly the empirical reconstruction (50) from Initialize MHR, which always belongs to [ 0 , 1 ] [0,1] [ 0 , 1 ] . The reconstruction calculation in the proof of Proposition 5.4 applies with frequency-error coefficient 2 − 18 2^{-18} 2 − 18 in place of 1 / 128 1/128 1/128 : its split at P k k = δ / 32 P_{kk}=\delta/32 P k k = δ /32 becomes P k k = 4 ⋅ 2 − 18 δ P_{kk}=4\cdot2^{-18}\delta P k k = 4 ⋅ 2 − 18 δ , and both cases give error at most 8 ⋅ 2 − 18 ( P i j + δ ) 8\cdot2^{-18}(P_{ij}+\delta) 8 ⋅ 2 − 18 ( P ij + δ ) . Since 8 ⋅ 2 − 18 = 2 − 15 8\cdot2^{-18}=2^{-15} 8 ⋅ 2 − 18 = 2 − 15 , multiplying by 0 ≤ q − p ≤ 1 0\le q-p\le1 0 ≤ q − p ≤ 1 gives the bound on ∣ μ ^ ( e ) − μ ( e ) ∣ |\widehat{\mu}(e)-\mu(e)| ∣ μ ( e ) − μ ( e ) ∣ in (92) : for e = ( p , q ) e=(p,q) e = ( p , q ) , use ( q − p ) P ( e ) = μ ( e ) (q-p)P(e)=\mu(e) ( q − p ) P ( e ) = μ ( e ) and ( q − p ) δ ≤ δ (q-p)\delta\le\delta ( q − p ) δ ≤ δ . With the prescribed diagonal and boundary estimates, these bounds hold for every e ∈ G K ∩ T e\in G_K\cap\mathcal{T} e ∈ G K ∩ T :
0 ≤ P ^ ( e ) ≤ 1 , ∣ P ^ ( e ) − P ( e ) ∣ ≤ 2 − 15 ( P ( e ) + δ ) , ∣ μ ^ ( e ) − μ ( e ) ∣ ≤ 2 − 15 ( μ ( e ) + δ ) , (92) \begin{aligned}
0&\le\widehat{P}(e)\le1,\qquad|\widehat{P}(e)-P(e)|\le2^{-15}(P(e)+\delta),\\
|\widehat{\mu}(e)-\mu(e)|&\le2^{-15}(\mu(e)+\delta), \tag*{(92)}
\end{aligned} 0 ∣ μ ( e ) − μ ( e ) ∣ ≤ P ( e ) ≤ 1 , ∣ P ( e ) − P ( e ) ∣ ≤ 2 − 15 ( P ( e ) + δ ) , ≤ 2 − 15 ( μ ( e ) + δ ) , (92)
where μ ^ ( p , q ) = ( q − p ) P ^ ( p , q ) \widehat{\mu}(p,q)=(q-p)\widehat{P}(p,q) μ ( p , q ) = ( q − p ) P ( p , q ) . In particular,
( 1 − 2 − 14 ) P ( e ) ≤ P ^ ( e ) ≤ ( 1 + 2 − 14 ) P ( e ) if P ( e ) ≥ δ . (93) (1-2^{-14})P(e)\le\widehat{P}(e)\le(1+2^{-14})P(e)\qquad\text{if }P(e)\ge\delta. \tag*{(93)} ( 1 − 2 − 14 ) P ( e ) ≤ P ( e ) ≤ ( 1 + 2 − 14 ) P ( e ) if P ( e ) ≥ δ . (93)
Step 3: retained prices. We show that every grid maximizer is retained and every retained pair has nearly optimal profit. By (90) , δ ≤ μ K ⋆ \delta\le\mu_K^\star δ ≤ μ K ⋆ . Thus (92) implies
∣ μ ^ ( e ) − μ ( e ) ∣ ≤ 2 − 14 μ K ⋆ ( e ∈ G K ∩ T ) , (94) |\widehat{\mu}(e)-\mu(e)|\le2^{-14}\mu_K^\star\qquad(e\in G_K\cap\mathcal{T}), \tag*{(94)} ∣ μ ( e ) − μ ( e ) ∣ ≤ 2 − 14 μ K ⋆ ( e ∈ G K ∩ T ) , (94)
and consequently
( 1 − 2 − 14 ) μ K ⋆ ≤ μ ^ K ⋆ ≤ ( 1 + 2 − 14 ) μ K ⋆ , μ ^ K ⋆ = max e ∈ G K ∩ T μ ^ ( e ) . (95) (1-2^{-14})\mu_K^\star\le\widehat{\mu}_K^\star\le(1+2^{-14})\mu_K^\star,\qquad
\widehat{\mu}_K^\star=\max_{e\in G_K\cap\mathcal{T}}\widehat{\mu}(e). \tag*{(95)} ( 1 − 2 − 14 ) μ K ⋆ ≤ μ K ⋆ ≤ ( 1 + 2 − 14 ) μ K ⋆ , μ K ⋆ = e ∈ G K ∩ T max μ ( e ) . (95)
The retention threshold 1 − 2 − 13 1-2^{-13} 1 − 2 − 13 leaves room for errors in both directions. Indeed, at a grid maximizer e ⋆ e^\star e ⋆ ,
μ ^ ( e ⋆ ) ≥ ( 1 − 2 − 14 ) μ K ⋆ ≥ ( 1 − 2 − 13 ) ( 1 + 2 − 14 ) μ K ⋆ ≥ ( 1 − 2 − 13 ) μ ^ K ⋆ . \widehat{\mu}(e^\star)\ge(1-2^{-14})\mu_K^\star\ge(1-2^{-13})(1+2^{-14})\mu_K^\star\ge(1-2^{-13})\widehat{\mu}_K^\star. μ ( e ⋆ ) ≥ ( 1 − 2 − 14 ) μ K ⋆ ≥ ( 1 − 2 − 13 ) ( 1 + 2 − 14 ) μ K ⋆ ≥ ( 1 − 2 − 13 ) μ K ⋆ .
Conversely, every retained pair e e e satisfies
μ ( e ) ≥ μ ^ ( e ) − 2 − 14 μ K ⋆ by xref‘(91)‘ ≥ ( ( 1 − 2 − 13 ) ( 1 − 2 − 14 ) − 2 − 14 ) μ K ⋆ by retention and xref‘(92)‘ ≥ ( 1 − 2 − 12 ) μ K ⋆ ≥ ( 1 − 0.0003 ) μ ⋆ . \begin{aligned}
\mu(e)&\ge\widehat{\mu}(e)-2^{-14}\mu_K^\star&&\text{by {xref}`(91)`}\\
&\ge\bigl((1-2^{-13})(1-2^{-14})-2^{-14}\bigr)\mu_K^\star&&\text{by retention and {xref}`(92)`}\\
&\ge(1-2^{-12})\mu_K^\star\ge(1-0.0003)\mu^\star.
\end{aligned} μ ( e ) ≥ μ ( e ) − 2 − 14 μ K ⋆ ≥ ( ( 1 − 2 − 13 ) ( 1 − 2 − 14 ) − 2 − 14 ) μ K ⋆ ≥ ( 1 − 2 − 12 ) μ K ⋆ ≥ ( 1 − 0.0003 ) μ ⋆ . by xref ‘(91)‘ by retention and xref ‘(92)‘
The last inequality uses (89) : its relative loss is at most 2 / ( K μ ⋆ ) ≤ 2 − 15 2/(K\mu^\star)\le2^{-15} 2/ ( K μ ⋆ ) ≤ 2 − 15 , since μ ⋆ ≥ δ ≥ 2 16 / K \mu^\star\ge\delta\ge2^{16}/K μ ⋆ ≥ δ ≥ 2 16 / K , and 2 − 12 + 2 − 15 < 0.0003 2^{-12}+2^{-15}<0.0003 2 − 12 + 2 − 15 < 0.0003 . We have therefore proved
arg max e ∈ G K ∩ T μ ( e ) ⊆ { e ∈ G K ∩ T : μ ^ ( e ) ≥ ( 1 − 2 − 13 ) μ ^ K ⋆ } ⊆ { e ∈ G K ∩ T : μ ( e ) ≥ ( 1 − 0.0003 ) μ ⋆ } . (96) \underset{e\in G_K\cap\mathcal{T}}{\operatorname{arg\,max}}\ \mu(e)
\subseteq\{e\in G_K\cap\mathcal{T}:\widehat{\mu}(e)\ge(1-2^{-13})\widehat{\mu}_K^\star\}
\subseteq\{e\in G_K\cap\mathcal{T}:\mu(e)\ge(1-0.0003)\mu^\star\}. \tag*{(96)} e ∈ G K ∩ T arg max μ ( e ) ⊆ { e ∈ G K ∩ T : μ ( e ) ≥ ( 1 − 2 − 13 ) μ K ⋆ } ⊆ { e ∈ G K ∩ T : μ ( e ) ≥ ( 1 − 0.0003 ) μ ⋆ } . (96)
Moreover, this implies that every retained pair belongs to int ( T ) \operatorname{int}(\mathcal{T}) int ( T ) .
Step 4: grouping by balance. We first check that each assigned pair has balance close to its root’s. Process the retained pairs in a fixed order. Assign x = ( p , q ) x=(p,q) x = ( p , q ) to an existing root w = ( u , v ) w=(u,v) w = ( u , v ) if both crosses belong to T \mathcal{T} T , both have estimated profit at least 0.99 μ ^ K ⋆ 0.99\widehat{\mu}_K^\star 0.99 μ K ⋆ , and
∣ 1 2 log P ^ ( p , v ) P ^ ( u , q ) ∣ ≤ 0.01. (97) \left|\frac{1}{2}\log\frac{\widehat{P}(p,v)}{\widehat{P}(u,q)}\right|\le0.01. \tag*{(97)} 2 1 log P ( u , q ) P ( p , v ) ≤ 0.01. (97)
If no root passes these tests, start a new group with root x x x . For a cross y y y passing the profit test, (94) and (95) give
P ( y ) ≥ μ ( y ) ≥ 0.99 μ ^ K ⋆ − 2 − 14 μ K ⋆ ≥ ( 0.99 ( 1 − 2 − 14 ) − 2 − 14 ) μ K ⋆ > δ , P(y)\ge\mu(y)\ge0.99\widehat{\mu}_K^\star-2^{-14}\mu_K^\star\ge\bigl(0.99(1-2^{-14})-2^{-14}\bigr)\mu_K^\star>\delta, P ( y ) ≥ μ ( y ) ≥ 0.99 μ K ⋆ − 2 − 14 μ K ⋆ ≥ ( 0.99 ( 1 − 2 − 14 ) − 2 − 14 ) μ K ⋆ > δ ,
using (90) in the last step. The relative-error bound (93) therefore applies to both crosses. Combining it with the balance test (97) and identity (49) gives
∣ log λ x λ w ∣ ≤ 0.01 + 1 2 log 1 + 2 − 14 1 − 2 − 14 < 0.011 (98) \left|\log\frac{\lambda_x}{\lambda_w}\right|
\le0.01+\frac{1}{2}\log\frac{1+2^{-14}}{1-2^{-14}}<0.011 \tag*{(98)} log λ w λ x ≤ 0.01 + 2 1 log 1 − 2 − 14 1 + 2 − 14 < 0.011 (98)
for every pair in the group rooted at w w w . We use this bound in Step 5 to show that these pairs have high auxiliary profit at the common parameter λ w \lambda_w λ w .
To bound the number of groups, we first show that any existing root w w w with ∣ log ( λ x / λ w ) ∣ ≤ 0.005 \lvert\log(\lambda_x/\lambda_w)\rvert\le0.005 ∣ log ( λ x / λ w )∣ ≤ 0.005 satisfies the feasibility, estimated-profit, and estimated-balance conditions in Step 4 for assigning x x x to its group. Both x x x and w w w are retained, so their actual profit gaps are at most 0.0003 μ ⋆ 0.0003\mu^\star 0.0003 μ ⋆ by (96) . At the root, λ w \lambda_w λ w is exactly balanced, so r λ w ( w ) = μ ( w ) r_{\lambda_w}(w)=\mu(w) r λ w ( w ) = μ ( w ) and μ ⋆ − r λ w ( w ) ≤ 0.0003 μ ⋆ \mu^\star-r_{\lambda_w}(w)\le0.0003\mu^\star μ ⋆ − r λ w ( w ) ≤ 0.0003 μ ⋆ . At x x x , using λ w \lambda_w λ w instead of λ x \lambda_x λ x introduces an additional loss bounded by Lemma 6.2:
μ ⋆ − r λ w ( x ) = ( μ ⋆ − μ ( x ) ) + ( μ ( x ) − r λ w ( x ) ) ≤ 0.0003 μ ⋆ + μ ( x ) ( log λ w λ x ) 2 ≤ ( 0.0003 + 0.005 2 ) μ ⋆ = 0.000325 μ ⋆ . \begin{aligned}
\mu^\star-r_{\lambda_w}(x)&=(\mu^\star-\mu(x))+(\mu(x)-r_{\lambda_w}(x))\\
&\le0.0003\mu^\star+\mu(x)\left(\log\frac{\lambda_w}{\lambda_x}\right)^2\\
&\le(0.0003+0.005^2)\mu^\star=0.000325\mu^\star.
\end{aligned} μ ⋆ − r λ w ( x ) = ( μ ⋆ − μ ( x )) + ( μ ( x ) − r λ w ( x )) ≤ 0.0003 μ ⋆ + μ ( x ) ( log λ x λ w ) 2 ≤ ( 0.0003 + 0.00 5 2 ) μ ⋆ = 0.000325 μ ⋆ .
Both auxiliary profits exceed 0.99 μ ⋆ 0.99\mu^\star 0.99 μ ⋆ , so Lemma 6.3 makes both crosses feasible with gaps at most 4 ( 0.0003 + 0.000325 ) μ ⋆ = 0.0025 μ ⋆ 4(0.0003+0.000325)\mu^\star=0.0025\mu^\star 4 ( 0.0003 + 0.000325 ) μ ⋆ = 0.0025 μ ⋆ . We now verify the two estimated-profit tests. For either cross y y y , its price difference is at most one, and (90) gives
P ( y ) ≥ μ ( y ) ≥ ( 1 − 0.0025 ) μ ⋆ ≥ ( 1 − 0.0025 ) 23 17 δ > δ . P(y)\ge\mu(y)\ge(1-0.0025)\mu^\star\ge(1-0.0025)\frac{23}{17}\delta>\delta. P ( y ) ≥ μ ( y ) ≥ ( 1 − 0.0025 ) μ ⋆ ≥ ( 1 − 0.0025 ) 17 23 δ > δ .
Thus (93) applies to both crosses and yields
μ ^ ( y ) ≥ ( 1 − 2 − 14 ) μ ( y ) ≥ ( 1 − 2 − 14 ) ( 1 − 0.0025 ) μ ⋆ > 0.99 ( 1 + 2 − 14 ) μ ⋆ ≥ 0.99 μ ^ K ⋆ , \begin{aligned}
\widehat{\mu}(y)&\ge(1-2^{-14})\mu(y)\ge(1-2^{-14})(1-0.0025)\mu^\star\\
&>0.99(1+2^{-14})\mu^\star\ge0.99\widehat{\mu}^\star_K,
\end{aligned} μ ( y ) ≥ ( 1 − 2 − 14 ) μ ( y ) ≥ ( 1 − 2 − 14 ) ( 1 − 0.0025 ) μ ⋆ > 0.99 ( 1 + 2 − 14 ) μ ⋆ ≥ 0.99 μ K ⋆ ,
where the last inequality uses (95) . The balance test also passes: the same probability-estimate comparison used above, together with (49) , gives
∣ 1 2 log P ^ ( p , v ) P ^ ( u , q ) ∣ ≤ ∣ log λ x λ w ∣ + 1 2 log 1 + 2 − 14 1 − 2 − 14 ≤ 0.005 + 1 2 log 1 + 2 − 14 1 − 2 − 14 < 0.01. \left|\frac{1}{2}\log\frac{\widehat{P}(p,v)}{\widehat{P}(u,q)}\right|
\le\left|\log\frac{\lambda_x}{\lambda_w}\right|+\frac{1}{2}\log\frac{1+2^{-14}}{1-2^{-14}}
\le0.005+\frac{1}{2}\log\frac{1+2^{-14}}{1-2^{-14}}<0.01. 2 1 log P ( u , q ) P ( p , v ) ≤ log λ w λ x + 2 1 log 1 − 2 − 14 1 + 2 − 14 ≤ 0.005 + 2 1 log 1 − 2 − 14 1 + 2 − 14 < 0.01.
Consequently, a retained pair within log distance 0.005 0.005 0.005 of an existing root joins an existing group. Since a new root is created only when all existing roots fail the tests, distinct roots have log balances more than 0.005 0.005 0.005 apart.
To bound the interval containing these log balances, (96) and μ ⋆ ≥ 2 16 / K \mu^\star\ge2^{16}/K μ ⋆ ≥ 2 16 / K from Step 1 give μ ( x ) ≥ ( 1 − 0.0003 ) μ ⋆ ≥ 1 / K \mu(x)\ge(1-0.0003)\mu^\star\ge1/K μ ( x ) ≥ ( 1 − 0.0003 ) μ ⋆ ≥ 1/ K at every retained pair. Each marginal acceptance probability is at least this profit and at most one. Hence (38) gives
∣ log λ x ∣ = 1 2 ∣ log F s ( p ) 1 − F b ( q ) ∣ ≤ 1 2 log K . \lvert\log\lambda_x\rvert=\frac{1}{2}\left\lvert\log\frac{F_s(p)}{1-F_b(q)}\right\rvert\le\frac{1}{2}\log K. ∣ log λ x ∣ = 2 1 log 1 − F b ( q ) F s ( p ) ≤ 2 1 log K .
Thus all root log balances lie in an interval of length log K \log K log K . Since they are more than 0.005 0.005 0.005 apart, there are at most 1 + log K / 0.005 = O ( log ( T ) ) 1+\log K/0.005=O(\log(T)) 1 + log K /0.005 = O ( log ( T )) roots, using K ≤ T K\le T K ≤ T .
Step 5: convex hulls. We now extend the profit and balance guarantees to every point of each convex hull. Each group’s convex hull is compact, contains its root, and lies in int ( T ) \operatorname{int}(T) int ( T ) . Every vertex x x x is a retained pair in the group rooted at w w w . Its profit is at least ( 1 − 0.0003 ) μ ⋆ (1-0.0003)\mu^\star ( 1 − 0.0003 ) μ ⋆ by (96) , and its log balance is within 0.011 0.011 0.011 of the root’s by (98) . The balancing identity (39) therefore gives
r λ w ( x ) = μ ( x ) sech 2 ( log λ w λ x ) ≥ ( 1 − 0.0003 ) μ ⋆ sech 2 ( 0.011 ) ≥ ( 1 − 1 / 2000 ) μ ⋆ . (99) \begin{aligned}
r_{\lambda_w}(x)&=\mu(x)\operatorname{sech}^2\left(\log\frac{\lambda_w}{\lambda_x}\right)\\
&\ge(1-0.0003)\mu^\star\operatorname{sech}^2(0.011)\ge(1-1/2000)\mu^\star. \tag*{(99)}
\end{aligned} r λ w ( x ) = μ ( x ) sech 2 ( log λ x λ w ) ≥ ( 1 − 0.0003 ) μ ⋆ sech 2 ( 0.011 ) ≥ ( 1 − 1/2000 ) μ ⋆ . (99)
By (99) , the reciprocal 1 / r λ w 1/r_{\lambda_w} 1/ r λ w is at most 1 / ( ( 1 − 1 / 2000 ) μ ⋆ ) 1/((1-1/2000)\mu^\star) 1/ (( 1 − 1/2000 ) μ ⋆ ) at every vertex. Its convexity, from Lemma 6.2, extends this upper bound throughout the hull, and hence extends the corresponding lower bound on r λ w r_{\lambda_w} r λ w .
Since μ ( x ) ≤ μ ⋆ \mu(x) \le\mu^\star μ ( x ) ≤ μ ⋆ , (39) then gives tanh 2 ( log λ x λ w ) = 1 − r λ w ( x ) μ ( x ) ≤ 1 / 2000 \tanh^2\left(\log\frac{\lambda_x}{\lambda_w}\right) = 1 - \frac{r_{\lambda_w}(x)}{\mu(x)} \le1/2000 tanh 2 ( log λ w λ x ) = 1 − μ ( x ) r λ w ( x ) ≤ 1/2000 . Thus ∣ log ( λ x / λ w ) ∣ ≤ D \left|\log(\lambda_x/\lambda_w)\right| \le D ∣ log ( λ x / λ w ) ∣ ≤ D throughout the hull. For any offset ∣ θ i ∣ ≤ D \lvert\theta_i\rvert\le D ∣ θ i ∣ ≤ D selected in Step 6 and ∣ u ∣ ≤ 10 − 3 \lvert u\rvert\le10^{-3} ∣ u ∣ ≤ 1 0 − 3 , the triangle inequality gives
∣ log e θ i + u λ w λ x ∣ ≤ ∣ θ i ∣ + ∣ u ∣ + ∣ log λ w λ x ∣ ≤ 2 D + 10 − 3 . \left|\log\frac{e^{\theta_i+u}\lambda_w}{\lambda_x}\right| \le\lvert\theta_i\rvert+ \lvert u\rvert+ \left|\log\frac{\lambda_w}{\lambda_x}\right| \le2D + 10^{-3}. log λ x e θ i + u λ w ≤ ∣ θ i ∣ + ∣ u ∣ + log λ x λ w ≤ 2 D + 1 0 − 3 .
Using μ ( x ) ≥ r λ w ( x ) ≥ ( 1 − 1 / 2000 ) μ ⋆ \mu(x) \ge r_{\lambda_w}(x) \ge(1-1/2000)\mu^\star μ ( x ) ≥ r λ w ( x ) ≥ ( 1 − 1/2000 ) μ ⋆ in (39) , it follows that
r e θ i + u λ w ( x ) ≥ ( 1 − 1 / 2000 ) μ ⋆ sech 2 ( 2 D + 10 − 3 ) > 0.99 μ ⋆ . r_{e^{\theta_i+u}\lambda_w}(x) \ge(1-1/2000)\mu^\star\operatorname{sech}^2(2D+10^{-3}) > 0.99\mu^\star. r e θ i + u λ w ( x ) ≥ ( 1 − 1/2000 ) μ ⋆ sech 2 ( 2 D + 1 0 − 3 ) > 0.99 μ ⋆ .
This proves the first assertion of part {(i)}.
For part {(ii)}, we first compare probabilities with the true root probability P ( w ) P(w) P ( w ) . By (99) , every hull point has auxiliary profit above 0.99 μ ⋆ 0.99\mu^\star 0.99 μ ⋆ at the common parameter λ w \lambda_w λ w , so Lemma 6.3 applies to any two hull points. Applying it to x x x and w w w gives
P ( w ) 3 ≤ P ( x ) ≤ 3 P ( w ) . \frac{P(w)}{3} \le P(x) \le3P(w). 3 P ( w ) ≤ P ( x ) ≤ 3 P ( w ) .
Applying it to x x x and another hull point z z z makes both crosses feasible and gives P ( x ) / 3 ≤ P ( y ) ≤ 3 P ( x ) P(x)/3 \le P(y) \le3P(x) P ( x ) /3 ≤ P ( y ) ≤ 3 P ( x ) for either cross y y y . Combining the two comparisons yields
P ( w ) 9 ≤ P ( x ) 3 ≤ P ( y ) ≤ 3 P ( x ) ≤ 9 P ( w ) . \frac{P(w)}{9} \le\frac{P(x)}{3} \le P(y) \le3P(x) \le9P(w). 9 P ( w ) ≤ 3 P ( x ) ≤ P ( y ) ≤ 3 P ( x ) ≤ 9 P ( w ) .
Thus the original points and their crosses all have probabilities between P ( w ) / 9 P(w)/9 P ( w ) /9 and 9 P ( w ) 9P(w) 9 P ( w ) .
Next we verify δ ≤ P ^ ( w ) ≤ 1 \delta\le\widehat{P}(w) \le1 δ ≤ P ( w ) ≤ 1 and replace P ( w ) P(w) P ( w ) by its stored empirical estimate. The root’s price difference is at most one, so the retention rule, (95) , and (90) give
P ^ ( w ) ≥ μ ^ ( w ) ≥ ( 1 − 2 − 13 ) μ ^ K ⋆ ≥ ( 1 − 2 − 13 ) ( 1 − 2 − 14 ) μ K ⋆ > δ . \widehat{P}(w) \ge\widehat{\mu}(w) \ge(1-2^{-13})\widehat{\mu}^{\star}_K \ge(1-2^{-13})(1-2^{-14})\mu^\star_K > \delta. P ( w ) ≥ μ ( w ) ≥ ( 1 − 2 − 13 ) μ K ⋆ ≥ ( 1 − 2 − 13 ) ( 1 − 2 − 14 ) μ K ⋆ > δ .
Also P ^ ( w ) ≤ 1 \widehat{P}(w) \le1 P ( w ) ≤ 1 by construction. The retained-profit bound (96) and (90) give P ( w ) ≥ μ ( w ) > δ P(w) \ge\mu(w) > \delta P ( w ) ≥ μ ( w ) > δ , so (93) implies
P ^ ( w ) 1 + 2 − 14 ≤ P ( w ) ≤ P ^ ( w ) 1 − 2 − 14 . \frac{\widehat{P}(w)}{1+2^{-14}} \le P(w) \le\frac{\widehat{P}(w)}{1-2^{-14}}. 1 + 2 − 14 P ( w ) ≤ P ( w ) ≤ 1 − 2 − 14 P ( w ) .
Combining this with the factor-9 comparison above yields, for every original or crossed pair y y y ,
P ^ ( w ) 9 ( 1 + 2 − 14 ) ≤ P ^ ( w ) 9 ( 1 + 2 − 14 ) ≤ P ( y ) ≤ 9 P ^ ( w ) 1 − 2 − 14 < 10 P ^ ( w ) . \frac{\widehat{P}(w)}{9(1+2^{-14})} \le\frac{\widehat{P}(w)}{9(1+2^{-14})} \le P(y) \le\frac{9\widehat{P}(w)}{1-2^{-14}} < 10\widehat{P}(w). 9 ( 1 + 2 − 14 ) P ( w ) ≤ 9 ( 1 + 2 − 14 ) P ( w ) ≤ P ( y ) ≤ 1 − 2 − 14 9 P ( w ) < 10 P ( w ) .
Finally, we prove the bound on δ / ( q − p ) \delta/(q-p) δ / ( q − p ) . By (99) , actual profit throughout the hull satisfies μ ( p , q ) ≥ r λ w ( p , q ) ≥ ( 1 − 1 / 2000 ) μ ⋆ > δ \mu(p,q) \ge r_{\lambda_w}(p,q) \ge(1-1/2000)\mu^\star> \delta μ ( p , q ) ≥ r λ w ( p , q ) ≥ ( 1 − 1/2000 ) μ ⋆ > δ ; the last inequality uses μ ⋆ ≥ ( 23 / 17 ) δ \mu^\star\ge(23/17)\delta μ ⋆ ≥ ( 23/17 ) δ from (90) . Consequently,
δ q − p = δ μ ( p , q ) P ( p , q ) ≤ P ( p , q ) ≤ 10 P ^ ( w ) . \frac{\delta}{q-p} = \frac{\delta}{\mu(p,q)}P(p,q) \le P(p,q) \le10\widehat{P}(w). q − p δ = μ ( p , q ) δ P ( p , q ) ≤ P ( p , q ) ≤ 10 P ( w ) .
This proves part {(ii)}.
Step 6: offsets and a near-optimal candidate. We show that the offset mesh gives few candidates while preserving a nearly optimal one. Divide [ − D , D ] [-D,D] [ − D , D ] into equal intervals of length at most 1 2 η / δ \frac{1}{2}\sqrt{\eta/\delta} 2 1 η / δ , using the smallest possible number, and use their endpoints as offsets. Each root gives O ( δ / η ) O(\sqrt{\delta/\eta}) O ( δ / η ) candidates, hence N = O ( δ / η log ( 2 T ) ) N = O(\sqrt{\delta/\eta}\log(2T)) N = O ( δ / η log ( 2 T )) in total. To identify the near-optimal candidate, choose a grid maximizer e ⋆ e^\star e ⋆ , retained by (96) . By the balance bound in Step 5, its log balancing parameter relative to its assigned root lies in [ − D , D ] [-D,D] [ − D , D ] . Choose the nearest mesh endpoint as the offset. Its distance from this log balancing parameter is at most half the interval length, so this fixes a candidate i ⋆ i^\star i ⋆ with
∣ log λ i ⋆ λ e ⋆ ∣ ≤ 1 4 η / δ . \left|\log\frac{\lambda_{i^\star}}{\lambda_{e^\star}}\right| \le\frac{1}{4}\sqrt{\eta/\delta}. log λ e ⋆ λ i ⋆ ≤ 4 1 η / δ .
For every ∣ u ∣ ≤ 10 − 3 η / δ |u| \le10^{-3}\sqrt{\eta/\delta} ∣ u ∣ ≤ 1 0 − 3 η / δ , the triangle inequality bounds the perturbed mismatch by ( 1 4 + 10 − 3 ) η / δ (\frac{1}{4}+10^{-3})\sqrt{\eta/\delta} ( 4 1 + 1 0 − 3 ) η / δ . Thus (39) , (89) , and μ ⋆ < 6 δ \mu^\star< 6\delta μ ⋆ < 6 δ give
r λ i ⋆ e u ( e ⋆ ) ≥ μ ⋆ − 2 K − 6 δ ( 1 4 + 10 − 3 ) 2 η δ ≥ μ ⋆ − 3 η . r_{\lambda_{i^\star}e^u}(e^\star) \ge\mu^\star- \frac{2}{K} - 6\delta\left(\frac{1}{4}+10^{-3}\right)^2\frac{\eta}{\delta} \ge\mu^\star- 3\eta. r λ i ⋆ e u ( e ⋆ ) ≥ μ ⋆ − K 2 − 6 δ ( 4 1 + 1 0 − 3 ) 2 δ η ≥ μ ⋆ − 3 η .
The last inequality uses 1 / K ≤ η 1/K \le\eta 1/ K ≤ η and 6 ( 1 4 + 10 − 3 ) 2 < 1 6(\frac{1}{4}+10^{-3})^2 < 1 6 ( 4 1 + 1 0 − 3 ) 2 < 1 . This proves the remaining assertion of part (i).
Regret. Only Steps 1 and 2 use trading rounds. The proof of Lemma 5.3 shows that, when Initialize MHR returns scale δ \delta δ , it has used O ( K log 2 ( T ) / δ ) O(K\log^2(T)/\delta) O ( K log 2 ( T ) / δ ) trading rounds in total. The additional sampling in Step 2 uses n ∣ S ∣ = O ( K log 2 ( T ) / δ ) n|S| = O(K\log^2(T)/\delta) n ∣ S ∣ = O ( K log 2 ( T ) / δ ) further trading rounds. All remaining steps use the stored estimates. Every posted pair is feasible and has regret at most μ ⋆ < 6 δ \mu^\star< 6\delta μ ⋆ < 6 δ , giving O ( K log 2 ( T ) ) O(K\log^2(T)) O ( K log 2 ( T )) regret. Together with P ( Q ) ≥ 1 − 1 / T \mathbb{P}(Q) \ge1 - 1/T P ( Q ) ≥ 1 − 1/ T , this proves the lemma. □ \square □
The Zooming Dimension of Bilateral Trade
In this appendix, we show that the tighter notion of zooming dimension used in Kleinberg et al. [30 ] does not improve over the “trivial” T 3 / 4 T^{3/4} T 3/4 regret bound for the stochastic setting. This implies that the generalized notion for the adversarial setting [39 ] does not help as well.
We then focus on the stochastic i.i.d. setting, and write Δ ( x ) = μ ⋆ − μ ( x ) \Delta(x) = \mu^\star- \mu(x) Δ ( x ) = μ ⋆ − μ ( x ) for the suboptimality gap, and N r ( S ) N_r(S) N r ( S ) for the minimum number of sets of diameter less than r r r covering S S S . The stochastic zooming dimension z z z in Kleinberg et al. [30, Definition 4.1] controls
N r / 8 { x : r / 2 < Δ ( x ) ≤ r } ≤ C 0 r − z . N_{r/8}\{x : r/2 < \Delta(x) \le r\} \le C_0r^{-z}. N r /8 { x : r /2 < Δ ( x ) ≤ r } ≤ C 0 r − z .
Their Theorem 4.2 gives regret O ~ ( T z + 1 / z + 2 ) \widetilde{O}(T^{z+1/z+2}) O ( T z + 1/ z + 2 ) . Their metric-only refinement, the max-min-covering dimension, is still 2 on T \mathcal{T} T : every nonempty relatively open subset of this triangle contains a two-dimensional ball, and the triangle itself has covering dimension 2.
Podimata and Slivkins [39 ] replace stochastic gaps by realized average gaps over prefixes of the horizon. Their adversarial zooming dimension covers arms whose gap is O ~ ( r ) \widetilde{O}(r) O ( r ) on some prefix of length Ω ( r − 2 ) \Omega(r^{-2}) Ω ( r − 2 ) , using a finite representative set. It yields the same exponent z + 1 / z + 2 z+1/z+2 z + 1/ z + 2 . In the stochastic case their comparison lemma identifies this exponent with that obtained from the sublevel sets { Δ ≤ O ( r ) } \{\Delta\le O(r)\} { Δ ≤ O ( r )} , allowing polylogarithmic changes in the covering multiplier. We distinguish these sublevel sets from the shells used above.
One-sided discretization is not a symmetric Lipschitz assumption. Before addressing the zooming dimension bound, we remark that profit is not even Lipschitz, but only one sided. For deterministic valuations ( s , b ) = ( 1 / 4 , 3 / 4 ) (s,b) = (1/4, 3/4) ( s , b ) = ( 1/4 , 3/4 ) , the optimal pair ( 1 / 4 , 3 / 4 ) (1/4,3/4) ( 1/4 , 3/4 ) earns 1 / 2 1/2 1/2 , whereas ( 1 / 4 − η , 3 / 4 ) (1/4-\eta,3/4) ( 1/4 − η , 3/4 ) earns zero for every 0 < η ≤ 1 / 4 0 < \eta\le1/4 0 < η ≤ 1/4 . Thus even the optimal-arm condition Δ ( x ) ≤ L ∥ x − x ⋆ ∥ \Delta(x) \le L\lVert x-x^\star\rVert Δ ( x ) ≤ L ∥ x − x ⋆ ∥ , allowed by Kleinberg et al. [30, Theorem 4.3], fails for every fixed L L L . The directed rounding argument in § 1.2 avoids this problem by increasing the seller price and decreasing the buyer price. This observation alone does not dispose of Podimata and Slivkins [39 ] : their full version also treats one-sided Lipschitzness for one-sided pricing. The next example isolates the dimension obstruction even when the mean profit is Lipschitz.
Proposition B.1. There is a joint distribution of valuations in [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 whose mean profit is Lipschitz on T \mathcal{T} T , has a unique maximizer, and has stochastic zooming dimension 2 in the price metric. The sublevel-set covering exponent is also 2.
Proof. Define the rectangle R = [ 0 , 1 / 4 ] × [ 3 / 4 , 1 ] R = [0,1/4] \times[3/4,1] R = [ 0 , 1/4 ] × [ 3/4 , 1 ] , the reference price x 0 = ( 1 / 8 , 7 / 8 ) x_0 = (1/8,7/8) x 0 = ( 1/8 , 7/8 ) , and the function
h ( x ) = { e − 1 / ∥ x − x 0 ∥ 2 2 , x ≠ x 0 , 0 , x = x 0 , h(x) =
\begin{cases}
e^{-1/\lVert x-x_0\rVert_2^2}, & x \ne x_0,\\
0, & x = x_0,
\end{cases} h ( x ) = { e − 1/ ∥ x − x 0 ∥ 2 2 , 0 , x = x 0 , x = x 0 ,
Consider the joint distribution with trade probability H H H defined as follows on R R R :
H ( p , q ) = c ⋅ 1 − h ( p , q ) q − p , H(p,q) = c \cdot\frac{1-h(p,q)}{q-p}, H ( p , q ) = c ⋅ q − p 1 − h ( p , q ) ,
where c c c is a normalization constant equal to 1 / [ 2 ( 1 − e − 32 ) ] 1/[2(1-e^{-32})] 1/ [ 2 ( 1 − e − 32 )] . It can be verified analytically that there exists an explicit distribution supported on R \mathbb{R} R whose trade probability is H H H .
For arbitrary prices, define p ′ = min { p , 1 / 4 } p'=\min\{p,1/4\} p ′ = min { p , 1/4 } and q ′ = max { q , 3 / 4 } q'=\max\{q,3/4\} q ′ = max { q , 3/4 } . The resulting trade probability is H ( p ′ , q ′ ) H(p',q') H ( p ′ , q ′ ) , so it is Lipschitz on [ 0 , 1 ] 2 [0,1]^2 [ 0 , 1 ] 2 . For generic prices ( p , q ) ∈ T (p,q)\in\mathcal{T} ( p , q ) ∈ T , we have the following bound on the expected profit
μ ( p , q ) = ( q − p ) H ( p ′ , q ′ ) ≤ ( q ′ − p ′ ) H ( p ′ , q ′ ) = c ( 1 − h ( p ′ , q ′ ) ) ≤ c . \mu(p,q)=(q-p)H(p',q')\le(q'-p')H(p',q')=c(1-h(p',q'))\le c. μ ( p , q ) = ( q − p ) H ( p ′ , q ′ ) ≤ ( q ′ − p ′ ) H ( p ′ , q ′ ) = c ( 1 − h ( p ′ , q ′ )) ≤ c .
Equality holds only at x 0 x_0 x 0 , so that the expected gap in R R R has the following clean form Δ ( x ) = c ⋅ h ( x ) \Delta(x)=c\cdot h(x) Δ ( x ) = c ⋅ h ( x ) ; outside R R R it is bounded away from zero because the projected point lies on the boundary of R R R . Consequently, for all sufficiently small r r r , the shell { r / 2 < Δ ≤ r } \{r/2<\Delta\le r\} { r /2 < Δ ≤ r } is an annulus centered at x 0 x_0 x 0 , of area
π ( 1 log ( c / r ) − 1 log ( 2 c / r ) ) = π log 2 log ( c / r ) log ( 2 c / r ) = Θ ( 1 log 2 ( 1 / r ) ) . \pi\left(\frac{1}{\log(c/r)}-\frac{1}{\log(2c/r)}\right)=\frac{\pi\log2}{\log(c/r)\log(2c/r)}=\Theta\left(\frac{1}{\log^2(1/r)}\right). π ( log ( c / r ) 1 − log ( 2 c / r ) 1 ) = log ( c / r ) log ( 2 c / r ) π log 2 = Θ ( log 2 ( 1/ r ) 1 ) .
A set of diameter r / 8 r/8 r /8 has area O ( r 2 ) O(r^2) O ( r 2 ) , hence the shell’s covering number is at least Ω ( r − 2 / log 2 ( 1 / r ) ) \Omega(r^{-2}/\log^2(1/r)) Ω ( r − 2 / log 2 ( 1/ r )) . This exceeds C 0 r − z C_0r^{-z} C 0 r − z for every fixed C 0 C_0 C 0 and every z < 2 z<2 z < 2 . The ambient two-dimensional covering bound supplies the reverse exponent. Likewise, { Δ ≤ r } \{\Delta\le r\} { Δ ≤ r } is a disk of area π / log ( c / r ) \pi/\log(c/r) π / log ( c / r ) , so its covering exponent is 2 as well.
Thus the stochastic zooming bound retains the exponent 3 / 4 3/4 3/4 on an admissible instance with correlated valuations. The stochastic specialization of the adversarial zooming bound has the same obstruction, including its finite-horizon multiplier. To see this quantitatively, take r = T − 1 / 4 r=T^{-1/4} r = T − 1/4 for sufficiently large T T T . The sublevel disk requires
N r ( { Δ ≤ r } ) = Ω ( T log T ) . N_r(\{\Delta\le r\})=\Omega\left(\frac{\sqrt{T}}{\log T}\right). N r ({ Δ ≤ r }) = Ω ( log T T ) .
Consequently, an exponent z < 2 z<2 z < 2 in a bound N r ≤ γ ( T ) r − z N_r\le\gamma(T)r^{-z} N r ≤ γ ( T ) r − z requires
γ ( T ) = Ω ( T ( 2 − z ) / 4 log T ) . \gamma(T)=\Omega\left(\frac{T^{(2-z)/4}}{\log T}\right). γ ( T ) = Ω ( log T T ( 2 − z ) /4 ) .
This is an admissible scale for the stochastic comparison in Podimata and Slivkins [39 ] : r − 2 = T 1 / 2 ≤ T r^{-2}=T^{1/2}\le T r − 2 = T 1/2 ≤ T , the representative mesh of order 1 / T 1/T 1/ T resolves the disk, and the uniform stochastic fluctuation on their polynomial-size representative set at time T T T is O ( log T / T ) = o ( r ) O(\sqrt{\log T/T})=o(r) O ( log T / T ) = o ( r ) . Their comparison transfers the covering obstruction to the adversarial notion up to polylogarithmic factors. The multiplier in their regret bound enters as γ 1 / ( z + 2 ) \gamma^{1/(z+2)} γ 1/ ( z + 2 ) , so the corresponding bound retains a factor of order at least
Ω ~ ( T ( z + 1 ) / ( z + 2 ) T ( 2 − z ) / ( 4 ( z + 2 ) ) ) = Ω ~ ( T 3 / 4 ) . \widetilde{\Omega}\left(T^{(z+1)/(z+2)}T^{(2-z)/(4(z+2))}\right)=\widetilde{\Omega}\left(T^{3/4}\right). Ω ( T ( z + 1 ) / ( z + 2 ) T ( 2 − z ) / ( 4 ( z + 2 )) ) = Ω ( T 3/4 ) .
This concerns the size of the dimension-based guarantee, not the actual regret. In particular, a dimension bound of at most 1 with a polylogarithmic multiplier, which would yield our worst-case exponent 2 / 3 2/3 2/3 , cannot follow from the bilateral-trade structure alone.
Complete Analysis of the Stochastic Warm-Up
We give the full successive-elimination proof of the warm-up result in the stochastic setting. Recall that in this setting the seller and buyer valuations may be arbitrarily correlated within a round, but i.i.d. across rounds.
Fix an integer K K K and recall the (discretized) action space
A = G K ∩ T = { ( i / K , j / K ) : 0 ≤ i ≤ j ≤ K } . \mathcal{A}=G_K\cap\mathcal{T}=\{(i/K,j/K):0\le i\le j\le K\}. A = G K ∩ T = {( i / K , j / K ) : 0 ≤ i ≤ j ≤ K } .
For a valuation distribution Q Q Q , write μ Q ( a ) \mu_Q(a) μ Q ( a ) for the expected profit of a a a , μ Q ⋆ = max a ∈ A μ Q ( a ) \mu_Q^\star=\max_{a\in\mathcal{A}}\mu_Q(a) μ Q ⋆ = max a ∈ A μ Q ( a ) , and Δ Q ( a ) = μ Q ⋆ − μ Q ( a ) \Delta_Q(a)=\mu_Q^\star-\mu_Q(a) Δ Q ( a ) = μ Q ⋆ − μ Q ( a ) for its suboptimality gap when the agents’ valuations are drawn according to Q Q Q . Finally, recall that the discretization lemma (Lemma A.6 ) tells us that if we fix upfront the grid step-size 1 / K 1/K 1/ K and pay the additive 2 T / K 2T/K 2 T / K in the regret, then we can assume that the underlying distribution D \mathcal{D} D is supported on the uniform grid G K G_K G K , without loss of generality.
The local estimation primitive. We repeatedly use the Localized Sampling Lemma (Lemma 4.8 , Remark 4.11) for various reference distributions Q \mathcal{Q} Q , scales ε \varepsilon ε , and failure probabilities δ ′ \delta' δ ′ . In each one of these calls, it estimates μ D ( a ) \mu_{\mathcal{D}}(a) μ D ( a ) , for the real underlying distribution D \mathcal{D} D , to error ε / 64 \varepsilon/64 ε /64 , simultaneously on
S ( Q , ε ) = { a ∈ A : Δ Q ( a ) ≤ ε } , S(\mathcal{Q},\varepsilon)=\{a\in\mathcal{A}:\Delta_{\mathcal{Q}}(a)\le\varepsilon\}, S ( Q , ε ) = { a ∈ A : Δ Q ( a ) ≤ ε } ,
while querying only actions satisfying Δ Q ( a ) ≤ 2 ε \Delta_{\mathcal{Q}}(a)\le2\varepsilon Δ Q ( a ) ≤ 2 ε . In each one of these calls, the number of queries is bounded as follows (for some universal constant C C C ):
N ( ε , δ ′ ) ≤ C log 2 ( K ) log ( K / δ ′ ) ( K ε 2 + log 3 ( K ) ε 3 ) . (100) N(\varepsilon,\delta')\le C\log^{2}(K)\log(K/\delta')\left(\frac{K}{\varepsilon^{2}}+\frac{\log^{3}(K)}{\varepsilon^{3}}\right). \tag*{(100)} N ( ε , δ ′ ) ≤ C log 2 ( K ) log ( K / δ ′ ) ( ε 2 K + ε 3 log 3 ( K ) ) . (100)
Confidence sets and phases. Let H = ⌊ log 4 K ⌋ H=\lfloor\log_{4}K\rfloor H = ⌊ log 4 K ⌋ , set ε h = 4 − h \varepsilon_h=4^{-h} ε h = 4 − h for h = 0 , … , H h=0,\ldots,H h = 0 , … , H , and let δ = T − 2 \delta=T^{-2} δ = T − 2 . Initially, C 0 \mathcal{C}_0 C 0 is the simplex of all distributions on the finite grid. Phase h h h consists of the following steps.
Choose any Q h ∈ C h \mathcal{Q}_h\in\mathcal{C}_h Q h ∈ C h , and set S h = S ( Q h , ε h ) S_h=S(\mathcal{Q}_h,\varepsilon_h) S h = S ( Q h , ε h ) .
Run the local estimator from Lemma 4.8 with reference Q h \mathcal{Q}_h Q h , scale ε h \varepsilon_h ε h , and failure probability δ / ( H + 1 ) \delta/(H+1) δ / ( H + 1 ) . Write μ ^ h ( a ) \widehat{\mu}_h(a) μ h ( a ) for its estimates on S h S_h S h .
Intersect the current confidence set with the new constraints:
C h + 1 = { Q ∈ C h : ∣ μ Q ( a ) − μ ^ h ( a ) ∣ ≤ ε h / 64 for every a ∈ S h } . (101) \mathcal{C}_{h+1}=\{\mathcal{Q}\in\mathcal{C}_h:|\mu_{\mathcal{Q}}(a)-\widehat{\mu}_h(a)|\le\varepsilon_h/64\text{ for every }a\in S_h\}. \tag*{(101)} C h + 1 = { Q ∈ C h : ∣ μ Q ( a ) − μ h ( a ) ∣ ≤ ε h /64 for every a ∈ S h } . (101)
If the horizon ends during a phase, stop immediately. If every phase finishes, use an empirical maximizer of μ ^ H \widehat{\mu}_H μ H on S H S_H S H for the remaining rounds. If a confidence set is empty, use any fixed action in A \mathcal{A} A thereafter. Let E \mathcal{E} E be the event that every completed estimator satisfies its simultaneous error guarantee. Conditional on the history at the start of a phase, Q h \mathcal{Q}_h Q h is fixed and all subsequent observations are generated by fresh samples from D \mathcal{D} D . Thus the estimator guarantee and a union bound give P ( E ) ≥ 1 − δ \mathbb{P}(\mathcal{E})\ge1-\delta P ( E ) ≥ 1 − δ . Conditioning on E \mathcal{E} E , the true distribution D \mathcal{D} D always belongs to the maintained confidence sets C h \mathcal{C}_h C h , so the fallback for an empty set is never used. The enlarged query region makes it necessary to retain all confidence constraints, including those from earlier phases. Indeed, Equation (101) implicitly considers the previous filtering of distributions, and thus the old constraints. Once an action is excluded, these constraints prevent a later reference distribution from making it look promising again.
Lemma C.1. For every completed phase h h h and every Q ∈ C h + 1 \mathcal{Q}\in\mathcal{C}_{h+1} Q ∈ C h + 1 ,
a ∉ S h ⟹ Δ Q ( a ) > 3 4 ε h . (102) a\notin S_h\quad\Longrightarrow\quad\Delta_{\mathcal{Q}}(a)>\frac{3}{4}\varepsilon_h. \tag*{(102)} a ∈ / S h ⟹ Δ Q ( a ) > 4 3 ε h . (102) Consequently, for h ≥ 1 h\ge1 h ≥ 1 , every query in phase h h h belongs to S h − 1 S_{h-1} S h − 1 . Conditioning on E \mathcal{E} E , each such query a a a has true gap
Δ D ( a ) ≤ 9 4 ε h . (103) \Delta_{\mathcal{D}}(a)\le\frac{9}{4}\varepsilon_h. \tag*{(103)} Δ D ( a ) ≤ 4 9 ε h . (103) The same bound holds in phase zero.
Proof. We start from the first statement, proving it by induction on the level. For h = 0 h=0 h = 0 , there is nothing to prove as ε 0 = 1 \varepsilon_0=1 ε 0 = 1 and all profits in A \mathcal{A} A lie in [ 0 , 1 ] [0,1] [ 0 , 1 ] , S 0 = A S_0=\mathcal{A} S 0 = A and Equation (102) holds vacuously. Suppose now that Equation (102) holds at the level h − 1 h-1 h − 1 . Since Q h ∈ C h \mathcal{Q}_h\in\mathcal{C}_h Q h ∈ C h , all actions outside S h − 1 S_{h-1} S h − 1 have Q h \mathcal{Q}_h Q h -gap greater than 3 ε h − 1 / 4 = 3 ε h 3\varepsilon_{h-1}/4=3\varepsilon_h 3 ε h − 1 /4 = 3 ε h . Thus S h ⊆ S h − 1 S_h\subseteq S_{h-1} S h ⊆ S h − 1 , every maximizer of μ ^ h \widehat{\mu}_h μ h belongs to S h − 1 S_{h-1} S h − 1 , and every query in phase h h h , whose Q h \mathcal{Q}_h Q h -gap is at most 2 ε h 2\varepsilon_h 2 ε h , belongs to S h − 1 S_{h-1} S h − 1 as well. Fix Q ∈ C h + 1 \mathcal{Q}\in\mathcal{C}_{h+1} Q ∈ C h + 1 . If a ∉ S h − 1 a\notin S_{h-1} a ∈ / S h − 1 , the induction hypothesis and C h + 1 ⊆ C h \mathcal{C}_{h+1}\subseteq\mathcal{C}_h C h + 1 ⊆ C h already give Δ Q ( a ) > 3 ε h − 1 / 4 > 3 ε h / 4 \Delta_{\mathcal{Q}}(a)>3\varepsilon_{h-1}/4>3\varepsilon_h/4 Δ Q ( a ) > 3 ε h − 1 /4 > 3 ε h /4 . Otherwise, let a ∈ S h − 1 ∖ S h a\in S_{h-1}\setminus S_h a ∈ S h − 1 ∖ S h and choose a maximizer a h ∗ a_h^* a h ∗ of μ Q h \mu_{\mathcal{Q}_h} μ Q h . Both a a a and a h ∗ a_h^* a h ∗ belong to S h − 1 S_{h-1} S h − 1 , and both Q \mathcal{Q} Q and Q h \mathcal{Q}_h Q h satisfy the constraints from phase h − 1 h-1 h − 1 . Hence
∣ μ Q ( x ) − μ Q h ( x ) ∣ ≤ ε h − 1 32 ( x ∈ S h − 1 ) . |\mu_{\mathcal{Q}}(x)-\mu_{\mathcal{Q}_h}(x)|\le\frac{\varepsilon_{h-1}}{32}\qquad(x\in S_{h-1}). ∣ μ Q ( x ) − μ Q h ( x ) ∣ ≤ 32 ε h − 1 ( x ∈ S h − 1 ) .
It follows that
Δ Q ( a ) ≥ μ Q ( a h ∗ ) − μ Q ( a ) ≥ Δ Q h ( a ) − ε h − 1 16 > ε h − ε h 4 = 3 4 ε h , (104) \Delta_{\mathcal{Q}}(a)\ge\mu_{\mathcal{Q}}(a_h^*)-\mu_{\mathcal{Q}}(a)\ge\Delta_{\mathcal{Q}_h}(a)-\frac{\varepsilon_{h-1}}{16}>\varepsilon_h-\frac{\varepsilon_h}{4}=\frac{3}{4}\varepsilon_h, \tag*{(104)} Δ Q ( a ) ≥ μ Q ( a h ∗ ) − μ Q ( a ) ≥ Δ Q h ( a ) − 16 ε h − 1 > ε h − 4 ε h = 4 3 ε h , (104)
分分彩计划 completing the induction.
For the regret bound, work on E \mathcal{E} E and fix a phase h ≥ 1 h \ge1 h ≥ 1 . The true distribution D \mathcal{D} D belongs to C h \mathcal{C}_h C h , so its maximizer a ⋆ a^\star a ⋆ also belongs to S h − 1 S_{h-1} S h − 1 , by the induction hypothesis. For any queried action a a a , both a a a and a ⋆ a^\star a ⋆ are in S h − 1 S_{h-1} S h − 1 . The same confidence constraints therefore imply
Δ D ( a ) = μ D ( a ⋆ ) − μ D ( a ) ≤ μ Q h ( a ⋆ ) − μ Q h ( a ) + ε h − 1 16 ≤ 2 ε h + ε h 4 = 9 4 ε h . \Delta_{\mathcal{D}}(a) = \mu_{\mathcal{D}}(a^\star) - \mu_{\mathcal{D}}(a) \le\mu_{\mathcal{Q}_h}(a^\star) - \mu_{\mathcal{Q}_h}(a) + \frac{\varepsilon_{h-1}}{16} \le2\varepsilon_h + \frac{\varepsilon_h}{4} = \frac{9}{4}\varepsilon_h. Δ D ( a ) = μ D ( a ⋆ ) − μ D ( a ) ≤ μ Q h ( a ⋆ ) − μ Q h ( a ) + 16 ε h − 1 ≤ 2 ε h + 4 ε h = 4 9 ε h .
In phase zero the gap is at most 1 ≤ 9 ε 0 / 4 1 \le9\varepsilon_0/4 1 ≤ 9 ε 0 /4 .
Warm-up Result (Stochastic Setting). Consider the problem of learning the profit-maximizing posted prices in bilateral trade in the stochastic setting. There exists a learning algorithm which enjoys O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret.
Proof. Choose K = ⌈ T 1 / 3 ⌉ K = \lceil T^{1/3}\rceil K = ⌈ T 1/3 ⌉ . The bounds above apply also to a phase interrupted by the horizon: the quality of its queries depends only on the confidence constraints from completed phases. By Lemma C.1 and (100) , on E \mathcal{E} E the total conditional mean regret during all estimation phases is at most
∑ h = 0 H 9 4 ε h N ( ε h , δ / H ) ≤ C ′ log 2 ( K ) log ( K H / δ ) ∑ h = 0 H ( K ε h + log 3 ( K ) ε h 2 ) ≤ C ′ ′ log 2 ( K ) log ( K H / δ ) ( K ε H + log 3 ( K ) ε H 2 ) = O ~ ( K 2 ) . \begin{aligned}
\sum_{h=0}^{H} \frac{9}{4}\varepsilon_h N(\varepsilon_h,\delta/H) \le C'\log^2(K)\log(KH/\delta)\sum_{h=0}^{H}\left(\frac{K}{\varepsilon_h}+\frac{\log^3(K)}{\varepsilon_h^2}\right) \\
&\le C''\log^2(K)\log(KH/\delta)\left(\frac{K}{\varepsilon_H}+\frac{\log^3(K)}{\varepsilon_H^2}\right) \\
&= \widetilde{O}(K^2).
\end{aligned} h = 0 ∑ H 4 9 ε h N ( ε h , δ / H ) ≤ C ′ log 2 ( K ) log ( K H / δ ) h = 0 ∑ H ( ε h K + ε h 2 log 3 ( K ) ) ≤ C ′′ log 2 ( K ) log ( K H / δ ) ( ε H K + ε H 2 log 3 ( K ) ) = O ( K 2 ) .
The final equality uses 1 / K ≤ ε H < 4 / K 1/K \le\varepsilon_H < 4/K 1/ K ≤ ε H < 4/ K . If all phases complete, Equation (102) applied to D \mathcal{D} D shows that a true grid optimum lies in S H S_H S H . The empirical maximizer therefore has gap at most ε H / 32 \varepsilon_H/32 ε H /32 , so that the remaining rounds contribute at most O ( T / K ) O(T/K) O ( T / K ) .
On E c \mathcal{E}^c E c , regret is at most T T T , so taking expectations adds at most T δ ≤ 1 T\delta\le1 T δ ≤ 1 . Finally, adding the discretization error 2 T / K 2T/K 2 T / K gives
R T ∈ O ~ ( K 2 + T / K ) = O ~ ( T 2 / 3 ) . R_T \in\widetilde{O}(K^2+T/K)=\widetilde{O}(T^{2/3}). R T ∈ O ( K 2 + T / K ) = O ( T 2/3 ) .