Introduction
Dynamic pricing is a central problem in revenue management. A seller must learn how demand responds to prices while simultaneously using the accumulated information to generate revenue. This learning problem becomes more challenging when customer, product, or market heterogeneity is observed through covariates. Although such information enables personalized pricing, every posted price also determines what the seller subsequently learns. This feedback creates the exploration–exploitation trade-off at the heart of contextual dynamic pricing; see [12 ] for a broad review and [24 ] for applications.
A widely studied framework models a customer’s latent valuation as a finite-dimensional function of the observed covariates plus an additive market shock. In semiparametric formulations, the contextual component is typically linear, whereas the distribution of the market shock is left unspecified ([20 , 22 , 23 , 13 ] ). This framework combines an interpretable representation of customer heterogeneity with a flexible demand curve. Binary purchase feedback dominates this single-index literature. Shape-free regret guarantees are available for Lipschitz links, whereas the higher-order-smooth literature typically couples smoothness with distributional coverage and/or stable revenue geometry.
At Lipschitz smoothness, [25 ] allow arbitrary adaptive contexts and [15 ] allow general valuation classes under independent and identically distributed contexts; both obtain the optimal shape-free two-thirds horizon exponent with binary feedback. For smoother links, recent improvements rely on nondegenerate or feature-diverse stochastic contexts together with stable revenue geometry ([16 ] ; [8 ] ), or on a smooth oracle-price map induced by strong unimodality ([14 ] ).
A particularly consequential restriction is strong unimodality ([26 ] ; [16 ] ; [14 ] ). In the form used by recent smooth-pricing analyses, this condition requires every contextual revenue function to have a unique interior maximizer and its revenue loss to be uniformly comparable to the squared distance from that maximizer. It is therefore considerably stronger than smoothness or a local second-order condition. Strong unimodality rules out separated revenue modes, flat optimal-price regions, and many asymmetric or boundary-optimal revenue landscapes. At the same time, it provides several powerful analytical advantages: the policy can safely localize around one estimated optimizer, price-estimation errors translate into quadratic revenue losses, and the context-dependent optimal price can often be represented by a regular oracle price map. Indeed, these properties are central to the algorithms and improved regret guarantees in [26 ] , [16 ] , and [14 ] .
This paper studies contextual pricing without these structural conveniences. In each period, the seller observes a covariate vector, posts a price, and observes a bounded purchase quantity, which may be binary, discrete, or continuous. Latent valuation is linear in the observed covariates and subject to an unknown market shock. After integrating out the shock, expected demand is an unknown function of the difference between price and a linear contextual valuation index; both the index parameter and the response function are unknown. The usual binary-purchase model is obtained as a special case, but the formulation also accommodates bounded unit sales and purchase volume. We allow the covariates to arrive arbitrarily and impose only Hölder smoothness on the observable response function. The induced revenue function may consequently be multimodal, may contain a flat optimal region, and need not have a unique maximizer. The smoothness assumption also has natural primitive foundations: it follows, for example, when the market-shock distribution is smooth and the quantity-response function has bounded variation, or when the quantity-response function is itself sufficiently smooth.
Removing strong unimodality fundamentally changes the learning problem. Hölder smoothness controls only local approximation and provides no information about the global geometry of revenue. Learning accurately near a provisional optimizer is therefore insufficient: another, statistically similar region may contain a better and well-separated revenue mode. A policy that localizes prematurely may never collect enough information to discover that region. Consequently, the algorithm must preserve and compare candidates across the entire price domain rather than organize its exploration around a single estimated optimum or oracle price map.
A second difficulty arises from the interaction between the unknown valuation parameter and the nonparametric response function. Directly inserting a pilot parameter estimate into a nonparametric regression transfers the pilot error to the fitted response curve at first order. This error can dominate the higher-order approximation accuracy that smoothness would otherwise provide. A natural remedy, used in classical single-index estimation and recent semiparametric pricing analyses, is to profile a nonparametric fit over candidate index parameters and refine the parameter through constrained least squares ([17 ] ; [19 ] ; [18 ] ; [27 ] ; Chen, 2025; [16 ] ). This remedy is statistically and computationally demanding. Because the local fits and their underlying sample assignments vary with the candidate parameter, the resulting objective is generally nonconvex, even in the twice-smooth case. Standard local optimization methods thus do not provide a global-solution guarantee. Moreover, the same observations are used both to construct the nonparametric fit and to evaluate the least-squares criterion, creating a complex dependence structure in online settings and making finite-sample confidence analysis delicate ([16 ] ). These issues motivate an approach that exploits higher-order smoothness without repeatedly solving a nonconvex joint-estimation problem.
Adaptive data collection introduces a third challenge. Posted prices, residual bins, and subsequent exploration decisions all depend on previous outcomes. If past observations are reassigned whenever the valuation estimate changes, membership in a local dataset is determined retrospectively using information that was unavailable when the observation was collected. This destroys the predictable sampling structure needed for martingale concentration and makes standard offline nonparametric guarantees inapplicable.
We address these challenges through a pilot-corrected layered decision-partitioning (LDP) policy. The pilot module uses uniform-price exploration to estimate the linear valuation component and evaluates its uncertainty only along the currently observed covariate direction. A well-covered direction enters the main pricing stage immediately, whereas an insufficiently covered direction triggers additional pilot exploration. This directional mechanism avoids distributional assumptions on the covariate sequence.
During the main pricing stage, prices are represented relative to the estimated valuation index, and demand is learned locally in the resulting residual coordinate. An augmented local-polynomial representation absorbs the leading pilot perturbation into the regression coefficients, leaving only a second-order pilot error in addition to the usual nonparametric approximation error. Thus, the policy benefits from higher-order smoothness without solving the nonconvex constrained least-squares problems used in alternative joint estimation procedures. Each observation is also assigned a permanent layer–bin label before its demand is observed. These labels are never recomputed after the pilot estimate changes, preserving predictable sampling and enabling self-normalized concentration within every local dataset.
Finally, the layered policy maintains candidate actions over the entire residual domain. An action is eliminated only when its optimistic revenue is statistically separated from the best surviving benchmark. This design protects distant modes and flat near-optimal regions and therefore performs global rather than local revenue learning.
Balancing pilot exploration, local approximation, and statistical uncertainty yields the minimax smoothness-dependent horizon rate, up to logarithmic factors, for fixed problem primitives. We establish a matching expected-regret lower bound over an admissible constant-context, binary-demand subclass. The construction begins with a smooth instance whose revenue is flat over a nondegenerate interval and introduces statistically indistinguishable perturbations in separated price regions. Together, the upper and lower bounds characterize the minimax dependence on the horizon for the general revenue class considered here.
Contributions. Our contributions are twofold.
sequences, general Hölder smoothness, and bounded quantity feedback in a translated-residual single-index model without concavity, strong unimodality, or a unique optimal price. Our pilot-corrected LDP policy integrates directional exploration, higher-order local polynomials, predictable permanent labels, and global elimination. Relative to the Lipschitz LDP framework of [15 ] , this replaces an episodic offline pilot and piecewise-constant learning with an online directional pilot, first-order error correction, and higher-order residual learning.
Minimax-optimal regret for shape-free smooth links. For fixed dimension and fixed problem primitives, the proposed policy achieves O ~ ( T β + 1 2 β + 1 ) \widetilde{O}\left(T^{\frac{\beta+1}{2\beta+1}}\right) O ( T 2 β + 1 β + 1 ) regret, matching Ω ( T β + 1 2 β + 1 ) \Omega\left(T^{\frac{\beta+1}{2\beta+1}}\right) Ω ( T 2 β + 1 β + 1 ) lower bound. Together, the upper and lower bounds characterize the minimax horizon dependence for the general revenue class studied in this paper.
Semiparametric contextual dynamic pricing. Contextual pricing with a linear valuation component and an additive shock has been studied under known parametric noise distributions ([20 ] ; [3 ] ), unknown nonparametric noise ([22 ] ; [23 ] ; [13 ] ; [10 ] ; [28 ] ; [8 ] ), and more general valuation or utility classes ([9 ] ; [15 ] ). These works differ in the feedback available, the distributional assumptions on contexts, and the geometry imposed on revenue. The papers most closely related to ours are summarized in Table 1 . The rates in the last column suppress logarithmic factors and polynomial dependence on fixed problem primitives.
Table 1. Representative results for smooth semiparametric contextual pricing. Here β \beta β denotes link or tail smoothness; rates suppress logarithmic factors and dependence on fixed problem primitives.
The principal comparison at β = 1 \beta= 1 β = 1 is [25 ] : it already allows arbitrary adaptive contexts and general revenue geometry, and attains the same T 2 / 3 T^{2/3} T 2/3 horizon exponent, but is restricted to binary feedback and Lipschitz smoothness. [15 ] introduce the LDP architecture for a shape-free Lipschitz problem with binary feedback and i.i.d. contexts. Thus, our claim at β = 1 \beta= 1 β = 1 is not a better horizon exponent. The contribution is to retain arbitrary contexts and shape-free revenue while covering general β \beta β and bounded quantity feedback.
For twice-smooth demand, [26 ] establish the sharp O ~ ( T 3 / 5 ) \widetilde{O}(T^{3/5}) O ( T 3/5 ) regret rate under strong unimodality and nondegenerate i.i.d. contexts, using contextual successive elimination together with semiparametric estimation. [16 ] extend this line to general Hölder smoothness by combining local-polynomial estimation with the stationary learning subroutine of [26 ] , while retaining similar revenue-geometry and context-coverage conditions. [14 ] also impose strong unimodality but allow arbitrary contexts; by directly learning the resulting smooth oracle-price map, they obtain a faster horizon exponent when β > 2 \beta> 2 β > 2 . Our result addresses the complementary regime in which higher-order smoothness is available but the revenue geometry remains unrestricted. The flat-optimum construction underlying our lower bound shows that this distinction is structural: without strong unimodality, the hard instances need not admit a unique and stable oracle-price map on which localization can be based.
Xu and Wang (2026) study an adjacent, nonnested regime: binary feedback with possibly discontinuous demand, including jumps induced by atoms, under stochastic well-conditioned contexts. Their model relaxes link regularity, whereas ours uses Hölder smoothness to handle arbitrary contexts and bounded quantity feedback.
Algorithmically, our closest antecedent is the policy of Gong et al. (2025). Relative to its episodic offline pilot and piecewise-constant residual learning, our policy uses an online directional pilot, higher-order local polynomials, an augmented correction that makes pilot error second order, and permanent residual labels that preserve predictable sampling as the pilot evolves. Uniform-price exploration in Javanmard and Nazerzadeh (2019); Fan et al. (2024); Chen et al. (2024) and the arbitrary-context mechanisms in Tullii et al. (2024) provide additional points of contact.
Nonparametric and multimodal pricing. Without covariates, nonparametric dynamic pricing has been studied under a range of smoothness and shape conditions (Besbes and Zeevi, 2009, 2015; Wang et al., 2021). In particular, Wang et al. (2021) establish the minimax rate O ~ ( T ( β + 1 ) / ( 2 β + 1 ) ) \widetilde{O}(T^{(\beta+1)/(2\beta+1)}) O ( T ( β + 1 ) / ( 2 β + 1 ) ) for smooth multimodal revenue using local-polynomial optimism. Their confidence indices explicitly incorporate a local approximation envelope, and thus require the corresponding smoothness radius.
Bounded quantity observations are not new by themselves. Wang et al. (2021) analyze general demand feedback without contexts, while Bu et al. (2022, 2025) study partially linear and separable contextual demand models. The present model instead places the unknown context effect inside the translated residual p − x T θ ⋆ p-x^{\mathsf{T}}\theta_{\star} p − x T θ ⋆ . Consequently, the valuation shift and the nonparametric link must be learned jointly under an arbitrary context sequence. This translated single-index structure, rather than nonbinary feedback alone, is the relevant distinction.
Layered contextual bandits. Our analysis draws on standard confidence-based methods for contextual bandits (Auer, 2002; Chu et al., 2011; Abbasi-Yadkori et al., 2011; Lattimore and Szepesvári, 2020). Classical layered partitioning separates observations by precision level to obtain sharper confidence control. In the present problem, however, the local regression model itself depends on an evolving pilot index and is only approximately specified. The permanent labeling rule is therefore essential: it fixes the residual coordinate, local bin, and stopping layer used for each observation before the outcome is revealed. This converts the adaptively collected local samples into predictable martingale arrays while retaining global price exploration.
Organization. Section 3 introduces the model, the induced demand link, and its regularity. Section 4 presents the directional pilot, pilot-corrected local regression, and layered pricing policy. Section 5 establishes the regret upper bound and the matching minimax lower bound. The paper concludes with directions for future research.
Notation. Throughout the paper, we write [ n ] : = { 1 , … , n } [n]:=\{1,\ldots,n\} [ n ] := { 1 , … , n } for any positive integer n n n , denote the cardinality of a set A A A by ∣ A ∣ |A| ∣ A ∣ , and use 1 { E } \mathbf{1}\{E\} 1 { E } for the indicator of an event E E E . For vectors, ∥ ⋅ ∥ p \|\cdot\|_{p} ∥ ⋅ ∥ p denotes the standard ℓ p \ell_p ℓ p norm for 1 ≤ p ≤ ∞ 1 \le p \le\infty 1 ≤ p ≤ ∞ . The notation O ~ \widetilde{O} O suppresses absolute constants and logarithmic factors. For a positive definite matrix A A A , define ∥ z ∥ A : = z T A z \|z\|_A := \sqrt{z^{\mathsf{T}}Az} ∥ z ∥ A := z T A z . For an interval I = [ a , b ] I=[a,b] I = [ a , b ] , let Π I ( z ) : = min { b , max { a , z } } \Pi_I(z):=\min\{b,\max\{a,z\}\} Π I ( z ) := min { b , max { a , z }} denote projection onto I I I .
Basic Model. We consider a contextual dynamic pricing problem over a finite horizon T T T . In each period t ∈ [ T ] t \in[T] t ∈ [ T ] , a customer arrives and the seller observes a covariate vector x t ∈ X ⊆ R d \boldsymbol{x}_t \in\mathcal{X} \subseteq\mathbb{R}^d x t ∈ X ⊆ R d that characterizes the observable customer, product, or market characteristics. Unlike standard i.i.d. assumptions, we do not impose any distributional assumption on the covariate sequence { x t } t = 1 T \{\boldsymbol{x}_t\}_{t=1}^T { x t } t = 1 T ; it can be arbitrary and possibly dependent. The covariate space is bounded, i.e., there exists C x < ∞ C_x < \infty C x < ∞ such that sup x ∈ X ∥ x ∥ 2 ≤ C x \sup_{\boldsymbol{x}\in\mathcal{X}}\|\boldsymbol{x}\|_2 \le C_x sup x ∈ X ∥ x ∥ 2 ≤ C x . For each t ∈ [ T ] t \in[T] t ∈ [ T ] , let F t − 1 \mathcal{F}_{t-1} F t − 1 denote the σ \sigma σ -field generated by all observations and policy randomization available up to the end of period t − 1 t-1 t − 1 . Let U t U_t U t collect all within-period randomization used by the policy after observing x t \boldsymbol{x}_t x t . Conditional on F t − 1 ∨ σ ( x t ) \mathcal{F}_{t-1}\vee\sigma(\boldsymbol{x}_t) F t − 1 ∨ σ ( x t ) , the seed U t U_t U t is drawn independently of the current valuation shock.
The customer’s valuation for the product is modeled as a linear function of the covariates plus random noise:
v t = x t T θ ⋆ + ϵ t , v_t = \boldsymbol{x}_t^{\mathsf{T}}\boldsymbol{\theta}_\star+ \epsilon_t, v t = x t T θ ⋆ + ϵ t ,
where θ ⋆ ∈ Θ = { θ ∈ R d : ∥ θ ∥ 2 ≤ C θ } \boldsymbol{\theta}_\star\in\Theta= \{\boldsymbol{\theta} \in\mathbb{R}^d : \|\boldsymbol{\theta}\|_2 \le C_\theta\} θ ⋆ ∈ Θ = { θ ∈ R d : ∥ θ ∥ 2 ≤ C θ } is an unknown parameter vector, and ϵ t \epsilon_t ϵ t is an unobserved random shock. The shocks { ϵ t } t = 1 T \{\epsilon_t\}_{t=1}^T { ϵ t } t = 1 T are independent and identically distributed according to an unknown cumulative distribution function F F F with zero mean. For every t ∈ [ T ] t \in[T] t ∈ [ T ] , ϵ t \epsilon_t ϵ t is independent of F t − 1 ∨ σ ( x t , U t ) \mathcal{F}_{t-1}\vee\sigma(\boldsymbol{x}_t,U_t) F t − 1 ∨ σ ( x t , U t ) .
Assumption 3.1 (Bounded Valuations) There exist B > 0 B>0 B > 0 and B ϵ ∈ ( 0 , B / 2 ) B_\epsilon\in(0,B/2) B ϵ ∈ ( 0 , B /2 ) such that:
The noise is uniformly bounded: ∣ ϵ t ∣ ≤ B ϵ |\epsilon_t| \le B_\epsilon ∣ ϵ t ∣ ≤ B ϵ almost surely.
The linear valuation satisfies x T θ ⋆ ∈ [ B ϵ , B − B ϵ ] \boldsymbol{x}^{\mathsf{T}}\boldsymbol{\theta}_\star\in[B_\epsilon,B-B_\epsilon] x T θ ⋆ ∈ [ B ϵ , B − B ϵ ] for all x ∈ X \boldsymbol{x}\in\mathcal{X} x ∈ X .
Assumption 3.1 guarantees that v t = x t T θ ⋆ + ϵ t ∈ [ 0 , B ] v_t=\boldsymbol{x}_t^{\mathsf{T}}\boldsymbol{\theta}_\star+\epsilon_t\in[0,B] v t = x t T θ ⋆ + ϵ t ∈ [ 0 , B ] almost surely, so the realized valuation and any feasible posted price lie in the same known interval [ 0 , B ] [0,B] [ 0 , B ] . This is a standard and natural requirement in practical pricing applications.
After observing x t \boldsymbol{x}_t x t , the seller posts a per-unit price p t ∈ [ 0 , B ] p_t\in[0,B] p t ∈ [ 0 , B ] . The price and every auxiliary action label constructed by the policy before demand is observed are measurable with respect to the full pre-demand information field G t : = F t − 1 ∨ σ ( x t , U t ) \mathcal{G}_t:=\mathcal{F}_{t-1}\vee\sigma(\boldsymbol{x}_t,U_t) G t := F t − 1 ∨ σ ( x t , U t ) . If the realized valuation satisfies v t ≥ p t v_t\ge p_t v t ≥ p t , a sale occurs and the seller observes a positive purchase quantity y t > 0 y_t>0 y t > 0 ; otherwise, no sale occurs and y t = 0 y_t=0 y t = 0 , i.e., 1 { y t > 0 } = 1 { v t ≥ p t } \mathbb{1}\{y_t>0\}=\mathbb{1}\{v_t\ge p_t\} 1 { y t > 0 } = 1 { v t ≥ p t } almost surely. The observed quantity y t y_t y t may be discrete (e.g., number of units) or continuous (e.g., physical volume or usage), depending on the application. We assume there exists a constant 0 < D < ∞ 0<D<\infty 0 < D < ∞ such that 0 ≤ y t ≤ D 0\le y_t\le D 0 ≤ y t ≤ D almost surely. The realized revenue at time t t t is p t y t p_ty_t p t y t , and each round yields the observation triple ( x t , p t , y t ) (\boldsymbol{x}_t,p_t,y_t) ( x t , p t , y t ) . We set F t : = G t ∨ σ ( y t ) \mathcal{F}_t:=\mathcal{G}_t\vee\sigma(y_t) F t := G t ∨ σ ( y t ) , so G t \mathcal{G}_t G t and F t \mathcal{F}_t F t describe, respectively, all information immediately before and after current demand is observed.
Generalized Single Index Model. The basic model above only specifies the qualitative relationship between y t y_t y t and the surplus v t − p t v_t-p_t v t − p t . However, to enable statistical learning and regret minimization, the seller must quantify how the expected demand varies with the posted price and covariates. To make the problem tractable while retaining flexibility, we adopt a surplus-dependent single-index structure. This formulation generalizes the widely studied binary purchase model ([14 ] ; [16 ] ; [23 ] ; [13 ] ; [15 ] ) by allowing the realized demand quantity to depend on the surplus through a function q ( ⋅ ) q(\cdot) q ( ⋅ ) , rather than restricting to a binary sale indicator.
Specifically, we posit that, conditional on the information available before demand is observed and on the realized valuation, the expected demand depends on the current price and covariates only through the net surplus v t − p t v_t-p_t v t − p t . This single-index form is natural: it implies that a customer’s purchasing quantity decision is driven by the perceived value relative to the price, rather than by their absolute levels separately. Under this premise, there exists an unknown baseline function q ( ⋅ ) q(\cdot) q ( ⋅ ) that maps the surplus to the expected quantity. Importantly, we impose no parametric form, such as a linear or logistic specification, on q q q , allowing it to accommodate richly heterogeneous and potentially nonmonotone demand responses. The formal statement is as follows.
Assumption 3.2 (Surplus-dependent demand response) There exists an unknown measurable function q : R → [ 0 , D ] q:\mathbb{R}\to[0,D] q : R → [ 0 , D ] with q ( z ) = 0 q(z)=0 q ( z ) = 0 for z < 0 z<0 z < 0 or z > B z>B z > B and q ( z ) > 0 q(z)>0 q ( z ) > 0 for z ∈ [ 0 , B ] z\in[0,B] z ∈ [ 0 , B ] , such that, for every t ∈ [ T ] t\in[T] t ∈ [ T ] ,
E [ y t ∣ G t ∨ σ ( v t ) ] = q ( v t − p t ) almost surely. \mathbb{E}[y_t\mid\mathcal{G}_t\vee\sigma(v_t)]=q(v_t-p_t)\quad\text{almost surely.} E [ y t ∣ G t ∨ σ ( v t )] = q ( v t − p t ) almost surely.
Assumption 3.2 formalizes the surplus-dependent single-index structure introduced above. It posits that, conditional on all pre-demand information and the realized valuation, the conditional mean demand is determined solely by the realized surplus v t − p t v_t-p_t v t − p t and the functional form q ( ⋅ ) q(\cdot) q ( ⋅ ) is left fully nonparametric. The zero restriction for negative surplus is consistent with the no-sale condition. The zero extension beyond B B B is only a technical convention, because such surplus cannot arise under Assumption 3.1. For an affordable purchase, the incidence condition implies positive realized quantity, and we choose the version of the conditional-mean function that is strictly positive on [ 0 , B ] [0,B] [ 0 , B ] . Crucially, we impose no monotonicity, concavity, or unimodality assumptions on q q q . This allows q q q to capture rich and potentially irregular demand patterns—such as quantity discounts, satiation effects, or Giffen-like behaviors at the micro level—that are often excluded by parametric specifications.
Define the effective residual domain I g = [ − B + B ϵ , B − B ϵ ] \mathcal{I}_g=[-B+B_{\epsilon},B-B_{\epsilon}] I g = [ − B + B ϵ , B − B ϵ ] . The function q q q characterizes the expected demand conditional on the realized valuation v t = s v_t=s v t = s , which is unobservable to the seller. For decision-making and regret analysis, however, we require the expected demand conditional on the information available before the current demand is observed. To bridge this gap, we integrate out the noise ϵ t \epsilon_t ϵ t from the surplus s − p t = θ ⋆ ⊤ x t + ϵ t − p t s-p_t=\theta_{\star}^{\top}x_t+\epsilon_t-p_t s − p t = θ ⋆ ⊤ x t + ϵ t − p t . This yields an induced link function g g g , defined as the convolution of q q q with the noise distribution F F F :
g ( u ) : = E [ q ( ϵ t − u ) ] , u ∈ I g , g(u):=\mathbb{E}[q(\epsilon_t-u)],\qquad u\in\mathcal{I}_g, g ( u ) := E [ q ( ϵ t − u )] , u ∈ I g ,
where u = p − θ ⋆ ⊤ x u=p-\theta_{\star}^{\top}x u = p − θ ⋆ ⊤ x . Then, by iterated expectations and the sequential exogeneity of ϵ t \epsilon_t ϵ t , we obtain,
E [ y t ∣ G t ] = g ( p t − x t ⊤ θ ⋆ ) . \mathbb{E}[y_t\mid\mathcal{G}_t]=g(p_t-x_t^{\top}\theta_{\star}). E [ y t ∣ G t ] = g ( p t − x t ⊤ θ ⋆ ) .
In this way, g g g serves as the demand link that replaces the latent q q q , and all subsequent estimation and regret guarantees will be stated in terms of g g g .
Revenue and Regret. Given covariate x x x and price p p p , the expected revenue is Rev ( x , p ) = p g ( p − θ ⋆ ⊤ x ) \operatorname{Rev}(x,p)=p\,g(p-\theta_{\star}^{\top}x) Rev ( x , p ) = p g ( p − θ ⋆ ⊤ x ) , Let p ⋆ ( x ) ∈ arg max p ∈ [ 0 , B ] Rev ( x , p ) p^{\star}(x)\in\arg\max_{p\in[0,B]}\operatorname{Rev}(x,p) p ⋆ ( x ) ∈ arg max p ∈ [ 0 , B ] Rev ( x , p ) be an arbitrary maximizer, and write p t ⋆ : = p ⋆ ( x t ) p_t^{\star}:=p^{\star}(x_t) p t ⋆ := p ⋆ ( x t ) for period t t t . The cumulative regret is
Reg ( T ) = ∑ t = 1 T ( Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ) . \operatorname{Reg}(T)=\sum_{t=1}^{T}\left(\operatorname{Rev}(x_t,p_t^{\star})-\operatorname{Rev}(x_t,p_t)\right). Reg ( T ) = t = 1 ∑ T ( Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ) .
The goal is to design a nonanticipating pricing policy that, for each t t t , selects p t p_t p t based on the current covariate x t x_t x t and all past observations { ( x s , p s , y s ) } s = 1 t − 1 \{(x_s,p_s,y_s)\}_{s=1}^{t-1} {( x s , p s , y s ) } s = 1 t − 1 , so as to minimize Reg ( T ) \operatorname{Reg}(T) Reg ( T ) while learning the unknown valuation parameter θ ⋆ \theta_\star θ ⋆ and the induced demand link g g g .
Regularity. To enable nonparametric estimation, we directly assume the observable induced link function g g g belongs to a Hölder class. This direct approach is standard in the dynamic pricing literature ([7 ] ; [27 ] ) and facilitates transparent convergence analysis, since g g g is the direct object of estimation.
Definition 3.3 (Hölder class) Let I ⊂ R \mathcal{I} \subset\mathbb{R} I ⊂ R be a compact interval, β ≥ 1 \beta\ge1 β ≥ 1 , and 0 < L < ∞ 0 < L < \infty 0 < L < ∞ . Define
ϖ ( β ) : = max { k ∈ Z ≥ 0 : k < β } . \varpi(\beta) := \max\{k \in\mathbb{Z}_{\ge0} : k < \beta\}. ϖ ( β ) := max { k ∈ Z ≥ 0 : k < β } .
A function f : I → R f : \mathcal{I} \to\mathbb{R} f : I → R belongs to H ( β , L ; I ) \mathcal{H}(\beta,L;\mathcal{I}) H ( β , L ; I ) if f f f is ϖ ( β ) \varpi(\beta) ϖ ( β ) -times differentiable on I \mathcal{I} I and
∣ f ( ϖ ( β ) ) ( u ) − f ( ϖ ( β ) ) ( u ′ ) ∣ ≤ L ∣ u − u ′ ∣ β − ϖ ( β ) , ∀ u , u ′ ∈ I . \left|f^{(\varpi(\beta))}(u)-f^{(\varpi(\beta))}(u')\right| \le L|u-u'|^{\beta-\varpi(\beta)}, \qquad\forall u,u' \in\mathcal{I}. f ( ϖ ( β )) ( u ) − f ( ϖ ( β )) ( u ′ ) ≤ L ∣ u − u ′ ∣ β − ϖ ( β ) , ∀ u , u ′ ∈ I .
Assumption 3.4 (Smoothness of the Induced Link) The induced link function satisfies g ∈ H ( β , L g ; I g ) g \in\mathcal{H}(\beta,L_g;\mathcal{I}_g) g ∈ H ( β , L g ; I g ) for some β ≥ 1 \beta\ge1 β ≥ 1 and 0 < L g < ∞ 0 < L_g < \infty 0 < L g < ∞ .
For the contextual policy, the smoothness parameters β \beta β and L g L_g L g are treated as known. Throughout this paper, C g C_g C g denotes a fixed, class-level upper envelope for the derivatives of g g g up to order ϖ ( β ) \varpi(\beta) ϖ ( β ) . Because 0 ≤ g ≤ D 0 \le g \le D 0 ≤ g ≤ D on the interval I g \mathcal{I}_g I g , a standard one-dimensional interpolation inequality allows C g C_g C g to be chosen as a function only of B B B , B ϵ B_\epsilon B ϵ , D D D , β \beta β , and L g L_g L g . Thus, the constants used by the policy do not depend on unknown instance-specific derivative values.
When the domain I \mathcal{I} I is clear from the context, we write H ( β ) \mathcal{H}(\beta) H ( β ) for simplicity.
Justification of Assumption 3.4. A natural question is whether directly assuming Hölder smoothness on g g g is too restrictive. In our framework, this assumption is in fact mild, because the convolutional structure g ( u ) = E [ q ( ϵ t − u ) ] g(u)=\mathbb{E}[q(\epsilon_t-u)] g ( u ) = E [ q ( ϵ t − u )] allows g g g to inherit smoothness from more primitive components. The following concrete scenario shows when Assumption 3.4 automatically holds.
Smooth noise distribution. Suppose the noise distribution F F F itself is Hölder smooth, i.e., F ∈ H ( β , L F ; [ − B , B ] ) F \in\mathcal{H}(\beta,L_F;[-B,B]) F ∈ H ( β , L F ; [ − B , B ]) . Even if the nonparametric demand response q q q only has bounded variation (allowing jumps and discontinuities), the convolution with F F F regularizes g g g . The following proposition formalizes this.
Proposition 3.5 Under Assumptions 3.1 and 3.2, and assuming that q q q is of bounded variation on [ 0 , B ] [0,B] [ 0 , B ] , if F ∈ H ( β , L F ; [ − B , B ] ) F \in\mathcal{H}(\beta,L_F;[-B,B]) F ∈ H ( β , L F ; [ − B , B ]) , then g ∈ H ( β , ( D + V q ) L F ; I g ) g \in\mathcal{H}(\beta,(D+V_q)L_F;\mathcal{I}_g) g ∈ H ( β , ( D + V q ) L F ; I g ) , where V q V_q V q denotes the total variation of q q q on [ 0 , B ] [0,B] [ 0 , B ] .
In the special case of binary demand where q ( z ) = 1 { 0 ≤ z ≤ B } q(z)=\mathbf{1}\{0 \le z \le B\} q ( z ) = 1 { 0 ≤ z ≤ B } , the induced link reduces to the survival function g ( u ) = 1 − F ( u ) g(u)=1-F(u) g ( u ) = 1 − F ( u ) whenever F F F is continuous, thereby recovering the classical binary-purchase dynamic pricing model. This special case encompasses a broad range of existing work that assumes F F F to be Lipschitz ([4 ] ; [15 ] ), to have a Lipschitz first derivative ([26 ] ), or to be m m m -times continuously differentiable ([13 ] ; [27 ] ). This example demonstrates that Assumption 3.4 is not an extraneous restriction: it follows from smooth valuation noise even when the quantity response itself has jumps. The model contains the bounded binary-purchase formulation as a special case while relaxing its revenue-geometry restrictions; it does not require the revenue function to be concave or the optimal price to be unique.
Remark 3.6 (Strong unimodality) Several recent studies impose a strong-unimodality condition: for every context x x x , the function Rev ( x , ⋅ ) \operatorname{Rev}(x,\cdot) Rev ( x , ⋅ ) has a unique interior maximizer, and the revenue loss is comparable to the squared distance from that maximizer [26 ] ; [16 ] ; [14 ] . This structure permits the policy to localize around a single price and, under additional smoothness, to learn a stable oracle-price map.
We do not impose strong unimodality. Consequently, Rev ( x , ⋅ ) \operatorname{Rev}(x,\cdot) Rev ( x , ⋅ ) may have multiple separated modes or nonunique maximizers, and small estimation errors may change which price region appears optimal. Localized search and oracle-price-map reductions are therefore not generally valid, motivating the global residual-space learning procedure developed in Section 4.
Algorithm
The policy has three coupled components, summarized in Figure 1 . First, uncertainty-triggered uniform-price exploration estimates the valuation index only along the current context direction. Second, a pilot-corrected local-polynomial feature absorbs the first-order index error, leaving the usual local approximation error and a second-order pilot term. Third, a layered rule compares candidate residual actions globally, rather than localizing around one provisional optimizer.
Figure 1. Overview of the proposed adaptive semiparametric pricing policy. Panel (a) shows the uncertainty-triggered pilot module. Panel (b) illustrates global residual-space learning and the augmented first-order correction for pilot-index error. Panel (c) depicts nested survivor sets under layered UCB exploration and elimination; outline, hatching, and solid fill distinguish active, surviving, and final actions in grayscale.
Every main-policy observation receives its residual-bin and stopping-layer labels before demand is observed, and these labels are never recomputed. The resulting predictable datasets support martingale concentration even though prices and visited bins are selected adaptively. The following subsections define the pilot, correction, confidence radius, and global elimination rule in that order.
Adaptive Pilot Estimation
The pilot module is introduced first because the subsequent residual representation requires an accurate valuation index at the current covariate. Importantly, the policy does not require θ ^ t \hat{\theta}_t θ ^ t to approximate θ ⋆ \theta_\star θ ⋆ uniformly in Euclidean norm. It only needs to certify the scalar index x t ⊤ θ ⋆ x_t^\top\theta_\star x t ⊤ θ ⋆ along the currently observed direction. This context-specific requirement is particularly useful when the covariates need not be independently sampled: directions that are already well covered can immediately enter the main pricing module, whereas insufficiently covered directions trigger additional pilot exploration.
The pilot estimator is constructed exclusively from uniform-price exploration rounds. On such a round, the pseudo-response B 1 { y t > 0 } B\mathbf{1}\{y_t > 0\} B 1 { y t > 0 } is an unbiased observation of x t T θ ⋆ x_t^{\mathsf{T}}\theta_{\star} x t T θ ⋆ . Accordingly, M t M_t M t and b t b_t b t form the regularized design matrix and response vector based on the pilot-exploration data, and θ ^ t = M t − 1 b t \widehat{\theta}_t = M_t^{-1}b_t θ t = M t − 1 b t . The quantity γ T ∥ x t ∥ M t − 1 \gamma_T\lVert x_t\rVert_{M_t^{-1}} γ T ∥ x t ∥ M t − 1 measures the remaining uncertainty in the current valuation index. If this quantity exceeds the target accuracy η \eta η , the policy collects an additional uniform-price observation. Otherwise, it invokes the layered pricing module.
Algorithm 1 Contextual Dynamic Pricing with Adaptive Pilot and Permanent Bin Labels
Require: Pilot accuracy η \eta η , horizon T T T , smoothness β \beta β , bounds C θ , C x , B , D C_{\theta}, C_x, B, D C θ , C x , B , D , number of bins N N N , regularization λ > 0 \lambda> 0 λ > 0 , confidence constants C z , C ψ , C l C_z, C_{\psi}, C_l C z , C ψ , C l , and confidence level δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 )
Set γ T ← B d log ( 1 + T C x 2 / d ) + 2 log ( 4 / δ ) + C θ \gamma_T \leftarrow B\sqrt{d\log(1+TC_x^2/d)+2\log(4/\delta)}+C_{\theta} γ T ← B d log ( 1 + T C x 2 / d ) + 2 log ( 4/ δ ) + C θ and S ← max { 1 , ⌈ log 2 T ⌉ } S \leftarrow\max\{1,\lceil\log_2\sqrt{T}\rceil\} S ← max { 1 , ⌈ log 2 T ⌉} ,
Initialize T exp ← ∅ \mathcal{T}^{\exp} \leftarrow\varnothing T e x p ← ∅ , M 1 ← I d M_1 \leftarrow I_d M 1 ← I d , b 1 ← 0 d b_1 \leftarrow\mathbf{0}_d b 1 ← 0 d , and Ψ 1 , s j ← ∅ \Psi^j_{1,s} \leftarrow\varnothing Ψ 1 , s j ← ∅ for every s ∈ [ S ] s \in[S] s ∈ [ S ] and j ∈ [ N ] j \in[N] j ∈ [ N ]
for t = 1 , … , T t = 1,\ldots,T t = 1 , … , T do
Observe x t x_t x t and set θ ^ t ← M t − 1 b t \widehat{\theta}_t \leftarrow M_t^{-1}b_t θ t ← M t − 1 b t
if γ T ∥ x t ∥ M t − 1 > η \gamma_T\lVert x_t\rVert_{M_t^{-1}} > \eta γ T ∥ x t ∥ M t − 1 > η then
Set T exp ← T exp ∪ { t } \mathcal{T}^{\exp} \leftarrow\mathcal{T}^{\exp} \cup\{t\} T e x p ← T e x p ∪ { t }
Draw p t ∼ Unif [ 0 , B ] p_t \sim\operatorname{Unif}[0,B] p t ∼ Unif [ 0 , B ] , post p t p_t p t , and observe y t ∈ [ 0 , D ] y_t \in[0,D] y t ∈ [ 0 , D ]
Set M t + 1 ← M t + x t x t T M_{t+1} \leftarrow M_t+x_tx_t^{\mathsf{T}} M t + 1 ← M t + x t x t T , b t + 1 ← b t + B 1 { y t > 0 } x t b_{t+1} \leftarrow b_t+B\mathbf{1}\{y_t>0\}x_t b t + 1 ← b t + B 1 { y t > 0 } x t
Set Ψ t + 1 , s j ← Ψ t , s j \Psi^j_{t+1,s} \leftarrow\Psi^j_{t,s} Ψ t + 1 , s j ← Ψ t , s j , for all s ∈ [ S ] s \in[S] s ∈ [ S ] and j ∈ [ N ] j \in[N] j ∈ [ N ]
else
Run Algorithm 2 and obtain ( s t , j t , w t , p t ) (s_t,j_t,w_t,p_t) ( s t , j t , w t , p t )
Record ( s t , j t , w t ) (s_t,j_t,w_t) ( s t , j t , w t ) as the permanent layer, bin, and residual-action labels of period t t t
Post p t p_t p t and observe y t ∈ [ 0 , D ] y_t \in[0,D] y t ∈ [ 0 , D ]
Set M t + 1 ← M t M_{t+1} \leftarrow M_t M t + 1 ← M t , b t + 1 ← b t b_{t+1} \leftarrow b_t b t + 1 ← b t , and
Ψ t + 1 , s j ← { Ψ t , s j ∪ { t } , s = s t , j = j t , Ψ t , s j , otherwise . \Psi^j_{t+1,s} \leftarrow
\begin{cases}
\Psi^j_{t,s} \cup\{t\}, & s=s_t,j=j_t,\\
\Psi^j_{t,s}, & \text{otherwise}.
\end{cases} Ψ t + 1 , s j ← { Ψ t , s j ∪ { t } , Ψ t , s j , s = s t , j = j t , otherwise .
Algorithm 1 gives the outer wrapper of the policy; the layered pricing subroutine is specified subsequently. We refer to rounds on which uniform pricing is used as pilot-exploration rounds, and to all remaining rounds as LDP rounds. Although later pilot-exploration rounds may update θ ^ t \widehat{\theta}_t θ t , the bin and layer labels assigned to an LDP observation are permanent and are never recomputed using a later pilot.
Algorithm 2 assigns each LDP round a stopping layer s t s_t s t and a permanent bin label j t j_t j t . For each pair ( s , j ) (s,j) ( s , j ) , the outer wrapper maintains a dataset Ψ t , s j \Psi^j_{t,s} Ψ t , s j , which stores all past LDP observations carrying that fixed label. The task of selecting ( s t , j t ) (s_t,j_t) ( s t , j t ) is entirely delegated to Algorithm 2; the outer algorithm only appends the current observation to the corresponding Ψ t + 1 , s t j t \Psi^{j_t}_{t+1,s_t} Ψ t + 1 , s t j t .
The following lemma validates the uncertainty gate in Algorithm 1. It provides a time-uniform, direction-dependent confidence bound and therefore guarantees that the valuation index is sufficiently accurate whenever the policy enters an LDP round.
Lemma 4.1 (Pilot estimation confidence) With the value of γ T \gamma_T γ T specified in Algorithm 1, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 , simultaneously for every t ≤ T t \le T t ≤ T and every x ∈ R d x \in\mathbb{R}^d x ∈ R d ,
∣ x T ( θ ^ t − θ ⋆ ) ∣ ≤ γ T ∥ x ∥ M t − 1 . (1) \left|x^{\mathsf{T}}(\widehat{\theta}_t-\theta_{\star})\right| \le\gamma_T\lVert x\rVert_{M_t^{-1}}. \tag*{(1)} x T ( θ t − θ ⋆ ) ≤ γ T ∥ x ∥ M t − 1 . (1)
Consequently, on the event in Lemma 4.1, every LDP round t t t satisfies ∣ Π [ 0 , B ] ( x t T θ ^ t ) − x t T θ ⋆ ∣ ≤ η \left|\Pi_{[0,B]}(x_t^{\mathsf{T}}\widehat{\theta}_t)-x_t^{\mathsf{T}}\theta_{\star}\right| \le\eta Π [ 0 , B ] ( x t T θ t ) − x t T θ ⋆ ≤ η .
Therefore, for every feasible pilot-residual action w w w ,
p ^ t ( w ) − x t T θ ⋆ = w + [ Π [ 0 , B ] ( x t T θ ^ t ) − x t T θ ⋆ ] . \widehat{p}_t(w)-x_t^{\mathsf T}\theta_\star
=
w+\left[\Pi_{[0,B]}(x_t^{\mathsf T}\widehat{\theta}_t)-x_t^{\mathsf T}\theta_\star\right]. p t ( w ) − x t T θ ⋆ = w + [ Π [ 0 , B ] ( x t T θ t ) − x t T θ ⋆ ] .
so the true residual differs from its pilot counterpart w w w by at most η \eta η . This certification localizes the evaluation point of g g g , but it does not by itself eliminate the resulting first-order error. We next incorporate this displacement directly into the local regression feature.
Pilot-Corrected Local Regression
The pilot certificate controls the valuation-index error at the current context, but does not by itself remove its effect on estimating the unknown demand link. On the pilot-confidence event, ∣ p ^ t ( w ) − w − x t T θ ⋆ ∣ ≤ η \left|\widehat{p}_t(w)-w-x_t^{\mathsf T}\theta_\star\right|\leq\eta p t ( w ) − w − x t T θ ⋆ ≤ η . An ordinary local-polynomial regression indexed by w w w ignores this displacement and therefore incurs a first-order error of order η \eta η . When β > 1 \beta>1 β > 1 , this error may dominate the target local-polynomial bias h β h^\beta h β , where h h h denotes the bin width defined below.
Existing smooth semiparametric pricing methods address this coupling by jointly refining the index parameter and the local approximation of g g g [26 , 16 ] . Conditional on a candidate θ \theta θ , both the local regressors and their Gram matrix depend on θ \theta θ . Profiling out the polynomial coefficients consequently leads to a constrained least-squares problem that is generally nonconvex and does not, in general, admit a globally certified solution.
Our key simplification is to target prediction rather than structural joint identification. Pricing requires accurate predictions of g ( p − x T θ ⋆ ) g(p-x^{\mathsf T}\theta_\star) g ( p − x T θ ⋆ ) , but does not require a separate refinement of θ ⋆ \theta_\star θ ⋆ within every local regression. We therefore retain the complete first-order effect of the pilot-index displacement and estimate the resulting products as composite coefficients. This prediction-preserving lifting converts the coupled estimation problem into a convex ridge regression while leaving only a second-order pilot remainder.
To formalize this construction, partition the residual space [ − B , B ] [-B,B] [ − B , B ] 1 into N N N bins of equal width h = 2 B / N h=2B/N h = 2 B / N . Let a j = − B + 2 B j N a_j=-B+\frac{2Bj}{N} a j = − B + N 2 B j , for j = 0 , 1 , … , N j=0,1,\ldots,N j = 0 , 1 , … , N , and define
I j = [ a j − 1 , a j ) , j = 1 , … , N − 1 , I N = [ a N − 1 , a N ] . I_j=[a_{j-1},a_j),\quad j=1,\ldots,N-1,\qquad I_N=[a_{N-1},a_N]. I j = [ a j − 1 , a j ) , j = 1 , … , N − 1 , I N = [ a N − 1 , a N ] .
For each I j I_j I j , define the local-polynomial feature anchored at a j − 1 a_{j-1} a j − 1 by
ϕ j ( u ) : = ( 1 , u − a j − 1 , … , ( u − a j − 1 ) ϖ ( β ) ) T ∈ R 1 + ϖ ( β ) , \phi_j(u):=\left(1,u-a_{j-1},\ldots,(u-a_{j-1})^{\varpi(\beta)}\right)^{\mathsf T}\in\mathbb{R}^{1+\varpi(\beta)}, ϕ j ( u ) := ( 1 , u − a j − 1 , … , ( u − a j − 1 ) ϖ ( β ) ) T ∈ R 1 + ϖ ( β ) ,
and let ϕ j ′ ( u ) \phi'_j(u) ϕ j ′ ( u ) denote its componentwise derivative. When ϖ ( β ) ≥ 1 \varpi(\beta)\geq1 ϖ ( β ) ≥ 1 , define
X j ( x , u ) : = ( x 2 ( u − a j − 1 ) x ⋮ ϖ ( β ) ( u − a j − 1 ) ϖ ( β ) − 1 x ) ∈ R ϖ ( β ) d , \mathbf{X}_j(x,u):=
\begin{pmatrix}
x\\
2(u-a_{j-1})x\\
\vdots\\
\varpi(\beta)(u-a_{j-1})^{\varpi(\beta)-1}x
\end{pmatrix}
\in\mathbb{R}^{\varpi(\beta)d}, X j ( x , u ) := x 2 ( u − a j − 1 ) x ⋮ ϖ ( β ) ( u − a j − 1 ) ϖ ( β ) − 1 x ∈ R ϖ ( β ) d ,
when ϖ ( β ) = 0 \varpi(\beta)=0 ϖ ( β ) = 0 , let X j ( x , u ) \mathbf{X}_j(x,u) X j ( x , u ) be the empty vector.
For w ∈ I j w \in I_j w ∈ I j , expanding around the bin anchor and retaining the first-order pilot displacement gives
g ( p ^ t ( w ) − x t ⊤ θ ⋆ ) = ϕ j ( w ) ⊤ ( g ( a j − 1 ) g ′ ( a j − 1 ) ⋮ g ( ϖ ( β ) ) ( a j − 1 ) ϖ ( β ) ! ) + ( p ^ t ( w ) − w − x t ⊤ θ ⋆ ) ϕ j ′ ( w ) ⊤ ( g ( a j − 1 ) g ′ ( a j − 1 ) ⋮ g ( ϖ ( β ) ) ( a j − 1 ) ϖ ( β ) ! ) + R t , j ( w ) , (2) \begin{aligned}
g\left(\widehat{p}_t(w)-x_t^\top\theta_\star\right)=\phi_j(w)^\top
\begin{pmatrix}
g(a_{j-1})\\
g'(a_{j-1})\\
\vdots\\
\dfrac{g^{(\varpi(\beta))}(a_{j-1})}{\varpi(\beta)!}
\end{pmatrix} \\
&+\left(\widehat{p}_t(w)-w-x_t^\top\theta_\star\right)\phi'_j(w)^\top
\begin{pmatrix}
g(a_{j-1})\\
g'(a_{j-1})\\
\vdots\\
\dfrac{g^{(\varpi(\beta))}(a_{j-1})}{\varpi(\beta)!}
\end{pmatrix}
+R_{t,j}(w), \tag*{(2)}
\end{aligned} g ( p t ( w ) − x t ⊤ θ ⋆ ) = ϕ j ( w ) ⊤ g ( a j − 1 ) g ′ ( a j − 1 ) ⋮ ϖ ( β )! g ( ϖ ( β )) ( a j − 1 ) + ( p t ( w ) − w − x t ⊤ θ ⋆ ) ϕ j ′ ( w ) ⊤ g ( a j − 1 ) g ′ ( a j − 1 ) ⋮ ϖ ( β )! g ( ϖ ( β )) ( a j − 1 ) + R t , j ( w ) , (2)
where, under η ≤ h \eta\le h η ≤ h , ∣ R t , j ( w ) ∣ ≤ C l ( h β + η 2 ) \lvert R_{t,j}(w)\rvert\le C_l\left(h^\beta+\eta^2\right) ∣ R t , j ( w )∣ ≤ C l ( h β + η 2 ) . The two components of the remainder correspond to the local-polynomial approximation error and the second-order pilot-displacement error, respectively.
Hence, the first-order representation in (2) can be rewritten as
[ ϕ j ( w ) + ( p ^ t ( w ) − w ) ϕ j ′ ( w ) ] ⊤ ( g ( a j − 1 ) g ′ ( a j − 1 ) ⋮ g ( ϖ ( β ) ) ( a j − 1 ) ϖ ( β ) ! ) − X j ( x t , w ) ⊤ ( g ′ ( a j − 1 ) θ ⋆ ⋮ g ( ϖ ( β ) ) ( a j − 1 ) ϖ ( β ) ! θ ⋆ ) . \left[\phi_j(w)+\left(\widehat{p}_t(w)-w\right)\phi'_j(w)\right]^\top
\begin{pmatrix}
g(a_{j-1})\\
g'(a_{j-1})\\
\vdots\\
\dfrac{g^{(\varpi(\beta))}(a_{j-1})}{\varpi(\beta)!}
\end{pmatrix}
-X_j(x_t,w)^\top
\begin{pmatrix}
g'(a_{j-1})\theta_\star\\
\vdots\\
\dfrac{g^{(\varpi(\beta))}(a_{j-1})}{\varpi(\beta)!}\theta_\star
\end{pmatrix}. [ ϕ j ( w ) + ( p t ( w ) − w ) ϕ j ′ ( w ) ] ⊤ g ( a j − 1 ) g ′ ( a j − 1 ) ⋮ ϖ ( β )! g ( ϖ ( β )) ( a j − 1 ) − X j ( x t , w ) ⊤ g ′ ( a j − 1 ) θ ⋆ ⋮ ϖ ( β )! g ( ϖ ( β )) ( a j − 1 ) θ ⋆ .
The second term contains products between the local derivative coefficients and θ ⋆ \theta_\star θ ⋆ . Enforcing their common factorization through θ ⋆ \theta_\star θ ⋆ would recover the nonlinear coupling underlying the constrained joint estimators in prior work. Instead, we treat these products directly as composite coefficients.
Specifically, define a 1 + ϖ ( β ) ( d + 1 ) 1+\varpi(\beta)(d+1) 1 + ϖ ( β ) ( d + 1 ) -dimensional column vector
z j : = ( g ( a j − 1 ) , g ′ ( a j − 1 ) , … , g ( ϖ ( β ) ) ( a j − 1 ) ϖ ( β ) ! , ( g ′ ( a j − 1 ) θ ⋆ ) ⊤ , … , ( g ( ϖ ( β ) ) ( a j − 1 ) ϖ ( β ) ! θ ⋆ ) ⊤ ) ⊤ , z_j:=\left(g(a_{j-1}),\ g'(a_{j-1}),\ \ldots,\ \frac{g^{(\varpi(\beta))}(a_{j-1})}{\varpi(\beta)!},\ \left(g'(a_{j-1})\theta_\star\right)^\top,\ \ldots,\ \left(\frac{g^{(\varpi(\beta))}(a_{j-1})}{\varpi(\beta)!}\theta_\star\right)^\top\right)^\top, z j := ( g ( a j − 1 ) , g ′ ( a j − 1 ) , … , ϖ ( β )! g ( ϖ ( β )) ( a j − 1 ) , ( g ′ ( a j − 1 ) θ ⋆ ) ⊤ , … , ( ϖ ( β )! g ( ϖ ( β )) ( a j − 1 ) θ ⋆ ) ⊤ ) ⊤ ,
and, for every t t t , j j j , and w ∈ I j w \in I_j w ∈ I j , define the pilot-corrected feature
ψ t , j ( w ) : = ( ϕ j ( w ) + ( p ^ t ( w ) − w ) ϕ j ′ ( w ) − X j ( x t , w ) ) ∈ R 1 + ϖ ( β ) ( d + 1 ) . (3) \psi_{t,j}(w):=
\begin{pmatrix}
\phi_j(w)+\left(\widehat{p}_t(w)-w\right)\phi'_j(w)\\
-X_j(x_t,w)
\end{pmatrix}
\in\mathbb{R}^{1+\varpi(\beta)(d+1)}. \tag*{(3)} ψ t , j ( w ) := ( ϕ j ( w ) + ( p t ( w ) − w ) ϕ j ′ ( w ) − X j ( x t , w ) ) ∈ R 1 + ϖ ( β ) ( d + 1 ) . (3)
Equation (2) then becomes
g ( p ^ t ( w ) − x t ⊤ θ ⋆ ) = ψ t , j ( w ) ⊤ z j + R t , j ( w ) , ∣ R t , j ( w ) ∣ ≤ C l ( h β + η 2 ) . (4) g\left(\widehat{p}_t(w)-x_t^\top\theta_\star\right)=\psi_{t,j}(w)^\top z_j+R_{t,j}(w),\qquad\lvert R_{t,j}(w)\rvert\le C_l\left(h^\beta+\eta^2\right). \tag*{(4)} g ( p t ( w ) − x t ⊤ θ ⋆ ) = ψ t , j ( w ) ⊤ z j + R t , j ( w ) , ∣ R t , j ( w )∣ ≤ C l ( h β + η 2 ) . (4)
Thus, up to a higher-order deterministic remainder, the conditional mean belongs to a finite-dimensional linear prediction class. When β = 1 \beta=1 β = 1 , we have ϖ ( β ) = 0 \varpi(\beta)=0 ϖ ( β ) = 0 ; the derivative block is absent, and the pilot displacement is absorbed into the O ( h ) O(h) O ( h ) approximation error under η ≤ h \eta\le h η ≤ h .
For each layer–bin pair ( s , j ) (s,j) ( s , j ) , we estimate the composite coefficient using observations carrying the corresponding permanent labels. Define Λ t , s j : = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ψ i , j ( w i ) ⊤ \Lambda^j_{t,s}:=\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)\psi_{i,j}(w_i)^\top Λ t , s j := ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ψ i , j ( w i ) ⊤ . Fix a regularization parameter λ > 0 \lambda>0 λ > 0 . The ridge estimator is
z ^ t , s j : = argmin z ∈ R 1 + ϖ ( β ) ( d + 1 ) { ∑ i ∈ Ψ t , s j ( y i − ψ i , j ( w i ) ⊤ z ) 2 + λ ∥ z ∥ 2 2 } = ( λ I + Λ t , s j ) − 1 ∑ i ∈ Ψ t , s j y i ψ i , j ( w i ) . (5) \widehat{z}^j_{t,s}:=\underset{z\in\mathbb{R}^{1+\varpi(\beta)(d+1)}}{\operatorname{argmin}}\left\{\sum_{i\in\Psi^j_{t,s}}\left(y_i-\psi_{i,j}(w_i)^\top z\right)^2+\lambda\lVert z\rVert_2^2\right\}=(\lambda I+\Lambda^j_{t,s})^{-1}\sum_{i\in\Psi^j_{t,s}}y_i\psi_{i,j}(w_i). \tag*{(5)} z t , s j := z ∈ R 1 + ϖ ( β ) ( d + 1 ) argmin ⎩ ⎨ ⎧ i ∈ Ψ t , s j ∑ ( y i − ψ i , j ( w i ) ⊤ z ) 2 + λ ∥ z ∥ 2 2 ⎭ ⎬ ⎫ = ( λ I + Λ t , s j ) − 1 i ∈ Ψ t , s j ∑ y i ψ i , j ( w i ) . (5)
For w ∈ I j w \in I_j w ∈ I j , the corresponding demand estimate based on Ψ t , s j \Psi^j_{t,s} Ψ t , s j is
g ^ t , s j ( w ) : = Π [ 0 , D ] ( ψ t , j ( w ) ⊤ z ^ t , s j ) . (6) \widehat{g}^j_{t,s}(w) := \Pi_{[0,D]}\left(\psi_{t,j}(w)^\top\widehat{z}^j_{t,s}\right). \tag*{(6)} g t , s j ( w ) := Π [ 0 , D ] ( ψ t , j ( w ) ⊤ z t , s j ) . (6)
The estimator in (5) does not approximate a local solution of the preceding nonconvex joint-estimation problem. Instead, it relaxes the factorization restrictions among the derivative coefficients and estimates the prediction-sufficient composite coefficients directly. The population vector z j z_j z j remains in the lifted class, so this relaxation introduces no additional first-order approximation error. Its deterministic misspecification is bounded by O ( h β + η 2 ) O(h^\beta+\eta^2) O ( h β + η 2 ) .
Once the lifted features have been constructed, (5) is a standard strictly convex ridge regression and therefore has the displayed unique global solution. Moreover, each feature ψ i , j ( w i ) \psi_{i,j}(w_i) ψ i , j ( w i ) is determined before y i y_i y i is observed, allowing the stochastic regression errors to be analyzed as a predictable martingale transform. Under η 2 ≲ h β \eta^2 \lesssim h^\beta η 2 ≲ h β , the lifting preserves the local-polynomial approximation order while avoiding nonconvex joint refinement altogether. The next subsection converts these point predictions into confidence bounds and incorporates them into the layered decision rule.
Layered Decision Making
The preceding pilot-corrected regression converts the coupled semiparametric estimation problem into a convex linear prediction problem and provides a point estimate of demand for each feasible residual action. Point predictions alone, however, are insufficient for sequential pricing: the amount of information varies across layer–bin pairs, and the lifted representation retains a deterministic approximation error of order O ( h β + η 2 ) O(h^\beta+\eta^2) O ( h β + η 2 ) . We therefore construct confidence bounds that jointly account for stochastic estimation error, ridge regularization, and this remaining approximation error, and use them to form optimistic estimates of the corresponding revenues.
Confidence radius. The explicit form of the radius is
r t , s j ( w ) = min { D , C ψ D ι T ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C z λ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) ∣ Ψ t , s j ∣ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) ∣ Ψ t , s j ∣ } . (7) r^j_{t,s}(w) = \min\left\{D,\ C_\psi D\sqrt{\iota_T}\left\|\psi_{t,j}(w)\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}} + C_z\sqrt{\lambda}\left\|\psi_{t,j}(w)\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}} + C_l(h^\beta+\eta^2)\sqrt{\left|\Psi^j_{t,s}\right|}\left\|\psi_{t,j}(w)\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}} + \frac{C_l(h^\beta+\eta^2)}{\phantom{\sqrt{\left|\Psi^j_{t,s}\right|}}}\right\}. \tag*{(7)} r t , s j ( w ) = min ⎩ ⎨ ⎧ D , C ψ D ι T ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C z λ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) Ψ t , s j ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + Ψ t , s j C l ( h β + η 2 ) ⎭ ⎬ ⎫ . (7)
with
ι T : = 2 log ( 4 S N δ ) + [ 1 + ϖ ( β ) ( d + 1 ) ] log ( 1 + T λ ) . (8) \iota_T := 2\log\left(\frac{4SN}{\delta}\right) + \left[1+\varpi(\beta)(d+1)\right]\log\left(1+\frac{T}{\lambda}\right). \tag*{(8)} ι T := 2 log ( δ 4 S N ) + [ 1 + ϖ ( β ) ( d + 1 ) ] log ( 1 + λ T ) . (8)
If ∣ Ψ t , s j ∣ = 0 \left|\Psi^j_{t,s}\right|=0 Ψ t , s j = 0 , we set r t , s j ( w ) = D r^j_{t,s}(w)=D r t , s j ( w ) = D .
The four summands in (7) have distinct roles: self-normalized concentration controls the centered demand noise; the ridge penalty contributes C z λ C_z\sqrt{\lambda} C z λ ; the accumulated historical remainders contribute the term proportional to ∣ Ψ t , s j ∣ \sqrt{\left|\Psi^j_{t,s}\right|} Ψ t , s j ; and the final term controls the current action’s approximation error. Projection onto [ 0 , D ] [0,D] [ 0 , D ] cannot increase error relative to a target in that interval. Proposition 4.2 formalizes this decomposition and proves that r t , s j ( w ) r^j_{t,s}(w) r t , s j ( w ) is a simultaneous confidence radius.
Proposition 4.2 (Confidence bound) *Suppose Assumptions 3.1, 3.2, and 3.4 hold. Let λ > 0 \lambda>0 λ > 0 , δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) , and 0 < η ≤ h 0<\eta\le h 0 < η ≤ h . Let C z , C l , C ψ > 0 C_z,C_l,C_\psi>0 C z , C l , C ψ > 0 be the finite constants specified in (25), (27), and (35). Then, with probability at least 1 − δ 1-\delta 1 − δ , simultaneously for every LDP round t t t , every s ∈ [ s t ] s \in[s_t] s ∈ [ s t ] , and every ( j , w ) ∈ A t , s (j,w) \in\mathcal{A}_{t,s} ( j , w ) ∈ A t , s ,
∣ g ^ t , s ( w ) − g ( p ^ t ( w ) − x t ⊤ θ ⋆ ) ∣ ≤ r t , s j ( w ) . (9) \left|\widehat{g}_{t,s}(w)-g\left(\widehat{p}_t(w)-x_t^{\top}\theta^\star\right)\right| \le r^j_{t,s}(w). \tag*{(9)} g t , s ( w ) − g ( p t ( w ) − x t ⊤ θ ⋆ ) ≤ r t , s j ( w ) . (9)
Algorithm 2 Layered Decision Making with Pilot-Residual Actions
Require: Current round t t t , context x t x_t x t , certified pilot θ ^ t \widehat{\theta}_t θ t , permanent datasets { Ψ t , s j } s ∈ [ S ] , j ∈ [ N ] \{\Psi^j_{t,s}\}_{s\in[S],j\in[N]} { Ψ t , s j } s ∈ [ S ] , j ∈ [ N ] , bin partition { I j } j ∈ [ N ] \{I_j\}_{j\in[N]} { I j } j ∈ [ N ] , and constants C ψ , C z , C l C_\psi,C_z,C_l C ψ , C z , C l
Discretize the residual-action space by W : = { − B + k T − 1 / 2 : k = 0 , 1 , … , ⌊ 2 B T ⌋ } ∪ { 0 , B } \mathcal{W}:=\{-B+kT^{-1/2}:k=0,1,\ldots,\lfloor2B\sqrt{T}\rfloor\}\cup\{0,B\} W := { − B + k T − 1/2 : k = 0 , 1 , … , ⌊ 2 B T ⌋} ∪ { 0 , B } .
Set u ^ t ← Π [ 0 , B ] ( x t ⊤ θ ^ t ) \widehat{u}_t\leftarrow\Pi_{[0,B]}(x_t^{\top}\widehat{\theta}_t) u t ← Π [ 0 , B ] ( x t ⊤ θ t ) , p ^ t ( w ) ← u ^ t + w \widehat{p}_t(w)\leftarrow\widehat{u}_t+w p t ( w ) ← u t + w .
for each j ∈ [ N ] j\in[N] j ∈ [ N ] do
Set W t , j ← { w ∈ W ∩ I j : 0 ≤ p ^ t ( w ) ≤ B } \mathcal{W}_{t,j}\leftarrow\{w\in\mathcal{W}\cap I_j:0\le\widehat{p}_t(w)\le B\} W t , j ← { w ∈ W ∩ I j : 0 ≤ p t ( w ) ≤ B } .
Set s ← 1 s\leftarrow1 s ← 1 , A t , 1 ← { ( j , w ) : j ∈ [ N ] , w ∈ W t , j } \mathcal{A}_{t,1}\leftarrow\{(j,w):j\in[N],\ w\in\mathcal{W}_{t,j}\} A t , 1 ← {( j , w ) : j ∈ [ N ] , w ∈ W t , j } .
repeat
Set J t , s ← { j ∈ [ N ] : ( j , w ) ∈ A t , s for some w } \mathcal{J}_{t,s}\leftarrow\{j\in[N]:(j,w)\in\mathcal{A}_{t,s}\text{ for some }w\} J t , s ← { j ∈ [ N ] : ( j , w ) ∈ A t , s for some w } .
for each j ∈ J t , s j\in\mathcal{J}_{t,s} j ∈ J t , s do
Compute z ^ t , s j \widehat{z}^j_{t,s} z t , s j by (5)
for each ( j , w ) ∈ A t , s (j,w)\in\mathcal{A}_{t,s} ( j , w ) ∈ A t , s do
Compute g ^ t , s ( w ) ← Π [ 0 , D ] ( ψ t , j ( w ) ⊤ z ^ t , s j ) \widehat{g}_{t,s}(w)\leftarrow\Pi_{[0,D]}(\psi_{t,j}(w)^{\top}\widehat{z}^j_{t,s}) g t , s ( w ) ← Π [ 0 , D ] ( ψ t , j ( w ) ⊤ z t , s j ) .
Compute r t , s j ( w ) r^j_{t,s}(w) r t , s j ( w ) by (7)
Set U t , s j ( w ) ← p ^ t ( w ) min { D , g ^ t , s ( w ) + r t , s j ( w ) } U^j_{t,s}(w)\leftarrow\widehat{p}_t(w)\min\{D,\widehat{g}_{t,s}(w)+r^j_{t,s}(w)\} U t , s j ( w ) ← p t ( w ) min { D , g t , s ( w ) + r t , s j ( w )} , wid t , s j ( w ) ← p ^ t ( w ) r t , s j ( w ) \operatorname{wid}^j_{t,s}(w)\leftarrow\widehat{p}_t(w)r^j_{t,s}(w) wid t , s j ( w ) ← p t ( w ) r t , s j ( w ) .
if s = S s=S s = S then
Select ( j t , w t ) ∈ arg max ( j , w ) ∈ A t , s U t , s j ( w ) (j_t,w_t)\in\arg\max_{(j,w)\in\mathcal{A}_{t,s}}U^j_{t,s}(w) ( j t , w t ) ∈ arg max ( j , w ) ∈ A t , s U t , s j ( w ) , s t ← s s_t\leftarrow s s t ← s .
else if max ( j , w ) ∈ A t , s wid t , s j ( w ) ≤ B D 2 − s \max_{(j,w)\in\mathcal{A}_{t,s}}\operatorname{wid}^j_{t,s}(w)\le BD2^{-s} max ( j , w ) ∈ A t , s wid t , s j ( w ) ≤ B D 2 − s then
Set A t , s + 1 ← { ( j , w ) ∈ A t , s : U t , s j ( w ) ≥ max ( j ′ , w ′ ) ∈ A t , s U t , s j ′ ( w ′ ) − B D 2 1 − s } \mathcal{A}_{t,s+1}\leftarrow\{(j,w)\in\mathcal{A}_{t,s}:U^j_{t,s}(w)\ge\max_{(j',w')\in\mathcal{A}_{t,s}}U^{j'}_{t,s}(w')-BD2^{1-s}\} A t , s + 1 ← {( j , w ) ∈ A t , s : U t , s j ( w ) ≥ max ( j ′ , w ′ ) ∈ A t , s U t , s j ′ ( w ′ ) − B D 2 1 − s } .
Set s ← s + 1 s\leftarrow s+1 s ← s + 1 .
else
Set A t , s u ← { ( j , w ) ∈ A t , s : wid t , s j ( w ) > B D 2 − s } \mathcal{A}^u_{t,s}\leftarrow\{(j,w)\in\mathcal{A}_{t,s}:\operatorname{wid}^j_{t,s}(w)>BD2^{-s}\} A t , s u ← {( j , w ) ∈ A t , s : wid t , s j ( w ) > B D 2 − s } .
Select ( j t , w t ) ∈ arg max ( j , w ) ∈ A t , s u U t , s j ( w ) (j_t,w_t)\in\arg\max_{(j,w)\in\mathcal{A}^u_{t,s}}U^j_{t,s}(w) ( j t , w t ) ∈ arg max ( j , w ) ∈ A t , s u U t , s j ( w ) , s t ← s s_t\leftarrow s s t ← s .
until ( j t , w t ) (j_t,w_t) ( j t , w t ) has been selected
Set p t ← p ^ t ( w t ) p_t\leftarrow\widehat{p}_t(w_t) p t ← p t ( w t ) .
return ( s t , j t , w t , p t ) (s_t,j_t,w_t,p_t) ( s t , j t , w t , p t ) .
Layered exploration and elimination. The layered decision rule proceeds by refining the revenue resolution level by level. At layer s s s , the target resolution is B D 2 − s BD2^{-s} B D 2 − s . For every action ( j , w ) (j,w) ( j , w ) surviving in the current active set A t , s \mathcal{A}_{t,s} A t , s , the algorithm maintains an optimistic revenue U t , s j ( w ) U^j_{t,s}(w) U t , s j ( w ) and a width wid t , s j ( w ) \operatorname{wid}^j_{t,s}(w) wid t , s j ( w ) that measures the remaining uncertainty in its estimated revenue. The policy then decides whether to sample at this layer or to advance.
If there exists an action in A t , s \mathcal{A}_{t,s} A t , s with width strictly greater than B D 2 − s BD2^{-s} B D 2 − s , the policy selects one such under-explored action and assigns the current observation permanently to layer s s s and the corresponding bin j j j . This sampling step directly reduces the uncertainty of that action. Otherwise, if every surviving action satisfies w t , s i j ( w ) ≤ B D 2 − s w_{t,s}^{ij}(w) \le BD 2^{-s} w t , s ij ( w ) ≤ B D 2 − s , then the layer is considered sufficiently resolved. Our policy eliminates all actions whose optimistic revenue falls more than B D 2 1 − s BD 2^{1-s} B D 2 1 − s below the largest optimistic revenue among the survivors, and passes the remaining actions to the next layer s + 1 s+1 s + 1 . The safety of this elimination follows from the confidence bounds. Crucially, this argument does not require strong unimodality, local curvature, or uniqueness of the revenue maximizer, because it compares all surviving actions globally.
If the policy proceeds to layer s + 1 s+1 s + 1 , it repeats the same check with the finer resolution B D 2 − ( s + 1 ) BD 2^{-(s+1)} B D 2 − ( s + 1 ) . Once all layers up to S S S have been passed, the final action is chosen as the one with the largest UCB among the remaining active set.
Algorithm 2 implements this layered procedure. It starts with the full feasible residual grid and sequentially examines the layers. At each layer, it first computes the required estimates and widths; if an under-explored action is found, the round stops and the observation is assigned to that layer. Otherwise, it performs elimination and continues without consuming the current observation. The process terminates when either an action is selected for sampling or the final layer is reached, in which case the action with the highest UCB is chosen.
Regret Analysis
Upper Bound
We now analyze the regret of the proposed policy. The proof follows the two operating modes of Algorithm 1. On LDP rounds, the confidence bounds translate the stopping layer into a one-period revenue guarantee. We then control how often the policy can stop at each layer by combining the layer-specific uncertainty threshold with an elliptical-potential bound for the permanently labeled observations. This yields a bound on the regret accumulated over all LDP rounds. Separately, the uncertainty gate limits the number of pilot-exploration rounds through the growth of the pilot design matrix. Combining these two components gives the finite-sample regret bound, after which we establish a matching lower bound in its dependence on the horizon.
We begin with the one-period analysis of an LDP round. The confidence event guarantees that the best action in the current discretized candidate set survives every completed layer. Consequently, the stopping layer controls the revenue loss relative to the best discretized action, while the mesh of the residual grid controls the additional loss relative to the continuous oracle price.
Lemma 5.1 Fix an LDP round t t t , and suppose that the confidence bounds (9) hold at every layer visited in period t t t . For each visited layer s s s , define V t , s : = max ( j , w ) ∈ A t , s Rev ( x t , p ^ t ( w ) ) V_{t,s} := \max_{(j,w)\in\mathcal{A}_{t,s}} \operatorname{Rev}(x_t,\hat{p}_t(w)) V t , s := max ( j , w ) ∈ A t , s Rev ( x t , p ^ t ( w )) . Then:
If the algorithm passes the precision check at a layer s < S s<S s < S , then V t , s + 1 = V t , s V_{t,s+1}=V_{t,s} V t , s + 1 = V t , s .
If s t ≥ 2 s_t \ge2 s t ≥ 2 , then V t , 1 − Rev ( x t , p t ) ≤ 8 B D 2 − s t V_{t,1}-\operatorname{Rev}(x_t,p_t)\le8BD 2^{-s_t} V t , 1 − Rev ( x t , p t ) ≤ 8 B D 2 − s t .
Moreover, there exists a finite constant L R e v > 0 L_{\mathrm{Rev}}>0 L Rev > 0 , such that, for every s t ∈ [ S ] s_t\in[S] s t ∈ [ S ] ,
Rev ( x t , p t ∗ ) − Rev ( x t , p t ) ≤ L R e v T + 8 B D 2 − s t . (10) \operatorname{Rev}(x_t,p_t^*)-\operatorname{Rev}(x_t,p_t)\le\frac{L_{\mathrm{Rev}}}{\sqrt{T}}+8BD 2^{-s_t}. \tag*{(10)} Rev ( x t , p t ∗ ) − Rev ( x t , p t ) ≤ T L Rev + 8 B D 2 − s t . (10)
Lemma 5.1 reduces the regret over LDP rounds to the weighted layer occupancy ∑ s = 1 S 2 − s ∣ Ψ T + 1 , s ∣ \sum_{s=1}^{S}2^{-s}|\Psi_{T+1,s}| ∑ s = 1 S 2 − s ∣ Ψ T + 1 , s ∣ . Here Ψ T + 1 , s : = ⋃ j = 1 J Ψ T + 1 , s j \Psi_{T+1,s}:=\bigcup_{j=1}^{J}\Psi_{T+1,s}^{j} Ψ T + 1 , s := ⋃ j = 1 J Ψ T + 1 , s j denotes the set of all LDP rounds assigned to layer s s s . It therefore remains to control the number of rounds terminating at each layer. The required auxiliary results are established in the appendix. Lemma A.2 bounds the cumulative revenue uncertainty within a layer by applying a sequential elliptical-potential argument within each permanently labeled residual bin and then aggregating across bins; the resulting bound separates statistical uncertainty from the approximation error induced by the local-polynomial remainder and the residual second-order pilot-index error. Lemma A.3 then exploits the fact that every round terminating at a nonterminal layer s < S s < S s < S has revenue width exceeding B D 2 − s BD2^{-s} B D 2 − s to convert the cumulative-width bound into an occupancy bound whenever the layer resolution dominates the approximation floor O ( h β + η 2 ) \mathcal{O}(h^\beta+ \eta^2) O ( h β + η 2 ) . These results provide the ingredients needed to aggregate the layer-dependent losses.
These ingredients lead to a natural three-regime decomposition of the LDP regret. For nonterminal layers whose resolution remains above the approximation floor, the occupancy bounds apply, and the resulting contribution is governed by statistical uncertainty. Once the resolution falls to the approximation scale, a separate occupancy bound is no longer needed, because the per-round loss at these finer layers can be charged directly to the approximation error. Finally, each round reaching the terminal layer contributes at most its prescribed resolution 2 − S 2^{-S} 2 − S . Summing the contributions from these three regimes yields the total regret incurred over the LDP rounds.
Proposition 5.2 (Regret of the discretized LDP step) On the uniform confidence event in Proposition 4.2, we have
∑ t ∈ [ T ] ∖ T e x p [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ 64 C ω B ( D T + λ ) N T ι T \sum_{t\in[T]\setminus\mathcal{T}^{\mathrm{exp}}} \left[\operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)\right]
\le64C_\omega B\left(D\sqrt{T}+\sqrt{\lambda}\right)\sqrt{NT\iota_T} t ∈ [ T ] ∖ T exp ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ 64 C ω B ( D T + λ ) N T ι T
+ 16 C ω B T ( h β + η 2 ) ι T + ( 8 B D + L R e v ) T . (11) {}+16C_\omega BT(h^\beta+\eta^2)\sqrt{\iota_T}+(8BD+L_{\mathrm{Rev}})\sqrt{T}. \tag*{(11)} + 16 C ω B T ( h β + η 2 ) ι T + ( 8 B D + L Rev ) T . (11)
Suppose that λ > 0 \lambda>0 λ > 0 is fixed independently of T T T , η 2 ≤ c η h β \eta^2\le c_\eta h^\beta η 2 ≤ c η h β for some fixed constant c η > 0 c_\eta>0 c η > 0 in addition to η ≤ h \eta\le h η ≤ h , N = ⌊ T 1 2 β + 1 ⌋ N=\left\lfloor T^{\frac{1}{2\beta+1}}\right\rfloor N = ⌊ T 2 β + 1 1 ⌋ and h = 2 B / N h=2B/N h = 2 B / N , then there exists a finite constant C L D P > 0 C_{\mathrm{LDP}}>0 C LDP > 0 such that
∑ t ∈ [ T ] ∖ T e x p [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ C L D P ι T T β + 1 2 β + 1 . (12) \sum_{t\in[T]\setminus\mathcal{T}^{\mathrm{exp}}} \left[\operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)\right]
\le C_{\mathrm{LDP}}\iota_T T^{\frac{\beta+1}{2\beta+1}}. \tag*{(12)} t ∈ [ T ] ∖ T exp ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ C LDP ι T T 2 β + 1 β + 1 . (12)
It remains to control the pilot-exploration rounds. Such a round is triggered only when the leverage score of the current covariate exceeds the target index accuracy. Each trigger produces a multiplicative increase in the determinant of the pilot design matrix. Since the determinant is also bounded above by the trace of that matrix, only a limited number of pilot-exploration rounds can occur. This argument is pathwise and does not consume any additional failure probability.
Lemma 5.3 (Pilot-exploration count and regret) Suppose that Algorithm 1 is implemented with η > 0 \eta>0 η > 0 . Then the number of pilot-exploration rounds satisfies, pathwise,
∣ T e x p ∣ ≤ min { T , d log ( 1 + T C x 2 / d ) log ( 1 + η 2 / γ T 2 ) } ≤ min { T , d ( 1 + γ T 2 η 2 ) log ( 1 + T C x 2 d ) } . (13) |\mathcal{T}^{\mathrm{exp}}|
\le\min\left\{T,\frac{d\log(1+TC_x^2/d)}{\log(1+\eta^2/\gamma_T^2)}\right\}
\le\min\left\{T,d\left(1+\frac{\gamma_T^2}{\eta^2}\right)\log\left(1+\frac{TC_x^2}{d}\right)\right\}. \tag*{(13)} ∣ T exp ∣ ≤ min { T , log ( 1 + η 2 / γ T 2 ) d log ( 1 + T C x 2 / d ) } ≤ min { T , d ( 1 + η 2 γ T 2 ) log ( 1 + d T C x 2 ) } . (13)
Consequently, the regret incurred on pilot-exploration rounds satisfies
∑ t ∈ T e x p [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ B D min { T , d ( 1 + γ T 2 η 2 ) log ( 1 + T C x 2 d ) } . (14) \sum_{t\in\mathcal{T}^{\mathrm{exp}}}\left[\operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)\right]
\le BD\min\left\{T,d\left(1+\frac{\gamma_T^2}{\eta^2}\right)\log\left(1+\frac{TC_x^2}{d}\right)\right\}. \tag*{(14)} t ∈ T exp ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ B D min { T , d ( 1 + η 2 γ T 2 ) log ( 1 + d T C x 2 ) } . (14)
The regret decomposition is now complete. The pilot-exploration bound in Lemma 5.3 holds pathwise, whereas the LDP bound in Proposition 5.2 holds on the uniform confidence event. Hence, their sum holds on the same event, without an additional union bound. The following theorem first states the resulting finite-sample guarantee and then specializes it to compatible choices of the residual-bin width and pilot accuracy.
Theorem 5.4 (Regret upper bound) Suppose Assumptions 3.1, 3.2, and 3.4 hold. Let 0 < η ≤ h 0 < \eta\le h 0 < η ≤ h , and γ T \gamma_T γ T , ι T \iota_T ι T be defined as in Algorithm 1 and (8), respectively. Then, with probability at least 1 − δ 1-\delta 1 − δ ,
Reg ( T ) ≤ B D min { T , d log ( 1 + T C χ 2 / d ) log ( 1 + η 2 / γ T 2 ) } + 64 C ω B ( D ι T + λ ) N T ι T + 16 C ω B T ( h β + η 2 ) ι T + ( 8 B D + L R e v ) T . (15) \operatorname{Reg}(T) \le BD \min\left\{T,\frac{d\log\left(1+TC_{\chi}^{2}/d\right)}{\log\left(1+\eta^{2}/\gamma_T^{2}\right)}\right\}+64C_{\omega}B\left(D\sqrt{\iota_T}+\sqrt{\lambda}\right)\sqrt{N_T\iota_T}
+16C_{\omega}BT\left(h^{\beta}+\eta^{2}\right)\sqrt{\iota_T}+\left(8BD+L_{\mathrm{Rev}}\right)\sqrt{T}. \tag*{(15)} Reg ( T ) ≤ B D min { T , log ( 1 + η 2 / γ T 2 ) d log ( 1 + T C χ 2 / d ) } + 64 C ω B ( D ι T + λ ) N T ι T + 16 C ω B T ( h β + η 2 ) ι T + ( 8 B D + L Rev ) T . (15)
where C ω C_{\omega} C ω and L R e v L_{\mathrm{Rev}} L Rev are the constants in Lemma A.2 and Lemma 5.1.
Moreover, let N = ⌈ T 1 2 β + 1 ⌉ N=\left\lceil T^{\frac{1}{2\beta+1}}\right\rceil N = ⌈ T 2 β + 1 1 ⌉ and η 2 = min { h 2 , h β } \eta^{2}=\min\{h^{2},h^{\beta}\} η 2 = min { h 2 , h β } . Then with probability at least 1 − δ 1-\delta 1 − δ , there exists a finite constant C > 0 C>0 C > 0 such that
Reg ( T ) ≤ C [ d ( 1 + γ T 2 ) log ( 1 + T C χ 2 d ) + ( 1 + λ ) ι T ] T β + 1 2 β + 1 . (16) \operatorname{Reg}(T) \le C\left[d\left(1+\gamma_T^{2}\right)\log\left(1+\frac{TC_{\chi}^{2}}{d}\right)+\left(1+\sqrt{\lambda}\right)\iota_T\right]T^{\frac{\beta+1}{2\beta+1}}. \tag*{(16)} Reg ( T ) ≤ C [ d ( 1 + γ T 2 ) log ( 1 + d T C χ 2 ) + ( 1 + λ ) ι T ] T 2 β + 1 β + 1 . (16)
The theorem separates the main sources of regret. Pilot exploration pays for certifying the valuation index along the covariate directions encountered by the policy; its contribution is controlled by the uncertainty-triggered determinant argument. On LDP rounds, N T \sqrt{N_T} N T is the statistical cost of learning across the N N N residual bins, whereas T h β Th^{\beta} T h β is the local-polynomial approximation cost. The pilot-corrected lifting leaves only the second-order contribution T η 2 T\eta^{2} T η 2 , so choosing η 2 ≤ h β \eta^{2}\le h^{\beta} η 2 ≤ h β prevents pilot-index error from worsening the nonparametric rate. The residual-grid discretization and terminal layer contribute only order T \sqrt{T} T . Choosing N ≍ T 1 2 β + 1 N\asymp T^{\frac{1}{2\beta+1}} N ≍ T 2 β + 1 1 , h ≍ T − 1 2 β + 1 h\asymp T^{-\frac{1}{2\beta+1}} h ≍ T − 2 β + 1 1 and η 2 ≤ h β \eta^{2}\le h^{\beta} η 2 ≤ h β balances the statistical and approximation terms. For fixed d d d and fixed problem primitives, this yields the regret on the order of O ~ ( T β + 1 2 β + 1 ) \widetilde{O}\left(T^{\frac{\beta+1}{2\beta+1}}\right) O ( T 2 β + 1 β + 1 ) .
Remark 5.5 (Adaptation to an unknown time horizon) The knowledge of T T T can be removed using the standard doubling trick, as commonly adopted in the dynamic-pricing literature [20 ] , [23 ] , [13 ] , [26 ] , [16 ] . Specifically, the policy is restarted over epochs of geometrically increasing lengths, with all horizon-dependent parameters, including N N N , calibrated to the current epoch length.
Lower Bound
We next establish a minimax lower bound that matches the horizon dependence of the regret upper bound in Theorem 5.4, up to logarithmic factors. The construction uses a constant-covariate, binary-demand subclass. Hence, the lower bound isolates the difficulty of learning an unknown smooth demand function and does not rely on estimating the contextual parameter.
A hard binary-demand subclass. Consider the admissible demand subclass
y t = D 1 { v t ≥ p t } , q ( z ) = D 1 { 0 ≤ z ≤ B } . (17) y_t = D\mathbf{1}\{v_t \ge p_t\}, \qquad q(z)=D\mathbf{1}\{0\le z\le B\}. \tag*{(17)} y t = D 1 { v t ≥ p t } , q ( z ) = D 1 { 0 ≤ z ≤ B } . (17)
For every u ∈ I g u\in\mathcal{I}_g u ∈ I g , this subclass gives g ( u ) = E [ q ( ϵ t − u ) ] = D P ( ϵ t ≥ u ) g(u)=\mathbb{E}[q(\epsilon_t-u)]=D\mathbb{P}(\epsilon_t\ge u) g ( u ) = E [ q ( ϵ t − u )] = D P ( ϵ t ≥ u ) . Indeed, if u ∈ I g u\in\mathcal{I}_g u ∈ I g and ϵ t ≥ u \epsilon_t\ge u ϵ t ≥ u , then 0 ≤ ϵ t − u ≤ B ϵ − ( − B + B ϵ ) = B 0\le\epsilon_t-u\le B_{\epsilon}-(-B+B_{\epsilon})=B 0 ≤ ϵ t − u ≤ B ϵ − ( − B + B ϵ ) = B . Thus, on the effective residual domain I g \mathcal{I}_g I g , the normalized link g / D g/D g / D is the survival function of the valuation shock.
Fix an admissible θ ∘ ∈ Θ \theta^{\circ} \in\Theta θ ∘ ∈ Θ satisfying Assumption 3.1 over X \mathcal{X} X , and fix any x ∘ ∈ X x^{\circ} \in\mathcal{X} x ∘ ∈ X . Because deterministic covariate sequences are admissible, the minimax problem contains the subclass x t ≡ x ∘ x_t \equiv x^{\circ} x t ≡ x ∘ , θ ⋆ = θ ∘ \theta_{\star}=\theta^{\circ} θ ⋆ = θ ∘ . For every p ∈ [ 0 , B ] p \in[0,B] p ∈ [ 0 , B ] , p − ( x ∘ ) ⊤ θ ∘ ∈ [ − ( x ∘ ) ⊤ θ ∘ , B − ( x ∘ ) ⊤ θ ∘ ] ⊆ I g p-(x^{\circ})^{\top}\theta^{\circ} \in[-(x^{\circ})^{\top}\theta^{\circ}, B-(x^{\circ})^{\top}\theta^{\circ}] \subseteq\mathcal{I}_g p − ( x ∘ ) ⊤ θ ∘ ∈ [ − ( x ∘ ) ⊤ θ ∘ , B − ( x ∘ ) ⊤ θ ∘ ] ⊆ I g , and the corresponding expected revenue reduces to Rev ( p ) = p g ( p − ( x ∘ ) ⊤ θ ∘ ) \operatorname{Rev}(p)=p\,g\left(p-(x^{\circ})^{\top}\theta^{\circ}\right) Rev ( p ) = p g ( p − ( x ∘ ) ⊤ θ ∘ ) .
Theorem 5.6 (Minimax lower bound) There exist finite constants L ‾ g > 0 \overline{L}_g>0 L g > 0 , c > 0 c>0 c > 0 , and T 0 < ∞ T_0<\infty T 0 < ∞ such that, for every L g ≥ L ‾ g L_g\geq\overline{L}_g L g ≥ L g and every T ≥ T 0 T\geq T_0 T ≥ T 0 ,
inf π sup P ( L g ) E π [ Reg ( T ) ] ≥ c T β + 1 2 β + 1 . (18) \inf_{\pi}\sup_{\mathcal{P}(L_g)}\mathbb{E}^{\pi}[\operatorname{Reg}(T)]\geq cT^{\frac{\beta+1}{2\beta+1}}. \tag*{(18)} π inf P ( L g ) sup E π [ Reg ( T )] ≥ c T 2 β + 1 β + 1 . (18)
Here, the infimum is over all nonanticipating, possibly randomized, pricing policies, and the supremum is over all instances in P ( L g ) \mathcal{P}(L_g) P ( L g ) . The class P ( L g ) \mathcal{P}(L_g) P ( L g ) contains all instances satisfying Assumptions 3.1, 3.2, and 3.4 with Hölder constant at most L g L_g L g , together with all admissible covariate sequences. The expectation is taken over both the demand randomness and randomization of the pricing policy.
Corollary 5.7 (Expected-regret upper bound and minimax horizon rate) Fix L g ≥ L ‾ g L_g\geq\overline{L}_g L g ≥ L g , the dimension, and the problem primitives, and suppose the confidence constants used by the policy are chosen uniformly over P ( L g ) \mathcal{P}(L_g) P ( L g ) . For all sufficiently large T T T , there are finite constants 0 < c ‾ ≤ C ‾ < ∞ 0<\underline{c}\leq\overline{C}<\infty 0 < c ≤ C < ∞ , independent of T T T , such that
c ‾ T β + 1 2 β + 1 ≤ inf π sup P ( L g ) E π [ Reg ( T ) ] ≤ C ‾ ( log T ) 2 T β + 1 2 β + 1 . (19) \underline{c}T^{\frac{\beta+1}{2\beta+1}}\leq\inf_{\pi}\sup_{\mathcal{P}(L_g)}\mathbb{E}^{\pi}[\operatorname{Reg}(T)]\leq\overline{C}(\log T)^2T^{\frac{\beta+1}{2\beta+1}}. \tag*{(19)} c T 2 β + 1 β + 1 ≤ π inf P ( L g ) sup E π [ Reg ( T )] ≤ C ( log T ) 2 T 2 β + 1 β + 1 . (19)
In particular, the upper and lower bounds match in their horizon exponent.
Construction overview. The main difficulty is to construct a hard family that simultaneously satisfies the smoothness, monotonicity, bounded-support, and zero-mean requirements imposed on the valuation shock. We first construct a smooth baseline link whose expected revenue is constant over a nondegenerate interior price interval. We then place a smooth local perturbation in one of K T K_T K T disjoint subintervals of this flat region. Each perturbation creates a hidden revenue improvement, while a smaller adjustment away from the flat region preserves the zero-mean condition.
The following lemma provides the required baseline link and the function used for the compensating adjustment.
Lemma 5.8 (A flat zero-mean baseline) There exist constants 0 < p L < p U < B 0<p_L<p_U<B 0 < p L < p U < B , a nonnegative infinitely differentiable function χ : R → [ 0 , ∞ ) \chi:\mathbb{R}\to[0,\infty) χ : R → [ 0 , ∞ ) , and a nonincreasing function g 0 : R → [ 0 , D ] g_0:\mathbb{R}\to[0,D] g 0 : R → [ 0 , D ] satisfying the following properties:
The function g 0 g_0 g 0 is infinitely differentiable, g 0 ( u ) = D g_0(u)=D g 0 ( u ) = D for u ≤ − B ε u\leq-B\varepsilon u ≤ − B ε , and g 0 ( u ) = 0 g_0(u)=0 g 0 ( u ) = 0 for u ≥ B ε u\geq B\varepsilon u ≥ B ε .
The function g 0 g_0 g 0 belongs to H ( β ) \mathcal{H}(\beta) H ( β ) on [ − B , B ] [-B,B] [ − B , B ] , with a finite Hölder constant.
The normalized function g 0 / D g_0/D g 0 / D is the survival function of a zero-mean distribution supported on [ − B ε , B ε ] [-B\varepsilon,B\varepsilon] [ − B ε , B ε ] . In view of the monotonicity and endpoint conditions above, its zero-mean property is equivalently expressed as ∫ − B ε B ε g 0 ( u ) d u = D B ε \int_{-B\varepsilon}^{B\varepsilon}g_0(u)\,\mathrm{d}u=DB\varepsilon ∫ − B ε B ε g 0 ( u ) d u = D B ε .
The baseline revenue satisfies
p g 0 ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ D ( ( x ∘ ) ⊤ θ ∘ − B ε 2 ) , p ∈ [ 0 , B ] , (20) p\,g_0\left(p-(x^{\circ})^{\top}\theta^{\circ}\right)\leq D\left((x^{\circ})^{\top}\theta^{\circ}-\frac{B\varepsilon}{2}\right),\qquad p\in[0,B], \tag*{(20)} p g 0 ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ D ( ( x ∘ ) ⊤ θ ∘ − 2 B ε ) , p ∈ [ 0 , B ] , (20)
with equality for every p ∈ [ p L , p U ] p\in[p_L,p_U] p ∈ [ p L , p U ] .
Figure 2. Illustration of the lower-bound construction. Panel (a) constructs a smooth, nonincreasing baseline demand function g 0 g_0 g 0 satisfying the zero-mean integral condition. Panel (b) shows the induced flat revenue region and a local perturbation of width O ( K T − 1 ) \mathcal{O}(K_T^{-1}) O ( K T − 1 ) and height κ K T − β \kappa K_T^{-\beta} κ K T − β . Panel (c) adds the compensating term − a k χ -a_k\chi − a k χ , where a k = ∫ Δ k ( v ) d v = O ( K T − ( β + 1 ) ) a_k=\int\Delta_k(v)\,\mathrm{d}v=\mathcal{O}(K_T^{-(\beta+1)}) a k = ∫ Δ k ( v ) d v = O ( K T − ( β + 1 ) ) , so that the perturbation preserves the zero-mean shock condition. Panel (d) summarizes the information–regret balance: choosing K T ≍ T 1 / ( 2 β + 1 ) K_T\asymp T^{1/(2\beta+1)} K T ≍ T 1/ ( 2 β + 1 ) keeps the KL divergence bounded and yields the regret lower bound Ω ( T ( β + 1 ) / ( 2 β + 1 ) ) \Omega(T^{(\beta+1)/(2\beta+1)}) Ω ( T ( β + 1 ) / ( 2 β + 1 ) ) .
The function χ \chi χ satisfies supp ( χ ) ⊂ ( − B ε , B ε ) ∖ [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] \operatorname{supp}(\chi)\subset(-B_{\varepsilon},B_{\varepsilon})\setminus[p_L-(x^\circ)^\top\theta^\circ,p_U-(x^\circ)^\top\theta^\circ] supp ( χ ) ⊂ ( − B ε , B ε ) ∖ [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] , ∫ R χ ( u ) d u = 1 \int_{\mathbb{R}}\chi(u)\,\mathrm{d}u=1 ∫ R χ ( u ) d u = 1 . On neighborhoods of both [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] [p_L-(x^\circ)^\top\theta^\circ,p_U-(x^\circ)^\top\theta^\circ] [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] and supp ( χ ) \operatorname{supp}(\chi) supp ( χ ) , the function g 0 g_0 g 0 is bounded away from 0 and D D D , and g 0 ′ g_0' g 0 ′ is uniformly negative.
Starting from Lemma 5.8, partition the flat revenue interval into K T K_T K T disjoint price regions. For the k k k th region, add a smooth local perturbation Δ k \Delta_k Δ k and subtract the compensating term a k χ a_k\chi a k χ , where a k = ∫ R Δ k ( v ) d v a_k=\int_{\mathbb{R}}\Delta_k(v)\,\mathrm{d}v a k = ∫ R Δ k ( v ) d v . Thus, the perturbed link is of the form g 0 ( u ) + Δ k ( u ) − a k χ ( u ) g_0(u)+\Delta_k(u)-a_k\chi(u) g 0 ( u ) + Δ k ( u ) − a k χ ( u ) . Because ∫ R χ ( u ) d u = 1 \int_{\mathbb{R}}\chi(u)\,\mathrm{d}u=1 ∫ R χ ( u ) d u = 1 , ∫ R ( Δ k ( u ) − a k χ ( u ) ) d u = 0 \int_{\mathbb{R}}(\Delta_k(u)-a_k\chi(u))\,\mathrm{d}u=0 ∫ R ( Δ k ( u ) − a k χ ( u )) d u = 0 . Consequently, the perturbation does not alter the integral condition corresponding to the zero-mean valuation shock.
The local perturbation has width O ( K T − 1 ) \mathcal{O}(K_T^{-1}) O ( K T − 1 ) and raises the revenue in its associated region by order K T − β K_T^{-\beta} K T − β . Its integral therefore satisfies a k = O ( K T − ( β + 1 ) ) a_k=\mathcal{O}(K_T^{-(\beta+1)}) a k = O ( K T − ( β + 1 ) ) . Hence, the compensating adjustment is smaller than the local perturbation by a factor of order K T − 1 K_T^{-1} K T − 1 . By placing its support away from the flat region and choosing the perturbation magnitude sufficiently small, all perturbed links remain nonincreasing, take values in [ 0 , D ] [0,D] [ 0 , D ] , and belong to a common Hölder ball.
Information–regret balance. Under the baseline instance, the expected total number of visits to all K T K_T K T disjoint perturbation regions is at most T T T ; hence some region is visited at most T / K T T/K_T T / K T times in expectation. In that region, the perturbation changes the Bernoulli success probability by Θ ( K T − β ) \Theta(K_T^{-\beta}) Θ ( K T − β ) , producing a per-observation KL divergence of order K T − 2 β K_T^{-2\beta} K T − 2 β . The local KL contribution is therefore O ( T K T K T − 2 β ) = O ( T K T − ( 2 β + 1 ) ) \mathcal{O}\left(\frac{T}{K_T}K_T^{-2\beta}\right)=\mathcal{O}(TK_T^{-(2\beta+1)}) O ( K T T K T − 2 β ) = O ( T K T − ( 2 β + 1 ) ) . The compensating adjustment, of magnitude O ( K T − ( β + 1 ) ) \mathcal{O}(K_T^{-(\beta+1)}) O ( K T − ( β + 1 ) ) , contributes at most O ( T K T − 2 ( β + 1 ) ) \mathcal{O}(TK_T^{-2(\beta+1)}) O ( T K T − 2 ( β + 1 ) ) to the KL divergence, which is of lower order. Hence the total KL divergence remains bounded when K T ≍ T 1 / ( 2 β + 1 ) K_T\asymp T^{1/(2\beta+1)} K T ≍ T 1/ ( 2 β + 1 ) . A standard testing argument then shows that with probability bounded away from zero, the policy fails to identify the perturbed region for a constant fraction of the horizon. Since any price outside that region incurs a revenue loss of order K T − β K_T^{-\beta} K T − β , the expected regret is at least of order T K T − β ≍ T β + 1 2 β + 1 T K_T^{-\beta} \asymp T^{\frac{\beta+1}{2\beta+1}} T K T − β ≍ T 2 β + 1 β + 1 , proving Theorem 5.6. Therefore, for fixed problem primitives, the regret rate O ~ ( T β + 1 2 β + 1 ) \widetilde{O}\left(T^{\frac{\beta+1}{2\beta+1}}\right) O ( T 2 β + 1 β + 1 ) is minimax optimal in its dependence on the horizon.
Remark 5.9 (The effect of strong unimodality) The hard family of instances underlying Theorem 5.6 is based on a revenue function that is flat over a nondegenerate price interval. It therefore lies outside any class that satisfies the strong unimodality condition. The strong unimodality condition, however, rules out such flat alternatives and ensures that price errors translate into quadratic revenue losses. For β ≥ 2 \beta\ge2 β ≥ 2 , [14 ] obtain the matching horizon exponent 2 β − 1 4 β − 3 \frac{2\beta-1}{4\beta-3} 4 β − 3 2 β − 1 . Thus, strong unimodality fundamentally changes the learning difficulty and permits higher-order smoothness to be exploited more effectively.
Conclusion and Future Research
This paper studies contextual dynamic pricing with an unknown linear valuation component and an unknown nonparametric demand response. The model accommodates bounded discrete or continuous purchase quantities and imposes only Hölder smoothness on the induced demand function. In particular, the revenue function may be multimodal and its maximizer need not be unique. The framework therefore applies beyond the binary purchase model and avoids the strong shape restrictions commonly used to make contextual pricing analytically tractable.
To address the interaction between parametric estimation, nonparametric learning, and adaptive pricing, we develop a layered decision-partitioning framework that combines directional pilot learning, pilot-corrected local polynomial regression, and permanent layer–bin assignments. The pilot correction reduces the remaining index error to second order, whereas the permanent data partition preserves predictability and enables concentration under adaptively collected observations. Together with a global elimination procedure, these ingredients yield the regret bound Reg ( T ) = O ~ ( T β + 1 2 β + 1 ) \operatorname{Reg}(T) = \widetilde{O}\left(T^{\frac{\beta+1}{2\beta+1}}\right) Reg ( T ) = O ( T 2 β + 1 β + 1 ) , which matches our lower bound in its dependence on the horizon.
Two directions appear particularly promising for future research. First, it would be valuable to develop a policy that adapts simultaneously to the unknown Hölder radius L g L_g L g and smoothness order β \beta β , while preserving predictable data assignment and valid confidence guarantees under adaptive sampling. An accompanying question is whether such simultaneous adaptation necessarily incurs an additional statistical cost. Second, although the present framework accommodates general, possibly multimodal revenue functions, it does not exploit favorable local revenue geometry when such structure is present. Developing a shape-adaptive policy that achieves faster regret under quadratic revenue growth or strong unimodality, while retaining the minimax guarantee over the unrestricted revenue class, would yield a desirable best-of-both-worlds result.
Proofs
Proof roadmap. The appendix follows the logical dependency of the results. We first justify the induced link and the uniform-price pilot identity. We then establish pilot confidence, the pilot-corrected local approximation, and the uniform layer–bin confidence event. These ingredients feed the one-period gap and layer-counting arguments used in the contextual upper bound. The lower-bound construction is proved next.
Proofs for the Model
Proof of Theorem 3.5. Proof. Let I g = [ − B + B ϵ , B − B ϵ ] \mathcal{I}_g=[-B+B_{\epsilon},B-B_{\epsilon}] I g = [ − B + B ϵ , B − B ϵ ] . Since q : R → [ 0 , D ] q:\mathbb{R}\to[0,D] q : R → [ 0 , D ] , the function g ( u ) = E [ q ( ϵ t − u ) ] g(u)=\mathbb{E}[q(\epsilon_t-u)] g ( u ) = E [ q ( ϵ t − u )] is well-defined and satisfies 0 ≤ g ( u ) ≤ D 0\le g(u)\le D 0 ≤ g ( u ) ≤ D for all u ∈ I g u\in\mathcal{I}_g u ∈ I g . Let F ˉ ( u ) : = P ( ϵ t ≥ u ) \bar{F}(u):=\mathbb{P}(\epsilon_t\ge u) F ˉ ( u ) := P ( ϵ t ≥ u ) be the survival function of the noise. Since F ∈ H ( β , L F ; [ − B , B ] ) F\in\mathcal{H}(\beta,L_F;[-B,B]) F ∈ H ( β , L F ; [ − B , B ]) with β ≥ 1 \beta\ge1 β ≥ 1 , F F F is continuous. Hence ϵ t \epsilon_t ϵ t has no atoms.
By Assumption 3.1, the support of ϵ t \epsilon_t ϵ t is contained in [ − B ϵ , B ϵ ] [-B_{\epsilon},B_{\epsilon}] [ − B ϵ , B ϵ ] . Therefore F ˉ \bar{F} F ˉ is constant on ( − ∞ , − B ϵ ] (-\infty,-B_{\epsilon}] ( − ∞ , − B ϵ ] and on [ B ϵ , ∞ ) [B_{\epsilon},\infty) [ B ϵ , ∞ ) . Since B ϵ < B B_{\epsilon}<B B ϵ < B , the function 1 − F 1-F 1 − F is already constant on [ − B , − B ϵ ] [-B,-B_{\epsilon}] [ − B , − B ϵ ] and on [ B ϵ , B ] [B_{\epsilon},B] [ B ϵ , B ] . Thus, when viewed on the interval [ − B + B ϵ , 2 B − B ϵ ] [-B+B_{\epsilon},2B-B_{\epsilon}] [ − B + B ϵ , 2 B − B ϵ ] , the survival function F ˉ \bar{F} F ˉ belongs to H ( β , L F ; [ − B + B ϵ , 2 B − B ϵ ] ) \mathcal{H}(\beta,L_F;[-B+B_{\epsilon},2B-B_{\epsilon}]) H ( β , L F ; [ − B + B ϵ , 2 B − B ϵ ]) .
Because q q q has bounded variation on [ 0 , B ] [0,B] [ 0 , B ] , it has at most countably many discontinuities. Let q + q^{+} q + be a right-continuous representative of q q q on [ 0 , B ] [0,B] [ 0 , B ] with TV [ 0 , B ] ( q + ) ≤ V q \operatorname{TV}_{[0,B]}(q^{+})\le V_q TV [ 0 , B ] ( q + ) ≤ V q , extend it by zero outside [ 0 , B ] [0,B] [ 0 , B ] , and continue to denote the resulting function by q + q^{+} q + . Since ϵ t \epsilon_t ϵ t has no atoms, for each fixed u ∈ I g u\in\mathcal{I}_g u ∈ I g , E [ q ( ϵ t − u ) ] = E [ q + ( ϵ t − u ) ] \mathbb{E}[q(\epsilon_t-u)]=\mathbb{E}[q^{+}(\epsilon_t-u)] E [ q ( ϵ t − u )] = E [ q + ( ϵ t − u )] . Hence g g g can be represented using q + q^{+} q + . By the Lebesgue–Stieltjes representation for bounded-variation functions, there exists a finite signed measure μ q \mu_q μ q on ( 0 , B ] (0,B] ( 0 , B ] such that
q + ( w ) = q + ( 0 ) + μ q ( ( 0 , w ] ) , w ∈ [ 0 , B ] , q^{+}(w)=q^{+}(0)+\mu_q((0,w]),\qquad w\in[0,B], q + ( w ) = q + ( 0 ) + μ q (( 0 , w ]) , w ∈ [ 0 , B ] ,
and moreover,
∣ μ q ∣ ( ( 0 , B ] ) ≤ V q , ∣ q + ( 0 ) ∣ ≤ D . |\mu_q|((0,B])\le V_q,\qquad|q^{+}(0)|\le D. ∣ μ q ∣ (( 0 , B ]) ≤ V q , ∣ q + ( 0 ) ∣ ≤ D .
For u ∈ I g u\in\mathcal{I}_g u ∈ I g , Assumption 3.1 implies that ϵ t − u ≤ B \epsilon_t-u\le B ϵ t − u ≤ B almost surely whenever ϵ t − u ≥ 0 \epsilon_t-u\ge0 ϵ t − u ≥ 0 . Therefore,
q + ( ϵ t − u ) = q + ( 0 ) 1 { ϵ t ≥ u } + ∫ ( 0 , B ] 1 { ϵ t ≥ u + a } μ q ( d a ) . q^{+}(\epsilon_t-u)=q^{+}(0)\mathbf{1}\{\epsilon_t\ge u\}+\int_{(0,B]}\mathbf{1}\{\epsilon_t\ge u+a\}\mu_q(\mathrm{d}a). q + ( ϵ t − u ) = q + ( 0 ) 1 { ϵ t ≥ u } + ∫ ( 0 , B ] 1 { ϵ t ≥ u + a } μ q ( d a ) .
Taking expectations and applying Fubini’s theorem for finite signed measures gives
g ( u ) = q + ( 0 ) F ˉ ( u ) + ∫ ( 0 , B ] F ˉ ( u + a ) μ q ( d a ) , ∀ u ∈ I g . g(u)=q^{+}(0)\bar{F}(u)+\int_{(0,B]}\bar{F}(u+a)\mu_q(\mathrm{d}a),\qquad\forall u\in\mathcal{I}_g. g ( u ) = q + ( 0 ) F ˉ ( u ) + ∫ ( 0 , B ] F ˉ ( u + a ) μ q ( d a ) , ∀ u ∈ I g .
Let m : = ϖ ( β ) m:=\varpi(\beta) m := ϖ ( β ) . Since μ q \mu_q μ q is finite and F ˉ ∈ H ( β , L F ; [ − B + B ϵ , 2 B − B ϵ ] ) \bar{F}\in\mathcal{H}(\beta,L_F;[-B+B_{\epsilon},2B-B_{\epsilon}]) F ˉ ∈ H ( β , L F ; [ − B + B ϵ , 2 B − B ϵ ]) , differentiating under the finite signed measure yields
g ( m ) ( u ) = q + ( 0 ) F ˉ ( m ) ( u ) + ∫ ( 0 , B ] F ˉ ( m ) ( u + a ) μ q ( d a ) . g^{(m)}(u)=q^{+}(0)\bar{F}^{(m)}(u)+\int_{(0,B]}\bar{F}^{(m)}(u+a)\mu_q(\mathrm{d}a). g ( m ) ( u ) = q + ( 0 ) F ˉ ( m ) ( u ) + ∫ ( 0 , B ] F ˉ ( m ) ( u + a ) μ q ( d a ) .
Thus, for any u , u ′ ∈ I g u,u'\in\mathcal{I}_g u , u ′ ∈ I g ,
∣ g ( m ) ( u ) − g ( m ) ( u ′ ) ∣ ≤ ∣ q + ( 0 ) ∣ ∣ F ˉ ( m ) ( u ) − F ˉ ( m ) ( u ′ ) ∣ + ∫ ( 0 , B ] ∣ F ˉ ( m ) ( u + a ) − F ˉ ( m ) ( u ′ + a ) ∣ ∣ μ q ∣ ( d a ) ≤ ( ∣ q + ( 0 ) ∣ + ∣ μ q ∣ ( ( 0 , B ] ) ) L F ∣ u − u ′ ∣ β − m ≤ ( D + V q ) L F ∣ u − u ′ ∣ β − m . \begin{aligned}
\left|g^{(m)}(u)-g^{(m)}(u')\right|
&\le|q^{+}(0)|\left|\bar{F}^{(m)}(u)-\bar{F}^{(m)}(u')\right| \\
&\quad+\int_{(0,B]}\left|\bar{F}^{(m)}(u+a)-\bar{F}^{(m)}(u'+a)\right||\mu_q|(\mathrm{d}a)\\
&\le\left(|q^{+}(0)|+|\mu_q|((0,B])\right)L_F|u-u'|^{\beta-m}\\
&\le(D+V_q)L_F|u-u'|^{\beta-m}.
\end{aligned} g ( m ) ( u ) − g ( m ) ( u ′ ) ≤ ∣ q + ( 0 ) ∣ F ˉ ( m ) ( u ) − F ˉ ( m ) ( u ′ ) + ∫ ( 0 , B ] F ˉ ( m ) ( u + a ) − F ˉ ( m ) ( u ′ + a ) ∣ μ q ∣ ( d a ) ≤ ( ∣ q + ( 0 ) ∣ + ∣ μ q ∣ (( 0 , B ]) ) L F ∣ u − u ′ ∣ β − m ≤ ( D + V q ) L F ∣ u − u ′ ∣ β − m .
Therefore g ∈ H ( β , L g ; I g ) g \in\mathcal{H}(\beta, L_g; \mathcal{I}_g) g ∈ H ( β , L g ; I g ) with L g ≤ ( D + V q ) L F L_g \le(D + V_q)L_F L g ≤ ( D + V q ) L F . The proof is complete. □
Proofs for the Contextual Policy
Proof of Theorem 4.1. Proof. On an exploratory round, p i ∼ Unif [ 0 , B ] p_i \sim\operatorname{Unif}[0, B] p i ∼ Unif [ 0 , B ] is drawn independently of ϵ i \epsilon_i ϵ i conditional on F i − 1 ∨ σ ( x i ) \mathcal{F}_{i-1} \vee\sigma(x_i) F i − 1 ∨ σ ( x i ) . Since 1 { y i > 0 } = 1 { v i ≥ p i } \mathbb{1}\{y_i > 0\} = \mathbb{1}\{v_i \ge p_i\} 1 { y i > 0 } = 1 { v i ≥ p i } and v i ∈ [ 0 , B ] v_i \in[0, B] v i ∈ [ 0 , B ] , we have
E [ B 1 { y i > 0 } ∣ F i − 1 , x i , { i ∈ T e x p } ] = E [ v i ∣ F i − 1 , x i ] = x i ⊤ θ ⋆ . \mathbb{E}[B\mathbb{1}\{y_i > 0\} \mid\mathcal{F}_{i-1}, x_i, \{i \in\mathcal{T}^{\mathrm{exp}}\}] = \mathbb{E}[v_i \mid\mathcal{F}_{i-1}, x_i] = x_i^{\top}\theta_\star. E [ B 1 { y i > 0 } ∣ F i − 1 , x i , { i ∈ T exp }] = E [ v i ∣ F i − 1 , x i ] = x i ⊤ θ ⋆ .
Thus, B 1 { y i > 0 } B\mathbb{1}\{y_i > 0\} B 1 { y i > 0 } is an unbiased measurement of x i ⊤ θ ⋆ x_i^{\top}\theta_\star x i ⊤ θ ⋆ on pilot rounds. The observation error is bounded by B B B in absolute value, and 1 { i ∈ T e x p } x i \mathbb{1}\{i \in\mathcal{T}^{\mathrm{exp}}\}x_i 1 { i ∈ T exp } x i is determined before y i y_i y i is revealed. By the standard self-normalized concentration inequality for adaptive linear regression, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 , simultaneously for all t ≤ T t \le T t ≤ T ,
∥ ∑ i < t , i ∈ T e x p x i ( B 1 { y i > 0 } − x i ⊤ θ ⋆ ) ∥ M t − 1 ≤ B log det ( M t ) + 2 log 4 δ . \left\|\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}} x_i\left(B\mathbb{1}\{y_i>0\}-x_i^{\top}\theta_\star\right)\right\|_{M_t^{-1}}
\le B\sqrt{\log\det(M_t)+2\log\frac{4}{\delta}}. i < t , i ∈ T exp ∑ x i ( B 1 { y i > 0 } − x i ⊤ θ ⋆ ) M t − 1 ≤ B log det ( M t ) + 2 log δ 4 .
The update rules give
M t = I d + ∑ i < t , i ∈ T e x p x i x i ⊤ , b t = ∑ i < t , i ∈ T e x p B 1 { y i > 0 } x i . M_t=I_d+\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}}x_i x_i^{\top},\qquad b_t=\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}}B\mathbb{1}\{y_i>0\}x_i. M t = I d + i < t , i ∈ T exp ∑ x i x i ⊤ , b t = i < t , i ∈ T exp ∑ B 1 { y i > 0 } x i .
Since ∥ x i ∥ 2 ≤ C x \|x_i\|_2\le C_x ∥ x i ∥ 2 ≤ C x , standard elliptical-potential arguments yield
log det ( M t ) ≤ d log ( 1 + T C x 2 d ) . \log\det(M_t)\le d\log\left(1+\frac{TC_x^2}{d}\right). log det ( M t ) ≤ d log ( 1 + d T C x 2 ) .
Since θ ^ t = M t − 1 b t \widehat{\theta}_t=M_t^{-1}b_t θ t = M t − 1 b t , we have
θ ^ t − θ ⋆ = M t − 1 b t − M t − 1 M t θ ⋆ = M t − 1 ( b t − M t θ ⋆ ) . (21) \widehat{\theta}_t-\theta_\star=M_t^{-1}b_t-M_t^{-1}M_t\theta_\star=M_t^{-1}(b_t-M_t\theta_\star). \tag*{(21)} θ t − θ ⋆ = M t − 1 b t − M t − 1 M t θ ⋆ = M t − 1 ( b t − M t θ ⋆ ) . (21)
Moreover,
M t θ ⋆ = ( I d + ∑ i < t , i ∈ T e x p x i x i ⊤ ) θ ⋆ = θ ⋆ + ∑ i < t , i ∈ T e x p x i ( x i ⊤ θ ⋆ ) . (22) M_t\theta_\star=\left(I_d+\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}}x_i x_i^{\top}\right)\theta_\star=\theta_\star+\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}}x_i\left(x_i^{\top}\theta_\star\right). \tag*{(22)} M t θ ⋆ = I d + i < t , i ∈ T exp ∑ x i x i ⊤ θ ⋆ = θ ⋆ + i < t , i ∈ T exp ∑ x i ( x i ⊤ θ ⋆ ) . (22)
Substituting (22) into (21) yields
θ ^ t − θ ⋆ = M t − 1 [ ∑ i < t , i ∈ T e x p B 1 { y i > 0 } x i − θ ⋆ − ∑ i < t , i ∈ T e x p x i ( x i ⊤ θ ⋆ ) ] = M t − 1 [ ∑ i < t , i ∈ T e x p x i ( B 1 { y i > 0 } − x i ⊤ θ ⋆ ) − θ ⋆ ] . \begin{aligned}
\widehat{\theta}_t-\theta_\star=M_t^{-1}\left[\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}}B\mathbb{1}\{y_i>0\}x_i-\theta_\star-\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}}x_i\left(x_i^{\top}\theta_\star\right)\right] \\
&=M_t^{-1}\left[\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}}x_i\left(B\mathbb{1}\{y_i>0\}-x_i^{\top}\theta_\star\right)-\theta_\star\right].
\end{aligned} θ t − θ ⋆ = M t − 1 i < t , i ∈ T exp ∑ B 1 { y i > 0 } x i − θ ⋆ − i < t , i ∈ T exp ∑ x i ( x i ⊤ θ ⋆ ) = M t − 1 i < t , i ∈ T exp ∑ x i ( B 1 { y i > 0 } − x i ⊤ θ ⋆ ) − θ ⋆ .
For any x ∈ R d x\in\mathbb{R}^d x ∈ R d , the Cauchy–Schwarz inequality in the M t − 1 M_t^{-1} M t − 1 -norm gives
∣ x ⊤ ( θ ^ t − θ ⋆ ) ∣ ≤ ∥ x ∥ M t − 1 [ ∥ ∑ i < t , i ∈ T e x p x i ( B 1 { y i > 0 } − x i ⊤ θ ⋆ ) ∥ M t − 1 + ∥ θ ⋆ ∥ M t − 1 ] ≤ ∥ x ∥ M t − 1 [ B d log ( 1 + T C x 2 d ) + 2 log 4 δ + C θ ] , \begin{aligned}
\left|x^{\top}\left(\widehat{\theta}_t-\theta_\star\right)\right|
\le\|x\|_{M_t^{-1}}\left[\left\|\sum_{\substack{i<t,\,i\in\mathcal{T}^{\mathrm{exp}}}}x_i\left(B\mathbb{1}\{y_i>0\}-x_i^{\top}\theta_\star\right)\right\|_{M_t^{-1}}+\|\theta_\star\|_{M_t^{-1}}\right] \\
&\le\|x\|_{M_t^{-1}}\left[B\sqrt{d\log\left(1+\frac{TC_x^2}{d}\right)+2\log\frac{4}{\delta}}+C_\theta\right],
\end{aligned} x ⊤ ( θ t − θ ⋆ ) ≤ ∥ x ∥ M t − 1 i < t , i ∈ T exp ∑ x i ( B 1 { y i > 0 } − x i ⊤ θ ⋆ ) M t − 1 + ∥ θ ⋆ ∥ M t − 1 ≤ ∥ x ∥ M t − 1 [ B d log ( 1 + d T C x 2 ) + 2 log δ 4 + C θ ] ,
where the second inequality uses ∥ θ ⋆ ∥ M t − 1 ≤ ∥ θ ⋆ ∥ 2 ≤ C θ \lVert\theta_{\star}\rVert_{M_{t}^{-1}} \le\lVert\theta_{\star}\rVert_{2} \le C_{\theta} ∥ θ ⋆ ∥ M t − 1 ≤ ∥ θ ⋆ ∥ 2 ≤ C θ , and M t − 1 ⪯ I d M_{t}^{-1} \preceq I_{d} M t − 1 ⪯ I d . The bracketed expression is precisely γ T \gamma_{T} γ T , completing the proof. □
The proof of Proposition 4.2 uses the following martingale property.
Proposition A.1 (Martingale property) For notational convenience, set s i = j i = w i = 0 s_i=j_i=w_i=0 s i = j i = w i = 0 whenever i ∈ T e x p i \in\mathcal{T}^{\mathrm{exp}} i ∈ T exp . Use the chronological fields defined in Section 3 and write
F ‾ i : = G i , F i = F ‾ i ∨ σ ( y i ) , i ∈ [ T ] . \overline{\mathcal{F}}_{i} := \mathcal{G}_{i}, \qquad\mathcal{F}_{i} = \overline{\mathcal{F}}_{i} \vee\sigma(y_i), \qquad i \in[T]. F i := G i , F i = F i ∨ σ ( y i ) , i ∈ [ T ] .
In particular, the exploration decision, price, stopping layer, bin, and residual label are all F ‾ i \overline{\mathcal{F}}_{i} F i -measurable. For each i ∈ [ T ] i \in[T] i ∈ [ T ] , define
ε i : = y i − g ( p i − x i T θ ⋆ ) . \varepsilon_i := y_i - g\left(p_i - x_i^{\mathsf T}\theta_{\star}\right). ε i := y i − g ( p i − x i T θ ⋆ ) .
Then
E [ ε i ∣ F ‾ i ] = 0 , ∣ ε i ∣ ≤ D a.s. \mathbb{E}\left[\varepsilon_i \mid\overline{\mathcal{F}}_{i}\right] = 0, \qquad\lvert\varepsilon_i\rvert\le D \quad\text{a.s.} E [ ε i ∣ F i ] = 0 , ∣ ε i ∣ ≤ D a.s.
For fixed s ∈ [ S ] s \in[S] s ∈ [ S ] and j ∈ [ N ] j \in[N] j ∈ [ N ] , define the zero-padded feature vector
ψ ‾ i , j , s : = { ψ i , j ( w i ) , if i ∉ T e x p , s i = s , j i = j , 0 , otherwise. \overline{\psi}_{i,j,s} :=
\begin{cases}
\psi_{i,j}(w_i), & \text{if } i \notin\mathcal{T}^{\mathrm{exp}},\ s_i=s,\ j_i=j,\\
0, & \text{otherwise.}
\end{cases} ψ i , j , s := { ψ i , j ( w i ) , 0 , if i ∈ / T exp , s i = s , j i = j , otherwise.
Then ψ ‾ i , j , s \overline{\psi}_{i,j,s} ψ i , j , s is F ‾ i \overline{\mathcal{F}}_{i} F i -measurable, ∥ ψ ‾ i , j , s ∥ 2 ≤ C ψ \lVert\overline{\psi}_{i,j,s}\rVert_{2} \le C_{\psi} ∥ ψ i , j , s ∥ 2 ≤ C ψ for some C ψ > 0 C_{\psi}>0 C ψ > 0 , a.s., and E [ ψ ‾ i , j , s ε i ∣ F ‾ i ] = 0 \mathbb{E}[\overline{\psi}_{i,j,s}\varepsilon_i \mid\overline{\mathcal{F}}_{i}] = 0 E [ ψ i , j , s ε i ∣ F i ] = 0 . Consequently,
M 0 s , j : = 0 , M t s , j : = ∑ i = 1 t ψ ‾ i , j , s ε i , t ∈ [ T ] , M_{0}^{s,j} := 0, \qquad M_{t}^{s,j} := \sum_{i=1}^{t}\overline{\psi}_{i,j,s}\varepsilon_i,\qquad t \in[T], M 0 s , j := 0 , M t s , j := i = 1 ∑ t ψ i , j , s ε i , t ∈ [ T ] ,
defines a vector-valued martingale with respect to { F t } t = 0 T \{\mathcal{F}_{t}\}_{t=0}^{T} { F t } t = 0 T , whose increments satisfy
∥ M t s , j − M t − 1 s , j ∥ 2 ≤ C ψ D a.s. \left\lVert M_{t}^{s,j} - M_{t-1}^{s,j}\right\rVert_{2} \le C_{\psi}D \quad\text{a.s.} M t s , j − M t − 1 s , j 2 ≤ C ψ D a.s.
In particular, for every t ∈ [ T ] t \in[T] t ∈ [ T ] ,
M t − 1 s , j = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i . M_{t-1}^{s,j} = \sum_{i \in\Psi_{t,s}^{j}} \psi_{i,j}(w_i)\varepsilon_i. M t − 1 s , j = i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i .
Proof. By policy nonanticipativity and the sequential exogeneity of the current valuation shock, P ( ε i ≤ u ∣ F ‾ i ) = F ( u ) \mathbb{P}(\varepsilon_i \le u \mid\overline{\mathcal{F}}_{i}) = F(u) P ( ε i ≤ u ∣ F i ) = F ( u ) . Moreover, the surplus-demand model gives E [ y i ∣ F ‾ i ∨ σ ( v i ) ] = q ( v i − p i ) \mathbb{E}[y_i \mid\overline{\mathcal{F}}_{i} \vee\sigma(v_i)] = q(v_i-p_i) E [ y i ∣ F i ∨ σ ( v i )] = q ( v i − p i ) . Therefore, by the tower property,
E [ y i ∣ F ‾ i ] = E [ q ( v i − p i ) ∣ F ‾ i ] = ∫ q ( x i T θ ⋆ + ϵ − p i ) d F ( ϵ ) = g ( p i − x i T θ ⋆ ) . \mathbb{E}[y_i \mid\overline{\mathcal{F}}_{i}] = \mathbb{E}[q(v_i-p_i) \mid\overline{\mathcal{F}}_{i}] = \int q\left(x_i^{\mathsf T}\theta_{\star}+\epsilon-p_i\right)\,\mathrm{d}F(\epsilon) = g\left(p_i-x_i^{\mathsf T}\theta_{\star}\right). E [ y i ∣ F i ] = E [ q ( v i − p i ) ∣ F i ] = ∫ q ( x i T θ ⋆ + ϵ − p i ) d F ( ϵ ) = g ( p i − x i T θ ⋆ ) .
It follows that E [ ε i ∣ F ‾ i ] = 0 \mathbb{E}[\varepsilon_i \mid\overline{\mathcal{F}}_{i}] = 0 E [ ε i ∣ F i ] = 0 . Since y i ∈ [ 0 , D ] y_i \in[0,D] y i ∈ [ 0 , D ] almost surely and g ∈ [ 0 , D ] g \in[0,D] g ∈ [ 0 , D ] , we have ε i ∈ [ − g ( p i − x i T θ ⋆ ) , D − g ( p i − x i T θ ⋆ ) ] \varepsilon_i \in[-g(p_i-x_i^{\mathsf T}\theta_{\star}),D-g(p_i-x_i^{\mathsf T}\theta_{\star})] ε i ∈ [ − g ( p i − x i T θ ⋆ ) , D − g ( p i − x i T θ ⋆ )] , and hence ∣ ε i ∣ ≤ D \lvert\varepsilon_i\rvert\le D ∣ ε i ∣ ≤ D .
Now fix s ∈ [ S ] s \in[S] s ∈ [ S ] and j ∈ [ N ] j \in[N] j ∈ [ N ] . The exploration decision, permanent layer–bin labels, residual action, and posted price are all determined before y i y_i y i is observed. Hence ψ ‾ i , j , s \overline{\psi}_{i,j,s} ψ i , j , s is F ‾ i \overline{\mathcal{F}}_{i} F i -measurable. Furthermore, whenever this feature is nonzero,
w i ∈ I j , p i − w i = p ^ i ( w i ) − w i ∈ [ 0 , B ] , ∥ x i ∥ 2 ≤ C x . w_i \in I_j,\qquad p_i-w_i=\widehat{p}_i(w_i)-w_i \in[0,B],\qquad\lVert x_i\rVert_2 \le C_x. w i ∈ I j , p i − w i = p i ( w i ) − w i ∈ [ 0 , B ] , ∥ x i ∥ 2 ≤ C x .
Since the local-polynomial degree is fixed, the definition of ψ ˉ i , j \bar{\psi}_{i,j} ψ ˉ i , j implies that there exists a deterministic constant C ψ < ∞ C_{\psi}<\infty C ψ < ∞ such that ∥ ψ ˉ i , j , s ∥ 2 ≤ C ψ \lVert\bar{\psi}_{i,j,s}\rVert_{2}\le C_{\psi} ∥ ψ ˉ i , j , s ∥ 2 ≤ C ψ . Consequently,
E [ ψ ˉ i , j , s ε i ∣ F i − ] = ψ ˉ i , j , s E [ ε i ∣ F i − ] = 0 , \mathbb{E}\left[\bar{\psi}_{i,j,s}\varepsilon_i\mid\mathcal{F}_{i}^{-}\right]=\bar{\psi}_{i,j,s}\mathbb{E}\left[\varepsilon_i\mid\mathcal{F}_{i}^{-}\right]=0, E [ ψ ˉ i , j , s ε i ∣ F i − ] = ψ ˉ i , j , s E [ ε i ∣ F i − ] = 0 ,
which proves the martingale-difference property.
To make the associated martingale explicit, define, within this proof,
M 0 s , j : = 0 , M t s , j : = ∑ i = 1 t ψ ˉ i , j , s ε i , t ∈ [ T ] . M_{0}^{s,j}:=0,\qquad M_{t}^{s,j}:=\sum_{i=1}^{t}\bar{\psi}_{i,j,s}\varepsilon_i,\quad t\in[T]. M 0 s , j := 0 , M t s , j := i = 1 ∑ t ψ ˉ i , j , s ε i , t ∈ [ T ] .
The process { M t s , j } t = 0 T \{M_{t}^{s,j}\}_{t=0}^{T} { M t s , j } t = 0 T is { F t } t = 0 T \{\mathcal{F}_{t}\}_{t=0}^{T} { F t } t = 0 T -adapted and integrable. Since F t − 1 ⊆ F t − \mathcal{F}_{t-1}\subseteq\mathcal{F}_{t}^{-} F t − 1 ⊆ F t − , the tower property yields
E [ M t s , j ∣ F t − 1 ] = M t − 1 s , j + E [ E [ ψ ˉ t , j , s ε t ∣ F t − ] ∣ F t − 1 ] = M t − 1 s , j . \mathbb{E}\left[M_{t}^{s,j}\mid\mathcal{F}_{t-1}\right]=M_{t-1}^{s,j}+\mathbb{E}\left[\mathbb{E}\left[\bar{\psi}_{t,j,s}\varepsilon_t\mid\mathcal{F}_{t}^{-}\right]\mid\mathcal{F}_{t-1}\right]=M_{t-1}^{s,j}. E [ M t s , j ∣ F t − 1 ] = M t − 1 s , j + E [ E [ ψ ˉ t , j , s ε t ∣ F t − ] ∣ F t − 1 ] = M t − 1 s , j .
Thus, { M t s , j , F t } t = 0 T \{M_{t}^{s,j},\mathcal{F}_{t}\}_{t=0}^{T} { M t s , j , F t } t = 0 T is a vector-valued martingale, interpreted componentwise, with increments satisfying
∥ M t s , j − M t − 1 s , j ∥ 2 ≤ C ψ D a.s. \left\lVert M_{t}^{s,j}-M_{t-1}^{s,j}\right\rVert_{2}\le C_{\psi}D\qquad\text{a.s.} M t s , j − M t − 1 s , j 2 ≤ C ψ D a.s.
Finally, by the permanent-label definition,
Ψ t , s j = { i < t : i ∉ T e x p , s i = s , j i = j } , \Psi_{t,s}^{j}=\{i<t:i\notin\mathcal{T}^{\mathrm{exp}},s_i=s,j_i=j\}, Ψ t , s j = { i < t : i ∈ / T exp , s i = s , j i = j } ,
and therefore
M t − 1 s , j = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i . M_{t-1}^{s,j}=\sum_{i\in\Psi_{t,s}^{j}}\psi_{i,j}(w_i)\varepsilon_i. M t − 1 s , j = i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i .
This is precisely the stochastic score appearing in the layer–bin ridge estimator. □
Proof of Theorem 4.2. Proof. We proceed in five steps.
Step 1: pilot-index control. Let
E p i l : = { ∣ x ⊤ ( θ ^ t − θ ⋆ ) ∣ ≤ γ T ∥ x ∥ M t − 1 for every t ≤ T and every x ∈ R d } . \mathcal{E}_{\mathrm{pil}}:=\left\{\left|\boldsymbol{x}^{\top}(\widehat{\boldsymbol{\theta}}_{t}-\boldsymbol{\theta}_{\star})\right|\le\gamma_T\lVert\boldsymbol{x}\rVert_{M_t^{-1}}\text{ for every }t\le T\text{ and every }\boldsymbol{x}\in\mathbb{R}^{d}\right\}. E pil := { x ⊤ ( θ t − θ ⋆ ) ≤ γ T ∥ x ∥ M t − 1 for every t ≤ T and every x ∈ R d } .
Lemma 4.1 gives P ( E p i l ) ≥ 1 − δ / 2 \mathbb{P}(\mathcal{E}_{\mathrm{pil}})\ge1-\delta/2 P ( E pil ) ≥ 1 − δ /2 . If i i i is a main-policy round, then the exploration test did not trigger, and therefore γ T ∥ x i ∥ M i − 1 − 1 ≤ η \gamma_T\lVert\boldsymbol{x}_i\rVert_{M_{i-1}^{-1}}\le\eta γ T ∥ x i ∥ M i − 1 − 1 ≤ η . On E p i l \mathcal{E}_{\mathrm{pil}} E pil , we have ∣ x i ⊤ ( θ ^ i − θ ⋆ ) ∣ ≤ η \left|\boldsymbol{x}_i^{\top}(\widehat{\boldsymbol{\theta}}_i-\boldsymbol{\theta}_{\star})\right|\le\eta x i ⊤ ( θ i − θ ⋆ ) ≤ η . Because Euclidean projection onto [ 0 , B ] [0,B] [ 0 , B ] is nonexpansive and x i ⊤ θ ⋆ ∈ [ 0 , B ] \boldsymbol{x}_i^{\top}\boldsymbol{\theta}_{\star}\in[0,B] x i ⊤ θ ⋆ ∈ [ 0 , B ] ,
∣ Π [ 0 , B ] ( x i ⊤ θ ^ i ) − x i ⊤ θ ⋆ ∣ ≤ η . (23) \left|\Pi_{[0,B]}(\boldsymbol{x}_i^{\top}\widehat{\boldsymbol{\theta}}_i)-\boldsymbol{x}_i^{\top}\boldsymbol{\theta}_{\star}\right|\le\eta. \tag*{(23)} Π [ 0 , B ] ( x i ⊤ θ i ) − x i ⊤ θ ⋆ ≤ η . (23)
The same inequality holds for the current LDP round t t t .
Step 2: local Taylor expansion and joint linearization. Since 0 ≤ g ≤ D 0\le g\le D 0 ≤ g ≤ D on I g \mathcal{I}_g I g , g ∈ H ( β , L g ; I g ) g\in\mathcal{H}(\beta,L_g;\mathcal{I}_g) g ∈ H ( β , L g ; I g ) , and I g \mathcal{I}_g I g is a fixed nondegenerate compact interval, a standard one-dimensional interpolation inequality implies
max 0 ≤ k ≤ ϖ ( β ) sup u ∈ I g ∣ g ( k ) ( u ) ∣ ≤ C g , \max_{0\le k\le\varpi(\beta)}\sup_{u\in\mathcal{I}_g}\left|g^{(k)}(u)\right|\le C_g, 0 ≤ k ≤ ϖ ( β ) max u ∈ I g sup g ( k ) ( u ) ≤ C g ,
where C g C_g C g is the fixed class-level envelope specified after Assumption 3.4.
To make the Taylor coefficients at all bin endpoints well defined, extend g g g from I g = [ − B + B ϵ , B − B ϵ ] \mathcal{I}_g=[-B+B_{\epsilon},B-B_{\epsilon}] I g = [ − B + B ϵ , B − B ϵ ] to [ − B , B ] [-B,B] [ − B , B ] by its endpoint Taylor polynomials of order m : = ϖ ( β ) m:=\varpi(\beta) m := ϖ ( β ) , while continuing to use the same notation g g g .
The extension agrees with g g g and its derivatives up to order m m m at both endpoints of I g \mathcal{I}_g I g , and is therefore m m m -times differentiable on [ − B , B ] [-B,B] [ − B , B ] . Moreover, g ( m ) g^{(m)} g ( m ) is constant on each added interval and equals the corresponding endpoint value. Hence, Assumption 3.4 implies
∣ g ( m ) ( u ) − g ( m ) ( u ′ ) ∣ ≤ L g ∣ u − u ′ ∣ β − m , u , u ′ ∈ [ − B , B ] . \left|g^{(m)}(u)-g^{(m)}(u')\right|\le L_g|u-u'|^{\beta-m},\qquad u,u'\in[-B,B]. g ( m ) ( u ) − g ( m ) ( u ′ ) ≤ L g ∣ u − u ′ ∣ β − m , u , u ′ ∈ [ − B , B ] .
Indeed, when u u u and u ′ u' u ′ lie in different regions, the original Hölder bound is applied to the corresponding endpoint or endpoints, whose separation is no greater than ∣ u − u ′ ∣ |u-u'| ∣ u − u ′ ∣ . Therefore, the extension belongs to H ( β , L g ; [ − B , B ] ) \mathcal{H}(\beta,L_g;[-B,B]) H ( β , L g ; [ − B , B ]) . After increasing the fixed value of C g C_g C g , if necessary, the extension therefore satisfies
max 0 ≤ k ≤ m sup u ∈ [ − B , B ] ∣ g ( k ) ( u ) ∣ ≤ C g . (24) \max_{0\le k\le m}\sup_{u\in[-B,B]}|g^{(k)}(u)|\le C_g. \tag*{(24)} 0 ≤ k ≤ m max u ∈ [ − B , B ] sup ∣ g ( k ) ( u ) ∣ ≤ C g . (24)
This extension is used only in the analysis and leaves g g g unchanged on its original domain I g \mathcal{I}_g I g ; hence it does not alter the demand model or the algorithm.
For the proof, use the convention g ( 0 ) = g g^{(0)}=g g ( 0 ) = g and define the coefficient vector corresponding to the joint feature. Recall
z j = g ( a j − 1 ) . z_j=g(a_{j-1}). z j = g ( a j − 1 ) .
if ϖ ( β ) = 0 \varpi(\beta)=0 ϖ ( β ) = 0 , and
z j = ( g ( a j − 1 ) g ′ ( a j − 1 ) ⋮ g ( ϖ ( β ) ) ( a j − 1 ) / ϖ ( β ) ! g ′ ( a j − 1 ) θ ⋆ ⋮ [ g ( ϖ ( β ) ) ( a j − 1 ) / ϖ ( β ) ! ] θ ⋆ ) z_j=
\begin{pmatrix}
g(a_{j-1})\\
g'(a_{j-1})\\
\vdots\\
g^{(\varpi(\beta))}(a_{j-1})/\varpi(\beta)!\\
g'(a_{j-1})\theta_{\star}\\
\vdots\\
[g^{(\varpi(\beta))}(a_{j-1})/\varpi(\beta)!]\theta_{\star}
\end{pmatrix} z j = g ( a j − 1 ) g ′ ( a j − 1 ) ⋮ g ( ϖ ( β )) ( a j − 1 ) / ϖ ( β )! g ′ ( a j − 1 ) θ ⋆ ⋮ [ g ( ϖ ( β )) ( a j − 1 ) / ϖ ( β )!] θ ⋆
if ϖ ( β ) ≥ 1 \varpi(\beta)\ge1 ϖ ( β ) ≥ 1 .
Define
C z : = C g [ ∑ k = 0 m 1 ( k ! ) 2 + C θ 2 ∑ k = 1 m 1 ( k ! ) 2 ] 1 / 2 . (25) C_z:=C_g\left[\sum_{k=0}^{m}\frac{1}{(k!)^2}+C_{\theta}^{2}\sum_{k=1}^{m}\frac{1}{(k!)^2}\right]^{1/2}. \tag*{(25)} C z := C g [ k = 0 ∑ m ( k ! ) 2 1 + C θ 2 k = 1 ∑ m ( k ! ) 2 1 ] 1/2 . (25)
Equation (24) and ∥ θ ⋆ ∥ 2 ≤ C θ \|\theta_{\star}\|_2\le C_{\theta} ∥ θ ⋆ ∥ 2 ≤ C θ imply that, uniformly over j ∈ [ N ] j\in[N] j ∈ [ N ] ,
∥ z j ∥ 2 2 = ∑ k = 0 m [ g ( k ) ( a j − 1 ) k ! ] 2 + ∥ θ ⋆ ∥ 2 2 ∑ k = 1 m [ g ( k ) ( a j − 1 ) k ! ] 2 ≤ C g 2 [ ∑ k = 0 m 1 ( k ! ) 2 + C θ 2 ∑ k = 1 m 1 ( k ! ) 2 ] = C z 2 . \|z_j\|_2^2=\sum_{k=0}^{m}\left[\frac{g^{(k)}(a_{j-1})}{k!}\right]^2+\|\theta_{\star}\|_2^2\sum_{k=1}^{m}\left[\frac{g^{(k)}(a_{j-1})}{k!}\right]^2
\le C_g^2\left[\sum_{k=0}^{m}\frac{1}{(k!)^2}+C_{\theta}^{2}\sum_{k=1}^{m}\frac{1}{(k!)^2}\right]=C_z^2. ∥ z j ∥ 2 2 = k = 0 ∑ m [ k ! g ( k ) ( a j − 1 ) ] 2 + ∥ θ ⋆ ∥ 2 2 k = 1 ∑ m [ k ! g ( k ) ( a j − 1 ) ] 2 ≤ C g 2 [ k = 0 ∑ m ( k ! ) 2 1 + C θ 2 k = 1 ∑ m ( k ! ) 2 1 ] = C z 2 .
Since the right-hand side does not depend on j j j , we obtain
sup j ∈ [ N ] ∥ z j ∥ 2 ≤ C z . (26) \sup_{j\in[N]}\|z_j\|_2\le C_z. \tag*{(26)} j ∈ [ N ] sup ∥ z j ∥ 2 ≤ C z . (26)
Define
C l : = max { 2 β L g ϖ ( β ) ! , C g ∑ k = 2 ϖ ( β ) ( 2 k − k − 1 ) ( 2 B ) k − 2 k ! } . (27) C_l:=\max\left\{\frac{2^\beta L_g}{\varpi(\beta)!},\ C_g\sum_{k=2}^{\varpi(\beta)}\frac{(2^k-k-1)(2B)^{k-2}}{k!}\right\}. \tag*{(27)} C l := max ⎩ ⎨ ⎧ ϖ ( β )! 2 β L g , C g k = 2 ∑ ϖ ( β ) k ! ( 2 k − k − 1 ) ( 2 B ) k − 2 ⎭ ⎬ ⎫ . (27)
We next establish the joint linearized representation on the event E p i l \mathcal{E}_{\mathrm{pil}} E pil . Fix i ∈ Ψ t , s j i \in\Psi_{t,s}^{j} i ∈ Ψ t , s j . Since the bin label assigned to period i i i is permanent, w i ∈ I j w_i \in I_j w i ∈ I j , and hence 0 ≤ w i − a j − 1 ≤ h 0 \le w_i-a_{j-1} \le h 0 ≤ w i − a j − 1 ≤ h . Moreover, since p i = p ^ i ( w i ) = Π [ 0 , B ] ( x i T θ ^ i ) + w i p_i=\widehat{p}_i(w_i)=\Pi_{[0,B]}(x_i^{\mathsf{T}}\widehat{\theta}_i)+w_i p i = p i ( w i ) = Π [ 0 , B ] ( x i T θ i ) + w i , (23) gives that
∣ ( p i − w i ) − x i T θ ⋆ ∣ ≤ η . (28) \left|(p_i-w_i)-x_i^{\mathsf{T}}\theta_\star\right|\le\eta. \tag*{(28)} ( p i − w i ) − x i T θ ⋆ ≤ η . (28)
The true residual can therefore be written exactly as
p i − x i T θ ⋆ = a j − 1 + ( w i − a j − 1 ) + ( ( p i − w i ) − x i T θ ⋆ ) . (29) p_i-x_i^{\mathsf{T}}\theta_\star=a_{j-1}+(w_i-a_{j-1})+\left((p_i-w_i)-x_i^{\mathsf{T}}\theta_\star\right). \tag*{(29)} p i − x i T θ ⋆ = a j − 1 + ( w i − a j − 1 ) + ( ( p i − w i ) − x i T θ ⋆ ) . (29)
Moreover, since p i ∈ [ 0 , B ] p_i\in[0,B] p i ∈ [ 0 , B ] and x i T θ ⋆ ∈ [ B ϵ , B − B ϵ ] x_i^{\mathsf{T}}\theta_\star\in[B_{\epsilon},B-B_{\epsilon}] x i T θ ⋆ ∈ [ B ϵ , B − B ϵ ] ,
p i − x i T θ ⋆ ∈ [ − B + B ϵ , B − B ϵ ] = I g . p_i-x_i^{\mathsf{T}}\theta_\star\in[-B+B_{\epsilon},B-B_{\epsilon}]=\mathcal{I}_g. p i − x i T θ ⋆ ∈ [ − B + B ϵ , B − B ϵ ] = I g .
Hence g ( p i − x i T θ ⋆ ) g(p_i-x_i^{\mathsf{T}}\theta_\star) g ( p i − x i T θ ⋆ ) is evaluated on the original domain of the induced demand link; the extension in Step 2 is used only to define the Taylor coefficients at a j − 1 a_{j-1} a j − 1 . Thus, relative to the expansion point a j − 1 a_{j-1} a j − 1 , the true residual contains two deviations: the within-bin deviation w i − a j − 1 w_i-a_{j-1} w i − a j − 1 , whose absolute value is at most h h h , and the pilot-index error ( p i − w i ) − x i T θ ⋆ (p_i-w_i)-x_i^{\mathsf{T}}\theta_\star ( p i − w i ) − x i T θ ⋆ , whose absolute value is at most η \eta η .
We next identify the function value represented by the joint feature. By the definitions of ψ i , j ( w i ) \psi_{i,j}(w_i) ψ i , j ( w i ) , ϕ j \phi_j ϕ j , ϕ j ′ \phi'_j ϕ j ′ , X j \mathbf{X}_j X j , and z j z_j z j , we have
ψ i , j ( w i ) T z j = ∑ k = 0 ϖ ( β ) g ( k ) ( a j − 1 ) k ! ( w i − a j − 1 ) k + [ ( p i − w i ) − x i T θ ⋆ ] ∑ k = 1 ϖ ( β ) g ( k ) ( a j − 1 ) ( k − 1 ) ! ( w i − a j − 1 ) k − 1 . (30) \psi_{i,j}(w_i)^{\mathsf{T}}z_j
=
\sum_{k=0}^{\varpi(\beta)}
\frac{g^{(k)}(a_{j-1})}{k!}(w_i-a_{j-1})^k
+
\left[(p_i-w_i)-x_i^{\mathsf{T}}\theta_\star\right]
\sum_{k=1}^{\varpi(\beta)}
\frac{g^{(k)}(a_{j-1})}{(k-1)!}(w_i-a_{j-1})^{k-1}. \tag*{(30)} ψ i , j ( w i ) T z j = k = 0 ∑ ϖ ( β ) k ! g ( k ) ( a j − 1 ) ( w i − a j − 1 ) k + [ ( p i − w i ) − x i T θ ⋆ ] k = 1 ∑ ϖ ( β ) ( k − 1 )! g ( k ) ( a j − 1 ) ( w i − a j − 1 ) k − 1 . (30)
The first sum in (30) is the local Taylor polynomial centered at a j − 1 a_{j-1} a j − 1 and evaluated at w i w_i w i . The second sum is the derivative of this polynomial at w i w_i w i , multiplied by the pilot-index error in (28). Hence, ψ i , j ( w i ) T z j \psi_{i,j}(w_i)^{\mathsf{T}}z_j ψ i , j ( w i ) T z j represents the local Taylor polynomial together with the complete first-order correction for the pilot-index error. When ϖ ( β ) = 0 \varpi(\beta)=0 ϖ ( β ) = 0 , the derivative term is absent.
If β = 1 \beta=1 β = 1 , then ϖ ( β ) = 0 \varpi(\beta)=0 ϖ ( β ) = 0 and
ψ i , j ( w i ) T z j = g ( a j − 1 ) . \psi_{i,j}(w_i)^{\mathsf{T}}z_j=g(a_{j-1}). ψ i , j ( w i ) T z j = g ( a j − 1 ) .
Since g g g is Lipschitz continuous, (29) gives
∣ g ( p i − x i T θ ⋆ ) − ψ i , j ( w i ) T z j ∣ = ∣ g ( p i − x i T θ ⋆ ) − g ( a j − 1 ) ∣ ≤ L g ( ∣ w i − a j − 1 ∣ + ∣ ( p i − w i ) − x i T θ ⋆ ∣ ) ≤ L g ( h + η ) ≤ 2 L g h ≤ C l ( h β + η 2 ) , \begin{aligned}
\left|g(p_i-x_i^{\mathsf{T}}\theta_\star)-\psi_{i,j}(w_i)^{\mathsf{T}}z_j\right|
&=\left|g(p_i-x_i^{\mathsf{T}}\theta_\star)-g(a_{j-1})\right|\\
&\le L_g\left(|w_i-a_{j-1}|+\left|(p_i-w_i)-x_i^{\mathsf{T}}\theta_\star\right|\right)\\
&\le L_g(h+\eta)\le2L_gh\le C_l(h^\beta+\eta^2),
\end{aligned} g ( p i − x i T θ ⋆ ) − ψ i , j ( w i ) T z j = g ( p i − x i T θ ⋆ ) − g ( a j − 1 ) ≤ L g ( ∣ w i − a j − 1 ∣ + ( p i − w i ) − x i T θ ⋆ ) ≤ L g ( h + η ) ≤ 2 L g h ≤ C l ( h β + η 2 ) ,
because h β = h h^\beta=h h β = h , η ≤ h \eta\le h η ≤ h , and C l ≥ 2 L g C_l\ge2L_g C l ≥ 2 L g .
Suppose now that β > 1 \beta>1 β > 1 . Applying the Hölder Taylor expansion at a j − 1 a_{j-1} a j − 1 and using (29), we obtain
g ( p i − x i T θ ⋆ ) = ∑ k = 0 ϖ ( β ) g ( k ) ( a j − 1 ) k ! [ ( w i − a j − 1 ) + ( p i − w i ) − x i T θ ⋆ ] k + R i . (31) g(p_i-x_i^{\mathsf{T}}\theta_\star)
=
\sum_{k=0}^{\varpi(\beta)}
\frac{g^{(k)}(a_{j-1})}{k!}
\left[(w_i-a_{j-1})+(p_i-w_i)-x_i^{\mathsf{T}}\theta_\star\right]^k
+R_i. \tag*{(31)} g ( p i − x i T θ ⋆ ) = k = 0 ∑ ϖ ( β ) k ! g ( k ) ( a j − 1 ) [ ( w i − a j − 1 ) + ( p i − w i ) − x i T θ ⋆ ] k + R i . (31)
where
∣ R i ∣ ≤ L g ϖ ( β ) ! ( ∣ w i − a j − 1 ∣ + ∣ ( p i − w i ) − x i T θ ⋆ ∣ ) β ≤ L g ϖ ( β ) ! ( h + η ) β ≤ 2 β L g ϖ ( β ) ! h β ≤ C l h β . |R_i|
\le
\frac{L_g}{\varpi(\beta)!}
\left(|w_i-a_{j-1}|+\left|(p_i-w_i)-x_i^{\mathsf{T}}\theta_\star\right|\right)^\beta
\le
\frac{L_g}{\varpi(\beta)!}(h+\eta)^\beta
\le
\frac{2^\beta L_g}{\varpi(\beta)!}h^\beta
\le C_lh^\beta. ∣ R i ∣ ≤ ϖ ( β )! L g ( ∣ w i − a j − 1 ∣ + ( p i − w i ) − x i T θ ⋆ ) β ≤ ϖ ( β )! L g ( h + η ) β ≤ ϖ ( β )! 2 β L g h β ≤ C l h β .
For each k ≥ 1 k \ge1 k ≥ 1 , applying the binomial formula to the k k k th power in (31) gives
[ ( w i − a j − 1 ) + ( p i − w i ) − x i T θ ⋆ ] k = ∑ r = 0 k ( k r ) ( w i − a j − 1 ) k − r ( ( p i − w i ) − x i T θ ⋆ ) r . \left[(w_i-a_{j-1})+(p_i-w_i)-\mathbf{x}_i^{\mathsf{T}}\theta_\star\right]^k=\sum_{r=0}^{k}\binom{k}{r}(w_i-a_{j-1})^{k-r}\left((p_i-w_i)-\mathbf{x}_i^{\mathsf{T}}\theta_\star\right)^r. [ ( w i − a j − 1 ) + ( p i − w i ) − x i T θ ⋆ ] k = r = 0 ∑ k ( r k ) ( w i − a j − 1 ) k − r ( ( p i − w i ) − x i T θ ⋆ ) r .
The term with r = 0 r=0 r = 0 does not contain the pilot-index error, while the term with r = 1 r=1 r = 1 is linear in the pilot-index error. After multiplying by the Taylor coefficients and summing over k k k , these terms are exactly the two terms in (30) . Therefore, ψ i , j ( w i ) T z j \psi_{i,j}(w_i)^{\mathsf{T}}z_j ψ i , j ( w i ) T z j contains all terms in the Taylor expansion that involve at most one factor of ( p i − w i ) − x i T θ ⋆ (p_i-w_i)-\mathbf{x}_i^{\mathsf{T}}\theta_\star ( p i − w i ) − x i T θ ⋆ .
The remaining terms correspond to r ≥ 2 r \ge2 r ≥ 2 . For every 2 ≤ r ≤ k 2 \le r \le k 2 ≤ r ≤ k , using
∣ w i − a j − 1 ∣ ≤ h , ∣ ( p i − w i ) − x i T θ ⋆ ∣ ≤ η , η ≤ h , \lvert w_i-a_{j-1}\rvert\le h,\qquad\left\lvert(p_i-w_i)-\mathbf{x}_i^{\mathsf{T}}\theta_\star\right\rvert\le\eta,\qquad\eta\le h, ∣ w i − a j − 1 ∣ ≤ h , ( p i − w i ) − x i T θ ⋆ ≤ η , η ≤ h ,
we obtain
∣ w i − a j − 1 ∣ k − r ∣ ( p i − w i ) − x i T θ ⋆ ∣ r ≤ h k − r η r ≤ h k − r η 2 h r − 2 = h k − 2 η 2 ≤ ( 2 B ) k − 2 η 2 , \lvert w_i-a_{j-1}\rvert^{k-r}\left\lvert(p_i-w_i)-\mathbf{x}_i^{\mathsf{T}}\theta_\star\right\rvert^r \le h^{k-r}\eta^r \le h^{k-r}\eta^2h^{r-2}=h^{k-2}\eta^2\le(2B)^{k-2}\eta^2, ∣ w i − a j − 1 ∣ k − r ( p i − w i ) − x i T θ ⋆ r ≤ h k − r η r ≤ h k − r η 2 h r − 2 = h k − 2 η 2 ≤ ( 2 B ) k − 2 η 2 ,
where the second inequality follows because η ≤ h \eta\le h η ≤ h and r ≥ 2 r \ge2 r ≥ 2 and the last inequality follows because h ≤ 2 B h \le2B h ≤ 2 B . Moreover, (24) uniformly bounds the Taylor coefficients by C g C_g C g , and the numbers of possible values of k k k and r r r depend only on β \beta β . Hence,
∑ k = 2 ϖ ( β ) ∣ g ( k ) ( a j − 1 ) ∣ k ! ∑ r = 2 k ( k r ) ∣ w i − a j − 1 ∣ k − r ∣ ( p i − w i ) − x i T θ ⋆ ∣ r ≤ C g η 2 ∑ k = 2 ϖ ( β ) ( 2 k − k − 1 ) ( 2 B ) k − 2 k ! ≤ C l η 2 , \sum_{k=2}^{\varpi(\beta)}\frac{\lvert g^{(k)}(a_{j-1})\rvert}{k!}\sum_{r=2}^{k}\binom{k}{r}\lvert w_i-a_{j-1}\rvert^{k-r}\left\lvert(p_i-w_i)-\mathbf{x}_i^{\mathsf{T}}\theta_\star\right\rvert^r\le C_g\eta^2\sum_{k=2}^{\varpi(\beta)}\frac{(2^k-k-1)(2B)^{k-2}}{k!}\le C_l\eta^2, k = 2 ∑ ϖ ( β ) k ! ∣ g ( k ) ( a j − 1 )∣ r = 2 ∑ k ( r k ) ∣ w i − a j − 1 ∣ k − r ( p i − w i ) − x i T θ ⋆ r ≤ C g η 2 k = 2 ∑ ϖ ( β ) k ! ( 2 k − k − 1 ) ( 2 B ) k − 2 ≤ C l η 2 ,
where we used ∑ r = 2 k ( k r ) = 2 k − k − 1 \sum_{r=2}^{k}\binom{k}{r}=2^k-k-1 ∑ r = 2 k ( r k ) = 2 k − k − 1 .
Combining this bound with the Taylor remainder ∣ R i ∣ ≤ C l h β \lvert R_i\rvert\le C_lh^\beta ∣ R i ∣ ≤ C l h β , we have
g ( p i − x i T θ ⋆ ) = ψ i , j ( w i ) T z j + R i , j ( w i ) , ∣ R i , j ( w i ) ∣ ≤ C l ( h β + η 2 ) . (32) g(p_i-\mathbf{x}_i^{\mathsf{T}}\theta_\star)=\psi_{i,j}(w_i)^{\mathsf{T}}z_j+R_{i,j}(w_i),\qquad\lvert R_{i,j}(w_i)\rvert\le C_l(h^\beta+\eta^2). \tag*{(32)} g ( p i − x i T θ ⋆ ) = ψ i , j ( w i ) T z j + R i , j ( w i ) , ∣ R i , j ( w i )∣ ≤ C l ( h β + η 2 ) . (32)
When 1 < β ≤ 2 1<\beta\le2 1 < β ≤ 2 , we have ϖ ( β ) = 1 \varpi(\beta)=1 ϖ ( β ) = 1 , so the sums over k ≥ 2 k\ge2 k ≥ 2 are empty. In this case, no quadratic or higher-order term in the pilot-index error is omitted, and the additional η 2 \eta^2 η 2 term is unnecessary but remains a valid uniform upper bound.
The same argument applies to every current candidate action w ∈ W t , j w\in\mathcal{W}_{t,j} w ∈ W t , j . Indeed, p ^ t ( w ) ∈ [ 0 , B ] \widehat{p}_t(w)\in[0,B] p t ( w ) ∈ [ 0 , B ] , so Assumption 3.1 implies
p ^ t ( w ) − x t T θ ⋆ ∈ I g . \widehat{p}_t(w)-\mathbf{x}_t^{\mathsf{T}}\theta_\star\in I_g. p t ( w ) − x t T θ ⋆ ∈ I g .
Moreover, w ∈ I j w\in I_j w ∈ I j and Step 1 give
0 ≤ w − a j − 1 ≤ h , ∣ p ^ t ( w ) − w − x t T θ ⋆ ∣ ≤ η . 0\le w-a_{j-1}\le h,\qquad\left\lvert\widehat{p}_t(w)-w-\mathbf{x}_t^{\mathsf{T}}\theta_\star\right\rvert\le\eta. 0 ≤ w − a j − 1 ≤ h , p t ( w ) − w − x t T θ ⋆ ≤ η .
Therefore,
g ( p ^ t ( w ) − x t T θ ⋆ ) = ψ t , j ( w ) T z j + R t , j ( w ) , ∣ R t , j ( w ) ∣ ≤ C l ( h β + η 2 ) . (33) g\left(\widehat{p}_t(w)-\mathbf{x}_t^{\mathsf{T}}\theta_\star\right)=\psi_{t,j}(w)^{\mathsf{T}}z_j+R_{t,j}(w),\qquad\lvert R_{t,j}(w)\rvert\le C_l(h^\beta+\eta^2). \tag*{(33)} g ( p t ( w ) − x t T θ ⋆ ) = ψ t , j ( w ) T z j + R t , j ( w ) , ∣ R t , j ( w )∣ ≤ C l ( h β + η 2 ) . (33)
Step 3: ridge-error decomposition. For every i ∈ Ψ t , s j i \in\Psi^{j}_{t,s} i ∈ Ψ t , s j , Proposition A.1 and (32) give y i = ψ i , j ( w i ) ⊤ z j + R i , j ( w i ) + ε i y_i = \psi_{i,j}(w_i)^{\top}z_j + R_{i,j}(w_i) + \varepsilon_i y i = ψ i , j ( w i ) ⊤ z j + R i , j ( w i ) + ε i . Thus each observed demand consists of the linear approximation ψ i , j ( w i ) ⊤ z j \psi_{i,j}(w_i)^{\top}z_j ψ i , j ( w i ) ⊤ z j , the approximation error R i , j ( w i ) R_{i,j}(w_i) R i , j ( w i ) , and the demand noise ε i \varepsilon_i ε i .
By the definition of z ^ t , s j \widehat{z}^{j}_{t,s} z t , s j ,
( Λ t , s j + λ I ) z ^ t , s j = ∑ i ∈ Ψ t , s j y i ψ i , j ( w i ) . (\Lambda^{j}_{t,s}+\lambda I)\widehat{z}^{j}_{t,s}=\sum_{i\in\Psi^{j}_{t,s}} y_i\psi_{i,j}(w_i). ( Λ t , s j + λ I ) z t , s j = i ∈ Ψ t , s j ∑ y i ψ i , j ( w i ) .
Substituting the preceding expression for y i y_i y i yields
( Λ t , s j + λ I ) z ^ t , s j = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ( ψ i , j ( w i ) ⊤ z j + R i , j ( w i ) + ε i ) = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ψ i , j ( w i ) ⊤ z j + ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) + ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i . \begin{aligned}
(\Lambda^{j}_{t,s}+\lambda I)\widehat{z}^{j}_{t,s}=\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)\left(\psi_{i,j}(w_i)^{\top}z_j+R_{i,j}(w_i)+\varepsilon_i\right) \\
&=\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)\psi_{i,j}(w_i)^{\top}z_j+\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)+\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)\varepsilon_i.
\end{aligned} ( Λ t , s j + λ I ) z t , s j = i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ( ψ i , j ( w i ) ⊤ z j + R i , j ( w i ) + ε i ) = i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ψ i , j ( w i ) ⊤ z j + i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) + i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i .
By the definition
Λ t , s j = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ψ i , j ( w i ) ⊤ , \Lambda^{j}_{t,s}=\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)\psi_{i,j}(w_i)^{\top}, Λ t , s j = i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ψ i , j ( w i ) ⊤ ,
the first term on the right-hand side equals Λ t , s j z j \Lambda^{j}_{t,s}z_j Λ t , s j z j . Hence
( Λ t , s j + λ I ) z ^ t , s j = Λ t , s j z j + ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) + ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i . (\Lambda^{j}_{t,s}+\lambda I)\widehat{z}^{j}_{t,s}=\Lambda^{j}_{t,s}z_j+\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)+\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)\varepsilon_i. ( Λ t , s j + λ I ) z t , s j = Λ t , s j z j + i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) + i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i .
Subtracting ( Λ t , s j + λ I ) z j (\Lambda^{j}_{t,s}+\lambda I)z_j ( Λ t , s j + λ I ) z j from both sides gives
( Λ t , s j + λ I ) ( z ^ t , s j − z j ) = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i + ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) − λ z j . (\Lambda^{j}_{t,s}+\lambda I)\left(\widehat{z}^{j}_{t,s}-z_j\right)=\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)\varepsilon_i+\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)-\lambda z_j. ( Λ t , s j + λ I ) ( z t , s j − z j ) = i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i + i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) − λ z j .
The term − λ z j -\lambda z_j − λ z j appears because
Λ t , s j z j − ( Λ t , s j + λ I ) z j = − λ z j . \Lambda^{j}_{t,s}z_j-(\Lambda^{j}_{t,s}+\lambda I)z_j=-\lambda z_j. Λ t , s j z j − ( Λ t , s j + λ I ) z j = − λ z j .
Since λ > 0 \lambda>0 λ > 0 , the matrix Λ t , s j + λ I \Lambda^{j}_{t,s}+\lambda I Λ t , s j + λ I is positive definite and therefore invertible. Multiplying both sides by ( Λ t , s j + λ I ) − 1 (\Lambda^{j}_{t,s}+\lambda I)^{-1} ( Λ t , s j + λ I ) − 1 gives
z ^ t , s j − z j = ( Λ t , s j + λ I ) − 1 ( ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i + ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) − λ z j ) (34) \widehat{z}^{j}_{t,s}-z_j=(\Lambda^{j}_{t,s}+\lambda I)^{-1}\left(\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)\varepsilon_i+\sum_{i\in\Psi^{j}_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)-\lambda z_j\right) \tag*{(34)} z t , s j − z j = ( Λ t , s j + λ I ) − 1 i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i + i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) − λ z j (34)
The three terms on the right-hand side arise, respectively, from the random demand noise, the approximation error in (32), and the ridge regularization.
Step 4: noise, approximation, and regularization bounds. Fix s ∈ [ S ] s \in[S] s ∈ [ S ] and j ∈ [ N ] j \in[N] j ∈ [ N ] . Proposition A.1 shows that ψ ‾ i , j , s \overline{\psi}_{i,j,s} ψ i , j , s is determined before y i y_i y i is observed and that E [ ψ ‾ i , j , s ε i ∣ F i − ] = 0 \mathbb{E}[\overline{\psi}_{i,j,s}\varepsilon_i \mid\mathcal{F}^{-}_{i}] = 0 E [ ψ i , j , s ε i ∣ F i − ] = 0 . Moreover, conditional on F i − \mathcal{F}^{-}_{i} F i − , ε i \varepsilon_i ε i has mean zero and lies in an interval of length D D D . Hence, by the conditional Hoeffding lemma, for every γ ∈ R \gamma\in\mathbb{R} γ ∈ R ,
E [ exp ( γ ε i ) ∣ F i − ] ≤ exp ( γ 2 D 2 8 ) . \mathbb{E}[\exp(\gamma\varepsilon_i)\mid\mathcal{F}^{-}_{i}]\leq\exp\left(\frac{\gamma^2D^2}{8}\right). E [ exp ( γ ε i ) ∣ F i − ] ≤ exp ( 8 γ 2 D 2 ) .
Thus, ε i \varepsilon_i ε i is conditionally D / 2 D/2 D /2 -sub-Gaussian, and the standard time-uniform self-normalized concentration inequality applies to the features ψ ˉ i , j , s \bar{\psi}_{i,j,s} ψ ˉ i , j , s .
By the definition of ψ ˉ i , j , s \bar{\psi}_{i,j,s} ψ ˉ i , j , s , for every t ≤ T + 1 t \le T + 1 t ≤ T + 1 ,
∑ i < t ψ ˉ i , j , s ε i = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i \sum_{i<t} \bar{\psi}_{i,j,s}\varepsilon_i = \sum_{i\in\Psi^j_{t,s}} \psi_{i,j}(w_i)\varepsilon_i i < t ∑ ψ ˉ i , j , s ε i = i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i
and
∑ i < t ψ ˉ i , j , s ( ψ ˉ i , j , s ) ⊤ = ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ψ i , j ( w i ) ⊤ = Λ t , s j . \sum_{i<t} \bar{\psi}_{i,j,s}\left(\bar{\psi}_{i,j,s}\right)^\top= \sum_{i\in\Psi^j_{t,s}} \psi_{i,j}(w_i)\psi_{i,j}(w_i)^\top= \Lambda^j_{t,s}. i < t ∑ ψ ˉ i , j , s ( ψ ˉ i , j , s ) ⊤ = i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ψ i , j ( w i ) ⊤ = Λ t , s j .
Hence, for pair ( s , j ) (s,j) ( s , j ) , with probability at least 1 − δ / ( 2 S N ) 1-\delta/(2SN) 1 − δ / ( 2 S N ) , for every t ≤ T + 1 t \le T+1 t ≤ T + 1 ,
∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i ∥ ( Λ t , s j + λ I ) − 1 ≤ D log det ( Λ t , s j + λ I ) det ( λ I ) + 2 log ( 2 S N δ ) . \left\lVert\sum_{i\in\Psi^j_{t,s}} \psi_{i,j}(w_i)\varepsilon_i \right\rVert_{\left(\Lambda^j_{t,s}+\lambda I\right)^{-1}} \le D\sqrt{\log\frac{\det\left(\Lambda^j_{t,s}+\lambda I\right)}{\det(\lambda I)}+2\log\left(\frac{2SN}{\delta}\right)}. i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i ( Λ t , s j + λ I ) − 1 ≤ D log det ( λ I ) det ( Λ t , s j + λ I ) + 2 log ( δ 2 S N ) .
The matrix Λ t , s j \Lambda^j_{t,s} Λ t , s j has dimension 1 + ϖ ( β ) ( d + 1 ) 1+\varpi(\beta)(d+1) 1 + ϖ ( β ) ( d + 1 ) . Define
C ψ 2 : = 1 + ∑ k = 1 ϖ ( β ) [ ( 2 B ) k + B k ( 2 B ) k − 1 ] 2 + C x 2 ∑ k = 1 ϖ ( β ) k 2 ( 2 B ) 2 ( k − 1 ) . (35) C_\psi^2 := 1+\sum_{k=1}^{\varpi(\beta)}\left[(2B)^k+Bk(2B)^{k-1}\right]^2+C_x^2\sum_{k=1}^{\varpi(\beta)}k^2(2B)^{2(k-1)}. \tag*{(35)} C ψ 2 := 1 + k = 1 ∑ ϖ ( β ) [ ( 2 B ) k + B k ( 2 B ) k − 1 ] 2 + C x 2 k = 1 ∑ ϖ ( β ) k 2 ( 2 B ) 2 ( k − 1 ) . (35)
Since 0 ≤ w i − a j − 1 ≤ h ≤ 2 B 0 \le w_i-a_{j-1}\le h\le2B 0 ≤ w i − a j − 1 ≤ h ≤ 2 B , 0 ≤ p i − w i ≤ B 0 \le p_i-w_i\le B 0 ≤ p i − w i ≤ B , and ∥ x i ∥ 2 ≤ C x \lVert x_i\rVert_2\le C_x ∥ x i ∥ 2 ≤ C x , the definition of the joint feature gives ∥ ψ i , j ( w i ) ∥ 2 ≤ C ψ \lVert\psi_{i,j}(w_i)\rVert_2\le C_\psi ∥ ψ i , j ( w i ) ∥ 2 ≤ C ψ . The product of positive numbers is no larger than the corresponding power of their average, and hence
log det ( Λ t , s j + λ I ) det ( λ I ) ≤ [ 1 + ϖ ( β ) ( d + 1 ) ] log ( 1 + tr ( Λ t , s j ) λ [ 1 + ϖ ( β ) ( d + 1 ) ] ) ≤ C ψ 2 [ 1 + ϖ ( β ) ( d + 1 ) ] log ( 1 + T λ ) ≤ C ψ 2 ι T , \begin{aligned}
\log\frac{\det\left(\Lambda^j_{t,s}+\lambda I\right)}{\det(\lambda I)}
\le[1+\varpi(\beta)(d+1)]\log\left(1+\frac{\operatorname{tr}\left(\Lambda^j_{t,s}\right)}{\lambda[1+\varpi(\beta)(d+1)]}\right) \\
&\le C_\psi^2[1+\varpi(\beta)(d+1)]\log\left(1+\frac{T}{\lambda}\right)\le C_\psi^2\iota_T,
\end{aligned} log det ( λ I ) det ( Λ t , s j + λ I ) ≤ [ 1 + ϖ ( β ) ( d + 1 )] log 1 + λ [ 1 + ϖ ( β ) ( d + 1 )] tr ( Λ t , s j ) ≤ C ψ 2 [ 1 + ϖ ( β ) ( d + 1 )] log ( 1 + λ T ) ≤ C ψ 2 ι T ,
where the second inequality follows because
tr ( Λ t , s j ) = ∑ i ∈ Ψ t , s j ∥ ψ i , j ( w i ) ∥ 2 2 ≤ C ψ 2 ∣ Ψ t , s j ∣ ≤ C ψ 2 T . \operatorname{tr}\left(\Lambda^j_{t,s}\right)=\sum_{i\in\Psi^j_{t,s}}\lVert\psi_{i,j}(w_i)\rVert_2^2\le C_\psi^2\lvert\Psi^j_{t,s}\rvert\le C_\psi^2T. tr ( Λ t , s j ) = i ∈ Ψ t , s j ∑ ∥ ψ i , j ( w i ) ∥ 2 2 ≤ C ψ 2 ∣ Ψ t , s j ∣ ≤ C ψ 2 T .
It follows that, for this fixed pair ( s , j ) (s,j) ( s , j ) , with probability at least 1 − δ / ( 2 S N ) 1-\delta/(2SN) 1 − δ / ( 2 S N ) ,
∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i ∥ ( Λ t , s j + λ I ) − 1 ≤ C ψ D ι T \left\lVert\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)\varepsilon_i \right\rVert_{\left(\Lambda^j_{t,s}+\lambda I\right)^{-1}}\le C_\psi D\sqrt{\iota_T} i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i ( Λ t , s j + λ I ) − 1 ≤ C ψ D ι T
simultaneously for every $t\le T+1.
There are S N SN S N layer–bin pairs. Since the failure probability for each fixed pair is at most δ / ( 2 S N ) \delta/(2SN) δ / ( 2 S N ) , the probability that the preceding bound fails for at least one pair is at most
S N δ 2 S N = δ 2 . SN\frac{\delta}{2SN}=\frac{\delta}{2}. S N 2 S N δ = 2 δ .
Therefore, there exists an event E s c \mathcal{E}_{\mathrm{sc}} E sc with P ( E s c ) ≥ 1 − δ / 2 \mathbb{P}(\mathcal{E}_{\mathrm{sc}}) \ge1-\delta/2 P ( E sc ) ≥ 1 − δ /2 , on which, simultaneously for every t ≤ T + 1 t \le T+1 t ≤ T + 1 , s ∈ [ S ] s \in[S] s ∈ [ S ] , and j ∈ [ N ] j \in[N] j ∈ [ N ] ,
∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i ∥ ( Λ t , s j + λ I ) − 1 ≤ C ψ D v T . (36) \left\|\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)\varepsilon_i\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}} \le C_\psi D\sqrt{vT}. \tag*{(36)} i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i ( Λ t , s j + λ I ) − 1 ≤ C ψ D v T . (36)
We first control the term involving the approximation errors R i , j ( w i ) R_{i,j}(w_i) R i , j ( w i ) . For any vector a a a , the Cauchy–Schwarz inequality gives
∣ a ⊤ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) ∣ = ∣ ∑ i ∈ Ψ t , s j ( a ⊤ ψ i , j ( w i ) ) R i , j ( w i ) ∣ ≤ [ ∑ i ∈ Ψ t , s j ( a ⊤ ψ i , j ( w i ) ) 2 ] 1 / 2 × [ ∑ i ∈ Ψ t , s j R i , j ( w i ) 2 ] 1 / 2 . \left|a^\top\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)\right| = \left|\sum_{i\in\Psi^j_{t,s}}(a^\top\psi_{i,j}(w_i))R_{i,j}(w_i)\right|
\le\left[\sum_{i\in\Psi^j_{t,s}}(a^\top\psi_{i,j}(w_i))^2\right]^{1/2}\times\left[\sum_{i\in\Psi^j_{t,s}}R_{i,j}(w_i)^2\right]^{1/2}. a ⊤ i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) = i ∈ Ψ t , s j ∑ ( a ⊤ ψ i , j ( w i )) R i , j ( w i ) ≤ i ∈ Ψ t , s j ∑ ( a ⊤ ψ i , j ( w i ) ) 2 1/2 × i ∈ Ψ t , s j ∑ R i , j ( w i ) 2 1/2 .
By the definition of Λ t , s j \Lambda^j_{t,s} Λ t , s j ,
∑ i ∈ Ψ t , s j ( a ⊤ ψ i , j ( w i ) ) 2 = a ⊤ Λ t , s j a ≤ a ⊤ ( Λ t , s j + λ I ) a = ∥ a ∥ Λ t , s j + λ I 2 . \sum_{i\in\Psi^j_{t,s}}(a^\top\psi_{i,j}(w_i))^2 = a^\top\Lambda^j_{t,s}a \le a^\top(\Lambda^j_{t,s}+\lambda I)a = \|a\|^2_{\Lambda^j_{t,s}+\lambda I}. i ∈ Ψ t , s j ∑ ( a ⊤ ψ i , j ( w i ) ) 2 = a ⊤ Λ t , s j a ≤ a ⊤ ( Λ t , s j + λ I ) a = ∥ a ∥ Λ t , s j + λ I 2 .
Therefore,
∣ a ⊤ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) ∣ ≤ ∥ a ∥ Λ t , s j + λ I [ ∑ i ∈ Ψ t , s j R i , j ( w i ) 2 ] 1 / 2 . \left|a^\top\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)\right| \le\|a\|_{\Lambda^j_{t,s}+\lambda I}\left[\sum_{i\in\Psi^j_{t,s}}R_{i,j}(w_i)^2\right]^{1/2}. a ⊤ i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) ≤ ∥ a ∥ Λ t , s j + λ I i ∈ Ψ t , s j ∑ R i , j ( w i ) 2 1/2 .
Taking
a = ( Λ t , s j + λ I ) − 1 ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) a=(\Lambda^j_{t,s}+\lambda I)^{-1}\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i) a = ( Λ t , s j + λ I ) − 1 i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i )
in the preceding inequality yields
∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) ∥ ( Λ t , s j + λ I ) − 1 2 ≤ ∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) ∥ ( Λ t , s j + λ I ) − 1 [ ∑ i ∈ Ψ t , s j R i , j ( w i ) 2 ] 1 / 2 . \left\|\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)\right\|^2_{(\Lambda^j_{t,s}+\lambda I)^{-1}}
\le\left\|\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}\left[\sum_{i\in\Psi^j_{t,s}}R_{i,j}(w_i)^2\right]^{1/2}. i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) ( Λ t , s j + λ I ) − 1 2 ≤ i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) ( Λ t , s j + λ I ) − 1 i ∈ Ψ t , s j ∑ R i , j ( w i ) 2 1/2 .
Canceling the common factor when it is nonzero, while the zero case is immediate, gives
∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) ∥ ( Λ t , s j + λ I ) − 1 ≤ [ ∑ i ∈ Ψ t , s j R i , j ( w i ) 2 ] 1 / 2 . \left\|\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}
\le\left[\sum_{i\in\Psi^j_{t,s}}R_{i,j}(w_i)^2\right]^{1/2}. i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) ( Λ t , s j + λ I ) − 1 ≤ i ∈ Ψ t , s j ∑ R i , j ( w i ) 2 1/2 .
Since ∣ R i , j ( w i ) ∣ ≤ C l ( h β + η 2 ) |R_{i,j}(w_i)| \le C_l(h^\beta+\eta^2) ∣ R i , j ( w i ) ∣ ≤ C l ( h β + η 2 ) , for every i ∈ Ψ t , s j i\in\Psi^j_{t,s} i ∈ Ψ t , s j , we have
∑ i ∈ Ψ t , s j R i , j ( w i ) 2 ≤ C l 2 ∣ Ψ t , s j ∣ ( h β + η 2 ) 2 . \sum_{i\in\Psi^j_{t,s}}R_{i,j}(w_i)^2 \le C_l^2|\Psi^j_{t,s}|(h^\beta+\eta^2)^2. i ∈ Ψ t , s j ∑ R i , j ( w i ) 2 ≤ C l 2 ∣ Ψ t , s j ∣ ( h β + η 2 ) 2 .
It follows that
∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) ∥ ( Λ t , s j + λ I ) − 1 ≤ C l ∣ Ψ t , s j ∣ ( h β + η 2 ) . (37) \left\|\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}} \le C_l\sqrt{|\Psi^j_{t,s}|}(h^\beta+\eta^2). \tag*{(37)} i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) ( Λ t , s j + λ I ) − 1 ≤ C l ∣ Ψ t , s j ∣ ( h β + η 2 ) . (37)
It remains to control the term caused by the ridge regularization. Since Λ t , s j + λ I ⪰ λ I \Lambda^j_{t,s}+\lambda I\succeq\lambda I Λ t , s j + λ I ⪰ λ I , we have
( Λ t , s j + λ I ) − 1 ⪯ 1 λ I . (\Lambda^j_{t,s}+\lambda I)^{-1}\preceq\frac{1}{\lambda}I. ( Λ t , s j + λ I ) − 1 ⪯ λ 1 I .
Therefore,
∥ λ z j ∥ ( Λ t , s j + λ I ) − 1 2 = λ 2 z j ⊤ ( Λ t , s j + λ I ) − 1 z j ≤ λ ∥ z j ∥ 2 . \|\lambda z_j\|^2_{(\Lambda^j_{t,s}+\lambda I)^{-1}}=\lambda^2z_j^\top(\Lambda^j_{t,s}+\lambda I)^{-1}z_j\le\lambda\|z_j\|^2. ∥ λ z j ∥ ( Λ t , s j + λ I ) − 1 2 = λ 2 z j ⊤ ( Λ t , s j + λ I ) − 1 z j ≤ λ ∥ z j ∥ 2 .
Taking square roots and using (26) gives
∥ λ z j ∥ ( Λ t , s j + λ I ) − 1 ≤ λ ∥ z j ∥ ≤ C z λ . (38) \|\lambda z_j\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}\le\sqrt{\lambda}\|z_j\|\le C_z\sqrt{\lambda}. \tag*{(38)} ∥ λ z j ∥ ( Λ t , s j + λ I ) − 1 ≤ λ ∥ z j ∥ ≤ C z λ . (38)
Step 5: uniform confidence bound and completion. We work on the event E p i l ∩ E s c \mathcal{E}_{\mathrm{pil}}\cap\mathcal{E}_{\mathrm{sc}} E pil ∩ E sc , which has probability at least 1 − δ 1-\delta 1 − δ . Fix a main-policy round t t t , a layer s ∈ [ s t ] s\in[s_t] s ∈ [ s t ] , and an action ( j , w ) ∈ A t , s (j,w)\in\mathcal{A}_{t,s} ( j , w ) ∈ A t , s . Assume first that ∣ Ψ t , s j ∣ ≥ 1 |\Psi^j_{t,s}|\ge1 ∣ Ψ t , s j ∣ ≥ 1 .
By (34) and multiplying both sides by ψ t , j ( w ) ⊤ \psi_{t,j}(w)^\top ψ t , j ( w ) ⊤ , taking absolute values, and applying the triangle inequality gives
∣ ψ t , j ( w ) ⊤ ( z ^ t , s j − z j ) ∣ ≤ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 [ ∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) ε i ∥ ( Λ t , s j + λ I ) − 1 + ∥ ∑ i ∈ Ψ t , s j ψ i , j ( w i ) R i , j ( w i ) ∥ ( Λ t , s j + λ I ) − 1 + ∥ λ z j ∥ ( Λ t , s j + λ I ) − 1 ] . \left|\psi_{t,j}(w)^\top\left(\widehat{z}^j_{t,s}-z_j\right)\right|
\le\|\psi_{t,j}(w)\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}
\left[
\left\|\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)\varepsilon_i\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}
+\left\|\sum_{i\in\Psi^j_{t,s}}\psi_{i,j}(w_i)R_{i,j}(w_i)\right\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}
+\|\lambda z_j\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}
\right]. ψ t , j ( w ) ⊤ ( z t , s j − z j ) ≤ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 i ∈ Ψ t , s j ∑ ψ i , j ( w i ) ε i ( Λ t , s j + λ I ) − 1 + i ∈ Ψ t , s j ∑ ψ i , j ( w i ) R i , j ( w i ) ( Λ t , s j + λ I ) − 1 + ∥ λ z j ∥ ( Λ t , s j + λ I ) − 1 .
Using (36) , (37) , and (38) , we obtain
∣ ψ t , j ( w ) ⊤ ( z ^ t , s j − z j ) ∣ ≤ ( C ψ D ι T + C z λ ) ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) ∣ Ψ t , s j ∣ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 . \left|\psi_{t,j}(w)^\top\left(\widehat{z}^j_{t,s}-z_j\right)\right|
\le\left(C_\psi D\sqrt{\iota_T}+C_z\sqrt{\lambda}\right)\|\psi_{t,j}(w)\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}
+C_l(h^\beta+\eta^2)\sqrt{|\Psi^j_{t,s}|}\,\|\psi_{t,j}(w)\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}. ψ t , j ( w ) ⊤ ( z t , s j − z j ) ≤ ( C ψ D ι T + C z λ ) ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) ∣ Ψ t , s j ∣ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 .
The preceding inequality controls the error caused by replacing z j z_j z j with its estimator z ^ t , s j \widehat{z}^j_{t,s} z t , s j . We must also account for the approximation error at the current action. By (33) ,
g ( p ^ t ( w ) − x t ⊤ θ ⋆ ) = ψ t , j ( w ) ⊤ z j + R t , j ( w ) , g(\widehat{p}_t(w)-x_t^\top\theta_\star)=\psi_{t,j}(w)^\top z_j+R_{t,j}(w), g ( p t ( w ) − x t ⊤ θ ⋆ ) = ψ t , j ( w ) ⊤ z j + R t , j ( w ) ,
where ∣ R t , j ( w ) ∣ ≤ C l ( h β + η 2 ) |R_{t,j}(w)|\le C_l(h^\beta+\eta^2) ∣ R t , j ( w ) ∣ ≤ C l ( h β + η 2 ) . Therefore,
∣ ψ t , j ( w ) ⊤ z ^ t , s j − g ( p ^ t ( w ) − x t ⊤ θ ⋆ ) ∣ = ∣ ψ t , j ( w ) ⊤ ( z ^ t , s j − z j ) − R t , j ( w ) ∣ ≤ ∣ ψ t , j ( w ) ⊤ ( z ^ t , s j − z j ) ∣ + ∣ R t , j ( w ) ∣ ≤ ( C ψ D ι T + C z λ ) ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) [ 1 + ∣ Ψ t , s j ∣ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 ] . \begin{aligned}
\left|\psi_{t,j}(w)^\top\widehat{z}^j_{t,s}-g(\widehat{p}_t(w)-x_t^\top\theta_\star)\right|
&=\left|\psi_{t,j}(w)^\top\left(\widehat{z}^j_{t,s}-z_j\right)-R_{t,j}(w)\right|\\
&\le\left|\psi_{t,j}(w)^\top\left(\widehat{z}^j_{t,s}-z_j\right)\right|+|R_{t,j}(w)|\\
&\le\left(C_\psi D\sqrt{\iota_T}+C_z\sqrt{\lambda}\right)\|\psi_{t,j}(w)\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}\\
&\quad+C_l(h^\beta+\eta^2)\left[1+\sqrt{|\Psi^j_{t,s}|}\,\|\psi_{t,j}(w)\|_{(\Lambda^j_{t,s}+\lambda I)^{-1}}\right].
\end{aligned} ψ t , j ( w ) ⊤ z t , s j − g ( p t ( w ) − x t ⊤ θ ⋆ ) = ψ t , j ( w ) ⊤ ( z t , s j − z j ) − R t , j ( w ) ≤ ψ t , j ( w ) ⊤ ( z t , s j − z j ) + ∣ R t , j ( w ) ∣ ≤ ( C ψ D ι T + C z λ ) ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) [ 1 + ∣ Ψ t , s j ∣ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 ] .
The algorithm restricts the estimated demand to the interval [ 0 , D ] [0,D] [ 0 , D ] :
g ^ t , s j ( w ) = Π [ 0 , D ] ( ψ t , j ( w ) T z ^ t , s j ) . \widehat{g}^{j}_{t,s}(w)=\Pi_{[0,D]}\left(\psi_{t,j}(w)^{\mathsf{T}}\widehat{z}^{j}_{t,s}\right). g t , s j ( w ) = Π [ 0 , D ] ( ψ t , j ( w ) T z t , s j ) .
This restriction cannot increase the error because the true mean demand also belongs to [ 0 , D ] [0,D] [ 0 , D ] . Indeed, for every a ∈ R a\in\mathbb{R} a ∈ R and every b ∈ [ 0 , D ] b\in[0,D] b ∈ [ 0 , D ] , ∣ Π [ 0 , D ] ( a ) − b ∣ ≤ ∣ a − b ∣ \left|\Pi_{[0,D]}(a)-b\right|\leq|a-b| Π [ 0 , D ] ( a ) − b ≤ ∣ a − b ∣ . Moreover, both Π [ 0 , D ] ( a ) \Pi_{[0,D]}(a) Π [ 0 , D ] ( a ) and b b b belong to [ 0 , D ] [0,D] [ 0 , D ] , so ∣ Π [ 0 , D ] ( a ) − b ∣ ≤ D \left|\Pi_{[0,D]}(a)-b\right|\leq D Π [ 0 , D ] ( a ) − b ≤ D . Applying these two inequalities with a = ψ t , j ( w ) T z ^ t , s j a=\psi_{t,j}(w)^{\mathsf{T}}\widehat{z}^{j}_{t,s} a = ψ t , j ( w ) T z t , s j and b = g ( p ^ t ( w ) − x t T θ ⋆ ) b=g(\widehat{p}_{t}(w)-x_{t}^{\mathsf{T}}\theta_{\star}) b = g ( p t ( w ) − x t T θ ⋆ ) gives
∣ g ^ t , s j ( w ) − g ( p ^ t ( w ) − x t T θ ⋆ ) ∣ ≤ min { D , ( C ψ D ι T + C z λ ) ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) [ 1 + ∣ Ψ t , s j ∣ ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 ] } ≤ r t , s j ( w ) . \begin{aligned}
\left|\widehat{g}^{j}_{t,s}(w)-g\left(\widehat{p}_{t}(w)-x_{t}^{\mathsf{T}}\theta_{\star}\right)\right|
&\leq\min\Biggl\{D,\left(C_{\psi}D\sqrt{\iota T}+C_{z}\sqrt{\lambda}\right)\left\|\psi_{t,j}(w)\right\|_{\left(\Lambda^{j}_{t,s}+\lambda I\right)^{-1}}\\
&\qquad+C_{l}\left(h^{\beta}+\eta^{2}\right)\left[1+\sqrt{\left|\Psi^{j}_{t,s}\right|}\left\|\psi_{t,j}(w)\right\|_{\left(\Lambda^{j}_{t,s}+\lambda I\right)^{-1}}\right]\Biggr\}\\
&\leq r^{j}_{t,s}(w).
\end{aligned} g t , s j ( w ) − g ( p t ( w ) − x t T θ ⋆ ) ≤ min { D , ( C ψ D ι T + C z λ ) ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 + C l ( h β + η 2 ) [ 1 + Ψ t , s j ∥ ψ t , j ( w ) ∥ ( Λ t , s j + λ I ) − 1 ] } ≤ r t , s j ( w ) .
It remains to consider the case ∣ Ψ t , s j ∣ = 0 \left|\Psi^{j}_{t,s}\right|=0 Ψ t , s j = 0 . Then Ψ t , s j = ∅ \Psi^{j}_{t,s}=\varnothing Ψ t , s j = ∅ and Λ t , s j = 0 \Lambda^{j}_{t,s}=0 Λ t , s j = 0 . The optimization problem reduces to min z λ ∥ z ∥ 2 2 \min_{z}\lambda\|z\|_{2}^{2} min z λ ∥ z ∥ 2 2 . Since λ > 0 \lambda>0 λ > 0 , its unique minimizer is z ^ t , s j = 0 \widehat{z}^{j}_{t,s}=0 z t , s j = 0 . It follows that g ^ t , s j ( w ) = Π [ 0 , D ] ( 0 ) = 0 \widehat{g}^{j}_{t,s}(w)=\Pi_{[0,D]}(0)=0 g t , s j ( w ) = Π [ 0 , D ] ( 0 ) = 0 . The algorithm sets r t , s j ( w ) = D r^{j}_{t,s}(w)=D r t , s j ( w ) = D . Since the true mean demand belongs to [ 0 , D ] [0,D] [ 0 , D ] ,
∣ g ^ t , s j ( w ) − g ( p ^ t ( w ) − x t T θ ⋆ ) ∣ = ∣ g ( p ^ t ( w ) − x t T θ ⋆ ) ∣ ≤ D = r t , s j ( w ) . \left|\widehat{g}^{j}_{t,s}(w)-g\left(\widehat{p}_{t}(w)-x_{t}^{\mathsf{T}}\theta_{\star}\right)\right|
=\left|g\left(\widehat{p}_{t}(w)-x_{t}^{\mathsf{T}}\theta_{\star}\right)\right|
\leq D=r^{j}_{t,s}(w). g t , s j ( w ) − g ( p t ( w ) − x t T θ ⋆ ) = g ( p t ( w ) − x t T θ ⋆ ) ≤ D = r t , s j ( w ) .
Finally, the event E p i l \mathcal{E}_{\mathrm{pil}} E pil controls all periods simultaneously, and the event E s c \mathcal{E}_{\mathrm{sc}} E sc controls all periods, layers, and bins simultaneously. Once these two events occur, the preceding inequalities can be applied to every residual action w w w because they use only the already established vector bounds and the Cauchy–Schwarz inequality. Thus no separate probability bound is needed for each w w w . Therefore, (9) holds simultaneously for every main-policy round t t t , every s ∈ [ s t ] s\in[s_{t}] s ∈ [ s t ] , and every ( j , w ) ∈ A t , s (j,w)\in\mathcal{A}_{t,s} ( j , w ) ∈ A t , s .
Proofs for the Contextual Regret Bounds
Proof of Theorem 5.1.
Proof. We proceed in four steps.
Step 1: Revenue-UCB sandwich. Fix a visited layer s s s and an action ( j , w ) ∈ A t , s (j,w)\in\mathcal{A}_{t,s} ( j , w ) ∈ A t , s . On the confidence event,
∣ g ^ t , s j ( w ) − g ( p ^ t ( w ) − x t T θ ⋆ ) ∣ ≤ r t , s j ( w ) . \left|\widehat{g}^{j}_{t,s}(w)-g\left(\widehat{p}_{t}(w)-x_{t}^{\mathsf{T}}\theta_{\star}\right)\right|\leq r^{j}_{t,s}(w). g t , s j ( w ) − g ( p t ( w ) − x t T θ ⋆ ) ≤ r t , s j ( w ) .
Because the true conditional mean belongs to [ 0 , D ] [0,D] [ 0 , D ] ,
g ( p ^ t ( w ) − x t T θ ⋆ ) ≤ min { D , g ^ t , s j ( w ) + r t , s j ( w ) } ≤ g ( p ^ t ( w ) − x t T θ ⋆ ) + 2 r t , s j ( w ) . g\left(\widehat{p}_{t}(w)-x_{t}^{\mathsf{T}}\theta_{\star}\right)\leq\min\left\{D,\widehat{g}^{j}_{t,s}(w)+r^{j}_{t,s}(w)\right\}\leq g\left(\widehat{p}_{t}(w)-x_{t}^{\mathsf{T}}\theta_{\star}\right)+2r^{j}_{t,s}(w). g ( p t ( w ) − x t T θ ⋆ ) ≤ min { D , g t , s j ( w ) + r t , s j ( w ) } ≤ g ( p t ( w ) − x t T θ ⋆ ) + 2 r t , s j ( w ) .
Multiplying by p ^ t ( w ) ≥ 0 \widehat{p}_{t}(w)\geq0 p t ( w ) ≥ 0 and using the definitions of U t , s j ( w ) U^{j}_{t,s}(w) U t , s j ( w ) and wid t , s j ( w ) \operatorname{wid}^{j}_{t,s}(w) wid t , s j ( w ) gives
Rev ( x t , p ^ t ( w ) ) ≤ U t , s j ( w ) ≤ Rev ( x t , p ^ t ( w ) ) + 2 wid t , s j ( w ) . (39) \operatorname{Rev}(x_{t},\widehat{p}_{t}(w))\leq U^{j}_{t,s}(w)\leq\operatorname{Rev}(x_{t},\widehat{p}_{t}(w))+2\operatorname{wid}^{j}_{t,s}(w). \tag*{(39)} Rev ( x t , p t ( w )) ≤ U t , s j ( w ) ≤ Rev ( x t , p t ( w )) + 2 wid t , s j ( w ) . (39)
Step 2: Preservation of the discrete benchmark. Suppose that the precision check passes at a layer s < S s < S s < S . Then
wid t , s j ( w ) ≤ B D 2 − s for every ( j , w ) ∈ A t , s . \operatorname{wid}_{t,s}^{j}(w) \le BD 2^{-s} \qquad\text{for every } (j,w) \in\mathcal{A}_{t,s}. wid t , s j ( w ) ≤ B D 2 − s for every ( j , w ) ∈ A t , s .
Choose ( j ⋆ , w ⋆ ) ∈ argmax ( j , w ) ∈ A t , s Rev ( x t , p ^ t ( w ) ) (j^\star,w^\star) \in\operatorname*{argmax}_{(j,w)\in\mathcal{A}_{t,s}} \operatorname{Rev}(x_t,\widehat{p}_t(w)) ( j ⋆ , w ⋆ ) ∈ argmax ( j , w ) ∈ A t , s Rev ( x t , p t ( w )) . By the lower side of (39), U t , s j ⋆ ( w ⋆ ) ≥ V t , s U_{t,s}^{j^\star}(w^\star) \ge V_{t,s} U t , s j ⋆ ( w ⋆ ) ≥ V t , s . By its upper side, for every ( j , w ) ∈ A t , s (j,w) \in\mathcal{A}_{t,s} ( j , w ) ∈ A t , s ,
U t , s j ( w ) ≤ V t , s + 2 B D 2 − s = V t , s + B D 2 1 − s . U_{t,s}^{j}(w) \le V_{t,s} + 2BD 2^{-s} = V_{t,s} + BD 2^{1-s}. U t , s j ( w ) ≤ V t , s + 2 B D 2 − s = V t , s + B D 2 1 − s .
Consequently,
U t , s j ⋆ ( w ⋆ ) ≥ max ( j , w ) ∈ A t , s U t , s j ( w ) − B D 2 1 − s . U_{t,s}^{j^\star}(w^\star) \ge\max_{(j,w)\in\mathcal{A}_{t,s}} U_{t,s}^{j}(w) - BD 2^{1-s}. U t , s j ⋆ ( w ⋆ ) ≥ ( j , w ) ∈ A t , s max U t , s j ( w ) − B D 2 1 − s .
By the definition of A t , s + 1 \mathcal{A}_{t,s+1} A t , s + 1 in Algorithm 2, the preceding inequality implies that ( j ⋆ , w ⋆ ) ∈ A t , s + 1 (j^\star,w^\star) \in\mathcal{A}_{t,s+1} ( j ⋆ , w ⋆ ) ∈ A t , s + 1 . Since ( j ⋆ , w ⋆ ) (j^\star,w^\star) ( j ⋆ , w ⋆ ) attains V t , s V_{t,s} V t , s , it follows that
V t , s + 1 ≥ Rev ( x t , p ^ t ( w ⋆ ) ) = V t , s . V_{t,s+1} \ge\operatorname{Rev}(x_t,\widehat{p}_t(w^\star)) = V_{t,s}. V t , s + 1 ≥ Rev ( x t , p t ( w ⋆ )) = V t , s .
On the other hand, A t , s + 1 ⊆ A t , s \mathcal{A}_{t,s+1} \subseteq\mathcal{A}_{t,s} A t , s + 1 ⊆ A t , s , and hence V t , s + 1 ≤ V t , s V_{t,s+1} \le V_{t,s} V t , s + 1 ≤ V t , s . Therefore, V t , s + 1 = V t , s V_{t,s+1} = V_{t,s} V t , s + 1 = V t , s .
Step 3: Gap between the selected price and the initial discrete benchmark. Suppose that s t ≥ 2 s_t \ge2 s t ≥ 2 . To reach layer s t s_t s t , the algorithm must have passed the precision check at layer s t − 1 s_t - 1 s t − 1 . Moreover, ( j t , w t ) ∈ A t , s t (j_t,w_t) \in\mathcal{A}_{t,s_t} ( j t , w t ) ∈ A t , s t , so this action survived the refinement from layer s t − 1 s_t - 1 s t − 1 to layer s t s_t s t . Hence
U t , s t − 1 j t ( w t ) ≥ max ( j , w ) ∈ A t , s t − 1 U t , s t − 1 j ( w ) − B D 2 2 − s t ≥ V t , s t − 1 − B D 2 2 − s t . (40) U_{t,s_t-1}^{j_t}(w_t) \ge\max_{(j,w)\in\mathcal{A}_{t,s_t-1}} U_{t,s_t-1}^{j}(w) - BD 2^{2-s_t} \ge V_{t,s_t-1} - BD 2^{2-s_t}. \tag*{(40)} U t , s t − 1 j t ( w t ) ≥ ( j , w ) ∈ A t , s t − 1 max U t , s t − 1 j ( w ) − B D 2 2 − s t ≥ V t , s t − 1 − B D 2 2 − s t . (40)
The upper side of (39) and the precision check at layer s t − 1 s_t - 1 s t − 1 give
U t , s t − 1 j t ( w t ) ≤ Rev ( x t , p t ) + 2 B D 2 − ( s t − 1 ) = Rev ( x t , p t ) + B D 2 2 − s t . (41) U_{t,s_t-1}^{j_t}(w_t) \le\operatorname{Rev}(x_t,p_t) + 2BD 2^{-(s_t-1)} = \operatorname{Rev}(x_t,p_t) + BD 2^{2-s_t}. \tag*{(41)} U t , s t − 1 j t ( w t ) ≤ Rev ( x t , p t ) + 2 B D 2 − ( s t − 1 ) = Rev ( x t , p t ) + B D 2 2 − s t . (41)
Combining (40) and (41) yields V t , s t − 1 − Rev ( x t , p t ) ≤ 2 B D 2 2 − s t V_{t,s_t-1} - \operatorname{Rev}(x_t,p_t) \le2BD 2^{2-s_t} V t , s t − 1 − Rev ( x t , p t ) ≤ 2 B D 2 2 − s t . Every layer preceding s t s_t s t passed the precision check. Repeated application of Step 2 therefore gives V t , s t − 1 = V t , 1 V_{t,s_t-1} = V_{t,1} V t , s t − 1 = V t , 1 , which proves the second assertion of the lemma.
Step 4: Discretization of the continuous oracle price. If β = 1 \beta= 1 β = 1 , Assumption 3.4 implies that g g g is L g L_g L g -Lipschitz. If β > 1 \beta> 1 β > 1 , Step 2 of the proof of Proposition 4.2 gives sup u ∈ I g ∣ g ′ ( u ) ∣ ≤ C g \sup_{u\in\mathcal{I}_g} |g'(u)| \le C_g sup u ∈ I g ∣ g ′ ( u ) ∣ ≤ C g . Hence g g g is Lipschitz on I g \mathcal{I}_g I g with constant max { L g , C g } \max\{L_g,C_g\} max { L g , C g } . Since both residual arguments below belong to I g \mathcal{I}_g I g by Assumption 3.1, for every x ∈ X x \in\mathcal{X} x ∈ X and p , p ′ ∈ [ 0 , B ] p,p' \in[0,B] p , p ′ ∈ [ 0 , B ] ,
∣ Rev ( x , p ) − Rev ( x , p ′ ) ∣ ≤ D ∣ p − p ′ ∣ + B ∣ g ( p − x ⊤ θ ⋆ ) − g ( p ′ − x ⊤ θ ⋆ ) ∣ ≤ L Rev ∣ p − p ′ ∣ , \left|\operatorname{Rev}(x,p) - \operatorname{Rev}(x,p')\right| \le D|p-p'| + B\left|g(p-x^\top\theta_\star)-g(p'-x^\top\theta_\star)\right| \le L_{\operatorname{Rev}}|p-p'|, ∣ Rev ( x , p ) − Rev ( x , p ′ ) ∣ ≤ D ∣ p − p ′ ∣ + B g ( p − x ⊤ θ ⋆ ) − g ( p ′ − x ⊤ θ ⋆ ) ≤ L Rev ∣ p − p ′ ∣ ,
where one may take L Rev = D + B max { L g , C g } L_{\operatorname{Rev}} = D + B\max\{L_g,C_g\} L Rev = D + B max { L g , C g } .
The oracle residual p t ⋆ − Π [ 0 , B ] ( x t ⊤ θ ^ t ) p_t^\star-\Pi_{[0,B]}(x_t^\top\widehat{\theta}_t) p t ⋆ − Π [ 0 , B ] ( x t ⊤ θ t ) belongs to [ − B , B ] [-B,B] [ − B , B ] . The feasible residual set is the interval [ − Π [ 0 , B ] ( x t ⊤ θ ^ t ) , B − Π [ 0 , B ] ( x t ⊤ θ ^ t ) ] [-\Pi_{[0,B]}(x_t^\top\widehat{\theta}_t), B-\Pi_{[0,B]}(x_t^\top\widehat{\theta}_t)] [ − Π [ 0 , B ] ( x t ⊤ θ t ) , B − Π [ 0 , B ] ( x t ⊤ θ t )] , which contains both zero and the oracle residual. By choosing a point of W \mathcal{W} W between zero and the oracle residual, the mesh definition of W \mathcal{W} W , together with the explicitly included endpoints, gives a feasible w ∈ W w \in\mathcal{W} w ∈ W satisfying
∣ w − [ p t ⋆ − Π [ 0 , B ] ( x t ⊤ θ ^ t ) ] ∣ ≤ T − 1 / 2 . \left|w-\left[p_t^\star-\Pi_{[0,B]}(x_t^\top\widehat{\theta}_t)\right]\right| \le T^{-1/2}. w − [ p t ⋆ − Π [ 0 , B ] ( x t ⊤ θ t ) ] ≤ T − 1/2 .
Assigning this w w w to its unique bin yields an action in A t , 1 \mathcal{A}_{t,1} A t , 1 , and ∣ p ^ t ( w ) − p t ⋆ ∣ ≤ T − 1 / 2 |\widehat{p}_t(w)-p_t^\star|\le T^{-1/2} ∣ p t ( w ) − p t ⋆ ∣ ≤ T − 1/2 . Hence
Rev ( x t , p t ⋆ ) − V t , 1 ≤ L Rev T . \operatorname{Rev}(x_t,p_t^\star)-V_{t,1}\le\frac{L_{\operatorname{Rev}}}{\sqrt{T}}. Rev ( x t , p t ⋆ ) − V t , 1 ≤ T L Rev .
If s t ≥ 2 s_t\ge2 s t ≥ 2 , combining this inequality with the bound established in Step 3 proves (10) . If s t = 1 s_t=1 s t = 1 , then 0 ≤ Rev ( x t , p ) ≤ B D 0\le\operatorname{Rev}(x_t,p)\le BD 0 ≤ Rev ( x t , p ) ≤ B D for every feasible price, so
Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ≤ L Rev T + V t , 1 − Rev ( x t , p t ) ≤ L Rev T + B D ≤ L Rev T + 8 B D 2 − s t . \operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)\le\frac{L_{\operatorname{Rev}}}{\sqrt{T}}+V_{t,1}-\operatorname{Rev}(x_t,p_t)\le\frac{L_{\operatorname{Rev}}}{\sqrt{T}}+BD\le\frac{L_{\operatorname{Rev}}}{\sqrt{T}}+8BD2^{-s_t}. Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ≤ T L Rev + V t , 1 − Rev ( x t , p t ) ≤ T L Rev + B D ≤ T L Rev + 8 B D 2 − s t .
Thus the same conclusion holds for s t = 1 s_t=1 s t = 1 . □
Proof of counting lemmas.
Lemma A.2 (Cumulative revenue uncertainty within one layer) There exists a finite constant C ω ≥ 1 C_\omega\ge1 C ω ≥ 1 , such that, for every s ∈ [ S ] s\in[S] s ∈ [ S ] , pathwise,
∑ t ∈ Ψ T + 1 , s wid t , s j t ( w t ) ≤ C ω B [ ( D T + λ ) N ∣ Ψ T + 1 , s ∣ ι T + ( h β + η 2 ) ∣ Ψ T + 1 , s ∣ ι T ] . (42) \sum_{t\in\Psi_{T+1,s}}\operatorname{wid}_{t,s}^{j_t}(w_t)\le C_\omega B\left[(D\sqrt{T}+\sqrt{\lambda})\sqrt{N|\Psi_{T+1,s}|}\sqrt{\iota_T}+(h^\beta+\eta^2)|\Psi_{T+1,s}|\sqrt{\iota_T}\right]. \tag*{(42)} t ∈ Ψ T + 1 , s ∑ wid t , s j t ( w t ) ≤ C ω B [ ( D T + λ ) N ∣ Ψ T + 1 , s ∣ ι T + ( h β + η 2 ) ∣ Ψ T + 1 , s ∣ ι T ] . (42)
The right-hand side is interpreted as zero when Ψ T + 1 , s = ∅ \Psi_{T+1,s}=\varnothing Ψ T + 1 , s = ∅ .
Proof. Fix s ∈ [ S ] s\in[S] s ∈ [ S ] . The claim is immediate if Ψ T + 1 , s = ∅ \Psi_{T+1,s}=\varnothing Ψ T + 1 , s = ∅ . We first work within one nonempty bin and then sum over bins.
Step 1: Sequential Gram matrices within a permanent bin. Fix j ∈ [ N ] j\in[N] j ∈ [ N ] with Ψ T + 1 , s j ≠ ∅ \Psi_{T+1,s}^{j}\ne\varnothing Ψ T + 1 , s j = ∅ , and enumerate its observations chronologically as
Ψ T + 1 , s j = { t j , 1 < ⋯ < t j , ∣ Ψ T + 1 , s j ∣ } . \Psi_{T+1,s}^{j}=\{t_{j,1}<\cdots<t_{j,|\Psi_{T+1,s}^{j}|}\}. Ψ T + 1 , s j = { t j , 1 < ⋯ < t j , ∣ Ψ T + 1 , s j ∣ } .
Permanent labeling and the definition of Λ t , s j \Lambda_{t,s}^{j} Λ t , s j imply that, for every k ≤ ∣ Ψ T + 1 , s j ∣ k\le|\Psi_{T+1,s}^{j}| k ≤ ∣ Ψ T + 1 , s j ∣ ,
Λ t j , k , s j + λ I = λ I + ∑ ℓ < k ψ t j , ℓ , j ( w t j , ℓ ) ψ t j , ℓ , j ( w t j , ℓ ) ⊤ , ∣ Ψ t j , k , s j ∣ = k − 1. (43) \Lambda_{t_{j,k},s}^{j}+\lambda I=\lambda I+\sum_{\ell<k}\psi_{t_{j,\ell},j}(w_{t_{j,\ell}})\psi_{t_{j,\ell},j}(w_{t_{j,\ell}})^\top,\qquad|\Psi_{t_{j,k},s}^{j}|=k-1. \tag*{(43)} Λ t j , k , s j + λ I = λ I + ℓ < k ∑ ψ t j , ℓ , j ( w t j , ℓ ) ψ t j , ℓ , j ( w t j , ℓ ) ⊤ , ∣ Ψ t j , k , s j ∣ = k − 1. (43)
Step 2: Elliptical-potential bound. The uniform feature bound established in the proof of Proposition 4.2 gives ∥ ψ t , j ( w t ) ∥ 2 ≤ C ψ \|\psi_{t,j}(w_t)\|_2\le C_\psi ∥ ψ t , j ( w t ) ∥ 2 ≤ C ψ . Applying the matrix determinant lemma sequentially to (43) , and using min { 1 , x } ≤ 2 log ( 1 + x ) \min\{1,x\}\le2\log(1+x) min { 1 , x } ≤ 2 log ( 1 + x ) for x ≥ 0 x\ge0 x ≥ 0 , yields
∑ k = 1 ∣ Ψ T + 1 , s j ∣ min { 1 , ∥ ψ t j , k , j ( w t j , k ) ∥ ( Λ t j , k , s j + λ I ) − 1 2 } ≤ 2 log det ( λ I + ∑ k = 1 ∣ Ψ T + 1 , s j ∣ ψ t j , k , j ( w t j , k ) ψ t j , k , j ( w t j , k ) ⊤ ) det ( λ I ) ≤ 2 [ 1 + ϖ ( β ) ( d + 1 ) ] log ( 1 + C ψ 2 ∣ Ψ T + 1 , s j ∣ λ [ 1 + ϖ ( β ) ( d + 1 ) ] ) ≤ 2 max { 1 , C ψ 2 } ι T . (44) \begin{aligned}
\sum_{k=1}^{|\Psi_{T+1,s}^{j}|}\min\left\{1,\left\|\psi_{t_{j,k},j}(w_{t_{j,k}})\right\|_{\left(\Lambda_{t_{j,k},s}^{j}+\lambda I\right)^{-1}}^2\right\}
&\le2\log\frac{\det\left(\lambda I+\sum_{k=1}^{|\Psi_{T+1,s}^{j}|}\psi_{t_{j,k},j}(w_{t_{j,k}})\psi_{t_{j,k},j}(w_{t_{j,k}})^\top\right)}{\det(\lambda I)}\\
&\le2[1+\varpi(\beta)(d+1)]\log\left(1+\frac{C_\psi^2|\Psi_{T+1,s}^{j}|}{\lambda[1+\varpi(\beta)(d+1)]}\right)\\
&\le2\max\{1,C_\psi^2\}\iota_T. \tag*{(44)}
\end{aligned} k = 1 ∑ ∣ Ψ T + 1 , s j ∣ min { 1 , ψ t j , k , j ( w t j , k ) ( Λ t j , k , s j + λ I ) − 1 2 } ≤ 2 log det ( λ I ) det ( λ I + ∑ k = 1 ∣ Ψ T + 1 , s j ∣ ψ t j , k , j ( w t j , k ) ψ t j , k , j ( w t j , k ) ⊤ ) ≤ 2 [ 1 + ϖ ( β ) ( d + 1 )] log ( 1 + λ [ 1 + ϖ ( β ) ( d + 1 )] C ψ 2 ∣ Ψ T + 1 , s j ∣ ) ≤ 2 max { 1 , C ψ 2 } ι T . (44)
The last inequality follows from ∣ Ψ T + 1 , s j ∣ ≤ T |\Psi^{j}_{T+1,s}| \le T ∣ Ψ T + 1 , s j ∣ ≤ T , the definition of ι T \iota_T ι T , and log ( 1 + a x ) ≤ max { 1 , a } log ( 1 + x ) \log(1+ax) \le\max\{1,a\}\log(1+x) log ( 1 + a x ) ≤ max { 1 , a } log ( 1 + x ) for a , x ≥ 0 a,x \ge0 a , x ≥ 0 .
By Cauchy–Schwarz and (44),
∑ k = 1 ∣ Ψ T + 1 , s j ∣ min { 1 , ∥ ψ t j , k , j ( w t j , k ) ∥ ( Λ t j , k , s j + λ I ) − 1 } ≤ ∣ Ψ T + 1 , s j ∣ ∑ k = 1 ∣ Ψ T + 1 , s j ∣ min { 1 , ∥ ψ t j , k , j ( w t j , k ) ∥ ( Λ t j , k , s j + λ I ) − 1 2 } ≤ 2 max { 1 , C ψ 2 } ∣ Ψ T + 1 , s j ∣ ι T . (45) \begin{aligned}
\sum_{k=1}^{|\Psi^{j}_{T+1,s}|} \min\left\{1,\left\|\psi_{t_{j,k,j}}(w_{t_{j,k}})\right\|\left(\Lambda^{j}_{t_{j,k,s}}+\lambda I\right)^{-1}\right\} \\
&\le\sqrt{|\Psi^{j}_{T+1,s}| \sum_{k=1}^{|\Psi^{j}_{T+1,s}|} \min\left\{1,\left\|\psi_{t_{j,k,j}}(w_{t_{j,k}})\right\|^{2}_{\left(\Lambda^{j}_{t_{j,k,s}}+\lambda I\right)^{-1}}\right\}}
\le\sqrt{2\max\{1,C_{\psi}^{2}\}}\sqrt{|\Psi^{j}_{T+1,s}|\iota_T}. \tag*{(45)}
\end{aligned} k = 1 ∑ ∣ Ψ T + 1 , s j ∣ min { 1 , ψ t j , k , j ( w t j , k ) ( Λ t j , k , s j + λ I ) − 1 } ≤ ∣ Ψ T + 1 , s j ∣ k = 1 ∑ ∣ Ψ T + 1 , s j ∣ min { 1 , ψ t j , k , j ( w t j , k ) ( Λ t j , k , s j + λ I ) − 1 2 } ≤ 2 max { 1 , C ψ 2 } ∣ Ψ T + 1 , s j ∣ ι T . (45)
Step 3: Sum of confidence radii within one bin. The first observation in the bin satisfies ∣ Ψ t j , 1 , s j ∣ = 0 |\Psi^{j}_{t_{j,1,s}}|=0 ∣ Ψ t j , 1 , s j ∣ = 0 , and therefore (7) gives r t j , 1 , s j ( w t j , 1 ) = D r^{j}_{t_{j,1,s}}(w_{t_{j,1}})=D r t j , 1 , s j ( w t j , 1 ) = D . For k ≥ 2 k\ge2 k ≥ 2 , we have ∣ Ψ t j , k , s j ∣ = k − 1 ≤ ∣ Ψ T + 1 , s j ∣ |\Psi^{j}_{t_{j,k,s}}|=k-1\le|\Psi^{j}_{T+1,s}| ∣ Ψ t j , k , s j ∣ = k − 1 ≤ ∣ Ψ T + 1 , s j ∣ . Therefore, the radius definition and its truncation at D D D imply
r t j , k , s j ( w t j , k ) ≤ max { C ψ , C z } ( ( D ι T + λ ) min { 1 , ∥ ψ t j , k , j ( w t j , k ) ∥ ( Λ t j , k , s j + λ I ) − 1 } + C l ( h β + η 2 ) [ 1 + ∣ Ψ T + 1 , s j ∣ min { 1 , ∥ ψ t j , k , j ( w t j , k ) ∥ ( Λ t j , k , s j + λ I ) − 1 } ] ) . (46) r^{j}_{t_{j,k,s}}(w_{t_{j,k}}) \le\max\{C_{\psi},C_{z}\}\left((D\sqrt{\iota_T}+\sqrt{\lambda})\min\left\{1,\left\|\psi_{t_{j,k,j}}(w_{t_{j,k}})\right\|_{\left(\Lambda^{j}_{t_{j,k,s}}+\lambda I\right)^{-1}}\right\} + C_l(h^{\beta}+\eta^{2})\left[1+\sqrt{|\Psi^{j}_{T+1,s}|}\min\left\{1,\left\|\psi_{t_{j,k,j}}(w_{t_{j,k}})\right\|_{\left(\Lambda^{j}_{t_{j,k,s}}+\lambda I\right)^{-1}}\right\}\right]\right). \tag*{(46)} r t j , k , s j ( w t j , k ) ≤ max { C ψ , C z } ( ( D ι T + λ ) min { 1 , ψ t j , k , j ( w t j , k ) ( Λ t j , k , s j + λ I ) − 1 } + C l ( h β + η 2 ) [ 1 + ∣ Ψ T + 1 , s j ∣ min { 1 , ψ t j , k , j ( w t j , k ) ( Λ t j , k , s j + λ I ) − 1 } ] ) . (46)
Summing (46) over k ≥ 2 k\ge2 k ≥ 2 , adding the first radius, and applying (45) gives
∑ t ∈ Ψ T + 1 , s j r t , s j ( w t ) ≤ D + max { C ψ , C z } 2 max { 1 , C ψ 2 } ( D ι T + λ ) ∣ Ψ T + 1 , s j ∣ ι T + C l ( h β + η 2 ) [ ∣ Ψ T + 1 , s j ∣ + 2 max { 1 , C ψ 2 } ∣ Ψ T + 1 , s j ∣ ι T ] . \sum_{t\in\Psi^{j}_{T+1,s}} r^{j}_{t,s}(w_t) \le D+\max\{C_{\psi},C_z\}\sqrt{2\max\{1,C_{\psi}^{2}\}}(D\sqrt{\iota_T}+\sqrt{\lambda})\sqrt{|\Psi^{j}_{T+1,s}| \iota_T}
+C_l(h^{\beta}+\eta^{2})\left[|\Psi^{j}_{T+1,s}|+\sqrt{2\max\{1,C_{\psi}^{2}\}}|\Psi^{j}_{T+1,s}|\sqrt{\iota_T}\right]. t ∈ Ψ T + 1 , s j ∑ r t , s j ( w t ) ≤ D + max { C ψ , C z } 2 max { 1 , C ψ 2 } ( D ι T + λ ) ∣ Ψ T + 1 , s j ∣ ι T + C l ( h β + η 2 ) [ ∣ Ψ T + 1 , s j ∣ + 2 max { 1 , C ψ 2 } ∣ Ψ T + 1 , s j ∣ ι T ] .
Since ∣ Ψ T + 1 , s j ∣ ≥ 1 |\Psi^{j}_{T+1,s}|\ge1 ∣ Ψ T + 1 , s j ∣ ≥ 1 , ι T > 1 \iota_T>1 ι T > 1 , and D ι T + λ ≥ D D\sqrt{\iota_T}+\sqrt{\lambda}\ge D D ι T + λ ≥ D , the first and third terms can be absorbed, yielding
∑ t ∈ Ψ T + 1 , s j r t , s j ( w t ) ≤ [ 1 + max { C ψ , C z } 2 max { 1 , C ψ 2 } ] ( D ι T + λ ) ∣ Ψ T + 1 , s j ∣ ι T + C l [ 1 + 2 max { 1 , C ψ 2 } ] ( h β + η 2 ) ∣ Ψ T + 1 , s j ∣ ι T . (47) \sum_{t\in\Psi^{j}_{T+1,s}} r^{j}_{t,s}(w_t) \le\left[1+\max\{C_{\psi},C_z\}\sqrt{2\max\{1,C_{\psi}^{2}\}}\right](D\sqrt{\iota_T}+\sqrt{\lambda})\sqrt{|\Psi^{j}_{T+1,s}| \iota_T}
+C_l\left[1+\sqrt{2\max\{1,C_{\psi}^{2}\}}\right](h^{\beta}+\eta^{2})|\Psi^{j}_{T+1,s}|\sqrt{\iota_T}. \tag*{(47)} t ∈ Ψ T + 1 , s j ∑ r t , s j ( w t ) ≤ [ 1 + max { C ψ , C z } 2 max { 1 , C ψ 2 } ] ( D ι T + λ ) ∣ Ψ T + 1 , s j ∣ ι T + C l [ 1 + 2 max { 1 , C ψ 2 } ] ( h β + η 2 ) ∣ Ψ T + 1 , s j ∣ ι T . (47)
Step 4: Sum over bins and convert demand radii to revenue widths. By Cauchy–Schwarz,
∑ j = 1 N ∣ Ψ T + 1 , s j ∣ ≤ N ∑ j = 1 N ∣ Ψ T + 1 , s j ∣ = N ∣ Ψ T + 1 , s ∣ . \sum_{j=1}^{N}\sqrt{|\Psi^{j}_{T+1,s}|}\le\sqrt{N\sum_{j=1}^{N}|\Psi^{j}_{T+1,s}|}=\sqrt{N|\Psi_{T+1,s}|}. j = 1 ∑ N ∣ Ψ T + 1 , s j ∣ ≤ N j = 1 ∑ N ∣ Ψ T + 1 , s j ∣ = N ∣ Ψ T + 1 , s ∣ .
and, by the permanent partition,
∑ j = 1 N ∣ Ψ T + 1 , s j ∣ = ∣ Ψ T + 1 , s ∣ . \sum_{j=1}^{N}|\Psi^{j}_{T+1,s}|=|\Psi_{T+1,s}|. j = 1 ∑ N ∣ Ψ T + 1 , s j ∣ = ∣ Ψ T + 1 , s ∣.
By choosing
C ω = max { 1 + max { C ψ , C z } 2 max { 1 , C ψ 2 } , C l ( 1 + 2 max { 1 , C ψ 2 } ) } C_{\omega}=\max\left\{1+\max\{C_{\psi},C_z\}\sqrt{2\max\{1,C_{\psi}^{2}\}},\ C_l\left(1+\sqrt{2\max\{1,C_{\psi}^{2}\}}\right)\right\} C ω = max { 1 + max { C ψ , C z } 2 max { 1 , C ψ 2 } , C l ( 1 + 2 max { 1 , C ψ 2 } ) }
both coefficients in (47) are bounded by C ω C_\omega C ω . Summing over bins and using
wid t , s j t ( w t ) = p ^ t ( w t ) r t , s j t ( w t ) ≤ B r t , s j t ( w t ) , \operatorname{wid}_{t,s}^{jt}(w_t)=\widehat{p}_t(w_t)r_{t,s}^{jt}(w_t)\le Br_{t,s}^{jt}(w_t), wid t , s j t ( w t ) = p t ( w t ) r t , s j t ( w t ) ≤ B r t , s j t ( w t ) ,
therefore proves (42). □
Lemma A.3 (Occupancy of a nonterminal LDP layer) Let C ω C_\omega C ω be as in Lemma A.2. For every s < S s<S s < S , pathwise,
D 2 − s ∣ Ψ T + 1 , s ∣ ≤ C ω [ ( D ν T + λ ) N ∣ Ψ T + 1 , s ∣ ν T + ( h β + η 2 ) ∣ Ψ T + 1 , s ∣ ν T ] . (48) D2^{-s}|\Psi_{T+1,s}|\le C_\omega\left[(D\sqrt{\nu_T}+\sqrt{\lambda})\sqrt{N|\Psi_{T+1,s}|\nu_T}+(h^\beta+\eta^2)|\Psi_{T+1,s}|\sqrt{\nu_T}\right]. \tag*{(48)} D 2 − s ∣ Ψ T + 1 , s ∣ ≤ C ω [ ( D ν T + λ ) N ∣ Ψ T + 1 , s ∣ ν T + ( h β + η 2 ) ∣ Ψ T + 1 , s ∣ ν T ] . (48)
Moreover, if
D 2 − s ≥ 2 C ω ( h β + η 2 ) ν T , (49) D2^{-s}\ge2C_\omega(h^\beta+\eta^2)\sqrt{\nu_T}, \tag*{(49)} D 2 − s ≥ 2 C ω ( h β + η 2 ) ν T , (49)
then
∣ Ψ T + 1 , s ∣ ≤ 4 C ω 2 ( D ν T + λ ) 2 D 2 2 2 s N ν T . (50) |\Psi_{T+1,s}|\le4C_\omega^2\frac{(D\sqrt{\nu_T}+\sqrt{\lambda})^2}{D^2}2^{2s}N\nu_T. \tag*{(50)} ∣ Ψ T + 1 , s ∣ ≤ 4 C ω 2 D 2 ( D ν T + λ ) 2 2 2 s N ν T . (50)
Proof. Fix s < S s<S s < S . If Ψ T + 1 , s = ∅ \Psi_{T+1,s}=\varnothing Ψ T + 1 , s = ∅ , both conclusions are immediate. Suppose that Ψ T + 1 , s ≠ ∅ \Psi_{T+1,s}\ne\varnothing Ψ T + 1 , s = ∅ . Every round t ∈ Ψ T + 1 , s t\in\Psi_{T+1,s} t ∈ Ψ T + 1 , s is assigned to the nonterminal layer s s s . Under Algorithm 2, selection at such a layer can occur only through the under-explored branch. Therefore, wid t , s j t ( w t ) > B D 2 − s \operatorname{wid}_{t,s}^{jt}(w_t)>BD2^{-s} wid t , s j t ( w t ) > B D 2 − s for t ∈ Ψ T + 1 , s t\in\Psi_{T+1,s} t ∈ Ψ T + 1 , s . Summing this inequality over t ∈ Ψ T + 1 , s t\in\Psi_{T+1,s} t ∈ Ψ T + 1 , s and applying Lemma A.2 gives
B D 2 − s ∣ Ψ T + 1 , s ∣ < ∑ t ∈ Ψ T + 1 , s wid t , s j t ( w t ) ≤ C ω B [ ( D ν T + λ ) N ∣ Ψ T + 1 , s ∣ ν T + ( h β + η 2 ) ∣ Ψ T + 1 , s ∣ ν T ] . \begin{aligned}
BD2^{-s}|\Psi_{T+1,s}|<
\sum_{t\in\Psi_{T+1,s}}\operatorname{wid}_{t,s}^{jt}(w_t) \\
&\le C_\omega B\left[(D\sqrt{\nu_T}+\sqrt{\lambda})\sqrt{N|\Psi_{T+1,s}|\nu_T}+(h^\beta+\eta^2)|\Psi_{T+1,s}|\sqrt{\nu_T}\right].
\end{aligned} B D 2 − s ∣ Ψ T + 1 , s ∣ < t ∈ Ψ T + 1 , s ∑ wid t , s j t ( w t ) ≤ C ω B [ ( D ν T + λ ) N ∣ Ψ T + 1 , s ∣ ν T + ( h β + η 2 ) ∣ Ψ T + 1 , s ∣ ν T ] .
Since B > 0 B>0 B > 0 , dividing by B B B and weakening the resulting strict inequality proves (48). Suppose now that (49) holds. Then
C ω ( h β + η 2 ) ν T ≤ D 2 2 − s . C_\omega(h^\beta+\eta^2)\sqrt{\nu_T}\le\frac{D}{2}2^{-s}. C ω ( h β + η 2 ) ν T ≤ 2 D 2 − s .
It follows from (48) that
D 2 2 − s ∣ Ψ T + 1 , s ∣ ≤ [ D 2 − s − C ω ( h β + η 2 ) ν T ] ∣ Ψ T + 1 , s ∣ ≤ C ω ( D ν T + λ ) N ∣ Ψ T + 1 , s ∣ ν T . \frac{D}{2}2^{-s}|\Psi_{T+1,s}|\le[D2^{-s}-C_\omega(h^\beta+\eta^2)\sqrt{\nu_T}]|\Psi_{T+1,s}|
\le C_\omega(D\sqrt{\nu_T}+\sqrt{\lambda})\sqrt{N|\Psi_{T+1,s}|\nu_T}. 2 D 2 − s ∣ Ψ T + 1 , s ∣ ≤ [ D 2 − s − C ω ( h β + η 2 ) ν T ] ∣ Ψ T + 1 , s ∣ ≤ C ω ( D ν T + λ ) N ∣ Ψ T + 1 , s ∣ ν T .
Both sides are nonnegative. Squaring this inequality gives
D 2 4 2 − 2 s ∣ Ψ T + 1 , s ∣ 2 ≤ C ω 2 ( D ν T + λ ) 2 N ∣ Ψ T + 1 , s ∣ ν T . \frac{D^2}{4}2^{-2s}|\Psi_{T+1,s}|^2\le C_\omega^2(D\sqrt{\nu_T}+\sqrt{\lambda})^2N|\Psi_{T+1,s}|\nu_T. 4 D 2 2 − 2 s ∣ Ψ T + 1 , s ∣ 2 ≤ C ω 2 ( D ν T + λ ) 2 N ∣ Ψ T + 1 , s ∣ ν T .
Because ∣ Ψ T + 1 , s ∣ > 0 |\Psi_{T+1,s}|>0 ∣ Ψ T + 1 , s ∣ > 0 , canceling this factor and rearranging yields
∣ Ψ T + 1 , s ∣ ≤ 4 C ω 2 ( D ν T + λ ) 2 D 2 2 2 s N ν T , |\Psi_{T+1,s}|\le4C_\omega^2\frac{(D\sqrt{\nu_T}+\sqrt{\lambda})^2}{D^2}2^{2s}N\nu_T, ∣ Ψ T + 1 , s ∣ ≤ 4 C ω 2 D 2 ( D ν T + λ ) 2 2 2 s N ν T ,
which proves (50). □
Proof of Theorem 5.2.
Proof. We proceed in five steps. Throughout the proof, C C C denotes a finite constant depending only on the fixed problem primitives and on C ψ , C z , C l C_\psi,C_z,C_l C ψ , C z , C l ; its value may increase from line to line.
Step 1: Reduction to a dyadic layer sum. Note that, the sets Ψ T + 1 , 1 , … , Ψ T + 1 , S \Psi_{T+1,1},\ldots,\Psi_{T+1,S} Ψ T + 1 , 1 , … , Ψ T + 1 , S are disjoint and contain all LDP rounds. Therefore,
∑ s = 1 S ∣ Ψ T + 1 , s ∣ ≤ T . (51) \sum_{s=1}^{S} \lvert\Psi_{T+1,s}\rvert\le T. \tag*{(51)} s = 1 ∑ S ∣ Ψ T + 1 , s ∣ ≤ T . (51)
By Lemma 5.1, every LDP round t t t satisfies
Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ≤ L Rev T + 8 B D 2 − s t . \operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)\le\frac{L_{\operatorname{Rev}}}{\sqrt{T}}+8BD\,2^{-s_t}. Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ≤ T L Rev + 8 B D 2 − s t .
Summing over the LDP rounds and using their layer partition gives
∑ t ∈ [ T ] ∖ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ L Rev T + 8 B D ∑ s = 1 S 2 − s ∣ Ψ T + 1 , s ∣ . (52) \sum_{t\in[T]\setminus\mathcal{T}^{\exp}}\left[\operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)\right]\le L_{\operatorname{Rev}}\sqrt{T}+8BD\sum_{s=1}^{S}2^{-s}\lvert\Psi_{T+1,s}\rvert. \tag*{(52)} t ∈ [ T ] ∖ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ L Rev T + 8 B D s = 1 ∑ S 2 − s ∣ Ψ T + 1 , s ∣ . (52)
Step 2: Nonterminal layers dominated by statistical uncertainty. For every nonterminal layer s < S s<S s < S satisfying (49), Lemma A.3 gives
∣ Ψ T + 1 , s ∣ ≤ 4 C ω 2 ( D L T + λ ) 2 D 2 2 2 s N ι T . \lvert\Psi_{T+1,s}\rvert\le4C_\omega^2\frac{\left(D\sqrt{L_T}+\sqrt{\lambda}\right)^2}{D^2}2^{2s}N\iota_T. ∣ Ψ T + 1 , s ∣ ≤ 4 C ω 2 D 2 ( D L T + λ ) 2 2 2 s N ι T .
The deterministic bound ∣ Ψ T + 1 , s ∣ ≤ T \lvert\Psi_{T+1,s}\rvert\le T ∣ Ψ T + 1 , s ∣ ≤ T also holds. Note that for every a > 0 a>0 a > 0 , we have
∑ s ≥ 1 2 − s min { T , a 2 2 s } ≤ 4 a T , (53) \sum_{s\ge1}2^{-s}\min\{T,a2^{2s}\}\le4\sqrt{aT}, \tag*{(53)} s ≥ 1 ∑ 2 − s min { T , a 2 2 s } ≤ 4 a T , (53)
because if a ≥ T a\ge T a ≥ T , the left-hand side is at most T ∑ s ≥ 1 2 − s ≤ T ≤ a T T\sum_{s\ge1}2^{-s}\le T\le\sqrt{aT} T ∑ s ≥ 1 2 − s ≤ T ≤ a T , while if a < T a<T a < T , choose an integer s 0 ≥ 0 s_0\ge0 s 0 ≥ 0 such that 2 s 0 ≤ T / a < 2 s 0 + 1 2^{s_0}\le\sqrt{T/a}<2^{s_0+1} 2 s 0 ≤ T / a < 2 s 0 + 1 , then
∑ s ≥ 1 2 − s min { T , a 2 2 s } ≤ a ∑ s ≤ s 0 2 s + T ∑ s > s 0 2 − s ≤ 2 a 2 s 0 + T 2 − s 0 ≤ 4 a T . \sum_{s\ge1}2^{-s}\min\{T,a2^{2s}\}\le a\sum_{s\le s_0}2^s+T\sum_{s>s_0}2^{-s}\le2a2^{s_0}+T2^{-s_0}\le4\sqrt{aT}. s ≥ 1 ∑ 2 − s min { T , a 2 2 s } ≤ a s ≤ s 0 ∑ 2 s + T s > s 0 ∑ 2 − s ≤ 2 a 2 s 0 + T 2 − s 0 ≤ 4 a T .
Applying (53) with
a = 4 C ω 2 ( D L T + λ ) 2 D 2 N ι T a=4C_\omega^2\frac{\left(D\sqrt{L_T}+\sqrt{\lambda}\right)^2}{D^2}N\iota_T a = 4 C ω 2 D 2 ( D L T + λ ) 2 N ι T
yields
∑ s < S : D 2 − s ≥ 2 C ω ( h β + η 2 ) ι T 2 − s ∣ Ψ T + 1 , s ∣ ≤ 4 a T = 8 C ω D L T + λ D N T ι T . (54) \sum_{\substack{s<S:\\D2^{-s}\ge2C_\omega(h^\beta+\eta^2)\sqrt{\iota_T}}}2^{-s}\lvert\Psi_{T+1,s}\rvert\le4\sqrt{aT}=8C_\omega\frac{D\sqrt{L_T}+\sqrt{\lambda}}{D}\sqrt{NT\iota_T}. \tag*{(54)} s < S : D 2 − s ≥ 2 C ω ( h β + η 2 ) ι T ∑ 2 − s ∣ Ψ T + 1 , s ∣ ≤ 4 a T = 8 C ω D D L T + λ N T ι T . (54)
Step 3: Nonterminal layers dominated by approximation bias. For every remaining nonterminal layer,
D 2 − s < 2 C ω ( h β + η 2 ) ι T , D2^{-s}<2C_\omega(h^\beta+\eta^2)\sqrt{\iota_T}, D 2 − s < 2 C ω ( h β + η 2 ) ι T ,
and hence
2 − s < 2 C ω ( h β + η 2 ) ι T D . 2^{-s}<\frac{2C_\omega(h^\beta+\eta^2)\sqrt{\iota_T}}{D}. 2 − s < D 2 C ω ( h β + η 2 ) ι T .
Consequently,
∑ s < S : D 2 − s < 2 C ω ( h β + η 2 ) ι T 2 − s ∣ Ψ T + 1 , s ∣ ≤ 2 C ω ( h β + η 2 ) ι T D ∑ s < S ∣ Ψ T + 1 , s ∣ ≤ 2 C ω ( h β + η 2 ) ι T D T . (55) \sum_{\substack{s<S:\\D2^{-s}<2C_\omega(h^\beta+\eta^2)\sqrt{\iota_T}}}2^{-s}\lvert\Psi_{T+1,s}\rvert\le\frac{2C_\omega(h^\beta+\eta^2)\sqrt{\iota_T}}{D}\sum_{s<S}\lvert\Psi_{T+1,s}\rvert\le\frac{2C_\omega(h^\beta+\eta^2)\sqrt{\iota_T}}{D}T. \tag*{(55)} s < S : D 2 − s < 2 C ω ( h β + η 2 ) ι T ∑ 2 − s ∣ Ψ T + 1 , s ∣ ≤ D 2 C ω ( h β + η 2 ) ι T s < S ∑ ∣ Ψ T + 1 , s ∣ ≤ D 2 C ω ( h β + η 2 ) ι T T . (55)
Step 4: Terminal layer and completion of the explicit bound. For the terminal layer,
2 − S ∣ Ψ T + 1 , S ∣ ≤ T 2 − S . (56) 2^{-S}|\Psi_{T+1,S}| \le T2^{-S}. \tag*{(56)} 2 − S ∣ Ψ T + 1 , S ∣ ≤ T 2 − S . (56)
Combining (54), (55), and (56), we obtain
∑ s = 1 S 2 − s ∣ Ψ T + 1 , s ∣ ≤ 8 C ω D τ T + λ D N T ι T + 2 C ω ( h β + η 2 ) ι T D T + T 2 − S . \sum_{s=1}^{S}2^{-s}|\Psi_{T+1,s}| \le8C_{\omega}\frac{D\sqrt{\tau_T}+\sqrt{\lambda}}{D}\sqrt{N T\iota_T}+\frac{2C_{\omega}(h^{\beta}+\eta^2)\sqrt{\iota_T}}{D}T+T2^{-S}. s = 1 ∑ S 2 − s ∣ Ψ T + 1 , s ∣ ≤ 8 C ω D D τ T + λ N T ι T + D 2 C ω ( h β + η 2 ) ι T T + T 2 − S .
Substituting this inequality into (52) gives
∑ t ∈ [ T ] ∖ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ 64 C ω B ( D τ T + λ ) N T ι T + 16 C ω B T ( h β + η 2 ) ι T + 8 B D T 2 − S + L Rev T . \sum_{t\in[T]\setminus\mathcal{T}^{\exp}}\left[\operatorname{Rev}(x_t,p_t^{\star})-\operatorname{Rev}(x_t,p_t)\right]
\le64C_{\omega}B(D\sqrt{\tau_T}+\sqrt{\lambda})\sqrt{N T\iota_T}+16C_{\omega}BT(h^{\beta}+\eta^2)\sqrt{\iota_T}+8BDT2^{-S}+L_{\operatorname{Rev}}\sqrt{T}. t ∈ [ T ] ∖ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ 64 C ω B ( D τ T + λ ) N T ι T + 16 C ω B T ( h β + η 2 ) ι T + 8 B D T 2 − S + L Rev T .
For S = max { 1 , ⌈ log 2 T ⌉ } S=\max\left\{1,\left\lceil\log_2\sqrt{T}\right\rceil\right\} S = max { 1 , ⌈ log 2 T ⌉ } , we have T 2 − S ≤ T T2^{-S}\le\sqrt{T} T 2 − S ≤ T . Therefore, (11) follows.
Step 5: Simplified rate under algorithmic tuning. Let λ > 0 \lambda>0 λ > 0 be fixed independently of T T T , let η 2 ≤ c η h β \eta^2\le c_{\eta}h^{\beta} η 2 ≤ c η h β , and let N = ⌈ T 1 2 β + 1 ⌉ N=\left\lceil T^{\frac{1}{2\beta+1}}\right\rceil N = ⌈ T 2 β + 1 1 ⌉ . Since T ≥ 1 T\ge1 T ≥ 1 , we have T 1 2 β + 1 ≤ N ≤ 2 T 1 2 β + 1 T^{\frac{1}{2\beta+1}}\le N\le2T^{\frac{1}{2\beta+1}} T 2 β + 1 1 ≤ N ≤ 2 T 2 β + 1 1 . It follows that
N T ≤ 2 T β + 1 2 β + 1 , \sqrt{NT}\le\sqrt{2}T^{\frac{\beta+1}{2\beta+1}}, N T ≤ 2 T 2 β + 1 β + 1 ,
and, because h = 2 B / N h=2B/N h = 2 B / N ,
T ( h β + η 2 ) ≤ ( 1 + c η ) T h β = ( 1 + c η ) ( 2 B ) β T N − β ≤ ( 1 + c η ) ( 2 B ) β T β + 1 2 β + 1 . T(h^{\beta}+\eta^2)\le(1+c_{\eta})Th^{\beta}=(1+c_{\eta})(2B)^{\beta}TN^{-\beta}\le(1+c_{\eta})(2B)^{\beta}T^{\frac{\beta+1}{2\beta+1}}. T ( h β + η 2 ) ≤ ( 1 + c η ) T h β = ( 1 + c η ) ( 2 B ) β T N − β ≤ ( 1 + c η ) ( 2 B ) β T 2 β + 1 β + 1 .
Moreover, T ≤ T β + 1 2 β + 1 \sqrt{T}\le T^{\frac{\beta+1}{2\beta+1}} T ≤ T 2 β + 1 β + 1 for every finite β ≥ 1 \beta\ge1 β ≥ 1 . By the definition of ι T \iota_T ι T , we have ι T > 1 \iota_T>1 ι T > 1 . Since λ > 0 \lambda>0 λ > 0 is fixed independently of T T T ,
( D τ T + λ ) ι T = D ι T + λ ι T ≤ ( D + λ ) ι T , (D\sqrt{\tau_T}+\sqrt{\lambda})\sqrt{\iota_T}=D\iota_T+\sqrt{\lambda\iota_T}\le(D+\sqrt{\lambda})\iota_T, ( D τ T + λ ) ι T = D ι T + λ ι T ≤ ( D + λ ) ι T ,
and ι T ≤ ι T \sqrt{\iota_T}\le\iota_T ι T ≤ ι T .
Applying these inequalities to (11) gives
∑ t ∈ [ T ] ∖ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ [ 64 2 C ω B ( D + λ ) + 16 C ω B ( 1 + c η ) ( 2 B ) β + 8 B D + L Rev ] ι T T β + 1 2 β + 1 . \sum_{t\in[T]\setminus\mathcal{T}^{\exp}}\left[\operatorname{Rev}(x_t,p_t^{\star})-\operatorname{Rev}(x_t,p_t)\right]
\le\left[64\sqrt{2}C_{\omega}B(D+\sqrt{\lambda})+16C_{\omega}B(1+c_{\eta})(2B)^{\beta}+8BD+L_{\operatorname{Rev}}\right]\iota_T T^{\frac{\beta+1}{2\beta+1}}. t ∈ [ T ] ∖ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ [ 64 2 C ω B ( D + λ ) + 16 C ω B ( 1 + c η ) ( 2 B ) β + 8 B D + L Rev ] ι T T 2 β + 1 β + 1 .
One may take
C L D P : = 64 2 C ω B ( D + λ ) + 16 C ω B ( 1 + c η ) ( 2 B ) β + 8 B D + L Rev . C_{\mathrm{LDP}}:=64\sqrt{2}C_{\omega}B(D+\sqrt{\lambda})+16C_{\omega}B(1+c_{\eta})(2B)^{\beta}+8BD+L_{\operatorname{Rev}}. C LDP := 64 2 C ω B ( D + λ ) + 16 C ω B ( 1 + c η ) ( 2 B ) β + 8 B D + L Rev .
This proves (12). □
Proof of Theorem 5.3.
Proof. Fix an arbitrary realization of the policy. If T exp = ∅ \mathcal{T}^{\exp}=\varnothing T e x p = ∅ , all conclusions are immediate. Otherwise, enumerate the pilot-exploration rounds chronologically as
T exp = { τ 1 < ⋯ < τ ∣ T exp ∣ } . \mathcal{T}^{\exp}=\{\tau_1<\cdots<\tau_{|\mathcal{T}^{\exp}|}\}. T e x p = { τ 1 < ⋯ < τ ∣ T e x p ∣ } .
Step 1: Leverage on exploration rounds. For every k = 1 , … , ∣ T e x p ∣ k = 1, \ldots, |\mathcal{T}^{\mathrm{exp}}| k = 1 , … , ∣ T exp ∣ , period τ k \tau_k τ k is a pilot-exploration round. Therefore, by the uncertainty gate in Algorithm 1, γ T ∥ x τ k ∥ M τ k − 1 > η \gamma_T \lVert x_{\tau_k}\rVert_{M_{\tau_k}^{-1}} > \eta γ T ∥ x τ k ∥ M τ k − 1 > η . Since γ T > 0 \gamma_T > 0 γ T > 0 , squaring both sides gives
∥ x τ k ∥ M τ k − 1 2 > η 2 γ T 2 . (57) \lVert x_{\tau_k}\rVert_{M_{\tau_k}^{-1}}^2 > \frac{\eta^2}{\gamma_T^2}. \tag*{(57)} ∥ x τ k ∥ M τ k − 1 2 > γ T 2 η 2 . (57)
Step 2: Exact determinant recursion. The matrix M t M_t M t is updated only on pilot-exploration rounds. In particular, M τ 1 = M 1 = I d M_{\tau_1} = M_1 = I_d M τ 1 = M 1 = I d , and, for every k = 1 , … , ∣ T e x p ∣ − 1 k = 1, \ldots, |\mathcal{T}^{\mathrm{exp}}| - 1 k = 1 , … , ∣ T exp ∣ − 1 ,
M τ k + 1 = M τ k + x τ k x τ k T , M_{\tau_{k+1}} = M_{\tau_k} + x_{\tau_k}x_{\tau_k}^{\mathsf{T}}, M τ k + 1 = M τ k + x τ k x τ k T ,
because M t M_t M t remains unchanged on all LDP rounds strictly between τ k \tau_k τ k and τ k + 1 \tau_{k+1} τ k + 1 . Similarly,
M T + 1 = M τ ∣ T e x p ∣ + x τ ∣ T e x p ∣ x τ ∣ T e x p ∣ T . M_{T+1} = M_{\tau_{|\mathcal{T}^{\mathrm{exp}}|}} + x_{\tau_{|\mathcal{T}^{\mathrm{exp}}|}}x_{\tau_{|\mathcal{T}^{\mathrm{exp}}|}}^{\mathsf{T}}. M T + 1 = M τ ∣ T exp ∣ + x τ ∣ T exp ∣ x τ ∣ T exp ∣ T .
Since M τ k ⪰ I d M_{\tau_k} \succeq I_d M τ k ⪰ I d , it is positive definite. The matrix determinant lemma therefore gives
det ( M τ k + x τ k x τ k T ) = det ( M τ k ) ( 1 + x τ k T M τ k − 1 x τ k ) = det ( M τ k ) ( 1 + ∥ x τ k ∥ M τ k − 1 2 ) , \det\left(M_{\tau_k} + x_{\tau_k}x_{\tau_k}^{\mathsf{T}}\right) = \det(M_{\tau_k})\left(1 + x_{\tau_k}^{\mathsf{T}}M_{\tau_k}^{-1}x_{\tau_k}\right) = \det(M_{\tau_k})\left(1 + \lVert x_{\tau_k}\rVert_{M_{\tau_k}^{-1}}^2\right), det ( M τ k + x τ k x τ k T ) = det ( M τ k ) ( 1 + x τ k T M τ k − 1 x τ k ) = det ( M τ k ) ( 1 + ∥ x τ k ∥ M τ k − 1 2 ) ,
Iterating this identity and using det ( M τ 1 ) = det ( I d ) = 1 \det(M_{\tau_1}) = \det(I_d) = 1 det ( M τ 1 ) = det ( I d ) = 1 yields
log det ( M T + 1 ) = ∑ k = 1 ∣ T e x p ∣ log ( 1 + ∥ x τ k ∥ M τ k − 1 2 ) . (58) \log\det(M_{T+1}) = \sum_{k=1}^{|\mathcal{T}^{\mathrm{exp}}|} \log\left(1 + \lVert x_{\tau_k}\rVert_{M_{\tau_k}^{-1}}^2\right). \tag*{(58)} log det ( M T + 1 ) = k = 1 ∑ ∣ T exp ∣ log ( 1 + ∥ x τ k ∥ M τ k − 1 2 ) . (58)
Step 3: Lower bound on the determinant growth. By (57), every summand in (58) satisfies
log ( 1 + ∥ x τ k ∥ M τ k − 1 2 ) > log ( 1 + η 2 γ T 2 ) . \log\left(1 + \lVert x_{\tau_k}\rVert_{M_{\tau_k}^{-1}}^2\right) > \log\left(1 + \frac{\eta^2}{\gamma_T^2}\right). log ( 1 + ∥ x τ k ∥ M τ k − 1 2 ) > log ( 1 + γ T 2 η 2 ) .
Summing over k k k gives,
∣ T e x p ∣ log ( 1 + η 2 γ T 2 ) < log det ( M T + 1 ) . (59) |\mathcal{T}^{\mathrm{exp}}| \log\left(1 + \frac{\eta^2}{\gamma_T^2}\right) < \log\det(M_{T+1}). \tag*{(59)} ∣ T exp ∣ log ( 1 + γ T 2 η 2 ) < log det ( M T + 1 ) . (59)
Step 4: Upper bound on the determinant. By the update rule,
M T + 1 = I d + ∑ t ∈ T e x p x t x t T . M_{T+1} = I_d + \sum_{t\in\mathcal{T}^{\mathrm{exp}}} x_t x_t^{\mathsf{T}}. M T + 1 = I d + t ∈ T exp ∑ x t x t T .
Therefore, we have
tr ( M T + 1 ) = d + ∑ t ∈ T e x p ∥ x t ∥ 2 2 ≤ d + T C x 2 . \operatorname{tr}(M_{T+1}) = d + \sum_{t\in\mathcal{T}^{\mathrm{exp}}} \lVert x_t\rVert_2^2 \le d + T C_x^2. tr ( M T + 1 ) = d + t ∈ T exp ∑ ∥ x t ∥ 2 2 ≤ d + T C x 2 .
Since M T + 1 M_{T+1} M T + 1 is positive definite, by the arithmetic–geometric mean inequality applied to the eigenvalues of M T + 1 M_{T+1} M T + 1 ,
det ( M T + 1 ) 1 / d ≤ tr ( M T + 1 ) d . \det(M_{T+1})^{1/d} \le\frac{\operatorname{tr}(M_{T+1})}{d}. det ( M T + 1 ) 1/ d ≤ d tr ( M T + 1 ) .
Consequently,
log det ( M T + 1 ) ≤ d log ( tr ( M T + 1 ) d ) ≤ d log ( 1 + T C x 2 d ) . (60) \log\det(M_{T+1}) \le d\log\left(\frac{\operatorname{tr}(M_{T+1})}{d}\right) \le d\log\left(1 + \frac{TC_x^2}{d}\right). \tag*{(60)} log det ( M T + 1 ) ≤ d log ( d tr ( M T + 1 ) ) ≤ d log ( 1 + d T C x 2 ) . (60)
Combining (59) and (60) yields
∣ T exp ∣ < d log ( 1 + T C χ 2 / d ) log ( 1 + η 2 / γ T 2 ) . \lvert\mathcal{T}^{\exp} \rvert< \frac{d\log\left(1+TC_{\chi}^{2}/d\right)}{\log\left(1+\eta^{2}/\gamma_{T}^{2}\right)}. ∣ T e x p ∣ < log ( 1 + η 2 / γ T 2 ) d log ( 1 + T C χ 2 / d ) .
Together with the trivial bound ∣ T exp ∣ ≤ T \lvert\mathcal{T}^{\exp} \rvert\le T ∣ T e x p ∣ ≤ T , this proves the first inequality in (13) .
For every x ≥ 0 x \ge0 x ≥ 0 , log ( 1 + x ) ≥ x / ( 1 + x ) \log(1+x) \ge x/(1+x) log ( 1 + x ) ≥ x / ( 1 + x ) . Applying this inequality with x = η 2 / γ T 2 x=\eta^{2}/\gamma_{T}^{2} x = η 2 / γ T 2 gives
1 log ( 1 + η 2 / γ T 2 ) ≤ 1 + γ T 2 η 2 . \frac{1}{\log\left(1+\eta^{2}/\gamma_{T}^{2}\right)} \le1+\frac{\gamma_{T}^{2}}{\eta^{2}}. log ( 1 + η 2 / γ T 2 ) 1 ≤ 1 + η 2 γ T 2 .
This proves the second inequality in (13) .
Step 5: Pilot-exploration regret. Since 0 ≤ g ( u ) ≤ D 0 \le g(u) \le D 0 ≤ g ( u ) ≤ D and 0 ≤ p ≤ B 0 \le p \le B 0 ≤ p ≤ B , 0 ≤ Rev ( x , p ) = p g ( p − x ⊤ θ ⋆ ) ≤ B D 0 \le\operatorname{Rev}(x,p)=p g(p-x^{\top}\theta_{\star}) \le BD 0 ≤ Rev ( x , p ) = p g ( p − x ⊤ θ ⋆ ) ≤ B D . Therefore, for every t ∈ T exp t \in\mathcal{T}^{\exp} t ∈ T e x p ,
0 ≤ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ≤ Rev ( x t , p t ⋆ ) ≤ B D . 0 \le\operatorname{Rev}(x_t,p_t^{\star})-\operatorname{Rev}(x_t,p_t) \le\operatorname{Rev}(x_t,p_t^{\star}) \le BD. 0 ≤ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ≤ Rev ( x t , p t ⋆ ) ≤ B D .
Summing over t ∈ T exp t \in\mathcal{T}^{\exp} t ∈ T e x p and applying (13) proves (14) .
Proof of Theorem 5.4.
Proof. We work throughout on the uniform confidence event in Proposition 4.2, which occurs with probability at least 1 − δ 1-\delta 1 − δ .
Step 1: Decomposition of the regret. The pilot-exploration rounds and the LDP rounds form a disjoint partition of the horizon. Therefore,
Reg ( T ) = ∑ t ∈ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] + ∑ t ∈ [ T ] ∖ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] . (61) \operatorname{Reg}(T)=\sum_{t\in\mathcal{T}^{\exp}}\left[\operatorname{Rev}(x_t,p_t^{\star})-\operatorname{Rev}(x_t,p_t)\right]+\sum_{t\in[T]\setminus\mathcal{T}^{\exp}}\left[\operatorname{Rev}(x_t,p_t^{\star})-\operatorname{Rev}(x_t,p_t)\right]. \tag*{(61)} Reg ( T ) = t ∈ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] + t ∈ [ T ] ∖ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] . (61)
Step 2: Finite-sample bound. Lemma 5.3 gives the pathwise
∑ t ∈ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ B D min { T , d log ( 1 + T C χ 2 / d ) log ( 1 + η 2 / γ T 2 ) } . (62) \sum_{t\in\mathcal{T}^{\exp}}\left[\operatorname{Rev}(x_t,p_t^{\star})-\operatorname{Rev}(x_t,p_t)\right]\le BD\min\left\{T,\frac{d\log\left(1+TC_{\chi}^{2}/d\right)}{\log\left(1+\eta^{2}/\gamma_{T}^{2}\right)}\right\}. \tag*{(62)} t ∈ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ B D min { T , log ( 1 + η 2 / γ T 2 ) d log ( 1 + T C χ 2 / d ) } . (62)
Proposition 5.2 and the choice S = max { 1 , ⌈ log 2 T ⌉ } S=\max\left\{1,\left\lceil\log_{2}\sqrt{T}\right\rceil\right\} S = max { 1 , ⌈ log 2 T ⌉ } give
∑ t ∈ [ T ] ∖ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ 64 C ω B ( D ι T + λ ) N T ι T + 16 C ω B T ( h β + η 2 ) ι T + ( 8 B D + L Rev ) T . (63) \begin{aligned}
\sum_{t\in[T]\setminus\mathcal{T}^{\exp}}\left[\operatorname{Rev}(x_t,p_t^{\star})-\operatorname{Rev}(x_t,p_t)\right]
&\le64C_{\omega}B(D\sqrt{\iota T}+\sqrt{\lambda})\sqrt{NT\iota_{T}} \\
&\quad+16C_{\omega}BT(h^{\beta}+\eta^{2})\sqrt{\iota_{T}}+(8BD+L_{\operatorname{Rev}})\sqrt{T}. \tag*{(63)}
\end{aligned} t ∈ [ T ] ∖ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ 64 C ω B ( D ι T + λ ) N T ι T + 16 C ω B T ( h β + η 2 ) ι T + ( 8 B D + L Rev ) T . (63)
Substituting (62) and (63) into (61) proves (15) .
Step 3: LDP regret under the prescribed tuning. Let N = ⌈ T 1 2 β + 1 ⌉ N=\left\lceil T^{\frac{1}{2\beta+1}}\right\rceil N = ⌈ T 2 β + 1 1 ⌉ and η 2 = min { h 2 , h β } \eta^{2}=\min\{h^{2},h^{\beta}\} η 2 = min { h 2 , h β } . Because η 2 ≤ h 2 \eta^{2}\le h^{2} η 2 ≤ h 2 , we have η ≤ h \eta\le h η ≤ h . Moreover, since η 2 ≤ h β \eta^{2}\le h^{\beta} η 2 ≤ h β , we have h β + η 2 ≤ 2 h β h^{\beta}+\eta^{2}\le2h^{\beta} h β + η 2 ≤ 2 h β . Thus the prescribed tuning satisfies the conditions used in Proposition 4.2 and Proposition 5.2.
Since T ≥ 1 T\ge1 T ≥ 1 , we have
T 1 2 β + 1 ≤ N ≤ T 1 2 β + 1 + 1 ≤ 2 T 1 2 β + 1 . T^{\frac{1}{2\beta+1}}\le N\le T^{\frac{1}{2\beta+1}}+1\le2T^{\frac{1}{2\beta+1}}. T 2 β + 1 1 ≤ N ≤ T 2 β + 1 1 + 1 ≤ 2 T 2 β + 1 1 .
It follows that
N T ≤ 2 T 1 2 + 1 2 ( 2 β + 1 ) = 2 T β + 1 2 β + 1 . (64) \sqrt{NT} \le\sqrt{2}T^{\frac{1}{2}+\frac{1}{2(2\beta+1)}}=\sqrt{2}T^{\frac{\beta+1}{2\beta+1}}. \tag*{(64)} N T ≤ 2 T 2 1 + 2 ( 2 β + 1 ) 1 = 2 T 2 β + 1 β + 1 . (64)
Since N ≥ T 1 / ( 2 β + 1 ) N \ge T^{1/(2\beta+1)} N ≥ T 1/ ( 2 β + 1 ) and h = 2 B / N h=2B/N h = 2 B / N ,
T h β = T ( 2 B N ) β ≤ ( 2 B ) β T 1 − β 2 β + 1 = ( 2 B ) β T β + 1 2 β + 1 . (65) Th^\beta= T\left(\frac{2B}{N}\right)^\beta\le(2B)^\beta T^{1-\frac{\beta}{2\beta+1}}=(2B)^\beta T^{\frac{\beta+1}{2\beta+1}}. \tag*{(65)} T h β = T ( N 2 B ) β ≤ ( 2 B ) β T 1 − 2 β + 1 β = ( 2 B ) β T 2 β + 1 β + 1 . (65)
Furthermore, since ( β + 1 ) / ( 2 β + 1 ) > 1 / 2 (\beta+1)/(2\beta+1)>1/2 ( β + 1 ) / ( 2 β + 1 ) > 1/2 , we have T ≤ T β + 1 2 β + 1 \sqrt{T}\le T^{\frac{\beta+1}{2\beta+1}} T ≤ T 2 β + 1 β + 1 . By the definition of ι T \iota_T ι T , ι T ≥ 2 log 4 > 1 \iota_T\ge2\log4>1 ι T ≥ 2 log 4 > 1 . Therefore, ι T ≤ ι T \sqrt{\iota_T}\le\iota_T ι T ≤ ι T . In addition,
( D ι T + λ ) N T ι T = ( D ι T + λ ι T ) N T ≤ ( D + 1 ) ( 1 + λ ) ι T N T . (66) (D\sqrt{\iota_T}+\sqrt{\lambda})\sqrt{N T\iota_T}=(D\iota_T+\sqrt{\lambda\iota_T})\sqrt{NT}\le(D+1)(1+\sqrt{\lambda})\iota_T\sqrt{NT}. \tag*{(66)} ( D ι T + λ ) N T ι T = ( D ι T + λ ι T ) N T ≤ ( D + 1 ) ( 1 + λ ) ι T N T . (66)
Combining (66) with (64) gives
64 C ω B ( D ι T + λ ) N T ι T ≤ 64 2 C ω B ( D + 1 ) ( 1 + λ ) ι T T β + 1 2 β + 1 . 64C_\omega B(D\sqrt{\iota_T}+\sqrt{\lambda})\sqrt{N T\iota_T}\le64\sqrt{2}C_\omega B(D+1)(1+\sqrt{\lambda})\iota_T T^{\frac{\beta+1}{2\beta+1}}. 64 C ω B ( D ι T + λ ) N T ι T ≤ 64 2 C ω B ( D + 1 ) ( 1 + λ ) ι T T 2 β + 1 β + 1 .
Next, by h β + η 2 ≤ 2 h β h^\beta+\eta^2\le2h^\beta h β + η 2 ≤ 2 h β , (65), and ι T ≤ ι T \sqrt{\iota_T}\le\iota_T ι T ≤ ι T ,
16 C ω B T ( h β + η 2 ) ι T ≤ 32 C ω B T h β ι T ≤ 32 C ω B ( 2 B ) β ( 1 + λ ) ι T T β + 1 2 β + 1 . 16C_\omega BT(h^\beta+\eta^2)\sqrt{\iota_T}\le32C_\omega BTh^\beta\iota_T\le32C_\omega B(2B)^\beta(1+\sqrt{\lambda})\iota_T T^{\frac{\beta+1}{2\beta+1}}. 16 C ω B T ( h β + η 2 ) ι T ≤ 32 C ω B T h β ι T ≤ 32 C ω B ( 2 B ) β ( 1 + λ ) ι T T 2 β + 1 β + 1 .
Finally, by T ≤ T β + 1 2 β + 1 \sqrt{T}\le T^{\frac{\beta+1}{2\beta+1}} T ≤ T 2 β + 1 β + 1 and ( 1 + λ ) ι T ≥ 1 (1+\sqrt{\lambda})\iota_T\ge1 ( 1 + λ ) ι T ≥ 1 ,
( 8 B D + L R e v ) T ≤ ( 8 B D + L R e v ) ( 1 + λ ) ι T T β + 1 2 β + 1 . (8BD+L_{\mathrm{Rev}})\sqrt{T}\le(8BD+L_{\mathrm{Rev}})(1+\sqrt{\lambda})\iota_T T^{\frac{\beta+1}{2\beta+1}}. ( 8 B D + L Rev ) T ≤ ( 8 B D + L Rev ) ( 1 + λ ) ι T T 2 β + 1 β + 1 .
Consequently, we have
∑ t ∈ [ T ] ∖ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ [ 64 2 C ω B ( D + 1 ) + 32 C ω B ( 2 B ) β + 8 B D + L R e v ] × ( 1 + λ ) ι T T β + 1 2 β + 1 . (67) \begin{aligned}
\sum_{t\in[T]\setminus\mathcal{T}^{\exp}}[\operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)]
&\le\left[64\sqrt{2}C_\omega B(D+1)+32C_\omega B(2B)^\beta+8BD+L_{\mathrm{Rev}}\right]\\
&\quad{}\times(1+\sqrt{\lambda})\iota_T T^{\frac{\beta+1}{2\beta+1}}. \tag*{(67)}
\end{aligned} t ∈ [ T ] ∖ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t )] ≤ [ 64 2 C ω B ( D + 1 ) + 32 C ω B ( 2 B ) β + 8 B D + L Rev ] × ( 1 + λ ) ι T T 2 β + 1 β + 1 . (67)
Step 4: Pilot-exploration regret under the prescribed tuning. For every x ≥ 0 x\ge0 x ≥ 0 , log ( 1 + x ) ≥ x / ( 1 + x ) \log(1+x)\ge x/(1+x) log ( 1 + x ) ≥ x / ( 1 + x ) . Applying this inequality with x = η 2 / γ T 2 x=\eta^2/\gamma_T^2 x = η 2 / γ T 2 gives
1 log ( 1 + η 2 / γ T 2 ) ≤ 1 + γ T 2 η 2 . \frac{1}{\log(1+\eta^2/\gamma_T^2)}\le1+\frac{\gamma_T^2}{\eta^2}. log ( 1 + η 2 / γ T 2 ) 1 ≤ 1 + η 2 γ T 2 .
Thus, by (62),
∑ t ∈ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ B D d ( 1 + γ T 2 η 2 ) log ( 1 + T C x 2 d ) . (68) \sum_{t\in\mathcal{T}^{\exp}}[\operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)]\le BDd\left(1+\frac{\gamma_T^2}{\eta^2}\right)\log\left(1+\frac{TC_x^2}{d}\right). \tag*{(68)} t ∈ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t )] ≤ B D d ( 1 + η 2 γ T 2 ) log ( 1 + d T C x 2 ) . (68)
Since N ≤ 2 T 1 2 β + 1 N\le2T^{\frac{1}{2\beta+1}} N ≤ 2 T 2 β + 1 1 , we have h = 2 B / N ≥ B T − 1 2 β + 1 h=2B/N\ge BT^{-\frac{1}{2\beta+1}} h = 2 B / N ≥ B T − 2 β + 1 1 . Because η 2 = min { h 2 , h β } \eta^2=\min\{h^2,h^\beta\} η 2 = min { h 2 , h β } , it follows that
1 η 2 = max { h − 2 , h − β } ≤ max { B − 2 T 2 2 β + 1 , B − β T β 2 β + 1 } ≤ max { B − 2 , B − β } T β + 1 2 β + 1 . (69) \frac{1}{\eta^2}=\max\{h^{-2},h^{-\beta}\}\le\max\left\{B^{-2}T^{\frac{2}{2\beta+1}},B^{-\beta}T^{\frac{\beta}{2\beta+1}}\right\}\le\max\{B^{-2},B^{-\beta}\}T^{\frac{\beta+1}{2\beta+1}}. \tag*{(69)} η 2 1 = max { h − 2 , h − β } ≤ max { B − 2 T 2 β + 1 2 , B − β T 2 β + 1 β } ≤ max { B − 2 , B − β } T 2 β + 1 β + 1 . (69)
Since T ( β + 1 ) / ( 2 β + 1 ) ≥ 1 T^{(\beta+1)/(2\beta+1)}\ge1 T ( β + 1 ) / ( 2 β + 1 ) ≥ 1 ,
1 + γ T 2 η 2 ≤ [ 1 + max { B − 2 , B − β } γ T 2 ] T β + 1 2 β + 1 ≤ max { 1 , B − 2 , B − β } ( 1 + γ T 2 ) T β + 1 2 β + 1 . 1+\frac{\gamma_T^2}{\eta^2}\le\left[1+\max\{B^{-2},B^{-\beta}\}\gamma_T^2\right]T^{\frac{\beta+1}{2\beta+1}}\le\max\{1,B^{-2},B^{-\beta}\}(1+\gamma_T^2)T^{\frac{\beta+1}{2\beta+1}}. 1 + η 2 γ T 2 ≤ [ 1 + max { B − 2 , B − β } γ T 2 ] T 2 β + 1 β + 1 ≤ max { 1 , B − 2 , B − β } ( 1 + γ T 2 ) T 2 β + 1 β + 1 .
Substituting this inequality into (68) gives
∑ t ∈ T exp [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t ) ] ≤ B D max { 1 , B − 2 , B − β } d ( 1 + γ T 2 ) log ( 1 + T C x 2 d ) T β + 1 2 β + 1 . (70) \sum_{t\in\mathcal{T}^{\exp}}[\operatorname{Rev}(x_t,p_t^\star)-\operatorname{Rev}(x_t,p_t)]\le BD\max\{1,B^{-2},B^{-\beta}\}d(1+\gamma_T^2)\log\left(1+\frac{TC_x^2}{d}\right)T^{\frac{\beta+1}{2\beta+1}}. \tag*{(70)} t ∈ T e x p ∑ [ Rev ( x t , p t ⋆ ) − Rev ( x t , p t )] ≤ B D max { 1 , B − 2 , B − β } d ( 1 + γ T 2 ) log ( 1 + d T C x 2 ) T 2 β + 1 β + 1 . (70)
Step 5: Completion of the tuned bound. Define
C : = max { B D max { 1 , B − 2 , B − β } , 64 2 C ω B ( D + 1 ) + 32 C ω B ( 2 B ) β + 8 B D + L Rev } . C := \max\left\{BD\max\{1,B^{-2},B^{-\beta}\},64\sqrt{2}C_{\omega B}(D+1)+32C_{\omega B}(2B)^\beta+8BD+L_{\operatorname{Rev}}\right\}. C := max { B D max { 1 , B − 2 , B − β } , 64 2 C ω B ( D + 1 ) + 32 C ω B ( 2 B ) β + 8 B D + L Rev } .
By Lemma A.2 and Lemma 5.1, C C C is finite and depends only on the fixed problem primitives. Combining (67) and (70) yields
Reg ( T ) ≤ C [ d ( 1 + γ T 2 ) log ( 1 + T C x 2 d ) + ( 1 + λ ) ι T ] T β + 1 2 β + 1 , \operatorname{Reg}(T) \le C\left[d(1+\gamma_T^2)\log\left(1+\frac{TC_x^2}{d}\right)+(1+\sqrt{\lambda})\iota_T\right]T^{\frac{\beta+1}{2\beta+1}}, Reg ( T ) ≤ C [ d ( 1 + γ T 2 ) log ( 1 + d T C x 2 ) + ( 1 + λ ) ι T ] T 2 β + 1 β + 1 ,
which proves (16). Finally, the definitions of γ T \gamma_T γ T and ι T \iota_T ι T imply γ T 2 = O ( log T ) , ι T = O ( log T ) \gamma_T^2=\mathcal{O}(\log T),\iota_T=\mathcal{O}(\log T) γ T 2 = O ( log T ) , ι T = O ( log T ) . Consequently,
d ( 1 + γ T 2 ) log ( 1 + T C x 2 d ) + ( 1 + λ ) ι T = O ( log 2 T ) , d(1+\gamma_T^2)\log\left(1+\frac{TC_x^2}{d}\right)+(1+\sqrt{\lambda})\iota_T=\mathcal{O}(\log^2 T), d ( 1 + γ T 2 ) log ( 1 + d T C x 2 ) + ( 1 + λ ) ι T = O ( log 2 T ) ,
and it follows that
Reg ( T ) = O ~ ( T β + 1 2 β + 1 ) . \operatorname{Reg}(T)=\widetilde{\mathcal{O}}\left(T^{\frac{\beta+1}{2\beta+1}}\right). Reg ( T ) = O ( T 2 β + 1 β + 1 ) .
with probability at least 1 − δ 1-\delta 1 − δ . This completes the proof. □
Proof of Lemma 5.8.
Proof. We construct g 0 g_0 g 0 in four steps. We first build a nonincreasing envelope that generates a flat globally optimal revenue region and has more mass than required by the zero-mean condition. We then smooth its only kink without changing the flat region, truncate its right tail so that the zero-mean condition holds exactly, and finally verify the remaining properties and construct the compensation function χ \chi χ .
Step 1: Constructing a flat-revenue envelope. We first construct a nonincreasing function whose associated revenue is constant at its global maximum over an interior price interval. The construction initially has more total mass than required by the zero-mean condition; this excess mass will be removed in Step 3.
Define
g ˉ ( u ) : = min { D , D ( ( x ∘ ) ⊤ θ ∘ − B ε / 2 ) ( x ∘ ) ⊤ θ ∘ + u } , u ∈ [ − B ε , B ε ] , \bar{g}(u):=\min\left\{D,\frac{D((x^\circ)^\top\theta^\circ-B_\varepsilon/2)}{(x^\circ)^\top\theta^\circ+u}\right\},\qquad u\in[-B_\varepsilon,B_\varepsilon], g ˉ ( u ) := min { D , ( x ∘ ) ⊤ θ ∘ + u D (( x ∘ ) ⊤ θ ∘ − B ε /2 ) } , u ∈ [ − B ε , B ε ] ,
where the ratio is interpreted as + ∞ +\infty + ∞ when ( x ∘ ) ⊤ θ ∘ + u = 0 (x^\circ)^\top\theta^\circ+u=0 ( x ∘ ) ⊤ θ ∘ + u = 0 . Then
g ˉ ( u ) = D , ∀ u ≤ − B ε 2 \bar{g}(u)=D,\quad\forall u\le-\frac{B_\varepsilon}{2} g ˉ ( u ) = D , ∀ u ≤ − 2 B ε
and
g ˉ ( u ) = D ( ( x ∘ ) ⊤ θ ∘ − B ε / 2 ) ( x ∘ ) ⊤ θ ∘ + u , ∀ u > − B ε 2 . \bar{g}(u)=\frac{D((x^\circ)^\top\theta^\circ-B_\varepsilon/2)}{(x^\circ)^\top\theta^\circ+u},\quad\forall u>-\frac{B_\varepsilon}{2}. g ˉ ( u ) = ( x ∘ ) ⊤ θ ∘ + u D (( x ∘ ) ⊤ θ ∘ − B ε /2 ) , ∀ u > − 2 B ε .
Consequently,
∫ − B ε B ε g ˉ ( u ) d u = D B ε 2 + D ( ( x ∘ ) ⊤ θ ∘ − B ε 2 ) log ( 1 + 3 B ε 2 ( ( x ∘ ) ⊤ θ ∘ − B ε / 2 ) ) . (71) \int_{-B_\varepsilon}^{B_\varepsilon}\bar{g}(u)\,\mathrm{d}u
=\frac{DB_\varepsilon}{2}+D\left((x^\circ)^\top\theta^\circ-\frac{B_\varepsilon}{2}\right)\log\left(1+\frac{3B_\varepsilon}{2((x^\circ)^\top\theta^\circ-B_\varepsilon/2)}\right). \tag*{(71)} ∫ − B ε B ε g ˉ ( u ) d u = 2 D B ε + D ( ( x ∘ ) ⊤ θ ∘ − 2 B ε ) log ( 1 + 2 (( x ∘ ) ⊤ θ ∘ − B ε /2 ) 3 B ε ) . (71)
For every a > 0 a>0 a > 0 , the function x log ( 1 + a / x ) x\log(1+a/x) x log ( 1 + a / x ) is strictly increasing in x x x on ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) , because
d d x [ x log ( 1 + a x ) ] = log ( 1 + a x ) − a x + a > 0. \frac{\mathrm{d}}{\mathrm{d}x}\left[x\log\left(1+\frac{a}{x}\right)\right]
=\log\left(1+\frac{a}{x}\right)-\frac{a}{x+a}>0. d x d [ x log ( 1 + x a ) ] = log ( 1 + x a ) − x + a a > 0.
Since ( x ∘ ) ⊤ θ ∘ − B ε / 2 ≥ B ε / 2 (x^\circ)^\top\theta^\circ-B_\varepsilon/2\ge B_\varepsilon/2 ( x ∘ ) ⊤ θ ∘ − B ε /2 ≥ B ε /2 , take a = 3 B ε / 2 a=3B_\varepsilon/2 a = 3 B ε /2 and equation (71) implies
∫ − B ε B ε g ˉ ( u ) d u ≥ D B ε 2 + D B ε 2 log 4 > D B ε . (72) \int_{-B_\varepsilon}^{B_\varepsilon}\bar{g}(u)\,\mathrm{d}u\ge\frac{DB_\varepsilon}{2}+\frac{DB_\varepsilon}{2}\log4>DB_\varepsilon. \tag*{(72)} ∫ − B ε B ε g ˉ ( u ) d u ≥ 2 D B ε + 2 D B ε log 4 > D B ε . (72)
Set
p L : = ( x ∘ ) ⊤ θ ∘ − 3 B ε 8 , p U : = ( x ∘ ) ⊤ θ ∘ − B ε 4 . p_L:=(x^\circ)^\top\theta^\circ-\frac{3B_\varepsilon}{8},\qquad p_U:=(x^\circ)^\top\theta^\circ-\frac{B_\varepsilon}{4}. p L := ( x ∘ ) ⊤ θ ∘ − 8 3 B ε , p U := ( x ∘ ) ⊤ θ ∘ − 4 B ε .
Because ( x ∘ ) ⊤ θ ∘ ∈ [ B ε , B − B ε ] (x^\circ)^\top\theta^\circ\in[B_\varepsilon,B-B_\varepsilon] ( x ∘ ) ⊤ θ ∘ ∈ [ B ε , B − B ε ] , we have 0 < p L < p U < B 0<p_L<p_U<B 0 < p L < p U < B and
[ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] = [ − 3 B ε 8 , − B ε 4 ] ⊂ ( − B ε , B ε ) . \left[p_L-(x^\circ)^\top\theta^\circ,p_U-(x^\circ)^\top\theta^\circ\right]=\left[-\frac{3B_\varepsilon}{8},-\frac{B_\varepsilon}{4}\right]\subset(-B_\varepsilon,B_\varepsilon). [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] = [ − 8 3 B ε , − 4 B ε ] ⊂ ( − B ε , B ε ) .
For every p ∈ [ p L , p U ] p\in[p_L,p_U] p ∈ [ p L , p U ] , we have p − ( x ∘ ) ⊤ θ ∘ ≥ − 3 B ε / 8 > − B ε / 2 p-(x^\circ)^\top\theta^\circ\ge-3B_\varepsilon/8>-B_\varepsilon/2 p − ( x ∘ ) ⊤ θ ∘ ≥ − 3 B ε /8 > − B ε /2 . Hence,
p g ˉ ( p − ( x ∘ ) ⊤ θ ∘ ) = p D ( ( x ∘ ) ⊤ θ ∘ − B ε / 2 ) p = D ( ( x ∘ ) ⊤ θ ∘ − B ε 2 ) . (73) p\bar{g}\left(p-(x^\circ)^\top\theta^\circ\right)=p\frac{D\left((x^\circ)^\top\theta^\circ-B_\varepsilon/2\right)}{p}=D\left((x^\circ)^\top\theta^\circ-\frac{B_\varepsilon}{2}\right). \tag*{(73)} p g ˉ ( p − ( x ∘ ) ⊤ θ ∘ ) = p p D ( ( x ∘ ) ⊤ θ ∘ − B ε /2 ) = D ( ( x ∘ ) ⊤ θ ∘ − 2 B ε ) . (73)
Thus, the revenue induced by g ˉ \bar{g} g ˉ is constant on [ p L , p U ] [p_L,p_U] [ p L , p U ] .
Step 2: Smoothing the envelope without changing the flat region. The envelope g ˉ \bar{g} g ˉ is continuous but has a kink at − B ε / 2 -B_\varepsilon/2 − B ε /2 . We smooth this kink over a short interval while preserving monotonicity, remaining below g ˉ \bar{g} g ˉ , and leaving the flat-revenue region unchanged.
Fix a nondecreasing infinitely differentiable function ρ : R → [ 0 , 1 ] \rho:\mathbb{R}\to[0,1] ρ : R → [ 0 , 1 ] satisfying
ρ ( z ) = 0 for z ≤ 0 , ρ ( z ) = 1 for z ≥ 1 , ρ ′ ( z ) > 0 for z ∈ ( 0 , 1 ) . \rho(z)=0\quad\text{for }z\le0,\qquad\rho(z)=1\quad\text{for }z\ge1,\qquad\rho'(z)>0\quad\text{for }z\in(0,1). ρ ( z ) = 0 for z ≤ 0 , ρ ( z ) = 1 for z ≥ 1 , ρ ′ ( z ) > 0 for z ∈ ( 0 , 1 ) .
Choose ζ > 0 \zeta>0 ζ > 0 sufficiently small that
ζ < B ε 64 and 6 D ζ < ∫ − B ε B ε g ˉ ( u ) d u − D B ε . (74) \zeta<\frac{B_\varepsilon}{64}\quad\text{and}\quad6D\zeta<\int_{-B_\varepsilon}^{B_\varepsilon}\bar{g}(u)\,\mathrm{d}u-DB_\varepsilon. \tag*{(74)} ζ < 64 B ε and 6 D ζ < ∫ − B ε B ε g ˉ ( u ) d u − D B ε . (74)
There exists z 0 ∈ ( − B ε / 2 − 2 ζ , − B ε / 2 ) z_0\in(-B_\varepsilon/2-2\zeta,-B_\varepsilon/2) z 0 ∈ ( − B ε /2 − 2 ζ , − B ε /2 ) such that
∫ − B ε / 2 − 2 ζ − B ε / 2 + 2 ζ D ( ( x ∘ ) ⊤ θ ∘ − B ε / 2 ) ( ( x ∘ ) ⊤ θ ∘ + v ) 2 ρ ( v − z 0 ζ ) d v = D − D ( ( x ∘ ) ⊤ θ ∘ − B ε / 2 ) ( x ∘ ) ⊤ θ ∘ − B ε / 2 + 2 ζ . (75) \int_{-B_\varepsilon/2-2\zeta}^{-B_\varepsilon/2+2\zeta}\frac{D\left((x^\circ)^\top\theta^\circ-B_\varepsilon/2\right)}{\left((x^\circ)^\top\theta^\circ+v\right)^2}\rho\left(\frac{v-z_0}{\zeta}\right)\,\mathrm{d}v
=
D-\frac{D\left((x^\circ)^\top\theta^\circ-B_\varepsilon/2\right)}{(x^\circ)^\top\theta^\circ-B_\varepsilon/2+2\zeta}. \tag*{(75)} ∫ − B ε /2 − 2 ζ − B ε /2 + 2 ζ ( ( x ∘ ) ⊤ θ ∘ + v ) 2 D ( ( x ∘ ) ⊤ θ ∘ − B ε /2 ) ρ ( ζ v − z 0 ) d v = D − ( x ∘ ) ⊤ θ ∘ − B ε /2 + 2 ζ D ( ( x ∘ ) ⊤ θ ∘ − B ε /2 ) . (75)
Since ( x ∘ ) ⊤ θ ∘ ≥ B ε (x^\circ)^\top\theta^\circ\ge B_\varepsilon ( x ∘ ) ⊤ θ ∘ ≥ B ε , we have ( x ∘ ) ⊤ θ ∘ − B ε / 2 − 2 ζ > 0 (x^\circ)^\top\theta^\circ-B_\varepsilon/2-2\zeta>0 ( x ∘ ) ⊤ θ ∘ − B ε /2 − 2 ζ > 0 , and the denominator above is strictly positive. Indeed, the left-hand side is continuous in z 0 z_0 z 0 , whereas the right-hand side equals
∫ − B ε / 2 − B ε / 2 + 2 ζ D ( ( x ∘ ) ⊤ θ ∘ − B ε / 2 ) ( ( x ∘ ) ⊤ θ ∘ + v ) 2 d v . \int_{-B_\varepsilon/2}^{-B_\varepsilon/2+2\zeta}\frac{D\left((x^\circ)^\top\theta^\circ-B_\varepsilon/2\right)}{\left((x^\circ)^\top\theta^\circ+v\right)^2}\,\mathrm{d}v. ∫ − B ε /2 − B ε /2 + 2 ζ ( ( x ∘ ) ⊤ θ ∘ + v ) 2 D ( ( x ∘ ) ⊤ θ ∘ − B ε /2 ) d v .
When z 0 = − B ε / 2 − 2 ζ z_0=-B_\varepsilon/2-2\zeta z 0 = − B ε /2 − 2 ζ , we have
ρ ( v − z 0 ζ ) = 1 for every v ≥ − B ε 2 − ζ . \rho\left(\frac{v-z_0}{\zeta}\right)=1\quad\text{for every }v\ge-\frac{B_\varepsilon}{2}-\zeta. ρ ( ζ v − z 0 ) = 1 for every v ≥ − 2 B ε − ζ .
Hence the left-hand side of (75) is at least
∫ − B ε / 2 − ζ − B ε / 2 + 2 ζ D ( ( x ∘ ) ⊤ θ ∘ − B ε / 2 ) ( ( x ∘ ) ⊤ θ ∘ + v ) 2 d v . \int_{-B_\varepsilon/2-\zeta}^{-B_\varepsilon/2+2\zeta}\frac{D\left((x^\circ)^\top\theta^\circ-B_\varepsilon/2\right)}{\left((x^\circ)^\top\theta^\circ+v\right)^2}\,\mathrm{d}v. ∫ − B ε /2 − ζ − B ε /2 + 2 ζ ( ( x ∘ ) ⊤ θ ∘ + v ) 2 D ( ( x ∘ ) ⊤ θ ∘ − B ε /2 ) d v .
which is strictly larger than the right-hand side since the additional integral over [ − B ε / 2 − ζ , − B ε / 2 ] [-B_{\varepsilon}/2-\zeta,-B_{\varepsilon}/2] [ − B ε /2 − ζ , − B ε /2 ] is strictly positive.
In contrast, when z 0 = − B ε / 2 z_0=-B_{\varepsilon}/2 z 0 = − B ε /2 , the multiplier is zero for v ≤ − B ε / 2 v\le-B_{\varepsilon}/2 v ≤ − B ε /2 , lies strictly between zero and one for v ∈ ( − B ε / 2 , − B ε / 2 + ζ ) v\in(-B_{\varepsilon}/2,-B_{\varepsilon}/2+\zeta) v ∈ ( − B ε /2 , − B ε /2 + ζ ) , and equals one for v ≥ − B ε / 2 + ζ v\ge-B_{\varepsilon}/2+\zeta v ≥ − B ε /2 + ζ . Therefore, the left-hand side is strictly smaller than
∫ − B ε / 2 − B ε / 2 + 2 ζ D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( ( x ∘ ) T θ ∘ + v ) 2 d v , \int_{-B_{\varepsilon}/2}^{-B_{\varepsilon}/2+2\zeta}
\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{((x^\circ)^{\mathsf T}\theta^\circ+v)^2}\,\mathrm{d}v, ∫ − B ε /2 − B ε /2 + 2 ζ (( x ∘ ) T θ ∘ + v ) 2 D (( x ∘ ) T θ ∘ − B ε /2 ) d v ,
which is precisely the right-hand side. Applying the intermediate value theorem therefore yields a z 0 ∈ ( − B ε / 2 − 2 ζ , − B ε / 2 ) z_0\in(-B_{\varepsilon}/2-2\zeta,-B_{\varepsilon}/2) z 0 ∈ ( − B ε /2 − 2 ζ , − B ε /2 ) for which (75) holds.
Define
g ~ ( u ) : = { D , u ≤ − B ε / 2 − 2 ζ , D − ∫ − B ε / 2 − 2 ζ u D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( ( x ∘ ) T θ ∘ + v ) 2 ρ ( v − z 0 ζ ) d v , − B ε / 2 − 2 ζ < u < − B ε / 2 + 2 ζ , D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( x ∘ ) T θ ∘ + u , u ≥ − B ε / 2 + 2 ζ . \widetilde{g}(u):=
\begin{cases}
D, & u\le-B_{\varepsilon}/2-2\zeta,\\
D-\int_{-B_{\varepsilon}/2-2\zeta}^{u}
\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{((x^\circ)^{\mathsf T}\theta^\circ+v)^2}
\rho\left(\frac{v-z_0}{\zeta}\right)\,\mathrm{d}v,
& -B_{\varepsilon}/2-2\zeta<u<-B_{\varepsilon}/2+2\zeta,\\
\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{(x^\circ)^{\mathsf T}\theta^\circ+u},
& u\ge-B_{\varepsilon}/2+2\zeta.
\end{cases} g ( u ) := ⎩ ⎨ ⎧ D , D − ∫ − B ε /2 − 2 ζ u (( x ∘ ) T θ ∘ + v ) 2 D (( x ∘ ) T θ ∘ − B ε /2 ) ρ ( ζ v − z 0 ) d v , ( x ∘ ) T θ ∘ + u D (( x ∘ ) T θ ∘ − B ε /2 ) , u ≤ − B ε /2 − 2 ζ , − B ε /2 − 2 ζ < u < − B ε /2 + 2 ζ , u ≥ − B ε /2 + 2 ζ .
The choice of z 0 z_0 z 0 in (75) ensures that the middle piece and the fractional piece of g ~ \widetilde{g} g have the same value at u = − B ε / 2 + 2 ζ u=-B_{\varepsilon}/2+2\zeta u = − B ε /2 + 2 ζ . At the left junction point, the multiplier involving ρ \rho ρ equals zero on a neighborhood, so the middle piece coincides locally with the constant D D D . At the right junction point, the multiplier equals one on a neighborhood, and (75) ensures that the middle piece coincides locally with the fractional piece. Therefore, all derivatives match at both junction points, and g ~ \widetilde{g} g is infinitely differentiable on [ − B ε , B ε ] [-B_{\varepsilon},B_{\varepsilon}] [ − B ε , B ε ] .
We verify the monotonicity and range of g ~ \widetilde{g} g . On the middle interval, we have
g ~ ′ ( u ) = − D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( ( x ∘ ) T θ ∘ + u ) 2 ρ ( u − z 0 ζ ) ≤ 0. \widetilde{g}'(u)
=
-\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{((x^\circ)^{\mathsf T}\theta^\circ+u)^2}
\rho\left(\frac{u-z_0}{\zeta}\right)
\le0. g ′ ( u ) = − (( x ∘ ) T θ ∘ + u ) 2 D (( x ∘ ) T θ ∘ − B ε /2 ) ρ ( ζ u − z 0 ) ≤ 0.
Hence the middle piece is nonincreasing. It starts from D D D at u = − B ε / 2 − 2 ζ u=-B_{\varepsilon}/2-2\zeta u = − B ε /2 − 2 ζ and, by (75), ends at
D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( x ∘ ) T θ ∘ − B ε / 2 + 2 ζ > 0. (76) \frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{(x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2+2\zeta}>0. \tag*{(76)} ( x ∘ ) T θ ∘ − B ε /2 + 2 ζ D (( x ∘ ) T θ ∘ − B ε /2 ) > 0. (76)
Since the left piece is constant at D D D and the right piece is positive and decreasing, g ~ \widetilde{g} g is nonincreasing on [ − B ε , B ε ] [-B_{\varepsilon},B_{\varepsilon}] [ − B ε , B ε ] and satisfies 0 ≤ g ~ ( u ) ≤ D 0\le\widetilde{g}(u)\le D 0 ≤ g ( u ) ≤ D for u ∈ [ − B ε , B ε ] u\in[-B_{\varepsilon},B_{\varepsilon}] u ∈ [ − B ε , B ε ] .
We next show that g ~ ≤ g ‾ \widetilde{g}\le\overline{g} g ≤ g . If u ≤ − B ε / 2 u\le-B_{\varepsilon}/2 u ≤ − B ε /2 , then g ‾ ( u ) = D \overline{g}(u)=D g ( u ) = D and g ~ ( u ) ≤ D \widetilde{g}(u)\le D g ( u ) ≤ D , so the desired inequality holds. If u ≥ − B ε / 2 + 2 ζ u\ge-B_{\varepsilon}/2+2\zeta u ≥ − B ε /2 + 2 ζ , we have g ~ ( u ) = g ‾ ( u ) \widetilde{g}(u)=\overline{g}(u) g ( u ) = g ( u ) by definition. It only remains to consider the case u ∈ [ − B ε / 2 , − B ε / 2 + 2 ζ ] u\in[-B_{\varepsilon}/2,-B_{\varepsilon}/2+2\zeta] u ∈ [ − B ε /2 , − B ε /2 + 2 ζ ] . For every such u u u , by (76), we have
g ~ ( u ) = D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( x ∘ ) T θ ∘ − B ε / 2 + 2 ζ + ∫ u − B ε / 2 + 2 ζ D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( ( x ∘ ) T θ ∘ + v ) 2 ρ ( v − z 0 ζ ) d v ≤ D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( x ∘ ) T θ ∘ − B ε / 2 + 2 ζ + ∫ u − B ε / 2 + 2 ζ D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( ( x ∘ ) T θ ∘ + v ) 2 d v = D ( ( x ∘ ) T θ ∘ − B ε / 2 ) ( x ∘ ) T θ ∘ + u = g ‾ ( u ) . \begin{aligned}
\widetilde{g}(u)
&=
\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{(x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2+2\zeta}
+
\int_u^{-B_{\varepsilon}/2+2\zeta}
\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{((x^\circ)^{\mathsf T}\theta^\circ+v)^2}
\rho\left(\frac{v-z_0}{\zeta}\right)\,\mathrm{d}v\\
&\le
\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{(x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2+2\zeta}
+
\int_u^{-B_{\varepsilon}/2+2\zeta}
\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{((x^\circ)^{\mathsf T}\theta^\circ+v)^2}\,\mathrm{d}v\\
&=
\frac{D((x^\circ)^{\mathsf T}\theta^\circ-B_{\varepsilon}/2)}
{(x^\circ)^{\mathsf T}\theta^\circ+u}
=\overline{g}(u).
\end{aligned} g ( u ) = ( x ∘ ) T θ ∘ − B ε /2 + 2 ζ D (( x ∘ ) T θ ∘ − B ε /2 ) + ∫ u − B ε /2 + 2 ζ (( x ∘ ) T θ ∘ + v ) 2 D (( x ∘ ) T θ ∘ − B ε /2 ) ρ ( ζ v − z 0 ) d v ≤ ( x ∘ ) T θ ∘ − B ε /2 + 2 ζ D (( x ∘ ) T θ ∘ − B ε /2 ) + ∫ u − B ε /2 + 2 ζ (( x ∘ ) T θ ∘ + v ) 2 D (( x ∘ ) T θ ∘ − B ε /2 ) d v = ( x ∘ ) T θ ∘ + u D (( x ∘ ) T θ ∘ − B ε /2 ) = g ( u ) .
Consequently, we have 0 ≤ g ~ ( u ) ≤ g ‾ ( u ) 0\le\widetilde{g}(u)\le\overline{g}(u) 0 ≤ g ( u ) ≤ g ( u ) for u ∈ [ − B ε , B ε ] u\in[-B_{\varepsilon},B_{\varepsilon}] u ∈ [ − B ε , B ε ] .
Finally, the smoothing modification does not affect the flat-revenue region. Indeed, by (74),
− B ϵ 2 + 2 ζ < − 3 B ϵ 8 = p L − ( x ∘ ) ⊤ θ ∘ . -\frac{B_{\epsilon}}{2}+2\zeta<-\frac{3B_{\epsilon}}{8}=p_L-(x^\circ)^\top\theta^\circ. − 2 B ϵ + 2 ζ < − 8 3 B ϵ = p L − ( x ∘ ) ⊤ θ ∘ .
Thus the smoothing interval lies strictly to the left of [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] [p_L-(x^\circ)^\top\theta^\circ,\ p_U-(x^\circ)^\top\theta^\circ] [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] . It follows from the definition of g ~ \widetilde{g} g that
g ~ ( u ) = D ( ( x ∘ ) ⊤ θ ∘ − B ϵ / 2 ) ( x ∘ ) ⊤ θ ∘ + u \widetilde{g}(u)=\frac{D((x^\circ)^\top\theta^\circ-B_{\epsilon}/2)}{(x^\circ)^\top\theta^\circ+u} g ( u ) = ( x ∘ ) ⊤ θ ∘ + u D (( x ∘ ) ⊤ θ ∘ − B ϵ /2 )
on a neighborhood of [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] [p_L-(x^\circ)^\top\theta^\circ,\ p_U-(x^\circ)^\top\theta^\circ] [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] . Hence the smoothing operation leaves the flat-revenue region unchanged.
Step 3: Enforcing the zero-mean condition. Although g ~ \widetilde{g} g is smooth and preserves the flat-revenue region, its integral remains larger than D B ϵ DB_{\epsilon} D B ϵ . Indeed, g ~ \widetilde{g} g differs from g ‾ \overline{g} g only on [ − B ϵ / 2 − 2 ζ , − B ϵ / 2 + 2 ζ ] [-B_{\epsilon}/2-2\zeta,-B_{\epsilon}/2+2\zeta] [ − B ϵ /2 − 2 ζ , − B ϵ /2 + 2 ζ ] , whose length is 4 ζ 4\zeta 4 ζ . Since both functions take values in [ 0 , D ] [0,D] [ 0 , D ] , equations (72) and (74) give
∫ − B ϵ B ϵ g ~ ( u ) d u ≥ ∫ − B ϵ B ϵ g ‾ ( u ) d u − 4 D ζ > D B ϵ . \int_{-B_{\epsilon}}^{B_{\epsilon}}\widetilde{g}(u)\,\mathrm{d}u\ge\int_{-B_{\epsilon}}^{B_{\epsilon}}\overline{g}(u)\,\mathrm{d}u-4D\zeta>DB_{\epsilon}. ∫ − B ϵ B ϵ g ( u ) d u ≥ ∫ − B ϵ B ϵ g ( u ) d u − 4 D ζ > D B ϵ .
We therefore remove part of the right tail of g ~ \widetilde{g} g smoothly, while leaving the flat-revenue region unchanged.
For every ξ ∈ [ p U − ( x ∘ ) ⊤ θ ∘ + ζ , B ϵ − 2 ζ ] \xi\in[p_U-(x^\circ)^\top\theta^\circ+\zeta,B_{\epsilon}-2\zeta] ξ ∈ [ p U − ( x ∘ ) ⊤ θ ∘ + ζ , B ϵ − 2 ζ ] , define
g ξ ( u ) : = g ~ ( u ) [ 1 − ρ ( u − ξ ζ ) ] , u ∈ [ − B ϵ , B ϵ ] . g_{\xi}(u):=\widetilde{g}(u)\left[1-\rho\left(\frac{u-\xi}{\zeta}\right)\right],\qquad u\in[-B_{\epsilon},B_{\epsilon}]. g ξ ( u ) := g ( u ) [ 1 − ρ ( ζ u − ξ ) ] , u ∈ [ − B ϵ , B ϵ ] .
For u ≤ ξ u\le\xi u ≤ ξ , the multiplicative factor equals one, whereas for u ≥ ξ + ζ u\ge\xi+\zeta u ≥ ξ + ζ , it equals zero. On the interval ( ξ , ξ + ζ ) (\xi,\xi+\zeta) ( ξ , ξ + ζ ) , it decreases smoothly from one to zero. Thus g ξ g_{\xi} g ξ agrees with g ~ \widetilde{g} g to the left of ξ \xi ξ and vanishes to the right of ξ + ζ \xi+\zeta ξ + ζ .
Since g ~ \widetilde{g} g is nonnegative and nonincreasing, and since ρ \rho ρ is nondecreasing,
g ξ ′ ( u ) = g ~ ′ ( u ) [ 1 − ρ ( u − ξ ζ ) ] − g ~ ( u ) ζ ρ ′ ( u − ξ ζ ) ≤ 0. g_{\xi}'(u)=\widetilde{g}'(u)\left[1-\rho\left(\frac{u-\xi}{\zeta}\right)\right]-\frac{\widetilde{g}(u)}{\zeta}\rho'\left(\frac{u-\xi}{\zeta}\right)\le0. g ξ ′ ( u ) = g ′ ( u ) [ 1 − ρ ( ζ u − ξ ) ] − ζ g ( u ) ρ ′ ( ζ u − ξ ) ≤ 0.
Hence g ξ g_{\xi} g ξ is nonincreasing. Moreover, we have 0 ≤ g ξ ( u ) ≤ g ~ ( u ) ≤ g ‾ ( u ) 0\le g_{\xi}(u)\le\widetilde{g}(u)\le\overline{g}(u) 0 ≤ g ξ ( u ) ≤ g ( u ) ≤ g ( u ) . Because ξ ≥ p U − ( x ∘ ) ⊤ θ ∘ + ζ \xi\ge p_U-(x^\circ)^\top\theta^\circ+\zeta ξ ≥ p U − ( x ∘ ) ⊤ θ ∘ + ζ , the multiplicative factor equals one throughout [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] [p_L-(x^\circ)^\top\theta^\circ,p_U-(x^\circ)^\top\theta^\circ] [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] . Therefore, on this interval,
g ξ ( u ) = g ~ ( u ) = D ( ( x ∘ ) ⊤ θ ∘ − B ϵ / 2 ) ( x ∘ ) ⊤ θ ∘ + u . g_{\xi}(u)=\widetilde{g}(u)=\frac{D((x^\circ)^\top\theta^\circ-B_{\epsilon}/2)}{(x^\circ)^\top\theta^\circ+u}. g ξ ( u ) = g ( u ) = ( x ∘ ) ⊤ θ ∘ + u D (( x ∘ ) ⊤ θ ∘ − B ϵ /2 ) .
In addition, g ξ = D g_{\xi}=D g ξ = D on a neighborhood of − B ϵ -B_{\epsilon} − B ϵ and g ξ = 0 g_{\xi}=0 g ξ = 0 on a neighborhood of B ϵ B_{\epsilon} B ϵ .
For every fixed u u u , the function g ξ ( u ) g_{\xi}(u) g ξ ( u ) is continuous and nondecreasing in ξ \xi ξ . Since 0 ≤ g ξ ( u ) ≤ D 0\le g_{\xi}(u)\le D 0 ≤ g ξ ( u ) ≤ D , the dominated convergence theorem implies that
ξ ⟼ ∫ − B ϵ B ϵ g ξ ( u ) d u \xi\longmapsto\int_{-B_{\epsilon}}^{B_{\epsilon}}g_{\xi}(u)\,\mathrm{d}u ξ ⟼ ∫ − B ϵ B ϵ g ξ ( u ) d u
is continuous and nondecreasing.
At ξ = p U − ( x ∘ ) ⊤ θ ∘ + ζ \xi=p_U-(x^\circ)^\top\theta^\circ+\zeta ξ = p U − ( x ∘ ) ⊤ θ ∘ + ζ , the function g ξ g_{\xi} g ξ vanishes for u ≥ p U − ( x ∘ ) ⊤ θ ∘ + 2 ζ u\ge p_U-(x^\circ)^\top\theta^\circ+2\zeta u ≥ p U − ( x ∘ ) ⊤ θ ∘ + 2 ζ . Consequently,
∫ − B ϵ B ϵ g ξ ( u ) d u ≤ D ( B ϵ + p U − ( x ∘ ) ⊤ θ ∘ + 2 ζ ) = D ( 3 B ϵ 4 + 2 ζ ) < D B ϵ . \int_{-B_{\epsilon}}^{B_{\epsilon}}g_{\xi}(u)\,\mathrm{d}u\le D\left(B_{\epsilon}+p_U-(x^\circ)^\top\theta^\circ+2\zeta\right)=D\left(\frac{3B_{\epsilon}}{4}+2\zeta\right)<DB_{\epsilon}. ∫ − B ϵ B ϵ g ξ ( u ) d u ≤ D ( B ϵ + p U − ( x ∘ ) ⊤ θ ∘ + 2 ζ ) = D ( 4 3 B ϵ + 2 ζ ) < D B ϵ .
At ξ = B ϵ − 2 ζ \xi= B_{\epsilon} - 2\zeta ξ = B ϵ − 2 ζ , the functions g ξ g_{\xi} g ξ and g ˉ \bar{g} g ˉ may differ only on
[ − B ϵ 2 − 2 ζ , − B ϵ 2 + 2 ζ ] ∪ [ B ϵ − 2 ζ , B ϵ ] , \left[-\frac{B_{\epsilon}}{2} - 2\zeta, -\frac{B_{\epsilon}}{2} + 2\zeta\right] \cup[B_{\epsilon} - 2\zeta, B_{\epsilon}], [ − 2 B ϵ − 2 ζ , − 2 B ϵ + 2 ζ ] ∪ [ B ϵ − 2 ζ , B ϵ ] ,
whose total length is at most 6 ζ 6\zeta 6 ζ . Since both functions take values in [ 0 , D ] [0,D] [ 0 , D ] , equations (72) and (74) imply
∫ − B ϵ B ϵ g B ϵ − 2 ζ ( u ) d u ≥ ∫ − B ϵ B ϵ g ˉ ( u ) d u − 6 D ζ > D B ϵ . \int_{-B_{\epsilon}}^{B_{\epsilon}} g_{B_{\epsilon}-2\zeta}(u)\,\mathrm{d}u \ge\int_{-B_{\epsilon}}^{B_{\epsilon}} \bar{g}(u)\,\mathrm{d}u - 6D\zeta> DB_{\epsilon}. ∫ − B ϵ B ϵ g B ϵ − 2 ζ ( u ) d u ≥ ∫ − B ϵ B ϵ g ˉ ( u ) d u − 6 D ζ > D B ϵ .
Therefore, by continuity, there exists ξ ⋆ ∈ [ p U − ( x ∘ ) T θ ∘ + ζ , B ϵ − 2 ζ ] \xi^{\star} \in[p_U - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} + \zeta, B_{\epsilon} - 2\zeta] ξ ⋆ ∈ [ p U − ( x ∘ ) T θ ∘ + ζ , B ϵ − 2 ζ ] such that
∫ − B ϵ B ϵ g ξ ⋆ ( u ) d u = D B ϵ . \int_{-B_{\epsilon}}^{B_{\epsilon}} g_{\xi^{\star}}(u)\,\mathrm{d}u = DB_{\epsilon}. ∫ − B ϵ B ϵ g ξ ⋆ ( u ) d u = D B ϵ .
Step 4: Verifying the baseline properties and constructing the compensation function. We now set g 0 = g ξ ⋆ g_0 = g_{\xi^{\star}} g 0 = g ξ ⋆ . We first verify that g 0 g_0 g 0 corresponds to an admissible zero-mean noise distribution and generates the desired flat globally optimal revenue region. We then construct χ \chi χ , which will be used to preserve the zero-mean condition in the subsequent perturbation construction.
Set g 0 = g ξ ⋆ g_0 = g_{\xi^{\star}} g 0 = g ξ ⋆ on [ − B ϵ , B ϵ ] [-B_{\epsilon}, B_{\epsilon}] [ − B ϵ , B ϵ ] , and extend it to R \mathbb{R} R by g 0 ( u ) = D g_0(u) = D g 0 ( u ) = D for u ≤ − B ϵ u \le-B_{\epsilon} u ≤ − B ϵ , and g 0 ( u ) = 0 g_0(u) = 0 g 0 ( u ) = 0 for u ≥ B ϵ u \ge B_{\epsilon} u ≥ B ϵ . Because g ξ ⋆ g_{\xi^{\star}} g ξ ⋆ is constant on neighborhoods of both endpoints, this extension is infinitely differentiable on R \mathbb{R} R . In particular, g 0 ∈ H ( β ) g_0 \in\mathcal{H}(\beta) g 0 ∈ H ( β ) on [ − B , B ] [-B,B] [ − B , B ] , with a finite Hölder constant.
The function g 0 / D g_0/D g 0 / D is continuous, nonincreasing, equals one to the left of − B ϵ -B_{\epsilon} − B ϵ , and equals zero to the right of B ϵ B_{\epsilon} B ϵ . It is therefore the survival function of a continuous distribution supported on [ − B ϵ , B ϵ ] [-B_{\epsilon}, B_{\epsilon}] [ − B ϵ , B ϵ ] . Moreover, by the choice of ξ ⋆ \xi^{\star} ξ ⋆ in Step 3,
E [ ϵ t ] = ∫ − B ϵ B ϵ g 0 ( u ) D d u − B ϵ = 0. \mathbb{E}[\epsilon_t] = \int_{-B_{\epsilon}}^{B_{\epsilon}} \frac{g_0(u)}{D}\,\mathrm{d}u - B_{\epsilon} = 0. E [ ϵ t ] = ∫ − B ϵ B ϵ D g 0 ( u ) d u − B ϵ = 0.
We next verify the revenue properties. If p − ( x ∘ ) T θ ∘ ∈ [ − B ϵ , B ϵ ] p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} \in[-B_{\epsilon}, B_{\epsilon}] p − ( x ∘ ) T θ ∘ ∈ [ − B ϵ , B ϵ ] and p > 0 p > 0 p > 0 , then g 0 ≤ g ˉ g_0 \le\bar{g} g 0 ≤ g ˉ , and hence
p g 0 ( p − ( x ∘ ) T θ ∘ ) ≤ p min { D , D ( ( x ∘ ) T θ ∘ − B ϵ / 2 ) p } ≤ D ( ( x ∘ ) T θ ∘ − B ϵ 2 ) . p g_0\left(p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ}\right) \le p \min\left\{D, \frac{D\left((\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} - B_{\epsilon}/2\right)}{p}\right\} \le D\left((\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} - \frac{B_{\epsilon}}{2}\right). p g 0 ( p − ( x ∘ ) T θ ∘ ) ≤ p min { D , p D ( ( x ∘ ) T θ ∘ − B ϵ /2 ) } ≤ D ( ( x ∘ ) T θ ∘ − 2 B ϵ ) .
If p = 0 p = 0 p = 0 , the same inequality is immediate. If p − ( x ∘ ) T θ ∘ ≤ − B ϵ p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} \le-B_{\epsilon} p − ( x ∘ ) T θ ∘ ≤ − B ϵ , then g 0 ( p − ( x ∘ ) T θ ∘ ) = D g_0\left(p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ}\right) = D g 0 ( p − ( x ∘ ) T θ ∘ ) = D and
p g 0 ( p − ( x ∘ ) T θ ∘ ) = D p ≤ D ( ( x ∘ ) T θ ∘ − B ϵ ) < D ( ( x ∘ ) T θ ∘ − B ϵ 2 ) . p g_0\left(p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ}\right) = Dp \le D\left((\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} - B_{\epsilon}\right) < D\left((\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} - \frac{B_{\epsilon}}{2}\right). p g 0 ( p − ( x ∘ ) T θ ∘ ) = D p ≤ D ( ( x ∘ ) T θ ∘ − B ϵ ) < D ( ( x ∘ ) T θ ∘ − 2 B ϵ ) .
If p − ( x ∘ ) T θ ∘ ≥ B ϵ p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} \ge B_{\epsilon} p − ( x ∘ ) T θ ∘ ≥ B ϵ , then g 0 ( p − ( x ∘ ) T θ ∘ ) = 0 g_0\left(p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ}\right) = 0 g 0 ( p − ( x ∘ ) T θ ∘ ) = 0 . This proves the inequality in (20).
For every p ∈ [ p L , p U ] p \in[p_L,p_U] p ∈ [ p L , p U ] , Step 3 gives
g 0 ( p − ( x ∘ ) T θ ∘ ) = D ( ( x ∘ ) T θ ∘ − B ϵ / 2 ) p . g_0\left(p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ}\right) = \frac{D\left((\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} - B_{\epsilon}/2\right)}{p}. g 0 ( p − ( x ∘ ) T θ ∘ ) = p D ( ( x ∘ ) T θ ∘ − B ϵ /2 ) .
Therefore,
p g 0 ( p − ( x ∘ ) T θ ∘ ) = D ( ( x ∘ ) T θ ∘ − B ϵ 2 ) , p ∈ [ p L , p U ] , p g_0\left(p - (\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ}\right) = D\left((\mathbf{x}^{\circ})^{\mathsf{T}}\theta^{\circ} - \frac{B_{\epsilon}}{2}\right), \quad p \in[p_L,p_U], p g 0 ( p − ( x ∘ ) T θ ∘ ) = D ( ( x ∘ ) T θ ∘ − 2 B ϵ ) , p ∈ [ p L , p U ] ,
which proves the equality in (20).
Furthermore, for every p ∈ [ p L , p U ] p \in[p_L,p_U] p ∈ [ p L , p U ] ,
0 < D ( ( x ∘ ) ⊤ θ ∘ − B ϵ / 2 ) p U ≤ g 0 ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ D ( ( x ∘ ) ⊤ θ ∘ − B ϵ / 2 ) p L < D . 0 < \frac{D((x^\circ)^\top\theta^\circ-B_\epsilon/2)}{p_U} \le g_0(p-(x^\circ)^\top\theta^\circ) \le\frac{D((x^\circ)^\top\theta^\circ-B_\epsilon/2)}{p_L} < D. 0 < p U D (( x ∘ ) ⊤ θ ∘ − B ϵ /2 ) ≤ g 0 ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ p L D (( x ∘ ) ⊤ θ ∘ − B ϵ /2 ) < D .
Because the same representation holds on a neighborhood of [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] [p_L-(x^\circ)^\top\theta^\circ,p_U-(x^\circ)^\top\theta^\circ] [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] , the function g 0 g_0 g 0 is bounded away from both 0 and D D D , and
g 0 ′ ( u ) = − D ( ( x ∘ ) ⊤ θ ∘ − B ϵ / 2 ) ( ( x ∘ ) ⊤ θ ∘ + u ) 2 < 0. g_0'(u)=-\frac{D((x^\circ)^\top\theta^\circ-B_\epsilon/2)}{((x^\circ)^\top\theta^\circ+u)^2}<0. g 0 ′ ( u ) = − (( x ∘ ) ⊤ θ ∘ + u ) 2 D (( x ∘ ) ⊤ θ ∘ − B ϵ /2 ) < 0.
throughout that neighborhood.
It remains to construct the compensation function. Define
χ ( u ) : = 2 ζ ρ ′ ( 2 ( u − ξ ⋆ ) ζ − 1 2 ) . \chi(u):=\frac{2}{\zeta}\rho'\left(\frac{2(u-\xi_\star)}{\zeta}-\frac{1}{2}\right). χ ( u ) := ζ 2 ρ ′ ( ζ 2 ( u − ξ ⋆ ) − 2 1 ) .
Since ρ ′ ≥ 0 \rho' \ge0 ρ ′ ≥ 0 , we have χ ( u ) ≥ 0 \chi(u) \ge0 χ ( u ) ≥ 0 . By the definition of χ \chi χ ,
∫ R χ ( u ) d u = ∫ R 2 ζ ρ ′ ( 2 ( u − ξ ⋆ ) ζ − 1 2 ) d u = [ ρ ( 2 ( u − ξ ⋆ ) ζ − 1 2 ) ] u = − ∞ u = ∞ = 1 − 0 = 1. \int_{\mathbb{R}}\chi(u)\,\mathrm{d}u=\int_{\mathbb{R}}\frac{2}{\zeta}\rho'\left(\frac{2(u-\xi_\star)}{\zeta}-\frac{1}{2}\right)\,\mathrm{d}u=\left[\rho\left(\frac{2(u-\xi_\star)}{\zeta}-\frac{1}{2}\right)\right]_{u=-\infty}^{u=\infty}=1-0=1. ∫ R χ ( u ) d u = ∫ R ζ 2 ρ ′ ( ζ 2 ( u − ξ ⋆ ) − 2 1 ) d u = [ ρ ( ζ 2 ( u − ξ ⋆ ) − 2 1 ) ] u = − ∞ u = ∞ = 1 − 0 = 1.
Moreover, since ρ ′ \rho' ρ ′ vanishes outside ( 0 , 1 ) (0,1) ( 0 , 1 ) , by the choice of ξ ⋆ \xi_\star ξ ⋆ in Step 3
supp ( χ ) ⊂ [ ξ ⋆ + ζ 4 , ξ ⋆ + 3 ζ 4 ] ⊂ ( − B ϵ , B ϵ ) ∖ [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] . \operatorname{supp}(\chi)\subset\left[\xi_\star+\frac{\zeta}{4},\xi_\star+\frac{3\zeta}{4}\right]\subset(-B_\epsilon,B_\epsilon)\setminus[p_L-(x^\circ)^\top\theta^\circ,p_U-(x^\circ)^\top\theta^\circ]. supp ( χ ) ⊂ [ ξ ⋆ + 4 ζ , ξ ⋆ + 4 3 ζ ] ⊂ ( − B ϵ , B ϵ ) ∖ [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] .
On a sufficiently small neighborhood of supp ( χ ) \operatorname{supp}(\chi) supp ( χ ) , the factor 1 − ρ ( u − ξ ⋆ ζ ) 1-\rho\left(\frac{u-\xi_\star}{\zeta}\right) 1 − ρ ( ζ u − ξ ⋆ ) is bounded away from both zero and one. Since g ~ \widetilde{g} g is also bounded away from both zero and D D D there, the same holds for g 0 g_0 g 0 .
Finally,
g 0 ′ ( u ) = g ~ ′ ( u ) [ 1 − ρ ( u − ξ ⋆ ζ ) ] − g ~ ( u ) ζ ρ ′ ( u − ξ ⋆ ζ ) < 0 , g_0'(u)=\widetilde{g}'(u)\left[1-\rho\left(\frac{u-\xi_\star}{\zeta}\right)\right]-\frac{\widetilde{g}(u)}{\zeta}\rho'\left(\frac{u-\xi_\star}{\zeta}\right)<0, g 0 ′ ( u ) = g ′ ( u ) [ 1 − ρ ( ζ u − ξ ⋆ ) ] − ζ g ( u ) ρ ′ ( ζ u − ξ ⋆ ) < 0 ,
because g ~ ′ ( u ) < 0 \widetilde{g}'(u)<0 g ′ ( u ) < 0 , g ~ ( u ) > 0 \widetilde{g}(u)>0 g ( u ) > 0 , and ρ ′ ≥ 0 \rho'\ge0 ρ ′ ≥ 0 there. Since g 0 ′ g_0' g 0 ′ is continuous, it is uniformly negative on a sufficiently small closed neighborhood of supp ( χ ) \operatorname{supp}(\chi) supp ( χ ) . □
Proof of Theorem 5.6.
Proof.
Step 1: Constructing the perturbed instances. Let g 0 g_0 g 0 , p L p_L p L , p U p_U p U , and χ \chi χ be as in Lemma 5.8. Set K T : = ⌈ T 1 / ( 2 β + 1 ) ⌉ K_T:=\lceil T^{1/(2\beta+1)}\rceil K T := ⌈ T 1/ ( 2 β + 1 ) ⌉ and, for every k ∈ [ K T ] k\in[K_T] k ∈ [ K T ] , define
p ˉ k : = p L + ( k − 1 2 ) p U − p L K T . \bar{p}_k:=p_L+\left(k-\frac{1}{2}\right)\frac{p_U-p_L}{K_T}. p ˉ k := p L + ( k − 2 1 ) K T p U − p L .
Thus, p ˉ 1 , … , p ˉ K T \bar{p}_1,\ldots,\bar{p}_{K_T} p ˉ 1 , … , p ˉ K T are equally spaced in [ p L , p U ] [p_L,p_U] [ p L , p U ] . For brevity, write u k : = p ˉ k − ( x ∘ ) ⊤ θ ∘ u_k:=\bar{p}_k-(x^\circ)^\top\theta^\circ u k := p ˉ k − ( x ∘ ) ⊤ θ ∘ . Choose a sufficiently small constant κ > 0 \kappa>0 κ > 0 , depending only on the fixed problem primitives. For every k ∈ [ K T ] k\in[K_T] k ∈ [ K T ] , define
Δ k ( u ) : = { κ K T − β ( ( x ∘ ) ⊤ θ ∘ + u ) ρ ′ ( 1 / 2 ) ρ ′ ( 4 K T ( u − u k ) p U − p L + 1 2 ) , ∣ u − u k ∣ < p U − p L 8 K T , 0 , otherwise , (77) \Delta_k(u):=
\begin{cases}
\frac{\kappa K_T^{-\beta}}{((x^\circ)^\top\theta^\circ+u)\rho'(1/2)}
\rho'\left(\frac{4K_T(u-u_k)}{p_U-p_L}+\frac{1}{2}\right), & |u-u_k|<\frac{p_U-p_L}{8K_T},\\
0, & \text{otherwise},
\end{cases} \tag*{(77)} Δ k ( u ) := { (( x ∘ ) ⊤ θ ∘ + u ) ρ ′ ( 1/2 ) κ K T − β ρ ′ ( p U − p L 4 K T ( u − u k ) + 2 1 ) , 0 , ∣ u − u k ∣ < 8 K T p U − p L , otherwise , (77)
and
g k ( u ) : = g 0 ( u ) + Δ k ( u ) − χ ( u ) ∫ R Δ k ( v ) d v . (78) g_k(u) := g_0(u) + \Delta_k(u) - \chi(u) \int_{\mathbb{R}} \Delta_k(v)\,\mathrm{d}v. \tag*{(78)} g k ( u ) := g 0 ( u ) + Δ k ( u ) − χ ( u ) ∫ R Δ k ( v ) d v . (78)
The term Δ k \Delta_k Δ k creates a local revenue increase near p ˉ k \bar{p}_k p ˉ k , while the term involving χ \chi χ removes the same total mass away from this region.
Because ρ ′ ( 1 / 2 ) > 0 \rho^{\prime}(1/2)>0 ρ ′ ( 1/2 ) > 0 , Δ k \Delta_k Δ k is well defined and nonnegative. Moreover,
supp ( Δ k ) ⊂ [ p ˉ k − ( x ∘ ) T θ ∘ − p U − p L 8 K T , p ˉ k − ( x ∘ ) T θ ∘ + p U − p L 8 K T ] . \operatorname{supp}(\Delta_k) \subset\left[\bar{p}_k-(x^\circ)^{\mathsf{T}}\theta^\circ-\frac{p_U-p_L}{8K_T},\bar{p}_k-(x^\circ)^{\mathsf{T}}\theta^\circ+\frac{p_U-p_L}{8K_T}\right]. supp ( Δ k ) ⊂ [ p ˉ k − ( x ∘ ) T θ ∘ − 8 K T p U − p L , p ˉ k − ( x ∘ ) T θ ∘ + 8 K T p U − p L ] .
These supports are pairwise disjoint and contained in [ p L − ( x ∘ ) T θ ∘ , p U − ( x ∘ ) T θ ∘ ] [p_L-(x^\circ)^{\mathsf{T}}\theta^\circ,p_U-(x^\circ)^{\mathsf{T}}\theta^\circ] [ p L − ( x ∘ ) T θ ∘ , p U − ( x ∘ ) T θ ∘ ] . Since ( x ∘ ) T θ ∘ + u ≥ p L > 0 (x^\circ)^{\mathsf{T}}\theta^\circ+u\ge p_L>0 ( x ∘ ) T θ ∘ + u ≥ p L > 0 on their union, the height of Δ k \Delta_k Δ k is of order K T − β K_T^{-\beta} K T − β , while the length of its support is of order K T − 1 K_T^{-1} K T − 1 . Hence
0 ≤ ∫ R Δ k ( u ) d u ≤ C l b κ K T − ( β + 1 ) (79) 0\le\int_{\mathbb{R}}\Delta_k(u)\,\mathrm{d}u\le C_{\mathrm{lb}}\kappa K_T^{-(\beta+1)} \tag*{(79)} 0 ≤ ∫ R Δ k ( u ) d u ≤ C lb κ K T − ( β + 1 ) (79)
uniformly in k k k and T T T , where C l b < ∞ C_{\mathrm{lb}}<\infty C lb < ∞ denotes a constant that may change from line to line.
Because ρ \rho ρ is infinitely differentiable and constant outside [ 0 , 1 ] [0,1] [ 0 , 1 ] , all derivatives of ρ ′ \rho^{\prime} ρ ′ vanish at zero and one. Therefore, the piecewise-defined function Δ k \Delta_k Δ k is infinitely differentiable on R \mathbb{R} R .
Since the supports of Δ k \Delta_k Δ k and χ \chi χ lie in ( − B ϵ , B ϵ ) (-B_\epsilon,B_\epsilon) ( − B ϵ , B ϵ ) , and ∫ R χ ( u ) d u = 1 \int_{\mathbb{R}}\chi(u)\,\mathrm{d}u=1 ∫ R χ ( u ) d u = 1 ,
∫ − B ϵ B ϵ g k ( u ) d u = ∫ − B ϵ B ϵ g 0 ( u ) d u + ∫ R Δ k ( u ) d u − ( ∫ R χ ( u ) d u ) ( ∫ R Δ k ( u ) d u ) = D B ϵ . \int_{-B_\epsilon}^{B_\epsilon}g_k(u)\,\mathrm{d}u
=
\int_{-B_\epsilon}^{B_\epsilon}g_0(u)\,\mathrm{d}u
+\int_{\mathbb{R}}\Delta_k(u)\,\mathrm{d}u
-\left(\int_{\mathbb{R}}\chi(u)\,\mathrm{d}u\right)\left(\int_{\mathbb{R}}\Delta_k(u)\,\mathrm{d}u\right)
=DB_\epsilon. ∫ − B ϵ B ϵ g k ( u ) d u = ∫ − B ϵ B ϵ g 0 ( u ) d u + ∫ R Δ k ( u ) d u − ( ∫ R χ ( u ) d u ) ( ∫ R Δ k ( u ) d u ) = D B ϵ .
Thus, g k g_k g k and g 0 g_0 g 0 have the same integral on [ − B ϵ , B ϵ ] [-B_\epsilon,B_\epsilon] [ − B ϵ , B ϵ ] .
We first verify that the constructed functions satisfy the required smoothness condition. By enlarging C l b C_{\mathrm{lb}} C lb , if necessary, repeated application of the product and chain rules gives
sup u ∈ R ∣ Δ k ( r ) ( u ) ∣ ≤ C l b κ K T r − β , r = 0 , 1 , … , ϖ ˉ ( β ) + 1. (80) \sup_{u\in\mathbb{R}}\left|\Delta_k^{(r)}(u)\right|\le C_{\mathrm{lb}}\kappa K_T^{r-\beta},\qquad r=0,1,\ldots,\bar{\varpi}(\beta)+1. \tag*{(80)} u ∈ R sup Δ k ( r ) ( u ) ≤ C lb κ K T r − β , r = 0 , 1 , … , ϖ ˉ ( β ) + 1. (80)
We now verify the Hölder condition. Fix u , u ′ ∈ R u,u'\in\mathbb{R} u , u ′ ∈ R . If ∣ u − u ′ ∣ ≤ K T − 1 |u-u'|\le K_T^{-1} ∣ u − u ′ ∣ ≤ K T − 1 , the mean-value theorem gives
∣ Δ k ( ϖ ˉ ( β ) ) ( u ) − Δ k ( ϖ ˉ ( β ) ) ( u ′ ) ∣ ≤ C l b κ K T ϖ ˉ ( β ) + 1 − β ∣ u − u ′ ∣ = C l b κ ∣ u − u ′ ∣ β − ϖ ˉ ( β ) ( K T ∣ u − u ′ ∣ ) ϖ ˉ ( β ) + 1 − β ≤ C l b κ ∣ u − u ′ ∣ β − ϖ ˉ ( β ) . \begin{aligned}
\left|\Delta_k^{(\bar{\varpi}(\beta))}(u)-\Delta_k^{(\bar{\varpi}(\beta))}(u')\right|
&\le C_{\mathrm{lb}}\kappa K_T^{\bar{\varpi}(\beta)+1-\beta}|u-u'|\\
&=C_{\mathrm{lb}}\kappa|u-u'|^{\beta-\bar{\varpi}(\beta)}
\left(K_T|u-u'|\right)^{\bar{\varpi}(\beta)+1-\beta}\\
&\le C_{\mathrm{lb}}\kappa|u-u'|^{\beta-\bar{\varpi}(\beta)}.
\end{aligned} Δ k ( ϖ ˉ ( β )) ( u ) − Δ k ( ϖ ˉ ( β )) ( u ′ ) ≤ C lb κ K T ϖ ˉ ( β ) + 1 − β ∣ u − u ′ ∣ = C lb κ ∣ u − u ′ ∣ β − ϖ ˉ ( β ) ( K T ∣ u − u ′ ∣ ) ϖ ˉ ( β ) + 1 − β ≤ C lb κ ∣ u − u ′ ∣ β − ϖ ˉ ( β ) .
If ∣ u − u ′ ∣ > K T − 1 |u-u'|>K_T^{-1} ∣ u − u ′ ∣ > K T − 1 , then
∣ Δ k ( ϖ ˉ ( β ) ) ( u ) − Δ k ( ϖ ˉ ( β ) ) ( u ′ ) ∣ ≤ 2 sup v ∈ R ∣ Δ k ( ϖ ˉ ( β ) ) ( v ) ∣ ≤ C l b κ K T ϖ ˉ ( β ) − β ≤ C l b κ ∣ u − u ′ ∣ β − ϖ ˉ ( β ) . \left|\Delta_k^{(\bar{\varpi}(\beta))}(u)-\Delta_k^{(\bar{\varpi}(\beta))}(u')\right|
\le2\sup_{v\in\mathbb{R}}\left|\Delta_k^{(\bar{\varpi}(\beta))}(v)\right|
\le C_{\mathrm{lb}}\kappa K_T^{\bar{\varpi}(\beta)-\beta}
\le C_{\mathrm{lb}}\kappa|u-u'|^{\beta-\bar{\varpi}(\beta)}. Δ k ( ϖ ˉ ( β )) ( u ) − Δ k ( ϖ ˉ ( β )) ( u ′ ) ≤ 2 v ∈ R sup Δ k ( ϖ ˉ ( β )) ( v ) ≤ C lb κ K T ϖ ˉ ( β ) − β ≤ C lb κ ∣ u − u ′ ∣ β − ϖ ˉ ( β ) .
Therefore, we have
∣ Δ k ( ϖ ˉ ( β ) ) ( u ) − Δ k ( ϖ ˉ ( β ) ) ( u ′ ) ∣ ≤ C l b κ ∣ u − u ′ ∣ β − ϖ ˉ ( β ) . (81) \left|\Delta_k^{(\bar{\varpi}(\beta))}(u)-\Delta_k^{(\bar{\varpi}(\beta))}(u')\right|
\le C_{\mathrm{lb}}\kappa|u-u'|^{\beta-\bar{\varpi}(\beta)}. \tag*{(81)} Δ k ( ϖ ˉ ( β )) ( u ) − Δ k ( ϖ ˉ ( β )) ( u ′ ) ≤ C lb κ ∣ u − u ′ ∣ β − ϖ ˉ ( β ) . (81)
By Lemma 5.8, g 0 g_0 g 0 satisfies the required Hölder condition. Moreover, χ \chi χ is fixed and infinitely differentiable, while (79) uniformly bounds the coefficient multiplying χ \chi χ . Combining these facts with (81), there exists a finite constant L ˉ g \bar{L}_g L ˉ g , independent of k k k and T T T , such that g k ∈ H ( β , L ˉ g ; [ − B , B ] ) g_k\in\mathcal{H}(\beta,\bar{L}_g;[-B,B]) g k ∈ H ( β , L ˉ g ; [ − B , B ]) for every k ∈ [ K T ] k\in[K_T] k ∈ [ K T ] .
We next verify the range and monotonicity requirements. The supports of Δ k \Delta_k Δ k and χ \chi χ are disjoint. By Lemma 5.8, g 0 g_0 g 0 is bounded away from both 0 and D D D , and g 0 ′ g'_0 g 0 ′ is uniformly negative, on neighborhoods of these supports.
By enlarging C l b C_{\mathrm{lb}} C lb , if necessary, (80) and (79) imply
∥ Δ k ∥ ∞ + ∥ Δ k ′ ∥ ∞ + ∥ χ ( ⋅ ) ∫ R Δ k ( v ) d v ∥ ∞ + ∥ χ ′ ( ⋅ ) ∫ R Δ k ( v ) d v ∥ ∞ ≤ C l b κ . \lVert\Delta_k\rVert_\infty+\lVert\Delta'_k\rVert_\infty+\left\lVert\chi(\cdot)\int_{\mathbb{R}}\Delta_k(v)\,\mathrm{d}v\right\rVert_\infty+\left\lVert\chi'(\cdot)\int_{\mathbb{R}}\Delta_k(v)\,\mathrm{d}v\right\rVert_\infty\le C_{\mathrm{lb}}\kappa. ∥ Δ k ∥ ∞ + ∥ Δ k ′ ∥ ∞ + χ ( ⋅ ) ∫ R Δ k ( v ) d v ∞ + χ ′ ( ⋅ ) ∫ R Δ k ( v ) d v ∞ ≤ C lb κ .
On supp ( Δ k ) \operatorname{supp}(\Delta_k) supp ( Δ k ) , g k = g 0 + Δ k g_k=g_0+\Delta_k g k = g 0 + Δ k , whereas on supp ( χ ) \operatorname{supp}(\chi) supp ( χ ) ,
g k = g 0 − χ ( ⋅ ) ∫ R Δ k ( v ) d v . g_k=g_0-\chi(\cdot)\int_{\mathbb{R}}\Delta_k(v)\,\mathrm{d}v. g k = g 0 − χ ( ⋅ ) ∫ R Δ k ( v ) d v .
Outside these two supports, g k = g 0 g_k=g_0 g k = g 0 . Combining these observations with the preceding bounds, for all sufficiently small κ > 0 \kappa>0 κ > 0 , uniformly in k k k and T T T , 0 ≤ g k ( u ) ≤ D 0\le g_k(u)\le D 0 ≤ g k ( u ) ≤ D and g k ′ ( u ) ≤ 0 g'_k(u)\le0 g k ′ ( u ) ≤ 0 for u ∈ R u\in\mathbb{R} u ∈ R .
Since the supports of Δ k \Delta_k Δ k and χ \chi χ lie strictly inside ( − B ϵ , B ϵ ) (-B_\epsilon,B_\epsilon) ( − B ϵ , B ϵ ) , g k ( u ) = D g_k(u)=D g k ( u ) = D for u ≤ − B ϵ u\le-B_\epsilon u ≤ − B ϵ and g k ( u ) = 0 g_k(u)=0 g k ( u ) = 0 for u ≥ B ϵ u\ge B_\epsilon u ≥ B ϵ . Therefore, g k / D g_k/D g k / D is the survival function of a distribution supported on [ − B ϵ , B ϵ ] [-B_\epsilon,B_\epsilon] [ − B ϵ , B ϵ ] . Moreover, the preceding integral identity gives
E [ ϵ t ] = ∫ − B ϵ B ϵ g k ( u ) D d u − B ϵ = 0. \mathbb{E}[\epsilon_t]=\int_{-B_\epsilon}^{B_\epsilon}\frac{g_k(u)}{D}\,\mathrm{d}u-B_\epsilon=0. E [ ϵ t ] = ∫ − B ϵ B ϵ D g k ( u ) d u − B ϵ = 0.
Under instance k k k , the shocks are i.i.d. with the constructed distribution and are independent of the policy randomization. Moreover, because y t = D 1 { v t ≥ p t } y_t=D1\{v_t\ge p_t\} y t = D 1 { v t ≥ p t } , the demand is measurable with respect to σ ( p t , v t ) \sigma(p_t,v_t) σ ( p t , v t ) . Therefore,
E [ y t ∣ F t − 1 ∨ σ ( x t , p t , v t ) ] = D 1 { v t ≥ p t } = q ( v t − p t ) . \mathbb{E}[y_t\mid\mathcal{F}_{t-1}\vee\sigma(x_t,p_t,v_t)]=D1\{v_t\ge p_t\}=q(v_t-p_t). E [ y t ∣ F t − 1 ∨ σ ( x t , p t , v t )] = D 1 { v t ≥ p t } = q ( v t − p t ) .
Hence, Assumption 3.2 holds. Furthermore, by (17), for every u ∈ I g u\in I_g u ∈ I g ,
E [ q ( ϵ t − u ) ] = D P ( ϵ t ≥ u ) = g k ( u ) . \mathbb{E}[q(\epsilon_t-u)]=D\mathbb{P}(\epsilon_t\ge u)=g_k(u). E [ q ( ϵ t − u )] = D P ( ϵ t ≥ u ) = g k ( u ) .
Thus, each constructed instance satisfies the original model assumptions provided that L g ≥ L ‾ g L_g\ge\overline{L}_g L g ≥ L g .
Step 2: Establishing the revenue gap and statistical closeness. Under instance k k k , the perturbation raises the revenue near p ‾ k \overline{p}_k p k , while prices outside its support receive no such increase.
Since the support of χ \chi χ is disjoint from [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] [p_L-(x^\circ)^\top\theta^\circ,p_U-(x^\circ)^\top\theta^\circ] [ p L − ( x ∘ ) ⊤ θ ∘ , p U − ( x ∘ ) ⊤ θ ∘ ] , the definition of Δ k \Delta_k Δ k gives
p ‾ k g k ( p ‾ k − ( x ∘ ) ⊤ θ ∘ ) = D ( ( x ∘ ) ⊤ θ ∘ − B ϵ 2 ) + κ K T − β . \overline{p}_kg_k\left(\overline{p}_k-(x^\circ)^\top\theta^\circ\right)=D\left((x^\circ)^\top\theta^\circ-\frac{B_\epsilon}{2}\right)+\kappa K T^{-\beta}. p k g k ( p k − ( x ∘ ) ⊤ θ ∘ ) = D ( ( x ∘ ) ⊤ θ ∘ − 2 B ϵ ) + κ K T − β .
If p − ( x ∘ ) ⊤ θ ∘ ∉ supp ( Δ k ) p-(x^\circ)^\top\theta^\circ\notin\operatorname{supp}(\Delta_k) p − ( x ∘ ) ⊤ θ ∘ ∈ / supp ( Δ k ) , then Δ k = 0 \Delta_k=0 Δ k = 0 . Since the term involving χ \chi χ is nonpositive,
p g k ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ p g 0 ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ D ( ( x ∘ ) ⊤ θ ∘ − B ϵ 2 ) . pg_k\left(p-(x^\circ)^\top\theta^\circ\right)\le pg_0\left(p-(x^\circ)^\top\theta^\circ\right)\le D\left((x^\circ)^\top\theta^\circ-\frac{B_\epsilon}{2}\right). p g k ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ p g 0 ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ D ( ( x ∘ ) ⊤ θ ∘ − 2 B ϵ ) .
Consequently,
max p ′ ∈ [ 0 , B ] p ′ g k ( p ′ − ( x ∘ ) ⊤ θ ∘ ) − p g k ( p − ( x ∘ ) ⊤ θ ∘ ) ≥ κ K T − β , p − ( x ∘ ) ⊤ θ ∘ ∉ supp ( Δ k ) . (82) \max_{p'\in[0,B]}p'g_k\left(p'-(x^\circ)^\top\theta^\circ\right)-pg_k\left(p-(x^\circ)^\top\theta^\circ\right)\ge\kappa K T^{-\beta},\qquad p-(x^\circ)^\top\theta^\circ\notin\operatorname{supp}(\Delta_k). \tag*{(82)} p ′ ∈ [ 0 , B ] max p ′ g k ( p ′ − ( x ∘ ) ⊤ θ ∘ ) − p g k ( p − ( x ∘ ) ⊤ θ ∘ ) ≥ κ K T − β , p − ( x ∘ ) ⊤ θ ∘ ∈ / supp ( Δ k ) . (82)
Fix an arbitrary nonanticipating policy π \pi π . Let P k \mathbb{P}_k P k denote the law of the complete observed history, including posted prices and demands, under instance k k k , and let P 0 \mathbb{P}_0 P 0 denote the corresponding law under the baseline instance g 0 g_0 g 0 . Conditional on the past and p t p_t p t , the normalized observation y t / D y_t/D y t / D is Bernoulli with success probability g k ( p t − ( x ∘ ) ⊤ θ ∘ ) / D g_k\left(p_t-(x^\circ)^\top\theta^\circ\right)/D g k ( p t − ( x ∘ ) ⊤ θ ∘ ) / D . Since the policy uses the same conditional decision rule under every instance, the chain rule for relative entropy gives
D K L ( P 0 ∥ P k ) = E 0 [ ∑ t = 1 T D K L ( Bern ( g 0 ( p t − ( x ∘ ) ⊤ θ ∘ ) D ) ∥ Bern ( g k ( p t − ( x ∘ ) ⊤ θ ∘ ) D ) ) ] . (83) D_{\mathrm{KL}}(\mathbb{P}_0 \mathbin\Vert\mathbb{P}_k)=\mathbb{E}_0\left[\sum_{t=1}^{T}D_{\mathrm{KL}}\left(\operatorname{Bern}\left(\frac{g_0\left(p_t-(x^\circ)^\top\theta^\circ\right)}{D}\right)\mathbin\Vert\operatorname{Bern}\left(\frac{g_k\left(p_t-(x^\circ)^\top\theta^\circ\right)}{D}\right)\right)\right]. \tag*{(83)} D KL ( P 0 ∥ P k ) = E 0 [ t = 1 ∑ T D KL ( Bern ( D g 0 ( p t − ( x ∘ ) ⊤ θ ∘ ) ) ∥ Bern ( D g k ( p t − ( x ∘ ) ⊤ θ ∘ ) ) ) ] . (83)
By the properties established in Step 1, the relevant Bernoulli probabilities on the supports of Δ k \Delta_k Δ k and χ \chi χ lie in a fixed subinterval of ( 0 , 1 ) (0,1) ( 0 , 1 ) . By enlarging C l b C_{\mathrm{lb}} C lb , if necessary,
D K L ( Bern ( a ) ∥ Bern ( b ) ) ≤ C l b ( a − b ) 2 D_{\mathrm{KL}}(\operatorname{Bern}(a)\mathbin\Vert\operatorname{Bern}(b))\leq C_{\mathrm{lb}}(a-b)^2 D KL ( Bern ( a ) ∥ Bern ( b )) ≤ C lb ( a − b ) 2
for all probabilities appearing above. Moreover,
∣ g k ( p − ( x ∘ ) ⊤ θ ∘ ) − g 0 ( p − ( x ∘ ) ⊤ θ ∘ ) ∣ ≤ C l b κ K T − β 1 { p − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ) } + C l b κ K T − ( β + 1 ) 1 { p − ( x ∘ ) ⊤ θ ∘ ∈ supp ( χ ) } . \left|g_k\left(p-(x^\circ)^\top\theta^\circ\right)-g_0\left(p-(x^\circ)^\top\theta^\circ\right)\right|
\leq C_{\mathrm{lb}}\kappa K_T^{-\beta}\mathbf{1}\left\{p-(x^\circ)^\top\theta^\circ\in\operatorname{supp}(\Delta_k)\right\}
+C_{\mathrm{lb}}\kappa K_T^{-(\beta+1)}\mathbf{1}\left\{p-(x^\circ)^\top\theta^\circ\in\operatorname{supp}(\chi)\right\}. g k ( p − ( x ∘ ) ⊤ θ ∘ ) − g 0 ( p − ( x ∘ ) ⊤ θ ∘ ) ≤ C lb κ K T − β 1 { p − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ) } + C lb κ K T − ( β + 1 ) 1 { p − ( x ∘ ) ⊤ θ ∘ ∈ supp ( χ ) } .
Using (80) with r = 0 r=0 r = 0 and (79), and enlarging C l b C_{\mathrm{lb}} C lb if necessary, we obtain
D K L ( P 0 ∥ P k ) ≤ C l b κ 2 K T − 2 β E 0 [ ∑ t = 1 T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ) } ] + C l b κ 2 K T − 2 β − 2 T . (84) D_{\mathrm{KL}}(\mathbb{P}_0\mathbin\Vert\mathbb{P}_k)\leq C_{\mathrm{lb}}\kappa^2K_T^{-2\beta}\mathbb{E}_0\left[\sum_{t=1}^{T}\mathbf{1}\left\{p_t-(x^\circ)^\top\theta^\circ\in\operatorname{supp}(\Delta_k)\right\}\right]+C_{\mathrm{lb}}\kappa^2K_T^{-2\beta-2}T. \tag*{(84)} D KL ( P 0 ∥ P k ) ≤ C lb κ 2 K T − 2 β E 0 [ t = 1 ∑ T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ) } ] + C lb κ 2 K T − 2 β − 2 T . (84)
Because the supports of Δ 1 , … , Δ K T \Delta_1,\ldots,\Delta_{K_T} Δ 1 , … , Δ K T are pairwise disjoint,
∑ k = 1 K T ∑ t = 1 T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ) } ≤ T \sum_{k=1}^{K_T}\sum_{t=1}^{T}\mathbf{1}\left\{p_t-(x^\circ)^\top\theta^\circ\in\operatorname{supp}(\Delta_k)\right\}\leq T k = 1 ∑ K T t = 1 ∑ T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ) } ≤ T
pathwise. Hence there exists k ⋆ ∈ [ K T ] k^\star\in[K_T] k ⋆ ∈ [ K T ] such that
E 0 [ ∑ t = 1 T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ⋆ ) } ] ≤ T K T . \mathbb{E}_0\left[\sum_{t=1}^{T}\mathbf{1}\left\{p_t-(x^\circ)^\top\theta^\circ\in\operatorname{supp}(\Delta_{k^\star})\right\}\right]\leq\frac{T}{K_T}. E 0 [ t = 1 ∑ T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ⋆ ) } ] ≤ K T T .
For this k ⋆ k^\star k ⋆ , we have
D K L ( P 0 ∥ P k ⋆ ) ≤ C l b κ 2 T K T − ( 2 β + 1 ) + C l b κ 2 T K T − ( 2 β + 2 ) . D_{\mathrm{KL}}(\mathbb{P}_0\mathbin\Vert\mathbb{P}_{k^\star})\leq C_{\mathrm{lb}}\kappa^2TK_T^{-(2\beta+1)}+C_{\mathrm{lb}}\kappa^2TK_T^{-(2\beta+2)}. D KL ( P 0 ∥ P k ⋆ ) ≤ C lb κ 2 T K T − ( 2 β + 1 ) + C lb κ 2 T K T − ( 2 β + 2 ) .
Since K T ≥ 1 K_T\geq1 K T ≥ 1 , by enlarging C l b C_{\mathrm{lb}} C lb , if necessary, we have
D K L ( P 0 ∥ P k ⋆ ) ≤ C l b κ 2 T K T − ( 2 β + 1 ) . (85) D_{\mathrm{KL}}(\mathbb{P}_0\mathbin\Vert\mathbb{P}_{k^\star})\leq C_{\mathrm{lb}}\kappa^2TK_T^{-(2\beta+1)}. \tag*{(85)} D KL ( P 0 ∥ P k ⋆ ) ≤ C lb κ 2 T K T − ( 2 β + 1 ) . (85)
Step 3: Deriving the regret lower bound. The KL bound in Step 2 shows that instance k ⋆ k^\star k ⋆ is difficult to distinguish from the baseline. We now use this fact to show that the policy misses the corresponding perturbation region sufficiently often. Under instance k ⋆ k^\star k ⋆ , (82) gives
Reg ( T ) ≥ κ K T − β ∑ t = 1 T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∉ supp ( Δ k ⋆ ) } . (86) \operatorname{Reg}(T)\geq\kappa K_T^{-\beta}\sum_{t=1}^{T}\mathbf{1}\left\{p_t-(x^\circ)^\top\theta^\circ\notin\operatorname{supp}(\Delta_{k^\star})\right\}. \tag*{(86)} Reg ( T ) ≥ κ K T − β t = 1 ∑ T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ / supp ( Δ k ⋆ ) } . (86)
For every t t t , apply the Bretagnolle–Huber inequality to the event { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ⋆ ) } \{p_t - (x^\circ)^\top\theta^\circ\in\operatorname{supp}(\Delta_{k^\star})\} { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ⋆ )} .
Summing over t t t gives
E 0 [ ∑ t = 1 T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ⋆ ) } ] + E k ⋆ [ ∑ t = 1 T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∉ supp ( Δ k ⋆ ) } ] ≥ T 2 exp { − D K L ( P 0 ∥ P k ⋆ ) } . (87) \mathbb{E}_0\left[\sum_{t=1}^{T}\mathbf{1}\left\{p_t-(x^\circ)^\top\theta^\circ\in\operatorname{supp}(\Delta_{k^\star})\right\}\right]+\mathbb{E}_{k^\star}\left[\sum_{t=1}^{T}\mathbf{1}\left\{p_t-(x^\circ)^\top\theta^\circ\notin\operatorname{supp}(\Delta_{k^\star})\right\}\right]\geq\frac{T}{2}\exp\left\{-D_{\mathrm{KL}}(\mathbb{P}_0\mathbin\Vert\mathbb{P}_{k^\star})\right\}. \tag*{(87)} E 0 [ t = 1 ∑ T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ⋆ ) } ] + E k ⋆ [ t = 1 ∑ T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ / supp ( Δ k ⋆ ) } ] ≥ 2 T exp { − D KL ( P 0 ∥ P k ⋆ ) } . (87)
By the definition of K T K_T K T , we have T K T − ( 2 β + 1 ) ≤ 1 T K_T^{-(2\beta+1)}\leq1 T K T − ( 2 β + 1 ) ≤ 1 . By further decreasing κ \kappa κ , if necessary, assume that C l b κ 2 ≤ log 2 C_{\mathrm{lb}}\kappa^2\leq\log2 C lb κ 2 ≤ log 2 . This choice preserves all the range, monotonicity, and smoothness properties established above. Equation (85) then implies D K L ( P 0 ∥ P k ⋆ ) ≤ log 2 D_{\mathrm{KL}}(\mathbb{P}_0\mathbin\Vert\mathbb{P}_{k^\star})\leq\log2 D KL ( P 0 ∥ P k ⋆ ) ≤ log 2 . Increasing T 0 T_0 T 0 , if necessary, also guarantees K T ≥ 8 K_T\geq8 K T ≥ 8 . Consequently,
E 0 [ ∑ t = 1 T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ⋆ ) } ] ≤ T K T ≤ T 8 . \mathbb{E}_0\left[\sum_{t=1}^{T}\mathbf{1}\left\{p_t-(x^\circ)^\top\theta^\circ\in\operatorname{supp}(\Delta_{k^\star})\right\}\right]\leq\frac{T}{K_T}\leq\frac{T}{8}. E 0 [ t = 1 ∑ T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ supp ( Δ k ⋆ ) } ] ≤ K T T ≤ 8 T .
Combining this bound with (87) gives
E k ⋆ [ ∑ t = 1 T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∉ supp ( Δ k ⋆ ) } ] ≥ T 4 − T 8 = T 8 . \mathbb{E}_{k^\star}\left[\sum_{t=1}^{T}\mathbf{1}\left\{p_t-(x^\circ)^\top\theta^\circ\notin\operatorname{supp}(\Delta_{k^\star})\right\}\right]\geq\frac{T}{4}-\frac{T}{8}=\frac{T}{8}. E k ⋆ [ t = 1 ∑ T 1 { p t − ( x ∘ ) ⊤ θ ∘ ∈ / supp ( Δ k ⋆ ) } ] ≥ 4 T − 8 T = 8 T .
Taking E k ⋆ \mathbb{E}_{k^\star} E k ⋆ on both sides of (86) therefore yields
E k ⋆ [ Reg ( T ) ] ≥ κ 8 T K T − β . \mathbb{E}_{k^\star}[\operatorname{Reg}(T)]\geq\frac{\kappa}{8}T K_T^{-\beta}. E k ⋆ [ Reg ( T )] ≥ 8 κ T K T − β .
Finally, since K T ≤ 2 T 1 / ( 2 β + 1 ) K_T\leq2T^{1/(2\beta+1)} K T ≤ 2 T 1/ ( 2 β + 1 ) , we obtain
E k ⋆ [ Reg ( T ) ] ≥ κ 2 β + 3 T β + 1 2 β + 1 . \mathbb{E}_{k^\star}[\operatorname{Reg}(T)]\geq\frac{\kappa}{2^{\beta+3}}T^{\frac{\beta+1}{2\beta+1}}. E k ⋆ [ Reg ( T )] ≥ 2 β + 3 κ T 2 β + 1 β + 1 .
The policy π \pi π was arbitrary. Taking the infimum over policies and the supremum over admissible instances completes the proof. □