Introduction
Despite rapid advances in machine learning, human judgment remains essential in many high-stakes decision-making tasks. In domains such as healthcare, AI systems can provide accurate predictions, yet often lack access to the full context available to human experts. Conversely, human decision-makers can incorporate rich contextual information but may benefit from algorithmic guidance. This complementary structure has led to growing interest in human-AI collaboration (HAC) as a framework for combining the strengths of both [23 , 9 , 15 , 6 , 3 , 1 , 14 , 4 , 2 ] .
A key but often overlooked aspect of human-AI collaboration is the impressionability of human decision-makers. In HAC settings, when an AI system defers a task to a human while providing a prediction or auxiliary signal, this information can influence the human’s perception of the task and alter their subsequent behavior and final decision. For example, Bondi et al. [ [9 ] show that, when deferring a task to a human, it is possible to improve human performance by informing the human of the deferral decision without revealing the AI prediction. Vodrahalli et al. [ [35 ] further demonstrate that manipulating the displayed confidence score of an AI system can alter subsequent human judgments. More recently, Bo et al. [ [8 ] reveal that, in collaboration with large language models, different reliance interventions can significantly affect both decision quality and the time required for users to engage with AI advice and answer questions. We use impressionability to mean that a human’s response distribution depends on how AI advice is presented. This impressionability creates a fundamental challenge: the presentation of AI outputs becomes a decision variable to optimize, while the resulting distribution of human responses depends on this choice. This dependence is reminiscent of performative prediction [31 , 27 , 26 ] . Despite growing empirical evidence of such effects [6 , 9 , 25 , 7 , 8 , 11 ] , many theoretical HAC models treat human responses as independent of the presentation of AI advice.
Another fundamental challenge in human–AI collaboration arises from deployment-time variability. In practice, the downstream environment in which a collaboration mechanism is used can differ significantly from the conditions assumed during training. In particular, the evaluation criteria and the cost of querying human expertise may vary across applications, time, and resource constraints [30 , 28 , 5 , 10 ] . For example, in a clinical setting, the cost of consulting a specialist may vary depending on workload or urgency, while the evaluation criterion may shift between minimizing diagnostic error and prioritizing high-risk patients, depending on the application. Moreover, once a system is deployed, access to training data is often limited or unavailable, making it impractical to retrain the model for each new setting. Addressing this challenge requires frameworks that can adapt collaboration policies to varying deployment-time prediction objectives and human-query costs without retraining the learned outcome model. However, existing approaches typically optimize for a single objective and do not directly support such adaptation [30 , 28 , 34 , 32 ] .
Our contributions. To address the challenges of human impressionability and deployment-time variability, we introduce a unified theoretical framework for human-AI collaboration based on collaborative omnimechanism design . Our main contribution is the theoretical foundation of Collaborative Outcome Indistinguishability (COI), a loss-matching condition underlying a computationally tractable omniprediction framework tailored for human-AI collaboration with impressionable human minds. Formally, the COI condition ensures that expected losses under our learned predictive model closely match those under the true data-generating environment for the policies in a prescribed comparison class and the policies induced by the learned model across supported loss-cost pairs. This enables a central planner to select a policy whose expected loss is close to that of the best policy in the comparison class for each supported loss-cost pair, without retraining or requiring an explicit parametric model of impressionable human behavior. The core technical contributions of our work are as follows:
(a). Efficient COI learning via boosting. We extend the omniprediction framework to the HAC setting and propose an efficient boosting algorithm, COI-Boost, that achieves outcome indistinguishability using a reliable auditor. We establish finite-sample guarantees for both the auditor and the boosting procedure, and validate the effectiveness of this approach empirically. Further, we reduce Decision-OI auditing to weighted calibration and obtain a computational guarantee under the coverage and smoothing assumptions. We also derive a sample-complexity lower bound for offline omnimechanism learning.
(b). Capturing impressionable human behavior. We do not posit a parametric model of human responses or require pointwise recovery of their true distribution. Instead, we learn a surrogate outcome model whose expected losses agree with those of the environment for the relevant policies and objectives under standard coverage and complexity assumptions. This enables near-optimal decision-making without requiring a precise model of human behavior.
(c). Extensions to richer collaboration settings. We extend our framework to settings with multiple human experts and to feature-based methods, demonstrating its flexibility and applicability to more complex human-AI collaboration scenarios.
(d). Evaluation across decision regimes. We evaluate cost adaptation with human responses generated by a simulator fitted to real interaction data in binary classification, observed radiologist–AI interactions, and house-price prediction. Together, these experiments test whether a learned predictive model can support input-dependent collaboration policies across distinct decision regimes and adapt to different costs without retraining.
Setup
In this section, we formally introduce our mathematical models and desiderata.
Basic mechanism design with a single loss. For clarity, we first formulate collaboration for a fixed prediction loss and human-query cost, then extend the objective to a family of losses and costs.
As illustrated in Figure 1 , for an input x ∈ X x \in\mathcal{X} x ∈ X , a black-box AI model M M M produces a prediction y M ∈ Y y_M \in\mathcal{Y} y M ∈ Y . In human-AI collaboration, given the context z = ( x , y M ) z = (x, y_M) z = ( x , y M ) , the mechanism first decides whether to request human input using a deferral rule r : Z → { 0 , 1 } r : \mathcal{Z} \to\{0,1\} r : Z → { 0 , 1 } , where Z = X × Y \mathcal{Z} = \mathcal{X} \times\mathcal{Y} Z = X × Y . This leads to two cases:
Figure 1. OmniMechanism setup and workflow diagram.
When r ( z ) = 0 r(z) = 0 r ( z ) = 0 , the process is fully automated, and we do not rely on any human input; the AI model’s prediction y M y_M y M serves as the final output.
When r ( z ) = 1 r(z) = 1 r ( z ) = 1 , our mechanism presents the transformed (possibly augmented, curated, or reframed) AI output, which we also refer to as the signal, s = ϕ ( z ) ∈ S s = \phi(z) \in\mathcal{S} s = ϕ ( z ) ∈ S , to a human decision-maker, resulting in a human output y H ∈ Y y_H \in\mathcal{Y} y H ∈ Y . Here, the function ϕ : Z → S \phi: \mathcal{Z} \to\mathcal{S} ϕ : Z → S denotes the signal-presentation rule. Examples include the parameterized confidence score manipulation function [35 ] and different types of reliance intervention [8 ] mentioned in the introduction. Importantly, because the human decision-maker is influenced by the presented signal, different choices of the signal s s s induce different distributions over the human output. Our mechanism then combines the AI model’s prediction and the human output through a collaboration function g g g , yielding the final output g ( y M , y H ) ∈ Y g(y_M,y_H) \in\mathcal{Y} g ( y M , y H ) ∈ Y . In this process, we must choose both ϕ \phi ϕ and g g g in order to optimize the quality of the final output. Let R \mathcal{R} R , Φ \Phi Φ , and G \mathcal{G} G denote prescribed classes of deferral rules r : Z → { 0 , 1 } r : \mathcal{Z} \to\{0,1\} r : Z → { 0 , 1 } , signal-presentation rules ϕ : Z → S \phi: \mathcal{Z} \to\mathcal{S} ϕ : Z → S , and collaboration functions g : Y × Y → Y g : \mathcal{Y} \times\mathcal{Y} \to\mathcal{Y} g : Y × Y → Y , respectively.
In both cases, the final output quality is evaluated against the ground truth y t r u e ∈ Y t r u e y_{\mathrm{true}} \in\mathcal{Y}_{\mathrm{true}} y true ∈ Y true corresponding to input x x x . For notational convenience, we write the final prediction as y f i n a l y_{\mathrm{final}} y final , which is set to be y M y_M y M when r ( z ) = 0 r(z) = 0 r ( z ) = 0 , and g ( y M , y H ) g(y_M,y_H) g ( y M , y H ) when r ( z ) = 1 r(z) = 1 r ( z ) = 1 . We define H = { ( r , ϕ , g ) ∣ r ∈ R , ϕ ∈ Φ , g ∈ G } \mathcal{H} = \{(r,\phi,g)|r \in\mathcal{R},\phi\in\Phi,g \in\mathcal{G}\} H = {( r , ϕ , g ) ∣ r ∈ R , ϕ ∈ Φ , g ∈ G } . Each triple h = ( r , ϕ , g ) ∈ H h = (r,\phi,g) \in\mathcal{H} h = ( r , ϕ , g ) ∈ H specifies a collaboration mechanism consisting of a deferral rule, a signal-presentation rule, and a collaboration function. Our objective must account for both the cost of human querying and the quality of the final prediction. Thus, given a fixed function pair ω = ( l ~ , c ) \omega= (\tilde{l},c) ω = ( l ~ , c ) , where loss l ~ \tilde{l} l ~ measures prediction error and cost function c ( x ) c(x) c ( x ) represents the cost of querying the human on input x x x , we consider the following total loss at h ∈ H h \in\mathcal{H} h ∈ H :
ℓ ω ( z , y H , y t r u e ; h ) = ( 1 − r ( z ) ) l ~ ( y M , y t r u e ) + r ( z ) [ l ~ ( g ( y M , y H ) , y t r u e ) + c ( x ) ] . \ell^\omega(z,y_H,y_{\mathrm{true}};h) = (1-r(z))\tilde{l}(y_M,y_{\mathrm{true}}) + r(z)\left[\tilde{l}(g(y_M,y_H),y_{\mathrm{true}}) + c(x)\right]. ℓ ω ( z , y H , y true ; h ) = ( 1 − r ( z )) l ~ ( y M , y true ) + r ( z ) [ l ~ ( g ( y M , y H ) , y true ) + c ( x ) ] .
Here, the expected loss depends on r , g r,g r , g and most importantly, depends on the signal through the distribution of y H y_H y H . We also note that, in general, Y t r u e \mathcal{Y}_{\mathrm{true}} Y true may differ from Y \mathcal{Y} Y . For example, Y t r u e = { 0 , 1 } \mathcal{Y}_{\mathrm{true}} = \{0,1\} Y true = { 0 , 1 } may represent binary ground-truth labels, while Y = [ 0 , 1 ] \mathcal{Y} = [0,1] Y = [ 0 , 1 ] represents probability-valued predictions.
Example 2.1 (Illustration of Notations). Given a patient’s medical image and clinical records x x x , a black-box AI model first produces a diagnosis prediction y M y_M y M . The system then decides whether to rely directly on the AI prediction or defer the case to a physician. If the task is deferred, the physician is shown an additional signal s = ϕ ( z ) s = \phi(z) s = ϕ ( z ) derived from the AI output, such as a displayed confidence score or a reliability warning. For example, presenting a highly confident AI prediction may increase the physician’s reliance on the AI recommendation, while a cautionary intervention may encourage more careful review. Because human judgments are impressionable, different presentations can lead to different physician decisions. The final diagnosis is then produced by combining the AI prediction and the physician’s assessment through a collaboration function g g g [4 ] .
From the previous discussion, we see that, to design an (optimal) mechanism for human-AI collaboration with impressionable human decision-makers, given a black-box AI model, we need to choose a deferral function r ∈ R r \in\mathcal{R} r ∈ R , a signal-presentation rule ϕ ∈ Φ \phi\in\Phi ϕ ∈ Φ , and a collaboration function g ∈ G g \in\mathcal{G} g ∈ G . In a typical supervised learning framework with a fixed prediction loss l ~ \tilde{l} l ~ and a cost function c c c , the goal will be achieving
min h ∈ H E z , y H , y t r u e ℓ ω ( z , y H , y t r u e ; h ) , \min_{h \in\mathcal{H}} \mathbb{E}_{z,y_H,y_{\mathrm{true}}}\ell^\omega(z,y_H,y_{\mathrm{true}};h), h ∈ H min E z , y H , y true ℓ ω ( z , y H , y true ; h ) ,
The expectation is taken under the distribution induced by the candidate signal presentation rule ϕ \phi ϕ , whose effect on human responses is unknown, while the context distribution is held fixed throughout.
Our ultimate goal: omnimechanism design for a loss family. As motivated in the introduction, in human-AI collaboration, in many cases, neither the prediction loss l ~ \tilde{l} l ~ nor the querying cost c c c is fixed or known a priori , as both depend on the downstream task, resource constraints, and deployment context. To address this, we adopt an omniprediction perspective [18 , 19 , 13 ] (Dwork et al. 2021) and seek a mechanism design that simultaneously performs well across a class of losses l ~ \tilde{l} l ~ and cost functions c c c without retraining, ensuring robustness and broad applicability without needing to tailor the model to a specific downstream objective.
Let us consider a family of prediction losses L \mathcal{L} L and a family of query-cost functions C \mathcal{C} C , and let Ω = L × C \Omega= \mathcal{L} \times\mathcal{C} Ω = L × C . We further denote Δ ( T ) \Delta(T) Δ ( T ) as the set of probability distributions over a set T T T . Inspired by the omniprediction theory [10 , 18 ] , we learn a base predictive distribution p : Z × { 0 , 1 } × S × G → Δ ( Y × Y t r u e ) p : \mathcal{Z} \times\{0,1\} \times\mathcal{S} \times\mathcal{G} \to\Delta(\mathcal{Y} \times\mathcal{Y}_{\mathrm{true}}) p : Z × { 0 , 1 } × S × G → Δ ( Y × Y true ) modeling the joint distribution of the resulting final prediction and the ground-truth label, such that, for any pair ω = ( l ~ , c ) ∈ Ω \omega= (\tilde{l},c) \in\Omega ω = ( l ~ , c ) ∈ Ω of loss and cost, there is a data-free post-processing map k ω k^\omega k ω satisfying [ k ω ( p ) ] ( z ) = ( r ω ( z ) , ϕ ω ( z ) , g ω ( z , ⋅ , ⋅ ) ) [k^\omega(p)](z) = (r^\omega(z),\phi^\omega(z),g^\omega(z,\cdot,\cdot)) [ k ω ( p )] ( z ) = ( r ω ( z ) , ϕ ω ( z ) , g ω ( z , ⋅ , ⋅ )) , which maps from Z \mathcal{Z} Z to { 0 , 1 } × S × G \{0,1\} \times\mathcal{S} \times\mathcal{G} { 0 , 1 } × S × G (i.e., a tuple specifying a deferral decision, signaling strategy, and a z z z -dependent collaboration function), thereby defining
ℓ ω ( z , y H , y t r u e ; k ω ( p ) ) = ( 1 − r ω ( z ) ) l ~ ( y M , y t r u e ) + r ω ( z ) [ l ~ ( g ω ( z , y M , y H ) , y t r u e ) + c ( x ) ] , \ell^\omega(z,y_H,y_{\mathrm{true}};k^\omega(p)) = (1-r^\omega(z))\tilde{l}(y_M,y_{\mathrm{true}}) + r^\omega(z)\left[\tilde{l}(g^\omega(z,y_M,y_H),y_{\mathrm{true}}) + c(x)\right], ℓ ω ( z , y H , y true ; k ω ( p )) = ( 1 − r ω ( z )) l ~ ( y M , y true ) + r ω ( z ) [ l ~ ( g ω ( z , y M , y H ) , y true ) + c ( x ) ] ,
where human output y H y_H y H implicitly depends on ϕ ω ( z ) \phi^\omega(z) ϕ ω ( z ) . We highlight a notational distinction: while any base collaboration function g ∈ G g \in\mathcal{G} g ∈ G operates exclusively on the predictions, mapping Y × Y \mathcal{Y} \times\mathcal{Y} Y × Y to Y \mathcal{Y} Y , our post-processing map extracts the collaboration function rule pointwise using the base predictive distribution p ( z ) p(z) p ( z ) for each state z z z . Therefore, for each context z z z , the selected collaboration function g ω ( z , ⋅ , ⋅ ) g^\omega(z,\cdot,\cdot) g ω ( z , ⋅ , ⋅ ) belongs to G \mathcal{G} G while the selected function may vary with z z z .
Loosely speaking, for any ϵ > 0 \epsilon> 0 ϵ > 0 , we say that p p p is an ( Ω , H , ϵ ) (\Omega,\mathcal{H},\epsilon) ( Ω , H , ϵ ) -collaborative omnimechanism if for any loss-cost pair ω ∈ Ω \omega\in\Omega ω ∈ Ω ,
E z , y H , y t r u e ℓ ω ( z , y H , y t r u e ; k ω ( p ) ) ≤ min h ∈ H E z , y H , y t r u e ℓ ω ( z , y H , y t r u e ; h ) + ϵ . \mathbb{E}_{z,y_H,y_{\mathrm{true}}}\ell^\omega(z,y_H,y_{\mathrm{true}};k^\omega(p)) \leq\min_{h \in\mathcal{H}}\mathbb{E}_{z,y_H,y_{\mathrm{true}}}\ell^\omega(z,y_H,y_{\mathrm{true}};h) + \epsilon. E z , y H , y true ℓ ω ( z , y H , y true ; k ω ( p )) ≤ h ∈ H min E z , y H , y true ℓ ω ( z , y H , y true ; h ) + ϵ .
The guarantee compares the induced policy with the best mechanism in H \mathcal{H} H . The induced policy can be more expressive because it may choose a different aggregation rule for each context. We remark that the left hand side expectation uses the response distribution induced by the signal-presentation rule ϕ ω \phi^\omega ϕ ω while each right-hand expectation uses the distribution induced by the corresponding prescribed base policy’s signal-presentation rule.
Our goal is to develop an omnimechanism that can simultaneously achieve approximate optimality across different choices of l ~ \tilde{l} l ~ and c c c through simple, data-free post-processing, without requiring retraining. For clarity, this section describes deterministic post-processing. § 3.1 introduces a smoothed randomized version and Theorem 4.1 quantifies the additional approximation error. Building on the omniprediction framework [Gopalan et al. 2021; Kim and [20 ] (Perdomo 2022), we will develop a sample-efficient boosting algorithm for the smoothed collaborative omnimechanism construction.
Methodology
To formally tackle the omnimechanism design objective, we must address the fundamental challenge of our setting: the impressionability of human behavior. Because the human output y H y_{\mathrm{H}} y H shifts based on the signal s s s they are shown, we recast our framework in the language of contextual bandits. Building on outcome indistinguishability [12 , 24 ] , we employ a boosting algorithm to construct the base predictive distribution p p p , thereby validating the omnimechanism. We then show how to post-process p p p in a data-free manner to obtain near-optimality guarantees.
Policy Learning and State-Action Space
We treat omnimechanism design as a policy learning problem. We first learn an outcome model from offline human-AI interaction data. At deployment, we use this model to evaluate the expected total losses of candidate actions under a supplied prediction loss and human-query cost, and obtain a collaboration policy by post-processing without retraining. We express this procedure in the language of contextual bandits, with the policies π h \pi_h π h induced by mechanisms h ∈ H h \in\mathcal{H} h ∈ H serving as the comparison class.
For that purpose, we treat z = ( x , y M ) z = (x,y_{\mathrm{M}}) z = ( x , y M ) as the state context and consider the action space A : = { 0 , 1 } × S × G \mathcal{A} := \{0,1\} \times\mathcal{S} \times\mathcal{G} A := { 0 , 1 } × S × G . In addition, we define the (augmented) output space as O : = Y × Y t r u e \mathcal{O} := \mathcal{Y} \times\mathcal{Y}_{\mathrm{true}} O := Y × Y true . Now, let us illustrate that given z z z , how each action a ∈ A a \in\mathcal{A} a ∈ A and outcome o = ( y f i n a l , y t r u e ) ∈ O o = (y_{\mathrm{final}},y_{\mathrm{true}}) \in\mathcal{O} o = ( y final , y true ) ∈ O are connected. With a slight abuse of notation, given z z z , for an action a = ( r , s , g ) ∈ A a = (r,s,g) \in\mathcal{A} a = ( r , s , g ) ∈ A 1 , it leads to the final augmented outcome o = ( y f i n a l , y t r u e ) o = (y_{\mathrm{final}},y_{\mathrm{true}}) o = ( y final , y true ) in the following way: if r = 0 r = 0 r = 0 , it leads to the final decision y f i n a l = y M y_{\mathrm{final}} = y_{\mathrm{M}} y final = y M ; if r = 1 r = 1 r = 1 , the signal s s s is presented to the human, and the final decision is given by y f i n a l = g ( y M , y H ) y_{\mathrm{final}} = g(y_{\mathrm{M}},y_{\mathrm{H}}) y final = g ( y M , y H ) . Crucially, we need to keep in mind that y H y_{\mathrm{H}} y H is implicitly dependent on s s s (i.e., the impressionability of human minds). For a pair ω = ( l ~ , c ) \omega= (\tilde{l},c) ω = ( l ~ , c ) of prediction error loss l ~ ∈ L \tilde{l} \in\mathcal{L} l ~ ∈ L and cost c ∈ C c \in\mathcal{C} c ∈ C , we denote the loss value at state z z z under action a = ( r , s , g ) a = (r,s,g) a = ( r , s , g ) and outcome o o o as:
ℓ a ω ( z , o ) = ( 1 − r ) l ~ ( y M , y t r u e ) + r [ l ~ ( g ( y M , y H ) , y t r u e ) + c ( x ) ] = l ~ ( o ) + r c ( x ) , \ell_a^\omega(z,o) = (1-r)\tilde{l}(y_{\mathrm{M}},y_{\mathrm{true}}) + r\left[\tilde{l}(g(y_{\mathrm{M}},y_{\mathrm{H}}),y_{\mathrm{true}}) + c(x)\right] = \tilde{l}(o) + rc(x), ℓ a ω ( z , o ) = ( 1 − r ) l ~ ( y M , y true ) + r [ l ~ ( g ( y M , y H ) , y true ) + c ( x ) ] = l ~ ( o ) + r c ( x ) ,
where the last equation follows the definition of y f i n a l y_{\mathrm{final}} y final and enables us to unify the outcome in automation and deferral processes.
In the following experimental and theoretical analysis, we assume finite action and outcome spaces; the context space need not be finite. For continuous-valued prediction problems, a finite approximation can be obtained by restricting candidate signals and collaboration functions to finite sets and discretizing the continuous coordinates of the outcome pair ( y f i n a l , y t r u e ) (y_{\mathrm{final}},y_{\mathrm{true}}) ( y final , y true ) into bins. We define a policy as a map π : Z → Δ A \pi: \mathcal{Z} \to\Delta\mathcal{A} π : Z → Δ A . We also define a base predictive distribution as a map p : Z × A → Δ ( O ) p : \mathcal{Z} \times\mathcal{A} \to\Delta(\mathcal{O}) p : Z × A → Δ ( O ) , and let p ∗ : Z × A → Δ ( O ) p^* : \mathcal{Z} \times\mathcal{A} \to\Delta(\mathcal{O}) p ∗ : Z × A → Δ ( O ) denote the true data-generating base predictive distribution. We remark that, as a special case, each h = ( r , ϕ , g ) ∈ H h = (r,\phi,g) \in\mathcal{H} h = ( r , ϕ , g ) ∈ H can induce a deterministic base policy π h \pi_h π h by setting π h ( z ) = ( r ( z ) , ϕ ( z ) , g ( ⋅ , ⋅ ) ) \pi_h(z) = (r(z),\phi(z),g(\cdot,\cdot)) π h ( z ) = ( r ( z ) , ϕ ( z ) , g ( ⋅ , ⋅ )) where g g g is globally applied to all z z z . We identify deterministic policies with point-mass distributions when writing π ( a ∣ z ) \pi(a \mid z) π ( a ∣ z ) . Then under a pair ω = ( l ~ , c ) \omega= (\tilde{l}, c) ω = ( l ~ , c ) , the typical supervised learning objective can be rewritten in the following form:
min h ∈ H E z , y H , y t r u e ℓ ω ( z , y H , y t r u e ; h ) = min h ∈ H E z E o ∼ p ∗ ( ⋅ ∣ z , π h ( z ) ) ℓ π h ( z ) ω ( z , o ) . \min_{h \in\mathcal{H}} \mathbb{E}_{z,y_H,y_{\mathrm{true}}}\ell^\omega(z,y_H,y_{\mathrm{true}};h) = \min_{h \in\mathcal{H}} \mathbb{E}_{z}\mathbb{E}_{o \sim p^*(\cdot\mid z,\pi_h(z))}\ell^{\omega}_{\pi_h(z)}(z,o). h ∈ H min E z , y H , y true ℓ ω ( z , y H , y true ; h ) = h ∈ H min E z E o ∼ p ∗ ( ⋅ ∣ z , π h ( z )) ℓ π h ( z ) ω ( z , o ) .
For given ε > 0 \varepsilon> 0 ε > 0 , our ε \varepsilon ε -omnimechanism design is equivalent to searching for a base predictive distribution p p p that enables a possibly randomized data-free post-processing π ω : = k ω ( p ) \pi^\omega:= k^\omega(p) π ω := k ω ( p ) such that the following near-optimal guarantee holds:
E z E a ∼ π ω ( ⋅ ∣ z ) E o ∼ p ( ⋅ ∣ z , a ) ℓ a ω ( z , o ) ≤ min h ∈ H E z E o ∼ p ∗ ( ⋅ ∣ z , π h ( z ) ) ℓ π h ( z ) ω ( z , o ) + ε . \mathbb{E}_{z}\mathbb{E}_{a \sim\pi^\omega(\cdot\mid z)}\mathbb{E}_{o \sim p(\cdot\mid z,a)}\ell^\omega_a(z,o) \leq\min_{h \in\mathcal{H}}\mathbb{E}_{z}\mathbb{E}_{o \sim p^*(\cdot\mid z,\pi_h(z))}\ell^\omega_{\pi_h(z)}(z,o) + \varepsilon. E z E a ∼ π ω ( ⋅ ∣ z ) E o ∼ p ( ⋅ ∣ z , a ) ℓ a ω ( z , o ) ≤ h ∈ H min E z E o ∼ p ∗ ( ⋅ ∣ z , π h ( z )) ℓ π h ( z ) ω ( z , o ) + ε .
To constructively instantiate this scheme with provable efficiency guarantees, we propose the following post-processing rule. We introduce and fix an inverse-temperature parameter τ > 0 \tau> 0 τ > 0 . Let P = { p : Z × A → Δ ( O ) } \mathcal{P} = \{p : \mathcal{Z} \times\mathcal{A} \to\Delta(\mathcal{O})\} P = { p : Z × A → Δ ( O )} denote the space of base predictive distributions, for p ∈ P p \in\mathcal{P} p ∈ P , we write p ( ⋅ ∣ z , a ) p(\cdot\mid z,a) p ( ⋅ ∣ z , a ) the predicted outcome distribution given state z z z and action a a a . Specifically, for any given loss-cost pair ω ∈ Ω \omega\in\Omega ω ∈ Ω , we define the post-processing procedure as the functional mapping k ω : P → Δ ( A ) Z k^\omega: \mathcal{P} \to\Delta(\mathcal{A})^\mathcal{Z} k ω : P → Δ ( A ) Z , such that for a given state z z z and action a a a , its probability mass at ( z , a ) (z,a) ( z , a ) is:
( k ω ( p ) ) z , a : = exp ( − τ E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ) ∑ a ′ ∈ A exp ( − τ E o ∼ p ( ⋅ ∣ z , a ′ ) [ ℓ a ′ ω ( z , o ) ] ) . (1) (k^\omega(p))_{z,a} := \frac{\exp\left(-\tau\mathbb{E}_{o \sim p(\cdot\mid z,a)}[\ell^\omega_a(z,o)]\right)}{\sum_{a' \in\mathcal{A}} \exp\left(-\tau\mathbb{E}_{o \sim p(\cdot\mid z,a')}[\ell^\omega_{a'}(z,o)]\right)}. \tag*{(1)} ( k ω ( p ) ) z , a := ∑ a ′ ∈ A exp ( − τ E o ∼ p ( ⋅ ∣ z , a ′ ) [ ℓ a ′ ω ( z , o )] ) exp ( − τ E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] ) . (1)
Given a state z z z , the probability of taking action a a a is defined to be π ω ( a ∣ z ) : = ( k ω ( p ) ) z , a \pi^\omega(a \mid z) := (k^\omega(p))_{z,a} π ω ( a ∣ z ) := ( k ω ( p ) ) z , a . When τ = ∞ \tau= \infty τ = ∞ , k ω ( p ) k^\omega(p) k ω ( p ) is a degenerate distribution at each state z z z given a fixed tie-breaking rule and the induced policy π ω ( ⋅ ∣ z ) \pi^\omega(\cdot\mid z) π ω ( ⋅ ∣ z ) deterministically takes loss-minimizing action, which recovers the post-processing method in our setup as a special case. Taking τ < ∞ \tau< \infty τ < ∞ enables us to prove time-efficiency result, bypassing the hardness of performative decision-calibration in Kim and Perdomo [ [24 ] .
By relying entirely on the simulated loss via base predictive distribution p p p , this formulation enables data-free adaptation to any prescribed downstream cost-loss pair without accessing fresh samples. With this explicit post-processing mechanism in place, we can now formally characterize the conditions under which a base predictive distribution satisfies the requirements of our omnimechanism.
Our Algorithm
To algorithmically construct a collaborative omnimechanism, we extend the concept of outcome indistinguishability in [12 , 24 ] to our human-AI collaboration setting, and introduce a boosting algorithm to construct a base predictive distribution p p p that achieves the omnimechanism.
Collaborative outcome indistinguishability (COI). A key property for obtaining a collaborative omnimechanism is what we call collaborative outcome indistinguishability.
Definition 3.1 (Collaborative Outcome Indistinguishability). A base predictive distribution p p p satisfies ( Ω , H , ε ) (\Omega, \mathcal{H}, \varepsilon) ( Ω , H , ε ) -Collaborative Outcome Indistinguishability (COI) if the following conditions hold:
(a). (Hypothesis-OI) For each loss-cost pair ω ∈ Ω \omega\in\Omega ω ∈ Ω , and deterministic base collaboration policy π h \pi_h π h induced by h ∈ H h \in\mathcal{H} h ∈ H , the simulated expected loss under p p p approximately matches the true expected loss under p ∗ p^* p ∗ :
∣ E z E o ∼ p ∗ ( ⋅ ∣ z , π h ( z ) ) [ ℓ π h ( z ) ω ( z , o ) ] − E z E o ∼ p ( ⋅ ∣ z , π h ( z ) ) [ ℓ π h ( z ) ω ( z , o ) ] ∣ ≤ ε \left|\mathbb{E}_{z}\mathbb{E}_{o \sim p^*(\cdot\mid z,\pi_h(z))}\left[\ell^{\omega}_{\pi_h(z)}(z,o)\right] - \mathbb{E}_{z}\mathbb{E}_{o \sim p(\cdot\mid z,\pi_h(z))}\left[\ell^{\omega}_{\pi_h(z)}(z,o)\right]\right| \leq\varepsilon E z E o ∼ p ∗ ( ⋅ ∣ z , π h ( z )) [ ℓ π h ( z ) ω ( z , o ) ] − E z E o ∼ p ( ⋅ ∣ z , π h ( z )) [ ℓ π h ( z ) ω ( z , o ) ] ≤ ε
(b). (Decision-OI) For each loss-cost pair ω ∈ Ω \omega\in\Omega ω ∈ Ω , given the randomized post-processing policy π ω \pi^\omega π ω defined in Eq. (1) , the simulated expected loss approximately matches the true expected loss:
∣ E z E a ∼ π ω ( ⋅ ∣ z ) E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] − E z E a ∼ π ω ( ⋅ ∣ z ) E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ∣ ≤ ε \left|\mathbb{E}_{z}\mathbb{E}_{a \sim\pi^\omega(\cdot\mid z)}\mathbb{E}_{o \sim p^*(\cdot\mid z,a)}\left[\ell^\omega_a(z,o)\right] - \mathbb{E}_{z}\mathbb{E}_{a \sim\pi^\omega(\cdot\mid z)}\mathbb{E}_{o \sim p(\cdot\mid z,a)}\left[\ell^\omega_a(z,o)\right]\right| \leq\varepsilon E z E a ∼ π ω ( ⋅ ∣ z ) E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] − E z E a ∼ π ω ( ⋅ ∣ z ) E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ≤ ε
COI requires approximate agreement of expected losses under the specified policies and loss–cost pairs, without requiring pointwise recovery of the true outcome distribution p ∗ ( ⋅ ∣ z , a ) p^*(\cdot\mid z,a) p ∗ ( ⋅ ∣ z , a ) . Crucially, satisfying the COI condition is sufficient to guarantee that a base predictive distribution p p p acts as a valid omnimechanism (we defer the formal proof of this sufficiency to Section B). Therefore, our algorithmic goal reduces to finding a base predictive distribution p p p that satisfies Definition 3.1. Here we provide an intuitive overview of these introduced key objects in our framework and their roles.
Intuitive walkthrough. Four objects connect offline learning and the COI-condition to deployment:
True outcome distribution p ∗ ( ⋅ ∣ z , a ) p^*(\cdot\mid z,a) p ∗ ( ⋅ ∣ z , a ) . This describes the outcome o = ( y f i n a l , y t r u e ) o = (y_{\mathrm{final}}, y_{\mathrm{true}}) o = ( y final , y true ) generated by action a a a in context z z z , including the effect of the displayed signal on human responses.
Logging policy π log ( ⋅ ∣ z ) \pi_{\log}(\cdot\mid z) π l o g ( ⋅ ∣ z ) . This determines which actions are taken during data collection. Offline observations follow
z ∼ D Z , a ∼ π log ( ⋅ ∣ z ) , o ∼ p ∗ ( ⋅ ∣ z , a ) , z \sim D_Z,\qquad a \sim\pi_{\log}(\cdot\mid z),\qquad o \sim p^*(\cdot\mid z,a), z ∼ D Z , a ∼ π l o g ( ⋅ ∣ z ) , o ∼ p ∗ ( ⋅ ∣ z , a ) ,
where D Z D_Z D Z is the fixed context distribution. Sufficient logging coverage allows us to estimate the expected losses of candidate policies.
Learned predictive distribution p ( ⋅ ∣ z , a ) p(\cdot\mid z,a) p ( ⋅ ∣ z , a ) . COI-Boost constructs a reusable surrogate outcome model, updating it when an auditor detects a discrepancy between simulated and true expected losses. COI requires agreement for the tested policies and loss-cost pairs, without requiring pointwise recovery of p ∗ p^* p ∗ .
Deployed policy π ω = k ω ( p ) \pi^\omega= k^\omega(p) π ω = k ω ( p ) . Given a supported loss-cost pair ω = ( l ~ , c ) \omega= (\tilde{l}, c) ω = ( l ~ , c ) , Eq. (1) converts the predicted action losses into a randomized policy. Changing ω \omega ω requires only this post-processing step, without retraining p p p or collecting new outcome samples; the context distribution and action-conditional outcome law remain fixed.
The near-optimality argument shall use the following three comparisons: First, Decision-OI transfers the deployed policy’s true expected loss to its simulated loss under p p p ; Second, the softmax rule makes this simulated loss close to the minimum, and hence no greater than that of any comparator π h \pi_h π h , h ∈ H h \in\mathcal{H} h ∈ H , up to the smoothing error; Finally, Hypothesis-OI then transfers the comparator’s simulated loss back to its true loss. The two COI conditions contribute at most 2 ε 2\varepsilon 2 ε , while smoothing contributes at most log ∣ A ∣ / τ \log|\mathcal{A}|/\tau log ∣ A ∣/ τ . Below, we present our algorithmic approach to produce a base predictive distribution satisfying the COI condition.
COI-boosting algorithm. We present here our algorithm assuming we have a reliable auditor, which we call a COI-auditor that defined below. To validate this auditor in practice, we assume access to an offline dataset collected via a fixed logging policy, denoted as π l o g \pi_{\mathrm{log}} π log . We will defer the details of data collection using π l o g \pi_{\mathrm{log}} π log and construction of such a reliable auditor in § 4 .
Definition 3.2 (COI Auditor). Fix ε > 0 \varepsilon> 0 ε > 0 , δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) , and the data collection policy π l o g \pi_{\mathrm{log}} π log . An ( Ω , H , ε , δ , m ) (\Omega,\mathcal{H},\varepsilon,\delta,m) ( Ω , H , ε , δ , m ) -COI auditor takes as input a base predictive distribution p p p and m m m independent fresh samples { ( z i , a i , o i ) } i = 1 m \{(z_i,a_i,o_i)\}_{i=1}^{m} {( z i , a i , o i ) } i = 1 m drawn using π l o g \pi_{\mathrm{log}} π log that are independent of p p p . It outputs either ⊥ \perp ⊥ or a certificate ( π , ω , σ ) (\pi,\omega,\sigma) ( π , ω , σ ) , where ω ∈ Ω \omega\in\Omega ω ∈ Ω , σ ∈ { − 1 , 1 } \sigma\in\{-1,1\} σ ∈ { − 1 , 1 } , and π \pi π is either a base policy π h \pi_h π h for some h ∈ H h \in\mathcal{H} h ∈ H or the post-processed policy π ω \pi^\omega π ω , such that: with probability at least 1 − δ 1-\delta 1 − δ , the following guarantees hold:
If the auditor returns ( π , ω , σ ) (\pi,\omega,\sigma) ( π , ω , σ ) , then
σ E z E a ∼ π ( ⋅ ∣ z ) [ E o ∼ p ω ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] − E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ] > ε 2 . \sigma\mathbb{E}_{z}\mathbb{E}_{a\sim\pi(\cdot\mid z)}\left[\mathbb{E}_{o\sim p^\omega(\cdot\mid z,a)}[\ell_a^\omega(z,o)]-\mathbb{E}_{o\sim p(\cdot\mid z,a)}[\ell_a^\omega(z,o)]\right] > \frac{\varepsilon}{2}. σ E z E a ∼ π ( ⋅ ∣ z ) [ E o ∼ p ω ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] − E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] ] > 2 ε . If the auditor returns ⊥ \perp ⊥ , then p p p satisfies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI.
In particular, if p p p does not satisfy ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI, the auditor must return a certificate on this event. However, when p p p satisfies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI but not ( Ω , H , ε / 2 ) (\Omega,\mathcal{H},\varepsilon/2) ( Ω , H , ε /2 ) -COI, either valid output is permitted.
Algorithm 1 COI-Boosting Algorithm
Given H \mathcal{H} H , L \mathcal{L} L , C \mathcal{C} C , ε \varepsilon ε , π l o g \pi_{\mathrm{log}} π log , δ \delta δ , and a ( Ω , H , ε , δ , m ) (\Omega,\mathcal{H},\varepsilon,\delta,m) ( Ω , H , ε , δ , m ) -COI auditor. Set learning rate η \eta η , maximal iteration round T max T_{\max} T m a x and the initial base predictive distribution p ( 0 ) ( o ∣ z , a ) = 1 / ∣ O ∣ p^{(0)}(o \mid z,a) = 1/|\mathcal{O}| p ( 0 ) ( o ∣ z , a ) = 1/∣ O ∣ for all ( z , a , o ) (z,a,o) ( z , a , o ) .
for 0 ≤ t < T max 0 \le t < T_{\max} 0 ≤ t < T m a x do
Draw m m m -samples { ( z i , a i , o i ) } i = 1 m \{(z_i,a_i,o_i)\}_{i=1}^{m} {( z i , a i , o i ) } i = 1 m and call the ( Ω , H , ε , δ , m ) (\Omega,\mathcal{H},\varepsilon,\delta,m) ( Ω , H , ε , δ , m ) -COI auditor with p ( t ) p^{(t)} p ( t ) .
if the auditor outputs ⊥ \perp ⊥ then Terminate.
else
The auditor outputs ( π ( t ) , ω ( t ) , σ ) (\pi^{(t)},\omega^{(t)},\sigma) ( π ( t ) , ω ( t ) , σ ) . Set g ( t ) ( z , a , o ) = σ π ( t ) ( a ∣ z ) π l o g ( a ∣ z ) ℓ a ω ( t ) ( z , o ) g^{(t)}(z,a,o) = \sigma\frac{\pi^{(t)}(a\mid z)}{\pi_{\mathrm{log}}(a\mid z)}\ell_a^{\omega^{(t)}}(z,o) g ( t ) ( z , a , o ) = σ π log ( a ∣ z ) π ( t ) ( a ∣ z ) ℓ a ω ( t ) ( z , o ) ;
Update the base predictive distribution with learning rate η \eta η ,
p ( t + 1 ) ( o ∣ z , a ) = p ( t ) ( o ∣ z , a ) exp ( η g ( t ) ( z , a , o ) ) ∑ o ′ ∈ O p ( t ) ( o ′ ∣ z , a ) exp ( η g ( t ) ( z , a , o ′ ) ) . p^{(t+1)}(o \mid z,a) =
\frac{p^{(t)}(o \mid z,a)\exp(\eta g^{(t)}(z,a,o))}
{\sum_{o'\in\mathcal{O}}p^{(t)}(o' \mid z,a)\exp(\eta g^{(t)}(z,a,o'))}. p ( t + 1 ) ( o ∣ z , a ) = ∑ o ′ ∈ O p ( t ) ( o ′ ∣ z , a ) exp ( η g ( t ) ( z , a , o ′ )) p ( t ) ( o ∣ z , a ) exp ( η g ( t ) ( z , a , o )) . end if
end for
return p ( t ) p^{(t)} p ( t ) if it terminates or reaches the maximal iteration round t = T max t = T_{\max} t = T m a x .
Table 1 summarizes the main notation. Throughout, Δ ( S ) \Delta(S) Δ ( S ) denotes the set of probability distributions over a set S S S .
Table 1. Main notation for collaborative omnimechanism design.
Theoretical Guarantees
In this section, we establish the theoretical foundations of our collaborative omnimechanism framework. We proceed in two steps. First, we prove that satisfying the Collaborative Outcome
Indistinguishability (COI) condition is sufficient to guarantee a valid omnimechanism. Second, we establish sample-complexity bounds for COI auditing and evaluate practical implementations of our framework empirically in § 6 .
We begin by formally establishing the sufficiency of COI. Theorem 4.1 shows that if our boosting algorithm yields a base predictive distribution that is indistinguishable from the true environment under our specified test classes, the resulting post-processed policy will be near-optimal for any downstream loss and cost, and thus guarantees that the base predictive distribution is a valid omnimechanism.
Theorem 4.1 (COI Implies Collaborative Omnimechanism.). If a base predictive distribution p : Z × A → Δ O p:\mathcal{Z}\times\mathcal{A}\to\Delta\mathcal{O} p : Z × A → Δ O satisfies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI (Definition 3.1 ), then it is an ( Ω , H , 2 ε + log ∣ A ∣ τ ) (\Omega,\mathcal{H},2\varepsilon+\frac{\log|\mathcal{A}|}{\tau}) ( Ω , H , 2 ε + τ l o g ∣ A ∣ ) -collaborative omnimechanism. Specifically, for any loss-cost pair ω ∈ Ω \omega\in\Omega ω ∈ Ω , we have:
E z E a ∼ π ω ( ⋅ ∣ z ) E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ≤ min h ∈ H E z E o ∼ p ∗ ( ⋅ ∣ z , π h ( z ) ) [ ℓ π h ( z ) ω ( z , o ) ] + 2 ε + log ∣ A ∣ τ . \mathbb{E}_{z}\mathbb{E}_{a\sim\pi^\omega(\cdot\mid z)}\mathbb{E}_{o\sim p^*(\cdot\mid z,a)}\left[\ell^\omega_a(z,o)\right]\leq\min_{h\in\mathcal{H}}\mathbb{E}_{z}\mathbb{E}_{o\sim p^*(\cdot\mid z,\pi_h(z))}\left[\ell^\omega_{\pi_h(z)}(z,o)\right]+2\varepsilon+\frac{\log|\mathcal{A}|}{\tau}. E z E a ∼ π ω ( ⋅ ∣ z ) E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ≤ h ∈ H min E z E o ∼ p ∗ ( ⋅ ∣ z , π h ( z )) [ ℓ π h ( z ) ω ( z , o ) ] + 2 ε + τ log ∣ A ∣ .
Theorem 4.1 confirms that collaborative omnimechanism can be achieved via a COI base predictive distribution. When τ \tau τ is large, the near-optimality gap becomes negligible. We shall show that the convergence guarantee of our algorithm (Algorithm 1) holds for both τ < ∞ \tau<\infty τ < ∞ and τ = ∞ \tau=\infty τ = ∞ and further validate empirically that our algorithm is efficient even if we take τ = ∞ \tau=\infty τ = ∞ . On the other hand, the convergence of the COI-Boosting algorithm (Algorithm 1) relies on the existence of a reliable and efficient COI-auditor (Definition 3.2 ). To practically operationalize an ( Ω , H , ε , δ , m ) (\Omega,\mathcal{H},\varepsilon,\delta,m) ( Ω , H , ε , δ , m ) -COI auditor, we assume access to a historical dataset of human-AI collaborations. Specifically, given a fixed data collection policy π l o g : Z → Δ A \pi_{\mathrm{log}}:\mathcal{Z}\to\Delta\mathcal{A} π log : Z → Δ A , the samples for the auditor input are drawn via the following process:
z ∼ D Z , a ∼ π l o g ( ⋅ ∣ z ) , o ∼ p ⋆ ( ⋅ ∣ z , a ) , z \sim D_{\mathcal{Z}},\ a \sim\pi_{\mathrm{log}}(\cdot\mid z),\ o \sim p^{\star}(\cdot\mid z,a), z ∼ D Z , a ∼ π log ( ⋅ ∣ z ) , o ∼ p ⋆ ( ⋅ ∣ z , a ) ,
where D Z D_{\mathcal{Z}} D Z denotes the marginal distribution over state z ∈ Z z \in\mathcal{Z} z ∈ Z . In Theorem 4.2 , we show that under standard data coverage and finite complexity assumptions, we can construct such a reliable, sample-efficient auditor. We defer the formal statements of assumptions on data coverage (Assumption B.1) and finiteness of complexity and boundedness for L , C , H \mathcal{L},\mathcal{C},\mathcal{H} L , C , H (Assumption B.2) to the Appendix § 11 .
Theorem 4.2 (Sample Efficiency of Auditor, Informal). Under Assumptions B.1 and B.2, fix τ > 0 \tau> 0 τ > 0 and ε , δ ∈ ( 0 , 1 ) \varepsilon,\delta\in(0,1) ε , δ ∈ ( 0 , 1 ) . For any base predictive distribution p p p independent of the audit batch, there exists an ( Ω , H , ε , δ , m ) (\Omega,\mathcal{H},\varepsilon,\delta,m) ( Ω , H , ε , δ , m ) -COI auditor using
m = O ( ( 1 + B C ) 2 γ 2 ε 2 ( ( d R + d Φ + log ∣ G ∣ ) log ∣ A ∣ + ( 1 + τ ) 2 ∣ O ∣ + log ∣ C ∣ + log ( 1 / δ ) ) ) m = \mathcal{O}\left(\frac{(1+B_{\mathcal{C}})^2}{\gamma^2\varepsilon^2}\left((d_{\mathcal{R}}+d_{\Phi}+\log|\mathcal{G}|)\log|\mathcal{A}|+(1+\tau)^2|\mathcal{O}|+\log|\mathcal{C}|+\log(1/\delta)\right)\right) m = O ( γ 2 ε 2 ( 1 + B C ) 2 ( ( d R + d Φ + log ∣ G ∣ ) log ∣ A ∣ + ( 1 + τ ) 2 ∣ O ∣ + log ∣ C ∣ + log ( 1/ δ ) ) ) independent offline samples collected under π l o g \pi_{\mathrm{log}} π log . With probability at least 1 − δ 1-\delta 1 − δ , the auditor outputs either a valid violation certificate ( π , ω , σ ) (\pi,\omega,\sigma) ( π , ω , σ ) or ⊥ \bot ⊥ , where returning ⊥ \bot ⊥ certifies that p p p satisfies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI. Here γ > 0 \gamma> 0 γ > 0 is the coverage constant for logging policy, B C B_{\mathcal{C}} B C is a uniform bound of the cost functions, and d R d_{\mathcal{R}} d R and d Φ d_{\Phi} d Φ are the Natarajan dimensions of R \mathcal{R} R and Φ \Phi Φ respectively.
This bound reveals that the sample complexity of the auditor scales linearly with the structural complexity of the collaboration policy classes (i.e., d R d_{\mathcal{R}} d R , d Φ d_{\Phi} d Φ , and log ∣ G ∣ \log|\mathcal{G}| log ∣ G ∣ ), which is of the same order as in previous theoretical learning to defer work Mozannar and Sontag [ [30 ] . Coupled with the convergence guarantees of the boosting procedure (Theorem B.5 in Appendix § 11 ), this ensures the sample-efficient construction of a collaborative omnimechanism.
We remark that, beyond introducing COI as a framework for human-AI collaboration with impressionable decision-makers, we establish new auditing guarantees that distinguish our setting from prior omniprediction and performative prediction frameworks. Inspired by Gopalan et al. [ [20 ] , we establish in Appendix § 12 efficient witness-based Decision-OI auditing through a reduction to weighted calibration. This yields an auditor with runtime poly ( ∣ A × O ∣ O ( C ( B , τ ) log 2 ( 1 / ε ) ) ) \operatorname{poly}(|\mathcal{A}\times\mathcal{O}|^{\mathcal{O}(\mathcal{C}(B,\tau)\log^2(1/\varepsilon))}) poly ( ∣ A × O ∣ O ( C ( B , τ ) l o g 2 ( 1/ ε )) ) under uniform logging policy with fixed scalar cost and finite inverse temperature. This runtime is polynomial in the action and outcome space cardinalities for fixed accuracy, and quasipolynomial in inverse accuracy when those cardinalities are fixed, which avoids the exponential dependence on loss-space dimension incurred by direct discretization, achieving polynomial dependence on the action and outcome cardinalities. It thereby addresses, in our smoothed setting, the computational bottleneck highlighted by Kim and Perdomo [ [24 ] for performative decision-calibration. Complementing our per-call auditing upper bound, we establish a sample-complexity lower bound for offline omnimechanism learning: m = Ω ( d H + log ( 1 / δ ) γ ε 2 ) m=\Omega\left(\frac{d_{\mathcal{H}}+\log(1/\delta)}{\gamma\varepsilon^2}\right) m = Ω ( γ ε 2 d H + l o g ( 1/ δ ) ) , this lower bound establishes unavoidable dependence on policy-class complexity, inverse-squared accuracy, and limited logging coverage. Together, these results show that COI is not only a conceptual extension of outcome indistinguishability to human-AI collaboration with impressionable decision-makers, but also admits efficient and nearly tight auditing guarantees.
Extension
We further extend our framework to settings with multiple human experts and feature-based methods, highlighting its flexibility and applicability to more complex human-AI collaboration scenarios. Due to space limitations, we defer the technical details and analysis to the appendix.
Incorporating opinions from multiple humans. In human–AI collaboration, a task may be deferred to one of several human experts with different specialties and query costs [28 , 34 , 32 ] . We consider routing each deferred task to a single expert, whose identity is included in the action. Appendix § 13 explains how our sample-complexity analysis extends to this enlarged action space, with the resulting guarantee depending on the logging coverage and the complexity of the expanded collaboration policy class.
Efficient auditor with feature-based methods. In previous human-AI collaboration frameworks, given a state z = ( x , y M ) z = (x,y_M) z = ( x , y M ) , the action a = ( r , s , g ) ∈ A a = (r,s,g) \in\mathcal{A} a = ( r , s , g ) ∈ A , consisting of the deferral decision, signal and collaboration function, is often determined based solely on the AI output y M y_M y M [35 , 9 , 4 , 10 ] . This can be viewed as a special case of feature-based methods, where we introduce a feature map that projects a state z z z into a low-dimensional feature vector. We show that under standard Lipschitz continuity assumptions, our framework extends to this setting, preserving the sample-efficiency guarantees for the COI-auditor. We defer the theoretical details to Appendix § 13 .
Experiments
We evaluate our framework in three settings. Experiment 1 is our primary experiment, in which we learn omnimechanisms using both COI-Boosting and a neural network and evaluate their optimally post-processed policies and COI violations. Experiment 2 uses empirical outcome distributions from radiologist–AI interaction data, without relying on a behavioral simulator. Experiment 3 extends the evaluation to continuous house-price prediction. Together, these experiments support the central claim that a learned omnimechanism can induce effective input-dependent policies for different downstream objectives. Implementation details and additional results are provided in the appendix.
Experiment 1: Impressionable Human Decisions
Dataset. We conduct experiments on the Art and Sarcasm tasks from the real human-AI interaction dataset of [35 ] , which contains over 30,000 interactions across several binary classification tasks. Each example records a real participant’s initial response, an AI signal, and the participant’s final response after observing the AI advice. Following [35 ] , we fit their two-stage behavioral model as a black-box simulator; as reported in [35 ] , the model was shown to align well with aggregate real human behavior on these tasks. The base predictive distribution is then trained and evaluated only on samples generated by the fitted simulator, without access to its parameters. This setup does not require white-box access to the human-behavior model or the convexity and sensitivity assumptions used in prior work [31 , 27 , 26 ] .
Implementation. We instantiate the omnimechanism in a setting where the downstream cost varies. For each example, the system observes a state z z z 2 that summarizes the pre-interaction information: the human’s initial confidence, the AI’s confidence, and their initial agreement indicator. The system then chooses an action a a a consisting of a deferral decision, a signal, and a collaboration function. The action either directly outputs the AI prediction, or routes the instance to the human with a transformed AI-advice signal [35 ] and combines the human and AI predictions through a linear collaboration function. We treat the initial human judgment as given; cost c c c and human involvement refer only to additional consultation.
For each state-action pair ( z , a ) (z,a) ( z , a ) , we use a trained base predictive distribution p ( ⋅ ∣ z , a ) p(\cdot\mid z,a) p ( ⋅ ∣ z , a ) to estimate the simulated expected loss. Given a pair ω = ( l ~ , c ) \omega= (\tilde{l},c) ω = ( l ~ , c ) of a loss function l ~ \tilde{l} l ~ and a constant human-involvement cost c c c 3 , the optimal post-processing policy chooses π ω ( z ) ∈ arg min a { E p ( ⋅ ∣ z , a ) [ ℓ ω ( z , o ) ] } \pi^\omega(z) \in\arg\min_a \{\mathbb{E}_{p(\cdot\mid z,a)}[\ell^{\omega}(z,o)]\} π ω ( z ) ∈ arg min a { E p ( ⋅ ∣ z , a ) [ ℓ ω ( z , o )]} 4 .
We implement our omnimechanism with the boosting algorithm (Algorithm 1) by discretizing z z z and maintaining an outcome distribution for each discrete state-action pair. To extend this approach to continuous spaces, we further develop a neural base predictive distribution utilizing a three-layer neural network that preserves the continuous nature of z z z . This provides a function-approximation implementation of the same omnimechanism. We present implementation details in Appendix § 14 .
Evaluation and analysis. Table 2 reports accuracy, utility, and human involvement rate for each method under different human cost c c c . We compare our post-processing policies with three global baselines: No Action , which always defers to the human with the original AI advice; Manipulation , which always routes to the human with a globally optimized signaling function ϕ \phi ϕ [35 ] ; and Global Best , which selects the policy in H \mathcal{H} H with the highest overall utility for each cost c c c and holds that policy fixed across inputs. Here, H \mathcal{H} H extends Manipulation with typical deferral functions and human-AI collaboration functions in [9 , 4 ] . More details are presented in Appendix § 14 .§ 6 .
Table 2. Policy performance under human-involvement costs c ∈ { 0 , 0.1 , 0.2 } c \in\{0,0.1,0.2\} c ∈ { 0 , 0.1 , 0.2 } . We report accuracy (Acc.), utility (Util. = Acc. − c ⋅ - c \cdot − c ⋅ Hum.), and human-involvement rate (Hum.). The best and second-best utilities in each setting are shown in bold and <u>underlined</u>, respectively.
The post-processed policies are competitive with Global Best across human-cost regimes. On Art, both implementations recover AI-only behavior at nonzero costs and match Global Best’s utility. On Sarcasm, the neural implementation achieves higher utility at all reported costs, while Boosting also improves upon Global Best at c = 0.1 c = 0.1 c = 0.1 . These results illustrate how a learned omnimechanism can support effective collaboration policies across downstream costs through post-processing.
As cost c c c increases, the post-processing policies π ω \pi^\omega π ω reduce or eliminate deferral, reflecting the trade-off between accuracy and growing human cost. We highlight that this adaptation is obtained through the post-processing of our omnimechanism, rather than by retraining a separate model for each cost parameter. Beyond the original setting, where AI advice accuracy is around 80%, we further evaluate shifted AI-advice settings with average AI accuracy reduced to approximately 70% (Table 5 ) and 60% (Table 6 ) in Appendix § 14 .§ 7 . We retrain the omnimechanisms under each shifted advice distribution; within each setting, the resulting predictors are reused across costs. The omnimechanism-based policies remain competitive with the strongest baselines across these settings, further supporting the effectiveness of our approach under different AI reliability levels.
Table 5: Policy performance after shifting AI advice to approximately 70% accuracy. We report accuracy (Acc.), utility (Util. = Acc. − c c c · Hum.), and human-involvement rate (Hum.). The best and second-best utilities in each setting are shown in bold and underline, respectively.
Table 5.
Table 6: Policy performance after shifting AI advice to approximately 60% accuracy. We report accuracy (Acc.), utility (Util. = Acc. − c c c · Hum.), and human-involvement rate (Hum.). The best and second-best utilities in each setting are shown in bold and underline, respectively.
Table 6.
Convergence of boosting-based omnimechanism. We find that the boosting-based omnimechanism exhibits stable finite-sample training dynamics consistent with our convergence analysis in Theorem B.5 . As shown in Figure 2 , the KL divergence from the smoothed simulator target to the learned predictive distribution decreases over boosting rounds. The maximum empirical COI violation also approaches the tolerance ε = 0.05 \varepsilon= 0.05 ε = 0.05 . These diagnostics show improved distributional alignment and agreement in expected loss over the audited policies and costs.
Figure 2. Training dynamics of the boosting-based omnimechanism over boosting rounds.
Experiment 2: Radiologist–AI Collaboration
Data and setting. We use the Design 1 subset of Collab-CXR, comprising probabilistic assessments by 159 professional radiologists on 324 chest X-rays under randomized information conditions [29 , 1 ] . We focus on Airspace Opacity and Cardiomediastinal Abnormality, the two pre-registered diagnostic categories in the original study. The action set consists of direct AI prediction (AI), unaided radiologist assessment (H), and AI-assisted radiologist assessment (AI+H), matching the main collaboration modes considered in the original study.
Implementation and evaluation. Following Experiment 1, we learn an action-conditional four-state predictive distribution and obtain a policy by deterministic expected-loss-minimizing post-processing. The training targets are constructed directly from the observed AI and radiologist probabilities. The model receives a MedSigLIP-448 X-ray embedding Sellergren et al. [ [33 ] , the AI probability, the pathology, and the candidate action. Following Agarwal et al. [ [1 ] , we evaluate diagnostic probabilities by their average deviation from the diagnostic standard. For a reported diagnostic probability ρ ∈ [ 0 , 1 ] \rho\in[0,1] ρ ∈ [ 0 , 1 ] and binary diagnostic-standard label y ∈ { 0 , 1 } y \in\{0,1\} y ∈ { 0 , 1 } , the utility of action a a a at human cost c c c is u c ( ρ , y , a ) = 1 − ∣ ρ − y ∣ − c 1 { a ≠ A I } u_c(\rho,y,a)=1-|\rho-y|-c\mathbf{1}\{a\ne\mathrm{AI}\} u c ( ρ , y , a ) = 1 − ∣ ρ − y ∣ − c 1 { a = AI } . We use a patient-level split with 259 training and 65 test X-rays, with model selection performed using only the training set. Post-processing selects the action with the highest estimated expected utility. For comparison, Global Best selects on the training set a single action for each c c c and applies it to all test inputs across both pathologies. Further details are given in Appendix § 15 .
Results. Table 3 reports diagnostic score, human-involvement rate, and utility separately for each pathology. The post-processed policy improves upon Global Best at low and moderate human costs and recovers the AI-only policy as the cost increases. When human input is free, it improves utility over Global Best for both Airspace Opacity (+0.024) and Cardiomediastinal Abnormality (+0.025); the overall gain is 0.025. At c = 0.02 c=0.02 c = 0.02 , it adapts differently across pathologies, retaining substantially more human involvement for Airspace Opacity while improving utility for both. At c = 0.05 c=0.05 c = 0.05 , the remaining gain is concentrated in Airspace Opacity. For c ∈ { 0.10 , 0.20 } c\in\{0.10,0.20\} c ∈ { 0.10 , 0.20 } , the same trained predictor selects AI for every test input and exactly matches Global Best . Averaged across pathologies, the utility improvements are 0.028 at c = 0.02 c=0.02 c = 0.02 and 0.010 at c = 0.05 c=0.05 c = 0.05 .
Table 3. Collab-CXR test performance by pathology. Score is 1 − ∣ ρ − y ∣ 1-|\rho-y| 1 − ∣ ρ − y ∣ and Util. = Score − c ⋅ -c\cdot − c ⋅ Hum.. Global Best is selected overall on training data, not separately by pathology.
Experiment 3: AI-Assisted House-Price Prediction
Data and setting. To complement the classification settings in Experiments 1 and 2, we use a simulated experiment based on the AMES AI-assisted continuous-valued house-price prediction dataset introduced by [22 ] . The released data contain 5,244 paired initial and assisted judgments from 437 participants on 240 houses. The action set has nine total actions, consisting of directly using the initial human estimate (H 0 \mathrm{H}_0 H 0 ), either of two AI predictions (A I 1 \mathrm{AI}_1 AI 1 and A I 2 \mathrm{AI}_2 AI 2 ), or requesting a human revision under one of six assistance conditions proposed by [22 ] : either AI model paired with no explanation, SHAP, or ILIV+SHAP. The decision is made after observing the initial human estimate, and the human cost applies only to an additional revision.
Implementation and evaluation. Following the NN method in Experiments 1 and 2, we learn an action-conditional predictive distribution and obtain a policy by deterministic expected-loss-minimizing post-processing. We represent outcomes by 13 bins of signed relative prediction error. Training targets are one-hot for the three observed direct outputs and empirical bin distributions for simulated human revisions. The behavioral simulator is fitted to the log-relative human update and generates responses by sampling prediction residuals. The omnimechanism receives six house features, the initial human estimate, both AI predictions, and the candidate action; the sale price is used only to construct training targets and evaluate losses. Following [22 ] , for a final estimate v v v and sale price y > 0 y > 0 y > 0 , we use absolute percentage error (APE), d A P E ( v , y ) = ∣ ( v − y ) / y ∣ d_{\mathrm{APE}}(v,y) = \lvert(v-y)/y\rvert d APE ( v , y ) = ∣( v − y ) / y ∣ , with total loss d A P E ( v , y ) + c 1 { a ∈ A 1 } d_{\mathrm{APE}}(v,y) + c\mathbf{1}\{a \in A_1\} d APE ( v , y ) + c 1 { a ∈ A 1 } , where A 1 A_1 A 1 is the set of six revision actions. We remark that the policy uses bin representatives to estimate losses when selecting actions; evaluation uses the continuous prediction errors. Appendix G provides the discretization, logged-response evaluation, and sensitivity analyses.
Results. Table 4 reports continuous APE, additional human-revision rate, and total loss. The post-processed policy improves upon Global Best throughout the reported cost range and reduces additional human involvement as the cost increases. At zero cost, the NN policy reduces APE from 0.1335 to 0.1053, while requesting revisions on 2.78% of contexts. As cost increases, additional human involvement declines, reaching zero at c = 0.10 c = 0.10 c = 0.10 . The policy retains gains at this cost by choosing among the initial human estimate and the two AI predictions, whereas Global Best uses AI2 _2 2 throughout. These results illustrate our framework’s ability to adapt to the human-involvement cost without retraining.
Table 4. AMES simulator test performance on 1,044 contexts from 87 held-out participants (seed 42). Error is continuous APE, Rev. is the fraction of contexts requesting an additional human revision, and Total = Error+c ⋅ c\cdot c ⋅ Rev.. Positive Δ \Delta Δ Loss indicates lower total loss for Ours. Global Best is selected separately for each cost on training data.
Conclusion
In this work, we present a novel collaborative omnimechanism framework that accounts for signal-dependent human responses and adapts collaboration policies to different deployment-time prediction losses and human-query costs. Unlike prior approaches that assume fixed human behavior or optimize a single objective, our method learns a unified predictive model that can be post-processed to adapt to different query costs and evaluation criteria without retraining. The framework may improve decision quality and reduce unnecessary human workload in high-stakes domains, but it also risks manipulating human judgments or amplifying overreliance on AI, requiring transparency, auditing, and safeguards against misuse. Our evaluation uses existing interaction datasets, and two experiments rely partly on fitted behavioral simulators. The results therefore depend on simulator fidelity, and selected human-revision interventions can have sparse logged support. The finite intervention menus also leave richer forms of presentation and interaction unexplored.
Acknowledgements
Vahid Tarokh was supported in part by the NSF under the National AI Institute for Edge Computing Leveraging Next Generation Wireless Networks, Grant CNS-2625580.
References
Additional Related Work
Human-AI Collaboration. Learning to defer (L2D) framework Geifman and El-Yaniv [ [15 ] ; Mozannar and Sontag [ [30 ] jointly optimize predictors and rejectors. Subsequent works have extended this to various settings, including capacity constraints Mozannar and Sontag [ [30 ] ; Mao et al. [ [28 ] , and routing to multiple experts Mao et al. [ [28 ] ; Verma et al. [ [34 ] ; Ramgolam [ [32 ] . Guo et al. [ [21 ] ; Alur et al. [ [3 , 4 ] studied the complementary information in human-AI collaboration and efficient aggregation method to improve human-AI team performance.
Impressionability of Human Behavior. A growing body of empirical research Bondi et al. [ [9 ] ; Vodrahalli et al. [ [35 ] ; Bo et al. [ [8 ] demonstrates that human decision-makers are highly impressionable when interacting with AI systems. Performative prediction Perdomo et al. [ [31 ] ; Lin and Zrnic [ [27 ] ; Li et al. [ [26 ] studied theoretically the situation where the deployment of a model shifts the underlying data distribution under convexity and smoothness assumptions.
Omniprediction and Outcome Indistinguishability. To handle the deployment-time variability of loss and cost functions without retraining, Gopalan et al. [ [18 ] initiated the theory of omniprediction. Dwork et al. [ [12 , 13 ] provide efficiency guarantees to achieve omniprediction via outcome indistinguishability (OI). Recently, omniprediction has been extended to selective classification Casacuberta and Kanade [ [10 ] and performative prediction Kim and Perdomo [ [24 ] .
Theoretical Results and Proofs
In this section, we formally state all the assumptions and theoretical results in § 4 , and provide detailed proofs.
Assumptions and Notations
We make the following assumption on data collection policy π l o g \pi_{\mathrm{log}} π log for COI-auditor (Definition 3.2 ) to guarantee that the offline dataset explores the action space and distribution shift induced by impressionable human behavior. Recall that we denote D Z \mathcal{D}_{\mathcal{Z}} D Z the marginal distribution of state z ∈ Z z \in\mathcal{Z} z ∈ Z .
Assumption B.1 (Data Coverage). We assume that offline dataset for the COI-auditor (Definition 3.2 ) is collected by a historical logging policy π l o g : Z → Δ A \pi_{\mathrm{log}} : \mathcal{Z} \to\Delta\mathcal{A} π log : Z → Δ A . Specifically, each sample ( z , a , o ) (z,a,o) ( z , a , o ) is drawn independently from a joint distribution D l o g \mathcal{D}_{\mathrm{log}} D log defined by the following procedure:
z ∼ D Z , a ∼ π l o g ( z ) , o ∼ p ⋆ ( ⋅ ∣ z , a ) . z \sim\mathcal{D}_{\mathcal{Z}}, \quad a \sim\pi_{\mathrm{log}}(z), \quad o \sim p^{\star}(\cdot\mid z,a). z ∼ D Z , a ∼ π log ( z ) , o ∼ p ⋆ ( ⋅ ∣ z , a ) .
We assume there exists a strictly positive constant γ ∈ ( 0 , 1 ) \gamma\in(0,1) γ ∈ ( 0 , 1 ) such that for all states z ∈ Z z \in\mathcal{Z} z ∈ Z and all actions a ∈ A a \in\mathcal{A} a ∈ A , the logging policy satisfies:
π l o g ( a ∣ z ) ≥ γ . \pi_{\mathrm{log}}(a \mid z) \ge\gamma. π log ( a ∣ z ) ≥ γ .
We make the following assumptions on the loss family L \mathcal{L} L , cost family C \mathcal{C} C , and base collaboration policies H \mathcal{H} H .
Assumption B.2. We assume that the loss, cost, and policy families ( L , C , H ) (\mathcal{L},\mathcal{C},\mathcal{H}) ( L , C , H ) satisfy the following properties:
(a) (Boundedness of L \mathcal{L} L ). The family of losses L \mathcal{L} L is the set of all bounded functions l ~ : O → [ 0 , 1 ] \tilde{l}: \mathcal{O} \to[0,1] l ~ : O → [ 0 , 1 ] .
(b) (Finiteness and Boundedness of Cost Functions C \mathcal{C} C ). The family of cost functions C \mathcal{C} C is a finite set, where each c ∈ C c \in\mathcal{C} c ∈ C is uniformly bounded such that sup x , c ∣ c ( x ) ∣ ≤ B c \sup_{x,c} |c(x)| \le B_c sup x , c ∣ c ( x ) ∣ ≤ B c .
(c) (Finite Complexity of Base Policy ). We assume the deterministic policy family R , Φ , H \mathcal{R},\Phi,\mathcal{H} R , Φ , H has finite Natarajan dimension d R , d Φ d_{\mathcal{R}},d_{\Phi} d R , d Φ and d H d_{\mathcal{H}} d H respectively, and that the sets of combination rules G \mathcal{G} G is finite.
Notations. In the following context, we shall fix the logging policy π log \pi_{\log} π l o g and collect offline dataset with it. Given base policy family H \mathcal{H} H , loss family L \mathcal{L} L and cost family C \mathcal{C} C and a base predictive distribution p p p , we define Π e v a l ( p ) : = { ( π h , ω ) : h ∈ H , ω ∈ Ω } ∪ { ( π ω , ω ) : ω ∈ Ω } \Pi_{\mathrm{eval}}(p) := \{(\pi_{h},\omega): h \in\mathcal{H},\omega\in\Omega\} \cup\{(\pi^{\omega},\omega): \omega\in\Omega\} Π eval ( p ) := {( π h , ω ) : h ∈ H , ω ∈ Ω } ∪ {( π ω , ω ) : ω ∈ Ω } . This set depends on base predictive distribution p p p through post-processing policy π ω = k ω ( p ) \pi^{\omega} = k^{\omega}(p) π ω = k ω ( p ) for a loss-cost pair ω = ( l ~ , c ) \omega= (\tilde{l},c) ω = ( l ~ , c ) .
For notation simplicity, given a base predictive distribution p p p , a pair ω = ( l ~ , c ) \omega= (\tilde{l},c) ω = ( l ~ , c ) of an evaluation loss l ~ ∈ L \tilde{l} \in\mathcal{L} l ~ ∈ L and a cost function c ∈ C c \in\mathcal{C} c ∈ C , the expected loss of a base collaboration policy h = ( r , ϕ , g ) ∈ H h = (r,\phi,g) \in\mathcal{H} h = ( r , ϕ , g ) ∈ H is defined as:
V ( h ; ω ; p ) ≜ E z E o ∼ p ( ⋅ ∣ z , π h ( z ) ) [ ℓ π h ( z ) ω ( z , o ) ] . V(h;\omega;p) \triangleq\mathbb{E}_{z}\mathbb{E}_{o \sim p(\cdot\mid z,\pi_{h}(z))}\left[\ell^{\omega}_{\pi_{h}(z)}(z,o)\right]. V ( h ; ω ; p ) ≜ E z E o ∼ p ( ⋅ ∣ z , π h ( z )) [ ℓ π h ( z ) ω ( z , o ) ] .
Similarly, for a random general policy π : Z → Δ A \pi:\mathcal{Z} \to\Delta\mathcal{A} π : Z → Δ A the expected loss is:
V ( π ; ω ; p ) ≜ E z E a ∼ π ( ⋅ ∣ z ) E o ∼ p ( ⋅ ∣ z , a ) ℓ a ω ( z , o ) . V(\pi;\omega;p) \triangleq\mathbb{E}_{z}\mathbb{E}_{a \sim\pi(\cdot\mid z)}\mathbb{E}_{o \sim p(\cdot\mid z,a)}\ell^{\omega}_{a}(z,o). V ( π ; ω ; p ) ≜ E z E a ∼ π ( ⋅ ∣ z ) E o ∼ p ( ⋅ ∣ z , a ) ℓ a ω ( z , o ) .
Further, for a possibly randomized policy π \pi π and a loss-cost pair ω = ( l ~ , c ) ∈ Ω \omega= (\tilde{l},c) \in\Omega ω = ( l ~ , c ) ∈ Ω , we define the expected COI gap as the true expected loss minus the predicted expected loss:
G ( π ; ω ; p ) = E z E a ∼ π ( ⋅ ∣ z ) [ E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] − E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ] = V ( π ; ω ; p ∗ ) − V ( π ; ω ; p ) . \begin{aligned}
G(\pi;\omega;p) &= \mathbb{E}_{z}\mathbb{E}_{a \sim\pi(\cdot\mid z)}\left[\mathbb{E}_{o \sim p^*(\cdot\mid z,a)}[\ell^{\omega}_{a}(z,o)]-\mathbb{E}_{o \sim p(\cdot\mid z,a)}[\ell^{\omega}_{a}(z,o)]\right] \\
&= V(\pi;\omega;p^*)-V(\pi;\omega;p).
\end{aligned} G ( π ; ω ; p ) = E z E a ∼ π ( ⋅ ∣ z ) [ E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] − E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] ] = V ( π ; ω ; p ∗ ) − V ( π ; ω ; p ) .
Then we use the following Inverse Propensity Score (IPS) method to establish an empirical estimator given m m m -samples { ( z i , a i , o i ) } i = 1 m \{(z_i,a_i,o_i)\}_{i=1}^{m} {( z i , a i , o i ) } i = 1 m collected via policy π log \pi_{\log} π l o g . Specifically, we define the empirical COI-violation for a base predictive distribution p p p with respect to a collaboration policy π \pi π as:
G ^ m ( π ; ω ; p ) = 1 m ∑ i = 1 m [ π ( a i ∣ z i ) π log ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − ∑ a ∈ A π ( a ∣ z i ) E o ∼ p ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o ) ] ] \widehat{G}_m(\pi;\omega;p)=\frac{1}{m}\sum_{i=1}^{m}\left[\frac{\pi(a_i\mid z_i)}{\pi_{\log}(a_i\mid z_i)}\ell^\omega_{a_i}(z_i,o_i)-\sum_{a\in\mathcal{A}}\pi(a\mid z_i)\mathbb{E}_{o\sim p(\cdot\mid z_i,a)}[\ell^\omega_a(z_i,o)]\right] G m ( π ; ω ; p ) = m 1 i = 1 ∑ m [ π l o g ( a i ∣ z i ) π ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − a ∈ A ∑ π ( a ∣ z i ) E o ∼ p ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o )] ]
COI implies Omnimechanism
In this subsection, we prove that ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI implies ( Ω , H , 2 ε + log ∣ A ∣ τ ) (\Omega,\mathcal{H},2\varepsilon+\frac{\log|\mathcal{A}|}{\tau}) ( Ω , H , 2 ε + τ l o g ∣ A ∣ ) -omnimechanism.
Theorem B.4 (Restatement of Theorem 4.1 ). Suppose a base predictive distribution p : Z × A → Δ O p:\mathcal{Z}\times\mathcal{A}\to\Delta\mathcal{O} p : Z × A → Δ O satisfies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI at level ε > 0 \varepsilon>0 ε > 0 . Then, for any evaluation loss l ~ ∈ L \tilde{l}\in\mathcal{L} l ~ ∈ L and cost c ∈ C c\in\mathcal{C} c ∈ C , the extracted post-processing collaboration policy π ω = k ω ( p ) \pi^\omega=k^\omega(p) π ω = k ω ( p ) achieves:
V ( π ω ; ω ; p ∗ ) ≤ min h ∈ H V ( h ; ω ; p ∗ ) + 2 ε + log ( 2 ∣ S ∣ ∣ G ∣ ) τ V(\pi^\omega;\omega;p^*)\leq\min_{h\in\mathcal{H}}V(h;\omega;p^*)+2\varepsilon+\frac{\log(2|\mathcal{S}||\mathcal{G}|)}{\tau} V ( π ω ; ω ; p ∗ ) ≤ h ∈ H min V ( h ; ω ; p ∗ ) + 2 ε + τ log ( 2∣ S ∣∣ G ∣ )
Proof of Theorem 4.1 . Fix an arbitrary evaluation loss l ~ ∈ L \tilde{l}\in\mathcal{L} l ~ ∈ L , cost c ∈ C c\in\mathcal{C} c ∈ C , and base collaboration policy h ∈ H h\in\mathcal{H} h ∈ H , with induced deterministic policy π h \pi_h π h . Write ω = ( l ~ , c ) \omega=(\tilde{l},c) ω = ( l ~ , c ) . For a state z = ( x , y M ) z=(x,y_M) z = ( x , y M ) and an action a = ( r , s , g ) ∈ A a=(r,s,g)\in\mathcal{A} a = ( r , s , g ) ∈ A , define the simulated expected loss under p p p as
L p ( z , a ) ≜ E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] = E o ∼ p ( ⋅ ∣ z , a ) [ l ~ ( o ) + r c ( x ) ] . L_p(z,a)\triangleq\mathbb{E}_{o\sim p(\cdot\mid z,a)}[\ell^\omega_a(z,o)]=\mathbb{E}_{o\sim p(\cdot\mid z,a)}[\tilde{l}(o)+rc(x)]. L p ( z , a ) ≜ E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] = E o ∼ p ( ⋅ ∣ z , a ) [ l ~ ( o ) + r c ( x )] .
For finite τ > 0 \tau>0 τ > 0 , the post-processed policy π ω = k ω ( p ) \pi^\omega=k^\omega(p) π ω = k ω ( p ) is given by
π ω ( a ∣ z ) = exp ( − τ L p ( z , a ) ) ∑ a ′ ∈ A exp ( − τ L p ( z , a ′ ) ) . \pi^\omega(a\mid z)=\frac{\exp(-\tau L_p(z,a))}{\sum_{a'\in\mathcal{A}}\exp(-\tau L_p(z,a'))}. π ω ( a ∣ z ) = ∑ a ′ ∈ A exp ( − τ L p ( z , a ′ )) exp ( − τ L p ( z , a )) .
At each state z z z , its simulated expected loss satisfies
∑ a ∈ A π ω ( a ∣ z ) L p ( z , a ) = − 1 τ log ( ∑ a ′ ∈ A exp ( − τ L p ( z , a ′ ) ) ) + 1 τ H ( π ω ( ⋅ ∣ z ) ) ≤ − 1 τ log ( exp ( − τ min a ′ ∈ A L p ( z , a ′ ) ) ) + log ∣ A ∣ τ = min a ′ ∈ A L p ( z , a ′ ) + log ∣ A ∣ τ ≤ L p ( z , π h ( z ) ) + log ∣ A ∣ τ , \begin{aligned}
\sum_{a\in\mathcal{A}}\pi^\omega(a\mid z)L_p(z,a)
&=-\frac{1}{\tau}\log\left(\sum_{a'\in\mathcal{A}}\exp(-\tau L_p(z,a'))\right)+\frac{1}{\tau}H(\pi^\omega(\cdot\mid z))\\
&\leq-\frac{1}{\tau}\log\left(\exp\left(-\tau\min_{a'\in\mathcal{A}}L_p(z,a')\right)\right)+\frac{\log|\mathcal{A}|}{\tau}\\
&=\min_{a'\in\mathcal{A}}L_p(z,a')+\frac{\log|\mathcal{A}|}{\tau}\\
&\leq L_p(z,\pi_h(z))+\frac{\log|\mathcal{A}|}{\tau},
\end{aligned} a ∈ A ∑ π ω ( a ∣ z ) L p ( z , a ) = − τ 1 log ( a ′ ∈ A ∑ exp ( − τ L p ( z , a ′ )) ) + τ 1 H ( π ω ( ⋅ ∣ z )) ≤ − τ 1 log ( exp ( − τ a ′ ∈ A min L p ( z , a ′ ) ) ) + τ log ∣ A ∣ = a ′ ∈ A min L p ( z , a ′ ) + τ log ∣ A ∣ ≤ L p ( z , π h ( z )) + τ log ∣ A ∣ ,
where H ( π ω ( ⋅ ∣ z ) ) H(\pi^\omega(\cdot\mid z)) H ( π ω ( ⋅ ∣ z )) denotes the entropy of the action distribution and is at most log ∣ A ∣ \log|\mathcal{A}| log ∣ A ∣ . Taking expectation over z z z gives
V ( π ω ; ω ; p ) ≤ V ( π h ; ω ; p ) + log ∣ A ∣ τ . V(\pi^\omega;\omega;p)\leq V(\pi_h;\omega;p)+\frac{\log|\mathcal{A}|}{\tau}. V ( π ω ; ω ; p ) ≤ V ( π h ; ω ; p ) + τ log ∣ A ∣ .
By Decision-OI and Hypothesis-OI, respectively, we obtain
V ( π ω ; ω ; p ∗ ) ≤ V ( π ω ; ω ; p ) + ε ≤ V ( π h ; ω ; p ) + ε + log ∣ A ∣ τ ≤ V ( π h ; ω ; p ∗ ) + 2 ε + log ∣ A ∣ τ . \begin{aligned}
V(\pi^\omega;\omega;p^*)\leq V(\pi^\omega;\omega;p)+\varepsilon\\
&\leq V(\pi_h;\omega;p)+\varepsilon+\frac{\log|\mathcal{A}|}{\tau} \\
&\leq V(\pi_h;\omega;p^*)+2\varepsilon+\frac{\log|\mathcal{A}|}{\tau}.
\end{aligned} V ( π ω ; ω ; p ∗ ) ≤ V ( π ω ; ω ; p ) + ε ≤ V ( π h ; ω ; p ) + ε + τ log ∣ A ∣ ≤ V ( π h ; ω ; p ∗ ) + 2 ε + τ log ∣ A ∣ .
Since this holds for every l ~ ∈ L \tilde{l} \in\mathcal{L} l ~ ∈ L , c ∈ C c \in\mathcal{C} c ∈ C , and h ∈ H h \in\mathcal{H} h ∈ H , the claimed guarantee follows.
When τ = ∞ \tau= \infty τ = ∞ , the post-processing rule selects an action minimizing L p ( z , a ) L_p(z,a) L p ( z , a ) at each state. The same comparison therefore holds with zero smoothing error.
Convergence of the COI-Boost Algorithm
In this section, we prove the convergence of the COI-boosting algorithm (Algorithm 1), showing that it will always terminate in finite round and output an ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI base predictive distribution with high probability.
Theorem B.5 (Convergence of Boosting Algorithm 1). Under Assumption B.1, suppose 0 ≤ l ~ ( o ) ≤ 1 0 \leq\tilde{l}(o) \leq1 0 ≤ l ~ ( o ) ≤ 1 for all l ~ ∈ L \tilde{l} \in\mathcal{L} l ~ ∈ L and o ∈ O o \in\mathcal{O} o ∈ O . For ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] and δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) , set η = γ ε 4 \eta= \frac{\gamma\varepsilon}{4} η = 4 γ ε , T max = ⌈ 16 log ∣ O ∣ γ ε 2 ⌉ T_{\max} = \left\lceil\frac{16\log|\mathcal{O}|}{\gamma\varepsilon^{2}} \right\rceil T m a x = ⌈ γ ε 2 16 l o g ∣ O ∣ ⌉ . Run Algorithm 1 for T max T_{\max} T m a x steps, using an independent batch of logged samples at each auditing call and an auditor with failure probability at most δ \delta δ per call. Then, with probability at least 1 − T max δ 1-T_{\max}\delta 1 − T m a x δ , the algorithm terminates within T max T_{\max} T m a x auditing calls and returns a valid ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI predictor.
Proof of Theorem B.5. If ∣ O ∣ = 1 |\mathcal{O}| = 1 ∣ O ∣ = 1 , the initial predictor equals p ∗ p^* p ∗ , and the conclusion is immediate. Hence, assume ∣ O ∣ ≥ 2 |\mathcal{O}| \geq2 ∣ O ∣ ≥ 2 . Conditional on the history before each auditing call, the current predictor is fixed and its audit batch is independent of that history. Thus, the auditor fails with probability at most δ \delta δ at each call. Let E E E be the event that every auditing call made by the algorithm succeeds. Since there are at most T max T_{\max} T m a x calls, the union bound gives
P ( E ) ≥ 1 − T max δ . \mathbb{P}(E) \geq1-T_{\max}\delta. P ( E ) ≥ 1 − T m a x δ .
Condition on E E E . If the auditor returns ⊥ \bot ⊥ at iteration t t t , the auditor guarantee implies that p ( t ) p^{(t)} p ( t ) satisfies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI. We now show that this must occur within T max T_{\max} T m a x auditing calls.
We track the expected KL-divergence potential
Φ t = E z ∼ D Z E a ∼ π log ( ⋅ ∣ z ) [ KL ( p ∗ ( ⋅ ∣ z , a ) ∥ p ( t ) ( ⋅ ∣ z , a ) ) ] . \Phi_t = \mathbb{E}_{z\sim D_Z}\mathbb{E}_{a\sim\pi_{\log}(\cdot\mid z)}\left[\operatorname{KL}\left(p^*(\cdot\mid z,a)\mathbin{\|}p^{(t)}(\cdot\mid z,a)\right)\right]. Φ t = E z ∼ D Z E a ∼ π l o g ( ⋅ ∣ z ) [ KL ( p ∗ ( ⋅ ∣ z , a ) ∥ p ( t ) ( ⋅ ∣ z , a ) ) ] .
We have Φ t ≥ 0 \Phi_t \geq0 Φ t ≥ 0 , and uniform initialization gives Φ 0 ≤ log ∣ O ∣ \Phi_0 \leq\log|\mathcal{O}| Φ 0 ≤ log ∣ O ∣ .
At an iteration where the auditor returns a certificate, write its output as ( π ( t ) , ω ( t ) , σ ) (\pi^{(t)},\omega^{(t)},\sigma) ( π ( t ) , ω ( t ) , σ ) , where ω ( t ) = ( l ~ ( t ) , c ( t ) ) \omega^{(t)}=(\tilde{l}^{(t)},c^{(t)}) ω ( t ) = ( l ~ ( t ) , c ( t ) ) . For a = ( r , s , g ) a=(r,s,g) a = ( r , s , g ) , the total loss is
ℓ a ω ( t ) ( z , o ) = l ~ ( t ) ( o ) + r c ( t ) ( x ) . \ell_a^{\omega^{(t)}}(z,o)=\tilde{l}^{(t)}(o)+r c^{(t)}(x). ℓ a ω ( t ) ( z , o ) = l ~ ( t ) ( o ) + r c ( t ) ( x ) .
The cost term is independent of o o o , so its exponential factor cancels between the numerator and denominator of the normalized update. Therefore, for the following analysis, we may equivalently use the score
g ( t ) ( z , a , o ) = σ π ( t ) ( a ∣ z ) π log ( a ∣ z ) l ~ ( t ) ( o ) . g^{(t)}(z,a,o)=\sigma\frac{\pi^{(t)}(a\mid z)}{\pi_{\log}(a\mid z)}\tilde{l}^{(t)}(o). g ( t ) ( z , a , o ) = σ π l o g ( a ∣ z ) π ( t ) ( a ∣ z ) l ~ ( t ) ( o ) .
This replacement leaves every predictor p ( t ) p^{(t)} p ( t ) unchanged. By the auditor guarantee,
σ E z ∼ D Z E a ∼ π ( t ) ( ⋅ ∣ z ) [ E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( t ) ( z , o ) ] − E o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ℓ a ω ( t ) ( z , o ) ] ] > ε 2 . \sigma\mathbb{E}_{z\sim D_Z}\mathbb{E}_{a\sim\pi^{(t)}(\cdot\mid z)}\left[\mathbb{E}_{o\sim p^*(\cdot\mid z,a)}\left[\ell_a^{\omega^{(t)}}(z,o)\right]-\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z,a)}\left[\ell_a^{\omega^{(t)}}(z,o)\right]\right] > \frac{\varepsilon}{2}. σ E z ∼ D Z E a ∼ π ( t ) ( ⋅ ∣ z ) [ E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( t ) ( z , o ) ] − E o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ℓ a ω ( t ) ( z , o ) ] ] > 2 ε .
The outcome-independent cost also cancels in this difference. Consequently,
E z ∼ D E a ∼ π log ( ⋅ ∣ z ) [ ∣ E θ 0 ∼ p ∗ ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) − E θ 0 ∼ p ( t ) ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) ∣ > ε 2 ] . \mathbb{E}_{z\sim\mathcal{D}}\mathbb{E}_{a\sim\pi_{\log}(\cdot\mid z)}\left[\left|\mathbb{E}_{\theta_0\sim p^*(\cdot\mid z,a)}g^{(t)}(z,a,o)-\mathbb{E}_{\theta_0\sim p^{(t)}(\cdot\mid z,a)}g^{(t)}(z,a,o)\right|>\frac{\varepsilon}{2}\right]. E z ∼ D E a ∼ π l o g ( ⋅ ∣ z ) [ E θ 0 ∼ p ∗ ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) − E θ 0 ∼ p ( t ) ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) > 2 ε ] .
Using the positive exponential update, we obtain
Φ t − Φ t + 1 = E z ∼ D , a ∼ π log ( ⋅ ∣ z ) [ ∑ o ∈ O p ∗ ( o ∣ z , a ) log p ( t + 1 ) ( o ∣ z , a ) p ( t ) ( o ∣ z , a ) ] = E z ∼ D , a ∼ π log ( ⋅ ∣ z ) [ η E θ 0 ∼ p ∗ ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) − log ( E θ 0 ∼ p ( t ) ( ⋅ ∣ z , a ) exp ( η g ( t ) ( z , a , o ) ) ) ] . \begin{aligned}
\Phi_t-\Phi_{t+1}
&=\mathbb{E}_{z\sim\mathcal{D},a\sim\pi_{\log}(\cdot\mid z)}\left[\sum_{o\in\mathcal{O}}p^*(o\mid z,a)\log\frac{p^{(t+1)}(o\mid z,a)}{p^{(t)}(o\mid z,a)}\right]\\
&=\mathbb{E}_{z\sim\mathcal{D},a\sim\pi_{\log}(\cdot\mid z)}\left[\eta\mathbb{E}_{\theta_0\sim p^*(\cdot\mid z,a)}g^{(t)}(z,a,o)-\log\left(\mathbb{E}_{\theta_0\sim p^{(t)}(\cdot\mid z,a)}\exp\left(\eta g^{(t)}(z,a,o)\right)\right)\right].
\end{aligned} Φ t − Φ t + 1 = E z ∼ D , a ∼ π l o g ( ⋅ ∣ z ) [ o ∈ O ∑ p ∗ ( o ∣ z , a ) log p ( t ) ( o ∣ z , a ) p ( t + 1 ) ( o ∣ z , a ) ] = E z ∼ D , a ∼ π l o g ( ⋅ ∣ z ) [ η E θ 0 ∼ p ∗ ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) − log ( E θ 0 ∼ p ( t ) ( ⋅ ∣ z , a ) exp ( η g ( t ) ( z , a , o ) ) ) ] .
By Assumption B.1 and the boundedness of l ~ ( t ) \tilde{l}^{(t)} l ~ ( t ) , we have ∣ g ( t ) ∣ ≤ 1 / γ |g^{(t)}|\leq1/\gamma ∣ g ( t ) ∣ ≤ 1/ γ . Since η = γ ε / 4 \eta=\gamma\varepsilon/4 η = γ ε /4 and ε ≤ 1 \varepsilon\leq1 ε ≤ 1 ,
∣ η g ( t ) ∣ ≤ 1 4 . |\eta g^{(t)}|\leq\frac{1}{4}. ∣ η g ( t ) ∣ ≤ 4 1 .
Applying exp ( η g ( t ) ) ≤ 1 + η g ( t ) + η 2 ( g ( t ) ) 2 \exp(\eta g^{(t)})\leq1+\eta g^{(t)}+\eta^2(g^{(t)})^2 exp ( η g ( t ) ) ≤ 1 + η g ( t ) + η 2 ( g ( t ) ) 2 and log x ≤ x − 1 \log x\leq x-1 log x ≤ x − 1 yields
Φ t − Φ t + 1 ≥ η E z ∼ D , a ∼ π log ( ⋅ ∣ z ) [ E θ 0 ∼ p ∗ ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) − E θ 0 ∼ p ( t ) ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) ] − η 2 E z ∼ D , a ∼ π log ( ⋅ ∣ z ) , o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ( g ( t ) ( z , a , o ) ) 2 ] . \begin{aligned}
\Phi_t-\Phi_{t+1}
&\geq\eta\mathbb{E}_{z\sim\mathcal{D},a\sim\pi_{\log}(\cdot\mid z)}\left[\mathbb{E}_{\theta_0\sim p^*(\cdot\mid z,a)}g^{(t)}(z,a,o)-\mathbb{E}_{\theta_0\sim p^{(t)}(\cdot\mid z,a)}g^{(t)}(z,a,o)\right]\\
&\quad-\eta^2\mathbb{E}_{z\sim\mathcal{D},a\sim\pi_{\log}(\cdot\mid z),o\sim p^{(t)}(\cdot\mid z,a)}\left[(g^{(t)}(z,a,o))^2\right].
\end{aligned} Φ t − Φ t + 1 ≥ η E z ∼ D , a ∼ π l o g ( ⋅ ∣ z ) [ E θ 0 ∼ p ∗ ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) − E θ 0 ∼ p ( t ) ( ⋅ ∣ z , a ) g ( t ) ( z , a , o ) ] − η 2 E z ∼ D , a ∼ π l o g ( ⋅ ∣ z ) , o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ( g ( t ) ( z , a , o ) ) 2 ] .
Moreover,
E z ∼ D , a ∼ π log ( ⋅ ∣ z ) , o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ( g ( t ) ( z , a , o ) ) 2 ] = E z ∼ D [ ∑ a ∈ A ( π ( t ) ( a ∣ z ) ) 2 π log ( a ∣ z ) E o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ( l ~ ( t ) ( o ) ) 2 ] ] ≤ 1 γ E z ∼ D [ ∑ a ∈ A ( π ( t ) ( a ∣ z ) ) 2 ] ≤ 1 γ . \begin{aligned}
\mathbb{E}_{z\sim\mathcal{D},a\sim\pi_{\log}(\cdot\mid z),o\sim p^{(t)}(\cdot\mid z,a)}\left[(g^{(t)}(z,a,o))^2\right]
&=\mathbb{E}_{z\sim\mathcal{D}}\left[\sum_{a\in\mathcal{A}}\frac{(\pi^{(t)}(a\mid z))^2}{\pi_{\log}(a\mid z)}\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z,a)}\left[(\tilde{l}^{(t)}(o))^2\right]\right]\\
&\leq\frac{1}{\gamma}\mathbb{E}_{z\sim\mathcal{D}}\left[\sum_{a\in\mathcal{A}}(\pi^{(t)}(a\mid z))^2\right]\leq\frac{1}{\gamma}.
\end{aligned} E z ∼ D , a ∼ π l o g ( ⋅ ∣ z ) , o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ( g ( t ) ( z , a , o ) ) 2 ] = E z ∼ D [ a ∈ A ∑ π l o g ( a ∣ z ) ( π ( t ) ( a ∣ z ) ) 2 E o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ( l ~ ( t ) ( o ) ) 2 ] ] ≤ γ 1 E z ∼ D [ a ∈ A ∑ ( π ( t ) ( a ∣ z ) ) 2 ] ≤ γ 1 .
Combining these bounds with the signed-gap guarantee, every update satisfies
Φ t − Φ t + 1 > η ε 2 − η 2 γ = γ ε 2 16 . \Phi_t-\Phi_{t+1}>\frac{\eta\varepsilon}{2}-\frac{\eta^2}{\gamma}=\frac{\gamma\varepsilon^2}{16}. Φ t − Φ t + 1 > 2 η ε − γ η 2 = 16 γ ε 2 .
If all T max T_{\max} T m a x auditing calls returned certificates, the algorithm would perform T max T_{\max} T m a x updates, giving
0 ≤ Φ T max < Φ 0 − γ ε 2 16 T max ≤ log ∣ O ∣ − γ ε 2 16 T max ≤ 0 , 0\leq\Phi_{T_{\max}}<\Phi_0-\frac{\gamma\varepsilon^2}{16}T_{\max}\leq\log|\mathcal{O}|-\frac{\gamma\varepsilon^2}{16}T_{\max}\leq0, 0 ≤ Φ T m a x < Φ 0 − 16 γ ε 2 T m a x ≤ log ∣ O ∣ − 16 γ ε 2 T m a x ≤ 0 ,
a contradiction. Hence, on E \mathcal{E} E , an auditor returns ⊥ \perp ⊥ within T max T_{\max} T m a x calls, and the returned predictor satisfies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI.
Remark B.6. We remark here that the above convergence theorem does not depend on the Lipschitzness of policy with smoothing constant τ < ∞ \tau<\infty τ < ∞ , so it applies to τ = ∞ \tau=\infty τ = ∞ .
Sample Efficiency of Auditor
In this subsection, we prove our main efficiency results.
Theorem B.7 (Restatement of Theorem 4.2 ). Under Assumptions B.1 and B.2, fix τ > 0 \tau> 0 τ > 0 and ε , δ ∈ ( 0 , 1 ) \varepsilon,\delta\in(0,1) ε , δ ∈ ( 0 , 1 ) . For a base predictive distribution p p p independent of the audit batch, there exists an ( Ω , H , ε , δ , m ) (\Omega,\mathcal{H},\varepsilon,\delta,m) ( Ω , H , ε , δ , m ) -COI auditor using m m m independent offline samples collected under π log \pi_{\log} π l o g , provided that
m ≳ ( 1 + B C ) 2 γ 2 ε 2 ( ( d R + d Φ + log 2 ∣ G ∣ ) log ∣ A ∣ + ( 1 + τ ) 2 ∣ O ∣ + log 2 ∣ C ∣ δ ) , m \gtrsim\frac{(1+B_{\mathcal{C}})^2}{\gamma^2\varepsilon^2}\left((d_{\mathcal{R}}+d_{\Phi}+\log_2|\mathcal{G}|)\log|\mathcal{A}|+(1+\tau)^2|\mathcal{O}|+\log\frac{2|\mathcal{C}|}{\delta}\right), m ≳ γ 2 ε 2 ( 1 + B C ) 2 ( ( d R + d Φ + log 2 ∣ G ∣ ) log ∣ A ∣ + ( 1 + τ ) 2 ∣ O ∣ + log δ 2∣ C ∣ ) , with a sufficiently large universal implicit constant.
Proof. We first construct the auditor by defining its empirical objective, then separately derive the sample complexity for Hypothesis-OI and Decision-OI. Throughout the proof, we use the sign convention
G ( π ; ω ; p ) = V ( π ; ω ; p ∗ ) − V ( π ; ω ; p ) . G(\pi;\omega;p)=V(\pi;\omega;p^*)-V(\pi;\omega;p). G ( π ; ω ; p ) = V ( π ; ω ; p ∗ ) − V ( π ; ω ; p ) .
Step 1: Construction of Auditor.
We denote the evaluation class by
Π e v a l = { ( π ω , ω ) : ω ∈ Ω } ∪ { ( π h , ω ) : h ∈ H , ω ∈ Ω } . \Pi_{\mathrm{eval}}=\{(\pi^\omega,\omega):\omega\in\Omega\}\cup\{(\pi_{h},\omega):h\in\mathcal{H},\omega\in\Omega\}. Π eval = {( π ω , ω ) : ω ∈ Ω } ∪ {( π h , ω ) : h ∈ H , ω ∈ Ω } .
The auditor approximately maximizes the absolute empirical COI gap:
sup ( π , ω ) ∈ Π e v a l ∣ G ^ m ( π ; ω ; p ) ∣ , \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}}\left|\widehat{G}_m(\pi;\omega;p)\right|, ( π , ω ) ∈ Π eval sup G m ( π ; ω ; p ) ,
where G ^ m \widehat{G}_m G m is defined below. It selects a pair ( π , ω ) ∈ Π e v a l (\pi,\omega)\in\Pi_{\mathrm{eval}} ( π , ω ) ∈ Π eval whose objective value is within ε / 8 \varepsilon/8 ε /8 of this supremum.
If ∣ G ^ m ( π ; ω ; p ) ∣ > 3 ε / 4 |\widehat{G}_m(\pi;\omega;p)|>3\varepsilon/4 ∣ G m ( π ; ω ; p ) ∣ > 3 ε /4 , the auditor returns ( π , ω , σ ) (\pi,\omega,\sigma) ( π , ω , σ ) with σ = sign ( G ^ m ( π ; ω ; p ) ) \sigma=\operatorname{sign}(\widehat{G}_m(\pi;\omega;p)) σ = sign ( G m ( π ; ω ; p )) . Otherwise, it returns ⊥ \bot ⊥ . We will show that this rule is valid whenever
sup ( π , ω ) ∈ Π e v a l ∣ G ^ m ( π ; ω ; p ) − G ( π ; ω ; p ) ∣ ≤ ε 8 . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}}\left|\widehat{G}_m(\pi;\omega;p)-G(\pi;\omega;p)\right|\leq\frac{\varepsilon}{8}. ( π , ω ) ∈ Π eval sup G m ( π ; ω ; p ) − G ( π ; ω ; p ) ≤ 8 ε .
Step 2: Sample Complexity for Hypothesis-OI.
Recall that the empirical COI gap is
G ^ m ( π ; ω ; p ) = 1 m ∑ i = 1 m [ π ( a i ∣ z i ) π log ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − ∑ a ∈ A π ( a ∣ z i ) E o ∼ p ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o ) ] ] . \widehat{G}_m(\pi;\omega;p)=\frac{1}{m}\sum_{i=1}^{m}\left[\frac{\pi(a_i\mid z_i)}{\pi_{\log}(a_i\mid z_i)}\ell^\omega_{a_i}(z_i,o_i)-\sum_{a\in\mathcal{A}}\pi(a\mid z_i)\mathbb{E}_{o\sim p(\cdot\mid z_i,a)}[\ell^\omega_a(z_i,o)]\right]. G m ( π ; ω ; p ) = m 1 i = 1 ∑ m [ π l o g ( a i ∣ z i ) π ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − a ∈ A ∑ π ( a ∣ z i ) E o ∼ p ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o )] ] .
This is an unbiased estimator of G ( π ; ω ; p ) G(\pi;\omega;p) G ( π ; ω ; p ) for every fixed pair ( π , ω ) (\pi,\omega) ( π , ω ) .
For fixed h h h and c c c , the estimation error G ^ m ( π h ; ω ; p ) − G ( π h ; ω ; p ) \widehat{G}_m(\pi_h;\omega;p)-G(\pi_h;\omega;p) G m ( π h ; ω ; p ) − G ( π h ; ω ; p ) is affine in ℓ ~ ∈ [ 0 , 1 ] ∣ O ∣ \widetilde{\ell}\in[0,1]^{|\mathcal{O}|} ℓ ∈ [ 0 , 1 ] ∣ O ∣ . Its maximum absolute value is therefore attained at one of the 2 ∣ O ∣ 2^{|\mathcal{O}|} 2 ∣ O ∣ vertices of this cube.
We next control the complexity of the induced deterministic policy class. Since π h \pi_h π h retains the three action components associated with h = ( r , ϕ , g ) h=(r,\phi,g) h = ( r , ϕ , g ) , we have
d H ≤ d R + d Φ + log 2 ∣ G ∣ . d_{\mathcal{H}}\leq d_{\mathcal{R}}+d_{\Phi}+\log_2|\mathcal{G}|. d H ≤ d R + d Φ + log 2 ∣ G ∣.
Indeed, partition any Natarajan-shattered set according to the first component in which its two witness action labels differ. The subsets assigned to the first two components are Natarajan-shattered by R \mathcal{R} R and Φ \Phi Φ , respectively. If the third subset contains q q q points, realizing all its binary choices requires at least 2 q 2^q 2 q distinct choices from G \mathcal{G} G . The stated bound follows.
Let Π H ( n ) \Pi_{\mathcal{H}}(n) Π H ( n ) denote the maximum number of action labelings induced on n n n contexts. The multi-class Sauer bound gives, for n ≥ d H ≥ 1 n \ge d_{\mathcal{H}} \ge1 n ≥ d H ≥ 1 ,
Π H ( n ) ≤ ∑ j = 0 d H ( n j ) ( ∣ A ∣ 2 ) j ≤ ( e n ∣ A ∣ 2 d H ) d H . \Pi_{\mathcal{H}}(n) \le\sum_{j=0}^{d_{\mathcal{H}}} \binom{n}{j}\binom{|\mathcal{A}|}{2}^{j} \le\left(\frac{\mathrm{e}n|\mathcal{A}|^2}{d_{\mathcal{H}}}\right)^{d_{\mathcal{H}}}. Π H ( n ) ≤ j = 0 ∑ d H ( j n ) ( 2 ∣ A ∣ ) j ≤ ( d H e n ∣ A ∣ 2 ) d H .
For fixed ω \omega ω , consider the scalar IPS contributions indexed by h ∈ H h \in\mathcal{H} h ∈ H :
( z , a , o ) ⟼ 1 { π h ( z ) = a } π log ( a ∣ z ) ℓ a ω ( z , o ) − E o ′ ∼ p ( ⋅ ∣ z , π h ( z ) ) [ ℓ π h ( z ) ω ( z , o ′ ) ] . (z,a,o) \longmapsto\frac{\mathbf{1}\{\pi_h(z)=a\}}{\pi_{\log}(a\mid z)}\ell_a^\omega(z,o)-\mathbb{E}_{o'\sim p(\cdot\mid z,\pi_h(z))}\left[\ell_{\pi_h(z)}^\omega(z,o')\right]. ( z , a , o ) ⟼ π l o g ( a ∣ z ) 1 { π h ( z ) = a } ℓ a ω ( z , o ) − E o ′ ∼ p ( ⋅ ∣ z , π h ( z )) [ ℓ π h ( z ) ω ( z , o ′ ) ] .
Each contribution has absolute value at most 2 ( 1 + B c ) / γ 2(1+B_c)/\gamma 2 ( 1 + B c ) / γ . On any collection of logged observations, its values are determined by the action labels assigned by π h \pi_h π h . Consequently, if this scalar class pseudo-shatters n n n observations, then
2 n ≤ Π H ( n ) . 2^n \le\Pi_{\mathcal{H}}(n). 2 n ≤ Π H ( n ) .
The growth bound above therefore implies that its pseudo-dimension is O ( d H log ∣ A ∣ ) O(d_{\mathcal{H}}\log|\mathcal{A}|) O ( d H log ∣ A ∣ ) . When d H = 0 d_{\mathcal{H}}=0 d H = 0 , the induced policy class contains only one distinct policy, and the same concentration conclusion follows directly.
Applying the VC-subgraph covering bound and Dudley’s entropy integral, followed by the standard Rademacher generalization bound, controls this scalar class uniformly over h h h . Taking a union bound over the 2 ∣ O ∣ 2^{|\mathcal{O}|} 2 ∣ O ∣ loss vertices and c ∈ C c \in\mathcal{C} c ∈ C gives, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 ,
sup h ∈ H , ω ∈ Ω ∣ G ^ m ( π h ; ω ; p ) − G ( π h ; ω ; p ) ∣ ≤ 1 + B c γ m d H log ∣ A ∣ + ∣ O ∣ + log 2 ∣ C ∣ δ . \sup_{h\in\mathcal{H},\omega\in\Omega}\left|\widehat{G}_m(\pi_h;\omega;p)-G(\pi_h;\omega;p)\right| \le\frac{1+B_c}{\gamma\sqrt{m}}\sqrt{d_{\mathcal{H}}\log|\mathcal{A}|+|\mathcal{O}|+\log\frac{2|\mathcal{C}|}{\delta}}. h ∈ H , ω ∈ Ω sup G m ( π h ; ω ; p ) − G ( π h ; ω ; p ) ≤ γ m 1 + B c d H log ∣ A ∣ + ∣ O ∣ + log δ 2∣ C ∣ .
Step 3: Sample Complexity for Decision-OI.
Fix a cost c ∈ C c \in\mathcal{C} c ∈ C and write ω = ( ℓ ~ , c ) \omega=(\tilde{\ell},c) ω = ( ℓ ~ , c ) . For an action a = ( r , s , g ) a=(r,s,g) a = ( r , s , g ) and a logged observation ( z i , a i , o i ) (z_i,a_i,o_i) ( z i , a i , o i ) , define
g i ( a ; ℓ ~ ) = 1 { a i = a } π log ( a i ∣ z i ) ℓ a ω ( z i , o i ) − E o ∼ p ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o ) ] . g_i(a;\tilde{\ell})=\frac{\mathbf{1}\{a_i=a\}}{\pi_{\log}(a_i\mid z_i)}\ell_a^\omega(z_i,o_i)-\mathbb{E}_{o\sim p(\cdot\mid z_i,a)}[\ell_a^\omega(z_i,o)]. g i ( a ; ℓ ~ ) = π l o g ( a i ∣ z i ) 1 { a i = a } ℓ a ω ( z i , o i ) − E o ∼ p ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o )] .
For a policy π \pi π , set
g i ( π , ℓ ~ ) = ∑ a ∈ A π ( a ∣ z i ) g i ( a ; ℓ ~ ) . g_i(\pi,\tilde{\ell})=\sum_{a\in\mathcal{A}}\pi(a\mid z_i)g_i(a;\tilde{\ell}). g i ( π , ℓ ~ ) = a ∈ A ∑ π ( a ∣ z i ) g i ( a ; ℓ ~ ) .
Thus
G ^ m ( π ω ; ω ; p ) = 1 m ∑ i = 1 m g i ( π ω , ℓ ~ ) . \widehat{G}_m(\pi^\omega;\omega;p)=\frac{1}{m}\sum_{i=1}^{m}g_i(\pi^\omega,\tilde{\ell}). G m ( π ω ; ω ; p ) = m 1 i = 1 ∑ m g i ( π ω , ℓ ~ ) .
Lemma B.8. For fixed c c c , the function ℓ ~ ↦ g i ( π ( ℓ ~ , c ) , ℓ ~ ) \tilde{\ell}\mapsto g_i(\pi^{(\tilde{\ell},c)},\tilde{\ell}) ℓ ~ ↦ g i ( π ( ℓ ~ , c ) , ℓ ~ ) is 4 ( 1 + B c ) ( 1 + τ ) γ \frac{4(1+B_c)(1+\tau)}{\gamma} γ 4 ( 1 + B c ) ( 1 + τ ) -Lipschitz with respect to the ℓ ∞ \ell_\infty ℓ ∞ norm on [ 0 , 1 ] ∣ O ∣ [0,1]^{|\mathcal{O}|} [ 0 , 1 ] ∣ O ∣ .
Proof of Lemma B.8. Using the product rule,
∇ l ~ g i ( π ω , l ~ ) = ∑ a ∈ A [ g i ( a ; l ~ ) ∇ l ~ π ω ( a ∣ z i ) + π ω ( a ∣ z i ) ∇ l ~ g i ( a ; l ~ ) ] . \nabla_{\tilde l}g_i(\pi^\omega,\tilde l)=\sum_{a\in\mathcal{A}}\left[g_i(a;\tilde l)\nabla_{\tilde l}\pi^\omega(a\mid z_i)+\pi^\omega(a\mid z_i)\nabla_{\tilde l}g_i(a;\tilde l)\right]. ∇ l ~ g i ( π ω , l ~ ) = a ∈ A ∑ [ g i ( a ; l ~ ) ∇ l ~ π ω ( a ∣ z i ) + π ω ( a ∣ z i ) ∇ l ~ g i ( a ; l ~ ) ] .
By the definition of the softmax policy and the fact that each p ( ⋅ ∣ z i , a ) p(\cdot\mid z_i,a) p ( ⋅ ∣ z i , a ) is a probability distribution,
∥ ∇ l ~ π ω ( a ∣ z i ) ∥ 1 ≤ 2 τ π ω ( a ∣ z i ) . \left\|\nabla_{\tilde l}\pi^\omega(a\mid z_i)\right\|_1\leq2\tau\pi^\omega(a\mid z_i). ∇ l ~ π ω ( a ∣ z i ) 1 ≤ 2 τ π ω ( a ∣ z i ) .
Also,
∣ g i ( a ; l ~ ) ∣ ≤ ( 1 + B c ) ( 1 + 1 γ ) ≤ 2 ( 1 + B c ) γ . \left|g_i(a;\tilde l)\right|\leq(1+B_c)\left(1+\frac{1}{\gamma}\right)\leq\frac{2(1+B_c)}{\gamma}. g i ( a ; l ~ ) ≤ ( 1 + B c ) ( 1 + γ 1 ) ≤ γ 2 ( 1 + B c ) .
Hence the first term satisfies
∑ a ∈ A ∣ g i ( a ; l ~ ) ∣ ∥ ∇ l ~ π ω ( a ∣ z i ) ∥ 1 ≤ 4 τ ( 1 + B c ) γ . \sum_{a\in\mathcal{A}}\left|g_i(a;\tilde l)\right|\left\|\nabla_{\tilde l}\pi^\omega(a\mid z_i)\right\|_1\leq\frac{4\tau(1+B_c)}{\gamma}. a ∈ A ∑ g i ( a ; l ~ ) ∇ l ~ π ω ( a ∣ z i ) 1 ≤ γ 4 τ ( 1 + B c ) .
For the second term, differentiating the affine dependence on l ~ \tilde l l ~ gives
∥ ∇ l ~ g i ( a ; l ~ ) ∥ 1 ≤ 1 + 1 { a i = a } τ π log ( a i ∣ z i ) ≤ 1 + 1 γ . \left\|\nabla_{\tilde l}g_i(a;\tilde l)\right\|_1\leq1+\frac{\mathbf{1}\{a_i=a\}}{\tau\pi_{\log}(a_i\mid z_i)}\leq1+\frac{1}{\gamma}. ∇ l ~ g i ( a ; l ~ ) 1 ≤ 1 + τ π l o g ( a i ∣ z i ) 1 { a i = a } ≤ 1 + γ 1 .
Therefore,
∥ ∇ l ~ g i ( π ω , l ~ ) ∥ 1 ≤ 4 τ ( 1 + B c ) γ + 1 + 1 γ ≤ 4 ( 1 + B c ) ( 1 + τ ) γ . \left\|\nabla_{\tilde l}g_i(\pi^\omega,\tilde l)\right\|_1\leq\frac{4\tau(1+B_c)}{\gamma}+1+\frac{1}{\gamma}\leq\frac{4(1+B_c)(1+\tau)}{\gamma}. ∇ l ~ g i ( π ω , l ~ ) 1 ≤ γ 4 τ ( 1 + B c ) + 1 + γ 1 ≤ γ 4 ( 1 + B c ) ( 1 + τ ) .
Since the loss cube is convex, this gradient bound establishes the claimed Lipschitz property with respect to the ℓ ∞ \ell_\infty ℓ ∞ norm. ■
For fixed c c c , let F c \mathcal{F}_c F c be the following class of functions on logged triples:
F c = { ( z , a , o ) ↦ π ( l ~ , c ) ( a ∣ z ) π log ( a ∣ z ) ℓ a ( l ~ , c ) ( z , o ) − ∑ a ′ ∈ A π ( l ~ , c ) ( a ′ ∣ z ) E o ′ ∼ p ( ⋅ ∣ z , a ′ ) [ ℓ a ′ ( l ~ , c ) ( z , o ′ ) ] : l ~ ∈ [ 0 , 1 ] ∣ O ∣ } . \mathcal{F}_c=\left\{(z,a,o)\mapsto\frac{\pi^{(\tilde l,c)}(a\mid z)}{\pi_{\log}(a\mid z)}\ell_a^{(\tilde l,c)}(z,o)-\sum_{a'\in\mathcal{A}}\pi^{(\tilde l,c)}(a'\mid z)\mathbb{E}_{o'\sim p(\cdot\mid z,a')}\left[\ell_{a'}^{(\tilde l,c)}(z,o')\right]:\tilde l\in[0,1]^{|\mathcal{O}|}\right\}. F c = { ( z , a , o ) ↦ π l o g ( a ∣ z ) π ( l ~ , c ) ( a ∣ z ) ℓ a ( l ~ , c ) ( z , o ) − a ′ ∈ A ∑ π ( l ~ , c ) ( a ′ ∣ z ) E o ′ ∼ p ( ⋅ ∣ z , a ′ ) [ ℓ a ′ ( l ~ , c ) ( z , o ′ ) ] : l ~ ∈ [ 0 , 1 ] ∣ O ∣ } .
For 0 < u ≤ 1 0<u\leq1 0 < u ≤ 1 , the loss cube admits an ℓ ∞ \ell_\infty ℓ ∞ cover of size at most ( 1 + 2 / u ) ∣ O ∣ (1+2/u)^{|\mathcal{O}|} ( 1 + 2/ u ) ∣ O ∣ . By Lemma B.8, this induces a cover of F c \mathcal{F}_c F c under the normalized empirical L 2 L_2 L 2 metric with radius 4 ( 1 + B c ) ( 1 + τ ) u / γ 4(1+B_c)(1+\tau)u/\gamma 4 ( 1 + B c ) ( 1 + τ ) u / γ . Subtracting the member corresponding to l ~ = 0 \tilde l=0 l ~ = 0 does not change empirical Rademacher complexity. Applying Dudley’s entropy integral to the resulting centered class gives
R ^ m ( F c ) ≲ ( 1 + B c ) ( 1 + τ ) γ m ∫ 0 1 ∣ O ∣ log ( 1 + 2 / u ) d u ≲ ( 1 + B c ) ( 1 + τ ) γ ∣ O ∣ m . \widehat{\mathcal{R}}_m(\mathcal{F}_c)\lesssim\frac{(1+B_c)(1+\tau)}{\gamma\sqrt{m}}\int_0^1\sqrt{|\mathcal{O}|\log(1+2/u)}\,du\lesssim\frac{(1+B_c)(1+\tau)}{\gamma}\sqrt{\frac{|\mathcal{O}|}{m}}. R m ( F c ) ≲ γ m ( 1 + B c ) ( 1 + τ ) ∫ 0 1 ∣ O ∣ log ( 1 + 2/ u ) d u ≲ γ ( 1 + B c ) ( 1 + τ ) m ∣ O ∣ .
Every member of F c \mathcal{F}_c F c has absolute value at most 2 ( 1 + B c ) / γ 2(1+B_c)/\gamma 2 ( 1 + B c ) / γ . The standard Rademacher generalization bound and a union bound over c ∈ C c\in\mathcal{C} c ∈ C therefore imply, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 ,
sup ω ∈ Ω ∣ G ^ m ( π ω ; ω ; p ) − G ( π ω ; ω ; p ) ∣ ≲ 1 + B c γ m ( ( 1 + τ ) ∣ O ∣ + log 2 ∣ C ∣ δ ) . \sup_{\omega\in\Omega}\left|\widehat{G}_m(\pi^\omega;\omega;p)-G(\pi^\omega;\omega;p)\right|\lesssim\frac{1+B_c}{\gamma\sqrt{m}}\left((1+\tau)\sqrt{|\mathcal{O}|}+\sqrt{\log\frac{2|\mathcal{C}|}{\delta}}\right). ω ∈ Ω sup G m ( π ω ; ω ; p ) − G ( π ω ; ω ; p ) ≲ γ m 1 + B c ( ( 1 + τ ) ∣ O ∣ + log δ 2∣ C ∣ ) .
Combining Steps 2 and 3, with probability at least 1 − δ 1-\delta 1 − δ ,
sup ( π , ω ) ∈ Π e v a l ∣ G ^ m ( π ; ω ; p ) − G ( π ; ω ; p ) ∣ ≲ 1 + B c γ m d H log ∣ A ∣ + ( 1 + τ ) 2 ∣ O ∣ + log 2 ∣ C ∣ δ . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}}\left|\widehat{G}_m(\pi;\omega;p)-G(\pi;\omega;p)\right|\lesssim\frac{1+B_c}{\gamma\sqrt{m}}\sqrt{d_{\mathcal{H}}\log|\mathcal{A}|+(1+\tau)2|\mathcal{O}|+\log\frac{2|\mathcal{C}|}{\delta}}. ( π , ω ) ∈ Π eval sup G m ( π ; ω ; p ) − G ( π ; ω ; p ) ≲ γ m 1 + B c d H log ∣ A ∣ + ( 1 + τ ) 2∣ O ∣ + log δ 2∣ C ∣ .
Under the stated sample-size condition, this error is at most ε / 8 \varepsilon/8 ε /8 .
On this event, suppose the auditor returns ( π , ω , σ ) (\pi,\omega,\sigma) ( π , ω , σ ) . Then
σ G ( π ; ω ; p ) ≥ ∣ G ^ m ( π ; ω ; p ) ∣ − ε 8 > 3 ε 4 − ε 8 = 5 ε 8 > ε 2 . \sigma G(\pi;\omega;p)\geq\left|\widehat{G}_m(\pi;\omega;p)\right|-\frac{\varepsilon}{8}>\frac{3\varepsilon}{4}-\frac{\varepsilon}{8}=\frac{5\varepsilon}{8}>\frac{\varepsilon}{2}. σ G ( π ; ω ; p ) ≥ G m ( π ; ω ; p ) − 8 ε > 4 3 ε − 8 ε = 8 5 ε > 2 ε .
Thus every returned tuple is a valid signed violation certificate.
Otherwise, the selected pair has absolute empirical gap at most 3 ε / 4 3\varepsilon/4 3 ε /4 . Since it is an ε / 8 \varepsilon/8 ε /8 -approximate maximizer,
sup ( π , ω ) ∈ Π e v a l ∣ G ^ m ( π ; ω ; p ) ∣ ≤ 7 ε 8 . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}}\left|\widehat{G}_m(\pi;\omega;p)\right|\leq\frac{7\varepsilon}{8}. ( π , ω ) ∈ Π eval sup G m ( π ; ω ; p ) ≤ 8 7 ε .
Consequently,
sup ( π , ω ) ∈ Π e v a l ∣ G ( π ; ω ; p ) ∣ ≤ 7 ε 8 + ε 8 = ε . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}}\left|G(\pi;\omega;p)\right|\leq\frac{7\varepsilon}{8}+\frac{\varepsilon}{8}=\varepsilon. ( π , ω ) ∈ Π eval sup ∣ G ( π ; ω ; p ) ∣ ≤ 8 7 ε + 8 ε = ε .
Returning ⊥ \bot ⊥ therefore certifies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI. This completes the proof. ■
The following lemma establishes the sample efficiency when τ = ∞ \tau=\infty τ = ∞ .
Lemma B.9 (Sample Complexity of COI Auditing for τ = ∞ \tau=\infty τ = ∞ ). Suppose Assumptions B.1 and B.2 hold, the action and outcome spaces are finite, and the logging propensities are known. Fix a base predictive distribution p p p independently of the audit batch. For τ = ∞ \tau=\infty τ = ∞ , we can define k ω ( p ) k^\omega(p) k ω ( p ) to assign probability one to the first action, under a fixed ordering of A \mathcal{A} A , that minimizes
E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] . \mathbb{E}_{o\sim p(\cdot\mid z,a)}[\ell^\omega_a(z,o)]. E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] . Let
Π e v a l ( p ) = { ( π h , ω ) : h ∈ H , ω ∈ Ω } ∪ { ( k ω ( p ) , ω ) : ω ∈ Ω } . \Pi_{\mathrm{eval}}(p)=\{(\pi_h,\omega):h\in\mathcal{H},\ \omega\in\Omega\}\cup\{(k^\omega(p),\omega):\omega\in\Omega\}. Π eval ( p ) = {( π h , ω ) : h ∈ H , ω ∈ Ω } ∪ {( k ω ( p ) , ω ) : ω ∈ Ω } . For ε , δ ∈ ( 0 , 1 ) \varepsilon,\delta\in(0,1) ε , δ ∈ ( 0 , 1 ) , whenever
m ≳ ( 1 + B c ) 2 γ 2 ε 2 [ ( d R + d Φ + log 2 ∣ G ∣ + ∣ O ∣ ) log ( 2 ∣ A ∣ ) + log 2 ∣ C ∣ δ ] , m\gtrsim\frac{(1+B_c)^2}{\gamma^2\varepsilon^2}\left[(d_{\mathcal{R}}+d_{\Phi}+\log_2|\mathcal{G}|+|\mathcal{O}|)\log(2|\mathcal{A}|)+\log\frac{2|\mathcal{C}|}{\delta}\right], m ≳ γ 2 ε 2 ( 1 + B c ) 2 [ ( d R + d Φ + log 2 ∣ G ∣ + ∣ O ∣ ) log ( 2∣ A ∣ ) + log δ 2∣ C ∣ ] , with a sufficiently large universal implicit constant, m m m independent samples from the logging distribution satisfy, with probability at least 1 − δ 1-\delta 1 − δ ,
sup ( π , ω ) ∈ Π e v a l ( p ) ∣ G ^ m ( π ; ω ; p ) − G ( π ; ω ; p ) ∣ ≤ ε 8 . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}(p)}\left|\widehat{G}_m(\pi;\omega;p)-G(\pi;\omega;p)\right|\leq\frac{\varepsilon}{8}. ( π , ω ) ∈ Π eval ( p ) sup G m ( π ; ω ; p ) − G ( π ; ω ; p ) ≤ 8 ε . Consequently, the auditor that approximately maximizes the absolute empirical COI gap to additive accuracy ε / 8 \varepsilon/8 ε /8 and uses threshold 3 ε / 4 3\varepsilon/4 3 ε /4 is an ( Ω , H , ε , δ , m ) (\Omega,\mathcal{H},\varepsilon,\delta,m) ( Ω , H , ε , δ , m ) -COI auditor.
Proof. We condition on the predictor p p p , which is independent of the audit batch. The Hypothesis-OI analysis is unchanged from the proof of Theorem 4.2 . We establish a uniform convergence bound for the deterministic post-processing policies and then combine the two bounds.
Step 1: Sample Complexity for Decision-OI.
Fix c ∈ C c \in\mathcal{C} c ∈ C . Consider the class of scalar IPS contributions
F c = { ( z , a , o ) ↦ k ( ℓ ~ , c ) ( p ) ( a ∣ z ) π l o g ( a ∣ z ) ℓ a ( ℓ ~ , c ) ( z , o ) − ∑ a ′ ∈ A k ( ℓ ~ , c ) ( p ) ( a ′ ∣ z ) E o ′ ∼ p ( ⋅ ∣ z , a ′ ) [ ℓ a ′ ( ℓ ~ , c ) ( z , o ′ ) ] : ℓ ~ ∈ [ 0 , 1 ] ∣ O ∣ } . \mathcal{F}_c = \left\{(z,a,o) \mapsto\frac{k^{(\tilde{\ell},c)}(p)(a \mid z)}{\pi_{\mathrm{log}}(a \mid z)}\ell_a^{(\tilde{\ell},c)}(z,o) - \sum_{a' \in\mathcal{A}} k^{(\tilde{\ell},c)}(p)(a' \mid z)\mathbb{E}_{o' \sim p(\cdot\mid z,a')}[\ell_{a'}^{(\tilde{\ell},c)}(z,o')] : \tilde{\ell} \in[0,1]^{|\mathcal{O}|}\right\}. F c = { ( z , a , o ) ↦ π log ( a ∣ z ) k ( ℓ ~ , c ) ( p ) ( a ∣ z ) ℓ a ( ℓ ~ , c ) ( z , o ) − a ′ ∈ A ∑ k ( ℓ ~ , c ) ( p ) ( a ′ ∣ z ) E o ′ ∼ p ( ⋅ ∣ z , a ′ ) [ ℓ a ′ ( ℓ ~ , c ) ( z , o ′ )] : ℓ ~ ∈ [ 0 , 1 ] ∣ O ∣ } .
We show that
Pdim ( F c ) ≲ ∣ O ∣ log ( 2 ∣ A ∣ ) . \operatorname{Pdim}(\mathcal{F}_c) \lesssim|\mathcal{O}|\log(2|\mathcal{A}|). Pdim ( F c ) ≲ ∣ O ∣ log ( 2∣ A ∣ ) .
To this end, fix arbitrary logged observations ( z i , a i , o i ) i = 1 n (z_i,a_i,o_i)_{i=1}^n ( z i , a i , o i ) i = 1 n and thresholds t 1 , … , t n ∈ R t_1,\ldots,t_n \in\mathbb{R} t 1 , … , t n ∈ R . For each action a ∈ A a \in\mathcal{A} a ∈ A , define
g i ( a ; ℓ ~ ) = 1 { a i = a } π l o g ( a i ∣ z i ) ℓ a ( ℓ ~ , c ) ( z i , o i ) − E o ∼ p ( ⋅ ∣ z i , a ) [ ℓ a ( ℓ ~ , c ) ( z i , o ) ] . g_i(a;\tilde{\ell}) = \frac{\mathbf{1}\{a_i=a\}}{\pi_{\mathrm{log}}(a_i \mid z_i)}\ell_a^{(\tilde{\ell},c)}(z_i,o_i) - \mathbb{E}_{o\sim p(\cdot\mid z_i,a)}[\ell_a^{(\tilde{\ell},c)}(z_i,o)]. g i ( a ; ℓ ~ ) = π log ( a i ∣ z i ) 1 { a i = a } ℓ a ( ℓ ~ , c ) ( z i , o i ) − E o ∼ p ( ⋅ ∣ z i , a ) [ ℓ a ( ℓ ~ , c ) ( z i , o )] .
For fixed c c c , this expression is affine in ℓ ~ ∈ R ∣ O ∣ \tilde{\ell} \in\mathbb{R}^{|\mathcal{O}|} ℓ ~ ∈ R ∣ O ∣ . The scalar contribution of the post-processing policy at observation i i i is
g i ( k ( ℓ ~ , c ) ( p ) , ℓ ~ ) = ∑ a ∈ A k ( ℓ ~ , c ) ( p ) ( a ∣ z i ) g i ( a ; ℓ ~ ) . g_i(k^{(\tilde{\ell},c)}(p),\tilde{\ell}) = \sum_{a\in\mathcal{A}} k^{(\tilde{\ell},c)}(p)(a\mid z_i)g_i(a;\tilde{\ell}). g i ( k ( ℓ ~ , c ) ( p ) , ℓ ~ ) = a ∈ A ∑ k ( ℓ ~ , c ) ( p ) ( a ∣ z i ) g i ( a ; ℓ ~ ) .
Since the policy is deterministic, this equals g i ( a ; ℓ ~ ) g_i(a;\tilde{\ell}) g i ( a ; ℓ ~ ) for its selected action.
The selected action at z i z_i z i is determined by the signs of the pairwise predicted-loss differences
∑ o ∈ O p ( o ∣ z i , a ) ℓ a ( ℓ ~ , c ) ( z i , o ) − ∑ o ∈ O p ( o ∣ z i , a ′ ) ℓ a ′ ( ℓ ~ , c ) ( z i , o ) , a ≠ a ′ . \sum_{o\in\mathcal{O}}p(o\mid z_i,a)\ell_a^{(\tilde{\ell},c)}(z_i,o) - \sum_{o\in\mathcal{O}}p(o\mid z_i,a')\ell_{a'}^{(\tilde{\ell},c)}(z_i,o), \qquad a\ne a'. o ∈ O ∑ p ( o ∣ z i , a ) ℓ a ( ℓ ~ , c ) ( z i , o ) − o ∈ O ∑ p ( o ∣ z i , a ′ ) ℓ a ′ ( ℓ ~ , c ) ( z i , o ) , a = a ′ .
Each difference is affine in ℓ ~ \tilde{\ell} ℓ ~ . Their signs, including zero signs, determine the minimizing actions, and the fixed ordering determines which minimizer is selected. Once that action is determined, whether the scalar contribution exceeds t i t_i t i is determined by the sign of
g i ( a ; ℓ ~ ) − t i . g_i(a;\tilde{\ell})-t_i. g i ( a ; ℓ ~ ) − t i .
Thus all n n n threshold comparisons are determined by the signs of at most
n ( ∣ A ∣ ( ∣ A ∣ − 1 ) + ∣ A ∣ ) = n ∣ A ∣ 2 n\bigl(|\mathcal{A}|(|\mathcal{A}|-1)+|\mathcal{A}|\bigr)=n|\mathcal{A}|^2 n ( ∣ A ∣ ( ∣ A ∣ − 1 ) + ∣ A ∣ ) = n ∣ A ∣ 2
affine functions of the ∣ O ∣ |\mathcal{O}| ∣ O ∣ loss coordinates. For n ≥ ∣ O ∣ n\ge|\mathcal{O}| n ≥ ∣ O ∣ , the sign-pattern bound, including zero signs, gives at most
( 8 e n ∣ A ∣ 2 ∣ O ∣ ) ∣ O ∣ \left(\frac{8e n|\mathcal{A}|^2}{|\mathcal{O}|}\right)^{|\mathcal{O}|} ( ∣ O ∣ 8 e n ∣ A ∣ 2 ) ∣ O ∣
possible sign patterns [17 Corollary 2.1] . Therefore, if F c \mathcal{F}_c F c pseudo-shatters these n n n observations, then
2 n ≤ ( 8 e n ∣ A ∣ 2 ∣ O ∣ ) ∣ O ∣ 2^n \le\left(\frac{8 e n|\mathcal{A}|^2}{|\mathcal{O}|}\right)^{|\mathcal{O}|} 2 n ≤ ( ∣ O ∣ 8 e n ∣ A ∣ 2 ) ∣ O ∣
This implies n ≲ ∣ O ∣ log ( 2 ∣ A ∣ ) n \lesssim|\mathcal{O}|\log(2|\mathcal{A}|) n ≲ ∣ O ∣ log ( 2∣ A ∣ ) . The same bound holds trivially when n < ∣ O ∣ n < |\mathcal{O}| n < ∣ O ∣ , which establishes the claimed pseudo-dimension bound.
Every member of F c \mathcal{F}_c F c has absolute value at most
( 1 + B c ) ( 1 + 1 γ ) ≤ 2 ( 1 + B c ) γ . (1+B_c)\left(1+\frac{1}{\gamma}\right) \le\frac{2(1+B_c)}{\gamma}. ( 1 + B c ) ( 1 + γ 1 ) ≤ γ 2 ( 1 + B c ) .
Applying the VC-subgraph covering bound and Dudley’s entropy integral, followed by the standard Rademacher generalization bound and a union bound over c ∈ C c \in\mathcal{C} c ∈ C , gives, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 ,
sup ω ∈ Ω ∣ G ^ m ( k ω ( p ) ; ω ; p ) − G ( k ω ( p ) ; ω ; p ) ∣ ≤ 1 + B c γ m ∣ O ∣ log ( 2 ∣ A ∣ ) + log 2 ∣ C ∣ δ . \sup_{\omega\in\Omega}\left|\widehat{G}_m(k^\omega(p);\omega;p)-G(k^\omega(p);\omega;p)\right|
\le\frac{1+B_c}{\gamma\sqrt{m}}\sqrt{|\mathcal{O}|\log(2|\mathcal{A}|)+\log\frac{2|\mathcal{C}|}{\delta}}. ω ∈ Ω sup G m ( k ω ( p ) ; ω ; p ) − G ( k ω ( p ) ; ω ; p ) ≤ γ m 1 + B c ∣ O ∣ log ( 2∣ A ∣ ) + log δ 2∣ C ∣ .
Step 2: Combining Hypothesis-OI and Decision-OI. The Hypothesis-OI argument in the proof of Theorem 4.2 gives, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 ,
sup h ∈ H , ω ∈ Ω ∣ G ^ m ( π h ; ω ; p ) − G ( π h ; ω ; p ) ∣ ≤ 1 + B c γ m d H log ( 2 ∣ A ∣ ) + ∣ O ∣ + log 2 ∣ C ∣ δ . \sup_{\substack{h\in\mathcal{H},\ \omega\in\Omega}}\left|\widehat{G}_m(\pi_h;\omega;p)-G(\pi_h;\omega;p)\right|
\le\frac{1+B_c}{\gamma\sqrt{m}}\sqrt{d_{\mathcal{H}}\log(2|\mathcal{A}|)+|\mathcal{O}|+\log\frac{2|\mathcal{C}|}{\delta}}. h ∈ H , ω ∈ Ω sup G m ( π h ; ω ; p ) − G ( π h ; ω ; p ) ≤ γ m 1 + B c d H log ( 2∣ A ∣ ) + ∣ O ∣ + log δ 2∣ C ∣ .
Combining the two bounds, with probability at least 1 − δ 1-\delta 1 − δ ,
sup ( π , ω ) ∈ Π e v a l ( p ) ∣ G ^ m ( π ; ω ; p ) − G ( π ; ω ; p ) ∣ ≤ 1 + B c γ m ( d H + ∣ O ∣ ) log ( 2 ∣ A ∣ ) + log 2 ∣ C ∣ δ . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}(p)}\left|\widehat{G}_m(\pi;\omega;p)-G(\pi;\omega;p)\right|
\le\frac{1+B_c}{\gamma\sqrt{m}}\sqrt{(d_{\mathcal{H}}+|\mathcal{O}|)\log(2|\mathcal{A}|)+\log\frac{2|\mathcal{C}|}{\delta}}. ( π , ω ) ∈ Π eval ( p ) sup G m ( π ; ω ; p ) − G ( π ; ω ; p ) ≤ γ m 1 + B c ( d H + ∣ O ∣ ) log ( 2∣ A ∣ ) + log δ 2∣ C ∣ .
Using d H ≤ d R + d Φ + log 2 ∣ G ∣ d_{\mathcal{H}}\le d_{\mathcal{R}}+d_{\Phi}+\log_2|\mathcal{G}| d H ≤ d R + d Φ + log 2 ∣ G ∣ , the stated sample-size condition makes this error at most ε / 8 \varepsilon/8 ε /8 .
On this event, let ( π , ω ) (\pi,\omega) ( π , ω ) be a pair whose absolute empirical gap is within ε / 8 \varepsilon/8 ε /8 of the supremum over Π e v a l ( p ) \Pi_{\mathrm{eval}}(p) Π eval ( p ) . If its absolute empirical gap exceeds 3 ε / 4 3\varepsilon/4 3 ε /4 , return the pair with σ = sign ( G ^ m ( π ; ω ; p ) ) \sigma=\operatorname{sign}(\widehat{G}_m(\pi;\omega;p)) σ = sign ( G m ( π ; ω ; p )) . Then
σ G ( π ; ω ; p ) ≥ ∣ G ^ m ( π ; ω ; p ) ∣ − ε 8 − 5 ε 8 ≥ ε 2 . \sigma G(\pi;\omega;p)\ge|\widehat{G}_m(\pi;\omega;p)|-\frac{\varepsilon}{8}-\frac{5\varepsilon}{8}\ge\frac{\varepsilon}{2}. σ G ( π ; ω ; p ) ≥ ∣ G m ( π ; ω ; p ) ∣ − 8 ε − 8 5 ε ≥ 2 ε .
Otherwise, the supremum of the absolute empirical gap is at most 7 ε / 8 7\varepsilon/8 7 ε /8 , and hence
sup ( π , ω ) ∈ Π e v a l ( p ) ∣ G ( π ; ω ; p ) ∣ ≤ 7 ε 8 + ε 8 = ε . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}(p)}|G(\pi;\omega;p)|
\le\frac{7\varepsilon}{8}+\frac{\varepsilon}{8}=\varepsilon. ( π , ω ) ∈ Π eval ( p ) sup ∣ G ( π ; ω ; p ) ∣ ≤ 8 7 ε + 8 ε = ε .
Returning ⊥ \bot ⊥ therefore certifies ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -COI. This completes the proof. ■
To complement the upper bounds provided in Theorem 4.2 , we establish a lower bound on sample complexity. This ensures that the sample complexity’s dependence on the Natarajan dimension d H d_{\mathcal{H}} d H and the logging policy’s support constant γ \gamma γ is inherent to the problem of offline collaborative omniprediction, rather than an artifact of our specific auditing algorithm.
Theorem B.10 (Sample Complexity Lower Bound for Offline Omnimechanism Learning). Let A \mathcal{A} A and O \mathcal{O} O be finite action and outcome spaces, with ∣ A ∣ , ∣ O ∣ ≥ 2 |\mathcal{A}|,|\mathcal{O}| \ge2 ∣ A ∣ , ∣ O ∣ ≥ 2 . Let H \mathcal{H} H be a predefined hypothesis class of deterministic policies with finite Natarajan dimension d H ≥ 1 d_{\mathcal{H}} \ge1 d H ≥ 1 . Assume that Ω \Omega Ω contains a loss-cost pair ω = ( l ~ , 0 ) \omega= (\tilde{l},0) ω = ( l ~ , 0 ) , where l ~ ( o ) = I { o = o 1 } \tilde{l}(o) = \mathbb{I}\{o = o_1\} l ~ ( o ) = I { o = o 1 } for some o 1 ∈ O o_1 \in\mathcal{O} o 1 ∈ O .
For any ϵ ∈ ( 0 , 1 / 64 ] \epsilon\in(0,1/64] ϵ ∈ ( 0 , 1/64 ] , δ ∈ ( 0 , 1 / 4 ) \delta\in(0,1/4) δ ∈ ( 0 , 1/4 ) , and γ ∈ ( 0 , 1 / ∣ A ∣ ] \gamma\in(0,1/|\mathcal{A}|] γ ∈ ( 0 , 1/∣ A ∣ ] , there exist a context distribution and a known logging policy with minimum support γ \gamma γ such that any algorithm that, uniformly over the outcome environments, outputs an ( Ω , H , ϵ ) (\Omega,\mathcal{H},\epsilon) ( Ω , H , ϵ ) -collaborative omnimechanism with probability at least 1 − δ 1-\delta 1 − δ requires an offline dataset of size
m = Ω ( d H + log ( 1 / δ ) γ ϵ 2 ) . m = \Omega\left(\frac{d_{\mathcal{H}}+\log(1/\delta)}{\gamma\epsilon^2}\right). m = Ω ( γ ϵ 2 d H + log ( 1/ δ ) ) .
Proof. The proof proceeds by reduction. We construct a family of outcome distributions, which we refer to as environments to distinguish them from the single ground-truth distribution p ∗ p^* p ∗ in the main text. We show that an algorithm producing an ( Ω , H , ϵ ) (\Omega,\mathcal{H},\epsilon) ( Ω , H , ϵ ) -collaborative omnimechanism must identify these environments with small Hamming error. We then establish the lower bound using information-theoretic inequalities.
Step 1: Construction of the State Space and Logging Policy.
Write d = d H d=d_{\mathcal{H}} d = d H . By the definition of Natarajan dimension, there exist states Z h a r d = { z 1 , … , z d } \mathcal{Z}_{\mathrm{hard}}=\{z_1,\ldots,z_d\} Z hard = { z 1 , … , z d } and two distinct actions a 0 , j , a 1 , j ∈ A a_{0,j},a_{1,j}\in\mathcal{A} a 0 , j , a 1 , j ∈ A at each state z j z_j z j such that every selection of one action from each pair is realized by some policy in H \mathcal{H} H . Let the context distribution be uniform on these states:
P ( z = z j ) = 1 d , j ∈ { 1 , … , d } . \mathbb{P}(z=z_j)=\frac{1}{d},\qquad j\in\{1,\ldots,d\}. P ( z = z j ) = d 1 , j ∈ { 1 , … , d } .
Choose the logging policy
π l o g ( a ∣ z j ) = { γ , a = a 1 , j , 1 − γ ∣ A ∣ − 1 , a ≠ a 1 , j . \pi_{\mathrm{log}}(a\mid z_j)=
\begin{cases}
\gamma, & a=a_{1,j},\\
\frac{1-\gamma}{|\mathcal{A}|-1}, & a\ne a_{1,j}.
\end{cases} π log ( a ∣ z j ) = { γ , ∣ A ∣ − 1 1 − γ , a = a 1 , j , a = a 1 , j .
Since γ ≤ 1 / ∣ A ∣ \gamma\le1/|\mathcal{A}| γ ≤ 1/∣ A ∣ , every action has logging probability at least γ \gamma γ . Outside Z h a r d \mathcal{Z}_{\mathrm{hard}} Z hard , extend the logging policy arbitrarily while preserving this coverage condition.
Choose an outcome o 0 ∈ O o_0\in\mathcal{O} o 0 ∈ O distinct from o 1 o_1 o 1 . We parameterize the environments by v ∈ { − 1 , 1 } d v\in\{-1,1\}^d v ∈ { − 1 , 1 } d . Each environment is supported on { o 0 , o 1 } \{o_0,o_1\} { o 0 , o 1 } and satisfies
P v ( o 1 ∣ z j , a ) = { 1 2 − 16 ϵ v j , a = a 1 , j , 1 2 , a ≠ a 1 , j , P v ( o 0 ∣ z j , a ) = 1 − P v ( o 1 ∣ z j , a ) . P_v(o_1\mid z_j,a)=
\begin{cases}
\frac{1}{2}-16\epsilon v_j, & a=a_{1,j},\\
\frac{1}{2}, & a\ne a_{1,j},
\end{cases}
\qquad
P_v(o_0\mid z_j,a)=1-P_v(o_1\mid z_j,a). P v ( o 1 ∣ z j , a ) = { 2 1 − 16 ϵ v j , 2 1 , a = a 1 , j , a = a 1 , j , P v ( o 0 ∣ z j , a ) = 1 − P v ( o 1 ∣ z j , a ) .
Because ϵ ≤ 1 / 64 \epsilon\le1/64 ϵ ≤ 1/64 , these probabilities lie in [ 1 / 4 , 3 / 4 ] [1/4,3/4] [ 1/4 , 3/4 ] . Consider the loss-cost pair ω = ( l ~ , 0 ) \omega=(\tilde{l},0) ω = ( l ~ , 0 ) in the theorem statement. Since l ~ ( o ) = I { o = o 1 } \tilde{l}(o)=\mathbb{I}\{o=o_1\} l ~ ( o ) = I { o = o 1 } , the expected loss of action a 1 , j a_{1,j} a 1 , j is 1 / 2 − 16 ϵ v j 1/2-16\epsilon v_j 1/2 − 16 ϵ v j , while every other action has expected loss 1 / 2 1/2 1/2 .
When v j = 1 v_j=1 v j = 1 , the optimal action is a 1 , j a_{1,j} a 1 , j . When v j = − 1 v_j=-1 v j = − 1 , the action a 0 , j a_{0,j} a 0 , j is optimal. By Natarajan shattering, H \mathcal{H} H contains a policy making these optimal choices simultaneously at all selected states.
Write V ( π ; P v ) V(\pi;P_v) V ( π ; P v ) for the expected loss of policy π \pi π under environment P v P_v P v and the fixed loss-cost pair ω \omega ω . For any randomized policy π \pi π ,
V ( π ; P v ) − min h ∈ H V ( h ; P v ) = 16 ε d ∑ j = 1 d [ I { v j = 1 } ( 1 − π ( a 1 , j ∣ z j ) ) + I { v j = − 1 } π ( a 1 , j ∣ z j ) ] . V(\pi;P_v)-\min_{h\in\mathcal{H}}V(h;P_v)=\frac{16\varepsilon}{d}\sum_{j=1}^{d}\left[\mathbb{I}\{v_j=1\}\left(1-\pi(a_{1,j}\mid z_j)\right)+\mathbb{I}\{v_j=-1\}\pi(a_{1,j}\mid z_j)\right]. V ( π ; P v ) − h ∈ H min V ( h ; P v ) = d 16 ε j = 1 ∑ d [ I { v j = 1 } ( 1 − π ( a 1 , j ∣ z j ) ) + I { v j = − 1 } π ( a 1 , j ∣ z j ) ] .
Step 3: Reduction from Omnimechanism Learning to Environment Identification.
Suppose the learner outputs a base predictive distribution p p p , with post-processed policy π ω = k ω ( p ) \pi^\omega=k^\omega(p) π ω = k ω ( p ) . Whenever p p p is an ( Ω , H , ε ) (\Omega,\mathcal{H},\varepsilon) ( Ω , H , ε ) -collaborative omnimechanism,
V ( π ω ; P v ) − min h ∈ H V ( h ; P v ) ≤ ε . V(\pi^\omega;P_v)-\min_{h\in\mathcal{H}}V(h;P_v)\leq\varepsilon. V ( π ω ; P v ) − h ∈ H min V ( h ; P v ) ≤ ε .
Define
v ^ j = { 1 , π ω ( a 1 , j ∣ z j ) ≥ 1 / 2 , − 1 , π ω ( a 1 , j ∣ z j ) < 1 / 2 , \widehat{v}_j=
\begin{cases}
1, & \pi^\omega(a_{1,j}\mid z_j)\geq1/2,\\
-1, & \pi^\omega(a_{1,j}\mid z_j)<1/2,
\end{cases} v j = { 1 , − 1 , π ω ( a 1 , j ∣ z j ) ≥ 1/2 , π ω ( a 1 , j ∣ z j ) < 1/2 ,
and let E = { j : v ^ j ≠ v j } E=\{j:\widehat{v}_j\ne v_j\} E = { j : v j = v j } . For every j ∈ E j\in E j ∈ E , the deployed policy places probability at least 1 / 2 1/2 1/2 on suboptimal actions at state z j z_j z j . The excess-loss identity from Step 2 therefore gives
8 ε d ∣ E ∣ ≤ V ( π ω ; P v ) − min h ∈ H V ( h ; P v ) ≤ ε . \frac{8\varepsilon}{d}|E|\leq V(\pi^\omega;P_v)-\min_{h\in\mathcal{H}}V(h;P_v)\leq\varepsilon. d 8 ε ∣ E ∣ ≤ V ( π ω ; P v ) − h ∈ H min V ( h ; P v ) ≤ ε .
Consequently, whenever the learner succeeds,
Hamming ( v ^ , v ) = ∣ E ∣ ≤ d 8 . \operatorname{Hamming}(\widehat{v},v)=|E|\leq\frac{d}{8}. Hamming ( v , v ) = ∣ E ∣ ≤ 8 d .
Since the learner succeeds with probability at least 1 − δ 1-\delta 1 − δ under every environment and the Hamming error is always at most d d d ,
sup v E v [ Hamming ( v ^ , v ) ] ≤ d 8 + δ d < 3 d 8 . \sup_v\mathbb{E}_v[\operatorname{Hamming}(\widehat{v},v)]\leq\frac{d}{8}+\delta d<\frac{3d}{8}. v sup E v [ Hamming ( v , v )] ≤ 8 d + δ d < 8 3 d .
Step 4: Information-Theoretic Distinguishability.
Let P v P_v P v denote the distribution of one logged sample ( z , a , o ) (z,a,o) ( z , a , o ) under the context distribution, logging policy, and environment P v P_v P v constructed above. Consider two environments v , v ′ v,v' v , v ′ differing only in coordinate j j j . Their logged-sample distributions differ only when z = z j z=z_j z = z j and a = a 1 , j a=a_{1,j} a = a 1 , j , so
D K L ( P v ∥ P v ′ ) = γ d D K L ( Ber ( 1 2 − 16 ε v j ) ∥ Ber ( 1 2 + 16 ε v j ) ) . D_{\mathrm{KL}}(P_v\mathbin\Vert P_{v'})=\frac{\gamma}{d}D_{\mathrm{KL}}\left(\operatorname{Ber}\left(\frac{1}{2}-16\varepsilon v_j\right)\mathbin\Vert\operatorname{Ber}\left(\frac{1}{2}+16\varepsilon v_j\right)\right). D KL ( P v ∥ P v ′ ) = d γ D KL ( Ber ( 2 1 − 16 ε v j ) ∥ Ber ( 2 1 + 16 ε v j ) ) .
For Bernoulli parameters in [ 1 / 4 , 3 / 4 ] [1/4,3/4] [ 1/4 , 3/4 ] , the inequality
D K L ( Ber ( p ) ∥ Ber ( q ) ) ≤ ( p − q ) 2 q ( 1 − q ) D_{\mathrm{KL}}(\operatorname{Ber}(p)\mathbin\Vert\operatorname{Ber}(q))\leq\frac{(p-q)^2}{q(1-q)} D KL ( Ber ( p ) ∥ Ber ( q )) ≤ q ( 1 − q ) ( p − q ) 2
implies
D K L ( P v ∥ P v ′ ) ≲ γ ε 2 d . D_{\mathrm{KL}}(P_v\mathbin\Vert P_{v'})\lesssim\frac{\gamma\varepsilon^2}{d}. D KL ( P v ∥ P v ′ ) ≲ d γ ε 2 .
By additivity of KL divergence for independent samples,
D K L ( P v ⊗ m ∥ P v ′ ⊗ m ) ≲ m γ ε 2 d . D_{\mathrm{KL}}\left(\mathbb{P}_{v}^{\otimes m}\mathbin\Vert\mathbb{P}_{v'}^{\otimes m}\right)\lesssim\frac{m\gamma\varepsilon^{2}}{d}. D KL ( P v ⊗ m ∥ P v ′ ⊗ m ) ≲ d mγ ε 2 .
Here and below, the implicit constants are universal.
Step 5: Applying Assouad’s Lemma and a Two-Environment Testing Bound.
Assouad’s lemma together with Pinsker’s inequality gives
sup v E v [ Hamming ( v ^ , v ) ] ≥ d 2 ( 1 − 1 2 max Hamming ( v , v ′ ) = 1 D K L ( P v ⊗ m ∥ P v ′ ⊗ m ) ) . \sup_{v}\mathbb{E}_{v}[\operatorname{Hamming}(\widehat{v},v)]\ge\frac{d}{2}\left(1-\sqrt{\frac{1}{2}\max_{\operatorname{Hamming}(v,v')=1}D_{\mathrm{KL}}\left(\mathbb{P}_{v}^{\otimes m}\mathbin\Vert\mathbb{P}_{v'}^{\otimes m}\right)}\right). v sup E v [ Hamming ( v , v )] ≥ 2 d ( 1 − 2 1 Hamming ( v , v ′ ) = 1 max D KL ( P v ⊗ m ∥ P v ′ ⊗ m ) ) .
If the maximum adjacent-environment KL divergence is at most 1 / 8 1/8 1/8 , this lower bound is at least 3 d / 8 3d/8 3 d /8 , contradicting Step 3. Combining this observation with Step 4 yields
m ≳ d γ ε 2 . m\gtrsim\frac{d}{\gamma\varepsilon^{2}}. m ≳ γ ε 2 d .
To establish the dependence on δ \delta δ , consider the two environments
v = ( 1 , … , 1 ) , v ′ = ( − 1 , … , − 1 ) . v=(1,\ldots,1),\qquad v'=(-1,\ldots,-1). v = ( 1 , … , 1 ) , v ′ = ( − 1 , … , − 1 ) .
Whenever the learner succeeds, at most d / 8 d/8 d /8 coordinates of v ^ \widehat{v} v are incorrect. Thus a majority vote over the coordinates of v ^ \widehat{v} v distinguishes these two environments with error probability at most δ \delta δ under each environment. These environments differ only on the actions a 1 , j a_{1,j} a 1 , j , whose total logging probability is γ \gamma γ . The same KL calculation therefore gives
D K L ( P v ⊗ m ∥ P v ′ ⊗ m ) ≲ m γ ε 2 . D_{\mathrm{KL}}\left(\mathbb{P}_{v}^{\otimes m}\mathbin\Vert\mathbb{P}_{v'}^{\otimes m}\right)\lesssim m\gamma\varepsilon^{2}. D KL ( P v ⊗ m ∥ P v ′ ⊗ m ) ≲ mγ ε 2 .
On the other hand, the testing guarantee and the Bretagnolle–Huber inequality imply
1 − 2 δ ≤ TV ( P v ⊗ m , P v ′ ⊗ m ) ≤ 1 − exp ( − D K L ( P v ⊗ m ∥ P v ′ ⊗ m ) ) . 1-2\delta\le\operatorname{TV}\left(\mathbb{P}_{v}^{\otimes m},\mathbb{P}_{v'}^{\otimes m}\right)\le\sqrt{1-\exp\left(-D_{\mathrm{KL}}\left(\mathbb{P}_{v}^{\otimes m}\mathbin\Vert\mathbb{P}_{v'}^{\otimes m}\right)\right)}. 1 − 2 δ ≤ TV ( P v ⊗ m , P v ′ ⊗ m ) ≤ 1 − exp ( − D KL ( P v ⊗ m ∥ P v ′ ⊗ m ) ) .
Consequently,
m γ ε 2 ≳ log 1 4 δ ( 1 − δ ) ≳ log 1 δ , m\gamma\varepsilon^{2}\gtrsim\log\frac{1}{4\delta(1-\delta)}\gtrsim\log\frac{1}{\delta}, mγ ε 2 ≳ log 4 δ ( 1 − δ ) 1 ≳ log δ 1 ,
where the last inequality holds uniformly for δ ∈ ( 0 , 1 / 4 ) \delta\in(0,1/4) δ ∈ ( 0 , 1/4 ) . Hence,
m ≳ log ( 1 / δ ) γ ε 2 . m\gtrsim\frac{\log(1/\delta)}{\gamma\varepsilon^{2}}. m ≳ γ ε 2 log ( 1/ δ ) .
Combining the two necessary bounds and using max { d , log ( 1 / δ ) } ≥ ( d + log ( 1 / δ ) ) / 2 \max\{d,\log(1/\delta)\}\ge(d+\log(1/\delta))/2 max { d , log ( 1/ δ )} ≥ ( d + log ( 1/ δ )) /2 , we obtain
m = Ω ( d H + log ( 1 / δ ) γ ε 2 ) . m=\Omega\left(\frac{d_H+\log(1/\delta)}{\gamma\varepsilon^{2}}\right). m = Ω ( γ ε 2 d H + log ( 1/ δ ) ) .
This completes the proof. ▪
Efficiency of Decision-OI Auditing
In this section, we analyze the computational efficiency of Decision-OI auditing, following Gopalan et al. [ [20 ] . We consider a constant query cost c c c . For a state z z z , an action a = ( r , s , g ) a=(r,s,g) a = ( r , s , g ) , and an outcome o o o , the total loss is independent of z z z and can therefore be written as
ℓ a ω ( o ) = ℓ ~ ( o ) + r c , \ell^\omega_a(o)=\widetilde{\ell}(o)+rc, ℓ a ω ( o ) = ℓ ( o ) + r c ,
where ω = ( ℓ ~ , c ) \omega=(\widetilde{\ell},c) ω = ( ℓ , c ) and, with a slight abuse of notation, c c c denotes the value of the constant cost function. Recall that L = [ 0 , 1 ] ∣ O ∣ \mathcal{L}=[0,1]^{|\mathcal{O}|} L = [ 0 , 1 ] ∣ O ∣ . Given a base predictive distribution p p p and a loss-cost pair ω ∈ Ω \omega\in\Omega ω ∈ Ω , define the signed Decision-OI gap as
G ( π ω ; ω ; p ) : = E z E a ∼ π ω ( ⋅ ∣ z ) [ E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( o ) ] − E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( o ) ] ] . G(\pi^\omega;\omega;p):=\mathbb{E}_z\mathbb{E}_{a\sim\pi^\omega(\cdot\mid z)}\left[\mathbb{E}_{o\sim p^*(\cdot\mid z,a)}[\ell^\omega_a(o)]-\mathbb{E}_{o\sim p(\cdot\mid z,a)}[\ell^\omega_a(o)]\right]. G ( π ω ; ω ; p ) := E z E a ∼ π ω ( ⋅ ∣ z ) [ E o ∼ p ∗ ( ⋅ ∣ z , a ) [ ℓ a ω ( o )] − E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( o )] ] .
where the post-processing policy is
π ω ( a ∣ z ) ∝ exp ( − τ E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( o ) ] ) , \pi^\omega(a\mid z)\propto\exp\left(-\tau\mathbb{E}_{o\sim p(\cdot\mid z,a)}[\ell^\omega_a(o)]\right), π ω ( a ∣ z ) ∝ exp ( − τ E o ∼ p ( ⋅ ∣ z , a ) [ ℓ a ω ( o )] ) ,
and τ > 0 \tau>0 τ > 0 is the inverse temperature. A base predictive distribution p p p satisfies ( Ω , ε ) (\Omega,\varepsilon) ( Ω , ε ) -Decision-OI if
sup ω ∈ Ω ∣ G ( π ω ; ω ; p ) ∣ ≤ ε . \sup_{\omega\in\Omega}|G(\pi^\omega;\omega;p)|\le\varepsilon. ω ∈ Ω sup ∣ G ( π ω ; ω ; p ) ∣ ≤ ε .
Throughout this section, let m = ∣ A ∣ m=|\mathcal{A}| m = ∣ A ∣ and k = ∣ A × O ∣ k=|\mathcal{A}\times\mathcal{O}| k = ∣ A × O ∣ . We assume uniform logging, π l o g ( a ∣ z ) = 1 / m \pi_{\mathrm{log}}(a\mid z)=1/m π log ( a ∣ z ) = 1/ m for every ( z , a ) (z,a) ( z , a ) , and a fixed constant query cost c ≥ 0 c\ge0 c ≥ 0 . Thus, Ω = L × { c } = [ 0 , 1 ] ∣ O ∣ × { c } \Omega=\mathcal{L}\times\{c\}=[0,1]^{|\mathcal{O}|}\times\{c\} Ω = L × { c } = [ 0 , 1 ] ∣ O ∣ × { c } , and we take B = 1 + c B=1+c B = 1 + c as an upper bound on the total loss.
Following Gopalan et al. [ [20 ] , we reduce Decision-OI auditing to weighted-calibration auditing over a class of polynomial weight functions in a multinomial RKHS. This reduction yields an auditor with witness : the auditor returns a bounded function whose correlation with the prediction residual provides an update direction, although this function need not be induced by a loss-cost pair. Such witnesses support convergence to Decision-OI through the corresponding boosting procedure in Lemma C.4 .
Theorem C.11 establishes the following guarantee. Fix τ > 0 \tau>0 τ > 0 , ε ∈ ( 0 , 1 / e ) \varepsilon\in(0,1/e) ε ∈ ( 0 , 1/ e ) , and δ ∈ ( 0 , 1 / 3 ) \delta\in(0,1/3) δ ∈ ( 0 , 1/3 ) . There exists a constant C ( B , τ ) > 0 C(B,\tau)>0 C ( B , τ ) > 0 , depending only on B , τ B,\tau B , τ and independent of ∣ A ∣ , ∣ O ∣ |\mathcal{A}|,|\mathcal{O}| ∣ A ∣ , ∣ O ∣ , such that, with
D = ⌈ C ( B , τ ) log 2 ( 4 / ε ) ⌉ , r = D + 1 ( m B e k ) D , D=\left\lceil C(B,\tau)\log^2(4/\varepsilon)\right\rceil,\qquad r=\sqrt{D+1}(mB\sqrt{e}k)^D, D = ⌈ C ( B , τ ) log 2 ( 4/ ε ) ⌉ , r = D + 1 ( m B e k ) D ,
there exists an ( Ω , ε ε 6 m B r D + 1 ) (\Omega,\varepsilon\frac{\varepsilon}{6mBr\sqrt{D+1}}) ( Ω , ε 6 m B r D + 1 ε ) -auditor with witness for Decision-OI. For a fixed input predictor, this auditor has failure probability at most δ \delta δ , sample complexity
n = O ( k m 2 B 2 r 2 ( D + 1 ) log ( 1 / δ ) ε 2 ) , n=\mathcal{O}\left(\frac{km^2B^2r^2(D+1)\log(1/\delta)}{\varepsilon^2}\right), n = O ( ε 2 k m 2 B 2 r 2 ( D + 1 ) log ( 1/ δ ) ) ,
and running time poly ( n , k , D ) \operatorname{poly}(n,k,D) poly ( n , k , D ) .
For fixed B , τ , ε B,\tau,\varepsilon B , τ , ε , these complexities are polynomial in m m m and k k k . The reduction therefore avoids the exponential dependence on ∣ O ∣ |\mathcal{O}| ∣ O ∣ incurred by exhaustive discretization of the loss cube L = [ 0 , 1 ] ∣ O ∣ \mathcal{L}=[0,1]^{|\mathcal{O}|} L = [ 0 , 1 ] ∣ O ∣ . For fixed B , τ , m , k B,\tau,m,k B , τ , m , k , the dependence on 1 / ε 1/\varepsilon 1/ ε is quasi-polynomial.
Reduction to Weighted Calibration
We first recall the notion of weighted calibration. For a positive integer k k k , let Δ k \Delta_k Δ k denote the probability simplex on { 1 , … , k } \{1,\ldots,k\} { 1 , … , k } and let E k \mathcal{E}_k E k denote the corresponding set of one-hot vectors.
Definition C.1 (Weighted calibration, [20 ] ). Let W \mathcal{W} W be a family of functions w : Δ k → [ − 1 , 1 ] k w:\Delta_k\to[-1,1]^k w : Δ k → [ − 1 , 1 ] k . The W \mathcal{W} W -calibration error of a distribution D \mathcal{D} D on Δ k × E k \Delta_k\times\mathcal{E}_k Δ k × E k is
CE W ( D ) = sup w ∈ W ∣ E ( v , y ) ∼ D [ ⟨ y − v , w ( v ) ⟩ ] ∣ . (2) \operatorname{CE}_{\mathcal{W}}(\mathcal{D})=\sup_{w\in\mathcal{W}}\left|\mathbb{E}_{(v,y)\sim\mathcal{D}}[\langle y-v,w(v)\rangle]\right|. \tag*{(2)} CE W ( D ) = w ∈ W sup E ( v , y ) ∼ D [⟨ y − v , w ( v )⟩] . (2) We say that D \mathcal{D} D is ( W , ϵ ) (\mathcal{W},\epsilon) ( W , ϵ ) -calibrated if CE W ( D ) ≤ ϵ \operatorname{CE}_{\mathcal{W}}(\mathcal{D})\le\epsilon CE W ( D ) ≤ ϵ .
Recall that m = ∣ A ∣ m=|\mathcal{A}| m = ∣ A ∣ , k = ∣ A × O ∣ k=|\mathcal{A}\times\mathcal{O}| k = ∣ A × O ∣ , and B = 1 + c B=1+c B = 1 + c bounds the total loss. We now reduce Decision-OI to a weighted-calibration condition. Under uniform logging, draw
z ∼ D Z , a ∼ Unif ( A ) , o ∼ p ∗ ( ⋅ ∣ z , a ) . z\sim\mathcal{D}_{\mathcal{Z}},\qquad a\sim\operatorname{Unif}(\mathcal{A}),\qquad o\sim p^*(\cdot\mid z,a). z ∼ D Z , a ∼ Unif ( A ) , o ∼ p ∗ ( ⋅ ∣ z , a ) .
Given ( z , a , o ) (z,a,o) ( z , a , o ) , define vectors v \mathbf{v} v and y \mathbf{y} y , indexed by ( a ′ , o ′ ) ∈ A × O (a',o')\in\mathcal{A}\times\mathcal{O} ( a ′ , o ′ ) ∈ A × O , by
v ( a ′ , o ′ ) ( z ) = p ( o ′ ∣ z , a ′ ) m , y ( a ′ , o ′ ) ( z , a , o ) = 1 { a = a ′ , o = o ′ } . (3) \mathbf{v}_{(a',o')}(z)=\frac{p(o'\mid z,a')}{m},\qquad\mathbf{y}_{(a',o')}(z,a,o)=\mathbf{1}\{a=a',o=o'\}. \tag*{(3)} v ( a ′ , o ′ ) ( z ) = m p ( o ′ ∣ z , a ′ ) , y ( a ′ , o ′ ) ( z , a , o ) = 1 { a = a ′ , o = o ′ } . (3)
Let D c a l \mathcal{D}^{\mathrm{cal}} D cal denote the induced distribution of ( v , y ) (\mathbf{v},\mathbf{y}) ( v , y ) . Since
∑ ( a ′ , o ′ ) ∈ A × O v ( a ′ , o ′ ) ( z ) = 1 m ∑ a ′ ∈ A ∑ o ′ ∈ O p ( o ′ ∣ z , a ′ ) = 1 , \sum_{(a',o')\in\mathcal{A}\times\mathcal{O}}\mathbf{v}_{(a',o')}(z)=\frac{1}{m}\sum_{a'\in\mathcal{A}}\sum_{o'\in\mathcal{O}}p(o'\mid z,a')=1, ( a ′ , o ′ ) ∈ A × O ∑ v ( a ′ , o ′ ) ( z ) = m 1 a ′ ∈ A ∑ o ′ ∈ O ∑ p ( o ′ ∣ z , a ′ ) = 1 ,
we have v ∈ Δ k \mathbf{v}\in\Delta_k v ∈ Δ k and y ∈ E k \mathbf{y}\in\mathcal{E}_k y ∈ E k .
For each ω ∈ Ω \omega\in\Omega ω ∈ Ω and ( a ′ , o ′ ) ∈ A × O (a',o')\in\mathcal{A}\times\mathcal{O} ( a ′ , o ′ ) ∈ A × O , define the scalar weight function
W ( a ′ , o ′ ) ω ( v ) = ℓ a ′ ω ( o ′ ) B exp ( − m τ ∑ o ∈ O ℓ a ω ( o ) v ( a ′ , o ) ) ∑ a ∈ A exp ( − m τ ∑ o ∈ O ℓ a ω ( o ) v ( a , o ) ) . W^{\omega}_{(a',o')}(\mathbf{v})=\frac{\ell^\omega_{a'}(o')}{B}\frac{\exp\left(-m\tau\sum_{o\in\mathcal{O}}\ell^\omega_a(o)\mathbf{v}_{(a',o)}\right)}{\sum_{a\in\mathcal{A}}\exp\left(-m\tau\sum_{o\in\mathcal{O}}\ell^\omega_a(o)\mathbf{v}_{(a,o)}\right)}. W ( a ′ , o ′ ) ω ( v ) = B ℓ a ′ ω ( o ′ ) ∑ a ∈ A exp ( − m τ ∑ o ∈ O ℓ a ω ( o ) v ( a , o ) ) exp ( − m τ ∑ o ∈ O ℓ a ω ( o ) v ( a ′ , o ) ) .
Let
W d e c = { W ( a ′ , o ′ ) ω : ω ∈ Ω , ( a ′ , o ′ ) ∈ A × O } . \mathcal{W}_{\mathrm{dec}}=\left\{W^\omega_{(a',o')}: \omega\in\Omega,\ (a',o')\in\mathcal{A}\times\mathcal{O}\right\}. W dec = { W ( a ′ , o ′ ) ω : ω ∈ Ω , ( a ′ , o ′ ) ∈ A × O } .
Each member of W d e c \mathcal{W}_{\mathrm{dec}} W dec maps Δ k \Delta_k Δ k into [ 0 , 1 ] [0,1] [ 0 , 1 ] . We write W d e c k \mathcal{W}_{\mathrm{dec}}^k W dec k for its k k k -fold Cartesian product, viewed as a family of functions from Δ k \Delta_k Δ k to [ 0 , 1 ] k [0,1]^k [ 0 , 1 ] k . At the encoded prediction v ( z ) \mathbf{v}(z) v ( z ) , the softmax factor in W ( a ′ , o ′ ) ω W^\omega_{(a',o')} W ( a ′ , o ′ ) ω equals π ω ( a ′ ∣ z ) \pi^\omega(a'\mid z) π ω ( a ′ ∣ z ) . Consequently,
W ( a ′ , o ′ ) ω ( v ( z ) ) = ℓ a ′ ω ( o ′ ) B π ω ( a ′ ∣ z ) . W^\omega_{(a',o')}(\mathbf{v}(z))=\frac{\ell^\omega_{a'}(o')}{B}\pi^\omega(a'\mid z). W ( a ′ , o ′ ) ω ( v ( z )) = B ℓ a ′ ω ( o ′ ) π ω ( a ′ ∣ z ) .
Lemma C.2. For the distribution D c a l \mathcal{D}^{\mathrm{cal}} D cal constructed in Eq. (3) ,
sup ω ∈ Ω ∣ G ( π ω ; ω ; p ) ∣ ≤ m B CE W d e c k ( D c a l ) . \sup_{\omega\in\Omega}\left|\mathcal{G}(\pi^\omega;\omega;p)\right|\le mB\operatorname{CE}_{\mathcal{W}_{\mathrm{dec}}^k}(\mathcal{D}^{\mathrm{cal}}). ω ∈ Ω sup ∣ G ( π ω ; ω ; p ) ∣ ≤ m B CE W dec k ( D cal ) . In particular, if D c a l \mathcal{D}^{\mathrm{cal}} D cal is ( W d e c k , ϵ / ( m B ) ) (\mathcal{W}_{\mathrm{dec}}^k,\epsilon/(mB)) ( W dec k , ϵ / ( m B )) -calibrated, then p p p satisfies ( Ω , ϵ ) (\Omega,\epsilon) ( Ω , ϵ ) -Decision-OI.
Proof. Fix ω ∈ Ω \omega\in\Omega ω ∈ Ω . By construction, the tuple ( W ( a ′ , o ′ ) ω ) ( a ′ , o ′ ) ∈ A × O (W^{\omega}_{(a',o')})_{(a',o')\in\mathcal{A}\times\mathcal{O}} ( W ( a ′ , o ′ ) ω ) ( a ′ , o ′ ) ∈ A × O belongs to W d e c k \mathcal{W}^{k}_{\mathrm{dec}} W dec k . Under uniform logging,
E [ y ( a ′ , o ′ ) ∣ z ] = 1 m p ∗ ( o ′ ∣ z , a ′ ) . \mathbb{E}[y_{(a',o')} \mid z] = \frac{1}{m}p^*(o' \mid z,a'). E [ y ( a ′ , o ′ ) ∣ z ] = m 1 p ∗ ( o ′ ∣ z , a ′ ) .
Therefore,
E ( v , y ) ∼ D c a l [ ∑ ( a ′ , o ′ ) ∈ A × O W ( a ′ , o ′ ) ω ( v ) ( y ( a ′ , o ′ ) − v ( a ′ , o ′ ) ) ] = 1 m E z [ ∑ ( a ′ , o ′ ) ∈ A × O W ( a ′ , o ′ ) ω ( v ( z ) ) ( p ∗ ( o ′ ∣ z , a ′ ) − p ( o ′ ∣ z , a ′ ) ) ] = 1 m B E z [ ∑ a ′ ∈ A π ω ( a ′ ∣ z ) ∑ o ′ ∈ O ℓ a ′ ( o ′ ) ω ( p ∗ ( o ′ ∣ z , a ′ ) − p ( o ′ ∣ z , a ′ ) ) ] = 1 m B G ( π ω ; ω ; p ) . \begin{aligned}
\mathbb{E}_{(\mathbf{v},\mathbf{y})\sim\mathcal{D}^{\mathrm{cal}}}\left[\sum_{(a',o')\in\mathcal{A}\times\mathcal{O}}W^{\omega}_{(a',o')}(\mathbf{v})\left(\mathbf{y}_{(a',o')}-\mathbf{v}_{(a',o')}\right)\right] \\
&= \frac{1}{m}\mathbb{E}_{z}\left[\sum_{(a',o')\in\mathcal{A}\times\mathcal{O}}W^{\omega}_{(a',o')}(\mathbf{v}(z))\left(p^*(o'\mid z,a')-p(o'\mid z,a')\right)\right] \\
&= \frac{1}{mB}\mathbb{E}_{z}\left[\sum_{a'\in\mathcal{A}}\pi^{\omega}(a'\mid z)\sum_{o'\in\mathcal{O}}\ell^{\omega}_{a'(o')}\left(p^*(o'\mid z,a')-p(o'\mid z,a')\right)\right] \\
&= \frac{1}{mB}G(\pi^{\omega};\omega;p).
\end{aligned} E ( v , y ) ∼ D cal ( a ′ , o ′ ) ∈ A × O ∑ W ( a ′ , o ′ ) ω ( v ) ( y ( a ′ , o ′ ) − v ( a ′ , o ′ ) ) = m 1 E z ( a ′ , o ′ ) ∈ A × O ∑ W ( a ′ , o ′ ) ω ( v ( z )) ( p ∗ ( o ′ ∣ z , a ′ ) − p ( o ′ ∣ z , a ′ ) ) = m B 1 E z [ a ′ ∈ A ∑ π ω ( a ′ ∣ z ) o ′ ∈ O ∑ ℓ a ′ ( o ′ ) ω ( p ∗ ( o ′ ∣ z , a ′ ) − p ( o ′ ∣ z , a ′ ) ) ] = m B 1 G ( π ω ; ω ; p ) .
Taking absolute values and using the definition of weighted calibration gives
∣ G ( π ω ; ω ; p ) ∣ ≤ m B CE W d e c k D c a l . \left|G(\pi^{\omega};\omega;p)\right| \le mB\operatorname{CE}^{\mathcal{D}^{\mathrm{cal}}}_{\mathcal{W}^{k}_{\mathrm{dec}}}. ∣ G ( π ω ; ω ; p ) ∣ ≤ m B CE W dec k D cal .
Since this holds for every ω ∈ Ω \omega\in\Omega ω ∈ Ω , taking the supremum proves the first claim. The calibration condition then implies sup ω ∈ Ω ∣ G ( π ω ; ω ; p ) ∣ ≤ ε \sup_{\omega\in\Omega}|G(\pi^{\omega};\omega;p)|\le\varepsilon sup ω ∈ Ω ∣ G ( π ω ; ω ; p ) ∣ ≤ ε , establishing Decision-OI.
Definition C.3 (Auditor with witness for Decision-OI). Fix a loss-cost family Ω \Omega Ω , ε , β > 0 \varepsilon,\beta>0 ε , β > 0 , and δ ∈ ( 0 , 1 / 3 ) \delta\in(0,1/3) δ ∈ ( 0 , 1/3 ) . An ( Ω , ε , β ) (\Omega,\varepsilon,\beta) ( Ω , ε , β ) -auditor with witness for Decision-OI takes a base predictive distribution p p p and independent samples from its induced distribution D c a l \mathcal{D}^{\mathrm{cal}} D cal . It returns either ⊥ \bot ⊥ or a function w ′ : Δ k → [ − 1 , 1 ] k w':\Delta_k\to[-1,1]^k w ′ : Δ k → [ − 1 , 1 ] k . With probability at least 1 − δ 1-\delta 1 − δ , the output satisfies the following guarantees:
If the auditor returns ⊥ \bot ⊥ , then sup ω ∈ Ω ∣ G ( π ω ; ω ; p ) ∣ ≤ ε \sup_{\omega\in\Omega}|G(\pi^{\omega};\omega;p)|\le\varepsilon sup ω ∈ Ω ∣ G ( π ω ; ω ; p ) ∣ ≤ ε .
If the auditor returns w ′ w' w ′ , then E ( v , y ) ∼ D c a l [ ⟨ y − v , w ′ ( v ) ⟩ ] ≥ β 2 \mathbb{E}_{(\mathbf{v},\mathbf{y})\sim\mathcal{D}^{\mathrm{cal}}}[\langle\mathbf{y}-\mathbf{v},w'(\mathbf{v})\rangle]\ge\frac{\beta}{2} E ( v , y ) ∼ D cal [⟨ y − v , w ′ ( v )⟩] ≥ 2 β .
Here the returned witness need not be induced by a loss-cost pair.
To construct the auditor, first run a witness-search procedure with failure probability δ / 2 \delta/2 δ /2 . This procedure returns a bounded function w ′ : Δ k → [ − 1 , 1 ] k w':\Delta_k\to[-1,1]^k w ′ : Δ k → [ − 1 , 1 ] k and guarantees correlation at least β \beta β whenever p p p violates ( Ω , ε ) (\Omega,\varepsilon) ( Ω , ε ) -Decision-OI.
Next, use an independent validation batch of O ( β − 2 log ( 1 / δ ) ) O(\beta^{-2}\log(1/\delta)) O ( β − 2 log ( 1/ δ )) samples to estimate
E D c a l [ ⟨ y − v , w ′ ( v ) ⟩ ] \mathbb{E}_{\mathcal{D}^{\mathrm{cal}}}[\langle\mathbf{y}-\mathbf{v},w'(\mathbf{v})\rangle] E D cal [⟨ y − v , w ′ ( v )⟩]
to accuracy β / 4 \beta/4 β /4 with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 . This follows from Hoeffding’s inequality, since
∣ ⟨ y − v , w ′ ( v ) ⟩ ∣ ≤ ∥ y − v ∥ 1 ≤ 2. |\langle\mathbf{y}-\mathbf{v},w'(\mathbf{v})\rangle|\le\|\mathbf{y}-\mathbf{v}\|_1\le2. ∣ ⟨ y − v , w ′ ( v )⟩ ∣ ≤ ∥ y − v ∥ 1 ≤ 2.
The auditor returns w ′ w' w ′ if its empirical correlation is at least 3 β / 4 3\beta/4 3 β /4 , and returns ⊥ \bot ⊥ otherwise.
With probability at least 1 − δ 1-\delta 1 − δ , both the search and validation guarantees hold. On this event, every returned witness has true correlation at least 3 β / 4 − β / 4 = β / 2 3\beta/4-\beta/4=\beta/2 3 β /4 − β /4 = β /2 . If the auditor returns ⊥ \bot ⊥ , the candidate’s true correlation is strictly less than 3 β / 4 + β / 4 = β 3\beta/4+\beta/4=\beta 3 β /4 + β /4 = β . By the search guarantee, this can occur only if p p p satisfies ( Ω , ε ) (\Omega,\varepsilon) ( Ω , ε ) -Decision-OI. Thus the auditor satisfies the definition above with failure probability at most δ \delta δ .
Lemma C.4. Suppose an ( Ω , ε , β ) (\Omega,\varepsilon,\beta) ( Ω , ε , β ) -auditor with witness for Decision-OI is available at any prescribed failure probability. Fix δ ∈ ( 0 , 1 / 3 ) \delta\in(0,1/3) δ ∈ ( 0 , 1/3 ) and initialize
p ( 0 ) ( o ∣ z , a ) = 1 ∣ O ∣ , η = β 2 , T max = ⌈ 8 log ∣ O ∣ β 2 ⌉ + 1. p^{(0)}(o\mid z,a)=\frac{1}{|\mathcal{O}|},\qquad\eta=\frac{\beta}{2},\qquad T_{\max}=\left\lceil\frac{8\log|\mathcal{O}|}{\beta^2}\right\rceil+1. p ( 0 ) ( o ∣ z , a ) = ∣ O ∣ 1 , η = 2 β , T m a x = ⌈ β 2 8 log ∣ O ∣ ⌉ + 1. The following algorithm returns an ( Ω , ε ) (\Omega,\varepsilon) ( Ω , ε ) -Decision-OI predictor with probability at least 1 − δ 1-\delta 1 − δ , after at most ⌈ 8 log ∣ O ∣ / β 2 ⌉ \lceil8\log|\mathcal{O}|/\beta^2\rceil ⌈ 8 log ∣ O ∣/ β 2 ⌉ updates.
For each round 0 ≤ t < T max 0\le t<T_{\max} 0 ≤ t < T m a x :
Call the auditor on p ( t ) p^{(t)} p ( t ) with failure probability δ / T max \delta/T_{\max} δ / T m a x , using logged observations independent of all preceding calls. If it returns ⊥ \perp ⊥ , terminate and return p ( t ) p^{(t)} p ( t ) .
Otherwise, let w ( t ) w^{(t)} w ( t ) be the returned witness and define
v ( t ) ( z ) = 1 m ( p ( t ) ( o ∣ z , a ) ) ( a , o ) ∈ A × O . \mathbf{v}^{(t)}(z)=\frac{1}{m}\left(p^{(t)}(o\mid z,a)\right)_{(a,o)\in\mathcal{A}\times\mathcal{O}}. v ( t ) ( z ) = m 1 ( p ( t ) ( o ∣ z , a ) ) ( a , o ) ∈ A × O . For every ( z , a , o ) (z,a,o) ( z , a , o ) , update
p ( t + 1 ) ( o ∣ z , a ) = p ( t ) ( o ∣ z , a ) exp ( η w ( a , o ) ( t ) ( v ( t ) ( z ) ) ) ∑ o ′ ∈ O p ( t ) ( o ′ ∣ z , a ) exp ( η w ( a , o ′ ) ( t ) ( v ( t ) ( z ) ) ) . p^{(t+1)}(o\mid z,a)=\frac{p^{(t)}(o\mid z,a)\exp\left(\eta w^{(t)}_{(a,o)}(\mathbf{v}^{(t)}(z))\right)}{\sum_{o'\in\mathcal{O}}p^{(t)}(o'\mid z,a)\exp\left(\eta w^{(t)}_{(a,o')}(\mathbf{v}^{(t)}(z))\right)}. p ( t + 1 ) ( o ∣ z , a ) = ∑ o ′ ∈ O p ( t ) ( o ′ ∣ z , a ) exp ( η w ( a , o ′ ) ( t ) ( v ( t ) ( z )) ) p ( t ) ( o ∣ z , a ) exp ( η w ( a , o ) ( t ) ( v ( t ) ( z )) ) . If the iteration limit is reached, return the current predictor. On the stated success event, the auditor returns ⊥ \perp ⊥ within T max T_{\max} T m a x calls.
Proof. By independence of the batches, the auditor guarantee applies conditionally on the preceding history at every round. A union bound over at most T max T_{\max} T m a x calls shows that all auditor outputs satisfy their guarantees simultaneously with probability at least 1 − δ 1-\delta 1 − δ . We condition on this event. Returning ⊥ \perp ⊥ then certifies ( Ω , ε ) (\Omega,\varepsilon) ( Ω , ε ) -Decision-OI.
To bound the number of updates, define the potential
Φ t = E z [ 1 m ∑ a ∈ A D K L ( p ∗ ( ⋅ ∣ z , a ) ∥ p ( t ) ( ⋅ ∣ z , a ) ) ] . \Phi_t=\mathbb{E}_z\left[\frac{1}{m}\sum_{a\in\mathcal{A}}D_{\mathrm{KL}}\left(p^*(\cdot\mid z,a)\mathbin\Vert p^{(t)}(\cdot\mid z,a)\right)\right]. Φ t = E z [ m 1 a ∈ A ∑ D KL ( p ∗ ( ⋅ ∣ z , a ) ∥ p ( t ) ( ⋅ ∣ z , a ) ) ] .
Uniform initialization gives Φ 0 ≤ log ∣ O ∣ \Phi_0\le\log|\mathcal{O}| Φ 0 ≤ log ∣ O ∣ , while Φ t ≥ 0 \Phi_t\ge0 Φ t ≥ 0 for every t t t .
Whenever the auditor returns a witness,
E D c a l [ ⟨ y − v , w ( t ) ( v ) ⟩ ] ≥ β 2 , \mathbb{E}_{\mathcal{D}^{\mathrm{cal}}}\left[\langle y-\mathbf{v},w^{(t)}(\mathbf{v})\rangle\right]\ge\frac{\beta}{2}, E D cal [ ⟨ y − v , w ( t ) ( v )⟩ ] ≥ 2 β ,
where D c a l \mathcal{D}^{\mathrm{cal}} D cal denotes the distribution induced by p ( t ) p^{(t)} p ( t ) . For fixed ( z , a ) (z,a) ( z , a ) , Hoeffding’s lemma and w ( a , o ) ( t ) ∈ [ − 1 , 1 ] w^{(t)}_{(a,o)} \in[-1,1] w ( a , o ) ( t ) ∈ [ − 1 , 1 ] imply
log ( ∑ o ∈ O p ( t ) ( o ∣ z , a ) e η w ( a , o ) ( t ) ( v ( t ) ( z ) ) ) ≤ η ∑ o ∈ O p ( t ) ( o ∣ z , a ) w ( a , o ) ( t ) ( v ( t ) ( z ) ) + η 2 2 . \log\left(\sum_{o\in\mathcal{O}}p^{(t)}(o\mid z,a)e^{\eta w^{(t)}_{(a,o)}(v^{(t)}(z))}\right)\leq\eta\sum_{o\in\mathcal{O}}p^{(t)}(o\mid z,a)w^{(t)}_{(a,o)}(v^{(t)}(z))+\frac{\eta^2}{2}. log ( o ∈ O ∑ p ( t ) ( o ∣ z , a ) e η w ( a , o ) ( t ) ( v ( t ) ( z )) ) ≤ η o ∈ O ∑ p ( t ) ( o ∣ z , a ) w ( a , o ) ( t ) ( v ( t ) ( z )) + 2 η 2 .
Substituting the update into the potential gives
Φ t + 1 − Φ t ≤ η m E z [ ∑ a ∈ A ∑ o ∈ O ( p ( t ) ( o ∣ z , a ) − p ∗ ( o ∣ z , a ) ) w ( a , o ) ( t ) ( v ( t ) ( z ) ) ] + η 2 2 = − η E D c a l [ ⟨ y − v , w ( t ) ( v ) ⟩ ] + η 2 2 ≤ − η β 2 + η 2 2 = − β 2 8 . \begin{aligned}
\Phi_{t+1}-\Phi_t&\leq\frac{\eta}{m}\mathbb{E}_{z}\left[\sum_{a\in\mathcal{A}}\sum_{o\in\mathcal{O}}\left(p^{(t)}(o\mid z,a)-p^*(o\mid z,a)\right)w^{(t)}_{(a,o)}(v^{(t)}(z))\right]+\frac{\eta^2}{2}\\
&=-\eta\,\mathbb{E}_{\mathcal{D}_{\mathrm{cal}}}\left[\langle y-v,w^{(t)}(v)\rangle\right]+\frac{\eta^2}{2}\\
&\leq-\frac{\eta\beta}{2}+\frac{\eta^2}{2}=-\frac{\beta^2}{8}.
\end{aligned} Φ t + 1 − Φ t ≤ m η E z [ a ∈ A ∑ o ∈ O ∑ ( p ( t ) ( o ∣ z , a ) − p ∗ ( o ∣ z , a ) ) w ( a , o ) ( t ) ( v ( t ) ( z )) ] + 2 η 2 = − η E D cal [ ⟨ y − v , w ( t ) ( v )⟩ ] + 2 η 2 ≤ − 2 η β + 2 η 2 = − 8 β 2 .
The equality follows from uniform logging and v ( a , o ) ( t ) ( z ) = p ( t ) ( o ∣ z , a ) / m v^{(t)}_{(a,o)}(z)=p^{(t)}(o\mid z,a)/m v ( a , o ) ( t ) ( z ) = p ( t ) ( o ∣ z , a ) / m .
Since Φ 0 ≤ log ∣ O ∣ \Phi_0\leq\log|\mathcal{O}| Φ 0 ≤ log ∣ O ∣ and the potential is nonnegative, there can be at most 8 log ∣ O ∣ / β 2 8\log|\mathcal{O}|/\beta^2 8 log ∣ O ∣/ β 2 updates. Consequently, the auditor must return ⊥ \perp ⊥ within T max T_{\max} T m a x calls, and its guarantee establishes Decision-OI for the returned predictor. ■
Efficient Auditing for Weighted Calibration
Lemma C.2 expresses Decision-OI violations as weighted-calibration violations. We approximate the associated weight functions by polynomials with controlled RKHS norms and apply a kernel auditing algorithm to obtain an efficient Decision-OI auditor.
Definition C.5 (Multinomial kernel, [20 ] ). For a nonnegative integer D D D , let T D = [ k ] 0 ∪ [ k ] 1 ∪ ⋯ ∪ [ k ] D T_D=[k]^0\cup[k]^1\cup\cdots\cup[k]^D T D = [ k ] 0 ∪ [ k ] 1 ∪ ⋯ ∪ [ k ] D . For v = ( v 1 , … , v k ) ∈ Δ k \mathbf{v}=(v_1,\ldots,v_k)\in\Delta_k v = ( v 1 , … , v k ) ∈ Δ k and a tuple t = ( t 1 , … , t j ) ∈ [ k ] j t=(t_1,\ldots,t_j)\in[k]^j t = ( t 1 , … , t j ) ∈ [ k ] j , define v t = v t 1 ⋯ v t j v_t=v_{t_1}\cdots v_{t_j} v t = v t 1 ⋯ v t j , with the empty product equal to 1 1 1 when j = 0 j=0 j = 0 . Define ψ : Δ k → R 1 + k + ⋯ + k D \psi:\Delta_k\to\mathbb{R}^{1+k+\cdots+k^D} ψ : Δ k → R 1 + k + ⋯ + k D by assigning the coordinate v t v_t v t to each t ∈ T D t\in T_D t ∈ T D . The degree-D D D multinomial kernel is
ker D ( v , v ′ ) = ∑ j = 0 D ⟨ v , v ′ ⟩ j = ⟨ ψ ( v ) , ψ ( v ′ ) ⟩ . (4) \operatorname{ker}_D(\mathbf{v},\mathbf{v}')=\sum_{j=0}^{D}\langle\mathbf{v},\mathbf{v}'\rangle^j=\langle\psi(\mathbf{v}),\psi(\mathbf{v}')\rangle. \tag*{(4)} ker D ( v , v ′ ) = j = 0 ∑ D ⟨ v , v ′ ⟩ j = ⟨ ψ ( v ) , ψ ( v ′ )⟩ . (4) We denote its reproducing kernel Hilbert space by Γ ( D ) \Gamma(D) Γ ( D ) and its radius-r r r ball by
B Γ ( D ) ( r ) = { w ∈ Γ ( D ) : ∥ w ∥ Γ ( D ) ≤ r } . B_{\Gamma(D)}(r)=\{w\in\Gamma(D):\|w\|_{\Gamma(D)}\leq r\}. B Γ ( D ) ( r ) = { w ∈ Γ ( D ) : ∥ w ∥ Γ ( D ) ≤ r } .
Lemma C.6 (Kernel auditing, [20 ] , Theorem 33). For every r > 0 r>0 r > 0 , nonnegative integer D D D , ε > 0 \varepsilon>0 ε > 0 , and δ ∈ ( 0 , 1 / 3 ) \delta\in(0,1/3) δ ∈ ( 0 , 1/3 ) , there exists an algorithm using
n = O ( k r 2 ( D + 1 ) log ( 1 / δ ) ε 2 ) n=O\left(\frac{kr^2(D+1)\log(1/\delta)}{\varepsilon^2}\right) n = O ( ε 2 k r 2 ( D + 1 ) log ( 1/ δ ) ) i.i.d. samples from a distribution D \mathcal{D} D on Δ k × E k \Delta_k\times\mathcal{E}_k Δ k × E k and running in time poly ( n , k , D ) \operatorname{poly}(n,k,D) poly ( n , k , D ) . The algorithm always returns a function
w ∈ ( B Γ ( D ) ( 1 D + 1 ) ) k . w\in\left(B_{\Gamma(D)}\left(\frac{1}{\sqrt{D+1}}\right)\right)^k. w ∈ ( B Γ ( D ) ( D + 1 1 ) ) k .
If there exists w ∈ ( B Γ ( D ) ( r ) ) k w \in(B_{\Gamma(D)}(r))^k w ∈ ( B Γ ( D ) ( r ) ) k such that
∣ E ( v , y ) ∼ D [ ⟨ y − v , w ( v ) ⟩ ] ∣ > ε , \left|\mathbb{E}_{(v,y)\sim D}[\langle y-v,w(v)\rangle]\right|>\varepsilon, E ( v , y ) ∼ D [⟨ y − v , w ( v )⟩] > ε ,
then, with probability at least 1 − δ 1-\delta 1 − δ ,
E ( v , y ) ∼ D [ ⟨ y − v , w ′ ( v ) ⟩ ] ≥ ε 3 r D + 1 . \mathbb{E}_{(v,y)\sim D}[\langle y-v,w'(v)\rangle]\ge\frac{\varepsilon}{3r\sqrt{D+1}}. E ( v , y ) ∼ D [⟨ y − v , w ′ ( v )⟩] ≥ 3 r D + 1 ε .
Moreover, w ′ w' w ′ takes values in [ − 1 , 1 ] k [-1,1]^k [ − 1 , 1 ] k .
Recall that m = ∣ A ∣ m=|A| m = ∣ A ∣ , k = ∣ A × O ∣ k=|A\times O| k = ∣ A × O ∣ , and B = 1 + c ≥ 1 B=1+c\ge1 B = 1 + c ≥ 1 bounds the total loss. Throughout the following analysis, we assume τ > 0 \tau>0 τ > 0 and ε ∈ ( 0 , 1 / e ) \varepsilon\in(0,1/e) ε ∈ ( 0 , 1/ e ) .
Lemma C.7. For i ∈ [ m ] i\in[m] i ∈ [ m ] , let σ i : [ 0 , B ] m → [ 0 , 1 ] \sigma_i:[0,B]^m\to[0,1] σ i : [ 0 , B ] m → [ 0 , 1 ] be the i i i -th component of the softmax function with inverse temperature τ \tau τ :
σ i ( t 1 , … , t m ) = exp ( − τ t i ) ∑ j = 1 m exp ( − τ t j ) . \sigma_i(t_1,\ldots,t_m)=\frac{\exp(-\tau t_i)}{\sum_{j=1}^{m}\exp(-\tau t_j)}. σ i ( t 1 , … , t m ) = ∑ j = 1 m exp ( − τ t j ) exp ( − τ t i ) . There exists a constant C 0 ( B , τ ) > 0 C_0(B,\tau)>0 C 0 ( B , τ ) > 0 , independent of m m m and k k k , and a polynomial p p p of degree at most C 0 ( B , τ ) log 2 ( 1 / ε ) C_0(B,\tau)\log^2(1/\varepsilon) C 0 ( B , τ ) log 2 ( 1/ ε ) such that
∣ p ( t 1 , … , t m ) − σ i ( t 1 , … , t m ) ∣ ≤ ε m B for all ( t 1 , … , t m ) ∈ [ 0 , B ] m . |p(t_1,\ldots,t_m)-\sigma_i(t_1,\ldots,t_m)|\le\frac{\varepsilon}{mB}
\qquad\text{for all }(t_1,\ldots,t_m)\in[0,B]^m. ∣ p ( t 1 , … , t m ) − σ i ( t 1 , … , t m ) ∣ ≤ m B ε for all ( t 1 , … , t m ) ∈ [ 0 , B ] m .
Proof. The proof proceeds in three steps. We first approximate the exponential and reciprocal functions separately, then combine these approximations to obtain a polynomial approximation of the softmax component.
Step 1. Approximation of the exponential function on [ 0 , B ] [0,B] [ 0 , B ] .
Set
d 1 = ⌈ max { 1 , 2 e τ B , log ( 8 B / ε ) + 2 τ B log 2 } ⌉ , p 1 ( t ) = ∑ j = 0 d 1 ( − τ t ) j j ! . d_1=\left\lceil\max\left\{1,2e^{\tau B},\frac{\log(8B/\varepsilon)+2\tau B}{\log2}\right\}\right\rceil,\qquad
p_1(t)=\sum_{j=0}^{d_1}\frac{(-\tau t)^j}{j!}. d 1 = ⌈ max { 1 , 2 e τ B , log 2 log ( 8 B / ε ) + 2 τ B } ⌉ , p 1 ( t ) = j = 0 ∑ d 1 j ! ( − τ t ) j .
By Taylor’s theorem, for every t ∈ [ 0 , B ] t\in[0,B] t ∈ [ 0 , B ] ,
∣ p 1 ( t ) − e − τ t ∣ ≤ ( τ B ) d 1 + 1 ( d 1 + 1 ) ! ≤ ( e τ B d 1 + 1 ) d 1 + 1 ≤ 2 − ( d 1 + 1 ) ≤ ε 8 B e − 2 τ B . |p_1(t)-e^{-\tau t}|
\le\frac{(\tau B)^{d_1+1}}{(d_1+1)!}
\le\left(\frac{e\tau B}{d_1+1}\right)^{d_1+1}
\le2^{-(d_1+1)}
\le\frac{\varepsilon}{8B}e^{-2\tau B}. ∣ p 1 ( t ) − e − τ t ∣ ≤ ( d 1 + 1 )! ( τ B ) d 1 + 1 ≤ ( d 1 + 1 e τ B ) d 1 + 1 ≤ 2 − ( d 1 + 1 ) ≤ 8 B ε e − 2 τ B .
Step 2. Approximation of the reciprocal function on [ e − τ B / 2 , 5 / 4 ] [e^{-\tau B}/2,5/4] [ e − τ B /2 , 5/4 ] .
Set
d 2 = ⌈ 5 2 e τ B ( log 16 B ε + 2 τ B ) ⌉ , p 2 ( t ) = 4 5 ∑ j = 0 d 2 ( 1 − 4 t 5 ) j . d_2=\left\lceil\frac{5}{2}e^{\tau B}\left(\log\frac{16B}{\varepsilon}+2\tau B\right)\right\rceil,\qquad
p_2(t)=\frac45\sum_{j=0}^{d_2}\left(1-\frac{4t}{5}\right)^j. d 2 = ⌈ 2 5 e τ B ( log ε 16 B + 2 τ B ) ⌉ , p 2 ( t ) = 5 4 j = 0 ∑ d 2 ( 1 − 5 4 t ) j .
For t ∈ [ e − τ B / 2 , 5 / 4 ] t\in[e^{-\tau B}/2,5/4] t ∈ [ e − τ B /2 , 5/4 ] , the geometric-series identity gives
∣ p 2 ( t ) − 1 t ∣ = 4 5 ∑ j = d 2 + 1 ∞ ( 1 − 4 t 5 ) j = 1 t ( 1 − 4 t 5 ) d 2 + 1 . \left|p_2(t)-\frac1t\right|
=\frac45\sum_{j=d_2+1}^{\infty}\left(1-\frac{4t}{5}\right)^j
=\frac1t\left(1-\frac{4t}{5}\right)^{d_2+1}. p 2 ( t ) − t 1 = 5 4 j = d 2 + 1 ∑ ∞ ( 1 − 5 4 t ) j = t 1 ( 1 − 5 4 t ) d 2 + 1 .
≤ 2 e τ B exp ( − 2 5 e − τ B ( d 2 + 1 ) ) ≤ ε e − τ B 8 B . \le2e^{\tau B}\exp\left(-\frac{2}{5}e^{-\tau B}(d_2+1)\right)\le\frac{\varepsilon e^{-\tau B}}{8B}. ≤ 2 e τ B exp ( − 5 2 e − τ B ( d 2 + 1 ) ) ≤ 8 B ε e − τ B .
Step 3. Approximation of the softmax component σ i \sigma_i σ i .
For ( t 1 , … , t m ) ∈ [ 0 , B ] m (t_1,\ldots,t_m)\in[0,B]^m ( t 1 , … , t m ) ∈ [ 0 , B ] m , define
p 3 ( t 1 , … , t m ) = 1 m p 1 ( t i ) p 2 ( 1 m ∑ j = 1 m p 1 ( t j ) ) . p_3(t_1,\ldots,t_m)=\frac{1}{m}p_1(t_i)p_2\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right). p 3 ( t 1 , … , t m ) = m 1 p 1 ( t i ) p 2 ( m 1 j = 1 ∑ m p 1 ( t j ) ) .
Write E j = e − τ t j E_j=e^{-\tau t_j} E j = e − τ t j for j ∈ [ m ] j\in[m] j ∈ [ m ] . Then
σ i ( t 1 , … , t m ) = 1 m E i 1 m ∑ j = 1 m E j , ∣ 1 m ∑ j = 1 m p 1 ( t j ) − 1 m ∑ j = 1 m E j ∣ ≤ ε 8 B e − 2 τ B . \sigma_i(t_1,\ldots,t_m)=\frac{1}{m}\frac{E_i}{\frac{1}{m}\sum_{j=1}^{m}E_j},\qquad\left|\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)-\frac{1}{m}\sum_{j=1}^{m}E_j\right|\le\frac{\varepsilon}{8B}e^{-2\tau B}. σ i ( t 1 , … , t m ) = m 1 m 1 ∑ j = 1 m E j E i , m 1 j = 1 ∑ m p 1 ( t j ) − m 1 j = 1 ∑ m E j ≤ 8 B ε e − 2 τ B .
Since B ≥ 1 B\ge1 B ≥ 1 and ε < 1 / e \varepsilon<1/e ε < 1/ e , Step 1 implies
1 2 e − τ B ≤ p 1 ( t ) ≤ 1 + ε 8 B e − 2 τ B < 5 4 for every t ∈ [ 0 , B ] . \frac{1}{2}e^{-\tau B}\le p_1(t)\le1+\frac{\varepsilon}{8B}e^{-2\tau B}<\frac{5}{4}\qquad\text{for every }t\in[0,B]. 2 1 e − τ B ≤ p 1 ( t ) ≤ 1 + 8 B ε e − 2 τ B < 4 5 for every t ∈ [ 0 , B ] .
Thus the average of the p 1 ( t j ) p_1(t_j) p 1 ( t j ) lies in the approximation interval from Step 2. Moreover, 1 m ∑ j = 1 m E j ≥ e − τ B \frac{1}{m}\sum_{j=1}^{m}E_j\ge e^{-\tau B} m 1 ∑ j = 1 m E j ≥ e − τ B . Consequently,
∣ p 2 ( 1 m ∑ j = 1 m p 1 ( t j ) ) − 1 1 m ∑ j = 1 m E j ∣ ≤ ∣ p 2 ( 1 m ∑ j = 1 m p 1 ( t j ) ) − 1 1 m ∑ j = 1 m p 1 ( t j ) ∣ + ∣ 1 1 m ∑ j = 1 m p 1 ( t j ) − 1 1 m ∑ j = 1 m E j ∣ ≤ ε e − τ B 8 B + ε 8 B e − 2 τ B ( 1 m ∑ j = 1 m p 1 ( t j ) ) ( 1 m ∑ j = 1 m E j ) ≤ 3 ε 8 B . \begin{aligned}
\left|p_2\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right)-\frac{1}{\frac{1}{m}\sum_{j=1}^{m}E_j}\right| \\
&\le\left|p_2\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right)-\frac{1}{\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)}\right|+\left|\frac{1}{\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)}-\frac{1}{\frac{1}{m}\sum_{j=1}^{m}E_j}\right| \\
&\le\frac{\varepsilon e^{-\tau B}}{8B}+\frac{\frac{\varepsilon}{8B}e^{-2\tau B}}{\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right)\left(\frac{1}{m}\sum_{j=1}^{m}E_j\right)} \\
&\le\frac{3\varepsilon}{8B}.
\end{aligned} p 2 ( m 1 j = 1 ∑ m p 1 ( t j ) ) − m 1 ∑ j = 1 m E j 1 ≤ p 2 ( m 1 j = 1 ∑ m p 1 ( t j ) ) − m 1 ∑ j = 1 m p 1 ( t j ) 1 + m 1 ∑ j = 1 m p 1 ( t j ) 1 − m 1 ∑ j = 1 m E j 1 ≤ 8 B ε e − τ B + ( m 1 ∑ j = 1 m p 1 ( t j ) ) ( m 1 ∑ j = 1 m E j ) 8 B ε e − 2 τ B ≤ 8 B 3 ε .
It follows that
∣ p 3 ( t 1 , … , t m ) − σ i ( t 1 , … , t m ) ∣ ≤ 1 m p 1 ( t i ) ∣ p 2 ( 1 m ∑ j = 1 m p 1 ( t j ) ) − 1 1 m ∑ j = 1 m E j ∣ + 1 m ∣ p 1 ( t i ) − E i ∣ 1 m ∑ j = 1 m E j ≤ 1 m ( 5 4 3 ε 8 B + ε e − τ B 8 B ) ≤ ε m B . \begin{aligned}
|p_3(t_1,\ldots,t_m)-\sigma_i(t_1,\ldots,t_m)| \\
&\le\frac{1}{m}p_1(t_i)\left|p_2\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right)-\frac{1}{\frac{1}{m}\sum_{j=1}^{m}E_j}\right|+\frac{1}{m}\frac{|p_1(t_i)-E_i|}{\frac{1}{m}\sum_{j=1}^{m}E_j} \\
&\le\frac{1}{m}\left(\frac{5}{4}\frac{3\varepsilon}{8B}+\frac{\varepsilon e^{-\tau B}}{8B}\right) \\
&\le\frac{\varepsilon}{mB}.
\end{aligned} ∣ p 3 ( t 1 , … , t m ) − σ i ( t 1 , … , t m ) ∣ ≤ m 1 p 1 ( t i ) p 2 ( m 1 j = 1 ∑ m p 1 ( t j ) ) − m 1 ∑ j = 1 m E j 1 + m 1 m 1 ∑ j = 1 m E j ∣ p 1 ( t i ) − E i ∣ ≤ m 1 ( 4 5 8 B 3 ε + 8 B ε e − τ B ) ≤ m B ε .
The polynomial p 3 p_3 p 3 has degree at most d 1 ( d 2 + 1 ) d_1(d_2+1) d 1 ( d 2 + 1 ) . Since log ( 1 / ε ) > 1 \log(1/\varepsilon)>1 log ( 1/ ε ) > 1 ,
d 1 ≤ [ 1 + max { 1 , 2 e τ B , 1 + log ( 8 B ) + 2 τ B log 2 } ] log ( 1 / ε ) , (5) d_1\le\left[1+\max\left\{1,2e\tau B,\frac{1+\log(8B)+2\tau B}{\log2}\right\}\right]\log(1/\varepsilon), \tag*{(5)} d 1 ≤ [ 1 + max { 1 , 2 e τ B , log 2 1 + log ( 8 B ) + 2 τ B } ] log ( 1/ ε ) , (5)
and
d 2 + 1 ≤ [ 2 + 5 2 e τ B ( 1 + log ( 16 B ) + 2 τ B ) ] log ( 1 / ε ) . d_2+1\leq\left[2+\frac{5}{2}e^{\tau B}(1+\log(16B)+2\tau B)\right]\log(1/\varepsilon). d 2 + 1 ≤ [ 2 + 2 5 e τ B ( 1 + log ( 16 B ) + 2 τ B ) ] log ( 1/ ε ) .
Define
C 0 ( B , τ ) = [ 1 + max { 1 , 2 e τ B , 1 + log ( 8 B ) + 2 τ B log 2 } ] [ 2 + 5 2 e τ B ( 1 + log ( 16 B ) + 2 τ B ) ] . C_0(B,\tau)=\left[1+\max\left\{1,2e\tau B,\frac{1+\log(8B)+2\tau B}{\log2}\right\}\right]\left[2+\frac{5}{2}e^{\tau B}(1+\log(16B)+2\tau B)\right]. C 0 ( B , τ ) = [ 1 + max { 1 , 2 e τ B , log 2 1 + log ( 8 B ) + 2 τ B } ] [ 2 + 2 5 e τ B ( 1 + log ( 16 B ) + 2 τ B ) ] .
Then
deg ( p 3 ) ≤ d 1 ( d 2 + 1 ) ≤ C 0 ( B , τ ) log 2 ( 1 / ε ) . \deg(p_3)\leq d_1(d_2+1)\leq C_0(B,\tau)\log^2(1/\varepsilon). deg ( p 3 ) ≤ d 1 ( d 2 + 1 ) ≤ C 0 ( B , τ ) log 2 ( 1/ ε ) .
The constant is independent of m m m and k k k . Taking p = p 3 p=p_3 p = p 3 proves the claim. ■
For an m m m -variate polynomial p p p and vectors u 1 , … , u m ∈ R k \mathbf{u}_1,\ldots,\mathbf{u}_m\in\mathbb{R}^k u 1 , … , u m ∈ R k , define
p u 1 , … , u m ( v ) = p ( ⟨ u 1 , v ⟩ , … , ⟨ u m , v ⟩ ) . p_{\mathbf{u}_1,\ldots,\mathbf{u}_m}(\mathbf{v})=p(\langle\mathbf{u}_1,\mathbf{v}\rangle,\ldots,\langle\mathbf{u}_m,\mathbf{v}\rangle). p u 1 , … , u m ( v ) = p (⟨ u 1 , v ⟩ , … , ⟨ u m , v ⟩) .
Lemma C.8. Let
p ( t 1 , … , t m ) = ∑ j = 0 d ∑ I ∈ [ m ] j β I t i 1 ⋯ t i j p(t_1,\ldots,t_m)=\sum_{j=0}^{d}\sum_{I\in[m]^j}\beta_I t_{i_1}\cdots t_{i_j} p ( t 1 , … , t m ) = j = 0 ∑ d I ∈ [ m ] j ∑ β I t i 1 ⋯ t i j be a representation of an m m m -variate polynomial of total degree at most d d d , where I = ( i 1 , … , i j ) I=(i_1,\ldots,i_j) I = ( i 1 , … , i j ) and ∣ I ∣ = j |I|=j ∣ I ∣ = j . The empty product represents the constant term. For vectors u 1 , … , u m ∈ [ − m B , m B ] k \mathbf{u}_1,\ldots,\mathbf{u}_m\in[-mB,mB]^k u 1 , … , u m ∈ [ − m B , m B ] k , the function p u 1 , … , u m p_{\mathbf{u}_1,\ldots,\mathbf{u}_m} p u 1 , … , u m belongs to Γ ( d ) \Gamma(d) Γ ( d ) and satisfies
∥ p u 1 , … , u m ∥ Γ ( d ) 2 ≤ ( d + 1 ) ( m B k ) 2 d ( ∑ ∣ I ∣ ≤ d ∣ β I ∣ ) 2 . \left\|p_{\mathbf{u}_1,\ldots,\mathbf{u}_m}\right\|_{\Gamma(d)}^2\leq(d+1)(mB\sqrt{k})^{2d}\left(\sum_{|I|\leq d}|\beta_I|\right)^2. ∥ p u 1 , … , u m ∥ Γ ( d ) 2 ≤ ( d + 1 ) ( m B k ) 2 d ∣ I ∣ ≤ d ∑ ∣ β I ∣ 2 . Proof. For I = ( i 1 , … , i j ) I=(i_1,\ldots,i_j) I = ( i 1 , … , i j ) , substituting t r = ⟨ u r , v ⟩ t_r=\langle\mathbf{u}_r,\mathbf{v}\rangle t r = ⟨ u r , v ⟩ gives
β I ∏ ℓ = 1 j ⟨ u i ℓ , v ⟩ = β I ∑ t ∈ [ k ] j A t , I v t 1 ⋯ v t j , A t , I = ∏ ℓ = 1 j ( u i ℓ ) t ℓ . \beta_I\prod_{\ell=1}^{j}\langle\mathbf{u}_{i_\ell},\mathbf{v}\rangle=\beta_I\sum_{\mathbf{t}\in[k]^j}A_{\mathbf{t},I}v_{t_1}\cdots v_{t_j},\qquad A_{\mathbf{t},I}=\prod_{\ell=1}^{j}(\mathbf{u}_{i_\ell})_{t_\ell}. β I ℓ = 1 ∏ j ⟨ u i ℓ , v ⟩ = β I t ∈ [ k ] j ∑ A t , I v t 1 ⋯ v t j , A t , I = ℓ = 1 ∏ j ( u i ℓ ) t ℓ . For j = 0 j=0 j = 0 , take A t , I = 1 A_{\mathbf{t},I}=1 A t , I = 1 . Since ∣ A t , I ∣ ≤ ( m B ) j |A_{\mathbf{t},I}|\leq(mB)^j ∣ A t , I ∣ ≤ ( m B ) j ,
p u 1 , … , u m ( v ) = ∑ j = 0 d ∑ t ∈ [ k ] j ( ∑ ∣ I ∣ = j β I A t , I ) v t 1 ⋯ v t j . p_{\mathbf{u}_1,\ldots,\mathbf{u}_m}(\mathbf{v})=\sum_{j=0}^{d}\sum_{\mathbf{t}\in[k]^j}\left(\sum_{|I|=j}\beta_I A_{\mathbf{t},I}\right)v_{t_1}\cdots v_{t_j}. p u 1 , … , u m ( v ) = j = 0 ∑ d t ∈ [ k ] j ∑ ∣ I ∣ = j ∑ β I A t , I v t 1 ⋯ v t j . This is a representation in the multinomial feature space. Its coefficient norm therefore bounds the RKHS norm:
∥ p u 1 , … , u m ∥ Γ ( d ) 2 ≤ ∑ j = 0 d ∑ t ∈ [ k ] j ∣ ∑ ∣ I ∣ = j β I A t , I ∣ 2 \left\|p_{\mathbf{u}_1,\ldots,\mathbf{u}_m}\right\|_{\Gamma(d)}^2\leq\sum_{j=0}^{d}\sum_{\mathbf{t}\in[k]^j}\left|\sum_{|I|=j}\beta_I A_{\mathbf{t},I}\right|^2 ∥ p u 1 , … , u m ∥ Γ ( d ) 2 ≤ j = 0 ∑ d t ∈ [ k ] j ∑ ∣ I ∣ = j ∑ β I A t , I 2 ≤ ∑ j = 0 d ( m B k ) 2 j ( ∑ ∣ I ∣ = j ∣ β I ∣ ) 2 \leq\sum_{j=0}^{d}(mB\sqrt{k})^{2j}\left(\sum_{|I|=j}|\beta_I|\right)^2 ≤ j = 0 ∑ d ( m B k ) 2 j ∣ I ∣ = j ∑ ∣ β I ∣ 2
≤ ( d + 1 ) ( m B k ) 2 d ( ∑ ∣ I ∣ ≤ d ∣ β I ∣ ) 2 , \leq(d+1)(mB\sqrt{k})^{2d}\left(\sum_{|I|\leq d}|\beta_I|\right)^2, ≤ ( d + 1 ) ( m B k ) 2 d ∣ I ∣ ≤ d ∑ ∣ β I ∣ 2 ,
where the last inequality uses m B k ≥ 1 mB\sqrt{k}\geq1 m B k ≥ 1 .
Remark C.9. A polynomial can have multiple representations in the ordered-monomial form used in Lemma C.8 , as noted in [16 ] . The bound holds for each specified representation. In the coefficient calculations below, we combine identical monomials before taking absolute coefficient sums.
We now bound the coefficient sum for the polynomial approximation of the softmax function.
Lemma C.10. There exists a constant C 1 ( B , τ ) > 0 C_1(B,\tau)>0 C 1 ( B , τ ) > 0 , independent of m m m and k k k , such that, for every ε ∈ ( 0 , 1 / e ) \varepsilon\in(0,1/e) ε ∈ ( 0 , 1/ e ) and u 1 , … , u m ∈ [ 0 , m B ] k \mathbf{u}_1,\ldots,\mathbf{u}_m\in[0,mB]^k u 1 , … , u m ∈ [ 0 , m B ] k , there exists a polynomial p ( t 1 , … , t m ) p(t_1,\ldots,t_m) p ( t 1 , … , t m ) satisfying the following conditions:
For every v ∈ Δ k \mathbf{v}\in\Delta_k v ∈ Δ k such that ⟨ u i , v ⟩ ∈ [ 0 , B ] \langle\mathbf{u}_i,\mathbf{v}\rangle\in[0,B] ⟨ u i , v ⟩ ∈ [ 0 , B ] for all i ∈ [ m ] i\in[m] i ∈ [ m ] ,
∣ σ 1 ( ⟨ u 1 , v ⟩ , … , ⟨ u m , v ⟩ ) − p u 1 , … , u m ( v ) ∣ ≤ ε 2 m B . \left|\sigma_1(\langle\mathbf{u}_1,\mathbf{v}\rangle,\ldots,\langle\mathbf{u}_m,\mathbf{v}\rangle)-p_{\mathbf{u}_1,\ldots,\mathbf{u}_m}(\mathbf{v})\right|\leq\frac{\varepsilon}{2mB}. ∣ σ 1 (⟨ u 1 , v ⟩ , … , ⟨ u m , v ⟩) − p u 1 , … , u m ( v ) ∣ ≤ 2 m B ε . It has degree deg ( p ) ≤ D : = ⌈ C 1 ( B , τ ) log 2 ( 4 / ε ) ⌉ \deg(p)\leq D:=\lceil C_1(B,\tau)\log^2(4/\varepsilon)\rceil deg ( p ) ≤ D := ⌈ C 1 ( B , τ ) log 2 ( 4/ ε )⌉ .
The polynomial p u 1 , … , u m p_{\mathbf{u}_1,\ldots,\mathbf{u}_m} p u 1 , … , u m induced by p p p has RKHS norm
∥ p u 1 , … , u m ∥ Γ ( D ) 2 ≤ ( D + 1 ) exp ( D ) ( m B k ) 2 D . \left\|p_{\mathbf{u}_1,\ldots,\mathbf{u}_m}\right\|_{\Gamma(D)}^2\leq(D+1)\exp(D)(mB\sqrt{k})^{2D}. ∥ p u 1 , … , u m ∥ Γ ( D ) 2 ≤ ( D + 1 ) exp ( D ) ( m B k ) 2 D . ∣ p ( t 1 , … , t m ) ∣ ≤ 1 |p(t_1,\ldots,t_m)|\leq1 ∣ p ( t 1 , … , t m ) ∣ ≤ 1 for every ( t 1 , … , t m ) ∈ [ 0 , B ] m (t_1,\ldots,t_m)\in[0,B]^m ( t 1 , … , t m ) ∈ [ 0 , B ] m .
Proof. Let p 1 p_1 p 1 and p 2 p_2 p 2 be the polynomials constructed in Lemma C.7 with accuracy parameter ε / 4 \varepsilon/4 ε /4 . Define
p ( t 1 , … , t m ) = 4 B 4 m B + ε p 1 ( t 1 ) p 2 ( 1 m ∑ j = 1 m p 1 ( t j ) ) , p(t_1,\ldots,t_m)=\frac{4B}{4mB+\varepsilon}p_1(t_1)p_2\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right), p ( t 1 , … , t m ) = 4 m B + ε 4 B p 1 ( t 1 ) p 2 ( m 1 j = 1 ∑ m p 1 ( t j ) ) ,
and
p 0 ( t 1 , … , t m ) = 1 m p 1 ( t 1 ) p 2 ( 1 m ∑ j = 1 m p 1 ( t j ) ) . p_0(t_1,\ldots,t_m)=\frac{1}{m}p_1(t_1)p_2\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right). p 0 ( t 1 , … , t m ) = m 1 p 1 ( t 1 ) p 2 ( m 1 j = 1 ∑ m p 1 ( t j ) ) .
By Lemma C.7 ,
∣ p 0 ( t 1 , … , t m ) − σ 1 ( t 1 , … , t m ) ∣ ≤ ε 4 m B \left|p_0(t_1,\ldots,t_m)-\sigma_1(t_1,\ldots,t_m)\right|\leq\frac{\varepsilon}{4mB} ∣ p 0 ( t 1 , … , t m ) − σ 1 ( t 1 , … , t m ) ∣ ≤ 4 m B ε
on [ 0 , B ] m [0,B]^m [ 0 , B ] m . Since p = 4 m B 4 m B + ε p 0 p=\frac{4mB}{4mB+\varepsilon}p_0 p = 4 m B + ε 4 m B p 0 ,
∣ p ( t 1 , … , t m ) − σ 1 ( t 1 , … , t m ) ∣ ≤ 4 m B 4 m B + ε ∣ p 0 ( t 1 , … , t m ) − σ 1 ( t 1 , … , t m ) ∣ + ε 4 m B + ε ∣ σ 1 ( t 1 , … , t m ) ∣ ≤ 2 ε 4 m B + ε ≤ ε 2 m B . \begin{aligned}
|p(t_1,\ldots,t_m)-\sigma_1(t_1,\ldots,t_m)|
&\leq\frac{4mB}{4mB+\varepsilon}|p_0(t_1,\ldots,t_m)-\sigma_1(t_1,\ldots,t_m)|\\
&\quad+\frac{\varepsilon}{4mB+\varepsilon}|\sigma_1(t_1,\ldots,t_m)|\\
&\leq\frac{2\varepsilon}{4mB+\varepsilon}\leq\frac{\varepsilon}{2mB}.
\end{aligned} ∣ p ( t 1 , … , t m ) − σ 1 ( t 1 , … , t m ) ∣ ≤ 4 m B + ε 4 m B ∣ p 0 ( t 1 , … , t m ) − σ 1 ( t 1 , … , t m ) ∣ + 4 m B + ε ε ∣ σ 1 ( t 1 , … , t m ) ∣ ≤ 4 m B + ε 2 ε ≤ 2 m B ε .
This proves condition 1. Writing d 1 = deg ( p 1 ) d_1 = \deg(p_1) d 1 = deg ( p 1 ) and d 2 = deg ( p 2 ) d_2 = \deg(p_2) d 2 = deg ( p 2 ) , the same construction gives
deg ( p ) ≤ d 1 ( d 2 + 1 ) ≤ C 0 ( B , τ ) log 2 ( 4 / ε ) . \deg(p) \le d_1(d_2 + 1) \le C_0(B,\tau)\log^2(4/\varepsilon). deg ( p ) ≤ d 1 ( d 2 + 1 ) ≤ C 0 ( B , τ ) log 2 ( 4/ ε ) .
Thus condition 2 holds whenever C 1 ( B , τ ) ≥ C 0 ( B , τ ) C_1(B,\tau) \ge C_0(B,\tau) C 1 ( B , τ ) ≥ C 0 ( B , τ ) .
To prove condition 3, write the standard monomial expansion of a polynomial P P P as
P ( t 1 , … , t m ) = ∑ I ∈ Z ≥ 0 m ∣ I ∣ ≤ d β I t 1 i 1 ⋯ t m i m , ∣ I ∣ = i 1 + ⋯ + i m P(t_1,\ldots,t_m) = \sum_{\substack{I \in\mathbb{Z}_{\ge0}^m\\ |I|\le d}} \beta_I t_1^{i_1}\cdots t_m^{i_m}, \qquad|I| = i_1+\cdots+i_m P ( t 1 , … , t m ) = I ∈ Z ≥ 0 m ∣ I ∣ ≤ d ∑ β I t 1 i 1 ⋯ t m i m , ∣ I ∣ = i 1 + ⋯ + i m
and define
A ( P ) = ∑ ∣ I ∣ ≤ d ∣ β I ∣ . A(P) = \sum_{|I|\le d} |\beta_I|. A ( P ) = ∣ I ∣ ≤ d ∑ ∣ β I ∣.
We use the following elementary properties:
(a) For polynomials P P P and Q Q Q ,
A ( P + Q ) ≤ A ( P ) + A ( Q ) , A ( P Q ) ≤ A ( P ) A ( Q ) . A(P+Q) \le A(P)+A(Q), \qquad A(PQ) \le A(P)A(Q). A ( P + Q ) ≤ A ( P ) + A ( Q ) , A ( P Q ) ≤ A ( P ) A ( Q ) .
(b) If P P P is univariate of degree d P d_P d P , then
A ( P ∘ Q ) ≤ A ( P ) max { A ( Q ) , 1 } d P . A(P \circ Q) \le A(P)\max\{A(Q),1\}^{d_P}. A ( P ∘ Q ) ≤ A ( P ) max { A ( Q ) , 1 } d P .
By the definition of p 1 p_1 p 1 ,
A ( p 1 ) = ∑ j = 0 d 1 τ j j ! ≤ e τ . A(p_1) = \sum_{j=0}^{d_1}\frac{\tau^j}{j!} \le e^\tau. A ( p 1 ) = j = 0 ∑ d 1 j ! τ j ≤ e τ .
For p 2 p_2 p 2 ,
p 2 ( t ) = 4 5 ∑ j = 0 d 2 ∑ s = 0 j ( j s ) ( − 4 5 ) s t s = 4 5 ∑ s = 0 d 2 ( − 4 5 ) s ( ∑ j = s d 2 ( j s ) ) t s . p_2(t) = \frac{4}{5}\sum_{j=0}^{d_2}\sum_{s=0}^{j}\binom{j}{s}\left(-\frac{4}{5}\right)^s t^s = \frac{4}{5}\sum_{s=0}^{d_2}\left(-\frac{4}{5}\right)^s\left(\sum_{j=s}^{d_2}\binom{j}{s}\right)t^s. p 2 ( t ) = 5 4 j = 0 ∑ d 2 s = 0 ∑ j ( s j ) ( − 5 4 ) s t s = 5 4 s = 0 ∑ d 2 ( − 5 4 ) s ( j = s ∑ d 2 ( s j ) ) t s .
The coefficient of t s t^s t s has sign ( − 1 ) s (-1)^s ( − 1 ) s , so
A ( p 2 ) = p 2 ( − 1 ) = ( 9 5 ) d 2 + 1 − 1 ≤ e d 2 + 1 . A(p_2) = p_2(-1) = \left(\frac{9}{5}\right)^{d_2+1}-1 \le e^{d_2+1}. A ( p 2 ) = p 2 ( − 1 ) = ( 5 9 ) d 2 + 1 − 1 ≤ e d 2 + 1 .
Moreover,
A ( 1 m ∑ j = 1 m p 1 ( t j ) ) ≤ A ( p 1 ) ≤ e τ . A\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right) \le A(p_1) \le e^\tau. A ( m 1 j = 1 ∑ m p 1 ( t j ) ) ≤ A ( p 1 ) ≤ e τ .
Since 4 B 4 m B + ε ≤ 1 \frac{4B}{4mB+\varepsilon}\le1 4 m B + ε 4 B ≤ 1 , properties (a) and (b) imply
A ( p ) ≤ A ( p 1 ) A ( p 2 ) max { A ( 1 m ∑ j = 1 m p 1 ( t j ) ) , 1 } d 2 ≤ e τ e d 2 + 1 e τ d 2 = exp ( ( τ + 1 ) ( d 2 + 1 ) ) . \begin{aligned}
A(p) \le A(p_1)A(p_2)\max\left\{A\left(\frac{1}{m}\sum_{j=1}^{m}p_1(t_j)\right),1\right\}^{d_2} \\
&\le e^\tau e^{d_2+1}e^{\tau d_2} = \exp\left((\tau+1)(d_2+1)\right).
\end{aligned} A ( p ) ≤ A ( p 1 ) A ( p 2 ) max { A ( m 1 j = 1 ∑ m p 1 ( t j ) ) , 1 } d 2 ≤ e τ e d 2 + 1 e τ d 2 = exp ( ( τ + 1 ) ( d 2 + 1 ) ) .
The construction in Lemma C.7 ensures d 1 ≥ max { 1 , τ } d_1 \ge\max\{1,\tau\} d 1 ≥ max { 1 , τ } . Therefore,
A ( p ) ≤ exp ( 2 d 1 ( d 2 + 1 ) ) ≤ exp ( 2 C 0 ( B , τ ) log 2 ( 4 / ε ) ) . A(p) \le\exp(2d_1(d_2+1)) \le\exp(2C_0(B,\tau)\log^2(4/\varepsilon)). A ( p ) ≤ exp ( 2 d 1 ( d 2 + 1 )) ≤ exp ( 2 C 0 ( B , τ ) log 2 ( 4/ ε )) .
Choose C 1 ( B , τ ) = 4 C 0 ( B , τ ) C_1(B,\tau)=4C_0(B,\tau) C 1 ( B , τ ) = 4 C 0 ( B , τ ) . Then
D = ⌈ C 1 ( B , τ ) log 2 ( 4 / ε ) ⌉ , deg ( p ) ≤ D , A ( p ) 2 ≤ e D . D=\left\lceil C_1(B,\tau)\log^2(4/\varepsilon)\right\rceil,\qquad\deg(p)\le D,\qquad A(p)^2\le e^D. D = ⌈ C 1 ( B , τ ) log 2 ( 4/ ε ) ⌉ , deg ( p ) ≤ D , A ( p ) 2 ≤ e D .
The standard monomial expansion gives a representation as in Lemma C.8 with coefficient sum A ( p ) A(p) A ( p ) . Applying that lemma with degree bound D D D yields
∥ p u 1 , … , u m ∥ Γ ( D ) 2 ≤ ( D + 1 ) ( m B k ) 2 D A ( p ) 2 ≤ ( D + 1 ) ( m B k ) 2 D exp ( D ) . \lVert p_{u_1,\ldots,u_m}\rVert_{\Gamma(D)}^2 \le(D+1)(mB\sqrt{k})^{2D}A(p)^2 \le(D+1)(mB\sqrt{k})^{2D}\exp(D). ∥ p u 1 , … , u m ∥ Γ ( D ) 2 ≤ ( D + 1 ) ( m B k ) 2 D A ( p ) 2 ≤ ( D + 1 ) ( m B k ) 2 D exp ( D ) .
This proves condition 3.
Finally, for every ( t 1 , … , t m ) ∈ [ 0 , B ] m (t_1,\ldots,t_m)\in[0,B]^m ( t 1 , … , t m ) ∈ [ 0 , B ] m ,
∣ p ( t 1 , … , t m ) ∣ = 4 m B 4 m B + ε ∣ p 0 ( t 1 , … , t m ) ∣ ≤ 4 m B 4 m B + ε ( ∣ p 0 ( t 1 , … , t m ) − σ 1 ( t 1 , … , t m ) ∣ + ∣ σ 1 ( t 1 , … , t m ) ∣ ) ≤ 4 m B 4 m B + ε ( 1 + ε 4 m B ) = 1. \begin{aligned}
\lvert p(t_1,\ldots,t_m)\rvert=\frac{4mB}{4mB+\varepsilon}\lvert p_0(t_1,\ldots,t_m)\rvert\\
&\le\frac{4mB}{4mB+\varepsilon}\left(\lvert p_0(t_1,\ldots,t_m)-\sigma_1(t_1,\ldots,t_m)\rvert+\lvert\sigma_1(t_1,\ldots,t_m)\rvert\right) \\
&\le\frac{4mB}{4mB+\varepsilon}\left(1+\frac{\varepsilon}{4mB}\right)=1.
\end{aligned} ∣ p ( t 1 , … , t m )∣ = 4 m B + ε 4 m B ∣ p 0 ( t 1 , … , t m )∣ ≤ 4 m B + ε 4 m B ( ∣ p 0 ( t 1 , … , t m ) − σ 1 ( t 1 , … , t m )∣ + ∣ σ 1 ( t 1 , … , t m )∣ ) ≤ 4 m B + ε 4 m B ( 1 + 4 m B ε ) = 1.
This proves condition 4. ■
We now state our main theorem.
Theorem C.11. Under the uniform-logging and fixed-cost assumptions of this section, fix τ > 0 \tau>0 τ > 0 . There exists a constant C ( B , τ ) > 0 C(B,\tau)>0 C ( B , τ ) > 0 , depending only on B , τ B,\tau B , τ and independent of ∣ A ∣ , ∣ O ∣ \lvert\mathcal{A}\rvert,\lvert\mathcal{O}\rvert ∣ A ∣ , ∣ O ∣ , such that the following holds. For every ε ∈ ( 0 , 1 / e ) \varepsilon\in(0,1/e) ε ∈ ( 0 , 1/ e ) and δ ∈ ( 0 , 1 / 3 ) \delta\in(0,1/3) δ ∈ ( 0 , 1/3 ) , let
D = ⌈ C ( B , τ ) log 2 ( 4 / ε ) ⌉ , r = D + 1 ( m B e k ) D , D=\left\lceil C(B,\tau)\log^2(4/\varepsilon)\right\rceil,\qquad r=\sqrt{D+1}(mB\sqrt{e k})^D, D = ⌈ C ( B , τ ) log 2 ( 4/ ε ) ⌉ , r = D + 1 ( m B e k ) D , where m = ∣ A ∣ m=\lvert\mathcal{A}\rvert m = ∣ A ∣ and k = m ∣ O ∣ k=m\lvert\mathcal{O}\rvert k = m ∣ O ∣ . There exists an
( Ω , ε , ε 6 m B r D + 1 ) \left(\Omega,\varepsilon,\frac{\varepsilon}{6mBr\sqrt{D+1}}\right) ( Ω , ε , 6 m B r D + 1 ε ) auditor with witness for Decision-OI, with failure probability at most δ \delta δ , sample complexity
n = O ( k m 2 B 2 r 2 ( D + 1 ) log ( 1 / δ ) ε 2 ) , n=O\left(\frac{k m^2 B^2 r^2(D+1)\log(1/\delta)}{\varepsilon^2}\right), n = O ( ε 2 k m 2 B 2 r 2 ( D + 1 ) log ( 1/ δ ) ) , and running time poly ( n , k , D ) \operatorname{poly}(n,k,D) poly ( n , k , D ) . The samples used in an auditor call are independent of its input predictor.
Remark C.12. Let D D D and r r r be as in Theorem C.11 . With uniform initialization p ( 0 ) ( o ∣ z , a ) = 1 / ∣ O ∣ p^{(0)}(o\mid z,a)=1/\lvert\mathcal{O}\rvert p ( 0 ) ( o ∣ z , a ) = 1/ ∣ O ∣ , multiplicative updates with factor exp ( η w ( a , o ) ( t ) ( v ( t ) ( z ) ) ) \exp(\eta w^{(t)}_{(a,o)}(\mathbf{v}^{(t)}(z))) exp ( η w ( a , o ) ( t ) ( v ( t ) ( z ))) , and termination when the auditor returns ⊥ \bot ⊥ , Lemma C.4 gives
O ( m 2 B 2 r 2 ( D + 1 ) log ∣ O ∣ ε 2 ) O\left(\frac{m^2B^2r^2(D+1)\log\lvert\mathcal{O}\rvert}{\varepsilon^2}\right) O ( ε 2 m 2 B 2 r 2 ( D + 1 ) log ∣ O ∣ )
updates on the joint success event of the auditor calls. Each call uses independent samples, with failure probabilities allocated across calls as in that lemma.
For fixed B , τ , ϵ B,\tau,\epsilon B , τ , ϵ , this update bound is polynomial in m m m and k k k . For fixed m , k , B , τ m,k,B,\tau m , k , B , τ , it is quasi-polynomial in 1 / ϵ 1/\epsilon 1/ ϵ . Combining the auditor with boosting therefore preserves these qualitative efficiency guarantees.
Remark C.13. The updates are normalized separately for each action, preserving total mass 1 / m 1/m 1/ m in each action block of v \mathbf{v} v . The averaged KL potential in Lemma C.4 accounts for this block structure and yields the stated iteration bound.
Proof of Theorem C.11 . For simplicity, write A = { a 1 , … , a m } \mathcal{A}=\{a_1,\ldots,a_m\} A = { a 1 , … , a m } . For a loss-cost pair ω = ( l ~ , c ) \omega=(\tilde{l},c) ω = ( l ~ , c ) , define
s a ( z ) = m ∑ o ∈ O ℓ a ω ( o ) v ( a , o ) ( z ) . s_a(z)=m\sum_{o\in\mathcal{O}}\ell_a^\omega(o)\mathbf{v}_{(a,o)}(z). s a ( z ) = m o ∈ O ∑ ℓ a ω ( o ) v ( a , o ) ( z ) .
and let l a ω ∈ [ 0 , m B ] k \mathbf{l}_a^\omega\in[0,mB]^k l a ω ∈ [ 0 , m B ] k have coordinates
( l a ω ) ( a ′ , o ′ ) = m ℓ a ω ( o ′ ) 1 { a = a ′ } . (\mathbf{l}_a^\omega)_{(a',o')}=m\ell_a^\omega(o')\mathbf{1}\{a=a'\}. ( l a ω ) ( a ′ , o ′ ) = m ℓ a ω ( o ′ ) 1 { a = a ′ } .
Since v ( a , o ) ( z ) = p ( o ∣ z , a ) / m \mathbf{v}_{(a,o)}(z)=p(o\mid z,a)/m v ( a , o ) ( z ) = p ( o ∣ z , a ) / m and 0 ≤ ℓ a ω ( o ) ≤ B 0\leq\ell_a^\omega(o)\leq B 0 ≤ ℓ a ω ( o ) ≤ B ,
s a ( z ) = ⟨ l a ω , v ( z ) ⟩ = ∑ o ∈ O ℓ a ω ( o ) p ( o ∣ z , a ) ∈ [ 0 , B ] . s_a(z)=\langle\mathbf{l}_a^\omega,\mathbf{v}(z)\rangle=\sum_{o\in\mathcal{O}}\ell_a^\omega(o)p(o\mid z,a)\in[0,B]. s a ( z ) = ⟨ l a ω , v ( z )⟩ = o ∈ O ∑ ℓ a ω ( o ) p ( o ∣ z , a ) ∈ [ 0 , B ] .
Consequently,
π ω ( a i ∣ z ) = σ i ( ⟨ l a 1 ω , v ( z ) ⟩ , … , ⟨ l a m ω , v ( z ) ⟩ ) . \pi^\omega(a_i\mid z)=\sigma_i(\langle\mathbf{l}_{a_1}^\omega,\mathbf{v}(z)\rangle,\ldots,\langle\mathbf{l}_{a_m}^\omega,\mathbf{v}(z)\rangle). π ω ( a i ∣ z ) = σ i (⟨ l a 1 ω , v ( z )⟩ , … , ⟨ l a m ω , v ( z )⟩) .
Take C ( B , τ ) = C 1 ( B , τ ) C(B,\tau)=C_1(B,\tau) C ( B , τ ) = C 1 ( B , τ ) from Lemma C.10 and set
D = ⌈ C ( B , τ ) log 2 ( 4 / ϵ ) ⌉ . D=\left\lceil C(B,\tau)\log^2(4/\epsilon)\right\rceil. D = ⌈ C ( B , τ ) log 2 ( 4/ ϵ ) ⌉ .
If p p p violates ( Ω , ϵ ) (\Omega,\epsilon) ( Ω , ϵ ) -Decision-OI, there exists ω ∈ Ω \omega\in\Omega ω ∈ Ω such that
∣ E z [ 1 m ∑ a ∈ A ∑ o ∈ O ℓ a ω ( o ) B π ω ( a ∣ z ) ( p ∗ ( o ∣ z , a ) − p ( o ∣ z , a ) ) ] ∣ > ϵ m B . \left|\mathbb{E}_z\left[\frac{1}{m}\sum_{a\in\mathcal{A}}\sum_{o\in\mathcal{O}}\frac{\ell_a^\omega(o)}{B}\pi^\omega(a\mid z)\bigl(p^*(o\mid z,a)-p(o\mid z,a)\bigr)\right]\right|>\frac{\epsilon}{mB}. E z [ m 1 a ∈ A ∑ o ∈ O ∑ B ℓ a ω ( o ) π ω ( a ∣ z ) ( p ∗ ( o ∣ z , a ) − p ( o ∣ z , a ) ) ] > m B ϵ .
By Lemma C.10 , with a permutation of coordinates for each softmax component, there exists an m m m -variate polynomial q ( a i ) q^{(a_i)} q ( a i ) of degree at most D D D such that
∣ q l a 1 ω , … , l a m ω ( a i ) ( v ( z ) ) − π ω ( a i ∣ z ) ∣ ≤ ϵ 2 m B , \left|q^{(a_i)}_{\mathbf{l}_{a_1}^\omega,\ldots,\mathbf{l}_{a_m}^\omega}(\mathbf{v}(z))-\pi^\omega(a_i\mid z)\right|\leq\frac{\epsilon}{2mB}, q l a 1 ω , … , l a m ω ( a i ) ( v ( z )) − π ω ( a i ∣ z ) ≤ 2 m B ϵ ,
and
∥ q l a 1 ω , … , l a m ω ( a i ) ∥ Γ ( D ) 2 ≤ ( D + 1 ) ( m B k ) 2 D exp ( D ) . \left\|q^{(a_i)}_{\mathbf{l}_{a_1}^\omega,\ldots,\mathbf{l}_{a_m}^\omega}\right\|_{\Gamma(D)}^2\leq(D+1)(mB\sqrt{k})^{2D}\exp(D). q l a 1 ω , … , l a m ω ( a i ) Γ ( D ) 2 ≤ ( D + 1 ) ( m B k ) 2 D exp ( D ) .
Here we used ⟨ l a i ω , v ( z ) ⟩ ∈ [ 0 , B ] \langle\mathbf{l}_{a_i}^\omega,\mathbf{v}(z)\rangle\in[0,B] ⟨ l a i ω , v ( z )⟩ ∈ [ 0 , B ] for every action a i a_i a i . Define
r = D + 1 ( m B k ) D exp ( D / 2 ) = D + 1 ( m B e k ) D . r=\sqrt{D+1}(mB\sqrt{k})^D\exp(D/2)=\sqrt{D+1}(mB\sqrt{ek})^D. r = D + 1 ( m B k ) D exp ( D /2 ) = D + 1 ( m B e k ) D .
Since 0 ≤ ℓ a ω ( o ) / B ≤ 1 0 \le\ell_a^\omega(o)/B \le1 0 ≤ ℓ a ω ( o ) / B ≤ 1 , the tuple
( ℓ a ω ( o ) B q I a 1 ω , … , a m ω ( a ) ) ( a , o ) ∈ A × O \left(\frac{\ell_a^\omega(o)}{B}q_{I_{a_1^\omega,\ldots,a_m^\omega}}^{(a)}\right)_{(a,o)\in\mathcal{A}\times\mathcal{O}} ( B ℓ a ω ( o ) q I a 1 ω , … , a m ω ( a ) ) ( a , o ) ∈ A × O
belongs to ( B Γ ( D ) ( r ) ) k (B_{\Gamma(D)}(r))^k ( B Γ ( D ) ( r ) ) k .
We next bound the change in correlation caused by the polynomial approximation. For every z z z and a a a , the normalized loss takes values in [ 0 , 1 ] [0,1] [ 0 , 1 ] , so
∣ ∑ o ∈ O ℓ a ω ( o ) B ( p ∗ ( o ∣ z , a ) − p ( o ∣ z , a ) ) ∣ ≤ 1. \left|\sum_{o\in\mathcal{O}}\frac{\ell_a^\omega(o)}{B}\left(p^*(o\mid z,a)-p(o\mid z,a)\right)\right|\le1. o ∈ O ∑ B ℓ a ω ( o ) ( p ∗ ( o ∣ z , a ) − p ( o ∣ z , a ) ) ≤ 1.
Grouping the outcome terms by action gives
∣ E z [ 1 m ∑ a ∈ A ( q I a 1 ω , … , a m ω ( a ) ( v ( z ) ) − π ω ( a ∣ z ) ) ∑ o ∈ O ℓ a ω ( o ) B ( p ∗ ( o ∣ z , a ) − p ( o ∣ z , a ) ) ] ∣ ≤ 1 m ∑ a ∈ A E z [ ∣ q I a 1 ω , … , a m ω ( a ) ( v ( z ) ) − π ω ( a ∣ z ) ∣ ] ≤ ε 2 m B . \left|\mathbb{E}_{z}\left[\frac{1}{m}\sum_{a\in\mathcal{A}}\left(q_{I_{a_1^\omega,\ldots,a_m^\omega}}^{(a)}(\mathbf{v}(z))-\pi^\omega(a\mid z)\right)\sum_{o\in\mathcal{O}}\frac{\ell_a^\omega(o)}{B}\left(p^*(o\mid z,a)-p(o\mid z,a)\right)\right]\right|
\le\frac{1}{m}\sum_{a\in\mathcal{A}}\mathbb{E}_{z}\left[\left|q_{I_{a_1^\omega,\ldots,a_m^\omega}}^{(a)}(\mathbf{v}(z))-\pi^\omega(a\mid z)\right|\right]\le\frac{\varepsilon}{2mB}. E z [ m 1 a ∈ A ∑ ( q I a 1 ω , … , a m ω ( a ) ( v ( z )) − π ω ( a ∣ z ) ) o ∈ O ∑ B ℓ a ω ( o ) ( p ∗ ( o ∣ z , a ) − p ( o ∣ z , a ) ) ] ≤ m 1 a ∈ A ∑ E z [ q I a 1 ω , … , a m ω ( a ) ( v ( z )) − π ω ( a ∣ z ) ] ≤ 2 m B ε .
Therefore,
∣ E z [ 1 m ∑ a ∈ A ∑ o ∈ O ℓ a ω ( o ) B q I a 1 ω , … , a m ω ( a ) ( v ( z ) ) ( p ∗ ( o ∣ z , a ) − p ( o ∣ z , a ) ) ] ∣ > ε 2 m B . \left|\mathbb{E}_{z}\left[\frac{1}{m}\sum_{a\in\mathcal{A}}\sum_{o\in\mathcal{O}}\frac{\ell_a^\omega(o)}{B}q_{I_{a_1^\omega,\ldots,a_m^\omega}}^{(a)}(\mathbf{v}(z))\left(p^*(o\mid z,a)-p(o\mid z,a)\right)\right]\right|>\frac{\varepsilon}{2mB}. E z [ m 1 a ∈ A ∑ o ∈ O ∑ B ℓ a ω ( o ) q I a 1 ω , … , a m ω ( a ) ( v ( z )) ( p ∗ ( o ∣ z , a ) − p ( o ∣ z , a ) ) ] > 2 m B ε .
Under uniform logging,
E [ y ( a , o ) ∣ z ] = 1 m p ∗ ( o ∣ z , a ) . \mathbb{E}\left[y_{(a,o)}\mid z\right]=\frac{1}{m}p^*(o\mid z,a). E [ y ( a , o ) ∣ z ] = m 1 p ∗ ( o ∣ z , a ) .
Thus ( B Γ ( D ) ( r ) ) k (B_{\Gamma(D)}(r))^k ( B Γ ( D ) ( r ) ) k contains a function whose absolute correlation with y − v \mathbf{y}-\mathbf{v} y − v under D c a l \mathcal{D}^{\mathrm{cal}} D cal exceeds ε / ( 2 m B ) \varepsilon/(2mB) ε / ( 2 m B ) .
Write
β = ε 6 m B r D + 1 . \beta=\frac{\varepsilon}{6mB r\sqrt{D+1}}. β = 6 m B r D + 1 ε .
Applying Lemma C.6 with accuracy parameter ε / ( 2 m B ) \varepsilon/(2mB) ε / ( 2 m B ) and failure probability δ / 2 \delta/2 δ /2 uses
O ( k m 2 B 2 r 2 ( D + 1 ) log ( 2 / δ ) ε 2 ) \mathcal{O}\left(\frac{km^2B^2r^2(D+1)\log(2/\delta)}{\varepsilon^2}\right) O ( ε 2 k m 2 B 2 r 2 ( D + 1 ) log ( 2/ δ ) )
samples and produces a function
w ′ ∈ ( B Γ ( D ) ( 1 D + 1 ) ) k . w' \in\left(B_{\Gamma(D)}\left(\frac{1}{\sqrt{D+1}}\right)\right)^k. w ′ ∈ ( B Γ ( D ) ( D + 1 1 ) ) k .
Whenever p p p violates Decision-OI, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 this function satisfies
E D c a l [ ⟨ y − v , w ′ ( v ) ⟩ ] ≥ β . \mathbb{E}_{\mathcal{D}^{\mathrm{cal}}}\left[\left\langle\mathbf{y}-\mathbf{v},w'(\mathbf{v})\right\rangle\right]\ge\beta. E D cal [ ⟨ y − v , w ′ ( v ) ⟩ ] ≥ β .
The kernel algorithm runs in time poly ( n , k , D ) \operatorname{poly}(n,k,D) poly ( n , k , D ) , and its output always takes values in [ − 1 , 1 ] k [-1,1]^k [ − 1 , 1 ] k .
On an independent batch of O ( β − 2 log ( 1 / δ ) ) O(\beta^{-2}\log(1/\delta)) O ( β − 2 log ( 1/ δ )) samples, the auditor estimates this correlation to accuracy β / 4 \beta/4 β /4 with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 . It returns w ′ w^\prime w ′ if the empirical correlation is at least 3 β / 4 3\beta/4 3 β /4 , and returns ⊥ \perp ⊥ otherwise. On the joint success event, every returned witness has true correlation at least β / 2 \beta/2 β /2 . If the auditor returns ⊥ \perp ⊥ , the candidate’s true correlation is strictly less than β \beta β , which implies that p p p satisfies ( Ω , ε ) (\Omega,\varepsilon) ( Ω , ε ) -Decision-OI. The failure probability is therefore at most δ \delta δ .
Since
β − 2 = 36 m 2 B 2 r 2 ( D + 1 ) ε 2 , \beta^{-2}=\frac{36m^2B^2r^2(D+1)}{\varepsilon^2}, β − 2 = ε 2 36 m 2 B 2 r 2 ( D + 1 ) ,
the total sample complexity is
n = O ( k m 2 B 2 r 2 ( D + 1 ) log ( 1 / δ ) ε 2 ) . n=O\left(\frac{km^2B^2r^2(D+1)\log(1/\delta)}{\varepsilon^2}\right). n = O ( ε 2 k m 2 B 2 r 2 ( D + 1 ) log ( 1/ δ ) ) .
The kernel representation permits evaluation of w ′ w^\prime w ′ on the independent batch in time polynomial in n , k , D n,k,D n , k , D , so the total running time remains poly ( n , k , D ) \operatorname{poly}(n,k,D) poly ( n , k , D ) . Each logged observation produces a sample from D \mathcal{D} D through Eq. (3) . By the definition of an auditor with witness for Decision-OI, this establishes the claimed ( Ω , ε , β ) (\Omega,\varepsilon,\beta) ( Ω , ε , β ) guarantee.
For fixed B , τ , ε B,\tau,\varepsilon B , τ , ε , these complexities are polynomial in m = ∣ A ∣ m=|\mathcal{A}| m = ∣ A ∣ and k = ∣ A × O ∣ k=|\mathcal{A}\times\mathcal{O}| k = ∣ A × O ∣ . For fixed B , τ , m , k B,\tau,m,k B , τ , m , k , they are quasi-polynomial in 1 / ε 1/\varepsilon 1/ ε . ■
Theoretical Guarantee for Extensions in Section 5
Extension to Feature-Based Collaboration Policies
While Theorems 4.2 and B.5 establish convergence and sample efficiency of COI-Boost, real-world human–AI collaboration systems typically process complex, high-dimensional states, such as medical images or lengthy textual contexts. To study the representation required by COI-Boost, we consider predefined collaboration policies operating on a fixed, lower-dimensional feature representation, such as the confidence scores of a black-box model [4 , 10 ] . We show that, when the costs and logging propensities also depend only on these features, the learned surrogate depends on the state through the same representation.
Lemma D.1 (Feature-Dependent Reduction of COI-Boost). *Assume that every policy in H \mathcal{H} H , every cost function in C \mathcal{C} C , and the logging propensities π l o g ( a ∣ z ) > 0 \pi_{\mathrm{log}}(a\mid z)>0 π log ( a ∣ z ) > 0 depend on the state z z z only through a fixed feature map ψ ( z ) = [ ψ 1 ( z ) , … , ψ k ( z ) ] ⊤ ∈ R k \psi(z)=[\psi_1(z),\ldots,\psi_k(z)]^\top\in\mathbb{R}^k ψ ( z ) = [ ψ 1 ( z ) , … , ψ k ( z ) ] ⊤ ∈ R k . Suppose Algorithm 1 uses the auditor constructed in Theorem 4.2 , which searches over Π e v a l \Pi_{\mathrm{eval}} Π eval . Then, at every iteration t t t , the base predictive distribution p ( t ) ( o ∣ z , a ) p^{(t)}(o\mid z,a) p ( t ) ( o ∣ z , a ) depends on z z z only through ψ ( z ) \psi(z) ψ ( z ) .
Proof. We proceed by mathematical induction. At t = 0 t=0 t = 0 , the algorithm initializes p ( 0 ) ( o ∣ z , a ) = 1 / ∣ O ∣ p^{(0)}(o\mid z,a)=1/|\mathcal{O}| p ( 0 ) ( o ∣ z , a ) = 1/∣ O ∣ , which does not depend on z z z . Assume that at iteration t t t , the predictor p ( t ) ( o ∣ z , a ) p^{(t)}(o\mid z,a) p ( t ) ( o ∣ z , a ) depends on z z z only through ψ ( z ) \psi(z) ψ ( z ) .
During the auditing step, the algorithm searches over Π e v a l \Pi_{\mathrm{eval}} Π eval for a violating policy and loss-cost pair. By assumption, every base collaboration policy h ∈ H h\in\mathcal{H} h ∈ H depends on z z z only through ψ ( z ) \psi(z) ψ ( z ) . Moreover, for ω = ( l ~ , c ) \omega= (\tilde{l},c) ω = ( l ~ , c ) and a = ( r , s , g ) a=(r,s,g) a = ( r , s , g ) , the total loss ℓ a ω ( z , o ) = l ~ ( o ) + r c ( x ) \ell^\omega_a(z,o)=\tilde{l}(o)+rc(x) ℓ a ω ( z , o ) = l ~ ( o ) + r c ( x ) depends on z z z only through ψ ( z ) \psi(z) ψ ( z ) .
The post-processing policy π ω \pi^\omega π ω is defined by applying softmax to the predicted action losses:
π ω ( a ∣ z ) = exp ( − τ E o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ) ∑ a ′ ∈ A exp ( − τ E o ∼ p ( t ) ( ⋅ ∣ z , a ′ ) [ ℓ a ′ ω ( z , o ) ] ) . \pi^\omega(a\mid z)=\frac{\exp\left(-\tau\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z,a)}[\ell^\omega_a(z,o)]\right)}{\sum_{a'\in\mathcal{A}}\exp\left(-\tau\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z,a')}[\ell^\omega_{a'}(z,o)]\right)}. π ω ( a ∣ z ) = ∑ a ′ ∈ A exp ( − τ E o ∼ p ( t ) ( ⋅ ∣ z , a ′ ) [ ℓ a ′ ω ( z , o )] ) exp ( − τ E o ∼ p ( t ) ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] ) .
By the induction hypothesis and the assumption on the costs, both p ( t ) p^{(t)} p ( t ) and ℓ a ω ( z , o ) \ell^\omega_a(z,o) ℓ a ω ( z , o ) depend on z z z only through ψ ( z ) \psi(z) ψ ( z ) . Therefore, the predicted action losses and π ω ( a ∣ z ) \pi^\omega(a\mid z) π ω ( a ∣ z ) have the same feature dependence.
Consequently, every policy in Π e v a l \Pi_{\mathrm{eval}} Π eval depends on z z z only through ψ ( z ) \psi(z) ψ ( z ) . If the auditor returns ⊥ \bot ⊥ , the algorithm terminates. Otherwise, let ( π ( t ) , ω ( t ) , σ ) (\pi^{(t)},\omega^{(t)},\sigma) ( π ( t ) , ω ( t ) , σ ) be its returned certificate. The update score is
g ( t ) ( z , a , o ) = σ π ( t ) ( a ∣ z ) π l o g ( a ∣ z ) ℓ a ω ( t ) ( z , o ) . g^{(t)}(z,a,o)=\sigma\frac{\pi^{(t)}(a\mid z)}{\pi_{\mathrm{log}}(a\mid z)}\ell^{\omega^{(t)}}_a(z,o). g ( t ) ( z , a , o ) = σ π log ( a ∣ z ) π ( t ) ( a ∣ z ) ℓ a ω ( t ) ( z , o ) .
By assumption, the logging propensities also depend on z z z only through ψ ( z ) \psi(z) ψ ( z ) . Thus the policy, logging propensity, and total loss all have the same feature dependence, while σ ∈ { − 1 , 1 } \sigma\in\{-1,1\} σ ∈ { − 1 , 1 } is fixed for this update. It follows that g ( t ) g^{(t)} g ( t ) depends on z z z only through ψ ( z ) \psi(z) ψ ( z ) .
The multiplicative weights update is
p ( t + 1 ) ( o ∣ z , a ) = p ( t ) ( o ∣ z , a ) exp ( η g ( t ) ( z , a , o ) ) ∑ o ′ ∈ O p ( t ) ( o ′ ∣ z , a ) exp ( η g ( t ) ( z , a , o ′ ) ) . p^{(t+1)}(o\mid z,a)=\frac{p^{(t)}(o\mid z,a)\exp\left(\eta g^{(t)}(z,a,o)\right)}{\sum_{o'\in\mathcal{O}}p^{(t)}(o'\mid z,a)\exp\left(\eta g^{(t)}(z,a,o')\right)}. p ( t + 1 ) ( o ∣ z , a ) = ∑ o ′ ∈ O p ( t ) ( o ′ ∣ z , a ) exp ( η g ( t ) ( z , a , o ′ ) ) p ( t ) ( o ∣ z , a ) exp ( η g ( t ) ( z , a , o ) ) .
Both the numerator and the normalization denominator depend on z z z only through ψ ( z ) \psi(z) ψ ( z ) . Hence p ( t + 1 ) ( o ∣ z , a ) p^{(t+1)}(o\mid z,a) p ( t + 1 ) ( o ∣ z , a ) has the same feature dependence, completing the induction. ■
We now consider randomized collaboration policies that operate on a fixed feature representation ψ ( z ) \psi(z) ψ ( z ) . Each policy forms linear action scores from these features and converts them into action probabilities through a fixed Lipschitz map. Under bounds on the features and policy parameters, we use Rademacher complexity to establish a sample-complexity guarantee for COI auditing over the resulting policy class H ψ \mathcal{H}_\psi H ψ .
Assumption D.2 (Lipschitz Policies over Bounded Features). Let ψ : Z → R k \psi:\mathcal{Z}\to\mathbb{R}^k ψ : Z → R k be a fixed feature map satisfying sup z ∈ Z ∥ ψ ( z ) ∥ 2 ≤ B \sup_{z\in\mathcal{Z}}\|\psi(z)\|_2\le B sup z ∈ Z ∥ ψ ( z ) ∥ 2 ≤ B . Let ϕ : R ∣ A ∣ → Δ ( A ) \phi:\mathbb{R}^{|\mathcal{A}|}\to\Delta(\mathcal{A}) ϕ : R ∣ A ∣ → Δ ( A ) be a fixed map from action scores to action probabilities satisfying
∥ ϕ ( u ) − ϕ ( v ) ∥ 2 ≤ Λ ∥ u − v ∥ 2 for all u , v ∈ R ∣ A ∣ . \|\phi(u)-\phi(v)\|_2\le\Lambda\|u-v\|_2\qquad\text{for all }u,v\in\mathbb{R}^{|\mathcal{A}|}. ∥ ϕ ( u ) − ϕ ( v ) ∥ 2 ≤ Λ∥ u − v ∥ 2 for all u , v ∈ R ∣ A ∣ . Fix W max > 0 W_{\max}>0 W m a x > 0 and define the randomized policy family
H ψ = { h ( ⋅ ∣ z ) = ϕ ( W ψ ( z ) ) : W ∈ R ∣ A ∣ × k , ∥ W ∥ F ≤ W max } , \mathcal{H}_\psi=\left\{h(\cdot\mid z)=\phi(W\psi(z)):W\in\mathbb{R}^{|\mathcal{A}|\times k},\ \|W\|_F\le W_{\max}\right\}, H ψ = { h ( ⋅ ∣ z ) = ϕ ( W ψ ( z )) : W ∈ R ∣ A ∣ × k , ∥ W ∥ F ≤ W m a x } , where
∥ W ∥ F 2 = ∑ a ∈ A ∑ j = 1 k W a , j 2 . \|W\|_F^2=\sum_{a\in\mathcal{A}}\sum_{j=1}^{k}W_{a,j}^2. ∥ W ∥ F 2 = a ∈ A ∑ j = 1 ∑ k W a , j 2 . The feature and Lipschitz bounds use Euclidean norms, and the parameter constraint uses the Frobenius norm.
Theorem D.3 (Sample Complexity for Feature-Based Policies). Suppose Assumptions B .1 B.1 B .1 , D .2 D.2 D .2 , and B .2 ( a , b ) B.2(a,b) B .2 ( a , b ) hold, and the logging propensities are known. Fix a finite inverse temperature τ > 0 \tau> 0 τ > 0 and a predictor p ( t ) p^{(t)} p ( t ) independently of the audit batch. For each ω ∈ Ω \omega\in\Omega ω ∈ Ω , let π ω = k ω ( p ( t ) ) \pi^\omega= k^\omega(p^{(t)}) π ω = k ω ( p ( t ) ) , and define
Π e v a l ( p ( t ) ) = ( H ψ × Ω ) ∪ { ( π ω , ω ) : ω ∈ Ω } . \Pi_{\mathrm{eval}}(p^{(t)}) = (\mathcal{H}_{\psi} \times\Omega) \cup\{(\pi^\omega,\omega) : \omega\in\Omega\}. Π eval ( p ( t ) ) = ( H ψ × Ω ) ∪ {( π ω , ω ) : ω ∈ Ω } . For ε , δ ∈ ( 0 , 1 ) \varepsilon,\delta\in(0,1) ε , δ ∈ ( 0 , 1 ) , draw m m m independent samples from the logging distribution. Whenever
m ≳ ( 1 + B c ) 2 γ 2 ε 2 [ ∣ A ∣ Λ 2 B 2 W max 2 + ( 1 + τ ) 2 ∣ O ∣ + log 2 ∣ C ∣ δ ] , m \gtrsim\frac{(1+B_c)^2}{\gamma^2\varepsilon^2}\left[|\mathcal{A}|\Lambda^2 B^2 W_{\max}^2 + (1+\tau)^2|\mathcal{O}| + \log\frac{2|\mathcal{C}|}{\delta}\right], m ≳ γ 2 ε 2 ( 1 + B c ) 2 [ ∣ A ∣ Λ 2 B 2 W m a x 2 + ( 1 + τ ) 2 ∣ O ∣ + log δ 2∣ C ∣ ] , where the implicit constant is universal, with probability at least 1 − δ 1-\delta 1 − δ ,
sup ( π , ω ) ∈ Π e v a l ( p ( t ) ) ∣ G ^ m ( π ; ω ; p ( t ) ) − G ( π ; ω ; p ( t ) ) ∣ ≤ ε 8 . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}(p^{(t)})}\left|\widehat{G}_m(\pi;\omega;p^{(t)})-G(\pi;\omega;p^{(t)})\right| \le\frac{\varepsilon}{8}. ( π , ω ) ∈ Π eval ( p ( t ) ) sup G m ( π ; ω ; p ( t ) ) − G ( π ; ω ; p ( t ) ) ≤ 8 ε . Consequently, consider an auditor that maximizes the absolute empirical COI gap over Π e v a l ( p ( t ) ) \Pi_{\mathrm{eval}}(p^{(t)}) Π eval ( p ( t ) ) to additive accuracy ε / 8 \varepsilon/8 ε /8 . If the selected pair has absolute empirical gap greater than 3 ε / 4 3\varepsilon/4 3 ε /4 , the auditor returns this pair together with the sign of its empirical gap; otherwise, it returns ⊥ \perp ⊥ . On the same event, the auditor satisfies:
Every returned certificate ( π , ω , σ ) (\pi,\omega,\sigma) ( π , ω , σ ) satisfies
σ ( V ( π ; ω ; p ∗ ) − V ( π ; ω ; p ( t ) ) ) > ε / 2. \sigma\left(V(\pi;\omega;p^*)-V(\pi;\omega;p^{(t)})\right) > \varepsilon/2. σ ( V ( π ; ω ; p ∗ ) − V ( π ; ω ; p ( t ) ) ) > ε /2. If the auditor returns ⊥ \perp ⊥ , then p ( t ) p^{(t)} p ( t ) satisfies ( Ω , H ψ , ε ) (\Omega,\mathcal{H}_{\psi},\varepsilon) ( Ω , H ψ , ε ) -COI.
Remark D.4. The sample bound has no explicit dependence on the feature dimension k k k when the feature norm bound B B B , the Frobenius norm bound W max W_{\max} W m a x , and the Lipschitz constant Λ \Lambda Λ are fixed. These quantities may themselves depend on the chosen feature representation, so the dimension dependence is controlled through the stated norm and smoothness bounds.
The auditing guarantee assumes that the absolute empirical COI gap can be maximized to additive accuracy ε / 8 \varepsilon/8 ε /8 . The theorem establishes the sample complexity and validity of the resulting auditor; its computational efficiency additionally depends on the procedure used to solve this empirical optimization problem.
Proof. The auditor’s objective is to estimate the difference in expected loss between the true environment and the base predictive distribution uniformly over ( π , ω ) ∈ Π e v a l ( p ( t ) ) (\pi,\omega) \in\Pi_{\mathrm{eval}}(p^{(t)}) ( π , ω ) ∈ Π eval ( p ( t ) ) . Recall that the population and empirical COI gaps are
G ( π ; ω ; p ( t ) ) = V ( π ; ω ; p ∗ ) − V ( π ; ω ; p ( t ) ) G(\pi;\omega;p^{(t)}) = V(\pi;\omega;p^*) - V(\pi;\omega;p^{(t)}) G ( π ; ω ; p ( t ) ) = V ( π ; ω ; p ∗ ) − V ( π ; ω ; p ( t ) )
and
G ^ m ( π ; ω ; p ( t ) ) = 1 m ∑ i = 1 m [ π ( a i ∣ z i ) π l o g ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − ∑ a ∈ A π ( a ∣ z i ) E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o ) ] ] . \widehat{G}_m(\pi;\omega;p^{(t)}) = \frac{1}{m}\sum_{i=1}^{m}\left[\frac{\pi(a_i\mid z_i)}{\pi_{\mathrm{log}}(a_i\mid z_i)}\ell^\omega_{a_i}(z_i,o_i)-\sum_{a\in\mathcal{A}}\pi(a\mid z_i)\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z_i,a)}[\ell^\omega_a(z_i,o)]\right]. G m ( π ; ω ; p ( t ) ) = m 1 i = 1 ∑ m [ π log ( a i ∣ z i ) π ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − a ∈ A ∑ π ( a ∣ z i ) E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o )] ] .
Since p ( t ) p^{(t)} p ( t ) is independent of the audit batch, we may condition on it throughout the proof. For each fixed ( π , ω ) (\pi,\omega) ( π , ω ) , the empirical gap is an unbiased estimator of the population gap.
By Assumption B.2(a,b), ∣ ℓ a ω ( z , o ) ∣ ≤ 1 + B c \lvert\ell^{\omega}_{a}(z,o)\rvert\le1+B_c ∣ ℓ a ω ( z , o )∣ ≤ 1 + B c . Set M = ( 1 + B c ) / γ M=(1+B_c)/\gamma M = ( 1 + B c ) / γ . Each scalar contribution to G ^ m \widehat{G}_m G m has absolute value at most
1 + B c γ + ( 1 + B c ) ≤ 2 M . \frac{1+B_c}{\gamma}+(1+B_c)\le2M. γ 1 + B c + ( 1 + B c ) ≤ 2 M .
We decompose the evaluation class as Π e v a l ( p ( t ) ) = Π H ∪ Π s o f t \Pi_{\mathrm{eval}}(p^{(t)})=\Pi_{\mathcal{H}}\cup\Pi_{\mathrm{soft}} Π eval ( p ( t ) ) = Π H ∪ Π soft , where
Π H = { ( h , ω ) : h ∈ H ψ , ω ∈ Ω } , Π s o f t = { ( k ω ( p ( t ) ) , ω ) : ω ∈ Ω } . \Pi_{\mathcal{H}}=\{(h,\omega):h\in\mathcal{H}_{\psi},\ \omega\in\Omega\},\qquad\Pi_{\mathrm{soft}}=\{(k^{\omega}(p^{(t)}),\omega):\omega\in\Omega\}. Π H = {( h , ω ) : h ∈ H ψ , ω ∈ Ω } , Π soft = {( k ω ( p ( t ) ) , ω ) : ω ∈ Ω } .
Step 1: Rademacher Complexity of Π H \Pi_{\mathcal{H}} Π H .
Fix ω \omega ω and the audit sample. For h ( ⋅ ∣ z ) = ϕ ( W ψ ( z ) ) h(\cdot\mid z)=\phi(W\psi(z)) h ( ⋅ ∣ z ) = ϕ ( W ψ ( z )) , the i i i -th term in G ^ m ( h ; ω ; p ( t ) ) \widehat{G}_m(h;\omega;p^{(t)}) G m ( h ; ω ; p ( t ) ) is
ϕ ( W ψ ( z i ) ) a i π log ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − ∑ a ∈ A ϕ ( W ψ ( z i ) ) a E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o ) ] . \frac{\phi(W\psi(z_i))_{a_i}}{\pi_{\log}(a_i\mid z_i)}\ell^{\omega}_{a_i}(z_i,o_i)-\sum_{a\in\mathcal{A}}\phi(W\psi(z_i))_a\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z_i,a)}[\ell^{\omega}_a(z_i,o)]. π l o g ( a i ∣ z i ) ϕ ( W ψ ( z i ) ) a i ℓ a i ω ( z i , o i ) − a ∈ A ∑ ϕ ( W ψ ( z i ) ) a E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o )] .
For two parameter matrices W W W and W ′ W' W ′ , we have
∣ ϕ ( W ψ ( z i ) ) a i − ϕ ( W ′ ψ ( z i ) ) a i π log ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − ∑ a ∈ A ( ϕ ( W ψ ( z i ) ) a − ϕ ( W ′ ψ ( z i ) ) a ) E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o ) ] ∣ ≤ 1 + B c π log ( a i ∣ z i ) ∣ ϕ ( W ψ ( z i ) ) a i − ϕ ( W ′ ψ ( z i ) ) a i ∣ + ( 1 + B c ) ∑ a ∈ A ∣ ϕ ( W ψ ( z i ) ) a − ϕ ( W ′ ψ ( z i ) ) a ∣ ≤ ( 1 + B c ) ( 1 π log ( a i ∣ z i ) + ∣ A ∣ ) ∥ ϕ ( W ψ ( z i ) ) − ϕ ( W ′ ψ ( z i ) ) ∥ 2 ≤ Λ ( 1 + B c ) ( 1 π log ( a i ∣ z i ) + ∣ A ∣ ) ∥ ( W − W ′ ) ψ ( z i ) ∥ 2 ≤ 2 Λ M ∥ ( W − W ′ ) ψ ( z i ) ∥ 2 . \begin{aligned}
&\left\lvert\frac{\phi(W\psi(z_i))_{a_i}-\phi(W'\psi(z_i))_{a_i}}{\pi_{\log}(a_i\mid z_i)}\ell^{\omega}_{a_i}(z_i,o_i)-\sum_{a\in\mathcal{A}}\bigl(\phi(W\psi(z_i))_a-\phi(W'\psi(z_i))_a\bigr)\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z_i,a)}[\ell^{\omega}_a(z_i,o)]\right\rvert\\
&\le\frac{1+B_c}{\pi_{\log}(a_i\mid z_i)}\left\lvert\phi(W\psi(z_i))_{a_i}-\phi(W'\psi(z_i))_{a_i}\right\rvert+(1+B_c)\sum_{a\in\mathcal{A}}\left\lvert\phi(W\psi(z_i))_a-\phi(W'\psi(z_i))_a\right\rvert\\
&\le(1+B_c)\left(\frac{1}{\pi_{\log}(a_i\mid z_i)}+\sqrt{\lvert\mathcal{A}\rvert}\right)\left\|\phi(W\psi(z_i))-\phi(W'\psi(z_i))\right\|_2\\
&\le\Lambda(1+B_c)\left(\frac{1}{\pi_{\log}(a_i\mid z_i)}+\sqrt{\lvert\mathcal{A}\rvert}\right)\left\|(W-W')\psi(z_i)\right\|_2\\
&\le2\Lambda M\left\|(W-W')\psi(z_i)\right\|_2.
\end{aligned} π l o g ( a i ∣ z i ) ϕ ( W ψ ( z i ) ) a i − ϕ ( W ′ ψ ( z i ) ) a i ℓ a i ω ( z i , o i ) − a ∈ A ∑ ( ϕ ( W ψ ( z i ) ) a − ϕ ( W ′ ψ ( z i ) ) a ) E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o )] ≤ π l o g ( a i ∣ z i ) 1 + B c ∣ ϕ ( W ψ ( z i ) ) a i − ϕ ( W ′ ψ ( z i ) ) a i ∣ + ( 1 + B c ) a ∈ A ∑ ∣ ϕ ( W ψ ( z i ) ) a − ϕ ( W ′ ψ ( z i ) ) a ∣ ≤ ( 1 + B c ) ( π l o g ( a i ∣ z i ) 1 + ∣ A ∣ ) ∥ ϕ ( W ψ ( z i )) − ϕ ( W ′ ψ ( z i )) ∥ 2 ≤ Λ ( 1 + B c ) ( π l o g ( a i ∣ z i ) 1 + ∣ A ∣ ) ∥ ( W − W ′ ) ψ ( z i ) ∥ 2 ≤ 2Λ M ∥ ( W − W ′ ) ψ ( z i ) ∥ 2 .
The first inequality uses the loss bound. The second follows from Cauchy–Schwarz, and the third uses the Euclidean Lipschitz assumption on ϕ \phi ϕ . The last inequality follows from π log ( a i ∣ z i ) ≥ γ \pi_{\log}(a_i\mid z_i)\ge\gamma π l o g ( a i ∣ z i ) ≥ γ and ∣ A ∣ γ ≤ 1 \lvert\mathcal{A}\rvert\gamma\le1 ∣ A ∣ γ ≤ 1 . Thus each scalar contribution to the IPS estimator is 2 Λ M 2\Lambda M 2Λ M -Lipschitz in the action-score vector W ψ ( z i ) W\psi(z_i) W ψ ( z i ) under the Euclidean norm.
Let σ i \sigma_i σ i and σ i , a \sigma_{i,a} σ i , a be independent Rademacher signs, each taking values + 1 +1 + 1 and − 1 -1 − 1 with equal probability. The empirical Rademacher complexity under consideration is
E σ sup ∥ W ∥ F ≤ W max 1 m ∑ i = 1 m σ i [ ϕ ( W ψ ( z i ) ) a i π log ( a i ∣ z i ) ℓ a i ω ( z i , o i ) − ∑ a ∈ A ϕ ( W ψ ( z i ) ) a E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o ) ] ] . \mathbb{E}_{\sigma}\sup_{\lVert W\rVert_F\le W_{\max}}\frac{1}{m}\sum_{i=1}^{m}\sigma_i\left[\frac{\phi(W\psi(z_i))_{a_i}}{\pi_{\log}(a_i\mid z_i)}\ell^{\omega}_{a_i}(z_i,o_i)-\sum_{a\in\mathcal{A}}\phi(W\psi(z_i))_a\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z_i,a)}[\ell^{\omega}_a(z_i,o)]\right]. E σ ∥ W ∥ F ≤ W m a x sup m 1 i = 1 ∑ m σ i [ π l o g ( a i ∣ z i ) ϕ ( W ψ ( z i ) ) a i ℓ a i ω ( z i , o i ) − a ∈ A ∑ ϕ ( W ψ ( z i ) ) a E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ω ( z i , o )] ] .
Since each expression in brackets is 2 Λ M 2\Lambda M 2Λ M -Lipschitz in the score vector W ψ ( z i ) W\psi(z_i) W ψ ( z i ) , the vector-contraction inequality bounds this quantity by
2 2 Λ M m E σ sup ∥ W ∥ F ≤ W max ∑ i = 1 m ∑ a ∈ A σ i , a ( W ψ ( z i ) ) a . \frac{2\sqrt{2}\Lambda M}{m}\mathbb{E}_{\sigma}\sup_{\lVert W\rVert_F\le W_{\max}}\sum_{i=1}^{m}\sum_{a\in\mathcal{A}}\sigma_{i,a}(W\psi(z_i))_a. m 2 2 Λ M E σ ∥ W ∥ F ≤ W m a x sup i = 1 ∑ m a ∈ A ∑ σ i , a ( W ψ ( z i ) ) a .
We now bound the remaining linear expression. Expanding the matrix-vector product gives
∑ i = 1 m ∑ a ∈ A σ i , a ( W ψ ( z i ) ) a = ∑ a ∈ A ∑ j = 1 k W a , j ( ∑ i = 1 m σ i , a ψ j ( z i ) ) . \sum_{i=1}^{m}\sum_{a\in\mathcal{A}}\sigma_{i,a}(W\psi(z_i))_a=\sum_{a\in\mathcal{A}}\sum_{j=1}^{k}W_{a,j}\left(\sum_{i=1}^{m}\sigma_{i,a}\psi_j(z_i)\right). i = 1 ∑ m a ∈ A ∑ σ i , a ( W ψ ( z i ) ) a = a ∈ A ∑ j = 1 ∑ k W a , j ( i = 1 ∑ m σ i , a ψ j ( z i ) ) .
By Cauchy–Schwarz and ∥ W ∥ F ≤ W max \lVert W\rVert_F \le W_{\max} ∥ W ∥ F ≤ W m a x
sup ∥ W ∥ F ≤ W max ∑ i = 1 m ∑ a ∈ A σ i , a ( W ψ ( z i ) ) a ≤ W max ( ∑ a ∈ A ∥ ∑ i = 1 m σ i , a ψ ( z i ) ∥ 2 2 ) 1 / 2 . \sup_{\lVert W\rVert_F\le W_{\max}}\sum_{i=1}^{m}\sum_{a\in\mathcal{A}}\sigma_{i,a}(W\psi(z_i))_a \le W_{\max}\left(\sum_{a\in\mathcal{A}}\left\lVert\sum_{i=1}^{m}\sigma_{i,a}\psi(z_i)\right\rVert_2^2\right)^{1/2}. ∥ W ∥ F ≤ W m a x sup i = 1 ∑ m a ∈ A ∑ σ i , a ( W ψ ( z i ) ) a ≤ W m a x a ∈ A ∑ i = 1 ∑ m σ i , a ψ ( z i ) 2 2 1/2 .
Taking expectations and applying Jensen’s inequality yields
E σ sup ∥ W ∥ F ≤ W max ∑ i = 1 m ∑ a ∈ A σ i , a ( W ψ ( z i ) ) a ≤ W max ( ∑ a ∈ A E σ ∥ ∑ i = 1 m σ i , a ψ ( z i ) ∥ 2 2 ) 1 / 2 . \mathbb{E}_{\sigma}\sup_{\lVert W\rVert_F\le W_{\max}}\sum_{i=1}^{m}\sum_{a\in\mathcal{A}}\sigma_{i,a}(W\psi(z_i))_a \le W_{\max}\left(\sum_{a\in\mathcal{A}}\mathbb{E}_{\sigma}\left\lVert\sum_{i=1}^{m}\sigma_{i,a}\psi(z_i)\right\rVert_2^2\right)^{1/2}. E σ ∥ W ∥ F ≤ W m a x sup i = 1 ∑ m a ∈ A ∑ σ i , a ( W ψ ( z i ) ) a ≤ W m a x a ∈ A ∑ E σ i = 1 ∑ m σ i , a ψ ( z i ) 2 2 1/2 .
Independence and zero means of the signs eliminate the cross terms, so
∑ a ∈ A E σ ∥ ∑ i = 1 m σ i , a ψ ( z i ) ∥ 2 2 = ∣ A ∣ ∑ i = 1 m ∥ ψ ( z i ) ∥ 2 2 ≤ ∣ A ∣ m B 2 . \sum_{a\in\mathcal{A}}\mathbb{E}_{\sigma}\left\lVert\sum_{i=1}^{m}\sigma_{i,a}\psi(z_i)\right\rVert_2^2=|\mathcal{A}|\sum_{i=1}^{m}\lVert\psi(z_i)\rVert_2^2\le|\mathcal{A}|mB^2. a ∈ A ∑ E σ i = 1 ∑ m σ i , a ψ ( z i ) 2 2 = ∣ A ∣ i = 1 ∑ m ∥ ψ ( z i ) ∥ 2 2 ≤ ∣ A ∣ m B 2 .
Consequently, the empirical Rademacher complexity of the scalar IPS contributions, with ω \omega ω fixed, is at most
2 2 Λ M B W max ∣ A ∣ m . \frac{2\sqrt{2}\Lambda MBW_{\max}\sqrt{|\mathcal{A}|}}{\sqrt{m}}. m 2 2 Λ M B W m a x ∣ A ∣ .
For fixed h h h and c c c , the estimation error G ^ m − G \widehat{G}_m-G G m − G is affine in the loss vector l ~ ∈ [ 0 , 1 ] ∣ O ∣ \widetilde{l}\in[0,1]^{|\mathcal{O}|} l ∈ [ 0 , 1 ] ∣ O ∣ . Its maximum absolute value is therefore attained at a vertex of this cube. Applying standard Rademacher generalization bounds uniformly over h ∈ H ψ h\in\mathcal{H}_{\psi} h ∈ H ψ and taking a union bound over the 2 ∣ O ∣ 2^{|\mathcal{O}|} 2 ∣ O ∣ vertices and c ∈ C c\in\mathcal{C} c ∈ C gives, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 ,
sup ( h , ω ) ∈ Π H ∣ G ^ m ( h ; ω ; p ( t ) ) − G ( h ; ω ; p ( t ) ) ∣ ≲ M m ( Λ B W max ∣ A ∣ + ∣ O ∣ + log 2 ∣ C ∣ δ ) , \sup_{(h,\omega)\in\Pi_H}\left|\widehat{G}_m(h;\omega;p^{(t)})-G(h;\omega;p^{(t)})\right|\lesssim\frac{M}{\sqrt{m}}\left(\Lambda BW_{\max}\sqrt{|\mathcal{A}|}+\sqrt{|\mathcal{O}|+\log\frac{2|\mathcal{C}|}{\delta}}\right), ( h , ω ) ∈ Π H sup G m ( h ; ω ; p ( t ) ) − G ( h ; ω ; p ( t ) ) ≲ m M ( Λ B W m a x ∣ A ∣ + ∣ O ∣ + log δ 2∣ C ∣ ) ,
where ≲ \lesssim ≲ denotes an inequality up to a universal multiplicative constant.
Step 2: Uniform Convergence.
We next control Π s o f t \Pi_{\mathrm{soft}} Π soft . For fixed c c c , changing the loss vector from l ~ \widetilde{l} l to l ~ ′ \widetilde{l}' l ′ changes each predicted action loss by at most ∥ l ~ − l ~ ′ ∥ ∞ \lVert\widetilde{l}-\widetilde{l}'\rVert_{\infty} ∥ l − l ′ ∥ ∞ . The softmax rule consequently satisfies
∥ k ( l ~ , c ) ( p ( t ) ) ( ⋅ ∣ z ) − k ( l ~ ′ , c ) ( p ( t ) ) ( ⋅ ∣ z ) ∥ 1 ≤ 2 τ ∥ l ~ − l ~ ′ ∥ ∞ . \left\lVert k^{(\widetilde{l},c)}(p^{(t)})(\cdot\mid z)-k^{(\widetilde{l}',c)}(p^{(t)})(\cdot\mid z)\right\rVert_1\le2\tau\lVert\widetilde{l}-\widetilde{l}'\rVert_{\infty}. k ( l , c ) ( p ( t ) ) ( ⋅ ∣ z ) − k ( l ′ , c ) ( p ( t ) ) ( ⋅ ∣ z ) 1 ≤ 2 τ ∥ l − l ′ ∥ ∞ .
Combining this with the loss and propensity bounds, the change in each sample’s contribution to G ^ m \widehat{G}_m G m satisfies
∣ k ( l ~ , c ) ( p ( t ) ) ( a i ∣ z i ) ℓ a i ( l ~ , c ) ( z i , o i ) − k ( l ~ ′ , c ) ( p ( t ) ) ( a i ∣ z i ) ℓ a i ( l ~ ′ , c ) ( z i , o i ) π log ( a i ∣ z i ) − ∑ a ∈ A ( k ( l ~ , c ) ( p ( t ) ) ( a ∣ z i ) E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ( l ~ , c ) ( z i , o ) ] − k ( l ~ ′ , c ) ( p ( t ) ) ( a ∣ z i ) E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ( l ~ ′ , c ) ( z i , o ) ] ) ∣ ≤ ( 1 + 1 γ ) ( 1 + 2 τ ( 1 + B c ) ) ∥ l ~ − l ~ ′ ∥ ∞ ≤ 4 M ( 1 + τ ) ∥ l ~ − l ~ ′ ∥ ∞ . \left|
\frac{k^{(\widetilde{l},c)}(p^{(t)})(a_i\mid z_i)\ell_{a_i}^{(\widetilde{l},c)}(z_i,o_i)-k^{(\widetilde{l}',c)}(p^{(t)})(a_i\mid z_i)\ell_{a_i}^{(\widetilde{l}',c)}(z_i,o_i)}{\pi_{\log}(a_i\mid z_i)}
-\sum_{a\in\mathcal{A}}\left(k^{(\widetilde{l},c)}(p^{(t)})(a\mid z_i)\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z_i,a)}[\ell_a^{(\widetilde{l},c)}(z_i,o)]-k^{(\widetilde{l}',c)}(p^{(t)})(a\mid z_i)\mathbb{E}_{o\sim p^{(t)}(\cdot\mid z_i,a)}[\ell_a^{(\widetilde{l}',c)}(z_i,o)]\right)
\right|
\le\left(1+\frac{1}{\gamma}\right)(1+2\tau(1+B_c))\lVert\widetilde{l}-\widetilde{l}'\rVert_{\infty}\le4M(1+\tau)\lVert\widetilde{l}-\widetilde{l}'\rVert_{\infty}. π l o g ( a i ∣ z i ) k ( l , c ) ( p ( t ) ) ( a i ∣ z i ) ℓ a i ( l , c ) ( z i , o i ) − k ( l ′ , c ) ( p ( t ) ) ( a i ∣ z i ) ℓ a i ( l ′ , c ) ( z i , o i ) − a ∈ A ∑ ( k ( l , c ) ( p ( t ) ) ( a ∣ z i ) E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ( l , c ) ( z i , o )] − k ( l ′ , c ) ( p ( t ) ) ( a ∣ z i ) E o ∼ p ( t ) ( ⋅ ∣ z i , a ) [ ℓ a ( l ′ , c ) ( z i , o )] ) ≤ ( 1 + γ 1 ) ( 1 + 2 τ ( 1 + B c )) ∥ l − l ′ ∥ ∞ ≤ 4 M ( 1 + τ ) ∥ l − l ′ ∥ ∞ .
An ℓ ∞ \ell_\infty ℓ ∞ cover of the loss cube therefore induces a cover of these scalar contributions under the normalized empirical L 2 L_2 L 2 metric. Centering at the loss vector ℓ ~ = 0 \tilde{\ell}=0 ℓ ~ = 0 removes a fixed offset without changing the empirical Rademacher complexity. Applying Dudley’s entropy integral then shows that this complexity, for fixed c c c , is ≲ M ( 1 + τ ) ∣ O ∣ / m \lesssim M(1+\tau)\sqrt{|\mathcal{O}|/m} ≲ M ( 1 + τ ) ∣ O ∣/ m .
Using the envelope bound 2 M 2M 2 M and a union bound over c ∈ C c\in\mathcal{C} c ∈ C , we obtain, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2 ,
sup ω ∈ Ω ∣ G ^ m ( k ω ( p ( t ) ) ; ω ; p ( t ) ) − G ( k ω ( p ( t ) ) ; ω ; p ( t ) ) ∣ ≲ M m ( ( 1 + τ ) ∣ O ∣ + log 2 ∣ C ∣ δ ) . \sup_{\omega\in\Omega}\left|\widehat{G}_m(k^\omega(p^{(t)});\omega;p^{(t)})-G(k^\omega(p^{(t)});\omega;p^{(t)})\right|
\lesssim\frac{M}{\sqrt{m}}\left((1+\tau)\sqrt{|\mathcal{O}|}+\sqrt{\log\frac{2|\mathcal{C}|}{\delta}}\right). ω ∈ Ω sup G m ( k ω ( p ( t ) ) ; ω ; p ( t ) ) − G ( k ω ( p ( t ) ) ; ω ; p ( t ) ) ≲ m M ( ( 1 + τ ) ∣ O ∣ + log δ 2∣ C ∣ ) .
Combining the two bounds, with probability at least 1 − δ 1-\delta 1 − δ ,
sup ( π , ω ) ∈ Π e v a l ( p ( t ) ) ∣ G ^ m ( π ; ω ; p ( t ) ) − G ( π ; ω ; p ( t ) ) ∣ ≲ 1 + B c γ m ( Λ B W max ∣ A ∣ + ( 1 + τ ) ∣ O ∣ + log 2 ∣ C ∣ δ ) . \sup_{(\pi,\omega)\in\Pi_{\mathrm{eval}}(p^{(t)})}\left|\widehat{G}_m(\pi;\omega;p^{(t)})-G(\pi;\omega;p^{(t)})\right|
\lesssim\frac{1+B_c}{\gamma\sqrt{m}}\left(\Lambda B W_{\max}\sqrt{|\mathcal{A}|}+(1+\tau)\sqrt{|\mathcal{O}|}+\sqrt{\log\frac{2|\mathcal{C}|}{\delta}}\right). ( π , ω ) ∈ Π eval ( p ( t ) ) sup G m ( π ; ω ; p ( t ) ) − G ( π ; ω ; p ( t ) ) ≲ γ m 1 + B c ( Λ B W m a x ∣ A ∣ + ( 1 + τ ) ∣ O ∣ + log δ 2∣ C ∣ ) .
Thus the uniform estimation error is at most ε / 8 \varepsilon/8 ε /8 whenever
m ≳ ( 1 + B c ) 2 γ 2 ε 2 ( ∣ A ∣ Λ 2 B 2 W max 2 + ( 1 + τ ) 2 ∣ O ∣ + log 2 ∣ C ∣ δ ) . m\gtrsim\frac{(1+B_c)^2}{\gamma^2\varepsilon^2}\left(|\mathcal{A}|\Lambda^2B^2W_{\max}^2+(1+\tau)^2|\mathcal{O}|+\log\frac{2|\mathcal{C}|}{\delta}\right). m ≳ γ 2 ε 2 ( 1 + B c ) 2 ( ∣ A ∣ Λ 2 B 2 W m a x 2 + ( 1 + τ ) 2 ∣ O ∣ + log δ 2∣ C ∣ ) .
where the universal implicit constant in this sample-size requirement is sufficiently large.
On this event, consider an ε / 8 \varepsilon/8 ε /8 -approximate maximizer ( π , ω ) (\pi,\omega) ( π , ω ) of the absolute empirical gap. If its absolute empirical gap exceeds 3 ε / 4 3\varepsilon/4 3 ε /4 , return this pair with σ = sign ( G ^ m ( π ; ω ; p ( t ) ) ) \sigma=\operatorname{sign}(\widehat{G}_m(\pi;\omega;p^{(t)})) σ = sign ( G m ( π ; ω ; p ( t ) )) . Its signed true gap satisfies
σ G ( π ; ω ; p ( t ) ) ≥ ∣ G ^ m ( π ; ω ; p ( t ) ) ∣ − ε 8 > 5 ε 8 > ε 2 . \sigma G(\pi;\omega;p^{(t)})\geq\left|\widehat{G}_m(\pi;\omega;p^{(t)})\right|-\frac{\varepsilon}{8}>\frac{5\varepsilon}{8}>\frac{\varepsilon}{2}. σ G ( π ; ω ; p ( t ) ) ≥ G m ( π ; ω ; p ( t ) ) − 8 ε > 8 5 ε > 2 ε .
Otherwise, the supremum of the absolute empirical gap is at most 7 ε / 8 7\varepsilon/8 7 ε /8 , so the supremum of the absolute true gap is at most ε \varepsilon ε . Returning ⊥ \perp ⊥ therefore certifies ( Ω , H ψ , ε ) (\Omega,\mathcal{H}_{\psi},\varepsilon) ( Ω , H ψ , ε ) -COI. This completes the proof. ■
Extension to Multi-Agent Settings
The proposed collaborative omnimechanism framework also accommodates routing among K K K human expert agents with different query costs. We incorporate the expert identity into the action space by writing a = ( r , s , g , j ) a=(r,s,g,j) a = ( r , s , g , j ) , where j ∈ [ K ] j\in[K] j ∈ [ K ] identifies the expert queried when r = 1 r=1 r = 1 , and r = 0 r=0 r = 0 corresponds to automation. Thus each deferred action queries a single expert. Under this product representation, the enlarged action space A ′ \mathcal{A}' A ′ has size 2 K ∣ S ∣ ∣ G ∣ 2K|\mathcal{S}||\mathcal{G}| 2 K ∣ S ∣∣ G ∣ . Differences in experts’ responses and impressionability are represented by the action-dependent outcome distribution p ⋆ ( o ∣ z , a ) p^\star(o\mid z,a) p ⋆ ( o ∣ z , a ) .
Let C \mathcal{C} C be a finite family of expert-cost profiles c = ( c 1 , … , c K ) c=(c_1,\ldots,c_K) c = ( c 1 , … , c K ) , with 0 ≤ c j ( x ) ≤ B c 0\leq c_j(x)\leq B_c 0 ≤ c j ( x ) ≤ B c uniformly over c ∈ C c\in\mathcal{C} c ∈ C , j ∈ [ K ] j\in[K] j ∈ [ K ] , and x x x . For ω = ( ℓ ~ , c ) \omega=(\tilde{\ell},c) ω = ( ℓ ~ , c ) , the total loss is
ℓ a ω ( z , o ) = ℓ ~ ( o ) + r c j ( x ) . \ell_a^\omega(z,o)=\tilde{\ell}(o)+rc_j(x). ℓ a ω ( z , o ) = ℓ ~ ( o ) + r c j ( x ) .
Since each deferred action queries only one expert, the total loss remains bounded by 1 + B c 1+B_c 1 + B c .
Let H ′ \mathcal{H}^{\prime} H ′ denote the resulting collaboration policy class, with Natarajan dimension d H ′ d_{\mathcal{H}^{\prime}} d H ′ . Assume that the logging propensities are known and satisfy π l o g ( a ∣ z ) ≥ γ ′ > 0 \pi_{\mathrm{log}}(a \mid z) \ge\gamma^{\prime} > 0 π log ( a ∣ z ) ≥ γ ′ > 0 for every a ∈ A ′ a \in\mathcal{A}^{\prime} a ∈ A ′ and state z z z . For fixed finite inverse temperature τ > 0 \tau> 0 τ > 0 , under the remaining assumptions of the auditing theorem, the sample-complexity analysis applies to this enlarged action space and expert-dependent loss. In particular, a sufficient number of samples per audit is
m ≳ ( 1 + B c ) 2 ( γ ′ ) 2 ε 2 [ d H ′ log ∣ A ′ ∣ + ( 1 + τ ) 2 ∣ O ∣ + log 2 ∣ C ∣ δ ] . m \gtrsim\frac{(1+B_c)^2}{(\gamma^{\prime})^2\varepsilon^2}\left[d_{\mathcal{H}^{\prime}}\log\lvert\mathcal{A}^{\prime}\rvert+(1+\tau)^2\lvert\mathcal{O}\rvert+\log\frac{2\lvert\mathcal{C}\rvert}{\delta}\right]. m ≳ ( γ ′ ) 2 ε 2 ( 1 + B c ) 2 [ d H ′ log ∣ A ′ ∣ + ( 1 + τ ) 2 ∣ O ∣ + log δ 2 ∣ C ∣ ] .
where ∣ C ∣ \lvert\mathcal{C}\rvert ∣ C ∣ counts the admissible expert-cost profiles.
The statistical cost of accommodating additional experts is therefore governed by the enlarged policy class, action space, logging coverage, and family of cost profiles. Computational efficiency additionally depends on the procedure used to optimize the empirical auditing objective over H ′ \mathcal{H}^{\prime} H ′ .
Additional Details for Experiment 1: Impressionable Human Decisions
Dataset
We use the human-AI interaction dataset of [35 ] . The dataset contains more than 30,000 human-AI interactions across several binary classification tasks. Each interaction records the human’s initial response, the AI recommendation, and the human’s final response after observing the AI advice. Responses are encoded on a signed confidence scale in [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] : the sign indicates whether the selected label agrees with the ground-truth label, and the magnitude indicates the confidence strength. Thus, positive values correspond to correct predictions, negative values correspond to incorrect predictions, and values near zero indicate low confidence. The dataset also includes participant-level demographic and survey variables, which are used to fit the behavioral simulator following the original setup, but are not used in our omnimechanism design.
We focus on the Art and Sarcasm tasks. These two tasks provide sufficient interactions to support our full experimental pipeline: fitting a behavioral simulator, optimizing policies, and evaluating the learned policies on held-out examples. The remaining tasks contain data only for training the behavioral simulator, but not additional held-out interactions for reliable policy optimization and evaluation. We therefore exclude them from the main experiments.
For each task, we reserve a separate participant pool for fitting the behavioral simulator, as in [35 ] . The remaining participants are split approximately equally between policy training and held-out evaluation using seed 42. All interactions from a participant remain in the same partition, and the simulator-fitting, policy-training, and evaluation pools are mutually disjoint.
Human Behavioral Simulator
We use the two-stage behavioral simulator from [35 ] as a proxy for human behavior. Given the initial human response r 1 r_1 r 1 and the AI advice, the simulator first predicts the probability that the human changes their response after seeing the advice, denoted by p a c t p_{\mathrm{act}} p act . Conditional on activation, it predicts the signed response change
Δ = sign ( r 1 ) ( r 2 − r 1 ) . \Delta= \operatorname{sign}(r_1)(r_2-r_1). Δ = sign ( r 1 ) ( r 2 − r 1 ) .
The corresponding activated response is
r ^ 2 a c t = clip ( r 1 + sign ( r 1 ) Δ ^ , − 1 , 1 ) . \widehat{r}_2^{\mathrm{act}}=\operatorname{clip}\left(r_1+\operatorname{sign}(r_1)\widehat{\Delta},-1,1\right). r 2 act = clip ( r 1 + sign ( r 1 ) Δ , − 1 , 1 ) .
If the human is not activated, the final response remains r 1 r_1 r 1 .
For any transformed AI advice, the simulator therefore induces a mixture over two possible final responses: r 1 r_1 r 1 with probability 1 − p a c t 1-p_{\mathrm{act}} 1 − p act , and r ^ 2 a c t \widehat{r}_2^{\mathrm{act}} r 2 act with probability p a c t p_{\mathrm{act}} p act . For a collaboration policy with human weight w w w and AI weight 1 − w 1-w 1 − w , the final output is the weighted combination of the human response and the original AI signal. Its expected accuracy is
Acc. = E [ ( 1 − p a c t ) 1 { w r 1 + ( 1 − w ) r A I > 0 } + p a c t 1 { w r ^ 2 a c t + ( 1 − w ) r A I > 0 } ] . \operatorname{Acc.}=\mathbb{E}\left[(1-p_{\mathrm{act}})\mathbf{1}\{wr_1+(1-w)r_{\mathrm{AI}}>0\}+p_{\mathrm{act}}\mathbf{1}\{w\widehat{r}_2^{\mathrm{act}}+(1-w)r_{\mathrm{AI}}>0\}\right]. Acc. = E [ ( 1 − p act ) 1 { w r 1 + ( 1 − w ) r AI > 0 } + p act 1 { w r 2 act + ( 1 − w ) r AI > 0 } ] .
Here, w = 1 w=1 w = 1 corresponds to using the human response alone. For the AI-only route, accuracy is E [ 1 { r A I > 0 } ] \mathbb{E}[\mathbf{1}\{r_{\mathrm{AI}}>0\}] E [ 1 { r AI > 0 }] .
Omnimechanism Design
Both the boosting-based and neural implementations instantiate the same omnimechanism objective: learning a base predictive distribution over outcomes for each state-action pair, which can then be post-processed into policies for different downstream costs. They differ only in how the base predictive distribution is represented and optimized.
State representation. The observable state z z z contains only pre-interaction information available before deciding how to use the AI signal. We use
z = ( ∣ r 1 ∣ , ∣ r A I ∣ , 1 { sign ( r 1 ) = sign ( r A I ) } ) . z=\left(|r_1|,|r_{\mathrm{AI}}|,\mathbf{1}\{\operatorname{sign}(r_1)=\operatorname{sign}(r_{\mathrm{AI}})\}\right). z = ( ∣ r 1 ∣ , ∣ r AI ∣ , 1 { sign ( r 1 ) = sign ( r AI )} ) .
where r 1 r_1 r 1 is the human’s initial signed confidence and r A I r_{\mathrm{AI}} r AI is the raw signed AI signal. The first two coordinates measure the human and AI confidence strengths, while the third records whether the human and AI initially agree on the label. This representation removes the direct correctness information carried by the signs of the signed responses, while still retaining whether the human and AI initially agree.
For the boosting-based implementation, we discretize the two confidence coordinates with step size s z s_z s z and keep the agreement coordinate binary. In the main experiments, we use s z = 0.1 s_z=0.1 s z = 0.1 . For the neural implementation, we use the same three state features in continuous form, together with action features.
Action space. An action a a a specifies how the AI advice is used. It consists of three components: a deferral decision, a signal transformation, and a collaboration function. The deferral decision has two possible outcomes: either the system directly outputs the AI prediction, or it routes the instance to the human. We refer to these as the AI-only route and the Human+AI route.
Under the AI-only route, the final output is the raw AI signal r A I r_{\mathrm{AI}} r AI . Under the Human+AI route, the AI signal is first transformed and then passed to the behavioral simulator to produce a simulated human response.
We follow the signal transformation in [35 ] , parameterized by ( α , β ) (\alpha,\beta) ( α , β ) . We convert the signed AI signal r A I r_{\mathrm{AI}} r AI into a probability
p A I = r A I + 1 2 , p_{\mathrm{AI}} = \frac{r_{\mathrm{AI}}+1}{2}, p AI = 2 r AI + 1 ,
which we clip to [ 10 − 6 , 1 − 10 − 6 ] [10^{-6},1-10^{-6}] [ 1 0 − 6 , 1 − 1 0 − 6 ] before converting to a logit:
A = log p A I 1 − p A I , A = \log\frac{p_{\mathrm{AI}}}{1-p_{\mathrm{AI}}}, A = log 1 − p AI p AI ,
Using the logistic sigmoid σ ( u ) = ( 1 + e − u ) − 1 \sigma(u)=(1+e^{-u})^{-1} σ ( u ) = ( 1 + e − u ) − 1 , the transformed AI confidence is
ϕ α , β ( A ) = σ ( sign ( A ) ( α ∣ A ∣ + β ) ) , \phi_{\alpha,\beta}(A)=\sigma\left(\operatorname{sign}(A)(\alpha|A|+\beta)\right), ϕ α , β ( A ) = σ ( sign ( A ) ( α ∣ A ∣ + β ) ) ,
and the corresponding transformed signed signal is
r A I ′ = 2 ϕ α , β ( A ) − 1. r'_{\mathrm{AI}}=2\phi_{\alpha,\beta}(A)-1. r AI ′ = 2 ϕ α , β ( A ) − 1.
We use the parameter grids α ∈ { 0.5 , 1.0 , 1.5 , 2.0 } \alpha\in\{0.5,1.0,1.5,2.0\} α ∈ { 0.5 , 1.0 , 1.5 , 2.0 } and β ∈ { 0 , 0.5 , 1.0 , 1.5 , 2.0 } \beta\in\{0,0.5,1.0,1.5,2.0\} β ∈ { 0 , 0.5 , 1.0 , 1.5 , 2.0 } .
The collaboration function is used only under the Human+AI route. Let r ^ h u m a n \hat{r}_{\mathrm{human}} r ^ human denote the simulator-induced human response after observing the transformed AI signal. The final routed output is
r f i n a l = w r ^ h u m a n + ( 1 − w ) r A I , r_{\mathrm{final}}=w\hat{r}_{\mathrm{human}}+(1-w)r_{\mathrm{AI}}, r final = w r ^ human + ( 1 − w ) r AI ,
where w ∈ ( 0 , 1 ] w\in(0,1] w ∈ ( 0 , 1 ] is a collaboration weight. We use a grid over w w w ; with step size 0.5, the candidate values are w ∈ { 0.5 , 1.0 } w\in\{0.5,1.0\} w ∈ { 0.5 , 1.0 } . The case w = 1 w=1 w = 1 corresponds to using only the simulated human final response.
In our policy class H \mathcal{H} H , deferral functions are implemented by a threshold rule on AI confidence: it routes the instance to AI-only when ∣ r A I ∣ |r_{\mathrm{AI}}| ∣ r AI ∣ exceeds a threshold, and otherwise routes it to Human+AI. We also include always-human and always-AI as special cases. Although there may be many different deferral functions, each ultimately outputs only one of the two route outcomes for any fixed state: AI-only or Human+AI. Thus, deferral functions define candidate policies over states, while the action outcome itself is determined by the binary route together with the chosen signal transformation and collaboration function.
The following remark explains how this experimental action-space construction fits the finite-action framework underlying our sample-complexity guarantees.
remaining assumptions of the corresponding auditing theorem, including known logging propensities and coverage over A ˉ \bar{A} A ˉ , the finite-action sample-complexity guarantees apply with ∣ A ˉ ∣ |\bar{A}| ∣ A ˉ ∣ and the complexity of the induced policy class. Thus, the displayed signal may depend on the full input even though policy selection and outcome prediction use the reduced features z z z .
Outcome representation. For each state-action pair ( z , a ) (z,a) ( z , a ) , the base predictive distribution predicts a four-class advice-relative outcome
o ∈ { 0 , 1 , 2 , 3 } . o \in\{0,1,2,3\}. o ∈ { 0 , 1 , 2 , 3 } .
Each outcome records two binary quantities: whether the final prediction agrees with the AI signal, and whether the AI signal is correct. We define the classes as
0 : final_agree_AI = 0 , AI_correct = 0 , 1 : final_agree_AI = 0 , AI_correct = 1 , 2 : final_agree_AI = 1 , AI_correct = 0 , 3 : final_agree_AI = 1 , AI_correct = 1. \begin{aligned}
0 &: \ \text{final\_agree\_AI}=0,\quad\text{AI\_correct}=0,\\
1 &: \ \text{final\_agree\_AI}=0,\quad\text{AI\_correct}=1,\\
2 &: \ \text{final\_agree\_AI}=1,\quad\text{AI\_correct}=0,\\
3 &: \ \text{final\_agree\_AI}=1,\quad\text{AI\_correct}=1.
\end{aligned} 0 1 2 3 : final_agree_AI = 0 , AI_correct = 0 , : final_agree_AI = 0 , AI_correct = 1 , : final_agree_AI = 1 , AI_correct = 0 , : final_agree_AI = 1 , AI_correct = 1.
This representation is sufficient to determine whether the final output is correct. If the AI signal is incorrect, then disagreeing with it is correct, so class 0 is correct and class 2 is incorrect. If the AI signal is correct, then agreeing with it is correct, so class 3 is correct and class 1 is incorrect. Therefore, classes 0 and 3 correspond to correct final outputs, while classes 1 and 2 correspond to errors.
For losses based on prediction error, we use
l e r r ( o ) = 1 { o ∈ { 1 , 2 } } . l_{\mathrm{err}}(o) = \mathbf{1}\{o \in\{1,2\}\}. l err ( o ) = 1 { o ∈ { 1 , 2 }} .
This error loss is combined with human-involvement cost when computing COI violations during auditing, and the same outcome representation is used for post-processing under different downstream costs.
Training targets. For Human+AI actions, we replace hard correctness with σ ( T r f i n a l ) \sigma(T r_{\mathrm{final}}) σ ( T r final ) and average over the simulator’s activation mixture. We map the resulting soft correctness to the four outcome classes above, obtaining q ( 0 ) q^{(0)} q ( 0 ) . For AI-only actions, q ( 0 ) q^{(0)} q ( 0 ) is one-hot. Both implementations use the smoothed target
q s i m = ( 1 − 4 λ ) q ( 0 ) + λ 1 4 . q_{\mathrm{sim}} = (1-4\lambda)q^{(0)} + \lambda\mathbf{1}_4. q sim = ( 1 − 4 λ ) q ( 0 ) + λ 1 4 .
with T = 5 T=5 T = 5 and λ = 10 − 4 \lambda=10^{-4} λ = 1 0 − 4 . This smoothing is used for learning; reported accuracy is computed using hard sign-based correctness.
Boosting-Based Omnimechanism
We implement the boosting-based omnimechanism by discretizing the state space and maintaining a tabular base predictive distribution over outcomes for each discrete state-action pair. Specifically, after discretizing the confidence coordinates of z z z , we maintain
p ~ ( t ) ( o ∣ z , a ) , \tilde{p}^{(t)}(o \mid z,a), p ~ ( t ) ( o ∣ z , a ) ,
where o ∈ { 0 , 1 , 2 , 3 } o \in\{0,1,2,3\} o ∈ { 0 , 1 , 2 , 3 } is the outcome described above and a a a is an action.
For a downstream specification ω = ( l e r r , c ) \omega= (l_{\mathrm{err}}, c) ω = ( l err , c ) , we define the action-dependent loss
ℓ a ω ( z , o ) = l e r r ( o ) + c ⋅ 1 { a routes to human } , \ell_a^\omega(z,o) = l_{\mathrm{err}}(o) + c \cdot\mathbf{1}\{a\text{ routes to human}\}, ℓ a ω ( z , o ) = l err ( o ) + c ⋅ 1 { a routes to human } ,
where
l e r r ( o ) = 1 { o ∈ { 1 , 2 } } . l_{\mathrm{err}}(o) = \mathbf{1}\{o \in\{1,2\}\}. l err ( o ) = 1 { o ∈ { 1 , 2 }} .
Here l e r r l_{\mathrm{err}} l err denotes the prediction-error component, while ℓ a ω \ell_a^\omega ℓ a ω denotes the full downstream loss including the human-involvement cost.
At each boosting round, we audit the current base predictive distribution against a collection of candidate policies. This collection includes all fixed policies in the candidate class H \mathcal{H} H , as well as the current optimal post-processing policy induced by the predictor for each downstream cost parameter c c c :
π ω ( z ) ∈ arg min a E p ~ ( t ) ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] . \pi^\omega(z) \in\arg\min_a \mathbb{E}_{\tilde{p}^{(t)}(\cdot\mid z,a)}[\ell_a^\omega(z,o)]. π ω ( z ) ∈ arg a min E p ~ ( t ) ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] .
For a candidate mechanism h h h and downstream specification ω \omega ω , we compute the empirical COI violation
ε ( h , ω ) = E s i m [ ℓ π h ( z ) ω ( z , o ) ] − E p ~ ( t ) [ ℓ π h ( z ) ω ( z , o ) ] , \varepsilon(h,\omega) = \mathbb{E}_{\mathrm{sim}}\left[\ell_{\pi_h(z)}^\omega(z,o)\right] - \mathbb{E}_{\tilde{p}^{(t)}}\left[\ell_{\pi_h(z)}^\omega(z,o)\right], ε ( h , ω ) = E sim [ ℓ π h ( z ) ω ( z , o ) ] − E p ~ ( t ) [ ℓ π h ( z ) ω ( z , o ) ] ,
where the simulator expectation is estimated using the behavioral simulator and the predictor expectation is computed from p ~ ( t ) ( ⋅ ∣ z , π h ( z ) ) \tilde{p}^{(t)}(\cdot\mid z,\pi_h(z)) p ~ ( t ) ( ⋅ ∣ z , π h ( z )) . We then select the candidate with the largest absolute violation:
( h t , ω t ) = arg max h , ω ∣ ε ( h , ω ) ∣ . (h_t,\omega_t) = \arg\max_{h,\omega} |\varepsilon(h,\omega)|. ( h t , ω t ) = arg h , ω max ∣ ε ( h , ω ) ∣.
If the maximum violation is below a tolerance threshold ε \varepsilon ε , the procedure terminates.
Otherwise, we update the base predictive distribution on the action selected by the violating policy. Let
σ t = sign ( ε ( h t , ω t ) ) . \sigma_t = \operatorname{sign}(\varepsilon(h_t,\omega_t)). σ t = sign ( ε ( h t , ω t )) .
For each discrete state z z z , only the entry corresponding to the selected action a = π h t ( z ) a = \pi_{h_t}(z) a = π h t ( z ) is updated:
p ~ ( t + 1 ) ( o ∣ z , a ) ∝ p ~ ( t ) ( o ∣ z , a ) exp ( η σ t l e r r ( o ) ) , a = π h t ( z ) , \tilde{p}^{(t+1)}(o \mid z,a) \propto\tilde{p}^{(t)}(o \mid z,a)\exp\left(\eta\sigma_t l_{\mathrm{err}}(o)\right), \qquad a = \pi_{h_t}(z), p ~ ( t + 1 ) ( o ∣ z , a ) ∝ p ~ ( t ) ( o ∣ z , a ) exp ( η σ t l err ( o ) ) , a = π h t ( z ) ,
followed by normalization over o o o . Entries for actions not selected by h t h_t h t remain unchanged. The human-involvement cost affects which policies are selected during auditing and post-processing, but it is outcome-independent and therefore is not included in the multiplicative update.
After training, the learned base predictive distribution is reused for different downstream costs without retraining. In the experiments, we use deterministic post-processing, selecting the action with the smallest predicted expected loss. Thus, for any ω = ( l e r r , c ) \omega= (l_{\mathrm{err}}, c) ω = ( l err , c ) , the induced policy is obtained by post-processing:
π ω ( z ) ∈ arg min a E p ~ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] . \pi^\omega(z) \in\arg\min_a \mathbb{E}_{\tilde{p}(\cdot\mid z,a)}[\ell_a^\omega(z,o)]. π ω ( z ) ∈ arg a min E p ~ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o )] .
In our experiments, we use discretization step 0.1, multiplicative-weights learning rate η = 0.15 \eta= 0.15 η = 0.15 , and COI tolerance ε = 0.05 \varepsilon= 0.05 ε = 0.05 . We also record the KL divergence from the smoothed simulator target to the learned predictive distribution over boosting rounds as a diagnostic of distribution-level convergence, as shown in Figure 2 .
Neural Omnimechanism
We also implement a neural version of the omnimechanism to avoid discretizing the state space. It uses the same state, primitive action, and outcome definitions as above, but represents the base predictive distribution with a neural network:
p ~ θ ( o ∣ z , a ) . \tilde{p}_{\theta}(o \mid z,a). p ~ θ ( o ∣ z , a ) .
The network takes the continuous state z z z and primitive action features ( α , β , r o u t e , w ) (\alpha,\beta,\mathrm{route},w) ( α , β , route , w ) as input, and outputs logits over the four outcome classes.
For each training example and primitive action, we query the behavioral simulator to construct the target distribution q s i m q_{\mathrm{sim}} q sim defined above.
The neural predictor is trained on the targets q s i m q_{\mathrm{sim}} q sim by minimizing soft-label cross entropy:
L N N = − E ( z , a ) ∑ o q s i m ( o ∣ z , a ) log p ~ θ ( o ∣ z , a ) . \mathcal{L}_{\mathrm{NN}}=-\mathbb{E}_{(z,a)}\sum_{o}q_{\mathrm{sim}}(o\mid z,a)\log\tilde{p}_{\theta}(o\mid z,a). L NN = − E ( z , a ) o ∑ q sim ( o ∣ z , a ) log p ~ θ ( o ∣ z , a ) .
We use a three-layer perceptron with two 64-unit hidden layers, ReLU activations, and dropout 0.1, trained with Adam at learning rate 10 − 3 10^{-3} 1 0 − 3 and batch size 512. We reserve 30% of the policy-training participants for validation and select the checkpoint with the lowest validation cross entropy. We also audit empirical COI violations on the validation set, using simulator losses based on accuracy.
At evaluation time, we use the same deterministic post-processing rule as in Appendix § 14 .§ 4 : for each downstream specification ω = ( l e r r , c ) \omega=(l_{\mathrm{err}},c) ω = ( l err , c ) , the neural omnimechanism induces the policy
π ω ( z ) ∈ arg min a E p ~ θ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] , \pi^{\omega}(z)\in\underset{a}{\arg\min}\ \mathbb{E}_{\tilde{p}_{\theta}(\cdot\mid z,a)}\left[\ell_{a}^{\omega}(z,o)\right], π ω ( z ) ∈ a arg min E p ~ θ ( ⋅ ∣ z , a ) [ ℓ a ω ( z , o ) ] ,
using the same loss definition and evaluation protocol as in Appendix § 14 .§ 4 . The selected action is then evaluated with the behavioral simulator rather than with the neural prediction itself.
Baselines and Metrics
We compare the post-processing policies induced by our omnimechanism with three baselines: No Action, Manipulation, and Global Best .
No Action. This corresponds to the basic Human+AI setting in which every example is deferred to the human with original AI advice and final prediction is human’s final response.
Manipulation. This baseline is the AI-advice manipulation method from [35 ] . It always routes the instance to the human and optimizes only the global AI signaling function ϕ α , β \phi_{\alpha,\beta} ϕ α , β . The optimization is performed over the continuous ( α , β ) (\alpha,\beta) ( α , β ) space using the same parameter range as our action space.
Global Best. This baseline selects the best single policy from the candidate policy class H \mathcal{H} H for each downstream specification ω \omega ω and holds that policy fixed across inputs. The policy class H \mathcal{H} H contains global combinations of the three action components introduced above: a deferral function, a signaling function, and a collaboration function. The deferral functions include always-human, always-AI, and threshold-based rules that defer to the human when the AI confidence is below a threshold; the thresholds are chosen on a grid with step size 0.2. The signaling functions are drawn from the grid of transformations ϕ α , β \phi_{\alpha,\beta} ϕ α , β , and the collaboration functions are linear combinations parameterized by the grid of weights w w w , as described in Appendix § 14 .§ 3 . For each cost parameter c c c , Global Best is chosen on the policy-optimization data using simulator utility and then evaluated on the held-out evaluation data. Unlike Global Best, which uses a fixed routing rule, signaling function, and collaboration weight, our post-processing can select the complete action separately for each input.
Metrics. All methods are evaluated on the held-out evaluation set using the behavioral simulator. Accuracy is computed using the simulator-induced activation mixture defined in Appendix § 14 .§ 2 . We report human involvement rate as
H u m . = E [ 1 { a routes to human } ] , \mathrm{Hum.} = \mathbb{E}[1\{a\ \text{routes to human}\}], Hum. = E [ 1 { a routes to human }] ,
and utility for cost parameter c c c as
U t i l . c = A c c . − c ⋅ H u m . \mathrm{Util.}_c = \mathrm{Acc.} - c \cdot\mathrm{Hum.} Util. c = Acc. − c ⋅ Hum.
We also assess stability across random seeds. Across the evaluated runs, the empirical standard deviation of the reported metrics is below one percentage point.
Supplementary Experiment: Varying AI Advice Accuracy
We conduct an additional experiment to further validate the effectiveness of our methods under different AI advice accuracy levels. Starting from the original AI advice distribution, whose average accuracy is around 80%, we apply a global downward shift to the signed AI signal and clip the result to [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] . This produces two additional settings in which the average AI advice accuracy is approximately 70% and 60%. We focus on downward shifts to examine settings in which human consultation may become more valuable. The realized AI-only accuracies are 69.8% and 73.7% for Art and Sarcasm in the first setting, and 59.6% and 63.1% in the second.
For each shifted setting, we rerun the full experimental pipeline, including training the boosting-based and neural omnimechanisms and reselecting the corresponding baselines. This allows us to compare our methods with the strongest available baselines under each AI-advice accuracy level, and to examine whether the learned post-processing policies still achieve competitive utility when the reliability of the AI signal changes.
Table 5 and Table 6 report the results. When AI accuracy is reduced to roughly 70%, the learned policies remain competitive with Global Best across both tasks. On Art, the differences between methods are small, and for nonzero costs both Global Best and our methods quickly move toward AI-only behavior. On Sarcasm, where the value of human involvement is more state-dependent, our methods obtain the best or near-best utilities across cost values.
When AI accuracy is reduced to roughly 60%, the trade-off between accuracy and human involvement becomes more pronounced. When human involvement is free, the learned policies make substantial use of human input and improve accuracy over AI-only. As the human cost increases, the policies reduce human involvement and become more selective. On Art, our methods closely match the strongest utilities across cost values, indicating that the learned policies can recover the appropriate low-deferral behavior when human involvement is not cost-effective. On Sarcasm, both implementations improve upon Global Best at c = 0.1 c = 0.1 c = 0.1 ; at c = 0.2 c = 0.2 c = 0.2 ,
NN retains selective human involvement and higher utility, while Boosting uses AI-only and matches Global Best.
Overall, this supplementary experiment shows that our conclusions are not specific to the original AI accuracy level. After retraining under each shifted AI-advice distribution, the omnimechanism-based policies remain competitive with the strongest global baselines and reflect the expected trade-off between accuracy and human cost.
Additional Details for Experiment 2: Radiologist–AI Collaboration
Data and Action Construction
Collab-CXR contains probabilistic assessments from 227 professional radiologists on 324 retrospective chest X-rays, collected under four information environments and three experimental designs [29 ] . Our analysis uses Design 1, in which 159 radiologists each read 60 unique cases, with 15 cases assigned to each randomized information environment. A radiologist encounters each patient at most once, yielding one-shot assessments that match our action-selection setting. Designs 2 and 3 study repeated or sequential assessments and are not included in this experiment.
We study Airspace Opacity and Cardiomediastinal Abnormality, the two pre-registered top-level pathologies for which AI predictions are available in the original analysis. The original experiment varies both AI assistance and the availability of clinical history to study how these information sources affect radiologists’ assessments. We restrict attention to conditions without clinical history and consider three actions: the direct algorithmic probability (AI), a radiologist’s probability based on the X-ray alone (H), and a radiologist’s probability after observing both the X-ray and the AI assessment (AI+H). We charge a human-involvement cost c c c whenever a radiologist is consulted: AI incurs no such cost, while H and AI+H each incur cost c c c . Clinical-history conditions could be incorporated as additional actions but may induce additional cost. We omit them to keep the action space focused on how AI and radiologists are combined, with a single cost for radiologist involvement.
After restricting to the two conditions without clinical history, the 159 radiologists contribute 159 × 30 = 4,770 159 \times30 = 4{,}770 159 × 30 = 4 , 770 X-ray reads in total: 159 × 15 = 2,385 159 \times15 = 2{,}385 159 × 15 = 2 , 385 under H and the same number under AI+H. Each read contains an assessment for both focal pathologies, yielding 2,385 × 2 = 4,770 2{,}385 \times2 = 4{,}770 2 , 385 × 2 = 4 , 770 pathology-level probability reports per human action and 9,540 9{,}540 9 , 540 across the two actions. Both human actions cover all 324 × 2 = 648 324 \times2 = 648 324 × 2 = 648 patient–pathology inputs. The direct AI probability is unique within each patient–pathology pair.
Omnimechanism Design
Let y ∈ { 0 , 1 } y \in\{0,1\} y ∈ { 0 , 1 } denote the binary diagnostic-standard label supplied with each patient–pathology pair. The accompanying study evaluates diagnostic probabilities using the average deviation from this standard [1 ] . For a reported probability ρ ∈ [ 0 , 1 ] \rho\in[0,1] ρ ∈ [ 0 , 1 ] , we encode the following fractional four-state target:
q 00 = ( 1 − y ) ( 1 − ρ ) , q 01 = y ( 1 − ρ ) , q 10 = ( 1 − y ) ρ , q 11 = y ρ . (6) \begin{aligned}
q_{00} &= (1-y)(1-\rho), & q_{01} &= y(1-\rho), \\
q_{10} &= (1-y)\rho, & q_{11} &= y\rho. \tag*{(6)}
\end{aligned} q 00 q 10 = ( 1 − y ) ( 1 − ρ ) , = ( 1 − y ) ρ , q 01 q 11 = y ( 1 − ρ ) , = y ρ . (6)
We interpret ρ \rho ρ as a distribution over the binary diagnostic label, assigning probability 1 − ρ 1-\rho 1 − ρ to label 0 and probability ρ \rho ρ to label 1. The first index of q q q denotes this predicted label, while the second denotes the diagnostic standard. The diagonal mass gives the diagnostic score used in the main text:
S ( ρ , y ) = q 00 + q 11 = 1 − ∣ ρ − y ∣ . (7) S(\rho,y) = q_{00} + q_{11} = 1 - |\rho-y|. \tag*{(7)} S ( ρ , y ) = q 00 + q 11 = 1 − ∣ ρ − y ∣. (7)
Maximizing S S S is therefore equivalent to minimizing the deviation measure. The fractional representation retains the reported probability rather than converting it to a binary prediction.
For H and AI+H, we construct Eq. (6) for every reader and average the four masses within each patient–pathology–action cell. For AI, we construct the target once from the direct AI probability. This gives 324 × 2 × 3 = 1,944 324 \times2 \times3 = 1{,}944 324 × 2 × 3 = 1 , 944 cells, which are weighted equally during training. Repeated radiologist assessments estimate the empirical outcome distribution within a cell; reader identity is not included as a model input.
Neural Omnimechanism
Patients are split 80/20 using seed 42, stratified over the four joint binary label combinations for the two pathologies, so that their relative frequencies are approximately preserved across the training and test sets. The resulting split contains 259 training patients and 65 test patients. All actions and pathologies associated with an X-ray remain in the same split.
We extract one 1,152-dimensional representation per X-ray using the MedSigLIP-448 vision encoder [33 ] . A 32-component PCA transform is fit on the training embeddings and then used to project both training and test embeddings, retaining 83.27% of the training embedding variance. During cross-validation, the same procedure is performed within each fold: PCA is fit on the fold’s training patients and applied to its validation patients. After L2 normalization, the model input concatenates the image vector, scalar AI probability, two-dimensional pathology one-hot encoding, and three-dimensional action one-hot encoding, for 38 input features.
The neural omnimechanism is a two-hidden-layer MLP with width 32, ReLU activations, and dropout 0.1 after the first hidden layer. A four-way softmax produces q ^ ( z , a ) \hat{q}(z,a) q ^ ( z , a ) . We minimize fractional-target cross-entropy using AdamW, learning rate 10 − 3 10^{-3} 1 0 − 3 , weight decay 10 − 4 10^{-4} 1 0 − 4 , and batch size 128. We select the number of training epochs by five-fold patient-grouped, joint-label-stratified cross-validation, choosing the earliest epoch whose mean validation loss lies within one standard error of the minimum. The final PCA transform and neural model are fit using all training patients, without access to the test set.
Evaluation and Additional Results
For each input, the same trained model is evaluated with all three actions. At cost c c c , it chooses
π c ( z ) ∈ arg max a ∈ { A I , H , A I + H } { q ^ 00 ( z , a ) + q ^ 11 ( z , a ) − c 1 { a ≠ A I } } . (8) \pi_c(z) \in\arg\max_{a \in\{\mathrm{AI},\mathrm{H},\mathrm{AI+H}\}} \left\{\hat{q}_{00}(z,a) + \hat{q}_{11}(z,a) - c\mathbf{1}\{a \ne\mathrm{AI}\}\right\}. \tag*{(8)} π c ( z ) ∈ arg a ∈ { AI , H , AI + H } max { q ^ 00 ( z , a ) + q ^ 11 ( z , a ) − c 1 { a = AI } } . (8)
The comparator is Global Best: separately for each c c c , we use training cells to select one action maximizing mean S ( ρ , y ) − c ⋅ H u m . S(\rho,y) - c \cdot\mathrm{Hum.} S ( ρ , y ) − c ⋅ Hum. and apply that single action to every test input. Both the input-dependent policy and Global Best are fixed before evaluation on the test patients.
Table 7 gives the action composition underlying the performance results in Table 3 . With no human cost, the policy uses both unaided and AI-assisted radiologist assessments. As c c c increases, human involvement declines first for Cardiomediastinal Abnormality and then for Airspace Opacity. At c ≥ 0.10 c \ge0.10 c ≥ 0.10 , all test inputs are assigned to AI.
Table 7. Actions selected by the post-processed policy on the Collab-CXR test set. Each pathology contains 65 test X-rays.
Additional Details for Experiment 3: AI-Assisted House-Price Prediction
Data and Action Construction
We use the AMES house-price judgment data of Guo et al. [ [22 ] . The released trial table contains 437 participants and 5,244 paired judgments on 240 distinct houses. Each participant evaluates 12 houses, first providing all 12 unaided estimates and then revising these estimates after receiving AI assistance. Participants are assigned to one of six information conditions, crossing two AI models with three explanation types: no explanation, SHAP, and ILIV+SHAP. The assigned condition remains fixed across a participant’s assisted trials.
For each participant–house pair, let H 0 H_0 H 0 and H 1 H_1 H 1 denote the initial and assisted human estimates, M 1 M_1 M 1 and M 2 M_2 M 2 the two AI predictions, and y > 0 y > 0 y > 0 the sale price. Human estimates in the released table are multiplied by 1,000 to place all predictions and labels on the same dollar scale. The context is z = ( x , H 0 , M 1 , M 2 ) z = (x,H_0,M_1,M_2) z = ( x , H 0 , M 1 , M 2 ) , where x x x contains the six house features: year built, above-ground living area, garage capacity, number of fireplaces, year remodeled, and overall quality. Sale price, assisted human responses, participant identity, and reward variables are excluded from the policy inputs.
We study a sequential decision made after observing H 0 H_0 H 0 and we consider the action space A \mathcal{A} A consisting of the two parts below. The first action set is defined by A 0 = { H 0 , A I 1 , A I 2 } \mathcal{A}_0 = \{H_0,\mathrm{AI}_1,\mathrm{AI}_2\} A 0 = { H 0 , AI 1 , AI 2 } where the actions directly output H 0 H_0 H 0 , M 1 M_1 M 1 , and M 2 M_2 M 2 , respectively. The second action set is defined by A 1 = { R e v j , e : j ∈ { 1 , 2 } , e ∈ { n o n e , S H A P , I L I V + S H A P } } \mathcal{A}_1 = \{\mathrm{Rev}_{j,e}: j \in\{1,2\}, e \in\{\mathrm{none},\mathrm{SHAP},\mathrm{ILIV+SHAP}\}\} A 1 = { Rev j , e : j ∈ { 1 , 2 } , e ∈ { none , SHAP , ILIV + SHAP }} , where the actions request a human revision after displaying model j j j with explanation e e e , and use that revised estimate as the final output. Thus A = A 0 ∪ A 1 \mathcal{A} = \mathcal{A}_0 \cup\mathcal{A}_1 A = A 0 ∪ A 1 contains nine actions. We charge a human-involvement cost c c c only for actions in A 1 \mathcal{A}_1 A 1 , while the initial estimate is already available to every policy, so its elicitation cost is treated as sunk. All six revision actions correspond to conditions present in the original experiment; we do not introduce manipulated AI predictions beyond these conditions.
The main analysis retains all 437 participants in the released table. We also report a sensitivity analysis using the original filtering rules, which exclude participants who never revise, have long runs of repeated answers, or give extreme overestimates in both rounds. These rules retain 421 participants and 5,052 paired judgments. We use the unfiltered population for the main analysis because these exclusions depend partly on assisted responses.
Omnimechanism Design
Let v v v denote an action’s final price estimate. Following Guo et al. [ [22 ] , we use absolute percentage error (APE):
d A P E ( v , y ) = ∣ v − y y ∣ . (9) d_{\mathrm{APE}}(v,y)=\left|\frac{v-y}{y}\right|. \tag*{(9)} d APE ( v , y ) = y v − y . (9)
The total decision loss is d A P E ( v , y ) + c r ( a ) d_{\mathrm{APE}}(v,y)+c\,r(a) d APE ( v , y ) + c r ( a ) , where r ( a ) = 1 { a ∈ A 1 } r(a)=1\{a\in\mathcal{A}_1\} r ( a ) = 1 { a ∈ A 1 } . We report loss directly, with lower values indicating better performance. The cost c c c is measured in APE units.
To obtain a finite outcome representation, we discretize the signed relative error R = ( v − y ) / y R=(v-y)/y R = ( v − y ) / y into 13 bins. The finite breakpoints are
( − 1.00 , − 0.75 , − 0.50 , − 0.30 , − 0.15 , − 0.05 , 0.05 , 0.15 , 0.30 , 0.50 , 0.75 , 1.00 ) . (-1.00,-0.75,-0.50,-0.30,-0.15,-0.05,0.05,0.15,0.30,0.50,0.75,1.00). ( − 1.00 , − 0.75 , − 0.50 , − 0.30 , − 0.15 , − 0.05 , 0.05 , 0.15 , 0.30 , 0.50 , 0.75 , 1.00 ) .
The outer endpoints are − ∞ -\infty − ∞ and + ∞ +\infty + ∞ , and finite bins include their lower endpoint. For an outcome R R R , let b ( R ) ∈ { 1 , … , 13 } b(R)\in\{1,\ldots,13\} b ( R ) ∈ { 1 , … , 13 } denote the resulting bin index. Writing the finite breakpoints as t 1 < ⋯ < t 12 t_1<\cdots<t_{12} t 1 < ⋯ < t 12 , we use the midpoint of each interior bin, ρ k = ( t k − 1 + t k ) / 2 \rho_k=(t_{k-1}+t_k)/2 ρ k = ( t k − 1 + t k ) /2 for k = 2 , … , 12 k=2,\ldots,12 k = 2 , … , 12 . For the lower tail, we set ρ 1 = − 1 \rho_1=-1 ρ 1 = − 1 at the limiting boundary implied by positive estimates and sale prices (R > − 1 R>-1 R > − 1 ); for the open-ended upper bin [ 1 , ∞ ) [1,\infty) [ 1 , ∞ ) , we use ρ 13 = 1.5 \rho_{13}=1.5 ρ 13 = 1.5 , the midpoint of the reference interval [ 1 , 2 ] [1,2] [ 1 , 2 ] , as a fixed heuristic representative. The neural omnimechanism predicts p ^ θ ( k ∣ z , a ) \widehat{p}_{\theta}(k\mid z,a) p θ ( k ∣ z , a ) , and the corresponding discrete loss is ℓ A P E ( k ) = ∣ ρ k ∣ \ell_{\mathrm{APE}}(k)=|\rho_k| ℓ APE ( k ) = ∣ ρ k ∣ . This loss approximates Eq. (9) for action selection; final evaluation uses the continuous errors rather than the bin representatives.
For each policy-training context and action in A 0 \mathcal{A}_0 A 0 , the observed output gives a one-hot target on b ( R ) b(R) b ( R ) . For an action in A 1 \mathcal{A}_1 A 1 , a behavioral simulator generates B = 256 B=256 B = 256 revised estimates H ~ 1 ( 1 ) ( z , a ) , … , H ~ 1 ( B ) ( z , a ) \widetilde{H}_1^{(1)}(z,a),\ldots,\widetilde{H}_1^{(B)}(z,a) H 1 ( 1 ) ( z , a ) , … , H 1 ( B ) ( z , a ) . We construct the averaged target for omnimechanism based on the behavioral simulator:
q k ( z , a ) = 1 B ∑ m = 1 B 1 { b ( H ~ 1 ( m ) ( z , a ) − y y ) = k } . (10) q_k(z,a)=\frac{1}{B}\sum_{m=1}^{B}\mathbb{1}\left\{b\left(\frac{\widetilde{H}_1^{(m)}(z,a)-y}{y}\right)=k\right\}. \tag*{(10)} q k ( z , a ) = B 1 m = 1 ∑ B 1 { b ( y H 1 ( m ) ( z , a ) − y ) = k } . (10)
Each context contributes one target distribution for each of the nine actions, weighted equally during training. This produces 9,396 context–action training targets in the main split. The sale price is used to construct these supervised targets, but is not an input to the predictor.
Neural Omnimechanism
We split participants into disjoint simulator-training, simulator-audit, policy-training, and evaluation sets, stratifying by their assigned AI and explanation condition. With seed 42, the sets contain 197, 66, 87, and 87 participants, respectively, corresponding to 2,364, 792, 1,044, and 1,044 paired judgments. All trials from a participant remain in the same split. The split evaluates generalization to held-out participants on the same house bank; houses are not required to be disjoint across splits. The behavioral simulator predicts the log-relative update log ( H 1 / H 0 ) \log(H_1/H_0) log ( H 1 / H 0 ) using a two-hidden-layer MLP of width 64 with ReLU activations and a scalar output. Its inputs are the six house features, log H 0 \log H_0 log H 0 , the logarithm of the displayed AI prediction, relative AI–human disagreement ( M j − H 0 ) / H 0 (M_j-H_0)/H_0 ( M j − H 0 ) / H 0 and its magnitude, and one-hot encodings of the model and explanation. The model is fit by squared error on simulator-training participants. To generate stochastic responses, we add residuals sampled from a pooled distribution obtained by four-fold participant-grouped, condition-stratified prediction within that training set. We transform back to dollars and clip simulated estimates to [ 1 , 10 6 ] [1,10^6] [ 1 , 1 0 6 ] . The same residual draws are shared across candidate revision actions for a context. The separate simulator-audit split is used only for held-out diagnostics.
The neural omnimechanism takes the six house features, the logarithms of H 0 H_0 H 0 , M 1 M_1 M 1 , and M 2 M_2 M 2 , and a nine-dimensional action one-hot encoding, giving 18 input features. Numerical context features are standardized using training data. A two-hidden-layer MLP of width 64 with ReLU activations and a 13-way softmax produces p ^ θ ( ⋅ ∣ z , a ) \hat{p}_{\theta}(\cdot\mid z,a) p ^ θ ( ⋅ ∣ z , a ) . We minimize soft-target cross-entropy,
L N N ( θ ) = − 1 ∣ A ∣ N t r a i n ∑ i = 1 N t r a i n ∑ a ∈ A ∑ k = 1 13 q k ( z i , a ) log p ^ θ ( k ∣ z i , a ) . (11) \mathcal{L}_{\mathrm{NN}}(\theta)=-\frac{1}{|\mathcal{A}|N_{\mathrm{train}}}\sum_{i=1}^{N_{\mathrm{train}}}\sum_{a\in\mathcal{A}}\sum_{k=1}^{13}q_k(z_i,a)\log\hat{p}_{\theta}(k\mid z_i,a). \tag*{(11)} L NN ( θ ) = − ∣ A ∣ N train 1 i = 1 ∑ N train a ∈ A ∑ k = 1 ∑ 13 q k ( z i , a ) log p ^ θ ( k ∣ z i , a ) . (11)
The simulator and omnimechanism both use AdamW with learning rate 10 − 3 10^{-3} 1 0 − 3 , weight decay 10 − 4 10^{-4} 1 0 − 4 , and batch size 256. For each model, an internal 20% participant-level, condition-stratified validation split of its training data selects the epoch with the lowest validation loss, with at most 250 epochs and early-stopping patience 25. Standardization is fit within the internal training split during selection. We then refit each model and its standardization on all of its designated training data for the selected number of epochs: 9 for the simulator and 200 for the omnimechanism in the main run. The omnimechanism is trained once and is reused for all costs.
Evaluation and Additional Results
Following the deterministic post-processing used in Experiments 1 and 2, the same trained predictor is evaluated with every action. For each cost c c c , it chooses
π c ( z ) ∈ arg min a ∈ A { ∑ k = 1 13 p ^ θ ( k ∣ z , a ) ℓ A P E ( k ) + c r ( a ) } . (12) \pi_c(z)\in\arg\min_{a\in\mathcal{A}}\left\{\sum_{k=1}^{13}\hat{p}_{\theta}(k\mid z,a)\ell_{\mathrm{APE}}(k)+c\,r(a)\right\}. \tag*{(12)} π c ( z ) ∈ arg a ∈ A min { k = 1 ∑ 13 p ^ θ ( k ∣ z , a ) ℓ APE ( k ) + c r ( a ) } . (12)
We evaluate costs c ∈ { 0 , 0.005 , 0.010 , 0.025 , 0.050 , 0.075 , 0.100 } c\in\{0,0.005,0.010,0.025,0.050,0.075,0.100\} c ∈ { 0 , 0.005 , 0.010 , 0.025 , 0.050 , 0.075 , 0.100 } . The comparator is Global Best: separately for each cost, we use policy-training contexts to select the single action with the lowest mean continuous loss plus cost, evaluating revision actions with simulator rollouts. That action is then applied to every evaluation context. Global Best selects A I 2 \mathrm{AI}_2 AI 2 at all tested costs in the main run. Both policies are fixed before evaluation.
Simulator evaluation uses the observed outputs for A 0 \mathcal{A}_0 A 0 and 1,024 fresh rollouts per context and revision action for A 1 \mathcal{A}_1 A 1 . It scores the selected outputs by Eq. (9) , independently of the omnimechanism’s predicted bin probabilities. We also evaluate the policies on held-out logged human responses. Each row reveals only one of the six assisted outcomes, so a selected revision can be scored directly only when its action matches the participant’s logged condition. Let A i A_i A i be the logged action, a i = π c ( z i ) a_i=\pi_c(z_i) a i = π c ( z i ) the selected action, and v i ( a ) v_i(a) v i ( a ) the known output for a ∈ A 0 a\in\mathcal{A}_0 a ∈ A 0 . Using the nominal equal-assignment propensity e a = 1 / 6 e_a = 1/6 e a = 1/6 for revision actions, the logged estimate is
d ^ i ( a ) = { d A P E ( v i ( a ) , y i ) , a ∈ A 0 , 1 { A i = a } e a d A P E ( H 1 , i , y i ) , a ∈ A 1 , L ^ c ( π ) = 1 N ∑ i = 1 N [ d ^ i ( a i ) + c r ( a i ) ] . (13) \begin{aligned}
\widehat{d}_i(a) &=
\begin{cases}
d_{\mathrm{APE}}(v_i(a),y_i), & a \in\mathcal{A}_0,\\
\frac{\mathbf{1}\{A_i=a\}}{e_a}d_{\mathrm{APE}}(H_{1,i},y_i), & a \in\mathcal{A}_1,
\end{cases}\\
\widehat{L}_c(\pi) &= \frac{1}{N}\sum_{i=1}^{N}\left[\widehat{d}_i(a_i)+c\,r(a_i)\right]. \tag*{(13)}
\end{aligned} d i ( a ) L c ( π ) = { d APE ( v i ( a ) , y i ) , e a 1 { A i = a } d APE ( H 1 , i , y i ) , a ∈ A 0 , a ∈ A 1 , = N 1 i = 1 ∑ N [ d i ( a i ) + c r ( a i ) ] . (13)
The cost is charged whenever revision is selected, including unmatched rows. We use 2,000 paired participant-bootstrap replicates for uncertainty intervals, keeping each participant’s 12 judgments together. These intervals condition on the fitted models and the observed house bank. Since treatment is assigned once per participant, interpreting the logged estimates for instance-dependent intervention choices requires no cross-trial carryover within the assisted round.
Table 8 reports simulator and logged performance. At zero cost, the NN policy achieves APE 0.1053 under simulator evaluation and 0.1061 under logged evaluation, compared with 0.1335 for Global Best. The paired logged APE difference, NN minus Global Best, is − 0.0274 -0.0274 − 0.0274 with a 95% interval of [ − 0.0346 , − 0.0189 ] [-0.0346,-0.0189] [ − 0.0346 , − 0.0189 ] .
Table 8. AMES evaluation loss (APE plus marginal revision cost) on 87 held-out participants and 1,044 paired judgments. Sim. uses simulator rollouts; Logged uses Eq. (13) . Global Best (GB) selects AI2 _2 2 , so its simulated and logged losses coincide. Lower values are better.
Table 9 gives the action composition underlying these results. At zero cost, the NN policy requests revision on 29 contexts (2.78%). Additional human involvement decreases with cost and disappears at c = 0.10 c = 0.10 c = 0.10 . The policy can still use the already-observed initial human estimate at this cost. Restricting the same predictor to A 0 A_0 A 0 gives zero-cost logged APE 0.1056; the full policy’s difference is +0.0005, with a paired 95% interval of [− 0.0047 , 0.0080 -0.0047,0.0080 − 0.0047 , 0.0080 ]. Thus the primary-run gain over Global Best is largely explained by input-dependent routing among the available estimates.
Table 9. Actions selected by the AMES policy on the 1,044 evaluation contexts. Rev. aggregates the six revision actions. Matched counts selected revisions whose action equals the logged condition; it is a subset of Rev.
We repeat the pipeline with seeds 7 and 2026 to assess sensitivity to participant splitting and training. At zero cost, logged APE is 0.1388 and 0.1026, respectively, compared with Global Best values 0.1377 and 0.1329. Under the 421-participant filtering sensitivity with seed 42, logged APE is 0.1117 compared with 0.1357 for Global Best. These checks show that the primary-run improvement is not uniform across splits. The finite-bin approximation and limited factual support for selected revisions are therefore relevant when interpreting the simulator and logged results.