betaFeedback

WorksMathematics

Principal-agent hypothesis testing under adaptive experimentation

Abstract

Principal-agent hypothesis testing studies statistical contracts in which a principal rewards an agent according to data the agent generates. Existing work focuses on the single-shot setting, where the agent produces a single dataset. We extend the framework to sequential settings, where evidence accumulates over time, rewards may be deferred to the end or disbursed incrementally, and the agent may stop the contract at a strategically chosen stopping time. We call a contract null-deterring if no ''null agent'' (those whose data-generating distribution lies in the null hypothesis) can profit in expectation under any stopping rule, and our main conceptual result shows that such contracts are characterized by e-processes. We then turn to the agent's best response and identify the optimal stopping rule through the Snell envelope of the reward process. We develop simpler, more interpretable approximations that often require the agent to have only partial knowledge of its distribution.

Introduction

Consider a drug regulator (e.g., the FDA) that controls access to a market. A pharmaceutical company can pay a cost C>0C > 0 to run a clinical trial of a new treatment. Based on the outcome of the trial, the regulator decides how widely the treatment may be marketed. There are competing interests at play: the regulator wants effective treatments to reach patients while keeping ineffective ones off the market, whereas the company wants its treatment authorized as widely, and as quickly, as possible.

This scenario is an example of a principal-agent problem: the strategic interaction between a principal, who controls access to a good, and an agent who is vying for access. Such examples are not hard to find in practice: recommendation algorithms, internal experimentation platforms at tech companies, and venture funding being a few common ones. Given their ubiquity, principal-agent problems have long been studied in economics (e.g., [19, 39, 47, 18]).

Recently, motivated by the FDA example above, Bates et al. [5] introduced a framework to analyze such dynamics from a statistical perspective, which they call principal-agent hypothesis testing. The framework is based on statistical contracts: protocols which guide the interaction between the principal and agent. The agent pays a cost to enter the contract and then generates data from a distribution—a distribution that is private and unknown to the principal, but possibly known to the agent. The principal allocates resources (rewards) based on inference made from this random sample. Importantly, the initial cost and reward mechanism was agreed to a priori by the agent, before generating the data.

In the example above, the trial outcome is a random sample drawn from the agent’s distribution. The principal does not know the treatment’s effectiveness beforehand. The reward, perhaps a license to profit or the scope of market authorization, is a function of the observed outcomes. In general, the reward may be profit, market share, a license, a binary accept/reject decision, etc.

Agents are assumed to be partitioned into null agents (e.g., companies with ineffective treatments) and non-null agents (those with effective treatments). Formally, there is a parameter space Θ\Theta partitioned into Θ0\Theta_0 (nulls) and Θ1\Theta_1 (non-nulls) and each agent has a private type/parameter θ∈Θ\theta\in\Theta. Their data is drawn from X∼PθX \sim P_\theta. With this setup in mind, the principal-agent hypothesis testing framework of Bates et al. [5] is based on the following statistical contract:

Single-shot statistical contract [5]

  1. The principal publicly announces a cost C>0C > 0 and a menu R\mathcal{R} of reward functions.

  2. An agent with hidden parameter θ\theta may pay a cost C>0C > 0 to enter the contract. Upon entry, the agent selects a reward function R∈RR \in\mathcal{R}.

  3. The agent then generates data X∼PθX \sim P_\theta and receives reward R(X)∈[0,∞)R(X) \in[0,\infty).

The central design problem in statistical contract theory is to control the expected rewards allocated to null agents while allowing non-null agents to earn rewards commensurate with the evidence they submit [25]. Bates et al. [5] show that null-deterring contracts1—those ensuring that null agents cannot earn a positive expected profit—must consist of rescaled e-values, a property that follows almost directly from the definition (and motivation) of e-values. They further show that e-values characterize maximin optimal contracts, thus providing actionable advice for any principal looking to design statistical contracts.

There is a drawback, however. The contract above assumes that a single dataset is produced by the agent. But in many applications it is natural to consider sequential experiments. Data may take time to procure, or the agent may wish to obtain partial rewards based on the results thus far. Clinical trials are a paradigmatic case. They run over months or years, so rather than wait for the trial to conclude, a regulator may grant accelerated or conditional approval on interim evidence and broaden the authorization as more data arrive. In fact, sequential allocation and experimentation is now pervasive in the technology industry and elsewhere (e.g., Spotify [41], Netflix [30], Eppo [31], Growthbook [20], Abridge [34], Adobe [1], as well as in government and public policy [22, 23]).

Our goal in this work is to extend the single-shot statistical contract of Bates et al. [5] to the sequential setting, thus enabling adaptive experimentation and interim decision-making. Our contributions are as follows:

  1. We propose two versions of a sequential statistical contract: one with deferred rewards and one with incremental rewards. Extending the notion of null deterrence from the single-shot setting, we define null-deterring sequential contracts as those where a null agent’s expected reward is at most the entry cost under every parameter and stopping strategy.

  1. We classify the set of null-deterring contracts under both reward schedules using e-processes. For incremental reward schedules in particular, we show how to construct such contracts using functions called adjusters [14, 15, 42], which also arise in preserving accumulated evidence and combining e-processes [8].

  1. We analyze the agent’s best response under both reward schedules, deriving utility bounds and simple policies with explicit performance guarantees. We also develop approximations that require less knowledge of the agent’s distribution and compare these policies experimentally.

We note that Bates et al. [5] also gave a sequential version of the single-shot setup above, but their time-horizon must be known in advance. That is, the contract can only proceed for some fixed number of rounds determined a priori. We give more detail on the differences between their sequential contract and ours in Appendix § 9, but here let us summarize three ways in which our contract differs: (i) we allow stopping rules for both the principal and the agent, (ii) we study two distinct reward schedules, and (iii) we handle unbounded horizons. That said, their sequential contract also has benefits over ours, such as the ability to consider per-round costs. We thus view our contribution as extending the single-shot contract in an alternative way to that originally proposed by Bates et al. [5].

Code is available at https://github.com/bchugg/statcontracts.

Related work

Since the original article of Bates et al. [5], there have been several follow-up works on principal-agent testing. Chen et al. [6] study a similar setup but focus on characterizing the principal’s optimal p-value threshold under strategic participation of the agent. Shi et al. [43] study the agent’s optimal response under general concave utility functions, and then later examine the situation of a principal interacting with multiple heterogeneous agents and focus on the issue of adverse selection [44]. Bates et al. [4] study a Bayesian version of the principal-agent problem, and Chugg et al. [9] study a multi-agent version.

Recently, Artola Velasco et al. [2] also studied principal-agent hypothesis testing in a sequential setting, though with different goals. In their model, the principal seeks to maximize social welfare by subsidizing a fraction of the agent’s experimentation cost, and the agent is uncertain about efficacy and updates a prior private belief. Despite these differences, the two works share key ingredients: both rely on e-values [37], and both characterize the agent’s optimal strategy via dynamic programming. We therefore view their work as complementary to ours, and together they highlight the many possible extensions of principal-agent problems to more dynamic settings.

Nguyen et al. [32] also study sequential statistical protocols in which an agent chooses whether to submit a project. The principal observes public testing outcomes and uses future access to testing to influence these decisions. In this setup, the agent repeatedly decides whether to submit a new project, whereas ours accumulates evidence about a fixed type. Velasievic et al. [48] use anytime-valid tests to design contracts that protect users of AI services from a provider covertly downgrading its model.

Our analysis of the agent’s parameter choice is closely related to recent work of Clerico et al. [11] on sequential testing with preferences over rejection times. Their EDO criterion coincides with our CGF-optimal criterion, and our bounds adapt their single-threshold bounds. The settings differ, however. They choose parameters to maximize a discounted reward for rejecting the null at a single significance level. Our agent faces a contract whose reward grows with the evidence and is not always paid out all at once. The bounds must therefore hold across many evidence levels and be combined with the agent’s stopping decision.

Contract theory has a long history outside of the particular statistical framing studied here. Its primary goal is to study the interactions of agents, at least one of whom has private information. We refer to Lanchim[a][29] and Laffont and Martimort [28] for overviews. The adjacent field of algorithmic (as opposed to statistical) contract theory [17] is also concerned with principal-agent problems, but emphasizes the computational aspect as opposed to the statistical aspect. There, the goals are to learn optimal payout vectors and to analyze the computational complexity of doing so [12, 7, 51].

Technical preliminaries

Let (Ω,F)(\Omega, \mathcal{F}) be a measurable space. An e-value for a set of probability distributions Q\mathcal{Q} on (Ω,F)(\Omega, \mathcal{F}) is a nonnegative random variable E:Ω→[0,+∞]E:\Omega\to[0,+\infty] such that EQ[E]≤1\mathbb{E}_{Q}[E] \le1 for all Q∈QQ \in\mathcal{Q}. For readers unfamiliar with e-values, it is helpful to think of e-values as statistical cousins of p-values. While developed for similar purposes (in particular, to provide a measure of evidence against the null [10]) their definition in terms of expectations instead of tail probabilities gives them useful properties that p-values do not have. We refer to Ramdas and Wang [37] and Grünwald et al. [21] for more.

As our motivation in this work is to consider sequential settings, we will require the sequential version of an e-value, the e-process. To define it, we suppose that the space (Ω,F)(\Omega, \mathcal{F}) admits a filtration (Ft)t≥0(\mathcal{F}_t)_{t\geq0}. Then an e-process for Q\mathcal{Q} is an (Ft)(\mathcal{F}_t)-adapted sequence of nonnegative random variables (Et)t≥0(E_t)_{t\geq0} (that is, EtE_t is Ft\mathcal{F}_t-measurable for all tt) such that EτE_\tau is an e-value for Q\mathcal{Q} at all stopping times τ\tau. Recall that a stopping time is a random variable such that {τ=t}\{\tau=t\} is Ft\mathcal{F}_t-measurable for all t≥0t\geq0. That is, at time tt, one can always determine whether the stopping time is now. All logarithms throughout the paper are natural logarithms.

Sequential Statistical Contracts

Our first order of business is to determine how to extend the reward menu R\mathcal{R} in the single-shot contract to the sequential setting. Instead of a set of functions, we will have a set of processes. More specifically, R\mathcal{R} will consist of a set of process families (Rt)(R_t), each potentially parameterized by a sequence (λt)(\lambda_t). To make things concrete, consider the process family defined by

R0λ=0,Rtλ=Rt(λ1t)=Cexp⁡{∑i=1tλiXi−18∑i=1tλi2},t≥1,(1)R^\lambda_0 = 0,\qquad R^\lambda_t = R_t(\lambda^t_1) = C\exp\left\{\sum_{i=1}^{t}\lambda_i X_i-\frac{1}{8}\sum_{i=1}^{t}\lambda_i^2\right\},\qquad t\geq1, \tag*{(1)}

where we use λ1t\lambda^t_1 as shorthand for (λ1,…,λt)(\lambda_1,\ldots,\lambda_t). When opting into the contract, the agent would choose a specific reward process (Rtλ)(R^\lambda_t) but specify only λ1\lambda_1 initially. Then, before round tt, the agent chooses λt\lambda_t based on past observations. This example was not pulled from whole cloth: the underlying exponential process, initialized at one, is a nonnegative supermartingale for iid data with mean 00 bounded in [−1/2,1/2][-1/2,1/2]. (More generally, for λi≥0\lambda_i \geq0, it is a nonnegative supermartingale whenever the data are iid, have nonpositive mean and are 1/21/2-sub-Gaussian, e.g., Xi∼N(θ,1/4)X_i \sim N(\theta,1/4) with θ≤0\theta\leq0.) As we’ll see below, nonnegative supermartingales result in null-deterring contracts.

We will consider two reward schedules: deferred and incremental. Under the deferred schedule, the agent receives payment only when the experiment ends. Under the incremental schedule, the agent receives rewards along the way, with each period’s reward becoming immediately available. Incremental rewards are natural when the rewards are themselves immediately actionable. A pharmaceutical company may want to take their drug to a corner of the market, or begin profiting, based on the evidence thus far. Deferred rewards arise in settings where evidence is accumulated sequentially, but the relevant transfer or decision is made only once, at the end. Examples include regulatory approval, prize competitions, grant funding, or venture financing. In such settings, the agent may gather evidence over time, while the eventual reward is determined only when the data gathering stops.

The reward schedule also restricts the reward process. Under the incremental schedule, previously disbursed rewards cannot be clawed back, so the cumulative reward should be nondecreasing over time. Under the deferred schedule, only the terminal reward is given. Thus, the total reward may fluctuate up and down according to the evidence.

With the example of the reward process above and the two reward schedules in mind, let us define a sequential statistical contract.

Sequential statistical contract C=(C,R,σ)\mathcal{C} = (C,\mathcal{R},\sigma)

  1. The principal publicly announces a cost C>0C > 0, a menu of reward processes R\mathcal{R}, a stopping time σ≥1\sigma\ge1 (which may be infinite), and the type of reward schedule.

  2. If the agent pays CC to opt-in, they choose a reward process (Rtλ)t≥0(R_{t}^{\lambda})_{t \ge0} from R\mathcal{R}. Set R0=0R_{0} = 0.

  3. For t=1,2,…t = 1, 2, \ldots

    • The agent chooses parameter(s) λt\lambda_t if applicable

    • A trial is run producing data Xt∼PθX_t \sim P_{\theta}, where θ\theta is the agent’s type

    • The total reward Rt=Rt(Xt,λt)≥0R_t = R_t(X_t,\lambda_t) \ge0 is calculated. On the incremental reward schedule, the agent is awarded Rt−Rt−1R_t - R_{t-1}, ensured to be nonnegative.

    • If σ=t\sigma= t the contract is ended. On the deferred schedule, the agent is awarded RtR_t.

    • The agent decides whether to stop or continue.

    • If they stop, on the deferred schedule, they are awarded RtR_t.

Several remarks are in order. First, in addition to allowing the agent to stop the contract, we also allow the principal to commit ex ante to a stopping rule. This is useful in a variety of settings. For example, the principal may face a finite budget, a fixed review horizon, or some other operational constraint that limits how long the experiment can continue. Of course, the stopping rule need not be so explicit; it may depend on the growth rate of some utility function, for instance.

Second, we do not impose per-period costs on the agent. That is, once the agent has paid the entry cost CC, they do not make additional payments each round in order to generate further data. We interpret CC as the upfront cost of mounting the research program: setting up the experiment, building infrastructure, hiring personnel, obtaining approval, and incurring associated opportunity costs. This differs from the sequential model of Bates et al. [5]; see Appendix § 9.

More broadly, the costs of continuation need not be symmetric across the two parties. For instance, once the FDA has authorized a clinical trial, their marginal cost of allowing the study to continue may be quite low. But the agent may still face substantial costs from administering the treatment and monitoring patients.

Null Deterrence in Sequential Contracts

Following the intuition of Bates et al. [5] in the single-shot setting, the principal would like to design a menu R\mathcal{R} such that, regardless of the agent’s strategy (i.e., their choice of R∈RR \in\mathcal{R}, their optimization of (λt)(\lambda_t) if applicable, and when they choose to stop), the expected reward allocated to a null agent is at most CC. In the single-shot setting, null deterrence (ND) is the condition that Eθ[R(X)]≤C\mathbb{E}_{\theta}[R(X)] \le C for all θ∈Θ0\theta\in\Theta_0. We extend this to the sequential setting as follows.

Definition 1. The sequential contract (C,R,σ)(C,\mathcal{R},\sigma) is null-deterring (ND) if

for all (Rt)∈R and all θ∈Θ0, Eθ[Rτ∧σ]≤C for any stopping time τ≥1.(2)\text{for all } (R_t) \in\mathcal{R} \text{ and all } \theta\in\Theta_0,\ \mathbb{E}_{\theta}[R_{\tau\wedge\sigma}] \le C \text{ for any stopping time } \tau\ge1. \tag*{(2)}

We may sometimes discuss null deterrence not of an entire contract itself, but of a single process

RR. In particular, if RR satisfies Eθ[Rτ∧σ]≤C\mathbb{E}_{\theta}[R_{\tau\wedge\sigma}]\le C for all θ∈Θ0\theta\in\Theta_{0} and all stopping times τ≥1\tau\ge1,2 we will say that it is ND. Importantly, null deterrence does not require the principal to know the agent’s utility function. The condition holds under every agent strategy and it is thus an asymmetric guarantee, prioritizing the principal over the agent.

On the deferred reward schedule, (ND) should remind us of the definition of an e-process. Indeed, we will show below that all ND contracts on the deferred reward schedule are characterized by e-processes.

Things are not quite as simple on the incremental reward schedule, however. Many, if not most, e-processes are not almost surely increasing, and thus do not satisfy Rt−Rt−1≥0R_t-R_{t-1}\ge0 for all t≥1t\ge1. Instead, we can construct ND contracts by using the concept of an adjuster [42, 8, 14, 15], which is a nonnegative, nondecreasing, right-continuous function A:[1,∞]→[0,∞]A:[1,\infty]\to[0,\infty] with A(∞)=lim⁡y→∞A(y)A(\infty)=\lim_{y\to\infty}A(y), satisfying

∫1∞A(y)y2 dy≤1.(3)\int_{1}^{\infty}\frac{A(y)}{y^{2}}\,\mathrm{d}y\le1. \tag*{(3)}

The adjuster is admissible if (3) holds with equality and A(∞)=∞A(\infty)=\infty [42]. Adjusters have their roots in the game-theoretic interpretation of e-processes, where they are viewed as the wealth process of a statistician playing a game against nature [38]. In such a setting, it is sometimes convenient to ensure that the wealth doesn’t decrease over time, which is what adjusters were developed to accomplish.

More formally, an adjuster satisfies the following guarantee: For any test supermartingale (Mt)(M_t) (meaning a nonnegative supermartingale with initial value 1) there exists a test supermartingale (Yt)(Y_t) such that A(Mt∗)≤YtA(M_t^{*})\le Y_t almost surely, where Mt∗=max⁡0≤s≤tMsM_t^{*}=\max_{0\le s\le t}M_s [42 Section 7]. This, together with the optional stopping theorem, implies that A(Mt∗)A(M_t^{*}) is an e-process, a fact which will undergird our results.

With the concept of adjusters in hand, we can state the following result regarding ND statistical contracts.

Proposition 1. Let C=(C,R,σ)C=(C,\mathcal{R},\sigma) be a sequential statistical contract. For the deferred reward schedule, CC is ND if and only if (Rt∧σ/C)(R_{t\wedge\sigma}/C) is an ee-process for Θ0\Theta_0 for all (Rt)∈R(R_t)\in\mathcal{R}. For the incremental reward schedule, suppose every (Rt)∈R(R_t)\in\mathcal{R} is nondecreasing. Then CC is ND if and only if Eθ[R∞σ]≤C\mathbb{E}_{\theta}[R_{\infty}^{\sigma}]\le C for all θ∈Θ0\theta\in\Theta_0 and all (Rt)∈R(R_t)\in\mathcal{R}, where R∞σ=lim⁡t→∞Rt∧σR_{\infty}^{\sigma}=\lim_{t\to\infty}R_{t\wedge\sigma}. In particular, CC is ND if (Rt/C)(R_t/C) is an ee-process for Θ0\Theta_0 for every (Rt)∈R(R_t)\in\mathcal{R}. Further, given an ee-process (Et)t≥1(E_t)_{t\ge1} for Θ0\Theta_0, (CA(Et∗))t≥1(CA(E_t^*))_{t\ge1} is ND for any adjuster AA, where Et∗=1∨max⁡1≤k≤tEkE_t^*=1\vee\max_{1\le k\le t}E_k.

Faced with Proposition 1, a natural question is whether all increasing ee-processes can be represented via adjusters. This turns out to be false. Appendix § 8.§ 3 gives a bounded nondecreasing ee-process that cannot be represented, or even dominated, by an adjusted running maximum.

Nevertheless, adjusted ee-processes will play an important role in our analysis in § 4, so let us linger on their details here. To foster some intuition, consider the following scheme. The principal fixes evidence levels 1<a1<⋯<aK1<a_1<\cdots<a_K, which we call milestones, and promises a milestone payment rj≥0r_j\ge0 the first time the ee-process (Et)(E_t) (i.e., the evidence) reaches aja_j. The cumulative rewards are then

Rt=∑j=1Krj1{Et∗≥aj}such that∑j=1Krjaj≤C,(4)R_t=\sum_{j=1}^{K}r_j\mathbf{1}\{E_t^*\ge a_j\}\quad\text{such that}\quad\sum_{j=1}^{K}\frac{r_j}{a_j}\le C, \tag*{(4)}

where Et∗E_t^* is the running maximum. When the agent is null, the probability that the evidence ever reaches aja_j is at most 1/aj1/a_j by Ville’s inequality [49, 40], so a payment of rjr_j at level aja_j costs at most rj/ajr_j/a_j in expectation. The constraint in (4) thus caps the expected payout to a null agent at the entry cost CC, which we call the null-payout budget, and it is exactly (3) for A(y)=C−1∑jrj1{y≥aj}A(y)=C^{-1}\sum_jr_j\mathbf{1}\{y\ge a_j\}. On this view, an adjuster is simply a choice of how to spend the null-payout budget across evidence levels, with milestone aja_j using the fraction rj/(Caj)r_j/(Ca_j).

The same reading applies to any adjuster, not only those with discrete milestone payments. Since AA is nondecreasing and right-continuous,

A(y)=A(1)+∫(1,∞)1{y≥a} dA(a),(5)A(y)=A(1)+\int_{(1,\infty)}\mathbf{1}\{y\ge a\}\,\mathrm{d}A(a), \tag*{(5)}

and Tonelli’s theorem turns (3) into the budget constraint

∫1∞A(y)y2 dy=A(1)+∫(1,∞)dA(a)a≤1.(6)\int_1^\infty\frac{A(y)}{y^2}\,\mathrm{d}y=A(1)+\int_{(1,\infty)}\frac{\mathrm{d}A(a)}{a}\le1. \tag*{(6)}

The contract therefore pays a baseline CA(1)CA(1) at the end of the first round, whatever the data, and pays C dA(a)C\,\mathrm{d}A(a) the first time the evidence reaches a>1a>1. The baseline is reached with probability one, so it costs its full value, whereas a payment at level aa costs only the fraction 1/a1/a of its value.

Shafer et al. [42] suggested the class of admissible adjusters Aκ(y)=κy1−κA_\kappa(y)=\kappa y^{1-\kappa} for κ∈(0,1)\kappa\in(0,1). Other examples include A(y)=y−1A(y)=\sqrt{y}-1 [8] and A(y)=y2log⁡2(1+y)log⁡2(1+y)A(y)=\frac{y^2\log2}{(1+y)\log^2(1+y)} [27]. We can also consider mixing over AκA_\kappa:

Aamix⁡(y)=∫01κy1−κaκ−1 dκ=ay∫01a−κ dκ.A_a^{\operatorname{mix}}(y)=\int_0^1\kappa y^{1-\kappa}a^{\kappa-1}\,\mathrm{d}\kappa=ay\int_0^1a^{-\kappa}\,\mathrm{d}\kappa.

The uniform mixture a=1a=1 recovers the adjuster used by Choe and Ramdas [8].

Figure 1 compares the growth rates of these adjusters. Although all are admissible, they allocate their null-payout budget differently (right panel). The baseline reward matters. In particular, A1/2(1)=1/2A_{1/2}(1)=1/2 and A1mix⁡(1)=1/2A_1^{\operatorname{mix}}(1)=1/2, so these adjusters return half of the entry cost in round one, while the KV adjuster returns 1/(2log⁡2)≈72%1/(2\log2) \approx72\% and the square-root adjuster has no baseline payment. The principal’s choice of adjuster thus encodes a preference over where on the evidence scale to reward the agent.

Figure 1. Depictions of the Shafer adjuster Aκ(y)=κy1−κA_{\kappa}(y)=\kappa y^{1-\kappa} with κ=0.5\kappa=0.5, the square-root adjuster y−1\sqrt{y}-1 [8], the KV adjuster y2log⁡2/((1+y)log⁡2(1+y))y^{2}\log2/((1+y)\log^{2}(1+y)) [27], and the Beta(a,1)(a,1) mixtures AamixA^{\mathrm{mix}}_{a} for a∈{0.5,1,2}a\in\{0.5,1,2\}. Left: the reward as a function of running maximum evidence. Right: the share of the null-payout budget spent on payments at evidence levels up to eue^{u}, namely A(1)+∫(1,eu]dA(a)/aA(1)+\int_{(1,e^{u}]} \mathrm{d}A(a)/a, against log-evidence uu. Dots at zero show the unconditional first-round payment CA(1)CA(1) as a fraction of CC, and the remaining budget funds later milestones.

Remark 1. Definition 1 was formulated linearly in terms of the reward and cost. One could instead require Eθ[g(Rτ∧σ)]≤g(C)\mathbb{E}_{\theta}[g(R_{\tau\wedge\sigma})] \le g(C) for a nonnegative measurable function gg with g(C)>0g(C)>0. The e-process characterization in Proposition 1 then applies to g(Rt∧σ)/g(C)g(R_{t\wedge\sigma})/g(C) for t≥1t\ge1, extended by zero at time zero.

The Agent’s Strategy

The discussion on null-deterring contracts in the preceding section focuses on designing useful contracts from the principal’s perspective. But what about the agent? Faced with a null-deterring contract, what is the agent’s best response?

To answer this, we need to endow the agent with a utility function. Once such a function is introduced, the agent’s optimal strategy can be handled using machinery from stochastic control theory [36, 45]. In particular, it can be written in terms of a Bellman equation (or the Snell envelope, if the parameter policy is fixed in advance). We give the details in Appendix § 8.§ 1. But such solutions are not closed-form, not necessarily computationally tractable, and require that the agent possess full knowledge of their data-generating distribution PθP_\theta. Therefore, in this section, we are interested in strategies that are more transparent, easier to interpret, and lend themselves to approximations that do not require full knowledge of the distribution.

The Bellman and Snell approaches given in the appendix handle general utility functions and e-processes, but our approximations require making a few simplifying assumptions. For one, we will study specific utility functions for the agent, one for the incremental reward schedule and one for the deferred schedule. For a fixed discount factor 0<δ<10<\delta<1 and t≥1t\ge1, define

GtI:=∑s=1tδs(Rs−Rs−1),andGtD:=δtlog⁡(Rt).G_t^{\mathrm{I}} := \sum_{s=1}^{t}\delta^s(R_s-R_{s-1}), \qquad\text{and} \qquad G_t^{\mathrm{D}} := \delta^t\log(R_t).

The superscripts refer to the reward schedule. Under the deferred schedule the agent is paid once, so only the final payout matters; under the incremental schedule every interim payment counts. Both objectives are common in financial models [see, e.g., 3, 26].

The two reward schedules (and their corresponding utility functions) affect the agent’s decision problem. Since the reward process must be nondecreasing on the incremental schedule, once the agent opts into the contract they should never exit. Therefore, the agent’s decision on the incremental schedule is how to choose the parameters (λt)(\lambda_t) (if any) to optimize the reward process. On the deferred schedule, meanwhile, the agent is concerned both with optimizing these parameters and also with potentially stopping and exiting the contract. Indeed, there’s no guarantee that (GtD)(G_t^{\mathrm{D}}) is nondecreasing in tt.

We also assume throughout the rest of this section that the principal imposes no stopping time (σ=∞\sigma=\infty) and that the underlying e-process takes the form of a product:

Etπ=∏i=1tfλi(Xi),E0π=1,Wtπ=log⁡Etπ,(7)E_t^\pi=\prod_{i=1}^{t}f_{\lambda_i}(X_i), \qquad E_0^\pi=1, \qquad W_t^\pi=\log E_t^\pi, \tag*{(7)}

where fλf_\lambda is positive and the superscript π\pi is the strategy determining the parameters λi\lambda_i and, if applicable, the stopping time. For t≥1t\ge1, the rewards under the deferred schedule will be RtD=CEtπR_t^{\mathrm{D}}=CE_t^\pi and those under the incremental schedule will be RtI=CA((Etπ)t)R_t^{\mathrm{I}}=CA((E_t^\pi)_t) for some adjuster AA.

Let us summarize our path forward.

  1. First we identify a parameter λCGF\lambda_{\mathrm{CGF}} defined by

λCGF∈argmax⁡λ∈Λ Eθ[fλ(X)ρ∗]wheresup⁡λ∈ΛEθ[fλ(X)ρ∗]=1δ.(8)\lambda_{\mathrm{CGF}} \in\underset{\lambda\in\Lambda}{\operatorname{argmax}}\ \mathbb{E}_{\theta}\left[f_{\lambda}(X)^{\rho^*}\right] \quad\text{where}\quad \sup_{\lambda\in\Lambda}\mathbb{E}_{\theta}\left[f_{\lambda}(X)^{\rho^*}\right] = \frac{1}{\delta}. \tag*{(8)}

where Λ\Lambda is the set of values λ\lambda can take. We call λCGF\lambda_{\mathrm{CGF}} the CGF-optimal parameter, so named because it can be defined by the CGF of the log-increment log⁡fλ(X)\log f_{\lambda}(X), as we’ll see below. Under both reward schedules, we bound the gap between the agent’s utility under λCGF\lambda_{\mathrm{CGF}} and the optimal adaptive strategy as a function of ρ∗\rho^*. In particular, the gap is small when ρ∗h\rho^*h is small, where hh bounds the log-increments.

  1. We then introduce the growth-optimal parameter λGRO\lambda_{\mathrm{GRO}}, a first-order approximation to λCGF\lambda_{\mathrm{CGF}}, defined by

λGRO∈argmax⁡λ∈Λ Eθ[log⁡fλ(X)],(9)\lambda_{\mathrm{GRO}} \in\underset{\lambda\in\Lambda}{\operatorname{argmax}}\ \mathbb{E}_{\theta}[\log f_{\lambda}(X)], \tag*{(9)}

which can be related to λCGF\lambda_{\mathrm{CGF}} via log⁡Eθ[fλ(X)ρ]=ρEθ[log⁡fλ(X)]+O(ρ2)\log\mathbb{E}_{\theta}[f_{\lambda}(X)^{\rho}] = \rho\mathbb{E}_{\theta}[\log f_{\lambda}(X)] + O(\rho^2). The GRO objective is easier to estimate from data by an empirical average and thus motivates several plug-in policies that can be instantiated even when the agent does not have full knowledge of their distribution. These plug-ins are presented in § 4.3 and the strategies are compared experimentally in § 5.

We begin by introducing the CGF-optimal parameter λCGF\lambda_{\mathrm{CGF}}.

The CGF-optimal rule

For a fixed parameter λ\lambda, define the iid log-increments and their drift by

Δi(λ):=log⁡fλ(Xi),μ(λ):=Eθ[Δ1(λ)].(10)\Delta_i(\lambda) := \log f_{\lambda}(X_i), \qquad\mu(\lambda) := \mathbb{E}_{\theta}[\Delta_1(\lambda)]. \tag*{(10)}

Recall that WtπW_t^\pi is the logarithm of the e-process in (7). For a threshold κ≥0\kappa\ge0, write

Tπ(κ)=inf⁡{t≥0:Wtπ≥κ},Hπ(κ)=Eθ,π[δTπ(κ)],(11)T_\pi(\kappa) = \inf\{t \ge0 : W_t^\pi\ge\kappa\}, \qquad H_\pi(\kappa) = \mathbb{E}_{\theta,\pi}[\delta^{T_\pi(\kappa)}], \tag*{(11)}

with δ∞=0\delta^\infty= 0. In words, Tπ(κ)T_\pi(\kappa) is the first time that the log-evidence WtπW_t^\pi reaches κ\kappa, and Hπ(κ)H_\pi(\kappa) is the expected discount at that time. For a strategy that plays a constant parameter λ\lambda, we write Tλ(κ)T_\lambda(\kappa) and Hλ(κ)H_\lambda(\kappa).

The reader may be wondering why we defined TπT_\pi. Why are we studying threshold rules? First, they are natural under the incremental schedule when the reward process is an adjusted e-process, which pays C dA(a)C\,\mathrm{d}A(a) exactly when the evidence first reaches aa (§ 3). Second, under the deferred schedule with discounted log utility, a reward-threshold rule is optimal at each fixed parameter with integrable log-increments (Appendix § 8.§ 2).

Fix λ\lambda and define the CGF

ψλ(s)=log⁡Eθ[esΔ1(λ)].(12)\psi_{\lambda}(s) = \log\mathbb{E}_{\theta}[e^{s\Delta_1(\lambda)}]. \tag*{(12)}

Consider what happens as the strategy which plays λ\lambda tries to reach a log-evidence of κ\kappa. Each round changes the log-evidence by Δt(λ)\Delta_t(\lambda) and multiplies the eventual discount by δ\delta. Since the increments are iid, conditioning on the first one gives the recurrence Hλ(κ)=δEθ[Hλ(κ−Δ1(λ))]H_\lambda(\kappa) = \delta\mathbb{E}_{\theta}[H_\lambda(\kappa- \Delta_1(\lambda))] for κ>0\kappa> 0, with Hλ(κ)=1H_\lambda(\kappa) = 1 for κ≤0\kappa\le0. If, for the moment, we posit that the solution to this recursive equation looks like Hλ(κ)=exp⁡(−sκ)H_{\lambda}(\kappa)=\exp(-s\kappa) for some fixed ss, then taking logs of both sides and rearranging gives log⁡(1/δ)=log⁡Eθ[exp⁡(sΔ1)]=ψλ(s)\log(1/\delta)=\log\mathbb{E}_{\theta}[\exp(s\Delta_1)]=\psi_{\lambda}(s). This motivates the definitions

ρλ:=sup⁡{s≥0:ψλ(s)≤log⁡(1/δ)},ρ∗:=inf⁡λ∈Λρλ.(13)\rho_{\lambda}:=\sup\{s\ge0:\psi_{\lambda}(s)\le\log(1/\delta)\},\qquad\rho^{*}:=\inf_{\lambda\in\Lambda}\rho_{\lambda}. \tag*{(13)}

For well-behaved CGFs, we have ψλ(ρλ)=log⁡(1/δ)\psi_{\lambda}(\rho_{\lambda})=\log(1/\delta). Then ρ∗\rho^{*} is simply the smallest across all such roots and, when 0<ρ∗<∞0<\rho^{*}<\infty, it satisfies δEθ[fλ(X)ρ∗]≤1\delta\mathbb{E}_{\theta}[f_{\lambda}(X)^{\rho^{*}}]\le1 for every λ∈Λ\lambda\in\Lambda.3

The approximation Hλ(κ)≈e−ρλκH_{\lambda}(\kappa)\approx e^{-\rho_{\lambda}\kappa} suggests minimising ρλ\rho_{\lambda}. For 0<ρ∗<∞0<\rho^{*}<\infty, we call a parameter λCGF\lambda_{\mathrm{CGF}} CGF-optimal if it attains the infimum and satisfies the recurrence relation with equality:

λCGF∈argmin⁡λ∈ΛρλandδEθ[fλCGF(X)ρ∗]=1.(14)\lambda_{\mathrm{CGF}}\in\mathop{\operatorname{argmin}}_{\lambda\in\Lambda}\rho_{\lambda}\quad\text{and}\quad\delta\mathbb{E}_{\theta}[f_{\lambda_{\mathrm{CGF}}}(X)^{\rho^{*}}]=1. \tag*{(14)}

Equivalently, the CGF-optimal parameter satisfies ψλCGF(ρ∗)=log⁡(1/δ)\psi_{\lambda_{\mathrm{CGF}}}(\rho^{*})=\log(1/\delta). Such a parameter maximizes Eθ[fλ(X)ρ∗]\mathbb{E}_{\theta}[f_{\lambda}(X)^{\rho^{*}}] over λ∈Λ\lambda\in\Lambda. Only the lower bounds below require that the minimum in (14) is achieved. The upper bounds hold regardless.

Intriguingly, the exponent ρ∗\rho^{*} has recently appeared in another context: it is the optimal exponent η⋆\eta_{\star} of the exponential-decay-optimal (EDO) criterion of Clerico et al. [11, Section 7, equation (4)]. The same exponent appears because both problems discount the time needed to reach a given level of evidence. Proposition 2 below is thus similar to their Proposition 3.

The motivation for λCGF\lambda_{\mathrm{CGF}} and ρ∗\rho^{*} rests on the approximation Hλ(κ)≈e−ρλκH_{\lambda}(\kappa)\approx e^{-\rho_{\lambda}\kappa}. We might wonder how tight this is. The following result shows that Hπ(κ)≤e−ρ∗κH_{\pi}(\kappa)\le e^{-\rho^{*}\kappa} for every strategy π\pi, and that a fixed parameter nearly attains this bound when the overshoot (the amount by which the log-evidence exceeds κ\kappa at the crossing time) is small.

Proposition 2. Suppose the observations are iid under PθP_{\theta}, and let s∈(0,∞)s\in(0,\infty) satisfy δEθ[fλ(X)s]≤1\delta\mathbb{E}_{\theta}[f_{\lambda}(X)^{s}]\le1 for every λ∈Λ\lambda\in\Lambda. Then, for every predictable parameter strategy π\pi, the process (δt(Etπ)s)(\delta^{t}(E_{t}^{\pi})^{s}) is a nonnegative supermartingale under PθP_{\theta}, and Hπ(κ)≤exp⁡{−sκ}H_{\pi}(\kappa)\le\exp\{-s\kappa\} for all κ≥0\kappa\ge0. Further, if a fixed parameter λ0∈Λ\lambda_{0}\in\Lambda satisfies δEθ[fλ0(X)s]=1\delta\mathbb{E}_{\theta}[f_{\lambda_{0}}(X)^{s}]=1 and Δ1(λ0)≤h\Delta_{1}(\lambda_{0})\le h almost surely for some h>0h>0, then, for every κ≥0\kappa\ge0,

exp⁡{−s(κ+h)}≤Hλ0(κ)≤sup⁡πHπ(κ)≤exp⁡{−sκ}.(15)\exp\{-s(\kappa+h)\}\le H_{\lambda_{0}}(\kappa)\le\sup_{\pi}H_{\pi}(\kappa)\le\exp\{-s\kappa\}. \tag*{(15)}

Thus, taking s=ρ∗s=\rho^{*} and λ0=λCGF\lambda_{0}=\lambda_{\mathrm{CGF}}, whenever the stated assumptions hold, gives HλCGF(κ)≥e−ρ∗hsup⁡πHπ(κ)H_{\lambda_{\mathrm{CGF}}}(\kappa)\ge e^{-\rho^{*}h}\sup_{\pi}H_{\pi}(\kappa). That is, the fixed CGF rule achieves at least the fraction e−ρ∗he^{-\rho^{*}h} of the optimal discounted value under any adaptive strategy. The proof is in Appendix § 7.

Next we study the properties of the CGF-optimal rule under each reward schedule. Before we do, however, let us emphasize that the CGF-optimal rule still requires essentially full distributional knowledge from the agent, since a CGF that is finite near zero determines the distribution. Thus, it requires about as much information as the Bellman equation. However, it is significantly more transparent and, via results such as Proposition 2, allows us to analyze how close it is to the optimal strategy. Moreover, as we’ll see in § 4.3, it motivates various plug-in strategies which can be implemented under limited (or no) distributional knowledge.

λ_CGF under both reward schedules

We begin with the incremental reward schedule. Fix an adjuster AA and cumulative rewards Rtπ=CA((Etπ)⋆)R_{t}^{\pi}=CA((E_{t}^{\pi})^{\star}) for t≥1t\ge1. The agent chooses predictable parameters to maximize

Jinc(π):=Eθ,π[∑t≥1δt(Rtπ−Rt−1π)]=δCA(1)+C∫(1,∞)Hπ(log⁡a) dA(a),(16)J_{\mathrm{inc}}(\pi):=\mathbb{E}_{\theta,\pi}\left[\sum_{t\ge1}\delta^{t}(R_{t}^{\pi}-R_{t-1}^{\pi})\right]=\delta CA(1)+C\int_{(1,\infty)}H_{\pi}(\log a)\,\mathrm{d}A(a), \tag*{(16)}

where the second equality follows from the adjuster representation (5) in § 3. We can integrate the crossing bounds in Proposition 2, taking s=ρ∗s=\rho^* and κ=log⁡a\kappa=\log a, to obtain bounds on the agent’s utility as follows.

Proposition 3. Suppose the observations are iid under PθP_\theta and 0<ρ∗<∞0<\rho^*<\infty. Let AA be an adjuster and consider rewards Rtπ=CA((Etπ)∗)R_t^\pi=CA((E_t^\pi)^*) for t≥1t\ge1 and σ=∞\sigma=\infty. Define

UA:=δCA(1)+C∫(1,∞)a−ρ∗ dA(a),VA∗:=sup⁡πJinc(π),U_A:=\delta CA(1)+C\int_{(1,\infty)}a^{-\rho^*}\,\mathrm{d}A(a),\qquad V_A^*:=\sup_\pi J_{\mathrm{inc}}(\pi),

and suppose UA<∞U_A<\infty. Then VA∗≤UAV_A^*\le U_A. If, in addition, there exists a parameter λCGF∈Λ\lambda_{\mathrm{CGF}}\in\Lambda satisfying δEθ[fλCGF(X)ρ∗]=1\delta\mathbb{E}_\theta[f_{\lambda_{\mathrm{CGF}}}(X)^{\rho^*}]=1 and Δ1(λCGF)≤h\Delta_1(\lambda_{\mathrm{CGF}})\le h almost surely for some h>0h>0, then

e−ρ∗hUA≤Jinc(λCGF)≤VA∗≤UA.(17)e^{-\rho^*h}U_A\le J_{\mathrm{inc}}(\lambda_{\mathrm{CGF}})\le V_A^*\le U_A. \tag*{(17)}

Remark 2. The upper bound also tells us when positive expected discounted profit is impossible. If ρ∗≥1\rho^*\ge1, then a−ρ∗≤a−1a^{-\rho^*}\le a^{-1} for a≥1a\ge1, so the adjuster budget gives

VA∗≤UA≤δCA(1)+C∫(1,∞)a−1 dA(a)≤C.V_A^*\le U_A\le\delta CA(1)+C\int_{(1,\infty)}a^{-1}\,\mathrm{d}A(a)\le C.

Thus ρ∗<1\rho^*<1 is necessary for the agent’s expected discounted receipts to exceed the entry cost CC.

Next let us consider the deferred schedule. Here, the agent chooses both a parameter strategy and when to collect the reward. Stopping times τ≥1\tau\ge1 may be infinite, in which case the agent never collects and we set G∞D:=0G_\infty^D:=0. For any fixed parameter λ\lambda with integrable log-increments, a threshold rule attains the optimal stopping value (Appendix § 8.§ 2). We thus study the utility GDG^D under the threshold rules Tλ(κ)T_\lambda(\kappa). For κ>0\kappa>0, define

Jλ(κ):=Eθ[GTλ(κ)D]=Eθ[δTλ(κ)log⁡RTλ(κ)(λ)].(18)J_\lambda(\kappa):=\mathbb{E}_\theta\left[G^D_{T_\lambda(\kappa)}\right]=\mathbb{E}_\theta\left[\delta^{T_\lambda(\kappa)}\log R_{T_\lambda(\kappa)}(\lambda)\right]. \tag*{(18)}

At the time T=Tλ(κ)T=T_\lambda(\kappa), we have WT=log⁡ET≥κW_T=\log E_T\ge\kappa and log⁡RT=log⁡C+WT\log R_T=\log C+W_T. Ignoring overshoot, so that WT≈κW_T\approx\kappa, gives the following approximation to the expected discounted utility:

Jλ(κ)≈J^λ(κ):=e−ρλκ(log⁡C+κ).(19)J_\lambda(\kappa)\approx\widehat{J}_\lambda(\kappa):=e^{-\rho_\lambda\kappa}(\log C+\kappa). \tag*{(19)}

where we’re using that Hλ(κ)≈exp⁡(−ρλκ)H_\lambda(\kappa)\approx\exp(-\rho_\lambda\kappa). When log⁡C<1/ρλ\log C<1/\rho_\lambda, maximizing this approximation over κ>0\kappa>0 gives

κλCGF=1/ρλ−log⁡C,J^λ(κλCGF)=Cρλ/(eρλ).(20)\kappa_\lambda^{\mathrm{CGF}}=1/\rho_\lambda-\log C,\qquad\widehat{J}_\lambda(\kappa_\lambda^{\mathrm{CGF}})=C^{\rho_\lambda}/(e\rho_\lambda). \tag*{(20)}

In this regime, the maximized value is a decreasing function of ρλ\rho_\lambda, hence λCGF\lambda_{\mathrm{CGF}}, when it exists, also optimizes this approximation. Thus, the CGF-optimal policy under the deferred schedule uses λt≡λCGF\lambda_t\equiv\lambda_{\mathrm{CGF}} and the stopping rule

τCGF=inf⁡{t≥1:Rt(λCGF)≥e1/ρ∗}.(21)\tau_{\mathrm{CGF}}=\inf\{t\ge1:R_t(\lambda_{\mathrm{CGF}})\ge e^{1/\rho^*}\}. \tag*{(21)}

For any other fixed parameter λ\lambda with 0<ρλ<∞0<\rho_\lambda<\infty, the corresponding reward threshold is RCGF(λ)=e1/ρλR_{\mathrm{CGF}}(\lambda)=e^{1/\rho_\lambda}. As in the case of incremental rewards, we can use properties of ρ∗\rho^* to give optimality guarantees for the CGF-optimal parameter and stopping rule.

Proposition 4. Suppose the observations are iid under PθP_\theta and 0<ρ∗<∞0 < \rho^* < \infty. Consider deferred rewards Rtπ=CEtπR_t^\pi= C E_t^\pi for t≥1t \ge1, with C>0C > 0. Let Vθ∗(C)V_\theta^*(C) be the supremum of expected utility over predictable parameter strategies and all stopping times τ≥1\tau\ge1, and define

UD(C):={Cρ∗/(eρ∗),log⁡C<1/ρ∗,log⁡C,log⁡C≥1/ρ∗.(22)U_D(C) := \begin{cases} C^{\rho^*}/(e\rho^*), & \log C < 1/\rho^*,\\ \log C, & \log C \ge1/\rho^*. \end{cases} \tag*{(22)}

Then Vθ∗(C)≤UD(C)V_\theta^*(C) \le U_D(C). If log⁡C<1/ρ∗\log C < 1/\rho^*, suppose also that there exists a parameter λCGF∈Λ\lambda_{\mathrm{CGF}} \in\Lambda satisfying δEθ[fλCGF(X)ρ∗]=1\delta\mathbb{E}_\theta[f_{\lambda_{\mathrm{CGF}}}(X)^{\rho^*}] = 1 and Δ1(λCGF)≤h\Delta_1(\lambda_{\mathrm{CGF}}) \le h almost surely for some h>0h > 0. Set κ=1/ρ∗−log⁡C\kappa= 1/\rho^* - \log C and q=(1+ρ∗h)e−ρ∗hq = (1+\rho^*h)e^{-\rho^*h}. Then

qUD(C)≤JλCGF(κ)≤Vθ∗(C)≤UD(C).(23)qU_D(C) \le J_{\lambda_{\mathrm{CGF}}}(\kappa) \le V_\theta^*(C) \le U_D(C). \tag*{(23)}

Thus the CGF-optimal policy attains at least the fraction qq of the optimal expected utility over all predictable strategies and stopping times. Appendix § 7 gives a corresponding guarantee for unbounded log-increments in terms of the overshoot’s second moment.

Approximate strategies

Let us now suppose that the agent does not know their full distribution. In this case they can try to approximate the CGF-optimal strategy via a first-order Taylor expansion. For fixed λ\lambda with μ(λ)>0\mu(\lambda) > 0 and a CGF finite near zero, ψλ(s)=sμ(λ)+o(s)\psi_\lambda(s) = s\mu(\lambda) + o(s) gives ρλ≈log⁡(1/δ)/μ(λ)\rho_\lambda\approx\log(1/\delta)/\mu(\lambda) as δ→1\delta\to1. (Discount factors close to one, which describe patient agents, are the natural regime.) Therefore, to minimize ρλ\rho_\lambda we want to maximize μ(λ)\mu(\lambda), which suggests choosing

λGRO∈argmax⁡λ∈Λ μ(λ).(24)\lambda_{\mathrm{GRO}} \in\underset{\lambda\in\Lambda}{\operatorname{argmax}}\ \mu(\lambda). \tag*{(24)}

Under the incremental schedule the agent never stops, so they use λGRO\lambda_{\mathrm{GRO}} throughout. Under deferred rewards, substituting 1/ρλ≈μ(λ)/log⁡(1/δ)1/\rho_\lambda\approx\mu(\lambda)/\log(1/\delta) and following (21) gives the threshold

RDRIFT(λ)=exp⁡{μ(λ)/log⁡(1/δ)}.(25)R_{\mathrm{DRIFT}}(\lambda) = \exp\{\mu(\lambda)/\log(1/\delta)\}. \tag*{(25)}

The agent collects at the first round t≥1t \ge1 at which Rt(λ)R_t(\lambda) reaches this threshold.

Of course, these choices require knowing the drift. But there are many distributions for which it is reasonable to know the drift, even when full distributional knowledge is unavailable. For the exponential reward (1), for instance, μ(λ)=λθ−λ2/8\mu(\lambda) = \lambda\theta- \lambda^2/8 depends only on the observation mean θ=Eθ[X]\theta= \mathbb{E}_\theta[X]. The same holds for “sub-ψ\psi” processes [24] with a deterministic variance process, which cover a wide range of e-processes.

Further, when the drift is unknown, the agent can estimate it with the plug-in estimator

μ^t(λ)=1t∑i=1tlog⁡fλ(Xi),(26)\widehat{\mu}_t(\lambda) = \frac{1}{t}\sum_{i=1}^{t}\log f_\lambda(X_i), \tag*{(26)}

and bet an approximate maximizer λ^t∈Λ\widehat{\lambda}_t \in\Lambda of μ^t\widehat{\mu}_t in the next round. Since the contract requires λt\lambda_t to be chosen before XtX_t is observed, the agent starts from some λ1∈Λ\lambda_1 \in\Lambda and, after observing XtX_t, sets λt+1=λ^t\lambda_{t+1} = \widehat{\lambda}_t. The resulting e-process is Etπ=fλ1(X1)∏i=2tfλ^i−1(Xi)E_t^\pi= f_{\lambda_1}(X_1)\prod_{i=2}^{t} f_{\widehat{\lambda}_{i-1}}(X_i). Such a strategy has been called the “method of predictable plug-ins” [50] in other contexts. Examples are given in § 5. Under the deferred reward schedule, we use the following stopping time alongside the plug-in:

τPLUG−IN=inf⁡{t≥1:log⁡Rt≥max⁡(log⁡C,μ^t(λ^t)log⁡(1/δ))}.(27)\tau_{\mathrm{PLUG-IN}} = \inf\left\{t \ge1 : \log R_t \ge\max\left(\log C,\frac{\widehat{\mu}_t(\widehat{\lambda}_t)}{\log(1/\delta)}\right)\right\}. \tag*{(27)}

The floor at log⁡C\log C keeps the agent from collecting less than the entry cost CC. Next let us experimentally compare all the strategies we’ve discussed.

Numerical Illustrations

We compare the strategies under both reward schedules, setting C=1C = 1. Appendix § 10 provides the implementation details.

Incremental schedule

We consider Bernoulli observations with mean p=0.7p = 0.7, null mean at most m=0.5m = 0.5, and increments fλ(x)=1+λ(x−m)f_{\lambda}(x) = 1 + \lambda(x - m) for λ∈[0,1/m)\lambda\in[0,1/m). The resulting “betting-style” e-process, Et=∏i≤tfλ(Xi)E_t = \prod_{i \le t} f_{\lambda}(X_i) is a common one in the e-value literature [50]. Since we are working under the incremental reward schedule, the reward process is CA⁡(Et∗)\operatorname{CA}(E_t^*) for some adjuster AA.

We are interested in understanding the relative performance of our strategies compared to the optimal Bellman benchmark (Appendix § 8) as well as the effect of different adjusters. We consider two adjusters with three discrete milestone payments (see § 3), which differ in where on the evidence scale they spend their null-payout budget. Milestone aja_j uses the fraction wjw_j of the budget, so the reward is rj=wjajr_j = w_j a_j, with (a1,a2,a3)=(2,4,8)(a_1, a_2, a_3) = (2,4,8). The low-milestone adjuster uses weights (0.8,0.1,0.1)(0.8, 0.1, 0.1) and the high-milestone adjuster uses (0.1,0.1,0.8)(0.1, 0.1, 0.8).

Figure 2 shows that all three fixed policies plotted attain at least 98% of the Bellman value when δ=0.995\delta= 0.995. As agents become less patient and δ\delta decreases, the gap widens. In other words, with stronger discounting, adaptive parameter choice matters more, regardless of the adjuster. At δ=0.9\delta= 0.9 under the high-milestone adjuster, the growth-optimal rule earns 63.3% of the Bellman value and the CGF-optimal rule 86.9%. We also compare both rules with the best fixed parameter, found numerically. Its gap to the CGF-optimal rule reflects the cost of overshoot in Proposition 3.

Although the plug-in improves as the agent becomes more patient, it still falls short of the fixed policies at δ=0.995\delta= 0.995. To see why, relate it to the growth-optimal rule. In the Bernoulli example, the plug-in uses λ^t=(p~t−m)+/(m(1−m))\widehat{\lambda}_t = (\widetilde{p}_t - m)^{+}/(m(1-m)), with p~t=min⁡{X‾t,0.99}\widetilde{p}_t = \min\{\overline{X}_t, 0.99\}, where X‾t\overline{X}_t is the running mean. It starts at λ1=0.1\lambda_1 = 0.1 and sets λt+1=λ^t\lambda_{t+1} = \widehat{\lambda}_t. This is exactly λGRO=(p−m)/(m(1−m))\lambda_{\mathrm{GRO}} = (p-m)/(m(1-m)) with pp replaced by p~t\widetilde{p}_t, so the plug-in is the growth-optimal rule with an estimated parameter. For δ\delta close to one, λGRO\lambda_{\mathrm{GRO}} is nearly optimal, λ^t\hat{\lambda}_t converges to it, and the plug-in’s gap reflects the cost of learning. At δ=0.9\delta= 0.9, however, λGRO=0.8\lambda_{\mathrm{GRO}} = 0.8 is far below the best fixed parameter of about 1.36. The plug-in’s early estimates are noisy, and after early successes they can be much larger than λGRO\lambda_{\mathrm{GRO}}, moving the bet toward the better parameter. As a result, the plug-in matches the growth-optimal rule under the low-milestone adjuster and exceeds it under the high-milestone adjuster (69.2% versus 63.3% of the Bellman value).

Deferred schedule

We use Gaussian observations Xi∼N(θ,0.52)X_i \sim N(\theta, 0.5^2) with the exponential reward (1), and Bernoulli observations with the betting reward above. The growth-optimal parameters are λGRO=4θ\lambda_{\mathrm{GRO}} = 4\theta and λGRO=(p−m)/(m(1−m))\lambda_{\mathrm{GRO}} = (p - m)/(m(1 - m)), respectively. For Bernoulli data, Figure 3 also includes a Hoeffding baseline, which uses the sub-Gaussian reward C∏i≤texp⁡{λ(Xi−m)−λ2/8}C\prod_{i\le t}\exp\{\lambda(X_i-m)-\lambda^2/8\}, valid for any data in [0,1][0,1] with mean at most mm. Its drift λ(p−m)−λ2/8\lambda(p-m)-\lambda^2/8 depends only on the mean, so, as with the Gaussian reward, an agent who knows only pp can implement the growth-optimal rule exactly, with λ=4(p−m)\lambda= 4(p-m) and the drift threshold (25). The price is a smaller drift, and hence lower utility. Appendix § 10 quantifies the gap using Pinsker’s inequality, derives the CGF-optimal parameters, and specifies the plug-in rules.

Figure 3. Deferred utility relative to the Bellman benchmark (B.2). Sweeps over θ\theta and pp use δ=0.995\delta= 0.995, and sweeps over δ\delta use θ=0.3\theta= 0.3 (Gaussian) and p=0.7p = 0.7 (Bernoulli), with m=0.5m = 0.5. The CGF-optimal policy (21) uses λCGF\lambda_{\mathrm{CGF}}. The CGF threshold RCGF(λGRO)=e1/ρλGROR_{\mathrm{CGF}}(\lambda_{\mathrm{GRO}}) = e^{1/\rho\lambda_{\mathrm{GRO}}} and the drift threshold (25) use λGRO\lambda_{\mathrm{GRO}}, and the plug-in uses the threshold rule (27). Ratios are estimated from 20,000 simulations with a cap of 2500 rounds.

In Figure 3, the CGF-optimal policy (21) is within 2% of the Bellman benchmark throughout the Gaussian settings and within 1.2% throughout the Bernoulli settings. The drift rule (25) performs similarly for patient agents, but loses more with stronger discounting. The plug-in rule (27) can lose substantially for weak signals or strong discounting.

For the bounded Bernoulli increments in Figure 3, Proposition 4 quantifies the loss from overshoot. Gaussian log-increments are unbounded above, but equation (A.5) in Appendix § 7 bounds the loss using the second moment of the overshoot. As hinted at by these results, the performance of our strategies can decline substantially when a single observation can carry the reward far past the threshold.

To give a concrete example, consider Bernoulli data with rare successes (m=0.01m = 0.01, p=0.02p = 0.02, δ=0.9\delta= 0.9). Under λCGF≈54.5\lambda_{\mathrm{CGF}} \approx54.5, one success multiplies the evidence by about 55, a log-jump of 4.0 against the log-evidence threshold 1/ρ∗≈1.171/\rho^{*} \approx1.17 in (21), so q≈0.14q \approx0.14. The CGF-optimal rule earns 0.184, whereas the smaller bet λ=15.5\lambda= 15.5 with its exact threshold β∗(15.5)\beta^{*}(15.5) from Proposition 6 earns 0.261. So the CGF-optimal rule loses at least 29.4% of the optimal value. More details are provided in Appendix § 10.

Conclusion

We have extended the framework of principal-agent hypothesis testing to settings with repeated and adaptive experimentation. We introduced sequential statistical contracts and showed how a principal can use e-processes to design contracts which deter the entry of null agents. We then focused on the agent, noting that techniques from stochastic control theory can be leveraged to optimize the reward process once they’ve opted into the contract. Such techniques are not closed-form, however, and require that the agent possess full knowledge of their distribution to implement them. We therefore provide several approximations which are more transparent and can be implemented with less information.

Acknowledgements. BC and AR acknowledge support from NSF grants IIS-2229881 and DMS-2310718.

Omitted Proofs

We give the proofs of the four propositions in the main text. When the parameter λ\lambda is fixed, we suppress its subscript in the increments, partial sums, and so on.

Proof of Proposition 1. If CC is ND, then by definition Eθ[Rτ∧σ]≤C\mathbb{E}_{\theta}[R_{\tau\wedge\sigma}] \le C for any stopping time τ≥1\tau\ge1. The bound also holds when τ\tau may be zero: since R0=0R_0 = 0 and rewards are nonnegative, Rτ∧σ≤R(τ∨1)∧σR_{\tau\wedge\sigma} \le R_{(\tau\vee1)\wedge\sigma}. Dividing by C>0C > 0 shows that (Rt∧σ/C)(R_{t\wedge\sigma}/C) is an e-process for Θ0\Theta_0.

Conversely, on the deferred schedule, if (Rt∧σ/C)(R_{t\wedge\sigma}/C) is an e-process, then Eθ[Rτ∧σ]≤C\mathbb{E}_{\theta}[R_{\tau\wedge\sigma}] \le C for all stopping times τ\tau, which is precisely ND.

On the incremental schedule, suppose every (Rt)∈R(R_t) \in\mathcal{R} is nondecreasing, so that R∞σ=lim⁡tRt∧σR^\sigma_\infty= \lim_{t} R_{t\wedge\sigma} exists in [0,∞][0,\infty] for each reward process. If CC is ND, then taking the deterministic stopping times τ=n\tau= n gives Eθ[Rn∧σ]≤C\mathbb{E}_{\theta}[R_{n\wedge\sigma}] \le C, and monotone convergence yields Eθ[R∞σ]≤C\mathbb{E}_{\theta}[R^\sigma_\infty] \le C. Conversely, if Eθ[R∞σ]≤C\mathbb{E}_{\theta}[R^\sigma_\infty] \le C, then Rτ∧σ≤R∞σR_{\tau\wedge\sigma} \le R^\sigma_\infty for every stopping time τ\tau, so Eθ[Rτ∧σ]≤C\mathbb{E}_{\theta}[R_{\tau\wedge\sigma}] \le C.

In particular, if (Rt/C)(R_t/C) is an e-process for every reward process, then Eθ[Rn∧σ]≤C\mathbb{E}_{\theta}[R_{n\wedge\sigma}] \le C for every nn, since n∧σn\wedge\sigma is a stopping time. Monotone convergence gives the required limit bound, hence ND.

It remains to verify the adjuster construction. Let (Et)(E_t) be an e-process for Θ0\Theta_0. By an equivalent definition of an e-process [38], for each θ∈Θ0\theta\in\Theta_0 there exists a test supermartingale (Mtθ)t≥0(M^\theta_t)_{t\ge0} under PθP_\theta such that Et≤MtθE_t \le M^\theta_t PθP_\theta-a.s. for all t≥1t \ge1. Moreover, since AA is an adjuster, there exists a test supermartingale (Ytθ)(Y^\theta_t) under PθP_\theta such that A(max⁡0≤k≤tMkθ)≤YtθA\left(\max_{0\le k\le t} M^\theta_k\right) \le Y^\theta_t PθP_\theta-a.s. Therefore, since AA is nondecreasing (and since Et∗=1∨max⁡k≤tEk≥1E_t^* = 1 \vee\max_{k\le t} E_k \ge1 lies in the domain of AA and M0θ=1M^\theta_0 = 1), for any t≥1t \ge1,

A(Et∗)≤A(max⁡0≤k≤tMkθ)≤YtθPθ-a.s.,A(E_t^*) \le A\left(\max_{0\le k\le t} M^\theta_k\right) \le Y^\theta_t \quad P_\theta\text{-a.s.},

hence Eθ[A(Et∗)]≤Eθ[Ytθ]≤1\mathbb{E}_{\theta}[A(E_t^*)] \le\mathbb{E}_{\theta}[Y^\theta_t] \le1. We conclude that (A(Et∗))t≥1(A(E_t^*))_{t\ge1} is an e-process for Θ0\Theta_0, immediately implying that (CA(Et∗))t≥1(CA(E_t^*))_{t\ge1} is null-determining. □

The following identity supplies the overshoot correction used in the proofs below and extends the performance guarantees to unbounded increments.

Lemma 1. Fix λ\lambda with ψλ(ρλ)=log⁡(1/δ)\psi_\lambda(\rho_\lambda) = \log(1/\delta) for some ρλ>0\rho_\lambda> 0. Let Pλ,ρλP_{\lambda,\rho_\lambda} be the tilted law defined by

dPλ,ρλdPθ∣Fn=exp⁡{ρλWn(λ)−nlog⁡(1/δ)}.\left.\frac{\mathrm{d}P_{\lambda,\rho_\lambda}}{\mathrm{d}P_\theta}\right|_{\mathcal{F}_n} = \exp\{\rho_\lambda W_n(\lambda) - n\log(1/\delta)\}.

For κ≥0\kappa\ge0, the crossing time Tλ(κ)T_{\lambda}(\kappa) is finite under this law. Write Oλ(κ)=WTλ(κ)(λ)−κO_{\lambda}(\kappa) = W_{T_{\lambda}(\kappa)}(\lambda) - \kappa for the overshoot. Then

Hλ(κ)=e−ρλκEPλ,ρλ[e−ρλOλ(κ)].(28)H_{\lambda}(\kappa) = e^{-\rho_{\lambda}\kappa}\mathbb{E}_{P_{\lambda,\rho_{\lambda}}}\left[e^{-\rho_{\lambda}O_{\lambda}(\kappa)}\right]. \tag*{(28)}

More generally, for any nonnegative measurable gg,

Eθ[δTλ(κ)g(WTλ(κ)(λ))1{Tλ(κ)<∞}]=e−ρλκEPλ,ρλ[e−ρλOλ(κ)g(κ+Oλ(κ))].(29)\mathbb{E}_{\theta}\left[\delta^{T_{\lambda}(\kappa)}g\left(W_{T_{\lambda}(\kappa)}(\lambda)\right)1\{T_{\lambda}(\kappa)<\infty\}\right] = e^{-\rho_{\lambda}\kappa}\mathbb{E}_{P_{\lambda,\rho_{\lambda}}}\left[e^{-\rho_{\lambda}O_{\lambda}(\kappa)}g\left(\kappa+O_{\lambda}(\kappa)\right)\right]. \tag*{(29)}

The formula extends to signed gg whenever one of its weighted positive or negative parts is integrable.

Proof. Suppress λ\lambda and write ρ=ρλ\rho=\rho_{\lambda}, T=Tλ(κ)T=T_{\lambda}(\kappa), and Mn=δneρWnM_n=\delta^n e^{\rho W_n}. Tilting the one-observation law by δeρΔ1\delta e^{\rho\Delta_1} and taking its iid product defines PρP_{\rho}, since δEθ[eρΔ1]=1\delta\mathbb{E}_{\theta}[e^{\rho\Delta_1}]=1. Its density on Fn\mathcal{F}_n is MnM_n. Hence EPρ[e−ρWn]=δn\mathbb{E}_{P_{\rho}}[e^{-\rho W_n}]=\delta^n, and

Pρ(T>n)≤Pρ(Wn<κ)≤eρκδn⟶0.P_{\rho}(T>n)\le P_{\rho}(W_n<\kappa)\le e^{\rho\kappa}\delta^n\longrightarrow0.

Changing measure on each event {T=n}\{T=n\} gives

Eθ[δTg(WT)1{T=n}]=EPρ[e−ρWTg(WT)1{T=n}].\mathbb{E}_{\theta}\left[\delta^Tg(W_T)1\{T=n\}\right]=\mathbb{E}_{P_{\rho}}\left[e^{-\rho W_T}g(W_T)1\{T=n\}\right].

Sum over n≥0n\ge0 and use WT=κ+OW_T=\kappa+O to obtain (A.2). Taking g≡1g\equiv1 gives (A.1). Applying the formula separately to positive and negative parts gives the signed extension.

For κ>0\kappa>0 and g(w)=log⁡C+wg(w)=\log C+w, both weighted parts are integrable on w≥κw\ge\kappa, because (∣log⁡C∣+w)e−ρλw(|\log C|+w)e^{-\rho_{\lambda}w} is bounded. Thus

Jλ(κ)=e−ρλκEPλ,ρλ[e−ρλOλ(κ)(log⁡C+κ+Oλ(κ))].(30)J_{\lambda}(\kappa)=e^{-\rho_{\lambda}\kappa}\mathbb{E}_{P_{\lambda,\rho_{\lambda}}}\left[e^{-\rho_{\lambda}O_{\lambda}(\kappa)}\left(\log C+\kappa+O_{\lambda}(\kappa)\right)\right]. \tag*{(30)}

In particular, if ψλCGF(ρ∗)=log⁡(1/δ)\psi_{\lambda_{\mathrm{CGF}}}(\rho^*)=\log(1/\delta) and log⁡C<1/ρ∗\log C<1/\rho^*, setting κ=1/ρ∗−log⁡C\kappa=1/\rho^*-\log C and O=OλCGF(κ)O=O_{\lambda_{\mathrm{CGF}}}(\kappa) gives

JλCGF(κ)UD(C)=EPλCGF,ρ∗[(1+ρ∗O)e−ρ∗O].(31)\frac{J_{\lambda_{\mathrm{CGF}}}(\kappa)}{U_{\mathrm{D}}(C)} =\mathbb{E}_{P_{\lambda_{\mathrm{CGF}},\rho^*}}\left[(1+\rho^*O)e^{-\rho^*O}\right]. \tag*{(31)}

This identity also applies to unbounded increments.

Under the same assumptions, since 0≤1−(1+x)e−x=∫0xte−t dt≤x2/20\le1-(1+x)e^{-x}=\int_0^x te^{-t}\,dt\le x^2/2 for x≥0x\ge0, it gives

0≤1−JλCGF(κ)UD(C)≤(ρ∗)22EPλCGF,ρ∗[O2].(32)0\le1-\frac{J_{\lambda_{\mathrm{CGF}}}(\kappa)}{U_{\mathrm{D}}(C)} \le\frac{(\rho^*)^2}{2}\mathbb{E}_{P_{\lambda_{\mathrm{CGF}},\rho^*}}[O^2]. \tag*{(32)}

Since JλCGF(κ)≤Vθ∗(C)≤UD(C)J_{\lambda_{\mathrm{CGF}}}(\kappa)\le V_{\theta}^*(C)\le U_{\mathrm{D}}(C), the same bound controls the relative loss against the optimal adaptive strategy. Near-optimality thus follows when (ρ∗)2EPλCGF,ρ∗[O2](\rho^*)^2\mathbb{E}_{P_{\lambda_{\mathrm{CGF}},\rho^*}}[O^2] is small, without a bounded-increment assumption. For Gaussian increments, this tilted second moment is finite. Indeed, tilting preserves normality, the lemma's geometric tail bound gives a finite tilted mean crossing time, and O2≤∑i=1TλCGF(κ)(Δi(λCGF)+)2O^2\le\sum_{i=1}^{T_{\lambda_{\mathrm{CGF}}}(\kappa)}(\Delta_i(\lambda_{\mathrm{CGF}})^+)^2 has finite tilted expectation by independence.

Proof of Proposition 2. For a predictable strategy π\pi, write Mt=δt(Etπ)sM_t=\delta^t(\mathbb{E}^{\pi}_t)^s. Independence of XtX_t from Ft−1\mathcal{F}_{t-1} and the conditional moment bound give

Eθ[Mt∣Ft−1]=Mt−1δEθ[fλt(Xt)s∣Ft−1]≤Mt−1.\mathbb{E}_{\theta}[M_t\mid\mathcal{F}_{t-1}] =M_{t-1}\delta\mathbb{E}_{\theta}[f_{\lambda_t}(X_t)^s\mid\mathcal{F}_{t-1}] \le M_{t-1}.

Thus (Mt)(M_t) is a nonnegative supermartingale with initial value one, and obeys Eθ[MT]≤1\mathbb{E}_{\theta}[M_T]\le1 for T=Tπ(κ)T=T_{\pi}(\kappa). If TT is finite, ETπ≥eκE_T^{\pi}\ge e^{\kappa} by definition, hence Hπ(κ)≤e−sκH_{\pi}(\kappa)\le e^{-s\kappa}.

Under the additional assumptions for λ0\lambda_0, Lemma 1 applies with ρλ0=s\rho_{\lambda_0}=s. Its overshoot satisfies 0≤Oλ0(κ)≤h0\le O_{\lambda_0}(\kappa)\le h, so

Hλ0(κ)=e−sκEPλ0,s[e−sOλ0(κ)]≥e−s(κ+h).H_{\lambda_0}(\kappa)=e^{-s\kappa}\mathbb{E}_{P_{\lambda_0,s}}\left[e^{-sO_{\lambda_0}(\kappa)}\right]\ge e^{-s(\kappa+h)}.

The remaining inequality follows because a fixed parameter is a predictable strategy.

Proof of Proposition 3. The baseline CA⁡(1)\operatorname{CA}(1) is paid in round one, and each increase C dA(a)C\,\mathrm{d}A(a) is paid at its first evidence crossing. Tonelli’s theorem gives (16), since these payments are nonnegative. Integrating the upper crossing bound gives Jinc(π)≤UAJ_{\mathrm{inc}}(\pi) \le U_A for every predictable policy.

Under the lower-bound assumptions, Proposition 2 gives

Jinc(λCGF)≥δCA(1)+e−ρ∗hC∫(1,∞)a−ρ∗ dA(a)≥e−ρ∗hUA.(33)J_{\mathrm{inc}}(\lambda_{\mathrm{CGF}}) \ge\delta C A(1) + e^{-\rho^* h} C \int_{(1,\infty)} a^{-\rho^*}\,\mathrm{d}A(a) \ge e^{-\rho^* h} U_A. \tag*{(33)}

The middle inequality in (4.11) follows because the fixed CGF policy is admissible.

Proof of Proposition 4. Write c=log⁡Cc=\log C and Yt=c+WtY_t=c+W_t, so that Yt=log⁡RtY_t=\log R_t for t≥1t\ge1. For an adaptive strategy, abbreviate Δt(λt)\Delta_t(\lambda_t) as Δt\Delta_t. Let Lt=δteρ∗WtL_t=\delta^t e^{\rho^* W_t}. Fix any strategy. Since λt\lambda_t is Ft−1\mathcal{F}_{t-1}-measurable, XtX_t is independent of Ft−1\mathcal{F}_{t-1}, and ψλ(ρ∗)≤log⁡(1/δ)\psi_\lambda(\rho^*)\le\log(1/\delta) for every λ\lambda (§ 4.1),

Eθ[δeρ∗Δt∣Ft−1]=δeψλt(ρ∗)≤1,\mathbb{E}_{\theta}\left[\delta e^{\rho^* \Delta_t}\mid\mathcal{F}_{t-1}\right]=\delta e^{\psi_{\lambda_t}(\rho^*)}\le1,

so (Lt)(L_t) is a nonnegative supermartingale with L0=1L_0=1. Thus Eθ[Lτ]≤1\mathbb{E}_{\theta}[L_\tau]\le1 for any stopping time τ≥1\tau\ge1. Since ze−ρ∗z≤1/(eρ∗)z e^{-\rho^* z}\le1/(e\rho^*) for all zz,

GτD=Cρ∗LτYτe−ρ∗Yτ1{τ<∞}≤Cρ∗Lτ1{τ<∞}eρ∗.G_\tau^{\mathrm{D}}=C^{\rho^*}L_\tau Y_\tau e^{-\rho^*Y_\tau}\mathbf{1}\{\tau<\infty\}\le\frac{C^{\rho^*}L_\tau\mathbf{1}\{\tau<\infty\}}{e\rho^*}.

The positive part of the utility is therefore integrable. If its negative part is not integrable, the upper bound is immediate. Otherwise, set d=Eθ[δτ1{τ<∞}]∈[0,δ]d=\mathbb{E}_{\theta}[\delta^\tau\mathbf{1}\{\tau<\infty\}]\in[0,\delta]. If d=0d=0, the utility is zero. For d>0d>0, Jensen’s inequality under the probability measure with density δτ1{τ<∞}/d\delta^\tau\mathbf{1}\{\tau<\infty\}/d gives

Eθ[GτD]≤dρ∗log⁡(Eθ[δτRτρ∗1{τ<∞}]d)≤dc−dlog⁡dρ∗,\mathbb{E}_{\theta}[G_\tau^{\mathrm{D}}]\le\frac{d}{\rho^*}\log\left(\frac{\mathbb{E}_{\theta}[\delta^\tau R_\tau^{\rho^*}\mathbf{1}\{\tau<\infty\}]}{d}\right)\le dc-\frac{d\log d}{\rho^*},

where the second inequality uses Eθ[δτRτρ∗1{τ<∞}]=Cρ∗Eθ[Lτ1{τ<∞}]≤Cρ∗\mathbb{E}_{\theta}[\delta^\tau R_\tau^{\rho^*}\mathbf{1}\{\tau<\infty\}]=C^{\rho^*}\mathbb{E}_{\theta}[L_\tau\mathbf{1}\{\tau<\infty\}]\le C^{\rho^*}. Enlarging the range to 0<d≤10<d\le1 and maximizing gives d=min⁡(1,Cρ∗/e)d=\min(1,C^{\rho^*}/e) and the bound UD(C)U_{\mathrm{D}}(C), which is positive and also covers d=0d=0.

Under the additional assumptions for the lower bound, set κ=1/ρ∗−c\kappa=1/\rho^*-c and O=OλCGF(κ)O=O_{\lambda_{\mathrm{CGF}}}(\kappa). The overshoot identity (A.4) gives

JλCGF(κ)UD(C)=EPλCGF,ρ∗[(1+ρ∗O)e−ρ∗O]≥(1+ρ∗h)e−ρ∗h=q,\frac{J_{\lambda_{\mathrm{CGF}}}(\kappa)}{U_{\mathrm{D}}(C)}=\mathbb{E}_{P_{\lambda_{\mathrm{CGF}},\rho^*}}\left[(1+\rho^*O)e^{-\rho^*O}\right]\ge(1+\rho^*h)e^{-\rho^*h}=q,

because 0≤O≤h0\le O\le h under the tilted law and (1+x)e−x(1+x)e^{-x} is nonincreasing for x≥0x\ge0. The threshold rule is admissible, so JλCGF(κ)≤V⋆(C)J_{\lambda_{\mathrm{CGF}}}(\kappa)\le V^\star(C).

Additional Results

Bellman and Snell characterizations

Once the agent has chosen a reward process, they face a joint optimization over parameter selection and stopping, which can be framed as a stochastic control problem. Let v(r,t)v(r,t) denote the agent’s utility from collecting reward rr at time tt. For deferred rewards with Rt>0R_t>0 for t≥1t\ge1, let Wt=log⁡(Rt/C)W_t=\log(R_t/C) and define, after round t≥1t\ge1 while the contract is active,

F(w,t)=sup⁡π,τ≥tEθ,π[v(Rτ∧σ,τ∧σ)∣Wt=w].(34)F(w,t)=\sup_{\pi,\tau\ge t}\mathbb{E}_{\theta,\pi}\left[v(R_{\tau\wedge\sigma},\tau\wedge\sigma)\mid W_t=w\right]. \tag*{(34)}

Here π\pi ranges over predictable parameter policies and τ\tau over almost surely finite stopping times. The reward and time arguments are frozen when the principal stops at σ\sigma.

The Bellman equation characterizes FF under a Markov assumption: the conditional law of Wt+1W_{t+1} given Ft\mathcal{F}_t and λt+1\lambda_{t+1} depends only on (Wt,t,λt+1)(W_t,t,\lambda_{t+1}), and the principal stops at the first entry of (Wt,t)(W_t,t) into some termination set DD, that is, σ=inf⁡{t:(Wt,t)∈D}\sigma=\inf\{t:(W_t,t)\in D\}. Examples include a deterministic horizon σ=N\sigma=N, a rule that stops once the reward reaches a target or leaves an interval, a time-varying boundary σ=inf⁡{t:Wt≥b(t)}\sigma=\inf\{t:W_t\ge b(t)\}, and a growth-rate rule σ=inf⁡{t:Wt/t≤γ}\sigma=\inf\{t:W_t/t\le\gamma\}. A path-dependent rule, or one driven by the principal’s private information (such as a budget), is not Markov in (Wt,t)(W_t,t). Augmenting the state can restore the Markov property, but at the cost of a larger state, or of state variables the agent cannot observe.

Proposition 5 (Bellman characterization). Suppose the Markov assumption holds and the stopped payoffs are uniformly integrable over admissible policies and stopping times. For t≥1t\ge1 on {t<σ}\{t<\sigma\},

F(w,t)=max⁡{v(Cew,t),sup⁡λ∈ΛEθ[F(Wt+1,t+1)∣Wt=w,λt+1=λ]}.(35)F(w,t)=\max\left\{v(Ce^w,t),\sup_{\lambda\in\Lambda}\mathbb{E}_{\theta}[F(W_{t+1},t+1)\mid W_t=w,\lambda_{t+1}=\lambda]\right\}. \tag*{(35)}

with F(w,t)=v(Cew,t)F(w,t)=v(Ce^w,t) at termination. The value at entry is sup⁡λ1∈ΛEθ[F(W1,1)]\sup_{\lambda_1\in\Lambda}\mathbb{E}_{\theta}[F(W_1,1)], since the agent has already opted in. If the supremum admits a measurable maximizing selector and the resulting first stopping time is almost surely finite, selecting that maximizer while the continuation value exceeds the stopping payoff is optimal.

This follows from the principle of optimality and optional stopping under uniform integrability [35 Chapter 3]. The selector and finiteness conditions are needed only for an optimal strategy to exist, not for the Bellman equation itself. The numerical benchmarks use bounded action sets with integrable envelopes for the log-increments, which ensure uniform integrability of discounted log payoffs. Incremental rewards require a state that also records past payments, since discounted receipts depend on when each payment was made; for milestone contracts, current evidence and the next unpaid milestone suffice (Appendix § 10).

The Bellman equation is rarely solvable in closed form. Even for Gaussian data it is a non-linear integral equation, coupled with a state-dependent optimization over λ\lambda, and must be solved numerically. In practice, however, e-processes are seldom optimized round by round. One either commits in advance to a predictable plug-in rule λt=gt(X1,…,Xt−1)\lambda_t=g_t(X_1,\ldots,X_{t-1}) [50] or mixes over λ\lambda with a data-free distribution. Either way, the reward process is determined by the data, and only the stopping decision remains. For t≥1t\ge1, write Yt=v(Rt∧σ,t∧σ)Y_t=v(R_{t\wedge\sigma},t\wedge\sigma) and define its Snell envelope, the smallest supermartingale that dominates (Yt)(Y_t), by

St=ess sup⁡τ≥t, τ<∞ a.s.Eθ[Yτ∣Ft].(36)S_t=\mathop{\operatorname{ess\,sup}}_{\tau\ge t,\ \tau<\infty\ \mathrm{a.s.}}\mathbb{E}_{\theta}[Y_\tau\mid\mathcal{F}_t]. \tag*{(36)}

Theorem 1 (Snell characterization). Suppose σ≥1\sigma\ge1 is an (Ft)(\mathcal{F}_t)-stopping time and Eθ[sup⁡t≥1∣Yt∣]<∞\mathbb{E}_{\theta}[\sup_{t\ge1}|Y_t|]<\infty. If τ∗=inf⁡{t≥1:Yt=St}∧σ\tau^*=\inf\{t\ge1:Y_t=S_t\}\wedge\sigma is almost surely finite, it attains sup⁡1≤τ<∞ a.s.Eθ[Yτ]\sup_{1\le\tau<\infty\ \mathrm{a.s.}}\mathbb{E}_{\theta}[Y_\tau].

Proof. For t≥1t\ge1, the recursion St=max⁡{Yt,Eθ[St+1∣Ft]}S_t=\max\{Y_t,\mathbb{E}_{\theta}[S_{t+1}\mid\mathcal{F}_t]\} makes (St∧τ∗)t≥1(S_{t\wedge\tau^*})_{t\ge1} a uniformly integrable martingale. Thus Eθ[Yτ∗]=Eθ[S1]\mathbb{E}_{\theta}[Y_{\tau^*}]=\mathbb{E}_{\theta}[S_1], while domination and optional stopping give Eθ[Yτ]≤Eθ[S1]\mathbb{E}_{\theta}[Y_\tau]\le\mathbb{E}_{\theta}[S_1] for every admissible τ≥1\tau\ge1.

The integrability condition can be weakened to uniform integrability of {Yτ:τ<∞ a.s.}\{Y_\tau:\tau<\infty\ \mathrm{a.s.}\}, the usual class-D assumption [35 Chapter 1]. For a fixed parameter with Eθ∣Δ1(λ)∣<∞\mathbb{E}_{\theta}|\Delta_1(\lambda)|<\infty, it holds for discounted log utility, since sup⁡t≥1∣Yt∣≤∣log⁡C∣+∑i≥1δi∣Δi(λ)∣\sup_{t\ge1}|Y_t|\le|\log C|+\sum_{i\ge1}\delta^i|\Delta_i(\lambda)|, whose expectation is at most ∣log⁡C∣+δEθ∣Δ1(λ)∣/(1−δ)|\log C|+\delta\mathbb{E}_{\theta}|\Delta_1(\lambda)|/(1-\delta). For conditions under which the expected log utility of an e-value is well-defined, see De Roos et al. [16 Lemma 2.5].

Unlike Proposition 5, Theorem 1 does not require σ\sigma to be Markov, only an (Ft)(\mathcal{F}_t)-stopping time. For instance, a principal who stops once the reward falls more than a fraction ϵ\epsilon below its running maximum, σ=inf⁡{t≥1:Rt≤(1−ϵ)max⁡s≤tRs}\sigma= \inf\{t \ge1 : R_t \le(1-\epsilon)\max_{s \le t} R_s\}, uses a path-dependent rule that cannot be written in terms of (Wt,t)(W_t,t). The Snell envelope thus trades per-round optimization of λt\lambda_t for more general principal stopping rules. Both characterizations generally require computing a value function, and both also apply to other adapted payoffs, including cumulative discounted incremental receipts.

Section 4.2 also allows τ=∞\tau= \infty, with payoff zero. For discounted log utility under the integrability conditions above, this changes neither value. Indeed, sup⁡t∣Yt∣\sup_t |Y_t| is then integrable and, by the strong law of large numbers, Yt→0Y_t \to0 almost surely on {σ=∞}\{\sigma= \infty\}, so dominated convergence gives Eθ[Yτ∧n]→Eθ[Yτ]\mathbb{E}_{\theta}[Y_{\tau\wedge n}] \to\mathbb{E}_{\theta}[Y_\tau] for every stopping time τ\tau.

Exact fixed-parameter stopping value

For a fixed parameter, the optimal stopping boundary has an exact series representation [13, 33].

Proposition 6. Consider deferred rewards R0=0R_0 = 0 and Rt=CEtR_t = C E_t for t≥1t \ge1, with σ=∞\sigma= \infty and utility GtD=δtlog⁡RtG_t^{\mathrm{D}} = \delta^t \log R_t, with G∞D=0G_\infty^{\mathrm{D}} = 0. Fix λ\lambda with Eθ∣Δ1(λ)∣<∞\mathbb{E}_{\theta}|\Delta_1(\lambda)| < \infty. Set Sn(λ)=∑i=1nΔi(λ)S_n(\lambda) = \sum_{i=1}^{n} \Delta_i(\lambda) and

β⋆(λ)=∑n=1∞δnnEθ[(Sn(λ))+],τβ(λ)=inf⁡{t≥1:log⁡Rt(λ)≥β⋆(λ)}.\beta^\star(\lambda) = \sum_{n=1}^{\infty} \frac{\delta^n}{n}\mathbb{E}_{\theta}[(S_n(\lambda))^+], \qquad\tau_{\beta}(\lambda) = \inf\{t \ge1 : \log R_t(\lambda) \ge\beta^\star(\lambda)\}.

The rule τβ(λ)\tau_{\beta}(\lambda) attains the supremum of expected utility over all stopping times τ≥1\tau\ge1 at this parameter, without a positive-drift assumption.

Under the assumptions of Proposition 6, let NδN_\delta be independent of the observations, with Pr⁡(Nδ=n)=(1−δ)δn\Pr(N_\delta= n) = (1-\delta)\delta^n for n=0,1,…n = 0,1,\ldots\Pr(N_\delta \ge t) = \delta^t$, discounting is equivalent to crossing before this deadline:

Hπ(κ)=Pr⁡θ,π(max⁡0≤t≤NδWtπ≥κ).(37)H_{\pi}(\kappa) = \Pr_{\theta,\pi}\left(\max_{0 \le t \le N_\delta} W_t^\pi\ge\kappa\right). \tag*{(37)}

For a fixed parameter set Mλ=max⁡0≤t≤NδSt(λ)M_\lambda= \max_{0 \le t \le N_\delta} S_t(\lambda). The boundary is the expected record high, β⋆(λ)=Eθ[Mλ]\beta^\star(\lambda) = \mathbb{E}_{\theta}[M_\lambda]. Since the agent must run the first trial, the exact value is

Vλ(C):=sup⁡τ≥1Eθ[GτD]=δEθ[(log⁡C+Δ1(λ)+Mλ′−β⋆(λ))+],(38)V_\lambda(C) := \sup_{\tau\ge1}\mathbb{E}_{\theta}[G_\tau^{\mathrm{D}}] = \delta\mathbb{E}_{\theta}\left[(\log C + \Delta_1(\lambda) + M'_\lambda- \beta^\star(\lambda))^+\right], \tag*{(38)}

where Mλ′M'_\lambda is an independent copy of MλM_\lambda, independent of Δ1(λ)\Delta_1(\lambda). When log⁡C≤β⋆(λ)\log C \le\beta^\star(\lambda), this reduces to Eθ[(log⁡C+Mλ−β⋆(λ))+]\mathbb{E}_{\theta}[(\log C + M_\lambda- \beta^\star(\lambda))^+]; this includes the experiments with C=1C = 1.

Proof of Proposition 6. Write x=log⁡Cx = \log C, so the payoff at time t≥1t \ge1 is δt(x+St)\delta^t(x + S_t). Use the independent geometric deadline NδN_\delta from (B.4) and let M=max⁡0≤k≤NδSkM = \max_{0 \le k \le N_\delta} S_k. Since M≤∑k≤NδΔk+M \le\sum_{k \le N_\delta} \Delta_k^+, we have E[M]≤E[Nδ]E[Δ1+]=δ1−δE[Δ1+]<∞\mathbb{E}[M] \le\mathbb{E}[N_\delta]\mathbb{E}[\Delta_1^+] = \frac{\delta}{1-\delta}\mathbb{E}[\Delta_1^+] < \infty.

Spitzer’s identity [46] gives E[max⁡0≤k≤nSk]=∑k=1nE[Sk+]/k\mathbb{E}[\max_{0 \le k \le n} S_k] = \sum_{k=1}^{n}\mathbb{E}[S_k^+]/k. Averaging over NδN_\delta and exchanging the order of summation,

E[M]=∑n≥0(1−δ)δn∑k=1nE[Sk+]k=∑k≥1E[Sk+]k∑n≥k(1−δ)δn=∑k≥1δkkE[Sk+]=β⋆.\mathbb{E}[M] = \sum_{n \ge0}(1-\delta)\delta^n\sum_{k=1}^{n}\frac{\mathbb{E}[S_k^+]}{k} = \sum_{k \ge1}\frac{\mathbb{E}[S_k^+]}{k}\sum_{n \ge k}(1-\delta)\delta^n = \sum_{k \ge1}\frac{\delta^k}{k}\mathbb{E}[S_k^+] = \beta^\star.

Define the candidate value function

U(y):=E[(y+M−β⋆)+],y∈R.U(y) := \mathbb{E}[(y + M - \beta^\star)^+], \qquad y \in\mathbb{R}.

Step 1: U dominates the payoff, with equality above β⋆\beta^\star. By Jensen’s inequality, U(y)≥(y+M−β⋆)+≥yU(y) \ge(y + M - \beta^\star)^+ \ge y. If y≥β⋆y \ge\beta^\star then y+M−β⋆≥0y + M - \beta^\star\ge0 (as M≥0M \ge0), so U(y)=y+EM−β⋆=yU(y) = y + \mathbb{E}M - \beta^\star= y.

Step 2: a one-step recursion. By the memoryless property of the geometric law, with probability 1−δ1-\delta we have Nδ=0N_\delta= 0 and M=0M = 0, while conditional on Nδ≥1N_\delta\ge1 we have M=dmax⁡(0,Δ1+M′)M \mathrel{\overset{d}{=}} \max(0,\Delta_1 + M'), where M′=dMM' \mathrel{\overset{d}{=}} M is independent of Δ1\Delta_1. Hence

U(y)=(1−δ)(y−β⋆)++δE[(y−β⋆+max⁡(0,Δ1+M′))+].U(y) = (1-\delta)(y-\beta^\star)^+ + \delta\mathbb{E}\left[(y-\beta^\star+\max(0,\Delta_1+M'))^+\right].

If y≤β⋆y \le\beta^\star, then (y−β⋆+max⁡(0,z))+=(y−β⋆+z)+(y-\beta^\star+\max(0,z))^+ = (y-\beta^\star+z)^+ for every zz (both vanish when z≤0z \le0), so U(y)=δE[U(y+Δ1)]U(y) = \delta\mathbb{E}[U(y+\Delta_1)]. If y≥β⋆y \ge\beta^\star, then (y−β⋆+max⁡(0,z))+≥(y−β⋆+z)+(y-\beta^\star+\max(0,z))^+ \ge(y-\beta^\star+z)^+, so U(y)≥δE[U(y+Δ1)]U(y) \ge\delta\mathbb{E}[U(y+\Delta_1)]. Thus (δtU(x+St))t≥1(\delta^tU(x+S_t))_{t\ge1} is a nonnegative supermartingale, and a martingale from time one up to the hitting time T:=inf⁡{t≥1:x+St≥β⋆}T := \inf\{t \ge1 : x + S_t \ge\beta^\star\}.

Step 3: upper bound. For any stopping time τ≥1\tau\ge1, Step 1 and optional stopping from time one (via Fatou’s lemma) give

E[GτD]≤E[δτU(x+Sτ)1{τ<∞}]≤δE[U(x+Δ1)],\mathbb{E}[G_\tau^{\mathrm{D}}] \le\mathbb{E}[\delta^\tau U(x+S_\tau)1\{\tau<\infty\}] \le\delta\mathbb{E}[U(x+\Delta_1)],

where the stopped term is defined as zero on {τ=∞}\{\tau=\infty\}.

Step 4: attainment. The stopped process from Step 2 is an integrable martingale from time one at every finite horizon. On {T≤n}\{T \le n\}, Step 1 gives U(x+ST)=x+STU(x+S_T)=x+S_T. Consequently, for n≥1n \ge1,

δE[U(x+Δ1)]=E[δT(x+ST)1{T≤n}]+δnE[U(x+Sn)1{T>n}].\delta\mathbb{E}[U(x+\Delta_1)] = \mathbb{E}[\delta^T(x+S_T)1\{T\le n\}] + \delta^n\mathbb{E}[U(x+S_n)1\{T>n\}].

On {T>n}\{T>n\}, x+Sn<β⋆x+S_n<\beta^\star, so monotonicity of UU bounds the second term by δnU(β⋆)=δnβ⋆→0\delta^nU(\beta^\star)=\delta^n\beta^\star\to0. The first term converges to E[GTD]\mathbb{E}[G_T^{\mathrm{D}}] by monotone convergence, since x+ST≥β⋆≥0x+S_T\ge\beta^\star\ge0 at every finite crossing. Thus TT attains the upper bound δE[U(x+Δ1)]\delta\mathbb{E}[U(x+\Delta_1)], proving (B.5). If x≤β⋆x\le\beta^\star, Step 2 also gives δE[U(x+Δ1)]=U(x)\delta\mathbb{E}[U(x+\Delta_1)]=U(x).

A limitation of the adjuster construction

Here we observe that not all nondecreasing ee-processes come from adjusters.

Proposition 7. There exists a bounded nondecreasing ee-process (Rt)t≥0(R_t)_{t\ge0} with R0=0R_0=0 for a singleton null {P}\{P\} that cannot be represented as

Rt=A(Et⋆),Et⋆=1∨max⁡1≤s≤tEs,t≥1,R_t=A(E_t^\star), \qquad E_t^\star=1\vee\max_{1\le s\le t}E_s,\qquad t\ge1,

for any deterministic adjuster AA and any ee-process (Et)(E_t) for PP on the same filtration. In fact, no such pair satisfies Rt≤A(Et⋆)R_t\le A(E_t^\star) almost surely for every t≥1t\ge1.

Proof. Let XX be uniform on {0,1,2}\{0,1,2\} under PP, with F0\mathcal{F}_0 trivial and Ft=σ(X)\mathcal{F}_t=\sigma(X) for t≥1t\ge1. Set R0=0R_0=0 and Rt=XR_t=X for t≥1t\ge1. This process is bounded and nondecreasing. For every stopping time τ\tau, 0≤Rτ≤X0\le R_\tau\le X, so EP[Rτ]≤EP[X]=1\mathbb{E}_P[R_\tau]\le\mathbb{E}_P[X]=1. Thus (Rt)(R_t) is an ee-process and defines a null-deterring incremental contract with entry cost C=1C=1.

Suppose X=A(1∨E1)X=A(1\vee E_1) almost surely. Since P(X=0)>0P(X=0)>0, nonnegativity and monotonicity of AA force A(1)=0A(1)=0. Moreover, every adjuster satisfies A(y)≤yA(y)\le y for y≥1y\ge1, because

1≥∫1∞A(u)u2 du≥∫y∞A(y)u2 du=A(y)y.1\ge\int_1^\infty\frac{A(u)}{u^2}\,\mathrm{d}u\ge\int_y^\infty\frac{A(y)}{u^2}\,\mathrm{d}u=\frac{A(y)}{y}.

On {X=1}\{X=1\}, we must have E1>1E_1>1: otherwise A(1∨E1)=A(1)=0A(1\vee E_1)=A(1)=0. On {X=2}\{X=2\}, we have E1>1E_1>1 and 2=A(E1)≤E12=A(E_1)\le E_1. On {X=0}\{X=0\}, nonnegativity gives E1≥0E_1\ge0. Hence E1≥XE_1\ge X almost surely, with strict inequality on an event of probability 1/31/3. It follows that EP[E1]>EP[X]=1E_P[E_1] > E_P[X] = 1, contradicting the e-process property.

Finally, if X≤A(1∨E1)X \le A(1 \vee E_1) almost surely, the adjuster guarantee in Proposition 1 gives 1=EP[X]≤EP[A(1∨E1)]≤11 = E_P[X] \le E_P[A(1 \vee E_1)] \le1. Equality of expectations and the assumed domination force X=A(1∨E1)X = A(1 \vee E_1) almost surely, which was just ruled out.

Comparison to the Sequential Contract of Bates et al.

As we discussed in the introduction, Bates et al. [5] also provided a sequential version of their one-shot contract. We state it here:

Bates et al. [5] multi-round contract

  1. The initial license value is L(0)=0L(0) = 0.

  2. For t=1,2,…,Tt = 1, 2, \ldots, T: (a) The agent chooses to withdraw P(t)∈(0,L(t−1))P(t) \in(0, L(t-1)) from their license, or continue without a withdrawal. (b) The agent may pay CtC_t to run stage tt of the trial, in which case: i. The agent selects an update function ftf_t from the menu Ut\mathcal{U}_t ii. The trial produces evidence Zt∼PθZ_t \sim P_\theta iii. The license value is updated as L(t)=(L(t−1)+Ct−P(t))⋅ft(Zt)L(t) = (L(t-1) + C_t - P(t)) \cdot f_t(Z_t). (c) Otherwise, license value is updated as L(t)=L(t−1)−P(t)L(t) = L(t-1) - P(t).

While their motivation is similar to ours, the two contracts differ in their treatment of costs, payments, horizons, and the agent’s optimization problem. Let us draw attention to each of these features in turn.

Per-round costs and payments. First, the contract above uses per-round costs, and allows the agent to withdraw money from the license won so far. This can be beneficial for some applications, as it makes the cost structure explicit and part of the agent’s strategy. In particular, the contract is well-suited to settings in which continued experimentation requires a sequence of distinct investments and the agent actively manages both investment and consumption.

We instead model CC as the up-front cost of mounting the research program. Any costs of delay or continuation are represented through the agent’s utility v(Rt,t)v(R_t,t) rather than through additional payments in the contract. In other words, per-round costs are possible in our setting, though they are modeled only on the agent’s side—they are not part of the structure of the contract itself.

Finite-horizon and stopping times. Per-round costs make an unbounded horizon more delicate. The agent’s net-profit supermartingale need not be nonnegative, so optional stopping does not apply to arbitrary stopping times without additional admissibility conditions. (For instance, if cumulative investment is bounded by BB, the net profit is bounded below by −B-B, and optional stopping can be applied to the shifted process.) Consequently, Bates et al. work with a finite number TT of possible rounds. This fixed horizon allows the problem to be solved by backwards induction. Our framework, meanwhile, allows an unbounded horizon and introduces stopping rules for both the principal and the agent.

The agent’s best response. Both we and Bates et al. study the agent’s best response, and both do so via dynamic programming. Bates et al. formulate the finite-horizon contract as a Markov decision process whose state includes the current time and license value. At each stage, the agent chooses whether to stop and, if continuing, which e-value update to use. Our Bellman formulation is analogous, but permits an unbounded horizon, an explicit principal stopping rule, and general time-dependent utility. For a fixed reward process, we additionally characterize the optimal stopping time through its Snell envelope.

Overall, the two frameworks should be viewed as complementary extensions of the single-shot contract. Bates et al. emphasize a finite sequence of costly investments with flexible withdrawals and reinvestment. We emphasize a potentially indefinite stream of evidence, a fixed entry cost, principal and agent stopping, and approximations to the optimal stopping rule under different levels of agent information.

Experimental Details

Code to replicate the experiments is available at https://github.com/bchugg/statcontracts.

For Figure 2, we use Bernoulli observations and the milestone allocations in § 5.1. For the betting multiplier fλ(X)=1+λ(X−m)f_{\lambda}(X) = 1 + \lambda(X - m), the log-increment CGF is

Figure 2. The utility of various strategies under the incremental reward schedule relative to the Bellman benchmark (B.2). The low- and high-milestone adjusters allocate 80% of their null-payout budget (see § 3) to the first and last milestone, respectively. The plug-in is that of § 4.3 and uses the clipped sample mean. Standard errors of the ratios are below 0.0016, so error bars are omitted.

ψλ(s)=log⁡[p(1+λ(1−m))s+(1−p)(1−λm)s].\psi_{\lambda}(s) = \log\left[p(1 + \lambda(1 - m))^{s} + (1 - p)(1 - \lambda m)^{s}\right].

We compute the CGF-optimal parameter by numerically solving sup⁡λ∈Λψλ(ρ∗)=log⁡(1/δ)\sup_{\lambda\in\Lambda} \psi_{\lambda}(\rho^{*}) = \log(1/\delta); a prescribed fixed bet uses its own root ρλ\rho_{\lambda}. All reported Bernoulli settings satisfy δp>m\delta p > m, so ρ∗∈(0,1)\rho^{*} \in(0,1) and an interior CGF-optimal bet exists. The mean plug-in starts at λ1=0.1\lambda_{1} = 0.1, then uses p~t=min⁡{X‾t,0.99}\widetilde{p}_{t} = \min\{\overline{X}_{t}, 0.99\} and λt+1=(p~t−m)+/(m(1−m))\lambda_{t+1} = (\widetilde{p}_{t} - m)^{+}/(m(1 - m)), continuing to sample when the bet is zero.

A policy’s value is Jinc(π)=∑jrjHπ(log⁡aj)J_{\mathrm{inc}}(\pi) = \sum_{j} r_{j}H_{\pi}(\log a_{j}). For fixed bets we compute these crossing values recursively and search numerically for the best parameter. The adaptive benchmark solves a discretized Bellman equation whose state records current evidence and the next unpaid milestone. We evaluate the adaptive and plug-in policies on 500,000 simulated paths each, and the plotted ratios have standard errors below 0.0016. Simulation horizons make the omitted expected receipts smaller than 10−810^{-8}.

Figure 3 uses the same Bernoulli family and a Gaussian family. For X∼N(θ,ς2)X \sim N(\theta, \varsigma^{2}) with θ>0\theta> 0, use fλ(X)=exp⁡(λX−λ2/8)f_{\lambda}(X) = \exp(\lambda X - \lambda^{2}/8), λ≥0\lambda\ge0. Then

μ(λ)=λθ−λ2/8,ψλ(s)=sμ(λ)+s2λ2ς2/2.\mu(\lambda) = \lambda\theta- \lambda^{2}/8,\qquad\psi_{\lambda}(s) = s\mu(\lambda) + s^{2}\lambda^{2}\varsigma^{2}/2.

Maximizing the drift gives λGRO=4θ\lambda_{\mathrm{GRO}} = 4\theta. Maximizing the CGF over λ\lambda and solving the moment-root equation gives

ρ∗=log⁡(1/δ)2(θ2+2ς2log⁡(1/δ)),λCGF=4θ+8ς2log⁡(1/δ)θ.\rho^{*} = \frac{\log(1/\delta)}{2(\theta^{2} + 2\varsigma^{2}\log(1/\delta))},\qquad\lambda_{\mathrm{CGF}} = 4\theta+ \frac{8\varsigma^{2}\log(1/\delta)}{\theta}.

The experiments use ς=0.5\varsigma= 0.5.

The Gaussian plug-in starts at λ1=1\lambda_{1} = 1, then uses λ^t=clip⁡(4X‾t,0,20)\widehat{\lambda}_{t} = \operatorname{clip}(4\overline{X}_{t}, 0, 20) and estimated drift λ^tX‾t−λ^t2/8\widehat{\lambda}_{t}\overline{X}_{t} - \widehat{\lambda}_{t}^{2}/8, with λt+1=λ^t\lambda_{t+1} = \widehat{\lambda}_{t}. For the Bernoulli plug-in’s deferred threshold, the estimated drift is KL⁡(p~t∥m)\operatorname{KL}(\widetilde{p}_{t}\mathbin\Vert m) when p~t>m\widetilde{p}_{t} > m, and zero otherwise. This uses the clipped fitted law, a variant of the empirical drift estimate in § 4.3.

The Hoeffding baseline uses fλ(X)=exp⁡{λ(X−m)−λ2/8}f_{\lambda}(X) = \exp\{\lambda(X - m) - \lambda^{2}/8\} with λ=4(p−m)\lambda= 4(p - m) and the drift threshold corresponding to μ=2(p−m)2\mu= 2(p - m)^{2}. By Pinsker’s inequality, its drift 2(p−m)22(p - m)^{2} is at most the betting drift KL⁡(p∥m)\operatorname{KL}(p\mathbin\Vert m). For patient agents this gap carries over to utility: with C=1C = 1 and ρλ≈log⁡(1/δ)/μ(λ)\rho_{\lambda} \approx\log(1/\delta)/\mu(\lambda), the maximized value (20) is approximately μ(λ)/(elog⁡(1/δ))\mu(\lambda)/(e\log(1/\delta)), which is linear in the drift. The Hoeffding baseline should therefore earn about the fraction 2(p−m)2/KL⁡(p∥m)2(p - m)^{2}/\operatorname{KL}(p\mathbin\Vert m) of the value of the drift rule with the betting reward. At δ=0.995\delta= 0.995, this fraction falls from 0.998 at p=0.55p = 0.55 to 0.934 at p=0.8p = 0.8, and the simulated ratios agree with it to within 0.003.

We obtain the deferred Bellman benchmark by policy iteration on a log-evidence grid, averaging over Gaussian increments by quadrature and summing over the two Bernoulli outcomes exactly. Each policy and the benchmark are evaluated on 20,000 simulated paths under the original observation law. All paths run for at least one trial and are capped at N=2500N = 2500, with payoff δτ∧Nlog⁡Rτ∧N\delta^{\tau\wedge N} \log R_{\tau\wedge N} even if the threshold was not reached. The ratios therefore have sampling uncertainty and Bellman discretization error.

For the large-overshoot example in § 5.2, we compare λCGF≈54.4951\lambda_{\mathrm{CGF}} \approx54.4951 with log-evidence threshold 1/ρ∗≈1.1719361/\rho^{*} \approx1.171936 against λ=15.5\lambda= 15.5 with exact threshold β∗(15.5)≈0.301722\beta^{*}(15.5) \approx0.301722. The error relative to the uncapped values is below 2⋅10−252 \cdot10^{-25}.

References

  1. [1]Adobe. Experimentation panel. https://experienceleague.adobe.com/en/docs/analytics-platform/using/cja-workspace/panels/experimentation, September 2026. Adobe Customer Journey Analytics documentation, last updated 28 Sep 2026, accessed 2026-10-14.
  2. [2]Ander Artola Velasco, Stratis Tsirtsis, and Manuel Gomez-Rodriguez. Optimizing social utility in sequential experiments. In Advances in Neural Information Processing Systems, volume 39, 2026. To appear.
  3. [4]Stephen Bates, Michael I. Jordan, Michael Sklar, and Jake A. Soloff. Incentive-theoretic Bayesian inference for collaborative science. arXiv preprint arXiv:2307.03748, 2023.arxiv.org/abs/2307.03748
  4. [5]Stephen Bates, Michael I. Jordan, Michael Sklar, and Jake A. Soloff. Principal-agent hypothesis testing. Journal of the American Statistical Association, 2026.arxiv.org/abs/2205.06812
  5. [6]Yatong Chen, Safwan Hossain, and Yiling Chen. Strategic hypothesis testing. In Advances in Neural Information Processing Systems, volume 38, pages 162375–162402. Curran Associates, Inc., 2025.
  6. [7]Yurong Chen, Zhaohua Chen, Xiaotie Deng, and Zhiyi Huang. Are bounded contracts learnable and approximately optimal? In Proceedings of the 25th ACM Conference on Economics and Computation, pages 315–344. Association for Computing Machinery, 2024.arxiv.org/abs/2402.14486
  7. [8]Yo Joong Choe and Aaditya Ramdas. Combining evidence across filtrations. Journal of the Royal Statistical Society Series B: Statistical Methodology, 88(4):1492–1514, 2026.arxiv.org/abs/2402.09698
  8. [9]Ben Chugg, Michael I. Jordan, and Aaditya Ramdas. Strategic multi-agent resource allocation via statistical contracts. arXiv preprint arXiv:2610.18336, 2026.arxiv.org/abs/2610.18336
  9. [10]Ben Chugg, Aaditya Ramdas, and Peter Grünwald. E-values as statistical evidence: A comparison to Bayes factors, likelihoods, and p-values. Synthese, 208(3):130, 2026.arxiv.org/abs/2603.24421
  10. [11]Eugenio Clerico, Tobias Wegel, Iskander Azangulov, and Patrick Rebeshini. Time-sensitive anytime-valid testing. In Advances in Neural Information Processing Systems, volume 39, 2026. To appear.arxiv.org/abs/2605.06521
  11. [12]Alon Cohen, Argyrios Deligkas, and Moran Koren. Learning approximately optimal contracts. In International Symposium on Algorithmic Game Theory, pages 331–346. Springer, 2022.
  12. [13]D. A. Darling, T. Liggett, and H. M. Taylor. Optimal stopping for partial sums. The Annals of Mathematical Statistics, 43(4):1363–1368, 1972.DOI
  13. [14]A. Philip Dawid, Steven de Rooij, Peter Grünwald, Wouter M. Koolen, Glenn Shafer, Alexander Shen, Nikolai Vereshchagin, and Vladimir Vovk. Probability-free pricing of adjusted American lookbacks. arXiv preprint arXiv:1108.4113, 2011.arxiv.org/abs/1108.4113
  14. [15]A. Philip Dawid, Steven de Rooij, Glenn Shafer, Alexander Shen, Nikolai Vereshchagin, and Vladimir Vovk. Insuring against loss of evidence in game-theoretic probability. Statistics & Probability Letters, 81(1):157–162, 2011.
  15. [16]Dante de Roos, Ben Chugg, Peter Grünwald, and Aaditya Ramdas. Rao–Blackwellized e-variables. Biometrika, 2026. Advance article.arxiv.org/abs/2512.16759
  16. [17]Paul Dütting, Michal Feldman, and Inbal Talgam-Cohen. Algorithmic contract theory: A survey. Foundations and Trends® in Theoretical Computer Science, 16(3–4):211–411, 2024.arxiv.org/abs/2412.16384
  17. [18]Dino Gerardi and Lucas Maestri. A principal-agent model of sequential testing. Theoretical Economics, 7(3):425–463, 2012.
  18. [19]Sanford J. Grossman and Oliver D. Hart. An analysis of the principal-agent problem. In G. Dionne and S. E. Harrington, editors, Foundations of Insurance Economics: Readings in Economics and Finance, Huebner International Series on Risk, Insurance and Economic Security, pages 302–340. Springer, 1992.
  19. [20]GrowthBook. Sequential testing. https://docs.growthbook.io/statistics/sequential, n.d. GrowthBook documentation, no publication date stated, accessed 2026-10-14.
  20. [21]Peter Grünwald, Rianne de Heide, and Wouter Koolen. Safe testing. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(5):1091–1128, 2024.
  21. [22]Peter Henderson, Ben Chugg, Brandon Anderson, and Daniel E Ho. Beyond ads: Sequential decision-making algorithms in law and public policy. In Proceedings of the 2022 Symposium on Computer Science and Law, pages 87–100, 2022.DOI
  22. [23]Peter Henderson, Ben Chugg, Brandon Anderson, Kristen Altenburger, Alex Turk, John Guyton, Jacob Goldin, and Daniel E Ho. Integrating reward maximization and population estimation: Sequential decision-making for internal revenue service audit selection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 5087–5095, 2023.
  23. [24]Steven R. Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. Time-uniform Chernoff bounds via nonnegative supermartingales. Probability Surveys, 17:257–317, 2020.DOI
  24. [25]Michael I. Jordan. A collectivist, economic vision for AI. Communications of the ACM, 69(9):67–75, 2026.DOI
  25. [27]Wouter M. Koolen and Vladimir Vovk. Buy low, sell high. Theoretical Computer Science, 558:144–158, 2014.
  26. [28]Jean-Jacques Laffont and David Martimort. The Theory of Incentives: The Principal-Agent Model. Princeton University Press, 2002.
  27. [29]Cintya Lanchimba. Contract theory. In Industrial Organization: The Economics of Markets, Firms, and Competition, pages 91–124. Palgrave Macmillan, 2026.
  28. [30]Michael Lindon, Chris Sanden, Vache Shirikian, Yanjun Liu, Minal Mishra, and Martin Tingley. Sequential A/B testing keeps the world streaming Netflix, part 1: Continuous data. https://netflixtechblog.com/sequential-a-b-testing-keeps-the-world-streaming-netflix-part-1-continuous-data-cba6c7ed49df, February 2024. Netflix Technology Blog, accessed 2026-10-14.
  29. [31]Ryan Lucht. What is sequential testing? (with examples). https://www.geteppo.com/blog/sequential-testing, July 2024. Eppo blog, accessed 2026-10-14.
  30. [32]Drew T. Nguyen, Alireza Fallah, and Michael I. Jordan. Incentive design in sequential statistical protocols. ICML 2026 Workshop on New Frontiers in Game-Theoretic Learning, 2026. URL https://openreview.net/forum?id=VKt5UUOYCK.
  31. [33]A. A. Novikov and A. N. Shiryaev. On an effective solution of the optimal stopping problem for random walks. Theory of Probability & Its Applications, 49(2):344–354, 2005.DOI
  32. [34]Michael Oberst, Davis Liang, and Zachary C. Lipton. Pioneering the science of AI evaluation. https://www.abridge.com/ai/science-ai-evaluation, September 2024. Abridge whitepaper, published 2024-09-19, last updated 2025-08-07, accessed 2026-10-14.
  33. [35]Goran Peskir and Albert Shiryaev. Optimal Stopping and Free-Boundary Problems. Birkhäuser, 2006.
  34. [36]Martin L. Puterman. Markov decision processes. In D. P. Heyman and M. J. Sobel, editors, Stochastic Models, volume 2 of Handbooks in Operations Research and Management Science, chapter 8, pages 331–434. Elsevier, 1990.
  35. [37]Aaditya Ramdas and Ruodu Wang. Hypothesis testing with e-values. Foundations and Trends® in Statistics, 1(1–2):1–390, 2025.DOI
  36. [38]Aaditya Ramdas, Peter Grünwald, Vladimir Vovk, and Glenn Shafer. Game-theoretic statistics and safe anytime-valid inference. Statistical Science, 38(4):576–601, 2023.arxiv.org/abs/2210.01948
  37. [39]William P. Rogerson. The first-order approach to principal-agent problems. Econometrica, 53(6):1357–1367, 1985.DOI
  38. [40]Johannes Ruf, Martin Larsson, Wouter M. Koolen, and Aaditya Ramdas. A composite generalization of Ville’s martingale theorem using e-processes. Electronic Journal of Probability, 28, 2023.arxiv.org/abs/2203.04485
  39. [41]Mårten Schultzberg and Sebastian Ankargre­n. Choosing a sequential testing framework — comparisons and discussions. https://engineering.atspotify.com/2023/03/choosing-sequential-testing-framework-comparisons-and-discussions, March 2023. Spotify Engineering Blog, accessed 2026-10-14.
  40. [42]Glenn Shafer, Alexander Shen, Nikolai Vereshchagin, and Vladimir Vovk. Test martingales, Bayes factors and p-values. Statistical Science, 26(1):84–101, 2011.
  41. [43]Flora C. Shi, Stephen Bates, and Martin J. Wainwright. Sharp results for hypothesis testing with risk-sensitive agents. arXiv preprint arXiv:2412.16452, 2024.arxiv.org/abs/2412.16452
  42. [44]Flora C. Shi, Martin J. Wainwright, and Stephen Bates. Instance-adaptive hypothesis tests with heterogeneous agents. arXiv preprint arXiv:2510.21178, 2025.arxiv.org/abs/2510.21178
  43. [45]Albert N. Shiryaev. Optimal Stopping Rules. Springer, 2008.
  44. [46]Frank Spitzer. A combinatorial lemma and its application to probability theory. Transactions of the American Mathematical Society, 82(2):323–339, 1956.
  45. [47]Roland Vaubel. Principal-agent problems in international organizations. The Review of International Organizations, 1(2):125–138, 2006.DOI
  46. [48]Boris Velasevic, Eden Saig, and Eric Mazumdar. Anytime contracts for AI services. arXiv preprint arXiv:2610.22781, 2026.arxiv.org/abs/2610.22781
  47. [49]Jean Ville. Étude critique de la notion de collectif. Gauthier-Villars, Paris, 1939.
  48. [50]Ian Waudby-Smith and Aaditya Ramdas. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(1):1–27, 2024.
  49. [51]Banghua Zhu, Stephen Bates, Zhuoran Yang, Yixin Wang, Jiantao Jiao, and Michael I. Jordan. The sample complexity of online contract design. In Proceedings of the 24th ACM Conference on Economics and Computation, page 1188. Association for Computing Machinery, 2023. Full version: arXiv:2211.05732.arxiv.org/abs/2211.05732

Paper details

vlnc:W_3729sfx5
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 2

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents