WorksComputer Science
Contracting with an Agent under Model Mismatch
Abstract
In the classical principal-agent setting, the principal delegates some task to the agent, who receives a payment conditioned on the outcome of the task, which in turn benefits the principal. To align their objectives, the principal designs a contract that incentivizes the agent to take actions beneficial to the principal. In practice, however, the agent may act under misspecified beliefs about the underlying environment, leading to systematically suboptimal decisions. In this work, we study this problem through a principal-agent Markov Decision Process (PAMDP), in which the principal repeatedly interacts with an agent whose transition model may differ from the true environment dynamics. The principal has access to the true transition model and designs contracts that account for the agent's belief and incentives. We characterize contract design under model mismatch and develop a framework that enables the principal to induce desirable long-term benefits even though the agent has a potentially inaccurate understanding of the environment.
Introduction
The incentive alignment between two parties has long been described through the principal-agent framework, in which a principal delegates a task to an agent by offering a contract that specifies an outcome-based payment. The design of optimal contracts has been explored by economists and computer scientists [5, 6, 13], where it plays a fundamental role in understanding strategic interactions and incentive mechanisms in both macro- and micro-economic settings.
In the era of increasing human-AI cooperation, the contract design problem has become more relevant than ever before. When delegating tasks to AI systems, we must account for the potential misalignment of the environment dynamics, where one party cannot always assume that the other possesses an accurate model of the environment and may act according to a misspecified understanding of the environment dynamics. This form of model disagreement has been studied in a two-agent sequential stochastic game, where agents may use different models of the world while sharing the same reward structure [8]. Existing formulations of the principal-agent Markov Decision Process (PAMDP) [3, 26] assume that both parties share the same model of the world. We, however, consider a case where the agent acts according to an unknown, potentially suboptimal misspecified model. Under this model misalignment, the principal’s goal is to design contracts that maximize its utility while remaining incentive compatible with respect to the agent’s transition model.
We study the problem of contract design under uncertainty about the agent’s environment model. Our approach is based on balancing exploration, in which the principal proposes contracts that reveal information about the agent’s model, and exploitation, in which the principal leverages the information to offer approximately optimal contracts. Exploration is crucial because the agent’s dynamics influence both the actions induced by a contract and the decision to accept it. Consequently, a principal who ignores these belief misalignment risks may design suboptimal contracts, resulting in suboptimal utility or even contract rejection.
We propose a Bayesian learning approach for contract design in PAMDPs with misspecified agent beliefs, where the principal knows the true transition dynamics and the agent uses a different model. We assume that the principal has access to a finite set of possible agent models. The principal maintains a posterior distribution over this set and updates this belief through their repeated interactions. Using the estimated agent’s transition model, the principal computes least payment contracts using backward induction and linear programming. We analyze how mismatched transition dynamics in a PAMDP setting influence contract design and the resulting incentive mechanism. Our main contributions are reformulating the optimal solutions for this setting and validating our approach through empirical experiments. We further investigate how the choice of the candidate model set influences the learning process, including the accuracy of the agent’s transition model estimation, the convergence of the posterior and the resulting contract design.
Related Work
Principal-agent problems, which formalize settings in which a principal delegates a task to an agent under moral hazard, have been extensively studied in economics and computer science as a fundamental framework [5, 13, 17, 18, 23]. [5] has provided an extensive overview of standard contract theory and incentive mechanism design. Recently, algorithmic contract design has been widely studied in the literature to investigate how contracts can be designed using computational methods [1, 7, 12, 13, 14, 20]. The complexity of finding optimal contracts has been studied in [2, 15, 19, 27]. Moreover, [6, 11] have considered multi-agent settings where the principal designs contracts for multiple agents by extending the single agent contract design approaches.
Optimal contract design is closely related to online learning when the principal does not have perfect information about the agent. In such settings, learning effective incentives over time through repeated interactions has become crucial in the literature [9, 10, 21, 28]. Furthermore, [4, 19, 25] have focused on the settings where the agent learns over time, investigating how to design suitable contracts by incentivizing the learning agent.
We focus on a PAMDP setting, where the principal learns how to design contracts to incentivize an agent making sequential decisions in an MDP. The principal-agent reinforcement learning framework has been introduced in [22]. More recently, different approaches to designing optimal contracts have been investigated, including reward shaping [3] or history-dependent policies instead of Markovian policies [4].
Our setting is related to the contractual reinforcement learning framework [26], but we consider a setting where the principal and the agent have different transition models, and the principal must infer this to design appropriate contracts. [8] has introduced multi-view decision processes with two agents, where they disagree on transition probabilities and share the same reward function. Moreover, [16] has introduced the Berk-Nash Equilibrium to model decision making under misspecified beliefs. Recently, [24] has analyzed the learning dynamics of a Bayesian agent with a finite set of misspecified beliefs and demonstrated that the action frequencies converge to such an equilibrium for a learning agent with two actions. However, these works do not consider an MDP setting. In contrast, we study model mismatch in a principal-agent MDP and investigate how learning the agent’s belief influences contract design.
Problem Setting
We consider a principal-agent MDP setting in which a principal interacts with an agent over a finite horizon. In each step, the principal offers a contract to the agent, and the agent selects an action based on its own transition dynamics. The resulting state transition is then realized according to the true transition dynamics of this environment. We assume that the principal has access to the true transition model while the agent may have a misspecified model. Figure 1 demonstrates how the principal and the agent interact in such an environment.

Figure 1. PAMDP setting under model mismatch
Formally, a finite horizon Markov Decision Process (MDP) is defined by a tuple where is a set of possible states, is an action space, and denotes the finite horizon length. In each step , the transition dynamics is determined by the true transition kernel , where specifies the probability distribution over the next state after taking action in state . is the environment reward function in step and the initial state is sampled from the initial state distribution .
A PAMDP is represented by a tuple . In addition to the underlying MDP , a PAMDP contains a contract space such that specifies the contract payment received by the agent based on the state and the next state. In each step , the principal observes the current state and offers a contract to the agent. The agent then selects an action according to its policy . While the agent makes decisions based on its own transition model , the next state is realized according to the true transition model The agent suffers a cost and the principal receives the reward defined in the underlying MDP. Thus, the principal designs contracts while accounting for the mismatch between the agent’s transition model and the true transition model .
Principal and Agent Value Functions
Given an action policy and a contract in a step , we define value functions for the principal and the agent as they optimize different objectives and have different beliefs about the transition dynamics. The principal evaluates the expected cumulative reward under the true transition model . In contrast, the agent evaluates its expected cumulative utility under its own transition model , considering the contract payments received and the costs of action incurred.
We define the principal’s state value function at step under any action policy and contract as:
where is the principal’s state-action value function which is written as
Similarly, the agent’s state value function is defined by:
where is the agent’s state-action value function which is expressed by
Contract Constraints
The optimal contract design problem is subject to several standard constraints containing Individual Rationality (IR), Incentive Compatibility (IC) and Limited Liability (LL). These constraints ensure that the designed contract is both acceptable to the agent and effective in inducing the desired behavior.
Definition 1 (Individual Rationality). A contract satisfies the IR constraint if
Otherwise, the agent rejects the contract. It means that the agent accepts the contract only if the expected utility obtained from the induced policy is non-negative.
Definition 2 (Incentive Compatibility). A contract is incentive compatible if it induces the agent to take the desired action by making the intended action optimal under the agent’s own model. Specifically, for a target action in state , a contract satisfies the IC constraint if
This constraint ensures that, given the contract and the agent’s transition model , the agent’s best response is the desired action .
Definition 3 (Limited liability). The principal cannot assign negative payments to the agent:
Remark 1. The agent is assumed to always participate in the game. This allows us to ignore the IR constraint by adding a terminate action to with for all and all step , which ensures nonnegative utility for the agent. Therefore, we only need to enforce the IC constraint.
Since the principal and the agent interact at each step, as illustrated in Figure 1, selecting the terminate action at any step can be interpreted as the agent rejecting the contract for that step. In this case, the agent receives no payment and incurs no additional cost, ensuring a nonnegative utility for the agent.
Optimal Solutions Using Backward Induction and Linear Program
In this section, we consider a solution approach similar to [26], where the contract parameters are computed iteratively using backward induction and linear program from the final step to the initial step 1. Unlike existing formulations, we reformulate the least payment equations under different beliefs about the transition dynamics. The goal of the optimization problem is to minimize the total contract payment of the principal subject to the agent’s incentive compatibility constraint. Thus, we derive a least payment solution under the assumption of full observability of the system.
In each step , the principal designs a contract that maximizes its expected utility under the true transition dynamics :
Then, given a contract , the agent selects an action according to its policy under its own transition model :
Since the principal cannot directly incentivize the agent’s action, the contract is designed to induce the desired action through the agent’s best response policy.
We first consider the terminal step , which serves as the base case of the backward induction. We then derive the recursive formulation for the remaining steps . Accordingly, we set and . Thus, the agent’s and principal’s state value functions at step are given by
with
and
with
In step , for each action , we calculate the optimal contract , which represents the least payment contract that induces the agent to select action . This contract can be computed by the following linear program:
subject to
This optimization is equivalent to maximizing the principal’s utility at step since the reward term is independent of the contract .
Using the resulting contract , the principal obtains the corresponding state-action value function . Therefore, with ties broken in favor of the principal, the optimal action policy for the principal to induce is determined by
We denote the optimal action under the policy by since we have a deterministic action policy. Then, the optimal contract under the action is subsequently obtained by
The optimal value functions of the principal and the agent in step , and , are then obtained under the best action policy and the optimal contract , respectively.
At each step , following the backward induction procedure, the principal computes the least payment contract required to induce action by solving the following:
subject to
where is under the optimal policy and the optimal contract . Thus, using the principal’s state-action value function under the contract , the best action policy for the principal to induce an action is:
and the optimal contract under the action can be determined by
Similarly, in step , the optimal value functions can be written using the best action policy and the optimal contract . As a result, we note that the principal can design a contract which is likely to incorporate the agent’s action costs and transition model, which enables the principal to incentivize the agent directly with the contract.
The optimal solutions are computed under the perfect information about the agent’s transition model to induce a target action policy. Therefore, we note that the resulting contracts are incentive compatible, as the principal’s induced policy is aligned with the best response policy of the agent.
Motivation
Figure 2 compares the value functions of both the principal and the agent under settings with shared and mismatched beliefs and the corresponding total payments offered by the principal. We apply the optimal solution derived in the previous section. The results demonstrate how misspecified beliefs about the transition dynamics influence their value functions and the total contract payments by the principal. We report the average results over 20 trials, where rewards and costs are randomly sampled from , and contract payments are bounded in . We use an MDP with , , and a uniform initial state distribution.
Figure 2. Comparison of optimal solutions under the same and different beliefs
We compare the total contract payments under different beliefs and the same beliefs. The promised payment is computed as the sum of all entries in the contract payment matrix. We also evaluate the expected total payment under both the agent’s transition model and the true transition dynamics by accounting for the action policy. This enables us to compare the payment the agent expects to receive based on its own model and the payment the agent actually receives under the true dynamics. For their expected utility, we focus on the state value function, where we apply the backward induction steps in the previous section.
The results in Figure 2 show that when the principal and the agent share the same belief about the transition model, the principal can induce the desired behavior with relatively lower contract payments. In contrast, under different beliefs, the principal offers higher contract payments to compensate for the discrepancy between the agent’s model and the true dynamics. The figure highlights the difference between the agent’s expectation and realized performance. According to its own model, the agent believes to receive a higher return; however, when evaluated under the true dynamics, the realized payments and the resulting state value of the agent are lower. Consequently, the agent overestimates its expected utility due to misspecified model, at the same time, the principal achieves higher utility under belief mismatch despite offering a higher contract payment to the agent. This is because the realized payments under the true transition dynamics are lower than those in the shared belief setting. These results demonstrate that different beliefs can significantly affect the contract design and the incentive mechanism in sequential decision-making.
Learning in a PAMDP Setting with Misspecified Model
The principal’s objective is to offer contracts that incentivize the agent to act in a way that maximizes the principal’s utility. We consider an episodic learning consisting of episodes, where the principal and the agent interact over steps in each episode. In each episode , at each step , the principal offers a contract and observes the agent’s response, where the agent is assumed to be best responding to the offered contract according to its own transition model. Since the state transitions are generated by the true transition dynamics (rather than ), the agent’s model is not directly observable from the realized transitions. The principal commits to a contract payment , which maximizes its expected utility, at the beginning of each episode, since the principal can anticipate the agent’s best response to the committed contract. The agent computes the optimal action policy after the principal offers the contract. This implies that updates are performed at the end of each episode.
To simplify the analysis, we assume that the principal has access to a finite set of possible agent models . The principal knows the true transition dynamics , observes in each step , which is stored in the trajectory for episode , where is the current state, is the action taken by the agent according to the policy , which is determined by the model of the agent, and is the next state realized according to the true dynamics . In addition, for any step , the action cost function is assumed to be known by the principal.
Algorithm 1 Bayesian Learning for Contract Design under Misspecified Model
Input:
Initialize .
for do
Sample
Compute using (22)
Initialize the trajectory .
Sample initial state
for do
Offer to the agent based on current state
The agent selects an action
The agent incurs the cost
The next state is realized
Observe and make the payment
Observe and receive reward
Update
end for
Update using Bayes’ rule based on
end for
Therefore, the principal maintains an estimate of the agent’s transition model and updates it through repeated interaction using a Bayesian learning approach. Based on the posterior, the principal obtains a prior estimate of the agent’s transition model and designs contracts accordingly. In particular, the principal uses the observed trajectories in each episode to update its posterior over . Using the updated belief, the principal can offer approximately optimal contracts according to the true dynamics , but is constrained by the estimated agent’s model . Therefore, for step in episode , the least payment contract is computed using estimates as follows:
subject to
where is an estimate of the agent’s value under the estimated induced policy and the committed contract. Then, we follow similar steps as in Section 3 to determine the best contract payment and the policy to induce for the principal.
Let denote the distribution over the set in episode . At the beginning of the interaction, a prior is initialized as the uniform distribution over . In each episode , the principal updates the posterior distribution using observed trajectories, samples a model from and designs the contract which is constrained by . The learning procedure for each episode consists of the steps in Algorithm 1.
Based on the trajectory in episode , the principal updates its posterior belief over the possible agent models using Bayes’ rule as follows:
Since transitions to the next states are realized according to the true dynamics , the observed transitions do not provide crucial information to distinguish possible agent models. Instead, the agent’s model only affects the action policy. Thus, in each episode , after computing the policy induced by each possible model , the log-likelihood term is determined by
where .
Experiments
In this section, we present the simulation results. First, we introduce the evaluation metrics we use throughout the experiments to demonstrate how learning the agent’s misspecified model affects the contract design and how even small differences between the estimated and agent’s transition models may result in significant changes in contract payments. Specifically, we consider regret to measure the principal’s loss throughout their interaction, the Kullback–Leibler (KL) divergence between estimated transition model and the agent’s model to evaluate estimation accuracy, and mean squared error (MSE) between learned and optimal contracts.
We evaluate the performance using expected regret over episodes, which measures the cumulative loss of the principal because of the uncertainty about the agent’s transition model compared to the optimal solutions. The expected regret can be expressed as
where represents the principal’s optimal value under the optimal contract and the optimal policy , which is obtained through the steps in § 3 when the agent’s transition model is known, denotes the contract offered by the principal and denotes the agent policy to induce for the principal by the estimated agent’s transition model in episode . We use the absolute value since the principal’s realized utility is not always lower than . As shown in Figure 2, misspecified beliefs may enable the principal to achieve higher utility than in the same belief setting. Moreover, under model mismatch, a contract computed using the estimated agent model may result in a higher principal value than a contract computed based on any other agent’s transition model.
In each episode, to measure how the principal accurately estimates the agent’s model, we compute the KL divergence between the agent’s true transition model and the estimated one . For every state action pair , we have the next state distribution and compute the average KL divergence over all pairs as
where and are the number of states and actions, respectively.
Furthermore, since the principal updates its posterior estimate over the set of possible agent models, we also evaluate the KL divergence between the agent’s and the estimated models using the posterior distributions over the set. Thus, the KL divergence with the posterior is defined as follows:
To evaluate how much the learned contracts differ from the optimal one, we compute the MSE between the learned contract and the optimal payments , which is defined as
To construct a finite set of candidate agent transition models, we generate transition probabilities by sampling from a Dirichlet distribution centered around a combination of the agent’s model and the true transition dynamics. For any , , and step , we sample
where is the concentration parameter of the Dirichlet distribution, controlling how closely the sampled transition probabilities are concentrated around the given distribution and determines the influence of the agent’s model and true dynamics. For example, if , then the sampled distribution is concentrated on the agent’s transition model.
We consider a PAMDP setting with , , and . The agent and true transition models are randomly generated and remain fixed throughout the experiments. The rewards and costs are also randomly generated from , and the contract payments are bounded within . We note that all results are averaged over 20 trials.
Figure 3 and Figure 4 compare the effect of the time horizon on learning the agent’s transition model when all sets contain and . We construct five different model sets by fixing and varying the coefficient , respectively, from Set 1 to 5. Each set contains 300 transition models, 150 randomly generated models, , and , while the remaining models are sampled from the Dirichlet distribution described above. We note that the average KL divergence between the true dynamics and the agent’s model, , is 0.41 for Figure 3 and 0.40 for Figure 4.

Figure 3. Comparison of different model sets containing the agent’s transition model for

Figure 4. Comparison of different model sets containing the agent’s transition model for
In Figure 3, we set , whereas in Figure 4, . With a higher number of interactions in each episode, the principal is able to distinguish the agent’s transition model more easily, and the KL divergence with the posterior consistently converges to across all sets, as shown in Figure 3. In contrast, in Figure 4, the principal observes only one transition per episode, which may lead to insufficient exploration of some states and actions in some trials. Since some of the state action pairs are rarely or never visited by the agent, some models may induce similar likelihoods over the observed trajectories. Consequently, when they interact for fewer steps in each episode, it may result in convergence to a suboptimal posterior by assigning a positive probability to some suboptimal models. We also observe that even small differences in transition probabilities may result in significant loss for the principal, which means that models with similar KL values may lead to different performance, while models with larger KL divergence can still achieve comparable results.
Furthermore, we present additional experiments for model sets which do not contain the agent’s true transition dynamics , as shown in Figure 5, to investigate whether the posterior distribution converges to the model that is closest to the agent’s model. We consider sets of models constructed in the same style as those in Figure 3, except that is excluded from the sets. We note that is 0.36 and the closest model to has KL divergence of 0.002 for all sets.

Figure 5. Comparison of different model sets that do not contain the agent’s transition model for
Since the agent’s model is not in the sets, exact convergence to the agent’s model, as observed in Figure 3 and Figure 4, is no longer possible. Thus, we update the posterior using a Boltzmann softmax policy with a temperature parameter of 1 to obtain a meaningful belief distribution over the model set while keeping the rest of the learning procedure unchanged. The results demonstrate that Set 1, which contains models closer to , achieves lower regret, but also results in larger fluctuations in the contract distance and the KL divergence, since this set contains more candidate models similar to , making the posterior distribution less concentrated. We observe a decreasing trend in the KL divergence and contract distance as the closest model consistently maintains a positive posterior probability. On the other hand, as shown in the KL divergence plot, multiple models maintain positive posterior probabilities throughout the learning process, preventing convergence in KL divergence and contract distance.
Consequently, when the sets contain the agent’s transition model, we observe convergence in the plots as long as the principal and agent interact sufficiently. In contrast, when the agent’s model is excluded from the sets, such convergence is no longer guaranteed. Our experiments show that while KL divergence and contract distance provide useful insight into the learning process, they do not fully capture the effectiveness of the learning approach. In particular, regret provides a more informative measure of performance. We also observe across all three experiments that Sets 1 and 2, which contain models closer to the agent’s transition model, consistently yield lower regret for the principal. We conclude that accurately learning the agent’s dynamics is essential to design effective contracts and provide incentives under belief mismatch.
Discussion
In this work, we study a PAMDP setting with model misalignment where the principal and the agent have different beliefs on transition dynamics. We investigate how this mismatch affects the contract design. In particular, we consider the setting where the principal has access to a set of possible agent transition models and designs contracts by learning the agent’s model through repeated interactions. Our experimental results show that accurately estimating the agent’s transition model is essential for effective contract design. As the principal’s estimate converges to the agent’s model, the learned contracts approach the optimal contracts computed under perfect information, and thus the principal incurs lower regret. Moreover, the experiments demonstrate that even small differences in learning the agent’s model may result in noticeable changes in contract payments and the principal’s loss. In future work, our aim is to extend our analysis to continuous model spaces and analyze how contracts can be used to estimate the misaligned belief of the agent. Moreover, an important extension is to establish theoretical guarantees for this learning framework. Finally, many PAMDP settings consider that the principal cannot directly observe the agent’s actions. Extending our approach to such settings would be an interesting direction, as it has additional challenges in learning the agent’s model and contract design.
References
- [1]Tal Alon, Paul Dütting, Yingkai Li, and Inbal Talgam-Cohen. Bayesian Analysis of Linear Contracts, July 2023. arXiv:2211.06850 [cs, econ].
- [2]Francesco Bacchiocchi, Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti. Learning optimal contracts: How to exploit small action spaces. In International Conference on Learning Representations, volume 2024, pages 11944–11970, 2024.
- [3]Omer Ben-Porat, Yishay Mansour, Michal Moshkovitz, and Boaz Taitler. Principal-Agent Reward Shaping in MDPs, December 2023. arXiv:2401.00298 [cs].
- [4]Matteo Bollini, Francesco Bacchiocchi, Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti. Contracting With a Reinforcement Learning Agent by Playing Trick or Treat, October 2024. arXiv:2410.13520 [cs].arxiv.org/abs/2410.13520
- [5]Patrick Bolton and Mathias Dewatripont. Contract theory. MIT Press, Cambridge, MA Londin, England, 2005.
- [6]Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti. Multi-Agent Contract Design: How to Commission Multiple Agents with Individual Outcomes. In Proceedings of the 24th ACM Conference on Economics and Computation, pages 412–448, London United Kingdom, July 2023. ACM.DOI
- [7]David Molina Concha, Kyeonghyeon Park, Hyun-Rok Lee, Taesik Lee, and Chi-Guhn Lee. Algorithmic Contract Design with Reinforcement Learning Agents, August 2024. arXiv:2408.09686 [cs].
- [8]Christos Dimitrakakis, David C. Parkes, Goran Radanovic, and Paul Tylkin. Multi-view decision processes: the helper-AI problem. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS ’17, pages 5449–5458, Red Hook, NY, USA, 2017. Curran Associates Inc.
- [9]Ilgin Dogan, Zuo-Jun Max Shen, and Anil Aswani. Estimating and Incentivizing Imperfect-Knowledge Agents with Hidden Rewards, August 2023. arXiv:2308.06717 [cs, stat].
- [10]Ilgin Dogan, Zuo-Jun Max Shen, and Anil Aswani. Repeated Principal-Agent Games with Unobserved Agent Rewards and Perfect-Knowledge Agents, May 2023. arXiv:2304.07407 [cs, stat].
- [11]Paul Dütting, Tomer Ezra, Michal Feldman, and Thomas Kesselheim. Multi-Agent Contracts, November 2022. arXiv:2211.05434 [cs].
- [12]Paul Dütting, Tomer Ezra, Michal Feldman, and Thomas Kesselheim. Multi-Agent Combinatorial Contracts, May 2024. arXiv:2405.08260 [cs].
- [13]Paul Dütting, Michal Feldman, and Inbal Talgam-Cohen. Algorithmic Contract Theory: A Survey, December 2024. arXiv:2412.16384 [cs].
- [14]Paul Dütting, Tim Roughgarden, and Inbal Talgam-Cohen. Simple versus Optimal Contracts. In Proceedings of the 2019 ACM Conference on Economics and Computation, pages 369–387, Phoenix AZ USA, June 2019. ACM.
- [15]Paul Dütting, Tim Roughgarden, and Inbal Talgam-Cohen. The Complexity of Contracts, February 2020. arXiv:2002.12034 [cs].
- [16]Ignacio Esponda and Demian Pouzo. Equilibrium in misspecified markov decision processes. Theoretical Economics, 16(2):717–757, 2021.arxiv.org/abs/1502.06901
- [17]Jiarui Gan, Minbiao Han, Jibang Wu, and Haifeng Xu. Generalized Principal-Agency: Contracts, Information, Games and Beyond, February 2024. arXiv:2209.01146 [cs].arxiv.org/abs/2209.01146
- [18]Sanford Grossman and Oliver Hart. An analysis of principal-agent problem. Econometrica, 51:7–45, 01 1983.
- [19]Guru Guruganesh, Yoav Kolumbus, Jon Schneider, Inbal Talgam-Cohen, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Joshua R. Wang, and S. Matthew Weinberg. Contracting with a Learning Agent, January 2024. arXiv:2401.16198 [cs, econ].
- [20]Guru Guruganesh, Jon Schneider, and Joshua R. Wang. Contracts under Moral Hazard and Adverse Selection. In Proceedings of the 22nd ACM Conference on Economics and Computation, pages 563–582, Budapest Hungary, July 2021. ACM.arxiv.org/abs/2010.06742
- [21]Chien-Ju Ho, Aleksandrs Slivkins, and Jennifer Wortman Vaughan. Adaptive contract design for crowdsourcing markets: bandit algorithms for repeated principal-agent problems. In Proceedings of the fifteenth ACM conference on Economics and computation, pages 359–376, Palo Alto California USA, June 2014. ACM.
- [22]Dima Ivanov, Paul Dütting, Inbal Talgam-Cohen, Tonghan Wang, and David C. Parkes. Principal-Agent Reinforcement Learning, July 2024. arXiv:2407.18074 [cs].arxiv.org/abs/2407.18074
- [23]Jean-Jacques Laffont and David Martimort. The Theory of Incentives: The Principal-Agent Model. Princeton University Press, December 2009.
- [24]Yingkai Li and Argyris Oikonomou. Dynamics and contracts for an agent with misspecified beliefs. arXiv preprint arXiv:2405.20423, 2024.arxiv.org/abs/2405.20423
- [25]Tao Lin and Yiling Chen. Generalized Principal-Agent Problem with a Learning Agent, October 2025. arXiv:2402.09721 [cs].arxiv.org/abs/2402.09721
- [26]Jibang Wu, Siyu Chen, Mengdi Wang, Huazheng Wang, and Haifeng Xu. Contractual Reinforcement Learning: Pulling Arms with Invisible Hands, July 2024. arXiv:2407.01458 [cs].arxiv.org/abs/2407.01458
- [27]Banghua Zhu, Stephen Bates, Zhuoran Yang, Yixin Wang, Jiantao Jiao, and Michael I Jordan. The sample complexity of online contract design. arXiv preprint arXiv:2211.05732, 2022.arxiv.org/abs/2211.05732
- [28]Banghua Zhu, Sai Praneeth Karimireddy, Jiantao Jiao, and Michael I. Jordan. Online Learning in a Creator Economy, May 2023. arXiv:2305.11381 [cs].arxiv.org/abs/2305.11381
